2024-04-26T23:23:24.922 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T23:23:24.922 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/scrub} --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/orig.config.yaml 2024-04-26T23:23:24.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T23:23:25.062 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/scrub} email: null first_in_suite: false job_id: '7675316' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCccQe7W9iD7DSxp8gyRYjwlClebnJTFO/4NDpkSDQGrk662jMn/JlxAduabfSm9zmxyew565VyvgfZ8yZlC0iE= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMf4qdrlQPGAtPkCwCEnTxpAWYC4z3GsD2yCxvLqRewf+NVWQI+Adl0jVO2NCa+ybHLQ+e2SaI8M5At2X7kfoqc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T23:23:25.062 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T23:23:25.063 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T23:23:25.063 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T23:23:25.065 INFO:teuthology.task.internal:Checking packages... 2024-04-26T23:23:25.088 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T23:23:25.088 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T23:23:25.089 INFO:teuthology.packaging:ref: None 2024-04-26T23:23:25.089 INFO:teuthology.packaging:tag: None 2024-04-26T23:23:25.089 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T23:23:25.089 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:23:25.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T23:23:25.229 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T23:23:25.230 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T23:23:25.237 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T23:23:25.237 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T23:23:25.256 INFO:teuthology.task.internal:Saving configuration 2024-04-26T23:23:25.273 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T23:23:25.279 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T23:23:25.305 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:18:16.685545', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCccQe7W9iD7DSxp8gyRYjwlClebnJTFO/4NDpkSDQGrk662jMn/JlxAduabfSm9zmxyew565VyvgfZ8yZlC0iE='} 2024-04-26T23:23:25.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:18:16.684095', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMf4qdrlQPGAtPkCwCEnTxpAWYC4z3GsD2yCxvLqRewf+NVWQI+Adl0jVO2NCa+ybHLQ+e2SaI8M5At2X7kfoqc='} 2024-04-26T23:23:25.330 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T23:23:25.336 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T23:23:25.336 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T23:23:25.336 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T23:23:25.404 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd323fde160>, signals=[15]) 2024-04-26T23:23:25.405 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T23:23:25.411 INFO:teuthology.task.internal:Opening connections... 2024-04-26T23:23:25.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:23:25.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:23:25.489 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:23:25.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:23:25.565 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T23:23:25.574 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-04-26T23:23:25.593 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-04-26T23:23:25.593 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-04-26T23:23:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-04-26T23:23:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-04-26T23:23:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-04-26T23:23:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:23:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:23:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:23:25.652 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-04-26T23:23:25.674 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-04-26T23:23:25.692 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-04-26T23:23:25.692 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-04-26T23:23:25.748 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:23:25.749 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:23:25.750 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:23:25.750 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-04-26T23:23:25.774 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T23:23:25.784 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T23:23:25.789 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T23:23:25.789 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:23:25.792 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:23:25.807 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T23:23:25.813 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T23:23:25.813 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:23:25.848 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:23:25.948 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T23:23:26.002 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T23:23:26.016 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T23:23:26.017 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-26T23:23:26.017 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-26T23:23:26.017 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:23:26.018 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:23:26.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:23:26.034 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T23:23:26.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:23:26.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T23:23:26.274 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:26.274 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T23:23:26.274 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-04-26T23:23:26.283 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:26.283 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T23:23:26.283 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-26T23:23:26.292 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-437.el9.x86_64 2024-04-26T23:23:26.292 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:26.292 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T23:23:26.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum upgrade -y linux-firmware 2024-04-26T23:23:26.301 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-437.el9.x86_64 2024-04-26T23:23:26.301 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:26.301 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T23:23:26.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum upgrade -y linux-firmware 2024-04-26T23:23:28.555 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T23:23:28.680 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T23:23:30.481 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-26T23:23:30.733 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T23:23:34.773 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T23:23:37.182 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-26T23:23:38.636 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T23:23:40.924 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 942 kB/s | 6.2 MB 00:06 2024-04-26T23:23:42.674 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T23:23:43.667 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T23:23:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T23:23:45.678 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-04-26T23:23:45.679 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-04-26T23:23:45.680 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-04-26T23:23:45.796 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-26T23:23:45.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:45.942 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T23:23:45.942 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T23:23:45.945 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi142.front.sepia.ceph.com to complete... 2024-04-26T23:23:49.025 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T23:23:51.075 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-04-26T23:23:51.076 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-04-26T23:23:51.076 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-04-26T23:23:51.192 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T23:23:51.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:23:51.340 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T23:23:51.340 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T23:23:51.343 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-04-26T23:24:03.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26 23:24:03 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T23:24:03.574 INFO:teuthology.orchestra.run.smithi142.stderr:FINISHED --2024-04-26 23:24:03-- 2024-04-26T23:24:03.574 INFO:teuthology.orchestra.run.smithi142.stderr:Total wall clock time: 18s 2024-04-26T23:24:03.574 INFO:teuthology.orchestra.run.smithi142.stderr:Downloaded: 1 files, 341M in 17s (19.7 MB/s) 2024-04-26T23:24:03.577 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-26T23:24:03.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T23:24:05.802 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying... ######################################## 2024-04-26T23:24:05.934 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing... ######################################## 2024-04-26T23:24:06.053 INFO:teuthology.orchestra.run.smithi142.stdout:Updating / installing... 2024-04-26T23:24:14.022 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T23:24:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26 23:24:19 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T23:24:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-04-26 23:24:19-- 2024-04-26T23:24:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 28s 2024-04-26T23:24:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 341M in 28s (12.3 MB/s) 2024-04-26T23:24:19.585 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T23:24:19.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying... ######################################## 2024-04-26T23:24:21.951 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing... ######################################## 2024-04-26T23:24:22.060 INFO:teuthology.orchestra.run.smithi046.stdout:Updating / installing... 2024-04-26T23:24:30.479 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T23:24:52.649 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T23:24:52.649 DEBUG:teuthology.orchestra.run.smithi142:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T23:24:52.738 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T23:24:52.740 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T23:24:52.742 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T23:24:52.743 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T23:24:52.744 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T23:24:52.745 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T23:24:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T23:24:52.747 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T23:24:52.748 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T23:24:52.749 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T23:24:52.750 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T23:24:52.751 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T23:24:52.754 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T23:24:52.755 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T23:24:52.756 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T23:24:52.757 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T23:24:52.758 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T23:24:52.759 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T23:24:52.760 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T23:24:52.761 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T23:24:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T23:24:52.763 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T23:24:52.764 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T23:24:52.765 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T23:24:52.766 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T23:24:52.767 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T23:24:52.768 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T23:24:52.769 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T23:24:52.770 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T23:24:52.771 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T23:24:52.772 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T23:24:52.773 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T23:24:52.774 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T23:24:52.775 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T23:24:52.776 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T23:24:52.777 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T23:24:52.778 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T23:24:52.779 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T23:24:52.780 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T23:24:52.781 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T23:24:52.782 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T23:24:52.783 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T23:24:52.784 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T23:24:52.785 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T23:24:52.786 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T23:24:52.787 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T23:24:52.788 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T23:24:52.789 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T23:24:52.790 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T23:24:52.791 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T23:24:52.792 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T23:24:52.793 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T23:24:52.794 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T23:24:52.795 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T23:24:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T23:24:52.797 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T23:24:52.798 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T23:24:52.799 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T23:24:52.800 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T23:24:52.801 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T23:24:52.802 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T23:24:52.803 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T23:24:52.804 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T23:24:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T23:24:52.806 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T23:24:52.807 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T23:24:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T23:24:52.809 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T23:24:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T23:24:52.811 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T23:24:52.812 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T23:24:52.813 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T23:24:52.814 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T23:24:52.815 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T23:24:52.816 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T23:24:52.817 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T23:24:52.818 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T23:24:52.819 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T23:24:52.820 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T23:24:52.821 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T23:24:52.822 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T23:24:52.823 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T23:24:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T23:24:52.825 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T23:24:52.826 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T23:24:52.827 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T23:24:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T23:24:52.829 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T23:24:52.830 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T23:24:52.831 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T23:24:52.832 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T23:24:52.833 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T23:24:52.834 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T23:24:52.835 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T23:24:52.836 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T23:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T23:24:52.838 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T23:24:52.839 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T23:24:52.840 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T23:24:52.841 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T23:24:52.842 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T23:24:52.843 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T23:24:52.844 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T23:24:52.845 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T23:24:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T23:24:52.847 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T23:24:52.848 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T23:24:52.849 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T23:24:52.850 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T23:24:52.851 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T23:24:52.852 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T23:24:52.853 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T23:24:52.854 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T23:24:52.855 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T23:24:52.856 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T23:24:52.857 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T23:24:52.858 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T23:24:52.859 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T23:24:52.860 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T23:24:52.861 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T23:24:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T23:24:52.863 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T23:24:52.864 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T23:24:52.865 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T23:24:52.866 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T23:24:52.867 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T23:24:52.868 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T23:24:52.869 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T23:24:52.870 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T23:24:52.871 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T23:24:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T23:24:52.873 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T23:24:52.874 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T23:24:52.875 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T23:24:52.876 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T23:24:52.877 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T23:24:52.878 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T23:24:52.879 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T23:24:52.880 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T23:24:52.881 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T23:24:52.882 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T23:24:52.883 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T23:24:52.884 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T23:24:52.886 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T23:24:52.887 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T23:24:52.888 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T23:24:52.890 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T23:24:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T23:24:52.892 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T23:24:52.893 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T23:24:52.894 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T23:24:52.894 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T23:24:52.894 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T23:24:52.894 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T23:24:52.894 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:24:52.898 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.898 DEBUG:teuthology.orchestra.run.smithi142:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout:for file in vmlinuz System.map config; do 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:24:52.921 INFO:teuthology.orchestra.run.smithi142.stdout: fi 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:done 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-04-26T23:24:52.922 INFO:teuthology.orchestra.run.smithi142.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T23:24:52.923 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:24:52.923 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.923 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-04-26T23:24:52.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-04-26T23:24:52.994 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-04-26T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:24:52.997 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T23:24:52.998 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T23:24:52.998 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:52.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T23:24:53.535 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-04-26T23:24:54.895 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T23:24:54.922 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-04-26T23:24:54.925 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-04-26T23:24:54.956 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.WCPfqmjdlj 2024-04-26T23:24:54.956 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WCPfqmjdlj 2024-04-26T23:24:55.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.WCPfqmjdlj 2024-04-26T23:24:55.152 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.WCPfqmjdlj is 6KB 2024-04-26T23:24:55.201 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.WCPfqmjdlj 2024-04-26T23:24:55.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T23:24:55.287 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T23:24:55.287 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T23:24:55.287 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T23:24:55.287 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T23:24:55.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T23:24:55.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-04-26T23:25:08.336 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T23:25:08.336 DEBUG:teuthology.orchestra.run.smithi046:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T23:25:08.432 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.432 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T23:25:08.433 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T23:25:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T23:25:08.435 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T23:25:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T23:25:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T23:25:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T23:25:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T23:25:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T23:25:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T23:25:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T23:25:08.443 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T23:25:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T23:25:08.445 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T23:25:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T23:25:08.447 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T23:25:08.448 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T23:25:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T23:25:08.450 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T23:25:08.451 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T23:25:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T23:25:08.453 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T23:25:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T23:25:08.455 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T23:25:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T23:25:08.457 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T23:25:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T23:25:08.459 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T23:25:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T23:25:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T23:25:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T23:25:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T23:25:08.464 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T23:25:08.465 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T23:25:08.466 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T23:25:08.467 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T23:25:08.468 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T23:25:08.469 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T23:25:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T23:25:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T23:25:08.472 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T23:25:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T23:25:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T23:25:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T23:25:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T23:25:08.477 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T23:25:08.478 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T23:25:08.479 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T23:25:08.480 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T23:25:08.481 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T23:25:08.482 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T23:25:08.483 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T23:25:08.484 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T23:25:08.485 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T23:25:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T23:25:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T23:25:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T23:25:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T23:25:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T23:25:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T23:25:08.492 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T23:25:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T23:25:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T23:25:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T23:25:08.496 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T23:25:08.497 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T23:25:08.498 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T23:25:08.499 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T23:25:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T23:25:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T23:25:08.502 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T23:25:08.503 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T23:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T23:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T23:25:08.506 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T23:25:08.507 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T23:25:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T23:25:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T23:25:08.510 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T23:25:08.511 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T23:25:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T23:25:08.513 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T23:25:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T23:25:08.515 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T23:25:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T23:25:08.517 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T23:25:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T23:25:08.519 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T23:25:08.520 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T23:25:08.521 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T23:25:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T23:25:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T23:25:08.524 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T23:25:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T23:25:08.526 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T23:25:08.527 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T23:25:08.528 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T23:25:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T23:25:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T23:25:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T23:25:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T23:25:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T23:25:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T23:25:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T23:25:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T23:25:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T23:25:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T23:25:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T23:25:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T23:25:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T23:25:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T23:25:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T23:25:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T23:25:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T23:25:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T23:25:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T23:25:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T23:25:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T23:25:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T23:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T23:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T23:25:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T23:25:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T23:25:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T23:25:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T23:25:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T23:25:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T23:25:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T23:25:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T23:25:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T23:25:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T23:25:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T23:25:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T23:25:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T23:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T23:25:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T23:25:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T23:25:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T23:25:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T23:25:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T23:25:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T23:25:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T23:25:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T23:25:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T23:25:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T23:25:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:25:08.581 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.581 DEBUG:teuthology.orchestra.run.smithi046:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T23:25:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T23:25:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:25:08.606 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:for file in vmlinuz System.map config; do 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: fi 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-26T23:25:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T23:25:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:25:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-26T23:25:08.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-04-26T23:25:08.691 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-04-26T23:25:08.692 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T23:25:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-04-26T23:25:08.694 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:25:08.694 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T23:25:08.694 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T23:25:08.695 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:08.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T23:25:09.296 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-04-26T23:25:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T23:25:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-04-26T23:25:10.753 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-04-26T23:25:10.768 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.UCwSEo9zkv 2024-04-26T23:25:10.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UCwSEo9zkv 2024-04-26T23:25:10.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.UCwSEo9zkv 2024-04-26T23:25:10.979 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.UCwSEo9zkv is 6KB 2024-04-26T23:25:11.027 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.UCwSEo9zkv 2024-04-26T23:25:11.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T23:25:11.109 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T23:25:11.109 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T23:25:11.109 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T23:25:11.109 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T23:25:11.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T23:25:11.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-04-26T23:25:25.471 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-26T23:25:25.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:25:25.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:25:41.287 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T23:25:41.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:25:41.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:25:43.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:25:52.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:25:52.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:25:55.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:25:59.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-26T23:26:07.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:26:07.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:26:08.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:26:08.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:26:10.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:26:11.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-26T23:26:23.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:26:23.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:26:25.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:26:25.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:26:29.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:26:42.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-26T23:26:47.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:26:47.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:26:50.114 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:26:57.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:26:57.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:27:00.289 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-26T23:27:11.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:27:11.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:27:14.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:27:18.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:27:18.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:27:18.684 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-26T23:27:19.429 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:27:19.429 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T23:27:19.430 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-26T23:27:19.511 INFO:teuthology.orchestra.run.smithi046.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T23:27:19.512 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T23:27:19.512 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T23:27:19.512 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T23:27:20.512 DEBUG:teuthology.parallel:result is None 2024-04-26T23:27:38.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:27:38.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:27:41.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-26T23:28:08.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:28:08.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:28:08.707 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-04-26T23:28:09.310 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:28:09.310 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T23:28:09.310 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-04-26T23:28:09.378 INFO:teuthology.orchestra.run.smithi142.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T23:28:09.378 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T23:28:09.378 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T23:28:09.378 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T23:28:10.381 DEBUG:teuthology.parallel:result is None 2024-04-26T23:28:10.382 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T23:28:10.388 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T23:28:10.389 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:28:10.397 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:28:10.455 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T23:28:10.463 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T23:28:10.477 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T23:28:10.477 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:28:10.482 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:28:10.553 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T23:28:10.564 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T23:28:10.564 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:28:10.571 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:28:10.704 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:28:10.725 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:28:10.753 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:28:10.778 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:28:10.783 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T23:28:10.791 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T23:28:10.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:28:10.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:28:10.883 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T23:28:10.896 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T23:28:10.896 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:28:10.903 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:28:10.961 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:28:11.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:28:11.149 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:28:11.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:28:11.326 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:28:11.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:28:11.415 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:28:11.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:28:11.600 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:28:11.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:28:11.768 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:28:11.769 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:28:11.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-04-26T23:28:11.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-04-26T23:28:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:28:12.003 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:28:12.296 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T23:28:12.305 INFO:teuthology.task.internal:Starting timer... 2024-04-26T23:28:12.306 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T23:28:12.326 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T23:28:12.338 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T23:28:12.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-04-26T23:28:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-04-26T23:28:12.444 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-04-26T23:28:12.546 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-04-26T23:28:12.550 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T23:28:12.550 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-04-26T23:28:12.609 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-04-26T23:28:12.609 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-04-26T23:28:12.667 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-04-26T23:28:12.667 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-04-26T23:28:12.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:28:12.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:28:12.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:28:12.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:28:12.838 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T23:28:12.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:28:12.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:28:12.987 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T23:28:12.997 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T23:28:13.013 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T23:28:13.015 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T23:33:29.614 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-04-26T23:33:29.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:33:29.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:33:29.728 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-26T23:33:29.888 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-26T23:33:29.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:33:29.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:33:29.989 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-04-26T23:33:30.138 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-26T23:33:30.138 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T23:33:30.149 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T23:33:30.149 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:33:30.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:33:30.157 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:33:30.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:33:30.243 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:33:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:33:30.287 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:33:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:33:30.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-04-26T23:33:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-04-26T23:33:30.390 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-04-26T23:33:30.411 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-04-26T23:33:30.427 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:33:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:33:30.468 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:33:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:33:30.610 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.618 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:33:30.631 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:33:30.632 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-26T23:33:30.632 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.632 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.632 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.632 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:33:30.634 INFO:teuthology.run_tasks:Running task install... 2024-04-26T23:33:30.642 DEBUG:teuthology.task.install:project ceph 2024-04-26T23:33:30.642 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T23:33:30.642 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T23:33:30.642 INFO:teuthology.task.install:Using flavor: default 2024-04-26T23:33:30.650 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T23:33:30.651 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T23:33:30.651 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:33:30.651 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:33:30.653 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:33:30.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:33:30.796 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:33:30.796 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:33:30.873 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:33:30.874 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:33:30.906 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:33:30.907 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:33:30.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:33:31.000 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:33:31.000 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:33:31.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:33:31.044 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:33:31.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:33:31.135 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:33:31.135 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:33:31.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:33:31.255 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:33:31.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:33:31.340 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-04-26T23:33:31.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-04-26T23:33:31.374 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:33:31.453 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-04-26T23:33:31.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-04-26T23:33:31.682 INFO:teuthology.orchestra.run.smithi046.stdout:48 files removed 2024-04-26T23:33:31.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:33:31.793 INFO:teuthology.orchestra.run.smithi142.stdout:48 files removed 2024-04-26T23:33:31.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:33:32.566 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 207 kB/s | 84 kB 00:00 2024-04-26T23:33:32.815 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 152 kB/s | 84 kB 00:00 2024-04-26T23:33:33.042 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T23:33:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-26T23:33:33.576 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-26T23:33:33.866 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T23:33:34.966 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-04-26T23:33:35.131 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T23:33:37.126 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-26T23:33:37.140 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T23:33:41.666 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-26T23:33:41.809 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-26T23:33:43.116 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 16 kB 00:00 2024-04-26T23:33:43.460 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-26T23:33:44.651 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T23:33:44.801 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-26T23:33:50.701 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T23:33:50.945 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T23:33:53.049 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.050 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.070 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:33:53.071 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.073 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:33:53.074 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:33:53.076 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:33:53.079 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:33:53.081 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:33:53.150 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-04-26T23:33:53.182 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-04-26T23:33:53.182 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-04-26T23:33:53.182 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-04-26T23:33:53.182 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-04-26T23:33:53.182 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:33:53.183 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:33:53.184 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-04-26T23:33:53.185 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:33:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:33:53.187 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:33:53.188 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:33:53.189 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:33:53.190 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:33:53.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:33:53.192 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 4 Packages 2024-04-26T23:33:53.193 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:33:53.194 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 222 M 2024-04-26T23:33:53.194 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-04-26T23:33:53.497 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.499 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.521 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:33:53.522 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:33:53.523 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:33:53.525 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:33:53.527 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:33:53.531 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:33:53.532 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:33:53.609 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:33:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:33:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:33:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:33:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:33:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:33:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:33:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:33:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:33:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:33:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:33:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-04-26T23:33:53.640 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-26T23:33:53.641 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-04-26T23:33:53.641 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 4 Packages 2024-04-26T23:33:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:33:53.641 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 222 M 2024-04-26T23:33:53.641 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-04-26T23:33:54.344 INFO:teuthology.orchestra.run.smithi142.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-26T23:33:54.536 INFO:teuthology.orchestra.run.smithi142.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.6 MB/s | 889 kB 00:00 2024-04-26T23:33:54.594 INFO:teuthology.orchestra.run.smithi142.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.5 MB/s | 145 kB 00:00 2024-04-26T23:33:54.903 INFO:teuthology.orchestra.run.smithi142.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 6.6 MB/s | 5.4 MB 00:00 2024-04-26T23:33:54.947 INFO:teuthology.orchestra.run.smithi046.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-26T23:33:54.970 INFO:teuthology.orchestra.run.smithi142.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 6.3 MB/s | 2.4 MB 00:00 2024-04-26T23:33:55.128 INFO:teuthology.orchestra.run.smithi142.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 6.8 MB/s | 1.5 MB 00:00 2024-04-26T23:33:55.745 INFO:teuthology.orchestra.run.smithi142.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.1 MB/s | 4.7 MB 00:00 2024-04-26T23:33:55.864 INFO:teuthology.orchestra.run.smithi046.stdout:(2/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-04-26T23:33:55.964 INFO:teuthology.orchestra.run.smithi046.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T23:33:56.006 INFO:teuthology.orchestra.run.smithi046.stdout:(4/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 841 kB/s | 889 kB 00:01 2024-04-26T23:33:56.506 INFO:teuthology.orchestra.run.smithi046.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 4.4 MB/s | 2.4 MB 00:00 2024-04-26T23:33:57.237 INFO:teuthology.orchestra.run.smithi142.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 8.0 MB/s | 17 MB 00:02 2024-04-26T23:33:57.287 INFO:teuthology.orchestra.run.smithi142.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 516 kB/s | 25 kB 00:00 2024-04-26T23:33:57.331 INFO:teuthology.orchestra.run.smithi046.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.7 MB/s | 4.7 MB 00:00 2024-04-26T23:33:57.446 INFO:teuthology.orchestra.run.smithi142.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 6.2 MB/s | 11 MB 00:01 2024-04-26T23:33:57.546 INFO:teuthology.orchestra.run.smithi142.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T23:33:57.548 INFO:teuthology.orchestra.run.smithi046.stdout:(7/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-26T23:33:57.630 INFO:teuthology.orchestra.run.smithi142.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 328 kB/s | 27 kB 00:00 2024-04-26T23:33:57.713 INFO:teuthology.orchestra.run.smithi142.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 384 kB/s | 32 kB 00:00 2024-04-26T23:33:57.897 INFO:teuthology.orchestra.run.smithi142.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.0 MB/s | 747 kB 00:00 2024-04-26T23:33:57.989 INFO:teuthology.orchestra.run.smithi142.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 163 kB 00:00 2024-04-26T23:33:58.089 INFO:teuthology.orchestra.run.smithi142.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.3 MB/s | 127 kB 00:00 2024-04-26T23:33:58.189 INFO:teuthology.orchestra.run.smithi142.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 4.9 MB/s | 501 kB 00:00 2024-04-26T23:33:58.363 INFO:teuthology.orchestra.run.smithi046.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 5.6 MB/s | 21 MB 00:03 2024-04-26T23:33:58.466 INFO:teuthology.orchestra.run.smithi046.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 248 kB/s | 25 kB 00:00 2024-04-26T23:33:58.981 INFO:teuthology.orchestra.run.smithi142.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 6.8 MB/s | 5.4 MB 00:00 2024-04-26T23:33:59.065 INFO:teuthology.orchestra.run.smithi142.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 550 kB/s | 45 kB 00:00 2024-04-26T23:33:59.140 INFO:teuthology.orchestra.run.smithi142.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T23:33:59.223 INFO:teuthology.orchestra.run.smithi142.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.9 MB/s | 161 kB 00:00 2024-04-26T23:33:59.316 INFO:teuthology.orchestra.run.smithi142.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.5 MB/s | 321 kB 00:00 2024-04-26T23:33:59.399 INFO:teuthology.orchestra.run.smithi142.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T23:33:59.474 INFO:teuthology.orchestra.run.smithi142.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T23:33:59.550 INFO:teuthology.orchestra.run.smithi142.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T23:34:00.042 INFO:teuthology.orchestra.run.smithi142.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.3 MB/s | 3.1 MB 00:00 2024-04-26T23:34:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T23:34:00.167 INFO:teuthology.orchestra.run.smithi142.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 592 kB/s | 34 kB 00:00 2024-04-26T23:34:00.234 INFO:teuthology.orchestra.run.smithi142.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T23:34:00.274 INFO:teuthology.orchestra.run.smithi046.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 5.7 MB/s | 17 MB 00:02 2024-04-26T23:34:00.441 INFO:teuthology.orchestra.run.smithi046.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.3 MB/s | 225 kB 00:00 2024-04-26T23:34:00.533 INFO:teuthology.orchestra.run.smithi046.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 297 kB/s | 27 kB 00:00 2024-04-26T23:34:00.633 INFO:teuthology.orchestra.run.smithi046.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 320 kB/s | 32 kB 00:00 2024-04-26T23:34:00.718 INFO:teuthology.orchestra.run.smithi142.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.3 MB/s | 3.5 MB 00:00 2024-04-26T23:34:00.800 INFO:teuthology.orchestra.run.smithi046.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.4 MB/s | 747 kB 00:00 2024-04-26T23:34:00.900 INFO:teuthology.orchestra.run.smithi046.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 163 kB 00:00 2024-04-26T23:34:00.992 INFO:teuthology.orchestra.run.smithi046.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.4 MB/s | 127 kB 00:00 2024-04-26T23:34:01.167 INFO:teuthology.orchestra.run.smithi046.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 2.8 MB/s | 501 kB 00:00 2024-04-26T23:34:01.745 INFO:teuthology.orchestra.run.smithi142.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.2 MB/s | 7.4 MB 00:01 2024-04-26T23:34:01.843 INFO:teuthology.orchestra.run.smithi142.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.5 MB/s | 249 kB 00:00 2024-04-26T23:34:01.927 INFO:teuthology.orchestra.run.smithi142.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 598 kB/s | 49 kB 00:00 2024-04-26T23:34:02.002 INFO:teuthology.orchestra.run.smithi142.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 222 kB/s | 16 kB 00:00 2024-04-26T23:34:02.094 INFO:teuthology.orchestra.run.smithi142.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T23:34:02.218 INFO:teuthology.orchestra.run.smithi046.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 5.1 MB/s | 5.4 MB 00:01 2024-04-26T23:34:02.244 INFO:teuthology.orchestra.run.smithi142.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T23:34:02.284 INFO:teuthology.orchestra.run.smithi046.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 689 kB/s | 45 kB 00:00 2024-04-26T23:34:02.303 INFO:teuthology.orchestra.run.smithi142.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 318 kB/s | 18 kB 00:00 2024-04-26T23:34:02.359 INFO:teuthology.orchestra.run.smithi046.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T23:34:02.426 INFO:teuthology.orchestra.run.smithi046.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T23:34:02.493 INFO:teuthology.orchestra.run.smithi046.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 4.7 MB/s | 321 kB 00:00 2024-04-26T23:34:02.552 INFO:teuthology.orchestra.run.smithi046.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 5.0 MB/s | 298 kB 00:00 2024-04-26T23:34:02.610 INFO:teuthology.orchestra.run.smithi046.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 99 kB 00:00 2024-04-26T23:34:02.669 INFO:teuthology.orchestra.run.smithi046.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T23:34:02.853 INFO:teuthology.orchestra.run.smithi142.stdout:(38/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.8 MB/s | 49 MB 00:05 2024-04-26T23:34:02.886 INFO:teuthology.orchestra.run.smithi142.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 138 kB/s | 80 kB 00:00 2024-04-26T23:34:02.995 INFO:teuthology.orchestra.run.smithi046.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 9.5 MB/s | 3.1 MB 00:00 2024-04-26T23:34:03.053 INFO:teuthology.orchestra.run.smithi046.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T23:34:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 699 kB/s | 34 kB 00:00 2024-04-26T23:34:03.145 INFO:teuthology.orchestra.run.smithi142.stdout:(40/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 21 MB 00:09 2024-04-26T23:34:03.162 INFO:teuthology.orchestra.run.smithi046.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.4 MB/s | 142 kB 00:00 2024-04-26T23:34:03.212 INFO:teuthology.orchestra.run.smithi142.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 602 kB/s | 195 kB 00:00 2024-04-26T23:34:03.246 INFO:teuthology.orchestra.run.smithi142.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-26T23:34:03.279 INFO:teuthology.orchestra.run.smithi142.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-26T23:34:03.346 INFO:teuthology.orchestra.run.smithi142.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T23:34:03.381 INFO:teuthology.orchestra.run.smithi142.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-26T23:34:03.430 INFO:teuthology.orchestra.run.smithi142.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.7 MB/s | 1.2 MB 00:00 2024-04-26T23:34:03.472 INFO:teuthology.orchestra.run.smithi142.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-26T23:34:03.522 INFO:teuthology.orchestra.run.smithi142.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-04-26T23:34:03.529 INFO:teuthology.orchestra.run.smithi046.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 9.7 MB/s | 3.5 MB 00:00 2024-04-26T23:34:03.572 INFO:teuthology.orchestra.run.smithi142.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-04-26T23:34:03.631 INFO:teuthology.orchestra.run.smithi142.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-04-26T23:34:03.656 INFO:teuthology.orchestra.run.smithi142.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-04-26T23:34:03.673 INFO:teuthology.orchestra.run.smithi142.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 714 kB/s | 107 kB 00:00 2024-04-26T23:34:03.732 INFO:teuthology.orchestra.run.smithi142.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 440 kB/s | 33 kB 00:00 2024-04-26T23:34:03.765 INFO:teuthology.orchestra.run.smithi142.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-26T23:34:03.807 INFO:teuthology.orchestra.run.smithi142.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 245 kB/s | 18 kB 00:00 2024-04-26T23:34:03.832 INFO:teuthology.orchestra.run.smithi142.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-04-26T23:34:03.871 INFO:teuthology.orchestra.run.smithi046.stdout:(31/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 1.7 MB/s | 11 MB 00:06 2024-04-26T23:34:03.899 INFO:teuthology.orchestra.run.smithi142.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 688 kB/s | 45 kB 00:00 2024-04-26T23:34:03.925 INFO:teuthology.orchestra.run.smithi142.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T23:34:04.005 INFO:teuthology.orchestra.run.smithi046.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.8 MB/s | 249 kB 00:00 2024-04-26T23:34:04.008 INFO:teuthology.orchestra.run.smithi142.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.0 MB/s | 3.0 MB 00:00 2024-04-26T23:34:04.058 INFO:teuthology.orchestra.run.smithi142.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-04-26T23:34:04.088 INFO:teuthology.orchestra.run.smithi046.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 599 kB/s | 49 kB 00:00 2024-04-26T23:34:04.100 INFO:teuthology.orchestra.run.smithi142.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-26T23:34:04.142 INFO:teuthology.orchestra.run.smithi142.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-26T23:34:04.163 INFO:teuthology.orchestra.run.smithi046.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 222 kB/s | 16 kB 00:00 2024-04-26T23:34:04.167 INFO:teuthology.orchestra.run.smithi142.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-04-26T23:34:04.218 INFO:teuthology.orchestra.run.smithi142.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 722 kB/s | 35 kB 00:00 2024-04-26T23:34:04.251 INFO:teuthology.orchestra.run.smithi142.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-26T23:34:04.276 INFO:teuthology.orchestra.run.smithi142.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-26T23:34:04.380 INFO:teuthology.orchestra.run.smithi046.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-04-26T23:34:04.414 INFO:teuthology.orchestra.run.smithi046.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 273 kB 00:00 2024-04-26T23:34:04.418 INFO:teuthology.orchestra.run.smithi142.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T23:34:04.489 INFO:teuthology.orchestra.run.smithi046.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-26T23:34:04.523 INFO:teuthology.orchestra.run.smithi046.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 169 kB/s | 18 kB 00:00 2024-04-26T23:34:04.544 INFO:teuthology.orchestra.run.smithi142.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-26T23:34:04.652 INFO:teuthology.orchestra.run.smithi142.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-26T23:34:04.711 INFO:teuthology.orchestra.run.smithi142.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-04-26T23:34:04.736 INFO:teuthology.orchestra.run.smithi142.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 696 kB/s | 133 kB 00:00 2024-04-26T23:34:04.803 INFO:teuthology.orchestra.run.smithi142.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T23:34:04.870 INFO:teuthology.orchestra.run.smithi142.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-26T23:34:05.012 INFO:teuthology.orchestra.run.smithi142.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-26T23:34:05.045 INFO:teuthology.orchestra.run.smithi142.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-26T23:34:05.096 INFO:teuthology.orchestra.run.smithi142.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 522 kB/s | 26 kB 00:00 2024-04-26T23:34:05.129 INFO:teuthology.orchestra.run.smithi142.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-26T23:34:05.180 INFO:teuthology.orchestra.run.smithi142.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-04-26T23:34:05.182 INFO:teuthology.orchestra.run.smithi046.stdout:(39/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 7.3 MB/s | 49 MB 00:06 2024-04-26T23:34:05.215 INFO:teuthology.orchestra.run.smithi046.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-04-26T23:34:05.297 INFO:teuthology.orchestra.run.smithi142.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-26T23:34:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 262 kB/s | 35 kB 00:00 2024-04-26T23:34:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 795 kB/s | 172 kB 00:00 2024-04-26T23:34:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 957 kB/s | 195 kB 00:00 2024-04-26T23:34:05.422 INFO:teuthology.orchestra.run.smithi142.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-26T23:34:05.456 INFO:teuthology.orchestra.run.smithi142.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-26T23:34:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 845 kB/s | 803 kB 00:00 2024-04-26T23:34:05.531 INFO:teuthology.orchestra.run.smithi142.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 715 kB/s | 77 kB 00:00 2024-04-26T23:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 781 kB/s | 253 kB 00:00 2024-04-26T23:34:05.556 INFO:teuthology.orchestra.run.smithi142.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 425 kB/s | 42 kB 00:00 2024-04-26T23:34:05.574 INFO:teuthology.orchestra.run.smithi046.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T23:34:05.641 INFO:teuthology.orchestra.run.smithi046.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-26T23:34:05.675 INFO:teuthology.orchestra.run.smithi046.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-26T23:34:05.707 INFO:teuthology.orchestra.run.smithi142.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-26T23:34:05.733 INFO:teuthology.orchestra.run.smithi046.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T23:34:05.740 INFO:teuthology.orchestra.run.smithi142.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 869 kB/s | 159 kB 00:00 2024-04-26T23:34:05.774 INFO:teuthology.orchestra.run.smithi142.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-04-26T23:34:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-04-26T23:34:05.824 INFO:teuthology.orchestra.run.smithi142.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 617 kB/s | 51 kB 00:00 2024-04-26T23:34:05.849 INFO:teuthology.orchestra.run.smithi142.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 723 kB/s | 54 kB 00:00 2024-04-26T23:34:05.874 INFO:teuthology.orchestra.run.smithi142.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T23:34:05.933 INFO:teuthology.orchestra.run.smithi142.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 504 kB/s | 42 kB 00:00 2024-04-26T23:34:05.950 INFO:teuthology.orchestra.run.smithi142.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-26T23:34:05.992 INFO:teuthology.orchestra.run.smithi142.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 885 kB/s | 36 kB 00:00 2024-04-26T23:34:06.025 INFO:teuthology.orchestra.run.smithi142.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-26T23:34:06.084 INFO:teuthology.orchestra.run.smithi142.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 520 kB/s | 47 kB 00:00 2024-04-26T23:34:06.143 INFO:teuthology.orchestra.run.smithi142.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-26T23:34:06.168 INFO:teuthology.orchestra.run.smithi142.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 701 kB/s | 58 kB 00:00 2024-04-26T23:34:06.227 INFO:teuthology.orchestra.run.smithi142.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T23:34:06.252 INFO:teuthology.orchestra.run.smithi142.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 771 kB/s | 64 kB 00:00 2024-04-26T23:34:06.302 INFO:teuthology.orchestra.run.smithi142.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-26T23:34:06.626 INFO:teuthology.orchestra.run.smithi046.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 127 kB/s | 107 kB 00:00 2024-04-26T23:34:06.636 INFO:teuthology.orchestra.run.smithi142.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-26T23:34:06.662 INFO:teuthology.orchestra.run.smithi142.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2024-04-26T23:34:06.870 INFO:teuthology.orchestra.run.smithi142.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-04-26T23:34:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 484 kB/s | 689 kB 00:01 2024-04-26T23:34:07.129 INFO:teuthology.orchestra.run.smithi142.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 662 kB/s | 309 kB 00:00 2024-04-26T23:34:07.214 INFO:teuthology.orchestra.run.smithi142.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 57 kB/s | 19 kB 00:00 2024-04-26T23:34:07.218 INFO:teuthology.orchestra.run.smithi046.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-04-26T23:34:07.254 INFO:teuthology.orchestra.run.smithi142.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 888 kB/s | 548 kB 00:00 2024-04-26T23:34:07.321 INFO:teuthology.orchestra.run.smithi142.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 244 kB/s | 26 kB 00:00 2024-04-26T23:34:07.346 INFO:teuthology.orchestra.run.smithi142.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 533 kB/s | 49 kB 00:00 2024-04-26T23:34:07.489 INFO:teuthology.orchestra.run.smithi142.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-26T23:34:07.514 INFO:teuthology.orchestra.run.smithi142.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 909 kB/s | 151 kB 00:00 2024-04-26T23:34:07.535 INFO:teuthology.orchestra.run.smithi046.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 524 kB/s | 943 kB 00:01 2024-04-26T23:34:07.539 INFO:teuthology.orchestra.run.smithi142.stdout:(112/152): libunwind-1.6.2-1.el9.x86_64.rpm 311 kB/s | 67 kB 00:00 2024-04-26T23:34:07.589 INFO:teuthology.orchestra.run.smithi142.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-04-26T23:34:07.639 INFO:teuthology.orchestra.run.smithi142.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-26T23:34:07.664 INFO:teuthology.orchestra.run.smithi142.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-04-26T23:34:07.682 INFO:teuthology.orchestra.run.smithi142.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 353 kB/s | 32 kB 00:00 2024-04-26T23:34:07.707 INFO:teuthology.orchestra.run.smithi142.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-04-26T23:34:07.765 INFO:teuthology.orchestra.run.smithi142.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T23:34:07.799 INFO:teuthology.orchestra.run.smithi142.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T23:34:07.824 INFO:teuthology.orchestra.run.smithi142.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-04-26T23:34:07.849 INFO:teuthology.orchestra.run.smithi142.stdout:(121/152): python3-cheroot-8.6.0-4.el9.noarch.r 934 kB/s | 172 kB 00:00 2024-04-26T23:34:07.883 INFO:teuthology.orchestra.run.smithi142.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T23:34:07.885 INFO:teuthology.orchestra.run.smithi046.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 49 kB/s | 33 kB 00:00 2024-04-26T23:34:07.950 INFO:teuthology.orchestra.run.smithi142.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-26T23:34:07.975 INFO:teuthology.orchestra.run.smithi142.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 118 kB/s | 11 kB 00:00 2024-04-26T23:34:08.034 INFO:teuthology.orchestra.run.smithi142.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-26T23:34:08.059 INFO:teuthology.orchestra.run.smithi142.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-26T23:34:08.084 INFO:teuthology.orchestra.run.smithi142.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-26T23:34:08.109 INFO:teuthology.orchestra.run.smithi142.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-04-26T23:34:08.160 INFO:teuthology.orchestra.run.smithi142.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 266 kB/s | 20 kB 00:00 2024-04-26T23:34:08.210 INFO:teuthology.orchestra.run.smithi142.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-04-26T23:34:08.218 INFO:teuthology.orchestra.run.smithi046.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 281 kB/s | 313 kB 00:01 2024-04-26T23:34:08.235 INFO:teuthology.orchestra.run.smithi142.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T23:34:08.277 INFO:teuthology.orchestra.run.smithi142.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 876 kB/s | 58 kB 00:00 2024-04-26T23:34:08.319 INFO:teuthology.orchestra.run.smithi142.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-26T23:34:08.344 INFO:teuthology.orchestra.run.smithi142.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 248 kB/s | 16 kB 00:00 2024-04-26T23:34:08.378 INFO:teuthology.orchestra.run.smithi142.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-26T23:34:08.403 INFO:teuthology.orchestra.run.smithi142.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T23:34:08.428 INFO:teuthology.orchestra.run.smithi142.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-04-26T23:34:08.444 INFO:teuthology.orchestra.run.smithi046.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-04-26T23:34:08.462 INFO:teuthology.orchestra.run.smithi142.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 369 kB/s | 31 kB 00:00 2024-04-26T23:34:08.487 INFO:teuthology.orchestra.run.smithi142.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T23:34:08.513 INFO:teuthology.orchestra.run.smithi142.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-04-26T23:34:08.538 INFO:teuthology.orchestra.run.smithi142.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-26T23:34:08.563 INFO:teuthology.orchestra.run.smithi142.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 684 kB/s | 51 kB 00:00 2024-04-26T23:34:08.597 INFO:teuthology.orchestra.run.smithi142.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T23:34:08.622 INFO:teuthology.orchestra.run.smithi142.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-26T23:34:08.672 INFO:teuthology.orchestra.run.smithi142.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-26T23:34:08.697 INFO:teuthology.orchestra.run.smithi142.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-26T23:34:08.731 INFO:teuthology.orchestra.run.smithi142.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T23:34:08.844 INFO:teuthology.orchestra.run.smithi046.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 18 kB/s | 11 kB 00:00 2024-04-26T23:34:08.864 INFO:teuthology.orchestra.run.smithi142.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T23:34:08.990 INFO:teuthology.orchestra.run.smithi142.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-04-26T23:34:09.140 INFO:teuthology.orchestra.run.smithi142.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-26T23:34:09.215 INFO:teuthology.orchestra.run.smithi142.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-04-26T23:34:09.291 INFO:teuthology.orchestra.run.smithi142.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-04-26T23:34:09.307 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:34:09.308 INFO:teuthology.orchestra.run.smithi142.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-26T23:34:09.311 INFO:teuthology.orchestra.run.smithi046.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 203 kB/s | 160 kB 00:00 2024-04-26T23:34:09.478 INFO:teuthology.orchestra.run.smithi046.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-04-26T23:34:10.816 INFO:teuthology.orchestra.run.smithi046.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 441 kB/s | 662 kB 00:01 2024-04-26T23:34:10.870 INFO:teuthology.orchestra.run.smithi046.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 419 kB/s | 582 kB 00:01 2024-04-26T23:34:11.412 INFO:teuthology.orchestra.run.smithi046.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-04-26T23:34:11.521 INFO:teuthology.orchestra.run.smithi046.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-04-26T23:34:11.604 INFO:teuthology.orchestra.run.smithi046.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 753 kB/s | 3.0 MB 00:04 2024-04-26T23:34:12.088 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-04-26T23:34:12.334 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-04-26T23:34:12.334 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-04-26T23:34:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 40 kB/s | 35 kB 00:00 2024-04-26T23:34:12.513 INFO:teuthology.orchestra.run.smithi046.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 190 kB/s | 188 kB 00:00 2024-04-26T23:34:12.547 INFO:teuthology.orchestra.run.smithi046.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 258 kB/s | 292 kB 00:01 2024-04-26T23:34:14.889 INFO:teuthology.orchestra.run.smithi046.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 695 kB/s | 1.6 MB 00:02 2024-04-26T23:34:15.080 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-04-26T23:34:15.080 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-04-26T23:34:16.464 INFO:teuthology.orchestra.run.smithi046.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 579 kB/s | 911 kB 00:01 2024-04-26T23:34:17.197 INFO:teuthology.orchestra.run.smithi046.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 182 kB/s | 133 kB 00:00 2024-04-26T23:34:18.089 INFO:teuthology.orchestra.run.smithi046.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 813 kB/s | 4.4 MB 00:05 2024-04-26T23:34:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 784 kB/s | 4.7 MB 00:06 2024-04-26T23:34:19.348 INFO:teuthology.orchestra.run.smithi046.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 130 kB/s | 87 kB 00:00 2024-04-26T23:34:19.890 INFO:teuthology.orchestra.run.smithi046.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 574 kB/s | 1.0 MB 00:01 2024-04-26T23:34:20.765 INFO:teuthology.orchestra.run.smithi046.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 280 kB/s | 244 kB 00:00 2024-04-26T23:34:21.482 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-04-26T23:34:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:34:21.607 INFO:teuthology.orchestra.run.smithi046.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 295 kB/s | 248 kB 00:00 2024-04-26T23:34:21.653 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:34:21.769 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:34:21.832 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:34:21.864 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:34:22.005 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:34:22.199 INFO:teuthology.orchestra.run.smithi046.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-26T23:34:22.443 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:34:22.537 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:34:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:34:22.755 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:34:22.774 INFO:teuthology.orchestra.run.smithi046.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 33 kB/s | 19 kB 00:00 2024-04-26T23:34:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:34:22.958 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:34:23.271 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:34:23.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:34:23.359 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:34:23.505 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:34:23.535 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:34:23.591 INFO:teuthology.orchestra.run.smithi046.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 211 kB/s | 172 kB 00:00 2024-04-26T23:34:23.613 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:34:23.644 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:34:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:34:24.046 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:34:24.102 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:34:24.199 INFO:teuthology.orchestra.run.smithi046.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-04-26T23:34:24.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:34:24.416 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:34:24.453 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:34:24.500 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:34:24.580 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:34:24.631 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:34:24.691 INFO:teuthology.orchestra.run.smithi046.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 790 kB/s | 5.8 MB 00:07 2024-04-26T23:34:24.698 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:34:24.751 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:34:24.784 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:34:24.894 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:34:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:34:25.003 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:34:25.057 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:34:25.164 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:34:25.206 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:34:25.255 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:34:25.287 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:34:25.398 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:34:25.456 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:34:25.553 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:34:25.591 INFO:teuthology.orchestra.run.smithi046.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 279 kB/s | 250 kB 00:00 2024-04-26T23:34:25.743 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:34:25.817 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:34:26.425 INFO:teuthology.orchestra.run.smithi046.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 267 kB/s | 222 kB 00:00 2024-04-26T23:34:26.485 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:34:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:34:26.618 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:34:26.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:34:26.703 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:34:26.756 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:34:27.003 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:34:27.080 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:34:27.092 INFO:teuthology.orchestra.run.smithi046.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 116 kB/s | 77 kB 00:00 2024-04-26T23:34:27.161 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:34:27.292 INFO:teuthology.orchestra.run.smithi046.stdout:(84/152): python3-babel-2.9.1-2.el9.noarch.rpm 769 kB/s | 6.0 MB 00:07 2024-04-26T23:34:27.446 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:34:27.491 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:34:27.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:34:27.622 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:34:27.671 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:34:27.734 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:34:27.789 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:34:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 56 kB/s | 42 kB 00:00 2024-04-26T23:34:27.986 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:34:28.042 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:34:28.079 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:34:28.126 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:34:28.201 INFO:teuthology.orchestra.run.smithi046.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 297 kB/s | 269 kB 00:00 2024-04-26T23:34:28.252 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:34:28.296 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:34:28.346 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:34:28.386 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:34:28.419 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:34:28.610 INFO:teuthology.orchestra.run.smithi046.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 210 kB/s | 159 kB 00:00 2024-04-26T23:34:28.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:34:28.789 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:34:28.835 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:34:28.871 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:34:28.929 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:34:28.996 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:34:29.063 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:34:29.112 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:34:29.134 INFO:teuthology.orchestra.run.smithi046.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 300 kB/s | 279 kB 00:00 2024-04-26T23:34:29.158 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:34:29.208 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:34:29.310 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:34:29.356 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:34:29.378 INFO:teuthology.orchestra.run.smithi046.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 67 kB/s | 51 kB 00:00 2024-04-26T23:34:29.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:34:29.559 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:34:29.604 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:34:29.672 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:34:29.820 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:34:29.892 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:34:29.969 INFO:teuthology.orchestra.run.smithi046.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 64 kB/s | 54 kB 00:00 2024-04-26T23:34:30.603 INFO:teuthology.orchestra.run.smithi046.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 66 kB/s | 42 kB 00:00 2024-04-26T23:34:30.851 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:34:30.899 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:34:30.932 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:34:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 764 kB/s | 5.1 MB 00:06 2024-04-26T23:34:31.052 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:34:31.118 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:34:31.215 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:34:31.259 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:34:31.262 INFO:teuthology.orchestra.run.smithi046.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 67 kB/s | 44 kB 00:00 2024-04-26T23:34:31.492 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:34:31.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:34:31.626 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:34:31.971 INFO:teuthology.orchestra.run.smithi046.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 237 kB/s | 221 kB 00:00 2024-04-26T23:34:32.021 INFO:teuthology.orchestra.run.smithi046.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-04-26T23:34:32.442 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:34:32.594 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:34:32.596 INFO:teuthology.orchestra.run.smithi046.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-04-26T23:34:32.971 INFO:teuthology.orchestra.run.smithi046.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 321 kB/s | 305 kB 00:00 2024-04-26T23:34:33.338 INFO:teuthology.orchestra.run.smithi046.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 78 kB/s | 58 kB 00:00 2024-04-26T23:34:33.697 INFO:teuthology.orchestra.run.smithi046.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 129 kB/s | 94 kB 00:00 2024-04-26T23:34:34.206 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:34:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-04-26T23:34:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:34:34.381 INFO:teuthology.orchestra.run.smithi046.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 342 kB/s | 22 kB 00:00 2024-04-26T23:34:34.381 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:34:34.433 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:34:34.481 INFO:teuthology.orchestra.run.smithi046.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.5 MB/s | 863 kB 00:00 2024-04-26T23:34:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:34:34.898 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:34:34.981 INFO:teuthology.orchestra.run.smithi046.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T23:34:35.021 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:34:35.055 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:34:35.081 INFO:teuthology.orchestra.run.smithi046.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-26T23:34:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:34:35.132 INFO:teuthology.orchestra.run.smithi046.stdout:(105/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 36 kB/s | 64 kB 00:01 2024-04-26T23:34:35.299 INFO:teuthology.orchestra.run.smithi046.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 91 kB/s | 19 kB 00:00 2024-04-26T23:34:35.407 INFO:teuthology.orchestra.run.smithi046.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-26T23:34:35.524 INFO:teuthology.orchestra.run.smithi046.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 420 kB/s | 49 kB 00:00 2024-04-26T23:34:35.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:34:35.717 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:34:35.726 INFO:teuthology.orchestra.run.smithi046.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 338 kB/s | 67 kB 00:00 2024-04-26T23:34:35.816 INFO:teuthology.orchestra.run.smithi046.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-26T23:34:35.818 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:34:35.845 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:34:35.975 INFO:teuthology.orchestra.run.smithi046.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-04-26T23:34:36.092 INFO:teuthology.orchestra.run.smithi046.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-04-26T23:34:36.134 INFO:teuthology.orchestra.run.smithi046.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-26T23:34:36.201 INFO:teuthology.orchestra.run.smithi046.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-04-26T23:34:36.268 INFO:teuthology.orchestra.run.smithi046.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-04-26T23:34:36.310 INFO:teuthology.orchestra.run.smithi046.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-04-26T23:34:36.374 INFO:teuthology.orchestra.run.smithi046.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-26T23:34:36.419 INFO:teuthology.orchestra.run.smithi046.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T23:34:36.469 INFO:teuthology.orchestra.run.smithi046.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T23:34:36.503 INFO:teuthology.orchestra.run.smithi046.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 426 kB/s | 35 kB 00:00 2024-04-26T23:34:36.562 INFO:teuthology.orchestra.run.smithi046.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-26T23:34:36.795 INFO:teuthology.orchestra.run.smithi046.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 621 kB/s | 144 kB 00:00 2024-04-26T23:34:36.887 INFO:teuthology.orchestra.run.smithi046.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 118 kB/s | 11 kB 00:00 2024-04-26T23:34:36.971 INFO:teuthology.orchestra.run.smithi046.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2024-04-26T23:34:37.062 INFO:teuthology.orchestra.run.smithi046.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 255 kB/s | 23 kB 00:00 2024-04-26T23:34:37.146 INFO:teuthology.orchestra.run.smithi046.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-26T23:34:37.255 INFO:teuthology.orchestra.run.smithi046.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 183 kB/s | 20 kB 00:00 2024-04-26T23:34:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-26T23:34:37.455 INFO:teuthology.orchestra.run.smithi046.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-04-26T23:34:37.480 INFO:teuthology.orchestra.run.smithi046.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 466 kB/s | 46 kB 00:00 2024-04-26T23:34:37.548 INFO:teuthology.orchestra.run.smithi046.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-04-26T23:34:37.573 INFO:teuthology.orchestra.run.smithi046.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 632 kB/s | 58 kB 00:00 2024-04-26T23:34:37.640 INFO:teuthology.orchestra.run.smithi046.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T23:34:37.812 INFO:teuthology.orchestra.run.smithi046.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 69 kB/s | 16 kB 00:00 2024-04-26T23:34:37.849 INFO:teuthology.orchestra.run.smithi046.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 432 kB/s | 90 kB 00:00 2024-04-26T23:34:37.899 INFO:teuthology.orchestra.run.smithi046.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 331 kB/s | 28 kB 00:00 2024-04-26T23:34:37.941 INFO:teuthology.orchestra.run.smithi046.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 337 kB/s | 31 kB 00:00 2024-04-26T23:34:37.991 INFO:teuthology.orchestra.run.smithi046.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T23:34:38.033 INFO:teuthology.orchestra.run.smithi046.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 648 kB/s | 59 kB 00:00 2024-04-26T23:34:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-04-26T23:34:38.150 INFO:teuthology.orchestra.run.smithi046.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 439 kB/s | 51 kB 00:00 2024-04-26T23:34:38.184 INFO:teuthology.orchestra.run.smithi046.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T23:34:38.243 INFO:teuthology.orchestra.run.smithi046.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 984 kB/s | 90 kB 00:00 2024-04-26T23:34:38.293 INFO:teuthology.orchestra.run.smithi046.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-26T23:34:38.327 INFO:teuthology.orchestra.run.smithi046.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-04-26T23:34:38.477 INFO:teuthology.orchestra.run.smithi046.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-26T23:34:38.523 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:34:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:34:39.059 INFO:teuthology.orchestra.run.smithi046.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-04-26T23:34:39.213 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:34:39.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:34:39.465 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:34:39.465 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:34:39.465 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:34:39.465 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:39.519 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:34:40.202 INFO:teuthology.orchestra.run.smithi046.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.0 MB/s | 3.4 MB 00:01 2024-04-26T23:34:40.286 INFO:teuthology.orchestra.run.smithi046.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-26T23:34:40.703 INFO:teuthology.orchestra.run.smithi046.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-04-26T23:34:40.837 INFO:teuthology.orchestra.run.smithi046.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.4 MB/s | 2.4 MB 00:00 2024-04-26T23:34:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 881 kB/s | 16 MB 00:18 2024-04-26T23:34:48.272 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:34:48.272 INFO:teuthology.orchestra.run.smithi046.stdout:Total 4.1 MB/s | 222 MB 00:54 2024-04-26T23:34:51.233 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-04-26T23:34:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-04-26T23:34:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-04-26T23:34:52.163 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:34:52.163 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-04-26T23:34:52.164 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:52.414 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:34:52.461 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:34:52.462 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:34:52.462 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:34:52.462 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:54.098 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:34:54.175 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:34:54.175 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:34:54.176 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:34:54.176 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:34:54.176 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:34:54.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:54.348 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-04-26T23:34:54.349 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-04-26T23:34:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:34:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:34:54.495 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:34:54.495 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:34:54.495 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:34:54.495 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:34:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:54.991 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:34:55.058 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:34:55.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:34:55.266 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:34:55.344 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:34:55.580 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:34:55.636 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:34:55.722 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:34:55.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:34:56.006 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:34:56.056 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:34:56.101 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:34:56.134 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:34:56.796 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:34:56.845 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:34:56.899 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:34:57.010 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:34:57.039 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:34:58.357 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:34:58.399 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:34:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:34:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:34:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:34:59.839 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:34:59.876 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:34:59.946 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:34:59.947 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:34:59.947 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:34:59.947 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:34:59.947 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:34:59.947 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:00.010 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:35:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:35:00.192 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:35:00.266 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:35:00.837 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-04-26T23:35:00.977 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:35:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:35:01.319 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:35:01.346 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:35:01.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:35:01.421 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:01.422 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:35:01.461 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:35:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:35:01.540 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:35:01.540 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:01.541 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:35:01.541 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:01.613 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:35:01.626 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:35:01.688 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:35:01.688 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:01.688 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:35:01.689 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:35:01.689 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:35:01.689 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:02.064 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:35:02.091 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:35:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:35:02.134 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:02.134 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:35:02.135 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:35:02.135 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:35:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:35:02.374 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:35:02.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:35:02.579 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:35:02.620 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:35:03.011 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:35:03.076 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:35:03.107 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:35:03.280 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:35:03.307 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:35:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:35:03.459 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:35:03.531 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:35:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:35:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:35:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:35:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:35:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:35:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:35:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:35:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:35:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:35:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:35:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:35:05.112 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:35:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:35:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:35:05.328 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:35:05.455 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:35:05.598 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:35:05.662 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:35:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:35:05.869 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:35:05.934 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:35:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:35:06.248 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:35:06.309 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:35:07.040 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:35:07.162 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:35:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:35:07.201 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:35:07.229 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:35:07.246 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:35:07.287 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:35:07.314 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:35:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:35:07.368 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:35:07.416 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:35:07.607 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:35:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:35:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:35:07.759 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:35:07.810 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:35:07.838 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:35:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:35:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:35:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:35:08.058 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:35:08.102 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:35:08.166 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:35:08.222 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:35:08.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:35:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:35:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:35:08.577 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:35:08.718 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:35:08.731 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:35:08.777 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:35:08.831 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:35:08.901 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:35:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:35:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:35:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:35:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:35:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:35:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:35:09.582 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:35:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:35:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:35:09.765 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:35:09.845 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:35:09.896 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:35:10.046 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:35:10.113 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:35:10.154 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:35:10.198 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:35:10.252 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:35:10.308 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:35:10.390 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:35:10.412 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:35:10.475 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:35:10.552 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:35:10.597 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:35:10.597 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:35:10.748 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:35:10.788 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:35:10.788 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:35:10.936 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:35:11.067 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:35:11.372 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:35:11.372 INFO:teuthology.orchestra.run.smithi142.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T23:35:11.372 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:11.481 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:35:11.548 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:35:11.583 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:35:11.718 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:35:11.937 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:35:12.093 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:35:12.142 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:35:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:35:12.472 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:35:12.619 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:35:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:35:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:35:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:35:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:35:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:35:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:35:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:35:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:35:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:35:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:35:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:35:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:35:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:35:17.044 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:35:17.045 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:35:17.046 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:35:17.046 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:35:17.057 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:35:17.057 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:35:17.057 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:35:17.057 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:35:17.057 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:35:17.058 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:35:17.059 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:35:17.060 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:35:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:35:17.062 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:35:17.063 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:35:17.064 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:35:17.065 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:35:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:35:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:35:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:35:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:35:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:35:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:35:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:35:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:35:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:35:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:35:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:35:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:35:19.457 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-04-26T23:35:20.069 DEBUG:teuthology.parallel:result is None 2024-04-26T23:35:20.246 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:35:20.282 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:35:20.967 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:35:21.027 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:35:21.248 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:35:21.248 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:35:21.248 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:35:21.249 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:21.295 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:35:34.932 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-04-26T23:35:34.933 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:35:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:35:35.226 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:35.227 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:35:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:35:36.991 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:37.276 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:35:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:37.876 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:35:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:38.086 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:35:38.148 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:35:38.231 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:35:38.478 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:35:38.531 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:35:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:35:38.738 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:35:38.919 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:35:38.973 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:35:39.013 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:35:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:35:39.782 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:35:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:35:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:35:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:35:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:35:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:35:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:35:42.598 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:35:42.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:35:42.822 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:35:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:35:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:35:43.126 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:35:43.129 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:43.130 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:35:43.130 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:35:43.130 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:35:43.130 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:43.196 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:35:43.239 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:35:43.375 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:35:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:35:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:35:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:35:44.660 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:35:44.661 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:44.661 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:35:44.661 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:35:44.661 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:35:44.661 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:44.708 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:35:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:35:44.793 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:44.793 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:35:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:44.901 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:35:44.998 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:35:44.998 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:44.999 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:35:44.999 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:35:44.999 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:35:44.999 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:45.384 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:35:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:35:45.457 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:35:45.457 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:35:45.457 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:35:45.458 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:35:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:35:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:35:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:35:50.924 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:35:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:35:51.019 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:35:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:35:51.428 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:35:51.456 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:35:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:52.595 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:35:54.002 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:35:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:35:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:35:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:35:54.441 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:35:54.441 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:35:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:35:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:35:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:35:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:35:54.949 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:35:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:35:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T23:35:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:36:00.962 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:36:00.963 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:36:00.964 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:36:00.978 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:36:00.979 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:36:00.980 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:36:00.981 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:36:00.982 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:36:00.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:36:00.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:36:00.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:36:00.986 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:36:00.987 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:36:00.987 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:36:00.987 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:36:03.429 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:36:03.429 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:03.429 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-04-26T23:36:03.429 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.430 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:36:03.432 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.433 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:36:03.434 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:36:03.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:36:03.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:36:03.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:36:03.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:36:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:36:03.440 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:36:03.441 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:36:03.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:03.441 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-04-26T23:36:04.037 DEBUG:teuthology.parallel:result is None 2024-04-26T23:36:04.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:36:04.181 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:36:04.234 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:36:04.234 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:36:04.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:36:04.455 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:36:04.527 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:36:04.528 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:36:04.528 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:36:04.530 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T23:36:04.530 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:04.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:36:04.606 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:04.606 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:36:04.701 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T23:36:04.701 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:04.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:36:04.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:36:04.892 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:04.893 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:36:04.982 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:36:05.062 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T23:36:05.062 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:05.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:36:05.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:36:05.232 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:05.232 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:36:05.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:36:05.421 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T23:36:05.421 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:05.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:36:05.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:36:05.587 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:05.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:36:05.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:36:05.788 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T23:36:05.880 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T23:36:05.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-04-26T23:36:05.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2024-04-26T23:36:05.968 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T23:36:05.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T23:36:05.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T23:36:06.043 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T23:36:06.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T23:36:06.048 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T23:36:06.121 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T23:36:06.122 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T23:36:06.122 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T23:36:06.122 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:06.122 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T23:36:06.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:06.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:06.408 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:06.408 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T23:36:06.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:06.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:06.746 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T23:36:06.747 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T23:36:06.747 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7675316', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCccQe7W9iD7DSxp8gyRYjwlClebnJTFO/4NDpkSDQGrk662jMn/JlxAduabfSm9zmxyew565VyvgfZ8yZlC0iE=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMf4qdrlQPGAtPkCwCEnTxpAWYC4z3GsD2yCxvLqRewf+NVWQI+Adl0jVO2NCa+ybHLQ+e2SaI8M5At2X7kfoqc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T23:36:06.747 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T23:36:06.752 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T23:36:06.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T23:36:06.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T23:36:06.926 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:06.926 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:36:06.978 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:36:06.979 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-04-26T23:36:07.020 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-26 23:35:55.794055594 +0000 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-26 23:33:01.300129291 +0000 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-26 23:33:01.300129291 +0000 2024-04-26T23:36:07.021 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-26 23:33:01.300129291 +0000 2024-04-26T23:36:07.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:36:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:36:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:36:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000244943 s, 2.1 MB/s 2024-04-26T23:36:07.133 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:36:07.176 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-26 23:35:55.794055594 +0000 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-26 23:33:01.943122191 +0000 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-26 23:33:01.943122191 +0000 2024-04-26T23:36:07.272 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-26 23:33:01.943122191 +0000 2024-04-26T23:36:07.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:36:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:36:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:36:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000261238 s, 2.0 MB/s 2024-04-26T23:36:07.346 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:36:07.401 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-04-26T23:36:07.448 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-26 23:35:55.796055572 +0000 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-26 23:33:02.543115566 +0000 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-26 23:33:02.543115566 +0000 2024-04-26T23:36:07.449 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-26 23:33:02.543115566 +0000 2024-04-26T23:36:07.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:36:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:36:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:36:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000200481 s, 2.6 MB/s 2024-04-26T23:36:07.519 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:36:07.561 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-04-26T23:36:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:36:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:07.645 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T23:36:07.645 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:07.646 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:07.646 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-26 23:35:55.796055572 +0000 2024-04-26T23:36:07.646 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-26 23:33:03.122109173 +0000 2024-04-26T23:36:07.646 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-26 23:33:03.122109173 +0000 2024-04-26T23:36:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-26 23:33:03.122109173 +0000 2024-04-26T23:36:07.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:36:07.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:36:07.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:36:07.751 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000209263 s, 2.4 MB/s 2024-04-26T23:36:07.755 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:36:07.796 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:07.796 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:07.796 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:36:07.854 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:36:07.854 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-26 23:35:11.870395730 +0000 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-26 23:32:51.238769760 +0000 2024-04-26T23:36:07.905 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-26 23:32:51.238769760 +0000 2024-04-26T23:36:07.906 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-04-26 23:32:51.238769760 +0000 2024-04-26T23:36:07.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:36:07.977 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-26T23:36:07.977 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-26T23:36:07.977 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000381712 s, 1.3 MB/s 2024-04-26T23:36:07.979 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:36:08.030 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-26 23:35:11.870395730 +0000 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-26 23:32:51.817758954 +0000 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-26 23:32:51.817758954 +0000 2024-04-26T23:36:08.077 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-04-26 23:32:51.817758954 +0000 2024-04-26T23:36:08.078 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:36:08.198 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-26T23:36:08.199 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-26T23:36:08.199 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000366881 s, 1.4 MB/s 2024-04-26T23:36:08.202 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:36:08.253 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-04-26T23:36:08.304 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:36:08.304 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-26 23:35:11.872395693 +0000 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-26 23:32:52.435747421 +0000 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-26 23:32:52.435747421 +0000 2024-04-26T23:36:08.305 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-04-26 23:32:52.435747421 +0000 2024-04-26T23:36:08.305 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:36:08.383 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-26T23:36:08.383 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-26T23:36:08.383 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000371532 s, 1.4 MB/s 2024-04-26T23:36:08.387 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:36:08.443 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-04-26T23:36:08.493 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:36:08.493 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:36:08.493 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-26 23:35:11.872395693 +0000 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-26 23:32:53.048735980 +0000 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-26 23:32:53.048735980 +0000 2024-04-26T23:36:08.494 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-04-26 23:32:53.048735980 +0000 2024-04-26T23:36:08.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:36:08.568 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-26T23:36:08.569 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-26T23:36:08.569 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000312699 s, 1.6 MB/s 2024-04-26T23:36:08.572 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:36:08.623 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:08.623 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T23:36:08.624 INFO:tasks.ceph:Generating config... 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] pid file = 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T23:36:08.625 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T23:36:08.625 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T23:36:08.626 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T23:36:08.626 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T23:36:08.626 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T23:36:08.626 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-26T23:36:08.626 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T23:36:08.627 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T23:36:08.627 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-26T23:36:08.627 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T23:36:08.628 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T23:36:08.628 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T23:36:08.628 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T23:36:08.628 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T23:36:08.629 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T23:36:08.629 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T23:36:08.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T23:36:08.749 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T23:36:08.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T23:36:08.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:36:08.920 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.142'), ('mon.c', '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]')] 2024-04-26T23:36:08.920 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T23:36:08.921 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:08.921 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T23:36:08.960 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.142 --addv c '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-04-26T23:36:09.084789+0000 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-04-26T23:36:09.084789+0000 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 17 (quincy) 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-26T23:36:09.087 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-04-26T23:36:09.088 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] mon.c 2024-04-26T23:36:09.088 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T23:36:09.090 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T23:36:09.131 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 554dfe58-9c53-4ba0-a961-d2b323744aa7... 2024-04-26T23:36:09.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:36:09.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-26T23:36:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:36:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.301 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-26T23:36:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-26T23:36:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-26T23:36:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-26T23:36:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-26T23:36:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-26T23:36:09.308 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: mds cache max size = 1000 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: mds log max segments = 1 2024-04-26T23:36:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-26T23:36:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-04-26T23:36:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:36:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:36:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-26T23:36:09.318 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-26T23:36:09.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-26T23:36:09.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:36:09.321 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-26T23:36:09.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:36:09.323 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:36:09.324 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:36:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-26T23:36:09.326 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-04-26T23:36:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-04-26T23:36:09.328 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds cache max size = 1000 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds log max segments = 1 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:36:09.329 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:36:09.353 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T23:36:09.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T23:36:09.465 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T23:36:09.465 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:09.466 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T23:36:09.510 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:09.510 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T23:36:09.593 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:36:09.593 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:09.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T23:36:09.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:36:09.726 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:09.726 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:09.814 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:36:09.815 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:09.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T23:36:09.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:36:09.938 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:09.939 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:10.034 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T23:36:10.035 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T23:36:10.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T23:36:10.168 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T23:36:10.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T23:36:10.357 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T23:36:10.369 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T23:36:10.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T23:36:10.496 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T23:36:10.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T23:36:10.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T23:36:10.684 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T23:36:10.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T23:36:10.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T23:36:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T23:36:10.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T23:36:11.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T23:36:11.158 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T23:36:11.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T23:36:11.249 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T23:36:11.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T23:36:11.350 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T23:36:11.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T23:36:11.480 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T23:36:11.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T23:36:11.645 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T23:36:11.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T23:36:11.816 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T23:36:11.861 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T23:36:11.861 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T23:36:11.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T23:36:11.958 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:11.958 INFO:tasks.ceph:role: osd.0 2024-04-26T23:36:11.958 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:36:11.958 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T23:36:12.037 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:12.038 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:12.070 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-26T23:36:12.076 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-26T23:36:12.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T23:36:12.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T23:36:12.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T23:36:12.320 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:12.321 INFO:tasks.ceph:role: osd.1 2024-04-26T23:36:12.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:36:12.321 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T23:36:12.387 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:12.388 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:12.421 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-26T23:36:12.426 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-26T23:36:12.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T23:36:12.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T23:36:12.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T23:36:12.663 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:12.663 INFO:tasks.ceph:role: osd.2 2024-04-26T23:36:12.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:36:12.663 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:12.734 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:12.735 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:12.735 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:12.735 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:12.768 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-26T23:36:12.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-26T23:36:12.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T23:36:12.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T23:36:12.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T23:36:13.018 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:13.018 INFO:tasks.ceph:role: osd.3 2024-04-26T23:36:13.019 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:36:13.019 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:13.125 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-26T23:36:13.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-26T23:36:13.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T23:36:13.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T23:36:13.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:13.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.490+0000 7f429b013640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T23:36:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.491+0000 7f429b013640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T23:36:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.491+0000 7f429b013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:36:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.491+0000 7f429b013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:36:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.491+0000 7f429b013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:36:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:13.495+0000 7f429b013640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T23:36:16.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T23:36:16.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.697+0000 7faf3af1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T23:36:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.698+0000 7faf3af1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T23:36:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.698+0000 7faf3af1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:36:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.698+0000 7faf3af1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:36:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.698+0000 7faf3af1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:36:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:16.700+0000 7faf3af1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T23:36:19.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T23:36:20.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.138+0000 7fb7419b1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T23:36:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.139+0000 7fb7419b1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T23:36:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.139+0000 7fb7419b1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:36:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.139+0000 7fb7419b1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:36:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.139+0000 7fb7419b1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:36:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:20.141+0000 7fb7419b1640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T23:36:23.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T23:36:23.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:23.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.582+0000 7f3112814640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T23:36:23.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.583+0000 7f3112814640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T23:36:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.583+0000 7f3112814640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:36:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.583+0000 7f3112814640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:36:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.583+0000 7f3112814640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:36:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:23.585+0000 7f3112814640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T23:36:26.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T23:36:26.897 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T23:36:26.980 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:26.980 INFO:tasks.ceph:role: osd.4 2024-04-26T23:36:26.980 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:36:26.981 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:27.064 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:27.065 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:27.065 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:27.065 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:27.473 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-26T23:36:27.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-26T23:36:27.480 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T23:36:27.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T23:36:27.722 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T23:36:27.804 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:27.804 INFO:tasks.ceph:role: osd.5 2024-04-26T23:36:27.805 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:36:27.805 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:27.887 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:27.888 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:27.888 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:28.243 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-26T23:36:28.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-26T23:36:28.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T23:36:28.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T23:36:28.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T23:36:28.522 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:28.523 INFO:tasks.ceph:role: osd.6 2024-04-26T23:36:28.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:36:28.523 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:28.611 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:28.612 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:28.993 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-26T23:36:29.000 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-26T23:36:29.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T23:36:29.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T23:36:29.225 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T23:36:29.306 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:36:29.307 INFO:tasks.ceph:role: osd.7 2024-04-26T23:36:29.307 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:36:29.307 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:36:29.392 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:36:29.393 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:36:29.809 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-26T23:36:29.815 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-26T23:36:29.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T23:36:29.981 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T23:36:30.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:30.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.204+0000 7ffaba301640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T23:36:30.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.205+0000 7ffaba301640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T23:36:30.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.205+0000 7ffaba301640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:36:30.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.205+0000 7ffaba301640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:36:30.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.205+0000 7ffaba301640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:36:30.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:30.208+0000 7ffaba301640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T23:36:33.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T23:36:33.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:33.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.408+0000 7f77aca1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T23:36:33.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.409+0000 7f77aca1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T23:36:33.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.409+0000 7f77aca1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:36:33.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.409+0000 7f77aca1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:36:33.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.409+0000 7f77aca1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:36:33.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:33.411+0000 7f77aca1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T23:36:36.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T23:36:36.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:36.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.571+0000 7f8b72e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T23:36:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.572+0000 7f8b72e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T23:36:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.572+0000 7f8b72e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:36:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.572+0000 7f8b72e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:36:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.572+0000 7f8b72e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:36:36.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:36.574+0000 7f8b72e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T23:36:36.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:36.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:39.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T23:36:39.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:39.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.990+0000 7fba98c16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T23:36:39.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.991+0000 7fba98c16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T23:36:39.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.991+0000 7fba98c16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:36:39.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.991+0000 7fba98c16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:36:39.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.991+0000 7fba98c16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:36:39.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:39.993+0000 7fba98c16640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T23:36:43.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T23:36:43.280 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T23:36:43.280 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T23:36:43.377 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T23:36:43.467 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T23:36:43.556 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T23:36:43.644 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T23:36:43.735 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T23:36:43.823 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:43.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T23:36:43.917 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:43.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T23:36:44.006 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T23:36:44.088 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T23:36:44.167 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T23:36:44.295 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T23:36:44.378 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T23:36:44.465 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T23:36:44.549 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:44.549 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T23:36:44.610 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:44.610 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T23:36:44.664 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.664 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T23:36:44.721 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:44.721 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T23:36:44.772 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T23:36:44.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T23:36:44.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T23:36:44.858 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.y] 2024-04-26T23:36:44.858 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.0] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.1] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.2] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.3] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw== 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.x] 2024-04-26T23:36:44.859 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.4] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.5] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.6] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.7] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:36:44.860 INFO:teuthology.orchestra.run.smithi142.stdout:[client.2] 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ== 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg== 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi142.stdout:[client.3] 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA== 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg== 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg== 2024-04-26T23:36:44.861 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g== 2024-04-26T23:36:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg== 2024-04-26T23:36:44.863 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2024-04-26T23:36:44.864 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA== 2024-04-26T23:36:44.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:36:44.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:36:45.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:45.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:45.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:45.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:45.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:45.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:36:45.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:36:46.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:46.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:46.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:46.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:36:46.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.729 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:36:46.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:46.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.015 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:36:47.431 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T23:36:47.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T23:36:47.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:36:47.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T23:36:47.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T23:36:47.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:36:48.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T23:36:48.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T23:36:48.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:36:48.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T23:36:48.376 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:48.384 DEBUG:teuthology.orchestra.run.smithi142:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:36:48.434 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T23:36:48.435 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T23:36:48.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T23:36:48.441 INFO:tasks.ceph.mon.a:Started 2024-04-26T23:36:48.441 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T23:36:48.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T23:36:48.446 INFO:tasks.ceph.mon.b:Started 2024-04-26T23:36:48.447 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T23:36:48.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T23:36:48.453 INFO:tasks.ceph.mon.c:Started 2024-04-26T23:36:48.453 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T23:36:48.453 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T23:36:48.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T23:36:48.458 INFO:tasks.ceph.mgr.y:Started 2024-04-26T23:36:48.458 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T23:36:48.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T23:36:48.463 INFO:tasks.ceph.mgr.x:Started 2024-04-26T23:36:48.463 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T23:36:48.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T23:36:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.633+0000 7fb23a600640 1 Processor -- start 2024-04-26T23:36:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.633+0000 7fb23a600640 1 -- start start 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb23a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb23a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 0x7fb23406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb23a600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23406eb90 con 0x7fb2340b53b0 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb23a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb2340b4310 con 0x7fb23415e220 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb23a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2340b45f0 con 0x7fb23415d730 2024-04-26T23:36:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 0x7fb23406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:32898/0 (socket says 172.21.15.46:32898) 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 learned_addr learned my addr 172.21.15.46:0/1117872594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233e00640 1 -- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 msgr2=0x7fb23406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233e00640 1 -- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 msgr2=0x7fb23406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233e00640 1 --2- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 0x7fb23406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 msgr2=0x7fb2340d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 msgr2=0x7fb2340d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.635+0000 7fb233e00640 1 --2- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 0x7fb23406dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:36:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.636+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:36:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.656+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213144456 0 0) 0x7fb23406eb90 con 0x7fb2340b53b0 2024-04-26T23:36:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.656+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb218003520 con 0x7fb2340b53b0 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912970084 0 0) 0x7fb218003520 con 0x7fb2340b53b0 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 msgr2=0x7fb2340d00d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2340d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 msgr2=0x7fb23406dc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 --2- 172.21.15.46:0/1117872594 >> v2:172.21.15.142:3300/0 conn(0x7fb23415e220 0x7fb23406dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2340b48d0 con 0x7fb2340b53b0 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb220001670 con 0x7fb2340b53b0 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 >> v1:172.21.15.142:6789/0 conn(0x7fb2340b53b0 legacy=0x7fb23415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb218004030 0x7fb218004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb218004d40 0x7fb2180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b4310 con 0x7fb218004030 2024-04-26T23:36:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b45f0 con 0x7fb218004d40 2024-04-26T23:36:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb21800bb20 con 0x7fb23415d730 2024-04-26T23:36:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.658+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233e00640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb218004d40 0x7fb2180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb232a00640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb218004030 0x7fb218004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb218004d40 msgr2=0x7fb2180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb218004d40 0x7fb2180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb218004030 msgr2=0x7fb218004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb218004030 0x7fb218004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233400640 1 -- 172.21.15.46:0/1117872594 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb218003520 con 0x7fb23415d730 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.659+0000 7fb233e00640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb218004d40 0x7fb2180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:36:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.660+0000 7fb233400640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2180039b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb224009cc0 tx=0x7fb218005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70f0fb77f6abaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.660+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224038070 con 0x7fb23415d730 2024-04-26T23:36:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.660+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb224033410 con 0x7fb23415d730 2024-04-26T23:36:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.660+0000 7fb232000640 1 -- 172.21.15.46:0/1117872594 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2240098c0 con 0x7fb23415d730 2024-04-26T23:36:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 -- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 msgr2=0x7fb2180039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 --2- 172.21.15.46:0/1117872594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415d730 0x7fb2180039b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb224009cc0 tx=0x7fb218005830 comp rx=0 tx=0).stop 2024-04-26T23:36:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb233e00640 1 -- 172.21.15.46:0/1117872594 reap_dead start 2024-04-26T23:36:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 -- 172.21.15.46:0/1117872594 shutdown_connections 2024-04-26T23:36:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 -- 172.21.15.46:0/1117872594 >> 172.21.15.46:0/1117872594 conn(0x7fb23406c980 msgr2=0x7fb2340b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 -- 172.21.15.46:0/1117872594 shutdown_connections 2024-04-26T23:36:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.661+0000 7fb23a600640 1 -- 172.21.15.46:0/1117872594 wait complete. 2024-04-26T23:36:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.663+0000 7fb23a600640 1 Processor -- start 2024-04-26T23:36:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.663+0000 7fb23a600640 1 -- start start 2024-04-26T23:36:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 0x7fb234145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415c610 0x7fb2340c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b3ec0 con 0x7fb23415d730 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b4e90 con 0x7fb2340c7a80 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.664+0000 7fb23a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2340b4040 con 0x7fb23415c610 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:52996/0 (socket says 172.21.15.46:52996) 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 -- 172.21.15.46:0/2112894261 learned_addr learned my addr 172.21.15.46:0/2112894261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb233e00640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415c610 0x7fb2340c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb233400640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 0x7fb234145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 -- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415c610 msgr2=0x7fb2340c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415c610 0x7fb2340c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 -- 172.21.15.46:0/2112894261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 msgr2=0x7fb234145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 0x7fb234145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb232a00640 1 -- 172.21.15.46:0/2112894261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2340c69e0 con 0x7fb2340c7a80 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.665+0000 7fb233400640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 0x7fb234145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb232a00640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e790 tx=0x7fb22000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000eea0 con 0x7fb2340c7a80 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb220018070 con 0x7fb2340c7a80 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000c6c0 con 0x7fb2340c7a80 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2340c6cc0 con 0x7fb2340c7a80 2024-04-26T23:36:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2340c7280 con 0x7fb2340c7a80 2024-04-26T23:36:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb22000c860 con 0x7fb2340c7a80 2024-04-26T23:36:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.666+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb220003dc0 con 0x7fb2340c7a80 2024-04-26T23:36:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.667+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb234069530 con 0x7fb2340c7a80 2024-04-26T23:36:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.671+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb22000cc90 con 0x7fb2340c7a80 2024-04-26T23:36:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.744+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb23406e500 con 0x7fb2340c7a80 2024-04-26T23:36:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.751+0000 7fb22b400640 1 -- 172.21.15.46:0/2112894261 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb2200035f0 con 0x7fb2340c7a80 2024-04-26T23:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-04-26T23:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.752+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 msgr2=0x7fb23415c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.752+0000 7fb23a600640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e790 tx=0x7fb22000ec50 comp rx=0 tx=0).stop 2024-04-26T23:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 shutdown_connections 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb23415c610 0x7fb2340c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2340c7a80 0x7fb23415c0d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 --2- 172.21.15.46:0/2112894261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23415d730 0x7fb234145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 >> 172.21.15.46:0/2112894261 conn(0x7fb23406c980 msgr2=0x7fb23415cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 shutdown_connections 2024-04-26T23:36:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:48.753+0000 7fb23a600640 1 -- 172.21.15.46:0/2112894261 wait complete. 2024-04-26T23:36:48.773 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T23:36:48.773 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T23:36:48.774 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:48.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T23:36:48.846 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:48.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T23:36:48.958 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:48.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T23:36:48.965 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:48.961+0000 7f3115211040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:36:48.982 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:48.978+0000 7fb5fd012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:36:49.031 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:36:49.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T23:36:49.144 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:49.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T23:36:49.167 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:49.163+0000 7f3115211040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:36:49.186 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:49.182+0000 7fb5fd012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:36:49.221 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:49.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T23:36:49.233 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:49.230+0000 7f3115211040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:36:49.253 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:49.249+0000 7fb5fd012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:36:49.326 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:49.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T23:36:49.443 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:36:49.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T23:36:49.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:49.522+0000 7f3115211040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:36:49.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f 0 2024-04-26T23:36:49.569 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:49.567+0000 7fb5fd012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:36:49.582 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:49.578+0000 7f3115211040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:36:49.627 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:49.624+0000 7fb5fd012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:36:49.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.751+0000 7ff0d7a00640 1 Processor -- start 2024-04-26T23:36:49.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.751+0000 7ff0d7a00640 1 -- start start 2024-04-26T23:36:49.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d7a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:49.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d7a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff0d00d5b90 0x7ff0d00d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d7a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d00b46f0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d7a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d00b44a0 con 0x7ff0d00d5b90 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d7a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d00b5020 con 0x7ff0d0162aa0 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34754/0 (socket says 172.21.15.142:34754) 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.753+0000 7ff0d5000640 1 -- 172.21.15.142:0/2200770477 learned_addr learned my addr 172.21.15.142:0/2200770477 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.754+0000 7ff0d5000640 1 -- 172.21.15.142:0/2200770477 >> v2:172.21.15.142:3300/0 conn(0x7ff0d00d5b90 msgr2=0x7ff0d00d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.754+0000 7ff0d5000640 1 --2- 172.21.15.142:0/2200770477 >> v2:172.21.15.142:3300/0 conn(0x7ff0d00d5b90 0x7ff0d00d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.754+0000 7ff0d5000640 1 -- 172.21.15.142:0/2200770477 >> v1:172.21.15.142:6789/0 conn(0x7ff0d00b54b0 legacy=0x7ff0d00b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:49.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.754+0000 7ff0d5000640 1 -- 172.21.15.142:0/2200770477 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d00b49d0 con 0x7ff0d0162aa0 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.755+0000 7ff0d5000640 1 --2- 172.21.15.142:0/2200770477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009870 tx=0x7ff0c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0267fe9fe992e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.755+0000 7ff0cf400640 1 -- 172.21.15.142:0/2200770477 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4037070 con 0x7ff0d0162aa0 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.755+0000 7ff0cf400640 1 -- 172.21.15.142:0/2200770477 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c4032440 con 0x7ff0d0162aa0 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.755+0000 7ff0cf400640 1 -- 172.21.15.142:0/2200770477 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4032810 con 0x7ff0d0162aa0 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2200770477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 msgr2=0x7ff0d00d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2200770477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009870 tx=0x7ff0c402a180 comp rx=0 tx=0).stop 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2200770477 shutdown_connections 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2200770477 >> v2:172.21.15.142:3300/0 conn(0x7ff0d00d5b90 0x7ff0d00d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2200770477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d0162aa0 0x7ff0d00d3650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2200770477 >> 172.21.15.142:0/2200770477 conn(0x7ff0d00d5070 msgr2=0x7ff0d00d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:49.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2200770477 shutdown_connections 2024-04-26T23:36:49.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.756+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2200770477 wait complete. 2024-04-26T23:36:49.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.758+0000 7ff0d7a00640 1 Processor -- start 2024-04-26T23:36:49.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.758+0000 7ff0d7a00640 1 -- start start 2024-04-26T23:36:49.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:49.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 0x7ff0d00c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:49.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:49.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d00d30f0 con 0x7ff0d00d5b90 2024-04-26T23:36:49.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0162e80 con 0x7ff0d0162aa0 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d7a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d00d0c90 con 0x7ff0d00b54b0 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34768/0 (socket says 172.21.15.142:34768) 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45350/0 (socket says 172.21.15.142:45350) 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 -- 172.21.15.142:0/2852890525 learned_addr learned my addr 172.21.15.142:0/2852890525 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0cfe00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 0x7ff0d00c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 -- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 msgr2=0x7ff0d014c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 -- 172.21.15.142:0/2852890525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 msgr2=0x7ff0d00c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 0x7ff0d00c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5000640 1 -- 172.21.15.142:0/2852890525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d014c9c0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0cfe00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 0x7ff0d00c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.759+0000 7ff0d5a00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0d5000640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009870 tx=0x7ff0c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4041070 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c403c870 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d00cb9e0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4037040 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d00c9580 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.760+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0c4044070 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.761+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff0c403d3f0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.761+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d0069530 con 0x7ff0d00b54b0 2024-04-26T23:36:49.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.766+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff0d014c9c0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.844+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f", "id": 0} v 0) -- 0x7ff0d0069be0 con 0x7ff0d00b54b0 2024-04-26T23:36:49.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.850+0000 7ff0ccc00640 1 -- 172.21.15.142:0/2852890525 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff0c403cc60 con 0x7ff0d00b54b0 2024-04-26T23:36:49.853 INFO:teuthology.orchestra.run.smithi142.stdout:0 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 msgr2=0x7ff0d00ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009870 tx=0x7ff0c40046c0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 shutdown_connections 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0d0162aa0 0x7ff0d014c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d00d5b90 0x7ff0d00c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 --2- 172.21.15.142:0/2852890525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0d00b54b0 0x7ff0d00ca340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 >> 172.21.15.142:0/2852890525 conn(0x7ff0d00d5070 msgr2=0x7ff0d006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 shutdown_connections 2024-04-26T23:36:49.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:49.852+0000 7ff0d7a00640 1 -- 172.21.15.142:0/2852890525 wait complete. 2024-04-26T23:36:49.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 95298e5e-636c-4b28-a96c-2c87dec49625 1 2024-04-26T23:36:50.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.027+0000 7fa1c0e00640 1 Processor -- start 2024-04-26T23:36:50.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.027+0000 7fa1c0e00640 1 -- start start 2024-04-26T23:36:50.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.028+0000 7fa1c0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.028+0000 7fa1c0e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.028+0000 7fa1c0e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1bc0b46f0 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.028+0000 7fa1c0e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa1bc0b44a0 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.028+0000 7fa1c0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa1bc0b5020 con 0x7fa1bc162aa0 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45352/0 (socket says 172.21.15.142:45352) 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 -- 172.21.15.142:0/1307585763 learned_addr learned my addr 172.21.15.142:0/1307585763 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1ba800640 1 --1- 172.21.15.142:0/1307585763 >> v1:172.21.15.142:6789/0 conn(0x7fa1bc0b54b0 0x7fa1bc0b58f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55062/0 (socket says 172.21.15.142:55062) 2024-04-26T23:36:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 -- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 msgr2=0x7fa1bc0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 --2- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 -- 172.21.15.142:0/1307585763 >> v1:172.21.15.142:6789/0 conn(0x7fa1bc0b54b0 legacy=0x7fa1bc0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b9400640 1 -- 172.21.15.142:0/1307585763 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc0b49d0 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.029+0000 7fa1b8a00640 1 -- 172.21.15.142:0/1307585763 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437059329 0 0) 0x7fa1bc0b46f0 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.030+0000 7fa1b9400640 1 --2- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009870 tx=0x7fa1a802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4d2ce7424a839c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.030+0000 7fa1b8a00640 1 -- 172.21.15.142:0/1307585763 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8037070 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.030+0000 7fa1b8a00640 1 -- 172.21.15.142:0/1307585763 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a8032440 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.030+0000 7fa1b8a00640 1 -- 172.21.15.142:0/1307585763 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8032810 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 -- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0d5b90 msgr2=0x7fa1bc0d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009870 tx=0x7fa1a802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 -- 172.21.15.142:0/1307585763 shutdown_connections 2024-04-26T23:36:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc0d3b90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/1307585763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 -- 172.21.15.142:0/1307585763 >> 172.21.15.142:0/1307585763 conn(0x7fa1bc0d5070 msgr2=0x7fa1bc0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 -- 172.21.15.142:0/1307585763 shutdown_connections 2024-04-26T23:36:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.031+0000 7fa1c0e00640 1 -- 172.21.15.142:0/1307585763 wait complete. 2024-04-26T23:36:50.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.033+0000 7fa1c0e00640 1 Processor -- start 2024-04-26T23:36:50.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.033+0000 7fa1c0e00640 1 -- start start 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc1e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc0d30f0 con 0x7fa1bc0d5b90 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc162e80 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1c0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa1bc0d0c90 con 0x7fa1bc162aa0 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1b9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1b9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45362/0 (socket says 172.21.15.142:45362) 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1ba800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.034+0000 7fa1b9e00640 1 -- 172.21.15.142:0/2514228429 learned_addr learned my addr 172.21.15.142:0/2514228429 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 -- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 msgr2=0x7fa1bc0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 -- 172.21.15.142:0/2514228429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc0d5b90 msgr2=0x7fa1bc1e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc1e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 -- 172.21.15.142:0/2514228429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc161010 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1ba800640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b9e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e870 tx=0x7fa1ac00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac017070 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac00c940 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00cd30 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc160030 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.035+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1605b0 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.036+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa1ac01a070 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.036+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa1ac005e00 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.037+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc069530 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.039+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa1bc161010 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.157+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "95298e5e-636c-4b28-a96c-2c87dec49625", "id": 1} v 0) -- 0x7fa1bc0baed0 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.163+0000 7fa1b2000640 1 -- 172.21.15.142:0/2514228429 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95298e5e-636c-4b28-a96c-2c87dec49625", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa1ac013470 con 0x7fa1bc0b54b0 2024-04-26T23:36:50.166 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-04-26T23:36:50.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 msgr2=0x7fa1bc160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e870 tx=0x7fa1ac00ed30 comp rx=0 tx=0).stop 2024-04-26T23:36:50.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 shutdown_connections 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1bc162aa0 0x7fa1bc0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc0d5b90 0x7fa1bc1e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 --2- 172.21.15.142:0/2514228429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1bc0b54b0 0x7fa1bc160ad0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 >> 172.21.15.142:0/2514228429 conn(0x7fa1bc0d5070 msgr2=0x7fa1bc06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 shutdown_connections 2024-04-26T23:36:50.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.164+0000 7fa1c0e00640 1 -- 172.21.15.142:0/2514228429 wait complete. 2024-04-26T23:36:50.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new ddd74cdc-da9f-49c8-a006-719a401998cd 2 2024-04-26T23:36:50.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.340+0000 7f3014e00640 1 Processor -- start 2024-04-26T23:36:50.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.341+0000 7f3014e00640 1 -- start start 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f3014e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30100723e0 0x7f3010163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f3014e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3010072880 con 0x7f3010071f80 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f3014e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3010072b40 con 0x7f301015dd60 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f3014e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f301015e140 con 0x7f30100723e0 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f300de00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f301015dd60 0x7f30100b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55070/0 (socket says 172.21.15.142:55070) 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.342+0000 7f300de00640 1 -- 172.21.15.142:0/3891596266 learned_addr learned my addr 172.21.15.142:0/3891596266 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383665212 0 0) 0x7f3010072b40 con 0x7f301015dd60 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ff4003520 con 0x7f301015dd60 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3537751790 0 0) 0x7f2ff4003520 con 0x7f301015dd60 2024-04-26T23:36:50.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 >> v2:172.21.15.46:3300/0 conn(0x7f30100723e0 msgr2=0x7f3010163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 --2- 172.21.15.142:0/3891596266 >> v2:172.21.15.46:3300/0 conn(0x7f30100723e0 0x7f3010163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 >> v1:172.21.15.46:6789/0 conn(0x7f3010071f80 legacy=0x7f301006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3010072e20 con 0x7f301015dd60 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2ff8001670 con 0x7f301015dd60 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 >> v1:172.21.15.142:6789/0 conn(0x7f301015dd60 legacy=0x7f30100b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ff4004800 0x7f2ff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ff40052e0 0x7f2ff400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff400c1b0 con 0x7f2ff4004800 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff400c360 con 0x7f2ff40039b0 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff400c560 con 0x7f2ff40052e0 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300de00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.343+0000 7f300e800640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ff40052e0 0x7f2ff400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300de00640 1 -- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ff40052e0 msgr2=0x7f2ff400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300de00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ff40052e0 0x7f2ff400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300de00640 1 -- 172.21.15.142:0/3891596266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ff4004800 msgr2=0x7f2ff4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300de00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ff4004800 0x7f2ff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300de00640 1 -- 172.21.15.142:0/3891596266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff4003520 con 0x7f2ff40039b0 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.344+0000 7f300e800640 1 -- 172.21.15.142:0/3891596266 reap_dead start 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.345+0000 7f300de00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003db0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c210 tx=0x7f2ff4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab398fd973429e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.345+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8007d10 con 0x7f2ff40039b0 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.345+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8005ad0 con 0x7f2ff40039b0 2024-04-26T23:36:50.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.345+0000 7f300d400640 1 -- 172.21.15.142:0/3891596266 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8007070 con 0x7f2ff40039b0 2024-04-26T23:36:50.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 -- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 msgr2=0x7f2ff4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003db0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c210 tx=0x7f2ff4005830 comp rx=0 tx=0).stop 2024-04-26T23:36:50.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 -- 172.21.15.142:0/3891596266 shutdown_connections 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 --2- 172.21.15.142:0/3891596266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003db0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 -- 172.21.15.142:0/3891596266 >> 172.21.15.142:0/3891596266 conn(0x7f301006c980 msgr2=0x7f30100b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 -- 172.21.15.142:0/3891596266 shutdown_connections 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.346+0000 7f3014e00640 1 -- 172.21.15.142:0/3891596266 wait complete. 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.348+0000 7f3014e00640 1 Processor -- start 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.348+0000 7f3014e00640 1 -- start start 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.348+0000 7f3014e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3014e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30101f9190 0x7f301015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3014e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 0x7f3010157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3014e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3010163bc0 con 0x7f30101f9190 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3014e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30100733e0 con 0x7f2ff40039b0 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3014e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3010163d40 con 0x7f30101fd490 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45388/0 (socket says 172.21.15.142:45388) 2024-04-26T23:36:50.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 -- 172.21.15.142:0/862208122 learned_addr learned my addr 172.21.15.142:0/862208122 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300e800640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 0x7f3010157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f3005400640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30101f9190 0x7f301015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 -- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 msgr2=0x7f3010157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 0x7f3010157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 -- 172.21.15.142:0/862208122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30101f9190 msgr2=0x7f301015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30101f9190 0x7f301015fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300de00640 1 -- 172.21.15.142:0/862208122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3010152840 con 0x7f2ff40039b0 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.349+0000 7f300e800640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 0x7f3010157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:50.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f300de00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802ada0 tx=0x7f3010072ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff803f070 con 0x7f2ff40039b0 2024-04-26T23:36:50.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8001430 con 0x7f2ff40039b0 2024-04-26T23:36:50.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3010152ab0 con 0x7f2ff40039b0 2024-04-26T23:36:50.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005930 con 0x7f2ff40039b0 2024-04-26T23:36:50.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3010153070 con 0x7f2ff40039b0 2024-04-26T23:36:50.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ff8042070 con 0x7f2ff40039b0 2024-04-26T23:36:50.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.350+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2ff8007490 con 0x7f2ff40039b0 2024-04-26T23:36:50.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.351+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3010069530 con 0x7f2ff40039b0 2024-04-26T23:36:50.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.353+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2ff8049020 con 0x7f2ff40039b0 2024-04-26T23:36:50.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.432+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ddd74cdc-da9f-49c8-a006-719a401998cd", "id": 2} v 0) -- 0x7f30100b36f0 con 0x7f2ff40039b0 2024-04-26T23:36:50.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.438+0000 7f3006a00640 1 -- 172.21.15.142:0/862208122 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddd74cdc-da9f-49c8-a006-719a401998cd", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2ff8007070 con 0x7f2ff40039b0 2024-04-26T23:36:50.441 INFO:teuthology.orchestra.run.smithi142.stdout:2 2024-04-26T23:36:50.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 msgr2=0x7f301014b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802ada0 tx=0x7f3010072ff0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 shutdown_connections 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30101fd490 0x7f3010157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30101f9190 0x7f301015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 --2- 172.21.15.142:0/862208122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ff40039b0 0x7f301014b2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 >> 172.21.15.142:0/862208122 conn(0x7f301006c980 msgr2=0x7f301015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 shutdown_connections 2024-04-26T23:36:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.440+0000 7f3014e00640 1 -- 172.21.15.142:0/862208122 wait complete. 2024-04-26T23:36:50.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 1798bf3a-bfea-4dbb-8467-b047a259e4d1 3 2024-04-26T23:36:50.582 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:50.578+0000 7f3115211040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:36:50.594 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:50.591+0000 7fb5fd012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:36:50.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.641+0000 7f8bada00640 1 Processor -- start 2024-04-26T23:36:50.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.641+0000 7f8bada00640 1 -- start start 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bada00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ba806d8c0 0x7f8ba80d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bada00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba80d0a80 con 0x7f8ba815f490 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bada00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba80d0d60 con 0x7f8ba806d430 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bada00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba80d3060 con 0x7f8ba806d8c0 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bad000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ba806d8c0 0x7f8ba80d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8ba7400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8ba806d430 0x7f8ba80d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55074/0 (socket says 172.21.15.142:55074) 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bad000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ba806d8c0 0x7f8ba80d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45404/0 (socket says 172.21.15.142:45404) 2024-04-26T23:36:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8ba7400640 1 -- 172.21.15.142:0/856747722 learned_addr learned my addr 172.21.15.142:0/856747722 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-04-26T23:36:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bad000640 1 -- 172.21.15.142:0/856747722 >> v1:172.21.15.142:6789/0 conn(0x7f8ba806d430 legacy=0x7f8ba80d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8bad000640 1 -- 172.21.15.142:0/856747722 >> v1:172.21.15.46:6789/0 conn(0x7f8ba815f490 legacy=0x7f8ba806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.643+0000 7f8ba6a00640 1 -- 172.21.15.142:0/856747722 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644482113 0 0) 0x7f8ba80d0d60 con 0x7f8ba806d430 2024-04-26T23:36:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.644+0000 7f8bad000640 1 -- 172.21.15.142:0/856747722 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba80d3340 con 0x7f8ba806d8c0 2024-04-26T23:36:50.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8bad000640 1 --2- 172.21.15.142:0/856747722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d8c0 0x7f8ba80d3f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009d90 tx=0x7f8ba002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba495c93e3974b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8ba6a00640 1 -- 172.21.15.142:0/856747722 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0007740 con 0x7f8ba806d8c0 2024-04-26T23:36:50.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8ba6a00640 1 -- 172.21.15.142:0/856747722 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba00078e0 con 0x7f8ba806d8c0 2024-04-26T23:36:50.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8ba6a00640 1 -- 172.21.15.142:0/856747722 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0007cb0 con 0x7f8ba806d8c0 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8bada00640 1 -- 172.21.15.142:0/856747722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d8c0 msgr2=0x7f8ba80d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.645+0000 7f8bada00640 1 --2- 172.21.15.142:0/856747722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d8c0 0x7f8ba80d3f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009d90 tx=0x7f8ba002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.646+0000 7f8bada00640 1 -- 172.21.15.142:0/856747722 shutdown_connections 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.646+0000 7f8bada00640 1 --2- 172.21.15.142:0/856747722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d8c0 0x7f8ba80d3f70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.646+0000 7f8bada00640 1 -- 172.21.15.142:0/856747722 >> 172.21.15.142:0/856747722 conn(0x7f8ba80d4fc0 msgr2=0x7f8ba80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.646+0000 7f8bada00640 1 -- 172.21.15.142:0/856747722 shutdown_connections 2024-04-26T23:36:50.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.646+0000 7f8bada00640 1 -- 172.21.15.142:0/856747722 wait complete. 2024-04-26T23:36:50.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.647+0000 7f8bada00640 1 Processor -- start 2024-04-26T23:36:50.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.647+0000 7f8bada00640 1 -- start start 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 0x7f8ba80b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba815f490 0x7f8ba80b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80d06e0 con 0x7f8ba815f490 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80d2880 con 0x7f8ba806d430 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.648+0000 7f8bada00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80d0860 con 0x7f8ba806d8c0 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 0x7f8ba80b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45418/0 (socket says 172.21.15.142:45418) 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 -- 172.21.15.142:0/2598481455 learned_addr learned my addr 172.21.15.142:0/2598481455 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8bad000640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba815f490 0x7f8ba80b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 -- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 msgr2=0x7f8ba80b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 0x7f8ba80b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 -- 172.21.15.142:0/2598481455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba815f490 msgr2=0x7f8ba80b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba815f490 0x7f8ba80b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 -- 172.21.15.142:0/2598481455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba815f930 con 0x7f8ba806d430 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7400640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 0x7f8ba80b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.649+0000 7f8ba7e00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c400 tx=0x7f8b9800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x7f8ba806d430 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980052d0 con 0x7f8ba806d430 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98003830 con 0x7f8ba806d430 2024-04-26T23:36:50.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba815fbf0 con 0x7f8ba806d430 2024-04-26T23:36:50.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81601b0 con 0x7f8ba806d430 2024-04-26T23:36:50.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b9801a070 con 0x7f8ba806d430 2024-04-26T23:36:50.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.650+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f8b98011590 con 0x7f8ba806d430 2024-04-26T23:36:50.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.651+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba8069ac0 con 0x7f8ba806d430 2024-04-26T23:36:50.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.653+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8b98021070 con 0x7f8ba806d430 2024-04-26T23:36:50.690 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:50.686+0000 7f3115211040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:36:50.704 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:50.701+0000 7fb5fd012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:36:50.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.734+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1798bf3a-bfea-4dbb-8467-b047a259e4d1", "id": 3} v 0) -- 0x7f8ba815eb40 con 0x7f8ba806d430 2024-04-26T23:36:50.743 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:50.739+0000 7f3115211040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:36:50.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.743+0000 7f8b9fe00640 1 -- 172.21.15.142:0/2598481455 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1798bf3a-bfea-4dbb-8467-b047a259e4d1", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8b98011b80 con 0x7f8ba806d430 2024-04-26T23:36:50.746 INFO:teuthology.orchestra.run.smithi142.stdout:3 2024-04-26T23:36:50.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 msgr2=0x7f8ba8156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c400 tx=0x7f8b9800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 shutdown_connections 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba815f490 0x7f8ba80b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba806d8c0 0x7f8ba80b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 --2- 172.21.15.142:0/2598481455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ba806d430 0x7f8ba8156600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 >> 172.21.15.142:0/2598481455 conn(0x7f8ba80d4fc0 msgr2=0x7f8ba80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.744+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 shutdown_connections 2024-04-26T23:36:50.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.745+0000 7f8bada00640 1 -- 172.21.15.142:0/2598481455 wait complete. 2024-04-26T23:36:50.757 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:50.754+0000 7fb5fd012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:36:50.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1 4 2024-04-26T23:36:50.942 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:50.938+0000 7f3115211040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:36:50.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.948+0000 7f7a4de00640 1 Processor -- start 2024-04-26T23:36:50.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.948+0000 7f7a4de00640 1 -- start start 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.949+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.949+0000 7f7a4de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a48160180 0x7f7a480b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a48068710 con 0x7f7a480cea30 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480b4c20 con 0x7f7a48160180 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a481608c0 con 0x7f7a4815be30 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34816/0 (socket says 172.21.15.142:34816) 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 -- 172.21.15.142:0/2374658250 learned_addr learned my addr 172.21.15.142:0/2374658250 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a47e00640 1 --1- 172.21.15.142:0/2374658250 >> v1:172.21.15.142:6789/0 conn(0x7f7a480cea30 0x7f7a4815fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55090/0 (socket says 172.21.15.142:55090) 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4d400640 1 --2- 172.21.15.142:0/2374658250 >> v2:172.21.15.46:3300/0 conn(0x7f7a48160180 0x7f7a480b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 -- 172.21.15.142:0/2374658250 >> v2:172.21.15.46:3300/0 conn(0x7f7a48160180 msgr2=0x7f7a480b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 --2- 172.21.15.142:0/2374658250 >> v2:172.21.15.46:3300/0 conn(0x7f7a48160180 0x7f7a480b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 -- 172.21.15.142:0/2374658250 >> v1:172.21.15.142:6789/0 conn(0x7f7a480cea30 legacy=0x7f7a4815fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a47400640 1 -- 172.21.15.142:0/2374658250 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749053539 0 0) 0x7f7a48068710 con 0x7f7a480cea30 2024-04-26T23:36:50.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.950+0000 7f7a4ca00640 1 -- 172.21.15.142:0/2374658250 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a48068430 con 0x7f7a4815be30 2024-04-26T23:36:50.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.951+0000 7f7a4ca00640 1 --2- 172.21.15.142:0/2374658250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009870 tx=0x7f7a4002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8250fcbb1cbf1045 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a47400640 1 -- 172.21.15.142:0/2374658250 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40037070 con 0x7f7a4815be30 2024-04-26T23:36:50.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a47400640 1 -- 172.21.15.142:0/2374658250 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a40032440 con 0x7f7a4815be30 2024-04-26T23:36:50.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a47400640 1 -- 172.21.15.142:0/2374658250 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40032810 con 0x7f7a4815be30 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 -- 172.21.15.142:0/2374658250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 msgr2=0x7f7a480b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 --2- 172.21.15.142:0/2374658250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009870 tx=0x7f7a4002a1c0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 -- 172.21.15.142:0/2374658250 shutdown_connections 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 --2- 172.21.15.142:0/2374658250 >> v2:172.21.15.46:3300/0 conn(0x7f7a48160180 0x7f7a480b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 --2- 172.21.15.142:0/2374658250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a4815be30 0x7f7a480b08d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 -- 172.21.15.142:0/2374658250 >> 172.21.15.142:0/2374658250 conn(0x7f7a480b31a0 msgr2=0x7f7a480b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.952+0000 7f7a4de00640 1 -- 172.21.15.142:0/2374658250 shutdown_connections 2024-04-26T23:36:50.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.953+0000 7f7a4de00640 1 -- 172.21.15.142:0/2374658250 wait complete. 2024-04-26T23:36:50.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.954+0000 7f7a4de00640 1 Processor -- start 2024-04-26T23:36:50.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.954+0000 7f7a4de00640 1 -- start start 2024-04-26T23:36:50.957 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:50.955+0000 7fb5fd012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:36:50.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cea30 0x7f7a4815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a4815be30 0x7f7a48152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48068090 con 0x7f7a480cea30 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480b48b0 con 0x7f7a4815be30 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a48068210 con 0x7f7a48160180 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34828/0 (socket says 172.21.15.142:34828) 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a4d400640 1 -- 172.21.15.142:0/1155970147 learned_addr learned my addr 172.21.15.142:0/1155970147 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.955+0000 7f7a47e00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a4815be30 0x7f7a48152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4ca00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cea30 0x7f7a4815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 -- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a4815be30 msgr2=0x7f7a48152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a4815be30 0x7f7a48152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 -- 172.21.15.142:0/1155970147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cea30 msgr2=0x7f7a4815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:50.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cea30 0x7f7a4815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:50.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 -- 172.21.15.142:0/1155970147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4815dcd0 con 0x7f7a48160180 2024-04-26T23:36:50.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4d400640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800cc80 tx=0x7f7a3800a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800ccb0 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a38003990 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38003d80 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.956+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4815dfb0 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.957+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4815e570 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.957+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a380042b0 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.957+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7a38018c70 con 0x7f7a48160180 2024-04-26T23:36:50.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.958+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a48069ac0 con 0x7f7a48160180 2024-04-26T23:36:50.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:50.962+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7a38014030 con 0x7f7a48160180 2024-04-26T23:36:51.019 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:51.015+0000 7f3115211040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:36:51.023 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:51.020+0000 7fb5fd012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:36:51.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.041+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1", "id": 4} v 0) -- 0x7f7a4806d550 con 0x7f7a48160180 2024-04-26T23:36:51.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.048+0000 7f7a44c00640 1 -- 172.21.15.142:0/1155970147 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7a380045a0 con 0x7f7a48160180 2024-04-26T23:36:51.051 INFO:teuthology.orchestra.run.smithi142.stdout:4 2024-04-26T23:36:51.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.049+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 msgr2=0x7f7a4815ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.049+0000 7f7a4de00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800cc80 tx=0x7f7a3800a5c0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 shutdown_connections 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48160180 0x7f7a4815ed70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a4815be30 0x7f7a48152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 --2- 172.21.15.142:0/1155970147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cea30 0x7f7a4815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 >> 172.21.15.142:0/1155970147 conn(0x7f7a480b31a0 msgr2=0x7f7a480b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 shutdown_connections 2024-04-26T23:36:51.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.050+0000 7f7a4de00640 1 -- 172.21.15.142:0/1155970147 wait complete. 2024-04-26T23:36:51.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 2df71ca1-4fe1-49cb-b231-4998fa699f46 5 2024-04-26T23:36:51.083 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:51.080+0000 7f3115211040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:36:51.090 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:51.087+0000 7fb5fd012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:36:51.217 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:51.213+0000 7f3115211040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:36:51.225 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:51.222+0000 7fb5fd012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:36:51.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.245+0000 7f253f200640 1 Processor -- start 2024-04-26T23:36:51.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.245+0000 7f253f200640 1 -- start start 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f253815f490 0x7f253806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25380d0a80 con 0x7f253806d260 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25380d0d60 con 0x7f25380d4870 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f25380d3060 con 0x7f253815f490 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f253815f490 0x7f253806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f253815f490 0x7f253806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45446/0 (socket says 172.21.15.142:45446) 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253de00640 1 -- 172.21.15.142:0/1987216320 learned_addr learned my addr 172.21.15.142:0/1987216320 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:51.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.247+0000 7f253e800640 1 --1- 172.21.15.142:0/1987216320 >> v1:172.21.15.142:6789/0 conn(0x7f25380d4870 0x7f25380d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55096/0 (socket says 172.21.15.142:55096) 2024-04-26T23:36:51.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.248+0000 7f253de00640 1 -- 172.21.15.142:0/1987216320 >> v1:172.21.15.142:6789/0 conn(0x7f25380d4870 legacy=0x7f25380d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.248+0000 7f253de00640 1 -- 172.21.15.142:0/1987216320 >> v1:172.21.15.46:6789/0 conn(0x7f253806d260 legacy=0x7f25380d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:51.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.248+0000 7f253de00640 1 -- 172.21.15.142:0/1987216320 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25380d3340 con 0x7f253815f490 2024-04-26T23:36:51.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253de00640 1 --2- 172.21.15.142:0/1987216320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f253815f490 0x7f253806cd20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f252c009870 tx=0x7f252c02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54087d36a2b44bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253ca00640 1 -- 172.21.15.142:0/1987216320 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c037070 con 0x7f253815f490 2024-04-26T23:36:51.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253ca00640 1 -- 172.21.15.142:0/1987216320 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c032440 con 0x7f253815f490 2024-04-26T23:36:51.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253ca00640 1 -- 172.21.15.142:0/1987216320 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c032810 con 0x7f253815f490 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253f200640 1 -- 172.21.15.142:0/1987216320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f253815f490 msgr2=0x7f253806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.249+0000 7f253f200640 1 --2- 172.21.15.142:0/1987216320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f253815f490 0x7f253806cd20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f252c009870 tx=0x7f252c02a2a0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.250+0000 7f253f200640 1 -- 172.21.15.142:0/1987216320 shutdown_connections 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.250+0000 7f253f200640 1 --2- 172.21.15.142:0/1987216320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f253815f490 0x7f253806cd20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.250+0000 7f253f200640 1 -- 172.21.15.142:0/1987216320 >> 172.21.15.142:0/1987216320 conn(0x7f25380d4fc0 msgr2=0x7f25380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.250+0000 7f253f200640 1 -- 172.21.15.142:0/1987216320 shutdown_connections 2024-04-26T23:36:51.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.250+0000 7f253f200640 1 -- 172.21.15.142:0/1987216320 wait complete. 2024-04-26T23:36:51.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.251+0000 7f253f200640 1 Processor -- start 2024-04-26T23:36:51.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.251+0000 7f253f200640 1 -- start start 2024-04-26T23:36:51.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253806d260 0x7f25380b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f253815f490 0x7f2538156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25380d0860 con 0x7f253806d260 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25380d06e0 con 0x7f25380d4870 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.252+0000 7f253f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25380d2690 con 0x7f253815f490 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f253815f490 0x7f2538156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45456/0 (socket says 172.21.15.142:45456) 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 -- 172.21.15.142:0/2649173649 learned_addr learned my addr 172.21.15.142:0/2649173649 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253de00640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253806d260 0x7f25380b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 -- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f253815f490 msgr2=0x7f2538156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f253815f490 0x7f2538156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 -- 172.21.15.142:0/2649173649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253806d260 msgr2=0x7f25380b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253806d260 0x7f25380b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 -- 172.21.15.142:0/2649173649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253815fc70 con 0x7f25380d4870 2024-04-26T23:36:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.253+0000 7f253d400640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f252800c430 tx=0x7f252800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528011790 con 0x7f25380d4870 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2528017070 con 0x7f25380d4870 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253815ff50 con 0x7f25380d4870 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528011b70 con 0x7f25380d4870 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2538160510 con 0x7f25380d4870 2024-04-26T23:36:51.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2528011d10 con 0x7f25380d4870 2024-04-26T23:36:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.254+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2528012c20 con 0x7f25380d4870 2024-04-26T23:36:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.255+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538069ac0 con 0x7f25380d4870 2024-04-26T23:36:51.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.257+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f25280126a0 con 0x7f25380d4870 2024-04-26T23:36:51.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.335+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2df71ca1-4fe1-49cb-b231-4998fa699f46", "id": 5} v 0) -- 0x7f25380cb6a0 con 0x7f25380d4870 2024-04-26T23:36:51.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.341+0000 7f2532000640 1 -- 172.21.15.142:0/2649173649 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2df71ca1-4fe1-49cb-b231-4998fa699f46", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2528012880 con 0x7f25380d4870 2024-04-26T23:36:51.344 INFO:teuthology.orchestra.run.smithi142.stdout:5 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 msgr2=0x7f2538155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f252800c430 tx=0x7f252800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 shutdown_connections 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f253815f490 0x7f2538156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25380d4870 0x7f2538155fd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 --2- 172.21.15.142:0/2649173649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253806d260 0x7f25380b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 >> 172.21.15.142:0/2649173649 conn(0x7f25380d4fc0 msgr2=0x7f25380b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 shutdown_connections 2024-04-26T23:36:51.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.343+0000 7f253f200640 1 -- 172.21.15.142:0/2649173649 wait complete. 2024-04-26T23:36:51.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 7fb1ccc0-6622-4d7d-8396-ce094d195e98 6 2024-04-26T23:36:51.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.586+0000 7fbc2f200640 1 Processor -- start 2024-04-26T23:36:51.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.587+0000 7fbc2f200640 1 -- start start 2024-04-26T23:36:51.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc3006d430 0x7fbc300d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbc3006d8c0 0x7fbc300d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc300d0a80 con 0x7fbc3015f490 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbc300d0d60 con 0x7fbc3006d430 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbc300d3060 con 0x7fbc3006d8c0 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbc3006d8c0 0x7fbc300d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2de00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fbc3015f490 0x7fbc3006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55104/0 (socket says 172.21.15.142:55104) 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.588+0000 7fbc2de00640 1 -- 172.21.15.142:0/2616903217 learned_addr learned my addr 172.21.15.142:0/2616903217 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.589+0000 7fbc2d400640 1 --2- 172.21.15.142:0/2616903217 >> v2:172.21.15.46:3300/0 conn(0x7fbc3006d430 0x7fbc300d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.589+0000 7fbc2e800640 1 -- 172.21.15.142:0/2616903217 >> v2:172.21.15.46:3300/0 conn(0x7fbc3006d430 msgr2=0x7fbc300d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.589+0000 7fbc2e800640 1 --2- 172.21.15.142:0/2616903217 >> v2:172.21.15.46:3300/0 conn(0x7fbc3006d430 0x7fbc300d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.589+0000 7fbc2e800640 1 -- 172.21.15.142:0/2616903217 >> v1:172.21.15.142:6789/0 conn(0x7fbc3015f490 legacy=0x7fbc3006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.589+0000 7fbc2e800640 1 -- 172.21.15.142:0/2616903217 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc300d3340 con 0x7fbc3006d8c0 2024-04-26T23:36:51.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.590+0000 7fbc2e800640 1 --2- 172.21.15.142:0/2616903217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d3da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc300bb040 tx=0x7fbc2802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba9bc385a61afdc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.590+0000 7fbc2ca00640 1 -- 172.21.15.142:0/2616903217 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28007740 con 0x7fbc3006d8c0 2024-04-26T23:36:51.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.590+0000 7fbc2ca00640 1 -- 172.21.15.142:0/2616903217 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc280078e0 con 0x7fbc3006d8c0 2024-04-26T23:36:51.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.590+0000 7fbc2ca00640 1 -- 172.21.15.142:0/2616903217 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28007cb0 con 0x7fbc3006d8c0 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 -- 172.21.15.142:0/2616903217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3006d8c0 msgr2=0x7fbc300d3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 --2- 172.21.15.142:0/2616903217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d3da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc300bb040 tx=0x7fbc2802a680 comp rx=0 tx=0).stop 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 -- 172.21.15.142:0/2616903217 shutdown_connections 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 --2- 172.21.15.142:0/2616903217 >> v2:172.21.15.46:3300/0 conn(0x7fbc3006d430 0x7fbc300d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 --2- 172.21.15.142:0/2616903217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d3da0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 -- 172.21.15.142:0/2616903217 >> 172.21.15.142:0/2616903217 conn(0x7fbc300d4fc0 msgr2=0x7fbc300d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 -- 172.21.15.142:0/2616903217 shutdown_connections 2024-04-26T23:36:51.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.591+0000 7fbc2f200640 1 -- 172.21.15.142:0/2616903217 wait complete. 2024-04-26T23:36:51.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.593+0000 7fbc2f200640 1 Processor -- start 2024-04-26T23:36:51.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.593+0000 7fbc2f200640 1 -- start start 2024-04-26T23:36:51.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 0x7fbc300d1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc300d06e0 con 0x7fbc3006d8c0 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc300d2880 con 0x7fbc3015f490 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc300d0860 con 0x7fbc3006d430 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 0x7fbc300d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45486/0 (socket says 172.21.15.142:45486) 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2e800640 1 -- 172.21.15.142:0/4075508810 learned_addr learned my addr 172.21.15.142:0/4075508810 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.594+0000 7fbc2d400640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 -- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 msgr2=0x7fbc301f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 -- 172.21.15.142:0/4075508810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3006d8c0 msgr2=0x7fbc300d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 -- 172.21.15.142:0/4075508810 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc300cc500 con 0x7fbc3006d430 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2e800640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:51.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2de00640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 0x7fbc300d1660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000f60 tx=0x7fbc2000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017070 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc20005ce0 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc300cc7e0 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20007310 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc300ccda0 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.595+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc2001a070 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.596+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fbc200074b0 con 0x7fbc3006d430 2024-04-26T23:36:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.596+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc30069ac0 con 0x7fbc3006d430 2024-04-26T23:36:51.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.600+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbc20021020 con 0x7fbc3006d430 2024-04-26T23:36:51.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.680+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7fb1ccc0-6622-4d7d-8396-ce094d195e98", "id": 6} v 0) -- 0x7fbc300d1040 con 0x7fbc3006d430 2024-04-26T23:36:51.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:51.689+0000 7f3115211040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:36:51.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.691+0000 7fbc26000640 1 -- 172.21.15.142:0/4075508810 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7fb1ccc0-6622-4d7d-8396-ce094d195e98", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fbc2001c440 con 0x7fbc3006d430 2024-04-26T23:36:51.694 INFO:teuthology.orchestra.run.smithi142.stdout:6 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 msgr2=0x7fbc300d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 0x7fbc300d1660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000f60 tx=0x7fbc2000c760 comp rx=0 tx=0).stop 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 shutdown_connections 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3015f490 0x7fbc301f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3006d8c0 0x7fbc300d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 --2- 172.21.15.142:0/4075508810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc3006d430 0x7fbc300d1660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 >> 172.21.15.142:0/4075508810 conn(0x7fbc300d4fc0 msgr2=0x7fbc300b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 shutdown_connections 2024-04-26T23:36:51.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.693+0000 7fbc2f200640 1 -- 172.21.15.142:0/4075508810 wait complete. 2024-04-26T23:36:51.708 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:51.705+0000 7fb5fd012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:36:51.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 51cd5087-296d-45f2-9aa7-832ca1a521ef 7 2024-04-26T23:36:51.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.881+0000 7f305fe00640 1 Processor -- start 2024-04-26T23:36:51.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.882+0000 7f305fe00640 1 -- start start 2024-04-26T23:36:51.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30600d0a80 con 0x7f306006d260 2024-04-26T23:36:51.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30600d0d60 con 0x7f30600d4870 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30600d3060 con 0x7f306015f490 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34836/0 (socket says 172.21.15.142:34836) 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305ea00640 1 -- 172.21.15.142:0/1417918148 learned_addr learned my addr 172.21.15.142:0/1417918148 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.883+0000 7f305f400640 1 --1- 172.21.15.142:0/1417918148 >> v1:172.21.15.142:6789/0 conn(0x7f30600d4870 0x7f30600d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:55108/0 (socket says 172.21.15.142:55108) 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.884+0000 7f305ea00640 1 -- 172.21.15.142:0/1417918148 >> v1:172.21.15.142:6789/0 conn(0x7f30600d4870 legacy=0x7f30600d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.884+0000 7f305ea00640 1 -- 172.21.15.142:0/1417918148 >> v1:172.21.15.46:6789/0 conn(0x7f306006d260 legacy=0x7f30600d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.884+0000 7f305ea00640 1 -- 172.21.15.142:0/1417918148 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600d3340 con 0x7f306015f490 2024-04-26T23:36:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.884+0000 7f305e000640 1 -- 172.21.15.142:0/1417918148 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569587413 0 0) 0x7f30600d0d60 con 0x7f30600d4870 2024-04-26T23:36:51.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.885+0000 7f305ea00640 1 --2- 172.21.15.142:0/1417918148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30500098a0 tx=0x7f305002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5328c4e7e68cfef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.885+0000 7f305e000640 1 -- 172.21.15.142:0/1417918148 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050037070 con 0x7f306015f490 2024-04-26T23:36:51.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305e000640 1 -- 172.21.15.142:0/1417918148 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050032440 con 0x7f306015f490 2024-04-26T23:36:51.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305e000640 1 -- 172.21.15.142:0/1417918148 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050032810 con 0x7f306015f490 2024-04-26T23:36:51.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 -- 172.21.15.142:0/1417918148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 msgr2=0x7f306006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 --2- 172.21.15.142:0/1417918148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30500098a0 tx=0x7f305002a200 comp rx=0 tx=0).stop 2024-04-26T23:36:51.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 -- 172.21.15.142:0/1417918148 shutdown_connections 2024-04-26T23:36:51.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 --2- 172.21.15.142:0/1417918148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f306015f490 0x7f306006cd20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 -- 172.21.15.142:0/1417918148 >> 172.21.15.142:0/1417918148 conn(0x7f30600d4fc0 msgr2=0x7f30600d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.886+0000 7f305fe00640 1 -- 172.21.15.142:0/1417918148 shutdown_connections 2024-04-26T23:36:51.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.887+0000 7f305fe00640 1 -- 172.21.15.142:0/1417918148 wait complete. 2024-04-26T23:36:51.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.888+0000 7f305fe00640 1 Processor -- start 2024-04-26T23:36:51.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.888+0000 7f305fe00640 1 -- start start 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306006d260 0x7f30601611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306015f490 0x7f3060161da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30600d0860 con 0x7f306006d260 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30600d06e0 con 0x7f306015f490 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.889+0000 7f305fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30600d2690 con 0x7f30600d4870 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f305f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306015f490 0x7f3060161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:34852/0 (socket says 172.21.15.142:34852) 2024-04-26T23:36:51.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 -- 172.21.15.142:0/3193845700 learned_addr learned my addr 172.21.15.142:0/3193845700 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f305ea00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306006d260 0x7f30601611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 -- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306015f490 msgr2=0x7f3060161da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306015f490 0x7f3060161da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 -- 172.21.15.142:0/3193845700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306006d260 msgr2=0x7f30601611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306006d260 0x7f30601611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 -- 172.21.15.142:0/3193845700 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600b7e30 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.890+0000 7f3057e00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f304800c460 tx=0x7f304800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048016020 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3048012870 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048012c60 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30600b8110 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30600b86d0 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3048019020 con 0x7f30600d4870 2024-04-26T23:36:51.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.891+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3048012e00 con 0x7f30600d4870 2024-04-26T23:36:51.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.892+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3060069ac0 con 0x7f30600d4870 2024-04-26T23:36:51.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.894+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3048020020 con 0x7f30600d4870 2024-04-26T23:36:51.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.974+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "51cd5087-296d-45f2-9aa7-832ca1a521ef", "id": 7} v 0) -- 0x7f30600b50f0 con 0x7f30600d4870 2024-04-26T23:36:51.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.984+0000 7f3056a00640 1 -- 172.21.15.142:0/3193845700 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51cd5087-296d-45f2-9aa7-832ca1a521ef", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f304801b860 con 0x7f30600d4870 2024-04-26T23:36:51.987 INFO:teuthology.orchestra.run.smithi142.stdout:7 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 msgr2=0x7f30600c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f304800c460 tx=0x7f304800c920 comp rx=0 tx=0).stop 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 shutdown_connections 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306015f490 0x7f3060161da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30600d4870 0x7f30600c8800 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f304800c460 tx=0x7f304800c920 comp rx=0 tx=0).stop 2024-04-26T23:36:51.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 --2- 172.21.15.142:0/3193845700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306006d260 0x7f30601611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:51.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 >> 172.21.15.142:0/3193845700 conn(0x7f30600d4fc0 msgr2=0x7f30600cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:51.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 shutdown_connections 2024-04-26T23:36:51.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:36:51.985+0000 7f305fe00640 1 -- 172.21.15.142:0/3193845700 wait complete. 2024-04-26T23:36:51.999 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:51.995+0000 7f3115211040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:36:52.005 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T23:36:52.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T23:36:52.012 INFO:tasks.ceph.osd.0:Started 2024-04-26T23:36:52.012 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T23:36:52.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T23:36:52.019 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.016+0000 7fb5fd012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:36:52.023 INFO:tasks.ceph.osd.1:Started 2024-04-26T23:36:52.023 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T23:36:52.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T23:36:52.028 INFO:tasks.ceph.osd.2:Started 2024-04-26T23:36:52.028 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T23:36:52.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T23:36:52.033 INFO:tasks.ceph.osd.3:Started 2024-04-26T23:36:52.033 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T23:36:52.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T23:36:52.041 INFO:tasks.ceph.osd.4:Started 2024-04-26T23:36:52.041 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T23:36:52.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T23:36:52.050 INFO:tasks.ceph.osd.5:Started 2024-04-26T23:36:52.050 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T23:36:52.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T23:36:52.055 INFO:tasks.ceph.osd.6:Started 2024-04-26T23:36:52.055 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T23:36:52.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T23:36:52.060 INFO:tasks.ceph.osd.7:Started 2024-04-26T23:36:52.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:36:52.074 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:52.070+0000 7f3115211040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:36:52.088 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.085+0000 7fb5fd012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:36:52.259 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.256+0000 7fb5fd012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:36:52.260 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:52.257+0000 7f3115211040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:36:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.267+0000 7fe80cc00640 1 Processor -- start 2024-04-26T23:36:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.268+0000 7fe80cc00640 1 -- start start 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8080b5140 0x7fe8080d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8080d5410 0x7fe808162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe80815f890 con 0x7fe8080b5140 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8080cfd60 con 0x7fe8080d5410 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe80cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8080d2910 con 0x7fe80815f4b0 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8080b5140 0x7fe8080d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:42472/0 (socket says 172.21.15.46:42472) 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 -- 172.21.15.46:0/4221303553 learned_addr learned my addr 172.21.15.46:0/4221303553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe806800640 1 --2- 172.21.15.46:0/4221303553 >> v2:172.21.15.142:3300/0 conn(0x7fe8080d5410 0x7fe808162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 -- 172.21.15.46:0/4221303553 >> v2:172.21.15.142:3300/0 conn(0x7fe8080d5410 msgr2=0x7fe808162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 --2- 172.21.15.46:0/4221303553 >> v2:172.21.15.142:3300/0 conn(0x7fe8080d5410 0x7fe808162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 -- 172.21.15.46:0/4221303553 >> v2:172.21.15.46:3300/0 conn(0x7fe8080b5140 msgr2=0x7fe8080d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 --2- 172.21.15.46:0/4221303553 >> v2:172.21.15.46:3300/0 conn(0x7fe8080b5140 0x7fe8080d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.269+0000 7fe805e00640 1 -- 172.21.15.46:0/4221303553 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8080d58a0 con 0x7fe80815f4b0 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.270+0000 7fe805e00640 1 --2- 172.21.15.46:0/4221303553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b7c30 tx=0x7fe7f802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86843f508edc459 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe804a00640 1 -- 172.21.15.46:0/4221303553 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8037070 con 0x7fe80815f4b0 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe804a00640 1 -- 172.21.15.46:0/4221303553 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f8032440 con 0x7fe80815f4b0 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe804a00640 1 -- 172.21.15.46:0/4221303553 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8032810 con 0x7fe80815f4b0 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 -- 172.21.15.46:0/4221303553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 msgr2=0x7fe8080d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 --2- 172.21.15.46:0/4221303553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b7c30 tx=0x7fe7f802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 -- 172.21.15.46:0/4221303553 shutdown_connections 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 --2- 172.21.15.46:0/4221303553 >> v2:172.21.15.142:3300/0 conn(0x7fe8080d5410 0x7fe808162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 --2- 172.21.15.46:0/4221303553 >> v2:172.21.15.46:3300/0 conn(0x7fe8080b5140 0x7fe8080d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 --2- 172.21.15.46:0/4221303553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe80815f4b0 0x7fe8080d23d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.271+0000 7fe80cc00640 1 -- 172.21.15.46:0/4221303553 >> 172.21.15.46:0/4221303553 conn(0x7fe80808eb50 msgr2=0x7fe80809e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.272+0000 7fe80cc00640 1 -- 172.21.15.46:0/4221303553 shutdown_connections 2024-04-26T23:36:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.272+0000 7fe80cc00640 1 -- 172.21.15.46:0/4221303553 wait complete. 2024-04-26T23:36:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.273+0000 7fe80cc00640 1 Processor -- start 2024-04-26T23:36:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.273+0000 7fe80cc00640 1 -- start start 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe80815f4b0 0x7fe8081f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 0x7fe808162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8080b5b90 con 0x7fe8080d5410 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8080b58b0 con 0x7fe80815f4b0 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe80cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8080b5e70 con 0x7fe808161de0 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.274+0000 7fe805e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52624/0 (socket says 172.21.15.46:52624) 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 -- 172.21.15.46:0/3179593919 learned_addr learned my addr 172.21.15.46:0/3179593919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805400640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe80815f4b0 0x7fe8081f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe806800640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 0x7fe808162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 -- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 msgr2=0x7fe808162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 0x7fe808162270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 -- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe80815f4b0 msgr2=0x7fe8081f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe80815f4b0 0x7fe8081f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 -- 172.21.15.46:0/3179593919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8080ba180 con 0x7fe8080d5410 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe806800640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 0x7fe808162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:36:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe805e00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe7f802a210 tx=0x7fe7f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.275+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8041020 con 0x7fe8080d5410 2024-04-26T23:36:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f803d620 con 0x7fe8080d5410 2024-04-26T23:36:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8037040 con 0x7fe8080d5410 2024-04-26T23:36:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80815fc60 con 0x7fe8080d5410 2024-04-26T23:36:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe808160220 con 0x7fe8080d5410 2024-04-26T23:36:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7f8044020 con 0x7fe8080d5410 2024-04-26T23:36:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.276+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fe7f8046b80 con 0x7fe8080d5410 2024-04-26T23:36:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.277+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe808069530 con 0x7fe8080d5410 2024-04-26T23:36:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.279+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe7f8046e20 con 0x7fe8080d5410 2024-04-26T23:36:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.345+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8080b4bd0 con 0x7fe8080d5410 2024-04-26T23:36:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.345+0000 7fe7fe000640 1 -- 172.21.15.46:0/3179593919 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fe7f804b030 con 0x7fe8080d5410 2024-04-26T23:36:52.350 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:36:52.350 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:36:51.976402+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.346+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 msgr2=0x7fe8081f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.346+0000 7fe80cc00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe7f802a210 tx=0x7fe7f80046c0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 shutdown_connections 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe808161de0 0x7fe808162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe80815f4b0 0x7fe8081f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 --2- 172.21.15.46:0/3179593919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8080d5410 0x7fe8081f5ca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 >> 172.21.15.46:0/3179593919 conn(0x7fe80808eb50 msgr2=0x7fe8080cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 shutdown_connections 2024-04-26T23:36:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:36:52.347+0000 7fe80cc00640 1 -- 172.21.15.46:0/3179593919 wait complete. 2024-04-26T23:36:52.366 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T23:36:52.366 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T23:36:52.585 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.582+0000 7fb5fd012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:36:52.617 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:52.613+0000 7f3115211040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:36:52.640 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.637+0000 7fb5fd012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:36:52.677 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:52.673+0000 7f3115211040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:36:52.715 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-26T23:36:52.711+0000 7f3bb100c640 -1 Falling back to public interface 2024-04-26T23:36:52.723 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-04-26T23:36:52.720+0000 7f472c7a7640 -1 Falling back to public interface 2024-04-26T23:36:52.724 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-04-26T23:36:52.721+0000 7fa0cdcb3640 -1 Falling back to public interface 2024-04-26T23:36:52.732 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-26T23:36:52.728+0000 7f7f94a16640 -1 Falling back to public interface 2024-04-26T23:36:52.740 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-26T23:36:52.736+0000 7f550281c640 -1 Falling back to public interface 2024-04-26T23:36:52.749 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-26T23:36:52.746+0000 7ff12ee0d640 -1 Falling back to public interface 2024-04-26T23:36:52.753 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-26T23:36:52.749+0000 7f1cc640c640 -1 Falling back to public interface 2024-04-26T23:36:52.758 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-04-26T23:36:52.755+0000 7fb6adeaf640 -1 Falling back to public interface 2024-04-26T23:36:52.830 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.827+0000 7fb5fd012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:36:52.872 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:52.868+0000 7f3115211040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:36:52.994 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:52.991+0000 7fb5fd012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:36:53.047 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.043+0000 7f3115211040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:36:53.055 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:53.052+0000 7fb5fd012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:36:53.109 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.105+0000 7f3115211040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:36:53.137 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:53.134+0000 7fb5fd012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:36:53.172 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T23:36:53.188 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:53.186+0000 7fb5fd012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:36:53.193 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.190+0000 7f3115211040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:36:53.250 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.247+0000 7f3115211040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:36:53.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.339+0000 7f3898200640 1 Processor -- start 2024-04-26T23:36:53.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.339+0000 7f3898200640 1 -- start start 2024-04-26T23:36:53.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3898200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f38900b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:53.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3898200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3890160820 con 0x7f38900730b0 2024-04-26T23:36:53.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3898200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389015d270 con 0x7f38900b10a0 2024-04-26T23:36:53.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3898200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f389015f800 con 0x7f38900b0cc0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3896200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38900730b0 0x7f38900734e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50938/0 (socket says 172.21.15.46:50938) 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3896200640 1 -- 172.21.15.46:0/4235356933 learned_addr learned my addr 172.21.15.46:0/4235356933 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f3895800640 1 --2- 172.21.15.46:0/4235356933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f38900b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311968703 0 0) 0x7f3890160820 con 0x7f38900730b0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.342+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3878003520 con 0x7f38900730b0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.343+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491093403 0 0) 0x7f389015d270 con 0x7f38900b10a0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.343+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3890160820 con 0x7f38900b10a0 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.343+0000 7f3895800640 1 -- 172.21.15.46:0/4235356933 >> v1:172.21.15.142:6789/0 conn(0x7f38900b10a0 legacy=0x7f38900b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.343+0000 7f3895800640 1 -- 172.21.15.46:0/4235356933 >> v1:172.21.15.46:6789/0 conn(0x7f38900730b0 legacy=0x7f38900734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.346 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.343+0000 7f3895800640 1 -- 172.21.15.46:0/4235356933 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38900b54a0 con 0x7f38900b0cc0 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.344+0000 7f3895800640 1 --2- 172.21.15.46:0/4235356933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f38900b4f60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38840097d0 tx=0x7f388402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9180f6bc8d8c28c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.344+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884037070 con 0x7f38900b0cc0 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.344+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884032440 con 0x7f38900b0cc0 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.344+0000 7f388fe00640 1 -- 172.21.15.46:0/4235356933 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884032810 con 0x7f38900b0cc0 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 -- 172.21.15.46:0/4235356933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 msgr2=0x7f38900b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 --2- 172.21.15.46:0/4235356933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f38900b4f60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38840097d0 tx=0x7f388402a210 comp rx=0 tx=0).stop 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 -- 172.21.15.46:0/4235356933 shutdown_connections 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 --2- 172.21.15.46:0/4235356933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f38900b4f60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 -- 172.21.15.46:0/4235356933 >> 172.21.15.46:0/4235356933 conn(0x7f38900b3ab0 msgr2=0x7f38900b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:53.348 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 -- 172.21.15.46:0/4235356933 shutdown_connections 2024-04-26T23:36:53.349 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.345+0000 7f3898200640 1 -- 172.21.15.46:0/4235356933 wait complete. 2024-04-26T23:36:53.350 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.348+0000 7f3898200640 1 Processor -- start 2024-04-26T23:36:53.350 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.348+0000 7f3898200640 1 -- start start 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 0x7f38901609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f3890162070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3890160020 con 0x7f38900b10a0 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f389015fd40 con 0x7f38900730b0 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.349+0000 7f3898200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3890160300 con 0x7f38900b0cc0 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:53.354 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3895800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 0x7f38901609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52634/0 (socket says 172.21.15.46:52634) 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 -- 172.21.15.46:0/4038874734 learned_addr learned my addr 172.21.15.46:0/4038874734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3894e00640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f3890162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 -- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 msgr2=0x7f3890162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f3890162070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 -- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 msgr2=0x7f38901609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 0x7f38901609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 -- 172.21.15.46:0/4038874734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3890150550 con 0x7f38900b10a0 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3895800640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 0x7f38901609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.350+0000 7f3896200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f388800cba0 tx=0x7f388800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888017070 con 0x7f38900b10a0 2024-04-26T23:36:53.355 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388800ace0 con 0x7f38900b10a0 2024-04-26T23:36:53.356 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888012620 con 0x7f38900b10a0 2024-04-26T23:36:53.356 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3890160fd0 con 0x7f38900b10a0 2024-04-26T23:36:53.356 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3890161590 con 0x7f38900b10a0 2024-04-26T23:36:53.356 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f388801a070 con 0x7f38900b10a0 2024-04-26T23:36:53.356 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.351+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3888007070 con 0x7f38900b10a0 2024-04-26T23:36:53.357 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.352+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38900b1500 con 0x7f38900b10a0 2024-04-26T23:36:53.357 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.355+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f38900b1500 con 0x7f38900b10a0 2024-04-26T23:36:53.423 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.420+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f389006cf70 con 0x7f38900b10a0 2024-04-26T23:36:53.424 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.421+0000 7f388d600640 1 -- 172.21.15.46:0/4038874734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3888021020 con 0x7f38900b10a0 2024-04-26T23:36:53.424 INFO:teuthology.misc.health.smithi046.stdout: 2024-04-26T23:36:53.424 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:36:51.976402+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:36:53.425 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 msgr2=0x7f3890150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:36:53.425 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f388800cba0 tx=0x7f388800a430 comp rx=0 tx=0).stop 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 shutdown_connections 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38900b10a0 0x7f3890150010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38900b0cc0 0x7f3890162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 --2- 172.21.15.46:0/4038874734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38900730b0 0x7f38901609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 >> 172.21.15.46:0/4038874734 conn(0x7f38900b3ab0 msgr2=0x7f38900b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 shutdown_connections 2024-04-26T23:36:53.426 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:36:53.422+0000 7f3898200640 1 -- 172.21.15.46:0/4038874734 wait complete. 2024-04-26T23:36:53.437 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T23:36:53.729 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:53.726+0000 7fb5fd012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:36:53.825 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.822+0000 7f3115211040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:36:53.859 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:53.856+0000 7fb5fd012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:36:53.968 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-26T23:36:53.966+0000 7f3115211040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:36:54.908 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:54.905+0000 7fb59ca00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T23:36:54.908 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-26T23:36:54.905+0000 7fb59ca00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T23:36:55.880 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-26T23:36:55.878+0000 7f1cc640c640 -1 osd.3 0 log_to_monitors true 2024-04-26T23:36:56.360 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-26T23:36:56.357+0000 7f550281c640 -1 osd.1 0 log_to_monitors true 2024-04-26T23:36:56.364 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-26T23:36:56.362+0000 7f7f94a16640 -1 osd.2 0 log_to_monitors true 2024-04-26T23:36:56.377 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-04-26T23:36:56.374+0000 7fa0cdcb3640 -1 osd.5 0 log_to_monitors true 2024-04-26T23:36:56.574 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-26T23:36:56.571+0000 7f3bb100c640 -1 osd.0 0 log_to_monitors true 2024-04-26T23:36:56.662 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-04-26T23:36:56.659+0000 7f472c7a7640 -1 osd.6 0 log_to_monitors true 2024-04-26T23:36:56.666 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-04-26T23:36:56.663+0000 7fb6adeaf640 -1 osd.4 0 log_to_monitors true 2024-04-26T23:36:56.675 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-26T23:36:56.672+0000 7ff12ee0d640 -1 osd.7 0 log_to_monitors true 2024-04-26T23:36:59.626 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-26T23:36:59.624+0000 7f54fd000640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T23:36:59.638 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-26T23:36:59.635+0000 7f1cc1600640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T23:36:59.639 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-26T23:36:59.636+0000 7f54f3000640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:36:59.652 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-26T23:36:59.650+0000 7f1cb6c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:36:59.661 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-26T23:36:59.658+0000 7f3bac200640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T23:36:59.670 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-26T23:36:59.667+0000 7f3ba1800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:36:59.677 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-26T23:36:59.674+0000 7f7f8f400640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T23:36:59.684 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-26T23:36:59.682+0000 7f7f85400640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:37:00.222 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-26T23:37:00.219+0000 7ff12a000640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T23:37:00.244 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:00.245 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-26T23:37:00.241+0000 7ff11f600640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:37:00.312 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-04-26T23:37:00.310+0000 7fb6a9800640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T23:37:00.318 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-04-26T23:37:00.315+0000 7fb69ee00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:37:00.356 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-04-26T23:37:00.353+0000 7fa0c8c00640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T23:37:00.361 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-04-26T23:37:00.358+0000 7f4728000640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T23:37:00.364 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-04-26T23:37:00.361+0000 7fa0bec00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:37:00.368 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-04-26T23:37:00.365+0000 7f471d600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:37:00.437 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.434+0000 7f6a8da00640 1 Processor -- start 2024-04-26T23:37:00.437 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.434+0000 7f6a8da00640 1 -- start start 2024-04-26T23:37:00.438 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a8815f330 0x7f6a880ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.439 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a880af2e0 0x7f6a880cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.439 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a880cb8c0 0x7f6a880ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.439 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6a880af830 con 0x7f6a880cb8c0 2024-04-26T23:37:00.439 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6a880afb10 con 0x7f6a880af2e0 2024-04-26T23:37:00.439 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880afdf0 con 0x7f6a8815f330 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a880cb8c0 0x7f6a880ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a880cb8c0 0x7f6a880ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52670/0 (socket says 172.21.15.46:52670) 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a8d000640 1 -- 172.21.15.46:0/3621619782 learned_addr learned my addr 172.21.15.46:0/3621619782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.436+0000 7f6a87e00640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a8815f330 0x7f6a880ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a87400640 1 --2- 172.21.15.46:0/3621619782 >> v2:172.21.15.142:3300/0 conn(0x7f6a880af2e0 0x7f6a880cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a8d000640 1 -- 172.21.15.46:0/3621619782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a8815f330 msgr2=0x7f6a880ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a8d000640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a8815f330 0x7f6a880ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.440 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a8d000640 1 -- 172.21.15.46:0/3621619782 >> v2:172.21.15.142:3300/0 conn(0x7f6a880af2e0 msgr2=0x7f6a880cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a8d000640 1 --2- 172.21.15.46:0/3621619782 >> v2:172.21.15.142:3300/0 conn(0x7f6a880af2e0 0x7f6a880cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.437+0000 7f6a8d000640 1 -- 172.21.15.46:0/3621619782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880b00d0 con 0x7f6a880cb8c0 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.438+0000 7f6a8d000640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a880cb8c0 0x7f6a880ced90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a880b0460 tx=0x7f6a8002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8202ea3092b0d46c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.438+0000 7f6a86a00640 1 -- 172.21.15.46:0/3621619782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80036070 con 0x7f6a880cb8c0 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.438+0000 7f6a86a00640 1 -- 172.21.15.46:0/3621619782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80031600 con 0x7f6a880cb8c0 2024-04-26T23:37:00.441 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.438+0000 7f6a86a00640 1 -- 172.21.15.46:0/3621619782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800319d0 con 0x7f6a880cb8c0 2024-04-26T23:37:00.442 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 -- 172.21.15.46:0/3621619782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a880cb8c0 msgr2=0x7f6a880ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.442 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a880cb8c0 0x7f6a880ced90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a880b0460 tx=0x7f6a8002a610 comp rx=0 tx=0).stop 2024-04-26T23:37:00.442 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 -- 172.21.15.46:0/3621619782 shutdown_connections 2024-04-26T23:37:00.442 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a880cb8c0 0x7f6a880ced90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.443 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 --2- 172.21.15.46:0/3621619782 >> v2:172.21.15.142:3300/0 conn(0x7f6a880af2e0 0x7f6a880cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.443 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 --2- 172.21.15.46:0/3621619782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a8815f330 0x7f6a880ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.443 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 -- 172.21.15.46:0/3621619782 >> 172.21.15.46:0/3621619782 conn(0x7f6a880d0b20 msgr2=0x7f6a880d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:00.443 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 -- 172.21.15.46:0/3621619782 shutdown_connections 2024-04-26T23:37:00.443 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.439+0000 7f6a8da00640 1 -- 172.21.15.46:0/3621619782 wait complete. 2024-04-26T23:37:00.444 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.441+0000 7f6a8da00640 1 Processor -- start 2024-04-26T23:37:00.444 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.442+0000 7f6a8da00640 1 -- start start 2024-04-26T23:37:00.446 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 0x7f6a880b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.446 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a880cb8c0 0x7f6a8814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880cf450 con 0x7f6a8815f330 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880cf2d0 con 0x7f6a880cb8c0 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880cf5d0 con 0x7f6a880af2e0 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52672/0 (socket says 172.21.15.46:52672) 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a8d000640 1 -- 172.21.15.46:0/2102816056 learned_addr learned my addr 172.21.15.46:0/2102816056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.443+0000 7f6a87e00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 0x7f6a880b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a87400640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a880cb8c0 0x7f6a8814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 -- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 msgr2=0x7f6a880b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 0x7f6a880b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.447 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 -- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a880cb8c0 msgr2=0x7f6a8814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.448 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a880cb8c0 0x7f6a8814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.448 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 -- 172.21.15.46:0/2102816056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8814a9a0 con 0x7f6a8815f330 2024-04-26T23:37:00.448 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a87e00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 0x7f6a880b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:00.448 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8d000640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009f40 tx=0x7f6a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80036050 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80042070 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80009510 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a880b52f0 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.444+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a880b58b0 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.445+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a880506c0 con 0x7f6a8815f330 2024-04-26T23:37:00.449 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.446+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f6a80007bd0 con 0x7f6a8815f330 2024-04-26T23:37:00.450 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.447+0000 7f6a7fe00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 0x7f6a6806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.450 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.447+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f6a800abbf0 con 0x7f6a8815f330 2024-04-26T23:37:00.450 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.448+0000 7f6a87e00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 0x7f6a6806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.452 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.449+0000 7f6a87e00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 0x7f6a6806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006fd0 tx=0x7f6a78008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.455 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.453+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a80045360 con 0x7f6a8815f330 2024-04-26T23:37:00.665 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.662+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a880d0530 con 0x7f6a8815f330 2024-04-26T23:37:00.667 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.664+0000 7f6a7fe00640 1 -- 172.21.15.46:0/2102816056 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10873 (secure 0 0 0) 0x7f6a8007adf0 con 0x7f6a8815f330 2024-04-26T23:37:00.667 INFO:teuthology.misc.health.smithi046.stdout: 2024-04-26T23:37:00.668 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":13,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:00.631953+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 msgr2=0x7f6a6806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 0x7f6a6806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006fd0 tx=0x7f6a78008040 comp rx=0 tx=0).stop 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 msgr2=0x7f6a881f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009f40 tx=0x7f6a80004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 shutdown_connections 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a6806c860 0x7f6a6806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8815f330 0x7f6a881f4ce0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.670 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a880cb8c0 0x7f6a8814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.671 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 --2- 172.21.15.46:0/2102816056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a880af2e0 0x7f6a880b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.671 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 >> 172.21.15.46:0/2102816056 conn(0x7f6a880d0b20 msgr2=0x7f6a880b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:00.671 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 shutdown_connections 2024-04-26T23:37:00.671 INFO:teuthology.misc.health.smithi046.stderr:2024-04-26T23:37:00.667+0000 7f6a8da00640 1 -- 172.21.15.46:0/2102816056 wait complete. 2024-04-26T23:37:00.683 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T23:37:00.683 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T23:37:00.683 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:37:00.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:37:00.689 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:37:00.689 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:37:00.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:37:00.694 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:37:00.694 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:37:00.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:37:00.702 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:37:00.703 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:37:00.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:37:00.710 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:37:00.710 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T23:37:00.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:00.824 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:37:00.825 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:37:00.854 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:37:00.855 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:37:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.878+0000 7fe15d600640 1 Processor -- start 2024-04-26T23:37:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.878+0000 7fe15d600640 1 -- start start 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe15d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 0x7fe15816a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe15d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1580d2f60 con 0x7fe15815f500 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe15d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1580d3240 con 0x7fe1580b3fb0 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe15d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe15815f8e0 con 0x7fe1580d5890 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe15815f500 0x7fe1580d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50956/0 (socket says 172.21.15.46:50956) 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156a00640 1 -- 172.21.15.46:0/927866000 learned_addr learned my addr 172.21.15.46:0/927866000 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe157400640 1 --2- 172.21.15.46:0/927866000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 0x7fe15816a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402025425 0 0) 0x7fe1580d2f60 con 0x7fe15815f500 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe13c003520 con 0x7fe15815f500 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889064218 0 0) 0x7fe1580d3240 con 0x7fe1580b3fb0 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1580d2f60 con 0x7fe1580b3fb0 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe157400640 1 -- 172.21.15.46:0/927866000 >> v1:172.21.15.142:6789/0 conn(0x7fe1580b3fb0 legacy=0x7fe1580d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe157400640 1 -- 172.21.15.46:0/927866000 >> v1:172.21.15.46:6789/0 conn(0x7fe15815f500 legacy=0x7fe1580d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.880+0000 7fe157400640 1 -- 172.21.15.46:0/927866000 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1580d3520 con 0x7fe1580d5890 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.881+0000 7fe157400640 1 --2- 172.21.15.46:0/927866000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 0x7fe15816a290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe150009dc0 tx=0x7fe15002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f880914b1f87735d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150007740 con 0x7fe1580d5890 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1500078e0 con 0x7fe1580d5890 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe156000640 1 -- 172.21.15.46:0/927866000 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150007cb0 con 0x7fe1580d5890 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 -- 172.21.15.46:0/927866000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 msgr2=0x7fe15816a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 --2- 172.21.15.46:0/927866000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 0x7fe15816a290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe150009dc0 tx=0x7fe15002a6d0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 -- 172.21.15.46:0/927866000 shutdown_connections 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 --2- 172.21.15.46:0/927866000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580d5890 0x7fe15816a290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 -- 172.21.15.46:0/927866000 >> 172.21.15.46:0/927866000 conn(0x7fe158074400 msgr2=0x7fe15809e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.882+0000 7fe15d600640 1 -- 172.21.15.46:0/927866000 shutdown_connections 2024-04-26T23:37:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.883+0000 7fe15d600640 1 -- 172.21.15.46:0/927866000 wait complete. 2024-04-26T23:37:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.884+0000 7fe15d600640 1 Processor -- start 2024-04-26T23:37:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.884+0000 7fe15d600640 1 -- start start 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580b3fb0 0x7fe158161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1580d5890 0x7fe1580cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1580d3ae0 con 0x7fe15815f500 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1580b5e80 con 0x7fe1580d5890 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe15d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1580d4000 con 0x7fe1580b3fb0 2024-04-26T23:37:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe157400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe157400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52722/0 (socket says 172.21.15.46:52722) 2024-04-26T23:37:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe157400640 1 -- 172.21.15.46:0/800165413 learned_addr learned my addr 172.21.15.46:0/800165413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe156a00640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580b3fb0 0x7fe158161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.885+0000 7fe14fe00640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1580d5890 0x7fe1580cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 -- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580b3fb0 msgr2=0x7fe158161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580b3fb0 0x7fe158161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 -- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1580d5890 msgr2=0x7fe1580cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1580d5890 0x7fe1580cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 -- 172.21.15.46:0/800165413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1581f5030 con 0x7fe15815f500 2024-04-26T23:37:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe157400640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe15002acc0 tx=0x7fe150004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150040070 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe150031b40 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150009450 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.886+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1580cd8b0 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.887+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1580ce150 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.888+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe158069530 con 0x7fe15815f500 2024-04-26T23:37:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.888+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe150039040 con 0x7fe15815f500 2024-04-26T23:37:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.889+0000 7fe14ea00640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 0x7fe13406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.889+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fe1500acd00 con 0x7fe15815f500 2024-04-26T23:37:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.890+0000 7fe156a00640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 0x7fe13406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.890+0000 7fe156a00640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 0x7fe13406ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f60 tx=0x7fe148007d70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:00.894+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe15007af60 con 0x7fe15815f500 2024-04-26T23:37:01.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.093+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1580b4c50 con 0x7fe15815f500 2024-04-26T23:37:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.095+0000 7fe14ea00640 1 -- 172.21.15.46:0/800165413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10873 (secure 0 0 0) 0x7fe15007f260 con 0x7fe15815f500 2024-04-26T23:37:01.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:00.631953+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.097+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 msgr2=0x7fe13406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.097+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 0x7fe13406ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f60 tx=0x7fe148007d70 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.097+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 msgr2=0x7fe15815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.097+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe15002acc0 tx=0x7fe150004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 shutdown_connections 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe13406c860 0x7fe13406ed20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15815f500 0x7fe15815fa60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1580d5890 0x7fe1580cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 --2- 172.21.15.46:0/800165413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1580b3fb0 0x7fe158161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 >> 172.21.15.46:0/800165413 conn(0x7fe158074400 msgr2=0x7fe1580d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 shutdown_connections 2024-04-26T23:37:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.098+0000 7fe15d600640 1 -- 172.21.15.46:0/800165413 wait complete. 2024-04-26T23:37:01.117 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-26T23:37:01.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.312+0000 7f2034c00640 1 Processor -- start 2024-04-26T23:37:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.313+0000 7f2034c00640 1 -- start start 2024-04-26T23:37:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.314+0000 7f2034c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20301682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.314+0000 7f2034c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2030161940 con 0x7f203006e510 2024-04-26T23:37:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.314+0000 7f2034c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2030071f80 con 0x7f2030160bd0 2024-04-26T23:37:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.314+0000 7f2034c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20300742d0 con 0x7f2030165ee0 2024-04-26T23:37:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f203006e510 0x7f203006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50970/0 (socket says 172.21.15.46:50970) 2024-04-26T23:37:01.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202de00640 1 -- 172.21.15.46:0/1489669066 learned_addr learned my addr 172.21.15.46:0/1489669066 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202e800640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20301682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803009142 0 0) 0x7f2030161940 con 0x7f203006e510 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2014003520 con 0x7f203006e510 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.315+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2831214725 0 0) 0x7f2014003520 con 0x7f203006e510 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 msgr2=0x7f20301682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20301682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 >> v1:172.21.15.142:6789/0 conn(0x7f2030160bd0 legacy=0x7f2030161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030072260 con 0x7f203006e510 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2020001670 con 0x7f203006e510 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 >> v1:172.21.15.46:6789/0 conn(0x7f203006e510 legacy=0x7f203006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20140039e0 0x7f2014003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2014004800 0x7f2014004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20300742d0 con 0x7f2014004800 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f201400baa0 con 0x7f20140039e0 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f201400bca0 con 0x7f2030165ee0 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2014004800 0x7f2014004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202de00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20140039e0 0x7f2014003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202e800640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 -- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 msgr2=0x7f20140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 -- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20140039e0 msgr2=0x7f2014003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20140039e0 0x7f2014003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202d400640 1 -- 172.21.15.46:0/1489669066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2014003520 con 0x7f2014004800 2024-04-26T23:37:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.316+0000 7f202e800640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2030165ee0 0x7f20140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f202d400640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2014004800 0x7f2014004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2018001720 tx=0x7f2014005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd35029751e30b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018001370 con 0x7f2014004800 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180052d0 con 0x7f2014004800 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f202ca00640 1 -- 172.21.15.46:0/1489669066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201802ab90 con 0x7f2014004800 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f2034c00640 1 -- 172.21.15.46:0/1489669066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2014004800 msgr2=0x7f2014004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f2034c00640 1 --2- 172.21.15.46:0/1489669066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2014004800 0x7f2014004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2018001720 tx=0x7f2014005830 comp rx=0 tx=0).stop 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f202e800640 1 -- 172.21.15.46:0/1489669066 reap_dead start 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f2034c00640 1 -- 172.21.15.46:0/1489669066 shutdown_connections 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.318+0000 7f2034c00640 1 -- 172.21.15.46:0/1489669066 >> 172.21.15.46:0/1489669066 conn(0x7f203006c530 msgr2=0x7f203006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.319+0000 7f2034c00640 1 -- 172.21.15.46:0/1489669066 shutdown_connections 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.319+0000 7f2034c00640 1 -- 172.21.15.46:0/1489669066 wait complete. 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.320+0000 7f2034c00640 1 Processor -- start 2024-04-26T23:37:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.321+0000 7f2034c00640 1 -- start start 2024-04-26T23:37:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 0x7f203015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20300b3f80 0x7f203013a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2030072d40 con 0x7f20300b4400 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2030072820 con 0x7f2014004800 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f2034c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2030072ec0 con 0x7f20300b3f80 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52738/0 (socket says 172.21.15.46:52738) 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 -- 172.21.15.46:0/525756363 learned_addr learned my addr 172.21.15.46:0/525756363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202de00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 0x7f203015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202d400640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20300b3f80 0x7f203013a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 -- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20300b3f80 msgr2=0x7f203013a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20300b3f80 0x7f203013a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 -- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 msgr2=0x7f203015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 0x7f203015fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202e800640 1 -- 172.21.15.46:0/525756363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203013c500 con 0x7f20300b4400 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.322+0000 7f202de00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 0x7f203015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f202e800640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2030161c40 tx=0x7f202800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801c070 con 0x7f20300b4400 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2028017400 con 0x7f20300b4400 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280177d0 con 0x7f20300b4400 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203013c7e0 con 0x7f20300b4400 2024-04-26T23:37:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.323+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203013cda0 con 0x7f20300b4400 2024-04-26T23:37:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.324+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2030069530 con 0x7f20300b4400 2024-04-26T23:37:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.325+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f202801f070 con 0x7f20300b4400 2024-04-26T23:37:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.326+0000 7f2026000640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 0x7f200c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.327+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f202808b700 con 0x7f20300b4400 2024-04-26T23:37:01.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.327+0000 7f202de00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 0x7f200c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:01.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.328+0000 7f202de00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 0x7f200c06ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20200057f0 tx=0x7f2020000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:01.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.332+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2028012070 con 0x7f20300b4400 2024-04-26T23:37:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.580+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203006cb50 con 0x7f20300b4400 2024-04-26T23:37:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.581+0000 7f2026000640 1 -- 172.21.15.46:0/525756363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f202805a0e0 con 0x7f20300b4400 2024-04-26T23:37:01.584 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:01.584 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T23:37:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 msgr2=0x7f200c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 0x7f200c06ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20200057f0 tx=0x7f2020000ed0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 msgr2=0x7f2030144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2030161c40 tx=0x7f202800f4a0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 shutdown_connections 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f200c06c860 0x7f200c06ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20300b4400 0x7f2030144190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20300b3f80 0x7f203013a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 --2- 172.21.15.46:0/525756363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2014004800 0x7f203015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 >> 172.21.15.46:0/525756363 conn(0x7f203006c530 msgr2=0x7f20300ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 shutdown_connections 2024-04-26T23:37:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:01.584+0000 7f2034c00640 1 -- 172.21.15.46:0/525756363 wait complete. 2024-04-26T23:37:03.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.825+0000 7f3b09000640 1 Processor -- start 2024-04-26T23:37:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.825+0000 7f3b09000640 1 -- start start 2024-04-26T23:37:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 0x7f3b040c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b04162e20 0x7f3b0406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b040cdaa0 0x7f3b040cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0406ce10 con 0x7f3b040cdaa0 2024-04-26T23:37:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3b041633c0 con 0x7f3b04162e20 2024-04-26T23:37:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.827+0000 7f3b09000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b041636a0 con 0x7f3b0415f1f0 2024-04-26T23:37:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b040cdaa0 0x7f3b040cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b040cdaa0 0x7f3b040cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55796/0 (socket says 172.21.15.46:55796) 2024-04-26T23:37:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 -- 172.21.15.46:0/300228676 learned_addr learned my addr 172.21.15.46:0/300228676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b02a00640 1 --2- 172.21.15.46:0/300228676 >> v2:172.21.15.142:3300/0 conn(0x7f3b04162e20 0x7f3b0406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03400640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 0x7f3b040c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 -- 172.21.15.46:0/300228676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 msgr2=0x7f3b040c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 0x7f3b040c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 -- 172.21.15.46:0/300228676 >> v2:172.21.15.142:3300/0 conn(0x7f3b04162e20 msgr2=0x7f3b0406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 --2- 172.21.15.46:0/300228676 >> v2:172.21.15.142:3300/0 conn(0x7f3b04162e20 0x7f3b0406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03e00640 1 -- 172.21.15.46:0/300228676 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b04163980 con 0x7f3b040cdaa0 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.828+0000 7f3b03400640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 0x7f3b040c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.829+0000 7f3b03e00640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040cdf30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b0406d430 tx=0x7f3afc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2bab17e72d6e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b02000640 1 -- 172.21.15.46:0/300228676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc036070 con 0x7f3b040cdaa0 2024-04-26T23:37:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b02000640 1 -- 172.21.15.46:0/300228676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc031600 con 0x7f3b040cdaa0 2024-04-26T23:37:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b02000640 1 -- 172.21.15.46:0/300228676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0319d0 con 0x7f3b040cdaa0 2024-04-26T23:37:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 -- 172.21.15.46:0/300228676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 msgr2=0x7f3b040cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040cdf30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b0406d430 tx=0x7f3afc02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 -- 172.21.15.46:0/300228676 shutdown_connections 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040cdf30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 --2- 172.21.15.46:0/300228676 >> v2:172.21.15.142:3300/0 conn(0x7f3b04162e20 0x7f3b0406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 --2- 172.21.15.46:0/300228676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b0415f1f0 0x7f3b040c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.830+0000 7f3b09000640 1 -- 172.21.15.46:0/300228676 >> 172.21.15.46:0/300228676 conn(0x7f3b040d1c70 msgr2=0x7f3b040d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.831+0000 7f3b09000640 1 -- 172.21.15.46:0/300228676 shutdown_connections 2024-04-26T23:37:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.831+0000 7f3b09000640 1 -- 172.21.15.46:0/300228676 wait complete. 2024-04-26T23:37:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.832+0000 7f3b09000640 1 Processor -- start 2024-04-26T23:37:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.832+0000 7f3b09000640 1 -- start start 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.833+0000 7f3b09000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.833+0000 7f3b09000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b0415f1f0 0x7f3b040b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.833+0000 7f3b09000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b04162e20 0x7f3b04153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b09000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0415f750 con 0x7f3b040cdaa0 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b09000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0415f5d0 con 0x7f3b0415f1f0 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b09000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b040cc910 con 0x7f3b04162e20 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55808/0 (socket says 172.21.15.46:55808) 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 -- 172.21.15.46:0/1895570350 learned_addr learned my addr 172.21.15.46:0/1895570350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 -- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b04162e20 msgr2=0x7f3b04153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b04162e20 0x7f3b04153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 -- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b0415f1f0 msgr2=0x7f3b040b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b0415f1f0 0x7f3b040b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.834+0000 7f3b03400640 1 -- 172.21.15.46:0/1895570350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b040b03a0 con 0x7f3b040cdaa0 2024-04-26T23:37:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3b03400640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c7f0 tx=0x7f3af400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4015070 con 0x7f3b040cdaa0 2024-04-26T23:37:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af4007a30 con 0x7f3b040cdaa0 2024-04-26T23:37:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4007e00 con 0x7f3b040cdaa0 2024-04-26T23:37:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b040b0680 con 0x7f3b040cdaa0 2024-04-26T23:37:03.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.835+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0415aeb0 con 0x7f3b040cdaa0 2024-04-26T23:37:03.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.836+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b04069ac0 con 0x7f3b040cdaa0 2024-04-26T23:37:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.838+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f3af4018070 con 0x7f3b040cdaa0 2024-04-26T23:37:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.839+0000 7f3afb400640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 0x7f3ae406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.839+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3af408b740 con 0x7f3b040cdaa0 2024-04-26T23:37:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.839+0000 7f3b02a00640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 0x7f3ae406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:03.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.841+0000 7f3b02a00640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 0x7f3ae406ed20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3af0006fd0 tx=0x7f3af0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:03.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:03.845+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3af4059f10 con 0x7f3b040cdaa0 2024-04-26T23:37:04.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.106+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b040d2a10 con 0x7f3b040cdaa0 2024-04-26T23:37:04.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.107+0000 7f3afb400640 1 -- 172.21.15.46:0/1895570350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f3af4059810 con 0x7f3b040cdaa0 2024-04-26T23:37:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:04.111 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T23:37:04.111 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-04-26T23:37:04.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 msgr2=0x7f3ae406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 0x7f3ae406ed20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3af0006fd0 tx=0x7f3af0008040 comp rx=0 tx=0).stop 2024-04-26T23:37:04.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 msgr2=0x7f3b040afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c7f0 tx=0x7f3af400ccb0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 shutdown_connections 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ae406c860 0x7f3ae406ed20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b04162e20 0x7f3b04153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b0415f1f0 0x7f3b040b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 --2- 172.21.15.46:0/1895570350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b040cdaa0 0x7f3b040afca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.110+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 >> 172.21.15.46:0/1895570350 conn(0x7f3b040d1c70 msgr2=0x7f3b040b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.111+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 shutdown_connections 2024-04-26T23:37:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.111+0000 7f3b09000640 1 -- 172.21.15.46:0/1895570350 wait complete. 2024-04-26T23:37:04.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.311+0000 7f9b73000640 1 Processor -- start 2024-04-26T23:37:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.312+0000 7f9b73000640 1 -- start start 2024-04-26T23:37:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.313+0000 7f9b73000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b6c15f150 0x7f9b6c071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.313+0000 7f9b73000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b6c06ca60 0x7f9b6c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.313+0000 7f9b73000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b6c1632a0 con 0x7f9b6c162d50 2024-04-26T23:37:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.313+0000 7f9b73000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6c0cf970 con 0x7f9b6c06ca60 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.313+0000 7f9b73000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6c163580 con 0x7f9b6c15f150 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b6b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9b6c162d50 0x7f9b6c06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59190/0 (socket says 172.21.15.46:59190) 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b6b400640 1 -- 172.21.15.46:0/3810774353 learned_addr learned my addr 172.21.15.46:0/3810774353 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b71000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b6c06ca60 0x7f9b6c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b6be00640 1 --2- 172.21.15.46:0/3810774353 >> v2:172.21.15.142:3300/0 conn(0x7f9b6c15f150 0x7f9b6c071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b6aa00640 1 -- 172.21.15.46:0/3810774353 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163777243 0 0) 0x7f9b6c1632a0 con 0x7f9b6c162d50 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b6aa00640 1 -- 172.21.15.46:0/3810774353 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b50003520 con 0x7f9b6c162d50 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b71000640 1 -- 172.21.15.46:0/3810774353 >> v2:172.21.15.142:3300/0 conn(0x7f9b6c15f150 msgr2=0x7f9b6c071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b71000640 1 --2- 172.21.15.46:0/3810774353 >> v2:172.21.15.142:3300/0 conn(0x7f9b6c15f150 0x7f9b6c071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b71000640 1 -- 172.21.15.46:0/3810774353 >> v1:172.21.15.46:6789/0 conn(0x7f9b6c162d50 legacy=0x7f9b6c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.314+0000 7f9b71000640 1 -- 172.21.15.46:0/3810774353 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c163860 con 0x7f9b6c06ca60 2024-04-26T23:37:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.315+0000 7f9b71000640 1 --2- 172.21.15.46:0/3810774353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c0cfe00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c06d430 tx=0x7f9b6402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a20b18a2890f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b6aa00640 1 -- 172.21.15.46:0/3810774353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64007740 con 0x7f9b6c06ca60 2024-04-26T23:37:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b6aa00640 1 -- 172.21.15.46:0/3810774353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b640078e0 con 0x7f9b6c06ca60 2024-04-26T23:37:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b6aa00640 1 -- 172.21.15.46:0/3810774353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64007cb0 con 0x7f9b6c06ca60 2024-04-26T23:37:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 -- 172.21.15.46:0/3810774353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 msgr2=0x7f9b6c0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 --2- 172.21.15.46:0/3810774353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c0cfe00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c06d430 tx=0x7f9b6402a5f0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 -- 172.21.15.46:0/3810774353 shutdown_connections 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 --2- 172.21.15.46:0/3810774353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c0cfe00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 --2- 172.21.15.46:0/3810774353 >> v2:172.21.15.142:3300/0 conn(0x7f9b6c15f150 0x7f9b6c071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.316+0000 7f9b73000640 1 -- 172.21.15.46:0/3810774353 >> 172.21.15.46:0/3810774353 conn(0x7f9b6c06da90 msgr2=0x7f9b6c0d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.317+0000 7f9b73000640 1 -- 172.21.15.46:0/3810774353 shutdown_connections 2024-04-26T23:37:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.317+0000 7f9b73000640 1 -- 172.21.15.46:0/3810774353 wait complete. 2024-04-26T23:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.318+0000 7f9b73000640 1 Processor -- start 2024-04-26T23:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.319+0000 7f9b73000640 1 -- start start 2024-04-26T23:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.319+0000 7f9b73000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.319+0000 7f9b73000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 0x7f9b6c0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.319+0000 7f9b73000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c162d50 0x7f9b6c072f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b73000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c15f530 con 0x7f9b6c06ca60 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b73000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c0719d0 con 0x7f9b6c162d50 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b73000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c0d0340 con 0x7f9b6c15f150 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55838/0 (socket says 172.21.15.46:55838) 2024-04-26T23:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 -- 172.21.15.46:0/1549300226 learned_addr learned my addr 172.21.15.46:0/1549300226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6b400640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 0x7f9b6c0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 -- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 msgr2=0x7f9b6c0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b71000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c162d50 0x7f9b6c072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 0x7f9b6c0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 -- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c162d50 msgr2=0x7f9b6c072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c162d50 0x7f9b6c072f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6be00640 1 -- 172.21.15.46:0/1549300226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c1f87e0 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.320+0000 7f9b6b400640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 0x7f9b6c0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b6be00640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00e9b0 tx=0x7f9b5c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c017070 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b5c00c9a0 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00cd90 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c1f8ac0 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.321+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c1f9080 con 0x7f9b6c06ca60 2024-04-26T23:37:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.322+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6c069530 con 0x7f9b6c06ca60 2024-04-26T23:37:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.323+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9b5c01a070 con 0x7f9b6c06ca60 2024-04-26T23:37:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.324+0000 7f9b63e00640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 0x7f9b4c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.324+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9b5c08bb00 con 0x7f9b6c06ca60 2024-04-26T23:37:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.325+0000 7f9b6b400640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 0x7f9b4c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.326+0000 7f9b6b400640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 0x7f9b4c06ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0c7740 tx=0x7f9b58009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.329+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b5c05a350 con 0x7f9b6c06ca60 2024-04-26T23:37:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.548+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b6c0cf450 con 0x7f9b6c06ca60 2024-04-26T23:37:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.550+0000 7f9b63e00640 1 -- 172.21.15.46:0/1549300226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12861 (secure 0 0 0) 0x7f9b5c059c50 con 0x7f9b6c06ca60 2024-04-26T23:37:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:03.643267+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.552+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 msgr2=0x7f9b4c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.552+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 0x7f9b4c06ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0c7740 tx=0x7f9b58009040 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.552+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 msgr2=0x7f9b6c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.552+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00e9b0 tx=0x7f9b5c00ee70 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 shutdown_connections 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b4c06c860 0x7f9b4c06ed20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c162d50 0x7f9b6c072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c15f150 0x7f9b6c0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 --2- 172.21.15.46:0/1549300226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b6c06ca60 0x7f9b6c15ae80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 >> 172.21.15.46:0/1549300226 conn(0x7f9b6c06da90 msgr2=0x7f9b6c0d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 shutdown_connections 2024-04-26T23:37:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.553+0000 7f9b73000640 1 -- 172.21.15.46:0/1549300226 wait complete. 2024-04-26T23:37:04.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:37:04.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.781+0000 7fa0eea00640 1 Processor -- start 2024-04-26T23:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.781+0000 7fa0eea00640 1 -- start start 2024-04-26T23:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.782+0000 7fa0eea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0e8162920 0x7fa0e80d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.782+0000 7fa0eea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0e80b3d60 0x7fa0e80d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.782+0000 7fa0eea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e80d3d90 con 0x7fa0e80b1cd0 2024-04-26T23:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.782+0000 7fa0eea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e80d2980 con 0x7fa0e8162920 2024-04-26T23:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.782+0000 7fa0eea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e80d24f0 con 0x7fa0e80b3d60 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0e8162920 0x7fa0e80d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0e8162920 0x7fa0e80d3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55854/0 (socket says 172.21.15.46:55854) 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 -- 172.21.15.46:0/3112112559 learned_addr learned my addr 172.21.15.46:0/3112112559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ecc00640 1 --2- 172.21.15.46:0/3112112559 >> v2:172.21.15.142:3300/0 conn(0x7fa0e80b3d60 0x7fa0e80d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 -- 172.21.15.46:0/3112112559 >> v2:172.21.15.142:3300/0 conn(0x7fa0e80b3d60 msgr2=0x7fa0e80d3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3112112559 >> v2:172.21.15.142:3300/0 conn(0x7fa0e80b3d60 0x7fa0e80d3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 -- 172.21.15.46:0/3112112559 >> v1:172.21.15.142:6789/0 conn(0x7fa0e80b1cd0 legacy=0x7fa0e80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.783+0000 7fa0ed600640 1 -- 172.21.15.46:0/3112112559 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e8162d00 con 0x7fa0e8162920 2024-04-26T23:37:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.784+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3112112559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80d3310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80b47d0 tx=0x7fa0dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d52f0f40b483aea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0e7e00640 1 -- 172.21.15.46:0/3112112559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc036070 con 0x7fa0e8162920 2024-04-26T23:37:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0e7e00640 1 -- 172.21.15.46:0/3112112559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0dc031600 con 0x7fa0e8162920 2024-04-26T23:37:04.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0e7e00640 1 -- 172.21.15.46:0/3112112559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc0319d0 con 0x7fa0e8162920 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 -- 172.21.15.46:0/3112112559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 msgr2=0x7fa0e80d3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3112112559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80d3310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80b47d0 tx=0x7fa0dc02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 -- 172.21.15.46:0/3112112559 shutdown_connections 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3112112559 >> v2:172.21.15.142:3300/0 conn(0x7fa0e80b3d60 0x7fa0e80d3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3112112559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80d3310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.785+0000 7fa0eea00640 1 -- 172.21.15.46:0/3112112559 >> 172.21.15.46:0/3112112559 conn(0x7fa0e80d5000 msgr2=0x7fa0e80d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.786+0000 7fa0eea00640 1 -- 172.21.15.46:0/3112112559 shutdown_connections 2024-04-26T23:37:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.786+0000 7fa0eea00640 1 -- 172.21.15.46:0/3112112559 wait complete. 2024-04-26T23:37:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.787+0000 7fa0eea00640 1 Processor -- start 2024-04-26T23:37:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.787+0000 7fa0eea00640 1 -- start start 2024-04-26T23:37:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 0x7fa0e815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0e80b3d60 0x7fa0e80ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e80d2180 con 0x7fa0e8162920 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e80d1ea0 con 0x7fa0e80b3d60 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.788+0000 7fa0eea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e80b4340 con 0x7fa0e80b1cd0 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55868/0 (socket says 172.21.15.46:55868) 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 -- 172.21.15.46:0/3304431466 learned_addr learned my addr 172.21.15.46:0/3304431466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ecc00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0e80b3d60 0x7fa0e80ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 0x7fa0e815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 -- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 msgr2=0x7fa0e815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 0x7fa0e815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 -- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0e80b3d60 msgr2=0x7fa0e80ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0e80b3d60 0x7fa0e80ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 -- 172.21.15.46:0/3304431466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e815e600 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 0x7fa0e815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.789+0000 7fa0ee000640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c390 tx=0x7fa0e000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.790+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0017070 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.790+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e00052d0 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.790+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0003810 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.790+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e80cc340 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.790+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e80ce960 con 0x7fa0e8162920 2024-04-26T23:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.791+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e8069ac0 con 0x7fa0e8162920 2024-04-26T23:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.792+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fa0e001a070 con 0x7fa0e8162920 2024-04-26T23:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.793+0000 7fa0e5600640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 0x7fa0cc06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.794+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa0e008a880 con 0x7fa0e8162920 2024-04-26T23:37:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.794+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 0x7fa0cc06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:04.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.795+0000 7fa0ed600640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 0x7fa0cc06ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc033000 tx=0x7fa0dc009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:04.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:04.798+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0e0058b20 con 0x7fa0e8162920 2024-04-26T23:37:05.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.008+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0e80d19f0 con 0x7fa0e8162920 2024-04-26T23:37:05.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.009+0000 7fa0e5600640 1 -- 172.21.15.46:0/3304431466 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fa0e005ce20 con 0x7fa0e8162920 2024-04-26T23:37:05.012 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 msgr2=0x7fa0cc06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 0x7fa0cc06ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc033000 tx=0x7fa0dc009000 comp rx=0 tx=0).stop 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 msgr2=0x7fa0e80b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c390 tx=0x7fa0e000c850 comp rx=0 tx=0).stop 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 shutdown_connections 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0cc06c890 0x7fa0cc06ed50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e8162920 0x7fa0e80b9150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0e80b3d60 0x7fa0e80ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 --2- 172.21.15.46:0/3304431466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0e80b1cd0 0x7fa0e815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 >> 172.21.15.46:0/3304431466 conn(0x7fa0e80d5000 msgr2=0x7fa0e80b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 shutdown_connections 2024-04-26T23:37:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.012+0000 7fa0eea00640 1 -- 172.21.15.46:0/3304431466 wait complete. 2024-04-26T23:37:05.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:37:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.223+0000 7f40ed800640 1 Processor -- start 2024-04-26T23:37:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.223+0000 7f40ed800640 1 -- start start 2024-04-26T23:37:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.224+0000 7f40ed800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40e815f580 0x7f40e816a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.224+0000 7f40ed800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e80d5e10 con 0x7f40e80d1c90 2024-04-26T23:37:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.224+0000 7f40ed800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e80cfc80 con 0x7f40e80d3d00 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.224+0000 7f40ed800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f40e80cff20 con 0x7f40e815f580 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f40e80d1c90 0x7f40e80d35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59194/0 (socket says 172.21.15.46:59194) 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40e7e00640 1 -- 172.21.15.46:0/3987530686 learned_addr learned my addr 172.21.15.46:0/3987530686 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40ece00640 1 --2- 172.21.15.46:0/3987530686 >> v2:172.21.15.46:3300/0 conn(0x7f40e815f580 0x7f40e816a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40ece00640 1 -- 172.21.15.46:0/3987530686 >> v1:172.21.15.142:6789/0 conn(0x7f40e80d3d00 legacy=0x7f40e815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40ece00640 1 -- 172.21.15.46:0/3987530686 >> v1:172.21.15.46:6789/0 conn(0x7f40e80d1c90 legacy=0x7f40e80d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.225+0000 7f40ece00640 1 -- 172.21.15.46:0/3987530686 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e80d0200 con 0x7f40e815f580 2024-04-26T23:37:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.226+0000 7f40ece00640 1 --2- 172.21.15.46:0/3987530686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e815f580 0x7f40e816a290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e808e940 tx=0x7f40e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455eb47d83bdb288 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40e6a00640 1 -- 172.21.15.46:0/3987530686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007740 con 0x7f40e815f580 2024-04-26T23:37:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40e6a00640 1 -- 172.21.15.46:0/3987530686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e00078e0 con 0x7f40e815f580 2024-04-26T23:37:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40e6a00640 1 -- 172.21.15.46:0/3987530686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007cb0 con 0x7f40e815f580 2024-04-26T23:37:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 -- 172.21.15.46:0/3987530686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e815f580 msgr2=0x7f40e816a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 --2- 172.21.15.46:0/3987530686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e815f580 0x7f40e816a290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e808e940 tx=0x7f40e002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 -- 172.21.15.46:0/3987530686 shutdown_connections 2024-04-26T23:37:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 --2- 172.21.15.46:0/3987530686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e815f580 0x7f40e816a290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 -- 172.21.15.46:0/3987530686 >> 172.21.15.46:0/3987530686 conn(0x7f40e80d5010 msgr2=0x7f40e80d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:05.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 -- 172.21.15.46:0/3987530686 shutdown_connections 2024-04-26T23:37:05.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.227+0000 7f40ed800640 1 -- 172.21.15.46:0/3987530686 wait complete. 2024-04-26T23:37:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.229+0000 7f40ed800640 1 Processor -- start 2024-04-26T23:37:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.229+0000 7f40ed800640 1 -- start start 2024-04-26T23:37:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e80d1c90 0x7f40e81504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e815f580 0x7f40e80cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80d2070 con 0x7f40e80d3d00 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80d07c0 con 0x7f40e80d1c90 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40ed800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40e80d21f0 con 0x7f40e815f580 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.230+0000 7f40e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55886/0 (socket says 172.21.15.46:55886) 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 -- 172.21.15.46:0/2315353061 learned_addr learned my addr 172.21.15.46:0/2315353061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40ece00640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e815f580 0x7f40e80cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7e00640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e80d1c90 0x7f40e81504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 -- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e815f580 msgr2=0x7f40e80cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e815f580 0x7f40e80cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 -- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e80d1c90 msgr2=0x7f40e81504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e80d1c90 0x7f40e81504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 -- 172.21.15.46:0/2315353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e815ac60 con 0x7f40e80d3d00 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.231+0000 7f40e7400640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c400 tx=0x7f40d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.232+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4017070 con 0x7f40e80d3d00 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.232+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d40035f0 con 0x7f40e80d3d00 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.232+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4003a20 con 0x7f40e80d3d00 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.232+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e815af40 con 0x7f40e80d3d00 2024-04-26T23:37:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.232+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e80cc330 con 0x7f40e80d3d00 2024-04-26T23:37:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.233+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e806a640 con 0x7f40e80d3d00 2024-04-26T23:37:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.234+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f40d401a070 con 0x7f40e80d3d00 2024-04-26T23:37:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.235+0000 7f40dfe00640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 0x7f40c806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.235+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f40d408aab0 con 0x7f40e80d3d00 2024-04-26T23:37:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.236+0000 7f40e7e00640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 0x7f40c806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.237+0000 7f40e7e00640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 0x7f40c806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f60 tx=0x7f40d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.241+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40e806a640 con 0x7f40e80d3d00 2024-04-26T23:37:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.451+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f40e80cc500 con 0x7f40e80d3d00 2024-04-26T23:37:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.452+0000 7f40dfe00640 1 -- 172.21.15.46:0/2315353061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f40d40010c0 con 0x7f40e80d3d00 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:37:05.456 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 msgr2=0x7f40c806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 0x7f40c806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f60 tx=0x7f40d8009040 comp rx=0 tx=0).stop 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 msgr2=0x7f40e80ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c400 tx=0x7f40d400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 shutdown_connections 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40c806c860 0x7f40c806ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e815f580 0x7f40e80cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e80d3d00 0x7f40e80ce4d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 --2- 172.21.15.46:0/2315353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e80d1c90 0x7f40e81504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 >> 172.21.15.46:0/2315353061 conn(0x7f40e80d5010 msgr2=0x7f40e80b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 shutdown_connections 2024-04-26T23:37:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.455+0000 7f40ed800640 1 -- 172.21.15.46:0/2315353061 wait complete. 2024-04-26T23:37:05.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:37:05.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:37:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.685+0000 7fc504e00640 1 Processor -- start 2024-04-26T23:37:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.685+0000 7fc504e00640 1 -- start start 2024-04-26T23:37:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50015f6c0 0x7fc5000d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc5000d5820 0x7fc5000d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5000d36f0 0x7fc5000d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc5000d0fe0 con 0x7fc5000d36f0 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc5000d5d00 con 0x7fc5000d5820 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc504e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000cffc0 con 0x7fc50015f6c0 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc4fe800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5000d36f0 0x7fc5000d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc4fe800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5000d36f0 0x7fc5000d3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55890/0 (socket says 172.21.15.46:55890) 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc4fe800640 1 -- 172.21.15.46:0/2341058344 learned_addr learned my addr 172.21.15.46:0/2341058344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc4fde00640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50015f6c0 0x7fc5000d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.687+0000 7fc4fd400640 1 --2- 172.21.15.46:0/2341058344 >> v2:172.21.15.142:3300/0 conn(0x7fc5000d5820 0x7fc5000d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.688+0000 7fc4fe800640 1 -- 172.21.15.46:0/2341058344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50015f6c0 msgr2=0x7fc5000d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.688+0000 7fc4fe800640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50015f6c0 0x7fc5000d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.688+0000 7fc4fe800640 1 -- 172.21.15.46:0/2341058344 >> v2:172.21.15.142:3300/0 conn(0x7fc5000d5820 msgr2=0x7fc5000d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.688+0000 7fc4fe800640 1 --2- 172.21.15.46:0/2341058344 >> v2:172.21.15.142:3300/0 conn(0x7fc5000d5820 0x7fc5000d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.688+0000 7fc4fe800640 1 -- 172.21.15.46:0/2341058344 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000d02a0 con 0x7fc5000d36f0 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.689+0000 7fc4fe800640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5000d36f0 0x7fc5000d3ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc50008f0e0 tx=0x7fc4f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f189718926d7b67e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.689+0000 7fc4fca00640 1 -- 172.21.15.46:0/2341058344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8036070 con 0x7fc5000d36f0 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.689+0000 7fc4fca00640 1 -- 172.21.15.46:0/2341058344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8031600 con 0x7fc5000d36f0 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.689+0000 7fc4fca00640 1 -- 172.21.15.46:0/2341058344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80319d0 con 0x7fc5000d36f0 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 -- 172.21.15.46:0/2341058344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5000d36f0 msgr2=0x7fc5000d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5000d36f0 0x7fc5000d3ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc50008f0e0 tx=0x7fc4f802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 -- 172.21.15.46:0/2341058344 shutdown_connections 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5000d36f0 0x7fc5000d3ba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 --2- 172.21.15.46:0/2341058344 >> v2:172.21.15.142:3300/0 conn(0x7fc5000d5820 0x7fc5000d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 --2- 172.21.15.46:0/2341058344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50015f6c0 0x7fc5000d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 -- 172.21.15.46:0/2341058344 >> 172.21.15.46:0/2341058344 conn(0x7fc50009e450 msgr2=0x7fc5000d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 -- 172.21.15.46:0/2341058344 shutdown_connections 2024-04-26T23:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.690+0000 7fc504e00640 1 -- 172.21.15.46:0/2341058344 wait complete. 2024-04-26T23:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.692+0000 7fc504e00640 1 Processor -- start 2024-04-26T23:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.692+0000 7fc504e00640 1 -- start start 2024-04-26T23:37:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5000d36f0 0x7fc5000b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 0x7fc50015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000cfda0 con 0x7fc50015f6c0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000cfc20 con 0x7fc5000d36f0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc504e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000d30b0 con 0x7fc5000d5820 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55894/0 (socket says 172.21.15.46:55894) 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fe800640 1 -- 172.21.15.46:0/1488524240 learned_addr learned my addr 172.21.15.46:0/1488524240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fde00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5000d36f0 0x7fc5000b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fd400640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 0x7fc50015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.693+0000 7fc4fe800640 1 -- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 msgr2=0x7fc50015b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fe800640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 0x7fc50015b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fe800640 1 -- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5000d36f0 msgr2=0x7fc5000b9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fe800640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5000d36f0 0x7fc5000b9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fe800640 1 -- 172.21.15.46:0/1488524240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000b8f10 con 0x7fc50015f6c0 2024-04-26T23:37:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fd400640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 0x7fc50015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4fe800640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009f40 tx=0x7fc4f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8041020 con 0x7fc50015f6c0 2024-04-26T23:37:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f80077a0 con 0x7fc50015f6c0 2024-04-26T23:37:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8036040 con 0x7fc50015f6c0 2024-04-26T23:37:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5000b91f0 con 0x7fc50015f6c0 2024-04-26T23:37:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.694+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5000b97b0 con 0x7fc50015f6c0 2024-04-26T23:37:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.695+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc500069530 con 0x7fc50015f6c0 2024-04-26T23:37:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.696+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fc4f8044020 con 0x7fc50015f6c0 2024-04-26T23:37:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.698+0000 7fc4f6000640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 0x7fc4e006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.698+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc4f80ac510 con 0x7fc50015f6c0 2024-04-26T23:37:05.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.698+0000 7fc4fde00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 0x7fc4e006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:05.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.700+0000 7fc4fde00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 0x7fc4e006ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:05.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.702+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4f804b030 con 0x7fc50015f6c0 2024-04-26T23:37:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:05.905+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc5000d6020 con 0x7fc50015f6c0 2024-04-26T23:37:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.696+0000 7fc4f6000640 1 -- 172.21.15.46:0/1488524240 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fc4f807acd0 con 0x7fc50015f6c0 2024-04-26T23:37:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.699+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 msgr2=0x7fc4e006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.699+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 0x7fc4e006ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f0008040 comp rx=0 tx=0).stop 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.699+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 msgr2=0x7fc5000cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.699+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009f40 tx=0x7fc4f8004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 shutdown_connections 2024-04-26T23:37:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c860 0x7fc4e006ed20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50015f6c0 0x7fc5000cc8d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5000d5820 0x7fc50015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 --2- 172.21.15.46:0/1488524240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5000d36f0 0x7fc5000b9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 >> 172.21.15.46:0/1488524240 conn(0x7fc50009e450 msgr2=0x7fc5000c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.700+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 shutdown_connections 2024-04-26T23:37:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.701+0000 7fc504e00640 1 -- 172.21.15.46:0/1488524240 wait complete. 2024-04-26T23:37:06.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T23:37:06.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:06.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.930+0000 7f29bfe00640 1 Processor -- start 2024-04-26T23:37:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.930+0000 7f29bfe00640 1 -- start start 2024-04-26T23:37:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 0x7f29c00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29c0162aa0 0x7f29c00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c00d07d0 con 0x7f29c00b5fb0 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f29c00d0ab0 con 0x7f29c0162aa0 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f29c00d0d90 con 0x7f29c01626c0 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29c0162aa0 0x7f29c00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29c0162aa0 0x7f29c00d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55902/0 (socket says 172.21.15.46:55902) 2024-04-26T23:37:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.932+0000 7f29bf400640 1 -- 172.21.15.46:0/2072723401 learned_addr learned my addr 172.21.15.46:0/2072723401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bea00640 1 --2- 172.21.15.46:0/2072723401 >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 0x7f29c00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bf400640 1 -- 172.21.15.46:0/2072723401 >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 msgr2=0x7f29c00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bf400640 1 --2- 172.21.15.46:0/2072723401 >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 0x7f29c00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bf400640 1 -- 172.21.15.46:0/2072723401 >> v1:172.21.15.142:6789/0 conn(0x7f29c00b5fb0 legacy=0x7f29c00d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bf400640 1 -- 172.21.15.46:0/2072723401 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00d2c80 con 0x7f29c0162aa0 2024-04-26T23:37:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.933+0000 7f29bea00640 1 --2- 172.21.15.46:0/2072723401 >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 0x7f29c00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.935+0000 7f29bf400640 1 --2- 172.21.15.46:0/2072723401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c0162aa0 0x7f29c00d1900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29c008f2e0 tx=0x7f29b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa9c4ada6b95d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.935+0000 7f29bd600640 1 -- 172.21.15.46:0/2072723401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8036070 con 0x7f29c0162aa0 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.935+0000 7f29bd600640 1 -- 172.21.15.46:0/2072723401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b8031600 con 0x7f29c0162aa0 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.935+0000 7f29bd600640 1 -- 172.21.15.46:0/2072723401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80319d0 con 0x7f29c0162aa0 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 -- 172.21.15.46:0/2072723401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c0162aa0 msgr2=0x7f29c00d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 --2- 172.21.15.46:0/2072723401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c0162aa0 0x7f29c00d1900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29c008f2e0 tx=0x7f29b802a730 comp rx=0 tx=0).stop 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 -- 172.21.15.46:0/2072723401 shutdown_connections 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 --2- 172.21.15.46:0/2072723401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c0162aa0 0x7f29c00d1900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 --2- 172.21.15.46:0/2072723401 >> v2:172.21.15.142:3300/0 conn(0x7f29c01626c0 0x7f29c00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 -- 172.21.15.46:0/2072723401 >> 172.21.15.46:0/2072723401 conn(0x7f29c009e2a0 msgr2=0x7f29c00d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 -- 172.21.15.46:0/2072723401 shutdown_connections 2024-04-26T23:37:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.936+0000 7f29bfe00640 1 -- 172.21.15.46:0/2072723401 wait complete. 2024-04-26T23:37:06.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.938+0000 7f29bfe00640 1 Processor -- start 2024-04-26T23:37:06.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.938+0000 7f29bfe00640 1 -- start start 2024-04-26T23:37:06.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c00b5fb0 0x7f29c00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29c0162aa0 0x7f29c01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00d2210 con 0x7f29c01626c0 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00d2090 con 0x7f29c00b5fb0 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29c00d31a0 con 0x7f29c0162aa0 2024-04-26T23:37:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c00b5fb0 0x7f29c00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55918/0 (socket says 172.21.15.46:55918) 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29be000640 1 -- 172.21.15.46:0/1551727472 learned_addr learned my addr 172.21.15.46:0/1551727472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.939+0000 7f29bf400640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29c0162aa0 0x7f29c01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 -- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29c0162aa0 msgr2=0x7f29c01f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29c0162aa0 0x7f29c01f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 -- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c00b5fb0 msgr2=0x7f29c00ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c00b5fb0 0x7f29c00ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 -- 172.21.15.46:0/1551727472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00cebd0 con 0x7f29c01626c0 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29be000640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c400 tx=0x7f29ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0052d0 con 0x7f29c01626c0 2024-04-26T23:37:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ac0054b0 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c00cc180 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac006a60 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.940+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c00cc740 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.942+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c00693b0 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.943+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29ac006c00 con 0x7f29c01626c0 2024-04-26T23:37:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.944+0000 7f29b6a00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 0x7f29a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:06.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.944+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f29ac08ae20 con 0x7f29c01626c0 2024-04-26T23:37:06.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.944+0000 7f29bea00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 0x7f29a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:06.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.945+0000 7f29bea00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 0x7f29a006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006e80 tx=0x7f29b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:06.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:06.949+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29ac058d40 con 0x7f29c01626c0 2024-04-26T23:37:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.159+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f29c00d25b0 con 0x7f29c01626c0 2024-04-26T23:37:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.703+0000 7f29b6a00640 1 -- 172.21.15.46:0/1551727472 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f29ac05d040 con 0x7f29c01626c0 2024-04-26T23:37:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 msgr2=0x7f29a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 0x7f29a006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006e80 tx=0x7f29b0008040 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 msgr2=0x7f29c01f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c400 tx=0x7f29ac00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 shutdown_connections 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29a006c8f0 0x7f29a006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29c0162aa0 0x7f29c01f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c01626c0 0x7f29c01f8150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 --2- 172.21.15.46:0/1551727472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c00b5fb0 0x7f29c00ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.707+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 >> 172.21.15.46:0/1551727472 conn(0x7f29c009e2a0 msgr2=0x7f29c00d3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.708+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 shutdown_connections 2024-04-26T23:37:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.708+0000 7f29bfe00640 1 -- 172.21.15.46:0/1551727472 wait complete. 2024-04-26T23:37:07.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:37:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.977+0000 7f8942200640 1 Processor -- start 2024-04-26T23:37:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.977+0000 7f8942200640 1 -- start start 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8942200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f893c06e470 0x7f893c169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8942200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c0d20b0 con 0x7f893c15f3e0 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8942200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c0cdf10 con 0x7f893c0cfa00 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8942200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f893c0d2390 con 0x7f893c06e470 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8940e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f893c15f3e0 0x7f893c0d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59206/0 (socket says 172.21.15.46:59206) 2024-04-26T23:37:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.978+0000 7f8940e00640 1 -- 172.21.15.46:0/151785139 learned_addr learned my addr 172.21.15.46:0/151785139 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f8941800640 1 --2- 172.21.15.46:0/151785139 >> v2:172.21.15.142:3300/0 conn(0x7f893c06e470 0x7f893c169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996249154 0 0) 0x7f893c0d20b0 con 0x7f893c15f3e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8920003520 con 0x7f893c15f3e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 741331722 0 0) 0x7f8920003520 con 0x7f893c15f3e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 >> v2:172.21.15.142:3300/0 conn(0x7f893c06e470 msgr2=0x7f893c169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 --2- 172.21.15.46:0/151785139 >> v2:172.21.15.142:3300/0 conn(0x7f893c06e470 0x7f893c169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 >> v1:172.21.15.142:6789/0 conn(0x7f893c0cfa00 legacy=0x7f893c0d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c0ce1f0 con 0x7f893c15f3e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f892c001670 con 0x7f893c15f3e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 >> v1:172.21.15.46:6789/0 conn(0x7f893c15f3e0 legacy=0x7f893c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89200039b0 0x7f8920003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.979+0000 7f893b400640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893b400640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 0x7f892000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0d2390 con 0x7f89200052e0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f892000c1b0 con 0x7f89200039b0 2024-04-26T23:37:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f892000c3b0 con 0x7f8920004800 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 0x7f892000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8940e00640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89200039b0 0x7f8920003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893be00640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 -- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8920004800 msgr2=0x7f8920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 -- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89200039b0 msgr2=0x7f8920003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89200039b0 0x7f8920003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8941800640 1 -- 172.21.15.46:0/151785139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8920003520 con 0x7f89200052e0 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f893be00640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.980+0000 7f8940e00640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89200039b0 0x7f8920003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.981+0000 7f8941800640 1 -- 172.21.15.46:0/151785139 reap_dead start 2024-04-26T23:37:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.981+0000 7f8941800640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 0x7f892000b9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8934009a90 tx=0x7f8920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9f9191839fe352 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934009690 con 0x7f89200052e0 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8934004020 con 0x7f89200052e0 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f893b400640 1 -- 172.21.15.46:0/151785139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934004410 con 0x7f89200052e0 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 -- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 msgr2=0x7f892000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 0x7f892000b9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8934009a90 tx=0x7f8920005830 comp rx=0 tx=0).stop 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 -- 172.21.15.46:0/151785139 shutdown_connections 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 --2- 172.21.15.46:0/151785139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89200052e0 0x7f892000b9c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 -- 172.21.15.46:0/151785139 >> 172.21.15.46:0/151785139 conn(0x7f893c0b48f0 msgr2=0x7f893c0b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 -- 172.21.15.46:0/151785139 shutdown_connections 2024-04-26T23:37:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.982+0000 7f8942200640 1 -- 172.21.15.46:0/151785139 wait complete. 2024-04-26T23:37:07.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.984+0000 7f8942200640 1 Processor -- start 2024-04-26T23:37:07.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.985+0000 7f8942200640 1 -- start start 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 0x7f893c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c157610 0x7f893c1e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0b40a0 con 0x7f893c1570c0 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0ce560 con 0x7f893c157610 2024-04-26T23:37:07.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8942200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f893c0b4220 con 0x7f89200052e0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55940/0 (socket says 172.21.15.46:55940) 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 -- 172.21.15.46:0/2224027945 learned_addr learned my addr 172.21.15.46:0/2224027945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8940e00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 0x7f893c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8941800640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c157610 0x7f893c1e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 -- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 msgr2=0x7f893c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 0x7f893c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 -- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c157610 msgr2=0x7f893c1e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c157610 0x7f893c1e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f893be00640 1 -- 172.21.15.46:0/2224027945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c156e70 con 0x7f893c1570c0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.986+0000 7f8940e00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 0x7f893c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f893be00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f892800d8e0 tx=0x7f892800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003da0 con 0x7f893c1570c0 2024-04-26T23:37:07.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8928017070 con 0x7f893c1570c0 2024-04-26T23:37:07.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c0b6a20 con 0x7f893c1570c0 2024-04-26T23:37:07.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800b8d0 con 0x7f893c1570c0 2024-04-26T23:37:07.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.987+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c0b6fe0 con 0x7f893c1570c0 2024-04-26T23:37:07.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.988+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c069ac0 con 0x7f893c1570c0 2024-04-26T23:37:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.989+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f892800ba70 con 0x7f893c1570c0 2024-04-26T23:37:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.990+0000 7f8938c00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 0x7f891c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.990+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f892808bdd0 con 0x7f893c1570c0 2024-04-26T23:37:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.991+0000 7f8940e00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 0x7f891c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.992+0000 7f8940e00640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 0x7f891c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f892c001eb0 tx=0x7f892c001080 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:07.995+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f892805a1c0 con 0x7f893c1570c0 2024-04-26T23:37:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.250+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f893c1e7630 con 0x7f893c1570c0 2024-04-26T23:37:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.753+0000 7f8938c00640 1 -- 172.21.15.46:0/2224027945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8928059ac0 con 0x7f893c1570c0 2024-04-26T23:37:08.759 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T23:37:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.765+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 msgr2=0x7f891c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.765+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 0x7f891c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f892c001eb0 tx=0x7f892c001080 comp rx=0 tx=0).stop 2024-04-26T23:37:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.765+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 msgr2=0x7f893c0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.765+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f892800d8e0 tx=0x7f892800dda0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 shutdown_connections 2024-04-26T23:37:08.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891c06c8f0 0x7f891c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c157610 0x7f893c1e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f893c1570c0 0x7f893c0b7ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 --2- 172.21.15.46:0/2224027945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89200052e0 0x7f893c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.768+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 >> 172.21.15.46:0/2224027945 conn(0x7f893c0b48f0 msgr2=0x7f893c070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.769+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 shutdown_connections 2024-04-26T23:37:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.769+0000 7f8942200640 1 -- 172.21.15.46:0/2224027945 wait complete. 2024-04-26T23:37:08.804 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T23:37:08.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T23:37:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.987+0000 7fce31e00640 1 Processor -- start 2024-04-26T23:37:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.988+0000 7fce31e00640 1 -- start start 2024-04-26T23:37:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.989+0000 7fce31e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce2c1628e0 0x7fce2c0b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.989+0000 7fce31e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.989+0000 7fce31e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c0b6140 con 0x7fce2c0d5660 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.989+0000 7fce31e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c0b82e0 con 0x7fce2c0d1fa0 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.989+0000 7fce31e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c0d2f30 con 0x7fce2c1628e0 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55956/0 (socket says 172.21.15.46:55956) 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 -- 172.21.15.46:0/2284570197 learned_addr learned my addr 172.21.15.46:0/2284570197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce30a00640 1 --2- 172.21.15.46:0/2284570197 >> v2:172.21.15.142:3300/0 conn(0x7fce2c1628e0 0x7fce2c0b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 -- 172.21.15.46:0/2284570197 >> v2:172.21.15.142:3300/0 conn(0x7fce2c1628e0 msgr2=0x7fce2c0b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 --2- 172.21.15.46:0/2284570197 >> v2:172.21.15.142:3300/0 conn(0x7fce2c1628e0 0x7fce2c0b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 -- 172.21.15.46:0/2284570197 >> v1:172.21.15.142:6789/0 conn(0x7fce2c0d5660 legacy=0x7fce2c0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.990+0000 7fce31400640 1 -- 172.21.15.46:0/2284570197 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c0d0f10 con 0x7fce2c0d1fa0 2024-04-26T23:37:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.991+0000 7fce31400640 1 --2- 172.21.15.46:0/2284570197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06cbe0 tx=0x7fce2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98abd9ef1234b43b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.992+0000 7fce2b400640 1 -- 172.21.15.46:0/2284570197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24036070 con 0x7fce2c0d1fa0 2024-04-26T23:37:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.992+0000 7fce2b400640 1 -- 172.21.15.46:0/2284570197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24031600 con 0x7fce2c0d1fa0 2024-04-26T23:37:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.992+0000 7fce2b400640 1 -- 172.21.15.46:0/2284570197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240319d0 con 0x7fce2c0d1fa0 2024-04-26T23:37:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.992+0000 7fce31e00640 1 -- 172.21.15.46:0/2284570197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c0d1fa0 msgr2=0x7fce2c0d33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.992+0000 7fce31e00640 1 --2- 172.21.15.46:0/2284570197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06cbe0 tx=0x7fce2402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 -- 172.21.15.46:0/2284570197 shutdown_connections 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 --2- 172.21.15.46:0/2284570197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c0d1fa0 0x7fce2c0d33c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 --2- 172.21.15.46:0/2284570197 >> v2:172.21.15.142:3300/0 conn(0x7fce2c1628e0 0x7fce2c0b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 -- 172.21.15.46:0/2284570197 >> 172.21.15.46:0/2284570197 conn(0x7fce2c09e2c0 msgr2=0x7fce2c0d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 -- 172.21.15.46:0/2284570197 shutdown_connections 2024-04-26T23:37:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.993+0000 7fce31e00640 1 -- 172.21.15.46:0/2284570197 wait complete. 2024-04-26T23:37:08.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.994+0000 7fce31e00640 1 Processor -- start 2024-04-26T23:37:08.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.995+0000 7fce31e00640 1 -- start start 2024-04-26T23:37:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c0d1fa0 0x7fce2c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 0x7fce2c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0d0c30 con 0x7fce2c1628e0 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0d0950 con 0x7fce2c0d5660 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c0d2bc0 con 0x7fce2c0d1fa0 2024-04-26T23:37:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55972/0 (socket says 172.21.15.46:55972) 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 -- 172.21.15.46:0/3252448958 learned_addr learned my addr 172.21.15.46:0/3252448958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce30a00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c0d1fa0 0x7fce2c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce2be00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 0x7fce2c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 -- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c0d1fa0 msgr2=0x7fce2c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c0d1fa0 0x7fce2c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 -- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 msgr2=0x7fce2c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 0x7fce2c0c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce31400640 1 -- 172.21.15.46:0/3252448958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c0c9140 con 0x7fce2c1628e0 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.996+0000 7fce2be00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 0x7fce2c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce31400640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce24034040 tx=0x7fce24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24036050 con 0x7fce2c1628e0 2024-04-26T23:37:09.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24042070 con 0x7fce2c1628e0 2024-04-26T23:37:09.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c0c9420 con 0x7fce2c1628e0 2024-04-26T23:37:09.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24009510 con 0x7fce2c1628e0 2024-04-26T23:37:09.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.997+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c0c99e0 con 0x7fce2c1628e0 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.998+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c0693b0 con 0x7fce2c1628e0 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:08.999+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce24007bd0 con 0x7fce2c1628e0 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.000+0000 7fce28c00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.000+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce240aca20 con 0x7fce2c1628e0 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.001+0000 7fce30a00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.002+0000 7fce30a00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:09.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.005+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce2c0b6140 con 0x7fce2c1628e0 2024-04-26T23:37:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.216+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fce2c0cbd90 con 0x7fce2c1628e0 2024-04-26T23:37:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.734+0000 7fce28c00640 1 -- 172.21.15.46:0/3252448958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fce2407ae40 con 0x7fce2c1628e0 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.737+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 msgr2=0x7fce0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.737+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).stop 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.737+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 msgr2=0x7fce2c161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.737+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce24034040 tx=0x7fce24004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 shutdown_connections 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce2c1628e0 0x7fce2c161050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0d5660 0x7fce2c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 --2- 172.21.15.46:0/3252448958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c0d1fa0 0x7fce2c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.738+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 >> 172.21.15.46:0/3252448958 conn(0x7fce2c09e2c0 msgr2=0x7fce2c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:09.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.739+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 shutdown_connections 2024-04-26T23:37:09.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.739+0000 7fce31e00640 1 -- 172.21.15.46:0/3252448958 wait complete. 2024-04-26T23:37:09.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:37:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.983+0000 7f72d1000640 1 Processor -- start 2024-04-26T23:37:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.983+0000 7f72d1000640 1 -- start start 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72d1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72cc06f240 0x7f72cc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72d1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc160610 0x7f72cc0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72d1000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72cc0d0380 con 0x7f72cc06f620 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72d1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f72cc0cfef0 con 0x7f72cc06f240 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72d1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72cc0cf2e0 con 0x7f72cc160610 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72cb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72cc06f240 0x7f72cc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72cb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72cc06f240 0x7f72cc0cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55982/0 (socket says 172.21.15.46:55982) 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.985+0000 7f72cb400640 1 -- 172.21.15.46:0/2086268766 learned_addr learned my addr 172.21.15.46:0/2086268766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72caa00640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc160610 0x7f72cc0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72cb400640 1 -- 172.21.15.46:0/2086268766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc160610 msgr2=0x7f72cc0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72cb400640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc160610 0x7f72cc0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72cb400640 1 -- 172.21.15.46:0/2086268766 >> v1:172.21.15.46:6789/0 conn(0x7f72cc06f620 legacy=0x7f72cc0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72ca000640 1 -- 172.21.15.46:0/2086268766 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670764286 0 0) 0x7f72cc0d0380 con 0x7f72cc06f620 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.986+0000 7f72cb400640 1 -- 172.21.15.46:0/2086268766 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc0cf5c0 con 0x7f72cc06f240 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.987+0000 7f72cb400640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc0cdee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009870 tx=0x7f72bc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e481fa8c9221a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.987+0000 7f72ca000640 1 -- 172.21.15.46:0/2086268766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc037070 con 0x7f72cc06f240 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.987+0000 7f72ca000640 1 -- 172.21.15.46:0/2086268766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc032440 con 0x7f72cc06f240 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.987+0000 7f72ca000640 1 -- 172.21.15.46:0/2086268766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc032810 con 0x7f72cc06f240 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 -- 172.21.15.46:0/2086268766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 msgr2=0x7f72cc0cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc0cdee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009870 tx=0x7f72bc02a250 comp rx=0 tx=0).stop 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 -- 172.21.15.46:0/2086268766 shutdown_connections 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc160610 0x7f72cc0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 --2- 172.21.15.46:0/2086268766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc0cdee0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 -- 172.21.15.46:0/2086268766 >> 172.21.15.46:0/2086268766 conn(0x7f72cc0b27b0 msgr2=0x7f72cc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 -- 172.21.15.46:0/2086268766 shutdown_connections 2024-04-26T23:37:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.988+0000 7f72d1000640 1 -- 172.21.15.46:0/2086268766 wait complete. 2024-04-26T23:37:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.990+0000 7f72d1000640 1 Processor -- start 2024-04-26T23:37:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.990+0000 7f72d1000640 1 -- start start 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc06f620 0x7f72cc15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72cc160610 0x7f72cc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc0ce420 con 0x7f72cc06f240 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc0cfb80 con 0x7f72cc160610 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72d1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72cc0cc2d0 con 0x7f72cc06f620 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55998/0 (socket says 172.21.15.46:55998) 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 -- 172.21.15.46:0/1829760045 learned_addr learned my addr 172.21.15.46:0/1829760045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 -- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc06f620 msgr2=0x7f72cc15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc06f620 0x7f72cc15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 -- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72cc160610 msgr2=0x7f72cc15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72cc160610 0x7f72cc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.991+0000 7f72cb400640 1 -- 172.21.15.46:0/1829760045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc149660 con 0x7f72cc06f240 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72cb400640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc037040 con 0x7f72cc06f240 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc042070 con 0x7f72cc06f240 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc030bd0 con 0x7f72cc06f240 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc1f5d70 con 0x7f72cc06f240 2024-04-26T23:37:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.992+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc15a4a0 con 0x7f72cc06f240 2024-04-26T23:37:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.993+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72cc15a6e0 con 0x7f72cc06f240 2024-04-26T23:37:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.994+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72bc047460 con 0x7f72cc06f240 2024-04-26T23:37:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.995+0000 7f72c3400640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 0x7f72ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.995+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f72bc0ac9a0 con 0x7f72cc06f240 2024-04-26T23:37:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.996+0000 7f72caa00640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 0x7f72ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:09.997+0000 7f72caa00640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 0x7f72ac06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72cc073430 tx=0x7f72b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:10.000+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72cc15a6e0 con 0x7f72cc06f240 2024-04-26T23:37:10.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:10.204+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f72cc0693b0 con 0x7f72cc06f240 2024-04-26T23:37:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.773+0000 7f72c3400640 1 -- 172.21.15.46:0/1829760045 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f72cc149660 con 0x7f72cc06f240 2024-04-26T23:37:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-04-26T23:37:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.777+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 msgr2=0x7f72ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.777+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 0x7f72ac06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72cc073430 tx=0x7f72b8008040 comp rx=0 tx=0).stop 2024-04-26T23:37:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.777+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 msgr2=0x7f72cc149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.777+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 shutdown_connections 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72ac06c8f0 0x7f72ac06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72cc160610 0x7f72cc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72cc06f620 0x7f72cc15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 --2- 172.21.15.46:0/1829760045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72cc06f240 0x7f72cc149120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.778+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 >> 172.21.15.46:0/1829760045 conn(0x7f72cc0b27b0 msgr2=0x7f72cc15e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.779+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 shutdown_connections 2024-04-26T23:37:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.779+0000 7f72d1000640 1 -- 172.21.15.46:0/1829760045 wait complete. 2024-04-26T23:37:11.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T23:37:11.850 INFO:tasks.ceph.mds.b.smithi142.stderr:2024-04-26T23:37:11.847+0000 7f246b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:37:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.994+0000 7f7dd5600640 1 Processor -- start 2024-04-26T23:37:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.994+0000 7f7dd5600640 1 -- start start 2024-04-26T23:37:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dd5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dd01469c0 0x7f7dd015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dd5600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7dd00cf560 0x7f7dd00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dd5600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd0146da0 con 0x7f7dd00cfa10 2024-04-26T23:37:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dd5600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd00d0ac0 con 0x7f7dd01469c0 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dd5600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd00cfeb0 con 0x7f7dd00cf560 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dcea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dd01469c0 0x7f7dd015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dcea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dd01469c0 0x7f7dd015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56940/0 (socket says 172.21.15.46:56940) 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dcea00640 1 -- 172.21.15.46:0/1493307463 learned_addr learned my addr 172.21.15.46:0/1493307463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.996+0000 7f7dce000640 1 --2- 172.21.15.46:0/1493307463 >> v2:172.21.15.142:3300/0 conn(0x7f7dd00cf560 0x7f7dd00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.997+0000 7f7dcea00640 1 -- 172.21.15.46:0/1493307463 >> v2:172.21.15.142:3300/0 conn(0x7f7dd00cf560 msgr2=0x7f7dd00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.997+0000 7f7dcea00640 1 --2- 172.21.15.46:0/1493307463 >> v2:172.21.15.142:3300/0 conn(0x7f7dd00cf560 0x7f7dd00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.997+0000 7f7dcea00640 1 -- 172.21.15.46:0/1493307463 >> v1:172.21.15.46:6789/0 conn(0x7f7dd00cfa10 legacy=0x7f7dd006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.997+0000 7f7dcea00640 1 -- 172.21.15.46:0/1493307463 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd00d0190 con 0x7f7dd01469c0 2024-04-26T23:37:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.998+0000 7f7dcea00640 1 --2- 172.21.15.46:0/1493307463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd015d6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00b6330 tx=0x7f7dc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd50db8b99f1890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.998+0000 7f7dcd600640 1 -- 172.21.15.46:0/1493307463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0037070 con 0x7f7dd01469c0 2024-04-26T23:37:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.998+0000 7f7dcd600640 1 -- 172.21.15.46:0/1493307463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc0032440 con 0x7f7dd01469c0 2024-04-26T23:37:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.998+0000 7f7dcd600640 1 -- 172.21.15.46:0/1493307463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0032810 con 0x7f7dd01469c0 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 -- 172.21.15.46:0/1493307463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 msgr2=0x7f7dd015d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 --2- 172.21.15.46:0/1493307463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd015d6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00b6330 tx=0x7f7dc002a270 comp rx=0 tx=0).stop 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 -- 172.21.15.46:0/1493307463 shutdown_connections 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 --2- 172.21.15.46:0/1493307463 >> v2:172.21.15.142:3300/0 conn(0x7f7dd00cf560 0x7f7dd00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 --2- 172.21.15.46:0/1493307463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd015d6d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 -- 172.21.15.46:0/1493307463 >> 172.21.15.46:0/1493307463 conn(0x7f7dd006c770 msgr2=0x7f7dd006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 -- 172.21.15.46:0/1493307463 shutdown_connections 2024-04-26T23:37:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:11.999+0000 7f7dd5600640 1 -- 172.21.15.46:0/1493307463 wait complete. 2024-04-26T23:37:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.001+0000 7f7dd5600640 1 Processor -- start 2024-04-26T23:37:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.001+0000 7f7dd5600640 1 -- start start 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 0x7f7dd013eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd00cfa10 0x7f7dd013c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00737e0 con 0x7f7dd01469c0 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d0750 con 0x7f7dd00cfa10 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dd5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd0073960 con 0x7f7dd00cf560 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dcf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dcf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56948/0 (socket says 172.21.15.46:56948) 2024-04-26T23:37:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.002+0000 7f7dcf400640 1 -- 172.21.15.46:0/917244831 learned_addr learned my addr 172.21.15.46:0/917244831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dce000640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd00cfa10 0x7f7dd013c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcea00640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 0x7f7dd013eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 -- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 msgr2=0x7f7dd013eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 0x7f7dd013eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 -- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd00cfa10 msgr2=0x7f7dd013c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd00cfa10 0x7f7dd013c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 -- 172.21.15.46:0/917244831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd013cbb0 con 0x7f7dd01469c0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcea00640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 0x7f7dd013eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dcf400640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c7d0 tx=0x7f7dc800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8003b70 con 0x7f7dd01469c0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc8003d30 con 0x7f7dd01469c0 2024-04-26T23:37:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800aa30 con 0x7f7dd01469c0 2024-04-26T23:37:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.003+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd013ce90 con 0x7f7dd01469c0 2024-04-26T23:37:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.004+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd013d450 con 0x7f7dd01469c0 2024-04-26T23:37:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.005+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0069530 con 0x7f7dd01469c0 2024-04-26T23:37:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.006+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dc800abd0 con 0x7f7dd01469c0 2024-04-26T23:37:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.007+0000 7f7dc6a00640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 0x7f7db006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.007+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f7dc808bdf0 con 0x7f7dd01469c0 2024-04-26T23:37:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.007+0000 7f7dcea00640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 0x7f7db006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:12.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.008+0000 7f7dcea00640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 0x7f7db006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0034000 tx=0x7f7dc002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.012+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7dc8059e60 con 0x7f7dd01469c0 2024-04-26T23:37:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.214+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f7dd00aefd0 con 0x7f7dd01469c0 2024-04-26T23:37:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.785+0000 7f7dc6a00640 1 -- 172.21.15.46:0/917244831 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f7dc8059760 con 0x7f7dd01469c0 2024-04-26T23:37:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T23:37:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.788+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 msgr2=0x7f7db006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 0x7f7db006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0034000 tx=0x7f7dc002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 msgr2=0x7f7dd013baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c7d0 tx=0x7f7dc800cc90 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 shutdown_connections 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7db006c8f0 0x7f7db006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd01469c0 0x7f7dd013baa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd00cfa10 0x7f7dd013c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 --2- 172.21.15.46:0/917244831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd00cf560 0x7f7dd013eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.789+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 >> 172.21.15.46:0/917244831 conn(0x7f7dd006c770 msgr2=0x7f7dd00b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.791+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 shutdown_connections 2024-04-26T23:37:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:12.791+0000 7f7dd5600640 1 -- 172.21.15.46:0/917244831 wait complete. 2024-04-26T23:37:12.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T23:37:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.008+0000 7f7db9a00640 1 Processor -- start 2024-04-26T23:37:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.008+0000 7f7db9a00640 1 -- start start 2024-04-26T23:37:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7db41595c0 0x7f7db4073a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7db40b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db40aeb30 con 0x7f7db4071f90 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7db40ae8e0 con 0x7f7db41595c0 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7db40aee10 con 0x7f7db40d0e90 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7dabe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7db4071f90 0x7f7db40d0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40258/0 (socket says 172.21.15.46:40258) 2024-04-26T23:37:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7dabe00640 1 -- 172.21.15.46:0/2427206107 learned_addr learned my addr 172.21.15.46:0/2427206107 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db9000640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7db40b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db3e00640 1 --2- 172.21.15.46:0/2427206107 >> v2:172.21.15.142:3300/0 conn(0x7f7db41595c0 0x7f7db4073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019723356 0 0) 0x7f7db40aeb30 con 0x7f7db4071f90 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.010+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d98003520 con 0x7f7db4071f90 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4178369972 0 0) 0x7f7d98003520 con 0x7f7db4071f90 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 msgr2=0x7f7db40b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7db40b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 >> v2:172.21.15.142:3300/0 conn(0x7f7db41595c0 msgr2=0x7f7db4073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 --2- 172.21.15.46:0/2427206107 >> v2:172.21.15.142:3300/0 conn(0x7f7db41595c0 0x7f7db4073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db40af0f0 con 0x7f7db4071f90 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7d9c001690 con 0x7f7db4071f90 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 >> v1:172.21.15.46:6789/0 conn(0x7f7db4071f90 legacy=0x7f7db40d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7d980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7d98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d98004d40 0x7f7d980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7db40ae8e0 con 0x7f7d98003f80 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7db40aee10 con 0x7f7d98004d40 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9800b9c0 con 0x7f7db40d0e90 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.011+0000 7f7dabe00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7d98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7db3e00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7d980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7dabe00640 1 -- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 msgr2=0x7f7d980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7dabe00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db40d0e90 0x7f7d980039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7dabe00640 1 -- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d98004d40 msgr2=0x7f7d980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7dabe00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d98004d40 0x7f7d980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.012+0000 7f7dabe00640 1 -- 172.21.15.46:0/2427206107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d98003520 con 0x7f7d98003f80 2024-04-26T23:37:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7dabe00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7d98004800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c160 tx=0x7f7d98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0e60db43ba6975 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c007c80 con 0x7f7d98003f80 2024-04-26T23:37:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c005ad0 con 0x7f7d98003f80 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db3400640 1 -- 172.21.15.46:0/2427206107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c007070 con 0x7f7d98003f80 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db9a00640 1 -- 172.21.15.46:0/2427206107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 msgr2=0x7f7d98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db9a00640 1 --2- 172.21.15.46:0/2427206107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7d98004800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c160 tx=0x7f7d98005830 comp rx=0 tx=0).stop 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.013+0000 7f7db9000640 1 -- 172.21.15.46:0/2427206107 reap_dead start 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.014+0000 7f7db9a00640 1 -- 172.21.15.46:0/2427206107 shutdown_connections 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.014+0000 7f7db9a00640 1 -- 172.21.15.46:0/2427206107 >> 172.21.15.46:0/2427206107 conn(0x7f7db406c350 msgr2=0x7f7db406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.014+0000 7f7db9a00640 1 -- 172.21.15.46:0/2427206107 shutdown_connections 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.014+0000 7f7db9a00640 1 -- 172.21.15.46:0/2427206107 wait complete. 2024-04-26T23:37:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.015+0000 7f7db9a00640 1 Processor -- start 2024-04-26T23:37:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.016+0000 7f7db9a00640 1 -- start start 2024-04-26T23:37:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.016+0000 7f7db9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db4143510 0x7f7db4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db41508a0 0x7f7db4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41599a0 con 0x7f7d98003f80 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7db40af6b0 con 0x7f7db41508a0 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7db4073f70 con 0x7f7db4143510 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56970/0 (socket says 172.21.15.46:56970) 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 -- 172.21.15.46:0/3105741173 learned_addr learned my addr 172.21.15.46:0/3105741173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7dabe00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db4143510 0x7f7db4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db9000640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db41508a0 0x7f7db4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 -- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db4143510 msgr2=0x7f7db4150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db4143510 0x7f7db4150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 -- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db41508a0 msgr2=0x7f7db4150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db41508a0 0x7f7db4150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.017+0000 7f7db3e00640 1 -- 172.21.15.46:0/3105741173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db40cdb70 con 0x7f7d98003f80 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db3e00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7da40108c0 tx=0x7f7da4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da400f040 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da400eb60 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da401c400 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db40cfcd0 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.018+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db40d0290 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.019+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db40693b0 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.020+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7da401c5a0 con 0x7f7d98003f80 2024-04-26T23:37:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.022+0000 7f7db0c00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.022+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f7da408c5d0 con 0x7f7d98003f80 2024-04-26T23:37:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.022+0000 7f7dabe00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.024+0000 7f7dabe00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 0x7f7d9006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c032000 tx=0x7f7d9c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.027+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7da4018030 con 0x7f7d98003f80 2024-04-26T23:37:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.282+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f7db4153720 con 0x7f7d98003f80 2024-04-26T23:37:13.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.788+0000 7f7db0c00640 1 -- 172.21.15.46:0/3105741173 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f7da405a640 con 0x7f7d98003f80 2024-04-26T23:37:13.792 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-04-26T23:37:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 msgr2=0x7f7d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 0x7f7d9006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c032000 tx=0x7f7d9c009030 comp rx=0 tx=0).stop 2024-04-26T23:37:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 msgr2=0x7f7db40cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7da40108c0 tx=0x7f7da4010d80 comp rx=0 tx=0).stop 2024-04-26T23:37:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 shutdown_connections 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db41508a0 0x7f7db4150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db4143510 0x7f7db4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 --2- 172.21.15.46:0/3105741173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d98003f80 0x7f7db40cd4d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 >> 172.21.15.46:0/3105741173 conn(0x7f7db406c350 msgr2=0x7f7db41541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.792+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 shutdown_connections 2024-04-26T23:37:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:13.793+0000 7f7db9a00640 1 -- 172.21.15.46:0/3105741173 wait complete. 2024-04-26T23:37:13.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.019+0000 7fcdffe00640 1 Processor -- start 2024-04-26T23:37:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.020+0000 7fcdffe00640 1 -- start start 2024-04-26T23:37:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.021+0000 7fcdffe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce0014c2e0 0x7fce000ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.021+0000 7fcdffe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fce000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.021+0000 7fcdffe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0014c6c0 con 0x7fce000cb0f0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.021+0000 7fcdffe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fce000cb8b0 con 0x7fce0014c2e0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.021+0000 7fcdffe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce00167e10 con 0x7fce000cdbc0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfe000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce000cb0f0 0x7fce000cd4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40270/0 (socket says 172.21.15.46:40270) 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfe000640 1 -- 172.21.15.46:0/1680147463 learned_addr learned my addr 172.21.15.46:0/1680147463 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1680147463 >> v2:172.21.15.142:3300/0 conn(0x7fce0014c2e0 0x7fce000ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615846990 0 0) 0x7fce0014c6c0 con 0x7fce000cb0f0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcde4003520 con 0x7fce000cb0f0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdff400640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fce000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3144637569 0 0) 0x7fcde4003520 con 0x7fce000cb0f0 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 msgr2=0x7fce000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fce000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 >> v2:172.21.15.142:3300/0 conn(0x7fce0014c2e0 msgr2=0x7fce000ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 --2- 172.21.15.46:0/1680147463 >> v2:172.21.15.142:3300/0 conn(0x7fce0014c2e0 0x7fce000ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce00167050 con 0x7fce000cb0f0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.022+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcdf00019f0 con 0x7fce000cb0f0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdff400640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fce000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 >> v1:172.21.15.46:6789/0 conn(0x7fce000cb0f0 legacy=0x7fce000cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fcde40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4003ef0 0x7fcde4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4004d40 0x7fcde40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce000cb8b0 con 0x7fcde4003ef0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce00167e10 con 0x7fcde4004d40 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcde400ba70 con 0x7fce000cdbc0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4003ef0 0x7fcde4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdff400640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4004d40 0x7fcde40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fcde40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 -- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 msgr2=0x7fcde40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000cdbc0 0x7fcde40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 -- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4004d40 msgr2=0x7fcde40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4004d40 0x7fcde40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.023+0000 7fcdfe000640 1 -- 172.21.15.46:0/1680147463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde4003520 con 0x7fcde4003ef0 2024-04-26T23:37:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.024+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4003ef0 0x7fcde4004800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000a920 tx=0x7fcde4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d022c2fbf3e17c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000a4c0 con 0x7fcde4003ef0 2024-04-26T23:37:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf002f070 con 0x7fcde4003ef0 2024-04-26T23:37:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdfd600640 1 -- 172.21.15.46:0/1680147463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002bd40 con 0x7fcde4003ef0 2024-04-26T23:37:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdffe00640 1 -- 172.21.15.46:0/1680147463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4003ef0 msgr2=0x7fcde4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1680147463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4003ef0 0x7fcde4004800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000a920 tx=0x7fcde4005830 comp rx=0 tx=0).stop 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdff400640 1 -- 172.21.15.46:0/1680147463 reap_dead start 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdffe00640 1 -- 172.21.15.46:0/1680147463 shutdown_connections 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.025+0000 7fcdffe00640 1 -- 172.21.15.46:0/1680147463 >> 172.21.15.46:0/1680147463 conn(0x7fce000b26d0 msgr2=0x7fce000b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.026+0000 7fcdffe00640 1 -- 172.21.15.46:0/1680147463 shutdown_connections 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.026+0000 7fcdffe00640 1 -- 172.21.15.46:0/1680147463 wait complete. 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.027+0000 7fcdffe00640 1 Processor -- start 2024-04-26T23:37:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.028+0000 7fcdffe00640 1 -- start start 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 0x7fce000b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000c8560 0x7fce00164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce001678f0 con 0x7fce000c8120 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce00167610 con 0x7fcde4003ef0 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdffe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce000cb580 con 0x7fce000c8560 2024-04-26T23:37:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56986/0 (socket says 172.21.15.46:56986) 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 -- 172.21.15.46:0/1297234798 learned_addr learned my addr 172.21.15.46:0/1297234798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 0x7fce000b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdff400640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000c8560 0x7fce00164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 -- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000c8560 msgr2=0x7fce00164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000c8560 0x7fce00164a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 -- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 msgr2=0x7fce000b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 0x7fce000b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfe000640 1 -- 172.21.15.46:0/1297234798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce00166860 con 0x7fce000c8120 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.029+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 0x7fce000b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdfe000640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000a920 tx=0x7fce00163930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002b8a0 con 0x7fce000c8120 2024-04-26T23:37:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf002ba40 con 0x7fce000c8120 2024-04-26T23:37:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002bdf0 con 0x7fce000c8120 2024-04-26T23:37:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce000b6590 con 0x7fce000c8120 2024-04-26T23:37:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.030+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce000b3c80 con 0x7fce000c8120 2024-04-26T23:37:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.031+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce000b6ab0 con 0x7fce000c8120 2024-04-26T23:37:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.032+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdf0034470 con 0x7fce000c8120 2024-04-26T23:37:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.034+0000 7fcdf6a00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 0x7fcddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.034+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcdf00ac970 con 0x7fce000c8120 2024-04-26T23:37:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.034+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 0x7fcddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.035+0000 7fcdfea00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 0x7fcddc06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009a70 tx=0x7fcde8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.038+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdf007a9c0 con 0x7fce000c8120 2024-04-26T23:37:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.282+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce000693b0 con 0x7fce000c8120 2024-04-26T23:37:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.283+0000 7fcdf6a00640 1 -- 172.21.15.46:0/1297234798 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7fcdf007a2c0 con 0x7fce000c8120 2024-04-26T23:37:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:14.287 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:13.283993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-04-26T23:37:14.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 msgr2=0x7fcddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 0x7fcddc06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009a70 tx=0x7fcde8008040 comp rx=0 tx=0).stop 2024-04-26T23:37:14.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 msgr2=0x7fce00146f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000a920 tx=0x7fce00163930 comp rx=0 tx=0).stop 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 shutdown_connections 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcddc06c8f0 0x7fcddc06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce000c8560 0x7fce00164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000c8120 0x7fce00146f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 --2- 172.21.15.46:0/1297234798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde4003ef0 0x7fce000b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.286+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 >> 172.21.15.46:0/1297234798 conn(0x7fce000b26d0 msgr2=0x7fce0006de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.287+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 shutdown_connections 2024-04-26T23:37:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.287+0000 7fcdffe00640 1 -- 172.21.15.46:0/1297234798 wait complete. 2024-04-26T23:37:14.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.518+0000 7f27cfe00640 1 Processor -- start 2024-04-26T23:37:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.519+0000 7f27cfe00640 1 -- start start 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.520+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 0x7f27d00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.520+0000 7f27cfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27d015f030 0x7f27d015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.520+0000 7f27cfe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d015f990 con 0x7f27d015fcb0 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.520+0000 7f27cfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f27d00cba60 con 0x7f27d015f030 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.520+0000 7f27cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27d00cbcb0 con 0x7f27d0073be0 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27d015f030 0x7f27d015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27d015fcb0 0x7f27d00c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40282/0 (socket says 172.21.15.46:40282) 2024-04-26T23:37:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cea00640 1 -- 172.21.15.46:0/792886611 learned_addr learned my addr 172.21.15.46:0/792886611 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27c7e00640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 0x7f27d00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cf400640 1 -- 172.21.15.46:0/792886611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 msgr2=0x7f27d00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cf400640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 0x7f27d00cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cf400640 1 -- 172.21.15.46:0/792886611 >> v1:172.21.15.46:6789/0 conn(0x7f27d015fcb0 legacy=0x7f27d00c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27ce000640 1 -- 172.21.15.46:0/792886611 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234158060 0 0) 0x7f27d015f990 con 0x7f27d015fcb0 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27cf400640 1 -- 172.21.15.46:0/792886611 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d00cbf90 con 0x7f27d015f030 2024-04-26T23:37:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.521+0000 7f27c7e00640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 0x7f27d00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.522+0000 7f27cf400640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d015f450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009d90 tx=0x7f27c802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d6d243a7e025e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27ce000640 1 -- 172.21.15.46:0/792886611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8007740 con 0x7f27d015f030 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27ce000640 1 -- 172.21.15.46:0/792886611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c80078e0 con 0x7f27d015f030 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27ce000640 1 -- 172.21.15.46:0/792886611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8007cb0 con 0x7f27d015f030 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 -- 172.21.15.46:0/792886611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 msgr2=0x7f27d015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d015f450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009d90 tx=0x7f27c802c7e0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 -- 172.21.15.46:0/792886611 shutdown_connections 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d015f450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 --2- 172.21.15.46:0/792886611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d0073be0 0x7f27d00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.523+0000 7f27cfe00640 1 -- 172.21.15.46:0/792886611 >> 172.21.15.46:0/792886611 conn(0x7f27d00b26d0 msgr2=0x7f27d00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.524+0000 7f27cfe00640 1 -- 172.21.15.46:0/792886611 shutdown_connections 2024-04-26T23:37:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.524+0000 7f27cfe00640 1 -- 172.21.15.46:0/792886611 wait complete. 2024-04-26T23:37:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.525+0000 7f27cfe00640 1 Processor -- start 2024-04-26T23:37:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 -- start start 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d0073be0 0x7f27d014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d015fcb0 0x7f27d015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27d00cda60 con 0x7f27d015f030 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0074000 con 0x7f27d0073be0 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.526+0000 7f27cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27d00cdbe0 con 0x7f27d015fcb0 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56994/0 (socket says 172.21.15.46:56994) 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 -- 172.21.15.46:0/1363976486 learned_addr learned my addr 172.21.15.46:0/1363976486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27cea00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d0073be0 0x7f27d014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27cf400640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d015fcb0 0x7f27d015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 -- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d015fcb0 msgr2=0x7f27d015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d015fcb0 0x7f27d015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 -- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d0073be0 msgr2=0x7f27d014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d0073be0 0x7f27d014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.527+0000 7f27c7e00640 1 -- 172.21.15.46:0/1363976486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d014dcf0 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27c7e00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c820 tx=0x7f27b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8015070 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b8007a30 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8007e00 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d00b44b0 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.528+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d00b4a70 con 0x7f27d015f030 2024-04-26T23:37:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.529+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d00693b0 con 0x7f27d015f030 2024-04-26T23:37:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.530+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27b8018070 con 0x7f27d015f030 2024-04-26T23:37:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.531+0000 7f27c7400640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 0x7f27ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.531+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f27b808bf00 con 0x7f27d015f030 2024-04-26T23:37:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.532+0000 7f27cea00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 0x7f27ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.533+0000 7f27cea00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 0x7f27ac06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000f60 tx=0x7f27c0007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.536+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27d014dcf0 con 0x7f27d015f030 2024-04-26T23:37:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.744+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27d00c68c0 con 0x7f27d015f030 2024-04-26T23:37:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.747+0000 7f27c7400640 1 -- 172.21.15.46:0/1363976486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f27b8059f50 con 0x7f27d015f030 2024-04-26T23:37:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:14.750 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.749+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 msgr2=0x7f27ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.749+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 0x7f27ac06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000f60 tx=0x7f27c0007c00 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.749+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 msgr2=0x7f27d00b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.749+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c820 tx=0x7f27b800cc20 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 shutdown_connections 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f27ac06c8f0 0x7f27ac06edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27d015fcb0 0x7f27d015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d015f030 0x7f27d00b5550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 --2- 172.21.15.46:0/1363976486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d0073be0 0x7f27d014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 >> 172.21.15.46:0/1363976486 conn(0x7f27d00b26d0 msgr2=0x7f27d00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 shutdown_connections 2024-04-26T23:37:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.750+0000 7f27cfe00640 1 -- 172.21.15.46:0/1363976486 wait complete. 2024-04-26T23:37:14.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.978+0000 7f621f000640 1 Processor -- start 2024-04-26T23:37:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.979+0000 7f621f000640 1 -- start start 2024-04-26T23:37:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.980+0000 7f621f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f621806e510 0x7f621806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.980+0000 7f621f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621806ee30 con 0x7f62180732c0 2024-04-26T23:37:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.980+0000 7f621f000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6218161940 con 0x7f6218160bd0 2024-04-26T23:37:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.980+0000 7f621f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6218071f80 con 0x7f621806e510 2024-04-26T23:37:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.981+0000 7f621d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f62180732c0 0x7f6218073710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40298/0 (socket says 172.21.15.46:40298) 2024-04-26T23:37:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.981+0000 7f621d000640 1 -- 172.21.15.46:0/1403005507 learned_addr learned my addr 172.21.15.46:0/1403005507 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.981+0000 7f6217e00640 1 --2- 172.21.15.46:0/1403005507 >> v2:172.21.15.142:3300/0 conn(0x7f621806e510 0x7f621806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.981+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685425253 0 0) 0x7f621806ee30 con 0x7f62180732c0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.981+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61fc003520 con 0x7f62180732c0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3751409571 0 0) 0x7f61fc003520 con 0x7f62180732c0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 >> v2:172.21.15.142:3300/0 conn(0x7f621806e510 msgr2=0x7f621806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 --2- 172.21.15.46:0/1403005507 >> v2:172.21.15.142:3300/0 conn(0x7f621806e510 0x7f621806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 >> v1:172.21.15.142:6789/0 conn(0x7f6218160bd0 legacy=0x7f6218161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6218072260 con 0x7f62180732c0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f62100019f0 con 0x7f62180732c0 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 >> v1:172.21.15.46:6789/0 conn(0x7f62180732c0 legacy=0x7f6218073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc003a00 0x7f61fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f61fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61fc0051c0 0x7f61fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6218071f80 con 0x7f61fc004800 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc00c0c0 con 0x7f61fc0051c0 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc00c2c0 con 0x7f61fc003a00 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f61fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217e00640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc003a00 0x7f61fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f621d000640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61fc0051c0 0x7f61fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 -- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc003a00 msgr2=0x7f61fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc003a00 0x7f61fc003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 -- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61fc0051c0 msgr2=0x7f61fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61fc0051c0 0x7f61fc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f621d000640 1 -- 172.21.15.46:0/1403005507 reap_dead start 2024-04-26T23:37:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.982+0000 7f6217400640 1 -- 172.21.15.46:0/1403005507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc003520 con 0x7f61fc004800 2024-04-26T23:37:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.983+0000 7f6217400640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f61fc004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6204001750 tx=0x7f61fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a2840a748c45e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204001370 con 0x7f61fc004800 2024-04-26T23:37:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62040052d0 con 0x7f61fc004800 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f6216a00640 1 -- 172.21.15.46:0/1403005507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620402abc0 con 0x7f61fc004800 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f621f000640 1 -- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 msgr2=0x7f61fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f621f000640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f61fc004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6204001750 tx=0x7f61fc005830 comp rx=0 tx=0).stop 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f621f000640 1 -- 172.21.15.46:0/1403005507 shutdown_connections 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f621f000640 1 --2- 172.21.15.46:0/1403005507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f61fc004c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.984+0000 7f621f000640 1 -- 172.21.15.46:0/1403005507 >> 172.21.15.46:0/1403005507 conn(0x7f621806c530 msgr2=0x7f621806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.985+0000 7f621f000640 1 -- 172.21.15.46:0/1403005507 shutdown_connections 2024-04-26T23:37:14.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.985+0000 7f621f000640 1 -- 172.21.15.46:0/1403005507 wait complete. 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.986+0000 7f621f000640 1 Processor -- start 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.986+0000 7f621f000640 1 -- start start 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6218154890 0x7f62181540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 0x7f621813dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6218072d40 con 0x7f61fc004800 2024-04-26T23:37:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6218072820 con 0x7f62180b1dc0 2024-04-26T23:37:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.987+0000 7f621f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6218072ec0 con 0x7f6218154890 2024-04-26T23:37:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57006/0 (socket says 172.21.15.46:57006) 2024-04-26T23:37:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 -- 172.21.15.46:0/4061589471 learned_addr learned my addr 172.21.15.46:0/4061589471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217400640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6218154890 0x7f62181540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f621d000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 0x7f621813dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 -- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6218154890 msgr2=0x7f62181540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6218154890 0x7f62181540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 -- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 msgr2=0x7f621813dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 0x7f621813dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 -- 172.21.15.46:0/4061589471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62181545e0 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f621d000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 0x7f621813dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.988+0000 7f6217e00640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62080107c0 tx=0x7f6208010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.989+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620800f040 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.989+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620800eb00 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.989+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620801c400 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.989+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621813b900 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.989+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621813c290 con 0x7f61fc004800 2024-04-26T23:37:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.990+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218069530 con 0x7f61fc004800 2024-04-26T23:37:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.991+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f620801c5a0 con 0x7f61fc004800 2024-04-26T23:37:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.992+0000 7f620fe00640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.992+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f620808c430 con 0x7f61fc004800 2024-04-26T23:37:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.992+0000 7f6217400640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.993+0000 7f6217400640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 0x7f61f806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6204032000 tx=0x7f620402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:14.996+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6208018030 con 0x7f61fc004800 2024-04-26T23:37:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.200+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62180cf1a0 con 0x7f61fc004800 2024-04-26T23:37:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.202+0000 7f620fe00640 1 -- 172.21.15.46:0/4061589471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f6208059e30 con 0x7f61fc004800 2024-04-26T23:37:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:15.206 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 msgr2=0x7f61f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 0x7f61f806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6204032000 tx=0x7f620402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 msgr2=0x7f62181f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62080107c0 tx=0x7f6208010c80 comp rx=0 tx=0).stop 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 shutdown_connections 2024-04-26T23:37:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62180b1dc0 0x7f621813dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6218154890 0x7f62181540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 --2- 172.21.15.46:0/4061589471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61fc004800 0x7f62181f92a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 >> 172.21.15.46:0/4061589471 conn(0x7f621806c530 msgr2=0x7f621806e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 shutdown_connections 2024-04-26T23:37:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.205+0000 7f621f000640 1 -- 172.21.15.46:0/4061589471 wait complete. 2024-04-26T23:37:15.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.441+0000 7fa3c4a00640 1 Processor -- start 2024-04-26T23:37:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.441+0000 7fa3c4a00640 1 -- start start 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3c4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3c0142280 0x7fa3c006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3c4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c00b43f0 con 0x7fa3c00b3e80 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3c4a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c00b46d0 con 0x7fa3c006ea50 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3c4a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c006eea0 con 0x7fa3c0142280 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3bde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3c0142280 0x7fa3c006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3bde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3c0142280 0x7fa3c006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57012/0 (socket says 172.21.15.46:57012) 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3bd400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3c00b3e80 0x7fa3c0144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40312/0 (socket says 172.21.15.46:40312) 2024-04-26T23:37:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.443+0000 7fa3bde00640 1 -- 172.21.15.46:0/209062774 learned_addr learned my addr 172.21.15.46:0/209062774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.444+0000 7fa3bde00640 1 -- 172.21.15.46:0/209062774 >> v1:172.21.15.142:6789/0 conn(0x7fa3c006ea50 legacy=0x7fa3c0144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.444+0000 7fa3bde00640 1 -- 172.21.15.46:0/209062774 >> v1:172.21.15.46:6789/0 conn(0x7fa3c00b3e80 legacy=0x7fa3c0144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.444+0000 7fa3bde00640 1 -- 172.21.15.46:0/209062774 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c00b49b0 con 0x7fa3c0142280 2024-04-26T23:37:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.446+0000 7fa3bde00640 1 --2- 172.21.15.46:0/209062774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c006e510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009870 tx=0x7fa3b002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e19918ba90d2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.446+0000 7fa3bca00640 1 -- 172.21.15.46:0/209062774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0037070 con 0x7fa3c0142280 2024-04-26T23:37:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.446+0000 7fa3bca00640 1 -- 172.21.15.46:0/209062774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b002ae50 con 0x7fa3c0142280 2024-04-26T23:37:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.446+0000 7fa3bca00640 1 -- 172.21.15.46:0/209062774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0032860 con 0x7fa3c0142280 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 -- 172.21.15.46:0/209062774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 msgr2=0x7fa3c006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/209062774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c006e510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009870 tx=0x7fa3b002a300 comp rx=0 tx=0).stop 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 -- 172.21.15.46:0/209062774 shutdown_connections 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/209062774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c006e510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 -- 172.21.15.46:0/209062774 >> 172.21.15.46:0/209062774 conn(0x7fa3c006c530 msgr2=0x7fa3c006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 -- 172.21.15.46:0/209062774 shutdown_connections 2024-04-26T23:37:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.447+0000 7fa3c4a00640 1 -- 172.21.15.46:0/209062774 wait complete. 2024-04-26T23:37:15.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.449+0000 7fa3c4a00640 1 Processor -- start 2024-04-26T23:37:15.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.449+0000 7fa3c4a00640 1 -- start start 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3c006ea50 0x7fa3c0158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 0x7fa3c00ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0145660 con 0x7fa3c0142280 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c00b4f70 con 0x7fa3c00b3e80 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.450+0000 7fa3c4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c01428a0 con 0x7fa3c006ea50 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57020/0 (socket says 172.21.15.46:57020) 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 -- 172.21.15.46:0/36372883 learned_addr learned my addr 172.21.15.46:0/36372883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3bd400640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 0x7fa3c00ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 -- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3c006ea50 msgr2=0x7fa3c0158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3c006ea50 0x7fa3c0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 -- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 msgr2=0x7fa3c00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 0x7fa3c00ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3be800640 1 -- 172.21.15.46:0/36372883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c00cf350 con 0x7fa3c0142280 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.451+0000 7fa3bd400640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 0x7fa3c00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3be800640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c6b0 tx=0x7fa3b800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8003b70 con 0x7fa3c0142280 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b8003d30 con 0x7fa3c0142280 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8005310 con 0x7fa3c0142280 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c00cf630 con 0x7fa3c0142280 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.452+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01589b0 con 0x7fa3c0142280 2024-04-26T23:37:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.453+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0069530 con 0x7fa3c0142280 2024-04-26T23:37:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.454+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3b80054b0 con 0x7fa3c0142280 2024-04-26T23:37:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.455+0000 7fa3b6000640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.455+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3b808bd00 con 0x7fa3c0142280 2024-04-26T23:37:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.456+0000 7fa3bde00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.457+0000 7fa3bde00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 0x7fa3a006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0034000 tx=0x7fa3b002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.459+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3b8059de0 con 0x7fa3c0142280 2024-04-26T23:37:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.705+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3c0073f70 con 0x7fa3c0142280 2024-04-26T23:37:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.706+0000 7fa3b6000640 1 -- 172.21.15.46:0/36372883 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7fa3b80596e0 con 0x7fa3c0142280 2024-04-26T23:37:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:15.709 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:13.283993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 msgr2=0x7fa3a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 0x7fa3a006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0034000 tx=0x7fa3b002c000 comp rx=0 tx=0).stop 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 msgr2=0x7fa3c01f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c6b0 tx=0x7fa3b800cb70 comp rx=0 tx=0).stop 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 shutdown_connections 2024-04-26T23:37:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c0142280 0x7fa3c01f4fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c00b3e80 0x7fa3c00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 --2- 172.21.15.46:0/36372883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3c006ea50 0x7fa3c0158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 >> 172.21.15.46:0/36372883 conn(0x7fa3c006c530 msgr2=0x7fa3c006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 shutdown_connections 2024-04-26T23:37:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.709+0000 7fa3c4a00640 1 -- 172.21.15.46:0/36372883 wait complete. 2024-04-26T23:37:15.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.932+0000 7fae86a00640 1 Processor -- start 2024-04-26T23:37:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.932+0000 7fae86a00640 1 -- start start 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae86a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae800b5140 0x7fae800d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae86a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae800cfd60 con 0x7fae800d5410 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae86a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae800d58c0 con 0x7fae8015f4b0 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae86a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fae8015f890 con 0x7fae800b5140 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae7f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae800b5140 0x7fae800d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae84a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fae800d5410 0x7fae800d2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40328/0 (socket says 172.21.15.46:40328) 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae7f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae800b5140 0x7fae800d4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57024/0 (socket says 172.21.15.46:57024) 2024-04-26T23:37:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.934+0000 7fae84a00640 1 -- 172.21.15.46:0/2005481692 learned_addr learned my addr 172.21.15.46:0/2005481692 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.935+0000 7fae7ea00640 1 -- 172.21.15.46:0/2005481692 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131933579 0 0) 0x7fae78001490 con 0x7fae800d5410 2024-04-26T23:37:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.935+0000 7fae7ea00640 1 -- 172.21.15.46:0/2005481692 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae800d5410 2024-04-26T23:37:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.935+0000 7fae7f400640 1 -- 172.21.15.46:0/2005481692 >> v1:172.21.15.142:6789/0 conn(0x7fae8015f4b0 legacy=0x7fae800d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.935+0000 7fae7f400640 1 -- 172.21.15.46:0/2005481692 >> v1:172.21.15.46:6789/0 conn(0x7fae800d5410 legacy=0x7fae800d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.935+0000 7fae7f400640 1 -- 172.21.15.46:0/2005481692 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae800b55d0 con 0x7fae800b5140 2024-04-26T23:37:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.936+0000 7fae7f400640 1 --2- 172.21.15.46:0/2005481692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae800b5140 0x7fae800d4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c205371ec0c35ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.936+0000 7fae7ea00640 1 -- 172.21.15.46:0/2005481692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037070 con 0x7fae800b5140 2024-04-26T23:37:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.936+0000 7fae7ea00640 1 -- 172.21.15.46:0/2005481692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c032440 con 0x7fae800b5140 2024-04-26T23:37:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.936+0000 7fae7ea00640 1 -- 172.21.15.46:0/2005481692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c032810 con 0x7fae800b5140 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 -- 172.21.15.46:0/2005481692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae800b5140 msgr2=0x7fae800d4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 --2- 172.21.15.46:0/2005481692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae800b5140 0x7fae800d4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a230 comp rx=0 tx=0).stop 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 -- 172.21.15.46:0/2005481692 shutdown_connections 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 --2- 172.21.15.46:0/2005481692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae800b5140 0x7fae800d4ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 -- 172.21.15.46:0/2005481692 >> 172.21.15.46:0/2005481692 conn(0x7fae8008eb50 msgr2=0x7fae8009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 -- 172.21.15.46:0/2005481692 shutdown_connections 2024-04-26T23:37:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.937+0000 7fae86a00640 1 -- 172.21.15.46:0/2005481692 wait complete. 2024-04-26T23:37:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.939+0000 7fae86a00640 1 Processor -- start 2024-04-26T23:37:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.939+0000 7fae86a00640 1 -- start start 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae800d5410 0x7fae8015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae80161e20 0x7fae800ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae800b5e70 con 0x7fae8015f4b0 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae800b5b90 con 0x7fae800d5410 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae86a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae800b4680 con 0x7fae80161e20 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57036/0 (socket says 172.21.15.46:57036) 2024-04-26T23:37:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.940+0000 7fae7f400640 1 -- 172.21.15.46:0/3287624403 learned_addr learned my addr 172.21.15.46:0/3287624403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 -- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae80161e20 msgr2=0x7fae800ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae80161e20 0x7fae800ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 -- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae800d5410 msgr2=0x7fae8015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7fe00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae800d5410 0x7fae8015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae800d5410 0x7fae8015af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 -- 172.21.15.46:0/3287624403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae801f4f30 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae7f400640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c041070 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c03c870 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae801f51f0 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.941+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037040 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.942+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae801f57b0 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.943+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae80069530 con 0x7fae8015f4b0 2024-04-26T23:37:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.944+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae6c044070 con 0x7fae8015f4b0 2024-04-26T23:37:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.945+0000 7fae77e00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 0x7fae6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.945+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fae6c0ac850 con 0x7fae8015f4b0 2024-04-26T23:37:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.946+0000 7fae7fe00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 0x7fae6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.947+0000 7fae7fe00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 0x7fae6006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae80163900 tx=0x7fae70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:15.950+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae6c07a930 con 0x7fae8015f4b0 2024-04-26T23:37:16.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.155+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae800b4bd0 con 0x7fae8015f4b0 2024-04-26T23:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.158+0000 7fae77e00640 1 -- 172.21.15.46:0/3287624403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fae6c07a230 con 0x7fae8015f4b0 2024-04-26T23:37:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:16.162 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 msgr2=0x7fae6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 0x7fae6006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae80163900 tx=0x7fae70009040 comp rx=0 tx=0).stop 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 msgr2=0x7fae801f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 shutdown_connections 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6006c8f0 0x7fae6006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae80161e20 0x7fae800ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8015f4b0 0x7fae801f8fc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 --2- 172.21.15.46:0/3287624403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae800d5410 0x7fae8015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 >> 172.21.15.46:0/3287624403 conn(0x7fae8008eb50 msgr2=0x7fae8009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 shutdown_connections 2024-04-26T23:37:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.161+0000 7fae86a00640 1 -- 172.21.15.46:0/3287624403 wait complete. 2024-04-26T23:37:16.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:37:08.251922+0000', 'modified': '2024-04-26T23:37:13.283993+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6836/2711334262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2711334262}, {'type': 'v1', 'addr': '172.21.15.142:6836', 'nonce': 2711334262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-26T23:37:16.588 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:37:16.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:37:16.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.824+0000 7f4e87e00640 1 Processor -- start 2024-04-26T23:37:16.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.824+0000 7f4e87e00640 1 -- start start 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e88160910 0x7f4e88160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e88073ad0 con 0x7f4e881466b0 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e88146a90 con 0x7f4e880720e0 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4e881612a0 con 0x7f4e88160910 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e88160910 0x7f4e88160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e88160910 0x7f4e88160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57042/0 (socket says 172.21.15.46:57042) 2024-04-26T23:37:16.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.826+0000 7f4e87400640 1 -- 172.21.15.46:0/2840328186 learned_addr learned my addr 172.21.15.46:0/2840328186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.827+0000 7f4e87400640 1 -- 172.21.15.46:0/2840328186 >> v1:172.21.15.142:6789/0 conn(0x7f4e880720e0 legacy=0x7f4e880cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.827+0000 7f4e87400640 1 -- 172.21.15.46:0/2840328186 >> v1:172.21.15.46:6789/0 conn(0x7f4e881466b0 legacy=0x7f4e8806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.827+0000 7f4e87400640 1 -- 172.21.15.46:0/2840328186 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e88073d70 con 0x7f4e88160910 2024-04-26T23:37:16.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.827+0000 7f4e85600640 1 -- 172.21.15.46:0/2840328186 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225626677 0 0) 0x7f4e88073ad0 con 0x7f4e881466b0 2024-04-26T23:37:16.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.828+0000 7f4e87400640 1 --2- 172.21.15.46:0/2840328186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e88160910 0x7f4e88160d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009e80 tx=0x7f4e8002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce0ee3bb2f166df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.828+0000 7f4e85600640 1 -- 172.21.15.46:0/2840328186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80036070 con 0x7f4e88160910 2024-04-26T23:37:16.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.828+0000 7f4e85600640 1 -- 172.21.15.46:0/2840328186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80031600 con 0x7f4e88160910 2024-04-26T23:37:16.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.828+0000 7f4e85600640 1 -- 172.21.15.46:0/2840328186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800319d0 con 0x7f4e88160910 2024-04-26T23:37:16.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 -- 172.21.15.46:0/2840328186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e88160910 msgr2=0x7f4e88160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 --2- 172.21.15.46:0/2840328186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e88160910 0x7f4e88160d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009e80 tx=0x7f4e8002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 -- 172.21.15.46:0/2840328186 shutdown_connections 2024-04-26T23:37:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 --2- 172.21.15.46:0/2840328186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e88160910 0x7f4e88160d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 -- 172.21.15.46:0/2840328186 >> 172.21.15.46:0/2840328186 conn(0x7f4e880b26d0 msgr2=0x7f4e880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 -- 172.21.15.46:0/2840328186 shutdown_connections 2024-04-26T23:37:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.829+0000 7f4e87e00640 1 -- 172.21.15.46:0/2840328186 wait complete. 2024-04-26T23:37:16.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.831+0000 7f4e87e00640 1 Processor -- start 2024-04-26T23:37:16.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.831+0000 7f4e87e00640 1 -- start start 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e880720e0 0x7f4e880b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e88160910 0x7f4e880c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880cb470 con 0x7f4e881466b0 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880cb190 con 0x7f4e880720e0 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e87e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e880cb990 con 0x7f4e88160910 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57046/0 (socket says 172.21.15.46:57046) 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.832+0000 7f4e86000640 1 -- 172.21.15.46:0/860789243 learned_addr learned my addr 172.21.15.46:0/860789243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e87400640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e88160910 0x7f4e880c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 -- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e88160910 msgr2=0x7f4e880c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e88160910 0x7f4e880c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 -- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e880720e0 msgr2=0x7f4e880b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e880720e0 0x7f4e880b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 -- 172.21.15.46:0/860789243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e880c9bd0 con 0x7f4e881466b0 2024-04-26T23:37:16.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.833+0000 7f4e86000640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c430 tx=0x7f4e7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.834+0000 7f4e7f400640 1 -- 172.21.15.46:0/860789243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011790 con 0x7f4e881466b0 2024-04-26T23:37:16.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.834+0000 7f4e7f400640 1 -- 172.21.15.46:0/860789243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70017070 con 0x7f4e881466b0 2024-04-26T23:37:16.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.834+0000 7f4e7f400640 1 -- 172.21.15.46:0/860789243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011bb0 con 0x7f4e881466b0 2024-04-26T23:37:16.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.834+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e880b1c50 con 0x7f4e881466b0 2024-04-26T23:37:16.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.834+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e880b2210 con 0x7f4e881466b0 2024-04-26T23:37:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.836+0000 7f4e7f400640 1 -- 172.21.15.46:0/860789243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e7001c460 con 0x7f4e881466b0 2024-04-26T23:37:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.837+0000 7f4e7f400640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 0x7f4e6406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.837+0000 7f4e7f400640 1 -- 172.21.15.46:0/860789243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e7008bd00 con 0x7f4e881466b0 2024-04-26T23:37:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.838+0000 7f4e90922740 1 Processor -- start 2024-04-26T23:37:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.838+0000 7f4e86a00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 0x7f4e6406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.838+0000 7f4e90922740 1 -- start start 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87ad5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e86a00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 0x7f4e6406ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001600 tx=0x7f4e78007910 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e87a37600 con 0x556e87b0fe80 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e879d4100 con 0x556e87afe270 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556e87a335c0 con 0x556e87ad37b0 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57050/0 (socket says 172.21.15.46:57050) 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.839+0000 7f4e86000640 1 -- 172.21.15.46:0/2568541397 learned_addr learned my addr 172.21.15.46:0/2568541397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e87400640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86a00640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87ad5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 -- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 msgr2=0x556e87ad5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87ad5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 -- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 msgr2=0x556e87afe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 -- 172.21.15.46:0/2568541397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e877492d0 con 0x556e87b0fe80 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e87400640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e86000640 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e70055bc0 tx=0x7f4e70059f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37993af07b74d759 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e77400640 1 -- 172.21.15.46:0/2568541397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7006f070 con 0x556e87b0fe80 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e77400640 1 -- 172.21.15.46:0/2568541397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70062340 con 0x556e87b0fe80 2024-04-26T23:37:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.840+0000 7f4e77400640 1 -- 172.21.15.46:0/2568541397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011d50 con 0x556e87b0fe80 2024-04-26T23:37:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 -- 172.21.15.46:0/2568541397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 msgr2=0x556e87ad3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e70055bc0 tx=0x7f4e70059f20 comp rx=0 tx=0).stop 2024-04-26T23:37:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 -- 172.21.15.46:0/2568541397 shutdown_connections 2024-04-26T23:37:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87ad3b90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 --2- 172.21.15.46:0/2568541397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87ad5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 -- 172.21.15.46:0/2568541397 >> 172.21.15.46:0/2568541397 conn(0x556e87a67d10 msgr2=0x556e87b19270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 -- 172.21.15.46:0/2568541397 shutdown_connections 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.841+0000 7f4e90922740 1 -- 172.21.15.46:0/2568541397 wait complete. 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.842+0000 7f4e90922740 1 -- start start 2024-04-26T23:37:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.842+0000 7f4e90922740 1 Processor -- start 2024-04-26T23:37:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e879bef70 con 0x556e87b0fe80 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e87afbdc0 con 0x556e87afe270 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556e87b2c8a0 con 0x556e87ad37b0 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e90922740 10 client.0 _ll_register_callbacks cb 0x556e87a2d480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57052/0 (socket says 172.21.15.46:57052) 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.843+0000 7f4e87400640 1 -- 172.21.15.46:0/3927843085 learned_addr learned my addr 172.21.15.46:0/3927843085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e86a00640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e86000640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 -- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 msgr2=0x556e87b29930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b29930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 -- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 msgr2=0x556e87afaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 -- 172.21.15.46:0/3927843085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e878136b0 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e86000640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e87400640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e800093e0 tx=0x7f4e80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccfd0b52be1a15c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e6be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80040030 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e800042f0 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.844+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e70062340 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f4e8004c050 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 10 client.4383 fetch_fsmap learned FSMap version 7 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556e87b29e70 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70062340 con 0x556e87b0fe80 2024-04-26T23:37:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f4e8004f020 con 0x556e87b0fe80 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 10 client.4383 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 10 client.4383 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 20 client.4383 populate_metadata read hostname 'smithi046' 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87b2b9d0 0x556e87b2bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 4 client.4383 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.845+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 --> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e87a45210 con 0x556e87b2b9d0 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.846+0000 7f4e86a00640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87b2b9d0 0x556e87b2bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.847+0000 7f4e86a00640 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87b2b9d0 0x556e87b2bdb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b4d9c2bdb6113c server_cookie=cf7c6cd9a5ae28b0 in_seq=0 out_seq=0 2024-04-26T23:37:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.847+0000 7f4e6be00640 10 client.4383 ms_handle_connect on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.849+0000 7f4e6be00640 1 -- 172.21.15.46:0/3927843085 <== mds.0 v2:172.21.15.142:6834/2711334262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e87a45210 con 0x556e87b2b9d0 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.849+0000 7f4e6be00640 10 client.4383 handle_command_reply: tid=0 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.849+0000 7f4e90922740 1 client.4383 shutdown 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.849+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 msgr2=0x556e87afb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e800093e0 tx=0x7f4e80004ab0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 shutdown_connections 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87b2b9d0 0x556e87b2bdb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87b0fe80 0x556e87afb470 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87afe270 0x556e87afaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 --2- 172.21.15.46:0/3927843085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b29930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e6be00640 0 client.4383 ms_handle_reset on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 >> 172.21.15.46:0/3927843085 conn(0x556e87a67d10 msgr2=0x556e87afbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 shutdown_connections 2024-04-26T23:37:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 1 -- 172.21.15.46:0/3927843085 wait complete. 2024-04-26T23:37:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.850+0000 7f4e90922740 20 client.4383 trim_cache size 0 max 16384 2024-04-26T23:37:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.870+0000 7f4e90922740 1 Processor -- start 2024-04-26T23:37:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.870+0000 7f4e90922740 1 -- start start 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b00ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b0fe80 0x556e87ad2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e882019c0 con 0x556e87afe270 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880cfe40 con 0x556e87b0fe80 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e88201370 con 0x556e87ad37b0 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57054/0 (socket says 172.21.15.46:57054) 2024-04-26T23:37:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.871+0000 7f4e86000640 1 -- 172.21.15.46:0/3138117300 learned_addr learned my addr 172.21.15.46:0/3138117300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86a00640 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b00ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 -- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 msgr2=0x556e87b00ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b00ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 -- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b0fe80 msgr2=0x556e87ad2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b0fe80 0x556e87ad2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 -- 172.21.15.46:0/3138117300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e87b134b0 con 0x556e87afe270 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e86000640 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000a040 tx=0x7f4e7006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d730339b5e5aeaab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.872+0000 7f4e76a00640 1 -- 172.21.15.46:0/3138117300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7007c030 con 0x556e87afe270 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e76a00640 1 -- 172.21.15.46:0/3138117300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70062340 con 0x556e87afe270 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e76a00640 1 -- 172.21.15.46:0/3138117300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70055a00 con 0x556e87afe270 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 -- 172.21.15.46:0/3138117300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 msgr2=0x556e87ad1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000a040 tx=0x7f4e7006f000 comp rx=0 tx=0).stop 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 -- 172.21.15.46:0/3138117300 shutdown_connections 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b0fe80 0x556e87ad2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87afe270 0x556e87ad1de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 --2- 172.21.15.46:0/3138117300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87ad37b0 0x556e87b00ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 -- 172.21.15.46:0/3138117300 >> 172.21.15.46:0/3138117300 conn(0x556e87a67d10 msgr2=0x556e87afdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 -- 172.21.15.46:0/3138117300 shutdown_connections 2024-04-26T23:37:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.873+0000 7f4e90922740 1 -- 172.21.15.46:0/3138117300 wait complete. 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.874+0000 7f4e90922740 1 -- start start 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.874+0000 7f4e90922740 1 Processor -- start 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87b0fe80 0x556e87b93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b2adf0 0x556e87b19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e87b00740 con 0x556e87ad37b0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e87b2c630 con 0x556e87b2adf0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556e87afeea0 con 0x556e87b0fe80 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e90922740 10 client.0 _ll_register_callbacks cb 0x7f4e80034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.875+0000 7f4e86a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57064/0 (socket says 172.21.15.46:57064) 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 -- 172.21.15.46:0/1089773157 learned_addr learned my addr 172.21.15.46:0/1089773157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86000640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87b0fe80 0x556e87b93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 -- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87b0fe80 msgr2=0x556e87b93480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87b0fe80 0x556e87b93480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e87400640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b2adf0 0x556e87b19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 -- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b2adf0 msgr2=0x556e87b19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b2adf0 0x556e87b19300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 -- 172.21.15.46:0/1089773157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e87b17f10 con 0x556e87ad37b0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e86a00640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001e90 tx=0x7f4e780010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a0398071c1205f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.876+0000 7f4e6be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:37:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78023450 con 0x556e87ad37b0 2024-04-26T23:37:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e780235f0 con 0x556e87ad37b0 2024-04-26T23:37:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e70062340 con 0x556e87ad37b0 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f4e78031050 con 0x556e87ad37b0 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 10 client.4389 fetch_fsmap learned FSMap version 7 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556e87b18c10 con 0x556e87ad37b0 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70062340 con 0x556e87ad37b0 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f4e78034020 con 0x556e87ad37b0 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 10 client.4389 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 10 client.4389 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:37:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 20 client.4389 populate_metadata read hostname 'smithi046' 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87ad2460 0x556e87ad2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 4 client.4389 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.877+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 --> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] -- command(tid 0: {"prefix": "status"}) -- 0x556e87ad4a00 con 0x556e87ad2460 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.878+0000 7f4e86000640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87ad2460 0x556e87ad2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.879+0000 7f4e86000640 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87ad2460 0x556e87ad2840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e009450d814399 server_cookie=197254545c33f9ae in_seq=0 out_seq=0 2024-04-26T23:37:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.879+0000 7f4e6be00640 10 client.4389 ms_handle_connect on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e6be00640 1 -- 172.21.15.46:0/1089773157 <== mds.0 v2:172.21.15.142:6834/2711334262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x556e87ad4a00 con 0x556e87ad2460 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e6be00640 10 client.4389 handle_command_reply: tid=0 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 client.4389 shutdown 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 msgr2=0x556e87ac9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001e90 tx=0x7f4e780010f0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 shutdown_connections 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x556e87ad2460 0x556e87ad2840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e87b2adf0 0x556e87b19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556e87b0fe80 0x556e87b93480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 --2- 172.21.15.46:0/1089773157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e87ad37b0 0x556e87ac9df0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e6be00640 0 client.4389 ms_handle_reset on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 >> 172.21.15.46:0/1089773157 conn(0x556e87a67d10 msgr2=0x556e87b293e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 shutdown_connections 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.880+0000 7f4e90922740 1 -- 172.21.15.46:0/1089773157 wait complete. 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e90922740 20 client.4389 trim_cache size 0 max 16384 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4285, 2024-04-26T23:37:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:37:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:37:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:37:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 8.1185714749999995, 2024-04-26T23:37:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 7, 2024-04-26T23:37:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 25, 2024-04-26T23:37:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T23:37:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 16.024047060000001 2024-04-26T23:37:16.887 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 msgr2=0x7f4e6406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 0x7f4e6406ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001600 tx=0x7f4e78007910 comp rx=0 tx=0).stop 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 msgr2=0x7f4e880ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c430 tx=0x7f4e7000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 shutdown_connections 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e6406c940 0x7f4e6406ee00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e88160910 0x7f4e880c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e881466b0 0x7f4e880ca390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 --2- 172.21.15.46:0/860789243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e880720e0 0x7f4e880b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 >> 172.21.15.46:0/860789243 conn(0x7f4e880b26d0 msgr2=0x7f4e880cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 shutdown_connections 2024-04-26T23:37:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:16.881+0000 7f4e87e00640 1 -- 172.21.15.46:0/860789243 wait complete. 2024-04-26T23:37:16.901 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:37:16.901 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:16.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:37:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.093+0000 7f49dda00640 1 Processor -- start 2024-04-26T23:37:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.093+0000 7f49dda00640 1 -- start start 2024-04-26T23:37:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.094+0000 7f49dda00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49d8073570 0x7f49d80d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.094+0000 7f49dda00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80cf950 con 0x7f49d815c5c0 2024-04-26T23:37:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.094+0000 7f49dda00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80cfc30 con 0x7f49d8145770 2024-04-26T23:37:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.094+0000 7f49dda00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80d0c50 con 0x7f49d8073570 2024-04-26T23:37:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.095+0000 7f49d7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f49d815c5c0 0x7f49d80b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40344/0 (socket says 172.21.15.46:40344) 2024-04-26T23:37:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.095+0000 7f49d7400640 1 -- 172.21.15.46:0/791886636 learned_addr learned my addr 172.21.15.46:0/791886636 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.095+0000 7f49d6000640 1 --2- 172.21.15.46:0/791886636 >> v2:172.21.15.142:3300/0 conn(0x7f49d8073570 0x7f49d80d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.095+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970959949 0 0) 0x7f49d80cf950 con 0x7f49d815c5c0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.095+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49bc005730 con 0x7f49d815c5c0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3815265231 0 0) 0x7f49bc005730 con 0x7f49d815c5c0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 >> v2:172.21.15.142:3300/0 conn(0x7f49d8073570 msgr2=0x7f49d80d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 --2- 172.21.15.46:0/791886636 >> v2:172.21.15.142:3300/0 conn(0x7f49d8073570 0x7f49d80d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 >> v1:172.21.15.142:6789/0 conn(0x7f49d8145770 legacy=0x7f49d8072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80cff10 con 0x7f49d815c5c0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f49d0001670 con 0x7f49d815c5c0 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 >> v1:172.21.15.46:6789/0 conn(0x7f49d815c5c0 legacy=0x7f49d80b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc005bc0 0x7f49bc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc006a10 0x7f49bc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 0x7f49bc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80d0c50 con 0x7f49bc0073d0 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc00e270 con 0x7f49bc005bc0 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc00e470 con 0x7f49bc006a10 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d7400640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 0x7f49bc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d6a00640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc005bc0 0x7f49bc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.096+0000 7f49d6000640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc006a10 0x7f49bc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 -- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc006a10 msgr2=0x7f49bc006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc006a10 0x7f49bc006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 -- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc005bc0 msgr2=0x7f49bc005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc005bc0 0x7f49bc005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 -- 172.21.15.46:0/791886636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc005730 con 0x7f49bc0073d0 2024-04-26T23:37:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.097+0000 7f49d7400640 1 -- 172.21.15.46:0/791886636 reap_dead start 2024-04-26T23:37:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49d7400640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 0x7f49bc00dbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c240 tx=0x7f49bc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dc38512e7d86b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0007cf0 con 0x7f49bc0073d0 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d00058d0 con 0x7f49bc0073d0 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49d5600640 1 -- 172.21.15.46:0/791886636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0005ce0 con 0x7f49bc0073d0 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49dda00640 1 -- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 msgr2=0x7f49bc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49dda00640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 0x7f49bc00dbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c240 tx=0x7f49bc007a40 comp rx=0 tx=0).stop 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49dda00640 1 -- 172.21.15.46:0/791886636 shutdown_connections 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49dda00640 1 --2- 172.21.15.46:0/791886636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49bc0073d0 0x7f49bc00dbd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.098+0000 7f49dda00640 1 -- 172.21.15.46:0/791886636 >> 172.21.15.46:0/791886636 conn(0x7f49d806c530 msgr2=0x7f49d806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.099+0000 7f49dda00640 1 -- 172.21.15.46:0/791886636 shutdown_connections 2024-04-26T23:37:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.099+0000 7f49dda00640 1 -- 172.21.15.46:0/791886636 wait complete. 2024-04-26T23:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.100+0000 7f49dda00640 1 Processor -- start 2024-04-26T23:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 -- start start 2024-04-26T23:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0073d0 0x7f49d80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d80c6fd0 0x7f49d80c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80cf5b0 con 0x7f49d80c8410 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49d815d490 con 0x7f49bc0073d0 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.101+0000 7f49dda00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80cf730 con 0x7f49d80c6fd0 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57074/0 (socket says 172.21.15.46:57074) 2024-04-26T23:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 -- 172.21.15.46:0/1592358788 learned_addr learned my addr 172.21.15.46:0/1592358788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d6a00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0073d0 0x7f49d80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 -- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d80c6fd0 msgr2=0x7f49d80c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d6000640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d80c6fd0 0x7f49d80c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d80c6fd0 0x7f49d80c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 -- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0073d0 msgr2=0x7f49d80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0073d0 0x7f49d80cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.102+0000 7f49d7400640 1 -- 172.21.15.46:0/1592358788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80ae980 con 0x7f49d80c8410 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49d7400640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c330 tx=0x7f49d80d00e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0038040 con 0x7f49d80c8410 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0041070 con 0x7f49d80c8410 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004a00 con 0x7f49d80c8410 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d80aec60 con 0x7f49d80c8410 2024-04-26T23:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.103+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d80af220 con 0x7f49d80c8410 2024-04-26T23:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.104+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d8069530 con 0x7f49d80c8410 2024-04-26T23:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.105+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49d0004ba0 con 0x7f49d80c8410 2024-04-26T23:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.106+0000 7f49cea00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.106+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f49d00ab9e0 con 0x7f49d80c8410 2024-04-26T23:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.107+0000 7f49d6a00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.108+0000 7f49d6a00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.111+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49d0079a40 con 0x7f49d80c8410 2024-04-26T23:37:17.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.355+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f49d80b6060 con 0x7f49d80c8410 2024-04-26T23:37:17.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.356+0000 7f49cea00640 1 -- 172.21.15.46:0/1592358788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f49d0079340 con 0x7f49d80c8410 2024-04-26T23:37:17.359 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:37:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.358+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 msgr2=0x7f49b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.358+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4009040 comp rx=0 tx=0).stop 2024-04-26T23:37:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 msgr2=0x7f49d80afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c330 tx=0x7f49d80d00e0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 shutdown_connections 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49d80c8410 0x7f49d80afa20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d80c6fd0 0x7f49d80c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 --2- 172.21.15.46:0/1592358788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0073d0 0x7f49d80cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 >> 172.21.15.46:0/1592358788 conn(0x7f49d806c530 msgr2=0x7f49d80b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 shutdown_connections 2024-04-26T23:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17.359+0000 7f49dda00640 1 -- 172.21.15.46:0/1592358788 wait complete. 2024-04-26T23:37:17.380 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:37:17.381 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:37:17.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:17.381 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:37:17.428 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42584sec preferred_lft 42584sec 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:37:17.430 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:17.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:17.937 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:37:18.008 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:37:18.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:18.012 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:37:18.071 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:37:18.071 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:37:18.071 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:37:18.071 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:37:18.071 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:37:18.072 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:18.072 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:18.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:37:18.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:37:18.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:37:18.073 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:18.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:18.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:18.834 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:37:18.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:18.889 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-26T23:37:18.940 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:18.940 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:18.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:18.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T23:37:18.940 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:19.128 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T23:37:19.128 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T23:37:19.129 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:19.435 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:19.435 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:19.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:37:19.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:37:19.435 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:19.648 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T23:37:19.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:19.648 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:19.691 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:37:19.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:19.691 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:19.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-26T23:37:19.858 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:37:19.905 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-26T23:37:19.905 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-26T23:37:19.905 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-26T23:37:19.906 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-26T23:37:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:19.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:19.909 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-26T23:37:19.956 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T23:37:19.957 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T23:37:19.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-26T23:37:20.029 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:37:20.078 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:37:20.079 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T23:37:20.080 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T23:37:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T23:37:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-26T23:37:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:20.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:20.082 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-26T23:37:20.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-04-26T23:37:20.151+0000 7f1b8840b480 -1 init, newargv = 0x560bb9987240 newargc=15 2024-04-26T23:37:20.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[38700]: starting ceph client 2024-04-26T23:37:20.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[38700]: starting fuse 2024-04-26T23:37:21.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-26T23:37:21.324 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:37:21.383 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-26T23:37:21.383 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-26T23:37:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T23:37:21.385 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T23:37:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-26T23:37:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:21.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:21.387 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-26T23:37:21.439 INFO:teuthology.orchestra.run.smithi046.stdout:41 2024-04-26T23:37:21.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T23:37:21.440 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T23:37:21.441 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-04-26T23:37:21.442 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:37:21.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.38700.asok 2024-04-26T23:37:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:21.638 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.38700.asok 2024-04-26T23:37:21.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:21.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.38700.asok status 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-04-26T23:37:21.875 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "38700", 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4401, 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4401 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:37:21.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2489619293 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2489619293 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4401 192.168.0.1:0/2489619293", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2489619293", 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 7, 2024-04-26T23:37:21.877 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 25, 2024-04-26T23:37:21.878 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-04-26T23:37:21.878 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-04-26T23:37:21.878 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-04-26T23:37:21.878 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:37:21.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:21.891 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-04-26T23:37:21.891 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:21.947 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-04-26T23:37:21.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:21.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:21.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:22.040 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T23:37:22.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:22.202 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-04-26T23:37:22.202 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T23:37:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:22.213 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:37:22.213 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:37:22.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:22.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.477+0000 7fd3d1000640 1 Processor -- start 2024-04-26T23:37:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.477+0000 7fd3d1000640 1 -- start start 2024-04-26T23:37:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.478+0000 7fd3d1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.478+0000 7fd3d1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc0b4000 0x7fd3cc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.478+0000 7fd3d1000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3cc0d11f0 con 0x7fd3cc15f4e0 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.478+0000 7fd3d1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc0d0ce0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.478+0000 7fd3d1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc0d2990 con 0x7fd3cc0b4000 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48256/0 (socket says 172.21.15.46:48256) 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 -- 172.21.15.46:0/986026724 learned_addr learned my addr 172.21.15.46:0/986026724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3cbe00640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc0b4000 0x7fd3cc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 -- 172.21.15.46:0/986026724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc0b4000 msgr2=0x7fd3cc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc0b4000 0x7fd3cc0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 -- 172.21.15.46:0/986026724 >> v1:172.21.15.142:6789/0 conn(0x7fd3cc15f4e0 legacy=0x7fd3cc0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.479+0000 7fd3caa00640 1 -- 172.21.15.46:0/986026724 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc0d5f20 con 0x7fd3cc0d2130 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.480+0000 7fd3caa00640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009870 tx=0x7fd3b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbee3f1ddd40f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3ca000640 1 -- 172.21.15.46:0/986026724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4037070 con 0x7fd3cc0d2130 2024-04-26T23:37:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3ca000640 1 -- 172.21.15.46:0/986026724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b4032440 con 0x7fd3cc0d2130 2024-04-26T23:37:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3ca000640 1 -- 172.21.15.46:0/986026724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4032810 con 0x7fd3cc0d2130 2024-04-26T23:37:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 -- 172.21.15.46:0/986026724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 msgr2=0x7fd3cc0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009870 tx=0x7fd3b402a210 comp rx=0 tx=0).stop 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 -- 172.21.15.46:0/986026724 shutdown_connections 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc0b4000 0x7fd3cc0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 --2- 172.21.15.46:0/986026724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc0d2df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.481+0000 7fd3d1000640 1 -- 172.21.15.46:0/986026724 >> 172.21.15.46:0/986026724 conn(0x7fd3cc0d5010 msgr2=0x7fd3cc0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.482+0000 7fd3d1000640 1 -- 172.21.15.46:0/986026724 shutdown_connections 2024-04-26T23:37:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.482+0000 7fd3d1000640 1 -- 172.21.15.46:0/986026724 wait complete. 2024-04-26T23:37:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.483+0000 7fd3d1000640 1 Processor -- start 2024-04-26T23:37:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.483+0000 7fd3d1000640 1 -- start start 2024-04-26T23:37:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 0x7fd3cc0ba820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 0x7fd3cc0b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0b1d60 con 0x7fd3cc0d2130 2024-04-26T23:37:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0d5c40 con 0x7fd3cc0b4000 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.484+0000 7fd3d1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc15f8c0 con 0x7fd3cc15f4e0 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48268/0 (socket says 172.21.15.46:48268) 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 -- 172.21.15.46:0/363514351 learned_addr learned my addr 172.21.15.46:0/363514351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3cbe00640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 0x7fd3cc0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 -- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 msgr2=0x7fd3cc0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3cb400640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 0x7fd3cc0ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 0x7fd3cc0b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 -- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 msgr2=0x7fd3cc0ba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 0x7fd3cc0ba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3caa00640 1 -- 172.21.15.46:0/363514351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc1f56c0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3cbe00640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 0x7fd3cc0b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.485+0000 7fd3cb400640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 0x7fd3cc0ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3caa00640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009870 tx=0x7fd3b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4041020 con 0x7fd3cc0d2130 2024-04-26T23:37:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b40306c0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc0b8880 con 0x7fd3cc0d2130 2024-04-26T23:37:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4037040 con 0x7fd3cc0d2130 2024-04-26T23:37:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.486+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc0b8e40 con 0x7fd3cc0d2130 2024-04-26T23:37:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.487+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3cc0693b0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.488+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3b4044020 con 0x7fd3cc0d2130 2024-04-26T23:37:22.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.489+0000 7fd3c3400640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.489+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3b40ac6d0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.490+0000 7fd3cb400640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.491+0000 7fd3cb400640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 0x7fd3a806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007e90 tx=0x7fd3bc0073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.495+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3b404b030 con 0x7fd3cc0d2130 2024-04-26T23:37:22.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.700+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3cc0d46d0 con 0x7fd3cc0d2130 2024-04-26T23:37:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.701+0000 7fd3c3400640 1 -- 172.21.15.46:0/363514351 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fd3b407a720 con 0x7fd3cc0d2130 2024-04-26T23:37:22.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:22.705 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 msgr2=0x7fd3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 0x7fd3a806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007e90 tx=0x7fd3bc0073a0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 msgr2=0x7fd3cc1f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009870 tx=0x7fd3b40046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 shutdown_connections 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3cc15f4e0 0x7fd3cc0b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc0d2130 0x7fd3cc1f6880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 --2- 172.21.15.46:0/363514351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3cc0b4000 0x7fd3cc0ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 >> 172.21.15.46:0/363514351 conn(0x7fd3cc0d5010 msgr2=0x7fd3cc15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 shutdown_connections 2024-04-26T23:37:22.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.704+0000 7fd3d1000640 1 -- 172.21.15.46:0/363514351 wait complete. 2024-04-26T23:37:22.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:37:22.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:37:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.938+0000 7f56eb400640 1 Processor -- start 2024-04-26T23:37:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.938+0000 7f56eb400640 1 -- start start 2024-04-26T23:37:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.940+0000 7f56eb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.940+0000 7f56eb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56e406f310 0x7f56e40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.940+0000 7f56eb400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e415c790 con 0x7f56e4070000 2024-04-26T23:37:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.940+0000 7f56eb400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f56e406ddc0 con 0x7f56e406f310 2024-04-26T23:37:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.940+0000 7f56eb400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f56e406e0a0 con 0x7f56e406ee60 2024-04-26T23:37:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.941+0000 7f56e3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.941+0000 7f56e8a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f56e4070000 0x7f56e41557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:42506/0 (socket says 172.21.15.46:42506) 2024-04-26T23:37:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.941+0000 7f56e8a00640 1 -- 172.21.15.46:0/2799397193 learned_addr learned my addr 172.21.15.46:0/2799397193 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.941+0000 7f56e3e00640 1 --2- 172.21.15.46:0/2799397193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:56846/0 (socket says 172.21.15.46:56846) 2024-04-26T23:37:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.941+0000 7f56e9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56e406f310 0x7f56e40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.942+0000 7f56e3e00640 1 -- 172.21.15.46:0/2799397193 >> v2:172.21.15.46:3300/0 conn(0x7f56e406f310 msgr2=0x7f56e40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.942+0000 7f56e3e00640 1 --2- 172.21.15.46:0/2799397193 >> v2:172.21.15.46:3300/0 conn(0x7f56e406f310 0x7f56e40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.942+0000 7f56e3e00640 1 -- 172.21.15.46:0/2799397193 >> v1:172.21.15.142:6789/0 conn(0x7f56e4070000 legacy=0x7f56e41557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.942+0000 7f56e3e00640 1 -- 172.21.15.46:0/2799397193 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e406e380 con 0x7f56e406ee60 2024-04-26T23:37:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.943+0000 7f56e3e00640 1 --2- 172.21.15.46:0/2799397193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097a0 tx=0x7f56d002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b6720fbb1bdeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.951+0000 7f56e3400640 1 -- 172.21.15.46:0/2799397193 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0037070 con 0x7f56e406ee60 2024-04-26T23:37:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56e3400640 1 -- 172.21.15.46:0/2799397193 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d002ae40 con 0x7f56e406ee60 2024-04-26T23:37:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56e3400640 1 -- 172.21.15.46:0/2799397193 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0032660 con 0x7f56e406ee60 2024-04-26T23:37:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 -- 172.21.15.46:0/2799397193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 msgr2=0x7f56e40d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 --2- 172.21.15.46:0/2799397193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097a0 tx=0x7f56d002a0f0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 -- 172.21.15.46:0/2799397193 shutdown_connections 2024-04-26T23:37:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 --2- 172.21.15.46:0/2799397193 >> v2:172.21.15.46:3300/0 conn(0x7f56e406f310 0x7f56e40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 --2- 172.21.15.46:0/2799397193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406ee60 0x7f56e40d0110 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.952+0000 7f56eb400640 1 -- 172.21.15.46:0/2799397193 >> 172.21.15.46:0/2799397193 conn(0x7f56e406c980 msgr2=0x7f56e40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.953+0000 7f56eb400640 1 -- 172.21.15.46:0/2799397193 shutdown_connections 2024-04-26T23:37:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.953+0000 7f56eb400640 1 -- 172.21.15.46:0/2799397193 wait complete. 2024-04-26T23:37:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.954+0000 7f56eb400640 1 Processor -- start 2024-04-26T23:37:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 -- start start 2024-04-26T23:37:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406f310 0x7f56e414bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 0x7f56e414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56e40d0650 con 0x7f56e406ee60 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f56e406e940 con 0x7f56e4070000 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.955+0000 7f56eb400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f56e40d07d0 con 0x7f56e406f310 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48284/0 (socket says 172.21.15.46:48284) 2024-04-26T23:37:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 -- 172.21.15.46:0/3449543199 learned_addr learned my addr 172.21.15.46:0/3449543199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e9400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 0x7f56e414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e3e00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406f310 0x7f56e414bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 -- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406f310 msgr2=0x7f56e414bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406f310 0x7f56e414bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 -- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 msgr2=0x7f56e414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 0x7f56e414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e8a00640 1 -- 172.21.15.46:0/3449543199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e4153ed0 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.956+0000 7f56e9400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 0x7f56e414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56e8a00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c400 tx=0x7f56d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8017070 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d8005ce0 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8007310 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e41541a0 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.957+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e4154760 con 0x7f56e406ee60 2024-04-26T23:37:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.958+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e4069530 con 0x7f56e406ee60 2024-04-26T23:37:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.960+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56d801a070 con 0x7f56e406ee60 2024-04-26T23:37:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.961+0000 7f56e0c00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.961+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f56d808b310 con 0x7f56e406ee60 2024-04-26T23:37:22.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.961+0000 7f56e3e00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.962+0000 7f56e3e00640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 0x7f56c406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56d0034000 tx=0x7f56d002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:22.965+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56d8059360 con 0x7f56e406ee60 2024-04-26T23:37:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.170+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56e40b5c70 con 0x7f56e406ee60 2024-04-26T23:37:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.172+0000 7f56e0c00640 1 -- 172.21.15.46:0/3449543199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f56d8058c60 con 0x7f56e406ee60 2024-04-26T23:37:23.176 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:37:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 msgr2=0x7f56c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 0x7f56c406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56d0034000 tx=0x7f56d002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 msgr2=0x7f56e4153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c400 tx=0x7f56d800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 shutdown_connections 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e4070000 0x7f56e414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f56e406f310 0x7f56e414bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 --2- 172.21.15.46:0/3449543199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e406ee60 0x7f56e4153990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.174+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 >> 172.21.15.46:0/3449543199 conn(0x7f56e406c980 msgr2=0x7f56e40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.175+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 shutdown_connections 2024-04-26T23:37:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.175+0000 7f56eb400640 1 -- 172.21.15.46:0/3449543199 wait complete. 2024-04-26T23:37:23.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:37:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.398+0000 7fe8c5000640 1 Processor -- start 2024-04-26T23:37:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.398+0000 7fe8c5000640 1 -- start start 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 0x7fe8c006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c0160160 0x7fe8c00b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c00684f0 0x7fe8c00b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c015f5d0 con 0x7fe8c0160160 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c01605f0 con 0x7fe8c015f1f0 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8c5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c006e930 con 0x7fe8c00684f0 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c0160160 0x7fe8c00b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c0160160 0x7fe8c00b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48296/0 (socket says 172.21.15.46:48296) 2024-04-26T23:37:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 -- 172.21.15.46:0/1777863239 learned_addr learned my addr 172.21.15.46:0/1777863239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 -- 172.21.15.46:0/1777863239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c00684f0 msgr2=0x7fe8c00b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 --2- 172.21.15.46:0/1777863239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c00684f0 0x7fe8c00b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bea00640 1 -- 172.21.15.46:0/1777863239 >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 msgr2=0x7fe8c006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.400+0000 7fe8bf400640 1 --2- 172.21.15.46:0/1777863239 >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 0x7fe8c006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.401+0000 7fe8bea00640 1 --2- 172.21.15.46:0/1777863239 >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 0x7fe8c006f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.401+0000 7fe8bea00640 1 -- 172.21.15.46:0/1777863239 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c006ec10 con 0x7fe8c0160160 2024-04-26T23:37:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.401+0000 7fe8bf400640 1 --2- 172.21.15.46:0/1777863239 >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 0x7fe8c006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.402+0000 7fe8bea00640 1 --2- 172.21.15.46:0/1777863239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c0160160 0x7fe8c00b6570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8c006dfa0 tx=0x7fe8a802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8452e4dcab155083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.402+0000 7fe8be000640 1 -- 172.21.15.46:0/1777863239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8037070 con 0x7fe8c0160160 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.402+0000 7fe8be000640 1 -- 172.21.15.46:0/1777863239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a8032440 con 0x7fe8c0160160 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.402+0000 7fe8be000640 1 -- 172.21.15.46:0/1777863239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8032810 con 0x7fe8c0160160 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 -- 172.21.15.46:0/1777863239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c0160160 msgr2=0x7fe8c00b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 --2- 172.21.15.46:0/1777863239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c0160160 0x7fe8c00b6570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8c006dfa0 tx=0x7fe8a802a200 comp rx=0 tx=0).stop 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 -- 172.21.15.46:0/1777863239 shutdown_connections 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 --2- 172.21.15.46:0/1777863239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c00684f0 0x7fe8c00b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 --2- 172.21.15.46:0/1777863239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c0160160 0x7fe8c00b6570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 --2- 172.21.15.46:0/1777863239 >> v2:172.21.15.142:3300/0 conn(0x7fe8c015f1f0 0x7fe8c006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 -- 172.21.15.46:0/1777863239 >> 172.21.15.46:0/1777863239 conn(0x7fe8c00d13f0 msgr2=0x7fe8c00d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 -- 172.21.15.46:0/1777863239 shutdown_connections 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.403+0000 7fe8c5000640 1 -- 172.21.15.46:0/1777863239 wait complete. 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.405+0000 7fe8c5000640 1 Processor -- start 2024-04-26T23:37:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.405+0000 7fe8c5000640 1 -- start start 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c00684f0 0x7fe8c01f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 0x7fe8c006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00b49e0 con 0x7fe8c006ca20 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c006f1d0 con 0x7fe8c00684f0 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8c5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c00b4b60 con 0x7fe8c015f1f0 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48312/0 (socket says 172.21.15.46:48312) 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 -- 172.21.15.46:0/23641172 learned_addr learned my addr 172.21.15.46:0/23641172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bea00640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 0x7fe8c006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bf400640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c00684f0 0x7fe8c01f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 -- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 msgr2=0x7fe8c006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 0x7fe8c006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 -- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c00684f0 msgr2=0x7fe8c01f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c00684f0 0x7fe8c01f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bfe00640 1 -- 172.21.15.46:0/23641172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c01f4f90 con 0x7fe8c006ca20 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.406+0000 7fe8bea00640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 0x7fe8c006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8bfe00640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b3f0 tx=0x7fe8b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8015020 con 0x7fe8c006ca20 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8011830 con 0x7fe8c006ca20 2024-04-26T23:37:23.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c014c0e0 con 0x7fe8c006ca20 2024-04-26T23:37:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011c20 con 0x7fe8c006ca20 2024-04-26T23:37:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.407+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01f5240 con 0x7fe8c006ca20 2024-04-26T23:37:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.408+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c0069ac0 con 0x7fe8c006ca20 2024-04-26T23:37:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.409+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8b8018020 con 0x7fe8c006ca20 2024-04-26T23:37:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.410+0000 7fe8b7400640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 0x7fe89c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.410+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8b808bde0 con 0x7fe8c006ca20 2024-04-26T23:37:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.411+0000 7fe8bf400640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 0x7fe89c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.412+0000 7fe8bf400640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 0x7fe89c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00075b0 tx=0x7fe8b0007540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.415+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8b801f020 con 0x7fe8c006ca20 2024-04-26T23:37:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.620+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe8c006ceb0 con 0x7fe8c006ca20 2024-04-26T23:37:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.621+0000 7fe8b7400640 1 -- 172.21.15.46:0/23641172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fe8b80172c0 con 0x7fe8c006ca20 2024-04-26T23:37:23.624 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 msgr2=0x7fe89c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 0x7fe89c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00075b0 tx=0x7fe8b0007540 comp rx=0 tx=0).stop 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 msgr2=0x7fe8c01f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b3f0 tx=0x7fe8b800b8b0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 shutdown_connections 2024-04-26T23:37:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe89c06c920 0x7fe89c06ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c006ca20 0x7fe8c01f6430 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8c015f1f0 0x7fe8c006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 --2- 172.21.15.46:0/23641172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c00684f0 0x7fe8c01f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 >> 172.21.15.46:0/23641172 conn(0x7fe8c00d13f0 msgr2=0x7fe8c00b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 shutdown_connections 2024-04-26T23:37:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.624+0000 7fe8c5000640 1 -- 172.21.15.46:0/23641172 wait complete. 2024-04-26T23:37:23.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:37:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.853+0000 7f0129000640 1 Processor -- start 2024-04-26T23:37:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.853+0000 7f0129000640 1 -- start start 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0129000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0124146740 0x7f012415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0129000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01240d0330 0x7f01241612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0129000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0124146b20 con 0x7f01240ce670 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0129000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f01240ccf30 con 0x7f0124146740 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0129000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0124160af0 con 0x7f01240d0330 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0123400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0124146740 0x7f012415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0123400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0124146740 0x7f012415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48322/0 (socket says 172.21.15.46:48322) 2024-04-26T23:37:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f0123400640 1 -- 172.21.15.46:0/3846790782 learned_addr learned my addr 172.21.15.46:0/3846790782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.855+0000 7f011be00640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01240d0330 0x7f01241612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.856+0000 7f0123400640 1 -- 172.21.15.46:0/3846790782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01240d0330 msgr2=0x7f01241612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.856+0000 7f0123400640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01240d0330 0x7f01241612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.856+0000 7f0123400640 1 -- 172.21.15.46:0/3846790782 >> v1:172.21.15.142:6789/0 conn(0x7f01240ce670 legacy=0x7f01240ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.856+0000 7f0123400640 1 -- 172.21.15.46:0/3846790782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0124160dd0 con 0x7f0124146740 2024-04-26T23:37:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.857+0000 7f0123400640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0124146740 0x7f012415df90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0114009870 tx=0x7f011402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a168965955566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.857+0000 7f0122a00640 1 -- 172.21.15.46:0/3846790782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114037070 con 0x7f0124146740 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.857+0000 7f0122a00640 1 -- 172.21.15.46:0/3846790782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0114032440 con 0x7f0124146740 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.857+0000 7f0122a00640 1 -- 172.21.15.46:0/3846790782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114032810 con 0x7f0124146740 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 -- 172.21.15.46:0/3846790782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0124146740 msgr2=0x7f012415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0124146740 0x7f012415df90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0114009870 tx=0x7f011402a210 comp rx=0 tx=0).stop 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 -- 172.21.15.46:0/3846790782 shutdown_connections 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01240d0330 0x7f01241612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 --2- 172.21.15.46:0/3846790782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0124146740 0x7f012415df90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 -- 172.21.15.46:0/3846790782 >> 172.21.15.46:0/3846790782 conn(0x7f01240b26d0 msgr2=0x7f01240b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 -- 172.21.15.46:0/3846790782 shutdown_connections 2024-04-26T23:37:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.858+0000 7f0129000640 1 -- 172.21.15.46:0/3846790782 wait complete. 2024-04-26T23:37:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.860+0000 7f0129000640 1 Processor -- start 2024-04-26T23:37:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.860+0000 7f0129000640 1 -- start start 2024-04-26T23:37:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01240ce670 0x7f01240c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0124146740 0x7f0124139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01240cccf0 con 0x7f01240d0330 2024-04-26T23:37:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01240ccb70 con 0x7f01240ce670 2024-04-26T23:37:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0129000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0124161830 con 0x7f0124146740 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f011be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f011be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48330/0 (socket says 172.21.15.46:48330) 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f011be00640 1 -- 172.21.15.46:0/1797936742 learned_addr learned my addr 172.21.15.46:0/1797936742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0123400640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01240ce670 0x7f01240c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.861+0000 7f0123e00640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0124146740 0x7f0124139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 -- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0124146740 msgr2=0x7f0124139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0124146740 0x7f0124139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 -- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01240ce670 msgr2=0x7f01240c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01240ce670 0x7f01240c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 -- 172.21.15.46:0/1797936742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012415df20 con 0x7f01240d0330 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011be00640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d830 tx=0x7f010c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00b6c0 con 0x7f01240d0330 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c019070 con 0x7f01240d0330 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ba80 con 0x7f01240d0330 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.862+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012415e200 con 0x7f01240d0330 2024-04-26T23:37:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.863+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012415e7c0 con 0x7f01240d0330 2024-04-26T23:37:23.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.864+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01240cfff0 con 0x7f01240d0330 2024-04-26T23:37:23.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.865+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f010c01e460 con 0x7f01240d0330 2024-04-26T23:37:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.866+0000 7f011b400640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 0x7f010006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.866+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f010c08c690 con 0x7f01240d0330 2024-04-26T23:37:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.866+0000 7f0123400640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 0x7f010006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.867+0000 7f0123400640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 0x7f010006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0114034000 tx=0x7f011402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:23.871+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01240cfff0 con 0x7f01240d0330 2024-04-26T23:37:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.076+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01240693b0 con 0x7f01240d0330 2024-04-26T23:37:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.077+0000 7f011b400640 1 -- 172.21.15.46:0/1797936742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f010c05a650 con 0x7f01240d0330 2024-04-26T23:37:24.080 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:37:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 msgr2=0x7f010006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 0x7f010006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0114034000 tx=0x7f011402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 msgr2=0x7f01240b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d830 tx=0x7f010c00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 shutdown_connections 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f010006c8f0 0x7f010006edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0124146740 0x7f0124139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01240d0330 0x7f01240b6570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 --2- 172.21.15.46:0/1797936742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01240ce670 0x7f01240c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.079+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 >> 172.21.15.46:0/1797936742 conn(0x7f01240b26d0 msgr2=0x7f01240b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.080+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 shutdown_connections 2024-04-26T23:37:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.080+0000 7f0129000640 1 -- 172.21.15.46:0/1797936742 wait complete. 2024-04-26T23:37:24.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:37:24.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.312+0000 7f165ce00640 1 Processor -- start 2024-04-26T23:37:24.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.312+0000 7f165ce00640 1 -- start start 2024-04-26T23:37:24.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.313+0000 7f165ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f16581644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f165ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1658073af0 0x7f16580cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f165ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16581649f0 con 0x7f16580cbec0 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f165ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f16580cb4f0 con 0x7f1658073af0 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f165ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1658071e80 con 0x7f16581640d0 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f1657e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16580cbec0 0x7f16580ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48632/0 (socket says 172.21.15.46:48632) 2024-04-26T23:37:24.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f1657e00640 1 -- 172.21.15.46:0/833009193 learned_addr learned my addr 172.21.15.46:0/833009193 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:24.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f1657400640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f16581644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f164fe00640 1 --2- 172.21.15.46:0/833009193 >> v2:172.21.15.142:3300/0 conn(0x7f1658073af0 0x7f16580cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.314+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1331113006 0 0) 0x7f16581649f0 con 0x7f16580cbec0 2024-04-26T23:37:24.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f163c003520 con 0x7f16580cbec0 2024-04-26T23:37:24.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1291223766 0 0) 0x7f163c003520 con 0x7f16580cbec0 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 msgr2=0x7f16581644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f16581644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 >> v2:172.21.15.142:3300/0 conn(0x7f1658073af0 msgr2=0x7f16580cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 --2- 172.21.15.46:0/833009193 >> v2:172.21.15.142:3300/0 conn(0x7f1658073af0 0x7f16580cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1658072160 con 0x7f16580cbec0 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f16500019f0 con 0x7f16580cbec0 2024-04-26T23:37:24.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 >> v1:172.21.15.46:6789/0 conn(0x7f16580cbec0 legacy=0x7f16580ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f163c0039b0 0x7f163c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f163c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c004d40 0x7f163c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16580cb4f0 con 0x7f163c0039b0 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1658071e80 con 0x7f163c004d40 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f163c00b9c0 con 0x7f16581640d0 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.315+0000 7f1657400640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f163c0039b0 0x7f163c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f164fe00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f163c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f1657400640 1 -- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 msgr2=0x7f163c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f1657400640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16581640d0 0x7f163c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f1657400640 1 -- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c004d40 msgr2=0x7f163c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f1657400640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c004d40 0x7f163c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.316+0000 7f1657400640 1 -- 172.21.15.46:0/833009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c003520 con 0x7f163c0039b0 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.317+0000 7f1657400640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f163c0039b0 0x7f163c003dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1648009ac0 tx=0x7f163c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75452cda4c222ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.317+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648036070 con 0x7f163c0039b0 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.317+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1648031400 con 0x7f163c0039b0 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.317+0000 7f1656a00640 1 -- 172.21.15.46:0/833009193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648009690 con 0x7f163c0039b0 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 -- 172.21.15.46:0/833009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f163c0039b0 msgr2=0x7f163c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 --2- 172.21.15.46:0/833009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f163c0039b0 0x7f163c003dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1648009ac0 tx=0x7f163c005830 comp rx=0 tx=0).stop 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f1657e00640 1 -- 172.21.15.46:0/833009193 reap_dead start 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 -- 172.21.15.46:0/833009193 shutdown_connections 2024-04-26T23:37:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 -- 172.21.15.46:0/833009193 >> 172.21.15.46:0/833009193 conn(0x7f16580b27b0 msgr2=0x7f16580b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 -- 172.21.15.46:0/833009193 shutdown_connections 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.318+0000 7f165ce00640 1 -- 172.21.15.46:0/833009193 wait complete. 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.320+0000 7f165ce00640 1 Processor -- start 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.320+0000 7f165ce00640 1 -- start start 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f163c0039b0 0x7f16581ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1658149e20 0x7f16581eef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16580ceee0 con 0x7f165815cf20 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1658073f70 con 0x7f1658149e20 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f165ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16580cf060 con 0x7f163c0039b0 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f1657e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f1657e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48356/0 (socket says 172.21.15.46:48356) 2024-04-26T23:37:24.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.321+0000 7f1657e00640 1 -- 172.21.15.46:0/1911610514 learned_addr learned my addr 172.21.15.46:0/1911610514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f164fe00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1658149e20 0x7f16581eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657400640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f163c0039b0 0x7f16581ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 -- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f163c0039b0 msgr2=0x7f16581ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f163c0039b0 0x7f16581ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 -- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1658149e20 msgr2=0x7f16581eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1658149e20 0x7f16581eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 -- 172.21.15.46:0/1911610514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16580cac10 con 0x7f165815cf20 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f1657e00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16580b3710 tx=0x7f165000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000a640 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165000a7e0 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16580c7d70 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000abb0 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.322+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16580c8330 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.324+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16580693b0 con 0x7f165815cf20 2024-04-26T23:37:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.325+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f165000ad50 con 0x7f165815cf20 2024-04-26T23:37:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.326+0000 7f164f400640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 0x7f163406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.326+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f165008b880 con 0x7f165815cf20 2024-04-26T23:37:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.326+0000 7f1657400640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 0x7f163406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.328+0000 7f1657400640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 0x7f163406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1648033000 tx=0x7f1648008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.330+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1650059840 con 0x7f165815cf20 2024-04-26T23:37:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.536+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f16580c4010 con 0x7f165815cf20 2024-04-26T23:37:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.537+0000 7f164f400640 1 -- 172.21.15.46:0/1911610514 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f1650059140 con 0x7f165815cf20 2024-04-26T23:37:24.539 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 msgr2=0x7f163406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 0x7f163406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1648033000 tx=0x7f1648008030 comp rx=0 tx=0).stop 2024-04-26T23:37:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 msgr2=0x7f16581ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16580b3710 tx=0x7f165000cc00 comp rx=0 tx=0).stop 2024-04-26T23:37:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 shutdown_connections 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f163406c8f0 0x7f163406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165815cf20 0x7f16581ef440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1658149e20 0x7f16581eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 --2- 172.21.15.46:0/1911610514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f163c0039b0 0x7f16581ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 >> 172.21.15.46:0/1911610514 conn(0x7f16580b27b0 msgr2=0x7f1658164bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.539+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 shutdown_connections 2024-04-26T23:37:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.540+0000 7f165ce00640 1 -- 172.21.15.46:0/1911610514 wait complete. 2024-04-26T23:37:24.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:37:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.775+0000 7fd709400640 1 Processor -- start 2024-04-26T23:37:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.776+0000 7fd709400640 1 -- start start 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd709400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd7040d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd709400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7040d11f0 con 0x7fd70415f4e0 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd709400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7040d5f20 con 0x7fd7040b4000 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd709400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040d0ce0 con 0x7fd7040d2130 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd703e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd70415f4e0 0x7fd7040d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48636/0 (socket says 172.21.15.46:48636) 2024-04-26T23:37:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.777+0000 7fd703e00640 1 -- 172.21.15.46:0/2634865554 learned_addr learned my addr 172.21.15.46:0/2634865554 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd703400640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd7040d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1967187507 0 0) 0x7fd7040d11f0 con 0x7fd70415f4e0 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6e8003520 con 0x7fd70415f4e0 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3620107523 0 0) 0x7fd6e8003520 con 0x7fd70415f4e0 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 msgr2=0x7fd7040d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd7040d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 >> v1:172.21.15.142:6789/0 conn(0x7fd7040b4000 legacy=0x7fd7040b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7040d0f10 con 0x7fd70415f4e0 2024-04-26T23:37:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.778+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd6f4001670 con 0x7fd70415f4e0 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 >> v1:172.21.15.46:6789/0 conn(0x7fd70415f4e0 legacy=0x7fd7040d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd6e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8003f60 0x7fd6e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6e8004d40 0x7fd6e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e800ba70 con 0x7fd6e8004d40 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e800bc20 con 0x7fd6e8003f60 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e800be20 con 0x7fd7040d2130 2024-04-26T23:37:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd708a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6e8004d40 0x7fd6e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.779+0000 7fd703e00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd6e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.780+0000 7fd708a00640 1 -- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 msgr2=0x7fd6e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.780+0000 7fd708a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040d2130 0x7fd6e80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.780+0000 7fd708a00640 1 -- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8003f60 msgr2=0x7fd6e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.780+0000 7fd708a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8003f60 0x7fd6e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.780+0000 7fd708a00640 1 -- 172.21.15.46:0/2634865554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e8003520 con 0x7fd6e8004d40 2024-04-26T23:37:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.781+0000 7fd708a00640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6e8004d40 0x7fd6e80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc001e50 tx=0x7fd6e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2fdb15a2f4021d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.781+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc001a70 con 0x7fd6e8004d40 2024-04-26T23:37:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.781+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc003970 con 0x7fd6e8004d40 2024-04-26T23:37:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.781+0000 7fd702a00640 1 -- 172.21.15.46:0/2634865554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc003d60 con 0x7fd6e8004d40 2024-04-26T23:37:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 -- 172.21.15.46:0/2634865554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6e8004d40 msgr2=0x7fd6e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 --2- 172.21.15.46:0/2634865554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6e8004d40 0x7fd6e80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc001e50 tx=0x7fd6e8005830 comp rx=0 tx=0).stop 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd708a00640 1 -- 172.21.15.46:0/2634865554 reap_dead start 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 -- 172.21.15.46:0/2634865554 shutdown_connections 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 -- 172.21.15.46:0/2634865554 >> 172.21.15.46:0/2634865554 conn(0x7fd7040d5010 msgr2=0x7fd7040d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 -- 172.21.15.46:0/2634865554 shutdown_connections 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.782+0000 7fd709400640 1 -- 172.21.15.46:0/2634865554 wait complete. 2024-04-26T23:37:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.784+0000 7fd709400640 1 Processor -- start 2024-04-26T23:37:24.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.784+0000 7fd709400640 1 -- start start 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8004d40 0x7fd704158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040ce8d0 0x7fd7040ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7040b1d60 con 0x7fd70415b2d0 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7040d2990 con 0x7fd6e8004d40 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd709400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd70415f8c0 con 0x7fd7040ce8d0 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd703400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd703400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48384/0 (socket says 172.21.15.46:48384) 2024-04-26T23:37:24.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.785+0000 7fd703400640 1 -- 172.21.15.46:0/723987156 learned_addr learned my addr 172.21.15.46:0/723987156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703e00640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8004d40 0x7fd704158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd708a00640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040ce8d0 0x7fd7040ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 -- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040ce8d0 msgr2=0x7fd7040ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040ce8d0 0x7fd7040ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 -- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8004d40 msgr2=0x7fd704158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8004d40 0x7fd704158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 -- 172.21.15.46:0/723987156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd704158860 con 0x7fd70415b2d0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.786+0000 7fd703400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e840 tx=0x7fd6f000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.787+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0017070 con 0x7fd70415b2d0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.787+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f000c940 con 0x7fd70415b2d0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.787+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7040cc9f0 con 0x7fd70415b2d0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.787+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000cd30 con 0x7fd70415b2d0 2024-04-26T23:37:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.787+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7040ccfb0 con 0x7fd70415b2d0 2024-04-26T23:37:24.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.788+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7040693b0 con 0x7fd70415b2d0 2024-04-26T23:37:24.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.789+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6f001a070 con 0x7fd70415b2d0 2024-04-26T23:37:24.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.790+0000 7fd6fbe00640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 0x7fd6e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:24.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.791+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6f008bdf0 con 0x7fd70415b2d0 2024-04-26T23:37:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.791+0000 7fd703e00640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 0x7fd6e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.792+0000 7fd703e00640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 0x7fd6e406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4001eb0 tx=0x7fd6f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:24.795+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6f0059ed0 con 0x7fd70415b2d0 2024-04-26T23:37:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.016+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7040d33d0 con 0x7fd70415b2d0 2024-04-26T23:37:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.017+0000 7fd6fbe00640 1 -- 172.21.15.46:0/723987156 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fd6f00597d0 con 0x7fd70415b2d0 2024-04-26T23:37:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 msgr2=0x7fd6e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 0x7fd6e406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4001eb0 tx=0x7fd6f4009040 comp rx=0 tx=0).stop 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 msgr2=0x7fd7040cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e840 tx=0x7fd6f000ed00 comp rx=0 tx=0).stop 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 shutdown_connections 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6e406c8f0 0x7fd6e406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7040ce8d0 0x7fd7040ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd70415b2d0 0x7fd7040cc0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 --2- 172.21.15.46:0/723987156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e8004d40 0x7fd704158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 >> 172.21.15.46:0/723987156 conn(0x7fd7040d5010 msgr2=0x7fd7040b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 shutdown_connections 2024-04-26T23:37:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.020+0000 7fd709400640 1 -- 172.21.15.46:0/723987156 wait complete. 2024-04-26T23:37:25.042 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:37:25.042 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:37:25.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.305+0000 7f864aa00640 1 Processor -- start 2024-04-26T23:37:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.305+0000 7f864aa00640 1 -- start start 2024-04-26T23:37:25.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.306+0000 7f864aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86440d2130 0x7f86440d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.306+0000 7f864aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f86440b4000 0x7f86440b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.306+0000 7f864aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86440d11f0 con 0x7f864415f4e0 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.306+0000 7f864aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f86440d0ce0 con 0x7f86440d2130 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.306+0000 7f864aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f86440d2990 con 0x7f86440b4000 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86440d2130 0x7f86440d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86440d2130 0x7f86440d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48388/0 (socket says 172.21.15.46:48388) 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 -- 172.21.15.46:0/2593309537 learned_addr learned my addr 172.21.15.46:0/2593309537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f864a000640 1 --2- 172.21.15.46:0/2593309537 >> v2:172.21.15.142:3300/0 conn(0x7f86440b4000 0x7f86440b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 -- 172.21.15.46:0/2593309537 >> v2:172.21.15.142:3300/0 conn(0x7f86440b4000 msgr2=0x7f86440b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 --2- 172.21.15.46:0/2593309537 >> v2:172.21.15.142:3300/0 conn(0x7f86440b4000 0x7f86440b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 -- 172.21.15.46:0/2593309537 >> v1:172.21.15.142:6789/0 conn(0x7f864415f4e0 legacy=0x7f86440d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.307+0000 7f8648c00640 1 -- 172.21.15.46:0/2593309537 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86440d5f20 con 0x7f86440d2130 2024-04-26T23:37:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.308+0000 7f8648c00640 1 --2- 172.21.15.46:0/2593309537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f86440d2df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a514171b7fa476f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.309+0000 7f8643e00640 1 -- 172.21.15.46:0/2593309537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630036070 con 0x7f86440d2130 2024-04-26T23:37:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.309+0000 7f8643e00640 1 -- 172.21.15.46:0/2593309537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630031600 con 0x7f86440d2130 2024-04-26T23:37:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.309+0000 7f8643e00640 1 -- 172.21.15.46:0/2593309537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300319d0 con 0x7f86440d2130 2024-04-26T23:37:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.309+0000 7f864aa00640 1 -- 172.21.15.46:0/2593309537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 msgr2=0x7f86440d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.309+0000 7f864aa00640 1 --2- 172.21.15.46:0/2593309537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f86440d2df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863002a710 comp rx=0 tx=0).stop 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 -- 172.21.15.46:0/2593309537 shutdown_connections 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 --2- 172.21.15.46:0/2593309537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f86440d2df0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 --2- 172.21.15.46:0/2593309537 >> v2:172.21.15.142:3300/0 conn(0x7f86440b4000 0x7f86440b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 -- 172.21.15.46:0/2593309537 >> 172.21.15.46:0/2593309537 conn(0x7f86440d5010 msgr2=0x7f86440d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 -- 172.21.15.46:0/2593309537 shutdown_connections 2024-04-26T23:37:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.310+0000 7f864aa00640 1 -- 172.21.15.46:0/2593309537 wait complete. 2024-04-26T23:37:25.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.312+0000 7f864aa00640 1 Processor -- start 2024-04-26T23:37:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.312+0000 7f864aa00640 1 -- start start 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 0x7f86440ba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f864415f4e0 0x7f86440b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86440b1d60 con 0x7f86440d2130 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86440d5c40 con 0x7f86440b4000 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.313+0000 7f864aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f864415f8c0 con 0x7f864415f4e0 2024-04-26T23:37:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48402/0 (socket says 172.21.15.46:48402) 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 -- 172.21.15.46:0/4128016785 learned_addr learned my addr 172.21.15.46:0/4128016785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8649600640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 0x7f86440ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f864a000640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f864415f4e0 0x7f86440b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 -- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f864415f4e0 msgr2=0x7f86440b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f864415f4e0 0x7f86440b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 -- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 msgr2=0x7f86440ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 0x7f86440ba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8648c00640 1 -- 172.21.15.46:0/4128016785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864415d030 con 0x7f86440d2130 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.314+0000 7f8649600640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 0x7f86440ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f8648c00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8630034040 tx=0x7f8630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630036050 con 0x7f86440d2130 2024-04-26T23:37:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630037040 con 0x7f86440d2130 2024-04-26T23:37:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864415d310 con 0x7f86440d2130 2024-04-26T23:37:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630009510 con 0x7f86440d2130 2024-04-26T23:37:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.315+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864415d8d0 con 0x7f86440d2130 2024-04-26T23:37:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.316+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86440693b0 con 0x7f86440d2130 2024-04-26T23:37:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.317+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8630007c50 con 0x7f86440d2130 2024-04-26T23:37:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.319+0000 7f8641600640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 0x7f862406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.319+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f86300acad0 con 0x7f86440d2130 2024-04-26T23:37:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.319+0000 7f8649600640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 0x7f862406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.320+0000 7f8649600640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 0x7f862406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8634003eb0 tx=0x7f8634009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.323+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f863007ab20 con 0x7f86440d2130 2024-04-26T23:37:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.526+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86440d33d0 con 0x7f86440d2130 2024-04-26T23:37:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.529+0000 7f8641600640 1 -- 172.21.15.46:0/4128016785 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f863007a420 con 0x7f86440d2130 2024-04-26T23:37:25.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:25.532 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.532+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 msgr2=0x7f862406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.532+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 0x7f862406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8634003eb0 tx=0x7f8634009040 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.532+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 msgr2=0x7f864415c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.532+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8630034040 tx=0x7f8630004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 shutdown_connections 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f862406c8f0 0x7f862406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f864415f4e0 0x7f86440b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86440d2130 0x7f864415c5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 --2- 172.21.15.46:0/4128016785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86440b4000 0x7f86440ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 >> 172.21.15.46:0/4128016785 conn(0x7f86440d5010 msgr2=0x7f864406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 shutdown_connections 2024-04-26T23:37:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.533+0000 7f864aa00640 1 -- 172.21.15.46:0/4128016785 wait complete. 2024-04-26T23:37:25.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:37:25.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:37:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.769+0000 7fe86aa00640 1 Processor -- start 2024-04-26T23:37:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.769+0000 7fe86aa00640 1 -- start start 2024-04-26T23:37:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.771+0000 7fe86aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 0x7fe864160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.771+0000 7fe86aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 0x7fe864072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.771+0000 7fe86aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8640cbe60 con 0x7fe86415cc40 2024-04-26T23:37:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.771+0000 7fe86aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8640cca70 con 0x7fe8640cdc60 2024-04-26T23:37:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.771+0000 7fe86aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8640cc140 con 0x7fe864160130 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe86a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 0x7fe864072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe869600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 0x7fe864160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe868c00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe86415cc40 0x7fe8640cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:42550/0 (socket says 172.21.15.46:42550) 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe868c00640 1 -- 172.21.15.46:0/1501555796 learned_addr learned my addr 172.21.15.46:0/1501555796 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe869600640 1 -- 172.21.15.46:0/1501555796 >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 msgr2=0x7fe864072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe869600640 1 --2- 172.21.15.46:0/1501555796 >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 0x7fe864072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe869600640 1 -- 172.21.15.46:0/1501555796 >> v1:172.21.15.142:6789/0 conn(0x7fe86415cc40 legacy=0x7fe8640cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe869600640 1 -- 172.21.15.46:0/1501555796 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8640cc420 con 0x7fe864160130 2024-04-26T23:37:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.772+0000 7fe86a000640 1 --2- 172.21.15.46:0/1501555796 >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 0x7fe864072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.774+0000 7fe869600640 1 --2- 172.21.15.46:0/1501555796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 0x7fe864160510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8580097a0 tx=0x7fe85802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c275e8648c29219 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.774+0000 7fe863e00640 1 -- 172.21.15.46:0/1501555796 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858037070 con 0x7fe864160130 2024-04-26T23:37:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe863e00640 1 -- 172.21.15.46:0/1501555796 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85802ae50 con 0x7fe864160130 2024-04-26T23:37:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe863e00640 1 -- 172.21.15.46:0/1501555796 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858032660 con 0x7fe864160130 2024-04-26T23:37:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 -- 172.21.15.46:0/1501555796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 msgr2=0x7fe864160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1501555796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 0x7fe864160510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8580097a0 tx=0x7fe85802a100 comp rx=0 tx=0).stop 2024-04-26T23:37:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 -- 172.21.15.46:0/1501555796 shutdown_connections 2024-04-26T23:37:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1501555796 >> v2:172.21.15.142:3300/0 conn(0x7fe8640cdc60 0x7fe864072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1501555796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe864160130 0x7fe864160510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 -- 172.21.15.46:0/1501555796 >> 172.21.15.46:0/1501555796 conn(0x7fe8640b27b0 msgr2=0x7fe8640b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 -- 172.21.15.46:0/1501555796 shutdown_connections 2024-04-26T23:37:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.775+0000 7fe86aa00640 1 -- 172.21.15.46:0/1501555796 wait complete. 2024-04-26T23:37:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.777+0000 7fe86aa00640 1 Processor -- start 2024-04-26T23:37:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.777+0000 7fe86aa00640 1 -- start start 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86415cc40 0x7fe8640c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe864160130 0x7fe86414dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe86415d220 con 0x7fe8640cdc60 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe86415d0a0 con 0x7fe864160130 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe86aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe864072760 con 0x7fe86415cc40 2024-04-26T23:37:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe869600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.778+0000 7fe869600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48408/0 (socket says 172.21.15.46:48408) 2024-04-26T23:37:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 -- 172.21.15.46:0/1018651071 learned_addr learned my addr 172.21.15.46:0/1018651071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe868c00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86415cc40 0x7fe8640c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 -- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86415cc40 msgr2=0x7fe8640c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86415cc40 0x7fe8640c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 -- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe864160130 msgr2=0x7fe86414dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe86a000640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe864160130 0x7fe86414dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe864160130 0x7fe86414dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 -- 172.21.15.46:0/1018651071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8640c4c70 con 0x7fe8640cdc60 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.779+0000 7fe869600640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe85802a6f0 tx=0x7fe8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.780+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858042020 con 0x7fe8640cdc60 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.780+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85803d690 con 0x7fe8640cdc60 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.780+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858037040 con 0x7fe8640cdc60 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.780+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8640c4f40 con 0x7fe8640cdc60 2024-04-26T23:37:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.780+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8640c5500 con 0x7fe8640cdc60 2024-04-26T23:37:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.781+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe864072550 con 0x7fe8640cdc60 2024-04-26T23:37:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.782+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe858045020 con 0x7fe8640cdc60 2024-04-26T23:37:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.784+0000 7fe861600640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.784+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8580ac5b0 con 0x7fe8640cdc60 2024-04-26T23:37:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.784+0000 7fe868c00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.785+0000 7fe868c00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 0x7fe84406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe86414bbe0 tx=0x7fe850009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.788+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe864072550 con 0x7fe8640cdc60 2024-04-26T23:37:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.996+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8640693b0 con 0x7fe8640cdc60 2024-04-26T23:37:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:25.997+0000 7fe861600640 1 -- 172.21.15.46:0/1018651071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe85804c030 con 0x7fe8640cdc60 2024-04-26T23:37:26.000 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:37:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 msgr2=0x7fe84406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 0x7fe84406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe86414bbe0 tx=0x7fe850009040 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 msgr2=0x7fe8641492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe85802a6f0 tx=0x7fe8580046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 shutdown_connections 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe864160130 0x7fe86414dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86415cc40 0x7fe8640c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 --2- 172.21.15.46:0/1018651071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8640cdc60 0x7fe8641492d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 >> 172.21.15.46:0/1018651071 conn(0x7fe8640b27b0 msgr2=0x7fe86415d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:26.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 shutdown_connections 2024-04-26T23:37:26.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.000+0000 7fe86aa00640 1 -- 172.21.15.46:0/1018651071 wait complete. 2024-04-26T23:37:26.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:37:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.228+0000 7f9ba5800640 1 Processor -- start 2024-04-26T23:37:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.228+0000 7f9ba5800640 1 -- start start 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.229+0000 7f9ba5800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ba00b5730 0x7f9ba00b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9ba5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0071f10 0x7f9ba0072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9ba5800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba0073120 con 0x7f9ba01597e0 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9ba5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba00aeff0 con 0x7f9ba0071f10 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9ba5800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba0159bc0 con 0x7f9ba00b5730 2024-04-26T23:37:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0071f10 0x7f9ba0072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0071f10 0x7f9ba0072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48412/0 (socket says 172.21.15.46:48412) 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9f400640 1 -- 172.21.15.46:0/2381762139 learned_addr learned my addr 172.21.15.46:0/2381762139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9e000640 1 --2- 172.21.15.46:0/2381762139 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00b5730 0x7f9ba00b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9f400640 1 -- 172.21.15.46:0/2381762139 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00b5730 msgr2=0x7f9ba00b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.230+0000 7f9b9f400640 1 --2- 172.21.15.46:0/2381762139 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00b5730 0x7f9ba00b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.231+0000 7f9b9f400640 1 -- 172.21.15.46:0/2381762139 >> v1:172.21.15.142:6789/0 conn(0x7f9ba01597e0 legacy=0x7f9ba00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.231+0000 7f9b9f400640 1 -- 172.21.15.46:0/2381762139 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba0072b60 con 0x7f9ba0071f10 2024-04-26T23:37:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9b9f400640 1 --2- 172.21.15.46:0/2381762139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0071f10 0x7f9ba0072340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00743c0 tx=0x7f9b9802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3224546715b1d181 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9b9d600640 1 -- 172.21.15.46:0/2381762139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98037070 con 0x7f9ba0071f10 2024-04-26T23:37:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9b9d600640 1 -- 172.21.15.46:0/2381762139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b98032440 con 0x7f9ba0071f10 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9b9d600640 1 -- 172.21.15.46:0/2381762139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98032810 con 0x7f9ba0071f10 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9ba5800640 1 -- 172.21.15.46:0/2381762139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0071f10 msgr2=0x7f9ba0072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.232+0000 7f9ba5800640 1 --2- 172.21.15.46:0/2381762139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0071f10 0x7f9ba0072340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00743c0 tx=0x7f9b9802a2c0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 -- 172.21.15.46:0/2381762139 shutdown_connections 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 --2- 172.21.15.46:0/2381762139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0071f10 0x7f9ba0072340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 --2- 172.21.15.46:0/2381762139 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00b5730 0x7f9ba00b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 -- 172.21.15.46:0/2381762139 >> 172.21.15.46:0/2381762139 conn(0x7f9ba006c980 msgr2=0x7f9ba00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 -- 172.21.15.46:0/2381762139 shutdown_connections 2024-04-26T23:37:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.233+0000 7f9ba5800640 1 -- 172.21.15.46:0/2381762139 wait complete. 2024-04-26T23:37:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.234+0000 7f9ba5800640 1 Processor -- start 2024-04-26T23:37:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.235+0000 7f9ba5800640 1 -- start start 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 0x7f9ba00b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00b5730 0x7f9ba01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba00b43b0 con 0x7f9ba01597e0 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba00b40d0 con 0x7f9ba0071f10 2024-04-26T23:37:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9ba5800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba0072880 con 0x7f9ba00b5730 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48428/0 (socket says 172.21.15.46:48428) 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 -- 172.21.15.46:0/1878990840 learned_addr learned my addr 172.21.15.46:0/1878990840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9e000640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00b5730 0x7f9ba01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 -- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00b5730 msgr2=0x7f9ba01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00b5730 0x7f9ba01562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 -- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 msgr2=0x7f9ba00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9ea00640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 0x7f9ba00b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 0x7f9ba00b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9f400640 1 -- 172.21.15.46:0/1878990840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba00b07c0 con 0x7f9ba01597e0 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.236+0000 7f9b9ea00640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 0x7f9ba00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9b9f400640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b9802a300 tx=0x7f9b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98041020 con 0x7f9ba01597e0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980306c0 con 0x7f9ba01597e0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba00b0aa0 con 0x7f9ba01597e0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98037040 con 0x7f9ba01597e0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.237+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba00b1060 con 0x7f9ba01597e0 2024-04-26T23:37:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.238+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba0069530 con 0x7f9ba01597e0 2024-04-26T23:37:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.239+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b98044020 con 0x7f9ba01597e0 2024-04-26T23:37:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.240+0000 7f9b96a00640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.240+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b980ac740 con 0x7f9ba01597e0 2024-04-26T23:37:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.241+0000 7f9b9ea00640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.242+0000 7f9b9ea00640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 0x7f9b8006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007e90 tx=0x7f9b900073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.245+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b9804b030 con 0x7f9ba01597e0 2024-04-26T23:37:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.443+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ba0076550 con 0x7f9ba01597e0 2024-04-26T23:37:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.444+0000 7f9b96a00640 1 -- 172.21.15.46:0/1878990840 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f9b9807a790 con 0x7f9ba01597e0 2024-04-26T23:37:26.447 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.446+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 msgr2=0x7f9b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.446+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 0x7f9b8006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007e90 tx=0x7f9b900073a0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.446+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 msgr2=0x7f9ba0156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.446+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b9802a300 tx=0x7f9b980046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 shutdown_connections 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01597e0 0x7f9ba0156800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00b5730 0x7f9ba01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 --2- 172.21.15.46:0/1878990840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0071f10 0x7f9ba00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 >> 172.21.15.46:0/1878990840 conn(0x7f9ba006c980 msgr2=0x7f9ba00b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 shutdown_connections 2024-04-26T23:37:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.447+0000 7f9ba5800640 1 -- 172.21.15.46:0/1878990840 wait complete. 2024-04-26T23:37:26.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:37:26.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.675+0000 7f0014a00640 1 Processor -- start 2024-04-26T23:37:26.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.675+0000 7f0014a00640 1 -- start start 2024-04-26T23:37:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.676+0000 7f0014a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 0x7f001015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.676+0000 7f0014a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.676+0000 7f0014a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0010146b20 con 0x7f00100d0330 2024-04-26T23:37:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.676+0000 7f0014a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0010161a00 con 0x7f0010146740 2024-04-26T23:37:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.676+0000 7f0014a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00100d07c0 con 0x7f00100ce670 2024-04-26T23:37:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.677+0000 7f000fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.677+0000 7f000f400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 0x7f001015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.677+0000 7f000fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:56956/0 (socket says 172.21.15.46:56956) 2024-04-26T23:37:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.677+0000 7f000fe00640 1 -- 172.21.15.46:0/3872304729 learned_addr learned my addr 172.21.15.46:0/3872304729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:26.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.678+0000 7f000fe00640 1 -- 172.21.15.46:0/3872304729 >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 msgr2=0x7f001015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:26.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.678+0000 7f000fe00640 1 --2- 172.21.15.46:0/3872304729 >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 0x7f001015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.678+0000 7f000fe00640 1 -- 172.21.15.46:0/3872304729 >> v1:172.21.15.142:6789/0 conn(0x7f00100d0330 legacy=0x7f00101612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.678+0000 7f000fe00640 1 -- 172.21.15.46:0/3872304729 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00100ccf30 con 0x7f00100ce670 2024-04-26T23:37:26.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.678+0000 7f000f400640 1 --2- 172.21.15.46:0/3872304729 >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 0x7f001015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.680+0000 7f000fe00640 1 --2- 172.21.15.46:0/3872304729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e50 tx=0x7f000802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a957faa9f8b62e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.680+0000 7f000ea00640 1 -- 172.21.15.46:0/3872304729 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008036070 con 0x7f00100ce670 2024-04-26T23:37:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f000ea00640 1 -- 172.21.15.46:0/3872304729 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008031600 con 0x7f00100ce670 2024-04-26T23:37:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f000ea00640 1 -- 172.21.15.46:0/3872304729 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00080319d0 con 0x7f00100ce670 2024-04-26T23:37:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 -- 172.21.15.46:0/3872304729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 msgr2=0x7f00100cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 --2- 172.21.15.46:0/3872304729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e50 tx=0x7f000802a710 comp rx=0 tx=0).stop 2024-04-26T23:37:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 -- 172.21.15.46:0/3872304729 shutdown_connections 2024-04-26T23:37:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 --2- 172.21.15.46:0/3872304729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100ce670 0x7f00100cea80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 --2- 172.21.15.46:0/3872304729 >> v2:172.21.15.142:3300/0 conn(0x7f0010146740 0x7f001015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.681+0000 7f0014a00640 1 -- 172.21.15.46:0/3872304729 >> 172.21.15.46:0/3872304729 conn(0x7f00100b26d0 msgr2=0x7f00100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.682+0000 7f0014a00640 1 -- 172.21.15.46:0/3872304729 shutdown_connections 2024-04-26T23:37:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.682+0000 7f0014a00640 1 -- 172.21.15.46:0/3872304729 wait complete. 2024-04-26T23:37:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.683+0000 7f0014a00640 1 Processor -- start 2024-04-26T23:37:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.683+0000 7f0014a00640 1 -- start start 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.684+0000 7f0014a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.684+0000 7f0014a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100d0330 0x7f0010140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.684+0000 7f0014a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0010146740 0x7f0010141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f0014a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00100ccb70 con 0x7f00100ce670 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f0014a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0010160dd0 con 0x7f0010146740 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f0014a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00100cccf0 con 0x7f00100d0330 2024-04-26T23:37:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48430/0 (socket says 172.21.15.46:48430) 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 -- 172.21.15.46:0/223678836 learned_addr learned my addr 172.21.15.46:0/223678836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f0007e00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100d0330 0x7f0010140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000fe00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0010146740 0x7f0010141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 -- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100d0330 msgr2=0x7f0010140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100d0330 0x7f0010140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 -- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0010146740 msgr2=0x7f0010141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0010146740 0x7f0010141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.685+0000 7f000f400640 1 -- 172.21.15.46:0/223678836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00101f5180 con 0x7f00100ce670 2024-04-26T23:37:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f000f400640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f000000e960 tx=0x7f000000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000017070 con 0x7f00100ce670 2024-04-26T23:37:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f000000ca00 con 0x7f00100ce670 2024-04-26T23:37:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000cdd0 con 0x7f00100ce670 2024-04-26T23:37:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101f5460 con 0x7f00100ce670 2024-04-26T23:37:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.686+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101f5f40 con 0x7f00100ce670 2024-04-26T23:37:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.688+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00100693b0 con 0x7f00100ce670 2024-04-26T23:37:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.688+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f000001a070 con 0x7f00100ce670 2024-04-26T23:37:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.689+0000 7f0007400640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 0x7effec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.690+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f000008c1a0 con 0x7f00100ce670 2024-04-26T23:37:26.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.690+0000 7f0007e00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 0x7effec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:26.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.691+0000 7f0007e00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 0x7effec06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00100d0170 tx=0x7efff8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.694+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f000005a1f0 con 0x7f00100ce670 2024-04-26T23:37:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.896+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f001006f400 con 0x7f00100ce670 2024-04-26T23:37:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.897+0000 7f0007400640 1 -- 172.21.15.46:0/223678836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f0000059af0 con 0x7f00100ce670 2024-04-26T23:37:26.900 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.900+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 msgr2=0x7effec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.900+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 0x7effec06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00100d0170 tx=0x7efff8009040 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.900+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 msgr2=0x7f00100c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.900+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f000000e960 tx=0x7f000000ee20 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 shutdown_connections 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effec06c8f0 0x7effec06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0010146740 0x7f0010141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00100d0330 0x7f0010140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 --2- 172.21.15.46:0/223678836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00100ce670 0x7f00100c6d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 >> 172.21.15.46:0/223678836 conn(0x7f00100b26d0 msgr2=0x7f00100cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 shutdown_connections 2024-04-26T23:37:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:26.901+0000 7f0014a00640 1 -- 172.21.15.46:0/223678836 wait complete. 2024-04-26T23:37:26.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:37:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.134+0000 7fd3d8a00640 1 Processor -- start 2024-04-26T23:37:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.134+0000 7fd3d8a00640 1 -- start start 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.136+0000 7fd3d8a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd3d415f210 0x7fd3d40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.136+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40cd610 0x7fd3d40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.136+0000 7fd3d8a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d40af570 con 0x7fd3d40ae8f0 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.136+0000 7fd3d8a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d40cdfb0 con 0x7fd3d415f210 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.136+0000 7fd3d8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d40af850 con 0x7fd3d40cd610 2024-04-26T23:37:27.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd3d415f210 0x7fd3d40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40cd610 0x7fd3d40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd3d415f210 0x7fd3d40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:57062/0 (socket says 172.21.15.46:57062) 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 -- 172.21.15.46:0/142103648 learned_addr learned my addr 172.21.15.46:0/142103648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 -- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40cd610 msgr2=0x7fd3d40cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 --2- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40cd610 0x7fd3d40cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.137+0000 7fd3d2800640 1 -- 172.21.15.46:0/142103648 >> v1:172.21.15.142:6789/0 conn(0x7fd3d40ae8f0 legacy=0x7fd3d40aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.138+0000 7fd3d2800640 1 -- 172.21.15.46:0/142103648 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d40cb680 con 0x7fd3d415f210 2024-04-26T23:37:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.139+0000 7fd3d2800640 1 --2- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40cc160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009870 tx=0x7fd3c402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99054a9170b26b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.139+0000 7fd3d1e00640 1 -- 172.21.15.46:0/142103648 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4037070 con 0x7fd3d415f210 2024-04-26T23:37:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.139+0000 7fd3d1e00640 1 -- 172.21.15.46:0/142103648 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c4032440 con 0x7fd3d415f210 2024-04-26T23:37:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.139+0000 7fd3d1e00640 1 -- 172.21.15.46:0/142103648 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4032810 con 0x7fd3d415f210 2024-04-26T23:37:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 -- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 msgr2=0x7fd3d40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40cc160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009870 tx=0x7fd3c402a250 comp rx=0 tx=0).stop 2024-04-26T23:37:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 -- 172.21.15.46:0/142103648 shutdown_connections 2024-04-26T23:37:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40cd610 0x7fd3d40cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/142103648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40cc160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.140+0000 7fd3d8a00640 1 -- 172.21.15.46:0/142103648 >> 172.21.15.46:0/142103648 conn(0x7fd3d40b27b0 msgr2=0x7fd3d40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.141+0000 7fd3d8a00640 1 -- 172.21.15.46:0/142103648 shutdown_connections 2024-04-26T23:37:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.141+0000 7fd3d8a00640 1 -- 172.21.15.46:0/142103648 wait complete. 2024-04-26T23:37:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.143+0000 7fd3d8a00640 1 Processor -- start 2024-04-26T23:37:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.143+0000 7fd3d8a00640 1 -- start start 2024-04-26T23:37:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.144+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40ae8f0 0x7fd3d414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.144+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d415f5f0 con 0x7fd3d40cd610 2024-04-26T23:37:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d40cbc40 con 0x7fd3d415f210 2024-04-26T23:37:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d4071e80 con 0x7fd3d40ae8f0 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48440/0 (socket says 172.21.15.46:48440) 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3c9e00640 1 -- 172.21.15.46:0/1952621855 learned_addr learned my addr 172.21.15.46:0/1952621855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d2800640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40ae8f0 0x7fd3d414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.145+0000 7fd3d3200640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 -- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40ae8f0 msgr2=0x7fd3d414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40ae8f0 0x7fd3d414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 -- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 msgr2=0x7fd3d40c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 -- 172.21.15.46:0/1952621855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d40c4a60 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3c9e00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9b0 tx=0x7fd3bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc017070 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00c9a0 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00cd90 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.146+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d40c4d40 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.147+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d40c5300 con 0x7fd3d40cd610 2024-04-26T23:37:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.148+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d40693b0 con 0x7fd3d40cd610 2024-04-26T23:37:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.149+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3bc01a070 con 0x7fd3d40cd610 2024-04-26T23:37:27.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.150+0000 7fd3cb400640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 0x7fd3b006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.150+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3bc08c120 con 0x7fd3d40cd610 2024-04-26T23:37:27.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.150+0000 7fd3d2800640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 0x7fd3b006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.152+0000 7fd3d2800640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 0x7fd3b006ee30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4034000 tx=0x7fd3c402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.154+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3bc059c10 con 0x7fd3d40cd610 2024-04-26T23:37:27.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.354+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd3d40b2da0 con 0x7fd3d40cd610 2024-04-26T23:37:27.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.355+0000 7fd3cb400640 1 -- 172.21.15.46:0/1952621855 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fd3bc05df10 con 0x7fd3d40cd610 2024-04-26T23:37:27.358 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:27.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 msgr2=0x7fd3b006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 0x7fd3b006ee30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4034000 tx=0x7fd3c402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:27.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 msgr2=0x7fd3d40c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9b0 tx=0x7fd3bc00ee70 comp rx=0 tx=0).stop 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 shutdown_connections 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3b006c970 0x7fd3b006ee30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d415f210 0x7fd3d40c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d40cd610 0x7fd3d40c5b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 --2- 172.21.15.46:0/1952621855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3d40ae8f0 0x7fd3d414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 >> 172.21.15.46:0/1952621855 conn(0x7fd3d40b27b0 msgr2=0x7fd3d415d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.358+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 shutdown_connections 2024-04-26T23:37:27.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.359+0000 7fd3d8a00640 1 -- 172.21.15.46:0/1952621855 wait complete. 2024-04-26T23:37:27.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:37:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.585+0000 7fcf76000640 1 Processor -- start 2024-04-26T23:37:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.585+0000 7fcf76000640 1 -- start start 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.587+0000 7fcf76000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7015f4e0 0x7fcf700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.587+0000 7fcf76000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf700b4000 0x7fcf700d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.587+0000 7fcf76000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf700d11f0 con 0x7fcf700d1f60 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.587+0000 7fcf76000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf700d2990 con 0x7fcf700b4000 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.587+0000 7fcf76000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700d5f20 con 0x7fcf7015f4e0 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf74c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7015f4e0 0x7fcf700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf700b4000 0x7fcf700d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf700b4000 0x7fcf700d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:57086/0 (socket says 172.21.15.46:57086) 2024-04-26T23:37:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 -- 172.21.15.46:0/1205153717 learned_addr learned my addr 172.21.15.46:0/1205153717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 -- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7015f4e0 msgr2=0x7fcf700d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7015f4e0 0x7fcf700d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 -- 172.21.15.46:0/1205153717 >> v1:172.21.15.142:6789/0 conn(0x7fcf700d1f60 legacy=0x7fcf700cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.588+0000 7fcf6fe00640 1 -- 172.21.15.46:0/1205153717 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf700d0f10 con 0x7fcf700b4000 2024-04-26T23:37:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.590+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700b4000 0x7fcf700d1a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009870 tx=0x7fcf5c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7eafedc4a929ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.590+0000 7fcf6f400640 1 -- 172.21.15.46:0/1205153717 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c037070 con 0x7fcf700b4000 2024-04-26T23:37:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.590+0000 7fcf6f400640 1 -- 172.21.15.46:0/1205153717 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c032440 con 0x7fcf700b4000 2024-04-26T23:37:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.590+0000 7fcf6f400640 1 -- 172.21.15.46:0/1205153717 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c032810 con 0x7fcf700b4000 2024-04-26T23:37:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 -- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700b4000 msgr2=0x7fcf700d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 --2- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700b4000 0x7fcf700d1a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009870 tx=0x7fcf5c02a250 comp rx=0 tx=0).stop 2024-04-26T23:37:27.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 -- 172.21.15.46:0/1205153717 shutdown_connections 2024-04-26T23:37:27.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 --2- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700b4000 0x7fcf700d1a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 --2- 172.21.15.46:0/1205153717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7015f4e0 0x7fcf700d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 -- 172.21.15.46:0/1205153717 >> 172.21.15.46:0/1205153717 conn(0x7fcf700d5010 msgr2=0x7fcf700d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 -- 172.21.15.46:0/1205153717 shutdown_connections 2024-04-26T23:37:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.591+0000 7fcf76000640 1 -- 172.21.15.46:0/1205153717 wait complete. 2024-04-26T23:37:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.593+0000 7fcf76000640 1 Processor -- start 2024-04-26T23:37:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.593+0000 7fcf76000640 1 -- start start 2024-04-26T23:37:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf700b4000 0x7fcf70155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700d1f60 0x7fcf700ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7015f8c0 con 0x7fcf7015f4e0 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700b1d60 con 0x7fcf700d1f60 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.594+0000 7fcf76000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700b3290 con 0x7fcf700b4000 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48452/0 (socket says 172.21.15.46:48452) 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 -- 172.21.15.46:0/2254540838 learned_addr learned my addr 172.21.15.46:0/2254540838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700d1f60 0x7fcf700ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf74c00640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf700b4000 0x7fcf70155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 -- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf700b4000 msgr2=0x7fcf70155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf700b4000 0x7fcf70155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 -- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700d1f60 msgr2=0x7fcf700ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700d1f60 0x7fcf700ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.595+0000 7fcf75600640 1 -- 172.21.15.46:0/2254540838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf70155850 con 0x7fcf7015f4e0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf75600640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c7a0 tx=0x7fcf6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003b70 con 0x7fcf7015f4e0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68003d30 con 0x7fcf7015f4e0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800aa90 con 0x7fcf7015f4e0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf700cce20 con 0x7fcf7015f4e0 2024-04-26T23:37:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.596+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf700cd3e0 con 0x7fcf7015f4e0 2024-04-26T23:37:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.597+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf700693b0 con 0x7fcf7015f4e0 2024-04-26T23:37:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.598+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf6800ac30 con 0x7fcf7015f4e0 2024-04-26T23:37:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.600+0000 7fcf6cc00640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 0x7fcf5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.600+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf6808bcc0 con 0x7fcf7015f4e0 2024-04-26T23:37:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.600+0000 7fcf74c00640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 0x7fcf5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.601+0000 7fcf74c00640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 0x7fcf5006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009a30 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.605+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf68059e40 con 0x7fcf7015f4e0 2024-04-26T23:37:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.820+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf700d33d0 con 0x7fcf7015f4e0 2024-04-26T23:37:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.820+0000 7fcf6cc00640 1 -- 172.21.15.46:0/2254540838 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fcf68059740 con 0x7fcf7015f4e0 2024-04-26T23:37:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:37:27.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 msgr2=0x7fcf5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 0x7fcf5006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009a30 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2024-04-26T23:37:27.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 msgr2=0x7fcf700cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:27.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c7a0 tx=0x7fcf6800cc60 comp rx=0 tx=0).stop 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 shutdown_connections 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5006c8f0 0x7fcf5006edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7015f4e0 0x7fcf700cf080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf700d1f60 0x7fcf700ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 --2- 172.21.15.46:0/2254540838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf700b4000 0x7fcf70155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.824+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 >> 172.21.15.46:0/2254540838 conn(0x7fcf700d5010 msgr2=0x7fcf7006dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.825+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 shutdown_connections 2024-04-26T23:37:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:27.825+0000 7fcf76000640 1 -- 172.21.15.46:0/2254540838 wait complete. 2024-04-26T23:37:27.845 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-04-26T23:37:27.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:27.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:37:27.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[38700]: fuse finished with error 0 and tester_r 0 2024-04-26T23:37:27.938 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:37:34.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:37:34.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:34.041 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T23:37:34.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:34.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.317+0000 7f846b400640 1 Processor -- start 2024-04-26T23:37:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.317+0000 7f846b400640 1 -- start start 2024-04-26T23:37:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.319+0000 7f846b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 0x7f84641682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.319+0000 7f846b400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8464161940 con 0x7f846406e510 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.319+0000 7f846b400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8464071f80 con 0x7f8464160bd0 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.319+0000 7f846b400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84640742d0 con 0x7f8464165ee0 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.320+0000 7f8468a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f846406e510 0x7f846406e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40078/0 (socket says 172.21.15.46:40078) 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.320+0000 7f8468a00640 1 -- 172.21.15.46:0/201810563 learned_addr learned my addr 172.21.15.46:0/201810563 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.320+0000 7f8469400640 1 --2- 172.21.15.46:0/201810563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 0x7f84641682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.320+0000 7f8463e00640 1 -- 172.21.15.46:0/201810563 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225277552 0 0) 0x7f8464161940 con 0x7f846406e510 2024-04-26T23:37:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.320+0000 7f8463e00640 1 -- 172.21.15.46:0/201810563 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8444003520 con 0x7f846406e510 2024-04-26T23:37:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.321+0000 7f8469400640 1 -- 172.21.15.46:0/201810563 >> v1:172.21.15.142:6789/0 conn(0x7f8464160bd0 legacy=0x7f8464161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.321+0000 7f8469400640 1 -- 172.21.15.46:0/201810563 >> v1:172.21.15.46:6789/0 conn(0x7f846406e510 legacy=0x7f846406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.321+0000 7f8469400640 1 -- 172.21.15.46:0/201810563 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8464072260 con 0x7f8464165ee0 2024-04-26T23:37:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.322+0000 7f8469400640 1 --2- 172.21.15.46:0/201810563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 0x7f84641682d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8464074450 tx=0x7f845c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a04ba9bb10f8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.323+0000 7f8463e00640 1 -- 172.21.15.46:0/201810563 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c007740 con 0x7f8464165ee0 2024-04-26T23:37:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.323+0000 7f8463e00640 1 -- 172.21.15.46:0/201810563 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c0078e0 con 0x7f8464165ee0 2024-04-26T23:37:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.323+0000 7f8463e00640 1 -- 172.21.15.46:0/201810563 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c007cb0 con 0x7f8464165ee0 2024-04-26T23:37:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.323+0000 7f846b400640 1 -- 172.21.15.46:0/201810563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 msgr2=0x7f84641682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.323+0000 7f846b400640 1 --2- 172.21.15.46:0/201810563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 0x7f84641682d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8464074450 tx=0x7f845c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.324+0000 7f846b400640 1 -- 172.21.15.46:0/201810563 shutdown_connections 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.324+0000 7f846b400640 1 --2- 172.21.15.46:0/201810563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464165ee0 0x7f84641682d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.324+0000 7f846b400640 1 -- 172.21.15.46:0/201810563 >> 172.21.15.46:0/201810563 conn(0x7f846406c530 msgr2=0x7f846406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.324+0000 7f846b400640 1 -- 172.21.15.46:0/201810563 shutdown_connections 2024-04-26T23:37:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.324+0000 7f846b400640 1 -- 172.21.15.46:0/201810563 wait complete. 2024-04-26T23:37:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.326+0000 7f846b400640 1 Processor -- start 2024-04-26T23:37:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.326+0000 7f846b400640 1 -- start start 2024-04-26T23:37:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 0x7f84641397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 0x7f846413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8464072d40 con 0x7f8464165ee0 2024-04-26T23:37:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8464072820 con 0x7f846406e510 2024-04-26T23:37:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f846b400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8464072ec0 con 0x7f8464160bd0 2024-04-26T23:37:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f8469400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f8469400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37268/0 (socket says 172.21.15.46:37268) 2024-04-26T23:37:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f8469400640 1 -- 172.21.15.46:0/3602464275 learned_addr learned my addr 172.21.15.46:0/3602464275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f8468a00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 0x7f84641397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.327+0000 7f845be00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 0x7f846413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 -- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 msgr2=0x7f846413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 0x7f846413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 -- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 msgr2=0x7f84641397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 0x7f84641397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 -- 172.21.15.46:0/3602464275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846413dd80 con 0x7f8464165ee0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8468a00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 0x7f84641397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f845be00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 0x7f846413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8469400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f845c009ee0 tx=0x7f845c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c03f070 con 0x7f8464165ee0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c031bc0 con 0x7f8464165ee0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.328+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846413e040 con 0x7f8464165ee0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.329+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c044460 con 0x7f8464165ee0 2024-04-26T23:37:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.329+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846413e600 con 0x7f8464165ee0 2024-04-26T23:37:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.330+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8464069530 con 0x7f8464165ee0 2024-04-26T23:37:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.331+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f845c03b410 con 0x7f8464165ee0 2024-04-26T23:37:34.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.332+0000 7f8461600640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 0x7f844006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.332+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f845c0ad690 con 0x7f8464165ee0 2024-04-26T23:37:34.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.333+0000 7f8468a00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 0x7f844006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.334+0000 7f8468a00640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 0x7f844006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84540011e0 tx=0x7f8454009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.337+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f845c07b720 con 0x7f8464165ee0 2024-04-26T23:37:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.542+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f846406fb00 con 0x7f8464165ee0 2024-04-26T23:37:34.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.545+0000 7f8461600640 1 -- 172.21.15.46:0/3602464275 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f845c07b020 con 0x7f8464165ee0 2024-04-26T23:37:34.549 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:34.549 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 msgr2=0x7f844006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 0x7f844006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84540011e0 tx=0x7f8454009040 comp rx=0 tx=0).stop 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 msgr2=0x7f846413ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f845c009ee0 tx=0x7f845c004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 shutdown_connections 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f844006c8f0 0x7f844006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8464165ee0 0x7f846413ee00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8464160bd0 0x7f846413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 --2- 172.21.15.46:0/3602464275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846406e510 0x7f84641397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.547+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 >> 172.21.15.46:0/3602464275 conn(0x7f846406c530 msgr2=0x7f846415ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:34.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.548+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 shutdown_connections 2024-04-26T23:37:34.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.548+0000 7f846b400640 1 -- 172.21.15.46:0/3602464275 wait complete. 2024-04-26T23:37:34.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:34.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.772+0000 7f6efdc00640 1 Processor -- start 2024-04-26T23:37:34.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.772+0000 7f6efdc00640 1 -- start start 2024-04-26T23:37:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.773+0000 7f6efdc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ef8165ee0 0x7f6ef81682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.773+0000 7f6efdc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef8161940 con 0x7f6ef806e510 2024-04-26T23:37:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.773+0000 7f6efdc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef8071f80 con 0x7f6ef8160bd0 2024-04-26T23:37:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.773+0000 7f6efdc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef80742d0 con 0x7f6ef8165ee0 2024-04-26T23:37:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.774+0000 7f6ef6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ef806e510 0x7f6ef806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40092/0 (socket says 172.21.15.46:40092) 2024-04-26T23:37:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.774+0000 7f6ef6a00640 1 -- 172.21.15.46:0/1814456888 learned_addr learned my addr 172.21.15.46:0/1814456888 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.774+0000 7f6ef7400640 1 --2- 172.21.15.46:0/1814456888 >> v2:172.21.15.142:3300/0 conn(0x7f6ef8165ee0 0x7f6ef81682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.774+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375533692 0 0) 0x7f6ef8161940 con 0x7f6ef806e510 2024-04-26T23:37:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.774+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6edc003520 con 0x7f6ef806e510 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2314807052 0 0) 0x7f6edc003520 con 0x7f6ef806e510 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 >> v2:172.21.15.142:3300/0 conn(0x7f6ef8165ee0 msgr2=0x7f6ef81682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 --2- 172.21.15.46:0/1814456888 >> v2:172.21.15.142:3300/0 conn(0x7f6ef8165ee0 0x7f6ef81682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 >> v1:172.21.15.142:6789/0 conn(0x7f6ef8160bd0 legacy=0x7f6ef8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8072260 con 0x7f6ef806e510 2024-04-26T23:37:34.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6ee8001670 con 0x7f6ef806e510 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 >> v1:172.21.15.46:6789/0 conn(0x7f6ef806e510 legacy=0x7f6ef806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6edc0051c0 0x7f6edc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef80742d0 con 0x7f6edc0039b0 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc00c060 con 0x7f6edc004800 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc00c260 con 0x7f6edc0051c0 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.775+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6eefe00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef7400640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6edc0051c0 0x7f6edc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef6a00640 1 -- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6edc0051c0 msgr2=0x7f6edc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6edc0051c0 0x7f6edc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef6a00640 1 -- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004800 msgr2=0x7f6edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef6a00640 1 -- 172.21.15.46:0/1814456888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc003520 con 0x7f6edc0039b0 2024-04-26T23:37:34.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6eefe00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:34.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.776+0000 7f6ef7400640 1 -- 172.21.15.46:0/1814456888 reap_dead start 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.777+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c240 tx=0x7f6edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632692e7665459b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007cf0 con 0x7f6edc0039b0 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8005af0 con 0x7f6edc0039b0 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6ef6000640 1 -- 172.21.15.46:0/1814456888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007070 con 0x7f6edc0039b0 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 -- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 msgr2=0x7f6edc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c240 tx=0x7f6edc005830 comp rx=0 tx=0).stop 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 -- 172.21.15.46:0/1814456888 shutdown_connections 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 --2- 172.21.15.46:0/1814456888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 -- 172.21.15.46:0/1814456888 >> 172.21.15.46:0/1814456888 conn(0x7f6ef806c530 msgr2=0x7f6ef806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:34.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.778+0000 7f6efdc00640 1 -- 172.21.15.46:0/1814456888 shutdown_connections 2024-04-26T23:37:34.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.779+0000 7f6efdc00640 1 -- 172.21.15.46:0/1814456888 wait complete. 2024-04-26T23:37:34.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.780+0000 7f6efdc00640 1 Processor -- start 2024-04-26T23:37:34.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.780+0000 7f6efdc00640 1 -- start start 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef81f81f0 0x7f6ef80b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 0x7f6ef8145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8072d40 con 0x7f6edc0039b0 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8072820 con 0x7f6ef80b5640 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.781+0000 7f6efdc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef8072ec0 con 0x7f6ef81f81f0 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37282/0 (socket says 172.21.15.46:37282) 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 -- 172.21.15.46:0/4150238464 learned_addr learned my addr 172.21.15.46:0/4150238464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef7400640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 0x7f6ef8145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6eefe00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef81f81f0 0x7f6ef80b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 -- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef81f81f0 msgr2=0x7f6ef80b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef81f81f0 0x7f6ef80b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 -- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 msgr2=0x7f6ef8145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 0x7f6ef8145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 -- 172.21.15.46:0/4150238464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef80b15c0 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef7400640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 0x7f6ef8145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.782+0000 7f6ef6a00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c240 tx=0x7f6ef8072150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.783+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8041020 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.783+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8038040 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.783+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8001670 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.783+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef815c600 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.783+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef80b1840 con 0x7f6edc0039b0 2024-04-26T23:37:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.784+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8069530 con 0x7f6edc0039b0 2024-04-26T23:37:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.785+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ee8044020 con 0x7f6edc0039b0 2024-04-26T23:37:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.786+0000 7f6eeea00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.786+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6ee80ac1d0 con 0x7f6edc0039b0 2024-04-26T23:37:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.787+0000 7f6eefe00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.788+0000 7f6eefe00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 0x7f6ed406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0007fd0 tx=0x7f6ee0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.790+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ef80b15c0 con 0x7f6edc0039b0 2024-04-26T23:37:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.994+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ef806fb00 con 0x7f6edc0039b0 2024-04-26T23:37:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:34.996+0000 7f6eeea00640 1 -- 172.21.15.46:0/4150238464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f6ee807a220 con 0x7f6edc0039b0 2024-04-26T23:37:35.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:35.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 msgr2=0x7f6ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 0x7f6ed406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0007fd0 tx=0x7f6ee0009040 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 msgr2=0x7f6ef815e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c240 tx=0x7f6ef8072150 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 shutdown_connections 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef80b5640 0x7f6ef8145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef81f81f0 0x7f6ef80b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 --2- 172.21.15.46:0/4150238464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc0039b0 0x7f6ef815e6a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 >> 172.21.15.46:0/4150238464 conn(0x7f6ef806c530 msgr2=0x7f6ef806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 shutdown_connections 2024-04-26T23:37:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.000+0000 7f6efdc00640 1 -- 172.21.15.46:0/4150238464 wait complete. 2024-04-26T23:37:35.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:37:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.235+0000 7f6ba7e00640 1 Processor -- start 2024-04-26T23:37:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.235+0000 7f6ba7e00640 1 -- start start 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba80d07d0 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80d0ab0 con 0x7f6ba81626c0 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80d0d90 con 0x7f6ba8162aa0 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6b9fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ba80b5fb0 0x7f6ba80d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40106/0 (socket says 172.21.15.46:40106) 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6b9fe00640 1 -- 172.21.15.46:0/32256217 learned_addr learned my addr 172.21.15.46:0/32256217 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37296/0 (socket says 172.21.15.46:37296) 2024-04-26T23:37:35.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.237+0000 7f6ba7400640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:35.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.238+0000 7f6ba6a00640 1 -- 172.21.15.46:0/32256217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 msgr2=0x7f6ba80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.238+0000 7f6ba6a00640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.238+0000 7f6ba6a00640 1 -- 172.21.15.46:0/32256217 >> v1:172.21.15.46:6789/0 conn(0x7f6ba80b5fb0 legacy=0x7f6ba80d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.238+0000 7f6ba6a00640 1 -- 172.21.15.46:0/32256217 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80d2c80 con 0x7f6ba81626c0 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.238+0000 7f6ba6000640 1 -- 172.21.15.46:0/32256217 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695840566 0 0) 0x7f6ba80d07d0 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba6a00640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6b980097a0 tx=0x7f6b9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8b2f0f4d514c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba6000640 1 -- 172.21.15.46:0/32256217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037070 con 0x7f6ba81626c0 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba6000640 1 -- 172.21.15.46:0/32256217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba81626c0 2024-04-26T23:37:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba6000640 1 -- 172.21.15.46:0/32256217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98032810 con 0x7f6ba81626c0 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba7e00640 1 -- 172.21.15.46:0/32256217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba81626c0 msgr2=0x7f6ba80b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.239+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6b980097a0 tx=0x7f6b9802a180 comp rx=0 tx=0).stop 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 -- 172.21.15.46:0/32256217 shutdown_connections 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/32256217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 -- 172.21.15.46:0/32256217 >> 172.21.15.46:0/32256217 conn(0x7f6ba809e2a0 msgr2=0x7f6ba80d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 -- 172.21.15.46:0/32256217 shutdown_connections 2024-04-26T23:37:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.240+0000 7f6ba7e00640 1 -- 172.21.15.46:0/32256217 wait complete. 2024-04-26T23:37:35.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.241+0000 7f6ba7e00640 1 Processor -- start 2024-04-26T23:37:35.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 -- start start 2024-04-26T23:37:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80d2210 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80d2090 con 0x7f6ba81626c0 2024-04-26T23:37:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.242+0000 7f6ba7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80d31a0 con 0x7f6ba8162aa0 2024-04-26T23:37:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37302/0 (socket says 172.21.15.46:37302) 2024-04-26T23:37:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 -- 172.21.15.46:0/3041285072 learned_addr learned my addr 172.21.15.46:0/3041285072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba7400640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 -- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 msgr2=0x7f6ba80b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 -- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ba81626c0 msgr2=0x7f6ba80b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 -- 172.21.15.46:0/3041285072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba815da80 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.243+0000 7f6ba6a00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b98035040 tx=0x7f6b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.244+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037040 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.244+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98030770 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.244+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98030b80 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.244+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba80ccb40 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.244+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba80cd100 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.245+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba80693b0 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.246+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b98044020 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.247+0000 7f6b9ea00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 0x7f6b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.247+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b980ac480 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.248+0000 7f6b9fe00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 0x7f6b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.249+0000 7f6b9fe00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 0x7f6b8406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007fd0 tx=0x7f6b900077c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:35.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.251+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b98032e20 con 0x7f6ba80b5fb0 2024-04-26T23:37:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:35.496+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6ba80d25b0 con 0x7f6ba80b5fb0 2024-04-26T23:37:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.011+0000 7f6b9ea00640 1 -- 172.21.15.46:0/3041285072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f6b9807a4d0 con 0x7f6ba80b5fb0 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.014+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 msgr2=0x7f6b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.014+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 0x7f6b8406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007fd0 tx=0x7f6b900077c0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.014+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 msgr2=0x7f6ba8158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.014+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b98035040 tx=0x7f6b980046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 shutdown_connections 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8406c8f0 0x7f6b8406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ba8162aa0 0x7f6ba80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ba81626c0 0x7f6ba80b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 --2- 172.21.15.46:0/3041285072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80b5fb0 0x7f6ba8158470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 >> 172.21.15.46:0/3041285072 conn(0x7f6ba809e2a0 msgr2=0x7f6ba80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:36.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 shutdown_connections 2024-04-26T23:37:36.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.015+0000 7f6ba7e00640 1 -- 172.21.15.46:0/3041285072 wait complete. 2024-04-26T23:37:36.042 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T23:37:36.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:37:36.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.250+0000 7f20fd000640 1 Processor -- start 2024-04-26T23:37:36.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.251+0000 7f20fd000640 1 -- start start 2024-04-26T23:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.252+0000 7f20fd000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f20f81597e0 0x7f20f80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.252+0000 7f20fd000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 0x7f20f80b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.252+0000 7f20fd000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f8073120 con 0x7f20f80b5730 2024-04-26T23:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.252+0000 7f20fd000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f20f8159bc0 con 0x7f20f81597e0 2024-04-26T23:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20fd000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20f8072b60 con 0x7f20f8072620 2024-04-26T23:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20f5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f20f80b5730 0x7f20f8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40112/0 (socket says 172.21.15.46:40112) 2024-04-26T23:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20f5400640 1 -- 172.21.15.46:0/2858661089 learned_addr learned my addr 172.21.15.46:0/2858661089 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20f5e00640 1 --2- 172.21.15.46:0/2858661089 >> v2:172.21.15.142:3300/0 conn(0x7f20f81597e0 0x7f20f80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20f6800640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 0x7f20f80b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.253+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377225727 0 0) 0x7f20f8073120 con 0x7f20f80b5730 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20dc003520 con 0x7f20f80b5730 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1470783558 0 0) 0x7f20dc003520 con 0x7f20f80b5730 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 msgr2=0x7f20f80b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 0x7f20f80b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 >> v2:172.21.15.142:3300/0 conn(0x7f20f81597e0 msgr2=0x7f20f80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 --2- 172.21.15.46:0/2858661089 >> v2:172.21.15.142:3300/0 conn(0x7f20f81597e0 0x7f20f80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8072e40 con 0x7f20f80b5730 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f20e0001670 con 0x7f20f80b5730 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 >> v1:172.21.15.46:6789/0 conn(0x7f20f80b5730 legacy=0x7f20f8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20dc0039b0 0x7f20dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 0x7f20dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc004d40 0x7f20dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8159bc0 con 0x7f20dc0039b0 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8072b60 con 0x7f20dc004d40 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc00b9f0 con 0x7f20f8072620 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.254+0000 7f20f5e00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20dc0039b0 0x7f20dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.255+0000 7f20f5e00640 1 -- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 msgr2=0x7f20dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.255+0000 7f20f5e00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8072620 0x7f20dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.255+0000 7f20f5e00640 1 -- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc004d40 msgr2=0x7f20dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.255+0000 7f20f5e00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc004d40 0x7f20dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.255+0000 7f20f5e00640 1 -- 172.21.15.46:0/2858661089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc003520 con 0x7f20dc0039b0 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.256+0000 7f20f5e00640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20dc0039b0 0x7f20dc003dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ac0 tx=0x7f20dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2996408ce8777db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.256+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8036070 con 0x7f20dc0039b0 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.256+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e8031400 con 0x7f20dc0039b0 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.256+0000 7f20f4a00640 1 -- 172.21.15.46:0/2858661089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8009690 con 0x7f20dc0039b0 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 -- 172.21.15.46:0/2858661089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20dc0039b0 msgr2=0x7f20dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 --2- 172.21.15.46:0/2858661089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20dc0039b0 0x7f20dc003dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ac0 tx=0x7f20dc005830 comp rx=0 tx=0).stop 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20f6800640 1 -- 172.21.15.46:0/2858661089 reap_dead start 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 -- 172.21.15.46:0/2858661089 shutdown_connections 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 -- 172.21.15.46:0/2858661089 >> 172.21.15.46:0/2858661089 conn(0x7f20f806c980 msgr2=0x7f20f80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 -- 172.21.15.46:0/2858661089 shutdown_connections 2024-04-26T23:37:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.257+0000 7f20fd000640 1 -- 172.21.15.46:0/2858661089 wait complete. 2024-04-26T23:37:36.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.259+0000 7f20fd000640 1 Processor -- start 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.259+0000 7f20fd000640 1 -- start start 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc0039b0 0x7f20f80cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 0x7f20f8160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80aee20 con 0x7f20f80cd6b0 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80b43b0 con 0x7f20dc0039b0 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20fd000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20f80aefa0 con 0x7f20f8160430 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37314/0 (socket says 172.21.15.46:37314) 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20f5400640 1 -- 172.21.15.46:0/285584602 learned_addr learned my addr 172.21.15.46:0/285584602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20f5e00640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc0039b0 0x7f20f80cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.260+0000 7f20f6800640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 0x7f20f8160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 -- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 msgr2=0x7f20f8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 0x7f20f8160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 -- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc0039b0 msgr2=0x7f20f80cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:36.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc0039b0 0x7f20f80cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:36.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 -- 172.21.15.46:0/285584602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f80cde50 con 0x7f20f80cd6b0 2024-04-26T23:37:36.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f6800640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 0x7f20f8160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:36.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20f5400640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c7a0 tx=0x7f20e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:36.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0018070 con 0x7f20f80cd6b0 2024-04-26T23:37:36.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e00074e0 con 0x7f20f80cd6b0 2024-04-26T23:37:36.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0004a20 con 0x7f20f80cd6b0 2024-04-26T23:37:36.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.261+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f80ce130 con 0x7f20f80cd6b0 2024-04-26T23:37:36.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.262+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f80ce6f0 con 0x7f20f80cd6b0 2024-04-26T23:37:36.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.263+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8069530 con 0x7f20f80cd6b0 2024-04-26T23:37:36.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.264+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20e001b070 con 0x7f20f80cd6b0 2024-04-26T23:37:36.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.265+0000 7f20ee000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 0x7f20d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:36.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.265+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f20e008b440 con 0x7f20f80cd6b0 2024-04-26T23:37:36.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.265+0000 7f20f5e00640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 0x7f20d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:36.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.266+0000 7f20f5e00640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 0x7f20d406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ac0 tx=0x7f20e8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:36.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.269+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20e0058e40 con 0x7f20f80cd6b0 2024-04-26T23:37:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:36.513+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f20f8158070 con 0x7f20f80cd6b0 2024-04-26T23:37:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.014+0000 7f20ee000640 1 -- 172.21.15.46:0/285584602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f20e005d140 con 0x7f20f80cd6b0 2024-04-26T23:37:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 msgr2=0x7f20d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 0x7f20d406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ac0 tx=0x7f20e8008030 comp rx=0 tx=0).stop 2024-04-26T23:37:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 msgr2=0x7f20f8155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c7a0 tx=0x7f20e000cc60 comp rx=0 tx=0).stop 2024-04-26T23:37:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 shutdown_connections 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20d406c8f0 0x7f20d406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20f8160430 0x7f20f8160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f80cd6b0 0x7f20f8155840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 --2- 172.21.15.46:0/285584602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20dc0039b0 0x7f20f80cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 >> 172.21.15.46:0/285584602 conn(0x7f20f806c980 msgr2=0x7f20f806f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 shutdown_connections 2024-04-26T23:37:37.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.018+0000 7f20fd000640 1 -- 172.21.15.46:0/285584602 wait complete. 2024-04-26T23:37:37.048 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:37:37.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.259+0000 7f3a20200640 1 Processor -- start 2024-04-26T23:37:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.260+0000 7f3a20200640 1 -- start start 2024-04-26T23:37:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.261+0000 7f3a20200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a1806e510 0x7f3a1806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.261+0000 7f3a20200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a18160bd0 0x7f3a18161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.261+0000 7f3a20200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1806ee30 con 0x7f3a1815d540 2024-04-26T23:37:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.261+0000 7f3a20200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3a180742d0 con 0x7f3a1806e510 2024-04-26T23:37:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.261+0000 7f3a20200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3a18161940 con 0x7f3a18160bd0 2024-04-26T23:37:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a1806e510 0x7f3a1806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a1806e510 0x7f3a1806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-04-26T23:37:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 -- 172.21.15.46:0/2668190395 learned_addr learned my addr 172.21.15.46:0/2668190395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1ce00640 1 --2- 172.21.15.46:0/2668190395 >> v2:172.21.15.142:3300/0 conn(0x7f3a18160bd0 0x7f3a18161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 -- 172.21.15.46:0/2668190395 >> v2:172.21.15.142:3300/0 conn(0x7f3a18160bd0 msgr2=0x7f3a18161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 --2- 172.21.15.46:0/2668190395 >> v2:172.21.15.142:3300/0 conn(0x7f3a18160bd0 0x7f3a18161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 -- 172.21.15.46:0/2668190395 >> v1:172.21.15.142:6789/0 conn(0x7f3a1815d540 legacy=0x7f3a1815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.262+0000 7f3a1d800640 1 -- 172.21.15.46:0/2668190395 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a18071f80 con 0x7f3a1806e510 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.263+0000 7f3a1d800640 1 --2- 172.21.15.46:0/2668190395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a1806e510 0x7f3a1806e8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009870 tx=0x7f3a0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6afa1012e2b1784 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a17e00640 1 -- 172.21.15.46:0/2668190395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c037070 con 0x7f3a1806e510 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a17e00640 1 -- 172.21.15.46:0/2668190395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c032440 con 0x7f3a1806e510 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a17e00640 1 -- 172.21.15.46:0/2668190395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c032810 con 0x7f3a1806e510 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 -- 172.21.15.46:0/2668190395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a1806e510 msgr2=0x7f3a1806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 --2- 172.21.15.46:0/2668190395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a1806e510 0x7f3a1806e8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009870 tx=0x7f3a0c02a270 comp rx=0 tx=0).stop 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 -- 172.21.15.46:0/2668190395 shutdown_connections 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 --2- 172.21.15.46:0/2668190395 >> v2:172.21.15.142:3300/0 conn(0x7f3a18160bd0 0x7f3a18161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 --2- 172.21.15.46:0/2668190395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a1806e510 0x7f3a1806e8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.264+0000 7f3a20200640 1 -- 172.21.15.46:0/2668190395 >> 172.21.15.46:0/2668190395 conn(0x7f3a1806c530 msgr2=0x7f3a1806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.265+0000 7f3a20200640 1 -- 172.21.15.46:0/2668190395 shutdown_connections 2024-04-26T23:37:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.265+0000 7f3a20200640 1 -- 172.21.15.46:0/2668190395 wait complete. 2024-04-26T23:37:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.267+0000 7f3a20200640 1 Processor -- start 2024-04-26T23:37:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.267+0000 7f3a20200640 1 -- start start 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 0x7f3a180afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 0x7f3a180b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18072820 con 0x7f3a18160bd0 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18072540 con 0x7f3a1806e510 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.268+0000 7f3a20200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3a181615a0 con 0x7f3a1815d540 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37334/0 (socket says 172.21.15.46:37334) 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 -- 172.21.15.46:0/3822541147 learned_addr learned my addr 172.21.15.46:0/3822541147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1d800640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 0x7f3a180afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1ce00640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 0x7f3a180b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 -- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 msgr2=0x7f3a180b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 0x7f3a180b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 -- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 msgr2=0x7f3a180afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 0x7f3a180afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 -- 172.21.15.46:0/3822541147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a180b4120 con 0x7f3a18160bd0 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1ce00640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 0x7f3a180b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1d800640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 0x7f3a180afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:37.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.269+0000 7f3a1e200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c780 tx=0x7f3a1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.270+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10004b60 con 0x7f3a18160bd0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.270+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10004d00 con 0x7f3a18160bd0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.270+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a180b43c0 con 0x7f3a18160bd0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.270+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000aa90 con 0x7f3a18160bd0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.270+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a180b4980 con 0x7f3a18160bd0 2024-04-26T23:37:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.271+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a18069530 con 0x7f3a18160bd0 2024-04-26T23:37:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.272+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a1000ac30 con 0x7f3a18160bd0 2024-04-26T23:37:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.273+0000 7f3a15600640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.273+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3a1008c1f0 con 0x7f3a18160bd0 2024-04-26T23:37:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.274+0000 7f3a1d800640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.275+0000 7f3a1d800640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 0x7f39f806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.278+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a10020070 con 0x7f3a18160bd0 2024-04-26T23:37:37.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.524+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a1806fb00 con 0x7f3a18160bd0 2024-04-26T23:37:37.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.525+0000 7f3a15600640 1 -- 172.21.15.46:0/3822541147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f3a1005a240 con 0x7f3a18160bd0 2024-04-26T23:37:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:37.529 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:37.529 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:37:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 msgr2=0x7f39f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 0x7f39f806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02e000 comp rx=0 tx=0).stop 2024-04-26T23:37:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 msgr2=0x7f3a1815b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c780 tx=0x7f3a1000cc40 comp rx=0 tx=0).stop 2024-04-26T23:37:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 shutdown_connections 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a18160bd0 0x7f3a1815b990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a1815d540 0x7f3a180b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 --2- 172.21.15.46:0/3822541147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a1806e510 0x7f3a180afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 >> 172.21.15.46:0/3822541147 conn(0x7f3a1806c530 msgr2=0x7f3a180b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 shutdown_connections 2024-04-26T23:37:37.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.528+0000 7f3a20200640 1 -- 172.21.15.46:0/3822541147 wait complete. 2024-04-26T23:37:37.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:37.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:37.730 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:37.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.745+0000 7f247f400640 1 Processor -- start 2024-04-26T23:37:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.746+0000 7f247f400640 1 -- start start 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f247f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24780d1af0 0x7f24780d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f247f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f247815ed30 0x7f247815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f247f400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247815fd30 con 0x7f2478162980 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f247f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2478162d60 con 0x7f24780d1af0 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f247f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f24780d1050 con 0x7f247815ed30 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f2477e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24780d1af0 0x7f24780d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f2477e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24780d1af0 0x7f24780d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37342/0 (socket says 172.21.15.46:37342) 2024-04-26T23:37:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.747+0000 7f2477e00640 1 -- 172.21.15.46:0/3300729878 learned_addr learned my addr 172.21.15.46:0/3300729878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:37.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.748+0000 7f247d400640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f247815ed30 0x7f247815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.748+0000 7f2477e00640 1 -- 172.21.15.46:0/3300729878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f247815ed30 msgr2=0x7f247815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:37.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.748+0000 7f2477e00640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f247815ed30 0x7f247815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.748+0000 7f2477e00640 1 -- 172.21.15.46:0/3300729878 >> v1:172.21.15.142:6789/0 conn(0x7f2478162980 legacy=0x7f24780d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:37.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.748+0000 7f2477e00640 1 -- 172.21.15.46:0/3300729878 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2478160010 con 0x7f24780d1af0 2024-04-26T23:37:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.749+0000 7f2477e00640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24780d1af0 0x7f24780d1f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2464009870 tx=0x7f246402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac348b09aa58cb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.750+0000 7f2477400640 1 -- 172.21.15.46:0/3300729878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464037070 con 0x7f24780d1af0 2024-04-26T23:37:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.750+0000 7f2477400640 1 -- 172.21.15.46:0/3300729878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2464032440 con 0x7f24780d1af0 2024-04-26T23:37:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.750+0000 7f2477400640 1 -- 172.21.15.46:0/3300729878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464032810 con 0x7f24780d1af0 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 -- 172.21.15.46:0/3300729878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24780d1af0 msgr2=0x7f24780d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24780d1af0 0x7f24780d1f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2464009870 tx=0x7f246402a270 comp rx=0 tx=0).stop 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 -- 172.21.15.46:0/3300729878 shutdown_connections 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f247815ed30 0x7f247815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 --2- 172.21.15.46:0/3300729878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24780d1af0 0x7f24780d1f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 -- 172.21.15.46:0/3300729878 >> 172.21.15.46:0/3300729878 conn(0x7f247809e420 msgr2=0x7f24780d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 -- 172.21.15.46:0/3300729878 shutdown_connections 2024-04-26T23:37:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.751+0000 7f247f400640 1 -- 172.21.15.46:0/3300729878 wait complete. 2024-04-26T23:37:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.753+0000 7f247f400640 1 Processor -- start 2024-04-26T23:37:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.753+0000 7f247f400640 1 -- start start 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 0x7f24780cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 0x7f247815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24781608b0 con 0x7f247815ed30 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24781605d0 con 0x7f2478162980 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f247f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2478160d50 con 0x7f24780d1af0 2024-04-26T23:37:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.754+0000 7f2477e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37346/0 (socket says 172.21.15.46:37346) 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 -- 172.21.15.46:0/1408159714 learned_addr learned my addr 172.21.15.46:0/1408159714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f247ca00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 0x7f24780cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f247d400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 0x7f247815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 -- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 msgr2=0x7f24780cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 0x7f24780cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 -- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 msgr2=0x7f247815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 0x7f247815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 -- 172.21.15.46:0/1408159714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24780c95d0 con 0x7f247815ed30 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f247d400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 0x7f247815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f247ca00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 0x7f24780cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:37.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.755+0000 7f2477e00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f24640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.756+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464037040 con 0x7f247815ed30 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.756+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2464042070 con 0x7f247815ed30 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.756+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464030880 con 0x7f247815ed30 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.756+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24780c98b0 con 0x7f247815ed30 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.756+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24780c9e70 con 0x7f247815ed30 2024-04-26T23:37:37.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.757+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2478069530 con 0x7f247815ed30 2024-04-26T23:37:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.759+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2464030a20 con 0x7f247815ed30 2024-04-26T23:37:37.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.760+0000 7f2474c00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 0x7f245806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:37.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.761+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24640ad230 con 0x7f247815ed30 2024-04-26T23:37:37.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.761+0000 7f247ca00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 0x7f245806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:37.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.762+0000 7f247ca00640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 0x7f245806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24780cf690 tx=0x7f246c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.764+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f246403a390 con 0x7f247815ed30 2024-04-26T23:37:37.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.970+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24780d3d00 con 0x7f247815ed30 2024-04-26T23:37:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.973+0000 7f2474c00640 1 -- 172.21.15.46:0/1408159714 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f246407b300 con 0x7f247815ed30 2024-04-26T23:37:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:37.978 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 msgr2=0x7f245806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 0x7f245806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24780cf690 tx=0x7f246c009040 comp rx=0 tx=0).stop 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 msgr2=0x7f24780c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f24640046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 shutdown_connections 2024-04-26T23:37:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f245806c8f0 0x7f245806edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2478162980 0x7f247815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f247815ed30 0x7f24780c8bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 --2- 172.21.15.46:0/1408159714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24780d1af0 0x7f24780cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 >> 172.21.15.46:0/1408159714 conn(0x7f247809e420 msgr2=0x7f247806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 shutdown_connections 2024-04-26T23:37:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:37.976+0000 7f247f400640 1 -- 172.21.15.46:0/1408159714 wait complete. 2024-04-26T23:37:38.000 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:37:38.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:37:38.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.189+0000 7fba9d400640 1 Processor -- start 2024-04-26T23:37:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.190+0000 7fba9d400640 1 -- start start 2024-04-26T23:37:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815f4e0 0x7fba980d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba9d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba980d2130 0x7fba980b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba9d400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980d11f0 con 0x7fba980b4000 2024-04-26T23:37:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba9d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fba980d0ce0 con 0x7fba9815f4e0 2024-04-26T23:37:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba9d400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fba980d2990 con 0x7fba980d2130 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.191+0000 7fba97e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815f4e0 0x7fba980d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815f4e0 0x7fba980d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37358/0 (socket says 172.21.15.46:37358) 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 -- 172.21.15.46:0/1734535766 learned_addr learned my addr 172.21.15.46:0/1734535766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba9ca00640 1 --2- 172.21.15.46:0/1734535766 >> v2:172.21.15.142:3300/0 conn(0x7fba980d2130 0x7fba980b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 -- 172.21.15.46:0/1734535766 >> v2:172.21.15.142:3300/0 conn(0x7fba980d2130 msgr2=0x7fba980b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 --2- 172.21.15.46:0/1734535766 >> v2:172.21.15.142:3300/0 conn(0x7fba980d2130 0x7fba980b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 -- 172.21.15.46:0/1734535766 >> v1:172.21.15.142:6789/0 conn(0x7fba980b4000 legacy=0x7fba980d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:38.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.192+0000 7fba97e00640 1 -- 172.21.15.46:0/1734535766 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980d5f20 con 0x7fba9815f4e0 2024-04-26T23:37:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.193+0000 7fba97e00640 1 --2- 172.21.15.46:0/1734535766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815f4e0 0x7fba980d2df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba980d4e20 tx=0x7fba8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a596052ba205781d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.194+0000 7fba96a00640 1 -- 172.21.15.46:0/1734535766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88037070 con 0x7fba9815f4e0 2024-04-26T23:37:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.194+0000 7fba96a00640 1 -- 172.21.15.46:0/1734535766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba88032440 con 0x7fba9815f4e0 2024-04-26T23:37:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.194+0000 7fba96a00640 1 -- 172.21.15.46:0/1734535766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88032810 con 0x7fba9815f4e0 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 -- 172.21.15.46:0/1734535766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815f4e0 msgr2=0x7fba980d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 --2- 172.21.15.46:0/1734535766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815f4e0 0x7fba980d2df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba980d4e20 tx=0x7fba8802a230 comp rx=0 tx=0).stop 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 -- 172.21.15.46:0/1734535766 shutdown_connections 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 --2- 172.21.15.46:0/1734535766 >> v2:172.21.15.142:3300/0 conn(0x7fba980d2130 0x7fba980b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 --2- 172.21.15.46:0/1734535766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815f4e0 0x7fba980d2df0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 -- 172.21.15.46:0/1734535766 >> 172.21.15.46:0/1734535766 conn(0x7fba980d5010 msgr2=0x7fba980d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 -- 172.21.15.46:0/1734535766 shutdown_connections 2024-04-26T23:37:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.195+0000 7fba9d400640 1 -- 172.21.15.46:0/1734535766 wait complete. 2024-04-26T23:37:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.197+0000 7fba9d400640 1 Processor -- start 2024-04-26T23:37:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.197+0000 7fba9d400640 1 -- start start 2024-04-26T23:37:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.198+0000 7fba9d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 0x7fba980cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 0x7fba98160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba980b1d60 con 0x7fba980d2130 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba980d5c40 con 0x7fba9815f4e0 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9d400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba9815f8c0 con 0x7fba980b4000 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37366/0 (socket says 172.21.15.46:37366) 2024-04-26T23:37:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba97400640 1 -- 172.21.15.46:0/2277240619 learned_addr learned my addr 172.21.15.46:0/2277240619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba9ca00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 0x7fba98160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.199+0000 7fba97e00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 0x7fba980cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 -- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 msgr2=0x7fba980cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 0x7fba980cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 -- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 msgr2=0x7fba98160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 0x7fba98160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 -- 172.21.15.46:0/2277240619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba98160750 con 0x7fba980d2130 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba9ca00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 0x7fba98160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97e00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 0x7fba980cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba97400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c430 tx=0x7fba8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84011790 con 0x7fba980d2130 2024-04-26T23:37:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba84017070 con 0x7fba980d2130 2024-04-26T23:37:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba98160a30 con 0x7fba980d2130 2024-04-26T23:37:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.200+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84011bb0 con 0x7fba980d2130 2024-04-26T23:37:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.201+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba98160ff0 con 0x7fba980d2130 2024-04-26T23:37:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.202+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba980693b0 con 0x7fba980d2130 2024-04-26T23:37:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.203+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba8401c460 con 0x7fba980d2130 2024-04-26T23:37:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.204+0000 7fba8fe00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.204+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba8408bdb0 con 0x7fba980d2130 2024-04-26T23:37:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.205+0000 7fba97e00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.206+0000 7fba97e00640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 0x7fba7806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba88034000 tx=0x7fba8802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.209+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba84090070 con 0x7fba980d2130 2024-04-26T23:37:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.408+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba980d33d0 con 0x7fba7806c920 2024-04-26T23:37:38.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.441+0000 7fba8fe00640 1 -- 172.21.15.46:0/2277240619 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7fba88093310 con 0x7fba7806c920 2024-04-26T23:37:38.450 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:37:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 msgr2=0x7fba7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 0x7fba7806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba88034000 tx=0x7fba8802e000 comp rx=0 tx=0).stop 2024-04-26T23:37:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 msgr2=0x7fba9815fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c430 tx=0x7fba8400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 shutdown_connections 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9815f4e0 0x7fba98160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba980d2130 0x7fba9815fad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 --2- 172.21.15.46:0/2277240619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b4000 0x7fba980cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 >> 172.21.15.46:0/2277240619 conn(0x7fba980d5010 msgr2=0x7fba9806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 shutdown_connections 2024-04-26T23:37:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.447+0000 7fba9d400640 1 -- 172.21.15.46:0/2277240619 wait complete. 2024-04-26T23:37:38.471 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T23:37:37.892044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223640,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751080,"statfs":{"total":773094113280,"available":772865105920,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004993"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795609+0000","last_change":"2024-04-26T23:37:07.740855+0000","last_active":"2024-04-26T23:37:13.795609+0000","last_peered":"2024-04-26T23:37:13.795609+0000","last_clean":"2024-04-26T23:37:13.795609+0000","last_became_active":"2024-04-26T23:37:07.739396+0000","last_became_peered":"2024-04-26T23:37:07.739396+0000","last_unstale":"2024-04-26T23:37:13.795609+0000","last_undegraded":"2024-04-26T23:37:13.795609+0000","last_fullsized":"2024-04-26T23:37:13.795609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:05.866636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892052+0000","last_change":"2024-04-26T23:37:08.773954+0000","last_active":"2024-04-26T23:37:13.892052+0000","last_peered":"2024-04-26T23:37:13.892052+0000","last_clean":"2024-04-26T23:37:13.892052+0000","last_became_active":"2024-04-26T23:37:08.771001+0000","last_became_peered":"2024-04-26T23:37:08.771001+0000","last_unstale":"2024-04-26T23:37:13.892052+0000","last_undegraded":"2024-04-26T23:37:13.892052+0000","last_fullsized":"2024-04-26T23:37:13.892052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890219+0000","last_change":"2024-04-26T23:37:13.098670+0000","last_active":"2024-04-26T23:37:13.890219+0000","last_peered":"2024-04-26T23:37:13.890219+0000","last_clean":"2024-04-26T23:37:13.890219+0000","last_became_active":"2024-04-26T23:37:13.098244+0000","last_became_peered":"2024-04-26T23:37:13.098244+0000","last_unstale":"2024-04-26T23:37:13.890219+0000","last_undegraded":"2024-04-26T23:37:13.890219+0000","last_fullsized":"2024-04-26T23:37:13.890219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892226+0000","last_change":"2024-04-26T23:37:07.729205+0000","last_active":"2024-04-26T23:37:13.892226+0000","last_peered":"2024-04-26T23:37:13.892226+0000","last_clean":"2024-04-26T23:37:13.892226+0000","last_became_active":"2024-04-26T23:37:07.728871+0000","last_became_peered":"2024-04-26T23:37:07.728871+0000","last_unstale":"2024-04-26T23:37:13.892226+0000","last_undegraded":"2024-04-26T23:37:13.892226+0000","last_fullsized":"2024-04-26T23:37:13.892226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795238+0000","last_change":"2024-04-26T23:37:08.793071+0000","last_active":"2024-04-26T23:37:13.795238+0000","last_peered":"2024-04-26T23:37:13.795238+0000","last_clean":"2024-04-26T23:37:13.795238+0000","last_became_active":"2024-04-26T23:37:08.792784+0000","last_became_peered":"2024-04-26T23:37:08.792784+0000","last_unstale":"2024-04-26T23:37:13.795238+0000","last_undegraded":"2024-04-26T23:37:13.795238+0000","last_fullsized":"2024-04-26T23:37:13.795238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798412+0000","last_change":"2024-04-26T23:37:12.830234+0000","last_active":"2024-04-26T23:37:13.798412+0000","last_peered":"2024-04-26T23:37:13.798412+0000","last_clean":"2024-04-26T23:37:13.798412+0000","last_became_active":"2024-04-26T23:37:12.829735+0000","last_became_peered":"2024-04-26T23:37:12.829735+0000","last_unstale":"2024-04-26T23:37:13.798412+0000","last_undegraded":"2024-04-26T23:37:13.798412+0000","last_fullsized":"2024-04-26T23:37:13.798412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:01.085576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893008+0000","last_change":"2024-04-26T23:37:07.755959+0000","last_active":"2024-04-26T23:37:13.893008+0000","last_peered":"2024-04-26T23:37:13.893008+0000","last_clean":"2024-04-26T23:37:13.893008+0000","last_became_active":"2024-04-26T23:37:07.754315+0000","last_became_peered":"2024-04-26T23:37:07.754315+0000","last_unstale":"2024-04-26T23:37:13.893008+0000","last_undegraded":"2024-04-26T23:37:13.893008+0000","last_fullsized":"2024-04-26T23:37:13.893008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802750+0000","last_change":"2024-04-26T23:37:08.786529+0000","last_active":"2024-04-26T23:37:13.802750+0000","last_peered":"2024-04-26T23:37:13.802750+0000","last_clean":"2024-04-26T23:37:13.802750+0000","last_became_active":"2024-04-26T23:37:08.781335+0000","last_became_peered":"2024-04-26T23:37:08.781335+0000","last_unstale":"2024-04-26T23:37:13.802750+0000","last_undegraded":"2024-04-26T23:37:13.802750+0000","last_fullsized":"2024-04-26T23:37:13.802750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798764+0000","last_change":"2024-04-26T23:37:12.830290+0000","last_active":"2024-04-26T23:37:13.798764+0000","last_peered":"2024-04-26T23:37:13.798764+0000","last_clean":"2024-04-26T23:37:13.798764+0000","last_became_active":"2024-04-26T23:37:12.829772+0000","last_became_peered":"2024-04-26T23:37:12.829772+0000","last_unstale":"2024-04-26T23:37:13.798764+0000","last_undegraded":"2024-04-26T23:37:13.798764+0000","last_fullsized":"2024-04-26T23:37:13.798764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:01.225018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892372+0000","last_change":"2024-04-26T23:37:07.756056+0000","last_active":"2024-04-26T23:37:13.892372+0000","last_peered":"2024-04-26T23:37:13.892372+0000","last_clean":"2024-04-26T23:37:13.892372+0000","last_became_active":"2024-04-26T23:37:07.754918+0000","last_became_peered":"2024-04-26T23:37:07.754918+0000","last_unstale":"2024-04-26T23:37:13.892372+0000","last_undegraded":"2024-04-26T23:37:13.892372+0000","last_fullsized":"2024-04-26T23:37:13.892372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:08.953262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803127+0000","last_change":"2024-04-26T23:37:08.780722+0000","last_active":"2024-04-26T23:37:13.803127+0000","last_peered":"2024-04-26T23:37:13.803127+0000","last_clean":"2024-04-26T23:37:13.803127+0000","last_became_active":"2024-04-26T23:37:08.780223+0000","last_became_peered":"2024-04-26T23:37:08.780223+0000","last_unstale":"2024-04-26T23:37:13.803127+0000","last_undegraded":"2024-04-26T23:37:13.803127+0000","last_fullsized":"2024-04-26T23:37:13.803127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798239+0000","last_change":"2024-04-26T23:37:12.831066+0000","last_active":"2024-04-26T23:37:13.798239+0000","last_peered":"2024-04-26T23:37:13.798239+0000","last_clean":"2024-04-26T23:37:13.798239+0000","last_became_active":"2024-04-26T23:37:12.829941+0000","last_became_peered":"2024-04-26T23:37:12.829941+0000","last_unstale":"2024-04-26T23:37:13.798239+0000","last_undegraded":"2024-04-26T23:37:13.798239+0000","last_fullsized":"2024-04-26T23:37:13.798239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:04.322071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890099+0000","last_change":"2024-04-26T23:37:08.768304+0000","last_active":"2024-04-26T23:37:13.890099+0000","last_peered":"2024-04-26T23:37:13.890099+0000","last_clean":"2024-04-26T23:37:13.890099+0000","last_became_active":"2024-04-26T23:37:08.767971+0000","last_became_peered":"2024-04-26T23:37:08.767971+0000","last_unstale":"2024-04-26T23:37:13.890099+0000","last_undegraded":"2024-04-26T23:37:13.890099+0000","last_fullsized":"2024-04-26T23:37:13.890099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892460+0000","last_change":"2024-04-26T23:37:07.755890+0000","last_active":"2024-04-26T23:37:13.892460+0000","last_peered":"2024-04-26T23:37:13.892460+0000","last_clean":"2024-04-26T23:37:13.892460+0000","last_became_active":"2024-04-26T23:37:07.753801+0000","last_became_peered":"2024-04-26T23:37:07.753801+0000","last_unstale":"2024-04-26T23:37:13.892460+0000","last_undegraded":"2024-04-26T23:37:13.892460+0000","last_fullsized":"2024-04-26T23:37:13.892460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800561+0000","last_change":"2024-04-26T23:37:12.821170+0000","last_active":"2024-04-26T23:37:13.800561+0000","last_peered":"2024-04-26T23:37:13.800561+0000","last_clean":"2024-04-26T23:37:13.800561+0000","last_became_active":"2024-04-26T23:37:12.820739+0000","last_became_peered":"2024-04-26T23:37:12.820739+0000","last_unstale":"2024-04-26T23:37:13.800561+0000","last_undegraded":"2024-04-26T23:37:13.800561+0000","last_fullsized":"2024-04-26T23:37:13.800561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891188+0000","last_change":"2024-04-26T23:37:08.772794+0000","last_active":"2024-04-26T23:37:13.891188+0000","last_peered":"2024-04-26T23:37:13.891188+0000","last_clean":"2024-04-26T23:37:13.891188+0000","last_became_active":"2024-04-26T23:37:08.770226+0000","last_became_peered":"2024-04-26T23:37:08.770226+0000","last_unstale":"2024-04-26T23:37:13.891188+0000","last_undegraded":"2024-04-26T23:37:13.891188+0000","last_fullsized":"2024-04-26T23:37:13.891188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888791+0000","last_change":"2024-04-26T23:37:07.739350+0000","last_active":"2024-04-26T23:37:13.888791+0000","last_peered":"2024-04-26T23:37:13.888791+0000","last_clean":"2024-04-26T23:37:13.888791+0000","last_became_active":"2024-04-26T23:37:07.739007+0000","last_became_peered":"2024-04-26T23:37:07.739007+0000","last_unstale":"2024-04-26T23:37:13.888791+0000","last_undegraded":"2024-04-26T23:37:13.888791+0000","last_fullsized":"2024-04-26T23:37:13.888791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800746+0000","last_change":"2024-04-26T23:37:12.831545+0000","last_active":"2024-04-26T23:37:13.800746+0000","last_peered":"2024-04-26T23:37:13.800746+0000","last_clean":"2024-04-26T23:37:13.800746+0000","last_became_active":"2024-04-26T23:37:12.829548+0000","last_became_peered":"2024-04-26T23:37:12.829548+0000","last_unstale":"2024-04-26T23:37:13.800746+0000","last_undegraded":"2024-04-26T23:37:13.800746+0000","last_fullsized":"2024-04-26T23:37:13.800746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:31.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889936+0000","last_change":"2024-04-26T23:37:08.761785+0000","last_active":"2024-04-26T23:37:13.889936+0000","last_peered":"2024-04-26T23:37:13.889936+0000","last_clean":"2024-04-26T23:37:13.889936+0000","last_became_active":"2024-04-26T23:37:08.759700+0000","last_became_peered":"2024-04-26T23:37:08.759700+0000","last_unstale":"2024-04-26T23:37:13.889936+0000","last_undegraded":"2024-04-26T23:37:13.889936+0000","last_fullsized":"2024-04-26T23:37:13.889936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888332+0000","last_change":"2024-04-26T23:37:07.738157+0000","last_active":"2024-04-26T23:37:13.888332+0000","last_peered":"2024-04-26T23:37:13.888332+0000","last_clean":"2024-04-26T23:37:13.888332+0000","last_became_active":"2024-04-26T23:37:07.737830+0000","last_became_peered":"2024-04-26T23:37:07.737830+0000","last_unstale":"2024-04-26T23:37:13.888332+0000","last_undegraded":"2024-04-26T23:37:13.888332+0000","last_fullsized":"2024-04-26T23:37:13.888332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892316+0000","last_change":"2024-04-26T23:37:13.099265+0000","last_active":"2024-04-26T23:37:13.892316+0000","last_peered":"2024-04-26T23:37:13.892316+0000","last_clean":"2024-04-26T23:37:13.892316+0000","last_became_active":"2024-04-26T23:37:13.095798+0000","last_became_peered":"2024-04-26T23:37:13.095798+0000","last_unstale":"2024-04-26T23:37:13.892316+0000","last_undegraded":"2024-04-26T23:37:13.892316+0000","last_fullsized":"2024-04-26T23:37:13.892316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800680+0000","last_change":"2024-04-26T23:37:08.764943+0000","last_active":"2024-04-26T23:37:13.800680+0000","last_peered":"2024-04-26T23:37:13.800680+0000","last_clean":"2024-04-26T23:37:13.800680+0000","last_became_active":"2024-04-26T23:37:08.764636+0000","last_became_peered":"2024-04-26T23:37:08.764636+0000","last_unstale":"2024-04-26T23:37:13.800680+0000","last_undegraded":"2024-04-26T23:37:13.800680+0000","last_fullsized":"2024-04-26T23:37:13.800680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:29.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890217+0000","last_change":"2024-04-26T23:37:07.753035+0000","last_active":"2024-04-26T23:37:13.890217+0000","last_peered":"2024-04-26T23:37:13.890217+0000","last_clean":"2024-04-26T23:37:13.890217+0000","last_became_active":"2024-04-26T23:37:07.752363+0000","last_became_peered":"2024-04-26T23:37:07.752363+0000","last_unstale":"2024-04-26T23:37:13.890217+0000","last_undegraded":"2024-04-26T23:37:13.890217+0000","last_fullsized":"2024-04-26T23:37:13.890217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799225+0000","last_change":"2024-04-26T23:37:12.830106+0000","last_active":"2024-04-26T23:37:13.799225+0000","last_peered":"2024-04-26T23:37:13.799225+0000","last_clean":"2024-04-26T23:37:13.799225+0000","last_became_active":"2024-04-26T23:37:12.829676+0000","last_became_peered":"2024-04-26T23:37:12.829676+0000","last_unstale":"2024-04-26T23:37:13.799225+0000","last_undegraded":"2024-04-26T23:37:13.799225+0000","last_fullsized":"2024-04-26T23:37:13.799225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795807+0000","last_change":"2024-04-26T23:37:08.764908+0000","last_active":"2024-04-26T23:37:13.795807+0000","last_peered":"2024-04-26T23:37:13.795807+0000","last_clean":"2024-04-26T23:37:13.795807+0000","last_became_active":"2024-04-26T23:37:08.764369+0000","last_became_peered":"2024-04-26T23:37:08.764369+0000","last_unstale":"2024-04-26T23:37:13.795807+0000","last_undegraded":"2024-04-26T23:37:13.795807+0000","last_fullsized":"2024-04-26T23:37:13.795807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892624+0000","last_change":"2024-04-26T23:37:07.751902+0000","last_active":"2024-04-26T23:37:13.892624+0000","last_peered":"2024-04-26T23:37:13.892624+0000","last_clean":"2024-04-26T23:37:13.892624+0000","last_became_active":"2024-04-26T23:37:07.751383+0000","last_became_peered":"2024-04-26T23:37:07.751383+0000","last_unstale":"2024-04-26T23:37:13.892624+0000","last_undegraded":"2024-04-26T23:37:13.892624+0000","last_fullsized":"2024-04-26T23:37:13.892624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800477+0000","last_change":"2024-04-26T23:37:13.090483+0000","last_active":"2024-04-26T23:37:13.800477+0000","last_peered":"2024-04-26T23:37:13.800477+0000","last_clean":"2024-04-26T23:37:13.800477+0000","last_became_active":"2024-04-26T23:37:13.088991+0000","last_became_peered":"2024-04-26T23:37:13.088991+0000","last_unstale":"2024-04-26T23:37:13.800477+0000","last_undegraded":"2024-04-26T23:37:13.800477+0000","last_fullsized":"2024-04-26T23:37:13.800477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892790+0000","last_change":"2024-04-26T23:37:08.777852+0000","last_active":"2024-04-26T23:37:13.892790+0000","last_peered":"2024-04-26T23:37:13.892790+0000","last_clean":"2024-04-26T23:37:13.892790+0000","last_became_active":"2024-04-26T23:37:08.777214+0000","last_became_peered":"2024-04-26T23:37:08.777214+0000","last_unstale":"2024-04-26T23:37:13.892790+0000","last_undegraded":"2024-04-26T23:37:13.892790+0000","last_fullsized":"2024-04-26T23:37:13.892790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890844+0000","last_change":"2024-04-26T23:37:07.744896+0000","last_active":"2024-04-26T23:37:13.890844+0000","last_peered":"2024-04-26T23:37:13.890844+0000","last_clean":"2024-04-26T23:37:13.890844+0000","last_became_active":"2024-04-26T23:37:07.744597+0000","last_became_peered":"2024-04-26T23:37:07.744597+0000","last_unstale":"2024-04-26T23:37:13.890844+0000","last_undegraded":"2024-04-26T23:37:13.890844+0000","last_fullsized":"2024-04-26T23:37:13.890844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889553+0000","last_change":"2024-04-26T23:37:13.097492+0000","last_active":"2024-04-26T23:37:13.889553+0000","last_peered":"2024-04-26T23:37:13.889553+0000","last_clean":"2024-04-26T23:37:13.889553+0000","last_became_active":"2024-04-26T23:37:13.096325+0000","last_became_peered":"2024-04-26T23:37:13.096325+0000","last_unstale":"2024-04-26T23:37:13.889553+0000","last_undegraded":"2024-04-26T23:37:13.889553+0000","last_fullsized":"2024-04-26T23:37:13.889553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889171+0000","last_change":"2024-04-26T23:37:08.762047+0000","last_active":"2024-04-26T23:37:13.889171+0000","last_peered":"2024-04-26T23:37:13.889171+0000","last_clean":"2024-04-26T23:37:13.889171+0000","last_became_active":"2024-04-26T23:37:08.761660+0000","last_became_peered":"2024-04-26T23:37:08.761660+0000","last_unstale":"2024-04-26T23:37:13.889171+0000","last_undegraded":"2024-04-26T23:37:13.889171+0000","last_fullsized":"2024-04-26T23:37:13.889171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893281+0000","last_change":"2024-04-26T23:37:07.753465+0000","last_active":"2024-04-26T23:37:13.893281+0000","last_peered":"2024-04-26T23:37:13.893281+0000","last_clean":"2024-04-26T23:37:13.893281+0000","last_became_active":"2024-04-26T23:37:07.752398+0000","last_became_peered":"2024-04-26T23:37:07.752398+0000","last_unstale":"2024-04-26T23:37:13.893281+0000","last_undegraded":"2024-04-26T23:37:13.893281+0000","last_fullsized":"2024-04-26T23:37:13.893281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795198+0000","last_change":"2024-04-26T23:37:13.089856+0000","last_active":"2024-04-26T23:37:13.795198+0000","last_peered":"2024-04-26T23:37:13.795198+0000","last_clean":"2024-04-26T23:37:13.795198+0000","last_became_active":"2024-04-26T23:37:13.088870+0000","last_became_peered":"2024-04-26T23:37:13.088870+0000","last_unstale":"2024-04-26T23:37:13.795198+0000","last_undegraded":"2024-04-26T23:37:13.795198+0000","last_fullsized":"2024-04-26T23:37:13.795198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803393+0000","last_change":"2024-04-26T23:37:08.785871+0000","last_active":"2024-04-26T23:37:13.803393+0000","last_peered":"2024-04-26T23:37:13.803393+0000","last_clean":"2024-04-26T23:37:13.803393+0000","last_became_active":"2024-04-26T23:37:08.785451+0000","last_became_peered":"2024-04-26T23:37:08.785451+0000","last_unstale":"2024-04-26T23:37:13.803393+0000","last_undegraded":"2024-04-26T23:37:13.803393+0000","last_fullsized":"2024-04-26T23:37:13.803393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889253+0000","last_change":"2024-04-26T23:37:07.730561+0000","last_active":"2024-04-26T23:37:13.889253+0000","last_peered":"2024-04-26T23:37:13.889253+0000","last_clean":"2024-04-26T23:37:13.889253+0000","last_became_active":"2024-04-26T23:37:07.730274+0000","last_became_peered":"2024-04-26T23:37:07.730274+0000","last_unstale":"2024-04-26T23:37:13.889253+0000","last_undegraded":"2024-04-26T23:37:13.889253+0000","last_fullsized":"2024-04-26T23:37:13.889253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888467+0000","last_change":"2024-04-26T23:37:12.819624+0000","last_active":"2024-04-26T23:37:13.888467+0000","last_peered":"2024-04-26T23:37:13.888467+0000","last_clean":"2024-04-26T23:37:13.888467+0000","last_became_active":"2024-04-26T23:37:12.819079+0000","last_became_peered":"2024-04-26T23:37:12.819079+0000","last_unstale":"2024-04-26T23:37:13.888467+0000","last_undegraded":"2024-04-26T23:37:13.888467+0000","last_fullsized":"2024-04-26T23:37:13.888467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799143+0000","last_change":"2024-04-26T23:37:08.760531+0000","last_active":"2024-04-26T23:37:13.799143+0000","last_peered":"2024-04-26T23:37:13.799143+0000","last_clean":"2024-04-26T23:37:13.799143+0000","last_became_active":"2024-04-26T23:37:08.760250+0000","last_became_peered":"2024-04-26T23:37:08.760250+0000","last_unstale":"2024-04-26T23:37:13.799143+0000","last_undegraded":"2024-04-26T23:37:13.799143+0000","last_fullsized":"2024-04-26T23:37:13.799143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:17.138976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802089+0000","last_change":"2024-04-26T23:37:07.743324+0000","last_active":"2024-04-26T23:37:13.802089+0000","last_peered":"2024-04-26T23:37:13.802089+0000","last_clean":"2024-04-26T23:37:13.802089+0000","last_became_active":"2024-04-26T23:37:07.742056+0000","last_became_peered":"2024-04-26T23:37:07.742056+0000","last_unstale":"2024-04-26T23:37:13.802089+0000","last_undegraded":"2024-04-26T23:37:13.802089+0000","last_fullsized":"2024-04-26T23:37:13.802089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889116+0000","last_change":"2024-04-26T23:37:12.835987+0000","last_active":"2024-04-26T23:37:13.889116+0000","last_peered":"2024-04-26T23:37:13.889116+0000","last_clean":"2024-04-26T23:37:13.889116+0000","last_became_active":"2024-04-26T23:37:12.831199+0000","last_became_peered":"2024-04-26T23:37:12.831199+0000","last_unstale":"2024-04-26T23:37:13.889116+0000","last_undegraded":"2024-04-26T23:37:13.889116+0000","last_fullsized":"2024-04-26T23:37:13.889116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888008+0000","last_change":"2024-04-26T23:37:08.768283+0000","last_active":"2024-04-26T23:37:13.888008+0000","last_peered":"2024-04-26T23:37:13.888008+0000","last_clean":"2024-04-26T23:37:13.888008+0000","last_became_active":"2024-04-26T23:37:08.766512+0000","last_became_peered":"2024-04-26T23:37:08.766512+0000","last_unstale":"2024-04-26T23:37:13.888008+0000","last_undegraded":"2024-04-26T23:37:13.888008+0000","last_fullsized":"2024-04-26T23:37:13.888008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888285+0000","last_change":"2024-04-26T23:37:07.728903+0000","last_active":"2024-04-26T23:37:13.888285+0000","last_peered":"2024-04-26T23:37:13.888285+0000","last_clean":"2024-04-26T23:37:13.888285+0000","last_became_active":"2024-04-26T23:37:07.728621+0000","last_became_peered":"2024-04-26T23:37:07.728621+0000","last_unstale":"2024-04-26T23:37:13.888285+0000","last_undegraded":"2024-04-26T23:37:13.888285+0000","last_fullsized":"2024-04-26T23:37:13.888285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892519+0000","last_change":"2024-04-26T23:37:12.833651+0000","last_active":"2024-04-26T23:37:13.892519+0000","last_peered":"2024-04-26T23:37:13.892519+0000","last_clean":"2024-04-26T23:37:13.892519+0000","last_became_active":"2024-04-26T23:37:12.831533+0000","last_became_peered":"2024-04-26T23:37:12.831533+0000","last_unstale":"2024-04-26T23:37:13.892519+0000","last_undegraded":"2024-04-26T23:37:13.892519+0000","last_fullsized":"2024-04-26T23:37:13.892519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794491+0000","last_change":"2024-04-26T23:37:08.749652+0000","last_active":"2024-04-26T23:37:13.794491+0000","last_peered":"2024-04-26T23:37:13.794491+0000","last_clean":"2024-04-26T23:37:13.794491+0000","last_became_active":"2024-04-26T23:37:08.748932+0000","last_became_peered":"2024-04-26T23:37:08.748932+0000","last_unstale":"2024-04-26T23:37:13.794491+0000","last_undegraded":"2024-04-26T23:37:13.794491+0000","last_fullsized":"2024-04-26T23:37:13.794491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:04.431242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890355+0000","last_change":"2024-04-26T23:37:07.753005+0000","last_active":"2024-04-26T23:37:13.890355+0000","last_peered":"2024-04-26T23:37:13.890355+0000","last_clean":"2024-04-26T23:37:13.890355+0000","last_became_active":"2024-04-26T23:37:07.752377+0000","last_became_peered":"2024-04-26T23:37:07.752377+0000","last_unstale":"2024-04-26T23:37:13.890355+0000","last_undegraded":"2024-04-26T23:37:13.890355+0000","last_fullsized":"2024-04-26T23:37:13.890355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795722+0000","last_change":"2024-04-26T23:37:13.097286+0000","last_active":"2024-04-26T23:37:13.795722+0000","last_peered":"2024-04-26T23:37:13.795722+0000","last_clean":"2024-04-26T23:37:13.795722+0000","last_became_active":"2024-04-26T23:37:13.096512+0000","last_became_peered":"2024-04-26T23:37:13.096512+0000","last_unstale":"2024-04-26T23:37:13.795722+0000","last_undegraded":"2024-04-26T23:37:13.795722+0000","last_fullsized":"2024-04-26T23:37:13.795722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890492+0000","last_change":"2024-04-26T23:37:08.791257+0000","last_active":"2024-04-26T23:37:13.890492+0000","last_peered":"2024-04-26T23:37:13.890492+0000","last_clean":"2024-04-26T23:37:13.890492+0000","last_became_active":"2024-04-26T23:37:08.790966+0000","last_became_peered":"2024-04-26T23:37:08.790966+0000","last_unstale":"2024-04-26T23:37:13.890492+0000","last_undegraded":"2024-04-26T23:37:13.890492+0000","last_fullsized":"2024-04-26T23:37:13.890492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802430+0000","last_change":"2024-04-26T23:37:07.742144+0000","last_active":"2024-04-26T23:37:13.802430+0000","last_peered":"2024-04-26T23:37:13.802430+0000","last_clean":"2024-04-26T23:37:13.802430+0000","last_became_active":"2024-04-26T23:37:07.741812+0000","last_became_peered":"2024-04-26T23:37:07.741812+0000","last_unstale":"2024-04-26T23:37:13.802430+0000","last_undegraded":"2024-04-26T23:37:13.802430+0000","last_fullsized":"2024-04-26T23:37:13.802430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890498+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:37:13.890498+0000","last_peered":"2024-04-26T23:37:13.890498+0000","last_clean":"2024-04-26T23:37:13.890498+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:37:13.890498+0000","last_undegraded":"2024-04-26T23:37:13.890498+0000","last_fullsized":"2024-04-26T23:37:13.890498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892073+0000","last_change":"2024-04-26T23:37:13.094077+0000","last_active":"2024-04-26T23:37:13.892073+0000","last_peered":"2024-04-26T23:37:13.892073+0000","last_clean":"2024-04-26T23:37:13.892073+0000","last_became_active":"2024-04-26T23:37:13.093035+0000","last_became_peered":"2024-04-26T23:37:13.093035+0000","last_unstale":"2024-04-26T23:37:13.892073+0000","last_undegraded":"2024-04-26T23:37:13.892073+0000","last_fullsized":"2024-04-26T23:37:13.892073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892721+0000","last_change":"2024-04-26T23:37:08.787518+0000","last_active":"2024-04-26T23:37:13.892721+0000","last_peered":"2024-04-26T23:37:13.892721+0000","last_clean":"2024-04-26T23:37:13.892721+0000","last_became_active":"2024-04-26T23:37:08.787303+0000","last_became_peered":"2024-04-26T23:37:08.787303+0000","last_unstale":"2024-04-26T23:37:13.892721+0000","last_undegraded":"2024-04-26T23:37:13.892721+0000","last_fullsized":"2024-04-26T23:37:13.892721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891144+0000","last_change":"2024-04-26T23:37:07.746465+0000","last_active":"2024-04-26T23:37:13.891144+0000","last_peered":"2024-04-26T23:37:13.891144+0000","last_clean":"2024-04-26T23:37:13.891144+0000","last_became_active":"2024-04-26T23:37:07.746109+0000","last_became_peered":"2024-04-26T23:37:07.746109+0000","last_unstale":"2024-04-26T23:37:13.891144+0000","last_undegraded":"2024-04-26T23:37:13.891144+0000","last_fullsized":"2024-04-26T23:37:13.891144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890061+0000","last_change":"2024-04-26T23:37:13.098348+0000","last_active":"2024-04-26T23:37:13.890061+0000","last_peered":"2024-04-26T23:37:13.890061+0000","last_clean":"2024-04-26T23:37:13.890061+0000","last_became_active":"2024-04-26T23:37:13.092005+0000","last_became_peered":"2024-04-26T23:37:13.092005+0000","last_unstale":"2024-04-26T23:37:13.890061+0000","last_undegraded":"2024-04-26T23:37:13.890061+0000","last_fullsized":"2024-04-26T23:37:13.890061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802973+0000","last_change":"2024-04-26T23:37:08.788947+0000","last_active":"2024-04-26T23:37:13.802973+0000","last_peered":"2024-04-26T23:37:13.802973+0000","last_clean":"2024-04-26T23:37:13.802973+0000","last_became_active":"2024-04-26T23:37:08.785540+0000","last_became_peered":"2024-04-26T23:37:08.785540+0000","last_unstale":"2024-04-26T23:37:13.802973+0000","last_undegraded":"2024-04-26T23:37:13.802973+0000","last_fullsized":"2024-04-26T23:37:13.802973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889865+0000","last_change":"2024-04-26T23:37:07.745996+0000","last_active":"2024-04-26T23:37:13.889865+0000","last_peered":"2024-04-26T23:37:13.889865+0000","last_clean":"2024-04-26T23:37:13.889865+0000","last_became_active":"2024-04-26T23:37:07.745526+0000","last_became_peered":"2024-04-26T23:37:07.745526+0000","last_unstale":"2024-04-26T23:37:13.889865+0000","last_undegraded":"2024-04-26T23:37:13.889865+0000","last_fullsized":"2024-04-26T23:37:13.889865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795637+0000","last_change":"2024-04-26T23:37:12.818381+0000","last_active":"2024-04-26T23:37:13.795637+0000","last_peered":"2024-04-26T23:37:13.795637+0000","last_clean":"2024-04-26T23:37:13.795637+0000","last_became_active":"2024-04-26T23:37:12.816556+0000","last_became_peered":"2024-04-26T23:37:12.816556+0000","last_unstale":"2024-04-26T23:37:13.795637+0000","last_undegraded":"2024-04-26T23:37:13.795637+0000","last_fullsized":"2024-04-26T23:37:13.795637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893859+0000","last_change":"2024-04-26T23:37:08.788613+0000","last_active":"2024-04-26T23:37:13.893859+0000","last_peered":"2024-04-26T23:37:13.893859+0000","last_clean":"2024-04-26T23:37:13.893859+0000","last_became_active":"2024-04-26T23:37:08.788341+0000","last_became_peered":"2024-04-26T23:37:08.788341+0000","last_unstale":"2024-04-26T23:37:13.893859+0000","last_undegraded":"2024-04-26T23:37:13.893859+0000","last_fullsized":"2024-04-26T23:37:13.893859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:05.835929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891528+0000","last_change":"2024-04-26T23:37:07.732372+0000","last_active":"2024-04-26T23:37:13.891528+0000","last_peered":"2024-04-26T23:37:13.891528+0000","last_clean":"2024-04-26T23:37:13.891528+0000","last_became_active":"2024-04-26T23:37:07.732085+0000","last_became_peered":"2024-04-26T23:37:07.732085+0000","last_unstale":"2024-04-26T23:37:13.891528+0000","last_undegraded":"2024-04-26T23:37:13.891528+0000","last_fullsized":"2024-04-26T23:37:13.891528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798168+0000","last_change":"2024-04-26T23:37:13.093306+0000","last_active":"2024-04-26T23:37:13.798168+0000","last_peered":"2024-04-26T23:37:13.798168+0000","last_clean":"2024-04-26T23:37:13.798168+0000","last_became_active":"2024-04-26T23:37:13.089425+0000","last_became_peered":"2024-04-26T23:37:13.089425+0000","last_unstale":"2024-04-26T23:37:13.798168+0000","last_undegraded":"2024-04-26T23:37:13.798168+0000","last_fullsized":"2024-04-26T23:37:13.798168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893002+0000","last_change":"2024-04-26T23:37:08.788931+0000","last_active":"2024-04-26T23:37:13.893002+0000","last_peered":"2024-04-26T23:37:13.893002+0000","last_clean":"2024-04-26T23:37:13.893002+0000","last_became_active":"2024-04-26T23:37:08.788287+0000","last_became_peered":"2024-04-26T23:37:08.788287+0000","last_unstale":"2024-04-26T23:37:13.893002+0000","last_undegraded":"2024-04-26T23:37:13.893002+0000","last_fullsized":"2024-04-26T23:37:13.893002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799208+0000","last_change":"2024-04-26T23:37:07.729179+0000","last_active":"2024-04-26T23:37:13.799208+0000","last_peered":"2024-04-26T23:37:13.799208+0000","last_clean":"2024-04-26T23:37:13.799208+0000","last_became_active":"2024-04-26T23:37:07.728840+0000","last_became_peered":"2024-04-26T23:37:07.728840+0000","last_unstale":"2024-04-26T23:37:13.799208+0000","last_undegraded":"2024-04-26T23:37:13.799208+0000","last_fullsized":"2024-04-26T23:37:13.799208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:09.969446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798616+0000","last_change":"2024-04-26T23:37:12.833356+0000","last_active":"2024-04-26T23:37:13.798616+0000","last_peered":"2024-04-26T23:37:13.798616+0000","last_clean":"2024-04-26T23:37:13.798616+0000","last_became_active":"2024-04-26T23:37:12.830528+0000","last_became_peered":"2024-04-26T23:37:12.830528+0000","last_unstale":"2024-04-26T23:37:13.798616+0000","last_undegraded":"2024-04-26T23:37:13.798616+0000","last_fullsized":"2024-04-26T23:37:13.798616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.662558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891353+0000","last_change":"2024-04-26T23:37:08.782130+0000","last_active":"2024-04-26T23:37:13.891353+0000","last_peered":"2024-04-26T23:37:13.891353+0000","last_clean":"2024-04-26T23:37:13.891353+0000","last_became_active":"2024-04-26T23:37:08.781906+0000","last_became_peered":"2024-04-26T23:37:08.781906+0000","last_unstale":"2024-04-26T23:37:13.891353+0000","last_undegraded":"2024-04-26T23:37:13.891353+0000","last_fullsized":"2024-04-26T23:37:13.891353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.887760+0000","last_change":"2024-04-26T23:37:07.728318+0000","last_active":"2024-04-26T23:37:13.887760+0000","last_peered":"2024-04-26T23:37:13.887760+0000","last_clean":"2024-04-26T23:37:13.887760+0000","last_became_active":"2024-04-26T23:37:07.727941+0000","last_became_peered":"2024-04-26T23:37:07.727941+0000","last_unstale":"2024-04-26T23:37:13.887760+0000","last_undegraded":"2024-04-26T23:37:13.887760+0000","last_fullsized":"2024-04-26T23:37:13.887760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795484+0000","last_change":"2024-04-26T23:37:12.832185+0000","last_active":"2024-04-26T23:37:13.795484+0000","last_peered":"2024-04-26T23:37:13.795484+0000","last_clean":"2024-04-26T23:37:13.795484+0000","last_became_active":"2024-04-26T23:37:12.830851+0000","last_became_peered":"2024-04-26T23:37:12.830851+0000","last_unstale":"2024-04-26T23:37:13.795484+0000","last_undegraded":"2024-04-26T23:37:13.795484+0000","last_fullsized":"2024-04-26T23:37:13.795484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889876+0000","last_change":"2024-04-26T23:37:08.768079+0000","last_active":"2024-04-26T23:37:13.889876+0000","last_peered":"2024-04-26T23:37:13.889876+0000","last_clean":"2024-04-26T23:37:13.889876+0000","last_became_active":"2024-04-26T23:37:08.767338+0000","last_became_peered":"2024-04-26T23:37:08.767338+0000","last_unstale":"2024-04-26T23:37:13.889876+0000","last_undegraded":"2024-04-26T23:37:13.889876+0000","last_fullsized":"2024-04-26T23:37:13.889876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890125+0000","last_change":"2024-04-26T23:37:07.746339+0000","last_active":"2024-04-26T23:37:13.890125+0000","last_peered":"2024-04-26T23:37:13.890125+0000","last_clean":"2024-04-26T23:37:13.890125+0000","last_became_active":"2024-04-26T23:37:07.746075+0000","last_became_peered":"2024-04-26T23:37:07.746075+0000","last_unstale":"2024-04-26T23:37:13.890125+0000","last_undegraded":"2024-04-26T23:37:13.890125+0000","last_fullsized":"2024-04-26T23:37:13.890125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892748+0000","last_change":"2024-04-26T23:37:13.091284+0000","last_active":"2024-04-26T23:37:13.892748+0000","last_peered":"2024-04-26T23:37:13.892748+0000","last_clean":"2024-04-26T23:37:13.892748+0000","last_became_active":"2024-04-26T23:37:13.090274+0000","last_became_peered":"2024-04-26T23:37:13.090274+0000","last_unstale":"2024-04-26T23:37:13.892748+0000","last_undegraded":"2024-04-26T23:37:13.892748+0000","last_fullsized":"2024-04-26T23:37:13.892748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:53.078576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892237+0000","last_change":"2024-04-26T23:37:08.782024+0000","last_active":"2024-04-26T23:37:13.892237+0000","last_peered":"2024-04-26T23:37:13.892237+0000","last_clean":"2024-04-26T23:37:13.892237+0000","last_became_active":"2024-04-26T23:37:08.781730+0000","last_became_peered":"2024-04-26T23:37:08.781730+0000","last_unstale":"2024-04-26T23:37:13.892237+0000","last_undegraded":"2024-04-26T23:37:13.892237+0000","last_fullsized":"2024-04-26T23:37:13.892237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801858+0000","last_change":"2024-04-26T23:37:07.741413+0000","last_active":"2024-04-26T23:37:13.801858+0000","last_peered":"2024-04-26T23:37:13.801858+0000","last_clean":"2024-04-26T23:37:13.801858+0000","last_became_active":"2024-04-26T23:37:07.741088+0000","last_became_peered":"2024-04-26T23:37:07.741088+0000","last_unstale":"2024-04-26T23:37:13.801858+0000","last_undegraded":"2024-04-26T23:37:13.801858+0000","last_fullsized":"2024-04-26T23:37:13.801858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891737+0000","last_change":"2024-04-26T23:37:13.089930+0000","last_active":"2024-04-26T23:37:13.891737+0000","last_peered":"2024-04-26T23:37:13.891737+0000","last_clean":"2024-04-26T23:37:13.891737+0000","last_became_active":"2024-04-26T23:37:13.088797+0000","last_became_peered":"2024-04-26T23:37:13.088797+0000","last_unstale":"2024-04-26T23:37:13.891737+0000","last_undegraded":"2024-04-26T23:37:13.891737+0000","last_fullsized":"2024-04-26T23:37:13.891737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:24.129454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795384+0000","last_change":"2024-04-26T23:37:08.772085+0000","last_active":"2024-04-26T23:37:13.795384+0000","last_peered":"2024-04-26T23:37:13.795384+0000","last_clean":"2024-04-26T23:37:13.795384+0000","last_became_active":"2024-04-26T23:37:08.771732+0000","last_became_peered":"2024-04-26T23:37:08.771732+0000","last_unstale":"2024-04-26T23:37:13.795384+0000","last_undegraded":"2024-04-26T23:37:13.795384+0000","last_fullsized":"2024-04-26T23:37:13.795384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:01.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796199+0000","last_change":"2024-04-26T23:37:07.739747+0000","last_active":"2024-04-26T23:37:13.796199+0000","last_peered":"2024-04-26T23:37:13.796199+0000","last_clean":"2024-04-26T23:37:13.796199+0000","last_became_active":"2024-04-26T23:37:07.739361+0000","last_became_peered":"2024-04-26T23:37:07.739361+0000","last_unstale":"2024-04-26T23:37:13.796199+0000","last_undegraded":"2024-04-26T23:37:13.796199+0000","last_fullsized":"2024-04-26T23:37:13.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:55.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802956+0000","last_change":"2024-04-26T23:37:13.100935+0000","last_active":"2024-04-26T23:37:13.802956+0000","last_peered":"2024-04-26T23:37:13.802956+0000","last_clean":"2024-04-26T23:37:13.802956+0000","last_became_active":"2024-04-26T23:37:13.098932+0000","last_became_peered":"2024-04-26T23:37:13.098932+0000","last_unstale":"2024-04-26T23:37:13.802956+0000","last_undegraded":"2024-04-26T23:37:13.802956+0000","last_fullsized":"2024-04-26T23:37:13.802956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892457+0000","last_change":"2024-04-26T23:37:08.780897+0000","last_active":"2024-04-26T23:37:13.892457+0000","last_peered":"2024-04-26T23:37:13.892457+0000","last_clean":"2024-04-26T23:37:13.892457+0000","last_became_active":"2024-04-26T23:37:08.780573+0000","last_became_peered":"2024-04-26T23:37:08.780573+0000","last_unstale":"2024-04-26T23:37:13.892457+0000","last_undegraded":"2024-04-26T23:37:13.892457+0000","last_fullsized":"2024-04-26T23:37:13.892457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:27.933683+0000","last_change":"2024-04-26T23:37:07.743107+0000","last_active":"2024-04-26T23:37:27.933683+0000","last_peered":"2024-04-26T23:37:27.933683+0000","last_clean":"2024-04-26T23:37:27.933683+0000","last_became_active":"2024-04-26T23:37:07.742729+0000","last_became_peered":"2024-04-26T23:37:07.742729+0000","last_unstale":"2024-04-26T23:37:27.933683+0000","last_undegraded":"2024-04-26T23:37:27.933683+0000","last_fullsized":"2024-04-26T23:37:27.933683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888705+0000","last_change":"2024-04-26T23:37:12.824914+0000","last_active":"2024-04-26T23:37:13.888705+0000","last_peered":"2024-04-26T23:37:13.888705+0000","last_clean":"2024-04-26T23:37:13.888705+0000","last_became_active":"2024-04-26T23:37:12.824383+0000","last_became_peered":"2024-04-26T23:37:12.824383+0000","last_unstale":"2024-04-26T23:37:13.888705+0000","last_undegraded":"2024-04-26T23:37:13.888705+0000","last_fullsized":"2024-04-26T23:37:13.888705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801058+0000","last_change":"2024-04-26T23:37:08.785202+0000","last_active":"2024-04-26T23:37:13.801058+0000","last_peered":"2024-04-26T23:37:13.801058+0000","last_clean":"2024-04-26T23:37:13.801058+0000","last_became_active":"2024-04-26T23:37:08.779537+0000","last_became_peered":"2024-04-26T23:37:08.779537+0000","last_unstale":"2024-04-26T23:37:13.801058+0000","last_undegraded":"2024-04-26T23:37:13.801058+0000","last_fullsized":"2024-04-26T23:37:13.801058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.971268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889719+0000","last_change":"2024-04-26T23:37:07.741707+0000","last_active":"2024-04-26T23:37:13.889719+0000","last_peered":"2024-04-26T23:37:13.889719+0000","last_clean":"2024-04-26T23:37:13.889719+0000","last_became_active":"2024-04-26T23:37:07.740731+0000","last_became_peered":"2024-04-26T23:37:07.740731+0000","last_unstale":"2024-04-26T23:37:13.889719+0000","last_undegraded":"2024-04-26T23:37:13.889719+0000","last_fullsized":"2024-04-26T23:37:13.889719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798812+0000","last_change":"2024-04-26T23:37:12.818753+0000","last_active":"2024-04-26T23:37:13.798812+0000","last_peered":"2024-04-26T23:37:13.798812+0000","last_clean":"2024-04-26T23:37:13.798812+0000","last_became_active":"2024-04-26T23:37:12.817885+0000","last_became_peered":"2024-04-26T23:37:12.817885+0000","last_unstale":"2024-04-26T23:37:13.798812+0000","last_undegraded":"2024-04-26T23:37:13.798812+0000","last_fullsized":"2024-04-26T23:37:13.798812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:06.283131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795457+0000","last_change":"2024-04-26T23:37:08.796476+0000","last_active":"2024-04-26T23:37:13.795457+0000","last_peered":"2024-04-26T23:37:13.795457+0000","last_clean":"2024-04-26T23:37:13.795457+0000","last_became_active":"2024-04-26T23:37:08.795730+0000","last_became_peered":"2024-04-26T23:37:08.795730+0000","last_unstale":"2024-04-26T23:37:13.795457+0000","last_undegraded":"2024-04-26T23:37:13.795457+0000","last_fullsized":"2024-04-26T23:37:13.795457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802755+0000","last_change":"2024-04-26T23:37:07.746232+0000","last_active":"2024-04-26T23:37:13.802755+0000","last_peered":"2024-04-26T23:37:13.802755+0000","last_clean":"2024-04-26T23:37:13.802755+0000","last_became_active":"2024-04-26T23:37:07.745874+0000","last_became_peered":"2024-04-26T23:37:07.745874+0000","last_unstale":"2024-04-26T23:37:13.802755+0000","last_undegraded":"2024-04-26T23:37:13.802755+0000","last_fullsized":"2024-04-26T23:37:13.802755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891356+0000","last_change":"2024-04-26T23:37:12.834267+0000","last_active":"2024-04-26T23:37:13.891356+0000","last_peered":"2024-04-26T23:37:13.891356+0000","last_clean":"2024-04-26T23:37:13.891356+0000","last_became_active":"2024-04-26T23:37:12.829458+0000","last_became_peered":"2024-04-26T23:37:12.829458+0000","last_unstale":"2024-04-26T23:37:13.891356+0000","last_undegraded":"2024-04-26T23:37:13.891356+0000","last_fullsized":"2024-04-26T23:37:13.891356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:05.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802727+0000","last_change":"2024-04-26T23:37:08.788911+0000","last_active":"2024-04-26T23:37:13.802727+0000","last_peered":"2024-04-26T23:37:13.802727+0000","last_clean":"2024-04-26T23:37:13.802727+0000","last_became_active":"2024-04-26T23:37:08.785658+0000","last_became_peered":"2024-04-26T23:37:08.785658+0000","last_unstale":"2024-04-26T23:37:13.802727+0000","last_undegraded":"2024-04-26T23:37:13.802727+0000","last_fullsized":"2024-04-26T23:37:13.802727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892184+0000","last_change":"2024-04-26T23:37:07.750014+0000","last_active":"2024-04-26T23:37:13.892184+0000","last_peered":"2024-04-26T23:37:13.892184+0000","last_clean":"2024-04-26T23:37:13.892184+0000","last_became_active":"2024-04-26T23:37:07.748991+0000","last_became_peered":"2024-04-26T23:37:07.748991+0000","last_unstale":"2024-04-26T23:37:13.892184+0000","last_undegraded":"2024-04-26T23:37:13.892184+0000","last_fullsized":"2024-04-26T23:37:13.892184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890500+0000","last_change":"2024-04-26T23:37:13.096386+0000","last_active":"2024-04-26T23:37:13.890500+0000","last_peered":"2024-04-26T23:37:13.890500+0000","last_clean":"2024-04-26T23:37:13.890500+0000","last_became_active":"2024-04-26T23:37:13.095901+0000","last_became_peered":"2024-04-26T23:37:13.095901+0000","last_unstale":"2024-04-26T23:37:13.890500+0000","last_undegraded":"2024-04-26T23:37:13.890500+0000","last_fullsized":"2024-04-26T23:37:13.890500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:22.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795244+0000","last_change":"2024-04-26T23:37:13.100870+0000","last_active":"2024-04-26T23:37:13.795244+0000","last_peered":"2024-04-26T23:37:13.795244+0000","last_clean":"2024-04-26T23:37:13.795244+0000","last_became_active":"2024-04-26T23:37:13.094872+0000","last_became_peered":"2024-04-26T23:37:13.094872+0000","last_unstale":"2024-04-26T23:37:13.795244+0000","last_undegraded":"2024-04-26T23:37:13.795244+0000","last_fullsized":"2024-04-26T23:37:13.795244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888033+0000","last_change":"2024-04-26T23:37:08.770189+0000","last_active":"2024-04-26T23:37:13.888033+0000","last_peered":"2024-04-26T23:37:13.888033+0000","last_clean":"2024-04-26T23:37:13.888033+0000","last_became_active":"2024-04-26T23:37:08.769911+0000","last_became_peered":"2024-04-26T23:37:08.769911+0000","last_unstale":"2024-04-26T23:37:13.888033+0000","last_undegraded":"2024-04-26T23:37:13.888033+0000","last_fullsized":"2024-04-26T23:37:13.888033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800060+0000","last_change":"2024-04-26T23:37:07.745063+0000","last_active":"2024-04-26T23:37:13.800060+0000","last_peered":"2024-04-26T23:37:13.800060+0000","last_clean":"2024-04-26T23:37:13.800060+0000","last_became_active":"2024-04-26T23:37:07.744383+0000","last_became_peered":"2024-04-26T23:37:07.744383+0000","last_unstale":"2024-04-26T23:37:13.800060+0000","last_undegraded":"2024-04-26T23:37:13.800060+0000","last_fullsized":"2024-04-26T23:37:13.800060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890337+0000","last_change":"2024-04-26T23:37:12.824766+0000","last_active":"2024-04-26T23:37:13.890337+0000","last_peered":"2024-04-26T23:37:13.890337+0000","last_clean":"2024-04-26T23:37:13.890337+0000","last_became_active":"2024-04-26T23:37:12.824314+0000","last_became_peered":"2024-04-26T23:37:12.824314+0000","last_unstale":"2024-04-26T23:37:13.890337+0000","last_undegraded":"2024-04-26T23:37:13.890337+0000","last_fullsized":"2024-04-26T23:37:13.890337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889801+0000","last_change":"2024-04-26T23:37:08.769161+0000","last_active":"2024-04-26T23:37:13.889801+0000","last_peered":"2024-04-26T23:37:13.889801+0000","last_clean":"2024-04-26T23:37:13.889801+0000","last_became_active":"2024-04-26T23:37:08.767929+0000","last_became_peered":"2024-04-26T23:37:08.767929+0000","last_unstale":"2024-04-26T23:37:13.889801+0000","last_undegraded":"2024-04-26T23:37:13.889801+0000","last_fullsized":"2024-04-26T23:37:13.889801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890886+0000","last_change":"2024-04-26T23:37:07.739492+0000","last_active":"2024-04-26T23:37:13.890886+0000","last_peered":"2024-04-26T23:37:13.890886+0000","last_clean":"2024-04-26T23:37:13.890886+0000","last_became_active":"2024-04-26T23:37:07.739185+0000","last_became_peered":"2024-04-26T23:37:07.739185+0000","last_unstale":"2024-04-26T23:37:13.890886+0000","last_undegraded":"2024-04-26T23:37:13.890886+0000","last_fullsized":"2024-04-26T23:37:13.890886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890373+0000","last_change":"2024-04-26T23:37:07.750874+0000","last_active":"2024-04-26T23:37:13.890373+0000","last_peered":"2024-04-26T23:37:13.890373+0000","last_clean":"2024-04-26T23:37:13.890373+0000","last_became_active":"2024-04-26T23:37:07.748284+0000","last_became_peered":"2024-04-26T23:37:07.748284+0000","last_unstale":"2024-04-26T23:37:13.890373+0000","last_undegraded":"2024-04-26T23:37:13.890373+0000","last_fullsized":"2024-04-26T23:37:13.890373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796082+0000","last_change":"2024-04-26T23:37:08.753110+0000","last_active":"2024-04-26T23:37:13.796082+0000","last_peered":"2024-04-26T23:37:13.796082+0000","last_clean":"2024-04-26T23:37:13.796082+0000","last_became_active":"2024-04-26T23:37:08.752712+0000","last_became_peered":"2024-04-26T23:37:08.752712+0000","last_unstale":"2024-04-26T23:37:13.796082+0000","last_undegraded":"2024-04-26T23:37:13.796082+0000","last_fullsized":"2024-04-26T23:37:13.796082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889412+0000","last_change":"2024-04-26T23:37:12.829960+0000","last_active":"2024-04-26T23:37:13.889412+0000","last_peered":"2024-04-26T23:37:13.889412+0000","last_clean":"2024-04-26T23:37:13.889412+0000","last_became_active":"2024-04-26T23:37:12.829520+0000","last_became_peered":"2024-04-26T23:37:12.829520+0000","last_unstale":"2024-04-26T23:37:13.889412+0000","last_undegraded":"2024-04-26T23:37:13.889412+0000","last_fullsized":"2024-04-26T23:37:13.889412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:05.418963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891289+0000","last_change":"2024-04-26T23:37:07.747637+0000","last_active":"2024-04-26T23:37:13.891289+0000","last_peered":"2024-04-26T23:37:13.891289+0000","last_clean":"2024-04-26T23:37:13.891289+0000","last_became_active":"2024-04-26T23:37:07.747308+0000","last_became_peered":"2024-04-26T23:37:07.747308+0000","last_unstale":"2024-04-26T23:37:13.891289+0000","last_undegraded":"2024-04-26T23:37:13.891289+0000","last_fullsized":"2024-04-26T23:37:13.891289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798586+0000","last_change":"2024-04-26T23:37:08.763567+0000","last_active":"2024-04-26T23:37:13.798586+0000","last_peered":"2024-04-26T23:37:13.798586+0000","last_clean":"2024-04-26T23:37:13.798586+0000","last_became_active":"2024-04-26T23:37:08.762771+0000","last_became_peered":"2024-04-26T23:37:08.762771+0000","last_unstale":"2024-04-26T23:37:13.798586+0000","last_undegraded":"2024-04-26T23:37:13.798586+0000","last_fullsized":"2024-04-26T23:37:13.798586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:19.641900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892195+0000","last_change":"2024-04-26T23:37:12.832076+0000","last_active":"2024-04-26T23:37:13.892195+0000","last_peered":"2024-04-26T23:37:13.892195+0000","last_clean":"2024-04-26T23:37:13.892195+0000","last_became_active":"2024-04-26T23:37:12.830641+0000","last_became_peered":"2024-04-26T23:37:12.830641+0000","last_unstale":"2024-04-26T23:37:13.892195+0000","last_undegraded":"2024-04-26T23:37:13.892195+0000","last_fullsized":"2024-04-26T23:37:13.892195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:21.722510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889621+0000","last_change":"2024-04-26T23:37:08.768406+0000","last_active":"2024-04-26T23:37:13.889621+0000","last_peered":"2024-04-26T23:37:13.889621+0000","last_clean":"2024-04-26T23:37:13.889621+0000","last_became_active":"2024-04-26T23:37:08.767730+0000","last_became_peered":"2024-04-26T23:37:08.767730+0000","last_unstale":"2024-04-26T23:37:13.889621+0000","last_undegraded":"2024-04-26T23:37:13.889621+0000","last_fullsized":"2024-04-26T23:37:13.889621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888136+0000","last_change":"2024-04-26T23:37:08.780725+0000","last_active":"2024-04-26T23:37:13.888136+0000","last_peered":"2024-04-26T23:37:13.888136+0000","last_clean":"2024-04-26T23:37:13.888136+0000","last_became_active":"2024-04-26T23:37:08.780362+0000","last_became_peered":"2024-04-26T23:37:08.780362+0000","last_unstale":"2024-04-26T23:37:13.888136+0000","last_undegraded":"2024-04-26T23:37:13.888136+0000","last_fullsized":"2024-04-26T23:37:13.888136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801811+0000","last_change":"2024-04-26T23:37:08.789772+0000","last_active":"2024-04-26T23:37:13.801811+0000","last_peered":"2024-04-26T23:37:13.801811+0000","last_clean":"2024-04-26T23:37:13.801811+0000","last_became_active":"2024-04-26T23:37:08.785818+0000","last_became_peered":"2024-04-26T23:37:08.785818+0000","last_unstale":"2024-04-26T23:37:13.801811+0000","last_undegraded":"2024-04-26T23:37:13.801811+0000","last_fullsized":"2024-04-26T23:37:13.801811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795710+0000","last_change":"2024-04-26T23:37:08.768747+0000","last_active":"2024-04-26T23:37:13.795710+0000","last_peered":"2024-04-26T23:37:13.795710+0000","last_clean":"2024-04-26T23:37:13.795710+0000","last_became_active":"2024-04-26T23:37:08.768413+0000","last_became_peered":"2024-04-26T23:37:08.768413+0000","last_unstale":"2024-04-26T23:37:13.795710+0000","last_undegraded":"2024-04-26T23:37:13.795710+0000","last_fullsized":"2024-04-26T23:37:13.795710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891826+0000","last_change":"2024-04-26T23:37:08.788540+0000","last_active":"2024-04-26T23:37:13.891826+0000","last_peered":"2024-04-26T23:37:13.891826+0000","last_clean":"2024-04-26T23:37:13.891826+0000","last_became_active":"2024-04-26T23:37:08.788200+0000","last_became_peered":"2024-04-26T23:37:08.788200+0000","last_unstale":"2024-04-26T23:37:13.891826+0000","last_undegraded":"2024-04-26T23:37:13.891826+0000","last_fullsized":"2024-04-26T23:37:13.891826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891950+0000","last_change":"2024-04-26T23:37:08.777509+0000","last_active":"2024-04-26T23:37:13.891950+0000","last_peered":"2024-04-26T23:37:13.891950+0000","last_clean":"2024-04-26T23:37:13.891950+0000","last_became_active":"2024-04-26T23:37:08.777061+0000","last_became_peered":"2024-04-26T23:37:08.777061+0000","last_unstale":"2024-04-26T23:37:13.891950+0000","last_undegraded":"2024-04-26T23:37:13.891950+0000","last_fullsized":"2024-04-26T23:37:13.891950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891626+0000","last_change":"2024-04-26T23:37:08.774973+0000","last_active":"2024-04-26T23:37:13.891626+0000","last_peered":"2024-04-26T23:37:13.891626+0000","last_clean":"2024-04-26T23:37:13.891626+0000","last_became_active":"2024-04-26T23:37:08.774707+0000","last_became_peered":"2024-04-26T23:37:08.774707+0000","last_unstale":"2024-04-26T23:37:13.891626+0000","last_undegraded":"2024-04-26T23:37:13.891626+0000","last_fullsized":"2024-04-26T23:37:13.891626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892661+0000","last_change":"2024-04-26T23:37:08.781269+0000","last_active":"2024-04-26T23:37:13.892661+0000","last_peered":"2024-04-26T23:37:13.892661+0000","last_clean":"2024-04-26T23:37:13.892661+0000","last_became_active":"2024-04-26T23:37:08.780337+0000","last_became_peered":"2024-04-26T23:37:08.780337+0000","last_unstale":"2024-04-26T23:37:13.892661+0000","last_undegraded":"2024-04-26T23:37:13.892661+0000","last_fullsized":"2024-04-26T23:37:13.892661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799802+0000","last_change":"2024-04-26T23:37:08.771877+0000","last_active":"2024-04-26T23:37:13.799802+0000","last_peered":"2024-04-26T23:37:13.799802+0000","last_clean":"2024-04-26T23:37:13.799802+0000","last_became_active":"2024-04-26T23:37:08.770810+0000","last_became_peered":"2024-04-26T23:37:08.770810+0000","last_unstale":"2024-04-26T23:37:13.799802+0000","last_undegraded":"2024-04-26T23:37:13.799802+0000","last_fullsized":"2024-04-26T23:37:13.799802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890670+0000","last_change":"2024-04-26T23:37:08.780735+0000","last_active":"2024-04-26T23:37:13.890670+0000","last_peered":"2024-04-26T23:37:13.890670+0000","last_clean":"2024-04-26T23:37:13.890670+0000","last_became_active":"2024-04-26T23:37:08.780384+0000","last_became_peered":"2024-04-26T23:37:08.780384+0000","last_unstale":"2024-04-26T23:37:13.890670+0000","last_undegraded":"2024-04-26T23:37:13.890670+0000","last_fullsized":"2024-04-26T23:37:13.890670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801555+0000","last_change":"2024-04-26T23:37:08.790073+0000","last_active":"2024-04-26T23:37:13.801555+0000","last_peered":"2024-04-26T23:37:13.801555+0000","last_clean":"2024-04-26T23:37:13.801555+0000","last_became_active":"2024-04-26T23:37:08.789431+0000","last_became_peered":"2024-04-26T23:37:08.789431+0000","last_unstale":"2024-04-26T23:37:13.801555+0000","last_undegraded":"2024-04-26T23:37:13.801555+0000","last_fullsized":"2024-04-26T23:37:13.801555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795276+0000","last_change":"2024-04-26T23:37:08.756179+0000","last_active":"2024-04-26T23:37:13.795276+0000","last_peered":"2024-04-26T23:37:13.795276+0000","last_clean":"2024-04-26T23:37:13.795276+0000","last_became_active":"2024-04-26T23:37:08.755805+0000","last_became_peered":"2024-04-26T23:37:08.755805+0000","last_unstale":"2024-04-26T23:37:13.795276+0000","last_undegraded":"2024-04-26T23:37:13.795276+0000","last_fullsized":"2024-04-26T23:37:13.795276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891285+0000","last_change":"2024-04-26T23:37:08.768542+0000","last_active":"2024-04-26T23:37:13.891285+0000","last_peered":"2024-04-26T23:37:13.891285+0000","last_clean":"2024-04-26T23:37:13.891285+0000","last_became_active":"2024-04-26T23:37:08.765841+0000","last_became_peered":"2024-04-26T23:37:08.765841+0000","last_unstale":"2024-04-26T23:37:13.891285+0000","last_undegraded":"2024-04-26T23:37:13.891285+0000","last_fullsized":"2024-04-26T23:37:13.891285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893105+0000","last_change":"2024-04-26T23:37:08.775312+0000","last_active":"2024-04-26T23:37:13.893105+0000","last_peered":"2024-04-26T23:37:13.893105+0000","last_clean":"2024-04-26T23:37:13.893105+0000","last_became_active":"2024-04-26T23:37:08.774944+0000","last_became_peered":"2024-04-26T23:37:08.774944+0000","last_unstale":"2024-04-26T23:37:13.893105+0000","last_undegraded":"2024-04-26T23:37:13.893105+0000","last_fullsized":"2024-04-26T23:37:13.893105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891448+0000","last_change":"2024-04-26T23:37:08.772525+0000","last_active":"2024-04-26T23:37:13.891448+0000","last_peered":"2024-04-26T23:37:13.891448+0000","last_clean":"2024-04-26T23:37:13.891448+0000","last_became_active":"2024-04-26T23:37:08.772177+0000","last_became_peered":"2024-04-26T23:37:08.772177+0000","last_unstale":"2024-04-26T23:37:13.891448+0000","last_undegraded":"2024-04-26T23:37:13.891448+0000","last_fullsized":"2024-04-26T23:37:13.891448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:55.136157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803582+0000","last_change":"2024-04-26T23:37:08.790236+0000","last_active":"2024-04-26T23:37:13.803582+0000","last_peered":"2024-04-26T23:37:13.803582+0000","last_clean":"2024-04-26T23:37:13.803582+0000","last_became_active":"2024-04-26T23:37:08.789885+0000","last_became_peered":"2024-04-26T23:37:08.789885+0000","last_unstale":"2024-04-26T23:37:13.803582+0000","last_undegraded":"2024-04-26T23:37:13.803582+0000","last_fullsized":"2024-04-26T23:37:13.803582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799434+0000","last_change":"2024-04-26T23:37:08.780732+0000","last_active":"2024-04-26T23:37:13.799434+0000","last_peered":"2024-04-26T23:37:13.799434+0000","last_clean":"2024-04-26T23:37:13.799434+0000","last_became_active":"2024-04-26T23:37:08.779046+0000","last_became_peered":"2024-04-26T23:37:08.779046+0000","last_unstale":"2024-04-26T23:37:13.799434+0000","last_undegraded":"2024-04-26T23:37:13.799434+0000","last_fullsized":"2024-04-26T23:37:13.799434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888183+0000","last_change":"2024-04-26T23:37:08.751809+0000","last_active":"2024-04-26T23:37:13.888183+0000","last_peered":"2024-04-26T23:37:13.888183+0000","last_clean":"2024-04-26T23:37:13.888183+0000","last_became_active":"2024-04-26T23:37:08.750955+0000","last_became_peered":"2024-04-26T23:37:08.750955+0000","last_unstale":"2024-04-26T23:37:13.888183+0000","last_undegraded":"2024-04-26T23:37:13.888183+0000","last_fullsized":"2024-04-26T23:37:13.888183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795084+0000","last_change":"2024-04-26T23:37:08.762870+0000","last_active":"2024-04-26T23:37:13.795084+0000","last_peered":"2024-04-26T23:37:13.795084+0000","last_clean":"2024-04-26T23:37:13.795084+0000","last_became_active":"2024-04-26T23:37:08.762554+0000","last_became_peered":"2024-04-26T23:37:08.762554+0000","last_unstale":"2024-04-26T23:37:13.795084+0000","last_undegraded":"2024-04-26T23:37:13.795084+0000","last_fullsized":"2024-04-26T23:37:13.795084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:05.526812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891059+0000","last_change":"2024-04-26T23:37:08.774910+0000","last_active":"2024-04-26T23:37:13.891059+0000","last_peered":"2024-04-26T23:37:13.891059+0000","last_clean":"2024-04-26T23:37:13.891059+0000","last_became_active":"2024-04-26T23:37:08.774587+0000","last_became_peered":"2024-04-26T23:37:08.774587+0000","last_unstale":"2024-04-26T23:37:13.891059+0000","last_undegraded":"2024-04-26T23:37:13.891059+0000","last_fullsized":"2024-04-26T23:37:13.891059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800959+0000","last_change":"2024-04-26T23:37:08.778389+0000","last_active":"2024-04-26T23:37:13.800959+0000","last_peered":"2024-04-26T23:37:13.800959+0000","last_clean":"2024-04-26T23:37:13.800959+0000","last_became_active":"2024-04-26T23:37:08.778116+0000","last_became_peered":"2024-04-26T23:37:08.778116+0000","last_unstale":"2024-04-26T23:37:13.800959+0000","last_undegraded":"2024-04-26T23:37:13.800959+0000","last_fullsized":"2024-04-26T23:37:13.800959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892362+0000","last_change":"2024-04-26T23:37:08.781929+0000","last_active":"2024-04-26T23:37:13.892362+0000","last_peered":"2024-04-26T23:37:13.892362+0000","last_clean":"2024-04-26T23:37:13.892362+0000","last_became_active":"2024-04-26T23:37:08.780724+0000","last_became_peered":"2024-04-26T23:37:08.780724+0000","last_unstale":"2024-04-26T23:37:13.892362+0000","last_undegraded":"2024-04-26T23:37:13.892362+0000","last_fullsized":"2024-04-26T23:37:13.892362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891324+0000","last_change":"2024-04-26T23:37:08.792579+0000","last_active":"2024-04-26T23:37:13.891324+0000","last_peered":"2024-04-26T23:37:13.891324+0000","last_clean":"2024-04-26T23:37:13.891324+0000","last_became_active":"2024-04-26T23:37:08.792171+0000","last_became_peered":"2024-04-26T23:37:08.792171+0000","last_unstale":"2024-04-26T23:37:13.891324+0000","last_undegraded":"2024-04-26T23:37:13.891324+0000","last_fullsized":"2024-04-26T23:37:13.891324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893856+0000","last_change":"2024-04-26T23:37:08.795179+0000","last_active":"2024-04-26T23:37:13.893856+0000","last_peered":"2024-04-26T23:37:13.893856+0000","last_clean":"2024-04-26T23:37:13.893856+0000","last_became_active":"2024-04-26T23:37:08.794890+0000","last_became_peered":"2024-04-26T23:37:08.794890+0000","last_unstale":"2024-04-26T23:37:13.893856+0000","last_undegraded":"2024-04-26T23:37:13.893856+0000","last_fullsized":"2024-04-26T23:37:13.893856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889647+0000","last_change":"2024-04-26T23:37:08.786739+0000","last_active":"2024-04-26T23:37:13.889647+0000","last_peered":"2024-04-26T23:37:13.889647+0000","last_clean":"2024-04-26T23:37:13.889647+0000","last_became_active":"2024-04-26T23:37:08.786360+0000","last_became_peered":"2024-04-26T23:37:08.786360+0000","last_unstale":"2024-04-26T23:37:13.889647+0000","last_undegraded":"2024-04-26T23:37:13.889647+0000","last_fullsized":"2024-04-26T23:37:13.889647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891212+0000","last_change":"2024-04-26T23:37:08.775748+0000","last_active":"2024-04-26T23:37:13.891212+0000","last_peered":"2024-04-26T23:37:13.891212+0000","last_clean":"2024-04-26T23:37:13.891212+0000","last_became_active":"2024-04-26T23:37:08.775440+0000","last_became_peered":"2024-04-26T23:37:08.775440+0000","last_unstale":"2024-04-26T23:37:13.891212+0000","last_undegraded":"2024-04-26T23:37:13.891212+0000","last_fullsized":"2024-04-26T23:37:13.891212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801324+0000","last_change":"2024-04-26T23:37:08.791665+0000","last_active":"2024-04-26T23:37:13.801324+0000","last_peered":"2024-04-26T23:37:13.801324+0000","last_clean":"2024-04-26T23:37:13.801324+0000","last_became_active":"2024-04-26T23:37:08.790305+0000","last_became_peered":"2024-04-26T23:37:08.790305+0000","last_unstale":"2024-04-26T23:37:13.801324+0000","last_undegraded":"2024-04-26T23:37:13.801324+0000","last_fullsized":"2024-04-26T23:37:13.801324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891978+0000","last_change":"2024-04-26T23:37:08.776000+0000","last_active":"2024-04-26T23:37:13.891978+0000","last_peered":"2024-04-26T23:37:13.891978+0000","last_clean":"2024-04-26T23:37:13.891978+0000","last_became_active":"2024-04-26T23:37:08.773615+0000","last_became_peered":"2024-04-26T23:37:08.773615+0000","last_unstale":"2024-04-26T23:37:13.891978+0000","last_undegraded":"2024-04-26T23:37:13.891978+0000","last_fullsized":"2024-04-26T23:37:13.891978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802456+0000","last_change":"2024-04-26T23:37:08.789506+0000","last_active":"2024-04-26T23:37:13.802456+0000","last_peered":"2024-04-26T23:37:13.802456+0000","last_clean":"2024-04-26T23:37:13.802456+0000","last_became_active":"2024-04-26T23:37:08.789180+0000","last_became_peered":"2024-04-26T23:37:08.789180+0000","last_unstale":"2024-04-26T23:37:13.802456+0000","last_undegraded":"2024-04-26T23:37:13.802456+0000","last_fullsized":"2024-04-26T23:37:13.802456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794898+0000","last_change":"2024-04-26T23:37:08.766757+0000","last_active":"2024-04-26T23:37:13.794898+0000","last_peered":"2024-04-26T23:37:13.794898+0000","last_clean":"2024-04-26T23:37:13.794898+0000","last_became_active":"2024-04-26T23:37:08.761659+0000","last_became_peered":"2024-04-26T23:37:08.761659+0000","last_unstale":"2024-04-26T23:37:13.794898+0000","last_undegraded":"2024-04-26T23:37:13.794898+0000","last_fullsized":"2024-04-26T23:37:13.794898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890719+0000","last_change":"2024-04-26T23:37:08.761968+0000","last_active":"2024-04-26T23:37:13.890719+0000","last_peered":"2024-04-26T23:37:13.890719+0000","last_clean":"2024-04-26T23:37:13.890719+0000","last_became_active":"2024-04-26T23:37:08.761546+0000","last_became_peered":"2024-04-26T23:37:08.761546+0000","last_unstale":"2024-04-26T23:37:13.890719+0000","last_undegraded":"2024-04-26T23:37:13.890719+0000","last_fullsized":"2024-04-26T23:37:13.890719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893557+0000","last_change":"2024-04-26T23:37:08.775290+0000","last_active":"2024-04-26T23:37:13.893557+0000","last_peered":"2024-04-26T23:37:13.893557+0000","last_clean":"2024-04-26T23:37:13.893557+0000","last_became_active":"2024-04-26T23:37:08.774646+0000","last_became_peered":"2024-04-26T23:37:08.774646+0000","last_unstale":"2024-04-26T23:37:13.893557+0000","last_undegraded":"2024-04-26T23:37:13.893557+0000","last_fullsized":"2024-04-26T23:37:13.893557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:37:38.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:37:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.709+0000 7ffb85e00640 1 Processor -- start 2024-04-26T23:37:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.709+0000 7ffb85e00640 1 -- start start 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.710+0000 7ffb85e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb80162a60 0x7ffb800d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.710+0000 7ffb85e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffb800d5a90 0x7ffb800b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.710+0000 7ffb85e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb800b65e0 con 0x7ffb800b4c50 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb85e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffb800d12e0 con 0x7ffb80162a60 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb85e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffb800d05f0 con 0x7ffb800d5a90 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb80162a60 0x7ffb800d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb80162a60 0x7ffb800d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37374/0 (socket says 172.21.15.46:37374) 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 -- 172.21.15.46:0/1474048990 learned_addr learned my addr 172.21.15.46:0/1474048990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb7fe00640 1 --2- 172.21.15.46:0/1474048990 >> v2:172.21.15.142:3300/0 conn(0x7ffb800d5a90 0x7ffb800b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 -- 172.21.15.46:0/1474048990 >> v2:172.21.15.142:3300/0 conn(0x7ffb800d5a90 msgr2=0x7ffb800b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 --2- 172.21.15.46:0/1474048990 >> v2:172.21.15.142:3300/0 conn(0x7ffb800d5a90 0x7ffb800b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 -- 172.21.15.46:0/1474048990 >> v1:172.21.15.142:6789/0 conn(0x7ffb800b4c50 legacy=0x7ffb800d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.711+0000 7ffb84a00640 1 -- 172.21.15.46:0/1474048990 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb800d1530 con 0x7ffb80162a60 2024-04-26T23:37:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.712+0000 7ffb84a00640 1 --2- 172.21.15.46:0/1474048990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb80162a60 0x7ffb800d0a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006f5e0 tx=0x7ffb7402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956dc6245e484ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb7f400640 1 -- 172.21.15.46:0/1474048990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74037070 con 0x7ffb80162a60 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb7f400640 1 -- 172.21.15.46:0/1474048990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7402ae20 con 0x7ffb80162a60 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb7f400640 1 -- 172.21.15.46:0/1474048990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74032660 con 0x7ffb80162a60 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 -- 172.21.15.46:0/1474048990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb80162a60 msgr2=0x7ffb800d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 --2- 172.21.15.46:0/1474048990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb80162a60 0x7ffb800d0a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006f5e0 tx=0x7ffb7402a110 comp rx=0 tx=0).stop 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 -- 172.21.15.46:0/1474048990 shutdown_connections 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 --2- 172.21.15.46:0/1474048990 >> v2:172.21.15.142:3300/0 conn(0x7ffb800d5a90 0x7ffb800b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 --2- 172.21.15.46:0/1474048990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb80162a60 0x7ffb800d0a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.713+0000 7ffb85e00640 1 -- 172.21.15.46:0/1474048990 >> 172.21.15.46:0/1474048990 conn(0x7ffb800d4fa0 msgr2=0x7ffb800d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.714+0000 7ffb85e00640 1 -- 172.21.15.46:0/1474048990 shutdown_connections 2024-04-26T23:37:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.714+0000 7ffb85e00640 1 -- 172.21.15.46:0/1474048990 wait complete. 2024-04-26T23:37:38.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.715+0000 7ffb85e00640 1 Processor -- start 2024-04-26T23:37:38.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.716+0000 7ffb85e00640 1 -- start start 2024-04-26T23:37:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.716+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.716+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 0x7ffb800c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 0x7ffb800c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb800b5200 con 0x7ffb800b4c50 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb800b5080 con 0x7ffb800d5a90 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb800b5380 con 0x7ffb80162a60 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37388/0 (socket says 172.21.15.46:37388) 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 -- 172.21.15.46:0/2667575628 learned_addr learned my addr 172.21.15.46:0/2667575628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb7fe00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 0x7ffb800c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 -- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 msgr2=0x7ffb800c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85400640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 0x7ffb800c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 0x7ffb800c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 -- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 msgr2=0x7ffb800c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 0x7ffb800c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb84a00640 1 -- 172.21.15.46:0/2667575628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb800cad30 con 0x7ffb800b4c50 2024-04-26T23:37:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb7fe00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 0x7ffb800c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.717+0000 7ffb85400640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 0x7ffb800c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb84a00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb74000c00 tx=0x7ffb740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74041070 con 0x7ffb800b4c50 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7403c670 con 0x7ffb800b4c50 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74037040 con 0x7ffb800b4c50 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb800cb010 con 0x7ffb800b4c50 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.718+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb800cb5d0 con 0x7ffb800b4c50 2024-04-26T23:37:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.719+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb800ce360 con 0x7ffb800b4c50 2024-04-26T23:37:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.721+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb74044070 con 0x7ffb800b4c50 2024-04-26T23:37:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.721+0000 7ffb7cc00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 0x7ffb6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.722+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb740ac630 con 0x7ffb800b4c50 2024-04-26T23:37:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.722+0000 7ffb7fe00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 0x7ffb6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.723+0000 7ffb7fe00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 0x7ffb6406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006d100 tx=0x7ffb70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:38.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.726+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb7407a680 con 0x7ffb800b4c50 2024-04-26T23:37:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.925+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb80069ac0 con 0x7ffb6406c8f0 2024-04-26T23:37:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.948+0000 7ffb7cc00640 1 -- 172.21.15.46:0/2667575628 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7ffb80069ac0 con 0x7ffb6406c8f0 2024-04-26T23:37:38.956 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.955+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 msgr2=0x7ffb6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.955+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 0x7ffb6406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006d100 tx=0x7ffb70008040 comp rx=0 tx=0).stop 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 msgr2=0x7ffb8014ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb74000c00 tx=0x7ffb740046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 shutdown_connections 2024-04-26T23:37:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb6406c8f0 0x7ffb6406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb80162a60 0x7ffb800c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb800d5a90 0x7ffb800c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 --2- 172.21.15.46:0/2667575628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb800b4c50 0x7ffb8014ef90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 >> 172.21.15.46:0/2667575628 conn(0x7ffb800d4fa0 msgr2=0x7ffb8015fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 shutdown_connections 2024-04-26T23:37:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:38.956+0000 7ffb85e00640 1 -- 172.21.15.46:0/2667575628 wait complete. 2024-04-26T23:37:38.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T23:37:37.892044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223640,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751080,"statfs":{"total":773094113280,"available":772865105920,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004993"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795609+0000","last_change":"2024-04-26T23:37:07.740855+0000","last_active":"2024-04-26T23:37:13.795609+0000","last_peered":"2024-04-26T23:37:13.795609+0000","last_clean":"2024-04-26T23:37:13.795609+0000","last_became_active":"2024-04-26T23:37:07.739396+0000","last_became_peered":"2024-04-26T23:37:07.739396+0000","last_unstale":"2024-04-26T23:37:13.795609+0000","last_undegraded":"2024-04-26T23:37:13.795609+0000","last_fullsized":"2024-04-26T23:37:13.795609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:05.866636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892052+0000","last_change":"2024-04-26T23:37:08.773954+0000","last_active":"2024-04-26T23:37:13.892052+0000","last_peered":"2024-04-26T23:37:13.892052+0000","last_clean":"2024-04-26T23:37:13.892052+0000","last_became_active":"2024-04-26T23:37:08.771001+0000","last_became_peered":"2024-04-26T23:37:08.771001+0000","last_unstale":"2024-04-26T23:37:13.892052+0000","last_undegraded":"2024-04-26T23:37:13.892052+0000","last_fullsized":"2024-04-26T23:37:13.892052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890219+0000","last_change":"2024-04-26T23:37:13.098670+0000","last_active":"2024-04-26T23:37:13.890219+0000","last_peered":"2024-04-26T23:37:13.890219+0000","last_clean":"2024-04-26T23:37:13.890219+0000","last_became_active":"2024-04-26T23:37:13.098244+0000","last_became_peered":"2024-04-26T23:37:13.098244+0000","last_unstale":"2024-04-26T23:37:13.890219+0000","last_undegraded":"2024-04-26T23:37:13.890219+0000","last_fullsized":"2024-04-26T23:37:13.890219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892226+0000","last_change":"2024-04-26T23:37:07.729205+0000","last_active":"2024-04-26T23:37:13.892226+0000","last_peered":"2024-04-26T23:37:13.892226+0000","last_clean":"2024-04-26T23:37:13.892226+0000","last_became_active":"2024-04-26T23:37:07.728871+0000","last_became_peered":"2024-04-26T23:37:07.728871+0000","last_unstale":"2024-04-26T23:37:13.892226+0000","last_undegraded":"2024-04-26T23:37:13.892226+0000","last_fullsized":"2024-04-26T23:37:13.892226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795238+0000","last_change":"2024-04-26T23:37:08.793071+0000","last_active":"2024-04-26T23:37:13.795238+0000","last_peered":"2024-04-26T23:37:13.795238+0000","last_clean":"2024-04-26T23:37:13.795238+0000","last_became_active":"2024-04-26T23:37:08.792784+0000","last_became_peered":"2024-04-26T23:37:08.792784+0000","last_unstale":"2024-04-26T23:37:13.795238+0000","last_undegraded":"2024-04-26T23:37:13.795238+0000","last_fullsized":"2024-04-26T23:37:13.795238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798412+0000","last_change":"2024-04-26T23:37:12.830234+0000","last_active":"2024-04-26T23:37:13.798412+0000","last_peered":"2024-04-26T23:37:13.798412+0000","last_clean":"2024-04-26T23:37:13.798412+0000","last_became_active":"2024-04-26T23:37:12.829735+0000","last_became_peered":"2024-04-26T23:37:12.829735+0000","last_unstale":"2024-04-26T23:37:13.798412+0000","last_undegraded":"2024-04-26T23:37:13.798412+0000","last_fullsized":"2024-04-26T23:37:13.798412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:01.085576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893008+0000","last_change":"2024-04-26T23:37:07.755959+0000","last_active":"2024-04-26T23:37:13.893008+0000","last_peered":"2024-04-26T23:37:13.893008+0000","last_clean":"2024-04-26T23:37:13.893008+0000","last_became_active":"2024-04-26T23:37:07.754315+0000","last_became_peered":"2024-04-26T23:37:07.754315+0000","last_unstale":"2024-04-26T23:37:13.893008+0000","last_undegraded":"2024-04-26T23:37:13.893008+0000","last_fullsized":"2024-04-26T23:37:13.893008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802750+0000","last_change":"2024-04-26T23:37:08.786529+0000","last_active":"2024-04-26T23:37:13.802750+0000","last_peered":"2024-04-26T23:37:13.802750+0000","last_clean":"2024-04-26T23:37:13.802750+0000","last_became_active":"2024-04-26T23:37:08.781335+0000","last_became_peered":"2024-04-26T23:37:08.781335+0000","last_unstale":"2024-04-26T23:37:13.802750+0000","last_undegraded":"2024-04-26T23:37:13.802750+0000","last_fullsized":"2024-04-26T23:37:13.802750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798764+0000","last_change":"2024-04-26T23:37:12.830290+0000","last_active":"2024-04-26T23:37:13.798764+0000","last_peered":"2024-04-26T23:37:13.798764+0000","last_clean":"2024-04-26T23:37:13.798764+0000","last_became_active":"2024-04-26T23:37:12.829772+0000","last_became_peered":"2024-04-26T23:37:12.829772+0000","last_unstale":"2024-04-26T23:37:13.798764+0000","last_undegraded":"2024-04-26T23:37:13.798764+0000","last_fullsized":"2024-04-26T23:37:13.798764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:01.225018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892372+0000","last_change":"2024-04-26T23:37:07.756056+0000","last_active":"2024-04-26T23:37:13.892372+0000","last_peered":"2024-04-26T23:37:13.892372+0000","last_clean":"2024-04-26T23:37:13.892372+0000","last_became_active":"2024-04-26T23:37:07.754918+0000","last_became_peered":"2024-04-26T23:37:07.754918+0000","last_unstale":"2024-04-26T23:37:13.892372+0000","last_undegraded":"2024-04-26T23:37:13.892372+0000","last_fullsized":"2024-04-26T23:37:13.892372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:08.953262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803127+0000","last_change":"2024-04-26T23:37:08.780722+0000","last_active":"2024-04-26T23:37:13.803127+0000","last_peered":"2024-04-26T23:37:13.803127+0000","last_clean":"2024-04-26T23:37:13.803127+0000","last_became_active":"2024-04-26T23:37:08.780223+0000","last_became_peered":"2024-04-26T23:37:08.780223+0000","last_unstale":"2024-04-26T23:37:13.803127+0000","last_undegraded":"2024-04-26T23:37:13.803127+0000","last_fullsized":"2024-04-26T23:37:13.803127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798239+0000","last_change":"2024-04-26T23:37:12.831066+0000","last_active":"2024-04-26T23:37:13.798239+0000","last_peered":"2024-04-26T23:37:13.798239+0000","last_clean":"2024-04-26T23:37:13.798239+0000","last_became_active":"2024-04-26T23:37:12.829941+0000","last_became_peered":"2024-04-26T23:37:12.829941+0000","last_unstale":"2024-04-26T23:37:13.798239+0000","last_undegraded":"2024-04-26T23:37:13.798239+0000","last_fullsized":"2024-04-26T23:37:13.798239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:04.322071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890099+0000","last_change":"2024-04-26T23:37:08.768304+0000","last_active":"2024-04-26T23:37:13.890099+0000","last_peered":"2024-04-26T23:37:13.890099+0000","last_clean":"2024-04-26T23:37:13.890099+0000","last_became_active":"2024-04-26T23:37:08.767971+0000","last_became_peered":"2024-04-26T23:37:08.767971+0000","last_unstale":"2024-04-26T23:37:13.890099+0000","last_undegraded":"2024-04-26T23:37:13.890099+0000","last_fullsized":"2024-04-26T23:37:13.890099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892460+0000","last_change":"2024-04-26T23:37:07.755890+0000","last_active":"2024-04-26T23:37:13.892460+0000","last_peered":"2024-04-26T23:37:13.892460+0000","last_clean":"2024-04-26T23:37:13.892460+0000","last_became_active":"2024-04-26T23:37:07.753801+0000","last_became_peered":"2024-04-26T23:37:07.753801+0000","last_unstale":"2024-04-26T23:37:13.892460+0000","last_undegraded":"2024-04-26T23:37:13.892460+0000","last_fullsized":"2024-04-26T23:37:13.892460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800561+0000","last_change":"2024-04-26T23:37:12.821170+0000","last_active":"2024-04-26T23:37:13.800561+0000","last_peered":"2024-04-26T23:37:13.800561+0000","last_clean":"2024-04-26T23:37:13.800561+0000","last_became_active":"2024-04-26T23:37:12.820739+0000","last_became_peered":"2024-04-26T23:37:12.820739+0000","last_unstale":"2024-04-26T23:37:13.800561+0000","last_undegraded":"2024-04-26T23:37:13.800561+0000","last_fullsized":"2024-04-26T23:37:13.800561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891188+0000","last_change":"2024-04-26T23:37:08.772794+0000","last_active":"2024-04-26T23:37:13.891188+0000","last_peered":"2024-04-26T23:37:13.891188+0000","last_clean":"2024-04-26T23:37:13.891188+0000","last_became_active":"2024-04-26T23:37:08.770226+0000","last_became_peered":"2024-04-26T23:37:08.770226+0000","last_unstale":"2024-04-26T23:37:13.891188+0000","last_undegraded":"2024-04-26T23:37:13.891188+0000","last_fullsized":"2024-04-26T23:37:13.891188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888791+0000","last_change":"2024-04-26T23:37:07.739350+0000","last_active":"2024-04-26T23:37:13.888791+0000","last_peered":"2024-04-26T23:37:13.888791+0000","last_clean":"2024-04-26T23:37:13.888791+0000","last_became_active":"2024-04-26T23:37:07.739007+0000","last_became_peered":"2024-04-26T23:37:07.739007+0000","last_unstale":"2024-04-26T23:37:13.888791+0000","last_undegraded":"2024-04-26T23:37:13.888791+0000","last_fullsized":"2024-04-26T23:37:13.888791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800746+0000","last_change":"2024-04-26T23:37:12.831545+0000","last_active":"2024-04-26T23:37:13.800746+0000","last_peered":"2024-04-26T23:37:13.800746+0000","last_clean":"2024-04-26T23:37:13.800746+0000","last_became_active":"2024-04-26T23:37:12.829548+0000","last_became_peered":"2024-04-26T23:37:12.829548+0000","last_unstale":"2024-04-26T23:37:13.800746+0000","last_undegraded":"2024-04-26T23:37:13.800746+0000","last_fullsized":"2024-04-26T23:37:13.800746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:31.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889936+0000","last_change":"2024-04-26T23:37:08.761785+0000","last_active":"2024-04-26T23:37:13.889936+0000","last_peered":"2024-04-26T23:37:13.889936+0000","last_clean":"2024-04-26T23:37:13.889936+0000","last_became_active":"2024-04-26T23:37:08.759700+0000","last_became_peered":"2024-04-26T23:37:08.759700+0000","last_unstale":"2024-04-26T23:37:13.889936+0000","last_undegraded":"2024-04-26T23:37:13.889936+0000","last_fullsized":"2024-04-26T23:37:13.889936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888332+0000","last_change":"2024-04-26T23:37:07.738157+0000","last_active":"2024-04-26T23:37:13.888332+0000","last_peered":"2024-04-26T23:37:13.888332+0000","last_clean":"2024-04-26T23:37:13.888332+0000","last_became_active":"2024-04-26T23:37:07.737830+0000","last_became_peered":"2024-04-26T23:37:07.737830+0000","last_unstale":"2024-04-26T23:37:13.888332+0000","last_undegraded":"2024-04-26T23:37:13.888332+0000","last_fullsized":"2024-04-26T23:37:13.888332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892316+0000","last_change":"2024-04-26T23:37:13.099265+0000","last_active":"2024-04-26T23:37:13.892316+0000","last_peered":"2024-04-26T23:37:13.892316+0000","last_clean":"2024-04-26T23:37:13.892316+0000","last_became_active":"2024-04-26T23:37:13.095798+0000","last_became_peered":"2024-04-26T23:37:13.095798+0000","last_unstale":"2024-04-26T23:37:13.892316+0000","last_undegraded":"2024-04-26T23:37:13.892316+0000","last_fullsized":"2024-04-26T23:37:13.892316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800680+0000","last_change":"2024-04-26T23:37:08.764943+0000","last_active":"2024-04-26T23:37:13.800680+0000","last_peered":"2024-04-26T23:37:13.800680+0000","last_clean":"2024-04-26T23:37:13.800680+0000","last_became_active":"2024-04-26T23:37:08.764636+0000","last_became_peered":"2024-04-26T23:37:08.764636+0000","last_unstale":"2024-04-26T23:37:13.800680+0000","last_undegraded":"2024-04-26T23:37:13.800680+0000","last_fullsized":"2024-04-26T23:37:13.800680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:29.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890217+0000","last_change":"2024-04-26T23:37:07.753035+0000","last_active":"2024-04-26T23:37:13.890217+0000","last_peered":"2024-04-26T23:37:13.890217+0000","last_clean":"2024-04-26T23:37:13.890217+0000","last_became_active":"2024-04-26T23:37:07.752363+0000","last_became_peered":"2024-04-26T23:37:07.752363+0000","last_unstale":"2024-04-26T23:37:13.890217+0000","last_undegraded":"2024-04-26T23:37:13.890217+0000","last_fullsized":"2024-04-26T23:37:13.890217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799225+0000","last_change":"2024-04-26T23:37:12.830106+0000","last_active":"2024-04-26T23:37:13.799225+0000","last_peered":"2024-04-26T23:37:13.799225+0000","last_clean":"2024-04-26T23:37:13.799225+0000","last_became_active":"2024-04-26T23:37:12.829676+0000","last_became_peered":"2024-04-26T23:37:12.829676+0000","last_unstale":"2024-04-26T23:37:13.799225+0000","last_undegraded":"2024-04-26T23:37:13.799225+0000","last_fullsized":"2024-04-26T23:37:13.799225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795807+0000","last_change":"2024-04-26T23:37:08.764908+0000","last_active":"2024-04-26T23:37:13.795807+0000","last_peered":"2024-04-26T23:37:13.795807+0000","last_clean":"2024-04-26T23:37:13.795807+0000","last_became_active":"2024-04-26T23:37:08.764369+0000","last_became_peered":"2024-04-26T23:37:08.764369+0000","last_unstale":"2024-04-26T23:37:13.795807+0000","last_undegraded":"2024-04-26T23:37:13.795807+0000","last_fullsized":"2024-04-26T23:37:13.795807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892624+0000","last_change":"2024-04-26T23:37:07.751902+0000","last_active":"2024-04-26T23:37:13.892624+0000","last_peered":"2024-04-26T23:37:13.892624+0000","last_clean":"2024-04-26T23:37:13.892624+0000","last_became_active":"2024-04-26T23:37:07.751383+0000","last_became_peered":"2024-04-26T23:37:07.751383+0000","last_unstale":"2024-04-26T23:37:13.892624+0000","last_undegraded":"2024-04-26T23:37:13.892624+0000","last_fullsized":"2024-04-26T23:37:13.892624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800477+0000","last_change":"2024-04-26T23:37:13.090483+0000","last_active":"2024-04-26T23:37:13.800477+0000","last_peered":"2024-04-26T23:37:13.800477+0000","last_clean":"2024-04-26T23:37:13.800477+0000","last_became_active":"2024-04-26T23:37:13.088991+0000","last_became_peered":"2024-04-26T23:37:13.088991+0000","last_unstale":"2024-04-26T23:37:13.800477+0000","last_undegraded":"2024-04-26T23:37:13.800477+0000","last_fullsized":"2024-04-26T23:37:13.800477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892790+0000","last_change":"2024-04-26T23:37:08.777852+0000","last_active":"2024-04-26T23:37:13.892790+0000","last_peered":"2024-04-26T23:37:13.892790+0000","last_clean":"2024-04-26T23:37:13.892790+0000","last_became_active":"2024-04-26T23:37:08.777214+0000","last_became_peered":"2024-04-26T23:37:08.777214+0000","last_unstale":"2024-04-26T23:37:13.892790+0000","last_undegraded":"2024-04-26T23:37:13.892790+0000","last_fullsized":"2024-04-26T23:37:13.892790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890844+0000","last_change":"2024-04-26T23:37:07.744896+0000","last_active":"2024-04-26T23:37:13.890844+0000","last_peered":"2024-04-26T23:37:13.890844+0000","last_clean":"2024-04-26T23:37:13.890844+0000","last_became_active":"2024-04-26T23:37:07.744597+0000","last_became_peered":"2024-04-26T23:37:07.744597+0000","last_unstale":"2024-04-26T23:37:13.890844+0000","last_undegraded":"2024-04-26T23:37:13.890844+0000","last_fullsized":"2024-04-26T23:37:13.890844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889553+0000","last_change":"2024-04-26T23:37:13.097492+0000","last_active":"2024-04-26T23:37:13.889553+0000","last_peered":"2024-04-26T23:37:13.889553+0000","last_clean":"2024-04-26T23:37:13.889553+0000","last_became_active":"2024-04-26T23:37:13.096325+0000","last_became_peered":"2024-04-26T23:37:13.096325+0000","last_unstale":"2024-04-26T23:37:13.889553+0000","last_undegraded":"2024-04-26T23:37:13.889553+0000","last_fullsized":"2024-04-26T23:37:13.889553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889171+0000","last_change":"2024-04-26T23:37:08.762047+0000","last_active":"2024-04-26T23:37:13.889171+0000","last_peered":"2024-04-26T23:37:13.889171+0000","last_clean":"2024-04-26T23:37:13.889171+0000","last_became_active":"2024-04-26T23:37:08.761660+0000","last_became_peered":"2024-04-26T23:37:08.761660+0000","last_unstale":"2024-04-26T23:37:13.889171+0000","last_undegraded":"2024-04-26T23:37:13.889171+0000","last_fullsized":"2024-04-26T23:37:13.889171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893281+0000","last_change":"2024-04-26T23:37:07.753465+0000","last_active":"2024-04-26T23:37:13.893281+0000","last_peered":"2024-04-26T23:37:13.893281+0000","last_clean":"2024-04-26T23:37:13.893281+0000","last_became_active":"2024-04-26T23:37:07.752398+0000","last_became_peered":"2024-04-26T23:37:07.752398+0000","last_unstale":"2024-04-26T23:37:13.893281+0000","last_undegraded":"2024-04-26T23:37:13.893281+0000","last_fullsized":"2024-04-26T23:37:13.893281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795198+0000","last_change":"2024-04-26T23:37:13.089856+0000","last_active":"2024-04-26T23:37:13.795198+0000","last_peered":"2024-04-26T23:37:13.795198+0000","last_clean":"2024-04-26T23:37:13.795198+0000","last_became_active":"2024-04-26T23:37:13.088870+0000","last_became_peered":"2024-04-26T23:37:13.088870+0000","last_unstale":"2024-04-26T23:37:13.795198+0000","last_undegraded":"2024-04-26T23:37:13.795198+0000","last_fullsized":"2024-04-26T23:37:13.795198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803393+0000","last_change":"2024-04-26T23:37:08.785871+0000","last_active":"2024-04-26T23:37:13.803393+0000","last_peered":"2024-04-26T23:37:13.803393+0000","last_clean":"2024-04-26T23:37:13.803393+0000","last_became_active":"2024-04-26T23:37:08.785451+0000","last_became_peered":"2024-04-26T23:37:08.785451+0000","last_unstale":"2024-04-26T23:37:13.803393+0000","last_undegraded":"2024-04-26T23:37:13.803393+0000","last_fullsized":"2024-04-26T23:37:13.803393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889253+0000","last_change":"2024-04-26T23:37:07.730561+0000","last_active":"2024-04-26T23:37:13.889253+0000","last_peered":"2024-04-26T23:37:13.889253+0000","last_clean":"2024-04-26T23:37:13.889253+0000","last_became_active":"2024-04-26T23:37:07.730274+0000","last_became_peered":"2024-04-26T23:37:07.730274+0000","last_unstale":"2024-04-26T23:37:13.889253+0000","last_undegraded":"2024-04-26T23:37:13.889253+0000","last_fullsized":"2024-04-26T23:37:13.889253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888467+0000","last_change":"2024-04-26T23:37:12.819624+0000","last_active":"2024-04-26T23:37:13.888467+0000","last_peered":"2024-04-26T23:37:13.888467+0000","last_clean":"2024-04-26T23:37:13.888467+0000","last_became_active":"2024-04-26T23:37:12.819079+0000","last_became_peered":"2024-04-26T23:37:12.819079+0000","last_unstale":"2024-04-26T23:37:13.888467+0000","last_undegraded":"2024-04-26T23:37:13.888467+0000","last_fullsized":"2024-04-26T23:37:13.888467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799143+0000","last_change":"2024-04-26T23:37:08.760531+0000","last_active":"2024-04-26T23:37:13.799143+0000","last_peered":"2024-04-26T23:37:13.799143+0000","last_clean":"2024-04-26T23:37:13.799143+0000","last_became_active":"2024-04-26T23:37:08.760250+0000","last_became_peered":"2024-04-26T23:37:08.760250+0000","last_unstale":"2024-04-26T23:37:13.799143+0000","last_undegraded":"2024-04-26T23:37:13.799143+0000","last_fullsized":"2024-04-26T23:37:13.799143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:17.138976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802089+0000","last_change":"2024-04-26T23:37:07.743324+0000","last_active":"2024-04-26T23:37:13.802089+0000","last_peered":"2024-04-26T23:37:13.802089+0000","last_clean":"2024-04-26T23:37:13.802089+0000","last_became_active":"2024-04-26T23:37:07.742056+0000","last_became_peered":"2024-04-26T23:37:07.742056+0000","last_unstale":"2024-04-26T23:37:13.802089+0000","last_undegraded":"2024-04-26T23:37:13.802089+0000","last_fullsized":"2024-04-26T23:37:13.802089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889116+0000","last_change":"2024-04-26T23:37:12.835987+0000","last_active":"2024-04-26T23:37:13.889116+0000","last_peered":"2024-04-26T23:37:13.889116+0000","last_clean":"2024-04-26T23:37:13.889116+0000","last_became_active":"2024-04-26T23:37:12.831199+0000","last_became_peered":"2024-04-26T23:37:12.831199+0000","last_unstale":"2024-04-26T23:37:13.889116+0000","last_undegraded":"2024-04-26T23:37:13.889116+0000","last_fullsized":"2024-04-26T23:37:13.889116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888008+0000","last_change":"2024-04-26T23:37:08.768283+0000","last_active":"2024-04-26T23:37:13.888008+0000","last_peered":"2024-04-26T23:37:13.888008+0000","last_clean":"2024-04-26T23:37:13.888008+0000","last_became_active":"2024-04-26T23:37:08.766512+0000","last_became_peered":"2024-04-26T23:37:08.766512+0000","last_unstale":"2024-04-26T23:37:13.888008+0000","last_undegraded":"2024-04-26T23:37:13.888008+0000","last_fullsized":"2024-04-26T23:37:13.888008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888285+0000","last_change":"2024-04-26T23:37:07.728903+0000","last_active":"2024-04-26T23:37:13.888285+0000","last_peered":"2024-04-26T23:37:13.888285+0000","last_clean":"2024-04-26T23:37:13.888285+0000","last_became_active":"2024-04-26T23:37:07.728621+0000","last_became_peered":"2024-04-26T23:37:07.728621+0000","last_unstale":"2024-04-26T23:37:13.888285+0000","last_undegraded":"2024-04-26T23:37:13.888285+0000","last_fullsized":"2024-04-26T23:37:13.888285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892519+0000","last_change":"2024-04-26T23:37:12.833651+0000","last_active":"2024-04-26T23:37:13.892519+0000","last_peered":"2024-04-26T23:37:13.892519+0000","last_clean":"2024-04-26T23:37:13.892519+0000","last_became_active":"2024-04-26T23:37:12.831533+0000","last_became_peered":"2024-04-26T23:37:12.831533+0000","last_unstale":"2024-04-26T23:37:13.892519+0000","last_undegraded":"2024-04-26T23:37:13.892519+0000","last_fullsized":"2024-04-26T23:37:13.892519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794491+0000","last_change":"2024-04-26T23:37:08.749652+0000","last_active":"2024-04-26T23:37:13.794491+0000","last_peered":"2024-04-26T23:37:13.794491+0000","last_clean":"2024-04-26T23:37:13.794491+0000","last_became_active":"2024-04-26T23:37:08.748932+0000","last_became_peered":"2024-04-26T23:37:08.748932+0000","last_unstale":"2024-04-26T23:37:13.794491+0000","last_undegraded":"2024-04-26T23:37:13.794491+0000","last_fullsized":"2024-04-26T23:37:13.794491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:04.431242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890355+0000","last_change":"2024-04-26T23:37:07.753005+0000","last_active":"2024-04-26T23:37:13.890355+0000","last_peered":"2024-04-26T23:37:13.890355+0000","last_clean":"2024-04-26T23:37:13.890355+0000","last_became_active":"2024-04-26T23:37:07.752377+0000","last_became_peered":"2024-04-26T23:37:07.752377+0000","last_unstale":"2024-04-26T23:37:13.890355+0000","last_undegraded":"2024-04-26T23:37:13.890355+0000","last_fullsized":"2024-04-26T23:37:13.890355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795722+0000","last_change":"2024-04-26T23:37:13.097286+0000","last_active":"2024-04-26T23:37:13.795722+0000","last_peered":"2024-04-26T23:37:13.795722+0000","last_clean":"2024-04-26T23:37:13.795722+0000","last_became_active":"2024-04-26T23:37:13.096512+0000","last_became_peered":"2024-04-26T23:37:13.096512+0000","last_unstale":"2024-04-26T23:37:13.795722+0000","last_undegraded":"2024-04-26T23:37:13.795722+0000","last_fullsized":"2024-04-26T23:37:13.795722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890492+0000","last_change":"2024-04-26T23:37:08.791257+0000","last_active":"2024-04-26T23:37:13.890492+0000","last_peered":"2024-04-26T23:37:13.890492+0000","last_clean":"2024-04-26T23:37:13.890492+0000","last_became_active":"2024-04-26T23:37:08.790966+0000","last_became_peered":"2024-04-26T23:37:08.790966+0000","last_unstale":"2024-04-26T23:37:13.890492+0000","last_undegraded":"2024-04-26T23:37:13.890492+0000","last_fullsized":"2024-04-26T23:37:13.890492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802430+0000","last_change":"2024-04-26T23:37:07.742144+0000","last_active":"2024-04-26T23:37:13.802430+0000","last_peered":"2024-04-26T23:37:13.802430+0000","last_clean":"2024-04-26T23:37:13.802430+0000","last_became_active":"2024-04-26T23:37:07.741812+0000","last_became_peered":"2024-04-26T23:37:07.741812+0000","last_unstale":"2024-04-26T23:37:13.802430+0000","last_undegraded":"2024-04-26T23:37:13.802430+0000","last_fullsized":"2024-04-26T23:37:13.802430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890498+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:37:13.890498+0000","last_peered":"2024-04-26T23:37:13.890498+0000","last_clean":"2024-04-26T23:37:13.890498+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:37:13.890498+0000","last_undegraded":"2024-04-26T23:37:13.890498+0000","last_fullsized":"2024-04-26T23:37:13.890498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892073+0000","last_change":"2024-04-26T23:37:13.094077+0000","last_active":"2024-04-26T23:37:13.892073+0000","last_peered":"2024-04-26T23:37:13.892073+0000","last_clean":"2024-04-26T23:37:13.892073+0000","last_became_active":"2024-04-26T23:37:13.093035+0000","last_became_peered":"2024-04-26T23:37:13.093035+0000","last_unstale":"2024-04-26T23:37:13.892073+0000","last_undegraded":"2024-04-26T23:37:13.892073+0000","last_fullsized":"2024-04-26T23:37:13.892073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892721+0000","last_change":"2024-04-26T23:37:08.787518+0000","last_active":"2024-04-26T23:37:13.892721+0000","last_peered":"2024-04-26T23:37:13.892721+0000","last_clean":"2024-04-26T23:37:13.892721+0000","last_became_active":"2024-04-26T23:37:08.787303+0000","last_became_peered":"2024-04-26T23:37:08.787303+0000","last_unstale":"2024-04-26T23:37:13.892721+0000","last_undegraded":"2024-04-26T23:37:13.892721+0000","last_fullsized":"2024-04-26T23:37:13.892721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891144+0000","last_change":"2024-04-26T23:37:07.746465+0000","last_active":"2024-04-26T23:37:13.891144+0000","last_peered":"2024-04-26T23:37:13.891144+0000","last_clean":"2024-04-26T23:37:13.891144+0000","last_became_active":"2024-04-26T23:37:07.746109+0000","last_became_peered":"2024-04-26T23:37:07.746109+0000","last_unstale":"2024-04-26T23:37:13.891144+0000","last_undegraded":"2024-04-26T23:37:13.891144+0000","last_fullsized":"2024-04-26T23:37:13.891144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890061+0000","last_change":"2024-04-26T23:37:13.098348+0000","last_active":"2024-04-26T23:37:13.890061+0000","last_peered":"2024-04-26T23:37:13.890061+0000","last_clean":"2024-04-26T23:37:13.890061+0000","last_became_active":"2024-04-26T23:37:13.092005+0000","last_became_peered":"2024-04-26T23:37:13.092005+0000","last_unstale":"2024-04-26T23:37:13.890061+0000","last_undegraded":"2024-04-26T23:37:13.890061+0000","last_fullsized":"2024-04-26T23:37:13.890061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802973+0000","last_change":"2024-04-26T23:37:08.788947+0000","last_active":"2024-04-26T23:37:13.802973+0000","last_peered":"2024-04-26T23:37:13.802973+0000","last_clean":"2024-04-26T23:37:13.802973+0000","last_became_active":"2024-04-26T23:37:08.785540+0000","last_became_peered":"2024-04-26T23:37:08.785540+0000","last_unstale":"2024-04-26T23:37:13.802973+0000","last_undegraded":"2024-04-26T23:37:13.802973+0000","last_fullsized":"2024-04-26T23:37:13.802973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889865+0000","last_change":"2024-04-26T23:37:07.745996+0000","last_active":"2024-04-26T23:37:13.889865+0000","last_peered":"2024-04-26T23:37:13.889865+0000","last_clean":"2024-04-26T23:37:13.889865+0000","last_became_active":"2024-04-26T23:37:07.745526+0000","last_became_peered":"2024-04-26T23:37:07.745526+0000","last_unstale":"2024-04-26T23:37:13.889865+0000","last_undegraded":"2024-04-26T23:37:13.889865+0000","last_fullsized":"2024-04-26T23:37:13.889865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795637+0000","last_change":"2024-04-26T23:37:12.818381+0000","last_active":"2024-04-26T23:37:13.795637+0000","last_peered":"2024-04-26T23:37:13.795637+0000","last_clean":"2024-04-26T23:37:13.795637+0000","last_became_active":"2024-04-26T23:37:12.816556+0000","last_became_peered":"2024-04-26T23:37:12.816556+0000","last_unstale":"2024-04-26T23:37:13.795637+0000","last_undegraded":"2024-04-26T23:37:13.795637+0000","last_fullsized":"2024-04-26T23:37:13.795637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893859+0000","last_change":"2024-04-26T23:37:08.788613+0000","last_active":"2024-04-26T23:37:13.893859+0000","last_peered":"2024-04-26T23:37:13.893859+0000","last_clean":"2024-04-26T23:37:13.893859+0000","last_became_active":"2024-04-26T23:37:08.788341+0000","last_became_peered":"2024-04-26T23:37:08.788341+0000","last_unstale":"2024-04-26T23:37:13.893859+0000","last_undegraded":"2024-04-26T23:37:13.893859+0000","last_fullsized":"2024-04-26T23:37:13.893859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:05.835929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891528+0000","last_change":"2024-04-26T23:37:07.732372+0000","last_active":"2024-04-26T23:37:13.891528+0000","last_peered":"2024-04-26T23:37:13.891528+0000","last_clean":"2024-04-26T23:37:13.891528+0000","last_became_active":"2024-04-26T23:37:07.732085+0000","last_became_peered":"2024-04-26T23:37:07.732085+0000","last_unstale":"2024-04-26T23:37:13.891528+0000","last_undegraded":"2024-04-26T23:37:13.891528+0000","last_fullsized":"2024-04-26T23:37:13.891528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798168+0000","last_change":"2024-04-26T23:37:13.093306+0000","last_active":"2024-04-26T23:37:13.798168+0000","last_peered":"2024-04-26T23:37:13.798168+0000","last_clean":"2024-04-26T23:37:13.798168+0000","last_became_active":"2024-04-26T23:37:13.089425+0000","last_became_peered":"2024-04-26T23:37:13.089425+0000","last_unstale":"2024-04-26T23:37:13.798168+0000","last_undegraded":"2024-04-26T23:37:13.798168+0000","last_fullsized":"2024-04-26T23:37:13.798168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893002+0000","last_change":"2024-04-26T23:37:08.788931+0000","last_active":"2024-04-26T23:37:13.893002+0000","last_peered":"2024-04-26T23:37:13.893002+0000","last_clean":"2024-04-26T23:37:13.893002+0000","last_became_active":"2024-04-26T23:37:08.788287+0000","last_became_peered":"2024-04-26T23:37:08.788287+0000","last_unstale":"2024-04-26T23:37:13.893002+0000","last_undegraded":"2024-04-26T23:37:13.893002+0000","last_fullsized":"2024-04-26T23:37:13.893002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799208+0000","last_change":"2024-04-26T23:37:07.729179+0000","last_active":"2024-04-26T23:37:13.799208+0000","last_peered":"2024-04-26T23:37:13.799208+0000","last_clean":"2024-04-26T23:37:13.799208+0000","last_became_active":"2024-04-26T23:37:07.728840+0000","last_became_peered":"2024-04-26T23:37:07.728840+0000","last_unstale":"2024-04-26T23:37:13.799208+0000","last_undegraded":"2024-04-26T23:37:13.799208+0000","last_fullsized":"2024-04-26T23:37:13.799208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:09.969446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798616+0000","last_change":"2024-04-26T23:37:12.833356+0000","last_active":"2024-04-26T23:37:13.798616+0000","last_peered":"2024-04-26T23:37:13.798616+0000","last_clean":"2024-04-26T23:37:13.798616+0000","last_became_active":"2024-04-26T23:37:12.830528+0000","last_became_peered":"2024-04-26T23:37:12.830528+0000","last_unstale":"2024-04-26T23:37:13.798616+0000","last_undegraded":"2024-04-26T23:37:13.798616+0000","last_fullsized":"2024-04-26T23:37:13.798616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.662558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891353+0000","last_change":"2024-04-26T23:37:08.782130+0000","last_active":"2024-04-26T23:37:13.891353+0000","last_peered":"2024-04-26T23:37:13.891353+0000","last_clean":"2024-04-26T23:37:13.891353+0000","last_became_active":"2024-04-26T23:37:08.781906+0000","last_became_peered":"2024-04-26T23:37:08.781906+0000","last_unstale":"2024-04-26T23:37:13.891353+0000","last_undegraded":"2024-04-26T23:37:13.891353+0000","last_fullsized":"2024-04-26T23:37:13.891353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.887760+0000","last_change":"2024-04-26T23:37:07.728318+0000","last_active":"2024-04-26T23:37:13.887760+0000","last_peered":"2024-04-26T23:37:13.887760+0000","last_clean":"2024-04-26T23:37:13.887760+0000","last_became_active":"2024-04-26T23:37:07.727941+0000","last_became_peered":"2024-04-26T23:37:07.727941+0000","last_unstale":"2024-04-26T23:37:13.887760+0000","last_undegraded":"2024-04-26T23:37:13.887760+0000","last_fullsized":"2024-04-26T23:37:13.887760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795484+0000","last_change":"2024-04-26T23:37:12.832185+0000","last_active":"2024-04-26T23:37:13.795484+0000","last_peered":"2024-04-26T23:37:13.795484+0000","last_clean":"2024-04-26T23:37:13.795484+0000","last_became_active":"2024-04-26T23:37:12.830851+0000","last_became_peered":"2024-04-26T23:37:12.830851+0000","last_unstale":"2024-04-26T23:37:13.795484+0000","last_undegraded":"2024-04-26T23:37:13.795484+0000","last_fullsized":"2024-04-26T23:37:13.795484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889876+0000","last_change":"2024-04-26T23:37:08.768079+0000","last_active":"2024-04-26T23:37:13.889876+0000","last_peered":"2024-04-26T23:37:13.889876+0000","last_clean":"2024-04-26T23:37:13.889876+0000","last_became_active":"2024-04-26T23:37:08.767338+0000","last_became_peered":"2024-04-26T23:37:08.767338+0000","last_unstale":"2024-04-26T23:37:13.889876+0000","last_undegraded":"2024-04-26T23:37:13.889876+0000","last_fullsized":"2024-04-26T23:37:13.889876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890125+0000","last_change":"2024-04-26T23:37:07.746339+0000","last_active":"2024-04-26T23:37:13.890125+0000","last_peered":"2024-04-26T23:37:13.890125+0000","last_clean":"2024-04-26T23:37:13.890125+0000","last_became_active":"2024-04-26T23:37:07.746075+0000","last_became_peered":"2024-04-26T23:37:07.746075+0000","last_unstale":"2024-04-26T23:37:13.890125+0000","last_undegraded":"2024-04-26T23:37:13.890125+0000","last_fullsized":"2024-04-26T23:37:13.890125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892748+0000","last_change":"2024-04-26T23:37:13.091284+0000","last_active":"2024-04-26T23:37:13.892748+0000","last_peered":"2024-04-26T23:37:13.892748+0000","last_clean":"2024-04-26T23:37:13.892748+0000","last_became_active":"2024-04-26T23:37:13.090274+0000","last_became_peered":"2024-04-26T23:37:13.090274+0000","last_unstale":"2024-04-26T23:37:13.892748+0000","last_undegraded":"2024-04-26T23:37:13.892748+0000","last_fullsized":"2024-04-26T23:37:13.892748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:53.078576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892237+0000","last_change":"2024-04-26T23:37:08.782024+0000","last_active":"2024-04-26T23:37:13.892237+0000","last_peered":"2024-04-26T23:37:13.892237+0000","last_clean":"2024-04-26T23:37:13.892237+0000","last_became_active":"2024-04-26T23:37:08.781730+0000","last_became_peered":"2024-04-26T23:37:08.781730+0000","last_unstale":"2024-04-26T23:37:13.892237+0000","last_undegraded":"2024-04-26T23:37:13.892237+0000","last_fullsized":"2024-04-26T23:37:13.892237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801858+0000","last_change":"2024-04-26T23:37:07.741413+0000","last_active":"2024-04-26T23:37:13.801858+0000","last_peered":"2024-04-26T23:37:13.801858+0000","last_clean":"2024-04-26T23:37:13.801858+0000","last_became_active":"2024-04-26T23:37:07.741088+0000","last_became_peered":"2024-04-26T23:37:07.741088+0000","last_unstale":"2024-04-26T23:37:13.801858+0000","last_undegraded":"2024-04-26T23:37:13.801858+0000","last_fullsized":"2024-04-26T23:37:13.801858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891737+0000","last_change":"2024-04-26T23:37:13.089930+0000","last_active":"2024-04-26T23:37:13.891737+0000","last_peered":"2024-04-26T23:37:13.891737+0000","last_clean":"2024-04-26T23:37:13.891737+0000","last_became_active":"2024-04-26T23:37:13.088797+0000","last_became_peered":"2024-04-26T23:37:13.088797+0000","last_unstale":"2024-04-26T23:37:13.891737+0000","last_undegraded":"2024-04-26T23:37:13.891737+0000","last_fullsized":"2024-04-26T23:37:13.891737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:24.129454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795384+0000","last_change":"2024-04-26T23:37:08.772085+0000","last_active":"2024-04-26T23:37:13.795384+0000","last_peered":"2024-04-26T23:37:13.795384+0000","last_clean":"2024-04-26T23:37:13.795384+0000","last_became_active":"2024-04-26T23:37:08.771732+0000","last_became_peered":"2024-04-26T23:37:08.771732+0000","last_unstale":"2024-04-26T23:37:13.795384+0000","last_undegraded":"2024-04-26T23:37:13.795384+0000","last_fullsized":"2024-04-26T23:37:13.795384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:01.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796199+0000","last_change":"2024-04-26T23:37:07.739747+0000","last_active":"2024-04-26T23:37:13.796199+0000","last_peered":"2024-04-26T23:37:13.796199+0000","last_clean":"2024-04-26T23:37:13.796199+0000","last_became_active":"2024-04-26T23:37:07.739361+0000","last_became_peered":"2024-04-26T23:37:07.739361+0000","last_unstale":"2024-04-26T23:37:13.796199+0000","last_undegraded":"2024-04-26T23:37:13.796199+0000","last_fullsized":"2024-04-26T23:37:13.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:55.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802956+0000","last_change":"2024-04-26T23:37:13.100935+0000","last_active":"2024-04-26T23:37:13.802956+0000","last_peered":"2024-04-26T23:37:13.802956+0000","last_clean":"2024-04-26T23:37:13.802956+0000","last_became_active":"2024-04-26T23:37:13.098932+0000","last_became_peered":"2024-04-26T23:37:13.098932+0000","last_unstale":"2024-04-26T23:37:13.802956+0000","last_undegraded":"2024-04-26T23:37:13.802956+0000","last_fullsized":"2024-04-26T23:37:13.802956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892457+0000","last_change":"2024-04-26T23:37:08.780897+0000","last_active":"2024-04-26T23:37:13.892457+0000","last_peered":"2024-04-26T23:37:13.892457+0000","last_clean":"2024-04-26T23:37:13.892457+0000","last_became_active":"2024-04-26T23:37:08.780573+0000","last_became_peered":"2024-04-26T23:37:08.780573+0000","last_unstale":"2024-04-26T23:37:13.892457+0000","last_undegraded":"2024-04-26T23:37:13.892457+0000","last_fullsized":"2024-04-26T23:37:13.892457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:27.933683+0000","last_change":"2024-04-26T23:37:07.743107+0000","last_active":"2024-04-26T23:37:27.933683+0000","last_peered":"2024-04-26T23:37:27.933683+0000","last_clean":"2024-04-26T23:37:27.933683+0000","last_became_active":"2024-04-26T23:37:07.742729+0000","last_became_peered":"2024-04-26T23:37:07.742729+0000","last_unstale":"2024-04-26T23:37:27.933683+0000","last_undegraded":"2024-04-26T23:37:27.933683+0000","last_fullsized":"2024-04-26T23:37:27.933683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888705+0000","last_change":"2024-04-26T23:37:12.824914+0000","last_active":"2024-04-26T23:37:13.888705+0000","last_peered":"2024-04-26T23:37:13.888705+0000","last_clean":"2024-04-26T23:37:13.888705+0000","last_became_active":"2024-04-26T23:37:12.824383+0000","last_became_peered":"2024-04-26T23:37:12.824383+0000","last_unstale":"2024-04-26T23:37:13.888705+0000","last_undegraded":"2024-04-26T23:37:13.888705+0000","last_fullsized":"2024-04-26T23:37:13.888705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801058+0000","last_change":"2024-04-26T23:37:08.785202+0000","last_active":"2024-04-26T23:37:13.801058+0000","last_peered":"2024-04-26T23:37:13.801058+0000","last_clean":"2024-04-26T23:37:13.801058+0000","last_became_active":"2024-04-26T23:37:08.779537+0000","last_became_peered":"2024-04-26T23:37:08.779537+0000","last_unstale":"2024-04-26T23:37:13.801058+0000","last_undegraded":"2024-04-26T23:37:13.801058+0000","last_fullsized":"2024-04-26T23:37:13.801058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.971268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889719+0000","last_change":"2024-04-26T23:37:07.741707+0000","last_active":"2024-04-26T23:37:13.889719+0000","last_peered":"2024-04-26T23:37:13.889719+0000","last_clean":"2024-04-26T23:37:13.889719+0000","last_became_active":"2024-04-26T23:37:07.740731+0000","last_became_peered":"2024-04-26T23:37:07.740731+0000","last_unstale":"2024-04-26T23:37:13.889719+0000","last_undegraded":"2024-04-26T23:37:13.889719+0000","last_fullsized":"2024-04-26T23:37:13.889719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798812+0000","last_change":"2024-04-26T23:37:12.818753+0000","last_active":"2024-04-26T23:37:13.798812+0000","last_peered":"2024-04-26T23:37:13.798812+0000","last_clean":"2024-04-26T23:37:13.798812+0000","last_became_active":"2024-04-26T23:37:12.817885+0000","last_became_peered":"2024-04-26T23:37:12.817885+0000","last_unstale":"2024-04-26T23:37:13.798812+0000","last_undegraded":"2024-04-26T23:37:13.798812+0000","last_fullsized":"2024-04-26T23:37:13.798812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:06.283131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795457+0000","last_change":"2024-04-26T23:37:08.796476+0000","last_active":"2024-04-26T23:37:13.795457+0000","last_peered":"2024-04-26T23:37:13.795457+0000","last_clean":"2024-04-26T23:37:13.795457+0000","last_became_active":"2024-04-26T23:37:08.795730+0000","last_became_peered":"2024-04-26T23:37:08.795730+0000","last_unstale":"2024-04-26T23:37:13.795457+0000","last_undegraded":"2024-04-26T23:37:13.795457+0000","last_fullsized":"2024-04-26T23:37:13.795457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802755+0000","last_change":"2024-04-26T23:37:07.746232+0000","last_active":"2024-04-26T23:37:13.802755+0000","last_peered":"2024-04-26T23:37:13.802755+0000","last_clean":"2024-04-26T23:37:13.802755+0000","last_became_active":"2024-04-26T23:37:07.745874+0000","last_became_peered":"2024-04-26T23:37:07.745874+0000","last_unstale":"2024-04-26T23:37:13.802755+0000","last_undegraded":"2024-04-26T23:37:13.802755+0000","last_fullsized":"2024-04-26T23:37:13.802755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891356+0000","last_change":"2024-04-26T23:37:12.834267+0000","last_active":"2024-04-26T23:37:13.891356+0000","last_peered":"2024-04-26T23:37:13.891356+0000","last_clean":"2024-04-26T23:37:13.891356+0000","last_became_active":"2024-04-26T23:37:12.829458+0000","last_became_peered":"2024-04-26T23:37:12.829458+0000","last_unstale":"2024-04-26T23:37:13.891356+0000","last_undegraded":"2024-04-26T23:37:13.891356+0000","last_fullsized":"2024-04-26T23:37:13.891356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:05.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802727+0000","last_change":"2024-04-26T23:37:08.788911+0000","last_active":"2024-04-26T23:37:13.802727+0000","last_peered":"2024-04-26T23:37:13.802727+0000","last_clean":"2024-04-26T23:37:13.802727+0000","last_became_active":"2024-04-26T23:37:08.785658+0000","last_became_peered":"2024-04-26T23:37:08.785658+0000","last_unstale":"2024-04-26T23:37:13.802727+0000","last_undegraded":"2024-04-26T23:37:13.802727+0000","last_fullsized":"2024-04-26T23:37:13.802727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892184+0000","last_change":"2024-04-26T23:37:07.750014+0000","last_active":"2024-04-26T23:37:13.892184+0000","last_peered":"2024-04-26T23:37:13.892184+0000","last_clean":"2024-04-26T23:37:13.892184+0000","last_became_active":"2024-04-26T23:37:07.748991+0000","last_became_peered":"2024-04-26T23:37:07.748991+0000","last_unstale":"2024-04-26T23:37:13.892184+0000","last_undegraded":"2024-04-26T23:37:13.892184+0000","last_fullsized":"2024-04-26T23:37:13.892184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890500+0000","last_change":"2024-04-26T23:37:13.096386+0000","last_active":"2024-04-26T23:37:13.890500+0000","last_peered":"2024-04-26T23:37:13.890500+0000","last_clean":"2024-04-26T23:37:13.890500+0000","last_became_active":"2024-04-26T23:37:13.095901+0000","last_became_peered":"2024-04-26T23:37:13.095901+0000","last_unstale":"2024-04-26T23:37:13.890500+0000","last_undegraded":"2024-04-26T23:37:13.890500+0000","last_fullsized":"2024-04-26T23:37:13.890500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:22.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795244+0000","last_change":"2024-04-26T23:37:13.100870+0000","last_active":"2024-04-26T23:37:13.795244+0000","last_peered":"2024-04-26T23:37:13.795244+0000","last_clean":"2024-04-26T23:37:13.795244+0000","last_became_active":"2024-04-26T23:37:13.094872+0000","last_became_peered":"2024-04-26T23:37:13.094872+0000","last_unstale":"2024-04-26T23:37:13.795244+0000","last_undegraded":"2024-04-26T23:37:13.795244+0000","last_fullsized":"2024-04-26T23:37:13.795244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888033+0000","last_change":"2024-04-26T23:37:08.770189+0000","last_active":"2024-04-26T23:37:13.888033+0000","last_peered":"2024-04-26T23:37:13.888033+0000","last_clean":"2024-04-26T23:37:13.888033+0000","last_became_active":"2024-04-26T23:37:08.769911+0000","last_became_peered":"2024-04-26T23:37:08.769911+0000","last_unstale":"2024-04-26T23:37:13.888033+0000","last_undegraded":"2024-04-26T23:37:13.888033+0000","last_fullsized":"2024-04-26T23:37:13.888033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800060+0000","last_change":"2024-04-26T23:37:07.745063+0000","last_active":"2024-04-26T23:37:13.800060+0000","last_peered":"2024-04-26T23:37:13.800060+0000","last_clean":"2024-04-26T23:37:13.800060+0000","last_became_active":"2024-04-26T23:37:07.744383+0000","last_became_peered":"2024-04-26T23:37:07.744383+0000","last_unstale":"2024-04-26T23:37:13.800060+0000","last_undegraded":"2024-04-26T23:37:13.800060+0000","last_fullsized":"2024-04-26T23:37:13.800060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890337+0000","last_change":"2024-04-26T23:37:12.824766+0000","last_active":"2024-04-26T23:37:13.890337+0000","last_peered":"2024-04-26T23:37:13.890337+0000","last_clean":"2024-04-26T23:37:13.890337+0000","last_became_active":"2024-04-26T23:37:12.824314+0000","last_became_peered":"2024-04-26T23:37:12.824314+0000","last_unstale":"2024-04-26T23:37:13.890337+0000","last_undegraded":"2024-04-26T23:37:13.890337+0000","last_fullsized":"2024-04-26T23:37:13.890337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889801+0000","last_change":"2024-04-26T23:37:08.769161+0000","last_active":"2024-04-26T23:37:13.889801+0000","last_peered":"2024-04-26T23:37:13.889801+0000","last_clean":"2024-04-26T23:37:13.889801+0000","last_became_active":"2024-04-26T23:37:08.767929+0000","last_became_peered":"2024-04-26T23:37:08.767929+0000","last_unstale":"2024-04-26T23:37:13.889801+0000","last_undegraded":"2024-04-26T23:37:13.889801+0000","last_fullsized":"2024-04-26T23:37:13.889801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890886+0000","last_change":"2024-04-26T23:37:07.739492+0000","last_active":"2024-04-26T23:37:13.890886+0000","last_peered":"2024-04-26T23:37:13.890886+0000","last_clean":"2024-04-26T23:37:13.890886+0000","last_became_active":"2024-04-26T23:37:07.739185+0000","last_became_peered":"2024-04-26T23:37:07.739185+0000","last_unstale":"2024-04-26T23:37:13.890886+0000","last_undegraded":"2024-04-26T23:37:13.890886+0000","last_fullsized":"2024-04-26T23:37:13.890886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890373+0000","last_change":"2024-04-26T23:37:07.750874+0000","last_active":"2024-04-26T23:37:13.890373+0000","last_peered":"2024-04-26T23:37:13.890373+0000","last_clean":"2024-04-26T23:37:13.890373+0000","last_became_active":"2024-04-26T23:37:07.748284+0000","last_became_peered":"2024-04-26T23:37:07.748284+0000","last_unstale":"2024-04-26T23:37:13.890373+0000","last_undegraded":"2024-04-26T23:37:13.890373+0000","last_fullsized":"2024-04-26T23:37:13.890373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796082+0000","last_change":"2024-04-26T23:37:08.753110+0000","last_active":"2024-04-26T23:37:13.796082+0000","last_peered":"2024-04-26T23:37:13.796082+0000","last_clean":"2024-04-26T23:37:13.796082+0000","last_became_active":"2024-04-26T23:37:08.752712+0000","last_became_peered":"2024-04-26T23:37:08.752712+0000","last_unstale":"2024-04-26T23:37:13.796082+0000","last_undegraded":"2024-04-26T23:37:13.796082+0000","last_fullsized":"2024-04-26T23:37:13.796082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889412+0000","last_change":"2024-04-26T23:37:12.829960+0000","last_active":"2024-04-26T23:37:13.889412+0000","last_peered":"2024-04-26T23:37:13.889412+0000","last_clean":"2024-04-26T23:37:13.889412+0000","last_became_active":"2024-04-26T23:37:12.829520+0000","last_became_peered":"2024-04-26T23:37:12.829520+0000","last_unstale":"2024-04-26T23:37:13.889412+0000","last_undegraded":"2024-04-26T23:37:13.889412+0000","last_fullsized":"2024-04-26T23:37:13.889412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:05.418963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891289+0000","last_change":"2024-04-26T23:37:07.747637+0000","last_active":"2024-04-26T23:37:13.891289+0000","last_peered":"2024-04-26T23:37:13.891289+0000","last_clean":"2024-04-26T23:37:13.891289+0000","last_became_active":"2024-04-26T23:37:07.747308+0000","last_became_peered":"2024-04-26T23:37:07.747308+0000","last_unstale":"2024-04-26T23:37:13.891289+0000","last_undegraded":"2024-04-26T23:37:13.891289+0000","last_fullsized":"2024-04-26T23:37:13.891289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798586+0000","last_change":"2024-04-26T23:37:08.763567+0000","last_active":"2024-04-26T23:37:13.798586+0000","last_peered":"2024-04-26T23:37:13.798586+0000","last_clean":"2024-04-26T23:37:13.798586+0000","last_became_active":"2024-04-26T23:37:08.762771+0000","last_became_peered":"2024-04-26T23:37:08.762771+0000","last_unstale":"2024-04-26T23:37:13.798586+0000","last_undegraded":"2024-04-26T23:37:13.798586+0000","last_fullsized":"2024-04-26T23:37:13.798586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:19.641900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892195+0000","last_change":"2024-04-26T23:37:12.832076+0000","last_active":"2024-04-26T23:37:13.892195+0000","last_peered":"2024-04-26T23:37:13.892195+0000","last_clean":"2024-04-26T23:37:13.892195+0000","last_became_active":"2024-04-26T23:37:12.830641+0000","last_became_peered":"2024-04-26T23:37:12.830641+0000","last_unstale":"2024-04-26T23:37:13.892195+0000","last_undegraded":"2024-04-26T23:37:13.892195+0000","last_fullsized":"2024-04-26T23:37:13.892195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:21.722510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889621+0000","last_change":"2024-04-26T23:37:08.768406+0000","last_active":"2024-04-26T23:37:13.889621+0000","last_peered":"2024-04-26T23:37:13.889621+0000","last_clean":"2024-04-26T23:37:13.889621+0000","last_became_active":"2024-04-26T23:37:08.767730+0000","last_became_peered":"2024-04-26T23:37:08.767730+0000","last_unstale":"2024-04-26T23:37:13.889621+0000","last_undegraded":"2024-04-26T23:37:13.889621+0000","last_fullsized":"2024-04-26T23:37:13.889621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888136+0000","last_change":"2024-04-26T23:37:08.780725+0000","last_active":"2024-04-26T23:37:13.888136+0000","last_peered":"2024-04-26T23:37:13.888136+0000","last_clean":"2024-04-26T23:37:13.888136+0000","last_became_active":"2024-04-26T23:37:08.780362+0000","last_became_peered":"2024-04-26T23:37:08.780362+0000","last_unstale":"2024-04-26T23:37:13.888136+0000","last_undegraded":"2024-04-26T23:37:13.888136+0000","last_fullsized":"2024-04-26T23:37:13.888136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801811+0000","last_change":"2024-04-26T23:37:08.789772+0000","last_active":"2024-04-26T23:37:13.801811+0000","last_peered":"2024-04-26T23:37:13.801811+0000","last_clean":"2024-04-26T23:37:13.801811+0000","last_became_active":"2024-04-26T23:37:08.785818+0000","last_became_peered":"2024-04-26T23:37:08.785818+0000","last_unstale":"2024-04-26T23:37:13.801811+0000","last_undegraded":"2024-04-26T23:37:13.801811+0000","last_fullsized":"2024-04-26T23:37:13.801811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795710+0000","last_change":"2024-04-26T23:37:08.768747+0000","last_active":"2024-04-26T23:37:13.795710+0000","last_peered":"2024-04-26T23:37:13.795710+0000","last_clean":"2024-04-26T23:37:13.795710+0000","last_became_active":"2024-04-26T23:37:08.768413+0000","last_became_peered":"2024-04-26T23:37:08.768413+0000","last_unstale":"2024-04-26T23:37:13.795710+0000","last_undegraded":"2024-04-26T23:37:13.795710+0000","last_fullsized":"2024-04-26T23:37:13.795710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891826+0000","last_change":"2024-04-26T23:37:08.788540+0000","last_active":"2024-04-26T23:37:13.891826+0000","last_peered":"2024-04-26T23:37:13.891826+0000","last_clean":"2024-04-26T23:37:13.891826+0000","last_became_active":"2024-04-26T23:37:08.788200+0000","last_became_peered":"2024-04-26T23:37:08.788200+0000","last_unstale":"2024-04-26T23:37:13.891826+0000","last_undegraded":"2024-04-26T23:37:13.891826+0000","last_fullsized":"2024-04-26T23:37:13.891826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891950+0000","last_change":"2024-04-26T23:37:08.777509+0000","last_active":"2024-04-26T23:37:13.891950+0000","last_peered":"2024-04-26T23:37:13.891950+0000","last_clean":"2024-04-26T23:37:13.891950+0000","last_became_active":"2024-04-26T23:37:08.777061+0000","last_became_peered":"2024-04-26T23:37:08.777061+0000","last_unstale":"2024-04-26T23:37:13.891950+0000","last_undegraded":"2024-04-26T23:37:13.891950+0000","last_fullsized":"2024-04-26T23:37:13.891950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891626+0000","last_change":"2024-04-26T23:37:08.774973+0000","last_active":"2024-04-26T23:37:13.891626+0000","last_peered":"2024-04-26T23:37:13.891626+0000","last_clean":"2024-04-26T23:37:13.891626+0000","last_became_active":"2024-04-26T23:37:08.774707+0000","last_became_peered":"2024-04-26T23:37:08.774707+0000","last_unstale":"2024-04-26T23:37:13.891626+0000","last_undegraded":"2024-04-26T23:37:13.891626+0000","last_fullsized":"2024-04-26T23:37:13.891626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892661+0000","last_change":"2024-04-26T23:37:08.781269+0000","last_active":"2024-04-26T23:37:13.892661+0000","last_peered":"2024-04-26T23:37:13.892661+0000","last_clean":"2024-04-26T23:37:13.892661+0000","last_became_active":"2024-04-26T23:37:08.780337+0000","last_became_peered":"2024-04-26T23:37:08.780337+0000","last_unstale":"2024-04-26T23:37:13.892661+0000","last_undegraded":"2024-04-26T23:37:13.892661+0000","last_fullsized":"2024-04-26T23:37:13.892661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799802+0000","last_change":"2024-04-26T23:37:08.771877+0000","last_active":"2024-04-26T23:37:13.799802+0000","last_peered":"2024-04-26T23:37:13.799802+0000","last_clean":"2024-04-26T23:37:13.799802+0000","last_became_active":"2024-04-26T23:37:08.770810+0000","last_became_peered":"2024-04-26T23:37:08.770810+0000","last_unstale":"2024-04-26T23:37:13.799802+0000","last_undegraded":"2024-04-26T23:37:13.799802+0000","last_fullsized":"2024-04-26T23:37:13.799802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890670+0000","last_change":"2024-04-26T23:37:08.780735+0000","last_active":"2024-04-26T23:37:13.890670+0000","last_peered":"2024-04-26T23:37:13.890670+0000","last_clean":"2024-04-26T23:37:13.890670+0000","last_became_active":"2024-04-26T23:37:08.780384+0000","last_became_peered":"2024-04-26T23:37:08.780384+0000","last_unstale":"2024-04-26T23:37:13.890670+0000","last_undegraded":"2024-04-26T23:37:13.890670+0000","last_fullsized":"2024-04-26T23:37:13.890670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801555+0000","last_change":"2024-04-26T23:37:08.790073+0000","last_active":"2024-04-26T23:37:13.801555+0000","last_peered":"2024-04-26T23:37:13.801555+0000","last_clean":"2024-04-26T23:37:13.801555+0000","last_became_active":"2024-04-26T23:37:08.789431+0000","last_became_peered":"2024-04-26T23:37:08.789431+0000","last_unstale":"2024-04-26T23:37:13.801555+0000","last_undegraded":"2024-04-26T23:37:13.801555+0000","last_fullsized":"2024-04-26T23:37:13.801555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795276+0000","last_change":"2024-04-26T23:37:08.756179+0000","last_active":"2024-04-26T23:37:13.795276+0000","last_peered":"2024-04-26T23:37:13.795276+0000","last_clean":"2024-04-26T23:37:13.795276+0000","last_became_active":"2024-04-26T23:37:08.755805+0000","last_became_peered":"2024-04-26T23:37:08.755805+0000","last_unstale":"2024-04-26T23:37:13.795276+0000","last_undegraded":"2024-04-26T23:37:13.795276+0000","last_fullsized":"2024-04-26T23:37:13.795276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891285+0000","last_change":"2024-04-26T23:37:08.768542+0000","last_active":"2024-04-26T23:37:13.891285+0000","last_peered":"2024-04-26T23:37:13.891285+0000","last_clean":"2024-04-26T23:37:13.891285+0000","last_became_active":"2024-04-26T23:37:08.765841+0000","last_became_peered":"2024-04-26T23:37:08.765841+0000","last_unstale":"2024-04-26T23:37:13.891285+0000","last_undegraded":"2024-04-26T23:37:13.891285+0000","last_fullsized":"2024-04-26T23:37:13.891285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893105+0000","last_change":"2024-04-26T23:37:08.775312+0000","last_active":"2024-04-26T23:37:13.893105+0000","last_peered":"2024-04-26T23:37:13.893105+0000","last_clean":"2024-04-26T23:37:13.893105+0000","last_became_active":"2024-04-26T23:37:08.774944+0000","last_became_peered":"2024-04-26T23:37:08.774944+0000","last_unstale":"2024-04-26T23:37:13.893105+0000","last_undegraded":"2024-04-26T23:37:13.893105+0000","last_fullsized":"2024-04-26T23:37:13.893105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891448+0000","last_change":"2024-04-26T23:37:08.772525+0000","last_active":"2024-04-26T23:37:13.891448+0000","last_peered":"2024-04-26T23:37:13.891448+0000","last_clean":"2024-04-26T23:37:13.891448+0000","last_became_active":"2024-04-26T23:37:08.772177+0000","last_became_peered":"2024-04-26T23:37:08.772177+0000","last_unstale":"2024-04-26T23:37:13.891448+0000","last_undegraded":"2024-04-26T23:37:13.891448+0000","last_fullsized":"2024-04-26T23:37:13.891448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:55.136157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803582+0000","last_change":"2024-04-26T23:37:08.790236+0000","last_active":"2024-04-26T23:37:13.803582+0000","last_peered":"2024-04-26T23:37:13.803582+0000","last_clean":"2024-04-26T23:37:13.803582+0000","last_became_active":"2024-04-26T23:37:08.789885+0000","last_became_peered":"2024-04-26T23:37:08.789885+0000","last_unstale":"2024-04-26T23:37:13.803582+0000","last_undegraded":"2024-04-26T23:37:13.803582+0000","last_fullsized":"2024-04-26T23:37:13.803582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799434+0000","last_change":"2024-04-26T23:37:08.780732+0000","last_active":"2024-04-26T23:37:13.799434+0000","last_peered":"2024-04-26T23:37:13.799434+0000","last_clean":"2024-04-26T23:37:13.799434+0000","last_became_active":"2024-04-26T23:37:08.779046+0000","last_became_peered":"2024-04-26T23:37:08.779046+0000","last_unstale":"2024-04-26T23:37:13.799434+0000","last_undegraded":"2024-04-26T23:37:13.799434+0000","last_fullsized":"2024-04-26T23:37:13.799434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888183+0000","last_change":"2024-04-26T23:37:08.751809+0000","last_active":"2024-04-26T23:37:13.888183+0000","last_peered":"2024-04-26T23:37:13.888183+0000","last_clean":"2024-04-26T23:37:13.888183+0000","last_became_active":"2024-04-26T23:37:08.750955+0000","last_became_peered":"2024-04-26T23:37:08.750955+0000","last_unstale":"2024-04-26T23:37:13.888183+0000","last_undegraded":"2024-04-26T23:37:13.888183+0000","last_fullsized":"2024-04-26T23:37:13.888183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795084+0000","last_change":"2024-04-26T23:37:08.762870+0000","last_active":"2024-04-26T23:37:13.795084+0000","last_peered":"2024-04-26T23:37:13.795084+0000","last_clean":"2024-04-26T23:37:13.795084+0000","last_became_active":"2024-04-26T23:37:08.762554+0000","last_became_peered":"2024-04-26T23:37:08.762554+0000","last_unstale":"2024-04-26T23:37:13.795084+0000","last_undegraded":"2024-04-26T23:37:13.795084+0000","last_fullsized":"2024-04-26T23:37:13.795084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:05.526812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891059+0000","last_change":"2024-04-26T23:37:08.774910+0000","last_active":"2024-04-26T23:37:13.891059+0000","last_peered":"2024-04-26T23:37:13.891059+0000","last_clean":"2024-04-26T23:37:13.891059+0000","last_became_active":"2024-04-26T23:37:08.774587+0000","last_became_peered":"2024-04-26T23:37:08.774587+0000","last_unstale":"2024-04-26T23:37:13.891059+0000","last_undegraded":"2024-04-26T23:37:13.891059+0000","last_fullsized":"2024-04-26T23:37:13.891059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800959+0000","last_change":"2024-04-26T23:37:08.778389+0000","last_active":"2024-04-26T23:37:13.800959+0000","last_peered":"2024-04-26T23:37:13.800959+0000","last_clean":"2024-04-26T23:37:13.800959+0000","last_became_active":"2024-04-26T23:37:08.778116+0000","last_became_peered":"2024-04-26T23:37:08.778116+0000","last_unstale":"2024-04-26T23:37:13.800959+0000","last_undegraded":"2024-04-26T23:37:13.800959+0000","last_fullsized":"2024-04-26T23:37:13.800959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892362+0000","last_change":"2024-04-26T23:37:08.781929+0000","last_active":"2024-04-26T23:37:13.892362+0000","last_peered":"2024-04-26T23:37:13.892362+0000","last_clean":"2024-04-26T23:37:13.892362+0000","last_became_active":"2024-04-26T23:37:08.780724+0000","last_became_peered":"2024-04-26T23:37:08.780724+0000","last_unstale":"2024-04-26T23:37:13.892362+0000","last_undegraded":"2024-04-26T23:37:13.892362+0000","last_fullsized":"2024-04-26T23:37:13.892362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891324+0000","last_change":"2024-04-26T23:37:08.792579+0000","last_active":"2024-04-26T23:37:13.891324+0000","last_peered":"2024-04-26T23:37:13.891324+0000","last_clean":"2024-04-26T23:37:13.891324+0000","last_became_active":"2024-04-26T23:37:08.792171+0000","last_became_peered":"2024-04-26T23:37:08.792171+0000","last_unstale":"2024-04-26T23:37:13.891324+0000","last_undegraded":"2024-04-26T23:37:13.891324+0000","last_fullsized":"2024-04-26T23:37:13.891324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893856+0000","last_change":"2024-04-26T23:37:08.795179+0000","last_active":"2024-04-26T23:37:13.893856+0000","last_peered":"2024-04-26T23:37:13.893856+0000","last_clean":"2024-04-26T23:37:13.893856+0000","last_became_active":"2024-04-26T23:37:08.794890+0000","last_became_peered":"2024-04-26T23:37:08.794890+0000","last_unstale":"2024-04-26T23:37:13.893856+0000","last_undegraded":"2024-04-26T23:37:13.893856+0000","last_fullsized":"2024-04-26T23:37:13.893856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889647+0000","last_change":"2024-04-26T23:37:08.786739+0000","last_active":"2024-04-26T23:37:13.889647+0000","last_peered":"2024-04-26T23:37:13.889647+0000","last_clean":"2024-04-26T23:37:13.889647+0000","last_became_active":"2024-04-26T23:37:08.786360+0000","last_became_peered":"2024-04-26T23:37:08.786360+0000","last_unstale":"2024-04-26T23:37:13.889647+0000","last_undegraded":"2024-04-26T23:37:13.889647+0000","last_fullsized":"2024-04-26T23:37:13.889647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891212+0000","last_change":"2024-04-26T23:37:08.775748+0000","last_active":"2024-04-26T23:37:13.891212+0000","last_peered":"2024-04-26T23:37:13.891212+0000","last_clean":"2024-04-26T23:37:13.891212+0000","last_became_active":"2024-04-26T23:37:08.775440+0000","last_became_peered":"2024-04-26T23:37:08.775440+0000","last_unstale":"2024-04-26T23:37:13.891212+0000","last_undegraded":"2024-04-26T23:37:13.891212+0000","last_fullsized":"2024-04-26T23:37:13.891212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801324+0000","last_change":"2024-04-26T23:37:08.791665+0000","last_active":"2024-04-26T23:37:13.801324+0000","last_peered":"2024-04-26T23:37:13.801324+0000","last_clean":"2024-04-26T23:37:13.801324+0000","last_became_active":"2024-04-26T23:37:08.790305+0000","last_became_peered":"2024-04-26T23:37:08.790305+0000","last_unstale":"2024-04-26T23:37:13.801324+0000","last_undegraded":"2024-04-26T23:37:13.801324+0000","last_fullsized":"2024-04-26T23:37:13.801324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891978+0000","last_change":"2024-04-26T23:37:08.776000+0000","last_active":"2024-04-26T23:37:13.891978+0000","last_peered":"2024-04-26T23:37:13.891978+0000","last_clean":"2024-04-26T23:37:13.891978+0000","last_became_active":"2024-04-26T23:37:08.773615+0000","last_became_peered":"2024-04-26T23:37:08.773615+0000","last_unstale":"2024-04-26T23:37:13.891978+0000","last_undegraded":"2024-04-26T23:37:13.891978+0000","last_fullsized":"2024-04-26T23:37:13.891978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802456+0000","last_change":"2024-04-26T23:37:08.789506+0000","last_active":"2024-04-26T23:37:13.802456+0000","last_peered":"2024-04-26T23:37:13.802456+0000","last_clean":"2024-04-26T23:37:13.802456+0000","last_became_active":"2024-04-26T23:37:08.789180+0000","last_became_peered":"2024-04-26T23:37:08.789180+0000","last_unstale":"2024-04-26T23:37:13.802456+0000","last_undegraded":"2024-04-26T23:37:13.802456+0000","last_fullsized":"2024-04-26T23:37:13.802456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794898+0000","last_change":"2024-04-26T23:37:08.766757+0000","last_active":"2024-04-26T23:37:13.794898+0000","last_peered":"2024-04-26T23:37:13.794898+0000","last_clean":"2024-04-26T23:37:13.794898+0000","last_became_active":"2024-04-26T23:37:08.761659+0000","last_became_peered":"2024-04-26T23:37:08.761659+0000","last_unstale":"2024-04-26T23:37:13.794898+0000","last_undegraded":"2024-04-26T23:37:13.794898+0000","last_fullsized":"2024-04-26T23:37:13.794898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890719+0000","last_change":"2024-04-26T23:37:08.761968+0000","last_active":"2024-04-26T23:37:13.890719+0000","last_peered":"2024-04-26T23:37:13.890719+0000","last_clean":"2024-04-26T23:37:13.890719+0000","last_became_active":"2024-04-26T23:37:08.761546+0000","last_became_peered":"2024-04-26T23:37:08.761546+0000","last_unstale":"2024-04-26T23:37:13.890719+0000","last_undegraded":"2024-04-26T23:37:13.890719+0000","last_fullsized":"2024-04-26T23:37:13.890719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893557+0000","last_change":"2024-04-26T23:37:08.775290+0000","last_active":"2024-04-26T23:37:13.893557+0000","last_peered":"2024-04-26T23:37:13.893557+0000","last_clean":"2024-04-26T23:37:13.893557+0000","last_became_active":"2024-04-26T23:37:08.774646+0000","last_became_peered":"2024-04-26T23:37:08.774646+0000","last_unstale":"2024-04-26T23:37:13.893557+0000","last_undegraded":"2024-04-26T23:37:13.893557+0000","last_fullsized":"2024-04-26T23:37:13.893557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:37:38.984 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:37:38.984 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T23:37:38.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T23:37:38.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T23:37:38.985 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T23:37:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.211+0000 7fef47400640 1 Processor -- start 2024-04-26T23:37:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.211+0000 7fef47400640 1 -- start start 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef47400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef4006e510 0x7fef4006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef47400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4006ee30 con 0x7fef400732c0 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef47400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef40161940 con 0x7fef40160bd0 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef47400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fef40071f80 con 0x7fef4006e510 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef44a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef4006e510 0x7fef4006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef44a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef4006e510 0x7fef4006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37400/0 (socket says 172.21.15.46:37400) 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef44a00640 1 -- 172.21.15.46:0/644812621 learned_addr learned my addr 172.21.15.46:0/644812621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.213+0000 7fef45400640 1 --1- 172.21.15.46:0/644812621 >> v1:172.21.15.46:6789/0 conn(0x7fef400732c0 0x7fef40073710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40122/0 (socket says 172.21.15.46:40122) 2024-04-26T23:37:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.214+0000 7fef44a00640 1 -- 172.21.15.46:0/644812621 >> v1:172.21.15.142:6789/0 conn(0x7fef40160bd0 legacy=0x7fef40161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.214+0000 7fef44a00640 1 -- 172.21.15.46:0/644812621 >> v1:172.21.15.46:6789/0 conn(0x7fef400732c0 legacy=0x7fef40073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.214+0000 7fef44a00640 1 -- 172.21.15.46:0/644812621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef40072260 con 0x7fef4006e510 2024-04-26T23:37:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.215+0000 7fef44a00640 1 --2- 172.21.15.46:0/644812621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef4006e510 0x7fef4006e8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fef340098a0 tx=0x7fef3402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a81f056478c29c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.215+0000 7fef3f400640 1 -- 172.21.15.46:0/644812621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34037070 con 0x7fef4006e510 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.215+0000 7fef3f400640 1 -- 172.21.15.46:0/644812621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef34032440 con 0x7fef4006e510 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.215+0000 7fef3f400640 1 -- 172.21.15.46:0/644812621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34032810 con 0x7fef4006e510 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 -- 172.21.15.46:0/644812621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef4006e510 msgr2=0x7fef4006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 --2- 172.21.15.46:0/644812621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef4006e510 0x7fef4006e8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fef340098a0 tx=0x7fef3402a2e0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 -- 172.21.15.46:0/644812621 shutdown_connections 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 --2- 172.21.15.46:0/644812621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef4006e510 0x7fef4006e8f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 -- 172.21.15.46:0/644812621 >> 172.21.15.46:0/644812621 conn(0x7fef4006c530 msgr2=0x7fef4006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 -- 172.21.15.46:0/644812621 shutdown_connections 2024-04-26T23:37:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.216+0000 7fef47400640 1 -- 172.21.15.46:0/644812621 wait complete. 2024-04-26T23:37:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.218+0000 7fef47400640 1 Processor -- start 2024-04-26T23:37:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.218+0000 7fef47400640 1 -- start start 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4006e510 0x7fef401566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 0x7fef4013c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef40072d40 con 0x7fef40160bd0 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef40072820 con 0x7fef400732c0 2024-04-26T23:37:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef47400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fef40072ec0 con 0x7fef4006e510 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef45400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef45400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37404/0 (socket says 172.21.15.46:37404) 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef45400640 1 -- 172.21.15.46:0/3104576203 learned_addr learned my addr 172.21.15.46:0/3104576203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef3fe00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 0x7fef4013c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.219+0000 7fef44a00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4006e510 0x7fef401566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 -- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4006e510 msgr2=0x7fef401566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4006e510 0x7fef401566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 -- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 msgr2=0x7fef4013c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 0x7fef4013c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 -- 172.21.15.46:0/3104576203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4015b0d0 con 0x7fef40160bd0 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef3fe00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 0x7fef4013c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef45400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c780 tx=0x7fef3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38017070 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3800a640 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800aa30 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef40158140 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.220+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef40156c30 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.221+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef401f5700 con 0x7fef40160bd0 2024-04-26T23:37:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.222+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef3801a070 con 0x7fef40160bd0 2024-04-26T23:37:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.224+0000 7fef3cc00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 0x7fef2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.224+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fef3808b3f0 con 0x7fef40160bd0 2024-04-26T23:37:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.225+0000 7fef44a00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 0x7fef2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.226+0000 7fef44a00640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 0x7fef2006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef34034000 tx=0x7fef3402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.229+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef38058e80 con 0x7fef40160bd0 2024-04-26T23:37:39.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.487+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fef40069530 con 0x7fef40160bd0 2024-04-26T23:37:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.495+0000 7fef3cc00640 1 -- 172.21.15.46:0/3104576203 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fef3805d180 con 0x7fef40160bd0 2024-04-26T23:37:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 msgr2=0x7fef2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 0x7fef2006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef34034000 tx=0x7fef3402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 msgr2=0x7fef400b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c780 tx=0x7fef3800cc40 comp rx=0 tx=0).stop 2024-04-26T23:37:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 shutdown_connections 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef2006c8f0 0x7fef2006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef40160bd0 0x7fef400b2050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef400732c0 0x7fef4013c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 --2- 172.21.15.46:0/3104576203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4006e510 0x7fef401566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 >> 172.21.15.46:0/3104576203 conn(0x7fef4006c530 msgr2=0x7fef40154150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.500+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 shutdown_connections 2024-04-26T23:37:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.501+0000 7fef47400640 1 -- 172.21.15.46:0/3104576203 wait complete. 2024-04-26T23:37:39.522 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1556001348},{"type":"v1","addr":"172.21.15.142:6833","nonce":1556001348}]},"active_addr":"172.21.15.142:6833/1556001348","active_change":"2024-04-26T23:36:53.861595+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2044279013}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2950152639}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":140345242}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1459395129}]}]} 2024-04-26T23:37:39.526 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T23:37:39.526 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T23:37:39.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.732+0000 7f9f40c00640 1 Processor -- start 2024-04-26T23:37:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.733+0000 7f9f40c00640 1 -- start start 2024-04-26T23:37:39.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.734+0000 7f9f40c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f3c160bd0 0x7f9f3c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.734+0000 7f9f40c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f3c0732c0 0x7f9f3c073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f40c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c161940 con 0x7f9f3c06e510 2024-04-26T23:37:39.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f40c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c071f80 con 0x7f9f3c160bd0 2024-04-26T23:37:39.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f40c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c072260 con 0x7f9f3c0732c0 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f33e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f3c160bd0 0x7f9f3c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f33e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f3c160bd0 0x7f9f3c161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37420/0 (socket says 172.21.15.46:37420) 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f33e00640 1 -- 172.21.15.46:0/4191060608 learned_addr learned my addr 172.21.15.46:0/4191060608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.735+0000 7f9f3a800640 1 --2- 172.21.15.46:0/4191060608 >> v2:172.21.15.142:3300/0 conn(0x7f9f3c0732c0 0x7f9f3c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.736+0000 7f9f33e00640 1 -- 172.21.15.46:0/4191060608 >> v2:172.21.15.142:3300/0 conn(0x7f9f3c0732c0 msgr2=0x7f9f3c073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.736+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4191060608 >> v2:172.21.15.142:3300/0 conn(0x7f9f3c0732c0 0x7f9f3c073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.736+0000 7f9f33e00640 1 -- 172.21.15.46:0/4191060608 >> v1:172.21.15.142:6789/0 conn(0x7f9f3c06e510 legacy=0x7f9f3c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.736+0000 7f9f33e00640 1 -- 172.21.15.46:0/4191060608 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c072540 con 0x7f9f3c160bd0 2024-04-26T23:37:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.737+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4191060608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c160bd0 0x7f9f3c161060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009870 tx=0x7f9f2402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f177e3a8264db24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.737+0000 7f9f39400640 1 -- 172.21.15.46:0/4191060608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24037070 con 0x7f9f3c160bd0 2024-04-26T23:37:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.737+0000 7f9f39400640 1 -- 172.21.15.46:0/4191060608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24032440 con 0x7f9f3c160bd0 2024-04-26T23:37:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.737+0000 7f9f39400640 1 -- 172.21.15.46:0/4191060608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24032810 con 0x7f9f3c160bd0 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 -- 172.21.15.46:0/4191060608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c160bd0 msgr2=0x7f9f3c161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4191060608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c160bd0 0x7f9f3c161060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009870 tx=0x7f9f2402a2a0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 -- 172.21.15.46:0/4191060608 shutdown_connections 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4191060608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c160bd0 0x7f9f3c161060 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4191060608 >> v2:172.21.15.142:3300/0 conn(0x7f9f3c0732c0 0x7f9f3c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 -- 172.21.15.46:0/4191060608 >> 172.21.15.46:0/4191060608 conn(0x7f9f3c06c530 msgr2=0x7f9f3c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:39.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 -- 172.21.15.46:0/4191060608 shutdown_connections 2024-04-26T23:37:39.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.738+0000 7f9f40c00640 1 -- 172.21.15.46:0/4191060608 wait complete. 2024-04-26T23:37:39.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.740+0000 7f9f40c00640 1 Processor -- start 2024-04-26T23:37:39.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.740+0000 7f9f40c00640 1 -- start start 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f3c0732c0 0x7f9f3c0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f3c160bd0 0x7f9f3c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c161720 con 0x7f9f3c06e510 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1615a0 con 0x7f9f3c0732c0 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f40c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c072d40 con 0x7f9f3c160bd0 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f39e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f39e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37422/0 (socket says 172.21.15.46:37422) 2024-04-26T23:37:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.741+0000 7f9f39e00640 1 -- 172.21.15.46:0/4202768738 learned_addr learned my addr 172.21.15.46:0/4202768738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f3c0732c0 0x7f9f3c0b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f3a800640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f3c160bd0 0x7f9f3c0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 -- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f3c160bd0 msgr2=0x7f9f3c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f3c160bd0 0x7f9f3c0b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 -- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f3c0732c0 msgr2=0x7f9f3c0b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f3c0732c0 0x7f9f3c0b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 -- 172.21.15.46:0/4202768738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c13ff60 con 0x7f9f3c06e510 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f39e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c3c0 tx=0x7f9f2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0052d0 con 0x7f9f3c06e510 2024-04-26T23:37:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.742+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c005490 con 0x7f9f3c06e510 2024-04-26T23:37:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.743+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c007260 con 0x7f9f3c06e510 2024-04-26T23:37:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.743+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c0b2e40 con 0x7f9f3c06e510 2024-04-26T23:37:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.743+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c0b3400 con 0x7f9f3c06e510 2024-04-26T23:37:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.744+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c069530 con 0x7f9f3c06e510 2024-04-26T23:37:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.745+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f2c007400 con 0x7f9f3c06e510 2024-04-26T23:37:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.746+0000 7f9f32000640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 0x7f9f1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.746+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f2c08b2b0 con 0x7f9f3c06e510 2024-04-26T23:37:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.746+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 0x7f9f1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.748+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 0x7f9f1806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f24034000 tx=0x7f9f2402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.751+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f2c059430 con 0x7f9f3c06e510 2024-04-26T23:37:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.946+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f3c06fb00 con 0x7f9f3c06e510 2024-04-26T23:37:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.948+0000 7f9f32000640 1 -- 172.21.15.46:0/4202768738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f9f2c058d30 con 0x7f9f3c06e510 2024-04-26T23:37:39.952 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:39.952 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.950+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 msgr2=0x7f9f1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.950+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 0x7f9f1806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f24034000 tx=0x7f9f2402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.950+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 msgr2=0x7f9f3c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.950+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c3c0 tx=0x7f9f2c00c880 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 shutdown_connections 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1806c920 0x7f9f1806ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f3c160bd0 0x7f9f3c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f3c0732c0 0x7f9f3c0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 --2- 172.21.15.46:0/4202768738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c06e510 0x7f9f3c15c5c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 >> 172.21.15.46:0/4202768738 conn(0x7f9f3c06c530 msgr2=0x7f9f3c145b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:39.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 shutdown_connections 2024-04-26T23:37:39.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:39.951+0000 7f9f40c00640 1 -- 172.21.15.46:0/4202768738 wait complete. 2024-04-26T23:37:39.973 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T23:37:39.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:40.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.173+0000 7f5a05e00640 1 Processor -- start 2024-04-26T23:37:40.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.173+0000 7f5a05e00640 1 -- start start 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f5a05e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 0x7f5a000d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f5a05e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a000d5410 0x7f5a000d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f5a05e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a000cfd60 con 0x7f5a0015f4b0 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f5a05e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5a0015f890 con 0x7f5a000d5410 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f5a05e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000d58c0 con 0x7f5a000b5140 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f59ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a000d5410 0x7f5a000d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f59ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a000d5410 0x7f5a000d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37424/0 (socket says 172.21.15.46:37424) 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.174+0000 7f59ff400640 1 -- 172.21.15.46:0/1316560819 learned_addr learned my addr 172.21.15.46:0/1316560819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59fe000640 1 --2- 172.21.15.46:0/1316560819 >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 0x7f5a000d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59ff400640 1 -- 172.21.15.46:0/1316560819 >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 msgr2=0x7f5a000d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59ff400640 1 --2- 172.21.15.46:0/1316560819 >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 0x7f5a000d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59ff400640 1 -- 172.21.15.46:0/1316560819 >> v1:172.21.15.142:6789/0 conn(0x7f5a0015f4b0 legacy=0x7f5a000d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59ff400640 1 -- 172.21.15.46:0/1316560819 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000b55d0 con 0x7f5a000d5410 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.175+0000 7f59fe000640 1 --2- 172.21.15.46:0/1316560819 >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 0x7f5a000d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.176+0000 7f59ff400640 1 --2- 172.21.15.46:0/1316560819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a000d2f50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a0008daa0 tx=0x7f59f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5bf71a56085df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.176+0000 7f59fd600640 1 -- 172.21.15.46:0/1316560819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8036070 con 0x7f5a000d5410 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.176+0000 7f59fd600640 1 -- 172.21.15.46:0/1316560819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f8031600 con 0x7f5a000d5410 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.176+0000 7f59fd600640 1 -- 172.21.15.46:0/1316560819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80319d0 con 0x7f5a000d5410 2024-04-26T23:37:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 -- 172.21.15.46:0/1316560819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 msgr2=0x7f5a000d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 --2- 172.21.15.46:0/1316560819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a000d2f50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a0008daa0 tx=0x7f59f802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 -- 172.21.15.46:0/1316560819 shutdown_connections 2024-04-26T23:37:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 --2- 172.21.15.46:0/1316560819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a000d2f50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 --2- 172.21.15.46:0/1316560819 >> v2:172.21.15.142:3300/0 conn(0x7f5a000b5140 0x7f5a000d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 -- 172.21.15.46:0/1316560819 >> 172.21.15.46:0/1316560819 conn(0x7f5a0008eb50 msgr2=0x7f5a0009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 -- 172.21.15.46:0/1316560819 shutdown_connections 2024-04-26T23:37:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.177+0000 7f5a05e00640 1 -- 172.21.15.46:0/1316560819 wait complete. 2024-04-26T23:37:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.179+0000 7f5a05e00640 1 Processor -- start 2024-04-26T23:37:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.179+0000 7f5a05e00640 1 -- start start 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a0015f4b0 0x7f5a000b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 0x7f5a000c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000b5e70 con 0x7f5a000d5410 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000b5b90 con 0x7f5a000ba390 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f5a05e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a000b4680 con 0x7f5a0015f4b0 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f59fea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f59fea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37428/0 (socket says 172.21.15.46:37428) 2024-04-26T23:37:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f59fea00640 1 -- 172.21.15.46:0/636127379 learned_addr learned my addr 172.21.15.46:0/636127379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f59fea00640 1 -- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a0015f4b0 msgr2=0x7f5a000b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.180+0000 7f59fea00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a0015f4b0 0x7f5a000b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59fea00640 1 -- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 msgr2=0x7f5a000c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59ff400640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 0x7f5a000c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59fea00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 0x7f5a000c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59fea00640 1 -- 172.21.15.46:0/636127379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000cc500 con 0x7f5a000d5410 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59ff400640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 0x7f5a000c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59fea00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c410 tx=0x7f59f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0017070 con 0x7f5a000d5410 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f00035f0 con 0x7f5a000d5410 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0003a20 con 0x7f5a000d5410 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a000c97e0 con 0x7f5a000d5410 2024-04-26T23:37:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.181+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a000c9da0 con 0x7f5a000d5410 2024-04-26T23:37:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.182+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a000ba7a0 con 0x7f5a000d5410 2024-04-26T23:37:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.184+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59f0003bc0 con 0x7f5a000d5410 2024-04-26T23:37:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.184+0000 7f59f7400640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 0x7f59dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.185+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f59f008b230 con 0x7f5a000d5410 2024-04-26T23:37:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.185+0000 7f59fe000640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 0x7f59dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.186+0000 7f59fe000640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 0x7f59dc06ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e8006fd0 tx=0x7f59e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.189+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a000ba7a0 con 0x7f5a000d5410 2024-04-26T23:37:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.397+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a000b4bd0 con 0x7f5a000d5410 2024-04-26T23:37:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.399+0000 7f59f7400640 1 -- 172.21.15.46:0/636127379 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f59f0059280 con 0x7f5a000d5410 2024-04-26T23:37:40.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:40.403 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 msgr2=0x7f59dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 0x7f59dc06ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e8006fd0 tx=0x7f59e8008040 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 msgr2=0x7f5a0015e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c410 tx=0x7f59f000c8d0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 shutdown_connections 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59dc06c920 0x7f59dc06ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000ba390 0x7f5a000c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a0015f4b0 0x7f5a000b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 --2- 172.21.15.46:0/636127379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a000d5410 0x7f5a0015e060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 >> 172.21.15.46:0/636127379 conn(0x7f5a0008eb50 msgr2=0x7f5a0006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 shutdown_connections 2024-04-26T23:37:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.402+0000 7f5a05e00640 1 -- 172.21.15.46:0/636127379 wait complete. 2024-04-26T23:37:40.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T23:37:40.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T23:37:40.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T23:37:40.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T23:37:40.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T23:37:40.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T23:37:40.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T23:37:40.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T23:37:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.729+0000 7f7770200640 1 Processor -- start 2024-04-26T23:37:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.729+0000 7f877a000640 1 Processor -- start 2024-04-26T23:37:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.729+0000 7f7770200640 1 -- start start 2024-04-26T23:37:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.730+0000 7f6dfb200640 1 Processor -- start 2024-04-26T23:37:40.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.730+0000 7f877a000640 1 -- start start 2024-04-26T23:37:40.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.730+0000 7f6dfb200640 1 -- start start 2024-04-26T23:37:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f6dfb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6df4162860 0x7f6df40d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f6dfb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6df406d470 0x7f6df40b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f7770200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77680d0fa0 0x7f77680d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f7770200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 0x7f776806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f7770200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77680d2360 con 0x7f7768166e00 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f6dfb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df406e110 con 0x7f6df40d3a60 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f6dfb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6df406dca0 con 0x7f6df406d470 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f6dfb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6df40d5b00 con 0x7f6df4162860 2024-04-26T23:37:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f7770200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f77680cfcd0 con 0x7f77680d0fa0 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f7770200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77680d1400 con 0x7f77680d1ed0 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f877a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f876c0e6e20 0x7f876c0e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f877a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876c0076f0 con 0x7f876c057b00 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f877a000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876c0079d0 con 0x7f876c0e31c0 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.732+0000 7f877a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f876c057670 con 0x7f876c0e6e20 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.733+0000 7f776d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7768166e00 0x7f77680d3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40136/0 (socket says 172.21.15.46:40136) 2024-04-26T23:37:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.733+0000 7f776d800640 1 -- 172.21.15.46:0/1145766507 learned_addr learned my addr 172.21.15.46:0/1145766507 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f6df9e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6df4162860 0x7f6df40d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f6df9e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6df4162860 0x7f6df40d3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59140/0 (socket says 172.21.15.46:59140) 2024-04-26T23:37:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.733+0000 7f8779600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f876c057b00 0x7f876c057f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40138/0 (socket says 172.21.15.46:40138) 2024-04-26T23:37:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.733+0000 7f8779600640 1 -- 172.21.15.46:0/1211723461 learned_addr learned my addr 172.21.15.46:0/1211723461 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f8773e00640 1 --2- 172.21.15.46:0/1211723461 >> v2:172.21.15.142:3300/0 conn(0x7f876c0e6e20 0x7f876c0e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f6df9e00640 1 -- 172.21.15.46:0/1538555657 learned_addr learned my addr 172.21.15.46:0/1538555657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f776e200640 1 --2- 172.21.15.46:0/1145766507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 0x7f776806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f776ce00640 1 --2- 172.21.15.46:0/1145766507 >> v2:172.21.15.46:3300/0 conn(0x7f77680d0fa0 0x7f77680d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.734+0000 7f6dfa800640 1 --2- 172.21.15.46:0/1538555657 >> v2:172.21.15.46:3300/0 conn(0x7f6df406d470 0x7f6df40b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f7767e00640 1 -- 172.21.15.46:0/1145766507 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372609121 0 0) 0x7f77680d2360 con 0x7f7768166e00 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f7767e00640 1 -- 172.21.15.46:0/1145766507 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7750003520 con 0x7f7768166e00 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f776e200640 1 -- 172.21.15.46:0/1145766507 >> v2:172.21.15.46:3300/0 conn(0x7f77680d0fa0 msgr2=0x7f77680d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f776e200640 1 --2- 172.21.15.46:0/1145766507 >> v2:172.21.15.46:3300/0 conn(0x7f77680d0fa0 0x7f77680d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f776e200640 1 -- 172.21.15.46:0/1145766507 >> v1:172.21.15.46:6789/0 conn(0x7f7768166e00 legacy=0x7f77680d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f776e200640 1 -- 172.21.15.46:0/1145766507 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680cff00 con 0x7f77680d1ed0 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f8773e00640 1 -- 172.21.15.46:0/1211723461 >> v1:172.21.15.142:6789/0 conn(0x7f876c0e31c0 legacy=0x7f876c006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f8773e00640 1 -- 172.21.15.46:0/1211723461 >> v1:172.21.15.46:6789/0 conn(0x7f876c057b00 legacy=0x7f876c057f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f8773e00640 1 -- 172.21.15.46:0/1211723461 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c057300 con 0x7f876c0e6e20 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f6df9e00640 1 -- 172.21.15.46:0/1538555657 >> v2:172.21.15.46:3300/0 conn(0x7f6df406d470 msgr2=0x7f6df40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f6df9e00640 1 --2- 172.21.15.46:0/1538555657 >> v2:172.21.15.46:3300/0 conn(0x7f6df406d470 0x7f6df40b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f6df9e00640 1 -- 172.21.15.46:0/1538555657 >> v1:172.21.15.46:6789/0 conn(0x7f6df40d3a60 legacy=0x7f6df406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.735+0000 7f6df9e00640 1 -- 172.21.15.46:0/1538555657 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df40d5de0 con 0x7f6df4162860 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.737+0000 7f776e200640 1 --2- 172.21.15.46:0/1145766507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 0x7f776806cf30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c3de3e9b5a8d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.737+0000 7f6df9e00640 1 --2- 172.21.15.46:0/1538555657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df40d3520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6df406de20 tx=0x7f6de802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f42b5762c258011 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f7767e00640 1 -- 172.21.15.46:0/1145766507 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760007740 con 0x7f77680d1ed0 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f7767e00640 1 -- 172.21.15.46:0/1145766507 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77600078e0 con 0x7f77680d1ed0 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f7767e00640 1 -- 172.21.15.46:0/1145766507 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760007cb0 con 0x7f77680d1ed0 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f6df8a00640 1 -- 172.21.15.46:0/1538555657 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8037070 con 0x7f6df4162860 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f6df8a00640 1 -- 172.21.15.46:0/1538555657 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de8032440 con 0x7f6df4162860 2024-04-26T23:37:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f6df8a00640 1 -- 172.21.15.46:0/1538555657 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8032810 con 0x7f6df4162860 2024-04-26T23:37:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f2ede200640 1 Processor -- start 2024-04-26T23:37:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 -- 172.21.15.46:0/1538555657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 msgr2=0x7f6df40d3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 --2- 172.21.15.46:0/1538555657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df40d3520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6df406de20 tx=0x7f6de802a210 comp rx=0 tx=0).stop 2024-04-26T23:37:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f2ede200640 1 -- start start 2024-04-26T23:37:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 -- 172.21.15.46:0/1538555657 shutdown_connections 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 --2- 172.21.15.46:0/1538555657 >> v2:172.21.15.46:3300/0 conn(0x7f6df406d470 0x7f6df40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 --2- 172.21.15.46:0/1538555657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df40d3520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.739+0000 7f6dfb200640 1 -- 172.21.15.46:0/1538555657 >> 172.21.15.46:0/1538555657 conn(0x7f6df40d4f40 msgr2=0x7f6df40d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.738+0000 7f8773e00640 1 --2- 172.21.15.46:0/1211723461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c0e7260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f87600097a0 tx=0x7f876002a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678d2884aa8b3643 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.740+0000 7f7770200640 1 -- 172.21.15.46:0/1145766507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 msgr2=0x7f776806cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.740+0000 7f2ede200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 0x7f2ed8073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.740+0000 7f2ede200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed8073960 con 0x7f2ed81607d0 2024-04-26T23:37:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f2ede200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed815d290 con 0x7f2ed815cc90 2024-04-26T23:37:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f2ede200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed815d570 con 0x7f2ed8073040 2024-04-26T23:37:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.740+0000 7f7770200640 1 --2- 172.21.15.46:0/1145766507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 0x7f776806cf30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776002a660 comp rx=0 tx=0).stop 2024-04-26T23:37:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f2ed6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2ed81607d0 0x7f2ed815c580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40152/0 (socket says 172.21.15.46:40152) 2024-04-26T23:37:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f2ed6a00640 1 -- 172.21.15.46:0/2812514684 learned_addr learned my addr 172.21.15.46:0/2812514684 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 -- 172.21.15.46:0/1145766507 shutdown_connections 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 --2- 172.21.15.46:0/1145766507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d1ed0 0x7f776806cf30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 --2- 172.21.15.46:0/1145766507 >> v2:172.21.15.46:3300/0 conn(0x7f77680d0fa0 0x7f77680d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 -- 172.21.15.46:0/1145766507 >> 172.21.15.46:0/1145766507 conn(0x7f776809e450 msgr2=0x7f77680d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 -- 172.21.15.46:0/1145766507 shutdown_connections 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f7770200640 1 -- 172.21.15.46:0/1145766507 wait complete. 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f6dfb200640 1 -- 172.21.15.46:0/1538555657 shutdown_connections 2024-04-26T23:37:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.741+0000 7f2ed7400640 1 --2- 172.21.15.46:0/2812514684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 0x7f2ed8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.742+0000 7f6dfb200640 1 -- 172.21.15.46:0/1538555657 wait complete. 2024-04-26T23:37:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.742+0000 7f2ed6000640 1 -- 172.21.15.46:0/2812514684 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634568843 0 0) 0x7f2ed8073960 con 0x7f2ed81607d0 2024-04-26T23:37:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.742+0000 7f2ed6000640 1 -- 172.21.15.46:0/2812514684 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ebc003520 con 0x7f2ed81607d0 2024-04-26T23:37:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.743+0000 7f7770200640 1 Processor -- start 2024-04-26T23:37:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.743+0000 7f7770200640 1 -- start start 2024-04-26T23:37:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.743+0000 7f2ed7400640 1 -- 172.21.15.46:0/2812514684 >> v1:172.21.15.142:6789/0 conn(0x7f2ed815cc90 legacy=0x7f2ed80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.743+0000 7f2ed7400640 1 -- 172.21.15.46:0/2812514684 >> v1:172.21.15.46:6789/0 conn(0x7f2ed81607d0 legacy=0x7f2ed815c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.743+0000 7f2ed7400640 1 -- 172.21.15.46:0/2812514684 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed815fcf0 con 0x7f2ed8073040 2024-04-26T23:37:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d0fa0 0x7f77681f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77680d1ed0 0x7f776806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 0x7f776806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77680d07a0 con 0x7f7768166e00 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77680d04c0 con 0x7f77680d1ed0 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f7770200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77680d0a80 con 0x7f77680d0fa0 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.744+0000 7f2ed7400640 1 --2- 172.21.15.46:0/2812514684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 0x7f2ed8073420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009750 tx=0x7f2ec802a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4161fc468a6c498b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f776e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 0x7f776806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f2ed6000640 1 -- 172.21.15.46:0/2812514684 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8037070 con 0x7f2ed8073040 2024-04-26T23:37:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f2ed6000640 1 -- 172.21.15.46:0/2812514684 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec802ad20 con 0x7f2ed8073040 2024-04-26T23:37:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f2ed6000640 1 -- 172.21.15.46:0/2812514684 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8032660 con 0x7f2ed8073040 2024-04-26T23:37:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f6dfb200640 1 Processor -- start 2024-04-26T23:37:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f6dfb200640 1 -- start start 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f776e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 0x7f776806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37462/0 (socket says 172.21.15.46:37462) 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f776e200640 1 -- 172.21.15.46:0/2514278483 learned_addr learned my addr 172.21.15.46:0/2514278483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f776ce00640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77680d1ed0 0x7f776806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.745+0000 7f776d800640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d0fa0 0x7f77681f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f8773400640 1 -- 172.21.15.46:0/1211723461 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760037070 con 0x7f876c0e6e20 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f8773400640 1 -- 172.21.15.46:0/1211723461 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8760032640 con 0x7f876c0e6e20 2024-04-26T23:37:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f8773400640 1 -- 172.21.15.46:0/1211723461 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760032a50 con 0x7f876c0e6e20 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 -- 172.21.15.46:0/2812514684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 msgr2=0x7f2ed8073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 --2- 172.21.15.46:0/2812514684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 0x7f2ed8073420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009750 tx=0x7f2ec802a010 comp rx=0 tx=0).stop 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 -- 172.21.15.46:0/2812514684 shutdown_connections 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 --2- 172.21.15.46:0/2812514684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed8073040 0x7f2ed8073420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 -- 172.21.15.46:0/2812514684 >> 172.21.15.46:0/2812514684 conn(0x7f2ed80d1d80 msgr2=0x7f2ed80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7fc127c00640 1 Processor -- start 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f3791e00640 1 Processor -- start 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f776e200640 1 -- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d0fa0 msgr2=0x7f77681f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f776e200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77680d0fa0 0x7f77681f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f776e200640 1 -- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77680d1ed0 msgr2=0x7f776806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f776e200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77680d1ed0 0x7f776806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 -- 172.21.15.46:0/2812514684 shutdown_connections 2024-04-26T23:37:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f776e200640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77681fd6c0 con 0x7f7768166e00 2024-04-26T23:37:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7fc127c00640 1 -- start start 2024-04-26T23:37:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f3791e00640 1 -- start start 2024-04-26T23:37:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.746+0000 7f2ede200640 1 -- 172.21.15.46:0/2812514684 wait complete. 2024-04-26T23:37:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f776e200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 0x7f776806e5f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7760009eb0 tx=0x7f7760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f877a000640 1 -- 172.21.15.46:0/1211723461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 msgr2=0x7f876c0e7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f877a000640 1 --2- 172.21.15.46:0/1211723461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c0e7260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f87600097a0 tx=0x7f876002a2e0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760040070 con 0x7f7768166e00 2024-04-26T23:37:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760031b40 con 0x7f7768166e00 2024-04-26T23:37:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.747+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760045460 con 0x7f7768166e00 2024-04-26T23:37:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f6dfb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6df406d470 0x7f6df4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f6dfb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df4070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f6dfb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6df4070e30 0x7f6df40b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f877a000640 1 -- 172.21.15.46:0/1211723461 shutdown_connections 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7fc127c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1200d21a0 0x7fc1200d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7fc127c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1200d0d20 0x7fc1201692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7fc127c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1200d2e80 con 0x7fc12015f3b0 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7fc127c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc1200d1170 con 0x7fc1200d21a0 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7fc127c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc1200d3140 con 0x7fc1200d0d20 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77681561c0 con 0x7f7768166e00 2024-04-26T23:37:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7768156780 con 0x7f7768166e00 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f877a000640 1 --2- 172.21.15.46:0/1211723461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c0e7260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.748+0000 7f877a000640 1 -- 172.21.15.46:0/1211723461 >> 172.21.15.46:0/1211723461 conn(0x7f876c03aa30 msgr2=0x7f876c03ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f2ede200640 1 Processor -- start 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6dfb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6df40d27e0 con 0x7f6df4070e30 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6dfb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6df406e3f0 con 0x7f6df4162860 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6dfb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6df4162c40 con 0x7f6df406d470 2024-04-26T23:37:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6df9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7fc11fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1200d21a0 0x7fc1200d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7fc11fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1200d21a0 0x7fc1200d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37482/0 (socket says 172.21.15.46:37482) 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7fc11fe00640 1 -- 172.21.15.46:0/562152221 learned_addr learned my addr 172.21.15.46:0/562152221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6df9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df4070550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59154/0 (socket says 172.21.15.46:59154) 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6df9400640 1 -- 172.21.15.46:0/2316875537 learned_addr learned my addr 172.21.15.46:0/2316875537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f6dfa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6df4070e30 0x7f6df40b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7f877a000640 1 -- 172.21.15.46:0/1211723461 shutdown_connections 2024-04-26T23:37:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7fc125c00640 1 --2- 172.21.15.46:0/562152221 >> v2:172.21.15.142:3300/0 conn(0x7fc1200d0d20 0x7fc1201692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f2ede200640 1 -- start start 2024-04-26T23:37:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.749+0000 7fe1b9e00640 1 Processor -- start 2024-04-26T23:37:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7fe1b9e00640 1 -- start start 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f6df9400640 1 -- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6df406d470 msgr2=0x7f6df4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f6df9400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6df406d470 0x7f6df4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f6df9400640 1 -- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6df4070e30 msgr2=0x7f6df40b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f3791e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f378c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f378aa00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f378c0d36f0 0x7f378c0d3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46818/0 (socket says 172.21.15.46:46818) 2024-04-26T23:37:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f378aa00640 1 -- 172.21.15.46:0/1575748262 learned_addr learned my addr 172.21.15.46:0/1575748262 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11f400640 1 -- 172.21.15.46:0/562152221 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290348033 0 0) 0x7fc1200d2e80 con 0x7fc12015f3b0 2024-04-26T23:37:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11f400640 1 -- 172.21.15.46:0/562152221 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc108003520 con 0x7fc12015f3b0 2024-04-26T23:37:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77680693b0 con 0x7f7768166e00 2024-04-26T23:37:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f3791e00640 1 --2- 172.21.15.46:0/1575748262 >> v2:172.21.15.46:3300/0 conn(0x7f378c0d21e0 0x7f378c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.750+0000 7f877a000640 1 -- 172.21.15.46:0/1211723461 wait complete. 2024-04-26T23:37:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378c0b1d60 con 0x7f378c0d36f0 2024-04-26T23:37:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f6df9400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6df4070e30 0x7f6df40b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f6df9400640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41fcdc0 con 0x7f6df4162860 2024-04-26T23:37:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f6dfa800640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6df4070e30 0x7f6df40b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f378c0d3300 con 0x7f378c0d21e0 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f378c0d6010 con 0x7f378c15f750 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11fe00640 1 -- 172.21.15.46:0/562152221 >> v2:172.21.15.142:3300/0 conn(0x7fc1200d0d20 msgr2=0x7fc1201692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11fe00640 1 --2- 172.21.15.46:0/562152221 >> v2:172.21.15.142:3300/0 conn(0x7fc1200d0d20 0x7fc1201692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11fe00640 1 -- 172.21.15.46:0/562152221 >> v1:172.21.15.142:6789/0 conn(0x7fc12015f3b0 legacy=0x7fc1200d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.751+0000 7fc11fe00640 1 -- 172.21.15.46:0/562152221 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1200d3420 con 0x7fc1200d21a0 2024-04-26T23:37:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.752+0000 7f378a000640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f378c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.752+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529580140 0 0) 0x7f378c0b1d60 con 0x7f378c0d36f0 2024-04-26T23:37:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.752+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3770003520 con 0x7f378c0d36f0 2024-04-26T23:37:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f2ede200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 0x7f2ed806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.752+0000 7f378b400640 1 --2- 172.21.15.46:0/1575748262 >> v2:172.21.15.46:3300/0 conn(0x7f378c0d21e0 0x7f378c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7760039040 con 0x7f7768166e00 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 748618573 0 0) 0x7f3770003520 con 0x7f378c0d36f0 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 msgr2=0x7f378c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f378c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 >> v2:172.21.15.46:3300/0 conn(0x7f378c0d21e0 msgr2=0x7f378c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7fe1b9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1b40d0d20 0x7fe1b41692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 --2- 172.21.15.46:0/1575748262 >> v2:172.21.15.46:3300/0 conn(0x7f378c0d21e0 0x7f378c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c0d3020 con 0x7f378c0d36f0 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3784001a10 con 0x7f378c0d36f0 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 >> v1:172.21.15.142:6789/0 conn(0x7f378c0d36f0 legacy=0x7f378c0d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f37700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3770003f30 0x7f3770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770004d40 0x7f37700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0d3300 con 0x7f3770004d40 2024-04-26T23:37:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0d6010 con 0x7f3770003f30 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fc11fe00640 1 --2- 172.21.15.46:0/562152221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200d21a0 0x7fc1200d07e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc12008ea20 tx=0x7fc11402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa815a9e1ac72069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f377000bb50 con 0x7f378c15f750 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f378b400640 1 --2- 172.21.15.46:0/1575748262 >> v2:172.21.15.46:3300/0 conn(0x7f378c0d21e0 0x7f378c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f2ede200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed815cc90 0x7f2ed8146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f2ede200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed81607d0 0x7f2ed8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.753+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 0x7f2ed806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 0x7f2ed806c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37500/0 (socket says 172.21.15.46:37500) 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b2a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe1b415f3b0 0x7fe1b40d1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46828/0 (socket says 172.21.15.46:46828) 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54528/0 (socket says 172.21.15.46:54528) 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f378aa00640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f37700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b2a00640 1 -- 172.21.15.46:0/2657933107 learned_addr learned my addr 172.21.15.46:0/2657933107 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f6df9400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df4070550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c430 tx=0x7f6de400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b40d2e80 con 0x7fe1b415f3b0 2024-04-26T23:37:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b40d3140 con 0x7fe1b40d0d20 2024-04-26T23:37:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b40d3420 con 0x7fe1b40d21a0 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f378a000640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3770003f30 0x7f3770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7fc11f400640 1 -- 172.21.15.46:0/562152221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114037070 con 0x7fc1200d21a0 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f7765600640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f774806c940 0x7f774806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f776d800640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f774806c940 0x7f774806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f77600ad5c0 con 0x7f7768166e00 2024-04-26T23:37:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f877a000640 1 Processor -- start 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f877a000640 1 -- start start 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f2ede200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81602b0 con 0x7f2ed8073040 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f2ede200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80ce360 con 0x7f2ed81607d0 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f2ede200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed8159e50 con 0x7f2ed815cc90 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f2ed7400640 1 -- 172.21.15.46:0/3976204056 learned_addr learned my addr 172.21.15.46:0/3976204056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f2ed6a00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed815cc90 0x7f2ed8146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.754+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4011790 con 0x7f6df4162860 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de4017070 con 0x7f6df4162860 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4011bb0 con 0x7f6df4162860 2024-04-26T23:37:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f6dfb200640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df41fd0a0 con 0x7f6df4162860 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f6dfb200640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41fd660 con 0x7f6df4162860 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fc11f400640 1 -- 172.21.15.46:0/562152221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc114032440 con 0x7fc1200d21a0 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fc11f400640 1 -- 172.21.15.46:0/562152221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140327f0 con 0x7fc1200d21a0 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b3400640 1 --2- 172.21.15.46:0/2657933107 >> v2:172.21.15.46:3300/0 conn(0x7fe1b40d0d20 0x7fe1b41692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b2000640 1 -- 172.21.15.46:0/2657933107 >> v2:172.21.15.46:3300/0 conn(0x7fe1b40d0d20 msgr2=0x7fe1b41692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b2000640 1 --2- 172.21.15.46:0/2657933107 >> v2:172.21.15.46:3300/0 conn(0x7fe1b40d0d20 0x7fe1b41692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b2000640 1 -- 172.21.15.46:0/2657933107 >> v1:172.21.15.142:6789/0 conn(0x7fe1b415f3b0 legacy=0x7fe1b40d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b1600640 1 -- 172.21.15.46:0/2657933107 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347962724 0 0) 0x7fe1b40d2e80 con 0x7fe1b415f3b0 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7fe1b2000640 1 -- 172.21.15.46:0/2657933107 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b40d3700 con 0x7fe1b40d21a0 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f378aa00640 1 -- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3770003f30 msgr2=0x7f3770004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f378aa00640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3770003f30 0x7f3770004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f378aa00640 1 -- 172.21.15.46:0/1575748262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770004d40 msgr2=0x7f37700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f378aa00640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770004d40 0x7f37700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.755+0000 7f378aa00640 1 -- 172.21.15.46:0/1575748262 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c0d3020 con 0x7f378c15f750 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f2ed7e00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed81607d0 0x7f2ed8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f7765600640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f77480760c0 0x7f7748078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7760043e50 con 0x7f77480760c0 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77600b4050 con 0x7f7768166e00 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 -- 172.21.15.46:0/562152221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200d21a0 msgr2=0x7fc1200d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 --2- 172.21.15.46:0/562152221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200d21a0 0x7fc1200d07e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc12008ea20 tx=0x7fc11402a270 comp rx=0 tx=0).stop 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 -- 172.21.15.46:0/562152221 shutdown_connections 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 --2- 172.21.15.46:0/562152221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200d21a0 0x7fc1200d07e0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc12008ea20 tx=0x7fc11402a270 comp rx=0 tx=0).stop 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 --2- 172.21.15.46:0/562152221 >> v2:172.21.15.142:3300/0 conn(0x7fc1200d0d20 0x7fc1201692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.756+0000 7fc127c00640 1 -- 172.21.15.46:0/562152221 >> 172.21.15.46:0/562152221 conn(0x7fc12009e450 msgr2=0x7fc1200d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed7400640 1 -- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed815cc90 msgr2=0x7f2ed8146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f877a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 0x7f876c0e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6dcc001030 con 0x7f6df4162860 2024-04-26T23:37:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed7400640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed815cc90 0x7f2ed8146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed7400640 1 -- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed81607d0 msgr2=0x7f2ed8147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed7400640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed81607d0 0x7f2ed8147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed7400640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80c7590 con 0x7f2ed8073040 2024-04-26T23:37:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f2ed6a00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed815cc90 0x7f2ed8146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7fc127c00640 1 -- 172.21.15.46:0/562152221 shutdown_connections 2024-04-26T23:37:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f877a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876c0e31c0 0x7f876c009f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7f776ce00640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f77480760c0 0x7f7748078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f877a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c03d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f8778c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 0x7f876c0e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f8778c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 0x7f876c0e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54546/0 (socket says 172.21.15.46:54546) 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f8778c00640 1 -- 172.21.15.46:0/2211000282 learned_addr learned my addr 172.21.15.46:0/2211000282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7fe1b2000640 1 --2- 172.21.15.46:0/2657933107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00097a0 tx=0x7fe1a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55abc3b028caff59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f378aa00640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f37700039b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f378400a8f0 tx=0x7f3770005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc4388959a76577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f876c007360 con 0x7f876c0e31c0 2024-04-26T23:37:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f8773e00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876c0e31c0 0x7f876c009f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f876c0e77a0 con 0x7f876c0e6e20 2024-04-26T23:37:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f776ce00640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f77480760c0 0x7f7748078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f876c03c0c0 con 0x7f876c057b00 2024-04-26T23:37:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378400a4f0 con 0x7f378c15f750 2024-04-26T23:37:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7fe1b1600640 1 -- 172.21.15.46:0/2657933107 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0037070 con 0x7fe1b40d21a0 2024-04-26T23:37:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7fe1b1600640 1 -- 172.21.15.46:0/2657933107 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a0032440 con 0x7fe1b40d21a0 2024-04-26T23:37:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378402f070 con 0x7f378c15f750 2024-04-26T23:37:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7f3789600640 1 -- 172.21.15.46:0/1575748262 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378402bce0 con 0x7f378c15f750 2024-04-26T23:37:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7f8779600640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c03d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7f2ed7400640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 0x7f2ed806c5c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009840 tx=0x7f2ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7f776d800640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f774806c940 0x7f774806ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f775c005880 tx=0x7f775c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 -- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 msgr2=0x7f876c03d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c0e6e20 0x7f876c03d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 -- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876c0e31c0 msgr2=0x7f876c009f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876c0e31c0 0x7f876c009f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c00a9c0 con 0x7f876c057b00 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.757+0000 7fc127c00640 1 -- 172.21.15.46:0/562152221 wait complete. 2024-04-26T23:37:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7fe1b1600640 1 -- 172.21.15.46:0/2657933107 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0032810 con 0x7fe1b40d21a0 2024-04-26T23:37:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 msgr2=0x7f37700039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f3791e00640 1 --2- 172.21.15.46:0/1575748262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c15f750 0x7f37700039b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f378400a8f0 tx=0x7f3770005830 comp rx=0 tx=0).stop 2024-04-26T23:37:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 msgr2=0x7fe1b40d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/2657933107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00097a0 tx=0x7fe1a002a180 comp rx=0 tx=0).stop 2024-04-26T23:37:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== osd.0 v2:172.21.15.46:6800/3755207586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7760043e50 con 0x7f77480760c0 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 shutdown_connections 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/2657933107 >> v2:172.21.15.46:3300/0 conn(0x7fe1b40d0d20 0x7fe1b41692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/2657933107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b40d21a0 0x7fe1b40d07e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 >> 172.21.15.46:0/2657933107 conn(0x7fe1b409e450 msgr2=0x7fe1b40d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f8778c00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 0x7f876c0e51d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f876800c450 tx=0x7f876800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.760+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6de401c460 con 0x7f6df4162860 2024-04-26T23:37:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f378b400640 1 -- 172.21.15.46:0/1575748262 reap_dead start 2024-04-26T23:37:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 shutdown_connections 2024-04-26T23:37:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.758+0000 7ff8ddc00640 1 Processor -- start 2024-04-26T23:37:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.759+0000 7ff8ddc00640 1 -- start start 2024-04-26T23:37:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 >> 172.21.15.46:0/1575748262 conn(0x7f378c0d5110 msgr2=0x7f378c0d54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7fc127c00640 1 Processor -- start 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 shutdown_connections 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7ff8ddc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff8d8167650 0x7ff8d80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7ff8ddc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8d80d1a00 0x7ff8d8167a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768017070 con 0x7f876c057b00 2024-04-26T23:37:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87680052d0 con 0x7f876c057b00 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768003830 con 0x7f876c057b00 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7ff8ddc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8d8163d10 con 0x7ff8d8070bc0 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7ff8ddc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d8168390 con 0x7ff8d80d1a00 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7ff8ddc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d81680b0 con 0x7ff8d8167650 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c180d60 con 0x7f876c057b00 2024-04-26T23:37:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c181320 con 0x7f876c057b00 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 shutdown_connections 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8037070 con 0x7f2ed8073040 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec8038040 con 0x7f2ed8073040 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004470 con 0x7f2ed8073040 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed80c7870 con 0x7f2ed8073040 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7ff8d7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8d8070bc0 0x7ff8d806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40158/0 (socket says 172.21.15.46:40158) 2024-04-26T23:37:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7ff8d7400640 1 -- 172.21.15.46:0/3943997957 learned_addr learned my addr 172.21.15.46:0/3943997957 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7fc127c00640 1 -- start start 2024-04-26T23:37:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7fe1b9e00640 1 -- 172.21.15.46:0/2657933107 wait complete. 2024-04-26T23:37:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed80c7e30 con 0x7f2ed8073040 2024-04-26T23:37:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f6dee000640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c940 0x7f6dd806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6de408c730 con 0x7f6df4162860 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f6dee000640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7f6dd80760c0 0x7f6dd8078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6dd8078bd0 con 0x7f6dd80760c0 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6de408cae0 con 0x7f6df4162860 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.763+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f876c0057e0 con 0x7f876c057b00 2024-04-26T23:37:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.763+0000 7f6df9e00640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c940 0x7f6dd806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.762+0000 7ff8d6a00640 1 --2- 172.21.15.46:0/3943997957 >> v2:172.21.15.142:3300/0 conn(0x7ff8d8167650 0x7ff8d80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.763+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673671743 0 0) 0x7ff8d8163d10 con 0x7ff8d8070bc0 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.763+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8bc003520 con 0x7ff8d8070bc0 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1619983086 0 0) 0x7ff8bc003520 con 0x7ff8d8070bc0 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 >> v2:172.21.15.142:3300/0 conn(0x7ff8d8167650 msgr2=0x7ff8d80b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 --2- 172.21.15.46:0/3943997957 >> v2:172.21.15.142:3300/0 conn(0x7ff8d8167650 0x7ff8d80b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 >> v2:172.21.15.46:3300/0 conn(0x7ff8d80d1a00 msgr2=0x7ff8d8167a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 --2- 172.21.15.46:0/3943997957 >> v2:172.21.15.46:3300/0 conn(0x7ff8d80d1a00 0x7ff8d8167a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d80cfb20 con 0x7ff8d8070bc0 2024-04-26T23:37:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7f6dfa800640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7f6dd80760c0 0x7f6dd8078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff8d00019f0 con 0x7ff8d8070bc0 2024-04-26T23:37:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 >> v1:172.21.15.46:6789/0 conn(0x7ff8d8070bc0 legacy=0x7ff8d806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8bc0039b0 0x7ff8bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1200d0d20 0x7fc1201e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7ff8d5600640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 0x7fc1200c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200c9120 0x7fc120161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc12015f790 con 0x7fc1200c9120 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200d39e0 con 0x7fc1200d0d20 2024-04-26T23:37:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7fc127c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc120160190 con 0x7fc12015f3b0 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7ff8d5600640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc0051c0 0x7ff8bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8168390 con 0x7ff8bc004800 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d81680b0 con 0x7ff8bc0051c0 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc00c140 con 0x7ff8bc0039b0 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7fe1b9e00640 1 Processor -- start 2024-04-26T23:37:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7fe1b9e00640 1 -- start start 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.761+0000 7f3791e00640 1 -- 172.21.15.46:0/1575748262 wait complete. 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.764+0000 7f3791e00640 1 Processor -- start 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7f3791e00640 1 -- start start 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7f6dfa800640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7f6dd80760c0 0x7f6dd8078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7ff8d6000640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7ff8d6a00640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8bc0039b0 0x7ff8bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7ff8d7400640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc0051c0 0x7ff8bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7fc11fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 0x7fc1200c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7fc11fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 0x7fc1200c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54558/0 (socket says 172.21.15.46:54558) 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.765+0000 7fc11fe00640 1 -- 172.21.15.46:0/2777409140 learned_addr learned my addr 172.21.15.46:0/2777409140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7fc11fe00640 1 -- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1200d0d20 msgr2=0x7fc1201e7bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7fc11fe00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1200d0d20 0x7fc1201e7bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7fc11fe00640 1 -- 172.21.15.46:0/2777409140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200c9120 msgr2=0x7fc120161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7fc11fe00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1200c9120 0x7fc120161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.766+0000 7fc11fe00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc120160470 con 0x7fc12015f3b0 2024-04-26T23:37:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.767+0000 7fc11fe00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 0x7fc1200c9810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc114035040 tx=0x7fc1140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.767+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 0x7fe1b4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.767+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1b40d21a0 0x7fe1b4162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.767+0000 7f6df9e00640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c940 0x7f6dd806ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6de8034000 tx=0x7f6de802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b415f3b0 0x7fe1b40b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114041070 con 0x7fc12015f3b0 2024-04-26T23:37:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc114004220 con 0x7fc12015f3b0 2024-04-26T23:37:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114037040 con 0x7fc12015f3b0 2024-04-26T23:37:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 0x7fe1b4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7f3791e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 0x7f378c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7ff8d6000640 1 -- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8bc0039b0 msgr2=0x7ff8bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b415f790 con 0x7fe1b40d0d20 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40d3f00 con 0x7fe1b40d21a0 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b41537c0 con 0x7fe1b415f3b0 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc120160750 con 0x7fc12015f3b0 2024-04-26T23:37:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc120160d10 con 0x7fc12015f3b0 2024-04-26T23:37:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 0x7fe1b4153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37560/0 (socket says 172.21.15.46:37560) 2024-04-26T23:37:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7fe1b2a00640 1 -- 172.21.15.46:0/3545978733 learned_addr learned my addr 172.21.15.46:0/3545978733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7f3791e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c0ce8d0 0x7f378c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2a00640 1 -- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b415f3b0 msgr2=0x7fe1b40b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7f3791e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 0x7f378c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f378b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 0x7f378c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f378aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 0x7f378c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f378aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 0x7f378c0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59222/0 (socket says 172.21.15.46:59222) 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f378b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 0x7f378c0cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37562/0 (socket says 172.21.15.46:37562) 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f378aa00640 1 -- 172.21.15.46:0/3020061521 learned_addr learned my addr 172.21.15.46:0/3020061521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0b4000 con 0x7f378c0c9650 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0d0f90 con 0x7f378c15c770 2024-04-26T23:37:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f378c0b4180 con 0x7f378c0ce8d0 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b3400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b415f3b0 0x7fe1b40b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2000640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1b40d21a0 0x7fe1b4162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2a00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b415f3b0 0x7fe1b40b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2a00640 1 -- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1b40d21a0 msgr2=0x7fe1b4162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2a00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1b40d21a0 0x7fe1b4162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2a00640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b4160fc0 con 0x7fe1b40d0d20 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b3400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1b415f3b0 0x7fe1b40b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7fe1b2000640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1b40d21a0 0x7fe1b4162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7ff8d6000640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8bc0039b0 0x7ff8bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7ff8d6000640 1 -- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc0051c0 msgr2=0x7ff8bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7ff8d6000640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc0051c0 0x7ff8bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.768+0000 7ff8d6000640 1 -- 172.21.15.46:0/3943997957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc003520 con 0x7ff8bc004800 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7ff8d7400640 1 -- 172.21.15.46:0/3943997957 reap_dead start 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7fe1b2a00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 0x7fe1b4153280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c780 tx=0x7fe1ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 -- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c0ce8d0 msgr2=0x7f378c0cafe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378c0ce8d0 0x7f378c0cafe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 -- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 msgr2=0x7f378c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 0x7f378c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c15a620 con 0x7f378c0c9650 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378aa00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378c15c770 0x7f378c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f378b400640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 0x7f378c0cb520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3784009000 tx=0x7f377800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac018070 con 0x7fe1b40d0d20 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac00a640 con 0x7fe1b40d0d20 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00aa70 con 0x7fe1b40d0d20 2024-04-26T23:37:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== osd.3 v2:172.21.15.46:6824/1706950386 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6dd8078bd0 con 0x7f6dd80760c0 2024-04-26T23:37:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b41612a0 con 0x7fe1b40d0d20 2024-04-26T23:37:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ec802abe0 con 0x7f2ed8073040 2024-04-26T23:37:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.769+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ed80cdb60 con 0x7f2ed8073040 2024-04-26T23:37:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800f040 con 0x7f378c0c9650 2024-04-26T23:37:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1200693b0 con 0x7fc12015f3b0 2024-04-26T23:37:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3778017e00 con 0x7f378c0c9650 2024-04-26T23:37:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800eea0 con 0x7f378c0c9650 2024-04-26T23:37:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.771+0000 7f2ecf400640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2eb806c9c0 0x7f2eb806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2ec80abd20 con 0x7f2ed8073040 2024-04-26T23:37:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c0c9bc0 con 0x7f378c0c9650 2024-04-26T23:37:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f2ecf400640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7f2eb8076130 0x7f2eb8078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ec80067f0 con 0x7f2eb8076130 2024-04-26T23:37:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ec80b1310 con 0x7f2ed8073040 2024-04-26T23:37:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c0ca180 con 0x7f378c0c9650 2024-04-26T23:37:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b4161860 con 0x7fe1b40d0d20 2024-04-26T23:37:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.773+0000 7f2ed6a00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2eb806c9c0 0x7f2eb806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.773+0000 7f2ed7e00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7f2eb8076130 0x7f2eb8078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.773+0000 7ff8d6000640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8bc004c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009800 tx=0x7ff8bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87430c9a68d70a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.774+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1b4161f20 con 0x7fe1b40d0d20 2024-04-26T23:37:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.774+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f378c069ac0 con 0x7f378c0c9650 2024-04-26T23:37:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.774+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4037070 con 0x7ff8bc004800 2024-04-26T23:37:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.774+0000 7f2ed6a00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2eb806c9c0 0x7f2eb806ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cc260 tx=0x7f2ec4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.770+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f876801a070 con 0x7f876c057b00 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f8770c00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c940 0x7f875406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f876808b9e0 con 0x7f876c057b00 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f8770c00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7f87540760c0 0x7f8754078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8754078bd0 con 0x7f87540760c0 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.772+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f876808bd90 con 0x7f876c057b00 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.774+0000 7f8779600640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7f87540760c0 0x7f8754078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.775+0000 7f8779600640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7f87540760c0 0x7f8754078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.775+0000 7ff8d5600640 1 -- 172.21.15.46:0/3943997957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c402a9f0 con 0x7ff8bc004800 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7ff8ddc00640 1 -- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 msgr2=0x7ff8bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7ff8ddc00640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8bc004c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009800 tx=0x7ff8bc005830 comp rx=0 tx=0).stop 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7ff8ddc00640 1 -- 172.21.15.46:0/3943997957 shutdown_connections 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7ff8ddc00640 1 --2- 172.21.15.46:0/3943997957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8bc004c80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.775+0000 7f8773e00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c940 0x7f875406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7ff8ddc00640 1 -- 172.21.15.46:0/3943997957 >> 172.21.15.46:0/3943997957 conn(0x7ff8d80d1e20 msgr2=0x7ff8d80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7ff8ddc00640 1 -- 172.21.15.46:0/3943997957 shutdown_connections 2024-04-26T23:37:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7f2ed7e00640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7f2eb8076130 0x7f2eb8078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc114003610 con 0x7fc12015f3b0 2024-04-26T23:37:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== osd.2 v2:172.21.15.46:6808/2486201425 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ec80067f0 con 0x7f2eb8076130 2024-04-26T23:37:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== osd.1 v2:172.21.15.46:6816/3879162742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8754078bd0 con 0x7f87540760c0 2024-04-26T23:37:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.776+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3778026060 con 0x7f378c0c9650 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7f3783400640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f376c06c940 0x7f376c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3778093620 con 0x7f378c0c9650 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f3783400640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f376c0760c0 0x7f376c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f376c078bd0 con 0x7f376c0760c0 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f377809a050 con 0x7f378c0c9650 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f378aa00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f376c06c940 0x7f376c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7f8773e00640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c940 0x7f875406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8760034000 tx=0x7f876002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1ac00ac10 con 0x7fe1b40d0d20 2024-04-26T23:37:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.777+0000 7ff8ddc00640 1 -- 172.21.15.46:0/3943997957 wait complete. 2024-04-26T23:37:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.778+0000 7fc11cc00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc10006c940 0x7fc10006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.779+0000 7f378a000640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f376c0760c0 0x7f376c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.779+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1140accf0 con 0x7fc12015f3b0 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.779+0000 7fc11cc00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7fc1000760c0 0x7fc100078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.779+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc114042ec0 con 0x7fc1000760c0 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.779+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1140ad1d0 con 0x7fc12015f3b0 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.780+0000 7fe1ab400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.780+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1ac08b2a0 con 0x7fe1b40d0d20 2024-04-26T23:37:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.780+0000 7f378aa00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f376c06c940 0x7f376c06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3784001350 tx=0x7f3784038040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.781+0000 7fe1ab400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7fe194076070 0x7fe1940784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.781+0000 7ff8ddc00640 1 Processor -- start 2024-04-26T23:37:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.781+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe194078b80 con 0x7fe194076070 2024-04-26T23:37:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.781+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe1ac090060 con 0x7fe1b40d0d20 2024-04-26T23:37:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.780+0000 7fc125200640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc10006c940 0x7fc10006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.780+0000 7fc125c00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7fc1000760c0 0x7fc100078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.781+0000 7ff8ddc00640 1 -- start start 2024-04-26T23:37:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.782+0000 7fc125200640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc10006c940 0x7fc10006ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc12015e1c0 tx=0x7fc10c0078a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.782+0000 7fe1b2000640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.782+0000 7f378a000640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f376c0760c0 0x7f376c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.782+0000 7fe1b3400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7fe194076070 0x7fe1940784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.783+0000 7fc125c00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7fc1000760c0 0x7fc100078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.783+0000 7ff8ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8d815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.783+0000 7ff8ddc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8d80b5bf0 0x7ff8d80c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.783+0000 7ff8ddc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8d80b7200 0x7ff8d80b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8d815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7fe1b3400640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7fe194076070 0x7fe1940784b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8d815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37578/0 (socket says 172.21.15.46:37578) 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8ddc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d80b13a0 con 0x7ff8bc004800 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8ddc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d80d1470 con 0x7ff8d80b7200 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8ddc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d80b1520 con 0x7ff8d80b5bf0 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 -- 172.21.15.46:0/1391551096 learned_addr learned my addr 172.21.15.46:0/1391551096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7fe1b2000640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0034000 tx=0x7fe1a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 -- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8d80b5bf0 msgr2=0x7ff8d80c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8d80b5bf0 0x7ff8d80c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 -- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8d80b7200 msgr2=0x7ff8d80b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8d80b7200 0x7ff8d80b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.784+0000 7ff8d6a00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d80b6160 con 0x7ff8bc004800 2024-04-26T23:37:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.785+0000 7ff8d6a00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8d815dc50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8011890 tx=0x7ff8c8011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8019070 con 0x7ff8bc004800 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c8004240 con 0x7ff8bc004800 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8009590 con 0x7ff8bc004800 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== osd.6 v2:172.21.15.142:6800/715004401 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f376c078bd0 con 0x7f376c0760c0 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d80b6440 con 0x7ff8bc004800 2024-04-26T23:37:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.786+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d80b6a00 con 0x7ff8bc004800 2024-04-26T23:37:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.787+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== osd.4 v2:172.21.15.142:6824/2432053156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc114042ec0 con 0x7fc1000760c0 2024-04-26T23:37:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.787+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8d80693b0 con 0x7ff8bc004800 2024-04-26T23:37:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.788+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== osd.5 v2:172.21.15.142:6801/82441601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe194078b80 con 0x7fe194076070 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.790+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8c801c070 con 0x7ff8bc004800 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.792+0000 7ff8cea00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff8b806c9f0 0x7ff8b806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.793+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff8c808c710 con 0x7ff8bc004800 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.793+0000 7ff8cea00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7ff8b8076170 0x7ff8b80785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.793+0000 7ff8d6000640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff8b806c9f0 0x7ff8b806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.794+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c8001d30 con 0x7ff8b8076170 2024-04-26T23:37:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.794+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8c808cb00 con 0x7ff8bc004800 2024-04-26T23:37:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.794+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 --> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77680d29f0 con 0x7f77480760c0 2024-04-26T23:37:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.794+0000 7ff8d7400640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7ff8b8076170 0x7ff8b80785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.795+0000 7f7765600640 1 -- 172.21.15.46:0/2514278483 <== osd.0 v2:172.21.15.46:6800/3755207586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f77680d29f0 con 0x7f77480760c0 2024-04-26T23:37:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.796+0000 7ff8d6000640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff8b806c9f0 0x7ff8b806eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40097d0 tx=0x7ff8c402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.796+0000 7ff8d7400640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7ff8b8076170 0x7ff8b80785b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.797+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f77480760c0 msgr2=0x7f7748078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.797+0000 7f7770200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f77480760c0 0x7f7748078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.797+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 --> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6dcc002dd0 con 0x7f6dd80760c0 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.798+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f774806c940 msgr2=0x7f774806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.798+0000 7f7770200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f774806c940 0x7f774806ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f775c005880 tx=0x7f775c009040 comp rx=0 tx=0).stop 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.798+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 msgr2=0x7f776806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.798+0000 7f7770200640 1 --2- 172.21.15.46:0/2514278483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7768166e00 0x7f776806e5f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7760009eb0 tx=0x7f7760004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.798+0000 7f776e200640 1 -- 172.21.15.46:0/2514278483 reap_dead start 2024-04-26T23:37:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 shutdown_connections 2024-04-26T23:37:40.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f6dee000640 1 -- 172.21.15.46:0/2316875537 <== osd.3 v2:172.21.15.46:6824/1706950386 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6dcc002dd0 con 0x7f6dd80760c0 2024-04-26T23:37:40.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 >> 172.21.15.46:0/2514278483 conn(0x7f776809e450 msgr2=0x7f7768166660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 shutdown_connections 2024-04-26T23:37:40.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f7770200640 1 -- 172.21.15.46:0/2514278483 wait complete. 2024-04-26T23:37:40.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== osd.7 v2:172.21.15.142:6816/631850901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8c8001d30 con 0x7ff8b8076170 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7f6dd80760c0 msgr2=0x7f6dd8078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.799+0000 7f6de3400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7f6dd80760c0 0x7f6dd8078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c940 msgr2=0x7f6dd806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c940 0x7f6dd806ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6de8034000 tx=0x7f6de802c000 comp rx=0 tx=0).stop 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 msgr2=0x7f6df4070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 --2- 172.21.15.46:0/2316875537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6df4162860 0x7f6df4070550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c430 tx=0x7f6de400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6dfa800640 1 -- 172.21.15.46:0/2316875537 reap_dead start 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 shutdown_connections 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.800+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 >> 172.21.15.46:0/2316875537 conn(0x7f6df40d4f40 msgr2=0x7f6df40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.803+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 shutdown_connections 2024-04-26T23:37:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.803+0000 7f6de3400640 1 -- 172.21.15.46:0/2316875537 wait complete. 2024-04-26T23:37:40.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.806+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f876c056ca0 con 0x7f87540760c0 2024-04-26T23:37:40.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.807+0000 7f8770c00640 1 -- 172.21.15.46:0/2211000282 <== osd.1 v2:172.21.15.46:6816/3879162742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f876c056ca0 con 0x7f87540760c0 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7f87540760c0 msgr2=0x7f8754078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7f87540760c0 0x7f8754078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c940 msgr2=0x7f875406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c940 0x7f875406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8760034000 tx=0x7f876002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 msgr2=0x7f876c0e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f877a000640 1 --2- 172.21.15.46:0/2211000282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f876c057b00 0x7f876c0e51d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f876800c450 tx=0x7f876800c850 comp rx=0 tx=0).stop 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.810+0000 7f8779600640 1 -- 172.21.15.46:0/2211000282 reap_dead start 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.811+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 shutdown_connections 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.811+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 >> 172.21.15.46:0/2211000282 conn(0x7f876c03aa30 msgr2=0x7f876c038840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.811+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 shutdown_connections 2024-04-26T23:37:40.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.811+0000 7f877a000640 1 -- 172.21.15.46:0/2211000282 wait complete. 2024-04-26T23:37:40.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.816+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ed80693b0 con 0x7f2eb8076130 2024-04-26T23:37:40.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.818+0000 7f2ecf400640 1 -- 172.21.15.46:0/3976204056 <== osd.2 v2:172.21.15.46:6808/2486201425 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2ed80693b0 con 0x7f2eb8076130 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7f2eb8076130 msgr2=0x7f2eb8078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7f2eb8076130 0x7f2eb8078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2eb806c9c0 msgr2=0x7f2eb806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2eb806c9c0 0x7f2eb806ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cc260 tx=0x7f2ec4009040 comp rx=0 tx=0).stop 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 msgr2=0x7f2ed806c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ede200640 1 --2- 172.21.15.46:0/3976204056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8073040 0x7f2ed806c5c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009840 tx=0x7f2ec80046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.819+0000 7f2ed7e00640 1 -- 172.21.15.46:0/3976204056 reap_dead start 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.820+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 shutdown_connections 2024-04-26T23:37:40.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.820+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 >> 172.21.15.46:0/3976204056 conn(0x7f2ed80d1d80 msgr2=0x7f2ed80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.820+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 shutdown_connections 2024-04-26T23:37:40.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.820+0000 7f2ede200640 1 -- 172.21.15.46:0/3976204056 wait complete. 2024-04-26T23:37:40.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.822+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 --> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1200c95f0 con 0x7fc1000760c0 2024-04-26T23:37:40.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.823+0000 7fc11cc00640 1 -- 172.21.15.46:0/2777409140 <== osd.4 v2:172.21.15.142:6824/2432053156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc1200c95f0 con 0x7fc1000760c0 2024-04-26T23:37:40.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.823+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 --> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f378c15a800 con 0x7f376c0760c0 2024-04-26T23:37:40.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7fc1000760c0 msgr2=0x7fc100078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7fc1000760c0 0x7fc100078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 --> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1b4073f40 con 0x7fe194076070 2024-04-26T23:37:40.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7f3783400640 1 -- 172.21.15.46:0/3020061521 <== osd.6 v2:172.21.15.142:6800/715004401 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f377c009a90 con 0x7f376c0760c0 2024-04-26T23:37:40.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc10006c940 msgr2=0x7fc10006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc10006c940 0x7fc10006ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc12015e1c0 tx=0x7fc10c0078a0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 msgr2=0x7fc1200c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.824+0000 7fc127c00640 1 --2- 172.21.15.46:0/2777409140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015f3b0 0x7fc1200c9810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc114035040 tx=0x7fc1140046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fc125c00640 1 -- 172.21.15.46:0/2777409140 reap_dead start 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 shutdown_connections 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 >> 172.21.15.46:0/2777409140 conn(0x7fc12009e450 msgr2=0x7fc1200d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fe1ab400640 1 -- 172.21.15.46:0/3545978733 <== osd.5 v2:172.21.15.142:6801/82441601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe1b4073f40 con 0x7fe194076070 2024-04-26T23:37:40.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 shutdown_connections 2024-04-26T23:37:40.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.825+0000 7fc127c00640 1 -- 172.21.15.46:0/2777409140 wait complete. 2024-04-26T23:37:40.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f376c0760c0 msgr2=0x7f376c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7f3791e00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f376c0760c0 0x7f376c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7fe194076070 msgr2=0x7fe1940784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7fe194076070 0x7fe1940784b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe19406c8f0 msgr2=0x7fe19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0034000 tx=0x7fe1a002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 msgr2=0x7fe1b4153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/3545978733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1b40d0d20 0x7fe1b4153280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c780 tx=0x7fe1ac00cc40 comp rx=0 tx=0).stop 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b3400640 1 -- 172.21.15.46:0/3545978733 reap_dead start 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 shutdown_connections 2024-04-26T23:37:40.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.826+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 >> 172.21.15.46:0/3545978733 conn(0x7fe1b409e450 msgr2=0x7fe1b40b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.843 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 shutdown_connections 2024-04-26T23:37:40.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T23:37:40.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7fe1b9e00640 1 -- 172.21.15.46:0/3545978733 wait complete. 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f376c06c940 msgr2=0x7f376c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f376c06c940 0x7f376c06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3784001350 tx=0x7f3784038040 comp rx=0 tx=0).stop 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 msgr2=0x7f378c0cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 --2- 172.21.15.46:0/3020061521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f378c0c9650 0x7f378c0cb520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3784009000 tx=0x7f377800e7e0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f378b400640 1 -- 172.21.15.46:0/3020061521 reap_dead start 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 shutdown_connections 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.827+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 >> 172.21.15.46:0/3020061521 conn(0x7f378c0d5110 msgr2=0x7f378c0b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.828+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 shutdown_connections 2024-04-26T23:37:40.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.828+0000 7f3791e00640 1 -- 172.21.15.46:0/3020061521 wait complete. 2024-04-26T23:37:40.846 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T23:37:40.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.830+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 --> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8d806d0a0 con 0x7ff8b8076170 2024-04-26T23:37:40.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.831+0000 7ff8cea00640 1 -- 172.21.15.46:0/1391551096 <== osd.7 v2:172.21.15.142:6816/631850901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff8d806d0a0 con 0x7ff8b8076170 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.831+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7ff8b8076170 msgr2=0x7ff8b80785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.831+0000 7ff8ddc00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7ff8b8076170 0x7ff8b80785b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff8b806c9f0 msgr2=0x7ff8b806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff8b806c9f0 0x7ff8b806eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40097d0 tx=0x7ff8c402c000 comp rx=0 tx=0).stop 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 msgr2=0x7ff8d815dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 --2- 172.21.15.46:0/1391551096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc004800 0x7ff8d815dc50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8011890 tx=0x7ff8c8011d50 comp rx=0 tx=0).stop 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8d7400640 1 -- 172.21.15.46:0/1391551096 reap_dead start 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 shutdown_connections 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 >> 172.21.15.46:0/1391551096 conn(0x7ff8d80d1e20 msgr2=0x7ff8d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 shutdown_connections 2024-04-26T23:37:40.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:40.834+0000 7ff8ddc00640 1 -- 172.21.15.46:0/1391551096 wait complete. 2024-04-26T23:37:40.849 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T23:37:40.850 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T23:37:40.854 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T23:37:40.860 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:40.860 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:40.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T23:37:40.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T23:37:40.862 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:40.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T23:37:41.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.126+0000 7f0bc8200640 1 Processor -- start 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.126+0000 7f0bc8200640 1 -- start start 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.128+0000 7f0bc8200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0bb80e3150 0x7f0bb80e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.128+0000 7f0bc8200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb8053ca0 0x7f0bb8054130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.128+0000 7f0bc8200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb8054670 con 0x7f0bb803c090 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.128+0000 7f0bc8200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb8007b60 con 0x7f0bb80e3150 2024-04-26T23:37:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.128+0000 7f0bc8200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb8058800 con 0x7f0bb8053ca0 2024-04-26T23:37:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.129+0000 7f0bc5800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0bb80e3150 0x7f0bb80e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.129+0000 7f0bc4e00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0bb803c090 0x7f0bb8058040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46834/0 (socket says 172.21.15.46:46834) 2024-04-26T23:37:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.129+0000 7f0bc4e00640 1 -- 172.21.15.46:0/1988130165 learned_addr learned my addr 172.21.15.46:0/1988130165 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.129+0000 7f0bc6200640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb8053ca0 0x7f0bb8054130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.130+0000 7f0bbfe00640 1 -- 172.21.15.46:0/1988130165 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086755594 0 0) 0x7f0bb8054670 con 0x7f0bb803c090 2024-04-26T23:37:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.130+0000 7f0bc5800640 1 -- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb8053ca0 msgr2=0x7f0bb8054130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.130+0000 7f0bc5800640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb8053ca0 0x7f0bb8054130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.130+0000 7f0bc5800640 1 -- 172.21.15.46:0/1988130165 >> v1:172.21.15.142:6789/0 conn(0x7f0bb803c090 legacy=0x7f0bb8058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.130+0000 7f0bc5800640 1 -- 172.21.15.46:0/1988130165 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb8058ae0 con 0x7f0bb80e3150 2024-04-26T23:37:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.132+0000 7f0bc5800640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb80e3150 0x7f0bb80e7770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00097a0 tx=0x7f0bb002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75401bd332c70627 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.133+0000 7f0bbfe00640 1 -- 172.21.15.46:0/1988130165 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0037070 con 0x7f0bb80e3150 2024-04-26T23:37:41.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.133+0000 7f0bbfe00640 1 -- 172.21.15.46:0/1988130165 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb002ae50 con 0x7f0bb80e3150 2024-04-26T23:37:41.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.133+0000 7f0bbfe00640 1 -- 172.21.15.46:0/1988130165 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0037070 con 0x7f0bb80e3150 2024-04-26T23:37:41.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.134+0000 7f0bc8200640 1 -- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb80e3150 msgr2=0x7f0bb80e7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.135+0000 7f0bc8200640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb80e3150 0x7f0bb80e7770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00097a0 tx=0x7f0bb002a100 comp rx=0 tx=0).stop 2024-04-26T23:37:41.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.135+0000 7f0bc8200640 1 -- 172.21.15.46:0/1988130165 shutdown_connections 2024-04-26T23:37:41.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.135+0000 7f0bc8200640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb8053ca0 0x7f0bb8054130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.135+0000 7f0bc8200640 1 --2- 172.21.15.46:0/1988130165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb80e3150 0x7f0bb80e7770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.135+0000 7f0bc8200640 1 -- 172.21.15.46:0/1988130165 >> 172.21.15.46:0/1988130165 conn(0x7f0bb8058ed0 msgr2=0x7f0bb80592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.136+0000 7f0bc8200640 1 -- 172.21.15.46:0/1988130165 shutdown_connections 2024-04-26T23:37:41.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.136+0000 7f0bc8200640 1 -- 172.21.15.46:0/1988130165 wait complete. 2024-04-26T23:37:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.139+0000 7f0bc8200640 1 Processor -- start 2024-04-26T23:37:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.139+0000 7f0bc8200640 1 -- start start 2024-04-26T23:37:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.140+0000 7f965d200640 1 Processor -- start 2024-04-26T23:37:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.140+0000 7f9c1ea00640 1 Processor -- start 2024-04-26T23:37:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.140+0000 7f965d200640 1 -- start start 2024-04-26T23:37:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.140+0000 7f9c1ea00640 1 -- start start 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 0x7f0bb80e1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8053ca0 0x7f0bb80d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80e3530 con 0x7f0bb80e3150 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb8006630 con 0x7f0bb8053ca0 2024-04-26T23:37:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.141+0000 7f0bc8200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb8006b50 con 0x7f0bb803c090 2024-04-26T23:37:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f965d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96580b4f90 0x7f96581601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f965d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9658160a20 0x7f9658160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f965d200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9658160720 con 0x7f965806f6f0 2024-04-26T23:37:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f965d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f965806fb90 con 0x7f96580b4f90 2024-04-26T23:37:41.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f965d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f965806fe70 con 0x7f9658160a20 2024-04-26T23:37:41.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f9655e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96580b4f90 0x7f96581601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 0x7f0bb80e1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37588/0 (socket says 172.21.15.46:37588) 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 0x7f0bb80e1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54622/0 (socket says 172.21.15.46:54622) 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 -- 172.21.15.46:0/3486812518 learned_addr learned my addr 172.21.15.46:0/3486812518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 -- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 msgr2=0x7f0bb80e1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 0x7f0bb80e1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 -- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8053ca0 msgr2=0x7f0bb80d74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8053ca0 0x7f0bb80d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.142+0000 7f0bc6200640 1 -- 172.21.15.46:0/3486812518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb804d250 con 0x7f0bb80e3150 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f0bc6200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006bcb0 tx=0x7f0bc00744a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0097480 con 0x7f0bb80e3150 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0097620 con 0x7f0bb80e3150 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0097a10 con 0x7f0bb80e3150 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f9655e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96580b4f90 0x7f96581601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37596/0 (socket says 172.21.15.46:37596) 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f9655e00640 1 -- 172.21.15.46:0/3082629697 learned_addr learned my addr 172.21.15.46:0/3082629697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f9656800640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9658160a20 0x7f9658160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.143+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb804d530 con 0x7f0bb80e3150 2024-04-26T23:37:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.144+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb804daf0 con 0x7f0bb80e3150 2024-04-26T23:37:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.145+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c005290 con 0x7f0bb80e3150 2024-04-26T23:37:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.145+0000 7f9c1ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c1815f4e0 0x7f9c180d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.144+0000 7f9655e00640 1 -- 172.21.15.46:0/3082629697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9658160a20 msgr2=0x7f9658160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.144+0000 7f9655e00640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9658160a20 0x7f9658160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.144+0000 7f9655e00640 1 -- 172.21.15.46:0/3082629697 >> v1:172.21.15.142:6789/0 conn(0x7f965806f6f0 legacy=0x7f96580b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.144+0000 7f9655e00640 1 -- 172.21.15.46:0/3082629697 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9658070150 con 0x7f96580b4f90 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 0x7f9c180d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c180d11f0 con 0x7f9c180d1f60 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9c180d2990 con 0x7f9c1815f4e0 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9c180d5f20 con 0x7f9c180b4000 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c1815f4e0 0x7f9c180d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 0x7f9c180d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c1815f4e0 0x7f9c180d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37606/0 (socket says 172.21.15.46:37606) 2024-04-26T23:37:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9c1d600640 1 -- 172.21.15.46:0/2625465226 learned_addr learned my addr 172.21.15.46:0/2625465226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.147+0000 7f9c1d600640 1 -- 172.21.15.46:0/2625465226 >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 msgr2=0x7f9c180d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.147+0000 7f9c17e00640 1 -- 172.21.15.46:0/2625465226 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545400861 0 0) 0x7f9c180d11f0 con 0x7f9c180d1f60 2024-04-26T23:37:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.146+0000 7f9655e00640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96580b4f90 0x7f96581601e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9644009870 tx=0x7f964402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13c16ef1b1e44c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.147+0000 7f9654a00640 1 -- 172.21.15.46:0/3082629697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644037070 con 0x7f96580b4f90 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.147+0000 7f9654a00640 1 -- 172.21.15.46:0/3082629697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9644032440 con 0x7f96580b4f90 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.147+0000 7f9654a00640 1 -- 172.21.15.46:0/3082629697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644032810 con 0x7f96580b4f90 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 -- 172.21.15.46:0/3082629697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96580b4f90 msgr2=0x7f96581601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f222d000640 1 Processor -- start 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.149+0000 7f222d000640 1 -- start start 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96580b4f90 0x7f96581601e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9644009870 tx=0x7f964402a210 comp rx=0 tx=0).stop 2024-04-26T23:37:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 -- 172.21.15.46:0/3082629697 shutdown_connections 2024-04-26T23:37:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9658160a20 0x7f9658160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 --2- 172.21.15.46:0/3082629697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96580b4f90 0x7f96581601e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.148+0000 7f965d200640 1 -- 172.21.15.46:0/3082629697 >> 172.21.15.46:0/3082629697 conn(0x7f965806cbd0 msgr2=0x7f96580b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.149+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bc006f450 con 0x7f0bb80e3150 2024-04-26T23:37:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.149+0000 7f084ca00640 1 Processor -- start 2024-04-26T23:37:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.149+0000 7f9c1d600640 1 --2- 172.21.15.46:0/2625465226 >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 0x7f9c180d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.149+0000 7f9c1d600640 1 -- 172.21.15.46:0/2625465226 >> v1:172.21.15.142:6789/0 conn(0x7f9c180d1f60 legacy=0x7f9c180cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f9c1d600640 1 -- 172.21.15.46:0/2625465226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c180d0f10 con 0x7f9c1815f4e0 2024-04-26T23:37:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f9c1cc00640 1 --2- 172.21.15.46:0/2625465226 >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 0x7f9c180d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f222d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 0x7f22280d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f222d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f222d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22280d11f0 con 0x7f222815f4e0 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f222d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f22280d0ce0 con 0x7f22280d2130 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.150+0000 7f222d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22280d2990 con 0x7f22280b4000 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.151+0000 7f084ca00640 1 -- start start 2024-04-26T23:37:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f2226a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 0x7f22280d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f0bbd600640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 0x7f0ba406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f2226a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 0x7f22280d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37612/0 (socket says 172.21.15.46:37612) 2024-04-26T23:37:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0bc0108840 con 0x7f0bb80e3150 2024-04-26T23:37:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f2226a00640 1 -- 172.21.15.46:0/1998634701 learned_addr learned my addr 172.21.15.46:0/1998634701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.152+0000 7f2227e00640 1 --2- 172.21.15.46:0/1998634701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f965d200640 1 -- 172.21.15.46:0/3082629697 shutdown_connections 2024-04-26T23:37:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f965d200640 1 -- 172.21.15.46:0/3082629697 wait complete. 2024-04-26T23:37:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f9c1d600640 1 --2- 172.21.15.46:0/2625465226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180d2df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c1808d880 tx=0x7f9c0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0151a23727d1eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f0bc5800640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 0x7f0ba406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f084ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b5140 0x7f08480d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f084ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08480d5410 0x7f08480d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f084ca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480cfd60 con 0x7f084815f4b0 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f084ca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f084815f890 con 0x7f08480d5410 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f2227e00640 1 -- 172.21.15.46:0/1998634701 >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 msgr2=0x7f22280d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f2227e00640 1 --2- 172.21.15.46:0/1998634701 >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 0x7f22280d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f2227e00640 1 -- 172.21.15.46:0/1998634701 >> v1:172.21.15.46:6789/0 conn(0x7f222815f4e0 legacy=0x7f22280d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.153+0000 7f2227e00640 1 -- 172.21.15.46:0/1998634701 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280d5f20 con 0x7f22280b4000 2024-04-26T23:37:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f084ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08480d58c0 con 0x7f08480b5140 2024-04-26T23:37:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f0845e00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f084815f4b0 0x7f08480d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46866/0 (socket says 172.21.15.46:46866) 2024-04-26T23:37:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.154+0000 7f0845e00640 1 -- 172.21.15.46:0/3665431746 learned_addr learned my addr 172.21.15.46:0/3665431746 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.155+0000 7f0845400640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b5140 0x7f08480d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.155+0000 7f0846800640 1 --2- 172.21.15.46:0/3665431746 >> v2:172.21.15.142:3300/0 conn(0x7f08480d5410 0x7f08480d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.155+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127426527 0 0) 0x7f0840001490 con 0x7f084815f4b0 2024-04-26T23:37:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.155+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f082c003520 con 0x7f084815f4b0 2024-04-26T23:37:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.155+0000 7f0bc5800640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 0x7f0ba406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0034000 tx=0x7f0bb002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 999119373 0 0) 0x7f082c003520 con 0x7f084815f4b0 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b5140 msgr2=0x7f08480d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b5140 0x7f08480d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 >> v2:172.21.15.142:3300/0 conn(0x7f08480d5410 msgr2=0x7f08480d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 --2- 172.21.15.46:0/3665431746 >> v2:172.21.15.142:3300/0 conn(0x7f08480d5410 0x7f08480d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480b55d0 con 0x7f084815f4b0 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0840001bc0 con 0x7f084815f4b0 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 >> v1:172.21.15.142:6789/0 conn(0x7f084815f4b0 legacy=0x7f08480d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082c0039b0 0x7f082c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f082c004800 0x7f082c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f082c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f084815f890 con 0x7f082c0051c0 2024-04-26T23:37:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08480d58c0 con 0x7f082c0039b0 2024-04-26T23:37:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f082c00bf00 con 0x7f082c004800 2024-04-26T23:37:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.156+0000 7f2227e00640 1 --2- 172.21.15.46:0/1998634701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280b2280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22180097a0 tx=0x7f221802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4673dea9a872b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f082c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0845400640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f082c004800 0x7f082c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 -- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f082c004800 msgr2=0x7f082c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f082c004800 0x7f082c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 -- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082c0039b0 msgr2=0x7f082c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082c0039b0 0x7f082c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.157+0000 7f0846800640 1 -- 172.21.15.46:0/3665431746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c003520 con 0x7f082c0051c0 2024-04-26T23:37:41.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f965d200640 1 Processor -- start 2024-04-26T23:37:41.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f2226000640 1 -- 172.21.15.46:0/1998634701 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218037070 con 0x7f22280b4000 2024-04-26T23:37:41.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7fc409200640 1 Processor -- start 2024-04-26T23:37:41.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f965d200640 1 -- start start 2024-04-26T23:37:41.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f37cbe00640 1 Processor -- start 2024-04-26T23:37:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f37cbe00640 1 -- start start 2024-04-26T23:37:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f2226000640 1 -- 172.21.15.46:0/1998634701 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2218032440 con 0x7f22280b4000 2024-04-26T23:37:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.158+0000 7f2226000640 1 -- 172.21.15.46:0/1998634701 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218032810 con 0x7f22280b4000 2024-04-26T23:37:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f222d000640 1 -- 172.21.15.46:0/1998634701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 msgr2=0x7f22280b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f0846800640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f082c00b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ad0 tx=0x7f082c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a23466ebb73703 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f222d000640 1 --2- 172.21.15.46:0/1998634701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280b2280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22180097a0 tx=0x7f221802a180 comp rx=0 tx=0).stop 2024-04-26T23:37:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7fc409200640 1 -- start start 2024-04-26T23:37:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f9c17e00640 1 -- 172.21.15.46:0/2625465226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04037070 con 0x7f9c1815f4e0 2024-04-26T23:37:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f9c17e00640 1 -- 172.21.15.46:0/2625465226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04032440 con 0x7f9c1815f4e0 2024-04-26T23:37:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f9c17e00640 1 -- 172.21.15.46:0/2625465226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04032810 con 0x7f9c1815f4e0 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.159+0000 7f965d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 0x7f9658144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f965d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 0x7f96581f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f965d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 0x7f9658073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f965d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96580b0330 con 0x7f9658160a20 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f965d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9658070710 con 0x7f96580b4f90 2024-04-26T23:37:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f965d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9658148b50 con 0x7f965806f6f0 2024-04-26T23:37:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830036070 con 0x7f082c0051c0 2024-04-26T23:37:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f37cbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37cc162a80 0x7f37cc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f0844a00640 1 -- 172.21.15.46:0/3665431746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0830031400 con 0x7f082c0051c0 2024-04-26T23:37:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f37cbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37cc0b6600 con 0x7f37cc0d5ab0 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f37cbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f37cc0d1300 con 0x7f37cc162a80 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f37cbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0d0610 con 0x7f37cc0b4e40 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f9655e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 0x7f9658144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f222d000640 1 -- 172.21.15.46:0/1998634701 shutdown_connections 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f222d000640 1 --2- 172.21.15.46:0/1998634701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280b2280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f222d000640 1 --2- 172.21.15.46:0/1998634701 >> v2:172.21.15.46:3300/0 conn(0x7f22280d2130 0x7f22280d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f222d000640 1 -- 172.21.15.46:0/1998634701 >> 172.21.15.46:0/1998634701 conn(0x7f22280d5010 msgr2=0x7f22280d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f9655e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 0x7f9658144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54674/0 (socket says 172.21.15.46:54674) 2024-04-26T23:37:41.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.160+0000 7f9655e00640 1 -- 172.21.15.46:0/735295803 learned_addr learned my addr 172.21.15.46:0/735295803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9656800640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 0x7f9658073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f222d000640 1 -- 172.21.15.46:0/1998634701 shutdown_connections 2024-04-26T23:37:41.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f37cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 -- 172.21.15.46:0/2625465226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 msgr2=0x7f9c180d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/2625465226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180d2df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c1808d880 tx=0x7f9c0402a270 comp rx=0 tx=0).stop 2024-04-26T23:37:41.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bc00d62c0 con 0x7f0bb80e3150 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 -- 172.21.15.46:0/2625465226 shutdown_connections 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/2625465226 >> v2:172.21.15.142:3300/0 conn(0x7f9c180b4000 0x7f9c180d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f37cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54676/0 (socket says 172.21.15.46:54676) 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/2625465226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180d2df0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 -- 172.21.15.46:0/2625465226 >> 172.21.15.46:0/2625465226 conn(0x7f9c180d5010 msgr2=0x7f9c180d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f084ca00640 1 -- 172.21.15.46:0/3665431746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 msgr2=0x7f082c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f084ca00640 1 --2- 172.21.15.46:0/3665431746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f082c00b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ad0 tx=0x7f082c005830 comp rx=0 tx=0).stop 2024-04-26T23:37:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f37cb400640 1 -- 172.21.15.46:0/1156395886 learned_addr learned my addr 172.21.15.46:0/1156395886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9655400640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 0x7f96581f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f0846800640 1 -- 172.21.15.46:0/3665431746 reap_dead start 2024-04-26T23:37:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f9c1ea00640 1 -- 172.21.15.46:0/2625465226 shutdown_connections 2024-04-26T23:37:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.161+0000 7f222d000640 1 -- 172.21.15.46:0/1998634701 wait complete. 2024-04-26T23:37:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f084ca00640 1 -- 172.21.15.46:0/3665431746 shutdown_connections 2024-04-26T23:37:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f084ca00640 1 -- 172.21.15.46:0/3665431746 >> 172.21.15.46:0/3665431746 conn(0x7f084808eb50 msgr2=0x7f084809e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9c1ea00640 1 -- 172.21.15.46:0/2625465226 wait complete. 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 -- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 msgr2=0x7f9658144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 0x7f9658144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 -- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 msgr2=0x7f96581f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 0x7f96581f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 -- 172.21.15.46:0/735295803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96581f9a30 con 0x7f9658160a20 2024-04-26T23:37:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9655400640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 0x7f96581f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f084ca00640 1 -- 172.21.15.46:0/3665431746 shutdown_connections 2024-04-26T23:37:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f9656800640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 0x7f9658073240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9650007cf0 tx=0x7f965000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f084ca00640 1 -- 172.21.15.46:0/3665431746 wait complete. 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f5c95e00640 1 Processor -- start 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f5c95e00640 1 -- start start 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7fc409200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc40415f500 0x7fc4040d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7fc409200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 0x7fc4040d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7fc409200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040d2a70 con 0x7fc4040d56c0 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7fc409200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc40415f8e0 con 0x7fc40415f500 2024-04-26T23:37:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7fc409200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4040b5e80 con 0x7fc4040b3fb0 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f37caa00640 1 --2- 172.21.15.46:0/1156395886 >> v2:172.21.15.46:3300/0 conn(0x7f37cc162a80 0x7f37cc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f37c9600640 1 -- 172.21.15.46:0/1156395886 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447638790 0 0) 0x7f37cc0b6600 con 0x7f37cc0d5ab0 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.162+0000 7f37c9600640 1 -- 172.21.15.46:0/1156395886 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37b0003520 con 0x7f37cc0d5ab0 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f37cb400640 1 -- 172.21.15.46:0/1156395886 >> v2:172.21.15.46:3300/0 conn(0x7f37cc162a80 msgr2=0x7f37cc0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f37cb400640 1 --2- 172.21.15.46:0/1156395886 >> v2:172.21.15.46:3300/0 conn(0x7f37cc162a80 0x7f37cc0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f37cb400640 1 -- 172.21.15.46:0/1156395886 >> v1:172.21.15.46:6789/0 conn(0x7f37cc0d5ab0 legacy=0x7f37cc0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.163+0000 7f37cb400640 1 -- 172.21.15.46:0/1156395886 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc0d1550 con 0x7f37cc0b4e40 2024-04-26T23:37:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.164+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000edd0 con 0x7f9658160a20 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.164+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581f9d10 con 0x7f9658160a20 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.164+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96580aca00 con 0x7f9658160a20 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650003990 con 0x7f9658160a20 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650003d80 con 0x7f9658160a20 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f222d000640 1 Processor -- start 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f222d000640 1 -- start start 2024-04-26T23:37:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f084ca00640 1 Processor -- start 2024-04-26T23:37:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f37cb400640 1 --2- 172.21.15.46:0/1156395886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009e50 tx=0x7f37c402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33be698b650d9d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f9c1ea00640 1 Processor -- start 2024-04-26T23:37:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f37c9600640 1 -- 172.21.15.46:0/1156395886 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4036070 con 0x7f37cc0b4e40 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7f5c95e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c880543c0 0x7f5c880e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c88058810 con 0x7f5c880e3160 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c88058af0 con 0x7f5c8803c0a0 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5c88007b70 con 0x7f5c880543c0 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f9c1ea00640 1 -- start start 2024-04-26T23:37:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f084ca00640 1 -- start start 2024-04-26T23:37:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c880543c0 0x7f5c880e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c880543c0 0x7f5c880e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59272/0 (socket says 172.21.15.46:59272) 2024-04-26T23:37:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7f5c95400640 1 -- 172.21.15.46:0/283260320 learned_addr learned my addr 172.21.15.46:0/283260320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f37c9600640 1 -- 172.21.15.46:0/1156395886 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c402ae50 con 0x7f37cc0b4e40 2024-04-26T23:37:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f37c9600640 1 -- 172.21.15.46:0/1156395886 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4031820 con 0x7f37cc0b4e40 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.164+0000 7fc401e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc40415f500 0x7fc4040d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7fc402800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc4040d56c0 0x7fc404162f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46868/0 (socket says 172.21.15.46:46868) 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7fc402800640 1 -- 172.21.15.46:0/687958623 learned_addr learned my addr 172.21.15.46:0/687958623 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.165+0000 7fc401400640 1 --2- 172.21.15.46:0/687958623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 0x7fc4040d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc400a00640 1 -- 172.21.15.46:0/687958623 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487467607 0 0) 0x7fc4040d2a70 con 0x7fc4040d56c0 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc400a00640 1 -- 172.21.15.46:0/687958623 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3e800bd30 con 0x7fc4040d56c0 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc401400640 1 -- 172.21.15.46:0/687958623 >> v2:172.21.15.46:3300/0 conn(0x7fc40415f500 msgr2=0x7fc4040d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc401400640 1 --2- 172.21.15.46:0/687958623 >> v2:172.21.15.46:3300/0 conn(0x7fc40415f500 0x7fc4040d2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc401400640 1 -- 172.21.15.46:0/687958623 >> v1:172.21.15.142:6789/0 conn(0x7fc4040d56c0 legacy=0x7fc404162f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.166+0000 7fc401400640 1 -- 172.21.15.46:0/687958623 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040d2f60 con 0x7fc4040b3fb0 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c180b4000 0x7f9c18154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1815f8c0 con 0x7f9c1815f4e0 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c180b1d60 con 0x7f9c180b4000 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f9c1ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9c180b3290 con 0x7f9c180d1f60 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f5c95400640 1 -- 172.21.15.46:0/283260320 >> v1:172.21.15.142:6789/0 conn(0x7f5c8803c0a0 legacy=0x7f5c88053cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f5c95400640 1 -- 172.21.15.46:0/283260320 >> v1:172.21.15.46:6789/0 conn(0x7f5c880e3160 legacy=0x7f5c88058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:41.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.167+0000 7f5c95400640 1 -- 172.21.15.46:0/283260320 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c88006360 con 0x7f5c880543c0 2024-04-26T23:37:41.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 -- 172.21.15.46:0/1156395886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 msgr2=0x7f37cc0d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 --2- 172.21.15.46:0/1156395886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009e50 tx=0x7f37c402a630 comp rx=0 tx=0).stop 2024-04-26T23:37:41.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f084ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f084815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 -- 172.21.15.46:0/1156395886 shutdown_connections 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 --2- 172.21.15.46:0/1156395886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc0b4e40 0x7f37cc0d19d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 --2- 172.21.15.46:0/1156395886 >> v2:172.21.15.46:3300/0 conn(0x7f37cc162a80 0x7f37cc0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f37cbe00640 1 -- 172.21.15.46:0/1156395886 >> 172.21.15.46:0/1156395886 conn(0x7f37cc0d4fc0 msgr2=0x7f37cc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f9c1cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f9c1cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54696/0 (socket says 172.21.15.46:54696) 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f9c1cc00640 1 -- 172.21.15.46:0/3633385967 learned_addr learned my addr 172.21.15.46:0/3633385967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f9c1d600640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c180b4000 0x7f9c18154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.168+0000 7f9c1e000640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f5c95400640 1 --2- 172.21.15.46:0/283260320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880543c0 0x7f5c880e7780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c9006b1d0 tx=0x7f5c900a68e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a7c7eb45ceaf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 0x7f22281f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f9c1cc00640 1 -- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c180b4000 msgr2=0x7f9c18154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f9c1cc00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c180b4000 0x7f9c18154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f9c1cc00640 1 -- 172.21.15.46:0/3633385967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 msgr2=0x7f9c180b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 0x7f22280ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22280b1d60 con 0x7f222815f4e0 2024-04-26T23:37:41.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22280d5c40 con 0x7f22280d2130 2024-04-26T23:37:41.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f222d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f222815f8c0 con 0x7f22280b4000 2024-04-26T23:37:41.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7fc401400640 1 --2- 172.21.15.46:0/687958623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 0x7fc4040d5180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097a0 tx=0x7fc3f002a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c2f73484df1fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f084ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 0x7f08480cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f5c8f400640 1 -- 172.21.15.46:0/283260320 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900a9070 con 0x7f5c880543c0 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f2227e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 0x7f22280ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f2227400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f2227e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 0x7f22280ba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37656/0 (socket says 172.21.15.46:37656) 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f2227e00640 1 -- 172.21.15.46:0/4005484498 learned_addr learned my addr 172.21.15.46:0/4005484498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f5c8f400640 1 -- 172.21.15.46:0/283260320 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c900ac6a0 con 0x7f5c880543c0 2024-04-26T23:37:41.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7fc400a00640 1 -- 172.21.15.46:0/687958623 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0037070 con 0x7fc4040b3fb0 2024-04-26T23:37:41.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f084ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 0x7f08480ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f5c8f400640 1 -- 172.21.15.46:0/283260320 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900acb30 con 0x7f5c880543c0 2024-04-26T23:37:41.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7fc400a00640 1 -- 172.21.15.46:0/687958623 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0032440 con 0x7fc4040b3fb0 2024-04-26T23:37:41.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7fc400a00640 1 -- 172.21.15.46:0/687958623 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f002ae60 con 0x7fc4040b3fb0 2024-04-26T23:37:41.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f084ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08480b5e70 con 0x7f082c0051c0 2024-04-26T23:37:41.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f9c1cc00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f9c1cc00640 1 -- 172.21.15.46:0/3633385967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c180b4730 con 0x7f9c180d1f60 2024-04-26T23:37:41.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f0845400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 0x7f08480cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f0845400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 0x7f08480cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54724/0 (socket says 172.21.15.46:54724) 2024-04-26T23:37:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f0845400640 1 -- 172.21.15.46:0/2380645935 learned_addr learned my addr 172.21.15.46:0/2380645935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f37cbe00640 1 -- 172.21.15.46:0/1156395886 shutdown_connections 2024-04-26T23:37:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f084ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08480b5b90 con 0x7f08480ce630 2024-04-26T23:37:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f5c95e00640 1 -- 172.21.15.46:0/283260320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880543c0 msgr2=0x7f5c880e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f5c95e00640 1 --2- 172.21.15.46:0/283260320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880543c0 0x7f5c880e7780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c9006b1d0 tx=0x7f5c900a68e0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f0846800640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 0x7f08480ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08480b4680 con 0x7f08480b26e0 2024-04-26T23:37:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f37cbe00640 1 -- 172.21.15.46:0/1156395886 wait complete. 2024-04-26T23:37:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.170+0000 7f2226a00640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 0x7f22281f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f0845e00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f084815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2226a00640 1 -- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 msgr2=0x7f22280cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2226a00640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2226a00640 1 -- 172.21.15.46:0/4005484498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 msgr2=0x7f22280ba2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2226a00640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 0x7f22280ba2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2226a00640 1 -- 172.21.15.46:0/4005484498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222815e6a0 con 0x7f22280d2130 2024-04-26T23:37:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f2227400640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:41.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7fc409200640 1 -- 172.21.15.46:0/687958623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 msgr2=0x7fc4040d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f5c95e00640 1 -- 172.21.15.46:0/283260320 shutdown_connections 2024-04-26T23:37:41.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f5c95e00640 1 --2- 172.21.15.46:0/283260320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880543c0 0x7f5c880e7780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f5c95e00640 1 -- 172.21.15.46:0/283260320 >> 172.21.15.46:0/283260320 conn(0x7f5c88058ee0 msgr2=0x7f5c880592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7fc409200640 1 --2- 172.21.15.46:0/687958623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 0x7fc4040d5180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097a0 tx=0x7fc3f002a150 comp rx=0 tx=0).stop 2024-04-26T23:37:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f5c95e00640 1 -- 172.21.15.46:0/283260320 shutdown_connections 2024-04-26T23:37:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f5c95e00640 1 -- 172.21.15.46:0/283260320 wait complete. 2024-04-26T23:37:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7fc409200640 1 -- 172.21.15.46:0/687958623 shutdown_connections 2024-04-26T23:37:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7fc409200640 1 --2- 172.21.15.46:0/687958623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040b3fb0 0x7fc4040d5180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7fc409200640 1 --2- 172.21.15.46:0/687958623 >> v2:172.21.15.46:3300/0 conn(0x7fc40415f500 0x7fc4040d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7fc409200640 1 -- 172.21.15.46:0/687958623 >> 172.21.15.46:0/687958623 conn(0x7fc404074400 msgr2=0x7fc40409e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0846800640 1 -- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 msgr2=0x7f08480cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0846800640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 0x7f08480cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0846800640 1 -- 172.21.15.46:0/2380645935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 msgr2=0x7f084815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f2226a00640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 0x7f22281f9270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2220007b20 tx=0x7f222000d0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0846800640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f084815d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0846800640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480b2b70 con 0x7f08480ce630 2024-04-26T23:37:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f9c1e000640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f0845e00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f084815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7fc409200640 1 -- 172.21.15.46:0/687958623 shutdown_connections 2024-04-26T23:37:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222001a070 con 0x7f22280d2130 2024-04-26T23:37:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222000dd10 con 0x7f22280d2130 2024-04-26T23:37:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220015600 con 0x7f22280d2130 2024-04-26T23:37:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22280cd7a0 con 0x7f22280d2130 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.174+0000 7fc409200640 1 -- 172.21.15.46:0/687958623 wait complete. 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.174+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22280cdd60 con 0x7f22280d2130 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.169+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9630005290 con 0x7f9658160a20 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.171+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f965001b020 con 0x7f9658160a20 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.172+0000 7f964e000640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 0x7f963806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.173+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9650003b40 con 0x7f9658160a20 2024-04-26T23:37:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.174+0000 7f9655e00640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 0x7f963806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.174+0000 7f9c1cc00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000cf60 tx=0x7f9c1000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1001a070 con 0x7f9c180d1f60 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1000d840 con 0x7f9c180d1f60 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000dc10 con 0x7f9c180d1f60 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c180b4a10 con 0x7f9c180d1f60 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c180b4fd0 con 0x7f9c180d1f60 2024-04-26T23:37:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f0846800640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 0x7f08480ceae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0830031000 tx=0x7f0830004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f37cbe00640 1 Processor -- start 2024-04-26T23:37:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f37cbe00640 1 -- start start 2024-04-26T23:37:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f5c95e00640 1 Processor -- start 2024-04-26T23:37:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f5c95e00640 1 -- start start 2024-04-26T23:37:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22280693b0 con 0x7f22280d2130 2024-04-26T23:37:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830036040 con 0x7f08480ce630 2024-04-26T23:37:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.175+0000 7f9655e00640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 0x7f963806efb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96440090b0 tx=0x7f964402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 0x7f5c880d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 0x7f5c880e5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 0x7f5c8803ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880ee050 con 0x7f5c880543c0 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880e3540 con 0x7f5c880e3160 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f5c95e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c880ee1d0 con 0x7f5c8803c0a0 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 0x7f5c880d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7fc409200640 1 Processor -- start 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc0d5ab0 0x7f37cc0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0830043070 con 0x7f08480ce630 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.176+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083003c870 con 0x7f08480ce630 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08480b2e50 con 0x7f08480ce630 2024-04-26T23:37:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08480b3410 con 0x7f08480ce630 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7fc409200640 1 -- start start 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 0x7f37cc0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0b5430 con 0x7f37cc0b4e40 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0b52b0 con 0x7f37cc0d5ab0 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f37cbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0d1f10 con 0x7f37cc162a80 2024-04-26T23:37:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 0x7f5c880d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54740/0 (socket says 172.21.15.46:54740) 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f5c94a00640 1 -- 172.21.15.46:0/1246918350 learned_addr learned my addr 172.21.15.46:0/1246918350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f5c95400640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 0x7f5c8803ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.177+0000 7f5c8fe00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 0x7f5c880e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 0x7f37cc0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37676/0 (socket says 172.21.15.46:37676) 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 -- 172.21.15.46:0/961141545 learned_addr learned my addr 172.21.15.46:0/961141545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 0x7f37cc0cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:54756/0 (socket says 172.21.15.46:54756) 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37ca000640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc0d5ab0 0x7f37cc0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c180693b0 con 0x7f9c180d1f60 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f222001d070 con 0x7f22280d2130 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f5c8fe00640 1 -- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 msgr2=0x7f5c880d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f5c8fe00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 0x7f5c880d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f5c8fe00640 1 -- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 msgr2=0x7f5c8803ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f5c8fe00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 0x7f5c8803ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.178+0000 7f5c8fe00640 1 -- 172.21.15.46:0/1246918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c880e5cd0 con 0x7f5c880543c0 2024-04-26T23:37:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f5c95400640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 0x7f5c8803ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 -- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 msgr2=0x7f37cc0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 0x7f37cc0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 -- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc0d5ab0 msgr2=0x7f37cc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.179+0000 7f37caa00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc0d5ab0 0x7f37cc0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.180+0000 7f37caa00640 1 -- 172.21.15.46:0/961141545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1f9380 con 0x7f37cc0b4e40 2024-04-26T23:37:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.180+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c1001d070 con 0x7f9c180d1f60 2024-04-26T23:37:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.180+0000 7f221f400640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 0x7f220806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.180+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f222008bde0 con 0x7f22280d2130 2024-04-26T23:37:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7f5c8fe00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 0x7f5c880e5360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c3c0 tx=0x7f5c7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7f2227400640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 0x7f220806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7f37caa00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00d5f0 tx=0x7f37bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7fc409200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7fc409200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040d56c0 0x7fc4040d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7fc401e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7f9c15600640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 0x7f9bf806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9c1008c670 con 0x7f9c180d1f60 2024-04-26T23:37:41.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7fc409200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40415f500 0x7fc40414f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.181+0000 7fc401e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37680/0 (socket says 172.21.15.46:37680) 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc409200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040d3800 con 0x7fc4040b3fb0 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc409200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040d3520 con 0x7fc40415f500 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc409200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4040d3ae0 con 0x7fc4040d56c0 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 -- 172.21.15.46:0/3076728116 learned_addr learned my addr 172.21.15.46:0/3076728116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401400640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040d56c0 0x7fc4040d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7f9c1d600640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 0x7f9bf806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7f2227400640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 0x7f220806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2214001ad0 tx=0x7f2214009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 -- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040d56c0 msgr2=0x7fc4040d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0848069530 con 0x7f08480ce630 2024-04-26T23:37:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040d56c0 0x7fc4040d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 -- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40415f500 msgr2=0x7fc40414f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40415f500 0x7fc40414f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.182+0000 7fc401e00640 1 -- 172.21.15.46:0/3076728116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404162780 con 0x7fc4040b3fb0 2024-04-26T23:37:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.183+0000 7fc401e00640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc007e70 tx=0x7fc3fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.184+0000 7f9c1d600640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 0x7f9bf806ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c0402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c017070 con 0x7f5c880543c0 2024-04-26T23:37:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.184+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f083003ca10 con 0x7f08480ce630 2024-04-26T23:37:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c880e5fb0 con 0x7f5c880543c0 2024-04-26T23:37:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c880e6570 con 0x7f5c880543c0 2024-04-26T23:37:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9650059db0 con 0x7f9658160a20 2024-04-26T23:37:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.184+0000 7f083e000640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 0x7f082406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f08300ac140 con 0x7f08480ce630 2024-04-26T23:37:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0125f0 con 0x7f37cc0b4e40 2024-04-26T23:37:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc014620 con 0x7fc4040b3fb0 2024-04-26T23:37:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4041504c0 con 0x7fc4040b3fb0 2024-04-26T23:37:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc0b7c80 con 0x7f37cc0b4e40 2024-04-26T23:37:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f0845e00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 0x7f082406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc0b8240 con 0x7f37cc0b4e40 2024-04-26T23:37:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc404150d60 con 0x7fc4040b3fb0 2024-04-26T23:37:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.186+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc019070 con 0x7f37cc0b4e40 2024-04-26T23:37:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.186+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc012990 con 0x7f37cc0b4e40 2024-04-26T23:37:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.186+0000 7f0845e00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 0x7f082406ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0840002230 tx=0x7f0840002160 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.186+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc019070 con 0x7fc4040b3fb0 2024-04-26T23:37:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.187+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc003b90 con 0x7fc4040b3fb0 2024-04-26T23:37:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0052d0 con 0x7f5c880543c0 2024-04-26T23:37:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.185+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c021060 con 0x7f5c880543c0 2024-04-26T23:37:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.187+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c005290 con 0x7f5c880543c0 2024-04-26T23:37:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.188+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a4005290 con 0x7f37cc0b4e40 2024-04-26T23:37:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.188+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc005290 con 0x7fc4040b3fb0 2024-04-26T23:37:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.189+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c7c01a070 con 0x7f5c880543c0 2024-04-26T23:37:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.190+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37bc012b30 con 0x7f37cc0b4e40 2024-04-26T23:37:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.191+0000 7f5c8cc00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 0x7f5c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.191+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5c7c08b7f0 con 0x7f5c880543c0 2024-04-26T23:37:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.191+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3fc0147c0 con 0x7fc4040b3fb0 2024-04-26T23:37:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.191+0000 7f37c2a00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.192+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f37bc08cad0 con 0x7f37cc0b4e40 2024-04-26T23:37:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.192+0000 7f5c94a00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 0x7f5c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.192+0000 7f37ca000640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.192+0000 7fc3fa000640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 0x7fc3e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.193+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc3fc08d2a0 con 0x7fc4040b3fb0 2024-04-26T23:37:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.193+0000 7f5c94a00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 0x7f5c7006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c840099a0 tx=0x7f5c840090e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.193+0000 7fc401400640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 0x7fc3e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.193+0000 7f37ca000640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 0x7f37ac06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37b8007fd0 tx=0x7f37b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.194+0000 7fc401400640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 0x7fc3e406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0034000 tx=0x7fc3f002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.199+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c7c059400 con 0x7f5c880543c0 2024-04-26T23:37:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.199+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2220010070 con 0x7f22280d2130 2024-04-26T23:37:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.199+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c1005a180 con 0x7f9c180d1f60 2024-04-26T23:37:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.205+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37bc08ce20 con 0x7f37cc0b4e40 2024-04-26T23:37:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.212+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f083004c020 con 0x7f08480ce630 2024-04-26T23:37:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.215+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3fc023020 con 0x7fc4040b3fb0 2024-04-26T23:37:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.605+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0b9c005580 con 0x7f0bb80e3150 2024-04-26T23:37:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.606+0000 7f0bbd600640 1 -- 172.21.15.46:0/3486812518 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0bc00da5c0 con 0x7f0bb80e3150 2024-04-26T23:37:41.610 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.610+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 msgr2=0x7f0ba406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.610+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 0x7f0ba406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0034000 tx=0x7f0bb002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.610+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 msgr2=0x7f0bb804ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.610+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006bcb0 tx=0x7f0bc00744a0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 shutdown_connections 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba406c9c0 0x7f0ba406ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80e3150 0x7f0bb804ca00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8053ca0 0x7f0bb80d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 --2- 172.21.15.46:0/3486812518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bb803c090 0x7f0bb80e1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 >> 172.21.15.46:0/3486812518 conn(0x7f0bb8058ed0 msgr2=0x7f0bb8039be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 shutdown_connections 2024-04-26T23:37:41.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.611+0000 7f0bc8200640 1 -- 172.21.15.46:0/3486812518 wait complete. 2024-04-26T23:37:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.631+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9630005850 con 0x7f9658160a20 2024-04-26T23:37:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.632+0000 7f964e000640 1 -- 172.21.15.46:0/735295803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96500596b0 con 0x7f9658160a20 2024-04-26T23:37:41.638 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 msgr2=0x7f963806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 0x7f963806efb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96440090b0 tx=0x7f964402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 msgr2=0x7f9658073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 0x7f9658073240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9650007cf0 tx=0x7f965000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 shutdown_connections 2024-04-26T23:37:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f963806caf0 0x7f963806efb0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96440090b0 tx=0x7f964402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9658160a20 0x7f9658073240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96580b4f90 0x7f96581f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 --2- 172.21.15.46:0/735295803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f965806f6f0 0x7f9658144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 >> 172.21.15.46:0/735295803 conn(0x7f965806cbd0 msgr2=0x7f9658160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9c180d33d0 con 0x7f9c180d1f60 2024-04-26T23:37:41.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.636+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 shutdown_connections 2024-04-26T23:37:41.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.637+0000 7f965d200640 1 -- 172.21.15.46:0/735295803 wait complete. 2024-04-26T23:37:41.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.637+0000 7f9c15600640 1 -- 172.21.15.46:0/3633385967 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c1005e480 con 0x7f9c180d1f60 2024-04-26T23:37:41.644 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-26T23:37:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.640+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 msgr2=0x7f9bf806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.640+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 0x7f9bf806ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c0402e000 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 msgr2=0x7f9c180b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000cf60 tx=0x7f9c1000cf90 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 shutdown_connections 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf806c9c0 0x7f9bf806ee80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c1815f4e0 0x7f9c180b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c180d1f60 0x7f9c180b57d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 --2- 172.21.15.46:0/3633385967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c180b4000 0x7f9c18154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 >> 172.21.15.46:0/3633385967 conn(0x7f9c180d5010 msgr2=0x7f9c180d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 shutdown_connections 2024-04-26T23:37:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.641+0000 7f9c1ea00640 1 -- 172.21.15.46:0/3633385967 wait complete. 2024-04-26T23:37:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.648+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f22280d33d0 con 0x7f22280d2130 2024-04-26T23:37:41.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.649+0000 7f221f400640 1 -- 172.21.15.46:0/4005484498 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2220059860 con 0x7f22280d2130 2024-04-26T23:37:41.654 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.651+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5c6c005850 con 0x7f5c880543c0 2024-04-26T23:37:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.651+0000 7f5c8cc00640 1 -- 172.21.15.46:0/1246918350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c7c05d700 con 0x7f5c880543c0 2024-04-26T23:37:41.655 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2024-04-26T23:37:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.652+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 msgr2=0x7f220806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.652+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 0x7f220806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2214001ad0 tx=0x7f2214009040 comp rx=0 tx=0).stop 2024-04-26T23:37:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.652+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 msgr2=0x7f22281f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.652+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 0x7f22281f9270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2220007b20 tx=0x7f222000d0d0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 shutdown_connections 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c9c0 0x7f220806ee80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222815f4e0 0x7f22280ba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22280d2130 0x7f22281f9270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 --2- 172.21.15.46:0/4005484498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280b4000 0x7f22280cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 >> 172.21.15.46:0/4005484498 conn(0x7f22280d5010 msgr2=0x7f222806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 shutdown_connections 2024-04-26T23:37:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.653+0000 7f222d000640 1 -- 172.21.15.46:0/4005484498 wait complete. 2024-04-26T23:37:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.654+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 msgr2=0x7f5c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.654+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 0x7f5c7006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c840099a0 tx=0x7f5c840090e0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.654+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 msgr2=0x7f5c880e5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.654+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 0x7f5c880e5360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c3c0 tx=0x7f5c7c00c880 comp rx=0 tx=0).stop 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 shutdown_connections 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7006c8f0 0x7f5c7006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c880e3160 0x7f5c8803ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c880543c0 0x7f5c880e5360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 --2- 172.21.15.46:0/1246918350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c8803c0a0 0x7f5c880d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 >> 172.21.15.46:0/1246918350 conn(0x7f5c88058ee0 msgr2=0x7f5c880db2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 shutdown_connections 2024-04-26T23:37:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.655+0000 7f5c95e00640 1 -- 172.21.15.46:0/1246918350 wait complete. 2024-04-26T23:37:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.659+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f37a4005850 con 0x7f37cc0b4e40 2024-04-26T23:37:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.660+0000 7f37c2a00640 1 -- 172.21.15.46:0/961141545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37bc05a650 con 0x7f37cc0b4e40 2024-04-26T23:37:41.664 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.662+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc3dc005850 con 0x7fc4040b3fb0 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.663+0000 7fc3fa000640 1 -- 172.21.15.46:0/3076728116 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3fc05b380 con 0x7fc4040b3fb0 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 msgr2=0x7f37ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 0x7f37ac06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37b8007fd0 tx=0x7f37b8009040 comp rx=0 tx=0).stop 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 msgr2=0x7f37cc0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00d5f0 tx=0x7f37bc00dab0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 shutdown_connections 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc162a80 0x7f37cc0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc0d5ab0 0x7f37cc0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 --2- 172.21.15.46:0/961141545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0b4e40 0x7f37cc0ce530 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 >> 172.21.15.46:0/961141545 conn(0x7f37cc0d4fc0 msgr2=0x7f37cc06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 shutdown_connections 2024-04-26T23:37:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.664+0000 7f37cbe00640 1 -- 172.21.15.46:0/961141545 wait complete. 2024-04-26T23:37:41.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.666+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 msgr2=0x7fc3e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.666+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 0x7fc3e406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0034000 tx=0x7fc3f002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.666+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 msgr2=0x7fc404162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.666+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc007e70 tx=0x7fc3fc00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 shutdown_connections 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e406c8f0 0x7fc3e406edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40415f500 0x7fc40414f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4040d56c0 0x7fc4040d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 --2- 172.21.15.46:0/3076728116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4040b3fb0 0x7fc404162240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 >> 172.21.15.46:0/3076728116 conn(0x7fc404074400 msgr2=0x7fc4040b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 shutdown_connections 2024-04-26T23:37:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f08480b4bd0 con 0x7f08480ce630 2024-04-26T23:37:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.667+0000 7fc409200640 1 -- 172.21.15.46:0/3076728116 wait complete. 2024-04-26T23:37:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.668+0000 7f083e000640 1 -- 172.21.15.46:0/2380645935 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0830079bd0 con 0x7f08480ce630 2024-04-26T23:37:41.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-26T23:37:41.673 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 msgr2=0x7f082406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 0x7f082406ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0840002230 tx=0x7f0840002160 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 msgr2=0x7f08480ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 0x7f08480ceae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0830031000 tx=0x7f0830004730 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 shutdown_connections 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f082406c920 0x7f082406ede0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08480ce630 0x7f08480ceae0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08480b26e0 0x7f08480cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 --2- 172.21.15.46:0/2380645935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0051c0 0x7f084815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.671+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 >> 172.21.15.46:0/2380645935 conn(0x7f084808eb50 msgr2=0x7f08480cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.672+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 shutdown_connections 2024-04-26T23:37:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:41.672+0000 7f084ca00640 1 -- 172.21.15.46:0/2380645935 wait complete. 2024-04-26T23:37:41.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-26T23:37:41.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-04-26T23:37:41.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-26T23:37:41.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-26T23:37:41.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-26T23:37:42.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T23:37:42.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T23:37:42.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T23:37:42.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T23:37:42.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T23:37:42.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T23:37:42.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T23:37:42.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T23:37:42.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.931+0000 7f37c1800640 1 Processor -- start 2024-04-26T23:37:42.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.931+0000 7f37c1800640 1 -- start start 2024-04-26T23:37:42.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.934+0000 7f37c1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37b40e30b0 0x7f37b4057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.935+0000 7f37c1800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37b40542f0 0x7f37b40068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.935+0000 7f37c1800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b40582c0 con 0x7f37b403e830 2024-04-26T23:37:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.936+0000 7f37c1800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f37b4058560 con 0x7f37b40e30b0 2024-04-26T23:37:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.936+0000 7f37c1800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37b4058840 con 0x7f37b40542f0 2024-04-26T23:37:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.937+0000 7f37baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37b40e30b0 0x7f37b4057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.937+0000 7f37ba000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f37b403e830 0x7f37b4053be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39824/0 (socket says 172.21.15.46:39824) 2024-04-26T23:37:42.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.937+0000 7f37ba000640 1 -- 172.21.15.46:0/3306619694 learned_addr learned my addr 172.21.15.46:0/3306619694 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:42.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.937+0000 7f37bb400640 1 --2- 172.21.15.46:0/3306619694 >> v2:172.21.15.142:3300/0 conn(0x7f37b40542f0 0x7f37b40068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.939+0000 7f37b9600640 1 -- 172.21.15.46:0/3306619694 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697179916 0 0) 0x7f37b40582c0 con 0x7f37b403e830 2024-04-26T23:37:42.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.939+0000 7f37b9600640 1 -- 172.21.15.46:0/3306619694 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a0003520 con 0x7f37b403e830 2024-04-26T23:37:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.940+0000 7f37baa00640 1 -- 172.21.15.46:0/3306619694 >> v2:172.21.15.142:3300/0 conn(0x7f37b40542f0 msgr2=0x7f37b40068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.940+0000 7f37baa00640 1 --2- 172.21.15.46:0/3306619694 >> v2:172.21.15.142:3300/0 conn(0x7f37b40542f0 0x7f37b40068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.940+0000 7f37baa00640 1 -- 172.21.15.46:0/3306619694 >> v1:172.21.15.46:6789/0 conn(0x7f37b403e830 legacy=0x7f37b4053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.940+0000 7f37baa00640 1 -- 172.21.15.46:0/3306619694 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4058b20 con 0x7f37b40e30b0 2024-04-26T23:37:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.942+0000 7f37baa00640 1 --2- 172.21.15.46:0/3306619694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b40e30b0 0x7f37b4057d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37b403be50 tx=0x7f37a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1224e72b20f627b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.943+0000 7f37b9600640 1 -- 172.21.15.46:0/3306619694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8037070 con 0x7f37b40e30b0 2024-04-26T23:37:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.943+0000 7f37b9600640 1 -- 172.21.15.46:0/3306619694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a8032440 con 0x7f37b40e30b0 2024-04-26T23:37:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.943+0000 7f37b9600640 1 -- 172.21.15.46:0/3306619694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8032810 con 0x7f37b40e30b0 2024-04-26T23:37:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.944+0000 7f37c1800640 1 -- 172.21.15.46:0/3306619694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b40e30b0 msgr2=0x7f37b4057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.944+0000 7f37c1800640 1 --2- 172.21.15.46:0/3306619694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b40e30b0 0x7f37b4057d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37b403be50 tx=0x7f37a802a270 comp rx=0 tx=0).stop 2024-04-26T23:37:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.945+0000 7f37c1800640 1 -- 172.21.15.46:0/3306619694 shutdown_connections 2024-04-26T23:37:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.945+0000 7f37c1800640 1 --2- 172.21.15.46:0/3306619694 >> v2:172.21.15.142:3300/0 conn(0x7f37b40542f0 0x7f37b40068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.945+0000 7f37c1800640 1 --2- 172.21.15.46:0/3306619694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b40e30b0 0x7f37b4057d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.945+0000 7f37c1800640 1 -- 172.21.15.46:0/3306619694 >> 172.21.15.46:0/3306619694 conn(0x7f37b40559d0 msgr2=0x7f37b4055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:42.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.946+0000 7f37c1800640 1 -- 172.21.15.46:0/3306619694 shutdown_connections 2024-04-26T23:37:42.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.946+0000 7f37c1800640 1 -- 172.21.15.46:0/3306619694 wait complete. 2024-04-26T23:37:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.950+0000 7f37c1800640 1 Processor -- start 2024-04-26T23:37:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.950+0000 7f37c1800640 1 -- start start 2024-04-26T23:37:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.952+0000 7f37c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 0x7f37b4179720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.953+0000 7f37c1800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.953+0000 7f37c1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b40e30b0 0x7f37b40399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37c1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37b40e3490 con 0x7f37b403e830 2024-04-26T23:37:42.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37ba000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37ba000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60498/0 (socket says 172.21.15.46:60498) 2024-04-26T23:37:42.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37ba000640 1 -- 172.21.15.46:0/1991216311 learned_addr learned my addr 172.21.15.46:0/1991216311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:42.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37bb400640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b40e30b0 0x7f37b40399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.954+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37b403bfb0 con 0x7f37b40e30b0 2024-04-26T23:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.955+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37b4006e00 con 0x7f37b40542f0 2024-04-26T23:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.955+0000 7f37baa00640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 0x7f37b4179720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.956+0000 7f37ba000640 1 -- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b40e30b0 msgr2=0x7f37b40399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.956+0000 7f37ba000640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b40e30b0 0x7f37b40399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.956+0000 7f37ba000640 1 -- 172.21.15.46:0/1991216311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 msgr2=0x7f37b4179720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.957+0000 7f37ba000640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 0x7f37b4179720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.957+0000 7f37ba000640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4179c60 con 0x7f37b40542f0 2024-04-26T23:37:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.957+0000 7f37baa00640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 0x7f37b4179720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.958+0000 7f37ba000640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37bc063ef0 tx=0x7f37bc0707c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.959+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc050c30 con 0x7f37b40542f0 2024-04-26T23:37:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.959+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc0515b0 con 0x7f37b40542f0 2024-04-26T23:37:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.959+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc074370 con 0x7f37b40542f0 2024-04-26T23:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.959+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b40e4c00 con 0x7f37b40542f0 2024-04-26T23:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.961+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b40e51c0 con 0x7f37b40542f0 2024-04-26T23:37:42.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.963+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b40057d0 con 0x7f37b40542f0 2024-04-26T23:37:42.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.965+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37bc095030 con 0x7f37b40542f0 2024-04-26T23:37:42.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.966+0000 7f37b2a00640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 0x7f379c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.967+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f37bc107650 con 0x7f37b40542f0 2024-04-26T23:37:42.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.967+0000 7f37baa00640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 0x7f379c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.969+0000 7f37baa00640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 0x7f379c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a8034000 tx=0x7f37a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.980+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37bc09c020 con 0x7f37b40542f0 2024-04-26T23:37:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.983+0000 7f995ae00640 1 Processor -- start 2024-04-26T23:37:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.984+0000 7f995ae00640 1 -- start start 2024-04-26T23:37:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.987+0000 7f9959a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f99540d1910 0x7f99540d1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39832/0 (socket says 172.21.15.46:39832) 2024-04-26T23:37:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.988+0000 7f8f5de00640 1 Processor -- start 2024-04-26T23:37:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.987+0000 7f995ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99540b98e0 0x7f99540d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.988+0000 7f995ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99540b7e60 con 0x7f99540d1910 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.988+0000 7f995ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99540d0c70 con 0x7f99540d0740 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.988+0000 7f995ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f99540b8440 con 0x7f99540b98e0 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.987+0000 7f9959000640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f99540d0740 0x7f9954163320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47038/0 (socket says 172.21.15.46:47038) 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.988+0000 7f9959000640 1 -- 172.21.15.46:0/4130651868 learned_addr learned my addr 172.21.15.46:0/4130651868 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.989+0000 7f8f5de00640 1 -- start start 2024-04-26T23:37:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.989+0000 7effdb200640 1 Processor -- start 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.989+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2750023960 0 0) 0x7f99540d0c70 con 0x7f99540d0740 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.989+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f993c003520 con 0x7f99540d0740 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.990+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3509059094 0 0) 0x7f993c003520 con 0x7f99540d0740 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.990+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 >> v2:172.21.15.46:3300/0 conn(0x7f99540b98e0 msgr2=0x7f99540d2400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.990+0000 7f995a400640 1 --2- 172.21.15.46:0/4130651868 >> v2:172.21.15.46:3300/0 conn(0x7f99540b98e0 0x7f99540d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.989+0000 7effdb200640 1 -- start start 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 --2- 172.21.15.46:0/4130651868 >> v2:172.21.15.46:3300/0 conn(0x7f99540b98e0 0x7f99540d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 >> v1:172.21.15.46:6789/0 conn(0x7f99540d1910 legacy=0x7f99540d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540b80d0 con 0x7f99540d0740 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9948001670 con 0x7f99540d0740 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 >> v1:172.21.15.142:6789/0 conn(0x7f99540d0740 legacy=0x7f9954163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 0x7f993c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c004800 0x7f993c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0053c0 0x7f993c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7effdb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effdc15b060 0x7effdc0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7effdb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7effdc0ce6f0 0x7effdc072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7effdb200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effdc0ced80 con 0x7effdc15ec90 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7effdb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7effdc0ceb60 con 0x7effdc15b060 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7effdb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7effdc0cf060 con 0x7effdc0ce6f0 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f993c014a10 con 0x7f993c003a00 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f993c014bc0 con 0x7f993c004800 2024-04-26T23:37:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.991+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f993c014dc0 con 0x7f993c0053c0 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effdc15b060 0x7effdc0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effdc15b060 0x7effdc0cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39228/0 (socket says 172.21.15.46:39228) 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 -- 172.21.15.46:0/3576104776 learned_addr learned my addr 172.21.15.46:0/3576104776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9400640 1 --2- 172.21.15.46:0/3576104776 >> v2:172.21.15.142:3300/0 conn(0x7effdc0ce6f0 0x7effdc072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7f8f5de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f5806e510 0x7f8f5806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 0x7f8f58161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f8f5de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f5806ee30 con 0x7f8f5815d540 2024-04-26T23:37:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f8f5de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8f580742d0 con 0x7f8f5806e510 2024-04-26T23:37:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f8f5de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8f58161940 con 0x7f8f58160bd0 2024-04-26T23:37:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7f9959a00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 0x7f993c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f995a400640 1 --2- 172.21.15.46:0/4130651868 >> v2:172.21.15.46:3300/0 conn(0x7f99540b98e0 0x7f99540d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 -- 172.21.15.46:0/3576104776 >> v2:172.21.15.142:3300/0 conn(0x7effdc0ce6f0 msgr2=0x7effdc072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 --2- 172.21.15.46:0/3576104776 >> v2:172.21.15.142:3300/0 conn(0x7effdc0ce6f0 0x7effdc072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 -- 172.21.15.46:0/3576104776 >> v1:172.21.15.142:6789/0 conn(0x7effdc15ec90 legacy=0x7effdc06e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.992+0000 7effd9e00640 1 -- 172.21.15.46:0/3576104776 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc0cf340 con 0x7effdc15b060 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.993+0000 7f9959000640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c004800 0x7f993c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7f8f57400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8f5815d540 0x7f8f5815d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47060/0 (socket says 172.21.15.46:47060) 2024-04-26T23:37:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7f8f57400640 1 -- 172.21.15.46:0/4226874766 learned_addr learned my addr 172.21.15.46:0/4226874766 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7f8f56a00640 1 --2- 172.21.15.46:0/4226874766 >> v2:172.21.15.46:3300/0 conn(0x7f8f5806e510 0x7f8f5806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7f995a400640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0053c0 0x7f993c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7effd9e00640 1 --2- 172.21.15.46:0/3576104776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc15b060 0x7effdc0cfe20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effdc0cd500 tx=0x7effcc02c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd2b5c6b1ee233d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7f8f4e000640 1 --2- 172.21.15.46:0/4226874766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 0x7f8f58161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.994+0000 7effd8a00640 1 -- 172.21.15.46:0/3576104776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc008be0 con 0x7effdc15b060 2024-04-26T23:37:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7effd8a00640 1 -- 172.21.15.46:0/3576104776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc008d80 con 0x7effdc15b060 2024-04-26T23:37:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7effd8a00640 1 -- 172.21.15.46:0/3576104776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc00a2f0 con 0x7effdc15b060 2024-04-26T23:37:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f56000640 1 -- 172.21.15.46:0/4226874766 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056500501 0 0) 0x7f8f5806ee30 con 0x7f8f5815d540 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f56000640 1 -- 172.21.15.46:0/4226874766 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f3c003520 con 0x7f8f5815d540 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f4e000640 1 -- 172.21.15.46:0/4226874766 >> v2:172.21.15.46:3300/0 conn(0x7f8f5806e510 msgr2=0x7f8f5806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f9959a00640 1 -- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0053c0 msgr2=0x7f993c0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f4e000640 1 --2- 172.21.15.46:0/4226874766 >> v2:172.21.15.46:3300/0 conn(0x7f8f5806e510 0x7f8f5806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f4e000640 1 -- 172.21.15.46:0/4226874766 >> v1:172.21.15.142:6789/0 conn(0x7f8f5815d540 legacy=0x7f8f5815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f8f4e000640 1 -- 172.21.15.46:0/4226874766 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f58071f80 con 0x7f8f58160bd0 2024-04-26T23:37:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f0526a00640 1 Processor -- start 2024-04-26T23:37:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.995+0000 7f0526a00640 1 -- start start 2024-04-26T23:37:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.996+0000 7f9959a00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0053c0 0x7f993c0141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.996+0000 7effdb200640 1 -- 172.21.15.46:0/3576104776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc15b060 msgr2=0x7effdc0cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.996+0000 7effdb200640 1 --2- 172.21.15.46:0/3576104776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc15b060 0x7effdc0cfe20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effdc0cd500 tx=0x7effcc02c3c0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.996+0000 7f9959a00640 1 -- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c004800 msgr2=0x7f993c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7f9959a00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c004800 0x7f993c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 -- 172.21.15.46:0/3576104776 shutdown_connections 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 --2- 172.21.15.46:0/3576104776 >> v2:172.21.15.142:3300/0 conn(0x7effdc0ce6f0 0x7effdc072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 --2- 172.21.15.46:0/3576104776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc15b060 0x7effdc0cfe20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 -- 172.21.15.46:0/3576104776 >> 172.21.15.46:0/3576104776 conn(0x7effdc0d0b20 msgr2=0x7effdc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 -- 172.21.15.46:0/3576104776 shutdown_connections 2024-04-26T23:37:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7f9959a00640 1 -- 172.21.15.46:0/4130651868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c003520 con 0x7f993c003a00 2024-04-26T23:37:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7f8f4e000640 1 --2- 172.21.15.46:0/4226874766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 0x7f8f58161030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009d90 tx=0x7f8f4802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc7d9a4f426c822 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7feb86800640 1 Processor -- start 2024-04-26T23:37:43.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f0526a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f0520170530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f8f56000640 1 -- 172.21.15.46:0/4226874766 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48007740 con 0x7f8f58160bd0 2024-04-26T23:37:43.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f8f56000640 1 -- 172.21.15.46:0/4226874766 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f480078e0 con 0x7f8f58160bd0 2024-04-26T23:37:43.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f8f56000640 1 -- 172.21.15.46:0/4226874766 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48007c90 con 0x7f8f58160bd0 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f051fe00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f052006f660 0x7f0520071f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47064/0 (socket says 172.21.15.46:47064) 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f051fe00640 1 -- 172.21.15.46:0/1352573568 learned_addr learned my addr 172.21.15.46:0/1352573568 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f051f400640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f0520170530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7f995a400640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0053c0 0x7f993c0141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f995a400640 1 -- 172.21.15.46:0/4130651868 reap_dead start 2024-04-26T23:37:43.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f9959000640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c004800 0x7f993c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.999+0000 7f9959a00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 0x7f993c003e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9940001750 tx=0x7f993c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379da9db417f284c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f3b5ec00640 1 Processor -- start 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7f3b5ec00640 1 -- start start 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.998+0000 7feb86800640 1 -- start start 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.999+0000 7feb86800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb800b5530 0x7feb800ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb86800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb800d0c90 con 0x7feb800ce8e0 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb86800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb800d0f30 con 0x7feb8009ba70 2024-04-26T23:37:43.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb86800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feb800b59c0 con 0x7feb800b5530 2024-04-26T23:37:43.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:42.997+0000 7effdb200640 1 -- 172.21.15.46:0/3576104776 wait complete. 2024-04-26T23:37:43.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7effdb200640 1 Processor -- start 2024-04-26T23:37:43.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb7fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feb800ce8e0 0x7feb8009be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39846/0 (socket says 172.21.15.46:39846) 2024-04-26T23:37:43.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb7fe00640 1 -- 172.21.15.46:0/2071193158 learned_addr learned my addr 172.21.15.46:0/2071193158 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:43.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7effdb200640 1 -- start start 2024-04-26T23:37:43.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0520072770 con 0x7f0520073bc0 2024-04-26T23:37:43.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f3b5ec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b580d08d0 0x7f3b580b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f3b5ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b5815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f3b5ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b5815c4f0 con 0x7f3b580d04f0 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f3b5ec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3b580b4090 con 0x7f3b580d08d0 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f3b5ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580b4370 con 0x7f3b5815f470 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0520072a50 con 0x7f052006f660 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400013a0 con 0x7f993c003a00 2024-04-26T23:37:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.000+0000 7feb7ea00640 1 --2- 172.21.15.46:0/2071193158 >> v2:172.21.15.142:3300/0 conn(0x7feb800b5530 0x7feb800ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0520073450 con 0x7f05200cd750 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994002e070 con 0x7f993c003a00 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f9953e00640 1 -- 172.21.15.46:0/4130651868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994002ad80 con 0x7f993c003a00 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f3b4f400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b580d08d0 0x7f3b580b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f3b4f400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b580d08d0 0x7f3b580b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35622/0 (socket says 172.21.15.46:35622) 2024-04-26T23:37:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f3b4f400640 1 -- 172.21.15.46:0/2796598304 learned_addr learned my addr 172.21.15.46:0/2796598304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f3b5cc00640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b5815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212073225 0 0) 0x7f0520072770 con 0x7f0520073bc0 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0504003420 con 0x7f0520073bc0 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140856696 0 0) 0x7f0520072a50 con 0x7f052006f660 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.001+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0520072770 con 0x7f052006f660 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7feb7ea00640 1 -- 172.21.15.46:0/2071193158 >> v1:172.21.15.142:6789/0 conn(0x7feb8009ba70 legacy=0x7feb800b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7feb7ea00640 1 -- 172.21.15.46:0/2071193158 >> v1:172.21.15.46:6789/0 conn(0x7feb800ce8e0 legacy=0x7feb8009be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7feb7e000640 1 -- 172.21.15.46:0/2071193158 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39921225 0 0) 0x7feb800d0f30 con 0x7feb8009ba70 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7feb7ea00640 1 -- 172.21.15.46:0/2071193158 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800cdba0 con 0x7feb800b5530 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7fbf6d800640 1 Processor -- start 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f3b4f400640 1 -- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 msgr2=0x7f3b5815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f3b4f400640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b5815aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f3b4f400640 1 -- 172.21.15.46:0/2796598304 >> v1:172.21.15.46:6789/0 conn(0x7f3b580d04f0 legacy=0x7f3b5815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f995ae00640 1 -- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 msgr2=0x7f993c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f995ae00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 0x7f993c003e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9940001750 tx=0x7f993c005830 comp rx=0 tx=0).stop 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f995ae00640 1 -- 172.21.15.46:0/4130651868 shutdown_connections 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f995ae00640 1 --2- 172.21.15.46:0/4130651868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c003a00 0x7f993c003e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f995ae00640 1 -- 172.21.15.46:0/4130651868 >> 172.21.15.46:0/4130651868 conn(0x7f99540d4f70 msgr2=0x7f99540d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f995ae00640 1 -- 172.21.15.46:0/4130651868 shutdown_connections 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f995ae00640 1 -- 172.21.15.46:0/4130651868 wait complete. 2024-04-26T23:37:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7effdb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7effdb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 0x7effdc0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7effd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7effdb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 0x7effdc0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7effd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39264/0 (socket says 172.21.15.46:39264) 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effd9e00640 1 -- 172.21.15.46:0/918767709 learned_addr learned my addr 172.21.15.46:0/918767709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effdc073010 con 0x7effdc0ce6f0 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7effdc0cf900 con 0x7effdc15ec90 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7effdc15b440 con 0x7effdc15b060 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effd9400640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 0x7effdc0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effda800640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 0x7effdc0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7effd9e00640 1 -- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 msgr2=0x7effdc0c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f8f5de00640 1 -- 172.21.15.46:0/4226874766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 msgr2=0x7f8f58161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f8f5de00640 1 --2- 172.21.15.46:0/4226874766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 0x7f8f58161030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009d90 tx=0x7f8f4802a660 comp rx=0 tx=0).stop 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f8f5de00640 1 -- 172.21.15.46:0/4226874766 shutdown_connections 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f8f5de00640 1 --2- 172.21.15.46:0/4226874766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f58160bd0 0x7f8f58161030 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009d90 tx=0x7f8f4802a660 comp rx=0 tx=0).stop 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f8f5de00640 1 --2- 172.21.15.46:0/4226874766 >> v2:172.21.15.46:3300/0 conn(0x7f8f5806e510 0x7f8f5806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f8f5de00640 1 -- 172.21.15.46:0/4226874766 >> 172.21.15.46:0/4226874766 conn(0x7f8f5806c530 msgr2=0x7f8f5806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f8f5de00640 1 -- 172.21.15.46:0/4226874766 shutdown_connections 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f8f5de00640 1 -- 172.21.15.46:0/4226874766 wait complete. 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f3b4f400640 1 -- 172.21.15.46:0/2796598304 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580b4650 con 0x7f3b580d08d0 2024-04-26T23:37:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7f3b4f400640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d08d0 0x7f3b580b4b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009870 tx=0x7f3b4002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fe4de8dd62d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2679423270 0 0) 0x7f0504003420 con 0x7f0520073bc0 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 msgr2=0x7f0520170530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f0520170530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 >> v1:172.21.15.142:6789/0 conn(0x7f052006f660 legacy=0x7f0520071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0520072d30 con 0x7f0520073bc0 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f05180019f0 con 0x7f0520073bc0 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 >> v1:172.21.15.46:6789/0 conn(0x7f0520073bc0 legacy=0x7f05200b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0504004c50 0x7f0504005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05040036f0 0x7f0504003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f050400b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0520073450 con 0x7f05040036f0 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0504004470 con 0x7f0504004c50 2024-04-26T23:37:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f050400bca0 con 0x7f05200cd750 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.003+0000 7f051fe00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0504004c50 0x7f0504005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f0524a00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f050400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051f400640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05040036f0 0x7f0504003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051fe00640 1 -- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 msgr2=0x7f050400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051fe00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cd750 0x7f050400b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051fe00640 1 -- 172.21.15.46:0/1352573568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05040036f0 msgr2=0x7f0504003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051fe00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05040036f0 0x7f0504003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051fe00640 1 -- 172.21.15.46:0/1352573568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0520072d30 con 0x7f0504004c50 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7f051f400640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05040036f0 0x7f0504003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7feb7ea00640 1 --2- 172.21.15.46:0/2071193158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb800ce3a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cf4515a046ace8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.002+0000 7fbf6d800640 1 -- start start 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.004+0000 7fbf6d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 0x7fbf680b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf6d800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 0x7fbf6814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf6d800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf6d800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbf680b4940 con 0x7fbf6806e780 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf6d800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbf680b4c20 con 0x7fbf680b43c0 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf6d800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf680b4f00 con 0x7fbf6806eb60 2024-04-26T23:37:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf67400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7feb7e000640 1 -- 172.21.15.46:0/2071193158 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c037070 con 0x7feb800b5530 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7feb7e000640 1 -- 172.21.15.46:0/2071193158 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c032440 con 0x7feb800b5530 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7feb7e000640 1 -- 172.21.15.46:0/2071193158 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c032810 con 0x7feb800b5530 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effd9e00640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 0x7effdc0c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effd9e00640 1 -- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 msgr2=0x7effdc0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effd9e00640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 0x7effdc0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effd9e00640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc203680 con 0x7effdc0ce6f0 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effda800640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 0x7effdc0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7effd9400640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 0x7effdc0c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf67400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60562/0 (socket says 172.21.15.46:60562) 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f3b57400640 1 -- 172.21.15.46:0/2796598304 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40037070 con 0x7f3b580d08d0 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f3b57400640 1 -- 172.21.15.46:0/2796598304 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b40032440 con 0x7f3b580d08d0 2024-04-26T23:37:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f3b57400640 1 -- 172.21.15.46:0/2796598304 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40032810 con 0x7f3b580d08d0 2024-04-26T23:37:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf67400640 1 -- 172.21.15.46:0/1618866215 learned_addr learned my addr 172.21.15.46:0/1618866215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.005+0000 7fbf66a00640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 0x7fbf680b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7effd9e00640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7effcc007450 tx=0x7effcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f051fe00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0504004c50 0x7f0504005050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05100019c0 tx=0x7f0504004720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcfa8fa2d7f1408 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f3b5ec00640 1 -- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d08d0 msgr2=0x7f3b580b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d08d0 0x7f3b580b4b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009870 tx=0x7f3b4002a210 comp rx=0 tx=0).stop 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100015e0 con 0x7f0504004c50 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051002e070 con 0x7f0504004c50 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7fbf66000640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 0x7fbf6814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.006+0000 7fbf67400640 1 -- 172.21.15.46:0/1618866215 >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 msgr2=0x7fbf6814caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf67400640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 0x7fbf6814caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf67400640 1 -- 172.21.15.46:0/1618866215 >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 msgr2=0x7fbf680b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 -- 172.21.15.46:0/2796598304 shutdown_connections 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b5815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/2796598304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d08d0 0x7f3b580b4b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 -- 172.21.15.46:0/2796598304 >> 172.21.15.46:0/2796598304 conn(0x7f3b5806c530 msgr2=0x7f3b5806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf67400640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 0x7fbf680b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf67400640 1 -- 172.21.15.46:0/1618866215 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf680b51e0 con 0x7fbf6806eb60 2024-04-26T23:37:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f995ae00640 1 Processor -- start 2024-04-26T23:37:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 -- 172.21.15.46:0/2796598304 shutdown_connections 2024-04-26T23:37:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f3b5ec00640 1 -- 172.21.15.46:0/2796598304 wait complete. 2024-04-26T23:37:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f051ea00640 1 -- 172.21.15.46:0/1352573568 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100048a0 con 0x7f0504004c50 2024-04-26T23:37:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7f995ae00640 1 -- start start 2024-04-26T23:37:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc040020 con 0x7effdc0ce6f0 2024-04-26T23:37:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf66000640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 0x7fbf6814caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7fbf66a00640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 0x7fbf680b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc03c800 con 0x7effdc0ce6f0 2024-04-26T23:37:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc03cbf0 con 0x7effdc0ce6f0 2024-04-26T23:37:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7feb86800640 1 -- 172.21.15.46:0/2071193158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 msgr2=0x7feb800ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.007+0000 7feb86800640 1 --2- 172.21.15.46:0/2071193158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb800ce3a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a230 comp rx=0 tx=0).stop 2024-04-26T23:37:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc0b4a80 con 0x7effdc0ce6f0 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc0b5040 con 0x7effdc0ce6f0 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7feb86800640 1 -- 172.21.15.46:0/2071193158 shutdown_connections 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7feb86800640 1 --2- 172.21.15.46:0/2071193158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb800ce3a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7feb86800640 1 -- 172.21.15.46:0/2071193158 >> 172.21.15.46:0/2071193158 conn(0x7feb800b4a10 msgr2=0x7feb800b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9954201680 0x7f99540c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954202590 0x7f99540cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7f8f5de00640 1 Processor -- start 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f3b5ec00640 1 Processor -- start 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0504004c50 msgr2=0x7f0504005050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7f0526a00640 1 --2- 172.21.15.46:0/1352573568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0504004c50 0x7f0504005050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05100019c0 tx=0x7f0504004720 comp rx=0 tx=0).stop 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f0524a00640 1 -- 172.21.15.46:0/1352573568 reap_dead start 2024-04-26T23:37:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 shutdown_connections 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 >> 172.21.15.46:0/1352573568 conn(0x7f052006cbd0 msgr2=0x7f05200b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.008+0000 7feb86800640 1 -- 172.21.15.46:0/2071193158 shutdown_connections 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7feb86800640 1 -- 172.21.15.46:0/2071193158 wait complete. 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99540d1e70 con 0x7f9954202590 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99540d1cf0 con 0x7f993c003a00 2024-04-26T23:37:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f995ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99540d1ff0 con 0x7f9954201680 2024-04-26T23:37:43.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 shutdown_connections 2024-04-26T23:37:43.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.009+0000 7f8f5de00640 1 -- start start 2024-04-26T23:37:43.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7fbf67400640 1 --2- 172.21.15.46:0/1618866215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b6550 tx=0x7fbf5802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd7bfde338e1918 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb4005290 con 0x7effdc0ce6f0 2024-04-26T23:37:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 0x7f8f5813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 0x7f8f58139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7fbf65600640 1 -- 172.21.15.46:0/1618866215 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58037070 con 0x7fbf6806eb60 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f9959000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9954201680 0x7f99540c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f9959a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f9959a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35648/0 (socket says 172.21.15.46:35648) 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f9959a00640 1 -- 172.21.15.46:0/1086563821 learned_addr learned my addr 172.21.15.46:0/1086563821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58072820 con 0x7f8f5815d540 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58072540 con 0x7f8f5813c540 2024-04-26T23:37:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f8f5de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8f58161570 con 0x7f8f5806e510 2024-04-26T23:37:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7fbf65600640 1 -- 172.21.15.46:0/1618866215 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58032440 con 0x7fbf6806eb60 2024-04-26T23:37:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f8f57400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f8f56a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 0x7f8f5813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f3b5ec00640 1 -- start start 2024-04-26T23:37:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f8f57400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35652/0 (socket says 172.21.15.46:35652) 2024-04-26T23:37:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f8f57400640 1 -- 172.21.15.46:0/278801649 learned_addr learned my addr 172.21.15.46:0/278801649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f3b5ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f3b5ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b580d08d0 0x7f3b580ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.011+0000 7f3b5ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b580cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f4e000640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 0x7f8f58139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b5ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5815f8d0 con 0x7f3b580d08d0 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b57e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b57e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35666/0 (socket says 172.21.15.46:35666) 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b57e00640 1 -- 172.21.15.46:0/1531581430 learned_addr learned my addr 172.21.15.46:0/1531581430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58070240 con 0x7f3b580d04f0 2024-04-26T23:37:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.010+0000 7f0526a00640 1 -- 172.21.15.46:0/1352573568 wait complete. 2024-04-26T23:37:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f995a400640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954202590 0x7f99540cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5815fa50 con 0x7f3b5815f470 2024-04-26T23:37:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7feb86800640 1 Processor -- start 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 -- 172.21.15.46:0/1618866215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 msgr2=0x7fbf6814d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf65600640 1 -- 172.21.15.46:0/1618866215 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58037070 con 0x7fbf6806eb60 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1618866215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b6550 tx=0x7fbf5802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 -- 172.21.15.46:0/1618866215 shutdown_connections 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1618866215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806eb60 0x7fbf6814d030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.142:3300/0 conn(0x7fbf680b43c0 0x7fbf6814caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1618866215 >> v2:172.21.15.46:3300/0 conn(0x7fbf6806e780 0x7fbf680b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7fbf6d800640 1 -- 172.21.15.46:0/1618866215 >> 172.21.15.46:0/1618866215 conn(0x7fbf6806c530 msgr2=0x7fbf6806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7feb86800640 1 -- start start 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f57400640 1 -- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 msgr2=0x7f8f5813c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f57400640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 0x7f8f5813c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f57400640 1 -- 172.21.15.46:0/278801649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 msgr2=0x7f8f58139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f57400640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 0x7f8f58139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f57400640 1 -- 172.21.15.46:0/278801649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5813ca70 con 0x7f8f5813c540 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.012+0000 7f8f56a00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 0x7f8f5813c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7f8f4e000640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 0x7f8f58139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7f3b4f400640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b580d08d0 0x7f3b580ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7f3b5cc00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b580cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7fbf6d800640 1 -- 172.21.15.46:0/1618866215 shutdown_connections 2024-04-26T23:37:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7fbf6d800640 1 -- 172.21.15.46:0/1618866215 wait complete. 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7f0526a00640 1 Processor -- start 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.013+0000 7f0526a00640 1 -- start start 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 -- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 msgr2=0x7f3b580cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b580cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 -- 172.21.15.46:0/1531581430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b580d08d0 msgr2=0x7f3b580ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b580d08d0 0x7f3b580ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b581f4d70 con 0x7f3b580d04f0 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f9959a00640 1 -- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9954201680 msgr2=0x7f99540c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f9959a00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9954201680 0x7f99540c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f9959a00640 1 -- 172.21.15.46:0/1086563821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954202590 msgr2=0x7f99540cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f9959a00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954202590 0x7f99540cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f9959a00640 1 -- 172.21.15.46:0/1086563821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9954201ae0 con 0x7f993c003a00 2024-04-26T23:37:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7f3b57e00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c430 tx=0x7f3b4800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7feb86800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb8009ba70 0x7feb801585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.014+0000 7feb86800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb8015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48011790 con 0x7f3b580d04f0 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b48017070 con 0x7f3b580d04f0 2024-04-26T23:37:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48011b70 con 0x7f3b580d04f0 2024-04-26T23:37:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb86800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb800d1210 con 0x7feb800ce8e0 2024-04-26T23:37:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb86800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb800b3f60 con 0x7feb800b5530 2024-04-26T23:37:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f9959a00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9940001870 tx=0x7f9940001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb86800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb800cdf10 con 0x7feb8009ba70 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b580cc6a0 con 0x7f3b580d04f0 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b580ccc60 con 0x7f3b580d04f0 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39300/0 (socket says 172.21.15.46:39300) 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.015+0000 7feb7fe00640 1 -- 172.21.15.46:0/1933301905 learned_addr learned my addr 172.21.15.46:0/1933301905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f8f57400640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c3c0 tx=0x7f8f50001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7fbf6d800640 1 Processor -- start 2024-04-26T23:37:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940007480 con 0x7f993c003a00 2024-04-26T23:37:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9940007620 con 0x7f993c003a00 2024-04-26T23:37:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7feb7f400640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb8009ba70 0x7feb801585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7fbf6d800640 1 -- start start 2024-04-26T23:37:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400049e0 con 0x7f993c003a00 2024-04-26T23:37:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99540ce9e0 con 0x7f993c003a00 2024-04-26T23:37:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9954201d90 con 0x7f993c003a00 2024-04-26T23:37:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50017070 con 0x7f8f5813c540 2024-04-26T23:37:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.016+0000 7feb7ea00640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb8015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f50005ce0 con 0x7f8f5813c540 2024-04-26T23:37:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50017070 con 0x7f8f5813c540 2024-04-26T23:37:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effcc043020 con 0x7effdc0ce6f0 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5813cd50 con 0x7f8f5813c540 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f58150c70 con 0x7f8f5813c540 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 -- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb8009ba70 msgr2=0x7feb801585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb8009ba70 0x7feb801585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 -- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 msgr2=0x7feb8015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb8015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 -- 172.21.15.46:0/1933301905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800b5e10 con 0x7feb800ce8e0 2024-04-26T23:37:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf6806eb60 0x7fbf680cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 0x7fbf680cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681672b0 con 0x7fbf680b43c0 2024-04-26T23:37:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.017+0000 7feb7fe00640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c420 tx=0x7feb7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68167130 con 0x7fbf6806eb60 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf6d800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf68167430 con 0x7fbf6806e780 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7fbf66a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99540693b0 con 0x7f993c003a00 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7fbf66a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60598/0 (socket says 172.21.15.46:60598) 2024-04-26T23:37:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7fbf66a00640 1 -- 172.21.15.46:0/1084490169 learned_addr learned my addr 172.21.15.46:0/1084490169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7effd2000640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 0x7effbc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7effcc0ac740 con 0x7effdc0ce6f0 2024-04-26T23:37:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 0x7f052016c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05200d01a0 0x7f052016f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 0x7f0520203740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0520073730 con 0x7f052016fa10 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05200b58b0 con 0x7f05200d01a0 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.019+0000 7f0526a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0520074070 con 0x7f05200cfdc0 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f051fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 0x7f052016c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f051f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05200d01a0 0x7f052016f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f0524a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 0x7f0520203740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30005290 con 0x7f8f5813c540 2024-04-26T23:37:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.018+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b58069530 con 0x7f3b580d04f0 2024-04-26T23:37:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7fbf67400640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 0x7fbf680cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f0524a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 0x7f0520203740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39302/0 (socket says 172.21.15.46:39302) 2024-04-26T23:37:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f0524a00640 1 -- 172.21.15.46:0/278990424 learned_addr learned my addr 172.21.15.46:0/278990424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7fbf66000640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf6806eb60 0x7fbf680cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7fbf66a00640 1 -- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf6806eb60 msgr2=0x7fbf680cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7fbf66a00640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf6806eb60 0x7fbf680cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b48011d10 con 0x7f3b580d04f0 2024-04-26T23:37:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7fbf66a00640 1 -- 172.21.15.46:0/1084490169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 msgr2=0x7fbf680cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7fbf66a00640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 0x7fbf680cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7fbf66a00640 1 -- 172.21.15.46:0/1084490169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf680b1650 con 0x7fbf6806e780 2024-04-26T23:37:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7fbf67400640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 0x7fbf680cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7f051fe00640 1 -- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05200d01a0 msgr2=0x7f052016f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7f051fe00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05200d01a0 0x7f052016f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7f051fe00640 1 -- 172.21.15.46:0/278990424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 msgr2=0x7f0520203740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7f051fe00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 0x7f0520203740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7f051fe00640 1 -- 172.21.15.46:0/278990424 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0520204470 con 0x7f05200cfdc0 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.020+0000 7effd9400640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 0x7effbc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.021+0000 7fbf66a00640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000efc0 tx=0x7fbf6000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7f051fe00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 0x7f052016c2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0510005000 tx=0x7f05100012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7effd9400640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 0x7effbc06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effdc152d40 tx=0x7effc8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60018020 con 0x7fbf6806e780 2024-04-26T23:37:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7f3b54c00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 0x7f3b3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf680b1930 con 0x7fbf6806e780 2024-04-26T23:37:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf680b1ef0 con 0x7fbf6806e780 2024-04-26T23:37:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b4808bce0 con 0x7f3b580d04f0 2024-04-26T23:37:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510043070 con 0x7f05200cfdc0 2024-04-26T23:37:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf60014680 con 0x7fbf6806e780 2024-04-26T23:37:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60014a50 con 0x7fbf6806e780 2024-04-26T23:37:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0520170050 con 0x7f05200cfdc0 2024-04-26T23:37:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f3b4f400640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 0x7f3b3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0520204a10 con 0x7f05200cfdc0 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510004cd0 con 0x7f05200cfdc0 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100072b0 con 0x7f05200cfdc0 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.022+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9940030050 con 0x7f993c003a00 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f9951600640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 0x7f993406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.024+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99400ab280 con 0x7f993c003a00 2024-04-26T23:37:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.024+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40005290 con 0x7fbf6806e780 2024-04-26T23:37:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.023+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f5001a070 con 0x7f8f5813c540 2024-04-26T23:37:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.024+0000 7f3b4f400640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 0x7f3b3406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b40034000 tx=0x7f3b4002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78017070 con 0x7feb800ce8e0 2024-04-26T23:37:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0520204c10 con 0x7f05200cfdc0 2024-04-26T23:37:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb78005ce0 con 0x7feb800ce8e0 2024-04-26T23:37:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780072d0 con 0x7feb800ce8e0 2024-04-26T23:37:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb800b6080 con 0x7feb800ce8e0 2024-04-26T23:37:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb800b6640 con 0x7feb800ce8e0 2024-04-26T23:37:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7effcc04a070 con 0x7effdc0ce6f0 2024-04-26T23:37:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7f9959000640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 0x7f993406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7f8f4fe00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 0x7f8f3406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.025+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8f5008ac40 con 0x7f8f5813c540 2024-04-26T23:37:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.026+0000 7f8f56a00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 0x7f8f3406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.026+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb58005290 con 0x7feb800ce8e0 2024-04-26T23:37:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.027+0000 7f9959000640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 0x7f993406ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9948001030 tx=0x7f9948007bb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.027+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf60005d10 con 0x7fbf6806e780 2024-04-26T23:37:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.028+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0510030050 con 0x7f05200cfdc0 2024-04-26T23:37:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.028+0000 7f8f56a00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 0x7f8f3406efe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009b90 tx=0x7f8f40008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.028+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb7801a070 con 0x7feb800ce8e0 2024-04-26T23:37:43.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.028+0000 7f0517e00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 0x7f050006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f05100ab400 con 0x7f05200cfdc0 2024-04-26T23:37:43.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7feb77400640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 0x7feb6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7f051f400640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 0x7f050006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7fbf5ea00640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 0x7fbf4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbf6008d1d0 con 0x7fbf6806e780 2024-04-26T23:37:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.029+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feb7808b150 con 0x7feb800ce8e0 2024-04-26T23:37:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.030+0000 7f051f400640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 0x7f050006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f052008c5e0 tx=0x7f050c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.030+0000 7feb7f400640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 0x7feb6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.030+0000 7fbf66000640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 0x7fbf4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.031+0000 7feb7f400640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 0x7feb6006ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb700012e0 tx=0x7feb70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.032+0000 7fbf66000640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 0x7fbf4806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf68069c70 tx=0x7fbf54008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.038+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb78058bd0 con 0x7feb800ce8e0 2024-04-26T23:37:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.039+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b48059d30 con 0x7f3b580d04f0 2024-04-26T23:37:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.041+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf60022030 con 0x7fbf6806e780 2024-04-26T23:37:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.052+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9940078bf0 con 0x7f993c003a00 2024-04-26T23:37:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.054+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0520204c10 con 0x7f05200cfdc0 2024-04-26T23:37:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.066+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f50007e20 con 0x7f8f5813c540 2024-04-26T23:37:43.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.390+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f37b4053bd0 con 0x7f37b40542f0 2024-04-26T23:37:43.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.391+0000 7f37b2a00640 1 -- 172.21.15.46:0/1991216311 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37b4179c60 con 0x7f37b40542f0 2024-04-26T23:37:43.399 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.395+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 msgr2=0x7f379c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.395+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 0x7f379c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a8034000 tx=0x7f37a802e000 comp rx=0 tx=0).stop 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.395+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 msgr2=0x7f37b4176020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.395+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37bc063ef0 tx=0x7f37bc0707c0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 shutdown_connections 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f379c06c8f0 0x7f379c06edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b40e30b0 0x7f37b40399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b40542f0 0x7f37b4176020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 --2- 172.21.15.46:0/1991216311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b403e830 0x7f37b4179720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.396+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 >> 172.21.15.46:0/1991216311 conn(0x7f37b40559d0 msgr2=0x7f37b4055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.397+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 shutdown_connections 2024-04-26T23:37:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.397+0000 7f37c1800640 1 -- 172.21.15.46:0/1991216311 wait complete. 2024-04-26T23:37:43.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-04-26T23:37:43.428 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.461+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8f30005580 con 0x7f8f5813c540 2024-04-26T23:37:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.462+0000 7f8f4fe00640 1 -- 172.21.15.46:0/278801649 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f5008a020 con 0x7f8f5813c540 2024-04-26T23:37:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.463+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7effb4005850 con 0x7effdc0ce6f0 2024-04-26T23:37:43.467 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.463+0000 7effd2000640 1 -- 172.21.15.46:0/918767709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effcc07a8a0 con 0x7effdc0ce6f0 2024-04-26T23:37:43.467 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 msgr2=0x7f8f3406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 0x7f8f3406efe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009b90 tx=0x7f8f40008040 comp rx=0 tx=0).stop 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 msgr2=0x7f8f5813a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c3c0 tx=0x7f8f50001e70 comp rx=0 tx=0).stop 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 shutdown_connections 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f3406cb20 0x7f8f3406efe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f5813c540 0x7f8f5813a3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5815d540 0x7f8f58139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 --2- 172.21.15.46:0/278801649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f5806e510 0x7f8f5813c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.465+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 >> 172.21.15.46:0/278801649 conn(0x7f8f5806c530 msgr2=0x7f8f580b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.466+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 shutdown_connections 2024-04-26T23:37:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.466+0000 7f8f5de00640 1 -- 172.21.15.46:0/278801649 wait complete. 2024-04-26T23:37:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 msgr2=0x7effbc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 0x7effbc06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effdc152d40 tx=0x7effc8008040 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 msgr2=0x7effdc203140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7effcc007450 tx=0x7effcc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 shutdown_connections 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7effbc06c9c0 0x7effbc06ee80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effdc15ec90 0x7effdc0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effdc15b060 0x7effdc0c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 --2- 172.21.15.46:0/918767709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effdc0ce6f0 0x7effdc203140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 >> 172.21.15.46:0/918767709 conn(0x7effdc0d0b20 msgr2=0x7effdc158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.467+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 shutdown_connections 2024-04-26T23:37:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.468+0000 7effdb200640 1 -- 172.21.15.46:0/918767709 wait complete. 2024-04-26T23:37:43.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.475+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3b580cf4f0 con 0x7f3b580d04f0 2024-04-26T23:37:43.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.476+0000 7f3b54c00640 1 -- 172.21.15.46:0/1531581430 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b48059630 con 0x7f3b580d04f0 2024-04-26T23:37:43.483 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 msgr2=0x7f3b3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 0x7f3b3406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b40034000 tx=0x7f3b4002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 msgr2=0x7f3b5815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c430 tx=0x7f3b4800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 shutdown_connections 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b3406c8f0 0x7f3b3406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b5815f470 0x7f3b580cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b580d08d0 0x7f3b580ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 --2- 172.21.15.46:0/1531581430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b580d04f0 0x7f3b5815c9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 >> 172.21.15.46:0/1531581430 conn(0x7f3b5806c530 msgr2=0x7f3b5815d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 shutdown_connections 2024-04-26T23:37:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.480+0000 7f3b5ec00640 1 -- 172.21.15.46:0/1531581430 wait complete. 2024-04-26T23:37:43.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.2 2024-04-26T23:37:43.491 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-04-26T23:37:43.497 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.497+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f99540c96a0 con 0x7f993c003a00 2024-04-26T23:37:43.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.498+0000 7f9951600640 1 -- 172.21.15.46:0/1086563821 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f994007cef0 con 0x7f993c003a00 2024-04-26T23:37:43.501 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 msgr2=0x7f993406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 0x7f993406ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9948001030 tx=0x7f9948007bb0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 msgr2=0x7f99540ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9940001870 tx=0x7f9940001120 comp rx=0 tx=0).stop 2024-04-26T23:37:43.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 shutdown_connections 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feb58005580 con 0x7feb800ce8e0 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f993406c920 0x7f993406ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954202590 0x7f99540cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9954201680 0x7f99540c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 --2- 172.21.15.46:0/1086563821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c003a00 0x7f99540ce340 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 >> 172.21.15.46:0/1086563821 conn(0x7f99540d4f70 msgr2=0x7f995406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 shutdown_connections 2024-04-26T23:37:43.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.500+0000 7f995ae00640 1 -- 172.21.15.46:0/1086563821 wait complete. 2024-04-26T23:37:43.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.501+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbf40005850 con 0x7fbf6806e780 2024-04-26T23:37:43.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.501+0000 7feb77400640 1 -- 172.21.15.46:0/1933301905 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb7805ced0 con 0x7feb800ce8e0 2024-04-26T23:37:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-26T23:37:43.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.501+0000 7fbf5ea00640 1 -- 172.21.15.46:0/1084490169 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf60010040 con 0x7fbf6806e780 2024-04-26T23:37:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.502+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0520069530 con 0x7f05200cfdc0 2024-04-26T23:37:43.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.503+0000 7f0517e00640 1 -- 172.21.15.46:0/278990424 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0510078e80 con 0x7f05200cfdc0 2024-04-26T23:37:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-26T23:37:43.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 msgr2=0x7feb6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 0x7feb6006ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb700012e0 tx=0x7feb70009040 comp rx=0 tx=0).stop 2024-04-26T23:37:43.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 msgr2=0x7feb8015d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c420 tx=0x7feb7800c8e0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 msgr2=0x7fbf4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 0x7fbf4806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf68069c70 tx=0x7fbf54008040 comp rx=0 tx=0).stop 2024-04-26T23:37:43.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 msgr2=0x7fbf68146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.504+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000efc0 tx=0x7fbf6000c490 comp rx=0 tx=0).stop 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 shutdown_connections 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb6006c940 0x7feb6006ee00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb800ce8e0 0x7feb8015d760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb800b5530 0x7feb8015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 --2- 172.21.15.46:0/1933301905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb8009ba70 0x7feb801585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 >> 172.21.15.46:0/1933301905 conn(0x7feb800b4a10 msgr2=0x7feb8015ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2024-04-26T23:37:43.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.505+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 shutdown_connections 2024-04-26T23:37:43.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7feb86800640 1 -- 172.21.15.46:0/1933301905 wait complete. 2024-04-26T23:37:43.512 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 shutdown_connections 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4806c8f0 0x7fbf4806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf680b43c0 0x7fbf680cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf6806eb60 0x7fbf680cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 --2- 172.21.15.46:0/1084490169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6806e780 0x7fbf68146930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 >> 172.21.15.46:0/1084490169 conn(0x7fbf6806c530 msgr2=0x7fbf680cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 msgr2=0x7f050006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 0x7f050006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f052008c5e0 tx=0x7f050c008040 comp rx=0 tx=0).stop 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 msgr2=0x7f052016c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 0x7f052016c2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0510005000 tx=0x7f05100012d0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 shutdown_connections 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f050006c920 0x7f050006ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052016fa10 0x7f0520203740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05200d01a0 0x7f052016f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 shutdown_connections 2024-04-26T23:37:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 --2- 172.21.15.46:0/278990424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05200cfdc0 0x7f052016c2a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 >> 172.21.15.46:0/278990424 conn(0x7f052006cbd0 msgr2=0x7f05200b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 shutdown_connections 2024-04-26T23:37:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7f0526a00640 1 -- 172.21.15.46:0/278990424 wait complete. 2024-04-26T23:37:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.506+0000 7fbf6d800640 1 -- 172.21.15.46:0/1084490169 wait complete. 2024-04-26T23:37:43.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.3 2024-04-26T23:37:43.526 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-04-26T23:37:43.529 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-04-26T23:37:43.536 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2024-04-26T23:37:43.539 DEBUG:teuthology.parallel:result is None 2024-04-26T23:37:43.539 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T23:37:43.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:37:43.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.777+0000 7f5b5de00640 1 Processor -- start 2024-04-26T23:37:43.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.777+0000 7f5b5de00640 1 -- start start 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5b5815f4e0 0x7f5b580d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5de00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b580d11f0 con 0x7f5b580d2130 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b580d5f20 con 0x7f5b580b4000 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5de00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5b580d0ce0 con 0x7f5b5815f4e0 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5b580d2130 0x7f5b580b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39856/0 (socket says 172.21.15.46:39856) 2024-04-26T23:37:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.779+0000 7f5b5d400640 1 -- 172.21.15.46:0/1942857114 learned_addr learned my addr 172.21.15.46:0/1942857114 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/1942857114 >> v2:172.21.15.142:3300/0 conn(0x7f5b5815f4e0 0x7f5b580d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696943243 0 0) 0x7f5b580d11f0 con 0x7f5b580d2130 2024-04-26T23:37:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b40003520 con 0x7f5b580d2130 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5969464 0 0) 0x7f5b40003520 con 0x7f5b580d2130 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 >> v2:172.21.15.142:3300/0 conn(0x7f5b5815f4e0 msgr2=0x7f5b580d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 --2- 172.21.15.46:0/1942857114 >> v2:172.21.15.142:3300/0 conn(0x7f5b5815f4e0 0x7f5b580d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 >> v1:172.21.15.142:6789/0 conn(0x7f5b580b4000 legacy=0x7f5b580d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580d0f10 con 0x7f5b580d2130 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5b500019f0 con 0x7f5b580d2130 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 >> v1:172.21.15.46:6789/0 conn(0x7f5b580d2130 legacy=0x7f5b580b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 0x7f5b40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40004800 0x7f5b40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b400051c0 0x7f5b4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4000c090 con 0x7f5b40003a00 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4000c240 con 0x7f5b400051c0 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.780+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4000c440 con 0x7f5b40004800 2024-04-26T23:37:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 0x7f5b40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 -- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40004800 msgr2=0x7f5b40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40004800 0x7f5b40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5d400640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b400051c0 0x7f5b4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 -- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b400051c0 msgr2=0x7f5b4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b400051c0 0x7f5b4000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5ca00640 1 -- 172.21.15.46:0/1942857114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b40003520 con 0x7f5b40003a00 2024-04-26T23:37:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.781+0000 7f5b5d400640 1 -- 172.21.15.46:0/1942857114 reap_dead start 2024-04-26T23:37:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.782+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 0x7f5b40003e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009ac0 tx=0x7f5b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2982d15df0f44d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.782+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036070 con 0x7f5b40003a00 2024-04-26T23:37:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.782+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c031400 con 0x7f5b40003a00 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.782+0000 7f5b57400640 1 -- 172.21.15.46:0/1942857114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009690 con 0x7f5b40003a00 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 -- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 msgr2=0x7f5b40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 0x7f5b40003e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009ac0 tx=0x7f5b40005830 comp rx=0 tx=0).stop 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 -- 172.21.15.46:0/1942857114 shutdown_connections 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 --2- 172.21.15.46:0/1942857114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b40003a00 0x7f5b40003e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 -- 172.21.15.46:0/1942857114 >> 172.21.15.46:0/1942857114 conn(0x7f5b580d5010 msgr2=0x7f5b580d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 -- 172.21.15.46:0/1942857114 shutdown_connections 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.783+0000 7f5b5de00640 1 -- 172.21.15.46:0/1942857114 wait complete. 2024-04-26T23:37:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.785+0000 7f5b5de00640 1 Processor -- start 2024-04-26T23:37:43.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.785+0000 7f5b5de00640 1 -- start start 2024-04-26T23:37:43.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40003a00 0x7f5b581f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 0x7f5b58158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580b1d60 con 0x7f5b58158f50 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580d2990 con 0x7f5b5815b530 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5815f8c0 con 0x7f5b40003a00 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39330/0 (socket says 172.21.15.46:39330) 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.786+0000 7f5b5d400640 1 -- 172.21.15.46:0/2248048545 learned_addr learned my addr 172.21.15.46:0/2248048545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40003a00 0x7f5b581f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b57e00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 0x7f5b58158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 -- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40003a00 msgr2=0x7f5b581f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40003a00 0x7f5b581f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 -- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 msgr2=0x7f5b58158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 0x7f5b58158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:43.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 -- 172.21.15.46:0/2248048545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5815bb80 con 0x7f5b58158f50 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b57e00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 0x7f5b58158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5d400640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b5808d650 tx=0x7f5b5000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50019070 con 0x7f5b58158f50 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5000b810 con 0x7f5b58158f50 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000bc00 con 0x7f5b58158f50 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.787+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5815be60 con 0x7f5b58158f50 2024-04-26T23:37:43.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.788+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580c9d90 con 0x7f5b58158f50 2024-04-26T23:37:43.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.789+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5815b910 con 0x7f5b58158f50 2024-04-26T23:37:43.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.790+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b5001c070 con 0x7f5b58158f50 2024-04-26T23:37:43.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.790+0000 7f5b54c00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:43.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.791+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5b5008b8d0 con 0x7f5b58158f50 2024-04-26T23:37:43.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.791+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:43.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.792+0000 7f5b5ca00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 0x7f5b3806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c033000 tx=0x7f5b4c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:43.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.795+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b50059920 con 0x7f5b58158f50 2024-04-26T23:37:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:43.987+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b580693b0 con 0x7f5b3806c8f0 2024-04-26T23:37:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.012+0000 7f5b54c00640 1 -- 172.21.15.46:0/2248048545 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332535 (secure 0 0 0) 0x7f5b580693b0 con 0x7f5b3806c8f0 2024-04-26T23:37:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 msgr2=0x7f5b3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 0x7f5b3806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c033000 tx=0x7f5b4c008030 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 msgr2=0x7f5b580cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b5808d650 tx=0x7f5b5000b0d0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 shutdown_connections 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b58158f50 0x7f5b580cd640 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b5815b530 0x7f5b58158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 --2- 172.21.15.46:0/2248048545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b40003a00 0x7f5b581f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 >> 172.21.15.46:0/2248048545 conn(0x7f5b580d5010 msgr2=0x7f5b58161b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 shutdown_connections 2024-04-26T23:37:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.020+0000 7f5b5de00640 1 -- 172.21.15.46:0/2248048545 wait complete. 2024-04-26T23:37:44.045 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-26T23:37:43.895489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223640,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751080,"statfs":{"total":773094113280,"available":772865105920,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005910"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795609+0000","last_change":"2024-04-26T23:37:07.740855+0000","last_active":"2024-04-26T23:37:13.795609+0000","last_peered":"2024-04-26T23:37:13.795609+0000","last_clean":"2024-04-26T23:37:13.795609+0000","last_became_active":"2024-04-26T23:37:07.739396+0000","last_became_peered":"2024-04-26T23:37:07.739396+0000","last_unstale":"2024-04-26T23:37:13.795609+0000","last_undegraded":"2024-04-26T23:37:13.795609+0000","last_fullsized":"2024-04-26T23:37:13.795609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:05.866636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892052+0000","last_change":"2024-04-26T23:37:08.773954+0000","last_active":"2024-04-26T23:37:13.892052+0000","last_peered":"2024-04-26T23:37:13.892052+0000","last_clean":"2024-04-26T23:37:13.892052+0000","last_became_active":"2024-04-26T23:37:08.771001+0000","last_became_peered":"2024-04-26T23:37:08.771001+0000","last_unstale":"2024-04-26T23:37:13.892052+0000","last_undegraded":"2024-04-26T23:37:13.892052+0000","last_fullsized":"2024-04-26T23:37:13.892052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890219+0000","last_change":"2024-04-26T23:37:13.098670+0000","last_active":"2024-04-26T23:37:13.890219+0000","last_peered":"2024-04-26T23:37:13.890219+0000","last_clean":"2024-04-26T23:37:13.890219+0000","last_became_active":"2024-04-26T23:37:13.098244+0000","last_became_peered":"2024-04-26T23:37:13.098244+0000","last_unstale":"2024-04-26T23:37:13.890219+0000","last_undegraded":"2024-04-26T23:37:13.890219+0000","last_fullsized":"2024-04-26T23:37:13.890219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892226+0000","last_change":"2024-04-26T23:37:07.729205+0000","last_active":"2024-04-26T23:37:13.892226+0000","last_peered":"2024-04-26T23:37:13.892226+0000","last_clean":"2024-04-26T23:37:13.892226+0000","last_became_active":"2024-04-26T23:37:07.728871+0000","last_became_peered":"2024-04-26T23:37:07.728871+0000","last_unstale":"2024-04-26T23:37:13.892226+0000","last_undegraded":"2024-04-26T23:37:13.892226+0000","last_fullsized":"2024-04-26T23:37:13.892226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795238+0000","last_change":"2024-04-26T23:37:08.793071+0000","last_active":"2024-04-26T23:37:13.795238+0000","last_peered":"2024-04-26T23:37:13.795238+0000","last_clean":"2024-04-26T23:37:13.795238+0000","last_became_active":"2024-04-26T23:37:08.792784+0000","last_became_peered":"2024-04-26T23:37:08.792784+0000","last_unstale":"2024-04-26T23:37:13.795238+0000","last_undegraded":"2024-04-26T23:37:13.795238+0000","last_fullsized":"2024-04-26T23:37:13.795238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798412+0000","last_change":"2024-04-26T23:37:12.830234+0000","last_active":"2024-04-26T23:37:13.798412+0000","last_peered":"2024-04-26T23:37:13.798412+0000","last_clean":"2024-04-26T23:37:13.798412+0000","last_became_active":"2024-04-26T23:37:12.829735+0000","last_became_peered":"2024-04-26T23:37:12.829735+0000","last_unstale":"2024-04-26T23:37:13.798412+0000","last_undegraded":"2024-04-26T23:37:13.798412+0000","last_fullsized":"2024-04-26T23:37:13.798412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:01.085576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893008+0000","last_change":"2024-04-26T23:37:07.755959+0000","last_active":"2024-04-26T23:37:13.893008+0000","last_peered":"2024-04-26T23:37:13.893008+0000","last_clean":"2024-04-26T23:37:13.893008+0000","last_became_active":"2024-04-26T23:37:07.754315+0000","last_became_peered":"2024-04-26T23:37:07.754315+0000","last_unstale":"2024-04-26T23:37:13.893008+0000","last_undegraded":"2024-04-26T23:37:13.893008+0000","last_fullsized":"2024-04-26T23:37:13.893008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802750+0000","last_change":"2024-04-26T23:37:08.786529+0000","last_active":"2024-04-26T23:37:13.802750+0000","last_peered":"2024-04-26T23:37:13.802750+0000","last_clean":"2024-04-26T23:37:13.802750+0000","last_became_active":"2024-04-26T23:37:08.781335+0000","last_became_peered":"2024-04-26T23:37:08.781335+0000","last_unstale":"2024-04-26T23:37:13.802750+0000","last_undegraded":"2024-04-26T23:37:13.802750+0000","last_fullsized":"2024-04-26T23:37:13.802750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798764+0000","last_change":"2024-04-26T23:37:12.830290+0000","last_active":"2024-04-26T23:37:13.798764+0000","last_peered":"2024-04-26T23:37:13.798764+0000","last_clean":"2024-04-26T23:37:13.798764+0000","last_became_active":"2024-04-26T23:37:12.829772+0000","last_became_peered":"2024-04-26T23:37:12.829772+0000","last_unstale":"2024-04-26T23:37:13.798764+0000","last_undegraded":"2024-04-26T23:37:13.798764+0000","last_fullsized":"2024-04-26T23:37:13.798764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:01.225018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892372+0000","last_change":"2024-04-26T23:37:07.756056+0000","last_active":"2024-04-26T23:37:13.892372+0000","last_peered":"2024-04-26T23:37:13.892372+0000","last_clean":"2024-04-26T23:37:13.892372+0000","last_became_active":"2024-04-26T23:37:07.754918+0000","last_became_peered":"2024-04-26T23:37:07.754918+0000","last_unstale":"2024-04-26T23:37:13.892372+0000","last_undegraded":"2024-04-26T23:37:13.892372+0000","last_fullsized":"2024-04-26T23:37:13.892372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:08.953262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803127+0000","last_change":"2024-04-26T23:37:08.780722+0000","last_active":"2024-04-26T23:37:13.803127+0000","last_peered":"2024-04-26T23:37:13.803127+0000","last_clean":"2024-04-26T23:37:13.803127+0000","last_became_active":"2024-04-26T23:37:08.780223+0000","last_became_peered":"2024-04-26T23:37:08.780223+0000","last_unstale":"2024-04-26T23:37:13.803127+0000","last_undegraded":"2024-04-26T23:37:13.803127+0000","last_fullsized":"2024-04-26T23:37:13.803127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798239+0000","last_change":"2024-04-26T23:37:12.831066+0000","last_active":"2024-04-26T23:37:13.798239+0000","last_peered":"2024-04-26T23:37:13.798239+0000","last_clean":"2024-04-26T23:37:13.798239+0000","last_became_active":"2024-04-26T23:37:12.829941+0000","last_became_peered":"2024-04-26T23:37:12.829941+0000","last_unstale":"2024-04-26T23:37:13.798239+0000","last_undegraded":"2024-04-26T23:37:13.798239+0000","last_fullsized":"2024-04-26T23:37:13.798239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:04.322071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890099+0000","last_change":"2024-04-26T23:37:08.768304+0000","last_active":"2024-04-26T23:37:13.890099+0000","last_peered":"2024-04-26T23:37:13.890099+0000","last_clean":"2024-04-26T23:37:13.890099+0000","last_became_active":"2024-04-26T23:37:08.767971+0000","last_became_peered":"2024-04-26T23:37:08.767971+0000","last_unstale":"2024-04-26T23:37:13.890099+0000","last_undegraded":"2024-04-26T23:37:13.890099+0000","last_fullsized":"2024-04-26T23:37:13.890099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892460+0000","last_change":"2024-04-26T23:37:07.755890+0000","last_active":"2024-04-26T23:37:13.892460+0000","last_peered":"2024-04-26T23:37:13.892460+0000","last_clean":"2024-04-26T23:37:13.892460+0000","last_became_active":"2024-04-26T23:37:07.753801+0000","last_became_peered":"2024-04-26T23:37:07.753801+0000","last_unstale":"2024-04-26T23:37:13.892460+0000","last_undegraded":"2024-04-26T23:37:13.892460+0000","last_fullsized":"2024-04-26T23:37:13.892460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800561+0000","last_change":"2024-04-26T23:37:12.821170+0000","last_active":"2024-04-26T23:37:13.800561+0000","last_peered":"2024-04-26T23:37:13.800561+0000","last_clean":"2024-04-26T23:37:13.800561+0000","last_became_active":"2024-04-26T23:37:12.820739+0000","last_became_peered":"2024-04-26T23:37:12.820739+0000","last_unstale":"2024-04-26T23:37:13.800561+0000","last_undegraded":"2024-04-26T23:37:13.800561+0000","last_fullsized":"2024-04-26T23:37:13.800561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891188+0000","last_change":"2024-04-26T23:37:08.772794+0000","last_active":"2024-04-26T23:37:13.891188+0000","last_peered":"2024-04-26T23:37:13.891188+0000","last_clean":"2024-04-26T23:37:13.891188+0000","last_became_active":"2024-04-26T23:37:08.770226+0000","last_became_peered":"2024-04-26T23:37:08.770226+0000","last_unstale":"2024-04-26T23:37:13.891188+0000","last_undegraded":"2024-04-26T23:37:13.891188+0000","last_fullsized":"2024-04-26T23:37:13.891188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888791+0000","last_change":"2024-04-26T23:37:07.739350+0000","last_active":"2024-04-26T23:37:13.888791+0000","last_peered":"2024-04-26T23:37:13.888791+0000","last_clean":"2024-04-26T23:37:13.888791+0000","last_became_active":"2024-04-26T23:37:07.739007+0000","last_became_peered":"2024-04-26T23:37:07.739007+0000","last_unstale":"2024-04-26T23:37:13.888791+0000","last_undegraded":"2024-04-26T23:37:13.888791+0000","last_fullsized":"2024-04-26T23:37:13.888791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800746+0000","last_change":"2024-04-26T23:37:12.831545+0000","last_active":"2024-04-26T23:37:13.800746+0000","last_peered":"2024-04-26T23:37:13.800746+0000","last_clean":"2024-04-26T23:37:13.800746+0000","last_became_active":"2024-04-26T23:37:12.829548+0000","last_became_peered":"2024-04-26T23:37:12.829548+0000","last_unstale":"2024-04-26T23:37:13.800746+0000","last_undegraded":"2024-04-26T23:37:13.800746+0000","last_fullsized":"2024-04-26T23:37:13.800746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:31.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889936+0000","last_change":"2024-04-26T23:37:08.761785+0000","last_active":"2024-04-26T23:37:13.889936+0000","last_peered":"2024-04-26T23:37:13.889936+0000","last_clean":"2024-04-26T23:37:13.889936+0000","last_became_active":"2024-04-26T23:37:08.759700+0000","last_became_peered":"2024-04-26T23:37:08.759700+0000","last_unstale":"2024-04-26T23:37:13.889936+0000","last_undegraded":"2024-04-26T23:37:13.889936+0000","last_fullsized":"2024-04-26T23:37:13.889936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888332+0000","last_change":"2024-04-26T23:37:07.738157+0000","last_active":"2024-04-26T23:37:13.888332+0000","last_peered":"2024-04-26T23:37:13.888332+0000","last_clean":"2024-04-26T23:37:13.888332+0000","last_became_active":"2024-04-26T23:37:07.737830+0000","last_became_peered":"2024-04-26T23:37:07.737830+0000","last_unstale":"2024-04-26T23:37:13.888332+0000","last_undegraded":"2024-04-26T23:37:13.888332+0000","last_fullsized":"2024-04-26T23:37:13.888332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892316+0000","last_change":"2024-04-26T23:37:13.099265+0000","last_active":"2024-04-26T23:37:13.892316+0000","last_peered":"2024-04-26T23:37:13.892316+0000","last_clean":"2024-04-26T23:37:13.892316+0000","last_became_active":"2024-04-26T23:37:13.095798+0000","last_became_peered":"2024-04-26T23:37:13.095798+0000","last_unstale":"2024-04-26T23:37:13.892316+0000","last_undegraded":"2024-04-26T23:37:13.892316+0000","last_fullsized":"2024-04-26T23:37:13.892316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800680+0000","last_change":"2024-04-26T23:37:08.764943+0000","last_active":"2024-04-26T23:37:13.800680+0000","last_peered":"2024-04-26T23:37:13.800680+0000","last_clean":"2024-04-26T23:37:13.800680+0000","last_became_active":"2024-04-26T23:37:08.764636+0000","last_became_peered":"2024-04-26T23:37:08.764636+0000","last_unstale":"2024-04-26T23:37:13.800680+0000","last_undegraded":"2024-04-26T23:37:13.800680+0000","last_fullsized":"2024-04-26T23:37:13.800680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:29.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890217+0000","last_change":"2024-04-26T23:37:07.753035+0000","last_active":"2024-04-26T23:37:13.890217+0000","last_peered":"2024-04-26T23:37:13.890217+0000","last_clean":"2024-04-26T23:37:13.890217+0000","last_became_active":"2024-04-26T23:37:07.752363+0000","last_became_peered":"2024-04-26T23:37:07.752363+0000","last_unstale":"2024-04-26T23:37:13.890217+0000","last_undegraded":"2024-04-26T23:37:13.890217+0000","last_fullsized":"2024-04-26T23:37:13.890217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799225+0000","last_change":"2024-04-26T23:37:12.830106+0000","last_active":"2024-04-26T23:37:13.799225+0000","last_peered":"2024-04-26T23:37:13.799225+0000","last_clean":"2024-04-26T23:37:13.799225+0000","last_became_active":"2024-04-26T23:37:12.829676+0000","last_became_peered":"2024-04-26T23:37:12.829676+0000","last_unstale":"2024-04-26T23:37:13.799225+0000","last_undegraded":"2024-04-26T23:37:13.799225+0000","last_fullsized":"2024-04-26T23:37:13.799225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795807+0000","last_change":"2024-04-26T23:37:08.764908+0000","last_active":"2024-04-26T23:37:13.795807+0000","last_peered":"2024-04-26T23:37:13.795807+0000","last_clean":"2024-04-26T23:37:13.795807+0000","last_became_active":"2024-04-26T23:37:08.764369+0000","last_became_peered":"2024-04-26T23:37:08.764369+0000","last_unstale":"2024-04-26T23:37:13.795807+0000","last_undegraded":"2024-04-26T23:37:13.795807+0000","last_fullsized":"2024-04-26T23:37:13.795807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892624+0000","last_change":"2024-04-26T23:37:07.751902+0000","last_active":"2024-04-26T23:37:13.892624+0000","last_peered":"2024-04-26T23:37:13.892624+0000","last_clean":"2024-04-26T23:37:13.892624+0000","last_became_active":"2024-04-26T23:37:07.751383+0000","last_became_peered":"2024-04-26T23:37:07.751383+0000","last_unstale":"2024-04-26T23:37:13.892624+0000","last_undegraded":"2024-04-26T23:37:13.892624+0000","last_fullsized":"2024-04-26T23:37:13.892624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800477+0000","last_change":"2024-04-26T23:37:13.090483+0000","last_active":"2024-04-26T23:37:13.800477+0000","last_peered":"2024-04-26T23:37:13.800477+0000","last_clean":"2024-04-26T23:37:13.800477+0000","last_became_active":"2024-04-26T23:37:13.088991+0000","last_became_peered":"2024-04-26T23:37:13.088991+0000","last_unstale":"2024-04-26T23:37:13.800477+0000","last_undegraded":"2024-04-26T23:37:13.800477+0000","last_fullsized":"2024-04-26T23:37:13.800477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892790+0000","last_change":"2024-04-26T23:37:08.777852+0000","last_active":"2024-04-26T23:37:13.892790+0000","last_peered":"2024-04-26T23:37:13.892790+0000","last_clean":"2024-04-26T23:37:13.892790+0000","last_became_active":"2024-04-26T23:37:08.777214+0000","last_became_peered":"2024-04-26T23:37:08.777214+0000","last_unstale":"2024-04-26T23:37:13.892790+0000","last_undegraded":"2024-04-26T23:37:13.892790+0000","last_fullsized":"2024-04-26T23:37:13.892790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890844+0000","last_change":"2024-04-26T23:37:07.744896+0000","last_active":"2024-04-26T23:37:13.890844+0000","last_peered":"2024-04-26T23:37:13.890844+0000","last_clean":"2024-04-26T23:37:13.890844+0000","last_became_active":"2024-04-26T23:37:07.744597+0000","last_became_peered":"2024-04-26T23:37:07.744597+0000","last_unstale":"2024-04-26T23:37:13.890844+0000","last_undegraded":"2024-04-26T23:37:13.890844+0000","last_fullsized":"2024-04-26T23:37:13.890844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889553+0000","last_change":"2024-04-26T23:37:13.097492+0000","last_active":"2024-04-26T23:37:13.889553+0000","last_peered":"2024-04-26T23:37:13.889553+0000","last_clean":"2024-04-26T23:37:13.889553+0000","last_became_active":"2024-04-26T23:37:13.096325+0000","last_became_peered":"2024-04-26T23:37:13.096325+0000","last_unstale":"2024-04-26T23:37:13.889553+0000","last_undegraded":"2024-04-26T23:37:13.889553+0000","last_fullsized":"2024-04-26T23:37:13.889553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889171+0000","last_change":"2024-04-26T23:37:08.762047+0000","last_active":"2024-04-26T23:37:13.889171+0000","last_peered":"2024-04-26T23:37:13.889171+0000","last_clean":"2024-04-26T23:37:13.889171+0000","last_became_active":"2024-04-26T23:37:08.761660+0000","last_became_peered":"2024-04-26T23:37:08.761660+0000","last_unstale":"2024-04-26T23:37:13.889171+0000","last_undegraded":"2024-04-26T23:37:13.889171+0000","last_fullsized":"2024-04-26T23:37:13.889171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893281+0000","last_change":"2024-04-26T23:37:07.753465+0000","last_active":"2024-04-26T23:37:13.893281+0000","last_peered":"2024-04-26T23:37:13.893281+0000","last_clean":"2024-04-26T23:37:13.893281+0000","last_became_active":"2024-04-26T23:37:07.752398+0000","last_became_peered":"2024-04-26T23:37:07.752398+0000","last_unstale":"2024-04-26T23:37:13.893281+0000","last_undegraded":"2024-04-26T23:37:13.893281+0000","last_fullsized":"2024-04-26T23:37:13.893281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795198+0000","last_change":"2024-04-26T23:37:13.089856+0000","last_active":"2024-04-26T23:37:13.795198+0000","last_peered":"2024-04-26T23:37:13.795198+0000","last_clean":"2024-04-26T23:37:13.795198+0000","last_became_active":"2024-04-26T23:37:13.088870+0000","last_became_peered":"2024-04-26T23:37:13.088870+0000","last_unstale":"2024-04-26T23:37:13.795198+0000","last_undegraded":"2024-04-26T23:37:13.795198+0000","last_fullsized":"2024-04-26T23:37:13.795198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803393+0000","last_change":"2024-04-26T23:37:08.785871+0000","last_active":"2024-04-26T23:37:13.803393+0000","last_peered":"2024-04-26T23:37:13.803393+0000","last_clean":"2024-04-26T23:37:13.803393+0000","last_became_active":"2024-04-26T23:37:08.785451+0000","last_became_peered":"2024-04-26T23:37:08.785451+0000","last_unstale":"2024-04-26T23:37:13.803393+0000","last_undegraded":"2024-04-26T23:37:13.803393+0000","last_fullsized":"2024-04-26T23:37:13.803393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889253+0000","last_change":"2024-04-26T23:37:07.730561+0000","last_active":"2024-04-26T23:37:13.889253+0000","last_peered":"2024-04-26T23:37:13.889253+0000","last_clean":"2024-04-26T23:37:13.889253+0000","last_became_active":"2024-04-26T23:37:07.730274+0000","last_became_peered":"2024-04-26T23:37:07.730274+0000","last_unstale":"2024-04-26T23:37:13.889253+0000","last_undegraded":"2024-04-26T23:37:13.889253+0000","last_fullsized":"2024-04-26T23:37:13.889253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888467+0000","last_change":"2024-04-26T23:37:12.819624+0000","last_active":"2024-04-26T23:37:13.888467+0000","last_peered":"2024-04-26T23:37:13.888467+0000","last_clean":"2024-04-26T23:37:13.888467+0000","last_became_active":"2024-04-26T23:37:12.819079+0000","last_became_peered":"2024-04-26T23:37:12.819079+0000","last_unstale":"2024-04-26T23:37:13.888467+0000","last_undegraded":"2024-04-26T23:37:13.888467+0000","last_fullsized":"2024-04-26T23:37:13.888467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799143+0000","last_change":"2024-04-26T23:37:08.760531+0000","last_active":"2024-04-26T23:37:13.799143+0000","last_peered":"2024-04-26T23:37:13.799143+0000","last_clean":"2024-04-26T23:37:13.799143+0000","last_became_active":"2024-04-26T23:37:08.760250+0000","last_became_peered":"2024-04-26T23:37:08.760250+0000","last_unstale":"2024-04-26T23:37:13.799143+0000","last_undegraded":"2024-04-26T23:37:13.799143+0000","last_fullsized":"2024-04-26T23:37:13.799143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:17.138976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802089+0000","last_change":"2024-04-26T23:37:07.743324+0000","last_active":"2024-04-26T23:37:13.802089+0000","last_peered":"2024-04-26T23:37:13.802089+0000","last_clean":"2024-04-26T23:37:13.802089+0000","last_became_active":"2024-04-26T23:37:07.742056+0000","last_became_peered":"2024-04-26T23:37:07.742056+0000","last_unstale":"2024-04-26T23:37:13.802089+0000","last_undegraded":"2024-04-26T23:37:13.802089+0000","last_fullsized":"2024-04-26T23:37:13.802089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889116+0000","last_change":"2024-04-26T23:37:12.835987+0000","last_active":"2024-04-26T23:37:13.889116+0000","last_peered":"2024-04-26T23:37:13.889116+0000","last_clean":"2024-04-26T23:37:13.889116+0000","last_became_active":"2024-04-26T23:37:12.831199+0000","last_became_peered":"2024-04-26T23:37:12.831199+0000","last_unstale":"2024-04-26T23:37:13.889116+0000","last_undegraded":"2024-04-26T23:37:13.889116+0000","last_fullsized":"2024-04-26T23:37:13.889116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888008+0000","last_change":"2024-04-26T23:37:08.768283+0000","last_active":"2024-04-26T23:37:13.888008+0000","last_peered":"2024-04-26T23:37:13.888008+0000","last_clean":"2024-04-26T23:37:13.888008+0000","last_became_active":"2024-04-26T23:37:08.766512+0000","last_became_peered":"2024-04-26T23:37:08.766512+0000","last_unstale":"2024-04-26T23:37:13.888008+0000","last_undegraded":"2024-04-26T23:37:13.888008+0000","last_fullsized":"2024-04-26T23:37:13.888008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888285+0000","last_change":"2024-04-26T23:37:07.728903+0000","last_active":"2024-04-26T23:37:13.888285+0000","last_peered":"2024-04-26T23:37:13.888285+0000","last_clean":"2024-04-26T23:37:13.888285+0000","last_became_active":"2024-04-26T23:37:07.728621+0000","last_became_peered":"2024-04-26T23:37:07.728621+0000","last_unstale":"2024-04-26T23:37:13.888285+0000","last_undegraded":"2024-04-26T23:37:13.888285+0000","last_fullsized":"2024-04-26T23:37:13.888285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892519+0000","last_change":"2024-04-26T23:37:12.833651+0000","last_active":"2024-04-26T23:37:13.892519+0000","last_peered":"2024-04-26T23:37:13.892519+0000","last_clean":"2024-04-26T23:37:13.892519+0000","last_became_active":"2024-04-26T23:37:12.831533+0000","last_became_peered":"2024-04-26T23:37:12.831533+0000","last_unstale":"2024-04-26T23:37:13.892519+0000","last_undegraded":"2024-04-26T23:37:13.892519+0000","last_fullsized":"2024-04-26T23:37:13.892519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794491+0000","last_change":"2024-04-26T23:37:08.749652+0000","last_active":"2024-04-26T23:37:13.794491+0000","last_peered":"2024-04-26T23:37:13.794491+0000","last_clean":"2024-04-26T23:37:13.794491+0000","last_became_active":"2024-04-26T23:37:08.748932+0000","last_became_peered":"2024-04-26T23:37:08.748932+0000","last_unstale":"2024-04-26T23:37:13.794491+0000","last_undegraded":"2024-04-26T23:37:13.794491+0000","last_fullsized":"2024-04-26T23:37:13.794491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:04.431242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890355+0000","last_change":"2024-04-26T23:37:07.753005+0000","last_active":"2024-04-26T23:37:13.890355+0000","last_peered":"2024-04-26T23:37:13.890355+0000","last_clean":"2024-04-26T23:37:13.890355+0000","last_became_active":"2024-04-26T23:37:07.752377+0000","last_became_peered":"2024-04-26T23:37:07.752377+0000","last_unstale":"2024-04-26T23:37:13.890355+0000","last_undegraded":"2024-04-26T23:37:13.890355+0000","last_fullsized":"2024-04-26T23:37:13.890355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795722+0000","last_change":"2024-04-26T23:37:13.097286+0000","last_active":"2024-04-26T23:37:13.795722+0000","last_peered":"2024-04-26T23:37:13.795722+0000","last_clean":"2024-04-26T23:37:13.795722+0000","last_became_active":"2024-04-26T23:37:13.096512+0000","last_became_peered":"2024-04-26T23:37:13.096512+0000","last_unstale":"2024-04-26T23:37:13.795722+0000","last_undegraded":"2024-04-26T23:37:13.795722+0000","last_fullsized":"2024-04-26T23:37:13.795722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890492+0000","last_change":"2024-04-26T23:37:08.791257+0000","last_active":"2024-04-26T23:37:13.890492+0000","last_peered":"2024-04-26T23:37:13.890492+0000","last_clean":"2024-04-26T23:37:13.890492+0000","last_became_active":"2024-04-26T23:37:08.790966+0000","last_became_peered":"2024-04-26T23:37:08.790966+0000","last_unstale":"2024-04-26T23:37:13.890492+0000","last_undegraded":"2024-04-26T23:37:13.890492+0000","last_fullsized":"2024-04-26T23:37:13.890492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802430+0000","last_change":"2024-04-26T23:37:07.742144+0000","last_active":"2024-04-26T23:37:13.802430+0000","last_peered":"2024-04-26T23:37:13.802430+0000","last_clean":"2024-04-26T23:37:13.802430+0000","last_became_active":"2024-04-26T23:37:07.741812+0000","last_became_peered":"2024-04-26T23:37:07.741812+0000","last_unstale":"2024-04-26T23:37:13.802430+0000","last_undegraded":"2024-04-26T23:37:13.802430+0000","last_fullsized":"2024-04-26T23:37:13.802430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890498+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:37:13.890498+0000","last_peered":"2024-04-26T23:37:13.890498+0000","last_clean":"2024-04-26T23:37:13.890498+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:37:13.890498+0000","last_undegraded":"2024-04-26T23:37:13.890498+0000","last_fullsized":"2024-04-26T23:37:13.890498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892073+0000","last_change":"2024-04-26T23:37:13.094077+0000","last_active":"2024-04-26T23:37:13.892073+0000","last_peered":"2024-04-26T23:37:13.892073+0000","last_clean":"2024-04-26T23:37:13.892073+0000","last_became_active":"2024-04-26T23:37:13.093035+0000","last_became_peered":"2024-04-26T23:37:13.093035+0000","last_unstale":"2024-04-26T23:37:13.892073+0000","last_undegraded":"2024-04-26T23:37:13.892073+0000","last_fullsized":"2024-04-26T23:37:13.892073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892721+0000","last_change":"2024-04-26T23:37:08.787518+0000","last_active":"2024-04-26T23:37:13.892721+0000","last_peered":"2024-04-26T23:37:13.892721+0000","last_clean":"2024-04-26T23:37:13.892721+0000","last_became_active":"2024-04-26T23:37:08.787303+0000","last_became_peered":"2024-04-26T23:37:08.787303+0000","last_unstale":"2024-04-26T23:37:13.892721+0000","last_undegraded":"2024-04-26T23:37:13.892721+0000","last_fullsized":"2024-04-26T23:37:13.892721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891144+0000","last_change":"2024-04-26T23:37:07.746465+0000","last_active":"2024-04-26T23:37:13.891144+0000","last_peered":"2024-04-26T23:37:13.891144+0000","last_clean":"2024-04-26T23:37:13.891144+0000","last_became_active":"2024-04-26T23:37:07.746109+0000","last_became_peered":"2024-04-26T23:37:07.746109+0000","last_unstale":"2024-04-26T23:37:13.891144+0000","last_undegraded":"2024-04-26T23:37:13.891144+0000","last_fullsized":"2024-04-26T23:37:13.891144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890061+0000","last_change":"2024-04-26T23:37:13.098348+0000","last_active":"2024-04-26T23:37:13.890061+0000","last_peered":"2024-04-26T23:37:13.890061+0000","last_clean":"2024-04-26T23:37:13.890061+0000","last_became_active":"2024-04-26T23:37:13.092005+0000","last_became_peered":"2024-04-26T23:37:13.092005+0000","last_unstale":"2024-04-26T23:37:13.890061+0000","last_undegraded":"2024-04-26T23:37:13.890061+0000","last_fullsized":"2024-04-26T23:37:13.890061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802973+0000","last_change":"2024-04-26T23:37:08.788947+0000","last_active":"2024-04-26T23:37:13.802973+0000","last_peered":"2024-04-26T23:37:13.802973+0000","last_clean":"2024-04-26T23:37:13.802973+0000","last_became_active":"2024-04-26T23:37:08.785540+0000","last_became_peered":"2024-04-26T23:37:08.785540+0000","last_unstale":"2024-04-26T23:37:13.802973+0000","last_undegraded":"2024-04-26T23:37:13.802973+0000","last_fullsized":"2024-04-26T23:37:13.802973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889865+0000","last_change":"2024-04-26T23:37:07.745996+0000","last_active":"2024-04-26T23:37:13.889865+0000","last_peered":"2024-04-26T23:37:13.889865+0000","last_clean":"2024-04-26T23:37:13.889865+0000","last_became_active":"2024-04-26T23:37:07.745526+0000","last_became_peered":"2024-04-26T23:37:07.745526+0000","last_unstale":"2024-04-26T23:37:13.889865+0000","last_undegraded":"2024-04-26T23:37:13.889865+0000","last_fullsized":"2024-04-26T23:37:13.889865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795637+0000","last_change":"2024-04-26T23:37:12.818381+0000","last_active":"2024-04-26T23:37:13.795637+0000","last_peered":"2024-04-26T23:37:13.795637+0000","last_clean":"2024-04-26T23:37:13.795637+0000","last_became_active":"2024-04-26T23:37:12.816556+0000","last_became_peered":"2024-04-26T23:37:12.816556+0000","last_unstale":"2024-04-26T23:37:13.795637+0000","last_undegraded":"2024-04-26T23:37:13.795637+0000","last_fullsized":"2024-04-26T23:37:13.795637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893859+0000","last_change":"2024-04-26T23:37:08.788613+0000","last_active":"2024-04-26T23:37:13.893859+0000","last_peered":"2024-04-26T23:37:13.893859+0000","last_clean":"2024-04-26T23:37:13.893859+0000","last_became_active":"2024-04-26T23:37:08.788341+0000","last_became_peered":"2024-04-26T23:37:08.788341+0000","last_unstale":"2024-04-26T23:37:13.893859+0000","last_undegraded":"2024-04-26T23:37:13.893859+0000","last_fullsized":"2024-04-26T23:37:13.893859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:05.835929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891528+0000","last_change":"2024-04-26T23:37:07.732372+0000","last_active":"2024-04-26T23:37:13.891528+0000","last_peered":"2024-04-26T23:37:13.891528+0000","last_clean":"2024-04-26T23:37:13.891528+0000","last_became_active":"2024-04-26T23:37:07.732085+0000","last_became_peered":"2024-04-26T23:37:07.732085+0000","last_unstale":"2024-04-26T23:37:13.891528+0000","last_undegraded":"2024-04-26T23:37:13.891528+0000","last_fullsized":"2024-04-26T23:37:13.891528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798168+0000","last_change":"2024-04-26T23:37:13.093306+0000","last_active":"2024-04-26T23:37:13.798168+0000","last_peered":"2024-04-26T23:37:13.798168+0000","last_clean":"2024-04-26T23:37:13.798168+0000","last_became_active":"2024-04-26T23:37:13.089425+0000","last_became_peered":"2024-04-26T23:37:13.089425+0000","last_unstale":"2024-04-26T23:37:13.798168+0000","last_undegraded":"2024-04-26T23:37:13.798168+0000","last_fullsized":"2024-04-26T23:37:13.798168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893002+0000","last_change":"2024-04-26T23:37:08.788931+0000","last_active":"2024-04-26T23:37:13.893002+0000","last_peered":"2024-04-26T23:37:13.893002+0000","last_clean":"2024-04-26T23:37:13.893002+0000","last_became_active":"2024-04-26T23:37:08.788287+0000","last_became_peered":"2024-04-26T23:37:08.788287+0000","last_unstale":"2024-04-26T23:37:13.893002+0000","last_undegraded":"2024-04-26T23:37:13.893002+0000","last_fullsized":"2024-04-26T23:37:13.893002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799208+0000","last_change":"2024-04-26T23:37:07.729179+0000","last_active":"2024-04-26T23:37:13.799208+0000","last_peered":"2024-04-26T23:37:13.799208+0000","last_clean":"2024-04-26T23:37:13.799208+0000","last_became_active":"2024-04-26T23:37:07.728840+0000","last_became_peered":"2024-04-26T23:37:07.728840+0000","last_unstale":"2024-04-26T23:37:13.799208+0000","last_undegraded":"2024-04-26T23:37:13.799208+0000","last_fullsized":"2024-04-26T23:37:13.799208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:09.969446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798616+0000","last_change":"2024-04-26T23:37:12.833356+0000","last_active":"2024-04-26T23:37:13.798616+0000","last_peered":"2024-04-26T23:37:13.798616+0000","last_clean":"2024-04-26T23:37:13.798616+0000","last_became_active":"2024-04-26T23:37:12.830528+0000","last_became_peered":"2024-04-26T23:37:12.830528+0000","last_unstale":"2024-04-26T23:37:13.798616+0000","last_undegraded":"2024-04-26T23:37:13.798616+0000","last_fullsized":"2024-04-26T23:37:13.798616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.662558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891353+0000","last_change":"2024-04-26T23:37:08.782130+0000","last_active":"2024-04-26T23:37:13.891353+0000","last_peered":"2024-04-26T23:37:13.891353+0000","last_clean":"2024-04-26T23:37:13.891353+0000","last_became_active":"2024-04-26T23:37:08.781906+0000","last_became_peered":"2024-04-26T23:37:08.781906+0000","last_unstale":"2024-04-26T23:37:13.891353+0000","last_undegraded":"2024-04-26T23:37:13.891353+0000","last_fullsized":"2024-04-26T23:37:13.891353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.887760+0000","last_change":"2024-04-26T23:37:07.728318+0000","last_active":"2024-04-26T23:37:13.887760+0000","last_peered":"2024-04-26T23:37:13.887760+0000","last_clean":"2024-04-26T23:37:13.887760+0000","last_became_active":"2024-04-26T23:37:07.727941+0000","last_became_peered":"2024-04-26T23:37:07.727941+0000","last_unstale":"2024-04-26T23:37:13.887760+0000","last_undegraded":"2024-04-26T23:37:13.887760+0000","last_fullsized":"2024-04-26T23:37:13.887760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795484+0000","last_change":"2024-04-26T23:37:12.832185+0000","last_active":"2024-04-26T23:37:13.795484+0000","last_peered":"2024-04-26T23:37:13.795484+0000","last_clean":"2024-04-26T23:37:13.795484+0000","last_became_active":"2024-04-26T23:37:12.830851+0000","last_became_peered":"2024-04-26T23:37:12.830851+0000","last_unstale":"2024-04-26T23:37:13.795484+0000","last_undegraded":"2024-04-26T23:37:13.795484+0000","last_fullsized":"2024-04-26T23:37:13.795484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889876+0000","last_change":"2024-04-26T23:37:08.768079+0000","last_active":"2024-04-26T23:37:13.889876+0000","last_peered":"2024-04-26T23:37:13.889876+0000","last_clean":"2024-04-26T23:37:13.889876+0000","last_became_active":"2024-04-26T23:37:08.767338+0000","last_became_peered":"2024-04-26T23:37:08.767338+0000","last_unstale":"2024-04-26T23:37:13.889876+0000","last_undegraded":"2024-04-26T23:37:13.889876+0000","last_fullsized":"2024-04-26T23:37:13.889876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890125+0000","last_change":"2024-04-26T23:37:07.746339+0000","last_active":"2024-04-26T23:37:13.890125+0000","last_peered":"2024-04-26T23:37:13.890125+0000","last_clean":"2024-04-26T23:37:13.890125+0000","last_became_active":"2024-04-26T23:37:07.746075+0000","last_became_peered":"2024-04-26T23:37:07.746075+0000","last_unstale":"2024-04-26T23:37:13.890125+0000","last_undegraded":"2024-04-26T23:37:13.890125+0000","last_fullsized":"2024-04-26T23:37:13.890125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892748+0000","last_change":"2024-04-26T23:37:13.091284+0000","last_active":"2024-04-26T23:37:13.892748+0000","last_peered":"2024-04-26T23:37:13.892748+0000","last_clean":"2024-04-26T23:37:13.892748+0000","last_became_active":"2024-04-26T23:37:13.090274+0000","last_became_peered":"2024-04-26T23:37:13.090274+0000","last_unstale":"2024-04-26T23:37:13.892748+0000","last_undegraded":"2024-04-26T23:37:13.892748+0000","last_fullsized":"2024-04-26T23:37:13.892748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:53.078576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892237+0000","last_change":"2024-04-26T23:37:08.782024+0000","last_active":"2024-04-26T23:37:13.892237+0000","last_peered":"2024-04-26T23:37:13.892237+0000","last_clean":"2024-04-26T23:37:13.892237+0000","last_became_active":"2024-04-26T23:37:08.781730+0000","last_became_peered":"2024-04-26T23:37:08.781730+0000","last_unstale":"2024-04-26T23:37:13.892237+0000","last_undegraded":"2024-04-26T23:37:13.892237+0000","last_fullsized":"2024-04-26T23:37:13.892237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801858+0000","last_change":"2024-04-26T23:37:07.741413+0000","last_active":"2024-04-26T23:37:13.801858+0000","last_peered":"2024-04-26T23:37:13.801858+0000","last_clean":"2024-04-26T23:37:13.801858+0000","last_became_active":"2024-04-26T23:37:07.741088+0000","last_became_peered":"2024-04-26T23:37:07.741088+0000","last_unstale":"2024-04-26T23:37:13.801858+0000","last_undegraded":"2024-04-26T23:37:13.801858+0000","last_fullsized":"2024-04-26T23:37:13.801858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891737+0000","last_change":"2024-04-26T23:37:13.089930+0000","last_active":"2024-04-26T23:37:13.891737+0000","last_peered":"2024-04-26T23:37:13.891737+0000","last_clean":"2024-04-26T23:37:13.891737+0000","last_became_active":"2024-04-26T23:37:13.088797+0000","last_became_peered":"2024-04-26T23:37:13.088797+0000","last_unstale":"2024-04-26T23:37:13.891737+0000","last_undegraded":"2024-04-26T23:37:13.891737+0000","last_fullsized":"2024-04-26T23:37:13.891737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:24.129454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795384+0000","last_change":"2024-04-26T23:37:08.772085+0000","last_active":"2024-04-26T23:37:13.795384+0000","last_peered":"2024-04-26T23:37:13.795384+0000","last_clean":"2024-04-26T23:37:13.795384+0000","last_became_active":"2024-04-26T23:37:08.771732+0000","last_became_peered":"2024-04-26T23:37:08.771732+0000","last_unstale":"2024-04-26T23:37:13.795384+0000","last_undegraded":"2024-04-26T23:37:13.795384+0000","last_fullsized":"2024-04-26T23:37:13.795384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:01.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796199+0000","last_change":"2024-04-26T23:37:07.739747+0000","last_active":"2024-04-26T23:37:13.796199+0000","last_peered":"2024-04-26T23:37:13.796199+0000","last_clean":"2024-04-26T23:37:13.796199+0000","last_became_active":"2024-04-26T23:37:07.739361+0000","last_became_peered":"2024-04-26T23:37:07.739361+0000","last_unstale":"2024-04-26T23:37:13.796199+0000","last_undegraded":"2024-04-26T23:37:13.796199+0000","last_fullsized":"2024-04-26T23:37:13.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:55.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802956+0000","last_change":"2024-04-26T23:37:13.100935+0000","last_active":"2024-04-26T23:37:13.802956+0000","last_peered":"2024-04-26T23:37:13.802956+0000","last_clean":"2024-04-26T23:37:13.802956+0000","last_became_active":"2024-04-26T23:37:13.098932+0000","last_became_peered":"2024-04-26T23:37:13.098932+0000","last_unstale":"2024-04-26T23:37:13.802956+0000","last_undegraded":"2024-04-26T23:37:13.802956+0000","last_fullsized":"2024-04-26T23:37:13.802956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892457+0000","last_change":"2024-04-26T23:37:08.780897+0000","last_active":"2024-04-26T23:37:13.892457+0000","last_peered":"2024-04-26T23:37:13.892457+0000","last_clean":"2024-04-26T23:37:13.892457+0000","last_became_active":"2024-04-26T23:37:08.780573+0000","last_became_peered":"2024-04-26T23:37:08.780573+0000","last_unstale":"2024-04-26T23:37:13.892457+0000","last_undegraded":"2024-04-26T23:37:13.892457+0000","last_fullsized":"2024-04-26T23:37:13.892457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:27.933683+0000","last_change":"2024-04-26T23:37:07.743107+0000","last_active":"2024-04-26T23:37:27.933683+0000","last_peered":"2024-04-26T23:37:27.933683+0000","last_clean":"2024-04-26T23:37:27.933683+0000","last_became_active":"2024-04-26T23:37:07.742729+0000","last_became_peered":"2024-04-26T23:37:07.742729+0000","last_unstale":"2024-04-26T23:37:27.933683+0000","last_undegraded":"2024-04-26T23:37:27.933683+0000","last_fullsized":"2024-04-26T23:37:27.933683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888705+0000","last_change":"2024-04-26T23:37:12.824914+0000","last_active":"2024-04-26T23:37:13.888705+0000","last_peered":"2024-04-26T23:37:13.888705+0000","last_clean":"2024-04-26T23:37:13.888705+0000","last_became_active":"2024-04-26T23:37:12.824383+0000","last_became_peered":"2024-04-26T23:37:12.824383+0000","last_unstale":"2024-04-26T23:37:13.888705+0000","last_undegraded":"2024-04-26T23:37:13.888705+0000","last_fullsized":"2024-04-26T23:37:13.888705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801058+0000","last_change":"2024-04-26T23:37:08.785202+0000","last_active":"2024-04-26T23:37:13.801058+0000","last_peered":"2024-04-26T23:37:13.801058+0000","last_clean":"2024-04-26T23:37:13.801058+0000","last_became_active":"2024-04-26T23:37:08.779537+0000","last_became_peered":"2024-04-26T23:37:08.779537+0000","last_unstale":"2024-04-26T23:37:13.801058+0000","last_undegraded":"2024-04-26T23:37:13.801058+0000","last_fullsized":"2024-04-26T23:37:13.801058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.971268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889719+0000","last_change":"2024-04-26T23:37:07.741707+0000","last_active":"2024-04-26T23:37:13.889719+0000","last_peered":"2024-04-26T23:37:13.889719+0000","last_clean":"2024-04-26T23:37:13.889719+0000","last_became_active":"2024-04-26T23:37:07.740731+0000","last_became_peered":"2024-04-26T23:37:07.740731+0000","last_unstale":"2024-04-26T23:37:13.889719+0000","last_undegraded":"2024-04-26T23:37:13.889719+0000","last_fullsized":"2024-04-26T23:37:13.889719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798812+0000","last_change":"2024-04-26T23:37:12.818753+0000","last_active":"2024-04-26T23:37:13.798812+0000","last_peered":"2024-04-26T23:37:13.798812+0000","last_clean":"2024-04-26T23:37:13.798812+0000","last_became_active":"2024-04-26T23:37:12.817885+0000","last_became_peered":"2024-04-26T23:37:12.817885+0000","last_unstale":"2024-04-26T23:37:13.798812+0000","last_undegraded":"2024-04-26T23:37:13.798812+0000","last_fullsized":"2024-04-26T23:37:13.798812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:06.283131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795457+0000","last_change":"2024-04-26T23:37:08.796476+0000","last_active":"2024-04-26T23:37:13.795457+0000","last_peered":"2024-04-26T23:37:13.795457+0000","last_clean":"2024-04-26T23:37:13.795457+0000","last_became_active":"2024-04-26T23:37:08.795730+0000","last_became_peered":"2024-04-26T23:37:08.795730+0000","last_unstale":"2024-04-26T23:37:13.795457+0000","last_undegraded":"2024-04-26T23:37:13.795457+0000","last_fullsized":"2024-04-26T23:37:13.795457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802755+0000","last_change":"2024-04-26T23:37:07.746232+0000","last_active":"2024-04-26T23:37:13.802755+0000","last_peered":"2024-04-26T23:37:13.802755+0000","last_clean":"2024-04-26T23:37:13.802755+0000","last_became_active":"2024-04-26T23:37:07.745874+0000","last_became_peered":"2024-04-26T23:37:07.745874+0000","last_unstale":"2024-04-26T23:37:13.802755+0000","last_undegraded":"2024-04-26T23:37:13.802755+0000","last_fullsized":"2024-04-26T23:37:13.802755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891356+0000","last_change":"2024-04-26T23:37:12.834267+0000","last_active":"2024-04-26T23:37:13.891356+0000","last_peered":"2024-04-26T23:37:13.891356+0000","last_clean":"2024-04-26T23:37:13.891356+0000","last_became_active":"2024-04-26T23:37:12.829458+0000","last_became_peered":"2024-04-26T23:37:12.829458+0000","last_unstale":"2024-04-26T23:37:13.891356+0000","last_undegraded":"2024-04-26T23:37:13.891356+0000","last_fullsized":"2024-04-26T23:37:13.891356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:05.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802727+0000","last_change":"2024-04-26T23:37:08.788911+0000","last_active":"2024-04-26T23:37:13.802727+0000","last_peered":"2024-04-26T23:37:13.802727+0000","last_clean":"2024-04-26T23:37:13.802727+0000","last_became_active":"2024-04-26T23:37:08.785658+0000","last_became_peered":"2024-04-26T23:37:08.785658+0000","last_unstale":"2024-04-26T23:37:13.802727+0000","last_undegraded":"2024-04-26T23:37:13.802727+0000","last_fullsized":"2024-04-26T23:37:13.802727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892184+0000","last_change":"2024-04-26T23:37:07.750014+0000","last_active":"2024-04-26T23:37:13.892184+0000","last_peered":"2024-04-26T23:37:13.892184+0000","last_clean":"2024-04-26T23:37:13.892184+0000","last_became_active":"2024-04-26T23:37:07.748991+0000","last_became_peered":"2024-04-26T23:37:07.748991+0000","last_unstale":"2024-04-26T23:37:13.892184+0000","last_undegraded":"2024-04-26T23:37:13.892184+0000","last_fullsized":"2024-04-26T23:37:13.892184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890500+0000","last_change":"2024-04-26T23:37:13.096386+0000","last_active":"2024-04-26T23:37:13.890500+0000","last_peered":"2024-04-26T23:37:13.890500+0000","last_clean":"2024-04-26T23:37:13.890500+0000","last_became_active":"2024-04-26T23:37:13.095901+0000","last_became_peered":"2024-04-26T23:37:13.095901+0000","last_unstale":"2024-04-26T23:37:13.890500+0000","last_undegraded":"2024-04-26T23:37:13.890500+0000","last_fullsized":"2024-04-26T23:37:13.890500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:22.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795244+0000","last_change":"2024-04-26T23:37:13.100870+0000","last_active":"2024-04-26T23:37:13.795244+0000","last_peered":"2024-04-26T23:37:13.795244+0000","last_clean":"2024-04-26T23:37:13.795244+0000","last_became_active":"2024-04-26T23:37:13.094872+0000","last_became_peered":"2024-04-26T23:37:13.094872+0000","last_unstale":"2024-04-26T23:37:13.795244+0000","last_undegraded":"2024-04-26T23:37:13.795244+0000","last_fullsized":"2024-04-26T23:37:13.795244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888033+0000","last_change":"2024-04-26T23:37:08.770189+0000","last_active":"2024-04-26T23:37:13.888033+0000","last_peered":"2024-04-26T23:37:13.888033+0000","last_clean":"2024-04-26T23:37:13.888033+0000","last_became_active":"2024-04-26T23:37:08.769911+0000","last_became_peered":"2024-04-26T23:37:08.769911+0000","last_unstale":"2024-04-26T23:37:13.888033+0000","last_undegraded":"2024-04-26T23:37:13.888033+0000","last_fullsized":"2024-04-26T23:37:13.888033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800060+0000","last_change":"2024-04-26T23:37:07.745063+0000","last_active":"2024-04-26T23:37:13.800060+0000","last_peered":"2024-04-26T23:37:13.800060+0000","last_clean":"2024-04-26T23:37:13.800060+0000","last_became_active":"2024-04-26T23:37:07.744383+0000","last_became_peered":"2024-04-26T23:37:07.744383+0000","last_unstale":"2024-04-26T23:37:13.800060+0000","last_undegraded":"2024-04-26T23:37:13.800060+0000","last_fullsized":"2024-04-26T23:37:13.800060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890337+0000","last_change":"2024-04-26T23:37:12.824766+0000","last_active":"2024-04-26T23:37:13.890337+0000","last_peered":"2024-04-26T23:37:13.890337+0000","last_clean":"2024-04-26T23:37:13.890337+0000","last_became_active":"2024-04-26T23:37:12.824314+0000","last_became_peered":"2024-04-26T23:37:12.824314+0000","last_unstale":"2024-04-26T23:37:13.890337+0000","last_undegraded":"2024-04-26T23:37:13.890337+0000","last_fullsized":"2024-04-26T23:37:13.890337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889801+0000","last_change":"2024-04-26T23:37:08.769161+0000","last_active":"2024-04-26T23:37:13.889801+0000","last_peered":"2024-04-26T23:37:13.889801+0000","last_clean":"2024-04-26T23:37:13.889801+0000","last_became_active":"2024-04-26T23:37:08.767929+0000","last_became_peered":"2024-04-26T23:37:08.767929+0000","last_unstale":"2024-04-26T23:37:13.889801+0000","last_undegraded":"2024-04-26T23:37:13.889801+0000","last_fullsized":"2024-04-26T23:37:13.889801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890886+0000","last_change":"2024-04-26T23:37:07.739492+0000","last_active":"2024-04-26T23:37:13.890886+0000","last_peered":"2024-04-26T23:37:13.890886+0000","last_clean":"2024-04-26T23:37:13.890886+0000","last_became_active":"2024-04-26T23:37:07.739185+0000","last_became_peered":"2024-04-26T23:37:07.739185+0000","last_unstale":"2024-04-26T23:37:13.890886+0000","last_undegraded":"2024-04-26T23:37:13.890886+0000","last_fullsized":"2024-04-26T23:37:13.890886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890373+0000","last_change":"2024-04-26T23:37:07.750874+0000","last_active":"2024-04-26T23:37:13.890373+0000","last_peered":"2024-04-26T23:37:13.890373+0000","last_clean":"2024-04-26T23:37:13.890373+0000","last_became_active":"2024-04-26T23:37:07.748284+0000","last_became_peered":"2024-04-26T23:37:07.748284+0000","last_unstale":"2024-04-26T23:37:13.890373+0000","last_undegraded":"2024-04-26T23:37:13.890373+0000","last_fullsized":"2024-04-26T23:37:13.890373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796082+0000","last_change":"2024-04-26T23:37:08.753110+0000","last_active":"2024-04-26T23:37:13.796082+0000","last_peered":"2024-04-26T23:37:13.796082+0000","last_clean":"2024-04-26T23:37:13.796082+0000","last_became_active":"2024-04-26T23:37:08.752712+0000","last_became_peered":"2024-04-26T23:37:08.752712+0000","last_unstale":"2024-04-26T23:37:13.796082+0000","last_undegraded":"2024-04-26T23:37:13.796082+0000","last_fullsized":"2024-04-26T23:37:13.796082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889412+0000","last_change":"2024-04-26T23:37:12.829960+0000","last_active":"2024-04-26T23:37:13.889412+0000","last_peered":"2024-04-26T23:37:13.889412+0000","last_clean":"2024-04-26T23:37:13.889412+0000","last_became_active":"2024-04-26T23:37:12.829520+0000","last_became_peered":"2024-04-26T23:37:12.829520+0000","last_unstale":"2024-04-26T23:37:13.889412+0000","last_undegraded":"2024-04-26T23:37:13.889412+0000","last_fullsized":"2024-04-26T23:37:13.889412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:05.418963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891289+0000","last_change":"2024-04-26T23:37:07.747637+0000","last_active":"2024-04-26T23:37:13.891289+0000","last_peered":"2024-04-26T23:37:13.891289+0000","last_clean":"2024-04-26T23:37:13.891289+0000","last_became_active":"2024-04-26T23:37:07.747308+0000","last_became_peered":"2024-04-26T23:37:07.747308+0000","last_unstale":"2024-04-26T23:37:13.891289+0000","last_undegraded":"2024-04-26T23:37:13.891289+0000","last_fullsized":"2024-04-26T23:37:13.891289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798586+0000","last_change":"2024-04-26T23:37:08.763567+0000","last_active":"2024-04-26T23:37:13.798586+0000","last_peered":"2024-04-26T23:37:13.798586+0000","last_clean":"2024-04-26T23:37:13.798586+0000","last_became_active":"2024-04-26T23:37:08.762771+0000","last_became_peered":"2024-04-26T23:37:08.762771+0000","last_unstale":"2024-04-26T23:37:13.798586+0000","last_undegraded":"2024-04-26T23:37:13.798586+0000","last_fullsized":"2024-04-26T23:37:13.798586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:19.641900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892195+0000","last_change":"2024-04-26T23:37:12.832076+0000","last_active":"2024-04-26T23:37:13.892195+0000","last_peered":"2024-04-26T23:37:13.892195+0000","last_clean":"2024-04-26T23:37:13.892195+0000","last_became_active":"2024-04-26T23:37:12.830641+0000","last_became_peered":"2024-04-26T23:37:12.830641+0000","last_unstale":"2024-04-26T23:37:13.892195+0000","last_undegraded":"2024-04-26T23:37:13.892195+0000","last_fullsized":"2024-04-26T23:37:13.892195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:21.722510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889621+0000","last_change":"2024-04-26T23:37:08.768406+0000","last_active":"2024-04-26T23:37:13.889621+0000","last_peered":"2024-04-26T23:37:13.889621+0000","last_clean":"2024-04-26T23:37:13.889621+0000","last_became_active":"2024-04-26T23:37:08.767730+0000","last_became_peered":"2024-04-26T23:37:08.767730+0000","last_unstale":"2024-04-26T23:37:13.889621+0000","last_undegraded":"2024-04-26T23:37:13.889621+0000","last_fullsized":"2024-04-26T23:37:13.889621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888136+0000","last_change":"2024-04-26T23:37:08.780725+0000","last_active":"2024-04-26T23:37:13.888136+0000","last_peered":"2024-04-26T23:37:13.888136+0000","last_clean":"2024-04-26T23:37:13.888136+0000","last_became_active":"2024-04-26T23:37:08.780362+0000","last_became_peered":"2024-04-26T23:37:08.780362+0000","last_unstale":"2024-04-26T23:37:13.888136+0000","last_undegraded":"2024-04-26T23:37:13.888136+0000","last_fullsized":"2024-04-26T23:37:13.888136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801811+0000","last_change":"2024-04-26T23:37:08.789772+0000","last_active":"2024-04-26T23:37:13.801811+0000","last_peered":"2024-04-26T23:37:13.801811+0000","last_clean":"2024-04-26T23:37:13.801811+0000","last_became_active":"2024-04-26T23:37:08.785818+0000","last_became_peered":"2024-04-26T23:37:08.785818+0000","last_unstale":"2024-04-26T23:37:13.801811+0000","last_undegraded":"2024-04-26T23:37:13.801811+0000","last_fullsized":"2024-04-26T23:37:13.801811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795710+0000","last_change":"2024-04-26T23:37:08.768747+0000","last_active":"2024-04-26T23:37:13.795710+0000","last_peered":"2024-04-26T23:37:13.795710+0000","last_clean":"2024-04-26T23:37:13.795710+0000","last_became_active":"2024-04-26T23:37:08.768413+0000","last_became_peered":"2024-04-26T23:37:08.768413+0000","last_unstale":"2024-04-26T23:37:13.795710+0000","last_undegraded":"2024-04-26T23:37:13.795710+0000","last_fullsized":"2024-04-26T23:37:13.795710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891826+0000","last_change":"2024-04-26T23:37:08.788540+0000","last_active":"2024-04-26T23:37:13.891826+0000","last_peered":"2024-04-26T23:37:13.891826+0000","last_clean":"2024-04-26T23:37:13.891826+0000","last_became_active":"2024-04-26T23:37:08.788200+0000","last_became_peered":"2024-04-26T23:37:08.788200+0000","last_unstale":"2024-04-26T23:37:13.891826+0000","last_undegraded":"2024-04-26T23:37:13.891826+0000","last_fullsized":"2024-04-26T23:37:13.891826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891950+0000","last_change":"2024-04-26T23:37:08.777509+0000","last_active":"2024-04-26T23:37:13.891950+0000","last_peered":"2024-04-26T23:37:13.891950+0000","last_clean":"2024-04-26T23:37:13.891950+0000","last_became_active":"2024-04-26T23:37:08.777061+0000","last_became_peered":"2024-04-26T23:37:08.777061+0000","last_unstale":"2024-04-26T23:37:13.891950+0000","last_undegraded":"2024-04-26T23:37:13.891950+0000","last_fullsized":"2024-04-26T23:37:13.891950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891626+0000","last_change":"2024-04-26T23:37:08.774973+0000","last_active":"2024-04-26T23:37:13.891626+0000","last_peered":"2024-04-26T23:37:13.891626+0000","last_clean":"2024-04-26T23:37:13.891626+0000","last_became_active":"2024-04-26T23:37:08.774707+0000","last_became_peered":"2024-04-26T23:37:08.774707+0000","last_unstale":"2024-04-26T23:37:13.891626+0000","last_undegraded":"2024-04-26T23:37:13.891626+0000","last_fullsized":"2024-04-26T23:37:13.891626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892661+0000","last_change":"2024-04-26T23:37:08.781269+0000","last_active":"2024-04-26T23:37:13.892661+0000","last_peered":"2024-04-26T23:37:13.892661+0000","last_clean":"2024-04-26T23:37:13.892661+0000","last_became_active":"2024-04-26T23:37:08.780337+0000","last_became_peered":"2024-04-26T23:37:08.780337+0000","last_unstale":"2024-04-26T23:37:13.892661+0000","last_undegraded":"2024-04-26T23:37:13.892661+0000","last_fullsized":"2024-04-26T23:37:13.892661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799802+0000","last_change":"2024-04-26T23:37:08.771877+0000","last_active":"2024-04-26T23:37:13.799802+0000","last_peered":"2024-04-26T23:37:13.799802+0000","last_clean":"2024-04-26T23:37:13.799802+0000","last_became_active":"2024-04-26T23:37:08.770810+0000","last_became_peered":"2024-04-26T23:37:08.770810+0000","last_unstale":"2024-04-26T23:37:13.799802+0000","last_undegraded":"2024-04-26T23:37:13.799802+0000","last_fullsized":"2024-04-26T23:37:13.799802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890670+0000","last_change":"2024-04-26T23:37:08.780735+0000","last_active":"2024-04-26T23:37:13.890670+0000","last_peered":"2024-04-26T23:37:13.890670+0000","last_clean":"2024-04-26T23:37:13.890670+0000","last_became_active":"2024-04-26T23:37:08.780384+0000","last_became_peered":"2024-04-26T23:37:08.780384+0000","last_unstale":"2024-04-26T23:37:13.890670+0000","last_undegraded":"2024-04-26T23:37:13.890670+0000","last_fullsized":"2024-04-26T23:37:13.890670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801555+0000","last_change":"2024-04-26T23:37:08.790073+0000","last_active":"2024-04-26T23:37:13.801555+0000","last_peered":"2024-04-26T23:37:13.801555+0000","last_clean":"2024-04-26T23:37:13.801555+0000","last_became_active":"2024-04-26T23:37:08.789431+0000","last_became_peered":"2024-04-26T23:37:08.789431+0000","last_unstale":"2024-04-26T23:37:13.801555+0000","last_undegraded":"2024-04-26T23:37:13.801555+0000","last_fullsized":"2024-04-26T23:37:13.801555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795276+0000","last_change":"2024-04-26T23:37:08.756179+0000","last_active":"2024-04-26T23:37:13.795276+0000","last_peered":"2024-04-26T23:37:13.795276+0000","last_clean":"2024-04-26T23:37:13.795276+0000","last_became_active":"2024-04-26T23:37:08.755805+0000","last_became_peered":"2024-04-26T23:37:08.755805+0000","last_unstale":"2024-04-26T23:37:13.795276+0000","last_undegraded":"2024-04-26T23:37:13.795276+0000","last_fullsized":"2024-04-26T23:37:13.795276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891285+0000","last_change":"2024-04-26T23:37:08.768542+0000","last_active":"2024-04-26T23:37:13.891285+0000","last_peered":"2024-04-26T23:37:13.891285+0000","last_clean":"2024-04-26T23:37:13.891285+0000","last_became_active":"2024-04-26T23:37:08.765841+0000","last_became_peered":"2024-04-26T23:37:08.765841+0000","last_unstale":"2024-04-26T23:37:13.891285+0000","last_undegraded":"2024-04-26T23:37:13.891285+0000","last_fullsized":"2024-04-26T23:37:13.891285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893105+0000","last_change":"2024-04-26T23:37:08.775312+0000","last_active":"2024-04-26T23:37:13.893105+0000","last_peered":"2024-04-26T23:37:13.893105+0000","last_clean":"2024-04-26T23:37:13.893105+0000","last_became_active":"2024-04-26T23:37:08.774944+0000","last_became_peered":"2024-04-26T23:37:08.774944+0000","last_unstale":"2024-04-26T23:37:13.893105+0000","last_undegraded":"2024-04-26T23:37:13.893105+0000","last_fullsized":"2024-04-26T23:37:13.893105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891448+0000","last_change":"2024-04-26T23:37:08.772525+0000","last_active":"2024-04-26T23:37:13.891448+0000","last_peered":"2024-04-26T23:37:13.891448+0000","last_clean":"2024-04-26T23:37:13.891448+0000","last_became_active":"2024-04-26T23:37:08.772177+0000","last_became_peered":"2024-04-26T23:37:08.772177+0000","last_unstale":"2024-04-26T23:37:13.891448+0000","last_undegraded":"2024-04-26T23:37:13.891448+0000","last_fullsized":"2024-04-26T23:37:13.891448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:55.136157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803582+0000","last_change":"2024-04-26T23:37:08.790236+0000","last_active":"2024-04-26T23:37:13.803582+0000","last_peered":"2024-04-26T23:37:13.803582+0000","last_clean":"2024-04-26T23:37:13.803582+0000","last_became_active":"2024-04-26T23:37:08.789885+0000","last_became_peered":"2024-04-26T23:37:08.789885+0000","last_unstale":"2024-04-26T23:37:13.803582+0000","last_undegraded":"2024-04-26T23:37:13.803582+0000","last_fullsized":"2024-04-26T23:37:13.803582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799434+0000","last_change":"2024-04-26T23:37:08.780732+0000","last_active":"2024-04-26T23:37:13.799434+0000","last_peered":"2024-04-26T23:37:13.799434+0000","last_clean":"2024-04-26T23:37:13.799434+0000","last_became_active":"2024-04-26T23:37:08.779046+0000","last_became_peered":"2024-04-26T23:37:08.779046+0000","last_unstale":"2024-04-26T23:37:13.799434+0000","last_undegraded":"2024-04-26T23:37:13.799434+0000","last_fullsized":"2024-04-26T23:37:13.799434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888183+0000","last_change":"2024-04-26T23:37:08.751809+0000","last_active":"2024-04-26T23:37:13.888183+0000","last_peered":"2024-04-26T23:37:13.888183+0000","last_clean":"2024-04-26T23:37:13.888183+0000","last_became_active":"2024-04-26T23:37:08.750955+0000","last_became_peered":"2024-04-26T23:37:08.750955+0000","last_unstale":"2024-04-26T23:37:13.888183+0000","last_undegraded":"2024-04-26T23:37:13.888183+0000","last_fullsized":"2024-04-26T23:37:13.888183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795084+0000","last_change":"2024-04-26T23:37:08.762870+0000","last_active":"2024-04-26T23:37:13.795084+0000","last_peered":"2024-04-26T23:37:13.795084+0000","last_clean":"2024-04-26T23:37:13.795084+0000","last_became_active":"2024-04-26T23:37:08.762554+0000","last_became_peered":"2024-04-26T23:37:08.762554+0000","last_unstale":"2024-04-26T23:37:13.795084+0000","last_undegraded":"2024-04-26T23:37:13.795084+0000","last_fullsized":"2024-04-26T23:37:13.795084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:05.526812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891059+0000","last_change":"2024-04-26T23:37:08.774910+0000","last_active":"2024-04-26T23:37:13.891059+0000","last_peered":"2024-04-26T23:37:13.891059+0000","last_clean":"2024-04-26T23:37:13.891059+0000","last_became_active":"2024-04-26T23:37:08.774587+0000","last_became_peered":"2024-04-26T23:37:08.774587+0000","last_unstale":"2024-04-26T23:37:13.891059+0000","last_undegraded":"2024-04-26T23:37:13.891059+0000","last_fullsized":"2024-04-26T23:37:13.891059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800959+0000","last_change":"2024-04-26T23:37:08.778389+0000","last_active":"2024-04-26T23:37:13.800959+0000","last_peered":"2024-04-26T23:37:13.800959+0000","last_clean":"2024-04-26T23:37:13.800959+0000","last_became_active":"2024-04-26T23:37:08.778116+0000","last_became_peered":"2024-04-26T23:37:08.778116+0000","last_unstale":"2024-04-26T23:37:13.800959+0000","last_undegraded":"2024-04-26T23:37:13.800959+0000","last_fullsized":"2024-04-26T23:37:13.800959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892362+0000","last_change":"2024-04-26T23:37:08.781929+0000","last_active":"2024-04-26T23:37:13.892362+0000","last_peered":"2024-04-26T23:37:13.892362+0000","last_clean":"2024-04-26T23:37:13.892362+0000","last_became_active":"2024-04-26T23:37:08.780724+0000","last_became_peered":"2024-04-26T23:37:08.780724+0000","last_unstale":"2024-04-26T23:37:13.892362+0000","last_undegraded":"2024-04-26T23:37:13.892362+0000","last_fullsized":"2024-04-26T23:37:13.892362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891324+0000","last_change":"2024-04-26T23:37:08.792579+0000","last_active":"2024-04-26T23:37:13.891324+0000","last_peered":"2024-04-26T23:37:13.891324+0000","last_clean":"2024-04-26T23:37:13.891324+0000","last_became_active":"2024-04-26T23:37:08.792171+0000","last_became_peered":"2024-04-26T23:37:08.792171+0000","last_unstale":"2024-04-26T23:37:13.891324+0000","last_undegraded":"2024-04-26T23:37:13.891324+0000","last_fullsized":"2024-04-26T23:37:13.891324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893856+0000","last_change":"2024-04-26T23:37:08.795179+0000","last_active":"2024-04-26T23:37:13.893856+0000","last_peered":"2024-04-26T23:37:13.893856+0000","last_clean":"2024-04-26T23:37:13.893856+0000","last_became_active":"2024-04-26T23:37:08.794890+0000","last_became_peered":"2024-04-26T23:37:08.794890+0000","last_unstale":"2024-04-26T23:37:13.893856+0000","last_undegraded":"2024-04-26T23:37:13.893856+0000","last_fullsized":"2024-04-26T23:37:13.893856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889647+0000","last_change":"2024-04-26T23:37:08.786739+0000","last_active":"2024-04-26T23:37:13.889647+0000","last_peered":"2024-04-26T23:37:13.889647+0000","last_clean":"2024-04-26T23:37:13.889647+0000","last_became_active":"2024-04-26T23:37:08.786360+0000","last_became_peered":"2024-04-26T23:37:08.786360+0000","last_unstale":"2024-04-26T23:37:13.889647+0000","last_undegraded":"2024-04-26T23:37:13.889647+0000","last_fullsized":"2024-04-26T23:37:13.889647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891212+0000","last_change":"2024-04-26T23:37:08.775748+0000","last_active":"2024-04-26T23:37:13.891212+0000","last_peered":"2024-04-26T23:37:13.891212+0000","last_clean":"2024-04-26T23:37:13.891212+0000","last_became_active":"2024-04-26T23:37:08.775440+0000","last_became_peered":"2024-04-26T23:37:08.775440+0000","last_unstale":"2024-04-26T23:37:13.891212+0000","last_undegraded":"2024-04-26T23:37:13.891212+0000","last_fullsized":"2024-04-26T23:37:13.891212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801324+0000","last_change":"2024-04-26T23:37:08.791665+0000","last_active":"2024-04-26T23:37:13.801324+0000","last_peered":"2024-04-26T23:37:13.801324+0000","last_clean":"2024-04-26T23:37:13.801324+0000","last_became_active":"2024-04-26T23:37:08.790305+0000","last_became_peered":"2024-04-26T23:37:08.790305+0000","last_unstale":"2024-04-26T23:37:13.801324+0000","last_undegraded":"2024-04-26T23:37:13.801324+0000","last_fullsized":"2024-04-26T23:37:13.801324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891978+0000","last_change":"2024-04-26T23:37:08.776000+0000","last_active":"2024-04-26T23:37:13.891978+0000","last_peered":"2024-04-26T23:37:13.891978+0000","last_clean":"2024-04-26T23:37:13.891978+0000","last_became_active":"2024-04-26T23:37:08.773615+0000","last_became_peered":"2024-04-26T23:37:08.773615+0000","last_unstale":"2024-04-26T23:37:13.891978+0000","last_undegraded":"2024-04-26T23:37:13.891978+0000","last_fullsized":"2024-04-26T23:37:13.891978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802456+0000","last_change":"2024-04-26T23:37:08.789506+0000","last_active":"2024-04-26T23:37:13.802456+0000","last_peered":"2024-04-26T23:37:13.802456+0000","last_clean":"2024-04-26T23:37:13.802456+0000","last_became_active":"2024-04-26T23:37:08.789180+0000","last_became_peered":"2024-04-26T23:37:08.789180+0000","last_unstale":"2024-04-26T23:37:13.802456+0000","last_undegraded":"2024-04-26T23:37:13.802456+0000","last_fullsized":"2024-04-26T23:37:13.802456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794898+0000","last_change":"2024-04-26T23:37:08.766757+0000","last_active":"2024-04-26T23:37:13.794898+0000","last_peered":"2024-04-26T23:37:13.794898+0000","last_clean":"2024-04-26T23:37:13.794898+0000","last_became_active":"2024-04-26T23:37:08.761659+0000","last_became_peered":"2024-04-26T23:37:08.761659+0000","last_unstale":"2024-04-26T23:37:13.794898+0000","last_undegraded":"2024-04-26T23:37:13.794898+0000","last_fullsized":"2024-04-26T23:37:13.794898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890719+0000","last_change":"2024-04-26T23:37:08.761968+0000","last_active":"2024-04-26T23:37:13.890719+0000","last_peered":"2024-04-26T23:37:13.890719+0000","last_clean":"2024-04-26T23:37:13.890719+0000","last_became_active":"2024-04-26T23:37:08.761546+0000","last_became_peered":"2024-04-26T23:37:08.761546+0000","last_unstale":"2024-04-26T23:37:13.890719+0000","last_undegraded":"2024-04-26T23:37:13.890719+0000","last_fullsized":"2024-04-26T23:37:13.890719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893557+0000","last_change":"2024-04-26T23:37:08.775290+0000","last_active":"2024-04-26T23:37:13.893557+0000","last_peered":"2024-04-26T23:37:13.893557+0000","last_clean":"2024-04-26T23:37:13.893557+0000","last_became_active":"2024-04-26T23:37:08.774646+0000","last_became_peered":"2024-04-26T23:37:08.774646+0000","last_unstale":"2024-04-26T23:37:13.893557+0000","last_undegraded":"2024-04-26T23:37:13.893557+0000","last_fullsized":"2024-04-26T23:37:13.893557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:37:44.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:37:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.270+0000 7f5939400640 1 Processor -- start 2024-04-26T23:37:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.270+0000 7f5939400640 1 -- start start 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5939400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f593415f4e0 0x7f59340d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5939400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59340d2130 0x7f59340b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5939400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59340d11f0 con 0x7f59340b4000 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5939400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f59340d0ce0 con 0x7f59340d2130 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5939400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f59340d2990 con 0x7f593415f4e0 2024-04-26T23:37:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59340d2130 0x7f59340b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59340d2130 0x7f59340b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39342/0 (socket says 172.21.15.46:39342) 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 -- 172.21.15.46:0/3161354417 learned_addr learned my addr 172.21.15.46:0/3161354417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5933e00640 1 --2- 172.21.15.46:0/3161354417 >> v2:172.21.15.142:3300/0 conn(0x7f593415f4e0 0x7f59340d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 -- 172.21.15.46:0/3161354417 >> v2:172.21.15.142:3300/0 conn(0x7f593415f4e0 msgr2=0x7f59340d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 --2- 172.21.15.46:0/3161354417 >> v2:172.21.15.142:3300/0 conn(0x7f593415f4e0 0x7f59340d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.272+0000 7f5938a00640 1 -- 172.21.15.46:0/3161354417 >> v1:172.21.15.142:6789/0 conn(0x7f59340b4000 legacy=0x7f59340d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.273+0000 7f5938a00640 1 -- 172.21.15.46:0/3161354417 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340d5f20 con 0x7f59340d2130 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5938a00640 1 --2- 172.21.15.46:0/3161354417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59340d2130 0x7f59340b2280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f593408d8b0 tx=0x7f592c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b67c2c124a8c534 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5932a00640 1 -- 172.21.15.46:0/3161354417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c036070 con 0x7f59340d2130 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5932a00640 1 -- 172.21.15.46:0/3161354417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c031600 con 0x7f59340d2130 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5932a00640 1 -- 172.21.15.46:0/3161354417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0319d0 con 0x7f59340d2130 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5939400640 1 -- 172.21.15.46:0/3161354417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59340d2130 msgr2=0x7f59340b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.274+0000 7f5939400640 1 --2- 172.21.15.46:0/3161354417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59340d2130 0x7f59340b2280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f593408d8b0 tx=0x7f592c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 -- 172.21.15.46:0/3161354417 shutdown_connections 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 --2- 172.21.15.46:0/3161354417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59340d2130 0x7f59340b2280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 --2- 172.21.15.46:0/3161354417 >> v2:172.21.15.142:3300/0 conn(0x7f593415f4e0 0x7f59340d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 -- 172.21.15.46:0/3161354417 >> 172.21.15.46:0/3161354417 conn(0x7f59340d5010 msgr2=0x7f59340d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 -- 172.21.15.46:0/3161354417 shutdown_connections 2024-04-26T23:37:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.275+0000 7f5939400640 1 -- 172.21.15.46:0/3161354417 wait complete. 2024-04-26T23:37:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.276+0000 7f5939400640 1 Processor -- start 2024-04-26T23:37:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.276+0000 7f5939400640 1 -- start start 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 0x7f59340ba440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59340d2130 0x7f59340b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59340b1d60 con 0x7f593415f4e0 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59340d5c40 con 0x7f59340b4000 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.277+0000 7f5939400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f593415f8c0 con 0x7f59340d2130 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39352/0 (socket says 172.21.15.46:39352) 2024-04-26T23:37:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 -- 172.21.15.46:0/3213662365 learned_addr learned my addr 172.21.15.46:0/3213662365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5933e00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 0x7f59340ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5933400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59340d2130 0x7f59340b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 -- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59340d2130 msgr2=0x7f59340b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59340d2130 0x7f59340b9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 -- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 msgr2=0x7f59340ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 0x7f59340ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 -- 172.21.15.46:0/3213662365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5934160b20 con 0x7f593415f4e0 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5933e00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 0x7f59340ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.278+0000 7f5938a00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f592c034040 tx=0x7f592c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.279+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c036050 con 0x7f593415f4e0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.279+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c042070 con 0x7f593415f4e0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.279+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c009510 con 0x7f593415f4e0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.279+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5934160dc0 con 0x7f593415f4e0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.279+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5934161380 con 0x7f593415f4e0 2024-04-26T23:37:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.280+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59340693b0 con 0x7f593415f4e0 2024-04-26T23:37:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.281+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f592c007bd0 con 0x7f593415f4e0 2024-04-26T23:37:44.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.282+0000 7f592be00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 0x7f591406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.282+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f592c0ace50 con 0x7f593415f4e0 2024-04-26T23:37:44.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.283+0000 7f5933e00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 0x7f591406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.284+0000 7f5933e00640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 0x7f591406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5924006fd0 tx=0x7f5924008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.287+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f592c045360 con 0x7f593415f4e0 2024-04-26T23:37:44.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.477+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59340a2a80 con 0x7f591406c8f0 2024-04-26T23:37:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.503+0000 7f592be00640 1 -- 172.21.15.46:0/3213662365 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332535 (secure 0 0 0) 0x7f59340a2a80 con 0x7f591406c8f0 2024-04-26T23:37:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:37:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 msgr2=0x7f591406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 0x7f591406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5924006fd0 tx=0x7f5924008040 comp rx=0 tx=0).stop 2024-04-26T23:37:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 msgr2=0x7f593406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f592c034040 tx=0x7f592c004a40 comp rx=0 tx=0).stop 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 shutdown_connections 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f591406c8f0 0x7f591406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593415f4e0 0x7f593406dff0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59340d2130 0x7f59340b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 --2- 172.21.15.46:0/3213662365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59340b4000 0x7f59340ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 >> 172.21.15.46:0/3213662365 conn(0x7f59340d5010 msgr2=0x7f593415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 shutdown_connections 2024-04-26T23:37:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.511+0000 7f5939400640 1 -- 172.21.15.46:0/3213662365 wait complete. 2024-04-26T23:37:44.531 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-26T23:37:43.895489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223640,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751080,"statfs":{"total":773094113280,"available":772865105920,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005910"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795609+0000","last_change":"2024-04-26T23:37:07.740855+0000","last_active":"2024-04-26T23:37:13.795609+0000","last_peered":"2024-04-26T23:37:13.795609+0000","last_clean":"2024-04-26T23:37:13.795609+0000","last_became_active":"2024-04-26T23:37:07.739396+0000","last_became_peered":"2024-04-26T23:37:07.739396+0000","last_unstale":"2024-04-26T23:37:13.795609+0000","last_undegraded":"2024-04-26T23:37:13.795609+0000","last_fullsized":"2024-04-26T23:37:13.795609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:05.866636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892052+0000","last_change":"2024-04-26T23:37:08.773954+0000","last_active":"2024-04-26T23:37:13.892052+0000","last_peered":"2024-04-26T23:37:13.892052+0000","last_clean":"2024-04-26T23:37:13.892052+0000","last_became_active":"2024-04-26T23:37:08.771001+0000","last_became_peered":"2024-04-26T23:37:08.771001+0000","last_unstale":"2024-04-26T23:37:13.892052+0000","last_undegraded":"2024-04-26T23:37:13.892052+0000","last_fullsized":"2024-04-26T23:37:13.892052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890219+0000","last_change":"2024-04-26T23:37:13.098670+0000","last_active":"2024-04-26T23:37:13.890219+0000","last_peered":"2024-04-26T23:37:13.890219+0000","last_clean":"2024-04-26T23:37:13.890219+0000","last_became_active":"2024-04-26T23:37:13.098244+0000","last_became_peered":"2024-04-26T23:37:13.098244+0000","last_unstale":"2024-04-26T23:37:13.890219+0000","last_undegraded":"2024-04-26T23:37:13.890219+0000","last_fullsized":"2024-04-26T23:37:13.890219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892226+0000","last_change":"2024-04-26T23:37:07.729205+0000","last_active":"2024-04-26T23:37:13.892226+0000","last_peered":"2024-04-26T23:37:13.892226+0000","last_clean":"2024-04-26T23:37:13.892226+0000","last_became_active":"2024-04-26T23:37:07.728871+0000","last_became_peered":"2024-04-26T23:37:07.728871+0000","last_unstale":"2024-04-26T23:37:13.892226+0000","last_undegraded":"2024-04-26T23:37:13.892226+0000","last_fullsized":"2024-04-26T23:37:13.892226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795238+0000","last_change":"2024-04-26T23:37:08.793071+0000","last_active":"2024-04-26T23:37:13.795238+0000","last_peered":"2024-04-26T23:37:13.795238+0000","last_clean":"2024-04-26T23:37:13.795238+0000","last_became_active":"2024-04-26T23:37:08.792784+0000","last_became_peered":"2024-04-26T23:37:08.792784+0000","last_unstale":"2024-04-26T23:37:13.795238+0000","last_undegraded":"2024-04-26T23:37:13.795238+0000","last_fullsized":"2024-04-26T23:37:13.795238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798412+0000","last_change":"2024-04-26T23:37:12.830234+0000","last_active":"2024-04-26T23:37:13.798412+0000","last_peered":"2024-04-26T23:37:13.798412+0000","last_clean":"2024-04-26T23:37:13.798412+0000","last_became_active":"2024-04-26T23:37:12.829735+0000","last_became_peered":"2024-04-26T23:37:12.829735+0000","last_unstale":"2024-04-26T23:37:13.798412+0000","last_undegraded":"2024-04-26T23:37:13.798412+0000","last_fullsized":"2024-04-26T23:37:13.798412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:01.085576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893008+0000","last_change":"2024-04-26T23:37:07.755959+0000","last_active":"2024-04-26T23:37:13.893008+0000","last_peered":"2024-04-26T23:37:13.893008+0000","last_clean":"2024-04-26T23:37:13.893008+0000","last_became_active":"2024-04-26T23:37:07.754315+0000","last_became_peered":"2024-04-26T23:37:07.754315+0000","last_unstale":"2024-04-26T23:37:13.893008+0000","last_undegraded":"2024-04-26T23:37:13.893008+0000","last_fullsized":"2024-04-26T23:37:13.893008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802750+0000","last_change":"2024-04-26T23:37:08.786529+0000","last_active":"2024-04-26T23:37:13.802750+0000","last_peered":"2024-04-26T23:37:13.802750+0000","last_clean":"2024-04-26T23:37:13.802750+0000","last_became_active":"2024-04-26T23:37:08.781335+0000","last_became_peered":"2024-04-26T23:37:08.781335+0000","last_unstale":"2024-04-26T23:37:13.802750+0000","last_undegraded":"2024-04-26T23:37:13.802750+0000","last_fullsized":"2024-04-26T23:37:13.802750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798764+0000","last_change":"2024-04-26T23:37:12.830290+0000","last_active":"2024-04-26T23:37:13.798764+0000","last_peered":"2024-04-26T23:37:13.798764+0000","last_clean":"2024-04-26T23:37:13.798764+0000","last_became_active":"2024-04-26T23:37:12.829772+0000","last_became_peered":"2024-04-26T23:37:12.829772+0000","last_unstale":"2024-04-26T23:37:13.798764+0000","last_undegraded":"2024-04-26T23:37:13.798764+0000","last_fullsized":"2024-04-26T23:37:13.798764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:01.225018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892372+0000","last_change":"2024-04-26T23:37:07.756056+0000","last_active":"2024-04-26T23:37:13.892372+0000","last_peered":"2024-04-26T23:37:13.892372+0000","last_clean":"2024-04-26T23:37:13.892372+0000","last_became_active":"2024-04-26T23:37:07.754918+0000","last_became_peered":"2024-04-26T23:37:07.754918+0000","last_unstale":"2024-04-26T23:37:13.892372+0000","last_undegraded":"2024-04-26T23:37:13.892372+0000","last_fullsized":"2024-04-26T23:37:13.892372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:08.953262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803127+0000","last_change":"2024-04-26T23:37:08.780722+0000","last_active":"2024-04-26T23:37:13.803127+0000","last_peered":"2024-04-26T23:37:13.803127+0000","last_clean":"2024-04-26T23:37:13.803127+0000","last_became_active":"2024-04-26T23:37:08.780223+0000","last_became_peered":"2024-04-26T23:37:08.780223+0000","last_unstale":"2024-04-26T23:37:13.803127+0000","last_undegraded":"2024-04-26T23:37:13.803127+0000","last_fullsized":"2024-04-26T23:37:13.803127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798239+0000","last_change":"2024-04-26T23:37:12.831066+0000","last_active":"2024-04-26T23:37:13.798239+0000","last_peered":"2024-04-26T23:37:13.798239+0000","last_clean":"2024-04-26T23:37:13.798239+0000","last_became_active":"2024-04-26T23:37:12.829941+0000","last_became_peered":"2024-04-26T23:37:12.829941+0000","last_unstale":"2024-04-26T23:37:13.798239+0000","last_undegraded":"2024-04-26T23:37:13.798239+0000","last_fullsized":"2024-04-26T23:37:13.798239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:04.322071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890099+0000","last_change":"2024-04-26T23:37:08.768304+0000","last_active":"2024-04-26T23:37:13.890099+0000","last_peered":"2024-04-26T23:37:13.890099+0000","last_clean":"2024-04-26T23:37:13.890099+0000","last_became_active":"2024-04-26T23:37:08.767971+0000","last_became_peered":"2024-04-26T23:37:08.767971+0000","last_unstale":"2024-04-26T23:37:13.890099+0000","last_undegraded":"2024-04-26T23:37:13.890099+0000","last_fullsized":"2024-04-26T23:37:13.890099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892460+0000","last_change":"2024-04-26T23:37:07.755890+0000","last_active":"2024-04-26T23:37:13.892460+0000","last_peered":"2024-04-26T23:37:13.892460+0000","last_clean":"2024-04-26T23:37:13.892460+0000","last_became_active":"2024-04-26T23:37:07.753801+0000","last_became_peered":"2024-04-26T23:37:07.753801+0000","last_unstale":"2024-04-26T23:37:13.892460+0000","last_undegraded":"2024-04-26T23:37:13.892460+0000","last_fullsized":"2024-04-26T23:37:13.892460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800561+0000","last_change":"2024-04-26T23:37:12.821170+0000","last_active":"2024-04-26T23:37:13.800561+0000","last_peered":"2024-04-26T23:37:13.800561+0000","last_clean":"2024-04-26T23:37:13.800561+0000","last_became_active":"2024-04-26T23:37:12.820739+0000","last_became_peered":"2024-04-26T23:37:12.820739+0000","last_unstale":"2024-04-26T23:37:13.800561+0000","last_undegraded":"2024-04-26T23:37:13.800561+0000","last_fullsized":"2024-04-26T23:37:13.800561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891188+0000","last_change":"2024-04-26T23:37:08.772794+0000","last_active":"2024-04-26T23:37:13.891188+0000","last_peered":"2024-04-26T23:37:13.891188+0000","last_clean":"2024-04-26T23:37:13.891188+0000","last_became_active":"2024-04-26T23:37:08.770226+0000","last_became_peered":"2024-04-26T23:37:08.770226+0000","last_unstale":"2024-04-26T23:37:13.891188+0000","last_undegraded":"2024-04-26T23:37:13.891188+0000","last_fullsized":"2024-04-26T23:37:13.891188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888791+0000","last_change":"2024-04-26T23:37:07.739350+0000","last_active":"2024-04-26T23:37:13.888791+0000","last_peered":"2024-04-26T23:37:13.888791+0000","last_clean":"2024-04-26T23:37:13.888791+0000","last_became_active":"2024-04-26T23:37:07.739007+0000","last_became_peered":"2024-04-26T23:37:07.739007+0000","last_unstale":"2024-04-26T23:37:13.888791+0000","last_undegraded":"2024-04-26T23:37:13.888791+0000","last_fullsized":"2024-04-26T23:37:13.888791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800746+0000","last_change":"2024-04-26T23:37:12.831545+0000","last_active":"2024-04-26T23:37:13.800746+0000","last_peered":"2024-04-26T23:37:13.800746+0000","last_clean":"2024-04-26T23:37:13.800746+0000","last_became_active":"2024-04-26T23:37:12.829548+0000","last_became_peered":"2024-04-26T23:37:12.829548+0000","last_unstale":"2024-04-26T23:37:13.800746+0000","last_undegraded":"2024-04-26T23:37:13.800746+0000","last_fullsized":"2024-04-26T23:37:13.800746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:31.665337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889936+0000","last_change":"2024-04-26T23:37:08.761785+0000","last_active":"2024-04-26T23:37:13.889936+0000","last_peered":"2024-04-26T23:37:13.889936+0000","last_clean":"2024-04-26T23:37:13.889936+0000","last_became_active":"2024-04-26T23:37:08.759700+0000","last_became_peered":"2024-04-26T23:37:08.759700+0000","last_unstale":"2024-04-26T23:37:13.889936+0000","last_undegraded":"2024-04-26T23:37:13.889936+0000","last_fullsized":"2024-04-26T23:37:13.889936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888332+0000","last_change":"2024-04-26T23:37:07.738157+0000","last_active":"2024-04-26T23:37:13.888332+0000","last_peered":"2024-04-26T23:37:13.888332+0000","last_clean":"2024-04-26T23:37:13.888332+0000","last_became_active":"2024-04-26T23:37:07.737830+0000","last_became_peered":"2024-04-26T23:37:07.737830+0000","last_unstale":"2024-04-26T23:37:13.888332+0000","last_undegraded":"2024-04-26T23:37:13.888332+0000","last_fullsized":"2024-04-26T23:37:13.888332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892316+0000","last_change":"2024-04-26T23:37:13.099265+0000","last_active":"2024-04-26T23:37:13.892316+0000","last_peered":"2024-04-26T23:37:13.892316+0000","last_clean":"2024-04-26T23:37:13.892316+0000","last_became_active":"2024-04-26T23:37:13.095798+0000","last_became_peered":"2024-04-26T23:37:13.095798+0000","last_unstale":"2024-04-26T23:37:13.892316+0000","last_undegraded":"2024-04-26T23:37:13.892316+0000","last_fullsized":"2024-04-26T23:37:13.892316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800680+0000","last_change":"2024-04-26T23:37:08.764943+0000","last_active":"2024-04-26T23:37:13.800680+0000","last_peered":"2024-04-26T23:37:13.800680+0000","last_clean":"2024-04-26T23:37:13.800680+0000","last_became_active":"2024-04-26T23:37:08.764636+0000","last_became_peered":"2024-04-26T23:37:08.764636+0000","last_unstale":"2024-04-26T23:37:13.800680+0000","last_undegraded":"2024-04-26T23:37:13.800680+0000","last_fullsized":"2024-04-26T23:37:13.800680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:29.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890217+0000","last_change":"2024-04-26T23:37:07.753035+0000","last_active":"2024-04-26T23:37:13.890217+0000","last_peered":"2024-04-26T23:37:13.890217+0000","last_clean":"2024-04-26T23:37:13.890217+0000","last_became_active":"2024-04-26T23:37:07.752363+0000","last_became_peered":"2024-04-26T23:37:07.752363+0000","last_unstale":"2024-04-26T23:37:13.890217+0000","last_undegraded":"2024-04-26T23:37:13.890217+0000","last_fullsized":"2024-04-26T23:37:13.890217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799225+0000","last_change":"2024-04-26T23:37:12.830106+0000","last_active":"2024-04-26T23:37:13.799225+0000","last_peered":"2024-04-26T23:37:13.799225+0000","last_clean":"2024-04-26T23:37:13.799225+0000","last_became_active":"2024-04-26T23:37:12.829676+0000","last_became_peered":"2024-04-26T23:37:12.829676+0000","last_unstale":"2024-04-26T23:37:13.799225+0000","last_undegraded":"2024-04-26T23:37:13.799225+0000","last_fullsized":"2024-04-26T23:37:13.799225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795807+0000","last_change":"2024-04-26T23:37:08.764908+0000","last_active":"2024-04-26T23:37:13.795807+0000","last_peered":"2024-04-26T23:37:13.795807+0000","last_clean":"2024-04-26T23:37:13.795807+0000","last_became_active":"2024-04-26T23:37:08.764369+0000","last_became_peered":"2024-04-26T23:37:08.764369+0000","last_unstale":"2024-04-26T23:37:13.795807+0000","last_undegraded":"2024-04-26T23:37:13.795807+0000","last_fullsized":"2024-04-26T23:37:13.795807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892624+0000","last_change":"2024-04-26T23:37:07.751902+0000","last_active":"2024-04-26T23:37:13.892624+0000","last_peered":"2024-04-26T23:37:13.892624+0000","last_clean":"2024-04-26T23:37:13.892624+0000","last_became_active":"2024-04-26T23:37:07.751383+0000","last_became_peered":"2024-04-26T23:37:07.751383+0000","last_unstale":"2024-04-26T23:37:13.892624+0000","last_undegraded":"2024-04-26T23:37:13.892624+0000","last_fullsized":"2024-04-26T23:37:13.892624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800477+0000","last_change":"2024-04-26T23:37:13.090483+0000","last_active":"2024-04-26T23:37:13.800477+0000","last_peered":"2024-04-26T23:37:13.800477+0000","last_clean":"2024-04-26T23:37:13.800477+0000","last_became_active":"2024-04-26T23:37:13.088991+0000","last_became_peered":"2024-04-26T23:37:13.088991+0000","last_unstale":"2024-04-26T23:37:13.800477+0000","last_undegraded":"2024-04-26T23:37:13.800477+0000","last_fullsized":"2024-04-26T23:37:13.800477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892790+0000","last_change":"2024-04-26T23:37:08.777852+0000","last_active":"2024-04-26T23:37:13.892790+0000","last_peered":"2024-04-26T23:37:13.892790+0000","last_clean":"2024-04-26T23:37:13.892790+0000","last_became_active":"2024-04-26T23:37:08.777214+0000","last_became_peered":"2024-04-26T23:37:08.777214+0000","last_unstale":"2024-04-26T23:37:13.892790+0000","last_undegraded":"2024-04-26T23:37:13.892790+0000","last_fullsized":"2024-04-26T23:37:13.892790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890844+0000","last_change":"2024-04-26T23:37:07.744896+0000","last_active":"2024-04-26T23:37:13.890844+0000","last_peered":"2024-04-26T23:37:13.890844+0000","last_clean":"2024-04-26T23:37:13.890844+0000","last_became_active":"2024-04-26T23:37:07.744597+0000","last_became_peered":"2024-04-26T23:37:07.744597+0000","last_unstale":"2024-04-26T23:37:13.890844+0000","last_undegraded":"2024-04-26T23:37:13.890844+0000","last_fullsized":"2024-04-26T23:37:13.890844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889553+0000","last_change":"2024-04-26T23:37:13.097492+0000","last_active":"2024-04-26T23:37:13.889553+0000","last_peered":"2024-04-26T23:37:13.889553+0000","last_clean":"2024-04-26T23:37:13.889553+0000","last_became_active":"2024-04-26T23:37:13.096325+0000","last_became_peered":"2024-04-26T23:37:13.096325+0000","last_unstale":"2024-04-26T23:37:13.889553+0000","last_undegraded":"2024-04-26T23:37:13.889553+0000","last_fullsized":"2024-04-26T23:37:13.889553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889171+0000","last_change":"2024-04-26T23:37:08.762047+0000","last_active":"2024-04-26T23:37:13.889171+0000","last_peered":"2024-04-26T23:37:13.889171+0000","last_clean":"2024-04-26T23:37:13.889171+0000","last_became_active":"2024-04-26T23:37:08.761660+0000","last_became_peered":"2024-04-26T23:37:08.761660+0000","last_unstale":"2024-04-26T23:37:13.889171+0000","last_undegraded":"2024-04-26T23:37:13.889171+0000","last_fullsized":"2024-04-26T23:37:13.889171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893281+0000","last_change":"2024-04-26T23:37:07.753465+0000","last_active":"2024-04-26T23:37:13.893281+0000","last_peered":"2024-04-26T23:37:13.893281+0000","last_clean":"2024-04-26T23:37:13.893281+0000","last_became_active":"2024-04-26T23:37:07.752398+0000","last_became_peered":"2024-04-26T23:37:07.752398+0000","last_unstale":"2024-04-26T23:37:13.893281+0000","last_undegraded":"2024-04-26T23:37:13.893281+0000","last_fullsized":"2024-04-26T23:37:13.893281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:02.540340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795198+0000","last_change":"2024-04-26T23:37:13.089856+0000","last_active":"2024-04-26T23:37:13.795198+0000","last_peered":"2024-04-26T23:37:13.795198+0000","last_clean":"2024-04-26T23:37:13.795198+0000","last_became_active":"2024-04-26T23:37:13.088870+0000","last_became_peered":"2024-04-26T23:37:13.088870+0000","last_unstale":"2024-04-26T23:37:13.795198+0000","last_undegraded":"2024-04-26T23:37:13.795198+0000","last_fullsized":"2024-04-26T23:37:13.795198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803393+0000","last_change":"2024-04-26T23:37:08.785871+0000","last_active":"2024-04-26T23:37:13.803393+0000","last_peered":"2024-04-26T23:37:13.803393+0000","last_clean":"2024-04-26T23:37:13.803393+0000","last_became_active":"2024-04-26T23:37:08.785451+0000","last_became_peered":"2024-04-26T23:37:08.785451+0000","last_unstale":"2024-04-26T23:37:13.803393+0000","last_undegraded":"2024-04-26T23:37:13.803393+0000","last_fullsized":"2024-04-26T23:37:13.803393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889253+0000","last_change":"2024-04-26T23:37:07.730561+0000","last_active":"2024-04-26T23:37:13.889253+0000","last_peered":"2024-04-26T23:37:13.889253+0000","last_clean":"2024-04-26T23:37:13.889253+0000","last_became_active":"2024-04-26T23:37:07.730274+0000","last_became_peered":"2024-04-26T23:37:07.730274+0000","last_unstale":"2024-04-26T23:37:13.889253+0000","last_undegraded":"2024-04-26T23:37:13.889253+0000","last_fullsized":"2024-04-26T23:37:13.889253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888467+0000","last_change":"2024-04-26T23:37:12.819624+0000","last_active":"2024-04-26T23:37:13.888467+0000","last_peered":"2024-04-26T23:37:13.888467+0000","last_clean":"2024-04-26T23:37:13.888467+0000","last_became_active":"2024-04-26T23:37:12.819079+0000","last_became_peered":"2024-04-26T23:37:12.819079+0000","last_unstale":"2024-04-26T23:37:13.888467+0000","last_undegraded":"2024-04-26T23:37:13.888467+0000","last_fullsized":"2024-04-26T23:37:13.888467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:14.326950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799143+0000","last_change":"2024-04-26T23:37:08.760531+0000","last_active":"2024-04-26T23:37:13.799143+0000","last_peered":"2024-04-26T23:37:13.799143+0000","last_clean":"2024-04-26T23:37:13.799143+0000","last_became_active":"2024-04-26T23:37:08.760250+0000","last_became_peered":"2024-04-26T23:37:08.760250+0000","last_unstale":"2024-04-26T23:37:13.799143+0000","last_undegraded":"2024-04-26T23:37:13.799143+0000","last_fullsized":"2024-04-26T23:37:13.799143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:17.138976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802089+0000","last_change":"2024-04-26T23:37:07.743324+0000","last_active":"2024-04-26T23:37:13.802089+0000","last_peered":"2024-04-26T23:37:13.802089+0000","last_clean":"2024-04-26T23:37:13.802089+0000","last_became_active":"2024-04-26T23:37:07.742056+0000","last_became_peered":"2024-04-26T23:37:07.742056+0000","last_unstale":"2024-04-26T23:37:13.802089+0000","last_undegraded":"2024-04-26T23:37:13.802089+0000","last_fullsized":"2024-04-26T23:37:13.802089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889116+0000","last_change":"2024-04-26T23:37:12.835987+0000","last_active":"2024-04-26T23:37:13.889116+0000","last_peered":"2024-04-26T23:37:13.889116+0000","last_clean":"2024-04-26T23:37:13.889116+0000","last_became_active":"2024-04-26T23:37:12.831199+0000","last_became_peered":"2024-04-26T23:37:12.831199+0000","last_unstale":"2024-04-26T23:37:13.889116+0000","last_undegraded":"2024-04-26T23:37:13.889116+0000","last_fullsized":"2024-04-26T23:37:13.889116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888008+0000","last_change":"2024-04-26T23:37:08.768283+0000","last_active":"2024-04-26T23:37:13.888008+0000","last_peered":"2024-04-26T23:37:13.888008+0000","last_clean":"2024-04-26T23:37:13.888008+0000","last_became_active":"2024-04-26T23:37:08.766512+0000","last_became_peered":"2024-04-26T23:37:08.766512+0000","last_unstale":"2024-04-26T23:37:13.888008+0000","last_undegraded":"2024-04-26T23:37:13.888008+0000","last_fullsized":"2024-04-26T23:37:13.888008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888285+0000","last_change":"2024-04-26T23:37:07.728903+0000","last_active":"2024-04-26T23:37:13.888285+0000","last_peered":"2024-04-26T23:37:13.888285+0000","last_clean":"2024-04-26T23:37:13.888285+0000","last_became_active":"2024-04-26T23:37:07.728621+0000","last_became_peered":"2024-04-26T23:37:07.728621+0000","last_unstale":"2024-04-26T23:37:13.888285+0000","last_undegraded":"2024-04-26T23:37:13.888285+0000","last_fullsized":"2024-04-26T23:37:13.888285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892519+0000","last_change":"2024-04-26T23:37:12.833651+0000","last_active":"2024-04-26T23:37:13.892519+0000","last_peered":"2024-04-26T23:37:13.892519+0000","last_clean":"2024-04-26T23:37:13.892519+0000","last_became_active":"2024-04-26T23:37:12.831533+0000","last_became_peered":"2024-04-26T23:37:12.831533+0000","last_unstale":"2024-04-26T23:37:13.892519+0000","last_undegraded":"2024-04-26T23:37:13.892519+0000","last_fullsized":"2024-04-26T23:37:13.892519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794491+0000","last_change":"2024-04-26T23:37:08.749652+0000","last_active":"2024-04-26T23:37:13.794491+0000","last_peered":"2024-04-26T23:37:13.794491+0000","last_clean":"2024-04-26T23:37:13.794491+0000","last_became_active":"2024-04-26T23:37:08.748932+0000","last_became_peered":"2024-04-26T23:37:08.748932+0000","last_unstale":"2024-04-26T23:37:13.794491+0000","last_undegraded":"2024-04-26T23:37:13.794491+0000","last_fullsized":"2024-04-26T23:37:13.794491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:04.431242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890355+0000","last_change":"2024-04-26T23:37:07.753005+0000","last_active":"2024-04-26T23:37:13.890355+0000","last_peered":"2024-04-26T23:37:13.890355+0000","last_clean":"2024-04-26T23:37:13.890355+0000","last_became_active":"2024-04-26T23:37:07.752377+0000","last_became_peered":"2024-04-26T23:37:07.752377+0000","last_unstale":"2024-04-26T23:37:13.890355+0000","last_undegraded":"2024-04-26T23:37:13.890355+0000","last_fullsized":"2024-04-26T23:37:13.890355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795722+0000","last_change":"2024-04-26T23:37:13.097286+0000","last_active":"2024-04-26T23:37:13.795722+0000","last_peered":"2024-04-26T23:37:13.795722+0000","last_clean":"2024-04-26T23:37:13.795722+0000","last_became_active":"2024-04-26T23:37:13.096512+0000","last_became_peered":"2024-04-26T23:37:13.096512+0000","last_unstale":"2024-04-26T23:37:13.795722+0000","last_undegraded":"2024-04-26T23:37:13.795722+0000","last_fullsized":"2024-04-26T23:37:13.795722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:09.908349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890492+0000","last_change":"2024-04-26T23:37:08.791257+0000","last_active":"2024-04-26T23:37:13.890492+0000","last_peered":"2024-04-26T23:37:13.890492+0000","last_clean":"2024-04-26T23:37:13.890492+0000","last_became_active":"2024-04-26T23:37:08.790966+0000","last_became_peered":"2024-04-26T23:37:08.790966+0000","last_unstale":"2024-04-26T23:37:13.890492+0000","last_undegraded":"2024-04-26T23:37:13.890492+0000","last_fullsized":"2024-04-26T23:37:13.890492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802430+0000","last_change":"2024-04-26T23:37:07.742144+0000","last_active":"2024-04-26T23:37:13.802430+0000","last_peered":"2024-04-26T23:37:13.802430+0000","last_clean":"2024-04-26T23:37:13.802430+0000","last_became_active":"2024-04-26T23:37:07.741812+0000","last_became_peered":"2024-04-26T23:37:07.741812+0000","last_unstale":"2024-04-26T23:37:13.802430+0000","last_undegraded":"2024-04-26T23:37:13.802430+0000","last_fullsized":"2024-04-26T23:37:13.802430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890498+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:37:13.890498+0000","last_peered":"2024-04-26T23:37:13.890498+0000","last_clean":"2024-04-26T23:37:13.890498+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:37:13.890498+0000","last_undegraded":"2024-04-26T23:37:13.890498+0000","last_fullsized":"2024-04-26T23:37:13.890498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892073+0000","last_change":"2024-04-26T23:37:13.094077+0000","last_active":"2024-04-26T23:37:13.892073+0000","last_peered":"2024-04-26T23:37:13.892073+0000","last_clean":"2024-04-26T23:37:13.892073+0000","last_became_active":"2024-04-26T23:37:13.093035+0000","last_became_peered":"2024-04-26T23:37:13.093035+0000","last_unstale":"2024-04-26T23:37:13.892073+0000","last_undegraded":"2024-04-26T23:37:13.892073+0000","last_fullsized":"2024-04-26T23:37:13.892073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892721+0000","last_change":"2024-04-26T23:37:08.787518+0000","last_active":"2024-04-26T23:37:13.892721+0000","last_peered":"2024-04-26T23:37:13.892721+0000","last_clean":"2024-04-26T23:37:13.892721+0000","last_became_active":"2024-04-26T23:37:08.787303+0000","last_became_peered":"2024-04-26T23:37:08.787303+0000","last_unstale":"2024-04-26T23:37:13.892721+0000","last_undegraded":"2024-04-26T23:37:13.892721+0000","last_fullsized":"2024-04-26T23:37:13.892721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891144+0000","last_change":"2024-04-26T23:37:07.746465+0000","last_active":"2024-04-26T23:37:13.891144+0000","last_peered":"2024-04-26T23:37:13.891144+0000","last_clean":"2024-04-26T23:37:13.891144+0000","last_became_active":"2024-04-26T23:37:07.746109+0000","last_became_peered":"2024-04-26T23:37:07.746109+0000","last_unstale":"2024-04-26T23:37:13.891144+0000","last_undegraded":"2024-04-26T23:37:13.891144+0000","last_fullsized":"2024-04-26T23:37:13.891144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890061+0000","last_change":"2024-04-26T23:37:13.098348+0000","last_active":"2024-04-26T23:37:13.890061+0000","last_peered":"2024-04-26T23:37:13.890061+0000","last_clean":"2024-04-26T23:37:13.890061+0000","last_became_active":"2024-04-26T23:37:13.092005+0000","last_became_peered":"2024-04-26T23:37:13.092005+0000","last_unstale":"2024-04-26T23:37:13.890061+0000","last_undegraded":"2024-04-26T23:37:13.890061+0000","last_fullsized":"2024-04-26T23:37:13.890061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802973+0000","last_change":"2024-04-26T23:37:08.788947+0000","last_active":"2024-04-26T23:37:13.802973+0000","last_peered":"2024-04-26T23:37:13.802973+0000","last_clean":"2024-04-26T23:37:13.802973+0000","last_became_active":"2024-04-26T23:37:08.785540+0000","last_became_peered":"2024-04-26T23:37:08.785540+0000","last_unstale":"2024-04-26T23:37:13.802973+0000","last_undegraded":"2024-04-26T23:37:13.802973+0000","last_fullsized":"2024-04-26T23:37:13.802973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889865+0000","last_change":"2024-04-26T23:37:07.745996+0000","last_active":"2024-04-26T23:37:13.889865+0000","last_peered":"2024-04-26T23:37:13.889865+0000","last_clean":"2024-04-26T23:37:13.889865+0000","last_became_active":"2024-04-26T23:37:07.745526+0000","last_became_peered":"2024-04-26T23:37:07.745526+0000","last_unstale":"2024-04-26T23:37:13.889865+0000","last_undegraded":"2024-04-26T23:37:13.889865+0000","last_fullsized":"2024-04-26T23:37:13.889865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795637+0000","last_change":"2024-04-26T23:37:12.818381+0000","last_active":"2024-04-26T23:37:13.795637+0000","last_peered":"2024-04-26T23:37:13.795637+0000","last_clean":"2024-04-26T23:37:13.795637+0000","last_became_active":"2024-04-26T23:37:12.816556+0000","last_became_peered":"2024-04-26T23:37:12.816556+0000","last_unstale":"2024-04-26T23:37:13.795637+0000","last_undegraded":"2024-04-26T23:37:13.795637+0000","last_fullsized":"2024-04-26T23:37:13.795637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.515560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893859+0000","last_change":"2024-04-26T23:37:08.788613+0000","last_active":"2024-04-26T23:37:13.893859+0000","last_peered":"2024-04-26T23:37:13.893859+0000","last_clean":"2024-04-26T23:37:13.893859+0000","last_became_active":"2024-04-26T23:37:08.788341+0000","last_became_peered":"2024-04-26T23:37:08.788341+0000","last_unstale":"2024-04-26T23:37:13.893859+0000","last_undegraded":"2024-04-26T23:37:13.893859+0000","last_fullsized":"2024-04-26T23:37:13.893859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:05.835929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891528+0000","last_change":"2024-04-26T23:37:07.732372+0000","last_active":"2024-04-26T23:37:13.891528+0000","last_peered":"2024-04-26T23:37:13.891528+0000","last_clean":"2024-04-26T23:37:13.891528+0000","last_became_active":"2024-04-26T23:37:07.732085+0000","last_became_peered":"2024-04-26T23:37:07.732085+0000","last_unstale":"2024-04-26T23:37:13.891528+0000","last_undegraded":"2024-04-26T23:37:13.891528+0000","last_fullsized":"2024-04-26T23:37:13.891528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798168+0000","last_change":"2024-04-26T23:37:13.093306+0000","last_active":"2024-04-26T23:37:13.798168+0000","last_peered":"2024-04-26T23:37:13.798168+0000","last_clean":"2024-04-26T23:37:13.798168+0000","last_became_active":"2024-04-26T23:37:13.089425+0000","last_became_peered":"2024-04-26T23:37:13.089425+0000","last_unstale":"2024-04-26T23:37:13.798168+0000","last_undegraded":"2024-04-26T23:37:13.798168+0000","last_fullsized":"2024-04-26T23:37:13.798168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:43.503693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893002+0000","last_change":"2024-04-26T23:37:08.788931+0000","last_active":"2024-04-26T23:37:13.893002+0000","last_peered":"2024-04-26T23:37:13.893002+0000","last_clean":"2024-04-26T23:37:13.893002+0000","last_became_active":"2024-04-26T23:37:08.788287+0000","last_became_peered":"2024-04-26T23:37:08.788287+0000","last_unstale":"2024-04-26T23:37:13.893002+0000","last_undegraded":"2024-04-26T23:37:13.893002+0000","last_fullsized":"2024-04-26T23:37:13.893002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.254530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799208+0000","last_change":"2024-04-26T23:37:07.729179+0000","last_active":"2024-04-26T23:37:13.799208+0000","last_peered":"2024-04-26T23:37:13.799208+0000","last_clean":"2024-04-26T23:37:13.799208+0000","last_became_active":"2024-04-26T23:37:07.728840+0000","last_became_peered":"2024-04-26T23:37:07.728840+0000","last_unstale":"2024-04-26T23:37:13.799208+0000","last_undegraded":"2024-04-26T23:37:13.799208+0000","last_fullsized":"2024-04-26T23:37:13.799208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:09.969446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798616+0000","last_change":"2024-04-26T23:37:12.833356+0000","last_active":"2024-04-26T23:37:13.798616+0000","last_peered":"2024-04-26T23:37:13.798616+0000","last_clean":"2024-04-26T23:37:13.798616+0000","last_became_active":"2024-04-26T23:37:12.830528+0000","last_became_peered":"2024-04-26T23:37:12.830528+0000","last_unstale":"2024-04-26T23:37:13.798616+0000","last_undegraded":"2024-04-26T23:37:13.798616+0000","last_fullsized":"2024-04-26T23:37:13.798616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.662558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891353+0000","last_change":"2024-04-26T23:37:08.782130+0000","last_active":"2024-04-26T23:37:13.891353+0000","last_peered":"2024-04-26T23:37:13.891353+0000","last_clean":"2024-04-26T23:37:13.891353+0000","last_became_active":"2024-04-26T23:37:08.781906+0000","last_became_peered":"2024-04-26T23:37:08.781906+0000","last_unstale":"2024-04-26T23:37:13.891353+0000","last_undegraded":"2024-04-26T23:37:13.891353+0000","last_fullsized":"2024-04-26T23:37:13.891353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.887760+0000","last_change":"2024-04-26T23:37:07.728318+0000","last_active":"2024-04-26T23:37:13.887760+0000","last_peered":"2024-04-26T23:37:13.887760+0000","last_clean":"2024-04-26T23:37:13.887760+0000","last_became_active":"2024-04-26T23:37:07.727941+0000","last_became_peered":"2024-04-26T23:37:07.727941+0000","last_unstale":"2024-04-26T23:37:13.887760+0000","last_undegraded":"2024-04-26T23:37:13.887760+0000","last_fullsized":"2024-04-26T23:37:13.887760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795484+0000","last_change":"2024-04-26T23:37:12.832185+0000","last_active":"2024-04-26T23:37:13.795484+0000","last_peered":"2024-04-26T23:37:13.795484+0000","last_clean":"2024-04-26T23:37:13.795484+0000","last_became_active":"2024-04-26T23:37:12.830851+0000","last_became_peered":"2024-04-26T23:37:12.830851+0000","last_unstale":"2024-04-26T23:37:13.795484+0000","last_undegraded":"2024-04-26T23:37:13.795484+0000","last_fullsized":"2024-04-26T23:37:13.795484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:20.169349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889876+0000","last_change":"2024-04-26T23:37:08.768079+0000","last_active":"2024-04-26T23:37:13.889876+0000","last_peered":"2024-04-26T23:37:13.889876+0000","last_clean":"2024-04-26T23:37:13.889876+0000","last_became_active":"2024-04-26T23:37:08.767338+0000","last_became_peered":"2024-04-26T23:37:08.767338+0000","last_unstale":"2024-04-26T23:37:13.889876+0000","last_undegraded":"2024-04-26T23:37:13.889876+0000","last_fullsized":"2024-04-26T23:37:13.889876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890125+0000","last_change":"2024-04-26T23:37:07.746339+0000","last_active":"2024-04-26T23:37:13.890125+0000","last_peered":"2024-04-26T23:37:13.890125+0000","last_clean":"2024-04-26T23:37:13.890125+0000","last_became_active":"2024-04-26T23:37:07.746075+0000","last_became_peered":"2024-04-26T23:37:07.746075+0000","last_unstale":"2024-04-26T23:37:13.890125+0000","last_undegraded":"2024-04-26T23:37:13.890125+0000","last_fullsized":"2024-04-26T23:37:13.890125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892748+0000","last_change":"2024-04-26T23:37:13.091284+0000","last_active":"2024-04-26T23:37:13.892748+0000","last_peered":"2024-04-26T23:37:13.892748+0000","last_clean":"2024-04-26T23:37:13.892748+0000","last_became_active":"2024-04-26T23:37:13.090274+0000","last_became_peered":"2024-04-26T23:37:13.090274+0000","last_unstale":"2024-04-26T23:37:13.892748+0000","last_undegraded":"2024-04-26T23:37:13.892748+0000","last_fullsized":"2024-04-26T23:37:13.892748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:53.078576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892237+0000","last_change":"2024-04-26T23:37:08.782024+0000","last_active":"2024-04-26T23:37:13.892237+0000","last_peered":"2024-04-26T23:37:13.892237+0000","last_clean":"2024-04-26T23:37:13.892237+0000","last_became_active":"2024-04-26T23:37:08.781730+0000","last_became_peered":"2024-04-26T23:37:08.781730+0000","last_unstale":"2024-04-26T23:37:13.892237+0000","last_undegraded":"2024-04-26T23:37:13.892237+0000","last_fullsized":"2024-04-26T23:37:13.892237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:00.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801858+0000","last_change":"2024-04-26T23:37:07.741413+0000","last_active":"2024-04-26T23:37:13.801858+0000","last_peered":"2024-04-26T23:37:13.801858+0000","last_clean":"2024-04-26T23:37:13.801858+0000","last_became_active":"2024-04-26T23:37:07.741088+0000","last_became_peered":"2024-04-26T23:37:07.741088+0000","last_unstale":"2024-04-26T23:37:13.801858+0000","last_undegraded":"2024-04-26T23:37:13.801858+0000","last_fullsized":"2024-04-26T23:37:13.801858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:54.118151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891737+0000","last_change":"2024-04-26T23:37:13.089930+0000","last_active":"2024-04-26T23:37:13.891737+0000","last_peered":"2024-04-26T23:37:13.891737+0000","last_clean":"2024-04-26T23:37:13.891737+0000","last_became_active":"2024-04-26T23:37:13.088797+0000","last_became_peered":"2024-04-26T23:37:13.088797+0000","last_unstale":"2024-04-26T23:37:13.891737+0000","last_undegraded":"2024-04-26T23:37:13.891737+0000","last_fullsized":"2024-04-26T23:37:13.891737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:24.129454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795384+0000","last_change":"2024-04-26T23:37:08.772085+0000","last_active":"2024-04-26T23:37:13.795384+0000","last_peered":"2024-04-26T23:37:13.795384+0000","last_clean":"2024-04-26T23:37:13.795384+0000","last_became_active":"2024-04-26T23:37:08.771732+0000","last_became_peered":"2024-04-26T23:37:08.771732+0000","last_unstale":"2024-04-26T23:37:13.795384+0000","last_undegraded":"2024-04-26T23:37:13.795384+0000","last_fullsized":"2024-04-26T23:37:13.795384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:01.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796199+0000","last_change":"2024-04-26T23:37:07.739747+0000","last_active":"2024-04-26T23:37:13.796199+0000","last_peered":"2024-04-26T23:37:13.796199+0000","last_clean":"2024-04-26T23:37:13.796199+0000","last_became_active":"2024-04-26T23:37:07.739361+0000","last_became_peered":"2024-04-26T23:37:07.739361+0000","last_unstale":"2024-04-26T23:37:13.796199+0000","last_undegraded":"2024-04-26T23:37:13.796199+0000","last_fullsized":"2024-04-26T23:37:13.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:55.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802956+0000","last_change":"2024-04-26T23:37:13.100935+0000","last_active":"2024-04-26T23:37:13.802956+0000","last_peered":"2024-04-26T23:37:13.802956+0000","last_clean":"2024-04-26T23:37:13.802956+0000","last_became_active":"2024-04-26T23:37:13.098932+0000","last_became_peered":"2024-04-26T23:37:13.098932+0000","last_unstale":"2024-04-26T23:37:13.802956+0000","last_undegraded":"2024-04-26T23:37:13.802956+0000","last_fullsized":"2024-04-26T23:37:13.802956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:21.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892457+0000","last_change":"2024-04-26T23:37:08.780897+0000","last_active":"2024-04-26T23:37:13.892457+0000","last_peered":"2024-04-26T23:37:13.892457+0000","last_clean":"2024-04-26T23:37:13.892457+0000","last_became_active":"2024-04-26T23:37:08.780573+0000","last_became_peered":"2024-04-26T23:37:08.780573+0000","last_unstale":"2024-04-26T23:37:13.892457+0000","last_undegraded":"2024-04-26T23:37:13.892457+0000","last_fullsized":"2024-04-26T23:37:13.892457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:27.933683+0000","last_change":"2024-04-26T23:37:07.743107+0000","last_active":"2024-04-26T23:37:27.933683+0000","last_peered":"2024-04-26T23:37:27.933683+0000","last_clean":"2024-04-26T23:37:27.933683+0000","last_became_active":"2024-04-26T23:37:07.742729+0000","last_became_peered":"2024-04-26T23:37:07.742729+0000","last_unstale":"2024-04-26T23:37:27.933683+0000","last_undegraded":"2024-04-26T23:37:27.933683+0000","last_fullsized":"2024-04-26T23:37:27.933683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.120970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888705+0000","last_change":"2024-04-26T23:37:12.824914+0000","last_active":"2024-04-26T23:37:13.888705+0000","last_peered":"2024-04-26T23:37:13.888705+0000","last_clean":"2024-04-26T23:37:13.888705+0000","last_became_active":"2024-04-26T23:37:12.824383+0000","last_became_peered":"2024-04-26T23:37:12.824383+0000","last_unstale":"2024-04-26T23:37:13.888705+0000","last_undegraded":"2024-04-26T23:37:13.888705+0000","last_fullsized":"2024-04-26T23:37:13.888705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801058+0000","last_change":"2024-04-26T23:37:08.785202+0000","last_active":"2024-04-26T23:37:13.801058+0000","last_peered":"2024-04-26T23:37:13.801058+0000","last_clean":"2024-04-26T23:37:13.801058+0000","last_became_active":"2024-04-26T23:37:08.779537+0000","last_became_peered":"2024-04-26T23:37:08.779537+0000","last_unstale":"2024-04-26T23:37:13.801058+0000","last_undegraded":"2024-04-26T23:37:13.801058+0000","last_fullsized":"2024-04-26T23:37:13.801058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.971268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889719+0000","last_change":"2024-04-26T23:37:07.741707+0000","last_active":"2024-04-26T23:37:13.889719+0000","last_peered":"2024-04-26T23:37:13.889719+0000","last_clean":"2024-04-26T23:37:13.889719+0000","last_became_active":"2024-04-26T23:37:07.740731+0000","last_became_peered":"2024-04-26T23:37:07.740731+0000","last_unstale":"2024-04-26T23:37:13.889719+0000","last_undegraded":"2024-04-26T23:37:13.889719+0000","last_fullsized":"2024-04-26T23:37:13.889719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:06.307745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798812+0000","last_change":"2024-04-26T23:37:12.818753+0000","last_active":"2024-04-26T23:37:13.798812+0000","last_peered":"2024-04-26T23:37:13.798812+0000","last_clean":"2024-04-26T23:37:13.798812+0000","last_became_active":"2024-04-26T23:37:12.817885+0000","last_became_peered":"2024-04-26T23:37:12.817885+0000","last_unstale":"2024-04-26T23:37:13.798812+0000","last_undegraded":"2024-04-26T23:37:13.798812+0000","last_fullsized":"2024-04-26T23:37:13.798812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:06.283131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795457+0000","last_change":"2024-04-26T23:37:08.796476+0000","last_active":"2024-04-26T23:37:13.795457+0000","last_peered":"2024-04-26T23:37:13.795457+0000","last_clean":"2024-04-26T23:37:13.795457+0000","last_became_active":"2024-04-26T23:37:08.795730+0000","last_became_peered":"2024-04-26T23:37:08.795730+0000","last_unstale":"2024-04-26T23:37:13.795457+0000","last_undegraded":"2024-04-26T23:37:13.795457+0000","last_fullsized":"2024-04-26T23:37:13.795457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802755+0000","last_change":"2024-04-26T23:37:07.746232+0000","last_active":"2024-04-26T23:37:13.802755+0000","last_peered":"2024-04-26T23:37:13.802755+0000","last_clean":"2024-04-26T23:37:13.802755+0000","last_became_active":"2024-04-26T23:37:07.745874+0000","last_became_peered":"2024-04-26T23:37:07.745874+0000","last_unstale":"2024-04-26T23:37:13.802755+0000","last_undegraded":"2024-04-26T23:37:13.802755+0000","last_fullsized":"2024-04-26T23:37:13.802755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:28.812783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891356+0000","last_change":"2024-04-26T23:37:12.834267+0000","last_active":"2024-04-26T23:37:13.891356+0000","last_peered":"2024-04-26T23:37:13.891356+0000","last_clean":"2024-04-26T23:37:13.891356+0000","last_became_active":"2024-04-26T23:37:12.829458+0000","last_became_peered":"2024-04-26T23:37:12.829458+0000","last_unstale":"2024-04-26T23:37:13.891356+0000","last_undegraded":"2024-04-26T23:37:13.891356+0000","last_fullsized":"2024-04-26T23:37:13.891356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:05.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802727+0000","last_change":"2024-04-26T23:37:08.788911+0000","last_active":"2024-04-26T23:37:13.802727+0000","last_peered":"2024-04-26T23:37:13.802727+0000","last_clean":"2024-04-26T23:37:13.802727+0000","last_became_active":"2024-04-26T23:37:08.785658+0000","last_became_peered":"2024-04-26T23:37:08.785658+0000","last_unstale":"2024-04-26T23:37:13.802727+0000","last_undegraded":"2024-04-26T23:37:13.802727+0000","last_fullsized":"2024-04-26T23:37:13.802727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:27.592917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892184+0000","last_change":"2024-04-26T23:37:07.750014+0000","last_active":"2024-04-26T23:37:13.892184+0000","last_peered":"2024-04-26T23:37:13.892184+0000","last_clean":"2024-04-26T23:37:13.892184+0000","last_became_active":"2024-04-26T23:37:07.748991+0000","last_became_peered":"2024-04-26T23:37:07.748991+0000","last_unstale":"2024-04-26T23:37:13.892184+0000","last_undegraded":"2024-04-26T23:37:13.892184+0000","last_fullsized":"2024-04-26T23:37:13.892184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:59.158543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890500+0000","last_change":"2024-04-26T23:37:13.096386+0000","last_active":"2024-04-26T23:37:13.890500+0000","last_peered":"2024-04-26T23:37:13.890500+0000","last_clean":"2024-04-26T23:37:13.890500+0000","last_became_active":"2024-04-26T23:37:13.095901+0000","last_became_peered":"2024-04-26T23:37:13.095901+0000","last_unstale":"2024-04-26T23:37:13.890500+0000","last_undegraded":"2024-04-26T23:37:13.890500+0000","last_fullsized":"2024-04-26T23:37:13.890500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:22.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795244+0000","last_change":"2024-04-26T23:37:13.100870+0000","last_active":"2024-04-26T23:37:13.795244+0000","last_peered":"2024-04-26T23:37:13.795244+0000","last_clean":"2024-04-26T23:37:13.795244+0000","last_became_active":"2024-04-26T23:37:13.094872+0000","last_became_peered":"2024-04-26T23:37:13.094872+0000","last_unstale":"2024-04-26T23:37:13.795244+0000","last_undegraded":"2024-04-26T23:37:13.795244+0000","last_fullsized":"2024-04-26T23:37:13.795244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:07.712253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888033+0000","last_change":"2024-04-26T23:37:08.770189+0000","last_active":"2024-04-26T23:37:13.888033+0000","last_peered":"2024-04-26T23:37:13.888033+0000","last_clean":"2024-04-26T23:37:13.888033+0000","last_became_active":"2024-04-26T23:37:08.769911+0000","last_became_peered":"2024-04-26T23:37:08.769911+0000","last_unstale":"2024-04-26T23:37:13.888033+0000","last_undegraded":"2024-04-26T23:37:13.888033+0000","last_fullsized":"2024-04-26T23:37:13.888033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800060+0000","last_change":"2024-04-26T23:37:07.745063+0000","last_active":"2024-04-26T23:37:13.800060+0000","last_peered":"2024-04-26T23:37:13.800060+0000","last_clean":"2024-04-26T23:37:13.800060+0000","last_became_active":"2024-04-26T23:37:07.744383+0000","last_became_peered":"2024-04-26T23:37:07.744383+0000","last_unstale":"2024-04-26T23:37:13.800060+0000","last_undegraded":"2024-04-26T23:37:13.800060+0000","last_fullsized":"2024-04-26T23:37:13.800060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:36.666159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890337+0000","last_change":"2024-04-26T23:37:12.824766+0000","last_active":"2024-04-26T23:37:13.890337+0000","last_peered":"2024-04-26T23:37:13.890337+0000","last_clean":"2024-04-26T23:37:13.890337+0000","last_became_active":"2024-04-26T23:37:12.824314+0000","last_became_peered":"2024-04-26T23:37:12.824314+0000","last_unstale":"2024-04-26T23:37:13.890337+0000","last_undegraded":"2024-04-26T23:37:13.890337+0000","last_fullsized":"2024-04-26T23:37:13.890337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:59.286047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889801+0000","last_change":"2024-04-26T23:37:08.769161+0000","last_active":"2024-04-26T23:37:13.889801+0000","last_peered":"2024-04-26T23:37:13.889801+0000","last_clean":"2024-04-26T23:37:13.889801+0000","last_became_active":"2024-04-26T23:37:08.767929+0000","last_became_peered":"2024-04-26T23:37:08.767929+0000","last_unstale":"2024-04-26T23:37:13.889801+0000","last_undegraded":"2024-04-26T23:37:13.889801+0000","last_fullsized":"2024-04-26T23:37:13.889801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890886+0000","last_change":"2024-04-26T23:37:07.739492+0000","last_active":"2024-04-26T23:37:13.890886+0000","last_peered":"2024-04-26T23:37:13.890886+0000","last_clean":"2024-04-26T23:37:13.890886+0000","last_became_active":"2024-04-26T23:37:07.739185+0000","last_became_peered":"2024-04-26T23:37:07.739185+0000","last_unstale":"2024-04-26T23:37:13.890886+0000","last_undegraded":"2024-04-26T23:37:13.890886+0000","last_fullsized":"2024-04-26T23:37:13.890886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890373+0000","last_change":"2024-04-26T23:37:07.750874+0000","last_active":"2024-04-26T23:37:13.890373+0000","last_peered":"2024-04-26T23:37:13.890373+0000","last_clean":"2024-04-26T23:37:13.890373+0000","last_became_active":"2024-04-26T23:37:07.748284+0000","last_became_peered":"2024-04-26T23:37:07.748284+0000","last_unstale":"2024-04-26T23:37:13.890373+0000","last_undegraded":"2024-04-26T23:37:13.890373+0000","last_fullsized":"2024-04-26T23:37:13.890373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:18.623894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.796082+0000","last_change":"2024-04-26T23:37:08.753110+0000","last_active":"2024-04-26T23:37:13.796082+0000","last_peered":"2024-04-26T23:37:13.796082+0000","last_clean":"2024-04-26T23:37:13.796082+0000","last_became_active":"2024-04-26T23:37:08.752712+0000","last_became_peered":"2024-04-26T23:37:08.752712+0000","last_unstale":"2024-04-26T23:37:13.796082+0000","last_undegraded":"2024-04-26T23:37:13.796082+0000","last_fullsized":"2024-04-26T23:37:13.796082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:47.570319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889412+0000","last_change":"2024-04-26T23:37:12.829960+0000","last_active":"2024-04-26T23:37:13.889412+0000","last_peered":"2024-04-26T23:37:13.889412+0000","last_clean":"2024-04-26T23:37:13.889412+0000","last_became_active":"2024-04-26T23:37:12.829520+0000","last_became_peered":"2024-04-26T23:37:12.829520+0000","last_unstale":"2024-04-26T23:37:13.889412+0000","last_undegraded":"2024-04-26T23:37:13.889412+0000","last_fullsized":"2024-04-26T23:37:13.889412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:05.418963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891289+0000","last_change":"2024-04-26T23:37:07.747637+0000","last_active":"2024-04-26T23:37:13.891289+0000","last_peered":"2024-04-26T23:37:13.891289+0000","last_clean":"2024-04-26T23:37:13.891289+0000","last_became_active":"2024-04-26T23:37:07.747308+0000","last_became_peered":"2024-04-26T23:37:07.747308+0000","last_unstale":"2024-04-26T23:37:13.891289+0000","last_undegraded":"2024-04-26T23:37:13.891289+0000","last_fullsized":"2024-04-26T23:37:13.891289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:06.678026+0000","last_clean_scrub_stamp":"2024-04-26T23:37:06.678026+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.874122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.798586+0000","last_change":"2024-04-26T23:37:08.763567+0000","last_active":"2024-04-26T23:37:13.798586+0000","last_peered":"2024-04-26T23:37:13.798586+0000","last_clean":"2024-04-26T23:37:13.798586+0000","last_became_active":"2024-04-26T23:37:08.762771+0000","last_became_peered":"2024-04-26T23:37:08.762771+0000","last_unstale":"2024-04-26T23:37:13.798586+0000","last_undegraded":"2024-04-26T23:37:13.798586+0000","last_fullsized":"2024-04-26T23:37:13.798586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:19.641900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892195+0000","last_change":"2024-04-26T23:37:12.832076+0000","last_active":"2024-04-26T23:37:13.892195+0000","last_peered":"2024-04-26T23:37:13.892195+0000","last_clean":"2024-04-26T23:37:13.892195+0000","last_became_active":"2024-04-26T23:37:12.830641+0000","last_became_peered":"2024-04-26T23:37:12.830641+0000","last_unstale":"2024-04-26T23:37:13.892195+0000","last_undegraded":"2024-04-26T23:37:13.892195+0000","last_fullsized":"2024-04-26T23:37:13.892195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:11.768452+0000","last_clean_scrub_stamp":"2024-04-26T23:37:11.768452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:21.722510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889621+0000","last_change":"2024-04-26T23:37:08.768406+0000","last_active":"2024-04-26T23:37:13.889621+0000","last_peered":"2024-04-26T23:37:13.889621+0000","last_clean":"2024-04-26T23:37:13.889621+0000","last_became_active":"2024-04-26T23:37:08.767730+0000","last_became_peered":"2024-04-26T23:37:08.767730+0000","last_unstale":"2024-04-26T23:37:13.889621+0000","last_undegraded":"2024-04-26T23:37:13.889621+0000","last_fullsized":"2024-04-26T23:37:13.889621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888136+0000","last_change":"2024-04-26T23:37:08.780725+0000","last_active":"2024-04-26T23:37:13.888136+0000","last_peered":"2024-04-26T23:37:13.888136+0000","last_clean":"2024-04-26T23:37:13.888136+0000","last_became_active":"2024-04-26T23:37:08.780362+0000","last_became_peered":"2024-04-26T23:37:08.780362+0000","last_unstale":"2024-04-26T23:37:13.888136+0000","last_undegraded":"2024-04-26T23:37:13.888136+0000","last_fullsized":"2024-04-26T23:37:13.888136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801811+0000","last_change":"2024-04-26T23:37:08.789772+0000","last_active":"2024-04-26T23:37:13.801811+0000","last_peered":"2024-04-26T23:37:13.801811+0000","last_clean":"2024-04-26T23:37:13.801811+0000","last_became_active":"2024-04-26T23:37:08.785818+0000","last_became_peered":"2024-04-26T23:37:08.785818+0000","last_unstale":"2024-04-26T23:37:13.801811+0000","last_undegraded":"2024-04-26T23:37:13.801811+0000","last_fullsized":"2024-04-26T23:37:13.801811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795710+0000","last_change":"2024-04-26T23:37:08.768747+0000","last_active":"2024-04-26T23:37:13.795710+0000","last_peered":"2024-04-26T23:37:13.795710+0000","last_clean":"2024-04-26T23:37:13.795710+0000","last_became_active":"2024-04-26T23:37:08.768413+0000","last_became_peered":"2024-04-26T23:37:08.768413+0000","last_unstale":"2024-04-26T23:37:13.795710+0000","last_undegraded":"2024-04-26T23:37:13.795710+0000","last_fullsized":"2024-04-26T23:37:13.795710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891826+0000","last_change":"2024-04-26T23:37:08.788540+0000","last_active":"2024-04-26T23:37:13.891826+0000","last_peered":"2024-04-26T23:37:13.891826+0000","last_clean":"2024-04-26T23:37:13.891826+0000","last_became_active":"2024-04-26T23:37:08.788200+0000","last_became_peered":"2024-04-26T23:37:08.788200+0000","last_unstale":"2024-04-26T23:37:13.891826+0000","last_undegraded":"2024-04-26T23:37:13.891826+0000","last_fullsized":"2024-04-26T23:37:13.891826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891950+0000","last_change":"2024-04-26T23:37:08.777509+0000","last_active":"2024-04-26T23:37:13.891950+0000","last_peered":"2024-04-26T23:37:13.891950+0000","last_clean":"2024-04-26T23:37:13.891950+0000","last_became_active":"2024-04-26T23:37:08.777061+0000","last_became_peered":"2024-04-26T23:37:08.777061+0000","last_unstale":"2024-04-26T23:37:13.891950+0000","last_undegraded":"2024-04-26T23:37:13.891950+0000","last_fullsized":"2024-04-26T23:37:13.891950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891626+0000","last_change":"2024-04-26T23:37:08.774973+0000","last_active":"2024-04-26T23:37:13.891626+0000","last_peered":"2024-04-26T23:37:13.891626+0000","last_clean":"2024-04-26T23:37:13.891626+0000","last_became_active":"2024-04-26T23:37:08.774707+0000","last_became_peered":"2024-04-26T23:37:08.774707+0000","last_unstale":"2024-04-26T23:37:13.891626+0000","last_undegraded":"2024-04-26T23:37:13.891626+0000","last_fullsized":"2024-04-26T23:37:13.891626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892661+0000","last_change":"2024-04-26T23:37:08.781269+0000","last_active":"2024-04-26T23:37:13.892661+0000","last_peered":"2024-04-26T23:37:13.892661+0000","last_clean":"2024-04-26T23:37:13.892661+0000","last_became_active":"2024-04-26T23:37:08.780337+0000","last_became_peered":"2024-04-26T23:37:08.780337+0000","last_unstale":"2024-04-26T23:37:13.892661+0000","last_undegraded":"2024-04-26T23:37:13.892661+0000","last_fullsized":"2024-04-26T23:37:13.892661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:02.210711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799802+0000","last_change":"2024-04-26T23:37:08.771877+0000","last_active":"2024-04-26T23:37:13.799802+0000","last_peered":"2024-04-26T23:37:13.799802+0000","last_clean":"2024-04-26T23:37:13.799802+0000","last_became_active":"2024-04-26T23:37:08.770810+0000","last_became_peered":"2024-04-26T23:37:08.770810+0000","last_unstale":"2024-04-26T23:37:13.799802+0000","last_undegraded":"2024-04-26T23:37:13.799802+0000","last_fullsized":"2024-04-26T23:37:13.799802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890670+0000","last_change":"2024-04-26T23:37:08.780735+0000","last_active":"2024-04-26T23:37:13.890670+0000","last_peered":"2024-04-26T23:37:13.890670+0000","last_clean":"2024-04-26T23:37:13.890670+0000","last_became_active":"2024-04-26T23:37:08.780384+0000","last_became_peered":"2024-04-26T23:37:08.780384+0000","last_unstale":"2024-04-26T23:37:13.890670+0000","last_undegraded":"2024-04-26T23:37:13.890670+0000","last_fullsized":"2024-04-26T23:37:13.890670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801555+0000","last_change":"2024-04-26T23:37:08.790073+0000","last_active":"2024-04-26T23:37:13.801555+0000","last_peered":"2024-04-26T23:37:13.801555+0000","last_clean":"2024-04-26T23:37:13.801555+0000","last_became_active":"2024-04-26T23:37:08.789431+0000","last_became_peered":"2024-04-26T23:37:08.789431+0000","last_unstale":"2024-04-26T23:37:13.801555+0000","last_undegraded":"2024-04-26T23:37:13.801555+0000","last_fullsized":"2024-04-26T23:37:13.801555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795276+0000","last_change":"2024-04-26T23:37:08.756179+0000","last_active":"2024-04-26T23:37:13.795276+0000","last_peered":"2024-04-26T23:37:13.795276+0000","last_clean":"2024-04-26T23:37:13.795276+0000","last_became_active":"2024-04-26T23:37:08.755805+0000","last_became_peered":"2024-04-26T23:37:08.755805+0000","last_unstale":"2024-04-26T23:37:13.795276+0000","last_undegraded":"2024-04-26T23:37:13.795276+0000","last_fullsized":"2024-04-26T23:37:13.795276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:12.270375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891285+0000","last_change":"2024-04-26T23:37:08.768542+0000","last_active":"2024-04-26T23:37:13.891285+0000","last_peered":"2024-04-26T23:37:13.891285+0000","last_clean":"2024-04-26T23:37:13.891285+0000","last_became_active":"2024-04-26T23:37:08.765841+0000","last_became_peered":"2024-04-26T23:37:08.765841+0000","last_unstale":"2024-04-26T23:37:13.891285+0000","last_undegraded":"2024-04-26T23:37:13.891285+0000","last_fullsized":"2024-04-26T23:37:13.891285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:25.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893105+0000","last_change":"2024-04-26T23:37:08.775312+0000","last_active":"2024-04-26T23:37:13.893105+0000","last_peered":"2024-04-26T23:37:13.893105+0000","last_clean":"2024-04-26T23:37:13.893105+0000","last_became_active":"2024-04-26T23:37:08.774944+0000","last_became_peered":"2024-04-26T23:37:08.774944+0000","last_unstale":"2024-04-26T23:37:13.893105+0000","last_undegraded":"2024-04-26T23:37:13.893105+0000","last_fullsized":"2024-04-26T23:37:13.893105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:16.096929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891448+0000","last_change":"2024-04-26T23:37:08.772525+0000","last_active":"2024-04-26T23:37:13.891448+0000","last_peered":"2024-04-26T23:37:13.891448+0000","last_clean":"2024-04-26T23:37:13.891448+0000","last_became_active":"2024-04-26T23:37:08.772177+0000","last_became_peered":"2024-04-26T23:37:08.772177+0000","last_unstale":"2024-04-26T23:37:13.891448+0000","last_undegraded":"2024-04-26T23:37:13.891448+0000","last_fullsized":"2024-04-26T23:37:13.891448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:55.136157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.803582+0000","last_change":"2024-04-26T23:37:08.790236+0000","last_active":"2024-04-26T23:37:13.803582+0000","last_peered":"2024-04-26T23:37:13.803582+0000","last_clean":"2024-04-26T23:37:13.803582+0000","last_became_active":"2024-04-26T23:37:08.789885+0000","last_became_peered":"2024-04-26T23:37:08.789885+0000","last_unstale":"2024-04-26T23:37:13.803582+0000","last_undegraded":"2024-04-26T23:37:13.803582+0000","last_fullsized":"2024-04-26T23:37:13.803582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.799434+0000","last_change":"2024-04-26T23:37:08.780732+0000","last_active":"2024-04-26T23:37:13.799434+0000","last_peered":"2024-04-26T23:37:13.799434+0000","last_clean":"2024-04-26T23:37:13.799434+0000","last_became_active":"2024-04-26T23:37:08.779046+0000","last_became_peered":"2024-04-26T23:37:08.779046+0000","last_unstale":"2024-04-26T23:37:13.799434+0000","last_undegraded":"2024-04-26T23:37:13.799434+0000","last_fullsized":"2024-04-26T23:37:13.799434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.888183+0000","last_change":"2024-04-26T23:37:08.751809+0000","last_active":"2024-04-26T23:37:13.888183+0000","last_peered":"2024-04-26T23:37:13.888183+0000","last_clean":"2024-04-26T23:37:13.888183+0000","last_became_active":"2024-04-26T23:37:08.750955+0000","last_became_peered":"2024-04-26T23:37:08.750955+0000","last_unstale":"2024-04-26T23:37:13.888183+0000","last_undegraded":"2024-04-26T23:37:13.888183+0000","last_fullsized":"2024-04-26T23:37:13.888183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.795084+0000","last_change":"2024-04-26T23:37:08.762870+0000","last_active":"2024-04-26T23:37:13.795084+0000","last_peered":"2024-04-26T23:37:13.795084+0000","last_clean":"2024-04-26T23:37:13.795084+0000","last_became_active":"2024-04-26T23:37:08.762554+0000","last_became_peered":"2024-04-26T23:37:08.762554+0000","last_unstale":"2024-04-26T23:37:13.795084+0000","last_undegraded":"2024-04-26T23:37:13.795084+0000","last_fullsized":"2024-04-26T23:37:13.795084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:05.526812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891059+0000","last_change":"2024-04-26T23:37:08.774910+0000","last_active":"2024-04-26T23:37:13.891059+0000","last_peered":"2024-04-26T23:37:13.891059+0000","last_clean":"2024-04-26T23:37:13.891059+0000","last_became_active":"2024-04-26T23:37:08.774587+0000","last_became_peered":"2024-04-26T23:37:08.774587+0000","last_unstale":"2024-04-26T23:37:13.891059+0000","last_undegraded":"2024-04-26T23:37:13.891059+0000","last_fullsized":"2024-04-26T23:37:13.891059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:57.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.800959+0000","last_change":"2024-04-26T23:37:08.778389+0000","last_active":"2024-04-26T23:37:13.800959+0000","last_peered":"2024-04-26T23:37:13.800959+0000","last_clean":"2024-04-26T23:37:13.800959+0000","last_became_active":"2024-04-26T23:37:08.778116+0000","last_became_peered":"2024-04-26T23:37:08.778116+0000","last_unstale":"2024-04-26T23:37:13.800959+0000","last_undegraded":"2024-04-26T23:37:13.800959+0000","last_fullsized":"2024-04-26T23:37:13.800959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:07.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.892362+0000","last_change":"2024-04-26T23:37:08.781929+0000","last_active":"2024-04-26T23:37:13.892362+0000","last_peered":"2024-04-26T23:37:13.892362+0000","last_clean":"2024-04-26T23:37:13.892362+0000","last_became_active":"2024-04-26T23:37:08.780724+0000","last_became_peered":"2024-04-26T23:37:08.780724+0000","last_unstale":"2024-04-26T23:37:13.892362+0000","last_undegraded":"2024-04-26T23:37:13.892362+0000","last_fullsized":"2024-04-26T23:37:13.892362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:00.249651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891324+0000","last_change":"2024-04-26T23:37:08.792579+0000","last_active":"2024-04-26T23:37:13.891324+0000","last_peered":"2024-04-26T23:37:13.891324+0000","last_clean":"2024-04-26T23:37:13.891324+0000","last_became_active":"2024-04-26T23:37:08.792171+0000","last_became_peered":"2024-04-26T23:37:08.792171+0000","last_unstale":"2024-04-26T23:37:13.891324+0000","last_undegraded":"2024-04-26T23:37:13.891324+0000","last_fullsized":"2024-04-26T23:37:13.891324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893856+0000","last_change":"2024-04-26T23:37:08.795179+0000","last_active":"2024-04-26T23:37:13.893856+0000","last_peered":"2024-04-26T23:37:13.893856+0000","last_clean":"2024-04-26T23:37:13.893856+0000","last_became_active":"2024-04-26T23:37:08.794890+0000","last_became_peered":"2024-04-26T23:37:08.794890+0000","last_unstale":"2024-04-26T23:37:13.893856+0000","last_undegraded":"2024-04-26T23:37:13.893856+0000","last_fullsized":"2024-04-26T23:37:13.893856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:09.443140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.889647+0000","last_change":"2024-04-26T23:37:08.786739+0000","last_active":"2024-04-26T23:37:13.889647+0000","last_peered":"2024-04-26T23:37:13.889647+0000","last_clean":"2024-04-26T23:37:13.889647+0000","last_became_active":"2024-04-26T23:37:08.786360+0000","last_became_peered":"2024-04-26T23:37:08.786360+0000","last_unstale":"2024-04-26T23:37:13.889647+0000","last_undegraded":"2024-04-26T23:37:13.889647+0000","last_fullsized":"2024-04-26T23:37:13.889647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:51.088047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891212+0000","last_change":"2024-04-26T23:37:08.775748+0000","last_active":"2024-04-26T23:37:13.891212+0000","last_peered":"2024-04-26T23:37:13.891212+0000","last_clean":"2024-04-26T23:37:13.891212+0000","last_became_active":"2024-04-26T23:37:08.775440+0000","last_became_peered":"2024-04-26T23:37:08.775440+0000","last_unstale":"2024-04-26T23:37:13.891212+0000","last_undegraded":"2024-04-26T23:37:13.891212+0000","last_fullsized":"2024-04-26T23:37:13.891212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.801324+0000","last_change":"2024-04-26T23:37:08.791665+0000","last_active":"2024-04-26T23:37:13.801324+0000","last_peered":"2024-04-26T23:37:13.801324+0000","last_clean":"2024-04-26T23:37:13.801324+0000","last_became_active":"2024-04-26T23:37:08.790305+0000","last_became_peered":"2024-04-26T23:37:08.790305+0000","last_unstale":"2024-04-26T23:37:13.801324+0000","last_undegraded":"2024-04-26T23:37:13.801324+0000","last_fullsized":"2024-04-26T23:37:13.801324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:57.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.891978+0000","last_change":"2024-04-26T23:37:08.776000+0000","last_active":"2024-04-26T23:37:13.891978+0000","last_peered":"2024-04-26T23:37:13.891978+0000","last_clean":"2024-04-26T23:37:13.891978+0000","last_became_active":"2024-04-26T23:37:08.773615+0000","last_became_peered":"2024-04-26T23:37:08.773615+0000","last_unstale":"2024-04-26T23:37:13.891978+0000","last_undegraded":"2024-04-26T23:37:13.891978+0000","last_fullsized":"2024-04-26T23:37:13.891978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:41.181102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.802456+0000","last_change":"2024-04-26T23:37:08.789506+0000","last_active":"2024-04-26T23:37:13.802456+0000","last_peered":"2024-04-26T23:37:13.802456+0000","last_clean":"2024-04-26T23:37:13.802456+0000","last_became_active":"2024-04-26T23:37:08.789180+0000","last_became_peered":"2024-04-26T23:37:08.789180+0000","last_unstale":"2024-04-26T23:37:13.802456+0000","last_undegraded":"2024-04-26T23:37:13.802456+0000","last_fullsized":"2024-04-26T23:37:13.802456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:55.213627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.794898+0000","last_change":"2024-04-26T23:37:08.766757+0000","last_active":"2024-04-26T23:37:13.794898+0000","last_peered":"2024-04-26T23:37:13.794898+0000","last_clean":"2024-04-26T23:37:13.794898+0000","last_became_active":"2024-04-26T23:37:08.761659+0000","last_became_peered":"2024-04-26T23:37:08.761659+0000","last_unstale":"2024-04-26T23:37:13.794898+0000","last_undegraded":"2024-04-26T23:37:13.794898+0000","last_fullsized":"2024-04-26T23:37:13.794898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:34.412737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.890719+0000","last_change":"2024-04-26T23:37:08.761968+0000","last_active":"2024-04-26T23:37:13.890719+0000","last_peered":"2024-04-26T23:37:13.890719+0000","last_clean":"2024-04-26T23:37:13.890719+0000","last_became_active":"2024-04-26T23:37:08.761546+0000","last_became_peered":"2024-04-26T23:37:08.761546+0000","last_unstale":"2024-04-26T23:37:13.890719+0000","last_undegraded":"2024-04-26T23:37:13.890719+0000","last_fullsized":"2024-04-26T23:37:13.890719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:34.590138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T23:37:13.893557+0000","last_change":"2024-04-26T23:37:08.775290+0000","last_active":"2024-04-26T23:37:13.893557+0000","last_peered":"2024-04-26T23:37:13.893557+0000","last_clean":"2024-04-26T23:37:13.893557+0000","last_became_active":"2024-04-26T23:37:08.774646+0000","last_became_peered":"2024-04-26T23:37:08.774646+0000","last_unstale":"2024-04-26T23:37:13.893557+0000","last_undegraded":"2024-04-26T23:37:13.893557+0000","last_fullsized":"2024-04-26T23:37:13.893557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:07.696032+0000","last_clean_scrub_stamp":"2024-04-26T23:37:07.696032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:03.639833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:37:44.536 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T23:37:44.536 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T23:37:44.536 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T23:37:44.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T23:37:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.777+0000 7ff0f7200640 1 Processor -- start 2024-04-26T23:37:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.777+0000 7ff0f7200640 1 -- start start 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f8159950 0x7ff0f80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff0f80ae8e0 0x7ff0f8162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0f81631d0 0x7ff0f815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f8163620 con 0x7ff0f81631d0 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f80af0c0 con 0x7ff0f80ae8e0 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.779+0000 7ff0f7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f80af3a0 con 0x7ff0f8159950 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0f81631d0 0x7ff0f815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0f81631d0 0x7ff0f815f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39358/0 (socket says 172.21.15.46:39358) 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 -- 172.21.15.46:0/233115726 learned_addr learned my addr 172.21.15.46:0/233115726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f5400640 1 --2- 172.21.15.46:0/233115726 >> v2:172.21.15.142:3300/0 conn(0x7ff0f80ae8e0 0x7ff0f8162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f5e00640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f8159950 0x7ff0f80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 -- 172.21.15.46:0/233115726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f8159950 msgr2=0x7ff0f80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f8159950 0x7ff0f80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 -- 172.21.15.46:0/233115726 >> v2:172.21.15.142:3300/0 conn(0x7ff0f80ae8e0 msgr2=0x7ff0f8162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 --2- 172.21.15.46:0/233115726 >> v2:172.21.15.142:3300/0 conn(0x7ff0f80ae8e0 0x7ff0f8162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.780+0000 7ff0f6800640 1 -- 172.21.15.46:0/233115726 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f80af680 con 0x7ff0f81631d0 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.781+0000 7ff0f6800640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815f6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80767f0 tx=0x7ff0f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4058b8197fdb8c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.781+0000 7ff0f4a00640 1 -- 172.21.15.46:0/233115726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037070 con 0x7ff0f81631d0 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f4a00640 1 -- 172.21.15.46:0/233115726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f0032440 con 0x7ff0f81631d0 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f4a00640 1 -- 172.21.15.46:0/233115726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0032810 con 0x7ff0f81631d0 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 -- 172.21.15.46:0/233115726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 msgr2=0x7ff0f815f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815f6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80767f0 tx=0x7ff0f002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 -- 172.21.15.46:0/233115726 shutdown_connections 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815f6a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 --2- 172.21.15.46:0/233115726 >> v2:172.21.15.142:3300/0 conn(0x7ff0f80ae8e0 0x7ff0f8162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 --2- 172.21.15.46:0/233115726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f8159950 0x7ff0f80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 -- 172.21.15.46:0/233115726 >> 172.21.15.46:0/233115726 conn(0x7ff0f80b2820 msgr2=0x7ff0f80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 -- 172.21.15.46:0/233115726 shutdown_connections 2024-04-26T23:37:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.782+0000 7ff0f7200640 1 -- 172.21.15.46:0/233115726 wait complete. 2024-04-26T23:37:44.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.784+0000 7ff0f7200640 1 Processor -- start 2024-04-26T23:37:44.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.784+0000 7ff0f7200640 1 -- start start 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0f80ae8e0 0x7ff0f815c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 0x7ff0f8158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f80b0160 con 0x7ff0f81631d0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f80afc40 con 0x7ff0f80ae8e0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.785+0000 7ff0f7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f80aed40 con 0x7ff0f814b6e0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39368/0 (socket says 172.21.15.46:39368) 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 -- 172.21.15.46:0/1592892656 learned_addr learned my addr 172.21.15.46:0/1592892656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5e00640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0f80ae8e0 0x7ff0f815c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f6800640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 0x7ff0f8158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 -- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 msgr2=0x7ff0f8158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 0x7ff0f8158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 -- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0f80ae8e0 msgr2=0x7ff0f815c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0f80ae8e0 0x7ff0f815c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 -- 172.21.15.46:0/1592892656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f815ac90 con 0x7ff0f81631d0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f6800640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 0x7ff0f8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.786+0000 7ff0f5400640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007e60 tx=0x7ff0e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.787+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014620 con 0x7ff0f81631d0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.787+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e0019070 con 0x7ff0f81631d0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.787+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014a00 con 0x7ff0f81631d0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.787+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f81609b0 con 0x7ff0f81631d0 2024-04-26T23:37:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.787+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f815cbf0 con 0x7ff0f81631d0 2024-04-26T23:37:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.788+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f80693b0 con 0x7ff0f81631d0 2024-04-26T23:37:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.789+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0e0014ba0 con 0x7ff0f81631d0 2024-04-26T23:37:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.790+0000 7ff0ee000640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 0x7ff0d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.790+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0e008c780 con 0x7ff0f81631d0 2024-04-26T23:37:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.790+0000 7ff0f5e00640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 0x7ff0d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.791+0000 7ff0f5e00640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 0x7ff0d406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8006fd0 tx=0x7ff0e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:44.794+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0e005a7d0 con 0x7ff0f81631d0 2024-04-26T23:37:45.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.041+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0f80b35e0 con 0x7ff0f81631d0 2024-04-26T23:37:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.041+0000 7ff0ee000640 1 -- 172.21.15.46:0/1592892656 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff0e005a0d0 con 0x7ff0f81631d0 2024-04-26T23:37:45.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:45.045 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T23:37:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 msgr2=0x7ff0d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 0x7ff0d406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8006fd0 tx=0x7ff0e8008040 comp rx=0 tx=0).stop 2024-04-26T23:37:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 msgr2=0x7ff0f815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007e60 tx=0x7ff0e000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 shutdown_connections 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0d406c920 0x7ff0d406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0f814b6e0 0x7ff0f8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0f81631d0 0x7ff0f815a750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 --2- 172.21.15.46:0/1592892656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0f80ae8e0 0x7ff0f815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 >> 172.21.15.46:0/1592892656 conn(0x7ff0f80b2820 msgr2=0x7ff0f80b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 shutdown_connections 2024-04-26T23:37:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.044+0000 7ff0f7200640 1 -- 172.21.15.46:0/1592892656 wait complete. 2024-04-26T23:37:45.067 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T23:37:45.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.266+0000 7faefa400640 1 Processor -- start 2024-04-26T23:37:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.266+0000 7faefa400640 1 -- start start 2024-04-26T23:37:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.267+0000 7faefa400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faef40cc690 0x7faef40cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.267+0000 7faefa400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef40c6a50 0x7faef40c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faefa400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef4072bb0 con 0x7faef41454a0 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faefa400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faef4072e90 con 0x7faef40cc690 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faefa400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faef4073170 con 0x7faef40c6a50 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef9000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faef41454a0 0x7faef4073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39864/0 (socket says 172.21.15.46:39864) 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef9000640 1 -- 172.21.15.46:0/1045140346 learned_addr learned my addr 172.21.15.46:0/1045140346 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3e00640 1 --2- 172.21.15.46:0/1045140346 >> v2:172.21.15.46:3300/0 conn(0x7faef40cc690 0x7faef40cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef9a00640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef40c6a50 0x7faef40c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3400640 1 -- 172.21.15.46:0/1045140346 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165325870 0 0) 0x7faef4072bb0 con 0x7faef41454a0 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3400640 1 -- 172.21.15.46:0/1045140346 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faedc003520 con 0x7faef41454a0 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3e00640 1 -- 172.21.15.46:0/1045140346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef40c6a50 msgr2=0x7faef40c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3e00640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef40c6a50 0x7faef40c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.268+0000 7faef3e00640 1 -- 172.21.15.46:0/1045140346 >> v1:172.21.15.46:6789/0 conn(0x7faef41454a0 legacy=0x7faef4073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.269+0000 7faef3e00640 1 -- 172.21.15.46:0/1045140346 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef4073450 con 0x7faef40cc690 2024-04-26T23:37:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faef3e00640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef40cdc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faee00097a0 tx=0x7faee002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b902e5aa765f6dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faef3400640 1 -- 172.21.15.46:0/1045140346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0037070 con 0x7faef40cc690 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faef3400640 1 -- 172.21.15.46:0/1045140346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0032440 con 0x7faef40cc690 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faef3400640 1 -- 172.21.15.46:0/1045140346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee00327f0 con 0x7faef40cc690 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faefa400640 1 -- 172.21.15.46:0/1045140346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 msgr2=0x7faef40cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.270+0000 7faefa400640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef40cdc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faee00097a0 tx=0x7faee002a180 comp rx=0 tx=0).stop 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 -- 172.21.15.46:0/1045140346 shutdown_connections 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef40c6a50 0x7faef40c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 --2- 172.21.15.46:0/1045140346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef40cdc30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 -- 172.21.15.46:0/1045140346 >> 172.21.15.46:0/1045140346 conn(0x7faef40b26d0 msgr2=0x7faef40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 -- 172.21.15.46:0/1045140346 shutdown_connections 2024-04-26T23:37:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.271+0000 7faefa400640 1 -- 172.21.15.46:0/1045140346 wait complete. 2024-04-26T23:37:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.272+0000 7faefa400640 1 Processor -- start 2024-04-26T23:37:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.273+0000 7faefa400640 1 -- start start 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef40c6a50 0x7faef4143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef41454a0 0x7faef406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faef4145880 con 0x7faef40cc690 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faef4072880 con 0x7faef40c6a50 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faefa400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faef4145a00 con 0x7faef41454a0 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faef3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faef3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39386/0 (socket says 172.21.15.46:39386) 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faef3e00640 1 -- 172.21.15.46:0/1265128256 learned_addr learned my addr 172.21.15.46:0/1265128256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faef9000640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef40c6a50 0x7faef4143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.274+0000 7faef9a00640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef41454a0 0x7faef406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 -- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef41454a0 msgr2=0x7faef406efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef41454a0 0x7faef406efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 -- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef40c6a50 msgr2=0x7faef4143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef40c6a50 0x7faef4143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 -- 172.21.15.46:0/1265128256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef413d540 con 0x7faef40cc690 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef3e00640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faee0035040 tx=0x7faee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0041020 con 0x7faef40cc690 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0030700 con 0x7faef40cc690 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0037040 con 0x7faef40cc690 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef413d820 con 0x7faef40cc690 2024-04-26T23:37:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.275+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef413dde0 con 0x7faef40cc690 2024-04-26T23:37:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.277+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef40cd340 con 0x7faef40cc690 2024-04-26T23:37:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.277+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faee0044020 con 0x7faef40cc690 2024-04-26T23:37:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.278+0000 7faef0c00640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 0x7faed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.278+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faee00ac700 con 0x7faef40cc690 2024-04-26T23:37:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.279+0000 7faef9000640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 0x7faed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.280+0000 7faef9000640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 0x7faed406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faee8005880 tx=0x7faee8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.283+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faef40cd340 con 0x7faef40cc690 2024-04-26T23:37:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.487+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faef40d0300 con 0x7faef40cc690 2024-04-26T23:37:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.490+0000 7faef0c00640 1 -- 172.21.15.46:0/1265128256 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7faef40d0300 con 0x7faef40cc690 2024-04-26T23:37:45.494 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:45.494 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 msgr2=0x7faed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 0x7faed406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faee8005880 tx=0x7faee8009040 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 msgr2=0x7faef413e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faee0035040 tx=0x7faee00046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 shutdown_connections 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faed406c8f0 0x7faed406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faef41454a0 0x7faef406efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cc690 0x7faef413e300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 --2- 172.21.15.46:0/1265128256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef40c6a50 0x7faef4143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 >> 172.21.15.46:0/1265128256 conn(0x7faef40b26d0 msgr2=0x7faef40aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.492+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 shutdown_connections 2024-04-26T23:37:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.493+0000 7faefa400640 1 -- 172.21.15.46:0/1265128256 wait complete. 2024-04-26T23:37:45.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:37:45.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:37:45.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.711+0000 7f5e99400640 1 Processor -- start 2024-04-26T23:37:45.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.711+0000 7f5e99400640 1 -- start start 2024-04-26T23:37:45.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.712+0000 7f5e99400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e940cdd20 0x7f5e940cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.712+0000 7f5e99400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e940ae8f0 0x7f5e940aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.712+0000 7f5e99400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940af2a0 con 0x7f5e9415f210 2024-04-26T23:37:45.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.712+0000 7f5e99400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5e940af570 con 0x7f5e940ae8f0 2024-04-26T23:37:45.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.712+0000 7f5e99400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5e940af850 con 0x7f5e940cdd20 2024-04-26T23:37:45.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e98a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e940ae8f0 0x7f5e940aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e93e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5e9415f210 0x7f5e940cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39866/0 (socket says 172.21.15.46:39866) 2024-04-26T23:37:45.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e93e00640 1 -- 172.21.15.46:0/434871138 learned_addr learned my addr 172.21.15.46:0/434871138 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:45.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e8be00640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e940cdd20 0x7f5e940cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e98a00640 1 -- 172.21.15.46:0/434871138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e940cdd20 msgr2=0x7f5e940cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e98a00640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e940cdd20 0x7f5e940cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e98a00640 1 -- 172.21.15.46:0/434871138 >> v1:172.21.15.46:6789/0 conn(0x7f5e9415f210 legacy=0x7f5e940cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.714+0000 7f5e98a00640 1 -- 172.21.15.46:0/434871138 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940cb680 con 0x7f5e940ae8f0 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.713+0000 7f5e93400640 1 -- 172.21.15.46:0/434871138 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082543198 0 0) 0x7f5e940af2a0 con 0x7f5e9415f210 2024-04-26T23:37:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.715+0000 7f5e98a00640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e940aed60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009d90 tx=0x7f5e8c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344371299e67ab67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.715+0000 7f5e93400640 1 -- 172.21.15.46:0/434871138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c007740 con 0x7f5e940ae8f0 2024-04-26T23:37:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.715+0000 7f5e93400640 1 -- 172.21.15.46:0/434871138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c0078e0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.715+0000 7f5e93400640 1 -- 172.21.15.46:0/434871138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c007cb0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 -- 172.21.15.46:0/434871138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 msgr2=0x7f5e940aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e940aed60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009d90 tx=0x7f5e8c02a660 comp rx=0 tx=0).stop 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 -- 172.21.15.46:0/434871138 shutdown_connections 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e940aed60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 --2- 172.21.15.46:0/434871138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e940cdd20 0x7f5e940cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 -- 172.21.15.46:0/434871138 >> 172.21.15.46:0/434871138 conn(0x7f5e940b27b0 msgr2=0x7f5e940b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 -- 172.21.15.46:0/434871138 shutdown_connections 2024-04-26T23:37:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.716+0000 7f5e99400640 1 -- 172.21.15.46:0/434871138 wait complete. 2024-04-26T23:37:45.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.718+0000 7f5e99400640 1 Processor -- start 2024-04-26T23:37:45.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.718+0000 7f5e99400640 1 -- start start 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e940cdd20 0x7f5e940b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e9415f210 0x7f5e940b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9415f5f0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940cbc40 con 0x7f5e940cdd20 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e99400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5e94071e80 con 0x7f5e9415f210 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39404/0 (socket says 172.21.15.46:39404) 2024-04-26T23:37:45.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e93e00640 1 -- 172.21.15.46:0/1639252120 learned_addr learned my addr 172.21.15.46:0/1639252120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:45.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e8be00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e940cdd20 0x7f5e940b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e93e00640 1 -- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e9415f210 msgr2=0x7f5e940b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.719+0000 7f5e98a00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e9415f210 0x7f5e940b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e93e00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e9415f210 0x7f5e940b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e93e00640 1 -- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e940cdd20 msgr2=0x7f5e940b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e93e00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e940cdd20 0x7f5e940b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e93e00640 1 -- 172.21.15.46:0/1639252120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940c97d0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e93e00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c430 tx=0x7f5e8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840074e0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e840047e0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84004bd0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e940c9ab0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.720+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e940b38e0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.722+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e940693b0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.722+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e84011070 con 0x7f5e940ae8f0 2024-04-26T23:37:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.724+0000 7f5e90c00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 0x7f5e7006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.724+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e8408baa0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.725+0000 7f5e8be00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 0x7f5e7006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.726+0000 7f5e8be00640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 0x7f5e7006ee00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e941f9180 tx=0x7f5e7c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.729+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e8400e280 con 0x7f5e940ae8f0 2024-04-26T23:37:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.932+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e9415c7c0 con 0x7f5e940ae8f0 2024-04-26T23:37:45.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.933+0000 7f5e90c00640 1 -- 172.21.15.46:0/1639252120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f5e84059c00 con 0x7f5e940ae8f0 2024-04-26T23:37:45.937 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:37:45.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.936+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 msgr2=0x7f5e7006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.936+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 0x7f5e7006ee00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e941f9180 tx=0x7f5e7c008040 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.936+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 msgr2=0x7f5e9415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.936+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c430 tx=0x7f5e8400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 shutdown_connections 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e7006c940 0x7f5e7006ee00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e9415f210 0x7f5e940b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e940cdd20 0x7f5e940b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 --2- 172.21.15.46:0/1639252120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940ae8f0 0x7f5e9415a330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 >> 172.21.15.46:0/1639252120 conn(0x7f5e940b27b0 msgr2=0x7f5e9415cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 shutdown_connections 2024-04-26T23:37:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:45.937+0000 7f5e99400640 1 -- 172.21.15.46:0/1639252120 wait complete. 2024-04-26T23:37:45.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:37:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.174+0000 7f745a000640 1 Processor -- start 2024-04-26T23:37:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.175+0000 7f745a000640 1 -- start start 2024-04-26T23:37:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.176+0000 7f745a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f745415cec0 0x7f74540cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.176+0000 7f745a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f74540cdb30 0x7f74540735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.176+0000 7f745a000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74540ce7d0 con 0x7f7454073b30 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.176+0000 7f745a000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f74540ceab0 con 0x7f745415cec0 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.176+0000 7f745a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f74540ced90 con 0x7f74540cdb30 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f745415cec0 0x7f74540cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f745415cec0 0x7f74540cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39416/0 (socket says 172.21.15.46:39416) 2024-04-26T23:37:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 -- 172.21.15.46:0/332270415 learned_addr learned my addr 172.21.15.46:0/332270415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7453e00640 1 --2- 172.21.15.46:0/332270415 >> v2:172.21.15.142:3300/0 conn(0x7f74540cdb30 0x7f74540735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 -- 172.21.15.46:0/332270415 >> v2:172.21.15.142:3300/0 conn(0x7f74540cdb30 msgr2=0x7f74540735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 --2- 172.21.15.46:0/332270415 >> v2:172.21.15.142:3300/0 conn(0x7f74540cdb30 0x7f74540735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 -- 172.21.15.46:0/332270415 >> v1:172.21.15.142:6789/0 conn(0x7f7454073b30 legacy=0x7f7454071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.177+0000 7f7458c00640 1 -- 172.21.15.46:0/332270415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74540cf070 con 0x7f745415cec0 2024-04-26T23:37:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.178+0000 7f7458c00640 1 --2- 172.21.15.46:0/332270415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f745415cec0 0x7f74540cd5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7454074300 tx=0x7f744802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdf275dbfa7e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f7453400640 1 -- 172.21.15.46:0/332270415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448037070 con 0x7f745415cec0 2024-04-26T23:37:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f7453400640 1 -- 172.21.15.46:0/332270415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7448032440 con 0x7f745415cec0 2024-04-26T23:37:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f7453400640 1 -- 172.21.15.46:0/332270415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448032810 con 0x7f745415cec0 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 -- 172.21.15.46:0/332270415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f745415cec0 msgr2=0x7f74540cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 --2- 172.21.15.46:0/332270415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f745415cec0 0x7f74540cd5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7454074300 tx=0x7f744802a230 comp rx=0 tx=0).stop 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 -- 172.21.15.46:0/332270415 shutdown_connections 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 --2- 172.21.15.46:0/332270415 >> v2:172.21.15.142:3300/0 conn(0x7f74540cdb30 0x7f74540735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 --2- 172.21.15.46:0/332270415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f745415cec0 0x7f74540cd5f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.179+0000 7f745a000640 1 -- 172.21.15.46:0/332270415 >> 172.21.15.46:0/332270415 conn(0x7f74540b27b0 msgr2=0x7f74540b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.180+0000 7f745a000640 1 -- 172.21.15.46:0/332270415 shutdown_connections 2024-04-26T23:37:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.180+0000 7f745a000640 1 -- 172.21.15.46:0/332270415 wait complete. 2024-04-26T23:37:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.181+0000 7f745a000640 1 Processor -- start 2024-04-26T23:37:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.181+0000 7f745a000640 1 -- start start 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7454073b30 0x7f74540ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 0x7f745415da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7454073f70 con 0x7f74540cdb30 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f745415d2a0 con 0x7f745415cec0 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.182+0000 7f745a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74540b4b50 con 0x7f7454073b30 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39422/0 (socket says 172.21.15.46:39422) 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 -- 172.21.15.46:0/341628604 learned_addr learned my addr 172.21.15.46:0/341628604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7459600640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 0x7f745415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7458c00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7454073b30 0x7f74540ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 -- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7454073b30 msgr2=0x7f74540ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7454073b30 0x7f74540ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 -- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 msgr2=0x7f745415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 0x7f745415da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7453e00640 1 -- 172.21.15.46:0/341628604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74540b11b0 con 0x7f74540cdb30 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.183+0000 7f7459600640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 0x7f745415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f7453e00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f744000c7f0 tx=0x7f744000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440015070 con 0x7f74540cdb30 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440004240 con 0x7f74540cdb30 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440007620 con 0x7f74540cdb30 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74540b1490 con 0x7f74540cdb30 2024-04-26T23:37:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.184+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74540b1a50 con 0x7f74540cdb30 2024-04-26T23:37:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.185+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74540b63b0 con 0x7f74540cdb30 2024-04-26T23:37:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.186+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74400079d0 con 0x7f74540cdb30 2024-04-26T23:37:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.187+0000 7f7450c00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 0x7f743406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.187+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f744008c220 con 0x7f74540cdb30 2024-04-26T23:37:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.188+0000 7f7458c00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 0x7f743406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.189+0000 7f7458c00640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 0x7f743406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.192+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74540b63b0 con 0x7f74540cdb30 2024-04-26T23:37:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.396+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f74540693b0 con 0x7f74540cdb30 2024-04-26T23:37:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.397+0000 7f7450c00640 1 -- 172.21.15.46:0/341628604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f744005a3a0 con 0x7f74540cdb30 2024-04-26T23:37:46.401 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.399+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 msgr2=0x7f743406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.399+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 0x7f743406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802e000 comp rx=0 tx=0).stop 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 msgr2=0x7f74541541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f744000c7f0 tx=0x7f744000ccb0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 shutdown_connections 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f743406c8f0 0x7f743406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f745415cec0 0x7f745415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74540cdb30 0x7f74541541d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 --2- 172.21.15.46:0/341628604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7454073b30 0x7f74540ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 >> 172.21.15.46:0/341628604 conn(0x7f74540b27b0 msgr2=0x7f74540b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 shutdown_connections 2024-04-26T23:37:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.400+0000 7f745a000640 1 -- 172.21.15.46:0/341628604 wait complete. 2024-04-26T23:37:46.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:37:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.633+0000 7f3e7f200640 1 Processor -- start 2024-04-26T23:37:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.634+0000 7f3e7f200640 1 -- start start 2024-04-26T23:37:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.635+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 0x7f3e8015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.635+0000 7f3e7f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e80146b20 con 0x7f3e800ce670 2024-04-26T23:37:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.635+0000 7f3e7f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e800ccf30 con 0x7f3e800d0330 2024-04-26T23:37:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.635+0000 7f3e7f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3e80161a00 con 0x7f3e80146740 2024-04-26T23:37:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.636+0000 7f3e7de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 0x7f3e8015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.636+0000 7f3e7d400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3e800d0330 0x7f3e801612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47116/0 (socket says 172.21.15.46:47116) 2024-04-26T23:37:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.636+0000 7f3e7d400640 1 -- 172.21.15.46:0/1671548916 learned_addr learned my addr 172.21.15.46:0/1671548916 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.637+0000 7f3e7de00640 1 -- 172.21.15.46:0/1671548916 >> v1:172.21.15.142:6789/0 conn(0x7f3e800d0330 legacy=0x7f3e801612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.637+0000 7f3e7de00640 1 -- 172.21.15.46:0/1671548916 >> v1:172.21.15.46:6789/0 conn(0x7f3e800ce670 legacy=0x7f3e800ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.637+0000 7f3e7de00640 1 -- 172.21.15.46:0/1671548916 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e80160af0 con 0x7f3e80146740 2024-04-26T23:37:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.638+0000 7f3e7de00640 1 --2- 172.21.15.46:0/1671548916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 0x7f3e8015df90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097d0 tx=0x7f3e7002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199a985802a56d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.639+0000 7f3e7ca00640 1 -- 172.21.15.46:0/1671548916 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037070 con 0x7f3e80146740 2024-04-26T23:37:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.639+0000 7f3e7ca00640 1 -- 172.21.15.46:0/1671548916 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e70032440 con 0x7f3e80146740 2024-04-26T23:37:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.639+0000 7f3e7ca00640 1 -- 172.21.15.46:0/1671548916 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70032810 con 0x7f3e80146740 2024-04-26T23:37:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.639+0000 7f3e7f200640 1 -- 172.21.15.46:0/1671548916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 msgr2=0x7f3e8015df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.639+0000 7f3e7f200640 1 --2- 172.21.15.46:0/1671548916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 0x7f3e8015df90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097d0 tx=0x7f3e7002a210 comp rx=0 tx=0).stop 2024-04-26T23:37:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.640+0000 7f3e7f200640 1 -- 172.21.15.46:0/1671548916 shutdown_connections 2024-04-26T23:37:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.640+0000 7f3e7f200640 1 --2- 172.21.15.46:0/1671548916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e80146740 0x7f3e8015df90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.640+0000 7f3e7f200640 1 -- 172.21.15.46:0/1671548916 >> 172.21.15.46:0/1671548916 conn(0x7f3e800b26d0 msgr2=0x7f3e800b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.640+0000 7f3e7f200640 1 -- 172.21.15.46:0/1671548916 shutdown_connections 2024-04-26T23:37:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.640+0000 7f3e7f200640 1 -- 172.21.15.46:0/1671548916 wait complete. 2024-04-26T23:37:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.642+0000 7f3e7f200640 1 Processor -- start 2024-04-26T23:37:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.642+0000 7f3e7f200640 1 -- start start 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e800ce670 0x7f3e800c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e80146740 0x7f3e800c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800ccb70 con 0x7f3e800d0330 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800d0790 con 0x7f3e80146740 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3e800cccf0 con 0x7f3e800ce670 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39434/0 (socket says 172.21.15.46:39434) 2024-04-26T23:37:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7d400640 1 -- 172.21.15.46:0/3613968697 learned_addr learned my addr 172.21.15.46:0/3613968697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.643+0000 7f3e7de00640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e800ce670 0x7f3e800c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e80146740 0x7f3e800c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 -- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e800ce670 msgr2=0x7f3e800c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e800ce670 0x7f3e800c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 -- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e80146740 msgr2=0x7f3e800c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e80146740 0x7f3e800c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 -- 172.21.15.46:0/3613968697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e800c6450 con 0x7f3e800d0330 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7d400640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c4a0 tx=0x7f3e6800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68017070 con 0x7f3e800d0330 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e68005ce0 con 0x7f3e800d0330 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68007270 con 0x7f3e800d0330 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e800c6730 con 0x7f3e800d0330 2024-04-26T23:37:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.644+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e800c6cf0 con 0x7f3e800d0330 2024-04-26T23:37:46.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.646+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e800693b0 con 0x7f3e800d0330 2024-04-26T23:37:46.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.647+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e6801a070 con 0x7f3e800d0330 2024-04-26T23:37:46.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.647+0000 7f3e76000640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:46.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.648+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3e6808b140 con 0x7f3e800d0330 2024-04-26T23:37:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.648+0000 7f3e7de00640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.649+0000 7f3e7de00640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e70034000 tx=0x7f3e7002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.652+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e68058bd0 con 0x7f3e800d0330 2024-04-26T23:37:46.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.855+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e8006f400 con 0x7f3e800d0330 2024-04-26T23:37:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.855+0000 7f3e76000640 1 -- 172.21.15.46:0/3613968697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3e6805ced0 con 0x7f3e800d0330 2024-04-26T23:37:46.859 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:37:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 msgr2=0x7f3e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e70034000 tx=0x7f3e7002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 msgr2=0x7f3e8013fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c4a0 tx=0x7f3e6800c960 comp rx=0 tx=0).stop 2024-04-26T23:37:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 shutdown_connections 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e80146740 0x7f3e800c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e800d0330 0x7f3e8013fa70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 --2- 172.21.15.46:0/3613968697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e800ce670 0x7f3e800c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 >> 172.21.15.46:0/3613968697 conn(0x7f3e800b26d0 msgr2=0x7f3e80144f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 shutdown_connections 2024-04-26T23:37:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:46.858+0000 7f3e7f200640 1 -- 172.21.15.46:0/3613968697 wait complete. 2024-04-26T23:37:46.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:37:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.089+0000 7fc029200640 1 Processor -- start 2024-04-26T23:37:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.089+0000 7fc029200640 1 -- start start 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc0240d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0240d5b90 0x7fc0240d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 0x7fc0240b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240b44a0 con 0x7fc0240d5b90 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240b5020 con 0x7fc0240b54b0 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc029200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc0240b46f0 con 0x7fc024162aa0 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc021400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0240d5b90 0x7fc0240d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc021400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0240d5b90 0x7fc0240d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39444/0 (socket says 172.21.15.46:39444) 2024-04-26T23:37:47.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.091+0000 7fc021400640 1 -- 172.21.15.46:0/1743630161 learned_addr learned my addr 172.21.15.46:0/1743630161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc022800640 1 --2- 172.21.15.46:0/1743630161 >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 0x7fc0240b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021e00640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc0240d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021400640 1 -- 172.21.15.46:0/1743630161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 msgr2=0x7fc0240d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021400640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc0240d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021400640 1 -- 172.21.15.46:0/1743630161 >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 msgr2=0x7fc0240b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021400640 1 --2- 172.21.15.46:0/1743630161 >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 0x7fc0240b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc021400640 1 -- 172.21.15.46:0/1743630161 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0240b49d0 con 0x7fc0240d5b90 2024-04-26T23:37:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.092+0000 7fc022800640 1 --2- 172.21.15.46:0/1743630161 >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 0x7fc0240b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:47.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.093+0000 7fc021400640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0240d3b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0240bb0f0 tx=0x7fc00c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0755bd78ff853a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.093+0000 7fc020a00640 1 -- 172.21.15.46:0/1743630161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c037070 con 0x7fc0240d5b90 2024-04-26T23:37:47.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.093+0000 7fc020a00640 1 -- 172.21.15.46:0/1743630161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c032440 con 0x7fc0240d5b90 2024-04-26T23:37:47.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.093+0000 7fc020a00640 1 -- 172.21.15.46:0/1743630161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c032810 con 0x7fc0240d5b90 2024-04-26T23:37:47.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 -- 172.21.15.46:0/1743630161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 msgr2=0x7fc0240d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0240d3b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0240bb0f0 tx=0x7fc00c02a140 comp rx=0 tx=0).stop 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 -- 172.21.15.46:0/1743630161 shutdown_connections 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 --2- 172.21.15.46:0/1743630161 >> v2:172.21.15.142:3300/0 conn(0x7fc0240b54b0 0x7fc0240b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0240d3b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 --2- 172.21.15.46:0/1743630161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc0240d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 -- 172.21.15.46:0/1743630161 >> 172.21.15.46:0/1743630161 conn(0x7fc0240d5070 msgr2=0x7fc0240d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 -- 172.21.15.46:0/1743630161 shutdown_connections 2024-04-26T23:37:47.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.094+0000 7fc029200640 1 -- 172.21.15.46:0/1743630161 wait complete. 2024-04-26T23:37:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.096+0000 7fc029200640 1 Processor -- start 2024-04-26T23:37:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.096+0000 7fc029200640 1 -- start start 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.097+0000 7fc029200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0240b54b0 0x7fc0240cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.097+0000 7fc029200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc029200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc024154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc029200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0240d30f0 con 0x7fc0240d5b90 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc029200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc024162e80 con 0x7fc0240b54b0 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc029200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc0240b5e30 con 0x7fc024162aa0 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39448/0 (socket says 172.21.15.46:39448) 2024-04-26T23:37:47.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 -- 172.21.15.46:0/697001674 learned_addr learned my addr 172.21.15.46:0/697001674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021e00640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0240b54b0 0x7fc0240cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc022800640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc024154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 -- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 msgr2=0x7fc024154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc024154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 -- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0240b54b0 msgr2=0x7fc0240cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0240b54b0 0x7fc0240cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.098+0000 7fc021400640 1 -- 172.21.15.46:0/697001674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0240c96e0 con 0x7fc0240d5b90 2024-04-26T23:37:47.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.099+0000 7fc021400640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000c00 tx=0x7fc00c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.100+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c041020 con 0x7fc0240d5b90 2024-04-26T23:37:47.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.100+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c0306c0 con 0x7fc0240d5b90 2024-04-26T23:37:47.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.100+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c037040 con 0x7fc0240d5b90 2024-04-26T23:37:47.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.100+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0240c99c0 con 0x7fc0240d5b90 2024-04-26T23:37:47.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.100+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0240c9f80 con 0x7fc0240d5b90 2024-04-26T23:37:47.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.101+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0240d3ec0 con 0x7fc0240d5b90 2024-04-26T23:37:47.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.102+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc00c044020 con 0x7fc0240d5b90 2024-04-26T23:37:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.103+0000 7fc01a000640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 0x7fc00006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.103+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc00c0ac810 con 0x7fc0240d5b90 2024-04-26T23:37:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.104+0000 7fc021e00640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 0x7fc00006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.105+0000 7fc021e00640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 0x7fc00006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e90 tx=0x7fc01400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.108+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0240d3ec0 con 0x7fc0240d5b90 2024-04-26T23:37:47.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.311+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc024069530 con 0x7fc0240d5b90 2024-04-26T23:37:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.313+0000 7fc01a000640 1 -- 172.21.15.46:0/697001674 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fc00c04b030 con 0x7fc0240d5b90 2024-04-26T23:37:47.317 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 msgr2=0x7fc00006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 0x7fc00006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e90 tx=0x7fc01400b040 comp rx=0 tx=0).stop 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 msgr2=0x7fc0241540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000c00 tx=0x7fc00c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 shutdown_connections 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc00006c8f0 0x7fc00006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc024162aa0 0x7fc024154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0240d5b90 0x7fc0241540d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 --2- 172.21.15.46:0/697001674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0240b54b0 0x7fc0240cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.315+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 >> 172.21.15.46:0/697001674 conn(0x7fc0240d5070 msgr2=0x7fc0240d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.316+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 shutdown_connections 2024-04-26T23:37:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.316+0000 7fc029200640 1 -- 172.21.15.46:0/697001674 wait complete. 2024-04-26T23:37:47.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:47.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.547+0000 7f39dbe00640 1 Processor -- start 2024-04-26T23:37:47.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.547+0000 7f39dbe00640 1 -- start start 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39dbe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39dc0720e0 0x7f39dc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 0x7f39dc160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39dbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39dc073ad0 con 0x7f39dc1466b0 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39dbe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39dc146a90 con 0x7f39dc0720e0 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc073d70 con 0x7f39dc160910 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39daa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f39dc1466b0 0x7f39dc06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39892/0 (socket says 172.21.15.46:39892) 2024-04-26T23:37:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39daa00640 1 -- 172.21.15.46:0/274442283 learned_addr learned my addr 172.21.15.46:0/274442283 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39d2000640 1 --2- 172.21.15.46:0/274442283 >> v2:172.21.15.142:3300/0 conn(0x7f39dc0720e0 0x7f39dc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39db400640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 0x7f39dc160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.549+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065013141 0 0) 0x7f39dc073ad0 con 0x7f39dc1466b0 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c0003520 con 0x7f39dc1466b0 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4109408185 0 0) 0x7f39c0003520 con 0x7f39dc1466b0 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 msgr2=0x7f39dc160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 0x7f39dc160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 >> v2:172.21.15.142:3300/0 conn(0x7f39dc0720e0 msgr2=0x7f39dc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 --2- 172.21.15.46:0/274442283 >> v2:172.21.15.142:3300/0 conn(0x7f39dc0720e0 0x7f39dc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc074050 con 0x7f39dc1466b0 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f39cc001670 con 0x7f39dc1466b0 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 >> v1:172.21.15.46:6789/0 conn(0x7f39dc1466b0 legacy=0x7f39dc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39c00039b0 0x7f39c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 0x7f39c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c0004d40 0x7f39c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc146a90 con 0x7f39c00039b0 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc073d70 con 0x7f39c0004d40 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39c000b9c0 con 0x7f39dc160910 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.550+0000 7f39daa00640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39c00039b0 0x7f39c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.551+0000 7f39daa00640 1 -- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 msgr2=0x7f39c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.551+0000 7f39daa00640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc160910 0x7f39c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.551+0000 7f39daa00640 1 -- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c0004d40 msgr2=0x7f39c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.551+0000 7f39daa00640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c0004d40 0x7f39c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.551+0000 7f39daa00640 1 -- 172.21.15.46:0/274442283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c0003520 con 0x7f39c00039b0 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.552+0000 7f39daa00640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39c00039b0 0x7f39c0003dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c210 tx=0x7f39c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f076b7a24388f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.552+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc007d30 con 0x7f39c00039b0 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.552+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc005ad0 con 0x7f39c00039b0 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.552+0000 7f39da000640 1 -- 172.21.15.46:0/274442283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc007070 con 0x7f39c00039b0 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 -- 172.21.15.46:0/274442283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39c00039b0 msgr2=0x7f39c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 --2- 172.21.15.46:0/274442283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39c00039b0 0x7f39c0003dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c210 tx=0x7f39c0005830 comp rx=0 tx=0).stop 2024-04-26T23:37:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39db400640 1 -- 172.21.15.46:0/274442283 reap_dead start 2024-04-26T23:37:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 -- 172.21.15.46:0/274442283 shutdown_connections 2024-04-26T23:37:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 -- 172.21.15.46:0/274442283 >> 172.21.15.46:0/274442283 conn(0x7f39dc0b26d0 msgr2=0x7f39dc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 -- 172.21.15.46:0/274442283 shutdown_connections 2024-04-26T23:37:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.553+0000 7f39dbe00640 1 -- 172.21.15.46:0/274442283 wait complete. 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.555+0000 7f39dbe00640 1 Processor -- start 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.555+0000 7f39dbe00640 1 -- start start 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39c00039b0 0x7f39dc157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 0x7f39dc0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0cdda0 con 0x7f39dc0afed0 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0cb470 con 0x7f39dc0c7b70 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc160da0 con 0x7f39c00039b0 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39472/0 (socket says 172.21.15.46:39472) 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.556+0000 7f39db400640 1 -- 172.21.15.46:0/3967017127 learned_addr learned my addr 172.21.15.46:0/3967017127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39d2000640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 0x7f39dc0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39daa00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39c00039b0 0x7f39dc157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 -- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39c00039b0 msgr2=0x7f39dc157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39c00039b0 0x7f39dc157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 -- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 msgr2=0x7f39dc0c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 0x7f39dc0c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 -- 172.21.15.46:0/3967017127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc0c86e0 con 0x7f39dc0afed0 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39d2000640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 0x7f39dc0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39db400640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39dc071c20 tx=0x7f39d400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4018020 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d400bae0 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc0c89c0 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.557+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4014400 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.558+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc0c8f80 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.559+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc0693b0 con 0x7f39dc0afed0 2024-04-26T23:37:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.560+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39d4005d10 con 0x7f39dc0afed0 2024-04-26T23:37:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.561+0000 7f39d3400640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 0x7f39b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.561+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f39d408c540 con 0x7f39dc0afed0 2024-04-26T23:37:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.562+0000 7f39daa00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 0x7f39b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.563+0000 7f39daa00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 0x7f39b806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39cc033000 tx=0x7f39cc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.566+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39d4022030 con 0x7f39dc0afed0 2024-04-26T23:37:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.813+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39dc0c7180 con 0x7f39dc0afed0 2024-04-26T23:37:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.814+0000 7f39d3400640 1 -- 172.21.15.46:0/3967017127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f39d405a620 con 0x7f39dc0afed0 2024-04-26T23:37:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:47.818 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 msgr2=0x7f39b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 0x7f39b806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39cc033000 tx=0x7f39cc02e000 comp rx=0 tx=0).stop 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 msgr2=0x7f39dc0b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39dc071c20 tx=0x7f39d400b480 comp rx=0 tx=0).stop 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 shutdown_connections 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39b806c920 0x7f39b806ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc0afed0 0x7f39dc0b0350 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc0c7b70 0x7f39dc0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 --2- 172.21.15.46:0/3967017127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39c00039b0 0x7f39dc157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 >> 172.21.15.46:0/3967017127 conn(0x7f39dc0b26d0 msgr2=0x7f39dc0c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 shutdown_connections 2024-04-26T23:37:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:47.817+0000 7f39dbe00640 1 -- 172.21.15.46:0/3967017127 wait complete. 2024-04-26T23:37:47.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.050+0000 7f1ef3200640 1 Processor -- start 2024-04-26T23:37:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.051+0000 7f1ef3200640 1 -- start start 2024-04-26T23:37:48.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.052+0000 7f1ef3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef415f4e0 0x7f1ef40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.052+0000 7f1ef3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40b4000 0x7f1ef40d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef40d11f0 con 0x7f1ef40d1f60 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef40d2990 con 0x7f1ef415f4e0 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef40d5f20 con 0x7f1ef40b4000 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef415f4e0 0x7f1ef40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef415f4e0 0x7f1ef40d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39482/0 (socket says 172.21.15.46:39482) 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ef40d1f60 0x7f1ef40cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39896/0 (socket says 172.21.15.46:39896) 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 -- 172.21.15.46:0/4015817988 learned_addr learned my addr 172.21.15.46:0/4015817988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1eebe00640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40b4000 0x7f1ef40d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 -- 172.21.15.46:0/4015817988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40b4000 msgr2=0x7f1ef40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40b4000 0x7f1ef40d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 -- 172.21.15.46:0/4015817988 >> v1:172.21.15.46:6789/0 conn(0x7f1ef40d1f60 legacy=0x7f1ef40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.053+0000 7f1ef1e00640 1 -- 172.21.15.46:0/4015817988 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef40d0f10 con 0x7f1ef415f4e0 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.054+0000 7f1ef1e00640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef415f4e0 0x7f1ef40d2df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009870 tx=0x7f1ee402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a2d3ab9165b2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef1400640 1 -- 172.21.15.46:0/4015817988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4037070 con 0x7f1ef415f4e0 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef1400640 1 -- 172.21.15.46:0/4015817988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee4032440 con 0x7f1ef415f4e0 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef1400640 1 -- 172.21.15.46:0/4015817988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4032810 con 0x7f1ef415f4e0 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 -- 172.21.15.46:0/4015817988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef415f4e0 msgr2=0x7f1ef40d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef415f4e0 0x7f1ef40d2df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009870 tx=0x7f1ee402a250 comp rx=0 tx=0).stop 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 -- 172.21.15.46:0/4015817988 shutdown_connections 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40b4000 0x7f1ef40d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 --2- 172.21.15.46:0/4015817988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef415f4e0 0x7f1ef40d2df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.055+0000 7f1ef3200640 1 -- 172.21.15.46:0/4015817988 >> 172.21.15.46:0/4015817988 conn(0x7f1ef40d5010 msgr2=0x7f1ef40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.056+0000 7f1ef3200640 1 -- 172.21.15.46:0/4015817988 shutdown_connections 2024-04-26T23:37:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.056+0000 7f1ef3200640 1 -- 172.21.15.46:0/4015817988 wait complete. 2024-04-26T23:37:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.057+0000 7f1ef3200640 1 Processor -- start 2024-04-26T23:37:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.058+0000 7f1ef3200640 1 -- start start 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40d1f60 0x7f1ef40b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 0x7f1ef415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef415f8c0 con 0x7f1ef40b4000 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef40b1d60 con 0x7f1ef415f4e0 2024-04-26T23:37:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef40b3290 con 0x7f1ef40d1f60 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39486/0 (socket says 172.21.15.46:39486) 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 -- 172.21.15.46:0/880710590 learned_addr learned my addr 172.21.15.46:0/880710590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1eebe00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40d1f60 0x7f1ef40b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 -- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40d1f60 msgr2=0x7f1ef40b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40d1f60 0x7f1ef40b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef2800640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 0x7f1ef415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 -- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 msgr2=0x7f1ef415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 0x7f1ef415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef1e00640 1 -- 172.21.15.46:0/880710590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef40b47f0 con 0x7f1ef40b4000 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.059+0000 7f1ef2800640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 0x7f1ef415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1ef1e00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4041020 con 0x7f1ef40b4000 2024-04-26T23:37:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee40306c0 con 0x7f1ef40b4000 2024-04-26T23:37:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4037040 con 0x7f1ef40b4000 2024-04-26T23:37:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef40b4ad0 con 0x7f1ef40b4000 2024-04-26T23:37:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.060+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef40b5090 con 0x7f1ef40b4000 2024-04-26T23:37:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.062+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef40693b0 con 0x7f1ef40b4000 2024-04-26T23:37:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.062+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ee4044020 con 0x7f1ef40b4000 2024-04-26T23:37:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.063+0000 7f1eea000640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 0x7f1ed006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.063+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ee40ac6c0 con 0x7f1ef40b4000 2024-04-26T23:37:48.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.064+0000 7f1eebe00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 0x7f1ed006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.065+0000 7f1eebe00640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 0x7f1ed006ee30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ef41f8830 tx=0x7f1edc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.067+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ee404b030 con 0x7f1ef40b4000 2024-04-26T23:37:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.274+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef40b79b0 con 0x7f1ef40b4000 2024-04-26T23:37:48.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.276+0000 7f1eea000640 1 -- 172.21.15.46:0/880710590 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f1ee407a790 con 0x7f1ef40b4000 2024-04-26T23:37:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:48.280 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 msgr2=0x7f1ed006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 0x7f1ed006ee30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ef41f8830 tx=0x7f1edc008040 comp rx=0 tx=0).stop 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 msgr2=0x7f1ef41573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee40046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 shutdown_connections 2024-04-26T23:37:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ed006c970 0x7f1ed006ee30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef415f4e0 0x7f1ef415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ef40d1f60 0x7f1ef40b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 --2- 172.21.15.46:0/880710590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef40b4000 0x7f1ef41573a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 >> 172.21.15.46:0/880710590 conn(0x7f1ef40d5010 msgr2=0x7f1ef41601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.279+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 shutdown_connections 2024-04-26T23:37:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.280+0000 7f1ef3200640 1 -- 172.21.15.46:0/880710590 wait complete. 2024-04-26T23:37:48.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:37:48.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:37:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.526+0000 7f8cd6c00640 1 Processor -- start 2024-04-26T23:37:48.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.526+0000 7f8cd6c00640 1 -- start start 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.527+0000 7f8cd6c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8cd01626c0 0x7f8cd00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd6c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cd0162aa0 0x7f8cd00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd6c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd00d07d0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd6c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd00d0ab0 con 0x7f8cd0162aa0 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd6c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd00d0d90 con 0x7f8cd01626c0 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd4e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8cd00b5fb0 0x7f8cd00d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39906/0 (socket says 172.21.15.46:39906) 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd4e00640 1 -- 172.21.15.46:0/1532154983 learned_addr learned my addr 172.21.15.46:0/1532154983 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd6200640 1 --2- 172.21.15.46:0/1532154983 >> v2:172.21.15.46:3300/0 conn(0x7f8cd0162aa0 0x7f8cd00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8ccfe00640 1 -- 172.21.15.46:0/1532154983 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491664059 0 0) 0x7f8cd00d07d0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8cd5800640 1 --2- 172.21.15.46:0/1532154983 >> v2:172.21.15.142:3300/0 conn(0x7f8cd01626c0 0x7f8cd00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.528+0000 7f8ccfe00640 1 -- 172.21.15.46:0/1532154983 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cb8003520 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.529+0000 7f8cd6200640 1 -- 172.21.15.46:0/1532154983 >> v2:172.21.15.142:3300/0 conn(0x7f8cd01626c0 msgr2=0x7f8cd00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.529+0000 7f8cd6200640 1 --2- 172.21.15.46:0/1532154983 >> v2:172.21.15.142:3300/0 conn(0x7f8cd01626c0 0x7f8cd00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.529+0000 7f8cd6200640 1 -- 172.21.15.46:0/1532154983 >> v1:172.21.15.46:6789/0 conn(0x7f8cd00b5fb0 legacy=0x7f8cd00d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.529+0000 7f8cd6200640 1 -- 172.21.15.46:0/1532154983 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd00d2c80 con 0x7f8cd0162aa0 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.530+0000 7f8cd6200640 1 --2- 172.21.15.46:0/1532154983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00d1900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8cd008f2e0 tx=0x7f8cc802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a25ac26b61d5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.530+0000 7f8ccfe00640 1 -- 172.21.15.46:0/1532154983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8007740 con 0x7f8cd0162aa0 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.530+0000 7f8ccfe00640 1 -- 172.21.15.46:0/1532154983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc80078e0 con 0x7f8cd0162aa0 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.530+0000 7f8ccfe00640 1 -- 172.21.15.46:0/1532154983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8007cb0 con 0x7f8cd0162aa0 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1532154983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd0162aa0 msgr2=0x7f8cd00d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1532154983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00d1900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8cd008f2e0 tx=0x7f8cc802a680 comp rx=0 tx=0).stop 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1532154983 shutdown_connections 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1532154983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00d1900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1532154983 >> v2:172.21.15.142:3300/0 conn(0x7f8cd01626c0 0x7f8cd00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1532154983 >> 172.21.15.46:0/1532154983 conn(0x7f8cd009e2a0 msgr2=0x7f8cd00d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1532154983 shutdown_connections 2024-04-26T23:37:48.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.531+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1532154983 wait complete. 2024-04-26T23:37:48.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.533+0000 7f8cd6c00640 1 Processor -- start 2024-04-26T23:37:48.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.533+0000 7f8cd6c00640 1 -- start start 2024-04-26T23:37:48.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd01626c0 0x7f8cd00b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00d2210 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00d2090 con 0x7f8cd0162aa0 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd00d31a0 con 0x7f8cd01626c0 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39512/0 (socket says 172.21.15.46:39512) 2024-04-26T23:37:48.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 -- 172.21.15.46:0/1679316673 learned_addr learned my addr 172.21.15.46:0/1679316673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:48.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd6200640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd4e00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd01626c0 0x7f8cd00b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 -- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd01626c0 msgr2=0x7f8cd00b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd01626c0 0x7f8cd00b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 -- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd0162aa0 msgr2=0x7f8cd00b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.534+0000 7f8cd5800640 1 -- 172.21.15.46:0/1679316673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd00b24c0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8cd5800640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d800 tx=0x7f8cc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400b6c0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc4019070 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd00b27a0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400ba80 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.535+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd01e9580 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.536+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd00693b0 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.537+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cc400bc20 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.538+0000 7f8ccd600640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 0x7f8cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.538+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8cc408c320 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.539+0000 7f8cd4e00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 0x7f8cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.540+0000 7f8cd4e00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 0x7f8cb006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc001360 tx=0x7f8cbc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.543+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cc405a400 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.746+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cd00b2970 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.747+0000 7f8ccd600640 1 -- 172.21.15.46:0/1679316673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f8cc4059d00 con 0x7f8cd00b5fb0 2024-04-26T23:37:48.751 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.749+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 msgr2=0x7f8cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.749+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 0x7f8cb006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc001360 tx=0x7f8cbc009040 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 msgr2=0x7f8cd01e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d800 tx=0x7f8cc400dcc0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 shutdown_connections 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8cb006c8f0 0x7f8cb006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd0162aa0 0x7f8cd00b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd01626c0 0x7f8cd00b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 --2- 172.21.15.46:0/1679316673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd00b5fb0 0x7f8cd01e9040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 >> 172.21.15.46:0/1679316673 conn(0x7f8cd009e2a0 msgr2=0x7f8cd00b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 shutdown_connections 2024-04-26T23:37:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.750+0000 7f8cd6c00640 1 -- 172.21.15.46:0/1679316673 wait complete. 2024-04-26T23:37:48.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:37:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.991+0000 7f167c000640 1 Processor -- start 2024-04-26T23:37:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.991+0000 7f167c000640 1 -- start start 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f167415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16740d07e0 con 0x7f167415e050 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167c000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167406eb90 con 0x7f167415d730 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16740b4310 con 0x7f16740b53b0 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f167415e050 0x7f167406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39914/0 (socket says 172.21.15.46:39914) 2024-04-26T23:37:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.993+0000 7f167a000640 1 -- 172.21.15.46:0/2619630031 learned_addr learned my addr 172.21.15.46:0/2619630031 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1678c00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f167415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898718266 0 0) 0x7f16740d07e0 con 0x7f167415e050 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f165c003520 con 0x7f167415e050 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1236802054 0 0) 0x7f165c003520 con 0x7f167415e050 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 msgr2=0x7f167415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f167415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 >> v1:172.21.15.142:6789/0 conn(0x7f167415d730 legacy=0x7f16740d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16740b45f0 con 0x7f167415e050 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f166c0019f0 con 0x7f167415e050 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 >> v1:172.21.15.46:6789/0 conn(0x7f167415e050 legacy=0x7f167406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165c0039e0 0x7f165c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.994+0000 7f1673e00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f165c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1673e00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f165c004d40 0x7f165c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16740b4310 con 0x7f165c0039e0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f165c00b9c0 con 0x7f165c004d40 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f165c00bbc0 con 0x7f16740b53b0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165c0039e0 0x7f165c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1678c00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f165c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 -- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 msgr2=0x7f165c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f167a000640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f165c004d40 0x7f165c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f165c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 -- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f165c004d40 msgr2=0x7f165c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f165c004d40 0x7f165c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1679600640 1 -- 172.21.15.46:0/2619630031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c003520 con 0x7f165c0039e0 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f1678c00640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16740b53b0 0x7f165c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.995+0000 7f167a000640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f165c004d40 0x7f165c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.996+0000 7f1679600640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165c0039e0 0x7f165c003dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1668001860 tx=0x7f165c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7608f0c0b1fda78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668037070 con 0x7f165c0039e0 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166802ab60 con 0x7f165c0039e0 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f1673e00640 1 -- 172.21.15.46:0/2619630031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668001430 con 0x7f165c0039e0 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 -- 172.21.15.46:0/2619630031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165c0039e0 msgr2=0x7f165c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 --2- 172.21.15.46:0/2619630031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165c0039e0 0x7f165c003dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1668001860 tx=0x7f165c005830 comp rx=0 tx=0).stop 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167a000640 1 -- 172.21.15.46:0/2619630031 reap_dead start 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 -- 172.21.15.46:0/2619630031 shutdown_connections 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 -- 172.21.15.46:0/2619630031 >> 172.21.15.46:0/2619630031 conn(0x7f167406c980 msgr2=0x7f16740b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 -- 172.21.15.46:0/2619630031 shutdown_connections 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.997+0000 7f167c000640 1 -- 172.21.15.46:0/2619630031 wait complete. 2024-04-26T23:37:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.999+0000 7f167c000640 1 Processor -- start 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:48.999+0000 7f167c000640 1 -- start start 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f165c0039e0 0x7f167413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16740cd120 0x7f167415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16740b4e90 con 0x7f16740cd580 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16740b4bb0 con 0x7f16740cd120 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.000+0000 7f167c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f167415e4c0 con 0x7f165c0039e0 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39522/0 (socket says 172.21.15.46:39522) 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 -- 172.21.15.46:0/2826738370 learned_addr learned my addr 172.21.15.46:0/2826738370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f1679600640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f165c0039e0 0x7f167413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f1678c00640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16740cd120 0x7f167415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 -- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f165c0039e0 msgr2=0x7f167413c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f165c0039e0 0x7f167413c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 -- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16740cd120 msgr2=0x7f167415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16740cd120 0x7f167415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.001+0000 7f167a000640 1 -- 172.21.15.46:0/2826738370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16740cb020 con 0x7f16740cd580 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f167a000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f167415e430 tx=0x7f166c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c00bb80 con 0x7f16740cd580 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c00bd20 con 0x7f16740cd580 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16740cb300 con 0x7f16740cd580 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c007270 con 0x7f16740cd580 2024-04-26T23:37:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.002+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16740cb8c0 con 0x7f16740cd580 2024-04-26T23:37:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.003+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1674069530 con 0x7f16740cd580 2024-04-26T23:37:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.004+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f166c01b020 con 0x7f16740cd580 2024-04-26T23:37:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.005+0000 7f1671600640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.005+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f166c08c1a0 con 0x7f16740cd580 2024-04-26T23:37:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.006+0000 7f1679600640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.007+0000 7f1679600640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1668034000 tx=0x7f1668009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.010+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f166c05a1f0 con 0x7f16740cd580 2024-04-26T23:37:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.218+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1674142de0 con 0x7f16740cd580 2024-04-26T23:37:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.219+0000 7f1671600640 1 -- 172.21.15.46:0/2826738370 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f166c059af0 con 0x7f16740cd580 2024-04-26T23:37:49.223 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 msgr2=0x7f165406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1668034000 tx=0x7f1668009030 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 msgr2=0x7f167415cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f167415e430 tx=0x7f166c00b4a0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 shutdown_connections 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16740cd580 0x7f167415cd40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16740cd120 0x7f167415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 --2- 172.21.15.46:0/2826738370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f165c0039e0 0x7f167413c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 >> 172.21.15.46:0/2826738370 conn(0x7f167406c980 msgr2=0x7f1674073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:49.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.222+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 shutdown_connections 2024-04-26T23:37:49.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.223+0000 7f167c000640 1 -- 172.21.15.46:0/2826738370 wait complete. 2024-04-26T23:37:49.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:37:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.454+0000 7ff23fe00640 1 Processor -- start 2024-04-26T23:37:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.454+0000 7ff23fe00640 1 -- start start 2024-04-26T23:37:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.455+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 0x7ff2380d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.455+0000 7ff23fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2380b53b0 0x7ff23815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23806eb90 con 0x7ff23815e050 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff2380b4310 con 0x7ff2380b53b0 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2380b45f0 con 0x7ff23815d730 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2380b53b0 0x7ff23815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2380b53b0 0x7ff23815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39524/0 (socket says 172.21.15.46:39524) 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23ca00640 1 -- 172.21.15.46:0/870031076 learned_addr learned my addr 172.21.15.46:0/870031076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.456+0000 7ff23d400640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 0x7ff2380d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.457+0000 7ff23ca00640 1 -- 172.21.15.46:0/870031076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 msgr2=0x7ff2380d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.457+0000 7ff23ca00640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 0x7ff2380d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.457+0000 7ff23ca00640 1 -- 172.21.15.46:0/870031076 >> v1:172.21.15.142:6789/0 conn(0x7ff23815e050 legacy=0x7ff23806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.457+0000 7ff23ca00640 1 -- 172.21.15.46:0/870031076 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2380b48d0 con 0x7ff2380b53b0 2024-04-26T23:37:49.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.457+0000 7ff23d400640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 0x7ff2380d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff23ca00640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2380b53b0 0x7ff23815db10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff224009870 tx=0x7ff22402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c619820050ea79a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff237e00640 1 -- 172.21.15.46:0/870031076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224037070 con 0x7ff2380b53b0 2024-04-26T23:37:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff237e00640 1 -- 172.21.15.46:0/870031076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff224032440 con 0x7ff2380b53b0 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff237e00640 1 -- 172.21.15.46:0/870031076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224032810 con 0x7ff2380b53b0 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff23fe00640 1 -- 172.21.15.46:0/870031076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2380b53b0 msgr2=0x7ff23815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.458+0000 7ff23fe00640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2380b53b0 0x7ff23815db10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff224009870 tx=0x7ff22402a270 comp rx=0 tx=0).stop 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 -- 172.21.15.46:0/870031076 shutdown_connections 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2380b53b0 0x7ff23815db10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 --2- 172.21.15.46:0/870031076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff23815d730 0x7ff2380d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 -- 172.21.15.46:0/870031076 >> 172.21.15.46:0/870031076 conn(0x7ff23806c980 msgr2=0x7ff2380b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 -- 172.21.15.46:0/870031076 shutdown_connections 2024-04-26T23:37:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.459+0000 7ff23fe00640 1 -- 172.21.15.46:0/870031076 wait complete. 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.460+0000 7ff23fe00640 1 Processor -- start 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.461+0000 7ff23fe00640 1 -- start start 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2380b53b0 0x7ff2380b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 0x7ff23815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff23815e4c0 con 0x7ff23815e050 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2380b4e90 con 0x7ff23815d730 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2380b3ec0 con 0x7ff2380b53b0 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39536/0 (socket says 172.21.15.46:39536) 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23de00640 1 -- 172.21.15.46:0/2124850890 learned_addr learned my addr 172.21.15.46:0/2124850890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23ca00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 0x7ff23815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.462+0000 7ff23d400640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2380b53b0 0x7ff2380b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 -- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2380b53b0 msgr2=0x7ff2380b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2380b53b0 0x7ff2380b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 -- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 msgr2=0x7ff23815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 0x7ff23815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 -- 172.21.15.46:0/2124850890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff238142df0 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23ca00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 0x7ff23815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23de00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c7a0 tx=0x7ff23000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230003b70 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff230003d30 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2380b2ed0 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23000aa90 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.463+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23815b630 con 0x7ff23815e050 2024-04-26T23:37:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.465+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238069530 con 0x7ff23815e050 2024-04-26T23:37:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.466+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff23000ac30 con 0x7ff23815e050 2024-04-26T23:37:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.467+0000 7ff235600640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 0x7ff21806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.467+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff23008bdc0 con 0x7ff23815e050 2024-04-26T23:37:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.467+0000 7ff23d400640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 0x7ff21806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.468+0000 7ff23d400640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 0x7ff21806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006e80 tx=0x7ff22c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.471+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff230059e10 con 0x7ff23815e050 2024-04-26T23:37:49.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.673+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff2380008d0 con 0x7ff23815e050 2024-04-26T23:37:49.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.673+0000 7ff235600640 1 -- 172.21.15.46:0/2124850890 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7ff230059710 con 0x7ff23815e050 2024-04-26T23:37:49.677 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:37:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 msgr2=0x7ff21806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 0x7ff21806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006e80 tx=0x7ff22c008040 comp rx=0 tx=0).stop 2024-04-26T23:37:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 msgr2=0x7ff2381ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c7a0 tx=0x7ff23000cc60 comp rx=0 tx=0).stop 2024-04-26T23:37:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 shutdown_connections 2024-04-26T23:37:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff21806c8f0 0x7ff21806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff23815e050 0x7ff2381ea9a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff23815d730 0x7ff23815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 --2- 172.21.15.46:0/2124850890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2380b53b0 0x7ff2380b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 >> 172.21.15.46:0/2124850890 conn(0x7ff23806c980 msgr2=0x7ff2380af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 shutdown_connections 2024-04-26T23:37:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.676+0000 7ff23fe00640 1 -- 172.21.15.46:0/2124850890 wait complete. 2024-04-26T23:37:49.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:37:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.905+0000 7f3513200640 1 Processor -- start 2024-04-26T23:37:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.905+0000 7f3513200640 1 -- start start 2024-04-26T23:37:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3513200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 0x7f3514160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3513200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f351415cc40 0x7f35140cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3513200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35140cbe60 con 0x7f351415d0a0 2024-04-26T23:37:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3513200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f35140cca70 con 0x7f351415cc40 2024-04-26T23:37:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3513200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f35140cc140 con 0x7f3514160130 2024-04-26T23:37:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f350be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f351415cc40 0x7f35140cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.907+0000 7f3511e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 0x7f3514160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f351415cc40 0x7f35140cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 -- 172.21.15.46:0/668324974 learned_addr learned my addr 172.21.15.46:0/668324974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 -- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 msgr2=0x7f3514160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 0x7f3514160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 -- 172.21.15.46:0/668324974 >> v1:172.21.15.142:6789/0 conn(0x7f351415d0a0 legacy=0x7f35140cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f350be00640 1 -- 172.21.15.46:0/668324974 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35140cc420 con 0x7f351415cc40 2024-04-26T23:37:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.908+0000 7f3511e00640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 0x7f3514160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:37:49.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.910+0000 7f350be00640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415cc40 0x7f35140cd550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009870 tx=0x7f34fc02a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28457f263427df97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.910+0000 7f3511400640 1 -- 172.21.15.46:0/668324974 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc037070 con 0x7f351415cc40 2024-04-26T23:37:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.910+0000 7f3511400640 1 -- 172.21.15.46:0/668324974 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc032440 con 0x7f351415cc40 2024-04-26T23:37:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.910+0000 7f3511400640 1 -- 172.21.15.46:0/668324974 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc032810 con 0x7f351415cc40 2024-04-26T23:37:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 -- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415cc40 msgr2=0x7f35140cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415cc40 0x7f35140cd550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009870 tx=0x7f34fc02a1d0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 -- 172.21.15.46:0/668324974 shutdown_connections 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415cc40 0x7f35140cd550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 --2- 172.21.15.46:0/668324974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3514160130 0x7f3514160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 -- 172.21.15.46:0/668324974 >> 172.21.15.46:0/668324974 conn(0x7f35140b27b0 msgr2=0x7f35140b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 -- 172.21.15.46:0/668324974 shutdown_connections 2024-04-26T23:37:49.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.911+0000 7f3513200640 1 -- 172.21.15.46:0/668324974 wait complete. 2024-04-26T23:37:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.913+0000 7f3513200640 1 Processor -- start 2024-04-26T23:37:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.914+0000 7f3513200640 1 -- start start 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f351415cc40 0x7f351414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 0x7f35140c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35140733b0 con 0x7f3514160130 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3514073230 con 0x7f351415d0a0 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3513200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3514073530 con 0x7f351415cc40 2024-04-26T23:37:49.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3512800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3512800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39540/0 (socket says 172.21.15.46:39540) 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3512800640 1 -- 172.21.15.46:0/1142956510 learned_addr learned my addr 172.21.15.46:0/1142956510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f3511e00640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f351415cc40 0x7f351414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.915+0000 7f350be00640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 0x7f35140c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 -- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f351415cc40 msgr2=0x7f351414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f351415cc40 0x7f351414c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 -- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 msgr2=0x7f35140c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 0x7f35140c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 -- 172.21.15.46:0/1142956510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35141506f0 con 0x7f3514160130 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f350be00640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 0x7f35140c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3512800640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c730 tx=0x7f350c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c003b70 con 0x7f3514160130 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c003d30 con 0x7f3514160130 2024-04-26T23:37:49.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c005310 con 0x7f3514160130 2024-04-26T23:37:49.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35140c6f30 con 0x7f3514160130 2024-04-26T23:37:49.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.916+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3514153be0 con 0x7f3514160130 2024-04-26T23:37:49.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.918+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35141508c0 con 0x7f3514160130 2024-04-26T23:37:49.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.918+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f350c0056c0 con 0x7f3514160130 2024-04-26T23:37:49.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.919+0000 7f350a000640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 0x7f34f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:49.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.920+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f350c08bf70 con 0x7f3514160130 2024-04-26T23:37:49.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.920+0000 7f3511e00640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 0x7f34f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:49.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.921+0000 7f3511e00640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 0x7f34f006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3504009a30 tx=0x7f3504008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:49.923+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f350c059fc0 con 0x7f3514160130 2024-04-26T23:37:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.121+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f35140693b0 con 0x7f3514160130 2024-04-26T23:37:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.125+0000 7f350a000640 1 -- 172.21.15.46:0/1142956510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f350c0598c0 con 0x7f3514160130 2024-04-26T23:37:50.129 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 msgr2=0x7f34f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 0x7f34f006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3504009a30 tx=0x7f3504008040 comp rx=0 tx=0).stop 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 msgr2=0x7f35140c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c730 tx=0x7f350c00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 shutdown_connections 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34f006c8f0 0x7f34f006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3514160130 0x7f35140c9ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f351415d0a0 0x7f35140c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 --2- 172.21.15.46:0/1142956510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f351415cc40 0x7f351414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 >> 172.21.15.46:0/1142956510 conn(0x7f35140b27b0 msgr2=0x7f351415b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.128+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 shutdown_connections 2024-04-26T23:37:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.129+0000 7f3513200640 1 -- 172.21.15.46:0/1142956510 wait complete. 2024-04-26T23:37:50.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:50.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.360+0000 7f60d6600640 1 Processor -- start 2024-04-26T23:37:50.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.360+0000 7f60d6600640 1 -- start start 2024-04-26T23:37:50.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d6600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60d015edf0 0x7f60d00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60d015f1d0 0x7f60d00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d6600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d00b03c0 con 0x7f60d00cb1a0 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d6600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f60d00cf5c0 con 0x7f60d015f1d0 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d6600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f60d00cb680 con 0x7f60d015edf0 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60d00cb1a0 0x7f60d00ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39926/0 (socket says 172.21.15.46:39926) 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d5c00640 1 -- 172.21.15.46:0/1036454531 learned_addr learned my addr 172.21.15.46:0/1036454531 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60cfe00640 1 --2- 172.21.15.46:0/1036454531 >> v2:172.21.15.46:3300/0 conn(0x7f60d015f1d0 0x7f60d00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.362+0000 7f60d5200640 1 --2- 172.21.15.46:0/1036454531 >> v2:172.21.15.142:3300/0 conn(0x7f60d015edf0 0x7f60d00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.363+0000 7f60cfe00640 1 -- 172.21.15.46:0/1036454531 >> v2:172.21.15.142:3300/0 conn(0x7f60d015edf0 msgr2=0x7f60d00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.363+0000 7f60cfe00640 1 --2- 172.21.15.46:0/1036454531 >> v2:172.21.15.142:3300/0 conn(0x7f60d015edf0 0x7f60d00cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.363+0000 7f60cfe00640 1 -- 172.21.15.46:0/1036454531 >> v1:172.21.15.46:6789/0 conn(0x7f60d00cb1a0 legacy=0x7f60d00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.363+0000 7f60cfe00640 1 -- 172.21.15.46:0/1036454531 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00cb960 con 0x7f60d015f1d0 2024-04-26T23:37:50.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.364+0000 7f60cfe00640 1 --2- 172.21.15.46:0/1036454531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d015f1d0 0x7f60d00cd060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d0a00 tx=0x7f60bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6045017523efbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.364+0000 7f60cf400640 1 -- 172.21.15.46:0/1036454531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc037070 con 0x7f60d015f1d0 2024-04-26T23:37:50.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.364+0000 7f60cf400640 1 -- 172.21.15.46:0/1036454531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc032440 con 0x7f60d015f1d0 2024-04-26T23:37:50.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.364+0000 7f60cf400640 1 -- 172.21.15.46:0/1036454531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc032810 con 0x7f60d015f1d0 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 -- 172.21.15.46:0/1036454531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d015f1d0 msgr2=0x7f60d00cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 --2- 172.21.15.46:0/1036454531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d015f1d0 0x7f60d00cd060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d0a00 tx=0x7f60bc02a230 comp rx=0 tx=0).stop 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 -- 172.21.15.46:0/1036454531 shutdown_connections 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 --2- 172.21.15.46:0/1036454531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d015f1d0 0x7f60d00cd060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 --2- 172.21.15.46:0/1036454531 >> v2:172.21.15.142:3300/0 conn(0x7f60d015edf0 0x7f60d00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 -- 172.21.15.46:0/1036454531 >> 172.21.15.46:0/1036454531 conn(0x7f60d00b26d0 msgr2=0x7f60d00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 -- 172.21.15.46:0/1036454531 shutdown_connections 2024-04-26T23:37:50.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.365+0000 7f60d6600640 1 -- 172.21.15.46:0/1036454531 wait complete. 2024-04-26T23:37:50.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.367+0000 7f60d6600640 1 Processor -- start 2024-04-26T23:37:50.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.367+0000 7f60d6600640 1 -- start start 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d00cb1a0 0x7f60d015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d015edf0 0x7f60d015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00cc200 con 0x7f60d014fc70 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00cbf20 con 0x7f60d00cb1a0 2024-04-26T23:37:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d6600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60d00c6950 con 0x7f60d015edf0 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39550/0 (socket says 172.21.15.46:39550) 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 -- 172.21.15.46:0/3226974763 learned_addr learned my addr 172.21.15.46:0/3226974763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5200640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d00cb1a0 0x7f60d015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60cfe00640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d015edf0 0x7f60d015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 -- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d015edf0 msgr2=0x7f60d015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d015edf0 0x7f60d015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 -- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d00cb1a0 msgr2=0x7f60d015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d00cb1a0 0x7f60d015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.368+0000 7f60d5c00640 1 -- 172.21.15.46:0/3226974763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d0160830 con 0x7f60d014fc70 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60d5c00640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60c800cb40 tx=0x7f60c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8017070 con 0x7f60d014fc70 2024-04-26T23:37:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c800abe0 con 0x7f60d014fc70 2024-04-26T23:37:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d0160b10 con 0x7f60d014fc70 2024-04-26T23:37:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8012400 con 0x7f60d014fc70 2024-04-26T23:37:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.369+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01610d0 con 0x7f60d014fc70 2024-04-26T23:37:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.370+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d00693b0 con 0x7f60d014fc70 2024-04-26T23:37:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.371+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60c801a070 con 0x7f60d014fc70 2024-04-26T23:37:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.372+0000 7f60ccc00640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 0x7f60b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.372+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60c808bc40 con 0x7f60d014fc70 2024-04-26T23:37:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.373+0000 7f60d5200640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 0x7f60b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.374+0000 7f60d5200640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 0x7f60b006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.377+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60c8059c90 con 0x7f60d014fc70 2024-04-26T23:37:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.621+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d015d770 con 0x7f60d014fc70 2024-04-26T23:37:50.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.622+0000 7f60ccc00640 1 -- 172.21.15.46:0/3226974763 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f60c808b070 con 0x7f60d014fc70 2024-04-26T23:37:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:50.626 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:50.626 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:37:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.624+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 msgr2=0x7f60b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.624+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 0x7f60b006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).stop 2024-04-26T23:37:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.624+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 msgr2=0x7f60d00cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.624+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60c800cb40 tx=0x7f60c800a060 comp rx=0 tx=0).stop 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 shutdown_connections 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60b006c920 0x7f60b006ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d014fc70 0x7f60d00cc620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d015edf0 0x7f60d015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 --2- 172.21.15.46:0/3226974763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d00cb1a0 0x7f60d015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 >> 172.21.15.46:0/3226974763 conn(0x7f60d00b26d0 msgr2=0x7f60d00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 shutdown_connections 2024-04-26T23:37:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.625+0000 7f60d6600640 1 -- 172.21.15.46:0/3226974763 wait complete. 2024-04-26T23:37:50.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:37:50.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.845+0000 7febb4200640 1 Processor -- start 2024-04-26T23:37:50.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.846+0000 7febb4200640 1 -- start start 2024-04-26T23:37:50.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.847+0000 7febb4200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febac145770 0x7febac0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.847+0000 7febb4200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7febac15ccd0 0x7febac0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.847+0000 7febb4200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac0cf950 con 0x7febac072e60 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.847+0000 7febb4200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7febac0d0c50 con 0x7febac145770 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.847+0000 7febb4200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7febac15d490 con 0x7febac15ccd0 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febac145770 0x7febac0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febac145770 0x7febac0d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39566/0 (socket says 172.21.15.46:39566) 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 -- 172.21.15.46:0/3035052694 learned_addr learned my addr 172.21.15.46:0/3035052694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:50.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb0e00640 1 --1- 172.21.15.46:0/3035052694 >> v1:172.21.15.46:6789/0 conn(0x7febac072e60 0x7febac15c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39932/0 (socket says 172.21.15.46:39932) 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb2200640 1 --2- 172.21.15.46:0/3035052694 >> v2:172.21.15.142:3300/0 conn(0x7febac15ccd0 0x7febac0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 -- 172.21.15.46:0/3035052694 >> v2:172.21.15.142:3300/0 conn(0x7febac15ccd0 msgr2=0x7febac0732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 --2- 172.21.15.46:0/3035052694 >> v2:172.21.15.142:3300/0 conn(0x7febac15ccd0 0x7febac0732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 -- 172.21.15.46:0/3035052694 >> v1:172.21.15.46:6789/0 conn(0x7febac072e60 legacy=0x7febac15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febb1800640 1 -- 172.21.15.46:0/3035052694 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0cfc30 con 0x7febac145770 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.848+0000 7febabe00640 1 -- 172.21.15.46:0/3035052694 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2887591943 0 0) 0x7febac0cf950 con 0x7febac072e60 2024-04-26T23:37:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.849+0000 7febb1800640 1 --2- 172.21.15.46:0/3035052694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac145770 0x7febac0d0710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febac0b5fc0 tx=0x7feba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4298cf2b73bdaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febabe00640 1 -- 172.21.15.46:0/3035052694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0037070 con 0x7febac145770 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febabe00640 1 -- 172.21.15.46:0/3035052694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba0032440 con 0x7febac145770 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febabe00640 1 -- 172.21.15.46:0/3035052694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0032810 con 0x7febac145770 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 -- 172.21.15.46:0/3035052694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac145770 msgr2=0x7febac0d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 --2- 172.21.15.46:0/3035052694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac145770 0x7febac0d0710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febac0b5fc0 tx=0x7feba002a270 comp rx=0 tx=0).stop 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 -- 172.21.15.46:0/3035052694 shutdown_connections 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 --2- 172.21.15.46:0/3035052694 >> v2:172.21.15.142:3300/0 conn(0x7febac15ccd0 0x7febac0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 --2- 172.21.15.46:0/3035052694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac145770 0x7febac0d0710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 -- 172.21.15.46:0/3035052694 >> 172.21.15.46:0/3035052694 conn(0x7febac06c530 msgr2=0x7febac06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 -- 172.21.15.46:0/3035052694 shutdown_connections 2024-04-26T23:37:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.850+0000 7febb4200640 1 -- 172.21.15.46:0/3035052694 wait complete. 2024-04-26T23:37:50.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.852+0000 7febb4200640 1 Processor -- start 2024-04-26T23:37:50.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.852+0000 7febb4200640 1 -- start start 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febac072e60 0x7febac1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 0x7febac143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d01f0 con 0x7febac15ccd0 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7febac073800 con 0x7febac145770 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb4200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7febac0cf5b0 con 0x7febac072e60 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39578/0 (socket says 172.21.15.46:39578) 2024-04-26T23:37:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.853+0000 7febb2200640 1 -- 172.21.15.46:0/370914166 learned_addr learned my addr 172.21.15.46:0/370914166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb1800640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febac072e60 0x7febac1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 -- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febac072e60 msgr2=0x7febac1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febac072e60 0x7febac1f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb0e00640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 0x7febac143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 -- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 msgr2=0x7febac143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 0x7febac143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 -- 172.21.15.46:0/370914166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac142070 con 0x7febac15ccd0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb0e00640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 0x7febac143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7febb2200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feba4007d80 tx=0x7feba400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.854+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4014420 con 0x7febac15ccd0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.855+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4019070 con 0x7febac15ccd0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.855+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac142350 con 0x7febac15ccd0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.855+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4014800 con 0x7febac15ccd0 2024-04-26T23:37:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.855+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac142910 con 0x7febac15ccd0 2024-04-26T23:37:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.856+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac069530 con 0x7febac15ccd0 2024-04-26T23:37:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.857+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feba40149a0 con 0x7febac15ccd0 2024-04-26T23:37:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.858+0000 7feba9600640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 0x7feb8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.858+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feba408c800 con 0x7febac15ccd0 2024-04-26T23:37:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.858+0000 7febb1800640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 0x7feb8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.859+0000 7febb1800640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 0x7feb8c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feba0034000 tx=0x7feba002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:50.862+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feba405a7c0 con 0x7febac15ccd0 2024-04-26T23:37:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.063+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febac0b6060 con 0x7febac15ccd0 2024-04-26T23:37:51.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.067+0000 7feba9600640 1 -- 172.21.15.46:0/370914166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7feba405a0c0 con 0x7febac15ccd0 2024-04-26T23:37:51.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:51.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:13.283679+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.069+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 msgr2=0x7feb8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.069+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 0x7feb8c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feba0034000 tx=0x7feba002e000 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.069+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 msgr2=0x7febac143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.069+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feba4007d80 tx=0x7feba400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 shutdown_connections 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb8c06c920 0x7feb8c06ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febac15ccd0 0x7febac143650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac145770 0x7febac143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 --2- 172.21.15.46:0/370914166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febac072e60 0x7febac1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 >> 172.21.15.46:0/370914166 conn(0x7febac06c530 msgr2=0x7febac06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 shutdown_connections 2024-04-26T23:37:51.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.070+0000 7febb4200640 1 -- 172.21.15.46:0/370914166 wait complete. 2024-04-26T23:37:51.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.302+0000 7f9148400640 1 Processor -- start 2024-04-26T23:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.302+0000 7f9148400640 1 -- start start 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.304+0000 7f9148400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9140165ee0 0x7f91401682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.304+0000 7f9148400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9140161940 con 0x7f914006e510 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.304+0000 7f9148400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9140071f80 con 0x7f9140160bd0 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.304+0000 7f9148400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f91400742d0 con 0x7f9140165ee0 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.304+0000 7f9145a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f914006e510 0x7f914006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39946/0 (socket says 172.21.15.46:39946) 2024-04-26T23:37:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f9145a00640 1 -- 172.21.15.46:0/4122741096 learned_addr learned my addr 172.21.15.46:0/4122741096 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f9146400640 1 --2- 172.21.15.46:0/4122741096 >> v2:172.21.15.142:3300/0 conn(0x7f9140165ee0 0x7f91401682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313558588 0 0) 0x7f9140161940 con 0x7f914006e510 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9128003520 con 0x7f914006e510 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2861504571 0 0) 0x7f9128003520 con 0x7f914006e510 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 >> v2:172.21.15.142:3300/0 conn(0x7f9140165ee0 msgr2=0x7f91401682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 --2- 172.21.15.46:0/4122741096 >> v2:172.21.15.142:3300/0 conn(0x7f9140165ee0 0x7f91401682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 >> v1:172.21.15.142:6789/0 conn(0x7f9140160bd0 legacy=0x7f9140161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9140072260 con 0x7f914006e510 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9134001670 con 0x7f914006e510 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.305+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 >> v1:172.21.15.46:6789/0 conn(0x7f914006e510 legacy=0x7f914006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280039b0 0x7f9128003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f9128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280051c0 0x7f912800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91400742d0 con 0x7f9128004800 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f912800c060 con 0x7f91280051c0 2024-04-26T23:37:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f912800c260 con 0x7f91280039b0 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f9128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9146400640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280051c0 0x7f912800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145a00640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280039b0 0x7f9128003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 -- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280039b0 msgr2=0x7f9128003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280039b0 0x7f9128003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 -- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280051c0 msgr2=0x7f912800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280051c0 0x7f912800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9145000640 1 -- 172.21.15.46:0/4122741096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9128003520 con 0x7f9128004800 2024-04-26T23:37:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.306+0000 7f9146400640 1 -- 172.21.15.46:0/4122741096 reap_dead start 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f9145000640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f9128004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f912c001750 tx=0x7f9128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60af6245b1fda402 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c001370 con 0x7f9128004800 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c0052d0 con 0x7f9128004800 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f913fe00640 1 -- 172.21.15.46:0/4122741096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c02abc0 con 0x7f9128004800 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f9148400640 1 -- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 msgr2=0x7f9128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.308+0000 7f9148400640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f9128004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f912c001750 tx=0x7f9128005830 comp rx=0 tx=0).stop 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.309+0000 7f9148400640 1 -- 172.21.15.46:0/4122741096 shutdown_connections 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.309+0000 7f9148400640 1 --2- 172.21.15.46:0/4122741096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f9128004c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.309+0000 7f9148400640 1 -- 172.21.15.46:0/4122741096 >> 172.21.15.46:0/4122741096 conn(0x7f914006c530 msgr2=0x7f914006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.309+0000 7f9148400640 1 -- 172.21.15.46:0/4122741096 shutdown_connections 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.309+0000 7f9148400640 1 -- 172.21.15.46:0/4122741096 wait complete. 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.310+0000 7f9148400640 1 Processor -- start 2024-04-26T23:37:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.311+0000 7f9148400640 1 -- start start 2024-04-26T23:37:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.311+0000 7f9148400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.311+0000 7f9148400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91401f7170 0x7f91400b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9148400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91400b1ae0 0x7f914015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9148400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9140072d40 con 0x7f9128004800 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9148400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9140072820 con 0x7f91400b1ae0 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9148400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9140072ec0 con 0x7f91401f7170 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39596/0 (socket says 172.21.15.46:39596) 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 -- 172.21.15.46:0/4038444275 learned_addr learned my addr 172.21.15.46:0/4038444275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145000640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91401f7170 0x7f91400b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9146400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91400b1ae0 0x7f914015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 -- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91401f7170 msgr2=0x7f91400b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91401f7170 0x7f91400b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 -- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91400b1ae0 msgr2=0x7f914015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91400b1ae0 0x7f914015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.312+0000 7f9145a00640 1 -- 172.21.15.46:0/4038444275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91400b3680 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f9145a00640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f913400d800 tx=0x7f913400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134019070 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91340074e0 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134004a00 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91400b0ac0 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.313+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91400b1080 con 0x7f9128004800 2024-04-26T23:37:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.314+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140069530 con 0x7f9128004800 2024-04-26T23:37:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.316+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f913401c070 con 0x7f9128004800 2024-04-26T23:37:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.316+0000 7f913d600640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 0x7f912006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.317+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f913408b830 con 0x7f9128004800 2024-04-26T23:37:51.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.317+0000 7f9145000640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 0x7f912006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:51.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.318+0000 7f9145000640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 0x7f912006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f912c032000 tx=0x7f912c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.321+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9134059910 con 0x7f9128004800 2024-04-26T23:37:51.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.567+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914006fb00 con 0x7f9128004800 2024-04-26T23:37:51.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.568+0000 7f913d600640 1 -- 172.21.15.46:0/4038444275 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9134059210 con 0x7f9128004800 2024-04-26T23:37:51.572 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:37:51.572 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:37:51.572 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 msgr2=0x7f912006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 0x7f912006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f912c032000 tx=0x7f912c02e000 comp rx=0 tx=0).stop 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 msgr2=0x7f914013c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f913400d800 tx=0x7f913400dcc0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 shutdown_connections 2024-04-26T23:37:51.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f912006c8f0 0x7f912006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91400b1ae0 0x7f914015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91401f7170 0x7f91400b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 --2- 172.21.15.46:0/4038444275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9128004800 0x7f914013c220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 >> 172.21.15.46:0/4038444275 conn(0x7f914006c530 msgr2=0x7f9140156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 shutdown_connections 2024-04-26T23:37:51.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:51.571+0000 7f9148400640 1 -- 172.21.15.46:0/4038444275 wait complete. 2024-04-26T23:37:51.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:37:08.251922+0000', 'modified': '2024-04-26T23:37:36.515301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6836/2711334262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2711334262}, {'type': 'v1', 'addr': '172.21.15.142:6836', 'nonce': 2711334262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-26T23:37:51.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:37:51.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:37:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.215+0000 7f9444600640 1 Processor -- start 2024-04-26T23:37:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.215+0000 7f9444600640 1 -- start start 2024-04-26T23:37:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9444600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f943c160bd0 0x7f943c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9444600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943c161940 con 0x7f943c0732c0 2024-04-26T23:37:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9444600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943c071f80 con 0x7f943c06e510 2024-04-26T23:37:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9444600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f943c072260 con 0x7f943c160bd0 2024-04-26T23:37:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9442600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f943c0732c0 0x7f943c073730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60168/0 (socket says 172.21.15.46:60168) 2024-04-26T23:37:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.217+0000 7f9442600640 1 -- 172.21.15.46:0/694461749 learned_addr learned my addr 172.21.15.46:0/694461749 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f9441200640 1 --2- 172.21.15.46:0/694461749 >> v2:172.21.15.142:3300/0 conn(0x7f943c160bd0 0x7f943c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583645041 0 0) 0x7f943c161940 con 0x7f943c0732c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9424003520 con 0x7f943c0732c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1076633910 0 0) 0x7f9424003520 con 0x7f943c0732c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 >> v2:172.21.15.142:3300/0 conn(0x7f943c160bd0 msgr2=0x7f943c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 --2- 172.21.15.46:0/694461749 >> v2:172.21.15.142:3300/0 conn(0x7f943c160bd0 0x7f943c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 >> v1:172.21.15.142:6789/0 conn(0x7f943c06e510 legacy=0x7f943c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c072540 con 0x7f943c0732c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9434001670 con 0x7f943c0732c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 >> v1:172.21.15.46:6789/0 conn(0x7f943c0732c0 legacy=0x7f943c073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9424003a30 0x7f9424003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9424004800 0x7f9424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 0x7f942400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f943c072260 con 0x7f94240051c0 2024-04-26T23:37:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f942400c0f0 con 0x7f9424003a30 2024-04-26T23:37:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.218+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f942400c2f0 con 0x7f9424004800 2024-04-26T23:37:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 0x7f942400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9441200640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9424004800 0x7f9424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9441c00640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9424003a30 0x7f9424003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 -- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9424004800 msgr2=0x7f9424004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9424004800 0x7f9424004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 -- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9424003a30 msgr2=0x7f9424003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9424003a30 0x7f9424003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9442600640 1 -- 172.21.15.46:0/694461749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9424003520 con 0x7f94240051c0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.219+0000 7f9441c00640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9424003a30 0x7f9424003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.220+0000 7f9442600640 1 -- 172.21.15.46:0/694461749 reap_dead start 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.220+0000 7f9442600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 0x7f942400b9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f943400c0e0 tx=0x7f9424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbda11b784255a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434007c20 con 0x7f94240051c0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9434005af0 con 0x7f94240051c0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f943be00640 1 -- 172.21.15.46:0/694461749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434007070 con 0x7f94240051c0 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 -- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 msgr2=0x7f942400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 0x7f942400b9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f943400c0e0 tx=0x7f9424005830 comp rx=0 tx=0).stop 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 -- 172.21.15.46:0/694461749 shutdown_connections 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 --2- 172.21.15.46:0/694461749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94240051c0 0x7f942400b9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 -- 172.21.15.46:0/694461749 >> 172.21.15.46:0/694461749 conn(0x7f943c06c530 msgr2=0x7f943c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 -- 172.21.15.46:0/694461749 shutdown_connections 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.221+0000 7f9444600640 1 -- 172.21.15.46:0/694461749 wait complete. 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.223+0000 7f9444600640 1 Processor -- start 2024-04-26T23:37:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.223+0000 7f9444600640 1 -- start start 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 0x7f943c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f943c1452c0 0x7f943c142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f943c072ec0 con 0x7f943c13bd60 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f943c072d40 con 0x7f943c1452c0 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.224+0000 7f9444600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f943c146b10 con 0x7f94240051c0 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55710/0 (socket says 172.21.15.46:55710) 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 -- 172.21.15.46:0/2349477361 learned_addr learned my addr 172.21.15.46:0/2349477361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9441c00640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 0x7f943c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9441200640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f943c1452c0 0x7f943c142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 -- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 msgr2=0x7f943c15a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 0x7f943c15a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 -- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f943c1452c0 msgr2=0x7f943c142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f943c1452c0 0x7f943c142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9442600640 1 -- 172.21.15.46:0/2349477361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c13c720 con 0x7f943c13bd60 2024-04-26T23:37:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.225+0000 7f9441c00640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 0x7f943c15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9442600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f943400c200 tx=0x7f943c161b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9439600640 1 -- 172.21.15.46:0/2349477361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434000ed0 con 0x7f943c13bd60 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9439600640 1 -- 172.21.15.46:0/2349477361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9434001640 con 0x7f943c13bd60 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9439600640 1 -- 172.21.15.46:0/2349477361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340047e0 con 0x7f943c13bd60 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c1457a0 con 0x7f943c13bd60 2024-04-26T23:37:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.226+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c145d60 con 0x7f943c13bd60 2024-04-26T23:37:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.229+0000 7f9439600640 1 -- 172.21.15.46:0/2349477361 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9434043020 con 0x7f943c13bd60 2024-04-26T23:37:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.230+0000 7f9439600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 0x7f941c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.230+0000 7f9439600640 1 -- 172.21.15.46:0/2349477361 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94340ab3d0 con 0x7f943c13bd60 2024-04-26T23:37:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.230+0000 7f94475d0740 1 Processor -- start 2024-04-26T23:37:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.230+0000 7f9441c00640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 0x7f941c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.230+0000 7f94475d0740 1 -- start start 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f7d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f7fd0a0 0x56018f7fd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56018f736600 con 0x56018f80ecb0 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56018f6d3100 con 0x56018f7d25e0 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.231+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56018f7325c0 con 0x56018f7fd0a0 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441c00640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 0x7f941c06ef80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9430000f80 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55722/0 (socket says 172.21.15.46:55722) 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 -- 172.21.15.46:0/2667929819 learned_addr learned my addr 172.21.15.46:0/2667929819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441c00640 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f7d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9442600640 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f7fd0a0 0x56018f7fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 -- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f7fd0a0 msgr2=0x56018f7fd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f7fd0a0 0x56018f7fd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 -- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 msgr2=0x56018f7d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f7d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.232+0000 7f9441200640 1 -- 172.21.15.46:0/2667929819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018f4482d0 con 0x56018f80ecb0 2024-04-26T23:37:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.233+0000 7f9441200640 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9428009a00 tx=0x7f9428009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351008940b06f082 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.233+0000 7f942e000640 1 -- 172.21.15.46:0/2667929819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801a070 con 0x56018f80ecb0 2024-04-26T23:37:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.233+0000 7f942e000640 1 -- 172.21.15.46:0/2667929819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428004230 con 0x56018f80ecb0 2024-04-26T23:37:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.233+0000 7f942e000640 1 -- 172.21.15.46:0/2667929819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428013960 con 0x56018f80ecb0 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 -- 172.21.15.46:0/2667929819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 msgr2=0x56018f7d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9428009a00 tx=0x7f9428009190 comp rx=0 tx=0).stop 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 -- 172.21.15.46:0/2667929819 shutdown_connections 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f7fd0a0 0x56018f7fd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f80ecb0 0x56018f7d29c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 --2- 172.21.15.46:0/2667929819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f7d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 -- 172.21.15.46:0/2667929819 >> 172.21.15.46:0/2667929819 conn(0x56018f766d10 msgr2=0x56018f8180a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 -- 172.21.15.46:0/2667929819 shutdown_connections 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.234+0000 7f94475d0740 1 -- 172.21.15.46:0/2667929819 wait complete. 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.235+0000 7f94475d0740 1 -- start start 2024-04-26T23:37:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.235+0000 7f94475d0740 1 Processor -- start 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56018f6bdf70 con 0x56018f7fd0a0 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56018f7fabf0 con 0x56018f7d25e0 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56018f81a5a0 con 0x56018f80ecb0 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f94475d0740 10 client.0 _ll_register_callbacks cb 0x56018f72c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f9441200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f9441200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55728/0 (socket says 172.21.15.46:55728) 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.236+0000 7f9441200640 1 -- 172.21.15.46:0/2507344223 learned_addr learned my addr 172.21.15.46:0/2507344223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441c00640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9442600640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 -- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 msgr2=0x56018f7f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 -- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 msgr2=0x56018f818700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 -- 172.21.15.46:0/2507344223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018f5126b0 con 0x56018f7fd0a0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9441200640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94280099d0 tx=0x7f94280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5584796b826e79e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9422a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801a070 con 0x56018f7fd0a0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9428005ba0 con 0x56018f7fd0a0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.237+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9428004230 con 0x56018f7fd0a0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9428005de0 con 0x56018f7fd0a0 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 10 client.4794 fetch_fsmap learned FSMap version 9 2024-04-26T23:37:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56018f81a2e0 con 0x56018f7fd0a0 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428004230 con 0x56018f7fd0a0 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9428032050 con 0x56018f7fd0a0 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 10 client.4794 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 20 client.4794 populate_metadata read hostname 'smithi046' 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f8193e0 0x56018f8283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 4 client.4794 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.238+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 --> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56018f744210 con 0x56018f8193e0 2024-04-26T23:37:52.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.239+0000 7f9441c00640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f8193e0 0x56018f8283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.240+0000 7f9441c00640 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f8193e0 0x56018f8283c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ed01d73e743759 server_cookie=c8b2a3ae6d68fa4c in_seq=0 out_seq=0 2024-04-26T23:37:52.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.240+0000 7f9422a00640 10 client.4794 ms_handle_connect on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f9422a00640 1 -- 172.21.15.46:0/2507344223 <== mds.0 v2:172.21.15.142:6834/2711334262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56018f744210 con 0x56018f8193e0 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f9422a00640 10 client.4794 handle_command_reply: tid=0 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 client.4794 shutdown 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 msgr2=0x56018f81acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94280099d0 tx=0x7f94280046c0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 shutdown_connections 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f8193e0 0x56018f8283c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f7fd0a0 0x56018f81acc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f9422a00640 0 client.4794 ms_handle_reset on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 --2- 172.21.15.46:0/2507344223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 >> 172.21.15.46:0/2507344223 conn(0x56018f766d10 msgr2=0x56018f7faff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 shutdown_connections 2024-04-26T23:37:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 1 -- 172.21.15.46:0/2507344223 wait complete. 2024-04-26T23:37:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.242+0000 7f94475d0740 20 client.4794 trim_cache size 0 max 16384 2024-04-26T23:37:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.262+0000 7f94475d0740 1 Processor -- start 2024-04-26T23:37:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.262+0000 7f94475d0740 1 -- start start 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f82b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f943c2023f0 con 0x56018f82bd50 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f943c2018a0 con 0x56018f7d25e0 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.263+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f943c201dd0 con 0x56018f80ecb0 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55742/0 (socket says 172.21.15.46:55742) 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 -- 172.21.15.46:0/2018931646 learned_addr learned my addr 172.21.15.46:0/2018931646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9441200640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f82b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9441c00640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 -- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 msgr2=0x56018f82b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f82b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 -- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 msgr2=0x56018f818760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 -- 172.21.15.46:0/2018931646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018f7f7f60 con 0x56018f82bd50 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9441200640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f82b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.264+0000 7f9442600640 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94340750f0 tx=0x7f943402ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a74eb47fa5d4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f942d600640 1 -- 172.21.15.46:0/2018931646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434078c90 con 0x56018f82bd50 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f942d600640 1 -- 172.21.15.46:0/2018931646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943407d040 con 0x56018f82bd50 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f942d600640 1 -- 172.21.15.46:0/2018931646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434081900 con 0x56018f82bd50 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 -- 172.21.15.46:0/2018931646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 msgr2=0x56018f81aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94340750f0 tx=0x7f943402ace0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 -- 172.21.15.46:0/2018931646 shutdown_connections 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f81aad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f82b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 --2- 172.21.15.46:0/2018931646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f818760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.265+0000 7f94475d0740 1 -- 172.21.15.46:0/2018931646 >> 172.21.15.46:0/2018931646 conn(0x56018f766d10 msgr2=0x56018f7f9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.266+0000 7f94475d0740 1 -- 172.21.15.46:0/2018931646 shutdown_connections 2024-04-26T23:37:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.266+0000 7f94475d0740 1 -- 172.21.15.46:0/2018931646 wait complete. 2024-04-26T23:37:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.266+0000 7f94475d0740 1 -- start start 2024-04-26T23:37:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.266+0000 7f94475d0740 1 Processor -- start 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.267+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f878ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.267+0000 7f94475d0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f94475d0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f94475d0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56018f819260 con 0x56018f82bd50 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56018f82c4a0 con 0x56018f7d25e0 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f94475d0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56018f819820 con 0x56018f80ecb0 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f94475d0740 10 client.0 _ll_register_callbacks cb 0x7f9428009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55752/0 (socket says 172.21.15.46:55752) 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 -- 172.21.15.46:0/2072583454 learned_addr learned my addr 172.21.15.46:0/2072583454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9441200640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9441c00640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f878ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 -- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 msgr2=0x56018f7f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 -- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 msgr2=0x56018f878ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f878ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9442600640 1 -- 172.21.15.46:0/2072583454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018f8932d0 con 0x56018f82bd50 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.268+0000 7f9441c00640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f878ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9442600640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9434078990 tx=0x7f94340ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e571553b518422f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434078c90 con 0x56018f82bd50 2024-04-26T23:37:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94340a9050 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943407d040 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9434076d30 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f94475d0740 10 client.4800 fetch_fsmap learned FSMap version 9 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56018f7fbc10 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943407d040 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.269+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9434075300 con 0x56018f82bd50 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 10 client.4800 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 20 client.4800 populate_metadata read hostname 'smithi046' 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f896c30 0x56018f897010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 4 client.4800 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 --> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] -- command(tid 0: {"prefix": "status"}) -- 0x56018f73e080 con 0x56018f896c30 2024-04-26T23:37:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.270+0000 7f9441c00640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f896c30 0x56018f897010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.271+0000 7f9441c00640 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f896c30 0x56018f897010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd38fbbf5b4e9324 server_cookie=525fd656da63d53e in_seq=0 out_seq=0 2024-04-26T23:37:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.271+0000 7f9422a00640 10 client.4800 ms_handle_connect on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f9422a00640 1 -- 172.21.15.46:0/2072583454 <== mds.0 v2:172.21.15.142:6834/2711334262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56018f73e080 con 0x56018f896c30 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f9422a00640 10 client.4800 handle_command_reply: tid=0 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 client.4800 shutdown 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 msgr2=0x56018f818000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9434078990 tx=0x7f94340ab040 comp rx=0 tx=0).stop 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 shutdown_connections 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:6834/2711334262,v1:172.21.15.142:6836/2711334262] conn(0x56018f896c30 0x56018f897010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56018f82bd50 0x56018f818000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56018f80ecb0 0x56018f7f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 --2- 172.21.15.46:0/2072583454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56018f7d25e0 0x56018f878ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f9422a00640 0 client.4800 ms_handle_reset on v2:172.21.15.142:6834/2711334262 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.272+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 >> 172.21.15.46:0/2072583454 conn(0x56018f766d10 msgr2=0x56018f800410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 shutdown_connections 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f94475d0740 1 -- 172.21.15.46:0/2072583454 wait complete. 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f94475d0740 20 client.4800 trim_cache size 0 max 16384 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:37:52.280 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4285, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 43.511585310000001, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 9, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 25, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 25, 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 51.417064676000003 2024-04-26T23:37:52.281 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:37:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 msgr2=0x7f941c06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 0x7f941c06ef80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9430000f80 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-04-26T23:37:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 msgr2=0x7f943c13c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.273+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f943400c200 tx=0x7f943c161b10 comp rx=0 tx=0).stop 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 shutdown_connections 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f941c06cac0 0x7f941c06ef80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c13bd60 0x7f943c13c1e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f943c1452c0 0x7f943c142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 --2- 172.21.15.46:0/2349477361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94240051c0 0x7f943c15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 >> 172.21.15.46:0/2349477361 conn(0x7f943c06c530 msgr2=0x7f943c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 shutdown_connections 2024-04-26T23:37:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:52.274+0000 7f9444600640 1 -- 172.21.15.46:0/2349477361 wait complete. 2024-04-26T23:37:52.298 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T23:37:52.309 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T23:37:52.309 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T23:37:52.309 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T23:37:52.309 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T23:37:52.337 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T23:37:52.337 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T23:37:52.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.338 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:37:52.397 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T23:37:52.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:37:52.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-26T23:37:52.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.605 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:37:52.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-26T23:37:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-04-26T23:37:52.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:37:52.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.786 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:37:52.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-04-26T23:37:52.930 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-04-26T23:37:52.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:37:52.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.935 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:37:52.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:52.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-04-26T23:37:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-04-26T23:37:53.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:37:53.076 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T23:37:53.076 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:37:53.076 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:37:53.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:37:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.285+0000 7ff5e7e00640 1 Processor -- start 2024-04-26T23:37:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.285+0000 7ff5e7e00640 1 -- start start 2024-04-26T23:37:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff5e8073db0 0x7ff5e8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e806f1d0 con 0x7ff5e80cf450 2024-04-26T23:37:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e80741c0 con 0x7ff5e80cff40 2024-04-26T23:37:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e80726d0 con 0x7ff5e8073db0 2024-04-26T23:37:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff5e80cf450 0x7ff5e80cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60172/0 (socket says 172.21.15.46:60172) 2024-04-26T23:37:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e6a00640 1 -- 172.21.15.46:0/3818705454 learned_addr learned my addr 172.21.15.46:0/3818705454 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.287+0000 7ff5e7400640 1 --2- 172.21.15.46:0/3818705454 >> v2:172.21.15.142:3300/0 conn(0x7ff5e8073db0 0x7ff5e8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4028933629 0 0) 0x7ff5e806f1d0 con 0x7ff5e80cf450 2024-04-26T23:37:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5cc003520 con 0x7ff5e80cf450 2024-04-26T23:37:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307918350 0 0) 0x7ff5cc003520 con 0x7ff5e80cf450 2024-04-26T23:37:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 >> v2:172.21.15.142:3300/0 conn(0x7ff5e8073db0 msgr2=0x7ff5e8072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 --2- 172.21.15.46:0/3818705454 >> v2:172.21.15.142:3300/0 conn(0x7ff5e8073db0 0x7ff5e8072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 >> v1:172.21.15.142:6789/0 conn(0x7ff5e80cff40 legacy=0x7ff5e8164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e806f470 con 0x7ff5e80cf450 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff5d8001670 con 0x7ff5e80cf450 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 >> v1:172.21.15.46:6789/0 conn(0x7ff5e80cf450 legacy=0x7ff5e80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80726d0 con 0x7ff5cc004800 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc00c160 con 0x7ff5cc0051c0 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.288+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff5cc00c360 con 0x7ff5cc0039b0 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6a00640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e7400640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 -- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc0039b0 msgr2=0x7ff5cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 -- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5cc0051c0 msgr2=0x7ff5cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e6000640 1 -- 172.21.15.46:0/3818705454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc003520 con 0x7ff5cc004800 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e7400640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.289+0000 7ff5e7400640 1 -- 172.21.15.46:0/3818705454 reap_dead start 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.290+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4001720 tx=0x7ff5cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb233c570046c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4001370 con 0x7ff5cc004800 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d40052d0 con 0x7ff5cc004800 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e5600640 1 -- 172.21.15.46:0/3818705454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d402ab90 con 0x7ff5cc004800 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 msgr2=0x7ff5cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4001720 tx=0x7ff5cc005830 comp rx=0 tx=0).stop 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3818705454 shutdown_connections 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3818705454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.291+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3818705454 >> 172.21.15.46:0/3818705454 conn(0x7ff5e80b2820 msgr2=0x7ff5e80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.292+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3818705454 shutdown_connections 2024-04-26T23:37:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.292+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3818705454 wait complete. 2024-04-26T23:37:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.293+0000 7ff5e7e00640 1 Processor -- start 2024-04-26T23:37:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.294+0000 7ff5e7e00640 1 -- start start 2024-04-26T23:37:53.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc004800 0x7ff5e80b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e81f8e00 0x7ff5e81f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8072ef0 con 0x7ff5e81db710 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8072c10 con 0x7ff5e81f8e00 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e80731d0 con 0x7ff5cc004800 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55762/0 (socket says 172.21.15.46:55762) 2024-04-26T23:37:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 -- 172.21.15.46:0/3968368284 learned_addr learned my addr 172.21.15.46:0/3968368284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6a00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc004800 0x7ff5e80b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e7400640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e81f8e00 0x7ff5e81f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 -- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc004800 msgr2=0x7ff5e80b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc004800 0x7ff5e80b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 -- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e81f8e00 msgr2=0x7ff5e81f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e81f8e00 0x7ff5e81f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.295+0000 7ff5e6000640 1 -- 172.21.15.46:0/3968368284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e80b5570 con 0x7ff5e81db710 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5e6000640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4040020 con 0x7ff5e81db710 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4005480 con 0x7ff5e81db710 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4001370 con 0x7ff5e81db710 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e80b4cd0 con 0x7ff5e81db710 2024-04-26T23:37:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.296+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e80b5290 con 0x7ff5e81db710 2024-04-26T23:37:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.297+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e80693b0 con 0x7ff5e81db710 2024-04-26T23:37:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.298+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5d4043020 con 0x7ff5e81db710 2024-04-26T23:37:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.299+0000 7ff5dea00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 0x7ff5c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.299+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff5d40abe80 con 0x7ff5e81db710 2024-04-26T23:37:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.300+0000 7ff5e6a00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 0x7ff5c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.301+0000 7ff5e6a00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 0x7ff5c806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8001b80 tx=0x7ff5d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.304+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5d404a030 con 0x7ff5e81db710 2024-04-26T23:37:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.548+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff5e80734a0 con 0x7ff5e81db710 2024-04-26T23:37:53.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.549+0000 7ff5dea00640 1 -- 172.21.15.46:0/3968368284 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff5d402be20 con 0x7ff5e81db710 2024-04-26T23:37:53.553 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 msgr2=0x7ff5c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 0x7ff5c806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8001b80 tx=0x7ff5d8009040 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 msgr2=0x7ff5e81f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4001120 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 shutdown_connections 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5c806c920 0x7ff5c806ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e81f8e00 0x7ff5e81f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e81db710 0x7ff5e81f88c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/3968368284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5cc004800 0x7ff5e80b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 >> 172.21.15.46:0/3968368284 conn(0x7ff5e80b2820 msgr2=0x7ff5e80ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 shutdown_connections 2024-04-26T23:37:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53.552+0000 7ff5e7e00640 1 -- 172.21.15.46:0/3968368284 wait complete. 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:37:53.576 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:37:53.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:53.577 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42547sec preferred_lft 42547sec 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:37:53.630 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:37:53.630 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:53.630 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:53.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:37:53.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-26T23:37:53.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-26T23:37:53.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:37:53.631 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:53.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:53.889 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:37:53.957 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:37:53.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:53.962 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:37:54.014 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:37:54.014 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:37:54.014 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:37:54.014 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:37:54.014 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:37:54.015 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:54.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:54.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:54.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:54.232 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:37:54.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:54.279 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-26T23:37:54.325 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:54.326 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:54.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:37:54.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T23:37:54.326 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:54.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T23:37:54.561 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:54.561 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:54.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T23:37:54.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:37:54.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T23:37:54.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T23:37:54.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T23:37:54.562 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:54.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:54.845 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:54.846 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:54.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:37:54.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:37:54.846 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:55.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:55.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:55.060 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:37:55.109 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:37:55.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:55.110 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:37:55.163 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:37:55.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:55.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:37:55.286 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:37:55.286 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:37:55.286 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:55.978 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:37:55.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:37:55.978 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:55.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:37:55.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:37:56.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:37:56.016 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:37:56.016 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:37:56.019 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:37:56.019 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:37:56.019 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:37:56.019 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:37:56.019 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:37:56.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:37:56.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:56.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:37:56.105 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:56.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:56.226 INFO:teuthology.orchestra.run.smithi046.stdout:client4699 2024-04-26T23:37:56.226 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:37:56.226 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:37:56.279 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:56.406 INFO:teuthology.orchestra.run.smithi046.stdout:client4699 2024-04-26T23:37:56.406 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:56.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:56.532 INFO:teuthology.orchestra.run.smithi046.stdout:client4699 2024-04-26T23:37:56.532 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/554dfe58-9c53-4ba0-a961-d2b323744aa7.client4699/status') 2024-04-26T23:37:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:56.677 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.4699 (3)172.21.15.46:0/1321849448 2024-04-26T23:37:56.677 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-04-26T23:37:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-04-26T23:37:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-04-26T23:37:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:70 bytes copied, 0.000105282 s, 665 kB/s 2024-04-26T23:37:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:56.691 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-26T23:37:56.691 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:37:56.691 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T23:37:56.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:37:56.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.900+0000 7f789d600640 1 Processor -- start 2024-04-26T23:37:56.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.900+0000 7f789d600640 1 -- start start 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.901+0000 7f789d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f789806e510 0x7f789806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.901+0000 7f789d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7898160bd0 0x7f7898161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.901+0000 7f789d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789806ee30 con 0x7f789815d540 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.901+0000 7f789d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f78980742d0 con 0x7f7898160bd0 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.901+0000 7f789d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7898161940 con 0x7f789806e510 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7898160bd0 0x7f7898161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7898160bd0 0x7f7898161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55766/0 (socket says 172.21.15.46:55766) 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 -- 172.21.15.46:0/4201263290 learned_addr learned my addr 172.21.15.46:0/4201263290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 -- 172.21.15.46:0/4201263290 >> v2:172.21.15.142:3300/0 conn(0x7f789806e510 msgr2=0x7f789806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 --2- 172.21.15.46:0/4201263290 >> v2:172.21.15.142:3300/0 conn(0x7f789806e510 0x7f789806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 -- 172.21.15.46:0/4201263290 >> v1:172.21.15.142:6789/0 conn(0x7f789815d540 legacy=0x7f789815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:56.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.902+0000 7f7896000640 1 -- 172.21.15.46:0/4201263290 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7898071f80 con 0x7f7898160bd0 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.903+0000 7f7896000640 1 --2- 172.21.15.46:0/4201263290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7898160bd0 0x7f7898161060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7880009e50 tx=0x7f788002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2455a5d854cbb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.904+0000 7f7895600640 1 -- 172.21.15.46:0/4201263290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880036070 con 0x7f7898160bd0 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.904+0000 7f7895600640 1 -- 172.21.15.46:0/4201263290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7880031600 con 0x7f7898160bd0 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.904+0000 7f7895600640 1 -- 172.21.15.46:0/4201263290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78800319d0 con 0x7f7898160bd0 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.904+0000 7f789d600640 1 -- 172.21.15.46:0/4201263290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7898160bd0 msgr2=0x7f7898161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.904+0000 7f789d600640 1 --2- 172.21.15.46:0/4201263290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7898160bd0 0x7f7898161060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7880009e50 tx=0x7f788002a710 comp rx=0 tx=0).stop 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 -- 172.21.15.46:0/4201263290 shutdown_connections 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 --2- 172.21.15.46:0/4201263290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7898160bd0 0x7f7898161060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 --2- 172.21.15.46:0/4201263290 >> v2:172.21.15.142:3300/0 conn(0x7f789806e510 0x7f789806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 -- 172.21.15.46:0/4201263290 >> 172.21.15.46:0/4201263290 conn(0x7f789806c530 msgr2=0x7f789806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 -- 172.21.15.46:0/4201263290 shutdown_connections 2024-04-26T23:37:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.905+0000 7f789d600640 1 -- 172.21.15.46:0/4201263290 wait complete. 2024-04-26T23:37:56.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.906+0000 7f789d600640 1 Processor -- start 2024-04-26T23:37:56.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.907+0000 7f789d600640 1 -- start start 2024-04-26T23:37:56.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f789815d540 0x7f789815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 0x7f7898142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7898072820 con 0x7f789806e510 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7898072540 con 0x7f7898160bd0 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f789d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78981615a0 con 0x7f789815d540 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55782/0 (socket says 172.21.15.46:55782) 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 -- 172.21.15.46:0/3960376531 learned_addr learned my addr 172.21.15.46:0/3960376531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7897400640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 0x7f7898142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896000640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f789815d540 0x7f789815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 -- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f789815d540 msgr2=0x7f789815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f789815d540 0x7f789815b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 -- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 msgr2=0x7f7898142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 0x7f7898142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7896a00640 1 -- 172.21.15.46:0/3960376531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7898145ed0 con 0x7f789806e510 2024-04-26T23:37:56.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.908+0000 7f7897400640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 0x7f7898142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f7896a00640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f789000b4d0 tx=0x7f789000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890014070 con 0x7f789806e510 2024-04-26T23:37:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7890010870 con 0x7f789806e510 2024-04-26T23:37:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7898145050 con 0x7f789806e510 2024-04-26T23:37:56.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890010c60 con 0x7f789806e510 2024-04-26T23:37:56.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.909+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789813f860 con 0x7f789806e510 2024-04-26T23:37:56.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.911+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7898069530 con 0x7f789806e510 2024-04-26T23:37:56.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.912+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7890017070 con 0x7f789806e510 2024-04-26T23:37:56.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.913+0000 7f788ea00640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 0x7f787406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:56.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.913+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f789008bd40 con 0x7f789806e510 2024-04-26T23:37:56.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.913+0000 7f7896000640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 0x7f787406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.914+0000 7f7896000640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 0x7f787406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7880033000 tx=0x7f7880009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:56.918+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7890059cd0 con 0x7f789806e510 2024-04-26T23:37:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.164+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7898069e80 con 0x7f789806e510 2024-04-26T23:37:57.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.165+0000 7f788ea00640 1 -- 172.21.15.46:0/3960376531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f78900595d0 con 0x7f789806e510 2024-04-26T23:37:57.169 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:37:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.167+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 msgr2=0x7f787406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.167+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 0x7f787406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7880033000 tx=0x7f7880009000 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.167+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 msgr2=0x7f7898144b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.167+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f789000b4d0 tx=0x7f789000b990 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 shutdown_connections 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f787406c920 0x7f787406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7898160bd0 0x7f7898142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f789815d540 0x7f789815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 --2- 172.21.15.46:0/3960376531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789806e510 0x7f7898144b10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 >> 172.21.15.46:0/3960376531 conn(0x7f789806c530 msgr2=0x7f78980b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 shutdown_connections 2024-04-26T23:37:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57.168+0000 7f789d600640 1 -- 172.21.15.46:0/3960376531 wait complete. 2024-04-26T23:37:57.190 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:37:57.191 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T23:37:57.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.191 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:37:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:37:57.245 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:57.246 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:37:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:57.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.249 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:37:57.330 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:37:57.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.334 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:37:57.391 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:37:57.391 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:37:57.391 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:37:57.391 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:37:57.391 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:37:57.392 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:57.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:57.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.620 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:37:57.671 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:37:57.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.672 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-26T23:37:57.724 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T23:37:57.726 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:57.726 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:57.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:37:57.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T23:37:57.726 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:57.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:57.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:57.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T23:37:58.008 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:37:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether f6:c2:a3:53:c8:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::f4c2:a3ff:fe53:c868/64 scope link 2024-04-26T23:37:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:58.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.101 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:37:58.102 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether f6:c2:a3:53:c8:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:37:58.102 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:37:58.102 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.102 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::f4c2:a3ff:fe53:c868/64 scope link 2024-04-26T23:37:58.102 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:37:58.106 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T23:37:58.107 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:58.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:58.388 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:37:58.388 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:37:58.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2024-04-26T23:37:58.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T23:37:58.388 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:37:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:58.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:58.607 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T23:37:58.655 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T23:37:58.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:37:58.655 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T23:37:58.707 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:37:58.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:58.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:37:58.842 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:37:58.842 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:37:58.842 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:58.866 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:37:58.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:37:58.867 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:58.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:37:58.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:37:58.897 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:37:58.901 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T23:37:58.901 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:37:58.901 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:37:58.901 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:37:58.901 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:37:58.905 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:37:58.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:37:58.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T23:37:58.988 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:59.116 INFO:teuthology.orchestra.run.smithi046.stdout:client4702 2024-04-26T23:37:59.116 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:37:59.116 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:37:59.168 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:59.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:59.343 INFO:teuthology.orchestra.run.smithi046.stdout:client4702 2024-04-26T23:37:59.343 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:37:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:37:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:59.471 INFO:teuthology.orchestra.run.smithi046.stdout:client4702 2024-04-26T23:37:59.471 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/554dfe58-9c53-4ba0-a961-d2b323744aa7.client4702/status') 2024-04-26T23:37:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:37:59.610 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.4702 (3)172.21.15.46:0/3500515784 2024-04-26T23:37:59.610 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-04-26T23:37:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-04-26T23:37:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-04-26T23:37:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:70 bytes copied, 9.8545e-05 s, 710 kB/s 2024-04-26T23:37:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:37:59.624 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T23:37:59.624 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:37:59.624 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T23:37:59.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:37:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.832+0000 7fa721000640 1 Processor -- start 2024-04-26T23:37:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.832+0000 7fa721000640 1 -- start start 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa721000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa71c16a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa721000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa71c0cfe50 con 0x7fa71c0d1990 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa721000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa71c0d0130 con 0x7fa71c0d0c10 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa721000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c0d0780 con 0x7fa71c1639c0 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa71b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa71c0d1990 0x7fa71c0d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60188/0 (socket says 172.21.15.46:60188) 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa71b400640 1 -- 172.21.15.46:0/172849767 learned_addr learned my addr 172.21.15.46:0/172849767 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:37:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa71be00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa71c16a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.834+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241717695 0 0) 0x7fa71c0cfe50 con 0x7fa71c0d1990 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa700003520 con 0x7fa71c0d1990 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930308477 0 0) 0x7fa700003520 con 0x7fa71c0d1990 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 msgr2=0x7fa71c16a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa71c16a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 >> v1:172.21.15.142:6789/0 conn(0x7fa71c0d0c10 legacy=0x7fa71c0d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c0d0410 con 0x7fa71c0d1990 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa704001670 con 0x7fa71c0d1990 2024-04-26T23:37:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 >> v1:172.21.15.46:6789/0 conn(0x7fa71c0d1990 legacy=0x7fa71c0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7000039e0 0x7fa700003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa700004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c0d0780 con 0x7fa700004d40 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000ba70 con 0x7fa7000039e0 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa70000bc70 con 0x7fa71c1639c0 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.835+0000 7fa71be00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71aa00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa700004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71b400640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7000039e0 0x7fa700003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71be00640 1 -- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 msgr2=0x7fa700004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71be00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c1639c0 0x7fa700004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71be00640 1 -- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7000039e0 msgr2=0x7fa700003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71be00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7000039e0 0x7fa700003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71be00640 1 -- 172.21.15.46:0/172849767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa700003520 con 0x7fa700004d40 2024-04-26T23:37:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.836+0000 7fa71b400640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7000039e0 0x7fa700003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.837+0000 7fa71be00640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa71c08d600 tx=0x7fa700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1350644c38907914 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.837+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714036070 con 0x7fa700004d40 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.837+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714031400 con 0x7fa700004d40 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.837+0000 7fa71a000640 1 -- 172.21.15.46:0/172849767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714009690 con 0x7fa700004d40 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 -- 172.21.15.46:0/172849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa700004d40 msgr2=0x7fa7000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 --2- 172.21.15.46:0/172849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa71c08d600 tx=0x7fa700005830 comp rx=0 tx=0).stop 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa71be00640 1 -- 172.21.15.46:0/172849767 reap_dead start 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 -- 172.21.15.46:0/172849767 shutdown_connections 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 -- 172.21.15.46:0/172849767 >> 172.21.15.46:0/172849767 conn(0x7fa71c09e2a0 msgr2=0x7fa71c0d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 -- 172.21.15.46:0/172849767 shutdown_connections 2024-04-26T23:37:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.838+0000 7fa721000640 1 -- 172.21.15.46:0/172849767 wait complete. 2024-04-26T23:37:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.840+0000 7fa721000640 1 Processor -- start 2024-04-26T23:37:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.840+0000 7fa721000640 1 -- start start 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 0x7fa71c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c156dd0 0x7fa71c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c0cfa90 con 0x7fa71c1569f0 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c0d5da0 con 0x7fa71c0ca2c0 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa721000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c0cfc10 con 0x7fa71c156dd0 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa71b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa71b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55806/0 (socket says 172.21.15.46:55806) 2024-04-26T23:37:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa71b400640 1 -- 172.21.15.46:0/2329855536 learned_addr learned my addr 172.21.15.46:0/2329855536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa71aa00640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 0x7fa71c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.841+0000 7fa71be00640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c156dd0 0x7fa71c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 -- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c156dd0 msgr2=0x7fa71c0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c156dd0 0x7fa71c0cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 -- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 msgr2=0x7fa71c0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 0x7fa71c0cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 -- 172.21.15.46:0/2329855536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c0c9220 con 0x7fa71c1569f0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71aa00640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 0x7fa71c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa71b400640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa70400dac0 tx=0x7fa70400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704019070 con 0x7fa71c1569f0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa704004cb0 con 0x7fa71c1569f0 2024-04-26T23:37:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704004e60 con 0x7fa71c1569f0 2024-04-26T23:37:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c0c9500 con 0x7fa71c1569f0 2024-04-26T23:37:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.842+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c0c9ac0 con 0x7fa71c1569f0 2024-04-26T23:37:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.843+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c0693b0 con 0x7fa71c1569f0 2024-04-26T23:37:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.844+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7040072f0 con 0x7fa71c1569f0 2024-04-26T23:37:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.845+0000 7fa713400640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 0x7fa6f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:37:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.845+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa70408bbc0 con 0x7fa71c1569f0 2024-04-26T23:37:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.846+0000 7fa71aa00640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 0x7fa6f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:37:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.847+0000 7fa71aa00640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 0x7fa6f806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa70c007fd0 tx=0x7fa70c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:37:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:37:59.850+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa704059b50 con 0x7fa71c1569f0 2024-04-26T23:38:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.100+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa71c0d1770 con 0x7fa71c1569f0 2024-04-26T23:38:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.101+0000 7fa713400640 1 -- 172.21.15.46:0/2329855536 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa704059450 con 0x7fa71c1569f0 2024-04-26T23:38:00.105 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:38:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 msgr2=0x7fa6f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 0x7fa6f806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa70c007fd0 tx=0x7fa70c009040 comp rx=0 tx=0).stop 2024-04-26T23:38:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 msgr2=0x7fa71c0c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa70400dac0 tx=0x7fa70400df80 comp rx=0 tx=0).stop 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 shutdown_connections 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa6f806c8f0 0x7fa6f806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa71c156dd0 0x7fa71c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa71c0ca2c0 0x7fa71c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 --2- 172.21.15.46:0/2329855536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71c1569f0 0x7fa71c0c8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 >> 172.21.15.46:0/2329855536 conn(0x7fa71c09e2a0 msgr2=0x7fa71c15ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 shutdown_connections 2024-04-26T23:38:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:00.103+0000 7fa721000640 1 -- 172.21.15.46:0/2329855536 wait complete. 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:38:00.127 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:38:00.128 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T23:38:00.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:00.128 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-04-26T23:38:00.186 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:38:00.186 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:38:00.186 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:38:00.186 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T23:38:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42582sec preferred_lft 42582sec 2024-04-26T23:38:00.188 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-04-26T23:38:00.188 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:00.188 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:38:00.189 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:00.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:00.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:00.667 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:38:00.728 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-04-26T23:38:00.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:00.734 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-04-26T23:38:00.786 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-04-26T23:38:00.786 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:38:00.786 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:38:00.786 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:38:00.786 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:38:00.787 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:00.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:01.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:01.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:01.500 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:38:01.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:01.548 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-04-26T23:38:01.590 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:01.590 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:01.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:38:01.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T23:38:01.590 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:01.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:01.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:01.800 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T23:38:01.800 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:01.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:02.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:02.076 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:02.076 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:02.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-04-26T23:38:02.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:38:02.076 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:02.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:02.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:02.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:38:02.262 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T23:38:02.308 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T23:38:02.309 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:38:02.309 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T23:38:02.398 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:38:02.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:02.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:38:02.537 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:38:02.538 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:38:02.538 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:03.213 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:38:03.213 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:38:03.213 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:03.213 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:38:03.213 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:38:03.247 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:38:03.248 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:38:03.248 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:38:03.252 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T23:38:03.252 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:38:03.252 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:38:03.252 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:38:03.252 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:38:03.256 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:38:03.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:03.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T23:38:03.333 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:03.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:03.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:client4520 2024-04-26T23:38:03.452 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:03.452 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:03.504 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:03.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:03.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:03.617 INFO:teuthology.orchestra.run.smithi142.stdout:client4520 2024-04-26T23:38:03.617 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:03.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:03.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:03.727 INFO:teuthology.orchestra.run.smithi142.stdout:client4520 2024-04-26T23:38:03.727 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/554dfe58-9c53-4ba0-a961-d2b323744aa7.client4520/status') 2024-04-26T23:38:03.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:03.857 INFO:teuthology.orchestra.run.smithi142.stdout:instance: client.4520 (3)192.168.0.1:0/2604217200 2024-04-26T23:38:03.858 INFO:teuthology.orchestra.run.smithi142.stdout:blocklisted: false 2024-04-26T23:38:03.858 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records in 2024-04-26T23:38:03.858 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records out 2024-04-26T23:38:03.858 INFO:teuthology.orchestra.run.smithi142.stderr:69 bytes copied, 7.6202e-05 s, 905 kB/s 2024-04-26T23:38:03.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:03.871 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-26T23:38:03.871 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:38:03.871 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T23:38:03.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:38:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.081+0000 7f011b200640 1 Processor -- start 2024-04-26T23:38:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.082+0000 7f011b200640 1 -- start start 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f011c072f90 0x7f011c0c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f011c0cc2e0 0x7f011c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 0x7f011c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f011c06f3c0 con 0x7f011c0cc2e0 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f011c0c6b30 con 0x7f011c072f90 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f011b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f011c070d20 con 0x7f011c06f850 2024-04-26T23:38:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f0119400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f011c0cc2e0 0x7f011c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f0119400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f011c0cc2e0 0x7f011c0cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54052/0 (socket says 172.21.15.46:54052) 2024-04-26T23:38:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.083+0000 7f0119400640 1 -- 172.21.15.46:0/2586317819 learned_addr learned my addr 172.21.15.46:0/2586317819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119e00640 1 --2- 172.21.15.46:0/2586317819 >> v2:172.21.15.142:3300/0 conn(0x7f011c072f90 0x7f011c0c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f011a800640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 0x7f011c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119400640 1 -- 172.21.15.46:0/2586317819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 msgr2=0x7f011c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119400640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 0x7f011c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119400640 1 -- 172.21.15.46:0/2586317819 >> v2:172.21.15.142:3300/0 conn(0x7f011c072f90 msgr2=0x7f011c0c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119400640 1 --2- 172.21.15.46:0/2586317819 >> v2:172.21.15.142:3300/0 conn(0x7f011c072f90 0x7f011c0c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f0119400640 1 -- 172.21.15.46:0/2586317819 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c0c6dd0 con 0x7f011c0cc2e0 2024-04-26T23:38:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.084+0000 7f011a800640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 0x7f011c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.086+0000 7f0119400640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c0cc2e0 0x7f011c0cc740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f011c071b40 tx=0x7f010802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5319a4b424f09f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.086+0000 7f0118a00640 1 -- 172.21.15.46:0/2586317819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108037070 con 0x7f011c0cc2e0 2024-04-26T23:38:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.086+0000 7f0118a00640 1 -- 172.21.15.46:0/2586317819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0108032440 con 0x7f011c0cc2e0 2024-04-26T23:38:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.086+0000 7f0118a00640 1 -- 172.21.15.46:0/2586317819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108032810 con 0x7f011c0cc2e0 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 -- 172.21.15.46:0/2586317819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c0cc2e0 msgr2=0x7f011c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c0cc2e0 0x7f011c0cc740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f011c071b40 tx=0x7f010802a140 comp rx=0 tx=0).stop 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 -- 172.21.15.46:0/2586317819 shutdown_connections 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c06f850 0x7f011c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 --2- 172.21.15.46:0/2586317819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c0cc2e0 0x7f011c0cc740 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 --2- 172.21.15.46:0/2586317819 >> v2:172.21.15.142:3300/0 conn(0x7f011c072f90 0x7f011c0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 -- 172.21.15.46:0/2586317819 >> 172.21.15.46:0/2586317819 conn(0x7f011c0d1c90 msgr2=0x7f011c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 -- 172.21.15.46:0/2586317819 shutdown_connections 2024-04-26T23:38:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.087+0000 7f011b200640 1 -- 172.21.15.46:0/2586317819 wait complete. 2024-04-26T23:38:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.089+0000 7f011b200640 1 Processor -- start 2024-04-26T23:38:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.089+0000 7f011b200640 1 -- start start 2024-04-26T23:38:04.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c072f90 0x7f011c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f011c0cc2e0 0x7f011c0b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f011c15f3b0 con 0x7f011c06f850 2024-04-26T23:38:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f011c0c70b0 con 0x7f011c0cc2e0 2024-04-26T23:38:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f011c15f530 con 0x7f011c072f90 2024-04-26T23:38:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f0119e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f0119e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54058/0 (socket says 172.21.15.46:54058) 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f0119e00640 1 -- 172.21.15.46:0/2890032214 learned_addr learned my addr 172.21.15.46:0/2890032214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.090+0000 7f011a800640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f011c0cc2e0 0x7f011c0b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119400640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c072f90 0x7f011c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 -- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c072f90 msgr2=0x7f011c151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c072f90 0x7f011c151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 -- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f011c0cc2e0 msgr2=0x7f011c0b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f011c0cc2e0 0x7f011c0b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 -- 172.21.15.46:0/2890032214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c0b4be0 con 0x7f011c06f850 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0119e00640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d5f0 tx=0x7f010c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0125f0 con 0x7f011c06f850 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c019070 con 0x7f011c06f850 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0129d0 con 0x7f011c06f850 2024-04-26T23:38:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.091+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c0b4ec0 con 0x7f011c06f850 2024-04-26T23:38:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.092+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c0b5480 con 0x7f011c06f850 2024-04-26T23:38:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.093+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c069ac0 con 0x7f011c06f850 2024-04-26T23:38:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.094+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f010c012b70 con 0x7f011c06f850 2024-04-26T23:38:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.095+0000 7f0112000640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 0x7f00fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.095+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f010c08c3f0 con 0x7f011c06f850 2024-04-26T23:38:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.095+0000 7f0119400640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 0x7f00fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.096+0000 7f0119400640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 0x7f00fc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0108034000 tx=0x7f010802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:04.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.099+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f010c05a380 con 0x7f011c06f850 2024-04-26T23:38:04.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.352+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f011c0b2ab0 con 0x7f011c06f850 2024-04-26T23:38:04.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.353+0000 7f0112000640 1 -- 172.21.15.46:0/2890032214 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f010c059c80 con 0x7f011c06f850 2024-04-26T23:38:04.357 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:38:04.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 msgr2=0x7f00fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:04.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 0x7f00fc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0108034000 tx=0x7f010802e000 comp rx=0 tx=0).stop 2024-04-26T23:38:04.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 msgr2=0x7f011c0cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d5f0 tx=0x7f010c00dab0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 shutdown_connections 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00fc06c8f0 0x7f00fc06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f011c0cc2e0 0x7f011c0b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f011c072f90 0x7f011c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 --2- 172.21.15.46:0/2890032214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f011c06f850 0x7f011c0cb990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 >> 172.21.15.46:0/2890032214 conn(0x7f011c0d1c90 msgr2=0x7f011c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 shutdown_connections 2024-04-26T23:38:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:04.356+0000 7f011b200640 1 -- 172.21.15.46:0/2890032214 wait complete. 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:38:04.380 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T23:38:04.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:04.381 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-04-26T23:38:04.439 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:38:04.440 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42577sec preferred_lft 42577sec 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether ae:b4:42:09:93:85 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::d8b2:48ff:fe27:8773/64 scope link 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether ae:b4:42:09:93:85 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::acb4:42ff:fe09:9385/64 scope link 2024-04-26T23:38:04.441 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:04.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:04.443 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:38:04.525 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-04-26T23:38:04.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:04.529 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-04-26T23:38:04.581 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-04-26T23:38:04.582 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:38:04.582 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:38:04.582 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:38:04.582 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:38:04.583 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:04.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:04.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:04.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:04.803 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:38:04.850 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T23:38:04.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:04.851 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-04-26T23:38:04.947 INFO:teuthology.orchestra.run.smithi142.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T23:38:04.948 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:04.948 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:04.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T23:38:04.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T23:38:04.948 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:05.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:05.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:05.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:05.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T23:38:05.212 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 46:74:56:97:50:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::4474:56ff:fe97:50b7/64 scope link 2024-04-26T23:38:05.213 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:05.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:38:05.298 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 46:74:56:97:50:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:38:05.299 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:38:05.299 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.299 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::4474:56ff:fe97:50b7/64 scope link 2024-04-26T23:38:05.299 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:38:05.302 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T23:38:05.303 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:05.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:05.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-04-26T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.1 up 2024-04-26T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-04-26T23:38:05.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:05.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:05.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:38:05.787 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T23:38:05.833 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T23:38:05.834 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:38:05.834 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T23:38:05.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:38:05.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:05.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:38:06.014 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:38:06.014 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:38:06.014 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:06.035 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:38:06.035 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:38:06.035 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:06.035 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:38:06.035 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:38:06.061 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:38:06.062 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T23:38:06.062 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:38:06.062 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:38:06.063 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:38:06.063 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:38:06.066 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:38:06.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:06.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T23:38:06.136 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:06.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:06.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:06.255 INFO:teuthology.orchestra.run.smithi142.stdout:client4714 2024-04-26T23:38:06.256 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:06.256 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:06.302 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:06.449 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:06.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:06.465 INFO:teuthology.orchestra.run.smithi142.stdout:client4714 2024-04-26T23:38:06.466 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:38:06.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:38:06.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:client4714 2024-04-26T23:38:06.582 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/554dfe58-9c53-4ba0-a961-d2b323744aa7.client4714/status') 2024-04-26T23:38:06.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:38:06.710 INFO:teuthology.orchestra.run.smithi142.stdout:instance: client.4714 (3)192.168.0.2:0/571950812 2024-04-26T23:38:06.710 INFO:teuthology.orchestra.run.smithi142.stdout:blocklisted: false 2024-04-26T23:38:06.710 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records in 2024-04-26T23:38:06.711 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records out 2024-04-26T23:38:06.711 INFO:teuthology.orchestra.run.smithi142.stderr:68 bytes copied, 7.6963e-05 s, 884 kB/s 2024-04-26T23:38:06.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-26T23:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:38:06.723 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T23:38:06.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:06.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.958+0000 7fda73e00640 1 Processor -- start 2024-04-26T23:38:06.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.959+0000 7fda73e00640 1 -- start start 2024-04-26T23:38:06.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.960+0000 7fda73e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda741466b0 0x7fda7406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.960+0000 7fda73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda740720e0 0x7fda740cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.960+0000 7fda73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda74160910 0x7fda74160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fda74073ad0 con 0x7fda74160910 2024-04-26T23:38:06.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fda74146a90 con 0x7fda741466b0 2024-04-26T23:38:06.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda74073d70 con 0x7fda740720e0 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda74160910 0x7fda74160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda74160910 0x7fda74160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54070/0 (socket says 172.21.15.46:54070) 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.961+0000 7fda73400640 1 -- 172.21.15.46:0/4009823597 learned_addr learned my addr 172.21.15.46:0/4009823597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda72a00640 1 --2- 172.21.15.46:0/4009823597 >> v2:172.21.15.142:3300/0 conn(0x7fda741466b0 0x7fda7406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda6be00640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda740720e0 0x7fda740cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda73400640 1 -- 172.21.15.46:0/4009823597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda740720e0 msgr2=0x7fda740cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda73400640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda740720e0 0x7fda740cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda73400640 1 -- 172.21.15.46:0/4009823597 >> v2:172.21.15.142:3300/0 conn(0x7fda741466b0 msgr2=0x7fda7406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:06.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda73400640 1 --2- 172.21.15.46:0/4009823597 >> v2:172.21.15.142:3300/0 conn(0x7fda741466b0 0x7fda7406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.962+0000 7fda73400640 1 -- 172.21.15.46:0/4009823597 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda74074050 con 0x7fda74160910 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.963+0000 7fda73400640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda74160da0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda740b3440 tx=0x7fda6c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b245e3498005e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.963+0000 7fda72000640 1 -- 172.21.15.46:0/4009823597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c036070 con 0x7fda74160910 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.963+0000 7fda72000640 1 -- 172.21.15.46:0/4009823597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c031600 con 0x7fda74160910 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.963+0000 7fda72000640 1 -- 172.21.15.46:0/4009823597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0319d0 con 0x7fda74160910 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 -- 172.21.15.46:0/4009823597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 msgr2=0x7fda74160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda74160da0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda740b3440 tx=0x7fda6c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 -- 172.21.15.46:0/4009823597 shutdown_connections 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda74160da0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 --2- 172.21.15.46:0/4009823597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda740720e0 0x7fda740cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 --2- 172.21.15.46:0/4009823597 >> v2:172.21.15.142:3300/0 conn(0x7fda741466b0 0x7fda7406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 -- 172.21.15.46:0/4009823597 >> 172.21.15.46:0/4009823597 conn(0x7fda740b26d0 msgr2=0x7fda740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 -- 172.21.15.46:0/4009823597 shutdown_connections 2024-04-26T23:38:06.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.964+0000 7fda73e00640 1 -- 172.21.15.46:0/4009823597 wait complete. 2024-04-26T23:38:06.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.966+0000 7fda73e00640 1 Processor -- start 2024-04-26T23:38:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.966+0000 7fda73e00640 1 -- start start 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda740720e0 0x7fda740af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 0x7fda74157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda7406e3c0 con 0x7fda74160910 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda740cb470 con 0x7fda740720e0 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda740cdda0 con 0x7fda741466b0 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54074/0 (socket says 172.21.15.46:54074) 2024-04-26T23:38:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.967+0000 7fda73400640 1 -- 172.21.15.46:0/1909474180 learned_addr learned my addr 172.21.15.46:0/1909474180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda6be00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 0x7fda74157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda72a00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda740720e0 0x7fda740af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 -- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 msgr2=0x7fda74157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 0x7fda74157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 -- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda740720e0 msgr2=0x7fda740af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda740720e0 0x7fda740af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 -- 172.21.15.46:0/1909474180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda740aea80 con 0x7fda74160910 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda6be00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 0x7fda74157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73400640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009f40 tx=0x7fda6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c041020 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c0077a0 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c036040 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.968+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda740aed60 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.969+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda741579a0 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.970+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda740693b0 con 0x7fda74160910 2024-04-26T23:38:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.971+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda6c044020 con 0x7fda74160910 2024-04-26T23:38:06.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.972+0000 7fda6aa00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 0x7fda5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:06.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.972+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fda6c0acc30 con 0x7fda74160910 2024-04-26T23:38:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.972+0000 7fda72a00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 0x7fda5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:06.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.974+0000 7fda72a00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 0x7fda5006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:06.976+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda6c04b030 con 0x7fda74160910 2024-04-26T23:38:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.180+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda74069ac0 con 0x7fda74160910 2024-04-26T23:38:07.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.184+0000 7fda6aa00640 1 -- 172.21.15.46:0/1909474180 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7fda6c07ac60 con 0x7fda74160910 2024-04-26T23:38:07.188 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:07.188 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 msgr2=0x7fda5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 0x7fda5006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 msgr2=0x7fda740c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009f40 tx=0x7fda6c004a40 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 shutdown_connections 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5006c8f0 0x7fda5006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda74160910 0x7fda740c5940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda741466b0 0x7fda74157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.186+0000 7fda73e00640 1 --2- 172.21.15.46:0/1909474180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda740720e0 0x7fda740af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.187+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 >> 172.21.15.46:0/1909474180 conn(0x7fda740b26d0 msgr2=0x7fda7406f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.187+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 shutdown_connections 2024-04-26T23:38:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.187+0000 7fda73e00640 1 -- 172.21.15.46:0/1909474180 wait complete. 2024-04-26T23:38:07.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:07.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.399+0000 7fc40e200640 1 Processor -- start 2024-04-26T23:38:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.399+0000 7fc40e200640 1 -- start start 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc408072f50 0x7fc40806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc40806c890 0x7fc408160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40815d6c0 con 0x7fc40815aac0 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc40815e2d0 con 0x7fc40806c890 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc40815aea0 con 0x7fc408072f50 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc40806c890 0x7fc408160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc40806c890 0x7fc408160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54090/0 (socket says 172.21.15.46:54090) 2024-04-26T23:38:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40d800640 1 -- 172.21.15.46:0/2397680426 learned_addr learned my addr 172.21.15.46:0/2397680426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40d800640 1 -- 172.21.15.46:0/2397680426 >> v2:172.21.15.142:3300/0 conn(0x7fc408072f50 msgr2=0x7fc40806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.401+0000 7fc40d800640 1 --2- 172.21.15.46:0/2397680426 >> v2:172.21.15.142:3300/0 conn(0x7fc408072f50 0x7fc40806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.402+0000 7fc40d800640 1 -- 172.21.15.46:0/2397680426 >> v1:172.21.15.142:6789/0 conn(0x7fc40815aac0 legacy=0x7fc408071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.402+0000 7fc40d800640 1 -- 172.21.15.46:0/2397680426 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40815d9a0 con 0x7fc40806c890 2024-04-26T23:38:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.402+0000 7fc40d800640 1 --2- 172.21.15.46:0/2397680426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc40806c890 0x7fc408160c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4080d2950 tx=0x7fc40002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73392e379163d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.403+0000 7fc407400640 1 -- 172.21.15.46:0/2397680426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400036070 con 0x7fc40806c890 2024-04-26T23:38:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.403+0000 7fc407400640 1 -- 172.21.15.46:0/2397680426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc400031600 con 0x7fc40806c890 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.403+0000 7fc407400640 1 -- 172.21.15.46:0/2397680426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000319d0 con 0x7fc40806c890 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.403+0000 7fc40e200640 1 -- 172.21.15.46:0/2397680426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc40806c890 msgr2=0x7fc408160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.403+0000 7fc40e200640 1 --2- 172.21.15.46:0/2397680426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc40806c890 0x7fc408160c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4080d2950 tx=0x7fc40002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 -- 172.21.15.46:0/2397680426 shutdown_connections 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 --2- 172.21.15.46:0/2397680426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc40806c890 0x7fc408160c50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 --2- 172.21.15.46:0/2397680426 >> v2:172.21.15.142:3300/0 conn(0x7fc408072f50 0x7fc40806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 -- 172.21.15.46:0/2397680426 >> 172.21.15.46:0/2397680426 conn(0x7fc4080d1c50 msgr2=0x7fc4080d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 -- 172.21.15.46:0/2397680426 shutdown_connections 2024-04-26T23:38:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.404+0000 7fc40e200640 1 -- 172.21.15.46:0/2397680426 wait complete. 2024-04-26T23:38:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.405+0000 7fc40e200640 1 Processor -- start 2024-04-26T23:38:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.406+0000 7fc40e200640 1 -- start start 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.406+0000 7fc40e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40806c890 0x7fc40814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc40e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc40e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40815aac0 0x7fc408160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc40e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc40815df60 con 0x7fc408072f50 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc40e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc408073390 con 0x7fc40815aac0 2024-04-26T23:38:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc40e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4080ce120 con 0x7fc40806c890 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54106/0 (socket says 172.21.15.46:54106) 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 -- 172.21.15.46:0/1066861248 learned_addr learned my addr 172.21.15.46:0/1066861248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 -- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40806c890 msgr2=0x7fc40814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40806c890 0x7fc40814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 -- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40815aac0 msgr2=0x7fc408160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40815aac0 0x7fc408160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 -- 172.21.15.46:0/1066861248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40814f600 con 0x7fc408072f50 2024-04-26T23:38:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.407+0000 7fc407e00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400b4d0 tx=0x7fc3f400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.408+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4014070 con 0x7fc408072f50 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.408+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f4010870 con 0x7fc408072f50 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.408+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4010c60 con 0x7fc408072f50 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.408+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4080ca300 con 0x7fc408072f50 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.408+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4080ca8c0 con 0x7fc408072f50 2024-04-26T23:38:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.409+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40806f740 con 0x7fc408072f50 2024-04-26T23:38:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.410+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3f4017070 con 0x7fc408072f50 2024-04-26T23:38:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.411+0000 7fc404c00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 0x7fc3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.411+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc3f408bdb0 con 0x7fc408072f50 2024-04-26T23:38:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.412+0000 7fc40ce00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 0x7fc3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.413+0000 7fc40ce00640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 0x7fc3e806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009650 tx=0x7fc3fc006ba0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.415+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc40806f740 con 0x7fc408072f50 2024-04-26T23:38:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.617+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc408069ac0 con 0x7fc408072f50 2024-04-26T23:38:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.618+0000 7fc404c00640 1 -- 172.21.15.46:0/1066861248 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc3f4090070 con 0x7fc408072f50 2024-04-26T23:38:07.622 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 msgr2=0x7fc3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 0x7fc3e806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009650 tx=0x7fc3fc006ba0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 msgr2=0x7fc4080c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400b4d0 tx=0x7fc3f400b990 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 shutdown_connections 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c8f0 0x7fc3e806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.620+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40815aac0 0x7fc408160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.621+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc408072f50 0x7fc4080c9880 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.621+0000 7fc40e200640 1 --2- 172.21.15.46:0/1066861248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40806c890 0x7fc40814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.621+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 >> 172.21.15.46:0/1066861248 conn(0x7fc4080d1c50 msgr2=0x7fc40815fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.621+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 shutdown_connections 2024-04-26T23:38:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.621+0000 7fc40e200640 1 -- 172.21.15.46:0/1066861248 wait complete. 2024-04-26T23:38:07.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.821+0000 7fd749800640 1 Processor -- start 2024-04-26T23:38:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.821+0000 7fd749800640 1 -- start start 2024-04-26T23:38:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd749800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7440cdd00 0x7fd7440735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd749800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd744073b30 0x7fd7440b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd749800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7440ce7d0 con 0x7fd74415cec0 2024-04-26T23:38:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd749800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd7440ceab0 con 0x7fd7440cdd00 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd749800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd7440ced90 con 0x7fd744073b30 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.822+0000 7fd743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7440cdd00 0x7fd7440735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7440cdd00 0x7fd7440735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54116/0 (socket says 172.21.15.46:54116) 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 -- 172.21.15.46:0/1163785439 learned_addr learned my addr 172.21.15.46:0/1163785439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd748e00640 1 --2- 172.21.15.46:0/1163785439 >> v2:172.21.15.142:3300/0 conn(0x7fd744073b30 0x7fd7440b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 -- 172.21.15.46:0/1163785439 >> v2:172.21.15.142:3300/0 conn(0x7fd744073b30 msgr2=0x7fd7440b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 --2- 172.21.15.46:0/1163785439 >> v2:172.21.15.142:3300/0 conn(0x7fd744073b30 0x7fd7440b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 -- 172.21.15.46:0/1163785439 >> v1:172.21.15.142:6789/0 conn(0x7fd74415cec0 legacy=0x7fd7440cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.823+0000 7fd743400640 1 -- 172.21.15.46:0/1163785439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440cf070 con 0x7fd7440cdd00 2024-04-26T23:38:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.824+0000 7fd743400640 1 --2- 172.21.15.46:0/1163785439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7440cdd00 0x7fd7440735f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7440c7c50 tx=0x7fd73002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174f94daf62d71de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.825+0000 7fd742a00640 1 -- 172.21.15.46:0/1163785439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730037070 con 0x7fd7440cdd00 2024-04-26T23:38:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.825+0000 7fd742a00640 1 -- 172.21.15.46:0/1163785439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd730032440 con 0x7fd7440cdd00 2024-04-26T23:38:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.825+0000 7fd742a00640 1 -- 172.21.15.46:0/1163785439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730032810 con 0x7fd7440cdd00 2024-04-26T23:38:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.825+0000 7fd749800640 1 -- 172.21.15.46:0/1163785439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7440cdd00 msgr2=0x7fd7440735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.825+0000 7fd749800640 1 --2- 172.21.15.46:0/1163785439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7440cdd00 0x7fd7440735f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7440c7c50 tx=0x7fd73002a230 comp rx=0 tx=0).stop 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 -- 172.21.15.46:0/1163785439 shutdown_connections 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 --2- 172.21.15.46:0/1163785439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7440cdd00 0x7fd7440735f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 --2- 172.21.15.46:0/1163785439 >> v2:172.21.15.142:3300/0 conn(0x7fd744073b30 0x7fd7440b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 -- 172.21.15.46:0/1163785439 >> 172.21.15.46:0/1163785439 conn(0x7fd7440b27b0 msgr2=0x7fd7440b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 -- 172.21.15.46:0/1163785439 shutdown_connections 2024-04-26T23:38:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.826+0000 7fd749800640 1 -- 172.21.15.46:0/1163785439 wait complete. 2024-04-26T23:38:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.828+0000 7fd749800640 1 Processor -- start 2024-04-26T23:38:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.828+0000 7fd749800640 1 -- start start 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd744073b30 0x7fd74414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7440cdd00 0x7fd7440c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd744073f70 con 0x7fd74415cec0 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd74415d2a0 con 0x7fd7440cdd00 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd749800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440b4080 con 0x7fd744073b30 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd748e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd748e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54128/0 (socket says 172.21.15.46:54128) 2024-04-26T23:38:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd748e00640 1 -- 172.21.15.46:0/3963992542 learned_addr learned my addr 172.21.15.46:0/3963992542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd743e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd744073b30 0x7fd74414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.830+0000 7fd743400640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7440cdd00 0x7fd7440c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 -- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd744073b30 msgr2=0x7fd74414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd744073b30 0x7fd74414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 -- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7440cdd00 msgr2=0x7fd7440c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7440cdd00 0x7fd7440c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 -- 172.21.15.46:0/3963992542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74415d670 con 0x7fd74415cec0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd748e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00cd20 tx=0x7fd73c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c003b90 con 0x7fd74415cec0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c003d30 con 0x7fd74415cec0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74415d950 con 0x7fd74415cec0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.831+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c004240 con 0x7fd74415cec0 2024-04-26T23:38:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.832+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74415df10 con 0x7fd74415cec0 2024-04-26T23:38:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.833+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440693b0 con 0x7fd74415cec0 2024-04-26T23:38:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.834+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73c0043e0 con 0x7fd74415cec0 2024-04-26T23:38:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.835+0000 7fd73be00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.835+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd73c08ca20 con 0x7fd74415cec0 2024-04-26T23:38:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.835+0000 7fd743e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.836+0000 7fd743e00640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fd0 tx=0x7fd734008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:07.839+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd73c05aa40 con 0x7fd74415cec0 2024-04-26T23:38:07.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:07.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.037+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7440b3860 con 0x7fd74415cec0 2024-04-26T23:38:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.038+0000 7fd73be00640 1 -- 172.21.15.46:0/3963992542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd73c05a340 con 0x7fd74415cec0 2024-04-26T23:38:08.042 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.040+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 msgr2=0x7fd72406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.040+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fd0 tx=0x7fd734008040 comp rx=0 tx=0).stop 2024-04-26T23:38:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.040+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 msgr2=0x7fd74415b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.040+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00cd20 tx=0x7fd73c00a660 comp rx=0 tx=0).stop 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 shutdown_connections 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74415cec0 0x7fd74415b1b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7440cdd00 0x7fd7440c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 --2- 172.21.15.46:0/3963992542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd744073b30 0x7fd74414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 >> 172.21.15.46:0/3963992542 conn(0x7fd7440b27b0 msgr2=0x7fd7440b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 shutdown_connections 2024-04-26T23:38:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.041+0000 7fd749800640 1 -- 172.21.15.46:0/3963992542 wait complete. 2024-04-26T23:38:08.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.254+0000 7f30dda00640 1 Processor -- start 2024-04-26T23:38:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.254+0000 7f30dda00640 1 -- start start 2024-04-26T23:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.256+0000 7f30dda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.256+0000 7f30dda00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30d8071f10 0x7f30d8072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.256+0000 7f30dda00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d80728c0 con 0x7f30d81597e0 2024-04-26T23:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.256+0000 7f30dda00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f30d80aeff0 con 0x7f30d8071f10 2024-04-26T23:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.256+0000 7f30dda00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30d80b60d0 con 0x7f30d80b5730 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.257+0000 7f30d7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30d8071f10 0x7f30d8072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.257+0000 7f30d6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.257+0000 7f30d7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30d8071f10 0x7f30d8072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:55036/0 (socket says 172.21.15.46:55036) 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.257+0000 7f30d6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:42068/0 (socket says 172.21.15.46:42068) 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.257+0000 7f30d7400640 1 -- 172.21.15.46:0/1760709076 learned_addr learned my addr 172.21.15.46:0/1760709076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.258+0000 7f30d7400640 1 -- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 msgr2=0x7f30d80b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.258+0000 7f30d7400640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.258+0000 7f30d7400640 1 -- 172.21.15.46:0/1760709076 >> v1:172.21.15.142:6789/0 conn(0x7f30d81597e0 legacy=0x7f30d80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.258+0000 7f30d7400640 1 -- 172.21.15.46:0/1760709076 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d8073120 con 0x7f30d8071f10 2024-04-26T23:38:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.258+0000 7f30d6000640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:38:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.259+0000 7f30d7400640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8072380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757adf6b55da5853 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.259+0000 7f30d5600640 1 -- 172.21.15.46:0/1760709076 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0037070 con 0x7f30d8071f10 2024-04-26T23:38:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.259+0000 7f30d5600640 1 -- 172.21.15.46:0/1760709076 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d002ae50 con 0x7f30d8071f10 2024-04-26T23:38:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.259+0000 7f30d5600640 1 -- 172.21.15.46:0/1760709076 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0032860 con 0x7f30d8071f10 2024-04-26T23:38:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 -- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 msgr2=0x7f30d8072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8072380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d002a300 comp rx=0 tx=0).stop 2024-04-26T23:38:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 -- 172.21.15.46:0/1760709076 shutdown_connections 2024-04-26T23:38:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d80b5730 0x7f30d80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 --2- 172.21.15.46:0/1760709076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8072380 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 -- 172.21.15.46:0/1760709076 >> 172.21.15.46:0/1760709076 conn(0x7f30d806c980 msgr2=0x7f30d80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 -- 172.21.15.46:0/1760709076 shutdown_connections 2024-04-26T23:38:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.260+0000 7f30dda00640 1 -- 172.21.15.46:0/1760709076 wait complete. 2024-04-26T23:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.262+0000 7f30dda00640 1 Processor -- start 2024-04-26T23:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.262+0000 7f30dda00640 1 -- start start 2024-04-26T23:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d81597e0 0x7f30d8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8072e40 con 0x7f30d80b5730 2024-04-26T23:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8072b60 con 0x7f30d8071f10 2024-04-26T23:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30dda00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30d80b40d0 con 0x7f30d81597e0 2024-04-26T23:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.263+0000 7f30d6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54144/0 (socket says 172.21.15.46:54144) 2024-04-26T23:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 -- 172.21.15.46:0/2328354893 learned_addr learned my addr 172.21.15.46:0/2328354893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d7400640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d81597e0 0x7f30d8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 -- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d81597e0 msgr2=0x7f30d8156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d81597e0 0x7f30d8156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 -- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 msgr2=0x7f30d8147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 -- 172.21.15.46:0/2328354893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d81f86b0 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.264+0000 7f30d6000640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009f60 tx=0x7f30c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.265+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0018020 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.265+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c0014680 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.265+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0014a50 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.265+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d81f8990 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.265+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81f8f50 con 0x7f30d80b5730 2024-04-26T23:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.266+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d80cf2b0 con 0x7f30d80b5730 2024-04-26T23:38:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.267+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30c0005d10 con 0x7f30d80b5730 2024-04-26T23:38:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.268+0000 7f30cea00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.268+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f30c008cdf0 con 0x7f30d80b5730 2024-04-26T23:38:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.269+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.270+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 0x7f30b406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f30c8007fd0 tx=0x7f30c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.272+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30d80cf2b0 con 0x7f30d80b5730 2024-04-26T23:38:08.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.469+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30d8158e60 con 0x7f30d80b5730 2024-04-26T23:38:08.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.469+0000 7f30cea00640 1 -- 172.21.15.46:0/2328354893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f30c0022030 con 0x7f30d80b5730 2024-04-26T23:38:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 msgr2=0x7f30b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 0x7f30b406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f30c8007fd0 tx=0x7f30c8009040 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 msgr2=0x7f30d80cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009f60 tx=0x7f30c0009f90 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 shutdown_connections 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d81597e0 0x7f30d8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d80b5730 0x7f30d80cd6b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 --2- 172.21.15.46:0/2328354893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8071f10 0x7f30d8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 >> 172.21.15.46:0/2328354893 conn(0x7f30d806c980 msgr2=0x7f30d806e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 shutdown_connections 2024-04-26T23:38:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.472+0000 7f30dda00640 1 -- 172.21.15.46:0/2328354893 wait complete. 2024-04-26T23:38:08.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.688+0000 7f42e8a00640 1 Processor -- start 2024-04-26T23:38:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.689+0000 7f42e8a00640 1 -- start start 2024-04-26T23:38:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42e8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42e406d3f0 0x7f42e4167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42e8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e40b5890 con 0x7f42e40cfa50 2024-04-26T23:38:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42e8a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e415d1d0 con 0x7f42e415cdf0 2024-04-26T23:38:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42e8a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f42e40cffe0 con 0x7f42e406d3f0 2024-04-26T23:38:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42dbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42e40cfa50 0x7f42e40b5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45060/0 (socket says 172.21.15.46:45060) 2024-04-26T23:38:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.690+0000 7f42dbe00640 1 -- 172.21.15.46:0/2512084267 learned_addr learned my addr 172.21.15.46:0/2512084267 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e3200640 1 --2- 172.21.15.46:0/2512084267 >> v2:172.21.15.46:3300/0 conn(0x7f42e406d3f0 0x7f42e4167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e1e00640 1 -- 172.21.15.46:0/2512084267 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650331124 0 0) 0x7f42e40b5890 con 0x7f42e40cfa50 2024-04-26T23:38:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e1e00640 1 -- 172.21.15.46:0/2512084267 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c8003520 con 0x7f42e40cfa50 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e3200640 1 -- 172.21.15.46:0/2512084267 >> v1:172.21.15.142:6789/0 conn(0x7f42e415cdf0 legacy=0x7f42e40b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e3200640 1 -- 172.21.15.46:0/2512084267 >> v1:172.21.15.46:6789/0 conn(0x7f42e40cfa50 legacy=0x7f42e40b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.691+0000 7f42e3200640 1 -- 172.21.15.46:0/2512084267 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e40d02c0 con 0x7f42e406d3f0 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.692+0000 7f42e3200640 1 --2- 172.21.15.46:0/2512084267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e406d3f0 0x7f42e4167cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009d90 tx=0x7f42dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0312adeb7299572 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.692+0000 7f42e1e00640 1 -- 172.21.15.46:0/2512084267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc007740 con 0x7f42e406d3f0 2024-04-26T23:38:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e1e00640 1 -- 172.21.15.46:0/2512084267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc0078e0 con 0x7f42e406d3f0 2024-04-26T23:38:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e1e00640 1 -- 172.21.15.46:0/2512084267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc007cb0 con 0x7f42e406d3f0 2024-04-26T23:38:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 -- 172.21.15.46:0/2512084267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e406d3f0 msgr2=0x7f42e4167cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 --2- 172.21.15.46:0/2512084267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e406d3f0 0x7f42e4167cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009d90 tx=0x7f42dc02a660 comp rx=0 tx=0).stop 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 -- 172.21.15.46:0/2512084267 shutdown_connections 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 --2- 172.21.15.46:0/2512084267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e406d3f0 0x7f42e4167cf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 -- 172.21.15.46:0/2512084267 >> 172.21.15.46:0/2512084267 conn(0x7f42e40d13f0 msgr2=0x7f42e40d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 -- 172.21.15.46:0/2512084267 shutdown_connections 2024-04-26T23:38:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.693+0000 7f42e8a00640 1 -- 172.21.15.46:0/2512084267 wait complete. 2024-04-26T23:38:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.695+0000 7f42e8a00640 1 Processor -- start 2024-04-26T23:38:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.695+0000 7f42e8a00640 1 -- start start 2024-04-26T23:38:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e406d3f0 0x7f42e414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e415cdf0 0x7f42e4155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40d0b60 con 0x7f42e40cfa50 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40d0880 con 0x7f42e406d3f0 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42e8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42e4068610 con 0x7f42e415cdf0 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54156/0 (socket says 172.21.15.46:54156) 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.696+0000 7f42dbe00640 1 -- 172.21.15.46:0/889008723 learned_addr learned my addr 172.21.15.46:0/889008723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42e2800640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e406d3f0 0x7f42e414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 -- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e415cdf0 msgr2=0x7f42e4155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e415cdf0 0x7f42e4155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 -- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e406d3f0 msgr2=0x7f42e414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e406d3f0 0x7f42e414ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 -- 172.21.15.46:0/889008723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e414bf90 con 0x7f42e40cfa50 2024-04-26T23:38:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42dbe00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00c500 tx=0x7f42cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc017070 con 0x7f42e40cfa50 2024-04-26T23:38:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42cc005ce0 con 0x7f42e40cfa50 2024-04-26T23:38:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc007330 con 0x7f42e40cfa50 2024-04-26T23:38:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.697+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e4156bf0 con 0x7f42e40cfa50 2024-04-26T23:38:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.698+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e4156910 con 0x7f42e40cfa50 2024-04-26T23:38:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.699+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e4069ac0 con 0x7f42e40cfa50 2024-04-26T23:38:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.700+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42cc01a070 con 0x7f42e40cfa50 2024-04-26T23:38:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.700+0000 7f42daa00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 0x7f42c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.700+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f42cc08b3a0 con 0x7f42e40cfa50 2024-04-26T23:38:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.701+0000 7f42e2800640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 0x7f42c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.702+0000 7f42e2800640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 0x7f42c006ee00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42e4159530 tx=0x7f42d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.705+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42cc059500 con 0x7f42e40cfa50 2024-04-26T23:38:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.914+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f42e415c400 con 0x7f42e40cfa50 2024-04-26T23:38:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.915+0000 7f42daa00640 1 -- 172.21.15.46:0/889008723 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f42cc058e00 con 0x7f42e40cfa50 2024-04-26T23:38:08.919 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:08.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 msgr2=0x7f42c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 0x7f42c006ee00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42e4159530 tx=0x7f42d4009040 comp rx=0 tx=0).stop 2024-04-26T23:38:08.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 msgr2=0x7f42e41538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:08.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00c500 tx=0x7f42cc00c9c0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 shutdown_connections 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42c006c940 0x7f42c006ee00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e415cdf0 0x7f42e4155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e40cfa50 0x7f42e41538e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 --2- 172.21.15.46:0/889008723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e406d3f0 0x7f42e414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 >> 172.21.15.46:0/889008723 conn(0x7f42e40d13f0 msgr2=0x7f42e414ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.917+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 shutdown_connections 2024-04-26T23:38:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:08.918+0000 7f42e8a00640 1 -- 172.21.15.46:0/889008723 wait complete. 2024-04-26T23:38:08.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.149+0000 7f9ccec00640 1 Processor -- start 2024-04-26T23:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.149+0000 7f9ccec00640 1 -- start start 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cc80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9cc8160910 0x7f9cc8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc8073ad0 con 0x7f9cc81466b0 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc8146a90 con 0x7f9cc8160910 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc8073d70 con 0x7f9cc80720e0 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccd800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9cc81466b0 0x7f9cc806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45072/0 (socket says 172.21.15.46:45072) 2024-04-26T23:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.151+0000 7f9ccd800640 1 -- 172.21.15.46:0/1910227178 learned_addr learned my addr 172.21.15.46:0/1910227178 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cce200640 1 --2- 172.21.15.46:0/1910227178 >> v2:172.21.15.142:3300/0 conn(0x7f9cc8160910 0x7f9cc8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9ccce00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cc80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311692515 0 0) 0x7f9cc8073ad0 con 0x7f9cc81466b0 2024-04-26T23:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb0003520 con 0x7f9cc81466b0 2024-04-26T23:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4124344974 0 0) 0x7f9cb0003520 con 0x7f9cc81466b0 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 msgr2=0x7f9cc80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cc80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 >> v2:172.21.15.142:3300/0 conn(0x7f9cc8160910 msgr2=0x7f9cc8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 --2- 172.21.15.46:0/1910227178 >> v2:172.21.15.142:3300/0 conn(0x7f9cc8160910 0x7f9cc8160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc8074050 con 0x7f9cc81466b0 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9cbc001670 con 0x7f9cc81466b0 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.152+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 >> v1:172.21.15.46:6789/0 conn(0x7f9cc81466b0 legacy=0x7f9cc806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb00039b0 0x7f9cb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cb0004d40 0x7f9cb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8146a90 con 0x7f9cb00039b0 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8073d70 con 0x7f9cb0004d40 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb000b9c0 con 0x7f9cc80720e0 2024-04-26T23:38:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb00039b0 0x7f9cb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccce00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 -- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 msgr2=0x7f9cb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cc80720e0 0x7f9cb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 -- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cb0004d40 msgr2=0x7f9cb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cb0004d40 0x7f9cb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.153+0000 7f9ccd800640 1 -- 172.21.15.46:0/1910227178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb0003520 con 0x7f9cb00039b0 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb00039b0 0x7f9cb0003dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c210 tx=0x7f9cb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19902a28293f4515 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc007d70 con 0x7f9cb00039b0 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc005af0 con 0x7f9cb00039b0 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9cc7e00640 1 -- 172.21.15.46:0/1910227178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc007070 con 0x7f9cb00039b0 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9ccec00640 1 -- 172.21.15.46:0/1910227178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb00039b0 msgr2=0x7f9cb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1910227178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb00039b0 0x7f9cb0003dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c210 tx=0x7f9cb0005830 comp rx=0 tx=0).stop 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.155+0000 7f9cce200640 1 -- 172.21.15.46:0/1910227178 reap_dead start 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.156+0000 7f9ccec00640 1 -- 172.21.15.46:0/1910227178 shutdown_connections 2024-04-26T23:38:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.156+0000 7f9ccec00640 1 -- 172.21.15.46:0/1910227178 >> 172.21.15.46:0/1910227178 conn(0x7f9cc80b26d0 msgr2=0x7f9cc80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.156+0000 7f9ccec00640 1 -- 172.21.15.46:0/1910227178 shutdown_connections 2024-04-26T23:38:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.156+0000 7f9ccec00640 1 -- 172.21.15.46:0/1910227178 wait complete. 2024-04-26T23:38:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.157+0000 7f9ccec00640 1 Processor -- start 2024-04-26T23:38:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.158+0000 7f9ccec00640 1 -- start start 2024-04-26T23:38:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cb00039b0 0x7f9cc80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc81f5120 0x7f9cc80b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80cd860 con 0x7f9cc80b09e0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80cb470 con 0x7f9cc81f5120 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc80cd9e0 con 0x7f9cb00039b0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54164/0 (socket says 172.21.15.46:54164) 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccce00640 1 -- 172.21.15.46:0/1546026257 learned_addr learned my addr 172.21.15.46:0/1546026257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cb00039b0 0x7f9cc80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9cce200640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc81f5120 0x7f9cc80b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccce00640 1 -- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cb00039b0 msgr2=0x7f9cc80ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.159+0000 7f9ccce00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cb00039b0 0x7f9cc80ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccce00640 1 -- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc81f5120 msgr2=0x7f9cc80b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccce00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc81f5120 0x7f9cc80b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccce00640 1 -- 172.21.15.46:0/1546026257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc80b0fa0 con 0x7f9cc80b09e0 2024-04-26T23:38:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccce00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011540 tx=0x7f9cb8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8015930 con 0x7f9cc80b09e0 2024-04-26T23:38:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb8015ad0 con 0x7f9cc80b09e0 2024-04-26T23:38:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800f460 con 0x7f9cc80b09e0 2024-04-26T23:38:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc80b1280 con 0x7f9cc80b09e0 2024-04-26T23:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.160+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc80aed60 con 0x7f9cc80b09e0 2024-04-26T23:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.161+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc80693b0 con 0x7f9cc80b09e0 2024-04-26T23:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.162+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cb800f600 con 0x7f9cc80b09e0 2024-04-26T23:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.165+0000 7f9cc5600640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 0x7f9cac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.165+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9cb808c8d0 con 0x7f9cc80b09e0 2024-04-26T23:38:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.165+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 0x7f9cac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.167+0000 7f9ccd800640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 0x7f9cac06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc033000 tx=0x7f9cbc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.169+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9cb8019030 con 0x7f9cc80b09e0 2024-04-26T23:38:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.368+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cc80c7180 con 0x7f9cc80b09e0 2024-04-26T23:38:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.371+0000 7f9cc5600640 1 -- 172.21.15.46:0/1546026257 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f9cb805a860 con 0x7f9cc80b09e0 2024-04-26T23:38:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:09.375 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 msgr2=0x7f9cac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 0x7f9cac06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc033000 tx=0x7f9cbc009030 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 msgr2=0x7f9cc8143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011540 tx=0x7f9cb8011a00 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 shutdown_connections 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9cac06c920 0x7f9cac06ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc81f5120 0x7f9cc80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc80b09e0 0x7f9cc8143e80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 --2- 172.21.15.46:0/1546026257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9cb00039b0 0x7f9cc80ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 >> 172.21.15.46:0/1546026257 conn(0x7f9cc80b26d0 msgr2=0x7f9cc806e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 shutdown_connections 2024-04-26T23:38:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.374+0000 7f9ccec00640 1 -- 172.21.15.46:0/1546026257 wait complete. 2024-04-26T23:38:09.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:09.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.607+0000 7eff30c00640 1 Processor -- start 2024-04-26T23:38:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.607+0000 7eff30c00640 1 -- start start 2024-04-26T23:38:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.608+0000 7eff30c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff2c1640d0 0x7eff2c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.608+0000 7eff30c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.608+0000 7eff30c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2c1649f0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.608+0000 7eff30c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c0cb4f0 con 0x7eff2c1640d0 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.608+0000 7eff30c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c071e80 con 0x7eff2c073af0 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.609+0000 7eff2b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff2c1640d0 0x7eff2c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.609+0000 7eff2b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff2c1640d0 0x7eff2c1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:55088/0 (socket says 172.21.15.46:55088) 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.609+0000 7eff2b400640 1 -- 172.21.15.46:0/1740522647 learned_addr learned my addr 172.21.15.46:0/1740522647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.609+0000 7eff2aa00640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.610+0000 7eff2b400640 1 -- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 msgr2=0x7eff2c0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.610+0000 7eff2b400640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.610+0000 7eff2b400640 1 -- 172.21.15.46:0/1740522647 >> v1:172.21.15.142:6789/0 conn(0x7eff2c0cbec0 legacy=0x7eff2c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.610+0000 7eff2b400640 1 -- 172.21.15.46:0/1740522647 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c072160 con 0x7eff2c1640d0 2024-04-26T23:38:09.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.612+0000 7eff2b400640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c1644b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009870 tx=0x7eff1c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c688dca56c7bd47a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.612+0000 7eff2a000640 1 -- 172.21.15.46:0/1740522647 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c037070 con 0x7eff2c1640d0 2024-04-26T23:38:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff2a000640 1 -- 172.21.15.46:0/1740522647 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1c032440 con 0x7eff2c1640d0 2024-04-26T23:38:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff2a000640 1 -- 172.21.15.46:0/1740522647 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c032810 con 0x7eff2c1640d0 2024-04-26T23:38:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 -- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 msgr2=0x7eff2c1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c1644b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009870 tx=0x7eff1c02a270 comp rx=0 tx=0).stop 2024-04-26T23:38:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 -- 172.21.15.46:0/1740522647 shutdown_connections 2024-04-26T23:38:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 --2- 172.21.15.46:0/1740522647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c1644b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 -- 172.21.15.46:0/1740522647 >> 172.21.15.46:0/1740522647 conn(0x7eff2c0b27b0 msgr2=0x7eff2c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:09.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 -- 172.21.15.46:0/1740522647 shutdown_connections 2024-04-26T23:38:09.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.613+0000 7eff30c00640 1 -- 172.21.15.46:0/1740522647 wait complete. 2024-04-26T23:38:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.615+0000 7eff30c00640 1 Processor -- start 2024-04-26T23:38:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.615+0000 7eff30c00640 1 -- start start 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c1ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c0ceee0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c073f70 con 0x7eff2c1640d0 2024-04-26T23:38:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.616+0000 7eff30c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0cf060 con 0x7eff2c073af0 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54178/0 (socket says 172.21.15.46:54178) 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 -- 172.21.15.46:0/1971677463 learned_addr learned my addr 172.21.15.46:0/1971677463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2be00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2b400640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c1ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 -- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 msgr2=0x7eff2c1ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c1ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 -- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 msgr2=0x7eff2c151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 -- 172.21.15.46:0/1971677463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c0c64c0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.617+0000 7eff2aa00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7eff1400efc0 tx=0x7eff1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.618+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14018020 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.618+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14014480 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.618+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14014850 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.618+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c0c7520 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.618+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c0c6a40 con 0x7eff2c0cbec0 2024-04-26T23:38:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.619+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c14f9a0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.620+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff14005d10 con 0x7eff2c0cbec0 2024-04-26T23:38:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.621+0000 7eff23400640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 0x7eff0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.621+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7eff1408cba0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.622+0000 7eff2b400640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 0x7eff0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.623+0000 7eff2b400640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 0x7eff0806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.626+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff2c14f9a0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.839+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff2c0693b0 con 0x7eff2c0cbec0 2024-04-26T23:38:09.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.841+0000 7eff23400640 1 -- 172.21.15.46:0/1971677463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7eff14022070 con 0x7eff2c0cbec0 2024-04-26T23:38:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 msgr2=0x7eff0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 0x7eff0806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 msgr2=0x7eff2c1ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7eff1400efc0 tx=0x7eff1400c490 comp rx=0 tx=0).stop 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 shutdown_connections 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0806c8f0 0x7eff0806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff2c1640d0 0x7eff2c151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c0cbec0 0x7eff2c1ef590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 --2- 172.21.15.46:0/1971677463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff2c073af0 0x7eff2c1ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 >> 172.21.15.46:0/1971677463 conn(0x7eff2c0b27b0 msgr2=0x7eff2c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 shutdown_connections 2024-04-26T23:38:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:09.843+0000 7eff30c00640 1 -- 172.21.15.46:0/1971677463 wait complete. 2024-04-26T23:38:09.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.077+0000 7f1517e00640 1 Processor -- start 2024-04-26T23:38:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.077+0000 7f1517e00640 1 -- start start 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f151806ca60 0x7f151815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1518072f50 con 0x7f151806f860 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1518073230 con 0x7f1518160b40 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f151806fc40 con 0x7f151806ca60 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f151806ca60 0x7f151815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f151806ca60 0x7f151815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54186/0 (socket says 172.21.15.46:54186) 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1516a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f151806f860 0x7f1518071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45082/0 (socket says 172.21.15.46:45082) 2024-04-26T23:38:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.079+0000 7f1517400640 1 -- 172.21.15.46:0/3805427896 learned_addr learned my addr 172.21.15.46:0/3805427896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.080+0000 7f1517400640 1 -- 172.21.15.46:0/3805427896 >> v1:172.21.15.142:6789/0 conn(0x7f1518160b40 legacy=0x7f151806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.080+0000 7f1517400640 1 -- 172.21.15.46:0/3805427896 >> v1:172.21.15.46:6789/0 conn(0x7f151806f860 legacy=0x7f1518071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.080+0000 7f1515600640 1 -- 172.21.15.46:0/3805427896 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88044881 0 0) 0x7f1518072f50 con 0x7f151806f860 2024-04-26T23:38:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.080+0000 7f1517400640 1 -- 172.21.15.46:0/3805427896 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1518073510 con 0x7f151806ca60 2024-04-26T23:38:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.081+0000 7f1517400640 1 --2- 172.21.15.46:0/3805427896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f151806ca60 0x7f151815d8a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1510009870 tx=0x7f151002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525d39698354f014 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.081+0000 7f1515600640 1 -- 172.21.15.46:0/3805427896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510037070 con 0x7f151806ca60 2024-04-26T23:38:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.081+0000 7f1515600640 1 -- 172.21.15.46:0/3805427896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1510032440 con 0x7f151806ca60 2024-04-26T23:38:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.081+0000 7f1515600640 1 -- 172.21.15.46:0/3805427896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510032810 con 0x7f151806ca60 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 -- 172.21.15.46:0/3805427896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f151806ca60 msgr2=0x7f151815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 --2- 172.21.15.46:0/3805427896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f151806ca60 0x7f151815d8a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1510009870 tx=0x7f151002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 -- 172.21.15.46:0/3805427896 shutdown_connections 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 --2- 172.21.15.46:0/3805427896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f151806ca60 0x7f151815d8a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 -- 172.21.15.46:0/3805427896 >> 172.21.15.46:0/3805427896 conn(0x7f15180d1c30 msgr2=0x7f15180d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 -- 172.21.15.46:0/3805427896 shutdown_connections 2024-04-26T23:38:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.082+0000 7f1517e00640 1 -- 172.21.15.46:0/3805427896 wait complete. 2024-04-26T23:38:10.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.084+0000 7f1517e00640 1 Processor -- start 2024-04-26T23:38:10.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.084+0000 7f1517e00640 1 -- start start 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f151806ca60 0x7f1518156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 0x7f15180b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1518160fd0 con 0x7f1518160b40 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1518073ad0 con 0x7f151806ca60 2024-04-26T23:38:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.085+0000 7f1517e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1518073db0 con 0x7f151806f860 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54202/0 (socket says 172.21.15.46:54202) 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 -- 172.21.15.46:0/2390079650 learned_addr learned my addr 172.21.15.46:0/2390079650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1516a00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f151806ca60 0x7f1518156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1516000640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 0x7f15180b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 -- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 msgr2=0x7f15180b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 0x7f15180b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 -- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f151806ca60 msgr2=0x7f1518156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f151806ca60 0x7f1518156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 -- 172.21.15.46:0/2390079650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180b6840 con 0x7f1518160b40 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1516000640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 0x7f15180b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.086+0000 7f1517400640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f151002a2e0 tx=0x7f15100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.087+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510041070 con 0x7f1518160b40 2024-04-26T23:38:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.087+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151003c950 con 0x7f1518160b40 2024-04-26T23:38:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.087+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510037040 con 0x7f1518160b40 2024-04-26T23:38:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.088+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15180b4ee0 con 0x7f1518160b40 2024-04-26T23:38:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.088+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15180b54a0 con 0x7f1518160b40 2024-04-26T23:38:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.089+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15180693b0 con 0x7f1518160b40 2024-04-26T23:38:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.090+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1510044070 con 0x7f1518160b40 2024-04-26T23:38:10.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.091+0000 7f150ea00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 0x7f14f806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.091+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f15100aca30 con 0x7f1518160b40 2024-04-26T23:38:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.091+0000 7f1516a00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 0x7f14f806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.093+0000 7f1516a00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 0x7f14f806ee00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15080012e0 tx=0x7f1508007ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.095+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f151007ab60 con 0x7f1518160b40 2024-04-26T23:38:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.302+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15180d2990 con 0x7f1518160b40 2024-04-26T23:38:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.303+0000 7f150ea00640 1 -- 172.21.15.46:0/2390079650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f151007a460 con 0x7f1518160b40 2024-04-26T23:38:10.307 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.305+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 msgr2=0x7f14f806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.305+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 0x7f14f806ee00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15080012e0 tx=0x7f1508007ed0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.305+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 msgr2=0x7f151815e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.305+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f151002a2e0 tx=0x7f15100046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 shutdown_connections 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14f806c940 0x7f14f806ee00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518160b40 0x7f151815e500 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f151806f860 0x7f15180b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 --2- 172.21.15.46:0/2390079650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f151806ca60 0x7f1518156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 >> 172.21.15.46:0/2390079650 conn(0x7f15180d1c30 msgr2=0x7f151815f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 shutdown_connections 2024-04-26T23:38:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.306+0000 7f1517e00640 1 -- 172.21.15.46:0/2390079650 wait complete. 2024-04-26T23:38:10.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:10.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.541+0000 7fc115200640 1 Processor -- start 2024-04-26T23:38:10.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.541+0000 7fc115200640 1 -- start start 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.543+0000 7fc115200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc11015f010 0x7fc1100cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.543+0000 7fc115200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1100cd940 con 0x7fc1100c6050 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.543+0000 7fc115200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1100cdc20 con 0x7fc110163650 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.543+0000 7fc115200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc1100ce830 con 0x7fc11015f010 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.543+0000 7fc10f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc11015f010 0x7fc1100cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.544+0000 7fc10f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc11015f010 0x7fc1100cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54218/0 (socket says 172.21.15.46:54218) 2024-04-26T23:38:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.544+0000 7fc10f400640 1 -- 172.21.15.46:0/1927119566 learned_addr learned my addr 172.21.15.46:0/1927119566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.544+0000 7fc10f400640 1 -- 172.21.15.46:0/1927119566 >> v1:172.21.15.142:6789/0 conn(0x7fc110163650 legacy=0x7fc1100c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.544+0000 7fc10f400640 1 -- 172.21.15.46:0/1927119566 >> v1:172.21.15.46:6789/0 conn(0x7fc1100c6050 legacy=0x7fc1100cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.544+0000 7fc10f400640 1 -- 172.21.15.46:0/1927119566 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100cdf00 con 0x7fc11015f010 2024-04-26T23:38:10.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.545+0000 7fc10f400640 1 --2- 172.21.15.46:0/1927119566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11015f010 0x7fc1100cd320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc100009870 tx=0x7fc10002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3eb8cabab15c587 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc10e000640 1 -- 172.21.15.46:0/1927119566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100037070 con 0x7fc11015f010 2024-04-26T23:38:10.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc10e000640 1 -- 172.21.15.46:0/1927119566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc100032440 con 0x7fc11015f010 2024-04-26T23:38:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc10e000640 1 -- 172.21.15.46:0/1927119566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100032810 con 0x7fc11015f010 2024-04-26T23:38:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 -- 172.21.15.46:0/1927119566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11015f010 msgr2=0x7fc1100cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 --2- 172.21.15.46:0/1927119566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11015f010 0x7fc1100cd320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc100009870 tx=0x7fc10002a180 comp rx=0 tx=0).stop 2024-04-26T23:38:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 -- 172.21.15.46:0/1927119566 shutdown_connections 2024-04-26T23:38:10.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 --2- 172.21.15.46:0/1927119566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11015f010 0x7fc1100cd320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 -- 172.21.15.46:0/1927119566 >> 172.21.15.46:0/1927119566 conn(0x7fc1100b26d0 msgr2=0x7fc1100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:10.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 -- 172.21.15.46:0/1927119566 shutdown_connections 2024-04-26T23:38:10.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.546+0000 7fc115200640 1 -- 172.21.15.46:0/1927119566 wait complete. 2024-04-26T23:38:10.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.548+0000 7fc115200640 1 Processor -- start 2024-04-26T23:38:10.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.549+0000 7fc115200640 1 -- start start 2024-04-26T23:38:10.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 0x7fc11014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc110163650 0x7fc1100b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc11006de60 con 0x7fc1100c6050 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc11015f3f0 con 0x7fc11015f010 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc115200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100caae0 con 0x7fc110163650 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc10f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc10f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54232/0 (socket says 172.21.15.46:54232) 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc10f400640 1 -- 172.21.15.46:0/567202092 learned_addr learned my addr 172.21.15.46:0/567202092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.550+0000 7fc10ea00640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 0x7fc11014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 -- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc110163650 msgr2=0x7fc1100b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc110163650 0x7fc1100b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 -- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 msgr2=0x7fc11014a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 0x7fc11014a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 -- 172.21.15.46:0/567202092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1101e98e0 con 0x7fc1100c6050 2024-04-26T23:38:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10ea00640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 0x7fc11014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc10f400640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc100000c00 tx=0x7fc1000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100041070 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10003c870 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100037040 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.551+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101e9b60 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.552+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101ea120 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.553+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11015adf0 con 0x7fc1100c6050 2024-04-26T23:38:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.554+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc100044070 con 0x7fc1100c6050 2024-04-26T23:38:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.555+0000 7fc107400640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.555+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1000acad0 con 0x7fc1100c6050 2024-04-26T23:38:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.555+0000 7fc10ea00640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.557+0000 7fc10ea00640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc007fd0 tx=0x7fc0fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.559+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc11015adf0 con 0x7fc1100c6050 2024-04-26T23:38:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.763+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1100693b0 con 0x7fc1100c6050 2024-04-26T23:38:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.764+0000 7fc107400640 1 -- 172.21.15.46:0/567202092 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc1101e98e0 con 0x7fc1100c6050 2024-04-26T23:38:10.768 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 msgr2=0x7fc0f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc007fd0 tx=0x7fc0fc009040 comp rx=0 tx=0).stop 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 msgr2=0x7fc1100ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc100000c00 tx=0x7fc1000046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 shutdown_connections 2024-04-26T23:38:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc110163650 0x7fc1100b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11015f010 0x7fc11014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 --2- 172.21.15.46:0/567202092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1100c6050 0x7fc1100ca5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 >> 172.21.15.46:0/567202092 conn(0x7fc1100b26d0 msgr2=0x7fc1100cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 shutdown_connections 2024-04-26T23:38:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:10.767+0000 7fc115200640 1 -- 172.21.15.46:0/567202092 wait complete. 2024-04-26T23:38:10.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.004+0000 7f442dc00640 1 Processor -- start 2024-04-26T23:38:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.004+0000 7f442dc00640 1 -- start start 2024-04-26T23:38:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.005+0000 7f442dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 0x7f44280cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.005+0000 7f442dc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44280cd610 0x7f44280cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f442dc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44280af570 con 0x7f44280ae8f0 2024-04-26T23:38:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f442dc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f44280cdfb0 con 0x7f44280cd610 2024-04-26T23:38:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f442dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f44280af850 con 0x7f442815f210 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f4427e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 0x7f44280cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f441fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44280cd610 0x7f44280cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f4427e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 0x7f44280cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:42188/0 (socket says 172.21.15.46:42188) 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f441fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44280cd610 0x7f44280cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:55120/0 (socket says 172.21.15.46:55120) 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.006+0000 7f4427e00640 1 -- 172.21.15.46:0/1549867621 learned_addr learned my addr 172.21.15.46:0/1549867621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.007+0000 7f441fe00640 1 -- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 msgr2=0x7f44280cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.007+0000 7f441fe00640 1 --2- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 0x7f44280cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.007+0000 7f441fe00640 1 -- 172.21.15.46:0/1549867621 >> v1:172.21.15.142:6789/0 conn(0x7f44280ae8f0 legacy=0x7f44280aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.007+0000 7f441fe00640 1 -- 172.21.15.46:0/1549867621 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44280cb680 con 0x7f44280cd610 2024-04-26T23:38:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.008+0000 7f441fe00640 1 --2- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44280cda70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d58af097d4862c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.009+0000 7f4427400640 1 -- 172.21.15.46:0/1549867621 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f44280cd610 2024-04-26T23:38:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.009+0000 7f4427400640 1 -- 172.21.15.46:0/1549867621 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410032440 con 0x7f44280cd610 2024-04-26T23:38:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.009+0000 7f4427400640 1 -- 172.21.15.46:0/1549867621 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410032810 con 0x7f44280cd610 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 -- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 msgr2=0x7f44280cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 --2- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44280cda70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002a180 comp rx=0 tx=0).stop 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 -- 172.21.15.46:0/1549867621 shutdown_connections 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 --2- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44280cda70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 --2- 172.21.15.46:0/1549867621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f442815f210 0x7f44280cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 -- 172.21.15.46:0/1549867621 >> 172.21.15.46:0/1549867621 conn(0x7f44280b27b0 msgr2=0x7f44280b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 -- 172.21.15.46:0/1549867621 shutdown_connections 2024-04-26T23:38:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.010+0000 7f442dc00640 1 -- 172.21.15.46:0/1549867621 wait complete. 2024-04-26T23:38:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.012+0000 7f442dc00640 1 Processor -- start 2024-04-26T23:38:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.012+0000 7f442dc00640 1 -- start start 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 0x7f442815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44281e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f442815f5f0 con 0x7f442815f210 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44280cbc40 con 0x7f44280cd610 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4428071e80 con 0x7f44280ae8f0 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54238/0 (socket says 172.21.15.46:54238) 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.013+0000 7f442d200640 1 -- 172.21.15.46:0/2797066227 learned_addr learned my addr 172.21.15.46:0/2797066227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f441fe00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44281e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f4427e00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 0x7f442815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 -- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 msgr2=0x7f442815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 0x7f442815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 -- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 msgr2=0x7f44281e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44281e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 -- 172.21.15.46:0/2797066227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44281f5560 con 0x7f442815f210 2024-04-26T23:38:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f4427e00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 0x7f442815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.014+0000 7f442d200640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f442000c730 tx=0x7f442000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.015+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420003b70 con 0x7f442815f210 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.015+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420003d30 con 0x7f442815f210 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.015+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420005310 con 0x7f442815f210 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.015+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44281f8af0 con 0x7f442815f210 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.015+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44281f9390 con 0x7f442815f210 2024-04-26T23:38:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.016+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44280693b0 con 0x7f442815f210 2024-04-26T23:38:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.017+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44200056c0 con 0x7f442815f210 2024-04-26T23:38:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.018+0000 7f4424c00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 0x7f440406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.018+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f442008c090 con 0x7f442815f210 2024-04-26T23:38:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.019+0000 7f4427e00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 0x7f440406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.020+0000 7f4427e00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 0x7f440406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4418009a20 tx=0x7f4418008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.023+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f442005a020 con 0x7f442815f210 2024-04-26T23:38:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.223+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f44281e8430 con 0x7f442815f210 2024-04-26T23:38:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.224+0000 7f4424c00640 1 -- 172.21.15.46:0/2797066227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f4420059920 con 0x7f442815f210 2024-04-26T23:38:11.228 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 msgr2=0x7f440406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 0x7f440406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4418009a20 tx=0x7f4418008040 comp rx=0 tx=0).stop 2024-04-26T23:38:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 msgr2=0x7f44281ea1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f442000c730 tx=0x7f442000cbf0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 shutdown_connections 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f440406c8f0 0x7f440406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442815f210 0x7f44281ea1b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44280cd610 0x7f44281e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 --2- 172.21.15.46:0/2797066227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44280ae8f0 0x7f442815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 >> 172.21.15.46:0/2797066227 conn(0x7f44280b27b0 msgr2=0x7f44280b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 shutdown_connections 2024-04-26T23:38:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.227+0000 7f442dc00640 1 -- 172.21.15.46:0/2797066227 wait complete. 2024-04-26T23:38:11.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.457+0000 7f520d000640 1 Processor -- start 2024-04-26T23:38:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.457+0000 7f520d000640 1 -- start start 2024-04-26T23:38:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.458+0000 7f520d000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f52080b4000 0x7f52080d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.458+0000 7f520d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080d11f0 con 0x7f52080d2130 2024-04-26T23:38:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.458+0000 7f520d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080d5f20 con 0x7f520815f4e0 2024-04-26T23:38:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.458+0000 7f520d000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f52080d0ce0 con 0x7f52080b4000 2024-04-26T23:38:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.459+0000 7f5206a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52080d2130 0x7f52080b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45110/0 (socket says 172.21.15.46:45110) 2024-04-26T23:38:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.459+0000 7f5206a00640 1 -- 172.21.15.46:0/3523556156 learned_addr learned my addr 172.21.15.46:0/3523556156 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.459+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113491040 0 0) 0x7f52080d11f0 con 0x7f52080d2130 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.459+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e8003520 con 0x7f52080d2130 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.459+0000 7f5207e00640 1 --2- 172.21.15.46:0/3523556156 >> v2:172.21.15.142:3300/0 conn(0x7f52080b4000 0x7f52080d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1598627150 0 0) 0x7f51e8003520 con 0x7f52080d2130 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 >> v2:172.21.15.142:3300/0 conn(0x7f52080b4000 msgr2=0x7f52080d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 --2- 172.21.15.46:0/3523556156 >> v2:172.21.15.142:3300/0 conn(0x7f52080b4000 0x7f52080d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 >> v1:172.21.15.142:6789/0 conn(0x7f520815f4e0 legacy=0x7f52080d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52080d0f10 con 0x7f52080d2130 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f51f0001670 con 0x7f52080d2130 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 >> v1:172.21.15.46:6789/0 conn(0x7f52080d2130 legacy=0x7f52080b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e8003a30 0x7f51e8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f51e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e80051c0 0x7f51e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51e800c0c0 con 0x7f51e8004800 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51e800c270 con 0x7f51e80051c0 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51e800c470 con 0x7f51e8003a30 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5206a00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f51e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.460+0000 7f5207e00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e80051c0 0x7f51e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5206a00640 1 -- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e8003a30 msgr2=0x7f51e8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5207400640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e8003a30 0x7f51e8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5206a00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e8003a30 0x7f51e8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5206a00640 1 -- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e80051c0 msgr2=0x7f51e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5206a00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e80051c0 0x7f51e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5206a00640 1 -- 172.21.15.46:0/3523556156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e8003520 con 0x7f51e8004800 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5207400640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e8003a30 0x7f51e8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5207e00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e80051c0 0x7f51e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.461+0000 7f5207e00640 1 -- 172.21.15.46:0/3523556156 reap_dead start 2024-04-26T23:38:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f5206a00640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f51e8004c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c110 tx=0x7f51e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a108d57419c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007c40 con 0x7f51e8004800 2024-04-26T23:38:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0005af0 con 0x7f51e8004800 2024-04-26T23:38:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f5206000640 1 -- 172.21.15.46:0/3523556156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007070 con 0x7f51e8004800 2024-04-26T23:38:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f520d000640 1 -- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 msgr2=0x7f51e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.462+0000 7f520d000640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f51e8004c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c110 tx=0x7f51e8005830 comp rx=0 tx=0).stop 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.463+0000 7f520d000640 1 -- 172.21.15.46:0/3523556156 shutdown_connections 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.463+0000 7f520d000640 1 --2- 172.21.15.46:0/3523556156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f51e8004c80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.463+0000 7f520d000640 1 -- 172.21.15.46:0/3523556156 >> 172.21.15.46:0/3523556156 conn(0x7f52080d5010 msgr2=0x7f52080d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.463+0000 7f520d000640 1 -- 172.21.15.46:0/3523556156 shutdown_connections 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.463+0000 7f520d000640 1 -- 172.21.15.46:0/3523556156 wait complete. 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.464+0000 7f520d000640 1 Processor -- start 2024-04-26T23:38:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.465+0000 7f520d000640 1 -- start start 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.465+0000 7f520d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.465+0000 7f520d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 0x7f520806cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f520d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 0x7f520806d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f520d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52080b1d60 con 0x7f51e8004800 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f520d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52080d2990 con 0x7f52080cae10 2024-04-26T23:38:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f520d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f520815f8c0 con 0x7f5208158090 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54264/0 (socket says 172.21.15.46:54264) 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 -- 172.21.15.46:0/558038225 learned_addr learned my addr 172.21.15.46:0/558038225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207e00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 0x7f520806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5206a00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 0x7f520806cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 -- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 msgr2=0x7f520806cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 0x7f520806cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 -- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 msgr2=0x7f520806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 0x7f520806d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207400640 1 -- 172.21.15.46:0/558038225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52080cc0e0 con 0x7f51e8004800 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5207e00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 0x7f520806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.466+0000 7f5206a00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 0x7f520806cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f5207400640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d4b0 tx=0x7f51f800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8005480 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8017070 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8003830 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52080cc3c0 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.467+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52080cc980 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.469+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52080693b0 con 0x7f51e8004800 2024-04-26T23:38:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.470+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51f80039d0 con 0x7f51e8004800 2024-04-26T23:38:11.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.471+0000 7f51ff400640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 0x7f51e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.471+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f51f808bd60 con 0x7f51e8004800 2024-04-26T23:38:11.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.471+0000 7f5206a00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 0x7f51e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.472+0000 7f5206a00640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 0x7f51e406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51f0032000 tx=0x7f51f0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.475+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51f8059dc0 con 0x7f51e8004800 2024-04-26T23:38:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.690+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5208158a50 con 0x7f51e8004800 2024-04-26T23:38:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.693+0000 7f51ff400640 1 -- 172.21.15.46:0/558038225 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f51f80596c0 con 0x7f51e8004800 2024-04-26T23:38:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:11.697 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 msgr2=0x7f51e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 0x7f51e406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51f0032000 tx=0x7f51f0009030 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 msgr2=0x7f52080ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d4b0 tx=0x7f51f800d970 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 shutdown_connections 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51e406c8f0 0x7f51e406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52080cae10 0x7f520806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5208158090 0x7f520806cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 --2- 172.21.15.46:0/558038225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51e8004800 0x7f52080ca8d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 >> 172.21.15.46:0/558038225 conn(0x7f52080d5010 msgr2=0x7f52080b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 shutdown_connections 2024-04-26T23:38:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.696+0000 7f520d000640 1 -- 172.21.15.46:0/558038225 wait complete. 2024-04-26T23:38:11.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:11.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.933+0000 7fa743200640 1 Processor -- start 2024-04-26T23:38:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.934+0000 7fa743200640 1 -- start start 2024-04-26T23:38:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.935+0000 7fa743200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 0x7fa7440cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.935+0000 7fa743200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7440ae8f0 0x7fa7440aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.935+0000 7fa743200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7440af570 con 0x7fa74415f210 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.935+0000 7fa743200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa7440af850 con 0x7fa7440ae8f0 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.935+0000 7fa743200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa7440cb680 con 0x7fa7440cdd20 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7440ae8f0 0x7fa7440aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7440ae8f0 0x7fa7440aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58276/0 (socket says 172.21.15.46:58276) 2024-04-26T23:38:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 -- 172.21.15.46:0/116045298 learned_addr learned my addr 172.21.15.46:0/116045298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa739400640 1 --2- 172.21.15.46:0/116045298 >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 0x7fa7440cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 -- 172.21.15.46:0/116045298 >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 msgr2=0x7fa7440cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 --2- 172.21.15.46:0/116045298 >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 0x7fa7440cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 -- 172.21.15.46:0/116045298 >> v1:172.21.15.142:6789/0 conn(0x7fa74415f210 legacy=0x7fa7440cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa742800640 1 -- 172.21.15.46:0/116045298 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7440cb960 con 0x7fa7440ae8f0 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.936+0000 7fa739400640 1 --2- 172.21.15.46:0/116045298 >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 0x7fa7440cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.937+0000 7fa742800640 1 --2- 172.21.15.46:0/116045298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440aed00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7440b3710 tx=0x7fa73c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913160fcbce9e09c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa741400640 1 -- 172.21.15.46:0/116045298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c036070 con 0x7fa7440ae8f0 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa741400640 1 -- 172.21.15.46:0/116045298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c031600 con 0x7fa7440ae8f0 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa741400640 1 -- 172.21.15.46:0/116045298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0319d0 con 0x7fa7440ae8f0 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 -- 172.21.15.46:0/116045298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 msgr2=0x7fa7440aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 --2- 172.21.15.46:0/116045298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440aed00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7440b3710 tx=0x7fa73c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 -- 172.21.15.46:0/116045298 shutdown_connections 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 --2- 172.21.15.46:0/116045298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440aed00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 --2- 172.21.15.46:0/116045298 >> v2:172.21.15.142:3300/0 conn(0x7fa7440cdd20 0x7fa7440cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 -- 172.21.15.46:0/116045298 >> 172.21.15.46:0/116045298 conn(0x7fa7440b27b0 msgr2=0x7fa7440b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.938+0000 7fa743200640 1 -- 172.21.15.46:0/116045298 shutdown_connections 2024-04-26T23:38:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.939+0000 7fa743200640 1 -- 172.21.15.46:0/116045298 wait complete. 2024-04-26T23:38:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.940+0000 7fa743200640 1 Processor -- start 2024-04-26T23:38:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.940+0000 7fa743200640 1 -- start start 2024-04-26T23:38:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 0x7fa744152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74415f210 0x7fa744155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa74415f5f0 con 0x7fa7440ae8f0 2024-04-26T23:38:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440af240 con 0x7fa7440cdd20 2024-04-26T23:38:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.941+0000 7fa743200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa744071e80 con 0x7fa74415f210 2024-04-26T23:38:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58290/0 (socket says 172.21.15.46:58290) 2024-04-26T23:38:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 -- 172.21.15.46:0/442454943 learned_addr learned my addr 172.21.15.46:0/442454943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa739400640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 0x7fa744152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 -- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74415f210 msgr2=0x7fa744155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa742800640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74415f210 0x7fa744155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74415f210 0x7fa744155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 -- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 msgr2=0x7fa744152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 0x7fa744152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 -- 172.21.15.46:0/442454943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa744152a60 con 0x7fa7440ae8f0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa739400640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 0x7fa744152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.942+0000 7fa741e00640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa73400b420 tx=0x7fa73400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.943+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734011830 con 0x7fa7440ae8f0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.943+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa734016070 con 0x7fa7440ae8f0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.943+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734011c10 con 0x7fa7440ae8f0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.943+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7440c8120 con 0x7fa7440ae8f0 2024-04-26T23:38:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.943+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7440c86e0 con 0x7fa7440ae8f0 2024-04-26T23:38:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.944+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7440693b0 con 0x7fa7440ae8f0 2024-04-26T23:38:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.945+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa73401b460 con 0x7fa7440ae8f0 2024-04-26T23:38:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.946+0000 7fa73aa00640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 0x7fa72006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.946+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa73408b980 con 0x7fa7440ae8f0 2024-04-26T23:38:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.946+0000 7fa739400640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 0x7fa72006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.948+0000 7fa739400640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 0x7fa72006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:11.950+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa734020020 con 0x7fa7440ae8f0 2024-04-26T23:38:12.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.155+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa744070020 con 0x7fa7440ae8f0 2024-04-26T23:38:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.156+0000 7fa73aa00640 1 -- 172.21.15.46:0/442454943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fa734059a40 con 0x7fa7440ae8f0 2024-04-26T23:38:12.160 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 msgr2=0x7fa72006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 0x7fa72006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c008040 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 msgr2=0x7fa7440c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa73400b420 tx=0x7fa73400b8e0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 shutdown_connections 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa72006c8f0 0x7fa72006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74415f210 0x7fa744155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7440cdd20 0x7fa744152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 --2- 172.21.15.46:0/442454943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7440ae8f0 0x7fa7440c91c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.159+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 >> 172.21.15.46:0/442454943 conn(0x7fa7440b27b0 msgr2=0x7fa7440738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.160+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 shutdown_connections 2024-04-26T23:38:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.160+0000 7fa743200640 1 -- 172.21.15.46:0/442454943 wait complete. 2024-04-26T23:38:12.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:12.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.391+0000 7fc614c00640 1 Processor -- start 2024-04-26T23:38:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.392+0000 7fc614c00640 1 -- start start 2024-04-26T23:38:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.393+0000 7fc614c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6101626c0 0x7fc6100b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.393+0000 7fc614c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6100d07d0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.393+0000 7fc614c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6100d0ab0 con 0x7fc610162aa0 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.393+0000 7fc614c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc6100d0d90 con 0x7fc6101626c0 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6101626c0 0x7fc6100b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6101626c0 0x7fc6100b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58300/0 (socket says 172.21.15.46:58300) 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 -- 172.21.15.46:0/2961077797 learned_addr learned my addr 172.21.15.46:0/2961077797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60ea00640 1 --1- 172.21.15.46:0/2961077797 >> v1:172.21.15.46:6789/0 conn(0x7fc6100b5fb0 0x7fc6100d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49496/0 (socket says 172.21.15.46:49496) 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 -- 172.21.15.46:0/2961077797 >> v1:172.21.15.142:6789/0 conn(0x7fc610162aa0 legacy=0x7fc6100d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 -- 172.21.15.46:0/2961077797 >> v1:172.21.15.46:6789/0 conn(0x7fc6100b5fb0 legacy=0x7fc6100d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.394+0000 7fc60f400640 1 -- 172.21.15.46:0/2961077797 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6100d2c80 con 0x7fc6101626c0 2024-04-26T23:38:12.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.395+0000 7fc60f400640 1 --2- 172.21.15.46:0/2961077797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6101626c0 0x7fc6100b47c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ac7f6937113ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc60e000640 1 -- 172.21.15.46:0/2961077797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600037070 con 0x7fc6101626c0 2024-04-26T23:38:12.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc60e000640 1 -- 172.21.15.46:0/2961077797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc600032440 con 0x7fc6101626c0 2024-04-26T23:38:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc60e000640 1 -- 172.21.15.46:0/2961077797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600032810 con 0x7fc6101626c0 2024-04-26T23:38:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc614c00640 1 -- 172.21.15.46:0/2961077797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6101626c0 msgr2=0x7fc6100b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc614c00640 1 --2- 172.21.15.46:0/2961077797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6101626c0 0x7fc6100b47c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc614c00640 1 -- 172.21.15.46:0/2961077797 shutdown_connections 2024-04-26T23:38:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc614c00640 1 --2- 172.21.15.46:0/2961077797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6101626c0 0x7fc6100b47c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.396+0000 7fc614c00640 1 -- 172.21.15.46:0/2961077797 >> 172.21.15.46:0/2961077797 conn(0x7fc61009e2a0 msgr2=0x7fc6100d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.397+0000 7fc614c00640 1 -- 172.21.15.46:0/2961077797 shutdown_connections 2024-04-26T23:38:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.397+0000 7fc614c00640 1 -- 172.21.15.46:0/2961077797 wait complete. 2024-04-26T23:38:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.398+0000 7fc614c00640 1 Processor -- start 2024-04-26T23:38:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.398+0000 7fc614c00640 1 -- start start 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6101626c0 0x7fc610161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc610162aa0 0x7fc6100cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc61016a2e0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100d2260 con 0x7fc6101626c0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.399+0000 7fc614c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc61016a460 con 0x7fc610162aa0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58308/0 (socket says 172.21.15.46:58308) 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 -- 172.21.15.46:0/1149087695 learned_addr learned my addr 172.21.15.46:0/1149087695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60fe00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc610162aa0 0x7fc6100cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60ea00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6101626c0 0x7fc610161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 -- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc610162aa0 msgr2=0x7fc6100cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc610162aa0 0x7fc6100cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 -- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6101626c0 msgr2=0x7fc610161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6101626c0 0x7fc610161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 -- 172.21.15.46:0/1149087695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6100c8d00 con 0x7fc6100b5fb0 2024-04-26T23:38:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.400+0000 7fc60f400640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc6000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.401+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600041070 con 0x7fc6100b5fb0 2024-04-26T23:38:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.401+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60003c870 con 0x7fc6100b5fb0 2024-04-26T23:38:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.401+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600037040 con 0x7fc6100b5fb0 2024-04-26T23:38:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.401+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6100c8fe0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.401+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6100c95a0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.402+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6100693b0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.403+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc600044070 con 0x7fc6100b5fb0 2024-04-26T23:38:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.404+0000 7fc607400640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.404+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc6000acd30 con 0x7fc6100b5fb0 2024-04-26T23:38:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.405+0000 7fc60ea00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.406+0000 7fc60ea00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 0x7fc5f006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc007fd0 tx=0x7fc5fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.409+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc60007ade0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.615+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6100d3460 con 0x7fc6100b5fb0 2024-04-26T23:38:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.616+0000 7fc607400640 1 -- 172.21.15.46:0/1149087695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc60007a6e0 con 0x7fc6100b5fb0 2024-04-26T23:38:12.620 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.618+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 msgr2=0x7fc5f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.618+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 0x7fc5f006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc007fd0 tx=0x7fc5fc009040 comp rx=0 tx=0).stop 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.618+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 msgr2=0x7fc610158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.618+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc6000046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 shutdown_connections 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc610162aa0 0x7fc6100cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6101626c0 0x7fc610161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 --2- 172.21.15.46:0/1149087695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6100b5fb0 0x7fc610158bd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 >> 172.21.15.46:0/1149087695 conn(0x7fc61009e2a0 msgr2=0x7fc610159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 shutdown_connections 2024-04-26T23:38:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.619+0000 7fc614c00640 1 -- 172.21.15.46:0/1149087695 wait complete. 2024-04-26T23:38:12.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.863+0000 7f3271600640 1 Processor -- start 2024-04-26T23:38:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.863+0000 7f3271600640 1 -- start start 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3271600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f326c06f4b0 0x7f326c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3271600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 0x7f326c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3271600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c0b4b30 con 0x7f326c0d04c0 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3271600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0b4e10 con 0x7f326c06f4b0 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3271600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0b50f0 con 0x7f326c0b4320 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f3262000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f326c06f4b0 0x7f326c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f326aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f326c0d04c0 0x7f326c06eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49508/0 (socket says 172.21.15.46:49508) 2024-04-26T23:38:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.865+0000 7f326aa00640 1 -- 172.21.15.46:0/2513450676 learned_addr learned my addr 172.21.15.46:0/2513450676 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f3262000640 1 -- 172.21.15.46:0/2513450676 >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 msgr2=0x7f326c0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f326b400640 1 --2- 172.21.15.46:0/2513450676 >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 0x7f326c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f3262000640 1 --2- 172.21.15.46:0/2513450676 >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 0x7f326c0ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f3262000640 1 -- 172.21.15.46:0/2513450676 >> v1:172.21.15.46:6789/0 conn(0x7f326c0d04c0 legacy=0x7f326c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f326a000640 1 -- 172.21.15.46:0/2513450676 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866319566 0 0) 0x7f326c0b4b30 con 0x7f326c0d04c0 2024-04-26T23:38:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f3262000640 1 -- 172.21.15.46:0/2513450676 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c0b53d0 con 0x7f326c06f4b0 2024-04-26T23:38:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.866+0000 7f326b400640 1 --2- 172.21.15.46:0/2513450676 >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 0x7f326c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.867+0000 7f3262000640 1 --2- 172.21.15.46:0/2513450676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c06f4b0 0x7f326c0b3de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f326c145e70 tx=0x7f325402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc3fc2c18b530cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.867+0000 7f326a000640 1 -- 172.21.15.46:0/2513450676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254037070 con 0x7f326c06f4b0 2024-04-26T23:38:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.867+0000 7f326a000640 1 -- 172.21.15.46:0/2513450676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3254032440 con 0x7f326c06f4b0 2024-04-26T23:38:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.867+0000 7f326a000640 1 -- 172.21.15.46:0/2513450676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325402ae60 con 0x7f326c06f4b0 2024-04-26T23:38:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.868+0000 7f3271600640 1 -- 172.21.15.46:0/2513450676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c06f4b0 msgr2=0x7f326c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.868+0000 7f3271600640 1 --2- 172.21.15.46:0/2513450676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c06f4b0 0x7f326c0b3de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f326c145e70 tx=0x7f325402a110 comp rx=0 tx=0).stop 2024-04-26T23:38:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 -- 172.21.15.46:0/2513450676 shutdown_connections 2024-04-26T23:38:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 --2- 172.21.15.46:0/2513450676 >> v2:172.21.15.142:3300/0 conn(0x7f326c0b4320 0x7f326c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 --2- 172.21.15.46:0/2513450676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c06f4b0 0x7f326c0b3de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 -- 172.21.15.46:0/2513450676 >> 172.21.15.46:0/2513450676 conn(0x7f326c06c980 msgr2=0x7f326c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 -- 172.21.15.46:0/2513450676 shutdown_connections 2024-04-26T23:38:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.869+0000 7f3271600640 1 -- 172.21.15.46:0/2513450676 wait complete. 2024-04-26T23:38:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.871+0000 7f3271600640 1 Processor -- start 2024-04-26T23:38:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.871+0000 7f3271600640 1 -- start start 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f326c06f4b0 0x7f326c140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326c0b4320 0x7f326c0aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0b4910 con 0x7f326c0d04c0 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0b4790 con 0x7f326c0b4320 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3271600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f326c0ce9b0 con 0x7f326c06f4b0 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f326b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f326b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58320/0 (socket says 172.21.15.46:58320) 2024-04-26T23:38:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f326b400640 1 -- 172.21.15.46:0/351821412 learned_addr learned my addr 172.21.15.46:0/351821412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f326aa00640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f326c06f4b0 0x7f326c140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.873+0000 7f3262000640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326c0b4320 0x7f326c0aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 -- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f326c06f4b0 msgr2=0x7f326c140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f326c06f4b0 0x7f326c140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 -- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326c0b4320 msgr2=0x7f326c0aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326c0b4320 0x7f326c0aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 -- 172.21.15.46:0/351821412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c141dc0 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f326b400640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f326400ae90 tx=0x7f326400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32640079b0 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264007b50 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326400ce10 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c1409c0 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.874+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c13f6c0 con 0x7f326c0d04c0 2024-04-26T23:38:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.875+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c069530 con 0x7f326c0d04c0 2024-04-26T23:38:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.877+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3264010440 con 0x7f326c0d04c0 2024-04-26T23:38:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.878+0000 7f3263400640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 0x7f324806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.878+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f326408dfa0 con 0x7f326c0d04c0 2024-04-26T23:38:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.878+0000 7f326aa00640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 0x7f324806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.880+0000 7f326aa00640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 0x7f324806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f325c0012e0 tx=0x7f325c005eb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:12.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:12.882+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f326c141dc0 con 0x7f326c0d04c0 2024-04-26T23:38:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.087+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f326c0b0530 con 0x7f326c0d04c0 2024-04-26T23:38:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.088+0000 7f3263400640 1 -- 172.21.15.46:0/351821412 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f326405bfd0 con 0x7f326c0d04c0 2024-04-26T23:38:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.090+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 msgr2=0x7f324806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.090+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 0x7f324806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f325c0012e0 tx=0x7f325c005eb0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.090+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 msgr2=0x7f326c0ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.090+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f326400ae90 tx=0x7f326400cbc0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 shutdown_connections 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f324806c920 0x7f324806ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326c0d04c0 0x7f326c0ae480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326c0b4320 0x7f326c0aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 --2- 172.21.15.46:0/351821412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f326c06f4b0 0x7f326c140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 >> 172.21.15.46:0/351821412 conn(0x7f326c06c980 msgr2=0x7f326c073b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 shutdown_connections 2024-04-26T23:38:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.091+0000 7f3271600640 1 -- 172.21.15.46:0/351821412 wait complete. 2024-04-26T23:38:13.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.327+0000 7f130a000640 1 Processor -- start 2024-04-26T23:38:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.327+0000 7f130a000640 1 -- start start 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.328+0000 7f130a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1304162920 0x7f13040d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.328+0000 7f130a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f13040d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.328+0000 7f130a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13040d3d90 con 0x7f13040b1cd0 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.328+0000 7f130a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f13040d2980 con 0x7f1304162920 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.328+0000 7f130a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f13040d24f0 con 0x7f13040b3d60 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1309600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f13040b1cd0 0x7f13040b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49512/0 (socket says 172.21.15.46:49512) 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1309600640 1 -- 172.21.15.46:0/3664416861 learned_addr learned my addr 172.21.15.46:0/3664416861 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1303e00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f13040d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1308c00640 1 --2- 172.21.15.46:0/3664416861 >> v2:172.21.15.142:3300/0 conn(0x7f1304162920 0x7f13040d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963368404 0 0) 0x7f13040d3d90 con 0x7f13040b1cd0 2024-04-26T23:38:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.329+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e8003520 con 0x7f13040b1cd0 2024-04-26T23:38:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1335843559 0 0) 0x7f12e8003520 con 0x7f13040b1cd0 2024-04-26T23:38:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 msgr2=0x7f13040d3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f13040d3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 >> v2:172.21.15.142:3300/0 conn(0x7f1304162920 msgr2=0x7f13040d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 --2- 172.21.15.46:0/3664416861 >> v2:172.21.15.142:3300/0 conn(0x7f1304162920 0x7f13040d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1304162d00 con 0x7f13040b1cd0 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f12fc0019f0 con 0x7f13040b1cd0 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 >> v1:172.21.15.46:6789/0 conn(0x7f13040b1cd0 legacy=0x7f13040b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e80039b0 0x7f12e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f12e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13040d2980 con 0x7f12e80039b0 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13040d24f0 con 0x7f12e8004800 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12e800b9c0 con 0x7f13040b3d60 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.330+0000 7f1308c00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e80039b0 0x7f12e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1309600640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f12e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1308c00640 1 -- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 msgr2=0x7f12e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1303e00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1308c00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f12e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1308c00640 1 -- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e8004800 msgr2=0x7f12e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1308c00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1308c00640 1 -- 172.21.15.46:0/3664416861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e8003520 con 0x7f12e80039b0 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.331+0000 7f1309600640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040b3d60 0x7f12e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.332+0000 7f1308c00640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e80039b0 0x7f12e8003dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009af0 tx=0x7f12e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf6c6c3470dfbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.332+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036070 con 0x7f12e80039b0 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.332+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4031400 con 0x7f12e80039b0 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.332+0000 7f1303400640 1 -- 172.21.15.46:0/3664416861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4009690 con 0x7f12e80039b0 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 -- 172.21.15.46:0/3664416861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e80039b0 msgr2=0x7f12e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 --2- 172.21.15.46:0/3664416861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e80039b0 0x7f12e8003dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009af0 tx=0x7f12e8005830 comp rx=0 tx=0).stop 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f1309600640 1 -- 172.21.15.46:0/3664416861 reap_dead start 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 -- 172.21.15.46:0/3664416861 shutdown_connections 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 -- 172.21.15.46:0/3664416861 >> 172.21.15.46:0/3664416861 conn(0x7f13040d5000 msgr2=0x7f13040d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 -- 172.21.15.46:0/3664416861 shutdown_connections 2024-04-26T23:38:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.333+0000 7f130a000640 1 -- 172.21.15.46:0/3664416861 wait complete. 2024-04-26T23:38:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.335+0000 7f130a000640 1 Processor -- start 2024-04-26T23:38:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.335+0000 7f130a000640 1 -- start start 2024-04-26T23:38:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 0x7f130414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040ca520 0x7f13041f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13040d2180 con 0x7f13041f89d0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13040d1ea0 con 0x7f12e80039b0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f130a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f13040b4340 con 0x7f13040ca520 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f1309600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f1309600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58332/0 (socket says 172.21.15.46:58332) 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.336+0000 7f1309600640 1 -- 172.21.15.46:0/1815812148 learned_addr learned my addr 172.21.15.46:0/1815812148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1303e00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040ca520 0x7f13041f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1308c00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 0x7f130414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 -- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040ca520 msgr2=0x7f13041f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040ca520 0x7f13041f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 -- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 msgr2=0x7f130414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 0x7f130414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 -- 172.21.15.46:0/1815812148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13041f8e70 con 0x7f13041f89d0 2024-04-26T23:38:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1308c00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 0x7f130414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1309600640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13040b7b80 tx=0x7f12fc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.337+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc004d60 con 0x7f13041f89d0 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.338+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc00bb80 con 0x7f13041f89d0 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.338+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc007070 con 0x7f13041f89d0 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.338+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13041f9150 con 0x7f13041f89d0 2024-04-26T23:38:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.338+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13040ca000 con 0x7f13041f89d0 2024-04-26T23:38:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.339+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13040ca200 con 0x7f13041f89d0 2024-04-26T23:38:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.340+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12fc01b020 con 0x7f13041f89d0 2024-04-26T23:38:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.341+0000 7f1300c00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 0x7f12e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.341+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12fc08c110 con 0x7f13041f89d0 2024-04-26T23:38:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.341+0000 7f1308c00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 0x7f12e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.343+0000 7f1308c00640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 0x7f12e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12f4033000 tx=0x7f12f4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.345+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13040ca200 con 0x7f13041f89d0 2024-04-26T23:38:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.554+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1304069ac0 con 0x7f13041f89d0 2024-04-26T23:38:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.555+0000 7f1300c00640 1 -- 172.21.15.46:0/1815812148 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f12fc05a0a0 con 0x7f13041f89d0 2024-04-26T23:38:13.559 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 msgr2=0x7f12e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 0x7f12e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12f4033000 tx=0x7f12f4008030 comp rx=0 tx=0).stop 2024-04-26T23:38:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 msgr2=0x7f13040b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13040b7b80 tx=0x7f12fc00b4a0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 shutdown_connections 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f12e006c8f0 0x7f12e006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13041f89d0 0x7f13040b2160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13040ca520 0x7f13041f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 --2- 172.21.15.46:0/1815812148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80039b0 0x7f130414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 >> 172.21.15.46:0/1815812148 conn(0x7f13040d5000 msgr2=0x7f13040d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.558+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 shutdown_connections 2024-04-26T23:38:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.559+0000 7f130a000640 1 -- 172.21.15.46:0/1815812148 wait complete. 2024-04-26T23:38:13.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:13.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.787+0000 7f77eb200640 1 Processor -- start 2024-04-26T23:38:13.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.787+0000 7f77eb200640 1 -- start start 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77eb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77ec0d0f50 0x7f77ec06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77eb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 0x7f77ec0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77eb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ec0d21d0 con 0x7f77ec0d1330 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77eb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f77ec0d1ef0 con 0x7f77ec0d0f50 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77eb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f77ec0b6bc0 con 0x7f77ec0b86b0 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77e9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77ec0d0f50 0x7f77ec06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77e9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77ec0d0f50 0x7f77ec06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58346/0 (socket says 172.21.15.46:58346) 2024-04-26T23:38:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77e9e00640 1 -- 172.21.15.46:0/660587365 learned_addr learned my addr 172.21.15.46:0/660587365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:13.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.789+0000 7f77ea800640 1 --2- 172.21.15.46:0/660587365 >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 0x7f77ec0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.790+0000 7f77e9e00640 1 -- 172.21.15.46:0/660587365 >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 msgr2=0x7f77ec0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.790+0000 7f77e9e00640 1 --2- 172.21.15.46:0/660587365 >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 0x7f77ec0d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.790+0000 7f77e9e00640 1 -- 172.21.15.46:0/660587365 >> v1:172.21.15.46:6789/0 conn(0x7f77ec0d1330 legacy=0x7f77ec0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.790+0000 7f77e9e00640 1 -- 172.21.15.46:0/660587365 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec0b6ea0 con 0x7f77ec0d0f50 2024-04-26T23:38:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.790+0000 7f77ea800640 1 --2- 172.21.15.46:0/660587365 >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 0x7f77ec0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:13.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.791+0000 7f77e9e00640 1 --2- 172.21.15.46:0/660587365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d0f50 0x7f77ec06d670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77ec069020 tx=0x7f77dc02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99403225263a1815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.791+0000 7f77e8a00640 1 -- 172.21.15.46:0/660587365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc037070 con 0x7f77ec0d0f50 2024-04-26T23:38:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.791+0000 7f77e8a00640 1 -- 172.21.15.46:0/660587365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc032440 con 0x7f77ec0d0f50 2024-04-26T23:38:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.791+0000 7f77e8a00640 1 -- 172.21.15.46:0/660587365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc032810 con 0x7f77ec0d0f50 2024-04-26T23:38:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 -- 172.21.15.46:0/660587365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d0f50 msgr2=0x7f77ec06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 --2- 172.21.15.46:0/660587365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d0f50 0x7f77ec06d670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77ec069020 tx=0x7f77dc02a1f0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 -- 172.21.15.46:0/660587365 shutdown_connections 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 --2- 172.21.15.46:0/660587365 >> v2:172.21.15.142:3300/0 conn(0x7f77ec0b86b0 0x7f77ec0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 --2- 172.21.15.46:0/660587365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d0f50 0x7f77ec06d670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 -- 172.21.15.46:0/660587365 >> 172.21.15.46:0/660587365 conn(0x7f77ec0d2cc0 msgr2=0x7f77ec0d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 -- 172.21.15.46:0/660587365 shutdown_connections 2024-04-26T23:38:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.792+0000 7f77eb200640 1 -- 172.21.15.46:0/660587365 wait complete. 2024-04-26T23:38:13.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.794+0000 7f77eb200640 1 Processor -- start 2024-04-26T23:38:13.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.794+0000 7f77eb200640 1 -- start start 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77ec0b86b0 0x7f77ec1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77ec0d0f50 0x7f77ec1490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec15f7c0 con 0x7f77ec0d1330 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec15f640 con 0x7f77ec0d0f50 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77eb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec0b3ff0 con 0x7f77ec0b86b0 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58362/0 (socket says 172.21.15.46:58362) 2024-04-26T23:38:13.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.795+0000 7f77ea800640 1 -- 172.21.15.46:0/553153940 learned_addr learned my addr 172.21.15.46:0/553153940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77e9e00640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77ec0b86b0 0x7f77ec1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77e9400640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77ec0d0f50 0x7f77ec1490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 -- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77ec0b86b0 msgr2=0x7f77ec1526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77ec0b86b0 0x7f77ec1526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 -- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77ec0d0f50 msgr2=0x7f77ec1490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77ec0d0f50 0x7f77ec1490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 -- 172.21.15.46:0/553153940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec0cfda0 con 0x7f77ec0d1330 2024-04-26T23:38:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77ea800640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f77e400ced0 tx=0x7f77e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.796+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4003b90 con 0x7f77ec0d1330 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.797+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e4003d50 con 0x7f77ec0d1330 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.797+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004260 con 0x7f77ec0d1330 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.797+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec0cf500 con 0x7f77ec0d1330 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.797+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec0cfac0 con 0x7f77ec0d1330 2024-04-26T23:38:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.798+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ec069ac0 con 0x7f77ec0d1330 2024-04-26T23:38:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.799+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77e4004400 con 0x7f77ec0d1330 2024-04-26T23:38:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.800+0000 7f77e2000640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.800+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f77e408c970 con 0x7f77ec0d1330 2024-04-26T23:38:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.800+0000 7f77e9e00640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:13.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.802+0000 7f77e9e00640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 0x7f77c806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77dc034000 tx=0x7f77dc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:13.804+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77e405a2e0 con 0x7f77ec0d1330 2024-04-26T23:38:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.010+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77ec1599f0 con 0x7f77ec0d1330 2024-04-26T23:38:14.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.013+0000 7f77e2000640 1 -- 172.21.15.46:0/553153940 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f77e408ae20 con 0x7f77ec0d1330 2024-04-26T23:38:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:14.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 msgr2=0x7f77c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 0x7f77c806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77dc034000 tx=0x7f77dc02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 msgr2=0x7f77ec0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f77e400ced0 tx=0x7f77e400a710 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 shutdown_connections 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77ec0d1330 0x7f77ec0ce7c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77ec0d0f50 0x7f77ec1490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.016+0000 7f77eb200640 1 --2- 172.21.15.46:0/553153940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77ec0b86b0 0x7f77ec1526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.017+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 >> 172.21.15.46:0/553153940 conn(0x7f77ec0d2cc0 msgr2=0x7f77ec0700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.017+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 shutdown_connections 2024-04-26T23:38:14.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.017+0000 7f77eb200640 1 -- 172.21.15.46:0/553153940 wait complete. 2024-04-26T23:38:14.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:14.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.257+0000 7f00ce400640 1 Processor -- start 2024-04-26T23:38:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.257+0000 7f00ce400640 1 -- start start 2024-04-26T23:38:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.258+0000 7f00ce400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 0x7f00c80b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.258+0000 7f00ce400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f00c80b4fb0 0x7f00c80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.258+0000 7f00ce400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c80ce500 con 0x7f00c815d170 2024-04-26T23:38:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.258+0000 7f00ce400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f00c80b5400 con 0x7f00c80b4fb0 2024-04-26T23:38:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.258+0000 7f00ce400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00c80ce7e0 con 0x7f00c80cf880 2024-04-26T23:38:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.259+0000 7f00c7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00c815d170 0x7f00c80cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49536/0 (socket says 172.21.15.46:49536) 2024-04-26T23:38:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.259+0000 7f00c7400640 1 -- 172.21.15.46:0/3486115348 learned_addr learned my addr 172.21.15.46:0/3486115348 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.259+0000 7f00c6a00640 1 --2- 172.21.15.46:0/3486115348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 0x7f00c80b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.259+0000 7f00c7e00640 1 --2- 172.21.15.46:0/3486115348 >> v2:172.21.15.142:3300/0 conn(0x7f00c80b4fb0 0x7f00c80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.260+0000 7f00c6a00640 1 -- 172.21.15.46:0/3486115348 >> v2:172.21.15.142:3300/0 conn(0x7f00c80b4fb0 msgr2=0x7f00c80c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:14.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.260+0000 7f00c6a00640 1 --2- 172.21.15.46:0/3486115348 >> v2:172.21.15.142:3300/0 conn(0x7f00c80b4fb0 0x7f00c80c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.260+0000 7f00c6a00640 1 -- 172.21.15.46:0/3486115348 >> v1:172.21.15.46:6789/0 conn(0x7f00c815d170 legacy=0x7f00c80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.260+0000 7f00c6a00640 1 -- 172.21.15.46:0/3486115348 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c80ceac0 con 0x7f00c80cf880 2024-04-26T23:38:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.261+0000 7f00c6a00640 1 --2- 172.21.15.46:0/3486115348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 0x7f00c80b4a70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097a0 tx=0x7f00b402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2b71a6fdba208e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.264+0000 7f00c6000640 1 -- 172.21.15.46:0/3486115348 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4037070 con 0x7f00c80cf880 2024-04-26T23:38:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.264+0000 7f00c6000640 1 -- 172.21.15.46:0/3486115348 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4032440 con 0x7f00c80cf880 2024-04-26T23:38:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.264+0000 7f00c6000640 1 -- 172.21.15.46:0/3486115348 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4032810 con 0x7f00c80cf880 2024-04-26T23:38:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.264+0000 7f00ce400640 1 -- 172.21.15.46:0/3486115348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 msgr2=0x7f00c80b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.264+0000 7f00ce400640 1 --2- 172.21.15.46:0/3486115348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 0x7f00c80b4a70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097a0 tx=0x7f00b402a180 comp rx=0 tx=0).stop 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 -- 172.21.15.46:0/3486115348 shutdown_connections 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 --2- 172.21.15.46:0/3486115348 >> v2:172.21.15.142:3300/0 conn(0x7f00c80b4fb0 0x7f00c80c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 --2- 172.21.15.46:0/3486115348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80cf880 0x7f00c80b4a70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 -- 172.21.15.46:0/3486115348 >> 172.21.15.46:0/3486115348 conn(0x7f00c809ba70 msgr2=0x7f00c809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 -- 172.21.15.46:0/3486115348 shutdown_connections 2024-04-26T23:38:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.265+0000 7f00ce400640 1 -- 172.21.15.46:0/3486115348 wait complete. 2024-04-26T23:38:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.267+0000 7f00ce400640 1 Processor -- start 2024-04-26T23:38:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.267+0000 7f00ce400640 1 -- start start 2024-04-26T23:38:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 0x7f00c81603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00c815d170 0x7f00c80b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00c80ce1a0 con 0x7f00c80cf880 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00c80c6be0 con 0x7f00c815d170 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00ce400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00c815d550 con 0x7f00c80b4fb0 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58364/0 (socket says 172.21.15.46:58364) 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00c6a00640 1 -- 172.21.15.46:0/2711443698 learned_addr learned my addr 172.21.15.46:0/2711443698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.268+0000 7f00c7400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 0x7f00c81603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 -- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 msgr2=0x7f00c81603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c7e00640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00c815d170 0x7f00c80b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 0x7f00c81603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 -- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00c815d170 msgr2=0x7f00c80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00c815d170 0x7f00c80b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 -- 172.21.15.46:0/2711443698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c80ccad0 con 0x7f00c80cf880 2024-04-26T23:38:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c7400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 0x7f00c81603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00c6a00640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f00b4035040 tx=0x7f00b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4037040 con 0x7f00c80cf880 2024-04-26T23:38:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4030500 con 0x7f00c80cf880 2024-04-26T23:38:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4030910 con 0x7f00c80cf880 2024-04-26T23:38:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c815d900 con 0x7f00c80cf880 2024-04-26T23:38:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.269+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c815dec0 con 0x7f00c80cf880 2024-04-26T23:38:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.271+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c8069530 con 0x7f00c80cf880 2024-04-26T23:38:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.272+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00b4044020 con 0x7f00c80cf880 2024-04-26T23:38:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.272+0000 7f00bf400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 0x7f00a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.273+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00b40ac220 con 0x7f00c80cf880 2024-04-26T23:38:14.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.273+0000 7f00c7400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 0x7f00a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.274+0000 7f00c7400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 0x7f00a806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00b8000f60 tx=0x7f00b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.277+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00b407a1c0 con 0x7f00c80cf880 2024-04-26T23:38:14.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.491+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00c81f4d50 con 0x7f00c80cf880 2024-04-26T23:38:14.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.492+0000 7f00bf400640 1 -- 172.21.15.46:0/2711443698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f00b4079ac0 con 0x7f00c80cf880 2024-04-26T23:38:14.496 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 msgr2=0x7f00a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 0x7f00a806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00b8000f60 tx=0x7f00b8009040 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 msgr2=0x7f00c81f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f00b4035040 tx=0x7f00b40046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 shutdown_connections 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f00a806c8f0 0x7f00a806edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00c815d170 0x7f00c80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c80cf880 0x7f00c81f4810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 --2- 172.21.15.46:0/2711443698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00c80b4fb0 0x7f00c81603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 >> 172.21.15.46:0/2711443698 conn(0x7f00c809ba70 msgr2=0x7f00c80d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 shutdown_connections 2024-04-26T23:38:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.495+0000 7f00ce400640 1 -- 172.21.15.46:0/2711443698 wait complete. 2024-04-26T23:38:14.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:14.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.732+0000 7f8973800640 1 Processor -- start 2024-04-26T23:38:14.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.732+0000 7f8973800640 1 -- start start 2024-04-26T23:38:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.733+0000 7f8973800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f896c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.733+0000 7f8973800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f896c0d1470 0x7f896c0d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.733+0000 7f8973800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896c163e70 con 0x7f896c0d3da0 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.733+0000 7f8973800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f896c0d0aa0 con 0x7f896c0d1470 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.733+0000 7f8973800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f896c0d0170 con 0x7f896c15f480 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f8971800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f896c0d3da0 0x7f896c1630a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49538/0 (socket says 172.21.15.46:49538) 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f8971800640 1 -- 172.21.15.46:0/1632569591 learned_addr learned my addr 172.21.15.46:0/1632569591 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f896be00640 1 --2- 172.21.15.46:0/1632569591 >> v2:172.21.15.142:3300/0 conn(0x7f896c0d1470 0x7f896c0d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f8970e00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f896c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1715099337 0 0) 0x7f896c163e70 con 0x7f896c0d3da0 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.734+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f894c003520 con 0x7f896c0d3da0 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3533998968 0 0) 0x7f894c003520 con 0x7f896c0d3da0 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 msgr2=0x7f896c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f896c0d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 >> v2:172.21.15.142:3300/0 conn(0x7f896c0d1470 msgr2=0x7f896c0d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 --2- 172.21.15.46:0/1632569591 >> v2:172.21.15.142:3300/0 conn(0x7f896c0d1470 0x7f896c0d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c0d0450 con 0x7f896c0d3da0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8964001670 con 0x7f896c0d3da0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 >> v1:172.21.15.46:6789/0 conn(0x7f896c0d3da0 legacy=0x7f896c1630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894c0039b0 0x7f894c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f894c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894c004d40 0x7f894c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f896c0d0aa0 con 0x7f894c0039b0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f896c0d0170 con 0x7f894c004d40 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f894c00b9f0 con 0x7f896c15f480 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.735+0000 7f8970e00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894c0039b0 0x7f894c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f896be00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f894c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8971800640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894c004d40 0x7f894c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8970e00640 1 -- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 msgr2=0x7f894c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8970e00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f894c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8970e00640 1 -- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894c004d40 msgr2=0x7f894c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8970e00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894c004d40 0x7f894c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8970e00640 1 -- 172.21.15.46:0/1632569591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c003520 con 0x7f894c0039b0 2024-04-26T23:38:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f896be00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f896c15f480 0x7f894c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.736+0000 7f8971800640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894c004d40 0x7f894c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.737+0000 7f8970e00640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894c0039b0 0x7f894c003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f895c009ac0 tx=0x7f894c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6b2edc39f87c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.737+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c036070 con 0x7f894c0039b0 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.737+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c031400 con 0x7f894c0039b0 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.737+0000 7f896b400640 1 -- 172.21.15.46:0/1632569591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c009690 con 0x7f894c0039b0 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 -- 172.21.15.46:0/1632569591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894c0039b0 msgr2=0x7f894c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 --2- 172.21.15.46:0/1632569591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894c0039b0 0x7f894c003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f895c009ac0 tx=0x7f894c005830 comp rx=0 tx=0).stop 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8971800640 1 -- 172.21.15.46:0/1632569591 reap_dead start 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 -- 172.21.15.46:0/1632569591 shutdown_connections 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 -- 172.21.15.46:0/1632569591 >> 172.21.15.46:0/1632569591 conn(0x7f896c0d5070 msgr2=0x7f896c0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 -- 172.21.15.46:0/1632569591 shutdown_connections 2024-04-26T23:38:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.738+0000 7f8973800640 1 -- 172.21.15.46:0/1632569591 wait complete. 2024-04-26T23:38:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.740+0000 7f8973800640 1 Processor -- start 2024-04-26T23:38:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.740+0000 7f8973800640 1 -- start start 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f894c0039b0 0x7f896c1e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c0b9f00 0x7f896c0ba380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f896c15f860 con 0x7f896c1ea040 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f896c0cfe60 con 0x7f896c0b9f00 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f8973800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f896c15f9e0 con 0x7f894c0039b0 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f896be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f896be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58380/0 (socket says 172.21.15.46:58380) 2024-04-26T23:38:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.741+0000 7f896be00640 1 -- 172.21.15.46:0/504431267 learned_addr learned my addr 172.21.15.46:0/504431267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f8971800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c0b9f00 0x7f896c0ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 -- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f894c0039b0 msgr2=0x7f896c1e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f894c0039b0 0x7f896c1e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 -- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c0b9f00 msgr2=0x7f896c0ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c0b9f00 0x7f896c0ba380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 -- 172.21.15.46:0/504431267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c1e9c30 con 0x7f896c1ea040 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f896be00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f895400f400 tx=0x7f895400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954003990 con 0x7f896c1ea040 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8954003b30 con 0x7f896c1ea040 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954004020 con 0x7f896c1ea040 2024-04-26T23:38:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.742+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c1e90c0 con 0x7f896c1ea040 2024-04-26T23:38:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.743+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1e9670 con 0x7f896c1ea040 2024-04-26T23:38:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.744+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c08d8a0 con 0x7f896c1ea040 2024-04-26T23:38:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.745+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f895401d030 con 0x7f896c1ea040 2024-04-26T23:38:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.746+0000 7f8968c00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.746+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f895408ca80 con 0x7f896c1ea040 2024-04-26T23:38:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.746+0000 7f8970e00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.747+0000 7f8970e00640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 0x7f894806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f895c009ac0 tx=0x7f895c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.750+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f896c08d8a0 con 0x7f896c1ea040 2024-04-26T23:38:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.954+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f896c069530 con 0x7f896c1ea040 2024-04-26T23:38:14.959 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:14.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.955+0000 7f8968c00640 1 -- 172.21.15.46:0/504431267 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f895408ee20 con 0x7f896c1ea040 2024-04-26T23:38:14.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 msgr2=0x7f894806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 0x7f894806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f895c009ac0 tx=0x7f895c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 msgr2=0x7f896c15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f895400f400 tx=0x7f895400f8c0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 shutdown_connections 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c0b9f00 0x7f896c0ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896c1ea040 0x7f896c15a5d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 --2- 172.21.15.46:0/504431267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f894c0039b0 0x7f896c1e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 >> 172.21.15.46:0/504431267 conn(0x7f896c0d5070 msgr2=0x7f896c160b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 shutdown_connections 2024-04-26T23:38:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:14.958+0000 7f8973800640 1 -- 172.21.15.46:0/504431267 wait complete. 2024-04-26T23:38:14.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:15.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.193+0000 7f510fe00640 1 Processor -- start 2024-04-26T23:38:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.194+0000 7f510fe00640 1 -- start start 2024-04-26T23:38:15.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.195+0000 7f510fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f511006f680 0x7f51100ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.195+0000 7f510fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51100cff80 con 0x7f5110152ba0 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.195+0000 7f510fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f511015f920 con 0x7f5110152f80 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.195+0000 7f510fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5110160530 con 0x7f511006f680 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5110152ba0 0x7f51100cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49544/0 (socket says 172.21.15.46:49544) 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510ea00640 1 -- 172.21.15.46:0/3889035018 learned_addr learned my addr 172.21.15.46:0/3889035018 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510f400640 1 --2- 172.21.15.46:0/3889035018 >> v2:172.21.15.46:3300/0 conn(0x7f511006f680 0x7f51100ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510d600640 1 -- 172.21.15.46:0/3889035018 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309513300 0 0) 0x7f51100cff80 con 0x7f5110152ba0 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510d600640 1 -- 172.21.15.46:0/3889035018 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f4003520 con 0x7f5110152ba0 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510f400640 1 -- 172.21.15.46:0/3889035018 >> v1:172.21.15.142:6789/0 conn(0x7f5110152f80 legacy=0x7f511006ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.196+0000 7f510f400640 1 -- 172.21.15.46:0/3889035018 >> v1:172.21.15.46:6789/0 conn(0x7f5110152ba0 legacy=0x7f51100cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.197+0000 7f510f400640 1 -- 172.21.15.46:0/3889035018 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511015fc00 con 0x7f511006f680 2024-04-26T23:38:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510f400640 1 --2- 172.21.15.46:0/3889035018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f51100ccb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51101606b0 tx=0x7f510802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf84faa1a8e68e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510d600640 1 -- 172.21.15.46:0/3889035018 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108007740 con 0x7f511006f680 2024-04-26T23:38:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510d600640 1 -- 172.21.15.46:0/3889035018 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080078e0 con 0x7f511006f680 2024-04-26T23:38:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510d600640 1 -- 172.21.15.46:0/3889035018 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108007cb0 con 0x7f511006f680 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510fe00640 1 -- 172.21.15.46:0/3889035018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 msgr2=0x7f51100ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.198+0000 7f510fe00640 1 --2- 172.21.15.46:0/3889035018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f51100ccb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51101606b0 tx=0x7f510802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.199+0000 7f510fe00640 1 -- 172.21.15.46:0/3889035018 shutdown_connections 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.199+0000 7f510fe00640 1 --2- 172.21.15.46:0/3889035018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f51100ccb80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.199+0000 7f510fe00640 1 -- 172.21.15.46:0/3889035018 >> 172.21.15.46:0/3889035018 conn(0x7f51100b26d0 msgr2=0x7f51100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.199+0000 7f510fe00640 1 -- 172.21.15.46:0/3889035018 shutdown_connections 2024-04-26T23:38:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.199+0000 7f510fe00640 1 -- 172.21.15.46:0/3889035018 wait complete. 2024-04-26T23:38:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.200+0000 7f510fe00640 1 Processor -- start 2024-04-26T23:38:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.201+0000 7f510fe00640 1 -- start start 2024-04-26T23:38:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.201+0000 7f510fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.201+0000 7f510fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 0x7f51100d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5110152f80 0x7f51100d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51100cf4c0 con 0x7f511006f680 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51100cf340 con 0x7f5110152ba0 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51100cf640 con 0x7f5110152f80 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58388/0 (socket says 172.21.15.46:58388) 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 -- 172.21.15.46:0/3825174243 learned_addr learned my addr 172.21.15.46:0/3825174243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510e000640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 0x7f51100d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 -- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5110152f80 msgr2=0x7f51100d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5110152f80 0x7f51100d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 -- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 msgr2=0x7f51100d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 0x7f51100d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510ea00640 1 -- 172.21.15.46:0/3825174243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101f8bf0 con 0x7f511006f680 2024-04-26T23:38:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.202+0000 7f510e000640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 0x7f51100d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f510ea00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f510000c500 tx=0x7f510000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100017070 con 0x7f511006f680 2024-04-26T23:38:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100005ce0 con 0x7f511006f680 2024-04-26T23:38:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100007330 con 0x7f511006f680 2024-04-26T23:38:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51101f8ed0 con 0x7f511006f680 2024-04-26T23:38:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.203+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51100aedd0 con 0x7f511006f680 2024-04-26T23:38:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.204+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5110159a80 con 0x7f511006f680 2024-04-26T23:38:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.205+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f510001a070 con 0x7f511006f680 2024-04-26T23:38:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.206+0000 7f5106a00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 0x7f50f006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.206+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f510008b3a0 con 0x7f511006f680 2024-04-26T23:38:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.207+0000 7f510e000640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 0x7f50f006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.208+0000 7f510e000640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 0x7f50f006ee00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f511015ced0 tx=0x7f50fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.211+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5110159a80 con 0x7f511006f680 2024-04-26T23:38:15.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.418+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51100693b0 con 0x7f511006f680 2024-04-26T23:38:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.419+0000 7f5106a00640 1 -- 172.21.15.46:0/3825174243 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f5100059500 con 0x7f511006f680 2024-04-26T23:38:15.423 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.421+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 msgr2=0x7f50f006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.421+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 0x7f50f006ee00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f511015ced0 tx=0x7f50fc009040 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.421+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 msgr2=0x7f5110144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.421+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f510000c500 tx=0x7f510000c9c0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 shutdown_connections 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f50f006c940 0x7f50f006ee00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5110152f80 0x7f51100d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110152ba0 0x7f51100d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 --2- 172.21.15.46:0/3825174243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f511006f680 0x7f5110144600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 >> 172.21.15.46:0/3825174243 conn(0x7f51100b26d0 msgr2=0x7f51100c6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 shutdown_connections 2024-04-26T23:38:15.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.422+0000 7f510fe00640 1 -- 172.21.15.46:0/3825174243 wait complete. 2024-04-26T23:38:15.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.657+0000 7fee2b400640 1 Processor -- start 2024-04-26T23:38:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.658+0000 7fee2b400640 1 -- start start 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.659+0000 7fee2b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 0x7fee240b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.659+0000 7fee2b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee24071f10 0x7fee24072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee2b400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee24073120 con 0x7fee241597e0 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee2b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fee240aeff0 con 0x7fee24071f10 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee2b400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fee24159bc0 con 0x7fee240b5730 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee29400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee24071f10 0x7fee24072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee29400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee24071f10 0x7fee24072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58392/0 (socket says 172.21.15.46:58392) 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee29400640 1 -- 172.21.15.46:0/2484465339 learned_addr learned my addr 172.21.15.46:0/2484465339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.660+0000 7fee23e00640 1 --2- 172.21.15.46:0/2484465339 >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 0x7fee240b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.661+0000 7fee29400640 1 -- 172.21.15.46:0/2484465339 >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 msgr2=0x7fee240b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.661+0000 7fee29400640 1 --2- 172.21.15.46:0/2484465339 >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 0x7fee240b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.661+0000 7fee29400640 1 -- 172.21.15.46:0/2484465339 >> v1:172.21.15.142:6789/0 conn(0x7fee241597e0 legacy=0x7fee240ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.661+0000 7fee29400640 1 -- 172.21.15.46:0/2484465339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee24072b60 con 0x7fee24071f10 2024-04-26T23:38:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.661+0000 7fee23e00640 1 --2- 172.21.15.46:0/2484465339 >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 0x7fee240b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.662+0000 7fee29400640 1 --2- 172.21.15.46:0/2484465339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee24072340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee240743c0 tx=0x7fee1c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70d9dc221be3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.662+0000 7fee23400640 1 -- 172.21.15.46:0/2484465339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c037070 con 0x7fee24071f10 2024-04-26T23:38:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.662+0000 7fee23400640 1 -- 172.21.15.46:0/2484465339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c02ae50 con 0x7fee24071f10 2024-04-26T23:38:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.662+0000 7fee23400640 1 -- 172.21.15.46:0/2484465339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c032860 con 0x7fee24071f10 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 -- 172.21.15.46:0/2484465339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 msgr2=0x7fee24072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 --2- 172.21.15.46:0/2484465339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee24072340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee240743c0 tx=0x7fee1c02a300 comp rx=0 tx=0).stop 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 -- 172.21.15.46:0/2484465339 shutdown_connections 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 --2- 172.21.15.46:0/2484465339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee24072340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 --2- 172.21.15.46:0/2484465339 >> v2:172.21.15.142:3300/0 conn(0x7fee240b5730 0x7fee240b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 -- 172.21.15.46:0/2484465339 >> 172.21.15.46:0/2484465339 conn(0x7fee2406c980 msgr2=0x7fee240b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 -- 172.21.15.46:0/2484465339 shutdown_connections 2024-04-26T23:38:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.663+0000 7fee2b400640 1 -- 172.21.15.46:0/2484465339 wait complete. 2024-04-26T23:38:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.665+0000 7fee2b400640 1 Processor -- start 2024-04-26T23:38:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.665+0000 7fee2b400640 1 -- start start 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee240b5730 0x7fee240cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee241597e0 0x7fee2406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee240b43b0 con 0x7fee24071f10 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee240b40d0 con 0x7fee240b5730 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee2b400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee24072880 con 0x7fee241597e0 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee28a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee28a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58398/0 (socket says 172.21.15.46:58398) 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.666+0000 7fee28a00640 1 -- 172.21.15.46:0/3120863701 learned_addr learned my addr 172.21.15.46:0/3120863701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee23e00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee240b5730 0x7fee240cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 -- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee241597e0 msgr2=0x7fee2406f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee241597e0 0x7fee2406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 -- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee240b5730 msgr2=0x7fee240cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee240b5730 0x7fee240cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 -- 172.21.15.46:0/3120863701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2414aca0 con 0x7fee24071f10 2024-04-26T23:38:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee28a00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c730 tx=0x7fee1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14003b70 con 0x7fee24071f10 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee14003d30 con 0x7fee24071f10 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400aa90 con 0x7fee24071f10 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.667+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee240ce520 con 0x7fee24071f10 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.668+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee240ceae0 con 0x7fee24071f10 2024-04-26T23:38:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.669+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24073720 con 0x7fee24071f10 2024-04-26T23:38:15.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.670+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee1400ac30 con 0x7fee24071f10 2024-04-26T23:38:15.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.671+0000 7fee20c00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 0x7fee0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:15.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.671+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fee1408bf20 con 0x7fee24071f10 2024-04-26T23:38:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.671+0000 7fee23e00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 0x7fee0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.673+0000 7fee23e00640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 0x7fee0406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee24157570 tx=0x7fee10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:15.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.675+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee24073720 con 0x7fee24071f10 2024-04-26T23:38:15.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.878+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fee24069530 con 0x7fee24071f10 2024-04-26T23:38:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.879+0000 7fee20c00640 1 -- 172.21.15.46:0/3120863701 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fee2414aca0 con 0x7fee24071f10 2024-04-26T23:38:15.883 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 msgr2=0x7fee0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 0x7fee0406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee24157570 tx=0x7fee10008040 comp rx=0 tx=0).stop 2024-04-26T23:38:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 msgr2=0x7fee2414a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c730 tx=0x7fee1400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 shutdown_connections 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee0406c8f0 0x7fee0406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee241597e0 0x7fee2406f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee240b5730 0x7fee240cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 --2- 172.21.15.46:0/3120863701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee24071f10 0x7fee2414a760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 >> 172.21.15.46:0/3120863701 conn(0x7fee2406c980 msgr2=0x7fee24161b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 shutdown_connections 2024-04-26T23:38:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:15.882+0000 7fee2b400640 1 -- 172.21.15.46:0/3120863701 wait complete. 2024-04-26T23:38:15.921 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-26T23:38:15.921 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:38:15.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.137+0000 7f7828c00640 1 Processor -- start 2024-04-26T23:38:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.137+0000 7f7828c00640 1 -- start start 2024-04-26T23:38:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7828c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f782406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7828c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7824073ad0 con 0x7f78240720e0 2024-04-26T23:38:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7828c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7824146a90 con 0x7f7824160910 2024-04-26T23:38:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7828c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7824073d70 con 0x7f78241466b0 2024-04-26T23:38:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7822a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78240720e0 0x7f78240cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49556/0 (socket says 172.21.15.46:49556) 2024-04-26T23:38:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7822a00640 1 -- 172.21.15.46:0/1207766758 learned_addr learned my addr 172.21.15.46:0/1207766758 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7823400640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f782406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.139+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2469465389 0 0) 0x7f7824073ad0 con 0x7f78240720e0 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7808003520 con 0x7f78240720e0 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3399446604 0 0) 0x7f7808003520 con 0x7f78240720e0 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 msgr2=0x7f782406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f782406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 >> v1:172.21.15.142:6789/0 conn(0x7f7824160910 legacy=0x7f78240cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7824074050 con 0x7f78240720e0 2024-04-26T23:38:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7810001670 con 0x7f78240720e0 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 >> v1:172.21.15.46:6789/0 conn(0x7f78240720e0 legacy=0x7f78240cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f78080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7808003f60 0x7f7808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7808004d40 0x7f78080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7824073d70 con 0x7f7808004d40 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f780800baa0 con 0x7f7808003f60 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f780800bca0 con 0x7f78241466b0 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.140+0000 7f7823e00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7808004d40 0x7f78080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823400640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f78080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7822a00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7808003f60 0x7f7808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823e00640 1 -- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 msgr2=0x7f78080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823e00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78241466b0 0x7f78080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823e00640 1 -- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7808003f60 msgr2=0x7f7808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823e00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7808003f60 0x7f7808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7823e00640 1 -- 172.21.15.46:0/1207766758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7808003520 con 0x7f7808004d40 2024-04-26T23:38:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.141+0000 7f7822a00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7808003f60 0x7f7808004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.142+0000 7f7823e00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7808004d40 0x7f78080051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f781c001e50 tx=0x7f7808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b64af2f10ec7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.142+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c001a70 con 0x7f7808004d40 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.142+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c003970 con 0x7f7808004d40 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.142+0000 7f7822000640 1 -- 172.21.15.46:0/1207766758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c003d60 con 0x7f7808004d40 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 -- 172.21.15.46:0/1207766758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7808004d40 msgr2=0x7f78080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 --2- 172.21.15.46:0/1207766758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7808004d40 0x7f78080051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f781c001e50 tx=0x7f7808005830 comp rx=0 tx=0).stop 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7823e00640 1 -- 172.21.15.46:0/1207766758 reap_dead start 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 -- 172.21.15.46:0/1207766758 shutdown_connections 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 -- 172.21.15.46:0/1207766758 >> 172.21.15.46:0/1207766758 conn(0x7f78240b26d0 msgr2=0x7f78240b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 -- 172.21.15.46:0/1207766758 shutdown_connections 2024-04-26T23:38:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.143+0000 7f7828c00640 1 -- 172.21.15.46:0/1207766758 wait complete. 2024-04-26T23:38:16.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.145+0000 7f7828c00640 1 Processor -- start 2024-04-26T23:38:16.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.146+0000 7f7828c00640 1 -- start start 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78241f5ff0 0x7f78240c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78240cffa0 0x7f78240c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78240cc0a0 con 0x7f78240b1db0 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78240cb470 con 0x7f78241f5ff0 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7828c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78240cc220 con 0x7f78240cffa0 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7823400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7823400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58408/0 (socket says 172.21.15.46:58408) 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7823400640 1 -- 172.21.15.46:0/1372608438 learned_addr learned my addr 172.21.15.46:0/1372608438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.147+0000 7f7822a00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78241f5ff0 0x7f78240c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 -- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78240cffa0 msgr2=0x7f78240c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823e00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78240cffa0 0x7f78240c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78240cffa0 0x7f78240c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 -- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78241f5ff0 msgr2=0x7f78240c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78241f5ff0 0x7f78240c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 -- 172.21.15.46:0/1372608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78241f52d0 con 0x7f78240b1db0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7823400640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7814011600 tx=0x7f7814011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400f920 con 0x7f78240b1db0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781400fac0 con 0x7f78240b1db0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400fe70 con 0x7f78240b1db0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.148+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241f5550 con 0x7f78240b1db0 2024-04-26T23:38:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.149+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782413df00 con 0x7f78240b1db0 2024-04-26T23:38:16.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.150+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240693b0 con 0x7f78240b1db0 2024-04-26T23:38:16.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.151+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7814015460 con 0x7f78240b1db0 2024-04-26T23:38:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.152+0000 7f781b400640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 0x7f780406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.152+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f781408c960 con 0x7f78240b1db0 2024-04-26T23:38:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.153+0000 7f7822a00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 0x7f780406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.154+0000 7f7822a00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 0x7f780406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78240cab50 tx=0x7f7810009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.157+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78241f52d0 con 0x7f78240b1db0 2024-04-26T23:38:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.361+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78240b2440 con 0x7f78240b1db0 2024-04-26T23:38:16.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.365+0000 7f781b400640 1 -- 172.21.15.46:0/1372608438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f7814019030 con 0x7f78240b1db0 2024-04-26T23:38:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:16.369 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.367+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 msgr2=0x7f780406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.367+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 0x7f780406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78240cab50 tx=0x7f7810009040 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.367+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 msgr2=0x7f782413d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.367+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7814011600 tx=0x7f7814011ac0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 shutdown_connections 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f780406c8f0 0x7f780406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78240cffa0 0x7f78240c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78241f5ff0 0x7f78240c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 --2- 172.21.15.46:0/1372608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78240b1db0 0x7f782413d5f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 >> 172.21.15.46:0/1372608438 conn(0x7f78240b26d0 msgr2=0x7f78241444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 shutdown_connections 2024-04-26T23:38:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.368+0000 7f7828c00640 1 -- 172.21.15.46:0/1372608438 wait complete. 2024-04-26T23:38:16.392 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:16.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:16.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.606+0000 7f07fb200640 1 Processor -- start 2024-04-26T23:38:16.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.606+0000 7f07fb200640 1 -- start start 2024-04-26T23:38:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.608+0000 7f07fb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07fc1640d0 0x7f07fc1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.608+0000 7f07fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.608+0000 7f07fb200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07fc1649f0 con 0x7f07fc073af0 2024-04-26T23:38:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.608+0000 7f07fb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f07fc0cb4f0 con 0x7f07fc1640d0 2024-04-26T23:38:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.608+0000 7f07fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07fc071e80 con 0x7f07fc0ce7d0 2024-04-26T23:38:16.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.609+0000 7f07f9e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07fc1640d0 0x7f07fc1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.609+0000 7f07f9e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07fc1640d0 0x7f07fc1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:48666/0 (socket says 172.21.15.46:48666) 2024-04-26T23:38:16.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.609+0000 7f07f9e00640 1 -- 172.21.15.46:0/766395878 learned_addr learned my addr 172.21.15.46:0/766395878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:16.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.609+0000 7f07fa800640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.610+0000 7f07f9e00640 1 -- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 msgr2=0x7f07fc0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.610+0000 7f07f9e00640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.610+0000 7f07f9e00640 1 -- 172.21.15.46:0/766395878 >> v1:172.21.15.142:6789/0 conn(0x7f07fc073af0 legacy=0x7f07fc0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.610+0000 7f07f9e00640 1 -- 172.21.15.46:0/766395878 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc072160 con 0x7f07fc1640d0 2024-04-26T23:38:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.612+0000 7f07f9e00640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc1640d0 0x7f07fc1644b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009870 tx=0x7f07ec02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4808366686377d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.612+0000 7f07f8a00640 1 -- 172.21.15.46:0/766395878 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec037070 con 0x7f07fc1640d0 2024-04-26T23:38:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.612+0000 7f07f8a00640 1 -- 172.21.15.46:0/766395878 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec032440 con 0x7f07fc1640d0 2024-04-26T23:38:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.612+0000 7f07f8a00640 1 -- 172.21.15.46:0/766395878 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec032810 con 0x7f07fc1640d0 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 -- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc1640d0 msgr2=0x7f07fc1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc1640d0 0x7f07fc1644b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009870 tx=0x7f07ec02a250 comp rx=0 tx=0).stop 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 -- 172.21.15.46:0/766395878 shutdown_connections 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 --2- 172.21.15.46:0/766395878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc1640d0 0x7f07fc1644b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 -- 172.21.15.46:0/766395878 >> 172.21.15.46:0/766395878 conn(0x7f07fc0b27b0 msgr2=0x7f07fc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 -- 172.21.15.46:0/766395878 shutdown_connections 2024-04-26T23:38:16.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.613+0000 7f07fb200640 1 -- 172.21.15.46:0/766395878 wait complete. 2024-04-26T23:38:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.615+0000 7f07fb200640 1 Processor -- start 2024-04-26T23:38:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.615+0000 7f07fb200640 1 -- start start 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc073af0 0x7f07fc153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc0cc090 con 0x7f07fc1640d0 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc073f70 con 0x7f07fc073af0 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.616+0000 7f07fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07fc0cc210 con 0x7f07fc0ce7d0 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58420/0 (socket says 172.21.15.46:58420) 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 -- 172.21.15.46:0/4275759237 learned_addr learned my addr 172.21.15.46:0/4275759237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07f9e00640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc073af0 0x7f07fc153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07f9400640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 -- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 msgr2=0x7f07fc0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 -- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc073af0 msgr2=0x7f07fc153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc073af0 0x7f07fc153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07fa800640 1 -- 172.21.15.46:0/4275759237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc0b4360 con 0x7f07fc1640d0 2024-04-26T23:38:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.617+0000 7f07f9400640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07fa800640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d20 tx=0x7f07f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018020 con 0x7f07fc1640d0 2024-04-26T23:38:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4009b10 con 0x7f07fc1640d0 2024-04-26T23:38:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f40146c0 con 0x7f07fc1640d0 2024-04-26T23:38:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc0b4640 con 0x7f07fc1640d0 2024-04-26T23:38:16.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.618+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc0b4c00 con 0x7f07fc1640d0 2024-04-26T23:38:16.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.619+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07fc0693b0 con 0x7f07fc1640d0 2024-04-26T23:38:16.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.621+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07f4005d10 con 0x7f07fc1640d0 2024-04-26T23:38:16.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.621+0000 7f07f2000640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 0x7f07dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:16.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.622+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f07f408ca60 con 0x7f07fc1640d0 2024-04-26T23:38:16.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.622+0000 7f07f9e00640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 0x7f07dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:16.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.623+0000 7f07f9e00640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 0x7f07dc06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07ec034000 tx=0x7f07ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:16.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.627+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07f4022070 con 0x7f07fc1640d0 2024-04-26T23:38:16.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.831+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07fc0ce000 con 0x7f07fc1640d0 2024-04-26T23:38:16.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.832+0000 7f07f2000640 1 -- 172.21.15.46:0/4275759237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f07f4010040 con 0x7f07fc1640d0 2024-04-26T23:38:16.836 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 msgr2=0x7f07dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 0x7f07dc06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07ec034000 tx=0x7f07ec02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:16.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 msgr2=0x7f07fc161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:16.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d20 tx=0x7f07f400c490 comp rx=0 tx=0).stop 2024-04-26T23:38:16.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 shutdown_connections 2024-04-26T23:38:16.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07dc06c920 0x7f07dc06ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1640d0 0x7f07fc161fb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07fc0ce7d0 0x7f07fc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 --2- 172.21.15.46:0/4275759237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07fc073af0 0x7f07fc153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.836+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 >> 172.21.15.46:0/4275759237 conn(0x7f07fc0b27b0 msgr2=0x7f07fc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:16.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.837+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 shutdown_connections 2024-04-26T23:38:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:16.837+0000 7f07fb200640 1 -- 172.21.15.46:0/4275759237 wait complete. 2024-04-26T23:38:16.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.074+0000 7f5b9b600640 1 Processor -- start 2024-04-26T23:38:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.074+0000 7f5b9b600640 1 -- start start 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b9b600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5b94162aa0 0x7f5b940d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b9b600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b940d3b90 0x7f5b940d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b9b600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b940b46f0 con 0x7f5b940d5b90 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b9b600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5b940d11d0 con 0x7f5b940d3b90 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b9b600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5b940b44a0 con 0x7f5b94162aa0 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b93e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5b940d5b90 0x7f5b940b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49562/0 (socket says 172.21.15.46:49562) 2024-04-26T23:38:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b93e00640 1 -- 172.21.15.46:0/3188449696 learned_addr learned my addr 172.21.15.46:0/3188449696 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b99600640 1 --2- 172.21.15.46:0/3188449696 >> v2:172.21.15.46:3300/0 conn(0x7f5b940d3b90 0x7f5b940d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.076+0000 7f5b98c00640 1 --2- 172.21.15.46:0/3188449696 >> v2:172.21.15.142:3300/0 conn(0x7f5b94162aa0 0x7f5b940d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b93400640 1 -- 172.21.15.46:0/3188449696 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347751991 0 0) 0x7f5b940b46f0 con 0x7f5b940d5b90 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b93400640 1 -- 172.21.15.46:0/3188449696 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b78003520 con 0x7f5b940d5b90 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b99600640 1 -- 172.21.15.46:0/3188449696 >> v2:172.21.15.142:3300/0 conn(0x7f5b94162aa0 msgr2=0x7f5b940d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b99600640 1 --2- 172.21.15.46:0/3188449696 >> v2:172.21.15.142:3300/0 conn(0x7f5b94162aa0 0x7f5b940d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b99600640 1 -- 172.21.15.46:0/3188449696 >> v1:172.21.15.46:6789/0 conn(0x7f5b940d5b90 legacy=0x7f5b940b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.077+0000 7f5b99600640 1 -- 172.21.15.46:0/3188449696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b940b49d0 con 0x7f5b940d3b90 2024-04-26T23:38:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.078+0000 7f5b99600640 1 --2- 172.21.15.46:0/3188449696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b940d3b90 0x7f5b940d0c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b940b7bc0 tx=0x7f5b8c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca97bdd1b602735b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.078+0000 7f5b93400640 1 -- 172.21.15.46:0/3188449696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c037070 con 0x7f5b940d3b90 2024-04-26T23:38:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.078+0000 7f5b93400640 1 -- 172.21.15.46:0/3188449696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c02add0 con 0x7f5b940d3b90 2024-04-26T23:38:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.078+0000 7f5b93400640 1 -- 172.21.15.46:0/3188449696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c032640 con 0x7f5b940d3b90 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 -- 172.21.15.46:0/3188449696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b940d3b90 msgr2=0x7f5b940d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 --2- 172.21.15.46:0/3188449696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b940d3b90 0x7f5b940d0c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b940b7bc0 tx=0x7f5b8c02a080 comp rx=0 tx=0).stop 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 -- 172.21.15.46:0/3188449696 shutdown_connections 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 --2- 172.21.15.46:0/3188449696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b940d3b90 0x7f5b940d0c90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 --2- 172.21.15.46:0/3188449696 >> v2:172.21.15.142:3300/0 conn(0x7f5b94162aa0 0x7f5b940d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 -- 172.21.15.46:0/3188449696 >> 172.21.15.46:0/3188449696 conn(0x7f5b940d5070 msgr2=0x7f5b940d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 -- 172.21.15.46:0/3188449696 shutdown_connections 2024-04-26T23:38:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.079+0000 7f5b9b600640 1 -- 172.21.15.46:0/3188449696 wait complete. 2024-04-26T23:38:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.081+0000 7f5b9b600640 1 Processor -- start 2024-04-26T23:38:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.081+0000 7f5b9b600640 1 -- start start 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b940d3b90 0x7f5b940c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940d5b90 0x7f5b941f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b940d3fd0 con 0x7f5b94162aa0 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b940b5020 con 0x7f5b940d5b90 2024-04-26T23:38:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.082+0000 7f5b9b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b94162e80 con 0x7f5b940d3b90 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58452/0 (socket says 172.21.15.46:58452) 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 -- 172.21.15.46:0/676680291 learned_addr learned my addr 172.21.15.46:0/676680291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b93e00640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940d5b90 0x7f5b941f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b98c00640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b940d3b90 0x7f5b940c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 -- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b940d3b90 msgr2=0x7f5b940c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b940d3b90 0x7f5b940c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 -- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940d5b90 msgr2=0x7f5b941f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940d5b90 0x7f5b941f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.083+0000 7f5b99600640 1 -- 172.21.15.46:0/676680291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b940c90e0 con 0x7f5b94162aa0 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b99600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c035040 tx=0x7f5b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c037040 con 0x7f5b94162aa0 2024-04-26T23:38:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c042070 con 0x7f5b94162aa0 2024-04-26T23:38:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c03d640 con 0x7f5b94162aa0 2024-04-26T23:38:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b940c9380 con 0x7f5b94162aa0 2024-04-26T23:38:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.084+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b940c9940 con 0x7f5b94162aa0 2024-04-26T23:38:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.085+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b94069530 con 0x7f5b94162aa0 2024-04-26T23:38:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.086+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b8c03d7e0 con 0x7f5b94162aa0 2024-04-26T23:38:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.087+0000 7f5b90c00640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.087+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5b8c0ac630 con 0x7f5b94162aa0 2024-04-26T23:38:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.088+0000 7f5b98c00640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.089+0000 7f5b98c00640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 0x7f5b7406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b840073f0 tx=0x7f5b8400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.092+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b8c07a5c0 con 0x7f5b94162aa0 2024-04-26T23:38:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.300+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b940b6590 con 0x7f5b94162aa0 2024-04-26T23:38:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.301+0000 7f5b90c00640 1 -- 172.21.15.46:0/676680291 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f5b8c079ec0 con 0x7f5b94162aa0 2024-04-26T23:38:17.305 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 msgr2=0x7f5b7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 0x7f5b7406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b840073f0 tx=0x7f5b8400b040 comp rx=0 tx=0).stop 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 msgr2=0x7f5b940cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c035040 tx=0x7f5b8c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 shutdown_connections 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b94162aa0 0x7f5b940cc600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940d5b90 0x7f5b941f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 --2- 172.21.15.46:0/676680291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b940d3b90 0x7f5b940c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 >> 172.21.15.46:0/676680291 conn(0x7f5b940d5070 msgr2=0x7f5b940d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 shutdown_connections 2024-04-26T23:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.304+0000 7f5b9b600640 1 -- 172.21.15.46:0/676680291 wait complete. 2024-04-26T23:38:17.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:17.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.528+0000 7f198ca00640 1 Processor -- start 2024-04-26T23:38:17.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.528+0000 7f198ca00640 1 -- start start 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f198ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f19880d04c0 0x7f198806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f198ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 0x7f19880ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f198ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19880b4b30 con 0x7f198806f2e0 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f198ca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f19880b4e10 con 0x7f19880d04c0 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f198ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19880b50f0 con 0x7f19880b44f0 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f1985400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f198806f2e0 0x7f19880b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49570/0 (socket says 172.21.15.46:49570) 2024-04-26T23:38:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f1985400640 1 -- 172.21.15.46:0/1511808643 learned_addr learned my addr 172.21.15.46:0/1511808643 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f1985e00640 1 --2- 172.21.15.46:0/1511808643 >> v2:172.21.15.142:3300/0 conn(0x7f19880d04c0 0x7f198806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.530+0000 7f1986800640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 0x7f19880ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185237368 0 0) 0x7f19880b4b30 con 0x7f198806f2e0 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f196c003520 con 0x7f198806f2e0 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2201446745 0 0) 0x7f196c003520 con 0x7f198806f2e0 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 msgr2=0x7f19880ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 0x7f19880ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 >> v2:172.21.15.142:3300/0 conn(0x7f19880d04c0 msgr2=0x7f198806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 --2- 172.21.15.46:0/1511808643 >> v2:172.21.15.142:3300/0 conn(0x7f19880d04c0 0x7f198806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19880b53d0 con 0x7f198806f2e0 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1974001690 con 0x7f198806f2e0 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 >> v1:172.21.15.46:6789/0 conn(0x7f198806f2e0 legacy=0x7f19880b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 0x7f196c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f196c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c004d40 0x7f196c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19880b4e10 con 0x7f196c003f80 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19880b50f0 con 0x7f196c004d40 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f196c005990 con 0x7f19880b44f0 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.531+0000 7f1985400640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f196c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1986800640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c004d40 0x7f196c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1985400640 1 -- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 msgr2=0x7f196c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1985400640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19880b44f0 0x7f196c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1985400640 1 -- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c004d40 msgr2=0x7f196c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1985400640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c004d40 0x7f196c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1985400640 1 -- 172.21.15.46:0/1511808643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c003520 con 0x7f196c003f80 2024-04-26T23:38:17.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.532+0000 7f1986800640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c004d40 0x7f196c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.533+0000 7f1985400640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f196c004800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f196c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e266568dd02031 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.533+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974007ca0 con 0x7f196c003f80 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.533+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974005ad0 con 0x7f196c003f80 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.533+0000 7f1984a00640 1 -- 172.21.15.46:0/1511808643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974007070 con 0x7f196c003f80 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 -- 172.21.15.46:0/1511808643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 msgr2=0x7f196c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 --2- 172.21.15.46:0/1511808643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f196c004800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f196c005830 comp rx=0 tx=0).stop 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f1986800640 1 -- 172.21.15.46:0/1511808643 reap_dead start 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 -- 172.21.15.46:0/1511808643 shutdown_connections 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 -- 172.21.15.46:0/1511808643 >> 172.21.15.46:0/1511808643 conn(0x7f198806c980 msgr2=0x7f19880b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 -- 172.21.15.46:0/1511808643 shutdown_connections 2024-04-26T23:38:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.534+0000 7f198ca00640 1 -- 172.21.15.46:0/1511808643 wait complete. 2024-04-26T23:38:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.536+0000 7f198ca00640 1 Processor -- start 2024-04-26T23:38:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.536+0000 7f198ca00640 1 -- start start 2024-04-26T23:38:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1988140360 0x7f1988139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f198813c4d0 0x7f198813c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f198806de10 con 0x7f196c003f80 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f198806dc90 con 0x7f1988140360 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f198ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f198806df90 con 0x7f198813c4d0 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58466/0 (socket says 172.21.15.46:58466) 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 -- 172.21.15.46:0/1068544315 learned_addr learned my addr 172.21.15.46:0/1068544315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 -- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f198813c4d0 msgr2=0x7f198813c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f198813c4d0 0x7f198813c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 -- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1988140360 msgr2=0x7f1988139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985400640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1988140360 0x7f1988139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1988140360 0x7f1988139e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.537+0000 7f1985e00640 1 -- 172.21.15.46:0/1068544315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198813a970 con 0x7f196c003f80 2024-04-26T23:38:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f1985e00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f197800e860 tx=0x7f197800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978017070 con 0x7f196c003f80 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197800c8c0 con 0x7f196c003f80 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800cc90 con 0x7f196c003f80 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198813ac10 con 0x7f196c003f80 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.538+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198815d600 con 0x7f196c003f80 2024-04-26T23:38:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.539+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1988069530 con 0x7f196c003f80 2024-04-26T23:38:17.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.540+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f197801a070 con 0x7f196c003f80 2024-04-26T23:38:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.541+0000 7f197e000640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 0x7f196806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.541+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f197808bfe0 con 0x7f196c003f80 2024-04-26T23:38:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.542+0000 7f1985400640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 0x7f196806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.543+0000 7f1985400640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 0x7f196806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1974032000 tx=0x7f1974034040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.545+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1978059f70 con 0x7f196c003f80 2024-04-26T23:38:17.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.747+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1988144cb0 con 0x7f196c003f80 2024-04-26T23:38:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.748+0000 7f197e000640 1 -- 172.21.15.46:0/1068544315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f1978059870 con 0x7f196c003f80 2024-04-26T23:38:17.752 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 msgr2=0x7f196806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 0x7f196806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1974032000 tx=0x7f1974034040 comp rx=0 tx=0).stop 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 msgr2=0x7f19881589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f197800e860 tx=0x7f197800ed20 comp rx=0 tx=0).stop 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 shutdown_connections 2024-04-26T23:38:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196806c8f0 0x7f196806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f198813c4d0 0x7f198813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1988140360 0x7f1988139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 --2- 172.21.15.46:0/1068544315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196c003f80 0x7f19881589b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 >> 172.21.15.46:0/1068544315 conn(0x7f198806c980 msgr2=0x7f19880d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 shutdown_connections 2024-04-26T23:38:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.751+0000 7f198ca00640 1 -- 172.21.15.46:0/1068544315 wait complete. 2024-04-26T23:38:17.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.982+0000 7fae88e00640 1 Processor -- start 2024-04-26T23:38:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.982+0000 7fae88e00640 1 -- start start 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae88e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae8406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae88e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae840d07e0 con 0x7fae840b53b0 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae88e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8406eb90 con 0x7fae8415d730 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae88e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae840b4310 con 0x7fae8415e220 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae81400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fae840b53b0 0x7fae8415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49586/0 (socket says 172.21.15.46:49586) 2024-04-26T23:38:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae81400640 1 -- 172.21.15.46:0/3416733689 learned_addr learned my addr 172.21.15.46:0/3416733689 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:17.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.984+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190536798 0 0) 0x7fae840d07e0 con 0x7fae840b53b0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae82800640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae8406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae68003520 con 0x7fae840b53b0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2053772057 0 0) 0x7fae68003520 con 0x7fae840b53b0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 msgr2=0x7fae8406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae8406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 >> v1:172.21.15.142:6789/0 conn(0x7fae8415d730 legacy=0x7fae840d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae840b45f0 con 0x7fae840b53b0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fae70001670 con 0x7fae840b53b0 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 >> v1:172.21.15.46:6789/0 conn(0x7fae840b53b0 legacy=0x7fae8415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae68004060 0x7fae68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae68004d40 0x7fae680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae840b4310 con 0x7fae68004060 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae6800bc00 con 0x7fae68004d40 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae6800be00 con 0x7fae8415e220 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.985+0000 7fae81400640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae68004060 0x7fae68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae82800640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae68004d40 0x7fae680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81e00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81400640 1 -- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 msgr2=0x7fae680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81400640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae8415e220 0x7fae680039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81400640 1 -- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae68004d40 msgr2=0x7fae680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81400640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae68004d40 0x7fae680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.986+0000 7fae81400640 1 -- 172.21.15.46:0/3416733689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae68003520 con 0x7fae68004060 2024-04-26T23:38:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.987+0000 7fae81400640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae68004060 0x7fae68004800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c110 tx=0x7fae68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2661f0a738bc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.987+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70007c40 con 0x7fae68004060 2024-04-26T23:38:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.987+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae70005af0 con 0x7fae68004060 2024-04-26T23:38:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.987+0000 7fae80a00640 1 -- 172.21.15.46:0/3416733689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70007070 con 0x7fae68004060 2024-04-26T23:38:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 -- 172.21.15.46:0/3416733689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae68004060 msgr2=0x7fae68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 --2- 172.21.15.46:0/3416733689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae68004060 0x7fae68004800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c110 tx=0x7fae68005830 comp rx=0 tx=0).stop 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae82800640 1 -- 172.21.15.46:0/3416733689 reap_dead start 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 -- 172.21.15.46:0/3416733689 shutdown_connections 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 -- 172.21.15.46:0/3416733689 >> 172.21.15.46:0/3416733689 conn(0x7fae8406c980 msgr2=0x7fae840b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 -- 172.21.15.46:0/3416733689 shutdown_connections 2024-04-26T23:38:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.988+0000 7fae88e00640 1 -- 172.21.15.46:0/3416733689 wait complete. 2024-04-26T23:38:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.990+0000 7fae88e00640 1 Processor -- start 2024-04-26T23:38:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.990+0000 7fae88e00640 1 -- start start 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 0x7fae840cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae840afa60 0x7fae840afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae840b4e90 con 0x7fae841431c0 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae840b4bb0 con 0x7fae840afa60 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae88e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae840b3ec0 con 0x7fae68004060 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58476/0 (socket says 172.21.15.46:58476) 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.991+0000 7fae82800640 1 -- 172.21.15.46:0/1427332505 learned_addr learned my addr 172.21.15.46:0/1427332505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae81e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 0x7fae840cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae81400640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae840afa60 0x7fae840afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 -- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 msgr2=0x7fae840cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 0x7fae840cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 -- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae840afa60 msgr2=0x7fae840afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae840afa60 0x7fae840afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 -- 172.21.15.46:0/1427332505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84143f20 con 0x7fae841431c0 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae81e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 0x7fae840cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae82800640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae840cf170 tx=0x7fae7c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c019070 con 0x7fae841431c0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c007c90 con 0x7fae841431c0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c014640 con 0x7fae841431c0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.992+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae84144200 con 0x7fae841431c0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.993+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841f5f40 con 0x7fae841431c0 2024-04-26T23:38:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.994+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae840b0420 con 0x7fae841431c0 2024-04-26T23:38:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.995+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae7c01c070 con 0x7fae841431c0 2024-04-26T23:38:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.996+0000 7fae7a000640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 0x7fae6406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.996+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fae7c08b910 con 0x7fae841431c0 2024-04-26T23:38:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.996+0000 7fae81e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 0x7fae6406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:17.998+0000 7fae81e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 0x7fae6406ee30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae74007fd0 tx=0x7fae74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.000+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae7c059ab0 con 0x7fae841431c0 2024-04-26T23:38:18.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.209+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fae840b1720 con 0x7fae841431c0 2024-04-26T23:38:18.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.210+0000 7fae7a000640 1 -- 172.21.15.46:0/1427332505 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fae7c0593b0 con 0x7fae841431c0 2024-04-26T23:38:18.214 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 msgr2=0x7fae6406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 0x7fae6406ee30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae74007fd0 tx=0x7fae74009040 comp rx=0 tx=0).stop 2024-04-26T23:38:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 msgr2=0x7fae84143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae840cf170 tx=0x7fae7c009d30 comp rx=0 tx=0).stop 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 shutdown_connections 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae6406c970 0x7fae6406ee30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae841431c0 0x7fae84143640 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae840afa60 0x7fae840afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 --2- 172.21.15.46:0/1427332505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae68004060 0x7fae840cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 >> 172.21.15.46:0/1427332505 conn(0x7fae8406c980 msgr2=0x7fae840b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 shutdown_connections 2024-04-26T23:38:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.212+0000 7fae88e00640 1 -- 172.21.15.46:0/1427332505 wait complete. 2024-04-26T23:38:18.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-26T23:38:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.440+0000 7f224b200640 1 Processor -- start 2024-04-26T23:38:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.440+0000 7f224b200640 1 -- start start 2024-04-26T23:38:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f224c072c30 0x7f224c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c0b41b0 con 0x7f224c15bf50 2024-04-26T23:38:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f224c15c330 con 0x7f224c072c30 2024-04-26T23:38:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0b4490 con 0x7f224c06f8f0 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f224c072c30 0x7f224c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f2243e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f224c072c30 0x7f224c0730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:48746/0 (socket says 172.21.15.46:48746) 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.442+0000 7f224a800640 1 -- 172.21.15.46:0/2021461733 learned_addr learned my addr 172.21.15.46:0/2021461733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.443+0000 7f224a800640 1 -- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 msgr2=0x7f224c0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.443+0000 7f224a800640 1 --2- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.443+0000 7f224a800640 1 -- 172.21.15.46:0/2021461733 >> v1:172.21.15.142:6789/0 conn(0x7f224c15bf50 legacy=0x7f224c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.443+0000 7f224a800640 1 -- 172.21.15.46:0/2021461733 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0b4770 con 0x7f224c072c30 2024-04-26T23:38:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.445+0000 7f224a800640 1 --2- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0730c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f224402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab73f899e7f7d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f2249400640 1 -- 172.21.15.46:0/2021461733 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244036070 con 0x7f224c072c30 2024-04-26T23:38:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f2249400640 1 -- 172.21.15.46:0/2021461733 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244031600 con 0x7f224c072c30 2024-04-26T23:38:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f2249400640 1 -- 172.21.15.46:0/2021461733 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440319d0 con 0x7f224c072c30 2024-04-26T23:38:18.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 -- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 msgr2=0x7f224c0730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 --2- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0730c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f224402a730 comp rx=0 tx=0).stop 2024-04-26T23:38:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 -- 172.21.15.46:0/2021461733 shutdown_connections 2024-04-26T23:38:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 --2- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 --2- 172.21.15.46:0/2021461733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0730c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 -- 172.21.15.46:0/2021461733 >> 172.21.15.46:0/2021461733 conn(0x7f224c0b2820 msgr2=0x7f224c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 -- 172.21.15.46:0/2021461733 shutdown_connections 2024-04-26T23:38:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.446+0000 7f224b200640 1 -- 172.21.15.46:0/2021461733 wait complete. 2024-04-26T23:38:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.448+0000 7f224b200640 1 Processor -- start 2024-04-26T23:38:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.448+0000 7f224b200640 1 -- start start 2024-04-26T23:38:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0b5250 con 0x7f224c15bf50 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0739f0 con 0x7f224c072c30 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.449+0000 7f224b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0b53d0 con 0x7f224c06f8f0 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58480/0 (socket says 172.21.15.46:58480) 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 -- 172.21.15.46:0/4207546585 learned_addr learned my addr 172.21.15.46:0/4207546585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f2243e00640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f2249e00640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 -- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 msgr2=0x7f224c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 -- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 msgr2=0x7f224c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 -- 172.21.15.46:0/4207546585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c15d1e0 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f2243e00640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.450+0000 7f224a800640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2244009f70 tx=0x7f2244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.451+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244041020 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.451+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244008730 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.451+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244036040 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.451+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c15d4c0 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.451+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c15da80 con 0x7f224c15bf50 2024-04-26T23:38:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.452+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c0693b0 con 0x7f224c15bf50 2024-04-26T23:38:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.453+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2244044020 con 0x7f224c15bf50 2024-04-26T23:38:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.454+0000 7f2242000640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.454+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f22440aca10 con 0x7f224c15bf50 2024-04-26T23:38:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.455+0000 7f2249e00640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:18.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.456+0000 7f2249e00640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 0x7f222806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f224c157f60 tx=0x7f223c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:18.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.459+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f224404b030 con 0x7f224c15bf50 2024-04-26T23:38:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:18.737+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) -- 0x7f224c0aea80 con 0x7f224c15bf50 2024-04-26T23:38:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.285+0000 7f2242000640 1 -- 172.21.15.46:0/4207546585 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v82) ==== 151+0+0 (secure 0 0 0) 0x7f224407aad0 con 0x7f224c15bf50 2024-04-26T23:38:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.289+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 msgr2=0x7f222806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.289+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 0x7f222806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f224c157f60 tx=0x7f223c009040 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.289+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 msgr2=0x7f224c0b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.289+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2244009f70 tx=0x7f2244004a40 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 shutdown_connections 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c15bf50 0x7f224c0b1be0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c072c30 0x7f224c0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 --2- 172.21.15.46:0/4207546585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f224c06f8f0 0x7f224c0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 >> 172.21.15.46:0/4207546585 conn(0x7f224c0b2820 msgr2=0x7f224c0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 shutdown_connections 2024-04-26T23:38:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.290+0000 7f224b200640 1 -- 172.21.15.46:0/4207546585 wait complete. 2024-04-26T23:38:19.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:38:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.537+0000 7fce5e800640 1 Processor -- start 2024-04-26T23:38:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.537+0000 7fce5e800640 1 -- start start 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce580cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5e800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580af570 con 0x7fce5815f210 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580af850 con 0x7fce580ae8f0 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce580cb680 con 0x7fce580cdd20 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce5815f210 0x7fce580cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49592/0 (socket says 172.21.15.46:49592) 2024-04-26T23:38:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5d400640 1 -- 172.21.15.46:0/1027757740 learned_addr learned my addr 172.21.15.46:0/1027757740 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.539+0000 7fce5ca00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce580cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258707628 0 0) 0x7fce580af570 con 0x7fce5815f210 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce40003520 con 0x7fce5815f210 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1954683744 0 0) 0x7fce40003520 con 0x7fce5815f210 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 msgr2=0x7fce580cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce580cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 >> v1:172.21.15.142:6789/0 conn(0x7fce580ae8f0 legacy=0x7fce580aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580cb960 con 0x7fce5815f210 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fce4c0019f0 con 0x7fce5815f210 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 >> v1:172.21.15.46:6789/0 conn(0x7fce5815f210 legacy=0x7fce580cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce400039e0 0x7fce40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce40004d40 0x7fce400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce580cb680 con 0x7fce400039e0 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce4000ba70 con 0x7fce40004d40 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.540+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce4000bc70 con 0x7fce580cdd20 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce400039e0 0x7fce40003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5ca00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5de00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce40004d40 0x7fce400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 -- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 msgr2=0x7fce40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce580cdd20 0x7fce40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 -- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce40004d40 msgr2=0x7fce400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce40004d40 0x7fce400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5d400640 1 -- 172.21.15.46:0/1027757740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce40003520 con 0x7fce400039e0 2024-04-26T23:38:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.541+0000 7fce5de00640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce40004d40 0x7fce400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.542+0000 7fce5d400640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce400039e0 0x7fce40003e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce4c002230 tx=0x7fce40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3e245c8093a0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00a4a0 con 0x7fce400039e0 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c02f070 con 0x7fce400039e0 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce57e00640 1 -- 172.21.15.46:0/1027757740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c02bca0 con 0x7fce400039e0 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 -- 172.21.15.46:0/1027757740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce400039e0 msgr2=0x7fce40003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 --2- 172.21.15.46:0/1027757740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce400039e0 0x7fce40003e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce4c002230 tx=0x7fce40005830 comp rx=0 tx=0).stop 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5de00640 1 -- 172.21.15.46:0/1027757740 reap_dead start 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 -- 172.21.15.46:0/1027757740 shutdown_connections 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 -- 172.21.15.46:0/1027757740 >> 172.21.15.46:0/1027757740 conn(0x7fce580b27b0 msgr2=0x7fce580b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 -- 172.21.15.46:0/1027757740 shutdown_connections 2024-04-26T23:38:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.543+0000 7fce5e800640 1 -- 172.21.15.46:0/1027757740 wait complete. 2024-04-26T23:38:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.545+0000 7fce5e800640 1 Processor -- start 2024-04-26T23:38:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.545+0000 7fce5e800640 1 -- start start 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce400039e0 0x7fce580c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce58151010 0x7fce580c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce580cc160 con 0x7fce580c9550 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce5815f5f0 con 0x7fce58151010 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.546+0000 7fce5e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce580cc2e0 con 0x7fce400039e0 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58488/0 (socket says 172.21.15.46:58488) 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 -- 172.21.15.46:0/3779417785 learned_addr learned my addr 172.21.15.46:0/3779417785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5d400640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce400039e0 0x7fce580c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5de00640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce58151010 0x7fce580c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 -- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce400039e0 msgr2=0x7fce580c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce400039e0 0x7fce580c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 -- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce58151010 msgr2=0x7fce580c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce58151010 0x7fce580c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.547+0000 7fce5ca00640 1 -- 172.21.15.46:0/3779417785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580c8210 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce5ca00640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce44011600 tx=0x7fce44011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4400f920 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4400fac0 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4400fe70 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce580c84b0 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.548+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce580c8a70 con 0x7fce580c9550 2024-04-26T23:38:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.549+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce580693b0 con 0x7fce580c9550 2024-04-26T23:38:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.550+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce44015460 con 0x7fce580c9550 2024-04-26T23:38:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.552+0000 7fce55600640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 0x7fce3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.552+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fce4408c870 con 0x7fce580c9550 2024-04-26T23:38:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.552+0000 7fce5d400640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 0x7fce3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.553+0000 7fce5d400640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 0x7fce3806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce4c033000 tx=0x7fce4c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.557+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce44019030 con 0x7fce580c9550 2024-04-26T23:38:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.767+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fce5815d870 con 0x7fce580c9550 2024-04-26T23:38:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.774+0000 7fce55600640 1 -- 172.21.15.46:0/3779417785 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fce4405a800 con 0x7fce580c9550 2024-04-26T23:38:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 msgr2=0x7fce3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 0x7fce3806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce4c033000 tx=0x7fce4c009030 comp rx=0 tx=0).stop 2024-04-26T23:38:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 msgr2=0x7fce580c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce44011600 tx=0x7fce44011ac0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 shutdown_connections 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3806c920 0x7fce3806ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce58151010 0x7fce580c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce580c9550 0x7fce580c7cd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 --2- 172.21.15.46:0/3779417785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce400039e0 0x7fce580c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 >> 172.21.15.46:0/3779417785 conn(0x7fce580b27b0 msgr2=0x7fce580b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 shutdown_connections 2024-04-26T23:38:19.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:19.777+0000 7fce5e800640 1 -- 172.21.15.46:0/3779417785 wait complete. 2024-04-26T23:38:19.800 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:38:19.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:38:19.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:38:20.023 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:38:20.024 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:20.024 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:20.085 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:20.085 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:20.141 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:20.141 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:20.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.404+0000 7f5dd7200640 1 Processor -- start 2024-04-26T23:38:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.404+0000 7f5dd7200640 1 -- start start 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 0x7f5dd815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd8152480 0x7f5dd8074010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd80cfe20 0x7f5dd80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd8152910 con 0x7f5dd8152480 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd815c060 con 0x7f5dd81520a0 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd815c340 con 0x7f5dd80cfe20 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd8152480 0x7f5dd8074010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd8152480 0x7f5dd8074010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58496/0 (socket says 172.21.15.46:58496) 2024-04-26T23:38:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd5400640 1 -- 172.21.15.46:0/3856014058 learned_addr learned my addr 172.21.15.46:0/3856014058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd6800640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd80cfe20 0x7f5dd80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.406+0000 7f5dd5e00640 1 --2- 172.21.15.46:0/3856014058 >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 0x7f5dd815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5400640 1 -- 172.21.15.46:0/3856014058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd80cfe20 msgr2=0x7f5dd80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd80cfe20 0x7f5dd80728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5400640 1 -- 172.21.15.46:0/3856014058 >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 msgr2=0x7f5dd815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3856014058 >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 0x7f5dd815ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5400640 1 -- 172.21.15.46:0/3856014058 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd815c620 con 0x7f5dd8152480 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.407+0000 7f5dd5e00640 1 --2- 172.21.15.46:0/3856014058 >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 0x7f5dd815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.408+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd8152480 0x7f5dd8074010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009870 tx=0x7f5dc402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aadf3e8da196b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.408+0000 7f5dd4a00640 1 -- 172.21.15.46:0/3856014058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4037070 con 0x7f5dd8152480 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.408+0000 7f5dd4a00640 1 -- 172.21.15.46:0/3856014058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc4032440 con 0x7f5dd8152480 2024-04-26T23:38:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.408+0000 7f5dd4a00640 1 -- 172.21.15.46:0/3856014058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032810 con 0x7f5dd8152480 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 -- 172.21.15.46:0/3856014058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd8152480 msgr2=0x7f5dd8074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd8152480 0x7f5dd8074010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009870 tx=0x7f5dc402a200 comp rx=0 tx=0).stop 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 -- 172.21.15.46:0/3856014058 shutdown_connections 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd80cfe20 0x7f5dd80728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3856014058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd8152480 0x7f5dd8074010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3856014058 >> v2:172.21.15.142:3300/0 conn(0x7f5dd81520a0 0x7f5dd815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 -- 172.21.15.46:0/3856014058 >> 172.21.15.46:0/3856014058 conn(0x7f5dd806e3c0 msgr2=0x7f5dd806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 -- 172.21.15.46:0/3856014058 shutdown_connections 2024-04-26T23:38:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.409+0000 7f5dd7200640 1 -- 172.21.15.46:0/3856014058 wait complete. 2024-04-26T23:38:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.411+0000 7f5dd7200640 1 Processor -- start 2024-04-26T23:38:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.411+0000 7f5dd7200640 1 -- start start 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd80cfe20 0x7f5dd8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd8152480 0x7f5dd81f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8074550 con 0x7f5dd81520a0 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8072390 con 0x7f5dd80cfe20 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd80d02b0 con 0x7f5dd8152480 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58512/0 (socket says 172.21.15.46:58512) 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.412+0000 7f5dd5400640 1 -- 172.21.15.46:0/3471200464 learned_addr learned my addr 172.21.15.46:0/3471200464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 -- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd8152480 msgr2=0x7f5dd81f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd8152480 0x7f5dd81f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 -- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd80cfe20 msgr2=0x7f5dd8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5e00640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd80cfe20 0x7f5dd8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd80cfe20 0x7f5dd8151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 -- 172.21.15.46:0/3471200464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd8155cb0 con 0x7f5dd81520a0 2024-04-26T23:38:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.413+0000 7f5dd5400640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4000c00 tx=0x7f5dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.414+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4041020 con 0x7f5dd81520a0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.414+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc40306c0 con 0x7f5dd81520a0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.414+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4037040 con 0x7f5dd81520a0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.414+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd80b22e0 con 0x7f5dd81520a0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.414+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd8151b70 con 0x7f5dd81520a0 2024-04-26T23:38:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.415+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd80b6220 con 0x7f5dd81520a0 2024-04-26T23:38:20.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.416+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dc4044020 con 0x7f5dd81520a0 2024-04-26T23:38:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.417+0000 7f5dce000640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 0x7f5db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.417+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5dc40ac740 con 0x7f5dd81520a0 2024-04-26T23:38:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.418+0000 7f5dd5e00640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 0x7f5db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.419+0000 7f5dd5e00640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 0x7f5db806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8008850 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.422+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5dd80b6220 con 0x7f5dd81520a0 2024-04-26T23:38:20.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.630+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dd80693b0 con 0x7f5dd81520a0 2024-04-26T23:38:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.631+0000 7f5dce000640 1 -- 172.21.15.46:0/3471200464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f5dc404b030 con 0x7f5dd81520a0 2024-04-26T23:38:20.636 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:20.636 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 msgr2=0x7f5db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 0x7f5db806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8008850 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 msgr2=0x7f5dd81f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4000c00 tx=0x7f5dc40046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 shutdown_connections 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5db806c8f0 0x7f5db806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dd8152480 0x7f5dd81f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd81520a0 0x7f5dd81f8980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 --2- 172.21.15.46:0/3471200464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd80cfe20 0x7f5dd8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 >> 172.21.15.46:0/3471200464 conn(0x7f5dd806e3c0 msgr2=0x7f5dd806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 shutdown_connections 2024-04-26T23:38:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.634+0000 7f5dd7200640 1 -- 172.21.15.46:0/3471200464 wait complete. 2024-04-26T23:38:20.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:20.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:20.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.855+0000 7fcca2a00640 1 Processor -- start 2024-04-26T23:38:20.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.856+0000 7fcca2a00640 1 -- start start 2024-04-26T23:38:20.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.857+0000 7fcca2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc9c15dd60 0x7fcc9c163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.857+0000 7fcca2a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 0x7fcc9c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.857+0000 7fcca2a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c072880 con 0x7fcc9c071f80 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.857+0000 7fcca2a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9c15e140 con 0x7fcc9c15dd60 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.857+0000 7fcca2a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9c072b40 con 0x7fcc9c0723e0 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc9c15dd60 0x7fcc9c163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcc9c071f80 0x7fcc9c0b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49608/0 (socket says 172.21.15.46:49608) 2024-04-26T23:38:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9b400640 1 -- 172.21.15.46:0/1004526332 learned_addr learned my addr 172.21.15.46:0/1004526332 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9be00640 1 -- 172.21.15.46:0/1004526332 >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 msgr2=0x7fcc9c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcca0a00640 1 --2- 172.21.15.46:0/1004526332 >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 0x7fcc9c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9aa00640 1 -- 172.21.15.46:0/1004526332 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738296353 0 0) 0x7fcc9c072880 con 0x7fcc9c071f80 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9be00640 1 --2- 172.21.15.46:0/1004526332 >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 0x7fcc9c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9be00640 1 -- 172.21.15.46:0/1004526332 >> v1:172.21.15.46:6789/0 conn(0x7fcc9c071f80 legacy=0x7fcc9c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcc9be00640 1 -- 172.21.15.46:0/1004526332 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c072e20 con 0x7fcc9c15dd60 2024-04-26T23:38:20.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.858+0000 7fcca0a00640 1 --2- 172.21.15.46:0/1004526332 >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 0x7fcc9c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.859+0000 7fcc9be00640 1 --2- 172.21.15.46:0/1004526332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c15dd60 0x7fcc9c163bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b3010 tx=0x7fcc8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9ee859b6d93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcc9aa00640 1 -- 172.21.15.46:0/1004526332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c037070 con 0x7fcc9c15dd60 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcc9aa00640 1 -- 172.21.15.46:0/1004526332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c032440 con 0x7fcc9c15dd60 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcc9aa00640 1 -- 172.21.15.46:0/1004526332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c032810 con 0x7fcc9c15dd60 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 -- 172.21.15.46:0/1004526332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c15dd60 msgr2=0x7fcc9c163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 --2- 172.21.15.46:0/1004526332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c15dd60 0x7fcc9c163bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b3010 tx=0x7fcc8c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 -- 172.21.15.46:0/1004526332 shutdown_connections 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 --2- 172.21.15.46:0/1004526332 >> v2:172.21.15.142:3300/0 conn(0x7fcc9c0723e0 0x7fcc9c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 --2- 172.21.15.46:0/1004526332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c15dd60 0x7fcc9c163bc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 -- 172.21.15.46:0/1004526332 >> 172.21.15.46:0/1004526332 conn(0x7fcc9c06c980 msgr2=0x7fcc9c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:20.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.860+0000 7fcca2a00640 1 -- 172.21.15.46:0/1004526332 shutdown_connections 2024-04-26T23:38:20.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.861+0000 7fcca2a00640 1 -- 172.21.15.46:0/1004526332 wait complete. 2024-04-26T23:38:20.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.862+0000 7fcca2a00640 1 Processor -- start 2024-04-26T23:38:20.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.863+0000 7fcca2a00640 1 -- start start 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.863+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c071f80 0x7fcc9c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.863+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.863+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc9c15dd60 0x7fcc9c161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcca2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c0b42c0 con 0x7fcc9c0723e0 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcca2a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c0733e0 con 0x7fcc9c071f80 2024-04-26T23:38:20.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcca2a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9c0b4440 con 0x7fcc9c15dd60 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58518/0 (socket says 172.21.15.46:58518) 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 -- 172.21.15.46:0/928993645 learned_addr learned my addr 172.21.15.46:0/928993645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9be00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c071f80 0x7fcc9c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 -- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc9c15dd60 msgr2=0x7fcc9c161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc9c15dd60 0x7fcc9c161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 -- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c071f80 msgr2=0x7fcc9c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c071f80 0x7fcc9c14b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.864+0000 7fcc9b400640 1 -- 172.21.15.46:0/928993645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c14e990 con 0x7fcc9c0723e0 2024-04-26T23:38:20.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcc9b400640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8c0 tx=0x7fcc8400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84018070 con 0x7fcc9c0723e0 2024-04-26T23:38:20.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8400a640 con 0x7fcc9c0723e0 2024-04-26T23:38:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400aa30 con 0x7fcc9c0723e0 2024-04-26T23:38:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c14ec70 con 0x7fcc9c0723e0 2024-04-26T23:38:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.865+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c0af360 con 0x7fcc9c0723e0 2024-04-26T23:38:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.866+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9c069530 con 0x7fcc9c0723e0 2024-04-26T23:38:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.867+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc8400abd0 con 0x7fcc9c0723e0 2024-04-26T23:38:20.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.868+0000 7fcc93e00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 0x7fcc7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:20.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.869+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcc8408b6f0 con 0x7fcc9c0723e0 2024-04-26T23:38:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.869+0000 7fcc9be00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 0x7fcc7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.870+0000 7fcc9be00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 0x7fcc7806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c034000 tx=0x7fcc8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:20.873+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc84059690 con 0x7fcc9c0723e0 2024-04-26T23:38:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.080+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc9c0b36f0 con 0x7fcc9c0723e0 2024-04-26T23:38:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.081+0000 7fcc93e00640 1 -- 172.21.15.46:0/928993645 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fcc84058f90 con 0x7fcc9c0723e0 2024-04-26T23:38:21.086 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:21.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 msgr2=0x7fcc7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 0x7fcc7806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c034000 tx=0x7fcc8c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:21.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 msgr2=0x7fcc9c14bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8c0 tx=0x7fcc8400cd80 comp rx=0 tx=0).stop 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 shutdown_connections 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcc7806c8f0 0x7fcc7806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc9c15dd60 0x7fcc9c161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc9c0723e0 0x7fcc9c14bcf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 --2- 172.21.15.46:0/928993645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c071f80 0x7fcc9c14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 >> 172.21.15.46:0/928993645 conn(0x7fcc9c06c980 msgr2=0x7fcc9c06ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 shutdown_connections 2024-04-26T23:38:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.084+0000 7fcca2a00640 1 -- 172.21.15.46:0/928993645 wait complete. 2024-04-26T23:38:21.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.309+0000 7fa522200640 1 Processor -- start 2024-04-26T23:38:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.309+0000 7fa522200640 1 -- start start 2024-04-26T23:38:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa51c15d730 0x7fa51c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa51c0b53b0 0x7fa51c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15e050 0x7fa51c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa51c06eb90 con 0x7fa51c0b53b0 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa51c0b4310 con 0x7fa51c15d730 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.311+0000 7fa522200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa51c0b45f0 con 0x7fa51c15e050 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa51c0b53b0 0x7fa51c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa51c0b53b0 0x7fa51c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58530/0 (socket says 172.21.15.46:58530) 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 -- 172.21.15.46:0/3261475042 learned_addr learned my addr 172.21.15.46:0/3261475042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51b400640 1 --2- 172.21.15.46:0/3261475042 >> v2:172.21.15.142:3300/0 conn(0x7fa51c15d730 0x7fa51c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51be00640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15e050 0x7fa51c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 -- 172.21.15.46:0/3261475042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15e050 msgr2=0x7fa51c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15e050 0x7fa51c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 -- 172.21.15.46:0/3261475042 >> v2:172.21.15.142:3300/0 conn(0x7fa51c15d730 msgr2=0x7fa51c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 --2- 172.21.15.46:0/3261475042 >> v2:172.21.15.142:3300/0 conn(0x7fa51c15d730 0x7fa51c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.312+0000 7fa51aa00640 1 -- 172.21.15.46:0/3261475042 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c0b48d0 con 0x7fa51c0b53b0 2024-04-26T23:38:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.313+0000 7fa51aa00640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c0b53b0 0x7fa51c15db10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa51c06fa10 tx=0x7fa50802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd134452038b8b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.314+0000 7fa51a000640 1 -- 172.21.15.46:0/3261475042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508037070 con 0x7fa51c0b53b0 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.314+0000 7fa51a000640 1 -- 172.21.15.46:0/3261475042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508032440 con 0x7fa51c0b53b0 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.314+0000 7fa51a000640 1 -- 172.21.15.46:0/3261475042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508032810 con 0x7fa51c0b53b0 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.314+0000 7fa522200640 1 -- 172.21.15.46:0/3261475042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c0b53b0 msgr2=0x7fa51c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.314+0000 7fa522200640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c0b53b0 0x7fa51c15db10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa51c06fa10 tx=0x7fa50802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 -- 172.21.15.46:0/3261475042 shutdown_connections 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15e050 0x7fa51c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 --2- 172.21.15.46:0/3261475042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c0b53b0 0x7fa51c15db10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 --2- 172.21.15.46:0/3261475042 >> v2:172.21.15.142:3300/0 conn(0x7fa51c15d730 0x7fa51c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 -- 172.21.15.46:0/3261475042 >> 172.21.15.46:0/3261475042 conn(0x7fa51c06c980 msgr2=0x7fa51c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 -- 172.21.15.46:0/3261475042 shutdown_connections 2024-04-26T23:38:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.315+0000 7fa522200640 1 -- 172.21.15.46:0/3261475042 wait complete. 2024-04-26T23:38:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.317+0000 7fa522200640 1 Processor -- start 2024-04-26T23:38:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.317+0000 7fa522200640 1 -- start start 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa51c0b53b0 0x7fa51c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 0x7fa51c136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c15e460 con 0x7fa51c15e050 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c0b4e90 con 0x7fa51c0b53b0 2024-04-26T23:38:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa522200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa51c0b3ec0 con 0x7fa51c15d730 2024-04-26T23:38:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa51be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa51be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58536/0 (socket says 172.21.15.46:58536) 2024-04-26T23:38:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa51be00640 1 -- 172.21.15.46:0/3548141787 learned_addr learned my addr 172.21.15.46:0/3548141787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.318+0000 7fa51b400640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa51c0b53b0 0x7fa51c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51aa00640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 0x7fa51c136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 -- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 msgr2=0x7fa51c136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 0x7fa51c136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 -- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa51c0b53b0 msgr2=0x7fa51c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa51c0b53b0 0x7fa51c0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 -- 172.21.15.46:0/3548141787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c144860 con 0x7fa51c15e050 2024-04-26T23:38:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51aa00640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 0x7fa51c136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa51be00640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa514007e40 tx=0x7fa51400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514014620 con 0x7fa51c15e050 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa514019070 con 0x7fa51c15e050 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514014a20 con 0x7fa51c15e050 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.319+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c144b40 con 0x7fa51c15e050 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.320+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c145100 con 0x7fa51c15e050 2024-04-26T23:38:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.321+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c069530 con 0x7fa51c15e050 2024-04-26T23:38:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.322+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa514014bc0 con 0x7fa51c15e050 2024-04-26T23:38:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.323+0000 7fa513400640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 0x7fa4fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.323+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa51408c8b0 con 0x7fa51c15e050 2024-04-26T23:38:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.323+0000 7fa51b400640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 0x7fa4fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.324+0000 7fa51b400640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 0x7fa4fc06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa50c006fd0 tx=0x7fa50c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.328+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa51405a840 con 0x7fa51c15e050 2024-04-26T23:38:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.529+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa51c13b630 con 0x7fa51c15e050 2024-04-26T23:38:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.530+0000 7fa513400640 1 -- 172.21.15.46:0/3548141787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa51405a140 con 0x7fa51c15e050 2024-04-26T23:38:21.534 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 msgr2=0x7fa4fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 0x7fa4fc06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa50c006fd0 tx=0x7fa50c008040 comp rx=0 tx=0).stop 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 msgr2=0x7fa51c145d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa514007e40 tx=0x7fa51400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 shutdown_connections 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4fc06c8f0 0x7fa4fc06edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa51c15e050 0x7fa51c145d80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa51c15d730 0x7fa51c136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 --2- 172.21.15.46:0/3548141787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa51c0b53b0 0x7fa51c0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 >> 172.21.15.46:0/3548141787 conn(0x7fa51c06c980 msgr2=0x7fa51c13bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 shutdown_connections 2024-04-26T23:38:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.533+0000 7fa522200640 1 -- 172.21.15.46:0/3548141787 wait complete. 2024-04-26T23:38:21.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.768+0000 7f7bc3e00640 1 Processor -- start 2024-04-26T23:38:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.769+0000 7f7bc3e00640 1 -- start start 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 0x7f7bc40d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc40d4090 con 0x7f7bc40d24f0 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc40b4020 con 0x7f7bc40d1980 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc40d3880 con 0x7f7bc415f840 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7bc40d24f0 0x7f7bc40b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49610/0 (socket says 172.21.15.46:49610) 2024-04-26T23:38:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.770+0000 7f7bc3400640 1 -- 172.21.15.46:0/3893236054 learned_addr learned my addr 172.21.15.46:0/3893236054 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc2a00640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 0x7f7bc40d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736617041 0 0) 0x7f7bbc001110 con 0x7f7bc40d24f0 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ba4003520 con 0x7f7bc40d24f0 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 602413952 0 0) 0x7f7ba4003520 con 0x7f7bc40d24f0 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 msgr2=0x7f7bc40d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 0x7f7bc40d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 >> v1:172.21.15.142:6789/0 conn(0x7f7bc40d1980 legacy=0x7f7bc40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc40d3ad0 con 0x7f7bc40d24f0 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7bbc001840 con 0x7f7bc40d24f0 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 >> v1:172.21.15.46:6789/0 conn(0x7f7bc40d24f0 legacy=0x7f7bc40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40039e0 0x7f7ba4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7ba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 0x7f7ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.771+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40d3880 con 0x7f7ba4004800 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba400ba70 con 0x7f7ba40039e0 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba400bc70 con 0x7f7bc415f840 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7ba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2a00640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40039e0 0x7f7ba4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 -- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 msgr2=0x7f7ba40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc415f840 0x7f7ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 -- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40039e0 msgr2=0x7f7ba4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40039e0 0x7f7ba4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.772+0000 7f7bc2000640 1 -- 172.21.15.46:0/3893236054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba4003520 con 0x7f7ba4004800 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.773+0000 7f7bc2000640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7ba4004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bac001e20 tx=0x7f7ba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf474266c196b82b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac001a70 con 0x7f7ba4004800 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bac02ae20 con 0x7f7ba4004800 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc1600640 1 -- 172.21.15.46:0/3893236054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac003bb0 con 0x7f7ba4004800 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 -- 172.21.15.46:0/3893236054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 msgr2=0x7f7ba4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/3893236054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7ba4004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bac001e20 tx=0x7f7ba4005830 comp rx=0 tx=0).stop 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3400640 1 -- 172.21.15.46:0/3893236054 reap_dead start 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 -- 172.21.15.46:0/3893236054 shutdown_connections 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 -- 172.21.15.46:0/3893236054 >> 172.21.15.46:0/3893236054 conn(0x7f7bc40d4f70 msgr2=0x7f7bc40d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 -- 172.21.15.46:0/3893236054 shutdown_connections 2024-04-26T23:38:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.774+0000 7f7bc3e00640 1 -- 172.21.15.46:0/3893236054 wait complete. 2024-04-26T23:38:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.776+0000 7f7bc3e00640 1 Processor -- start 2024-04-26T23:38:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.776+0000 7f7bc3e00640 1 -- start start 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc4153460 0x7f7bc40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 0x7f7bc40ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40d2e70 con 0x7f7ba4004800 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40b27d0 con 0x7f7bc40b5a30 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc40d2ff0 con 0x7f7bc4153460 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.777+0000 7f7bc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58556/0 (socket says 172.21.15.46:58556) 2024-04-26T23:38:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 -- 172.21.15.46:0/4121337241 learned_addr learned my addr 172.21.15.46:0/4121337241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc3400640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 0x7f7bc40ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2000640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc4153460 0x7f7bc40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 -- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc4153460 msgr2=0x7f7bc40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc4153460 0x7f7bc40b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 -- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 msgr2=0x7f7bc40ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 0x7f7bc40ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 -- 172.21.15.46:0/4121337241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc40b7cc0 con 0x7f7ba4004800 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc3400640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 0x7f7bc40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.778+0000 7f7bc2a00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4011600 tx=0x7f7bb4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.779+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400f920 con 0x7f7ba4004800 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.779+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb400fac0 con 0x7f7ba4004800 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.779+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400fe70 con 0x7f7ba4004800 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.779+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc40b7fa0 con 0x7f7ba4004800 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.779+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc40b8560 con 0x7f7ba4004800 2024-04-26T23:38:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.780+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc4162cd0 con 0x7f7ba4004800 2024-04-26T23:38:21.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.781+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bb4015460 con 0x7f7ba4004800 2024-04-26T23:38:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.782+0000 7f7bbb400640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 0x7f7ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.782+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7bb408c900 con 0x7f7ba4004800 2024-04-26T23:38:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.783+0000 7f7bc2000640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 0x7f7ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.784+0000 7f7bc2000640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 0x7f7ba006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7bac032000 tx=0x7f7bac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.786+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bc4162cd0 con 0x7f7ba4004800 2024-04-26T23:38:21.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.991+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7bc40b5e40 con 0x7f7ba4004800 2024-04-26T23:38:21.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.992+0000 7f7bbb400640 1 -- 172.21.15.46:0/4121337241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f7bc40b5e40 con 0x7f7ba4004800 2024-04-26T23:38:21.996 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 msgr2=0x7f7ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 0x7f7ba006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7bac032000 tx=0x7f7bac02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 msgr2=0x7f7bc41f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4011600 tx=0x7f7bb4011ac0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 shutdown_connections 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ba006c8f0 0x7f7ba006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc40b5a30 0x7f7bc40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc4153460 0x7f7bc40b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 --2- 172.21.15.46:0/4121337241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba4004800 0x7f7bc41f6820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:21.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 >> 172.21.15.46:0/4121337241 conn(0x7f7bc40d4f70 msgr2=0x7f7bc415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:22.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 shutdown_connections 2024-04-26T23:38:22.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:21.995+0000 7f7bc3e00640 1 -- 172.21.15.46:0/4121337241 wait complete. 2024-04-26T23:38:22.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.232+0000 7fc503e00640 1 Processor -- start 2024-04-26T23:38:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.232+0000 7fc503e00640 1 -- start start 2024-04-26T23:38:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc503e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc50415f300 0x7fc50406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc503e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc50406c890 0x7fc5040af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc503e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5040cccf0 con 0x7fc50415ef20 2024-04-26T23:38:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc503e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc50406cd00 con 0x7fc50415f300 2024-04-26T23:38:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc503e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc5040ccfd0 con 0x7fc50406c890 2024-04-26T23:38:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.234+0000 7fc502000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc50415f300 0x7fc50406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc503400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc50406c890 0x7fc5040af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc50415ef20 0x7fc5040cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:42862/0 (socket says 172.21.15.46:42862) 2024-04-26T23:38:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502a00640 1 -- 172.21.15.46:0/2525175816 learned_addr learned my addr 172.21.15.46:0/2525175816 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502000640 1 -- 172.21.15.46:0/2525175816 >> v2:172.21.15.142:3300/0 conn(0x7fc50406c890 msgr2=0x7fc5040af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502000640 1 --2- 172.21.15.46:0/2525175816 >> v2:172.21.15.142:3300/0 conn(0x7fc50406c890 0x7fc5040af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502000640 1 -- 172.21.15.46:0/2525175816 >> v1:172.21.15.142:6789/0 conn(0x7fc50415ef20 legacy=0x7fc5040cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.235+0000 7fc502000640 1 -- 172.21.15.46:0/2525175816 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5040aea20 con 0x7fc50415f300 2024-04-26T23:38:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.236+0000 7fc502000640 1 --2- 172.21.15.46:0/2525175816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc50406c350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5040ce010 tx=0x7fc4ec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669233d0f6c89edf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc501600640 1 -- 172.21.15.46:0/2525175816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec037070 con 0x7fc50415f300 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc501600640 1 -- 172.21.15.46:0/2525175816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec032440 con 0x7fc50415f300 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc501600640 1 -- 172.21.15.46:0/2525175816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec02ae60 con 0x7fc50415f300 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 -- 172.21.15.46:0/2525175816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 msgr2=0x7fc50406c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 --2- 172.21.15.46:0/2525175816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc50406c350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5040ce010 tx=0x7fc4ec02a110 comp rx=0 tx=0).stop 2024-04-26T23:38:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 -- 172.21.15.46:0/2525175816 shutdown_connections 2024-04-26T23:38:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 --2- 172.21.15.46:0/2525175816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc50406c350 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 --2- 172.21.15.46:0/2525175816 >> v2:172.21.15.142:3300/0 conn(0x7fc50406c890 0x7fc5040af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 -- 172.21.15.46:0/2525175816 >> 172.21.15.46:0/2525175816 conn(0x7fc5040d1c50 msgr2=0x7fc5040d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 -- 172.21.15.46:0/2525175816 shutdown_connections 2024-04-26T23:38:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.237+0000 7fc503e00640 1 -- 172.21.15.46:0/2525175816 wait complete. 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.239+0000 7fc503e00640 1 Processor -- start 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.239+0000 7fc503e00640 1 -- start start 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 0x7fc504158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50415ef20 0x7fc5040c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040cc9b0 con 0x7fc50415f300 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040aefe0 con 0x7fc50406c890 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.240+0000 7fc503e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc504163b30 con 0x7fc50415ef20 2024-04-26T23:38:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41708/0 (socket says 172.21.15.46:41708) 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 -- 172.21.15.46:0/1104137338 learned_addr learned my addr 172.21.15.46:0/1104137338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc502a00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 0x7fc504158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 -- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50415ef20 msgr2=0x7fc5040c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50415ef20 0x7fc5040c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 -- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 msgr2=0x7fc504158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 0x7fc504158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 -- 172.21.15.46:0/1104137338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5040cb900 con 0x7fc50415f300 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc502a00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 0x7fc504158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.241+0000 7fc503400640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e930 tx=0x7fc4fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.242+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc017070 con 0x7fc50415f300 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.242+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc00c980 con 0x7fc50415f300 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.242+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cd70 con 0x7fc50415f300 2024-04-26T23:38:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.242+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5040c6330 con 0x7fc50415f300 2024-04-26T23:38:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.242+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5040c68f0 con 0x7fc50415f300 2024-04-26T23:38:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.243+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5040c6140 con 0x7fc50415f300 2024-04-26T23:38:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.245+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4fc01a070 con 0x7fc50415f300 2024-04-26T23:38:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.246+0000 7fc4faa00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 0x7fc4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.246+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4fc08bfe0 con 0x7fc50415f300 2024-04-26T23:38:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.246+0000 7fc502a00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 0x7fc4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.248+0000 7fc502a00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 0x7fc4e006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40012e0 tx=0x7fc4f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.251+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4fc05a030 con 0x7fc50415f300 2024-04-26T23:38:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.455+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc504069ac0 con 0x7fc50415f300 2024-04-26T23:38:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.456+0000 7fc4faa00640 1 -- 172.21.15.46:0/1104137338 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc4fc059930 con 0x7fc50415f300 2024-04-26T23:38:22.460 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.458+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 msgr2=0x7fc4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.458+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 0x7fc4e006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40012e0 tx=0x7fc4f4009040 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.458+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 msgr2=0x7fc5040c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.458+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e930 tx=0x7fc4fc00edf0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 shutdown_connections 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4e006c8f0 0x7fc4e006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415f300 0x7fc5040c7bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc50415ef20 0x7fc5040c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 --2- 172.21.15.46:0/1104137338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc50406c890 0x7fc504158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 >> 172.21.15.46:0/1104137338 conn(0x7fc5040d1c50 msgr2=0x7fc50406fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 shutdown_connections 2024-04-26T23:38:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.459+0000 7fc503e00640 1 -- 172.21.15.46:0/1104137338 wait complete. 2024-04-26T23:38:22.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:38:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 Processor -- start 2024-04-26T23:38:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 -- start start 2024-04-26T23:38:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca40160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca40073ad0 con 0x7fca401466b0 2024-04-26T23:38:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca40146a90 con 0x7fca400720e0 2024-04-26T23:38:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.686+0000 7fca45c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca40073d70 con 0x7fca40160910 2024-04-26T23:38:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.687+0000 7fca3fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca401466b0 0x7fca4006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41058/0 (socket says 172.21.15.46:41058) 2024-04-26T23:38:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.687+0000 7fca3fe00640 1 -- 172.21.15.46:0/126017235 learned_addr learned my addr 172.21.15.46:0/126017235 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.687+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358531537 0 0) 0x7fca40073ad0 con 0x7fca401466b0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.687+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca24003520 con 0x7fca401466b0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.687+0000 7fca45200640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca40160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4189287611 0 0) 0x7fca24003520 con 0x7fca401466b0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 msgr2=0x7fca40160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca40160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 >> v1:172.21.15.142:6789/0 conn(0x7fca400720e0 legacy=0x7fca400cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40074050 con 0x7fca401466b0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fca30001670 con 0x7fca401466b0 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 >> v1:172.21.15.46:6789/0 conn(0x7fca401466b0 legacy=0x7fca4006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca24004060 0x7fca24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004d40 0x7fca240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca40073d70 con 0x7fca24004060 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca2400bc00 con 0x7fca24004d40 2024-04-26T23:38:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca2400be00 con 0x7fca40160910 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3f400640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca24004060 0x7fca24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3fe00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca45200640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004d40 0x7fca240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.688+0000 7fca3f400640 1 -- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 msgr2=0x7fca240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca3f400640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca3f400640 1 -- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004d40 msgr2=0x7fca240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca3f400640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004d40 0x7fca240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca3f400640 1 -- 172.21.15.46:0/126017235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca24003520 con 0x7fca24004060 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca45200640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004d40 0x7fca240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.689+0000 7fca3fe00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca40160910 0x7fca240039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca3f400640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca24004060 0x7fca24004800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca28001e50 tx=0x7fca24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69df791b7593c94b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28001a70 con 0x7fca24004060 2024-04-26T23:38:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca28003970 con 0x7fca24004060 2024-04-26T23:38:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca3ea00640 1 -- 172.21.15.46:0/126017235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28003d60 con 0x7fca24004060 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca45c00640 1 -- 172.21.15.46:0/126017235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca24004060 msgr2=0x7fca24004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca45c00640 1 --2- 172.21.15.46:0/126017235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca24004060 0x7fca24004800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca28001e50 tx=0x7fca24005830 comp rx=0 tx=0).stop 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.690+0000 7fca45200640 1 -- 172.21.15.46:0/126017235 reap_dead start 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.691+0000 7fca45c00640 1 -- 172.21.15.46:0/126017235 shutdown_connections 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.691+0000 7fca45c00640 1 -- 172.21.15.46:0/126017235 >> 172.21.15.46:0/126017235 conn(0x7fca400b26d0 msgr2=0x7fca400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.691+0000 7fca45c00640 1 -- 172.21.15.46:0/126017235 shutdown_connections 2024-04-26T23:38:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.691+0000 7fca45c00640 1 -- 172.21.15.46:0/126017235 wait complete. 2024-04-26T23:38:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.692+0000 7fca45c00640 1 Processor -- start 2024-04-26T23:38:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 -- start start 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 0x7fca400b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca401f85a0 0x7fca400b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca40160da0 con 0x7fca4013ad90 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca400cb470 con 0x7fca24004060 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.693+0000 7fca45c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca40160f20 con 0x7fca401f85a0 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41728/0 (socket says 172.21.15.46:41728) 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 -- 172.21.15.46:0/3551625113 learned_addr learned my addr 172.21.15.46:0/3551625113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca3fe00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 0x7fca400b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 -- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca401f85a0 msgr2=0x7fca400b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca401f85a0 0x7fca400b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 -- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 msgr2=0x7fca400b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 0x7fca400b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca45200640 1 -- 172.21.15.46:0/3551625113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca400b40c0 con 0x7fca4013ad90 2024-04-26T23:38:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.694+0000 7fca3fe00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 0x7fca400b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca45200640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fca400742f0 tx=0x7fca3800e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38017020 con 0x7fca4013ad90 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca38013610 con 0x7fca4013ad90 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800f040 con 0x7fca4013ad90 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca400b43a0 con 0x7fca4013ad90 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.695+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca400b51e0 con 0x7fca4013ad90 2024-04-26T23:38:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.696+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca400b46e0 con 0x7fca4013ad90 2024-04-26T23:38:22.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.697+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca3801a020 con 0x7fca4013ad90 2024-04-26T23:38:22.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.698+0000 7fca37e00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:22.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.698+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fca3808ba30 con 0x7fca4013ad90 2024-04-26T23:38:22.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.699+0000 7fca3fe00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.700+0000 7fca3fe00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 0x7fca1c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca30001eb0 tx=0x7fca30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:22.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.703+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca38021020 con 0x7fca4013ad90 2024-04-26T23:38:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.911+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca400693b0 con 0x7fca4013ad90 2024-04-26T23:38:22.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.912+0000 7fca37e00640 1 -- 172.21.15.46:0/3551625113 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fca38059a50 con 0x7fca4013ad90 2024-04-26T23:38:22.916 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 msgr2=0x7fca1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 0x7fca1c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca30001eb0 tx=0x7fca30009040 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 msgr2=0x7fca4015fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fca400742f0 tx=0x7fca3800e8b0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 shutdown_connections 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca4013ad90 0x7fca4015fc20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca401f85a0 0x7fca400b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 --2- 172.21.15.46:0/3551625113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca24004060 0x7fca400b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 >> 172.21.15.46:0/3551625113 conn(0x7fca400b26d0 msgr2=0x7fca400c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 shutdown_connections 2024-04-26T23:38:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:22.915+0000 7fca45c00640 1 -- 172.21.15.46:0/3551625113 wait complete. 2024-04-26T23:38:22.938 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:38:22.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:22.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T23:38:23.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:38:23.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:23.022 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:38:23.072 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:23.072 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:23.129 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:23.129 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:23.182 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:23.182 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:23.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.448+0000 7f2498a00640 1 Processor -- start 2024-04-26T23:38:23.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.448+0000 7f2498a00640 1 -- start start 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.450+0000 7f2498a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 0x7f2494161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.450+0000 7f2498a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 0x7f2494073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.450+0000 7f2498a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2494161940 con 0x7f249406e510 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.450+0000 7f2498a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2494071f80 con 0x7f24940732c0 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.450+0000 7f2498a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2494072260 con 0x7f2494160bd0 2024-04-26T23:38:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2491400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 0x7f2494161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2492800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 0x7f2494073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2492800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 0x7f2494073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:53832/0 (socket says 172.21.15.46:53832) 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2492800640 1 -- 172.21.15.46:0/971067849 learned_addr learned my addr 172.21.15.46:0/971067849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2491400640 1 -- 172.21.15.46:0/971067849 >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 msgr2=0x7f2494073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.451+0000 7f2491400640 1 --2- 172.21.15.46:0/971067849 >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 0x7f2494073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.452+0000 7f2491400640 1 -- 172.21.15.46:0/971067849 >> v1:172.21.15.142:6789/0 conn(0x7f249406e510 legacy=0x7f249406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.452+0000 7f2491400640 1 -- 172.21.15.46:0/971067849 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2494072540 con 0x7f2494160bd0 2024-04-26T23:38:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.453+0000 7f2491400640 1 --2- 172.21.15.46:0/971067849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 0x7f2494161030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584a00fe63e7e63a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.453+0000 7f2490a00640 1 -- 172.21.15.46:0/971067849 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c037070 con 0x7f2494160bd0 2024-04-26T23:38:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2490a00640 1 -- 172.21.15.46:0/971067849 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c032440 con 0x7f2494160bd0 2024-04-26T23:38:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2490a00640 1 -- 172.21.15.46:0/971067849 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c032810 con 0x7f2494160bd0 2024-04-26T23:38:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 -- 172.21.15.46:0/971067849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 msgr2=0x7f2494161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 --2- 172.21.15.46:0/971067849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 0x7f2494161030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c02a180 comp rx=0 tx=0).stop 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 -- 172.21.15.46:0/971067849 shutdown_connections 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 --2- 172.21.15.46:0/971067849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2494160bd0 0x7f2494161030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 --2- 172.21.15.46:0/971067849 >> v2:172.21.15.142:3300/0 conn(0x7f24940732c0 0x7f2494073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 -- 172.21.15.46:0/971067849 >> 172.21.15.46:0/971067849 conn(0x7f249406c530 msgr2=0x7f249406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.454+0000 7f2498a00640 1 -- 172.21.15.46:0/971067849 shutdown_connections 2024-04-26T23:38:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.455+0000 7f2498a00640 1 -- 172.21.15.46:0/971067849 wait complete. 2024-04-26T23:38:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.456+0000 7f2498a00640 1 Processor -- start 2024-04-26T23:38:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.456+0000 7f2498a00640 1 -- start start 2024-04-26T23:38:23.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24940732c0 0x7f2494145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2494160bd0 0x7f249413be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24941616f0 con 0x7f249406e510 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2494161570 con 0x7f2494160bd0 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.457+0000 7f2498a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2494072d40 con 0x7f24940732c0 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41742/0 (socket says 172.21.15.46:41742) 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 -- 172.21.15.46:0/2493824560 learned_addr learned my addr 172.21.15.46:0/2493824560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2492800640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2494160bd0 0x7f249413be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491400640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24940732c0 0x7f2494145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 -- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24940732c0 msgr2=0x7f2494145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24940732c0 0x7f2494145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 -- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2494160bd0 msgr2=0x7f249413be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2494160bd0 0x7f249413be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.458+0000 7f2491e00640 1 -- 172.21.15.46:0/2493824560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24940b2470 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f2491e00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f248400c730 tx=0x7f248400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484016020 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2484006d50 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24940b2750 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484003bb0 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.459+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24940b2d10 con 0x7f249406e510 2024-04-26T23:38:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.460+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494069530 con 0x7f249406e510 2024-04-26T23:38:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.461+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f248400a850 con 0x7f249406e510 2024-04-26T23:38:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.462+0000 7f248a000640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 0x7f247006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.462+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f248408bbf0 con 0x7f249406e510 2024-04-26T23:38:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.463+0000 7f2491400640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 0x7f247006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.464+0000 7f2491400640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 0x7f247006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f247c034000 tx=0x7f247c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.468+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2484020020 con 0x7f249406e510 2024-04-26T23:38:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.674+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f249406f860 con 0x7f249406e510 2024-04-26T23:38:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.677+0000 7f248a000640 1 -- 172.21.15.46:0/2493824560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f2484019310 con 0x7f249406e510 2024-04-26T23:38:23.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:23.682 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 msgr2=0x7f247006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 0x7f247006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f247c034000 tx=0x7f247c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 msgr2=0x7f249415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f248400c730 tx=0x7f248400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 shutdown_connections 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f247006c8f0 0x7f247006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2494160bd0 0x7f249413be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24940732c0 0x7f2494145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 --2- 172.21.15.46:0/2493824560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249406e510 0x7f249415b880 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 >> 172.21.15.46:0/2493824560 conn(0x7f249406c530 msgr2=0x7f24940ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 shutdown_connections 2024-04-26T23:38:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.680+0000 7f2498a00640 1 -- 172.21.15.46:0/2493824560 wait complete. 2024-04-26T23:38:23.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:23.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.910+0000 7f27baa00640 1 Processor -- start 2024-04-26T23:38:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.910+0000 7f27baa00640 1 -- start start 2024-04-26T23:38:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.911+0000 7f27baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27b40cfdf0 0x7f27b40b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.911+0000 7f27baa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27b406d790 0x7f27b406de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.911+0000 7f27baa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b40b3360 con 0x7f27b406e3b0 2024-04-26T23:38:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.911+0000 7f27baa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f27b406e820 con 0x7f27b40cfdf0 2024-04-26T23:38:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.911+0000 7f27baa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f27b40b3f70 con 0x7f27b406d790 2024-04-26T23:38:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27b40cfdf0 0x7f27b40b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b8a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27b406e3b0 0x7f27b40afb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41062/0 (socket says 172.21.15.46:41062) 2024-04-26T23:38:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b8a00640 1 -- 172.21.15.46:0/618880806 learned_addr learned my addr 172.21.15.46:0/618880806 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b3400640 1 --2- 172.21.15.46:0/618880806 >> v2:172.21.15.142:3300/0 conn(0x7f27b406d790 0x7f27b406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b2a00640 1 -- 172.21.15.46:0/618880806 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608472614 0 0) 0x7f27b40b3360 con 0x7f27b406e3b0 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.912+0000 7f27b2a00640 1 -- 172.21.15.46:0/618880806 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2798003520 con 0x7f27b406e3b0 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.913+0000 7f27b3e00640 1 -- 172.21.15.46:0/618880806 >> v2:172.21.15.142:3300/0 conn(0x7f27b406d790 msgr2=0x7f27b406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.913+0000 7f27b3e00640 1 --2- 172.21.15.46:0/618880806 >> v2:172.21.15.142:3300/0 conn(0x7f27b406d790 0x7f27b406de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.913+0000 7f27b3e00640 1 -- 172.21.15.46:0/618880806 >> v1:172.21.15.46:6789/0 conn(0x7f27b406e3b0 legacy=0x7f27b40afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.913+0000 7f27b3e00640 1 -- 172.21.15.46:0/618880806 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b40b3640 con 0x7f27b40cfdf0 2024-04-26T23:38:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.914+0000 7f27b3e00640 1 --2- 172.21.15.46:0/618880806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b40cfdf0 0x7f27b40b4400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27b40b8f40 tx=0x7f27a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36fb16066101a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.914+0000 7f27b2a00640 1 -- 172.21.15.46:0/618880806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4037070 con 0x7f27b40cfdf0 2024-04-26T23:38:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.914+0000 7f27b2a00640 1 -- 172.21.15.46:0/618880806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a4032440 con 0x7f27b40cfdf0 2024-04-26T23:38:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.914+0000 7f27b2a00640 1 -- 172.21.15.46:0/618880806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4032810 con 0x7f27b40cfdf0 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 -- 172.21.15.46:0/618880806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b40cfdf0 msgr2=0x7f27b40b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 --2- 172.21.15.46:0/618880806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b40cfdf0 0x7f27b40b4400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27b40b8f40 tx=0x7f27a402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 -- 172.21.15.46:0/618880806 shutdown_connections 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 --2- 172.21.15.46:0/618880806 >> v2:172.21.15.142:3300/0 conn(0x7f27b406d790 0x7f27b406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 --2- 172.21.15.46:0/618880806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b40cfdf0 0x7f27b40b4400 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 -- 172.21.15.46:0/618880806 >> 172.21.15.46:0/618880806 conn(0x7f27b40d2e90 msgr2=0x7f27b40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 -- 172.21.15.46:0/618880806 shutdown_connections 2024-04-26T23:38:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.915+0000 7f27baa00640 1 -- 172.21.15.46:0/618880806 wait complete. 2024-04-26T23:38:23.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.918+0000 7f27baa00640 1 Processor -- start 2024-04-26T23:38:23.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.918+0000 7f27baa00640 1 -- start start 2024-04-26T23:38:23.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b406e3b0 0x7f27b41594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 0x7f27b41599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27b406dbf0 con 0x7f27b406d790 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27b40b3c00 con 0x7f27b406e3b0 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27baa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27b40d01d0 con 0x7f27b40cfdf0 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41754/0 (socket says 172.21.15.46:41754) 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 -- 172.21.15.46:0/1647698512 learned_addr learned my addr 172.21.15.46:0/1647698512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b8a00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 0x7f27b41599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3400640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b406e3b0 0x7f27b41594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 -- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 msgr2=0x7f27b41599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 0x7f27b41599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 -- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b406e3b0 msgr2=0x7f27b41594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b406e3b0 0x7f27b41594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 -- 172.21.15.46:0/1647698512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b415bf90 con 0x7f27b406d790 2024-04-26T23:38:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b8a00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 0x7f27b41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27b3e00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000c00 tx=0x7f27a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.919+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4037040 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.920+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a4042070 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.920+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4030960 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.920+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b41589d0 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.920+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b4158f90 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.921+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b4069530 con 0x7f27b406d790 2024-04-26T23:38:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.922+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27a4030b00 con 0x7f27b406d790 2024-04-26T23:38:23.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.923+0000 7f27abe00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:23.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.923+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f27a40ac8f0 con 0x7f27b406d790 2024-04-26T23:38:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.924+0000 7f27b3400640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.925+0000 7f27b3400640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 0x7f279006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f279c006fd0 tx=0x7f279c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:23.928+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27a4045360 con 0x7f27b406d790 2024-04-26T23:38:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.134+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27b4069bd0 con 0x7f27b406d790 2024-04-26T23:38:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.134+0000 7f27abe00640 1 -- 172.21.15.46:0/1647698512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f27a407a910 con 0x7f27b406d790 2024-04-26T23:38:24.139 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.137+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 msgr2=0x7f279006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.137+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 0x7f279006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f279c006fd0 tx=0x7f279c008040 comp rx=0 tx=0).stop 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.137+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 msgr2=0x7f27b40c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.137+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000c00 tx=0x7f27a40046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 shutdown_connections 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40cfdf0 0x7f27b41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b406e3b0 0x7f27b41594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 --2- 172.21.15.46:0/1647698512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b406d790 0x7f27b40c6ce0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 >> 172.21.15.46:0/1647698512 conn(0x7f27b40d2e90 msgr2=0x7f27b4162430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 shutdown_connections 2024-04-26T23:38:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.138+0000 7f27baa00640 1 -- 172.21.15.46:0/1647698512 wait complete. 2024-04-26T23:38:24.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.374+0000 7ff4a6200640 1 Processor -- start 2024-04-26T23:38:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.374+0000 7ff4a6200640 1 -- start start 2024-04-26T23:38:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a6200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a01626c0 0x7ff4a00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4a0162aa0 0x7ff4a00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a6200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4a00d07d0 con 0x7ff4a00b5fb0 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a6200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a00d0ab0 con 0x7ff4a0162aa0 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a6200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a00d0d90 con 0x7ff4a01626c0 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4a0162aa0 0x7ff4a00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4a0162aa0 0x7ff4a00d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41764/0 (socket says 172.21.15.46:41764) 2024-04-26T23:38:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 -- 172.21.15.46:0/2561487437 learned_addr learned my addr 172.21.15.46:0/2561487437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a4e00640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a01626c0 0x7ff4a00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 -- 172.21.15.46:0/2561487437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a01626c0 msgr2=0x7ff4a00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a01626c0 0x7ff4a00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 -- 172.21.15.46:0/2561487437 >> v1:172.21.15.142:6789/0 conn(0x7ff4a00b5fb0 legacy=0x7ff4a00d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.376+0000 7ff4a5800640 1 -- 172.21.15.46:0/2561487437 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a00d2c80 con 0x7ff4a0162aa0 2024-04-26T23:38:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.378+0000 7ff4a5800640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00d1900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff498009e50 tx=0x7ff49802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab80b7ec71767e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.378+0000 7ff49f400640 1 -- 172.21.15.46:0/2561487437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498036070 con 0x7ff4a0162aa0 2024-04-26T23:38:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.378+0000 7ff49f400640 1 -- 172.21.15.46:0/2561487437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498031600 con 0x7ff4a0162aa0 2024-04-26T23:38:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.378+0000 7ff49f400640 1 -- 172.21.15.46:0/2561487437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980319d0 con 0x7ff4a0162aa0 2024-04-26T23:38:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 -- 172.21.15.46:0/2561487437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0162aa0 msgr2=0x7ff4a00d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00d1900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff498009e50 tx=0x7ff49802a710 comp rx=0 tx=0).stop 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 -- 172.21.15.46:0/2561487437 shutdown_connections 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00d1900 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 --2- 172.21.15.46:0/2561487437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a01626c0 0x7ff4a00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 -- 172.21.15.46:0/2561487437 >> 172.21.15.46:0/2561487437 conn(0x7ff4a009e2a0 msgr2=0x7ff4a00d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 -- 172.21.15.46:0/2561487437 shutdown_connections 2024-04-26T23:38:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.379+0000 7ff4a6200640 1 -- 172.21.15.46:0/2561487437 wait complete. 2024-04-26T23:38:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.381+0000 7ff4a6200640 1 Processor -- start 2024-04-26T23:38:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.382+0000 7ff4a6200640 1 -- start start 2024-04-26T23:38:24.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a00b5fb0 0x7ff4a00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a00d21d0 con 0x7ff4a01626c0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a00d2050 con 0x7ff4a0162aa0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a6200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a00d31a0 con 0x7ff4a00b5fb0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41774/0 (socket says 172.21.15.46:41774) 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 -- 172.21.15.46:0/1811783893 learned_addr learned my addr 172.21.15.46:0/1811783893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a4e00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a00b5fb0 0x7ff4a00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff4a5800640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 -- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a00b5fb0 msgr2=0x7ff4a00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a00b5fb0 0x7ff4a00ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 -- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 msgr2=0x7ff4a00cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.383+0000 7ff49fe00640 1 -- 172.21.15.46:0/1811783893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a00badc0 con 0x7ff4a01626c0 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff4a5800640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff49fe00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c400 tx=0x7ff49000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900052d0 con 0x7ff4a01626c0 2024-04-26T23:38:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4900054b0 con 0x7ff4a01626c0 2024-04-26T23:38:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490006a40 con 0x7ff4a01626c0 2024-04-26T23:38:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a014f3e0 con 0x7ff4a01626c0 2024-04-26T23:38:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.384+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a014f960 con 0x7ff4a01626c0 2024-04-26T23:38:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.386+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a00693b0 con 0x7ff4a01626c0 2024-04-26T23:38:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.387+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff490006be0 con 0x7ff4a01626c0 2024-04-26T23:38:24.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.390+0000 7ff49cc00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.390+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff49008b360 con 0x7ff4a01626c0 2024-04-26T23:38:24.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.390+0000 7ff4a4e00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.390+0000 7ff4a4e00640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 0x7ff48406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff494006fd0 tx=0x7ff494008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.394+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff490059420 con 0x7ff4a01626c0 2024-04-26T23:38:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.597+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4a00d41e0 con 0x7ff4a01626c0 2024-04-26T23:38:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.598+0000 7ff49cc00640 1 -- 172.21.15.46:0/1811783893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7ff490058d20 con 0x7ff4a01626c0 2024-04-26T23:38:24.602 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 msgr2=0x7ff48406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 0x7ff48406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff494006fd0 tx=0x7ff494008040 comp rx=0 tx=0).stop 2024-04-26T23:38:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 msgr2=0x7ff4a00cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c400 tx=0x7ff49000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 shutdown_connections 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0162aa0 0x7ff4a00cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a01626c0 0x7ff4a00cd050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 --2- 172.21.15.46:0/1811783893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a00b5fb0 0x7ff4a00ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 >> 172.21.15.46:0/1811783893 conn(0x7ff4a009e2a0 msgr2=0x7ff4a00b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 shutdown_connections 2024-04-26T23:38:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.601+0000 7ff4a6200640 1 -- 172.21.15.46:0/1811783893 wait complete. 2024-04-26T23:38:24.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.845+0000 7f6dd2800640 1 Processor -- start 2024-04-26T23:38:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.845+0000 7f6dd2800640 1 -- start start 2024-04-26T23:38:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dcc15f840 0x7f6dcc0d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd2800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 0x7f6dcc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd2800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dcc0d4090 con 0x7f6dcc0d2320 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6dcc0d3880 con 0x7f6dcc15f840 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd2800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6dcc0d2790 con 0x7f6dcc0d1980 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dcc15f840 0x7f6dcc0d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dcc15f840 0x7f6dcc0d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41786/0 (socket says 172.21.15.46:41786) 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd1400640 1 -- 172.21.15.46:0/3518534298 learned_addr learned my addr 172.21.15.46:0/3518534298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.847+0000 7f6dd0a00640 1 --2- 172.21.15.46:0/3518534298 >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 0x7f6dcc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.848+0000 7f6dd1400640 1 -- 172.21.15.46:0/3518534298 >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 msgr2=0x7f6dcc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.848+0000 7f6dd1400640 1 --2- 172.21.15.46:0/3518534298 >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 0x7f6dcc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.848+0000 7f6dd1400640 1 -- 172.21.15.46:0/3518534298 >> v1:172.21.15.142:6789/0 conn(0x7f6dcc0d2320 legacy=0x7f6dcc0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.848+0000 7f6dd1400640 1 -- 172.21.15.46:0/3518534298 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc0b4020 con 0x7f6dcc15f840 2024-04-26T23:38:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.848+0000 7f6dd0a00640 1 --2- 172.21.15.46:0/3518534298 >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 0x7f6dcc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.849+0000 7f6dd1400640 1 --2- 172.21.15.46:0/3518534298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0d2e70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc08d000 tx=0x7f6dbc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895f38d66f9e89f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.849+0000 7f6dcbe00640 1 -- 172.21.15.46:0/3518534298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc036070 con 0x7f6dcc15f840 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.849+0000 7f6dcbe00640 1 -- 172.21.15.46:0/3518534298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc031600 con 0x7f6dcc15f840 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.849+0000 7f6dcbe00640 1 -- 172.21.15.46:0/3518534298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0319d0 con 0x7f6dcc15f840 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 -- 172.21.15.46:0/3518534298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 msgr2=0x7f6dcc0d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 --2- 172.21.15.46:0/3518534298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0d2e70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc08d000 tx=0x7f6dbc02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 -- 172.21.15.46:0/3518534298 shutdown_connections 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 --2- 172.21.15.46:0/3518534298 >> v2:172.21.15.142:3300/0 conn(0x7f6dcc0d1980 0x7f6dcc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 --2- 172.21.15.46:0/3518534298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0d2e70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 -- 172.21.15.46:0/3518534298 >> 172.21.15.46:0/3518534298 conn(0x7f6dcc0d4f70 msgr2=0x7f6dcc0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 -- 172.21.15.46:0/3518534298 shutdown_connections 2024-04-26T23:38:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.850+0000 7f6dd2800640 1 -- 172.21.15.46:0/3518534298 wait complete. 2024-04-26T23:38:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.852+0000 7f6dd2800640 1 Processor -- start 2024-04-26T23:38:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.852+0000 7f6dd2800640 1 -- start start 2024-04-26T23:38:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 0x7f6dcc15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dcc0d2320 0x7f6dcc0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0b27d0 con 0x7f6dcc15f840 2024-04-26T23:38:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0d3db0 con 0x7f6dcc0d1980 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc0d33b0 con 0x7f6dcc0d2320 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41796/0 (socket says 172.21.15.46:41796) 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 -- 172.21.15.46:0/1490176721 learned_addr learned my addr 172.21.15.46:0/1490176721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1400640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 0x7f6dcc15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd0a00640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dcc0d2320 0x7f6dcc0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 -- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dcc0d2320 msgr2=0x7f6dcc0b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dcc0d2320 0x7f6dcc0b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 -- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 msgr2=0x7f6dcc15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 0x7f6dcc15c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 -- 172.21.15.46:0/1490176721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc0b9980 con 0x7f6dcc15f840 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1400640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 0x7f6dcc15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.853+0000 7f6dd1e00640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c390 tx=0x7f6dc400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.855+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4017070 con 0x7f6dcc15f840 2024-04-26T23:38:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.855+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc40035f0 con 0x7f6dcc15f840 2024-04-26T23:38:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.855+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc0b9c60 con 0x7f6dcc15f840 2024-04-26T23:38:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.855+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40039e0 con 0x7f6dcc15f840 2024-04-26T23:38:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.855+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc0ba220 con 0x7f6dcc15f840 2024-04-26T23:38:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.856+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc0693b0 con 0x7f6dcc15f840 2024-04-26T23:38:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.857+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dc401a070 con 0x7f6dcc15f840 2024-04-26T23:38:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.858+0000 7f6dc9600640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.858+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6dc408b370 con 0x7f6dcc15f840 2024-04-26T23:38:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.859+0000 7f6dd1400640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.860+0000 7f6dd1400640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 0x7f6dac06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc033000 tx=0x7f6dbc009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:24.863+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dc4059390 con 0x7f6dcc15f840 2024-04-26T23:38:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.069+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6dcc0d5d10 con 0x7f6dcc15f840 2024-04-26T23:38:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.070+0000 7f6dc9600640 1 -- 172.21.15.46:0/1490176721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f6dc4058c90 con 0x7f6dcc15f840 2024-04-26T23:38:25.074 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.073+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 msgr2=0x7f6dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.073+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 0x7f6dac06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc033000 tx=0x7f6dbc009000 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.073+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 msgr2=0x7f6dcc0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.073+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c390 tx=0x7f6dc400c850 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 shutdown_connections 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15f840 0x7f6dcc0b57b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dcc0d2320 0x7f6dcc0b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 --2- 172.21.15.46:0/1490176721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc0d1980 0x7f6dcc15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 >> 172.21.15.46:0/1490176721 conn(0x7f6dcc0d4f70 msgr2=0x7f6dcc06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 shutdown_connections 2024-04-26T23:38:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.074+0000 7f6dd2800640 1 -- 172.21.15.46:0/1490176721 wait complete. 2024-04-26T23:38:25.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:25.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.303+0000 7f909a800640 1 Processor -- start 2024-04-26T23:38:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.304+0000 7f909a800640 1 -- start start 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f909a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f909415cec0 0x7f90940cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f909a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90940cdb30 0x7f90940735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f909a800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90940ce7d0 con 0x7f9094073b30 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f909a800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f90940ceab0 con 0x7f90940cdb30 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f909a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f90940ced90 con 0x7f909415cec0 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f9098a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90940cdb30 0x7f90940735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f9098a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90940cdb30 0x7f90940735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41804/0 (socket says 172.21.15.46:41804) 2024-04-26T23:38:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.305+0000 7f9098a00640 1 -- 172.21.15.46:0/1552882138 learned_addr learned my addr 172.21.15.46:0/1552882138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:25.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.306+0000 7f9099400640 1 --2- 172.21.15.46:0/1552882138 >> v2:172.21.15.142:3300/0 conn(0x7f909415cec0 0x7f90940cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.306+0000 7f9098a00640 1 -- 172.21.15.46:0/1552882138 >> v2:172.21.15.142:3300/0 conn(0x7f909415cec0 msgr2=0x7f90940cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:25.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.306+0000 7f9098a00640 1 --2- 172.21.15.46:0/1552882138 >> v2:172.21.15.142:3300/0 conn(0x7f909415cec0 0x7f90940cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.306+0000 7f9098a00640 1 -- 172.21.15.46:0/1552882138 >> v1:172.21.15.142:6789/0 conn(0x7f9094073b30 legacy=0x7f9094071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.306+0000 7f9098a00640 1 -- 172.21.15.46:0/1552882138 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940cf070 con 0x7f90940cdb30 2024-04-26T23:38:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.307+0000 7f9098a00640 1 --2- 172.21.15.46:0/1552882138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90940cdb30 0x7f90940735f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90940b3830 tx=0x7f908002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fa59478c0fb91d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.307+0000 7f9093e00640 1 -- 172.21.15.46:0/1552882138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080037070 con 0x7f90940cdb30 2024-04-26T23:38:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.307+0000 7f9093e00640 1 -- 172.21.15.46:0/1552882138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9080032440 con 0x7f90940cdb30 2024-04-26T23:38:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.307+0000 7f9093e00640 1 -- 172.21.15.46:0/1552882138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080032810 con 0x7f90940cdb30 2024-04-26T23:38:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 -- 172.21.15.46:0/1552882138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90940cdb30 msgr2=0x7f90940735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 --2- 172.21.15.46:0/1552882138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90940cdb30 0x7f90940735f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90940b3830 tx=0x7f908002a230 comp rx=0 tx=0).stop 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 -- 172.21.15.46:0/1552882138 shutdown_connections 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 --2- 172.21.15.46:0/1552882138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90940cdb30 0x7f90940735f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 --2- 172.21.15.46:0/1552882138 >> v2:172.21.15.142:3300/0 conn(0x7f909415cec0 0x7f90940cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 -- 172.21.15.46:0/1552882138 >> 172.21.15.46:0/1552882138 conn(0x7f90940b27b0 msgr2=0x7f90940b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 -- 172.21.15.46:0/1552882138 shutdown_connections 2024-04-26T23:38:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.308+0000 7f909a800640 1 -- 172.21.15.46:0/1552882138 wait complete. 2024-04-26T23:38:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.310+0000 7f909a800640 1 Processor -- start 2024-04-26T23:38:25.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.310+0000 7f909a800640 1 -- start start 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90940cdb30 0x7f90940c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f909415cec0 0x7f90940b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9094073f70 con 0x7f9094073b30 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f909415d2a0 con 0x7f90940cdb30 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f909a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90940b4b50 con 0x7f909415cec0 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f9099400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f9099400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41806/0 (socket says 172.21.15.46:41806) 2024-04-26T23:38:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.311+0000 7f9099400640 1 -- 172.21.15.46:0/1938626928 learned_addr learned my addr 172.21.15.46:0/1938626928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 -- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f909415cec0 msgr2=0x7f90940b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f909415cec0 0x7f90940b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 -- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90940cdb30 msgr2=0x7f90940c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90940cdb30 0x7f90940c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 -- 172.21.15.46:0/1938626928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940cbc60 con 0x7f9094073b30 2024-04-26T23:38:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9099400640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f908c007e40 tx=0x7f908c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.312+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c014620 con 0x7f9094073b30 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.313+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c019070 con 0x7f9094073b30 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.313+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c014a00 con 0x7f9094073b30 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.313+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90940cbf40 con 0x7f9094073b30 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.313+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90940cc500 con 0x7f9094073b30 2024-04-26T23:38:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.314+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940693b0 con 0x7f9094073b30 2024-04-26T23:38:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.316+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f908c014ba0 con 0x7f9094073b30 2024-04-26T23:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.317+0000 7f9091600640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 0x7f907406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.317+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f908c08c820 con 0x7f9094073b30 2024-04-26T23:38:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.318+0000 7f9098a00640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 0x7f907406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.319+0000 7f9098a00640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 0x7f907406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9080034000 tx=0x7f908002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.323+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f908c05a7b0 con 0x7f9094073b30 2024-04-26T23:38:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.532+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9094072f60 con 0x7f9094073b30 2024-04-26T23:38:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.533+0000 7f9091600640 1 -- 172.21.15.46:0/1938626928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f908c05a0b0 con 0x7f9094073b30 2024-04-26T23:38:25.536 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 msgr2=0x7f907406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 0x7f907406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9080034000 tx=0x7f908002c000 comp rx=0 tx=0).stop 2024-04-26T23:38:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 msgr2=0x7f9094156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f908c007e40 tx=0x7f908c00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 shutdown_connections 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f907406c8f0 0x7f907406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f909415cec0 0x7f90940b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90940cdb30 0x7f90940c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 --2- 172.21.15.46:0/1938626928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9094073b30 0x7f9094156280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.536+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 >> 172.21.15.46:0/1938626928 conn(0x7f90940b27b0 msgr2=0x7f90940b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.537+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 shutdown_connections 2024-04-26T23:38:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.537+0000 7f909a800640 1 -- 172.21.15.46:0/1938626928 wait complete. 2024-04-26T23:38:25.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:38:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.770+0000 7f5a2f400640 1 Processor -- start 2024-04-26T23:38:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.770+0000 7f5a2f400640 1 -- start start 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a2806e510 0x7f5a2806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2f400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2806ee30 con 0x7f5a28160bd0 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2f400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a28161940 con 0x7f5a280732c0 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5a28071f80 con 0x7f5a2806e510 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a2806e510 0x7f5a2806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a2806e510 0x7f5a2806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41808/0 (socket says 172.21.15.46:41808) 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a2ca00640 1 -- 172.21.15.46:0/3320903630 learned_addr learned my addr 172.21.15.46:0/3320903630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.772+0000 7f5a27e00640 1 --1- 172.21.15.46:0/3320903630 >> v1:172.21.15.46:6789/0 conn(0x7f5a28160bd0 0x7f5a28161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41074/0 (socket says 172.21.15.46:41074) 2024-04-26T23:38:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.773+0000 7f5a2ca00640 1 -- 172.21.15.46:0/3320903630 >> v1:172.21.15.142:6789/0 conn(0x7f5a280732c0 legacy=0x7f5a28073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.773+0000 7f5a2ca00640 1 -- 172.21.15.46:0/3320903630 >> v1:172.21.15.46:6789/0 conn(0x7f5a28160bd0 legacy=0x7f5a28161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.773+0000 7f5a2ca00640 1 -- 172.21.15.46:0/3320903630 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a28072260 con 0x7f5a2806e510 2024-04-26T23:38:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.773+0000 7f5a27400640 1 -- 172.21.15.46:0/3320903630 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549224327 0 0) 0x7f5a2806ee30 con 0x7f5a28160bd0 2024-04-26T23:38:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.774+0000 7f5a2ca00640 1 --2- 172.21.15.46:0/3320903630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a2806e510 0x7f5a2806e8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a1802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7c2f693c29c674 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.774+0000 7f5a27400640 1 -- 172.21.15.46:0/3320903630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18037070 con 0x7f5a2806e510 2024-04-26T23:38:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.774+0000 7f5a27400640 1 -- 172.21.15.46:0/3320903630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a18032440 con 0x7f5a2806e510 2024-04-26T23:38:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.774+0000 7f5a27400640 1 -- 172.21.15.46:0/3320903630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18032810 con 0x7f5a2806e510 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 -- 172.21.15.46:0/3320903630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a2806e510 msgr2=0x7f5a2806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 --2- 172.21.15.46:0/3320903630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a2806e510 0x7f5a2806e8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a1802a2e0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 -- 172.21.15.46:0/3320903630 shutdown_connections 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 --2- 172.21.15.46:0/3320903630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a2806e510 0x7f5a2806e8f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 -- 172.21.15.46:0/3320903630 >> 172.21.15.46:0/3320903630 conn(0x7f5a2806c530 msgr2=0x7f5a2806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 -- 172.21.15.46:0/3320903630 shutdown_connections 2024-04-26T23:38:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.775+0000 7f5a2f400640 1 -- 172.21.15.46:0/3320903630 wait complete. 2024-04-26T23:38:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.777+0000 7f5a2f400640 1 Processor -- start 2024-04-26T23:38:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.777+0000 7f5a2f400640 1 -- start start 2024-04-26T23:38:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a2806e510 0x7f5a281573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a28160bd0 0x7f5a281f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28072d40 con 0x7f5a280732c0 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28072820 con 0x7f5a28160bd0 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a2f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a28072ec0 con 0x7f5a2806e510 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41824/0 (socket says 172.21.15.46:41824) 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.778+0000 7f5a27e00640 1 -- 172.21.15.46:0/1602733408 learned_addr learned my addr 172.21.15.46:0/1602733408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 -- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a2806e510 msgr2=0x7f5a281573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a2806e510 0x7f5a281573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 -- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a28160bd0 msgr2=0x7f5a281f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a28160bd0 0x7f5a281f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 -- 172.21.15.46:0/1602733408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280b2010 con 0x7f5a280732c0 2024-04-26T23:38:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a27e00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000c430 tx=0x7f5a1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a100074e0 con 0x7f5a280732c0 2024-04-26T23:38:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a100047e0 con 0x7f5a280732c0 2024-04-26T23:38:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004bd0 con 0x7f5a280732c0 2024-04-26T23:38:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a280b22f0 con 0x7f5a280732c0 2024-04-26T23:38:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.779+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a280b28b0 con 0x7f5a280732c0 2024-04-26T23:38:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.780+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28069530 con 0x7f5a280732c0 2024-04-26T23:38:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.781+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a10011070 con 0x7f5a280732c0 2024-04-26T23:38:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.782+0000 7f5a24c00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 0x7f5a0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.782+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a1008bc80 con 0x7f5a280732c0 2024-04-26T23:38:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.783+0000 7f5a2ca00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 0x7f5a0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.785+0000 7f5a2ca00640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 0x7f5a0406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5a18034000 tx=0x7f5a1802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:25.787+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a10059c10 con 0x7f5a280732c0 2024-04-26T23:38:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.003+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a28160740 con 0x7f5a280732c0 2024-04-26T23:38:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.004+0000 7f5a24c00640 1 -- 172.21.15.46:0/1602733408 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5a10059510 con 0x7f5a280732c0 2024-04-26T23:38:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:38:26.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 msgr2=0x7f5a0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 0x7f5a0406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5a18034000 tx=0x7f5a1802c000 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 msgr2=0x7f5a2813c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000c430 tx=0x7f5a1000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 shutdown_connections 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c8f0 0x7f5a0406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a28160bd0 0x7f5a281f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280732c0 0x7f5a2813c2b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 --2- 172.21.15.46:0/1602733408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a2806e510 0x7f5a281573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 >> 172.21.15.46:0/1602733408 conn(0x7f5a2806c530 msgr2=0x7f5a28141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 shutdown_connections 2024-04-26T23:38:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.007+0000 7f5a2f400640 1 -- 172.21.15.46:0/1602733408 wait complete. 2024-04-26T23:38:26.029 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T23:38:26.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:26.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T23:38:26.129 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:38:26.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:26.130 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T23:38:26.188 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:26.189 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:26.250 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:26.250 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:26.310 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:38:26.310 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:26.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.581+0000 7fce35000640 1 Processor -- start 2024-04-26T23:38:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.581+0000 7fce35000640 1 -- start start 2024-04-26T23:38:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.583+0000 7fce35000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce300b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.583+0000 7fce35000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce300b43c0 0x7fce3014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.583+0000 7fce35000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300b4940 con 0x7fce3006eb60 2024-04-26T23:38:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.583+0000 7fce35000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fce300b4c20 con 0x7fce300b43c0 2024-04-26T23:38:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.583+0000 7fce35000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce300b4f00 con 0x7fce3006e780 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.584+0000 7fce25400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce300b43c0 0x7fce3014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.584+0000 7fce2de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce300b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.584+0000 7fce25400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce300b43c0 0x7fce3014caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:53932/0 (socket says 172.21.15.46:53932) 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.584+0000 7fce25400640 1 -- 172.21.15.46:0/1083796386 learned_addr learned my addr 172.21.15.46:0/1083796386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.585+0000 7fce25400640 1 -- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 msgr2=0x7fce300b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.585+0000 7fce25400640 1 --2- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce300b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.585+0000 7fce25400640 1 -- 172.21.15.46:0/1083796386 >> v1:172.21.15.142:6789/0 conn(0x7fce3006eb60 legacy=0x7fce30167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.585+0000 7fce25400640 1 -- 172.21.15.46:0/1083796386 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce300b51e0 con 0x7fce300b43c0 2024-04-26T23:38:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.586+0000 7fce25400640 1 --2- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014caf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce18009870 tx=0x7fce1802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3115f4d46cd4bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.586+0000 7fce2d400640 1 -- 172.21.15.46:0/1083796386 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18037070 con 0x7fce300b43c0 2024-04-26T23:38:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce2d400640 1 -- 172.21.15.46:0/1083796386 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18032440 con 0x7fce300b43c0 2024-04-26T23:38:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce2d400640 1 -- 172.21.15.46:0/1083796386 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18032810 con 0x7fce300b43c0 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 -- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 msgr2=0x7fce3014caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 --2- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014caf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce18009870 tx=0x7fce1802a270 comp rx=0 tx=0).stop 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 -- 172.21.15.46:0/1083796386 shutdown_connections 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 --2- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014caf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 --2- 172.21.15.46:0/1083796386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce300b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.587+0000 7fce35000640 1 -- 172.21.15.46:0/1083796386 >> 172.21.15.46:0/1083796386 conn(0x7fce3006c530 msgr2=0x7fce3006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.588+0000 7fce35000640 1 -- 172.21.15.46:0/1083796386 shutdown_connections 2024-04-26T23:38:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.588+0000 7fce35000640 1 -- 172.21.15.46:0/1083796386 wait complete. 2024-04-26T23:38:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.589+0000 7fce35000640 1 Processor -- start 2024-04-26T23:38:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.590+0000 7fce35000640 1 -- start start 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce30139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce300734f0 con 0x7fce3006eb60 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce30073370 con 0x7fce300b43c0 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce35000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce30073670 con 0x7fce3006e780 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41830/0 (socket says 172.21.15.46:41830) 2024-04-26T23:38:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce25400640 1 -- 172.21.15.46:0/1844405177 learned_addr learned my addr 172.21.15.46:0/1844405177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce2de00640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce30139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce2e800640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce25400640 1 -- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 msgr2=0x7fce30139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.591+0000 7fce25400640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce30139c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce25400640 1 -- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 msgr2=0x7fce3014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce25400640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce25400640 1 -- 172.21.15.46:0/1844405177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3013a150 con 0x7fce3006eb60 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce2de00640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce30139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce2e800640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce25400640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce1802a2b0 tx=0x7fce180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18041030 con 0x7fce3006eb60 2024-04-26T23:38:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18030440 con 0x7fce3006eb60 2024-04-26T23:38:26.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18037040 con 0x7fce3006eb60 2024-04-26T23:38:26.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce30166190 con 0x7fce3006eb60 2024-04-26T23:38:26.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.592+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce30166750 con 0x7fce3006eb60 2024-04-26T23:38:26.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.594+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce30069530 con 0x7fce3006eb60 2024-04-26T23:38:26.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.595+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce18044070 con 0x7fce3006eb60 2024-04-26T23:38:26.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.595+0000 7fce26a00640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:26.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.596+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fce180ac3f0 con 0x7fce3006eb60 2024-04-26T23:38:26.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.596+0000 7fce2de00640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:26.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.597+0000 7fce2de00640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce20009ac0 tx=0x7fce20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:26.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.600+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce1804b070 con 0x7fce3006eb60 2024-04-26T23:38:26.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.806+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce3006cc70 con 0x7fce3006eb60 2024-04-26T23:38:26.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.808+0000 7fce26a00640 1 -- 172.21.15.46:0/1844405177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7fce1807a380 con 0x7fce3006eb60 2024-04-26T23:38:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:26.812 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.811+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 msgr2=0x7fce0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.811+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce20009ac0 tx=0x7fce20008040 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.811+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 msgr2=0x7fce3014aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.811+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce1802a2b0 tx=0x7fce180046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.811+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 shutdown_connections 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce300b43c0 0x7fce3014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce3006eb60 0x7fce3014aa50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 --2- 172.21.15.46:0/1844405177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce3006e780 0x7fce30139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 >> 172.21.15.46:0/1844405177 conn(0x7fce3006c530 msgr2=0x7fce30162fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 shutdown_connections 2024-04-26T23:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:26.812+0000 7fce35000640 1 -- 172.21.15.46:0/1844405177 wait complete. 2024-04-26T23:38:26.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:26.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:27.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.045+0000 7fc64be00640 1 Processor -- start 2024-04-26T23:38:27.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.045+0000 7fc64be00640 1 -- start start 2024-04-26T23:38:27.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.047+0000 7fc64be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc64415dd60 0x7fc644163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.047+0000 7fc64be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644071f80 0x7fc6440b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.047+0000 7fc64be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc644072880 con 0x7fc6440723e0 2024-04-26T23:38:27.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.047+0000 7fc64be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc64415e140 con 0x7fc64415dd60 2024-04-26T23:38:27.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.047+0000 7fc64be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc644072b40 con 0x7fc644071f80 2024-04-26T23:38:27.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc649e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6440723e0 0x7fc644167200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41090/0 (socket says 172.21.15.46:41090) 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc649e00640 1 -- 172.21.15.46:0/355729270 learned_addr learned my addr 172.21.15.46:0/355729270 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc648a00640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644071f80 0x7fc6440b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc649400640 1 --2- 172.21.15.46:0/355729270 >> v2:172.21.15.142:3300/0 conn(0x7fc64415dd60 0x7fc644163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc643e00640 1 -- 172.21.15.46:0/355729270 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164119001 0 0) 0x7fc644072880 con 0x7fc6440723e0 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.048+0000 7fc643e00640 1 -- 172.21.15.46:0/355729270 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc62c003520 con 0x7fc6440723e0 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.049+0000 7fc649400640 1 -- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644071f80 msgr2=0x7fc6440b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.049+0000 7fc649400640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644071f80 0x7fc6440b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.049+0000 7fc649400640 1 -- 172.21.15.46:0/355729270 >> v1:172.21.15.46:6789/0 conn(0x7fc6440723e0 legacy=0x7fc644167200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.049+0000 7fc649400640 1 -- 172.21.15.46:0/355729270 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc644072e20 con 0x7fc64415dd60 2024-04-26T23:38:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.050+0000 7fc649400640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64415dd60 0x7fc644163bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe61293d40343b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.051+0000 7fc643e00640 1 -- 172.21.15.46:0/355729270 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638037070 con 0x7fc64415dd60 2024-04-26T23:38:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.051+0000 7fc643e00640 1 -- 172.21.15.46:0/355729270 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc638032440 con 0x7fc64415dd60 2024-04-26T23:38:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.051+0000 7fc643e00640 1 -- 172.21.15.46:0/355729270 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380327f0 con 0x7fc64415dd60 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.051+0000 7fc64be00640 1 -- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64415dd60 msgr2=0x7fc644163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.051+0000 7fc64be00640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64415dd60 0x7fc644163bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802a180 comp rx=0 tx=0).stop 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 -- 172.21.15.46:0/355729270 shutdown_connections 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644071f80 0x7fc6440b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 --2- 172.21.15.46:0/355729270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64415dd60 0x7fc644163bc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 -- 172.21.15.46:0/355729270 >> 172.21.15.46:0/355729270 conn(0x7fc64406c980 msgr2=0x7fc6440b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 -- 172.21.15.46:0/355729270 shutdown_connections 2024-04-26T23:38:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.052+0000 7fc64be00640 1 -- 172.21.15.46:0/355729270 wait complete. 2024-04-26T23:38:27.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.054+0000 7fc64be00640 1 Processor -- start 2024-04-26T23:38:27.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.054+0000 7fc64be00640 1 -- start start 2024-04-26T23:38:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 0x7fc64415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64415dd60 0x7fc64415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc64406eca0 con 0x7fc6440723e0 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440733e0 con 0x7fc644071f80 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc64be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc64406ee20 con 0x7fc64415dd60 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41844/0 (socket says 172.21.15.46:41844) 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc648a00640 1 -- 172.21.15.46:0/346482149 learned_addr learned my addr 172.21.15.46:0/346482149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc649400640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 0x7fc64415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.056+0000 7fc649e00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64415dd60 0x7fc64415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 -- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64415dd60 msgr2=0x7fc64415a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64415dd60 0x7fc64415a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 -- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 msgr2=0x7fc64415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 0x7fc64415f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 -- 172.21.15.46:0/346482149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6441482d0 con 0x7fc6440723e0 2024-04-26T23:38:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc649400640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 0x7fc64415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc648a00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc63000e9b0 tx=0x7fc63000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630017070 con 0x7fc6440723e0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63000c9a0 con 0x7fc6440723e0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63000cd90 con 0x7fc6440723e0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.057+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6441601b0 con 0x7fc6440723e0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.058+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc644160770 con 0x7fc6440723e0 2024-04-26T23:38:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.059+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc644069530 con 0x7fc6440723e0 2024-04-26T23:38:27.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.060+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc63001a070 con 0x7fc6440723e0 2024-04-26T23:38:27.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.061+0000 7fc641600640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 0x7fc62406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.061+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc63008bff0 con 0x7fc6440723e0 2024-04-26T23:38:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.061+0000 7fc649400640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 0x7fc62406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.063+0000 7fc649400640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 0x7fc62406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.065+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc630059f90 con 0x7fc6440723e0 2024-04-26T23:38:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.270+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc64415d420 con 0x7fc6440723e0 2024-04-26T23:38:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.271+0000 7fc641600640 1 -- 172.21.15.46:0/346482149 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc630059890 con 0x7fc6440723e0 2024-04-26T23:38:27.274 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 msgr2=0x7fc62406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 0x7fc62406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 msgr2=0x7fc644147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc63000e9b0 tx=0x7fc63000ee70 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 shutdown_connections 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62406c8f0 0x7fc62406edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64415dd60 0x7fc64415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440723e0 0x7fc644147d90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 --2- 172.21.15.46:0/346482149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc644071f80 0x7fc64415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 >> 172.21.15.46:0/346482149 conn(0x7fc64406c980 msgr2=0x7fc6440d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 shutdown_connections 2024-04-26T23:38:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.274+0000 7fc64be00640 1 -- 172.21.15.46:0/346482149 wait complete. 2024-04-26T23:38:27.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.502+0000 7f84a7e00640 1 Processor -- start 2024-04-26T23:38:27.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.502+0000 7f84a7e00640 1 -- start start 2024-04-26T23:38:27.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f84a80b4b50 0x7f84a8073d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a80ce7d0 con 0x7f84a80cdd00 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a80ceab0 con 0x7f84a815cec0 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f84a80ced90 con 0x7f84a80b4b50 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f849fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84a80cdd00 0x7f84a80735f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41096/0 (socket says 172.21.15.46:41096) 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f849fe00640 1 -- 172.21.15.46:0/3304831946 learned_addr learned my addr 172.21.15.46:0/3304831946 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a7400640 1 --2- 172.21.15.46:0/3304831946 >> v2:172.21.15.142:3300/0 conn(0x7f84a80b4b50 0x7f84a8073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.504+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209064460 0 0) 0x7f84a80ce7d0 con 0x7f84a80cdd00 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f848c003520 con 0x7f84a80cdd00 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 278992903 0 0) 0x7f848c003520 con 0x7f84a80cdd00 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 >> v2:172.21.15.142:3300/0 conn(0x7f84a80b4b50 msgr2=0x7f84a8073d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 --2- 172.21.15.46:0/3304831946 >> v2:172.21.15.142:3300/0 conn(0x7f84a80b4b50 0x7f84a8073d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 >> v1:172.21.15.142:6789/0 conn(0x7f84a815cec0 legacy=0x7f84a80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a80cf070 con 0x7f84a80cdd00 2024-04-26T23:38:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8490001690 con 0x7f84a80cdd00 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 >> v1:172.21.15.46:6789/0 conn(0x7f84a80cdd00 legacy=0x7f84a80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 0x7f848c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c004800 0x7f848c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c0051c0 0x7f848c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80ced90 con 0x7f848c003a00 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f848c00c0c0 con 0x7f848c0051c0 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f848c00c2c0 con 0x7f848c004800 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.505+0000 7f84a6a00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 0x7f848c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f849fe00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c004800 0x7f848c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a6a00640 1 -- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c004800 msgr2=0x7f848c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a6a00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c004800 0x7f848c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a6a00640 1 -- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c0051c0 msgr2=0x7f848c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a6a00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c0051c0 0x7f848c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a6a00640 1 -- 172.21.15.46:0/3304831946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c003520 con 0x7f848c003a00 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.506+0000 7f84a7400640 1 -- 172.21.15.46:0/3304831946 reap_dead start 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.507+0000 7f84a6a00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 0x7f848c003e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8498001750 tx=0x7f848c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fe7fc056f2306c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.507+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498001370 con 0x7f848c003a00 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.507+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84980052d0 con 0x7f848c003a00 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.507+0000 7f84a6000640 1 -- 172.21.15.46:0/3304831946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849802abe0 con 0x7f848c003a00 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 -- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 msgr2=0x7f848c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 0x7f848c003e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8498001750 tx=0x7f848c005830 comp rx=0 tx=0).stop 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 -- 172.21.15.46:0/3304831946 shutdown_connections 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 --2- 172.21.15.46:0/3304831946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f848c003a00 0x7f848c003e20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 -- 172.21.15.46:0/3304831946 >> 172.21.15.46:0/3304831946 conn(0x7f84a80b27b0 msgr2=0x7f84a80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 -- 172.21.15.46:0/3304831946 shutdown_connections 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.508+0000 7f84a7e00640 1 -- 172.21.15.46:0/3304831946 wait complete. 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.510+0000 7f84a7e00640 1 Processor -- start 2024-04-26T23:38:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.510+0000 7f84a7e00640 1 -- start start 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c003a00 0x7f84a80c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 0x7f84a815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80cd5f0 con 0x7f84a815f890 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84a815d2a0 con 0x7f848c003a00 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.511+0000 7f84a7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84a80cd770 con 0x7f84a815fd10 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41848/0 (socket says 172.21.15.46:41848) 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 -- 172.21.15.46:0/2198355008 learned_addr learned my addr 172.21.15.46:0/2198355008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f84a6a00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c003a00 0x7f84a80c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f84a7400640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 0x7f84a815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 -- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 msgr2=0x7f84a815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 0x7f84a815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 -- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c003a00 msgr2=0x7f84a80c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c003a00 0x7f84a80c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 -- 172.21.15.46:0/2198355008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a81f84f0 con 0x7f84a815f890 2024-04-26T23:38:27.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f84a7400640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 0x7f84a815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.512+0000 7f849fe00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8490001010 tx=0x7f849000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.513+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490019070 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.513+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84900072d0 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.513+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a81f87d0 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.513+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900047e0 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.513+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a81f8d90 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.514+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a80693b0 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.515+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f849001c070 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.516+0000 7f849ea00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 0x7f848406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.516+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f849008b8b0 con 0x7f84a815f890 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.517+0000 7f84a6a00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 0x7f848406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.518+0000 7f84a6a00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 0x7f848406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8498032000 tx=0x7f849802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.520+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8490059840 con 0x7f84a815f890 2024-04-26T23:38:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.724+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84a81f8f60 con 0x7f84a815f890 2024-04-26T23:38:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.725+0000 7f849ea00640 1 -- 172.21.15.46:0/2198355008 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f8490059140 con 0x7f84a815f890 2024-04-26T23:38:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 msgr2=0x7f848406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 0x7f848406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8498032000 tx=0x7f849802e000 comp rx=0 tx=0).stop 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 msgr2=0x7f84a815a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8490001010 tx=0x7f849000dc00 comp rx=0 tx=0).stop 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 shutdown_connections 2024-04-26T23:38:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848406c8f0 0x7f848406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a815fd10 0x7f84a815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a815f890 0x7f84a815a850 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 --2- 172.21.15.46:0/2198355008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c003a00 0x7f84a80c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 >> 172.21.15.46:0/2198355008 conn(0x7f84a80b27b0 msgr2=0x7f84a80b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 shutdown_connections 2024-04-26T23:38:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.728+0000 7f84a7e00640 1 -- 172.21.15.46:0/2198355008 wait complete. 2024-04-26T23:38:27.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.961+0000 7f7243e00640 1 Processor -- start 2024-04-26T23:38:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.961+0000 7f7243e00640 1 -- start start 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7243e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f724415f010 0x7f72440cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7243e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7244163650 0x7f72440c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7243e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72440cd940 con 0x7f72440c5e80 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7243e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f72440ce830 con 0x7f724415f010 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7243e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72440c62b0 con 0x7f7244163650 2024-04-26T23:38:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7242a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f724415f010 0x7f72440cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7242a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f724415f010 0x7f72440cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41862/0 (socket says 172.21.15.46:41862) 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.963+0000 7f7242a00640 1 -- 172.21.15.46:0/158027332 learned_addr learned my addr 172.21.15.46:0/158027332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.964+0000 7f7242000640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7244163650 0x7f72440c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.964+0000 7f7242a00640 1 -- 172.21.15.46:0/158027332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7244163650 msgr2=0x7f72440c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.964+0000 7f7242a00640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7244163650 0x7f72440c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.964+0000 7f7242a00640 1 -- 172.21.15.46:0/158027332 >> v1:172.21.15.142:6789/0 conn(0x7f72440c5e80 legacy=0x7f72440ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.964+0000 7f7242a00640 1 -- 172.21.15.46:0/158027332 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72440cdc20 con 0x7f724415f010 2024-04-26T23:38:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.965+0000 7f7242a00640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f724415f010 0x7f72440cd320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f723402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84fd911a32bbf74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.965+0000 7f7241600640 1 -- 172.21.15.46:0/158027332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234037070 con 0x7f724415f010 2024-04-26T23:38:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.965+0000 7f7241600640 1 -- 172.21.15.46:0/158027332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7234032440 con 0x7f724415f010 2024-04-26T23:38:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.965+0000 7f7241600640 1 -- 172.21.15.46:0/158027332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234032810 con 0x7f724415f010 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 -- 172.21.15.46:0/158027332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f724415f010 msgr2=0x7f72440cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f724415f010 0x7f72440cd320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f723402a130 comp rx=0 tx=0).stop 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 -- 172.21.15.46:0/158027332 shutdown_connections 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7244163650 0x7f72440c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 --2- 172.21.15.46:0/158027332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f724415f010 0x7f72440cd320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 -- 172.21.15.46:0/158027332 >> 172.21.15.46:0/158027332 conn(0x7f72440b26d0 msgr2=0x7f72440b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 -- 172.21.15.46:0/158027332 shutdown_connections 2024-04-26T23:38:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.966+0000 7f7243e00640 1 -- 172.21.15.46:0/158027332 wait complete. 2024-04-26T23:38:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.968+0000 7f7243e00640 1 Processor -- start 2024-04-26T23:38:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.968+0000 7f7243e00640 1 -- start start 2024-04-26T23:38:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f724415f010 0x7f72440c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7244163650 0x7f72441e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f724415f3f0 con 0x7f72440c5e80 2024-04-26T23:38:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7244163ab0 con 0x7f7244163650 2024-04-26T23:38:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7243e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f724406de60 con 0x7f724415f010 2024-04-26T23:38:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41878/0 (socket says 172.21.15.46:41878) 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 -- 172.21.15.46:0/3436555476 learned_addr learned my addr 172.21.15.46:0/3436555476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 -- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f724415f010 msgr2=0x7f72440c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f724415f010 0x7f72440c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 -- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7244163650 msgr2=0x7f72441e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.969+0000 7f7242a00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7244163650 0x7f72441e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f7242a00640 1 -- 172.21.15.46:0/3436555476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72440c9680 con 0x7f72440c5e80 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f7242a00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f723402a170 tx=0x7f72340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234041020 con 0x7f72440c5e80 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72340306c0 con 0x7f72440c5e80 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72440c9960 con 0x7f72440c5e80 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234037040 con 0x7f72440c5e80 2024-04-26T23:38:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.970+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72440c9f20 con 0x7f72440c5e80 2024-04-26T23:38:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.972+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72440693b0 con 0x7f72440c5e80 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.972+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7234044020 con 0x7f72440c5e80 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.973+0000 7f723aa00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.973+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f72340ac680 con 0x7f72440c5e80 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.974+0000 7f7242000640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.975+0000 7f7242000640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 0x7f722406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f724415f620 tx=0x7f7230008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:27.978+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f723404b030 con 0x7f72440c5e80 2024-04-26T23:38:28.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.180+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f72440cffb0 con 0x7f72440c5e80 2024-04-26T23:38:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.181+0000 7f723aa00640 1 -- 172.21.15.46:0/3436555476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f723407a740 con 0x7f72440c5e80 2024-04-26T23:38:28.184 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 msgr2=0x7f722406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 0x7f722406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f724415f620 tx=0x7f7230008040 comp rx=0 tx=0).stop 2024-04-26T23:38:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 msgr2=0x7f724415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f723402a170 tx=0x7f72340046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 shutdown_connections 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7244163650 0x7f72441e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f724415f010 0x7f72440c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 --2- 172.21.15.46:0/3436555476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72440c5e80 0x7f724415a210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 >> 172.21.15.46:0/3436555476 conn(0x7f72440b26d0 msgr2=0x7f724406efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 shutdown_connections 2024-04-26T23:38:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.184+0000 7f7243e00640 1 -- 172.21.15.46:0/3436555476 wait complete. 2024-04-26T23:38:28.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.417+0000 7ff6e8800640 1 Processor -- start 2024-04-26T23:38:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.418+0000 7ff6e8800640 1 -- start start 2024-04-26T23:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.419+0000 7ff6e8800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6e015d730 0x7ff6e00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.419+0000 7ff6e8800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e8800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e006eb90 con 0x7ff6e015e050 2024-04-26T23:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e8800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e00b4310 con 0x7ff6e015d730 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e8800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e00b45f0 con 0x7ff6e00b53b0 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6e015d730 0x7ff6e00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6e015d730 0x7ff6e00d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41884/0 (socket says 172.21.15.46:41884) 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 -- 172.21.15.46:0/3922776353 learned_addr learned my addr 172.21.15.46:0/3922776353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5400640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 -- 172.21.15.46:0/3922776353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 msgr2=0x7ff6e015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.420+0000 7ff6e5e00640 1 -- 172.21.15.46:0/3922776353 >> v1:172.21.15.142:6789/0 conn(0x7ff6e015e050 legacy=0x7ff6e006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.421+0000 7ff6e5e00640 1 -- 172.21.15.46:0/3922776353 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e00b48d0 con 0x7ff6e015d730 2024-04-26T23:38:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.422+0000 7ff6e5e00640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e00d00d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa06afbab1df4fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.422+0000 7ff6e4a00640 1 -- 172.21.15.46:0/3922776353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4037070 con 0x7ff6e015d730 2024-04-26T23:38:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.422+0000 7ff6e4a00640 1 -- 172.21.15.46:0/3922776353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d4032440 con 0x7ff6e015d730 2024-04-26T23:38:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.422+0000 7ff6e4a00640 1 -- 172.21.15.46:0/3922776353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4032810 con 0x7ff6e015d730 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 -- 172.21.15.46:0/3922776353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 msgr2=0x7ff6e00d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e00d00d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d402a250 comp rx=0 tx=0).stop 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 -- 172.21.15.46:0/3922776353 shutdown_connections 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 --2- 172.21.15.46:0/3922776353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e00d00d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 -- 172.21.15.46:0/3922776353 >> 172.21.15.46:0/3922776353 conn(0x7ff6e006c980 msgr2=0x7ff6e00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 -- 172.21.15.46:0/3922776353 shutdown_connections 2024-04-26T23:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.423+0000 7ff6e8800640 1 -- 172.21.15.46:0/3922776353 wait complete. 2024-04-26T23:38:28.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.425+0000 7ff6e8800640 1 Processor -- start 2024-04-26T23:38:28.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.425+0000 7ff6e8800640 1 -- start start 2024-04-26T23:38:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e01fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 0x7ff6e01fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e015e4c0 con 0x7ff6e015d730 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e00b4e90 con 0x7ff6e015e050 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e8800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e00b3ec0 con 0x7ff6e00b53b0 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41888/0 (socket says 172.21.15.46:41888) 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e5400640 1 -- 172.21.15.46:0/2282302295 learned_addr learned my addr 172.21.15.46:0/2282302295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.426+0000 7ff6e5e00640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e01fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e6800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 0x7ff6e01fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 -- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 msgr2=0x7ff6e01fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e01fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 -- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 msgr2=0x7ff6e01fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 0x7ff6e01fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 -- 172.21.15.46:0/2282302295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e0144060 con 0x7ff6e015d730 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e6800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 0x7ff6e01fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e5400640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000d620 tx=0x7ff6d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000b620 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d0019070 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000ba00 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.427+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e0144340 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.428+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e0144900 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.429+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e0069530 con 0x7ff6e015d730 2024-04-26T23:38:28.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.430+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6d000bba0 con 0x7ff6e015d730 2024-04-26T23:38:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.431+0000 7ff6da000640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 0x7ff6c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.431+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff6d008c3f0 con 0x7ff6e015d730 2024-04-26T23:38:28.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.431+0000 7ff6e5e00640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 0x7ff6c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.433+0000 7ff6e5e00640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 0x7ff6c406ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4034000 tx=0x7ff6d402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.435+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6d005a380 con 0x7ff6e015d730 2024-04-26T23:38:28.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.641+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff6e015b050 con 0x7ff6e015d730 2024-04-26T23:38:28.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.642+0000 7ff6da000640 1 -- 172.21.15.46:0/2282302295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff6d0059c80 con 0x7ff6e015d730 2024-04-26T23:38:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 msgr2=0x7ff6c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 0x7ff6c406ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4034000 tx=0x7ff6d402e000 comp rx=0 tx=0).stop 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 msgr2=0x7ff6e01e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000d620 tx=0x7ff6d000dae0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 shutdown_connections 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6c406c920 0x7ff6c406ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e015e050 0x7ff6e01fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e015d730 0x7ff6e01e3570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 --2- 172.21.15.46:0/2282302295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6e00b53b0 0x7ff6e01fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 >> 172.21.15.46:0/2282302295 conn(0x7ff6e006c980 msgr2=0x7ff6e00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 shutdown_connections 2024-04-26T23:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.645+0000 7ff6e8800640 1 -- 172.21.15.46:0/2282302295 wait complete. 2024-04-26T23:38:28.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:38:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.883+0000 7f9724400640 1 Processor -- start 2024-04-26T23:38:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.883+0000 7f9724400640 1 -- start start 2024-04-26T23:38:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.885+0000 7f9724400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f971c167a70 0x7f971c167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.885+0000 7f9724400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f971c0cc3f0 0x7f971c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.885+0000 7f9724400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c0cceb0 con 0x7f971c06c350 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.885+0000 7f9724400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f971c0cfe80 con 0x7f971c167a70 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.885+0000 7f9724400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f971c168390 con 0x7f971c0cc3f0 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9722400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f971c06c350 0x7f971c0c49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41098/0 (socket says 172.21.15.46:41098) 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9722400640 1 -- 172.21.15.46:0/2580381520 learned_addr learned my addr 172.21.15.46:0/2580381520 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721a00640 1 --2- 172.21.15.46:0/2580381520 >> v2:172.21.15.46:3300/0 conn(0x7f971c167a70 0x7f971c167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721000640 1 --2- 172.21.15.46:0/2580381520 >> v2:172.21.15.142:3300/0 conn(0x7f971c0cc3f0 0x7f971c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f971be00640 1 -- 172.21.15.46:0/2580381520 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378241544 0 0) 0x7f971c0cceb0 con 0x7f971c06c350 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f971be00640 1 -- 172.21.15.46:0/2580381520 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9700003520 con 0x7f971c06c350 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721a00640 1 -- 172.21.15.46:0/2580381520 >> v2:172.21.15.142:3300/0 conn(0x7f971c0cc3f0 msgr2=0x7f971c0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721a00640 1 --2- 172.21.15.46:0/2580381520 >> v2:172.21.15.142:3300/0 conn(0x7f971c0cc3f0 0x7f971c0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721a00640 1 -- 172.21.15.46:0/2580381520 >> v1:172.21.15.46:6789/0 conn(0x7f971c06c350 legacy=0x7f971c0c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.886+0000 7f9721a00640 1 -- 172.21.15.46:0/2580381520 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c0d00b0 con 0x7f971c167a70 2024-04-26T23:38:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.887+0000 7f9721a00640 1 --2- 172.21.15.46:0/2580381520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c167a70 0x7f971c167e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f971c0cf920 tx=0x7f970c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eca7e548534ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f971be00640 1 -- 172.21.15.46:0/2580381520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c037070 con 0x7f971c167a70 2024-04-26T23:38:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f971be00640 1 -- 172.21.15.46:0/2580381520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c032440 con 0x7f971c167a70 2024-04-26T23:38:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f971be00640 1 -- 172.21.15.46:0/2580381520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c032810 con 0x7f971c167a70 2024-04-26T23:38:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 -- 172.21.15.46:0/2580381520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c167a70 msgr2=0x7f971c167e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 --2- 172.21.15.46:0/2580381520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c167a70 0x7f971c167e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f971c0cf920 tx=0x7f970c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 -- 172.21.15.46:0/2580381520 shutdown_connections 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 --2- 172.21.15.46:0/2580381520 >> v2:172.21.15.142:3300/0 conn(0x7f971c0cc3f0 0x7f971c0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 --2- 172.21.15.46:0/2580381520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c167a70 0x7f971c167e50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 -- 172.21.15.46:0/2580381520 >> 172.21.15.46:0/2580381520 conn(0x7f971c06da90 msgr2=0x7f971c0d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.888+0000 7f9724400640 1 -- 172.21.15.46:0/2580381520 shutdown_connections 2024-04-26T23:38:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.889+0000 7f9724400640 1 -- 172.21.15.46:0/2580381520 wait complete. 2024-04-26T23:38:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.890+0000 7f9724400640 1 Processor -- start 2024-04-26T23:38:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.890+0000 7f9724400640 1 -- start start 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f971c06c350 0x7f971c0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c167a70 0x7f971c1fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f971c06f040 con 0x7f971c0cc3f0 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0d0700 con 0x7f971c06c350 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.891+0000 7f9724400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f971c06f1c0 con 0x7f971c167a70 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41906/0 (socket says 172.21.15.46:41906) 2024-04-26T23:38:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 -- 172.21.15.46:0/487983598 learned_addr learned my addr 172.21.15.46:0/487983598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 -- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c167a70 msgr2=0x7f971c1fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c167a70 0x7f971c1fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 -- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f971c06c350 msgr2=0x7f971c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f971c06c350 0x7f971c0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 -- 172.21.15.46:0/487983598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c1fd300 con 0x7f971c0cc3f0 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.892+0000 7f9721000640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f970800d830 tx=0x7f970800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.893+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800b6c0 con 0x7f971c0cc3f0 2024-04-26T23:38:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.893+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9708019070 con 0x7f971c0cc3f0 2024-04-26T23:38:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.893+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c1fd590 con 0x7f971c0cc3f0 2024-04-26T23:38:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.893+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800ba80 con 0x7f971c0cc3f0 2024-04-26T23:38:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.893+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c0b4ed0 con 0x7f971c0cc3f0 2024-04-26T23:38:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.894+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c069530 con 0x7f971c0cc3f0 2024-04-26T23:38:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.895+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f970801e460 con 0x7f971c0cc3f0 2024-04-26T23:38:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.896+0000 7f9719600640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.897+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f970808c640 con 0x7f971c0cc3f0 2024-04-26T23:38:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.897+0000 7f9721a00640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.898+0000 7f9721a00640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f970c034000 tx=0x7f970c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:28.901+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f970805a660 con 0x7f971c0cc3f0 2024-04-26T23:38:29.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.123+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f971c0d2b10 con 0x7f971c0cc3f0 2024-04-26T23:38:29.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.123+0000 7f9719600640 1 -- 172.21.15.46:0/487983598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f9708059f60 con 0x7f971c0cc3f0 2024-04-26T23:38:29.126 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 msgr2=0x7f96fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f970c034000 tx=0x7f970c02c000 comp rx=0 tx=0).stop 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 msgr2=0x7f971c1fc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f970800d830 tx=0x7f970800dcf0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 shutdown_connections 2024-04-26T23:38:29.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c167a70 0x7f971c1fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c0cc3f0 0x7f971c1fc880 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 --2- 172.21.15.46:0/487983598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f971c06c350 0x7f971c0b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.126+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 >> 172.21.15.46:0/487983598 conn(0x7f971c06da90 msgr2=0x7f971c06fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.127+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 shutdown_connections 2024-04-26T23:38:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.127+0000 7f9724400640 1 -- 172.21.15.46:0/487983598 wait complete. 2024-04-26T23:38:29.150 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T23:38:29.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:29.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T23:38:29.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:38:29.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:29.239 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T23:38:29.293 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:29.293 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:29.352 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:29.352 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:29.403 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:38:29.403 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:38:29.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.675+0000 7f466b200640 1 Processor -- start 2024-04-26T23:38:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.675+0000 7f466b200640 1 -- start start 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.676+0000 7f466b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f466c0b4000 0x7f466c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.676+0000 7f466b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c0d11f0 con 0x7f466c0d2130 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f466b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c0d5f20 con 0x7f466c15f4e0 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f466b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f466c0d0ce0 con 0x7f466c0b4000 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f4669400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f466c0d2130 0x7f466c0b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41100/0 (socket says 172.21.15.46:41100) 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f4669400640 1 -- 172.21.15.46:0/3975664581 learned_addr learned my addr 172.21.15.46:0/3975664581 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f466a800640 1 --2- 172.21.15.46:0/3975664581 >> v2:172.21.15.142:3300/0 conn(0x7f466c0b4000 0x7f466c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.677+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854973049 0 0) 0x7f466c0d11f0 con 0x7f466c0d2130 2024-04-26T23:38:29.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4650003520 con 0x7f466c0d2130 2024-04-26T23:38:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2152143989 0 0) 0x7f4650003520 con 0x7f466c0d2130 2024-04-26T23:38:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 >> v2:172.21.15.142:3300/0 conn(0x7f466c0b4000 msgr2=0x7f466c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 --2- 172.21.15.46:0/3975664581 >> v2:172.21.15.142:3300/0 conn(0x7f466c0b4000 0x7f466c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 >> v1:172.21.15.142:6789/0 conn(0x7f466c15f4e0 legacy=0x7f466c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0d0f10 con 0x7f466c0d2130 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f465c001670 con 0x7f466c0d2130 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 >> v1:172.21.15.46:6789/0 conn(0x7f466c0d2130 legacy=0x7f466c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46500039e0 0x7f4650003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f4650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46500051c0 0x7f465000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f465000c090 con 0x7f4650004800 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f465000c210 con 0x7f46500051c0 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f465000c410 con 0x7f46500039e0 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.678+0000 7f4669400640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f4650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f466a800640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46500051c0 0x7f465000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669400640 1 -- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46500039e0 msgr2=0x7f4650003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669e00640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46500039e0 0x7f4650003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669400640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46500039e0 0x7f4650003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669400640 1 -- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46500051c0 msgr2=0x7f465000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669400640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46500051c0 0x7f465000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669400640 1 -- 172.21.15.46:0/3975664581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650003520 con 0x7f4650004800 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f466a800640 1 -- 172.21.15.46:0/3975664581 reap_dead start 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.679+0000 7f4669e00640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46500039e0 0x7f4650003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.680+0000 7f4669400640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f4650004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c240 tx=0x7f4650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc53817d2b9e072 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.680+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c007d10 con 0x7f4650004800 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.680+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c005ad0 con 0x7f4650004800 2024-04-26T23:38:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.680+0000 7f4668a00640 1 -- 172.21.15.46:0/3975664581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c007070 con 0x7f4650004800 2024-04-26T23:38:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 -- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 msgr2=0x7f4650004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f4650004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c240 tx=0x7f4650005830 comp rx=0 tx=0).stop 2024-04-26T23:38:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 -- 172.21.15.46:0/3975664581 shutdown_connections 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 --2- 172.21.15.46:0/3975664581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f4650004c80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 -- 172.21.15.46:0/3975664581 >> 172.21.15.46:0/3975664581 conn(0x7f466c0d5010 msgr2=0x7f466c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 -- 172.21.15.46:0/3975664581 shutdown_connections 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.681+0000 7f466b200640 1 -- 172.21.15.46:0/3975664581 wait complete. 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.683+0000 7f466b200640 1 Processor -- start 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.683+0000 7f466b200640 1 -- start start 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 0x7f466c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f466c1f4f30 0x7f466c151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0b1d60 con 0x7f4650004800 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0d2990 con 0x7f466c15b6b0 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f466b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f466c15f8c0 con 0x7f466c1f4f30 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f4669e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f4669e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41930/0 (socket says 172.21.15.46:41930) 2024-04-26T23:38:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.684+0000 7f4669e00640 1 -- 172.21.15.46:0/808848687 learned_addr learned my addr 172.21.15.46:0/808848687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669400640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 0x7f466c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f466a800640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f466c1f4f30 0x7f466c151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 -- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f466c1f4f30 msgr2=0x7f466c151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f466c1f4f30 0x7f466c151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 -- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 msgr2=0x7f466c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 0x7f466c15d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 -- 172.21.15.46:0/808848687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0cd180 con 0x7f4650004800 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669400640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 0x7f466c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4669e00640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f465400d4b0 tx=0x7f465400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.685+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654005480 con 0x7f4650004800 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.686+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4654017070 con 0x7f4650004800 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.686+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654003830 con 0x7f4650004800 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.686+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c0cd460 con 0x7f4650004800 2024-04-26T23:38:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.686+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c0cdd00 con 0x7f4650004800 2024-04-26T23:38:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.687+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c0693b0 con 0x7f4650004800 2024-04-26T23:38:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.688+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46540039d0 con 0x7f4650004800 2024-04-26T23:38:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.689+0000 7f4662000640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 0x7f464806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.689+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f465408bd80 con 0x7f4650004800 2024-04-26T23:38:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.690+0000 7f4669400640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 0x7f464806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.691+0000 7f4669400640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 0x7f464806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c210 tx=0x7f465c02d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.693+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4654059dd0 con 0x7f4650004800 2024-04-26T23:38:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.896+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f466c0d33d0 con 0x7f4650004800 2024-04-26T23:38:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.898+0000 7f4662000640 1 -- 172.21.15.46:0/808848687 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f46540596d0 con 0x7f4650004800 2024-04-26T23:38:29.901 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:29.901 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.900+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 msgr2=0x7f464806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.900+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 0x7f464806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c210 tx=0x7f465c02d000 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.900+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 msgr2=0x7f466c0cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.900+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f465400d4b0 tx=0x7f465400d970 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 shutdown_connections 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f464806c8f0 0x7f464806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f466c1f4f30 0x7f466c151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f466c15b6b0 0x7f466c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 --2- 172.21.15.46:0/808848687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4650004800 0x7f466c0cb1d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 >> 172.21.15.46:0/808848687 conn(0x7f466c0d5010 msgr2=0x7f466c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 shutdown_connections 2024-04-26T23:38:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:29.901+0000 7f466b200640 1 -- 172.21.15.46:0/808848687 wait complete. 2024-04-26T23:38:29.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:29.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.137+0000 7f68e5600640 1 Processor -- start 2024-04-26T23:38:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.138+0000 7f68e5600640 1 -- start start 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68e5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e015d730 0x7f68e00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68e5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68e015e220 0x7f68e006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68e5600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e006eb90 con 0x7f68e00b53b0 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68e5600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f68e00b4310 con 0x7f68e015e220 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68e5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68e00b45f0 con 0x7f68e015d730 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68df400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68e015e220 0x7f68e006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68df400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68e015e220 0x7f68e006dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41932/0 (socket says 172.21.15.46:41932) 2024-04-26T23:38:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.139+0000 7f68df400640 1 -- 172.21.15.46:0/2298401799 learned_addr learned my addr 172.21.15.46:0/2298401799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.140+0000 7f68dea00640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e015d730 0x7f68e00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.140+0000 7f68df400640 1 -- 172.21.15.46:0/2298401799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e015d730 msgr2=0x7f68e00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.140+0000 7f68df400640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e015d730 0x7f68e00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.140+0000 7f68df400640 1 -- 172.21.15.46:0/2298401799 >> v1:172.21.15.142:6789/0 conn(0x7f68e00b53b0 legacy=0x7f68e015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.140+0000 7f68df400640 1 -- 172.21.15.46:0/2298401799 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e00b48d0 con 0x7f68e015e220 2024-04-26T23:38:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.141+0000 7f68df400640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015e220 0x7f68e006dc90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fae12397ba575bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.141+0000 7f68dd600640 1 -- 172.21.15.46:0/2298401799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8037070 con 0x7f68e015e220 2024-04-26T23:38:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.141+0000 7f68dd600640 1 -- 172.21.15.46:0/2298401799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d8032440 con 0x7f68e015e220 2024-04-26T23:38:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.141+0000 7f68dd600640 1 -- 172.21.15.46:0/2298401799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8032810 con 0x7f68e015e220 2024-04-26T23:38:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 -- 172.21.15.46:0/2298401799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015e220 msgr2=0x7f68e006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015e220 0x7f68e006dc90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802a210 comp rx=0 tx=0).stop 2024-04-26T23:38:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 -- 172.21.15.46:0/2298401799 shutdown_connections 2024-04-26T23:38:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015e220 0x7f68e006dc90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 --2- 172.21.15.46:0/2298401799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e015d730 0x7f68e00d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 -- 172.21.15.46:0/2298401799 >> 172.21.15.46:0/2298401799 conn(0x7f68e006c980 msgr2=0x7f68e00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 -- 172.21.15.46:0/2298401799 shutdown_connections 2024-04-26T23:38:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.142+0000 7f68e5600640 1 -- 172.21.15.46:0/2298401799 wait complete. 2024-04-26T23:38:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.144+0000 7f68e5600640 1 Processor -- start 2024-04-26T23:38:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.144+0000 7f68e5600640 1 -- start start 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e00b53b0 0x7f68e0073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e015e220 0x7f68e00af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68e00b3ec0 con 0x7f68e015d730 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68e00b4e90 con 0x7f68e015e220 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68e5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68e00b4040 con 0x7f68e00b53b0 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68de000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68de000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41936/0 (socket says 172.21.15.46:41936) 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.145+0000 7f68de000640 1 -- 172.21.15.46:0/2767171167 learned_addr learned my addr 172.21.15.46:0/2767171167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68dea00640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e00b53b0 0x7f68e0073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 -- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e00b53b0 msgr2=0x7f68e0073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e00b53b0 0x7f68e0073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 -- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e015e220 msgr2=0x7f68e00af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e015e220 0x7f68e00af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 -- 172.21.15.46:0/2767171167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e00aeb30 con 0x7f68e015d730 2024-04-26T23:38:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68de000640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b3e0 tx=0x7f68c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8015020 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c8011850 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8011c40 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.146+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e00aee10 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.147+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01f5cc0 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.148+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e0069530 con 0x7f68e015d730 2024-04-26T23:38:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.149+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68c8018020 con 0x7f68e015d730 2024-04-26T23:38:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.150+0000 7f68d6a00640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.150+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f68c808bfa0 con 0x7f68e015d730 2024-04-26T23:38:30.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.150+0000 7f68dea00640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.151+0000 7f68dea00640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 0x7f68bc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68e0142400 tx=0x7f68d000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.154+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68c801f020 con 0x7f68e015d730 2024-04-26T23:38:30.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.360+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68e00ced00 con 0x7f68e015d730 2024-04-26T23:38:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.362+0000 7f68d6a00640 1 -- 172.21.15.46:0/2767171167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f68c80172c0 con 0x7f68e015d730 2024-04-26T23:38:30.364 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.364+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 msgr2=0x7f68bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.364+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 0x7f68bc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68e0142400 tx=0x7f68d000b040 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.364+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 msgr2=0x7f68e013ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.364+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b3e0 tx=0x7f68c800b8a0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 shutdown_connections 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e015e220 0x7f68e00af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e015d730 0x7f68e013ab90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 --2- 172.21.15.46:0/2767171167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e00b53b0 0x7f68e0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 >> 172.21.15.46:0/2767171167 conn(0x7f68e006c980 msgr2=0x7f68e015bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 shutdown_connections 2024-04-26T23:38:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.365+0000 7f68e5600640 1 -- 172.21.15.46:0/2767171167 wait complete. 2024-04-26T23:38:30.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.597+0000 7f7ae2e00640 1 Processor -- start 2024-04-26T23:38:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.597+0000 7f7ae2e00640 1 -- start start 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.598+0000 7f7ae2e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7adc15cec0 0x7f7adc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.598+0000 7f7ae2e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc0ce7d0 con 0x7f7adc0cdb30 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.598+0000 7f7ae2e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc0ceab0 con 0x7f7adc073d00 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.598+0000 7f7ae2e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc0ced90 con 0x7f7adc15cec0 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7adc15cec0 0x7f7adc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7adc15cec0 0x7f7adc0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41940/0 (socket says 172.21.15.46:41940) 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2736906616 learned_addr learned my addr 172.21.15.46:0/2736906616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2736906616 >> v1:172.21.15.142:6789/0 conn(0x7f7adc073d00 legacy=0x7f7adc0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2736906616 >> v1:172.21.15.46:6789/0 conn(0x7f7adc0cdb30 legacy=0x7f7adc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.599+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2736906616 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0cf070 con 0x7f7adc15cec0 2024-04-26T23:38:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.600+0000 7f7ae1a00640 1 --2- 172.21.15.46:0/2736906616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc15cec0 0x7f7adc0cd5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a579dbcd30d676f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7adbe00640 1 -- 172.21.15.46:0/2736906616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0037070 con 0x7f7adc15cec0 2024-04-26T23:38:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7adbe00640 1 -- 172.21.15.46:0/2736906616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad0032440 con 0x7f7adc15cec0 2024-04-26T23:38:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7adbe00640 1 -- 172.21.15.46:0/2736906616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0032810 con 0x7f7adc15cec0 2024-04-26T23:38:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2736906616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc15cec0 msgr2=0x7f7adc0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2736906616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc15cec0 0x7f7adc0cd5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2736906616 shutdown_connections 2024-04-26T23:38:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2736906616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc15cec0 0x7f7adc0cd5f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2736906616 >> 172.21.15.46:0/2736906616 conn(0x7f7adc0b27b0 msgr2=0x7f7adc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2736906616 shutdown_connections 2024-04-26T23:38:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.601+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2736906616 wait complete. 2024-04-26T23:38:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.603+0000 7f7ae2e00640 1 Processor -- start 2024-04-26T23:38:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.603+0000 7f7ae2e00640 1 -- start start 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7adc0cdb30 0x7f7adc0c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 0x7f7adc0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0b4250 con 0x7f7adc073d00 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc15d2a0 con 0x7f7adc15cec0 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.604+0000 7f7ae2e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc0b43d0 con 0x7f7adc0cdb30 2024-04-26T23:38:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41954/0 (socket says 172.21.15.46:41954) 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2756223203 learned_addr learned my addr 172.21.15.46:0/2756223203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7adc0cdb30 msgr2=0x7f7adc0c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7adc0cdb30 0x7f7adc0c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 msgr2=0x7f7adc0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae2400640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 0x7f7adc0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 0x7f7adc0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 -- 172.21.15.46:0/2756223203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0c9870 con 0x7f7adc073d00 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae2400640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 0x7f7adc0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.605+0000 7f7ae1a00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.606+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0041070 con 0x7f7adc073d00 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.606+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad003c870 con 0x7f7adc073d00 2024-04-26T23:38:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.606+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0037040 con 0x7f7adc073d00 2024-04-26T23:38:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.606+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc0c9b50 con 0x7f7adc073d00 2024-04-26T23:38:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.606+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc0ca110 con 0x7f7adc073d00 2024-04-26T23:38:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.607+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc0693b0 con 0x7f7adc073d00 2024-04-26T23:38:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.608+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad0044070 con 0x7f7adc073d00 2024-04-26T23:38:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.609+0000 7f7ad9600640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 0x7f7abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.609+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7ad00acc10 con 0x7f7adc073d00 2024-04-26T23:38:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.610+0000 7f7ae1000640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 0x7f7abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.611+0000 7f7ae1000640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 0x7f7abc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8000f60 tx=0x7f7ac80079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.613+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ad007acc0 con 0x7f7adc073d00 2024-04-26T23:38:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.817+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7adc0b6090 con 0x7f7adc073d00 2024-04-26T23:38:30.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.819+0000 7f7ad9600640 1 -- 172.21.15.46:0/2756223203 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f7ad007a5c0 con 0x7f7adc073d00 2024-04-26T23:38:30.822 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.821+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 msgr2=0x7f7abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.821+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 0x7f7abc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8000f60 tx=0x7f7ac80079d0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.821+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 msgr2=0x7f7adc151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.821+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad00046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 shutdown_connections 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7abc06c8f0 0x7f7abc06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7adc15cec0 0x7f7adc0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7adc0cdb30 0x7f7adc0c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 --2- 172.21.15.46:0/2756223203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7adc073d00 0x7f7adc151f80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 >> 172.21.15.46:0/2756223203 conn(0x7f7adc0b27b0 msgr2=0x7f7adc0758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 shutdown_connections 2024-04-26T23:38:30.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:30.822+0000 7f7ae2e00640 1 -- 172.21.15.46:0/2756223203 wait complete. 2024-04-26T23:38:30.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.046+0000 7f0369e00640 1 Processor -- start 2024-04-26T23:38:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.047+0000 7f0369e00640 1 -- start start 2024-04-26T23:38:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.048+0000 7f0369e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364146740 0x7f036415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.048+0000 7f0369e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03640ce670 0x7f03640ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.048+0000 7f0369e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0364146b20 con 0x7f03640d0330 2024-04-26T23:38:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0369e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0364161a00 con 0x7f03640ce670 2024-04-26T23:38:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0369e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03640ccf30 con 0x7f0364146740 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0369400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03640ce670 0x7f03640ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0369400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03640ce670 0x7f03640ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:54098/0 (socket says 172.21.15.46:54098) 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0369400640 1 -- 172.21.15.46:0/2709003716 learned_addr learned my addr 172.21.15.46:0/2709003716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.049+0000 7f0368a00640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364146740 0x7f036415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.050+0000 7f0369400640 1 -- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364146740 msgr2=0x7f036415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.050+0000 7f0369400640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364146740 0x7f036415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.050+0000 7f0369400640 1 -- 172.21.15.46:0/2709003716 >> v1:172.21.15.142:6789/0 conn(0x7f03640d0330 legacy=0x7f03641612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.050+0000 7f0369400640 1 -- 172.21.15.46:0/2709003716 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0364160af0 con 0x7f03640ce670 2024-04-26T23:38:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.052+0000 7f0369400640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640ce670 0x7f03640ceaa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f035c009870 tx=0x7f035c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47570a45296b96b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.052+0000 7f0363400640 1 -- 172.21.15.46:0/2709003716 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c037070 con 0x7f03640ce670 2024-04-26T23:38:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.052+0000 7f0363400640 1 -- 172.21.15.46:0/2709003716 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c032440 con 0x7f03640ce670 2024-04-26T23:38:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.052+0000 7f0363400640 1 -- 172.21.15.46:0/2709003716 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c032810 con 0x7f03640ce670 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 -- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640ce670 msgr2=0x7f03640ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640ce670 0x7f03640ceaa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f035c009870 tx=0x7f035c02a250 comp rx=0 tx=0).stop 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 -- 172.21.15.46:0/2709003716 shutdown_connections 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640ce670 0x7f03640ceaa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 --2- 172.21.15.46:0/2709003716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364146740 0x7f036415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 -- 172.21.15.46:0/2709003716 >> 172.21.15.46:0/2709003716 conn(0x7f03640b26d0 msgr2=0x7f03640b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 -- 172.21.15.46:0/2709003716 shutdown_connections 2024-04-26T23:38:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.053+0000 7f0369e00640 1 -- 172.21.15.46:0/2709003716 wait complete. 2024-04-26T23:38:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.055+0000 7f0369e00640 1 Processor -- start 2024-04-26T23:38:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.055+0000 7f0369e00640 1 -- start start 2024-04-26T23:38:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 0x7f03640b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640d0330 0x7f036415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03640cefe0 con 0x7f0364146740 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03640d0790 con 0x7f03640d0330 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03640b5760 con 0x7f03640ce670 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41968/0 (socket says 172.21.15.46:41968) 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0369400640 1 -- 172.21.15.46:0/787020631 learned_addr learned my addr 172.21.15.46:0/787020631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0368a00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 0x7f03640b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.056+0000 7f0363e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640d0330 0x7f036415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 -- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 msgr2=0x7f03640b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 0x7f03640b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 -- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640d0330 msgr2=0x7f036415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640d0330 0x7f036415c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 -- 172.21.15.46:0/787020631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03640b59e0 con 0x7f0364146740 2024-04-26T23:38:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0368a00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 0x7f03640b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369400640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f035c009870 tx=0x7f035c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c041020 con 0x7f0364146740 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c03d640 con 0x7f0364146740 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c037040 con 0x7f0364146740 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.057+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03640b5cc0 con 0x7f0364146740 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.058+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03640c8770 con 0x7f0364146740 2024-04-26T23:38:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.059+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03640cc4d0 con 0x7f0364146740 2024-04-26T23:38:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.060+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f035c044020 con 0x7f0364146740 2024-04-26T23:38:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.060+0000 7f0360c00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.061+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f035c0ac700 con 0x7f0364146740 2024-04-26T23:38:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.061+0000 7f0368a00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.062+0000 7f0368a00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0354009af0 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.065+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03640cc4d0 con 0x7f0364146740 2024-04-26T23:38:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.267+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03640c8560 con 0x7f0364146740 2024-04-26T23:38:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.268+0000 7f0360c00640 1 -- 172.21.15.46:0/787020631 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f035c04b030 con 0x7f0364146740 2024-04-26T23:38:31.272 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.271+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 msgr2=0x7f034406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.271+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0354009af0 tx=0x7f0354008040 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.271+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 msgr2=0x7f03641e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.271+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f035c009870 tx=0x7f035c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 shutdown_connections 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364146740 0x7f03641e6bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03640d0330 0x7f036415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 --2- 172.21.15.46:0/787020631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03640ce670 0x7f03640b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 >> 172.21.15.46:0/787020631 conn(0x7f03640b26d0 msgr2=0x7f03640b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 shutdown_connections 2024-04-26T23:38:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.272+0000 7f0369e00640 1 -- 172.21.15.46:0/787020631 wait complete. 2024-04-26T23:38:31.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:31.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.510+0000 7f37d2200640 1 Processor -- start 2024-04-26T23:38:31.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.510+0000 7f37d2200640 1 -- start start 2024-04-26T23:38:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.512+0000 7f37d2200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37cc15d460 0x7f37cc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.512+0000 7f37d2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 0x7f37cc0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.512+0000 7f37d2200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37cc143690 con 0x7f37cc06c890 2024-04-26T23:38:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.512+0000 7f37d2200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37cc073920 con 0x7f37cc15d460 2024-04-26T23:38:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.512+0000 7f37d2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc072ff0 con 0x7f37cc143040 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37c3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f37cc06c890 0x7f37cc142930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41122/0 (socket says 172.21.15.46:41122) 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37c3e00640 1 -- 172.21.15.46:0/3027353101 learned_addr learned my addr 172.21.15.46:0/3027353101 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37d0e00640 1 --2- 172.21.15.46:0/3027353101 >> v2:172.21.15.142:3300/0 conn(0x7f37cc15d460 0x7f37cc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601654667 0 0) 0x7f37cc143690 con 0x7f37cc06c890 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37b0007930 con 0x7f37cc06c890 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.513+0000 7f37d1800640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 0x7f37cc0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2086284118 0 0) 0x7f37b0007930 con 0x7f37cc06c890 2024-04-26T23:38:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 msgr2=0x7f37cc0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 0x7f37cc0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 >> v2:172.21.15.142:3300/0 conn(0x7f37cc15d460 msgr2=0x7f37cc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 --2- 172.21.15.46:0/3027353101 >> v2:172.21.15.142:3300/0 conn(0x7f37cc15d460 0x7f37cc06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc0732d0 con 0x7f37cc06c890 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f37b4001690 con 0x7f37cc06c890 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 >> v1:172.21.15.46:6789/0 conn(0x7f37cc06c890 legacy=0x7f37cc142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b0007dc0 0x7f37b00081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 0x7f37b0008c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b0009150 0x7f37b00095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc073920 con 0x7f37b0007dc0 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc072ff0 con 0x7f37b0009150 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37b000fdd0 con 0x7f37cc143040 2024-04-26T23:38:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.514+0000 7f37d0e00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b0007dc0 0x7f37b00081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d1800640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b0009150 0x7f37b00095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d0e00640 1 -- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 msgr2=0x7f37b0008c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d0e00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37cc143040 0x7f37b0008c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d0e00640 1 -- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b0009150 msgr2=0x7f37b00095d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d0e00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37b0009150 0x7f37b00095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.515+0000 7f37d0e00640 1 -- 172.21.15.46:0/3027353101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b0007930 con 0x7f37b0007dc0 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.516+0000 7f37d0e00640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b0007dc0 0x7f37b00081e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009fc0 tx=0x7f37b0009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f37829e6de9e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.516+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc009c10 con 0x7f37b0007dc0 2024-04-26T23:38:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.516+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc036070 con 0x7f37b0007dc0 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.516+0000 7f37cbe00640 1 -- 172.21.15.46:0/3027353101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc02d070 con 0x7f37b0007dc0 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 -- 172.21.15.46:0/3027353101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b0007dc0 msgr2=0x7f37b00081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 --2- 172.21.15.46:0/3027353101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37b0007dc0 0x7f37b00081e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009fc0 tx=0x7f37b0009c40 comp rx=0 tx=0).stop 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d1800640 1 -- 172.21.15.46:0/3027353101 reap_dead start 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 -- 172.21.15.46:0/3027353101 shutdown_connections 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 -- 172.21.15.46:0/3027353101 >> 172.21.15.46:0/3027353101 conn(0x7f37cc0b3930 msgr2=0x7f37cc0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 -- 172.21.15.46:0/3027353101 shutdown_connections 2024-04-26T23:38:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.517+0000 7f37d2200640 1 -- 172.21.15.46:0/3027353101 wait complete. 2024-04-26T23:38:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.519+0000 7f37d2200640 1 Processor -- start 2024-04-26T23:38:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.519+0000 7f37d2200640 1 -- start start 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b0007dc0 0x7f37cc15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc13ed80 0x7f37cc13f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc073db0 con 0x7f37cc0ce800 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc15d840 con 0x7f37cc13ed80 2024-04-26T23:38:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.520+0000 7f37d2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0d0270 con 0x7f37b0007dc0 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41984/0 (socket says 172.21.15.46:41984) 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 -- 172.21.15.46:0/484041170 learned_addr learned my addr 172.21.15.46:0/484041170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37d0e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b0007dc0 0x7f37cc15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37d1800640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc13ed80 0x7f37cc13f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 -- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b0007dc0 msgr2=0x7f37cc15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b0007dc0 0x7f37cc15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 -- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc13ed80 msgr2=0x7f37cc13f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc13ed80 0x7f37cc13f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 -- 172.21.15.46:0/484041170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc15cbc0 con 0x7f37cc0ce800 2024-04-26T23:38:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.521+0000 7f37c3e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c700 tx=0x7f37b400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.522+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4018070 con 0x7f37cc0ce800 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.522+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b40072f0 con 0x7f37cc0ce800 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.522+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40047e0 con 0x7f37cc0ce800 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.522+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc13d6d0 con 0x7f37cc0ce800 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.522+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc13dc90 con 0x7f37cc0ce800 2024-04-26T23:38:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.523+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc06a4a0 con 0x7f37cc0ce800 2024-04-26T23:38:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.524+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37b401b070 con 0x7f37cc0ce800 2024-04-26T23:38:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.526+0000 7f37c9600640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 0x7f37a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.526+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f37b408b450 con 0x7f37cc0ce800 2024-04-26T23:38:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.526+0000 7f37d0e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 0x7f37a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.528+0000 7f37d0e00640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 0x7f37a806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37bc035000 tx=0x7f37bc02f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.530+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37cc06a4a0 con 0x7f37cc0ce800 2024-04-26T23:38:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.734+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f37cc069ac0 con 0x7f37cc0ce800 2024-04-26T23:38:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.735+0000 7f37c9600640 1 -- 172.21.15.46:0/484041170 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f37b40593e0 con 0x7f37cc0ce800 2024-04-26T23:38:31.738 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 msgr2=0x7f37a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 0x7f37a806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37bc035000 tx=0x7f37bc02f000 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 msgr2=0x7f37cc13c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c700 tx=0x7f37b400cbc0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 shutdown_connections 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37a806c8f0 0x7f37a806edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc13ed80 0x7f37cc13f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37cc0ce800 0x7f37cc13c2c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.737+0000 7f37d2200640 1 --2- 172.21.15.46:0/484041170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37b0007dc0 0x7f37cc15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.738+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 >> 172.21.15.46:0/484041170 conn(0x7f37cc0b3930 msgr2=0x7f37cc0d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.738+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 shutdown_connections 2024-04-26T23:38:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.738+0000 7f37d2200640 1 -- 172.21.15.46:0/484041170 wait complete. 2024-04-26T23:38:31.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:38:31.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.965+0000 7fb80f000640 1 Processor -- start 2024-04-26T23:38:31.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.965+0000 7fb80f000640 1 -- start start 2024-04-26T23:38:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.967+0000 7fb80f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb80815edf0 0x7fb8080cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.967+0000 7fb80f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080cd770 0x7fb8080c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.967+0000 7fb80f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8080b03c0 con 0x7fb80815f1d0 2024-04-26T23:38:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.967+0000 7fb80f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb8080cf5c0 con 0x7fb8080cd770 2024-04-26T23:38:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.967+0000 7fb80f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb8080cb680 con 0x7fb80815edf0 2024-04-26T23:38:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080cd770 0x7fb8080c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080cd770 0x7fb8080c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36138/0 (socket says 172.21.15.46:36138) 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 -- 172.21.15.46:0/3156024235 learned_addr learned my addr 172.21.15.46:0/3156024235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb80815f1d0 0x7fb8080cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42246/0 (socket says 172.21.15.46:42246) 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80dc00640 1 --2- 172.21.15.46:0/3156024235 >> v2:172.21.15.142:3300/0 conn(0x7fb80815edf0 0x7fb8080cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 -- 172.21.15.46:0/3156024235 >> v2:172.21.15.142:3300/0 conn(0x7fb80815edf0 msgr2=0x7fb8080cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 --2- 172.21.15.46:0/3156024235 >> v2:172.21.15.142:3300/0 conn(0x7fb80815edf0 0x7fb8080cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 -- 172.21.15.46:0/3156024235 >> v1:172.21.15.46:6789/0 conn(0x7fb80815f1d0 legacy=0x7fb8080cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb80e600640 1 -- 172.21.15.46:0/3156024235 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8080cb960 con 0x7fb8080cd770 2024-04-26T23:38:31.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.968+0000 7fb807e00640 1 -- 172.21.15.46:0/3156024235 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158675439 0 0) 0x7fb8080b03c0 con 0x7fb80815f1d0 2024-04-26T23:38:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.969+0000 7fb80e600640 1 --2- 172.21.15.46:0/3156024235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080cd770 0x7fb8080c5940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8080d0a00 tx=0x7fb80002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70b5c449983786e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.970+0000 7fb807e00640 1 -- 172.21.15.46:0/3156024235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800036070 con 0x7fb8080cd770 2024-04-26T23:38:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.970+0000 7fb807e00640 1 -- 172.21.15.46:0/3156024235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800031600 con 0x7fb8080cd770 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.970+0000 7fb807e00640 1 -- 172.21.15.46:0/3156024235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8000319f0 con 0x7fb8080cd770 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.970+0000 7fb80f000640 1 -- 172.21.15.46:0/3156024235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080cd770 msgr2=0x7fb8080c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.970+0000 7fb80f000640 1 --2- 172.21.15.46:0/3156024235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080cd770 0x7fb8080c5940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8080d0a00 tx=0x7fb80002a7c0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 -- 172.21.15.46:0/3156024235 shutdown_connections 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 --2- 172.21.15.46:0/3156024235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080cd770 0x7fb8080c5940 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 --2- 172.21.15.46:0/3156024235 >> v2:172.21.15.142:3300/0 conn(0x7fb80815edf0 0x7fb8080cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 -- 172.21.15.46:0/3156024235 >> 172.21.15.46:0/3156024235 conn(0x7fb8080b26d0 msgr2=0x7fb8080b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 -- 172.21.15.46:0/3156024235 shutdown_connections 2024-04-26T23:38:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.971+0000 7fb80f000640 1 -- 172.21.15.46:0/3156024235 wait complete. 2024-04-26T23:38:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.973+0000 7fb80f000640 1 Processor -- start 2024-04-26T23:38:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.973+0000 7fb80f000640 1 -- start start 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 0x7fb808073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 0x7fb8080c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8080cc200 con 0x7fb80815bb10 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8080cbf20 con 0x7fb8080cd770 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.974+0000 7fb80f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb8080cb1a0 con 0x7fb80815edf0 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36144/0 (socket says 172.21.15.46:36144) 2024-04-26T23:38:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 -- 172.21.15.46:0/1975871731 learned_addr learned my addr 172.21.15.46:0/1975871731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80dc00640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 0x7fb808073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80d200640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 0x7fb8080c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 -- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 msgr2=0x7fb8080c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 0x7fb8080c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 -- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 msgr2=0x7fb808073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 0x7fb808073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80e600640 1 -- 172.21.15.46:0/1975871731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb808073c70 con 0x7fb80815bb10 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80d200640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 0x7fb8080c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.975+0000 7fb80dc00640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 0x7fb808073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb80e600640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb80002adb0 tx=0x7fb800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800036050 con 0x7fb80815bb10 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800042070 con 0x7fb80815bb10 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb808072690 con 0x7fb80815bb10 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800009510 con 0x7fb80815bb10 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.976+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb808072c50 con 0x7fb80815bb10 2024-04-26T23:38:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.977+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8080693b0 con 0x7fb80815bb10 2024-04-26T23:38:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.979+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb800004470 con 0x7fb80815bb10 2024-04-26T23:38:31.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.980+0000 7fb805600640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:31.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.980+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb8000ace80 con 0x7fb80815bb10 2024-04-26T23:38:31.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.981+0000 7fb80dc00640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.982+0000 7fb80dc00640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 0x7fb7e806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc006fd0 tx=0x7fb7fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:31.985+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb80007ae10 con 0x7fb80815bb10 2024-04-26T23:38:32.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.202+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb80815d190 con 0x7fb80815bb10 2024-04-26T23:38:32.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.203+0000 7fb805600640 1 -- 172.21.15.46:0/1975871731 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fb80007a710 con 0x7fb80815bb10 2024-04-26T23:38:32.206 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 msgr2=0x7fb7e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 0x7fb7e806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc006fd0 tx=0x7fb7fc008040 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 msgr2=0x7fb808071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb80002adb0 tx=0x7fb800004a40 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 shutdown_connections 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80815bb10 0x7fb808071d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb80815edf0 0x7fb8080c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 --2- 172.21.15.46:0/1975871731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8080cd770 0x7fb808073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.205+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 >> 172.21.15.46:0/1975871731 conn(0x7fb8080b26d0 msgr2=0x7fb8080c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.206+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 shutdown_connections 2024-04-26T23:38:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.206+0000 7fb80f000640 1 -- 172.21.15.46:0/1975871731 wait complete. 2024-04-26T23:38:32.227 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T23:38:32.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:32.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T23:38:32.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:38:32.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:38:32.322 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T23:38:32.374 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:38:32.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:38:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.590+0000 7fe225400640 1 Processor -- start 2024-04-26T23:38:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.590+0000 7fe225400640 1 -- start start 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.591+0000 7fe225400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22015f1d0 0x7fe2200cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.591+0000 7fe225400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2200cd770 0x7fe2200c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.591+0000 7fe225400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2200b03c0 con 0x7fe22015edf0 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.591+0000 7fe225400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe2200cf5c0 con 0x7fe2200cd770 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.591+0000 7fe225400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe2200cb680 con 0x7fe22015f1d0 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe21fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe22015edf0 0x7fe2200cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42262/0 (socket says 172.21.15.46:42262) 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe21fe00640 1 -- 172.21.15.46:0/2973169546 learned_addr learned my addr 172.21.15.46:0/2973169546 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> v2:172.21.15.142:3300/0 conn(0x7fe2200cd770 0x7fe2200c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe21f400640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22015f1d0 0x7fe2200cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024742557 0 0) 0x7fe2200b03c0 con 0x7fe22015edf0 2024-04-26T23:38:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.592+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe204003520 con 0x7fe22015edf0 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2996878716 0 0) 0x7fe204003520 con 0x7fe22015edf0 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22015f1d0 msgr2=0x7fe2200cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22015f1d0 0x7fe2200cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 >> v2:172.21.15.142:3300/0 conn(0x7fe2200cd770 msgr2=0x7fe2200c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 --2- 172.21.15.46:0/2973169546 >> v2:172.21.15.142:3300/0 conn(0x7fe2200cd770 0x7fe2200c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2200cb960 con 0x7fe22015edf0 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe210001690 con 0x7fe22015edf0 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 >> v1:172.21.15.46:6789/0 conn(0x7fe22015edf0 legacy=0x7fe2200cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2040039b0 0x7fe204003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe204004360 0x7fe204004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe20400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200cf5c0 con 0x7fe204004f60 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200cb680 con 0x7fe204004360 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe20400bf00 con 0x7fe2040039b0 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> v2:172.21.15.142:3300/0 conn(0x7fe2200cd770 0x7fe2200c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe20400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.593+0000 7fe21fe00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2040039b0 0x7fe204003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe21f400640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe204004360 0x7fe204004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe224a00640 1 -- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2040039b0 msgr2=0x7fe204003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2040039b0 0x7fe204003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe224a00640 1 -- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe204004360 msgr2=0x7fe204004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe204004360 0x7fe204004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.594+0000 7fe224a00640 1 -- 172.21.15.46:0/2973169546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe204003520 con 0x7fe204004f60 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.595+0000 7fe224a00640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe20400b9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2180073e0 tx=0x7fe204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93345240536d7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.595+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218009c70 con 0x7fe204004f60 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.595+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21802c680 con 0x7fe204004f60 2024-04-26T23:38:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.595+0000 7fe21ea00640 1 -- 172.21.15.46:0/2973169546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21802ca30 con 0x7fe204004f60 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 -- 172.21.15.46:0/2973169546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 msgr2=0x7fe20400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 --2- 172.21.15.46:0/2973169546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe20400b9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2180073e0 tx=0x7fe204005830 comp rx=0 tx=0).stop 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe224a00640 1 -- 172.21.15.46:0/2973169546 reap_dead start 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 -- 172.21.15.46:0/2973169546 shutdown_connections 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 -- 172.21.15.46:0/2973169546 >> 172.21.15.46:0/2973169546 conn(0x7fe2200b26d0 msgr2=0x7fe2200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 -- 172.21.15.46:0/2973169546 shutdown_connections 2024-04-26T23:38:32.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.596+0000 7fe225400640 1 -- 172.21.15.46:0/2973169546 wait complete. 2024-04-26T23:38:32.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.598+0000 7fe225400640 1 Processor -- start 2024-04-26T23:38:32.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.598+0000 7fe225400640 1 -- start start 2024-04-26T23:38:32.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2200b5e90 0x7fe2201510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 0x7fe220151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200cc200 con 0x7fe204004f60 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200cbf20 con 0x7fe2200b6310 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe225400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe2200cb1a0 con 0x7fe2200b5e90 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36168/0 (socket says 172.21.15.46:36168) 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.599+0000 7fe21fe00640 1 -- 172.21.15.46:0/455607405 learned_addr learned my addr 172.21.15.46:0/455607405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21f400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2200b5e90 0x7fe2201510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe224a00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 0x7fe220151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 -- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2200b5e90 msgr2=0x7fe2201510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2200b5e90 0x7fe2201510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 -- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 msgr2=0x7fe220151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 0x7fe220151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 -- 172.21.15.46:0/455607405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2200c8900 con 0x7fe204004f60 2024-04-26T23:38:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe224a00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 0x7fe220151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe21fe00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e7b0 tx=0x7fe21000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.600+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000c8d0 con 0x7fe204004f60 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.601+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe210018070 con 0x7fe204004f60 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.601+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000ccd0 con 0x7fe204004f60 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.601+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2200c8be0 con 0x7fe204004f60 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.601+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2200c91a0 con 0x7fe204004f60 2024-04-26T23:38:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.602+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2200693b0 con 0x7fe204004f60 2024-04-26T23:38:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.603+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe21001d460 con 0x7fe204004f60 2024-04-26T23:38:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.604+0000 7fe217e00640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.604+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe21008beb0 con 0x7fe204004f60 2024-04-26T23:38:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.605+0000 7fe21f400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:32.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.606+0000 7fe21f400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 0x7fe1fc06ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe220162460 tx=0x7fe208008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.609+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe210059ed0 con 0x7fe204004f60 2024-04-26T23:38:32.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.860+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2200cede0 con 0x7fe204004f60 2024-04-26T23:38:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.861+0000 7fe217e00640 1 -- 172.21.15.46:0/455607405 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fe2100597d0 con 0x7fe204004f60 2024-04-26T23:38:32.864 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:32.865 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:38:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:38:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 msgr2=0x7fe1fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 0x7fe1fc06ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe220162460 tx=0x7fe208008040 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 msgr2=0x7fe2200b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e7b0 tx=0x7fe21000ec70 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 shutdown_connections 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2200b6310 0x7fe220151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2200b5e90 0x7fe2201510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 --2- 172.21.15.46:0/455607405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe204004f60 0x7fe2200b3f50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.864+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 >> 172.21.15.46:0/455607405 conn(0x7fe2200b26d0 msgr2=0x7fe2200b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.865+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 shutdown_connections 2024-04-26T23:38:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:32.865+0000 7fe225400640 1 -- 172.21.15.46:0/455607405 wait complete. 2024-04-26T23:38:32.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.090+0000 7fc69b200640 1 Processor -- start 2024-04-26T23:38:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.090+0000 7fc69b200640 1 -- start start 2024-04-26T23:38:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.092+0000 7fc69b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.092+0000 7fc69b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc69c160910 0x7fc69c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.092+0000 7fc69b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c073ad0 con 0x7fc69c1466b0 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.092+0000 7fc69b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c146a90 con 0x7fc69c160910 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.092+0000 7fc69b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c073d70 con 0x7fc69c0720e0 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc699e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc69c1466b0 0x7fc69c06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42274/0 (socket says 172.21.15.46:42274) 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc699e00640 1 -- 172.21.15.46:0/2162348012 learned_addr learned my addr 172.21.15.46:0/2162348012 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc69a800640 1 --2- 172.21.15.46:0/2162348012 >> v2:172.21.15.46:3300/0 conn(0x7fc69c160910 0x7fc69c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc699400640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc69a800640 1 -- 172.21.15.46:0/2162348012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 msgr2=0x7fc69c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc69a800640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc69a800640 1 -- 172.21.15.46:0/2162348012 >> v1:172.21.15.46:6789/0 conn(0x7fc69c1466b0 legacy=0x7fc69c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc69a800640 1 -- 172.21.15.46:0/2162348012 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c074050 con 0x7fc69c160910 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.093+0000 7fc698a00640 1 -- 172.21.15.46:0/2162348012 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480203434 0 0) 0x7fc69c073ad0 con 0x7fc69c1466b0 2024-04-26T23:38:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.095+0000 7fc69a800640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c160910 0x7fc69c160da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc694009d90 tx=0x7fc69402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313d1a9b0047b79c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.095+0000 7fc698a00640 1 -- 172.21.15.46:0/2162348012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694007740 con 0x7fc69c160910 2024-04-26T23:38:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.095+0000 7fc698a00640 1 -- 172.21.15.46:0/2162348012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6940078e0 con 0x7fc69c160910 2024-04-26T23:38:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.095+0000 7fc698a00640 1 -- 172.21.15.46:0/2162348012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694007cb0 con 0x7fc69c160910 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 -- 172.21.15.46:0/2162348012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c160910 msgr2=0x7fc69c160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c160910 0x7fc69c160da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc694009d90 tx=0x7fc69402a660 comp rx=0 tx=0).stop 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 -- 172.21.15.46:0/2162348012 shutdown_connections 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c160910 0x7fc69c160da0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 --2- 172.21.15.46:0/2162348012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 -- 172.21.15.46:0/2162348012 >> 172.21.15.46:0/2162348012 conn(0x7fc69c0b26d0 msgr2=0x7fc69c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 -- 172.21.15.46:0/2162348012 shutdown_connections 2024-04-26T23:38:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.096+0000 7fc69b200640 1 -- 172.21.15.46:0/2162348012 wait complete. 2024-04-26T23:38:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.098+0000 7fc69b200640 1 Processor -- start 2024-04-26T23:38:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.098+0000 7fc69b200640 1 -- start start 2024-04-26T23:38:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c160910 0x7fc69c144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0cdda0 con 0x7fc69c1466b0 2024-04-26T23:38:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0cb470 con 0x7fc69c160910 2024-04-26T23:38:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.099+0000 7fc69b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0cb990 con 0x7fc69c0720e0 2024-04-26T23:38:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36186/0 (socket says 172.21.15.46:36186) 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 -- 172.21.15.46:0/1608457430 learned_addr learned my addr 172.21.15.46:0/1608457430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc69a800640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c160910 0x7fc69c144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 -- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 msgr2=0x7fc69c0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 -- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c160910 msgr2=0x7fc69c144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c160910 0x7fc69c144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.100+0000 7fc699400640 1 -- 172.21.15.46:0/1608457430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0ca480 con 0x7fc69c1466b0 2024-04-26T23:38:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc699400640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d800 tx=0x7fc68800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68800b6c0 con 0x7fc69c1466b0 2024-04-26T23:38:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc688019070 con 0x7fc69c1466b0 2024-04-26T23:38:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c0c7fb0 con 0x7fc69c1466b0 2024-04-26T23:38:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68800ba80 con 0x7fc69c1466b0 2024-04-26T23:38:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.101+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0c8570 con 0x7fc69c1466b0 2024-04-26T23:38:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.103+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c0693b0 con 0x7fc69c1466b0 2024-04-26T23:38:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.104+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc68800bc20 con 0x7fc69c1466b0 2024-04-26T23:38:33.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.105+0000 7fc692000640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 0x7fc67c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.105+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc68808c420 con 0x7fc69c1466b0 2024-04-26T23:38:33.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.106+0000 7fc699e00640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 0x7fc67c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.107+0000 7fc699e00640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 0x7fc67c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc68c000f60 tx=0x7fc68c007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.109+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc68805a3b0 con 0x7fc69c1466b0 2024-04-26T23:38:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.318+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc69c0b61e0 con 0x7fc69c1466b0 2024-04-26T23:38:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.320+0000 7fc692000640 1 -- 172.21.15.46:0/1608457430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7fc688059cb0 con 0x7fc69c1466b0 2024-04-26T23:38:33.324 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:33.324 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 msgr2=0x7fc67c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 0x7fc67c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc68c000f60 tx=0x7fc68c007d50 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 msgr2=0x7fc69c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d800 tx=0x7fc68800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 shutdown_connections 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67c06c8f0 0x7fc67c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c160910 0x7fc69c144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c1466b0 0x7fc69c1599b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 --2- 172.21.15.46:0/1608457430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0720e0 0x7fc69c0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 >> 172.21.15.46:0/1608457430 conn(0x7fc69c0b26d0 msgr2=0x7fc69c0726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 shutdown_connections 2024-04-26T23:38:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.323+0000 7fc69b200640 1 -- 172.21.15.46:0/1608457430 wait complete. 2024-04-26T23:38:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:37:05.921840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:37:07.172664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:37:10.790040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:38:33.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:33.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.554+0000 7ffb51800640 1 Processor -- start 2024-04-26T23:38:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.555+0000 7ffb51800640 1 -- start start 2024-04-26T23:38:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.556+0000 7ffb51800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffb4c0b5fb0 0x7ffb4c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.556+0000 7ffb51800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb4c0d07d0 con 0x7ffb4c162aa0 2024-04-26T23:38:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.556+0000 7ffb51800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb4c0d0ab0 con 0x7ffb4c1626c0 2024-04-26T23:38:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.556+0000 7ffb51800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c0d0d90 con 0x7ffb4c0b5fb0 2024-04-26T23:38:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.557+0000 7ffb50e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb4c162aa0 0x7ffb4c0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42290/0 (socket says 172.21.15.46:42290) 2024-04-26T23:38:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.557+0000 7ffb50e00640 1 -- 172.21.15.46:0/319648252 learned_addr learned my addr 172.21.15.46:0/319648252 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.557+0000 7ffb4b400640 1 --2- 172.21.15.46:0/319648252 >> v2:172.21.15.142:3300/0 conn(0x7ffb4c0b5fb0 0x7ffb4c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.557+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365176927 0 0) 0x7ffb4c0d07d0 con 0x7ffb4c162aa0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.557+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb30003520 con 0x7ffb4c162aa0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1294968310 0 0) 0x7ffb30003520 con 0x7ffb4c162aa0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 >> v2:172.21.15.142:3300/0 conn(0x7ffb4c0b5fb0 msgr2=0x7ffb4c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 --2- 172.21.15.46:0/319648252 >> v2:172.21.15.142:3300/0 conn(0x7ffb4c0b5fb0 0x7ffb4c0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 >> v1:172.21.15.142:6789/0 conn(0x7ffb4c1626c0 legacy=0x7ffb4c0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c0d2c80 con 0x7ffb4c162aa0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ffb440019f0 con 0x7ffb4c162aa0 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 >> v1:172.21.15.46:6789/0 conn(0x7ffb4c162aa0 legacy=0x7ffb4c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb30003a00 0x7ffb30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb30004800 0x7ffb30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 0x7ffb3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d0d90 con 0x7ffb300051c0 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3000c0c0 con 0x7ffb30003a00 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3000c2c0 con 0x7ffb30004800 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.558+0000 7ffb50e00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 0x7ffb3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb4be00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb30003a00 0x7ffb30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb4b400640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb30004800 0x7ffb30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb50e00640 1 -- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb30004800 msgr2=0x7ffb30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb50e00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb30004800 0x7ffb30004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb50e00640 1 -- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb30003a00 msgr2=0x7ffb30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb50e00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb30003a00 0x7ffb30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.559+0000 7ffb50e00640 1 -- 172.21.15.46:0/319648252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb30003520 con 0x7ffb300051c0 2024-04-26T23:38:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.560+0000 7ffb50e00640 1 -- 172.21.15.46:0/319648252 reap_dead start 2024-04-26T23:38:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.560+0000 7ffb50e00640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 0x7ffb3000b9c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400a7b0 tx=0x7ffb30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a0c552c5a78da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.560+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400a3d0 con 0x7ffb300051c0 2024-04-26T23:38:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.560+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4402f070 con 0x7ffb300051c0 2024-04-26T23:38:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.560+0000 7ffb4aa00640 1 -- 172.21.15.46:0/319648252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402bc20 con 0x7ffb300051c0 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 -- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 msgr2=0x7ffb3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 0x7ffb3000b9c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400a7b0 tx=0x7ffb30005830 comp rx=0 tx=0).stop 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 -- 172.21.15.46:0/319648252 shutdown_connections 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 --2- 172.21.15.46:0/319648252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb300051c0 0x7ffb3000b9c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 -- 172.21.15.46:0/319648252 >> 172.21.15.46:0/319648252 conn(0x7ffb4c09e2a0 msgr2=0x7ffb4c0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 -- 172.21.15.46:0/319648252 shutdown_connections 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.561+0000 7ffb51800640 1 -- 172.21.15.46:0/319648252 wait complete. 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.563+0000 7ffb51800640 1 Processor -- start 2024-04-26T23:38:33.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.563+0000 7ffb51800640 1 -- start start 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb300051c0 0x7ffb4c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb4c0b2b50 0x7ffb4c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0b4b50 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0b49d0 con 0x7ffb4c0b2b50 2024-04-26T23:38:33.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb51800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d5850 con 0x7ffb300051c0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb50e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb50e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36210/0 (socket says 172.21.15.46:36210) 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb50e00640 1 -- 172.21.15.46:0/2572336451 learned_addr learned my addr 172.21.15.46:0/2572336451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb4be00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb300051c0 0x7ffb4c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.564+0000 7ffb4b400640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb4c0b2b50 0x7ffb4c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 -- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb300051c0 msgr2=0x7ffb4c0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb300051c0 0x7ffb4c0cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 -- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb4c0b2b50 msgr2=0x7ffb4c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb4c0b2b50 0x7ffb4c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 -- 172.21.15.46:0/2572336451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c0ce940 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb50e00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffb4402b470 tx=0x7ffb4402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402bbd0 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4402bd70 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44034670 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.565+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c0cec20 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.566+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c0cef00 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.567+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4c0693b0 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.568+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb44034810 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.569+0000 7ffb43e00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 0x7ffb2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:33.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.569+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffb440abef0 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.569+0000 7ffb4be00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 0x7ffb2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.570+0000 7ffb4be00640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 0x7ffb2c06ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005410 tx=0x7ffb3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.573+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb44046060 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.776+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb4c161ee0 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.777+0000 7ffb43e00640 1 -- 172.21.15.46:0/2572336451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ffb44079e80 con 0x7ffb4c1f54d0 2024-04-26T23:38:33.780 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.779+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 msgr2=0x7ffb2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.779+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 0x7ffb2c06ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005410 tx=0x7ffb3c009040 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.779+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 msgr2=0x7ffb4c15fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.779+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffb4402b470 tx=0x7ffb4402d070 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 shutdown_connections 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2c06c920 0x7ffb2c06ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4c1f54d0 0x7ffb4c15fe30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb4c0b2b50 0x7ffb4c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 --2- 172.21.15.46:0/2572336451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb300051c0 0x7ffb4c0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 >> 172.21.15.46:0/2572336451 conn(0x7ffb4c09e2a0 msgr2=0x7ffb4c1636d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 shutdown_connections 2024-04-26T23:38:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:33.780+0000 7ffb51800640 1 -- 172.21.15.46:0/2572336451 wait complete. 2024-04-26T23:38:33.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:38:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.011+0000 7f1391800640 1 Processor -- start 2024-04-26T23:38:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.012+0000 7f1391800640 1 -- start start 2024-04-26T23:38:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.013+0000 7f1391800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f138c1597e0 0x7f138c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.013+0000 7f1391800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f138c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.013+0000 7f1391800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138c073120 con 0x7f138c071f10 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.013+0000 7f1391800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f138c0b60d0 con 0x7f138c1597e0 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.013+0000 7f1391800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f138c159bc0 con 0x7f138c0b5730 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.014+0000 7f138b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f138c071f10 0x7f138c072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42304/0 (socket says 172.21.15.46:42304) 2024-04-26T23:38:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.014+0000 7f138b400640 1 -- 172.21.15.46:0/3936482844 learned_addr learned my addr 172.21.15.46:0/3936482844 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.014+0000 7f138a000640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f138c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.014+0000 7f138aa00640 1 --2- 172.21.15.46:0/3936482844 >> v2:172.21.15.142:3300/0 conn(0x7f138c1597e0 0x7f138c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.014+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277749327 0 0) 0x7f138c073120 con 0x7f138c071f10 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1370003520 con 0x7f138c071f10 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4237280417 0 0) 0x7f1370003520 con 0x7f138c071f10 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 msgr2=0x7f138c0b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f138c0b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 >> v2:172.21.15.142:3300/0 conn(0x7f138c1597e0 msgr2=0x7f138c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 --2- 172.21.15.46:0/3936482844 >> v2:172.21.15.142:3300/0 conn(0x7f138c1597e0 0x7f138c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c072b60 con 0x7f138c071f10 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1384001670 con 0x7f138c071f10 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 >> v1:172.21.15.46:6789/0 conn(0x7f138c071f10 legacy=0x7f138c072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13700039b0 0x7f1370003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f1370004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1370004d40 0x7f13700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f138c159bc0 con 0x7f13700039b0 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f137000b9c0 con 0x7f1370004d40 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f137000bb40 con 0x7f138c0b5730 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.015+0000 7f138aa00640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13700039b0 0x7f1370003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138a000640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f1370004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138b400640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1370004d40 0x7f13700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138aa00640 1 -- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 msgr2=0x7f1370004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138aa00640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0b5730 0x7f1370004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138aa00640 1 -- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1370004d40 msgr2=0x7f13700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138aa00640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1370004d40 0x7f13700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.016+0000 7f138aa00640 1 -- 172.21.15.46:0/3936482844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1370003520 con 0x7f13700039b0 2024-04-26T23:38:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.017+0000 7f138aa00640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13700039b0 0x7f1370003dd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f137c009ac0 tx=0x7f1370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51bf53c88969622 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.017+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c036070 con 0x7f13700039b0 2024-04-26T23:38:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.017+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c031400 con 0x7f13700039b0 2024-04-26T23:38:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.017+0000 7f1389600640 1 -- 172.21.15.46:0/3936482844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c009690 con 0x7f13700039b0 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 -- 172.21.15.46:0/3936482844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13700039b0 msgr2=0x7f1370003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 --2- 172.21.15.46:0/3936482844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13700039b0 0x7f1370003dd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f137c009ac0 tx=0x7f1370005830 comp rx=0 tx=0).stop 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f138b400640 1 -- 172.21.15.46:0/3936482844 reap_dead start 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 -- 172.21.15.46:0/3936482844 shutdown_connections 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 -- 172.21.15.46:0/3936482844 >> 172.21.15.46:0/3936482844 conn(0x7f138c06c980 msgr2=0x7f138c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 -- 172.21.15.46:0/3936482844 shutdown_connections 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.018+0000 7f1391800640 1 -- 172.21.15.46:0/3936482844 wait complete. 2024-04-26T23:38:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.020+0000 7f1391800640 1 Processor -- start 2024-04-26T23:38:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.020+0000 7f1391800640 1 -- start start 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13700039b0 0x7f138c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0c6c10 0x7f138c0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0b43b0 con 0x7f138c1622b0 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0b40d0 con 0x7f13700039b0 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.021+0000 7f1391800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0aee20 con 0x7f138c0c6c10 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36220/0 (socket says 172.21.15.46:36220) 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 -- 172.21.15.46:0/1263959184 learned_addr learned my addr 172.21.15.46:0/1263959184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138aa00640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13700039b0 0x7f138c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138b400640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0c6c10 0x7f138c0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 -- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0c6c10 msgr2=0x7f138c0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0c6c10 0x7f138c0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 -- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13700039b0 msgr2=0x7f138c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13700039b0 0x7f138c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.022+0000 7f138a000640 1 -- 172.21.15.46:0/1263959184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c1475c0 con 0x7f138c1622b0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f138a000640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1374007f30 tx=0x7f1374011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137401a020 con 0x7f138c1622b0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1374016470 con 0x7f138c1622b0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374016860 con 0x7f138c1622b0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c147860 con 0x7f138c1622b0 2024-04-26T23:38:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.023+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c0c79a0 con 0x7f138c1622b0 2024-04-26T23:38:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.024+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c069530 con 0x7f138c1622b0 2024-04-26T23:38:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.025+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1374005d10 con 0x7f138c1622b0 2024-04-26T23:38:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.026+0000 7f1382a00640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 0x7f136806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.026+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f137408cf60 con 0x7f138c1622b0 2024-04-26T23:38:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.027+0000 7f138aa00640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 0x7f136806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.028+0000 7f138aa00640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 0x7f136806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f137c033000 tx=0x7f137c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.031+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1374024030 con 0x7f138c1622b0 2024-04-26T23:38:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.236+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f138c1626b0 con 0x7f138c1622b0 2024-04-26T23:38:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.237+0000 7f1382a00640 1 -- 172.21.15.46:0/1263959184 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f137405aef0 con 0x7f138c1622b0 2024-04-26T23:38:34.240 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 msgr2=0x7f136806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 0x7f136806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f137c033000 tx=0x7f137c008030 comp rx=0 tx=0).stop 2024-04-26T23:38:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 msgr2=0x7f138c0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1374007f30 tx=0x7f1374011660 comp rx=0 tx=0).stop 2024-04-26T23:38:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 shutdown_connections 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136806c920 0x7f136806ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138c0c6c10 0x7f138c0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f138c1622b0 0x7f138c0cba90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 --2- 172.21.15.46:0/1263959184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13700039b0 0x7f138c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 >> 172.21.15.46:0/1263959184 conn(0x7f138c06c980 msgr2=0x7f138c158610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 shutdown_connections 2024-04-26T23:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.240+0000 7f1391800640 1 -- 172.21.15.46:0/1263959184 wait complete. 2024-04-26T23:38:34.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:38:34.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.480+0000 7f60ea600640 1 Processor -- start 2024-04-26T23:38:34.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.480+0000 7f60ea600640 1 -- start start 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60ea600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60e415e220 0x7f60e406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60ea600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e40d07e0 con 0x7f60e415d730 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60ea600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e406eb90 con 0x7f60e40b53b0 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60ea600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40b4310 con 0x7f60e415e220 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60e3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60e415d730 0x7f60e40d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42310/0 (socket says 172.21.15.46:42310) 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.481+0000 7f60e3400640 1 -- 172.21.15.46:0/4058477197 learned_addr learned my addr 172.21.15.46:0/4058477197 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4058477197 >> v2:172.21.15.142:3300/0 conn(0x7f60e415e220 0x7f60e406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852398111 0 0) 0x7f60e40d07e0 con 0x7f60e415d730 2024-04-26T23:38:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c8007930 con 0x7f60e415d730 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3993240168 0 0) 0x7f60c8007930 con 0x7f60e415d730 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 >> v2:172.21.15.142:3300/0 conn(0x7f60e415e220 msgr2=0x7f60e406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 --2- 172.21.15.46:0/4058477197 >> v2:172.21.15.142:3300/0 conn(0x7f60e415e220 0x7f60e406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 >> v1:172.21.15.142:6789/0 conn(0x7f60e40b53b0 legacy=0x7f60e415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e40b45f0 con 0x7f60e415d730 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.482+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f60d4001670 con 0x7f60e415d730 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 >> v1:172.21.15.46:6789/0 conn(0x7f60e415d730 legacy=0x7f60e40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60c8007e10 0x7f60c8008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c8008c10 0x7f60c8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 0x7f60c800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40b4310 con 0x7f60c80095d0 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60c80104d0 con 0x7f60c8007e10 2024-04-26T23:38:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80106d0 con 0x7f60c8008c10 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 0x7f60c800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3400640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60c8007e10 0x7f60c8008230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e2a00640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c8008c10 0x7f60c8009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 -- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c8008c10 msgr2=0x7f60c8009090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c8008c10 0x7f60c8009090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 -- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60c8007e10 msgr2=0x7f60c8008230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60c8007e10 0x7f60c8008230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.483+0000 7f60e3e00640 1 -- 172.21.15.46:0/4058477197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8007930 con 0x7f60c80095d0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.484+0000 7f60e3e00640 1 -- 172.21.15.46:0/4058477197 reap_dead start 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 0x7f60c800fdd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009fc0 tx=0x7f60c8009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cbace60e5d0bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc035020 con 0x7f60c80095d0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc003b90 con 0x7f60c80095d0 2024-04-26T23:38:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60e2000640 1 -- 172.21.15.46:0/4058477197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc009c10 con 0x7f60c80095d0 2024-04-26T23:38:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60ea600640 1 -- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 msgr2=0x7f60c800fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60ea600640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 0x7f60c800fdd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009fc0 tx=0x7f60c8009c40 comp rx=0 tx=0).stop 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60ea600640 1 -- 172.21.15.46:0/4058477197 shutdown_connections 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60ea600640 1 --2- 172.21.15.46:0/4058477197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c80095d0 0x7f60c800fdd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.485+0000 7f60ea600640 1 -- 172.21.15.46:0/4058477197 >> 172.21.15.46:0/4058477197 conn(0x7f60e406c980 msgr2=0x7f60e40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.486+0000 7f60ea600640 1 -- 172.21.15.46:0/4058477197 shutdown_connections 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.486+0000 7f60ea600640 1 -- 172.21.15.46:0/4058477197 wait complete. 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.487+0000 7f60ea600640 1 Processor -- start 2024-04-26T23:38:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.487+0000 7f60ea600640 1 -- start start 2024-04-26T23:38:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.488+0000 7f60ea600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c80095d0 0x7f60e413c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.488+0000 7f60ea600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 0x7f60e415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.488+0000 7f60ea600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.488+0000 7f60ea600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40b4e90 con 0x7f60e4140350 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.488+0000 7f60ea600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40b4bb0 con 0x7f60e40b0720 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60ea600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60e40b3ec0 con 0x7f60c80095d0 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36240/0 (socket says 172.21.15.46:36240) 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 -- 172.21.15.46:0/4280185035 learned_addr learned my addr 172.21.15.46:0/4280185035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3400640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c80095d0 0x7f60e413c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e2a00640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 0x7f60e415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 -- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c80095d0 msgr2=0x7f60e413c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c80095d0 0x7f60e413c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 -- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 msgr2=0x7f60e415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 0x7f60e415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e3e00640 1 -- 172.21.15.46:0/4280185035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e41416f0 con 0x7f60e4140350 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.489+0000 7f60e2a00640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 0x7f60e415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60e3e00640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e230 tx=0x7f60dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc03f030 con 0x7f60e4140350 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc003990 con 0x7f60e4140350 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc035070 con 0x7f60e4140350 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e41419d0 con 0x7f60e4140350 2024-04-26T23:38:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.490+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41398d0 con 0x7f60e4140350 2024-04-26T23:38:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.491+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4139ad0 con 0x7f60e4140350 2024-04-26T23:38:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.492+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60dc042030 con 0x7f60e4140350 2024-04-26T23:38:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.493+0000 7f60db400640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 0x7f60c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.493+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f60dc0ac780 con 0x7f60e4140350 2024-04-26T23:38:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.494+0000 7f60e3400640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 0x7f60c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.495+0000 7f60e3400640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 0x7f60c406ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60d4001050 tx=0x7f60d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.498+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60e4139ad0 con 0x7f60e4140350 2024-04-26T23:38:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.701+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60e4141ef0 con 0x7f60e4140350 2024-04-26T23:38:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.703+0000 7f60db400640 1 -- 172.21.15.46:0/4280185035 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f60dc07a7a0 con 0x7f60e4140350 2024-04-26T23:38:34.705 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:38:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 msgr2=0x7f60c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 0x7f60c406ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60d4001050 tx=0x7f60d4009040 comp rx=0 tx=0).stop 2024-04-26T23:38:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 msgr2=0x7f60e41fc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e230 tx=0x7f60dc009750 comp rx=0 tx=0).stop 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 shutdown_connections 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c920 0x7f60c406ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e4140350 0x7f60e41fc350 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e40b0720 0x7f60e415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 --2- 172.21.15.46:0/4280185035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60c80095d0 0x7f60e413c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 >> 172.21.15.46:0/4280185035 conn(0x7f60e406c980 msgr2=0x7f60e4142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.705+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 shutdown_connections 2024-04-26T23:38:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.706+0000 7f60ea600640 1 -- 172.21.15.46:0/4280185035 wait complete. 2024-04-26T23:38:34.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:38:34.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.946+0000 7f3d99a00640 1 Processor -- start 2024-04-26T23:38:34.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.946+0000 7f3d99a00640 1 -- start start 2024-04-26T23:38:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d99a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d941626c0 0x7f3d940b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d99a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d94162aa0 0x7f3d940d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d99a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d940d07d0 con 0x7f3d940b5fb0 2024-04-26T23:38:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d99a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3d940d0ab0 con 0x7f3d941626c0 2024-04-26T23:38:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d99a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3d940d0d90 con 0x7f3d94162aa0 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d93e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d941626c0 0x7f3d940b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d93e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d941626c0 0x7f3d940b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36252/0 (socket says 172.21.15.46:36252) 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.948+0000 7f3d93e00640 1 -- 172.21.15.46:0/1176373313 learned_addr learned my addr 172.21.15.46:0/1176373313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.949+0000 7f3d99000640 1 --2- 172.21.15.46:0/1176373313 >> v2:172.21.15.142:3300/0 conn(0x7f3d94162aa0 0x7f3d940d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.949+0000 7f3d93e00640 1 -- 172.21.15.46:0/1176373313 >> v2:172.21.15.142:3300/0 conn(0x7f3d94162aa0 msgr2=0x7f3d940d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.949+0000 7f3d93e00640 1 --2- 172.21.15.46:0/1176373313 >> v2:172.21.15.142:3300/0 conn(0x7f3d94162aa0 0x7f3d940d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.949+0000 7f3d93e00640 1 -- 172.21.15.46:0/1176373313 >> v1:172.21.15.142:6789/0 conn(0x7f3d940b5fb0 legacy=0x7f3d940d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:34.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.949+0000 7f3d93e00640 1 -- 172.21.15.46:0/1176373313 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d940d2c80 con 0x7f3d941626c0 2024-04-26T23:38:34.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.950+0000 7f3d93e00640 1 --2- 172.21.15.46:0/1176373313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d941626c0 0x7f3d940b47c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d940bafe0 tx=0x7f3d8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1b85b83f63d968 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.950+0000 7f3d93400640 1 -- 172.21.15.46:0/1176373313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84037070 con 0x7f3d941626c0 2024-04-26T23:38:34.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.950+0000 7f3d93400640 1 -- 172.21.15.46:0/1176373313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d84032440 con 0x7f3d941626c0 2024-04-26T23:38:34.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d93400640 1 -- 172.21.15.46:0/1176373313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84032810 con 0x7f3d941626c0 2024-04-26T23:38:34.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 -- 172.21.15.46:0/1176373313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d941626c0 msgr2=0x7f3d940b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1176373313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d941626c0 0x7f3d940b47c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d940bafe0 tx=0x7f3d8402a230 comp rx=0 tx=0).stop 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 -- 172.21.15.46:0/1176373313 shutdown_connections 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1176373313 >> v2:172.21.15.142:3300/0 conn(0x7f3d94162aa0 0x7f3d940d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1176373313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d941626c0 0x7f3d940b47c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 -- 172.21.15.46:0/1176373313 >> 172.21.15.46:0/1176373313 conn(0x7f3d9409e2a0 msgr2=0x7f3d940d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 -- 172.21.15.46:0/1176373313 shutdown_connections 2024-04-26T23:38:34.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.951+0000 7f3d99a00640 1 -- 172.21.15.46:0/1176373313 wait complete. 2024-04-26T23:38:34.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.953+0000 7f3d99a00640 1 Processor -- start 2024-04-26T23:38:34.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.953+0000 7f3d99a00640 1 -- start start 2024-04-26T23:38:34.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d940b5fb0 0x7f3d9414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 0x7f3d940c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d940d2210 con 0x7f3d94162aa0 2024-04-26T23:38:34.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d940d2090 con 0x7f3d941626c0 2024-04-26T23:38:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d940d31a0 con 0x7f3d940b5fb0 2024-04-26T23:38:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.954+0000 7f3d99000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36266/0 (socket says 172.21.15.46:36266) 2024-04-26T23:38:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 -- 172.21.15.46:0/1459844410 learned_addr learned my addr 172.21.15.46:0/1459844410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d93e00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d940b5fb0 0x7f3d9414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d8be00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 0x7f3d940c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 -- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d940b5fb0 msgr2=0x7f3d9414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d940b5fb0 0x7f3d9414e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 -- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 msgr2=0x7f3d940c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 0x7f3d940c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 -- 172.21.15.46:0/1459844410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d940cb9f0 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d8be00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 0x7f3d940c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.955+0000 7f3d99000640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c007e60 tx=0x7f3d8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.956+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c014420 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.956+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c019070 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.956+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c014820 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.956+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d940cbcd0 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.956+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d94150880 con 0x7f3d94162aa0 2024-04-26T23:38:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.957+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d940693b0 con 0x7f3d94162aa0 2024-04-26T23:38:34.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.958+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d8c0149c0 con 0x7f3d94162aa0 2024-04-26T23:38:34.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.959+0000 7f3d90c00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:34.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.959+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3d8c08c720 con 0x7f3d94162aa0 2024-04-26T23:38:34.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.960+0000 7f3d93e00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.961+0000 7f3d93e00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d84034000 tx=0x7f3d8402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:34.964+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d8c05a7e0 con 0x7f3d94162aa0 2024-04-26T23:38:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.175+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3d940b4040 con 0x7f3d94162aa0 2024-04-26T23:38:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.175+0000 7f3d90c00640 1 -- 172.21.15.46:0/1459844410 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f3d8c05a0e0 con 0x7f3d94162aa0 2024-04-26T23:38:35.179 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:38:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 msgr2=0x7f3d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d84034000 tx=0x7f3d8402e000 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 msgr2=0x7f3d941f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c007e60 tx=0x7f3d8c00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 shutdown_connections 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94162aa0 0x7f3d941f5510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d941626c0 0x7f3d940c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 --2- 172.21.15.46:0/1459844410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d940b5fb0 0x7f3d9414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.179+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 >> 172.21.15.46:0/1459844410 conn(0x7f3d9409e2a0 msgr2=0x7f3d940d3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.180+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 shutdown_connections 2024-04-26T23:38:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.180+0000 7f3d99a00640 1 -- 172.21.15.46:0/1459844410 wait complete. 2024-04-26T23:38:35.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:38:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.415+0000 7fc67f200640 1 Processor -- start 2024-04-26T23:38:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.415+0000 7fc67f200640 1 -- start start 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 0x7fc6780d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6780b4e20 0x7fc6780d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6780b65e0 con 0x7fc6780d5a90 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc6780d12e0 con 0x7fc6780b4e20 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6780d05f0 con 0x7fc678162a60 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6780d5a90 0x7fc6780b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42322/0 (socket says 172.21.15.46:42322) 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67d400640 1 -- 172.21.15.46:0/2093729509 learned_addr learned my addr 172.21.15.46:0/2093729509 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.417+0000 7fc67e800640 1 --2- 172.21.15.46:0/2093729509 >> v2:172.21.15.46:3300/0 conn(0x7fc6780b4e20 0x7fc6780d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67de00640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 0x7fc6780d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67ca00640 1 -- 172.21.15.46:0/2093729509 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2859841047 0 0) 0x7fc6780b65e0 con 0x7fc6780d5a90 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67ca00640 1 -- 172.21.15.46:0/2093729509 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc660003520 con 0x7fc6780d5a90 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67e800640 1 -- 172.21.15.46:0/2093729509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 msgr2=0x7fc6780d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67e800640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 0x7fc6780d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67e800640 1 -- 172.21.15.46:0/2093729509 >> v1:172.21.15.46:6789/0 conn(0x7fc6780d5a90 legacy=0x7fc6780b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67e800640 1 -- 172.21.15.46:0/2093729509 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6780d1530 con 0x7fc6780b4e20 2024-04-26T23:38:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.418+0000 7fc67de00640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 0x7fc6780d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.419+0000 7fc67e800640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b4e20 0x7fc6780d19b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc674009d90 tx=0x7fc67402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f37d0d716273c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.419+0000 7fc67ca00640 1 -- 172.21.15.46:0/2093729509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674007740 con 0x7fc6780b4e20 2024-04-26T23:38:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.419+0000 7fc67ca00640 1 -- 172.21.15.46:0/2093729509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6740078e0 con 0x7fc6780b4e20 2024-04-26T23:38:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.419+0000 7fc67ca00640 1 -- 172.21.15.46:0/2093729509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674007cb0 con 0x7fc6780b4e20 2024-04-26T23:38:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 -- 172.21.15.46:0/2093729509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b4e20 msgr2=0x7fc6780d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b4e20 0x7fc6780d19b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc674009d90 tx=0x7fc67402a530 comp rx=0 tx=0).stop 2024-04-26T23:38:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 -- 172.21.15.46:0/2093729509 shutdown_connections 2024-04-26T23:38:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b4e20 0x7fc6780d19b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 --2- 172.21.15.46:0/2093729509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc678162a60 0x7fc6780d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 -- 172.21.15.46:0/2093729509 >> 172.21.15.46:0/2093729509 conn(0x7fc6780d4fa0 msgr2=0x7fc6780d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 -- 172.21.15.46:0/2093729509 shutdown_connections 2024-04-26T23:38:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.420+0000 7fc67f200640 1 -- 172.21.15.46:0/2093729509 wait complete. 2024-04-26T23:38:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.422+0000 7fc67f200640 1 Processor -- start 2024-04-26T23:38:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.422+0000 7fc67f200640 1 -- start start 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6780b4e20 0x7fc6780c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 0x7fc6780d4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780b52d0 con 0x7fc6780d5a90 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780d1ef0 con 0x7fc678162a60 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.423+0000 7fc67f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6780c9160 con 0x7fc6780b4e20 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36290/0 (socket says 172.21.15.46:36290) 2024-04-26T23:38:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 -- 172.21.15.46:0/433070290 learned_addr learned my addr 172.21.15.46:0/433070290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67de00640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6780b4e20 0x7fc6780c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67e800640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 0x7fc6780d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 -- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6780b4e20 msgr2=0x7fc6780c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6780b4e20 0x7fc6780c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 -- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 msgr2=0x7fc6780d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 0x7fc6780d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67d400640 1 -- 172.21.15.46:0/433070290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6781f9220 con 0x7fc6780d5a90 2024-04-26T23:38:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.424+0000 7fc67e800640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 0x7fc6780d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc67d400640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c460 tx=0x7fc66800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668017070 con 0x7fc6780d5a90 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc668005d00 con 0x7fc6780d5a90 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6781f8480 con 0x7fc6780d5a90 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6680072b0 con 0x7fc6780d5a90 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.425+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781f8a20 con 0x7fc6780d5a90 2024-04-26T23:38:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.426+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc678069ac0 con 0x7fc6780d5a90 2024-04-26T23:38:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.427+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc66801a070 con 0x7fc6780d5a90 2024-04-26T23:38:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.428+0000 7fc672000640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 0x7fc65c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.428+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc66808b2d0 con 0x7fc6780d5a90 2024-04-26T23:38:35.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.429+0000 7fc67de00640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 0x7fc65c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.430+0000 7fc67de00640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 0x7fc65c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006fd0 tx=0x7fc66c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.433+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6680592f0 con 0x7fc6780d5a90 2024-04-26T23:38:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.677+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6781609f0 con 0x7fc6780d5a90 2024-04-26T23:38:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.678+0000 7fc672000640 1 -- 172.21.15.46:0/433070290 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fc668058bf0 con 0x7fc6780d5a90 2024-04-26T23:38:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:35.682 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:38:35.682 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:38:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 msgr2=0x7fc65c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 0x7fc65c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006fd0 tx=0x7fc66c008040 comp rx=0 tx=0).stop 2024-04-26T23:38:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 msgr2=0x7fc678152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c460 tx=0x7fc66800c920 comp rx=0 tx=0).stop 2024-04-26T23:38:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 shutdown_connections 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc65c06c8f0 0x7fc65c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc678162a60 0x7fc6780d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d5a90 0x7fc678152430 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 --2- 172.21.15.46:0/433070290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6780b4e20 0x7fc6780c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 >> 172.21.15.46:0/433070290 conn(0x7fc6780d4fa0 msgr2=0x7fc6780b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 shutdown_connections 2024-04-26T23:38:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.681+0000 7fc67f200640 1 -- 172.21.15.46:0/433070290 wait complete. 2024-04-26T23:38:35.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.899+0000 7f4b41a00640 1 Processor -- start 2024-04-26T23:38:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.899+0000 7f4b41a00640 1 -- start start 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4b3c0cc690 0x7f4b3c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b3c072bb0 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b3c072e90 con 0x7f4b3c1454a0 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3c073170 con 0x7f4b3c0cc690 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b3c0c6a50 0x7f4b3c0cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42334/0 (socket says 172.21.15.46:42334) 2024-04-26T23:38:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.901+0000 7f4b41000640 1 -- 172.21.15.46:0/2792291953 learned_addr learned my addr 172.21.15.46:0/2792291953 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b33e00640 1 --2- 172.21.15.46:0/2792291953 >> v2:172.21.15.142:3300/0 conn(0x7f4b3c0cc690 0x7f4b3c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246351113 0 0) 0x7f4b3c072bb0 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b20005730 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2816460034 0 0) 0x7f4b20005730 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 >> v2:172.21.15.142:3300/0 conn(0x7f4b3c0cc690 msgr2=0x7f4b3c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 --2- 172.21.15.46:0/2792291953 >> v2:172.21.15.142:3300/0 conn(0x7f4b3c0cc690 0x7f4b3c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 >> v1:172.21.15.142:6789/0 conn(0x7f4b3c1454a0 legacy=0x7f4b3c073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c073450 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4b340019f0 con 0x7f4b3c0c6a50 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 >> v1:172.21.15.46:6789/0 conn(0x7f4b3c0c6a50 legacy=0x7f4b3c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b20005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b20006a10 0x7f4b20006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b200073d0 0x7f4b2000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c073170 con 0x7f4b20005bc0 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2000e320 con 0x7f4b20006a10 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.902+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2000e520 con 0x7f4b200073d0 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b20005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 -- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b200073d0 msgr2=0x7f4b2000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b200073d0 0x7f4b2000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 -- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b20006a10 msgr2=0x7f4b20006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b20006a10 0x7f4b20006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b3be00640 1 -- 172.21.15.46:0/2792291953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b20005730 con 0x7f4b20005bc0 2024-04-26T23:38:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.903+0000 7f4b41000640 1 -- 172.21.15.46:0/2792291953 reap_dead start 2024-04-26T23:38:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.904+0000 7f4b3be00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b20005fe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c001750 tx=0x7f4b20007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc70431e6b0322 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.904+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c001370 con 0x7f4b20005bc0 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.904+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0052d0 con 0x7f4b20005bc0 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.904+0000 7f4b3b400640 1 -- 172.21.15.46:0/2792291953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c02abe0 con 0x7f4b20005bc0 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 -- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 msgr2=0x7f4b20005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b20005fe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c001750 tx=0x7f4b20007a40 comp rx=0 tx=0).stop 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 -- 172.21.15.46:0/2792291953 shutdown_connections 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 --2- 172.21.15.46:0/2792291953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b20005fe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 -- 172.21.15.46:0/2792291953 >> 172.21.15.46:0/2792291953 conn(0x7f4b3c0b26d0 msgr2=0x7f4b3c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 -- 172.21.15.46:0/2792291953 shutdown_connections 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.905+0000 7f4b41a00640 1 -- 172.21.15.46:0/2792291953 wait complete. 2024-04-26T23:38:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.907+0000 7f4b41a00640 1 Processor -- start 2024-04-26T23:38:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.907+0000 7f4b41a00640 1 -- start start 2024-04-26T23:38:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b3c0af8f0 0x7f4b3c1f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b3c15b170 0x7f4b3c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c145a00 con 0x7f4b20005bc0 2024-04-26T23:38:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c145880 con 0x7f4b3c0af8f0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b41a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c0c6ea0 con 0x7f4b3c15b170 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36320/0 (socket says 172.21.15.46:36320) 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.908+0000 7f4b3be00640 1 -- 172.21.15.46:0/1068383935 learned_addr learned my addr 172.21.15.46:0/1068383935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 -- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b3c15b170 msgr2=0x7f4b3c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b3c15b170 0x7f4b3c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 -- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b3c0af8f0 msgr2=0x7f4b3c1f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b3c0af8f0 0x7f4b3c1f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 -- 172.21.15.46:0/1068383935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c0cbaf0 con 0x7f4b20005bc0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b3be00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c03f030 con 0x7f4b20005bc0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c005480 con 0x7f4b20005bc0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c1f9460 con 0x7f4b20005bc0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.909+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c037040 con 0x7f4b20005bc0 2024-04-26T23:38:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.910+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c0aee10 con 0x7f4b20005bc0 2024-04-26T23:38:35.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.911+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c0693b0 con 0x7f4b20005bc0 2024-04-26T23:38:35.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.912+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b2c042070 con 0x7f4b20005bc0 2024-04-26T23:38:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.913+0000 7f4b38c00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 0x7f4b1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.913+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4b2c0ab6e0 con 0x7f4b20005bc0 2024-04-26T23:38:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.914+0000 7f4b33e00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 0x7f4b1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.915+0000 7f4b33e00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 0x7f4b1806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006e80 tx=0x7f4b24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:35.917+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b2c049070 con 0x7f4b20005bc0 2024-04-26T23:38:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.117+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b3c0cefa0 con 0x7f4b20005bc0 2024-04-26T23:38:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.119+0000 7f4b38c00640 1 -- 172.21.15.46:0/1068383935 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f4b2c0433b0 con 0x7f4b20005bc0 2024-04-26T23:38:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:36.123 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.122+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 msgr2=0x7f4b1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.122+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 0x7f4b1806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006e80 tx=0x7f4b24008040 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.122+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 msgr2=0x7f4b3c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.122+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c001120 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 shutdown_connections 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b1806c920 0x7f4b1806ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b3c15b170 0x7f4b3c0cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b3c0af8f0 0x7f4b3c1f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 --2- 172.21.15.46:0/1068383935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20005bc0 0x7f4b3c15e750 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 >> 172.21.15.46:0/1068383935 conn(0x7f4b3c0b26d0 msgr2=0x7f4b3c1426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 shutdown_connections 2024-04-26T23:38:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.123+0000 7f4b41a00640 1 -- 172.21.15.46:0/1068383935 wait complete. 2024-04-26T23:38:36.144 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:38:36.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:38:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.348+0000 7f2757e00640 1 Processor -- start 2024-04-26T23:38:36.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.349+0000 7f2757e00640 1 -- start start 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.350+0000 7f2757e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f275815edf0 0x7f27580cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.350+0000 7f2757e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f275815f1d0 0x7f27580cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.350+0000 7f2757e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27580b03c0 con 0x7f27580cb1a0 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.350+0000 7f2757e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f27580cf5c0 con 0x7f275815f1d0 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.350+0000 7f2757e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f27580cb680 con 0x7f275815edf0 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f275815f1d0 0x7f27580cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f275815f1d0 0x7f27580cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36322/0 (socket says 172.21.15.46:36322) 2024-04-26T23:38:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 -- 172.21.15.46:0/4075901078 learned_addr learned my addr 172.21.15.46:0/4075901078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2757400640 1 --1- 172.21.15.46:0/4075901078 >> v1:172.21.15.46:6789/0 conn(0x7f27580cb1a0 0x7f27580ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42338/0 (socket says 172.21.15.46:42338) 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756a00640 1 --2- 172.21.15.46:0/4075901078 >> v2:172.21.15.142:3300/0 conn(0x7f275815edf0 0x7f27580cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 -- 172.21.15.46:0/4075901078 >> v2:172.21.15.142:3300/0 conn(0x7f275815edf0 msgr2=0x7f27580cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 --2- 172.21.15.46:0/4075901078 >> v2:172.21.15.142:3300/0 conn(0x7f275815edf0 0x7f27580cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 -- 172.21.15.46:0/4075901078 >> v1:172.21.15.46:6789/0 conn(0x7f27580cb1a0 legacy=0x7f27580ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.351+0000 7f2756000640 1 -- 172.21.15.46:0/4075901078 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27580cb960 con 0x7f275815f1d0 2024-04-26T23:38:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.352+0000 7f2756000640 1 --2- 172.21.15.46:0/4075901078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815f1d0 0x7f27580cd060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f27580d0a00 tx=0x7f274002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b11d05af0c528b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2755600640 1 -- 172.21.15.46:0/4075901078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740036070 con 0x7f275815f1d0 2024-04-26T23:38:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2755600640 1 -- 172.21.15.46:0/4075901078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2740031600 con 0x7f275815f1d0 2024-04-26T23:38:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2755600640 1 -- 172.21.15.46:0/4075901078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27400319f0 con 0x7f275815f1d0 2024-04-26T23:38:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 -- 172.21.15.46:0/4075901078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815f1d0 msgr2=0x7f27580cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 --2- 172.21.15.46:0/4075901078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815f1d0 0x7f27580cd060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f27580d0a00 tx=0x7f274002a7c0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 -- 172.21.15.46:0/4075901078 shutdown_connections 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 --2- 172.21.15.46:0/4075901078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815f1d0 0x7f27580cd060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 --2- 172.21.15.46:0/4075901078 >> v2:172.21.15.142:3300/0 conn(0x7f275815edf0 0x7f27580cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 -- 172.21.15.46:0/4075901078 >> 172.21.15.46:0/4075901078 conn(0x7f27580b26d0 msgr2=0x7f27580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 -- 172.21.15.46:0/4075901078 shutdown_connections 2024-04-26T23:38:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.353+0000 7f2757e00640 1 -- 172.21.15.46:0/4075901078 wait complete. 2024-04-26T23:38:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.355+0000 7f2757e00640 1 Processor -- start 2024-04-26T23:38:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.355+0000 7f2757e00640 1 -- start start 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 0x7f275815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f275814fc70 0x7f27580cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27580cc200 con 0x7f275815edf0 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27580cbf20 con 0x7f27580cb1a0 2024-04-26T23:38:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2757e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27580c6950 con 0x7f275814fc70 2024-04-26T23:38:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2756000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2756000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36338/0 (socket says 172.21.15.46:36338) 2024-04-26T23:38:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.356+0000 7f2756000640 1 -- 172.21.15.46:0/4221125463 learned_addr learned my addr 172.21.15.46:0/4221125463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756a00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 0x7f275815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 -- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f275814fc70 msgr2=0x7f27580cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f275814fc70 0x7f27580cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 -- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 msgr2=0x7f275815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 0x7f275815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 -- 172.21.15.46:0/4221125463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2758160830 con 0x7f275815edf0 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756a00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 0x7f275815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.357+0000 7f2756000640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f274002adb0 tx=0x7f2740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.358+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740040020 con 0x7f275815edf0 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.358+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2740004040 con 0x7f275815edf0 2024-04-26T23:38:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.358+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740036040 con 0x7f275815edf0 2024-04-26T23:38:36.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.358+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2758160b10 con 0x7f275815edf0 2024-04-26T23:38:36.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.358+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27581610d0 con 0x7f275815edf0 2024-04-26T23:38:36.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.359+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27580693b0 con 0x7f275815edf0 2024-04-26T23:38:36.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.360+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2740043020 con 0x7f275815edf0 2024-04-26T23:38:36.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.361+0000 7f274ea00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 0x7f273406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.361+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f27400acc60 con 0x7f275815edf0 2024-04-26T23:38:36.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.362+0000 7f2756a00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 0x7f273406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.363+0000 7f2756a00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 0x7f273406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2748008890 tx=0x7f2748008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.366+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f274004a020 con 0x7f275815edf0 2024-04-26T23:38:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.602+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f275815d770 con 0x7f275815edf0 2024-04-26T23:38:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.603+0000 7f274ea00640 1 -- 172.21.15.46:0/4221125463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f274007abf0 con 0x7f275815edf0 2024-04-26T23:38:36.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:36.606 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:38:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 msgr2=0x7f273406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 0x7f273406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2748008890 tx=0x7f2748008040 comp rx=0 tx=0).stop 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 msgr2=0x7f275815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f274002adb0 tx=0x7f2740004a40 comp rx=0 tx=0).stop 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 shutdown_connections 2024-04-26T23:38:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f273406c920 0x7f273406ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f275814fc70 0x7f27580cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f275815edf0 0x7f275815ba50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 --2- 172.21.15.46:0/4221125463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27580cb1a0 0x7f275815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 >> 172.21.15.46:0/4221125463 conn(0x7f27580b26d0 msgr2=0x7f27580ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 shutdown_connections 2024-04-26T23:38:36.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.605+0000 7f2757e00640 1 -- 172.21.15.46:0/4221125463 wait complete. 2024-04-26T23:38:36.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:38:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.822+0000 7f5a9e800640 1 Processor -- start 2024-04-26T23:38:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.822+0000 7f5a9e800640 1 -- start start 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.823+0000 7f5a9e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a981466b0 0x7f5a9806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.823+0000 7f5a9e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 0x7f5a98160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.823+0000 7f5a9e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a98073ad0 con 0x7f5a980720e0 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.823+0000 7f5a9e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5a981612a0 con 0x7f5a981466b0 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.823+0000 7f5a9e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5a98146a90 con 0x7f5a98160910 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a981466b0 0x7f5a9806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a981466b0 0x7f5a9806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36342/0 (socket says 172.21.15.46:36342) 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 -- 172.21.15.46:0/157020776 learned_addr learned my addr 172.21.15.46:0/157020776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9de00640 1 --2- 172.21.15.46:0/157020776 >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 0x7f5a98160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 -- 172.21.15.46:0/157020776 >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 msgr2=0x7f5a98160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 --2- 172.21.15.46:0/157020776 >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 0x7f5a98160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 -- 172.21.15.46:0/157020776 >> v1:172.21.15.142:6789/0 conn(0x7f5a980720e0 legacy=0x7f5a980cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9d400640 1 -- 172.21.15.46:0/157020776 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a98073d70 con 0x7f5a981466b0 2024-04-26T23:38:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.824+0000 7f5a9de00640 1 --2- 172.21.15.46:0/157020776 >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 0x7f5a98160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.825+0000 7f5a9d400640 1 --2- 172.21.15.46:0/157020776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a9806de80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009870 tx=0x7f5a8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdc6bd9c2ea37df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a97e00640 1 -- 172.21.15.46:0/157020776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88037070 con 0x7f5a981466b0 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a97e00640 1 -- 172.21.15.46:0/157020776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a88032440 con 0x7f5a981466b0 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a97e00640 1 -- 172.21.15.46:0/157020776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88032810 con 0x7f5a981466b0 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 -- 172.21.15.46:0/157020776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 msgr2=0x7f5a9806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 --2- 172.21.15.46:0/157020776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a9806de80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009870 tx=0x7f5a8802a270 comp rx=0 tx=0).stop 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 -- 172.21.15.46:0/157020776 shutdown_connections 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 --2- 172.21.15.46:0/157020776 >> v2:172.21.15.142:3300/0 conn(0x7f5a98160910 0x7f5a98160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 --2- 172.21.15.46:0/157020776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a9806de80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 -- 172.21.15.46:0/157020776 >> 172.21.15.46:0/157020776 conn(0x7f5a980b26d0 msgr2=0x7f5a980b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.826+0000 7f5a9e800640 1 -- 172.21.15.46:0/157020776 shutdown_connections 2024-04-26T23:38:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.827+0000 7f5a9e800640 1 -- 172.21.15.46:0/157020776 wait complete. 2024-04-26T23:38:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.828+0000 7f5a9e800640 1 Processor -- start 2024-04-26T23:38:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.828+0000 7f5a9e800640 1 -- start start 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a980720e0 0x7f5a98153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a98160910 0x7f5a980ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a980cb470 con 0x7f5a981466b0 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a980cb190 con 0x7f5a980720e0 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9806e3c0 con 0x7f5a98160910 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36354/0 (socket says 172.21.15.46:36354) 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.829+0000 7f5a9ca00640 1 -- 172.21.15.46:0/2615473929 learned_addr learned my addr 172.21.15.46:0/2615473929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 -- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a98160910 msgr2=0x7f5a980ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a98160910 0x7f5a980ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 -- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a980720e0 msgr2=0x7f5a98153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a980720e0 0x7f5a98153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 -- 172.21.15.46:0/2615473929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a980b5210 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a9ca00640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400b3e0 tx=0x7f5a8400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84015020 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a84011850 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.830+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84011c40 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.831+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9813bcc0 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.831+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9813c280 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.832+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9813c480 con 0x7f5a981466b0 2024-04-26T23:38:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.833+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a84018020 con 0x7f5a981466b0 2024-04-26T23:38:36.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.834+0000 7f5a95600640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 0x7f5a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:36.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.834+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a8408bfe0 con 0x7f5a981466b0 2024-04-26T23:38:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.834+0000 7f5a9d400640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 0x7f5a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.836+0000 7f5a9d400640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 0x7f5a7806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a88034000 tx=0x7f5a8802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:36.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:36.838+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a9813c480 con 0x7f5a981466b0 2024-04-26T23:38:37.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.083+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a980693b0 con 0x7f5a981466b0 2024-04-26T23:38:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.084+0000 7f5a95600640 1 -- 172.21.15.46:0/2615473929 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f5a8401f020 con 0x7f5a981466b0 2024-04-26T23:38:37.087 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:37.087 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2350021464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2350021464},{"type":"v1","addr":"172.21.15.46:6835","nonce":2350021464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1529786296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1529786296},{"type":"v1","addr":"172.21.15.46:6834","nonce":1529786296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3730998103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3730998103},{"type":"v1","addr":"172.21.15.142:6837","nonce":3730998103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:37:08.251922+0000","modified":"2024-04-26T23:37:36.515301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/2711334262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2711334262},{"type":"v1","addr":"172.21.15.142:6836","nonce":2711334262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-26T23:38:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 msgr2=0x7f5a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 0x7f5a7806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a88034000 tx=0x7f5a8802c000 comp rx=0 tx=0).stop 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 msgr2=0x7f5a980c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400b3e0 tx=0x7f5a8400b8a0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 shutdown_connections 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a7806c8f0 0x7f5a7806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a98160910 0x7f5a980ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a981466b0 0x7f5a980c9c40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 --2- 172.21.15.46:0/2615473929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a980720e0 0x7f5a98153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 >> 172.21.15.46:0/2615473929 conn(0x7f5a980b26d0 msgr2=0x7f5a980c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:37.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 shutdown_connections 2024-04-26T23:38:37.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.087+0000 7f5a9e800640 1 -- 172.21.15.46:0/2615473929 wait complete. 2024-04-26T23:38:37.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.317+0000 7f6e4b000640 1 Processor -- start 2024-04-26T23:38:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.317+0000 7f6e4b000640 1 -- start start 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6e4415edf0 0x7f6e440cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e4415f1d0 0x7f6e440cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e440cb1a0 0x7f6e440cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6e440cf5c0 con 0x7f6e4415f1d0 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6e440b03c0 con 0x7f6e4415edf0 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e4b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e440cb680 con 0x7f6e440cb1a0 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.319+0000 7f6e49200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e4415f1d0 0x7f6e440cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e4415f1d0 0x7f6e440cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36366/0 (socket says 172.21.15.46:36366) 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 -- 172.21.15.46:0/2382927550 learned_addr learned my addr 172.21.15.46:0/2382927550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e4a600640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e440cb1a0 0x7f6e440cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49c00640 1 --2- 172.21.15.46:0/2382927550 >> v2:172.21.15.142:3300/0 conn(0x7f6e4415edf0 0x7f6e440cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 -- 172.21.15.46:0/2382927550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e440cb1a0 msgr2=0x7f6e440cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e440cb1a0 0x7f6e440cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 -- 172.21.15.46:0/2382927550 >> v2:172.21.15.142:3300/0 conn(0x7f6e4415edf0 msgr2=0x7f6e440cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 --2- 172.21.15.46:0/2382927550 >> v2:172.21.15.142:3300/0 conn(0x7f6e4415edf0 0x7f6e440cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.320+0000 7f6e49200640 1 -- 172.21.15.46:0/2382927550 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e440cb960 con 0x7f6e4415f1d0 2024-04-26T23:38:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.322+0000 7f6e49200640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e4415f1d0 0x7f6e440cd060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6e440d0a00 tx=0x7f6e3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1d9e879cb50db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.322+0000 7f6e43e00640 1 -- 172.21.15.46:0/2382927550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30037070 con 0x7f6e4415f1d0 2024-04-26T23:38:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.322+0000 7f6e43e00640 1 -- 172.21.15.46:0/2382927550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e30032440 con 0x7f6e4415f1d0 2024-04-26T23:38:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.322+0000 7f6e43e00640 1 -- 172.21.15.46:0/2382927550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30032810 con 0x7f6e4415f1d0 2024-04-26T23:38:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 -- 172.21.15.46:0/2382927550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e4415f1d0 msgr2=0x7f6e440cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e4415f1d0 0x7f6e440cd060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6e440d0a00 tx=0x7f6e3002a270 comp rx=0 tx=0).stop 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 -- 172.21.15.46:0/2382927550 shutdown_connections 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e440cb1a0 0x7f6e440cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 --2- 172.21.15.46:0/2382927550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e4415f1d0 0x7f6e440cd060 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 --2- 172.21.15.46:0/2382927550 >> v2:172.21.15.142:3300/0 conn(0x7f6e4415edf0 0x7f6e440cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 -- 172.21.15.46:0/2382927550 >> 172.21.15.46:0/2382927550 conn(0x7f6e440b26d0 msgr2=0x7f6e440b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 -- 172.21.15.46:0/2382927550 shutdown_connections 2024-04-26T23:38:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.323+0000 7f6e4b000640 1 -- 172.21.15.46:0/2382927550 wait complete. 2024-04-26T23:38:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.325+0000 7f6e4b000640 1 Processor -- start 2024-04-26T23:38:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.325+0000 7f6e4b000640 1 -- start start 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4415edf0 0x7f6e44073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 0x7f6e440c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440cc200 con 0x7f6e440cb1a0 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440cbf20 con 0x7f6e4415edf0 2024-04-26T23:38:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.326+0000 7f6e4b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e440ae8e0 con 0x7f6e441f80f0 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36376/0 (socket says 172.21.15.46:36376) 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 -- 172.21.15.46:0/4119636398 learned_addr learned my addr 172.21.15.46:0/4119636398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e4a600640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 0x7f6e440c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49200640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4415edf0 0x7f6e44073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 -- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 msgr2=0x7f6e440c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 0x7f6e440c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 -- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4415edf0 msgr2=0x7f6e44073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4415edf0 0x7f6e44073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e49c00640 1 -- 172.21.15.46:0/4119636398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e441f5a50 con 0x7f6e440cb1a0 2024-04-26T23:38:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.327+0000 7f6e4a600640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 0x7f6e440c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e49c00640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800d830 tx=0x7f6e3800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800b6c0 con 0x7f6e440cb1a0 2024-04-26T23:38:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e38019070 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800ba80 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e441f8a60 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.328+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e441f4d10 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.329+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e441f4b00 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.330+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e3801e460 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.331+0000 7f6e41600640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.331+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6e3808c650 con 0x7f6e440cb1a0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.332+0000 7f6e49200640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.333+0000 7f6e49200640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 0x7f6e2406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e3002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.335+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e441f5a50 con 0x7f6e440cb1a0 2024-04-26T23:38:37.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.541+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e440b2e30 con 0x7f6e440cb1a0 2024-04-26T23:38:37.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.543+0000 7f6e41600640 1 -- 172.21.15.46:0/4119636398 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18864 (secure 0 0 0) 0x7f6e3805a5e0 con 0x7f6e440cb1a0 2024-04-26T23:38:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:37.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:37:55.152765+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:37:05.921840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:37:07.172664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:37:10.790040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.545+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 msgr2=0x7f6e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.545+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 0x7f6e2406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e3002e000 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.545+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 msgr2=0x7f6e441f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.545+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800d830 tx=0x7f6e3800dcf0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 shutdown_connections 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e441f80f0 0x7f6e440c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4415edf0 0x7f6e44073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 --2- 172.21.15.46:0/4119636398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440cb1a0 0x7f6e441f5510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 >> 172.21.15.46:0/4119636398 conn(0x7f6e440b26d0 msgr2=0x7f6e440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:37.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 shutdown_connections 2024-04-26T23:38:37.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.546+0000 7f6e4b000640 1 -- 172.21.15.46:0/4119636398 wait complete. 2024-04-26T23:38:37.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:38:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.774+0000 7f265de00640 1 Processor -- start 2024-04-26T23:38:37.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.775+0000 7f265de00640 1 -- start start 2024-04-26T23:38:37.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f265de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2658160bd0 0x7f2658161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f265de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f26580732c0 0x7f2658073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f265de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2658161940 con 0x7f265806e510 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f265de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2658071f80 con 0x7f2658160bd0 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f265de00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2658072260 con 0x7f26580732c0 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f2656000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2658160bd0 0x7f2658161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f2656000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2658160bd0 0x7f2658161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36392/0 (socket says 172.21.15.46:36392) 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.776+0000 7f2656000640 1 -- 172.21.15.46:0/3760141036 learned_addr learned my addr 172.21.15.46:0/3760141036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.777+0000 7f2657400640 1 --2- 172.21.15.46:0/3760141036 >> v2:172.21.15.142:3300/0 conn(0x7f26580732c0 0x7f2658073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.777+0000 7f2656000640 1 -- 172.21.15.46:0/3760141036 >> v2:172.21.15.142:3300/0 conn(0x7f26580732c0 msgr2=0x7f2658073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.777+0000 7f2656000640 1 --2- 172.21.15.46:0/3760141036 >> v2:172.21.15.142:3300/0 conn(0x7f26580732c0 0x7f2658073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.777+0000 7f2656000640 1 -- 172.21.15.46:0/3760141036 >> v1:172.21.15.142:6789/0 conn(0x7f265806e510 legacy=0x7f265806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.777+0000 7f2656000640 1 -- 172.21.15.46:0/3760141036 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658072540 con 0x7f2658160bd0 2024-04-26T23:38:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.779+0000 7f2656000640 1 --2- 172.21.15.46:0/3760141036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2658160bd0 0x7f2658161060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2640009870 tx=0x7f264002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c32c62a42167d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.779+0000 7f2655600640 1 -- 172.21.15.46:0/3760141036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640037070 con 0x7f2658160bd0 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.779+0000 7f2655600640 1 -- 172.21.15.46:0/3760141036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2640032440 con 0x7f2658160bd0 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.779+0000 7f2655600640 1 -- 172.21.15.46:0/3760141036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640032810 con 0x7f2658160bd0 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 -- 172.21.15.46:0/3760141036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2658160bd0 msgr2=0x7f2658161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 --2- 172.21.15.46:0/3760141036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2658160bd0 0x7f2658161060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2640009870 tx=0x7f264002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 -- 172.21.15.46:0/3760141036 shutdown_connections 2024-04-26T23:38:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 --2- 172.21.15.46:0/3760141036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2658160bd0 0x7f2658161060 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 --2- 172.21.15.46:0/3760141036 >> v2:172.21.15.142:3300/0 conn(0x7f26580732c0 0x7f2658073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 -- 172.21.15.46:0/3760141036 >> 172.21.15.46:0/3760141036 conn(0x7f265806c530 msgr2=0x7f265806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 -- 172.21.15.46:0/3760141036 shutdown_connections 2024-04-26T23:38:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.780+0000 7f265de00640 1 -- 172.21.15.46:0/3760141036 wait complete. 2024-04-26T23:38:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.782+0000 7f265de00640 1 Processor -- start 2024-04-26T23:38:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.782+0000 7f265de00640 1 -- start start 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f265806e510 0x7f2658153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 0x7f2658152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2658161720 con 0x7f26580732c0 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26581615a0 con 0x7f2658160bd0 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.783+0000 7f265de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2658072d40 con 0x7f265806e510 2024-04-26T23:38:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36398/0 (socket says 172.21.15.46:36398) 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 -- 172.21.15.46:0/1966796269 learned_addr learned my addr 172.21.15.46:0/1966796269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656a00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f265806e510 0x7f2658153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2657400640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 0x7f2658152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 -- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f265806e510 msgr2=0x7f2658153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f265806e510 0x7f2658153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 -- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 msgr2=0x7f2658152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 0x7f2658152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 -- 172.21.15.46:0/1966796269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658142180 con 0x7f26580732c0 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2657400640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 0x7f2658152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.784+0000 7f2656000640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2640000c00 tx=0x7f26400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.785+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640037040 con 0x7f26580732c0 2024-04-26T23:38:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.785+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2640042070 con 0x7f26580732c0 2024-04-26T23:38:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.785+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640030880 con 0x7f26580732c0 2024-04-26T23:38:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.785+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2658142490 con 0x7f26580732c0 2024-04-26T23:38:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.785+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2658142cd0 con 0x7f26580732c0 2024-04-26T23:38:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.786+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2658069530 con 0x7f26580732c0 2024-04-26T23:38:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.787+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2640030a20 con 0x7f26580732c0 2024-04-26T23:38:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.788+0000 7f264f400640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.788+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f26400ac8c0 con 0x7f26580732c0 2024-04-26T23:38:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.789+0000 7f2656a00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.790+0000 7f2656a00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2648007fd0 tx=0x7f2648009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:37.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:37.792+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2658142490 con 0x7f26580732c0 2024-04-26T23:38:37.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:37.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.039+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f265806fb00 con 0x7f26580732c0 2024-04-26T23:38:38.042 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:38:38.040+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:38:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.045+0000 7f264f400640 1 -- 172.21.15.46:0/1966796269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f264007a7c0 con 0x7f26580732c0 2024-04-26T23:38:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 msgr2=0x7f263406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2648007fd0 tx=0x7f2648009040 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 msgr2=0x7f2658144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2640000c00 tx=0x7f26400046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 shutdown_connections 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2648007fd0 tx=0x7f2648009040 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2658160bd0 0x7f2658152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26580732c0 0x7f2658144bf0 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2640000c00 tx=0x7f26400046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 --2- 172.21.15.46:0/1966796269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f265806e510 0x7f2658153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.048+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 >> 172.21.15.46:0/1966796269 conn(0x7f265806c530 msgr2=0x7f26580b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.049+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 shutdown_connections 2024-04-26T23:38:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.050+0000 7f265de00640 1 -- 172.21.15.46:0/1966796269 wait complete. 2024-04-26T23:38:38.068 INFO:tasks.ceph.mds.b.smithi142.stderr: -6835> 2024-04-26T23:37:11.847+0000 7f246b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:38:38.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:38:38.132 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:38:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.261+0000 7f1fe0a00640 1 Processor -- start 2024-04-26T23:38:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.262+0000 7f1fe0a00640 1 -- start start 2024-04-26T23:38:38.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.263+0000 7f1fe0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 0x7f1fdc0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.263+0000 7f1fe0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fdc0d1cb0 0x7f1fdc0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.263+0000 7f1fe0a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fdc0d20c0 con 0x7f1fdc162a70 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.263+0000 7f1fe0a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1fdc0d3400 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.263+0000 7f1fe0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1fdc0d2370 con 0x7f1fdc0d3830 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fdb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fdc0d1cb0 0x7f1fdc0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fda800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1fdc162a70 0x7f1fdc0d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42346/0 (socket says 172.21.15.46:42346) 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fda800640 1 -- 172.21.15.46:0/797673874 learned_addr learned my addr 172.21.15.46:0/797673874 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fd9e00640 1 --2- 172.21.15.46:0/797673874 >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 0x7f1fdc0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fdb200640 1 -- 172.21.15.46:0/797673874 >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 msgr2=0x7f1fdc0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fdb200640 1 --2- 172.21.15.46:0/797673874 >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 0x7f1fdc0d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fdb200640 1 -- 172.21.15.46:0/797673874 >> v1:172.21.15.46:6789/0 conn(0x7f1fdc162a70 legacy=0x7f1fdc0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fdb200640 1 -- 172.21.15.46:0/797673874 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc0d2650 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.264+0000 7f1fd9e00640 1 --2- 172.21.15.46:0/797673874 >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 0x7f1fdc0d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:38.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.265+0000 7f1fdb200640 1 --2- 172.21.15.46:0/797673874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0b29e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc06f370 tx=0x7f1fd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b3f50310681514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:38.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fd9400640 1 -- 172.21.15.46:0/797673874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4037070 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fd9400640 1 -- 172.21.15.46:0/797673874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd4032440 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fd9400640 1 -- 172.21.15.46:0/797673874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4032810 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 -- 172.21.15.46:0/797673874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc0d1cb0 msgr2=0x7f1fdc0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/797673874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0b29e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc06f370 tx=0x7f1fd402a230 comp rx=0 tx=0).stop 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 -- 172.21.15.46:0/797673874 shutdown_connections 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/797673874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0b29e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/797673874 >> v2:172.21.15.142:3300/0 conn(0x7f1fdc0d3830 0x7f1fdc0d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 -- 172.21.15.46:0/797673874 >> 172.21.15.46:0/797673874 conn(0x7f1fdc0d4f20 msgr2=0x7f1fdc0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 -- 172.21.15.46:0/797673874 shutdown_connections 2024-04-26T23:38:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.266+0000 7f1fe0a00640 1 -- 172.21.15.46:0/797673874 wait complete. 2024-04-26T23:38:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.268+0000 7f1fe0a00640 1 Processor -- start 2024-04-26T23:38:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.268+0000 7f1fe0a00640 1 -- start start 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fdc0d3830 0x7f1fdc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc162fd0 con 0x7f1fdc162a70 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc162e50 con 0x7f1fdc0d1cb0 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.269+0000 7f1fe0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b2f20 con 0x7f1fdc0d3830 2024-04-26T23:38:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36422/0 (socket says 172.21.15.46:36422) 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 -- 172.21.15.46:0/30659887 learned_addr learned my addr 172.21.15.46:0/30659887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fda800640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fd9e00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fdc0d3830 0x7f1fdc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 -- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fdc0d3830 msgr2=0x7f1fdc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fdc0d3830 0x7f1fdc0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 -- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fdc0d1cb0 msgr2=0x7f1fdc0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.270+0000 7f1fdb200640 1 -- 172.21.15.46:0/30659887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc0c9370 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fdb200640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4035040 tx=0x7f1fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4037040 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd4042070 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4030b30 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc0c9650 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.271+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc0c9c10 con 0x7f1fdc162a70 2024-04-26T23:38:38.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.272+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc0693b0 con 0x7f1fdc162a70 2024-04-26T23:38:38.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.273+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fd4030cd0 con 0x7f1fdc162a70 2024-04-26T23:38:38.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.274+0000 7f1fd2a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:38.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.275+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f1fd40acc20 con 0x7f1fdc162a70 2024-04-26T23:38:38.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.275+0000 7f1fda800640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:38.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.276+0000 7f1fda800640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc1594d0 tx=0x7f1fcc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:38.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.280+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fd4045360 con 0x7f1fdc162a70 2024-04-26T23:38:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:38.530+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1fdc0c8fb0 con 0x7f1fdc162a70 2024-04-26T23:38:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.049+0000 7f1fd2a00640 1 -- 172.21.15.46:0/30659887 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1fd407ab50 con 0x7f1fdc162a70 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.052+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 msgr2=0x7f1fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.052+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc1594d0 tx=0x7f1fcc009040 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.052+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 msgr2=0x7f1fdc155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.052+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4035040 tx=0x7f1fd40046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 shutdown_connections 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fdc162a70 0x7f1fdc155fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fdc0d3830 0x7f1fdc0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 --2- 172.21.15.46:0/30659887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fdc0d1cb0 0x7f1fdc0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 >> 172.21.15.46:0/30659887 conn(0x7f1fdc0d4f20 msgr2=0x7f1fdc161d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 shutdown_connections 2024-04-26T23:38:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.053+0000 7f1fe0a00640 1 -- 172.21.15.46:0/30659887 wait complete. 2024-04-26T23:38:39.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:38:39.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.286+0000 7f2650a00640 1 Processor -- start 2024-04-26T23:38:39.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.287+0000 7f2650a00640 1 -- start start 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2650a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 0x7f264c0b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2650a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0b6140 con 0x7f264c0d5660 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2650a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0d0f10 con 0x7f264c0d1fa0 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2650a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0b82e0 con 0x7f264c1628e0 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2649e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f264c0d5660 0x7f264c0d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42350/0 (socket says 172.21.15.46:42350) 2024-04-26T23:38:39.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.288+0000 7f2649e00640 1 -- 172.21.15.46:0/3174598904 learned_addr learned my addr 172.21.15.46:0/3174598904 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f264a800640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 0x7f264c0b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588177053 0 0) 0x7f2634001490 con 0x7f264c0d5660 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f262c003520 con 0x7f264c0d5660 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3672454149 0 0) 0x7f262c003520 con 0x7f264c0d5660 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 msgr2=0x7f264c0b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 0x7f264c0b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 >> v1:172.21.15.142:6789/0 conn(0x7f264c0d1fa0 legacy=0x7f264c0d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c162cc0 con 0x7f264c0d5660 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2634001bc0 con 0x7f264c0d5660 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.289+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 >> v1:172.21.15.46:6789/0 conn(0x7f264c0d5660 legacy=0x7f264c0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f262c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f262c004800 0x7f262c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 0x7f262c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00ba70 con 0x7f262c0039e0 2024-04-26T23:38:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00bc20 con 0x7f262c004800 2024-04-26T23:38:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f262c00be20 con 0x7f264c1628e0 2024-04-26T23:38:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f262c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f2649e00640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f262c004800 0x7f262c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 -- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 msgr2=0x7f262c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c1628e0 0x7f262c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 -- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f262c004800 msgr2=0x7f262c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f262c004800 0x7f262c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.290+0000 7f264a800640 1 -- 172.21.15.46:0/3174598904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c003520 con 0x7f262c0039e0 2024-04-26T23:38:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.291+0000 7f264a800640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f262c003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f263c009a90 tx=0x7f262c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ca45db443a4a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c036070 con 0x7f262c0039e0 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c031400 con 0x7f262c0039e0 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2649400640 1 -- 172.21.15.46:0/3174598904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c009690 con 0x7f262c0039e0 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2650a00640 1 -- 172.21.15.46:0/3174598904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 msgr2=0x7f262c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2650a00640 1 --2- 172.21.15.46:0/3174598904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f262c003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f263c009a90 tx=0x7f262c005830 comp rx=0 tx=0).stop 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f264b200640 1 -- 172.21.15.46:0/3174598904 reap_dead start 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2650a00640 1 -- 172.21.15.46:0/3174598904 shutdown_connections 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2650a00640 1 -- 172.21.15.46:0/3174598904 >> 172.21.15.46:0/3174598904 conn(0x7f264c09e2c0 msgr2=0x7f264c0d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.292+0000 7f2650a00640 1 -- 172.21.15.46:0/3174598904 shutdown_connections 2024-04-26T23:38:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.293+0000 7f2650a00640 1 -- 172.21.15.46:0/3174598904 wait complete. 2024-04-26T23:38:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.294+0000 7f2650a00640 1 Processor -- start 2024-04-26T23:38:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.294+0000 7f2650a00640 1 -- start start 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f264c1f2300 0x7f264c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c159450 0x7f264c1f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0d0c30 con 0x7f262c0039e0 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0d2f30 con 0x7f264c1f2300 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f2650a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0d2bc0 con 0x7f264c159450 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.295+0000 7f264a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36436/0 (socket says 172.21.15.46:36436) 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 -- 172.21.15.46:0/779191946 learned_addr learned my addr 172.21.15.46:0/779191946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 -- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c159450 msgr2=0x7f264c1f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c159450 0x7f264c1f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 -- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f264c1f2300 msgr2=0x7f264c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f264c1f2300 0x7f264c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 -- 172.21.15.46:0/779191946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0cc420 con 0x7f262c0039e0 2024-04-26T23:38:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f264a800640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f263c034040 tx=0x7f263c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.296+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c040020 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.297+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c003c40 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.297+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c036040 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.297+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c0cc700 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.297+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1f3070 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.298+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c160a90 con 0x7f262c0039e0 2024-04-26T23:38:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.299+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f263c043020 con 0x7f262c0039e0 2024-04-26T23:38:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.300+0000 7f2642a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 0x7f262806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.300+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f263c0ac190 con 0x7f262c0039e0 2024-04-26T23:38:39.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.300+0000 7f2649e00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 0x7f262806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:39.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.301+0000 7f2649e00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 0x7f262806ee30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2634007c40 tx=0x7f2634009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:39.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.304+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f264c160a90 con 0x7f262c0039e0 2024-04-26T23:38:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:39.502+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f264c0693b0 con 0x7f262c0039e0 2024-04-26T23:38:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.417+0000 7f2642a00640 1 -- 172.21.15.46:0/779191946 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f263c04a070 con 0x7f262c0039e0 2024-04-26T23:38:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:38:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 msgr2=0x7f262806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 0x7f262806ee30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2634007c40 tx=0x7f2634009040 comp rx=0 tx=0).stop 2024-04-26T23:38:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 msgr2=0x7f264c1f2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f263c034040 tx=0x7f263c004730 comp rx=0 tx=0).stop 2024-04-26T23:38:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 shutdown_connections 2024-04-26T23:38:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f262806c970 0x7f262806ee30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f264c159450 0x7f264c1f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f264c1f2300 0x7f264c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 --2- 172.21.15.46:0/779191946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f262c0039e0 0x7f264c1f2b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.421+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 >> 172.21.15.46:0/779191946 conn(0x7f264c09e2c0 msgr2=0x7f264c160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:40.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.422+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 shutdown_connections 2024-04-26T23:38:40.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.424+0000 7f2650a00640 1 -- 172.21.15.46:0/779191946 wait complete. 2024-04-26T23:38:40.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:38:40.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.649+0000 7f0ae3200640 1 Processor -- start 2024-04-26T23:38:40.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.649+0000 7f0ae3200640 1 -- start start 2024-04-26T23:38:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.651+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 0x7f0ae40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.651+0000 7f0ae3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.651+0000 7f0ae3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae40b3ce0 con 0x7f0ae4146b20 2024-04-26T23:38:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.651+0000 7f0ae3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae40b48f0 con 0x7f0ae40d17f0 2024-04-26T23:38:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.651+0000 7f0ae3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40b3fc0 con 0x7f0ae4146740 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.652+0000 7f0ae2800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.652+0000 7f0ae1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 0x7f0ae40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.652+0000 7f0ae2800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:37280/0 (socket says 172.21.15.46:37280) 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.652+0000 7f0ae1400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0ae4146b20 0x7f0ae40d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:54972/0 (socket says 172.21.15.46:54972) 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.652+0000 7f0ae2800640 1 -- 172.21.15.46:0/372019496 learned_addr learned my addr 172.21.15.46:0/372019496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.653+0000 7f0ae1e00640 1 -- 172.21.15.46:0/372019496 >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 msgr2=0x7f0ae40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.653+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/372019496 >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.653+0000 7f0ae1e00640 1 -- 172.21.15.46:0/372019496 >> v1:172.21.15.142:6789/0 conn(0x7f0ae4146b20 legacy=0x7f0ae40d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.653+0000 7f0ae1e00640 1 -- 172.21.15.46:0/372019496 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40b42a0 con 0x7f0ae4146740 2024-04-26T23:38:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.653+0000 7f0ae2800640 1 --2- 172.21.15.46:0/372019496 >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:38:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.654+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/372019496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 0x7f0ae40cd6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097a0 tx=0x7f0ad402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78addc00d9d7df52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae0a00640 1 -- 172.21.15.46:0/372019496 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4037070 con 0x7f0ae4146740 2024-04-26T23:38:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae0a00640 1 -- 172.21.15.46:0/372019496 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad402ae40 con 0x7f0ae4146740 2024-04-26T23:38:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae0a00640 1 -- 172.21.15.46:0/372019496 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4032660 con 0x7f0ae4146740 2024-04-26T23:38:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 -- 172.21.15.46:0/372019496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 msgr2=0x7f0ae40cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 --2- 172.21.15.46:0/372019496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 0x7f0ae40cd6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097a0 tx=0x7f0ad402a0f0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 -- 172.21.15.46:0/372019496 shutdown_connections 2024-04-26T23:38:40.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 --2- 172.21.15.46:0/372019496 >> v2:172.21.15.142:3300/0 conn(0x7f0ae40d17f0 0x7f0ae40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 --2- 172.21.15.46:0/372019496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae4146740 0x7f0ae40cd6d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 -- 172.21.15.46:0/372019496 >> 172.21.15.46:0/372019496 conn(0x7f0ae406c770 msgr2=0x7f0ae406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:40.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 -- 172.21.15.46:0/372019496 shutdown_connections 2024-04-26T23:38:40.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.655+0000 7f0ae3200640 1 -- 172.21.15.46:0/372019496 wait complete. 2024-04-26T23:38:40.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.657+0000 7f0ae3200640 1 Processor -- start 2024-04-26T23:38:40.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.657+0000 7f0ae3200640 1 -- start start 2024-04-26T23:38:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 0x7f0ae4144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 0x7f0ae413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cdc10 con 0x7f0ae4146b20 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae415e560 con 0x7f0ae4146740 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.658+0000 7f0ae3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40cdd90 con 0x7f0ae40d17f0 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36448/0 (socket says 172.21.15.46:36448) 2024-04-26T23:38:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 -- 172.21.15.46:0/3131395125 learned_addr learned my addr 172.21.15.46:0/3131395125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 0x7f0ae4144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 -- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 msgr2=0x7f0ae4144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae1400640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 0x7f0ae413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 0x7f0ae4144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 -- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 msgr2=0x7f0ae413d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 0x7f0ae413d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae2800640 1 -- 172.21.15.46:0/3131395125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae413dcd0 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 0x7f0ae4144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.659+0000 7f0ae1400640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 0x7f0ae413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ae2800640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00efc0 tx=0x7f0adc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00cb20 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc00ccc0 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc003bd0 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae413dfb0 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.660+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae4142f80 con 0x7f0ae4146b20 2024-04-26T23:38:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.662+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae415d0d0 con 0x7f0ae4146b20 2024-04-26T23:38:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.663+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0adc01b030 con 0x7f0ae4146b20 2024-04-26T23:38:40.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.663+0000 7f0ada000640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 0x7f0ac406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:40.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.664+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f0adc08c100 con 0x7f0ae4146b20 2024-04-26T23:38:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.664+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 0x7f0ac406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.666+0000 7f0ae1e00640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 0x7f0ac406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4034000 tx=0x7f0ad402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:40.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.669+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ae415d0d0 con 0x7f0ae4146b20 2024-04-26T23:38:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:40.881+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ae4143180 con 0x7f0ae4146b20 2024-04-26T23:38:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.431+0000 7f0ada000640 1 -- 172.21.15.46:0/3131395125 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f0adc022020 con 0x7f0ae4146b20 2024-04-26T23:38:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 msgr2=0x7f0ac406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 0x7f0ac406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4034000 tx=0x7f0ad402e000 comp rx=0 tx=0).stop 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 msgr2=0x7f0ae413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00efc0 tx=0x7f0adc00c490 comp rx=0 tx=0).stop 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 shutdown_connections 2024-04-26T23:38:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ac406c920 0x7f0ac406ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ae4146b20 0x7f0ae413ea00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae4146740 0x7f0ae413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 --2- 172.21.15.46:0/3131395125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ae40d17f0 0x7f0ae4144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.436+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 >> 172.21.15.46:0/3131395125 conn(0x7f0ae406c770 msgr2=0x7f0ae415d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.438+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 shutdown_connections 2024-04-26T23:38:41.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.440+0000 7f0ae3200640 1 -- 172.21.15.46:0/3131395125 wait complete. 2024-04-26T23:38:41.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T23:38:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.668+0000 7f4333a00640 1 Processor -- start 2024-04-26T23:38:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.668+0000 7f4333a00640 1 -- start start 2024-04-26T23:38:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f4333a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f432c1469c0 0x7f432c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f4333a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f432c0cfa10 0x7f432c0727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f4333a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c146da0 con 0x7f432c0cf560 2024-04-26T23:38:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f4333a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d0ac0 con 0x7f432c0cfa10 2024-04-26T23:38:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f4333a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0cfeb0 con 0x7f432c1469c0 2024-04-26T23:38:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f432be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f432c0cf560 0x7f432c0b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42360/0 (socket says 172.21.15.46:42360) 2024-04-26T23:38:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.669+0000 7f432be00640 1 -- 172.21.15.46:0/916988696 learned_addr learned my addr 172.21.15.46:0/916988696 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331a00640 1 --2- 172.21.15.46:0/916988696 >> v2:172.21.15.46:3300/0 conn(0x7f432c0cfa10 0x7f432c0727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331000640 1 --2- 172.21.15.46:0/916988696 >> v2:172.21.15.142:3300/0 conn(0x7f432c1469c0 0x7f432c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f432b400640 1 -- 172.21.15.46:0/916988696 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728785748 0 0) 0x7f432c146da0 con 0x7f432c0cf560 2024-04-26T23:38:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331a00640 1 -- 172.21.15.46:0/916988696 >> v2:172.21.15.142:3300/0 conn(0x7f432c1469c0 msgr2=0x7f432c15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331a00640 1 --2- 172.21.15.46:0/916988696 >> v2:172.21.15.142:3300/0 conn(0x7f432c1469c0 0x7f432c15d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331a00640 1 -- 172.21.15.46:0/916988696 >> v1:172.21.15.46:6789/0 conn(0x7f432c0cf560 legacy=0x7f432c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.670+0000 7f4331a00640 1 -- 172.21.15.46:0/916988696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0d0190 con 0x7f432c0cfa10 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.671+0000 7f4331a00640 1 --2- 172.21.15.46:0/916988696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c0cfa10 0x7f432c0727d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f432c06fd00 tx=0x7f432402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4c87c9779e1aac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.671+0000 7f432b400640 1 -- 172.21.15.46:0/916988696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324037070 con 0x7f432c0cfa10 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f432b400640 1 -- 172.21.15.46:0/916988696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324032440 con 0x7f432c0cfa10 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f432b400640 1 -- 172.21.15.46:0/916988696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324032810 con 0x7f432c0cfa10 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 -- 172.21.15.46:0/916988696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c0cfa10 msgr2=0x7f432c0727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 --2- 172.21.15.46:0/916988696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c0cfa10 0x7f432c0727d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f432c06fd00 tx=0x7f432402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 -- 172.21.15.46:0/916988696 shutdown_connections 2024-04-26T23:38:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 --2- 172.21.15.46:0/916988696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c0cfa10 0x7f432c0727d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 --2- 172.21.15.46:0/916988696 >> v2:172.21.15.142:3300/0 conn(0x7f432c1469c0 0x7f432c15d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 -- 172.21.15.46:0/916988696 >> 172.21.15.46:0/916988696 conn(0x7f432c06c770 msgr2=0x7f432c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 -- 172.21.15.46:0/916988696 shutdown_connections 2024-04-26T23:38:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.672+0000 7f4333a00640 1 -- 172.21.15.46:0/916988696 wait complete. 2024-04-26T23:38:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.674+0000 7f4333a00640 1 Processor -- start 2024-04-26T23:38:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.674+0000 7f4333a00640 1 -- start start 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 0x7f432c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 0x7f432c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0b35d0 con 0x7f432c1469c0 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d0750 con 0x7f432c0cf560 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4333a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0b3750 con 0x7f432c0cfa10 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4331a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4331a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36474/0 (socket says 172.21.15.46:36474) 2024-04-26T23:38:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.675+0000 7f4331a00640 1 -- 172.21.15.46:0/3391032874 learned_addr learned my addr 172.21.15.46:0/3391032874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331000640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 0x7f432c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f432be00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 0x7f432c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 -- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 msgr2=0x7f432c0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 0x7f432c0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 -- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 msgr2=0x7f432c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 0x7f432c0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 -- 172.21.15.46:0/3391032874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c135770 con 0x7f432c1469c0 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331000640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 0x7f432c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f432be00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 0x7f432c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.676+0000 7f4331a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4324009890 tx=0x7f43240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.677+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324037040 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.677+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324042070 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.677+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324030b30 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.677+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c135a50 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.677+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c0c5d10 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.678+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c069530 con 0x7f432c1469c0 2024-04-26T23:38:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.679+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4324030cd0 con 0x7f432c1469c0 2024-04-26T23:38:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.680+0000 7f4328c00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.680+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f43240ac800 con 0x7f432c1469c0 2024-04-26T23:38:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.680+0000 7f4331000640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.681+0000 7f4331000640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f431c009f50 tx=0x7f431c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.684+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4324032e20 con 0x7f432c1469c0 2024-04-26T23:38:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:41.888+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f432c0b2fa0 con 0x7f432c1469c0 2024-04-26T23:38:42.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.442+0000 7f4328c00640 1 -- 172.21.15.46:0/3391032874 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f432403a350 con 0x7f432c1469c0 2024-04-26T23:38:42.445 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-04-26T23:38:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.445+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 msgr2=0x7f430806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.445+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f431c009f50 tx=0x7f431c00b040 comp rx=0 tx=0).stop 2024-04-26T23:38:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.445+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 msgr2=0x7f432c137660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.445+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4324009890 tx=0x7f43240046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 shutdown_connections 2024-04-26T23:38:42.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1469c0 0x7f432c137660 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432c0cfa10 0x7f432c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 --2- 172.21.15.46:0/3391032874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f432c0cf560 0x7f432c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.447+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 >> 172.21.15.46:0/3391032874 conn(0x7f432c06c770 msgr2=0x7f432c0afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.448+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 shutdown_connections 2024-04-26T23:38:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.448+0000 7f4333a00640 1 -- 172.21.15.46:0/3391032874 wait complete. 2024-04-26T23:38:42.478 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:38:42.479 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:38:42.479 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:38:42.479 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:38:42.479 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:38:42.479 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:38:42.479 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:38:42.480 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:38:42.480 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:38:42.480 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:38:42.480 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:38:42.480 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:38:42.480 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:38:42.480 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:38:42.481 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:38:42.481 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:38:42.497 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:38:42.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:38:42.501 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:38:42.501 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:38:42.507 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:38:42.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:38:42.508 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:38:42.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:38:42.555 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:38:42.555 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:38:42.556 DEBUG:teuthology.parallel:result is None 2024-04-26T23:38:42.556 DEBUG:teuthology.parallel:result is None 2024-04-26T23:38:42.561 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:38:42.562 DEBUG:teuthology.parallel:result is None 2024-04-26T23:38:42.562 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:38:42.562 DEBUG:teuthology.parallel:result is None 2024-04-26T23:38:42.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:38:42.708 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:38:42.709 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:38:42.710 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:38:42.713 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:38:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.744+0000 7fbaf5c00640 1 Processor -- start 2024-04-26T23:38:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.745+0000 7fbaf5c00640 1 -- start start 2024-04-26T23:38:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaf5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbaf0160bd0 0x7fbaf0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaf5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf00732c0 0x7fbaf0073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaf5c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf0161940 con 0x7fbaf006e510 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaf5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf0071f80 con 0x7fbaf0160bd0 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaf5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf0072260 con 0x7fbaf00732c0 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbae7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbaf0160bd0 0x7fbaf0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbae7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbaf0160bd0 0x7fbaf0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56882/0 (socket says 172.21.15.46:56882) 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbae7e00640 1 -- 172.21.15.46:0/1713918373 learned_addr learned my addr 172.21.15.46:0/1713918373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.746+0000 7fbaef400640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf00732c0 0x7fbaf0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.747+0000 7fbae7e00640 1 -- 172.21.15.46:0/1713918373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf00732c0 msgr2=0x7fbaf0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.747+0000 7fbae7e00640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf00732c0 0x7fbaf0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.747+0000 7fbae7e00640 1 -- 172.21.15.46:0/1713918373 >> v1:172.21.15.142:6789/0 conn(0x7fbaf006e510 legacy=0x7fbaf006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.747+0000 7fbae7e00640 1 -- 172.21.15.46:0/1713918373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0072540 con 0x7fbaf0160bd0 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.748+0000 7fbae7e00640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf0160bd0 0x7fbaf0161060 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b74ed520547081 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.748+0000 7fbaee000640 1 -- 172.21.15.46:0/1713918373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8037070 con 0x7fbaf0160bd0 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.748+0000 7fbaee000640 1 -- 172.21.15.46:0/1713918373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8032440 con 0x7fbaf0160bd0 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.748+0000 7fbaee000640 1 -- 172.21.15.46:0/1713918373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8032810 con 0x7fbaf0160bd0 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 -- 172.21.15.46:0/1713918373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf0160bd0 msgr2=0x7fbaf0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf0160bd0 0x7fbaf0161060 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 -- 172.21.15.46:0/1713918373 shutdown_connections 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf00732c0 0x7fbaf0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 --2- 172.21.15.46:0/1713918373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf0160bd0 0x7fbaf0161060 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 -- 172.21.15.46:0/1713918373 >> 172.21.15.46:0/1713918373 conn(0x7fbaf006c530 msgr2=0x7fbaf006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 -- 172.21.15.46:0/1713918373 shutdown_connections 2024-04-26T23:38:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.749+0000 7fbaf5c00640 1 -- 172.21.15.46:0/1713918373 wait complete. 2024-04-26T23:38:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.751+0000 7fbaf5c00640 1 Processor -- start 2024-04-26T23:38:42.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.751+0000 7fbaf5c00640 1 -- start start 2024-04-26T23:38:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 0x7fbaf013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf0160bd0 0x7fbaf0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0161720 con 0x7fbaf006e510 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01615a0 con 0x7fbaf00732c0 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaf5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf0072d40 con 0x7fbaf0160bd0 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 0x7fbaf013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56884/0 (socket says 172.21.15.46:56884) 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.752+0000 7fbaeea00640 1 -- 172.21.15.46:0/2437380189 learned_addr learned my addr 172.21.15.46:0/2437380189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbaef400640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf0160bd0 0x7fbaf0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 -- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf0160bd0 msgr2=0x7fbaf0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf0160bd0 0x7fbaf0154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 -- 172.21.15.46:0/2437380189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 msgr2=0x7fbaf0141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 -- 172.21.15.46:0/2437380189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0142220 con 0x7fbaf00732c0 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbaeea00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.753+0000 7fbae7e00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 0x7fbaf013c3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbad8031000 tx=0x7fbad80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.754+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8041020 con 0x7fbaf00732c0 2024-04-26T23:38:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.754+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad803d6e0 con 0x7fbaf00732c0 2024-04-26T23:38:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.754+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8037040 con 0x7fbaf00732c0 2024-04-26T23:38:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.754+0000 7fbaf5c00640 1 -- 172.21.15.46:0/2437380189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf0142500 con 0x7fbaf00732c0 2024-04-26T23:38:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.754+0000 7fbaf5c00640 1 -- 172.21.15.46:0/2437380189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf0142ac0 con 0x7fbaf00732c0 2024-04-26T23:38:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.755+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf0069530 con 0x7fbaf00732c0 2024-04-26T23:38:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.756+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbad8044020 con 0x7fbaf00732c0 2024-04-26T23:38:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.757+0000 7fbae6a00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 0x7fbacc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.757+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7fbad80ac0d0 con 0x7fbaf00732c0 2024-04-26T23:38:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.757+0000 7fbaeea00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 0x7fbacc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.758+0000 7fbaeea00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 0x7fbacc06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbae0001b90 tx=0x7fbae0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.763+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbad804b030 con 0x7fbaf00732c0 2024-04-26T23:38:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:42.979+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbaf0069e80 con 0x7fbaf00732c0 2024-04-26T23:38:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.482+0000 7fbae6a00640 1 -- 172.21.15.46:0/2437380189 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7fbad8079f00 con 0x7fbaf00732c0 2024-04-26T23:38:43.485 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:38:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 msgr2=0x7fbacc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 0x7fbacc06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbae0001b90 tx=0x7fbae0009040 comp rx=0 tx=0).stop 2024-04-26T23:38:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 msgr2=0x7fbaf013c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 0x7fbaf013c3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbad8031000 tx=0x7fbad80046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 shutdown_connections 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbacc06c8f0 0x7fbacc06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbaf0160bd0 0x7fbaf0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf00732c0 0x7fbaf013c3b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 --2- 172.21.15.46:0/2437380189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaf006e510 0x7fbaf0141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.486+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 >> 172.21.15.46:0/2437380189 conn(0x7fbaf006c530 msgr2=0x7fbaf0073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.487+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 shutdown_connections 2024-04-26T23:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.487+0000 7fbadfe00640 1 -- 172.21.15.46:0/2437380189 wait complete. 2024-04-26T23:38:43.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:38:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.711+0000 7f78efe00640 1 Processor -- start 2024-04-26T23:38:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.711+0000 7f78efe00640 1 -- start start 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78efe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78f00cf450 0x7f78f0073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f0072190 0x7f78f00cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78efe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f006f470 con 0x7f78f00cfd70 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78efe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f78f0072930 con 0x7f78f0072190 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78efe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f78f0072c10 con 0x7f78f00cf450 2024-04-26T23:38:43.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f0072190 0x7f78f00cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f0072190 0x7f78f00cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56888/0 (socket says 172.21.15.46:56888) 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.713+0000 7f78ee000640 1 -- 172.21.15.46:0/792155719 learned_addr learned my addr 172.21.15.46:0/792155719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.714+0000 7f78ee000640 1 -- 172.21.15.46:0/792155719 >> v2:172.21.15.142:3300/0 conn(0x7f78f00cf450 msgr2=0x7f78f0073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.714+0000 7f78ee000640 1 --2- 172.21.15.46:0/792155719 >> v2:172.21.15.142:3300/0 conn(0x7f78f00cf450 0x7f78f0073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.714+0000 7f78ee000640 1 -- 172.21.15.46:0/792155719 >> v1:172.21.15.142:6789/0 conn(0x7f78f00cfd70 legacy=0x7f78f006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.714+0000 7f78ee000640 1 -- 172.21.15.46:0/792155719 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0072ef0 con 0x7f78f0072190 2024-04-26T23:38:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.715+0000 7f78ee000640 1 --2- 172.21.15.46:0/792155719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f0072190 0x7f78f00cf830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f78f0070090 tx=0x7f78dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee41e43d96db38ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.715+0000 7f78ed600640 1 -- 172.21.15.46:0/792155719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc037070 con 0x7f78f0072190 2024-04-26T23:38:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.715+0000 7f78ed600640 1 -- 172.21.15.46:0/792155719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc032440 con 0x7f78f0072190 2024-04-26T23:38:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.715+0000 7f78ed600640 1 -- 172.21.15.46:0/792155719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc032810 con 0x7f78f0072190 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 -- 172.21.15.46:0/792155719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f0072190 msgr2=0x7f78f00cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 --2- 172.21.15.46:0/792155719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f0072190 0x7f78f00cf830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f78f0070090 tx=0x7f78dc02a230 comp rx=0 tx=0).stop 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 -- 172.21.15.46:0/792155719 shutdown_connections 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 --2- 172.21.15.46:0/792155719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f0072190 0x7f78f00cf830 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 --2- 172.21.15.46:0/792155719 >> v2:172.21.15.142:3300/0 conn(0x7f78f00cf450 0x7f78f0073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 -- 172.21.15.46:0/792155719 >> 172.21.15.46:0/792155719 conn(0x7f78f00b2820 msgr2=0x7f78f00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 -- 172.21.15.46:0/792155719 shutdown_connections 2024-04-26T23:38:43.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.716+0000 7f78efe00640 1 -- 172.21.15.46:0/792155719 wait complete. 2024-04-26T23:38:43.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.718+0000 7f78efe00640 1 Processor -- start 2024-04-26T23:38:43.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.718+0000 7f78efe00640 1 -- start start 2024-04-26T23:38:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78f0072190 0x7f78f01e0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78f00cf450 0x7f78f0142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0072620 con 0x7f78f00cfd70 2024-04-26T23:38:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78f00734b0 con 0x7f78f00cf450 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78efe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78f00d0180 con 0x7f78f0072190 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56904/0 (socket says 172.21.15.46:56904) 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.719+0000 7f78ef400640 1 -- 172.21.15.46:0/3233265549 learned_addr learned my addr 172.21.15.46:0/3233265549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78eea00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78f0072190 0x7f78f01e0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ee000640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78f00cf450 0x7f78f0142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 -- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78f0072190 msgr2=0x7f78f01e0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78f0072190 0x7f78f01e0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 -- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78f00cf450 msgr2=0x7f78f0142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78f00cf450 0x7f78f0142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 -- 172.21.15.46:0/3233265549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f01df3d0 con 0x7f78f00cfd70 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78ef400640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c780 tx=0x7f78e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8017070 con 0x7f78f00cfd70 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e8003da0 con 0x7f78f00cfd70 2024-04-26T23:38:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80052b0 con 0x7f78f00cfd70 2024-04-26T23:38:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.720+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f01df6b0 con 0x7f78f00cfd70 2024-04-26T23:38:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.721+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01dfc70 con 0x7f78f00cfd70 2024-04-26T23:38:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.722+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f00693b0 con 0x7f78f00cfd70 2024-04-26T23:38:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.723+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78e8005450 con 0x7f78f00cfd70 2024-04-26T23:38:43.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.725+0000 7f78e6a00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:43.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.725+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f78e808b020 con 0x7f78f00cfd70 2024-04-26T23:38:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.725+0000 7f78eea00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.727+0000 7f78eea00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 0x7f78d006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009890 tx=0x7f78e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:43.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:43.730+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78e8059570 con 0x7f78f00cfd70 2024-04-26T23:38:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.005+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f78f0073be0 con 0x7f78f00cfd70 2024-04-26T23:38:44.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.006+0000 7f78e6a00640 1 -- 172.21.15.46:0/3233265549 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f78e8058e70 con 0x7f78f00cfd70 2024-04-26T23:38:44.009 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:38:44.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.012 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.015 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:38:44.016 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:38:44.019 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixm2M3/JRAA8wMPFwFG0PE/z1KFRait6g==", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T23:38:44.020 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixmGdj/JRAAH0/Oyb7WWeUU88lD62cdQA==", 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixmjuH/JRAA9T7o7A74fLl1lDnPapYldA==", 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixmVuv/JRAAwlshriGaKJUkTcK4jEg91A==", 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T23:38:44.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixm+PT/JRAAFUqqy1mWascl2NBuLLVgxQ==", 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCQOixmkP7/JRAAW+pLF9ib4FP7hkzK2ZUwOQ==", 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:44.024 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:38:44.024 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 msgr2=0x7f78d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 0x7f78d006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009890 tx=0x7f78e0008040 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 msgr2=0x7f78f01e7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c780 tx=0x7f78e800cc40 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 shutdown_connections 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cfd70 0x7f78f01e7440 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78f00cf450 0x7f78f0142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 --2- 172.21.15.46:0/3233265549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78f0072190 0x7f78f01e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 >> 172.21.15.46:0/3233265549 conn(0x7f78f00b2820 msgr2=0x7f78f00b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 shutdown_connections 2024-04-26T23:38:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.009+0000 7f78efe00640 1 -- 172.21.15.46:0/3233265549 wait complete. 2024-04-26T23:38:44.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T23:38:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.236+0000 7f9eea000640 1 Processor -- start 2024-04-26T23:38:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.236+0000 7f9eea000640 1 -- start start 2024-04-26T23:38:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9eea000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 0x7f9ee40d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9eea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ee40d0d40 0x7f9ee40d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9eea000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee40cff80 con 0x7f9ee40d5d20 2024-04-26T23:38:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9eea000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee40d08b0 con 0x7f9ee40d0d40 2024-04-26T23:38:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9eea000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee40d0260 con 0x7f9ee4162720 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9ee2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ee40d0d40 0x7f9ee40d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9ee2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ee40d0d40 0x7f9ee40d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56916/0 (socket says 172.21.15.46:56916) 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9ee2a00640 1 -- 172.21.15.46:0/1528254914 learned_addr learned my addr 172.21.15.46:0/1528254914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.238+0000 7f9ee3400640 1 --2- 172.21.15.46:0/1528254914 >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 0x7f9ee40d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.239+0000 7f9ee2a00640 1 -- 172.21.15.46:0/1528254914 >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 msgr2=0x7f9ee40d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.239+0000 7f9ee2a00640 1 --2- 172.21.15.46:0/1528254914 >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 0x7f9ee40d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.239+0000 7f9ee2a00640 1 -- 172.21.15.46:0/1528254914 >> v1:172.21.15.46:6789/0 conn(0x7f9ee40d5d20 legacy=0x7f9ee4162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.239+0000 7f9ee2a00640 1 -- 172.21.15.46:0/1528254914 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee40d0540 con 0x7f9ee40d0d40 2024-04-26T23:38:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.239+0000 7f9ee3400640 1 --2- 172.21.15.46:0/1528254914 >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 0x7f9ee40d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.240+0000 7f9ee2a00640 1 --2- 172.21.15.46:0/1528254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee40d3130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9ee406cdf0 tx=0x7f9ecc02a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aed9873bdb385ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.240+0000 7f9ee2000640 1 -- 172.21.15.46:0/1528254914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc037070 con 0x7f9ee40d0d40 2024-04-26T23:38:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.240+0000 7f9ee2000640 1 -- 172.21.15.46:0/1528254914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ecc032440 con 0x7f9ee40d0d40 2024-04-26T23:38:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.240+0000 7f9ee2000640 1 -- 172.21.15.46:0/1528254914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc032810 con 0x7f9ee40d0d40 2024-04-26T23:38:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 -- 172.21.15.46:0/1528254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 msgr2=0x7f9ee40d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 --2- 172.21.15.46:0/1528254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee40d3130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9ee406cdf0 tx=0x7f9ecc02a280 comp rx=0 tx=0).stop 2024-04-26T23:38:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 -- 172.21.15.46:0/1528254914 shutdown_connections 2024-04-26T23:38:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 --2- 172.21.15.46:0/1528254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee40d3130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 --2- 172.21.15.46:0/1528254914 >> v2:172.21.15.142:3300/0 conn(0x7f9ee4162720 0x7f9ee40d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.241+0000 7f9eea000640 1 -- 172.21.15.46:0/1528254914 >> 172.21.15.46:0/1528254914 conn(0x7f9ee40d50b0 msgr2=0x7f9ee40d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.242+0000 7f9eea000640 1 -- 172.21.15.46:0/1528254914 shutdown_connections 2024-04-26T23:38:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.242+0000 7f9eea000640 1 -- 172.21.15.46:0/1528254914 wait complete. 2024-04-26T23:38:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.244+0000 7f9eea000640 1 Processor -- start 2024-04-26T23:38:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.244+0000 7f9eea000640 1 -- start start 2024-04-26T23:38:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ee40d5d20 0x7f9ee40a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ee4162720 0x7f9ee40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4162b00 con 0x7f9ee40d0d40 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee40cfbc0 con 0x7f9ee4162720 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9eea000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee409f230 con 0x7f9ee40d5d20 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9ee3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9ee3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56918/0 (socket says 172.21.15.46:56918) 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9ee3400640 1 -- 172.21.15.46:0/285743272 learned_addr learned my addr 172.21.15.46:0/285743272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9ee3e00640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ee4162720 0x7f9ee40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.245+0000 7f9ee2a00640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ee40d5d20 0x7f9ee40a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 -- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ee40d5d20 msgr2=0x7f9ee40a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ee40d5d20 0x7f9ee40a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 -- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ee4162720 msgr2=0x7f9ee40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ee4162720 0x7f9ee40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 -- 172.21.15.46:0/285743272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee409f510 con 0x7f9ee40d0d40 2024-04-26T23:38:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9ee3400640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400cf60 tx=0x7f9ed400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed401a070 con 0x7f9ee40d0d40 2024-04-26T23:38:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed400d840 con 0x7f9ee40d0d40 2024-04-26T23:38:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400dc10 con 0x7f9ee40d0d40 2024-04-26T23:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee409f7f0 con 0x7f9ee40d0d40 2024-04-26T23:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.246+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee409fdb0 con 0x7f9ee40d0d40 2024-04-26T23:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.248+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee409ffb0 con 0x7f9ee40d0d40 2024-04-26T23:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.249+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ed401d070 con 0x7f9ee40d0d40 2024-04-26T23:38:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.250+0000 7f9edb400640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 0x7f9ec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.250+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9ed408c290 con 0x7f9ee40d0d40 2024-04-26T23:38:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.250+0000 7f9ee2a00640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 0x7f9ec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.251+0000 7f9ee2a00640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 0x7f9ec006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc034000 tx=0x7f9ecc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.255+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ed405a100 con 0x7f9ee40d0d40 2024-04-26T23:38:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.536+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f9ee40693b0 con 0x7f9ee40d0d40 2024-04-26T23:38:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.541+0000 7f9edb400640 1 -- 172.21.15.46:0/285743272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f9ed405e400 con 0x7f9ee40d0d40 2024-04-26T23:38:44.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 msgr2=0x7f9ec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 0x7f9ec006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc034000 tx=0x7f9ecc02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:44.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 msgr2=0x7f9ee414e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400cf60 tx=0x7f9ed400cf90 comp rx=0 tx=0).stop 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 shutdown_connections 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ec006c8f0 0x7f9ec006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ee4162720 0x7f9ee40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ee40d5d20 0x7f9ee40a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 --2- 172.21.15.46:0/285743272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee40d0d40 0x7f9ee414e4e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 >> 172.21.15.46:0/285743272 conn(0x7f9ee40d50b0 msgr2=0x7f9ee40ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 shutdown_connections 2024-04-26T23:38:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.544+0000 7f9eea000640 1 -- 172.21.15.46:0/285743272 wait complete. 2024-04-26T23:38:44.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T23:38:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.767+0000 7fa112800640 1 Processor -- start 2024-04-26T23:38:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.767+0000 7fa112800640 1 -- start start 2024-04-26T23:38:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.769+0000 7fa112800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa10c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.769+0000 7fa112800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10c0c7a00 con 0x7fa10c15e070 2024-04-26T23:38:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.769+0000 7fa112800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10c06df90 con 0x7fa10c071db0 2024-04-26T23:38:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.769+0000 7fa112800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa10c06dd40 con 0x7fa10c0736b0 2024-04-26T23:38:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.770+0000 7fa110a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa10c15e070 0x7fa10c0cb830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33272/0 (socket says 172.21.15.46:33272) 2024-04-26T23:38:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.770+0000 7fa110a00640 1 -- 172.21.15.46:0/2934502672 learned_addr learned my addr 172.21.15.46:0/2934502672 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.770+0000 7fa111e00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa10c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.770+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991394390 0 0) 0x7fa10c0c7a00 con 0x7fa10c15e070 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.770+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f4003520 con 0x7fa10c15e070 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 278720283 0 0) 0x7fa0f4003520 con 0x7fa10c15e070 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 msgr2=0x7fa10c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa10c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 >> v1:172.21.15.142:6789/0 conn(0x7fa10c071db0 legacy=0x7fa10c072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c073c10 con 0x7fa10c15e070 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa0f8001670 con 0x7fa10c15e070 2024-04-26T23:38:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 >> v1:172.21.15.46:6789/0 conn(0x7fa10c15e070 legacy=0x7fa10c0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa0f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa0f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0f4004d40 0x7fa0f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c06dd40 con 0x7fa0f40039e0 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f400bb50 con 0x7fa0f4004d40 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f400bd50 con 0x7fa10c0736b0 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa111400640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa0f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.771+0000 7fa110a00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa0f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111e00640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0f4004d40 0x7fa0f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111400640 1 -- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 msgr2=0x7fa0f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111400640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c0736b0 0x7fa0f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111400640 1 -- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0f4004d40 msgr2=0x7fa0f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111400640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0f4004d40 0x7fa0f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.772+0000 7fa111400640 1 -- 172.21.15.46:0/2934502672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f4003520 con 0x7fa0f40039e0 2024-04-26T23:38:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.773+0000 7fa111400640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa0f4003e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa100001750 tx=0x7fa0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d7c473c5e68bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.773+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100001370 con 0x7fa0f40039e0 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.773+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1000052d0 con 0x7fa0f40039e0 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.773+0000 7fa10be00640 1 -- 172.21.15.46:0/2934502672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10002abe0 con 0x7fa0f40039e0 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 -- 172.21.15.46:0/2934502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 msgr2=0x7fa0f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 --2- 172.21.15.46:0/2934502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa0f4003e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa100001750 tx=0x7fa0f4005830 comp rx=0 tx=0).stop 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa111e00640 1 -- 172.21.15.46:0/2934502672 reap_dead start 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 -- 172.21.15.46:0/2934502672 shutdown_connections 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 -- 172.21.15.46:0/2934502672 >> 172.21.15.46:0/2934502672 conn(0x7fa10c0d0b20 msgr2=0x7fa10c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 -- 172.21.15.46:0/2934502672 shutdown_connections 2024-04-26T23:38:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.774+0000 7fa112800640 1 -- 172.21.15.46:0/2934502672 wait complete. 2024-04-26T23:38:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.776+0000 7fa112800640 1 Processor -- start 2024-04-26T23:38:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.776+0000 7fa112800640 1 -- start start 2024-04-26T23:38:44.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa10c0cf460 0x7fa10c150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c152b60 0x7fa10c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c0728a0 con 0x7fa0f40039e0 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c0cb310 con 0x7fa10c0cf460 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa112800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa10c072a20 con 0x7fa10c152b60 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa111400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa111400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56938/0 (socket says 172.21.15.46:56938) 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.777+0000 7fa111400640 1 -- 172.21.15.46:0/961074670 learned_addr learned my addr 172.21.15.46:0/961074670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 -- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c152b60 msgr2=0x7fa10c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c152b60 0x7fa10c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 -- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa10c0cf460 msgr2=0x7fa10c150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa10c0cf460 0x7fa10c150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 -- 172.21.15.46:0/961074670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c0d06a0 con 0x7fa0f40039e0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa111400640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa100000c00 tx=0x7fa100001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100042070 con 0x7fa0f40039e0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa100000ed0 con 0x7fa0f40039e0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100037040 con 0x7fa0f40039e0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.778+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c150680 con 0x7fa0f40039e0 2024-04-26T23:38:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.779+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c0c8fa0 con 0x7fa0f40039e0 2024-04-26T23:38:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.780+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c0693b0 con 0x7fa0f40039e0 2024-04-26T23:38:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.781+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1000052d0 con 0x7fa0f40039e0 2024-04-26T23:38:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.782+0000 7fa109600640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 0x7fa0ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.782+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa1000ab280 con 0x7fa0f40039e0 2024-04-26T23:38:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.782+0000 7fa110a00640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 0x7fa0ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.784+0000 7fa110a00640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 0x7fa0ec06ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8001050 tx=0x7fa0f8000f40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:44.786+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa100079740 con 0x7fa0f40039e0 2024-04-26T23:38:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.063+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fa10c0c67d0 con 0x7fa0f40039e0 2024-04-26T23:38:45.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.067+0000 7fa109600640 1 -- 172.21.15.46:0/961074670 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fa100079040 con 0x7fa0f40039e0 2024-04-26T23:38:45.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 msgr2=0x7fa0ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 0x7fa0ec06ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8001050 tx=0x7fa0f8000f40 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 msgr2=0x7fa10c0c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa100000c00 tx=0x7fa100001120 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 shutdown_connections 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa0ec06c920 0x7fa0ec06ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa10c152b60 0x7fa10c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa10c0cf460 0x7fa10c150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 --2- 172.21.15.46:0/961074670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f40039e0 0x7fa10c0c85b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 >> 172.21.15.46:0/961074670 conn(0x7fa10c0d0b20 msgr2=0x7fa10c15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 shutdown_connections 2024-04-26T23:38:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.070+0000 7fa112800640 1 -- 172.21.15.46:0/961074670 wait complete. 2024-04-26T23:38:45.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T23:38:45.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.291+0000 7fceca600640 1 Processor -- start 2024-04-26T23:38:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.291+0000 7fceca600640 1 -- start start 2024-04-26T23:38:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fceca600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcec40af160 0x7fcec4161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fceca600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcec4073880 0x7fcec4071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fceca600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec4073df0 con 0x7fcec40cc8a0 2024-04-26T23:38:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fceca600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcec4072460 con 0x7fcec4073880 2024-04-26T23:38:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fceca600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcec41613b0 con 0x7fcec40af160 2024-04-26T23:38:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fcec2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcec4073880 0x7fcec4071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fcec3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcec40cc8a0 0x7fcec415c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33288/0 (socket says 172.21.15.46:33288) 2024-04-26T23:38:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.293+0000 7fcec3e00640 1 -- 172.21.15.46:0/2216659458 learned_addr learned my addr 172.21.15.46:0/2216659458 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.294+0000 7fcec3400640 1 --2- 172.21.15.46:0/2216659458 >> v2:172.21.15.142:3300/0 conn(0x7fcec40af160 0x7fcec4161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.294+0000 7fcec2a00640 1 -- 172.21.15.46:0/2216659458 >> v2:172.21.15.142:3300/0 conn(0x7fcec40af160 msgr2=0x7fcec4161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.294+0000 7fcec2a00640 1 --2- 172.21.15.46:0/2216659458 >> v2:172.21.15.142:3300/0 conn(0x7fcec40af160 0x7fcec4161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.294+0000 7fcec2a00640 1 -- 172.21.15.46:0/2216659458 >> v1:172.21.15.46:6789/0 conn(0x7fcec40cc8a0 legacy=0x7fcec415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.294+0000 7fcec2a00640 1 -- 172.21.15.46:0/2216659458 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec40740d0 con 0x7fcec4073880 2024-04-26T23:38:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.295+0000 7fcec2a00640 1 --2- 172.21.15.46:0/2216659458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec4073880 0x7fcec4071f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcec409f0f0 tx=0x7fceb002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803b81b18493a0e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.295+0000 7fcec2000640 1 -- 172.21.15.46:0/2216659458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0037070 con 0x7fcec4073880 2024-04-26T23:38:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.295+0000 7fcec2000640 1 -- 172.21.15.46:0/2216659458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb0032440 con 0x7fcec4073880 2024-04-26T23:38:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.295+0000 7fcec2000640 1 -- 172.21.15.46:0/2216659458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0032810 con 0x7fcec4073880 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 -- 172.21.15.46:0/2216659458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec4073880 msgr2=0x7fcec4071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 --2- 172.21.15.46:0/2216659458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec4073880 0x7fcec4071f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcec409f0f0 tx=0x7fceb002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 -- 172.21.15.46:0/2216659458 shutdown_connections 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 --2- 172.21.15.46:0/2216659458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec4073880 0x7fcec4071f20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 --2- 172.21.15.46:0/2216659458 >> v2:172.21.15.142:3300/0 conn(0x7fcec40af160 0x7fcec4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 -- 172.21.15.46:0/2216659458 >> 172.21.15.46:0/2216659458 conn(0x7fcec40d0d10 msgr2=0x7fcec40d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 -- 172.21.15.46:0/2216659458 shutdown_connections 2024-04-26T23:38:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.296+0000 7fceca600640 1 -- 172.21.15.46:0/2216659458 wait complete. 2024-04-26T23:38:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.298+0000 7fceca600640 1 Processor -- start 2024-04-26T23:38:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.298+0000 7fceca600640 1 -- start start 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 0x7fcec414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcec40cc8a0 0x7fcec41515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcec40b05c0 con 0x7fcec40af160 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4161040 con 0x7fcec4073880 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.299+0000 7fceca600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcec40af540 con 0x7fcec40cc8a0 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56958/0 (socket says 172.21.15.46:56958) 2024-04-26T23:38:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 -- 172.21.15.46:0/961100040 learned_addr learned my addr 172.21.15.46:0/961100040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec3400640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 0x7fcec414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 -- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcec40cc8a0 msgr2=0x7fcec41515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcec40cc8a0 0x7fcec41515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 -- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 msgr2=0x7fcec414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 0x7fcec414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 -- 172.21.15.46:0/961100040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec414f280 con 0x7fcec40af160 2024-04-26T23:38:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec3400640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 0x7fcec414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.300+0000 7fcec2a00640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009890 tx=0x7fceb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.301+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0040020 con 0x7fcec40af160 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.301+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb0038040 con 0x7fcec40af160 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.301+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0037070 con 0x7fcec40af160 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.301+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec40cf380 con 0x7fcec40af160 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.301+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec40cf940 con 0x7fcec40af160 2024-04-26T23:38:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.302+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec40ce620 con 0x7fcec40af160 2024-04-26T23:38:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.303+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceb0043020 con 0x7fcec40af160 2024-04-26T23:38:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.304+0000 7fcebb400640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.304+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fceb00abff0 con 0x7fcec40af160 2024-04-26T23:38:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.305+0000 7fcec3400640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.306+0000 7fcec3400640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 0x7fcea406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fceb4007f50 tx=0x7fceb40074a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.309+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcec40ce620 con 0x7fcec40af160 2024-04-26T23:38:45.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.599+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fcec40693b0 con 0x7fcec40af160 2024-04-26T23:38:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.603+0000 7fcebb400640 1 -- 172.21.15.46:0/961100040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fceb004a030 con 0x7fcec40af160 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 msgr2=0x7fcea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 0x7fcea406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fceb4007f50 tx=0x7fceb40074a0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 msgr2=0x7fcec40ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009890 tx=0x7fceb00046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 shutdown_connections 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcec40cc8a0 0x7fcec41515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40af160 0x7fcec40ce8e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 --2- 172.21.15.46:0/961100040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcec4073880 0x7fcec414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 >> 172.21.15.46:0/961100040 conn(0x7fcec40d0d10 msgr2=0x7fcec40af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 shutdown_connections 2024-04-26T23:38:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.606+0000 7fceca600640 1 -- 172.21.15.46:0/961100040 wait complete. 2024-04-26T23:38:45.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T23:38:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.840+0000 7f6b0f000640 1 Processor -- start 2024-04-26T23:38:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.840+0000 7f6b0f000640 1 -- start start 2024-04-26T23:38:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b08163720 0x7f6b080c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6b0806fae0 0x7f6b080c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b08167350 con 0x7f6b0806f590 2024-04-26T23:38:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6b08163b00 con 0x7f6b08163720 2024-04-26T23:38:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6b080cd590 con 0x7f6b0806fae0 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b08163720 0x7f6b080c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6b0806f590 0x7f6b080ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33294/0 (socket says 172.21.15.46:33294) 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.842+0000 7f6b0d000640 1 -- 172.21.15.46:0/3366869513 learned_addr learned my addr 172.21.15.46:0/3366869513 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b07400640 1 --2- 172.21.15.46:0/3366869513 >> v2:172.21.15.142:3300/0 conn(0x7f6b0806fae0 0x7f6b080c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b07e00640 1 -- 172.21.15.46:0/3366869513 >> v2:172.21.15.142:3300/0 conn(0x7f6b0806fae0 msgr2=0x7f6b080c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3366869513 >> v2:172.21.15.142:3300/0 conn(0x7f6b0806fae0 0x7f6b080c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b07e00640 1 -- 172.21.15.46:0/3366869513 >> v1:172.21.15.46:6789/0 conn(0x7f6b0806f590 legacy=0x7f6b080ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b07e00640 1 -- 172.21.15.46:0/3366869513 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081675f0 con 0x7f6b08163720 2024-04-26T23:38:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.843+0000 7f6b06a00640 1 -- 172.21.15.46:0/3366869513 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952373678 0 0) 0x7f6b08167350 con 0x7f6b0806f590 2024-04-26T23:38:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.844+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3366869513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08163720 0x7f6b080c3d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b3820 tx=0x7f6af802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9003275a3e0ecd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b06a00640 1 -- 172.21.15.46:0/3366869513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8037070 con 0x7f6b08163720 2024-04-26T23:38:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b06a00640 1 -- 172.21.15.46:0/3366869513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8032440 con 0x7f6b08163720 2024-04-26T23:38:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b06a00640 1 -- 172.21.15.46:0/3366869513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8032810 con 0x7f6b08163720 2024-04-26T23:38:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 -- 172.21.15.46:0/3366869513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08163720 msgr2=0x7f6b080c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 --2- 172.21.15.46:0/3366869513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08163720 0x7f6b080c3d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b3820 tx=0x7f6af802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 -- 172.21.15.46:0/3366869513 shutdown_connections 2024-04-26T23:38:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 --2- 172.21.15.46:0/3366869513 >> v2:172.21.15.142:3300/0 conn(0x7f6b0806fae0 0x7f6b080c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 --2- 172.21.15.46:0/3366869513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08163720 0x7f6b080c3d00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 -- 172.21.15.46:0/3366869513 >> 172.21.15.46:0/3366869513 conn(0x7f6b080d0d10 msgr2=0x7f6b080d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 -- 172.21.15.46:0/3366869513 shutdown_connections 2024-04-26T23:38:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.845+0000 7f6b0f000640 1 -- 172.21.15.46:0/3366869513 wait complete. 2024-04-26T23:38:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.847+0000 7f6b0f000640 1 Processor -- start 2024-04-26T23:38:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.847+0000 7f6b0f000640 1 -- start start 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 0x7f6b08165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b08163720 0x7f6b080aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08167e90 con 0x7f6b0806fae0 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08167bb0 con 0x7f6b0806f590 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.848+0000 7f6b0f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6b080d07a0 con 0x7f6b08163720 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56984/0 (socket says 172.21.15.46:56984) 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 -- 172.21.15.46:0/2392634024 learned_addr learned my addr 172.21.15.46:0/2392634024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 0x7f6b08165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 -- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b08163720 msgr2=0x7f6b080aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b08163720 0x7f6b080aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 -- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 msgr2=0x7f6b08165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 0x7f6b08165990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 -- 172.21.15.46:0/2392634024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b08164ed0 con 0x7f6b0806fae0 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 0x7f6b08165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.849+0000 7f6b07400640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e870 tx=0x7f6af400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.850+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4017070 con 0x7f6b0806fae0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.850+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af400c940 con 0x7f6b0806fae0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.850+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400cd30 con 0x7f6b0806fae0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.850+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b08072a00 con 0x7f6b0806fae0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.850+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b08165190 con 0x7f6b0806fae0 2024-04-26T23:38:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.851+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b080693b0 con 0x7f6b0806fae0 2024-04-26T23:38:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.852+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6af401a070 con 0x7f6b0806fae0 2024-04-26T23:38:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.853+0000 7f6affe00640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 0x7f6ae806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.853+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f6af408b800 con 0x7f6b0806fae0 2024-04-26T23:38:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.854+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 0x7f6ae806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.855+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 0x7f6ae806ee00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6af8034000 tx=0x7f6af802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:45.858+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6af4059dd0 con 0x7f6b0806fae0 2024-04-26T23:38:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.145+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f6b08160de0 con 0x7f6b0806fae0 2024-04-26T23:38:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.149+0000 7f6affe00640 1 -- 172.21.15.46:0/2392634024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f6af40596d0 con 0x7f6b0806fae0 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 msgr2=0x7f6ae806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 0x7f6ae806ee00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6af8034000 tx=0x7f6af802e000 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 msgr2=0x7f6b08071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e870 tx=0x7f6af400ed30 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 shutdown_connections 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ae806c940 0x7f6ae806ee00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b08163720 0x7f6b080aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b0806fae0 0x7f6b08071f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 --2- 172.21.15.46:0/2392634024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b0806f590 0x7f6b08165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 >> 172.21.15.46:0/2392634024 conn(0x7f6b080d0d10 msgr2=0x7f6b080b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 shutdown_connections 2024-04-26T23:38:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.152+0000 7f6b0f000640 1 -- 172.21.15.46:0/2392634024 wait complete. 2024-04-26T23:38:46.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T23:38:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.373+0000 7fd1e3200640 1 Processor -- start 2024-04-26T23:38:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.373+0000 7fd1e3200640 1 -- start start 2024-04-26T23:38:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.374+0000 7fd1e3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1dc06f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1dc071db0 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1dc06fbf0 con 0x7fd1dc06e5d0 2024-04-26T23:38:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc0cff10 con 0x7fd1dc0cba90 2024-04-26T23:38:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd1dc15b5b0 0x7fd1dc0cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33296/0 (socket says 172.21.15.46:33296) 2024-04-26T23:38:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e1e00640 1 -- 172.21.15.46:0/1377512670 learned_addr learned my addr 172.21.15.46:0/1377512670 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1dc06f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860011581 0 0) 0x7fd1dc071db0 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.375+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1c4003520 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1124170116 0 0) 0x7fd1c4003520 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 msgr2=0x7fd1dc06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1dc06f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 >> v1:172.21.15.142:6789/0 conn(0x7fd1dc06e5d0 legacy=0x7fd1dc072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc06fed0 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd1d0001670 con 0x7fd1dc15b5b0 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 >> v1:172.21.15.46:6789/0 conn(0x7fd1dc15b5b0 legacy=0x7fd1dc0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c40039e0 0x7fd1c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c4004800 0x7fd1c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1c40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400baa0 con 0x7fd1c40039e0 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400bc50 con 0x7fd1c4004800 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c400be50 con 0x7fd1dc0cba90 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.376+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c40039e0 0x7fd1c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c4004800 0x7fd1c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e2800640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1c40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1e00640 1 -- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 msgr2=0x7fd1c40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1c40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1e00640 1 -- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c4004800 msgr2=0x7fd1c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c4004800 0x7fd1c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1e00640 1 -- 172.21.15.46:0/1377512670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c4003520 con 0x7fd1c40039e0 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e2800640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc0cba90 0x7fd1c40051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.377+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c4004800 0x7fd1c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.378+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c40039e0 0x7fd1c4003e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c110 tx=0x7fd1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e902c2dfc489fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.378+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0007c40 con 0x7fd1c40039e0 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.378+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0005af0 con 0x7fd1c40039e0 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.378+0000 7fd1e0a00640 1 -- 172.21.15.46:0/1377512670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0007070 con 0x7fd1c40039e0 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 -- 172.21.15.46:0/1377512670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c40039e0 msgr2=0x7fd1c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1377512670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c40039e0 0x7fd1c4003e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c110 tx=0x7fd1c4005830 comp rx=0 tx=0).stop 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e2800640 1 -- 172.21.15.46:0/1377512670 reap_dead start 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 -- 172.21.15.46:0/1377512670 shutdown_connections 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 -- 172.21.15.46:0/1377512670 >> 172.21.15.46:0/1377512670 conn(0x7fd1dc0d0b20 msgr2=0x7fd1dc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 -- 172.21.15.46:0/1377512670 shutdown_connections 2024-04-26T23:38:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.379+0000 7fd1e3200640 1 -- 172.21.15.46:0/1377512670 wait complete. 2024-04-26T23:38:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.381+0000 7fd1e3200640 1 Processor -- start 2024-04-26T23:38:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.381+0000 7fd1e3200640 1 -- start start 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 0x7fd1dc150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc159530 0x7fd1dc15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc15b990 con 0x7fd1dc1590b0 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc072330 con 0x7fd1c40039e0 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc06f4c0 con 0x7fd1dc159530 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56996/0 (socket says 172.21.15.46:56996) 2024-04-26T23:38:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.382+0000 7fd1e1400640 1 -- 172.21.15.46:0/1072791978 learned_addr learned my addr 172.21.15.46:0/1072791978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 0x7fd1dc150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 -- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc159530 msgr2=0x7fd1dc15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e2800640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc159530 0x7fd1dc15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc159530 0x7fd1dc15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 -- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 msgr2=0x7fd1dc150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 0x7fd1dc150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 -- 172.21.15.46:0/1072791978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc0c5e50 con 0x7fd1dc1590b0 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 0x7fd1dc150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1e1400640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc010960 tx=0x7fd1cc010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.383+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00f040 con 0x7fd1dc1590b0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.384+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00ebc0 con 0x7fd1dc1590b0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.384+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc01c400 con 0x7fd1dc1590b0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.384+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc0c6130 con 0x7fd1dc1590b0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.384+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc0c66f0 con 0x7fd1dc1590b0 2024-04-26T23:38:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.385+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1dc070780 con 0x7fd1dc1590b0 2024-04-26T23:38:46.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.386+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1cc01c5a0 con 0x7fd1dc1590b0 2024-04-26T23:38:46.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.387+0000 7fd1d6000640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 0x7fd1c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.387+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fd1cc08bfb0 con 0x7fd1dc1590b0 2024-04-26T23:38:46.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.387+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 0x7fd1c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.389+0000 7fd1e1e00640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 0x7fd1c006ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0032000 tx=0x7fd1d0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.392+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1dc070780 con 0x7fd1dc1590b0 2024-04-26T23:38:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.668+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd1dc0693b0 con 0x7fd1dc1590b0 2024-04-26T23:38:46.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.673+0000 7fd1d6000640 1 -- 172.21.15.46:0/1072791978 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fd1cc018030 con 0x7fd1dc1590b0 2024-04-26T23:38:46.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 msgr2=0x7fd1c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 0x7fd1c006ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0032000 tx=0x7fd1d0009030 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 msgr2=0x7fd1dc159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc010960 tx=0x7fd1cc010e20 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 shutdown_connections 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1c006c920 0x7fd1c006ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1dc159530 0x7fd1dc15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1dc1590b0 0x7fd1dc159d10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 --2- 172.21.15.46:0/1072791978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c40039e0 0x7fd1dc150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.675+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 >> 172.21.15.46:0/1072791978 conn(0x7fd1dc0d0b20 msgr2=0x7fd1dc0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.676+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 shutdown_connections 2024-04-26T23:38:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.676+0000 7fd1e3200640 1 -- 172.21.15.46:0/1072791978 wait complete. 2024-04-26T23:38:46.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T23:38:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.906+0000 7fa071400640 1 Processor -- start 2024-04-26T23:38:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.907+0000 7fa071400640 1 -- start start 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.908+0000 7fa071400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa06c15c350 0x7fa06c15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.908+0000 7fa071400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 0x7fa06c0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.908+0000 7fa071400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06c0cee60 con 0x7fa06c0cb170 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.908+0000 7fa071400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa06c155ae0 con 0x7fa06c15c350 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.908+0000 7fa071400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c0cec30 con 0x7fa06c06fea0 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa06c15c350 0x7fa06c15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa06c15c350 0x7fa06c15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57008/0 (socket says 172.21.15.46:57008) 2024-04-26T23:38:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 -- 172.21.15.46:0/3757564910 learned_addr learned my addr 172.21.15.46:0/3757564910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa063e00640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 0x7fa06c0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 -- 172.21.15.46:0/3757564910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 msgr2=0x7fa06c0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 0x7fa06c0ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 -- 172.21.15.46:0/3757564910 >> v1:172.21.15.46:6789/0 conn(0x7fa06c0cb170 legacy=0x7fa06c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa06be00640 1 -- 172.21.15.46:0/3757564910 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c0cf140 con 0x7fa06c15c350 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.909+0000 7fa063e00640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 0x7fa06c0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.910+0000 7fa06be00640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c15c350 0x7fa06c15b5b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64e5dcb6ca99b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa06b400640 1 -- 172.21.15.46:0/3757564910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c037070 con 0x7fa06c15c350 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa06b400640 1 -- 172.21.15.46:0/3757564910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c032440 con 0x7fa06c15c350 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa06b400640 1 -- 172.21.15.46:0/3757564910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c032810 con 0x7fa06c15c350 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 -- 172.21.15.46:0/3757564910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c15c350 msgr2=0x7fa06c15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c15c350 0x7fa06c15b5b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02a250 comp rx=0 tx=0).stop 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 -- 172.21.15.46:0/3757564910 shutdown_connections 2024-04-26T23:38:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c06fea0 0x7fa06c0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 --2- 172.21.15.46:0/3757564910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c15c350 0x7fa06c15b5b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.911+0000 7fa071400640 1 -- 172.21.15.46:0/3757564910 >> 172.21.15.46:0/3757564910 conn(0x7fa06c0d0b20 msgr2=0x7fa06c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.912+0000 7fa071400640 1 -- 172.21.15.46:0/3757564910 shutdown_connections 2024-04-26T23:38:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.912+0000 7fa071400640 1 -- 172.21.15.46:0/3757564910 wait complete. 2024-04-26T23:38:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.913+0000 7fa071400640 1 Processor -- start 2024-04-26T23:38:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.914+0000 7fa071400640 1 -- start start 2024-04-26T23:38:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c0cb170 0x7fa06c1e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06c15c350 0x7fa06c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0cf700 con 0x7fa06c06fea0 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0cfa70 con 0x7fa06c15c350 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa071400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c06f130 con 0x7fa06c0cb170 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57020/0 (socket says 172.21.15.46:57020) 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 -- 172.21.15.46:0/2528967996 learned_addr learned my addr 172.21.15.46:0/2528967996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 -- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c0cb170 msgr2=0x7fa06c1e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c0cb170 0x7fa06c1e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 -- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06c15c350 msgr2=0x7fa06c06e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06c15c350 0x7fa06c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.915+0000 7fa06be00640 1 -- 172.21.15.46:0/2528967996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1546c0 con 0x7fa06c06fea0 2024-04-26T23:38:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa06be00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa05c02a290 tx=0x7fa05c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c037040 con 0x7fa06c06fea0 2024-04-26T23:38:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c042070 con 0x7fa06c06fea0 2024-04-26T23:38:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c1e8f50 con 0x7fa06c06fea0 2024-04-26T23:38:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c030bd0 con 0x7fa06c06fea0 2024-04-26T23:38:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.916+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1e9510 con 0x7fa06c06fea0 2024-04-26T23:38:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.917+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c0693b0 con 0x7fa06c06fea0 2024-04-26T23:38:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.918+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa05c047460 con 0x7fa06c06fea0 2024-04-26T23:38:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.919+0000 7fa068c00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 0x7fa04806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.919+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa05c0ac5f0 con 0x7fa06c06fea0 2024-04-26T23:38:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.920+0000 7fa063e00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 0x7fa04806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.921+0000 7fa063e00640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 0x7fa04806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:46.923+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa05c07abd0 con 0x7fa06c06fea0 2024-04-26T23:38:47.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.200+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa06c15e660 con 0x7fa06c06fea0 2024-04-26T23:38:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.205+0000 7fa068c00640 1 -- 172.21.15.46:0/2528967996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fa05c07a4d0 con 0x7fa06c06fea0 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.207+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 msgr2=0x7fa04806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.207+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 0x7fa04806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054008040 comp rx=0 tx=0).stop 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.207+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 msgr2=0x7fa06c154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.207+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa05c02a290 tx=0x7fa05c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 shutdown_connections 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa04806c8f0 0x7fa04806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06c15c350 0x7fa06c06e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa06c0cb170 0x7fa06c1e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 --2- 172.21.15.46:0/2528967996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa06c06fea0 0x7fa06c154180 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 >> 172.21.15.46:0/2528967996 conn(0x7fa06c0d0b20 msgr2=0x7fa06c154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 shutdown_connections 2024-04-26T23:38:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.208+0000 7fa071400640 1 -- 172.21.15.46:0/2528967996 wait complete. 2024-04-26T23:38:47.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.438+0000 7f0d4a000640 1 Processor -- start 2024-04-26T23:38:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.438+0000 7f0d4a000640 1 -- start start 2024-04-26T23:38:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.439+0000 7f0d4a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d4406e510 0x7f0d4406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.439+0000 7f0d4a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 0x7f0d44073710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.439+0000 7f0d4a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4406ee30 con 0x7f0d44160bd0 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.439+0000 7f0d4a000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0d44161940 con 0x7f0d4406e510 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.439+0000 7f0d4a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0d44071f80 con 0x7f0d440732c0 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d4406e510 0x7f0d4406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d42a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0d44160bd0 0x7f0d44161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33314/0 (socket says 172.21.15.46:33314) 2024-04-26T23:38:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d42a00640 1 -- 172.21.15.46:0/2438076045 learned_addr learned my addr 172.21.15.46:0/2438076045 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43e00640 1 --2- 172.21.15.46:0/2438076045 >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 0x7f0d44073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43400640 1 -- 172.21.15.46:0/2438076045 >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 msgr2=0x7f0d44073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43400640 1 --2- 172.21.15.46:0/2438076045 >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 0x7f0d44073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43400640 1 -- 172.21.15.46:0/2438076045 >> v1:172.21.15.46:6789/0 conn(0x7f0d44160bd0 legacy=0x7f0d44161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43400640 1 -- 172.21.15.46:0/2438076045 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d44072260 con 0x7f0d4406e510 2024-04-26T23:38:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.440+0000 7f0d43e00640 1 --2- 172.21.15.46:0/2438076045 >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 0x7f0d44073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:47.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.441+0000 7f0d43400640 1 --2- 172.21.15.46:0/2438076045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4406e510 0x7f0d4406e8f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009d90 tx=0x7f0d3c02a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc8b3db2610d352 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d42000640 1 -- 172.21.15.46:0/2438076045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c007740 con 0x7f0d4406e510 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d42000640 1 -- 172.21.15.46:0/2438076045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c0078e0 con 0x7f0d4406e510 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d42000640 1 -- 172.21.15.46:0/2438076045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c007cb0 con 0x7f0d4406e510 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 -- 172.21.15.46:0/2438076045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4406e510 msgr2=0x7f0d4406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 --2- 172.21.15.46:0/2438076045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4406e510 0x7f0d4406e8f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009d90 tx=0x7f0d3c02a6b0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 -- 172.21.15.46:0/2438076045 shutdown_connections 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 --2- 172.21.15.46:0/2438076045 >> v2:172.21.15.142:3300/0 conn(0x7f0d440732c0 0x7f0d44073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 --2- 172.21.15.46:0/2438076045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4406e510 0x7f0d4406e8f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.442+0000 7f0d4a000640 1 -- 172.21.15.46:0/2438076045 >> 172.21.15.46:0/2438076045 conn(0x7f0d4406c530 msgr2=0x7f0d4406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.443+0000 7f0d4a000640 1 -- 172.21.15.46:0/2438076045 shutdown_connections 2024-04-26T23:38:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.443+0000 7f0d4a000640 1 -- 172.21.15.46:0/2438076045 wait complete. 2024-04-26T23:38:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.444+0000 7f0d4a000640 1 Processor -- start 2024-04-26T23:38:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.444+0000 7f0d4a000640 1 -- start start 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 0x7f0d441588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d44160bd0 0x7f0d440ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44072d40 con 0x7f0d440732c0 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44072820 con 0x7f0d4406e510 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.445+0000 7f0d4a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d44072ec0 con 0x7f0d44160bd0 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57044/0 (socket says 172.21.15.46:57044) 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 -- 172.21.15.46:0/539880321 learned_addr learned my addr 172.21.15.46:0/539880321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d43400640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 0x7f0d441588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d43e00640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d44160bd0 0x7f0d440ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 -- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d44160bd0 msgr2=0x7f0d440ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d44160bd0 0x7f0d440ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 -- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 msgr2=0x7f0d441588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 0x7f0d441588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 -- 172.21.15.46:0/539880321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d44158e00 con 0x7f0d440732c0 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d43400640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 0x7f0d441588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.446+0000 7f0d42a00640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c400 tx=0x7f0d2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.447+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d440732c0 2024-04-26T23:38:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.447+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c005ce0 con 0x7f0d440732c0 2024-04-26T23:38:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.447+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4415a080 con 0x7f0d440732c0 2024-04-26T23:38:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.447+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0072f0 con 0x7f0d440732c0 2024-04-26T23:38:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.447+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4415a640 con 0x7f0d440732c0 2024-04-26T23:38:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.448+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d44069530 con 0x7f0d440732c0 2024-04-26T23:38:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.449+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d2c01a070 con 0x7f0d440732c0 2024-04-26T23:38:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.450+0000 7f0d3b400640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 0x7f0d2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.450+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f0d2c08adb0 con 0x7f0d440732c0 2024-04-26T23:38:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.451+0000 7f0d43400640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 0x7f0d2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.452+0000 7f0d43400640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 0x7f0d2006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c034000 tx=0x7f0d3c033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.455+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d2c0593a0 con 0x7f0d440732c0 2024-04-26T23:38:47.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.660+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d4413d3c0 con 0x7f0d440732c0 2024-04-26T23:38:47.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.661+0000 7f0d3b400640 1 -- 172.21.15.46:0/539880321 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13128 (secure 0 0 0) 0x7f0d2c058ca0 con 0x7f0d440732c0 2024-04-26T23:38:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:47.665 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:38:43.489267+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 msgr2=0x7f0d2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 0x7f0d2006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c034000 tx=0x7f0d3c033040 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 msgr2=0x7f0d440ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c400 tx=0x7f0d2c00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 shutdown_connections 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d2006c920 0x7f0d2006ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d44160bd0 0x7f0d440ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d440732c0 0x7f0d440ce5f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 --2- 172.21.15.46:0/539880321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d4406e510 0x7f0d441588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.664+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 >> 172.21.15.46:0/539880321 conn(0x7f0d4406c530 msgr2=0x7f0d440b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.665+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 shutdown_connections 2024-04-26T23:38:47.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.665+0000 7f0d4a000640 1 -- 172.21.15.46:0/539880321 wait complete. 2024-04-26T23:38:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:38:47.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:38:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.896+0000 7fd4cea00640 1 Processor -- start 2024-04-26T23:38:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.897+0000 7fd4cea00640 1 -- start start 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.898+0000 7fd4cea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4c80d38a0 0x7fd4c815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.898+0000 7fd4cea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd4c80cd900 0x7fd4c80cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.898+0000 7fd4cea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c80cf530 con 0x7fd4c80d0e00 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4cea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c80d3d00 con 0x7fd4c80d38a0 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4cea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c80d05d0 con 0x7fd4c80cd900 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4c80d38a0 0x7fd4c815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4c80d38a0 0x7fd4c815ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57052/0 (socket says 172.21.15.46:57052) 2024-04-26T23:38:47.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 -- 172.21.15.46:0/2980612622 learned_addr learned my addr 172.21.15.46:0/2980612622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 -- 172.21.15.46:0/2980612622 >> v2:172.21.15.142:3300/0 conn(0x7fd4c80cd900 msgr2=0x7fd4c80cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 --2- 172.21.15.46:0/2980612622 >> v2:172.21.15.142:3300/0 conn(0x7fd4c80cd900 0x7fd4c80cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 -- 172.21.15.46:0/2980612622 >> v1:172.21.15.142:6789/0 conn(0x7fd4c80d0e00 legacy=0x7fd4c80d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.899+0000 7fd4ccc00640 1 -- 172.21.15.46:0/2980612622 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c80cf810 con 0x7fd4c80d38a0 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.901+0000 7fd4ccc00640 1 --2- 172.21.15.46:0/2980612622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c815ddf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80c6770 tx=0x7fd4b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73a38ed41f8c64a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.901+0000 7fd4c7e00640 1 -- 172.21.15.46:0/2980612622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4037070 con 0x7fd4c80d38a0 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.901+0000 7fd4c7e00640 1 -- 172.21.15.46:0/2980612622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b4032440 con 0x7fd4c80d38a0 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.901+0000 7fd4c7e00640 1 -- 172.21.15.46:0/2980612622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4032810 con 0x7fd4c80d38a0 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 -- 172.21.15.46:0/2980612622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 msgr2=0x7fd4c815ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 --2- 172.21.15.46:0/2980612622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c815ddf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80c6770 tx=0x7fd4b402a230 comp rx=0 tx=0).stop 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 -- 172.21.15.46:0/2980612622 shutdown_connections 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 --2- 172.21.15.46:0/2980612622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c815ddf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 --2- 172.21.15.46:0/2980612622 >> v2:172.21.15.142:3300/0 conn(0x7fd4c80cd900 0x7fd4c80cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 -- 172.21.15.46:0/2980612622 >> 172.21.15.46:0/2980612622 conn(0x7fd4c80d2d80 msgr2=0x7fd4c80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 -- 172.21.15.46:0/2980612622 shutdown_connections 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.902+0000 7fd4cea00640 1 -- 172.21.15.46:0/2980612622 wait complete. 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.904+0000 7fd4cea00640 1 Processor -- start 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.904+0000 7fd4cea00640 1 -- start start 2024-04-26T23:38:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c80cd900 0x7fd4c80b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 0x7fd4c80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80d00b0 con 0x7fd4c80d38a0 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80cfdd0 con 0x7fd4c80cd900 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c815e330 con 0x7fd4c80d0e00 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57064/0 (socket says 172.21.15.46:57064) 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 -- 172.21.15.46:0/158231188 learned_addr learned my addr 172.21.15.46:0/158231188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4cd600640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c80cd900 0x7fd4c80b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ccc00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 0x7fd4c80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 -- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 msgr2=0x7fd4c80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 0x7fd4c80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 -- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c80cd900 msgr2=0x7fd4c80b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c80cd900 0x7fd4c80b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ce000640 1 -- 172.21.15.46:0/158231188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c814b6e0 con 0x7fd4c80d38a0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.905+0000 7fd4ccc00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 0x7fd4c80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4ce000640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b410 tx=0x7fd4c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0015020 con 0x7fd4c80d38a0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0011850 con 0x7fd4c80d38a0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011c40 con 0x7fd4c80d38a0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c80b5fd0 con 0x7fd4c80d38a0 2024-04-26T23:38:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.906+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c80b6590 con 0x7fd4c80d38a0 2024-04-26T23:38:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.907+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8069ac0 con 0x7fd4c80d38a0 2024-04-26T23:38:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.908+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4c0018020 con 0x7fd4c80d38a0 2024-04-26T23:38:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.909+0000 7fd4c5600640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 0x7fd4a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.909+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fd4c008b970 con 0x7fd4c80d38a0 2024-04-26T23:38:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.910+0000 7fd4cd600640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 0x7fd4a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.911+0000 7fd4cd600640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 0x7fd4a806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80c68e0 tx=0x7fd4b80094d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:47.914+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4c001f020 con 0x7fd4c80d38a0 2024-04-26T23:38:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.121+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4c80b8d00 con 0x7fd4c80d38a0 2024-04-26T23:38:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.122+0000 7fd4c5600640 1 -- 172.21.15.46:0/158231188 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fd4c814b6e0 con 0x7fd4c80d38a0 2024-04-26T23:38:48.125 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.124+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 msgr2=0x7fd4a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.124+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 0x7fd4a806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80c68e0 tx=0x7fd4b80094d0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 msgr2=0x7fd4c814b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b410 tx=0x7fd4c000b8d0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 shutdown_connections 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd4a806c8f0 0x7fd4a806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4c80d38a0 0x7fd4c814b1a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4c80d0e00 0x7fd4c80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 --2- 172.21.15.46:0/158231188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c80cd900 0x7fd4c80b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 >> 172.21.15.46:0/158231188 conn(0x7fd4c80d2d80 msgr2=0x7fd4c80d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 shutdown_connections 2024-04-26T23:38:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.125+0000 7fd4cea00640 1 -- 172.21.15.46:0/158231188 wait complete. 2024-04-26T23:38:48.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:38:48.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.356+0000 7f5363200640 1 Processor -- start 2024-04-26T23:38:48.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.356+0000 7f5363200640 1 -- start start 2024-04-26T23:38:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.358+0000 7f5363200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f5364162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.358+0000 7f5363200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53640af0c0 con 0x7f53641631d0 2024-04-26T23:38:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.358+0000 7f5363200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53640af3a0 con 0x7f5364159950 2024-04-26T23:38:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.358+0000 7f5363200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5364163620 con 0x7f53640b48a0 2024-04-26T23:38:48.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.359+0000 7f5362800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f53641631d0 0x7f536406e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33322/0 (socket says 172.21.15.46:33322) 2024-04-26T23:38:48.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.359+0000 7f5362800640 1 -- 172.21.15.46:0/1351559251 learned_addr learned my addr 172.21.15.46:0/1351559251 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.359+0000 7f535be00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f5364162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.359+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999407082 0 0) 0x7f53640af0c0 con 0x7f53641631d0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.359+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5348003520 con 0x7f53641631d0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 723684217 0 0) 0x7f5348003520 con 0x7f53641631d0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 msgr2=0x7f5364162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f5364162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 >> v1:172.21.15.142:6789/0 conn(0x7f5364159950 legacy=0x7f53640ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53640af680 con 0x7f53641631d0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f535c0019f0 con 0x7f53641631d0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 >> v1:172.21.15.46:6789/0 conn(0x7f53641631d0 legacy=0x7f536406e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f53480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5348003f60 0x7f5348004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5348004d40 0x7f53480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5364163620 con 0x7f5348003f60 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f534800baa0 con 0x7f5348004d40 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f534800bca0 con 0x7f53640b48a0 2024-04-26T23:38:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5348003f60 0x7f5348004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5361e00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f53480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f5362800640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5348004d40 0x7f53480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 -- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 msgr2=0x7f53480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f53480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 -- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5348004d40 msgr2=0x7f53480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5348004d40 0x7f53480051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.360+0000 7f535be00640 1 -- 172.21.15.46:0/1351559251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5348003520 con 0x7f5348003f60 2024-04-26T23:38:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.361+0000 7f5361e00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53640b48a0 0x7f53480039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.361+0000 7f535be00640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5348003f60 0x7f5348004800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f534c009ac0 tx=0x7f5348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e62c38ad29d0800 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c036070 con 0x7f5348003f60 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c031400 con 0x7f5348003f60 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5361400640 1 -- 172.21.15.46:0/1351559251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c009690 con 0x7f5348003f60 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5363200640 1 -- 172.21.15.46:0/1351559251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5348003f60 msgr2=0x7f5348004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5363200640 1 --2- 172.21.15.46:0/1351559251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5348003f60 0x7f5348004800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f534c009ac0 tx=0x7f5348005830 comp rx=0 tx=0).stop 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5362800640 1 -- 172.21.15.46:0/1351559251 reap_dead start 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5363200640 1 -- 172.21.15.46:0/1351559251 shutdown_connections 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5363200640 1 -- 172.21.15.46:0/1351559251 >> 172.21.15.46:0/1351559251 conn(0x7f53640b2820 msgr2=0x7f53640b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.362+0000 7f5363200640 1 -- 172.21.15.46:0/1351559251 shutdown_connections 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.363+0000 7f5363200640 1 -- 172.21.15.46:0/1351559251 wait complete. 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.364+0000 7f5363200640 1 Processor -- start 2024-04-26T23:38:48.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.364+0000 7f5363200640 1 -- start start 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5348003f60 0x7f53641eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53641440d0 0x7f5364144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53640b0160 con 0x7f5364152b40 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53640afc40 con 0x7f53641440d0 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.365+0000 7f5363200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5364162c90 con 0x7f5348003f60 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57082/0 (socket says 172.21.15.46:57082) 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 -- 172.21.15.46:0/56818638 learned_addr learned my addr 172.21.15.46:0/56818638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5361e00640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5348003f60 0x7f53641eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f535be00640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53641440d0 0x7f5364144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 -- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5348003f60 msgr2=0x7f53641eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5348003f60 0x7f53641eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 -- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53641440d0 msgr2=0x7f5364144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53641440d0 0x7f5364144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 -- 172.21.15.46:0/56818638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53641538e0 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.366+0000 7f5362800640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f53641635b0 tx=0x7f535c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.367+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00de90 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.367+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c00bb80 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.367+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c007070 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.367+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5364153bc0 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.367+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536415a560 con 0x7f5364152b40 2024-04-26T23:38:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.368+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53640693b0 con 0x7f5364152b40 2024-04-26T23:38:48.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.369+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f535c01b020 con 0x7f5364152b40 2024-04-26T23:38:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.370+0000 7f535a000640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 0x7f534006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.370+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f535c08bc60 con 0x7f5364152b40 2024-04-26T23:38:48.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.371+0000 7f5361e00640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 0x7f534006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.372+0000 7f5361e00640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 0x7f534006ef50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.375+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f535c05a1b0 con 0x7f5364152b40 2024-04-26T23:38:48.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.579+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f53640b35e0 con 0x7f5364152b40 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.580+0000 7f535a000640 1 -- 172.21.15.46:0/56818638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f535c059ab0 con 0x7f5364152b40 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:38:48.583 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:38:48.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 msgr2=0x7f534006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 0x7f534006ef50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354009040 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 msgr2=0x7f5364152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f53641635b0 tx=0x7f535c00b4a0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 shutdown_connections 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f534006ca90 0x7f534006ef50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5364152b40 0x7f5364152fa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53641440d0 0x7f5364144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 --2- 172.21.15.46:0/56818638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5348003f60 0x7f53641eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.583+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 >> 172.21.15.46:0/56818638 conn(0x7f53640b2820 msgr2=0x7f536406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.584+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 shutdown_connections 2024-04-26T23:38:48.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.584+0000 7f5363200640 1 -- 172.21.15.46:0/56818638 wait complete. 2024-04-26T23:38:48.607 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:38:48.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:38:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.806+0000 7f05c5e00640 1 Processor -- start 2024-04-26T23:38:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.806+0000 7f05c5e00640 1 -- start start 2024-04-26T23:38:48.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.807+0000 7f05c5e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05c01627f0 0x7f05c00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.807+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d2080 0x7f05c00d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.807+0000 7f05c5e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c00d01a0 con 0x7f05c00d3af0 2024-04-26T23:38:48.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.807+0000 7f05c5e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f05c00d0480 con 0x7f05c01627f0 2024-04-26T23:38:48.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.807+0000 7f05c5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05c00d0760 con 0x7f05c00d2080 2024-04-26T23:38:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.808+0000 7f05c4a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05c01627f0 0x7f05c00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.808+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d2080 0x7f05c00d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.808+0000 7f05c4a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05c01627f0 0x7f05c00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:56602/0 (socket says 172.21.15.46:56602) 2024-04-26T23:38:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.808+0000 7f05c4a00640 1 -- 172.21.15.46:0/3437795205 learned_addr learned my addr 172.21.15.46:0/3437795205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.808+0000 7f05c5400640 1 --1- 172.21.15.46:0/3437795205 >> v1:172.21.15.142:6789/0 conn(0x7f05c00d3af0 0x7f05c0162f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:39938/0 (socket says 172.21.15.46:39938) 2024-04-26T23:38:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.809+0000 7f05c4a00640 1 -- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d2080 msgr2=0x7f05c00d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.809+0000 7f05c4a00640 1 --2- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d2080 0x7f05c00d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.809+0000 7f05c4a00640 1 -- 172.21.15.46:0/3437795205 >> v1:172.21.15.142:6789/0 conn(0x7f05c00d3af0 legacy=0x7f05c0162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.809+0000 7f05c4a00640 1 -- 172.21.15.46:0/3437795205 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c00d0a40 con 0x7f05c01627f0 2024-04-26T23:38:48.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.810+0000 7f05c4a00640 1 --2- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cfbc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009e50 tx=0x7f05b002a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf965191ed6c970 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.810+0000 7f05bf400640 1 -- 172.21.15.46:0/3437795205 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0036070 con 0x7f05c01627f0 2024-04-26T23:38:48.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05bf400640 1 -- 172.21.15.46:0/3437795205 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0031600 con 0x7f05c01627f0 2024-04-26T23:38:48.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05bf400640 1 -- 172.21.15.46:0/3437795205 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00319d0 con 0x7f05c01627f0 2024-04-26T23:38:48.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 -- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 msgr2=0x7f05c00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 --2- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cfbc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009e50 tx=0x7f05b002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 -- 172.21.15.46:0/3437795205 shutdown_connections 2024-04-26T23:38:48.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 --2- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d2080 0x7f05c00d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 --2- 172.21.15.46:0/3437795205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cfbc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 -- 172.21.15.46:0/3437795205 >> 172.21.15.46:0/3437795205 conn(0x7f05c009e380 msgr2=0x7f05c00d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:48.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 -- 172.21.15.46:0/3437795205 shutdown_connections 2024-04-26T23:38:48.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.811+0000 7f05c5e00640 1 -- 172.21.15.46:0/3437795205 wait complete. 2024-04-26T23:38:48.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.813+0000 7f05c5e00640 1 Processor -- start 2024-04-26T23:38:48.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.813+0000 7f05c5e00640 1 -- start start 2024-04-26T23:38:48.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d3af0 0x7f05c00ba350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0162bd0 con 0x7f05c00d2080 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05c00d24f0 con 0x7f05c01627f0 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.814+0000 7f05c5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05c00b5560 con 0x7f05c00d3af0 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57088/0 (socket says 172.21.15.46:57088) 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 -- 172.21.15.46:0/483888300 learned_addr learned my addr 172.21.15.46:0/483888300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05bfe00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d3af0 0x7f05c00ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c5400640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 -- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d3af0 msgr2=0x7f05c00ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d3af0 0x7f05c00ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 -- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 msgr2=0x7f05c00cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.815+0000 7f05c4a00640 1 -- 172.21.15.46:0/483888300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c00ba890 con 0x7f05c00d2080 2024-04-26T23:38:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05c4a00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034040 tx=0x7f05b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0041020 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0037040 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c00b92b0 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0036050 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.816+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c00b9870 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.817+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c00693b0 con 0x7f05c00d2080 2024-04-26T23:38:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.818+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05b0007c50 con 0x7f05c00d2080 2024-04-26T23:38:48.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.819+0000 7f05bcc00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:48.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.819+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f05b00ac050 con 0x7f05c00d2080 2024-04-26T23:38:48.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.820+0000 7f05bfe00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:48.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.821+0000 7f05bfe00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 0x7f059c06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05c00b8e00 tx=0x7f05a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:48.824+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05b004b060 con 0x7f05c00d2080 2024-04-26T23:38:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.029+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f05c00d56e0 con 0x7f05c00d2080 2024-04-26T23:38:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.637+0000 7f05bcc00640 1 -- 172.21.15.46:0/483888300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f05b007a650 con 0x7f05c00d2080 2024-04-26T23:38:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:38:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.642+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 msgr2=0x7f059c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.642+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 0x7f059c06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05c00b8e00 tx=0x7f05a8008040 comp rx=0 tx=0).stop 2024-04-26T23:38:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.642+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 msgr2=0x7f05c00b88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.642+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034040 tx=0x7f05b0004a40 comp rx=0 tx=0).stop 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 shutdown_connections 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c01627f0 0x7f05c00cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05c00d3af0 0x7f05c00ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 --2- 172.21.15.46:0/483888300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05c00d2080 0x7f05c00b88c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.643+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 >> 172.21.15.46:0/483888300 conn(0x7f05c009e380 msgr2=0x7f05c00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:49.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.644+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 shutdown_connections 2024-04-26T23:38:49.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.645+0000 7f05c5e00640 1 -- 172.21.15.46:0/483888300 wait complete. 2024-04-26T23:38:49.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:38:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.883+0000 7fea9e600640 1 Processor -- start 2024-04-26T23:38:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.884+0000 7fea9e600640 1 -- start start 2024-04-26T23:38:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.885+0000 7fea9e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea9815d730 0x7fea980d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.885+0000 7fea9e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea9806eb90 con 0x7fea980b53b0 2024-04-26T23:38:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.885+0000 7fea9e600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea980b4310 con 0x7fea9815e220 2024-04-26T23:38:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.885+0000 7fea9e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fea980b45f0 con 0x7fea9815d730 2024-04-26T23:38:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.886+0000 7fea97400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea9815d730 0x7fea980d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.886+0000 7fea96a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fea980b53b0 0x7fea9815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33324/0 (socket says 172.21.15.46:33324) 2024-04-26T23:38:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.886+0000 7fea96a00640 1 -- 172.21.15.46:0/3853319403 learned_addr learned my addr 172.21.15.46:0/3853319403 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.886+0000 7fea97400640 1 -- 172.21.15.46:0/3853319403 >> v1:172.21.15.142:6789/0 conn(0x7fea9815e220 legacy=0x7fea9806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.886+0000 7fea97400640 1 -- 172.21.15.46:0/3853319403 >> v1:172.21.15.46:6789/0 conn(0x7fea980b53b0 legacy=0x7fea9815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.887+0000 7fea96000640 1 -- 172.21.15.46:0/3853319403 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458494951 0 0) 0x7fea9806eb90 con 0x7fea980b53b0 2024-04-26T23:38:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.887+0000 7fea97400640 1 -- 172.21.15.46:0/3853319403 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea980b48d0 con 0x7fea9815d730 2024-04-26T23:38:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.888+0000 7fea97400640 1 --2- 172.21.15.46:0/3853319403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815d730 0x7fea980d00d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea800097a0 tx=0x7fea8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87e97399bfced8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.888+0000 7fea96000640 1 -- 172.21.15.46:0/3853319403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80037070 con 0x7fea9815d730 2024-04-26T23:38:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.888+0000 7fea96000640 1 -- 172.21.15.46:0/3853319403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea80032440 con 0x7fea9815d730 2024-04-26T23:38:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.888+0000 7fea96000640 1 -- 172.21.15.46:0/3853319403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80032810 con 0x7fea9815d730 2024-04-26T23:38:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 -- 172.21.15.46:0/3853319403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815d730 msgr2=0x7fea980d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 --2- 172.21.15.46:0/3853319403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815d730 0x7fea980d00d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea800097a0 tx=0x7fea8002a230 comp rx=0 tx=0).stop 2024-04-26T23:38:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 -- 172.21.15.46:0/3853319403 shutdown_connections 2024-04-26T23:38:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 --2- 172.21.15.46:0/3853319403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815d730 0x7fea980d00d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 -- 172.21.15.46:0/3853319403 >> 172.21.15.46:0/3853319403 conn(0x7fea9806c980 msgr2=0x7fea980b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 -- 172.21.15.46:0/3853319403 shutdown_connections 2024-04-26T23:38:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.889+0000 7fea9e600640 1 -- 172.21.15.46:0/3853319403 wait complete. 2024-04-26T23:38:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.891+0000 7fea9e600640 1 Processor -- start 2024-04-26T23:38:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.891+0000 7fea9e600640 1 -- start start 2024-04-26T23:38:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 0x7fea981f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea9815d730 0x7fea98143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea980b3ec0 con 0x7fea9815e220 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea980b4e90 con 0x7fea9815d730 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.892+0000 7fea9e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea980b4040 con 0x7fea980b53b0 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57104/0 (socket says 172.21.15.46:57104) 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 -- 172.21.15.46:0/3773927892 learned_addr learned my addr 172.21.15.46:0/3773927892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea96a00640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea9815d730 0x7fea98143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97400640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 0x7fea981f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 -- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 msgr2=0x7fea981f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 0x7fea981f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 -- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea9815d730 msgr2=0x7fea98143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea9815d730 0x7fea98143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97e00640 1 -- 172.21.15.46:0/3773927892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea981449e0 con 0x7fea9815e220 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.893+0000 7fea97400640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 0x7fea981f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea97e00640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fea90001eb0 tx=0x7fea9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90017070 con 0x7fea9815e220 2024-04-26T23:38:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea900052d0 con 0x7fea9815e220 2024-04-26T23:38:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90003810 con 0x7fea9815e220 2024-04-26T23:38:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea98144cc0 con 0x7fea9815e220 2024-04-26T23:38:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.894+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea98145280 con 0x7fea9815e220 2024-04-26T23:38:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.895+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea98069530 con 0x7fea9815e220 2024-04-26T23:38:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.897+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea9001a070 con 0x7fea9815e220 2024-04-26T23:38:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.898+0000 7fea8f400640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.898+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7fea9008b390 con 0x7fea9815e220 2024-04-26T23:38:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.898+0000 7fea97400640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.900+0000 7fea97400640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fea80034000 tx=0x7fea8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:49.902+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea90021020 con 0x7fea9815e220 2024-04-26T23:38:50.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.108+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fea980cf560 con 0x7fea9815e220 2024-04-26T23:38:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.654+0000 7fea8f400640 1 -- 172.21.15.46:0/3773927892 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7fea900597f0 con 0x7fea9815e220 2024-04-26T23:38:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:38:50.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.657+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 msgr2=0x7fea7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.657+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fea80034000 tx=0x7fea8002e000 comp rx=0 tx=0).stop 2024-04-26T23:38:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.657+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 msgr2=0x7fea98145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.657+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fea90001eb0 tx=0x7fea9000c890 comp rx=0 tx=0).stop 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 shutdown_connections 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9815e220 0x7fea98145d10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea9815d730 0x7fea98143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 --2- 172.21.15.46:0/3773927892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea980b53b0 0x7fea981f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.659+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 >> 172.21.15.46:0/3773927892 conn(0x7fea9806c980 msgr2=0x7fea9806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.660+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 shutdown_connections 2024-04-26T23:38:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.660+0000 7fea9e600640 1 -- 172.21.15.46:0/3773927892 wait complete. 2024-04-26T23:38:50.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:38:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.885+0000 7f6271c00640 1 Processor -- start 2024-04-26T23:38:50.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.886+0000 7f6271c00640 1 -- start start 2024-04-26T23:38:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.887+0000 7f6271c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f626c160130 0x7f626c160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.887+0000 7f6271c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.887+0000 7f6271c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626c0cbe60 con 0x7f626c15cc40 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.887+0000 7f6271c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f626c0cca70 con 0x7f626c160130 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.887+0000 7f6271c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f626c0cc140 con 0x7f626c0cdc60 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.888+0000 7f6271200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.888+0000 7f6271200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:51396/0 (socket says 172.21.15.46:51396) 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.888+0000 7f6271200640 1 -- 172.21.15.46:0/2044694365 learned_addr learned my addr 172.21.15.46:0/2044694365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.888+0000 7f626be00640 1 --2- 172.21.15.46:0/2044694365 >> v2:172.21.15.142:3300/0 conn(0x7f626c160130 0x7f626c160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.889+0000 7f626be00640 1 -- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 msgr2=0x7f626c072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.889+0000 7f626be00640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.889+0000 7f626be00640 1 -- 172.21.15.46:0/2044694365 >> v1:172.21.15.142:6789/0 conn(0x7f626c15cc40 legacy=0x7f626c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.889+0000 7f626be00640 1 -- 172.21.15.46:0/2044694365 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c0cc420 con 0x7f626c160130 2024-04-26T23:38:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.889+0000 7f6271200640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:38:50.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.890+0000 7f626be00640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c160130 0x7f626c160510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f625c0097a0 tx=0x7f625c02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e0fd3daca1fd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:50.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.891+0000 7f626b400640 1 -- 172.21.15.46:0/2044694365 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c037070 con 0x7f626c160130 2024-04-26T23:38:50.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.891+0000 7f626b400640 1 -- 172.21.15.46:0/2044694365 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c02ae40 con 0x7f626c160130 2024-04-26T23:38:50.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.891+0000 7f626b400640 1 -- 172.21.15.46:0/2044694365 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c032660 con 0x7f626c160130 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.891+0000 7f6271c00640 1 -- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c160130 msgr2=0x7f626c160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.891+0000 7f6271c00640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c160130 0x7f626c160510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f625c0097a0 tx=0x7f625c02a0f0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 -- 172.21.15.46:0/2044694365 shutdown_connections 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c0cdc60 0x7f626c072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 --2- 172.21.15.46:0/2044694365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c160130 0x7f626c160510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 -- 172.21.15.46:0/2044694365 >> 172.21.15.46:0/2044694365 conn(0x7f626c0b27b0 msgr2=0x7f626c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 -- 172.21.15.46:0/2044694365 shutdown_connections 2024-04-26T23:38:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.892+0000 7f6271c00640 1 -- 172.21.15.46:0/2044694365 wait complete. 2024-04-26T23:38:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.894+0000 7f6271c00640 1 Processor -- start 2024-04-26T23:38:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.894+0000 7f6271c00640 1 -- start start 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 0x7f626c0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c160130 0x7f626c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f626c15d220 con 0x7f626c15cc40 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f626c15d0a0 con 0x7f626c0cdc60 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6271c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f626c072760 con 0x7f626c160130 2024-04-26T23:38:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6263400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6263400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57110/0 (socket says 172.21.15.46:57110) 2024-04-26T23:38:50.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.895+0000 7f6263400640 1 -- 172.21.15.46:0/2554169602 learned_addr learned my addr 172.21.15.46:0/2554169602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6271200640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c160130 0x7f626c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f626be00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 0x7f626c0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 -- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c160130 msgr2=0x7f626c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c160130 0x7f626c15b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 -- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 msgr2=0x7f626c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 0x7f626c0c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 -- 172.21.15.46:0/2554169602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c14ce00 con 0x7f626c15cc40 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f626be00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 0x7f626c0c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.896+0000 7f6263400640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f625400c470 tx=0x7f6254001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.897+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540052d0 con 0x7f626c15cc40 2024-04-26T23:38:50.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.897+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6254005470 con 0x7f626c15cc40 2024-04-26T23:38:50.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.897+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254007260 con 0x7f626c15cc40 2024-04-26T23:38:50.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.897+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c14d0e0 con 0x7f626c15cc40 2024-04-26T23:38:50.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.897+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c14d6a0 con 0x7f626c15cc40 2024-04-26T23:38:50.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.898+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c0693b0 con 0x7f626c15cc40 2024-04-26T23:38:50.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.900+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6254007400 con 0x7f626c15cc40 2024-04-26T23:38:50.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.901+0000 7f6268c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:50.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.901+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f625408b1f0 con 0x7f626c15cc40 2024-04-26T23:38:50.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.902+0000 7f626be00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:50.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.903+0000 7f626be00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 0x7f624806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f625c034000 tx=0x7f625c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:50.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:50.907+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62540594e0 con 0x7f626c15cc40 2024-04-26T23:38:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.148+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f626c0d0740 con 0x7f626c15cc40 2024-04-26T23:38:51.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.658+0000 7f6268c00640 1 -- 172.21.15.46:0/2554169602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6254058de0 con 0x7f626c15cc40 2024-04-26T23:38:51.661 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T23:38:51.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.663+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 msgr2=0x7f624806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.663+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 0x7f624806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f625c034000 tx=0x7f625c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.663+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 msgr2=0x7f626c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.663+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f625400c470 tx=0x7f6254001e70 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 shutdown_connections 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626c160130 0x7f626c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f626c15cc40 0x7f626c15ae00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 --2- 172.21.15.46:0/2554169602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626c0cdc60 0x7f626c0c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 >> 172.21.15.46:0/2554169602 conn(0x7f626c0b27b0 msgr2=0x7f626c06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 shutdown_connections 2024-04-26T23:38:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.664+0000 7f6271c00640 1 -- 172.21.15.46:0/2554169602 wait complete. 2024-04-26T23:38:51.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.881+0000 7f1529600640 1 Processor -- start 2024-04-26T23:38:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.882+0000 7f1529600640 1 -- start start 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1529600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 0x7f152406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1529600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152406ee30 con 0x7f1524160bd0 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1529600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1524161940 con 0x7f15240732c0 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1529600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1524071f80 con 0x7f152406e510 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1522000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1524160bd0 0x7f1524161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53364/0 (socket says 172.21.15.46:53364) 2024-04-26T23:38:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.883+0000 7f1522000640 1 -- 172.21.15.46:0/3287364337 learned_addr learned my addr 172.21.15.46:0/3287364337 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1522a00640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 0x7f152406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330967727 0 0) 0x7f152406ee30 con 0x7f1524160bd0 2024-04-26T23:38:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1508003520 con 0x7f1524160bd0 2024-04-26T23:38:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 434957607 0 0) 0x7f1508003520 con 0x7f1524160bd0 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 msgr2=0x7f152406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 0x7f152406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 >> v1:172.21.15.142:6789/0 conn(0x7f15240732c0 legacy=0x7f1524073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524072260 con 0x7f1524160bd0 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f150c0019f0 con 0x7f1524160bd0 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 >> v1:172.21.15.46:6789/0 conn(0x7f1524160bd0 legacy=0x7f1524161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.884+0000 7f1521600640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15080039e0 0x7f1508003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1521600640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 0x7f1508004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1521600640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1508004d40 0x7f15080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1524071f80 con 0x7f15080039e0 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f150800baa0 con 0x7f1508004d40 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f150800bca0 con 0x7f152406e510 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15080039e0 0x7f1508003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 -- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 msgr2=0x7f1508004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f152406e510 0x7f1508004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 -- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1508004d40 msgr2=0x7f15080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1508004d40 0x7f15080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.885+0000 7f1522a00640 1 -- 172.21.15.46:0/3287364337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1508003520 con 0x7f15080039e0 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.886+0000 7f1522a00640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15080039e0 0x7f1508003e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1514009a90 tx=0x7f1508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5e2de002b4ef4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.886+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514036070 con 0x7f15080039e0 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.886+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514031400 con 0x7f15080039e0 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.886+0000 7f1521600640 1 -- 172.21.15.46:0/3287364337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514009690 con 0x7f15080039e0 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 -- 172.21.15.46:0/3287364337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15080039e0 msgr2=0x7f1508003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 --2- 172.21.15.46:0/3287364337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15080039e0 0x7f1508003e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1514009a90 tx=0x7f1508005830 comp rx=0 tx=0).stop 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1523400640 1 -- 172.21.15.46:0/3287364337 reap_dead start 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 -- 172.21.15.46:0/3287364337 shutdown_connections 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 -- 172.21.15.46:0/3287364337 >> 172.21.15.46:0/3287364337 conn(0x7f152406c530 msgr2=0x7f152406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 -- 172.21.15.46:0/3287364337 shutdown_connections 2024-04-26T23:38:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.887+0000 7f1529600640 1 -- 172.21.15.46:0/3287364337 wait complete. 2024-04-26T23:38:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.889+0000 7f1529600640 1 Processor -- start 2024-04-26T23:38:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.889+0000 7f1529600640 1 -- start start 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15080039e0 0x7f15240afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1524144610 0x7f1524144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072d40 con 0x7f15240b5a60 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072820 con 0x7f1524144610 2024-04-26T23:38:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1529600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1524072ec0 con 0x7f15080039e0 2024-04-26T23:38:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1522000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1522000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42402/0 (socket says 172.21.15.46:42402) 2024-04-26T23:38:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.890+0000 7f1522000640 1 -- 172.21.15.46:0/1497369400 learned_addr learned my addr 172.21.15.46:0/1497369400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522a00640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15080039e0 0x7f15240afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 -- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15080039e0 msgr2=0x7f15240afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15080039e0 0x7f15240afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 -- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1524144610 msgr2=0x7f1524144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1524144610 0x7f1524144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 -- 172.21.15.46:0/1497369400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152415c410 con 0x7f15240b5a60 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1522000640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f150c00caa0 tx=0x7f150c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c018070 con 0x7f15240b5a60 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c00a790 con 0x7f15240b5a60 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00ab80 con 0x7f15240b5a60 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.891+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15240b5240 con 0x7f15240b5a60 2024-04-26T23:38:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.892+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15240af2f0 con 0x7f15240b5a60 2024-04-26T23:38:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.893+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1524069530 con 0x7f15240b5a60 2024-04-26T23:38:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.894+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f150c0070d0 con 0x7f15240b5a60 2024-04-26T23:38:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.894+0000 7f151aa00640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.895+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f150c08b5b0 con 0x7f15240b5a60 2024-04-26T23:38:51.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.895+0000 7f1522a00640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.896+0000 7f1522a00640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 0x7f150006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1514033000 tx=0x7f1514008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:51.899+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f150c059720 con 0x7f15240b5a60 2024-04-26T23:38:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.101+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f152406fb00 con 0x7f15240b5a60 2024-04-26T23:38:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.104+0000 7f151aa00640 1 -- 172.21.15.46:0/1497369400 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16919 (secure 0 0 0) 0x7f150c059020 con 0x7f15240b5a60 2024-04-26T23:38:52.107 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:52.107 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:38:51.149853+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.106+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 msgr2=0x7f150006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.106+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 0x7f150006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1514033000 tx=0x7f1514008030 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.106+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 msgr2=0x7f152415bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.106+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f150c00caa0 tx=0x7f150c00cf60 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 shutdown_connections 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1524144610 0x7f1524144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15240b5a60 0x7f152415bed0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 --2- 172.21.15.46:0/1497369400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15080039e0 0x7f15240afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 >> 172.21.15.46:0/1497369400 conn(0x7f152406c530 msgr2=0x7f152406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 shutdown_connections 2024-04-26T23:38:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.107+0000 7f1529600640 1 -- 172.21.15.46:0/1497369400 wait complete. 2024-04-26T23:38:52.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.328+0000 7f189e000640 1 Processor -- start 2024-04-26T23:38:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.329+0000 7f189e000640 1 -- start start 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f189e000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1898162980 0x7f18980d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f189e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f189815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f189e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189815fd30 con 0x7f18980d31f0 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f189e000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1898162d60 con 0x7f1898162980 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f189e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18980d1050 con 0x7f18980d2200 2024-04-26T23:38:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f1896a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f18980d31f0 0x7f18980d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53372/0 (socket says 172.21.15.46:53372) 2024-04-26T23:38:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.330+0000 7f1896a00640 1 -- 172.21.15.46:0/142890243 learned_addr learned my addr 172.21.15.46:0/142890243 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1897e00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f189815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664048498 0 0) 0x7f189815fd30 con 0x7f18980d31f0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1897400640 1 --2- 172.21.15.46:0/142890243 >> v2:172.21.15.142:3300/0 conn(0x7f1898162980 0x7f18980d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f187c003520 con 0x7f18980d31f0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1085349043 0 0) 0x7f187c003520 con 0x7f18980d31f0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 msgr2=0x7f189815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f189815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 >> v2:172.21.15.142:3300/0 conn(0x7f1898162980 msgr2=0x7f18980d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 --2- 172.21.15.46:0/142890243 >> v2:172.21.15.142:3300/0 conn(0x7f1898162980 0x7f18980d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.331+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1898160010 con 0x7f18980d31f0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1884001670 con 0x7f18980d31f0 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 >> v1:172.21.15.46:6789/0 conn(0x7f18980d31f0 legacy=0x7f18980d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f187c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f187c003f30 0x7f187c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c004d40 0x7f187c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1898162d60 con 0x7f187c003f30 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18980d1050 con 0x7f187c004d40 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f187c00b9c0 con 0x7f18980d2200 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f187c003f30 0x7f187c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1897400640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f187c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1897e00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c004d40 0x7f187c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 -- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 msgr2=0x7f187c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980d2200 0x7f187c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 -- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c004d40 msgr2=0x7f187c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c004d40 0x7f187c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.332+0000 7f1896a00640 1 -- 172.21.15.46:0/142890243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c003520 con 0x7f187c003f30 2024-04-26T23:38:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.333+0000 7f1897e00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c004d40 0x7f187c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f1896a00640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f187c003f30 0x7f187c004800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f188400c210 tx=0x7f187c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e044a00a71bf1911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884007d50 con 0x7f187c003f30 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188402e070 con 0x7f187c003f30 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f1896000640 1 -- 172.21.15.46:0/142890243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884005d70 con 0x7f187c003f30 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f189e000640 1 -- 172.21.15.46:0/142890243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f187c003f30 msgr2=0x7f187c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f189e000640 1 --2- 172.21.15.46:0/142890243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f187c003f30 0x7f187c004800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f188400c210 tx=0x7f187c005830 comp rx=0 tx=0).stop 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f1897e00640 1 -- 172.21.15.46:0/142890243 reap_dead start 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f189e000640 1 -- 172.21.15.46:0/142890243 shutdown_connections 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.334+0000 7f189e000640 1 -- 172.21.15.46:0/142890243 >> 172.21.15.46:0/142890243 conn(0x7f189809e420 msgr2=0x7f18980d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.335+0000 7f189e000640 1 -- 172.21.15.46:0/142890243 shutdown_connections 2024-04-26T23:38:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.335+0000 7f189e000640 1 -- 172.21.15.46:0/142890243 wait complete. 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.336+0000 7f189e000640 1 Processor -- start 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.336+0000 7f189e000640 1 -- start start 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 0x7f1898151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980b5570 0x7f18980b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18981608b0 con 0x7f18980b3ec0 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18981605d0 con 0x7f187c003f30 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.337+0000 7f189e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1898160d50 con 0x7f18980b5570 2024-04-26T23:38:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42422/0 (socket says 172.21.15.46:42422) 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 -- 172.21.15.46:0/1169279878 learned_addr learned my addr 172.21.15.46:0/1169279878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1896a00640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980b5570 0x7f18980b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897400640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 0x7f1898151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 -- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980b5570 msgr2=0x7f18980b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980b5570 0x7f18980b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 -- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 msgr2=0x7f1898151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 0x7f1898151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897e00640 1 -- 172.21.15.46:0/1169279878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18980b44d0 con 0x7f18980b3ec0 2024-04-26T23:38:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.338+0000 7f1897400640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 0x7f1898151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f1897e00640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f189808e3a0 tx=0x7f189000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189001c070 con 0x7f18980b3ec0 2024-04-26T23:38:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890017400 con 0x7f18980b3ec0 2024-04-26T23:38:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18900177f0 con 0x7f18980b3ec0 2024-04-26T23:38:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18980b47b0 con 0x7f18980b3ec0 2024-04-26T23:38:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.339+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18980b4d70 con 0x7f18980b3ec0 2024-04-26T23:38:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.340+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18980d5060 con 0x7f18980b3ec0 2024-04-26T23:38:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.341+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f189001f070 con 0x7f18980b3ec0 2024-04-26T23:38:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.342+0000 7f188f400640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 0x7f187806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.342+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f189008bee0 con 0x7f18980b3ec0 2024-04-26T23:38:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.343+0000 7f1897400640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 0x7f187806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.344+0000 7f1897400640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 0x7f187806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18980d5680 tx=0x7f1888008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.347+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1890012070 con 0x7f18980b3ec0 2024-04-26T23:38:52.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.548+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1898069530 con 0x7f18980b3ec0 2024-04-26T23:38:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.550+0000 7f188f400640 1 -- 172.21.15.46:0/1169279878 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16919 (secure 0 0 0) 0x7f189005a0e0 con 0x7f18980b3ec0 2024-04-26T23:38:52.553 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:52.553 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:38:51.149853+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.552+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 msgr2=0x7f187806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.552+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 0x7f187806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18980d5680 tx=0x7f1888008040 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.552+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 msgr2=0x7f18980ba080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.552+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f189808e3a0 tx=0x7f189000f4a0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 shutdown_connections 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f187806c8f0 0x7f187806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18980b3ec0 0x7f18980ba080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18980b5570 0x7f18980b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 --2- 172.21.15.46:0/1169279878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c003f30 0x7f1898151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 >> 172.21.15.46:0/1169279878 conn(0x7f189809e420 msgr2=0x7f18980d1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 shutdown_connections 2024-04-26T23:38:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.553+0000 7f189e000640 1 -- 172.21.15.46:0/1169279878 wait complete. 2024-04-26T23:38:52.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:38:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.784+0000 7f82d9400640 1 Processor -- start 2024-04-26T23:38:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.785+0000 7f82d9400640 1 -- start start 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82d40d2010 0x7f82d40b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d9400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40d07d0 con 0x7f82d40b5fb0 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d9400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40d0ab0 con 0x7f82d41626c0 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d9400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f82d40d0d90 con 0x7f82d40d2010 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f82d40b5fb0 0x7f82d40d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53380/0 (socket says 172.21.15.46:53380) 2024-04-26T23:38:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.786+0000 7f82d3400640 1 -- 172.21.15.46:0/1602424387 learned_addr learned my addr 172.21.15.46:0/1602424387 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231812770 0 0) 0x7f82d40d07d0 con 0x7f82d40b5fb0 2024-04-26T23:38:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82b8003520 con 0x7f82d40b5fb0 2024-04-26T23:38:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d8a00640 1 --2- 172.21.15.46:0/1602424387 >> v2:172.21.15.142:3300/0 conn(0x7f82d40d2010 0x7f82d40b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1911533916 0 0) 0x7f82b8003520 con 0x7f82d40b5fb0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 >> v2:172.21.15.142:3300/0 conn(0x7f82d40d2010 msgr2=0x7f82d40b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 --2- 172.21.15.46:0/1602424387 >> v2:172.21.15.142:3300/0 conn(0x7f82d40d2010 0x7f82d40b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 >> v1:172.21.15.142:6789/0 conn(0x7f82d41626c0 legacy=0x7f82d40d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40d2c80 con 0x7f82d40b5fb0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f82bc0017f0 con 0x7f82d40b5fb0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 >> v1:172.21.15.46:6789/0 conn(0x7f82d40b5fb0 legacy=0x7f82d40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82b80039b0 0x7f82b8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b80051c0 0x7f82b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40d0d90 con 0x7f82b8004800 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82b800c0f0 con 0x7f82b80051c0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.787+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82b800c2f0 con 0x7f82b80039b0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3e00640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82b80039b0 0x7f82b8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 -- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82b80039b0 msgr2=0x7f82b8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82b80039b0 0x7f82b8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 -- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b80051c0 msgr2=0x7f82b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b80051c0 0x7f82b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3400640 1 -- 172.21.15.46:0/1602424387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b8003520 con 0x7f82b8004800 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d8a00640 1 -- 172.21.15.46:0/1602424387 reap_dead start 2024-04-26T23:38:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.788+0000 7f82d3e00640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82b80039b0 0x7f82b8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.789+0000 7f82d3400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 0x7f82b8004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c210 tx=0x7f82b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa743266bb63c129 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc007d40 con 0x7f82b8004800 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc007290 con 0x7f82b8004800 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d2a00640 1 -- 172.21.15.46:0/1602424387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0057a0 con 0x7f82b8004800 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 -- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 msgr2=0x7f82b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 0x7f82b8004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c210 tx=0x7f82b8005830 comp rx=0 tx=0).stop 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 -- 172.21.15.46:0/1602424387 shutdown_connections 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 --2- 172.21.15.46:0/1602424387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 -- 172.21.15.46:0/1602424387 >> 172.21.15.46:0/1602424387 conn(0x7f82d409e2a0 msgr2=0x7f82d40d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.790+0000 7f82d9400640 1 -- 172.21.15.46:0/1602424387 shutdown_connections 2024-04-26T23:38:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.791+0000 7f82d9400640 1 -- 172.21.15.46:0/1602424387 wait complete. 2024-04-26T23:38:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.792+0000 7f82d9400640 1 Processor -- start 2024-04-26T23:38:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.792+0000 7f82d9400640 1 -- start start 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.793+0000 7f82d9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b8004800 0x7f82d415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.793+0000 7f82d9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d40a29b0 0x7f82d415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40b47c0 con 0x7f82d40a2530 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d9400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4162cb0 con 0x7f82b8004800 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40b4940 con 0x7f82d40a29b0 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42442/0 (socket says 172.21.15.46:42442) 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 -- 172.21.15.46:0/2025241072 learned_addr learned my addr 172.21.15.46:0/2025241072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3e00640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b8004800 0x7f82d415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d8a00640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d40a29b0 0x7f82d415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 -- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d40a29b0 msgr2=0x7f82d415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d40a29b0 0x7f82d415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 -- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b8004800 msgr2=0x7f82d415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b8004800 0x7f82d415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.794+0000 7f82d3400640 1 -- 172.21.15.46:0/2025241072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40a17d0 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82d3400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f82bc034040 tx=0x7f82bc001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc041030 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc0059b0 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005dc0 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d41fa7e0 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.795+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d40a1d30 con 0x7f82d40a2530 2024-04-26T23:38:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.797+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d40693b0 con 0x7f82d40a2530 2024-04-26T23:38:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.797+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82bc044030 con 0x7f82d40a2530 2024-04-26T23:38:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.798+0000 7f82cbe00640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 0x7f82b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.798+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f82bc0ac1e0 con 0x7f82d40a2530 2024-04-26T23:38:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.799+0000 7f82d3e00640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 0x7f82b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.800+0000 7f82d3e00640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 0x7f82b006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82c4007fd0 tx=0x7f82c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:52.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:52.803+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82bc04b070 con 0x7f82d40a2530 2024-04-26T23:38:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.058+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f82d40d25b0 con 0x7f82d40a2530 2024-04-26T23:38:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.704+0000 7f82cbe00640 1 -- 172.21.15.46:0/2025241072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f82d40a17d0 con 0x7f82d40a2530 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.707+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 msgr2=0x7f82b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.707+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 0x7f82b006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82c4007fd0 tx=0x7f82c4009040 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.707+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 msgr2=0x7f82d415ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.707+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f82bc034040 tx=0x7f82bc001bc0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 shutdown_connections 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82b006c920 0x7f82b006ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d40a29b0 0x7f82d415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82d40a2530 0x7f82d415ddb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 --2- 172.21.15.46:0/2025241072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b8004800 0x7f82d415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 >> 172.21.15.46:0/2025241072 conn(0x7f82d409e2a0 msgr2=0x7f82d406cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 shutdown_connections 2024-04-26T23:38:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.708+0000 7f82d9400640 1 -- 172.21.15.46:0/2025241072 wait complete. 2024-04-26T23:38:53.731 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:38:53.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:38:53.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.948+0000 7fddfbe00640 1 Processor -- start 2024-04-26T23:38:53.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.949+0000 7fddfbe00640 1 -- start start 2024-04-26T23:38:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddfc15cec0 0x7fddfc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfbe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddfc0ce7d0 con 0x7fddfc073b30 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0cdf90 con 0x7fddfc15cec0 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc0ceab0 con 0x7fddfc0cdb30 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.950+0000 7fddfaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddfc15cec0 0x7fddfc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddfc15cec0 0x7fddfc0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42456/0 (socket says 172.21.15.46:42456) 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 -- 172.21.15.46:0/3557736771 learned_addr learned my addr 172.21.15.46:0/3557736771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfa000640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 -- 172.21.15.46:0/3557736771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 msgr2=0x7fddfc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 -- 172.21.15.46:0/3557736771 >> v1:172.21.15.142:6789/0 conn(0x7fddfc073b30 legacy=0x7fddfc071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfaa00640 1 -- 172.21.15.46:0/3557736771 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0ced90 con 0x7fddfc15cec0 2024-04-26T23:38:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.951+0000 7fddfa000640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.952+0000 7fddfaa00640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc15cec0 0x7fddfc0cd5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b3830 tx=0x7fddec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b076c139c8bb08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddf9600640 1 -- 172.21.15.46:0/3557736771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec037070 con 0x7fddfc15cec0 2024-04-26T23:38:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddf9600640 1 -- 172.21.15.46:0/3557736771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec032440 con 0x7fddfc15cec0 2024-04-26T23:38:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddf9600640 1 -- 172.21.15.46:0/3557736771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec032810 con 0x7fddfc15cec0 2024-04-26T23:38:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddfbe00640 1 -- 172.21.15.46:0/3557736771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc15cec0 msgr2=0x7fddfc0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddfbe00640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc15cec0 0x7fddfc0cd5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b3830 tx=0x7fddec02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.953+0000 7fddfbe00640 1 -- 172.21.15.46:0/3557736771 shutdown_connections 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.954+0000 7fddfbe00640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.954+0000 7fddfbe00640 1 --2- 172.21.15.46:0/3557736771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc15cec0 0x7fddfc0cd5f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.954+0000 7fddfbe00640 1 -- 172.21.15.46:0/3557736771 >> 172.21.15.46:0/3557736771 conn(0x7fddfc0b27b0 msgr2=0x7fddfc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.954+0000 7fddfbe00640 1 -- 172.21.15.46:0/3557736771 shutdown_connections 2024-04-26T23:38:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.954+0000 7fddfbe00640 1 -- 172.21.15.46:0/3557736771 wait complete. 2024-04-26T23:38:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.955+0000 7fddfbe00640 1 Processor -- start 2024-04-26T23:38:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.956+0000 7fddfbe00640 1 -- start start 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 0x7fddfc0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15d2a0 con 0x7fddfc073b30 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc0cf350 con 0x7fddfc15cec0 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc073fa0 con 0x7fddfc0cdb30 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42460/0 (socket says 172.21.15.46:42460) 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 -- 172.21.15.46:0/2744893994 learned_addr learned my addr 172.21.15.46:0/2744893994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfa000640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfb400640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 0x7fddfc0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 -- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 msgr2=0x7fddfc1f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc1f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 -- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 msgr2=0x7fddfc0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 0x7fddfc0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfaa00640 1 -- 172.21.15.46:0/2744893994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0c8f70 con 0x7fddfc073b30 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfa000640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.957+0000 7fddfb400640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 0x7fddfc0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddfaa00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddec000c00 tx=0x7fddec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec041020 con 0x7fddfc073b30 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec030970 con 0x7fddfc073b30 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec037040 con 0x7fddfc073b30 2024-04-26T23:38:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc0c9250 con 0x7fddfc073b30 2024-04-26T23:38:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.958+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc0c9810 con 0x7fddfc073b30 2024-04-26T23:38:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.959+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc0693b0 con 0x7fddfc073b30 2024-04-26T23:38:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.960+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddec044020 con 0x7fddfc073b30 2024-04-26T23:38:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.961+0000 7fddf2a00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.961+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fddec0ac760 con 0x7fddfc073b30 2024-04-26T23:38:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.962+0000 7fddfa000640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.963+0000 7fddfa000640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006fd0 tx=0x7fdde8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:53.966+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddec04b030 con 0x7fddfc073b30 2024-04-26T23:38:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.219+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fddfc0c44b0 con 0x7fddfc073b30 2024-04-26T23:38:54.478 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:38:54.475+0000 7f6919000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:38:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.714+0000 7fddf2a00640 1 -- 172.21.15.46:0/2744893994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fddec07a9f0 con 0x7fddfc073b30 2024-04-26T23:38:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 msgr2=0x7fdddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006fd0 tx=0x7fdde8008040 comp rx=0 tx=0).stop 2024-04-26T23:38:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 msgr2=0x7fddfc0aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddec000c00 tx=0x7fddec0046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 shutdown_connections 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc15cec0 0x7fddfc0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddfc0cdb30 0x7fddfc1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 --2- 172.21.15.46:0/2744893994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddfc073b30 0x7fddfc0aeb60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 >> 172.21.15.46:0/2744893994 conn(0x7fddfc0b27b0 msgr2=0x7fddfc0b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 shutdown_connections 2024-04-26T23:38:54.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.718+0000 7fddfbe00640 1 -- 172.21.15.46:0/2744893994 wait complete. 2024-04-26T23:38:54.743 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:38:54.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:38:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.945+0000 7f7f56400640 1 Processor -- start 2024-04-26T23:38:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.945+0000 7f7f56400640 1 -- start start 2024-04-26T23:38:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.947+0000 7f7f56400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f50160bd0 0x7f7f50161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.947+0000 7f7f56400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f50161940 con 0x7f7f5006e510 2024-04-26T23:38:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.947+0000 7f7f56400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f50071f80 con 0x7f7f500732c0 2024-04-26T23:38:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.947+0000 7f7f56400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7f50072260 con 0x7f7f50160bd0 2024-04-26T23:38:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.947+0000 7f7f4ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f50160bd0 0x7f7f50161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.948+0000 7f7f4ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f50160bd0 0x7f7f50161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42474/0 (socket says 172.21.15.46:42474) 2024-04-26T23:38:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.948+0000 7f7f4ea00640 1 -- 172.21.15.46:0/2160786208 learned_addr learned my addr 172.21.15.46:0/2160786208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.948+0000 7f7f4ea00640 1 -- 172.21.15.46:0/2160786208 >> v1:172.21.15.142:6789/0 conn(0x7f7f500732c0 legacy=0x7f7f50073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.948+0000 7f7f4ea00640 1 -- 172.21.15.46:0/2160786208 >> v1:172.21.15.46:6789/0 conn(0x7f7f5006e510 legacy=0x7f7f5006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.948+0000 7f7f4ea00640 1 -- 172.21.15.46:0/2160786208 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f50072540 con 0x7f7f50160bd0 2024-04-26T23:38:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.949+0000 7f7f4ea00640 1 --2- 172.21.15.46:0/2160786208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50161060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f3802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8528ee62724c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f4e000640 1 -- 172.21.15.46:0/2160786208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f50160bd0 2024-04-26T23:38:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f4e000640 1 -- 172.21.15.46:0/2160786208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38032640 con 0x7f7f50160bd0 2024-04-26T23:38:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f4e000640 1 -- 172.21.15.46:0/2160786208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38032a10 con 0x7f7f50160bd0 2024-04-26T23:38:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 -- 172.21.15.46:0/2160786208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 msgr2=0x7f7f50161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 --2- 172.21.15.46:0/2160786208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50161060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f3802a330 comp rx=0 tx=0).stop 2024-04-26T23:38:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 -- 172.21.15.46:0/2160786208 shutdown_connections 2024-04-26T23:38:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 --2- 172.21.15.46:0/2160786208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50161060 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 -- 172.21.15.46:0/2160786208 >> 172.21.15.46:0/2160786208 conn(0x7f7f5006c530 msgr2=0x7f7f5006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 -- 172.21.15.46:0/2160786208 shutdown_connections 2024-04-26T23:38:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.950+0000 7f7f56400640 1 -- 172.21.15.46:0/2160786208 wait complete. 2024-04-26T23:38:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.952+0000 7f7f56400640 1 Processor -- start 2024-04-26T23:38:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.952+0000 7f7f56400640 1 -- start start 2024-04-26T23:38:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5006e510 0x7f7f50139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 0x7f7f5013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50161720 con 0x7f7f50160bd0 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501615a0 con 0x7f7f500732c0 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.953+0000 7f7f56400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f50072d40 con 0x7f7f5006e510 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42482/0 (socket says 172.21.15.46:42482) 2024-04-26T23:38:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 -- 172.21.15.46:0/1291038897 learned_addr learned my addr 172.21.15.46:0/1291038897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:54.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4ea00640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 0x7f7f5013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:54.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4f400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5006e510 0x7f7f50139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:54.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 -- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5006e510 msgr2=0x7f7f50139280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5006e510 0x7f7f50139280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 -- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 msgr2=0x7f7f5013d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 0x7f7f5013d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 -- 172.21.15.46:0/1291038897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f501397c0 con 0x7f7f50160bd0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4ea00640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 0x7f7f5013d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.954+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c7a0 tx=0x7f7f4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.955+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48003b70 con 0x7f7f50160bd0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.955+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f48003d30 con 0x7f7f50160bd0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.955+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800aa90 con 0x7f7f50160bd0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.955+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501403f0 con 0x7f7f50160bd0 2024-04-26T23:38:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.955+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f500b10d0 con 0x7f7f50160bd0 2024-04-26T23:38:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.956+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50069530 con 0x7f7f50160bd0 2024-04-26T23:38:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.957+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f4800ac30 con 0x7f7f50160bd0 2024-04-26T23:38:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.958+0000 7f7f47400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.958+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7f4808bbf0 con 0x7f7f50160bd0 2024-04-26T23:38:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.959+0000 7f7f4f400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.960+0000 7f7f4f400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:54.964+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f48059f20 con 0x7f7f50160bd0 2024-04-26T23:38:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.209+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f500b1840 con 0x7f7f50160bd0 2024-04-26T23:38:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.211+0000 7f7f47400640 1 -- 172.21.15.46:0/1291038897 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7f7f48059820 con 0x7f7f50160bd0 2024-04-26T23:38:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:38:54.220291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:38:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-26T23:38:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 msgr2=0x7f7f2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40009040 comp rx=0 tx=0).stop 2024-04-26T23:38:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 msgr2=0x7f7f50140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c7a0 tx=0x7f7f4800cc60 comp rx=0 tx=0).stop 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 shutdown_connections 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50160bd0 0x7f7f50140b80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f500732c0 0x7f7f5013d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 --2- 172.21.15.46:0/1291038897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5006e510 0x7f7f50139280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.213+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 >> 172.21.15.46:0/1291038897 conn(0x7f7f5006c530 msgr2=0x7f7f50142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.214+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 shutdown_connections 2024-04-26T23:38:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.214+0000 7f7f56400640 1 -- 172.21.15.46:0/1291038897 wait complete. 2024-04-26T23:38:55.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:38:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.448+0000 7fcbb9000640 1 Processor -- start 2024-04-26T23:38:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.449+0000 7fcbb9000640 1 -- start start 2024-04-26T23:38:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.450+0000 7fcbb9000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcbb4162940 0x7fcbb40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.450+0000 7fcbb9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb40b1fa0 con 0x7fcbb40d38c0 2024-04-26T23:38:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.450+0000 7fcbb9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb40b2280 con 0x7fcbb40b3e00 2024-04-26T23:38:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.450+0000 7fcbb9000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb40d5ad0 con 0x7fcbb4162940 2024-04-26T23:38:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.451+0000 7fcbb3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcbb40d38c0 0x7fcbb40b2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53390/0 (socket says 172.21.15.46:53390) 2024-04-26T23:38:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.451+0000 7fcbb3e00640 1 -- 172.21.15.46:0/4173393178 learned_addr learned my addr 172.21.15.46:0/4173393178 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.451+0000 7fcbb3400640 1 --2- 172.21.15.46:0/4173393178 >> v2:172.21.15.142:3300/0 conn(0x7fcbb4162940 0x7fcbb40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.451+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934680353 0 0) 0x7fcbb40b1fa0 con 0x7fcbb40d38c0 2024-04-26T23:38:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.451+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb98003520 con 0x7fcbb40d38c0 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3529897356 0 0) 0x7fcb98003520 con 0x7fcbb40d38c0 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 >> v2:172.21.15.142:3300/0 conn(0x7fcbb4162940 msgr2=0x7fcbb40cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 --2- 172.21.15.46:0/4173393178 >> v2:172.21.15.142:3300/0 conn(0x7fcbb4162940 0x7fcbb40cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 >> v1:172.21.15.142:6789/0 conn(0x7fcbb40b3e00 legacy=0x7fcbb40d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb40b2560 con 0x7fcbb40d38c0 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcbac001690 con 0x7fcbb40d38c0 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 >> v1:172.21.15.46:6789/0 conn(0x7fcbb40d38c0 legacy=0x7fcbb40b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb98003a00 0x7fcb98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb98004800 0x7fcb98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 0x7fcb9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb40d5ad0 con 0x7fcb980051c0 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800c0c0 con 0x7fcb98003a00 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9800c2c0 con 0x7fcb98004800 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.452+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 0x7fcb9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3400640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb98003a00 0x7fcb98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb2a00640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb98004800 0x7fcb98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3e00640 1 -- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb98004800 msgr2=0x7fcb98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb98004800 0x7fcb98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3e00640 1 -- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb98003a00 msgr2=0x7fcb98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb98003a00 0x7fcb98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3e00640 1 -- 172.21.15.46:0/4173393178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb98003520 con 0x7fcb980051c0 2024-04-26T23:38:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.453+0000 7fcbb3400640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb98003a00 0x7fcb98003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:55.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.454+0000 7fcbb3e00640 1 -- 172.21.15.46:0/4173393178 reap_dead start 2024-04-26T23:38:55.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.454+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 0x7fcb9800b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc1a0 tx=0x7fcb98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eb6bb62c2ef540 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.454+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007c80 con 0x7fcb980051c0 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.454+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac005ad0 con 0x7fcb980051c0 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.454+0000 7fcbb2000640 1 -- 172.21.15.46:0/4173393178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007070 con 0x7fcb980051c0 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 -- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 msgr2=0x7fcb9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 0x7fcb9800b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc1a0 tx=0x7fcb98005830 comp rx=0 tx=0).stop 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 -- 172.21.15.46:0/4173393178 shutdown_connections 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 --2- 172.21.15.46:0/4173393178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb980051c0 0x7fcb9800b9c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 -- 172.21.15.46:0/4173393178 >> 172.21.15.46:0/4173393178 conn(0x7fcbb409e460 msgr2=0x7fcbb40d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 -- 172.21.15.46:0/4173393178 shutdown_connections 2024-04-26T23:38:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.455+0000 7fcbb9000640 1 -- 172.21.15.46:0/4173393178 wait complete. 2024-04-26T23:38:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.457+0000 7fcbb9000640 1 Processor -- start 2024-04-26T23:38:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.457+0000 7fcbb9000640 1 -- start start 2024-04-26T23:38:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb980051c0 0x7fcbb40cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb40b2c60 0x7fcbb40b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4162d20 con 0x7fcbb41510e0 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb40b1d50 con 0x7fcbb40b2c60 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb40cfa50 con 0x7fcb980051c0 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42506/0 (socket says 172.21.15.46:42506) 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.458+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3382638565 learned_addr learned my addr 172.21.15.46:0/3382638565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3400640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb980051c0 0x7fcbb40cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb2a00640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb40b2c60 0x7fcbb40b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb980051c0 msgr2=0x7fcbb40cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb980051c0 0x7fcbb40cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb40b2c60 msgr2=0x7fcbb40b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb40b2c60 0x7fcbb40b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3382638565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb41f47c0 con 0x7fcbb41510e0 2024-04-26T23:38:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.459+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40c8c20 tx=0x7fcbac001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.460+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac040070 con 0x7fcbb41510e0 2024-04-26T23:38:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.460+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac0047e0 con 0x7fcbb41510e0 2024-04-26T23:38:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.460+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41f4a40 con 0x7fcbb41510e0 2024-04-26T23:38:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.460+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac004bb0 con 0x7fcbb41510e0 2024-04-26T23:38:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.460+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41f5000 con 0x7fcbb41510e0 2024-04-26T23:38:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.461+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb40b3600 con 0x7fcbb41510e0 2024-04-26T23:38:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.462+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbac043070 con 0x7fcbb41510e0 2024-04-26T23:38:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.463+0000 7fcbab400640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 0x7fcb9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.463+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcbac0aae10 con 0x7fcbb41510e0 2024-04-26T23:38:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.464+0000 7fcbb3400640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 0x7fcb9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.465+0000 7fcbb3400640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 0x7fcb9406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007440 tx=0x7fcba400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.468+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbb41f47c0 con 0x7fcbb41510e0 2024-04-26T23:38:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.671+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbb40693b0 con 0x7fcbb41510e0 2024-04-26T23:38:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.674+0000 7fcbab400640 1 -- 172.21.15.46:0/3382638565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16906 (secure 0 0 0) 0x7fcbac079020 con 0x7fcbb41510e0 2024-04-26T23:38:55.678 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:55.678 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:38:52.675934+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:38:55.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.676+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 msgr2=0x7fcb9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.676+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 0x7fcb9406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007440 tx=0x7fcba400b040 comp rx=0 tx=0).stop 2024-04-26T23:38:55.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.676+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 msgr2=0x7fcbb4151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.676+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40c8c20 tx=0x7fcbac001990 comp rx=0 tx=0).stop 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 shutdown_connections 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb9406c920 0x7fcb9406ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41510e0 0x7fcbb4151560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb40b2c60 0x7fcbb40b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 --2- 172.21.15.46:0/3382638565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb980051c0 0x7fcbb40cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 >> 172.21.15.46:0/3382638565 conn(0x7fcbb409e460 msgr2=0x7fcbb415f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 shutdown_connections 2024-04-26T23:38:55.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.677+0000 7fcbb9000640 1 -- 172.21.15.46:0/3382638565 wait complete. 2024-04-26T23:38:55.700 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:38:55.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:38:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.907+0000 7fd682000640 1 Processor -- start 2024-04-26T23:38:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.907+0000 7fd682000640 1 -- start start 2024-04-26T23:38:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.908+0000 7fd682000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd67c15f4e0 0x7fd67c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.908+0000 7fd682000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c0d11f0 con 0x7fd67c0b4000 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.908+0000 7fd682000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c0d5f20 con 0x7fd67c0d2130 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd682000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c0d0ce0 con 0x7fd67c15f4e0 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd67c15f4e0 0x7fd67c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd67c15f4e0 0x7fd67c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42520/0 (socket says 172.21.15.46:42520) 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 -- 172.21.15.46:0/3124152226 learned_addr learned my addr 172.21.15.46:0/3124152226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd67be00640 1 --1- 172.21.15.46:0/3124152226 >> v1:172.21.15.46:6789/0 conn(0x7fd67c0b4000 0x7fd67c0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53398/0 (socket says 172.21.15.46:53398) 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 -- 172.21.15.46:0/3124152226 >> v1:172.21.15.142:6789/0 conn(0x7fd67c0d2130 legacy=0x7fd67c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 -- 172.21.15.46:0/3124152226 >> v1:172.21.15.46:6789/0 conn(0x7fd67c0b4000 legacy=0x7fd67c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd680c00640 1 -- 172.21.15.46:0/3124152226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c0d0f10 con 0x7fd67c15f4e0 2024-04-26T23:38:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.909+0000 7fd67b400640 1 -- 172.21.15.46:0/3124152226 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635800547 0 0) 0x7fd67c0d11f0 con 0x7fd67c0b4000 2024-04-26T23:38:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.910+0000 7fd680c00640 1 --2- 172.21.15.46:0/3124152226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c15f4e0 0x7fd67c0d2df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4023df26e6d14c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd67b400640 1 -- 172.21.15.46:0/3124152226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c037070 con 0x7fd67c15f4e0 2024-04-26T23:38:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd67b400640 1 -- 172.21.15.46:0/3124152226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c032440 con 0x7fd67c15f4e0 2024-04-26T23:38:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd67b400640 1 -- 172.21.15.46:0/3124152226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c032810 con 0x7fd67c15f4e0 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 -- 172.21.15.46:0/3124152226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c15f4e0 msgr2=0x7fd67c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 --2- 172.21.15.46:0/3124152226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c15f4e0 0x7fd67c0d2df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c02a2a0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 -- 172.21.15.46:0/3124152226 shutdown_connections 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 --2- 172.21.15.46:0/3124152226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c15f4e0 0x7fd67c0d2df0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 -- 172.21.15.46:0/3124152226 >> 172.21.15.46:0/3124152226 conn(0x7fd67c0d5010 msgr2=0x7fd67c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.911+0000 7fd682000640 1 -- 172.21.15.46:0/3124152226 shutdown_connections 2024-04-26T23:38:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.912+0000 7fd682000640 1 -- 172.21.15.46:0/3124152226 wait complete. 2024-04-26T23:38:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.913+0000 7fd682000640 1 Processor -- start 2024-04-26T23:38:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.913+0000 7fd682000640 1 -- start start 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 0x7fd67c0b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd67c15f4e0 0x7fd67c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0b1d60 con 0x7fd67c0d2130 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0d2990 con 0x7fd67c15f4e0 2024-04-26T23:38:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.914+0000 7fd682000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c15f8c0 con 0x7fd67c0b4000 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42536/0 (socket says 172.21.15.46:42536) 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 -- 172.21.15.46:0/2675327391 learned_addr learned my addr 172.21.15.46:0/2675327391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd681600640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd67c15f4e0 0x7fd67c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd680c00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 0x7fd67c0b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 -- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 msgr2=0x7fd67c0b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 0x7fd67c0b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 -- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd67c15f4e0 msgr2=0x7fd67c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd67c15f4e0 0x7fd67c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd67be00640 1 -- 172.21.15.46:0/2675327391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c1f6440 con 0x7fd67c0d2130 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.915+0000 7fd680c00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 0x7fd67c0b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd67be00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c7c0 tx=0x7fd66400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664018070 con 0x7fd67c0d2130 2024-04-26T23:38:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66400a5e0 con 0x7fd67c0d2130 2024-04-26T23:38:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c15c8a0 con 0x7fd67c0d2130 2024-04-26T23:38:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400a9d0 con 0x7fd67c0d2130 2024-04-26T23:38:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.916+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c15a290 con 0x7fd67c0d2130 2024-04-26T23:38:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.917+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c0693b0 con 0x7fd67c0d2130 2024-04-26T23:38:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.918+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd66401b070 con 0x7fd67c0d2130 2024-04-26T23:38:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.919+0000 7fd678c00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.919+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd66408b2a0 con 0x7fd67c0d2130 2024-04-26T23:38:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.920+0000 7fd680c00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.921+0000 7fd680c00640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd66c034000 tx=0x7fd66c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:55.923+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd67c1f6440 con 0x7fd67c0d2130 2024-04-26T23:38:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.125+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd67c0d33d0 con 0x7fd65806c8f0 2024-04-26T23:38:56.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.146+0000 7fd678c00640 1 -- 172.21.15.46:0/2675327391 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fd67c0d33d0 con 0x7fd65806c8f0 2024-04-26T23:38:56.153 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:56.154 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 msgr2=0x7fd65806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd66c034000 tx=0x7fd66c02e000 comp rx=0 tx=0).stop 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 msgr2=0x7fd67c0ba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c7c0 tx=0x7fd66400cc80 comp rx=0 tx=0).stop 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 shutdown_connections 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd67c15f4e0 0x7fd67c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd67c0d2130 0x7fd67c0ba880 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 --2- 172.21.15.46:0/2675327391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd67c0b4000 0x7fd67c0b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 >> 172.21.15.46:0/2675327391 conn(0x7fd67c0d5010 msgr2=0x7fd67c0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:56.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 shutdown_connections 2024-04-26T23:38:56.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.152+0000 7fd682000640 1 -- 172.21.15.46:0/2675327391 wait complete. 2024-04-26T23:38:56.175 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-26T23:38:55.922008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233088,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741632,"statfs":{"total":773094113280,"available":772855431168,"internally_reserved":0,"allocated":4546560,"data_stored":2600788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001900"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677871+0000","last_change":"2024-04-26T23:38:50.677871+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677871+0000","last_undegraded":"2024-04-26T23:38:50.677871+0000","last_fullsized":"2024-04-26T23:38:50.677871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680384+0000","last_change":"2024-04-26T23:38:50.680384+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680384+0000","last_undegraded":"2024-04-26T23:38:50.680384+0000","last_fullsized":"2024-04-26T23:38:50.680384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669294+0000","last_change":"2024-04-26T23:38:50.669294+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669294+0000","last_undegraded":"2024-04-26T23:38:50.669294+0000","last_fullsized":"2024-04-26T23:38:50.669294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.675317+0000","last_change":"2024-04-26T23:38:50.675317+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.675317+0000","last_undegraded":"2024-04-26T23:38:50.675317+0000","last_fullsized":"2024-04-26T23:38:50.675317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672056+0000","last_change":"2024-04-26T23:38:50.672056+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672056+0000","last_undegraded":"2024-04-26T23:38:50.672056+0000","last_fullsized":"2024-04-26T23:38:50.672056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677312+0000","last_change":"2024-04-26T23:38:50.677312+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677312+0000","last_undegraded":"2024-04-26T23:38:50.677312+0000","last_fullsized":"2024-04-26T23:38:50.677312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667316+0000","last_change":"2024-04-26T23:38:50.667316+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667316+0000","last_undegraded":"2024-04-26T23:38:50.667316+0000","last_fullsized":"2024-04-26T23:38:50.667316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672414+0000","last_change":"2024-04-26T23:38:50.672414+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672414+0000","last_undegraded":"2024-04-26T23:38:50.672414+0000","last_fullsized":"2024-04-26T23:38:50.672414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667307+0000","last_change":"2024-04-26T23:38:50.667307+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667307+0000","last_undegraded":"2024-04-26T23:38:50.667307+0000","last_fullsized":"2024-04-26T23:38:50.667307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669299+0000","last_change":"2024-04-26T23:38:50.669299+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669299+0000","last_undegraded":"2024-04-26T23:38:50.669299+0000","last_fullsized":"2024-04-26T23:38:50.669299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674349+0000","last_change":"2024-04-26T23:38:50.674349+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674349+0000","last_undegraded":"2024-04-26T23:38:50.674349+0000","last_fullsized":"2024-04-26T23:38:50.674349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667276+0000","last_change":"2024-04-26T23:38:50.667276+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667276+0000","last_undegraded":"2024-04-26T23:38:50.667276+0000","last_fullsized":"2024-04-26T23:38:50.667276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672590+0000","last_change":"2024-04-26T23:38:50.672590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672590+0000","last_undegraded":"2024-04-26T23:38:50.672590+0000","last_fullsized":"2024-04-26T23:38:50.672590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671236+0000","last_change":"2024-04-26T23:38:50.671236+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671236+0000","last_undegraded":"2024-04-26T23:38:50.671236+0000","last_fullsized":"2024-04-26T23:38:50.671236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666175+0000","last_change":"2024-04-26T23:38:50.666175+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666175+0000","last_undegraded":"2024-04-26T23:38:50.666175+0000","last_fullsized":"2024-04-26T23:38:50.666175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676905+0000","last_change":"2024-04-26T23:38:50.676905+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676905+0000","last_undegraded":"2024-04-26T23:38:50.676905+0000","last_fullsized":"2024-04-26T23:38:50.676905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.685115+0000","last_change":"2024-04-26T23:38:50.685115+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.685115+0000","last_undegraded":"2024-04-26T23:38:50.685115+0000","last_fullsized":"2024-04-26T23:38:50.685115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676567+0000","last_change":"2024-04-26T23:38:50.676567+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676567+0000","last_undegraded":"2024-04-26T23:38:50.676567+0000","last_fullsized":"2024-04-26T23:38:50.676567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673936+0000","last_change":"2024-04-26T23:38:50.673936+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673936+0000","last_undegraded":"2024-04-26T23:38:50.673936+0000","last_fullsized":"2024-04-26T23:38:50.673936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677945+0000","last_change":"2024-04-26T23:38:50.677945+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677945+0000","last_undegraded":"2024-04-26T23:38:50.677945+0000","last_fullsized":"2024-04-26T23:38:50.677945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671386+0000","last_change":"2024-04-26T23:38:50.671386+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671386+0000","last_undegraded":"2024-04-26T23:38:50.671386+0000","last_fullsized":"2024-04-26T23:38:50.671386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669210+0000","last_change":"2024-04-26T23:38:50.669210+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669210+0000","last_undegraded":"2024-04-26T23:38:50.669210+0000","last_fullsized":"2024-04-26T23:38:50.669210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679803+0000","last_change":"2024-04-26T23:38:50.679803+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679803+0000","last_undegraded":"2024-04-26T23:38:50.679803+0000","last_fullsized":"2024-04-26T23:38:50.679803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674451+0000","last_change":"2024-04-26T23:38:50.674451+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674451+0000","last_undegraded":"2024-04-26T23:38:50.674451+0000","last_fullsized":"2024-04-26T23:38:50.674451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677442+0000","last_change":"2024-04-26T23:38:50.677442+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677442+0000","last_undegraded":"2024-04-26T23:38:50.677442+0000","last_fullsized":"2024-04-26T23:38:50.677442+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678807+0000","last_change":"2024-04-26T23:38:50.678807+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678807+0000","last_undegraded":"2024-04-26T23:38:50.678807+0000","last_fullsized":"2024-04-26T23:38:50.678807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676493+0000","last_change":"2024-04-26T23:38:50.676493+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676493+0000","last_undegraded":"2024-04-26T23:38:50.676493+0000","last_fullsized":"2024-04-26T23:38:50.676493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673334+0000","last_change":"2024-04-26T23:38:50.673334+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673334+0000","last_undegraded":"2024-04-26T23:38:50.673334+0000","last_fullsized":"2024-04-26T23:38:50.673334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678505+0000","last_change":"2024-04-26T23:38:50.678505+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678505+0000","last_undegraded":"2024-04-26T23:38:50.678505+0000","last_fullsized":"2024-04-26T23:38:50.678505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676624+0000","last_change":"2024-04-26T23:38:50.676624+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676624+0000","last_undegraded":"2024-04-26T23:38:50.676624+0000","last_fullsized":"2024-04-26T23:38:50.676624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679221+0000","last_change":"2024-04-26T23:38:50.679221+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679221+0000","last_undegraded":"2024-04-26T23:38:50.679221+0000","last_fullsized":"2024-04-26T23:38:50.679221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676259+0000","last_change":"2024-04-26T23:38:50.676259+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676259+0000","last_undegraded":"2024-04-26T23:38:50.676259+0000","last_fullsized":"2024-04-26T23:38:50.676259+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682341+0000","last_change":"2024-04-26T23:38:50.682341+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682341+0000","last_undegraded":"2024-04-26T23:38:50.682341+0000","last_fullsized":"2024-04-26T23:38:50.682341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694038+0000","last_change":"2024-04-26T23:38:50.691324+0000","last_active":"2024-04-26T23:38:50.694038+0000","last_peered":"2024-04-26T23:38:50.694038+0000","last_clean":"2024-04-26T23:38:50.694038+0000","last_became_active":"2024-04-26T23:38:50.691023+0000","last_became_peered":"2024-04-26T23:38:50.691023+0000","last_unstale":"2024-04-26T23:38:50.694038+0000","last_undegraded":"2024-04-26T23:38:50.694038+0000","last_fullsized":"2024-04-26T23:38:50.694038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:55.670095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678183+0000","last_change":"2024-04-26T23:38:50.678183+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678183+0000","last_undegraded":"2024-04-26T23:38:50.678183+0000","last_fullsized":"2024-04-26T23:38:50.678183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693660+0000","last_change":"2024-04-26T23:38:50.691001+0000","last_active":"2024-04-26T23:38:50.693660+0000","last_peered":"2024-04-26T23:38:50.693660+0000","last_clean":"2024-04-26T23:38:50.693660+0000","last_became_active":"2024-04-26T23:38:50.690668+0000","last_became_peered":"2024-04-26T23:38:50.690668+0000","last_unstale":"2024-04-26T23:38:50.693660+0000","last_undegraded":"2024-04-26T23:38:50.693660+0000","last_fullsized":"2024-04-26T23:38:50.693660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665322+0000","last_change":"2024-04-26T23:38:50.665322+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665322+0000","last_undegraded":"2024-04-26T23:38:50.665322+0000","last_fullsized":"2024-04-26T23:38:50.665322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694003+0000","last_change":"2024-04-26T23:38:50.689919+0000","last_active":"2024-04-26T23:38:50.694003+0000","last_peered":"2024-04-26T23:38:50.694003+0000","last_clean":"2024-04-26T23:38:50.694003+0000","last_became_active":"2024-04-26T23:38:50.689629+0000","last_became_peered":"2024-04-26T23:38:50.689629+0000","last_unstale":"2024-04-26T23:38:50.694003+0000","last_undegraded":"2024-04-26T23:38:50.694003+0000","last_fullsized":"2024-04-26T23:38:50.694003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.753794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678342+0000","last_change":"2024-04-26T23:38:50.678342+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678342+0000","last_undegraded":"2024-04-26T23:38:50.678342+0000","last_fullsized":"2024-04-26T23:38:50.678342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690252+0000","last_change":"2024-04-26T23:38:50.684308+0000","last_active":"2024-04-26T23:38:50.690252+0000","last_peered":"2024-04-26T23:38:50.690252+0000","last_clean":"2024-04-26T23:38:50.690252+0000","last_became_active":"2024-04-26T23:38:50.684017+0000","last_became_peered":"2024-04-26T23:38:50.684017+0000","last_unstale":"2024-04-26T23:38:50.690252+0000","last_undegraded":"2024-04-26T23:38:50.690252+0000","last_fullsized":"2024-04-26T23:38:50.690252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.549403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667266+0000","last_change":"2024-04-26T23:38:50.667266+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667266+0000","last_undegraded":"2024-04-26T23:38:50.667266+0000","last_fullsized":"2024-04-26T23:38:50.667266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693328+0000","last_change":"2024-04-26T23:38:50.687042+0000","last_active":"2024-04-26T23:38:50.693328+0000","last_peered":"2024-04-26T23:38:50.693328+0000","last_clean":"2024-04-26T23:38:50.693328+0000","last_became_active":"2024-04-26T23:38:50.685547+0000","last_became_peered":"2024-04-26T23:38:50.685547+0000","last_unstale":"2024-04-26T23:38:50.693328+0000","last_undegraded":"2024-04-26T23:38:50.693328+0000","last_fullsized":"2024-04-26T23:38:50.693328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.664806+0000","last_change":"2024-04-26T23:38:50.664806+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.664806+0000","last_undegraded":"2024-04-26T23:38:50.664806+0000","last_fullsized":"2024-04-26T23:38:50.664806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697047+0000","last_change":"2024-04-26T23:38:50.695028+0000","last_active":"2024-04-26T23:38:50.697047+0000","last_peered":"2024-04-26T23:38:50.697047+0000","last_clean":"2024-04-26T23:38:50.697047+0000","last_became_active":"2024-04-26T23:38:50.694628+0000","last_became_peered":"2024-04-26T23:38:50.694628+0000","last_unstale":"2024-04-26T23:38:50.697047+0000","last_undegraded":"2024-04-26T23:38:50.697047+0000","last_fullsized":"2024-04-26T23:38:50.697047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667992+0000","last_change":"2024-04-26T23:38:50.667992+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667992+0000","last_undegraded":"2024-04-26T23:38:50.667992+0000","last_fullsized":"2024-04-26T23:38:50.667992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699401+0000","last_change":"2024-04-26T23:38:50.698125+0000","last_active":"2024-04-26T23:38:50.699401+0000","last_peered":"2024-04-26T23:38:50.699401+0000","last_clean":"2024-04-26T23:38:50.699401+0000","last_became_active":"2024-04-26T23:38:50.697795+0000","last_became_peered":"2024-04-26T23:38:50.697795+0000","last_unstale":"2024-04-26T23:38:50.699401+0000","last_undegraded":"2024-04-26T23:38:50.699401+0000","last_fullsized":"2024-04-26T23:38:50.699401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673039+0000","last_change":"2024-04-26T23:38:50.673039+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673039+0000","last_undegraded":"2024-04-26T23:38:50.673039+0000","last_fullsized":"2024-04-26T23:38:50.673039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689406+0000","last_change":"2024-04-26T23:38:50.680954+0000","last_active":"2024-04-26T23:38:50.689406+0000","last_peered":"2024-04-26T23:38:50.689406+0000","last_clean":"2024-04-26T23:38:50.689406+0000","last_became_active":"2024-04-26T23:38:50.680546+0000","last_became_peered":"2024-04-26T23:38:50.680546+0000","last_unstale":"2024-04-26T23:38:50.689406+0000","last_undegraded":"2024-04-26T23:38:50.689406+0000","last_fullsized":"2024-04-26T23:38:50.689406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:51.613896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674846+0000","last_change":"2024-04-26T23:38:50.674846+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674846+0000","last_undegraded":"2024-04-26T23:38:50.674846+0000","last_fullsized":"2024-04-26T23:38:50.674846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696898+0000","last_change":"2024-04-26T23:38:50.694593+0000","last_active":"2024-04-26T23:38:50.696898+0000","last_peered":"2024-04-26T23:38:50.696898+0000","last_clean":"2024-04-26T23:38:50.696898+0000","last_became_active":"2024-04-26T23:38:50.694289+0000","last_became_peered":"2024-04-26T23:38:50.694289+0000","last_unstale":"2024-04-26T23:38:50.696898+0000","last_undegraded":"2024-04-26T23:38:50.696898+0000","last_fullsized":"2024-04-26T23:38:50.696898+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.683436+0000","last_change":"2024-04-26T23:38:50.683436+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.683436+0000","last_undegraded":"2024-04-26T23:38:50.683436+0000","last_fullsized":"2024-04-26T23:38:50.683436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.701161+0000","last_change":"2024-04-26T23:38:50.696626+0000","last_active":"2024-04-26T23:38:50.701161+0000","last_peered":"2024-04-26T23:38:50.701161+0000","last_clean":"2024-04-26T23:38:50.701161+0000","last_became_active":"2024-04-26T23:38:50.696142+0000","last_became_peered":"2024-04-26T23:38:50.696142+0000","last_unstale":"2024-04-26T23:38:50.701161+0000","last_undegraded":"2024-04-26T23:38:50.701161+0000","last_fullsized":"2024-04-26T23:38:50.701161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666542+0000","last_change":"2024-04-26T23:38:50.666542+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666542+0000","last_undegraded":"2024-04-26T23:38:50.666542+0000","last_fullsized":"2024-04-26T23:38:50.666542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.698415+0000","last_change":"2024-04-26T23:38:50.695193+0000","last_active":"2024-04-26T23:38:50.698415+0000","last_peered":"2024-04-26T23:38:50.698415+0000","last_clean":"2024-04-26T23:38:50.698415+0000","last_became_active":"2024-04-26T23:38:50.694850+0000","last_became_peered":"2024-04-26T23:38:50.694850+0000","last_unstale":"2024-04-26T23:38:50.698415+0000","last_undegraded":"2024-04-26T23:38:50.698415+0000","last_fullsized":"2024-04-26T23:38:50.698415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.668261+0000","last_change":"2024-04-26T23:38:50.668261+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.668261+0000","last_undegraded":"2024-04-26T23:38:50.668261+0000","last_fullsized":"2024-04-26T23:38:50.668261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697007+0000","last_change":"2024-04-26T23:38:50.695400+0000","last_active":"2024-04-26T23:38:50.697007+0000","last_peered":"2024-04-26T23:38:50.697007+0000","last_clean":"2024-04-26T23:38:50.697007+0000","last_became_active":"2024-04-26T23:38:50.695178+0000","last_became_peered":"2024-04-26T23:38:50.695178+0000","last_unstale":"2024-04-26T23:38:50.697007+0000","last_undegraded":"2024-04-26T23:38:50.697007+0000","last_fullsized":"2024-04-26T23:38:50.697007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:22.297685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666548+0000","last_change":"2024-04-26T23:38:50.666548+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666548+0000","last_undegraded":"2024-04-26T23:38:50.666548+0000","last_fullsized":"2024-04-26T23:38:50.666548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696932+0000","last_change":"2024-04-26T23:38:50.695534+0000","last_active":"2024-04-26T23:38:50.696932+0000","last_peered":"2024-04-26T23:38:50.696932+0000","last_clean":"2024-04-26T23:38:50.696932+0000","last_became_active":"2024-04-26T23:38:50.686496+0000","last_became_peered":"2024-04-26T23:38:50.686496+0000","last_unstale":"2024-04-26T23:38:50.696932+0000","last_undegraded":"2024-04-26T23:38:50.696932+0000","last_fullsized":"2024-04-26T23:38:50.696932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.662014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672274+0000","last_change":"2024-04-26T23:38:50.672274+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672274+0000","last_undegraded":"2024-04-26T23:38:50.672274+0000","last_fullsized":"2024-04-26T23:38:50.672274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699742+0000","last_change":"2024-04-26T23:38:50.698660+0000","last_active":"2024-04-26T23:38:50.699742+0000","last_peered":"2024-04-26T23:38:50.699742+0000","last_clean":"2024-04-26T23:38:50.699742+0000","last_became_active":"2024-04-26T23:38:50.698295+0000","last_became_peered":"2024-04-26T23:38:50.698295+0000","last_unstale":"2024-04-26T23:38:50.699742+0000","last_undegraded":"2024-04-26T23:38:50.699742+0000","last_fullsized":"2024-04-26T23:38:50.699742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679622+0000","last_change":"2024-04-26T23:38:50.679622+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679622+0000","last_undegraded":"2024-04-26T23:38:50.679622+0000","last_fullsized":"2024-04-26T23:38:50.679622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689132+0000","last_change":"2024-04-26T23:38:50.686322+0000","last_active":"2024-04-26T23:38:50.689132+0000","last_peered":"2024-04-26T23:38:50.689132+0000","last_clean":"2024-04-26T23:38:50.689132+0000","last_became_active":"2024-04-26T23:38:50.684566+0000","last_became_peered":"2024-04-26T23:38:50.684566+0000","last_unstale":"2024-04-26T23:38:50.689132+0000","last_undegraded":"2024-04-26T23:38:50.689132+0000","last_fullsized":"2024-04-26T23:38:50.689132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:39.251641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672490+0000","last_change":"2024-04-26T23:38:50.672490+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672490+0000","last_undegraded":"2024-04-26T23:38:50.672490+0000","last_fullsized":"2024-04-26T23:38:50.672490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697031+0000","last_change":"2024-04-26T23:38:50.694214+0000","last_active":"2024-04-26T23:38:50.697031+0000","last_peered":"2024-04-26T23:38:50.697031+0000","last_clean":"2024-04-26T23:38:50.697031+0000","last_became_active":"2024-04-26T23:38:50.693909+0000","last_became_peered":"2024-04-26T23:38:50.693909+0000","last_unstale":"2024-04-26T23:38:50.697031+0000","last_undegraded":"2024-04-26T23:38:50.697031+0000","last_fullsized":"2024-04-26T23:38:50.697031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679586+0000","last_change":"2024-04-26T23:38:50.679586+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679586+0000","last_undegraded":"2024-04-26T23:38:50.679586+0000","last_fullsized":"2024-04-26T23:38:50.679586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692161+0000","last_change":"2024-04-26T23:38:50.686368+0000","last_active":"2024-04-26T23:38:50.692161+0000","last_peered":"2024-04-26T23:38:50.692161+0000","last_clean":"2024-04-26T23:38:50.692161+0000","last_became_active":"2024-04-26T23:38:50.686085+0000","last_became_peered":"2024-04-26T23:38:50.686085+0000","last_unstale":"2024-04-26T23:38:50.692161+0000","last_undegraded":"2024-04-26T23:38:50.692161+0000","last_fullsized":"2024-04-26T23:38:50.692161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671636+0000","last_change":"2024-04-26T23:38:50.671636+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671636+0000","last_undegraded":"2024-04-26T23:38:50.671636+0000","last_fullsized":"2024-04-26T23:38:50.671636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690421+0000","last_change":"2024-04-26T23:38:50.685721+0000","last_active":"2024-04-26T23:38:50.690421+0000","last_peered":"2024-04-26T23:38:50.690421+0000","last_clean":"2024-04-26T23:38:50.690421+0000","last_became_active":"2024-04-26T23:38:50.685381+0000","last_became_peered":"2024-04-26T23:38:50.685381+0000","last_unstale":"2024-04-26T23:38:50.690421+0000","last_undegraded":"2024-04-26T23:38:50.690421+0000","last_fullsized":"2024-04-26T23:38:50.690421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:05.204342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666638+0000","last_change":"2024-04-26T23:38:50.666638+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666638+0000","last_undegraded":"2024-04-26T23:38:50.666638+0000","last_fullsized":"2024-04-26T23:38:50.666638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692041+0000","last_change":"2024-04-26T23:38:50.690198+0000","last_active":"2024-04-26T23:38:50.692041+0000","last_peered":"2024-04-26T23:38:50.692041+0000","last_clean":"2024-04-26T23:38:50.692041+0000","last_became_active":"2024-04-26T23:38:50.689613+0000","last_became_peered":"2024-04-26T23:38:50.689613+0000","last_unstale":"2024-04-26T23:38:50.692041+0000","last_undegraded":"2024-04-26T23:38:50.692041+0000","last_fullsized":"2024-04-26T23:38:50.692041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:57:08.503076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671249+0000","last_change":"2024-04-26T23:38:50.671249+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671249+0000","last_undegraded":"2024-04-26T23:38:50.671249+0000","last_fullsized":"2024-04-26T23:38:50.671249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693306+0000","last_change":"2024-04-26T23:38:50.691793+0000","last_active":"2024-04-26T23:38:50.693306+0000","last_peered":"2024-04-26T23:38:50.693306+0000","last_clean":"2024-04-26T23:38:50.693306+0000","last_became_active":"2024-04-26T23:38:50.691492+0000","last_became_peered":"2024-04-26T23:38:50.691492+0000","last_unstale":"2024-04-26T23:38:50.693306+0000","last_undegraded":"2024-04-26T23:38:50.693306+0000","last_fullsized":"2024-04-26T23:38:50.693306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676346+0000","last_change":"2024-04-26T23:38:50.676346+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676346+0000","last_undegraded":"2024-04-26T23:38:50.676346+0000","last_fullsized":"2024-04-26T23:38:50.676346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690651+0000","last_change":"2024-04-26T23:38:50.687713+0000","last_active":"2024-04-26T23:38:50.690651+0000","last_peered":"2024-04-26T23:38:50.690651+0000","last_clean":"2024-04-26T23:38:50.690651+0000","last_became_active":"2024-04-26T23:38:50.687314+0000","last_became_peered":"2024-04-26T23:38:50.687314+0000","last_unstale":"2024-04-26T23:38:50.690651+0000","last_undegraded":"2024-04-26T23:38:50.690651+0000","last_fullsized":"2024-04-26T23:38:50.690651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:38.872504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665851+0000","last_change":"2024-04-26T23:38:50.665851+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665851+0000","last_undegraded":"2024-04-26T23:38:50.665851+0000","last_fullsized":"2024-04-26T23:38:50.665851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.684746+0000","last_change":"2024-04-26T23:38:50.682745+0000","last_active":"2024-04-26T23:38:50.684746+0000","last_peered":"2024-04-26T23:38:50.684746+0000","last_clean":"2024-04-26T23:38:50.684746+0000","last_became_active":"2024-04-26T23:38:50.682462+0000","last_became_peered":"2024-04-26T23:38:50.682462+0000","last_unstale":"2024-04-26T23:38:50.684746+0000","last_undegraded":"2024-04-26T23:38:50.684746+0000","last_fullsized":"2024-04-26T23:38:50.684746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:19.185932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670016+0000","last_change":"2024-04-26T23:38:50.670016+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670016+0000","last_undegraded":"2024-04-26T23:38:50.670016+0000","last_fullsized":"2024-04-26T23:38:50.670016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.700135+0000","last_change":"2024-04-26T23:38:50.699004+0000","last_active":"2024-04-26T23:38:50.700135+0000","last_peered":"2024-04-26T23:38:50.700135+0000","last_clean":"2024-04-26T23:38:50.700135+0000","last_became_active":"2024-04-26T23:38:50.698643+0000","last_became_peered":"2024-04-26T23:38:50.698643+0000","last_unstale":"2024-04-26T23:38:50.700135+0000","last_undegraded":"2024-04-26T23:38:50.700135+0000","last_fullsized":"2024-04-26T23:38:50.700135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671845+0000","last_change":"2024-04-26T23:38:50.671845+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671845+0000","last_undegraded":"2024-04-26T23:38:50.671845+0000","last_fullsized":"2024-04-26T23:38:50.671845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.691431+0000","last_change":"2024-04-26T23:38:50.688989+0000","last_active":"2024-04-26T23:38:50.691431+0000","last_peered":"2024-04-26T23:38:50.691431+0000","last_clean":"2024-04-26T23:38:50.691431+0000","last_became_active":"2024-04-26T23:38:50.688711+0000","last_became_peered":"2024-04-26T23:38:50.688711+0000","last_unstale":"2024-04-26T23:38:50.691431+0000","last_undegraded":"2024-04-26T23:38:50.691431+0000","last_fullsized":"2024-04-26T23:38:50.691431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:23.924574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680231+0000","last_change":"2024-04-26T23:38:50.680231+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680231+0000","last_undegraded":"2024-04-26T23:38:50.680231+0000","last_fullsized":"2024-04-26T23:38:50.680231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696348+0000","last_change":"2024-04-26T23:38:50.693273+0000","last_active":"2024-04-26T23:38:50.696348+0000","last_peered":"2024-04-26T23:38:50.696348+0000","last_clean":"2024-04-26T23:38:50.696348+0000","last_became_active":"2024-04-26T23:38:50.692984+0000","last_became_peered":"2024-04-26T23:38:50.692984+0000","last_unstale":"2024-04-26T23:38:50.696348+0000","last_undegraded":"2024-04-26T23:38:50.696348+0000","last_fullsized":"2024-04-26T23:38:50.696348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682956+0000","last_change":"2024-04-26T23:38:50.682956+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682956+0000","last_undegraded":"2024-04-26T23:38:50.682956+0000","last_fullsized":"2024-04-26T23:38:50.682956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693238+0000","last_change":"2024-04-26T23:38:50.691632+0000","last_active":"2024-04-26T23:38:50.693238+0000","last_peered":"2024-04-26T23:38:50.693238+0000","last_clean":"2024-04-26T23:38:50.693238+0000","last_became_active":"2024-04-26T23:38:50.691073+0000","last_became_peered":"2024-04-26T23:38:50.691073+0000","last_unstale":"2024-04-26T23:38:50.693238+0000","last_undegraded":"2024-04-26T23:38:50.693238+0000","last_fullsized":"2024-04-26T23:38:50.693238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:05.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676918+0000","last_change":"2024-04-26T23:38:50.676918+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676918+0000","last_undegraded":"2024-04-26T23:38:50.676918+0000","last_fullsized":"2024-04-26T23:38:50.676918+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690054+0000","last_change":"2024-04-26T23:38:50.686345+0000","last_active":"2024-04-26T23:38:50.690054+0000","last_peered":"2024-04-26T23:38:50.690054+0000","last_clean":"2024-04-26T23:38:50.690054+0000","last_became_active":"2024-04-26T23:38:50.685727+0000","last_became_peered":"2024-04-26T23:38:50.685727+0000","last_unstale":"2024-04-26T23:38:50.690054+0000","last_undegraded":"2024-04-26T23:38:50.690054+0000","last_fullsized":"2024-04-26T23:38:50.690054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665923+0000","last_change":"2024-04-26T23:38:50.665923+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665923+0000","last_undegraded":"2024-04-26T23:38:50.665923+0000","last_fullsized":"2024-04-26T23:38:50.665923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690132+0000","last_change":"2024-04-26T23:38:50.688232+0000","last_active":"2024-04-26T23:38:50.690132+0000","last_peered":"2024-04-26T23:38:50.690132+0000","last_clean":"2024-04-26T23:38:50.690132+0000","last_became_active":"2024-04-26T23:38:50.687890+0000","last_became_peered":"2024-04-26T23:38:50.687890+0000","last_unstale":"2024-04-26T23:38:50.690132+0000","last_undegraded":"2024-04-26T23:38:50.690132+0000","last_fullsized":"2024-04-26T23:38:50.690132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677590+0000","last_change":"2024-04-26T23:38:50.677590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677590+0000","last_undegraded":"2024-04-26T23:38:50.677590+0000","last_fullsized":"2024-04-26T23:38:50.677590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692825+0000","last_change":"2024-04-26T23:38:50.688814+0000","last_active":"2024-04-26T23:38:50.692825+0000","last_peered":"2024-04-26T23:38:50.692825+0000","last_clean":"2024-04-26T23:38:50.692825+0000","last_became_active":"2024-04-26T23:38:50.688521+0000","last_became_peered":"2024-04-26T23:38:50.688521+0000","last_unstale":"2024-04-26T23:38:50.692825+0000","last_undegraded":"2024-04-26T23:38:50.692825+0000","last_fullsized":"2024-04-26T23:38:50.692825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:00.898736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670198+0000","last_change":"2024-04-26T23:38:50.670198+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670198+0000","last_undegraded":"2024-04-26T23:38:50.670198+0000","last_fullsized":"2024-04-26T23:38:50.670198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.682702+0000","last_change":"2024-04-26T23:38:50.678514+0000","last_active":"2024-04-26T23:38:50.682702+0000","last_peered":"2024-04-26T23:38:50.682702+0000","last_clean":"2024-04-26T23:38:50.682702+0000","last_became_active":"2024-04-26T23:38:50.678094+0000","last_became_peered":"2024-04-26T23:38:50.678094+0000","last_unstale":"2024-04-26T23:38:50.682702+0000","last_undegraded":"2024-04-26T23:38:50.682702+0000","last_fullsized":"2024-04-26T23:38:50.682702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:03.457234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680407+0000","last_change":"2024-04-26T23:38:50.680407+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680407+0000","last_undegraded":"2024-04-26T23:38:50.680407+0000","last_fullsized":"2024-04-26T23:38:50.680407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696274+0000","last_change":"2024-04-26T23:38:50.685335+0000","last_active":"2024-04-26T23:38:50.696274+0000","last_peered":"2024-04-26T23:38:50.696274+0000","last_clean":"2024-04-26T23:38:50.696274+0000","last_became_active":"2024-04-26T23:38:50.685055+0000","last_became_peered":"2024-04-26T23:38:50.685055+0000","last_unstale":"2024-04-26T23:38:50.696274+0000","last_undegraded":"2024-04-26T23:38:50.696274+0000","last_fullsized":"2024-04-26T23:38:50.696274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670944+0000","last_change":"2024-04-26T23:38:50.670944+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670944+0000","last_undegraded":"2024-04-26T23:38:50.670944+0000","last_fullsized":"2024-04-26T23:38:50.670944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697010+0000","last_change":"2024-04-26T23:38:50.694147+0000","last_active":"2024-04-26T23:38:50.697010+0000","last_peered":"2024-04-26T23:38:50.697010+0000","last_clean":"2024-04-26T23:38:50.697010+0000","last_became_active":"2024-04-26T23:38:50.692579+0000","last_became_peered":"2024-04-26T23:38:50.692579+0000","last_unstale":"2024-04-26T23:38:50.697010+0000","last_undegraded":"2024-04-26T23:38:50.697010+0000","last_fullsized":"2024-04-26T23:38:50.697010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.687375+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:38:50.687375+0000","last_peered":"2024-04-26T23:38:50.687375+0000","last_clean":"2024-04-26T23:38:50.687375+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:38:50.687375+0000","last_undegraded":"2024-04-26T23:38:50.687375+0000","last_fullsized":"2024-04-26T23:38:50.687375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":771208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:38:56.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:38:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.415+0000 7fdcb0a00640 1 Processor -- start 2024-04-26T23:38:56.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.416+0000 7fdcb0a00640 1 -- start start 2024-04-26T23:38:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcac1466b0 0x7fdcac06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac0720e0 0x7fdcac0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdcac160910 0x7fdcac160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdcac073ad0 con 0x7fdcac1466b0 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdcac146a90 con 0x7fdcac160910 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcb0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac073d70 con 0x7fdcac0720e0 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcaa800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcac1466b0 0x7fdcac06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcaa800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcac1466b0 0x7fdcac06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42550/0 (socket says 172.21.15.46:42550) 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.417+0000 7fdcaa800640 1 -- 172.21.15.46:0/3335935617 learned_addr learned my addr 172.21.15.46:0/3335935617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdca1e00640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac0720e0 0x7fdcac0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcab200640 1 --2- 172.21.15.46:0/3335935617 >> v2:172.21.15.142:3300/0 conn(0x7fdcac160910 0x7fdcac160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcaa800640 1 -- 172.21.15.46:0/3335935617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac0720e0 msgr2=0x7fdcac0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcaa800640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac0720e0 0x7fdcac0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcaa800640 1 -- 172.21.15.46:0/3335935617 >> v2:172.21.15.142:3300/0 conn(0x7fdcac160910 msgr2=0x7fdcac160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcaa800640 1 --2- 172.21.15.46:0/3335935617 >> v2:172.21.15.142:3300/0 conn(0x7fdcac160910 0x7fdcac160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.418+0000 7fdcaa800640 1 -- 172.21.15.46:0/3335935617 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac074050 con 0x7fdcac1466b0 2024-04-26T23:38:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.420+0000 7fdcaa800640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac1466b0 0x7fdcac06de80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0cc920 tx=0x7fdc9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dd80796ac8175c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.420+0000 7fdca9e00640 1 -- 172.21.15.46:0/3335935617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c037070 con 0x7fdcac1466b0 2024-04-26T23:38:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.420+0000 7fdca9e00640 1 -- 172.21.15.46:0/3335935617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c032440 con 0x7fdcac1466b0 2024-04-26T23:38:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.420+0000 7fdca9e00640 1 -- 172.21.15.46:0/3335935617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c032810 con 0x7fdcac1466b0 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 -- 172.21.15.46:0/3335935617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac1466b0 msgr2=0x7fdcac06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac1466b0 0x7fdcac06de80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0cc920 tx=0x7fdc9c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 -- 172.21.15.46:0/3335935617 shutdown_connections 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/3335935617 >> v2:172.21.15.142:3300/0 conn(0x7fdcac160910 0x7fdcac160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac0720e0 0x7fdcac0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/3335935617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac1466b0 0x7fdcac06de80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 -- 172.21.15.46:0/3335935617 >> 172.21.15.46:0/3335935617 conn(0x7fdcac0b26d0 msgr2=0x7fdcac0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 -- 172.21.15.46:0/3335935617 shutdown_connections 2024-04-26T23:38:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.421+0000 7fdcb0a00640 1 -- 172.21.15.46:0/3335935617 wait complete. 2024-04-26T23:38:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.423+0000 7fdcb0a00640 1 Processor -- start 2024-04-26T23:38:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.423+0000 7fdcb0a00640 1 -- start start 2024-04-26T23:38:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac1466b0 0x7fdcac0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcac160910 0x7fdcac0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06e3c0 con 0x7fdcac0720e0 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac0cb470 con 0x7fdcac160910 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcb0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac0cdda0 con 0x7fdcac1466b0 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcaa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcaa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42566/0 (socket says 172.21.15.46:42566) 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcaa800640 1 -- 172.21.15.46:0/1418046562 learned_addr learned my addr 172.21.15.46:0/1418046562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcaa800640 1 -- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac1466b0 msgr2=0x7fdcac0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdca1e00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac1466b0 0x7fdcac0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.424+0000 7fdcaa800640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac1466b0 0x7fdcac0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcaa800640 1 -- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcac160910 msgr2=0x7fdcac0b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcaa800640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcac160910 0x7fdcac0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcaa800640 1 -- 172.21.15.46:0/1418046562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac15ebc0 con 0x7fdcac0720e0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcaa800640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c02a220 tx=0x7fdc9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c041070 con 0x7fdcac0720e0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c03c870 con 0x7fdcac0720e0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c037040 con 0x7fdcac0720e0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac15ee60 con 0x7fdcac0720e0 2024-04-26T23:38:56.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.425+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac15f420 con 0x7fdcac0720e0 2024-04-26T23:38:56.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.427+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac0693b0 con 0x7fdcac0720e0 2024-04-26T23:38:56.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.428+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc9c044070 con 0x7fdcac0720e0 2024-04-26T23:38:56.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.429+0000 7fdca3400640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 0x7fdc8806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.429+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdc9c0ac930 con 0x7fdcac0720e0 2024-04-26T23:38:56.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.429+0000 7fdca1e00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 0x7fdc8806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.431+0000 7fdca1e00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 0x7fdc8806ee30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006fd0 tx=0x7fdc94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.433+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc9c07acd0 con 0x7fdcac0720e0 2024-04-26T23:38:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.632+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcac0c7180 con 0x7fdc8806c970 2024-04-26T23:38:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.651+0000 7fdca3400640 1 -- 172.21.15.46:0/1418046562 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fdcac0c7180 con 0x7fdc8806c970 2024-04-26T23:38:56.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:38:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 msgr2=0x7fdc8806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 0x7fdc8806ee30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006fd0 tx=0x7fdc94008040 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 msgr2=0x7fdcac138dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c02a220 tx=0x7fdc9c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 shutdown_connections 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc8806c970 0x7fdc8806ee30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcac160910 0x7fdcac0b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcac1466b0 0x7fdcac0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 --2- 172.21.15.46:0/1418046562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcac0720e0 0x7fdcac138dd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 >> 172.21.15.46:0/1418046562 conn(0x7fdcac0b26d0 msgr2=0x7fdcac0cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 shutdown_connections 2024-04-26T23:38:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.657+0000 7fdcb0a00640 1 -- 172.21.15.46:0/1418046562 wait complete. 2024-04-26T23:38:56.679 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-26T23:38:55.922008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233088,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741632,"statfs":{"total":773094113280,"available":772855431168,"internally_reserved":0,"allocated":4546560,"data_stored":2600788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001900"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677871+0000","last_change":"2024-04-26T23:38:50.677871+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677871+0000","last_undegraded":"2024-04-26T23:38:50.677871+0000","last_fullsized":"2024-04-26T23:38:50.677871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680384+0000","last_change":"2024-04-26T23:38:50.680384+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680384+0000","last_undegraded":"2024-04-26T23:38:50.680384+0000","last_fullsized":"2024-04-26T23:38:50.680384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669294+0000","last_change":"2024-04-26T23:38:50.669294+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669294+0000","last_undegraded":"2024-04-26T23:38:50.669294+0000","last_fullsized":"2024-04-26T23:38:50.669294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.675317+0000","last_change":"2024-04-26T23:38:50.675317+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.675317+0000","last_undegraded":"2024-04-26T23:38:50.675317+0000","last_fullsized":"2024-04-26T23:38:50.675317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672056+0000","last_change":"2024-04-26T23:38:50.672056+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672056+0000","last_undegraded":"2024-04-26T23:38:50.672056+0000","last_fullsized":"2024-04-26T23:38:50.672056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677312+0000","last_change":"2024-04-26T23:38:50.677312+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677312+0000","last_undegraded":"2024-04-26T23:38:50.677312+0000","last_fullsized":"2024-04-26T23:38:50.677312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667316+0000","last_change":"2024-04-26T23:38:50.667316+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667316+0000","last_undegraded":"2024-04-26T23:38:50.667316+0000","last_fullsized":"2024-04-26T23:38:50.667316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672414+0000","last_change":"2024-04-26T23:38:50.672414+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672414+0000","last_undegraded":"2024-04-26T23:38:50.672414+0000","last_fullsized":"2024-04-26T23:38:50.672414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667307+0000","last_change":"2024-04-26T23:38:50.667307+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667307+0000","last_undegraded":"2024-04-26T23:38:50.667307+0000","last_fullsized":"2024-04-26T23:38:50.667307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669299+0000","last_change":"2024-04-26T23:38:50.669299+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669299+0000","last_undegraded":"2024-04-26T23:38:50.669299+0000","last_fullsized":"2024-04-26T23:38:50.669299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674349+0000","last_change":"2024-04-26T23:38:50.674349+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674349+0000","last_undegraded":"2024-04-26T23:38:50.674349+0000","last_fullsized":"2024-04-26T23:38:50.674349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667276+0000","last_change":"2024-04-26T23:38:50.667276+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667276+0000","last_undegraded":"2024-04-26T23:38:50.667276+0000","last_fullsized":"2024-04-26T23:38:50.667276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672590+0000","last_change":"2024-04-26T23:38:50.672590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672590+0000","last_undegraded":"2024-04-26T23:38:50.672590+0000","last_fullsized":"2024-04-26T23:38:50.672590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671236+0000","last_change":"2024-04-26T23:38:50.671236+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671236+0000","last_undegraded":"2024-04-26T23:38:50.671236+0000","last_fullsized":"2024-04-26T23:38:50.671236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666175+0000","last_change":"2024-04-26T23:38:50.666175+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666175+0000","last_undegraded":"2024-04-26T23:38:50.666175+0000","last_fullsized":"2024-04-26T23:38:50.666175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676905+0000","last_change":"2024-04-26T23:38:50.676905+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676905+0000","last_undegraded":"2024-04-26T23:38:50.676905+0000","last_fullsized":"2024-04-26T23:38:50.676905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.685115+0000","last_change":"2024-04-26T23:38:50.685115+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.685115+0000","last_undegraded":"2024-04-26T23:38:50.685115+0000","last_fullsized":"2024-04-26T23:38:50.685115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676567+0000","last_change":"2024-04-26T23:38:50.676567+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676567+0000","last_undegraded":"2024-04-26T23:38:50.676567+0000","last_fullsized":"2024-04-26T23:38:50.676567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673936+0000","last_change":"2024-04-26T23:38:50.673936+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673936+0000","last_undegraded":"2024-04-26T23:38:50.673936+0000","last_fullsized":"2024-04-26T23:38:50.673936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677945+0000","last_change":"2024-04-26T23:38:50.677945+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677945+0000","last_undegraded":"2024-04-26T23:38:50.677945+0000","last_fullsized":"2024-04-26T23:38:50.677945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671386+0000","last_change":"2024-04-26T23:38:50.671386+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671386+0000","last_undegraded":"2024-04-26T23:38:50.671386+0000","last_fullsized":"2024-04-26T23:38:50.671386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669210+0000","last_change":"2024-04-26T23:38:50.669210+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669210+0000","last_undegraded":"2024-04-26T23:38:50.669210+0000","last_fullsized":"2024-04-26T23:38:50.669210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679803+0000","last_change":"2024-04-26T23:38:50.679803+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679803+0000","last_undegraded":"2024-04-26T23:38:50.679803+0000","last_fullsized":"2024-04-26T23:38:50.679803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674451+0000","last_change":"2024-04-26T23:38:50.674451+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674451+0000","last_undegraded":"2024-04-26T23:38:50.674451+0000","last_fullsized":"2024-04-26T23:38:50.674451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677442+0000","last_change":"2024-04-26T23:38:50.677442+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677442+0000","last_undegraded":"2024-04-26T23:38:50.677442+0000","last_fullsized":"2024-04-26T23:38:50.677442+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678807+0000","last_change":"2024-04-26T23:38:50.678807+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678807+0000","last_undegraded":"2024-04-26T23:38:50.678807+0000","last_fullsized":"2024-04-26T23:38:50.678807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676493+0000","last_change":"2024-04-26T23:38:50.676493+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676493+0000","last_undegraded":"2024-04-26T23:38:50.676493+0000","last_fullsized":"2024-04-26T23:38:50.676493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673334+0000","last_change":"2024-04-26T23:38:50.673334+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673334+0000","last_undegraded":"2024-04-26T23:38:50.673334+0000","last_fullsized":"2024-04-26T23:38:50.673334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678505+0000","last_change":"2024-04-26T23:38:50.678505+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678505+0000","last_undegraded":"2024-04-26T23:38:50.678505+0000","last_fullsized":"2024-04-26T23:38:50.678505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676624+0000","last_change":"2024-04-26T23:38:50.676624+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676624+0000","last_undegraded":"2024-04-26T23:38:50.676624+0000","last_fullsized":"2024-04-26T23:38:50.676624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679221+0000","last_change":"2024-04-26T23:38:50.679221+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679221+0000","last_undegraded":"2024-04-26T23:38:50.679221+0000","last_fullsized":"2024-04-26T23:38:50.679221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676259+0000","last_change":"2024-04-26T23:38:50.676259+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676259+0000","last_undegraded":"2024-04-26T23:38:50.676259+0000","last_fullsized":"2024-04-26T23:38:50.676259+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682341+0000","last_change":"2024-04-26T23:38:50.682341+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682341+0000","last_undegraded":"2024-04-26T23:38:50.682341+0000","last_fullsized":"2024-04-26T23:38:50.682341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694038+0000","last_change":"2024-04-26T23:38:50.691324+0000","last_active":"2024-04-26T23:38:50.694038+0000","last_peered":"2024-04-26T23:38:50.694038+0000","last_clean":"2024-04-26T23:38:50.694038+0000","last_became_active":"2024-04-26T23:38:50.691023+0000","last_became_peered":"2024-04-26T23:38:50.691023+0000","last_unstale":"2024-04-26T23:38:50.694038+0000","last_undegraded":"2024-04-26T23:38:50.694038+0000","last_fullsized":"2024-04-26T23:38:50.694038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:55.670095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678183+0000","last_change":"2024-04-26T23:38:50.678183+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678183+0000","last_undegraded":"2024-04-26T23:38:50.678183+0000","last_fullsized":"2024-04-26T23:38:50.678183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693660+0000","last_change":"2024-04-26T23:38:50.691001+0000","last_active":"2024-04-26T23:38:50.693660+0000","last_peered":"2024-04-26T23:38:50.693660+0000","last_clean":"2024-04-26T23:38:50.693660+0000","last_became_active":"2024-04-26T23:38:50.690668+0000","last_became_peered":"2024-04-26T23:38:50.690668+0000","last_unstale":"2024-04-26T23:38:50.693660+0000","last_undegraded":"2024-04-26T23:38:50.693660+0000","last_fullsized":"2024-04-26T23:38:50.693660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665322+0000","last_change":"2024-04-26T23:38:50.665322+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665322+0000","last_undegraded":"2024-04-26T23:38:50.665322+0000","last_fullsized":"2024-04-26T23:38:50.665322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694003+0000","last_change":"2024-04-26T23:38:50.689919+0000","last_active":"2024-04-26T23:38:50.694003+0000","last_peered":"2024-04-26T23:38:50.694003+0000","last_clean":"2024-04-26T23:38:50.694003+0000","last_became_active":"2024-04-26T23:38:50.689629+0000","last_became_peered":"2024-04-26T23:38:50.689629+0000","last_unstale":"2024-04-26T23:38:50.694003+0000","last_undegraded":"2024-04-26T23:38:50.694003+0000","last_fullsized":"2024-04-26T23:38:50.694003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.753794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678342+0000","last_change":"2024-04-26T23:38:50.678342+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678342+0000","last_undegraded":"2024-04-26T23:38:50.678342+0000","last_fullsized":"2024-04-26T23:38:50.678342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690252+0000","last_change":"2024-04-26T23:38:50.684308+0000","last_active":"2024-04-26T23:38:50.690252+0000","last_peered":"2024-04-26T23:38:50.690252+0000","last_clean":"2024-04-26T23:38:50.690252+0000","last_became_active":"2024-04-26T23:38:50.684017+0000","last_became_peered":"2024-04-26T23:38:50.684017+0000","last_unstale":"2024-04-26T23:38:50.690252+0000","last_undegraded":"2024-04-26T23:38:50.690252+0000","last_fullsized":"2024-04-26T23:38:50.690252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.549403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667266+0000","last_change":"2024-04-26T23:38:50.667266+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667266+0000","last_undegraded":"2024-04-26T23:38:50.667266+0000","last_fullsized":"2024-04-26T23:38:50.667266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693328+0000","last_change":"2024-04-26T23:38:50.687042+0000","last_active":"2024-04-26T23:38:50.693328+0000","last_peered":"2024-04-26T23:38:50.693328+0000","last_clean":"2024-04-26T23:38:50.693328+0000","last_became_active":"2024-04-26T23:38:50.685547+0000","last_became_peered":"2024-04-26T23:38:50.685547+0000","last_unstale":"2024-04-26T23:38:50.693328+0000","last_undegraded":"2024-04-26T23:38:50.693328+0000","last_fullsized":"2024-04-26T23:38:50.693328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.664806+0000","last_change":"2024-04-26T23:38:50.664806+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.664806+0000","last_undegraded":"2024-04-26T23:38:50.664806+0000","last_fullsized":"2024-04-26T23:38:50.664806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697047+0000","last_change":"2024-04-26T23:38:50.695028+0000","last_active":"2024-04-26T23:38:50.697047+0000","last_peered":"2024-04-26T23:38:50.697047+0000","last_clean":"2024-04-26T23:38:50.697047+0000","last_became_active":"2024-04-26T23:38:50.694628+0000","last_became_peered":"2024-04-26T23:38:50.694628+0000","last_unstale":"2024-04-26T23:38:50.697047+0000","last_undegraded":"2024-04-26T23:38:50.697047+0000","last_fullsized":"2024-04-26T23:38:50.697047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667992+0000","last_change":"2024-04-26T23:38:50.667992+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667992+0000","last_undegraded":"2024-04-26T23:38:50.667992+0000","last_fullsized":"2024-04-26T23:38:50.667992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699401+0000","last_change":"2024-04-26T23:38:50.698125+0000","last_active":"2024-04-26T23:38:50.699401+0000","last_peered":"2024-04-26T23:38:50.699401+0000","last_clean":"2024-04-26T23:38:50.699401+0000","last_became_active":"2024-04-26T23:38:50.697795+0000","last_became_peered":"2024-04-26T23:38:50.697795+0000","last_unstale":"2024-04-26T23:38:50.699401+0000","last_undegraded":"2024-04-26T23:38:50.699401+0000","last_fullsized":"2024-04-26T23:38:50.699401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673039+0000","last_change":"2024-04-26T23:38:50.673039+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673039+0000","last_undegraded":"2024-04-26T23:38:50.673039+0000","last_fullsized":"2024-04-26T23:38:50.673039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689406+0000","last_change":"2024-04-26T23:38:50.680954+0000","last_active":"2024-04-26T23:38:50.689406+0000","last_peered":"2024-04-26T23:38:50.689406+0000","last_clean":"2024-04-26T23:38:50.689406+0000","last_became_active":"2024-04-26T23:38:50.680546+0000","last_became_peered":"2024-04-26T23:38:50.680546+0000","last_unstale":"2024-04-26T23:38:50.689406+0000","last_undegraded":"2024-04-26T23:38:50.689406+0000","last_fullsized":"2024-04-26T23:38:50.689406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:51.613896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674846+0000","last_change":"2024-04-26T23:38:50.674846+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674846+0000","last_undegraded":"2024-04-26T23:38:50.674846+0000","last_fullsized":"2024-04-26T23:38:50.674846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696898+0000","last_change":"2024-04-26T23:38:50.694593+0000","last_active":"2024-04-26T23:38:50.696898+0000","last_peered":"2024-04-26T23:38:50.696898+0000","last_clean":"2024-04-26T23:38:50.696898+0000","last_became_active":"2024-04-26T23:38:50.694289+0000","last_became_peered":"2024-04-26T23:38:50.694289+0000","last_unstale":"2024-04-26T23:38:50.696898+0000","last_undegraded":"2024-04-26T23:38:50.696898+0000","last_fullsized":"2024-04-26T23:38:50.696898+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.683436+0000","last_change":"2024-04-26T23:38:50.683436+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.683436+0000","last_undegraded":"2024-04-26T23:38:50.683436+0000","last_fullsized":"2024-04-26T23:38:50.683436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.701161+0000","last_change":"2024-04-26T23:38:50.696626+0000","last_active":"2024-04-26T23:38:50.701161+0000","last_peered":"2024-04-26T23:38:50.701161+0000","last_clean":"2024-04-26T23:38:50.701161+0000","last_became_active":"2024-04-26T23:38:50.696142+0000","last_became_peered":"2024-04-26T23:38:50.696142+0000","last_unstale":"2024-04-26T23:38:50.701161+0000","last_undegraded":"2024-04-26T23:38:50.701161+0000","last_fullsized":"2024-04-26T23:38:50.701161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666542+0000","last_change":"2024-04-26T23:38:50.666542+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666542+0000","last_undegraded":"2024-04-26T23:38:50.666542+0000","last_fullsized":"2024-04-26T23:38:50.666542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.698415+0000","last_change":"2024-04-26T23:38:50.695193+0000","last_active":"2024-04-26T23:38:50.698415+0000","last_peered":"2024-04-26T23:38:50.698415+0000","last_clean":"2024-04-26T23:38:50.698415+0000","last_became_active":"2024-04-26T23:38:50.694850+0000","last_became_peered":"2024-04-26T23:38:50.694850+0000","last_unstale":"2024-04-26T23:38:50.698415+0000","last_undegraded":"2024-04-26T23:38:50.698415+0000","last_fullsized":"2024-04-26T23:38:50.698415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.668261+0000","last_change":"2024-04-26T23:38:50.668261+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.668261+0000","last_undegraded":"2024-04-26T23:38:50.668261+0000","last_fullsized":"2024-04-26T23:38:50.668261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697007+0000","last_change":"2024-04-26T23:38:50.695400+0000","last_active":"2024-04-26T23:38:50.697007+0000","last_peered":"2024-04-26T23:38:50.697007+0000","last_clean":"2024-04-26T23:38:50.697007+0000","last_became_active":"2024-04-26T23:38:50.695178+0000","last_became_peered":"2024-04-26T23:38:50.695178+0000","last_unstale":"2024-04-26T23:38:50.697007+0000","last_undegraded":"2024-04-26T23:38:50.697007+0000","last_fullsized":"2024-04-26T23:38:50.697007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:22.297685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666548+0000","last_change":"2024-04-26T23:38:50.666548+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666548+0000","last_undegraded":"2024-04-26T23:38:50.666548+0000","last_fullsized":"2024-04-26T23:38:50.666548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696932+0000","last_change":"2024-04-26T23:38:50.695534+0000","last_active":"2024-04-26T23:38:50.696932+0000","last_peered":"2024-04-26T23:38:50.696932+0000","last_clean":"2024-04-26T23:38:50.696932+0000","last_became_active":"2024-04-26T23:38:50.686496+0000","last_became_peered":"2024-04-26T23:38:50.686496+0000","last_unstale":"2024-04-26T23:38:50.696932+0000","last_undegraded":"2024-04-26T23:38:50.696932+0000","last_fullsized":"2024-04-26T23:38:50.696932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.662014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672274+0000","last_change":"2024-04-26T23:38:50.672274+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672274+0000","last_undegraded":"2024-04-26T23:38:50.672274+0000","last_fullsized":"2024-04-26T23:38:50.672274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699742+0000","last_change":"2024-04-26T23:38:50.698660+0000","last_active":"2024-04-26T23:38:50.699742+0000","last_peered":"2024-04-26T23:38:50.699742+0000","last_clean":"2024-04-26T23:38:50.699742+0000","last_became_active":"2024-04-26T23:38:50.698295+0000","last_became_peered":"2024-04-26T23:38:50.698295+0000","last_unstale":"2024-04-26T23:38:50.699742+0000","last_undegraded":"2024-04-26T23:38:50.699742+0000","last_fullsized":"2024-04-26T23:38:50.699742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679622+0000","last_change":"2024-04-26T23:38:50.679622+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679622+0000","last_undegraded":"2024-04-26T23:38:50.679622+0000","last_fullsized":"2024-04-26T23:38:50.679622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689132+0000","last_change":"2024-04-26T23:38:50.686322+0000","last_active":"2024-04-26T23:38:50.689132+0000","last_peered":"2024-04-26T23:38:50.689132+0000","last_clean":"2024-04-26T23:38:50.689132+0000","last_became_active":"2024-04-26T23:38:50.684566+0000","last_became_peered":"2024-04-26T23:38:50.684566+0000","last_unstale":"2024-04-26T23:38:50.689132+0000","last_undegraded":"2024-04-26T23:38:50.689132+0000","last_fullsized":"2024-04-26T23:38:50.689132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:39.251641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672490+0000","last_change":"2024-04-26T23:38:50.672490+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672490+0000","last_undegraded":"2024-04-26T23:38:50.672490+0000","last_fullsized":"2024-04-26T23:38:50.672490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697031+0000","last_change":"2024-04-26T23:38:50.694214+0000","last_active":"2024-04-26T23:38:50.697031+0000","last_peered":"2024-04-26T23:38:50.697031+0000","last_clean":"2024-04-26T23:38:50.697031+0000","last_became_active":"2024-04-26T23:38:50.693909+0000","last_became_peered":"2024-04-26T23:38:50.693909+0000","last_unstale":"2024-04-26T23:38:50.697031+0000","last_undegraded":"2024-04-26T23:38:50.697031+0000","last_fullsized":"2024-04-26T23:38:50.697031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679586+0000","last_change":"2024-04-26T23:38:50.679586+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679586+0000","last_undegraded":"2024-04-26T23:38:50.679586+0000","last_fullsized":"2024-04-26T23:38:50.679586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692161+0000","last_change":"2024-04-26T23:38:50.686368+0000","last_active":"2024-04-26T23:38:50.692161+0000","last_peered":"2024-04-26T23:38:50.692161+0000","last_clean":"2024-04-26T23:38:50.692161+0000","last_became_active":"2024-04-26T23:38:50.686085+0000","last_became_peered":"2024-04-26T23:38:50.686085+0000","last_unstale":"2024-04-26T23:38:50.692161+0000","last_undegraded":"2024-04-26T23:38:50.692161+0000","last_fullsized":"2024-04-26T23:38:50.692161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671636+0000","last_change":"2024-04-26T23:38:50.671636+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671636+0000","last_undegraded":"2024-04-26T23:38:50.671636+0000","last_fullsized":"2024-04-26T23:38:50.671636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690421+0000","last_change":"2024-04-26T23:38:50.685721+0000","last_active":"2024-04-26T23:38:50.690421+0000","last_peered":"2024-04-26T23:38:50.690421+0000","last_clean":"2024-04-26T23:38:50.690421+0000","last_became_active":"2024-04-26T23:38:50.685381+0000","last_became_peered":"2024-04-26T23:38:50.685381+0000","last_unstale":"2024-04-26T23:38:50.690421+0000","last_undegraded":"2024-04-26T23:38:50.690421+0000","last_fullsized":"2024-04-26T23:38:50.690421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:05.204342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666638+0000","last_change":"2024-04-26T23:38:50.666638+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666638+0000","last_undegraded":"2024-04-26T23:38:50.666638+0000","last_fullsized":"2024-04-26T23:38:50.666638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692041+0000","last_change":"2024-04-26T23:38:50.690198+0000","last_active":"2024-04-26T23:38:50.692041+0000","last_peered":"2024-04-26T23:38:50.692041+0000","last_clean":"2024-04-26T23:38:50.692041+0000","last_became_active":"2024-04-26T23:38:50.689613+0000","last_became_peered":"2024-04-26T23:38:50.689613+0000","last_unstale":"2024-04-26T23:38:50.692041+0000","last_undegraded":"2024-04-26T23:38:50.692041+0000","last_fullsized":"2024-04-26T23:38:50.692041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:57:08.503076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671249+0000","last_change":"2024-04-26T23:38:50.671249+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671249+0000","last_undegraded":"2024-04-26T23:38:50.671249+0000","last_fullsized":"2024-04-26T23:38:50.671249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693306+0000","last_change":"2024-04-26T23:38:50.691793+0000","last_active":"2024-04-26T23:38:50.693306+0000","last_peered":"2024-04-26T23:38:50.693306+0000","last_clean":"2024-04-26T23:38:50.693306+0000","last_became_active":"2024-04-26T23:38:50.691492+0000","last_became_peered":"2024-04-26T23:38:50.691492+0000","last_unstale":"2024-04-26T23:38:50.693306+0000","last_undegraded":"2024-04-26T23:38:50.693306+0000","last_fullsized":"2024-04-26T23:38:50.693306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676346+0000","last_change":"2024-04-26T23:38:50.676346+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676346+0000","last_undegraded":"2024-04-26T23:38:50.676346+0000","last_fullsized":"2024-04-26T23:38:50.676346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690651+0000","last_change":"2024-04-26T23:38:50.687713+0000","last_active":"2024-04-26T23:38:50.690651+0000","last_peered":"2024-04-26T23:38:50.690651+0000","last_clean":"2024-04-26T23:38:50.690651+0000","last_became_active":"2024-04-26T23:38:50.687314+0000","last_became_peered":"2024-04-26T23:38:50.687314+0000","last_unstale":"2024-04-26T23:38:50.690651+0000","last_undegraded":"2024-04-26T23:38:50.690651+0000","last_fullsized":"2024-04-26T23:38:50.690651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:38.872504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665851+0000","last_change":"2024-04-26T23:38:50.665851+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665851+0000","last_undegraded":"2024-04-26T23:38:50.665851+0000","last_fullsized":"2024-04-26T23:38:50.665851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.684746+0000","last_change":"2024-04-26T23:38:50.682745+0000","last_active":"2024-04-26T23:38:50.684746+0000","last_peered":"2024-04-26T23:38:50.684746+0000","last_clean":"2024-04-26T23:38:50.684746+0000","last_became_active":"2024-04-26T23:38:50.682462+0000","last_became_peered":"2024-04-26T23:38:50.682462+0000","last_unstale":"2024-04-26T23:38:50.684746+0000","last_undegraded":"2024-04-26T23:38:50.684746+0000","last_fullsized":"2024-04-26T23:38:50.684746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:19.185932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670016+0000","last_change":"2024-04-26T23:38:50.670016+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670016+0000","last_undegraded":"2024-04-26T23:38:50.670016+0000","last_fullsized":"2024-04-26T23:38:50.670016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.700135+0000","last_change":"2024-04-26T23:38:50.699004+0000","last_active":"2024-04-26T23:38:50.700135+0000","last_peered":"2024-04-26T23:38:50.700135+0000","last_clean":"2024-04-26T23:38:50.700135+0000","last_became_active":"2024-04-26T23:38:50.698643+0000","last_became_peered":"2024-04-26T23:38:50.698643+0000","last_unstale":"2024-04-26T23:38:50.700135+0000","last_undegraded":"2024-04-26T23:38:50.700135+0000","last_fullsized":"2024-04-26T23:38:50.700135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671845+0000","last_change":"2024-04-26T23:38:50.671845+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671845+0000","last_undegraded":"2024-04-26T23:38:50.671845+0000","last_fullsized":"2024-04-26T23:38:50.671845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.691431+0000","last_change":"2024-04-26T23:38:50.688989+0000","last_active":"2024-04-26T23:38:50.691431+0000","last_peered":"2024-04-26T23:38:50.691431+0000","last_clean":"2024-04-26T23:38:50.691431+0000","last_became_active":"2024-04-26T23:38:50.688711+0000","last_became_peered":"2024-04-26T23:38:50.688711+0000","last_unstale":"2024-04-26T23:38:50.691431+0000","last_undegraded":"2024-04-26T23:38:50.691431+0000","last_fullsized":"2024-04-26T23:38:50.691431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:23.924574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680231+0000","last_change":"2024-04-26T23:38:50.680231+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680231+0000","last_undegraded":"2024-04-26T23:38:50.680231+0000","last_fullsized":"2024-04-26T23:38:50.680231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696348+0000","last_change":"2024-04-26T23:38:50.693273+0000","last_active":"2024-04-26T23:38:50.696348+0000","last_peered":"2024-04-26T23:38:50.696348+0000","last_clean":"2024-04-26T23:38:50.696348+0000","last_became_active":"2024-04-26T23:38:50.692984+0000","last_became_peered":"2024-04-26T23:38:50.692984+0000","last_unstale":"2024-04-26T23:38:50.696348+0000","last_undegraded":"2024-04-26T23:38:50.696348+0000","last_fullsized":"2024-04-26T23:38:50.696348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682956+0000","last_change":"2024-04-26T23:38:50.682956+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682956+0000","last_undegraded":"2024-04-26T23:38:50.682956+0000","last_fullsized":"2024-04-26T23:38:50.682956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693238+0000","last_change":"2024-04-26T23:38:50.691632+0000","last_active":"2024-04-26T23:38:50.693238+0000","last_peered":"2024-04-26T23:38:50.693238+0000","last_clean":"2024-04-26T23:38:50.693238+0000","last_became_active":"2024-04-26T23:38:50.691073+0000","last_became_peered":"2024-04-26T23:38:50.691073+0000","last_unstale":"2024-04-26T23:38:50.693238+0000","last_undegraded":"2024-04-26T23:38:50.693238+0000","last_fullsized":"2024-04-26T23:38:50.693238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:05.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676918+0000","last_change":"2024-04-26T23:38:50.676918+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676918+0000","last_undegraded":"2024-04-26T23:38:50.676918+0000","last_fullsized":"2024-04-26T23:38:50.676918+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690054+0000","last_change":"2024-04-26T23:38:50.686345+0000","last_active":"2024-04-26T23:38:50.690054+0000","last_peered":"2024-04-26T23:38:50.690054+0000","last_clean":"2024-04-26T23:38:50.690054+0000","last_became_active":"2024-04-26T23:38:50.685727+0000","last_became_peered":"2024-04-26T23:38:50.685727+0000","last_unstale":"2024-04-26T23:38:50.690054+0000","last_undegraded":"2024-04-26T23:38:50.690054+0000","last_fullsized":"2024-04-26T23:38:50.690054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665923+0000","last_change":"2024-04-26T23:38:50.665923+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665923+0000","last_undegraded":"2024-04-26T23:38:50.665923+0000","last_fullsized":"2024-04-26T23:38:50.665923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690132+0000","last_change":"2024-04-26T23:38:50.688232+0000","last_active":"2024-04-26T23:38:50.690132+0000","last_peered":"2024-04-26T23:38:50.690132+0000","last_clean":"2024-04-26T23:38:50.690132+0000","last_became_active":"2024-04-26T23:38:50.687890+0000","last_became_peered":"2024-04-26T23:38:50.687890+0000","last_unstale":"2024-04-26T23:38:50.690132+0000","last_undegraded":"2024-04-26T23:38:50.690132+0000","last_fullsized":"2024-04-26T23:38:50.690132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677590+0000","last_change":"2024-04-26T23:38:50.677590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677590+0000","last_undegraded":"2024-04-26T23:38:50.677590+0000","last_fullsized":"2024-04-26T23:38:50.677590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692825+0000","last_change":"2024-04-26T23:38:50.688814+0000","last_active":"2024-04-26T23:38:50.692825+0000","last_peered":"2024-04-26T23:38:50.692825+0000","last_clean":"2024-04-26T23:38:50.692825+0000","last_became_active":"2024-04-26T23:38:50.688521+0000","last_became_peered":"2024-04-26T23:38:50.688521+0000","last_unstale":"2024-04-26T23:38:50.692825+0000","last_undegraded":"2024-04-26T23:38:50.692825+0000","last_fullsized":"2024-04-26T23:38:50.692825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:00.898736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670198+0000","last_change":"2024-04-26T23:38:50.670198+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670198+0000","last_undegraded":"2024-04-26T23:38:50.670198+0000","last_fullsized":"2024-04-26T23:38:50.670198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.682702+0000","last_change":"2024-04-26T23:38:50.678514+0000","last_active":"2024-04-26T23:38:50.682702+0000","last_peered":"2024-04-26T23:38:50.682702+0000","last_clean":"2024-04-26T23:38:50.682702+0000","last_became_active":"2024-04-26T23:38:50.678094+0000","last_became_peered":"2024-04-26T23:38:50.678094+0000","last_unstale":"2024-04-26T23:38:50.682702+0000","last_undegraded":"2024-04-26T23:38:50.682702+0000","last_fullsized":"2024-04-26T23:38:50.682702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:03.457234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680407+0000","last_change":"2024-04-26T23:38:50.680407+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680407+0000","last_undegraded":"2024-04-26T23:38:50.680407+0000","last_fullsized":"2024-04-26T23:38:50.680407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696274+0000","last_change":"2024-04-26T23:38:50.685335+0000","last_active":"2024-04-26T23:38:50.696274+0000","last_peered":"2024-04-26T23:38:50.696274+0000","last_clean":"2024-04-26T23:38:50.696274+0000","last_became_active":"2024-04-26T23:38:50.685055+0000","last_became_peered":"2024-04-26T23:38:50.685055+0000","last_unstale":"2024-04-26T23:38:50.696274+0000","last_undegraded":"2024-04-26T23:38:50.696274+0000","last_fullsized":"2024-04-26T23:38:50.696274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670944+0000","last_change":"2024-04-26T23:38:50.670944+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670944+0000","last_undegraded":"2024-04-26T23:38:50.670944+0000","last_fullsized":"2024-04-26T23:38:50.670944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697010+0000","last_change":"2024-04-26T23:38:50.694147+0000","last_active":"2024-04-26T23:38:50.697010+0000","last_peered":"2024-04-26T23:38:50.697010+0000","last_clean":"2024-04-26T23:38:50.697010+0000","last_became_active":"2024-04-26T23:38:50.692579+0000","last_became_peered":"2024-04-26T23:38:50.692579+0000","last_unstale":"2024-04-26T23:38:50.697010+0000","last_undegraded":"2024-04-26T23:38:50.697010+0000","last_fullsized":"2024-04-26T23:38:50.697010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.687375+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:38:50.687375+0000","last_peered":"2024-04-26T23:38:50.687375+0000","last_clean":"2024-04-26T23:38:50.687375+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:38:50.687375+0000","last_undegraded":"2024-04-26T23:38:50.687375+0000","last_fullsized":"2024-04-26T23:38:50.687375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":771208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:38:56.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:38:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.933+0000 7f783e200640 1 Processor -- start 2024-04-26T23:38:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.933+0000 7f783e200640 1 -- start start 2024-04-26T23:38:56.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.934+0000 7f783e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.934+0000 7f783e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7838160bd0 0x7f7838161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.934+0000 7f783e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783806ee30 con 0x7f783815d540 2024-04-26T23:38:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.934+0000 7f783e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f78380742d0 con 0x7f7838160bd0 2024-04-26T23:38:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.935+0000 7f783e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7838161940 con 0x7f783806e510 2024-04-26T23:38:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.935+0000 7f7837400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.935+0000 7f7837400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:57884/0 (socket says 172.21.15.46:57884) 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.935+0000 7f7837400640 1 -- 172.21.15.46:0/785793386 learned_addr learned my addr 172.21.15.46:0/785793386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.936+0000 7f7836a00640 1 --2- 172.21.15.46:0/785793386 >> v2:172.21.15.142:3300/0 conn(0x7f7838160bd0 0x7f7838161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.936+0000 7f7837400640 1 -- 172.21.15.46:0/785793386 >> v2:172.21.15.142:3300/0 conn(0x7f7838160bd0 msgr2=0x7f7838161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.936+0000 7f7837400640 1 --2- 172.21.15.46:0/785793386 >> v2:172.21.15.142:3300/0 conn(0x7f7838160bd0 0x7f7838161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.936+0000 7f7837400640 1 -- 172.21.15.46:0/785793386 >> v1:172.21.15.142:6789/0 conn(0x7f783815d540 legacy=0x7f783815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.936+0000 7f7837400640 1 -- 172.21.15.46:0/785793386 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7838071f80 con 0x7f783806e510 2024-04-26T23:38:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.938+0000 7f7837400640 1 --2- 172.21.15.46:0/785793386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7828009870 tx=0x7f782802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff4678bf5a2cbdd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.938+0000 7f7836000640 1 -- 172.21.15.46:0/785793386 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828037070 con 0x7f783806e510 2024-04-26T23:38:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.938+0000 7f7836000640 1 -- 172.21.15.46:0/785793386 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828032440 con 0x7f783806e510 2024-04-26T23:38:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.938+0000 7f7836000640 1 -- 172.21.15.46:0/785793386 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828032810 con 0x7f783806e510 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 -- 172.21.15.46:0/785793386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 msgr2=0x7f783806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 --2- 172.21.15.46:0/785793386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7828009870 tx=0x7f782802a250 comp rx=0 tx=0).stop 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 -- 172.21.15.46:0/785793386 shutdown_connections 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 --2- 172.21.15.46:0/785793386 >> v2:172.21.15.142:3300/0 conn(0x7f7838160bd0 0x7f7838161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 --2- 172.21.15.46:0/785793386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783806e8f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 -- 172.21.15.46:0/785793386 >> 172.21.15.46:0/785793386 conn(0x7f783806c530 msgr2=0x7f783806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:56.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 -- 172.21.15.46:0/785793386 shutdown_connections 2024-04-26T23:38:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.939+0000 7f783e200640 1 -- 172.21.15.46:0/785793386 wait complete. 2024-04-26T23:38:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.941+0000 7f783e200640 1 Processor -- start 2024-04-26T23:38:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.941+0000 7f783e200640 1 -- start start 2024-04-26T23:38:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783813bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 0x7f7838158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7838072820 con 0x7f7838160bd0 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7838072540 con 0x7f783815d540 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f783e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7838161570 con 0x7f783806e510 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.942+0000 7f7837e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42572/0 (socket says 172.21.15.46:42572) 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 -- 172.21.15.46:0/2305590374 learned_addr learned my addr 172.21.15.46:0/2305590374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837400640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783813bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7836a00640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 0x7f7838158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 -- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 msgr2=0x7f783813bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783813bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 -- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 msgr2=0x7f7838158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 0x7f7838158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 -- 172.21.15.46:0/2305590374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7838139b10 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7836a00640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 0x7f7838158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.943+0000 7f7837e00640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f783000c7a0 tx=0x7f783000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.944+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830003b70 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.944+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7830003d30 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.944+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000aa90 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.944+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7838139db0 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.944+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783814cc70 con 0x7f7838160bd0 2024-04-26T23:38:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.945+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838069530 con 0x7f7838160bd0 2024-04-26T23:38:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.946+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f783000ac30 con 0x7f7838160bd0 2024-04-26T23:38:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.947+0000 7f782f400640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 0x7f781806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.947+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f783008bd10 con 0x7f7838160bd0 2024-04-26T23:38:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.948+0000 7f7837400640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 0x7f781806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.949+0000 7f7837400640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 0x7f781806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7828034000 tx=0x7f782802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:56.951+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7830091020 con 0x7f7838160bd0 2024-04-26T23:38:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.155+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78380d0aa0 con 0x7f781806c920 2024-04-26T23:38:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.174+0000 7f782f400640 1 -- 172.21.15.46:0/2305590374 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f78380d0aa0 con 0x7f781806c920 2024-04-26T23:38:57.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.178+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 msgr2=0x7f781806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.178+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 0x7f781806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7828034000 tx=0x7f782802e000 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.178+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 msgr2=0x7f78380b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.178+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f783000c7a0 tx=0x7f783000cc60 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 shutdown_connections 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781806c920 0x7f781806ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160bd0 0x7f78380b2f40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f783815d540 0x7f7838158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 --2- 172.21.15.46:0/2305590374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f783806e510 0x7f783813bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 >> 172.21.15.46:0/2305590374 conn(0x7f783806c530 msgr2=0x7f78380b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 shutdown_connections 2024-04-26T23:38:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.179+0000 7f783e200640 1 -- 172.21.15.46:0/2305590374 wait complete. 2024-04-26T23:38:57.201 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-26T23:38:55.922008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233088,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741632,"statfs":{"total":773094113280,"available":772855431168,"internally_reserved":0,"allocated":4546560,"data_stored":2600788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001900"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677871+0000","last_change":"2024-04-26T23:38:50.677871+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677871+0000","last_undegraded":"2024-04-26T23:38:50.677871+0000","last_fullsized":"2024-04-26T23:38:50.677871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680384+0000","last_change":"2024-04-26T23:38:50.680384+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680384+0000","last_undegraded":"2024-04-26T23:38:50.680384+0000","last_fullsized":"2024-04-26T23:38:50.680384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669294+0000","last_change":"2024-04-26T23:38:50.669294+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669294+0000","last_undegraded":"2024-04-26T23:38:50.669294+0000","last_fullsized":"2024-04-26T23:38:50.669294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.675317+0000","last_change":"2024-04-26T23:38:50.675317+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.675317+0000","last_undegraded":"2024-04-26T23:38:50.675317+0000","last_fullsized":"2024-04-26T23:38:50.675317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672056+0000","last_change":"2024-04-26T23:38:50.672056+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672056+0000","last_undegraded":"2024-04-26T23:38:50.672056+0000","last_fullsized":"2024-04-26T23:38:50.672056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677312+0000","last_change":"2024-04-26T23:38:50.677312+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677312+0000","last_undegraded":"2024-04-26T23:38:50.677312+0000","last_fullsized":"2024-04-26T23:38:50.677312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667316+0000","last_change":"2024-04-26T23:38:50.667316+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667316+0000","last_undegraded":"2024-04-26T23:38:50.667316+0000","last_fullsized":"2024-04-26T23:38:50.667316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672414+0000","last_change":"2024-04-26T23:38:50.672414+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672414+0000","last_undegraded":"2024-04-26T23:38:50.672414+0000","last_fullsized":"2024-04-26T23:38:50.672414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667307+0000","last_change":"2024-04-26T23:38:50.667307+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667307+0000","last_undegraded":"2024-04-26T23:38:50.667307+0000","last_fullsized":"2024-04-26T23:38:50.667307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669299+0000","last_change":"2024-04-26T23:38:50.669299+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669299+0000","last_undegraded":"2024-04-26T23:38:50.669299+0000","last_fullsized":"2024-04-26T23:38:50.669299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674349+0000","last_change":"2024-04-26T23:38:50.674349+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674349+0000","last_undegraded":"2024-04-26T23:38:50.674349+0000","last_fullsized":"2024-04-26T23:38:50.674349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667276+0000","last_change":"2024-04-26T23:38:50.667276+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667276+0000","last_undegraded":"2024-04-26T23:38:50.667276+0000","last_fullsized":"2024-04-26T23:38:50.667276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672590+0000","last_change":"2024-04-26T23:38:50.672590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672590+0000","last_undegraded":"2024-04-26T23:38:50.672590+0000","last_fullsized":"2024-04-26T23:38:50.672590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671236+0000","last_change":"2024-04-26T23:38:50.671236+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671236+0000","last_undegraded":"2024-04-26T23:38:50.671236+0000","last_fullsized":"2024-04-26T23:38:50.671236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666175+0000","last_change":"2024-04-26T23:38:50.666175+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666175+0000","last_undegraded":"2024-04-26T23:38:50.666175+0000","last_fullsized":"2024-04-26T23:38:50.666175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676905+0000","last_change":"2024-04-26T23:38:50.676905+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676905+0000","last_undegraded":"2024-04-26T23:38:50.676905+0000","last_fullsized":"2024-04-26T23:38:50.676905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.685115+0000","last_change":"2024-04-26T23:38:50.685115+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.685115+0000","last_undegraded":"2024-04-26T23:38:50.685115+0000","last_fullsized":"2024-04-26T23:38:50.685115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676567+0000","last_change":"2024-04-26T23:38:50.676567+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676567+0000","last_undegraded":"2024-04-26T23:38:50.676567+0000","last_fullsized":"2024-04-26T23:38:50.676567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673936+0000","last_change":"2024-04-26T23:38:50.673936+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673936+0000","last_undegraded":"2024-04-26T23:38:50.673936+0000","last_fullsized":"2024-04-26T23:38:50.673936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677945+0000","last_change":"2024-04-26T23:38:50.677945+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677945+0000","last_undegraded":"2024-04-26T23:38:50.677945+0000","last_fullsized":"2024-04-26T23:38:50.677945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671386+0000","last_change":"2024-04-26T23:38:50.671386+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671386+0000","last_undegraded":"2024-04-26T23:38:50.671386+0000","last_fullsized":"2024-04-26T23:38:50.671386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.669210+0000","last_change":"2024-04-26T23:38:50.669210+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.669210+0000","last_undegraded":"2024-04-26T23:38:50.669210+0000","last_fullsized":"2024-04-26T23:38:50.669210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679803+0000","last_change":"2024-04-26T23:38:50.679803+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679803+0000","last_undegraded":"2024-04-26T23:38:50.679803+0000","last_fullsized":"2024-04-26T23:38:50.679803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674451+0000","last_change":"2024-04-26T23:38:50.674451+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674451+0000","last_undegraded":"2024-04-26T23:38:50.674451+0000","last_fullsized":"2024-04-26T23:38:50.674451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677442+0000","last_change":"2024-04-26T23:38:50.677442+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677442+0000","last_undegraded":"2024-04-26T23:38:50.677442+0000","last_fullsized":"2024-04-26T23:38:50.677442+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678807+0000","last_change":"2024-04-26T23:38:50.678807+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678807+0000","last_undegraded":"2024-04-26T23:38:50.678807+0000","last_fullsized":"2024-04-26T23:38:50.678807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676493+0000","last_change":"2024-04-26T23:38:50.676493+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676493+0000","last_undegraded":"2024-04-26T23:38:50.676493+0000","last_fullsized":"2024-04-26T23:38:50.676493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673334+0000","last_change":"2024-04-26T23:38:50.673334+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673334+0000","last_undegraded":"2024-04-26T23:38:50.673334+0000","last_fullsized":"2024-04-26T23:38:50.673334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678505+0000","last_change":"2024-04-26T23:38:50.678505+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678505+0000","last_undegraded":"2024-04-26T23:38:50.678505+0000","last_fullsized":"2024-04-26T23:38:50.678505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676624+0000","last_change":"2024-04-26T23:38:50.676624+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676624+0000","last_undegraded":"2024-04-26T23:38:50.676624+0000","last_fullsized":"2024-04-26T23:38:50.676624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679221+0000","last_change":"2024-04-26T23:38:50.679221+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679221+0000","last_undegraded":"2024-04-26T23:38:50.679221+0000","last_fullsized":"2024-04-26T23:38:50.679221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676259+0000","last_change":"2024-04-26T23:38:50.676259+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676259+0000","last_undegraded":"2024-04-26T23:38:50.676259+0000","last_fullsized":"2024-04-26T23:38:50.676259+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682341+0000","last_change":"2024-04-26T23:38:50.682341+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682341+0000","last_undegraded":"2024-04-26T23:38:50.682341+0000","last_fullsized":"2024-04-26T23:38:50.682341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694038+0000","last_change":"2024-04-26T23:38:50.691324+0000","last_active":"2024-04-26T23:38:50.694038+0000","last_peered":"2024-04-26T23:38:50.694038+0000","last_clean":"2024-04-26T23:38:50.694038+0000","last_became_active":"2024-04-26T23:38:50.691023+0000","last_became_peered":"2024-04-26T23:38:50.691023+0000","last_unstale":"2024-04-26T23:38:50.694038+0000","last_undegraded":"2024-04-26T23:38:50.694038+0000","last_fullsized":"2024-04-26T23:38:50.694038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:55.670095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678183+0000","last_change":"2024-04-26T23:38:50.678183+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678183+0000","last_undegraded":"2024-04-26T23:38:50.678183+0000","last_fullsized":"2024-04-26T23:38:50.678183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693660+0000","last_change":"2024-04-26T23:38:50.691001+0000","last_active":"2024-04-26T23:38:50.693660+0000","last_peered":"2024-04-26T23:38:50.693660+0000","last_clean":"2024-04-26T23:38:50.693660+0000","last_became_active":"2024-04-26T23:38:50.690668+0000","last_became_peered":"2024-04-26T23:38:50.690668+0000","last_unstale":"2024-04-26T23:38:50.693660+0000","last_undegraded":"2024-04-26T23:38:50.693660+0000","last_fullsized":"2024-04-26T23:38:50.693660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665322+0000","last_change":"2024-04-26T23:38:50.665322+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665322+0000","last_undegraded":"2024-04-26T23:38:50.665322+0000","last_fullsized":"2024-04-26T23:38:50.665322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.694003+0000","last_change":"2024-04-26T23:38:50.689919+0000","last_active":"2024-04-26T23:38:50.694003+0000","last_peered":"2024-04-26T23:38:50.694003+0000","last_clean":"2024-04-26T23:38:50.694003+0000","last_became_active":"2024-04-26T23:38:50.689629+0000","last_became_peered":"2024-04-26T23:38:50.689629+0000","last_unstale":"2024-04-26T23:38:50.694003+0000","last_undegraded":"2024-04-26T23:38:50.694003+0000","last_fullsized":"2024-04-26T23:38:50.694003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.753794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.678342+0000","last_change":"2024-04-26T23:38:50.678342+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.678342+0000","last_undegraded":"2024-04-26T23:38:50.678342+0000","last_fullsized":"2024-04-26T23:38:50.678342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690252+0000","last_change":"2024-04-26T23:38:50.684308+0000","last_active":"2024-04-26T23:38:50.690252+0000","last_peered":"2024-04-26T23:38:50.690252+0000","last_clean":"2024-04-26T23:38:50.690252+0000","last_became_active":"2024-04-26T23:38:50.684017+0000","last_became_peered":"2024-04-26T23:38:50.684017+0000","last_unstale":"2024-04-26T23:38:50.690252+0000","last_undegraded":"2024-04-26T23:38:50.690252+0000","last_fullsized":"2024-04-26T23:38:50.690252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.549403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667266+0000","last_change":"2024-04-26T23:38:50.667266+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667266+0000","last_undegraded":"2024-04-26T23:38:50.667266+0000","last_fullsized":"2024-04-26T23:38:50.667266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693328+0000","last_change":"2024-04-26T23:38:50.687042+0000","last_active":"2024-04-26T23:38:50.693328+0000","last_peered":"2024-04-26T23:38:50.693328+0000","last_clean":"2024-04-26T23:38:50.693328+0000","last_became_active":"2024-04-26T23:38:50.685547+0000","last_became_peered":"2024-04-26T23:38:50.685547+0000","last_unstale":"2024-04-26T23:38:50.693328+0000","last_undegraded":"2024-04-26T23:38:50.693328+0000","last_fullsized":"2024-04-26T23:38:50.693328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.664806+0000","last_change":"2024-04-26T23:38:50.664806+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.664806+0000","last_undegraded":"2024-04-26T23:38:50.664806+0000","last_fullsized":"2024-04-26T23:38:50.664806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697047+0000","last_change":"2024-04-26T23:38:50.695028+0000","last_active":"2024-04-26T23:38:50.697047+0000","last_peered":"2024-04-26T23:38:50.697047+0000","last_clean":"2024-04-26T23:38:50.697047+0000","last_became_active":"2024-04-26T23:38:50.694628+0000","last_became_peered":"2024-04-26T23:38:50.694628+0000","last_unstale":"2024-04-26T23:38:50.697047+0000","last_undegraded":"2024-04-26T23:38:50.697047+0000","last_fullsized":"2024-04-26T23:38:50.697047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.667992+0000","last_change":"2024-04-26T23:38:50.667992+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.667992+0000","last_undegraded":"2024-04-26T23:38:50.667992+0000","last_fullsized":"2024-04-26T23:38:50.667992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699401+0000","last_change":"2024-04-26T23:38:50.698125+0000","last_active":"2024-04-26T23:38:50.699401+0000","last_peered":"2024-04-26T23:38:50.699401+0000","last_clean":"2024-04-26T23:38:50.699401+0000","last_became_active":"2024-04-26T23:38:50.697795+0000","last_became_peered":"2024-04-26T23:38:50.697795+0000","last_unstale":"2024-04-26T23:38:50.699401+0000","last_undegraded":"2024-04-26T23:38:50.699401+0000","last_fullsized":"2024-04-26T23:38:50.699401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.673039+0000","last_change":"2024-04-26T23:38:50.673039+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.673039+0000","last_undegraded":"2024-04-26T23:38:50.673039+0000","last_fullsized":"2024-04-26T23:38:50.673039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689406+0000","last_change":"2024-04-26T23:38:50.680954+0000","last_active":"2024-04-26T23:38:50.689406+0000","last_peered":"2024-04-26T23:38:50.689406+0000","last_clean":"2024-04-26T23:38:50.689406+0000","last_became_active":"2024-04-26T23:38:50.680546+0000","last_became_peered":"2024-04-26T23:38:50.680546+0000","last_unstale":"2024-04-26T23:38:50.689406+0000","last_undegraded":"2024-04-26T23:38:50.689406+0000","last_fullsized":"2024-04-26T23:38:50.689406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:51.613896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.674846+0000","last_change":"2024-04-26T23:38:50.674846+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.674846+0000","last_undegraded":"2024-04-26T23:38:50.674846+0000","last_fullsized":"2024-04-26T23:38:50.674846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696898+0000","last_change":"2024-04-26T23:38:50.694593+0000","last_active":"2024-04-26T23:38:50.696898+0000","last_peered":"2024-04-26T23:38:50.696898+0000","last_clean":"2024-04-26T23:38:50.696898+0000","last_became_active":"2024-04-26T23:38:50.694289+0000","last_became_peered":"2024-04-26T23:38:50.694289+0000","last_unstale":"2024-04-26T23:38:50.696898+0000","last_undegraded":"2024-04-26T23:38:50.696898+0000","last_fullsized":"2024-04-26T23:38:50.696898+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.683436+0000","last_change":"2024-04-26T23:38:50.683436+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.683436+0000","last_undegraded":"2024-04-26T23:38:50.683436+0000","last_fullsized":"2024-04-26T23:38:50.683436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.701161+0000","last_change":"2024-04-26T23:38:50.696626+0000","last_active":"2024-04-26T23:38:50.701161+0000","last_peered":"2024-04-26T23:38:50.701161+0000","last_clean":"2024-04-26T23:38:50.701161+0000","last_became_active":"2024-04-26T23:38:50.696142+0000","last_became_peered":"2024-04-26T23:38:50.696142+0000","last_unstale":"2024-04-26T23:38:50.701161+0000","last_undegraded":"2024-04-26T23:38:50.701161+0000","last_fullsized":"2024-04-26T23:38:50.701161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666542+0000","last_change":"2024-04-26T23:38:50.666542+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666542+0000","last_undegraded":"2024-04-26T23:38:50.666542+0000","last_fullsized":"2024-04-26T23:38:50.666542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.698415+0000","last_change":"2024-04-26T23:38:50.695193+0000","last_active":"2024-04-26T23:38:50.698415+0000","last_peered":"2024-04-26T23:38:50.698415+0000","last_clean":"2024-04-26T23:38:50.698415+0000","last_became_active":"2024-04-26T23:38:50.694850+0000","last_became_peered":"2024-04-26T23:38:50.694850+0000","last_unstale":"2024-04-26T23:38:50.698415+0000","last_undegraded":"2024-04-26T23:38:50.698415+0000","last_fullsized":"2024-04-26T23:38:50.698415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.668261+0000","last_change":"2024-04-26T23:38:50.668261+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.668261+0000","last_undegraded":"2024-04-26T23:38:50.668261+0000","last_fullsized":"2024-04-26T23:38:50.668261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697007+0000","last_change":"2024-04-26T23:38:50.695400+0000","last_active":"2024-04-26T23:38:50.697007+0000","last_peered":"2024-04-26T23:38:50.697007+0000","last_clean":"2024-04-26T23:38:50.697007+0000","last_became_active":"2024-04-26T23:38:50.695178+0000","last_became_peered":"2024-04-26T23:38:50.695178+0000","last_unstale":"2024-04-26T23:38:50.697007+0000","last_undegraded":"2024-04-26T23:38:50.697007+0000","last_fullsized":"2024-04-26T23:38:50.697007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:22.297685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666548+0000","last_change":"2024-04-26T23:38:50.666548+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666548+0000","last_undegraded":"2024-04-26T23:38:50.666548+0000","last_fullsized":"2024-04-26T23:38:50.666548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696932+0000","last_change":"2024-04-26T23:38:50.695534+0000","last_active":"2024-04-26T23:38:50.696932+0000","last_peered":"2024-04-26T23:38:50.696932+0000","last_clean":"2024-04-26T23:38:50.696932+0000","last_became_active":"2024-04-26T23:38:50.686496+0000","last_became_peered":"2024-04-26T23:38:50.686496+0000","last_unstale":"2024-04-26T23:38:50.696932+0000","last_undegraded":"2024-04-26T23:38:50.696932+0000","last_fullsized":"2024-04-26T23:38:50.696932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.662014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672274+0000","last_change":"2024-04-26T23:38:50.672274+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672274+0000","last_undegraded":"2024-04-26T23:38:50.672274+0000","last_fullsized":"2024-04-26T23:38:50.672274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.699742+0000","last_change":"2024-04-26T23:38:50.698660+0000","last_active":"2024-04-26T23:38:50.699742+0000","last_peered":"2024-04-26T23:38:50.699742+0000","last_clean":"2024-04-26T23:38:50.699742+0000","last_became_active":"2024-04-26T23:38:50.698295+0000","last_became_peered":"2024-04-26T23:38:50.698295+0000","last_unstale":"2024-04-26T23:38:50.699742+0000","last_undegraded":"2024-04-26T23:38:50.699742+0000","last_fullsized":"2024-04-26T23:38:50.699742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679622+0000","last_change":"2024-04-26T23:38:50.679622+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679622+0000","last_undegraded":"2024-04-26T23:38:50.679622+0000","last_fullsized":"2024-04-26T23:38:50.679622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.689132+0000","last_change":"2024-04-26T23:38:50.686322+0000","last_active":"2024-04-26T23:38:50.689132+0000","last_peered":"2024-04-26T23:38:50.689132+0000","last_clean":"2024-04-26T23:38:50.689132+0000","last_became_active":"2024-04-26T23:38:50.684566+0000","last_became_peered":"2024-04-26T23:38:50.684566+0000","last_unstale":"2024-04-26T23:38:50.689132+0000","last_undegraded":"2024-04-26T23:38:50.689132+0000","last_fullsized":"2024-04-26T23:38:50.689132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:39.251641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.672490+0000","last_change":"2024-04-26T23:38:50.672490+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.672490+0000","last_undegraded":"2024-04-26T23:38:50.672490+0000","last_fullsized":"2024-04-26T23:38:50.672490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697031+0000","last_change":"2024-04-26T23:38:50.694214+0000","last_active":"2024-04-26T23:38:50.697031+0000","last_peered":"2024-04-26T23:38:50.697031+0000","last_clean":"2024-04-26T23:38:50.697031+0000","last_became_active":"2024-04-26T23:38:50.693909+0000","last_became_peered":"2024-04-26T23:38:50.693909+0000","last_unstale":"2024-04-26T23:38:50.697031+0000","last_undegraded":"2024-04-26T23:38:50.697031+0000","last_fullsized":"2024-04-26T23:38:50.697031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.679586+0000","last_change":"2024-04-26T23:38:50.679586+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.679586+0000","last_undegraded":"2024-04-26T23:38:50.679586+0000","last_fullsized":"2024-04-26T23:38:50.679586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692161+0000","last_change":"2024-04-26T23:38:50.686368+0000","last_active":"2024-04-26T23:38:50.692161+0000","last_peered":"2024-04-26T23:38:50.692161+0000","last_clean":"2024-04-26T23:38:50.692161+0000","last_became_active":"2024-04-26T23:38:50.686085+0000","last_became_peered":"2024-04-26T23:38:50.686085+0000","last_unstale":"2024-04-26T23:38:50.692161+0000","last_undegraded":"2024-04-26T23:38:50.692161+0000","last_fullsized":"2024-04-26T23:38:50.692161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671636+0000","last_change":"2024-04-26T23:38:50.671636+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671636+0000","last_undegraded":"2024-04-26T23:38:50.671636+0000","last_fullsized":"2024-04-26T23:38:50.671636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690421+0000","last_change":"2024-04-26T23:38:50.685721+0000","last_active":"2024-04-26T23:38:50.690421+0000","last_peered":"2024-04-26T23:38:50.690421+0000","last_clean":"2024-04-26T23:38:50.690421+0000","last_became_active":"2024-04-26T23:38:50.685381+0000","last_became_peered":"2024-04-26T23:38:50.685381+0000","last_unstale":"2024-04-26T23:38:50.690421+0000","last_undegraded":"2024-04-26T23:38:50.690421+0000","last_fullsized":"2024-04-26T23:38:50.690421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:05.204342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.666638+0000","last_change":"2024-04-26T23:38:50.666638+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.666638+0000","last_undegraded":"2024-04-26T23:38:50.666638+0000","last_fullsized":"2024-04-26T23:38:50.666638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692041+0000","last_change":"2024-04-26T23:38:50.690198+0000","last_active":"2024-04-26T23:38:50.692041+0000","last_peered":"2024-04-26T23:38:50.692041+0000","last_clean":"2024-04-26T23:38:50.692041+0000","last_became_active":"2024-04-26T23:38:50.689613+0000","last_became_peered":"2024-04-26T23:38:50.689613+0000","last_unstale":"2024-04-26T23:38:50.692041+0000","last_undegraded":"2024-04-26T23:38:50.692041+0000","last_fullsized":"2024-04-26T23:38:50.692041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:57:08.503076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671249+0000","last_change":"2024-04-26T23:38:50.671249+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671249+0000","last_undegraded":"2024-04-26T23:38:50.671249+0000","last_fullsized":"2024-04-26T23:38:50.671249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693306+0000","last_change":"2024-04-26T23:38:50.691793+0000","last_active":"2024-04-26T23:38:50.693306+0000","last_peered":"2024-04-26T23:38:50.693306+0000","last_clean":"2024-04-26T23:38:50.693306+0000","last_became_active":"2024-04-26T23:38:50.691492+0000","last_became_peered":"2024-04-26T23:38:50.691492+0000","last_unstale":"2024-04-26T23:38:50.693306+0000","last_undegraded":"2024-04-26T23:38:50.693306+0000","last_fullsized":"2024-04-26T23:38:50.693306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676346+0000","last_change":"2024-04-26T23:38:50.676346+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676346+0000","last_undegraded":"2024-04-26T23:38:50.676346+0000","last_fullsized":"2024-04-26T23:38:50.676346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690651+0000","last_change":"2024-04-26T23:38:50.687713+0000","last_active":"2024-04-26T23:38:50.690651+0000","last_peered":"2024-04-26T23:38:50.690651+0000","last_clean":"2024-04-26T23:38:50.690651+0000","last_became_active":"2024-04-26T23:38:50.687314+0000","last_became_peered":"2024-04-26T23:38:50.687314+0000","last_unstale":"2024-04-26T23:38:50.690651+0000","last_undegraded":"2024-04-26T23:38:50.690651+0000","last_fullsized":"2024-04-26T23:38:50.690651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:38.872504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665851+0000","last_change":"2024-04-26T23:38:50.665851+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665851+0000","last_undegraded":"2024-04-26T23:38:50.665851+0000","last_fullsized":"2024-04-26T23:38:50.665851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.684746+0000","last_change":"2024-04-26T23:38:50.682745+0000","last_active":"2024-04-26T23:38:50.684746+0000","last_peered":"2024-04-26T23:38:50.684746+0000","last_clean":"2024-04-26T23:38:50.684746+0000","last_became_active":"2024-04-26T23:38:50.682462+0000","last_became_peered":"2024-04-26T23:38:50.682462+0000","last_unstale":"2024-04-26T23:38:50.684746+0000","last_undegraded":"2024-04-26T23:38:50.684746+0000","last_fullsized":"2024-04-26T23:38:50.684746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:19.185932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670016+0000","last_change":"2024-04-26T23:38:50.670016+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670016+0000","last_undegraded":"2024-04-26T23:38:50.670016+0000","last_fullsized":"2024-04-26T23:38:50.670016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.700135+0000","last_change":"2024-04-26T23:38:50.699004+0000","last_active":"2024-04-26T23:38:50.700135+0000","last_peered":"2024-04-26T23:38:50.700135+0000","last_clean":"2024-04-26T23:38:50.700135+0000","last_became_active":"2024-04-26T23:38:50.698643+0000","last_became_peered":"2024-04-26T23:38:50.698643+0000","last_unstale":"2024-04-26T23:38:50.700135+0000","last_undegraded":"2024-04-26T23:38:50.700135+0000","last_fullsized":"2024-04-26T23:38:50.700135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.671845+0000","last_change":"2024-04-26T23:38:50.671845+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.671845+0000","last_undegraded":"2024-04-26T23:38:50.671845+0000","last_fullsized":"2024-04-26T23:38:50.671845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.691431+0000","last_change":"2024-04-26T23:38:50.688989+0000","last_active":"2024-04-26T23:38:50.691431+0000","last_peered":"2024-04-26T23:38:50.691431+0000","last_clean":"2024-04-26T23:38:50.691431+0000","last_became_active":"2024-04-26T23:38:50.688711+0000","last_became_peered":"2024-04-26T23:38:50.688711+0000","last_unstale":"2024-04-26T23:38:50.691431+0000","last_undegraded":"2024-04-26T23:38:50.691431+0000","last_fullsized":"2024-04-26T23:38:50.691431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:23.924574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680231+0000","last_change":"2024-04-26T23:38:50.680231+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680231+0000","last_undegraded":"2024-04-26T23:38:50.680231+0000","last_fullsized":"2024-04-26T23:38:50.680231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696348+0000","last_change":"2024-04-26T23:38:50.693273+0000","last_active":"2024-04-26T23:38:50.696348+0000","last_peered":"2024-04-26T23:38:50.696348+0000","last_clean":"2024-04-26T23:38:50.696348+0000","last_became_active":"2024-04-26T23:38:50.692984+0000","last_became_peered":"2024-04-26T23:38:50.692984+0000","last_unstale":"2024-04-26T23:38:50.696348+0000","last_undegraded":"2024-04-26T23:38:50.696348+0000","last_fullsized":"2024-04-26T23:38:50.696348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.682956+0000","last_change":"2024-04-26T23:38:50.682956+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.682956+0000","last_undegraded":"2024-04-26T23:38:50.682956+0000","last_fullsized":"2024-04-26T23:38:50.682956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.693238+0000","last_change":"2024-04-26T23:38:50.691632+0000","last_active":"2024-04-26T23:38:50.693238+0000","last_peered":"2024-04-26T23:38:50.693238+0000","last_clean":"2024-04-26T23:38:50.693238+0000","last_became_active":"2024-04-26T23:38:50.691073+0000","last_became_peered":"2024-04-26T23:38:50.691073+0000","last_unstale":"2024-04-26T23:38:50.693238+0000","last_undegraded":"2024-04-26T23:38:50.693238+0000","last_fullsized":"2024-04-26T23:38:50.693238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:05.766590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.676918+0000","last_change":"2024-04-26T23:38:50.676918+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.676918+0000","last_undegraded":"2024-04-26T23:38:50.676918+0000","last_fullsized":"2024-04-26T23:38:50.676918+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690054+0000","last_change":"2024-04-26T23:38:50.686345+0000","last_active":"2024-04-26T23:38:50.690054+0000","last_peered":"2024-04-26T23:38:50.690054+0000","last_clean":"2024-04-26T23:38:50.690054+0000","last_became_active":"2024-04-26T23:38:50.685727+0000","last_became_peered":"2024-04-26T23:38:50.685727+0000","last_unstale":"2024-04-26T23:38:50.690054+0000","last_undegraded":"2024-04-26T23:38:50.690054+0000","last_fullsized":"2024-04-26T23:38:50.690054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.665923+0000","last_change":"2024-04-26T23:38:50.665923+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.665923+0000","last_undegraded":"2024-04-26T23:38:50.665923+0000","last_fullsized":"2024-04-26T23:38:50.665923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.690132+0000","last_change":"2024-04-26T23:38:50.688232+0000","last_active":"2024-04-26T23:38:50.690132+0000","last_peered":"2024-04-26T23:38:50.690132+0000","last_clean":"2024-04-26T23:38:50.690132+0000","last_became_active":"2024-04-26T23:38:50.687890+0000","last_became_peered":"2024-04-26T23:38:50.687890+0000","last_unstale":"2024-04-26T23:38:50.690132+0000","last_undegraded":"2024-04-26T23:38:50.690132+0000","last_fullsized":"2024-04-26T23:38:50.690132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.677590+0000","last_change":"2024-04-26T23:38:50.677590+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.677590+0000","last_undegraded":"2024-04-26T23:38:50.677590+0000","last_fullsized":"2024-04-26T23:38:50.677590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.692825+0000","last_change":"2024-04-26T23:38:50.688814+0000","last_active":"2024-04-26T23:38:50.692825+0000","last_peered":"2024-04-26T23:38:50.692825+0000","last_clean":"2024-04-26T23:38:50.692825+0000","last_became_active":"2024-04-26T23:38:50.688521+0000","last_became_peered":"2024-04-26T23:38:50.688521+0000","last_unstale":"2024-04-26T23:38:50.692825+0000","last_undegraded":"2024-04-26T23:38:50.692825+0000","last_fullsized":"2024-04-26T23:38:50.692825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:00.898736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670198+0000","last_change":"2024-04-26T23:38:50.670198+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670198+0000","last_undegraded":"2024-04-26T23:38:50.670198+0000","last_fullsized":"2024-04-26T23:38:50.670198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.682702+0000","last_change":"2024-04-26T23:38:50.678514+0000","last_active":"2024-04-26T23:38:50.682702+0000","last_peered":"2024-04-26T23:38:50.682702+0000","last_clean":"2024-04-26T23:38:50.682702+0000","last_became_active":"2024-04-26T23:38:50.678094+0000","last_became_peered":"2024-04-26T23:38:50.678094+0000","last_unstale":"2024-04-26T23:38:50.682702+0000","last_undegraded":"2024-04-26T23:38:50.682702+0000","last_fullsized":"2024-04-26T23:38:50.682702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:03.457234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.680407+0000","last_change":"2024-04-26T23:38:50.680407+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.680407+0000","last_undegraded":"2024-04-26T23:38:50.680407+0000","last_fullsized":"2024-04-26T23:38:50.680407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.696274+0000","last_change":"2024-04-26T23:38:50.685335+0000","last_active":"2024-04-26T23:38:50.696274+0000","last_peered":"2024-04-26T23:38:50.696274+0000","last_clean":"2024-04-26T23:38:50.696274+0000","last_became_active":"2024-04-26T23:38:50.685055+0000","last_became_peered":"2024-04-26T23:38:50.685055+0000","last_unstale":"2024-04-26T23:38:50.696274+0000","last_undegraded":"2024-04-26T23:38:50.696274+0000","last_fullsized":"2024-04-26T23:38:50.696274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T23:38:50.670944+0000","last_change":"2024-04-26T23:38:50.670944+0000","last_active":"2024-04-26T23:38:50.638842+0000","last_peered":"2024-04-26T23:38:50.638842+0000","last_clean":"2024-04-26T23:38:50.638842+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:38:50.670944+0000","last_undegraded":"2024-04-26T23:38:50.670944+0000","last_fullsized":"2024-04-26T23:38:50.670944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.697010+0000","last_change":"2024-04-26T23:38:50.694147+0000","last_active":"2024-04-26T23:38:50.697010+0000","last_peered":"2024-04-26T23:38:50.697010+0000","last_clean":"2024-04-26T23:38:50.697010+0000","last_became_active":"2024-04-26T23:38:50.692579+0000","last_became_peered":"2024-04-26T23:38:50.692579+0000","last_unstale":"2024-04-26T23:38:50.697010+0000","last_undegraded":"2024-04-26T23:38:50.697010+0000","last_fullsized":"2024-04-26T23:38:50.697010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T23:38:50.687375+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:38:50.687375+0000","last_peered":"2024-04-26T23:38:50.687375+0000","last_clean":"2024-04-26T23:38:50.687375+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:38:50.687375+0000","last_undegraded":"2024-04-26T23:38:50.687375+0000","last_fullsized":"2024-04-26T23:38:50.687375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":174173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":771208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.677871+0000', 'last_change': '2024-04-26T23:38:50.677871+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.677871+0000', 'last_undegraded': '2024-04-26T23:38:50.677871+0000', 'last_fullsized': '2024-04-26T23:38:50.677871+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.680384+0000', 'last_change': '2024-04-26T23:38:50.680384+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.680384+0000', 'last_undegraded': '2024-04-26T23:38:50.680384+0000', 'last_fullsized': '2024-04-26T23:38:50.680384+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.669294+0000', 'last_change': '2024-04-26T23:38:50.669294+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.669294+0000', 'last_undegraded': '2024-04-26T23:38:50.669294+0000', 'last_fullsized': '2024-04-26T23:38:50.669294+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.675317+0000', 'last_change': '2024-04-26T23:38:50.675317+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.675317+0000', 'last_undegraded': '2024-04-26T23:38:50.675317+0000', 'last_fullsized': '2024-04-26T23:38:50.675317+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.672056+0000', 'last_change': '2024-04-26T23:38:50.672056+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.672056+0000', 'last_undegraded': '2024-04-26T23:38:50.672056+0000', 'last_fullsized': '2024-04-26T23:38:50.672056+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.677312+0000', 'last_change': '2024-04-26T23:38:50.677312+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.677312+0000', 'last_undegraded': '2024-04-26T23:38:50.677312+0000', 'last_fullsized': '2024-04-26T23:38:50.677312+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.667316+0000', 'last_change': '2024-04-26T23:38:50.667316+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.667316+0000', 'last_undegraded': '2024-04-26T23:38:50.667316+0000', 'last_fullsized': '2024-04-26T23:38:50.667316+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.672414+0000', 'last_change': '2024-04-26T23:38:50.672414+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.672414+0000', 'last_undegraded': '2024-04-26T23:38:50.672414+0000', 'last_fullsized': '2024-04-26T23:38:50.672414+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-26T23:38:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.667307+0000', 'last_change': '2024-04-26T23:38:50.667307+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.667307+0000', 'last_undegraded': '2024-04-26T23:38:50.667307+0000', 'last_fullsized': '2024-04-26T23:38:50.667307+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.669299+0000', 'last_change': '2024-04-26T23:38:50.669299+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.669299+0000', 'last_undegraded': '2024-04-26T23:38:50.669299+0000', 'last_fullsized': '2024-04-26T23:38:50.669299+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.674349+0000', 'last_change': '2024-04-26T23:38:50.674349+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.674349+0000', 'last_undegraded': '2024-04-26T23:38:50.674349+0000', 'last_fullsized': '2024-04-26T23:38:50.674349+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.667276+0000', 'last_change': '2024-04-26T23:38:50.667276+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.667276+0000', 'last_undegraded': '2024-04-26T23:38:50.667276+0000', 'last_fullsized': '2024-04-26T23:38:50.667276+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.672590+0000', 'last_change': '2024-04-26T23:38:50.672590+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.672590+0000', 'last_undegraded': '2024-04-26T23:38:50.672590+0000', 'last_fullsized': '2024-04-26T23:38:50.672590+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.671236+0000', 'last_change': '2024-04-26T23:38:50.671236+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.671236+0000', 'last_undegraded': '2024-04-26T23:38:50.671236+0000', 'last_fullsized': '2024-04-26T23:38:50.671236+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.666175+0000', 'last_change': '2024-04-26T23:38:50.666175+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.666175+0000', 'last_undegraded': '2024-04-26T23:38:50.666175+0000', 'last_fullsized': '2024-04-26T23:38:50.666175+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676905+0000', 'last_change': '2024-04-26T23:38:50.676905+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676905+0000', 'last_undegraded': '2024-04-26T23:38:50.676905+0000', 'last_fullsized': '2024-04-26T23:38:50.676905+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.685115+0000', 'last_change': '2024-04-26T23:38:50.685115+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.685115+0000', 'last_undegraded': '2024-04-26T23:38:50.685115+0000', 'last_fullsized': '2024-04-26T23:38:50.685115+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676567+0000', 'last_change': '2024-04-26T23:38:50.676567+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676567+0000', 'last_undegraded': '2024-04-26T23:38:50.676567+0000', 'last_fullsized': '2024-04-26T23:38:50.676567+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.673936+0000', 'last_change': '2024-04-26T23:38:50.673936+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.673936+0000', 'last_undegraded': '2024-04-26T23:38:50.673936+0000', 'last_fullsized': '2024-04-26T23:38:50.673936+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.677945+0000', 'last_change': '2024-04-26T23:38:50.677945+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.677945+0000', 'last_undegraded': '2024-04-26T23:38:50.677945+0000', 'last_fullsized': '2024-04-26T23:38:50.677945+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.671386+0000', 'last_change': '2024-04-26T23:38:50.671386+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.671386+0000', 'last_undegraded': '2024-04-26T23:38:50.671386+0000', 'last_fullsized': '2024-04-26T23:38:50.671386+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.669210+0000', 'last_change': '2024-04-26T23:38:50.669210+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.669210+0000', 'last_undegraded': '2024-04-26T23:38:50.669210+0000', 'last_fullsized': '2024-04-26T23:38:50.669210+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.679803+0000', 'last_change': '2024-04-26T23:38:50.679803+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.679803+0000', 'last_undegraded': '2024-04-26T23:38:50.679803+0000', 'last_fullsized': '2024-04-26T23:38:50.679803+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.674451+0000', 'last_change': '2024-04-26T23:38:50.674451+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.674451+0000', 'last_undegraded': '2024-04-26T23:38:50.674451+0000', 'last_fullsized': '2024-04-26T23:38:50.674451+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.677442+0000', 'last_change': '2024-04-26T23:38:50.677442+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.677442+0000', 'last_undegraded': '2024-04-26T23:38:50.677442+0000', 'last_fullsized': '2024-04-26T23:38:50.677442+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.678807+0000', 'last_change': '2024-04-26T23:38:50.678807+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.678807+0000', 'last_undegraded': '2024-04-26T23:38:50.678807+0000', 'last_fullsized': '2024-04-26T23:38:50.678807+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676493+0000', 'last_change': '2024-04-26T23:38:50.676493+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676493+0000', 'last_undegraded': '2024-04-26T23:38:50.676493+0000', 'last_fullsized': '2024-04-26T23:38:50.676493+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.673334+0000', 'last_change': '2024-04-26T23:38:50.673334+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.673334+0000', 'last_undegraded': '2024-04-26T23:38:50.673334+0000', 'last_fullsized': '2024-04-26T23:38:50.673334+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.678505+0000', 'last_change': '2024-04-26T23:38:50.678505+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.678505+0000', 'last_undegraded': '2024-04-26T23:38:50.678505+0000', 'last_fullsized': '2024-04-26T23:38:50.678505+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676624+0000', 'last_change': '2024-04-26T23:38:50.676624+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676624+0000', 'last_undegraded': '2024-04-26T23:38:50.676624+0000', 'last_fullsized': '2024-04-26T23:38:50.676624+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.679221+0000', 'last_change': '2024-04-26T23:38:50.679221+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.679221+0000', 'last_undegraded': '2024-04-26T23:38:50.679221+0000', 'last_fullsized': '2024-04-26T23:38:50.679221+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676259+0000', 'last_change': '2024-04-26T23:38:50.676259+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676259+0000', 'last_undegraded': '2024-04-26T23:38:50.676259+0000', 'last_fullsized': '2024-04-26T23:38:50.676259+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.682341+0000', 'last_change': '2024-04-26T23:38:50.682341+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.682341+0000', 'last_undegraded': '2024-04-26T23:38:50.682341+0000', 'last_fullsized': '2024-04-26T23:38:50.682341+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.678183+0000', 'last_change': '2024-04-26T23:38:50.678183+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.678183+0000', 'last_undegraded': '2024-04-26T23:38:50.678183+0000', 'last_fullsized': '2024-04-26T23:38:50.678183+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.665322+0000', 'last_change': '2024-04-26T23:38:50.665322+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.665322+0000', 'last_undegraded': '2024-04-26T23:38:50.665322+0000', 'last_fullsized': '2024-04-26T23:38:50.665322+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.678342+0000', 'last_change': '2024-04-26T23:38:50.678342+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.678342+0000', 'last_undegraded': '2024-04-26T23:38:50.678342+0000', 'last_fullsized': '2024-04-26T23:38:50.678342+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.667266+0000', 'last_change': '2024-04-26T23:38:50.667266+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.667266+0000', 'last_undegraded': '2024-04-26T23:38:50.667266+0000', 'last_fullsized': '2024-04-26T23:38:50.667266+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.664806+0000', 'last_change': '2024-04-26T23:38:50.664806+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.664806+0000', 'last_undegraded': '2024-04-26T23:38:50.664806+0000', 'last_fullsized': '2024-04-26T23:38:50.664806+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.667992+0000', 'last_change': '2024-04-26T23:38:50.667992+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.667992+0000', 'last_undegraded': '2024-04-26T23:38:50.667992+0000', 'last_fullsized': '2024-04-26T23:38:50.667992+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.673039+0000', 'last_change': '2024-04-26T23:38:50.673039+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.673039+0000', 'last_undegraded': '2024-04-26T23:38:50.673039+0000', 'last_fullsized': '2024-04-26T23:38:50.673039+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.674846+0000', 'last_change': '2024-04-26T23:38:50.674846+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.674846+0000', 'last_undegraded': '2024-04-26T23:38:50.674846+0000', 'last_fullsized': '2024-04-26T23:38:50.674846+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.683436+0000', 'last_change': '2024-04-26T23:38:50.683436+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.683436+0000', 'last_undegraded': '2024-04-26T23:38:50.683436+0000', 'last_fullsized': '2024-04-26T23:38:50.683436+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.666542+0000', 'last_change': '2024-04-26T23:38:50.666542+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.666542+0000', 'last_undegraded': '2024-04-26T23:38:50.666542+0000', 'last_fullsized': '2024-04-26T23:38:50.666542+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.668261+0000', 'last_change': '2024-04-26T23:38:50.668261+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.668261+0000', 'last_undegraded': '2024-04-26T23:38:50.668261+0000', 'last_fullsized': '2024-04-26T23:38:50.668261+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.666548+0000', 'last_change': '2024-04-26T23:38:50.666548+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.666548+0000', 'last_undegraded': '2024-04-26T23:38:50.666548+0000', 'last_fullsized': '2024-04-26T23:38:50.666548+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.672274+0000', 'last_change': '2024-04-26T23:38:50.672274+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.672274+0000', 'last_undegraded': '2024-04-26T23:38:50.672274+0000', 'last_fullsized': '2024-04-26T23:38:50.672274+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.679622+0000', 'last_change': '2024-04-26T23:38:50.679622+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.679622+0000', 'last_undegraded': '2024-04-26T23:38:50.679622+0000', 'last_fullsized': '2024-04-26T23:38:50.679622+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.672490+0000', 'last_change': '2024-04-26T23:38:50.672490+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.672490+0000', 'last_undegraded': '2024-04-26T23:38:50.672490+0000', 'last_fullsized': '2024-04-26T23:38:50.672490+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-26T23:38:57.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.679586+0000', 'last_change': '2024-04-26T23:38:50.679586+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.679586+0000', 'last_undegraded': '2024-04-26T23:38:50.679586+0000', 'last_fullsized': '2024-04-26T23:38:50.679586+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.671636+0000', 'last_change': '2024-04-26T23:38:50.671636+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.671636+0000', 'last_undegraded': '2024-04-26T23:38:50.671636+0000', 'last_fullsized': '2024-04-26T23:38:50.671636+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.666638+0000', 'last_change': '2024-04-26T23:38:50.666638+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.666638+0000', 'last_undegraded': '2024-04-26T23:38:50.666638+0000', 'last_fullsized': '2024-04-26T23:38:50.666638+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.671249+0000', 'last_change': '2024-04-26T23:38:50.671249+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.671249+0000', 'last_undegraded': '2024-04-26T23:38:50.671249+0000', 'last_fullsized': '2024-04-26T23:38:50.671249+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676346+0000', 'last_change': '2024-04-26T23:38:50.676346+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676346+0000', 'last_undegraded': '2024-04-26T23:38:50.676346+0000', 'last_fullsized': '2024-04-26T23:38:50.676346+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.665851+0000', 'last_change': '2024-04-26T23:38:50.665851+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.665851+0000', 'last_undegraded': '2024-04-26T23:38:50.665851+0000', 'last_fullsized': '2024-04-26T23:38:50.665851+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-26T23:38:57.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.670016+0000', 'last_change': '2024-04-26T23:38:50.670016+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.670016+0000', 'last_undegraded': '2024-04-26T23:38:50.670016+0000', 'last_fullsized': '2024-04-26T23:38:50.670016+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.671845+0000', 'last_change': '2024-04-26T23:38:50.671845+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.671845+0000', 'last_undegraded': '2024-04-26T23:38:50.671845+0000', 'last_fullsized': '2024-04-26T23:38:50.671845+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.680231+0000', 'last_change': '2024-04-26T23:38:50.680231+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.680231+0000', 'last_undegraded': '2024-04-26T23:38:50.680231+0000', 'last_fullsized': '2024-04-26T23:38:50.680231+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.682956+0000', 'last_change': '2024-04-26T23:38:50.682956+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.682956+0000', 'last_undegraded': '2024-04-26T23:38:50.682956+0000', 'last_fullsized': '2024-04-26T23:38:50.682956+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.676918+0000', 'last_change': '2024-04-26T23:38:50.676918+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.676918+0000', 'last_undegraded': '2024-04-26T23:38:50.676918+0000', 'last_fullsized': '2024-04-26T23:38:50.676918+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.665923+0000', 'last_change': '2024-04-26T23:38:50.665923+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.665923+0000', 'last_undegraded': '2024-04-26T23:38:50.665923+0000', 'last_fullsized': '2024-04-26T23:38:50.665923+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-26T23:38:57.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.677590+0000', 'last_change': '2024-04-26T23:38:50.677590+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.677590+0000', 'last_undegraded': '2024-04-26T23:38:50.677590+0000', 'last_fullsized': '2024-04-26T23:38:50.677590+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.670198+0000', 'last_change': '2024-04-26T23:38:50.670198+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.670198+0000', 'last_undegraded': '2024-04-26T23:38:50.670198+0000', 'last_fullsized': '2024-04-26T23:38:50.670198+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.680407+0000', 'last_change': '2024-04-26T23:38:50.680407+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.680407+0000', 'last_undegraded': '2024-04-26T23:38:50.680407+0000', 'last_fullsized': '2024-04-26T23:38:50.680407+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-26T23:38:57.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T23:38:50.670944+0000', 'last_change': '2024-04-26T23:38:50.670944+0000', 'last_active': '2024-04-26T23:38:50.638842+0000', 'last_peered': '2024-04-26T23:38:50.638842+0000', 'last_clean': '2024-04-26T23:38:50.638842+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T23:38:50.670944+0000', 'last_undegraded': '2024-04-26T23:38:50.670944+0000', 'last_fullsized': '2024-04-26T23:38:50.670944+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'last_clean_scrub_stamp': '2024-04-26T23:38:50.638842+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T23:38:57.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:38:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.453+0000 7fefc1e00640 1 Processor -- start 2024-04-26T23:38:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.453+0000 7fefc1e00640 1 -- start start 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefbc06f8f0 0x7fefbc0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 0x7fefbc073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefbc0b41b0 con 0x7fefbc15bf50 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fefbc15c330 con 0x7fefbc06f8f0 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fefbc0735c0 con 0x7fefbc072c30 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc0a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fefbc15bf50 0x7fefbc0ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53414/0 (socket says 172.21.15.46:53414) 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc0a00640 1 -- 172.21.15.46:0/1611254693 learned_addr learned my addr 172.21.15.46:0/1611254693 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:38:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefbbe00640 1 --2- 172.21.15.46:0/1611254693 >> v2:172.21.15.46:3300/0 conn(0x7fefbc06f8f0 0x7fefbc0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.455+0000 7fefc1400640 1 --2- 172.21.15.46:0/1611254693 >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 0x7fefbc073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbb400640 1 -- 172.21.15.46:0/1611254693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652661851 0 0) 0x7fefbc0b41b0 con 0x7fefbc15bf50 2024-04-26T23:38:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbb400640 1 -- 172.21.15.46:0/1611254693 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa4003520 con 0x7fefbc15bf50 2024-04-26T23:38:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbbe00640 1 -- 172.21.15.46:0/1611254693 >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 msgr2=0x7fefbc073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbbe00640 1 --2- 172.21.15.46:0/1611254693 >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 0x7fefbc073080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbbe00640 1 -- 172.21.15.46:0/1611254693 >> v1:172.21.15.46:6789/0 conn(0x7fefbc15bf50 legacy=0x7fefbc0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefbbe00640 1 -- 172.21.15.46:0/1611254693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc0b4490 con 0x7fefbc06f8f0 2024-04-26T23:38:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.456+0000 7fefc1400640 1 --2- 172.21.15.46:0/1611254693 >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 0x7fefbc073080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:38:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.457+0000 7fefbbe00640 1 --2- 172.21.15.46:0/1611254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc06f8f0 0x7fefbc0b3b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0b0740 tx=0x7fefa802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f83978672960f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.457+0000 7fefbb400640 1 -- 172.21.15.46:0/1611254693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8037070 con 0x7fefbc06f8f0 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.457+0000 7fefbb400640 1 -- 172.21.15.46:0/1611254693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa8032440 con 0x7fefbc06f8f0 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.457+0000 7fefbb400640 1 -- 172.21.15.46:0/1611254693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8032810 con 0x7fefbc06f8f0 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 -- 172.21.15.46:0/1611254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc06f8f0 msgr2=0x7fefbc0b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 --2- 172.21.15.46:0/1611254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc06f8f0 0x7fefbc0b3b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0b0740 tx=0x7fefa802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 -- 172.21.15.46:0/1611254693 shutdown_connections 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 --2- 172.21.15.46:0/1611254693 >> v2:172.21.15.142:3300/0 conn(0x7fefbc072c30 0x7fefbc073080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 --2- 172.21.15.46:0/1611254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc06f8f0 0x7fefbc0b3b20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 -- 172.21.15.46:0/1611254693 >> 172.21.15.46:0/1611254693 conn(0x7fefbc0b2820 msgr2=0x7fefbc0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 -- 172.21.15.46:0/1611254693 shutdown_connections 2024-04-26T23:38:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.458+0000 7fefc1e00640 1 -- 172.21.15.46:0/1611254693 wait complete. 2024-04-26T23:38:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.460+0000 7fefc1e00640 1 Processor -- start 2024-04-26T23:38:57.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.460+0000 7fefc1e00640 1 -- start start 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc06f8f0 0x7fefbc13d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc072c30 0x7fefbc156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc0739f0 con 0x7fefbc15bf50 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc0b4a50 con 0x7fefbc06f8f0 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc0b5250 con 0x7fefbc072c30 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42578/0 (socket says 172.21.15.46:42578) 2024-04-26T23:38:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc1400640 1 -- 172.21.15.46:0/4147464600 learned_addr learned my addr 172.21.15.46:0/4147464600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.461+0000 7fefc0a00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc06f8f0 0x7fefbc13d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefbbe00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc072c30 0x7fefbc156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 -- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc072c30 msgr2=0x7fefbc156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc072c30 0x7fefbc156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 -- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc06f8f0 msgr2=0x7fefbc13d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc06f8f0 0x7fefbc13d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 -- 172.21.15.46:0/4147464600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc155460 con 0x7fefbc15bf50 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1400640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007ef0 tx=0x7fefb400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb401a070 con 0x7fefbc15bf50 2024-04-26T23:38:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb4003b90 con 0x7fefbc15bf50 2024-04-26T23:38:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4004030 con 0x7fefbc15bf50 2024-04-26T23:38:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.462+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc155740 con 0x7fefbc15bf50 2024-04-26T23:38:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.463+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc13dca0 con 0x7fefbc15bf50 2024-04-26T23:38:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.464+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc069d20 con 0x7fefbc15bf50 2024-04-26T23:38:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.465+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb401d070 con 0x7fefbc15bf50 2024-04-26T23:38:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.466+0000 7fefb8c00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 0x7fef9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.466+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fefb408bee0 con 0x7fefbc15bf50 2024-04-26T23:38:57.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.466+0000 7fefc0a00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 0x7fef9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.467+0000 7fefc0a00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 0x7fef9c06ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fd0 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:57.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.470+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefb4010070 con 0x7fefbc15bf50 2024-04-26T23:38:57.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.751+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fefbc0693b0 con 0x7fefbc15bf50 2024-04-26T23:38:57.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.751+0000 7fefb8c00640 1 -- 172.21.15.46:0/4147464600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1034 (secure 0 0 0) 0x7fefb4059b30 con 0x7fefbc15bf50 2024-04-26T23:38:57.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:57.755 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":238682112,"bytes_avail":772855431168,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5280}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-26T23:38:11.906082+0000","services":{}},"progress_events":{}} 2024-04-26T23:38:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 msgr2=0x7fef9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 0x7fef9c06ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fd0 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2024-04-26T23:38:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 msgr2=0x7fefbc156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007ef0 tx=0x7fefb400d4a0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 shutdown_connections 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c920 0x7fef9c06ede0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc15bf50 0x7fefbc156760 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc072c30 0x7fefbc156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 --2- 172.21.15.46:0/4147464600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc06f8f0 0x7fefbc13d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 >> 172.21.15.46:0/4147464600 conn(0x7fefbc0b2820 msgr2=0x7fefbc0b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.754+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 shutdown_connections 2024-04-26T23:38:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.755+0000 7fefc1e00640 1 -- 172.21.15.46:0/4147464600 wait complete. 2024-04-26T23:38:57.778 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T23:38:57.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:38:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.988+0000 7f9a68800640 1 Processor -- start 2024-04-26T23:38:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.988+0000 7f9a68800640 1 -- start start 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a68800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a6006e510 0x7f9a6006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a68800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a600732c0 0x7f9a60073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a68800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6006ee30 con 0x7f9a60160bd0 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a68800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9a60161940 con 0x7f9a600732c0 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a68800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a60071f80 con 0x7f9a6006e510 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a66800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a600732c0 0x7f9a60073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a66800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a600732c0 0x7f9a60073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42586/0 (socket says 172.21.15.46:42586) 2024-04-26T23:38:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.990+0000 7f9a66800640 1 -- 172.21.15.46:0/616288944 learned_addr learned my addr 172.21.15.46:0/616288944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.991+0000 7f9a65e00640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a6006e510 0x7f9a6006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.991+0000 7f9a66800640 1 -- 172.21.15.46:0/616288944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a6006e510 msgr2=0x7f9a6006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.991+0000 7f9a66800640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a6006e510 0x7f9a6006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.991+0000 7f9a66800640 1 -- 172.21.15.46:0/616288944 >> v1:172.21.15.142:6789/0 conn(0x7f9a60160bd0 legacy=0x7f9a60161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:38:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.991+0000 7f9a66800640 1 -- 172.21.15.46:0/616288944 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a60072260 con 0x7f9a600732c0 2024-04-26T23:38:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.992+0000 7f9a66800640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600732c0 0x7f9a60073750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009e50 tx=0x7f9a5c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7c170c59ade750 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.992+0000 7f9a64a00640 1 -- 172.21.15.46:0/616288944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c036070 con 0x7f9a600732c0 2024-04-26T23:38:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.992+0000 7f9a64a00640 1 -- 172.21.15.46:0/616288944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5c031600 con 0x7f9a600732c0 2024-04-26T23:38:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.992+0000 7f9a64a00640 1 -- 172.21.15.46:0/616288944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0319d0 con 0x7f9a600732c0 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 -- 172.21.15.46:0/616288944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600732c0 msgr2=0x7f9a60073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600732c0 0x7f9a60073750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009e50 tx=0x7f9a5c02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 -- 172.21.15.46:0/616288944 shutdown_connections 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600732c0 0x7f9a60073750 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 --2- 172.21.15.46:0/616288944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a6006e510 0x7f9a6006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 -- 172.21.15.46:0/616288944 >> 172.21.15.46:0/616288944 conn(0x7f9a6006c530 msgr2=0x7f9a6006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 -- 172.21.15.46:0/616288944 shutdown_connections 2024-04-26T23:38:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.993+0000 7f9a68800640 1 -- 172.21.15.46:0/616288944 wait complete. 2024-04-26T23:38:57.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.995+0000 7f9a68800640 1 Processor -- start 2024-04-26T23:38:57.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.996+0000 7f9a68800640 1 -- start start 2024-04-26T23:38:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 0x7f9a60139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a60160bd0 0x7f9a600b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60072d40 con 0x7f9a6006e510 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60072820 con 0x7f9a600732c0 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a68800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a60072ec0 con 0x7f9a60160bd0 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42588/0 (socket says 172.21.15.46:42588) 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.997+0000 7f9a65e00640 1 -- 172.21.15.46:0/1112143730 learned_addr learned my addr 172.21.15.46:0/1112143730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a66800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a60160bd0 0x7f9a600b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65400640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 0x7f9a60139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 -- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a60160bd0 msgr2=0x7f9a600b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a60160bd0 0x7f9a600b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 -- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 msgr2=0x7f9a60139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 0x7f9a60139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 -- 172.21.15.46:0/1112143730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a601f94f0 con 0x7f9a6006e510 2024-04-26T23:38:58.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65400640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 0x7f9a60139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:38:58.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a65e00640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400d620 tx=0x7f9a5400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400b620 con 0x7f9a6006e510 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a54019070 con 0x7f9a6006e510 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400ba00 con 0x7f9a6006e510 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.998+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a600b1bc0 con 0x7f9a6006e510 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:57.999+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a600b2180 con 0x7f9a6006e510 2024-04-26T23:38:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.000+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60069530 con 0x7f9a6006e510 2024-04-26T23:38:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.001+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a5400bba0 con 0x7f9a6006e510 2024-04-26T23:38:58.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.002+0000 7f9a5a000640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:38:58.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.002+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9a5408c0f0 con 0x7f9a6006e510 2024-04-26T23:38:58.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.003+0000 7f9a65400640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:38:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.004+0000 7f9a65400640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 0x7f9a4406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a601f8500 tx=0x7f9a50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:38:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.007+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a601f94f0 con 0x7f9a6006e510 2024-04-26T23:38:58.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.211+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a6006fb00 con 0x7f9a4406c8f0 2024-04-26T23:38:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.230+0000 7f9a5a000640 1 -- 172.21.15.46:0/1112143730 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f9a50055850 con 0x7f9a4406c8f0 2024-04-26T23:38:58.351 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 msgr2=0x7f9a4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 0x7f9a4406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a601f8500 tx=0x7f9a50009040 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 msgr2=0x7f9a601f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400d620 tx=0x7f9a5400dae0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 shutdown_connections 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.236+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a60160bd0 0x7f9a600b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a600732c0 0x7f9a60139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 --2- 172.21.15.46:0/1112143730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6006e510 0x7f9a601f8fb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:38:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 >> 172.21.15.46:0/1112143730 conn(0x7f9a6006c530 msgr2=0x7f9a600b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:38:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 shutdown_connections 2024-04-26T23:38:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:38:58.237+0000 7f9a68800640 1 -- 172.21.15.46:0/1112143730 wait complete. 2024-04-26T23:38:58.354 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-26T23:38:57.922545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237580,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737140,"statfs":{"total":773094113280,"available":772850831360,"internally_reserved":0,"allocated":4857856,"data_stored":2743148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.263806"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916247+0000","last_change":"2024-04-26T23:38:51.700162+0000","last_active":"2024-04-26T23:38:52.916247+0000","last_peered":"2024-04-26T23:38:52.916247+0000","last_clean":"2024-04-26T23:38:52.916247+0000","last_became_active":"2024-04-26T23:38:51.699746+0000","last_became_peered":"2024-04-26T23:38:51.699746+0000","last_unstale":"2024-04-26T23:38:52.916247+0000","last_undegraded":"2024-04-26T23:38:52.916247+0000","last_fullsized":"2024-04-26T23:38:52.916247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:30.479535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914828+0000","last_change":"2024-04-26T23:38:51.692550+0000","last_active":"2024-04-26T23:38:52.914828+0000","last_peered":"2024-04-26T23:38:52.914828+0000","last_clean":"2024-04-26T23:38:52.914828+0000","last_became_active":"2024-04-26T23:38:51.692217+0000","last_became_peered":"2024-04-26T23:38:51.692217+0000","last_unstale":"2024-04-26T23:38:52.914828+0000","last_undegraded":"2024-04-26T23:38:52.914828+0000","last_fullsized":"2024-04-26T23:38:52.914828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:56.091602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691779+0000","last_change":"2024-04-26T23:38:51.706273+0000","last_active":"2024-04-26T23:38:52.691779+0000","last_peered":"2024-04-26T23:38:52.691779+0000","last_clean":"2024-04-26T23:38:52.691779+0000","last_became_active":"2024-04-26T23:38:51.705967+0000","last_became_peered":"2024-04-26T23:38:51.705967+0000","last_unstale":"2024-04-26T23:38:52.691779+0000","last_undegraded":"2024-04-26T23:38:52.691779+0000","last_fullsized":"2024-04-26T23:38:52.691779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.202801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916540+0000","last_change":"2024-04-26T23:38:51.700480+0000","last_active":"2024-04-26T23:38:52.916540+0000","last_peered":"2024-04-26T23:38:52.916540+0000","last_clean":"2024-04-26T23:38:52.916540+0000","last_became_active":"2024-04-26T23:38:51.697403+0000","last_became_peered":"2024-04-26T23:38:51.697403+0000","last_unstale":"2024-04-26T23:38:52.916540+0000","last_undegraded":"2024-04-26T23:38:52.916540+0000","last_fullsized":"2024-04-26T23:38:52.916540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:00.855520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916775+0000","last_change":"2024-04-26T23:38:51.696355+0000","last_active":"2024-04-26T23:38:52.916775+0000","last_peered":"2024-04-26T23:38:52.916775+0000","last_clean":"2024-04-26T23:38:52.916775+0000","last_became_active":"2024-04-26T23:38:51.696019+0000","last_became_peered":"2024-04-26T23:38:51.696019+0000","last_unstale":"2024-04-26T23:38:52.916775+0000","last_undegraded":"2024-04-26T23:38:52.916775+0000","last_fullsized":"2024-04-26T23:38:52.916775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:31.825522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918037+0000","last_change":"2024-04-26T23:38:51.686585+0000","last_active":"2024-04-26T23:38:52.918037+0000","last_peered":"2024-04-26T23:38:52.918037+0000","last_clean":"2024-04-26T23:38:52.918037+0000","last_became_active":"2024-04-26T23:38:51.686296+0000","last_became_peered":"2024-04-26T23:38:51.686296+0000","last_unstale":"2024-04-26T23:38:52.918037+0000","last_undegraded":"2024-04-26T23:38:52.918037+0000","last_fullsized":"2024-04-26T23:38:52.918037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915473+0000","last_change":"2024-04-26T23:38:51.711658+0000","last_active":"2024-04-26T23:38:52.915473+0000","last_peered":"2024-04-26T23:38:52.915473+0000","last_clean":"2024-04-26T23:38:52.915473+0000","last_became_active":"2024-04-26T23:38:51.709010+0000","last_became_peered":"2024-04-26T23:38:51.709010+0000","last_unstale":"2024-04-26T23:38:52.915473+0000","last_undegraded":"2024-04-26T23:38:52.915473+0000","last_fullsized":"2024-04-26T23:38:52.915473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:14.397549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914187+0000","last_change":"2024-04-26T23:38:51.705613+0000","last_active":"2024-04-26T23:38:52.914187+0000","last_peered":"2024-04-26T23:38:52.914187+0000","last_clean":"2024-04-26T23:38:52.914187+0000","last_became_active":"2024-04-26T23:38:51.705328+0000","last_became_peered":"2024-04-26T23:38:51.705328+0000","last_unstale":"2024-04-26T23:38:52.914187+0000","last_undegraded":"2024-04-26T23:38:52.914187+0000","last_fullsized":"2024-04-26T23:38:52.914187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692261+0000","last_change":"2024-04-26T23:38:51.710316+0000","last_active":"2024-04-26T23:38:52.692261+0000","last_peered":"2024-04-26T23:38:52.692261+0000","last_clean":"2024-04-26T23:38:52.692261+0000","last_became_active":"2024-04-26T23:38:51.710040+0000","last_became_peered":"2024-04-26T23:38:51.710040+0000","last_unstale":"2024-04-26T23:38:52.692261+0000","last_undegraded":"2024-04-26T23:38:52.692261+0000","last_fullsized":"2024-04-26T23:38:52.692261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:03.121714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916761+0000","last_change":"2024-04-26T23:38:51.706764+0000","last_active":"2024-04-26T23:38:52.916761+0000","last_peered":"2024-04-26T23:38:52.916761+0000","last_clean":"2024-04-26T23:38:52.916761+0000","last_became_active":"2024-04-26T23:38:51.703609+0000","last_became_peered":"2024-04-26T23:38:51.703609+0000","last_unstale":"2024-04-26T23:38:52.916761+0000","last_undegraded":"2024-04-26T23:38:52.916761+0000","last_fullsized":"2024-04-26T23:38:52.916761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:31.130045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916341+0000","last_change":"2024-04-26T23:38:51.704822+0000","last_active":"2024-04-26T23:38:52.916341+0000","last_peered":"2024-04-26T23:38:52.916341+0000","last_clean":"2024-04-26T23:38:52.916341+0000","last_became_active":"2024-04-26T23:38:51.703220+0000","last_became_peered":"2024-04-26T23:38:51.703220+0000","last_unstale":"2024-04-26T23:38:52.916341+0000","last_undegraded":"2024-04-26T23:38:52.916341+0000","last_fullsized":"2024-04-26T23:38:52.916341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691729+0000","last_change":"2024-04-26T23:38:51.695761+0000","last_active":"2024-04-26T23:38:52.691729+0000","last_peered":"2024-04-26T23:38:52.691729+0000","last_clean":"2024-04-26T23:38:52.691729+0000","last_became_active":"2024-04-26T23:38:51.695439+0000","last_became_peered":"2024-04-26T23:38:51.695439+0000","last_unstale":"2024-04-26T23:38:52.691729+0000","last_undegraded":"2024-04-26T23:38:52.691729+0000","last_fullsized":"2024-04-26T23:38:52.691729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:36.121201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915646+0000","last_change":"2024-04-26T23:38:51.701482+0000","last_active":"2024-04-26T23:38:52.915646+0000","last_peered":"2024-04-26T23:38:52.915646+0000","last_clean":"2024-04-26T23:38:52.915646+0000","last_became_active":"2024-04-26T23:38:51.701126+0000","last_became_peered":"2024-04-26T23:38:51.701126+0000","last_unstale":"2024-04-26T23:38:52.915646+0000","last_undegraded":"2024-04-26T23:38:52.915646+0000","last_fullsized":"2024-04-26T23:38:52.915646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:19.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695713+0000","last_change":"2024-04-26T23:38:51.714648+0000","last_active":"2024-04-26T23:38:52.695713+0000","last_peered":"2024-04-26T23:38:52.695713+0000","last_clean":"2024-04-26T23:38:52.695713+0000","last_became_active":"2024-04-26T23:38:51.713949+0000","last_became_peered":"2024-04-26T23:38:51.713949+0000","last_unstale":"2024-04-26T23:38:52.695713+0000","last_undegraded":"2024-04-26T23:38:52.695713+0000","last_fullsized":"2024-04-26T23:38:52.695713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:23.313646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915568+0000","last_change":"2024-04-26T23:38:51.713558+0000","last_active":"2024-04-26T23:38:52.915568+0000","last_peered":"2024-04-26T23:38:52.915568+0000","last_clean":"2024-04-26T23:38:52.915568+0000","last_became_active":"2024-04-26T23:38:51.708077+0000","last_became_peered":"2024-04-26T23:38:51.708077+0000","last_unstale":"2024-04-26T23:38:52.915568+0000","last_undegraded":"2024-04-26T23:38:52.915568+0000","last_fullsized":"2024-04-26T23:38:52.915568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.563249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919169+0000","last_change":"2024-04-26T23:38:51.708323+0000","last_active":"2024-04-26T23:38:52.919169+0000","last_peered":"2024-04-26T23:38:52.919169+0000","last_clean":"2024-04-26T23:38:52.919169+0000","last_became_active":"2024-04-26T23:38:51.706314+0000","last_became_peered":"2024-04-26T23:38:51.706314+0000","last_unstale":"2024-04-26T23:38:52.919169+0000","last_undegraded":"2024-04-26T23:38:52.919169+0000","last_fullsized":"2024-04-26T23:38:52.919169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918225+0000","last_change":"2024-04-26T23:38:51.706767+0000","last_active":"2024-04-26T23:38:52.918225+0000","last_peered":"2024-04-26T23:38:52.918225+0000","last_clean":"2024-04-26T23:38:52.918225+0000","last_became_active":"2024-04-26T23:38:51.706403+0000","last_became_peered":"2024-04-26T23:38:51.706403+0000","last_unstale":"2024-04-26T23:38:52.918225+0000","last_undegraded":"2024-04-26T23:38:52.918225+0000","last_fullsized":"2024-04-26T23:38:52.918225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:19.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698688+0000","last_change":"2024-04-26T23:38:51.704154+0000","last_active":"2024-04-26T23:38:52.698688+0000","last_peered":"2024-04-26T23:38:52.698688+0000","last_clean":"2024-04-26T23:38:52.698688+0000","last_became_active":"2024-04-26T23:38:51.703729+0000","last_became_peered":"2024-04-26T23:38:51.703729+0000","last_unstale":"2024-04-26T23:38:52.698688+0000","last_undegraded":"2024-04-26T23:38:52.698688+0000","last_fullsized":"2024-04-26T23:38:52.698688+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.135646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916131+0000","last_change":"2024-04-26T23:38:51.693174+0000","last_active":"2024-04-26T23:38:52.916131+0000","last_peered":"2024-04-26T23:38:52.916131+0000","last_clean":"2024-04-26T23:38:52.916131+0000","last_became_active":"2024-04-26T23:38:51.692843+0000","last_became_peered":"2024-04-26T23:38:51.692843+0000","last_unstale":"2024-04-26T23:38:52.916131+0000","last_undegraded":"2024-04-26T23:38:52.916131+0000","last_fullsized":"2024-04-26T23:38:52.916131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:37.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698304+0000","last_change":"2024-04-26T23:38:51.712580+0000","last_active":"2024-04-26T23:38:52.698304+0000","last_peered":"2024-04-26T23:38:52.698304+0000","last_clean":"2024-04-26T23:38:52.698304+0000","last_became_active":"2024-04-26T23:38:51.711933+0000","last_became_peered":"2024-04-26T23:38:51.711933+0000","last_unstale":"2024-04-26T23:38:52.698304+0000","last_undegraded":"2024-04-26T23:38:52.698304+0000","last_fullsized":"2024-04-26T23:38:52.698304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:29.725135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916021+0000","last_change":"2024-04-26T23:38:51.704480+0000","last_active":"2024-04-26T23:38:52.916021+0000","last_peered":"2024-04-26T23:38:52.916021+0000","last_clean":"2024-04-26T23:38:52.916021+0000","last_became_active":"2024-04-26T23:38:51.700760+0000","last_became_peered":"2024-04-26T23:38:51.700760+0000","last_unstale":"2024-04-26T23:38:52.916021+0000","last_undegraded":"2024-04-26T23:38:52.916021+0000","last_fullsized":"2024-04-26T23:38:52.916021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696264+0000","last_change":"2024-04-26T23:38:51.703706+0000","last_active":"2024-04-26T23:38:52.696264+0000","last_peered":"2024-04-26T23:38:52.696264+0000","last_clean":"2024-04-26T23:38:52.696264+0000","last_became_active":"2024-04-26T23:38:51.703118+0000","last_became_peered":"2024-04-26T23:38:51.703118+0000","last_unstale":"2024-04-26T23:38:52.696264+0000","last_undegraded":"2024-04-26T23:38:52.696264+0000","last_fullsized":"2024-04-26T23:38:52.696264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917687+0000","last_change":"2024-04-26T23:38:51.692232+0000","last_active":"2024-04-26T23:38:52.917687+0000","last_peered":"2024-04-26T23:38:52.917687+0000","last_clean":"2024-04-26T23:38:52.917687+0000","last_became_active":"2024-04-26T23:38:51.691045+0000","last_became_peered":"2024-04-26T23:38:51.691045+0000","last_unstale":"2024-04-26T23:38:52.917687+0000","last_undegraded":"2024-04-26T23:38:52.917687+0000","last_fullsized":"2024-04-26T23:38:52.917687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:49.516055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916650+0000","last_change":"2024-04-26T23:38:51.694705+0000","last_active":"2024-04-26T23:38:52.916650+0000","last_peered":"2024-04-26T23:38:52.916650+0000","last_clean":"2024-04-26T23:38:52.916650+0000","last_became_active":"2024-04-26T23:38:51.693597+0000","last_became_peered":"2024-04-26T23:38:51.693597+0000","last_unstale":"2024-04-26T23:38:52.916650+0000","last_undegraded":"2024-04-26T23:38:52.916650+0000","last_fullsized":"2024-04-26T23:38:52.916650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:19.771077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698999+0000","last_change":"2024-04-26T23:38:51.716170+0000","last_active":"2024-04-26T23:38:52.698999+0000","last_peered":"2024-04-26T23:38:52.698999+0000","last_clean":"2024-04-26T23:38:52.698999+0000","last_became_active":"2024-04-26T23:38:51.715894+0000","last_became_peered":"2024-04-26T23:38:51.715894+0000","last_unstale":"2024-04-26T23:38:52.698999+0000","last_undegraded":"2024-04-26T23:38:52.698999+0000","last_fullsized":"2024-04-26T23:38:52.698999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:36.414404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919230+0000","last_change":"2024-04-26T23:38:51.706456+0000","last_active":"2024-04-26T23:38:52.919230+0000","last_peered":"2024-04-26T23:38:52.919230+0000","last_clean":"2024-04-26T23:38:52.919230+0000","last_became_active":"2024-04-26T23:38:51.706161+0000","last_became_peered":"2024-04-26T23:38:51.706161+0000","last_unstale":"2024-04-26T23:38:52.919230+0000","last_undegraded":"2024-04-26T23:38:52.919230+0000","last_fullsized":"2024-04-26T23:38:52.919230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.202801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698821+0000","last_change":"2024-04-26T23:38:51.713320+0000","last_active":"2024-04-26T23:38:52.698821+0000","last_peered":"2024-04-26T23:38:52.698821+0000","last_clean":"2024-04-26T23:38:52.698821+0000","last_became_active":"2024-04-26T23:38:51.713038+0000","last_became_peered":"2024-04-26T23:38:51.713038+0000","last_unstale":"2024-04-26T23:38:52.698821+0000","last_undegraded":"2024-04-26T23:38:52.698821+0000","last_fullsized":"2024-04-26T23:38:52.698821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:21.439839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916125+0000","last_change":"2024-04-26T23:38:51.701247+0000","last_active":"2024-04-26T23:38:52.916125+0000","last_peered":"2024-04-26T23:38:52.916125+0000","last_clean":"2024-04-26T23:38:52.916125+0000","last_became_active":"2024-04-26T23:38:51.700921+0000","last_became_peered":"2024-04-26T23:38:51.700921+0000","last_unstale":"2024-04-26T23:38:52.916125+0000","last_undegraded":"2024-04-26T23:38:52.916125+0000","last_fullsized":"2024-04-26T23:38:52.916125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698168+0000","last_change":"2024-04-26T23:38:51.708914+0000","last_active":"2024-04-26T23:38:52.698168+0000","last_peered":"2024-04-26T23:38:52.698168+0000","last_clean":"2024-04-26T23:38:52.698168+0000","last_became_active":"2024-04-26T23:38:51.708535+0000","last_became_peered":"2024-04-26T23:38:51.708535+0000","last_unstale":"2024-04-26T23:38:52.698168+0000","last_undegraded":"2024-04-26T23:38:52.698168+0000","last_fullsized":"2024-04-26T23:38:52.698168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:13.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698029+0000","last_change":"2024-04-26T23:38:51.712495+0000","last_active":"2024-04-26T23:38:52.698029+0000","last_peered":"2024-04-26T23:38:52.698029+0000","last_clean":"2024-04-26T23:38:52.698029+0000","last_became_active":"2024-04-26T23:38:51.712155+0000","last_became_peered":"2024-04-26T23:38:51.712155+0000","last_unstale":"2024-04-26T23:38:52.698029+0000","last_undegraded":"2024-04-26T23:38:52.698029+0000","last_fullsized":"2024-04-26T23:38:52.698029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:31.825522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918147+0000","last_change":"2024-04-26T23:38:51.692347+0000","last_active":"2024-04-26T23:38:52.918147+0000","last_peered":"2024-04-26T23:38:52.918147+0000","last_clean":"2024-04-26T23:38:52.918147+0000","last_became_active":"2024-04-26T23:38:51.691440+0000","last_became_peered":"2024-04-26T23:38:51.691440+0000","last_unstale":"2024-04-26T23:38:52.918147+0000","last_undegraded":"2024-04-26T23:38:52.918147+0000","last_fullsized":"2024-04-26T23:38:52.918147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.563249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917783+0000","last_change":"2024-04-26T23:38:51.690092+0000","last_active":"2024-04-26T23:38:52.917783+0000","last_peered":"2024-04-26T23:38:52.917783+0000","last_clean":"2024-04-26T23:38:52.917783+0000","last_became_active":"2024-04-26T23:38:51.689821+0000","last_became_peered":"2024-04-26T23:38:51.689821+0000","last_unstale":"2024-04-26T23:38:52.917783+0000","last_undegraded":"2024-04-26T23:38:52.917783+0000","last_fullsized":"2024-04-26T23:38:52.917783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918113+0000","last_change":"2024-04-26T23:38:51.700456+0000","last_active":"2024-04-26T23:38:52.918113+0000","last_peered":"2024-04-26T23:38:52.918113+0000","last_clean":"2024-04-26T23:38:52.918113+0000","last_became_active":"2024-04-26T23:38:51.699994+0000","last_became_peered":"2024-04-26T23:38:51.699994+0000","last_unstale":"2024-04-26T23:38:52.918113+0000","last_undegraded":"2024-04-26T23:38:52.918113+0000","last_fullsized":"2024-04-26T23:38:52.918113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:14.397549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.699029+0000","last_change":"2024-04-26T23:38:50.691324+0000","last_active":"2024-04-26T23:38:52.699029+0000","last_peered":"2024-04-26T23:38:52.699029+0000","last_clean":"2024-04-26T23:38:52.699029+0000","last_became_active":"2024-04-26T23:38:50.691023+0000","last_became_peered":"2024-04-26T23:38:50.691023+0000","last_unstale":"2024-04-26T23:38:52.699029+0000","last_undegraded":"2024-04-26T23:38:52.699029+0000","last_fullsized":"2024-04-26T23:38:52.699029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:05.204342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919175+0000","last_change":"2024-04-26T23:38:51.707984+0000","last_active":"2024-04-26T23:38:52.919175+0000","last_peered":"2024-04-26T23:38:52.919175+0000","last_clean":"2024-04-26T23:38:52.919175+0000","last_became_active":"2024-04-26T23:38:51.705155+0000","last_became_peered":"2024-04-26T23:38:51.705155+0000","last_unstale":"2024-04-26T23:38:52.919175+0000","last_undegraded":"2024-04-26T23:38:52.919175+0000","last_fullsized":"2024-04-26T23:38:52.919175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:36.121201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692085+0000","last_change":"2024-04-26T23:38:50.691001+0000","last_active":"2024-04-26T23:38:52.692085+0000","last_peered":"2024-04-26T23:38:52.692085+0000","last_clean":"2024-04-26T23:38:52.692085+0000","last_became_active":"2024-04-26T23:38:50.690668+0000","last_became_peered":"2024-04-26T23:38:50.690668+0000","last_unstale":"2024-04-26T23:38:52.692085+0000","last_undegraded":"2024-04-26T23:38:52.692085+0000","last_fullsized":"2024-04-26T23:38:52.692085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696483+0000","last_change":"2024-04-26T23:38:51.701138+0000","last_active":"2024-04-26T23:38:52.696483+0000","last_peered":"2024-04-26T23:38:52.696483+0000","last_clean":"2024-04-26T23:38:52.696483+0000","last_became_active":"2024-04-26T23:38:51.700784+0000","last_became_peered":"2024-04-26T23:38:51.700784+0000","last_unstale":"2024-04-26T23:38:52.696483+0000","last_undegraded":"2024-04-26T23:38:52.696483+0000","last_fullsized":"2024-04-26T23:38:52.696483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919553+0000","last_change":"2024-04-26T23:38:50.689919+0000","last_active":"2024-04-26T23:38:52.919553+0000","last_peered":"2024-04-26T23:38:52.919553+0000","last_clean":"2024-04-26T23:38:52.919553+0000","last_became_active":"2024-04-26T23:38:50.689629+0000","last_became_peered":"2024-04-26T23:38:50.689629+0000","last_unstale":"2024-04-26T23:38:52.919553+0000","last_undegraded":"2024-04-26T23:38:52.919553+0000","last_fullsized":"2024-04-26T23:38:52.919553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916176+0000","last_change":"2024-04-26T23:38:51.701222+0000","last_active":"2024-04-26T23:38:52.916176+0000","last_peered":"2024-04-26T23:38:52.916176+0000","last_clean":"2024-04-26T23:38:52.916176+0000","last_became_active":"2024-04-26T23:38:51.700968+0000","last_became_peered":"2024-04-26T23:38:51.700968+0000","last_unstale":"2024-04-26T23:38:52.916176+0000","last_undegraded":"2024-04-26T23:38:52.916176+0000","last_fullsized":"2024-04-26T23:38:52.916176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:12.316705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697321+0000","last_change":"2024-04-26T23:38:50.684308+0000","last_active":"2024-04-26T23:38:52.697321+0000","last_peered":"2024-04-26T23:38:52.697321+0000","last_clean":"2024-04-26T23:38:52.697321+0000","last_became_active":"2024-04-26T23:38:50.684017+0000","last_became_peered":"2024-04-26T23:38:50.684017+0000","last_unstale":"2024-04-26T23:38:52.697321+0000","last_undegraded":"2024-04-26T23:38:52.697321+0000","last_fullsized":"2024-04-26T23:38:52.697321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691248+0000","last_change":"2024-04-26T23:38:51.683069+0000","last_active":"2024-04-26T23:38:52.691248+0000","last_peered":"2024-04-26T23:38:52.691248+0000","last_clean":"2024-04-26T23:38:52.691248+0000","last_became_active":"2024-04-26T23:38:51.682770+0000","last_became_peered":"2024-04-26T23:38:51.682770+0000","last_unstale":"2024-04-26T23:38:52.691248+0000","last_undegraded":"2024-04-26T23:38:52.691248+0000","last_fullsized":"2024-04-26T23:38:52.691248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916322+0000","last_change":"2024-04-26T23:38:50.687042+0000","last_active":"2024-04-26T23:38:52.916322+0000","last_peered":"2024-04-26T23:38:52.916322+0000","last_clean":"2024-04-26T23:38:52.916322+0000","last_became_active":"2024-04-26T23:38:50.685547+0000","last_became_peered":"2024-04-26T23:38:50.685547+0000","last_unstale":"2024-04-26T23:38:52.916322+0000","last_undegraded":"2024-04-26T23:38:52.916322+0000","last_fullsized":"2024-04-26T23:38:52.916322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915904+0000","last_change":"2024-04-26T23:38:51.713337+0000","last_active":"2024-04-26T23:38:52.915904+0000","last_peered":"2024-04-26T23:38:52.915904+0000","last_clean":"2024-04-26T23:38:52.915904+0000","last_became_active":"2024-04-26T23:38:51.709380+0000","last_became_peered":"2024-04-26T23:38:51.709380+0000","last_unstale":"2024-04-26T23:38:52.915904+0000","last_undegraded":"2024-04-26T23:38:52.915904+0000","last_fullsized":"2024-04-26T23:38:52.915904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:28.231903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917645+0000","last_change":"2024-04-26T23:38:50.695028+0000","last_active":"2024-04-26T23:38:52.917645+0000","last_peered":"2024-04-26T23:38:52.917645+0000","last_clean":"2024-04-26T23:38:52.917645+0000","last_became_active":"2024-04-26T23:38:50.694628+0000","last_became_peered":"2024-04-26T23:38:50.694628+0000","last_unstale":"2024-04-26T23:38:52.917645+0000","last_undegraded":"2024-04-26T23:38:52.917645+0000","last_fullsized":"2024-04-26T23:38:52.917645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:12.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696428+0000","last_change":"2024-04-26T23:38:51.715811+0000","last_active":"2024-04-26T23:38:52.696428+0000","last_peered":"2024-04-26T23:38:52.696428+0000","last_clean":"2024-04-26T23:38:52.696428+0000","last_became_active":"2024-04-26T23:38:51.715494+0000","last_became_peered":"2024-04-26T23:38:51.715494+0000","last_unstale":"2024-04-26T23:38:52.696428+0000","last_undegraded":"2024-04-26T23:38:52.696428+0000","last_fullsized":"2024-04-26T23:38:52.696428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918107+0000","last_change":"2024-04-26T23:38:50.698125+0000","last_active":"2024-04-26T23:38:52.918107+0000","last_peered":"2024-04-26T23:38:52.918107+0000","last_clean":"2024-04-26T23:38:52.918107+0000","last_became_active":"2024-04-26T23:38:50.697795+0000","last_became_peered":"2024-04-26T23:38:50.697795+0000","last_unstale":"2024-04-26T23:38:52.918107+0000","last_undegraded":"2024-04-26T23:38:52.918107+0000","last_fullsized":"2024-04-26T23:38:52.918107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914696+0000","last_change":"2024-04-26T23:38:51.690867+0000","last_active":"2024-04-26T23:38:52.914696+0000","last_peered":"2024-04-26T23:38:52.914696+0000","last_clean":"2024-04-26T23:38:52.914696+0000","last_became_active":"2024-04-26T23:38:51.690555+0000","last_became_peered":"2024-04-26T23:38:51.690555+0000","last_unstale":"2024-04-26T23:38:52.914696+0000","last_undegraded":"2024-04-26T23:38:52.914696+0000","last_fullsized":"2024-04-26T23:38:52.914696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695190+0000","last_change":"2024-04-26T23:38:50.680954+0000","last_active":"2024-04-26T23:38:52.695190+0000","last_peered":"2024-04-26T23:38:52.695190+0000","last_clean":"2024-04-26T23:38:52.695190+0000","last_became_active":"2024-04-26T23:38:50.680546+0000","last_became_peered":"2024-04-26T23:38:50.680546+0000","last_unstale":"2024-04-26T23:38:52.695190+0000","last_undegraded":"2024-04-26T23:38:52.695190+0000","last_fullsized":"2024-04-26T23:38:52.695190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917018+0000","last_change":"2024-04-26T23:38:51.701106+0000","last_active":"2024-04-26T23:38:52.917018+0000","last_peered":"2024-04-26T23:38:52.917018+0000","last_clean":"2024-04-26T23:38:52.917018+0000","last_became_active":"2024-04-26T23:38:51.697725+0000","last_became_peered":"2024-04-26T23:38:51.697725+0000","last_unstale":"2024-04-26T23:38:52.917018+0000","last_undegraded":"2024-04-26T23:38:52.917018+0000","last_fullsized":"2024-04-26T23:38:52.917018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:29.725135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916464+0000","last_change":"2024-04-26T23:38:50.694593+0000","last_active":"2024-04-26T23:38:52.916464+0000","last_peered":"2024-04-26T23:38:52.916464+0000","last_clean":"2024-04-26T23:38:52.916464+0000","last_became_active":"2024-04-26T23:38:50.694289+0000","last_became_peered":"2024-04-26T23:38:50.694289+0000","last_unstale":"2024-04-26T23:38:52.916464+0000","last_undegraded":"2024-04-26T23:38:52.916464+0000","last_fullsized":"2024-04-26T23:38:52.916464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:04.045799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918519+0000","last_change":"2024-04-26T23:38:51.718151+0000","last_active":"2024-04-26T23:38:52.918519+0000","last_peered":"2024-04-26T23:38:52.918519+0000","last_clean":"2024-04-26T23:38:52.918519+0000","last_became_active":"2024-04-26T23:38:51.717851+0000","last_became_peered":"2024-04-26T23:38:51.717851+0000","last_unstale":"2024-04-26T23:38:52.918519+0000","last_undegraded":"2024-04-26T23:38:52.918519+0000","last_fullsized":"2024-04-26T23:38:52.918519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917075+0000","last_change":"2024-04-26T23:38:50.696626+0000","last_active":"2024-04-26T23:38:52.917075+0000","last_peered":"2024-04-26T23:38:52.917075+0000","last_clean":"2024-04-26T23:38:52.917075+0000","last_became_active":"2024-04-26T23:38:50.696142+0000","last_became_peered":"2024-04-26T23:38:50.696142+0000","last_unstale":"2024-04-26T23:38:52.917075+0000","last_undegraded":"2024-04-26T23:38:52.917075+0000","last_fullsized":"2024-04-26T23:38:52.917075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692252+0000","last_change":"2024-04-26T23:38:51.684545+0000","last_active":"2024-04-26T23:38:52.692252+0000","last_peered":"2024-04-26T23:38:52.692252+0000","last_clean":"2024-04-26T23:38:52.692252+0000","last_became_active":"2024-04-26T23:38:51.684163+0000","last_became_peered":"2024-04-26T23:38:51.684163+0000","last_unstale":"2024-04-26T23:38:52.692252+0000","last_undegraded":"2024-04-26T23:38:52.692252+0000","last_fullsized":"2024-04-26T23:38:52.692252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:49.516055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916617+0000","last_change":"2024-04-26T23:38:50.695193+0000","last_active":"2024-04-26T23:38:52.916617+0000","last_peered":"2024-04-26T23:38:52.916617+0000","last_clean":"2024-04-26T23:38:52.916617+0000","last_became_active":"2024-04-26T23:38:50.694850+0000","last_became_peered":"2024-04-26T23:38:50.694850+0000","last_unstale":"2024-04-26T23:38:52.916617+0000","last_undegraded":"2024-04-26T23:38:52.916617+0000","last_fullsized":"2024-04-26T23:38:52.916617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697868+0000","last_change":"2024-04-26T23:38:51.705826+0000","last_active":"2024-04-26T23:38:52.697868+0000","last_peered":"2024-04-26T23:38:52.697868+0000","last_clean":"2024-04-26T23:38:52.697868+0000","last_became_active":"2024-04-26T23:38:51.705401+0000","last_became_peered":"2024-04-26T23:38:51.705401+0000","last_unstale":"2024-04-26T23:38:52.697868+0000","last_undegraded":"2024-04-26T23:38:52.697868+0000","last_fullsized":"2024-04-26T23:38:52.697868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914739+0000","last_change":"2024-04-26T23:38:50.695400+0000","last_active":"2024-04-26T23:38:52.914739+0000","last_peered":"2024-04-26T23:38:52.914739+0000","last_clean":"2024-04-26T23:38:52.914739+0000","last_became_active":"2024-04-26T23:38:50.695178+0000","last_became_peered":"2024-04-26T23:38:50.695178+0000","last_unstale":"2024-04-26T23:38:52.914739+0000","last_undegraded":"2024-04-26T23:38:52.914739+0000","last_fullsized":"2024-04-26T23:38:52.914739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.225065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691248+0000","last_change":"2024-04-26T23:38:51.704728+0000","last_active":"2024-04-26T23:38:52.691248+0000","last_peered":"2024-04-26T23:38:52.691248+0000","last_clean":"2024-04-26T23:38:52.691248+0000","last_became_active":"2024-04-26T23:38:51.702291+0000","last_became_peered":"2024-04-26T23:38:51.702291+0000","last_unstale":"2024-04-26T23:38:52.691248+0000","last_undegraded":"2024-04-26T23:38:52.691248+0000","last_fullsized":"2024-04-26T23:38:52.691248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915344+0000","last_change":"2024-04-26T23:38:50.695534+0000","last_active":"2024-04-26T23:38:52.915344+0000","last_peered":"2024-04-26T23:38:52.915344+0000","last_clean":"2024-04-26T23:38:52.915344+0000","last_became_active":"2024-04-26T23:38:50.686496+0000","last_became_peered":"2024-04-26T23:38:50.686496+0000","last_unstale":"2024-04-26T23:38:52.915344+0000","last_undegraded":"2024-04-26T23:38:52.915344+0000","last_fullsized":"2024-04-26T23:38:52.915344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:03.043110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914218+0000","last_change":"2024-04-26T23:38:51.698615+0000","last_active":"2024-04-26T23:38:52.914218+0000","last_peered":"2024-04-26T23:38:52.914218+0000","last_clean":"2024-04-26T23:38:52.914218+0000","last_became_active":"2024-04-26T23:38:51.698306+0000","last_became_peered":"2024-04-26T23:38:51.698306+0000","last_unstale":"2024-04-26T23:38:52.914218+0000","last_undegraded":"2024-04-26T23:38:52.914218+0000","last_fullsized":"2024-04-26T23:38:52.914218+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918006+0000","last_change":"2024-04-26T23:38:50.698660+0000","last_active":"2024-04-26T23:38:52.918006+0000","last_peered":"2024-04-26T23:38:52.918006+0000","last_clean":"2024-04-26T23:38:52.918006+0000","last_became_active":"2024-04-26T23:38:50.698295+0000","last_became_peered":"2024-04-26T23:38:50.698295+0000","last_unstale":"2024-04-26T23:38:52.918006+0000","last_undegraded":"2024-04-26T23:38:52.918006+0000","last_fullsized":"2024-04-26T23:38:52.918006+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917780+0000","last_change":"2024-04-26T23:38:51.690639+0000","last_active":"2024-04-26T23:38:52.917780+0000","last_peered":"2024-04-26T23:38:52.917780+0000","last_clean":"2024-04-26T23:38:52.917780+0000","last_became_active":"2024-04-26T23:38:51.690325+0000","last_became_peered":"2024-04-26T23:38:51.690325+0000","last_unstale":"2024-04-26T23:38:52.917780+0000","last_undegraded":"2024-04-26T23:38:52.917780+0000","last_fullsized":"2024-04-26T23:38:52.917780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:09.831540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697431+0000","last_change":"2024-04-26T23:38:50.686322+0000","last_active":"2024-04-26T23:38:52.697431+0000","last_peered":"2024-04-26T23:38:52.697431+0000","last_clean":"2024-04-26T23:38:52.697431+0000","last_became_active":"2024-04-26T23:38:50.684566+0000","last_became_peered":"2024-04-26T23:38:50.684566+0000","last_unstale":"2024-04-26T23:38:52.697431+0000","last_undegraded":"2024-04-26T23:38:52.697431+0000","last_fullsized":"2024-04-26T23:38:52.697431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:03.457234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915102+0000","last_change":"2024-04-26T23:38:51.702639+0000","last_active":"2024-04-26T23:38:52.915102+0000","last_peered":"2024-04-26T23:38:52.915102+0000","last_clean":"2024-04-26T23:38:52.915102+0000","last_became_active":"2024-04-26T23:38:51.701779+0000","last_became_peered":"2024-04-26T23:38:51.701779+0000","last_unstale":"2024-04-26T23:38:52.915102+0000","last_undegraded":"2024-04-26T23:38:52.915102+0000","last_fullsized":"2024-04-26T23:38:52.915102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:31.130045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917989+0000","last_change":"2024-04-26T23:38:50.694214+0000","last_active":"2024-04-26T23:38:52.917989+0000","last_peered":"2024-04-26T23:38:52.917989+0000","last_clean":"2024-04-26T23:38:52.917989+0000","last_became_active":"2024-04-26T23:38:50.693909+0000","last_became_peered":"2024-04-26T23:38:50.693909+0000","last_unstale":"2024-04-26T23:38:52.917989+0000","last_undegraded":"2024-04-26T23:38:52.917989+0000","last_fullsized":"2024-04-26T23:38:52.917989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914550+0000","last_change":"2024-04-26T23:38:51.717413+0000","last_active":"2024-04-26T23:38:52.914550+0000","last_peered":"2024-04-26T23:38:52.914550+0000","last_clean":"2024-04-26T23:38:52.914550+0000","last_became_active":"2024-04-26T23:38:51.717162+0000","last_became_peered":"2024-04-26T23:38:51.717162+0000","last_unstale":"2024-04-26T23:38:52.914550+0000","last_undegraded":"2024-04-26T23:38:52.914550+0000","last_fullsized":"2024-04-26T23:38:52.914550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917822+0000","last_change":"2024-04-26T23:38:50.686368+0000","last_active":"2024-04-26T23:38:52.917822+0000","last_peered":"2024-04-26T23:38:52.917822+0000","last_clean":"2024-04-26T23:38:52.917822+0000","last_became_active":"2024-04-26T23:38:50.686085+0000","last_became_peered":"2024-04-26T23:38:50.686085+0000","last_unstale":"2024-04-26T23:38:52.917822+0000","last_undegraded":"2024-04-26T23:38:52.917822+0000","last_fullsized":"2024-04-26T23:38:52.917822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:12.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916489+0000","last_change":"2024-04-26T23:38:51.704560+0000","last_active":"2024-04-26T23:38:52.916489+0000","last_peered":"2024-04-26T23:38:52.916489+0000","last_clean":"2024-04-26T23:38:52.916489+0000","last_became_active":"2024-04-26T23:38:51.702902+0000","last_became_peered":"2024-04-26T23:38:51.702902+0000","last_unstale":"2024-04-26T23:38:52.916489+0000","last_undegraded":"2024-04-26T23:38:52.916489+0000","last_fullsized":"2024-04-26T23:38:52.916489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:56.091602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697386+0000","last_change":"2024-04-26T23:38:50.685721+0000","last_active":"2024-04-26T23:38:52.697386+0000","last_peered":"2024-04-26T23:38:52.697386+0000","last_clean":"2024-04-26T23:38:52.697386+0000","last_became_active":"2024-04-26T23:38:50.685381+0000","last_became_peered":"2024-04-26T23:38:50.685381+0000","last_unstale":"2024-04-26T23:38:52.697386+0000","last_undegraded":"2024-04-26T23:38:52.697386+0000","last_fullsized":"2024-04-26T23:38:52.697386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696126+0000","last_change":"2024-04-26T23:38:51.698379+0000","last_active":"2024-04-26T23:38:52.696126+0000","last_peered":"2024-04-26T23:38:52.696126+0000","last_clean":"2024-04-26T23:38:52.696126+0000","last_became_active":"2024-04-26T23:38:51.698063+0000","last_became_peered":"2024-04-26T23:38:51.698063+0000","last_unstale":"2024-04-26T23:38:52.696126+0000","last_undegraded":"2024-04-26T23:38:52.696126+0000","last_fullsized":"2024-04-26T23:38:52.696126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:14.809468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916240+0000","last_change":"2024-04-26T23:38:50.690198+0000","last_active":"2024-04-26T23:38:52.916240+0000","last_peered":"2024-04-26T23:38:52.916240+0000","last_clean":"2024-04-26T23:38:52.916240+0000","last_became_active":"2024-04-26T23:38:50.689613+0000","last_became_peered":"2024-04-26T23:38:50.689613+0000","last_unstale":"2024-04-26T23:38:52.916240+0000","last_undegraded":"2024-04-26T23:38:52.916240+0000","last_fullsized":"2024-04-26T23:38:52.916240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:55.670095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916945+0000","last_change":"2024-04-26T23:38:51.704333+0000","last_active":"2024-04-26T23:38:52.916945+0000","last_peered":"2024-04-26T23:38:52.916945+0000","last_clean":"2024-04-26T23:38:52.916945+0000","last_became_active":"2024-04-26T23:38:51.704138+0000","last_became_peered":"2024-04-26T23:38:51.704138+0000","last_unstale":"2024-04-26T23:38:52.916945+0000","last_undegraded":"2024-04-26T23:38:52.916945+0000","last_fullsized":"2024-04-26T23:38:52.916945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:13.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919474+0000","last_change":"2024-04-26T23:38:50.691793+0000","last_active":"2024-04-26T23:38:52.919474+0000","last_peered":"2024-04-26T23:38:52.919474+0000","last_clean":"2024-04-26T23:38:52.919474+0000","last_became_active":"2024-04-26T23:38:50.691492+0000","last_became_peered":"2024-04-26T23:38:50.691492+0000","last_unstale":"2024-04-26T23:38:52.919474+0000","last_undegraded":"2024-04-26T23:38:52.919474+0000","last_fullsized":"2024-04-26T23:38:52.919474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918434+0000","last_change":"2024-04-26T23:38:51.691389+0000","last_active":"2024-04-26T23:38:52.918434+0000","last_peered":"2024-04-26T23:38:52.918434+0000","last_clean":"2024-04-26T23:38:52.918434+0000","last_became_active":"2024-04-26T23:38:51.690984+0000","last_became_peered":"2024-04-26T23:38:51.690984+0000","last_unstale":"2024-04-26T23:38:52.918434+0000","last_undegraded":"2024-04-26T23:38:52.918434+0000","last_fullsized":"2024-04-26T23:38:52.918434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:28.231903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917047+0000","last_change":"2024-04-26T23:38:50.687713+0000","last_active":"2024-04-26T23:38:52.917047+0000","last_peered":"2024-04-26T23:38:52.917047+0000","last_clean":"2024-04-26T23:38:52.917047+0000","last_became_active":"2024-04-26T23:38:50.687314+0000","last_became_peered":"2024-04-26T23:38:50.687314+0000","last_unstale":"2024-04-26T23:38:52.917047+0000","last_undegraded":"2024-04-26T23:38:52.917047+0000","last_fullsized":"2024-04-26T23:38:52.917047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:03.017735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691414+0000","last_change":"2024-04-26T23:38:51.685574+0000","last_active":"2024-04-26T23:38:52.691414+0000","last_peered":"2024-04-26T23:38:52.691414+0000","last_clean":"2024-04-26T23:38:52.691414+0000","last_became_active":"2024-04-26T23:38:51.685269+0000","last_became_peered":"2024-04-26T23:38:51.685269+0000","last_unstale":"2024-04-26T23:38:52.691414+0000","last_undegraded":"2024-04-26T23:38:52.691414+0000","last_fullsized":"2024-04-26T23:38:52.691414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917867+0000","last_change":"2024-04-26T23:38:50.682745+0000","last_active":"2024-04-26T23:38:52.917867+0000","last_peered":"2024-04-26T23:38:52.917867+0000","last_clean":"2024-04-26T23:38:52.917867+0000","last_became_active":"2024-04-26T23:38:50.682462+0000","last_became_peered":"2024-04-26T23:38:50.682462+0000","last_unstale":"2024-04-26T23:38:52.917867+0000","last_undegraded":"2024-04-26T23:38:52.917867+0000","last_fullsized":"2024-04-26T23:38:52.917867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:39.251641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915938+0000","last_change":"2024-04-26T23:38:51.716394+0000","last_active":"2024-04-26T23:38:52.915938+0000","last_peered":"2024-04-26T23:38:52.915938+0000","last_clean":"2024-04-26T23:38:52.915938+0000","last_became_active":"2024-04-26T23:38:51.712486+0000","last_became_peered":"2024-04-26T23:38:51.712486+0000","last_unstale":"2024-04-26T23:38:52.915938+0000","last_undegraded":"2024-04-26T23:38:52.915938+0000","last_fullsized":"2024-04-26T23:38:52.915938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918202+0000","last_change":"2024-04-26T23:38:50.699004+0000","last_active":"2024-04-26T23:38:52.918202+0000","last_peered":"2024-04-26T23:38:52.918202+0000","last_clean":"2024-04-26T23:38:52.918202+0000","last_became_active":"2024-04-26T23:38:50.698643+0000","last_became_peered":"2024-04-26T23:38:50.698643+0000","last_unstale":"2024-04-26T23:38:52.918202+0000","last_undegraded":"2024-04-26T23:38:52.918202+0000","last_fullsized":"2024-04-26T23:38:52.918202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915455+0000","last_change":"2024-04-26T23:38:51.687019+0000","last_active":"2024-04-26T23:38:52.915455+0000","last_peered":"2024-04-26T23:38:52.915455+0000","last_clean":"2024-04-26T23:38:52.915455+0000","last_became_active":"2024-04-26T23:38:51.686726+0000","last_became_peered":"2024-04-26T23:38:51.686726+0000","last_unstale":"2024-04-26T23:38:52.915455+0000","last_undegraded":"2024-04-26T23:38:52.915455+0000","last_fullsized":"2024-04-26T23:38:52.915455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:25.223837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916928+0000","last_change":"2024-04-26T23:38:50.688989+0000","last_active":"2024-04-26T23:38:52.916928+0000","last_peered":"2024-04-26T23:38:52.916928+0000","last_clean":"2024-04-26T23:38:52.916928+0000","last_became_active":"2024-04-26T23:38:50.688711+0000","last_became_peered":"2024-04-26T23:38:50.688711+0000","last_unstale":"2024-04-26T23:38:52.916928+0000","last_undegraded":"2024-04-26T23:38:52.916928+0000","last_fullsized":"2024-04-26T23:38:52.916928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:13.381141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919135+0000","last_change":"2024-04-26T23:38:51.706900+0000","last_active":"2024-04-26T23:38:52.919135+0000","last_peered":"2024-04-26T23:38:52.919135+0000","last_clean":"2024-04-26T23:38:52.919135+0000","last_became_active":"2024-04-26T23:38:51.705041+0000","last_became_peered":"2024-04-26T23:38:51.705041+0000","last_unstale":"2024-04-26T23:38:52.919135+0000","last_undegraded":"2024-04-26T23:38:52.919135+0000","last_fullsized":"2024-04-26T23:38:52.919135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917847+0000","last_change":"2024-04-26T23:38:50.693273+0000","last_active":"2024-04-26T23:38:52.917847+0000","last_peered":"2024-04-26T23:38:52.917847+0000","last_clean":"2024-04-26T23:38:52.917847+0000","last_became_active":"2024-04-26T23:38:50.692984+0000","last_became_peered":"2024-04-26T23:38:50.692984+0000","last_unstale":"2024-04-26T23:38:52.917847+0000","last_undegraded":"2024-04-26T23:38:52.917847+0000","last_fullsized":"2024-04-26T23:38:52.917847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:20.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918732+0000","last_change":"2024-04-26T23:38:51.702639+0000","last_active":"2024-04-26T23:38:52.918732+0000","last_peered":"2024-04-26T23:38:52.918732+0000","last_clean":"2024-04-26T23:38:52.918732+0000","last_became_active":"2024-04-26T23:38:51.702345+0000","last_became_peered":"2024-04-26T23:38:51.702345+0000","last_unstale":"2024-04-26T23:38:52.918732+0000","last_undegraded":"2024-04-26T23:38:52.918732+0000","last_fullsized":"2024-04-26T23:38:52.918732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916717+0000","last_change":"2024-04-26T23:38:50.691632+0000","last_active":"2024-04-26T23:38:52.916717+0000","last_peered":"2024-04-26T23:38:52.916717+0000","last_clean":"2024-04-26T23:38:52.916717+0000","last_became_active":"2024-04-26T23:38:50.691073+0000","last_became_peered":"2024-04-26T23:38:50.691073+0000","last_unstale":"2024-04-26T23:38:52.916717+0000","last_undegraded":"2024-04-26T23:38:52.916717+0000","last_fullsized":"2024-04-26T23:38:52.916717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:00.898736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915208+0000","last_change":"2024-04-26T23:38:51.694847+0000","last_active":"2024-04-26T23:38:52.915208+0000","last_peered":"2024-04-26T23:38:52.915208+0000","last_clean":"2024-04-26T23:38:52.915208+0000","last_became_active":"2024-04-26T23:38:51.694541+0000","last_became_peered":"2024-04-26T23:38:51.694541+0000","last_unstale":"2024-04-26T23:38:52.915208+0000","last_undegraded":"2024-04-26T23:38:52.915208+0000","last_fullsized":"2024-04-26T23:38:52.915208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:07.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691887+0000","last_change":"2024-04-26T23:38:50.686345+0000","last_active":"2024-04-26T23:38:52.691887+0000","last_peered":"2024-04-26T23:38:52.691887+0000","last_clean":"2024-04-26T23:38:52.691887+0000","last_became_active":"2024-04-26T23:38:50.685727+0000","last_became_peered":"2024-04-26T23:38:50.685727+0000","last_unstale":"2024-04-26T23:38:52.691887+0000","last_undegraded":"2024-04-26T23:38:52.691887+0000","last_fullsized":"2024-04-26T23:38:52.691887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691507+0000","last_change":"2024-04-26T23:38:51.684652+0000","last_active":"2024-04-26T23:38:52.691507+0000","last_peered":"2024-04-26T23:38:52.691507+0000","last_clean":"2024-04-26T23:38:52.691507+0000","last_became_active":"2024-04-26T23:38:51.684329+0000","last_became_peered":"2024-04-26T23:38:51.684329+0000","last_unstale":"2024-04-26T23:38:52.691507+0000","last_undegraded":"2024-04-26T23:38:52.691507+0000","last_fullsized":"2024-04-26T23:38:52.691507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691668+0000","last_change":"2024-04-26T23:38:50.688232+0000","last_active":"2024-04-26T23:38:52.691668+0000","last_peered":"2024-04-26T23:38:52.691668+0000","last_clean":"2024-04-26T23:38:52.691668+0000","last_became_active":"2024-04-26T23:38:50.687890+0000","last_became_peered":"2024-04-26T23:38:50.687890+0000","last_unstale":"2024-04-26T23:38:52.691668+0000","last_undegraded":"2024-04-26T23:38:52.691668+0000","last_fullsized":"2024-04-26T23:38:52.691668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918086+0000","last_change":"2024-04-26T23:38:51.714014+0000","last_active":"2024-04-26T23:38:52.918086+0000","last_peered":"2024-04-26T23:38:52.918086+0000","last_clean":"2024-04-26T23:38:52.918086+0000","last_became_active":"2024-04-26T23:38:51.713684+0000","last_became_peered":"2024-04-26T23:38:51.713684+0000","last_unstale":"2024-04-26T23:38:52.918086+0000","last_undegraded":"2024-04-26T23:38:52.918086+0000","last_fullsized":"2024-04-26T23:38:52.918086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698225+0000","last_change":"2024-04-26T23:38:50.688814+0000","last_active":"2024-04-26T23:38:52.698225+0000","last_peered":"2024-04-26T23:38:52.698225+0000","last_clean":"2024-04-26T23:38:52.698225+0000","last_became_active":"2024-04-26T23:38:50.688521+0000","last_became_peered":"2024-04-26T23:38:50.688521+0000","last_unstale":"2024-04-26T23:38:52.698225+0000","last_undegraded":"2024-04-26T23:38:52.698225+0000","last_fullsized":"2024-04-26T23:38:52.698225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:51.613896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695819+0000","last_change":"2024-04-26T23:38:51.704947+0000","last_active":"2024-04-26T23:38:52.695819+0000","last_peered":"2024-04-26T23:38:52.695819+0000","last_clean":"2024-04-26T23:38:52.695819+0000","last_became_active":"2024-04-26T23:38:51.704070+0000","last_became_peered":"2024-04-26T23:38:51.704070+0000","last_unstale":"2024-04-26T23:38:52.695819+0000","last_undegraded":"2024-04-26T23:38:52.695819+0000","last_fullsized":"2024-04-26T23:38:52.695819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.677975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695798+0000","last_change":"2024-04-26T23:38:50.678514+0000","last_active":"2024-04-26T23:38:52.695798+0000","last_peered":"2024-04-26T23:38:52.695798+0000","last_clean":"2024-04-26T23:38:52.695798+0000","last_became_active":"2024-04-26T23:38:50.678094+0000","last_became_peered":"2024-04-26T23:38:50.678094+0000","last_unstale":"2024-04-26T23:38:52.695798+0000","last_undegraded":"2024-04-26T23:38:52.695798+0000","last_fullsized":"2024-04-26T23:38:52.695798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919036+0000","last_change":"2024-04-26T23:38:51.710701+0000","last_active":"2024-04-26T23:38:52.919036+0000","last_peered":"2024-04-26T23:38:52.919036+0000","last_clean":"2024-04-26T23:38:52.919036+0000","last_became_active":"2024-04-26T23:38:51.710457+0000","last_became_peered":"2024-04-26T23:38:51.710457+0000","last_unstale":"2024-04-26T23:38:52.919036+0000","last_undegraded":"2024-04-26T23:38:52.919036+0000","last_fullsized":"2024-04-26T23:38:52.919036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914125+0000","last_change":"2024-04-26T23:38:50.685335+0000","last_active":"2024-04-26T23:38:52.914125+0000","last_peered":"2024-04-26T23:38:52.914125+0000","last_clean":"2024-04-26T23:38:52.914125+0000","last_became_active":"2024-04-26T23:38:50.685055+0000","last_became_peered":"2024-04-26T23:38:50.685055+0000","last_unstale":"2024-04-26T23:38:52.914125+0000","last_undegraded":"2024-04-26T23:38:52.914125+0000","last_fullsized":"2024-04-26T23:38:52.914125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:13.793507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917354+0000","last_change":"2024-04-26T23:38:51.707157+0000","last_active":"2024-04-26T23:38:52.917354+0000","last_peered":"2024-04-26T23:38:52.917354+0000","last_clean":"2024-04-26T23:38:52.917354+0000","last_became_active":"2024-04-26T23:38:51.706848+0000","last_became_peered":"2024-04-26T23:38:51.706848+0000","last_unstale":"2024-04-26T23:38:52.917354+0000","last_undegraded":"2024-04-26T23:38:52.917354+0000","last_fullsized":"2024-04-26T23:38:52.917354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:07.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918381+0000","last_change":"2024-04-26T23:38:50.694147+0000","last_active":"2024-04-26T23:38:52.918381+0000","last_peered":"2024-04-26T23:38:52.918381+0000","last_clean":"2024-04-26T23:38:52.918381+0000","last_became_active":"2024-04-26T23:38:50.692579+0000","last_became_peered":"2024-04-26T23:38:50.692579+0000","last_unstale":"2024-04-26T23:38:52.918381+0000","last_undegraded":"2024-04-26T23:38:52.918381+0000","last_fullsized":"2024-04-26T23:38:52.918381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:20.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918735+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:38:52.918735+0000","last_peered":"2024-04-26T23:38:52.918735+0000","last_clean":"2024-04-26T23:38:52.918735+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:38:52.918735+0000","last_undegraded":"2024-04-26T23:38:52.918735+0000","last_fullsized":"2024-04-26T23:38:52.918735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":786749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":196345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":785101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:39:01.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:39:01.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.576+0000 7f627e600640 1 Processor -- start 2024-04-26T23:39:01.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.577+0000 7f627e600640 1 -- start start 2024-04-26T23:39:01.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.578+0000 7f627e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f62780d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.578+0000 7f627e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62780b4000 0x7f62780d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.578+0000 7f627e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62780d11f0 con 0x7f62780d1f60 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.578+0000 7f627e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f62780d2990 con 0x7f62780b4000 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.578+0000 7f627e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f62780d5f20 con 0x7f627815f4e0 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f627dc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f62780d1f60 0x7f62780cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53426/0 (socket says 172.21.15.46:53426) 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f627dc00640 1 -- 172.21.15.46:0/2456602777 learned_addr learned my addr 172.21.15.46:0/2456602777 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:39:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f6277e00640 1 --2- 172.21.15.46:0/2456602777 >> v2:172.21.15.46:3300/0 conn(0x7f62780b4000 0x7f62780d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f627d200640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f62780d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f6277400640 1 -- 172.21.15.46:0/2456602777 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336414845 0 0) 0x7f62780d11f0 con 0x7f62780d1f60 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f6277400640 1 -- 172.21.15.46:0/2456602777 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f625c003520 con 0x7f62780d1f60 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f6277e00640 1 -- 172.21.15.46:0/2456602777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 msgr2=0x7f62780d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.579+0000 7f6277e00640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f62780d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.580+0000 7f6277e00640 1 -- 172.21.15.46:0/2456602777 >> v1:172.21.15.46:6789/0 conn(0x7f62780d1f60 legacy=0x7f62780cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.580+0000 7f6277e00640 1 -- 172.21.15.46:0/2456602777 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62780d0f10 con 0x7f62780b4000 2024-04-26T23:39:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.580+0000 7f627d200640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f62780d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:39:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.581+0000 7f6277e00640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780d1a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6260009d90 tx=0x7f626002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9a9f97ba606c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.581+0000 7f6277400640 1 -- 172.21.15.46:0/2456602777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260007740 con 0x7f62780b4000 2024-04-26T23:39:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.581+0000 7f6277400640 1 -- 172.21.15.46:0/2456602777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62600078e0 con 0x7f62780b4000 2024-04-26T23:39:01.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.581+0000 7f6277400640 1 -- 172.21.15.46:0/2456602777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260007c90 con 0x7f62780b4000 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 -- 172.21.15.46:0/2456602777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 msgr2=0x7f62780d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780d1a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6260009d90 tx=0x7f626002a660 comp rx=0 tx=0).stop 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 -- 172.21.15.46:0/2456602777 shutdown_connections 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780d1a20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 --2- 172.21.15.46:0/2456602777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f62780d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 -- 172.21.15.46:0/2456602777 >> 172.21.15.46:0/2456602777 conn(0x7f62780d5010 msgr2=0x7f62780d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 -- 172.21.15.46:0/2456602777 shutdown_connections 2024-04-26T23:39:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.582+0000 7f627e600640 1 -- 172.21.15.46:0/2456602777 wait complete. 2024-04-26T23:39:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.584+0000 7f627e600640 1 Processor -- start 2024-04-26T23:39:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.584+0000 7f627e600640 1 -- start start 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 0x7f62780b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f6278155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f627815f8c0 con 0x7f62780b4000 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62780b1d60 con 0x7f62780d1f60 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f62780b3290 con 0x7f627815f4e0 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42598/0 (socket says 172.21.15.46:42598) 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.585+0000 7f627d200640 1 -- 172.21.15.46:0/663760820 learned_addr learned my addr 172.21.15.46:0/663760820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 -- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 msgr2=0x7f6278155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f6278155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 -- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 msgr2=0x7f62780b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f6277e00640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 0x7f62780b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 0x7f62780b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 -- 172.21.15.46:0/663760820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6278155980 con 0x7f62780b4000 2024-04-26T23:39:01.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f6277e00640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 0x7f62780b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:01.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.586+0000 7f627d200640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f626800d800 tx=0x7f626800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.587+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268004030 con 0x7f62780b4000 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.587+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62680041d0 con 0x7f62780b4000 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.587+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62680096a0 con 0x7f62780b4000 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.587+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62780cd360 con 0x7f62780b4000 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.587+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62780cd920 con 0x7f62780b4000 2024-04-26T23:39:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.588+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62780693b0 con 0x7f62780b4000 2024-04-26T23:39:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.589+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6268009840 con 0x7f62780b4000 2024-04-26T23:39:01.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.590+0000 7f6274c00640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 0x7f625406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:01.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.591+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f626808c040 con 0x7f62780b4000 2024-04-26T23:39:01.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.591+0000 7f6277e00640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 0x7f625406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:01.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.592+0000 7f6277e00640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 0x7f625406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6260034000 tx=0x7f626002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:01.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.597+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6268011340 con 0x7f62780b4000 2024-04-26T23:39:01.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.795+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62780d33d0 con 0x7f625406c8f0 2024-04-26T23:39:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.814+0000 7f6274c00640 1 -- 172.21.15.46:0/663760820 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f62780d33d0 con 0x7f625406c8f0 2024-04-26T23:39:01.821 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:39:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:39:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 msgr2=0x7f625406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 0x7f625406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6260034000 tx=0x7f626002f000 comp rx=0 tx=0).stop 2024-04-26T23:39:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 msgr2=0x7f62780ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f626800d800 tx=0x7f626800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 shutdown_connections 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f625406c8f0 0x7f625406edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f627815f4e0 0x7f6278155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62780d1f60 0x7f62780b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 --2- 172.21.15.46:0/663760820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62780b4000 0x7f62780ccc80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 >> 172.21.15.46:0/663760820 conn(0x7f62780d5010 msgr2=0x7f62780cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 shutdown_connections 2024-04-26T23:39:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:01.821+0000 7f627e600640 1 -- 172.21.15.46:0/663760820 wait complete. 2024-04-26T23:39:01.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-04-26T23:38:59.923890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237580,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737140,"statfs":{"total":773094113280,"available":772850831360,"internally_reserved":0,"allocated":4857856,"data_stored":2743148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.262058"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916247+0000","last_change":"2024-04-26T23:38:51.700162+0000","last_active":"2024-04-26T23:38:52.916247+0000","last_peered":"2024-04-26T23:38:52.916247+0000","last_clean":"2024-04-26T23:38:52.916247+0000","last_became_active":"2024-04-26T23:38:51.699746+0000","last_became_peered":"2024-04-26T23:38:51.699746+0000","last_unstale":"2024-04-26T23:38:52.916247+0000","last_undegraded":"2024-04-26T23:38:52.916247+0000","last_fullsized":"2024-04-26T23:38:52.916247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:30.479535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914828+0000","last_change":"2024-04-26T23:38:51.692550+0000","last_active":"2024-04-26T23:38:52.914828+0000","last_peered":"2024-04-26T23:38:52.914828+0000","last_clean":"2024-04-26T23:38:52.914828+0000","last_became_active":"2024-04-26T23:38:51.692217+0000","last_became_peered":"2024-04-26T23:38:51.692217+0000","last_unstale":"2024-04-26T23:38:52.914828+0000","last_undegraded":"2024-04-26T23:38:52.914828+0000","last_fullsized":"2024-04-26T23:38:52.914828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:56.091602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691779+0000","last_change":"2024-04-26T23:38:51.706273+0000","last_active":"2024-04-26T23:38:52.691779+0000","last_peered":"2024-04-26T23:38:52.691779+0000","last_clean":"2024-04-26T23:38:52.691779+0000","last_became_active":"2024-04-26T23:38:51.705967+0000","last_became_peered":"2024-04-26T23:38:51.705967+0000","last_unstale":"2024-04-26T23:38:52.691779+0000","last_undegraded":"2024-04-26T23:38:52.691779+0000","last_fullsized":"2024-04-26T23:38:52.691779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.202801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916540+0000","last_change":"2024-04-26T23:38:51.700480+0000","last_active":"2024-04-26T23:38:52.916540+0000","last_peered":"2024-04-26T23:38:52.916540+0000","last_clean":"2024-04-26T23:38:52.916540+0000","last_became_active":"2024-04-26T23:38:51.697403+0000","last_became_peered":"2024-04-26T23:38:51.697403+0000","last_unstale":"2024-04-26T23:38:52.916540+0000","last_undegraded":"2024-04-26T23:38:52.916540+0000","last_fullsized":"2024-04-26T23:38:52.916540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:00.855520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916775+0000","last_change":"2024-04-26T23:38:51.696355+0000","last_active":"2024-04-26T23:38:52.916775+0000","last_peered":"2024-04-26T23:38:52.916775+0000","last_clean":"2024-04-26T23:38:52.916775+0000","last_became_active":"2024-04-26T23:38:51.696019+0000","last_became_peered":"2024-04-26T23:38:51.696019+0000","last_unstale":"2024-04-26T23:38:52.916775+0000","last_undegraded":"2024-04-26T23:38:52.916775+0000","last_fullsized":"2024-04-26T23:38:52.916775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:31.825522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918037+0000","last_change":"2024-04-26T23:38:51.686585+0000","last_active":"2024-04-26T23:38:52.918037+0000","last_peered":"2024-04-26T23:38:52.918037+0000","last_clean":"2024-04-26T23:38:52.918037+0000","last_became_active":"2024-04-26T23:38:51.686296+0000","last_became_peered":"2024-04-26T23:38:51.686296+0000","last_unstale":"2024-04-26T23:38:52.918037+0000","last_undegraded":"2024-04-26T23:38:52.918037+0000","last_fullsized":"2024-04-26T23:38:52.918037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915473+0000","last_change":"2024-04-26T23:38:51.711658+0000","last_active":"2024-04-26T23:38:52.915473+0000","last_peered":"2024-04-26T23:38:52.915473+0000","last_clean":"2024-04-26T23:38:52.915473+0000","last_became_active":"2024-04-26T23:38:51.709010+0000","last_became_peered":"2024-04-26T23:38:51.709010+0000","last_unstale":"2024-04-26T23:38:52.915473+0000","last_undegraded":"2024-04-26T23:38:52.915473+0000","last_fullsized":"2024-04-26T23:38:52.915473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:14.397549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914187+0000","last_change":"2024-04-26T23:38:51.705613+0000","last_active":"2024-04-26T23:38:52.914187+0000","last_peered":"2024-04-26T23:38:52.914187+0000","last_clean":"2024-04-26T23:38:52.914187+0000","last_became_active":"2024-04-26T23:38:51.705328+0000","last_became_peered":"2024-04-26T23:38:51.705328+0000","last_unstale":"2024-04-26T23:38:52.914187+0000","last_undegraded":"2024-04-26T23:38:52.914187+0000","last_fullsized":"2024-04-26T23:38:52.914187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692261+0000","last_change":"2024-04-26T23:38:51.710316+0000","last_active":"2024-04-26T23:38:52.692261+0000","last_peered":"2024-04-26T23:38:52.692261+0000","last_clean":"2024-04-26T23:38:52.692261+0000","last_became_active":"2024-04-26T23:38:51.710040+0000","last_became_peered":"2024-04-26T23:38:51.710040+0000","last_unstale":"2024-04-26T23:38:52.692261+0000","last_undegraded":"2024-04-26T23:38:52.692261+0000","last_fullsized":"2024-04-26T23:38:52.692261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:03.121714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916761+0000","last_change":"2024-04-26T23:38:51.706764+0000","last_active":"2024-04-26T23:38:52.916761+0000","last_peered":"2024-04-26T23:38:52.916761+0000","last_clean":"2024-04-26T23:38:52.916761+0000","last_became_active":"2024-04-26T23:38:51.703609+0000","last_became_peered":"2024-04-26T23:38:51.703609+0000","last_unstale":"2024-04-26T23:38:52.916761+0000","last_undegraded":"2024-04-26T23:38:52.916761+0000","last_fullsized":"2024-04-26T23:38:52.916761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:31.130045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916341+0000","last_change":"2024-04-26T23:38:51.704822+0000","last_active":"2024-04-26T23:38:52.916341+0000","last_peered":"2024-04-26T23:38:52.916341+0000","last_clean":"2024-04-26T23:38:52.916341+0000","last_became_active":"2024-04-26T23:38:51.703220+0000","last_became_peered":"2024-04-26T23:38:51.703220+0000","last_unstale":"2024-04-26T23:38:52.916341+0000","last_undegraded":"2024-04-26T23:38:52.916341+0000","last_fullsized":"2024-04-26T23:38:52.916341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691729+0000","last_change":"2024-04-26T23:38:51.695761+0000","last_active":"2024-04-26T23:38:52.691729+0000","last_peered":"2024-04-26T23:38:52.691729+0000","last_clean":"2024-04-26T23:38:52.691729+0000","last_became_active":"2024-04-26T23:38:51.695439+0000","last_became_peered":"2024-04-26T23:38:51.695439+0000","last_unstale":"2024-04-26T23:38:52.691729+0000","last_undegraded":"2024-04-26T23:38:52.691729+0000","last_fullsized":"2024-04-26T23:38:52.691729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:36.121201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915646+0000","last_change":"2024-04-26T23:38:51.701482+0000","last_active":"2024-04-26T23:38:52.915646+0000","last_peered":"2024-04-26T23:38:52.915646+0000","last_clean":"2024-04-26T23:38:52.915646+0000","last_became_active":"2024-04-26T23:38:51.701126+0000","last_became_peered":"2024-04-26T23:38:51.701126+0000","last_unstale":"2024-04-26T23:38:52.915646+0000","last_undegraded":"2024-04-26T23:38:52.915646+0000","last_fullsized":"2024-04-26T23:38:52.915646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:19.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695713+0000","last_change":"2024-04-26T23:38:51.714648+0000","last_active":"2024-04-26T23:38:52.695713+0000","last_peered":"2024-04-26T23:38:52.695713+0000","last_clean":"2024-04-26T23:38:52.695713+0000","last_became_active":"2024-04-26T23:38:51.713949+0000","last_became_peered":"2024-04-26T23:38:51.713949+0000","last_unstale":"2024-04-26T23:38:52.695713+0000","last_undegraded":"2024-04-26T23:38:52.695713+0000","last_fullsized":"2024-04-26T23:38:52.695713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:23.313646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915568+0000","last_change":"2024-04-26T23:38:51.713558+0000","last_active":"2024-04-26T23:38:52.915568+0000","last_peered":"2024-04-26T23:38:52.915568+0000","last_clean":"2024-04-26T23:38:52.915568+0000","last_became_active":"2024-04-26T23:38:51.708077+0000","last_became_peered":"2024-04-26T23:38:51.708077+0000","last_unstale":"2024-04-26T23:38:52.915568+0000","last_undegraded":"2024-04-26T23:38:52.915568+0000","last_fullsized":"2024-04-26T23:38:52.915568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.563249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919169+0000","last_change":"2024-04-26T23:38:51.708323+0000","last_active":"2024-04-26T23:38:52.919169+0000","last_peered":"2024-04-26T23:38:52.919169+0000","last_clean":"2024-04-26T23:38:52.919169+0000","last_became_active":"2024-04-26T23:38:51.706314+0000","last_became_peered":"2024-04-26T23:38:51.706314+0000","last_unstale":"2024-04-26T23:38:52.919169+0000","last_undegraded":"2024-04-26T23:38:52.919169+0000","last_fullsized":"2024-04-26T23:38:52.919169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918225+0000","last_change":"2024-04-26T23:38:51.706767+0000","last_active":"2024-04-26T23:38:52.918225+0000","last_peered":"2024-04-26T23:38:52.918225+0000","last_clean":"2024-04-26T23:38:52.918225+0000","last_became_active":"2024-04-26T23:38:51.706403+0000","last_became_peered":"2024-04-26T23:38:51.706403+0000","last_unstale":"2024-04-26T23:38:52.918225+0000","last_undegraded":"2024-04-26T23:38:52.918225+0000","last_fullsized":"2024-04-26T23:38:52.918225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:19.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698688+0000","last_change":"2024-04-26T23:38:51.704154+0000","last_active":"2024-04-26T23:38:52.698688+0000","last_peered":"2024-04-26T23:38:52.698688+0000","last_clean":"2024-04-26T23:38:52.698688+0000","last_became_active":"2024-04-26T23:38:51.703729+0000","last_became_peered":"2024-04-26T23:38:51.703729+0000","last_unstale":"2024-04-26T23:38:52.698688+0000","last_undegraded":"2024-04-26T23:38:52.698688+0000","last_fullsized":"2024-04-26T23:38:52.698688+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.135646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916131+0000","last_change":"2024-04-26T23:38:51.693174+0000","last_active":"2024-04-26T23:38:52.916131+0000","last_peered":"2024-04-26T23:38:52.916131+0000","last_clean":"2024-04-26T23:38:52.916131+0000","last_became_active":"2024-04-26T23:38:51.692843+0000","last_became_peered":"2024-04-26T23:38:51.692843+0000","last_unstale":"2024-04-26T23:38:52.916131+0000","last_undegraded":"2024-04-26T23:38:52.916131+0000","last_fullsized":"2024-04-26T23:38:52.916131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:37.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698304+0000","last_change":"2024-04-26T23:38:51.712580+0000","last_active":"2024-04-26T23:38:52.698304+0000","last_peered":"2024-04-26T23:38:52.698304+0000","last_clean":"2024-04-26T23:38:52.698304+0000","last_became_active":"2024-04-26T23:38:51.711933+0000","last_became_peered":"2024-04-26T23:38:51.711933+0000","last_unstale":"2024-04-26T23:38:52.698304+0000","last_undegraded":"2024-04-26T23:38:52.698304+0000","last_fullsized":"2024-04-26T23:38:52.698304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:29.725135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916021+0000","last_change":"2024-04-26T23:38:51.704480+0000","last_active":"2024-04-26T23:38:52.916021+0000","last_peered":"2024-04-26T23:38:52.916021+0000","last_clean":"2024-04-26T23:38:52.916021+0000","last_became_active":"2024-04-26T23:38:51.700760+0000","last_became_peered":"2024-04-26T23:38:51.700760+0000","last_unstale":"2024-04-26T23:38:52.916021+0000","last_undegraded":"2024-04-26T23:38:52.916021+0000","last_fullsized":"2024-04-26T23:38:52.916021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696264+0000","last_change":"2024-04-26T23:38:51.703706+0000","last_active":"2024-04-26T23:38:52.696264+0000","last_peered":"2024-04-26T23:38:52.696264+0000","last_clean":"2024-04-26T23:38:52.696264+0000","last_became_active":"2024-04-26T23:38:51.703118+0000","last_became_peered":"2024-04-26T23:38:51.703118+0000","last_unstale":"2024-04-26T23:38:52.696264+0000","last_undegraded":"2024-04-26T23:38:52.696264+0000","last_fullsized":"2024-04-26T23:38:52.696264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917687+0000","last_change":"2024-04-26T23:38:51.692232+0000","last_active":"2024-04-26T23:38:52.917687+0000","last_peered":"2024-04-26T23:38:52.917687+0000","last_clean":"2024-04-26T23:38:52.917687+0000","last_became_active":"2024-04-26T23:38:51.691045+0000","last_became_peered":"2024-04-26T23:38:51.691045+0000","last_unstale":"2024-04-26T23:38:52.917687+0000","last_undegraded":"2024-04-26T23:38:52.917687+0000","last_fullsized":"2024-04-26T23:38:52.917687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:49.516055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916650+0000","last_change":"2024-04-26T23:38:51.694705+0000","last_active":"2024-04-26T23:38:52.916650+0000","last_peered":"2024-04-26T23:38:52.916650+0000","last_clean":"2024-04-26T23:38:52.916650+0000","last_became_active":"2024-04-26T23:38:51.693597+0000","last_became_peered":"2024-04-26T23:38:51.693597+0000","last_unstale":"2024-04-26T23:38:52.916650+0000","last_undegraded":"2024-04-26T23:38:52.916650+0000","last_fullsized":"2024-04-26T23:38:52.916650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:19.771077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698999+0000","last_change":"2024-04-26T23:38:51.716170+0000","last_active":"2024-04-26T23:38:52.698999+0000","last_peered":"2024-04-26T23:38:52.698999+0000","last_clean":"2024-04-26T23:38:52.698999+0000","last_became_active":"2024-04-26T23:38:51.715894+0000","last_became_peered":"2024-04-26T23:38:51.715894+0000","last_unstale":"2024-04-26T23:38:52.698999+0000","last_undegraded":"2024-04-26T23:38:52.698999+0000","last_fullsized":"2024-04-26T23:38:52.698999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:36.414404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919230+0000","last_change":"2024-04-26T23:38:51.706456+0000","last_active":"2024-04-26T23:38:52.919230+0000","last_peered":"2024-04-26T23:38:52.919230+0000","last_clean":"2024-04-26T23:38:52.919230+0000","last_became_active":"2024-04-26T23:38:51.706161+0000","last_became_peered":"2024-04-26T23:38:51.706161+0000","last_unstale":"2024-04-26T23:38:52.919230+0000","last_undegraded":"2024-04-26T23:38:52.919230+0000","last_fullsized":"2024-04-26T23:38:52.919230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.202801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698821+0000","last_change":"2024-04-26T23:38:51.713320+0000","last_active":"2024-04-26T23:38:52.698821+0000","last_peered":"2024-04-26T23:38:52.698821+0000","last_clean":"2024-04-26T23:38:52.698821+0000","last_became_active":"2024-04-26T23:38:51.713038+0000","last_became_peered":"2024-04-26T23:38:51.713038+0000","last_unstale":"2024-04-26T23:38:52.698821+0000","last_undegraded":"2024-04-26T23:38:52.698821+0000","last_fullsized":"2024-04-26T23:38:52.698821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:21.439839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916125+0000","last_change":"2024-04-26T23:38:51.701247+0000","last_active":"2024-04-26T23:38:52.916125+0000","last_peered":"2024-04-26T23:38:52.916125+0000","last_clean":"2024-04-26T23:38:52.916125+0000","last_became_active":"2024-04-26T23:38:51.700921+0000","last_became_peered":"2024-04-26T23:38:51.700921+0000","last_unstale":"2024-04-26T23:38:52.916125+0000","last_undegraded":"2024-04-26T23:38:52.916125+0000","last_fullsized":"2024-04-26T23:38:52.916125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698168+0000","last_change":"2024-04-26T23:38:51.708914+0000","last_active":"2024-04-26T23:38:52.698168+0000","last_peered":"2024-04-26T23:38:52.698168+0000","last_clean":"2024-04-26T23:38:52.698168+0000","last_became_active":"2024-04-26T23:38:51.708535+0000","last_became_peered":"2024-04-26T23:38:51.708535+0000","last_unstale":"2024-04-26T23:38:52.698168+0000","last_undegraded":"2024-04-26T23:38:52.698168+0000","last_fullsized":"2024-04-26T23:38:52.698168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:13.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698029+0000","last_change":"2024-04-26T23:38:51.712495+0000","last_active":"2024-04-26T23:38:52.698029+0000","last_peered":"2024-04-26T23:38:52.698029+0000","last_clean":"2024-04-26T23:38:52.698029+0000","last_became_active":"2024-04-26T23:38:51.712155+0000","last_became_peered":"2024-04-26T23:38:51.712155+0000","last_unstale":"2024-04-26T23:38:52.698029+0000","last_undegraded":"2024-04-26T23:38:52.698029+0000","last_fullsized":"2024-04-26T23:38:52.698029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:31.825522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918147+0000","last_change":"2024-04-26T23:38:51.692347+0000","last_active":"2024-04-26T23:38:52.918147+0000","last_peered":"2024-04-26T23:38:52.918147+0000","last_clean":"2024-04-26T23:38:52.918147+0000","last_became_active":"2024-04-26T23:38:51.691440+0000","last_became_peered":"2024-04-26T23:38:51.691440+0000","last_unstale":"2024-04-26T23:38:52.918147+0000","last_undegraded":"2024-04-26T23:38:52.918147+0000","last_fullsized":"2024-04-26T23:38:52.918147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.563249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917783+0000","last_change":"2024-04-26T23:38:51.690092+0000","last_active":"2024-04-26T23:38:52.917783+0000","last_peered":"2024-04-26T23:38:52.917783+0000","last_clean":"2024-04-26T23:38:52.917783+0000","last_became_active":"2024-04-26T23:38:51.689821+0000","last_became_peered":"2024-04-26T23:38:51.689821+0000","last_unstale":"2024-04-26T23:38:52.917783+0000","last_undegraded":"2024-04-26T23:38:52.917783+0000","last_fullsized":"2024-04-26T23:38:52.917783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918113+0000","last_change":"2024-04-26T23:38:51.700456+0000","last_active":"2024-04-26T23:38:52.918113+0000","last_peered":"2024-04-26T23:38:52.918113+0000","last_clean":"2024-04-26T23:38:52.918113+0000","last_became_active":"2024-04-26T23:38:51.699994+0000","last_became_peered":"2024-04-26T23:38:51.699994+0000","last_unstale":"2024-04-26T23:38:52.918113+0000","last_undegraded":"2024-04-26T23:38:52.918113+0000","last_fullsized":"2024-04-26T23:38:52.918113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:14.397549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.699029+0000","last_change":"2024-04-26T23:38:50.691324+0000","last_active":"2024-04-26T23:38:52.699029+0000","last_peered":"2024-04-26T23:38:52.699029+0000","last_clean":"2024-04-26T23:38:52.699029+0000","last_became_active":"2024-04-26T23:38:50.691023+0000","last_became_peered":"2024-04-26T23:38:50.691023+0000","last_unstale":"2024-04-26T23:38:52.699029+0000","last_undegraded":"2024-04-26T23:38:52.699029+0000","last_fullsized":"2024-04-26T23:38:52.699029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:05.204342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919175+0000","last_change":"2024-04-26T23:38:51.707984+0000","last_active":"2024-04-26T23:38:52.919175+0000","last_peered":"2024-04-26T23:38:52.919175+0000","last_clean":"2024-04-26T23:38:52.919175+0000","last_became_active":"2024-04-26T23:38:51.705155+0000","last_became_peered":"2024-04-26T23:38:51.705155+0000","last_unstale":"2024-04-26T23:38:52.919175+0000","last_undegraded":"2024-04-26T23:38:52.919175+0000","last_fullsized":"2024-04-26T23:38:52.919175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:36.121201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692085+0000","last_change":"2024-04-26T23:38:50.691001+0000","last_active":"2024-04-26T23:38:52.692085+0000","last_peered":"2024-04-26T23:38:52.692085+0000","last_clean":"2024-04-26T23:38:52.692085+0000","last_became_active":"2024-04-26T23:38:50.690668+0000","last_became_peered":"2024-04-26T23:38:50.690668+0000","last_unstale":"2024-04-26T23:38:52.692085+0000","last_undegraded":"2024-04-26T23:38:52.692085+0000","last_fullsized":"2024-04-26T23:38:52.692085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696483+0000","last_change":"2024-04-26T23:38:51.701138+0000","last_active":"2024-04-26T23:38:52.696483+0000","last_peered":"2024-04-26T23:38:52.696483+0000","last_clean":"2024-04-26T23:38:52.696483+0000","last_became_active":"2024-04-26T23:38:51.700784+0000","last_became_peered":"2024-04-26T23:38:51.700784+0000","last_unstale":"2024-04-26T23:38:52.696483+0000","last_undegraded":"2024-04-26T23:38:52.696483+0000","last_fullsized":"2024-04-26T23:38:52.696483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919553+0000","last_change":"2024-04-26T23:38:50.689919+0000","last_active":"2024-04-26T23:38:52.919553+0000","last_peered":"2024-04-26T23:38:52.919553+0000","last_clean":"2024-04-26T23:38:52.919553+0000","last_became_active":"2024-04-26T23:38:50.689629+0000","last_became_peered":"2024-04-26T23:38:50.689629+0000","last_unstale":"2024-04-26T23:38:52.919553+0000","last_undegraded":"2024-04-26T23:38:52.919553+0000","last_fullsized":"2024-04-26T23:38:52.919553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:12.200561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916176+0000","last_change":"2024-04-26T23:38:51.701222+0000","last_active":"2024-04-26T23:38:52.916176+0000","last_peered":"2024-04-26T23:38:52.916176+0000","last_clean":"2024-04-26T23:38:52.916176+0000","last_became_active":"2024-04-26T23:38:51.700968+0000","last_became_peered":"2024-04-26T23:38:51.700968+0000","last_unstale":"2024-04-26T23:38:52.916176+0000","last_undegraded":"2024-04-26T23:38:52.916176+0000","last_fullsized":"2024-04-26T23:38:52.916176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:12.316705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697321+0000","last_change":"2024-04-26T23:38:50.684308+0000","last_active":"2024-04-26T23:38:52.697321+0000","last_peered":"2024-04-26T23:38:52.697321+0000","last_clean":"2024-04-26T23:38:52.697321+0000","last_became_active":"2024-04-26T23:38:50.684017+0000","last_became_peered":"2024-04-26T23:38:50.684017+0000","last_unstale":"2024-04-26T23:38:52.697321+0000","last_undegraded":"2024-04-26T23:38:52.697321+0000","last_fullsized":"2024-04-26T23:38:52.697321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:36.939563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691248+0000","last_change":"2024-04-26T23:38:51.683069+0000","last_active":"2024-04-26T23:38:52.691248+0000","last_peered":"2024-04-26T23:38:52.691248+0000","last_clean":"2024-04-26T23:38:52.691248+0000","last_became_active":"2024-04-26T23:38:51.682770+0000","last_became_peered":"2024-04-26T23:38:51.682770+0000","last_unstale":"2024-04-26T23:38:52.691248+0000","last_undegraded":"2024-04-26T23:38:52.691248+0000","last_fullsized":"2024-04-26T23:38:52.691248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916322+0000","last_change":"2024-04-26T23:38:50.687042+0000","last_active":"2024-04-26T23:38:52.916322+0000","last_peered":"2024-04-26T23:38:52.916322+0000","last_clean":"2024-04-26T23:38:52.916322+0000","last_became_active":"2024-04-26T23:38:50.685547+0000","last_became_peered":"2024-04-26T23:38:50.685547+0000","last_unstale":"2024-04-26T23:38:52.916322+0000","last_undegraded":"2024-04-26T23:38:52.916322+0000","last_fullsized":"2024-04-26T23:38:52.916322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915904+0000","last_change":"2024-04-26T23:38:51.713337+0000","last_active":"2024-04-26T23:38:52.915904+0000","last_peered":"2024-04-26T23:38:52.915904+0000","last_clean":"2024-04-26T23:38:52.915904+0000","last_became_active":"2024-04-26T23:38:51.709380+0000","last_became_peered":"2024-04-26T23:38:51.709380+0000","last_unstale":"2024-04-26T23:38:52.915904+0000","last_undegraded":"2024-04-26T23:38:52.915904+0000","last_fullsized":"2024-04-26T23:38:52.915904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:28.231903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917645+0000","last_change":"2024-04-26T23:38:50.695028+0000","last_active":"2024-04-26T23:38:52.917645+0000","last_peered":"2024-04-26T23:38:52.917645+0000","last_clean":"2024-04-26T23:38:52.917645+0000","last_became_active":"2024-04-26T23:38:50.694628+0000","last_became_peered":"2024-04-26T23:38:50.694628+0000","last_unstale":"2024-04-26T23:38:52.917645+0000","last_undegraded":"2024-04-26T23:38:52.917645+0000","last_fullsized":"2024-04-26T23:38:52.917645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:12.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696428+0000","last_change":"2024-04-26T23:38:51.715811+0000","last_active":"2024-04-26T23:38:52.696428+0000","last_peered":"2024-04-26T23:38:52.696428+0000","last_clean":"2024-04-26T23:38:52.696428+0000","last_became_active":"2024-04-26T23:38:51.715494+0000","last_became_peered":"2024-04-26T23:38:51.715494+0000","last_unstale":"2024-04-26T23:38:52.696428+0000","last_undegraded":"2024-04-26T23:38:52.696428+0000","last_fullsized":"2024-04-26T23:38:52.696428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:17.241026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918107+0000","last_change":"2024-04-26T23:38:50.698125+0000","last_active":"2024-04-26T23:38:52.918107+0000","last_peered":"2024-04-26T23:38:52.918107+0000","last_clean":"2024-04-26T23:38:52.918107+0000","last_became_active":"2024-04-26T23:38:50.697795+0000","last_became_peered":"2024-04-26T23:38:50.697795+0000","last_unstale":"2024-04-26T23:38:52.918107+0000","last_undegraded":"2024-04-26T23:38:52.918107+0000","last_fullsized":"2024-04-26T23:38:52.918107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914696+0000","last_change":"2024-04-26T23:38:51.690867+0000","last_active":"2024-04-26T23:38:52.914696+0000","last_peered":"2024-04-26T23:38:52.914696+0000","last_clean":"2024-04-26T23:38:52.914696+0000","last_became_active":"2024-04-26T23:38:51.690555+0000","last_became_peered":"2024-04-26T23:38:51.690555+0000","last_unstale":"2024-04-26T23:38:52.914696+0000","last_undegraded":"2024-04-26T23:38:52.914696+0000","last_fullsized":"2024-04-26T23:38:52.914696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:57.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695190+0000","last_change":"2024-04-26T23:38:50.680954+0000","last_active":"2024-04-26T23:38:52.695190+0000","last_peered":"2024-04-26T23:38:52.695190+0000","last_clean":"2024-04-26T23:38:52.695190+0000","last_became_active":"2024-04-26T23:38:50.680546+0000","last_became_peered":"2024-04-26T23:38:50.680546+0000","last_unstale":"2024-04-26T23:38:52.695190+0000","last_undegraded":"2024-04-26T23:38:52.695190+0000","last_fullsized":"2024-04-26T23:38:52.695190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:59.839559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917018+0000","last_change":"2024-04-26T23:38:51.701106+0000","last_active":"2024-04-26T23:38:52.917018+0000","last_peered":"2024-04-26T23:38:52.917018+0000","last_clean":"2024-04-26T23:38:52.917018+0000","last_became_active":"2024-04-26T23:38:51.697725+0000","last_became_peered":"2024-04-26T23:38:51.697725+0000","last_unstale":"2024-04-26T23:38:52.917018+0000","last_undegraded":"2024-04-26T23:38:52.917018+0000","last_fullsized":"2024-04-26T23:38:52.917018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:29.725135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916464+0000","last_change":"2024-04-26T23:38:50.694593+0000","last_active":"2024-04-26T23:38:52.916464+0000","last_peered":"2024-04-26T23:38:52.916464+0000","last_clean":"2024-04-26T23:38:52.916464+0000","last_became_active":"2024-04-26T23:38:50.694289+0000","last_became_peered":"2024-04-26T23:38:50.694289+0000","last_unstale":"2024-04-26T23:38:52.916464+0000","last_undegraded":"2024-04-26T23:38:52.916464+0000","last_fullsized":"2024-04-26T23:38:52.916464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:04.045799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918519+0000","last_change":"2024-04-26T23:38:51.718151+0000","last_active":"2024-04-26T23:38:52.918519+0000","last_peered":"2024-04-26T23:38:52.918519+0000","last_clean":"2024-04-26T23:38:52.918519+0000","last_became_active":"2024-04-26T23:38:51.717851+0000","last_became_peered":"2024-04-26T23:38:51.717851+0000","last_unstale":"2024-04-26T23:38:52.918519+0000","last_undegraded":"2024-04-26T23:38:52.918519+0000","last_fullsized":"2024-04-26T23:38:52.918519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917075+0000","last_change":"2024-04-26T23:38:50.696626+0000","last_active":"2024-04-26T23:38:52.917075+0000","last_peered":"2024-04-26T23:38:52.917075+0000","last_clean":"2024-04-26T23:38:52.917075+0000","last_became_active":"2024-04-26T23:38:50.696142+0000","last_became_peered":"2024-04-26T23:38:50.696142+0000","last_unstale":"2024-04-26T23:38:52.917075+0000","last_undegraded":"2024-04-26T23:38:52.917075+0000","last_fullsized":"2024-04-26T23:38:52.917075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.692252+0000","last_change":"2024-04-26T23:38:51.684545+0000","last_active":"2024-04-26T23:38:52.692252+0000","last_peered":"2024-04-26T23:38:52.692252+0000","last_clean":"2024-04-26T23:38:52.692252+0000","last_became_active":"2024-04-26T23:38:51.684163+0000","last_became_peered":"2024-04-26T23:38:51.684163+0000","last_unstale":"2024-04-26T23:38:52.692252+0000","last_undegraded":"2024-04-26T23:38:52.692252+0000","last_fullsized":"2024-04-26T23:38:52.692252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:49.516055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916617+0000","last_change":"2024-04-26T23:38:50.695193+0000","last_active":"2024-04-26T23:38:52.916617+0000","last_peered":"2024-04-26T23:38:52.916617+0000","last_clean":"2024-04-26T23:38:52.916617+0000","last_became_active":"2024-04-26T23:38:50.694850+0000","last_became_peered":"2024-04-26T23:38:50.694850+0000","last_unstale":"2024-04-26T23:38:52.916617+0000","last_undegraded":"2024-04-26T23:38:52.916617+0000","last_fullsized":"2024-04-26T23:38:52.916617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:28.709174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697868+0000","last_change":"2024-04-26T23:38:51.705826+0000","last_active":"2024-04-26T23:38:52.697868+0000","last_peered":"2024-04-26T23:38:52.697868+0000","last_clean":"2024-04-26T23:38:52.697868+0000","last_became_active":"2024-04-26T23:38:51.705401+0000","last_became_peered":"2024-04-26T23:38:51.705401+0000","last_unstale":"2024-04-26T23:38:52.697868+0000","last_undegraded":"2024-04-26T23:38:52.697868+0000","last_fullsized":"2024-04-26T23:38:52.697868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914739+0000","last_change":"2024-04-26T23:38:50.695400+0000","last_active":"2024-04-26T23:38:52.914739+0000","last_peered":"2024-04-26T23:38:52.914739+0000","last_clean":"2024-04-26T23:38:52.914739+0000","last_became_active":"2024-04-26T23:38:50.695178+0000","last_became_peered":"2024-04-26T23:38:50.695178+0000","last_unstale":"2024-04-26T23:38:52.914739+0000","last_undegraded":"2024-04-26T23:38:52.914739+0000","last_fullsized":"2024-04-26T23:38:52.914739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.225065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691248+0000","last_change":"2024-04-26T23:38:51.704728+0000","last_active":"2024-04-26T23:38:52.691248+0000","last_peered":"2024-04-26T23:38:52.691248+0000","last_clean":"2024-04-26T23:38:52.691248+0000","last_became_active":"2024-04-26T23:38:51.702291+0000","last_became_peered":"2024-04-26T23:38:51.702291+0000","last_unstale":"2024-04-26T23:38:52.691248+0000","last_undegraded":"2024-04-26T23:38:52.691248+0000","last_fullsized":"2024-04-26T23:38:52.691248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:05.061761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915344+0000","last_change":"2024-04-26T23:38:50.695534+0000","last_active":"2024-04-26T23:38:52.915344+0000","last_peered":"2024-04-26T23:38:52.915344+0000","last_clean":"2024-04-26T23:38:52.915344+0000","last_became_active":"2024-04-26T23:38:50.686496+0000","last_became_peered":"2024-04-26T23:38:50.686496+0000","last_unstale":"2024-04-26T23:38:52.915344+0000","last_undegraded":"2024-04-26T23:38:52.915344+0000","last_fullsized":"2024-04-26T23:38:52.915344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:03.043110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914218+0000","last_change":"2024-04-26T23:38:51.698615+0000","last_active":"2024-04-26T23:38:52.914218+0000","last_peered":"2024-04-26T23:38:52.914218+0000","last_clean":"2024-04-26T23:38:52.914218+0000","last_became_active":"2024-04-26T23:38:51.698306+0000","last_became_peered":"2024-04-26T23:38:51.698306+0000","last_unstale":"2024-04-26T23:38:52.914218+0000","last_undegraded":"2024-04-26T23:38:52.914218+0000","last_fullsized":"2024-04-26T23:38:52.914218+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918006+0000","last_change":"2024-04-26T23:38:50.698660+0000","last_active":"2024-04-26T23:38:52.918006+0000","last_peered":"2024-04-26T23:38:52.918006+0000","last_clean":"2024-04-26T23:38:52.918006+0000","last_became_active":"2024-04-26T23:38:50.698295+0000","last_became_peered":"2024-04-26T23:38:50.698295+0000","last_unstale":"2024-04-26T23:38:52.918006+0000","last_undegraded":"2024-04-26T23:38:52.918006+0000","last_fullsized":"2024-04-26T23:38:52.918006+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917780+0000","last_change":"2024-04-26T23:38:51.690639+0000","last_active":"2024-04-26T23:38:52.917780+0000","last_peered":"2024-04-26T23:38:52.917780+0000","last_clean":"2024-04-26T23:38:52.917780+0000","last_became_active":"2024-04-26T23:38:51.690325+0000","last_became_peered":"2024-04-26T23:38:51.690325+0000","last_unstale":"2024-04-26T23:38:52.917780+0000","last_undegraded":"2024-04-26T23:38:52.917780+0000","last_fullsized":"2024-04-26T23:38:52.917780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:09.831540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697431+0000","last_change":"2024-04-26T23:38:50.686322+0000","last_active":"2024-04-26T23:38:52.697431+0000","last_peered":"2024-04-26T23:38:52.697431+0000","last_clean":"2024-04-26T23:38:52.697431+0000","last_became_active":"2024-04-26T23:38:50.684566+0000","last_became_peered":"2024-04-26T23:38:50.684566+0000","last_unstale":"2024-04-26T23:38:52.697431+0000","last_undegraded":"2024-04-26T23:38:52.697431+0000","last_fullsized":"2024-04-26T23:38:52.697431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:03.457234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915102+0000","last_change":"2024-04-26T23:38:51.702639+0000","last_active":"2024-04-26T23:38:52.915102+0000","last_peered":"2024-04-26T23:38:52.915102+0000","last_clean":"2024-04-26T23:38:52.915102+0000","last_became_active":"2024-04-26T23:38:51.701779+0000","last_became_peered":"2024-04-26T23:38:51.701779+0000","last_unstale":"2024-04-26T23:38:52.915102+0000","last_undegraded":"2024-04-26T23:38:52.915102+0000","last_fullsized":"2024-04-26T23:38:52.915102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:31.130045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917989+0000","last_change":"2024-04-26T23:38:50.694214+0000","last_active":"2024-04-26T23:38:52.917989+0000","last_peered":"2024-04-26T23:38:52.917989+0000","last_clean":"2024-04-26T23:38:52.917989+0000","last_became_active":"2024-04-26T23:38:50.693909+0000","last_became_peered":"2024-04-26T23:38:50.693909+0000","last_unstale":"2024-04-26T23:38:52.917989+0000","last_undegraded":"2024-04-26T23:38:52.917989+0000","last_fullsized":"2024-04-26T23:38:52.917989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:51.260608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914550+0000","last_change":"2024-04-26T23:38:51.717413+0000","last_active":"2024-04-26T23:38:52.914550+0000","last_peered":"2024-04-26T23:38:52.914550+0000","last_clean":"2024-04-26T23:38:52.914550+0000","last_became_active":"2024-04-26T23:38:51.717162+0000","last_became_peered":"2024-04-26T23:38:51.717162+0000","last_unstale":"2024-04-26T23:38:52.914550+0000","last_undegraded":"2024-04-26T23:38:52.914550+0000","last_fullsized":"2024-04-26T23:38:52.914550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917822+0000","last_change":"2024-04-26T23:38:50.686368+0000","last_active":"2024-04-26T23:38:52.917822+0000","last_peered":"2024-04-26T23:38:52.917822+0000","last_clean":"2024-04-26T23:38:52.917822+0000","last_became_active":"2024-04-26T23:38:50.686085+0000","last_became_peered":"2024-04-26T23:38:50.686085+0000","last_unstale":"2024-04-26T23:38:52.917822+0000","last_undegraded":"2024-04-26T23:38:52.917822+0000","last_fullsized":"2024-04-26T23:38:52.917822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:12.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916489+0000","last_change":"2024-04-26T23:38:51.704560+0000","last_active":"2024-04-26T23:38:52.916489+0000","last_peered":"2024-04-26T23:38:52.916489+0000","last_clean":"2024-04-26T23:38:52.916489+0000","last_became_active":"2024-04-26T23:38:51.702902+0000","last_became_peered":"2024-04-26T23:38:51.702902+0000","last_unstale":"2024-04-26T23:38:52.916489+0000","last_undegraded":"2024-04-26T23:38:52.916489+0000","last_fullsized":"2024-04-26T23:38:52.916489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:56.091602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.697386+0000","last_change":"2024-04-26T23:38:50.685721+0000","last_active":"2024-04-26T23:38:52.697386+0000","last_peered":"2024-04-26T23:38:52.697386+0000","last_clean":"2024-04-26T23:38:52.697386+0000","last_became_active":"2024-04-26T23:38:50.685381+0000","last_became_peered":"2024-04-26T23:38:50.685381+0000","last_unstale":"2024-04-26T23:38:52.697386+0000","last_undegraded":"2024-04-26T23:38:52.697386+0000","last_fullsized":"2024-04-26T23:38:52.697386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.696126+0000","last_change":"2024-04-26T23:38:51.698379+0000","last_active":"2024-04-26T23:38:52.696126+0000","last_peered":"2024-04-26T23:38:52.696126+0000","last_clean":"2024-04-26T23:38:52.696126+0000","last_became_active":"2024-04-26T23:38:51.698063+0000","last_became_peered":"2024-04-26T23:38:51.698063+0000","last_unstale":"2024-04-26T23:38:52.696126+0000","last_undegraded":"2024-04-26T23:38:52.696126+0000","last_fullsized":"2024-04-26T23:38:52.696126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:14.809468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916240+0000","last_change":"2024-04-26T23:38:50.690198+0000","last_active":"2024-04-26T23:38:52.916240+0000","last_peered":"2024-04-26T23:38:52.916240+0000","last_clean":"2024-04-26T23:38:52.916240+0000","last_became_active":"2024-04-26T23:38:50.689613+0000","last_became_peered":"2024-04-26T23:38:50.689613+0000","last_unstale":"2024-04-26T23:38:52.916240+0000","last_undegraded":"2024-04-26T23:38:52.916240+0000","last_fullsized":"2024-04-26T23:38:52.916240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:55.670095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916945+0000","last_change":"2024-04-26T23:38:51.704333+0000","last_active":"2024-04-26T23:38:52.916945+0000","last_peered":"2024-04-26T23:38:52.916945+0000","last_clean":"2024-04-26T23:38:52.916945+0000","last_became_active":"2024-04-26T23:38:51.704138+0000","last_became_peered":"2024-04-26T23:38:51.704138+0000","last_unstale":"2024-04-26T23:38:52.916945+0000","last_undegraded":"2024-04-26T23:38:52.916945+0000","last_fullsized":"2024-04-26T23:38:52.916945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:13.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919474+0000","last_change":"2024-04-26T23:38:50.691793+0000","last_active":"2024-04-26T23:38:52.919474+0000","last_peered":"2024-04-26T23:38:52.919474+0000","last_clean":"2024-04-26T23:38:52.919474+0000","last_became_active":"2024-04-26T23:38:50.691492+0000","last_became_peered":"2024-04-26T23:38:50.691492+0000","last_unstale":"2024-04-26T23:38:52.919474+0000","last_undegraded":"2024-04-26T23:38:52.919474+0000","last_fullsized":"2024-04-26T23:38:52.919474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918434+0000","last_change":"2024-04-26T23:38:51.691389+0000","last_active":"2024-04-26T23:38:52.918434+0000","last_peered":"2024-04-26T23:38:52.918434+0000","last_clean":"2024-04-26T23:38:52.918434+0000","last_became_active":"2024-04-26T23:38:51.690984+0000","last_became_peered":"2024-04-26T23:38:51.690984+0000","last_unstale":"2024-04-26T23:38:52.918434+0000","last_undegraded":"2024-04-26T23:38:52.918434+0000","last_fullsized":"2024-04-26T23:38:52.918434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:28.231903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917047+0000","last_change":"2024-04-26T23:38:50.687713+0000","last_active":"2024-04-26T23:38:52.917047+0000","last_peered":"2024-04-26T23:38:52.917047+0000","last_clean":"2024-04-26T23:38:52.917047+0000","last_became_active":"2024-04-26T23:38:50.687314+0000","last_became_peered":"2024-04-26T23:38:50.687314+0000","last_unstale":"2024-04-26T23:38:52.917047+0000","last_undegraded":"2024-04-26T23:38:52.917047+0000","last_fullsized":"2024-04-26T23:38:52.917047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:03.017735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691414+0000","last_change":"2024-04-26T23:38:51.685574+0000","last_active":"2024-04-26T23:38:52.691414+0000","last_peered":"2024-04-26T23:38:52.691414+0000","last_clean":"2024-04-26T23:38:52.691414+0000","last_became_active":"2024-04-26T23:38:51.685269+0000","last_became_peered":"2024-04-26T23:38:51.685269+0000","last_unstale":"2024-04-26T23:38:52.691414+0000","last_undegraded":"2024-04-26T23:38:52.691414+0000","last_fullsized":"2024-04-26T23:38:52.691414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917867+0000","last_change":"2024-04-26T23:38:50.682745+0000","last_active":"2024-04-26T23:38:52.917867+0000","last_peered":"2024-04-26T23:38:52.917867+0000","last_clean":"2024-04-26T23:38:52.917867+0000","last_became_active":"2024-04-26T23:38:50.682462+0000","last_became_peered":"2024-04-26T23:38:50.682462+0000","last_unstale":"2024-04-26T23:38:52.917867+0000","last_undegraded":"2024-04-26T23:38:52.917867+0000","last_fullsized":"2024-04-26T23:38:52.917867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:39.251641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915938+0000","last_change":"2024-04-26T23:38:51.716394+0000","last_active":"2024-04-26T23:38:52.915938+0000","last_peered":"2024-04-26T23:38:52.915938+0000","last_clean":"2024-04-26T23:38:52.915938+0000","last_became_active":"2024-04-26T23:38:51.712486+0000","last_became_peered":"2024-04-26T23:38:51.712486+0000","last_unstale":"2024-04-26T23:38:52.915938+0000","last_undegraded":"2024-04-26T23:38:52.915938+0000","last_fullsized":"2024-04-26T23:38:52.915938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:57.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918202+0000","last_change":"2024-04-26T23:38:50.699004+0000","last_active":"2024-04-26T23:38:52.918202+0000","last_peered":"2024-04-26T23:38:52.918202+0000","last_clean":"2024-04-26T23:38:52.918202+0000","last_became_active":"2024-04-26T23:38:50.698643+0000","last_became_peered":"2024-04-26T23:38:50.698643+0000","last_unstale":"2024-04-26T23:38:52.918202+0000","last_undegraded":"2024-04-26T23:38:52.918202+0000","last_fullsized":"2024-04-26T23:38:52.918202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:35.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915455+0000","last_change":"2024-04-26T23:38:51.687019+0000","last_active":"2024-04-26T23:38:52.915455+0000","last_peered":"2024-04-26T23:38:52.915455+0000","last_clean":"2024-04-26T23:38:52.915455+0000","last_became_active":"2024-04-26T23:38:51.686726+0000","last_became_peered":"2024-04-26T23:38:51.686726+0000","last_unstale":"2024-04-26T23:38:52.915455+0000","last_undegraded":"2024-04-26T23:38:52.915455+0000","last_fullsized":"2024-04-26T23:38:52.915455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:25.223837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916928+0000","last_change":"2024-04-26T23:38:50.688989+0000","last_active":"2024-04-26T23:38:52.916928+0000","last_peered":"2024-04-26T23:38:52.916928+0000","last_clean":"2024-04-26T23:38:52.916928+0000","last_became_active":"2024-04-26T23:38:50.688711+0000","last_became_peered":"2024-04-26T23:38:50.688711+0000","last_unstale":"2024-04-26T23:38:52.916928+0000","last_undegraded":"2024-04-26T23:38:52.916928+0000","last_fullsized":"2024-04-26T23:38:52.916928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:13.381141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919135+0000","last_change":"2024-04-26T23:38:51.706900+0000","last_active":"2024-04-26T23:38:52.919135+0000","last_peered":"2024-04-26T23:38:52.919135+0000","last_clean":"2024-04-26T23:38:52.919135+0000","last_became_active":"2024-04-26T23:38:51.705041+0000","last_became_peered":"2024-04-26T23:38:51.705041+0000","last_unstale":"2024-04-26T23:38:52.919135+0000","last_undegraded":"2024-04-26T23:38:52.919135+0000","last_fullsized":"2024-04-26T23:38:52.919135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917847+0000","last_change":"2024-04-26T23:38:50.693273+0000","last_active":"2024-04-26T23:38:52.917847+0000","last_peered":"2024-04-26T23:38:52.917847+0000","last_clean":"2024-04-26T23:38:52.917847+0000","last_became_active":"2024-04-26T23:38:50.692984+0000","last_became_peered":"2024-04-26T23:38:50.692984+0000","last_unstale":"2024-04-26T23:38:52.917847+0000","last_undegraded":"2024-04-26T23:38:52.917847+0000","last_fullsized":"2024-04-26T23:38:52.917847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:20.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918732+0000","last_change":"2024-04-26T23:38:51.702639+0000","last_active":"2024-04-26T23:38:52.918732+0000","last_peered":"2024-04-26T23:38:52.918732+0000","last_clean":"2024-04-26T23:38:52.918732+0000","last_became_active":"2024-04-26T23:38:51.702345+0000","last_became_peered":"2024-04-26T23:38:51.702345+0000","last_unstale":"2024-04-26T23:38:52.918732+0000","last_undegraded":"2024-04-26T23:38:52.918732+0000","last_fullsized":"2024-04-26T23:38:52.918732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:37.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.916717+0000","last_change":"2024-04-26T23:38:50.691632+0000","last_active":"2024-04-26T23:38:52.916717+0000","last_peered":"2024-04-26T23:38:52.916717+0000","last_clean":"2024-04-26T23:38:52.916717+0000","last_became_active":"2024-04-26T23:38:50.691073+0000","last_became_peered":"2024-04-26T23:38:50.691073+0000","last_unstale":"2024-04-26T23:38:52.916717+0000","last_undegraded":"2024-04-26T23:38:52.916717+0000","last_fullsized":"2024-04-26T23:38:52.916717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:00.898736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.915208+0000","last_change":"2024-04-26T23:38:51.694847+0000","last_active":"2024-04-26T23:38:52.915208+0000","last_peered":"2024-04-26T23:38:52.915208+0000","last_clean":"2024-04-26T23:38:52.915208+0000","last_became_active":"2024-04-26T23:38:51.694541+0000","last_became_peered":"2024-04-26T23:38:51.694541+0000","last_unstale":"2024-04-26T23:38:52.915208+0000","last_undegraded":"2024-04-26T23:38:52.915208+0000","last_fullsized":"2024-04-26T23:38:52.915208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:07.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691887+0000","last_change":"2024-04-26T23:38:50.686345+0000","last_active":"2024-04-26T23:38:52.691887+0000","last_peered":"2024-04-26T23:38:52.691887+0000","last_clean":"2024-04-26T23:38:52.691887+0000","last_became_active":"2024-04-26T23:38:50.685727+0000","last_became_peered":"2024-04-26T23:38:50.685727+0000","last_unstale":"2024-04-26T23:38:52.691887+0000","last_undegraded":"2024-04-26T23:38:52.691887+0000","last_fullsized":"2024-04-26T23:38:52.691887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.809561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691507+0000","last_change":"2024-04-26T23:38:51.684652+0000","last_active":"2024-04-26T23:38:52.691507+0000","last_peered":"2024-04-26T23:38:52.691507+0000","last_clean":"2024-04-26T23:38:52.691507+0000","last_became_active":"2024-04-26T23:38:51.684329+0000","last_became_peered":"2024-04-26T23:38:51.684329+0000","last_unstale":"2024-04-26T23:38:52.691507+0000","last_undegraded":"2024-04-26T23:38:52.691507+0000","last_fullsized":"2024-04-26T23:38:52.691507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:39.363434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.691668+0000","last_change":"2024-04-26T23:38:50.688232+0000","last_active":"2024-04-26T23:38:52.691668+0000","last_peered":"2024-04-26T23:38:52.691668+0000","last_clean":"2024-04-26T23:38:52.691668+0000","last_became_active":"2024-04-26T23:38:50.687890+0000","last_became_peered":"2024-04-26T23:38:50.687890+0000","last_unstale":"2024-04-26T23:38:52.691668+0000","last_undegraded":"2024-04-26T23:38:52.691668+0000","last_fullsized":"2024-04-26T23:38:52.691668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:18.755116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918086+0000","last_change":"2024-04-26T23:38:51.714014+0000","last_active":"2024-04-26T23:38:52.918086+0000","last_peered":"2024-04-26T23:38:52.918086+0000","last_clean":"2024-04-26T23:38:52.918086+0000","last_became_active":"2024-04-26T23:38:51.713684+0000","last_became_peered":"2024-04-26T23:38:51.713684+0000","last_unstale":"2024-04-26T23:38:52.918086+0000","last_undegraded":"2024-04-26T23:38:52.918086+0000","last_fullsized":"2024-04-26T23:38:52.918086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.698225+0000","last_change":"2024-04-26T23:38:50.688814+0000","last_active":"2024-04-26T23:38:52.698225+0000","last_peered":"2024-04-26T23:38:52.698225+0000","last_clean":"2024-04-26T23:38:52.698225+0000","last_became_active":"2024-04-26T23:38:50.688521+0000","last_became_peered":"2024-04-26T23:38:50.688521+0000","last_unstale":"2024-04-26T23:38:52.698225+0000","last_undegraded":"2024-04-26T23:38:52.698225+0000","last_fullsized":"2024-04-26T23:38:52.698225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:51.613896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695819+0000","last_change":"2024-04-26T23:38:51.704947+0000","last_active":"2024-04-26T23:38:52.695819+0000","last_peered":"2024-04-26T23:38:52.695819+0000","last_clean":"2024-04-26T23:38:52.695819+0000","last_became_active":"2024-04-26T23:38:51.704070+0000","last_became_peered":"2024-04-26T23:38:51.704070+0000","last_unstale":"2024-04-26T23:38:52.695819+0000","last_undegraded":"2024-04-26T23:38:52.695819+0000","last_fullsized":"2024-04-26T23:38:52.695819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.677975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.695798+0000","last_change":"2024-04-26T23:38:50.678514+0000","last_active":"2024-04-26T23:38:52.695798+0000","last_peered":"2024-04-26T23:38:52.695798+0000","last_clean":"2024-04-26T23:38:52.695798+0000","last_became_active":"2024-04-26T23:38:50.678094+0000","last_became_peered":"2024-04-26T23:38:50.678094+0000","last_unstale":"2024-04-26T23:38:52.695798+0000","last_undegraded":"2024-04-26T23:38:52.695798+0000","last_fullsized":"2024-04-26T23:38:52.695798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:08.430797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.919036+0000","last_change":"2024-04-26T23:38:51.710701+0000","last_active":"2024-04-26T23:38:52.919036+0000","last_peered":"2024-04-26T23:38:52.919036+0000","last_clean":"2024-04-26T23:38:52.919036+0000","last_became_active":"2024-04-26T23:38:51.710457+0000","last_became_peered":"2024-04-26T23:38:51.710457+0000","last_unstale":"2024-04-26T23:38:52.919036+0000","last_undegraded":"2024-04-26T23:38:52.919036+0000","last_fullsized":"2024-04-26T23:38:52.919036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:43.813343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.914125+0000","last_change":"2024-04-26T23:38:50.685335+0000","last_active":"2024-04-26T23:38:52.914125+0000","last_peered":"2024-04-26T23:38:52.914125+0000","last_clean":"2024-04-26T23:38:52.914125+0000","last_became_active":"2024-04-26T23:38:50.685055+0000","last_became_peered":"2024-04-26T23:38:50.685055+0000","last_unstale":"2024-04-26T23:38:52.914125+0000","last_undegraded":"2024-04-26T23:38:52.914125+0000","last_fullsized":"2024-04-26T23:38:52.914125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:13.793507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.917354+0000","last_change":"2024-04-26T23:38:51.707157+0000","last_active":"2024-04-26T23:38:52.917354+0000","last_peered":"2024-04-26T23:38:52.917354+0000","last_clean":"2024-04-26T23:38:52.917354+0000","last_became_active":"2024-04-26T23:38:51.706848+0000","last_became_peered":"2024-04-26T23:38:51.706848+0000","last_unstale":"2024-04-26T23:38:52.917354+0000","last_undegraded":"2024-04-26T23:38:52.917354+0000","last_fullsized":"2024-04-26T23:38:52.917354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:50.638842+0000","last_clean_scrub_stamp":"2024-04-26T23:38:50.638842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:07.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918381+0000","last_change":"2024-04-26T23:38:50.694147+0000","last_active":"2024-04-26T23:38:52.918381+0000","last_peered":"2024-04-26T23:38:52.918381+0000","last_clean":"2024-04-26T23:38:52.918381+0000","last_became_active":"2024-04-26T23:38:50.692579+0000","last_became_peered":"2024-04-26T23:38:50.692579+0000","last_unstale":"2024-04-26T23:38:52.918381+0000","last_undegraded":"2024-04-26T23:38:52.918381+0000","last_fullsized":"2024-04-26T23:38:52.918381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:38:49.622881+0000","last_clean_scrub_stamp":"2024-04-26T23:38:49.622881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:20.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T23:38:52.918735+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:38:52.918735+0000","last_peered":"2024-04-26T23:38:52.918735+0000","last_clean":"2024-04-26T23:38:52.918735+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:38:52.918735+0000","last_undegraded":"2024-04-26T23:38:52.918735+0000","last_fullsized":"2024-04-26T23:38:52.918735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":786749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":196345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":785101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:39:01.848 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:39:01.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:39:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.103+0000 7fe539e00640 1 Processor -- start 2024-04-26T23:39:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.104+0000 7fe539e00640 1 -- start start 2024-04-26T23:39:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe53415cec0 0x7fe5340cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5340cdb30 0x7fe5340735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe534073b30 0x7fe534071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe5340ce7d0 con 0x7fe53415cec0 2024-04-26T23:39:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5340ceab0 con 0x7fe5340cdb30 2024-04-26T23:39:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.105+0000 7fe539e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5340ced90 con 0x7fe534073b30 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe538a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe53415cec0 0x7fe5340cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe538a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe53415cec0 0x7fe5340cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33770/0 (socket says 172.21.15.46:33770) 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe538a00640 1 -- 172.21.15.46:0/3250670306 learned_addr learned my addr 172.21.15.46:0/3250670306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe533e00640 1 --2- 172.21.15.46:0/3250670306 >> v2:172.21.15.142:3300/0 conn(0x7fe5340cdb30 0x7fe5340735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe539400640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe534073b30 0x7fe534071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe538a00640 1 -- 172.21.15.46:0/3250670306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe534073b30 msgr2=0x7fe534071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.106+0000 7fe538a00640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe534073b30 0x7fe534071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.107+0000 7fe538a00640 1 -- 172.21.15.46:0/3250670306 >> v2:172.21.15.142:3300/0 conn(0x7fe5340cdb30 msgr2=0x7fe5340735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.107+0000 7fe538a00640 1 --2- 172.21.15.46:0/3250670306 >> v2:172.21.15.142:3300/0 conn(0x7fe5340cdb30 0x7fe5340735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.107+0000 7fe538a00640 1 -- 172.21.15.46:0/3250670306 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5340cf070 con 0x7fe53415cec0 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe538a00640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53415cec0 0x7fe5340cd5f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe534074300 tx=0x7fe52802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e05dd76ae21ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe533400640 1 -- 172.21.15.46:0/3250670306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528037070 con 0x7fe53415cec0 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe533400640 1 -- 172.21.15.46:0/3250670306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe528032440 con 0x7fe53415cec0 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe533400640 1 -- 172.21.15.46:0/3250670306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528032810 con 0x7fe53415cec0 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe539e00640 1 -- 172.21.15.46:0/3250670306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53415cec0 msgr2=0x7fe5340cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.108+0000 7fe539e00640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53415cec0 0x7fe5340cd5f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe534074300 tx=0x7fe52802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 -- 172.21.15.46:0/3250670306 shutdown_connections 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe534073b30 0x7fe534071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 --2- 172.21.15.46:0/3250670306 >> v2:172.21.15.142:3300/0 conn(0x7fe5340cdb30 0x7fe5340735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 --2- 172.21.15.46:0/3250670306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53415cec0 0x7fe5340cd5f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 -- 172.21.15.46:0/3250670306 >> 172.21.15.46:0/3250670306 conn(0x7fe5340b27b0 msgr2=0x7fe5340b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 -- 172.21.15.46:0/3250670306 shutdown_connections 2024-04-26T23:39:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.109+0000 7fe539e00640 1 -- 172.21.15.46:0/3250670306 wait complete. 2024-04-26T23:39:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.110+0000 7fe539e00640 1 Processor -- start 2024-04-26T23:39:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.111+0000 7fe539e00640 1 -- start start 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe534073b30 0x7fe53415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe53415cec0 0x7fe53415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe534073f70 con 0x7fe5340cdb30 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe53415d2a0 con 0x7fe534073b30 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5340723c0 con 0x7fe53415cec0 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33778/0 (socket says 172.21.15.46:33778) 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 -- 172.21.15.46:0/4157571531 learned_addr learned my addr 172.21.15.46:0/4157571531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe538a00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe534073b30 0x7fe53415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe539400640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe53415cec0 0x7fe53415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 -- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe53415cec0 msgr2=0x7fe53415ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe53415cec0 0x7fe53415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 -- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe534073b30 msgr2=0x7fe53415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe534073b30 0x7fe53415af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.112+0000 7fe533e00640 1 -- 172.21.15.46:0/4157571531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5340c5df0 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe533e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe52000d800 tx=0x7fe52000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52000b6c0 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520019070 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52000ba80 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5340c60d0 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.113+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5340c6690 con 0x7fe5340cdb30 2024-04-26T23:39:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.114+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5340693b0 con 0x7fe5340cdb30 2024-04-26T23:39:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.116+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe52000bc20 con 0x7fe5340cdb30 2024-04-26T23:39:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.117+0000 7fe530c00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.117+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe52008c140 con 0x7fe5340cdb30 2024-04-26T23:39:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.117+0000 7fe538a00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.118+0000 7fe538a00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 0x7fe51406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe528034000 tx=0x7fe52802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.121+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe52005a3d0 con 0x7fe5340cdb30 2024-04-26T23:39:02.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.400+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe534160680 con 0x7fe5340cdb30 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.405+0000 7fe530c00640 1 -- 172.21.15.46:0/4157571531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fe520059cd0 con 0x7fe5340cdb30 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:39:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:39:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 msgr2=0x7fe51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 0x7fe51406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe528034000 tx=0x7fe52802e000 comp rx=0 tx=0).stop 2024-04-26T23:39:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 msgr2=0x7fe5341f3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe52000d800 tx=0x7fe52000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 shutdown_connections 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe53415cec0 0x7fe53415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5340cdb30 0x7fe5341f3da0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 --2- 172.21.15.46:0/4157571531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe534073b30 0x7fe53415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 >> 172.21.15.46:0/4157571531 conn(0x7fe5340b27b0 msgr2=0x7fe53406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 shutdown_connections 2024-04-26T23:39:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.409+0000 7fe539e00640 1 -- 172.21.15.46:0/4157571531 wait complete. 2024-04-26T23:39:02.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:39:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.632+0000 7fa93e000640 1 Processor -- start 2024-04-26T23:39:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.632+0000 7fa93e000640 1 -- start start 2024-04-26T23:39:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.633+0000 7fa93e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa9380d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.633+0000 7fa93e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9380d11f0 con 0x7fa93815f4e0 2024-04-26T23:39:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.633+0000 7fa93e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9380d5f20 con 0x7fa9380b4000 2024-04-26T23:39:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.633+0000 7fa93e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380d0ce0 con 0x7fa9380d2130 2024-04-26T23:39:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.634+0000 7fa93cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa93815f4e0 0x7fa9380d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58544/0 (socket says 172.21.15.46:58544) 2024-04-26T23:39:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.634+0000 7fa93cc00640 1 -- 172.21.15.46:0/671630103 learned_addr learned my addr 172.21.15.46:0/671630103 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:39:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.634+0000 7fa937e00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa9380d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.634+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 936667729 0 0) 0x7fa9380d11f0 con 0x7fa93815f4e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.634+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa91c003520 con 0x7fa93815f4e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791425807 0 0) 0x7fa91c003520 con 0x7fa93815f4e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 msgr2=0x7fa9380d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa9380d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 >> v1:172.21.15.142:6789/0 conn(0x7fa9380b4000 legacy=0x7fa9380b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380d0f10 con 0x7fa93815f4e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa928001670 con 0x7fa93815f4e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 >> v1:172.21.15.46:6789/0 conn(0x7fa93815f4e0 legacy=0x7fa9380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa91c0039e0 0x7fa91c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa91c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91c004d40 0x7fa91c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c00ba70 con 0x7fa91c0039e0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c00bc20 con 0x7fa91c004d40 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c00be20 con 0x7fa9380d2130 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.635+0000 7fa93cc00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa91c0039e0 0x7fa91c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa937e00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa91c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93d600640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91c004d40 0x7fa91c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93cc00640 1 -- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 msgr2=0x7fa91c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:02.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93cc00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9380d2130 0x7fa91c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93cc00640 1 -- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91c004d40 msgr2=0x7fa91c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:02.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93cc00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91c004d40 0x7fa91c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.636+0000 7fa93cc00640 1 -- 172.21.15.46:0/671630103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c003520 con 0x7fa91c0039e0 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.637+0000 7fa93cc00640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa91c0039e0 0x7fa91c003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa928007fd0 tx=0x7fa91c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f0298664ae3dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.637+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928007c20 con 0x7fa91c0039e0 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.637+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa928005af0 con 0x7fa91c0039e0 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.637+0000 7fa937400640 1 -- 172.21.15.46:0/671630103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928007070 con 0x7fa91c0039e0 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 -- 172.21.15.46:0/671630103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa91c0039e0 msgr2=0x7fa91c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 --2- 172.21.15.46:0/671630103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa91c0039e0 0x7fa91c003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa928007fd0 tx=0x7fa91c005830 comp rx=0 tx=0).stop 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93d600640 1 -- 172.21.15.46:0/671630103 reap_dead start 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 -- 172.21.15.46:0/671630103 shutdown_connections 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 -- 172.21.15.46:0/671630103 >> 172.21.15.46:0/671630103 conn(0x7fa9380d5010 msgr2=0x7fa9380d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 -- 172.21.15.46:0/671630103 shutdown_connections 2024-04-26T23:39:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.638+0000 7fa93e000640 1 -- 172.21.15.46:0/671630103 wait complete. 2024-04-26T23:39:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.640+0000 7fa93e000640 1 Processor -- start 2024-04-26T23:39:02.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.640+0000 7fa93e000640 1 -- start start 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91c0039e0 0x7fa93809ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 0x7fa9380a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380b1d60 con 0x7fa9380cdd10 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380d2990 con 0x7fa9380ce170 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa93815f8c0 con 0x7fa91c0039e0 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33808/0 (socket says 172.21.15.46:33808) 2024-04-26T23:39:02.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 -- 172.21.15.46:0/4216121630 learned_addr learned my addr 172.21.15.46:0/4216121630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93cc00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91c0039e0 0x7fa93809ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93d600640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 0x7fa9380a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 -- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91c0039e0 msgr2=0x7fa93809ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91c0039e0 0x7fa93809ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 -- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 msgr2=0x7fa9380a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 0x7fa9380a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa937e00640 1 -- 172.21.15.46:0/4216121630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380a1e50 con 0x7fa9380cdd10 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.641+0000 7fa93d600640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 0x7fa9380a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa937e00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa924007f90 tx=0x7fa924011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92401a020 con 0x7fa9380cdd10 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924011e90 con 0x7fa9380cdd10 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240168c0 con 0x7fa9380cdd10 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9380a2130 con 0x7fa9380cdd10 2024-04-26T23:39:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.642+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380a26f0 con 0x7fa9380cdd10 2024-04-26T23:39:02.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.643+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9380693b0 con 0x7fa9380cdd10 2024-04-26T23:39:02.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.644+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa924005d10 con 0x7fa9380cdd10 2024-04-26T23:39:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.645+0000 7fa934c00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.645+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa92408ccb0 con 0x7fa9380cdd10 2024-04-26T23:39:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.646+0000 7fa93cc00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:02.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.647+0000 7fa93cc00640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 0x7fa91806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa928032000 tx=0x7fa928009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.650+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa924024030 con 0x7fa9380cdd10 2024-04-26T23:39:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.890+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9380b3bc0 con 0x7fa9380cdd10 2024-04-26T23:39:02.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.891+0000 7fa934c00640 1 -- 172.21.15.46:0/4216121630 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7fa92405ae20 con 0x7fa9380cdd10 2024-04-26T23:39:02.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:39:02.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:38:54.220291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:39:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-26T23:39:02.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 msgr2=0x7fa91806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 0x7fa91806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa928032000 tx=0x7fa928009030 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 msgr2=0x7fa93809fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa924007f90 tx=0x7fa924011660 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 shutdown_connections 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9380ce170 0x7fa9380a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9380cdd10 0x7fa93809fb60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 --2- 172.21.15.46:0/4216121630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91c0039e0 0x7fa93809ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 >> 172.21.15.46:0/4216121630 conn(0x7fa9380d5010 msgr2=0x7fa9380b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 shutdown_connections 2024-04-26T23:39:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:02.894+0000 7fa93e000640 1 -- 172.21.15.46:0/4216121630 wait complete. 2024-04-26T23:39:03.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:38:51.149821+0000', 'modified': '2024-04-26T23:38:54.220291+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5280': {'gid': 5280, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1027205255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 1027205255}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1027205255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5280, 'qdb_cluster': [5280]} 2024-04-26T23:39:03.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:39:03.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:39:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.533+0000 7fee73e00640 1 Processor -- start 2024-04-26T23:39:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.533+0000 7fee73e00640 1 -- start start 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 0x7fee7406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee740720e0 0x7fee740cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee73e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee74073ad0 con 0x7fee74160910 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee73e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fee74146a90 con 0x7fee740720e0 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee73e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee74073d70 con 0x7fee741466b0 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.534+0000 7fee6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee740720e0 0x7fee740cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee740720e0 0x7fee740cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33818/0 (socket says 172.21.15.46:33818) 2024-04-26T23:39:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 -- 172.21.15.46:0/2340056510 learned_addr learned my addr 172.21.15.46:0/2340056510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee72a00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 0x7fee7406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 -- 172.21.15.46:0/2340056510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 msgr2=0x7fee7406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 0x7fee7406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 -- 172.21.15.46:0/2340056510 >> v1:172.21.15.46:6789/0 conn(0x7fee74160910 legacy=0x7fee740cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee6be00640 1 -- 172.21.15.46:0/2340056510 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee74074050 con 0x7fee740720e0 2024-04-26T23:39:03.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.535+0000 7fee72a00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 0x7fee7406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.536+0000 7fee6be00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee740720e0 0x7fee740cd860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef4712b68955dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.536+0000 7fee72000640 1 -- 172.21.15.46:0/2340056510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c037070 con 0x7fee740720e0 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.536+0000 7fee72000640 1 -- 172.21.15.46:0/2340056510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c032440 con 0x7fee740720e0 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.536+0000 7fee72000640 1 -- 172.21.15.46:0/2340056510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c032810 con 0x7fee740720e0 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 -- 172.21.15.46:0/2340056510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee740720e0 msgr2=0x7fee740cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee740720e0 0x7fee740cd860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c02a250 comp rx=0 tx=0).stop 2024-04-26T23:39:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 -- 172.21.15.46:0/2340056510 shutdown_connections 2024-04-26T23:39:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee740720e0 0x7fee740cd860 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 --2- 172.21.15.46:0/2340056510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee741466b0 0x7fee7406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 -- 172.21.15.46:0/2340056510 >> 172.21.15.46:0/2340056510 conn(0x7fee740b26d0 msgr2=0x7fee740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 -- 172.21.15.46:0/2340056510 shutdown_connections 2024-04-26T23:39:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.537+0000 7fee73e00640 1 -- 172.21.15.46:0/2340056510 wait complete. 2024-04-26T23:39:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.539+0000 7fee73e00640 1 Processor -- start 2024-04-26T23:39:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.539+0000 7fee73e00640 1 -- start start 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 0x7fee74141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 0x7fee740b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee7406e3c0 con 0x7fee741466b0 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee740cb470 con 0x7fee740720e0 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.540+0000 7fee73e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee740cdda0 con 0x7fee74160910 2024-04-26T23:39:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33832/0 (socket says 172.21.15.46:33832) 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 -- 172.21.15.46:0/1989254976 learned_addr learned my addr 172.21.15.46:0/1989254976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee73400640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 0x7fee740b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 -- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 msgr2=0x7fee740b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee72a00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 0x7fee74141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 0x7fee740b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 -- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 msgr2=0x7fee74141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 0x7fee74141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 -- 172.21.15.46:0/1989254976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee740b6390 con 0x7fee741466b0 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee73400640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 0x7fee740b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee72a00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 0x7fee74141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.541+0000 7fee6be00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.542+0000 7fee6aa00640 1 -- 172.21.15.46:0/1989254976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c041070 con 0x7fee741466b0 2024-04-26T23:39:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.542+0000 7fee6aa00640 1 -- 172.21.15.46:0/1989254976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c03c870 con 0x7fee741466b0 2024-04-26T23:39:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.542+0000 7fee6aa00640 1 -- 172.21.15.46:0/1989254976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c037040 con 0x7fee741466b0 2024-04-26T23:39:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.542+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee740b6670 con 0x7fee741466b0 2024-04-26T23:39:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.542+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee740ae8e0 con 0x7fee741466b0 2024-04-26T23:39:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.544+0000 7fee6aa00640 1 -- 172.21.15.46:0/1989254976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee5c044070 con 0x7fee741466b0 2024-04-26T23:39:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.545+0000 7fee6aa00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 0x7fee5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.545+0000 7fee6aa00640 1 -- 172.21.15.46:0/1989254976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fee5c0ac890 con 0x7fee741466b0 2024-04-26T23:39:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.546+0000 7fee72a00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 0x7fee5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.546+0000 7fee7c399740 1 Processor -- start 2024-04-26T23:39:03.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.546+0000 7fee7c399740 1 -- start start 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a027804b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a02782f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a027768600 con 0x55a0278047b0 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a027705100 con 0x55a027840e80 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0277645c0 con 0x55a02782f270 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee72a00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 0x7fee5006ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33844/0 (socket says 172.21.15.46:33844) 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee72a00640 1 -- 172.21.15.46:0/79472555 learned_addr learned my addr 172.21.15.46:0/79472555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.547+0000 7fee6be00640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a027804b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee73400640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a02782f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 -- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 msgr2=0x55a02782f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a02782f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 -- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 msgr2=0x55a027804b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a027804b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 -- 172.21.15.46:0/79472555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02747a2d0 con 0x55a0278047b0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee73400640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a02782f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee72a00640 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fee6400db00 tx=0x7fee64010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263a1f004aaaa082 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee63000640 1 -- 172.21.15.46:0/79472555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64019050 con 0x55a0278047b0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee63000640 1 -- 172.21.15.46:0/79472555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64023070 con 0x55a0278047b0 2024-04-26T23:39:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.548+0000 7fee63000640 1 -- 172.21.15.46:0/79472555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64012cc0 con 0x55a0278047b0 2024-04-26T23:39:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 -- 172.21.15.46:0/79472555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 msgr2=0x55a027806910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fee6400db00 tx=0x7fee64010000 comp rx=0 tx=0).stop 2024-04-26T23:39:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 -- 172.21.15.46:0/79472555 shutdown_connections 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a02782f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a027804b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 --2- 172.21.15.46:0/79472555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a027806910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 -- 172.21.15.46:0/79472555 >> 172.21.15.46:0/79472555 conn(0x55a027798d10 msgr2=0x55a02784a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 -- 172.21.15.46:0/79472555 shutdown_connections 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.549+0000 7fee7c399740 1 -- 172.21.15.46:0/79472555 wait complete. 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.550+0000 7fee7c399740 1 -- start start 2024-04-26T23:39:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.550+0000 7fee7c399740 1 Processor -- start 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0278047b0 0x55a02782be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a02784d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0276eff70 con 0x55a027840e80 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a02782cdc0 con 0x55a02782f270 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a02784b5b0 con 0x55a0278047b0 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee7c399740 10 client.0 _ll_register_callbacks cb 0x55a02775e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.551+0000 7fee73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33848/0 (socket says 172.21.15.46:33848) 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 -- 172.21.15.46:0/2083884379 learned_addr learned my addr 172.21.15.46:0/2083884379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee72a00640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0278047b0 0x55a02782be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee6be00640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a02784d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 -- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0278047b0 msgr2=0x55a02782be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0278047b0 0x55a02782be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 -- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 msgr2=0x55a02784d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a02784d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 -- 172.21.15.46:0/2083884379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0275446b0 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee73400640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00e4a0 tx=0x7fee6c00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a630d141065f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.552+0000 7fee57400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c003b70 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fee6c003d10 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64023070 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fee6c01f050 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 10 client.5484 fetch_fsmap learned FSMap version 18 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55a02785a970 con 0x55a027840e80 2024-04-26T23:39:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64023070 con 0x55a027840e80 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fee6c022020 con 0x55a027840e80 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 10 client.5484 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 10 client.5484 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 20 client.5484 populate_metadata read hostname 'smithi046' 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.553+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a02785d5a0 0x55a027824810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.554+0000 7fee7c399740 4 client.5484 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.554+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a027776210 con 0x55a02785d5a0 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.554+0000 7fee72a00640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a02785d5a0 0x55a027824810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.555+0000 7fee72a00640 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a02785d5a0 0x55a027824810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab265ca81ab7cda server_cookie=f81ec5d0f822f20f in_seq=0 out_seq=0 2024-04-26T23:39:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.555+0000 7fee57400640 10 client.5484 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:39:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.555+0000 7fee57400640 1 -- 172.21.15.46:0/2083884379 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a027776210 con 0x55a02785d5a0 2024-04-26T23:39:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.555+0000 7fee57400640 10 client.5484 handle_command_reply: tid=0 2024-04-26T23:39:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.555+0000 7fee7c399740 1 client.5484 shutdown 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 msgr2=0x55a0278187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00e4a0 tx=0x7fee6c00e960 comp rx=0 tx=0).stop 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 shutdown_connections 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a02785d5a0 0x55a027824810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a027840e80 0x55a0278187f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a02784d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 --2- 172.21.15.46:0/2083884379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0278047b0 0x55a02782be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee57400640 0 client.5484 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 >> 172.21.15.46:0/2083884379 conn(0x55a027798d10 msgr2=0x55a02782cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 shutdown_connections 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 1 -- 172.21.15.46:0/2083884379 wait complete. 2024-04-26T23:39:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.556+0000 7fee7c399740 20 client.5484 trim_cache size 0 max 16384 2024-04-26T23:39:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.576+0000 7fee7c399740 1 Processor -- start 2024-04-26T23:39:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.576+0000 7fee7c399740 1 -- start start 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a027818600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a027840e80 0x55a02782b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee74200d00 con 0x55a0278047b0 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee740cfbb0 con 0x55a02782f270 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee742006b0 con 0x55a027840e80 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33862/0 (socket says 172.21.15.46:33862) 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.577+0000 7fee72a00640 1 -- 172.21.15.46:0/3841037674 learned_addr learned my addr 172.21.15.46:0/3841037674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee6be00640 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a027818600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 -- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a027840e80 msgr2=0x55a02782b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a027840e80 0x55a02782b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 -- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 msgr2=0x55a027818600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a027818600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 -- 172.21.15.46:0/3841037674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a027825c60 con 0x55a0278047b0 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.578+0000 7fee72a00640 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee6400dad0 tx=0x7fee6401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaa00015b27333a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee62600640 1 -- 172.21.15.46:0/3841037674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6403b070 con 0x55a0278047b0 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee62600640 1 -- 172.21.15.46:0/3841037674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64023040 con 0x55a0278047b0 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee62600640 1 -- 172.21.15.46:0/3841037674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64012ed0 con 0x55a0278047b0 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 -- 172.21.15.46:0/3841037674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 msgr2=0x55a02782f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee6400dad0 tx=0x7fee6401c000 comp rx=0 tx=0).stop 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 -- 172.21.15.46:0/3841037674 shutdown_connections 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a027840e80 0x55a02782b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a02782f270 0x55a027818600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 --2- 172.21.15.46:0/3841037674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782f650 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.579+0000 7fee7c399740 1 -- 172.21.15.46:0/3841037674 >> 172.21.15.46:0/3841037674 conn(0x55a027798d10 msgr2=0x55a027832400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.580+0000 7fee7c399740 1 -- 172.21.15.46:0/3841037674 shutdown_connections 2024-04-26T23:39:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.580+0000 7fee7c399740 1 -- 172.21.15.46:0/3841037674 wait complete. 2024-04-26T23:39:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.580+0000 7fee7c399740 1 -- start start 2024-04-26T23:39:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.580+0000 7fee7c399740 1 Processor -- start 2024-04-26T23:39:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a0278cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a02786cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a02785d420 con 0x55a0278047b0 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a027818d30 con 0x55a027840e80 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a02785da70 con 0x55a02782f270 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.581+0000 7fee7c399740 10 client.0 _ll_register_callbacks cb 0x7fee6c0014c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33864/0 (socket says 172.21.15.46:33864) 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 -- 172.21.15.46:0/2536582838 learned_addr learned my addr 172.21.15.46:0/2536582838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee6be00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a0278cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee73400640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a02786cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 -- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 msgr2=0x55a0278cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a0278cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 -- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 msgr2=0x55a02786cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a02786cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 -- 172.21.15.46:0/2536582838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02782b6c0 con 0x55a0278047b0 2024-04-26T23:39:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee72a00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee6400db00 tx=0x7fee64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aff51183457791 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.582+0000 7fee57400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6403b040 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fee64005990 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64023040 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fee6404c050 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee7c399740 10 client.5490 fetch_fsmap learned FSMap version 18 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55a0278186c0 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64023040 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fee6404f020 con 0x55a0278047b0 2024-04-26T23:39:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee7c399740 10 client.5490 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.583+0000 7fee7c399740 10 client.5490 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee7c399740 20 client.5490 populate_metadata read hostname 'smithi046' 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a027817200 0x55a0278175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee7c399740 4 client.5490 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "status"}) -- 0x55a0278301a0 con 0x55a027817200 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee6be00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a027817200 0x55a0278175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee6be00640 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a027817200 0x55a0278175e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466410248d30a6da server_cookie=5b4c0504ca8c66b1 in_seq=0 out_seq=0 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.584+0000 7fee57400640 10 client.5490 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee57400640 1 -- 172.21.15.46:0/2536582838 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55a0278301a0 con 0x55a027817200 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee57400640 10 client.5490 handle_command_reply: tid=0 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 client.5490 shutdown 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 msgr2=0x55a02782a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee6400db00 tx=0x7fee64004730 comp rx=0 tx=0).stop 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 shutdown_connections 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a027817200 0x55a0278175e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a027840e80 0x55a02786cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a02782f270 0x55a0278cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 --2- 172.21.15.46:0/2536582838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0278047b0 0x55a02782a160 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee57400640 0 client.5490 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 >> 172.21.15.46:0/2536582838 conn(0x55a027798d10 msgr2=0x55a027832400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 shutdown_connections 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 1 -- 172.21.15.46:0/2536582838 wait complete. 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.585+0000 7fee7c399740 20 client.5490 trim_cache size 0 max 16384 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5280, 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:39:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 11.925985286, 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 18, 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 39, 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 39, 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 20.87357364 2024-04-26T23:39:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 msgr2=0x7fee5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 0x7fee5006ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64008040 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 msgr2=0x7fee740b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 shutdown_connections 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5006c920 0x7fee5006ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee74160910 0x7fee740b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee741466b0 0x7fee740b55d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 --2- 172.21.15.46:0/1989254976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee740720e0 0x7fee74141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 >> 172.21.15.46:0/1989254976 conn(0x7fee740b26d0 msgr2=0x7fee74160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 shutdown_connections 2024-04-26T23:39:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.586+0000 7fee73e00640 1 -- 172.21.15.46:0/1989254976 wait complete. 2024-04-26T23:39:03.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:39:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.817+0000 7f0dfb000640 1 Processor -- start 2024-04-26T23:39:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.818+0000 7f0dfb000640 1 -- start start 2024-04-26T23:39:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0dfb000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0df40d0c10 0x7f0df40d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0dfb000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df40cfe50 con 0x7f0df40d1990 2024-04-26T23:39:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0dfb000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df40d0130 con 0x7f0df41639c0 2024-04-26T23:39:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0dfb000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0df40d0780 con 0x7f0df40d0c10 2024-04-26T23:39:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0df9c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0df40d1990 0x7f0df40d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58568/0 (socket says 172.21.15.46:58568) 2024-04-26T23:39:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.819+0000 7f0df9c00640 1 -- 172.21.15.46:0/4011958893 learned_addr learned my addr 172.21.15.46:0/4011958893 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:39:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df9200640 1 --2- 172.21.15.46:0/4011958893 >> v2:172.21.15.46:3300/0 conn(0x7f0df40d0c10 0x7f0df40d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df3e00640 1 -- 172.21.15.46:0/4011958893 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471169895 0 0) 0x7f0df40cfe50 con 0x7f0df40d1990 2024-04-26T23:39:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df3e00640 1 -- 172.21.15.46:0/4011958893 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ddc003520 con 0x7f0df40d1990 2024-04-26T23:39:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df9200640 1 -- 172.21.15.46:0/4011958893 >> v1:172.21.15.142:6789/0 conn(0x7f0df41639c0 legacy=0x7f0df40b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df9200640 1 -- 172.21.15.46:0/4011958893 >> v1:172.21.15.46:6789/0 conn(0x7f0df40d1990 legacy=0x7f0df40d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.820+0000 7f0df9200640 1 -- 172.21.15.46:0/4011958893 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df40d0410 con 0x7f0df40d0c10 2024-04-26T23:39:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.822+0000 7f0df9200640 1 --2- 172.21.15.46:0/4011958893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df40d0c10 0x7f0df40d5690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0de00097a0 tx=0x7f0de002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bebb3ab0f14cb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.822+0000 7f0df3e00640 1 -- 172.21.15.46:0/4011958893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0037070 con 0x7f0df40d0c10 2024-04-26T23:39:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.822+0000 7f0df3e00640 1 -- 172.21.15.46:0/4011958893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de0032440 con 0x7f0df40d0c10 2024-04-26T23:39:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.822+0000 7f0df3e00640 1 -- 172.21.15.46:0/4011958893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0032810 con 0x7f0df40d0c10 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 -- 172.21.15.46:0/4011958893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df40d0c10 msgr2=0x7f0df40d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 --2- 172.21.15.46:0/4011958893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df40d0c10 0x7f0df40d5690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0de00097a0 tx=0x7f0de002a230 comp rx=0 tx=0).stop 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 -- 172.21.15.46:0/4011958893 shutdown_connections 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 --2- 172.21.15.46:0/4011958893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df40d0c10 0x7f0df40d5690 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 -- 172.21.15.46:0/4011958893 >> 172.21.15.46:0/4011958893 conn(0x7f0df409e2a0 msgr2=0x7f0df40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 -- 172.21.15.46:0/4011958893 shutdown_connections 2024-04-26T23:39:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.823+0000 7f0dfb000640 1 -- 172.21.15.46:0/4011958893 wait complete. 2024-04-26T23:39:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.824+0000 7f0dfb000640 1 Processor -- start 2024-04-26T23:39:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.825+0000 7f0dfb000640 1 -- start start 2024-04-26T23:39:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df40d0c10 0x7f0df41e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df40d1990 0x7f0df415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0df40cfc10 con 0x7f0df41639c0 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0df40cfa90 con 0x7f0df40d1990 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfb000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0df40d5bd0 con 0x7f0df40d0c10 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33886/0 (socket says 172.21.15.46:33886) 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0dfa600640 1 -- 172.21.15.46:0/1468471584 learned_addr learned my addr 172.21.15.46:0/1468471584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0df9c00640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df40d0c10 0x7f0df41e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.826+0000 7f0df9200640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df40d1990 0x7f0df415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 -- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df40d0c10 msgr2=0x7f0df41e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df40d0c10 0x7f0df41e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 -- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df40d1990 msgr2=0x7f0df415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df40d1990 0x7f0df415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 -- 172.21.15.46:0/1468471584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df40ca6e0 con 0x7f0df41639c0 2024-04-26T23:39:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfa600640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c760 tx=0x7f0dec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec003b70 con 0x7f0df41639c0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec003d10 con 0x7f0df41639c0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df40ca9c0 con 0x7f0df41639c0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.827+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00aa90 con 0x7f0df41639c0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.828+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df40caf80 con 0x7f0df41639c0 2024-04-26T23:39:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.829+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df40693b0 con 0x7f0df41639c0 2024-04-26T23:39:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.830+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dec00ac30 con 0x7f0df41639c0 2024-04-26T23:39:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.831+0000 7f0df1600640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 0x7f0dd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.831+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0dec08bad0 con 0x7f0df41639c0 2024-04-26T23:39:03.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.832+0000 7f0df9c00640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 0x7f0dd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:03.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.833+0000 7f0df9c00640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 0x7f0dd406ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0de80011e0 tx=0x7f0de8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:03.836+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0dec059d60 con 0x7f0df41639c0 2024-04-26T23:39:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.083+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0df40d3290 con 0x7f0df41639c0 2024-04-26T23:39:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.084+0000 7f0df1600640 1 -- 172.21.15.46:0/1468471584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f0dec059660 con 0x7f0df41639c0 2024-04-26T23:39:04.088 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:39:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 msgr2=0x7f0dd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 0x7f0dd406ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0de80011e0 tx=0x7f0de8009040 comp rx=0 tx=0).stop 2024-04-26T23:39:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 msgr2=0x7f0df40c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c760 tx=0x7f0dec00cc20 comp rx=0 tx=0).stop 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 shutdown_connections 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dd406c920 0x7f0dd406ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0df41639c0 0x7f0df40c9eb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df40d1990 0x7f0df415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 --2- 172.21.15.46:0/1468471584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df40d0c10 0x7f0df41e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 >> 172.21.15.46:0/1468471584 conn(0x7f0df409e2a0 msgr2=0x7f0df40d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 shutdown_connections 2024-04-26T23:39:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04.087+0000 7f0dfb000640 1 -- 172.21.15.46:0/1468471584 wait complete. 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:39:04.111 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:39:04.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.112 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:39:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42477sec preferred_lft 42477sec 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:39:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:39:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:39:04.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.169 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:39:04.252 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:39:04.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.256 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:39:04.310 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:39:04.310 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:39:04.310 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:39:04.310 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:39:04.310 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:39:04.311 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:39:04.311 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:39:04.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:39:04.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:39:04.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:39:04.312 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:39:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:39:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:39:04.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.525 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:39:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:39:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:39:04.577 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:39:04.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:39:04.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:39:04.667 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:39:04.724 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:39:04.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:39:04.724 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:39:04.779 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:39:04.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:39:04.911 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:39:04.911 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:39:04.911 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:39:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:39:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:39:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:39:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:39:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:39:04.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:39:04.966 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:39:04.968 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:39:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:39:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:39:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:39:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:39:04.973 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:39:04.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:04.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:39:05.061 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:39:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:39:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:39:05.200 INFO:teuthology.orchestra.run.smithi046.stdout:client4802 2024-04-26T23:39:05.200 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:39:05.201 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:39:05.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-26T23:39:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.511+0000 7f177be00640 1 Processor -- start 2024-04-26T23:39:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.512+0000 7f177be00640 1 -- start start 2024-04-26T23:39:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.513+0000 7f177be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f177c0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.513+0000 7f177be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f177c0d2080 0x7f177c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.513+0000 7f177be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177c0d01a0 con 0x7f177c1627f0 2024-04-26T23:39:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f177c0d0480 con 0x7f177c0d2080 2024-04-26T23:39:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0d0760 con 0x7f177c0d3cc0 2024-04-26T23:39:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f177c1627f0 0x7f177c0d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58582/0 (socket says 172.21.15.46:58582) 2024-04-26T23:39:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177aa00640 1 -- 172.21.15.46:0/4001744597 learned_addr learned my addr 172.21.15.46:0/4001744597 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177a000640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f177c0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f177b400640 1 --2- 172.21.15.46:0/4001744597 >> v2:172.21.15.142:3300/0 conn(0x7f177c0d2080 0x7f177c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141217472 0 0) 0x7f177c0d01a0 con 0x7f177c1627f0 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.514+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1760003520 con 0x7f177c1627f0 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1583062118 0 0) 0x7f1760003520 con 0x7f177c1627f0 2024-04-26T23:39:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 msgr2=0x7f177c0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f177c0cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 >> v2:172.21.15.142:3300/0 conn(0x7f177c0d2080 msgr2=0x7f177c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 --2- 172.21.15.46:0/4001744597 >> v2:172.21.15.142:3300/0 conn(0x7f177c0d2080 0x7f177c0b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0d0a40 con 0x7f177c1627f0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f176c001670 con 0x7f177c1627f0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 >> v1:172.21.15.46:6789/0 conn(0x7f177c1627f0 legacy=0x7f177c0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f17600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1760003f80 0x7f1760004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f17600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d0480 con 0x7f1760004d40 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d0760 con 0x7f1760003f80 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f176000ba40 con 0x7f177c0d3cc0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.515+0000 7f177b400640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f17600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177aa00640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f17600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177a000640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1760003f80 0x7f1760004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177b400640 1 -- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 msgr2=0x7f17600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177b400640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f17600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177b400640 1 -- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1760003f80 msgr2=0x7f1760004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177b400640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1760003f80 0x7f1760004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177b400640 1 -- 172.21.15.46:0/4001744597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1760003520 con 0x7f1760004d40 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177aa00640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c0d3cc0 0x7f17600039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:05.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.516+0000 7f177a000640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1760003f80 0x7f1760004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.517+0000 7f177b400640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f17600051c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ac0 tx=0x7f1760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8071ca8d5ee544 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.517+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774036070 con 0x7f1760004d40 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.517+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1774031400 con 0x7f1760004d40 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.517+0000 7f1779600640 1 -- 172.21.15.46:0/4001744597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774009690 con 0x7f1760004d40 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 -- 172.21.15.46:0/4001744597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 msgr2=0x7f17600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 --2- 172.21.15.46:0/4001744597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f17600051c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ac0 tx=0x7f1760005830 comp rx=0 tx=0).stop 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177b400640 1 -- 172.21.15.46:0/4001744597 reap_dead start 2024-04-26T23:39:05.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 -- 172.21.15.46:0/4001744597 shutdown_connections 2024-04-26T23:39:05.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 -- 172.21.15.46:0/4001744597 >> 172.21.15.46:0/4001744597 conn(0x7f177c09e380 msgr2=0x7f177c0d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:05.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 -- 172.21.15.46:0/4001744597 shutdown_connections 2024-04-26T23:39:05.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.518+0000 7f177be00640 1 -- 172.21.15.46:0/4001744597 wait complete. 2024-04-26T23:39:05.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.520+0000 7f177be00640 1 Processor -- start 2024-04-26T23:39:05.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.520+0000 7f177be00640 1 -- start start 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177c0c8aa0 0x7f177c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 0x7f177c06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f177c162bd0 con 0x7f1760004d40 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d2500 con 0x7f177c0c8aa0 2024-04-26T23:39:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0cfbc0 con 0x7f177c06d970 2024-04-26T23:39:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.521+0000 7f177aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33902/0 (socket says 172.21.15.46:33902) 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 -- 172.21.15.46:0/1054746328 learned_addr learned my addr 172.21.15.46:0/1054746328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177a000640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177c0c8aa0 0x7f177c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177b400640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 0x7f177c06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 -- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 msgr2=0x7f177c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 0x7f177c06ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 -- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177c0c8aa0 msgr2=0x7f177c0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177c0c8aa0 0x7f177c0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 -- 172.21.15.46:0/1054746328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0c9b20 con 0x7f1760004d40 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177b400640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 0x7f177c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.522+0000 7f177aa00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d8e0 tx=0x7f176c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.523+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c019070 con 0x7f1760004d40 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.523+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c0074e0 con 0x7f1760004d40 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.523+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0049e0 con 0x7f1760004d40 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.523+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c0c9560 con 0x7f1760004d40 2024-04-26T23:39:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.523+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c0b7c90 con 0x7f1760004d40 2024-04-26T23:39:05.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.524+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c0693b0 con 0x7f1760004d40 2024-04-26T23:39:05.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.525+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f176c01c070 con 0x7f1760004d40 2024-04-26T23:39:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.526+0000 7f1773400640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:39:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.526+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f176c08ba30 con 0x7f1760004d40 2024-04-26T23:39:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.526+0000 7f177a000640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:39:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.528+0000 7f177a000640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 0x7f175806ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f177c0d3970 tx=0x7f176400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:39:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.530+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f176c059c30 con 0x7f1760004d40 2024-04-26T23:39:05.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:05.778+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f177c0d3390 con 0x7f1760004d40 2024-04-26T23:39:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.782+0000 7f1773400640 1 -- 172.21.15.46:0/1054746328 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v19) ==== 167+0+0 (secure 0 0 0) 0x7f176c059530 con 0x7f1760004d40 2024-04-26T23:39:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:enabled new snapshots 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.785+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 msgr2=0x7f175806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.785+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 0x7f175806ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f177c0d3970 tx=0x7f176400b040 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 msgr2=0x7f177c159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d8e0 tx=0x7f176c00dda0 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 shutdown_connections 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f177c06d970 0x7f177c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177c0c8aa0 0x7f177c0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 --2- 172.21.15.46:0/1054746328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760004d40 0x7f177c159590 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:39:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 >> 172.21.15.46:0/1054746328 conn(0x7f177c09e380 msgr2=0x7f177c1602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:39:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 shutdown_connections 2024-04-26T23:39:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:39:06.786+0000 7f177be00640 1 -- 172.21.15.46:0/1054746328 wait complete. 2024-04-26T23:39:06.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:39:06.813 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-26T23:39:06.879 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:39:06.879 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:39:06.932 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> import uuid 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> n = 2000 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = False 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> if 3 > 0: 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = True 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> dpath = os.path.dirname(path) 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> fnameprefix = os.path.basename(path) 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> os.makedirs(dpath, exist_ok=True) 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:39:06.933 DEBUG:teuthology.orchestra.run.smithi046:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> for i in range(n): 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> f.write(f"{i}") 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> f.flush() 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(f.fileno()) 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> os.unlink(fpath) 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> if create_hard_links: 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> for j in range(3): 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T23:39:06.934 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:39:06.935 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:39:06.935 DEBUG:teuthology.orchestra.run.smithi046:> finally: 2024-04-26T23:39:06.935 DEBUG:teuthology.orchestra.run.smithi046:> os.close(dirfd) 2024-04-26T23:39:06.935 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:39:08.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:08.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:38.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:38.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:40:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:40:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:40:05.694 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-26T23:40:05.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:40:05.813 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-26T23:40:08.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:08.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:38.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:38.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:04.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:04.129 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-26T23:41:04.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-26T23:41:04.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.359+0000 7ff73de00640 1 Processor -- start 2024-04-26T23:41:04.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.359+0000 7ff73de00640 1 -- start start 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.360+0000 7ff73de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff738145750 0x7ff73806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.360+0000 7ff73de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 0x7ff7380b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.360+0000 7ff73de00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7380ce0c0 con 0x7ff7380b5120 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.360+0000 7ff73de00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff73806f860 con 0x7ff738145750 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.360+0000 7ff73de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7380b5940 con 0x7ff73806f400 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff73d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff7380b5120 0x7ff7380d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52574/0 (socket says 172.21.15.46:52574) 2024-04-26T23:41:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff73d400640 1 -- 172.21.15.46:0/114840814 learned_addr learned my addr 172.21.15.46:0/114840814 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:04.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff73ca00640 1 --2- 172.21.15.46:0/114840814 >> v2:172.21.15.142:3300/0 conn(0x7ff738145750 0x7ff73806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff737e00640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 0x7ff7380b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864749231 0 0) 0x7ff7380ce0c0 con 0x7ff7380b5120 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.361+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff720003520 con 0x7ff7380b5120 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 458065138 0 0) 0x7ff720003520 con 0x7ff7380b5120 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 msgr2=0x7ff7380b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 0x7ff7380b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 >> v2:172.21.15.142:3300/0 conn(0x7ff738145750 msgr2=0x7ff73806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 --2- 172.21.15.46:0/114840814 >> v2:172.21.15.142:3300/0 conn(0x7ff738145750 0x7ff73806eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7380ce330 con 0x7ff7380b5120 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff7300019f0 con 0x7ff7380b5120 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 >> v1:172.21.15.46:6789/0 conn(0x7ff7380b5120 legacy=0x7ff7380d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 0x7ff720004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7380b5940 con 0x7ff720004d40 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff72000dbd0 con 0x7ff7200039b0 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff72000dd50 con 0x7ff73806f400 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.362+0000 7ff73d400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73ca00640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73d400640 1 -- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 msgr2=0x7ff720004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73d400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73806f400 0x7ff720004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73d400640 1 -- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7200039b0 msgr2=0x7ff720003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73d400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.363+0000 7ff73d400640 1 -- 172.21.15.46:0/114840814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff720003520 con 0x7ff720004d40 2024-04-26T23:41:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.364+0000 7ff73d400640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff73000a870 tx=0x7ff720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb11bd2f672da5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.364+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000a4c0 con 0x7ff720004d40 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.364+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73002f070 con 0x7ff720004d40 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.364+0000 7ff737400640 1 -- 172.21.15.46:0/114840814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73002bcc0 con 0x7ff720004d40 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 -- 172.21.15.46:0/114840814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff720004d40 msgr2=0x7ff7200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 --2- 172.21.15.46:0/114840814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff73000a870 tx=0x7ff720005830 comp rx=0 tx=0).stop 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73d400640 1 -- 172.21.15.46:0/114840814 reap_dead start 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 -- 172.21.15.46:0/114840814 shutdown_connections 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 -- 172.21.15.46:0/114840814 >> 172.21.15.46:0/114840814 conn(0x7ff73806c350 msgr2=0x7ff73806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 -- 172.21.15.46:0/114840814 shutdown_connections 2024-04-26T23:41:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.365+0000 7ff73de00640 1 -- 172.21.15.46:0/114840814 wait complete. 2024-04-26T23:41:04.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.367+0000 7ff73de00640 1 Processor -- start 2024-04-26T23:41:04.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.367+0000 7ff73de00640 1 -- start start 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.368+0000 7ff73de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73813b370 0x7ff7380c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.368+0000 7ff73de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73813b750 0x7ff7380ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.368+0000 7ff73de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7380b55b0 con 0x7ff7380c7dc0 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7380b5c20 con 0x7ff73813b370 2024-04-26T23:41:04.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7380cfb40 con 0x7ff73813b750 2024-04-26T23:41:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58620/0 (socket says 172.21.15.46:58620) 2024-04-26T23:41:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 -- 172.21.15.46:0/994632799 learned_addr learned my addr 172.21.15.46:0/994632799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73ca00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73813b370 0x7ff7380c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff737e00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73813b750 0x7ff7380ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 -- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73813b750 msgr2=0x7ff7380ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73813b750 0x7ff7380ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 -- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73813b370 msgr2=0x7ff7380c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.369+0000 7ff73d400640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73813b370 0x7ff7380c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff73d400640 1 -- 172.21.15.46:0/994632799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7380c6c50 con 0x7ff7380c7dc0 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff73d400640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f80 tx=0x7ff730001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff734c00640 1 -- 172.21.15.46:0/994632799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000a4c0 con 0x7ff7380c7dc0 2024-04-26T23:41:04.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff734c00640 1 -- 172.21.15.46:0/994632799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff730034470 con 0x7ff7380c7dc0 2024-04-26T23:41:04.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff734c00640 1 -- 172.21.15.46:0/994632799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730002030 con 0x7ff7380c7dc0 2024-04-26T23:41:04.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7381f4d20 con 0x7ff7380c7dc0 2024-04-26T23:41:04.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.370+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7381f52e0 con 0x7ff7380c7dc0 2024-04-26T23:41:04.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.373+0000 7ff734c00640 1 -- 172.21.15.46:0/994632799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff730034840 con 0x7ff7380c7dc0 2024-04-26T23:41:04.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.374+0000 7ff734c00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.374+0000 7ff734c00640 1 -- 172.21.15.46:0/994632799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7ff7300abe30 con 0x7ff7380c7dc0 2024-04-26T23:41:04.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.374+0000 7ff742279740 1 Processor -- start 2024-04-26T23:41:04.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.374+0000 7ff73ca00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.374+0000 7ff742279740 1 -- start start 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0f89e20 0x5561a0fd00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5561a0eed900 con 0x5561a0fd19e0 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5561a0e8a1d0 con 0x5561a0fb4140 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.375+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5561a0ee98c0 con 0x5561a0f89e20 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58622/0 (socket says 172.21.15.46:58622) 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 -- 172.21.15.46:0/3285074068 learned_addr learned my addr 172.21.15.46:0/3285074068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff73ca00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 0x7ff71806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009a90 tx=0x7ff72c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff73ca00640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fb4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff73d400640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0f89e20 0x5561a0fd00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 -- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0f89e20 msgr2=0x5561a0fd00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0f89e20 0x5561a0fd00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 -- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 msgr2=0x5561a0fb4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fb4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff737e00640 1 -- 172.21.15.46:0/3285074068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a0cc97d0 con 0x5561a0fd19e0 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.376+0000 7ff73ca00640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fb4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.377+0000 7ff737e00640 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7240099a0 tx=0x7ff724009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83e20036c5acbdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.377+0000 7ff729200640 1 -- 172.21.15.46:0/3285074068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724019070 con 0x5561a0fd19e0 2024-04-26T23:41:04.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.377+0000 7ff729200640 1 -- 172.21.15.46:0/3285074068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff724011b00 con 0x5561a0fd19e0 2024-04-26T23:41:04.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.377+0000 7ff729200640 1 -- 172.21.15.46:0/3285074068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724015460 con 0x5561a0fd19e0 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 -- 172.21.15.46:0/3285074068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 msgr2=0x5561a0fd1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7240099a0 tx=0x7ff724009e60 comp rx=0 tx=0).stop 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 -- 172.21.15.46:0/3285074068 shutdown_connections 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0f89e20 0x5561a0fd00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fd1f30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 --2- 172.21.15.46:0/3285074068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fb4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 -- 172.21.15.46:0/3285074068 >> 172.21.15.46:0/3285074068 conn(0x5561a0f86750 msgr2=0x5561a0fcef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 -- 172.21.15.46:0/3285074068 shutdown_connections 2024-04-26T23:41:04.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.378+0000 7ff742279740 1 -- 172.21.15.46:0/3285074068 wait complete. 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.379+0000 7ff742279740 1 -- start start 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.379+0000 7ff742279740 1 Processor -- start 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fa0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fa0660 0x5561a0fc6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5561a0f8abc0 con 0x5561a0fd19e0 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5561a0e75120 con 0x5561a0fb4140 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5561a0f84680 con 0x5561a0fa0660 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff742279740 10 client.0 _ll_register_callbacks cb 0x5561a0ead750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58628/0 (socket says 172.21.15.46:58628) 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.380+0000 7ff737e00640 1 -- 172.21.15.46:0/2259274260 learned_addr learned my addr 172.21.15.46:0/2259274260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fa0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff73d400640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fa0660 0x5561a0fc6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 -- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fa0660 msgr2=0x5561a0fc6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fa0660 0x5561a0fc6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 -- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 msgr2=0x5561a0fa0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fa0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 -- 172.21.15.46:0/2259274260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a0bbf0f0 con 0x5561a0fd19e0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff737e00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff724004770 tx=0x7ff7240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e0b7e3fb1129cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff71d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.381+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724019070 con 0x5561a0fd19e0 2024-04-26T23:41:04.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7ff724005b90 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff724011b00 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff724031050 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 10 client.5517 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5561a0d29f60 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff724011b00 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff724034020 con 0x5561a0fd19e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 10 client.5517 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 10 client.5517 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 20 client.5517 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f9f1e0 0x5561a0f9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 4 client.5517 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.382+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561a0efb510 con 0x5561a0f9f1e0 2024-04-26T23:41:04.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.383+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f9f1e0 0x5561a0f9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.383+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f9f1e0 0x5561a0f9f620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b14d0f97af67f3d server_cookie=d512b7ddab82e993 in_seq=0 out_seq=0 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.383+0000 7ff71d600640 10 client.5517 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff71d600640 1 -- 172.21.15.46:0/2259274260 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561a0efb510 con 0x5561a0f9f1e0 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff71d600640 10 client.5517 handle_command_reply: tid=0 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 client.5517 shutdown 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 msgr2=0x5561a0fc6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff724004770 tx=0x7ff7240046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 shutdown_connections 2024-04-26T23:41:04.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f9f1e0 0x5561a0f9f620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fa0660 0x5561a0fc6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6000 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff71d600640 0 client.5517 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.384+0000 7ff742279740 1 --2- 172.21.15.46:0/2259274260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fb4140 0x5561a0fa0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.385+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 >> 172.21.15.46:0/2259274260 conn(0x5561a0f89e20 msgr2=0x5561a0fd24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.385+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 shutdown_connections 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.385+0000 7ff742279740 1 -- 172.21.15.46:0/2259274260 wait complete. 2024-04-26T23:41:04.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.385+0000 7ff742279740 20 client.5517 trim_cache size 0 max 16384 2024-04-26T23:41:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.403+0000 7ff742279740 1 Processor -- start 2024-04-26T23:41:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.403+0000 7ff742279740 1 -- start start 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fa0660 0x5561a0f835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0fb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff738202770 con 0x5561a0fd19e0 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7382017b0 con 0x5561a0fa0660 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7381f6d60 con 0x5561a0fb4140 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58630/0 (socket says 172.21.15.46:58630) 2024-04-26T23:41:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 -- 172.21.15.46:0/3528390004 learned_addr learned my addr 172.21.15.46:0/3528390004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 -- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 msgr2=0x5561a0fb7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0fb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 -- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fa0660 msgr2=0x5561a0f835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fa0660 0x5561a0f835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.404+0000 7ff73d400640 1 -- 172.21.15.46:0/3528390004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a0b981b0 con 0x5561a0fd19e0 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff73d400640 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff730001cf0 tx=0x7ff730080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464615e52d77d1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff71fe00640 1 -- 172.21.15.46:0/3528390004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300799a0 con 0x5561a0fd19e0 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff71fe00640 1 -- 172.21.15.46:0/3528390004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73002f070 con 0x5561a0fd19e0 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff71fe00640 1 -- 172.21.15.46:0/3528390004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73007dd50 con 0x5561a0fd19e0 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff742279740 1 -- 172.21.15.46:0/3528390004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 msgr2=0x5561a0fa3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.405+0000 7ff742279740 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff730001cf0 tx=0x7ff730080040 comp rx=0 tx=0).stop 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 -- 172.21.15.46:0/3528390004 shutdown_connections 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fd19e0 0x5561a0fa3390 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0fb7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 --2- 172.21.15.46:0/3528390004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fa0660 0x5561a0f835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 -- 172.21.15.46:0/3528390004 >> 172.21.15.46:0/3528390004 conn(0x5561a0f89e20 msgr2=0x5561a0fce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 -- 172.21.15.46:0/3528390004 shutdown_connections 2024-04-26T23:41:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 -- 172.21.15.46:0/3528390004 wait complete. 2024-04-26T23:41:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.406+0000 7ff742279740 1 -- start start 2024-04-26T23:41:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.407+0000 7ff742279740 1 Processor -- start 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.407+0000 7ff742279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.407+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0f9f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.407+0000 7ff742279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff742279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5561a0f275f0 con 0x5561a0fa0660 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff738201160 con 0x5561a0fd19e0 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff742279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5561a0ff30b0 con 0x5561a0fb4140 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff742279740 10 client.0 _ll_register_callbacks cb 0x7ff724009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58632/0 (socket says 172.21.15.46:58632) 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 -- 172.21.15.46:0/2627832939 learned_addr learned my addr 172.21.15.46:0/2627832939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 -- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 msgr2=0x5561a0f9f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff737e00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0f9f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73d400640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0f9f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 -- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fd19e0 msgr2=0x5561a0fc6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.408+0000 7ff73ca00640 1 -- 172.21.15.46:0/2627832939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a0fcfc60 con 0x5561a0fa0660 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff73ca00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5561a0f8ac50 tx=0x7ff72c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61d936486fc74ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff71d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c004020 con 0x5561a0fa0660 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7ff72c0041c0 con 0x5561a0fa0660 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff73002f070 con 0x5561a0fa0660 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff72c037050 con 0x5561a0fa0660 2024-04-26T23:41:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff742279740 10 client.5523 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5561a0ff2760 con 0x5561a0fa0660 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.409+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73002f070 con 0x5561a0fa0660 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff72c03a020 con 0x5561a0fa0660 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 10 client.5523 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 10 client.5523 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 20 client.5523 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f84c70 0x5561a0fc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 4 client.5523 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "perf dump"}) -- 0x5561a0fc7c80 con 0x5561a0f84c70 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.410+0000 7ff737e00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f84c70 0x5561a0fc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.411+0000 7ff737e00640 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f84c70 0x5561a0fc6550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc72028bd046358d server_cookie=7453576345d0ecd4 in_seq=0 out_seq=0 2024-04-26T23:41:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.411+0000 7ff71d600640 10 client.5523 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.411+0000 7ff71d600640 1 -- 172.21.15.46:0/2627832939 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17991 (crc 0 0 0) 0x5561a0fc7c80 con 0x5561a0f84c70 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.411+0000 7ff71d600640 10 client.5523 handle_command_reply: tid=0 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.411+0000 7ff742279740 1 client.5523 shutdown 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 msgr2=0x5561a1048840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5561a0f8ac50 tx=0x7ff72c009190 comp rx=0 tx=0).stop 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 shutdown_connections 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5561a0f84c70 0x5561a0fc6550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a0fd19e0 0x5561a0fc6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5561a0fb4140 0x5561a0f9f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 --2- 172.21.15.46:0/2627832939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561a0fa0660 0x5561a1048840 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 >> 172.21.15.46:0/2627832939 conn(0x5561a0f89e20 msgr2=0x5561a0fb2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff71d600640 0 client.5523 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 shutdown_connections 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 1 -- 172.21.15.46:0/2627832939 wait complete. 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.412+0000 7ff742279740 20 client.5523 trim_cache size 0 max 16384 2024-04-26T23:41:04.420 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1215, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1207, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 411728, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 15366556, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.336637761, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.180711428, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.087596459, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.066955914, 2024-04-26T23:41:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1207, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.486623384, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000403167 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000275, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000137 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 125216, 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 31920 2024-04-26T23:41:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1207, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1236, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 303253, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 21993906, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.346987597, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.204963031, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.052145623, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.067395165, 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1235, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.568455731, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000460288 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 102992 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:41:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 29286, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 51290, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 10219074, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 76397197, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 9.080760874, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 6.458203608, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 2.798059203, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.271089933, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 51289, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.952582490, 2024-04-26T23:41:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000077064 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 46464, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.025660688, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000552 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-26T23:41:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1461, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 31.442784867, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.021521413 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2090, 2024-04-26T23:41:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.962028908, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000938769 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 25580, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 25580, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 25580, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 615.630217364, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.024066857 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 12, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 19913, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 19913, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 19912, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-26T23:41:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 33, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 33, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 2, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 46585, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 46014, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 111, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-26T23:41:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:41:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 979, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 4031, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 3045, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 2207, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 54, 2024-04-26T23:41:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 14103 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 2000, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 4176, 2024-04-26T23:41:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 1969, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 2231, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-26T23:41:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 24738, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 24738, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-04-26T23:41:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 34, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 34, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 4, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 94853956, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2680, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 145.106325508, 2024-04-26T23:41:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.054144151 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 2014, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 2014, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 0, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 13, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 13, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 0, 2024-04-26T23:41:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 19913, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 19914, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 1, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 33, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 7474, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 7441, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 107876, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231788 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 28585, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-04-26T23:41:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 25580, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 8, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2000, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.977572468, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001988786 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 30, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009375929, 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000312530 2024-04-26T23:41:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6000, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 17.657319442, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002942886 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-26T23:41:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7305, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 8.751032896, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001197951 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000110948, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000110948 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.006087646, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006087646 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.027099108, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.027099108 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000151502, 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000151502 2024-04-26T23:41:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.216850655, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.027106331 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2231, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 544.647727673, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.244127175 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008542158, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008542158 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000725756, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000725756 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-26T23:41:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8000, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 40.327621183, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.005040952 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:41:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 1, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 0, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-04-26T23:41:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 14092, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 14092, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 117, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:41:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-26T23:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-26T23:41:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 17342107, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 6362, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 616, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 7, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-26T23:41:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 5520, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 78, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 21687729, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 889516, 2024-04-26T23:41:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 5026, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 91146924, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 37, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 5026, 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-26T23:41:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5026, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 43.424521202, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008639976 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5026, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 5186 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 5026, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 81, 2024-04-26T23:41:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 4945, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 11, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 81, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 2761, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 50, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 1969, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 33, 2024-04-26T23:41:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 22, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:41:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-26T23:41:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 41, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 3, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 8, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 8, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-26T23:41:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 212, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 47 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-26T23:41:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 32, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 32, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 1969, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 1969, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 31702, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 8499947, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 31702, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 31702, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 8499947, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 5026, 2024-04-26T23:41:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 91361545, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 5026, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 91361545, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-26T23:41:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 5026, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 5026, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 5026, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 5026, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 1969, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 214621, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 1969, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 81, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 214621, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x556b3e7be500": { 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 84, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 24738, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 90659652, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 24738, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 2680, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 90659568, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:04.472 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 msgr2=0x7ff71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 0x7ff71806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009a90 tx=0x7ff72c008040 comp rx=0 tx=0).stop 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 msgr2=0x7ff7380aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f80 tx=0x7ff730001360 comp rx=0 tx=0).stop 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 shutdown_connections 2024-04-26T23:41:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7380c7dc0 0x7ff7380aebf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff73813b750 0x7ff7380ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 --2- 172.21.15.46:0/994632799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73813b370 0x7ff7380c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 >> 172.21.15.46:0/994632799 conn(0x7ff73806c350 msgr2=0x7ff7380d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 shutdown_connections 2024-04-26T23:41:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.413+0000 7ff73de00640 1 -- 172.21.15.46:0/994632799 wait complete. 2024-04-26T23:41:04.475 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-26T23:41:04.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:04.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-26T23:41:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.693+0000 7f666ca00640 1 Processor -- start 2024-04-26T23:41:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.693+0000 7f666ca00640 1 -- start start 2024-04-26T23:41:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f666ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f666815f690 0x7f666806cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f666ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 0x7f66680d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f666ca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66680d2180 con 0x7f666815f2b0 2024-04-26T23:41:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f666ca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f66680cfcb0 con 0x7f666815f690 2024-04-26T23:41:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f666ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66680d2740 con 0x7f66680b4690 2024-04-26T23:41:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f6667400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f666815f2b0 0x7f66680d59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:60392/0 (socket says 172.21.15.46:60392) 2024-04-26T23:41:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.695+0000 7f6667400640 1 -- 172.21.15.46:0/729839253 learned_addr learned my addr 172.21.15.46:0/729839253 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6667e00640 1 --2- 172.21.15.46:0/729839253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 0x7f66680d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6666a00640 1 --2- 172.21.15.46:0/729839253 >> v2:172.21.15.142:3300/0 conn(0x7f666815f690 0x7f666806cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6667e00640 1 -- 172.21.15.46:0/729839253 >> v2:172.21.15.142:3300/0 conn(0x7f666815f690 msgr2=0x7f666806cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6667e00640 1 --2- 172.21.15.46:0/729839253 >> v2:172.21.15.142:3300/0 conn(0x7f666815f690 0x7f666806cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6667e00640 1 -- 172.21.15.46:0/729839253 >> v1:172.21.15.142:6789/0 conn(0x7f666815f2b0 legacy=0x7f66680d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.696+0000 7f6667e00640 1 -- 172.21.15.46:0/729839253 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680d2460 con 0x7f66680b4690 2024-04-26T23:41:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.698+0000 7f6667e00640 1 --2- 172.21.15.46:0/729839253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 0x7f66680d3680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f666002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc682efe6640f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.699+0000 7f6666000640 1 -- 172.21.15.46:0/729839253 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660031600 con 0x7f66680b4690 2024-04-26T23:41:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.699+0000 7f6666000640 1 -- 172.21.15.46:0/729839253 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600317a0 con 0x7f66680b4690 2024-04-26T23:41:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.699+0000 7f6666000640 1 -- 172.21.15.46:0/729839253 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660031b70 con 0x7f66680b4690 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.699+0000 7f666ca00640 1 -- 172.21.15.46:0/729839253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 msgr2=0x7f66680d3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.699+0000 7f666ca00640 1 --2- 172.21.15.46:0/729839253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 0x7f66680d3680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f666002a5d0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 -- 172.21.15.46:0/729839253 shutdown_connections 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 --2- 172.21.15.46:0/729839253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66680b4690 0x7f66680d3680 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 --2- 172.21.15.46:0/729839253 >> v2:172.21.15.142:3300/0 conn(0x7f666815f690 0x7f666806cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 -- 172.21.15.46:0/729839253 >> 172.21.15.46:0/729839253 conn(0x7f666809e3c0 msgr2=0x7f66680d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 -- 172.21.15.46:0/729839253 shutdown_connections 2024-04-26T23:41:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.700+0000 7f666ca00640 1 -- 172.21.15.46:0/729839253 wait complete. 2024-04-26T23:41:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.702+0000 7f666ca00640 1 Processor -- start 2024-04-26T23:41:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.702+0000 7f666ca00640 1 -- start start 2024-04-26T23:41:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f666815f2b0 0x7f6668157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f666815fa80 0x7f6668161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f666806d430 con 0x7f66680b4690 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66680d0250 con 0x7f666815f2b0 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f666ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f666806d5b0 con 0x7f666815fa80 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f6667400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f6667400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58638/0 (socket says 172.21.15.46:58638) 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f6667400640 1 -- 172.21.15.46:0/4105849260 learned_addr learned my addr 172.21.15.46:0/4105849260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.703+0000 7f6666a00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f666815f2b0 0x7f6668157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667e00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f666815fa80 0x7f6668161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 -- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f666815fa80 msgr2=0x7f6668161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f666815fa80 0x7f6668161290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 -- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f666815f2b0 msgr2=0x7f6668157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f666815f2b0 0x7f6668157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 -- 172.21.15.46:0/4105849260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66681601f0 con 0x7f66680b4690 2024-04-26T23:41:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f6667400640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f665800c430 tx=0x7f665800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f665f400640 1 -- 172.21.15.46:0/4105849260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658017070 con 0x7f66680b4690 2024-04-26T23:41:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f665f400640 1 -- 172.21.15.46:0/4105849260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6658005ce0 con 0x7f66680b4690 2024-04-26T23:41:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f665f400640 1 -- 172.21.15.46:0/4105849260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580072f0 con 0x7f66680b4690 2024-04-26T23:41:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.704+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66681604d0 con 0x7f66680b4690 2024-04-26T23:41:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.705+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6668160a90 con 0x7f66680b4690 2024-04-26T23:41:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.707+0000 7f665f400640 1 -- 172.21.15.46:0/4105849260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f665801a070 con 0x7f66680b4690 2024-04-26T23:41:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.708+0000 7f665f400640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 0x7f664806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.709+0000 7f665f400640 1 -- 172.21.15.46:0/4105849260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f665808b1c0 con 0x7f66680b4690 2024-04-26T23:41:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.709+0000 7f6670e34740 1 Processor -- start 2024-04-26T23:41:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.709+0000 7f6666a00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 0x7f664806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.709+0000 7f6670e34740 1 -- start start 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d152ee10 0x55a4d150b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 0x55a4d150bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4d146f990 con 0x55a4d15062b0 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4d140bb80 con 0x55a4d150d9f0 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a4d146b950 con 0x55a4d152ee10 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.710+0000 7f6666a00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 0x7f664806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6654006fd0 tx=0x7f6654008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58654/0 (socket says 172.21.15.46:58654) 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 -- 172.21.15.46:0/1014669506 learned_addr learned my addr 172.21.15.46:0/1014669506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6667400640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d152ee10 0x55a4d150b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6667e00640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 0x55a4d150bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 -- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d152ee10 msgr2=0x55a4d150b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d152ee10 0x55a4d150b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 -- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 msgr2=0x55a4d150bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 0x55a4d150bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 -- 172.21.15.46:0/1014669506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d124d940 con 0x55a4d15062b0 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6667e00640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 0x55a4d150bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.711+0000 7f6666a00640 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f665400f930 tx=0x7f6654012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2932a3ecd2ae584 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6653400640 1 -- 172.21.15.46:0/1014669506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665401b050 con 0x55a4d15062b0 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6653400640 1 -- 172.21.15.46:0/1014669506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654025070 con 0x55a4d15062b0 2024-04-26T23:41:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6653400640 1 -- 172.21.15.46:0/1014669506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654014de0 con 0x55a4d15062b0 2024-04-26T23:41:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 -- 172.21.15.46:0/1014669506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 msgr2=0x55a4d150b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f665400f930 tx=0x7f6654012000 comp rx=0 tx=0).stop 2024-04-26T23:41:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 -- 172.21.15.46:0/1014669506 shutdown_connections 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d150d9f0 0x55a4d150bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d15062b0 0x55a4d150b980 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 --2- 172.21.15.46:0/1014669506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d152ee10 0x55a4d150b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.712+0000 7f6670e34740 1 -- 172.21.15.46:0/1014669506 >> 172.21.15.46:0/1014669506 conn(0x55a4d1508730 msgr2=0x55a4d1551290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.713+0000 7f6670e34740 1 -- 172.21.15.46:0/1014669506 shutdown_connections 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.713+0000 7f6670e34740 1 -- 172.21.15.46:0/1014669506 wait complete. 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.713+0000 7f6670e34740 1 -- start start 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.714+0000 7f6670e34740 1 Processor -- start 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1532000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d15306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4d1509f00 con 0x55a4d1524460 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4d13f6a40 con 0x55a4d152ee10 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a4d152f1f0 con 0x55a4d15302d0 2024-04-26T23:41:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6670e34740 10 client.0 _ll_register_callbacks cb 0x55a4d143a200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58668/0 (socket says 172.21.15.46:58668) 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 -- 172.21.15.46:0/3290924183 learned_addr learned my addr 172.21.15.46:0/3290924183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667400640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1532000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6666a00640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d15306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 -- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 msgr2=0x55a4d15306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d15306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 -- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 msgr2=0x55a4d1532000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1532000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6667e00640 1 -- 172.21.15.46:0/3290924183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d12ae0d0 con 0x55a4d1524460 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.715+0000 7f6666a00640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d15306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6667e00640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f666002abc0 tx=0x7f6660004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6494c593341d04e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6647e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660009450 con 0x55a4d1524460 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f666003b850 con 0x55a4d1524460 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6654025070 con 0x55a4d1524460 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f666003ba50 con 0x55a4d1524460 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6670e34740 10 client.5532 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.716+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55a4d1141230 con 0x55a4d1524460 2024-04-26T23:41:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654025070 con 0x55a4d1524460 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f6660052020 con 0x55a4d1524460 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 10 client.5532 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 10 client.5532 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 20 client.5532 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1565370 0x55a4d1565750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 4 client.5532 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4d147d5a0 con 0x55a4d1565370 2024-04-26T23:41:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.717+0000 7f6667400640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1565370 0x55a4d1565750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.718+0000 7f6667400640 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1565370 0x55a4d1565750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7b8ad669c4b301 server_cookie=ab2d0b3a07c6a98a in_seq=0 out_seq=0 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.718+0000 7f6647e00640 10 client.5532 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.718+0000 7f6647e00640 1 -- 172.21.15.46:0/3290924183 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4d147d5a0 con 0x55a4d1565370 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.718+0000 7f6647e00640 10 client.5532 handle_command_reply: tid=0 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 client.5532 shutdown 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 msgr2=0x55a4d15248c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f666002abc0 tx=0x7f6660004ab0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 shutdown_connections 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1565370 0x55a4d1565750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15248c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6647e00640 0 client.5532 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d15306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 --2- 172.21.15.46:0/3290924183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1532000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 >> 172.21.15.46:0/3290924183 conn(0x55a4d150d9f0 msgr2=0x55a4d1554760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 shutdown_connections 2024-04-26T23:41:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 1 -- 172.21.15.46:0/3290924183 wait complete. 2024-04-26T23:41:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.719+0000 7f6670e34740 20 client.5532 trim_cache size 0 max 16384 2024-04-26T23:41:04.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.739+0000 7f6670e34740 1 Processor -- start 2024-04-26T23:41:04.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 -- start start 2024-04-26T23:41:04.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d1524460 0x55a4d1575880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 0x55a4d1539dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6668201c70 con 0x55a4d152ee10 2024-04-26T23:41:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b5920 con 0x55a4d15302d0 2024-04-26T23:41:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.740+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66680b4c80 con 0x55a4d1524460 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58676/0 (socket says 172.21.15.46:58676) 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 -- 172.21.15.46:0/3324041887 learned_addr learned my addr 172.21.15.46:0/3324041887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6667e00640 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 0x55a4d1539dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 -- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d1524460 msgr2=0x55a4d1575880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d1524460 0x55a4d1575880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 -- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 msgr2=0x55a4d1539dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 0x55a4d1539dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6666a00640 1 -- 172.21.15.46:0/3324041887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d1530a20 con 0x55a4d152ee10 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.741+0000 7f6667e00640 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 0x55a4d1539dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:04.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6666a00640 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6654022040 tx=0x7f6654014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6c4b102db79170 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6652a00640 1 -- 172.21.15.46:0/3324041887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654014be0 con 0x55a4d152ee10 2024-04-26T23:41:04.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6652a00640 1 -- 172.21.15.46:0/3324041887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654025040 con 0x55a4d152ee10 2024-04-26T23:41:04.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6652a00640 1 -- 172.21.15.46:0/3324041887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654034440 con 0x55a4d152ee10 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6670e34740 1 -- 172.21.15.46:0/3324041887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 msgr2=0x55a4d1539880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.742+0000 7f6670e34740 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6654022040 tx=0x7f6654014000 comp rx=0 tx=0).stop 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 -- 172.21.15.46:0/3324041887 shutdown_connections 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d15302d0 0x55a4d1539dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d152ee10 0x55a4d1539880 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 --2- 172.21.15.46:0/3324041887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d1524460 0x55a4d1575880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 -- 172.21.15.46:0/3324041887 >> 172.21.15.46:0/3324041887 conn(0x55a4d150d9f0 msgr2=0x55a4d1547400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 -- 172.21.15.46:0/3324041887 shutdown_connections 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 -- 172.21.15.46:0/3324041887 wait complete. 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.743+0000 7f6670e34740 1 -- start start 2024-04-26T23:41:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.744+0000 7f6670e34740 1 Processor -- start 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1524840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d1524d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4d1555270 con 0x55a4d1524460 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b52d0 con 0x55a4d152ee10 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a4d1524090 con 0x55a4d15302d0 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6670e34740 10 client.0 _ll_register_callbacks cb 0x7f6660009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58680/0 (socket says 172.21.15.46:58680) 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 -- 172.21.15.46:0/3757248696 learned_addr learned my addr 172.21.15.46:0/3757248696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6666a00640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1524840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667e00640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d1524d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 -- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 msgr2=0x55a4d1524d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d1524d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 -- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 msgr2=0x55a4d1524840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1524840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6667400640 1 -- 172.21.15.46:0/3757248696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d1527120 con 0x55a4d1524460 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.745+0000 7f6666a00640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1524840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6667400640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6658061350 tx=0x7f6658014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cee9c5036f72292 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6647e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658061880 con 0x55a4d1524460 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f66580895b0 con 0x55a4d1524460 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6654025040 con 0x55a4d1524460 2024-04-26T23:41:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6658056fa0 con 0x55a4d1524460 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6670e34740 10 client.5538 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55a4d15628c0 con 0x55a4d1524460 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.746+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654025040 con 0x55a4d1524460 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f6658078170 con 0x55a4d1524460 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 10 client.5538 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 10 client.5538 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 20 client.5538 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1563d50 0x55a4d152a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 4 client.5538 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) -- 0x55a4d15336f0 con 0x55a4d1563d50 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.747+0000 7f6666a00640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1563d50 0x55a4d152a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:04.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.748+0000 7f6666a00640 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1563d50 0x55a4d152a590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617025cf8dc5c8ac server_cookie=1e816b9f0b7c1b8d in_seq=0 out_seq=0 2024-04-26T23:41:04.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.748+0000 7f6647e00640 10 client.5538 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6647e00640 1 -- 172.21.15.46:0/3757248696 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4d15336f0 con 0x55a4d1563d50 2024-04-26T23:41:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6647e00640 10 client.5538 handle_command_reply: tid=0 2024-04-26T23:41:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 client.5538 shutdown 2024-04-26T23:41:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 msgr2=0x55a4d15cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6658061350 tx=0x7f6658014040 comp rx=0 tx=0).stop 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 shutdown_connections 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55a4d1563d50 0x55a4d152a590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4d15302d0 0x55a4d1524d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4d152ee10 0x55a4d1524840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 --2- 172.21.15.46:0/3757248696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4d1524460 0x55a4d15cc690 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 >> 172.21.15.46:0/3757248696 conn(0x55a4d150d9f0 msgr2=0x55a4d152c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.753+0000 7f6647e00640 0 client.5538 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 shutdown_connections 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f6670e34740 1 -- 172.21.15.46:0/3757248696 wait complete. 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f6670e34740 20 client.5538 trim_cache size 0 max 16384 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a0e937dc-7082-4da5-b6ed-87aaa7473e00", 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:41:04.759 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 msgr2=0x7f664806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 0x7f664806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6654006fd0 tx=0x7f6654008040 comp rx=0 tx=0).stop 2024-04-26T23:41:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 msgr2=0x7f6668157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.754+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f665800c430 tx=0x7f665800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 shutdown_connections 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f664806c8f0 0x7f664806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f666815fa80 0x7f6668161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f666815f2b0 0x7f6668157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 --2- 172.21.15.46:0/4105849260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66680b4690 0x7f6668157230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 >> 172.21.15.46:0/4105849260 conn(0x7f666809e3c0 msgr2=0x7f66680d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 shutdown_connections 2024-04-26T23:41:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:04.755+0000 7f666ca00640 1 -- 172.21.15.46:0/4105849260 wait complete. 2024-04-26T23:41:04.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:04.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-26T23:41:05.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.008+0000 7f5741800640 1 Processor -- start 2024-04-26T23:41:05.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.008+0000 7f5741800640 1 -- start start 2024-04-26T23:41:05.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f573c162aa0 0x7f573c0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 0x7f573c0d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 0x7f573c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f573c0b44a0 con 0x7f573c162aa0 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f573c0b5020 con 0x7f573c0d5b90 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f5741800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f573c0b46f0 con 0x7f573c0b54b0 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f573aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f573c162aa0 0x7f573c0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f573aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f573c162aa0 0x7f573c0d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58692/0 (socket says 172.21.15.46:58692) 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.010+0000 7f573aa00640 1 -- 172.21.15.46:0/2142891950 learned_addr learned my addr 172.21.15.46:0/2142891950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573a000640 1 --2- 172.21.15.46:0/2142891950 >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 0x7f573c0d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573b400640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 0x7f573c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573aa00640 1 -- 172.21.15.46:0/2142891950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 msgr2=0x7f573c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573aa00640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 0x7f573c0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573aa00640 1 -- 172.21.15.46:0/2142891950 >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 msgr2=0x7f573c0d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573aa00640 1 --2- 172.21.15.46:0/2142891950 >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 0x7f573c0d3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573aa00640 1 -- 172.21.15.46:0/2142891950 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c0b49d0 con 0x7f573c162aa0 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573a000640 1 --2- 172.21.15.46:0/2142891950 >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 0x7f573c0d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.011+0000 7f573b400640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 0x7f573c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.012+0000 7f573aa00640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c162aa0 0x7f573c0d3650 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f573c0c89a0 tx=0x7f572c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75107ee0c7b3905d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.012+0000 7f5739600640 1 -- 172.21.15.46:0/2142891950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c037070 con 0x7f573c162aa0 2024-04-26T23:41:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.012+0000 7f5739600640 1 -- 172.21.15.46:0/2142891950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c02ae00 con 0x7f573c162aa0 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5739600640 1 -- 172.21.15.46:0/2142891950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c032660 con 0x7f573c162aa0 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 -- 172.21.15.46:0/2142891950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c162aa0 msgr2=0x7f573c0d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c162aa0 0x7f573c0d3650 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f573c0c89a0 tx=0x7f572c02a0b0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 -- 172.21.15.46:0/2142891950 shutdown_connections 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0b54b0 0x7f573c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 --2- 172.21.15.46:0/2142891950 >> v2:172.21.15.142:3300/0 conn(0x7f573c0d5b90 0x7f573c0d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 --2- 172.21.15.46:0/2142891950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c162aa0 0x7f573c0d3650 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 -- 172.21.15.46:0/2142891950 >> 172.21.15.46:0/2142891950 conn(0x7f573c0d5070 msgr2=0x7f573c0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.013+0000 7f5741800640 1 -- 172.21.15.46:0/2142891950 shutdown_connections 2024-04-26T23:41:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.014+0000 7f5741800640 1 -- 172.21.15.46:0/2142891950 wait complete. 2024-04-26T23:41:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.016+0000 7f5741800640 1 Processor -- start 2024-04-26T23:41:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.016+0000 7f5741800640 1 -- start start 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 0x7f573c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f573c162aa0 0x7f573c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f573c0d30f0 con 0x7f573c0b54b0 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f573c162e80 con 0x7f573c162aa0 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f5741800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f573c0b5e30 con 0x7f573c0d5b90 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58702/0 (socket says 172.21.15.46:58702) 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.018+0000 7f573aa00640 1 -- 172.21.15.46:0/90359821 learned_addr learned my addr 172.21.15.46:0/90359821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573a000640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 0x7f573c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 -- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 msgr2=0x7f573c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 0x7f573c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 -- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f573c162aa0 msgr2=0x7f573c0cce60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:41:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f573c162aa0 0x7f573c0cce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 -- 172.21.15.46:0/90359821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c0cd770 con 0x7f573c0b54b0 2024-04-26T23:41:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573a000640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 0x7f573c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f573aa00640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f572c02a0f0 tx=0x7f572c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.019+0000 7f5732a00640 1 -- 172.21.15.46:0/90359821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c041070 con 0x7f573c0b54b0 2024-04-26T23:41:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.020+0000 7f5732a00640 1 -- 172.21.15.46:0/90359821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c03c670 con 0x7f573c0b54b0 2024-04-26T23:41:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.020+0000 7f5732a00640 1 -- 172.21.15.46:0/90359821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c037040 con 0x7f573c0b54b0 2024-04-26T23:41:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.020+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c0b7d50 con 0x7f573c0b54b0 2024-04-26T23:41:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.020+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c0cda50 con 0x7f573c0b54b0 2024-04-26T23:41:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.022+0000 7f5732a00640 1 -- 172.21.15.46:0/90359821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f572c03a070 con 0x7f573c0b54b0 2024-04-26T23:41:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.023+0000 7f5732a00640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 0x7f571c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.023+0000 7f5732a00640 1 -- 172.21.15.46:0/90359821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f572c02ae00 con 0x7f573c0b54b0 2024-04-26T23:41:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.024+0000 7f573a000640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 0x7f571c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.025+0000 7f573a000640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 0x7f571c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5728006fd0 tx=0x7f5728008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.026+0000 7f57447bb740 1 Processor -- start 2024-04-26T23:41:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.026+0000 7f57447bb740 1 -- start start 2024-04-26T23:41:05.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d82920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5594c6c68030 con 0x5594c6d58220 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594c6c837f0 con 0x5594c6d82540 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.027+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5594c6c3e7e0 con 0x5594c6d9fde0 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58718/0 (socket says 172.21.15.46:58718) 2024-04-26T23:41:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 -- 172.21.15.46:0/2984759901 learned_addr learned my addr 172.21.15.46:0/2984759901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573aa00640 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d82920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573a000640 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 -- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 msgr2=0x5594c6da0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 -- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 msgr2=0x5594c6d82920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d82920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 -- 172.21.15.46:0/2984759901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c6c4a050 con 0x5594c6d58220 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.028+0000 7f573b400640 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5734007e20 tx=0x7f573400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c1071abe714f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.029+0000 7f5727400640 1 -- 172.21.15.46:0/2984759901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400d840 con 0x5594c6d58220 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.029+0000 7f5727400640 1 -- 172.21.15.46:0/2984759901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573400d9e0 con 0x5594c6d58220 2024-04-26T23:41:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.029+0000 7f5727400640 1 -- 172.21.15.46:0/2984759901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400ddd0 con 0x5594c6d58220 2024-04-26T23:41:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 -- 172.21.15.46:0/2984759901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 msgr2=0x5594c6d9e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5734007e20 tx=0x7f573400c790 comp rx=0 tx=0).stop 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 -- 172.21.15.46:0/2984759901 shutdown_connections 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d58220 0x5594c6d9e4a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 --2- 172.21.15.46:0/2984759901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d82920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 -- 172.21.15.46:0/2984759901 >> 172.21.15.46:0/2984759901 conn(0x5594c6d54b50 msgr2=0x5594c6d9d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 -- 172.21.15.46:0/2984759901 shutdown_connections 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.030+0000 7f57447bb740 1 -- 172.21.15.46:0/2984759901 wait complete. 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.031+0000 7f57447bb740 1 -- start start 2024-04-26T23:41:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.031+0000 7f57447bb740 1 Processor -- start 2024-04-26T23:41:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d84870 0x5594c6da0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5594c6cc0200 con 0x5594c6d82540 2024-04-26T23:41:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594c6cbaee0 con 0x5594c6d84870 2024-04-26T23:41:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5594c6cc7e70 con 0x5594c6d9fde0 2024-04-26T23:41:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.032+0000 7f57447bb740 10 client.0 _ll_register_callbacks cb 0x5594c6cb1270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58732/0 (socket says 172.21.15.46:58732) 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 -- 172.21.15.46:0/292677427 learned_addr learned my addr 172.21.15.46:0/292677427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573a000640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573b400640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d84870 0x5594c6da0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 -- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 msgr2=0x5594c6da08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 -- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d84870 msgr2=0x5594c6da0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d84870 0x5594c6da0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.033+0000 7f573aa00640 1 -- 172.21.15.46:0/292677427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c6a977d0 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f573aa00640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f572c079910 tx=0x7f572c07a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed146604781d901f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f571be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c04a040 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f572c0820d0 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573400d9e0 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f572c082560 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f57447bb740 10 client.5550 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5594c69cd3f0 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.034+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573400d9e0 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f572c04d060 con 0x5594c6d82540 2024-04-26T23:41:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 10 client.5550 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 10 client.5550 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 20 client.5550 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d85720 0x5594c6d6d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 4 client.5550 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c6cc97e0 con 0x5594c6d85720 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.035+0000 7f573a000640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d85720 0x5594c6d6d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.036+0000 7f571be00640 10 client.5550 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.036+0000 7f573a000640 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d85720 0x5594c6d6d510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3626a1276c2ecd server_cookie=1670919117079f95 in_seq=0 out_seq=0 2024-04-26T23:41:05.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.045+0000 7f571be00640 1 -- 172.21.15.46:0/292677427 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594c6cc97e0 con 0x5594c6d85720 2024-04-26T23:41:05.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.045+0000 7f571be00640 10 client.5550 handle_command_reply: tid=0 2024-04-26T23:41:05.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 client.5550 shutdown 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 msgr2=0x5594c6d851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f572c079910 tx=0x7f572c07a140 comp rx=0 tx=0).stop 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 shutdown_connections 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d85720 0x5594c6d6d510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d84870 0x5594c6da0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6da08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 --2- 172.21.15.46:0/292677427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6d851e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.046+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 >> 172.21.15.46:0/292677427 conn(0x5594c6d58220 msgr2=0x5594c6d57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.047+0000 7f571be00640 0 client.5550 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.047+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 shutdown_connections 2024-04-26T23:41:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.047+0000 7f57447bb740 1 -- 172.21.15.46:0/292677427 wait complete. 2024-04-26T23:41:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.047+0000 7f57447bb740 20 client.5550 trim_cache size 0 max 16384 2024-04-26T23:41:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.076+0000 7f57447bb740 1 Processor -- start 2024-04-26T23:41:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.076+0000 7f57447bb740 1 -- start start 2024-04-26T23:41:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.077+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.077+0000 7f57447bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6d9e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f57447bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f573c201e70 con 0x5594c6d84870 2024-04-26T23:41:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f573c200eb0 con 0x5594c6d82540 2024-04-26T23:41:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f573c1f8ec0 con 0x5594c6d9fde0 2024-04-26T23:41:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f573a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f573a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58746/0 (socket says 172.21.15.46:58746) 2024-04-26T23:41:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.078+0000 7f573a000640 1 -- 172.21.15.46:0/4281902844 learned_addr learned my addr 172.21.15.46:0/4281902844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573b400640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6d9e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573aa00640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573a000640 1 -- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 msgr2=0x5594c6d9e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573a000640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6d9e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573a000640 1 -- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 msgr2=0x5594c6d85220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573a000640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d85220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573a000640 1 -- 172.21.15.46:0/4281902844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c698d0f0 con 0x5594c6d84870 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.079+0000 7f573b400640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6d9e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.080+0000 7f573a000640 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f572800faa0 tx=0x7f5728015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9ed0110865233 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.080+0000 7f5726a00640 1 -- 172.21.15.46:0/4281902844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572801b040 con 0x5594c6d84870 2024-04-26T23:41:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.081+0000 7f5726a00640 1 -- 172.21.15.46:0/4281902844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572801c040 con 0x5594c6d84870 2024-04-26T23:41:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.081+0000 7f5726a00640 1 -- 172.21.15.46:0/4281902844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728004030 con 0x5594c6d84870 2024-04-26T23:41:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.081+0000 7f57447bb740 1 -- 172.21.15.46:0/4281902844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 msgr2=0x5594c6d9e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.081+0000 7f57447bb740 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f572800faa0 tx=0x7f5728015000 comp rx=0 tx=0).stop 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 -- 172.21.15.46:0/4281902844 shutdown_connections 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d9fde0 0x5594c6d9e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d84870 0x5594c6d9e360 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 --2- 172.21.15.46:0/4281902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d82540 0x5594c6d85220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 -- 172.21.15.46:0/4281902844 >> 172.21.15.46:0/4281902844 conn(0x5594c6d58220 msgr2=0x5594c6d4e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 -- 172.21.15.46:0/4281902844 shutdown_connections 2024-04-26T23:41:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.082+0000 7f57447bb740 1 -- 172.21.15.46:0/4281902844 wait complete. 2024-04-26T23:41:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.083+0000 7f57447bb740 1 -- start start 2024-04-26T23:41:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.083+0000 7f57447bb740 1 Processor -- start 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d84870 0x5594c6d7f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d9fde0 0x5594c6e16de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5594c6cc8fd0 con 0x5594c6d82540 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f573c200900 con 0x5594c6d9fde0 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5594c6d6dc00 con 0x5594c6d84870 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f57447bb740 10 client.0 _ll_register_callbacks cb 0x7f572c079590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58762/0 (socket says 172.21.15.46:58762) 2024-04-26T23:41:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f573aa00640 1 -- 172.21.15.46:0/2058929896 learned_addr learned my addr 172.21.15.46:0/2058929896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f573a000640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d84870 0x5594c6d7f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.085+0000 7f573b400640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d9fde0 0x5594c6e16de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 -- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d84870 msgr2=0x5594c6d7f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d84870 0x5594c6d7f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 -- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d9fde0 msgr2=0x5594c6e16de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d9fde0 0x5594c6e16de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 -- 172.21.15.46:0/2058929896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c6c83a30 con 0x5594c6d82540 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f573aa00640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f572c082720 tx=0x7f572c0aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f9644a226c37fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.086+0000 7f571be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c04a040 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f572c098c80 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572801c040 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f572c075c20 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f57447bb740 10 client.5556 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5594c6d81a40 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.087+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572801c040 con 0x5594c6d82540 2024-04-26T23:41:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f572c09b060 con 0x5594c6d82540 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 10 client.5556 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 10 client.5556 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 20 client.5556 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d7f920 0x5594c6e17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 4 client.5556 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594c6cc3650 con 0x5594c6d7f920 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.088+0000 7f573a000640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d7f920 0x5594c6e17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.089+0000 7f573a000640 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d7f920 0x5594c6e17320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd749a226e980b97 server_cookie=d73046b96828cb57 in_seq=0 out_seq=0 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.089+0000 7f571be00640 10 client.5556 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f571be00640 1 -- 172.21.15.46:0/2058929896 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+274 (crc 0 0 0) 0x5594c6cc3650 con 0x5594c6d7f920 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f571be00640 10 client.5556 handle_command_reply: tid=0 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 client.5556 shutdown 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 msgr2=0x5594c6e16010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f572c082720 tx=0x7f572c0aa050 comp rx=0 tx=0).stop 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 shutdown_connections 2024-04-26T23:41:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x5594c6d7f920 0x5594c6e17320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c6d9fde0 0x5594c6e16de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594c6d84870 0x5594c6d7f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 --2- 172.21.15.46:0/2058929896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594c6d82540 0x5594c6e16010 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 >> 172.21.15.46:0/2058929896 conn(0x5594c6d58220 msgr2=0x5594c6d4de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "scrub active (913 inodes in the stack)", 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stdout: "a0e937dc-7082-4da5-b6ed-87aaa7473e00": { 2024-04-26T23:41:05.095 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a0e937dc-7082-4da5-b6ed-87aaa7473e00", 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,scrub_mdsdir" 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f571be00640 0 client.5556 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 shutdown_connections 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.090+0000 7f57447bb740 1 -- 172.21.15.46:0/2058929896 wait complete. 2024-04-26T23:41:05.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.091+0000 7f57447bb740 20 client.5556 trim_cache size 0 max 16384 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 msgr2=0x7f571c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 0x7f571c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5728006fd0 tx=0x7f5728008040 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 msgr2=0x7f573c0b8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f572c02a0f0 tx=0x7f572c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 shutdown_connections 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f571c06c8f0 0x7f571c06edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f573c162aa0 0x7f573c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f573c0d5b90 0x7f573c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 --2- 172.21.15.46:0/90359821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c0b54b0 0x7f573c0b8830 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.092+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 >> 172.21.15.46:0/90359821 conn(0x7f573c0d5070 msgr2=0x7f573c160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:05.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.093+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 shutdown_connections 2024-04-26T23:41:05.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:05.093+0000 7f5741800640 1 -- 172.21.15.46:0/90359821 wait complete. 2024-04-26T23:41:05.119 INFO:tasks.cephfs.filesystem:scrub status for tag:a0e937dc-7082-4da5-b6ed-87aaa7473e00 - {'path': '/', 'tag': 'a0e937dc-7082-4da5-b6ed-87aaa7473e00', 'options': 'recursive,scrub_mdsdir'} 2024-04-26T23:41:08.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:08.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:35.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:35.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-26T23:41:35.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.345+0000 7f6e3be00640 1 Processor -- start 2024-04-26T23:41:35.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.345+0000 7f6e3be00640 1 -- start start 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e3c145c10 0x7f6e3c073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6e3c0d0f90 0x7f6e3c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e3c06ec30 con 0x7f6e3c0cfed0 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6e3c145ff0 con 0x7f6e3c145c10 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6e3c0cf3f0 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6e3c0cfed0 0x7f6e3c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35722/0 (socket says 172.21.15.46:35722) 2024-04-26T23:41:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3a000640 1 -- 172.21.15.46:0/505115125 learned_addr learned my addr 172.21.15.46:0/505115125 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/505115125 >> v2:172.21.15.46:3300/0 conn(0x7f6e3c145c10 0x7f6e3c073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.347+0000 7f6e3b400640 1 --2- 172.21.15.46:0/505115125 >> v2:172.21.15.142:3300/0 conn(0x7f6e3c0d0f90 0x7f6e3c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e39600640 1 -- 172.21.15.46:0/505115125 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599513740 0 0) 0x7f6e3c06ec30 con 0x7f6e3c0cfed0 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e39600640 1 -- 172.21.15.46:0/505115125 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e20003520 con 0x7f6e3c0cfed0 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e3aa00640 1 -- 172.21.15.46:0/505115125 >> v2:172.21.15.142:3300/0 conn(0x7f6e3c0d0f90 msgr2=0x7f6e3c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/505115125 >> v2:172.21.15.142:3300/0 conn(0x7f6e3c0d0f90 0x7f6e3c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e3aa00640 1 -- 172.21.15.46:0/505115125 >> v1:172.21.15.46:6789/0 conn(0x7f6e3c0cfed0 legacy=0x7f6e3c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.348+0000 7f6e3aa00640 1 -- 172.21.15.46:0/505115125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c0cf6d0 con 0x7f6e3c145c10 2024-04-26T23:41:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.349+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/505115125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c145c10 0x7f6e3c073c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c06fe00 tx=0x7f6e2c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07dc440bfca6df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.349+0000 7f6e39600640 1 -- 172.21.15.46:0/505115125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c037070 con 0x7f6e3c145c10 2024-04-26T23:41:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.349+0000 7f6e39600640 1 -- 172.21.15.46:0/505115125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c032440 con 0x7f6e3c145c10 2024-04-26T23:41:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.349+0000 7f6e39600640 1 -- 172.21.15.46:0/505115125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0327f0 con 0x7f6e3c145c10 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 -- 172.21.15.46:0/505115125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c145c10 msgr2=0x7f6e3c073c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 --2- 172.21.15.46:0/505115125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c145c10 0x7f6e3c073c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c06fe00 tx=0x7f6e2c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 -- 172.21.15.46:0/505115125 shutdown_connections 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 --2- 172.21.15.46:0/505115125 >> v2:172.21.15.142:3300/0 conn(0x7f6e3c0d0f90 0x7f6e3c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 --2- 172.21.15.46:0/505115125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c145c10 0x7f6e3c073c00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 -- 172.21.15.46:0/505115125 >> 172.21.15.46:0/505115125 conn(0x7f6e3c06c350 msgr2=0x7f6e3c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 -- 172.21.15.46:0/505115125 shutdown_connections 2024-04-26T23:41:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.350+0000 7f6e3be00640 1 -- 172.21.15.46:0/505115125 wait complete. 2024-04-26T23:41:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.352+0000 7f6e3be00640 1 Processor -- start 2024-04-26T23:41:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.352+0000 7f6e3be00640 1 -- start start 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 0x7f6e3c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e3c145c10 0x7f6e3c13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1468b0 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cf0a0 con 0x7f6e3c0cfed0 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.353+0000 7f6e3be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c0d0360 con 0x7f6e3c145c10 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51772/0 (socket says 172.21.15.46:51772) 2024-04-26T23:41:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 -- 172.21.15.46:0/4232178275 learned_addr learned my addr 172.21.15.46:0/4232178275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3b400640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e3c145c10 0x7f6e3c13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 0x7f6e3c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 -- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e3c145c10 msgr2=0x7f6e3c13fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e3c145c10 0x7f6e3c13fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 -- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 msgr2=0x7f6e3c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 0x7f6e3c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 -- 172.21.15.46:0/4232178275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c0cb280 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 0x7f6e3c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.354+0000 7f6e3a000640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c3c0 tx=0x7f6e2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.355+0000 7f6e32a00640 1 -- 172.21.15.46:0/4232178275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.355+0000 7f6e32a00640 1 -- 172.21.15.46:0/4232178275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e28005ce0 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.355+0000 7f6e32a00640 1 -- 172.21.15.46:0/4232178275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280072f0 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.355+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c0cb560 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.355+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c0cbb20 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.357+0000 7f6e32a00640 1 -- 172.21.15.46:0/4232178275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e2801a070 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.358+0000 7f6e32a00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 0x7f6e1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.358+0000 7f6e32a00640 1 -- 172.21.15.46:0/4232178275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f6e2808b150 con 0x7f6e3c0d0f90 2024-04-26T23:41:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.359+0000 7f6e447b4740 1 Processor -- start 2024-04-26T23:41:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.359+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 0x7f6e1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.359+0000 7f6e447b4740 1 -- start start 2024-04-26T23:41:35.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 0x558f543fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543b33f0 0x558f543f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f542c3030 con 0x558f543dd710 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558f542de7f0 con 0x558f543fafb0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558f542997e0 con 0x558f543b33f0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51776/0 (socket says 172.21.15.46:51776) 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 -- 172.21.15.46:0/2154438913 learned_addr learned my addr 172.21.15.46:0/2154438913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 0x7f6e1c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c034000 tx=0x7f6e2c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3a000640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 0x558f543fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3b400640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543b33f0 0x558f543f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 -- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543b33f0 msgr2=0x558f543f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543b33f0 0x558f543f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.360+0000 7f6e3aa00640 1 -- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 msgr2=0x558f543fb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 0x558f543fb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e3aa00640 1 -- 172.21.15.46:0/2154438913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f542a5050 con 0x558f543dd710 2024-04-26T23:41:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e3a000640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 0x558f543fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c02a790 tx=0x7f6e2c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e951d902566f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e27400640 1 -- 172.21.15.46:0/2154438913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c037040 con 0x558f543dd710 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e27400640 1 -- 172.21.15.46:0/2154438913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c04c070 con 0x558f543dd710 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.361+0000 7f6e27400640 1 -- 172.21.15.46:0/2154438913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c03c660 con 0x558f543dd710 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 -- 172.21.15.46:0/2154438913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 msgr2=0x558f543ddaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c02a790 tx=0x7f6e2c008030 comp rx=0 tx=0).stop 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 -- 172.21.15.46:0/2154438913 shutdown_connections 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543b33f0 0x558f543f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543fafb0 0x558f543fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2154438913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543dd710 0x558f543ddaf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 -- 172.21.15.46:0/2154438913 >> 172.21.15.46:0/2154438913 conn(0x558f543afd20 msgr2=0x558f543f8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 -- 172.21.15.46:0/2154438913 shutdown_connections 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.362+0000 7f6e447b4740 1 -- 172.21.15.46:0/2154438913 wait complete. 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.363+0000 7f6e447b4740 1 -- start start 2024-04-26T23:41:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.363+0000 7f6e447b4740 1 Processor -- start 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543c4130 0x558f543c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f5431b200 con 0x558f543fafb0 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558f54315ee0 con 0x558f543dd710 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558f54322e70 con 0x558f543c4130 2024-04-26T23:41:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e447b4740 10 client.0 _ll_register_callbacks cb 0x558f5430c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51790/0 (socket says 172.21.15.46:51790) 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.364+0000 7f6e3a000640 1 -- 172.21.15.46:0/3115631426 learned_addr learned my addr 172.21.15.46:0/3115631426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 -- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543c4130 msgr2=0x558f543c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543c4130 0x558f543c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 -- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 msgr2=0x558f543d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 -- 172.21.15.46:0/3115631426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f540f27d0 con 0x558f543fafb0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e28089420 tx=0x7f6e28058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5813d3f89b34d046 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e1be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28007490 con 0x558f543fafb0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f6e28007630 con 0x558f543fafb0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.365+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c04c070 con 0x558f543fafb0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e28089120 con 0x558f543fafb0 2024-04-26T23:41:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 10 client.5571 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558f540283f0 con 0x558f543fafb0 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c04c070 con 0x558f543fafb0 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f6e28058070 con 0x558f543fafb0 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 10 client.5571 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 10 client.5571 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 20 client.5571 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543dab20 0x558f543dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 4 client.5571 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:41:35.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.366+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f543247e0 con 0x558f543dab20 2024-04-26T23:41:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.367+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543dab20 0x558f543dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.367+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543dab20 0x558f543dafb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af24aebae606f3e6 server_cookie=fc3a741dd254bf88 in_seq=0 out_seq=0 2024-04-26T23:41:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.367+0000 7f6e1be00640 10 client.5571 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e1be00640 1 -- 172.21.15.46:0/3115631426 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f543247e0 con 0x558f543dab20 2024-04-26T23:41:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e1be00640 10 client.5571 handle_command_reply: tid=0 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 client.5571 shutdown 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 msgr2=0x558f543d6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e28089420 tx=0x7f6e28058ce0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 shutdown_connections 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543dab20 0x558f543dafb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543c4130 0x558f543c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543d6090 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3115631426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 >> 172.21.15.46:0/3115631426 conn(0x558f543b33f0 msgr2=0x558f543ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e1be00640 0 client.5571 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 shutdown_connections 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.368+0000 7f6e447b4740 1 -- 172.21.15.46:0/3115631426 wait complete. 2024-04-26T23:41:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.369+0000 7f6e447b4740 20 client.5571 trim_cache size 0 max 16384 2024-04-26T23:41:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.388+0000 7f6e447b4740 1 Processor -- start 2024-04-26T23:41:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.388+0000 7f6e447b4740 1 -- start start 2024-04-26T23:41:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 0x558f543c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543dd710 0x558f5441c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c2019e0 con 0x558f543fafb0 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c200a80 con 0x558f543c4130 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c1ffe10 con 0x558f543dd710 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51804/0 (socket says 172.21.15.46:51804) 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 -- 172.21.15.46:0/2565090991 learned_addr learned my addr 172.21.15.46:0/2565090991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 -- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543dd710 msgr2=0x558f5441c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543dd710 0x558f5441c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3b400640 1 -- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 msgr2=0x558f543c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.389+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 0x558f543c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e3b400640 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 0x558f543c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e3b400640 1 -- 172.21.15.46:0/2565090991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53fe80f0 con 0x558f543fafb0 2024-04-26T23:41:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 0x558f543c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e3b400640 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ed0 tx=0x7f6e34003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106705f375d3a55d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e26a00640 1 -- 172.21.15.46:0/2565090991 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e340047b0 con 0x558f543fafb0 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e26a00640 1 -- 172.21.15.46:0/2565090991 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3401a070 con 0x558f543fafb0 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.390+0000 7f6e26a00640 1 -- 172.21.15.46:0/2565090991 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34015660 con 0x558f543fafb0 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 -- 172.21.15.46:0/2565090991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 msgr2=0x558f543f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ed0 tx=0x7f6e34003d60 comp rx=0 tx=0).stop 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 -- 172.21.15.46:0/2565090991 shutdown_connections 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543fafb0 0x558f543f7870 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543dd710 0x558f5441c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 --2- 172.21.15.46:0/2565090991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543c4130 0x558f543c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 -- 172.21.15.46:0/2565090991 >> 172.21.15.46:0/2565090991 conn(0x558f543b33f0 msgr2=0x558f543d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 -- 172.21.15.46:0/2565090991 shutdown_connections 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.391+0000 7f6e447b4740 1 -- 172.21.15.46:0/2565090991 wait complete. 2024-04-26T23:41:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.392+0000 7f6e447b4740 1 -- start start 2024-04-26T23:41:35.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.392+0000 7f6e447b4740 1 Processor -- start 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 0x558f543f7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f54323fd0 con 0x558f543c4130 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c200430 con 0x558f543dd710 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558f543fa810 con 0x558f543fafb0 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e447b4740 10 client.0 _ll_register_callbacks cb 0x7f6e28054ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51812/0 (socket says 172.21.15.46:51812) 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.393+0000 7f6e3aa00640 1 -- 172.21.15.46:0/3011788987 learned_addr learned my addr 172.21.15.46:0/3011788987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3b400640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 0x558f543f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 -- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 msgr2=0x558f543f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 0x558f543f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 -- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 msgr2=0x558f543d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 -- 172.21.15.46:0/3011788987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f542dea30 con 0x558f543c4130 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3b400640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 0x558f543f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009890 tx=0x7f6e2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b23379cbdd88c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e1be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c037040 con 0x558f543c4130 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f6e2c03c7a0 con 0x558f543c4130 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.394+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3401a070 con 0x558f543c4130 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c03c9a0 con 0x558f543c4130 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 10 client.5577 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558f543dba10 con 0x558f543c4130 2024-04-26T23:41:35.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3401a070 con 0x558f543c4130 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f6e2c067080 con 0x558f543c4130 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 10 client.5577 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 10 client.5577 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 20 client.5577 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543d2250 0x558f543f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 4 client.5577 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f5431e650 con 0x558f543d2250 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.395+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543d2250 0x558f543f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e3a000640 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543d2250 0x558f543f7bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d9bf987dfe2cb6 server_cookie=a78a935ad778a0b4 in_seq=0 out_seq=0 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e1be00640 10 client.5577 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e1be00640 1 -- 172.21.15.46:0/3011788987 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558f5431e650 con 0x558f543d2250 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e1be00640 10 client.5577 handle_command_reply: tid=0 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e447b4740 1 client.5577 shutdown 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 msgr2=0x558f543b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009890 tx=0x7f6e2c004730 comp rx=0 tx=0).stop 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.396+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 shutdown_connections 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x558f543d2250 0x558f543f7bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558f543fafb0 0x558f543f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f543dd710 0x558f543d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 --2- 172.21.15.46:0/3011788987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f543c4130 0x558f543b1e90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e1be00640 0 client.5577 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 >> 172.21.15.46:0/3011788987 conn(0x558f543b33f0 msgr2=0x558f543b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 shutdown_connections 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 1 -- 172.21.15.46:0/3011788987 wait complete. 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e447b4740 20 client.5577 trim_cache size 0 max 16384 2024-04-26T23:41:35.402 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:41:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:41:35.403 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:35.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 msgr2=0x7f6e1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.397+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 0x7f6e1c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c034000 tx=0x7f6e2c02e000 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 msgr2=0x7f6e3c13f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c3c0 tx=0x7f6e2800c880 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 shutdown_connections 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e1c06c8f0 0x7f6e1c06edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e3c145c10 0x7f6e3c13fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c0d0f90 0x7f6e3c13f990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 --2- 172.21.15.46:0/4232178275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e3c0cfed0 0x7f6e3c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 >> 172.21.15.46:0/4232178275 conn(0x7f6e3c06c350 msgr2=0x7f6e3c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 shutdown_connections 2024-04-26T23:41:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.398+0000 7f6e3be00640 1 -- 172.21.15.46:0/4232178275 wait complete. 2024-04-26T23:41:35.419 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:41:35.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-26T23:41:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.624+0000 7f77a3e00640 1 Processor -- start 2024-04-26T23:41:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.625+0000 7f77a3e00640 1 -- start start 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.626+0000 7f77a3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 0x7f77a40d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.626+0000 7f77a3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77a40d1cb0 0x7f77a40b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.626+0000 7f77a3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a40d20c0 con 0x7f77a40d3830 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.626+0000 7f77a3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f77a40d3400 con 0x7f77a40d1cb0 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.626+0000 7f77a3e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f77a40d2370 con 0x7f77a4162a70 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77a40d1cb0 0x7f77a40b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77a40d1cb0 0x7f77a40b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51816/0 (socket says 172.21.15.46:51816) 2024-04-26T23:41:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 -- 172.21.15.46:0/2894240078 learned_addr learned my addr 172.21.15.46:0/2894240078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a2a00640 1 --2- 172.21.15.46:0/2894240078 >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 0x7f77a40d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 -- 172.21.15.46:0/2894240078 >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 msgr2=0x7f77a40d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 --2- 172.21.15.46:0/2894240078 >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 0x7f77a40d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 -- 172.21.15.46:0/2894240078 >> v1:172.21.15.142:6789/0 conn(0x7f77a40d3830 legacy=0x7f77a40d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a3400640 1 -- 172.21.15.46:0/2894240078 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a40d2650 con 0x7f77a40d1cb0 2024-04-26T23:41:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.627+0000 7f77a2a00640 1 --2- 172.21.15.46:0/2894240078 >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 0x7f77a40d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.628+0000 7f77a3400640 1 --2- 172.21.15.46:0/2894240078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a40d1cb0 0x7f77a40b29e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f77a4074b40 tx=0x7f779c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d5a3fd5aae2eda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.629+0000 7f77a1600640 1 -- 172.21.15.46:0/2894240078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c037070 con 0x7f77a40d1cb0 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.629+0000 7f77a1600640 1 -- 172.21.15.46:0/2894240078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c032440 con 0x7f77a40d1cb0 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.629+0000 7f77a1600640 1 -- 172.21.15.46:0/2894240078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c032810 con 0x7f77a40d1cb0 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.629+0000 7f77a3e00640 1 -- 172.21.15.46:0/2894240078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a40d1cb0 msgr2=0x7f77a40b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.629+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2894240078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a40d1cb0 0x7f77a40b29e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f77a4074b40 tx=0x7f779c02a2c0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 -- 172.21.15.46:0/2894240078 shutdown_connections 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2894240078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a40d1cb0 0x7f77a40b29e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2894240078 >> v2:172.21.15.142:3300/0 conn(0x7f77a4162a70 0x7f77a40d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 -- 172.21.15.46:0/2894240078 >> 172.21.15.46:0/2894240078 conn(0x7f77a40d4f20 msgr2=0x7f77a40d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 -- 172.21.15.46:0/2894240078 shutdown_connections 2024-04-26T23:41:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.630+0000 7f77a3e00640 1 -- 172.21.15.46:0/2894240078 wait complete. 2024-04-26T23:41:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.632+0000 7f77a3e00640 1 Processor -- start 2024-04-26T23:41:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.632+0000 7f77a3e00640 1 -- start start 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77a40d1cb0 0x7f77a4155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a40d3830 0x7f77a4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4162fd0 con 0x7f77a4162a70 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4162e50 con 0x7f77a40d3830 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77a40b2f20 con 0x7f77a40d1cb0 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51826/0 (socket says 172.21.15.46:51826) 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a3400640 1 -- 172.21.15.46:0/2342721865 learned_addr learned my addr 172.21.15.46:0/2342721865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a2a00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77a40d1cb0 0x7f77a4155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.633+0000 7f77a2000640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a40d3830 0x7f77a4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 -- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77a40d1cb0 msgr2=0x7f77a4155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77a40d1cb0 0x7f77a4155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 -- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a40d3830 msgr2=0x7f77a4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a40d3830 0x7f77a4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 -- 172.21.15.46:0/2342721865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4155cb0 con 0x7f77a4162a70 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f77a3400640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f779c02a300 tx=0x7f779c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f779aa00640 1 -- 172.21.15.46:0/2342721865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c037040 con 0x7f77a4162a70 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f779aa00640 1 -- 172.21.15.46:0/2342721865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c042070 con 0x7f77a4162a70 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.634+0000 7f779aa00640 1 -- 172.21.15.46:0/2342721865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c030c90 con 0x7f77a4162a70 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.635+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a4157600 con 0x7f77a4162a70 2024-04-26T23:41:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.635+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a415b230 con 0x7f77a4162a70 2024-04-26T23:41:35.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.637+0000 7f779aa00640 1 -- 172.21.15.46:0/2342721865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f779c047460 con 0x7f77a4162a70 2024-04-26T23:41:35.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.638+0000 7f779aa00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 0x7f778006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.638+0000 7f779aa00640 1 -- 172.21.15.46:0/2342721865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f779c0ac9f0 con 0x7f77a4162a70 2024-04-26T23:41:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.639+0000 7f77a2a00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 0x7f778006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.639+0000 7f77ac3e6740 1 Processor -- start 2024-04-26T23:41:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.639+0000 7f77ac3e6740 1 -- start start 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874fed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fc2e20 0x55f8750090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f874f26900 con 0x55f87500a9e0 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f874ec31d0 con 0x55f874fc2e20 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f874f228c0 con 0x55f874fed140 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2a00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 0x7f778006edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f778c00a950 tx=0x7f778c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51830/0 (socket says 172.21.15.46:51830) 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 -- 172.21.15.46:0/1854373521 learned_addr learned my addr 172.21.15.46:0/1854373521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a3400640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fc2e20 0x55f8750090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874fed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 -- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 msgr2=0x55f874fed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874fed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 -- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fc2e20 msgr2=0x55f8750090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fc2e20 0x55f8750090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2000640 1 -- 172.21.15.46:0/1854373521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f874d027d0 con 0x55f87500a9e0 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.640+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874fed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.641+0000 7f77a2000640 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7794005440 tx=0x7f7794005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2f44c93fde7557 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.641+0000 7f7793000640 1 -- 172.21.15.46:0/1854373521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794006b50 con 0x55f87500a9e0 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.641+0000 7f7793000640 1 -- 172.21.15.46:0/1854373521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7794006d10 con 0x55f87500a9e0 2024-04-26T23:41:35.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.641+0000 7f7793000640 1 -- 172.21.15.46:0/1854373521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400ead0 con 0x55f87500a9e0 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1854373521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 msgr2=0x55f87500af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7794005440 tx=0x7f7794005900 comp rx=0 tx=0).stop 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1854373521 shutdown_connections 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fc2e20 0x55f8750090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f87500af30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1854373521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874fed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1854373521 >> 172.21.15.46:0/1854373521 conn(0x55f874fbf750 msgr2=0x55f875007f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1854373521 shutdown_connections 2024-04-26T23:41:35.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.642+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1854373521 wait complete. 2024-04-26T23:41:35.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.643+0000 7f77ac3e6740 1 -- start start 2024-04-26T23:41:35.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.643+0000 7f77ac3e6740 1 Processor -- start 2024-04-26T23:41:35.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f874fe1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 0x55f874fdd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f874fc3bc0 con 0x55f874fffa10 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f874eae120 con 0x55f874fed140 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f874fbd680 con 0x55f87500a9e0 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77ac3e6740 10 client.0 _ll_register_callbacks cb 0x55f874ee6750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51838/0 (socket says 172.21.15.46:51838) 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77a3400640 1 -- 172.21.15.46:0/1156048972 learned_addr learned my addr 172.21.15.46:0/1156048972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f874fe1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.644+0000 7f77a2000640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 0x55f874fdd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 -- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 msgr2=0x55f874fdd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 0x55f874fdd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 -- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 msgr2=0x55f874fe1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f874fe1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 -- 172.21.15.46:0/1156048972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f874bf80f0 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a2000640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 0x55f874fdd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77a3400640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f779c0aacc0 tx=0x7f779c032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b1b9574337c00f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f7787400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c07a9d0 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f779c082c10 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794006d10 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f779c07e870 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77ac3e6740 10 client.5589 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.645+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55f874d62f60 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7794006d10 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f779c03a030 con 0x55f874fffa10 2024-04-26T23:41:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 10 client.5589 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 10 client.5589 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 20 client.5589 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875006c40 0x55f875007050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 4 client.5589 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f874f34510 con 0x55f875006c40 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.646+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875006c40 0x55f875007050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.647+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875006c40 0x55f875007050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a000b1c69b317a6 server_cookie=66a041fc2c18b85b in_seq=0 out_seq=0 2024-04-26T23:41:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.647+0000 7f7787400640 10 client.5589 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.647+0000 7f7787400640 1 -- 172.21.15.46:0/1156048972 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f874f34510 con 0x55f875006c40 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.647+0000 7f7787400640 10 client.5589 handle_command_reply: tid=0 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 client.5589 shutdown 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 msgr2=0x55f874fddc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f779c0aacc0 tx=0x7f779c032440 comp rx=0 tx=0).stop 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 shutdown_connections 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875006c40 0x55f875007050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f874fffa10 0x55f874fddc30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f87500a9e0 0x55f874fdd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1156048972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f874fe1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f7787400640 0 client.5589 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 >> 172.21.15.46:0/1156048972 conn(0x55f874fc2e20 msgr2=0x55f87500bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 shutdown_connections 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1156048972 wait complete. 2024-04-26T23:41:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.648+0000 7f77ac3e6740 20 client.5589 trim_cache size 0 max 16384 2024-04-26T23:41:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.666+0000 7f77ac3e6740 1 Processor -- start 2024-04-26T23:41:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 -- start start 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f8750174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fffa10 0x55f874fff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4201d50 con 0x55f87500a9e0 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4200da0 con 0x55f874fed140 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.667+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77a415c620 con 0x55f874fffa10 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51848/0 (socket says 172.21.15.46:51848) 2024-04-26T23:41:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 -- 172.21.15.46:0/3542571506 learned_addr learned my addr 172.21.15.46:0/3542571506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 -- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fffa10 msgr2=0x55f874fff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fffa10 0x55f874fff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 -- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 msgr2=0x55f8750174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f8750174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 -- 172.21.15.46:0/3542571506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f874bd11b0 con 0x55f87500a9e0 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.668+0000 7f77a3400640 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f779c082b70 tx=0x7f779c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8e869edc180ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f7792600640 1 -- 172.21.15.46:0/3542571506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c07a7d0 con 0x55f87500a9e0 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f7792600640 1 -- 172.21.15.46:0/3542571506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c07a970 con 0x55f87500a9e0 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f7792600640 1 -- 172.21.15.46:0/3542571506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c079bc0 con 0x55f87500a9e0 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 -- 172.21.15.46:0/3542571506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 msgr2=0x55f874fe0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f779c082b70 tx=0x7f779c07c000 comp rx=0 tx=0).stop 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 -- 172.21.15.46:0/3542571506 shutdown_connections 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fe0230 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fffa10 0x55f874fff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/3542571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fed140 0x55f8750174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 -- 172.21.15.46:0/3542571506 >> 172.21.15.46:0/3542571506 conn(0x55f874fc2e20 msgr2=0x55f874fb8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 -- 172.21.15.46:0/3542571506 shutdown_connections 2024-04-26T23:41:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.669+0000 7f77ac3e6740 1 -- 172.21.15.46:0/3542571506 wait complete. 2024-04-26T23:41:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.670+0000 7f77ac3e6740 1 -- start start 2024-04-26T23:41:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.670+0000 7f77ac3e6740 1 Processor -- start 2024-04-26T23:41:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874ffe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fffa10 0x55f874ff07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f874ff14f0 con 0x55f87500a9e0 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4200850 con 0x55f874fffa10 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f875004a60 con 0x55f874fed140 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.671+0000 7f77ac3e6740 10 client.0 _ll_register_callbacks cb 0x7f779c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51862/0 (socket says 172.21.15.46:51862) 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 -- 172.21.15.46:0/1838233669 learned_addr learned my addr 172.21.15.46:0/1838233669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 -- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 msgr2=0x55f874ffe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874ffe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 -- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fffa10 msgr2=0x55f874ff07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fffa10 0x55f874ff07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 -- 172.21.15.46:0/1838233669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f874fc3200 con 0x55f87500a9e0 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f77a3400640 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f779c0aacc0 tx=0x7f779c0aab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d72c7a33f90b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f7787400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.672+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c07a7d0 con 0x55f87500a9e0 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f779c079430 con 0x55f87500a9e0 2024-04-26T23:41:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c07a970 con 0x55f87500a9e0 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f779c0a0050 con 0x55f87500a9e0 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 10 client.5595 fetch_fsmap learned FSMap version 19 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55f874fc3200 con 0x55f87500a9e0 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c07a970 con 0x55f87500a9e0 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f779c079630 con 0x55f87500a9e0 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 10 client.5595 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 10 client.5595 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 20 client.5595 populate_metadata read hostname 'smithi046' 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875080dd0 0x55f874fdd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 4 client.5595 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.673+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 --> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55f874f2e380 con 0x55f875080dd0 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.674+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875080dd0 0x55f874fdd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.674+0000 7f77a2a00640 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875080dd0 0x55f874fdd680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aca497ac2138bb0 server_cookie=14a7e3ece22a8059 in_seq=0 out_seq=0 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.674+0000 7f7787400640 10 client.5595 ms_handle_connect on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f7787400640 1 -- 172.21.15.46:0/1838233669 <== mds.0 v2:172.21.15.46:6833/1027205255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18004 (crc 0 0 0) 0x55f874f2e380 con 0x55f875080dd0 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f7787400640 10 client.5595 handle_command_reply: tid=0 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 client.5595 shutdown 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 msgr2=0x55f874fdd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f779c0aacc0 tx=0x7f779c0aab90 comp rx=0 tx=0).stop 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 shutdown_connections 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:6833/1027205255,v1:172.21.15.46:6835/1027205255] conn(0x55f875080dd0 0x55f874fdd680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f87500a9e0 0x55f874fdd140 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f874fffa10 0x55f874ff07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 --2- 172.21.15.46:0/1838233669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f874fed140 0x55f874ffe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f7787400640 0 client.5595 ms_handle_reset on v2:172.21.15.46:6833/1027205255 2024-04-26T23:41:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 >> 172.21.15.46:0/1838233669 conn(0x55f874fc2e20 msgr2=0x55f874fe0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 shutdown_connections 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 1 -- 172.21.15.46:0/1838233669 wait complete. 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.675+0000 7f77ac3e6740 20 client.5595 trim_cache size 0 max 16384 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2225, 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2217, 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 668220, 2024-04-26T23:41:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 15810627, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 16, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.458698220, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.221926091, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.117135917, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.102978715, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2217, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.697489069, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000314609 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 10, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000001134, 2024-04-26T23:41:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000113 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 129248, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 38416 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2815, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2851, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 711791, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 23747694, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.548144454, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.291966880, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.091642642, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.126997301, 2024-04-26T23:41:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2850, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.909170068, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000319007 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 115648 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 30764, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 52736, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 10802854, 2024-04-26T23:41:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 77074373, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 9.259823504, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 6.529892425, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 2.838699915, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.325323841, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 52735, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.290803638, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000081365 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 46466, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.025663035, 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000552 2024-04-26T23:41:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2970, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 32.007195365, 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.010776833 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3893, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.390124551, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000613954 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 25580, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 25580, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 25580, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 615.630217364, 2024-04-26T23:41:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.024066857 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 12, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 17913, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 17913, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 17912, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 2, 2024-04-26T23:41:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 2, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 2, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 46586, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 46015, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 111, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 2013, 2024-04-26T23:41:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 2013, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 13, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 2, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 11, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 2000, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 979, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 4031, 2024-04-26T23:41:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 3045, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 2207, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 55, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 14103 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-26T23:41:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 4176, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 3969, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 2231, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 1, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-26T23:41:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 26745, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 26745, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 36, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 36, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 4, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-04-26T23:41:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 96065794, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2684, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 145.168402816, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.054086588 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 14, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 2016, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 2002, 2024-04-26T23:41:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 13, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 13, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 0, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 17913, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 19914, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 2001, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 2, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 7474, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 7472, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 98712, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231788 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 28585, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 25580, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 10, 2024-04-26T23:41:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2000, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.977572468, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001988786 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 30, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009375929, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000312530 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6000, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 17.657319442, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002942886 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7305, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 8.751032896, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001197951 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000110948, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000110948 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.006087646, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006087646 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.027099108, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.027099108 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000151502, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000151502 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8, 2024-04-26T23:41:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.216850655, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.027106331 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2231, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 544.647727673, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.244127175 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008542158, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008542158 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000725756, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000725756 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8000, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 40.327621183, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.005040952 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 1, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 0, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 14092, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 14092, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 147, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-26T23:41:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-26T23:41:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-26T23:41:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 250852, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 92, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 440, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 5, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-26T23:41:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 5624, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 80, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 13490797, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 661853, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-26T23:41:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 9066, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 92573573, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 37, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 9066, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 9066, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 60.968327705, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006724942 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 9066, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 9228 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 9066, 2024-04-26T23:41:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 2104, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 6962, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 11, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 89, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 2771, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 54, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 1969, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 2033, 2024-04-26T23:41:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 2015, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 22, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-04-26T23:41:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-26T23:41:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 43, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 5, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 8, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 8, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 216, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 48 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-26T23:41:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 32, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 32, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 3969, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 3969, 2024-04-26T23:41:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 35798, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 9379998, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 35798, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 35798, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 9379998, 2024-04-26T23:41:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:41:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 9066, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 92990194, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 9066, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 92990194, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:41:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 9066, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 9066, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 9066, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 9066, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 3969, 2024-04-26T23:41:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 416621, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 3969, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 87, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 416621, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x556b3e7be500": { 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:41:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 26745, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 91871490, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 26745, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 2684, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 91871490, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:35.723 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 msgr2=0x7f778006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 0x7f778006edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f778c00a950 tx=0x7f778c008040 comp rx=0 tx=0).stop 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 msgr2=0x7f77a40ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f779c02a300 tx=0x7f779c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 shutdown_connections 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f778006c8f0 0x7f778006edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a4162a70 0x7f77a40ca4d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a40d3830 0x7f77a4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 --2- 172.21.15.46:0/2342721865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77a40d1cb0 0x7f77a4155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:35.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 >> 172.21.15.46:0/2342721865 conn(0x7f77a40d4f20 msgr2=0x7f77a40cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:35.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 shutdown_connections 2024-04-26T23:41:35.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:35.676+0000 7f77a3e00640 1 -- 172.21.15.46:0/2342721865 wait complete. 2024-04-26T23:41:35.725 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:41:35.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:41:35.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:41:35.950 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:41:35.950 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:41:35.950 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:41:35.950 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:36.012 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:41:36.012 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:36.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:36.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.280+0000 7f29ec400640 1 Processor -- start 2024-04-26T23:41:36.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.280+0000 7f29ec400640 1 -- start start 2024-04-26T23:41:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29e4142280 0x7f29e406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e40b3e80 0x7f29e406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 0x7f29e4144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40b43f0 con 0x7f29e4142280 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40b46d0 con 0x7f29e4144840 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29ec400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29e40b49b0 con 0x7f29e40b3e80 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.282+0000 7f29e9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29e4142280 0x7f29e406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29e4142280 0x7f29e406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51868/0 (socket says 172.21.15.46:51868) 2024-04-26T23:41:36.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 -- 172.21.15.46:0/1298334770 learned_addr learned my addr 172.21.15.46:0/1298334770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9000640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e40b3e80 0x7f29e406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29ea400640 1 --2- 172.21.15.46:0/1298334770 >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 0x7f29e4144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 -- 172.21.15.46:0/1298334770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e40b3e80 msgr2=0x7f29e406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e40b3e80 0x7f29e406ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 -- 172.21.15.46:0/1298334770 >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 msgr2=0x7f29e4144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1298334770 >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 0x7f29e4144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29e9a00640 1 -- 172.21.15.46:0/1298334770 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40b4c90 con 0x7f29e4142280 2024-04-26T23:41:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.283+0000 7f29ea400640 1 --2- 172.21.15.46:0/1298334770 >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 0x7f29e4144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.285+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e4142280 0x7f29e406e510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29e4070170 tx=0x7f29d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2de12a3b7bec89b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.285+0000 7f29e3e00640 1 -- 172.21.15.46:0/1298334770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8036070 con 0x7f29e4142280 2024-04-26T23:41:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.285+0000 7f29e3e00640 1 -- 172.21.15.46:0/1298334770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d8031600 con 0x7f29e4142280 2024-04-26T23:41:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.285+0000 7f29e3e00640 1 -- 172.21.15.46:0/1298334770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80319d0 con 0x7f29e4142280 2024-04-26T23:41:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 -- 172.21.15.46:0/1298334770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e4142280 msgr2=0x7f29e406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e4142280 0x7f29e406e510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29e4070170 tx=0x7f29d802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 -- 172.21.15.46:0/1298334770 shutdown_connections 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 --2- 172.21.15.46:0/1298334770 >> v2:172.21.15.142:3300/0 conn(0x7f29e4144840 0x7f29e4144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e40b3e80 0x7f29e406ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 --2- 172.21.15.46:0/1298334770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e4142280 0x7f29e406e510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 -- 172.21.15.46:0/1298334770 >> 172.21.15.46:0/1298334770 conn(0x7f29e406c530 msgr2=0x7f29e406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 -- 172.21.15.46:0/1298334770 shutdown_connections 2024-04-26T23:41:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.286+0000 7f29ec400640 1 -- 172.21.15.46:0/1298334770 wait complete. 2024-04-26T23:41:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.288+0000 7f29ec400640 1 Processor -- start 2024-04-26T23:41:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.288+0000 7f29ec400640 1 -- start start 2024-04-26T23:41:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29e4142280 0x7f29e40b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e4144840 0x7f29e41f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4142a20 con 0x7f29e40b3e80 2024-04-26T23:41:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41428a0 con 0x7f29e4142280 2024-04-26T23:41:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.289+0000 7f29ec400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29e41451f0 con 0x7f29e4144840 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51876/0 (socket says 172.21.15.46:51876) 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 -- 172.21.15.46:0/1340867000 learned_addr learned my addr 172.21.15.46:0/1340867000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29ea400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e4144840 0x7f29e41f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9000640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29e4142280 0x7f29e40b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 -- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e4144840 msgr2=0x7f29e41f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e4144840 0x7f29e41f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 -- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29e4142280 msgr2=0x7f29e40b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29e4142280 0x7f29e40b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.290+0000 7f29e9a00640 1 -- 172.21.15.46:0/1340867000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40afe60 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29e9a00640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f29d802ac90 tx=0x7f29d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8041020 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d8008790 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e40ce410 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8036040 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.291+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e40d0ab0 con 0x7f29e40b3e80 2024-04-26T23:41:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.292+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4069530 con 0x7f29e40b3e80 2024-04-26T23:41:36.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.293+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29d8044020 con 0x7f29e40b3e80 2024-04-26T23:41:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.295+0000 7f29e1600640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.295+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f29d80aca10 con 0x7f29e40b3e80 2024-04-26T23:41:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.296+0000 7f29e9000640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.297+0000 7f29e9000640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 0x7f29c406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f29e40cf430 tx=0x7f29d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.300+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29d804b070 con 0x7f29e40b3e80 2024-04-26T23:41:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.507+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29e406ee80 con 0x7f29e40b3e80 2024-04-26T23:41:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.510+0000 7f29e1600640 1 -- 172.21.15.46:0/1340867000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17020 (secure 0 0 0) 0x7f29d807ac70 con 0x7f29e40b3e80 2024-04-26T23:41:36.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:36.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:13.670571+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 msgr2=0x7f29c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 0x7f29c406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f29e40cf430 tx=0x7f29d0008040 comp rx=0 tx=0).stop 2024-04-26T23:41:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 msgr2=0x7f29e40d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f29d802ac90 tx=0x7f29d8004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 shutdown_connections 2024-04-26T23:41:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29e4144840 0x7f29e41f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29e4142280 0x7f29e40b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 --2- 172.21.15.46:0/1340867000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e40b3e80 0x7f29e40d0570 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 >> 172.21.15.46:0/1340867000 conn(0x7f29e406c530 msgr2=0x7f29e4143580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 shutdown_connections 2024-04-26T23:41:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.513+0000 7f29ec400640 1 -- 172.21.15.46:0/1340867000 wait complete. 2024-04-26T23:41:36.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:38:49.046203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:38:50.124386+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:41:36.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:41:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.758+0000 7fb789c00640 1 Processor -- start 2024-04-26T23:41:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.758+0000 7fb789c00640 1 -- start start 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb7840d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7840b53b0 0x7fb78415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb78415e050 0x7fb78406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb78406eb90 con 0x7fb78415e050 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb7840b4310 con 0x7fb7840b53b0 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb789c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840b45f0 con 0x7fb78415d730 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb783400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb78415e050 0x7fb78406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb783400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb78415e050 0x7fb78406dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51880/0 (socket says 172.21.15.46:51880) 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb783400640 1 -- 172.21.15.46:0/3793656341 learned_addr learned my addr 172.21.15.46:0/3793656341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb782000640 1 --2- 172.21.15.46:0/3793656341 >> v2:172.21.15.142:3300/0 conn(0x7fb7840b53b0 0x7fb78415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.760+0000 7fb782a00640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb7840d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.761+0000 7fb783400640 1 -- 172.21.15.46:0/3793656341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 msgr2=0x7fb7840d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.761+0000 7fb783400640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb7840d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.761+0000 7fb783400640 1 -- 172.21.15.46:0/3793656341 >> v2:172.21.15.142:3300/0 conn(0x7fb7840b53b0 msgr2=0x7fb78415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.761+0000 7fb783400640 1 --2- 172.21.15.46:0/3793656341 >> v2:172.21.15.142:3300/0 conn(0x7fb7840b53b0 0x7fb78415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.761+0000 7fb783400640 1 -- 172.21.15.46:0/3793656341 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840b48d0 con 0x7fb78415e050 2024-04-26T23:41:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.763+0000 7fb783400640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb78406dc90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb7840741d0 tx=0x7fb77c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad2c634bc835c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.765+0000 7fb781600640 1 -- 172.21.15.46:0/3793656341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c036070 con 0x7fb78415e050 2024-04-26T23:41:36.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.765+0000 7fb781600640 1 -- 172.21.15.46:0/3793656341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c031600 con 0x7fb78415e050 2024-04-26T23:41:36.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.765+0000 7fb781600640 1 -- 172.21.15.46:0/3793656341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0319d0 con 0x7fb78415e050 2024-04-26T23:41:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 -- 172.21.15.46:0/3793656341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 msgr2=0x7fb78406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb78406dc90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb7840741d0 tx=0x7fb77c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 -- 172.21.15.46:0/3793656341 shutdown_connections 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb78406dc90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 --2- 172.21.15.46:0/3793656341 >> v2:172.21.15.142:3300/0 conn(0x7fb7840b53b0 0x7fb78415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 --2- 172.21.15.46:0/3793656341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb7840d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 -- 172.21.15.46:0/3793656341 >> 172.21.15.46:0/3793656341 conn(0x7fb78406c980 msgr2=0x7fb7840b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 -- 172.21.15.46:0/3793656341 shutdown_connections 2024-04-26T23:41:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.766+0000 7fb789c00640 1 -- 172.21.15.46:0/3793656341 wait complete. 2024-04-26T23:41:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.768+0000 7fb789c00640 1 Processor -- start 2024-04-26T23:41:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.768+0000 7fb789c00640 1 -- start start 2024-04-26T23:41:36.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 0x7fb7841e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb78413b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb78415e4c0 con 0x7fb78415e050 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840b4e90 con 0x7fb7840b53b0 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.769+0000 7fb789c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840b3ec0 con 0x7fb78415d730 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51896/0 (socket says 172.21.15.46:51896) 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 -- 172.21.15.46:0/702925512 learned_addr learned my addr 172.21.15.46:0/702925512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb782a00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 0x7fb7841e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb782000640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb78413b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 -- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 msgr2=0x7fb78413b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb78413b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 -- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 msgr2=0x7fb7841e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 0x7fb7841e8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 -- 172.21.15.46:0/702925512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840b2920 con 0x7fb78415e050 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb782a00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 0x7fb7841e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:36.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.770+0000 7fb783400640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02ac90 tx=0x7fb77c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.771+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c03f020 con 0x7fb78415e050 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.771+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c037040 con 0x7fb78415e050 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.771+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c036070 con 0x7fb78415e050 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.771+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7840b1880 con 0x7fb78415e050 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.771+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7840b1e40 con 0x7fb78415e050 2024-04-26T23:41:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.772+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb784069530 con 0x7fb78415e050 2024-04-26T23:41:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.773+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c042020 con 0x7fb78415e050 2024-04-26T23:41:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.774+0000 7fb77aa00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.774+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fb77c0ac500 con 0x7fb78415e050 2024-04-26T23:41:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.775+0000 7fb782a00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.776+0000 7fb782a00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 0x7fb76406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb774006fd0 tx=0x7fb774008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.779+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb77c049020 con 0x7fb78415e050 2024-04-26T23:41:36.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.987+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7840d01c0 con 0x7fb78415e050 2024-04-26T23:41:36.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.988+0000 7fb77aa00640 1 -- 172.21.15.46:0/702925512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7fb77c07a760 con 0x7fb78415e050 2024-04-26T23:41:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.990+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 msgr2=0x7fb76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.990+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 0x7fb76406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb774006fd0 tx=0x7fb774008040 comp rx=0 tx=0).stop 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.990+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 msgr2=0x7fb7840b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.990+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02ac90 tx=0x7fb77c004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 shutdown_connections 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78415e050 0x7fb7840b1130 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb78415d730 0x7fb78413b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 --2- 172.21.15.46:0/702925512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840b53b0 0x7fb7841e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:36.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 >> 172.21.15.46:0/702925512 conn(0x7fb78406c980 msgr2=0x7fb7840cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:36.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 shutdown_connections 2024-04-26T23:41:36.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:36.991+0000 7fb789c00640 1 -- 172.21.15.46:0/702925512 wait complete. 2024-04-26T23:41:37.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:41:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.226+0000 7ff44a200640 1 Processor -- start 2024-04-26T23:41:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.226+0000 7ff44a200640 1 -- start start 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff44a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4441597e0 0x7ff4440ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff44a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 0x7ff4440b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff44a200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff444073120 con 0x7ff4440b5730 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff44a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff444159bc0 con 0x7ff4441597e0 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff44a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff444072b60 con 0x7ff444072620 2024-04-26T23:41:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff443400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4441597e0 0x7ff4440ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff443400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4441597e0 0x7ff4440ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51912/0 (socket says 172.21.15.46:51912) 2024-04-26T23:41:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.228+0000 7ff443400640 1 -- 172.21.15.46:0/508093444 learned_addr learned my addr 172.21.15.46:0/508093444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443e00640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 0x7ff4440b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443400640 1 -- 172.21.15.46:0/508093444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 msgr2=0x7ff4440b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443400640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 0x7ff4440b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443400640 1 -- 172.21.15.46:0/508093444 >> v1:172.21.15.46:6789/0 conn(0x7ff4440b5730 legacy=0x7ff444071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443400640 1 -- 172.21.15.46:0/508093444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff444072e40 con 0x7ff4441597e0 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.229+0000 7ff443e00640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 0x7ff4440b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.230+0000 7ff443400640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4441597e0 0x7ff4440ae8e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff434009870 tx=0x7ff43402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f50bcd0bdf69db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.230+0000 7ff442000640 1 -- 172.21.15.46:0/508093444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434037070 con 0x7ff4441597e0 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.230+0000 7ff442000640 1 -- 172.21.15.46:0/508093444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43402ae50 con 0x7ff4441597e0 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.230+0000 7ff442000640 1 -- 172.21.15.46:0/508093444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434032860 con 0x7ff4441597e0 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 -- 172.21.15.46:0/508093444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4441597e0 msgr2=0x7ff4440ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4441597e0 0x7ff4440ae8e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff434009870 tx=0x7ff43402a300 comp rx=0 tx=0).stop 2024-04-26T23:41:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 -- 172.21.15.46:0/508093444 shutdown_connections 2024-04-26T23:41:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444072620 0x7ff4440b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 --2- 172.21.15.46:0/508093444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4441597e0 0x7ff4440ae8e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 -- 172.21.15.46:0/508093444 >> 172.21.15.46:0/508093444 conn(0x7ff44406c980 msgr2=0x7ff4440b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 -- 172.21.15.46:0/508093444 shutdown_connections 2024-04-26T23:41:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.231+0000 7ff44a200640 1 -- 172.21.15.46:0/508093444 wait complete. 2024-04-26T23:41:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.233+0000 7ff44a200640 1 Processor -- start 2024-04-26T23:41:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.233+0000 7ff44a200640 1 -- start start 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 0x7ff44414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4441597e0 0x7ff4440cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440aee20 con 0x7ff4440b5730 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440b43b0 con 0x7ff444072620 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff44a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440aefa0 con 0x7ff4441597e0 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff442a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff442a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51928/0 (socket says 172.21.15.46:51928) 2024-04-26T23:41:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff442a00640 1 -- 172.21.15.46:0/1938642690 learned_addr learned my addr 172.21.15.46:0/1938642690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff443e00640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4441597e0 0x7ff4440cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff443400640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 0x7ff44414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.234+0000 7ff442a00640 1 -- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4441597e0 msgr2=0x7ff4440cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff442a00640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4441597e0 0x7ff4440cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff442a00640 1 -- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 msgr2=0x7ff44414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff442a00640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 0x7ff44414d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff442a00640 1 -- 172.21.15.46:0/1938642690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4441f8850 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff443400640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 0x7ff44414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff442a00640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c400 tx=0x7ff43000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300052d0 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff430005490 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4441f8b30 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430007260 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.235+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4441f90f0 con 0x7ff4440b5730 2024-04-26T23:41:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.237+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff444069530 con 0x7ff4440b5730 2024-04-26T23:41:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.238+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff430007400 con 0x7ff4440b5730 2024-04-26T23:41:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.239+0000 7ff43b400640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 0x7ff42406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.239+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7ff43008b260 con 0x7ff4440b5730 2024-04-26T23:41:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.239+0000 7ff443400640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 0x7ff42406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.241+0000 7ff443400640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 0x7ff42406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff434034000 tx=0x7ff43402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.243+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4300594d0 con 0x7ff4440b5730 2024-04-26T23:41:37.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.445+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4440b5c60 con 0x7ff4440b5730 2024-04-26T23:41:37.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.446+0000 7ff43b400640 1 -- 172.21.15.46:0/1938642690 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) ==== 104+0+11 (secure 0 0 0) 0x7ff430058dd0 con 0x7ff4440b5730 2024-04-26T23:41:37.449 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:41:37.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 msgr2=0x7ff42406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 0x7ff42406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff434034000 tx=0x7ff43402e000 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 msgr2=0x7ff44414f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c400 tx=0x7ff43000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 shutdown_connections 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42406c8f0 0x7ff42406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4441597e0 0x7ff4440cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4440b5730 0x7ff44414f9f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 --2- 172.21.15.46:0/1938642690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff444072620 0x7ff44414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 >> 172.21.15.46:0/1938642690 conn(0x7ff44406c980 msgr2=0x7ff444162b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 shutdown_connections 2024-04-26T23:41:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.448+0000 7ff44a200640 1 -- 172.21.15.46:0/1938642690 wait complete. 2024-04-26T23:41:37.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:41:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.678+0000 7f7fa3e00640 1 Processor -- start 2024-04-26T23:41:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.678+0000 7f7fa3e00640 1 -- start start 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.679+0000 7f7fa3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7fa4146740 0x7f7fa415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa4146b20 con 0x7f7fa40d0330 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa40d07c0 con 0x7f7fa40ce670 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa3e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa4161a00 con 0x7f7fa4146740 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7f9be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7fa40d0330 0x7f7fa41612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35730/0 (socket says 172.21.15.46:35730) 2024-04-26T23:41:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7f9be00640 1 -- 172.21.15.46:0/4104385048 learned_addr learned my addr 172.21.15.46:0/4104385048 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/4104385048 >> v2:172.21.15.142:3300/0 conn(0x7f7fa4146740 0x7f7fa415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553691536 0 0) 0x7f7fa4146b20 con 0x7f7fa40d0330 2024-04-26T23:41:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.680+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f84003520 con 0x7f7fa40d0330 2024-04-26T23:41:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128544249 0 0) 0x7f7f84003520 con 0x7f7fa40d0330 2024-04-26T23:41:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 >> v2:172.21.15.142:3300/0 conn(0x7f7fa4146740 msgr2=0x7f7fa415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 --2- 172.21.15.46:0/4104385048 >> v2:172.21.15.142:3300/0 conn(0x7f7fa4146740 0x7f7fa415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 >> v1:172.21.15.142:6789/0 conn(0x7f7fa40ce670 legacy=0x7f7fa40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa40ccf30 con 0x7f7fa40d0330 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7f8c001670 con 0x7f7fa40d0330 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 >> v1:172.21.15.46:6789/0 conn(0x7f7fa40d0330 legacy=0x7f7fa41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7f84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f84004800 0x7f7f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f840051c0 0x7f7f8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4161a00 con 0x7f7f84003a00 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8400c0c0 con 0x7f7f840051c0 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f840106d0 con 0x7f7f84004800 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.681+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7f84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa3400640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f840051c0 0x7f7f8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa2a00640 1 -- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f84004800 msgr2=0x7f7f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f84004800 0x7f7f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa2a00640 1 -- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f840051c0 msgr2=0x7f7f8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f840051c0 0x7f7f8400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa2a00640 1 -- 172.21.15.46:0/4104385048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f84003520 con 0x7f7f84003a00 2024-04-26T23:41:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.682+0000 7f7fa3400640 1 -- 172.21.15.46:0/4104385048 reap_dead start 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7f84003e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009ac0 tx=0x7f7f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8089982390d4340 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94036070 con 0x7f7f84003a00 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94031400 con 0x7f7f84003a00 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa2000640 1 -- 172.21.15.46:0/4104385048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94009690 con 0x7f7f84003a00 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa3e00640 1 -- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 msgr2=0x7f7f84003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.683+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7f84003e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009ac0 tx=0x7f7f84005830 comp rx=0 tx=0).stop 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.684+0000 7f7fa3e00640 1 -- 172.21.15.46:0/4104385048 shutdown_connections 2024-04-26T23:41:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.684+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/4104385048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7f84003e20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.684+0000 7f7fa3e00640 1 -- 172.21.15.46:0/4104385048 >> 172.21.15.46:0/4104385048 conn(0x7f7fa40b26d0 msgr2=0x7f7fa40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.684+0000 7f7fa3e00640 1 -- 172.21.15.46:0/4104385048 shutdown_connections 2024-04-26T23:41:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.684+0000 7f7fa3e00640 1 -- 172.21.15.46:0/4104385048 wait complete. 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.685+0000 7f7fa3e00640 1 Processor -- start 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.686+0000 7f7fa3e00640 1 -- start start 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.686+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.686+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40c5940 0x7f7fa41fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa413f110 0x7f7fa413f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa40ccb70 con 0x7f7f84003a00 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4160dd0 con 0x7f7fa40c5940 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa40cccf0 con 0x7f7fa413f110 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51936/0 (socket says 172.21.15.46:51936) 2024-04-26T23:41:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1254335813 learned_addr learned my addr 172.21.15.46:0/1254335813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:37.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa413f110 msgr2=0x7f7fa413f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa413f110 0x7f7fa413f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40c5940 msgr2=0x7f7fa41fcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40c5940 0x7f7fa41fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1254335813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa413fad0 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.687+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f94034040 tx=0x7f7f94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.688+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9403f020 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.688+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94037040 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.688+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94036070 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.688+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa40c6120 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.688+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa40c66e0 con 0x7f7f84003a00 2024-04-26T23:41:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.689+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa40693b0 con 0x7f7f84003a00 2024-04-26T23:41:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.690+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f94042020 con 0x7f7f84003a00 2024-04-26T23:41:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.691+0000 7f7f9aa00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 0x7f7f8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.691+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f7f940abdd0 con 0x7f7f84003a00 2024-04-26T23:41:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.691+0000 7f7f9be00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 0x7f7f8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.693+0000 7f7f9be00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 0x7f7f8006ede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4073d40 tx=0x7f7f8c001590 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:37.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.695+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f94049030 con 0x7f7f84003a00 2024-04-26T23:41:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.898+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fa40b0900 con 0x7f7f84003a00 2024-04-26T23:41:37.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.899+0000 7f7f9aa00640 1 -- 172.21.15.46:0/1254335813 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) ==== 100+0+11 (secure 0 0 0) 0x7f7f9407a060 con 0x7f7f84003a00 2024-04-26T23:41:37.903 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 msgr2=0x7f7f8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 0x7f7f8006ede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4073d40 tx=0x7f7f8c001590 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 msgr2=0x7f7fa4144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f94034040 tx=0x7f7f94004730 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 shutdown_connections 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f8006c920 0x7f7f8006ede0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa413f110 0x7f7fa413f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40c5940 0x7f7fa41fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1254335813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84003a00 0x7f7fa4144150 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 >> 172.21.15.46:0/1254335813 conn(0x7f7fa40b26d0 msgr2=0x7f7fa40cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.902+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 shutdown_connections 2024-04-26T23:41:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:37.903+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1254335813 wait complete. 2024-04-26T23:41:37.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:41:38.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.138+0000 7fefe7200640 1 Processor -- start 2024-04-26T23:41:38.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.138+0000 7fefe7200640 1 -- start start 2024-04-26T23:41:38.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fefe81454a0 0x7fefe80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe8072bb0 con 0x7fefe8073970 2024-04-26T23:41:38.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe8072e90 con 0x7fefe80c6a50 2024-04-26T23:41:38.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fefe8073170 con 0x7fefe81454a0 2024-04-26T23:41:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fefe8073970 0x7fefe80cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35732/0 (socket says 172.21.15.46:35732) 2024-04-26T23:41:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.140+0000 7fefe5400640 1 -- 172.21.15.46:0/2142616501 learned_addr learned my addr 172.21.15.46:0/2142616501 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe5e00640 1 --2- 172.21.15.46:0/2142616501 >> v2:172.21.15.142:3300/0 conn(0x7fefe81454a0 0x7fefe80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260661094 0 0) 0x7fefe8072bb0 con 0x7fefe8073970 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefcc003520 con 0x7fefe8073970 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2783370805 0 0) 0x7fefcc003520 con 0x7fefe8073970 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 >> v2:172.21.15.142:3300/0 conn(0x7fefe81454a0 msgr2=0x7fefe80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 --2- 172.21.15.46:0/2142616501 >> v2:172.21.15.142:3300/0 conn(0x7fefe81454a0 0x7fefe80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 >> v1:172.21.15.142:6789/0 conn(0x7fefe80c6a50 legacy=0x7fefe80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8073450 con 0x7fefe8073970 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fefd40019f0 con 0x7fefe8073970 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 >> v1:172.21.15.46:6789/0 conn(0x7fefe8073970 legacy=0x7fefe80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 0x7fefcc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc004800 0x7fefcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefcc0051c0 0x7fefcc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8073170 con 0x7fefcc003a00 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc00e2d0 con 0x7fefcc0051c0 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.141+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc00e4d0 con 0x7fefcc004800 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 0x7fefcc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 -- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc004800 msgr2=0x7fefcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc004800 0x7fefcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 -- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefcc0051c0 msgr2=0x7fefcc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefcc0051c0 0x7fefcc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe5e00640 1 -- 172.21.15.46:0/2142616501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc003520 con 0x7fefcc003a00 2024-04-26T23:41:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.142+0000 7fefe6800640 1 -- 172.21.15.46:0/2142616501 reap_dead start 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.143+0000 7fefe5e00640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 0x7fefcc003e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009ac0 tx=0x7fefcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c8f5e119ecefc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.143+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8036070 con 0x7fefcc003a00 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.143+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd8031400 con 0x7fefcc003a00 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.143+0000 7fefe4a00640 1 -- 172.21.15.46:0/2142616501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8009690 con 0x7fefcc003a00 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 -- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 msgr2=0x7fefcc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 0x7fefcc003e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009ac0 tx=0x7fefcc005830 comp rx=0 tx=0).stop 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 -- 172.21.15.46:0/2142616501 shutdown_connections 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 --2- 172.21.15.46:0/2142616501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefcc003a00 0x7fefcc003e20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 -- 172.21.15.46:0/2142616501 >> 172.21.15.46:0/2142616501 conn(0x7fefe80b26d0 msgr2=0x7fefe80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 -- 172.21.15.46:0/2142616501 shutdown_connections 2024-04-26T23:41:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.144+0000 7fefe7200640 1 -- 172.21.15.46:0/2142616501 wait complete. 2024-04-26T23:41:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.147+0000 7fefe7200640 1 Processor -- start 2024-04-26T23:41:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.147+0000 7fefe7200640 1 -- start start 2024-04-26T23:41:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc003a00 0x7fefe8140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe815c2c0 0x7fefe815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8145880 con 0x7fefe815b6d0 2024-04-26T23:41:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8073e00 con 0x7fefe815c2c0 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefe8145a00 con 0x7fefcc003a00 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.148+0000 7fefe5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51948/0 (socket says 172.21.15.46:51948) 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 -- 172.21.15.46:0/524887748 learned_addr learned my addr 172.21.15.46:0/524887748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5e00640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc003a00 0x7fefe8140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 -- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc003a00 msgr2=0x7fefe8140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc003a00 0x7fefe8140260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 -- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe815c2c0 msgr2=0x7fefe815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe6800640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe815c2c0 0x7fefe815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe815c2c0 0x7fefe815c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 -- 172.21.15.46:0/524887748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe815b4e0 con 0x7fefe815b6d0 2024-04-26T23:41:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.149+0000 7fefe5400640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fefd400caa0 tx=0x7fefd400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.150+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4018070 con 0x7fefe815b6d0 2024-04-26T23:41:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.150+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd400a6d0 con 0x7fefe815b6d0 2024-04-26T23:41:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.150+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400aac0 con 0x7fefe815b6d0 2024-04-26T23:41:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.150+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe80b4e80 con 0x7fefe815b6d0 2024-04-26T23:41:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.150+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe8141b80 con 0x7fefe815b6d0 2024-04-26T23:41:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.151+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe80693b0 con 0x7fefe815b6d0 2024-04-26T23:41:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.152+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefd401b070 con 0x7fefe815b6d0 2024-04-26T23:41:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.153+0000 7fefde000640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 0x7fefc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.153+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fefd408b440 con 0x7fefe815b6d0 2024-04-26T23:41:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.154+0000 7fefe5e00640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 0x7fefc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.155+0000 7fefe5e00640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 0x7fefc806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefd8033000 tx=0x7fefd8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.157+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefd4059740 con 0x7fefe815b6d0 2024-04-26T23:41:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.377+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fefe80cefa0 con 0x7fefe815b6d0 2024-04-26T23:41:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.377+0000 7fefde000640 1 -- 172.21.15.46:0/524887748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7fefd4059040 con 0x7fefe815b6d0 2024-04-26T23:41:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:41:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 msgr2=0x7fefc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 0x7fefc806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefd8033000 tx=0x7fefd8008030 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 msgr2=0x7fefe815afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fefd400caa0 tx=0x7fefd400cf60 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 shutdown_connections 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fefc806c8f0 0x7fefc806edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe815c2c0 0x7fefe815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe815b6d0 0x7fefe815afa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 --2- 172.21.15.46:0/524887748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefcc003a00 0x7fefe8140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 >> 172.21.15.46:0/524887748 conn(0x7fefe80b26d0 msgr2=0x7fefe8142a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.380+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 shutdown_connections 2024-04-26T23:41:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.381+0000 7fefe7200640 1 -- 172.21.15.46:0/524887748 wait complete. 2024-04-26T23:41:38.401 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:41:38.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:38.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:41:38.523 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:41:38.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:41:38.523 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:41:38.582 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:41:38.582 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:41:38.582 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:38.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:41:38.622 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:41:38.622 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:41:38.622 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:38.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:41:38.661 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:41:38.662 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:41:38.662 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:38.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:41:38.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:41:38.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:38.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.909+0000 7fd181e00640 1 Processor -- start 2024-04-26T23:41:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.910+0000 7fd181e00640 1 -- start start 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd181e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd17c147ee0 0x7fd17c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd181e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd17c148540 0x7fd17c163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd181e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17c070b00 con 0x7fd17c06c350 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd181e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd17c163d40 con 0x7fd17c148540 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd181e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd17c162db0 con 0x7fd17c147ee0 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.911+0000 7fd173e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd17c148540 0x7fd17c163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd17b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd17c06c350 0x7fd17c073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35734/0 (socket says 172.21.15.46:35734) 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd173e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd17c148540 0x7fd17c163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51964/0 (socket says 172.21.15.46:51964) 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd17b400640 1 -- 172.21.15.46:0/913136585 learned_addr learned my addr 172.21.15.46:0/913136585 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd17aa00640 1 --2- 172.21.15.46:0/913136585 >> v2:172.21.15.142:3300/0 conn(0x7fd17c147ee0 0x7fd17c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd173e00640 1 -- 172.21.15.46:0/913136585 >> v2:172.21.15.142:3300/0 conn(0x7fd17c147ee0 msgr2=0x7fd17c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd173e00640 1 --2- 172.21.15.46:0/913136585 >> v2:172.21.15.142:3300/0 conn(0x7fd17c147ee0 0x7fd17c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd173e00640 1 -- 172.21.15.46:0/913136585 >> v1:172.21.15.46:6789/0 conn(0x7fd17c06c350 legacy=0x7fd17c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.912+0000 7fd173e00640 1 -- 172.21.15.46:0/913136585 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c070de0 con 0x7fd17c148540 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.913+0000 7fd173e00640 1 --2- 172.21.15.46:0/913136585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c163800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d2da0 tx=0x7fd16402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c74a96e6c843a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.913+0000 7fd17a000640 1 -- 172.21.15.46:0/913136585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd17c148540 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.913+0000 7fd17a000640 1 -- 172.21.15.46:0/913136585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164032440 con 0x7fd17c148540 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd17a000640 1 -- 172.21.15.46:0/913136585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164032810 con 0x7fd17c148540 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 -- 172.21.15.46:0/913136585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 msgr2=0x7fd17c163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 --2- 172.21.15.46:0/913136585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c163800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d2da0 tx=0x7fd16402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 -- 172.21.15.46:0/913136585 shutdown_connections 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 --2- 172.21.15.46:0/913136585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c163800 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 --2- 172.21.15.46:0/913136585 >> v2:172.21.15.142:3300/0 conn(0x7fd17c147ee0 0x7fd17c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 -- 172.21.15.46:0/913136585 >> 172.21.15.46:0/913136585 conn(0x7fd17c0d1e50 msgr2=0x7fd17c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 -- 172.21.15.46:0/913136585 shutdown_connections 2024-04-26T23:41:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.914+0000 7fd181e00640 1 -- 172.21.15.46:0/913136585 wait complete. 2024-04-26T23:41:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.916+0000 7fd181e00640 1 Processor -- start 2024-04-26T23:41:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.916+0000 7fd181e00640 1 -- start start 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd17c06c350 0x7fd17c0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd17c147ee0 0x7fd17c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1482c0 con 0x7fd17c148540 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c163370 con 0x7fd17c147ee0 2024-04-26T23:41:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.917+0000 7fd181e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c148c90 con 0x7fd17c06c350 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51978/0 (socket says 172.21.15.46:51978) 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17b400640 1 -- 172.21.15.46:0/4162946101 learned_addr learned my addr 172.21.15.46:0/4162946101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17aa00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd17c06c350 0x7fd17c0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd173e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd17c147ee0 0x7fd17c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17b400640 1 -- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd17c06c350 msgr2=0x7fd17c0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.918+0000 7fd17b400640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd17c06c350 0x7fd17c0b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd17b400640 1 -- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd17c147ee0 msgr2=0x7fd17c1547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd17b400640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd17c147ee0 0x7fd17c1547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd17b400640 1 -- 172.21.15.46:0/4162946101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c0b3f10 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd17b400640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd17400cab0 tx=0x7fd17400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174017070 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17400ac30 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174012400 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c0b41f0 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.919+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c0b47b0 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.920+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c069530 con 0x7fd17c148540 2024-04-26T23:41:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.922+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd17401a070 con 0x7fd17c148540 2024-04-26T23:41:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.922+0000 7fd173400640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.923+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fd17408b940 con 0x7fd17c148540 2024-04-26T23:41:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.923+0000 7fd17aa00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.924+0000 7fd17aa00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0079a0 tx=0x7fd16c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:38.928+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd174059550 con 0x7fd17c148540 2024-04-26T23:41:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.176+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd17c0700a0 con 0x7fd17c148540 2024-04-26T23:41:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.177+0000 7fd173400640 1 -- 172.21.15.46:0/4162946101 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7fd17405d850 con 0x7fd17c148540 2024-04-26T23:41:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:39.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:39:05.780519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:41:39.181 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 msgr2=0x7fd15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0079a0 tx=0x7fd16c008040 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 msgr2=0x7fd17c0b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd17400cab0 tx=0x7fd17400cf70 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 shutdown_connections 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd17c148540 0x7fd17c0b5880 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd17c147ee0 0x7fd17c1547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 --2- 172.21.15.46:0/4162946101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd17c06c350 0x7fd17c0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 >> 172.21.15.46:0/4162946101 conn(0x7fd17c0d1e50 msgr2=0x7fd17c15f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.179+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 shutdown_connections 2024-04-26T23:41:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.180+0000 7fd181e00640 1 -- 172.21.15.46:0/4162946101 wait complete. 2024-04-26T23:41:39.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.459+0000 7fa9b5800640 1 Processor -- start 2024-04-26T23:41:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.460+0000 7fa9b5800640 1 -- start start 2024-04-26T23:41:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.461+0000 7fa9b5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9b01466b0 0x7fa9b006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.461+0000 7fa9b5800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 0x7fa9b0160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.461+0000 7fa9b5800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b0073ad0 con 0x7fa9b00720e0 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.461+0000 7fa9b5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b01612a0 con 0x7fa9b01466b0 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.461+0000 7fa9b5800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b0146a90 con 0x7fa9b0160910 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9b01466b0 0x7fa9b006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9af400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9b00720e0 0x7fa9b00cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35746/0 (socket says 172.21.15.46:35746) 2024-04-26T23:41:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9af400640 1 -- 172.21.15.46:0/466607556 learned_addr learned my addr 172.21.15.46:0/466607556 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9b4e00640 1 --2- 172.21.15.46:0/466607556 >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 0x7fa9b0160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9afe00640 1 -- 172.21.15.46:0/466607556 >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 msgr2=0x7fa9b0160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9afe00640 1 --2- 172.21.15.46:0/466607556 >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 0x7fa9b0160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9afe00640 1 -- 172.21.15.46:0/466607556 >> v1:172.21.15.46:6789/0 conn(0x7fa9b00720e0 legacy=0x7fa9b00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9afe00640 1 -- 172.21.15.46:0/466607556 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b0073d70 con 0x7fa9b01466b0 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9aea00640 1 -- 172.21.15.46:0/466607556 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925358794 0 0) 0x7fa9b0073ad0 con 0x7fa9b00720e0 2024-04-26T23:41:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.462+0000 7fa9b4e00640 1 --2- 172.21.15.46:0/466607556 >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 0x7fa9b0160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.463+0000 7fa9afe00640 1 --2- 172.21.15.46:0/466607556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b01466b0 0x7fa9b006de80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00097a0 tx=0x7fa9a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dc89e4c0527283 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9aea00640 1 -- 172.21.15.46:0/466607556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0037070 con 0x7fa9b01466b0 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9aea00640 1 -- 172.21.15.46:0/466607556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a0032440 con 0x7fa9b01466b0 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9aea00640 1 -- 172.21.15.46:0/466607556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0032810 con 0x7fa9b01466b0 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 -- 172.21.15.46:0/466607556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b01466b0 msgr2=0x7fa9b006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 --2- 172.21.15.46:0/466607556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b01466b0 0x7fa9b006de80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00097a0 tx=0x7fa9a002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 -- 172.21.15.46:0/466607556 shutdown_connections 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 --2- 172.21.15.46:0/466607556 >> v2:172.21.15.142:3300/0 conn(0x7fa9b0160910 0x7fa9b0160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 --2- 172.21.15.46:0/466607556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b01466b0 0x7fa9b006de80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.464+0000 7fa9b5800640 1 -- 172.21.15.46:0/466607556 >> 172.21.15.46:0/466607556 conn(0x7fa9b00b26d0 msgr2=0x7fa9b00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.465+0000 7fa9b5800640 1 -- 172.21.15.46:0/466607556 shutdown_connections 2024-04-26T23:41:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.465+0000 7fa9b5800640 1 -- 172.21.15.46:0/466607556 wait complete. 2024-04-26T23:41:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.466+0000 7fa9b5800640 1 Processor -- start 2024-04-26T23:41:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.467+0000 7fa9b5800640 1 -- start start 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.467+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b00720e0 0x7fa9b013f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b01466b0 0x7fa9b00b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b00cb470 con 0x7fa9b0160910 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b5800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b00cb190 con 0x7fa9b00720e0 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b5800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b006e3c0 con 0x7fa9b01466b0 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51992/0 (socket says 172.21.15.46:51992) 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 -- 172.21.15.46:0/2377313197 learned_addr learned my addr 172.21.15.46:0/2377313197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9afe00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b00720e0 0x7fa9b013f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9af400640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b01466b0 0x7fa9b00b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 -- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b01466b0 msgr2=0x7fa9b00b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b01466b0 0x7fa9b00b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 -- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b00720e0 msgr2=0x7fa9b013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b00720e0 0x7fa9b013f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.468+0000 7fa9b4e00640 1 -- 172.21.15.46:0/2377313197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b00ce670 con 0x7fa9b0160910 2024-04-26T23:41:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9b4e00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800cd80 tx=0x7fa9a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800f680 con 0x7fa9b0160910 2024-04-26T23:41:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a800f840 con 0x7fa9b0160910 2024-04-26T23:41:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800fc30 con 0x7fa9b0160910 2024-04-26T23:41:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b00ce950 con 0x7fa9b0160910 2024-04-26T23:41:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.469+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b00cef10 con 0x7fa9b0160910 2024-04-26T23:41:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.470+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b00693b0 con 0x7fa9b0160910 2024-04-26T23:41:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.471+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9a80213f0 con 0x7fa9b0160910 2024-04-26T23:41:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.472+0000 7fa9a7e00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 0x7fa98c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.473+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fa9a808cbb0 con 0x7fa9b0160910 2024-04-26T23:41:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.473+0000 7fa9afe00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 0x7fa98c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.474+0000 7fa9afe00640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 0x7fa98c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0034000 tx=0x7fa9a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.478+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9a8014030 con 0x7fa9b0160910 2024-04-26T23:41:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.681+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9b0076430 con 0x7fa9b0160910 2024-04-26T23:41:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.684+0000 7fa9a7e00640 1 -- 172.21.15.46:0/2377313197 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17020 (secure 0 0 0) 0x7fa9a805a730 con 0x7fa9b0160910 2024-04-26T23:41:39.689 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:39.689 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:13.670571+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.687+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 msgr2=0x7fa98c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.687+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 0x7fa98c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0034000 tx=0x7fa9a002e000 comp rx=0 tx=0).stop 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.687+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 msgr2=0x7fa9b00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.687+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800cd80 tx=0x7fa9a800a650 comp rx=0 tx=0).stop 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 shutdown_connections 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa98c06c8f0 0x7fa98c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b0160910 0x7fa9b00b61c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b01466b0 0x7fa9b00b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 --2- 172.21.15.46:0/2377313197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b00720e0 0x7fa9b013f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 >> 172.21.15.46:0/2377313197 conn(0x7fa9b00b26d0 msgr2=0x7fa9b00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:39.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 shutdown_connections 2024-04-26T23:41:39.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.688+0000 7fa9b5800640 1 -- 172.21.15.46:0/2377313197 wait complete. 2024-04-26T23:41:39.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:38:49.046203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:38:50.124386+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:41:39.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:41:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.926+0000 7fa847200640 1 Processor -- start 2024-04-26T23:41:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.926+0000 7fa847200640 1 -- start start 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.927+0000 7fa847200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa8480cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.927+0000 7fa847200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa848073d00 0x7fa8480b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.927+0000 7fa847200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8480ce7d0 con 0x7fa8480cdb30 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.927+0000 7fa847200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa8480cdf90 con 0x7fa848073d00 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.927+0000 7fa847200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480ceab0 con 0x7fa84815cec0 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa846800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa848073d00 0x7fa8480b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa845400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa8480cdb30 0x7fa8480735f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35750/0 (socket says 172.21.15.46:35750) 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa845400640 1 -- 172.21.15.46:0/2228084929 learned_addr learned my addr 172.21.15.46:0/2228084929 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa845e00640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa8480cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa846800640 1 -- 172.21.15.46:0/2228084929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 msgr2=0x7fa8480cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.928+0000 7fa846800640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa8480cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.929+0000 7fa846800640 1 -- 172.21.15.46:0/2228084929 >> v1:172.21.15.46:6789/0 conn(0x7fa8480cdb30 legacy=0x7fa8480735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.929+0000 7fa846800640 1 -- 172.21.15.46:0/2228084929 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8480ced90 con 0x7fa848073d00 2024-04-26T23:41:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.929+0000 7fa844a00640 1 -- 172.21.15.46:0/2228084929 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403479544 0 0) 0x7fa8480ce7d0 con 0x7fa8480cdb30 2024-04-26T23:41:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.930+0000 7fa846800640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa848073d00 0x7fa8480b3b40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99adc7de59c0897 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.930+0000 7fa844a00640 1 -- 172.21.15.46:0/2228084929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa848073d00 2024-04-26T23:41:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.930+0000 7fa844a00640 1 -- 172.21.15.46:0/2228084929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840032440 con 0x7fa848073d00 2024-04-26T23:41:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.930+0000 7fa844a00640 1 -- 172.21.15.46:0/2228084929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840032810 con 0x7fa848073d00 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 -- 172.21.15.46:0/2228084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa848073d00 msgr2=0x7fa8480b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa848073d00 0x7fa8480b3b40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a180 comp rx=0 tx=0).stop 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 -- 172.21.15.46:0/2228084929 shutdown_connections 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa848073d00 0x7fa8480b3b40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 --2- 172.21.15.46:0/2228084929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa8480cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 -- 172.21.15.46:0/2228084929 >> 172.21.15.46:0/2228084929 conn(0x7fa8480b27b0 msgr2=0x7fa8480b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 -- 172.21.15.46:0/2228084929 shutdown_connections 2024-04-26T23:41:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.931+0000 7fa847200640 1 -- 172.21.15.46:0/2228084929 wait complete. 2024-04-26T23:41:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.933+0000 7fa847200640 1 Processor -- start 2024-04-26T23:41:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.933+0000 7fa847200640 1 -- start start 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa848073d00 0x7fa84814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa84814e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa84815d2a0 con 0x7fa8480cdb30 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8480cf350 con 0x7fa848073d00 2024-04-26T23:41:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa847200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480b4080 con 0x7fa84815cec0 2024-04-26T23:41:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa845400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa845400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52008/0 (socket says 172.21.15.46:52008) 2024-04-26T23:41:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.934+0000 7fa845400640 1 -- 172.21.15.46:0/308662314 learned_addr learned my addr 172.21.15.46:0/308662314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845e00640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa848073d00 0x7fa84814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa846800640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa84814e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 -- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 msgr2=0x7fa84814e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa84814e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 -- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa848073d00 msgr2=0x7fa84814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa848073d00 0x7fa84814b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 -- 172.21.15.46:0/308662314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84814bca0 con 0x7fa8480cdb30 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa846800640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa84814e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa845400640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c4a0 tx=0x7fa83000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830017070 con 0x7fa8480cdb30 2024-04-26T23:41:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa830005ce0 con 0x7fa8480cdb30 2024-04-26T23:41:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300072b0 con 0x7fa8480cdb30 2024-04-26T23:41:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.935+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8480c8c30 con 0x7fa8480cdb30 2024-04-26T23:41:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.936+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8480c91f0 con 0x7fa8480cdb30 2024-04-26T23:41:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.937+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8480693b0 con 0x7fa8480cdb30 2024-04-26T23:41:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.938+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa83001a070 con 0x7fa8480cdb30 2024-04-26T23:41:39.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.939+0000 7fa83e000640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:39.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.939+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fa83008b040 con 0x7fa8480cdb30 2024-04-26T23:41:39.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.939+0000 7fa845e00640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.941+0000 7fa845e00640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 0x7fa82406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa8480ca280 tx=0x7fa83800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:39.944+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa830059340 con 0x7fa8480cdb30 2024-04-26T23:41:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.147+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8480cd710 con 0x7fa8480cdb30 2024-04-26T23:41:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.149+0000 7fa83e000640 1 -- 172.21.15.46:0/308662314 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7fa830058c40 con 0x7fa8480cdb30 2024-04-26T23:41:40.152 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:41:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 msgr2=0x7fa82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 0x7fa82406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa8480ca280 tx=0x7fa83800b040 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 msgr2=0x7fa84814e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c4a0 tx=0x7fa83000c960 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 shutdown_connections 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa84815cec0 0x7fa84814e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8480cdb30 0x7fa84814e020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 --2- 172.21.15.46:0/308662314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa848073d00 0x7fa84814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 >> 172.21.15.46:0/308662314 conn(0x7fa8480b27b0 msgr2=0x7fa848075a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.151+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 shutdown_connections 2024-04-26T23:41:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.152+0000 7fa847200640 1 -- 172.21.15.46:0/308662314 wait complete. 2024-04-26T23:41:40.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:41:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.388+0000 7f1f13e00640 1 Processor -- start 2024-04-26T23:41:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.388+0000 7f1f13e00640 1 -- start start 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.389+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.389+0000 7f1f13e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f140ce7d0 0x7f1f140cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.389+0000 7f1f13e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f141649f0 con 0x7f1f14073af0 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.389+0000 7f1f13e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1f140cb4f0 con 0x7f1f140ce7d0 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.389+0000 7f1f13e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f14071e80 con 0x7f1f141640d0 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.390+0000 7f1f12a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.390+0000 7f1f12a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:45386/0 (socket says 172.21.15.46:45386) 2024-04-26T23:41:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.390+0000 7f1f12a00640 1 -- 172.21.15.46:0/1085642525 learned_addr learned my addr 172.21.15.46:0/1085642525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.390+0000 7f1f13400640 1 --2- 172.21.15.46:0/1085642525 >> v2:172.21.15.142:3300/0 conn(0x7f1f140ce7d0 0x7f1f140cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.391+0000 7f1f12a00640 1 -- 172.21.15.46:0/1085642525 >> v2:172.21.15.142:3300/0 conn(0x7f1f140ce7d0 msgr2=0x7f1f140cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.391+0000 7f1f12a00640 1 --2- 172.21.15.46:0/1085642525 >> v2:172.21.15.142:3300/0 conn(0x7f1f140ce7d0 0x7f1f140cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.391+0000 7f1f12a00640 1 -- 172.21.15.46:0/1085642525 >> v1:172.21.15.142:6789/0 conn(0x7f1f14073af0 legacy=0x7f1f140cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.391+0000 7f1f12a00640 1 -- 172.21.15.46:0/1085642525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f14072160 con 0x7f1f141640d0 2024-04-26T23:41:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.391+0000 7f1f11600640 1 -- 172.21.15.46:0/1085642525 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309756999 0 0) 0x7f1f141649f0 con 0x7f1f14073af0 2024-04-26T23:41:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.393+0000 7f1f12a00640 1 --2- 172.21.15.46:0/1085642525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009870 tx=0x7f1f0402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96dfe5ed2f636575 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.393+0000 7f1f11600640 1 -- 172.21.15.46:0/1085642525 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04037070 con 0x7f1f141640d0 2024-04-26T23:41:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.393+0000 7f1f11600640 1 -- 172.21.15.46:0/1085642525 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04032440 con 0x7f1f141640d0 2024-04-26T23:41:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.393+0000 7f1f11600640 1 -- 172.21.15.46:0/1085642525 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04032810 con 0x7f1f141640d0 2024-04-26T23:41:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 -- 172.21.15.46:0/1085642525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 msgr2=0x7f1f141644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 --2- 172.21.15.46:0/1085642525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009870 tx=0x7f1f0402a250 comp rx=0 tx=0).stop 2024-04-26T23:41:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 -- 172.21.15.46:0/1085642525 shutdown_connections 2024-04-26T23:41:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 --2- 172.21.15.46:0/1085642525 >> v2:172.21.15.142:3300/0 conn(0x7f1f140ce7d0 0x7f1f140cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 --2- 172.21.15.46:0/1085642525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f141640d0 0x7f1f141644b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 -- 172.21.15.46:0/1085642525 >> 172.21.15.46:0/1085642525 conn(0x7f1f140b27b0 msgr2=0x7f1f140b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 -- 172.21.15.46:0/1085642525 shutdown_connections 2024-04-26T23:41:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.394+0000 7f1f13e00640 1 -- 172.21.15.46:0/1085642525 wait complete. 2024-04-26T23:41:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.396+0000 7f1f13e00640 1 Processor -- start 2024-04-26T23:41:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.396+0000 7f1f13e00640 1 -- start start 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f140ce7d0 0x7f1f140b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 0x7f1f1415c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140cc090 con 0x7f1f14073af0 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14073f70 con 0x7f1f141640d0 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f13e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f140cc210 con 0x7f1f140ce7d0 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52020/0 (socket says 172.21.15.46:52020) 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.397+0000 7f1f12a00640 1 -- 172.21.15.46:0/142843573 learned_addr learned my addr 172.21.15.46:0/142843573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f13400640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 0x7f1f1415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12000640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f140ce7d0 0x7f1f140b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 -- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f140ce7d0 msgr2=0x7f1f140b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f140ce7d0 0x7f1f140b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 -- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 msgr2=0x7f1f1415c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 0x7f1f1415c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 -- 172.21.15.46:0/142843573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f140c9fd0 con 0x7f1f14073af0 2024-04-26T23:41:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f13400640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 0x7f1f1415c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f12a00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1f04000c00 tx=0x7f1f040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.398+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04037040 con 0x7f1f14073af0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.399+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04042070 con 0x7f1f14073af0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.399+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04030940 con 0x7f1f14073af0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.399+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f140ca2b0 con 0x7f1f14073af0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.399+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1415cc70 con 0x7f1f14073af0 2024-04-26T23:41:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.400+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f140693b0 con 0x7f1f14073af0 2024-04-26T23:41:40.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.401+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f04030ae0 con 0x7f1f14073af0 2024-04-26T23:41:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.402+0000 7f1f0aa00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 0x7f1ef006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.402+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f1f040ac5c0 con 0x7f1f14073af0 2024-04-26T23:41:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.402+0000 7f1f12000640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 0x7f1ef006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.403+0000 7f1f12000640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 0x7f1ef006ede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f140ca880 tx=0x7f1efc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.406+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f0407a790 con 0x7f1f14073af0 2024-04-26T23:41:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.612+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f140cc5a0 con 0x7f1f14073af0 2024-04-26T23:41:40.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.613+0000 7f1f0aa00640 1 -- 172.21.15.46:0/142843573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) ==== 104+0+11 (secure 0 0 0) 0x7f1f0407a090 con 0x7f1f14073af0 2024-04-26T23:41:40.617 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 msgr2=0x7f1ef006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 0x7f1ef006ede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f140ca880 tx=0x7f1efc009040 comp rx=0 tx=0).stop 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 msgr2=0x7f1f140c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1f04000c00 tx=0x7f1f040046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 shutdown_connections 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ef006c920 0x7f1ef006ede0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f141640d0 0x7f1f1415c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f140ce7d0 0x7f1f140b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.616+0000 7f1f13e00640 1 --2- 172.21.15.46:0/142843573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f14073af0 0x7f1f140c7160 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.617+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 >> 172.21.15.46:0/142843573 conn(0x7f1f140b27b0 msgr2=0x7f1f140b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.617+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 shutdown_connections 2024-04-26T23:41:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.617+0000 7f1f13e00640 1 -- 172.21.15.46:0/142843573 wait complete. 2024-04-26T23:41:40.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:41:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.860+0000 7fc4ed800640 1 Processor -- start 2024-04-26T23:41:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.861+0000 7fc4ed800640 1 -- start start 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.862+0000 7fc4ed800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e815f840 0x7fc4e80d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.862+0000 7fc4ed800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 0x7fc4e80b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.862+0000 7fc4ed800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e80d4090 con 0x7fc4e80d1980 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.862+0000 7fc4ed800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e80d3880 con 0x7fc4e815f840 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.862+0000 7fc4ed800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80b4020 con 0x7fc4e80d24f0 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e815f840 0x7fc4e80d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e815f840 0x7fc4e80d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52022/0 (socket says 172.21.15.46:52022) 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 -- 172.21.15.46:0/4218081863 learned_addr learned my addr 172.21.15.46:0/4218081863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4ece00640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 0x7fc4e80b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 -- 172.21.15.46:0/4218081863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 msgr2=0x7fc4e80b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 0x7fc4e80b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 -- 172.21.15.46:0/4218081863 >> v1:172.21.15.142:6789/0 conn(0x7fc4e80d1980 legacy=0x7fc4e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4e7e00640 1 -- 172.21.15.46:0/4218081863 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e80d3ad0 con 0x7fc4e815f840 2024-04-26T23:41:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.863+0000 7fc4ece00640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 0x7fc4e80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.864+0000 7fc4e7e00640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e815f840 0x7fc4e80d2e70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009870 tx=0x7fc4d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f7f60490bfccf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.864+0000 7fc4e6a00640 1 -- 172.21.15.46:0/4218081863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8037070 con 0x7fc4e815f840 2024-04-26T23:41:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4e6a00640 1 -- 172.21.15.46:0/4218081863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d8032440 con 0x7fc4e815f840 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4e6a00640 1 -- 172.21.15.46:0/4218081863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8032810 con 0x7fc4e815f840 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 -- 172.21.15.46:0/4218081863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e815f840 msgr2=0x7fc4e80d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e815f840 0x7fc4e80d2e70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009870 tx=0x7fc4d802a210 comp rx=0 tx=0).stop 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 -- 172.21.15.46:0/4218081863 shutdown_connections 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d24f0 0x7fc4e80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 --2- 172.21.15.46:0/4218081863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e815f840 0x7fc4e80d2e70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 -- 172.21.15.46:0/4218081863 >> 172.21.15.46:0/4218081863 conn(0x7fc4e80d4f70 msgr2=0x7fc4e80d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 -- 172.21.15.46:0/4218081863 shutdown_connections 2024-04-26T23:41:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.865+0000 7fc4ed800640 1 -- 172.21.15.46:0/4218081863 wait complete. 2024-04-26T23:41:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.867+0000 7fc4ed800640 1 Processor -- start 2024-04-26T23:41:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.867+0000 7fc4ed800640 1 -- start start 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d1980 0x7fc4e80b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e815f840 0x7fc4e81ec780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80d33b0 con 0x7fc4e80d24f0 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80b27d0 con 0x7fc4e815f840 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.868+0000 7fc4ed800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80b3230 con 0x7fc4e80d1980 2024-04-26T23:41:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52034/0 (socket says 172.21.15.46:52034) 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 -- 172.21.15.46:0/1820859438 learned_addr learned my addr 172.21.15.46:0/1820859438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7e00640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d1980 0x7fc4e80b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4ece00640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e815f840 0x7fc4e81ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 -- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d1980 msgr2=0x7fc4e80b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d1980 0x7fc4e80b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 -- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e815f840 msgr2=0x7fc4e81ec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e815f840 0x7fc4e81ec780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.869+0000 7fc4e7400640 1 -- 172.21.15.46:0/1820859438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e80b7d70 con 0x7fc4e80d24f0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4e7400640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c460 tx=0x7fc4d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4016020 con 0x7fc4e80d24f0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4012870 con 0x7fc4e80d24f0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4012c60 con 0x7fc4e80d24f0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e80ba7f0 con 0x7fc4e80d24f0 2024-04-26T23:41:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.870+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e80b8050 con 0x7fc4e80d24f0 2024-04-26T23:41:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.871+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e8162f10 con 0x7fc4e80d24f0 2024-04-26T23:41:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.872+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4d4019020 con 0x7fc4e80d24f0 2024-04-26T23:41:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.873+0000 7fc4dfe00640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.873+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fc4d408b7b0 con 0x7fc4e80d24f0 2024-04-26T23:41:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.874+0000 7fc4e7e00640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:40.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.875+0000 7fc4e7e00640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 0x7fc4c806ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8034000 tx=0x7fc4d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:40.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:40.878+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4e8162f10 con 0x7fc4e80d24f0 2024-04-26T23:41:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.082+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4e80693b0 con 0x7fc4e80d24f0 2024-04-26T23:41:41.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.083+0000 7fc4dfe00640 1 -- 172.21.15.46:0/1820859438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) ==== 100+0+11 (secure 0 0 0) 0x7fc4d4020070 con 0x7fc4e80d24f0 2024-04-26T23:41:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.085+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 msgr2=0x7fc4c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.085+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 0x7fc4c806ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8034000 tx=0x7fc4d802e000 comp rx=0 tx=0).stop 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.085+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 msgr2=0x7fc4e80ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.085+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c460 tx=0x7fc4d400c920 comp rx=0 tx=0).stop 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 shutdown_connections 2024-04-26T23:41:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e815f840 0x7fc4e81ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80d24f0 0x7fc4e80ced10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 --2- 172.21.15.46:0/1820859438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d1980 0x7fc4e80b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 >> 172.21.15.46:0/1820859438 conn(0x7fc4e80d4f70 msgr2=0x7fc4e815d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 shutdown_connections 2024-04-26T23:41:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.086+0000 7fc4ed800640 1 -- 172.21.15.46:0/1820859438 wait complete. 2024-04-26T23:41:41.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:41:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.318+0000 7fb10f200640 1 Processor -- start 2024-04-26T23:41:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.319+0000 7fb10f200640 1 -- start start 2024-04-26T23:41:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.320+0000 7fb10f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb108162980 0x7fb1080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.320+0000 7fb10f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1080d2200 0x7fb10815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.320+0000 7fb10f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10815fd30 con 0x7fb1080d31f0 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.320+0000 7fb10f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb108162d60 con 0x7fb1080d2200 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.320+0000 7fb10f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb1080d1050 con 0x7fb108162980 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1080d2200 0x7fb10815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1080d2200 0x7fb10815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52048/0 (socket says 172.21.15.46:52048) 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb107400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb1080d31f0 0x7fb1080d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35764/0 (socket says 172.21.15.46:35764) 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 -- 172.21.15.46:0/3844271779 learned_addr learned my addr 172.21.15.46:0/3844271779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb107e00640 1 --2- 172.21.15.46:0/3844271779 >> v2:172.21.15.142:3300/0 conn(0x7fb108162980 0x7fb1080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 -- 172.21.15.46:0/3844271779 >> v2:172.21.15.142:3300/0 conn(0x7fb108162980 msgr2=0x7fb1080d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 --2- 172.21.15.46:0/3844271779 >> v2:172.21.15.142:3300/0 conn(0x7fb108162980 0x7fb1080d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 -- 172.21.15.46:0/3844271779 >> v1:172.21.15.46:6789/0 conn(0x7fb1080d31f0 legacy=0x7fb1080d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.321+0000 7fb10d200640 1 -- 172.21.15.46:0/3844271779 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb108160010 con 0x7fb1080d2200 2024-04-26T23:41:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.322+0000 7fb10d200640 1 --2- 172.21.15.46:0/3844271779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb10815ed30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb10808f2b0 tx=0x7fb10002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28bf8c47cc5dec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.323+0000 7fb106a00640 1 -- 172.21.15.46:0/3844271779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100036070 con 0x7fb1080d2200 2024-04-26T23:41:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.323+0000 7fb106a00640 1 -- 172.21.15.46:0/3844271779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100031600 con 0x7fb1080d2200 2024-04-26T23:41:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.323+0000 7fb106a00640 1 -- 172.21.15.46:0/3844271779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000319d0 con 0x7fb1080d2200 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.323+0000 7fb10f200640 1 -- 172.21.15.46:0/3844271779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 msgr2=0x7fb10815ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.323+0000 7fb10f200640 1 --2- 172.21.15.46:0/3844271779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb10815ed30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb10808f2b0 tx=0x7fb10002a730 comp rx=0 tx=0).stop 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 -- 172.21.15.46:0/3844271779 shutdown_connections 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 --2- 172.21.15.46:0/3844271779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb10815ed30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 --2- 172.21.15.46:0/3844271779 >> v2:172.21.15.142:3300/0 conn(0x7fb108162980 0x7fb1080d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 -- 172.21.15.46:0/3844271779 >> 172.21.15.46:0/3844271779 conn(0x7fb10809e420 msgr2=0x7fb1080d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 -- 172.21.15.46:0/3844271779 shutdown_connections 2024-04-26T23:41:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.324+0000 7fb10f200640 1 -- 172.21.15.46:0/3844271779 wait complete. 2024-04-26T23:41:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.326+0000 7fb10f200640 1 Processor -- start 2024-04-26T23:41:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.326+0000 7fb10f200640 1 -- start start 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1080d31f0 0x7fb108152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb108162980 0x7fb10806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081608b0 con 0x7fb1080d2200 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081605d0 con 0x7fb108162980 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.327+0000 7fb10f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb108160d50 con 0x7fb1080d31f0 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52050/0 (socket says 172.21.15.46:52050) 2024-04-26T23:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 -- 172.21.15.46:0/1857380142 learned_addr learned my addr 172.21.15.46:0/1857380142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 -- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1080d31f0 msgr2=0x7fb108152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1080d31f0 0x7fb108152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 -- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb108162980 msgr2=0x7fb10806d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb108162980 0x7fb10806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 -- 172.21.15.46:0/1857380142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1081f9de0 con 0x7fb1080d2200 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.328+0000 7fb107e00640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d800 tx=0x7fb0f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.329+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800b6c0 con 0x7fb1080d2200 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.329+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8019070 con 0x7fb1080d2200 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.329+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1081f9820 con 0x7fb1080d2200 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.329+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800ba80 con 0x7fb1080d2200 2024-04-26T23:41:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.329+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb108151d70 con 0x7fb1080d2200 2024-04-26T23:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.330+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb108069530 con 0x7fb1080d2200 2024-04-26T23:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.331+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0f800bc20 con 0x7fb1080d2200 2024-04-26T23:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.332+0000 7fb0ffe00640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 0x7fb0e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.332+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fb0f808c240 con 0x7fb1080d2200 2024-04-26T23:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.333+0000 7fb107400640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 0x7fb0e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.334+0000 7fb107400640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 0x7fb0e406ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000f60 tx=0x7fb0f0007a00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.336+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0f805a410 con 0x7fb1080d2200 2024-04-26T23:41:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.588+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1080d15f0 con 0x7fb1080d2200 2024-04-26T23:41:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.589+0000 7fb0ffe00640 1 -- 172.21.15.46:0/1857380142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7fb0f8059d10 con 0x7fb1080d2200 2024-04-26T23:41:41.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:41.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:39:05.780519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:41:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2024-04-26T23:41:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 msgr2=0x7fb0e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 0x7fb0e406ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000f60 tx=0x7fb0f0007a00 comp rx=0 tx=0).stop 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 msgr2=0x7fb1081f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d800 tx=0x7fb0f800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 shutdown_connections 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e406c920 0x7fb0e406ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb108162980 0x7fb10806d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.591+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1080d31f0 0x7fb108152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.592+0000 7fb10f200640 1 --2- 172.21.15.46:0/1857380142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1080d2200 0x7fb1081f8f20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.592+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 >> 172.21.15.46:0/1857380142 conn(0x7fb10809e420 msgr2=0x7fb10806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.592+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 shutdown_connections 2024-04-26T23:41:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.592+0000 7fb10f200640 1 -- 172.21.15.46:0/1857380142 wait complete. 2024-04-26T23:41:41.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.820+0000 7f34d2000640 1 Processor -- start 2024-04-26T23:41:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.820+0000 7f34d2000640 1 -- start start 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.822+0000 7f34d2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34cc160bd0 0x7f34cc161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.822+0000 7f34d2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34cc073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.822+0000 7f34d2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc161940 con 0x7f34cc06e510 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.822+0000 7f34d2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc071f80 con 0x7f34cc160bd0 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.822+0000 7f34d2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc072260 con 0x7f34cc0732c0 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34cb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f34cc06e510 0x7f34cc06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35772/0 (socket says 172.21.15.46:35772) 2024-04-26T23:41:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34cb400640 1 -- 172.21.15.46:0/4043641178 learned_addr learned my addr 172.21.15.46:0/4043641178 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34c3e00640 1 --2- 172.21.15.46:0/4043641178 >> v2:172.21.15.142:3300/0 conn(0x7f34cc160bd0 0x7f34cc161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34cbe00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34cc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977777557 0 0) 0x7f34cc161940 con 0x7f34cc06e510 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.823+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b0003520 con 0x7f34cc06e510 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071841794 0 0) 0x7f34b0003520 con 0x7f34cc06e510 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 msgr2=0x7f34cc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34cc073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 >> v2:172.21.15.142:3300/0 conn(0x7f34cc160bd0 msgr2=0x7f34cc161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 --2- 172.21.15.46:0/4043641178 >> v2:172.21.15.142:3300/0 conn(0x7f34cc160bd0 0x7f34cc161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc072540 con 0x7f34cc06e510 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f34bc001670 con 0x7f34cc06e510 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 >> v1:172.21.15.46:6789/0 conn(0x7f34cc06e510 legacy=0x7f34cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34b00039b0 0x7f34b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc071f80 con 0x7f34b0004d40 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc072260 con 0x7f34b00039b0 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34b000b9c0 con 0x7f34cc0732c0 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34cbe00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34c3e00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.824+0000 7f34cb400640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34b00039b0 0x7f34b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.825+0000 7f34cbe00640 1 -- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 msgr2=0x7f34b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:41.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.825+0000 7f34cbe00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc0732c0 0x7f34b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.825+0000 7f34cbe00640 1 -- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34b00039b0 msgr2=0x7f34b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:41.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.825+0000 7f34cbe00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34b00039b0 0x7f34b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.825+0000 7f34cbe00640 1 -- 172.21.15.46:0/4043641178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0003520 con 0x7f34b0004d40 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.826+0000 7f34cbe00640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34b00051c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fc0 tx=0x7f34b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45df331a69aca36e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.826+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4009c10 con 0x7f34b0004d40 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.826+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c4036070 con 0x7f34b0004d40 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.826+0000 7f34caa00640 1 -- 172.21.15.46:0/4043641178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4031880 con 0x7f34b0004d40 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 -- 172.21.15.46:0/4043641178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 msgr2=0x7f34b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 --2- 172.21.15.46:0/4043641178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34b00051c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fc0 tx=0x7f34b0005830 comp rx=0 tx=0).stop 2024-04-26T23:41:41.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34cbe00640 1 -- 172.21.15.46:0/4043641178 reap_dead start 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 -- 172.21.15.46:0/4043641178 shutdown_connections 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 -- 172.21.15.46:0/4043641178 >> 172.21.15.46:0/4043641178 conn(0x7f34cc06c530 msgr2=0x7f34cc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 -- 172.21.15.46:0/4043641178 shutdown_connections 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.827+0000 7f34d2000640 1 -- 172.21.15.46:0/4043641178 wait complete. 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.829+0000 7f34d2000640 1 Processor -- start 2024-04-26T23:41:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.829+0000 7f34d2000640 1 -- start start 2024-04-26T23:41:41.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 0x7f34cc145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 0x7f34cc145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc161720 con 0x7f34b0004d40 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1615a0 con 0x7f34cc13aea0 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34d2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc072d40 con 0x7f34cc13b320 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52056/0 (socket says 172.21.15.46:52056) 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34cb400640 1 -- 172.21.15.46:0/260645291 learned_addr learned my addr 172.21.15.46:0/260645291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34c3e00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 0x7f34cc145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34cbe00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 0x7f34cc145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.830+0000 7f34cb400640 1 -- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 msgr2=0x7f34cc145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cb400640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 0x7f34cc145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cb400640 1 -- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 msgr2=0x7f34cc145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cb400640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 0x7f34cc145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:41.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cb400640 1 -- 172.21.15.46:0/260645291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc13b8b0 con 0x7f34b0004d40 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34c3e00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 0x7f34cc145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cbe00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 0x7f34cc145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34cb400640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00e760 tx=0x7f34bc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00ee70 con 0x7f34b0004d40 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc018070 con 0x7f34b0004d40 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00cb10 con 0x7f34b0004d40 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc13bb90 con 0x7f34b0004d40 2024-04-26T23:41:41.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.831+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc143000 con 0x7f34b0004d40 2024-04-26T23:41:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.832+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc069530 con 0x7f34b0004d40 2024-04-26T23:41:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.834+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34bc00ccb0 con 0x7f34b0004d40 2024-04-26T23:41:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.835+0000 7f34c3400640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 0x7f34a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.835+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f34bc08bbc0 con 0x7f34b0004d40 2024-04-26T23:41:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.836+0000 7f34c3e00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 0x7f34a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:41.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.837+0000 7f34c3e00640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 0x7f34a806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34cc1f8480 tx=0x7f34b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:41.840+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34bc059e20 con 0x7f34b0004d40 2024-04-26T23:41:42.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.048+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34cc069b60 con 0x7f34b0004d40 2024-04-26T23:41:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.051+0000 7f34c3400640 1 -- 172.21.15.46:0/260645291 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17020 (secure 0 0 0) 0x7f34bc059720 con 0x7f34b0004d40 2024-04-26T23:41:42.054 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:42.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:13.670571+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.053+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 msgr2=0x7f34a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 0x7f34a806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34cc1f8480 tx=0x7f34b4008040 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 msgr2=0x7f34cc139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00e760 tx=0x7f34bc00ec20 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 shutdown_connections 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f34a806c8f0 0x7f34a806edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34cc13b320 0x7f34cc145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34cc13aea0 0x7f34cc145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 --2- 172.21.15.46:0/260645291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0004d40 0x7f34cc139640 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 >> 172.21.15.46:0/260645291 conn(0x7f34cc06c530 msgr2=0x7f34cc141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 shutdown_connections 2024-04-26T23:41:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.054+0000 7f34d2000640 1 -- 172.21.15.46:0/260645291 wait complete. 2024-04-26T23:41:42.077 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:41:42.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:41:42.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.272+0000 7f2e38a00640 1 Processor -- start 2024-04-26T23:41:42.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.272+0000 7f2e38a00640 1 -- start start 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e38a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e340720e0 0x7f2e340cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e38a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e34160910 0x7f2e34160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e38a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e34073ad0 con 0x7f2e341466b0 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e38a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2e341612a0 con 0x7f2e340720e0 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e38a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2e34146a90 con 0x7f2e34160910 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e31e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e340720e0 0x7f2e340cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e32800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e341466b0 0x7f2e3406de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51858/0 (socket says 172.21.15.46:51858) 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e32800640 1 -- 172.21.15.46:0/1630226863 learned_addr learned my addr 172.21.15.46:0/1630226863 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.274+0000 7f2e33200640 1 --2- 172.21.15.46:0/1630226863 >> v2:172.21.15.142:3300/0 conn(0x7f2e34160910 0x7f2e34160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.275+0000 7f2e31e00640 1 -- 172.21.15.46:0/1630226863 >> v2:172.21.15.142:3300/0 conn(0x7f2e34160910 msgr2=0x7f2e34160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.275+0000 7f2e31e00640 1 --2- 172.21.15.46:0/1630226863 >> v2:172.21.15.142:3300/0 conn(0x7f2e34160910 0x7f2e34160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.275+0000 7f2e31e00640 1 -- 172.21.15.46:0/1630226863 >> v1:172.21.15.46:6789/0 conn(0x7f2e341466b0 legacy=0x7f2e3406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.275+0000 7f2e31e00640 1 -- 172.21.15.46:0/1630226863 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e34073d70 con 0x7f2e340720e0 2024-04-26T23:41:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.275+0000 7f2e31400640 1 -- 172.21.15.46:0/1630226863 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 579976996 0 0) 0x7f2e34073ad0 con 0x7f2e341466b0 2024-04-26T23:41:42.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.276+0000 7f2e31e00640 1 --2- 172.21.15.46:0/1630226863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e340cd860 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097a0 tx=0x7f2e1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15f083506aed529 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.276+0000 7f2e31400640 1 -- 172.21.15.46:0/1630226863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c037070 con 0x7f2e340720e0 2024-04-26T23:41:42.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.276+0000 7f2e31400640 1 -- 172.21.15.46:0/1630226863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c032440 con 0x7f2e340720e0 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.276+0000 7f2e31400640 1 -- 172.21.15.46:0/1630226863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c032810 con 0x7f2e340720e0 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 -- 172.21.15.46:0/1630226863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 msgr2=0x7f2e340cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 --2- 172.21.15.46:0/1630226863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e340cd860 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097a0 tx=0x7f2e1c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 -- 172.21.15.46:0/1630226863 shutdown_connections 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 --2- 172.21.15.46:0/1630226863 >> v2:172.21.15.142:3300/0 conn(0x7f2e34160910 0x7f2e34160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 --2- 172.21.15.46:0/1630226863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e340cd860 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 -- 172.21.15.46:0/1630226863 >> 172.21.15.46:0/1630226863 conn(0x7f2e340b26d0 msgr2=0x7f2e340b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 -- 172.21.15.46:0/1630226863 shutdown_connections 2024-04-26T23:41:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.277+0000 7f2e38a00640 1 -- 172.21.15.46:0/1630226863 wait complete. 2024-04-26T23:41:42.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.279+0000 7f2e38a00640 1 Processor -- start 2024-04-26T23:41:42.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.279+0000 7f2e38a00640 1 -- start start 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 0x7f2e340d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e34160910 0x7f2e340d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cb470 con 0x7f2e340720e0 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cb190 con 0x7f2e341466b0 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e38a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340cdda0 con 0x7f2e34160910 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44780/0 (socket says 172.21.15.46:44780) 2024-04-26T23:41:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.280+0000 7f2e32800640 1 -- 172.21.15.46:0/3723808512 learned_addr learned my addr 172.21.15.46:0/3723808512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e31e00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 0x7f2e340d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e33200640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e34160910 0x7f2e340d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 -- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e34160910 msgr2=0x7f2e340d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e34160910 0x7f2e340d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 -- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 msgr2=0x7f2e340d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 0x7f2e340d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 -- 172.21.15.46:0/3723808512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e341f7db0 con 0x7f2e340720e0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e31e00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 0x7f2e340d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e32800640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c430 tx=0x7f2e2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240074e0 con 0x7f2e340720e0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e240047e0 con 0x7f2e340720e0 2024-04-26T23:41:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e340c9980 con 0x7f2e340720e0 2024-04-26T23:41:42.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24004bd0 con 0x7f2e340720e0 2024-04-26T23:41:42.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.281+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e340c9f40 con 0x7f2e340720e0 2024-04-26T23:41:42.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.283+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e340726b0 con 0x7f2e340720e0 2024-04-26T23:41:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.284+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e24011070 con 0x7f2e340720e0 2024-04-26T23:41:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.284+0000 7f2e2aa00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 0x7f2e1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.285+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f2e2408b9b0 con 0x7f2e340720e0 2024-04-26T23:41:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.285+0000 7f2e31e00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 0x7f2e1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.286+0000 7f2e31e00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 0x7f2e1006ede0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c034000 tx=0x7f2e1c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.289+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e340726b0 con 0x7f2e340720e0 2024-04-26T23:41:42.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.537+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e340c7180 con 0x7f2e340720e0 2024-04-26T23:41:42.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.538+0000 7f2e2aa00640 1 -- 172.21.15.46:0/3723808512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7f2e2400e280 con 0x7f2e340720e0 2024-04-26T23:41:42.542 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:42.542 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:39:05.780519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:41:42.542 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 msgr2=0x7f2e1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 0x7f2e1006ede0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c034000 tx=0x7f2e1c02e000 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 msgr2=0x7f2e341f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c430 tx=0x7f2e2400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 shutdown_connections 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e1006c920 0x7f2e1006ede0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e34160910 0x7f2e340d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e341466b0 0x7f2e340d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 --2- 172.21.15.46:0/3723808512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e340720e0 0x7f2e341f7870 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.541+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 >> 172.21.15.46:0/3723808512 conn(0x7f2e340b26d0 msgr2=0x7f2e340c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.542+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 shutdown_connections 2024-04-26T23:41:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.542+0000 7f2e38a00640 1 -- 172.21.15.46:0/3723808512 wait complete. 2024-04-26T23:41:42.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:41:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.779+0000 7fa498e00640 1 Processor -- start 2024-04-26T23:41:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.780+0000 7fa498e00640 1 -- start start 2024-04-26T23:41:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.781+0000 7fa498e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4940b4000 0x7fa4940d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.781+0000 7fa498e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4940d11f0 con 0x7fa4940d2130 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.781+0000 7fa498e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4940d5f20 con 0x7fa49415f4e0 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.781+0000 7fa498e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa4940d0ce0 con 0x7fa4940b4000 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.781+0000 7fa492a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa4940d2130 0x7fa4940b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51862/0 (socket says 172.21.15.46:51862) 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa492a00640 1 -- 172.21.15.46:0/2442472052 learned_addr learned my addr 172.21.15.46:0/2442472052 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa493e00640 1 --2- 172.21.15.46:0/2442472052 >> v2:172.21.15.46:3300/0 conn(0x7fa4940b4000 0x7fa4940d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa492000640 1 -- 172.21.15.46:0/2442472052 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004832809 0 0) 0x7fa4940d11f0 con 0x7fa4940d2130 2024-04-26T23:41:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa492000640 1 -- 172.21.15.46:0/2442472052 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa478003520 con 0x7fa4940d2130 2024-04-26T23:41:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa493e00640 1 -- 172.21.15.46:0/2442472052 >> v1:172.21.15.142:6789/0 conn(0x7fa49415f4e0 legacy=0x7fa4940d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa493e00640 1 -- 172.21.15.46:0/2442472052 >> v1:172.21.15.46:6789/0 conn(0x7fa4940d2130 legacy=0x7fa4940b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.782+0000 7fa493e00640 1 -- 172.21.15.46:0/2442472052 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4940d0f10 con 0x7fa4940b4000 2024-04-26T23:41:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.783+0000 7fa493e00640 1 --2- 172.21.15.46:0/2442472052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940b4000 0x7fa4940d2df0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009dc0 tx=0x7fa48c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b867deb07a692 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa492000640 1 -- 172.21.15.46:0/2442472052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c007740 con 0x7fa4940b4000 2024-04-26T23:41:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa492000640 1 -- 172.21.15.46:0/2442472052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48c0078e0 con 0x7fa4940b4000 2024-04-26T23:41:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa492000640 1 -- 172.21.15.46:0/2442472052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c007cb0 con 0x7fa4940b4000 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 -- 172.21.15.46:0/2442472052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940b4000 msgr2=0x7fa4940d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 --2- 172.21.15.46:0/2442472052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940b4000 0x7fa4940d2df0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009dc0 tx=0x7fa48c02a730 comp rx=0 tx=0).stop 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 -- 172.21.15.46:0/2442472052 shutdown_connections 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 --2- 172.21.15.46:0/2442472052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940b4000 0x7fa4940d2df0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 -- 172.21.15.46:0/2442472052 >> 172.21.15.46:0/2442472052 conn(0x7fa4940d5010 msgr2=0x7fa4940d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 -- 172.21.15.46:0/2442472052 shutdown_connections 2024-04-26T23:41:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.784+0000 7fa498e00640 1 -- 172.21.15.46:0/2442472052 wait complete. 2024-04-26T23:41:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.786+0000 7fa498e00640 1 Processor -- start 2024-04-26T23:41:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.786+0000 7fa498e00640 1 -- start start 2024-04-26T23:41:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 0x7fa4940cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 0x7fa4941e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4940b1d60 con 0x7fa4940d2130 2024-04-26T23:41:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4940d2990 con 0x7fa49415f4e0 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa498e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa49415f8c0 con 0x7fa4940b4000 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa492a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa492a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44802/0 (socket says 172.21.15.46:44802) 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.787+0000 7fa492a00640 1 -- 172.21.15.46:0/2775865947 learned_addr learned my addr 172.21.15.46:0/2775865947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa493e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 0x7fa4941e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa493400640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 0x7fa4940cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 -- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 msgr2=0x7fa4940cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 0x7fa4940cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 -- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 msgr2=0x7fa4941e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 0x7fa4941e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 -- 172.21.15.46:0/2775865947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4940ba550 con 0x7fa4940d2130 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa493e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 0x7fa4941e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa493400640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 0x7fa4940cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.788+0000 7fa492a00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c500 tx=0x7fa48000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.789+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa4940d2130 2024-04-26T23:41:42.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.789+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480005ce0 con 0x7fa4940d2130 2024-04-26T23:41:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.789+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480007330 con 0x7fa4940d2130 2024-04-26T23:41:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.789+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49414c1d0 con 0x7fa4940d2130 2024-04-26T23:41:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.789+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49414c790 con 0x7fa4940d2130 2024-04-26T23:41:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.790+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4940693b0 con 0x7fa4940d2130 2024-04-26T23:41:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.791+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa48001a070 con 0x7fa4940d2130 2024-04-26T23:41:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.792+0000 7fa48b400640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 0x7fa47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.792+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7fa48008b200 con 0x7fa4940d2130 2024-04-26T23:41:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.793+0000 7fa493400640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 0x7fa47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.794+0000 7fa493400640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 0x7fa47406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa484000f60 tx=0x7fa484009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:42.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:42.797+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa480059520 con 0x7fa4940d2130 2024-04-26T23:41:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.045+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4940d33d0 con 0x7fa4940d2130 2024-04-26T23:41:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.046+0000 7fa48b400640 1 -- 172.21.15.46:0/2775865947 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7fa480058e20 con 0x7fa4940d2130 2024-04-26T23:41:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:43.050 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:38:51.149821+0000","modified":"2024-04-26T23:39:05.780519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1027205255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1027205255},{"type":"v1","addr":"172.21.15.46:6835","nonce":1027205255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5280,"qdb_cluster":[5280]},"id":2}]} 2024-04-26T23:41:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2024-04-26T23:41:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 msgr2=0x7fa47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 0x7fa47406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa484000f60 tx=0x7fa484009040 comp rx=0 tx=0).stop 2024-04-26T23:41:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 msgr2=0x7fa4941e7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c500 tx=0x7fa48000c9c0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 shutdown_connections 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa47406c8f0 0x7fa47406edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49415f4e0 0x7fa4941e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4940d2130 0x7fa4941e7a90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 --2- 172.21.15.46:0/2775865947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4940b4000 0x7fa4940cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 >> 172.21.15.46:0/2775865947 conn(0x7fa4940d5010 msgr2=0x7fa4940d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 shutdown_connections 2024-04-26T23:41:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.049+0000 7fa498e00640 1 -- 172.21.15.46:0/2775865947 wait complete. 2024-04-26T23:41:43.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.282+0000 7f6f9ea00640 1 Processor -- start 2024-04-26T23:41:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.283+0000 7f6f9ea00640 1 -- start start 2024-04-26T23:41:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.284+0000 7f6f9ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.284+0000 7f6f9ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6f980720e0 0x7f6f980cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.284+0000 7f6f9ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f98073ad0 con 0x7f6f98160910 2024-04-26T23:41:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.284+0000 7f6f9ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6f98146a90 con 0x7f6f980720e0 2024-04-26T23:41:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.284+0000 7f6f9ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6f98073d70 con 0x7f6f981466b0 2024-04-26T23:41:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.285+0000 7f6f9d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.285+0000 7f6f9d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60118/0 (socket says 172.21.15.46:60118) 2024-04-26T23:41:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.285+0000 7f6f9e000640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f6f98160910 0x7f6f980cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:59902/0 (socket says 172.21.15.46:59902) 2024-04-26T23:41:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.285+0000 7f6f9d600640 1 -- 172.21.15.46:0/4180007117 learned_addr learned my addr 172.21.15.46:0/4180007117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.285+0000 7f6f9cc00640 1 --2- 172.21.15.46:0/4180007117 >> v2:172.21.15.142:3300/0 conn(0x7f6f980720e0 0x7f6f980cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.286+0000 7f6f97e00640 1 -- 172.21.15.46:0/4180007117 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727833438 0 0) 0x7f6f98073ad0 con 0x7f6f98160910 2024-04-26T23:41:43.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.286+0000 7f6f9d600640 1 -- 172.21.15.46:0/4180007117 >> v2:172.21.15.142:3300/0 conn(0x7f6f980720e0 msgr2=0x7f6f980cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:43.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.286+0000 7f6f9d600640 1 --2- 172.21.15.46:0/4180007117 >> v2:172.21.15.142:3300/0 conn(0x7f6f980720e0 0x7f6f980cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.286+0000 7f6f9d600640 1 -- 172.21.15.46:0/4180007117 >> v1:172.21.15.142:6789/0 conn(0x7f6f98160910 legacy=0x7f6f980cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.286+0000 7f6f9d600640 1 -- 172.21.15.46:0/4180007117 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f98074050 con 0x7f6f981466b0 2024-04-26T23:41:43.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.287+0000 7f6f9d600640 1 --2- 172.21.15.46:0/4180007117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009870 tx=0x7f6f8c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f42f6818d81925d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f97e00640 1 -- 172.21.15.46:0/4180007117 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c037070 con 0x7f6f981466b0 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f97e00640 1 -- 172.21.15.46:0/4180007117 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c032440 con 0x7f6f981466b0 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f97e00640 1 -- 172.21.15.46:0/4180007117 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c032810 con 0x7f6f981466b0 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f9ea00640 1 -- 172.21.15.46:0/4180007117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 msgr2=0x7f6f9806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/4180007117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009870 tx=0x7f6f8c02a250 comp rx=0 tx=0).stop 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f9ea00640 1 -- 172.21.15.46:0/4180007117 shutdown_connections 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/4180007117 >> v2:172.21.15.142:3300/0 conn(0x7f6f980720e0 0x7f6f980cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.288+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/4180007117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f9806de80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.289+0000 7f6f9ea00640 1 -- 172.21.15.46:0/4180007117 >> 172.21.15.46:0/4180007117 conn(0x7f6f980b26d0 msgr2=0x7f6f980b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.289+0000 7f6f9ea00640 1 -- 172.21.15.46:0/4180007117 shutdown_connections 2024-04-26T23:41:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.289+0000 7f6f9ea00640 1 -- 172.21.15.46:0/4180007117 wait complete. 2024-04-26T23:41:43.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.290+0000 7f6f9ea00640 1 Processor -- start 2024-04-26T23:41:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.291+0000 7f6f9ea00640 1 -- start start 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.291+0000 7f6f9ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f980720e0 0x7f6f980cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f980ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9806e3c0 con 0x7f6f98160910 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f980cb470 con 0x7f6f980720e0 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6f980cdda0 con 0x7f6f981466b0 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44810/0 (socket says 172.21.15.46:44810) 2024-04-26T23:41:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 -- 172.21.15.46:0/2424680187 learned_addr learned my addr 172.21.15.46:0/2424680187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9d600640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f980720e0 0x7f6f980cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9cc00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f980ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 -- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 msgr2=0x7f6f980ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f980ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 -- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f980720e0 msgr2=0x7f6f980cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f980720e0 0x7f6f980cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.292+0000 7f6f9e000640 1 -- 172.21.15.46:0/2424680187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f980af250 con 0x7f6f98160910 2024-04-26T23:41:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f9e000640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c700 tx=0x7f6f9000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90018070 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9000ce10 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000a860 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f980af530 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.293+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f980afaf0 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.295+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f980693b0 con 0x7f6f98160910 2024-04-26T23:41:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.296+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f9000aa00 con 0x7f6f98160910 2024-04-26T23:41:43.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.297+0000 7f6f95600640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 0x7f6f7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.298+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f6f9008b2d0 con 0x7f6f98160910 2024-04-26T23:41:43.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.298+0000 7f6f9d600640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 0x7f6f7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.299+0000 7f6f9d600640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 0x7f6f7806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c034000 tx=0x7f6f8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.302+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f900595d0 con 0x7f6f98160910 2024-04-26T23:41:43.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.510+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f9806e790 con 0x7f6f98160910 2024-04-26T23:41:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.512+0000 7f6f95600640 1 -- 172.21.15.46:0/2424680187 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17020 (secure 0 0 0) 0x7f6f90058ed0 con 0x7f6f98160910 2024-04-26T23:41:43.517 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:43.517 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:13.670571+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:38:49.046203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:38:50.124386+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:43.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 msgr2=0x7f6f7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 0x7f6f7806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c034000 tx=0x7f6f8c02e000 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 msgr2=0x7f6f98141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c700 tx=0x7f6f9000cbc0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 shutdown_connections 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f7806c8f0 0x7f6f7806edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f98160910 0x7f6f98141480 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f981466b0 0x7f6f980ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 --2- 172.21.15.46:0/2424680187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f980720e0 0x7f6f980cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 >> 172.21.15.46:0/2424680187 conn(0x7f6f980b26d0 msgr2=0x7f6f980b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 shutdown_connections 2024-04-26T23:41:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.515+0000 7f6f9ea00640 1 -- 172.21.15.46:0/2424680187 wait complete. 2024-04-26T23:41:43.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:41:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.748+0000 7f75bae00640 1 Processor -- start 2024-04-26T23:41:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.748+0000 7f75bae00640 1 -- start start 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.749+0000 7f75bae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f75b40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.749+0000 7f75bae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b40cf6f0 con 0x7f75b415d150 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.749+0000 7f75bae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b40cf990 con 0x7f75b40cd890 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.749+0000 7f75bae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40d1c70 con 0x7f75b40d12f0 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.750+0000 7f75b9a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75b415d150 0x7f75b40d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51870/0 (socket says 172.21.15.46:51870) 2024-04-26T23:41:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.750+0000 7f75b9a00640 1 -- 172.21.15.46:0/2683332793 learned_addr learned my addr 172.21.15.46:0/2683332793 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.750+0000 7f75b9000640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f75b40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.750+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842668622 0 0) 0x7f75b40cf6f0 con 0x7f75b415d150 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.750+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f759c003520 con 0x7f75b415d150 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167907257 0 0) 0x7f759c003520 con 0x7f75b415d150 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 msgr2=0x7f75b40d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f75b40d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 >> v1:172.21.15.142:6789/0 conn(0x7f75b40cd890 legacy=0x7f75b40cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b40cfc70 con 0x7f75b415d150 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f75a4001690 con 0x7f75b415d150 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 >> v1:172.21.15.46:6789/0 conn(0x7f75b415d150 legacy=0x7f75b40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f759c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f759c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c004d40 0x7f759c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40d1c70 con 0x7f759c0039e0 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f759c00bb30 con 0x7f759c004d40 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f759c00bd30 con 0x7f75b40d12f0 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.751+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f759c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9000640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f759c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75ba400640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c004d40 0x7f759c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9a00640 1 -- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 msgr2=0x7f759c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b40d12f0 0x7f759c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9a00640 1 -- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c004d40 msgr2=0x7f759c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c004d40 0x7f759c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.752+0000 7f75b9a00640 1 -- 172.21.15.46:0/2683332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c003520 con 0x7f759c0039e0 2024-04-26T23:41:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.753+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f759c003e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75b40cf8c0 tx=0x7f759c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7efa7b02e3b229 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.753+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4007c60 con 0x7f759c0039e0 2024-04-26T23:41:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.753+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a4005ad0 con 0x7f759c0039e0 2024-04-26T23:41:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.753+0000 7f75b3e00640 1 -- 172.21.15.46:0/2683332793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4007070 con 0x7f759c0039e0 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 -- 172.21.15.46:0/2683332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 msgr2=0x7f759c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 --2- 172.21.15.46:0/2683332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f759c003e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75b40cf8c0 tx=0x7f759c005830 comp rx=0 tx=0).stop 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75ba400640 1 -- 172.21.15.46:0/2683332793 reap_dead start 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 -- 172.21.15.46:0/2683332793 shutdown_connections 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 -- 172.21.15.46:0/2683332793 >> 172.21.15.46:0/2683332793 conn(0x7f75b40b4880 msgr2=0x7f75b40b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 -- 172.21.15.46:0/2683332793 shutdown_connections 2024-04-26T23:41:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.754+0000 7f75bae00640 1 -- 172.21.15.46:0/2683332793 wait complete. 2024-04-26T23:41:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.756+0000 7f75bae00640 1 Processor -- start 2024-04-26T23:41:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.756+0000 7f75bae00640 1 -- start start 2024-04-26T23:41:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 0x7f75b40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b40b7e30 0x7f75b406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75b415d530 con 0x7f759c0039e0 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40d0230 con 0x7f75b40b7e30 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.757+0000 7f75bae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40d0ff0 con 0x7f75b41f2580 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44838/0 (socket says 172.21.15.46:44838) 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 -- 172.21.15.46:0/2790880183 learned_addr learned my addr 172.21.15.46:0/2790880183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75ba400640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b40b7e30 0x7f75b406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9000640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 0x7f75b40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 -- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 msgr2=0x7f75b40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 0x7f75b40b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 -- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b40b7e30 msgr2=0x7f75b406dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b40b7e30 0x7f75b406dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9a00640 1 -- 172.21.15.46:0/2790880183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b40b8400 con 0x7f759c0039e0 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.758+0000 7f75b9000640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 0x7f75b40b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75b9a00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f75a402ad00 tx=0x7f75b40cfb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4040020 con 0x7f759c0039e0 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a40013d0 con 0x7f759c0039e0 2024-04-26T23:41:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b40b86e0 con 0x7f759c0039e0 2024-04-26T23:41:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4037040 con 0x7f759c0039e0 2024-04-26T23:41:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.759+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41f8c80 con 0x7f759c0039e0 2024-04-26T23:41:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.760+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b41f8790 con 0x7f759c0039e0 2024-04-26T23:41:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.761+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75a4043020 con 0x7f759c0039e0 2024-04-26T23:41:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.763+0000 7f75b1600640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.764+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6158+0+0 (secure 0 0 0) 0x7f75a40abba0 con 0x7f759c0039e0 2024-04-26T23:41:43.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.764+0000 7f75b9000640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:43.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.765+0000 7f75b9000640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 0x7f759806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75a8012430 tx=0x7f75a800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:43.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:43.768+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75a4005aa0 con 0x7f759c0039e0 2024-04-26T23:41:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.014+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75b40b88b0 con 0x7f759c0039e0 2024-04-26T23:41:44.018 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:41:44.015+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:41:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.018+0000 7f75b1600640 1 -- 172.21.15.46:0/2790880183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7f75a4079e30 con 0x7f759c0039e0 2024-04-26T23:41:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:41:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 msgr2=0x7f759806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 0x7f759806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75a8012430 tx=0x7f75a800b040 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 msgr2=0x7f75b41f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f75a402ad00 tx=0x7f75b40cfb60 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 shutdown_connections 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b40b7e30 0x7f75b406dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b41f2580 0x7f75b40b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 --2- 172.21.15.46:0/2790880183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c0039e0 0x7f75b41f91a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.021+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 >> 172.21.15.46:0/2790880183 conn(0x7f75b40b4880 msgr2=0x7f75b40d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.022+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 shutdown_connections 2024-04-26T23:41:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.022+0000 7f75bae00640 1 -- 172.21.15.46:0/2790880183 wait complete. 2024-04-26T23:41:44.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:41:44.120 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:41:44.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.244+0000 7f629d000640 1 Processor -- start 2024-04-26T23:41:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.244+0000 7f629d000640 1 -- start start 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f629d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62980d0a10 0x7f62980d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f629d000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f62980d34c0 0x7f62980d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f629d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62980cfc50 con 0x7f629815f2a0 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f629d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f62980d3e90 con 0x7f62980d0a10 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f629d000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f62980d0580 con 0x7f62980d34c0 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f6296a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62980d0a10 0x7f62980d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f6296a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62980d0a10 0x7f62980d2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44852/0 (socket says 172.21.15.46:44852) 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.246+0000 7f6296a00640 1 -- 172.21.15.46:0/3588151039 learned_addr learned my addr 172.21.15.46:0/3588151039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.247+0000 7f6297e00640 1 --2- 172.21.15.46:0/3588151039 >> v2:172.21.15.142:3300/0 conn(0x7f62980d34c0 0x7f62980d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.247+0000 7f6296a00640 1 -- 172.21.15.46:0/3588151039 >> v2:172.21.15.142:3300/0 conn(0x7f62980d34c0 msgr2=0x7f62980d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.247+0000 7f6296a00640 1 --2- 172.21.15.46:0/3588151039 >> v2:172.21.15.142:3300/0 conn(0x7f62980d34c0 0x7f62980d3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.247+0000 7f6296a00640 1 -- 172.21.15.46:0/3588151039 >> v1:172.21.15.142:6789/0 conn(0x7f629815f2a0 legacy=0x7f62980d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.247+0000 7f6296a00640 1 -- 172.21.15.46:0/3588151039 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62980cff30 con 0x7f62980d0a10 2024-04-26T23:41:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.248+0000 7f6296a00640 1 --2- 172.21.15.46:0/3588151039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62980d2480 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f62980cf7c0 tx=0x7f628002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d19b521d1490ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f6296000640 1 -- 172.21.15.46:0/3588151039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280037070 con 0x7f62980d0a10 2024-04-26T23:41:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f6296000640 1 -- 172.21.15.46:0/3588151039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6280032440 con 0x7f62980d0a10 2024-04-26T23:41:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f6296000640 1 -- 172.21.15.46:0/3588151039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280032810 con 0x7f62980d0a10 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 -- 172.21.15.46:0/3588151039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 msgr2=0x7f62980d2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 --2- 172.21.15.46:0/3588151039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62980d2480 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f62980cf7c0 tx=0x7f628002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 -- 172.21.15.46:0/3588151039 shutdown_connections 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 --2- 172.21.15.46:0/3588151039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62980d2480 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 --2- 172.21.15.46:0/3588151039 >> v2:172.21.15.142:3300/0 conn(0x7f62980d34c0 0x7f62980d3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 -- 172.21.15.46:0/3588151039 >> 172.21.15.46:0/3588151039 conn(0x7f62980d4f20 msgr2=0x7f62980d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.249+0000 7f629d000640 1 -- 172.21.15.46:0/3588151039 shutdown_connections 2024-04-26T23:41:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.250+0000 7f629d000640 1 -- 172.21.15.46:0/3588151039 wait complete. 2024-04-26T23:41:44.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.251+0000 7f629d000640 1 Processor -- start 2024-04-26T23:41:44.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.251+0000 7f629d000640 1 -- start start 2024-04-26T23:41:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 0x7f6298160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f629815f2a0 0x7f6298153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f629815f680 con 0x7f62980d0a10 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62980cf9f0 con 0x7f629815f2a0 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.252+0000 7f629d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f629806cd60 con 0x7f62980d34c0 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44866/0 (socket says 172.21.15.46:44866) 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 -- 172.21.15.46:0/1901847915 learned_addr learned my addr 172.21.15.46:0/1901847915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6296a00640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 0x7f6298160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297e00640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f629815f2a0 0x7f6298153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 -- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 msgr2=0x7f6298160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 0x7f6298160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 -- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f629815f2a0 msgr2=0x7f6298153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f629815f2a0 0x7f6298153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6297400640 1 -- 172.21.15.46:0/1901847915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62980b8d50 con 0x7f62980d0a10 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.253+0000 7f6296a00640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 0x7f6298160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f6297400640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f628800b3f0 tx=0x7f628800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288015020 con 0x7f62980d0a10 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6288011830 con 0x7f62980d0a10 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288011c20 con 0x7f62980d0a10 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62980b7fa0 con 0x7f62980d0a10 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.254+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62980b8560 con 0x7f62980d0a10 2024-04-26T23:41:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.255+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62980693b0 con 0x7f62980d0a10 2024-04-26T23:41:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.257+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6288018020 con 0x7f62980d0a10 2024-04-26T23:41:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.257+0000 7f628f400640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 0x7f627406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.258+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f628808b920 con 0x7f62980d0a10 2024-04-26T23:41:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.258+0000 7f6296a00640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 0x7f627406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.259+0000 7f6296a00640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 0x7f627406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6280034000 tx=0x7f628002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.262+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f628801f030 con 0x7f62980d0a10 2024-04-26T23:41:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:44.509+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6298068e80 con 0x7f62980d0a10 2024-04-26T23:41:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.022+0000 7f628f400640 1 -- 172.21.15.46:0/1901847915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f62880172c0 con 0x7f62980d0a10 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 msgr2=0x7f627406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 0x7f627406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6280034000 tx=0x7f628002e000 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 msgr2=0x7f62981f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f628800b3f0 tx=0x7f628800b8b0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 shutdown_connections 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.026+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f627406c8f0 0x7f627406edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f629815f2a0 0x7f6298153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62980d34c0 0x7f6298160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 --2- 172.21.15.46:0/1901847915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62980d0a10 0x7f62981f93b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 >> 172.21.15.46:0/1901847915 conn(0x7f62980d4f20 msgr2=0x7f62980d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 shutdown_connections 2024-04-26T23:41:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.027+0000 7f629d000640 1 -- 172.21.15.46:0/1901847915 wait complete. 2024-04-26T23:41:45.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:41:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.268+0000 7f7dbf800640 1 Processor -- start 2024-04-26T23:41:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.268+0000 7f7dbf800640 1 -- start start 2024-04-26T23:41:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dbf800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7db806d140 0x7f7db806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dbf800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7db806daf0 0x7f7db80aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dbf800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db815f850 con 0x7f7db815f470 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dbf800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80cc4f0 con 0x7f7db806d140 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dbf800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80b00e0 con 0x7f7db806daf0 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.270+0000 7f7dafe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7db806d140 0x7f7db806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7db806d140 0x7f7db806d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44882/0 (socket says 172.21.15.46:44882) 2024-04-26T23:41:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 -- 172.21.15.46:0/3951340182 learned_addr learned my addr 172.21.15.46:0/3951340182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 -- 172.21.15.46:0/3951340182 >> v2:172.21.15.142:3300/0 conn(0x7f7db806daf0 msgr2=0x7f7db80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dbd800640 1 --2- 172.21.15.46:0/3951340182 >> v2:172.21.15.142:3300/0 conn(0x7f7db806daf0 0x7f7db80aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 --2- 172.21.15.46:0/3951340182 >> v2:172.21.15.142:3300/0 conn(0x7f7db806daf0 0x7f7db80aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 -- 172.21.15.46:0/3951340182 >> v1:172.21.15.142:6789/0 conn(0x7f7db815f470 legacy=0x7f7db80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.271+0000 7f7dafe00640 1 -- 172.21.15.46:0/3951340182 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db80cc740 con 0x7f7db806d140 2024-04-26T23:41:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.272+0000 7f7dafe00640 1 --2- 172.21.15.46:0/3951340182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806d140 0x7f7db806d5b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7db80682f0 tx=0x7f7da002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67732ea0d8294e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.272+0000 7f7db7e00640 1 -- 172.21.15.46:0/3951340182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0037070 con 0x7f7db806d140 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.272+0000 7f7db7e00640 1 -- 172.21.15.46:0/3951340182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da0032440 con 0x7f7db806d140 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.272+0000 7f7db7e00640 1 -- 172.21.15.46:0/3951340182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0032810 con 0x7f7db806d140 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 -- 172.21.15.46:0/3951340182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806d140 msgr2=0x7f7db806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 --2- 172.21.15.46:0/3951340182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806d140 0x7f7db806d5b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7db80682f0 tx=0x7f7da002a2c0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 -- 172.21.15.46:0/3951340182 shutdown_connections 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 --2- 172.21.15.46:0/3951340182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806d140 0x7f7db806d5b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 --2- 172.21.15.46:0/3951340182 >> v2:172.21.15.142:3300/0 conn(0x7f7db806daf0 0x7f7db80aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 -- 172.21.15.46:0/3951340182 >> 172.21.15.46:0/3951340182 conn(0x7f7db80ce030 msgr2=0x7f7db80ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 -- 172.21.15.46:0/3951340182 shutdown_connections 2024-04-26T23:41:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.273+0000 7f7dbf800640 1 -- 172.21.15.46:0/3951340182 wait complete. 2024-04-26T23:41:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.275+0000 7f7dbf800640 1 Processor -- start 2024-04-26T23:41:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.275+0000 7f7dbf800640 1 -- start start 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 0x7f7db8156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 0x7f7db80ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80af010 con 0x7f7db806daf0 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80b0c50 con 0x7f7db806d140 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.276+0000 7f7dbf800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80af190 con 0x7f7db815f470 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44886/0 (socket says 172.21.15.46:44886) 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 -- 172.21.15.46:0/1741782307 learned_addr learned my addr 172.21.15.46:0/1741782307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dbd800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 0x7f7db80ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 -- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 msgr2=0x7f7db80ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 0x7f7db80ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 -- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 msgr2=0x7f7db8156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dbce00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 0x7f7db8156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 0x7f7db8156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 -- 172.21.15.46:0/1741782307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db815fb50 con 0x7f7db806daf0 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dbd800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 0x7f7db80ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dbce00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 0x7f7db8156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.277+0000 7f7dafe00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000c00 tx=0x7f7da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.278+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0041020 con 0x7f7db806daf0 2024-04-26T23:41:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.278+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da0030970 con 0x7f7db806daf0 2024-04-26T23:41:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.278+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0037040 con 0x7f7db806daf0 2024-04-26T23:41:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.278+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db80caa20 con 0x7f7db806daf0 2024-04-26T23:41:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.278+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81600d0 con 0x7f7db806daf0 2024-04-26T23:41:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.279+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8069530 con 0x7f7db806daf0 2024-04-26T23:41:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.280+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7da0044020 con 0x7f7db806daf0 2024-04-26T23:41:45.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.281+0000 7f7db5600640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 0x7f7d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:45.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.281+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f7da00ac7d0 con 0x7f7db806daf0 2024-04-26T23:41:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.282+0000 7f7dbce00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 0x7f7d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.283+0000 7f7dbce00640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 0x7f7d9406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7da8003eb0 tx=0x7f7da8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.286+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7da004b030 con 0x7f7db806daf0 2024-04-26T23:41:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:45.491+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7db8069ac0 con 0x7f7db806daf0 2024-04-26T23:41:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.025+0000 7f7db5600640 1 -- 172.21.15.46:0/1741782307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7f7da007a9f0 con 0x7f7db806daf0 2024-04-26T23:41:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 msgr2=0x7f7d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 0x7f7d9406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7da8003eb0 tx=0x7f7da8009040 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 msgr2=0x7f7db8161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000c00 tx=0x7f7da00046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 shutdown_connections 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d9406c8f0 0x7f7d9406edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db815f470 0x7f7db80ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db806daf0 0x7f7db8161df0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 --2- 172.21.15.46:0/1741782307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db806d140 0x7f7db8156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 >> 172.21.15.46:0/1741782307 conn(0x7f7db80ce030 msgr2=0x7f7db80af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.031+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 shutdown_connections 2024-04-26T23:41:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.032+0000 7f7dbf800640 1 -- 172.21.15.46:0/1741782307 wait complete. 2024-04-26T23:41:46.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:41:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.259+0000 7ff4c7200640 1 Processor -- start 2024-04-26T23:41:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.259+0000 7ff4c7200640 1 -- start start 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.260+0000 7ff4c7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4c8146740 0x7ff4c80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.260+0000 7ff4c7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4c8146b20 0x7ff4c80d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c80b3ce0 con 0x7ff4c80cdc10 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80b48f0 con 0x7ff4c8146b20 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80b3fc0 con 0x7ff4c8146740 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4c8146b20 0x7ff4c80d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4c8146b20 0x7ff4c80d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44902/0 (socket says 172.21.15.46:44902) 2024-04-26T23:41:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c5400640 1 -- 172.21.15.46:0/3615314166 learned_addr learned my addr 172.21.15.46:0/3615314166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.261+0000 7ff4c5e00640 1 --2- 172.21.15.46:0/3615314166 >> v2:172.21.15.142:3300/0 conn(0x7ff4c8146740 0x7ff4c80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.262+0000 7ff4c5400640 1 -- 172.21.15.46:0/3615314166 >> v2:172.21.15.142:3300/0 conn(0x7ff4c8146740 msgr2=0x7ff4c80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.262+0000 7ff4c5400640 1 --2- 172.21.15.46:0/3615314166 >> v2:172.21.15.142:3300/0 conn(0x7ff4c8146740 0x7ff4c80cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.262+0000 7ff4c5400640 1 -- 172.21.15.46:0/3615314166 >> v1:172.21.15.142:6789/0 conn(0x7ff4c80cdc10 legacy=0x7ff4c8160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.262+0000 7ff4c5400640 1 -- 172.21.15.46:0/3615314166 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80b42a0 con 0x7ff4c8146b20 2024-04-26T23:41:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.264+0000 7ff4c5400640 1 --2- 172.21.15.46:0/3615314166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c80d10e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b4a70 tx=0x7ff4b402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1d5334ec0e8bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.264+0000 7ff4c4a00640 1 -- 172.21.15.46:0/3615314166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037070 con 0x7ff4c8146b20 2024-04-26T23:41:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.264+0000 7ff4c4a00640 1 -- 172.21.15.46:0/3615314166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4032440 con 0x7ff4c8146b20 2024-04-26T23:41:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.264+0000 7ff4c4a00640 1 -- 172.21.15.46:0/3615314166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4032810 con 0x7ff4c8146b20 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.265+0000 7ff4c7200640 1 -- 172.21.15.46:0/3615314166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 msgr2=0x7ff4c80d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.265+0000 7ff4c7200640 1 --2- 172.21.15.46:0/3615314166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c80d10e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b4a70 tx=0x7ff4b402a1d0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 -- 172.21.15.46:0/3615314166 shutdown_connections 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 --2- 172.21.15.46:0/3615314166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c80d10e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 --2- 172.21.15.46:0/3615314166 >> v2:172.21.15.142:3300/0 conn(0x7ff4c8146740 0x7ff4c80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 -- 172.21.15.46:0/3615314166 >> 172.21.15.46:0/3615314166 conn(0x7ff4c806c770 msgr2=0x7ff4c806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 -- 172.21.15.46:0/3615314166 shutdown_connections 2024-04-26T23:41:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.266+0000 7ff4c7200640 1 -- 172.21.15.46:0/3615314166 wait complete. 2024-04-26T23:41:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.269+0000 7ff4c7200640 1 Processor -- start 2024-04-26T23:41:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.269+0000 7ff4c7200640 1 -- start start 2024-04-26T23:41:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.270+0000 7ff4c7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4c80cdc10 0x7ff4c813dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c8146740 0x7ff4c815c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80d05a0 con 0x7ff4c8146b20 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c815e560 con 0x7ff4c8146740 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c80d0720 con 0x7ff4c80cdc10 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44908/0 (socket says 172.21.15.46:44908) 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c6800640 1 -- 172.21.15.46:0/2511340344 learned_addr learned my addr 172.21.15.46:0/2511340344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.271+0000 7ff4c5e00640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4c80cdc10 0x7ff4c813dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c5400640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c8146740 0x7ff4c815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c6800640 1 -- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4c80cdc10 msgr2=0x7ff4c813dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c6800640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4c80cdc10 0x7ff4c813dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c6800640 1 -- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c8146740 msgr2=0x7ff4c815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c6800640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c8146740 0x7ff4c815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.272+0000 7ff4c6800640 1 -- 172.21.15.46:0/2511340344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8142930 con 0x7ff4c8146b20 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.273+0000 7ff4c6800640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c730 tx=0x7ff4c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.273+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0003b70 con 0x7ff4c8146b20 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.273+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c0003d30 con 0x7ff4c8146b20 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.273+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0005310 con 0x7ff4c8146b20 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.273+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c8145200 con 0x7ff4c8146b20 2024-04-26T23:41:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.274+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c8142e80 con 0x7ff4c8146b20 2024-04-26T23:41:46.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.275+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c80693b0 con 0x7ff4c8146b20 2024-04-26T23:41:46.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.276+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4c00054b0 con 0x7ff4c8146b20 2024-04-26T23:41:46.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.277+0000 7ff4be000640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:46.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.277+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5778+0+0 (secure 0 0 0) 0x7ff4c008b9b0 con 0x7ff4c8146b20 2024-04-26T23:41:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.278+0000 7ff4c5e00640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:46.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.279+0000 7ff4c5e00640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009aa0 tx=0x7ff4b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.281+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4c8145200 con 0x7ff4c8146b20 2024-04-26T23:41:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:46.496+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff4c80b5e40 con 0x7ff4c8146b20 2024-04-26T23:41:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.037+0000 7ff4be000640 1 -- 172.21.15.46:0/2511340344 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7ff4c0059d90 con 0x7ff4c8146b20 2024-04-26T23:41:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:41:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.041+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 msgr2=0x7ff4a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.041+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009aa0 tx=0x7ff4b8009190 comp rx=0 tx=0).stop 2024-04-26T23:41:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.041+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 msgr2=0x7ff4c81f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.041+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c730 tx=0x7ff4c000cbf0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 shutdown_connections 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c8146b20 0x7ff4c81f2cd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c8146740 0x7ff4c815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 --2- 172.21.15.46:0/2511340344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4c80cdc10 0x7ff4c813dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.043+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 >> 172.21.15.46:0/2511340344 conn(0x7ff4c806c770 msgr2=0x7ff4c8145950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.044+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 shutdown_connections 2024-04-26T23:41:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.049+0000 7ff4c7200640 1 -- 172.21.15.46:0/2511340344 wait complete. 2024-04-26T23:41:47.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:41:47.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.266+0000 7fae70e00640 1 Processor -- start 2024-04-26T23:41:47.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.266+0000 7fae70e00640 1 -- start start 2024-04-26T23:41:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.267+0000 7fae70e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae6c071f80 0x7fae6c163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.267+0000 7fae70e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c0723e0 0x7fae6c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.267+0000 7fae70e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6c072880 con 0x7fae6c15dd60 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.267+0000 7fae70e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c15e140 con 0x7fae6c071f80 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.267+0000 7fae70e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c072b40 con 0x7fae6c0723e0 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae6c071f80 0x7fae6c163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae6c071f80 0x7fae6c163bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44920/0 (socket says 172.21.15.46:44920) 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 -- 172.21.15.46:0/1875492218 learned_addr learned my addr 172.21.15.46:0/1875492218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 -- 172.21.15.46:0/1875492218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c0723e0 msgr2=0x7fae6c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae6a800640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c0723e0 0x7fae6c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c0723e0 0x7fae6c06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 -- 172.21.15.46:0/1875492218 >> v1:172.21.15.142:6789/0 conn(0x7fae6c15dd60 legacy=0x7fae6c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.268+0000 7fae69400640 1 -- 172.21.15.46:0/1875492218 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c072e20 con 0x7fae6c071f80 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.269+0000 7fae69400640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c071f80 0x7fae6c163bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3548867f03ca97aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae68a00640 1 -- 172.21.15.46:0/1875492218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037070 con 0x7fae6c071f80 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae68a00640 1 -- 172.21.15.46:0/1875492218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c032440 con 0x7fae6c071f80 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae68a00640 1 -- 172.21.15.46:0/1875492218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c032810 con 0x7fae6c071f80 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 -- 172.21.15.46:0/1875492218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c071f80 msgr2=0x7fae6c163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c071f80 0x7fae6c163bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a210 comp rx=0 tx=0).stop 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 -- 172.21.15.46:0/1875492218 shutdown_connections 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c0723e0 0x7fae6c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 --2- 172.21.15.46:0/1875492218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c071f80 0x7fae6c163bc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.270+0000 7fae70e00640 1 -- 172.21.15.46:0/1875492218 >> 172.21.15.46:0/1875492218 conn(0x7fae6c06c980 msgr2=0x7fae6c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.271+0000 7fae70e00640 1 -- 172.21.15.46:0/1875492218 shutdown_connections 2024-04-26T23:41:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.271+0000 7fae70e00640 1 -- 172.21.15.46:0/1875492218 wait complete. 2024-04-26T23:41:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.272+0000 7fae70e00640 1 Processor -- start 2024-04-26T23:41:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.272+0000 7fae70e00640 1 -- start start 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 0x7fae6c0aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 0x7fae6c166ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0b42c0 con 0x7fae6c0723e0 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0733e0 con 0x7fae6c15dd60 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.273+0000 7fae70e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0b4440 con 0x7fae6c071f80 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44928/0 (socket says 172.21.15.46:44928) 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 -- 172.21.15.46:0/691678003 learned_addr learned my addr 172.21.15.46:0/691678003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 0x7fae6c0aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae6a800640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 0x7fae6c166ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 -- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 msgr2=0x7fae6c0aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 0x7fae6c0aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 -- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 msgr2=0x7fae6c166ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 0x7fae6c166ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69400640 1 -- 172.21.15.46:0/691678003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0aaab0 con 0x7fae6c0723e0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae6a800640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 0x7fae6c166ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.274+0000 7fae69e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 0x7fae6c0aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae69400640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c041020 con 0x7fae6c0723e0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c030970 con 0x7fae6c0723e0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037040 con 0x7fae6c0723e0 2024-04-26T23:41:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c0aae60 con 0x7fae6c0723e0 2024-04-26T23:41:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.275+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c0ab420 con 0x7fae6c0723e0 2024-04-26T23:41:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.276+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c069530 con 0x7fae6c0723e0 2024-04-26T23:41:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.277+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae5c044020 con 0x7fae6c0723e0 2024-04-26T23:41:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.278+0000 7fae62000640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.278+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7fae5c0ac470 con 0x7fae6c0723e0 2024-04-26T23:41:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.279+0000 7fae69e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.280+0000 7fae69e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fae5400a830 tx=0x7fae54008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.283+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae5c04b030 con 0x7fae6c0723e0 2024-04-26T23:41:47.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.490+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fae6c06ef40 con 0x7fae6c0723e0 2024-04-26T23:41:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.495+0000 7fae62000640 1 -- 172.21.15.46:0/691678003 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fae5c07a970 con 0x7fae6c0723e0 2024-04-26T23:41:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.498+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 msgr2=0x7fae4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.498+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fae5400a830 tx=0x7fae54008040 comp rx=0 tx=0).stop 2024-04-26T23:41:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.498+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 msgr2=0x7fae6c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.498+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 shutdown_connections 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae6c15dd60 0x7fae6c166ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae6c0723e0 0x7fae6c15a350 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 --2- 172.21.15.46:0/691678003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae6c071f80 0x7fae6c0aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 >> 172.21.15.46:0/691678003 conn(0x7fae6c06c980 msgr2=0x7fae6c06dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 shutdown_connections 2024-04-26T23:41:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.499+0000 7fae70e00640 1 -- 172.21.15.46:0/691678003 wait complete. 2024-04-26T23:41:47.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-26T23:41:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.732+0000 7fe795000640 1 Processor -- start 2024-04-26T23:41:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.732+0000 7fe795000640 1 -- start start 2024-04-26T23:41:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe795000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe79006ec20 0x7fe7900734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe795000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe7900b3e80 0x7fe7900739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe795000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7900b43f0 con 0x7fe790142280 2024-04-26T23:41:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe795000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe7900b46d0 con 0x7fe79006ec20 2024-04-26T23:41:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe795000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe7900b49b0 con 0x7fe7900b3e80 2024-04-26T23:41:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe787e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe79006ec20 0x7fe7900734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe787e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe79006ec20 0x7fe7900734a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44938/0 (socket says 172.21.15.46:44938) 2024-04-26T23:41:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.734+0000 7fe787e00640 1 -- 172.21.15.46:0/220515598 learned_addr learned my addr 172.21.15.46:0/220515598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.735+0000 7fe787e00640 1 -- 172.21.15.46:0/220515598 >> v2:172.21.15.142:3300/0 conn(0x7fe7900b3e80 msgr2=0x7fe7900739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.735+0000 7fe787e00640 1 --2- 172.21.15.46:0/220515598 >> v2:172.21.15.142:3300/0 conn(0x7fe7900b3e80 0x7fe7900739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.735+0000 7fe787e00640 1 -- 172.21.15.46:0/220515598 >> v1:172.21.15.142:6789/0 conn(0x7fe790142280 legacy=0x7fe79006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.735+0000 7fe787e00640 1 -- 172.21.15.46:0/220515598 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7900b4c90 con 0x7fe79006ec20 2024-04-26T23:41:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.736+0000 7fe787e00640 1 --2- 172.21.15.46:0/220515598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7900734a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe79006f570 tx=0x7fe77802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6ed91fc727980c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.736+0000 7fe78d400640 1 -- 172.21.15.46:0/220515598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe79006ec20 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.736+0000 7fe78d400640 1 -- 172.21.15.46:0/220515598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe778032440 con 0x7fe79006ec20 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.736+0000 7fe78d400640 1 -- 172.21.15.46:0/220515598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778032810 con 0x7fe79006ec20 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 -- 172.21.15.46:0/220515598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 msgr2=0x7fe7900734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 --2- 172.21.15.46:0/220515598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7900734a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe79006f570 tx=0x7fe77802a230 comp rx=0 tx=0).stop 2024-04-26T23:41:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 -- 172.21.15.46:0/220515598 shutdown_connections 2024-04-26T23:41:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 --2- 172.21.15.46:0/220515598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7900734a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 --2- 172.21.15.46:0/220515598 >> v2:172.21.15.142:3300/0 conn(0x7fe7900b3e80 0x7fe7900739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 -- 172.21.15.46:0/220515598 >> 172.21.15.46:0/220515598 conn(0x7fe79006c530 msgr2=0x7fe79006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 -- 172.21.15.46:0/220515598 shutdown_connections 2024-04-26T23:41:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.737+0000 7fe795000640 1 -- 172.21.15.46:0/220515598 wait complete. 2024-04-26T23:41:47.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.739+0000 7fe795000640 1 Processor -- start 2024-04-26T23:41:47.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.739+0000 7fe795000640 1 -- start start 2024-04-26T23:41:47.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7900b3e80 0x7fe79015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe790142280 0x7fe7900cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe790142a20 con 0x7fe79006ec20 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7901428a0 con 0x7fe7900b3e80 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe795000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe790073f20 con 0x7fe790142280 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe78de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe78de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44950/0 (socket says 172.21.15.46:44950) 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.740+0000 7fe78de00640 1 -- 172.21.15.46:0/1546650166 learned_addr learned my addr 172.21.15.46:0/1546650166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe787e00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7900b3e80 0x7fe79015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78e800640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe790142280 0x7fe7900cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 -- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe790142280 msgr2=0x7fe7900cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe790142280 0x7fe7900cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 -- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7900b3e80 msgr2=0x7fe79015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7900b3e80 0x7fe79015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 -- 172.21.15.46:0/1546650166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79015f1d0 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe78de00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b3e0 tx=0x7fe78000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.741+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780015020 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.742+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe780011850 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.742+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780011c40 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.742+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79015f4b0 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.742+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79015fa70 con 0x7fe79006ec20 2024-04-26T23:41:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.743+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790069530 con 0x7fe79006ec20 2024-04-26T23:41:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.744+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe780018020 con 0x7fe79006ec20 2024-04-26T23:41:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.745+0000 7fe786000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 0x7fe76c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.745+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7fe78008b970 con 0x7fe79006ec20 2024-04-26T23:41:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.746+0000 7fe787e00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 0x7fe76c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.746+0000 7fe787e00640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 0x7fe76c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe77802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:47.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:47.749+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe78001f020 con 0x7fe79006ec20 2024-04-26T23:41:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.035+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) -- 0x7fe790160c40 con 0x7fe79006ec20 2024-04-26T23:41:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.040+0000 7fe786000640 1 -- 172.21.15.46:0/1546650166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v219) ==== 148+0+0 (secure 0 0 0) 0x7fe7800172c0 con 0x7fe79006ec20 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.042+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 msgr2=0x7fe76c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.042+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 0x7fe76c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe77802e000 comp rx=0 tx=0).stop 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.042+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 msgr2=0x7fe7901f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.042+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b3e0 tx=0x7fe78000b8a0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 shutdown_connections 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe76c06c920 0x7fe76c06ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe790142280 0x7fe7900cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7900b3e80 0x7fe79015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 --2- 172.21.15.46:0/1546650166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79006ec20 0x7fe7901f7d80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 >> 172.21.15.46:0/1546650166 conn(0x7fe79006c530 msgr2=0x7fe79015bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 shutdown_connections 2024-04-26T23:41:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.043+0000 7fe795000640 1 -- 172.21.15.46:0/1546650166 wait complete. 2024-04-26T23:41:48.067 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-26T23:41:48.067 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:41:48.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.276+0000 7ff42ea00640 1 Processor -- start 2024-04-26T23:41:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.276+0000 7ff42ea00640 1 -- start start 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.278+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815f1d0 0x7ff4280cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.278+0000 7ff42ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4280cd770 0x7ff4280c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.278+0000 7ff42ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280b03c0 con 0x7ff42815edf0 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.278+0000 7ff42ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280cf5c0 con 0x7ff4280cd770 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.278+0000 7ff42ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280cb680 con 0x7ff42815f1d0 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4280cd770 0x7ff4280c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4280cd770 0x7ff4280c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44952/0 (socket says 172.21.15.46:44952) 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 -- 172.21.15.46:0/1770413691 learned_addr learned my addr 172.21.15.46:0/1770413691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42d600640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff42815edf0 0x7ff4280cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:59974/0 (socket says 172.21.15.46:59974) 2024-04-26T23:41:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42cc00640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815f1d0 0x7ff4280cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 -- 172.21.15.46:0/1770413691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815f1d0 msgr2=0x7ff4280cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815f1d0 0x7ff4280cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 -- 172.21.15.46:0/1770413691 >> v1:172.21.15.142:6789/0 conn(0x7ff42815edf0 legacy=0x7ff4280cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.279+0000 7ff42e000640 1 -- 172.21.15.46:0/1770413691 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280cb960 con 0x7ff4280cd770 2024-04-26T23:41:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.281+0000 7ff42e000640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4280cd770 0x7ff4280c5940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff420009e50 tx=0x7ff42002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a133149bbda2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.281+0000 7ff427e00640 1 -- 172.21.15.46:0/1770413691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420036070 con 0x7ff4280cd770 2024-04-26T23:41:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.281+0000 7ff427e00640 1 -- 172.21.15.46:0/1770413691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420031600 con 0x7ff4280cd770 2024-04-26T23:41:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.281+0000 7ff427e00640 1 -- 172.21.15.46:0/1770413691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200319d0 con 0x7ff4280cd770 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 -- 172.21.15.46:0/1770413691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4280cd770 msgr2=0x7ff4280c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4280cd770 0x7ff4280c5940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff420009e50 tx=0x7ff42002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 -- 172.21.15.46:0/1770413691 shutdown_connections 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4280cd770 0x7ff4280c5940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 --2- 172.21.15.46:0/1770413691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815f1d0 0x7ff4280cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 -- 172.21.15.46:0/1770413691 >> 172.21.15.46:0/1770413691 conn(0x7ff4280b26d0 msgr2=0x7ff4280b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 -- 172.21.15.46:0/1770413691 shutdown_connections 2024-04-26T23:41:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.282+0000 7ff42ea00640 1 -- 172.21.15.46:0/1770413691 wait complete. 2024-04-26T23:41:48.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.284+0000 7ff42ea00640 1 Processor -- start 2024-04-26T23:41:48.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.284+0000 7ff42ea00640 1 -- start start 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4280cd770 0x7ff4281516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815edf0 0x7ff428161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280cc200 con 0x7ff428159f40 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280cbf20 con 0x7ff4280cd770 2024-04-26T23:41:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280cb1a0 con 0x7ff42815edf0 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44960/0 (socket says 172.21.15.46:44960) 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42e000640 1 -- 172.21.15.46:0/251933647 learned_addr learned my addr 172.21.15.46:0/251933647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42d600640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4280cd770 0x7ff4281516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.285+0000 7ff42cc00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815edf0 0x7ff428161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 -- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815edf0 msgr2=0x7ff428161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815edf0 0x7ff428161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 -- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4280cd770 msgr2=0x7ff4281516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4280cd770 0x7ff4281516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 -- 172.21.15.46:0/251933647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42815cbf0 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42e000640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff420009f70 tx=0x7ff420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420036050 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420042070 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420009510 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.286+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42815ced0 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.287+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42815d490 con 0x7ff428159f40 2024-04-26T23:41:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.288+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4280693b0 con 0x7ff428159f40 2024-04-26T23:41:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.289+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff420007bb0 con 0x7ff428159f40 2024-04-26T23:41:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.290+0000 7ff425600640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.290+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7ff4200ac8a0 con 0x7ff428159f40 2024-04-26T23:41:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.290+0000 7ff42d600640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.291+0000 7ff42d600640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 0x7ff40806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c007c20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.294+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff42007ada0 con 0x7ff428159f40 2024-04-26T23:41:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.499+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4280d1810 con 0x7ff428159f40 2024-04-26T23:41:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.501+0000 7ff425600640 1 -- 172.21.15.46:0/251933647 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13257 (secure 0 0 0) 0x7ff42007a6a0 con 0x7ff428159f40 2024-04-26T23:41:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:48.505 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:47.026674+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1027205255":"2024-04-27T23:41:44.015540+0000","172.21.15.46:6833/1027205255":"2024-04-27T23:41:44.015540+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 msgr2=0x7ff40806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 0x7ff40806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c007c20 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 msgr2=0x7ff42815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff420009f70 tx=0x7ff420004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 shutdown_connections 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff428159f40 0x7ff42815a3d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff42815edf0 0x7ff428161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 --2- 172.21.15.46:0/251933647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4280cd770 0x7ff4281516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 >> 172.21.15.46:0/251933647 conn(0x7ff4280b26d0 msgr2=0x7ff4280758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 shutdown_connections 2024-04-26T23:41:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.504+0000 7ff42ea00640 1 -- 172.21.15.46:0/251933647 wait complete. 2024-04-26T23:41:48.527 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:41:48.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:41:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.745+0000 7f695fe00640 1 Processor -- start 2024-04-26T23:41:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.745+0000 7f695fe00640 1 -- start start 2024-04-26T23:41:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69601626c0 0x7f69600b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 0x7f69600d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69600d07d0 con 0x7f6960162aa0 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f69600d0ab0 con 0x7f69601626c0 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f69600d0d90 con 0x7f69600b5fb0 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69601626c0 0x7f69600b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6960162aa0 0x7f69600d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51874/0 (socket says 172.21.15.46:51874) 2024-04-26T23:41:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.747+0000 7f695f400640 1 -- 172.21.15.46:0/2707385719 learned_addr learned my addr 172.21.15.46:0/2707385719 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f6956000640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 0x7f69600d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f695ea00640 1 -- 172.21.15.46:0/2707385719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 msgr2=0x7f69600d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f695ea00640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 0x7f69600d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f695ea00640 1 -- 172.21.15.46:0/2707385719 >> v1:172.21.15.46:6789/0 conn(0x7f6960162aa0 legacy=0x7f69600d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f695ea00640 1 -- 172.21.15.46:0/2707385719 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600d2c80 con 0x7f69601626c0 2024-04-26T23:41:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f695e000640 1 -- 172.21.15.46:0/2707385719 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031126816 0 0) 0x7f69600d07d0 con 0x7f6960162aa0 2024-04-26T23:41:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.748+0000 7f6956000640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 0x7f69600d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.749+0000 7f695ea00640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69601626c0 0x7f69600b47c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f69500097a0 tx=0x7f695002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784ac3380783d9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.749+0000 7f695e000640 1 -- 172.21.15.46:0/2707385719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950037070 con 0x7f69601626c0 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695e000640 1 -- 172.21.15.46:0/2707385719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6950032440 con 0x7f69601626c0 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695e000640 1 -- 172.21.15.46:0/2707385719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950032810 con 0x7f69601626c0 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 -- 172.21.15.46:0/2707385719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69601626c0 msgr2=0x7f69600b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69601626c0 0x7f69600b47c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f69500097a0 tx=0x7f695002a180 comp rx=0 tx=0).stop 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 -- 172.21.15.46:0/2707385719 shutdown_connections 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600b5fb0 0x7f69600d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 --2- 172.21.15.46:0/2707385719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69601626c0 0x7f69600b47c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 -- 172.21.15.46:0/2707385719 >> 172.21.15.46:0/2707385719 conn(0x7f696009e2a0 msgr2=0x7f69600d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 -- 172.21.15.46:0/2707385719 shutdown_connections 2024-04-26T23:41:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.750+0000 7f695fe00640 1 -- 172.21.15.46:0/2707385719 wait complete. 2024-04-26T23:41:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.752+0000 7f695fe00640 1 Processor -- start 2024-04-26T23:41:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.752+0000 7f695fe00640 1 -- start start 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 0x7f696009ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 0x7f696015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69600d2190 con 0x7f6960162aa0 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69600d2010 con 0x7f69600b5fb0 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.753+0000 7f695fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f69600d31a0 con 0x7f69600ce000 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44978/0 (socket says 172.21.15.46:44978) 2024-04-26T23:41:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 -- 172.21.15.46:0/2672351697 learned_addr learned my addr 172.21.15.46:0/2672351697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f695ea00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 0x7f696009ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 -- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 msgr2=0x7f696015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f695f400640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 0x7f696015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 0x7f696015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 -- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 msgr2=0x7f696009ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 0x7f696009ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f6956000640 1 -- 172.21.15.46:0/2672351697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600ce6d0 con 0x7f6960162aa0 2024-04-26T23:41:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f695ea00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 0x7f696009ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.754+0000 7f695f400640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 0x7f696015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f6956000640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f694800c7f0 tx=0x7f694800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948005b90 con 0x7f6960162aa0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6948005d30 con 0x7f6960162aa0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948007a30 con 0x7f6960162aa0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69600ce9b0 con 0x7f6960162aa0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.755+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69600a1380 con 0x7f6960162aa0 2024-04-26T23:41:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.756+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69600a1580 con 0x7f6960162aa0 2024-04-26T23:41:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.757+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6948007bd0 con 0x7f6960162aa0 2024-04-26T23:41:48.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.759+0000 7f6957e00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 0x7f693c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:48.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.759+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f694808bae0 con 0x7f6960162aa0 2024-04-26T23:41:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.760+0000 7f695ea00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 0x7f693c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.761+0000 7f695ea00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 0x7f693c06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6950034000 tx=0x7f695002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:48.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.764+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69600a1580 con 0x7f6960162aa0 2024-04-26T23:41:48.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.968+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69600693b0 con 0x7f6960162aa0 2024-04-26T23:41:48.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.968+0000 7f6957e00640 1 -- 172.21.15.46:0/2672351697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f6948015020 con 0x7f6960162aa0 2024-04-26T23:41:48.972 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 msgr2=0x7f693c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 0x7f693c06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6950034000 tx=0x7f695002e000 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 msgr2=0x7f69600cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f694800c7f0 tx=0x7f694800cbf0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 shutdown_connections 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f693c06c8f0 0x7f693c06edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69600ce000 0x7f696015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6960162aa0 0x7f69600cdac0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 --2- 172.21.15.46:0/2672351697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69600b5fb0 0x7f696009ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.971+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 >> 172.21.15.46:0/2672351697 conn(0x7f696009e2a0 msgr2=0x7f69600b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.972+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 shutdown_connections 2024-04-26T23:41:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:48.972+0000 7f695fe00640 1 -- 172.21.15.46:0/2672351697 wait complete. 2024-04-26T23:41:48.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-26T23:41:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.211+0000 7f6883400640 1 Processor -- start 2024-04-26T23:41:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.211+0000 7f6883400640 1 -- start start 2024-04-26T23:41:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6883400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f687c160bd0 0x7f687c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6883400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f687c0732c0 0x7f687c073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6883400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c161940 con 0x7f687c06e510 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6883400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f687c071f80 con 0x7f687c0732c0 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6883400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f687c072260 con 0x7f687c160bd0 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6881400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f687c0732c0 0x7f687c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6881400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f687c0732c0 0x7f687c073730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44994/0 (socket says 172.21.15.46:44994) 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.213+0000 7f6881400640 1 -- 172.21.15.46:0/3446041107 learned_addr learned my addr 172.21.15.46:0/3446041107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.214+0000 7f687be00640 1 --2- 172.21.15.46:0/3446041107 >> v2:172.21.15.142:3300/0 conn(0x7f687c160bd0 0x7f687c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.214+0000 7f6881400640 1 -- 172.21.15.46:0/3446041107 >> v2:172.21.15.142:3300/0 conn(0x7f687c160bd0 msgr2=0x7f687c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.214+0000 7f6881400640 1 --2- 172.21.15.46:0/3446041107 >> v2:172.21.15.142:3300/0 conn(0x7f687c160bd0 0x7f687c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.214+0000 7f6881400640 1 -- 172.21.15.46:0/3446041107 >> v1:172.21.15.142:6789/0 conn(0x7f687c06e510 legacy=0x7f687c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.214+0000 7f6881400640 1 -- 172.21.15.46:0/3446041107 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c072540 con 0x7f687c0732c0 2024-04-26T23:41:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.215+0000 7f6881400640 1 --2- 172.21.15.46:0/3446041107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c0732c0 0x7f687c073730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a100c2e2edf4ef4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.215+0000 7f687b400640 1 -- 172.21.15.46:0/3446041107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874036070 con 0x7f687c0732c0 2024-04-26T23:41:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.215+0000 7f687b400640 1 -- 172.21.15.46:0/3446041107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874031600 con 0x7f687c0732c0 2024-04-26T23:41:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.215+0000 7f687b400640 1 -- 172.21.15.46:0/3446041107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68740319d0 con 0x7f687c0732c0 2024-04-26T23:41:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 -- 172.21.15.46:0/3446041107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c0732c0 msgr2=0x7f687c073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 --2- 172.21.15.46:0/3446041107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c0732c0 0x7f687c073730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687402a710 comp rx=0 tx=0).stop 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 -- 172.21.15.46:0/3446041107 shutdown_connections 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 --2- 172.21.15.46:0/3446041107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c0732c0 0x7f687c073730 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 --2- 172.21.15.46:0/3446041107 >> v2:172.21.15.142:3300/0 conn(0x7f687c160bd0 0x7f687c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 -- 172.21.15.46:0/3446041107 >> 172.21.15.46:0/3446041107 conn(0x7f687c06c530 msgr2=0x7f687c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 -- 172.21.15.46:0/3446041107 shutdown_connections 2024-04-26T23:41:49.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.216+0000 7f6883400640 1 -- 172.21.15.46:0/3446041107 wait complete. 2024-04-26T23:41:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.218+0000 7f6883400640 1 Processor -- start 2024-04-26T23:41:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.218+0000 7f6883400640 1 -- start start 2024-04-26T23:41:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c06e510 0x7f687c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687c0732c0 0x7f687c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f687c161720 con 0x7f687c160bd0 2024-04-26T23:41:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1615a0 con 0x7f687c06e510 2024-04-26T23:41:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6883400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f687c072d40 con 0x7f687c0732c0 2024-04-26T23:41:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6881400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6881400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45008/0 (socket says 172.21.15.46:45008) 2024-04-26T23:41:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.219+0000 7f6881400640 1 -- 172.21.15.46:0/898575971 learned_addr learned my addr 172.21.15.46:0/898575971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6880a00640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c06e510 0x7f687c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f687be00640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687c0732c0 0x7f687c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 -- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687c0732c0 msgr2=0x7f687c1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687c0732c0 0x7f687c1f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 -- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c06e510 msgr2=0x7f687c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c06e510 0x7f687c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 -- 172.21.15.46:0/898575971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0b3280 con 0x7f687c160bd0 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.220+0000 7f6881400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6874009f70 tx=0x7f6874004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.221+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874036050 con 0x7f687c160bd0 2024-04-26T23:41:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.221+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874042070 con 0x7f687c160bd0 2024-04-26T23:41:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.221+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874009510 con 0x7f687c160bd0 2024-04-26T23:41:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.221+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c0b3560 con 0x7f687c160bd0 2024-04-26T23:41:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.221+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c1547e0 con 0x7f687c160bd0 2024-04-26T23:41:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.222+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c069530 con 0x7f687c160bd0 2024-04-26T23:41:49.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.223+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6874007bb0 con 0x7f687c160bd0 2024-04-26T23:41:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.224+0000 7f6878c00640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 0x7f685c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.224+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f68740ac860 con 0x7f687c160bd0 2024-04-26T23:41:49.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.225+0000 7f6880a00640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 0x7f685c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:49.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.226+0000 7f6880a00640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 0x7f685c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6870006fd0 tx=0x7f6870008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.229+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f687407adf0 con 0x7f687c160bd0 2024-04-26T23:41:49.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:49.500+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) -- 0x7f687c06fb00 con 0x7f687c160bd0 2024-04-26T23:41:50.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.055+0000 7f6878c00640 1 -- 172.21.15.46:0/898575971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v221) ==== 146+0+0 (secure 0 0 0) 0x7f687407a6f0 con 0x7f687c160bd0 2024-04-26T23:41:50.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.058+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 msgr2=0x7f685c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.058+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 0x7f685c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6870006fd0 tx=0x7f6870008040 comp rx=0 tx=0).stop 2024-04-26T23:41:50.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.058+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 msgr2=0x7f687c1f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.058+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6874009f70 tx=0x7f6874004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 shutdown_connections 2024-04-26T23:41:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685c06c8f0 0x7f685c06edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c160bd0 0x7f687c1f8980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687c0732c0 0x7f687c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 --2- 172.21.15.46:0/898575971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c06e510 0x7f687c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 >> 172.21.15.46:0/898575971 conn(0x7f687c06c530 msgr2=0x7f687c06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 shutdown_connections 2024-04-26T23:41:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.059+0000 7f6883400640 1 -- 172.21.15.46:0/898575971 wait complete. 2024-04-26T23:41:50.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:41:50.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.299+0000 7f9a5cc00640 1 Processor -- start 2024-04-26T23:41:50.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.299+0000 7f9a5cc00640 1 -- start start 2024-04-26T23:41:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.301+0000 7f9a5cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a5806d460 0x7f9a580d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.301+0000 7f9a5cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a5806d8c0 0x7f9a580d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.301+0000 7f9a5cc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5815f6e0 con 0x7f9a5815f300 2024-04-26T23:41:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.301+0000 7f9a5cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9a580cfc50 con 0x7f9a5806d460 2024-04-26T23:41:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.301+0000 7f9a5cc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9a580cff30 con 0x7f9a5806d8c0 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a5806d460 0x7f9a580d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a5806d460 0x7f9a580d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45016/0 (socket says 172.21.15.46:45016) 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 -- 172.21.15.46:0/184873986 learned_addr learned my addr 172.21.15.46:0/184873986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a56800640 1 --2- 172.21.15.46:0/184873986 >> v2:172.21.15.142:3300/0 conn(0x7f9a5806d8c0 0x7f9a580d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 -- 172.21.15.46:0/184873986 >> v2:172.21.15.142:3300/0 conn(0x7f9a5806d8c0 msgr2=0x7f9a580d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 --2- 172.21.15.46:0/184873986 >> v2:172.21.15.142:3300/0 conn(0x7f9a5806d8c0 0x7f9a580d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 -- 172.21.15.46:0/184873986 >> v1:172.21.15.142:6789/0 conn(0x7f9a5815f300 legacy=0x7f9a580d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.302+0000 7f9a55400640 1 -- 172.21.15.46:0/184873986 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a580d0210 con 0x7f9a5806d460 2024-04-26T23:41:50.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.304+0000 7f9a55400640 1 --2- 172.21.15.46:0/184873986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5806d460 0x7f9a580d2e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9a580d2d90 tx=0x7f9a4002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c2ee2ea83caab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:50.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.304+0000 7f9a54a00640 1 -- 172.21.15.46:0/184873986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40037070 con 0x7f9a5806d460 2024-04-26T23:41:50.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.304+0000 7f9a54a00640 1 -- 172.21.15.46:0/184873986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40032440 con 0x7f9a5806d460 2024-04-26T23:41:50.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.304+0000 7f9a54a00640 1 -- 172.21.15.46:0/184873986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40032810 con 0x7f9a5806d460 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 -- 172.21.15.46:0/184873986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5806d460 msgr2=0x7f9a580d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/184873986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5806d460 0x7f9a580d2e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9a580d2d90 tx=0x7f9a4002a230 comp rx=0 tx=0).stop 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 -- 172.21.15.46:0/184873986 shutdown_connections 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/184873986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5806d460 0x7f9a580d2e60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/184873986 >> v2:172.21.15.142:3300/0 conn(0x7f9a5806d8c0 0x7f9a580d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 -- 172.21.15.46:0/184873986 >> 172.21.15.46:0/184873986 conn(0x7f9a5809ba70 msgr2=0x7f9a5809be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 -- 172.21.15.46:0/184873986 shutdown_connections 2024-04-26T23:41:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.305+0000 7f9a5cc00640 1 -- 172.21.15.46:0/184873986 wait complete. 2024-04-26T23:41:50.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.308+0000 7f9a5cc00640 1 Processor -- start 2024-04-26T23:41:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.308+0000 7f9a5cc00640 1 -- start start 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a5806d460 0x7f9a58154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 0x7f9a580cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58163230 con 0x7f9a5815f300 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580cf8e0 con 0x7f9a5806d8c0 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.309+0000 7f9a5cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a581633b0 con 0x7f9a5806d460 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45030/0 (socket says 172.21.15.46:45030) 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 -- 172.21.15.46:0/2054201615 learned_addr learned my addr 172.21.15.46:0/2054201615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a55400640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 0x7f9a580cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a55e00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a5806d460 0x7f9a58154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 -- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a5806d460 msgr2=0x7f9a58154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a5806d460 0x7f9a58154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 -- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 msgr2=0x7f9a580cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 0x7f9a580cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 -- 172.21.15.46:0/2054201615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5815f8d0 con 0x7f9a5815f300 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a55400640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 0x7f9a580cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.310+0000 7f9a56800640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d5d0 tx=0x7f9a5000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.311+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500125f0 con 0x7f9a5815f300 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.311+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a50019070 con 0x7f9a5815f300 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.311+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a58160650 con 0x7f9a5815f300 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.311+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500129d0 con 0x7f9a5815f300 2024-04-26T23:41:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.311+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5815fb70 con 0x7f9a5815f300 2024-04-26T23:41:50.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.312+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a58069530 con 0x7f9a5815f300 2024-04-26T23:41:50.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.313+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a50012b70 con 0x7f9a5815f300 2024-04-26T23:41:50.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.314+0000 7f9a4e000640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 0x7f9a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:50.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.314+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f9a5008be40 con 0x7f9a5815f300 2024-04-26T23:41:50.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.315+0000 7f9a55e00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 0x7f9a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:50.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.316+0000 7f9a55e00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 0x7f9a3406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a580c99e0 tx=0x7f9a480094d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.318+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a5005a3d0 con 0x7f9a5815f300 2024-04-26T23:41:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.523+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9a580b8e50 con 0x7f9a5815f300 2024-04-26T23:41:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.528+0000 7f9a4e000640 1 -- 172.21.15.46:0/2054201615 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f9a50059cd0 con 0x7f9a5815f300 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.531+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 msgr2=0x7f9a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.531+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 0x7f9a3406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a580c99e0 tx=0x7f9a480094d0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.531+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 msgr2=0x7f9a580d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.531+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d5d0 tx=0x7f9a5000da90 comp rx=0 tx=0).stop 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 shutdown_connections 2024-04-26T23:41:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a3406c8f0 0x7f9a3406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5815f300 0x7f9a580d1060 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a5806d8c0 0x7f9a580cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 --2- 172.21.15.46:0/2054201615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a5806d460 0x7f9a58154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 >> 172.21.15.46:0/2054201615 conn(0x7f9a5809ba70 msgr2=0x7f9a580cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 shutdown_connections 2024-04-26T23:41:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:50.532+0000 7f9a5cc00640 1 -- 172.21.15.46:0/2054201615 wait complete. 2024-04-26T23:41:50.553 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:41:50.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:41:50.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:41:50.774 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:41:50.774 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:41:50.774 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:50.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:41:50.834 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:41:50.834 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:50.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:41:50.891 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:41:50.891 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:50.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:41:50.953 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:41:50.953 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:41:51.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:41:51.009 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:41:51.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:41:51.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.225+0000 7f1b08c00640 1 Processor -- start 2024-04-26T23:41:51.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.226+0000 7f1b08c00640 1 -- start start 2024-04-26T23:41:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b08c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b041466b0 0x7f1b0406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b08c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b04160910 0x7f1b04160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b08c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b04073ad0 con 0x7f1b040720e0 2024-04-26T23:41:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b08c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b041612a0 con 0x7f1b04160910 2024-04-26T23:41:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b08c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b04146a90 con 0x7f1b041466b0 2024-04-26T23:41:51.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.227+0000 7f1b03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b04160910 0x7f1b04160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b04160910 0x7f1b04160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45032/0 (socket says 172.21.15.46:45032) 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 -- 172.21.15.46:0/160438881 learned_addr learned my addr 172.21.15.46:0/160438881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03400640 1 --2- 172.21.15.46:0/160438881 >> v2:172.21.15.142:3300/0 conn(0x7f1b041466b0 0x7f1b0406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 -- 172.21.15.46:0/160438881 >> v2:172.21.15.142:3300/0 conn(0x7f1b041466b0 msgr2=0x7f1b0406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 --2- 172.21.15.46:0/160438881 >> v2:172.21.15.142:3300/0 conn(0x7f1b041466b0 0x7f1b0406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 -- 172.21.15.46:0/160438881 >> v1:172.21.15.46:6789/0 conn(0x7f1b040720e0 legacy=0x7f1b040cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.228+0000 7f1b03e00640 1 -- 172.21.15.46:0/160438881 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b04073d70 con 0x7f1b04160910 2024-04-26T23:41:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b03e00640 1 --2- 172.21.15.46:0/160438881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04160d60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff901315196d5cff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b02a00640 1 -- 172.21.15.46:0/160438881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc036070 con 0x7f1b04160910 2024-04-26T23:41:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b02a00640 1 -- 172.21.15.46:0/160438881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc031600 con 0x7f1b04160910 2024-04-26T23:41:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b02a00640 1 -- 172.21.15.46:0/160438881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0319d0 con 0x7f1b04160910 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b08c00640 1 -- 172.21.15.46:0/160438881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 msgr2=0x7f1b04160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.230+0000 7f1b08c00640 1 --2- 172.21.15.46:0/160438881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04160d60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc02a760 comp rx=0 tx=0).stop 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 -- 172.21.15.46:0/160438881 shutdown_connections 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 --2- 172.21.15.46:0/160438881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04160d60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 --2- 172.21.15.46:0/160438881 >> v2:172.21.15.142:3300/0 conn(0x7f1b041466b0 0x7f1b0406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 -- 172.21.15.46:0/160438881 >> 172.21.15.46:0/160438881 conn(0x7f1b040b26d0 msgr2=0x7f1b040b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 -- 172.21.15.46:0/160438881 shutdown_connections 2024-04-26T23:41:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.231+0000 7f1b08c00640 1 -- 172.21.15.46:0/160438881 wait complete. 2024-04-26T23:41:51.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.232+0000 7f1b08c00640 1 Processor -- start 2024-04-26T23:41:51.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.233+0000 7f1b08c00640 1 -- start start 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b040720e0 0x7f1b04140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b041466b0 0x7f1b041429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b040cb470 con 0x7f1b04160910 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b040cb190 con 0x7f1b040720e0 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b08c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0406e3c0 con 0x7f1b041466b0 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45044/0 (socket says 172.21.15.46:45044) 2024-04-26T23:41:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 -- 172.21.15.46:0/1749259382 learned_addr learned my addr 172.21.15.46:0/1749259382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03400640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b040720e0 0x7f1b04140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1afaa00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b041466b0 0x7f1b041429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 -- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b041466b0 msgr2=0x7f1b041429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b041466b0 0x7f1b041429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 -- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b040720e0 msgr2=0x7f1b04140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b040720e0 0x7f1b04140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.234+0000 7f1b03e00640 1 -- 172.21.15.46:0/1749259382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b04141950 con 0x7f1b04160910 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1b03e00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc036050 con 0x7f1b04160910 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc042070 con 0x7f1b04160910 2024-04-26T23:41:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc009510 con 0x7f1b04160910 2024-04-26T23:41:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b04141c30 con 0x7f1b04160910 2024-04-26T23:41:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.235+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b041421f0 con 0x7f1b04160910 2024-04-26T23:41:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.237+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b040693b0 con 0x7f1b04160910 2024-04-26T23:41:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.238+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1afc007bd0 con 0x7f1b04160910 2024-04-26T23:41:51.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.239+0000 7f1afbe00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.239+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f1afc0ac7c0 con 0x7f1b04160910 2024-04-26T23:41:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.239+0000 7f1b03400640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.241+0000 7f1b03400640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 0x7f1ae006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1af4006fd0 tx=0x7f1af4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.244+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1afc045360 con 0x7f1b04160910 2024-04-26T23:41:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.490+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b040c7180 con 0x7f1b04160910 2024-04-26T23:41:51.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.491+0000 7f1afbe00640 1 -- 172.21.15.46:0/1749259382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+3875 (secure 0 0 0) 0x7f1afc07ad60 con 0x7f1b04160910 2024-04-26T23:41:51.495 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:51.495 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3876034456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3876034456},{"type":"v1","addr":"172.21.15.142:6837","nonce":3876034456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1668102552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1668102552},{"type":"v1","addr":"172.21.15.142:6835","nonce":1668102552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1264841868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1264841868},{"type":"v1","addr":"172.21.15.46:6834","nonce":1264841868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2357357157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2357357157},{"type":"v1","addr":"172.21.15.46:6835","nonce":2357357157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[]} 2024-04-26T23:41:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 21 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 msgr2=0x7f1ae006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 0x7f1ae006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1af4006fd0 tx=0x7f1af4008040 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 msgr2=0x7f1b04140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 shutdown_connections 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b04160910 0x7f1b04140e90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b041466b0 0x7f1b041429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 --2- 172.21.15.46:0/1749259382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b040720e0 0x7f1b04140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.494+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 >> 172.21.15.46:0/1749259382 conn(0x7f1b040b26d0 msgr2=0x7f1b0406f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.495+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 shutdown_connections 2024-04-26T23:41:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.495+0000 7f1b08c00640 1 -- 172.21.15.46:0/1749259382 wait complete. 2024-04-26T23:41:51.517 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:41:51.517 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:41:51.517 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:41:51.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:41:51.517 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:41:51.518 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:41:51.518 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:41:51.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:41:51.518 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:41:51.518 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:41:51.518 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:41:51.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:41:51.519 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:41:51.519 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:41:51.519 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:41:51.519 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:41:51.534 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:41:51.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:41:51.537 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:41:51.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:41:51.540 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:41:51.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:41:51.548 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:41:51.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:41:51.584 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:41:51.584 DEBUG:teuthology.parallel:result is None 2024-04-26T23:41:51.590 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:41:51.591 DEBUG:teuthology.parallel:result is None 2024-04-26T23:41:51.596 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:41:51.596 DEBUG:teuthology.parallel:result is None 2024-04-26T23:41:51.602 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:41:51.602 DEBUG:teuthology.parallel:result is None 2024-04-26T23:41:51.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:41:51.741 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:41:51.741 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:41:51.750 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:41:51.757 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:41:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.786+0000 7f81ca000640 1 Processor -- start 2024-04-26T23:41:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.786+0000 7f81ca000640 1 -- start start 2024-04-26T23:41:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.787+0000 7f81ca000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 0x7f81c406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.787+0000 7f81ca000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.787+0000 7f81ca000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f81c415d540 0x7f81c415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.787+0000 7f81ca000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f81c406ee30 con 0x7f81c4160bd0 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81ca000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f81c4161940 con 0x7f81c415d540 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81ca000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81c4071f80 con 0x7f81c406e510 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45102/0 (socket says 172.21.15.46:45102) 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81bbe00640 1 -- 172.21.15.46:0/2824942348 learned_addr learned my addr 172.21.15.46:0/2824942348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 --2- 172.21.15.46:0/2824942348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 0x7f81c406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3e00640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.142:3300/0 conn(0x7f81c415d540 0x7f81c415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 -- 172.21.15.46:0/2824942348 >> v2:172.21.15.142:3300/0 conn(0x7f81c415d540 msgr2=0x7f81c415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.142:3300/0 conn(0x7f81c415d540 0x7f81c415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 -- 172.21.15.46:0/2824942348 >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 msgr2=0x7f81c4161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81c3400640 1 -- 172.21.15.46:0/2824942348 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c4072260 con 0x7f81c406e510 2024-04-26T23:41:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.788+0000 7f81bbe00640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.789+0000 7f81c3400640 1 --2- 172.21.15.46:0/2824942348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 0x7f81c406e8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81c40ce520 tx=0x7f81b402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d087e43bd12b1af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81c2a00640 1 -- 172.21.15.46:0/2824942348 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4037070 con 0x7f81c406e510 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81c2a00640 1 -- 172.21.15.46:0/2824942348 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b4032440 con 0x7f81c406e510 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81c2a00640 1 -- 172.21.15.46:0/2824942348 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b402ae60 con 0x7f81c406e510 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 -- 172.21.15.46:0/2824942348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 msgr2=0x7f81c406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 --2- 172.21.15.46:0/2824942348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 0x7f81c406e8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81c40ce520 tx=0x7f81b402a110 comp rx=0 tx=0).stop 2024-04-26T23:41:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 -- 172.21.15.46:0/2824942348 shutdown_connections 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.142:3300/0 conn(0x7f81c415d540 0x7f81c415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 --2- 172.21.15.46:0/2824942348 >> v2:172.21.15.46:3300/0 conn(0x7f81c4160bd0 0x7f81c4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 --2- 172.21.15.46:0/2824942348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c406e510 0x7f81c406e8f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 -- 172.21.15.46:0/2824942348 >> 172.21.15.46:0/2824942348 conn(0x7f81c406c530 msgr2=0x7f81c406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 -- 172.21.15.46:0/2824942348 shutdown_connections 2024-04-26T23:41:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.790+0000 7f81ca000640 1 -- 172.21.15.46:0/2824942348 wait complete. 2024-04-26T23:41:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.792+0000 7f81ca000640 1 Processor -- start 2024-04-26T23:41:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.792+0000 7f81ca000640 1 -- start start 2024-04-26T23:41:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c415d540 0x7f81c40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c4160bd0 0x7f81c4142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4161570 con 0x7f81c406e510 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4072820 con 0x7f81c415d540 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.793+0000 7f81ca000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81c41616f0 con 0x7f81c4160bd0 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45112/0 (socket says 172.21.15.46:45112) 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 -- 172.21.15.46:0/997418470 learned_addr learned my addr 172.21.15.46:0/997418470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3e00640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c4160bd0 0x7f81c4142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81bbe00640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c415d540 0x7f81c40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 -- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c4160bd0 msgr2=0x7f81c4142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c4160bd0 0x7f81c4142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 -- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c415d540 msgr2=0x7f81c40b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c415d540 0x7f81c40b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 -- 172.21.15.46:0/997418470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c4143940 con 0x7f81c406e510 2024-04-26T23:41:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.794+0000 7f81c3400640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009890 tx=0x7f81b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.795+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4041020 con 0x7f81c406e510 2024-04-26T23:41:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.795+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b403d640 con 0x7f81c406e510 2024-04-26T23:41:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.795+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4037040 con 0x7f81c406e510 2024-04-26T23:41:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.795+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c4158210 con 0x7f81c406e510 2024-04-26T23:41:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.795+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41587d0 con 0x7f81c406e510 2024-04-26T23:41:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.796+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c4069530 con 0x7f81c406e510 2024-04-26T23:41:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.797+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81b4044020 con 0x7f81c406e510 2024-04-26T23:41:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.798+0000 7f81bb400640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.799+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f81b40ac100 con 0x7f81c406e510 2024-04-26T23:41:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.799+0000 7f81bbe00640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.800+0000 7f81bbe00640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 0x7f81a006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f81c413d490 tx=0x7f81ac009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:51.803+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81b404b030 con 0x7f81c406e510 2024-04-26T23:41:52.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.019+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f81c406fb00 con 0x7f81c406e510 2024-04-26T23:41:52.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.084+0000 7f81bb400640 1 -- 172.21.15.46:0/997418470 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7f81b4079fc0 con 0x7f81c406e510 2024-04-26T23:41:52.088 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 msgr2=0x7f81a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 0x7f81a006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f81c413d490 tx=0x7f81ac009240 comp rx=0 tx=0).stop 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 msgr2=0x7f81c4158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009890 tx=0x7f81b40046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 shutdown_connections 2024-04-26T23:41:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81c4160bd0 0x7f81c4142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c415d540 0x7f81c40b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 --2- 172.21.15.46:0/997418470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c406e510 0x7f81c4158cf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 >> 172.21.15.46:0/997418470 conn(0x7f81c406c530 msgr2=0x7f81c415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 shutdown_connections 2024-04-26T23:41:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.088+0000 7f81ca000640 1 -- 172.21.15.46:0/997418470 wait complete. 2024-04-26T23:41:52.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:41:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.315+0000 7feb0e800640 1 Processor -- start 2024-04-26T23:41:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.315+0000 7feb0e800640 1 -- start start 2024-04-26T23:41:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.317+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb080cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.317+0000 7feb0e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb0806f2d0 0x7feb08072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.317+0000 7feb0e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb080d0730 con 0x7feb08071ef0 2024-04-26T23:41:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.317+0000 7feb0e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feb080b3bc0 con 0x7feb0806f2d0 2024-04-26T23:41:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.317+0000 7feb0e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb080b3ea0 con 0x7feb0806e7e0 2024-04-26T23:41:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.318+0000 7feb0de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb0806f2d0 0x7feb08072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.318+0000 7feb0d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb080cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.318+0000 7feb0de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb0806f2d0 0x7feb08072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:47532/0 (socket says 172.21.15.46:47532) 2024-04-26T23:41:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.318+0000 7feb0de00640 1 -- 172.21.15.46:0/1905385461 learned_addr learned my addr 172.21.15.46:0/1905385461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.318+0000 7feb0ca00640 1 --1- 172.21.15.46:0/1905385461 >> v1:172.21.15.142:6789/0 conn(0x7feb08071ef0 0x7feb0806ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:39594/0 (socket says 172.21.15.46:39594) 2024-04-26T23:41:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.319+0000 7feb0de00640 1 -- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 msgr2=0x7feb080cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.319+0000 7feb0de00640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb080cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.319+0000 7feb0de00640 1 -- 172.21.15.46:0/1905385461 >> v1:172.21.15.142:6789/0 conn(0x7feb08071ef0 legacy=0x7feb0806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.319+0000 7feb0de00640 1 -- 172.21.15.46:0/1905385461 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb080b4180 con 0x7feb0806f2d0 2024-04-26T23:41:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.319+0000 7feb0d400640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb080cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:41:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.320+0000 7feb0de00640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb08072de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb0002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2950abb4392f6bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.320+0000 7feb07e00640 1 -- 172.21.15.46:0/1905385461 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00036070 con 0x7feb0806f2d0 2024-04-26T23:41:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb07e00640 1 -- 172.21.15.46:0/1905385461 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb00031600 con 0x7feb0806f2d0 2024-04-26T23:41:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb07e00640 1 -- 172.21.15.46:0/1905385461 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000319d0 con 0x7feb0806f2d0 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 -- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 msgr2=0x7feb08072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb08072de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb0002a680 comp rx=0 tx=0).stop 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 -- 172.21.15.46:0/1905385461 shutdown_connections 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb08072de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 --2- 172.21.15.46:0/1905385461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb080cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 -- 172.21.15.46:0/1905385461 >> 172.21.15.46:0/1905385461 conn(0x7feb080b2820 msgr2=0x7feb080b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 -- 172.21.15.46:0/1905385461 shutdown_connections 2024-04-26T23:41:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.321+0000 7feb0e800640 1 -- 172.21.15.46:0/1905385461 wait complete. 2024-04-26T23:41:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.323+0000 7feb0e800640 1 Processor -- start 2024-04-26T23:41:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.323+0000 7feb0e800640 1 -- start start 2024-04-26T23:41:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb0815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb081462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb080b4c60 con 0x7feb08071ef0 2024-04-26T23:41:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb080b4740 con 0x7feb0806f2d0 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.324+0000 7feb0e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb080b4de0 con 0x7feb0806e7e0 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36228/0 (socket says 172.21.15.46:36228) 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 -- 172.21.15.46:0/4075539464 learned_addr learned my addr 172.21.15.46:0/4075539464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0d400640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb0815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0ca00640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb081462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 -- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 msgr2=0x7feb0815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb0815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 -- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 msgr2=0x7feb081462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb081462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.325+0000 7feb0de00640 1 -- 172.21.15.46:0/4075539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0815c860 con 0x7feb08071ef0 2024-04-26T23:41:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb0de00640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00041020 con 0x7feb08071ef0 2024-04-26T23:41:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000077a0 con 0x7feb08071ef0 2024-04-26T23:41:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0815cb40 con 0x7feb08071ef0 2024-04-26T23:41:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00036040 con 0x7feb08071ef0 2024-04-26T23:41:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.326+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0815d100 con 0x7feb08071ef0 2024-04-26T23:41:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.327+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0814d0f0 con 0x7feb08071ef0 2024-04-26T23:41:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.328+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb00044020 con 0x7feb08071ef0 2024-04-26T23:41:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.329+0000 7feb05600640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.329+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7feb000ac610 con 0x7feb08071ef0 2024-04-26T23:41:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.330+0000 7feb0d400640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.331+0000 7feb0d400640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 0x7feae806ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feafc009a30 tx=0x7feafc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.334+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb0004b030 con 0x7feb08071ef0 2024-04-26T23:41:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.615+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feb080693b0 con 0x7feb08071ef0 2024-04-26T23:41:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.616+0000 7feb05600640 1 -- 172.21.15.46:0/4075539464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7feb0007ab60 con 0x7feb08071ef0 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:41:52.622 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.624 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:41:52.625 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:41:52.627 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:41:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:41:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.630 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:41:52.632 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 msgr2=0x7feae806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 0x7feae806ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feafc009a30 tx=0x7feafc008040 comp rx=0 tx=0).stop 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 msgr2=0x7feb08151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb00004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 shutdown_connections 2024-04-26T23:41:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08071ef0 0x7feb08151ce0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0806f2d0 0x7feb081462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 --2- 172.21.15.46:0/4075539464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0806e7e0 0x7feb0815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 >> 172.21.15.46:0/4075539464 conn(0x7feb080b2820 msgr2=0x7feb081538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 shutdown_connections 2024-04-26T23:41:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.619+0000 7feb0e800640 1 -- 172.21.15.46:0/4075539464 wait complete. 2024-04-26T23:41:52.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.848+0000 7f0299e00640 1 Processor -- start 2024-04-26T23:41:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.848+0000 7f0299e00640 1 -- start start 2024-04-26T23:41:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.850+0000 7f0299e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02941466b0 0x7f029406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.850+0000 7f0299e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0294160910 0x7f0294160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.850+0000 7f0299e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0294073ad0 con 0x7f02940720e0 2024-04-26T23:41:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.850+0000 7f0299e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f02941612a0 con 0x7f02941466b0 2024-04-26T23:41:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.850+0000 7f0299e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0294146a90 con 0x7f0294160910 2024-04-26T23:41:52.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02941466b0 0x7f029406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02941466b0 0x7f029406de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36238/0 (socket says 172.21.15.46:36238) 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 -- 172.21.15.46:0/3539990541 learned_addr learned my addr 172.21.15.46:0/3539990541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0299400640 1 --2- 172.21.15.46:0/3539990541 >> v2:172.21.15.142:3300/0 conn(0x7f0294160910 0x7f0294160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 -- 172.21.15.46:0/3539990541 >> v2:172.21.15.142:3300/0 conn(0x7f0294160910 msgr2=0x7f0294160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 --2- 172.21.15.46:0/3539990541 >> v2:172.21.15.142:3300/0 conn(0x7f0294160910 0x7f0294160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 -- 172.21.15.46:0/3539990541 >> v1:172.21.15.142:6789/0 conn(0x7f02940720e0 legacy=0x7f02940cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.851+0000 7f0298a00640 1 -- 172.21.15.46:0/3539990541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0294073d70 con 0x7f02941466b0 2024-04-26T23:41:52.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.853+0000 7f0298a00640 1 --2- 172.21.15.46:0/3539990541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029406de80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0288009870 tx=0x7f028802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cecf7fb7f66b6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.853+0000 7f0293400640 1 -- 172.21.15.46:0/3539990541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288037070 con 0x7f02941466b0 2024-04-26T23:41:52.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.853+0000 7f0293400640 1 -- 172.21.15.46:0/3539990541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0288032440 con 0x7f02941466b0 2024-04-26T23:41:52.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.853+0000 7f0293400640 1 -- 172.21.15.46:0/3539990541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288032810 con 0x7f02941466b0 2024-04-26T23:41:52.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 -- 172.21.15.46:0/3539990541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 msgr2=0x7f029406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 --2- 172.21.15.46:0/3539990541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029406de80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0288009870 tx=0x7f028802a270 comp rx=0 tx=0).stop 2024-04-26T23:41:52.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 -- 172.21.15.46:0/3539990541 shutdown_connections 2024-04-26T23:41:52.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 --2- 172.21.15.46:0/3539990541 >> v2:172.21.15.142:3300/0 conn(0x7f0294160910 0x7f0294160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 --2- 172.21.15.46:0/3539990541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029406de80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 -- 172.21.15.46:0/3539990541 >> 172.21.15.46:0/3539990541 conn(0x7f02940b26d0 msgr2=0x7f02940b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 -- 172.21.15.46:0/3539990541 shutdown_connections 2024-04-26T23:41:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.854+0000 7f0299e00640 1 -- 172.21.15.46:0/3539990541 wait complete. 2024-04-26T23:41:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.856+0000 7f0299e00640 1 Processor -- start 2024-04-26T23:41:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.856+0000 7f0299e00640 1 -- start start 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 0x7f029415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0294160910 0x7f02941e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02940cb470 con 0x7f02941466b0 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02940cb190 con 0x7f02940720e0 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0299e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f029406e3c0 con 0x7f0294160910 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0293e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0293e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36248/0 (socket says 172.21.15.46:36248) 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.857+0000 7f0293e00640 1 -- 172.21.15.46:0/4221083450 learned_addr learned my addr 172.21.15.46:0/4221083450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0299400640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0294160910 0x7f02941e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 -- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0294160910 msgr2=0x7f02941e7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0298a00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 0x7f029415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0294160910 0x7f02941e7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 -- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 msgr2=0x7f029415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 0x7f029415f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 -- 172.21.15.46:0/4221083450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941e6be0 con 0x7f02941466b0 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0298a00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 0x7f029415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0293e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f028000b3f0 tx=0x7f028000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280015020 con 0x7f02941466b0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280011830 con 0x7f02941466b0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280011c20 con 0x7f02941466b0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941e6ec0 con 0x7f02941466b0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.858+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941e7480 con 0x7f02941466b0 2024-04-26T23:41:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.860+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02940693b0 con 0x7f02941466b0 2024-04-26T23:41:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.861+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0280018020 con 0x7f02941466b0 2024-04-26T23:41:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.862+0000 7f0290c00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 0x7f027406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.862+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f028008b6d0 con 0x7f02941466b0 2024-04-26T23:41:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.862+0000 7f0298a00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 0x7f027406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.864+0000 7f0298a00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 0x7f027406edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0288034000 tx=0x7f028802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:52.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:52.866+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f028001f030 con 0x7f02941466b0 2024-04-26T23:41:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.072+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f029413b600 con 0x7f02941466b0 2024-04-26T23:41:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.074+0000 7f0290c00640 1 -- 172.21.15.46:0/4221083450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13128 (secure 0 0 0) 0x7f02800172c0 con 0x7f02941466b0 2024-04-26T23:41:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:53.078 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:52.099457+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.078+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 msgr2=0x7f027406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.078+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 0x7f027406edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0288034000 tx=0x7f028802e000 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.078+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 msgr2=0x7f029413e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.078+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f028000b3f0 tx=0x7f028000b8b0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 shutdown_connections 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027406c8f0 0x7f027406edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0294160910 0x7f02941e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02941466b0 0x7f029413e3f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 --2- 172.21.15.46:0/4221083450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02940720e0 0x7f029415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 >> 172.21.15.46:0/4221083450 conn(0x7f02940b26d0 msgr2=0x7f0294144790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 shutdown_connections 2024-04-26T23:41:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.079+0000 7f0299e00640 1 -- 172.21.15.46:0/4221083450 wait complete. 2024-04-26T23:41:53.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:41:53.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:41:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.315+0000 7f6eb5a00640 1 Processor -- start 2024-04-26T23:41:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.316+0000 7f6eb5a00640 1 -- start start 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.317+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 0x7f6eb00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.317+0000 7f6eb5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6eb00b53b0 0x7f6eb015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.317+0000 7f6eb5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb006eb90 con 0x7f6eb015e050 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.317+0000 7f6eb5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00b4310 con 0x7f6eb00b53b0 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.317+0000 7f6eb5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b45f0 con 0x7f6eb015d730 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6eaf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6eb015e050 0x7f6eb006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48148/0 (socket says 172.21.15.46:48148) 2024-04-26T23:41:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6eaf400640 1 -- 172.21.15.46:0/3295231059 learned_addr learned my addr 172.21.15.46:0/3295231059 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6ea7e00640 1 --2- 172.21.15.46:0/3295231059 >> v2:172.21.15.142:3300/0 conn(0x7f6eb00b53b0 0x7f6eb015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 0x7f6eb00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483519934 0 0) 0x7f6eb006eb90 con 0x7f6eb015e050 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.318+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e90003520 con 0x7f6eb015e050 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305568372 0 0) 0x7f6e90003520 con 0x7f6eb015e050 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 msgr2=0x7f6eb00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 0x7f6eb00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 >> v2:172.21.15.142:3300/0 conn(0x7f6eb00b53b0 msgr2=0x7f6eb015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 --2- 172.21.15.46:0/3295231059 >> v2:172.21.15.142:3300/0 conn(0x7f6eb00b53b0 0x7f6eb015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00b48d0 con 0x7f6eb015e050 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6ea80019f0 con 0x7f6eb015e050 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 >> v1:172.21.15.46:6789/0 conn(0x7f6eb015e050 legacy=0x7f6eb006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e900039b0 0x7f6e90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90004800 0x7f6e90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 0x7f6e900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00b4310 con 0x7f6e900039b0 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00b45f0 con 0x7f6e90004800 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9000b9c0 con 0x7f6eb015d730 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.319+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e900039b0 0x7f6e90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6ea7e00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90004800 0x7f6e90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6eaea00640 1 -- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 msgr2=0x7f6e900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb015d730 0x7f6e900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6eaea00640 1 -- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90004800 msgr2=0x7f6e90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90004800 0x7f6e90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.320+0000 7f6eaea00640 1 -- 172.21.15.46:0/3295231059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e90003520 con 0x7f6e900039b0 2024-04-26T23:41:53.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.321+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e900039b0 0x7f6e90003dd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009ac0 tx=0x7f6e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38edcced249b82d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.321+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0036070 con 0x7f6e900039b0 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.321+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0031400 con 0x7f6e900039b0 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.321+0000 7f6eae000640 1 -- 172.21.15.46:0/3295231059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0009690 con 0x7f6e900039b0 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3295231059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e900039b0 msgr2=0x7f6e90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3295231059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e900039b0 0x7f6e90003dd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009ac0 tx=0x7f6e90005830 comp rx=0 tx=0).stop 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eaf400640 1 -- 172.21.15.46:0/3295231059 reap_dead start 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3295231059 shutdown_connections 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3295231059 >> 172.21.15.46:0/3295231059 conn(0x7f6eb006c980 msgr2=0x7f6eb00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3295231059 shutdown_connections 2024-04-26T23:41:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.322+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3295231059 wait complete. 2024-04-26T23:41:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.324+0000 7f6eb5a00640 1 Processor -- start 2024-04-26T23:41:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.324+0000 7f6eb5a00640 1 -- start start 2024-04-26T23:41:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 0x7f6eb0143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 0x7f6eb01f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb015e4c0 con 0x7f6eb00cbf40 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00b4e90 con 0x7f6eb0141790 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eb5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b3ec0 con 0x7f6e900039b0 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36256/0 (socket says 172.21.15.46:36256) 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eaf400640 1 -- 172.21.15.46:0/3144562542 learned_addr learned my addr 172.21.15.46:0/3144562542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6ea7e00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 0x7f6eb01f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.325+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 0x7f6eb0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 -- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 msgr2=0x7f6eb0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 0x7f6eb0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 -- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 msgr2=0x7f6eb01f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 0x7f6eb01f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 -- 172.21.15.46:0/3144562542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb0145750 con 0x7f6eb00cbf40 2024-04-26T23:41:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 0x7f6eb0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6ea7e00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 0x7f6eb01f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eaf400640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80012e0 tx=0x7f6ea800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800a5b0 con 0x7f6eb00cbf40 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea800a750 con 0x7f6eb00cbf40 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800ab40 con 0x7f6eb00cbf40 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb0143670 con 0x7f6eb00cbf40 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.326+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb0143c30 con 0x7f6eb00cbf40 2024-04-26T23:41:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.328+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb0069530 con 0x7f6eb00cbf40 2024-04-26T23:41:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.329+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea8011070 con 0x7f6eb00cbf40 2024-04-26T23:41:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.330+0000 7f6ea6a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 0x7f6e8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.330+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f6ea808b350 con 0x7f6eb00cbf40 2024-04-26T23:41:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.330+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 0x7f6e8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.332+0000 7f6eaea00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 0x7f6e8c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0033000 tx=0x7f6ea0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.334+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ea8059940 con 0x7f6eb00cbf40 2024-04-26T23:41:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.543+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6eb00cf530 con 0x7f6eb00cbf40 2024-04-26T23:41:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.544+0000 7f6ea6a00640 1 -- 172.21.15.46:0/3144562542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f6ea8059240 con 0x7f6eb00cbf40 2024-04-26T23:41:53.548 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 msgr2=0x7f6e8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 0x7f6e8c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0033000 tx=0x7f6ea0008030 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 msgr2=0x7f6eb00cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80012e0 tx=0x7f6ea800cf00 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 shutdown_connections 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c8f0 0x7f6e8c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00cbf40 0x7f6eb00cc3c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0141790 0x7f6eb01f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 --2- 172.21.15.46:0/3144562542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e900039b0 0x7f6eb0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 >> 172.21.15.46:0/3144562542 conn(0x7f6eb006c980 msgr2=0x7f6eb00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 shutdown_connections 2024-04-26T23:41:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.547+0000 7f6eb5a00640 1 -- 172.21.15.46:0/3144562542 wait complete. 2024-04-26T23:41:53.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:41:53.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.778+0000 7f39f5a00640 1 Processor -- start 2024-04-26T23:41:53.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.778+0000 7f39f5a00640 1 -- start start 2024-04-26T23:41:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39f00d1520 0x7f39f00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f015d510 0x7f39f00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d3af0 0x7f39f00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f39f00ce560 con 0x7f39f015d510 2024-04-26T23:41:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39f0070230 con 0x7f39f00d1520 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39f5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00d1e40 con 0x7f39f00d3af0 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f015d510 0x7f39f00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f015d510 0x7f39f00d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36264/0 (socket says 172.21.15.46:36264) 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.780+0000 7f39ee000640 1 -- 172.21.15.46:0/1892318406 learned_addr learned my addr 172.21.15.46:0/1892318406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39eea00640 1 --2- 172.21.15.46:0/1892318406 >> v2:172.21.15.142:3300/0 conn(0x7f39f00d1520 0x7f39f00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ef400640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d3af0 0x7f39f00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ee000640 1 -- 172.21.15.46:0/1892318406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d3af0 msgr2=0x7f39f00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ee000640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d3af0 0x7f39f00cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ee000640 1 -- 172.21.15.46:0/1892318406 >> v2:172.21.15.142:3300/0 conn(0x7f39f00d1520 msgr2=0x7f39f00d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ee000640 1 --2- 172.21.15.46:0/1892318406 >> v2:172.21.15.142:3300/0 conn(0x7f39f00d1520 0x7f39f00d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.781+0000 7f39ee000640 1 -- 172.21.15.46:0/1892318406 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f00ce7b0 con 0x7f39f015d510 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.782+0000 7f39ee000640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00d0b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f39f00b5af0 tx=0x7f39d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff3a9e4ec5fad9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.782+0000 7f39ed600640 1 -- 172.21.15.46:0/1892318406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8037070 con 0x7f39f015d510 2024-04-26T23:41:53.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.782+0000 7f39ed600640 1 -- 172.21.15.46:0/1892318406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d8032440 con 0x7f39f015d510 2024-04-26T23:41:53.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.782+0000 7f39ed600640 1 -- 172.21.15.46:0/1892318406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8032810 con 0x7f39f015d510 2024-04-26T23:41:53.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 -- 172.21.15.46:0/1892318406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 msgr2=0x7f39f00d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00d0b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f39f00b5af0 tx=0x7f39d802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 -- 172.21.15.46:0/1892318406 shutdown_connections 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d3af0 0x7f39f00cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 --2- 172.21.15.46:0/1892318406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00d0b10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 --2- 172.21.15.46:0/1892318406 >> v2:172.21.15.142:3300/0 conn(0x7f39f00d1520 0x7f39f00d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 -- 172.21.15.46:0/1892318406 >> 172.21.15.46:0/1892318406 conn(0x7f39f00d2f40 msgr2=0x7f39f00d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 -- 172.21.15.46:0/1892318406 shutdown_connections 2024-04-26T23:41:53.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.783+0000 7f39f5a00640 1 -- 172.21.15.46:0/1892318406 wait complete. 2024-04-26T23:41:53.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.785+0000 7f39f5a00640 1 Processor -- start 2024-04-26T23:41:53.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.785+0000 7f39f5a00640 1 -- start start 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d1520 0x7f39f00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 0x7f39f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00d1050 con 0x7f39f015d510 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00ced70 con 0x7f39f00d3af0 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39f5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00d11d0 con 0x7f39f00d1520 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36266/0 (socket says 172.21.15.46:36266) 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39ef400640 1 -- 172.21.15.46:0/607194970 learned_addr learned my addr 172.21.15.46:0/607194970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.786+0000 7f39eea00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d1520 0x7f39f00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ee000640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 0x7f39f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 -- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d1520 msgr2=0x7f39f00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d1520 0x7f39f00cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 -- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 msgr2=0x7f39f006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 0x7f39f006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 -- 172.21.15.46:0/607194970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f0152a60 con 0x7f39f015d510 2024-04-26T23:41:53.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ee000640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 0x7f39f006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39ef400640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c540 tx=0x7f39e800ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8015070 con 0x7f39f015d510 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e8004220 con 0x7f39f015d510 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.787+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8003610 con 0x7f39f015d510 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.788+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f0152d40 con 0x7f39f015d510 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.788+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f00c8b00 con 0x7f39f015d510 2024-04-26T23:41:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.789+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0069530 con 0x7f39f015d510 2024-04-26T23:41:53.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.790+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39e80037b0 con 0x7f39f015d510 2024-04-26T23:41:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.791+0000 7f39e6a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.791+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f39e808bae0 con 0x7f39f015d510 2024-04-26T23:41:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.791+0000 7f39eea00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.792+0000 7f39eea00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f39f0071420 tx=0x7f39e00077f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:53.796+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39e805a0c0 con 0x7f39f015d510 2024-04-26T23:41:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.000+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f39f00cc200 con 0x7f39f015d510 2024-04-26T23:41:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.001+0000 7f39e6a00640 1 -- 172.21.15.46:0/607194970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7f39e80599c0 con 0x7f39f015d510 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:41:54.005 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:41:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 msgr2=0x7f39cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f39f0071420 tx=0x7f39e00077f0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 msgr2=0x7f39f00caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c540 tx=0x7f39e800ca00 comp rx=0 tx=0).stop 2024-04-26T23:41:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 shutdown_connections 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f015d510 0x7f39f00caa60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f00d3af0 0x7f39f006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 --2- 172.21.15.46:0/607194970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f00d1520 0x7f39f00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 >> 172.21.15.46:0/607194970 conn(0x7f39f00d2f40 msgr2=0x7f39f0071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 shutdown_connections 2024-04-26T23:41:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.004+0000 7f39f5a00640 1 -- 172.21.15.46:0/607194970 wait complete. 2024-04-26T23:41:54.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:41:54.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:41:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.225+0000 7efc2a800640 1 Processor -- start 2024-04-26T23:41:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.225+0000 7efc2a800640 1 -- start start 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc2a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc24146b20 0x7efc240d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc2a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efc240d1620 0x7efc240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc2a800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240b3ce0 con 0x7efc24146740 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc2a800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7efc240d1a70 con 0x7efc24146b20 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc2a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efc240b48f0 con 0x7efc240d1620 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc28a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc24146b20 0x7efc240d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc28a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc24146b20 0x7efc240d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36278/0 (socket says 172.21.15.46:36278) 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.227+0000 7efc28a00640 1 -- 172.21.15.46:0/674670554 learned_addr learned my addr 172.21.15.46:0/674670554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.228+0000 7efc29e00640 1 --2- 172.21.15.46:0/674670554 >> v2:172.21.15.142:3300/0 conn(0x7efc240d1620 0x7efc240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.228+0000 7efc28a00640 1 -- 172.21.15.46:0/674670554 >> v2:172.21.15.142:3300/0 conn(0x7efc240d1620 msgr2=0x7efc240cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.228+0000 7efc28a00640 1 --2- 172.21.15.46:0/674670554 >> v2:172.21.15.142:3300/0 conn(0x7efc240d1620 0x7efc240cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.228+0000 7efc28a00640 1 -- 172.21.15.46:0/674670554 >> v1:172.21.15.142:6789/0 conn(0x7efc24146740 legacy=0x7efc240cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.228+0000 7efc28a00640 1 -- 172.21.15.46:0/674670554 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240b3fc0 con 0x7efc24146b20 2024-04-26T23:41:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.229+0000 7efc28a00640 1 --2- 172.21.15.46:0/674670554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc240d10e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efc2406fe90 tx=0x7efc1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ca4b20e6eeb33d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.230+0000 7efc23e00640 1 -- 172.21.15.46:0/674670554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10037070 con 0x7efc24146b20 2024-04-26T23:41:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.230+0000 7efc23e00640 1 -- 172.21.15.46:0/674670554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc10032440 con 0x7efc24146b20 2024-04-26T23:41:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.230+0000 7efc23e00640 1 -- 172.21.15.46:0/674670554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10032810 con 0x7efc24146b20 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.230+0000 7efc2a800640 1 -- 172.21.15.46:0/674670554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 msgr2=0x7efc240d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.230+0000 7efc2a800640 1 --2- 172.21.15.46:0/674670554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc240d10e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efc2406fe90 tx=0x7efc1002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 -- 172.21.15.46:0/674670554 shutdown_connections 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 --2- 172.21.15.46:0/674670554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc240d10e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 --2- 172.21.15.46:0/674670554 >> v2:172.21.15.142:3300/0 conn(0x7efc240d1620 0x7efc240cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 -- 172.21.15.46:0/674670554 >> 172.21.15.46:0/674670554 conn(0x7efc2406c770 msgr2=0x7efc2406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 -- 172.21.15.46:0/674670554 shutdown_connections 2024-04-26T23:41:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.231+0000 7efc2a800640 1 -- 172.21.15.46:0/674670554 wait complete. 2024-04-26T23:41:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.233+0000 7efc2a800640 1 Processor -- start 2024-04-26T23:41:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.233+0000 7efc2a800640 1 -- start start 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 0x7efc2413dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 0x7efc241510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc2415e560 con 0x7efc24146b20 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc240b4580 con 0x7efc24146740 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc2a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc240cdde0 con 0x7efc240d1620 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36288/0 (socket says 172.21.15.46:36288) 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc29e00640 1 -- 172.21.15.46:0/3859113234 learned_addr learned my addr 172.21.15.46:0/3859113234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc28a00640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 0x7efc241510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc29e00640 1 -- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 msgr2=0x7efc2413dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.234+0000 7efc29400640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 0x7efc2413dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29e00640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 0x7efc2413dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29e00640 1 -- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 msgr2=0x7efc241510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29e00640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 0x7efc241510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29e00640 1 -- 172.21.15.46:0/3859113234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2413e2d0 con 0x7efc24146b20 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc28a00640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 0x7efc241510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29400640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 0x7efc2413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc29e00640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00d4c0 tx=0x7efc1c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0125f0 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c019070 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241431d0 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0129f0 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.235+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc24143a30 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.236+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc240693b0 con 0x7efc24146b20 2024-04-26T23:41:54.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.237+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc1c012b90 con 0x7efc24146b20 2024-04-26T23:41:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.238+0000 7efc21600640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.238+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7efc1c08bca0 con 0x7efc24146b20 2024-04-26T23:41:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.239+0000 7efc29400640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.240+0000 7efc29400640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc18007e90 tx=0x7efc180073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.243+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc2413e2d0 con 0x7efc24146b20 2024-04-26T23:41:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:54.451+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efc24143520 con 0x7efc24146b20 2024-04-26T23:41:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.103+0000 7efc21600640 1 -- 172.21.15.46:0/3859113234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7efc1c05a280 con 0x7efc24146b20 2024-04-26T23:41:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 msgr2=0x7efc0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc18007e90 tx=0x7efc180073a0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 msgr2=0x7efc24140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00d4c0 tx=0x7efc1c00d980 comp rx=0 tx=0).stop 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 shutdown_connections 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc24146b20 0x7efc24140ff0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc24146740 0x7efc241510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 --2- 172.21.15.46:0/3859113234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc240d1620 0x7efc2413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.109+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 >> 172.21.15.46:0/3859113234 conn(0x7efc2406c770 msgr2=0x7efc240cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.111+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 shutdown_connections 2024-04-26T23:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.112+0000 7efc2a800640 1 -- 172.21.15.46:0/3859113234 wait complete. 2024-04-26T23:41:55.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:41:55.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.362+0000 7fab7e800640 1 Processor -- start 2024-04-26T23:41:55.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.363+0000 7fab7e800640 1 -- start start 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab781626c0 0x7fab780b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab780b5fb0 0x7fab780d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fab780d07d0 con 0x7fab780b5fb0 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fab780d0ab0 con 0x7fab781626c0 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.364+0000 7fab7e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab780d0d90 con 0x7fab78162aa0 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab780b5fb0 0x7fab780d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab780b5fb0 0x7fab780d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36304/0 (socket says 172.21.15.46:36304) 2024-04-26T23:41:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 -- 172.21.15.46:0/2214653276 learned_addr learned my addr 172.21.15.46:0/2214653276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7d400640 1 --2- 172.21.15.46:0/2214653276 >> v2:172.21.15.142:3300/0 conn(0x7fab781626c0 0x7fab780b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7de00640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 -- 172.21.15.46:0/2214653276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 msgr2=0x7fab780d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 -- 172.21.15.46:0/2214653276 >> v2:172.21.15.142:3300/0 conn(0x7fab781626c0 msgr2=0x7fab780b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 --2- 172.21.15.46:0/2214653276 >> v2:172.21.15.142:3300/0 conn(0x7fab781626c0 0x7fab780b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.365+0000 7fab7ca00640 1 -- 172.21.15.46:0/2214653276 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab780d2c80 con 0x7fab780b5fb0 2024-04-26T23:41:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.367+0000 7fab7ca00640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab780b5fb0 0x7fab780d5850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fab7808f2e0 tx=0x7fab6402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906b36629bd47fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.367+0000 7fab77e00640 1 -- 172.21.15.46:0/2214653276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64037070 con 0x7fab780b5fb0 2024-04-26T23:41:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.367+0000 7fab77e00640 1 -- 172.21.15.46:0/2214653276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab64032440 con 0x7fab780b5fb0 2024-04-26T23:41:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.367+0000 7fab77e00640 1 -- 172.21.15.46:0/2214653276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64032810 con 0x7fab780b5fb0 2024-04-26T23:41:55.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 -- 172.21.15.46:0/2214653276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab780b5fb0 msgr2=0x7fab780d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:55.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab780b5fb0 0x7fab780d5850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fab7808f2e0 tx=0x7fab6402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 -- 172.21.15.46:0/2214653276 shutdown_connections 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 --2- 172.21.15.46:0/2214653276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab780b5fb0 0x7fab780d5850 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 --2- 172.21.15.46:0/2214653276 >> v2:172.21.15.142:3300/0 conn(0x7fab781626c0 0x7fab780b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 -- 172.21.15.46:0/2214653276 >> 172.21.15.46:0/2214653276 conn(0x7fab7809e2a0 msgr2=0x7fab780d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 -- 172.21.15.46:0/2214653276 shutdown_connections 2024-04-26T23:41:55.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.368+0000 7fab7e800640 1 -- 172.21.15.46:0/2214653276 wait complete. 2024-04-26T23:41:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.370+0000 7fab7e800640 1 Processor -- start 2024-04-26T23:41:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.370+0000 7fab7e800640 1 -- start start 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 0x7fab780cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab780d1fc0 con 0x7fab781626c0 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fab780d1e40 con 0x7fab780b5fb0 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab780d2140 con 0x7fab78162aa0 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36308/0 (socket says 172.21.15.46:36308) 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.371+0000 7fab7ca00640 1 -- 172.21.15.46:0/1301831702 learned_addr learned my addr 172.21.15.46:0/1301831702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7d400640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 0x7fab780cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 -- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 msgr2=0x7fab780ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7de00640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 -- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 msgr2=0x7fab780cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 0x7fab780cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 -- 172.21.15.46:0/1301831702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab780cd220 con 0x7fab781626c0 2024-04-26T23:41:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7d400640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 0x7fab780cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.372+0000 7fab7ca00640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab6402a200 tx=0x7fab640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.373+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64041020 con 0x7fab781626c0 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.373+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab640306c0 con 0x7fab781626c0 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.373+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64037040 con 0x7fab781626c0 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.373+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab780ccf40 con 0x7fab781626c0 2024-04-26T23:41:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.373+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab781f51b0 con 0x7fab781626c0 2024-04-26T23:41:55.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.374+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7806d920 con 0x7fab781626c0 2024-04-26T23:41:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.376+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab64044020 con 0x7fab781626c0 2024-04-26T23:41:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.377+0000 7fab75600640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.377+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7fab640ac8c0 con 0x7fab781626c0 2024-04-26T23:41:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.377+0000 7fab7d400640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.379+0000 7fab7d400640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 0x7fab5806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab68006fd0 tx=0x7fab68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.381+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab7806d920 con 0x7fab781626c0 2024-04-26T23:41:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:55.591+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fab780693b0 con 0x7fab781626c0 2024-04-26T23:41:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.118+0000 7fab75600640 1 -- 172.21.15.46:0/1301831702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7fab6404b030 con 0x7fab781626c0 2024-04-26T23:41:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:41:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 msgr2=0x7fab5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 0x7fab5806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab68006fd0 tx=0x7fab68008040 comp rx=0 tx=0).stop 2024-04-26T23:41:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 msgr2=0x7fab7815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab6402a200 tx=0x7fab640046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 shutdown_connections 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab78162aa0 0x7fab780ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab781626c0 0x7fab7815d3e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 --2- 172.21.15.46:0/1301831702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab780b5fb0 0x7fab780cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.123+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 >> 172.21.15.46:0/1301831702 conn(0x7fab7809e2a0 msgr2=0x7fab781618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.127+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 shutdown_connections 2024-04-26T23:41:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.127+0000 7fab7e800640 1 -- 172.21.15.46:0/1301831702 wait complete. 2024-04-26T23:41:56.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:41:56.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.346+0000 7f094ca00640 1 Processor -- start 2024-04-26T23:41:56.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.346+0000 7f094ca00640 1 -- start start 2024-04-26T23:41:56.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f094815d730 0x7f09480d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09480b53b0 0x7f094815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 0x7f094806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f094806eb90 con 0x7f094815d730 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f09480b4310 con 0x7f094815e050 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f094ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09480b45f0 con 0x7f09480b53b0 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f0945e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f094815d730 0x7f09480d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f0945e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f094815d730 0x7f09480d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36312/0 (socket says 172.21.15.46:36312) 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.349+0000 7f0945e00640 1 -- 172.21.15.46:0/2267314884 learned_addr learned my addr 172.21.15.46:0/2267314884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0946800640 1 --2- 172.21.15.46:0/2267314884 >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 0x7f094806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f093d400640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09480b53b0 0x7f094815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0945e00640 1 -- 172.21.15.46:0/2267314884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09480b53b0 msgr2=0x7f094815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0945e00640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09480b53b0 0x7f094815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0945e00640 1 -- 172.21.15.46:0/2267314884 >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 msgr2=0x7f094806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0945e00640 1 --2- 172.21.15.46:0/2267314884 >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 0x7f094806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0945e00640 1 -- 172.21.15.46:0/2267314884 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09480b48d0 con 0x7f094815d730 2024-04-26T23:41:56.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.350+0000 7f0946800640 1 --2- 172.21.15.46:0/2267314884 >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 0x7f094806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:56.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.351+0000 7f0945e00640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f094815d730 0x7f09480d00d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0948074200 tx=0x7f093802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbecb416f297a832 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:56.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.351+0000 7f0945400640 1 -- 172.21.15.46:0/2267314884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938037070 con 0x7f094815d730 2024-04-26T23:41:56.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.351+0000 7f0945400640 1 -- 172.21.15.46:0/2267314884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938032440 con 0x7f094815d730 2024-04-26T23:41:56.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.351+0000 7f0945400640 1 -- 172.21.15.46:0/2267314884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938032810 con 0x7f094815d730 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 -- 172.21.15.46:0/2267314884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f094815d730 msgr2=0x7f09480d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f094815d730 0x7f09480d00d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0948074200 tx=0x7f093802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 -- 172.21.15.46:0/2267314884 shutdown_connections 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 --2- 172.21.15.46:0/2267314884 >> v2:172.21.15.142:3300/0 conn(0x7f094815e050 0x7f094806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09480b53b0 0x7f094815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 --2- 172.21.15.46:0/2267314884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f094815d730 0x7f09480d00d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 -- 172.21.15.46:0/2267314884 >> 172.21.15.46:0/2267314884 conn(0x7f094806c980 msgr2=0x7f09480b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 -- 172.21.15.46:0/2267314884 shutdown_connections 2024-04-26T23:41:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.352+0000 7f094ca00640 1 -- 172.21.15.46:0/2267314884 wait complete. 2024-04-26T23:41:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.354+0000 7f094ca00640 1 Processor -- start 2024-04-26T23:41:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.354+0000 7f094ca00640 1 -- start start 2024-04-26T23:41:56.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f094815d730 0x7f094813b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094815e050 0x7f09481e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f094815e4c0 con 0x7f09480b53b0 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09480b4e90 con 0x7f094815e050 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f094ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09480b3ec0 con 0x7f094815d730 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f0945e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f0945e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36316/0 (socket says 172.21.15.46:36316) 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.355+0000 7f0945e00640 1 -- 172.21.15.46:0/1903446110 learned_addr learned my addr 172.21.15.46:0/1903446110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 -- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f094815d730 msgr2=0x7f094813b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f094815d730 0x7f094813b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 -- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094815e050 msgr2=0x7f09481e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094815e050 0x7f09481e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 -- 172.21.15.46:0/1903446110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0948141d60 con 0x7f09480b53b0 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f0945e00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f093802a220 tx=0x7f09380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938037040 con 0x7f09480b53b0 2024-04-26T23:41:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938042070 con 0x7f09480b53b0 2024-04-26T23:41:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938030bb0 con 0x7f09480b53b0 2024-04-26T23:41:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.356+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0948142040 con 0x7f09480b53b0 2024-04-26T23:41:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.357+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094813b790 con 0x7f09480b53b0 2024-04-26T23:41:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.358+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0948069530 con 0x7f09480b53b0 2024-04-26T23:41:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.359+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0938047460 con 0x7f09480b53b0 2024-04-26T23:41:56.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.360+0000 7f093ea00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 0x7f092406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:56.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.360+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7f09380ac950 con 0x7f09480b53b0 2024-04-26T23:41:56.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.360+0000 7f093d400640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 0x7f092406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.362+0000 7f093d400640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 0x7f092406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f09481424f0 tx=0x7f0930008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.365+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0938045360 con 0x7f09480b53b0 2024-04-26T23:41:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:56.610+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f09480d0300 con 0x7f09480b53b0 2024-04-26T23:41:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.122+0000 7f093ea00640 1 -- 172.21.15.46:0/1903446110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v24) ==== 172+0+0 (secure 0 0 0) 0x7f093807ac40 con 0x7f09480b53b0 2024-04-26T23:41:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 7 and data pool 8 2024-04-26T23:41:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.126+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 msgr2=0x7f092406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.126+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 0x7f092406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f09481424f0 tx=0x7f0930008040 comp rx=0 tx=0).stop 2024-04-26T23:41:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.126+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 msgr2=0x7f09481416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.126+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f093802a220 tx=0x7f09380046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 shutdown_connections 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f092406c8f0 0x7f092406edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094815e050 0x7f09481e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f094815d730 0x7f094813b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 --2- 172.21.15.46:0/1903446110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09480b53b0 0x7f09481416f0 secure :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f093802a220 tx=0x7f09380046c0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.127+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 >> 172.21.15.46:0/1903446110 conn(0x7f094806c980 msgr2=0x7f09480b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.130+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 shutdown_connections 2024-04-26T23:41:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.130+0000 7f094ca00640 1 -- 172.21.15.46:0/1903446110 wait complete. 2024-04-26T23:41:57.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.347+0000 7f3739200640 1 Processor -- start 2024-04-26T23:41:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.347+0000 7f3739200640 1 -- start start 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3739200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373406e780 0x7f37340b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3739200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37340b43c0 0x7f373414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3739200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37340b4940 con 0x7f373406eb60 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3739200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37340b4c20 con 0x7f37340b43c0 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3739200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37340b4f00 con 0x7f373406e780 2024-04-26T23:41:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3732800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f373406eb60 0x7f3734167130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48154/0 (socket says 172.21.15.46:48154) 2024-04-26T23:41:57.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.349+0000 7f3732800640 1 -- 172.21.15.46:0/1412207265 learned_addr learned my addr 172.21.15.46:0/1412207265 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3729400640 1 --2- 172.21.15.46:0/1412207265 >> v2:172.21.15.142:3300/0 conn(0x7f37340b43c0 0x7f373414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731e00640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373406e780 0x7f37340b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607052700 0 0) 0x7f37340b4940 con 0x7f373406eb60 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3718003520 con 0x7f373406eb60 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1663587124 0 0) 0x7f3718003520 con 0x7f373406eb60 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373406e780 msgr2=0x7f37340b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373406e780 0x7f37340b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 >> v2:172.21.15.142:3300/0 conn(0x7f37340b43c0 msgr2=0x7f373414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 --2- 172.21.15.46:0/1412207265 >> v2:172.21.15.142:3300/0 conn(0x7f37340b43c0 0x7f373414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37340b51e0 con 0x7f373406eb60 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f372c0019f0 con 0x7f373406eb60 2024-04-26T23:41:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 >> v1:172.21.15.46:6789/0 conn(0x7f373406eb60 legacy=0x7f3734167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37180039b0 0x7f3718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3718004800 0x7f3718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37180051c0 0x7f371800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37340b4c20 con 0x7f37180051c0 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37340b4f00 con 0x7f3718004800 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f371800c000 con 0x7f37180039b0 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.350+0000 7f3731e00640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373406e780 0x7f37340b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37180051c0 0x7f371800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3731e00640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37180039b0 0x7f3718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3729400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3718004800 0x7f3718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 -- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37180039b0 msgr2=0x7f3718003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37180039b0 0x7f3718003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 -- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3718004800 msgr2=0x7f3718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3718004800 0x7f3718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3732800640 1 -- 172.21.15.46:0/1412207265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718003520 con 0x7f37180051c0 2024-04-26T23:41:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.351+0000 7f3729400640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3718004800 0x7f3718004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:57.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.352+0000 7f3732800640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37180051c0 0x7f371800b9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f372c00a950 tx=0x7f3718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b16e560ec2a116 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.352+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00a4c0 con 0x7f37180051c0 2024-04-26T23:41:57.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.352+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372c02f070 con 0x7f37180051c0 2024-04-26T23:41:57.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.352+0000 7f3731400640 1 -- 172.21.15.46:0/1412207265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c02bd40 con 0x7f37180051c0 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 -- 172.21.15.46:0/1412207265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37180051c0 msgr2=0x7f371800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 --2- 172.21.15.46:0/1412207265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37180051c0 0x7f371800b9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f372c00a950 tx=0x7f3718005830 comp rx=0 tx=0).stop 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3732800640 1 -- 172.21.15.46:0/1412207265 reap_dead start 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 -- 172.21.15.46:0/1412207265 shutdown_connections 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 -- 172.21.15.46:0/1412207265 >> 172.21.15.46:0/1412207265 conn(0x7f373406c530 msgr2=0x7f373406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 -- 172.21.15.46:0/1412207265 shutdown_connections 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.353+0000 7f3739200640 1 -- 172.21.15.46:0/1412207265 wait complete. 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.355+0000 7f3739200640 1 Processor -- start 2024-04-26T23:41:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.355+0000 7f3739200640 1 -- start start 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f373415d0c0 0x7f3734142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373415d4a0 0x7f37341475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37340734f0 con 0x7f3734143150 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3734073370 con 0x7f373415d0c0 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3739200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3734073670 con 0x7f373415d4a0 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36334/0 (socket says 172.21.15.46:36334) 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.356+0000 7f3732800640 1 -- 172.21.15.46:0/1928477175 learned_addr learned my addr 172.21.15.46:0/1928477175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3731e00640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f373415d0c0 0x7f3734142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 -- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373415d4a0 msgr2=0x7f37341475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373415d4a0 0x7f37341475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 -- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f373415d0c0 msgr2=0x7f3734142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f373415d0c0 0x7f3734142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 -- 172.21.15.46:0/1928477175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373415e310 con 0x7f3734143150 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3732800640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f372c00a950 tx=0x7f372c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00a4c0 con 0x7f3734143150 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372c034470 con 0x7f3734143150 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c002000 con 0x7f3734143150 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.357+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373415e5f0 con 0x7f3734143150 2024-04-26T23:41:57.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.358+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373415ebb0 con 0x7f3734143150 2024-04-26T23:41:57.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.359+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3734069530 con 0x7f3734143150 2024-04-26T23:41:57.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.360+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f372c034820 con 0x7f3734143150 2024-04-26T23:41:57.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.361+0000 7f372aa00640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.361+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f372c0ab5f0 con 0x7f3734143150 2024-04-26T23:41:57.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.362+0000 7f3731e00640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.363+0000 7f3731e00640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3724006fd0 tx=0x7f3724008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.366+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f372c079880 con 0x7f3734143150 2024-04-26T23:41:57.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.575+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37340b1410 con 0x7f3734143150 2024-04-26T23:41:57.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.578+0000 7f372aa00640 1 -- 172.21.15.46:0/1928477175 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16919 (secure 0 0 0) 0x7f372c079180 con 0x7f3734143150 2024-04-26T23:41:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:57.582 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:56.611232+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:57.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 msgr2=0x7f371006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3724006fd0 tx=0x7f3724008040 comp rx=0 tx=0).stop 2024-04-26T23:41:57.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 msgr2=0x7f3734147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f372c00a950 tx=0x7f372c001dc0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 shutdown_connections 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3734143150 0x7f3734147b10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f373415d4a0 0x7f37341475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 --2- 172.21.15.46:0/1928477175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f373415d0c0 0x7f3734142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 >> 172.21.15.46:0/1928477175 conn(0x7f373406c530 msgr2=0x7f373414a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 shutdown_connections 2024-04-26T23:41:57.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.581+0000 7f3739200640 1 -- 172.21.15.46:0/1928477175 wait complete. 2024-04-26T23:41:57.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:41:57.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.813+0000 7f5d91600640 1 Processor -- start 2024-04-26T23:41:57.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.814+0000 7f5d91600640 1 -- start start 2024-04-26T23:41:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d91600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d91600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c0d2130 0x7f5d8c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d91600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8c0d11f0 con 0x7f5d8c0b4000 2024-04-26T23:41:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d91600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5d8c0d0ce0 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d91600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c0d2990 con 0x7f5d8c0d2130 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d8be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d8be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36338/0 (socket says 172.21.15.46:36338) 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d8be00640 1 -- 172.21.15.46:0/16816687 learned_addr learned my addr 172.21.15.46:0/16816687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.815+0000 7f5d90c00640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c0d2130 0x7f5d8c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.816+0000 7f5d8be00640 1 -- 172.21.15.46:0/16816687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c0d2130 msgr2=0x7f5d8c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.816+0000 7f5d8be00640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c0d2130 0x7f5d8c0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.816+0000 7f5d8be00640 1 -- 172.21.15.46:0/16816687 >> v1:172.21.15.142:6789/0 conn(0x7f5d8c0b4000 legacy=0x7f5d8c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.816+0000 7f5d8be00640 1 -- 172.21.15.46:0/16816687 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c0d5f20 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.817+0000 7f5d8be00640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009e50 tx=0x7f5d7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935bb950885a2577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.818+0000 7f5d8aa00640 1 -- 172.21.15.46:0/16816687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c036070 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.818+0000 7f5d8aa00640 1 -- 172.21.15.46:0/16816687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c031600 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.818+0000 7f5d8aa00640 1 -- 172.21.15.46:0/16816687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0319d0 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 -- 172.21.15.46:0/16816687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c15f4e0 msgr2=0x7f5d8c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009e50 tx=0x7f5d7c02a710 comp rx=0 tx=0).stop 2024-04-26T23:41:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 -- 172.21.15.46:0/16816687 shutdown_connections 2024-04-26T23:41:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c0d2130 0x7f5d8c0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 --2- 172.21.15.46:0/16816687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c15f4e0 0x7f5d8c0d2df0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 -- 172.21.15.46:0/16816687 >> 172.21.15.46:0/16816687 conn(0x7f5d8c0d5010 msgr2=0x7f5d8c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:57.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 -- 172.21.15.46:0/16816687 shutdown_connections 2024-04-26T23:41:57.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.819+0000 7f5d91600640 1 -- 172.21.15.46:0/16816687 wait complete. 2024-04-26T23:41:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.821+0000 7f5d91600640 1 Processor -- start 2024-04-26T23:41:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.821+0000 7f5d91600640 1 -- start start 2024-04-26T23:41:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c0d2130 0x7f5d8c06cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c15f4e0 0x7f5d8c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c0b1d60 con 0x7f5d8c0b4000 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c0d5c40 con 0x7f5d8c0d2130 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d91600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c15f8c0 con 0x7f5d8c15f4e0 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36340/0 (socket says 172.21.15.46:36340) 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 -- 172.21.15.46:0/1428626584 learned_addr learned my addr 172.21.15.46:0/1428626584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 -- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c15f4e0 msgr2=0x7f5d8c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c15f4e0 0x7f5d8c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8b400640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c0d2130 0x7f5d8c06cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.822+0000 7f5d8be00640 1 -- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c0d2130 msgr2=0x7f5d8c06cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d8be00640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c0d2130 0x7f5d8c06cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d8be00640 1 -- 172.21.15.46:0/1428626584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c0cf290 con 0x7f5d8c0b4000 2024-04-26T23:41:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d8be00640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f80 tx=0x7f5d7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c036050 con 0x7f5d8c0b4000 2024-04-26T23:41:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c042070 con 0x7f5d8c0b4000 2024-04-26T23:41:57.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c009510 con 0x7f5d8c0b4000 2024-04-26T23:41:57.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.823+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c14ee00 con 0x7f5d8c0b4000 2024-04-26T23:41:57.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.824+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c14c1e0 con 0x7f5d8c0b4000 2024-04-26T23:41:57.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.825+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8c0d46d0 con 0x7f5d8c0b4000 2024-04-26T23:41:57.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.826+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d7c007c70 con 0x7f5d8c0b4000 2024-04-26T23:41:57.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.827+0000 7f5d83e00640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 0x7f5d6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:57.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.827+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5d7c0acbc0 con 0x7f5d8c0b4000 2024-04-26T23:41:57.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.827+0000 7f5d8b400640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 0x7f5d6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:57.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.828+0000 7f5d8b400640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 0x7f5d6806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c06dab0 tx=0x7f5d74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:57.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:57.831+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d7c045360 con 0x7f5d8c0b4000 2024-04-26T23:41:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.039+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d8c0d33d0 con 0x7f5d8c0b4000 2024-04-26T23:41:58.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.042+0000 7f5d83e00640 1 -- 172.21.15.46:0/1428626584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16919 (secure 0 0 0) 0x7f5d7c07ae50 con 0x7f5d8c0b4000 2024-04-26T23:41:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:41:58.046 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:56.611232+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:41:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 msgr2=0x7f5d6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 0x7f5d6806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c06dab0 tx=0x7f5d74009040 comp rx=0 tx=0).stop 2024-04-26T23:41:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 msgr2=0x7f5d8c14e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f80 tx=0x7f5d7c004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 shutdown_connections 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d6806c8f0 0x7f5d6806edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d8c15f4e0 0x7f5d8c06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c0d2130 0x7f5d8c06cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 --2- 172.21.15.46:0/1428626584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b4000 0x7f5d8c14e8c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 >> 172.21.15.46:0/1428626584 conn(0x7f5d8c0d5010 msgr2=0x7f5d8c160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 shutdown_connections 2024-04-26T23:41:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.045+0000 7f5d91600640 1 -- 172.21.15.46:0/1428626584 wait complete. 2024-04-26T23:41:58.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:41:58.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.274+0000 7f7916400640 1 Processor -- start 2024-04-26T23:41:58.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.274+0000 7f7916400640 1 -- start start 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.275+0000 7f7916400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7910160130 0x7f7910160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.275+0000 7f7916400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 0x7f79100cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.275+0000 7f7916400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f791015d0a0 0x7f79100cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.275+0000 7f7916400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f79100cca70 con 0x7f791015d0a0 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7916400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79100cbe60 con 0x7f791015cc40 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7916400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79100cc140 con 0x7f7910160130 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7915a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f791015d0a0 0x7f79100cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7915a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f791015d0a0 0x7f79100cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36352/0 (socket says 172.21.15.46:36352) 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7915a00640 1 -- 172.21.15.46:0/729972663 learned_addr learned my addr 172.21.15.46:0/729972663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f790fe00640 1 --2- 172.21.15.46:0/729972663 >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 0x7f79100cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.276+0000 7f7915000640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7910160130 0x7f7910160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f7915a00640 1 -- 172.21.15.46:0/729972663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7910160130 msgr2=0x7f7910160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f7915a00640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7910160130 0x7f7910160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f7915a00640 1 -- 172.21.15.46:0/729972663 >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 msgr2=0x7f79100cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f7915a00640 1 --2- 172.21.15.46:0/729972663 >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 0x7f79100cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f7915a00640 1 -- 172.21.15.46:0/729972663 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100cc420 con 0x7f791015d0a0 2024-04-26T23:41:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.277+0000 7f790fe00640 1 --2- 172.21.15.46:0/729972663 >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 0x7f79100cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.278+0000 7f7915a00640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f791015d0a0 0x7f79100cda90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f79100b3740 tx=0x7f790802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f5ad91a8d96ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.278+0000 7f790f400640 1 -- 172.21.15.46:0/729972663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908036070 con 0x7f791015d0a0 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.278+0000 7f790f400640 1 -- 172.21.15.46:0/729972663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908031600 con 0x7f791015d0a0 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.278+0000 7f790f400640 1 -- 172.21.15.46:0/729972663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080319d0 con 0x7f791015d0a0 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 -- 172.21.15.46:0/729972663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f791015d0a0 msgr2=0x7f79100cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f791015d0a0 0x7f79100cda90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f79100b3740 tx=0x7f790802a610 comp rx=0 tx=0).stop 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 -- 172.21.15.46:0/729972663 shutdown_connections 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f791015d0a0 0x7f79100cda90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 --2- 172.21.15.46:0/729972663 >> v2:172.21.15.142:3300/0 conn(0x7f791015cc40 0x7f79100cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 --2- 172.21.15.46:0/729972663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7910160130 0x7f7910160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 -- 172.21.15.46:0/729972663 >> 172.21.15.46:0/729972663 conn(0x7f79100b27b0 msgr2=0x7f79100b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 -- 172.21.15.46:0/729972663 shutdown_connections 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.279+0000 7f7916400640 1 -- 172.21.15.46:0/729972663 wait complete. 2024-04-26T23:41:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.281+0000 7f7916400640 1 Processor -- start 2024-04-26T23:41:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.281+0000 7f7916400640 1 -- start start 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f791015cc40 0x7f79100c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791015d0a0 0x7f7910155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79100723a0 con 0x7f7910160130 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7910072220 con 0x7f791015d0a0 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7916400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7910072520 con 0x7f791015cc40 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7915a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7915a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36354/0 (socket says 172.21.15.46:36354) 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7915a00640 1 -- 172.21.15.46:0/1425130692 learned_addr learned my addr 172.21.15.46:0/1425130692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f790fe00640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791015d0a0 0x7f7910155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.282+0000 7f7915000640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f791015cc40 0x7f79100c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 -- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f791015cc40 msgr2=0x7f79100c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f791015cc40 0x7f79100c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 -- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791015d0a0 msgr2=0x7f7910155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791015d0a0 0x7f7910155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 -- 172.21.15.46:0/1425130692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791014def0 con 0x7f7910160130 2024-04-26T23:41:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7915a00640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7908009f40 tx=0x7f7908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908036050 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908042070 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908009510 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791014e1d0 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.283+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791014e790 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.285+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79100693b0 con 0x7f7910160130 2024-04-26T23:41:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.286+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7908007bf0 con 0x7f7910160130 2024-04-26T23:41:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.287+0000 7f790cc00640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 0x7f78f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.287+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f79080acc00 con 0x7f7910160130 2024-04-26T23:41:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.287+0000 7f7915000640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 0x7f78f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:58.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.289+0000 7f7915000640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 0x7f78f006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7904006fd0 tx=0x7f7904008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:58.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.291+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7908045360 con 0x7f7910160130 2024-04-26T23:41:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:58.536+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f791006ff60 con 0x7f7910160130 2024-04-26T23:41:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.170+0000 7f790cc00640 1 -- 172.21.15.46:0/1425130692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f790807ae00 con 0x7f7910160130 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 msgr2=0x7f78f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 0x7f78f006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7904006fd0 tx=0x7f7904008040 comp rx=0 tx=0).stop 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 msgr2=0x7f791015b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7908009f40 tx=0x7f7908004a40 comp rx=0 tx=0).stop 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 shutdown_connections 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78f006c8f0 0x7f78f006edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7910160130 0x7f791015b870 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791015d0a0 0x7f7910155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 --2- 172.21.15.46:0/1425130692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f791015cc40 0x7f79100c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.174+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 >> 172.21.15.46:0/1425130692 conn(0x7f79100b27b0 msgr2=0x7f791014f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.175+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 shutdown_connections 2024-04-26T23:41:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.175+0000 7f7916400640 1 -- 172.21.15.46:0/1425130692 wait complete. 2024-04-26T23:41:59.201 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:41:59.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:41:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.410+0000 7fed91a00640 1 Processor -- start 2024-04-26T23:41:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.410+0000 7fed91a00640 1 -- start start 2024-04-26T23:41:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.411+0000 7fed91a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed8c15d170 0x7fed8c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.411+0000 7fed91a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 0x7fed8c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.411+0000 7fed91a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8c0ce500 con 0x7fed8c0cf880 2024-04-26T23:41:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.411+0000 7fed91a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fed8c0ce7e0 con 0x7fed8c15d170 2024-04-26T23:41:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.411+0000 7fed91a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fed8c0ceac0 con 0x7fed8c0b5180 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed8c15d170 0x7fed8c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed8c15d170 0x7fed8c0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36360/0 (socket says 172.21.15.46:36360) 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 -- 172.21.15.46:0/1511561604 learned_addr learned my addr 172.21.15.46:0/1511561604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8b400640 1 --2- 172.21.15.46:0/1511561604 >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 0x7fed8c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 -- 172.21.15.46:0/1511561604 >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 msgr2=0x7fed8c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 --2- 172.21.15.46:0/1511561604 >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 0x7fed8c0c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 -- 172.21.15.46:0/1511561604 >> v1:172.21.15.142:6789/0 conn(0x7fed8c0cf880 legacy=0x7fed8c0b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8aa00640 1 -- 172.21.15.46:0/1511561604 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c0ceda0 con 0x7fed8c15d170 2024-04-26T23:41:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.412+0000 7fed8b400640 1 --2- 172.21.15.46:0/1511561604 >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 0x7fed8c0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:41:59.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.413+0000 7fed8aa00640 1 --2- 172.21.15.46:0/1511561604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c15d170 0x7fed8c0cda90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed8c09cab0 tx=0x7fed7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486d16bd422c9a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:59.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed8a000640 1 -- 172.21.15.46:0/1511561604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037070 con 0x7fed8c15d170 2024-04-26T23:41:59.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed8a000640 1 -- 172.21.15.46:0/1511561604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed8c15d170 2024-04-26T23:41:59.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed8a000640 1 -- 172.21.15.46:0/1511561604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032810 con 0x7fed8c15d170 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 -- 172.21.15.46:0/1511561604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c15d170 msgr2=0x7fed8c0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 --2- 172.21.15.46:0/1511561604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c15d170 0x7fed8c0cda90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed8c09cab0 tx=0x7fed7c02a180 comp rx=0 tx=0).stop 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 -- 172.21.15.46:0/1511561604 shutdown_connections 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 --2- 172.21.15.46:0/1511561604 >> v2:172.21.15.142:3300/0 conn(0x7fed8c0b5180 0x7fed8c0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 --2- 172.21.15.46:0/1511561604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c15d170 0x7fed8c0cda90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 -- 172.21.15.46:0/1511561604 >> 172.21.15.46:0/1511561604 conn(0x7fed8c09ba70 msgr2=0x7fed8c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 -- 172.21.15.46:0/1511561604 shutdown_connections 2024-04-26T23:41:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.414+0000 7fed91a00640 1 -- 172.21.15.46:0/1511561604 wait complete. 2024-04-26T23:41:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.416+0000 7fed91a00640 1 Processor -- start 2024-04-26T23:41:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.416+0000 7fed91a00640 1 -- start start 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed8c0b5180 0x7fed8c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c15d170 0x7fed8c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c15d550 con 0x7fed8c0cf880 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c0cdfd0 con 0x7fed8c15d170 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.417+0000 7fed91a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c160810 con 0x7fed8c0b5180 2024-04-26T23:41:59.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36372/0 (socket says 172.21.15.46:36372) 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 -- 172.21.15.46:0/541183072 learned_addr learned my addr 172.21.15.46:0/541183072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed8b400640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c15d170 0x7fed8c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 -- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed8c0b5180 msgr2=0x7fed8c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed8c0b5180 0x7fed8c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 -- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c15d170 msgr2=0x7fed8c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c15d170 0x7fed8c0b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 -- 172.21.15.46:0/541183072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c1f4eb0 con 0x7fed8c0cf880 2024-04-26T23:41:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.418+0000 7fed83e00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c400 tx=0x7fed7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.419+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74011790 con 0x7fed8c0cf880 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.419+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed74017070 con 0x7fed8c0cf880 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.419+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74011bb0 con 0x7fed8c0cf880 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.419+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c0cad30 con 0x7fed8c0cf880 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.419+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c160530 con 0x7fed8c0cf880 2024-04-26T23:41:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.420+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c069530 con 0x7fed8c0cf880 2024-04-26T23:41:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.421+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed7401c460 con 0x7fed8c0cf880 2024-04-26T23:41:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.422+0000 7fed82a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:41:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.422+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fed7408bc20 con 0x7fed8c0cf880 2024-04-26T23:41:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.423+0000 7fed8aa00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:41:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.424+0000 7fed8aa00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 0x7fed6806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:41:59.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.427+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed74059e20 con 0x7fed8c0cf880 2024-04-26T23:41:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:41:59.680+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fed8c070720 con 0x7fed8c0cf880 2024-04-26T23:42:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.174+0000 7fed82a00640 1 -- 172.21.15.46:0/541183072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fed74059720 con 0x7fed8c0cf880 2024-04-26T23:42:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 msgr2=0x7fed6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 0x7fed6806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c02c000 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 msgr2=0x7fed8c14d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c400 tx=0x7fed7400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 shutdown_connections 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c15d170 0x7fed8c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed8c0cf880 0x7fed8c14d270 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 --2- 172.21.15.46:0/541183072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed8c0b5180 0x7fed8c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.178+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 >> 172.21.15.46:0/541183072 conn(0x7fed8c09ba70 msgr2=0x7fed8c06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.179+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 shutdown_connections 2024-04-26T23:42:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.179+0000 7fed91a00640 1 -- 172.21.15.46:0/541183072 wait complete. 2024-04-26T23:42:00.208 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:42:00.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.414+0000 7f64b3200640 1 Processor -- start 2024-04-26T23:42:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.414+0000 7f64b3200640 1 -- start start 2024-04-26T23:42:00.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 0x7f64b406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b40720e0 0x7f64b40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b4073ad0 con 0x7f64b4160910 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f64b4146a90 con 0x7f64b40720e0 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64b4073d70 con 0x7f64b41466b0 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b40720e0 0x7f64b40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b40720e0 0x7f64b40cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36374/0 (socket says 172.21.15.46:36374) 2024-04-26T23:42:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.416+0000 7f64b1400640 1 -- 172.21.15.46:0/4005048666 learned_addr learned my addr 172.21.15.46:0/4005048666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1e00640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 0x7f64b406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1400640 1 -- 172.21.15.46:0/4005048666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 msgr2=0x7f64b406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1400640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 0x7f64b406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1400640 1 -- 172.21.15.46:0/4005048666 >> v1:172.21.15.142:6789/0 conn(0x7f64b4160910 legacy=0x7f64b40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1400640 1 -- 172.21.15.46:0/4005048666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b4074050 con 0x7f64b40720e0 2024-04-26T23:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.417+0000 7f64b1e00640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 0x7f64b406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.418+0000 7f64b1400640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b40720e0 0x7f64b40cd860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009870 tx=0x7f64a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d258e486caa68c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b0a00640 1 -- 172.21.15.46:0/4005048666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0037070 con 0x7f64b40720e0 2024-04-26T23:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b0a00640 1 -- 172.21.15.46:0/4005048666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a0032440 con 0x7f64b40720e0 2024-04-26T23:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b0a00640 1 -- 172.21.15.46:0/4005048666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0032810 con 0x7f64b40720e0 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 -- 172.21.15.46:0/4005048666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b40720e0 msgr2=0x7f64b40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b40720e0 0x7f64b40cd860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009870 tx=0x7f64a002a210 comp rx=0 tx=0).stop 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 -- 172.21.15.46:0/4005048666 shutdown_connections 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b40720e0 0x7f64b40cd860 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 --2- 172.21.15.46:0/4005048666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b41466b0 0x7f64b406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.419+0000 7f64b3200640 1 -- 172.21.15.46:0/4005048666 >> 172.21.15.46:0/4005048666 conn(0x7f64b40b26d0 msgr2=0x7f64b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.420+0000 7f64b3200640 1 -- 172.21.15.46:0/4005048666 shutdown_connections 2024-04-26T23:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.420+0000 7f64b3200640 1 -- 172.21.15.46:0/4005048666 wait complete. 2024-04-26T23:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.421+0000 7f64b3200640 1 Processor -- start 2024-04-26T23:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.422+0000 7f64b3200640 1 -- start start 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 0x7f64b40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 0x7f64b40ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64b406e3c0 con 0x7f64b4160910 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40cb470 con 0x7f64b41466b0 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64b40cdda0 con 0x7f64b40720e0 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36380/0 (socket says 172.21.15.46:36380) 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 -- 172.21.15.46:0/838551064 learned_addr learned my addr 172.21.15.46:0/838551064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b1400640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 0x7f64b40ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b1e00640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 0x7f64b40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 -- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 msgr2=0x7f64b40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 0x7f64b40b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 -- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 msgr2=0x7f64b40ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 0x7f64b40ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.423+0000 7f64b2800640 1 -- 172.21.15.46:0/838551064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b40b4a20 con 0x7f64b4160910 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64b1400640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 0x7f64b40ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64b1e00640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 0x7f64b40b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64b2800640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00b440 tx=0x7f64ac00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac015020 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac011830 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac011c20 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b40b4cd0 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.424+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b40b5290 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.426+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b40693b0 con 0x7f64b4160910 2024-04-26T23:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.427+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64ac018020 con 0x7f64b4160910 2024-04-26T23:42:00.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.427+0000 7f64aa000640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 0x7f649406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.428+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f64ac08b830 con 0x7f64b4160910 2024-04-26T23:42:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.428+0000 7f64b1e00640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 0x7f649406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.429+0000 7f64b1e00640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 0x7f649406edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f64a4007f40 tx=0x7f64a4007490 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.432+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64ac01f030 con 0x7f64b4160910 2024-04-26T23:42:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.675+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64b40b2440 con 0x7f64b4160910 2024-04-26T23:42:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.677+0000 7f64aa000640 1 -- 172.21.15.46:0/838551064 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f64ac0172c0 con 0x7f64b4160910 2024-04-26T23:42:00.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:00.680 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:00.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 msgr2=0x7f649406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 0x7f649406edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f64a4007f40 tx=0x7f64a4007490 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 msgr2=0x7f64b40b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00b440 tx=0x7f64ac00b900 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 shutdown_connections 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f649406c8f0 0x7f649406edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b4160910 0x7f64b40b44e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b41466b0 0x7f64b40ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 --2- 172.21.15.46:0/838551064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64b40720e0 0x7f64b40b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 >> 172.21.15.46:0/838551064 conn(0x7f64b40b26d0 msgr2=0x7f64b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 shutdown_connections 2024-04-26T23:42:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.679+0000 7f64b3200640 1 -- 172.21.15.46:0/838551064 wait complete. 2024-04-26T23:42:00.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.913+0000 7f2f92800640 1 Processor -- start 2024-04-26T23:42:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.914+0000 7f2f92800640 1 -- start start 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.915+0000 7f2f92800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.915+0000 7f2f92800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2f8c0b4000 0x7f2f8c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.915+0000 7f2f92800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c0d11f0 con 0x7f2f8c0d1f60 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.915+0000 7f2f92800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8c0d2990 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.915+0000 7f2f92800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8c0d5f20 con 0x7f2f8c0b4000 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36394/0 (socket says 172.21.15.46:36394) 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 -- 172.21.15.46:0/1958425486 learned_addr learned my addr 172.21.15.46:0/1958425486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 -- 172.21.15.46:0/1958425486 >> v2:172.21.15.142:3300/0 conn(0x7f2f8c0b4000 msgr2=0x7f2f8c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 --2- 172.21.15.46:0/1958425486 >> v2:172.21.15.142:3300/0 conn(0x7f2f8c0b4000 0x7f2f8c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 -- 172.21.15.46:0/1958425486 >> v1:172.21.15.142:6789/0 conn(0x7f2f8c0d1f60 legacy=0x7f2f8c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.916+0000 7f2f91400640 1 -- 172.21.15.46:0/1958425486 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c0d0f10 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.917+0000 7f2f91400640 1 --2- 172.21.15.46:0/1958425486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c08d880 tx=0x7f2f7c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f608fe2938ac8301 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.917+0000 7f2f8be00640 1 -- 172.21.15.46:0/1958425486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c036070 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.917+0000 7f2f8be00640 1 -- 172.21.15.46:0/1958425486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c031600 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.917+0000 7f2f8be00640 1 -- 172.21.15.46:0/1958425486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0319d0 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 -- 172.21.15.46:0/1958425486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15f4e0 msgr2=0x7f2f8c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 --2- 172.21.15.46:0/1958425486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c08d880 tx=0x7f2f7c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 -- 172.21.15.46:0/1958425486 shutdown_connections 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 --2- 172.21.15.46:0/1958425486 >> v2:172.21.15.142:3300/0 conn(0x7f2f8c0b4000 0x7f2f8c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 --2- 172.21.15.46:0/1958425486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c0d2df0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 -- 172.21.15.46:0/1958425486 >> 172.21.15.46:0/1958425486 conn(0x7f2f8c0d5010 msgr2=0x7f2f8c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 -- 172.21.15.46:0/1958425486 shutdown_connections 2024-04-26T23:42:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.918+0000 7f2f92800640 1 -- 172.21.15.46:0/1958425486 wait complete. 2024-04-26T23:42:00.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.920+0000 7f2f92800640 1 Processor -- start 2024-04-26T23:42:00.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.920+0000 7f2f92800640 1 -- start start 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8c0d1f60 0x7f2f8c0a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c15f8c0 con 0x7f2f8c0b4000 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0b1d60 con 0x7f2f8c15f4e0 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f92800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8c0b3290 con 0x7f2f8c0d1f60 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f91400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f91400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36406/0 (socket says 172.21.15.46:36406) 2024-04-26T23:42:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.921+0000 7f2f91400640 1 -- 172.21.15.46:0/21291670 learned_addr learned my addr 172.21.15.46:0/21291670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 -- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8c0d1f60 msgr2=0x7f2f8c0a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8c0d1f60 0x7f2f8c0a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 -- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8c15f4e0 msgr2=0x7f2f8c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 -- 172.21.15.46:0/21291670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c15dd20 con 0x7f2f8c0b4000 2024-04-26T23:42:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f91400640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0093e0 tx=0x7f2f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c041020 con 0x7f2f8c0b4000 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c007800 con 0x7f2f8c0b4000 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c036040 con 0x7f2f8c0b4000 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.922+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c0cc0e0 con 0x7f2f8c0b4000 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.923+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c0a25a0 con 0x7f2f8c0b4000 2024-04-26T23:42:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.924+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c0693b0 con 0x7f2f8c0b4000 2024-04-26T23:42:00.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.925+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7c044020 con 0x7f2f8c0b4000 2024-04-26T23:42:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.926+0000 7f2f89600640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.926+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2f7c0acd30 con 0x7f2f8c0b4000 2024-04-26T23:42:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.926+0000 7f2f90a00640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.928+0000 7f2f90a00640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c1f8be0 tx=0x7f2f78006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:00.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:00.931+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f7c04b030 con 0x7f2f8c0b4000 2024-04-26T23:42:01.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.136+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f8c0d46d0 con 0x7f2f8c0b4000 2024-04-26T23:42:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.140+0000 7f2f89600640 1 -- 172.21.15.46:0/21291670 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16906 (secure 0 0 0) 0x7f2f7c07af60 con 0x7f2f8c0b4000 2024-04-26T23:42:01.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:58.143214+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:01.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.142+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 msgr2=0x7f2f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.142+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c1f8be0 tx=0x7f2f78006c90 comp rx=0 tx=0).stop 2024-04-26T23:42:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.142+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 msgr2=0x7f2f8c15d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.142+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0093e0 tx=0x7f2f7c004a40 comp rx=0 tx=0).stop 2024-04-26T23:42:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 shutdown_connections 2024-04-26T23:42:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8c15f4e0 0x7f2f8c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8c0d1f60 0x7f2f8c0a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 --2- 172.21.15.46:0/21291670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0b4000 0x7f2f8c15d7e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 >> 172.21.15.46:0/21291670 conn(0x7f2f8c0d5010 msgr2=0x7f2f8c0cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 shutdown_connections 2024-04-26T23:42:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.143+0000 7f2f92800640 1 -- 172.21.15.46:0/21291670 wait complete. 2024-04-26T23:42:01.166 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:42:01.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:42:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.374+0000 7fd3bec00640 1 Processor -- start 2024-04-26T23:42:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.374+0000 7fd3bec00640 1 -- start start 2024-04-26T23:42:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b815f1d0 0x7fd3b80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b80cd770 0x7fd3b80c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b80b03c0 con 0x7fd3b815edf0 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80cf5c0 con 0x7fd3b80cd770 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b80cb680 con 0x7fd3b815f1d0 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3be200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b80cd770 0x7fd3b80c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bd800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3b815edf0 0x7fd3b80cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48158/0 (socket says 172.21.15.46:48158) 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bd800640 1 -- 172.21.15.46:0/1554625013 learned_addr learned my addr 172.21.15.46:0/1554625013 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.376+0000 7fd3bce00640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b815f1d0 0x7fd3b80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.377+0000 7fd3be200640 1 -- 172.21.15.46:0/1554625013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b815f1d0 msgr2=0x7fd3b80cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:01.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.377+0000 7fd3be200640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b815f1d0 0x7fd3b80cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.377+0000 7fd3be200640 1 -- 172.21.15.46:0/1554625013 >> v1:172.21.15.46:6789/0 conn(0x7fd3b815edf0 legacy=0x7fd3b80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.377+0000 7fd3b7e00640 1 -- 172.21.15.46:0/1554625013 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837814847 0 0) 0x7fd3b80b03c0 con 0x7fd3b815edf0 2024-04-26T23:42:01.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.377+0000 7fd3be200640 1 -- 172.21.15.46:0/1554625013 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80cb960 con 0x7fd3b80cd770 2024-04-26T23:42:01.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.378+0000 7fd3be200640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b80c5940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009d90 tx=0x7fd3b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e1a572b2e6781a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3b7e00640 1 -- 172.21.15.46:0/1554625013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0007740 con 0x7fd3b80cd770 2024-04-26T23:42:01.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3b7e00640 1 -- 172.21.15.46:0/1554625013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b00078e0 con 0x7fd3b80cd770 2024-04-26T23:42:01.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3b7e00640 1 -- 172.21.15.46:0/1554625013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0007cb0 con 0x7fd3b80cd770 2024-04-26T23:42:01.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 -- 172.21.15.46:0/1554625013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 msgr2=0x7fd3b80c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b80c5940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009d90 tx=0x7fd3b002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 -- 172.21.15.46:0/1554625013 shutdown_connections 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b80c5940 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 --2- 172.21.15.46:0/1554625013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b815f1d0 0x7fd3b80cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 -- 172.21.15.46:0/1554625013 >> 172.21.15.46:0/1554625013 conn(0x7fd3b80b26d0 msgr2=0x7fd3b80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.379+0000 7fd3bec00640 1 -- 172.21.15.46:0/1554625013 shutdown_connections 2024-04-26T23:42:01.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.380+0000 7fd3bec00640 1 -- 172.21.15.46:0/1554625013 wait complete. 2024-04-26T23:42:01.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.381+0000 7fd3bec00640 1 Processor -- start 2024-04-26T23:42:01.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.381+0000 7fd3bec00640 1 -- start start 2024-04-26T23:42:01.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b815edf0 0x7fd3b80c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b814f7d0 0x7fd3b81f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b80cc200 con 0x7fd3b80cd770 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b80cbf20 con 0x7fd3b815edf0 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b80cb1a0 con 0x7fd3b814f7d0 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36436/0 (socket says 172.21.15.46:36436) 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bd800640 1 -- 172.21.15.46:0/133779875 learned_addr learned my addr 172.21.15.46:0/133779875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3bce00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b815edf0 0x7fd3b80c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.383+0000 7fd3be200640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b814f7d0 0x7fd3b81f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 -- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b814f7d0 msgr2=0x7fd3b81f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b814f7d0 0x7fd3b81f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 -- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b815edf0 msgr2=0x7fd3b80c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b815edf0 0x7fd3b80c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 -- 172.21.15.46:0/133779875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b81f5980 con 0x7fd3b80cd770 2024-04-26T23:42:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bd800640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c4a0 tx=0x7fd3ac00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac005ce0 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0072b0 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.384+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b81f5c60 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.385+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b80c92a0 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.386+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b80693b0 con 0x7fd3b80cd770 2024-04-26T23:42:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.387+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3ac01a070 con 0x7fd3b80cd770 2024-04-26T23:42:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.387+0000 7fd3b5600640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 0x7fd39806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.388+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd3ac08b0d0 con 0x7fd3b80cd770 2024-04-26T23:42:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.388+0000 7fd3bce00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 0x7fd39806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.389+0000 7fd3bce00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 0x7fd39806ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4006fd0 tx=0x7fd3a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.392+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3ac0592d0 con 0x7fd3b80cd770 2024-04-26T23:42:01.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.594+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3b80c60a0 con 0x7fd39806c920 2024-04-26T23:42:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.613+0000 7fd3b5600640 1 -- 172.21.15.46:0/133779875 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240078 (secure 0 0 0) 0x7fd3b80c60a0 con 0x7fd39806c920 2024-04-26T23:42:01.621 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:42:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.619+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 msgr2=0x7fd39806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.619+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 0x7fd39806ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4006fd0 tx=0x7fd3a4008040 comp rx=0 tx=0).stop 2024-04-26T23:42:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.619+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 msgr2=0x7fd3b814f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.619+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c4a0 tx=0x7fd3ac00c960 comp rx=0 tx=0).stop 2024-04-26T23:42:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 shutdown_connections 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39806c920 0x7fd39806ede0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b814f7d0 0x7fd3b81f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b815edf0 0x7fd3b80c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 --2- 172.21.15.46:0/133779875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b80cd770 0x7fd3b814f160 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 >> 172.21.15.46:0/133779875 conn(0x7fd3b80b26d0 msgr2=0x7fd3b80758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 shutdown_connections 2024-04-26T23:42:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.620+0000 7fd3bec00640 1 -- 172.21.15.46:0/133779875 wait complete. 2024-04-26T23:42:01.644 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-04-26T23:41:59.993962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296924,"kb_used_data":5884,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754677796,"statfs":{"total":773094113280,"available":772790063104,"internally_reserved":0,"allocated":6025216,"data_stored":3315711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.887978"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.118148+0000","last_change":"2024-04-26T23:41:55.118148+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.118148+0000","last_undegraded":"2024-04-26T23:41:55.118148+0000","last_fullsized":"2024-04-26T23:41:55.118148+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.115084+0000","last_change":"2024-04-26T23:41:55.115084+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.115084+0000","last_undegraded":"2024-04-26T23:41:55.115084+0000","last_fullsized":"2024-04-26T23:41:55.115084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.115470+0000","last_change":"2024-04-26T23:41:55.115470+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.115470+0000","last_undegraded":"2024-04-26T23:41:55.115470+0000","last_fullsized":"2024-04-26T23:41:55.115470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.121171+0000","last_change":"2024-04-26T23:41:55.121171+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.121171+0000","last_undegraded":"2024-04-26T23:41:55.121171+0000","last_fullsized":"2024-04-26T23:41:55.121171+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.116378+0000","last_change":"2024-04-26T23:41:55.116378+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.116378+0000","last_undegraded":"2024-04-26T23:41:55.116378+0000","last_fullsized":"2024-04-26T23:41:55.116378+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.110635+0000","last_change":"2024-04-26T23:41:55.110635+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.110635+0000","last_undegraded":"2024-04-26T23:41:55.110635+0000","last_fullsized":"2024-04-26T23:41:55.110635+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.116468+0000","last_change":"2024-04-26T23:41:55.116468+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.116468+0000","last_undegraded":"2024-04-26T23:41:55.116468+0000","last_fullsized":"2024-04-26T23:41:55.116468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.113376+0000","last_change":"2024-04-26T23:41:55.113376+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.113376+0000","last_undegraded":"2024-04-26T23:41:55.113376+0000","last_fullsized":"2024-04-26T23:41:55.113376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.120095+0000","last_change":"2024-04-26T23:41:55.120095+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.120095+0000","last_undegraded":"2024-04-26T23:41:55.120095+0000","last_fullsized":"2024-04-26T23:41:55.120095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.110543+0000","last_change":"2024-04-26T23:41:55.110543+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.110543+0000","last_undegraded":"2024-04-26T23:41:55.110543+0000","last_fullsized":"2024-04-26T23:41:55.110543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.118799+0000","last_change":"2024-04-26T23:41:55.118799+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.118799+0000","last_undegraded":"2024-04-26T23:41:55.118799+0000","last_fullsized":"2024-04-26T23:41:55.118799+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117814+0000","last_change":"2024-04-26T23:41:55.117814+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117814+0000","last_undegraded":"2024-04-26T23:41:55.117814+0000","last_fullsized":"2024-04-26T23:41:55.117814+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.114857+0000","last_change":"2024-04-26T23:41:55.114857+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.114857+0000","last_undegraded":"2024-04-26T23:41:55.114857+0000","last_fullsized":"2024-04-26T23:41:55.114857+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.116566+0000","last_change":"2024-04-26T23:41:55.116566+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.116566+0000","last_undegraded":"2024-04-26T23:41:55.116566+0000","last_fullsized":"2024-04-26T23:41:55.116566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.116621+0000","last_change":"2024-04-26T23:41:55.116621+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.116621+0000","last_undegraded":"2024-04-26T23:41:55.116621+0000","last_fullsized":"2024-04-26T23:41:55.116621+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117514+0000","last_change":"2024-04-26T23:41:55.117514+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117514+0000","last_undegraded":"2024-04-26T23:41:55.117514+0000","last_fullsized":"2024-04-26T23:41:55.117514+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.112902+0000","last_change":"2024-04-26T23:41:55.112902+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.112902+0000","last_undegraded":"2024-04-26T23:41:55.112902+0000","last_fullsized":"2024-04-26T23:41:55.112902+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.110710+0000","last_change":"2024-04-26T23:41:55.110710+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.110710+0000","last_undegraded":"2024-04-26T23:41:55.110710+0000","last_fullsized":"2024-04-26T23:41:55.110710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.118864+0000","last_change":"2024-04-26T23:41:55.118864+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.118864+0000","last_undegraded":"2024-04-26T23:41:55.118864+0000","last_fullsized":"2024-04-26T23:41:55.118864+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117582+0000","last_change":"2024-04-26T23:41:55.117582+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117582+0000","last_undegraded":"2024-04-26T23:41:55.117582+0000","last_fullsized":"2024-04-26T23:41:55.117582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.115566+0000","last_change":"2024-04-26T23:41:55.115566+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.115566+0000","last_undegraded":"2024-04-26T23:41:55.115566+0000","last_fullsized":"2024-04-26T23:41:55.115566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117979+0000","last_change":"2024-04-26T23:41:55.117979+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117979+0000","last_undegraded":"2024-04-26T23:41:55.117979+0000","last_fullsized":"2024-04-26T23:41:55.117979+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.112466+0000","last_change":"2024-04-26T23:41:55.112466+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.112466+0000","last_undegraded":"2024-04-26T23:41:55.112466+0000","last_fullsized":"2024-04-26T23:41:55.112466+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.110729+0000","last_change":"2024-04-26T23:41:55.110729+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.110729+0000","last_undegraded":"2024-04-26T23:41:55.110729+0000","last_fullsized":"2024-04-26T23:41:55.110729+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.116757+0000","last_change":"2024-04-26T23:41:55.116757+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.116757+0000","last_undegraded":"2024-04-26T23:41:55.116757+0000","last_fullsized":"2024-04-26T23:41:55.116757+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117668+0000","last_change":"2024-04-26T23:41:55.117668+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117668+0000","last_undegraded":"2024-04-26T23:41:55.117668+0000","last_fullsized":"2024-04-26T23:41:55.117668+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.112460+0000","last_change":"2024-04-26T23:41:55.112460+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.112460+0000","last_undegraded":"2024-04-26T23:41:55.112460+0000","last_fullsized":"2024-04-26T23:41:55.112460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117659+0000","last_change":"2024-04-26T23:41:55.117659+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117659+0000","last_undegraded":"2024-04-26T23:41:55.117659+0000","last_fullsized":"2024-04-26T23:41:55.117659+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.117457+0000","last_change":"2024-04-26T23:41:55.117457+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.117457+0000","last_undegraded":"2024-04-26T23:41:55.117457+0000","last_fullsized":"2024-04-26T23:41:55.117457+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.110544+0000","last_change":"2024-04-26T23:41:55.110544+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.110544+0000","last_undegraded":"2024-04-26T23:41:55.110544+0000","last_fullsized":"2024-04-26T23:41:55.110544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.120936+0000","last_change":"2024-04-26T23:41:55.120936+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.120936+0000","last_undegraded":"2024-04-26T23:41:55.120936+0000","last_fullsized":"2024-04-26T23:41:55.120936+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:41:56.104712+0000","last_change":"2024-04-26T23:41:56.104712+0000","last_active":"2024-04-26T23:41:56.104712+0000","last_peered":"2024-04-26T23:41:56.104712+0000","last_clean":"2024-04-26T23:41:56.104712+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:56.104712+0000","last_undegraded":"2024-04-26T23:41:56.104712+0000","last_fullsized":"2024-04-26T23:41:56.104712+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-26T23:41:55.113593+0000","last_change":"2024-04-26T23:41:55.113593+0000","last_active":"2024-04-26T23:41:55.089236+0000","last_peered":"2024-04-26T23:41:55.089236+0000","last_clean":"2024-04-26T23:41:55.089236+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:41:55.113593+0000","last_undegraded":"2024-04-26T23:41:55.113593+0000","last_fullsized":"2024-04-26T23:41:55.113593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":226,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T23:41:55.118381+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:41:55.118381+0000","last_peered":"2024-04-26T23:41:55.118381+0000","last_clean":"2024-04-26T23:41:55.118381+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:41:55.118381+0000","last_undegraded":"2024-04-26T23:41:55.118381+0000","last_fullsized":"2024-04-26T23:41:55.118381+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1200128,"data_stored":858016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35920,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335920,"statfs":{"total":96636764160,"available":96599982080,"internally_reserved":0,"allocated":593920,"data_stored":261439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":267648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":267648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40092,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331748,"statfs":{"total":96636764160,"available":96595709952,"internally_reserved":0,"allocated":606208,"data_stored":267648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":267648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":267648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40672,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331168,"statfs":{"total":96636764160,"available":96595116032,"internally_reserved":0,"allocated":1200128,"data_stored":858016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:42:01.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:42:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.892+0000 7f7f56600640 1 Processor -- start 2024-04-26T23:42:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.892+0000 7f7f56600640 1 -- start start 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.893+0000 7f7f56600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f5006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.893+0000 7f7f56600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7f500b86b0 0x7f7f500d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.893+0000 7f7f56600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f500d21d0 con 0x7f7f500d1330 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.893+0000 7f7f56600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7f500d1ef0 con 0x7f7f500b86b0 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.893+0000 7f7f56600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f500b6bc0 con 0x7f7f500d0f50 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f500d1330 0x7f7f500b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33332/0 (socket says 172.21.15.46:33332) 2024-04-26T23:42:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4fe00640 1 -- 172.21.15.46:0/2366607070 learned_addr learned my addr 172.21.15.46:0/2366607070 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f55200640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f5006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f55c00640 1 --2- 172.21.15.46:0/2366607070 >> v2:172.21.15.142:3300/0 conn(0x7f7f500b86b0 0x7f7f500d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642038494 0 0) 0x7f7f500d21d0 con 0x7f7f500d1330 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f34003520 con 0x7f7f500d1330 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2960750601 0 0) 0x7f7f34003520 con 0x7f7f500d1330 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 msgr2=0x7f7f5006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.894+0000 7f7f4f400640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f5006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 >> v2:172.21.15.142:3300/0 conn(0x7f7f500b86b0 msgr2=0x7f7f500d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 --2- 172.21.15.46:0/2366607070 >> v2:172.21.15.142:3300/0 conn(0x7f7f500b86b0 0x7f7f500d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f500b6ea0 con 0x7f7f500d1330 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7f38001670 con 0x7f7f500d1330 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 >> v1:172.21.15.46:6789/0 conn(0x7f7f500d1330 legacy=0x7f7f500b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34003f30 0x7f7f34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f34004d40 0x7f7f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500d1ef0 con 0x7f7f34004d40 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500b6bc0 con 0x7f7f34003f30 2024-04-26T23:42:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3400ba10 con 0x7f7f500d0f50 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f55c00640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f34004d40 0x7f7f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34003f30 0x7f7f34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.895+0000 7f7f55200640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.896+0000 7f7f55c00640 1 -- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 msgr2=0x7f7f340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.896+0000 7f7f55c00640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f500d0f50 0x7f7f340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.896+0000 7f7f55c00640 1 -- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34003f30 msgr2=0x7f7f34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.896+0000 7f7f55c00640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34003f30 0x7f7f34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.896+0000 7f7f55c00640 1 -- 172.21.15.46:0/2366607070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f34003520 con 0x7f7f34004d40 2024-04-26T23:42:01.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.897+0000 7f7f55c00640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f34004d40 0x7f7f340051c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7f48007360 tx=0x7f7f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a6401a22df9e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.897+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48009be0 con 0x7f7f34004d40 2024-04-26T23:42:01.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.897+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4802e620 con 0x7f7f34004d40 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.897+0000 7f7f4f400640 1 -- 172.21.15.46:0/2366607070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4802e9f0 con 0x7f7f34004d40 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 -- 172.21.15.46:0/2366607070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f34004d40 msgr2=0x7f7f340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 --2- 172.21.15.46:0/2366607070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f34004d40 0x7f7f340051c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7f48007360 tx=0x7f7f34005830 comp rx=0 tx=0).stop 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f55c00640 1 -- 172.21.15.46:0/2366607070 reap_dead start 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 -- 172.21.15.46:0/2366607070 shutdown_connections 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 -- 172.21.15.46:0/2366607070 >> 172.21.15.46:0/2366607070 conn(0x7f7f500d2cc0 msgr2=0x7f7f500d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 -- 172.21.15.46:0/2366607070 shutdown_connections 2024-04-26T23:42:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.898+0000 7f7f56600640 1 -- 172.21.15.46:0/2366607070 wait complete. 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.900+0000 7f7f56600640 1 Processor -- start 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.900+0000 7f7f56600640 1 -- start start 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 0x7f7f501f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5013fed0 0x7f7f500b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5015f7c0 con 0x7f7f5013fa50 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5015f640 con 0x7f7f34004d40 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f56600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f500b3ff0 con 0x7f7f5013fed0 2024-04-26T23:42:01.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40430/0 (socket says 172.21.15.46:40430) 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.901+0000 7f7f4fe00640 1 -- 172.21.15.46:0/269952601 learned_addr learned my addr 172.21.15.46:0/269952601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f55c00640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5013fed0 0x7f7f500b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f55200640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 0x7f7f501f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 -- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5013fed0 msgr2=0x7f7f500b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5013fed0 0x7f7f500b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 -- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 msgr2=0x7f7f501f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 0x7f7f501f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 -- 172.21.15.46:0/269952601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f500b0470 con 0x7f7f5013fa50 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f55200640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 0x7f7f501f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4fe00640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e870 tx=0x7f7f3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800c8d0 con 0x7f7f5013fa50 2024-04-26T23:42:01.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38018070 con 0x7f7f5013fa50 2024-04-26T23:42:01.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800ccd0 con 0x7f7f5013fa50 2024-04-26T23:42:01.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.902+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f500b0750 con 0x7f7f5013fa50 2024-04-26T23:42:01.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.903+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f500b0d10 con 0x7f7f5013fa50 2024-04-26T23:42:01.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.904+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50069ac0 con 0x7f7f5013fa50 2024-04-26T23:42:01.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.905+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f3801d460 con 0x7f7f5013fa50 2024-04-26T23:42:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.906+0000 7f7f4cc00640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.906+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7f3808bdb0 con 0x7f7f5013fa50 2024-04-26T23:42:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.906+0000 7f7f55200640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:01.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.907+0000 7f7f55200640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009a00 tx=0x7f7f40008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:01.910+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f38090070 con 0x7f7f5013fa50 2024-04-26T23:42:02.079 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:42:02.076+0000 7f45afa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:42:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.118+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f5006dc50 con 0x7f7f2c06c8f0 2024-04-26T23:42:02.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.138+0000 7f7f4cc00640 1 -- 172.21.15.46:0/269952601 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f7f5006dc50 con 0x7f7f2c06c8f0 2024-04-26T23:42:02.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 msgr2=0x7f7f2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009a00 tx=0x7f7f40008040 comp rx=0 tx=0).stop 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 msgr2=0x7f7f500af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e870 tx=0x7f7f3800ed30 comp rx=0 tx=0).stop 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 shutdown_connections 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7f2c06c8f0 0x7f7f2c06edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f5013fed0 0x7f7f500b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5013fa50 0x7f7f500af9f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 --2- 172.21.15.46:0/269952601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f34004d40 0x7f7f501f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 >> 172.21.15.46:0/269952601 conn(0x7f7f500d2cc0 msgr2=0x7f7f500b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 shutdown_connections 2024-04-26T23:42:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.143+0000 7f7f56600640 1 -- 172.21.15.46:0/269952601 wait complete. 2024-04-26T23:42:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-26T23:42:01.994564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297320,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754677400,"statfs":{"total":773094113280,"available":772789657600,"internally_reserved":0,"allocated":6397952,"data_stored":3490948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001921"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947467+0000","last_change":"2024-04-26T23:41:57.152224+0000","last_active":"2024-04-26T23:41:58.947467+0000","last_peered":"2024-04-26T23:41:58.947467+0000","last_clean":"2024-04-26T23:41:58.947467+0000","last_became_active":"2024-04-26T23:41:57.151954+0000","last_became_peered":"2024-04-26T23:41:57.151954+0000","last_unstale":"2024-04-26T23:41:58.947467+0000","last_undegraded":"2024-04-26T23:41:58.947467+0000","last_fullsized":"2024-04-26T23:41:58.947467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:13.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.166542+0000","last_change":"2024-04-26T23:41:57.157840+0000","last_active":"2024-04-26T23:41:58.166542+0000","last_peered":"2024-04-26T23:41:58.166542+0000","last_clean":"2024-04-26T23:41:58.166542+0000","last_became_active":"2024-04-26T23:41:57.157495+0000","last_became_peered":"2024-04-26T23:41:57.157495+0000","last_unstale":"2024-04-26T23:41:58.166542+0000","last_undegraded":"2024-04-26T23:41:58.166542+0000","last_fullsized":"2024-04-26T23:41:58.166542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:02.017547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157492+0000","last_change":"2024-04-26T23:41:57.163921+0000","last_active":"2024-04-26T23:41:58.157492+0000","last_peered":"2024-04-26T23:41:58.157492+0000","last_clean":"2024-04-26T23:41:58.157492+0000","last_became_active":"2024-04-26T23:41:57.163637+0000","last_became_peered":"2024-04-26T23:41:57.163637+0000","last_unstale":"2024-04-26T23:41:58.157492+0000","last_undegraded":"2024-04-26T23:41:58.157492+0000","last_fullsized":"2024-04-26T23:41:58.157492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:17:47.601785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.165336+0000","last_change":"2024-04-26T23:41:57.163985+0000","last_active":"2024-04-26T23:41:58.165336+0000","last_peered":"2024-04-26T23:41:58.165336+0000","last_clean":"2024-04-26T23:41:58.165336+0000","last_became_active":"2024-04-26T23:41:57.160717+0000","last_became_peered":"2024-04-26T23:41:57.160717+0000","last_unstale":"2024-04-26T23:41:58.165336+0000","last_undegraded":"2024-04-26T23:41:58.165336+0000","last_fullsized":"2024-04-26T23:41:58.165336+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.497942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967635+0000","last_change":"2024-04-26T23:41:57.169883+0000","last_active":"2024-04-26T23:41:58.967635+0000","last_peered":"2024-04-26T23:41:58.967635+0000","last_clean":"2024-04-26T23:41:58.967635+0000","last_became_active":"2024-04-26T23:41:57.169303+0000","last_became_peered":"2024-04-26T23:41:57.169303+0000","last_unstale":"2024-04-26T23:41:58.967635+0000","last_undegraded":"2024-04-26T23:41:58.967635+0000","last_fullsized":"2024-04-26T23:41:58.967635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:48.879053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946868+0000","last_change":"2024-04-26T23:41:57.169207+0000","last_active":"2024-04-26T23:41:58.946868+0000","last_peered":"2024-04-26T23:41:58.946868+0000","last_clean":"2024-04-26T23:41:58.946868+0000","last_became_active":"2024-04-26T23:41:57.168875+0000","last_became_peered":"2024-04-26T23:41:57.168875+0000","last_unstale":"2024-04-26T23:41:58.946868+0000","last_undegraded":"2024-04-26T23:41:58.946868+0000","last_fullsized":"2024-04-26T23:41:58.946868+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:11:17.337609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.156808+0000","last_change":"2024-04-26T23:41:57.160501+0000","last_active":"2024-04-26T23:41:58.156808+0000","last_peered":"2024-04-26T23:41:58.156808+0000","last_clean":"2024-04-26T23:41:58.156808+0000","last_became_active":"2024-04-26T23:41:57.160176+0000","last_became_peered":"2024-04-26T23:41:57.160176+0000","last_unstale":"2024-04-26T23:41:58.156808+0000","last_undegraded":"2024-04-26T23:41:58.156808+0000","last_fullsized":"2024-04-26T23:41:58.156808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:09.984406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967956+0000","last_change":"2024-04-26T23:41:57.150830+0000","last_active":"2024-04-26T23:41:58.967956+0000","last_peered":"2024-04-26T23:41:58.967956+0000","last_clean":"2024-04-26T23:41:58.967956+0000","last_became_active":"2024-04-26T23:41:57.150399+0000","last_became_peered":"2024-04-26T23:41:57.150399+0000","last_unstale":"2024-04-26T23:41:58.967956+0000","last_undegraded":"2024-04-26T23:41:58.967956+0000","last_fullsized":"2024-04-26T23:41:58.967956+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:44.401901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970598+0000","last_change":"2024-04-26T23:41:57.167754+0000","last_active":"2024-04-26T23:41:58.970598+0000","last_peered":"2024-04-26T23:41:58.970598+0000","last_clean":"2024-04-26T23:41:58.970598+0000","last_became_active":"2024-04-26T23:41:57.167348+0000","last_became_peered":"2024-04-26T23:41:57.167348+0000","last_unstale":"2024-04-26T23:41:58.970598+0000","last_undegraded":"2024-04-26T23:41:58.970598+0000","last_fullsized":"2024-04-26T23:41:58.970598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:30.200458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947350+0000","last_change":"2024-04-26T23:41:57.170635+0000","last_active":"2024-04-26T23:41:58.947350+0000","last_peered":"2024-04-26T23:41:58.947350+0000","last_clean":"2024-04-26T23:41:58.947350+0000","last_became_active":"2024-04-26T23:41:57.170362+0000","last_became_peered":"2024-04-26T23:41:57.170362+0000","last_unstale":"2024-04-26T23:41:58.947350+0000","last_undegraded":"2024-04-26T23:41:58.947350+0000","last_fullsized":"2024-04-26T23:41:58.947350+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:06.159391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970139+0000","last_change":"2024-04-26T23:41:57.166901+0000","last_active":"2024-04-26T23:41:58.970139+0000","last_peered":"2024-04-26T23:41:58.970139+0000","last_clean":"2024-04-26T23:41:58.970139+0000","last_became_active":"2024-04-26T23:41:57.166487+0000","last_became_peered":"2024-04-26T23:41:57.166487+0000","last_unstale":"2024-04-26T23:41:58.970139+0000","last_undegraded":"2024-04-26T23:41:58.970139+0000","last_fullsized":"2024-04-26T23:41:58.970139+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:57.703880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970093+0000","last_change":"2024-04-26T23:41:57.168957+0000","last_active":"2024-04-26T23:41:58.970093+0000","last_peered":"2024-04-26T23:41:58.970093+0000","last_clean":"2024-04-26T23:41:58.970093+0000","last_became_active":"2024-04-26T23:41:57.168191+0000","last_became_peered":"2024-04-26T23:41:57.168191+0000","last_unstale":"2024-04-26T23:41:58.970093+0000","last_undegraded":"2024-04-26T23:41:58.970093+0000","last_fullsized":"2024-04-26T23:41:58.970093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:08.967561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946724+0000","last_change":"2024-04-26T23:41:57.167755+0000","last_active":"2024-04-26T23:41:58.946724+0000","last_peered":"2024-04-26T23:41:58.946724+0000","last_clean":"2024-04-26T23:41:58.946724+0000","last_became_active":"2024-04-26T23:41:57.167408+0000","last_became_peered":"2024-04-26T23:41:57.167408+0000","last_unstale":"2024-04-26T23:41:58.946724+0000","last_undegraded":"2024-04-26T23:41:58.946724+0000","last_fullsized":"2024-04-26T23:41:58.946724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:22.808268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157180+0000","last_change":"2024-04-26T23:41:57.172335+0000","last_active":"2024-04-26T23:41:58.157180+0000","last_peered":"2024-04-26T23:41:58.157180+0000","last_clean":"2024-04-26T23:41:58.157180+0000","last_became_active":"2024-04-26T23:41:57.153581+0000","last_became_peered":"2024-04-26T23:41:57.153581+0000","last_unstale":"2024-04-26T23:41:58.157180+0000","last_undegraded":"2024-04-26T23:41:58.157180+0000","last_fullsized":"2024-04-26T23:41:58.157180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:00.972010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.165316+0000","last_change":"2024-04-26T23:41:57.163535+0000","last_active":"2024-04-26T23:41:58.165316+0000","last_peered":"2024-04-26T23:41:58.165316+0000","last_clean":"2024-04-26T23:41:58.165316+0000","last_became_active":"2024-04-26T23:41:57.157487+0000","last_became_peered":"2024-04-26T23:41:57.157487+0000","last_unstale":"2024-04-26T23:41:58.165316+0000","last_undegraded":"2024-04-26T23:41:58.165316+0000","last_fullsized":"2024-04-26T23:41:58.165316+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:50.183015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946390+0000","last_change":"2024-04-26T23:41:57.166277+0000","last_active":"2024-04-26T23:41:58.946390+0000","last_peered":"2024-04-26T23:41:58.946390+0000","last_clean":"2024-04-26T23:41:58.946390+0000","last_became_active":"2024-04-26T23:41:57.165941+0000","last_became_peered":"2024-04-26T23:41:57.165941+0000","last_unstale":"2024-04-26T23:41:58.946390+0000","last_undegraded":"2024-04-26T23:41:58.946390+0000","last_fullsized":"2024-04-26T23:41:58.946390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:04.014218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158049+0000","last_change":"2024-04-26T23:41:57.160204+0000","last_active":"2024-04-26T23:41:58.158049+0000","last_peered":"2024-04-26T23:41:58.158049+0000","last_clean":"2024-04-26T23:41:58.158049+0000","last_became_active":"2024-04-26T23:41:57.155070+0000","last_became_peered":"2024-04-26T23:41:57.155070+0000","last_unstale":"2024-04-26T23:41:58.158049+0000","last_undegraded":"2024-04-26T23:41:58.158049+0000","last_fullsized":"2024-04-26T23:41:58.158049+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:06.124241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.166522+0000","last_change":"2024-04-26T23:41:57.164699+0000","last_active":"2024-04-26T23:41:58.166522+0000","last_peered":"2024-04-26T23:41:58.166522+0000","last_clean":"2024-04-26T23:41:58.166522+0000","last_became_active":"2024-04-26T23:41:57.164424+0000","last_became_peered":"2024-04-26T23:41:57.164424+0000","last_unstale":"2024-04-26T23:41:58.166522+0000","last_undegraded":"2024-04-26T23:41:58.166522+0000","last_fullsized":"2024-04-26T23:41:58.166522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:44.447784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157736+0000","last_change":"2024-04-26T23:41:57.171374+0000","last_active":"2024-04-26T23:41:58.157736+0000","last_peered":"2024-04-26T23:41:58.157736+0000","last_clean":"2024-04-26T23:41:58.157736+0000","last_became_active":"2024-04-26T23:41:57.171089+0000","last_became_peered":"2024-04-26T23:41:57.171089+0000","last_unstale":"2024-04-26T23:41:58.157736+0000","last_undegraded":"2024-04-26T23:41:58.157736+0000","last_fullsized":"2024-04-26T23:41:58.157736+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:09.410447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358701+0000","last_change":"2024-04-26T23:41:57.170697+0000","last_active":"2024-04-26T23:41:58.358701+0000","last_peered":"2024-04-26T23:41:58.358701+0000","last_clean":"2024-04-26T23:41:58.358701+0000","last_became_active":"2024-04-26T23:41:57.170372+0000","last_became_peered":"2024-04-26T23:41:57.170372+0000","last_unstale":"2024-04-26T23:41:58.358701+0000","last_undegraded":"2024-04-26T23:41:58.358701+0000","last_fullsized":"2024-04-26T23:41:58.358701+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:48.180555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358254+0000","last_change":"2024-04-26T23:41:57.163536+0000","last_active":"2024-04-26T23:41:58.358254+0000","last_peered":"2024-04-26T23:41:58.358254+0000","last_clean":"2024-04-26T23:41:58.358254+0000","last_became_active":"2024-04-26T23:41:57.163232+0000","last_became_peered":"2024-04-26T23:41:57.163232+0000","last_unstale":"2024-04-26T23:41:58.358254+0000","last_undegraded":"2024-04-26T23:41:58.358254+0000","last_fullsized":"2024-04-26T23:41:58.358254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:22.551344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.968302+0000","last_change":"2024-04-26T23:41:57.173589+0000","last_active":"2024-04-26T23:41:58.968302+0000","last_peered":"2024-04-26T23:41:58.968302+0000","last_clean":"2024-04-26T23:41:58.968302+0000","last_became_active":"2024-04-26T23:41:57.173267+0000","last_became_peered":"2024-04-26T23:41:57.173267+0000","last_unstale":"2024-04-26T23:41:58.968302+0000","last_undegraded":"2024-04-26T23:41:58.968302+0000","last_fullsized":"2024-04-26T23:41:58.968302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:51.761571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.968760+0000","last_change":"2024-04-26T23:41:57.166207+0000","last_active":"2024-04-26T23:41:58.968760+0000","last_peered":"2024-04-26T23:41:58.968760+0000","last_clean":"2024-04-26T23:41:58.968760+0000","last_became_active":"2024-04-26T23:41:57.165261+0000","last_became_peered":"2024-04-26T23:41:57.165261+0000","last_unstale":"2024-04-26T23:41:58.968760+0000","last_undegraded":"2024-04-26T23:41:58.968760+0000","last_fullsized":"2024-04-26T23:41:58.968760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:55.120192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.156851+0000","last_change":"2024-04-26T23:41:57.156308+0000","last_active":"2024-04-26T23:41:58.156851+0000","last_peered":"2024-04-26T23:41:58.156851+0000","last_clean":"2024-04-26T23:41:58.156851+0000","last_became_active":"2024-04-26T23:41:57.156040+0000","last_became_peered":"2024-04-26T23:41:57.156040+0000","last_unstale":"2024-04-26T23:41:58.156851+0000","last_undegraded":"2024-04-26T23:41:58.156851+0000","last_fullsized":"2024-04-26T23:41:58.156851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:44.447784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158230+0000","last_change":"2024-04-26T23:41:57.165199+0000","last_active":"2024-04-26T23:41:58.158230+0000","last_peered":"2024-04-26T23:41:58.158230+0000","last_clean":"2024-04-26T23:41:58.158230+0000","last_became_active":"2024-04-26T23:41:57.160400+0000","last_became_peered":"2024-04-26T23:41:57.160400+0000","last_unstale":"2024-04-26T23:41:58.158230+0000","last_undegraded":"2024-04-26T23:41:58.158230+0000","last_fullsized":"2024-04-26T23:41:58.158230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:36.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.969936+0000","last_change":"2024-04-26T23:41:57.156560+0000","last_active":"2024-04-26T23:41:58.969936+0000","last_peered":"2024-04-26T23:41:58.969936+0000","last_clean":"2024-04-26T23:41:58.969936+0000","last_became_active":"2024-04-26T23:41:57.154906+0000","last_became_peered":"2024-04-26T23:41:57.154906+0000","last_unstale":"2024-04-26T23:41:58.969936+0000","last_undegraded":"2024-04-26T23:41:58.969936+0000","last_fullsized":"2024-04-26T23:41:58.969936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:59.671057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947532+0000","last_change":"2024-04-26T23:41:57.153054+0000","last_active":"2024-04-26T23:41:58.947532+0000","last_peered":"2024-04-26T23:41:58.947532+0000","last_clean":"2024-04-26T23:41:58.947532+0000","last_became_active":"2024-04-26T23:41:57.152707+0000","last_became_peered":"2024-04-26T23:41:57.152707+0000","last_unstale":"2024-04-26T23:41:58.947532+0000","last_undegraded":"2024-04-26T23:41:58.947532+0000","last_fullsized":"2024-04-26T23:41:58.947532+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:18.933429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358766+0000","last_change":"2024-04-26T23:41:57.160504+0000","last_active":"2024-04-26T23:41:58.358766+0000","last_peered":"2024-04-26T23:41:58.358766+0000","last_clean":"2024-04-26T23:41:58.358766+0000","last_became_active":"2024-04-26T23:41:57.160147+0000","last_became_peered":"2024-04-26T23:41:57.160147+0000","last_unstale":"2024-04-26T23:41:58.358766+0000","last_undegraded":"2024-04-26T23:41:58.358766+0000","last_fullsized":"2024-04-26T23:41:58.358766+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:48.879053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157353+0000","last_change":"2024-04-26T23:41:57.165490+0000","last_active":"2024-04-26T23:41:58.157353+0000","last_peered":"2024-04-26T23:41:58.157353+0000","last_clean":"2024-04-26T23:41:58.157353+0000","last_became_active":"2024-04-26T23:41:57.165083+0000","last_became_peered":"2024-04-26T23:41:57.165083+0000","last_unstale":"2024-04-26T23:41:58.157353+0000","last_undegraded":"2024-04-26T23:41:58.157353+0000","last_fullsized":"2024-04-26T23:41:58.157353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.164911+0000","last_change":"2024-04-26T23:41:57.165947+0000","last_active":"2024-04-26T23:41:58.164911+0000","last_peered":"2024-04-26T23:41:58.164911+0000","last_clean":"2024-04-26T23:41:58.164911+0000","last_became_active":"2024-04-26T23:41:57.163314+0000","last_became_peered":"2024-04-26T23:41:57.163314+0000","last_unstale":"2024-04-26T23:41:58.164911+0000","last_undegraded":"2024-04-26T23:41:58.164911+0000","last_fullsized":"2024-04-26T23:41:58.164911+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:52.961130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157197+0000","last_change":"2024-04-26T23:41:57.153159+0000","last_active":"2024-04-26T23:41:58.157197+0000","last_peered":"2024-04-26T23:41:58.157197+0000","last_clean":"2024-04-26T23:41:58.157197+0000","last_became_active":"2024-04-26T23:41:57.152633+0000","last_became_peered":"2024-04-26T23:41:57.152633+0000","last_unstale":"2024-04-26T23:41:58.157197+0000","last_undegraded":"2024-04-26T23:41:58.157197+0000","last_fullsized":"2024-04-26T23:41:58.157197+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:22.309427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.164040+0000","last_change":"2024-04-26T23:41:57.161480+0000","last_active":"2024-04-26T23:41:58.164040+0000","last_peered":"2024-04-26T23:41:58.164040+0000","last_clean":"2024-04-26T23:41:58.164040+0000","last_became_active":"2024-04-26T23:41:57.158683+0000","last_became_peered":"2024-04-26T23:41:57.158683+0000","last_unstale":"2024-04-26T23:41:58.164040+0000","last_undegraded":"2024-04-26T23:41:58.164040+0000","last_fullsized":"2024-04-26T23:41:58.164040+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:09.984406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163296+0000","last_change":"2024-04-26T23:41:57.168758+0000","last_active":"2024-04-26T23:41:58.163296+0000","last_peered":"2024-04-26T23:41:58.163296+0000","last_clean":"2024-04-26T23:41:58.163296+0000","last_became_active":"2024-04-26T23:41:57.167319+0000","last_became_peered":"2024-04-26T23:41:57.167319+0000","last_unstale":"2024-04-26T23:41:58.163296+0000","last_undegraded":"2024-04-26T23:41:58.163296+0000","last_fullsized":"2024-04-26T23:41:58.163296+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:22.252147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358394+0000","last_change":"2024-04-26T23:41:56.157068+0000","last_active":"2024-04-26T23:41:58.358394+0000","last_peered":"2024-04-26T23:41:58.358394+0000","last_clean":"2024-04-26T23:41:58.358394+0000","last_became_active":"2024-04-26T23:41:56.156695+0000","last_became_peered":"2024-04-26T23:41:56.156695+0000","last_unstale":"2024-04-26T23:41:58.358394+0000","last_undegraded":"2024-04-26T23:41:58.358394+0000","last_fullsized":"2024-04-26T23:41:58.358394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:03.732557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163290+0000","last_change":"2024-04-26T23:41:57.164753+0000","last_active":"2024-04-26T23:41:58.163290+0000","last_peered":"2024-04-26T23:41:58.163290+0000","last_clean":"2024-04-26T23:41:58.163290+0000","last_became_active":"2024-04-26T23:41:57.164042+0000","last_became_peered":"2024-04-26T23:41:57.164042+0000","last_unstale":"2024-04-26T23:41:58.163290+0000","last_undegraded":"2024-04-26T23:41:58.163290+0000","last_fullsized":"2024-04-26T23:41:58.163290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:50.183015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947257+0000","last_change":"2024-04-26T23:41:56.150848+0000","last_active":"2024-04-26T23:41:58.947257+0000","last_peered":"2024-04-26T23:41:58.947257+0000","last_clean":"2024-04-26T23:41:58.947257+0000","last_became_active":"2024-04-26T23:41:56.150570+0000","last_became_peered":"2024-04-26T23:41:56.150570+0000","last_unstale":"2024-04-26T23:41:58.947257+0000","last_undegraded":"2024-04-26T23:41:58.947257+0000","last_fullsized":"2024-04-26T23:41:58.947257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:40.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358382+0000","last_change":"2024-04-26T23:41:57.165537+0000","last_active":"2024-04-26T23:41:58.358382+0000","last_peered":"2024-04-26T23:41:58.358382+0000","last_clean":"2024-04-26T23:41:58.358382+0000","last_became_active":"2024-04-26T23:41:57.165246+0000","last_became_peered":"2024-04-26T23:41:57.165246+0000","last_unstale":"2024-04-26T23:41:58.358382+0000","last_undegraded":"2024-04-26T23:41:58.358382+0000","last_fullsized":"2024-04-26T23:41:58.358382+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:31.096145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947124+0000","last_change":"2024-04-26T23:41:56.156931+0000","last_active":"2024-04-26T23:41:58.947124+0000","last_peered":"2024-04-26T23:41:58.947124+0000","last_clean":"2024-04-26T23:41:58.947124+0000","last_became_active":"2024-04-26T23:41:56.156469+0000","last_became_peered":"2024-04-26T23:41:56.156469+0000","last_unstale":"2024-04-26T23:41:58.947124+0000","last_undegraded":"2024-04-26T23:41:58.947124+0000","last_fullsized":"2024-04-26T23:41:58.947124+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:46.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.969503+0000","last_change":"2024-04-26T23:41:57.156868+0000","last_active":"2024-04-26T23:41:58.969503+0000","last_peered":"2024-04-26T23:41:58.969503+0000","last_clean":"2024-04-26T23:41:58.969503+0000","last_became_active":"2024-04-26T23:41:57.156594+0000","last_became_peered":"2024-04-26T23:41:57.156594+0000","last_unstale":"2024-04-26T23:41:58.969503+0000","last_undegraded":"2024-04-26T23:41:58.969503+0000","last_fullsized":"2024-04-26T23:41:58.969503+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.812337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163299+0000","last_change":"2024-04-26T23:41:56.153625+0000","last_active":"2024-04-26T23:41:58.163299+0000","last_peered":"2024-04-26T23:41:58.163299+0000","last_clean":"2024-04-26T23:41:58.163299+0000","last_became_active":"2024-04-26T23:41:56.153270+0000","last_became_peered":"2024-04-26T23:41:56.153270+0000","last_unstale":"2024-04-26T23:41:58.163299+0000","last_undegraded":"2024-04-26T23:41:58.163299+0000","last_fullsized":"2024-04-26T23:41:58.163299+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:27.498593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967879+0000","last_change":"2024-04-26T23:41:57.177462+0000","last_active":"2024-04-26T23:41:58.967879+0000","last_peered":"2024-04-26T23:41:58.967879+0000","last_clean":"2024-04-26T23:41:58.967879+0000","last_became_active":"2024-04-26T23:41:57.177194+0000","last_became_peered":"2024-04-26T23:41:57.177194+0000","last_unstale":"2024-04-26T23:41:58.967879+0000","last_undegraded":"2024-04-26T23:41:58.967879+0000","last_fullsized":"2024-04-26T23:41:58.967879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:23.792807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157321+0000","last_change":"2024-04-26T23:41:56.148461+0000","last_active":"2024-04-26T23:41:58.157321+0000","last_peered":"2024-04-26T23:41:58.157321+0000","last_clean":"2024-04-26T23:41:58.157321+0000","last_became_active":"2024-04-26T23:41:56.148170+0000","last_became_peered":"2024-04-26T23:41:56.148170+0000","last_unstale":"2024-04-26T23:41:58.157321+0000","last_undegraded":"2024-04-26T23:41:58.157321+0000","last_fullsized":"2024-04-26T23:41:58.157321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:30.467786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.164552+0000","last_change":"2024-04-26T23:41:57.163470+0000","last_active":"2024-04-26T23:41:58.164552+0000","last_peered":"2024-04-26T23:41:58.164552+0000","last_clean":"2024-04-26T23:41:58.164552+0000","last_became_active":"2024-04-26T23:41:57.163145+0000","last_became_peered":"2024-04-26T23:41:57.163145+0000","last_unstale":"2024-04-26T23:41:58.164552+0000","last_undegraded":"2024-04-26T23:41:58.164552+0000","last_fullsized":"2024-04-26T23:41:58.164552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:03.954149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.966997+0000","last_change":"2024-04-26T23:41:56.157357+0000","last_active":"2024-04-26T23:41:58.966997+0000","last_peered":"2024-04-26T23:41:58.966997+0000","last_clean":"2024-04-26T23:41:58.966997+0000","last_became_active":"2024-04-26T23:41:56.157050+0000","last_became_peered":"2024-04-26T23:41:56.157050+0000","last_unstale":"2024-04-26T23:41:58.966997+0000","last_undegraded":"2024-04-26T23:41:58.966997+0000","last_fullsized":"2024-04-26T23:41:58.966997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:46.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970358+0000","last_change":"2024-04-26T23:41:57.156079+0000","last_active":"2024-04-26T23:41:58.970358+0000","last_peered":"2024-04-26T23:41:58.970358+0000","last_clean":"2024-04-26T23:41:58.970358+0000","last_became_active":"2024-04-26T23:41:57.155808+0000","last_became_peered":"2024-04-26T23:41:57.155808+0000","last_unstale":"2024-04-26T23:41:58.970358+0000","last_undegraded":"2024-04-26T23:41:58.970358+0000","last_fullsized":"2024-04-26T23:41:58.970358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:09.179731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358029+0000","last_change":"2024-04-26T23:41:56.156056+0000","last_active":"2024-04-26T23:41:58.358029+0000","last_peered":"2024-04-26T23:41:58.358029+0000","last_clean":"2024-04-26T23:41:58.358029+0000","last_became_active":"2024-04-26T23:41:56.155779+0000","last_became_peered":"2024-04-26T23:41:56.155779+0000","last_unstale":"2024-04-26T23:41:58.358029+0000","last_undegraded":"2024-04-26T23:41:58.358029+0000","last_fullsized":"2024-04-26T23:41:58.358029+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:07.633029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.166406+0000","last_change":"2024-04-26T23:41:57.174142+0000","last_active":"2024-04-26T23:41:58.166406+0000","last_peered":"2024-04-26T23:41:58.166406+0000","last_clean":"2024-04-26T23:41:58.166406+0000","last_became_active":"2024-04-26T23:41:57.173756+0000","last_became_peered":"2024-04-26T23:41:57.173756+0000","last_unstale":"2024-04-26T23:41:58.166406+0000","last_undegraded":"2024-04-26T23:41:58.166406+0000","last_fullsized":"2024-04-26T23:41:58.166406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:09.984406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.156747+0000","last_change":"2024-04-26T23:41:56.155210+0000","last_active":"2024-04-26T23:41:58.156747+0000","last_peered":"2024-04-26T23:41:58.156747+0000","last_clean":"2024-04-26T23:41:58.156747+0000","last_became_active":"2024-04-26T23:41:56.154932+0000","last_became_peered":"2024-04-26T23:41:56.154932+0000","last_unstale":"2024-04-26T23:41:58.156747+0000","last_undegraded":"2024-04-26T23:41:58.156747+0000","last_fullsized":"2024-04-26T23:41:58.156747+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.535868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967220+0000","last_change":"2024-04-26T23:41:57.172321+0000","last_active":"2024-04-26T23:41:58.967220+0000","last_peered":"2024-04-26T23:41:58.967220+0000","last_clean":"2024-04-26T23:41:58.967220+0000","last_became_active":"2024-04-26T23:41:57.172016+0000","last_became_peered":"2024-04-26T23:41:57.172016+0000","last_unstale":"2024-04-26T23:41:58.967220+0000","last_undegraded":"2024-04-26T23:41:58.967220+0000","last_fullsized":"2024-04-26T23:41:58.967220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:31.096145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.166544+0000","last_change":"2024-04-26T23:41:56.156001+0000","last_active":"2024-04-26T23:41:58.166544+0000","last_peered":"2024-04-26T23:41:58.166544+0000","last_clean":"2024-04-26T23:41:58.166544+0000","last_became_active":"2024-04-26T23:41:56.155044+0000","last_became_peered":"2024-04-26T23:41:56.155044+0000","last_unstale":"2024-04-26T23:41:58.166544+0000","last_undegraded":"2024-04-26T23:41:58.166544+0000","last_fullsized":"2024-04-26T23:41:58.166544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:22.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.162933+0000","last_change":"2024-04-26T23:41:57.170934+0000","last_active":"2024-04-26T23:41:58.162933+0000","last_peered":"2024-04-26T23:41:58.162933+0000","last_clean":"2024-04-26T23:41:58.162933+0000","last_became_active":"2024-04-26T23:41:57.170648+0000","last_became_peered":"2024-04-26T23:41:57.170648+0000","last_unstale":"2024-04-26T23:41:58.162933+0000","last_undegraded":"2024-04-26T23:41:58.162933+0000","last_fullsized":"2024-04-26T23:41:58.162933+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:03.954149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967781+0000","last_change":"2024-04-26T23:41:56.158679+0000","last_active":"2024-04-26T23:41:58.967781+0000","last_peered":"2024-04-26T23:41:58.967781+0000","last_clean":"2024-04-26T23:41:58.967781+0000","last_became_active":"2024-04-26T23:41:56.158358+0000","last_became_peered":"2024-04-26T23:41:56.158358+0000","last_unstale":"2024-04-26T23:41:58.967781+0000","last_undegraded":"2024-04-26T23:41:58.967781+0000","last_fullsized":"2024-04-26T23:41:58.967781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:32.950899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157176+0000","last_change":"2024-04-26T23:41:57.171222+0000","last_active":"2024-04-26T23:41:58.157176+0000","last_peered":"2024-04-26T23:41:58.157176+0000","last_clean":"2024-04-26T23:41:58.157176+0000","last_became_active":"2024-04-26T23:41:57.170888+0000","last_became_peered":"2024-04-26T23:41:57.170888+0000","last_unstale":"2024-04-26T23:41:58.157176+0000","last_undegraded":"2024-04-26T23:41:58.157176+0000","last_fullsized":"2024-04-26T23:41:58.157176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:02.017547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.165905+0000","last_change":"2024-04-26T23:41:56.152097+0000","last_active":"2024-04-26T23:41:58.165905+0000","last_peered":"2024-04-26T23:41:58.165905+0000","last_clean":"2024-04-26T23:41:58.165905+0000","last_became_active":"2024-04-26T23:41:56.151764+0000","last_became_peered":"2024-04-26T23:41:56.151764+0000","last_unstale":"2024-04-26T23:41:58.165905+0000","last_undegraded":"2024-04-26T23:41:58.165905+0000","last_fullsized":"2024-04-26T23:41:58.165905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:47.245558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967758+0000","last_change":"2024-04-26T23:41:57.162360+0000","last_active":"2024-04-26T23:41:58.967758+0000","last_peered":"2024-04-26T23:41:58.967758+0000","last_clean":"2024-04-26T23:41:58.967758+0000","last_became_active":"2024-04-26T23:41:57.161473+0000","last_became_peered":"2024-04-26T23:41:57.161473+0000","last_unstale":"2024-04-26T23:41:58.967758+0000","last_undegraded":"2024-04-26T23:41:58.967758+0000","last_fullsized":"2024-04-26T23:41:58.967758+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:48.261034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.969921+0000","last_change":"2024-04-26T23:41:56.148307+0000","last_active":"2024-04-26T23:41:58.969921+0000","last_peered":"2024-04-26T23:41:58.969921+0000","last_clean":"2024-04-26T23:41:58.969921+0000","last_became_active":"2024-04-26T23:41:56.148064+0000","last_became_peered":"2024-04-26T23:41:56.148064+0000","last_unstale":"2024-04-26T23:41:58.969921+0000","last_undegraded":"2024-04-26T23:41:58.969921+0000","last_fullsized":"2024-04-26T23:41:58.969921+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:02.938674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.162796+0000","last_change":"2024-04-26T23:41:57.152349+0000","last_active":"2024-04-26T23:41:58.162796+0000","last_peered":"2024-04-26T23:41:58.162796+0000","last_clean":"2024-04-26T23:41:58.162796+0000","last_became_active":"2024-04-26T23:41:57.152008+0000","last_became_peered":"2024-04-26T23:41:57.152008+0000","last_unstale":"2024-04-26T23:41:58.162796+0000","last_undegraded":"2024-04-26T23:41:58.162796+0000","last_fullsized":"2024-04-26T23:41:58.162796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:22.252147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157516+0000","last_change":"2024-04-26T23:41:56.144484+0000","last_active":"2024-04-26T23:41:58.157516+0000","last_peered":"2024-04-26T23:41:58.157516+0000","last_clean":"2024-04-26T23:41:58.157516+0000","last_became_active":"2024-04-26T23:41:56.141852+0000","last_became_peered":"2024-04-26T23:41:56.141852+0000","last_unstale":"2024-04-26T23:41:58.157516+0000","last_undegraded":"2024-04-26T23:41:58.157516+0000","last_fullsized":"2024-04-26T23:41:58.157516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:30.080670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157948+0000","last_change":"2024-04-26T23:41:57.157751+0000","last_active":"2024-04-26T23:41:58.157948+0000","last_peered":"2024-04-26T23:41:58.157948+0000","last_clean":"2024-04-26T23:41:58.157948+0000","last_became_active":"2024-04-26T23:41:57.157488+0000","last_became_peered":"2024-04-26T23:41:57.157488+0000","last_unstale":"2024-04-26T23:41:58.157948+0000","last_undegraded":"2024-04-26T23:41:58.157948+0000","last_fullsized":"2024-04-26T23:41:58.157948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:58.987249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.357902+0000","last_change":"2024-04-26T23:41:56.159426+0000","last_active":"2024-04-26T23:41:58.357902+0000","last_peered":"2024-04-26T23:41:58.357902+0000","last_clean":"2024-04-26T23:41:58.357902+0000","last_became_active":"2024-04-26T23:41:56.159071+0000","last_became_peered":"2024-04-26T23:41:56.159071+0000","last_unstale":"2024-04-26T23:41:58.357902+0000","last_undegraded":"2024-04-26T23:41:58.357902+0000","last_fullsized":"2024-04-26T23:41:58.357902+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:40.109977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970096+0000","last_change":"2024-04-26T23:41:57.167632+0000","last_active":"2024-04-26T23:41:58.970096+0000","last_peered":"2024-04-26T23:41:58.970096+0000","last_clean":"2024-04-26T23:41:58.970096+0000","last_became_active":"2024-04-26T23:41:57.167210+0000","last_became_peered":"2024-04-26T23:41:57.167210+0000","last_unstale":"2024-04-26T23:41:58.970096+0000","last_undegraded":"2024-04-26T23:41:58.970096+0000","last_fullsized":"2024-04-26T23:41:58.970096+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:00.515873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.165457+0000","last_change":"2024-04-26T23:41:56.151493+0000","last_active":"2024-04-26T23:41:58.165457+0000","last_peered":"2024-04-26T23:41:58.165457+0000","last_clean":"2024-04-26T23:41:58.165457+0000","last_became_active":"2024-04-26T23:41:56.151087+0000","last_became_peered":"2024-04-26T23:41:56.151087+0000","last_unstale":"2024-04-26T23:41:58.165457+0000","last_undegraded":"2024-04-26T23:41:58.165457+0000","last_fullsized":"2024-04-26T23:41:58.165457+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:27.498593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158109+0000","last_change":"2024-04-26T23:41:57.164878+0000","last_active":"2024-04-26T23:41:58.158109+0000","last_peered":"2024-04-26T23:41:58.158109+0000","last_clean":"2024-04-26T23:41:58.158109+0000","last_became_active":"2024-04-26T23:41:57.159830+0000","last_became_peered":"2024-04-26T23:41:57.159830+0000","last_unstale":"2024-04-26T23:41:58.158109+0000","last_undegraded":"2024-04-26T23:41:58.158109+0000","last_fullsized":"2024-04-26T23:41:58.158109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:51.761571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.968068+0000","last_change":"2024-04-26T23:41:56.147391+0000","last_active":"2024-04-26T23:41:58.968068+0000","last_peered":"2024-04-26T23:41:58.968068+0000","last_clean":"2024-04-26T23:41:58.968068+0000","last_became_active":"2024-04-26T23:41:56.147096+0000","last_became_peered":"2024-04-26T23:41:56.147096+0000","last_unstale":"2024-04-26T23:41:58.968068+0000","last_undegraded":"2024-04-26T23:41:58.968068+0000","last_fullsized":"2024-04-26T23:41:58.968068+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:28.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967989+0000","last_change":"2024-04-26T23:41:57.168453+0000","last_active":"2024-04-26T23:41:58.967989+0000","last_peered":"2024-04-26T23:41:58.967989+0000","last_clean":"2024-04-26T23:41:58.967989+0000","last_became_active":"2024-04-26T23:41:57.165083+0000","last_became_peered":"2024-04-26T23:41:57.165083+0000","last_unstale":"2024-04-26T23:41:58.967989+0000","last_undegraded":"2024-04-26T23:41:58.967989+0000","last_fullsized":"2024-04-26T23:41:58.967989+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:06.124241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158169+0000","last_change":"2024-04-26T23:41:56.156862+0000","last_active":"2024-04-26T23:41:58.158169+0000","last_peered":"2024-04-26T23:41:58.158169+0000","last_clean":"2024-04-26T23:41:58.158169+0000","last_became_active":"2024-04-26T23:41:56.156367+0000","last_became_peered":"2024-04-26T23:41:56.156367+0000","last_unstale":"2024-04-26T23:41:58.158169+0000","last_undegraded":"2024-04-26T23:41:58.158169+0000","last_fullsized":"2024-04-26T23:41:58.158169+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:22.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946476+0000","last_change":"2024-04-26T23:41:57.148104+0000","last_active":"2024-04-26T23:41:58.946476+0000","last_peered":"2024-04-26T23:41:58.946476+0000","last_clean":"2024-04-26T23:41:58.946476+0000","last_became_active":"2024-04-26T23:41:57.147698+0000","last_became_peered":"2024-04-26T23:41:57.147698+0000","last_unstale":"2024-04-26T23:41:58.946476+0000","last_undegraded":"2024-04-26T23:41:58.946476+0000","last_fullsized":"2024-04-26T23:41:58.946476+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:28.514069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970165+0000","last_change":"2024-04-26T23:41:56.155786+0000","last_active":"2024-04-26T23:41:58.970165+0000","last_peered":"2024-04-26T23:41:58.970165+0000","last_clean":"2024-04-26T23:41:58.970165+0000","last_became_active":"2024-04-26T23:41:56.155512+0000","last_became_peered":"2024-04-26T23:41:56.155512+0000","last_unstale":"2024-04-26T23:41:58.970165+0000","last_undegraded":"2024-04-26T23:41:58.970165+0000","last_fullsized":"2024-04-26T23:41:58.970165+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:30.701488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967539+0000","last_change":"2024-04-26T23:41:57.175391+0000","last_active":"2024-04-26T23:41:58.967539+0000","last_peered":"2024-04-26T23:41:58.967539+0000","last_clean":"2024-04-26T23:41:58.967539+0000","last_became_active":"2024-04-26T23:41:57.175101+0000","last_became_peered":"2024-04-26T23:41:57.175101+0000","last_unstale":"2024-04-26T23:41:58.967539+0000","last_undegraded":"2024-04-26T23:41:58.967539+0000","last_fullsized":"2024-04-26T23:41:58.967539+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:48.180555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358332+0000","last_change":"2024-04-26T23:41:56.154812+0000","last_active":"2024-04-26T23:41:58.358332+0000","last_peered":"2024-04-26T23:41:58.358332+0000","last_clean":"2024-04-26T23:41:58.358332+0000","last_became_active":"2024-04-26T23:41:56.154487+0000","last_became_peered":"2024-04-26T23:41:56.154487+0000","last_unstale":"2024-04-26T23:41:58.358332+0000","last_undegraded":"2024-04-26T23:41:58.358332+0000","last_fullsized":"2024-04-26T23:41:58.358332+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:18.639407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946898+0000","last_change":"2024-04-26T23:41:57.154218+0000","last_active":"2024-04-26T23:41:58.946898+0000","last_peered":"2024-04-26T23:41:58.946898+0000","last_clean":"2024-04-26T23:41:58.946898+0000","last_became_active":"2024-04-26T23:41:57.153942+0000","last_became_peered":"2024-04-26T23:41:57.153942+0000","last_unstale":"2024-04-26T23:41:58.946898+0000","last_undegraded":"2024-04-26T23:41:58.946898+0000","last_fullsized":"2024-04-26T23:41:58.946898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:14.790895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358783+0000","last_change":"2024-04-26T23:41:56.153668+0000","last_active":"2024-04-26T23:41:58.358783+0000","last_peered":"2024-04-26T23:41:58.358783+0000","last_clean":"2024-04-26T23:41:58.358783+0000","last_became_active":"2024-04-26T23:41:56.153241+0000","last_became_peered":"2024-04-26T23:41:56.153241+0000","last_unstale":"2024-04-26T23:41:58.358783+0000","last_undegraded":"2024-04-26T23:41:58.358783+0000","last_fullsized":"2024-04-26T23:41:58.358783+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:23.020103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163175+0000","last_change":"2024-04-26T23:41:57.168021+0000","last_active":"2024-04-26T23:41:58.163175+0000","last_peered":"2024-04-26T23:41:58.163175+0000","last_clean":"2024-04-26T23:41:58.163175+0000","last_became_active":"2024-04-26T23:41:57.167449+0000","last_became_peered":"2024-04-26T23:41:57.167449+0000","last_unstale":"2024-04-26T23:41:58.163175+0000","last_undegraded":"2024-04-26T23:41:58.163175+0000","last_fullsized":"2024-04-26T23:41:58.163175+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:52.961130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946892+0000","last_change":"2024-04-26T23:41:56.153251+0000","last_active":"2024-04-26T23:41:58.946892+0000","last_peered":"2024-04-26T23:41:58.946892+0000","last_clean":"2024-04-26T23:41:58.946892+0000","last_became_active":"2024-04-26T23:41:56.152597+0000","last_became_peered":"2024-04-26T23:41:56.152597+0000","last_unstale":"2024-04-26T23:41:58.946892+0000","last_undegraded":"2024-04-26T23:41:58.946892+0000","last_fullsized":"2024-04-26T23:41:58.946892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:49.353985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358252+0000","last_change":"2024-04-26T23:41:57.171575+0000","last_active":"2024-04-26T23:41:58.358252+0000","last_peered":"2024-04-26T23:41:58.358252+0000","last_clean":"2024-04-26T23:41:58.358252+0000","last_became_active":"2024-04-26T23:41:57.171098+0000","last_became_peered":"2024-04-26T23:41:57.171098+0000","last_unstale":"2024-04-26T23:41:58.358252+0000","last_undegraded":"2024-04-26T23:41:58.358252+0000","last_fullsized":"2024-04-26T23:41:58.358252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.483261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.968377+0000","last_change":"2024-04-26T23:41:56.146939+0000","last_active":"2024-04-26T23:41:58.968377+0000","last_peered":"2024-04-26T23:41:58.968377+0000","last_clean":"2024-04-26T23:41:58.968377+0000","last_became_active":"2024-04-26T23:41:56.146634+0000","last_became_peered":"2024-04-26T23:41:56.146634+0000","last_unstale":"2024-04-26T23:41:58.968377+0000","last_undegraded":"2024-04-26T23:41:58.968377+0000","last_fullsized":"2024-04-26T23:41:58.968377+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:51.945654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158027+0000","last_change":"2024-04-26T23:41:57.162795+0000","last_active":"2024-04-26T23:41:58.158027+0000","last_peered":"2024-04-26T23:41:58.158027+0000","last_clean":"2024-04-26T23:41:58.158027+0000","last_became_active":"2024-04-26T23:41:57.159561+0000","last_became_peered":"2024-04-26T23:41:57.159561+0000","last_unstale":"2024-04-26T23:41:58.158027+0000","last_undegraded":"2024-04-26T23:41:58.158027+0000","last_fullsized":"2024-04-26T23:41:58.158027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:44.401901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.156861+0000","last_change":"2024-04-26T23:41:56.138134+0000","last_active":"2024-04-26T23:41:58.156861+0000","last_peered":"2024-04-26T23:41:58.156861+0000","last_clean":"2024-04-26T23:41:58.156861+0000","last_became_active":"2024-04-26T23:41:56.137688+0000","last_became_peered":"2024-04-26T23:41:56.137688+0000","last_unstale":"2024-04-26T23:41:58.156861+0000","last_undegraded":"2024-04-26T23:41:58.156861+0000","last_fullsized":"2024-04-26T23:41:58.156861+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:47.245558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.947551+0000","last_change":"2024-04-26T23:41:57.157647+0000","last_active":"2024-04-26T23:41:58.947551+0000","last_peered":"2024-04-26T23:41:58.947551+0000","last_clean":"2024-04-26T23:41:58.947551+0000","last_became_active":"2024-04-26T23:41:57.157312+0000","last_became_peered":"2024-04-26T23:41:57.157312+0000","last_unstale":"2024-04-26T23:41:58.947551+0000","last_undegraded":"2024-04-26T23:41:58.947551+0000","last_fullsized":"2024-04-26T23:41:58.947551+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:29.410959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.967970+0000","last_change":"2024-04-26T23:41:56.150127+0000","last_active":"2024-04-26T23:41:58.967970+0000","last_peered":"2024-04-26T23:41:58.967970+0000","last_clean":"2024-04-26T23:41:58.967970+0000","last_became_active":"2024-04-26T23:41:56.149765+0000","last_became_peered":"2024-04-26T23:41:56.149765+0000","last_unstale":"2024-04-26T23:41:58.967970+0000","last_undegraded":"2024-04-26T23:41:58.967970+0000","last_fullsized":"2024-04-26T23:41:58.967970+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:12.254144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.162426+0000","last_change":"2024-04-26T23:41:57.158964+0000","last_active":"2024-04-26T23:41:58.162426+0000","last_peered":"2024-04-26T23:41:58.162426+0000","last_clean":"2024-04-26T23:41:58.162426+0000","last_became_active":"2024-04-26T23:41:57.158279+0000","last_became_peered":"2024-04-26T23:41:57.158279+0000","last_unstale":"2024-04-26T23:41:58.162426+0000","last_undegraded":"2024-04-26T23:41:58.162426+0000","last_fullsized":"2024-04-26T23:41:58.162426+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:29.410959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.358493+0000","last_change":"2024-04-26T23:41:56.159937+0000","last_active":"2024-04-26T23:41:58.358493+0000","last_peered":"2024-04-26T23:41:58.358493+0000","last_clean":"2024-04-26T23:41:58.358493+0000","last_became_active":"2024-04-26T23:41:56.159506+0000","last_became_peered":"2024-04-26T23:41:56.159506+0000","last_unstale":"2024-04-26T23:41:58.358493+0000","last_undegraded":"2024-04-26T23:41:58.358493+0000","last_fullsized":"2024-04-26T23:41:58.358493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:12.112512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.946817+0000","last_change":"2024-04-26T23:41:57.151539+0000","last_active":"2024-04-26T23:41:58.946817+0000","last_peered":"2024-04-26T23:41:58.946817+0000","last_clean":"2024-04-26T23:41:58.946817+0000","last_became_active":"2024-04-26T23:41:57.151240+0000","last_became_peered":"2024-04-26T23:41:57.151240+0000","last_unstale":"2024-04-26T23:41:58.946817+0000","last_undegraded":"2024-04-26T23:41:58.946817+0000","last_fullsized":"2024-04-26T23:41:58.946817+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:05.613366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.166207+0000","last_change":"2024-04-26T23:41:56.150011+0000","last_active":"2024-04-26T23:41:58.166207+0000","last_peered":"2024-04-26T23:41:58.166207+0000","last_clean":"2024-04-26T23:41:58.166207+0000","last_became_active":"2024-04-26T23:41:56.149715+0000","last_became_peered":"2024-04-26T23:41:56.149715+0000","last_unstale":"2024-04-26T23:41:58.166207+0000","last_undegraded":"2024-04-26T23:41:58.166207+0000","last_fullsized":"2024-04-26T23:41:58.166207+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:02.998742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.969297+0000","last_change":"2024-04-26T23:41:57.164150+0000","last_active":"2024-04-26T23:41:58.969297+0000","last_peered":"2024-04-26T23:41:58.969297+0000","last_clean":"2024-04-26T23:41:58.969297+0000","last_became_active":"2024-04-26T23:41:57.163831+0000","last_became_peered":"2024-04-26T23:41:57.163831+0000","last_unstale":"2024-04-26T23:41:58.969297+0000","last_undegraded":"2024-04-26T23:41:58.969297+0000","last_fullsized":"2024-04-26T23:41:58.969297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:36.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157542+0000","last_change":"2024-04-26T23:41:56.158249+0000","last_active":"2024-04-26T23:41:58.157542+0000","last_peered":"2024-04-26T23:41:58.157542+0000","last_clean":"2024-04-26T23:41:58.157542+0000","last_became_active":"2024-04-26T23:41:56.157926+0000","last_became_peered":"2024-04-26T23:41:56.157926+0000","last_unstale":"2024-04-26T23:41:58.157542+0000","last_undegraded":"2024-04-26T23:41:58.157542+0000","last_fullsized":"2024-04-26T23:41:58.157542+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:47.165080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157571+0000","last_change":"2024-04-26T23:41:57.154732+0000","last_active":"2024-04-26T23:41:58.157571+0000","last_peered":"2024-04-26T23:41:58.157571+0000","last_clean":"2024-04-26T23:41:58.157571+0000","last_became_active":"2024-04-26T23:41:57.154434+0000","last_became_peered":"2024-04-26T23:41:57.154434+0000","last_unstale":"2024-04-26T23:41:58.157571+0000","last_undegraded":"2024-04-26T23:41:58.157571+0000","last_fullsized":"2024-04-26T23:41:58.157571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:23.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.969455+0000","last_change":"2024-04-26T23:41:56.149243+0000","last_active":"2024-04-26T23:41:58.969455+0000","last_peered":"2024-04-26T23:41:58.969455+0000","last_clean":"2024-04-26T23:41:58.969455+0000","last_became_active":"2024-04-26T23:41:56.148978+0000","last_became_peered":"2024-04-26T23:41:56.148978+0000","last_unstale":"2024-04-26T23:41:58.969455+0000","last_undegraded":"2024-04-26T23:41:58.969455+0000","last_fullsized":"2024-04-26T23:41:58.969455+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:21.236671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163885+0000","last_change":"2024-04-26T23:41:57.165309+0000","last_active":"2024-04-26T23:41:58.163885+0000","last_peered":"2024-04-26T23:41:58.163885+0000","last_clean":"2024-04-26T23:41:58.163885+0000","last_became_active":"2024-04-26T23:41:57.165008+0000","last_became_peered":"2024-04-26T23:41:57.165008+0000","last_unstale":"2024-04-26T23:41:58.163885+0000","last_undegraded":"2024-04-26T23:41:58.163885+0000","last_fullsized":"2024-04-26T23:41:58.163885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.497942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.968979+0000","last_change":"2024-04-26T23:41:56.144988+0000","last_active":"2024-04-26T23:41:58.968979+0000","last_peered":"2024-04-26T23:41:58.968979+0000","last_clean":"2024-04-26T23:41:58.968979+0000","last_became_active":"2024-04-26T23:41:56.144711+0000","last_became_peered":"2024-04-26T23:41:56.144711+0000","last_unstale":"2024-04-26T23:41:58.968979+0000","last_undegraded":"2024-04-26T23:41:58.968979+0000","last_fullsized":"2024-04-26T23:41:58.968979+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:49.167539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.157850+0000","last_change":"2024-04-26T23:41:57.159851+0000","last_active":"2024-04-26T23:41:58.157850+0000","last_peered":"2024-04-26T23:41:58.157850+0000","last_clean":"2024-04-26T23:41:58.157850+0000","last_became_active":"2024-04-26T23:41:57.159169+0000","last_became_peered":"2024-04-26T23:41:57.159169+0000","last_unstale":"2024-04-26T23:41:58.157850+0000","last_undegraded":"2024-04-26T23:41:58.157850+0000","last_fullsized":"2024-04-26T23:41:58.157850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:14.048397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.966922+0000","last_change":"2024-04-26T23:41:56.154732+0000","last_active":"2024-04-26T23:41:58.966922+0000","last_peered":"2024-04-26T23:41:58.966922+0000","last_clean":"2024-04-26T23:41:58.966922+0000","last_became_active":"2024-04-26T23:41:56.152800+0000","last_became_peered":"2024-04-26T23:41:56.152800+0000","last_unstale":"2024-04-26T23:41:58.966922+0000","last_undegraded":"2024-04-26T23:41:58.966922+0000","last_fullsized":"2024-04-26T23:41:58.966922+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:40.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.970827+0000","last_change":"2024-04-26T23:41:57.153234+0000","last_active":"2024-04-26T23:41:58.970827+0000","last_peered":"2024-04-26T23:41:58.970827+0000","last_clean":"2024-04-26T23:41:58.970827+0000","last_became_active":"2024-04-26T23:41:57.149321+0000","last_became_peered":"2024-04-26T23:41:57.149321+0000","last_unstale":"2024-04-26T23:41:58.970827+0000","last_undegraded":"2024-04-26T23:41:58.970827+0000","last_fullsized":"2024-04-26T23:41:58.970827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:19.202077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.162577+0000","last_change":"2024-04-26T23:41:56.150661+0000","last_active":"2024-04-26T23:41:58.162577+0000","last_peered":"2024-04-26T23:41:58.162577+0000","last_clean":"2024-04-26T23:41:58.162577+0000","last_became_active":"2024-04-26T23:41:56.147698+0000","last_became_peered":"2024-04-26T23:41:56.147698+0000","last_unstale":"2024-04-26T23:41:58.162577+0000","last_undegraded":"2024-04-26T23:41:58.162577+0000","last_fullsized":"2024-04-26T23:41:58.162577+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:02.998742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.357879+0000","last_change":"2024-04-26T23:41:57.161868+0000","last_active":"2024-04-26T23:41:58.357879+0000","last_peered":"2024-04-26T23:41:58.357879+0000","last_clean":"2024-04-26T23:41:58.357879+0000","last_became_active":"2024-04-26T23:41:57.161548+0000","last_became_peered":"2024-04-26T23:41:57.161548+0000","last_unstale":"2024-04-26T23:41:58.357879+0000","last_undegraded":"2024-04-26T23:41:58.357879+0000","last_fullsized":"2024-04-26T23:41:58.357879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:56.104712+0000","last_clean_scrub_stamp":"2024-04-26T23:41:56.104712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:10.417240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.158143+0000","last_change":"2024-04-26T23:41:56.142083+0000","last_active":"2024-04-26T23:41:58.158143+0000","last_peered":"2024-04-26T23:41:58.158143+0000","last_clean":"2024-04-26T23:41:58.158143+0000","last_became_active":"2024-04-26T23:41:56.141765+0000","last_became_peered":"2024-04-26T23:41:56.141765+0000","last_unstale":"2024-04-26T23:41:58.158143+0000","last_undegraded":"2024-04-26T23:41:58.158143+0000","last_fullsized":"2024-04-26T23:41:58.158143+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:41:55.089236+0000","last_clean_scrub_stamp":"2024-04-26T23:41:55.089236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:47.863578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:41:58.163931+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:41:58.163931+0000","last_peered":"2024-04-26T23:41:58.163931+0000","last_clean":"2024-04-26T23:41:58.163931+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:41:58.163931+0000","last_undegraded":"2024-04-26T23:41:58.163931+0000","last_fullsized":"2024-04-26T23:41:58.163931+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1245184,"data_stored":878666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":651264,"data_stored":288826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35976,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335864,"statfs":{"total":96636764160,"available":96599924736,"internally_reserved":0,"allocated":643072,"data_stored":288174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":647168,"data_stored":288756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":659456,"data_stored":289808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35984,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335856,"statfs":{"total":96636764160,"available":96599916544,"internally_reserved":0,"allocated":655360,"data_stored":289786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":651264,"data_stored":288334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40716,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331124,"statfs":{"total":96636764160,"available":96595070976,"internally_reserved":0,"allocated":1245184,"data_stored":878598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:42:02.169 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:42:02.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:42:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.426+0000 7f44ffe00640 1 Processor -- start 2024-04-26T23:42:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.426+0000 7f44ffe00640 1 -- start start 2024-04-26T23:42:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.428+0000 7f44ffe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f45001640d0 0x7f45001644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.428+0000 7f44ffe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45000ce7d0 0x7f45000cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.428+0000 7f44ffe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45001649f0 con 0x7f4500073af0 2024-04-26T23:42:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.428+0000 7f44ffe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f45000cb4f0 con 0x7f45000ce7d0 2024-04-26T23:42:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.428+0000 7f44ffe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4500071e80 con 0x7f45001640d0 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45000ce7d0 0x7f45000cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45000ce7d0 0x7f45000cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40434/0 (socket says 172.21.15.46:40434) 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 -- 172.21.15.46:0/3735847862 learned_addr learned my addr 172.21.15.46:0/3735847862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44fea00640 1 --2- 172.21.15.46:0/3735847862 >> v2:172.21.15.142:3300/0 conn(0x7f45001640d0 0x7f45001644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 -- 172.21.15.46:0/3735847862 >> v2:172.21.15.142:3300/0 conn(0x7f45001640d0 msgr2=0x7f45001644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 --2- 172.21.15.46:0/3735847862 >> v2:172.21.15.142:3300/0 conn(0x7f45001640d0 0x7f45001644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 -- 172.21.15.46:0/3735847862 >> v1:172.21.15.142:6789/0 conn(0x7f4500073af0 legacy=0x7f45000cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.429+0000 7f44ff400640 1 -- 172.21.15.46:0/3735847862 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4500072160 con 0x7f45000ce7d0 2024-04-26T23:42:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.430+0000 7f44ff400640 1 --2- 172.21.15.46:0/3735847862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45000ce7d0 0x7f45000cec20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f45000cb1e0 tx=0x7f44f802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99120d4a7c3ec3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.431+0000 7f44fd600640 1 -- 172.21.15.46:0/3735847862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8036070 con 0x7f45000ce7d0 2024-04-26T23:42:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.431+0000 7f44fd600640 1 -- 172.21.15.46:0/3735847862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f8031600 con 0x7f45000ce7d0 2024-04-26T23:42:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.431+0000 7f44fd600640 1 -- 172.21.15.46:0/3735847862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f80319d0 con 0x7f45000ce7d0 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.431+0000 7f44ffe00640 1 -- 172.21.15.46:0/3735847862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45000ce7d0 msgr2=0x7f45000cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.431+0000 7f44ffe00640 1 --2- 172.21.15.46:0/3735847862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45000ce7d0 0x7f45000cec20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f45000cb1e0 tx=0x7f44f802a6b0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 -- 172.21.15.46:0/3735847862 shutdown_connections 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 --2- 172.21.15.46:0/3735847862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45000ce7d0 0x7f45000cec20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 --2- 172.21.15.46:0/3735847862 >> v2:172.21.15.142:3300/0 conn(0x7f45001640d0 0x7f45001644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 -- 172.21.15.46:0/3735847862 >> 172.21.15.46:0/3735847862 conn(0x7f45000b27b0 msgr2=0x7f45000b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 -- 172.21.15.46:0/3735847862 shutdown_connections 2024-04-26T23:42:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.432+0000 7f44ffe00640 1 -- 172.21.15.46:0/3735847862 wait complete. 2024-04-26T23:42:02.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.434+0000 7f44ffe00640 1 Processor -- start 2024-04-26T23:42:02.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.434+0000 7f44ffe00640 1 -- start start 2024-04-26T23:42:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4500073af0 0x7f45001f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45000ce7d0 0x7f45001f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45000cc090 con 0x7f45001640d0 2024-04-26T23:42:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4500073f70 con 0x7f45000ce7d0 2024-04-26T23:42:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ffe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45000cc210 con 0x7f4500073af0 2024-04-26T23:42:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40436/0 (socket says 172.21.15.46:40436) 2024-04-26T23:42:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44ff400640 1 -- 172.21.15.46:0/1894258417 learned_addr learned my addr 172.21.15.46:0/1894258417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.436+0000 7f44fe000640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45000ce7d0 0x7f45001f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44fea00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4500073af0 0x7f45001f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 -- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4500073af0 msgr2=0x7f45001f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4500073af0 0x7f45001f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 -- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45000ce7d0 msgr2=0x7f45001f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45000ce7d0 0x7f45001f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 -- 172.21.15.46:0/1894258417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001fa020 con 0x7f45001640d0 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.437+0000 7f44ff400640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f44f8034040 tx=0x7f44f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.438+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8036050 con 0x7f45001640d0 2024-04-26T23:42:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.438+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f8042070 con 0x7f45001640d0 2024-04-26T23:42:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.438+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8009510 con 0x7f45001640d0 2024-04-26T23:42:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.438+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45000c46c0 con 0x7f45001640d0 2024-04-26T23:42:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.438+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45000c4f20 con 0x7f45001640d0 2024-04-26T23:42:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.439+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45000693b0 con 0x7f45001640d0 2024-04-26T23:42:02.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.440+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44f8007bd0 con 0x7f45001640d0 2024-04-26T23:42:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.442+0000 7f44f6a00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 0x7f44e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.442+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f44f80acbf0 con 0x7f45001640d0 2024-04-26T23:42:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.442+0000 7f44fea00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 0x7f44e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.444+0000 7f44fea00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 0x7f44e006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f44f0006fd0 tx=0x7f44f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.447+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45001fa020 con 0x7f45001640d0 2024-04-26T23:42:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.727+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f45000cbd00 con 0x7f45001640d0 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.732+0000 7f44f6a00640 1 -- 172.21.15.46:0/1894258417 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f44f8045360 con 0x7f45001640d0 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:42:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:42:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:42:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 msgr2=0x7f44e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 0x7f44e006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f44f0006fd0 tx=0x7f44f0008040 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 msgr2=0x7f45001fa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f44f8034040 tx=0x7f44f8004a40 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 shutdown_connections 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44e006c8f0 0x7f44e006edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45001640d0 0x7f45001fa220 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45000ce7d0 0x7f45001f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 --2- 172.21.15.46:0/1894258417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4500073af0 0x7f45001f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 >> 172.21.15.46:0/1894258417 conn(0x7f45000b27b0 msgr2=0x7f45000b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 shutdown_connections 2024-04-26T23:42:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.735+0000 7f44ffe00640 1 -- 172.21.15.46:0/1894258417 wait complete. 2024-04-26T23:42:02.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.963+0000 7f111fe00640 1 Processor -- start 2024-04-26T23:42:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.963+0000 7f111fe00640 1 -- start start 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11201466b0 0x7f112006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1120160910 0x7f1120160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1120073ad0 con 0x7f11200720e0 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f11201612a0 con 0x7f1120160910 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1120146a90 con 0x7f11201466b0 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1120160910 0x7f1120160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.965+0000 7f111f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1120160910 0x7f1120160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40438/0 (socket says 172.21.15.46:40438) 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111f400640 1 -- 172.21.15.46:0/3691393671 learned_addr learned my addr 172.21.15.46:0/3691393671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111ea00640 1 --2- 172.21.15.46:0/3691393671 >> v2:172.21.15.142:3300/0 conn(0x7f11201466b0 0x7f112006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111f400640 1 -- 172.21.15.46:0/3691393671 >> v2:172.21.15.142:3300/0 conn(0x7f11201466b0 msgr2=0x7f112006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111f400640 1 --2- 172.21.15.46:0/3691393671 >> v2:172.21.15.142:3300/0 conn(0x7f11201466b0 0x7f112006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111f400640 1 -- 172.21.15.46:0/3691393671 >> v1:172.21.15.142:6789/0 conn(0x7f11200720e0 legacy=0x7f11200cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.966+0000 7f111f400640 1 -- 172.21.15.46:0/3691393671 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1120073d70 con 0x7f1120160910 2024-04-26T23:42:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.967+0000 7f111f400640 1 --2- 172.21.15.46:0/3691393671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1120160910 0x7f1120160d60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de2233bfb6695b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.968+0000 7f111e000640 1 -- 172.21.15.46:0/3691393671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118036070 con 0x7f1120160910 2024-04-26T23:42:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.968+0000 7f111e000640 1 -- 172.21.15.46:0/3691393671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118031600 con 0x7f1120160910 2024-04-26T23:42:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.968+0000 7f111e000640 1 -- 172.21.15.46:0/3691393671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180319d0 con 0x7f1120160910 2024-04-26T23:42:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.968+0000 7f111fe00640 1 -- 172.21.15.46:0/3691393671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1120160910 msgr2=0x7f1120160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.968+0000 7f111fe00640 1 --2- 172.21.15.46:0/3691393671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1120160910 0x7f1120160d60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a710 comp rx=0 tx=0).stop 2024-04-26T23:42:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 -- 172.21.15.46:0/3691393671 shutdown_connections 2024-04-26T23:42:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 --2- 172.21.15.46:0/3691393671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1120160910 0x7f1120160d60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 --2- 172.21.15.46:0/3691393671 >> v2:172.21.15.142:3300/0 conn(0x7f11201466b0 0x7f112006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 -- 172.21.15.46:0/3691393671 >> 172.21.15.46:0/3691393671 conn(0x7f11200b26d0 msgr2=0x7f11200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 -- 172.21.15.46:0/3691393671 shutdown_connections 2024-04-26T23:42:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.969+0000 7f111fe00640 1 -- 172.21.15.46:0/3691393671 wait complete. 2024-04-26T23:42:02.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.971+0000 7f111fe00640 1 Processor -- start 2024-04-26T23:42:02.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.971+0000 7f111fe00640 1 -- start start 2024-04-26T23:42:02.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11200720e0 0x7f11200b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 0x7f11200b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11200cb470 con 0x7f11201466b0 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11200cb190 con 0x7f11200720e0 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.972+0000 7f111fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f112006e3c0 con 0x7f1120160910 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40442/0 (socket says 172.21.15.46:40442) 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 -- 172.21.15.46:0/1842209222 learned_addr learned my addr 172.21.15.46:0/1842209222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f111ea00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11200720e0 0x7f11200b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f111f400640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 0x7f11200b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 -- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 msgr2=0x7f11200b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 0x7f11200b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 -- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11200720e0 msgr2=0x7f11200b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11200720e0 0x7f11200b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f1117e00640 1 -- 172.21.15.46:0/1842209222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112015ee00 con 0x7f11201466b0 2024-04-26T23:42:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.973+0000 7f111f400640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 0x7f11200b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f1117e00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f110800b3f0 tx=0x7f110800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108015020 con 0x7f11201466b0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108011830 con 0x7f11201466b0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112015f0e0 con 0x7f11201466b0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108011c20 con 0x7f11201466b0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.974+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11201e6ad0 con 0x7f11201466b0 2024-04-26T23:42:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.976+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11200693b0 con 0x7f11201466b0 2024-04-26T23:42:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.976+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1108018020 con 0x7f11201466b0 2024-04-26T23:42:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.977+0000 7f1116a00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 0x7f10fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.977+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f110808b930 con 0x7f11201466b0 2024-04-26T23:42:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.978+0000 7f111ea00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 0x7f10fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:02.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.979+0000 7f111ea00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 0x7f10fc06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1110006fd0 tx=0x7f1110008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:02.982+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f110801f030 con 0x7f11201466b0 2024-04-26T23:42:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.229+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11200c7180 con 0x7f11201466b0 2024-04-26T23:42:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.230+0000 7f1116a00640 1 -- 172.21.15.46:0/1842209222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f11080172c0 con 0x7f11201466b0 2024-04-26T23:42:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:03.234 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 msgr2=0x7f10fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 0x7f10fc06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1110006fd0 tx=0x7f1110008040 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 msgr2=0x7f1120158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f110800b3f0 tx=0x7f110800b8b0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 shutdown_connections 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10fc06c8f0 0x7f10fc06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1120160910 0x7f11200b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11201466b0 0x7f1120158fa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 --2- 172.21.15.46:0/1842209222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11200720e0 0x7f11200b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 >> 172.21.15.46:0/1842209222 conn(0x7f11200b26d0 msgr2=0x7f11200cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 shutdown_connections 2024-04-26T23:42:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.233+0000 7f111fe00640 1 -- 172.21.15.46:0/1842209222 wait complete. 2024-04-26T23:42:03.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:41:56.611187+0000', 'modified': '2024-04-26T23:41:59.681274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5793}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5793': {'gid': 5793, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6834/970201181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 970201181}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 970201181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5793, 'qdb_cluster': [5793]} 2024-04-26T23:42:03.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:42:03.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:42:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.878+0000 7f2ead000640 1 Processor -- start 2024-04-26T23:42:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.878+0000 7f2ead000640 1 -- start start 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ead000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ea81454a0 0x7f2ea80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ead000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ea8073970 0x7f2ea80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ead000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea8072bb0 con 0x7f2ea80c6a50 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ead000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea8072e90 con 0x7f2ea8073970 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ead000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea8073170 con 0x7f2ea81454a0 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ea6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ea8073970 0x7f2ea80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ea6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ea8073970 0x7f2ea80cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40452/0 (socket says 172.21.15.46:40452) 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ea6a00640 1 -- 172.21.15.46:0/197531318 learned_addr learned my addr 172.21.15.46:0/197531318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.880+0000 7f2ea7400640 1 --2- 172.21.15.46:0/197531318 >> v2:172.21.15.142:3300/0 conn(0x7f2ea81454a0 0x7f2ea80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.881+0000 7f2ea6a00640 1 -- 172.21.15.46:0/197531318 >> v2:172.21.15.142:3300/0 conn(0x7f2ea81454a0 msgr2=0x7f2ea80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.881+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/197531318 >> v2:172.21.15.142:3300/0 conn(0x7f2ea81454a0 0x7f2ea80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.881+0000 7f2ea6a00640 1 -- 172.21.15.46:0/197531318 >> v1:172.21.15.142:6789/0 conn(0x7f2ea80c6a50 legacy=0x7f2ea80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.881+0000 7f2ea6a00640 1 -- 172.21.15.46:0/197531318 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea8073450 con 0x7f2ea8073970 2024-04-26T23:42:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.882+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/197531318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea8073970 0x7f2ea80cdc30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80d1ba0 tx=0x7f2e9402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453bea7eac51d1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.882+0000 7f2ea6000640 1 -- 172.21.15.46:0/197531318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94037070 con 0x7f2ea8073970 2024-04-26T23:42:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.882+0000 7f2ea6000640 1 -- 172.21.15.46:0/197531318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94032440 con 0x7f2ea8073970 2024-04-26T23:42:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.882+0000 7f2ea6000640 1 -- 172.21.15.46:0/197531318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94032810 con 0x7f2ea8073970 2024-04-26T23:42:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 -- 172.21.15.46:0/197531318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea8073970 msgr2=0x7f2ea80cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 --2- 172.21.15.46:0/197531318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea8073970 0x7f2ea80cdc30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80d1ba0 tx=0x7f2e9402a230 comp rx=0 tx=0).stop 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 -- 172.21.15.46:0/197531318 shutdown_connections 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 --2- 172.21.15.46:0/197531318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea8073970 0x7f2ea80cdc30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 --2- 172.21.15.46:0/197531318 >> v2:172.21.15.142:3300/0 conn(0x7f2ea81454a0 0x7f2ea80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 -- 172.21.15.46:0/197531318 >> 172.21.15.46:0/197531318 conn(0x7f2ea80b26d0 msgr2=0x7f2ea80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 -- 172.21.15.46:0/197531318 shutdown_connections 2024-04-26T23:42:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.883+0000 7f2ead000640 1 -- 172.21.15.46:0/197531318 wait complete. 2024-04-26T23:42:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.885+0000 7f2ead000640 1 Processor -- start 2024-04-26T23:42:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.885+0000 7f2ead000640 1 -- start start 2024-04-26T23:42:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea8073970 0x7f2ea815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea80c6a50 0x7f2ea815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8073e00 con 0x7f2ea81454a0 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8072880 con 0x7f2ea80c6a50 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ead000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea8145880 con 0x7f2ea8073970 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40458/0 (socket says 172.21.15.46:40458) 2024-04-26T23:42:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.886+0000 7f2ea7e00640 1 -- 172.21.15.46:0/865377833 learned_addr learned my addr 172.21.15.46:0/865377833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea80c6a50 0x7f2ea815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7400640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea8073970 0x7f2ea815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 -- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea8073970 msgr2=0x7f2ea815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea8073970 0x7f2ea815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 -- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea80c6a50 msgr2=0x7f2ea815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea80c6a50 0x7f2ea815db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 -- 172.21.15.46:0/865377833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea815bb60 con 0x7f2ea81454a0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c7b0 tx=0x7f2ea000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2e9f400640 1 -- 172.21.15.46:0/865377833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0004b60 con 0x7f2ea81454a0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2e9f400640 1 -- 172.21.15.46:0/865377833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea0004d00 con 0x7f2ea81454a0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea80b3850 con 0x7f2ea81454a0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.887+0000 7f2e9f400640 1 -- 172.21.15.46:0/865377833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000aa90 con 0x7f2ea81454a0 2024-04-26T23:42:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.888+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea80b3e10 con 0x7f2ea81454a0 2024-04-26T23:42:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.890+0000 7f2e9f400640 1 -- 172.21.15.46:0/865377833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ea000ac30 con 0x7f2ea81454a0 2024-04-26T23:42:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.891+0000 7f2e9f400640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 0x7f2e8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.891+0000 7f2e9f400640 1 -- 172.21.15.46:0/865377833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2ea008bfe0 con 0x7f2ea81454a0 2024-04-26T23:42:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.891+0000 7f2eb1424740 1 Processor -- start 2024-04-26T23:42:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.891+0000 7f2eb1424740 1 -- start start 2024-04-26T23:42:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2ea7400640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 0x7f2e8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b82e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578b824a600 con 0x5578b8322e80 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5578b81e7100 con 0x5578b8311270 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.892+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5578b82465c0 con 0x5578b82e67b0 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40460/0 (socket says 172.21.15.46:40460) 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 -- 172.21.15.46:0/1198056940 learned_addr learned my addr 172.21.15.46:0/1198056940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea7400640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 0x7f2e8806ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 -- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 msgr2=0x5578b82e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b82e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 -- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 msgr2=0x5578b83116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 -- 172.21.15.46:0/1198056940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b7f5c2d0 con 0x5578b8322e80 2024-04-26T23:42:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.893+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009960 tx=0x7f2e9402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ac71cda01920c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2e93e00640 1 -- 172.21.15.46:0/1198056940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94037040 con 0x5578b8322e80 2024-04-26T23:42:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2e93e00640 1 -- 172.21.15.46:0/1198056940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94042070 con 0x5578b8322e80 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2e93e00640 1 -- 172.21.15.46:0/1198056940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9403c860 con 0x5578b8322e80 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2eb1424740 1 -- 172.21.15.46:0/1198056940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 msgr2=0x5578b82e6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009960 tx=0x7f2e9402c000 comp rx=0 tx=0).stop 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2eb1424740 1 -- 172.21.15.46:0/1198056940 shutdown_connections 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.894+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b82e6b90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.895+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1198056940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b82e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.895+0000 7f2eb1424740 1 -- 172.21.15.46:0/1198056940 >> 172.21.15.46:0/1198056940 conn(0x5578b827ad10 msgr2=0x5578b832c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.895+0000 7f2eb1424740 1 -- 172.21.15.46:0/1198056940 shutdown_connections 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.895+0000 7f2eb1424740 1 -- 172.21.15.46:0/1198056940 wait complete. 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.895+0000 7f2eb1424740 1 -- start start 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.896+0000 7f2eb1424740 1 Processor -- start 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.896+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8325550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578b81d1f70 con 0x5578b82e67b0 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5578b830edc0 con 0x5578b8311270 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5578b83241f0 con 0x5578b8322e80 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2eb1424740 10 client.0 _ll_register_callbacks cb 0x5578b8240480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40464/0 (socket says 172.21.15.46:40464) 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 -- 172.21.15.46:0/1338019027 learned_addr learned my addr 172.21.15.46:0/1338019027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8325550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 -- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 msgr2=0x5578b8325550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8325550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 -- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 msgr2=0x5578b83404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7400640 1 -- 172.21.15.46:0/1338019027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b80266b0 con 0x5578b82e67b0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.897+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8325550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2ea7400640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e980095d0 tx=0x7f2e9800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752dadf63f3b2d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2e90c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98025060 con 0x5578b82e67b0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2e98015d50 con 0x5578b82e67b0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e94042070 con 0x5578b82e67b0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e9802b050 con 0x5578b82e67b0 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2eb1424740 10 client.5925 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.898+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5578b8323f30 con 0x5578b82e67b0 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94042070 con 0x5578b82e67b0 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f2e9802e020 con 0x5578b82e67b0 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 10 client.5925 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 10 client.5925 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 20 client.5925 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b833c6a0 0x5578b833ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 4 client.5925 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b8258210 con 0x5578b833c6a0 2024-04-26T23:42:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.899+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b833c6a0 0x5578b833ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.900+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b833c6a0 0x5578b833ca80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0bd41f92f7ae394 server_cookie=246afedcc75929e4 in_seq=0 out_seq=0 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.900+0000 7f2e90c00640 10 client.5925 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2e90c00640 1 -- 172.21.15.46:0/1338019027 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578b8258210 con 0x5578b833c6a0 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2e90c00640 10 client.5925 handle_command_reply: tid=0 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 client.5925 shutdown 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 msgr2=0x5578b82feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e980095d0 tx=0x7f2e9800ed20 comp rx=0 tx=0).stop 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 shutdown_connections 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8325550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b833c6a0 0x5578b833ca80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b83404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 --2- 172.21.15.46:0/1338019027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82feba0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 >> 172.21.15.46:0/1338019027 conn(0x5578b827ad10 msgr2=0x5578b830f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2e90c00640 0 client.5925 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 shutdown_connections 2024-04-26T23:42:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 1 -- 172.21.15.46:0/1338019027 wait complete. 2024-04-26T23:42:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.901+0000 7f2eb1424740 20 client.5925 trim_cache size 0 max 16384 2024-04-26T23:42:03.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.921+0000 7f2eb1424740 1 Processor -- start 2024-04-26T23:42:03.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 -- start start 2024-04-26T23:42:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b8350340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8340120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81fc9d0 con 0x5578b82e67b0 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea80b5940 con 0x5578b8311270 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.922+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea81fc480 con 0x5578b8322e80 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40466/0 (socket says 172.21.15.46:40466) 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 -- 172.21.15.46:0/952122612 learned_addr learned my addr 172.21.15.46:0/952122612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 -- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 msgr2=0x5578b8340120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8340120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 -- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 msgr2=0x5578b8350340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b8350340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.923+0000 7f2ea7400640 1 -- 172.21.15.46:0/952122612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b832d1a0 con 0x5578b82e67b0 2024-04-26T23:42:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2ea7400640 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2e98008010 tx=0x7f2e9801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57fd613a183479f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2e93400640 1 -- 172.21.15.46:0/952122612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800f040 con 0x5578b82e67b0 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2e93400640 1 -- 172.21.15.46:0/952122612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98015d50 con 0x5578b82e67b0 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2e93400640 1 -- 172.21.15.46:0/952122612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800e900 con 0x5578b82e67b0 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2eb1424740 1 -- 172.21.15.46:0/952122612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 msgr2=0x5578b82fdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.924+0000 7f2eb1424740 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2e98008010 tx=0x7f2e9801e000 comp rx=0 tx=0).stop 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 -- 172.21.15.46:0/952122612 shutdown_connections 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b8322e80 0x5578b8340120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b8350340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 --2- 172.21.15.46:0/952122612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b82e67b0 0x5578b82fdbe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 -- 172.21.15.46:0/952122612 >> 172.21.15.46:0/952122612 conn(0x5578b827ad10 msgr2=0x5578b8312f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 -- 172.21.15.46:0/952122612 shutdown_connections 2024-04-26T23:42:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 -- 172.21.15.46:0/952122612 wait complete. 2024-04-26T23:42:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.925+0000 7f2eb1424740 1 -- start start 2024-04-26T23:42:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.926+0000 7f2eb1424740 1 Processor -- start 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.926+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b839d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b832ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578b8311700 con 0x5578b8322e80 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5578b8312df0 con 0x5578b8311270 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5578b8323440 con 0x5578b82e67b0 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2eb1424740 10 client.0 _ll_register_callbacks cb 0x7f2e98009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40472/0 (socket says 172.21.15.46:40472) 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 -- 172.21.15.46:0/2692582196 learned_addr learned my addr 172.21.15.46:0/2692582196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea6a00640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b832ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 -- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 msgr2=0x5578b839d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b839d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 -- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 msgr2=0x5578b832ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b832ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.927+0000 7f2ea7e00640 1 -- 172.21.15.46:0/2692582196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b839fed0 con 0x5578b8322e80 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2ea7e00640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0059a70 tx=0x7f2ea0059ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5033d57ab53abdc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2e90c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0087070 con 0x5578b8322e80 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2ea0056150 con 0x5578b8322e80 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e98015d50 con 0x5578b8322e80 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ea0079120 con 0x5578b8322e80 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2eb1424740 10 client.5931 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5578b830fd70 con 0x5578b8322e80 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.928+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98015d50 con 0x5578b8322e80 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f2ea0062230 con 0x5578b8322e80 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 10 client.5931 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 10 client.5931 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 20 client.5931 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b8314e90 0x5578b833a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 4 client.5931 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "status"}) -- 0x5578b8252080 con 0x5578b8314e90 2024-04-26T23:42:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.929+0000 7f2ea7400640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b8314e90 0x5578b833a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:03.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2ea7400640 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b8314e90 0x5578b833a970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0aae663e56a28f server_cookie=ddc9dd06e877ed03 in_seq=0 out_seq=0 2024-04-26T23:42:03.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2e90c00640 10 client.5931 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2e90c00640 1 -- 172.21.15.46:0/2692582196 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5578b8252080 con 0x5578b8314e90 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2e90c00640 10 client.5931 handle_command_reply: tid=0 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 client.5931 shutdown 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 msgr2=0x5578b832bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0059a70 tx=0x7f2ea0059ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 shutdown_connections 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578b8322e80 0x5578b832bfc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5578b8314e90 0x5578b833a970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578b8311270 0x5578b832ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 --2- 172.21.15.46:0/2692582196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5578b82e67b0 0x5578b839d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 >> 172.21.15.46:0/2692582196 conn(0x5578b827ad10 msgr2=0x5578b82fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2e90c00640 0 client.5931 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 shutdown_connections 2024-04-26T23:42:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.930+0000 7f2eb1424740 1 -- 172.21.15.46:0/2692582196 wait complete. 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2eb1424740 20 client.5931 trim_cache size 0 max 16384 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5793, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.8056097900000001, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 28, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 54, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 54, 2024-04-26T23:42:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.188461708 2024-04-26T23:42:03.938 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 msgr2=0x7f2e8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 0x7f2e8806ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2024-04-26T23:42:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 msgr2=0x7f2ea815e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c7b0 tx=0x7f2ea000cc70 comp rx=0 tx=0).stop 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 shutdown_connections 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e8806c920 0x7f2e8806ede0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea81454a0 0x7f2ea815e090 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea80c6a50 0x7f2ea815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 --2- 172.21.15.46:0/865377833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea8073970 0x7f2ea815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 >> 172.21.15.46:0/865377833 conn(0x7f2ea80b26d0 msgr2=0x7f2ea80b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 shutdown_connections 2024-04-26T23:42:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:03.931+0000 7f2ead000640 1 -- 172.21.15.46:0/865377833 wait complete. 2024-04-26T23:42:03.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:42:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.143+0000 7f683f200640 1 Processor -- start 2024-04-26T23:42:04.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.143+0000 7f683f200640 1 -- start start 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f684015f5b0 0x7f68400b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 0x7f684006d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684006e860 con 0x7f684006dd40 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f68400d2280 con 0x7f684015f5b0 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f684006e580 con 0x7f68400b3c60 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f684015f5b0 0x7f68400b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f684015f5b0 0x7f68400b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40476/0 (socket says 172.21.15.46:40476) 2024-04-26T23:42:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f683de00640 1 -- 172.21.15.46:0/613727185 learned_addr learned my addr 172.21.15.46:0/613727185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.145+0000 7f6835400640 1 --2- 172.21.15.46:0/613727185 >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 0x7f684006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.146+0000 7f683de00640 1 -- 172.21.15.46:0/613727185 >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 msgr2=0x7f684006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.146+0000 7f683de00640 1 --2- 172.21.15.46:0/613727185 >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 0x7f684006d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.146+0000 7f683de00640 1 -- 172.21.15.46:0/613727185 >> v1:172.21.15.142:6789/0 conn(0x7f684006dd40 legacy=0x7f68400cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.146+0000 7f683de00640 1 -- 172.21.15.46:0/613727185 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400d1c30 con 0x7f684015f5b0 2024-04-26T23:42:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.146+0000 7f6835400640 1 --2- 172.21.15.46:0/613727185 >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 0x7f684006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:04.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.147+0000 7f683de00640 1 --2- 172.21.15.46:0/613727185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684015f5b0 0x7f68400b4190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68400a0db0 tx=0x7f683002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1525c03a70d7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:04.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.147+0000 7f683d400640 1 -- 172.21.15.46:0/613727185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830037070 con 0x7f684015f5b0 2024-04-26T23:42:04.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.147+0000 7f683d400640 1 -- 172.21.15.46:0/613727185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830032440 con 0x7f684015f5b0 2024-04-26T23:42:04.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.147+0000 7f683d400640 1 -- 172.21.15.46:0/613727185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830032810 con 0x7f684015f5b0 2024-04-26T23:42:04.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 -- 172.21.15.46:0/613727185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684015f5b0 msgr2=0x7f68400b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:04.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 --2- 172.21.15.46:0/613727185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684015f5b0 0x7f68400b4190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68400a0db0 tx=0x7f683002a230 comp rx=0 tx=0).stop 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 -- 172.21.15.46:0/613727185 shutdown_connections 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 --2- 172.21.15.46:0/613727185 >> v2:172.21.15.142:3300/0 conn(0x7f68400b3c60 0x7f684006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 --2- 172.21.15.46:0/613727185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684015f5b0 0x7f68400b4190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 -- 172.21.15.46:0/613727185 >> 172.21.15.46:0/613727185 conn(0x7f68400d2da0 msgr2=0x7f68400d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.148+0000 7f683f200640 1 -- 172.21.15.46:0/613727185 shutdown_connections 2024-04-26T23:42:04.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.149+0000 7f683f200640 1 -- 172.21.15.46:0/613727185 wait complete. 2024-04-26T23:42:04.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.150+0000 7f683f200640 1 Processor -- start 2024-04-26T23:42:04.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.151+0000 7f683f200640 1 -- start start 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.151+0000 7f683f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f684006dd40 0x7f6840151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f683f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f683f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f684015f5b0 0x7f68400b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f683f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f684006e1d0 con 0x7f68400b3c60 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f683f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f684015f990 con 0x7f684006dd40 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f683f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68400ceb60 con 0x7f684015f5b0 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:04.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40484/0 (socket says 172.21.15.46:40484) 2024-04-26T23:42:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 -- 172.21.15.46:0/3401075408 learned_addr learned my addr 172.21.15.46:0/3401075408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 -- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f684015f5b0 msgr2=0x7f68400b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f684015f5b0 0x7f68400b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 -- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f684006dd40 msgr2=0x7f6840151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f684006dd40 0x7f6840151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.152+0000 7f6835400640 1 -- 172.21.15.46:0/3401075408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400cee40 con 0x7f68400b3c60 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f6835400640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f682800c7f0 tx=0x7f682800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828015070 con 0x7f68400b3c60 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828007820 con 0x7f68400b3c60 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828007bd0 con 0x7f68400b3c60 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68400cf120 con 0x7f68400b3c60 2024-04-26T23:42:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.153+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68400cfe60 con 0x7f68400b3c60 2024-04-26T23:42:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.155+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68400693b0 con 0x7f68400b3c60 2024-04-26T23:42:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.156+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6828018070 con 0x7f68400b3c60 2024-04-26T23:42:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.157+0000 7f6836a00640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.157+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f682808be00 con 0x7f68400b3c60 2024-04-26T23:42:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.157+0000 7f683de00640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.159+0000 7f683de00640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6830034000 tx=0x7f683002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.162+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f682805a090 con 0x7f68400b3c60 2024-04-26T23:42:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.412+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68400b5710 con 0x7f68400b3c60 2024-04-26T23:42:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.413+0000 7f6836a00640 1 -- 172.21.15.46:0/3401075408 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f6828059990 con 0x7f68400b3c60 2024-04-26T23:42:04.416 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:42:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 msgr2=0x7f681c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6830034000 tx=0x7f683002c000 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 msgr2=0x7f68400cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f682800c7f0 tx=0x7f682800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 shutdown_connections 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f684015f5b0 0x7f68400b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68400b3c60 0x7f68400cf920 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 --2- 172.21.15.46:0/3401075408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f684006dd40 0x7f6840151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 >> 172.21.15.46:0/3401075408 conn(0x7f68400d2da0 msgr2=0x7f684015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 shutdown_connections 2024-04-26T23:42:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04.416+0000 7f683f200640 1 -- 172.21.15.46:0/3401075408 wait complete. 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:42:04.439 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:42:04.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:04.440 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:42:04.496 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:42:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:42:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:42:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42296sec preferred_lft 42296sec 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:42:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:42:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:42:04.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:04.499 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:42:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:42:04.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:04.590 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:42:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:42:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:42:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:42:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:42:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:42:04.647 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:42:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:42:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:42:04.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:04.879 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:42:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:42:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:42:04.934 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:42:04.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:04.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:42:05.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:42:05.016 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:05.075 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:42:05.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:42:05.075 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:05.132 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:42:05.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:05.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:42:05.269 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:42:05.269 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:42:05.270 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:42:05.291 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:42:05.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:42:05.291 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:42:05.292 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:42:05.292 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:42:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:42:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:42:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:42:05.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:42:05.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:42:05.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:42:05.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:42:05.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:42:05.321 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:42:05.322 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:42:05.322 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:42:05.322 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:42:05.322 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:42:05.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:42:05.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:05.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:05.397 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:42:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:42:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:42:05.534 INFO:teuthology.orchestra.run.smithi046.stdout:client4967 2024-04-26T23:42:05.535 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:05.535 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:05.587 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:05.587 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-26T23:42:05.587 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-26T23:42:05.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:05.588 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-26T23:42:05.652 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:42:05.652 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:42:05.652 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:42:05.652 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-04-26T23:42:05.653 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:42:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:05 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-26T23:42:05.813 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-26T23:42:05.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:05.813 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-26T23:42:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-26T23:42:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-26T23:42:08.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:08.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:Updating files: 16% (234/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 32% (450/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 45% (640/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 69% (971/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 85% (1195/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 99% (1386/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-26T23:42:14.944 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-26T23:42:14.944 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-26T23:42:14.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.197+0000 7f64f2200640 1 Processor -- start 2024-04-26T23:42:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.197+0000 7f64f2200640 1 -- start start 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.198+0000 7f64f2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.198+0000 7f64f2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64ec06ca60 0x7f64ec0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.198+0000 7f64f2200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ec163710 con 0x7f64ec163200 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.198+0000 7f64f2200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f64ec1639f0 con 0x7f64ec06ca60 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.198+0000 7f64f2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec0cff50 con 0x7f64ec162620 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64ec06ca60 0x7f64ec0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64ec06ca60 0x7f64ec0d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39554/0 (socket says 172.21.15.46:39554) 2024-04-26T23:42:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 -- 172.21.15.46:0/276571742 learned_addr learned my addr 172.21.15.46:0/276571742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 -- 172.21.15.46:0/276571742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 msgr2=0x7f64ec162cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 --2- 172.21.15.46:0/276571742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 -- 172.21.15.46:0/276571742 >> v1:172.21.15.142:6789/0 conn(0x7f64ec163200 legacy=0x7f64ec06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.199+0000 7f64f1800640 1 -- 172.21.15.46:0/276571742 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec0d0230 con 0x7f64ec06ca60 2024-04-26T23:42:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.200+0000 7f64f1800640 1 --2- 172.21.15.46:0/276571742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec0d0750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009e50 tx=0x7f64e402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd1319db6dca045 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.200+0000 7f64eb400640 1 -- 172.21.15.46:0/276571742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4036070 con 0x7f64ec06ca60 2024-04-26T23:42:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64eb400640 1 -- 172.21.15.46:0/276571742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4031600 con 0x7f64ec06ca60 2024-04-26T23:42:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64eb400640 1 -- 172.21.15.46:0/276571742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40319d0 con 0x7f64ec06ca60 2024-04-26T23:42:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 -- 172.21.15.46:0/276571742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 msgr2=0x7f64ec0d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 --2- 172.21.15.46:0/276571742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec0d0750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009e50 tx=0x7f64e402a6d0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 -- 172.21.15.46:0/276571742 shutdown_connections 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 --2- 172.21.15.46:0/276571742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec0d0750 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 --2- 172.21.15.46:0/276571742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 -- 172.21.15.46:0/276571742 >> 172.21.15.46:0/276571742 conn(0x7f64ec0d1cd0 msgr2=0x7f64ec0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.201+0000 7f64f2200640 1 -- 172.21.15.46:0/276571742 shutdown_connections 2024-04-26T23:42:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.202+0000 7f64f2200640 1 -- 172.21.15.46:0/276571742 wait complete. 2024-04-26T23:42:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.203+0000 7f64f2200640 1 Processor -- start 2024-04-26T23:42:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.203+0000 7f64f2200640 1 -- start start 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 0x7f64ec160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec0b1320 con 0x7f64ec06ca60 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec0b11a0 con 0x7f64ec163200 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.204+0000 7f64f2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec0d0c90 con 0x7f64ec162620 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39568/0 (socket says 172.21.15.46:39568) 2024-04-26T23:42:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 -- 172.21.15.46:0/861317929 learned_addr learned my addr 172.21.15.46:0/861317929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f1800640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 0x7f64ec160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 -- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 msgr2=0x7f64ec0c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64ebe00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec0c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 -- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 msgr2=0x7f64ec160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 0x7f64ec160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 -- 172.21.15.46:0/861317929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec0ca240 con 0x7f64ec06ca60 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f1800640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 0x7f64ec160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64ebe00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.205+0000 7f64f0e00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b4a0 tx=0x7f64e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.206+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0014070 con 0x7f64ec06ca60 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.206+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e0010610 con 0x7f64ec06ca60 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.206+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0010a00 con 0x7f64ec06ca60 2024-04-26T23:42:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.206+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec0ca520 con 0x7f64ec06ca60 2024-04-26T23:42:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.206+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec0caae0 con 0x7f64ec06ca60 2024-04-26T23:42:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.207+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec069ac0 con 0x7f64ec06ca60 2024-04-26T23:42:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.208+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64e0017070 con 0x7f64ec06ca60 2024-04-26T23:42:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.209+0000 7f64e8c00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.209+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f64e008b840 con 0x7f64ec06ca60 2024-04-26T23:42:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.210+0000 7f64ebe00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.211+0000 7f64ebe00640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 0x7f64cc06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64ec161ac0 tx=0x7f64d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.214+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64e0059a40 con 0x7f64ec06ca60 2024-04-26T23:42:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.459+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64ec162ec0 con 0x7f64ec06ca60 2024-04-26T23:42:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.460+0000 7f64e8c00640 1 -- 172.21.15.46:0/861317929 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f64e0059340 con 0x7f64ec06ca60 2024-04-26T23:42:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:15.464 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 msgr2=0x7f64cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 0x7f64cc06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64ec161ac0 tx=0x7f64d8008040 comp rx=0 tx=0).stop 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 msgr2=0x7f64ec14d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b4a0 tx=0x7f64e000b960 comp rx=0 tx=0).stop 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 shutdown_connections 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64cc06c8f0 0x7f64cc06edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64ec163200 0x7f64ec160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64ec162620 0x7f64ec0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 --2- 172.21.15.46:0/861317929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec06ca60 0x7f64ec14d190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.462+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 >> 172.21.15.46:0/861317929 conn(0x7f64ec0d1cd0 msgr2=0x7f64ec15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.463+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 shutdown_connections 2024-04-26T23:42:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.463+0000 7f64f2200640 1 -- 172.21.15.46:0/861317929 wait complete. 2024-04-26T23:42:15.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-26T23:42:15.700 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": -2 2024-04-26T23:42:15.701 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:15.716 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-26T23:42:15.716 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-26T23:42:15.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-26T23:42:15.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.928+0000 7f0935200640 1 Processor -- start 2024-04-26T23:42:15.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.929+0000 7f0935200640 1 -- start start 2024-04-26T23:42:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.930+0000 7f0935200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f093015cf40 0x7f093006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f0935200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 0x7f093006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f0935200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0930160c60 con 0x7f093006f060 2024-04-26T23:42:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f0935200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0930160f00 con 0x7f093015cf40 2024-04-26T23:42:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f0935200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f09301611e0 con 0x7f09300af530 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f092de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f093015cf40 0x7f093006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f092de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f093015cf40 0x7f093006c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39582/0 (socket says 172.21.15.46:39582) 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f092de00640 1 -- 172.21.15.46:0/1225720145 learned_addr learned my addr 172.21.15.46:0/1225720145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f092e800640 1 --2- 172.21.15.46:0/1225720145 >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 0x7f093006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.931+0000 7f092de00640 1 -- 172.21.15.46:0/1225720145 >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 msgr2=0x7f093006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.932+0000 7f092de00640 1 --2- 172.21.15.46:0/1225720145 >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 0x7f093006c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.932+0000 7f092de00640 1 -- 172.21.15.46:0/1225720145 >> v1:172.21.15.46:6789/0 conn(0x7f093006f060 legacy=0x7f09300aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.932+0000 7f092de00640 1 -- 172.21.15.46:0/1225720145 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09301614c0 con 0x7f093015cf40 2024-04-26T23:42:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.932+0000 7f092ca00640 1 -- 172.21.15.46:0/1225720145 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339272401 0 0) 0x7f0930160c60 con 0x7f093006f060 2024-04-26T23:42:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.932+0000 7f092e800640 1 --2- 172.21.15.46:0/1225720145 >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 0x7f093006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.933+0000 7f092de00640 1 --2- 172.21.15.46:0/1225720145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f093006c350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f09300d24c0 tx=0x7f092002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cb4289cfcf387e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.933+0000 7f092ca00640 1 -- 172.21.15.46:0/1225720145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920037070 con 0x7f093015cf40 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.933+0000 7f092ca00640 1 -- 172.21.15.46:0/1225720145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920032440 con 0x7f093015cf40 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.933+0000 7f092ca00640 1 -- 172.21.15.46:0/1225720145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920032810 con 0x7f093015cf40 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 -- 172.21.15.46:0/1225720145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 msgr2=0x7f093006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 --2- 172.21.15.46:0/1225720145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f093006c350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f09300d24c0 tx=0x7f092002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 -- 172.21.15.46:0/1225720145 shutdown_connections 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 --2- 172.21.15.46:0/1225720145 >> v2:172.21.15.142:3300/0 conn(0x7f09300af530 0x7f093006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 --2- 172.21.15.46:0/1225720145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f093006c350 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 -- 172.21.15.46:0/1225720145 >> 172.21.15.46:0/1225720145 conn(0x7f093006da90 msgr2=0x7f09300d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 -- 172.21.15.46:0/1225720145 shutdown_connections 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.934+0000 7f0935200640 1 -- 172.21.15.46:0/1225720145 wait complete. 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.936+0000 7f0935200640 1 Processor -- start 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.936+0000 7f0935200640 1 -- start start 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 0x7f09300b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 0x7f09300b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09300cc840 con 0x7f093015cf40 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f093015d320 con 0x7f09300af530 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f0935200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09300cc9c0 con 0x7f093006f060 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39586/0 (socket says 172.21.15.46:39586) 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.937+0000 7f092e800640 1 -- 172.21.15.46:0/1181193138 learned_addr learned my addr 172.21.15.46:0/1181193138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092de00640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 0x7f09300b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092d400640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 0x7f09300b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 -- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 msgr2=0x7f09300b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 0x7f09300b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 -- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 msgr2=0x7f09300b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 0x7f09300b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 -- 172.21.15.46:0/1181193138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093015a5a0 con 0x7f093015cf40 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092d400640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 0x7f09300b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092de00640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 0x7f09300b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f092e800640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f092800c820 tx=0x7f092800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f0926000640 1 -- 172.21.15.46:0/1181193138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928015070 con 0x7f093015cf40 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f0926000640 1 -- 172.21.15.46:0/1181193138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928007a30 con 0x7f093015cf40 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.938+0000 7f0926000640 1 -- 172.21.15.46:0/1181193138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928007e20 con 0x7f093015cf40 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.939+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09300b50b0 con 0x7f093015cf40 2024-04-26T23:42:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.939+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09300b5950 con 0x7f093015cf40 2024-04-26T23:42:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.941+0000 7f0926000640 1 -- 172.21.15.46:0/1181193138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0928018070 con 0x7f093015cf40 2024-04-26T23:42:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.942+0000 7f0926000640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 0x7f090c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.942+0000 7f0926000640 1 -- 172.21.15.46:0/1181193138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f092808bcb0 con 0x7f093015cf40 2024-04-26T23:42:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.942+0000 7f092de00640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 0x7f090c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.943+0000 7f093803b740 1 Processor -- start 2024-04-26T23:42:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.943+0000 7f093803b740 1 -- start start 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615136aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 0x5561513696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092de00640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 0x7f090c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0920034000 tx=0x7f092002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5561512cfb90 con 0x556151369c20 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55615126c350 con 0x556151365970 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5561512cbb50 con 0x55615136aae0 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39602/0 (socket says 172.21.15.46:39602) 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 -- 172.21.15.46:0/1936000898 learned_addr learned my addr 172.21.15.46:0/1936000898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092d400640 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 0x5561513696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 -- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 msgr2=0x55615136aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615136aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 -- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 msgr2=0x5561513696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 0x5561513696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092e800640 1 -- 172.21.15.46:0/1936000898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561510ad9f0 con 0x556151369c20 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.944+0000 7f092d400640 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 0x5561513696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.945+0000 7f092e800640 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0928061e60 tx=0x7f092805db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d84c9987ba0337 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.945+0000 7f091ea00640 1 -- 172.21.15.46:0/1936000898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092801f040 con 0x556151369c20 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.945+0000 7f091ea00640 1 -- 172.21.15.46:0/1936000898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928060070 con 0x556151369c20 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.945+0000 7f091ea00640 1 -- 172.21.15.46:0/1936000898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928055e20 con 0x556151369c20 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 -- 172.21.15.46:0/1936000898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 msgr2=0x55615136a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0928061e60 tx=0x7f092805db60 comp rx=0 tx=0).stop 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 -- 172.21.15.46:0/1936000898 shutdown_connections 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x55615136a060 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151365970 0x5561513696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 --2- 172.21.15.46:0/1936000898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615136aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 -- 172.21.15.46:0/1936000898 >> 172.21.15.46:0/1936000898 conn(0x556151367be0 msgr2=0x5561513b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 -- 172.21.15.46:0/1936000898 shutdown_connections 2024-04-26T23:42:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.946+0000 7f093803b740 1 -- 172.21.15.46:0/1936000898 wait complete. 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.947+0000 7f093803b740 1 -- start start 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.947+0000 7f093803b740 1 Processor -- start 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 0x5561513b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556151396670 con 0x556151369c20 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5561512571a0 con 0x5561513ad890 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556151395af0 con 0x55615136aae0 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f093803b740 10 client.0 _ll_register_callbacks cb 0x556151270ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f092de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f092de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39618/0 (socket says 172.21.15.46:39618) 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f092de00640 1 -- 172.21.15.46:0/2225539737 learned_addr learned my addr 172.21.15.46:0/2225539737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f092e800640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 0x5561513b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.948+0000 7f092d400640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 -- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 msgr2=0x5561513ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 -- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 msgr2=0x5561513b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 0x5561513b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 -- 172.21.15.46:0/2225539737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556150fa12e0 con 0x556151369c20 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092d400640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092e800640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 0x5561513b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f092de00640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0920009930 tx=0x7f09200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10e1b43860ac287 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f0917400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920037040 con 0x556151369c20 2024-04-26T23:42:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f092003c8e0 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.949+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0928060070 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f092003cae0 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 10 client.5958 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55615110e180 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928060070 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f0920045070 con 0x556151369c20 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 10 client.5958 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 10 client.5958 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 20 client.5958 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5561513b27a0 0x5561513b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 4 client.5958 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.950+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561512a3550 con 0x5561513b27a0 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.951+0000 7f092d400640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5561513b27a0 0x5561513b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.951+0000 7f092d400640 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5561513b27a0 0x5561513b2bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb09e8fd031a935c server_cookie=da1ce65b563ed92b in_seq=0 out_seq=0 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.951+0000 7f0917400640 10 client.5958 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f0917400640 1 -- 172.21.15.46:0/2225539737 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561512a3550 con 0x5561513b27a0 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f0917400640 10 client.5958 handle_command_reply: tid=0 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 client.5958 shutdown 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 msgr2=0x5561513b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0920009930 tx=0x7f09200046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 shutdown_connections 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5561513b27a0 0x5561513b2bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561513ad890 0x5561513b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 --2- 172.21.15.46:0/2225539737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556151369c20 0x5561513b5a30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 >> 172.21.15.46:0/2225539737 conn(0x556151365970 msgr2=0x5561513650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f0917400640 0 client.5958 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 shutdown_connections 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.952+0000 7f093803b740 1 -- 172.21.15.46:0/2225539737 wait complete. 2024-04-26T23:42:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.953+0000 7f093803b740 20 client.5958 trim_cache size 0 max 16384 2024-04-26T23:42:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.972+0000 7f093803b740 1 Processor -- start 2024-04-26T23:42:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.972+0000 7f093803b740 1 -- start start 2024-04-26T23:42:15.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0930201c30 con 0x5561513ad890 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0930200c50 con 0x556151369c20 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09300c9f90 con 0x55615136aae0 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39624/0 (socket says 172.21.15.46:39624) 2024-04-26T23:42:15.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.973+0000 7f092e800640 1 -- 172.21.15.46:0/196675459 learned_addr learned my addr 172.21.15.46:0/196675459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 -- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 msgr2=0x5561513b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 -- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 msgr2=0x5561513b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 -- 172.21.15.46:0/196675459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556150f7a3a0 con 0x5561513ad890 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f092e800640 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0928007410 tx=0x7f0928060000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5886ff80abfb135 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f091e000640 1 -- 172.21.15.46:0/196675459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092801f040 con 0x5561513ad890 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f091e000640 1 -- 172.21.15.46:0/196675459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928060070 con 0x5561513ad890 2024-04-26T23:42:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.974+0000 7f091e000640 1 -- 172.21.15.46:0/196675459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928059eb0 con 0x5561513ad890 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 -- 172.21.15.46:0/196675459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 msgr2=0x5561513a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0928007410 tx=0x7f0928060000 comp rx=0 tx=0).stop 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 -- 172.21.15.46:0/196675459 shutdown_connections 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x5561513a6a80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x5561513b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 --2- 172.21.15.46:0/196675459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 -- 172.21.15.46:0/196675459 >> 172.21.15.46:0/196675459 conn(0x556151365970 msgr2=0x5561513651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 -- 172.21.15.46:0/196675459 shutdown_connections 2024-04-26T23:42:15.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.975+0000 7f093803b740 1 -- 172.21.15.46:0/196675459 wait complete. 2024-04-26T23:42:15.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.976+0000 7f093803b740 1 -- start start 2024-04-26T23:42:15.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.976+0000 7f093803b740 1 Processor -- start 2024-04-26T23:42:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615142a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09300ca5e0 con 0x5561513ad890 2024-04-26T23:42:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55615136af50 con 0x556151369c20 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55615138e070 con 0x55615136aae0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f093803b740 10 client.0 _ll_register_callbacks cb 0x7f0920035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f092e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39638/0 (socket says 172.21.15.46:39638) 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.977+0000 7f092e800640 1 -- 172.21.15.46:0/1691195851 learned_addr learned my addr 172.21.15.46:0/1691195851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092d400640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615142a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092de00640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 -- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 msgr2=0x55615142a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615142a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 -- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 msgr2=0x5561513a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 -- 172.21.15.46:0/1691195851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561513b11f0 con 0x5561513ad890 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f092e800640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f092800c7f0 tx=0x7f092801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2595c407809b2a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:15.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f0917400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092807b070 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.978+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0928055e20 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0928060070 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f092801f070 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 10 client.5964 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x556151427340 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928060070 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f0928058c00 con 0x5561513ad890 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 10 client.5964 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 20 client.5964 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55615138c650 0x5561513b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 4 client.5964 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) -- 0x5561512dd7a0 con 0x55615138c650 2024-04-26T23:42:15.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.979+0000 7f092de00640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55615138c650 0x5561513b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f092de00640 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55615138c650 0x5561513b2b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1fcf1c442d7e201 server_cookie=9112466b4953bded in_seq=0 out_seq=0 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f0917400640 10 client.5964 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f0917400640 1 -- 172.21.15.46:0/1691195851 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+26 (crc 0 0 0) 0x5561512dd7a0 con 0x55615138c650 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f0917400640 10 client.5964 handle_command_reply: tid=0 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f093803b740 1 client.5964 shutdown 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 msgr2=0x55615138d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.980+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f092800c7f0 tx=0x7f092801c000 comp rx=0 tx=0).stop 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 shutdown_connections 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561513ad890 0x55615138d650 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55615136aae0 0x55615142a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55615138c650 0x5561513b2b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 --2- 172.21.15.46:0/1691195851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556151369c20 0x5561513a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 >> 172.21.15.46:0/1691195851 conn(0x556151365970 msgr2=0x5561513d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f0917400640 0 client.5964 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 shutdown_connections 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 1 -- 172.21.15.46:0/1691195851 wait complete. 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.981+0000 7f093803b740 20 client.5964 trim_cache size 0 max 16384 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": -2 2024-04-26T23:42:15.988 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 msgr2=0x7f090c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 0x7f090c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0920034000 tx=0x7f092002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 msgr2=0x7f0930152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f092800c820 tx=0x7f092800cce0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 shutdown_connections 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f090c06c8f0 0x7f090c06edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f093015cf40 0x7f0930152120 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09300af530 0x7f09300b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 --2- 172.21.15.46:0/1181193138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f093006f060 0x7f09300b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 >> 172.21.15.46:0/1181193138 conn(0x7f093006da90 msgr2=0x7f093006fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:15.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 shutdown_connections 2024-04-26T23:42:15.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:15.982+0000 7f0935200640 1 -- 172.21.15.46:0/1181193138 wait complete. 2024-04-26T23:42:16.003 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-26T23:42:16.003 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-26T23:42:16.003 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-26T23:42:16.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.197+0000 7f6101000640 1 Processor -- start 2024-04-26T23:42:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.197+0000 7f6101000640 1 -- start start 2024-04-26T23:42:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f6101000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60fc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f6101000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc073ad0 con 0x7f60fc1466b0 2024-04-26T23:42:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f6101000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc146a90 con 0x7f60fc160910 2024-04-26T23:42:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f6101000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc073d70 con 0x7f60fc0720e0 2024-04-26T23:42:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f60fb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60fc1466b0 0x7f60fc06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36656/0 (socket says 172.21.15.46:36656) 2024-04-26T23:42:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f60fb400640 1 -- 172.21.15.46:0/2520745982 learned_addr learned my addr 172.21.15.46:0/2520745982 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.199+0000 7f60faa00640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60fc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285408408 0 0) 0x7f60fc073ad0 con 0x7f60fc1466b0 2024-04-26T23:42:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e0003520 con 0x7f60fc1466b0 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 725490314 0 0) 0x7f60e0003520 con 0x7f60fc1466b0 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 msgr2=0x7f60fc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60fc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 >> v1:172.21.15.142:6789/0 conn(0x7f60fc160910 legacy=0x7f60fc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc074050 con 0x7f60fc1466b0 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f60ec001670 con 0x7f60fc1466b0 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 >> v1:172.21.15.46:6789/0 conn(0x7f60fc1466b0 legacy=0x7f60fc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e00039e0 0x7f60e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e0004d40 0x7f60e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc073d70 con 0x7f60e00039e0 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60e000baa0 con 0x7f60e0004d40 2024-04-26T23:42:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.200+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60e000bca0 con 0x7f60fc0720e0 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e00039e0 0x7f60e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 -- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 msgr2=0x7f60e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60faa00640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60fc0720e0 0x7f60e0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 -- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e0004d40 msgr2=0x7f60e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e0004d40 0x7f60e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.201+0000 7f60fb400640 1 -- 172.21.15.46:0/2520745982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e0003520 con 0x7f60e00039e0 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.202+0000 7f60fb400640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e00039e0 0x7f60e0003e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f60ec001eb0 tx=0x7f60e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faeeaa242a11af95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.202+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007d30 con 0x7f60e00039e0 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.202+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec0058d0 con 0x7f60e00039e0 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.202+0000 7f60fa000640 1 -- 172.21.15.46:0/2520745982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec005ce0 con 0x7f60e00039e0 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 -- 172.21.15.46:0/2520745982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e00039e0 msgr2=0x7f60e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 --2- 172.21.15.46:0/2520745982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e00039e0 0x7f60e0003e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f60ec001eb0 tx=0x7f60e0005830 comp rx=0 tx=0).stop 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f60fbe00640 1 -- 172.21.15.46:0/2520745982 reap_dead start 2024-04-26T23:42:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 -- 172.21.15.46:0/2520745982 shutdown_connections 2024-04-26T23:42:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 -- 172.21.15.46:0/2520745982 >> 172.21.15.46:0/2520745982 conn(0x7f60fc0b26d0 msgr2=0x7f60fc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 -- 172.21.15.46:0/2520745982 shutdown_connections 2024-04-26T23:42:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.203+0000 7f6101000640 1 -- 172.21.15.46:0/2520745982 wait complete. 2024-04-26T23:42:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.205+0000 7f6101000640 1 Processor -- start 2024-04-26T23:42:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.205+0000 7f6101000640 1 -- start start 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e00039e0 0x7f60fc160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc0b5550 0x7f60fc137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0cc0a0 con 0x7f60fc0b6410 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0cb470 con 0x7f60fc0b5550 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f6101000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0cc220 con 0x7f60e00039e0 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f60faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f60faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39654/0 (socket says 172.21.15.46:39654) 2024-04-26T23:42:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.206+0000 7f60faa00640 1 -- 172.21.15.46:0/692501032 learned_addr learned my addr 172.21.15.46:0/692501032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60fb400640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e00039e0 0x7f60fc160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 -- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e00039e0 msgr2=0x7f60fc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e00039e0 0x7f60fc160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60fbe00640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc0b5550 0x7f60fc137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 -- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc0b5550 msgr2=0x7f60fc137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc0b5550 0x7f60fc137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 -- 172.21.15.46:0/692501032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0b44b0 con 0x7f60fc0b6410 2024-04-26T23:42:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60faa00640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60e8010860 tx=0x7f60e8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800f040 con 0x7f60fc0b6410 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e800eb60 con 0x7f60fc0b6410 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e801c400 con 0x7f60fc0b6410 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.207+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc0b4790 con 0x7f60fc0b6410 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.208+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc0b4d50 con 0x7f60fc0b6410 2024-04-26T23:42:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.209+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc0693b0 con 0x7f60fc0b6410 2024-04-26T23:42:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.210+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60e801c5a0 con 0x7f60fc0b6410 2024-04-26T23:42:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.211+0000 7f60f3400640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 0x7f60dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.211+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f60e808c2e0 con 0x7f60fc0b6410 2024-04-26T23:42:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.211+0000 7f60fb400640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 0x7f60dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.212+0000 7f60fb400640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 0x7f60dc06ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60ec033000 tx=0x7f60ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.215+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60e8018030 con 0x7f60fc0b6410 2024-04-26T23:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.460+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60fc0ceef0 con 0x7f60fc0b6410 2024-04-26T23:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.461+0000 7f60f3400640 1 -- 172.21.15.46:0/692501032 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f60e805a4e0 con 0x7f60fc0b6410 2024-04-26T23:42:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:16.466 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 msgr2=0x7f60dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 0x7f60dc06ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60ec033000 tx=0x7f60ec02e000 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 msgr2=0x7f60fc1e6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60e8010860 tx=0x7f60e8010d20 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 shutdown_connections 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60dc06c920 0x7f60dc06ede0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc0b5550 0x7f60fc137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60fc0b6410 0x7f60fc1e6750 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 --2- 172.21.15.46:0/692501032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e00039e0 0x7f60fc160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 >> 172.21.15.46:0/692501032 conn(0x7f60fc0b26d0 msgr2=0x7f60fc06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 shutdown_connections 2024-04-26T23:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.464+0000 7f6101000640 1 -- 172.21.15.46:0/692501032 wait complete. 2024-04-26T23:42:16.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-26T23:42:16.711 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:16.711 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:16.728 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:16.728 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-26T23:42:16.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-26T23:42:16.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.964+0000 7fe4c6a00640 1 Processor -- start 2024-04-26T23:42:16.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.965+0000 7fe4c6a00640 1 -- start start 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.966+0000 7fe4c6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4c00d3860 0x7fe4c00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.966+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c01630a0 0x7fe4c0163510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.966+0000 7fe4c6a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c0163e70 con 0x7fe4c015f480 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.966+0000 7fe4c6a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c0163a50 con 0x7fe4c00d3860 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.966+0000 7fe4c6a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c00d0aa0 con 0x7fe4c01630a0 2024-04-26T23:42:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.967+0000 7fe4bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4c00d3860 0x7fe4c00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.967+0000 7fe4bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4c00d3860 0x7fe4c00d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39664/0 (socket says 172.21.15.46:39664) 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.967+0000 7fe4bf400640 1 -- 172.21.15.46:0/3428298639 learned_addr learned my addr 172.21.15.46:0/3428298639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c01630a0 0x7fe4c0163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bea00640 1 -- 172.21.15.46:0/3428298639 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707173736 0 0) 0x7fe4c0163e70 con 0x7fe4c015f480 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bea00640 1 -- 172.21.15.46:0/3428298639 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4a4003520 con 0x7fe4c015f480 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bf400640 1 -- 172.21.15.46:0/3428298639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c01630a0 msgr2=0x7fe4c0163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bf400640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c01630a0 0x7fe4c0163510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bf400640 1 -- 172.21.15.46:0/3428298639 >> v1:172.21.15.142:6789/0 conn(0x7fe4c015f480 legacy=0x7fe4c00d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.968+0000 7fe4bf400640 1 -- 172.21.15.46:0/3428298639 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c00d0170 con 0x7fe4c00d3860 2024-04-26T23:42:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.971+0000 7fe4bf400640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c00d3860 0x7fe4c00d3cd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009870 tx=0x7fe4ac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b87f481a78546c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.972+0000 7fe4bea00640 1 -- 172.21.15.46:0/3428298639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037070 con 0x7fe4c00d3860 2024-04-26T23:42:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.972+0000 7fe4bea00640 1 -- 172.21.15.46:0/3428298639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac02ae50 con 0x7fe4c00d3860 2024-04-26T23:42:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.972+0000 7fe4bea00640 1 -- 172.21.15.46:0/3428298639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac032860 con 0x7fe4c00d3860 2024-04-26T23:42:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 -- 172.21.15.46:0/3428298639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c00d3860 msgr2=0x7fe4c00d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c00d3860 0x7fe4c00d3cd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009870 tx=0x7fe4ac02a300 comp rx=0 tx=0).stop 2024-04-26T23:42:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 -- 172.21.15.46:0/3428298639 shutdown_connections 2024-04-26T23:42:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c01630a0 0x7fe4c0163510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3428298639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c00d3860 0x7fe4c00d3cd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 -- 172.21.15.46:0/3428298639 >> 172.21.15.46:0/3428298639 conn(0x7fe4c00d5070 msgr2=0x7fe4c00d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 -- 172.21.15.46:0/3428298639 shutdown_connections 2024-04-26T23:42:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.973+0000 7fe4c6a00640 1 -- 172.21.15.46:0/3428298639 wait complete. 2024-04-26T23:42:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.975+0000 7fe4c6a00640 1 Processor -- start 2024-04-26T23:42:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.976+0000 7fe4c6a00640 1 -- start start 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.976+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c00d3860 0x7fe4c00b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c015f480 0x7fe4c0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c00d1640 con 0x7fe4c01630a0 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c6a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c00d0730 con 0x7fe4c00d3860 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c6a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c00cfe60 con 0x7fe4c015f480 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39676/0 (socket says 172.21.15.46:39676) 2024-04-26T23:42:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 -- 172.21.15.46:0/790937716 learned_addr learned my addr 172.21.15.46:0/790937716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4bf400640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c015f480 0x7fe4c0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c00d3860 0x7fe4c00b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 -- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c015f480 msgr2=0x7fe4c0153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c015f480 0x7fe4c0153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 -- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c00d3860 msgr2=0x7fe4c00b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c00d3860 0x7fe4c00b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.977+0000 7fe4c4a00640 1 -- 172.21.15.46:0/790937716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c00b8c50 con 0x7fe4c01630a0 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.978+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800efc0 tx=0x7fe4b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.978+0000 7fe4b7e00640 1 -- 172.21.15.46:0/790937716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8018020 con 0x7fe4c01630a0 2024-04-26T23:42:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.978+0000 7fe4b7e00640 1 -- 172.21.15.46:0/790937716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8014480 con 0x7fe4c01630a0 2024-04-26T23:42:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.978+0000 7fe4b7e00640 1 -- 172.21.15.46:0/790937716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8014870 con 0x7fe4c01630a0 2024-04-26T23:42:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.979+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c01f5190 con 0x7fe4c01630a0 2024-04-26T23:42:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.979+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01f5750 con 0x7fe4c01630a0 2024-04-26T23:42:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.981+0000 7fe4b7e00640 1 -- 172.21.15.46:0/790937716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4b8005d10 con 0x7fe4c01630a0 2024-04-26T23:42:16.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.982+0000 7fe4b7e00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 0x7fe4a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.983+0000 7fe4b7e00640 1 -- 172.21.15.46:0/790937716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4b808cc20 con 0x7fe4c01630a0 2024-04-26T23:42:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.983+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 0x7fe4a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.983+0000 7fe4c9837740 1 Processor -- start 2024-04-26T23:42:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 -- start start 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6f6120 0x55e90c6a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 0x55e90c6ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e90c612c30 con 0x55e90c6d9240 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e90c5af250 con 0x55e90c6d8b00 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e90c60ebf0 con 0x55e90c6f6120 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.984+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 0x7fe4a006edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00010e0 tx=0x7fe4b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39684/0 (socket says 172.21.15.46:39684) 2024-04-26T23:42:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 -- 172.21.15.46:0/2406645112 learned_addr learned my addr 172.21.15.46:0/2406645112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6f6120 0x55e90c6a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 0x55e90c6ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 -- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6f6120 msgr2=0x55e90c6a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6f6120 0x55e90c6a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 -- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 msgr2=0x55e90c6ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 0x55e90c6ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4bf400640 1 -- 172.21.15.46:0/2406645112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90c3f0b00 con 0x55e90c6d9240 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.985+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 0x55e90c6ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4bf400640 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c5bf3e3209ae4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4abe00640 1 -- 172.21.15.46:0/2406645112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac042070 con 0x55e90c6d9240 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4abe00640 1 -- 172.21.15.46:0/2406645112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac02ae50 con 0x55e90c6d9240 2024-04-26T23:42:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4abe00640 1 -- 172.21.15.46:0/2406645112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037040 con 0x55e90c6d9240 2024-04-26T23:42:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4c9837740 1 -- 172.21.15.46:0/2406645112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 msgr2=0x55e90c6d85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.986+0000 7fe4c9837740 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac02c000 comp rx=0 tx=0).stop 2024-04-26T23:42:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 -- 172.21.15.46:0/2406645112 shutdown_connections 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d8b00 0x55e90c6ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6d9240 0x55e90c6d85c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 --2- 172.21.15.46:0/2406645112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6f6120 0x55e90c6a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 -- 172.21.15.46:0/2406645112 >> 172.21.15.46:0/2406645112 conn(0x55e90c6aad20 msgr2=0x55e90c6f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 -- 172.21.15.46:0/2406645112 shutdown_connections 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 -- 172.21.15.46:0/2406645112 wait complete. 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.987+0000 7fe4c9837740 1 -- start start 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.988+0000 7fe4c9837740 1 Processor -- start 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 0x55e90c6dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d4120 0x55e90c6f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e90c6aced0 con 0x55e90c6f6120 2024-04-26T23:42:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e90c59a2e0 con 0x55e90c6d4120 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e90c6ac2c0 con 0x55e90c6d9240 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4c9837740 10 client.0 _ll_register_callbacks cb 0x55e90c608300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39694/0 (socket says 172.21.15.46:39694) 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 -- 172.21.15.46:0/48039043 learned_addr learned my addr 172.21.15.46:0/48039043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 0x55e90c6dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 -- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 msgr2=0x55e90c6dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 0x55e90c6dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 -- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d4120 msgr2=0x55e90c6f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d4120 0x55e90c6f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bf400640 1 -- 172.21.15.46:0/48039043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90c2e43f0 con 0x55e90c6f6120 2024-04-26T23:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.989+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 0x55e90c6dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4bf400640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009990 tx=0x7fe4ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab888e7ff6676e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037030 con 0x55e90c6f6120 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4ac049c90 con 0x55e90c6f6120 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ac02ae50 con 0x55e90c6f6120 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4ac042040 con 0x55e90c6f6120 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4c9837740 10 client.5982 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.990+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55e90c451290 con 0x55e90c6f6120 2024-04-26T23:42:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac02ae50 con 0x55e90c6f6120 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fe4ac003a20 con 0x55e90c6f6120 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 10 client.5982 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 10 client.5982 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 20 client.5982 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6d9c00 0x55e90c6da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 4 client.5982 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e90c5b18c0 con 0x55e90c6d9c00 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.991+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6d9c00 0x55e90c6da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.992+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6d9c00 0x55e90c6da040 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8624558f600723 server_cookie=a6c728ed1f85d3c5 in_seq=0 out_seq=0 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.992+0000 7fe4a8c00640 10 client.5982 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.992+0000 7fe4a8c00640 1 -- 172.21.15.46:0/48039043 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e90c5b18c0 con 0x55e90c6d9c00 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.992+0000 7fe4a8c00640 10 client.5982 handle_command_reply: tid=0 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 client.5982 shutdown 2024-04-26T23:42:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 msgr2=0x55e90c6f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009990 tx=0x7fe4ac004730 comp rx=0 tx=0).stop 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 shutdown_connections 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6d9c00 0x55e90c6da040 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d4120 0x55e90c6f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6f35c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 --2- 172.21.15.46:0/48039043 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d9240 0x55e90c6dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4a8c00640 0 client.5982 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 >> 172.21.15.46:0/48039043 conn(0x55e90c6d8b00 msgr2=0x55e90c6d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 shutdown_connections 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 1 -- 172.21.15.46:0/48039043 wait complete. 2024-04-26T23:42:17.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:16.993+0000 7fe4c9837740 20 client.5982 trim_cache size 0 max 16384 2024-04-26T23:42:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.012+0000 7fe4c9837740 1 Processor -- start 2024-04-26T23:42:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 -- start start 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0201df0 con 0x55e90c6f6120 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0200e40 con 0x55e90c6d9240 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.013+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c01f6b20 con 0x55e90c6d4120 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39710/0 (socket says 172.21.15.46:39710) 2024-04-26T23:42:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 -- 172.21.15.46:0/1735919991 learned_addr learned my addr 172.21.15.46:0/1735919991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 -- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 msgr2=0x55e90c6f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 -- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 msgr2=0x55e90c6d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 -- 172.21.15.46:0/1735919991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90c2bd4b0 con 0x55e90c6f6120 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.014+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80094c0 tx=0x7fe4b805a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a758ed32e118e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4ab400640 1 -- 172.21.15.46:0/1735919991 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b805ea50 con 0x55e90c6f6120 2024-04-26T23:42:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4ab400640 1 -- 172.21.15.46:0/1735919991 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8063280 con 0x55e90c6f6120 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4ab400640 1 -- 172.21.15.46:0/1735919991 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8056d10 con 0x55e90c6f6120 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 -- 172.21.15.46:0/1735919991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 msgr2=0x55e90c6ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80094c0 tx=0x7fe4b805a0c0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 -- 172.21.15.46:0/1735919991 shutdown_connections 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6ac4d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 --2- 172.21.15.46:0/1735919991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 -- 172.21.15.46:0/1735919991 >> 172.21.15.46:0/1735919991 conn(0x55e90c6d8b00 msgr2=0x55e90c702f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 -- 172.21.15.46:0/1735919991 shutdown_connections 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.015+0000 7fe4c9837740 1 -- 172.21.15.46:0/1735919991 wait complete. 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.016+0000 7fe4c9837740 1 -- start start 2024-04-26T23:42:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.016+0000 7fe4c9837740 1 Processor -- start 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c02007f0 con 0x55e90c6f6120 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e90c60d470 con 0x55e90c6d9240 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e90c6dc260 con 0x55e90c6d4120 2024-04-26T23:42:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c9837740 10 client.0 _ll_register_callbacks cb 0x7fe4a40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.017+0000 7fe4c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39722/0 (socket says 172.21.15.46:39722) 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 -- 172.21.15.46:0/3324950990 learned_addr learned my addr 172.21.15.46:0/3324950990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 -- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 msgr2=0x55e90c6db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 -- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 msgr2=0x55e90c6a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 -- 172.21.15.46:0/3324950990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90c6dbdb0 con 0x55e90c6f6120 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c4a00640 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80094c0 tx=0x7fe4b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b6f6d8fd36b967 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8062ff0 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4b8024b90 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.018+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b8063280 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4b805ea50 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 10 client.5988 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55e90c6d2b20 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8063280 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fe4b8059af0 con 0x55e90c6f6120 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 10 client.5988 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 10 client.5988 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 20 client.5988 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6f5840 0x55e90c76d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 4 client.5988 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) -- 0x55e90c620840 con 0x55e90c6f5840 2024-04-26T23:42:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.019+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6f5840 0x55e90c76d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.020+0000 7fe4bfe00640 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6f5840 0x55e90c76d520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39c5b102dceec2c server_cookie=109f50f0d4790b5e in_seq=0 out_seq=0 2024-04-26T23:42:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.020+0000 7fe4a8c00640 10 client.5988 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.020+0000 7fe4a8c00640 1 -- 172.21.15.46:0/3324950990 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e90c620840 con 0x55e90c6f5840 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.020+0000 7fe4a8c00640 10 client.5988 handle_command_reply: tid=0 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.020+0000 7fe4c9837740 1 client.5988 shutdown 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 msgr2=0x55e90c6a82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80094c0 tx=0x7fe4b801f000 comp rx=0 tx=0).stop 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 shutdown_connections 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e90c6f6120 0x55e90c6a82d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55e90c6f5840 0x55e90c76d520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e90c6d9240 0x55e90c6a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 --2- 172.21.15.46:0/3324950990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e90c6d4120 0x55e90c6db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 >> 172.21.15.46:0/3324950990 conn(0x55e90c6d8b00 msgr2=0x55e90c6ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4a8c00640 0 client.5988 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 shutdown_connections 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 1 -- 172.21.15.46:0/3324950990 wait complete. 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.021+0000 7fe4c9837740 20 client.5988 trim_cache size 0 max 16384 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9a021ed3-7cec-41a8-a84b-27d4a205cf8c", 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:17.029 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 msgr2=0x7fe4a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 0x7fe4a006edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00010e0 tx=0x7fe4b0009040 comp rx=0 tx=0).stop 2024-04-26T23:42:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 msgr2=0x7fe4c01596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800efc0 tx=0x7fe4b800c490 comp rx=0 tx=0).stop 2024-04-26T23:42:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 shutdown_connections 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a006c8f0 0x7fe4a006edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c01630a0 0x7fe4c01596d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c015f480 0x7fe4c0153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/790937716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c00d3860 0x7fe4c00b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 >> 172.21.15.46:0/790937716 conn(0x7fe4c00d5070 msgr2=0x7fe4c006cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 shutdown_connections 2024-04-26T23:42:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.022+0000 7fe4c6a00640 1 -- 172.21.15.46:0/790937716 wait complete. 2024-04-26T23:42:17.042 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '9a021ed3-7cec-41a8-a84b-27d4a205cf8c', 'mode': 'asynchronous'}' 2024-04-26T23:42:17.043 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-26T23:42:17.043 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-26T23:42:17.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:17.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.232+0000 7f653d600640 1 Processor -- start 2024-04-26T23:42:17.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.233+0000 7f653d600640 1 -- start start 2024-04-26T23:42:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f653d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653815f4e0 0x7f65380d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f653d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f65380b4000 0x7f65380d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f653d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65380d11f0 con 0x7f65380d1f60 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f653d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f65380d2990 con 0x7f653815f4e0 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f653d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f65380d5f20 con 0x7f65380b4000 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f6537e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653815f4e0 0x7f65380d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f6537e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653815f4e0 0x7f65380d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39726/0 (socket says 172.21.15.46:39726) 2024-04-26T23:42:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.234+0000 7f6537e00640 1 -- 172.21.15.46:0/528229352 learned_addr learned my addr 172.21.15.46:0/528229352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.235+0000 7f652fe00640 1 --2- 172.21.15.46:0/528229352 >> v2:172.21.15.142:3300/0 conn(0x7f65380b4000 0x7f65380d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.235+0000 7f6537e00640 1 -- 172.21.15.46:0/528229352 >> v2:172.21.15.142:3300/0 conn(0x7f65380b4000 msgr2=0x7f65380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.235+0000 7f6537e00640 1 --2- 172.21.15.46:0/528229352 >> v2:172.21.15.142:3300/0 conn(0x7f65380b4000 0x7f65380d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.235+0000 7f6537e00640 1 -- 172.21.15.46:0/528229352 >> v1:172.21.15.142:6789/0 conn(0x7f65380d1f60 legacy=0x7f65380cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.235+0000 7f6537e00640 1 -- 172.21.15.46:0/528229352 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380d0f10 con 0x7f653815f4e0 2024-04-26T23:42:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.236+0000 7f6537e00640 1 --2- 172.21.15.46:0/528229352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653815f4e0 0x7f65380d2df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f653808d880 tx=0x7f652802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c653be75aeae8a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.236+0000 7f6537400640 1 -- 172.21.15.46:0/528229352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037070 con 0x7f653815f4e0 2024-04-26T23:42:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.236+0000 7f6537400640 1 -- 172.21.15.46:0/528229352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528032440 con 0x7f653815f4e0 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.236+0000 7f6537400640 1 -- 172.21.15.46:0/528229352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528032810 con 0x7f653815f4e0 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 -- 172.21.15.46:0/528229352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653815f4e0 msgr2=0x7f65380d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 --2- 172.21.15.46:0/528229352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653815f4e0 0x7f65380d2df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f653808d880 tx=0x7f652802a230 comp rx=0 tx=0).stop 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 -- 172.21.15.46:0/528229352 shutdown_connections 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 --2- 172.21.15.46:0/528229352 >> v2:172.21.15.142:3300/0 conn(0x7f65380b4000 0x7f65380d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 --2- 172.21.15.46:0/528229352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653815f4e0 0x7f65380d2df0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 -- 172.21.15.46:0/528229352 >> 172.21.15.46:0/528229352 conn(0x7f65380d5010 msgr2=0x7f65380d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 -- 172.21.15.46:0/528229352 shutdown_connections 2024-04-26T23:42:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.237+0000 7f653d600640 1 -- 172.21.15.46:0/528229352 wait complete. 2024-04-26T23:42:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.239+0000 7f653d600640 1 Processor -- start 2024-04-26T23:42:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.239+0000 7f653d600640 1 -- start start 2024-04-26T23:42:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65380d1f60 0x7f653809ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f653815f4e0 0x7f6538152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f653815f8c0 con 0x7f65380b4000 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65380b1d60 con 0x7f65380d1f60 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f653d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65380b3290 con 0x7f653815f4e0 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.240+0000 7f6537e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39738/0 (socket says 172.21.15.46:39738) 2024-04-26T23:42:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 -- 172.21.15.46:0/3101004302 learned_addr learned my addr 172.21.15.46:0/3101004302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 -- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f653815f4e0 msgr2=0x7f6538152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f653815f4e0 0x7f6538152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 -- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65380d1f60 msgr2=0x7f653809ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65380d1f60 0x7f653809ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 -- 172.21.15.46:0/3101004302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380a18c0 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.241+0000 7f6537e00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6528000c00 tx=0x7f65280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.242+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528041020 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.242+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65280306c0 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.242+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037040 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.242+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380a1ba0 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.242+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65380a2160 con 0x7f65380b4000 2024-04-26T23:42:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.243+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65380693b0 con 0x7f65380b4000 2024-04-26T23:42:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.244+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6528044020 con 0x7f65380b4000 2024-04-26T23:42:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.245+0000 7f6534c00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.245+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f65280ac4f0 con 0x7f65380b4000 2024-04-26T23:42:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.246+0000 7f652fe00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.247+0000 7f652fe00640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65380cc410 tx=0x7f6520008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.250+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f652804b030 con 0x7f65380b4000 2024-04-26T23:42:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.497+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65380b7870 con 0x7f65380b4000 2024-04-26T23:42:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.498+0000 7f6534c00640 1 -- 172.21.15.46:0/3101004302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f652807a780 con 0x7f65380b4000 2024-04-26T23:42:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:17.503 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.501+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 msgr2=0x7f651406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.501+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65380cc410 tx=0x7f6520008040 comp rx=0 tx=0).stop 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.501+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 msgr2=0x7f65380ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.501+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6528000c00 tx=0x7f65280046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 shutdown_connections 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f653815f4e0 0x7f6538152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65380d1f60 0x7f653809ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 --2- 172.21.15.46:0/3101004302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b4000 0x7f65380ceb90 secure :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6528000c00 tx=0x7f65280046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 >> 172.21.15.46:0/3101004302 conn(0x7f65380d5010 msgr2=0x7f65380d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 shutdown_connections 2024-04-26T23:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.502+0000 7f653d600640 1 -- 172.21.15.46:0/3101004302 wait complete. 2024-04-26T23:42:17.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-26T23:42:17.743 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:17.743 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:17.758 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:17.759 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-26T23:42:17.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-26T23:42:17.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.969+0000 7f3c2fe00640 1 Processor -- start 2024-04-26T23:42:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.969+0000 7f3c2fe00640 1 -- start start 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c301626c0 0x7f3c300b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c30162aa0 0x7f3c300d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c300d07d0 con 0x7f3c300b5fb0 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3c300d0ab0 con 0x7f3c30162aa0 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c300d0d90 con 0x7f3c301626c0 2024-04-26T23:42:17.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c30162aa0 0x7f3c300d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c300b5fb0 0x7f3c300d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36660/0 (socket says 172.21.15.46:36660) 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.971+0000 7f3c2e000640 1 -- 172.21.15.46:0/1526307902 learned_addr learned my addr 172.21.15.46:0/1526307902 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c301626c0 0x7f3c300b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2f400640 1 -- 172.21.15.46:0/1526307902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c301626c0 msgr2=0x7f3c300b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2f400640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c301626c0 0x7f3c300b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2f400640 1 -- 172.21.15.46:0/1526307902 >> v1:172.21.15.46:6789/0 conn(0x7f3c300b5fb0 legacy=0x7f3c300d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2f400640 1 -- 172.21.15.46:0/1526307902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300d2c80 con 0x7f3c30162aa0 2024-04-26T23:42:17.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.972+0000 7f3c2d600640 1 -- 172.21.15.46:0/1526307902 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638977487 0 0) 0x7f3c300d07d0 con 0x7f3c300b5fb0 2024-04-26T23:42:17.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.973+0000 7f3c2f400640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c30162aa0 0x7f3c300d1900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009d90 tx=0x7f3c2802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a15b60066735b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.973+0000 7f3c2d600640 1 -- 172.21.15.46:0/1526307902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28007740 con 0x7f3c30162aa0 2024-04-26T23:42:17.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.973+0000 7f3c2d600640 1 -- 172.21.15.46:0/1526307902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c280078e0 con 0x7f3c30162aa0 2024-04-26T23:42:17.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.973+0000 7f3c2d600640 1 -- 172.21.15.46:0/1526307902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28007cb0 con 0x7f3c30162aa0 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 -- 172.21.15.46:0/1526307902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c30162aa0 msgr2=0x7f3c300d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c30162aa0 0x7f3c300d1900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009d90 tx=0x7f3c2802a660 comp rx=0 tx=0).stop 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 -- 172.21.15.46:0/1526307902 shutdown_connections 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c30162aa0 0x7f3c300d1900 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/1526307902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c301626c0 0x7f3c300b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 -- 172.21.15.46:0/1526307902 >> 172.21.15.46:0/1526307902 conn(0x7f3c3009e2a0 msgr2=0x7f3c300d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 -- 172.21.15.46:0/1526307902 shutdown_connections 2024-04-26T23:42:17.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.974+0000 7f3c2fe00640 1 -- 172.21.15.46:0/1526307902 wait complete. 2024-04-26T23:42:17.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.976+0000 7f3c2fe00640 1 Processor -- start 2024-04-26T23:42:17.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.976+0000 7f3c2fe00640 1 -- start start 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 0x7f3c300cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c30162aa0 0x7f3c300b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c300d21d0 con 0x7f3c300b5fb0 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c300d2050 con 0x7f3c301626c0 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.977+0000 7f3c2fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c300d31a0 con 0x7f3c30162aa0 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39760/0 (socket says 172.21.15.46:39760) 2024-04-26T23:42:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 -- 172.21.15.46:0/131493737 learned_addr learned my addr 172.21.15.46:0/131493737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2e000640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 0x7f3c300cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2f400640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c30162aa0 0x7f3c300b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 -- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c30162aa0 msgr2=0x7f3c300b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c30162aa0 0x7f3c300b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 -- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 msgr2=0x7f3c300cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 0x7f3c300cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 -- 172.21.15.46:0/131493737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300cdc20 con 0x7f3c300b5fb0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2e000640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 0x7f3c300cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.978+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000d800 tx=0x7f3c2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.979+0000 7f3c26a00640 1 -- 172.21.15.46:0/131493737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000b6c0 con 0x7f3c300b5fb0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.979+0000 7f3c26a00640 1 -- 172.21.15.46:0/131493737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c20019070 con 0x7f3c300b5fb0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.979+0000 7f3c26a00640 1 -- 172.21.15.46:0/131493737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000ba80 con 0x7f3c300b5fb0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.979+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c300cdf00 con 0x7f3c300b5fb0 2024-04-26T23:42:17.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.979+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c300ce4c0 con 0x7f3c300b5fb0 2024-04-26T23:42:17.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.981+0000 7f3c26a00640 1 -- 172.21.15.46:0/131493737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c2000bc20 con 0x7f3c300b5fb0 2024-04-26T23:42:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.982+0000 7f3c26a00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 0x7f3c1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.982+0000 7f3c26a00640 1 -- 172.21.15.46:0/131493737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c2008c270 con 0x7f3c300b5fb0 2024-04-26T23:42:17.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.983+0000 7f3c38a38740 1 Processor -- start 2024-04-26T23:42:17.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.983+0000 7f3c2e000640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 0x7f3c1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.983+0000 7f3c38a38740 1 -- start start 2024-04-26T23:42:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7accf40 0x5629a7a7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7aaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629a79e7d50 con 0x5629a7aaf920 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5629a79848c0 con 0x5629a7accf40 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5629a79e3d10 con 0x5629a7ab0060 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c2e000640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 0x7f3c1006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c300b8930 tx=0x7f3c1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39766/0 (socket says 172.21.15.46:39766) 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.984+0000 7f3c2f400640 1 -- 172.21.15.46:0/1167390830 learned_addr learned my addr 172.21.15.46:0/1167390830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2e000640 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7aaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7accf40 0x5629a7a7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 -- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 msgr2=0x5629a7aaf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7aaf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 -- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7accf40 msgr2=0x5629a7a7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7accf40 0x5629a7a7f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 -- 172.21.15.46:0/1167390830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a77c5f00 con 0x5629a7aaf920 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c2f400640 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3c28007300 tx=0x7f3c2802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d3690ee7e0bd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.985+0000 7f3c1b400640 1 -- 172.21.15.46:0/1167390830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28040070 con 0x5629a7aaf920 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c1b400640 1 -- 172.21.15.46:0/1167390830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c28007840 con 0x5629a7aaf920 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c1b400640 1 -- 172.21.15.46:0/1167390830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28007c10 con 0x5629a7aaf920 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 -- 172.21.15.46:0/1167390830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 msgr2=0x5629a7a84730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3c28007300 tx=0x7f3c2802f000 comp rx=0 tx=0).stop 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 -- 172.21.15.46:0/1167390830 shutdown_connections 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7aaf920 0x5629a7a84730 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7aaf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 --2- 172.21.15.46:0/1167390830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7accf40 0x5629a7a7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 -- 172.21.15.46:0/1167390830 >> 172.21.15.46:0/1167390830 conn(0x5629a7a81b40 msgr2=0x5629a7aca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 -- 172.21.15.46:0/1167390830 shutdown_connections 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.986+0000 7f3c38a38740 1 -- 172.21.15.46:0/1167390830 wait complete. 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.987+0000 7f3c38a38740 1 -- start start 2024-04-26T23:42:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.987+0000 7f3c38a38740 1 Processor -- start 2024-04-26T23:42:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.988+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 0x5629a7aab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.988+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7acd800 0x5629a7acdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c38a38740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629a7a83cf0 con 0x5629a7accf40 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5629a798c280 con 0x5629a7ab0060 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5629a7a830e0 con 0x5629a7acd800 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c38a38740 10 client.0 _ll_register_callbacks cb 0x5629a79954c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39782/0 (socket says 172.21.15.46:39782) 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 -- 172.21.15.46:0/3069246684 learned_addr learned my addr 172.21.15.46:0/3069246684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 -- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7acd800 msgr2=0x5629a7acdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2f400640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7acd800 0x5629a7acdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 0x5629a7aab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7acd800 0x5629a7acdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 -- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 msgr2=0x5629a7aab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 0x5629a7aab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2e000640 1 -- 172.21.15.46:0/3069246684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a76b97f0 con 0x5629a7accf40 2024-04-26T23:42:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.989+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 0x5629a7aab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c2e000640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c005a20 tx=0x7f3c1c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e737bb665b08724 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c026060 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c1c021c20 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c28007840 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c021e20 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c38a38740 10 client.6009 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.990+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5629a7826690 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c28007840 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f3c1c012410 con 0x5629a7accf40 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 10 client.6009 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 10 client.6009 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 20 client.6009 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7ab0a20 0x5629a7ab0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 4 client.6009 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629a79c3580 con 0x5629a7ab0a20 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.991+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7ab0a20 0x5629a7ab0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.992+0000 7f3c0fe00640 10 client.6009 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.992+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7ab0a20 0x5629a7ab0ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352a067cdf7890f7 server_cookie=1a9fb5a1ec1cc3ca in_seq=0 out_seq=0 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.992+0000 7f3c0fe00640 1 -- 172.21.15.46:0/3069246684 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629a79c3580 con 0x5629a7ab0a20 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.992+0000 7f3c0fe00640 10 client.6009 handle_command_reply: tid=0 2024-04-26T23:42:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.992+0000 7f3c38a38740 1 client.6009 shutdown 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 msgr2=0x5629a7aaddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c005a20 tx=0x7f3c1c001040 comp rx=0 tx=0).stop 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 shutdown_connections 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7acd800 0x5629a7acdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7aaddb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7ab0a20 0x5629a7ab0ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 --2- 172.21.15.46:0/3069246684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7ab0060 0x5629a7aab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 >> 172.21.15.46:0/3069246684 conn(0x5629a7aaf920 msgr2=0x5629a7a80330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c0fe00640 0 client.6009 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 shutdown_connections 2024-04-26T23:42:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 1 -- 172.21.15.46:0/3069246684 wait complete. 2024-04-26T23:42:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:17.993+0000 7f3c38a38740 20 client.6009 trim_cache size 0 max 16384 2024-04-26T23:42:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 Processor -- start 2024-04-26T23:42:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 -- start start 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7accf40 0x5629a7ac72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30205f10 con 0x5629a7ab0060 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30204f10 con 0x5629a7acd800 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.013+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c301f9d30 con 0x5629a7accf40 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39790/0 (socket says 172.21.15.46:39790) 2024-04-26T23:42:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 -- 172.21.15.46:0/2518653462 learned_addr learned my addr 172.21.15.46:0/2518653462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2e000640 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7accf40 0x5629a7ac72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 -- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7accf40 msgr2=0x5629a7ac72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7accf40 0x5629a7ac72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2f400640 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 -- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 msgr2=0x5629a7a80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a80330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.014+0000 7f3c2ea00640 1 -- 172.21.15.46:0/2518653462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a76928b0 con 0x5629a7ab0060 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3c200561d0 tx=0x7f3c200599d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf468add039a86d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c1aa00640 1 -- 172.21.15.46:0/2518653462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2008a270 con 0x5629a7ab0060 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c1aa00640 1 -- 172.21.15.46:0/2518653462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c20089070 con 0x5629a7ab0060 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c1aa00640 1 -- 172.21.15.46:0/2518653462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20062950 con 0x5629a7ab0060 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c38a38740 1 -- 172.21.15.46:0/2518653462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 msgr2=0x5629a7acaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.015+0000 7f3c38a38740 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3c200561d0 tx=0x7f3c200599d0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 -- 172.21.15.46:0/2518653462 shutdown_connections 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7accf40 0x5629a7ac72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 --2- 172.21.15.46:0/2518653462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7ab0060 0x5629a7acaac0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 -- 172.21.15.46:0/2518653462 >> 172.21.15.46:0/2518653462 conn(0x5629a7aaf920 msgr2=0x5629a7a7aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 -- 172.21.15.46:0/2518653462 shutdown_connections 2024-04-26T23:42:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 -- 172.21.15.46:0/2518653462 wait complete. 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.016+0000 7f3c38a38740 1 -- start start 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.017+0000 7f3c38a38740 1 Processor -- start 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7abfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629a7ac1b60 con 0x5629a7accf40 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301fa380 con 0x5629a7acd800 2024-04-26T23:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5629a7a826e0 con 0x5629a7ab0060 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c38a38740 10 client.0 _ll_register_callbacks cb 0x7f3c1c001c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39800/0 (socket says 172.21.15.46:39800) 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 -- 172.21.15.46:0/206357414 learned_addr learned my addr 172.21.15.46:0/206357414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 -- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 msgr2=0x5629a7abfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7abfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7abfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 -- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 msgr2=0x5629a7a832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.018+0000 7f3c2e000640 1 -- 172.21.15.46:0/206357414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a7aecd00 con 0x5629a7accf40 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c2e000640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c022fd0 tx=0x7f3c1c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d74049f659435 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c039060 con 0x5629a7accf40 2024-04-26T23:42:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c1c012bc0 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c20089070 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c012dc0 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c38a38740 10 client.6015 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5629a7b44410 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.019+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c20089070 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f3c1c012070 con 0x5629a7accf40 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 10 client.6015 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 20 client.6015 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7a83830 0x5629a7aaa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 4 client.6015 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) -- 0x5629a79f5960 con 0x5629a7a83830 2024-04-26T23:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.020+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7a83830 0x5629a7aaa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c2ea00640 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7a83830 0x5629a7aaa090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e812c018adf022 server_cookie=3ab1a1405cdd5da6 in_seq=0 out_seq=0 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c0fe00640 10 client.6015 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c0fe00640 1 -- 172.21.15.46:0/206357414 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629a79f5960 con 0x5629a7a83830 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c0fe00640 10 client.6015 handle_command_reply: tid=0 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c38a38740 1 client.6015 shutdown 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 msgr2=0x5629a7b438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c022fd0 tx=0x7f3c1c007000 comp rx=0 tx=0).stop 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.021+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 shutdown_connections 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5629a7a83830 0x5629a7aaa090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629a7acd800 0x5629a7a832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629a7accf40 0x5629a7b438d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 --2- 172.21.15.46:0/206357414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629a7ab0060 0x5629a7abfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 >> 172.21.15.46:0/206357414 conn(0x5629a7aaf920 msgr2=0x5629a7a80330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c0fe00640 0 client.6015 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 shutdown_connections 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 1 -- 172.21.15.46:0/206357414 wait complete. 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.022+0000 7f3c38a38740 20 client.6015 trim_cache size 0 max 16384 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "8f000adc-f873-4b65-9ef1-e2319c38e685", 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:18.029 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 msgr2=0x7f3c1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 0x7f3c1006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c300b8930 tx=0x7f3c1c009040 comp rx=0 tx=0).stop 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 msgr2=0x7f3c301545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000d800 tx=0x7f3c2000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 shutdown_connections 2024-04-26T23:42:18.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3c1006c8f0 0x7f3c1006edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c30162aa0 0x7f3c300b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c301626c0 0x7f3c300cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 --2- 172.21.15.46:0/131493737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c300b5fb0 0x7f3c301545e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 >> 172.21.15.46:0/131493737 conn(0x7f3c3009e2a0 msgr2=0x7f3c300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 shutdown_connections 2024-04-26T23:42:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.023+0000 7f3c2fe00640 1 -- 172.21.15.46:0/131493737 wait complete. 2024-04-26T23:42:18.044 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '8f000adc-f873-4b65-9ef1-e2319c38e685', 'mode': 'asynchronous'}' 2024-04-26T23:42:18.044 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-26T23:42:18.044 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-26T23:42:18.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:18.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.250+0000 7f11e1a00640 1 Processor -- start 2024-04-26T23:42:18.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.251+0000 7f11e1a00640 1 -- start start 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.252+0000 7f11e1a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11dc0d2130 0x7f11dc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0b4000 0x7f11dc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11e1a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11dc0d11f0 con 0x7f11dc15f4e0 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11e1a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f11dc0d0ce0 con 0x7f11dc0d2130 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11e1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11dc0d2990 con 0x7f11dc0b4000 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11dbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11dc15f4e0 0x7f11dc0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36674/0 (socket says 172.21.15.46:36674) 2024-04-26T23:42:18.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11dbe00640 1 -- 172.21.15.46:0/375254449 learned_addr learned my addr 172.21.15.46:0/375254449 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11db400640 1 --2- 172.21.15.46:0/375254449 >> v2:172.21.15.142:3300/0 conn(0x7f11dc0d2130 0x7f11dc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.253+0000 7f11e1000640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0b4000 0x7f11dc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11daa00640 1 -- 172.21.15.46:0/375254449 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 127445282 0 0) 0x7f11dc0d11f0 con 0x7f11dc15f4e0 2024-04-26T23:42:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11daa00640 1 -- 172.21.15.46:0/375254449 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11c0003520 con 0x7f11dc15f4e0 2024-04-26T23:42:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11db400640 1 -- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0b4000 msgr2=0x7f11dc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11db400640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0b4000 0x7f11dc0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11db400640 1 -- 172.21.15.46:0/375254449 >> v1:172.21.15.46:6789/0 conn(0x7f11dc15f4e0 legacy=0x7f11dc0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.254+0000 7f11db400640 1 -- 172.21.15.46:0/375254449 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc0d5f20 con 0x7f11dc0d2130 2024-04-26T23:42:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.255+0000 7f11db400640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc0d2130 0x7f11dc0d2df0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f11c40097a0 tx=0x7f11c402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f082c8ce729cbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.259+0000 7f11daa00640 1 -- 172.21.15.46:0/375254449 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037070 con 0x7f11dc0d2130 2024-04-26T23:42:18.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.259+0000 7f11daa00640 1 -- 172.21.15.46:0/375254449 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c4032440 con 0x7f11dc0d2130 2024-04-26T23:42:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.259+0000 7f11daa00640 1 -- 172.21.15.46:0/375254449 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4032810 con 0x7f11dc0d2130 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 -- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc0d2130 msgr2=0x7f11dc0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc0d2130 0x7f11dc0d2df0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f11c40097a0 tx=0x7f11c402a180 comp rx=0 tx=0).stop 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 -- 172.21.15.46:0/375254449 shutdown_connections 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0b4000 0x7f11dc0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 --2- 172.21.15.46:0/375254449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc0d2130 0x7f11dc0d2df0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 -- 172.21.15.46:0/375254449 >> 172.21.15.46:0/375254449 conn(0x7f11dc0d5010 msgr2=0x7f11dc0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 -- 172.21.15.46:0/375254449 shutdown_connections 2024-04-26T23:42:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.260+0000 7f11e1a00640 1 -- 172.21.15.46:0/375254449 wait complete. 2024-04-26T23:42:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.262+0000 7f11e1a00640 1 Processor -- start 2024-04-26T23:42:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.262+0000 7f11e1a00640 1 -- start start 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.263+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.263+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0d2130 0x7f11dc0a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc15f4e0 0x7f11dc0b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11e1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0b1d60 con 0x7f11dc0b4000 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11e1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0d5c40 con 0x7f11dc15f4e0 2024-04-26T23:42:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11e1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11dc15f8c0 con 0x7f11dc0d2130 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39812/0 (socket says 172.21.15.46:39812) 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 -- 172.21.15.46:0/88727077 learned_addr learned my addr 172.21.15.46:0/88727077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 -- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0d2130 msgr2=0x7f11dc0a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11db400640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0d2130 0x7f11dc0a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0d2130 0x7f11dc0a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 -- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc15f4e0 msgr2=0x7f11dc0b9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc15f4e0 0x7f11dc0b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.264+0000 7f11dbe00640 1 -- 172.21.15.46:0/88727077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc0a1360 con 0x7f11dc0b4000 2024-04-26T23:42:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11dbe00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c390 tx=0x7f11cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc017070 con 0x7f11dc0b4000 2024-04-26T23:42:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11cc005ce0 con 0x7f11dc0b4000 2024-04-26T23:42:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0072f0 con 0x7f11dc0b4000 2024-04-26T23:42:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc0a1640 con 0x7f11dc0b4000 2024-04-26T23:42:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.265+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc0a1c00 con 0x7f11dc0b4000 2024-04-26T23:42:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.266+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc0693b0 con 0x7f11dc0b4000 2024-04-26T23:42:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.267+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11cc01a070 con 0x7f11dc0b4000 2024-04-26T23:42:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.268+0000 7f11d3e00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 0x7f11b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.268+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f11cc08b100 con 0x7f11dc0b4000 2024-04-26T23:42:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.269+0000 7f11db400640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 0x7f11b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.270+0000 7f11db400640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 0x7f11b806edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f11c4034000 tx=0x7f11c402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.272+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11cc059300 con 0x7f11dc0b4000 2024-04-26T23:42:18.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.518+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11dc0d33d0 con 0x7f11dc0b4000 2024-04-26T23:42:18.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.519+0000 7f11d3e00640 1 -- 172.21.15.46:0/88727077 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f11cc058c00 con 0x7f11dc0b4000 2024-04-26T23:42:18.524 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:18.524 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:18.524 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:18.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 msgr2=0x7f11b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 0x7f11b806edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f11c4034000 tx=0x7f11c402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 msgr2=0x7f11dc09eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c390 tx=0x7f11cc00c850 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 shutdown_connections 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11b806c8f0 0x7f11b806edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc15f4e0 0x7f11dc0b9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11dc0d2130 0x7f11dc0a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 --2- 172.21.15.46:0/88727077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11dc0b4000 0x7f11dc09eed0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 >> 172.21.15.46:0/88727077 conn(0x7f11dc0d5010 msgr2=0x7f11dc0b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 shutdown_connections 2024-04-26T23:42:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.522+0000 7f11e1a00640 1 -- 172.21.15.46:0/88727077 wait complete. 2024-04-26T23:42:18.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-26T23:42:18.760 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:18.760 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:18.760 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:18.777 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:18.777 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-26T23:42:18.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-26T23:42:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.988+0000 7f75d9400640 1 Processor -- start 2024-04-26T23:42:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.988+0000 7f75d9400640 1 -- start start 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.989+0000 7f75d9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75d415f1f0 0x7f75d406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.989+0000 7f75d9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 0x7f75d4162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.989+0000 7f75d9400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d406f200 con 0x7f75d4163270 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75d9400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f75d4163d00 con 0x7f75d415f1f0 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75d9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75d406ccf0 con 0x7f75d406c890 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75d3200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75d4163270 0x7f75d40730b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36690/0 (socket says 172.21.15.46:36690) 2024-04-26T23:42:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75d3200640 1 -- 172.21.15.46:0/2255317047 learned_addr learned my addr 172.21.15.46:0/2255317047 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75d2800640 1 --2- 172.21.15.46:0/2255317047 >> v2:172.21.15.142:3300/0 conn(0x7f75d415f1f0 0x7f75d406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.990+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 0x7f75d4162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 684614566 0 0) 0x7f75d406f200 con 0x7f75d4163270 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75b8003520 con 0x7f75d4163270 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2887537745 0 0) 0x7f75b8003520 con 0x7f75d4163270 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 msgr2=0x7f75d4162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 0x7f75d4162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 >> v2:172.21.15.142:3300/0 conn(0x7f75d415f1f0 msgr2=0x7f75d406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 --2- 172.21.15.46:0/2255317047 >> v2:172.21.15.142:3300/0 conn(0x7f75d415f1f0 0x7f75d406c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d4163a20 con 0x7f75d4163270 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f75cc0019f0 con 0x7f75d4163270 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 >> v1:172.21.15.46:6789/0 conn(0x7f75d4163270 legacy=0x7f75d40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b80039b0 0x7f75b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b8004800 0x7f75b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 0x7f75b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4163d00 con 0x7f75b8004800 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800b9c0 con 0x7f75b80039b0 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75b800bb40 con 0x7f75d406c890 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.991+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b8004800 0x7f75b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75d2800640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b80039b0 0x7f75b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75c9e00640 1 -- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 msgr2=0x7f75b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75d406c890 0x7f75b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75c9e00640 1 -- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b80039b0 msgr2=0x7f75b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b80039b0 0x7f75b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.992+0000 7f75c9e00640 1 -- 172.21.15.46:0/2255317047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b8003520 con 0x7f75b8004800 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.993+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b8004800 0x7f75b8004c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009ac0 tx=0x7f75b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b75213d7fdec427 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.993+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc036070 con 0x7f75b8004800 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.993+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc031400 con 0x7f75b8004800 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.993+0000 7f75d1e00640 1 -- 172.21.15.46:0/2255317047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc009690 con 0x7f75b8004800 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 -- 172.21.15.46:0/2255317047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b8004800 msgr2=0x7f75b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 --2- 172.21.15.46:0/2255317047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b8004800 0x7f75b8004c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009ac0 tx=0x7f75b8005830 comp rx=0 tx=0).stop 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d3200640 1 -- 172.21.15.46:0/2255317047 reap_dead start 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 -- 172.21.15.46:0/2255317047 shutdown_connections 2024-04-26T23:42:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 -- 172.21.15.46:0/2255317047 >> 172.21.15.46:0/2255317047 conn(0x7f75d406da90 msgr2=0x7f75d40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 -- 172.21.15.46:0/2255317047 shutdown_connections 2024-04-26T23:42:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.994+0000 7f75d9400640 1 -- 172.21.15.46:0/2255317047 wait complete. 2024-04-26T23:42:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.996+0000 7f75d9400640 1 Processor -- start 2024-04-26T23:42:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.996+0000 7f75d9400640 1 -- start start 2024-04-26T23:42:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b8004800 0x7f75d40cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d41f53a0 0x7f75d415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75d415f5d0 con 0x7f75d40ce750 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75d406f810 con 0x7f75d41f53a0 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75d9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75d4163700 con 0x7f75b8004800 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39822/0 (socket says 172.21.15.46:39822) 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.997+0000 7f75c9e00640 1 -- 172.21.15.46:0/1467955251 learned_addr learned my addr 172.21.15.46:0/1467955251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 -- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b8004800 msgr2=0x7f75d40cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b8004800 0x7f75d40cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 -- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d41f53a0 msgr2=0x7f75d415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d41f53a0 0x7f75d415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 -- 172.21.15.46:0/1467955251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41f62d0 con 0x7f75d40ce750 2024-04-26T23:42:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75c9e00640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034040 tx=0x7f75bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75cb400640 1 -- 172.21.15.46:0/1467955251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc040020 con 0x7f75d40ce750 2024-04-26T23:42:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75cb400640 1 -- 172.21.15.46:0/1467955251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc0038d0 con 0x7f75d40ce750 2024-04-26T23:42:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75cb400640 1 -- 172.21.15.46:0/1467955251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc036040 con 0x7f75d40ce750 2024-04-26T23:42:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.998+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41f5d10 con 0x7f75d40ce750 2024-04-26T23:42:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:18.999+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41f8a50 con 0x7f75d40ce750 2024-04-26T23:42:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.001+0000 7f75cb400640 1 -- 172.21.15.46:0/1467955251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75bc043020 con 0x7f75d40ce750 2024-04-26T23:42:19.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.002+0000 7f75cb400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.003+0000 7f75cb400640 1 -- 172.21.15.46:0/1467955251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f75bc0abd70 con 0x7f75d40ce750 2024-04-26T23:42:19.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.003+0000 7f75d2800640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.003+0000 7f75dc272740 1 Processor -- start 2024-04-26T23:42:19.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.003+0000 7f75dc272740 1 -- start start 2024-04-26T23:42:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75d2800640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 0x7f75b006edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f75d40cc590 tx=0x7f75c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e25b0 0x55f7461b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f74611b8d0 con 0x55f7461e2cf0 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f7460b8070 con 0x55f7461e25b0 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.004+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f746117890 con 0x55f7461ffbd0 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39830/0 (socket says 172.21.15.46:39830) 2024-04-26T23:42:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 -- 172.21.15.46:0/3773401494 learned_addr learned my addr 172.21.15.46:0/3773401494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 -- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 msgr2=0x55f7461b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 -- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e25b0 msgr2=0x55f7461b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e25b0 0x55f7461b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 -- 172.21.15.46:0/3773401494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7460e47b0 con 0x55f7461e2cf0 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c9e00640 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0aa070 tx=0x7f75bc079900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9aef9823655e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c3000640 1 -- 172.21.15.46:0/3773401494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0824e0 con 0x55f7461e2cf0 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c3000640 1 -- 172.21.15.46:0/3773401494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc077c00 con 0x55f7461e2cf0 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.005+0000 7f75c3000640 1 -- 172.21.15.46:0/3773401494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc080070 con 0x55f7461e2cf0 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 -- 172.21.15.46:0/3773401494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 msgr2=0x55f7461e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0aa070 tx=0x7f75bc079900 comp rx=0 tx=0).stop 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 -- 172.21.15.46:0/3773401494 shutdown_connections 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e25b0 0x55f7461b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e2070 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 --2- 172.21.15.46:0/3773401494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 -- 172.21.15.46:0/3773401494 >> 172.21.15.46:0/3773401494 conn(0x55f7461b47d0 msgr2=0x55f7461fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 -- 172.21.15.46:0/3773401494 shutdown_connections 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.006+0000 7f75dc272740 1 -- 172.21.15.46:0/3773401494 wait complete. 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.007+0000 7f75dc272740 1 -- start start 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.007+0000 7f75dc272740 1 Processor -- start 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e4ef0 0x55f7461e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f746117890 con 0x55f7461e2cf0 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f7460a2ec0 con 0x55f7461e4ef0 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f74611b8d0 con 0x55f7461ffbd0 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.008+0000 7f75dc272740 10 client.0 _ll_register_callbacks cb 0x55f746128870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39832/0 (socket says 172.21.15.46:39832) 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 -- 172.21.15.46:0/2911671047 learned_addr learned my addr 172.21.15.46:0/2911671047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 -- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 msgr2=0x55f7461cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 -- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e4ef0 msgr2=0x55f7461e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e4ef0 0x55f7461e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 -- 172.21.15.46:0/2911671047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f745ef9690 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75d2800640 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f75c4015470 tx=0x7f75c4015870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc48c7a62b37be8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.009+0000 7f75b7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c401c050 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f75c40145f0 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75bc077c00 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75c4014840 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 10 client.6033 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55f745f59e20 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc077c00 con 0x55f7461e2cf0 2024-04-26T23:42:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f75c4031020 con 0x55f7461e2cf0 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 10 client.6033 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 20 client.6033 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461d0610 0x55f7461e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 4 client.6033 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.010+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7461294e0 con 0x55f7461d0610 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.011+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461d0610 0x55f7461e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.011+0000 7f75b7400640 10 client.6033 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.011+0000 7f75c9e00640 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461d0610 0x55f7461e11a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b66f2f43a2cdac server_cookie=db15a4a6e7867e7d in_seq=0 out_seq=0 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75b7400640 1 -- 172.21.15.46:0/2911671047 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7461294e0 con 0x55f7461d0610 2024-04-26T23:42:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75b7400640 10 client.6033 handle_command_reply: tid=0 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 client.6033 shutdown 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 msgr2=0x55f7461e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f75c4015470 tx=0x7f75c4015870 comp rx=0 tx=0).stop 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 shutdown_connections 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461d0610 0x55f7461e11a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e4ef0 0x55f7461e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461ffbd0 0x55f7461cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 --2- 172.21.15.46:0/2911671047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f7461e42e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 >> 172.21.15.46:0/2911671047 conn(0x55f7461e25b0 msgr2=0x55f7461b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.012+0000 7f75b7400640 0 client.6033 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.013+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 shutdown_connections 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.013+0000 7f75dc272740 1 -- 172.21.15.46:0/2911671047 wait complete. 2024-04-26T23:42:19.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.013+0000 7f75dc272740 20 client.6033 trim_cache size 0 max 16384 2024-04-26T23:42:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.032+0000 7f75dc272740 1 Processor -- start 2024-04-26T23:42:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.032+0000 7f75dc272740 1 -- start start 2024-04-26T23:42:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f7461b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461ffbd0 0x55f7461b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4202500 con 0x55f7461e2cf0 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75d42014e0 con 0x55f7461ffbd0 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75d40c9840 con 0x55f7461e4ef0 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.033+0000 7f75d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39838/0 (socket says 172.21.15.46:39838) 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 -- 172.21.15.46:0/220651187 learned_addr learned my addr 172.21.15.46:0/220651187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 -- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 msgr2=0x55f7461b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f7461b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 -- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461ffbd0 msgr2=0x55f7461b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461ffbd0 0x55f7461b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 -- 172.21.15.46:0/220651187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f745decf80 con 0x55f7461e2cf0 2024-04-26T23:42:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.034+0000 7f75d2800640 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f75c4006d40 tx=0x7f75c4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4829e4ab4c20eb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75c2600640 1 -- 172.21.15.46:0/220651187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c401c040 con 0x55f7461e2cf0 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75c2600640 1 -- 172.21.15.46:0/220651187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4026070 con 0x55f7461e2cf0 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75c2600640 1 -- 172.21.15.46:0/220651187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40149e0 con 0x55f7461e2cf0 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 -- 172.21.15.46:0/220651187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 msgr2=0x55f74620c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f75c4006d40 tx=0x7f75c4020000 comp rx=0 tx=0).stop 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 -- 172.21.15.46:0/220651187 shutdown_connections 2024-04-26T23:42:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461ffbd0 0x55f7461b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f7461b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 --2- 172.21.15.46:0/220651187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461e2cf0 0x55f74620c990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 -- 172.21.15.46:0/220651187 >> 172.21.15.46:0/220651187 conn(0x55f7461e25b0 msgr2=0x55f7461cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 -- 172.21.15.46:0/220651187 shutdown_connections 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.035+0000 7f75dc272740 1 -- 172.21.15.46:0/220651187 wait complete. 2024-04-26T23:42:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.036+0000 7f75dc272740 1 -- start start 2024-04-26T23:42:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.036+0000 7f75dc272740 1 Processor -- start 2024-04-26T23:42:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e2cf0 0x55f74626d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f746220210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7460b8070 con 0x55f7461ffbd0 2024-04-26T23:42:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4200ec0 con 0x55f7461e2cf0 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f7461dd3d0 con 0x55f7461e4ef0 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.037+0000 7f75dc272740 10 client.0 _ll_register_callbacks cb 0x7f75c40095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39852/0 (socket says 172.21.15.46:39852) 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 -- 172.21.15.46:0/1004646372 learned_addr learned my addr 172.21.15.46:0/1004646372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d2800640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e2cf0 0x55f74626d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75c9e00640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f746220210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 -- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 msgr2=0x55f746220210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f746220210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 -- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e2cf0 msgr2=0x55f74626d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e2cf0 0x55f74626d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.038+0000 7f75d3200640 1 -- 172.21.15.46:0/1004646372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f746276ff0 con 0x55f7461ffbd0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75d3200640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f75d4163650 tx=0x7f75cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f267a5ff1c04a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75b7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007110 con 0x55f7461ffbd0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f75cc010aa0 con 0x55f7461ffbd0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c4026070 con 0x55f7461ffbd0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75cc010ca0 con 0x55f7461ffbd0 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75dc272740 10 client.6039 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.039+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55f74620b910 con 0x55f7461ffbd0 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4026070 con 0x55f7461ffbd0 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f75cc0102e0 con 0x55f7461ffbd0 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 10 client.6039 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 20 client.6039 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461b82c0 0x55f7461b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 4 client.6039 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-26T23:42:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) -- 0x55f746123350 con 0x55f7461b82c0 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.040+0000 7f75d2800640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461b82c0 0x55f7461b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75d2800640 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461b82c0 0x55f7461b2000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df590b77a09f2e47 server_cookie=dc7ac2118cce8034 in_seq=0 out_seq=0 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75b7400640 10 client.6039 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75b7400640 1 -- 172.21.15.46:0/1004646372 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f746123350 con 0x55f7461b82c0 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75b7400640 10 client.6039 handle_command_reply: tid=0 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 client.6039 shutdown 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 msgr2=0x55f7461b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f75d4163650 tx=0x7f75cc00c8e0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 shutdown_connections 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7461ffbd0 0x55f7461b7d80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f7461e4ef0 0x55f746220210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55f7461b82c0 0x55f7461b2000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 --2- 172.21.15.46:0/1004646372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7461e2cf0 0x55f74626d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 >> 172.21.15.46:0/1004646372 conn(0x55f7461e25b0 msgr2=0x55f7461b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.041+0000 7f75b7400640 0 client.6039 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 shutdown_connections 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75dc272740 1 -- 172.21.15.46:0/1004646372 wait complete. 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75dc272740 20 client.6039 trim_cache size 0 max 16384 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ccf4d7d2-4ded-4559-8612-965efb20ea19", 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:19.059 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 msgr2=0x7f75b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 0x7f75b006edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f75d40cc590 tx=0x7f75c4008040 comp rx=0 tx=0).stop 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 msgr2=0x7f75d41f9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.042+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034040 tx=0x7f75bc004730 comp rx=0 tx=0).stop 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 shutdown_connections 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d41f53a0 0x7f75d415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d40ce750 0x7f75d41f9250 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 --2- 172.21.15.46:0/1467955251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b8004800 0x7f75d40cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 >> 172.21.15.46:0/1467955251 conn(0x7f75d406da90 msgr2=0x7f75d4070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 shutdown_connections 2024-04-26T23:42:19.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.043+0000 7f75d9400640 1 -- 172.21.15.46:0/1467955251 wait complete. 2024-04-26T23:42:19.065 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'ccf4d7d2-4ded-4559-8612-965efb20ea19', 'mode': 'asynchronous'}' 2024-04-26T23:42:19.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:19.066 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-26T23:42:19.120 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-26T23:42:19.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:19.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.329+0000 7f5087200640 1 Processor -- start 2024-04-26T23:42:19.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.330+0000 7f5087200640 1 -- start start 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.331+0000 7f5087200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508815f4e0 0x7f50880d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.331+0000 7f5087200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50880d11f0 con 0x7f50880b4000 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.331+0000 7f5087200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50880d5f20 con 0x7f50880d2130 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.331+0000 7f5087200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f50880d0ce0 con 0x7f508815f4e0 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508815f4e0 0x7f50880d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508815f4e0 0x7f50880d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39864/0 (socket says 172.21.15.46:39864) 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 -- 172.21.15.46:0/2716084692 learned_addr learned my addr 172.21.15.46:0/2716084692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f507fe00640 1 --1- 172.21.15.46:0/2716084692 >> v1:172.21.15.46:6789/0 conn(0x7f50880b4000 0x7f50880d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36700/0 (socket says 172.21.15.46:36700) 2024-04-26T23:42:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 -- 172.21.15.46:0/2716084692 >> v1:172.21.15.142:6789/0 conn(0x7f50880d2130 legacy=0x7f50880b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 -- 172.21.15.46:0/2716084692 >> v1:172.21.15.46:6789/0 conn(0x7f50880b4000 legacy=0x7f50880d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.332+0000 7f5085e00640 1 -- 172.21.15.46:0/2716084692 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50880d0f10 con 0x7f508815f4e0 2024-04-26T23:42:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.333+0000 7f5085e00640 1 --2- 172.21.15.46:0/2716084692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880d2df0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5078009870 tx=0x7f507802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac791644d1687d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5085400640 1 -- 172.21.15.46:0/2716084692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078037070 con 0x7f508815f4e0 2024-04-26T23:42:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5085400640 1 -- 172.21.15.46:0/2716084692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5078032440 con 0x7f508815f4e0 2024-04-26T23:42:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5085400640 1 -- 172.21.15.46:0/2716084692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078032810 con 0x7f508815f4e0 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 -- 172.21.15.46:0/2716084692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 msgr2=0x7f50880d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 --2- 172.21.15.46:0/2716084692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880d2df0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5078009870 tx=0x7f507802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 -- 172.21.15.46:0/2716084692 shutdown_connections 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 --2- 172.21.15.46:0/2716084692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880d2df0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 -- 172.21.15.46:0/2716084692 >> 172.21.15.46:0/2716084692 conn(0x7f50880d5010 msgr2=0x7f50880d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 -- 172.21.15.46:0/2716084692 shutdown_connections 2024-04-26T23:42:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.334+0000 7f5087200640 1 -- 172.21.15.46:0/2716084692 wait complete. 2024-04-26T23:42:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.336+0000 7f5087200640 1 Processor -- start 2024-04-26T23:42:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.336+0000 7f5087200640 1 -- start start 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50880b4000 0x7f50881f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 0x7f508815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50880b1d60 con 0x7f508815f4e0 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50880d2990 con 0x7f50880b4000 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.337+0000 7f5087200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f508815f8c0 con 0x7f50880d2130 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39878/0 (socket says 172.21.15.46:39878) 2024-04-26T23:42:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 -- 172.21.15.46:0/3689317714 learned_addr learned my addr 172.21.15.46:0/3689317714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:19.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f507fe00640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 0x7f508815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5085e00640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50880b4000 0x7f50881f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 -- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 msgr2=0x7f508815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 0x7f508815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 -- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50880b4000 msgr2=0x7f50881f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50880b4000 0x7f50881f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 -- 172.21.15.46:0/3689317714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50880b8180 con 0x7f508815f4e0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f507fe00640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 0x7f508815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.338+0000 7f5086800640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f508000c700 tx=0x7f508000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.339+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080016020 con 0x7f508815f4e0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.339+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508000aa50 con 0x7f508815f4e0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.339+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508000ce10 con 0x7f508815f4e0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.339+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50880b8460 con 0x7f508815f4e0 2024-04-26T23:42:19.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.339+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50880b8a20 con 0x7f508815f4e0 2024-04-26T23:42:19.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.340+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50880693b0 con 0x7f508815f4e0 2024-04-26T23:42:19.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.341+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5080019020 con 0x7f508815f4e0 2024-04-26T23:42:19.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.342+0000 7f507e000640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 0x7f506406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:19.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.342+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f508008bff0 con 0x7f508815f4e0 2024-04-26T23:42:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.343+0000 7f5085e00640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 0x7f506406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.344+0000 7f5085e00640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 0x7f506406ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5078034000 tx=0x7f507802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.347+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5080020020 con 0x7f508815f4e0 2024-04-26T23:42:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.600+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50880d33d0 con 0x7f508815f4e0 2024-04-26T23:42:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.602+0000 7f507e000640 1 -- 172.21.15.46:0/3689317714 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7f508005ae20 con 0x7f508815f4e0 2024-04-26T23:42:19.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:19.606 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 msgr2=0x7f506406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 0x7f506406ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5078034000 tx=0x7f507802e000 comp rx=0 tx=0).stop 2024-04-26T23:42:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 msgr2=0x7f50880ba1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f508000c700 tx=0x7f508000cbc0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 shutdown_connections 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506406c920 0x7f506406ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508815f4e0 0x7f50880ba1a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50880d2130 0x7f508815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 --2- 172.21.15.46:0/3689317714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50880b4000 0x7f50881f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 >> 172.21.15.46:0/3689317714 conn(0x7f50880d5010 msgr2=0x7f508815e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 shutdown_connections 2024-04-26T23:42:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:19.605+0000 7f5087200640 1 -- 172.21.15.46:0/3689317714 wait complete. 2024-04-26T23:42:19.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-26T23:42:19.855 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:19.855 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:19.855 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:19.871 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:19.872 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:19.872 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:19.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:19.922 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T23:42:20.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:42:20.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:42:20.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399951 s, 26.2 MB/s 2024-04-26T23:42:20.018 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-26T23:42:20.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.233+0000 7fbc3d200640 1 Processor -- start 2024-04-26T23:42:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.233+0000 7fbc3d200640 1 -- start start 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc3d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc380cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc3d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbc38073970 0x7fbc380cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc3d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc38072bb0 con 0x7fbc380c6a50 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc3d200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbc38072e90 con 0x7fbc38073970 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc3d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc38073170 con 0x7fbc381454a0 2024-04-26T23:42:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc37e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc380c6a50 0x7fbc380cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36702/0 (socket says 172.21.15.46:36702) 2024-04-26T23:42:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.235+0000 7fbc37e00640 1 -- 172.21.15.46:0/4083103847 learned_addr learned my addr 172.21.15.46:0/4083103847 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36a00640 1 --2- 172.21.15.46:0/4083103847 >> v2:172.21.15.142:3300/0 conn(0x7fbc38073970 0x7fbc380cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557324928 0 0) 0x7fbc38072bb0 con 0x7fbc380c6a50 2024-04-26T23:42:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc37400640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc380cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc1c007930 con 0x7fbc380c6a50 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3844983885 0 0) 0x7fbc1c007930 con 0x7fbc380c6a50 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 msgr2=0x7fbc380cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc380cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 >> v2:172.21.15.142:3300/0 conn(0x7fbc38073970 msgr2=0x7fbc380cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 --2- 172.21.15.46:0/4083103847 >> v2:172.21.15.142:3300/0 conn(0x7fbc38073970 0x7fbc380cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc38073450 con 0x7fbc380c6a50 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fbc300019f0 con 0x7fbc380c6a50 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 >> v1:172.21.15.46:6789/0 conn(0x7fbc380c6a50 legacy=0x7fbc380cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc1c007dc0 0x7fbc1c0081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc1c008c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c009150 0x7fbc1c0095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38072e90 con 0x7fbc1c009150 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38073170 con 0x7fbc1c007dc0 2024-04-26T23:42:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.236+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c00fe50 con 0x7fbc381454a0 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c009150 0x7fbc1c0095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37400640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc1c007dc0 0x7fbc1c0081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc36a00640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc1c008c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 -- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 msgr2=0x7fbc1c008c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc381454a0 0x7fbc1c008c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 -- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc1c007dc0 msgr2=0x7fbc1c0081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc1c007dc0 0x7fbc1c0081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37e00640 1 -- 172.21.15.46:0/4083103847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c007930 con 0x7fbc1c009150 2024-04-26T23:42:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.237+0000 7fbc37400640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc1c007dc0 0x7fbc1c0081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:20.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.238+0000 7fbc37e00640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c009150 0x7fbc1c0095d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000a7b0 tx=0x7fbc1c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383f0da18634522d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:20.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000a3d0 con 0x7fbc1c009150 2024-04-26T23:42:20.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3002f070 con 0x7fbc1c009150 2024-04-26T23:42:20.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc36000640 1 -- 172.21.15.46:0/4083103847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3002bc20 con 0x7fbc1c009150 2024-04-26T23:42:20.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 -- 172.21.15.46:0/4083103847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c009150 msgr2=0x7fbc1c0095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 --2- 172.21.15.46:0/4083103847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c009150 0x7fbc1c0095d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000a7b0 tx=0x7fbc1c009c40 comp rx=0 tx=0).stop 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc37e00640 1 -- 172.21.15.46:0/4083103847 reap_dead start 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 -- 172.21.15.46:0/4083103847 shutdown_connections 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 -- 172.21.15.46:0/4083103847 >> 172.21.15.46:0/4083103847 conn(0x7fbc380b26d0 msgr2=0x7fbc380b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 -- 172.21.15.46:0/4083103847 shutdown_connections 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.239+0000 7fbc3d200640 1 -- 172.21.15.46:0/4083103847 wait complete. 2024-04-26T23:42:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.241+0000 7fbc3d200640 1 Processor -- start 2024-04-26T23:42:20.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.241+0000 7fbc3d200640 1 -- start start 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.242+0000 7fbc3d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.242+0000 7fbc3d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 0x7fbc380b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.242+0000 7fbc3d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc380c7d30 0x7fbc380b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc3d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38073dd0 con 0x7fbc3815a9d0 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc3d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38072880 con 0x7fbc380ca750 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc3d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc380c9d20 con 0x7fbc380c7d30 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39896/0 (socket says 172.21.15.46:39896) 2024-04-26T23:42:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 -- 172.21.15.46:0/3294529448 learned_addr learned my addr 172.21.15.46:0/3294529448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37e00640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc380c7d30 0x7fbc380b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc36a00640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 0x7fbc380b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 -- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc380c7d30 msgr2=0x7fbc380b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc380c7d30 0x7fbc380b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 -- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 msgr2=0x7fbc380b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 0x7fbc380b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc37400640 1 -- 172.21.15.46:0/3294529448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc380c8260 con 0x7fbc3815a9d0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.243+0000 7fbc36a00640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 0x7fbc380b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc37400640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e7e0 tx=0x7fbc2800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28017070 con 0x7fbc3815a9d0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2800c6d0 con 0x7fbc3815a9d0 2024-04-26T23:42:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800f040 con 0x7fbc3815a9d0 2024-04-26T23:42:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc380c8540 con 0x7fbc3815a9d0 2024-04-26T23:42:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.244+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc380ca230 con 0x7fbc3815a9d0 2024-04-26T23:42:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.245+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc380693b0 con 0x7fbc3815a9d0 2024-04-26T23:42:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.247+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc2801a070 con 0x7fbc3815a9d0 2024-04-26T23:42:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.247+0000 7fbc2f400640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.248+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fbc2808b980 con 0x7fbc3815a9d0 2024-04-26T23:42:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.248+0000 7fbc36a00640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.249+0000 7fbc36a00640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 0x7fbc1806ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b5ef0 tx=0x7fbc24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.252+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc28059b80 con 0x7fbc3815a9d0 2024-04-26T23:42:20.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.501+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc380cefa0 con 0x7fbc3815a9d0 2024-04-26T23:42:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.502+0000 7fbc2f400640 1 -- 172.21.15.46:0/3294529448 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7fbc28059480 con 0x7fbc3815a9d0 2024-04-26T23:42:20.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:20.507 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.504+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 msgr2=0x7fbc1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.504+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 0x7fbc1806ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b5ef0 tx=0x7fbc24008040 comp rx=0 tx=0).stop 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 msgr2=0x7fbc3815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e7e0 tx=0x7fbc2800eca0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 shutdown_connections 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc380c7d30 0x7fbc380b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc380ca750 0x7fbc380b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 --2- 172.21.15.46:0/3294529448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3815a9d0 0x7fbc3815adb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 >> 172.21.15.46:0/3294529448 conn(0x7fbc380b26d0 msgr2=0x7fbc381445f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 shutdown_connections 2024-04-26T23:42:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20.505+0000 7fbc3d200640 1 -- 172.21.15.46:0/3294529448 wait complete. 2024-04-26T23:42:20.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-26T23:42:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:20.746 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:20.762 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:20.762 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:42:20.762 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:42:20.762 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:42:20.762 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:20.763 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-26T23:42:20.763 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:42:20.907 INFO:teuthology.orchestra.run.smithi046.stdout:1099511630412 2024-04-26T23:42:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:42:20.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-26T23:42:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.185+0000 7f981c000640 1 Processor -- start 2024-04-26T23:42:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.185+0000 7f981c000640 1 -- start start 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.186+0000 7f981c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98141597e0 0x7f98140ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98140b5730 0x7f98140b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981c000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 0x7f9814072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f98140b60d0 con 0x7f98140b5730 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981c000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f98140728c0 con 0x7f9814071f10 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9814073120 con 0x7f98141597e0 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f9818c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98140b5730 0x7f98140b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f9818c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98140b5730 0x7f98140b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39904/0 (socket says 172.21.15.46:39904) 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f9818c00640 1 -- 172.21.15.46:0/37509713 learned_addr learned my addr 172.21.15.46:0/37509713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f981a000640 1 --2- 172.21.15.46:0/37509713 >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 0x7f9814072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.187+0000 7f9819600640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98141597e0 0x7f98140ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f9818c00640 1 -- 172.21.15.46:0/37509713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98141597e0 msgr2=0x7f98140ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f9818c00640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98141597e0 0x7f98140ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f9818c00640 1 -- 172.21.15.46:0/37509713 >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 msgr2=0x7f9814072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f9818c00640 1 --2- 172.21.15.46:0/37509713 >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 0x7f9814072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f9818c00640 1 -- 172.21.15.46:0/37509713 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9814159bc0 con 0x7f98140b5730 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.188+0000 7f981a000640 1 --2- 172.21.15.46:0/37509713 >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 0x7f9814072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.189+0000 7f9818c00640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98140b5730 0x7f98140b5b90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f98140b3850 tx=0x7f980002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1e99ac3d7ed3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.189+0000 7f9813e00640 1 -- 172.21.15.46:0/37509713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800037070 con 0x7f98140b5730 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.189+0000 7f9813e00640 1 -- 172.21.15.46:0/37509713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9800032440 con 0x7f98140b5730 2024-04-26T23:42:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.189+0000 7f9813e00640 1 -- 172.21.15.46:0/37509713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800032810 con 0x7f98140b5730 2024-04-26T23:42:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 -- 172.21.15.46:0/37509713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98140b5730 msgr2=0x7f98140b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98140b5730 0x7f98140b5b90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f98140b3850 tx=0x7f980002a270 comp rx=0 tx=0).stop 2024-04-26T23:42:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 -- 172.21.15.46:0/37509713 shutdown_connections 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 --2- 172.21.15.46:0/37509713 >> v2:172.21.15.142:3300/0 conn(0x7f9814071f10 0x7f9814072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98140b5730 0x7f98140b5b90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 --2- 172.21.15.46:0/37509713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98141597e0 0x7f98140ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 -- 172.21.15.46:0/37509713 >> 172.21.15.46:0/37509713 conn(0x7f981406c980 msgr2=0x7f98140b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 -- 172.21.15.46:0/37509713 shutdown_connections 2024-04-26T23:42:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.190+0000 7f981c000640 1 -- 172.21.15.46:0/37509713 wait complete. 2024-04-26T23:42:21.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.192+0000 7f981c000640 1 Processor -- start 2024-04-26T23:42:21.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.192+0000 7f981c000640 1 -- start start 2024-04-26T23:42:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.193+0000 7f981c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.193+0000 7f981c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 0x7f98140cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.193+0000 7f981c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 0x7f9814156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.193+0000 7f981c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98140b40d0 con 0x7f9814071f10 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f981c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9814072e40 con 0x7f98141597e0 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f981c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f98140b43b0 con 0x7f98140b5730 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39920/0 (socket says 172.21.15.46:39920) 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 -- 172.21.15.46:0/2621551420 learned_addr learned my addr 172.21.15.46:0/2621551420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9818c00640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 0x7f98140cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 -- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 msgr2=0x7f98140cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 0x7f98140cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f981a000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 0x7f9814156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 -- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 msgr2=0x7f9814156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 0x7f9814156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9819600640 1 -- 172.21.15.46:0/2621551420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98140d0020 con 0x7f9814071f10 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f9818c00640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 0x7f98140cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.194+0000 7f981a000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 0x7f9814156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f9819600640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f980800d620 tx=0x7f980800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f9811600640 1 -- 172.21.15.46:0/2621551420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800b620 con 0x7f9814071f10 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f9811600640 1 -- 172.21.15.46:0/2621551420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808019070 con 0x7f9814071f10 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f9811600640 1 -- 172.21.15.46:0/2621551420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800ba00 con 0x7f9814071f10 2024-04-26T23:42:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98140d0300 con 0x7f9814071f10 2024-04-26T23:42:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.195+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9814144bd0 con 0x7f9814071f10 2024-04-26T23:42:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.197+0000 7f9811600640 1 -- 172.21.15.46:0/2621551420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f980800bba0 con 0x7f9814071f10 2024-04-26T23:42:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.198+0000 7f9811600640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.198+0000 7f9811600640 1 -- 172.21.15.46:0/2621551420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f980808c120 con 0x7f9814071f10 2024-04-26T23:42:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.199+0000 7f9818c00640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.199+0000 7f981ed9a740 1 Processor -- start 2024-04-26T23:42:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 -- start start 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f9818c00640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 0x7f97f406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9800034000 tx=0x7f980002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2ac82f50 0x55dd2ac86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd2abebc70 con 0x55dd2acb02f0 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dd2aba4980 con 0x55dd2ac82f50 2024-04-26T23:42:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.200+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55dd2abe7c30 con 0x55dd2acb06d0 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39926/0 (socket says 172.21.15.46:39926) 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 -- 172.21.15.46:0/2606691222 learned_addr learned my addr 172.21.15.46:0/2606691222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9818c00640 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 -- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 msgr2=0x55dd2acb3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acb3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 -- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2ac82f50 msgr2=0x55dd2ac86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2ac82f50 0x55dd2ac86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 -- 172.21.15.46:0/2606691222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd2ab7c7b0 con 0x55dd2acb02f0 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.201+0000 7f9819600640 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f98080622d0 tx=0x7f9808059640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761f9f7753c00074 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.202+0000 7f9805c00640 1 -- 172.21.15.46:0/2606691222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080721c0 con 0x55dd2acb02f0 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.202+0000 7f9805c00640 1 -- 172.21.15.46:0/2606691222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808062800 con 0x55dd2acb02f0 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.202+0000 7f9805c00640 1 -- 172.21.15.46:0/2606691222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808056290 con 0x55dd2acb02f0 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.202+0000 7f981ed9a740 1 -- 172.21.15.46:0/2606691222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 msgr2=0x55dd2acb2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.202+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f98080622d0 tx=0x7f9808059640 comp rx=0 tx=0).stop 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 -- 172.21.15.46:0/2606691222 shutdown_connections 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2ac82f50 0x55dd2ac86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acb3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2606691222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb02f0 0x55dd2acb2b00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 -- 172.21.15.46:0/2606691222 >> 172.21.15.46:0/2606691222 conn(0x55dd2ac851c0 msgr2=0x55dd2accdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 -- 172.21.15.46:0/2606691222 shutdown_connections 2024-04-26T23:42:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 -- 172.21.15.46:0/2606691222 wait complete. 2024-04-26T23:42:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.203+0000 7f981ed9a740 1 -- start start 2024-04-26T23:42:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.204+0000 7f981ed9a740 1 Processor -- start 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2acd0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acd0c40 0x55dd2acb5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd2abeaf80 con 0x55dd2acb06d0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dd2ab98260 con 0x55dd2acb02f0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55dd2abf02a0 con 0x55dd2acd0c40 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981ed9a740 10 client.0 _ll_register_callbacks cb 0x55dd2abe2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f9818c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f9818c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39936/0 (socket says 172.21.15.46:39936) 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f9818c00640 1 -- 172.21.15.46:0/1145016367 learned_addr learned my addr 172.21.15.46:0/1145016367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f9819600640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2acd0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.205+0000 7f981a000640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acd0c40 0x55dd2acb5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 -- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acd0c40 msgr2=0x55dd2acb5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acd0c40 0x55dd2acb5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 -- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 msgr2=0x55dd2acd0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2acd0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 -- 172.21.15.46:0/1145016367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd2ab7a1c0 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f9818c00640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9800009960 tx=0x7f980002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa10879cbe305d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f97fa000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980004d060 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f980003cb70 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.206+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808062800 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9800053050 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 10 client.6066 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55dd2a8fd3f0 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808062800 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f9800056020 con 0x55dd2acb06d0 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 10 client.6066 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 10 client.6066 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 20 client.6066 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2acdd850 0x55dd2acddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 4 client.6066 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.207+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd2abf9880 con 0x55dd2acdd850 2024-04-26T23:42:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.208+0000 7f9819600640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2acdd850 0x55dd2acddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.208+0000 7f9819600640 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2acdd850 0x55dd2acddc80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96b5eb98a851981 server_cookie=b32395db7dfb68e4 in_seq=0 out_seq=0 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.208+0000 7f97fa000640 10 client.6066 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f97fa000640 1 -- 172.21.15.46:0/1145016367 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd2abf9880 con 0x55dd2acdd850 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f97fa000640 10 client.6066 handle_command_reply: tid=0 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 client.6066 shutdown 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 msgr2=0x55dd2acb4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9800009960 tx=0x7f980002c070 comp rx=0 tx=0).stop 2024-04-26T23:42:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 shutdown_connections 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acd0c40 0x55dd2acb5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acb06d0 0x55dd2acb4f50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2acdd850 0x55dd2acddc80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 --2- 172.21.15.46:0/1145016367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2acd0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 >> 172.21.15.46:0/1145016367 conn(0x55dd2ac82f50 msgr2=0x55dd2ac87b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f97fa000640 0 client.6066 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.209+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 shutdown_connections 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.210+0000 7f981ed9a740 1 -- 172.21.15.46:0/1145016367 wait complete. 2024-04-26T23:42:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.210+0000 7f981ed9a740 20 client.6066 trim_cache size 0 max 16384 2024-04-26T23:42:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.225+0000 7f981ed9a740 1 Processor -- start 2024-04-26T23:42:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.225+0000 7f981ed9a740 1 -- start start 2024-04-26T23:42:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2ac86950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acdd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9814201e20 con 0x55dd2acd0c40 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9814200e50 con 0x55dd2acb02f0 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9814153370 con 0x55dd2acb06d0 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39942/0 (socket says 172.21.15.46:39942) 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 -- 172.21.15.46:0/2898175632 learned_addr learned my addr 172.21.15.46:0/2898175632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f9818c00640 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acdd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f9819600640 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2ac86950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 -- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 msgr2=0x55dd2acdd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acdd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 -- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 msgr2=0x55dd2ac86950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2ac86950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.226+0000 7f981a000640 1 -- 172.21.15.46:0/2898175632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd2acb0120 con 0x55dd2acd0c40 2024-04-26T23:42:21.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.227+0000 7f981a000640 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f980c008f60 tx=0x7f980c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeeec92f622590b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.227+0000 7f9805200640 1 -- 172.21.15.46:0/2898175632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c00b6e0 con 0x55dd2acd0c40 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.227+0000 7f9805200640 1 -- 172.21.15.46:0/2898175632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c00b880 con 0x55dd2acd0c40 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.227+0000 7f9805200640 1 -- 172.21.15.46:0/2898175632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c00bc30 con 0x55dd2acd0c40 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 -- 172.21.15.46:0/2898175632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 msgr2=0x55dd2acddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f980c008f60 tx=0x7f980c008f90 comp rx=0 tx=0).stop 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 -- 172.21.15.46:0/2898175632 shutdown_connections 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2acddbd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb06d0 0x55dd2acdd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 --2- 172.21.15.46:0/2898175632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb02f0 0x55dd2ac86950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 -- 172.21.15.46:0/2898175632 >> 172.21.15.46:0/2898175632 conn(0x55dd2ac82f50 msgr2=0x55dd2ac7e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 -- 172.21.15.46:0/2898175632 shutdown_connections 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.228+0000 7f981ed9a740 1 -- 172.21.15.46:0/2898175632 wait complete. 2024-04-26T23:42:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.229+0000 7f981ed9a740 1 -- start start 2024-04-26T23:42:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.229+0000 7f981ed9a740 1 Processor -- start 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb02f0 0x55dd2acb6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb06d0 0x55dd2ad53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98141539c0 con 0x55dd2acd0c40 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dd2accd3a0 con 0x55dd2acb06d0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55dd2acdcba0 con 0x55dd2acb02f0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981ed9a740 10 client.0 _ll_register_callbacks cb 0x7f9800035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39956/0 (socket says 172.21.15.46:39956) 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f981a000640 1 -- 172.21.15.46:0/3996626454 learned_addr learned my addr 172.21.15.46:0/3996626454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f9819600640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb02f0 0x55dd2acb6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.230+0000 7f9818c00640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb06d0 0x55dd2ad53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 -- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb02f0 msgr2=0x55dd2acb6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb02f0 0x55dd2acb6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 -- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb06d0 msgr2=0x55dd2ad53310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb06d0 0x55dd2ad53310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 -- 172.21.15.46:0/3996626454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd2acd61d0 con 0x55dd2acd0c40 2024-04-26T23:42:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981a000640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f980c00bc40 tx=0x7f980c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd50e4c61e768443 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f97fa000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c024070 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c00b880 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.231+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f980c020640 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f980c020840 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 10 client.6072 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55dd2a9c77d0 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c00b880 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f980c015070 con 0x55dd2acd0c40 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 10 client.6072 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 10 client.6072 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 20 client.6072 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2ace6a70 0x55dd2acc3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 4 client.6072 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd2abf36f0 con 0x55dd2ace6a70 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.232+0000 7f9819600640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2ace6a70 0x55dd2acc3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f9819600640 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2ace6a70 0x55dd2acc3490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253bb993d33ac8a0 server_cookie=7bfc73a086265551 in_seq=0 out_seq=0 2024-04-26T23:42:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f97fa000640 10 client.6072 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f97fa000640 1 -- 172.21.15.46:0/3996626454 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd2abf36f0 con 0x55dd2ace6a70 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f97fa000640 10 client.6072 handle_command_reply: tid=0 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 client.6072 shutdown 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 msgr2=0x55dd2ace6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f980c00bc40 tx=0x7f980c004ab0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 shutdown_connections 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2acd0c40 0x55dd2ace6530 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.233+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55dd2ace6a70 0x55dd2acc3490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd2acb06d0 0x55dd2ad53310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 1 --2- 172.21.15.46:0/3996626454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd2acb02f0 0x55dd2acb6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f97fa000640 0 client.6072 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 >> 172.21.15.46:0/3996626454 conn(0x55dd2ac82f50 msgr2=0x55dd2ac86ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 shutdown_connections 2024-04-26T23:42:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 1 -- 172.21.15.46:0/3996626454 wait complete. 2024-04-26T23:42:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981ed9a740 20 client.6072 trim_cache size 0 max 16384 2024-04-26T23:42:21.241 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 msgr2=0x7f97f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 0x7f97f406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9800034000 tx=0x7f980002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 msgr2=0x7f9814144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.234+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f980800d620 tx=0x7f980800dae0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 shutdown_connections 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98141597e0 0x7f9814156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98140b5730 0x7f98140cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 --2- 172.21.15.46:0/2621551420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9814071f10 0x7f9814144690 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 >> 172.21.15.46:0/2621551420 conn(0x7f981406c980 msgr2=0x7f9814158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 shutdown_connections 2024-04-26T23:42:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.235+0000 7f981c000640 1 -- 172.21.15.46:0/2621551420 wait complete. 2024-04-26T23:42:21.259 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4c.00000000 parent 2024-04-26T23:42:21.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.366+0000 7fa94c88cf40 1 Processor -- start 2024-04-26T23:42:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.366+0000 7fa94c88cf40 1 -- start start 2024-04-26T23:42:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b0092b5a0 0x559b0092c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 0x559b009a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b00886d20 con 0x559b009275a0 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b008dd2d0 con 0x559b009b6b20 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559b009b7d20 con 0x559b0092b5a0 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa949a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa949a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39976/0 (socket says 172.21.15.46:39976) 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.367+0000 7fa949a00640 1 -- 172.21.15.46:0/3121070542 learned_addr learned my addr 172.21.15.46:0/3121070542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 -- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b0092b5a0 msgr2=0x559b0092c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b0092b5a0 0x559b0092c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa94a400640 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 0x559b009a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 -- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 msgr2=0x559b009a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 0x559b009a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 -- 172.21.15.46:0/3121070542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b00911550 con 0x559b009275a0 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa94a400640 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 0x559b009a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.368+0000 7fa949a00640 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa93c054040 tx=0x7fa93c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1742e439f45a368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa943400640 1 -- 172.21.15.46:0/3121070542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c06a070 con 0x559b009275a0 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa943400640 1 -- 172.21.15.46:0/3121070542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c05d8c0 con 0x559b009275a0 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa943400640 1 -- 172.21.15.46:0/3121070542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c05dcb0 con 0x559b009275a0 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 -- 172.21.15.46:0/3121070542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 msgr2=0x559b009ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa93c054040 tx=0x7fa93c050000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 -- 172.21.15.46:0/3121070542 shutdown_connections 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b009b6b20 0x559b009a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b0092b5a0 0x559b0092c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.369+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/3121070542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009ad700 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.370+0000 7fa94c88cf40 1 -- 172.21.15.46:0/3121070542 >> 172.21.15.46:0/3121070542 conn(0x559b008e4a00 msgr2=0x559b0092ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.370+0000 7fa94c88cf40 1 -- 172.21.15.46:0/3121070542 shutdown_connections 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.370+0000 7fa94c88cf40 1 -- 172.21.15.46:0/3121070542 wait complete. 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.371+0000 7fa94c88cf40 1 Processor -- start 2024-04-26T23:42:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.371+0000 7fa94c88cf40 1 -- start start 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b0092b5a0 0x559b009ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b009b6b20 0x559b00921e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b009b76d0 con 0x559b009275a0 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b00911830 con 0x559b0092b5a0 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa94c88cf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559b009b79b0 con 0x559b009b6b20 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa949a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa949a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009a2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39988/0 (socket says 172.21.15.46:39988) 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.372+0000 7fa949a00640 1 -- 172.21.15.46:0/4092171548 learned_addr learned my addr 172.21.15.46:0/4092171548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 -- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b009b6b20 msgr2=0x559b00921e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b009b6b20 0x559b00921e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949000640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b0092b5a0 0x559b009ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 -- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b0092b5a0 msgr2=0x559b009ba4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b0092b5a0 0x559b009ba4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 -- 172.21.15.46:0/4092171548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b009a5810 con 0x559b009275a0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa949a00640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009a2d60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0259a0 tx=0x7fa93c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa942000640 1 -- 172.21.15.46:0/4092171548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c075020 con 0x559b009275a0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa942000640 1 -- 172.21.15.46:0/4092171548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c06b040 con 0x559b009275a0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa942000640 1 -- 172.21.15.46:0/4092171548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c06a050 con 0x559b009275a0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.373+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b009a36e0 con 0x559b009275a0 2024-04-26T23:42:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.374+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b009a5530 con 0x559b009275a0 2024-04-26T23:42:21.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.376+0000 7fa942000640 1 -- 172.21.15.46:0/4092171548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa93c078070 con 0x559b009275a0 2024-04-26T23:42:21.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa942000640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06b300 0x7fa92c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa942000640 1 -- 172.21.15.46:0/4092171548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa93c0cfa50 con 0x559b009275a0 2024-04-26T23:42:21.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x559b009baa00 0x559b009228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- osd_op(unknown.0.0:1 8.3a 8:5fa3b718:::10000000a4c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) -- 0x559b009241c0 con 0x559b009baa00 2024-04-26T23:42:21.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa949000640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06b300 0x7fa92c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.377+0000 7fa94a400640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x559b009baa00 0x559b009228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.378+0000 7fa94a400640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x559b009baa00 0x559b009228a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.379+0000 7fa949000640 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06b300 0x7fa92c06d7c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559b009112e0 tx=0x7fa9380054c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94a400640 1 -- 172.21.15.46:0/4092171548 <== osd.1 v2:172.21.15.46:6816/3879162742 1 ==== osd_op_reply(1 10000000a4c.00000000 [rmxattr] v54'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa94400e330 con 0x559b009baa00 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x559b009baa00 msgr2=0x559b009228a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x559b009baa00 0x559b009228a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06b300 msgr2=0x7fa92c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06b300 0x7fa92c06d7c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559b009112e0 tx=0x7fa9380054c0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 msgr2=0x559b009a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 --2- 172.21.15.46:0/4092171548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b009275a0 0x559b009a2d60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0259a0 tx=0x7fa93c003330 comp rx=0 tx=0).stop 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94a400640 1 -- 172.21.15.46:0/4092171548 reap_dead start 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 shutdown_connections 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 >> 172.21.15.46:0/4092171548 conn(0x559b008e4a00 msgr2=0x559b008e4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 shutdown_connections 2024-04-26T23:42:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.381+0000 7fa94c88cf40 1 -- 172.21.15.46:0/4092171548 wait complete. 2024-04-26T23:42:21.390 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-26T23:42:21.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-26T23:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.587+0000 7f8e93000640 1 Processor -- start 2024-04-26T23:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.587+0000 7f8e93000640 1 -- start start 2024-04-26T23:42:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e93000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e8c15cec0 0x7f8e8c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e93000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8c0ce7d0 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e93000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8c0ceab0 con 0x7f8e8c073d00 2024-04-26T23:42:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e93000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8c0ced90 con 0x7f8e8c15cec0 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e91c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e8c15cec0 0x7f8e8c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e91200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e8c0cdb30 0x7f8e8c0735f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36716/0 (socket says 172.21.15.46:36716) 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.589+0000 7f8e91200640 1 -- 172.21.15.46:0/3557282284 learned_addr learned my addr 172.21.15.46:0/3557282284 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.590+0000 7f8e91c00640 1 -- 172.21.15.46:0/3557282284 >> v1:172.21.15.142:6789/0 conn(0x7f8e8c073d00 legacy=0x7f8e8c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.590+0000 7f8e91c00640 1 -- 172.21.15.46:0/3557282284 >> v1:172.21.15.46:6789/0 conn(0x7f8e8c0cdb30 legacy=0x7f8e8c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.590+0000 7f8e91c00640 1 -- 172.21.15.46:0/3557282284 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c0cf070 con 0x7f8e8c15cec0 2024-04-26T23:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.591+0000 7f8e91c00640 1 --2- 172.21.15.46:0/3557282284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c0cd5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31177639358e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.591+0000 7f8e8be00640 1 -- 172.21.15.46:0/3557282284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80037070 con 0x7f8e8c15cec0 2024-04-26T23:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.591+0000 7f8e8be00640 1 -- 172.21.15.46:0/3557282284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80032440 con 0x7f8e8c15cec0 2024-04-26T23:42:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.591+0000 7f8e8be00640 1 -- 172.21.15.46:0/3557282284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80032810 con 0x7f8e8c15cec0 2024-04-26T23:42:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 -- 172.21.15.46:0/3557282284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15cec0 msgr2=0x7f8e8c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 --2- 172.21.15.46:0/3557282284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c0cd5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 -- 172.21.15.46:0/3557282284 shutdown_connections 2024-04-26T23:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 --2- 172.21.15.46:0/3557282284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c0cd5f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 -- 172.21.15.46:0/3557282284 >> 172.21.15.46:0/3557282284 conn(0x7f8e8c0b27b0 msgr2=0x7f8e8c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 -- 172.21.15.46:0/3557282284 shutdown_connections 2024-04-26T23:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.592+0000 7f8e93000640 1 -- 172.21.15.46:0/3557282284 wait complete. 2024-04-26T23:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.594+0000 7f8e93000640 1 Processor -- start 2024-04-26T23:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.594+0000 7f8e93000640 1 -- start start 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 0x7f8e8c1f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c0b4250 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c15d2a0 con 0x7f8e8c15cec0 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.595+0000 7f8e93000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c0b43d0 con 0x7f8e8c073d00 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39994/0 (socket says 172.21.15.46:39994) 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 -- 172.21.15.46:0/2830560420 learned_addr learned my addr 172.21.15.46:0/2830560420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e92600640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 0x7f8e8c1f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 -- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 msgr2=0x7f8e8c1f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 0x7f8e8c1f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 -- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 msgr2=0x7f8e8c159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 -- 172.21.15.46:0/2830560420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1f2e30 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 0x7f8e8c1f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e92600640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.596+0000 7f8e91200640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c400 tx=0x7f8e7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.597+0000 7f8e89600640 1 -- 172.21.15.46:0/2830560420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78017070 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.597+0000 7f8e89600640 1 -- 172.21.15.46:0/2830560420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e78005ce0 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.597+0000 7f8e89600640 1 -- 172.21.15.46:0/2830560420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e780072f0 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.597+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1f3110 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.597+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1f36d0 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.599+0000 7f8e89600640 1 -- 172.21.15.46:0/2830560420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e7801a070 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.600+0000 7f8e89600640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 0x7f8e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.600+0000 7f8e89600640 1 -- 172.21.15.46:0/2830560420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8e7808b100 con 0x7f8e8c0cdb30 2024-04-26T23:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.601+0000 7f8e97470740 1 Processor -- start 2024-04-26T23:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.601+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 0x7f8e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.601+0000 7f8e97470740 1 -- start start 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e8d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563b5e811e10 con 0x563b5e8d88e0 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563b5e7ae120 con 0x563b5e8d9020 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563b5e80ddd0 con 0x563b5e8f5f00 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 0x7f8e6c06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034000 tx=0x7f8e8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39996/0 (socket says 172.21.15.46:39996) 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.602+0000 7f8e92600640 1 -- 172.21.15.46:0/3936770242 learned_addr learned my addr 172.21.15.46:0/3936770242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e91c00640 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e91200640 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e8d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 -- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 msgr2=0x563b5e8a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 -- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 msgr2=0x563b5e8d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e8d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 -- 172.21.15.46:0/3936770242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5e5efc00 con 0x563b5e8d88e0 2024-04-26T23:42:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e92600640 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8e840045d0 tx=0x7f8e8400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52ef1b7a54630b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e7e000640 1 -- 172.21.15.46:0/3936770242 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84006b50 con 0x563b5e8d88e0 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e7e000640 1 -- 172.21.15.46:0/3936770242 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e84006cf0 con 0x563b5e8d88e0 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.603+0000 7f8e7e000640 1 -- 172.21.15.46:0/3936770242 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004a20 con 0x563b5e8d88e0 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 -- 172.21.15.46:0/3936770242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 msgr2=0x563b5e8ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8e840045d0 tx=0x7f8e8400d520 comp rx=0 tx=0).stop 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 -- 172.21.15.46:0/3936770242 shutdown_connections 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d88e0 0x563b5e8ad6f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e8d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 --2- 172.21.15.46:0/3936770242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 -- 172.21.15.46:0/3936770242 >> 172.21.15.46:0/3936770242 conn(0x563b5e8aab00 msgr2=0x563b5e8f33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 -- 172.21.15.46:0/3936770242 shutdown_connections 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.604+0000 7f8e97470740 1 -- 172.21.15.46:0/3936770242 wait complete. 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.605+0000 7f8e97470740 1 -- start start 2024-04-26T23:42:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.605+0000 7f8e97470740 1 Processor -- start 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e9030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563b5e8accb0 con 0x563b5e8ea250 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563b5e798c40 con 0x563b5e8d9020 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563b5e8ac0a0 con 0x563b5e8f5f00 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.606+0000 7f8e97470740 10 client.0 _ll_register_callbacks cb 0x563b5e7b9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40008/0 (socket says 172.21.15.46:40008) 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 -- 172.21.15.46:0/2783863080 learned_addr learned my addr 172.21.15.46:0/2783863080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e9030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e91200640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 -- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 msgr2=0x563b5e8f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 -- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 msgr2=0x563b5e9030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e9030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.607+0000 7f8e92600640 1 -- 172.21.15.46:0/2783863080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5e4e34f0 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e92600640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8e84007000 tx=0x7f8e84003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da00fb63850b75c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84006b50 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8e840078b0 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e84006cf0 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e8402d050 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e97470740 10 client.6096 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x563b5e650390 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.608+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e84006cf0 con 0x563b5e8ea250 2024-04-26T23:42:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f8e84030020 con 0x563b5e8ea250 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 10 client.6096 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 20 client.6096 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8d99e0 0x563b5e8d9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 4 client.6096 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b5e81fa20 con 0x563b5e8d99e0 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.609+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8d99e0 0x563b5e8d9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.610+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8d99e0 0x563b5e8d9df0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81d3d794eec0dbb server_cookie=5de3ca3db45796d4 in_seq=0 out_seq=0 2024-04-26T23:42:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.610+0000 7f8e72a00640 10 client.6096 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.610+0000 7f8e72a00640 1 -- 172.21.15.46:0/2783863080 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b5e81fa20 con 0x563b5e8d99e0 2024-04-26T23:42:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.610+0000 7f8e72a00640 10 client.6096 handle_command_reply: tid=0 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.610+0000 7f8e97470740 1 client.6096 shutdown 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 msgr2=0x563b5e8f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8e84007000 tx=0x7f8e84003a70 comp rx=0 tx=0).stop 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 shutdown_connections 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8ea250 0x563b5e8f4c30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8d99e0 0x563b5e8d9df0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 --2- 172.21.15.46:0/2783863080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8d9020 0x563b5e9030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 >> 172.21.15.46:0/2783863080 conn(0x563b5e8d88e0 msgr2=0x563b5e8f67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e72a00640 0 client.6096 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 shutdown_connections 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 1 -- 172.21.15.46:0/2783863080 wait complete. 2024-04-26T23:42:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.611+0000 7f8e97470740 20 client.6096 trim_cache size 0 max 16384 2024-04-26T23:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.631+0000 7f8e97470740 1 Processor -- start 2024-04-26T23:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.631+0000 7f8e97470740 1 -- start start 2024-04-26T23:42:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e8f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1ffb20 con 0x563b5e8d9020 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1feb40 con 0x563b5e8ea250 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c1fded0 con 0x563b5e8f5f00 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40010/0 (socket says 172.21.15.46:40010) 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 -- 172.21.15.46:0/2670270496 learned_addr learned my addr 172.21.15.46:0/2670270496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 -- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 msgr2=0x563b5e8f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 -- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 msgr2=0x563b5e8f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e8f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.632+0000 7f8e91c00640 1 -- 172.21.15.46:0/2670270496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5e4bc670 con 0x563b5e8d9020 2024-04-26T23:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002a7d0 tx=0x7f8e8002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fa7345fb65de2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e7d600640 1 -- 172.21.15.46:0/2670270496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80037040 con 0x563b5e8d9020 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e7d600640 1 -- 172.21.15.46:0/2670270496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80052070 con 0x563b5e8d9020 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e7d600640 1 -- 172.21.15.46:0/2670270496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8004c590 con 0x563b5e8d9020 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e97470740 1 -- 172.21.15.46:0/2670270496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 msgr2=0x563b5e8adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.633+0000 7f8e97470740 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002a7d0 tx=0x7f8e8002c000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 -- 172.21.15.46:0/2670270496 shutdown_connections 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8f5f00 0x563b5e8f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e8f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 --2- 172.21.15.46:0/2670270496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8d9020 0x563b5e8adf20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 -- 172.21.15.46:0/2670270496 >> 172.21.15.46:0/2670270496 conn(0x563b5e8d88e0 msgr2=0x563b5e8ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 -- 172.21.15.46:0/2670270496 shutdown_connections 2024-04-26T23:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 -- 172.21.15.46:0/2670270496 wait complete. 2024-04-26T23:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.634+0000 7f8e97470740 1 -- start start 2024-04-26T23:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.635+0000 7f8e97470740 1 Processor -- start 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8d9020 0x563b5e8f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e973100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1fe4f0 con 0x563b5e8f5f00 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563b5e916ee0 con 0x563b5e8ea250 2024-04-26T23:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563b5e8f2910 con 0x563b5e8d9020 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e97470740 10 client.0 _ll_register_callbacks cb 0x7f8e840043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40012/0 (socket says 172.21.15.46:40012) 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 -- 172.21.15.46:0/2552786687 learned_addr learned my addr 172.21.15.46:0/2552786687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 -- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8d9020 msgr2=0x563b5e8f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8d9020 0x563b5e8f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 -- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 msgr2=0x563b5e973100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e973100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.636+0000 7f8e92600640 1 -- 172.21.15.46:0/2552786687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5e8f2700 con 0x563b5e8f5f00 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e92600640 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400d560 tx=0x7f8e84009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a847902d43796dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e840047e0 con 0x563b5e8f5f00 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8e84037aa0 con 0x563b5e8f5f00 2024-04-26T23:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e80052070 con 0x563b5e8f5f00 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e84006d60 con 0x563b5e8f5f00 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e97470740 10 client.6102 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x563b5e8ae400 con 0x563b5e8f5f00 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.637+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80052070 con 0x563b5e8f5f00 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f8e84044070 con 0x563b5e8f5f00 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 10 client.6102 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 20 client.6102 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8ec540 0x563b5e8db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 4 client.6102 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) -- 0x563b5e819890 con 0x563b5e8ec540 2024-04-26T23:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8ec540 0x563b5e8db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e91c00640 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8ec540 0x563b5e8db140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87b10aa4d1de1cd server_cookie=bcab025f8ba85cd2 in_seq=0 out_seq=0 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.638+0000 7f8e72a00640 10 client.6102 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e72a00640 1 -- 172.21.15.46:0/2552786687 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b5e819890 con 0x563b5e8ec540 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e72a00640 10 client.6102 handle_command_reply: tid=0 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 client.6102 shutdown 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 msgr2=0x563b5e973640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400d560 tx=0x7f8e84009040 comp rx=0 tx=0).stop 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 shutdown_connections 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b5e8f5f00 0x563b5e973640 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x563b5e8ec540 0x563b5e8db140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b5e8ea250 0x563b5e973100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 --2- 172.21.15.46:0/2552786687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563b5e8d9020 0x563b5e8f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e72a00640 0 client.6102 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 >> 172.21.15.46:0/2552786687 conn(0x563b5e8d88e0 msgr2=0x563b5e8f4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.647 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:42:21.639+0000 7f45aaa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v453 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4967=0-4194304@1} caps={4967=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4967 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cdae700100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4c:[<0x10000000001/new_file_0 v453>,<0x10000000000/ceph-qa-suite v63>,<0x1/scrub_test_path v26>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-26T23:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 shutdown_connections 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e97470740 1 -- 172.21.15.46:0/2552786687 wait complete. 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e97470740 20 client.6102 trim_cache size 0 max 16384 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "74a99dee-f03c-49d3-b5fc-51c46aafb488", 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:21.648 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 msgr2=0x7f8e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 0x7f8e6c06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034000 tx=0x7f8e8002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 msgr2=0x7f8e8c156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.640+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c400 tx=0x7f8e7800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 shutdown_connections 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e6c06c8f0 0x7f8e6c06edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c15cec0 0x7f8e8c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cdb30 0x7f8e8c156d60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 --2- 172.21.15.46:0/2830560420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e8c073d00 0x7f8e8c1f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 >> 172.21.15.46:0/2830560420 conn(0x7f8e8c0b27b0 msgr2=0x7f8e8c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 shutdown_connections 2024-04-26T23:42:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.641+0000 7f8e93000640 1 -- 172.21.15.46:0/2830560420 wait complete. 2024-04-26T23:42:21.661 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '74a99dee-f03c-49d3-b5fc-51c46aafb488', 'mode': 'asynchronous'}' 2024-04-26T23:42:21.662 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:42:21.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-26T23:42:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.882+0000 7fc867800640 1 Processor -- start 2024-04-26T23:42:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.882+0000 7fc867800640 1 -- start start 2024-04-26T23:42:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc867800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc8600d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc867800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8600d5b90 0x7fc8600d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc867800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8600b46f0 con 0x7fc8600b54b0 2024-04-26T23:42:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc867800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc8600b44a0 con 0x7fc8600d5b90 2024-04-26T23:42:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc867800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600b5020 con 0x7fc860162aa0 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.883+0000 7fc857e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8600d5b90 0x7fc8600d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8600d5b90 0x7fc8600d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32844/0 (socket says 172.21.15.46:32844) 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 -- 172.21.15.46:0/2931650279 learned_addr learned my addr 172.21.15.46:0/2931650279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc864e00640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc8600d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 -- 172.21.15.46:0/2931650279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 msgr2=0x7fc8600d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc8600d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 -- 172.21.15.46:0/2931650279 >> v1:172.21.15.142:6789/0 conn(0x7fc8600b54b0 legacy=0x7fc8600b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.884+0000 7fc857e00640 1 -- 172.21.15.46:0/2931650279 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600b49d0 con 0x7fc8600d5b90 2024-04-26T23:42:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.885+0000 7fc857e00640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600d5b90 0x7fc8600d3b90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec9d1364c8c049 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.885+0000 7fc85fe00640 1 -- 172.21.15.46:0/2931650279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848037070 con 0x7fc8600d5b90 2024-04-26T23:42:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc85fe00640 1 -- 172.21.15.46:0/2931650279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848032440 con 0x7fc8600d5b90 2024-04-26T23:42:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc85fe00640 1 -- 172.21.15.46:0/2931650279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848032810 con 0x7fc8600d5b90 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 -- 172.21.15.46:0/2931650279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600d5b90 msgr2=0x7fc8600d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600d5b90 0x7fc8600d3b90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802a180 comp rx=0 tx=0).stop 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 -- 172.21.15.46:0/2931650279 shutdown_connections 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600d5b90 0x7fc8600d3b90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 --2- 172.21.15.46:0/2931650279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc8600d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 -- 172.21.15.46:0/2931650279 >> 172.21.15.46:0/2931650279 conn(0x7fc8600d5070 msgr2=0x7fc8600d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 -- 172.21.15.46:0/2931650279 shutdown_connections 2024-04-26T23:42:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.886+0000 7fc867800640 1 -- 172.21.15.46:0/2931650279 wait complete. 2024-04-26T23:42:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.888+0000 7fc867800640 1 Processor -- start 2024-04-26T23:42:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.888+0000 7fc867800640 1 -- start start 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600d5b90 0x7fc8600c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc860154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8600d30f0 con 0x7fc8600b54b0 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc860162e80 con 0x7fc8600d5b90 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc867800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600d0c90 con 0x7fc860162aa0 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.889+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32848/0 (socket says 172.21.15.46:32848) 2024-04-26T23:42:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 -- 172.21.15.46:0/3349409181 learned_addr learned my addr 172.21.15.46:0/3349409181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc865800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc860154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 -- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 msgr2=0x7fc860154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc857e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600d5b90 0x7fc8600c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc860154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 -- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600d5b90 msgr2=0x7fc8600c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600d5b90 0x7fc8600c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 -- 172.21.15.46:0/3349409181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600c9e00 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc865800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc860154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.890+0000 7fc864e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d620 tx=0x7fc85000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.891+0000 7fc85d600640 1 -- 172.21.15.46:0/3349409181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000b620 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.891+0000 7fc85d600640 1 -- 172.21.15.46:0/3349409181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc850019070 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.891+0000 7fc85d600640 1 -- 172.21.15.46:0/3349409181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000ba00 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.891+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8600ca0e0 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.891+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8600ca6a0 con 0x7fc8600b54b0 2024-04-26T23:42:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.893+0000 7fc85d600640 1 -- 172.21.15.46:0/3349409181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc85000bba0 con 0x7fc8600b54b0 2024-04-26T23:42:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.894+0000 7fc85d600640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 0x7fc83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.894+0000 7fc85d600640 1 -- 172.21.15.46:0/3349409181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc85008c1b0 con 0x7fc8600b54b0 2024-04-26T23:42:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.895+0000 7fc86a62e740 1 Processor -- start 2024-04-26T23:42:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.895+0000 7fc857e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 0x7fc83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.895+0000 7fc86a62e740 1 -- start start 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5b9ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9a4220 0x561e5b9ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e5b8b4030 con 0x561e5b9ebde0 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561e5b8cf7f0 con 0x561e5b9ce540 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561e5b88a7e0 con 0x561e5b9a4220 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32850/0 (socket says 172.21.15.46:32850) 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 -- 172.21.15.46:0/3370261553 learned_addr learned my addr 172.21.15.46:0/3370261553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 0x7fc83c06edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc848034000 tx=0x7fc84802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc864e00640 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5b9ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 -- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9a4220 msgr2=0x561e5b9ea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9a4220 0x561e5b9ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 -- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 msgr2=0x561e5b9ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5b9ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.896+0000 7fc857e00640 1 -- 172.21.15.46:0/3370261553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5b896050 con 0x561e5b9ebde0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.897+0000 7fc857e00640 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc8480090b0 tx=0x7fc848009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f2b9ebcb4e91ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.897+0000 7fc855600640 1 -- 172.21.15.46:0/3370261553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84804d070 con 0x561e5b9ebde0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.897+0000 7fc855600640 1 -- 172.21.15.46:0/3370261553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84803c460 con 0x561e5b9ebde0 2024-04-26T23:42:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.897+0000 7fc855600640 1 -- 172.21.15.46:0/3370261553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848037040 con 0x561e5b9ebde0 2024-04-26T23:42:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 -- 172.21.15.46:0/3370261553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 msgr2=0x561e5b9ec330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc8480090b0 tx=0x7fc848009240 comp rx=0 tx=0).stop 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 -- 172.21.15.46:0/3370261553 shutdown_connections 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9a4220 0x561e5b9ea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ebde0 0x561e5b9ec330 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3370261553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5b9ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 -- 172.21.15.46:0/3370261553 >> 172.21.15.46:0/3370261553 conn(0x561e5b9a0b50 msgr2=0x561e5b9e9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 -- 172.21.15.46:0/3370261553 shutdown_connections 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.898+0000 7fc86a62e740 1 -- 172.21.15.46:0/3370261553 wait complete. 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.899+0000 7fc86a62e740 1 -- start start 2024-04-26T23:42:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.899+0000 7fc86a62e740 1 Processor -- start 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 0x561e5b9f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e5b90c200 con 0x561e5b9ce540 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561e5b906ee0 con 0x561e5b9e8b30 2024-04-26T23:42:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561e5b913e70 con 0x561e5b9ebde0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc86a62e740 10 client.0 _ll_register_callbacks cb 0x561e5b8fd270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32856/0 (socket says 172.21.15.46:32856) 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc864e00640 1 -- 172.21.15.46:0/1173412770 learned_addr learned my addr 172.21.15.46:0/1173412770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc865800640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 0x561e5b9f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.900+0000 7fc857e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 -- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 msgr2=0x561e5b9ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 -- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 msgr2=0x561e5b9f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 0x561e5b9f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 -- 172.21.15.46:0/1173412770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5b6e37d0 con 0x561e5b9ce540 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc865800640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 0x561e5b9f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc864e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc85008a450 tx=0x7fc85005a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ff77da3630f112 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc84e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850072020 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc850056320 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84803c460 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc850062890 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc86a62e740 10 client.6114 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.901+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x561e5b6193f0 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84803c460 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fc850013070 con 0x561e5b9ce540 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 10 client.6114 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 10 client.6114 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 20 client.6114 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b9c7440 0x561e5b9f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 4 client.6114 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5b9157e0 con 0x561e5b9c7440 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.902+0000 7fc857e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b9c7440 0x561e5b9f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.903+0000 7fc857e00640 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b9c7440 0x561e5b9f9e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6600a8fbd2ab3c12 server_cookie=e63f5d5efb1c58c6 in_seq=0 out_seq=0 2024-04-26T23:42:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.903+0000 7fc84e000640 10 client.6114 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.903+0000 7fc84e000640 1 -- 172.21.15.46:0/1173412770 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e5b9157e0 con 0x561e5b9c7440 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.903+0000 7fc84e000640 10 client.6114 handle_command_reply: tid=0 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 client.6114 shutdown 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 msgr2=0x561e5b9ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc85008a450 tx=0x7fc85005a470 comp rx=0 tx=0).stop 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 shutdown_connections 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b9c7440 0x561e5b9f9e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9e8b30 0x561e5b9f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 --2- 172.21.15.46:0/1173412770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9ea2d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc84e000640 0 client.6114 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 >> 172.21.15.46:0/1173412770 conn(0x561e5b9a4220 msgr2=0x561e5b9a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 shutdown_connections 2024-04-26T23:42:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 1 -- 172.21.15.46:0/1173412770 wait complete. 2024-04-26T23:42:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.904+0000 7fc86a62e740 20 client.6114 trim_cache size 0 max 16384 2024-04-26T23:42:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.923+0000 7fc86a62e740 1 Processor -- start 2024-04-26T23:42:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 -- start start 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5ba0d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc860201f30 con 0x561e5b9e8b30 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc860200fa0 con 0x561e5b9ce540 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.924+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600cd5d0 con 0x561e5b9ebde0 2024-04-26T23:42:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32868/0 (socket says 172.21.15.46:32868) 2024-04-26T23:42:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 -- 172.21.15.46:0/3154027315 learned_addr learned my addr 172.21.15.46:0/3154027315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 -- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 msgr2=0x561e5b9f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 -- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 msgr2=0x561e5ba0d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5ba0d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 -- 172.21.15.46:0/3154027315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5b5d90f0 con 0x561e5b9e8b30 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.925+0000 7fc857e00640 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc84802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b5edc795442910 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc854c00640 1 -- 172.21.15.46:0/3154027315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84804d040 con 0x561e5b9e8b30 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc854c00640 1 -- 172.21.15.46:0/3154027315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848003610 con 0x561e5b9e8b30 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc854c00640 1 -- 172.21.15.46:0/3154027315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848037040 con 0x561e5b9e8b30 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 -- 172.21.15.46:0/3154027315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 msgr2=0x561e5b9e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc84802c000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 -- 172.21.15.46:0/3154027315 shutdown_connections 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9ebde0 0x561e5b9f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9e8b30 0x561e5b9e9570 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 --2- 172.21.15.46:0/3154027315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ce540 0x561e5ba0d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 -- 172.21.15.46:0/3154027315 >> 172.21.15.46:0/3154027315 conn(0x561e5b9a4220 msgr2=0x561e5b9a55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 -- 172.21.15.46:0/3154027315 shutdown_connections 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.926+0000 7fc86a62e740 1 -- 172.21.15.46:0/3154027315 wait complete. 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.927+0000 7fc86a62e740 1 -- start start 2024-04-26T23:42:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.927+0000 7fc86a62e740 1 Processor -- start 2024-04-26T23:42:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9e8b30 0x561e5b9e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ebde0 0x561e5b9e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e5b914fd0 con 0x561e5b9ce540 2024-04-26T23:42:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc860200a00 con 0x561e5b9ebde0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561e5b99e660 con 0x561e5b9e8b30 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.928+0000 7fc86a62e740 10 client.0 _ll_register_callbacks cb 0x7fc85000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32882/0 (socket says 172.21.15.46:32882) 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 -- 172.21.15.46:0/2782482957 learned_addr learned my addr 172.21.15.46:0/2782482957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc857e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9e8b30 0x561e5b9e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 -- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9e8b30 msgr2=0x561e5b9e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9e8b30 0x561e5b9e9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 -- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ebde0 msgr2=0x561e5b9e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc865800640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ebde0 0x561e5b9e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ebde0 0x561e5b9e6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.929+0000 7fc864e00640 1 -- 172.21.15.46:0/2782482957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5b8cfa30 con 0x561e5b9ce540 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc864e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc850062360 tx=0x7fc850000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac07cc83b8115bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850072040 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc850073cd0 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc848003610 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc850059610 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 10 client.6120 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x561e5b9cabe0 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848003610 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fc850060260 con 0x561e5b9ce540 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 10 client.6120 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 10 client.6120 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 20 client.6120 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b99e920 0x561e5b9e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 4 client.6120 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.930+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e5b90f650 con 0x561e5b99e920 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc857e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b99e920 0x561e5b9e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc857e00640 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b99e920 0x561e5b9e7410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfad6932a4f8538c server_cookie=5f471bbd6dc1659b in_seq=0 out_seq=0 2024-04-26T23:42:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc84e000640 10 client.6120 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc84e000640 1 -- 172.21.15.46:0/2782482957 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561e5b90f650 con 0x561e5b99e920 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc84e000640 10 client.6120 handle_command_reply: tid=0 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.931+0000 7fc86a62e740 1 client.6120 shutdown 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 msgr2=0x561e5b9e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc850062360 tx=0x7fc850000bc0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 shutdown_connections 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x561e5b99e920 0x561e5b9e7410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e5b9ebde0 0x561e5b9e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561e5b9e8b30 0x561e5b9e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 --2- 172.21.15.46:0/2782482957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e5b9ce540 0x561e5b9e9570 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 >> 172.21.15.46:0/2782482957 conn(0x561e5b9a4220 msgr2=0x561e5b9f62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc84e000640 0 client.6120 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 shutdown_connections 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 1 -- 172.21.15.46:0/2782482957 wait complete. 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.932+0000 7fc86a62e740 20 client.6120 trim_cache size 0 max 16384 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:42:21.940 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 msgr2=0x7fc83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 0x7fc83c06edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc848034000 tx=0x7fc84802e000 comp rx=0 tx=0).stop 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 msgr2=0x7fc8600caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d620 tx=0x7fc85000dae0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 shutdown_connections 2024-04-26T23:42:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc83c06c8f0 0x7fc83c06edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc860162aa0 0x7fc860154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600d5b90 0x7fc8600c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 --2- 172.21.15.46:0/3349409181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8600b54b0 0x7fc8600caea0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 >> 172.21.15.46:0/3349409181 conn(0x7fc8600d5070 msgr2=0x7fc8600b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 shutdown_connections 2024-04-26T23:42:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:21.933+0000 7fc867800640 1 -- 172.21.15.46:0/3349409181 wait complete. 2024-04-26T23:42:21.954 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:42:21.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-26T23:42:22.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.144+0000 7f9b20c00640 1 Processor -- start 2024-04-26T23:42:22.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.144+0000 7f9b20c00640 1 -- start start 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b20c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b1c162a70 0x7f9b1c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b20c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b20c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1c0d3400 con 0x7f9b1c0d1ae0 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b20c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1c0d2370 con 0x7f9b1c0d3830 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b20c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1c0d2650 con 0x7f9b1c162a70 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b1aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b1aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32892/0 (socket says 172.21.15.46:32892) 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b1aa00640 1 -- 172.21.15.46:0/2957879103 learned_addr learned my addr 172.21.15.46:0/2957879103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.146+0000 7f9b1b400640 1 --2- 172.21.15.46:0/2957879103 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c162a70 0x7f9b1c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.147+0000 7f9b1aa00640 1 -- 172.21.15.46:0/2957879103 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c162a70 msgr2=0x7f9b1c0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.147+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/2957879103 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c162a70 0x7f9b1c0d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.147+0000 7f9b1aa00640 1 -- 172.21.15.46:0/2957879103 >> v1:172.21.15.142:6789/0 conn(0x7f9b1c0d1ae0 legacy=0x7f9b1c0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.147+0000 7f9b1aa00640 1 -- 172.21.15.46:0/2957879103 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c0d0df0 con 0x7f9b1c0d3830 2024-04-26T23:42:22.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.148+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/2957879103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c074b40 tx=0x7f9b0802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d2086b9f061aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.148+0000 7f9b1a000640 1 -- 172.21.15.46:0/2957879103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037070 con 0x7f9b1c0d3830 2024-04-26T23:42:22.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.148+0000 7f9b1a000640 1 -- 172.21.15.46:0/2957879103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b08032440 con 0x7f9b1c0d3830 2024-04-26T23:42:22.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.148+0000 7f9b1a000640 1 -- 172.21.15.46:0/2957879103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08032810 con 0x7f9b1c0d3830 2024-04-26T23:42:22.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 -- 172.21.15.46:0/2957879103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c0d3830 msgr2=0x7f9b1c0d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 --2- 172.21.15.46:0/2957879103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c074b40 tx=0x7f9b0802a2c0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 -- 172.21.15.46:0/2957879103 shutdown_connections 2024-04-26T23:42:22.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 --2- 172.21.15.46:0/2957879103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0d15a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 --2- 172.21.15.46:0/2957879103 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c162a70 0x7f9b1c0d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 -- 172.21.15.46:0/2957879103 >> 172.21.15.46:0/2957879103 conn(0x7f9b1c0d4f20 msgr2=0x7f9b1c0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 -- 172.21.15.46:0/2957879103 shutdown_connections 2024-04-26T23:42:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.149+0000 7f9b20c00640 1 -- 172.21.15.46:0/2957879103 wait complete. 2024-04-26T23:42:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.151+0000 7f9b20c00640 1 Processor -- start 2024-04-26T23:42:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.151+0000 7f9b20c00640 1 -- start start 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c0d1ae0 0x7f9b1c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0d2070 con 0x7f9b1c162a70 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0d1ef0 con 0x7f9b1c0d3830 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b20c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c0b39f0 con 0x7f9b1c0d1ae0 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32902/0 (socket says 172.21.15.46:32902) 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.152+0000 7f9b1be00640 1 -- 172.21.15.46:0/3369416904 learned_addr learned my addr 172.21.15.46:0/3369416904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c0d1ae0 0x7f9b1c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 -- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c0d1ae0 msgr2=0x7f9b1c154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c0d1ae0 0x7f9b1c154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 -- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 msgr2=0x7f9b1c0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 -- 172.21.15.46:0/3369416904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c0ca460 con 0x7f9b1c162a70 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b1be00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c7b0 tx=0x7f9b1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b13400640 1 -- 172.21.15.46:0/3369416904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004b60 con 0x7f9b1c162a70 2024-04-26T23:42:22.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b13400640 1 -- 172.21.15.46:0/3369416904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b14004d00 con 0x7f9b1c162a70 2024-04-26T23:42:22.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b13400640 1 -- 172.21.15.46:0/3369416904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400aa30 con 0x7f9b1c162a70 2024-04-26T23:42:22.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.153+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c0c93c0 con 0x7f9b1c162a70 2024-04-26T23:42:22.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.154+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c0c9980 con 0x7f9b1c162a70 2024-04-26T23:42:22.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.156+0000 7f9b13400640 1 -- 172.21.15.46:0/3369416904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b1400abd0 con 0x7f9b1c162a70 2024-04-26T23:42:22.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.157+0000 7f9b13400640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 0x7f9afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.157+0000 7f9b13400640 1 -- 172.21.15.46:0/3369416904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9b1408bfa0 con 0x7f9b1c162a70 2024-04-26T23:42:22.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.158+0000 7f9b24fce740 1 Processor -- start 2024-04-26T23:42:22.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.158+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 0x7f9afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.158+0000 7f9b24fce740 1 -- start start 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e28100 0x555ef7e2b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ef7d90c70 con 0x555ef7e55880 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ef7d49980 con 0x555ef7e28100 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555ef7d8cc30 con 0x555ef7e554a0 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 0x7f9afc06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006fd0 tx=0x7f9b0c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32908/0 (socket says 172.21.15.46:32908) 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.159+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3367953960 learned_addr learned my addr 172.21.15.46:0/3367953960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1be00640 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e28100 0x555ef7e2b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 msgr2=0x555ef7e57cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e57cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e28100 msgr2=0x555ef7e2b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e28100 0x555ef7e2b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3367953960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef7d217b0 con 0x555ef7e55880 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.160+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9b080090b0 tx=0x7f9b0802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dde45fb202e6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b07400640 1 -- 172.21.15.46:0/3367953960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08041070 con 0x555ef7e55880 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b07400640 1 -- 172.21.15.46:0/3367953960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b080326c0 con 0x555ef7e55880 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b07400640 1 -- 172.21.15.46:0/3367953960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037040 con 0x555ef7e55880 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 -- 172.21.15.46:0/3367953960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 msgr2=0x555ef7e581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9b080090b0 tx=0x7f9b0802c000 comp rx=0 tx=0).stop 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 -- 172.21.15.46:0/3367953960 shutdown_connections 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e28100 0x555ef7e2b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e581f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3367953960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e57cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.161+0000 7f9b24fce740 1 -- 172.21.15.46:0/3367953960 >> 172.21.15.46:0/3367953960 conn(0x555ef7e2a370 msgr2=0x555ef7e72cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.162+0000 7f9b24fce740 1 -- 172.21.15.46:0/3367953960 shutdown_connections 2024-04-26T23:42:22.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.162+0000 7f9b24fce740 1 -- 172.21.15.46:0/3367953960 wait complete. 2024-04-26T23:42:22.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.162+0000 7f9b24fce740 1 -- start start 2024-04-26T23:42:22.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.163+0000 7f9b24fce740 1 Processor -- start 2024-04-26T23:42:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e591f0 0x555ef7e75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ef7d8ff80 con 0x555ef7e55880 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ef7d3d260 con 0x555ef7e554a0 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555ef7d952a0 con 0x555ef7e591f0 2024-04-26T23:42:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b24fce740 10 client.0 _ll_register_callbacks cb 0x555ef7d87960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32914/0 (socket says 172.21.15.46:32914) 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3902025213 learned_addr learned my addr 172.21.15.46:0/3902025213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1be00640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e591f0 0x555ef7e75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e591f0 msgr2=0x555ef7e75a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e591f0 0x555ef7e75a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 msgr2=0x555ef7e5b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e5b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.164+0000 7f9b1aa00640 1 -- 172.21.15.46:0/3902025213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef7d1f1c0 con 0x555ef7e55880 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a8b0 tx=0x7f9b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fac22025d29d06c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037040 con 0x555ef7e55880 2024-04-26T23:42:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9b08032a80 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b080326c0 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9b08053050 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.165+0000 7f9b24fce740 10 client.6132 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x555ef7aa23f0 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b080326c0 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f9b08056020 con 0x555ef7e55880 2024-04-26T23:42:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 10 client.6132 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 10 client.6132 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 20 client.6132 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e79890 0x555ef7e79cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 4 client.6132 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef7d9e880 con 0x555ef7e79890 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.166+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e79890 0x555ef7e79cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.167+0000 7f9b1b400640 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e79890 0x555ef7e79cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60f7afb2c6b594 server_cookie=37cdbc74e8e9155c in_seq=0 out_seq=0 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.167+0000 7f9afbe00640 10 client.6132 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9afbe00640 1 -- 172.21.15.46:0/3902025213 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ef7d9e880 con 0x555ef7e79890 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9afbe00640 10 client.6132 handle_command_reply: tid=0 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 client.6132 shutdown 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 msgr2=0x555ef7e54870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a8b0 tx=0x7f9b08004730 comp rx=0 tx=0).stop 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 shutdown_connections 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e591f0 0x555ef7e75a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e54870 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e79890 0x555ef7e79cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 --2- 172.21.15.46:0/3902025213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9afbe00640 0 client.6132 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 >> 172.21.15.46:0/3902025213 conn(0x555ef7e28100 msgr2=0x555ef7e2d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 shutdown_connections 2024-04-26T23:42:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 1 -- 172.21.15.46:0/3902025213 wait complete. 2024-04-26T23:42:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.168+0000 7f9b24fce740 20 client.6132 trim_cache size 0 max 16384 2024-04-26T23:42:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.184+0000 7f9b24fce740 1 Processor -- start 2024-04-26T23:42:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.184+0000 7f9b24fce740 1 -- start start 2024-04-26T23:42:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e7b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 0x555ef7e2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ef7e5bbc0 con 0x555ef7e591f0 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c201d10 con 0x555ef7e554a0 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c200d30 con 0x555ef7e55880 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.185+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32920/0 (socket says 172.21.15.46:32920) 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 -- 172.21.15.46:0/2180108631 learned_addr learned my addr 172.21.15.46:0/2180108631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 0x555ef7e2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 -- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 msgr2=0x555ef7e2e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1b400640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e7b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 0x555ef7e2e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 -- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 msgr2=0x555ef7e7b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e7b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 -- 172.21.15.46:0/2180108631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef7b6c7d0 con 0x555ef7e591f0 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 0x555ef7e2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:22.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.186+0000 7f9b1be00640 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9b140597a0 tx=0x7f9b14060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c2878fa19cf745 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b06a00640 1 -- 172.21.15.46:0/2180108631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14087020 con 0x555ef7e591f0 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b06a00640 1 -- 172.21.15.46:0/2180108631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1405de50 con 0x555ef7e591f0 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b06a00640 1 -- 172.21.15.46:0/2180108631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14062680 con 0x555ef7e591f0 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 -- 172.21.15.46:0/2180108631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 msgr2=0x555ef7e2d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9b140597a0 tx=0x7f9b14060040 comp rx=0 tx=0).stop 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 -- 172.21.15.46:0/2180108631 shutdown_connections 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e591f0 0x555ef7e2d190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e55880 0x555ef7e2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 --2- 172.21.15.46:0/2180108631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e554a0 0x555ef7e7b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 -- 172.21.15.46:0/2180108631 >> 172.21.15.46:0/2180108631 conn(0x555ef7e28100 msgr2=0x555ef7e5b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.187+0000 7f9b24fce740 1 -- 172.21.15.46:0/2180108631 shutdown_connections 2024-04-26T23:42:22.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.188+0000 7f9b24fce740 1 -- 172.21.15.46:0/2180108631 wait complete. 2024-04-26T23:42:22.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.188+0000 7f9b24fce740 1 -- start start 2024-04-26T23:42:22.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.188+0000 7f9b24fce740 1 Processor -- start 2024-04-26T23:42:22.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e591f0 0x555ef7ef4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0b9270 con 0x555ef7e55880 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0b8c20 con 0x555ef7e591f0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555ef7e5bbc0 con 0x555ef7e554a0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.189+0000 7f9b24fce740 10 client.0 _ll_register_callbacks cb 0x7f9b080096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32936/0 (socket says 172.21.15.46:32936) 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 -- 172.21.15.46:0/858333752 learned_addr learned my addr 172.21.15.46:0/858333752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1b400640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1be00640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e591f0 0x555ef7ef4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 -- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 msgr2=0x555ef7e72cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e72cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 -- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e591f0 msgr2=0x555ef7ef4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e591f0 0x555ef7ef4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 -- 172.21.15.46:0/858333752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef7d372a0 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9b1aa00640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9b08032e60 tx=0x7f9b0802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6c2b9baf9840b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.190+0000 7f9afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037030 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9b08058c30 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b1405de50 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9b08041040 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 10 client.6138 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x555ef7e95670 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1405de50 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f9b08056040 con 0x555ef7e55880 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 10 client.6138 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:22.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 20 client.6138 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e70e80 0x555ef7ef4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 4 client.6138 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.191+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ef7d986f0 con 0x555ef7e70e80 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.192+0000 7f9b1b400640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e70e80 0x555ef7ef4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.192+0000 7f9b1b400640 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e70e80 0x555ef7ef4780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc6b145be29e5a8 server_cookie=6a8868da19179289 in_seq=0 out_seq=0 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.192+0000 7f9afbe00640 10 client.6138 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.192+0000 7f9afbe00640 1 -- 172.21.15.46:0/858333752 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+168 (crc 0 0 0) 0x555ef7d986f0 con 0x555ef7e70e80 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.192+0000 7f9afbe00640 10 client.6138 handle_command_reply: tid=0 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 client.6138 shutdown 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 msgr2=0x555ef7e731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9b08032e60 tx=0x7f9b0802e000 comp rx=0 tx=0).stop 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 shutdown_connections 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x555ef7e70e80 0x555ef7ef4780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ef7e591f0 0x555ef7ef4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ef7e55880 0x555ef7e731f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 --2- 172.21.15.46:0/858333752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555ef7e554a0 0x555ef7e72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9afbe00640 0 client.6138 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 >> 172.21.15.46:0/858333752 conn(0x555ef7e28100 msgr2=0x555ef7e2d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 shutdown_connections 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 1 -- 172.21.15.46:0/858333752 wait complete. 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.193+0000 7f9b24fce740 20 client.6138 trim_cache size 0 max 16384 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-26T23:42:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3536701558, 2024-04-26T23:42:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511630412, 2024-04-26T23:42:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-26T23:42:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:22.206 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:42:22.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 msgr2=0x7f9afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 0x7f9afc06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006fd0 tx=0x7f9b0c008040 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 msgr2=0x7f9b1c0ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c7b0 tx=0x7f9b1400cc70 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 shutdown_connections 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9afc06c8f0 0x7f9afc06edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b1c162a70 0x7f9b1c0ca720 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0d3830 0x7f9b1c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 --2- 172.21.15.46:0/3369416904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c0d1ae0 0x7f9b1c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 >> 172.21.15.46:0/3369416904 conn(0x7f9b1c0d4f20 msgr2=0x7f9b1c1613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 shutdown_connections 2024-04-26T23:42:22.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.194+0000 7f9b20c00640 1 -- 172.21.15.46:0/3369416904 wait complete. 2024-04-26T23:42:22.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.411+0000 7fad5e200640 1 Processor -- start 2024-04-26T23:42:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.412+0000 7fad5e200640 1 -- start start 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad5815f4e0 0x7fad580d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad580b4000 0x7fad580d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad580d1f60 0x7fad580cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fad580d2990 con 0x7fad580b4000 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fad580d11f0 con 0x7fad580d1f60 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.413+0000 7fad5e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad580d5f20 con 0x7fad5815f4e0 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad580b4000 0x7fad580d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad580b4000 0x7fad580d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32938/0 (socket says 172.21.15.46:32938) 2024-04-26T23:42:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 -- 172.21.15.46:0/1537243506 learned_addr learned my addr 172.21.15.46:0/1537243506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad5d800640 1 --2- 172.21.15.46:0/1537243506 >> v2:172.21.15.142:3300/0 conn(0x7fad580d1f60 0x7fad580cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad5ce00640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad5815f4e0 0x7fad580d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 -- 172.21.15.46:0/1537243506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad5815f4e0 msgr2=0x7fad580d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad5815f4e0 0x7fad580d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 -- 172.21.15.46:0/1537243506 >> v2:172.21.15.142:3300/0 conn(0x7fad580d1f60 msgr2=0x7fad580cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 --2- 172.21.15.46:0/1537243506 >> v2:172.21.15.142:3300/0 conn(0x7fad580d1f60 0x7fad580cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.414+0000 7fad57e00640 1 -- 172.21.15.46:0/1537243506 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580d0f10 con 0x7fad580b4000 2024-04-26T23:42:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.415+0000 7fad57e00640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580d1a20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fad580bb0c0 tx=0x7fad4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad6db86f47456dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.416+0000 7fad57400640 1 -- 172.21.15.46:0/1537243506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40037070 con 0x7fad580b4000 2024-04-26T23:42:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.416+0000 7fad57400640 1 -- 172.21.15.46:0/1537243506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40032440 con 0x7fad580b4000 2024-04-26T23:42:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.416+0000 7fad57400640 1 -- 172.21.15.46:0/1537243506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40032810 con 0x7fad580b4000 2024-04-26T23:42:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.416+0000 7fad5e200640 1 -- 172.21.15.46:0/1537243506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 msgr2=0x7fad580d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.416+0000 7fad5e200640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580d1a20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fad580bb0c0 tx=0x7fad4002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 -- 172.21.15.46:0/1537243506 shutdown_connections 2024-04-26T23:42:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 --2- 172.21.15.46:0/1537243506 >> v2:172.21.15.142:3300/0 conn(0x7fad580d1f60 0x7fad580cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580d1a20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 --2- 172.21.15.46:0/1537243506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad5815f4e0 0x7fad580d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 -- 172.21.15.46:0/1537243506 >> 172.21.15.46:0/1537243506 conn(0x7fad580d5010 msgr2=0x7fad580d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 -- 172.21.15.46:0/1537243506 shutdown_connections 2024-04-26T23:42:22.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.417+0000 7fad5e200640 1 -- 172.21.15.46:0/1537243506 wait complete. 2024-04-26T23:42:22.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.419+0000 7fad5e200640 1 Processor -- start 2024-04-26T23:42:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.419+0000 7fad5e200640 1 -- start start 2024-04-26T23:42:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 0x7fad5809fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5815f4e0 0x7fad58161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad5815f8c0 con 0x7fad580b4000 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad580b1d60 con 0x7fad5815f4e0 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad580d0210 con 0x7fad580d1f60 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32948/0 (socket says 172.21.15.46:32948) 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 -- 172.21.15.46:0/139324409 learned_addr learned my addr 172.21.15.46:0/139324409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5d800640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5815f4e0 0x7fad58161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad57e00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 0x7fad5809fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 -- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 msgr2=0x7fad5809fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 0x7fad5809fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 -- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5815f4e0 msgr2=0x7fad58161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.420+0000 7fad5ce00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5815f4e0 0x7fad58161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad5ce00640 1 -- 172.21.15.46:0/139324409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580cd670 con 0x7fad580b4000 2024-04-26T23:42:22.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad57e00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 0x7fad5809fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:22.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad5ce00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fad48007e40 tx=0x7fad4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48014620 con 0x7fad580b4000 2024-04-26T23:42:22.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48019070 con 0x7fad580b4000 2024-04-26T23:42:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48014a00 con 0x7fad580b4000 2024-04-26T23:42:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad580a00b0 con 0x7fad580b4000 2024-04-26T23:42:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.421+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad580cdbd0 con 0x7fad580b4000 2024-04-26T23:42:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.423+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad580693b0 con 0x7fad580b4000 2024-04-26T23:42:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.424+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad48014ba0 con 0x7fad580b4000 2024-04-26T23:42:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.425+0000 7fad54c00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 0x7fad3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.425+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fad4808c4f0 con 0x7fad580b4000 2024-04-26T23:42:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.426+0000 7fad57e00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 0x7fad3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.427+0000 7fad57e00640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 0x7fad3406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fad40034000 tx=0x7fad4002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.430+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad580cd670 con 0x7fad580b4000 2024-04-26T23:42:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.675+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad580b3de0 con 0x7fad580b4000 2024-04-26T23:42:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.676+0000 7fad54c00640 1 -- 172.21.15.46:0/139324409 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7fad4805a820 con 0x7fad580b4000 2024-04-26T23:42:22.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:22.681 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.679+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 msgr2=0x7fad3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.679+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 0x7fad3406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fad40034000 tx=0x7fad4002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 msgr2=0x7fad580cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fad48007e40 tx=0x7fad4800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 shutdown_connections 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad3406c8f0 0x7fad3406edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5815f4e0 0x7fad58161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad580d1f60 0x7fad5809fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 --2- 172.21.15.46:0/139324409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad580b4000 0x7fad580cd130 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 >> 172.21.15.46:0/139324409 conn(0x7fad580d5010 msgr2=0x7fad580b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 shutdown_connections 2024-04-26T23:42:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.680+0000 7fad5e200640 1 -- 172.21.15.46:0/139324409 wait complete. 2024-04-26T23:42:22.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 3536701558 2024-04-26T23:42:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.909+0000 7f90f5c00640 1 Processor -- start 2024-04-26T23:42:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.909+0000 7f90f5c00640 1 -- start start 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.910+0000 7f90f5c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f90f00cdd20 0x7f90f00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90f5c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00af570 con 0x7f90f015f210 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90f5c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00af850 con 0x7f90f00ae8f0 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90f5c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f90f00cb680 con 0x7f90f00cdd20 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90efe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f90f015f210 0x7f90f00cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46068/0 (socket says 172.21.15.46:46068) 2024-04-26T23:42:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90efe00640 1 -- 172.21.15.46:0/3771766997 learned_addr learned my addr 172.21.15.46:0/3771766997 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90ef400640 1 --2- 172.21.15.46:0/3771766997 >> v2:172.21.15.142:3300/0 conn(0x7f90f00cdd20 0x7f90f00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633909198 0 0) 0x7f90f00af570 con 0x7f90f015f210 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.911+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90d4003520 con 0x7f90f015f210 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4204300818 0 0) 0x7f90d4003520 con 0x7f90f015f210 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 >> v2:172.21.15.142:3300/0 conn(0x7f90f00cdd20 msgr2=0x7f90f00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 --2- 172.21.15.46:0/3771766997 >> v2:172.21.15.142:3300/0 conn(0x7f90f00cdd20 0x7f90f00cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 >> v1:172.21.15.142:6789/0 conn(0x7f90f00ae8f0 legacy=0x7f90f00aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00cb960 con 0x7f90f015f210 2024-04-26T23:42:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f90e0001670 con 0x7f90f015f210 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 >> v1:172.21.15.46:6789/0 conn(0x7f90f015f210 legacy=0x7f90f00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d40039b0 0x7f90d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 0x7f90d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d40051c0 0x7f90d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00cb680 con 0x7f90d4004800 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90d400c060 con 0x7f90d40039b0 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90d400c260 con 0x7f90d40051c0 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90ef400640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 0x7f90d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.912+0000 7f90efe00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d40039b0 0x7f90d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90f5200640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d40051c0 0x7f90d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90ef400640 1 -- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d40051c0 msgr2=0x7f90d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90ef400640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d40051c0 0x7f90d400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90ef400640 1 -- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d40039b0 msgr2=0x7f90d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90ef400640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d40039b0 0x7f90d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90ef400640 1 -- 172.21.15.46:0/3771766997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d4003520 con 0x7f90d4004800 2024-04-26T23:42:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.913+0000 7f90f5200640 1 -- 172.21.15.46:0/3771766997 reap_dead start 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.914+0000 7f90ef400640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 0x7f90d4004c80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009ac0 tx=0x7f90d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88643082b1fd67d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.914+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc036070 con 0x7f90d4004800 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.914+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90dc031400 con 0x7f90d4004800 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.914+0000 7f90eea00640 1 -- 172.21.15.46:0/3771766997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc009690 con 0x7f90d4004800 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 -- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 msgr2=0x7f90d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 0x7f90d4004c80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009ac0 tx=0x7f90d4005830 comp rx=0 tx=0).stop 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 -- 172.21.15.46:0/3771766997 shutdown_connections 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 --2- 172.21.15.46:0/3771766997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4004800 0x7f90d4004c80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 -- 172.21.15.46:0/3771766997 >> 172.21.15.46:0/3771766997 conn(0x7f90f00b27b0 msgr2=0x7f90f00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 -- 172.21.15.46:0/3771766997 shutdown_connections 2024-04-26T23:42:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.915+0000 7f90f5c00640 1 -- 172.21.15.46:0/3771766997 wait complete. 2024-04-26T23:42:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.917+0000 7f90f5c00640 1 Processor -- start 2024-04-26T23:42:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.917+0000 7f90f5c00640 1 -- start start 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d4004800 0x7f90f01f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f015ad90 0x7f90f00c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00cc160 con 0x7f90f00c9230 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f015f5f0 con 0x7f90f015ad90 2024-04-26T23:42:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90f5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90f00cc2e0 con 0x7f90d4004800 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32978/0 (socket says 172.21.15.46:32978) 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.918+0000 7f90ef400640 1 -- 172.21.15.46:0/2126774693 learned_addr learned my addr 172.21.15.46:0/2126774693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90efe00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d4004800 0x7f90f01f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90f5200640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f015ad90 0x7f90f00c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 -- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d4004800 msgr2=0x7f90f01f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d4004800 0x7f90f01f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 -- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f015ad90 msgr2=0x7f90f00c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f015ad90 0x7f90f00c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 -- 172.21.15.46:0/2126774693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f01f5c00 con 0x7f90f00c9230 2024-04-26T23:42:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90ef400640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009be0 tx=0x7f90dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.919+0000 7f90e7e00640 1 -- 172.21.15.46:0/2126774693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc03f020 con 0x7f90f00c9230 2024-04-26T23:42:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.920+0000 7f90e7e00640 1 -- 172.21.15.46:0/2126774693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90dc037040 con 0x7f90f00c9230 2024-04-26T23:42:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.920+0000 7f90e7e00640 1 -- 172.21.15.46:0/2126774693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc036070 con 0x7f90f00c9230 2024-04-26T23:42:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.920+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f00c8190 con 0x7f90f00c9230 2024-04-26T23:42:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.920+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f00c8750 con 0x7f90f00c9230 2024-04-26T23:42:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.923+0000 7f90e7e00640 1 -- 172.21.15.46:0/2126774693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90dc042020 con 0x7f90f00c9230 2024-04-26T23:42:22.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.924+0000 7f90e7e00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.924+0000 7f90e7e00640 1 -- 172.21.15.46:0/2126774693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f90dc0abd70 con 0x7f90f00c9230 2024-04-26T23:42:22.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.924+0000 7f90f9fa1740 1 Processor -- start 2024-04-26T23:42:22.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.924+0000 7f90efe00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 -- start start 2024-04-26T23:42:22.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05530 0x55d7e5f05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 0x55d7e5ed56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d7e5e3f900 con 0x55d7e5ed5bf0 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7e5ddc1f0 con 0x55d7e5f05530 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.925+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d7e5e3b8c0 con 0x55d7e5f05f50 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90efe00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 0x7f90d006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90e0005a70 tx=0x7f90e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32982/0 (socket says 172.21.15.46:32982) 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 -- 172.21.15.46:0/3019532966 learned_addr learned my addr 172.21.15.46:0/3019532966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90efe00640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05530 0x55d7e5f05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90ef400640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 0x55d7e5ed56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 -- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 msgr2=0x55d7e5ed56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 0x55d7e5ed56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 -- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05530 msgr2=0x55d7e5f05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05530 0x55d7e5f05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90f5200640 1 -- 172.21.15.46:0/3019532966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e5c1d710 con 0x55d7e5ed5bf0 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.926+0000 7f90ef400640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 0x55d7e5ed56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90f5200640 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f90f0070390 tx=0x7f90e8001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4570b94cc1690370 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90e4c00640 1 -- 172.21.15.46:0/3019532966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003b70 con 0x55d7e5ed5bf0 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90e4c00640 1 -- 172.21.15.46:0/3019532966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e8003d10 con 0x55d7e5ed5bf0 2024-04-26T23:42:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90e4c00640 1 -- 172.21.15.46:0/3019532966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800e9a0 con 0x55d7e5ed5bf0 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3019532966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 msgr2=0x55d7e5efd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.927+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f90f0070390 tx=0x7f90e8001de0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3019532966 shutdown_connections 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5ed5bf0 0x55d7e5efd9b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05f50 0x55d7e5ed56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3019532966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05530 0x55d7e5f05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3019532966 >> 172.21.15.46:0/3019532966 conn(0x55d7e5ed7920 msgr2=0x55d7e5f20340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3019532966 shutdown_connections 2024-04-26T23:42:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.928+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3019532966 wait complete. 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.929+0000 7f90f9fa1740 1 -- start start 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.929+0000 7f90f9fa1740 1 Processor -- start 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f07fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d7e5f01c80 con 0x55d7e5f1edb0 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7e5dc7040 con 0x55d7e5f05f50 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d7e5f01f20 con 0x55d7e5f05530 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f9fa1740 10 client.0 _ll_register_callbacks cb 0x55d7e5df7220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32988/0 (socket says 172.21.15.46:32988) 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.930+0000 7f90f5200640 1 -- 172.21.15.46:0/3232519130 learned_addr learned my addr 172.21.15.46:0/3232519130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90ef400640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 -- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 msgr2=0x55d7e5f07fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90efe00640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f07fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f07fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 -- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 msgr2=0x55d7e5f04180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f04180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 -- 172.21.15.46:0/3232519130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e5c7dea0 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90ef400640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f5200640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e590 tx=0x7f90e8001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae8fed2c1c16850 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90d9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8022020 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f90e800eb80 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.931+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8003d10 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90e800ed80 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 10 client.6156 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55d7e5b11000 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e8003d10 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f90e8005070 con 0x55d7e5f1edb0 2024-04-26T23:42:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 10 client.6156 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 10 client.6156 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 20 client.6156 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5efe120 0x55d7e5efe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 4 client.6156 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e5e4d510 con 0x55d7e5efe120 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.932+0000 7f90efe00640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5efe120 0x55d7e5efe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.933+0000 7f90efe00640 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5efe120 0x55d7e5efe500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b862d4f3dfa3c5f server_cookie=ea33b669294e37b8 in_seq=0 out_seq=0 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.933+0000 7f90d9600640 10 client.6156 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90d9600640 1 -- 172.21.15.46:0/3232519130 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7e5e4d510 con 0x55d7e5efe120 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90d9600640 10 client.6156 handle_command_reply: tid=0 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 client.6156 shutdown 2024-04-26T23:42:22.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 msgr2=0x55d7e5f046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e590 tx=0x7f90e8001510 comp rx=0 tx=0).stop 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 shutdown_connections 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f046c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5efe120 0x55d7e5efe500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3232519130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f07fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90d9600640 0 client.6156 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 >> 172.21.15.46:0/3232519130 conn(0x55d7e5ed5bf0 msgr2=0x55d7e5ed90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 shutdown_connections 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.934+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3232519130 wait complete. 2024-04-26T23:42:22.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.935+0000 7f90f9fa1740 20 client.6156 trim_cache size 0 max 16384 2024-04-26T23:42:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.949+0000 7f90f9fa1740 1 Processor -- start 2024-04-26T23:42:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.949+0000 7f90f9fa1740 1 -- start start 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f1edb0 0x55d7e5f328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0202480 con 0x55d7e5f05530 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f02015d0 con 0x55d7e5f05f50 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.950+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90f0160a00 con 0x55d7e5f1edb0 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32994/0 (socket says 172.21.15.46:32994) 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 -- 172.21.15.46:0/1461182072 learned_addr learned my addr 172.21.15.46:0/1461182072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 -- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f1edb0 msgr2=0x55d7e5f328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f1edb0 0x55d7e5f328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 -- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 msgr2=0x55d7e5f43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 -- 172.21.15.46:0/1461182072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e5f00860 con 0x55d7e5f05530 2024-04-26T23:42:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.951+0000 7f90efe00640 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f90e00058a0 tx=0x7f90e000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54deeaa61a8423a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90dbe00640 1 -- 172.21.15.46:0/1461182072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0027440 con 0x55d7e5f05530 2024-04-26T23:42:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90dbe00640 1 -- 172.21.15.46:0/1461182072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e00275e0 con 0x55d7e5f05530 2024-04-26T23:42:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90dbe00640 1 -- 172.21.15.46:0/1461182072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0024400 con 0x55d7e5f05530 2024-04-26T23:42:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 -- 172.21.15.46:0/1461182072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 msgr2=0x55d7e5edb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f90e00058a0 tx=0x7f90e000e000 comp rx=0 tx=0).stop 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 -- 172.21.15.46:0/1461182072 shutdown_connections 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f1edb0 0x55d7e5f328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f05f50 0x55d7e5f43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/1461182072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05530 0x55d7e5edb9a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 -- 172.21.15.46:0/1461182072 >> 172.21.15.46:0/1461182072 conn(0x55d7e5ed5bf0 msgr2=0x55d7e5efcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 -- 172.21.15.46:0/1461182072 shutdown_connections 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.952+0000 7f90f9fa1740 1 -- 172.21.15.46:0/1461182072 wait complete. 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.953+0000 7f90f9fa1740 1 -- start start 2024-04-26T23:42:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.953+0000 7f90f9fa1740 1 Processor -- start 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d7e5f06e10 con 0x55d7e5f05f50 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0161050 con 0x55d7e5f1edb0 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d7e5efec80 con 0x55d7e5f05530 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.954+0000 7f90f9fa1740 10 client.0 _ll_register_callbacks cb 0x7f90d4004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33004/0 (socket says 172.21.15.46:33004) 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 -- 172.21.15.46:0/3743306703 learned_addr learned my addr 172.21.15.46:0/3743306703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90efe00640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 -- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 msgr2=0x55d7e5f83040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f83040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90f5200640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f9dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 -- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f1edb0 msgr2=0x55d7e5f9dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f9dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 -- 172.21.15.46:0/3743306703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e5f32ba0 con 0x55d7e5f05f50 2024-04-26T23:42:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90ef400640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f90dc079960 tx=0x7f90dc079a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d011d5f145bd284 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90d9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.955+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0825b0 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f90dc079c30 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e00275e0 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90dc079e30 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 10 client.6162 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55d7e5f16920 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e00275e0 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f90dc082120 con 0x55d7e5f05f50 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 10 client.6162 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 10 client.6162 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 20 client.6162 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5f08af0 0x55d7e5f42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 4 client.6162 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3536701558}] 2024-04-26T23:42:22.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3536701558}) -- 0x55d7e5e47380 con 0x55d7e5f08af0 2024-04-26T23:42:22.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.956+0000 7f90efe00640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5f08af0 0x55d7e5f42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.957+0000 7f90efe00640 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5f08af0 0x55d7e5f42d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d6c1b02b33d38f server_cookie=2798921704dcab90 in_seq=0 out_seq=0 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.957+0000 7f90d9600640 10 client.6162 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.957+0000 7f90d9600640 1 -- 172.21.15.46:0/3743306703 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55d7e5e47380 con 0x55d7e5f08af0 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.957+0000 7f90d9600640 10 client.6162 handle_command_reply: tid=0 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.957+0000 7f90f9fa1740 1 client.6162 shutdown 2024-04-26T23:42:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 msgr2=0x55d7e5f16160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f90dc079960 tx=0x7f90dc079a90 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 shutdown_connections 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55d7e5f08af0 0x55d7e5f42d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7e5f1edb0 0x55d7e5f9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7e5f05f50 0x55d7e5f16160 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 --2- 172.21.15.46:0/3743306703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7e5f05530 0x55d7e5f83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90d9600640 0 client.6162 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 >> 172.21.15.46:0/3743306703 conn(0x55d7e5ed5bf0 msgr2=0x55d7e5f43f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 shutdown_connections 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 1 -- 172.21.15.46:0/3743306703 wait complete. 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.958+0000 7f90f9fa1740 20 client.6162 trim_cache size 0 max 16384 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 msgr2=0x7f90d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 0x7f90d006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90e0005a70 tx=0x7f90e0009040 comp rx=0 tx=0).stop 2024-04-26T23:42:22.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 msgr2=0x7f90f015a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009be0 tx=0x7f90dc004730 comp rx=0 tx=0).stop 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 shutdown_connections 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f015ad90 0x7f90f00c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f00c9230 0x7f90f015a850 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 --2- 172.21.15.46:0/2126774693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d4004800 0x7f90f01f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 >> 172.21.15.46:0/2126774693 conn(0x7f90f00b27b0 msgr2=0x7f90f00724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 shutdown_connections 2024-04-26T23:42:22.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:22.959+0000 7f90f5c00640 1 -- 172.21.15.46:0/2126774693 wait complete. 2024-04-26T23:42:22.980 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-26T23:42:22.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.186+0000 7fc22b200640 1 Processor -- start 2024-04-26T23:42:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.187+0000 7fc22b200640 1 -- start start 2024-04-26T23:42:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc22b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc22c15f4e0 0x7fc22c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc22b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 0x7fc22c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc22b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c0d11f0 con 0x7fc22c0d1f60 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc22b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc22c0d2990 con 0x7fc22c15f4e0 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc22b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0d5f20 con 0x7fc22c0b4000 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.188+0000 7fc229e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc22c15f4e0 0x7fc22c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc22c15f4e0 0x7fc22c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33020/0 (socket says 172.21.15.46:33020) 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 -- 172.21.15.46:0/364883173 learned_addr learned my addr 172.21.15.46:0/364883173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc221400640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 0x7fc22c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 -- 172.21.15.46:0/364883173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 msgr2=0x7fc22c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 0x7fc22c0d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 -- 172.21.15.46:0/364883173 >> v1:172.21.15.142:6789/0 conn(0x7fc22c0d1f60 legacy=0x7fc22c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc229e00640 1 -- 172.21.15.46:0/364883173 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0d0f10 con 0x7fc22c15f4e0 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.189+0000 7fc221400640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 0x7fc22c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.190+0000 7fc229e00640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c15f4e0 0x7fc22c0d2df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fa29a9a9a5e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:23.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc229400640 1 -- 172.21.15.46:0/364883173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c037070 con 0x7fc22c15f4e0 2024-04-26T23:42:23.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc229400640 1 -- 172.21.15.46:0/364883173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c032440 con 0x7fc22c15f4e0 2024-04-26T23:42:23.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc229400640 1 -- 172.21.15.46:0/364883173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c032810 con 0x7fc22c15f4e0 2024-04-26T23:42:23.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 -- 172.21.15.46:0/364883173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c15f4e0 msgr2=0x7fc22c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:23.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c15f4e0 0x7fc22c0d2df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c02a210 comp rx=0 tx=0).stop 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 -- 172.21.15.46:0/364883173 shutdown_connections 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c0b4000 0x7fc22c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 --2- 172.21.15.46:0/364883173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c15f4e0 0x7fc22c0d2df0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.191+0000 7fc22b200640 1 -- 172.21.15.46:0/364883173 >> 172.21.15.46:0/364883173 conn(0x7fc22c0d5010 msgr2=0x7fc22c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.192+0000 7fc22b200640 1 -- 172.21.15.46:0/364883173 shutdown_connections 2024-04-26T23:42:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.192+0000 7fc22b200640 1 -- 172.21.15.46:0/364883173 wait complete. 2024-04-26T23:42:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.193+0000 7fc22b200640 1 Processor -- start 2024-04-26T23:42:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.193+0000 7fc22b200640 1 -- start start 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c0d1f60 0x7fc22c0ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c15f4e0 0x7fc22c0ba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c15f8c0 con 0x7fc22c0b4000 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0b1d60 con 0x7fc22c0d1f60 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.194+0000 7fc22b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0b3290 con 0x7fc22c15f4e0 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33028/0 (socket says 172.21.15.46:33028) 2024-04-26T23:42:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 -- 172.21.15.46:0/3742217304 learned_addr learned my addr 172.21.15.46:0/3742217304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:23.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 -- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c15f4e0 msgr2=0x7fc22c0ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:23.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c15f4e0 0x7fc22c0ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc221400640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c0d1f60 0x7fc22c0ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 -- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c0d1f60 msgr2=0x7fc22c0ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c0d1f60 0x7fc22c0ba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 -- 172.21.15.46:0/3742217304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0b8920 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.195+0000 7fc229e00640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.196+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c041020 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.196+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c030970 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.196+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c037040 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.196+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c0b8c00 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.196+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c0b91c0 con 0x7fc22c0b4000 2024-04-26T23:42:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.197+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c0693b0 con 0x7fc22c0b4000 2024-04-26T23:42:23.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.198+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc21c044020 con 0x7fc22c0b4000 2024-04-26T23:42:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.199+0000 7fc222a00640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 0x7fc20806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.199+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc21c0ac770 con 0x7fc22c0b4000 2024-04-26T23:42:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.200+0000 7fc221400640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 0x7fc20806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.201+0000 7fc221400640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 0x7fc20806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0bb090 tx=0x7fc214008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.204+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc21c04b030 con 0x7fc22c0b4000 2024-04-26T23:42:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.451+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc22c0d0c50 con 0x7fc22c0b4000 2024-04-26T23:42:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.452+0000 7fc222a00640 1 -- 172.21.15.46:0/3742217304 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7fc21c07a970 con 0x7fc22c0b4000 2024-04-26T23:42:23.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:23.456 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 msgr2=0x7fc20806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 0x7fc20806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0bb090 tx=0x7fc214008040 comp rx=0 tx=0).stop 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 msgr2=0x7fc22c152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 shutdown_connections 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc20806c8f0 0x7fc20806edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc22c15f4e0 0x7fc22c0ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c0d1f60 0x7fc22c0ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.454+0000 7fc22b200640 1 --2- 172.21.15.46:0/3742217304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c0b4000 0x7fc22c152510 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.455+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 >> 172.21.15.46:0/3742217304 conn(0x7fc22c0d5010 msgr2=0x7fc22c1600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.455+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 shutdown_connections 2024-04-26T23:42:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23.455+0000 7fc22b200640 1 -- 172.21.15.46:0/3742217304 wait complete. 2024-04-26T23:42:23.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-26T23:42:23.697 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:23.697 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:23.697 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:23.713 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:23.713 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:23.713 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-26T23:42:23.713 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-26T23:42:23.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:23.713 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:23.769 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-04-26T23:42:23.770 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:42:23.921 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:23.921 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mode": 16877, 2024-04-26T23:42:23.921 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ino": 1099511627777, 2024-04-26T23:42:23.921 INFO:teuthology.orchestra.run.smithi046.stdout: "st_dev": 41, 2024-04-26T23:42:23.921 INFO:teuthology.orchestra.run.smithi046.stdout: "st_nlink": 16, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_uid": 1000, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_gid": 1316, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_size": 18, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_atime": 1714174925.9129894, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mtime": 1714174939.9718351, 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ctime": 1714174939.9718351 2024-04-26T23:42:23.922 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:42:23.935 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-26T23:42:23.935 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-26T23:42:23.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.192+0000 7fd3a0a00640 1 Processor -- start 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.192+0000 7fd3a0a00640 1 -- start start 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd3a0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd39c0d2130 0x7fd39c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd3a0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd39c0b4000 0x7fd39c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd3a0a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd39c0d11f0 con 0x7fd39c15f4e0 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd3a0a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c0d0ce0 con 0x7fd39c0d2130 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd3a0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c0d2990 con 0x7fd39c0b4000 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd39a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd39c15f4e0 0x7fd39c0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46072/0 (socket says 172.21.15.46:46072) 2024-04-26T23:42:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd39a800640 1 -- 172.21.15.46:0/4192346223 learned_addr learned my addr 172.21.15.46:0/4192346223 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:24.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd391e00640 1 --2- 172.21.15.46:0/4192346223 >> v2:172.21.15.46:3300/0 conn(0x7fd39c0d2130 0x7fd39c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd39b200640 1 --2- 172.21.15.46:0/4192346223 >> v2:172.21.15.142:3300/0 conn(0x7fd39c0b4000 0x7fd39c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd399e00640 1 -- 172.21.15.46:0/4192346223 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266248748 0 0) 0x7fd39c0d11f0 con 0x7fd39c15f4e0 2024-04-26T23:42:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.194+0000 7fd399e00640 1 -- 172.21.15.46:0/4192346223 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd380003520 con 0x7fd39c15f4e0 2024-04-26T23:42:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.195+0000 7fd391e00640 1 -- 172.21.15.46:0/4192346223 >> v2:172.21.15.142:3300/0 conn(0x7fd39c0b4000 msgr2=0x7fd39c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.195+0000 7fd391e00640 1 --2- 172.21.15.46:0/4192346223 >> v2:172.21.15.142:3300/0 conn(0x7fd39c0b4000 0x7fd39c0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.195+0000 7fd391e00640 1 -- 172.21.15.46:0/4192346223 >> v1:172.21.15.46:6789/0 conn(0x7fd39c15f4e0 legacy=0x7fd39c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.195+0000 7fd391e00640 1 -- 172.21.15.46:0/4192346223 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c0d5f20 con 0x7fd39c0d2130 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.196+0000 7fd391e00640 1 --2- 172.21.15.46:0/4192346223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0d2df0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097a0 tx=0x7fd38402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261006d53422a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.196+0000 7fd399e00640 1 -- 172.21.15.46:0/4192346223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384037070 con 0x7fd39c0d2130 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.196+0000 7fd399e00640 1 -- 172.21.15.46:0/4192346223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd384032440 con 0x7fd39c0d2130 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.196+0000 7fd399e00640 1 -- 172.21.15.46:0/4192346223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384032810 con 0x7fd39c0d2130 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 -- 172.21.15.46:0/4192346223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 msgr2=0x7fd39c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/4192346223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0d2df0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097a0 tx=0x7fd38402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 -- 172.21.15.46:0/4192346223 shutdown_connections 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/4192346223 >> v2:172.21.15.142:3300/0 conn(0x7fd39c0b4000 0x7fd39c0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/4192346223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0d2df0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 -- 172.21.15.46:0/4192346223 >> 172.21.15.46:0/4192346223 conn(0x7fd39c0d5010 msgr2=0x7fd39c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 -- 172.21.15.46:0/4192346223 shutdown_connections 2024-04-26T23:42:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.197+0000 7fd3a0a00640 1 -- 172.21.15.46:0/4192346223 wait complete. 2024-04-26T23:42:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.199+0000 7fd3a0a00640 1 Processor -- start 2024-04-26T23:42:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.199+0000 7fd3a0a00640 1 -- start start 2024-04-26T23:42:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39c0b4000 0x7fd39c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 0x7fd39c0ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c0b1d60 con 0x7fd39c0d2130 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c0d5c40 con 0x7fd39c0b4000 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd3a0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd39c15f8c0 con 0x7fd39c15f4e0 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.200+0000 7fd391e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33046/0 (socket says 172.21.15.46:33046) 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 -- 172.21.15.46:0/582799319 learned_addr learned my addr 172.21.15.46:0/582799319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd39a800640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39c0b4000 0x7fd39c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd39b200640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 0x7fd39c0ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 -- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 msgr2=0x7fd39c0ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 0x7fd39c0ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 -- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39c0b4000 msgr2=0x7fd39c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39c0b4000 0x7fd39c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 -- 172.21.15.46:0/582799319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c1557a0 con 0x7fd39c0d2130 2024-04-26T23:42:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd39b200640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 0x7fd39c0ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.201+0000 7fd391e00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098c0 tx=0x7fd3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.202+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384037040 con 0x7fd39c0d2130 2024-04-26T23:42:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.202+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd384030500 con 0x7fd39c0d2130 2024-04-26T23:42:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.202+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840308d0 con 0x7fd39c0d2130 2024-04-26T23:42:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.202+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c155a80 con 0x7fd39c0d2130 2024-04-26T23:42:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.202+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c156040 con 0x7fd39c0d2130 2024-04-26T23:42:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.203+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c0693b0 con 0x7fd39c0d2130 2024-04-26T23:42:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.204+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd384044020 con 0x7fd39c0d2130 2024-04-26T23:42:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.205+0000 7fd393400640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 0x7fd37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.205+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd3840abfc0 con 0x7fd39c0d2130 2024-04-26T23:42:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.206+0000 7fd39a800640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 0x7fd37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.207+0000 7fd39a800640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 0x7fd37806edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005c00 tx=0x7fd38c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.210+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd38407a260 con 0x7fd39c0d2130 2024-04-26T23:42:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.458+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39c0d33d0 con 0x7fd39c0d2130 2024-04-26T23:42:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.459+0000 7fd393400640 1 -- 172.21.15.46:0/582799319 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5181 (secure 0 0 0) 0x7fd384079b60 con 0x7fd39c0d2130 2024-04-26T23:42:24.463 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:24.463 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:41:59.681274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 msgr2=0x7fd37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 0x7fd37806edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005c00 tx=0x7fd38c009040 comp rx=0 tx=0).stop 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 msgr2=0x7fd39c0ba0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098c0 tx=0x7fd3840046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 shutdown_connections 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd37806c8f0 0x7fd37806edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd39c15f4e0 0x7fd39c0ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0d2130 0x7fd39c0ba0f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 --2- 172.21.15.46:0/582799319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39c0b4000 0x7fd39c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.461+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 >> 172.21.15.46:0/582799319 conn(0x7fd39c0d5010 msgr2=0x7fd39c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.462+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 shutdown_connections 2024-04-26T23:42:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.462+0000 7fd3a0a00640 1 -- 172.21.15.46:0/582799319 wait complete. 2024-04-26T23:42:24.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-26T23:42:24.693 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:24.693 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": -2 2024-04-26T23:42:24.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:24.709 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-26T23:42:24.709 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-26T23:42:24.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-26T23:42:24.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.920+0000 7f1987600640 1 Processor -- start 2024-04-26T23:42:24.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.920+0000 7f1987600640 1 -- start start 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.921+0000 7f1987600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f198015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.921+0000 7f1987600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198015e050 0x7f198006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.921+0000 7f1987600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19800d07e0 con 0x7f198015d730 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.921+0000 7f1987600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f198006eb90 con 0x7f198015e050 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.921+0000 7f1987600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19800b4310 con 0x7f19800b53b0 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198015e050 0x7f198006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198015e050 0x7f198006dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33050/0 (socket says 172.21.15.46:33050) 2024-04-26T23:42:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 -- 172.21.15.46:0/3143925670 learned_addr learned my addr 172.21.15.46:0/3143925670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f197fe00640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f198015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 -- 172.21.15.46:0/3143925670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 msgr2=0x7f198015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f198015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 -- 172.21.15.46:0/3143925670 >> v1:172.21.15.142:6789/0 conn(0x7f198015d730 legacy=0x7f19800d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.922+0000 7f1985600640 1 -- 172.21.15.46:0/3143925670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19800b45f0 con 0x7f198015e050 2024-04-26T23:42:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f1985600640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015e050 0x7f198006dc90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1978009e50 tx=0x7f197802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac432d2c2ba1e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f197f400640 1 -- 172.21.15.46:0/3143925670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978036070 con 0x7f198015e050 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f197f400640 1 -- 172.21.15.46:0/3143925670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1978031600 con 0x7f198015e050 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f197f400640 1 -- 172.21.15.46:0/3143925670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780319d0 con 0x7f198015e050 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f1987600640 1 -- 172.21.15.46:0/3143925670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015e050 msgr2=0x7f198006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.924+0000 7f1987600640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015e050 0x7f198006dc90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1978009e50 tx=0x7f197802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 -- 172.21.15.46:0/3143925670 shutdown_connections 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015e050 0x7f198006dc90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 --2- 172.21.15.46:0/3143925670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f198015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 -- 172.21.15.46:0/3143925670 >> 172.21.15.46:0/3143925670 conn(0x7f198006c980 msgr2=0x7f19800b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 -- 172.21.15.46:0/3143925670 shutdown_connections 2024-04-26T23:42:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.925+0000 7f1987600640 1 -- 172.21.15.46:0/3143925670 wait complete. 2024-04-26T23:42:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.926+0000 7f1987600640 1 Processor -- start 2024-04-26T23:42:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.926+0000 7f1987600640 1 -- start start 2024-04-26T23:42:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f19801eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 0x7f19801f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19800b4e90 con 0x7f198015d730 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19800b4bb0 con 0x7f198015e050 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.927+0000 7f1987600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f198015e4c0 con 0x7f19800b53b0 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33056/0 (socket says 172.21.15.46:33056) 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 -- 172.21.15.46:0/1081270639 learned_addr learned my addr 172.21.15.46:0/1081270639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f1985600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 0x7f19801f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f1984c00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f19801eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 -- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 msgr2=0x7f19801eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f19801eaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 -- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 msgr2=0x7f19801f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 0x7f19801f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f197fe00640 1 -- 172.21.15.46:0/1081270639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801f1100 con 0x7f198015d730 2024-04-26T23:42:24.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f1985600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 0x7f19801f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:24.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.928+0000 7f1984c00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f19801eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:24.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f197fe00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f196c00d620 tx=0x7f196c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f197cc00640 1 -- 172.21.15.46:0/1081270639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00b620 con 0x7f198015d730 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f197cc00640 1 -- 172.21.15.46:0/1081270639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c019070 con 0x7f198015d730 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f197cc00640 1 -- 172.21.15.46:0/1081270639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00ba00 con 0x7f198015d730 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801f13e0 con 0x7f198015d730 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.929+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19800b1d10 con 0x7f198015d730 2024-04-26T23:42:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.931+0000 7f197cc00640 1 -- 172.21.15.46:0/1081270639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f196c00bba0 con 0x7f198015d730 2024-04-26T23:42:24.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.932+0000 7f197cc00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 0x7f196006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.932+0000 7f197cc00640 1 -- 172.21.15.46:0/1081270639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f196c08c0b0 con 0x7f198015d730 2024-04-26T23:42:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.933+0000 7f1984c00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 0x7f196006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.933+0000 7f198a448740 1 Processor -- start 2024-04-26T23:42:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.933+0000 7f198a448740 1 -- start start 2024-04-26T23:42:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f1984c00640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 0x7f196006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1974000f60 tx=0x7f1974009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e97bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5637e9722b90 con 0x5637e97b8970 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5637e96bf350 con 0x5637e97bcc20 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.934+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5637e971eb50 con 0x5637e97bdae0 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33072/0 (socket says 172.21.15.46:33072) 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 -- 172.21.15.46:0/3488941364 learned_addr learned my addr 172.21.15.46:0/3488941364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f1984c00640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e97bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 -- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 msgr2=0x5637e97bdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f1985600640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e97bdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 -- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 msgr2=0x5637e97bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97bd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 -- 172.21.15.46:0/3488941364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e95009f0 con 0x5637e97b8970 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f1984c00640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e97bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.935+0000 7f197fe00640 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b0e0 tx=0x7f196c059b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3012c0f9fe2c6ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f1971200640 1 -- 172.21.15.46:0/3488941364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c072020 con 0x5637e97b8970 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f1971200640 1 -- 172.21.15.46:0/3488941364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c060040 con 0x5637e97b8970 2024-04-26T23:42:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f1971200640 1 -- 172.21.15.46:0/3488941364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c062820 con 0x5637e97b8970 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 -- 172.21.15.46:0/3488941364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 msgr2=0x5637e97bc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b0e0 tx=0x7f196c059b40 comp rx=0 tx=0).stop 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 -- 172.21.15.46:0/3488941364 shutdown_connections 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e97bc6e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 --2- 172.21.15.46:0/3488941364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e97bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 -- 172.21.15.46:0/3488941364 >> 172.21.15.46:0/3488941364 conn(0x5637e97babe0 msgr2=0x5637e98036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 -- 172.21.15.46:0/3488941364 shutdown_connections 2024-04-26T23:42:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.936+0000 7f198a448740 1 -- 172.21.15.46:0/3488941364 wait complete. 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.937+0000 7f198a448740 1 -- start start 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.937+0000 7f198a448740 1 Processor -- start 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9807d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5637e97e9670 con 0x5637e980a5f0 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5637e96aa1a0 con 0x5637e97bcc20 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5637e97e8af0 con 0x5637e97bdae0 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.938+0000 7f198a448740 10 client.0 _ll_register_callbacks cb 0x5637e96c3ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33078/0 (socket says 172.21.15.46:33078) 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 -- 172.21.15.46:0/2991495830 learned_addr learned my addr 172.21.15.46:0/2991495830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1984c00640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 -- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 msgr2=0x5637e9807d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f197fe00640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9807d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9807d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 -- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 msgr2=0x5637e97ddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97ddd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 -- 172.21.15.46:0/2991495830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e93f42e0 con 0x5637e980a5f0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1985600640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1978009f70 tx=0x7f1978004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e377c9cf0ce8fe4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.939+0000 7f1965600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978041070 con 0x5637e980a5f0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1978031de0 con 0x5637e980a5f0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c060040 con 0x5637e980a5f0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1978036040 con 0x5637e980a5f0 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 10 client.6189 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5637e9561180 con 0x5637e980a5f0 2024-04-26T23:42:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c060040 con 0x5637e980a5f0 2024-04-26T23:42:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f1978008040 con 0x5637e980a5f0 2024-04-26T23:42:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 10 client.6189 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 10 client.6189 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 20 client.6189 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e9809570 0x5637e9809970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 4 client.6189 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.940+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e96f6550 con 0x5637e9809570 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.941+0000 7f1984c00640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e9809570 0x5637e9809970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.941+0000 7f1984c00640 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e9809570 0x5637e9809970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df5aa511f2cb982 server_cookie=d799c6f28345073d in_seq=0 out_seq=0 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.941+0000 7f1965600640 10 client.6189 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f1965600640 1 -- 172.21.15.46:0/2991495830 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637e96f6550 con 0x5637e9809570 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f1965600640 10 client.6189 handle_command_reply: tid=0 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 client.6189 shutdown 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 msgr2=0x5637e98082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1978009f70 tx=0x7f1978004ab0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 shutdown_connections 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e980a5f0 0x5637e98082b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9807d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e9809570 0x5637e9809970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 --2- 172.21.15.46:0/2991495830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e97ddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f1965600640 0 client.6189 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.942+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 >> 172.21.15.46:0/2991495830 conn(0x5637e97b8970 msgr2=0x5637e97b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.943+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 shutdown_connections 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.943+0000 7f198a448740 1 -- 172.21.15.46:0/2991495830 wait complete. 2024-04-26T23:42:24.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.943+0000 7f198a448740 20 client.6189 trim_cache size 0 max 16384 2024-04-26T23:42:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.963+0000 7f198a448740 1 Processor -- start 2024-04-26T23:42:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.963+0000 7f198a448740 1 -- start start 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97b8970 0x5637e98041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9827960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19801fdbc0 con 0x5637e97bcc20 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19801fcc00 con 0x5637e97b8970 2024-04-26T23:42:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f198013c2e0 con 0x5637e97bdae0 2024-04-26T23:42:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33088/0 (socket says 172.21.15.46:33088) 2024-04-26T23:42:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 -- 172.21.15.46:0/2588646332 learned_addr learned my addr 172.21.15.46:0/2588646332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f1984c00640 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97b8970 0x5637e98041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 -- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 msgr2=0x5637e9827960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9827960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 -- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97b8970 msgr2=0x5637e98041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97b8970 0x5637e98041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.964+0000 7f197fe00640 1 -- 172.21.15.46:0/2588646332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e93cd3a0 con 0x5637e97bcc20 2024-04-26T23:42:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.965+0000 7f197fe00640 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f196c08a440 tx=0x7f196c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b652f5a927b4d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.965+0000 7f1967e00640 1 -- 172.21.15.46:0/2588646332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c072040 con 0x5637e97bcc20 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.965+0000 7f1967e00640 1 -- 172.21.15.46:0/2588646332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c060040 con 0x5637e97bcc20 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.965+0000 7f1967e00640 1 -- 172.21.15.46:0/2588646332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c059a10 con 0x5637e97bcc20 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 -- 172.21.15.46:0/2588646332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 msgr2=0x5637e97e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f196c08a440 tx=0x7f196c020000 comp rx=0 tx=0).stop 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 -- 172.21.15.46:0/2588646332 shutdown_connections 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9827960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97bcc20 0x5637e97e8a00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 --2- 172.21.15.46:0/2588646332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97b8970 0x5637e98041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 -- 172.21.15.46:0/2588646332 >> 172.21.15.46:0/2588646332 conn(0x5637e97bbaf0 msgr2=0x5637e9808c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 -- 172.21.15.46:0/2588646332 shutdown_connections 2024-04-26T23:42:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.966+0000 7f198a448740 1 -- 172.21.15.46:0/2588646332 wait complete. 2024-04-26T23:42:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.967+0000 7f198a448740 1 -- start start 2024-04-26T23:42:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.967+0000 7f198a448740 1 Processor -- start 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e9808a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9808f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5637e9807e10 con 0x5637e97b8970 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19801fc5b0 con 0x5637e97bcc20 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5637e97b3500 con 0x5637e97bdae0 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f198a448740 10 client.0 _ll_register_callbacks cb 0x7f1978034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33100/0 (socket says 172.21.15.46:33100) 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 -- 172.21.15.46:0/2542186273 learned_addr learned my addr 172.21.15.46:0/2542186273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 -- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 msgr2=0x5637e9808f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9808f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 -- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 msgr2=0x5637e9808a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e9808a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.968+0000 7f1984c00640 1 -- 172.21.15.46:0/2542186273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e97dc4a0 con 0x5637e97b8970 2024-04-26T23:42:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1984c00640 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f197400d940 tx=0x7f1974000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248515b0a8923f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1965600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974027070 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1974003cc0 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c060040 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1974032050 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f198a448740 10 client.6195 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5637e97dcda0 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.969+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c060040 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f1974023420 con 0x5637e97b8970 2024-04-26T23:42:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 10 client.6195 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 10 client.6195 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 20 client.6195 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e980a5f0 0x5637e97e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 4 client.6195 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) -- 0x5637e97307a0 con 0x5637e980a5f0 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f197fe00640 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e980a5f0 0x5637e97e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f197fe00640 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e980a5f0 0x5637e97e7eb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73f85ad59c7ea58 server_cookie=70173bc04a4c825e in_seq=0 out_seq=0 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.970+0000 7f1965600640 10 client.6195 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f1965600640 1 -- 172.21.15.46:0/2542186273 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+26 (crc 0 0 0) 0x5637e97307a0 con 0x5637e980a5f0 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f1965600640 10 client.6195 handle_command_reply: tid=0 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 client.6195 shutdown 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 msgr2=0x5637e9806590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f197400d940 tx=0x7f1974000ed0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 shutdown_connections 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5637e97bdae0 0x5637e9808f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5637e980a5f0 0x5637e97e7eb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637e97bcc20 0x5637e9808a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 --2- 172.21.15.46:0/2542186273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637e97b8970 0x5637e9806590 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f1965600640 0 client.6195 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 >> 172.21.15.46:0/2542186273 conn(0x5637e97bbaf0 msgr2=0x5637e9807f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 shutdown_connections 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.971+0000 7f198a448740 1 -- 172.21.15.46:0/2542186273 wait complete. 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f198a448740 20 client.6195 trim_cache size 0 max 16384 2024-04-26T23:42:24.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": -2 2024-04-26T23:42:24.979 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:24.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 msgr2=0x7f196006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 0x7f196006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1974000f60 tx=0x7f1974009040 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 msgr2=0x7f19801f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f196c00d620 tx=0x7f196c00dae0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 shutdown_connections 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f196006c8f0 0x7f196006edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f198015e050 0x7f19801f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198015d730 0x7f19801f0a00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 --2- 172.21.15.46:0/1081270639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19800b53b0 0x7f19801eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 >> 172.21.15.46:0/1081270639 conn(0x7f198006c980 msgr2=0x7f198006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 shutdown_connections 2024-04-26T23:42:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:24.972+0000 7f1987600640 1 -- 172.21.15.46:0/1081270639 wait complete. 2024-04-26T23:42:24.992 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-26T23:42:24.992 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-26T23:42:24.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-26T23:42:25.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.178+0000 7f0c2fe00640 1 Processor -- start 2024-04-26T23:42:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.178+0000 7f0c2fe00640 1 -- start start 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.180+0000 7f0c2fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0c300cdd00 0x7f0c300735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.180+0000 7f0c2fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c300ce7d0 con 0x7f0c30073b30 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.180+0000 7f0c2fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c300ceab0 con 0x7f0c3015cec0 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.180+0000 7f0c2fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0c300ced90 con 0x7f0c300cdd00 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0c30073b30 0x7f0c300b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46078/0 (socket says 172.21.15.46:46078) 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2f400640 1 -- 172.21.15.46:0/1772760547 learned_addr learned my addr 172.21.15.46:0/1772760547 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c26000640 1 --2- 172.21.15.46:0/1772760547 >> v2:172.21.15.142:3300/0 conn(0x7f0c300cdd00 0x7f0c300735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455408034 0 0) 0x7f0c300ce7d0 con 0x7f0c30073b30 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c14003520 con 0x7f0c30073b30 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2514501859 0 0) 0x7f0c14003520 con 0x7f0c30073b30 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 >> v2:172.21.15.142:3300/0 conn(0x7f0c300cdd00 msgr2=0x7f0c300735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.181+0000 7f0c2e000640 1 --2- 172.21.15.46:0/1772760547 >> v2:172.21.15.142:3300/0 conn(0x7f0c300cdd00 0x7f0c300735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 >> v1:172.21.15.142:6789/0 conn(0x7f0c3015cec0 legacy=0x7f0c300cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c300cf070 con 0x7f0c30073b30 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0c280019f0 con 0x7f0c30073b30 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 >> v1:172.21.15.46:6789/0 conn(0x7f0c30073b30 legacy=0x7f0c300b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c14004800 0x7f0c14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c140051c0 0x7f0c1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c300ced90 con 0x7f0c140039b0 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400c060 con 0x7f0c14004800 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1400c260 con 0x7f0c140051c0 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c26000640 1 --2- 172.21.15.46:0/1772760547 >> v2:172.21.15.142:3300/0 conn(0x7f0c300cdd00 0x7f0c300735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c26000640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c14004800 0x7f0c14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 -- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c140051c0 msgr2=0x7f0c1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c140051c0 0x7f0c1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 -- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c14004800 msgr2=0x7f0c14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c14004800 0x7f0c14004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2ea00640 1 -- 172.21.15.46:0/1772760547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14003520 con 0x7f0c140039b0 2024-04-26T23:42:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.182+0000 7f0c2f400640 1 -- 172.21.15.46:0/1772760547 reap_dead start 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c14003dd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0c20001750 tx=0x7f0c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c810fd8286a6aea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20001370 con 0x7f0c140039b0 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c200052d0 con 0x7f0c140039b0 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2e000640 1 -- 172.21.15.46:0/1772760547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2002abe0 con 0x7f0c140039b0 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2fe00640 1 -- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 msgr2=0x7f0c14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.184+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c14003dd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0c20001750 tx=0x7f0c14005830 comp rx=0 tx=0).stop 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.185+0000 7f0c2fe00640 1 -- 172.21.15.46:0/1772760547 shutdown_connections 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.185+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/1772760547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c14003dd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.185+0000 7f0c2fe00640 1 -- 172.21.15.46:0/1772760547 >> 172.21.15.46:0/1772760547 conn(0x7f0c300b27b0 msgr2=0x7f0c300b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.185+0000 7f0c2fe00640 1 -- 172.21.15.46:0/1772760547 shutdown_connections 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.185+0000 7f0c2fe00640 1 -- 172.21.15.46:0/1772760547 wait complete. 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.187+0000 7f0c2fe00640 1 Processor -- start 2024-04-26T23:42:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.187+0000 7f0c2fe00640 1 -- start start 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c3015daf0 0x7f0c301f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c301f88d0 0x7f0c300b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30073f70 con 0x7f0c140039b0 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3015d2a0 con 0x7f0c3015daf0 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c300735f0 con 0x7f0c301f88d0 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33124/0 (socket says 172.21.15.46:33124) 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.188+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2198512979 learned_addr learned my addr 172.21.15.46:0/2198512979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c26000640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c3015daf0 0x7f0c301f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c301f88d0 msgr2=0x7f0c300b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c301f88d0 0x7f0c300b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c3015daf0 msgr2=0x7f0c301f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c3015daf0 0x7f0c301f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2198512979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c300b0680 con 0x7f0c140039b0 2024-04-26T23:42:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005040 tx=0x7f0c20001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c27400640 1 -- 172.21.15.46:0/2198512979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20037040 con 0x7f0c140039b0 2024-04-26T23:42:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c27400640 1 -- 172.21.15.46:0/2198512979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c20040070 con 0x7f0c140039b0 2024-04-26T23:42:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c27400640 1 -- 172.21.15.46:0/2198512979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20001370 con 0x7f0c140039b0 2024-04-26T23:42:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.189+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c300b0900 con 0x7f0c140039b0 2024-04-26T23:42:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.190+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c300b0ec0 con 0x7f0c140039b0 2024-04-26T23:42:25.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.192+0000 7f0c27400640 1 -- 172.21.15.46:0/2198512979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c200052d0 con 0x7f0c140039b0 2024-04-26T23:42:25.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.193+0000 7f0c27400640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 0x7f0c0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.193+0000 7f0c27400640 1 -- 172.21.15.46:0/2198512979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0c200ab8f0 con 0x7f0c140039b0 2024-04-26T23:42:25.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.193+0000 7f0c26000640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 0x7f0c0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.193+0000 7f0c38581740 1 Processor -- start 2024-04-26T23:42:25.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 -- start start 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c26000640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 0x7f0c0c06ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c3015df10 tx=0x7f0c18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953fe0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 0x562953fb5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562953f1ac30 con 0x562953ffe2f0 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562953eb7250 con 0x562953fe0cd0 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.194+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562953f16bf0 con 0x562953fe1410 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33138/0 (socket says 172.21.15.46:33138) 2024-04-26T23:42:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2484156588 learned_addr learned my addr 172.21.15.46:0/2484156588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2f400640 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 0x562953fb5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 msgr2=0x562953fe0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953fe0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 msgr2=0x562953fb5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 0x562953fb5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 -- 172.21.15.46:0/2484156588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562953cf8b00 con 0x562953ffe2f0 2024-04-26T23:42:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2f400640 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 0x562953fb5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.195+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005620 tx=0x7f0c20079060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85afdd6ca326db89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c1f400640 1 -- 172.21.15.46:0/2484156588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200793f0 con 0x562953ffe2f0 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c1f400640 1 -- 172.21.15.46:0/2484156588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2007d7a0 con 0x562953ffe2f0 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c1f400640 1 -- 172.21.15.46:0/2484156588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20081fd0 con 0x562953ffe2f0 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 -- 172.21.15.46:0/2484156588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 msgr2=0x562953fb0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005620 tx=0x7f0c20079060 comp rx=0 tx=0).stop 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 -- 172.21.15.46:0/2484156588 shutdown_connections 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe0cd0 0x562953fb5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953fe0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 --2- 172.21.15.46:0/2484156588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fb0c80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 -- 172.21.15.46:0/2484156588 >> 172.21.15.46:0/2484156588 conn(0x562953fb2ef0 msgr2=0x562953ffb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 -- 172.21.15.46:0/2484156588 shutdown_connections 2024-04-26T23:42:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.196+0000 7f0c38581740 1 -- 172.21.15.46:0/2484156588 wait complete. 2024-04-26T23:42:25.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.197+0000 7f0c38581740 1 -- start start 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.197+0000 7f0c38581740 1 Processor -- start 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x562953fe44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953fce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562953eb7250 con 0x562953ffe2f0 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562953ea22e0 con 0x562953fe1410 2024-04-26T23:42:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.198+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562953f1ac30 con 0x562953ff2880 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c38581740 10 client.0 _ll_register_callbacks cb 0x562953f10300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33144/0 (socket says 172.21.15.46:33144) 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 -- 172.21.15.46:0/3847943776 learned_addr learned my addr 172.21.15.46:0/3847943776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x562953fe44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 -- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 msgr2=0x562953fce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953fce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 -- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 msgr2=0x562953fe44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x562953fe44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c26000640 1 -- 172.21.15.46:0/3847943776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562953bec3f0 con 0x562953ffe2f0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.199+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x562953fe44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c26000640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0c18006d40 tx=0x7f0c18009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9dd7a4b6fc914f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801d050 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0c18022640 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2007d7a0 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c1800f040 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c38581740 10 client.6207 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x562953d59290 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.200+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2007d7a0 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f0c18032020 con 0x562953ffe2f0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 10 client.6207 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 20 client.6207 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ffcdb0 0x562953fe1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 4 client.6207 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562953eb98c0 con 0x562953ffcdb0 2024-04-26T23:42:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ffcdb0 0x562953fe1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.201+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ffcdb0 0x562953fe1900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaedaff509c1f857 server_cookie=bf8f9f585e112ac4 in_seq=0 out_seq=0 2024-04-26T23:42:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c13e00640 10 client.6207 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c13e00640 1 -- 172.21.15.46:0/3847943776 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562953eb98c0 con 0x562953ffcdb0 2024-04-26T23:42:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c13e00640 10 client.6207 handle_command_reply: tid=0 2024-04-26T23:42:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c38581740 1 client.6207 shutdown 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 msgr2=0x562953fce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.202+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0c18006d40 tx=0x7f0c18009190 comp rx=0 tx=0).stop 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 shutdown_connections 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953fce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953fce0c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ffcdb0 0x562953fe1900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 --2- 172.21.15.46:0/3847943776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x562953fe44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c13e00640 0 client.6207 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 >> 172.21.15.46:0/3847943776 conn(0x562953fe0cd0 msgr2=0x562953fb03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 shutdown_connections 2024-04-26T23:42:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 1 -- 172.21.15.46:0/3847943776 wait complete. 2024-04-26T23:42:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.203+0000 7f0c38581740 20 client.6207 trim_cache size 0 max 16384 2024-04-26T23:42:25.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.222+0000 7f0c38581740 1 Processor -- start 2024-04-26T23:42:25.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.222+0000 7f0c38581740 1 -- start start 2024-04-26T23:42:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953ffb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953ffe2f0 0x562953fccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30202630 con 0x562953ff2880 2024-04-26T23:42:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30201630 con 0x562953ffe2f0 2024-04-26T23:42:25.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c30200990 con 0x562953fe1410 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.223+0000 7f0c26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33146/0 (socket says 172.21.15.46:33146) 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 -- 172.21.15.46:0/3356876886 learned_addr learned my addr 172.21.15.46:0/3356876886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 -- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 msgr2=0x562953ffb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953ffb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 -- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953ffe2f0 msgr2=0x562953fccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953ffe2f0 0x562953fccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 -- 172.21.15.46:0/3356876886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562953bc54b0 con 0x562953ff2880 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.224+0000 7f0c26000640 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a010 tx=0x7f0c18021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c8eb8cd0a0bc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c1ea00640 1 -- 172.21.15.46:0/3356876886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18037070 con 0x562953ff2880 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c1ea00640 1 -- 172.21.15.46:0/3356876886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c18027070 con 0x562953ff2880 2024-04-26T23:42:25.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c1ea00640 1 -- 172.21.15.46:0/3356876886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801d070 con 0x562953ff2880 2024-04-26T23:42:25.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 -- 172.21.15.46:0/3356876886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 msgr2=0x562953fcc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a010 tx=0x7f0c18021000 comp rx=0 tx=0).stop 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 -- 172.21.15.46:0/3356876886 shutdown_connections 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953ffe2f0 0x562953fccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ff2880 0x562953fcc570 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 --2- 172.21.15.46:0/3356876886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953fe1410 0x562953ffb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 -- 172.21.15.46:0/3356876886 >> 172.21.15.46:0/3356876886 conn(0x562953fe0cd0 msgr2=0x562953fb19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 -- 172.21.15.46:0/3356876886 shutdown_connections 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.225+0000 7f0c38581740 1 -- 172.21.15.46:0/3356876886 wait complete. 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.226+0000 7f0c38581740 1 -- start start 2024-04-26T23:42:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.226+0000 7f0c38581740 1 Processor -- start 2024-04-26T23:42:25.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x5629540780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953ff3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562953ffaef0 con 0x562953ffe2f0 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30200fe0 con 0x562953fe1410 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562953ff1120 con 0x562953ff2880 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c38581740 10 client.0 _ll_register_callbacks cb 0x7f0c18009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33156/0 (socket says 172.21.15.46:33156) 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.227+0000 7f0c2f400640 1 -- 172.21.15.46:0/1558756533 learned_addr learned my addr 172.21.15.46:0/1558756533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x5629540780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 -- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 msgr2=0x562953ff3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953ff3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 -- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 msgr2=0x5629540780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x5629540780a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 -- 172.21.15.46:0/1558756533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562953fdcfa0 con 0x562953ffe2f0 2024-04-26T23:42:25.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c2f400640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0c300cfe50 tx=0x7f0c28007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e3876f30143c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28007110 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0c28015c70 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.228+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c18027070 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c280072b0 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 10 client.6213 fetch_fsmap learned FSMap version 28 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x562953fb1380 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c18027070 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f0c2801c050 con 0x562953ffe2f0 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 10 client.6213 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 20 client.6213 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ff47f0 0x562953ffecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 4 client.6213 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-26T23:42:25.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) -- 0x562953f28840 con 0x562953ff47f0 2024-04-26T23:42:25.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.229+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ff47f0 0x562953ffecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.230+0000 7f0c13e00640 10 client.6213 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.230+0000 7f0c2ea00640 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ff47f0 0x562953ffecb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc00af1df20325 server_cookie=aa9d40bac996e19b in_seq=0 out_seq=0 2024-04-26T23:42:25.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c13e00640 1 -- 172.21.15.46:0/1558756533 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562953f28840 con 0x562953ff47f0 2024-04-26T23:42:25.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c13e00640 10 client.6213 handle_command_reply: tid=0 2024-04-26T23:42:25.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 client.6213 shutdown 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 msgr2=0x562953ff40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0c300cfe50 tx=0x7f0c28007f90 comp rx=0 tx=0).stop 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 shutdown_connections 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562953ffe2f0 0x562953ff40e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562953ff2880 0x562953ff3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x562953ff47f0 0x562953ffecb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 --2- 172.21.15.46:0/1558756533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562953fe1410 0x5629540780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 >> 172.21.15.46:0/1558756533 conn(0x562953fe0cd0 msgr2=0x562953fe4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c13e00640 0 client.6213 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 shutdown_connections 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 1 -- 172.21.15.46:0/1558756533 wait complete. 2024-04-26T23:42:25.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.231+0000 7f0c38581740 20 client.6213 trim_cache size 0 max 16384 2024-04-26T23:42:25.239 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "1971d7ea-aca8-4254-a726-78aa93397525", 2024-04-26T23:42:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:25.239 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 msgr2=0x7f0c0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 0x7f0c0c06ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c3015df10 tx=0x7f0c18008040 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 msgr2=0x7f0c30150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005040 tx=0x7f0c20001120 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 shutdown_connections 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0c06c920 0x7f0c0c06ede0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c301f88d0 0x7f0c300b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c3015daf0 0x7f0c301f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 --2- 172.21.15.46:0/2198512979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c140039b0 0x7f0c30150fa0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 >> 172.21.15.46:0/2198512979 conn(0x7f0c300b27b0 msgr2=0x7f0c30072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 shutdown_connections 2024-04-26T23:42:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.232+0000 7f0c2fe00640 1 -- 172.21.15.46:0/2198512979 wait complete. 2024-04-26T23:42:25.254 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '1971d7ea-aca8-4254-a726-78aa93397525', 'mode': 'asynchronous'}' 2024-04-26T23:42:25.254 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-26T23:42:25.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-26T23:42:25.304 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:42:25.300+0000 7f0755a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-26T23:42:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.450+0000 7fdbab000640 1 Processor -- start 2024-04-26T23:42:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.451+0000 7fdbab000640 1 -- start start 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.452+0000 7fdbab000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdba40cdd20 0x7fdba40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.452+0000 7fdbab000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdba40ae8f0 0x7fdba40aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.452+0000 7fdbab000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba40af570 con 0x7fdba415f210 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.452+0000 7fdbab000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdba40af850 con 0x7fdba40ae8f0 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.452+0000 7fdbab000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdba40cb680 con 0x7fdba40cdd20 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdbaa600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdba40ae8f0 0x7fdba40aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdba9c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdba415f210 0x7fdba40cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46094/0 (socket says 172.21.15.46:46094) 2024-04-26T23:42:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdba9c00640 1 -- 172.21.15.46:0/399120765 learned_addr learned my addr 172.21.15.46:0/399120765 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdba9200640 1 --2- 172.21.15.46:0/399120765 >> v2:172.21.15.142:3300/0 conn(0x7fdba40cdd20 0x7fdba40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdbaa600640 1 -- 172.21.15.46:0/399120765 >> v2:172.21.15.142:3300/0 conn(0x7fdba40cdd20 msgr2=0x7fdba40cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdbaa600640 1 --2- 172.21.15.46:0/399120765 >> v2:172.21.15.142:3300/0 conn(0x7fdba40cdd20 0x7fdba40cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdbaa600640 1 -- 172.21.15.46:0/399120765 >> v1:172.21.15.46:6789/0 conn(0x7fdba415f210 legacy=0x7fdba40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdbaa600640 1 -- 172.21.15.46:0/399120765 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba40cb960 con 0x7fdba40ae8f0 2024-04-26T23:42:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.453+0000 7fdba3e00640 1 -- 172.21.15.46:0/399120765 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032048873 0 0) 0x7fdba40af570 con 0x7fdba415f210 2024-04-26T23:42:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.454+0000 7fdbaa600640 1 --2- 172.21.15.46:0/399120765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba40aed00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdba40b3710 tx=0x7fdb9c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5e22f133187195 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.455+0000 7fdba3e00640 1 -- 172.21.15.46:0/399120765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c007740 con 0x7fdba40ae8f0 2024-04-26T23:42:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.455+0000 7fdba3e00640 1 -- 172.21.15.46:0/399120765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c0078e0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.455+0000 7fdba3e00640 1 -- 172.21.15.46:0/399120765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c007cb0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.455+0000 7fdbab000640 1 -- 172.21.15.46:0/399120765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 msgr2=0x7fdba40aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.455+0000 7fdbab000640 1 --2- 172.21.15.46:0/399120765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba40aed00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdba40b3710 tx=0x7fdb9c02a680 comp rx=0 tx=0).stop 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 -- 172.21.15.46:0/399120765 shutdown_connections 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 --2- 172.21.15.46:0/399120765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba40aed00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 --2- 172.21.15.46:0/399120765 >> v2:172.21.15.142:3300/0 conn(0x7fdba40cdd20 0x7fdba40cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 -- 172.21.15.46:0/399120765 >> 172.21.15.46:0/399120765 conn(0x7fdba40b27b0 msgr2=0x7fdba40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 -- 172.21.15.46:0/399120765 shutdown_connections 2024-04-26T23:42:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.456+0000 7fdbab000640 1 -- 172.21.15.46:0/399120765 wait complete. 2024-04-26T23:42:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.457+0000 7fdbab000640 1 Processor -- start 2024-04-26T23:42:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.458+0000 7fdbab000640 1 -- start start 2024-04-26T23:42:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.458+0000 7fdbab000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.458+0000 7fdbab000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdba40cdd20 0x7fdba40b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.458+0000 7fdbab000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdba415f210 0x7fdba40b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdbab000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdba415f5f0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdbab000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdba40af240 con 0x7fdba415f210 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdbab000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdba4071e80 con 0x7fdba40cdd20 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33174/0 (socket says 172.21.15.46:33174) 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 -- 172.21.15.46:0/1929456750 learned_addr learned my addr 172.21.15.46:0/1929456750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdbaa600640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdba415f210 0x7fdba40b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 -- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdba40cdd20 msgr2=0x7fdba40b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdba40cdd20 0x7fdba40b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 -- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdba415f210 msgr2=0x7fdba40b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdba415f210 0x7fdba40b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.459+0000 7fdba9c00640 1 -- 172.21.15.46:0/1929456750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba41f4ca0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdba9c00640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c460 tx=0x7fdb9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdba1600640 1 -- 172.21.15.46:0/1929456750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98018070 con 0x7fdba40ae8f0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdba1600640 1 -- 172.21.15.46:0/1929456750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb98007500 con 0x7fdba40ae8f0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdba1600640 1 -- 172.21.15.46:0/1929456750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98004a20 con 0x7fdba40ae8f0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba41f4f80 con 0x7fdba40ae8f0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.460+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41f4910 con 0x7fdba40ae8f0 2024-04-26T23:42:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.463+0000 7fdba1600640 1 -- 172.21.15.46:0/1929456750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb98004bc0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.464+0000 7fdba1600640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.464+0000 7fdba1600640 1 -- 172.21.15.46:0/1929456750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fdb9808afe0 con 0x7fdba40ae8f0 2024-04-26T23:42:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.464+0000 7fdbaf58f740 1 Processor -- start 2024-04-26T23:42:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.464+0000 7fdbaf58f740 1 -- start start 2024-04-26T23:42:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdba9200640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851f958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fc5920 0x556851f9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556851efdd50 con 0x556851fc6060 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556851e9a8c0 con 0x556851fe2f40 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.465+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556851ef9d10 con 0x556851fc5920 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33184/0 (socket says 172.21.15.46:33184) 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 -- 172.21.15.46:0/1039517596 learned_addr learned my addr 172.21.15.46:0/1039517596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9c00640 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851f958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 -- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fc5920 msgr2=0x556851f9a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fc5920 0x556851f9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 -- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 msgr2=0x556851f958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851f958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 -- 172.21.15.46:0/1039517596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556851cdbf00 con 0x556851fc6060 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9c00640 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851f958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 0x7fdb8406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdba40c6000 tx=0x7fdb90009d10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.466+0000 7fdba9200640 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000d8e0 tx=0x7fdb9001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0e7fdc3da02f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdb96000640 1 -- 172.21.15.46:0/1039517596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9001c070 con 0x556851fc6060 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdb96000640 1 -- 172.21.15.46:0/1039517596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90014af0 con 0x556851fc6060 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdb96000640 1 -- 172.21.15.46:0/1039517596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90021460 con 0x556851fc6060 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 -- 172.21.15.46:0/1039517596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 msgr2=0x556851fc53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000d8e0 tx=0x7fdb9001a000 comp rx=0 tx=0).stop 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 -- 172.21.15.46:0/1039517596 shutdown_connections 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fc5920 0x556851f9a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fc53e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/1039517596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851f958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.467+0000 7fdbaf58f740 1 -- 172.21.15.46:0/1039517596 >> 172.21.15.46:0/1039517596 conn(0x556851f97b40 msgr2=0x556851fe03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.468+0000 7fdbaf58f740 1 -- 172.21.15.46:0/1039517596 shutdown_connections 2024-04-26T23:42:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.468+0000 7fdbaf58f740 1 -- 172.21.15.46:0/1039517596 wait complete. 2024-04-26T23:42:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.468+0000 7fdbaf58f740 1 -- start start 2024-04-26T23:42:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.469+0000 7fdbaf58f740 1 Processor -- start 2024-04-26T23:42:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851fc49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556851f99cf0 con 0x556851fc6060 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556851ea2280 con 0x556851fe2f40 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556851f990e0 con 0x556851fb2f80 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdbaf58f740 10 client.0 _ll_register_callbacks cb 0x556851eab4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33186/0 (socket says 172.21.15.46:33186) 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 -- 172.21.15.46:0/2600369465 learned_addr learned my addr 172.21.15.46:0/2600369465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 -- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 msgr2=0x556851fc7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 -- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 msgr2=0x556851fc49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851fc49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.470+0000 7fdba9c00640 1 -- 172.21.15.46:0/2600369465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556851bcf7f0 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdb98089350 tx=0x7fdb980592b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7635c1a8ada5375a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdb8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98058200 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fdb9805f170 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb90014af0 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb98071170 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdbaf58f740 10 client.6228 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x556851d3c690 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.471+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90014af0 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fdb98054900 con 0x556851fc6060 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 10 client.6228 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 20 client.6228 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe1290 0x556851fe16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 4 client.6228 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556851ed9580 con 0x556851fe1290 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdba9200640 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe1290 0x556851fe16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdba9200640 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe1290 0x556851fe16f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0eeb300aaade70 server_cookie=bfe4233bb5fcb55e in_seq=0 out_seq=0 2024-04-26T23:42:25.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.472+0000 7fdb8aa00640 10 client.6228 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.473+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2600369465 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556851ed9580 con 0x556851fe1290 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.473+0000 7fdb8aa00640 10 client.6228 handle_command_reply: tid=0 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.473+0000 7fdbaf58f740 1 client.6228 shutdown 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.473+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 msgr2=0x556851fb36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.473+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdb98089350 tx=0x7fdb980592b0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 shutdown_connections 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fc7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe1290 0x556851fe16f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fe2f40 0x556851fc49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2600369465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fc6060 0x556851fb36b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdb8aa00640 0 client.6228 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 >> 172.21.15.46:0/2600369465 conn(0x556851fc5920 msgr2=0x556851f963a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 shutdown_connections 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2600369465 wait complete. 2024-04-26T23:42:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.474+0000 7fdbaf58f740 20 client.6228 trim_cache size 0 max 16384 2024-04-26T23:42:25.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.494+0000 7fdbaf58f740 1 Processor -- start 2024-04-26T23:42:25.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.494+0000 7fdbaf58f740 1 -- start start 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.494+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fdd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fe1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdba42019e0 con 0x556851fe2f40 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4200a60 con 0x556851fc6060 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdba41f8bd0 con 0x556851fb2f80 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33202/0 (socket says 172.21.15.46:33202) 2024-04-26T23:42:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 -- 172.21.15.46:0/3987603464 learned_addr learned my addr 172.21.15.46:0/3987603464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdba9200640 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fe1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdba9c00640 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fdd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 -- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 msgr2=0x556851fdd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fdd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 -- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 msgr2=0x556851fe1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fe1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.495+0000 7fdbaa600640 1 -- 172.21.15.46:0/3987603464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556851ba88b0 con 0x556851fe2f40 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.496+0000 7fdbaa600640 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0094e0 tx=0x7fdb9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a4bd164c6448a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.496+0000 7fdb95600640 1 -- 172.21.15.46:0/3987603464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0077c0 con 0x556851fe2f40 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.496+0000 7fdb95600640 1 -- 172.21.15.46:0/3987603464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c041070 con 0x556851fe2f40 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.496+0000 7fdb95600640 1 -- 172.21.15.46:0/3987603464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c007b80 con 0x556851fe2f40 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 -- 172.21.15.46:0/3987603464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 msgr2=0x556851fe2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0094e0 tx=0x7fdb9c004ab0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 -- 172.21.15.46:0/3987603464 shutdown_connections 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fe2f40 0x556851fe2510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fe1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/3987603464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fb2f80 0x556851fdd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 -- 172.21.15.46:0/3987603464 >> 172.21.15.46:0/3987603464 conn(0x556851fc5920 msgr2=0x556851f90ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 -- 172.21.15.46:0/3987603464 shutdown_connections 2024-04-26T23:42:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.497+0000 7fdbaf58f740 1 -- 172.21.15.46:0/3987603464 wait complete. 2024-04-26T23:42:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.498+0000 7fdbaf58f740 1 -- start start 2024-04-26T23:42:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.498+0000 7fdbaf58f740 1 Processor -- start 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fdd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fe2f40 0x556851fe3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4200410 con 0x556851fb2f80 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55685205d6a0 con 0x556851fc6060 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556851fb2290 con 0x556851fe2f40 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdbaf58f740 10 client.0 _ll_register_callbacks cb 0x7fdb98004eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdba9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33206/0 (socket says 172.21.15.46:33206) 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.499+0000 7fdba9c00640 1 -- 172.21.15.46:0/2532590378 learned_addr learned my addr 172.21.15.46:0/2532590378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 -- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fe2f40 msgr2=0x556851fe3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fe2f40 0x556851fe3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 -- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 msgr2=0x556851fdd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9200640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fdd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fdd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 -- 172.21.15.46:0/2532590378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556852059a50 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdba9c00640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb98056e60 tx=0x7fdb98089280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d26fe8a819a1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdb8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98078060 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fdb98057a70 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.500+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c041070 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb9807e050 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 10 client.6234 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55685205d440 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c041070 con 0x556851fb2f80 2024-04-26T23:42:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fdb98081020 con 0x556851fb2f80 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 10 client.6234 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 10 client.6234 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 20 client.6234 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe3f40 0x556851fc0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 4 client.6234 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-26T23:42:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) -- 0x556851f96f90 con 0x556851fe3f40 2024-04-26T23:42:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.501+0000 7fdba9200640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe3f40 0x556851fc0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.502+0000 7fdba9200640 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe3f40 0x556851fc0a80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55e6b189e1b3566 server_cookie=f3cf56f41371cac1 in_seq=0 out_seq=0 2024-04-26T23:42:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.502+0000 7fdb8aa00640 10 client.6234 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdb8aa00640 1 -- 172.21.15.46:0/2532590378 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556851f96f90 con 0x556851fe3f40 2024-04-26T23:42:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdb8aa00640 10 client.6234 handle_command_reply: tid=0 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 client.6234 shutdown 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 msgr2=0x556852059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb98056e60 tx=0x7fdb98089280 comp rx=0 tx=0).stop 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 shutdown_connections 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556851fe2f40 0x556851fe3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x556851fe3f40 0x556851fc0a80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556851fc6060 0x556851fdd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 --2- 172.21.15.46:0/2532590378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556851fb2f80 0x556852059510 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdb8aa00640 0 client.6234 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 >> 172.21.15.46:0/2532590378 conn(0x556851fc5920 msgr2=0x556851fb3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 shutdown_connections 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 1 -- 172.21.15.46:0/2532590378 wait complete. 2024-04-26T23:42:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.503+0000 7fdbaf58f740 20 client.6234 trim_cache size 0 max 16384 2024-04-26T23:42:25.511 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "afde6561-9792-40d2-afce-69022315ffc5", 2024-04-26T23:42:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:25.511 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 msgr2=0x7fdb8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 0x7fdb8406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdba40c6000 tx=0x7fdb90009d10 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 msgr2=0x7fdba4154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c460 tx=0x7fdb9800c920 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 shutdown_connections 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdba415f210 0x7fdba40b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdba40cdd20 0x7fdba40b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 --2- 172.21.15.46:0/1929456750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40ae8f0 0x7fdba4154140 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 >> 172.21.15.46:0/1929456750 conn(0x7fdba40b27b0 msgr2=0x7fdba40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 shutdown_connections 2024-04-26T23:42:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.504+0000 7fdbab000640 1 -- 172.21.15.46:0/1929456750 wait complete. 2024-04-26T23:42:25.526 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'afde6561-9792-40d2-afce-69022315ffc5', 'mode': 'asynchronous'}' 2024-04-26T23:42:25.526 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-26T23:42:25.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-26T23:42:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.730+0000 7f2a0d600640 1 Processor -- start 2024-04-26T23:42:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.730+0000 7f2a0d600640 1 -- start start 2024-04-26T23:42:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a0d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a080cdd00 0x7f2a080735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a0d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a08073b30 0x7f2a080b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a0d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a080ce7d0 con 0x7f2a0815cec0 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a0d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2a080ceab0 con 0x7f2a080cdd00 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a0d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2a080ced90 con 0x7f2a08073b30 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a07400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a080cdd00 0x7f2a080735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a07e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a0815cec0 0x7f2a080cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46102/0 (socket says 172.21.15.46:46102) 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.732+0000 7f2a07e00640 1 -- 172.21.15.46:0/921727999 learned_addr learned my addr 172.21.15.46:0/921727999 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a06a00640 1 -- 172.21.15.46:0/921727999 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238443689 0 0) 0x7f2a080ce7d0 con 0x7f2a0815cec0 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/921727999 >> v2:172.21.15.142:3300/0 conn(0x7f2a08073b30 0x7f2a080b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a06a00640 1 -- 172.21.15.46:0/921727999 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29ec003520 con 0x7f2a0815cec0 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a07400640 1 -- 172.21.15.46:0/921727999 >> v2:172.21.15.142:3300/0 conn(0x7f2a08073b30 msgr2=0x7f2a080b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a07400640 1 --2- 172.21.15.46:0/921727999 >> v2:172.21.15.142:3300/0 conn(0x7f2a08073b30 0x7f2a080b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a07400640 1 -- 172.21.15.46:0/921727999 >> v1:172.21.15.46:6789/0 conn(0x7f2a0815cec0 legacy=0x7f2a080cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.733+0000 7f2a07400640 1 -- 172.21.15.46:0/921727999 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a080cf070 con 0x7f2a080cdd00 2024-04-26T23:42:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.734+0000 7f2a07400640 1 --2- 172.21.15.46:0/921727999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080cdd00 0x7f2a080735f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a080c7c50 tx=0x7f29f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b567f821a409a573 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.734+0000 7f2a06a00640 1 -- 172.21.15.46:0/921727999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4037070 con 0x7f2a080cdd00 2024-04-26T23:42:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.734+0000 7f2a06a00640 1 -- 172.21.15.46:0/921727999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4032440 con 0x7f2a080cdd00 2024-04-26T23:42:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.734+0000 7f2a06a00640 1 -- 172.21.15.46:0/921727999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40327f0 con 0x7f2a080cdd00 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 -- 172.21.15.46:0/921727999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080cdd00 msgr2=0x7f2a080735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 --2- 172.21.15.46:0/921727999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080cdd00 0x7f2a080735f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a080c7c50 tx=0x7f29f402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 -- 172.21.15.46:0/921727999 shutdown_connections 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 --2- 172.21.15.46:0/921727999 >> v2:172.21.15.142:3300/0 conn(0x7f2a08073b30 0x7f2a080b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 --2- 172.21.15.46:0/921727999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080cdd00 0x7f2a080735f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 -- 172.21.15.46:0/921727999 >> 172.21.15.46:0/921727999 conn(0x7f2a080b27b0 msgr2=0x7f2a080b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 -- 172.21.15.46:0/921727999 shutdown_connections 2024-04-26T23:42:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.735+0000 7f2a0d600640 1 -- 172.21.15.46:0/921727999 wait complete. 2024-04-26T23:42:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.737+0000 7f2a0d600640 1 Processor -- start 2024-04-26T23:42:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.737+0000 7f2a0d600640 1 -- start start 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a080cdd00 0x7f2a081f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 0x7f2a081f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a08073f70 con 0x7f2a08073b30 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0815d2a0 con 0x7f2a0815cec0 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a0d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a080b4080 con 0x7f2a080cdd00 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.738+0000 7f2a07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33224/0 (socket says 172.21.15.46:33224) 2024-04-26T23:42:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 -- 172.21.15.46:0/2896783051 learned_addr learned my addr 172.21.15.46:0/2896783051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 0x7f2a081f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07400640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a080cdd00 0x7f2a081f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 -- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a080cdd00 msgr2=0x7f2a081f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a080cdd00 0x7f2a081f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 -- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 msgr2=0x7f2a081f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 0x7f2a081f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 -- 172.21.15.46:0/2896783051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0815b9a0 con 0x7f2a08073b30 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 0x7f2a081f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.739+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c390 tx=0x7f29f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.740+0000 7f29ffe00640 1 -- 172.21.15.46:0/2896783051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8017070 con 0x7f2a08073b30 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.740+0000 7f29ffe00640 1 -- 172.21.15.46:0/2896783051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8005ce0 con 0x7f2a08073b30 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.740+0000 7f29ffe00640 1 -- 172.21.15.46:0/2896783051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80072f0 con 0x7f2a08073b30 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.740+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081f8a90 con 0x7f2a08073b30 2024-04-26T23:42:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.740+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0815d7a0 con 0x7f2a08073b30 2024-04-26T23:42:25.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.742+0000 7f29ffe00640 1 -- 172.21.15.46:0/2896783051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29f801a070 con 0x7f2a08073b30 2024-04-26T23:42:25.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.743+0000 7f29ffe00640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.743+0000 7f29ffe00640 1 -- 172.21.15.46:0/2896783051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f29f808b1a0 con 0x7f2a08073b30 2024-04-26T23:42:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.744+0000 7f2a07400640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.744+0000 7f2a11be8740 1 Processor -- start 2024-04-26T23:42:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.744+0000 7f2a11be8740 1 -- start start 2024-04-26T23:42:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b33256d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a07400640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 0x7f29e806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f29f4034000 tx=0x7f29f402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 0x55b332572570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b3324d68d0 con 0x55b33259dea0 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b332473070 con 0x55b33259d760 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b3324d2890 con 0x55b3325bad80 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33228/0 (socket says 172.21.15.46:33228) 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.745+0000 7f2a07400640 1 -- 172.21.15.46:0/3036830990 learned_addr learned my addr 172.21.15.46:0/3036830990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 0x55b332572570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07e00640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b33256d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 -- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 msgr2=0x55b33256d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b33256d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 -- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 msgr2=0x55b332572570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 0x55b332572570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 -- 172.21.15.46:0/3036830990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33249f7b0 con 0x55b33259dea0 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 0x55b332572570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f2a07400640 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f29f40326f0 tx=0x7f29f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03b5543edcb85ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.746+0000 7f29f3e00640 1 -- 172.21.15.46:0/3036830990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4037040 con 0x55b33259dea0 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f29f3e00640 1 -- 172.21.15.46:0/3036830990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4032da0 con 0x55b33259dea0 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f29f3e00640 1 -- 172.21.15.46:0/3036830990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004230 con 0x55b33259dea0 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 -- 172.21.15.46:0/3036830990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 msgr2=0x55b33259d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f29f40326f0 tx=0x7f29f402c000 comp rx=0 tx=0).stop 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 -- 172.21.15.46:0/3036830990 shutdown_connections 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259d760 0x55b332572570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b33259d220 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 --2- 172.21.15.46:0/3036830990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b33256d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 -- 172.21.15.46:0/3036830990 >> 172.21.15.46:0/3036830990 conn(0x55b33256f980 msgr2=0x55b3325b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 -- 172.21.15.46:0/3036830990 shutdown_connections 2024-04-26T23:42:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.747+0000 7f2a11be8740 1 -- 172.21.15.46:0/3036830990 wait complete. 2024-04-26T23:42:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.748+0000 7f2a11be8740 1 -- start start 2024-04-26T23:42:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.748+0000 7f2a11be8740 1 Processor -- start 2024-04-26T23:42:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 0x55b3325c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 0x55b3325a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b332571b30 con 0x55b3325bad80 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b33245dec0 con 0x55b33259fa20 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b332570f20 con 0x55b33259dea0 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.749+0000 7f2a11be8740 10 client.0 _ll_register_callbacks cb 0x55b3324e3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33240/0 (socket says 172.21.15.46:33240) 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 -- 172.21.15.46:0/203492345 learned_addr learned my addr 172.21.15.46:0/203492345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07e00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 0x55b3325c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 0x55b3325a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 -- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 msgr2=0x55b3325c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 0x55b3325c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 -- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 msgr2=0x55b3325a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 0x55b3325a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07400640 1 -- 172.21.15.46:0/203492345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3322b4690 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a07e00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 0x55b3325c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.750+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 0x55b3325a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f2a07400640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f29f4049040 tx=0x7f29f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4a3a4d0950b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f29f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4056060 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f29f4044040 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f4032da0 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29f405c050 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f2a11be8740 10 client.6249 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55b332314e20 con 0x55b3325bad80 2024-04-26T23:42:25.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.751+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4032da0 con 0x55b3325bad80 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f29f405f020 con 0x55b3325bad80 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 10 client.6249 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 20 client.6249 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325cb200 0x55b3325b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 4 client.6249 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3324e44e0 con 0x55b3325cb200 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a07e00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325cb200 0x55b3325b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f2a07e00640 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325cb200 0x55b3325b01b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c1c2f53ec9c48f server_cookie=edc4385d2393a7e8 in_seq=0 out_seq=0 2024-04-26T23:42:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.752+0000 7f29f0c00640 10 client.6249 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f29f0c00640 1 -- 172.21.15.46:0/203492345 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3324e44e0 con 0x55b3325cb200 2024-04-26T23:42:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f29f0c00640 10 client.6249 handle_command_reply: tid=0 2024-04-26T23:42:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 client.6249 shutdown 2024-04-26T23:42:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 msgr2=0x55b33259f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f29f4049040 tx=0x7f29f4004730 comp rx=0 tx=0).stop 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 shutdown_connections 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325cb200 0x55b3325b01b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259fa20 0x55b3325a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3325bad80 0x55b33259f4e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 --2- 172.21.15.46:0/203492345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259dea0 0x55b3325c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f29f0c00640 0 client.6249 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.753+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 >> 172.21.15.46:0/203492345 conn(0x55b33259d760 msgr2=0x55b3325b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.754+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 shutdown_connections 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.754+0000 7f2a11be8740 1 -- 172.21.15.46:0/203492345 wait complete. 2024-04-26T23:42:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.754+0000 7f2a11be8740 20 client.6249 trim_cache size 0 max 16384 2024-04-26T23:42:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.774+0000 7f2a11be8740 1 Processor -- start 2024-04-26T23:42:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.774+0000 7f2a11be8740 1 -- start start 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 0x55b3325daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3325bad80 0x55b3325dafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a082018d0 con 0x55b33259dea0 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a082008e0 con 0x55b3325bad80 2024-04-26T23:42:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0815ee80 con 0x55b33259fa20 2024-04-26T23:42:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33248/0 (socket says 172.21.15.46:33248) 2024-04-26T23:42:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 -- 172.21.15.46:0/2999470343 learned_addr learned my addr 172.21.15.46:0/2999470343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07400640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 0x55b3325daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 -- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 msgr2=0x55b3325daa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 0x55b3325daa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a0cc00640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3325bad80 0x55b3325dafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.775+0000 7f2a07e00640 1 -- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3325bad80 msgr2=0x55b3325dafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3325bad80 0x55b3325dafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f2a07e00640 1 -- 172.21.15.46:0/2999470343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3321a7f80 con 0x55b33259dea0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f2a07400640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 0x55b3325daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f2a07e00640 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29f80070a0 tx=0x7f29f808b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12035955ebaf28fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f29f3400640 1 -- 172.21.15.46:0/2999470343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8087070 con 0x55b33259dea0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f29f3400640 1 -- 172.21.15.46:0/2999470343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8056d40 con 0x55b33259dea0 2024-04-26T23:42:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.776+0000 7f29f3400640 1 -- 172.21.15.46:0/2999470343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80711b0 con 0x55b33259dea0 2024-04-26T23:42:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 -- 172.21.15.46:0/2999470343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 msgr2=0x55b3325bb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29f80070a0 tx=0x7f29f808b040 comp rx=0 tx=0).stop 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 -- 172.21.15.46:0/2999470343 shutdown_connections 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3325bad80 0x55b3325dafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b33259fa20 0x55b3325daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 --2- 172.21.15.46:0/2999470343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259dea0 0x55b3325bb830 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 -- 172.21.15.46:0/2999470343 >> 172.21.15.46:0/2999470343 conn(0x55b33259d760 msgr2=0x55b3325b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 -- 172.21.15.46:0/2999470343 shutdown_connections 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.777+0000 7f2a11be8740 1 -- 172.21.15.46:0/2999470343 wait complete. 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.778+0000 7f2a11be8740 1 -- start start 2024-04-26T23:42:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.778+0000 7f2a11be8740 1 Processor -- start 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259dea0 0x55b3325c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b3325adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b3325b15c0 con 0x55b33259fa20 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a082002c0 con 0x55b33259dea0 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b3325a12f0 con 0x55b3325bad80 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a11be8740 10 client.0 _ll_register_callbacks cb 0x7f29f4032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33262/0 (socket says 172.21.15.46:33262) 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.779+0000 7f2a07400640 1 -- 172.21.15.46:0/1819512807 learned_addr learned my addr 172.21.15.46:0/1819512807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07e00640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259dea0 0x55b3325c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 -- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 msgr2=0x55b3325adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b3325adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 -- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259dea0 msgr2=0x55b3325c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259dea0 0x55b3325c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 -- 172.21.15.46:0/1819512807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b332572c80 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a07400640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f29f4032460 tx=0x7f29f4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d991a66d6c4793d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f29f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4037040 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f29f403c670 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.780+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f8056d40 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29f40040b0 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 10 client.6255 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55b332598320 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8056d40 con 0x55b33259fa20 2024-04-26T23:42:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f29f405f040 con 0x55b33259fa20 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 10 client.6255 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 20 client.6255 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325bbde0 0x55b3325ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 4 client.6255 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) -- 0x55b33256edd0 con 0x55b3325bbde0 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.781+0000 7f2a07e00640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325bbde0 0x55b3325ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f2a07e00640 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325bbde0 0x55b3325ae0f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4eb46d53159883 server_cookie=757a237475749dec in_seq=0 out_seq=0 2024-04-26T23:42:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f29f0c00640 10 client.6255 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f29f0c00640 1 -- 172.21.15.46:0/1819512807 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b33256edd0 con 0x55b3325bbde0 2024-04-26T23:42:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f29f0c00640 10 client.6255 handle_command_reply: tid=0 2024-04-26T23:42:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f2a11be8740 1 client.6255 shutdown 2024-04-26T23:42:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 msgr2=0x55b3325c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.782+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f29f4032460 tx=0x7f29f4008030 comp rx=0 tx=0).stop 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 shutdown_connections 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b3325bad80 0x55b3325adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b33259fa20 0x55b3325c7070 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55b3325bbde0 0x55b3325ae0f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 --2- 172.21.15.46:0/1819512807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b33259dea0 0x55b3325c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 >> 172.21.15.46:0/1819512807 conn(0x55b33259d760 msgr2=0x55b3325b9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f29f0c00640 0 client.6255 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 shutdown_connections 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 1 -- 172.21.15.46:0/1819512807 wait complete. 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a11be8740 20 client.6255 trim_cache size 0 max 16384 2024-04-26T23:42:25.790 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:25.791 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:25.791 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a4b69158-7363-4187-bda2-17bac2da0ae3", 2024-04-26T23:42:25.791 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:25.791 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 msgr2=0x7f29e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 0x7f29e806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f29f4034000 tx=0x7f29f402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.783+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 msgr2=0x7f2a08153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c390 tx=0x7f29f800c850 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 shutdown_connections 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0815cec0 0x7f2a081f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a080cdd00 0x7f2a081f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 --2- 172.21.15.46:0/2896783051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a08073b30 0x7f2a08153fa0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 >> 172.21.15.46:0/2896783051 conn(0x7f2a080b27b0 msgr2=0x7f2a080ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 shutdown_connections 2024-04-26T23:42:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:25.784+0000 7f2a0d600640 1 -- 172.21.15.46:0/2896783051 wait complete. 2024-04-26T23:42:25.805 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'a4b69158-7363-4187-bda2-17bac2da0ae3', 'mode': 'asynchronous'}' 2024-04-26T23:42:25.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:25.805 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-26T23:42:25.851 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-26T23:42:25.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.103+0000 7fb6f3200640 1 Processor -- start 2024-04-26T23:42:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.103+0000 7fb6f3200640 1 -- start start 2024-04-26T23:42:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.105+0000 7fb6f3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 0x7fb6f40cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.105+0000 7fb6f3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415f1d0 0x7fb6f40cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.105+0000 7fb6f3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f40b03c0 con 0x7fb6f40cb1a0 2024-04-26T23:42:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.105+0000 7fb6f3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f40cf5c0 con 0x7fb6f415f1d0 2024-04-26T23:42:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.105+0000 7fb6f3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f40cb680 con 0x7fb6f415edf0 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415f1d0 0x7fb6f40cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6f2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb6f40cb1a0 0x7fb6f40ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46110/0 (socket says 172.21.15.46:46110) 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6f2800640 1 -- 172.21.15.46:0/2758379424 learned_addr learned my addr 172.21.15.46:0/2758379424 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415f1d0 0x7fb6f40cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33270/0 (socket says 172.21.15.46:33270) 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/2758379424 >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 0x7fb6f40cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 -- 172.21.15.46:0/2758379424 >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 msgr2=0x7fb6f40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 --2- 172.21.15.46:0/2758379424 >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 0x7fb6f40cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 -- 172.21.15.46:0/2758379424 >> v1:172.21.15.46:6789/0 conn(0x7fb6f40cb1a0 legacy=0x7fb6f40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6ebe00640 1 -- 172.21.15.46:0/2758379424 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f40cb960 con 0x7fb6f415f1d0 2024-04-26T23:42:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.106+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/2758379424 >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 0x7fb6f40cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.107+0000 7fb6ebe00640 1 --2- 172.21.15.46:0/2758379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415f1d0 0x7fb6f40cd060 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40d0a00 tx=0x7fb6dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47016de641df2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.107+0000 7fb6f1400640 1 -- 172.21.15.46:0/2758379424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc037070 con 0x7fb6f415f1d0 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f1400640 1 -- 172.21.15.46:0/2758379424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc032440 con 0x7fb6f415f1d0 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f1400640 1 -- 172.21.15.46:0/2758379424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc032810 con 0x7fb6f415f1d0 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 -- 172.21.15.46:0/2758379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415f1d0 msgr2=0x7fb6f40cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 --2- 172.21.15.46:0/2758379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415f1d0 0x7fb6f40cd060 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40d0a00 tx=0x7fb6dc02a230 comp rx=0 tx=0).stop 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 -- 172.21.15.46:0/2758379424 shutdown_connections 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 --2- 172.21.15.46:0/2758379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415f1d0 0x7fb6f40cd060 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 --2- 172.21.15.46:0/2758379424 >> v2:172.21.15.142:3300/0 conn(0x7fb6f415edf0 0x7fb6f40cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 -- 172.21.15.46:0/2758379424 >> 172.21.15.46:0/2758379424 conn(0x7fb6f40b26d0 msgr2=0x7fb6f40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 -- 172.21.15.46:0/2758379424 shutdown_connections 2024-04-26T23:42:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.108+0000 7fb6f3200640 1 -- 172.21.15.46:0/2758379424 wait complete. 2024-04-26T23:42:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.110+0000 7fb6f3200640 1 Processor -- start 2024-04-26T23:42:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.110+0000 7fb6f3200640 1 -- start start 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 0x7fb6f40c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 0x7fb6f40b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40cc200 con 0x7fb6f415edf0 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40cbf20 con 0x7fb6f4153b60 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.111+0000 7fb6f3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f40c6950 con 0x7fb6f40cb1a0 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33280/0 (socket says 172.21.15.46:33280) 2024-04-26T23:42:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 -- 172.21.15.46:0/206183407 learned_addr learned my addr 172.21.15.46:0/206183407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6f2800640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 0x7fb6f40b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 0x7fb6f40c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 -- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 msgr2=0x7fb6f40c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 0x7fb6f40c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 -- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 msgr2=0x7fb6f40b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 0x7fb6f40b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 -- 172.21.15.46:0/206183407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f41f5030 con 0x7fb6f415edf0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 0x7fb6f40c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6f2800640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 0x7fb6f40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.112+0000 7fb6ebe00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc000c00 tx=0x7fb6dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.113+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc037040 con 0x7fb6f415edf0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.113+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc042070 con 0x7fb6f415edf0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.113+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc030a50 con 0x7fb6f415edf0 2024-04-26T23:42:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.113+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f40ca330 con 0x7fb6f415edf0 2024-04-26T23:42:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.113+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f41f52a0 con 0x7fb6f415edf0 2024-04-26T23:42:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.114+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f40693b0 con 0x7fb6f415edf0 2024-04-26T23:42:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.115+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6dc030bf0 con 0x7fb6f415edf0 2024-04-26T23:42:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.116+0000 7fb6ea000640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 0x7fb6d006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.116+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb6dc0ac760 con 0x7fb6f415edf0 2024-04-26T23:42:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.117+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 0x7fb6d006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.118+0000 7fb6f1e00640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 0x7fb6d006ee00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.121+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6dc045360 con 0x7fb6f415edf0 2024-04-26T23:42:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.375+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f415de10 con 0x7fb6f415edf0 2024-04-26T23:42:26.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.376+0000 7fb6ea000640 1 -- 172.21.15.46:0/206183407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5182 (secure 0 0 0) 0x7fb6dc07a9e0 con 0x7fb6f415edf0 2024-04-26T23:42:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:26.380 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:25.176741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:26.381 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-04-26T23:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 msgr2=0x7fb6d006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 0x7fb6d006ee00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4008040 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 msgr2=0x7fb6f41f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc000c00 tx=0x7fb6dc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 shutdown_connections 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6d006c940 0x7fb6d006ee00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6f4153b60 0x7fb6f40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415edf0 0x7fb6f41f57c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 --2- 172.21.15.46:0/206183407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6f40cb1a0 0x7fb6f40c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 >> 172.21.15.46:0/206183407 conn(0x7fb6f40b26d0 msgr2=0x7fb6f40af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 shutdown_connections 2024-04-26T23:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.379+0000 7fb6f3200640 1 -- 172.21.15.46:0/206183407 wait complete. 2024-04-26T23:42:26.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-26T23:42:26.626 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:26.626 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:26.642 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:26.643 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:26.643 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:26.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:26.697 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-26T23:42:26.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:42:26.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:42:26.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338839 s, 30.9 MB/s 2024-04-26T23:42:26.784 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-26T23:42:26.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:27.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.996+0000 7f01f9200640 1 Processor -- start 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.997+0000 7f01f9200640 1 -- start start 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.998+0000 7f01f9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01f40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f40d11f0 con 0x7f01f40d2130 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f9200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f40d5f20 con 0x7f01f40b4000 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01f40d0ce0 con 0x7f01f415f4e0 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f01f40d2130 0x7f01f40b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46124/0 (socket says 172.21.15.46:46124) 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f3e00640 1 -- 172.21.15.46:0/2593957688 learned_addr learned my addr 172.21.15.46:0/2593957688 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:26.999+0000 7f01f3400640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01f40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636954380 0 0) 0x7f01f40d11f0 con 0x7f01f40d2130 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01d4003520 con 0x7f01f40d2130 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274638123 0 0) 0x7f01f40d5f20 con 0x7f01f40b4000 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f40d11f0 con 0x7f01f40b4000 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1403964339 0 0) 0x7f01d4003520 con 0x7f01f40d2130 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 msgr2=0x7f01f40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01f40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 >> v1:172.21.15.142:6789/0 conn(0x7f01f40b4000 legacy=0x7f01f40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f40d0f10 con 0x7f01f40d2130 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f01ec001670 con 0x7f01f40d2130 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 >> v1:172.21.15.46:6789/0 conn(0x7f01f40d2130 legacy=0x7f01f40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d4003b70 0x7f01d4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400ba70 con 0x7f01d4004d40 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400bbf0 con 0x7f01d4003b70 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.000+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01d400bda0 con 0x7f01f415f4e0 2024-04-26T23:42:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3400640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d4003b70 0x7f01d4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f2a00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 -- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 msgr2=0x7f01d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 -- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d4003b70 msgr2=0x7f01d4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d4003b70 0x7f01d4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f3e00640 1 -- 172.21.15.46:0/2593957688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4003520 con 0x7f01d4004d40 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.001+0000 7f01f2a00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f415f4e0 0x7f01d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.003+0000 7f01f3e00640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c190 tx=0x7f01d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f5df98967e01b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.003+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007cc0 con 0x7f01d4004d40 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.003+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec005af0 con 0x7f01d4004d40 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.003+0000 7f01f2000640 1 -- 172.21.15.46:0/2593957688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007070 con 0x7f01d4004d40 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 -- 172.21.15.46:0/2593957688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 msgr2=0x7f01d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 --2- 172.21.15.46:0/2593957688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c190 tx=0x7f01d4005830 comp rx=0 tx=0).stop 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f3e00640 1 -- 172.21.15.46:0/2593957688 reap_dead start 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 -- 172.21.15.46:0/2593957688 shutdown_connections 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 -- 172.21.15.46:0/2593957688 >> 172.21.15.46:0/2593957688 conn(0x7f01f40d5010 msgr2=0x7f01f40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 -- 172.21.15.46:0/2593957688 shutdown_connections 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.004+0000 7f01f9200640 1 -- 172.21.15.46:0/2593957688 wait complete. 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.006+0000 7f01f9200640 1 Processor -- start 2024-04-26T23:42:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.006+0000 7f01f9200640 1 -- start start 2024-04-26T23:42:27.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f40c8970 0x7f01f414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01f414fc70 0x7f01f4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01f40b1d60 con 0x7f01d4004d40 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01f40d2990 con 0x7f01f414fc70 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.007+0000 7f01f9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01f415f8c0 con 0x7f01f40c8970 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33288/0 (socket says 172.21.15.46:33288) 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 -- 172.21.15.46:0/2865351148 learned_addr learned my addr 172.21.15.46:0/2865351148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3e00640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01f414fc70 0x7f01f4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f2a00640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f40c8970 0x7f01f414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 -- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f40c8970 msgr2=0x7f01f414f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f40c8970 0x7f01f414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 -- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01f414fc70 msgr2=0x7f01f4160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01f414fc70 0x7f01f4160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.008+0000 7f01f3400640 1 -- 172.21.15.46:0/2865351148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f414f990 con 0x7f01d4004d40 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01f3400640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f01e4011a00 tx=0x7f01e4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400fb50 con 0x7f01d4004d40 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e400fcf0 con 0x7f01d4004d40 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400fea0 con 0x7f01d4004d40 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f40cd610 con 0x7f01d4004d40 2024-04-26T23:42:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.009+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f40cdbd0 con 0x7f01d4004d40 2024-04-26T23:42:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.010+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f40693b0 con 0x7f01d4004d40 2024-04-26T23:42:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.011+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01e4012040 con 0x7f01d4004d40 2024-04-26T23:42:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.012+0000 7f01eb400640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 0x7f01d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.012+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f01e408c480 con 0x7f01d4004d40 2024-04-26T23:42:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.013+0000 7f01f2a00640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 0x7f01d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.014+0000 7f01f2a00640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 0x7f01d006edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01f40d59e0 tx=0x7f01dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.017+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01f414f990 con 0x7f01d4004d40 2024-04-26T23:42:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.262+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f40d33d0 con 0x7f01d4004d40 2024-04-26T23:42:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.263+0000 7f01eb400640 1 -- 172.21.15.46:0/2865351148 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5182 (secure 0 0 0) 0x7f01e4019030 con 0x7f01d4004d40 2024-04-26T23:42:27.268 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:27.268 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:25.176741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 msgr2=0x7f01d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 0x7f01d006edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01f40d59e0 tx=0x7f01dc009040 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 msgr2=0x7f01f415b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f01e4011a00 tx=0x7f01e4011ec0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 shutdown_connections 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f01d006c8f0 0x7f01d006edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01f414fc70 0x7f01f4160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01f40c8970 0x7f01f414f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 --2- 172.21.15.46:0/2865351148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4004d40 0x7f01f415b840 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 >> 172.21.15.46:0/2865351148 conn(0x7f01f40d5010 msgr2=0x7f01f40b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.266+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 shutdown_connections 2024-04-26T23:42:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.267+0000 7f01f9200640 1 -- 172.21.15.46:0/2865351148 wait complete. 2024-04-26T23:42:27.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-26T23:42:27.499 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:27.499 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:27.499 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:27.514 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:27.514 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:42:27.515 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:42:27.515 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:42:27.515 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:42:27.515 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-26T23:42:27.515 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:42:27.651 INFO:teuthology.orchestra.run.smithi046.stdout:1099511630414 2024-04-26T23:42:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:42:27.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-26T23:42:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.925+0000 7fadbbe00640 1 Processor -- start 2024-04-26T23:42:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.925+0000 7fadbbe00640 1 -- start start 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadbbe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fadbc145c10 0x7fadbc073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadbbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadbc06ec30 con 0x7fadbc0cfed0 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadbbe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadbc0cf3f0 con 0x7fadbc0d0f90 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadbbe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fadbc145ff0 con 0x7fadbc145c10 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadba000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fadbc0cfed0 0x7fadbc0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46130/0 (socket says 172.21.15.46:46130) 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.927+0000 7fadba000640 1 -- 172.21.15.46:0/2688940625 learned_addr learned my addr 172.21.15.46:0/2688940625 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadbaa00640 1 --2- 172.21.15.46:0/2688940625 >> v2:172.21.15.142:3300/0 conn(0x7fadbc145c10 0x7fadbc073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436880309 0 0) 0x7fadbc06ec30 con 0x7fadbc0cfed0 2024-04-26T23:42:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada0003520 con 0x7fadbc0cfed0 2024-04-26T23:42:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3929611012 0 0) 0x7fada0003520 con 0x7fadbc0cfed0 2024-04-26T23:42:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 >> v2:172.21.15.142:3300/0 conn(0x7fadbc145c10 msgr2=0x7fadbc073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 --2- 172.21.15.46:0/2688940625 >> v2:172.21.15.142:3300/0 conn(0x7fadbc145c10 0x7fadbc073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 >> v1:172.21.15.142:6789/0 conn(0x7fadbc0d0f90 legacy=0x7fadbc071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc0cf6d0 con 0x7fadbc0cfed0 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fada80019f0 con 0x7fadbc0cfed0 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 >> v1:172.21.15.46:6789/0 conn(0x7fadbc0cfed0 legacy=0x7fadbc0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fada00039b0 0x7fada0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00051c0 0x7fada000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc145ff0 con 0x7fada0004800 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fada000c060 con 0x7fada00051c0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.928+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fada000c260 con 0x7fada00039b0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadbaa00640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fada00039b0 0x7fada0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadbb400640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00051c0 0x7fada000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 -- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fada00039b0 msgr2=0x7fada0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fada00039b0 0x7fada0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 -- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00051c0 msgr2=0x7fada000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00051c0 0x7fada000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadba000640 1 -- 172.21.15.46:0/2688940625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0003520 con 0x7fada0004800 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.929+0000 7fadbb400640 1 -- 172.21.15.46:0/2688940625 reap_dead start 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.930+0000 7fadba000640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fada0004c80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fada800a840 tx=0x7fada0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10996c37ceba42de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada800a460 con 0x7fada0004800 2024-04-26T23:42:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada802f070 con 0x7fada0004800 2024-04-26T23:42:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadb9600640 1 -- 172.21.15.46:0/2688940625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada802bcf0 con 0x7fada0004800 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 -- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 msgr2=0x7fada0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fada0004c80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fada800a840 tx=0x7fada0005830 comp rx=0 tx=0).stop 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 -- 172.21.15.46:0/2688940625 shutdown_connections 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 --2- 172.21.15.46:0/2688940625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 -- 172.21.15.46:0/2688940625 >> 172.21.15.46:0/2688940625 conn(0x7fadbc06c350 msgr2=0x7fadbc06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 -- 172.21.15.46:0/2688940625 shutdown_connections 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.931+0000 7fadbbe00640 1 -- 172.21.15.46:0/2688940625 wait complete. 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.933+0000 7fadbbe00640 1 Processor -- start 2024-04-26T23:42:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.933+0000 7fadbbe00640 1 -- start start 2024-04-26T23:42:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadbc15c680 0x7fadbc1e6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadbc1f6c80 0x7fadbc0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1468b0 con 0x7fada0004800 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0cf0a0 con 0x7fadbc15c680 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc0d0360 con 0x7fadbc1f6c80 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.934+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33306/0 (socket says 172.21.15.46:33306) 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 -- 172.21.15.46:0/58415354 learned_addr learned my addr 172.21.15.46:0/58415354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbb400640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadbc1f6c80 0x7fadbc0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadba000640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadbc15c680 0x7fadbc1e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 -- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadbc1f6c80 msgr2=0x7fadbc0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadbc1f6c80 0x7fadbc0b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 -- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadbc15c680 msgr2=0x7fadbc1e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadbc15c680 0x7fadbc1e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 -- 172.21.15.46:0/58415354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1e70b0 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.935+0000 7fadbaa00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fadac0114e0 tx=0x7fadac0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.936+0000 7fadb2a00640 1 -- 172.21.15.46:0/58415354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac015870 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.936+0000 7fadb2a00640 1 -- 172.21.15.46:0/58415354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac015a10 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.936+0000 7fadb2a00640 1 -- 172.21.15.46:0/58415354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac015dc0 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.936+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1e7390 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.936+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1f6480 con 0x7fada0004800 2024-04-26T23:42:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.938+0000 7fadb2a00640 1 -- 172.21.15.46:0/58415354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadac013400 con 0x7fada0004800 2024-04-26T23:42:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.939+0000 7fadb2a00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 0x7fad9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.939+0000 7fadb2a00640 1 -- 172.21.15.46:0/58415354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fadac08c630 con 0x7fada0004800 2024-04-26T23:42:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.940+0000 7fadc45ed740 1 Processor -- start 2024-04-26T23:42:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.940+0000 7fadc45ed740 1 -- start start 2024-04-26T23:42:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.940+0000 7fadba000640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 0x7fad9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1dc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 0x55fcf1d99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fcf1cfbc70 con 0x55fcf1ddee00 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fcf1cb4980 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55fcf1cf7c30 con 0x55fcf1d93100 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33314/0 (socket says 172.21.15.46:33314) 2024-04-26T23:42:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadbb400640 1 -- 172.21.15.46:0/1047685683 learned_addr learned my addr 172.21.15.46:0/1047685683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadba000640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 0x7fad9c06edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fada8033000 tx=0x7fada8038040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadba000640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 0x55fcf1d99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.941+0000 7fadbaa00640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1dc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 -- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 msgr2=0x55fcf1d99130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 0x55fcf1d99130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 -- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 msgr2=0x55fcf1dc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1dc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 -- 172.21.15.46:0/1047685683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf1c8c7b0 con 0x55fcf1ddee00 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadba000640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 0x55fcf1d99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbaa00640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1dc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fadbb400640 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0b3a60 tx=0x7fadb4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c0aa5ac0dedbce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fada7000640 1 -- 172.21.15.46:0/1047685683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40052e0 con 0x55fcf1ddee00 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fada7000640 1 -- 172.21.15.46:0/1047685683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb4018070 con 0x55fcf1ddee00 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.942+0000 7fada7000640 1 -- 172.21.15.46:0/1047685683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003830 con 0x55fcf1ddee00 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 -- 172.21.15.46:0/1047685683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 msgr2=0x55fcf1ddf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0b3a60 tx=0x7fadb4007f90 comp rx=0 tx=0).stop 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 -- 172.21.15.46:0/1047685683 shutdown_connections 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1ddf280 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1d93100 0x55fcf1d99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 --2- 172.21.15.46:0/1047685683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1dc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 -- 172.21.15.46:0/1047685683 >> 172.21.15.46:0/1047685683 conn(0x55fcf1d95370 msgr2=0x55fcf1dddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 -- 172.21.15.46:0/1047685683 shutdown_connections 2024-04-26T23:42:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.943+0000 7fadc45ed740 1 -- 172.21.15.46:0/1047685683 wait complete. 2024-04-26T23:42:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.944+0000 7fadc45ed740 1 -- start start 2024-04-26T23:42:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.944+0000 7fadc45ed740 1 Processor -- start 2024-04-26T23:42:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dc12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 0x55fcf1dabac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fcf1cfaf80 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fcf1ca8260 con 0x55fcf1ddee00 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55fcf1d002a0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.945+0000 7fadc45ed740 10 client.0 _ll_register_callbacks cb 0x55fcf1cf2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33326/0 (socket says 172.21.15.46:33326) 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 -- 172.21.15.46:0/562405671 learned_addr learned my addr 172.21.15.46:0/562405671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadba000640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dc12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbb400640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 0x55fcf1dabac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 -- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 msgr2=0x55fcf1dabac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 0x55fcf1dabac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 -- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 msgr2=0x55fcf1dc12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dc12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 -- 172.21.15.46:0/562405671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf1c8a1c0 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbb400640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 0x55fcf1dabac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadba000640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dc12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fadbaa00640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fadac08a960 tx=0x7fadac05e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbbd18ae08780f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.946+0000 7fad9b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac06edd0 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fadac062910 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4018070 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadac062b30 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 10 client.6282 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55fcf1a0d3f0 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb4018070 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fadac01cbc0 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 10 client.6282 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 10 client.6282 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 20 client.6282 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1dbdb40 0x55fcf1dbdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 4 client.6282 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.947+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcf1d09880 con 0x55fcf1dbdb40 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.948+0000 7fadba000640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1dbdb40 0x55fcf1dbdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.948+0000 7fadba000640 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1dbdb40 0x55fcf1dbdfd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb114cc06d77a6f server_cookie=f73131b3a0ab0814 in_seq=0 out_seq=0 2024-04-26T23:42:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.948+0000 7fad9b400640 10 client.6282 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.949+0000 7fad9b400640 1 -- 172.21.15.46:0/562405671 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcf1d09880 con 0x55fcf1dbdb40 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.949+0000 7fad9b400640 10 client.6282 handle_command_reply: tid=0 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.949+0000 7fadc45ed740 1 client.6282 shutdown 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.949+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 msgr2=0x55fcf1da92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.949+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fadac08a960 tx=0x7fadac05e6b0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 shutdown_connections 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc17e0 0x55fcf1dabac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1dbdb40 0x55fcf1dbdfd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dc12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 --2- 172.21.15.46:0/562405671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc2ea0 0x55fcf1da92e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fad9b400640 0 client.6282 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 >> 172.21.15.46:0/562405671 conn(0x55fcf1d93100 msgr2=0x55fcf1dde610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 shutdown_connections 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 1 -- 172.21.15.46:0/562405671 wait complete. 2024-04-26T23:42:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.950+0000 7fadc45ed740 20 client.6282 trim_cache size 0 max 16384 2024-04-26T23:42:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.965+0000 7fadc45ed740 1 Processor -- start 2024-04-26T23:42:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.965+0000 7fadc45ed740 1 -- start start 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc17e0 0x55fcf1ddaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1fe020 con 0x55fcf1ddee00 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0b57b0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc142d20 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33342/0 (socket says 172.21.15.46:33342) 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.966+0000 7fadbb400640 1 -- 172.21.15.46:0/3664974889 learned_addr learned my addr 172.21.15.46:0/3664974889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadba000640 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbaa00640 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc17e0 0x55fcf1ddaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 -- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 msgr2=0x55fcf1d97f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d97f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 -- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc17e0 msgr2=0x55fcf1ddaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc17e0 0x55fcf1ddaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 -- 172.21.15.46:0/3664974889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf1ad77d0 con 0x55fcf1ddee00 2024-04-26T23:42:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.967+0000 7fadbb400640 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fadb4014000 tx=0x7fadb4005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61912f63ff398b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fada6600640 1 -- 172.21.15.46:0/3664974889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4007240 con 0x55fcf1ddee00 2024-04-26T23:42:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fada6600640 1 -- 172.21.15.46:0/3664974889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb4021030 con 0x55fcf1ddee00 2024-04-26T23:42:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fada6600640 1 -- 172.21.15.46:0/3664974889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4007640 con 0x55fcf1ddee00 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 -- 172.21.15.46:0/3664974889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 msgr2=0x55fcf1d98480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fadb4014000 tx=0x7fadb4005000 comp rx=0 tx=0).stop 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 -- 172.21.15.46:0/3664974889 shutdown_connections 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1ddee00 0x55fcf1d98480 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3664974889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1dc17e0 0x55fcf1ddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 -- 172.21.15.46:0/3664974889 >> 172.21.15.46:0/3664974889 conn(0x55fcf1d93100 msgr2=0x55fcf1dc0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 -- 172.21.15.46:0/3664974889 shutdown_connections 2024-04-26T23:42:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.968+0000 7fadc45ed740 1 -- 172.21.15.46:0/3664974889 wait complete. 2024-04-26T23:42:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.969+0000 7fadc45ed740 1 -- start start 2024-04-26T23:42:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.969+0000 7fadc45ed740 1 Processor -- start 2024-04-26T23:42:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dbf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0b50e0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fcf1d99680 con 0x55fcf1ddee00 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55fcf1dc02c0 con 0x55fcf1dc2ea0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.970+0000 7fadc45ed740 10 client.0 _ll_register_callbacks cb 0x7fadac059d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33356/0 (socket says 172.21.15.46:33356) 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 -- 172.21.15.46:0/3297168317 learned_addr learned my addr 172.21.15.46:0/3297168317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbb400640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dbf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadba000640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 -- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 msgr2=0x55fcf1d91f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d91f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 -- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 msgr2=0x55fcf1dbf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dbf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 -- 172.21.15.46:0/3297168317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf1ca22a0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbb400640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dbf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fadbaa00640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fadac062870 tx=0x7fadac01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d151058af75aace server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.971+0000 7fad9b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac07e060 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fadac01cbc0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4021030 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadac059f90 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 10 client.6288 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55fcf1d92c40 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb4021030 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fadac06ebd0 con 0x55fcf1dc17e0 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 10 client.6288 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 10 client.6288 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 20 client.6288 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1e5bd00 0x55fcf1dbfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 4 client.6288 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:42:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.972+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcf1d036f0 con 0x55fcf1e5bd00 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fadba000640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1e5bd00 0x55fcf1dbfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fadba000640 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1e5bd00 0x55fcf1dbfa00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98fa1c5bb4d1af0e server_cookie=bc220d39ce9f4eef in_seq=0 out_seq=0 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fad9b400640 10 client.6288 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fad9b400640 1 -- 172.21.15.46:0/3297168317 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcf1d036f0 con 0x55fcf1e5bd00 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fad9b400640 10 client.6288 handle_command_reply: tid=0 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.973+0000 7fadc45ed740 1 client.6288 shutdown 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 msgr2=0x55fcf1e5ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fadac062870 tx=0x7fadac01f040 comp rx=0 tx=0).stop 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 shutdown_connections 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55fcf1e5bd00 0x55fcf1dbfa00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcf1ddee00 0x55fcf1dbf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55fcf1dc2ea0 0x55fcf1d91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 --2- 172.21.15.46:0/3297168317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcf1dc17e0 0x55fcf1e5ab50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fad9b400640 0 client.6288 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 >> 172.21.15.46:0/3297168317 conn(0x55fcf1d93100 msgr2=0x55fcf1d9a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 shutdown_connections 2024-04-26T23:42:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 1 -- 172.21.15.46:0/3297168317 wait complete. 2024-04-26T23:42:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.974+0000 7fadc45ed740 20 client.6288 trim_cache size 0 max 16384 2024-04-26T23:42:27.982 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 msgr2=0x7fad9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 0x7fad9c06edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fada8033000 tx=0x7fada8038040 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 msgr2=0x7fadbc13a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fadac0114e0 tx=0x7fadac0119a0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 shutdown_connections 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fad9c06c8f0 0x7fad9c06edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadbc1f6c80 0x7fadbc0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadbc15c680 0x7fadbc1e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 --2- 172.21.15.46:0/58415354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0004800 0x7fadbc13a030 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 >> 172.21.15.46:0/58415354 conn(0x7fadbc06c350 msgr2=0x7fadbc15ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 shutdown_connections 2024-04-26T23:42:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:27.975+0000 7fadbbe00640 1 -- 172.21.15.46:0/58415354 wait complete. 2024-04-26T23:42:27.997 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4e.00000000 parent 2024-04-26T23:42:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.087+0000 7f541991bf40 1 Processor -- start 2024-04-26T23:42:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 -- start start 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x56027895bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278a00b20 0x56027895c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602788d0d20 con 0x560278927d20 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602789272d0 con 0x5602789715a0 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.088+0000 7f541991bf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56027895ae60 con 0x560278a00b20 2024-04-26T23:42:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33366/0 (socket says 172.21.15.46:33366) 2024-04-26T23:42:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 -- 172.21.15.46:0/3820389943 learned_addr learned my addr 172.21.15.46:0/3820389943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5415600640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x56027895bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416a00640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278a00b20 0x56027895c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 -- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278a00b20 msgr2=0x56027895c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278a00b20 0x56027895c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 -- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 msgr2=0x56027895bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x56027895bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.089+0000 7f5416000640 1 -- 172.21.15.46:0/3820389943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027895b140 con 0x560278927d20 2024-04-26T23:42:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.090+0000 7f5415600640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x56027895bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.090+0000 7f5416000640 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5408009750 tx=0x7f5408005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771295a9edf70386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.090+0000 7f540fe00640 1 -- 172.21.15.46:0/3820389943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408019070 con 0x560278927d20 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.090+0000 7f540fe00640 1 -- 172.21.15.46:0/3820389943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5408008890 con 0x560278927d20 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.090+0000 7f540fe00640 1 -- 172.21.15.46:0/3820389943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408008c60 con 0x560278927d20 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 -- 172.21.15.46:0/3820389943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 msgr2=0x5602789ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5408009750 tx=0x7f5408005a00 comp rx=0 tx=0).stop 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 -- 172.21.15.46:0/3820389943 shutdown_connections 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278a00b20 0x56027895c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x56027895bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 --2- 172.21.15.46:0/3820389943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278927d20 0x5602789ff2e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 -- 172.21.15.46:0/3820389943 >> 172.21.15.46:0/3820389943 conn(0x56027892ea00 msgr2=0x56027895a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 -- 172.21.15.46:0/3820389943 shutdown_connections 2024-04-26T23:42:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.091+0000 7f541991bf40 1 -- 172.21.15.46:0/3820389943 wait complete. 2024-04-26T23:42:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.092+0000 7f541991bf40 1 Processor -- start 2024-04-26T23:42:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.092+0000 7f541991bf40 1 -- start start 2024-04-26T23:42:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278927d20 0x5602789fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x560278955640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 0x560278955b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560278957ea0 con 0x560278a00b20 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56027895b420 con 0x5602789715a0 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f541991bf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560278a001e0 con 0x560278927d20 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 0x560278955b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 0x560278955b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33374/0 (socket says 172.21.15.46:33374) 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416a00640 1 -- 172.21.15.46:0/3702080906 learned_addr learned my addr 172.21.15.46:0/3702080906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5415600640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x560278955640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416000640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278927d20 0x5602789fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416a00640 1 -- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278927d20 msgr2=0x5602789fbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.093+0000 7f5416a00640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560278927d20 0x5602789fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f5416a00640 1 -- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 msgr2=0x560278955640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f5416a00640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x560278955640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f5416a00640 1 -- 172.21.15.46:0/3702080906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560278a9ab00 con 0x560278a00b20 2024-04-26T23:42:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f5415600640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602789715a0 0x560278955640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f5416a00640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 0x560278955b80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5410045ed0 tx=0x7f5410002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f540ea00640 1 -- 172.21.15.46:0/3702080906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410054b70 con 0x560278a00b20 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f540ea00640 1 -- 172.21.15.46:0/3702080906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5410054d30 con 0x560278a00b20 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f540ea00640 1 -- 172.21.15.46:0/3702080906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410068670 con 0x560278a00b20 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560278a9ade0 con 0x560278a00b20 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.094+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560278a9b3a0 con 0x560278a00b20 2024-04-26T23:42:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.096+0000 7f540ea00640 1 -- 172.21.15.46:0/3702080906 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5410071440 con 0x560278a00b20 2024-04-26T23:42:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.097+0000 7f540ea00640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53f806b2d0 0x7f53f806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.097+0000 7f540ea00640 1 -- 172.21.15.46:0/3702080906 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f54100cf630 con 0x560278a00b20 2024-04-26T23:42:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.098+0000 7f541991bf40 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x56027896c130 0x56027896c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.098+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 --> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] -- osd_op(unknown.0.0:1 8.2 8:43adcc81:::10000000a4e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) -- 0x56027896cac0 con 0x56027896c130 2024-04-26T23:42:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.098+0000 7f5416000640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53f806b2d0 0x7f53f806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.098+0000 7f5415600640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x56027896c130 0x56027896c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.099+0000 7f5416000640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53f806b2d0 0x7f53f806d790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x560278958bb0 tx=0x7f5408008000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.100+0000 7f5415600640 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x56027896c130 0x56027896c580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f5415600640 1 -- 172.21.15.46:0/3702080906 <== osd.4 v2:172.21.15.142:6824/2432053156 1 ==== osd_op_reply(1 10000000a4e.00000000 [rmxattr] v54'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f54040025d0 con 0x56027896c130 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x56027896c130 msgr2=0x56027896c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x56027896c130 0x56027896c580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53f806b2d0 msgr2=0x7f53f806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53f806b2d0 0x7f53f806d790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x560278958bb0 tx=0x7f5408008000 comp rx=0 tx=0).stop 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 msgr2=0x560278955b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f541991bf40 1 --2- 172.21.15.46:0/3702080906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560278a00b20 0x560278955b80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5410045ed0 tx=0x7f5410002fb0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.102+0000 7f5416a00640 1 -- 172.21.15.46:0/3702080906 reap_dead start 2024-04-26T23:42:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.103+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 shutdown_connections 2024-04-26T23:42:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.103+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 >> 172.21.15.46:0/3702080906 conn(0x56027892ea00 msgr2=0x56027895a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.103+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 shutdown_connections 2024-04-26T23:42:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.103+0000 7f541991bf40 1 -- 172.21.15.46:0/3702080906 wait complete. 2024-04-26T23:42:28.112 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-26T23:42:28.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-26T23:42:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.315+0000 7f8bbec00640 1 Processor -- start 2024-04-26T23:42:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.315+0000 7f8bbec00640 1 -- start start 2024-04-26T23:42:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8bb81640d0 0x7f8bb81644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bb8073af0 0x7f8bb80cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb81649f0 con 0x7f8bb80cbec0 2024-04-26T23:42:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb80cb4f0 con 0x7f8bb8073af0 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb8071e80 con 0x7f8bb81640d0 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bb8073af0 0x7f8bb80cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bb8073af0 0x7f8bb80cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33376/0 (socket says 172.21.15.46:33376) 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.317+0000 7f8bbce00640 1 -- 172.21.15.46:0/211113071 learned_addr learned my addr 172.21.15.46:0/211113071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.318+0000 7f8bbd800640 1 --2- 172.21.15.46:0/211113071 >> v2:172.21.15.142:3300/0 conn(0x7f8bb81640d0 0x7f8bb81644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.318+0000 7f8bbce00640 1 -- 172.21.15.46:0/211113071 >> v2:172.21.15.142:3300/0 conn(0x7f8bb81640d0 msgr2=0x7f8bb81644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.318+0000 7f8bbce00640 1 --2- 172.21.15.46:0/211113071 >> v2:172.21.15.142:3300/0 conn(0x7f8bb81640d0 0x7f8bb81644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.318+0000 7f8bbce00640 1 -- 172.21.15.46:0/211113071 >> v1:172.21.15.46:6789/0 conn(0x7f8bb80cbec0 legacy=0x7f8bb80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.318+0000 7f8bbce00640 1 -- 172.21.15.46:0/211113071 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb8072160 con 0x7f8bb8073af0 2024-04-26T23:42:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.319+0000 7f8bbce00640 1 --2- 172.21.15.46:0/211113071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb8073af0 0x7f8bb80cb980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b6d70 tx=0x7f8ba402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97f27d8b0285b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.319+0000 7f8bb7e00640 1 -- 172.21.15.46:0/211113071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4037070 con 0x7f8bb8073af0 2024-04-26T23:42:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.319+0000 7f8bb7e00640 1 -- 172.21.15.46:0/211113071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4032440 con 0x7f8bb8073af0 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.319+0000 7f8bb7e00640 1 -- 172.21.15.46:0/211113071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4032810 con 0x7f8bb8073af0 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 -- 172.21.15.46:0/211113071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb8073af0 msgr2=0x7f8bb80cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 --2- 172.21.15.46:0/211113071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb8073af0 0x7f8bb80cb980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b6d70 tx=0x7f8ba402a270 comp rx=0 tx=0).stop 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 -- 172.21.15.46:0/211113071 shutdown_connections 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 --2- 172.21.15.46:0/211113071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb8073af0 0x7f8bb80cb980 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 --2- 172.21.15.46:0/211113071 >> v2:172.21.15.142:3300/0 conn(0x7f8bb81640d0 0x7f8bb81644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 -- 172.21.15.46:0/211113071 >> 172.21.15.46:0/211113071 conn(0x7f8bb80b27b0 msgr2=0x7f8bb80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 -- 172.21.15.46:0/211113071 shutdown_connections 2024-04-26T23:42:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.320+0000 7f8bbec00640 1 -- 172.21.15.46:0/211113071 wait complete. 2024-04-26T23:42:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.322+0000 7f8bbec00640 1 Processor -- start 2024-04-26T23:42:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.322+0000 7f8bbec00640 1 -- start start 2024-04-26T23:42:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bb81640d0 0x7f8bb80c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 0x7f8bb8165e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb80ceee0 con 0x7f8bb80cbec0 2024-04-26T23:42:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8073f70 con 0x7f8bb80c9dc0 2024-04-26T23:42:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb80cf060 con 0x7f8bb81640d0 2024-04-26T23:42:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33386/0 (socket says 172.21.15.46:33386) 2024-04-26T23:42:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.323+0000 7f8bbd800640 1 -- 172.21.15.46:0/450260189 learned_addr learned my addr 172.21.15.46:0/450260189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbe200640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 0x7f8bb8165e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 -- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bb81640d0 msgr2=0x7f8bb80c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bb81640d0 0x7f8bb80c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 -- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 msgr2=0x7f8bb8165e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 0x7f8bb8165e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 -- 172.21.15.46:0/450260189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb80c7d70 con 0x7f8bb80cbec0 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbe200640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 0x7f8bb8165e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bbd800640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00d620 tx=0x7f8bac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.324+0000 7f8bb5600640 1 -- 172.21.15.46:0/450260189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00b620 con 0x7f8bb80cbec0 2024-04-26T23:42:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.325+0000 7f8bb5600640 1 -- 172.21.15.46:0/450260189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bac019070 con 0x7f8bb80cbec0 2024-04-26T23:42:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.325+0000 7f8bb5600640 1 -- 172.21.15.46:0/450260189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ba00 con 0x7f8bb80cbec0 2024-04-26T23:42:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.325+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb8155670 con 0x7f8bb80cbec0 2024-04-26T23:42:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.325+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb80c7fe0 con 0x7f8bb80cbec0 2024-04-26T23:42:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.327+0000 7f8bb5600640 1 -- 172.21.15.46:0/450260189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bac00bba0 con 0x7f8bb80cbec0 2024-04-26T23:42:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.328+0000 7f8bb5600640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 0x7f8b9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.328+0000 7f8bb5600640 1 -- 172.21.15.46:0/450260189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8bac08c170 con 0x7f8bb80cbec0 2024-04-26T23:42:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.329+0000 7f8bc3026740 1 Processor -- start 2024-04-26T23:42:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.329+0000 7f8bbce00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 0x7f8b9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.329+0000 7f8bc3026740 1 -- start start 2024-04-26T23:42:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c5cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 0x55ed8c5d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c538e10 con 0x55ed8c600020 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c4d5120 con 0x55ed8c5ff8e0 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ed8c534dd0 con 0x55ed8c61cf00 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bbce00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 0x7f8b9806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33394/0 (socket says 172.21.15.46:33394) 2024-04-26T23:42:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.330+0000 7f8bbce00640 1 -- 172.21.15.46:0/2754616272 learned_addr learned my addr 172.21.15.46:0/2754616272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbd800640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c5cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbe200640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 0x55ed8c5d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 -- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 msgr2=0x55ed8c5cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c5cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 -- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 msgr2=0x55ed8c5d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 0x55ed8c5d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 -- 172.21.15.46:0/2754616272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c316c00 con 0x55ed8c600020 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbe200640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 0x55ed8c5d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.331+0000 7f8bbce00640 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4006c80 tx=0x7f8ba402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77151661368e1f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8baa000640 1 -- 172.21.15.46:0/2754616272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba404d070 con 0x55ed8c600020 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8baa000640 1 -- 172.21.15.46:0/2754616272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4032dc0 con 0x55ed8c600020 2024-04-26T23:42:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8baa000640 1 -- 172.21.15.46:0/2754616272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4037040 con 0x55ed8c600020 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 -- 172.21.15.46:0/2754616272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 msgr2=0x55ed8c5ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4006c80 tx=0x7f8ba402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 -- 172.21.15.46:0/2754616272 shutdown_connections 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c5ff8e0 0x55ed8c5d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c5ff3a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2754616272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c5cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.332+0000 7f8bc3026740 1 -- 172.21.15.46:0/2754616272 >> 172.21.15.46:0/2754616272 conn(0x55ed8c5d1b00 msgr2=0x55ed8c61a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.333+0000 7f8bc3026740 1 -- 172.21.15.46:0/2754616272 shutdown_connections 2024-04-26T23:42:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.333+0000 7f8bc3026740 1 -- 172.21.15.46:0/2754616272 wait complete. 2024-04-26T23:42:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.333+0000 7f8bc3026740 1 -- start start 2024-04-26T23:42:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.334+0000 7f8bc3026740 1 Processor -- start 2024-04-26T23:42:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c601b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c612690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c5d3cb0 con 0x55ed8c600020 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c4bfc40 con 0x55ed8c612210 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ed8c5d30a0 con 0x55ed8c61cf00 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bc3026740 10 client.0 _ll_register_callbacks cb 0x55ed8c4e0b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33410/0 (socket says 172.21.15.46:33410) 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 -- 172.21.15.46:0/2562559339 learned_addr learned my addr 172.21.15.46:0/2562559339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbe200640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c612690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 -- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 msgr2=0x55ed8c601b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c601b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 -- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 msgr2=0x55ed8c612690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c612690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbd800640 1 -- 172.21.15.46:0/2562559339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c20a4f0 con 0x55ed8c600020 2024-04-26T23:42:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.335+0000 7f8bbe200640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c612690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8bbd800640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8bac08a410 tx=0x7f8bac05a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4e2239801d149f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8b9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac072020 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8bac0562e0 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba4032dc0 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bac062850 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8bc3026740 10 client.6309 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55ed8c377390 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.336+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4032dc0 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f8bac013070 con 0x55ed8c600020 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 10 client.6309 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 10 client.6309 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 20 client.6309 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5edad0 0x55ed8c5edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 4 client.6309 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8c546a20 con 0x55ed8c5edad0 2024-04-26T23:42:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.337+0000 7f8bbce00640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5edad0 0x55ed8c5edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.338+0000 7f8bbce00640 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5edad0 0x55ed8c5edf50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f761dc7c96214160 server_cookie=9f3c62346db7af8e in_seq=0 out_seq=0 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.338+0000 7f8b9ea00640 10 client.6309 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.338+0000 7f8b9ea00640 1 -- 172.21.15.46:0/2562559339 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed8c546a20 con 0x55ed8c5edad0 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.338+0000 7f8b9ea00640 10 client.6309 handle_command_reply: tid=0 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.338+0000 7f8bc3026740 1 client.6309 shutdown 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 msgr2=0x55ed8c61bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8bac08a410 tx=0x7f8bac05a430 comp rx=0 tx=0).stop 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 shutdown_connections 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5edad0 0x55ed8c5edf50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c612690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c61cf00 0x55ed8c601b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 --2- 172.21.15.46:0/2562559339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c600020 0x55ed8c61bac0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8b9ea00640 0 client.6309 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 >> 172.21.15.46:0/2562559339 conn(0x55ed8c5ff8e0 msgr2=0x55ed8c61d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 shutdown_connections 2024-04-26T23:42:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 1 -- 172.21.15.46:0/2562559339 wait complete. 2024-04-26T23:42:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.339+0000 7f8bc3026740 20 client.6309 trim_cache size 0 max 16384 2024-04-26T23:42:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 Processor -- start 2024-04-26T23:42:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 -- start start 2024-04-26T23:42:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c5d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c63ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8206680 con 0x55ed8c61cf00 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8205660 con 0x55ed8c612210 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.359+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb80c5c20 con 0x55ed8c600020 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33412/0 (socket says 172.21.15.46:33412) 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 -- 172.21.15.46:0/1454550448 learned_addr learned my addr 172.21.15.46:0/1454550448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbce00640 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c63ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbd800640 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c5d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 -- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 msgr2=0x55ed8c5d5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c5d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 -- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 msgr2=0x55ed8c63ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c63ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.360+0000 7f8bbe200640 1 -- 172.21.15.46:0/1454550448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c1e3670 con 0x55ed8c61cf00 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.361+0000 7f8bbe200640 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0001cc0 tx=0x7f8bb0002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e174e51ffd3bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.361+0000 7f8ba9600640 1 -- 172.21.15.46:0/1454550448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003da0 con 0x55ed8c61cf00 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.361+0000 7f8ba9600640 1 -- 172.21.15.46:0/1454550448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb0018070 con 0x55ed8c61cf00 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.361+0000 7f8ba9600640 1 -- 172.21.15.46:0/1454550448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000fb90 con 0x55ed8c61cf00 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 -- 172.21.15.46:0/1454550448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 msgr2=0x55ed8c63e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0001cc0 tx=0x7f8bb0002180 comp rx=0 tx=0).stop 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 -- 172.21.15.46:0/1454550448 shutdown_connections 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c63e2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c63ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 --2- 172.21.15.46:0/1454550448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c5d5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 -- 172.21.15.46:0/1454550448 >> 172.21.15.46:0/1454550448 conn(0x55ed8c5ff8e0 msgr2=0x55ed8c5edeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 -- 172.21.15.46:0/1454550448 shutdown_connections 2024-04-26T23:42:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.362+0000 7f8bc3026740 1 -- 172.21.15.46:0/1454550448 wait complete. 2024-04-26T23:42:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.363+0000 7f8bc3026740 1 -- start start 2024-04-26T23:42:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.363+0000 7f8bc3026740 1 Processor -- start 2024-04-26T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c6014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c601a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c4d91c0 con 0x55ed8c61cf00 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8205080 con 0x55ed8c612210 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ed8c5fae00 con 0x55ed8c600020 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bc3026740 10 client.0 _ll_register_callbacks cb 0x7f8bac00d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33416/0 (socket says 172.21.15.46:33416) 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.364+0000 7f8bbe200640 1 -- 172.21.15.46:0/3566948178 learned_addr learned my addr 172.21.15.46:0/3566948178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 -- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 msgr2=0x55ed8c6014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbce00640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c601a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c6014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 -- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 msgr2=0x55ed8c601a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c601a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 -- 172.21.15.46:0/3566948178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c629b30 con 0x55ed8c61cf00 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bbe200640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0015040 tx=0x7f8bb0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfc7a9349b85b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8b9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000fda0 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8bb00050d0 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.365+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb0018070 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bb002f050 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 10 client.6315 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55ed8c63df50 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb0018070 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f8bb0032020 con 0x55ed8c61cf00 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 10 client.6315 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 10 client.6315 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 20 client.6315 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5ef780 0x55ed8c697d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 4 client.6315 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) -- 0x55ed8c540890 con 0x55ed8c5ef780 2024-04-26T23:42:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.366+0000 7f8bbd800640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5ef780 0x55ed8c697d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.367+0000 7f8bbd800640 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5ef780 0x55ed8c697d90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1129c86557d2744 server_cookie=3a86aa7cd42bfe77 in_seq=0 out_seq=0 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.367+0000 7f8b9ea00640 10 client.6315 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.367+0000 7f8b9ea00640 1 -- 172.21.15.46:0/3566948178 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed8c540890 con 0x55ed8c5ef780 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.367+0000 7f8b9ea00640 10 client.6315 handle_command_reply: tid=0 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.367+0000 7f8bc3026740 1 client.6315 shutdown 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 msgr2=0x55ed8c697850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0015040 tx=0x7f8bb0000f40 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 shutdown_connections 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8c61cf00 0x55ed8c697850 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55ed8c5ef780 0x55ed8c697d90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8c612210 0x55ed8c601a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 --2- 172.21.15.46:0/3566948178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ed8c600020 0x55ed8c6014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8b9ea00640 0 client.6315 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 >> 172.21.15.46:0/3566948178 conn(0x55ed8c5ff8e0 msgr2=0x55ed8c5d5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 shutdown_connections 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 1 -- 172.21.15.46:0/3566948178 wait complete. 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f8bc3026740 20 client.6315 trim_cache size 0 max 16384 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7f25f28b-3f3f-4b5e-90bf-3bf53f668c08", 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:28.375 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:42:28.368+0000 7f45aaa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v467 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4967=0-4194304@1} caps={4967=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4967 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cdae700680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4e:[<0x10000000001/new_file_1 v467>,<0x10000000000/ceph-qa-suite v79>,<0x1/scrub_test_path v34>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 msgr2=0x7f8b9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 0x7f8b9806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba402c000 comp rx=0 tx=0).stop 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 msgr2=0x7f8bb8155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00d620 tx=0x7f8bac00dae0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 shutdown_connections 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b9806c8f0 0x7f8b9806edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb80c9dc0 0x7f8bb8165e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bb81640d0 0x7f8bb80c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 --2- 172.21.15.46:0/450260189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bb80cbec0 0x7f8bb8155130 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 >> 172.21.15.46:0/450260189 conn(0x7f8bb80b27b0 msgr2=0x7f8bb80d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 shutdown_connections 2024-04-26T23:42:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.369+0000 7f8bbec00640 1 -- 172.21.15.46:0/450260189 wait complete. 2024-04-26T23:42:28.390 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '7f25f28b-3f3f-4b5e-90bf-3bf53f668c08', 'mode': 'asynchronous'}' 2024-04-26T23:42:28.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:42:28.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-26T23:42:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.598+0000 7f9e8a200640 1 Processor -- start 2024-04-26T23:42:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.599+0000 7f9e8a200640 1 -- start start 2024-04-26T23:42:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.600+0000 7f9e8a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8415d730 0x7f9e840d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.600+0000 7f9e8a200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e8406eb90 con 0x7f9e840b53b0 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e8a200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e840b4310 con 0x7f9e8415e220 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e8a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9e840b45f0 con 0x7f9e8415d730 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8415d730 0x7f9e840d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8415d730 0x7f9e840d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33424/0 (socket says 172.21.15.46:33424) 2024-04-26T23:42:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 -- 172.21.15.46:0/1578768192 learned_addr learned my addr 172.21.15.46:0/1578768192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 -- 172.21.15.46:0/1578768192 >> v1:172.21.15.142:6789/0 conn(0x7f9e8415e220 legacy=0x7f9e8406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 -- 172.21.15.46:0/1578768192 >> v1:172.21.15.46:6789/0 conn(0x7f9e840b53b0 legacy=0x7f9e8415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.601+0000 7f9e83400640 1 -- 172.21.15.46:0/1578768192 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e840b48d0 con 0x7f9e8415d730 2024-04-26T23:42:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.603+0000 7f9e83400640 1 --2- 172.21.15.46:0/1578768192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e840d00d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009e50 tx=0x7f9e7402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03926e5af707762 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.603+0000 7f9e82000640 1 -- 172.21.15.46:0/1578768192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74036070 con 0x7f9e8415d730 2024-04-26T23:42:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.603+0000 7f9e82000640 1 -- 172.21.15.46:0/1578768192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e74031600 con 0x7f9e8415d730 2024-04-26T23:42:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.603+0000 7f9e82000640 1 -- 172.21.15.46:0/1578768192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e740319d0 con 0x7f9e8415d730 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 -- 172.21.15.46:0/1578768192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 msgr2=0x7f9e840d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1578768192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e840d00d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009e50 tx=0x7f9e7402a760 comp rx=0 tx=0).stop 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 -- 172.21.15.46:0/1578768192 shutdown_connections 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1578768192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e840d00d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 -- 172.21.15.46:0/1578768192 >> 172.21.15.46:0/1578768192 conn(0x7f9e8406c980 msgr2=0x7f9e840b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 -- 172.21.15.46:0/1578768192 shutdown_connections 2024-04-26T23:42:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.604+0000 7f9e8a200640 1 -- 172.21.15.46:0/1578768192 wait complete. 2024-04-26T23:42:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.606+0000 7f9e8a200640 1 Processor -- start 2024-04-26T23:42:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.606+0000 7f9e8a200640 1 -- start start 2024-04-26T23:42:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 0x7f9e840c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e8415e220 0x7f9e84146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e840b3ec0 con 0x7f9e8415d730 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e840b4e90 con 0x7f9e840b53b0 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e8a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9e840b4040 con 0x7f9e8415e220 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33438/0 (socket says 172.21.15.46:33438) 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.607+0000 7f9e82a00640 1 -- 172.21.15.46:0/1737218870 learned_addr learned my addr 172.21.15.46:0/1737218870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e83400640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 0x7f9e840c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 -- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e8415e220 msgr2=0x7f9e84146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e83e00640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e8415e220 0x7f9e84146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e8415e220 0x7f9e84146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 -- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 msgr2=0x7f9e840c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 0x7f9e840c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 -- 172.21.15.46:0/1737218870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e84144e00 con 0x7f9e8415d730 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e83400640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 0x7f9e840c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e82a00640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c3c0 tx=0x7f9e6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e7b400640 1 -- 172.21.15.46:0/1737218870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0052d0 con 0x7f9e8415d730 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e7b400640 1 -- 172.21.15.46:0/1737218870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c005490 con 0x7f9e8415d730 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e7b400640 1 -- 172.21.15.46:0/1737218870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c007260 con 0x7f9e8415d730 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.608+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e84145080 con 0x7f9e8415d730 2024-04-26T23:42:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.609+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e84145640 con 0x7f9e8415d730 2024-04-26T23:42:28.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.612+0000 7f9e7b400640 1 -- 172.21.15.46:0/1737218870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e6c007400 con 0x7f9e8415d730 2024-04-26T23:42:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.613+0000 7f9e7b400640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 0x7f9e6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.613+0000 7f9e7b400640 1 -- 172.21.15.46:0/1737218870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9e6c08b270 con 0x7f9e8415d730 2024-04-26T23:42:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.613+0000 7f9e8d05b740 1 Processor -- start 2024-04-26T23:42:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.613+0000 7f9e83400640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 0x7f9e6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.613+0000 7f9e8d05b740 1 -- start start 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2afbde0 0x5583f2afc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583f29c4030 con 0x5583f2ab4220 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5583f29df7f0 con 0x5583f2ade540 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.614+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5583f299a7e0 con 0x5583f2afbde0 2024-04-26T23:42:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83400640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 0x7f9e6006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e74033000 tx=0x7f9e74009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33448/0 (socket says 172.21.15.46:33448) 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 -- 172.21.15.46:0/3813138661 learned_addr learned my addr 172.21.15.46:0/3813138661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83400640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e82a00640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2afbde0 0x5583f2afc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 -- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2afbde0 msgr2=0x5583f2afc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2afbde0 0x5583f2afc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 -- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 msgr2=0x5583f2ade920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83e00640 1 -- 172.21.15.46:0/3813138661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f29a6050 con 0x5583f2ab4220 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.615+0000 7f9e83400640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e83e00640 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c003db0 tx=0x7f9e7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d371ad994ab52440 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e73400640 1 -- 172.21.15.46:0/3813138661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c016020 con 0x5583f2ab4220 2024-04-26T23:42:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e73400640 1 -- 172.21.15.46:0/3813138661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c00db60 con 0x5583f2ab4220 2024-04-26T23:42:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e73400640 1 -- 172.21.15.46:0/3813138661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0123f0 con 0x5583f2ab4220 2024-04-26T23:42:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e8d05b740 1 -- 172.21.15.46:0/3813138661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 msgr2=0x5583f2afa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.616+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c003db0 tx=0x7f9e7c004270 comp rx=0 tx=0).stop 2024-04-26T23:42:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 -- 172.21.15.46:0/3813138661 shutdown_connections 2024-04-26T23:42:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2ab4220 0x5583f2afa4a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2afbde0 0x5583f2afc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/3813138661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 -- 172.21.15.46:0/3813138661 >> 172.21.15.46:0/3813138661 conn(0x5583f2ab0b50 msgr2=0x5583f2af9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 -- 172.21.15.46:0/3813138661 shutdown_connections 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 -- 172.21.15.46:0/3813138661 wait complete. 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.617+0000 7f9e8d05b740 1 -- start start 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.618+0000 7f9e8d05b740 1 Processor -- start 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ad6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2afbde0 0x5583f2ae1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583f2a1c200 con 0x5583f2b09680 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5583f2a16ee0 con 0x5583f2afbde0 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5583f2a23e70 con 0x5583f2ade540 2024-04-26T23:42:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e8d05b740 10 client.0 _ll_register_callbacks cb 0x5583f2a0d270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33458/0 (socket says 172.21.15.46:33458) 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 -- 172.21.15.46:0/2593266973 learned_addr learned my addr 172.21.15.46:0/2593266973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83400640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ad6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 -- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 msgr2=0x5583f2ad6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ad6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 -- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2afbde0 msgr2=0x5583f2ae1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2afbde0 0x5583f2ae1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.619+0000 7f9e83e00640 1 -- 172.21.15.46:0/2593266973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f27f37d0 con 0x5583f2b09680 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e83e00640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c015040 tx=0x7f9e7c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7381047868c03e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e67e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c029060 con 0x5583f2b09680 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9e7c005450 con 0x5583f2b09680 2024-04-26T23:42:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7c00db60 con 0x5583f2b09680 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c02f050 con 0x5583f2b09680 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e8d05b740 10 client.6330 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.620+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x5583f27293f0 con 0x5583f2b09680 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c00db60 con 0x5583f2b09680 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f9e7c032020 con 0x5583f2b09680 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 10 client.6330 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 20 client.6330 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2ae15b0 0x5583f2ae0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 4 client.6330 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f2a257e0 con 0x5583f2ae15b0 2024-04-26T23:42:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.621+0000 7f9e83400640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2ae15b0 0x5583f2ae0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.622+0000 7f9e83400640 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2ae15b0 0x5583f2ae0210 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee187476d6ae55c3 server_cookie=22fe283bac0ebae4 in_seq=0 out_seq=0 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.622+0000 7f9e67e00640 10 client.6330 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.622+0000 7f9e67e00640 1 -- 172.21.15.46:0/2593266973 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583f2a257e0 con 0x5583f2ae15b0 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e67e00640 10 client.6330 handle_command_reply: tid=0 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 client.6330 shutdown 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 msgr2=0x5583f2aad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c015040 tx=0x7f9e7c0036f0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 shutdown_connections 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2b09680 0x5583f2aad7b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2ae15b0 0x5583f2ae0210 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2afbde0 0x5583f2ae1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/2593266973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ad6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e67e00640 0 client.6330 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 >> 172.21.15.46:0/2593266973 conn(0x5583f2ab4220 msgr2=0x5583f2ab20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 shutdown_connections 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 1 -- 172.21.15.46:0/2593266973 wait complete. 2024-04-26T23:42:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.623+0000 7f9e8d05b740 20 client.6330 trim_cache size 0 max 16384 2024-04-26T23:42:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.642+0000 7f9e8d05b740 1 Processor -- start 2024-04-26T23:42:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 -- start start 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ab2e80 0x5583f2ab2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e842020a0 con 0x5583f2afbde0 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84201080 con 0x5583f2ade540 2024-04-26T23:42:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.643+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9e841437a0 con 0x5583f2ab2e80 2024-04-26T23:42:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33468/0 (socket says 172.21.15.46:33468) 2024-04-26T23:42:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 -- 172.21.15.46:0/344011617 learned_addr learned my addr 172.21.15.46:0/344011617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e83400640 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 -- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ab2e80 msgr2=0x5583f2ab2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ab2e80 0x5583f2ab2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 -- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 msgr2=0x5583f2ade920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 -- 172.21.15.46:0/344011617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f2aadb20 con 0x5583f2afbde0 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.644+0000 7f9e82a00640 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c390 tx=0x7f9e6c058ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7a572d41fbe5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e72a00640 1 -- 172.21.15.46:0/344011617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c087070 con 0x5583f2afbde0 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e72a00640 1 -- 172.21.15.46:0/344011617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c08b0c0 con 0x5583f2afbde0 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e72a00640 1 -- 172.21.15.46:0/344011617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c071070 con 0x5583f2afbde0 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 -- 172.21.15.46:0/344011617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 msgr2=0x5583f2ab20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c390 tx=0x7f9e6c058ec0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 -- 172.21.15.46:0/344011617 shutdown_connections 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ab2e80 0x5583f2ab2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2ab20f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/344011617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ade540 0x5583f2ade920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 -- 172.21.15.46:0/344011617 >> 172.21.15.46:0/344011617 conn(0x5583f2ab4220 msgr2=0x5583f2ab5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 -- 172.21.15.46:0/344011617 shutdown_connections 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.645+0000 7f9e8d05b740 1 -- 172.21.15.46:0/344011617 wait complete. 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.646+0000 7f9e8d05b740 1 -- start start 2024-04-26T23:42:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.646+0000 7f9e8d05b740 1 Processor -- start 2024-04-26T23:42:28.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ab2e80 0x5583f2ab20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ab2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583f2a24fd0 con 0x5583f2afbde0 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84200a40 con 0x5583f2ab2e80 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5583f2b0c9d0 con 0x5583f2ade540 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.647+0000 7f9e8d05b740 10 client.0 _ll_register_callbacks cb 0x7f9e7c003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33478/0 (socket says 172.21.15.46:33478) 2024-04-26T23:42:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 -- 172.21.15.46:0/4083808399 learned_addr learned my addr 172.21.15.46:0/4083808399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83400640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ab2e80 0x5583f2ab20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e82a00640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ab2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 -- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 msgr2=0x5583f2ab2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ab2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 -- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ab2e80 msgr2=0x5583f2ab20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ab2e80 0x5583f2ab20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e83e00640 1 -- 172.21.15.46:0/4083808399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f26e90f0 con 0x5583f2afbde0 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.648+0000 7f9e82a00640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ab2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e83e00640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c004510 tx=0x7f9e7c003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0f3dfcf5211093 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e67e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c016030 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9e7c004c80 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c08b0c0 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c004ea0 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e8d05b740 10 client.6336 fetch_fsmap learned FSMap version 29 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.649+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x5583f29dfa30 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c08b0c0 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f9e7c042020 con 0x5583f2afbde0 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 10 client.6336 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 10 client.6336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 20 client.6336 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2b1cca0 0x5583f2b130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 4 client.6336 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:42:28.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.650+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583f2aae6e0 con 0x5583f2b1cca0 2024-04-26T23:42:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e83400640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2b1cca0 0x5583f2b130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e83400640 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2b1cca0 0x5583f2b130f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e15425b93488a3 server_cookie=8c0ccb0b079bad61 in_seq=0 out_seq=0 2024-04-26T23:42:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e67e00640 10 client.6336 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e67e00640 1 -- 172.21.15.46:0/4083808399 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+284 (crc 0 0 0) 0x5583f2aae6e0 con 0x5583f2b1cca0 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e67e00640 10 client.6336 handle_command_reply: tid=0 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.651+0000 7f9e8d05b740 1 client.6336 shutdown 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 msgr2=0x5583f2b12bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c004510 tx=0x7f9e7c003760 comp rx=0 tx=0).stop 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 shutdown_connections 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583f2afbde0 0x5583f2b12bb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583f2ade540 0x5583f2ab2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5583f2b1cca0 0x5583f2b130f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 --2- 172.21.15.46:0/4083808399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f2ab2e80 0x5583f2ab20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e67e00640 0 client.6336 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 >> 172.21.15.46:0/4083808399 conn(0x5583f2ab4220 msgr2=0x5583f2ab5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 shutdown_connections 2024-04-26T23:42:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 1 -- 172.21.15.46:0/4083808399 wait complete. 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.652+0000 7f9e8d05b740 20 client.6336 trim_cache size 0 max 16384 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "7f25f28b-3f3f-4b5e-90bf-3bf53f668c08": { 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7f25f28b-3f3f-4b5e-90bf-3bf53f668c08", 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "" 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:28.661 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 msgr2=0x7f9e6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 0x7f9e6006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e74033000 tx=0x7f9e74009000 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 msgr2=0x7f9e841448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c3c0 tx=0x7f9e6c00c880 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 shutdown_connections 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e6006c8f0 0x7f9e6006edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e8415e220 0x7f9e84146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8415d730 0x7f9e841448c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 --2- 172.21.15.46:0/1737218870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e840b53b0 0x7f9e840c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 >> 172.21.15.46:0/1737218870 conn(0x7f9e8406c980 msgr2=0x7f9e84071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 shutdown_connections 2024-04-26T23:42:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:28.653+0000 7f9e8a200640 1 -- 172.21.15.46:0/1737218870 wait complete. 2024-04-26T23:42:33.360 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:42:33.356+0000 7f0755a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-26T23:42:33.677 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:42:33.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-26T23:42:33.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.901+0000 7fe4c7e00640 1 Processor -- start 2024-04-26T23:42:33.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.901+0000 7fe4c7e00640 1 -- start start 2024-04-26T23:42:33.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.903+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c8145c10 0x7fe4c8073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.903+0000 7fe4c7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4c80cfed0 0x7fe4c80d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.903+0000 7fe4c7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c806ec30 con 0x7fe4c80d0dc0 2024-04-26T23:42:33.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.903+0000 7fe4c7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c8145ff0 con 0x7fe4c80cfed0 2024-04-26T23:42:33.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.903+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c80d1230 con 0x7fe4c8145c10 2024-04-26T23:42:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.904+0000 7fe4c6000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4c80cfed0 0x7fe4c80d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.904+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c8145c10 0x7fe4c8073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.904+0000 7fe4c6000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4c80cfed0 0x7fe4c80d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:57912/0 (socket says 172.21.15.46:57912) 2024-04-26T23:42:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.904+0000 7fe4c6000640 1 -- 172.21.15.46:0/313515453 learned_addr learned my addr 172.21.15.46:0/313515453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.905+0000 7fe4c6000640 1 -- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c8145c10 msgr2=0x7fe4c8073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.905+0000 7fe4c6000640 1 --2- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c8145c10 0x7fe4c8073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.905+0000 7fe4c6000640 1 -- 172.21.15.46:0/313515453 >> v1:172.21.15.142:6789/0 conn(0x7fe4c80d0dc0 legacy=0x7fe4c8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.905+0000 7fe4c6000640 1 -- 172.21.15.46:0/313515453 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c80cf3f0 con 0x7fe4c80cfed0 2024-04-26T23:42:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.906+0000 7fe4c6000640 1 --2- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c80d0880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098a0 tx=0x7fe4b402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891b72e12dbab038 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.907+0000 7fe4c5600640 1 -- 172.21.15.46:0/313515453 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037070 con 0x7fe4c80cfed0 2024-04-26T23:42:33.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.907+0000 7fe4c5600640 1 -- 172.21.15.46:0/313515453 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4032440 con 0x7fe4c80cfed0 2024-04-26T23:42:33.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.907+0000 7fe4c5600640 1 -- 172.21.15.46:0/313515453 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4032810 con 0x7fe4c80cfed0 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.907+0000 7fe4c7e00640 1 -- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 msgr2=0x7fe4c80d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c80d0880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098a0 tx=0x7fe4b402a290 comp rx=0 tx=0).stop 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 -- 172.21.15.46:0/313515453 shutdown_connections 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c80d0880 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/313515453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c8145c10 0x7fe4c8073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 -- 172.21.15.46:0/313515453 >> 172.21.15.46:0/313515453 conn(0x7fe4c806c350 msgr2=0x7fe4c806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 -- 172.21.15.46:0/313515453 shutdown_connections 2024-04-26T23:42:33.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.908+0000 7fe4c7e00640 1 -- 172.21.15.46:0/313515453 wait complete. 2024-04-26T23:42:33.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.909+0000 7fe4c7e00640 1 Processor -- start 2024-04-26T23:42:33.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.910+0000 7fe4c7e00640 1 -- start start 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.910+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c81417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.910+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c80d0dc0 0x7fe4c81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c80cf0a0 con 0x7fe4c8145c10 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c80cf9b0 con 0x7fe4c80cfed0 2024-04-26T23:42:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c81468b0 con 0x7fe4c80d0dc0 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52766/0 (socket says 172.21.15.46:52766) 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c7400640 1 -- 172.21.15.46:0/2360428535 learned_addr learned my addr 172.21.15.46:0/2360428535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c81417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.911+0000 7fe4c6000640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c80d0dc0 0x7fe4c81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 -- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c80d0dc0 msgr2=0x7fe4c81535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c80d0dc0 0x7fe4c81535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 -- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 msgr2=0x7fe4c81417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c81417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 -- 172.21.15.46:0/2360428535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c813ae20 con 0x7fe4c8145c10 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7400640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c7d0 tx=0x7fe4c000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4bea00640 1 -- 172.21.15.46:0/2360428535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0003b70 con 0x7fe4c8145c10 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4bea00640 1 -- 172.21.15.46:0/2360428535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0003d30 con 0x7fe4c8145c10 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4bea00640 1 -- 172.21.15.46:0/2360428535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000aa90 con 0x7fe4c8145c10 2024-04-26T23:42:33.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.912+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c8140110 con 0x7fe4c8145c10 2024-04-26T23:42:33.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.913+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81406d0 con 0x7fe4c8145c10 2024-04-26T23:42:33.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.915+0000 7fe4bea00640 1 -- 172.21.15.46:0/2360428535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4c000ac30 con 0x7fe4c8145c10 2024-04-26T23:42:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.916+0000 7fe4bea00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 0x7fe4a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.916+0000 7fe4bea00640 1 -- 172.21.15.46:0/2360428535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4c008bbc0 con 0x7fe4c8145c10 2024-04-26T23:42:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.916+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 0x7fe4a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.916+0000 7fe4d0794740 1 Processor -- start 2024-04-26T23:42:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.917+0000 7fe4d0794740 1 -- start start 2024-04-26T23:42:33.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.917+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.917+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da67716500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.917+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 0x55da67714670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4d0794740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da675de030 con 0x55da676f8710 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da675f97f0 con 0x55da676ce3f0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55da675b47e0 con 0x55da67715fb0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52772/0 (socket says 172.21.15.46:52772) 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 -- 172.21.15.46:0/549337434 learned_addr learned my addr 172.21.15.46:0/549337434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 0x7fe4a806ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009aa0 tx=0x7fe4b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6000640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da67716500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c7400640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 0x55da67714670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 -- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 msgr2=0x55da67716500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da67716500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 -- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 msgr2=0x55da67714670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 0x55da67714670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c6a00640 1 -- 172.21.15.46:0/549337434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da675c0050 con 0x55da676f8710 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.918+0000 7fe4c7400640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 0x55da67714670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.919+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011dd0 tx=0x7fe4b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1b3a6a3ac734f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.919+0000 7fe4b3400640 1 -- 172.21.15.46:0/549337434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8025070 con 0x55da676f8710 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.919+0000 7fe4b3400640 1 -- 172.21.15.46:0/549337434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b801b070 con 0x55da676f8710 2024-04-26T23:42:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.919+0000 7fe4b3400640 1 -- 172.21.15.46:0/549337434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8014dd0 con 0x55da676f8710 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 -- 172.21.15.46:0/549337434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 msgr2=0x55da676f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011dd0 tx=0x7fe4b8014000 comp rx=0 tx=0).stop 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 -- 172.21.15.46:0/549337434 shutdown_connections 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676ce3f0 0x55da67714670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da67716500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 --2- 172.21.15.46:0/549337434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676f8af0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 -- 172.21.15.46:0/549337434 >> 172.21.15.46:0/549337434 conn(0x55da676cad20 msgr2=0x55da67713520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 -- 172.21.15.46:0/549337434 shutdown_connections 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.920+0000 7fe4d0794740 1 -- 172.21.15.46:0/549337434 wait complete. 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.921+0000 7fe4d0794740 1 -- start start 2024-04-26T23:42:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.921+0000 7fe4d0794740 1 Processor -- start 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 0x55da676f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da67636200 con 0x55da676e9570 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da67630ee0 con 0x55da67715fb0 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55da6763de70 con 0x55da676f8710 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4d0794740 10 client.0 _ll_register_callbacks cb 0x55da67627270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52784/0 (socket says 172.21.15.46:52784) 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.922+0000 7fe4c7400640 1 -- 172.21.15.46:0/1169405434 learned_addr learned my addr 172.21.15.46:0/1169405434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 -- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 msgr2=0x55da676f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c6000640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 0x55da676f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 -- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 msgr2=0x55da676f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 0x55da676f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 -- 172.21.15.46:0/1169405434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6740d7d0 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c6000640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 0x55da676f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4c7400640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000a430 tx=0x7fe4c00597a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549b84c989e8dc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0070070 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.923+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4c0058b30 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b801b070 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0058d30 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 10 client.6345 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55da673433f0 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b801b070 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7fe4c0055430 con 0x55da676e9570 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 10 client.6345 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 10 client.6345 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 20 client.6345 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676f0f90 0x55da676f1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 4 client.6345 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.924+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da6763f7e0 con 0x55da676f0f90 2024-04-26T23:42:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.925+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676f0f90 0x55da676f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.925+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676f0f90 0x55da676f1370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f63196d28a97889 server_cookie=c60a2c64f3abdfd3 in_seq=0 out_seq=0 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.925+0000 7fe4a7e00640 10 client.6345 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4a7e00640 1 -- 172.21.15.46:0/1169405434 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da6763f7e0 con 0x55da676f0f90 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4a7e00640 10 client.6345 handle_command_reply: tid=0 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 client.6345 shutdown 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 msgr2=0x55da676f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000a430 tx=0x7fe4c00597a0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 shutdown_connections 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676e9570 0x55da676f7110 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676f0f90 0x55da676f1370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da67715fb0 0x55da676f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1169405434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 >> 172.21.15.46:0/1169405434 conn(0x55da676ce3f0 msgr2=0x55da676c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.926+0000 7fe4a7e00640 0 client.6345 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.927+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 shutdown_connections 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.927+0000 7fe4d0794740 1 -- 172.21.15.46:0/1169405434 wait complete. 2024-04-26T23:42:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.927+0000 7fe4d0794740 20 client.6345 trim_cache size 0 max 16384 2024-04-26T23:42:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.946+0000 7fe4d0794740 1 Processor -- start 2024-04-26T23:42:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.946+0000 7fe4d0794740 1 -- start start 2024-04-26T23:42:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da6778fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da676e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8201e50 con 0x55da676f8710 2024-04-26T23:42:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8200e30 con 0x55da676e9570 2024-04-26T23:42:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c813f7a0 con 0x55da67715fb0 2024-04-26T23:42:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52790/0 (socket says 172.21.15.46:52790) 2024-04-26T23:42:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.947+0000 7fe4c6000640 1 -- 172.21.15.46:0/1957052263 learned_addr learned my addr 172.21.15.46:0/1957052263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c7400640 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da676e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da6778fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 -- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 msgr2=0x55da676e88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da676e88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 -- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 msgr2=0x55da6778fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da6778fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 -- 172.21.15.46:0/1957052263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da676f16e0 con 0x55da676f8710 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.948+0000 7fe4c6000640 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098a0 tx=0x7fe4b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3507d2e9d5316f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4b2a00640 1 -- 172.21.15.46:0/1957052263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4042070 con 0x55da676f8710 2024-04-26T23:42:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4b2a00640 1 -- 172.21.15.46:0/1957052263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4032700 con 0x55da676f8710 2024-04-26T23:42:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4b2a00640 1 -- 172.21.15.46:0/1957052263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037040 con 0x55da676f8710 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 -- 172.21.15.46:0/1957052263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 msgr2=0x55da676e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098a0 tx=0x7fe4b402c000 comp rx=0 tx=0).stop 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 -- 172.21.15.46:0/1957052263 shutdown_connections 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da67715fb0 0x55da676e88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da676f8710 0x55da676e83b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 --2- 172.21.15.46:0/1957052263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da6778fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 -- 172.21.15.46:0/1957052263 >> 172.21.15.46:0/1957052263 conn(0x55da676ce3f0 msgr2=0x55da676c4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 -- 172.21.15.46:0/1957052263 shutdown_connections 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.949+0000 7fe4d0794740 1 -- 172.21.15.46:0/1957052263 wait complete. 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.950+0000 7fe4d0794740 1 -- start start 2024-04-26T23:42:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.950+0000 7fe4d0794740 1 Processor -- start 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da676f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da6763efd0 con 0x55da67715fb0 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c82007e0 con 0x55da676e9570 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55da67779340 con 0x55da676f8710 2024-04-26T23:42:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.951+0000 7fe4d0794740 10 client.0 _ll_register_callbacks cb 0x7fe4c0001830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52794/0 (socket says 172.21.15.46:52794) 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 -- 172.21.15.46:0/3156409989 learned_addr learned my addr 172.21.15.46:0/3156409989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da676f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c6000640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 -- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 msgr2=0x55da676e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 -- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 msgr2=0x55da676f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da676f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 -- 172.21.15.46:0/3156409989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da673030f0 con 0x55da67715fb0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4c7400640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0002230 tx=0x7fe4c0089d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e096ee28286b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.952+0000 7fe4a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0070040 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe4c00587b0 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b4032700 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0059220 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 10 client.6351 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55da675f9a30 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4032700 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7fe4c0078040 con 0x55da67715fb0 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 10 client.6351 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 10 client.6351 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 20 client.6351 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676ebec0 0x55da676c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 4 client.6351 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:42:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.953+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da67639650 con 0x55da676ebec0 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676ebec0 0x55da676c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4c6a00640 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676ebec0 0x55da676c7960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6c6105b8fd9eaf server_cookie=49e0ad21538b40ea in_seq=0 out_seq=0 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4a7e00640 10 client.6351 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4a7e00640 1 -- 172.21.15.46:0/3156409989 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55da67639650 con 0x55da676ebec0 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4a7e00640 10 client.6351 handle_command_reply: tid=0 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.954+0000 7fe4d0794740 1 client.6351 shutdown 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 msgr2=0x55da676eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0002230 tx=0x7fe4c0089d90 comp rx=0 tx=0).stop 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 shutdown_connections 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da67715fb0 0x55da676eb980 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55da676f8710 0x55da676e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55da676ebec0 0x55da676c7960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 --2- 172.21.15.46:0/3156409989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da676e9570 0x55da676f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4a7e00640 0 client.6351 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 >> 172.21.15.46:0/3156409989 conn(0x55da676ce3f0 msgr2=0x55da676ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 shutdown_connections 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 1 -- 172.21.15.46:0/3156409989 wait complete. 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.955+0000 7fe4d0794740 20 client.6351 trim_cache size 0 max 16384 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:42:33.963 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 msgr2=0x7fe4a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 0x7fe4a806ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009aa0 tx=0x7fe4b8009190 comp rx=0 tx=0).stop 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 msgr2=0x7fe4c8153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c7d0 tx=0x7fe4c000cc90 comp rx=0 tx=0).stop 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 shutdown_connections 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4a806c920 0x7fe4a806ede0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4c8145c10 0x7fe4c8153b00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4c80d0dc0 0x7fe4c81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 --2- 172.21.15.46:0/2360428535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80cfed0 0x7fe4c81417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 >> 172.21.15.46:0/2360428535 conn(0x7fe4c806c350 msgr2=0x7fe4c8157150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 shutdown_connections 2024-04-26T23:42:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:33.956+0000 7fe4c7e00640 1 -- 172.21.15.46:0/2360428535 wait complete. 2024-04-26T23:42:33.979 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:42:33.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-26T23:42:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.191+0000 7f4841c00640 1 Processor -- start 2024-04-26T23:42:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.192+0000 7f4841c00640 1 -- start start 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.193+0000 7f4841c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f483c162a70 0x7f483c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.193+0000 7f4841c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d3830 0x7f483c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.193+0000 7f4841c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c0d3400 con 0x7f483c0d1ae0 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.193+0000 7f4841c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f483c0d2370 con 0x7f483c162a70 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.193+0000 7f4841c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0d2650 con 0x7f483c0d3830 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f483c162a70 0x7f483c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f483c162a70 0x7f483c0d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52810/0 (socket says 172.21.15.46:52810) 2024-04-26T23:42:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 -- 172.21.15.46:0/1363335339 learned_addr learned my addr 172.21.15.46:0/1363335339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483b400640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d3830 0x7f483c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 -- 172.21.15.46:0/1363335339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d3830 msgr2=0x7f483c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d3830 0x7f483c0d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 -- 172.21.15.46:0/1363335339 >> v1:172.21.15.142:6789/0 conn(0x7f483c0d1ae0 legacy=0x7f483c0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.194+0000 7f483be00640 1 -- 172.21.15.46:0/1363335339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c0d0df0 con 0x7f483c162a70 2024-04-26T23:42:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.195+0000 7f483be00640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c0d2e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4824009870 tx=0x7f482402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aa2f91e58be9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f483aa00640 1 -- 172.21.15.46:0/1363335339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f483c162a70 2024-04-26T23:42:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f483aa00640 1 -- 172.21.15.46:0/1363335339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4824032440 con 0x7f483c162a70 2024-04-26T23:42:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f483aa00640 1 -- 172.21.15.46:0/1363335339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824032810 con 0x7f483c162a70 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 -- 172.21.15.46:0/1363335339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 msgr2=0x7f483c0d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c0d2e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4824009870 tx=0x7f482402a2a0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 -- 172.21.15.46:0/1363335339 shutdown_connections 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d3830 0x7f483c0d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 --2- 172.21.15.46:0/1363335339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c0d2e20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.196+0000 7f4841c00640 1 -- 172.21.15.46:0/1363335339 >> 172.21.15.46:0/1363335339 conn(0x7f483c0d4f20 msgr2=0x7f483c0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.197+0000 7f4841c00640 1 -- 172.21.15.46:0/1363335339 shutdown_connections 2024-04-26T23:42:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.197+0000 7f4841c00640 1 -- 172.21.15.46:0/1363335339 wait complete. 2024-04-26T23:42:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.199+0000 7f4841c00640 1 Processor -- start 2024-04-26T23:42:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.199+0000 7f4841c00640 1 -- start start 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d1ae0 0x7f483c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f483c0d3830 0x7f483c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0d20f0 con 0x7f483c162a70 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0d1f70 con 0x7f483c0d3830 2024-04-26T23:42:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.200+0000 7f4841c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0b39f0 con 0x7f483c0d1ae0 2024-04-26T23:42:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52814/0 (socket says 172.21.15.46:52814) 2024-04-26T23:42:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 -- 172.21.15.46:0/3904455741 learned_addr learned my addr 172.21.15.46:0/3904455741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f483be00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d1ae0 0x7f483c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f483b400640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f483c0d3830 0x7f483c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 -- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d1ae0 msgr2=0x7f483c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d1ae0 0x7f483c0cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 -- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f483c0d3830 msgr2=0x7f483c0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f483c0d3830 0x7f483c0cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.201+0000 7f4841200640 1 -- 172.21.15.46:0/3904455741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c0cc660 con 0x7f483c162a70 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4841200640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f483400c410 tx=0x7f483400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4833e00640 1 -- 172.21.15.46:0/3904455741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834011790 con 0x7f483c162a70 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4833e00640 1 -- 172.21.15.46:0/3904455741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834017070 con 0x7f483c162a70 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1f8210 con 0x7f483c162a70 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4833e00640 1 -- 172.21.15.46:0/3904455741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834011b70 con 0x7f483c162a70 2024-04-26T23:42:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.202+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1f87d0 con 0x7f483c162a70 2024-04-26T23:42:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.205+0000 7f4833e00640 1 -- 172.21.15.46:0/3904455741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4834011d10 con 0x7f483c162a70 2024-04-26T23:42:34.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.206+0000 7f4833e00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 0x7f481806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.206+0000 7f4833e00640 1 -- 172.21.15.46:0/3904455741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f483408b880 con 0x7f483c162a70 2024-04-26T23:42:34.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.207+0000 7f4846178740 1 Processor -- start 2024-04-26T23:42:34.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.207+0000 7f483be00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 0x7f481806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.207+0000 7f4846178740 1 -- start start 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0bc100 0x55baeb0bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55baeb024c70 con 0x55baeb0e9880 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55baeafdd980 con 0x55baeb0bc100 2024-04-26T23:42:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55baeb020c30 con 0x55baeb0e94a0 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f483b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f483be00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 0x7f481806ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f483b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52820/0 (socket says 172.21.15.46:52820) 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.208+0000 7f483b400640 1 -- 172.21.15.46:0/2800255466 learned_addr learned my addr 172.21.15.46:0/2800255466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483be00640 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f4841200640 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0bc100 0x55baeb0bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 -- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 msgr2=0x55baeb0ebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0ebcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 -- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0bc100 msgr2=0x55baeb0bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0bc100 0x55baeb0bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 -- 172.21.15.46:0/2800255466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baeafb57b0 con 0x55baeb0e9880 2024-04-26T23:42:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.209+0000 7f483b400640 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f482c009ad0 tx=0x7f482c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba22cd1f9d09775f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4830c00640 1 -- 172.21.15.46:0/2800255466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c014610 con 0x55baeb0e9880 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4830c00640 1 -- 172.21.15.46:0/2800255466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c019070 con 0x55baeb0e9880 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4830c00640 1 -- 172.21.15.46:0/2800255466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0149d0 con 0x55baeb0e9880 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 -- 172.21.15.46:0/2800255466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 msgr2=0x55baeb0ec1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f482c009ad0 tx=0x7f482c009f90 comp rx=0 tx=0).stop 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 -- 172.21.15.46:0/2800255466 shutdown_connections 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0bc100 0x55baeb0bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0ec1f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 --2- 172.21.15.46:0/2800255466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0ebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 -- 172.21.15.46:0/2800255466 >> 172.21.15.46:0/2800255466 conn(0x55baeb0be370 msgr2=0x55baeb106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 -- 172.21.15.46:0/2800255466 shutdown_connections 2024-04-26T23:42:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.210+0000 7f4846178740 1 -- 172.21.15.46:0/2800255466 wait complete. 2024-04-26T23:42:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.211+0000 7f4846178740 1 -- start start 2024-04-26T23:42:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.211+0000 7f4846178740 1 Processor -- start 2024-04-26T23:42:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e94a0 0x55baeb0e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 0x55baeb0e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55baeb023f80 con 0x55baeb0e9880 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55baeafd1260 con 0x55baeb0e94a0 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55baeb0292a0 con 0x55baeb0e7250 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.212+0000 7f4846178740 10 client.0 _ll_register_callbacks cb 0x55baeb01b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52822/0 (socket says 172.21.15.46:52822) 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 -- 172.21.15.46:0/1128819965 learned_addr learned my addr 172.21.15.46:0/1128819965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483be00640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e94a0 0x55baeb0e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f4841200640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 0x55baeb0e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 -- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 msgr2=0x55baeb0e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 0x55baeb0e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 -- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e94a0 msgr2=0x55baeb0e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e94a0 0x55baeb0e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 -- 172.21.15.46:0/1128819965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baeafb31c0 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f4841200640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 0x55baeb0e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f483b400640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55baeb0c3a00 tx=0x7f482c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57a80c2a54f8710 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.213+0000 7f4829600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c014dc0 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f482c029c70 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c019070 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f482c030050 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 10 client.6363 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55baead363f0 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c019070 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f482c033020 con 0x55baeb0e9880 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 10 client.6363 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 10 client.6363 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 20 client.6363 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb0edd90 0x55baeb0ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 4 client.6363 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.214+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baeb032880 con 0x55baeb0edd90 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.215+0000 7f483be00640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb0edd90 0x55baeb0ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.215+0000 7f483be00640 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb0edd90 0x55baeb0ee1a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c3c5b9bac6c1ac server_cookie=ef9dcb7902bf83e6 in_seq=0 out_seq=0 2024-04-26T23:42:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.215+0000 7f4829600640 10 client.6363 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4829600640 1 -- 172.21.15.46:0/1128819965 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55baeb032880 con 0x55baeb0edd90 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4829600640 10 client.6363 handle_command_reply: tid=0 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 client.6363 shutdown 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 msgr2=0x55baeb10b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55baeb0c3a00 tx=0x7f482c012000 comp rx=0 tx=0).stop 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 shutdown_connections 2024-04-26T23:42:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e7250 0x55baeb0e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb10b3f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb0edd90 0x55baeb0ee1a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.216+0000 7f4846178740 1 --2- 172.21.15.46:0/1128819965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e94a0 0x55baeb0e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.217+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 >> 172.21.15.46:0/1128819965 conn(0x55baeb0bc100 msgr2=0x55baeb0c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.217+0000 7f4829600640 0 client.6363 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.217+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 shutdown_connections 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.217+0000 7f4846178740 1 -- 172.21.15.46:0/1128819965 wait complete. 2024-04-26T23:42:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.217+0000 7f4846178740 20 client.6363 trim_cache size 0 max 16384 2024-04-26T23:42:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.232+0000 7f4846178740 1 Processor -- start 2024-04-26T23:42:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.232+0000 7f4846178740 1 -- start start 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e7250 0x55baeb0e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f483c201da0 con 0x55baeb0e9880 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f483c200e00 con 0x55baeb0e7250 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0b89b0 con 0x55baeb0e94a0 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52838/0 (socket says 172.21.15.46:52838) 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 -- 172.21.15.46:0/1809900264 learned_addr learned my addr 172.21.15.46:0/1809900264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f483b400640 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 -- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 msgr2=0x55baeb0baf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 -- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e7250 msgr2=0x55baeb0e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e7250 0x55baeb0e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.233+0000 7f4841200640 1 -- 172.21.15.46:0/1809900264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baeae007d0 con 0x55baeb0e9880 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4841200640 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f483400cdd0 tx=0x7f483405d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f08c9e177da27c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f482be00640 1 -- 172.21.15.46:0/1809900264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834071070 con 0x55baeb0e9880 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f482be00640 1 -- 172.21.15.46:0/1809900264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834057710 con 0x55baeb0e9880 2024-04-26T23:42:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f482be00640 1 -- 172.21.15.46:0/1809900264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834061b60 con 0x55baeb0e9880 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 -- 172.21.15.46:0/1809900264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 msgr2=0x55baeb0c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f483400cdd0 tx=0x7f483405d7c0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 -- 172.21.15.46:0/1809900264 shutdown_connections 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e9880 0x55baeb0c14f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb0baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 --2- 172.21.15.46:0/1809900264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e7250 0x55baeb0e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.234+0000 7f4846178740 1 -- 172.21.15.46:0/1809900264 >> 172.21.15.46:0/1809900264 conn(0x55baeb0bc100 msgr2=0x55baeb0e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.235+0000 7f4846178740 1 -- 172.21.15.46:0/1809900264 shutdown_connections 2024-04-26T23:42:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.235+0000 7f4846178740 1 -- 172.21.15.46:0/1809900264 wait complete. 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.235+0000 7f4846178740 1 -- start start 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.235+0000 7f4846178740 1 Processor -- start 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb107480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e9880 0x55baeb127e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55baeb10b040 con 0x55baeb0e7250 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f483c2007b0 con 0x55baeb0e9880 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55baeb0bcd20 con 0x55baeb0e94a0 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.236+0000 7f4846178740 10 client.0 _ll_register_callbacks cb 0x7f482c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52846/0 (socket says 172.21.15.46:52846) 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 -- 172.21.15.46:0/1308644443 learned_addr learned my addr 172.21.15.46:0/1308644443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483b400640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 -- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 msgr2=0x55baeb107480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb107480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 -- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e9880 msgr2=0x55baeb127e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f4841200640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e9880 0x55baeb127e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e9880 0x55baeb127e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483be00640 1 -- 172.21.15.46:0/1308644443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baeafcb2a0 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.237+0000 7f483b400640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f483be00640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4824009990 tx=0x7f4824004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675cb01b4eea6b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824037040 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4824032c10 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4834057710 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f482405d050 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4846178740 10 client.6369 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55baeb116220 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834057710 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.238+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f4824060020 con 0x55baeb0e7250 2024-04-26T23:42:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 10 client.6369 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 10 client.6369 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 20 client.6369 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb128350 0x55baeb0c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 4 client.6369 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55baeb02c6f0 con 0x55baeb128350 2024-04-26T23:42:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f483b400640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb128350 0x55baeb0c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f483b400640 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb128350 0x55baeb0c12c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf15bc48bbce13f server_cookie=c77829c1a61d7387 in_seq=0 out_seq=0 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.239+0000 7f4829600640 10 client.6369 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4829600640 1 -- 172.21.15.46:0/1308644443 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+168 (crc 0 0 0) 0x55baeb02c6f0 con 0x55baeb128350 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4829600640 10 client.6369 handle_command_reply: tid=0 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 client.6369 shutdown 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 msgr2=0x55baeb129270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4824009990 tx=0x7f4824004730 comp rx=0 tx=0).stop 2024-04-26T23:42:34.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 shutdown_connections 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x55baeb128350 0x55baeb0c12c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55baeb0e9880 0x55baeb127e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55baeb0e94a0 0x55baeb107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 --2- 172.21.15.46:0/1308644443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55baeb0e7250 0x55baeb129270 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4829600640 0 client.6369 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 >> 172.21.15.46:0/1308644443 conn(0x55baeb0bc100 msgr2=0x55baeb0c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 shutdown_connections 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 1 -- 172.21.15.46:0/1308644443 wait complete. 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.240+0000 7f4846178740 20 client.6369 trim_cache size 0 max 16384 2024-04-26T23:42:34.248 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3274666785, 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511630414, 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:34.249 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 msgr2=0x7f481806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 0x7f481806ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 msgr2=0x7f483c15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f483400c410 tx=0x7f483400c8d0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 shutdown_connections 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f481806c920 0x7f481806ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f483c162a70 0x7f483c15faa0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f483c0d3830 0x7f483c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 --2- 172.21.15.46:0/3904455741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483c0d1ae0 0x7f483c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 >> 172.21.15.46:0/3904455741 conn(0x7f483c0d4f20 msgr2=0x7f483c160960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 shutdown_connections 2024-04-26T23:42:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.241+0000 7f4841c00640 1 -- 172.21.15.46:0/3904455741 wait complete. 2024-04-26T23:42:34.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.471+0000 7fefc1600640 1 Processor -- start 2024-04-26T23:42:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.471+0000 7fefc1600640 1 -- start start 2024-04-26T23:42:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefc1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefc1600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefc1600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefbc14c6c0 con 0x7fefbc0cb0f0 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefc1600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fefbc0cb8b0 con 0x7fefbc14c2e0 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefc1600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc167e10 con 0x7fefbc0cdbc0 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.473+0000 7fefbbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52852/0 (socket says 172.21.15.46:52852) 2024-04-26T23:42:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 -- 172.21.15.46:0/1788240757 learned_addr learned my addr 172.21.15.46:0/1788240757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefc0c00640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 -- 172.21.15.46:0/1788240757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 msgr2=0x7fefbc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 -- 172.21.15.46:0/1788240757 >> v1:172.21.15.142:6789/0 conn(0x7fefbc0cb0f0 legacy=0x7fefbc0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.474+0000 7fefbbe00640 1 -- 172.21.15.46:0/1788240757 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc167050 con 0x7fefbc14c2e0 2024-04-26T23:42:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.475+0000 7fefbbe00640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fefac009870 tx=0x7fefac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc17a0dc7628235d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefbaa00640 1 -- 172.21.15.46:0/1788240757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac037070 con 0x7fefbc14c2e0 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefbaa00640 1 -- 172.21.15.46:0/1788240757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefac032440 con 0x7fefbc14c2e0 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefbaa00640 1 -- 172.21.15.46:0/1788240757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac032810 con 0x7fefbc14c2e0 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 -- 172.21.15.46:0/1788240757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 msgr2=0x7fefbc0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fefac009870 tx=0x7fefac02a180 comp rx=0 tx=0).stop 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 -- 172.21.15.46:0/1788240757 shutdown_connections 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 --2- 172.21.15.46:0/1788240757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc0ccdd0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 -- 172.21.15.46:0/1788240757 >> 172.21.15.46:0/1788240757 conn(0x7fefbc0b26d0 msgr2=0x7fefbc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 -- 172.21.15.46:0/1788240757 shutdown_connections 2024-04-26T23:42:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.476+0000 7fefc1600640 1 -- 172.21.15.46:0/1788240757 wait complete. 2024-04-26T23:42:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.478+0000 7fefc1600640 1 Processor -- start 2024-04-26T23:42:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.478+0000 7fefc1600640 1 -- start start 2024-04-26T23:42:34.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc0cb0f0 0x7fefbc147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1678f0 con 0x7fefbc14c2e0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc167610 con 0x7fefbc0cb0f0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc1600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc0cb580 con 0x7fefbc0cdbc0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.479+0000 7fefc0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52862/0 (socket says 172.21.15.46:52862) 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 -- 172.21.15.46:0/441019161 learned_addr learned my addr 172.21.15.46:0/441019161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefbb400640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 -- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 msgr2=0x7fefbc165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefbbe00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc0cb0f0 0x7fefbc147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 -- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc0cb0f0 msgr2=0x7fefbc147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc0cb0f0 0x7fefbc147870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 -- 172.21.15.46:0/441019161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc147f70 con 0x7fefbc14c2e0 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefbb400640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.480+0000 7fefc0c00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007de0 tx=0x7fefb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.481+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4014620 con 0x7fefbc14c2e0 2024-04-26T23:42:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.481+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb4019070 con 0x7fefbc14c2e0 2024-04-26T23:42:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.481+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc0b3960 con 0x7fefbc14c2e0 2024-04-26T23:42:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.481+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4014a20 con 0x7fefbc14c2e0 2024-04-26T23:42:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.481+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc0b3f20 con 0x7fefbc14c2e0 2024-04-26T23:42:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.482+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc0b4120 con 0x7fefbc14c2e0 2024-04-26T23:42:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.483+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb4014bc0 con 0x7fefbc14c2e0 2024-04-26T23:42:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.484+0000 7fefb3e00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.484+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fefb408c650 con 0x7fefbc14c2e0 2024-04-26T23:42:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.485+0000 7fefbbe00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.486+0000 7fefbbe00640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 0x7fef9c06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefac034000 tx=0x7fefac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.489+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefbc0b4120 con 0x7fefbc14c2e0 2024-04-26T23:42:34.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.733+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefbc0693b0 con 0x7fefbc14c2e0 2024-04-26T23:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.734+0000 7fefb3e00640 1 -- 172.21.15.46:0/441019161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5182 (secure 0 0 0) 0x7fefb405a8e0 con 0x7fefbc14c2e0 2024-04-26T23:42:34.739 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:34.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:33.176845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 31 2024-04-26T23:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 msgr2=0x7fef9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 0x7fef9c06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefac034000 tx=0x7fefac02e000 comp rx=0 tx=0).stop 2024-04-26T23:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 msgr2=0x7fefbc146610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007de0 tx=0x7fefb400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 shutdown_connections 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefbc14c2e0 0x7fefbc146610 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefbc0cdbc0 0x7fefbc165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 --2- 172.21.15.46:0/441019161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefbc0cb0f0 0x7fefbc147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 >> 172.21.15.46:0/441019161 conn(0x7fefbc0b26d0 msgr2=0x7fefbc06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 shutdown_connections 2024-04-26T23:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.738+0000 7fefc1600640 1 -- 172.21.15.46:0/441019161 wait complete. 2024-04-26T23:42:34.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 3274666785 2024-04-26T23:42:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.971+0000 7f7ee7e00640 1 Processor -- start 2024-04-26T23:42:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.971+0000 7f7ee7e00640 1 -- start start 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ee01597e0 0x7f7ee00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee00b5730 0x7f7ee00b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 0x7f7ee0072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee00b60d0 con 0x7f7ee01597e0 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee00728c0 con 0x7f7ee0071f10 2024-04-26T23:42:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee0073120 con 0x7f7ee00b5730 2024-04-26T23:42:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ee01597e0 0x7f7ee00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ee01597e0 0x7f7ee00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52878/0 (socket says 172.21.15.46:52878) 2024-04-26T23:42:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.973+0000 7f7ee5400640 1 -- 172.21.15.46:0/325098279 learned_addr learned my addr 172.21.15.46:0/325098279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee00b5730 0x7f7ee00b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/325098279 >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 0x7f7ee0072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5400640 1 -- 172.21.15.46:0/325098279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee00b5730 msgr2=0x7f7ee00b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5400640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee00b5730 0x7f7ee00b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5400640 1 -- 172.21.15.46:0/325098279 >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 msgr2=0x7f7ee0072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5400640 1 --2- 172.21.15.46:0/325098279 >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 0x7f7ee0072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5400640 1 -- 172.21.15.46:0/325098279 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0159bc0 con 0x7f7ee01597e0 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.974+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/325098279 >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 0x7f7ee0072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.975+0000 7f7ee5400640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00ae8e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00b3850 tx=0x7f7ed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f9db7cd78bc4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.976+0000 7f7edfe00640 1 -- 172.21.15.46:0/325098279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4037070 con 0x7f7ee01597e0 2024-04-26T23:42:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.976+0000 7f7edfe00640 1 -- 172.21.15.46:0/325098279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4032440 con 0x7f7ee01597e0 2024-04-26T23:42:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.976+0000 7f7edfe00640 1 -- 172.21.15.46:0/325098279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4032810 con 0x7f7ee01597e0 2024-04-26T23:42:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.976+0000 7f7ee7e00640 1 -- 172.21.15.46:0/325098279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 msgr2=0x7f7ee00ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.976+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00ae8e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00b3850 tx=0x7f7ed402a270 comp rx=0 tx=0).stop 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 -- 172.21.15.46:0/325098279 shutdown_connections 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/325098279 >> v2:172.21.15.142:3300/0 conn(0x7f7ee0071f10 0x7f7ee0072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee00b5730 0x7f7ee00b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/325098279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00ae8e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 -- 172.21.15.46:0/325098279 >> 172.21.15.46:0/325098279 conn(0x7f7ee006c980 msgr2=0x7f7ee00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 -- 172.21.15.46:0/325098279 shutdown_connections 2024-04-26T23:42:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.977+0000 7f7ee7e00640 1 -- 172.21.15.46:0/325098279 wait complete. 2024-04-26T23:42:34.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.979+0000 7f7ee7e00640 1 Processor -- start 2024-04-26T23:42:34.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.979+0000 7f7ee7e00640 1 -- start start 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee0071f10 0x7f7ee014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ee00b5730 0x7f7ee00ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00b40d0 con 0x7f7ee01597e0 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0072e40 con 0x7f7ee00b5730 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee00b43b0 con 0x7f7ee0071f10 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52890/0 (socket says 172.21.15.46:52890) 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee5e00640 1 -- 172.21.15.46:0/138097247 learned_addr learned my addr 172.21.15.46:0/138097247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ee00b5730 0x7f7ee00ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.980+0000 7f7ee5400640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee0071f10 0x7f7ee014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 -- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee0071f10 msgr2=0x7f7ee014a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee0071f10 0x7f7ee014a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 -- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ee00b5730 msgr2=0x7f7ee00ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ee00b5730 0x7f7ee00ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 -- 172.21.15.46:0/138097247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee00ce910 con 0x7f7ee01597e0 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8007e40 tx=0x7f7ed800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7edd600640 1 -- 172.21.15.46:0/138097247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed801a070 con 0x7f7ee01597e0 2024-04-26T23:42:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7edd600640 1 -- 172.21.15.46:0/138097247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed8003b90 con 0x7f7ee01597e0 2024-04-26T23:42:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7edd600640 1 -- 172.21.15.46:0/138097247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800de80 con 0x7f7ee01597e0 2024-04-26T23:42:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.981+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee00cebf0 con 0x7f7ee01597e0 2024-04-26T23:42:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.982+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee00cf1b0 con 0x7f7ee01597e0 2024-04-26T23:42:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.984+0000 7f7edd600640 1 -- 172.21.15.46:0/138097247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ed801d070 con 0x7f7ee01597e0 2024-04-26T23:42:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.985+0000 7f7edd600640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 0x7f7ec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.986+0000 7f7edd600640 1 -- 172.21.15.46:0/138097247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7ed808c040 con 0x7f7ee01597e0 2024-04-26T23:42:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.986+0000 7f7eead8f740 1 Processor -- start 2024-04-26T23:42:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.986+0000 7f7ee5400640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 0x7f7ec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.986+0000 7f7eead8f740 1 -- start start 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.987+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848320 0x5558fc848700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.987+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.987+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc8189e0 0x5558fc8407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7eead8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5558fc782900 con 0x5558fc848d40 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5558fc71f1f0 con 0x5558fc848320 2024-04-26T23:42:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5558fc77e8c0 con 0x5558fc8189e0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee5400640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 0x7f7ec006edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4034000 tx=0x7f7ed402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52906/0 (socket says 172.21.15.46:52906) 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 -- 172.21.15.46:0/1547180645 learned_addr learned my addr 172.21.15.46:0/1547180645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee5400640 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848320 0x5558fc848700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc8189e0 0x5558fc8407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 -- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc8189e0 msgr2=0x5558fc8407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc8189e0 0x5558fc8407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 -- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848320 msgr2=0x5558fc848700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848320 0x5558fc848700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.988+0000 7f7ee4a00640 1 -- 172.21.15.46:0/1547180645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fc560710 con 0x5558fc848d40 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00b900 tx=0x7f7ecc00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677b71e614ff36d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7ed1c00640 1 -- 172.21.15.46:0/1547180645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc018020 con 0x5558fc848d40 2024-04-26T23:42:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7ed1c00640 1 -- 172.21.15.46:0/1547180645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00fa80 con 0x5558fc848d40 2024-04-26T23:42:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7ed1c00640 1 -- 172.21.15.46:0/1547180645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00fe50 con 0x5558fc848d40 2024-04-26T23:42:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7eead8f740 1 -- 172.21.15.46:0/1547180645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 msgr2=0x5558fc8184a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.989+0000 7f7eead8f740 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00b900 tx=0x7f7ecc00bdc0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 -- 172.21.15.46:0/1547180645 shutdown_connections 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc8189e0 0x5558fc8407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848d40 0x5558fc8184a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 --2- 172.21.15.46:0/1547180645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848320 0x5558fc848700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 -- 172.21.15.46:0/1547180645 >> 172.21.15.46:0/1547180645 conn(0x5558fc81a710 msgr2=0x5558fc863130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 -- 172.21.15.46:0/1547180645 shutdown_connections 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 -- 172.21.15.46:0/1547180645 wait complete. 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.990+0000 7f7eead8f740 1 -- start start 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.991+0000 7f7eead8f740 1 Processor -- start 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc83ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc847190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5558fc844a70 con 0x5558fc848320 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5558fc70a040 con 0x5558fc846d10 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5558fc844d10 con 0x5558fc848d40 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7eead8f740 10 client.0 _ll_register_callbacks cb 0x5558fc73a220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7ee5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7ee5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52914/0 (socket says 172.21.15.46:52914) 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.992+0000 7f7ee5400640 1 -- 172.21.15.46:0/2679917184 learned_addr learned my addr 172.21.15.46:0/2679917184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc83ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc847190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 -- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 msgr2=0x5558fc83ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc83ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 -- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 msgr2=0x5558fc847190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc847190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 -- 172.21.15.46:0/2679917184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fc5c0ea0 con 0x5558fc848320 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc847190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ee5400640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40090b0 tx=0x7f7ed402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f342babdf6dd0095 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ec6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed404e060 con 0x5558fc848320 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7ed403ebb0 con 0x5558fc848320 2024-04-26T23:42:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.993+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ecc00fa80 con 0x5558fc848320 2024-04-26T23:42:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ed4054050 con 0x5558fc848320 2024-04-26T23:42:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 10 client.6387 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x5558fc454000 con 0x5558fc848320 2024-04-26T23:42:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00fa80 con 0x5558fc848320 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f7ed4057020 con 0x5558fc848320 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 10 client.6387 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 10 client.6387 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 20 client.6387 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc849770 0x5558fc841900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 4 client.6387 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.994+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558fc790510 con 0x5558fc849770 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.995+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc849770 0x5558fc841900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.995+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc849770 0x5558fc841900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebd09b33e9e272b server_cookie=f48aa1b8a0d7f5ce in_seq=0 out_seq=0 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.995+0000 7f7ec6000640 10 client.6387 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7ec6000640 1 -- 172.21.15.46:0/2679917184 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558fc790510 con 0x5558fc849770 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7ec6000640 10 client.6387 handle_command_reply: tid=0 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 client.6387 shutdown 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 msgr2=0x5558fc83e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40090b0 tx=0x7f7ed402c070 comp rx=0 tx=0).stop 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 shutdown_connections 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc849770 0x5558fc841900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc847190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc83ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 --2- 172.21.15.46:0/2679917184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc83e680 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 >> 172.21.15.46:0/2679917184 conn(0x5558fc8189e0 msgr2=0x5558fc817bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.996+0000 7f7ec6000640 0 client.6387 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.997+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 shutdown_connections 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.997+0000 7f7eead8f740 1 -- 172.21.15.46:0/2679917184 wait complete. 2024-04-26T23:42:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:34.997+0000 7f7eead8f740 20 client.6387 trim_cache size 0 max 16384 2024-04-26T23:42:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.012+0000 7f7eead8f740 1 Processor -- start 2024-04-26T23:42:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.012+0000 7f7eead8f740 1 -- start start 2024-04-26T23:42:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc841880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc861b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0200e50 con 0x5558fc848320 2024-04-26T23:42:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01606d0 con 0x5558fc846d10 2024-04-26T23:42:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee0200830 con 0x5558fc848d40 2024-04-26T23:42:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-04-26T23:42:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 -- 172.21.15.46:0/3637518688 learned_addr learned my addr 172.21.15.46:0/3637518688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:35.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 -- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 msgr2=0x5558fc861b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc861b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 -- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 msgr2=0x5558fc841880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc841880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.013+0000 7f7ee4a00640 1 -- 172.21.15.46:0/3637518688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fc83df00 con 0x5558fc848320 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x5558fc819bb0 tx=0x7f7ecc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6639c0ee1a6d3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7ed1200640 1 -- 172.21.15.46:0/3637518688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc018040 con 0x5558fc848320 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7ed1200640 1 -- 172.21.15.46:0/3637518688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00fcb0 con 0x5558fc848320 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7ed1200640 1 -- 172.21.15.46:0/3637518688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc004250 con 0x5558fc848320 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7eead8f740 1 -- 172.21.15.46:0/3637518688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 msgr2=0x5558fc841dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.014+0000 7f7eead8f740 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x5558fc819bb0 tx=0x7f7ecc014000 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 -- 172.21.15.46:0/3637518688 shutdown_connections 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc848d40 0x5558fc861b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc841dc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 --2- 172.21.15.46:0/3637518688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc846d10 0x5558fc841880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 -- 172.21.15.46:0/3637518688 >> 172.21.15.46:0/3637518688 conn(0x5558fc8189e0 msgr2=0x5558fc84c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 -- 172.21.15.46:0/3637518688 shutdown_connections 2024-04-26T23:42:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 -- 172.21.15.46:0/3637518688 wait complete. 2024-04-26T23:42:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.015+0000 7f7eead8f740 1 -- start start 2024-04-26T23:42:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.016+0000 7f7eead8f740 1 Processor -- start 2024-04-26T23:42:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.016+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 0x5558fc8dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.016+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7eead8f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848d40 0x5558fc81c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7eead8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5558fc8495f0 con 0x5558fc848320 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5558fc83ff20 con 0x5558fc848d40 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7eead8f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5558fc849c40 con 0x5558fc846d10 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7eead8f740 10 client.0 _ll_register_callbacks cb 0x7f7ed4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52930/0 (socket says 172.21.15.46:52930) 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 -- 172.21.15.46:0/4230282036 learned_addr learned my addr 172.21.15.46:0/4230282036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee5400640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 0x5558fc8dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee5e00640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848d40 0x5558fc81c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 -- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 msgr2=0x5558fc8dc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 0x5558fc8dc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 -- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848d40 msgr2=0x5558fc81c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848d40 0x5558fc81c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee4a00640 1 -- 172.21.15.46:0/4230282036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fc81dad0 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.017+0000 7f7ee5400640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 0x5558fc8dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ee4a00640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00be00 tx=0x7f7ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b817921d46b2059 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ec6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc018070 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7ecc004240 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ecc00fcb0 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ecc00b220 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7eead8f740 10 client.6393 fetch_fsmap learned FSMap version 31 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x5558fc81d850 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.018+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00fcb0 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f7ecc03e020 con 0x5558fc848320 2024-04-26T23:42:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 10 client.6393 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 10 client.6393 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 20 client.6393 populate_metadata read hostname 'smithi046' 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc841b90 0x5558fc8dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 4 client.6393 mds_command: new command op to 5793 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3274666785}] 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 --> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3274666785}) -- 0x5558fc843190 con 0x5558fc841b90 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.019+0000 7f7ee5400640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc841b90 0x5558fc8dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7ee5400640 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc841b90 0x5558fc8dff60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529b47add6921da1 server_cookie=45c7a3c7d485bc57 in_seq=0 out_seq=0 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7ec6000640 10 client.6393 ms_handle_connect on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7ec6000640 1 -- 172.21.15.46:0/4230282036 <== mds.0 v2:172.21.15.46:6832/970201181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5558fc843190 con 0x5558fc841b90 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7ec6000640 10 client.6393 handle_command_reply: tid=0 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 client.6393 shutdown 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 msgr2=0x5558fc81d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00be00 tx=0x7f7ecc004730 comp rx=0 tx=0).stop 2024-04-26T23:42:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 shutdown_connections 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:6832/970201181,v1:172.21.15.46:6834/970201181] conn(0x5558fc841b90 0x5558fc8dff60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558fc848d40 0x5558fc81c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558fc848320 0x5558fc81d310 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 --2- 172.21.15.46:0/4230282036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5558fc846d10 0x5558fc8dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 >> 172.21.15.46:0/4230282036 conn(0x5558fc8189e0 msgr2=0x5558fc83e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7ec6000640 0 client.6393 ms_handle_reset on v2:172.21.15.46:6832/970201181 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 shutdown_connections 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 1 -- 172.21.15.46:0/4230282036 wait complete. 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.020+0000 7f7eead8f740 20 client.6393 trim_cache size 0 max 16384 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 msgr2=0x7f7ec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 0x7f7ec006edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4034000 tx=0x7f7ed402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 msgr2=0x7f7ee00cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8007e40 tx=0x7f7ed800d4a0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 shutdown_connections 2024-04-26T23:42:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ec006c8f0 0x7f7ec006edb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ee01597e0 0x7f7ee00cfce0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ee00b5730 0x7f7ee00ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 --2- 172.21.15.46:0/138097247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ee0071f10 0x7f7ee014a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 >> 172.21.15.46:0/138097247 conn(0x7f7ee006c980 msgr2=0x7f7ee006e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 shutdown_connections 2024-04-26T23:42:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.021+0000 7f7ee7e00640 1 -- 172.21.15.46:0/138097247 wait complete. 2024-04-26T23:42:35.043 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-26T23:42:35.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.242+0000 7f19e2000640 1 Processor -- start 2024-04-26T23:42:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.242+0000 7f19e2000640 1 -- start start 2024-04-26T23:42:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19dc15f1d0 0x7f19dc0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f19dc0cd770 0x7f19dc0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19dc0b03c0 con 0x7f19dc15edf0 2024-04-26T23:42:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc0cf5c0 con 0x7f19dc0cd770 2024-04-26T23:42:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0cb680 con 0x7f19dc15f1d0 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e0c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f19dc15edf0 0x7f19dc0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45654/0 (socket says 172.21.15.46:45654) 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19e0c00640 1 -- 172.21.15.46:0/2592131054 learned_addr learned my addr 172.21.15.46:0/2592131054 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.244+0000 7f19dbe00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19dc15f1d0 0x7f19dc0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19e1600640 1 --2- 172.21.15.46:0/2592131054 >> v2:172.21.15.142:3300/0 conn(0x7f19dc0cd770 0x7f19dc0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566138800 0 0) 0x7f19dc0b03c0 con 0x7f19dc15edf0 2024-04-26T23:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19c4003520 con 0x7f19dc15edf0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031310651 0 0) 0x7f19c4003520 con 0x7f19dc15edf0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19dc15f1d0 msgr2=0x7f19dc0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19dc15f1d0 0x7f19dc0cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 >> v2:172.21.15.142:3300/0 conn(0x7f19dc0cd770 msgr2=0x7f19dc0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 --2- 172.21.15.46:0/2592131054 >> v2:172.21.15.142:3300/0 conn(0x7f19dc0cd770 0x7f19dc0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc0cb960 con 0x7f19dc15edf0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f19d0001690 con 0x7f19dc15edf0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 >> v1:172.21.15.46:6789/0 conn(0x7f19dc15edf0 legacy=0x7f19dc0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c40039b0 0x7f19c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19c4004360 0x7f19c4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19c4004eb0 0x7f19c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0cf5c0 con 0x7f19c4004360 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0cb680 con 0x7f19c4004eb0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.245+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19c400bf00 con 0x7f19c40039b0 2024-04-26T23:42:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19c4004360 0x7f19c4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19e0c00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c40039b0 0x7f19c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19e1600640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19c4004eb0 0x7f19c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 -- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c40039b0 msgr2=0x7f19c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c40039b0 0x7f19c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 -- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19c4004eb0 msgr2=0x7f19c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19c4004eb0 0x7f19c400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.246+0000 7f19dbe00640 1 -- 172.21.15.46:0/2592131054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c4003520 con 0x7f19c4004360 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.247+0000 7f19dbe00640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19c4004360 0x7f19c4004840 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009ac0 tx=0x7f19c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32565e403f32f714 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.247+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8036070 con 0x7f19c4004360 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.247+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c8031400 con 0x7f19c4004360 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.247+0000 7f19db400640 1 -- 172.21.15.46:0/2592131054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8009690 con 0x7f19c4004360 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 -- 172.21.15.46:0/2592131054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19c4004360 msgr2=0x7f19c4004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 --2- 172.21.15.46:0/2592131054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19c4004360 0x7f19c4004840 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009ac0 tx=0x7f19c4005830 comp rx=0 tx=0).stop 2024-04-26T23:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e1600640 1 -- 172.21.15.46:0/2592131054 reap_dead start 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 -- 172.21.15.46:0/2592131054 shutdown_connections 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 -- 172.21.15.46:0/2592131054 >> 172.21.15.46:0/2592131054 conn(0x7f19dc0b26d0 msgr2=0x7f19dc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 -- 172.21.15.46:0/2592131054 shutdown_connections 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.248+0000 7f19e2000640 1 -- 172.21.15.46:0/2592131054 wait complete. 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.250+0000 7f19e2000640 1 Processor -- start 2024-04-26T23:42:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.250+0000 7f19e2000640 1 -- start start 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c4004360 0x7f19dc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19dc1588f0 0x7f19dc0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0cc200 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0cbf20 con 0x7f19dc1588f0 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0cb1a0 con 0x7f19c4004360 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52948/0 (socket says 172.21.15.46:52948) 2024-04-26T23:42:35.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.251+0000 7f19e1600640 1 -- 172.21.15.46:0/3120875738 learned_addr learned my addr 172.21.15.46:0/3120875738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19dbe00640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19dc1588f0 0x7f19dc0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e0c00640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c4004360 0x7f19dc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 -- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c4004360 msgr2=0x7f19dc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c4004360 0x7f19dc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 -- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19dc1588f0 msgr2=0x7f19dc0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19dc1588f0 0x7f19dc0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 -- 172.21.15.46:0/3120875738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc072e20 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e1600640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009fd0 tx=0x7f19d400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4016620 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d401b070 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4016a20 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.252+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc159640 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.253+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc159c00 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.254+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc0693b0 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.255+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19d4005d10 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.257+0000 7f19d8c00640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 0x7f19bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.257+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f19d408c5e0 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.257+0000 7f19e0c00640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 0x7f19bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.258+0000 7f19e0c00640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 0x7f19bc06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f19d0001050 tx=0x7f19d0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.262+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19d4024020 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.509+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19dc076430 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.510+0000 7f19d8c00640 1 -- 172.21.15.46:0/3120875738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5182 (secure 0 0 0) 0x7f19d405a7e0 con 0x7f19dc0b3ac0 2024-04-26T23:42:35.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:35.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:33.176845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 31 2024-04-26T23:42:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.513+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 msgr2=0x7f19bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.513+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 0x7f19bc06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f19d0001050 tx=0x7f19d0009040 comp rx=0 tx=0).stop 2024-04-26T23:42:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.513+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 msgr2=0x7f19dc1f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.513+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009fd0 tx=0x7f19d400f520 comp rx=0 tx=0).stop 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 shutdown_connections 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f19bc06c8f0 0x7f19bc06edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19dc0b3ac0 0x7f19dc1f7a60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19dc1588f0 0x7f19dc0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 --2- 172.21.15.46:0/3120875738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19c4004360 0x7f19dc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 >> 172.21.15.46:0/3120875738 conn(0x7f19dc0b26d0 msgr2=0x7f19dc0ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 shutdown_connections 2024-04-26T23:42:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:35.514+0000 7f19e2000640 1 -- 172.21.15.46:0/3120875738 wait complete. 2024-04-26T23:42:35.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-26T23:42:35.756 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:42:35.756 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:35.772 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-26T23:42:35.772 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:42:35.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:42:35.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:42:35.991 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:42:35.991 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:42:35.992 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:35.992 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:36.052 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:36.053 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:36.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.326+0000 7f48dca00640 1 Processor -- start 2024-04-26T23:42:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.326+0000 7f48dca00640 1 -- start start 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48dca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48d81454a0 0x7f48d80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48dca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d8072bb0 con 0x7f48d80c6a50 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48dca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d8072e90 con 0x7f48d8073970 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48dca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f48d8073170 con 0x7f48d81454a0 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48d7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f48d80c6a50 0x7f48d80cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45656/0 (socket says 172.21.15.46:45656) 2024-04-26T23:42:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.328+0000 7f48d7e00640 1 -- 172.21.15.46:0/470085914 learned_addr learned my addr 172.21.15.46:0/470085914 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d7400640 1 --2- 172.21.15.46:0/470085914 >> v2:172.21.15.142:3300/0 conn(0x7f48d81454a0 0x7f48d80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452137860 0 0) 0x7f48d8072bb0 con 0x7f48d80c6a50 2024-04-26T23:42:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48bc003520 con 0x7f48d80c6a50 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2205047543 0 0) 0x7f48bc003520 con 0x7f48d80c6a50 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 >> v2:172.21.15.142:3300/0 conn(0x7f48d81454a0 msgr2=0x7f48d80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 --2- 172.21.15.46:0/470085914 >> v2:172.21.15.142:3300/0 conn(0x7f48d81454a0 0x7f48d80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 >> v1:172.21.15.142:6789/0 conn(0x7f48d8073970 legacy=0x7f48d80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d8073450 con 0x7f48d80c6a50 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f48d00019f0 con 0x7f48d80c6a50 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 >> v1:172.21.15.46:6789/0 conn(0x7f48d80c6a50 legacy=0x7f48d80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0039b0 0x7f48bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc004800 0x7f48bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48bc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8073170 con 0x7f48bc0051c0 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc00e270 con 0x7f48bc004800 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.329+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc00e470 con 0x7f48bc0039b0 2024-04-26T23:42:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48bc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 -- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0039b0 msgr2=0x7f48bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7400640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0039b0 0x7f48bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d6a00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc004800 0x7f48bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0039b0 0x7f48bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 -- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc004800 msgr2=0x7f48bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc004800 0x7f48bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7e00640 1 -- 172.21.15.46:0/470085914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc003520 con 0x7f48bc0051c0 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d7400640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0039b0 0x7f48bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.330+0000 7f48d6a00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc004800 0x7f48bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.331+0000 7f48d7e00640 1 -- 172.21.15.46:0/470085914 reap_dead start 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.331+0000 7f48d7e00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48bc00dbd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f48d000a840 tx=0x7f48bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeb3a7a4eecfedf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d000a460 con 0x7f48bc0051c0 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d002f070 con 0x7f48bc0051c0 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48d6000640 1 -- 172.21.15.46:0/470085914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d002bcb0 con 0x7f48bc0051c0 2024-04-26T23:42:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 -- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 msgr2=0x7f48bc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48bc00dbd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f48d000a840 tx=0x7f48bc005830 comp rx=0 tx=0).stop 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 -- 172.21.15.46:0/470085914 shutdown_connections 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 --2- 172.21.15.46:0/470085914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48bc00dbd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 -- 172.21.15.46:0/470085914 >> 172.21.15.46:0/470085914 conn(0x7f48d80b26d0 msgr2=0x7f48d80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 -- 172.21.15.46:0/470085914 shutdown_connections 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.332+0000 7f48dca00640 1 -- 172.21.15.46:0/470085914 wait complete. 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.334+0000 7f48dca00640 1 Processor -- start 2024-04-26T23:42:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.334+0000 7f48dca00640 1 -- start start 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 0x7f48d80c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d80c8490 0x7f48d80c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8145880 con 0x7f48bc0051c0 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8073e00 con 0x7f48d80c8490 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.335+0000 7f48dca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48d8145a00 con 0x7f48d813f7d0 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52960/0 (socket says 172.21.15.46:52960) 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 -- 172.21.15.46:0/420172654 learned_addr learned my addr 172.21.15.46:0/420172654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d6a00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 0x7f48d80c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7e00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d80c8490 0x7f48d80c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 -- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 msgr2=0x7f48d80c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 0x7f48d80c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 -- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d80c8490 msgr2=0x7f48d80c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d80c8490 0x7f48d80c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 -- 172.21.15.46:0/420172654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d80c8e50 con 0x7f48bc0051c0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d6a00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 0x7f48d80c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.336+0000 7f48d7400640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f48c80108b0 tx=0x7f48c8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.337+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c800f040 con 0x7f48bc0051c0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.337+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c800eb60 con 0x7f48bc0051c0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.337+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c801c400 con 0x7f48bc0051c0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.337+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d8142860 con 0x7f48bc0051c0 2024-04-26T23:42:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.337+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d8142e20 con 0x7f48bc0051c0 2024-04-26T23:42:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.338+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d80693b0 con 0x7f48bc0051c0 2024-04-26T23:42:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.339+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48c801c5a0 con 0x7f48bc0051c0 2024-04-26T23:42:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.340+0000 7f48cf400640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 0x7f48b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.340+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f48c808c4d0 con 0x7f48bc0051c0 2024-04-26T23:42:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.341+0000 7f48d6a00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 0x7f48b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.342+0000 7f48d6a00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 0x7f48b806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48d815f400 tx=0x7f48c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.345+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48d80c8e50 con 0x7f48bc0051c0 2024-04-26T23:42:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.550+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48d806fe80 con 0x7f48bc0051c0 2024-04-26T23:42:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.553+0000 7f48cf400640 1 -- 172.21.15.46:0/420172654 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16906 (secure 0 0 0) 0x7f48c8018030 con 0x7f48bc0051c0 2024-04-26T23:42:36.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:36.558 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:58.143214+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 msgr2=0x7f48b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 0x7f48b806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48d815f400 tx=0x7f48c4009040 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 msgr2=0x7f48d8143900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f48c80108b0 tx=0x7f48c8010d70 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 shutdown_connections 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b806c8f0 0x7f48b806edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d80c8490 0x7f48d80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d813f7d0 0x7f48d80c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 --2- 172.21.15.46:0/420172654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0051c0 0x7f48d8143900 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 >> 172.21.15.46:0/420172654 conn(0x7f48d80b26d0 msgr2=0x7f48d80c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 shutdown_connections 2024-04-26T23:42:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.556+0000 7f48dca00640 1 -- 172.21.15.46:0/420172654 wait complete. 2024-04-26T23:42:36.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:41:54.469953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:41:55.605157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:42:36.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.775+0000 7f0b3cc00640 1 Processor -- start 2024-04-26T23:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.775+0000 7f0b3cc00640 1 -- start start 2024-04-26T23:42:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b3cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b380b6a40 0x7f0b380708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b3cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380d1fd0 0x7f0b380d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b3cc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b3806d160 con 0x7f0b38162890 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b3cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0b380b6ea0 con 0x7f0b380d1fd0 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b3cc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3806cba0 con 0x7f0b380b6a40 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b36800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380d1fd0 0x7f0b380d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b36800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380d1fd0 0x7f0b380d2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52974/0 (socket says 172.21.15.46:52974) 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b36800640 1 -- 172.21.15.46:0/2361695622 learned_addr learned my addr 172.21.15.46:0/2361695622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.777+0000 7f0b2fe00640 1 --2- 172.21.15.46:0/2361695622 >> v2:172.21.15.142:3300/0 conn(0x7f0b380b6a40 0x7f0b380708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.778+0000 7f0b36800640 1 -- 172.21.15.46:0/2361695622 >> v2:172.21.15.142:3300/0 conn(0x7f0b380b6a40 msgr2=0x7f0b380708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.778+0000 7f0b36800640 1 --2- 172.21.15.46:0/2361695622 >> v2:172.21.15.142:3300/0 conn(0x7f0b380b6a40 0x7f0b380708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.778+0000 7f0b36800640 1 -- 172.21.15.46:0/2361695622 >> v1:172.21.15.142:6789/0 conn(0x7f0b38162890 legacy=0x7f0b3806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.778+0000 7f0b36800640 1 -- 172.21.15.46:0/2361695622 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3806ce80 con 0x7f0b380d1fd0 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.779+0000 7f0b36800640 1 --2- 172.21.15.46:0/2361695622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380d1fd0 0x7f0b380d2460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0b3806d3c0 tx=0x7f0b3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158994f8a1009113 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.779+0000 7f0b35400640 1 -- 172.21.15.46:0/2361695622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30036070 con 0x7f0b380d1fd0 2024-04-26T23:42:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.779+0000 7f0b35400640 1 -- 172.21.15.46:0/2361695622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b30031600 con 0x7f0b380d1fd0 2024-04-26T23:42:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.779+0000 7f0b35400640 1 -- 172.21.15.46:0/2361695622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b300319d0 con 0x7f0b380d1fd0 2024-04-26T23:42:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 -- 172.21.15.46:0/2361695622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380d1fd0 msgr2=0x7f0b380d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/2361695622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380d1fd0 0x7f0b380d2460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0b3806d3c0 tx=0x7f0b3002a660 comp rx=0 tx=0).stop 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 -- 172.21.15.46:0/2361695622 shutdown_connections 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/2361695622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380d1fd0 0x7f0b380d2460 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/2361695622 >> v2:172.21.15.142:3300/0 conn(0x7f0b380b6a40 0x7f0b380708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 -- 172.21.15.46:0/2361695622 >> 172.21.15.46:0/2361695622 conn(0x7f0b3806d640 msgr2=0x7f0b3806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 -- 172.21.15.46:0/2361695622 shutdown_connections 2024-04-26T23:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.780+0000 7f0b3cc00640 1 -- 172.21.15.46:0/2361695622 wait complete. 2024-04-26T23:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.782+0000 7f0b3cc00640 1 Processor -- start 2024-04-26T23:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.783+0000 7f0b3cc00640 1 -- start start 2024-04-26T23:42:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 0x7f0b3814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d1fd0 0x7f0b381e9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38070e30 con 0x7f0b38162890 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380703d0 con 0x7f0b380b6a40 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b3cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b38162d30 con 0x7f0b380d1fd0 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b36800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b36800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52978/0 (socket says 172.21.15.46:52978) 2024-04-26T23:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.784+0000 7f0b36800640 1 -- 172.21.15.46:0/163747478 learned_addr learned my addr 172.21.15.46:0/163747478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b35e00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 0x7f0b3814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b2fe00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d1fd0 0x7f0b381e9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 -- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d1fd0 msgr2=0x7f0b381e9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d1fd0 0x7f0b381e9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 -- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 msgr2=0x7f0b3814b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 0x7f0b3814b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 -- 172.21.15.46:0/163747478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3814bdb0 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b35e00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 0x7f0b3814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.785+0000 7f0b36800640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f40 tx=0x7f0b30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.786+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30036050 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.786+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b30042070 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.786+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381eb5a0 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.786+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009510 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.786+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381ea7e0 con 0x7f0b38162890 2024-04-26T23:42:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.787+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38069530 con 0x7f0b38162890 2024-04-26T23:42:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.788+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b30007bd0 con 0x7f0b38162890 2024-04-26T23:42:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.789+0000 7f0b2e000640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 0x7f0b1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.789+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0b300acc60 con 0x7f0b38162890 2024-04-26T23:42:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.790+0000 7f0b35e00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 0x7f0b1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.791+0000 7f0b35e00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 0x7f0b1406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800a830 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:36.794+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b3814bdb0 con 0x7f0b38162890 2024-04-26T23:42:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.003+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b38073b80 con 0x7f0b38162890 2024-04-26T23:42:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.004+0000 7f0b2e000640 1 -- 172.21.15.46:0/163747478 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f0b3007ae70 con 0x7f0b38162890 2024-04-26T23:42:37.008 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:42:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 msgr2=0x7f0b1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 0x7f0b1406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800a830 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2024-04-26T23:42:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 msgr2=0x7f0b380cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f40 tx=0x7f0b30004a40 comp rx=0 tx=0).stop 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 shutdown_connections 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1406c8f0 0x7f0b1406edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b38162890 0x7f0b380cf800 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d1fd0 0x7f0b381e9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 --2- 172.21.15.46:0/163747478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b6a40 0x7f0b3814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 >> 172.21.15.46:0/163747478 conn(0x7f0b3806d640 msgr2=0x7f0b3806eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 shutdown_connections 2024-04-26T23:42:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.007+0000 7f0b3cc00640 1 -- 172.21.15.46:0/163747478 wait complete. 2024-04-26T23:42:37.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:42:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.249+0000 7f64e2000640 1 Processor -- start 2024-04-26T23:42:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.249+0000 7f64e2000640 1 -- start start 2024-04-26T23:42:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64e2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f64dc0b53b0 0x7f64dc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64e2000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64dc06eb90 con 0x7f64dc15e050 2024-04-26T23:42:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64e2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc0b4310 con 0x7f64dc0b53b0 2024-04-26T23:42:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc0b45f0 con 0x7f64dc15d730 2024-04-26T23:42:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64db400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64daa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f64dc0b53b0 0x7f64dc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64db400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:38096/0 (socket says 172.21.15.46:38096) 2024-04-26T23:42:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.251+0000 7f64db400640 1 -- 172.21.15.46:0/2782439820 learned_addr learned my addr 172.21.15.46:0/2782439820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.252+0000 7f64daa00640 1 -- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 msgr2=0x7f64dc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.252+0000 7f64daa00640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.252+0000 7f64daa00640 1 -- 172.21.15.46:0/2782439820 >> v1:172.21.15.142:6789/0 conn(0x7f64dc15e050 legacy=0x7f64dc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.252+0000 7f64daa00640 1 -- 172.21.15.46:0/2782439820 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc0b48d0 con 0x7f64dc0b53b0 2024-04-26T23:42:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.252+0000 7f64db400640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:42:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.254+0000 7f64daa00640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15db10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64c40097a0 tx=0x7f64c402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ae3cb2b96e2f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.254+0000 7f64da000640 1 -- 172.21.15.46:0/2782439820 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4037070 con 0x7f64dc0b53b0 2024-04-26T23:42:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.254+0000 7f64da000640 1 -- 172.21.15.46:0/2782439820 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c4032440 con 0x7f64dc0b53b0 2024-04-26T23:42:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.254+0000 7f64da000640 1 -- 172.21.15.46:0/2782439820 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4032810 con 0x7f64dc0b53b0 2024-04-26T23:42:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 -- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 msgr2=0x7f64dc15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15db10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64c40097a0 tx=0x7f64c402a180 comp rx=0 tx=0).stop 2024-04-26T23:42:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 -- 172.21.15.46:0/2782439820 shutdown_connections 2024-04-26T23:42:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15db10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 --2- 172.21.15.46:0/2782439820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15d730 0x7f64dc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 -- 172.21.15.46:0/2782439820 >> 172.21.15.46:0/2782439820 conn(0x7f64dc06c980 msgr2=0x7f64dc0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 -- 172.21.15.46:0/2782439820 shutdown_connections 2024-04-26T23:42:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.255+0000 7f64e2000640 1 -- 172.21.15.46:0/2782439820 wait complete. 2024-04-26T23:42:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.257+0000 7f64e2000640 1 Processor -- start 2024-04-26T23:42:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.257+0000 7f64e2000640 1 -- start start 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15e050 0x7f64dc073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc15e4c0 con 0x7f64dc15d730 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc0b4e90 con 0x7f64dc0b53b0 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.258+0000 7f64e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc0b3ec0 con 0x7f64dc15e050 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52986/0 (socket says 172.21.15.46:52986) 2024-04-26T23:42:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 -- 172.21.15.46:0/2412347916 learned_addr learned my addr 172.21.15.46:0/2412347916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64dbe00640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15e050 0x7f64dc073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64db400640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 -- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15e050 msgr2=0x7f64dc073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15e050 0x7f64dc073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 -- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 msgr2=0x7f64dc15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 -- 172.21.15.46:0/2412347916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc073c40 con 0x7f64dc15d730 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.259+0000 7f64daa00640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64c4035040 tx=0x7f64c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.260+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4037040 con 0x7f64dc15d730 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.260+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c4030500 con 0x7f64dc15d730 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.260+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc15a970 con 0x7f64dc15d730 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.260+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4030910 con 0x7f64dc15d730 2024-04-26T23:42:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.260+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc13aa50 con 0x7f64dc15d730 2024-04-26T23:42:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.261+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc069530 con 0x7f64dc15d730 2024-04-26T23:42:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.262+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64c4044020 con 0x7f64dc15d730 2024-04-26T23:42:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.263+0000 7f64d3400640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 0x7f64b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.263+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f64c40abf60 con 0x7f64dc15d730 2024-04-26T23:42:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.264+0000 7f64db400640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 0x7f64b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.265+0000 7f64db400640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 0x7f64b806ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009b70 tx=0x7f64cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.268+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64c407a160 con 0x7f64dc15d730 2024-04-26T23:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.475+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64dc072e80 con 0x7f64dc15d730 2024-04-26T23:42:37.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.476+0000 7f64d3400640 1 -- 172.21.15.46:0/2412347916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f64c4079a60 con 0x7f64dc15d730 2024-04-26T23:42:37.480 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 msgr2=0x7f64b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 0x7f64b806ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009b70 tx=0x7f64cc008040 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 msgr2=0x7f64dc1f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64c4035040 tx=0x7f64c40046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 shutdown_connections 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f64b806c920 0x7f64b806ede0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64dc15e050 0x7f64dc073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc15d730 0x7f64dc1f4910 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 --2- 172.21.15.46:0/2412347916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64dc0b53b0 0x7f64dc15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 >> 172.21.15.46:0/2412347916 conn(0x7f64dc06c980 msgr2=0x7f64dc15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 shutdown_connections 2024-04-26T23:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.479+0000 7f64e2000640 1 -- 172.21.15.46:0/2412347916 wait complete. 2024-04-26T23:42:37.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:42:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.711+0000 7f2f72800640 1 Processor -- start 2024-04-26T23:42:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.711+0000 7f2f72800640 1 -- start start 2024-04-26T23:42:37.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.712+0000 7f2f72800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2f6c15d730 0x7f2f6c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.712+0000 7f2f72800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f6c15e220 0x7f2f6c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.712+0000 7f2f72800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c06eb90 con 0x7f2f6c0b53b0 2024-04-26T23:42:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.712+0000 7f2f72800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0b4310 con 0x7f2f6c15e220 2024-04-26T23:42:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.712+0000 7f2f72800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0b45f0 con 0x7f2f6c15d730 2024-04-26T23:42:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f6c15e220 0x7f2f6c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f6c15e220 0x7f2f6c06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53002/0 (socket says 172.21.15.46:53002) 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 -- 172.21.15.46:0/3457602944 learned_addr learned my addr 172.21.15.46:0/3457602944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6b400640 1 --2- 172.21.15.46:0/3457602944 >> v2:172.21.15.142:3300/0 conn(0x7f2f6c15d730 0x7f2f6c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 -- 172.21.15.46:0/3457602944 >> v2:172.21.15.142:3300/0 conn(0x7f2f6c15d730 msgr2=0x7f2f6c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 --2- 172.21.15.46:0/3457602944 >> v2:172.21.15.142:3300/0 conn(0x7f2f6c15d730 0x7f2f6c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 -- 172.21.15.46:0/3457602944 >> v1:172.21.15.142:6789/0 conn(0x7f2f6c0b53b0 legacy=0x7f2f6c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.713+0000 7f2f6be00640 1 -- 172.21.15.46:0/3457602944 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c0b48d0 con 0x7f2f6c15e220 2024-04-26T23:42:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.715+0000 7f2f6be00640 1 --2- 172.21.15.46:0/3457602944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c06dc90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06fa10 tx=0x7f2f6402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6798d046a4c1fa63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.715+0000 7f2f6aa00640 1 -- 172.21.15.46:0/3457602944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036070 con 0x7f2f6c15e220 2024-04-26T23:42:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.715+0000 7f2f6aa00640 1 -- 172.21.15.46:0/3457602944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f64031600 con 0x7f2f6c15e220 2024-04-26T23:42:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.715+0000 7f2f6aa00640 1 -- 172.21.15.46:0/3457602944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640319d0 con 0x7f2f6c15e220 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 -- 172.21.15.46:0/3457602944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 msgr2=0x7f2f6c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 --2- 172.21.15.46:0/3457602944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c06dc90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06fa10 tx=0x7f2f6402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 -- 172.21.15.46:0/3457602944 shutdown_connections 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 --2- 172.21.15.46:0/3457602944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c06dc90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 --2- 172.21.15.46:0/3457602944 >> v2:172.21.15.142:3300/0 conn(0x7f2f6c15d730 0x7f2f6c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 -- 172.21.15.46:0/3457602944 >> 172.21.15.46:0/3457602944 conn(0x7f2f6c06c980 msgr2=0x7f2f6c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 -- 172.21.15.46:0/3457602944 shutdown_connections 2024-04-26T23:42:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.716+0000 7f2f72800640 1 -- 172.21.15.46:0/3457602944 wait complete. 2024-04-26T23:42:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.717+0000 7f2f72800640 1 Processor -- start 2024-04-26T23:42:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.718+0000 7f2f72800640 1 -- start start 2024-04-26T23:42:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f6c0b53b0 0x7f2f6c1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f6c15d730 0x7f2f6c1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0b3ec0 con 0x7f2f6c15e220 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0b4e90 con 0x7f2f6c0b53b0 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f72800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0b4040 con 0x7f2f6c15d730 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53014/0 (socket says 172.21.15.46:53014) 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 -- 172.21.15.46:0/4225307671 learned_addr learned my addr 172.21.15.46:0/4225307671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6b400640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f6c0b53b0 0x7f2f6c1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f63e00640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f6c15d730 0x7f2f6c1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 -- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f6c15d730 msgr2=0x7f2f6c1430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f6c15d730 0x7f2f6c1430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 -- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f6c0b53b0 msgr2=0x7f2f6c1f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f6c0b53b0 0x7f2f6c1f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.719+0000 7f2f6be00640 1 -- 172.21.15.46:0/4225307671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c0aeb30 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f6be00640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009f40 tx=0x7f2f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036050 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f64042070 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64009510 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c0aee10 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.720+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c0af3d0 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.722+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c069530 con 0x7f2f6c15e220 2024-04-26T23:42:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.722+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f64007bd0 con 0x7f2f6c15e220 2024-04-26T23:42:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.723+0000 7f2f63400640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.724+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2f640acca0 con 0x7f2f6c15e220 2024-04-26T23:42:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.724+0000 7f2f6b400640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.725+0000 7f2f6b400640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006fd0 tx=0x7f2f5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:37.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.728+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f64045360 con 0x7f2f6c15e220 2024-04-26T23:42:37.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.934+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f6c0af5a0 con 0x7f2f6c15e220 2024-04-26T23:42:37.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.935+0000 7f2f63400640 1 -- 172.21.15.46:0/4225307671 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f2f6407aea0 con 0x7f2f6c15e220 2024-04-26T23:42:37.939 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.937+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 msgr2=0x7f2f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.937+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006fd0 tx=0x7f2f5c008040 comp rx=0 tx=0).stop 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.937+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 msgr2=0x7f2f6c143610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.937+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009f40 tx=0x7f2f64004a40 comp rx=0 tx=0).stop 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 shutdown_connections 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c15e220 0x7f2f6c143610 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f6c15d730 0x7f2f6c1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 --2- 172.21.15.46:0/4225307671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f6c0b53b0 0x7f2f6c1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 >> 172.21.15.46:0/4225307671 conn(0x7f2f6c06c980 msgr2=0x7f2f6c15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 shutdown_connections 2024-04-26T23:42:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:37.938+0000 7f2f72800640 1 -- 172.21.15.46:0/4225307671 wait complete. 2024-04-26T23:42:37.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:42:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.165+0000 7fa450e00640 1 Processor -- start 2024-04-26T23:42:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.165+0000 7fa450e00640 1 -- start start 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.166+0000 7fa450e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 0x7fa44c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.166+0000 7fa450e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa44c0d31f0 0x7fa44c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.166+0000 7fa450e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c15fd30 con 0x7fa44c0d2030 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.166+0000 7fa450e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c162d60 con 0x7fa44c0d31f0 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.166+0000 7fa450e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c0d24a0 con 0x7fa44c162980 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa44c0d31f0 0x7fa44c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa44c0d31f0 0x7fa44c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53022/0 (socket says 172.21.15.46:53022) 2024-04-26T23:42:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 -- 172.21.15.46:0/3410686386 learned_addr learned my addr 172.21.15.46:0/3410686386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449e00640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 0x7fa44c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 -- 172.21.15.46:0/3410686386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 msgr2=0x7fa44c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 0x7fa44c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 -- 172.21.15.46:0/3410686386 >> v1:172.21.15.142:6789/0 conn(0x7fa44c0d2030 legacy=0x7fa44c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449400640 1 -- 172.21.15.46:0/3410686386 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0d1050 con 0x7fa44c0d31f0 2024-04-26T23:42:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.167+0000 7fa449e00640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 0x7fa44c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa449400640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c0d31f0 0x7fa44c0d1af0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009e50 tx=0x7fa43c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d41cb20bff70a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa448a00640 1 -- 172.21.15.46:0/3410686386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c036070 con 0x7fa44c0d31f0 2024-04-26T23:42:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa448a00640 1 -- 172.21.15.46:0/3410686386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c031600 con 0x7fa44c0d31f0 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa448a00640 1 -- 172.21.15.46:0/3410686386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0319d0 con 0x7fa44c0d31f0 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa450e00640 1 -- 172.21.15.46:0/3410686386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c0d31f0 msgr2=0x7fa44c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.169+0000 7fa450e00640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c0d31f0 0x7fa44c0d1af0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009e50 tx=0x7fa43c02a6d0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 -- 172.21.15.46:0/3410686386 shutdown_connections 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c0d31f0 0x7fa44c0d1af0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 --2- 172.21.15.46:0/3410686386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c162980 0x7fa44c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 -- 172.21.15.46:0/3410686386 >> 172.21.15.46:0/3410686386 conn(0x7fa44c09e420 msgr2=0x7fa44c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 -- 172.21.15.46:0/3410686386 shutdown_connections 2024-04-26T23:42:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.170+0000 7fa450e00640 1 -- 172.21.15.46:0/3410686386 wait complete. 2024-04-26T23:42:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.171+0000 7fa450e00640 1 Processor -- start 2024-04-26T23:42:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.172+0000 7fa450e00640 1 -- start start 2024-04-26T23:42:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c0d2030 0x7fa44c0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44c0d31f0 0x7fa44c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1605d0 con 0x7fa44c162980 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1602f0 con 0x7fa44c0d31f0 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa450e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c1608b0 con 0x7fa44c0d2030 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53028/0 (socket says 172.21.15.46:53028) 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 -- 172.21.15.46:0/3625115230 learned_addr learned my addr 172.21.15.46:0/3625115230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa449e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c0d2030 0x7fa44c0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa449400640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44c0d31f0 0x7fa44c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 -- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c0d2030 msgr2=0x7fa44c0cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c0d2030 0x7fa44c0cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 -- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44c0d31f0 msgr2=0x7fa44c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.173+0000 7fa44a800640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44c0d31f0 0x7fa44c1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa44a800640 1 -- 172.21.15.46:0/3625115230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c15dfb0 con 0x7fa44c162980 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa44a800640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b440 tx=0x7fa44400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444015020 con 0x7fa44c162980 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444011830 con 0x7fa44c162980 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444011c20 con 0x7fa44c162980 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c0cf440 con 0x7fa44c162980 2024-04-26T23:42:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.174+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c0b4e80 con 0x7fa44c162980 2024-04-26T23:42:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.175+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c069530 con 0x7fa44c162980 2024-04-26T23:42:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.176+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa444018020 con 0x7fa44c162980 2024-04-26T23:42:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.177+0000 7fa442000640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.178+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa44408b860 con 0x7fa44c162980 2024-04-26T23:42:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.178+0000 7fa449e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.179+0000 7fa449e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa43400a990 tx=0x7fa434008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:38.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.182+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa44401f030 con 0x7fa44c162980 2024-04-26T23:42:38.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.399+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa44c0d2690 con 0x7fa44c162980 2024-04-26T23:42:38.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.400+0000 7fa442000640 1 -- 172.21.15.46:0/3625115230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7fa44c15dfb0 con 0x7fa44c162980 2024-04-26T23:42:38.405 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 msgr2=0x7fa42806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa43400a990 tx=0x7fa434008040 comp rx=0 tx=0).stop 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 msgr2=0x7fa44c15efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b440 tx=0x7fa44400b900 comp rx=0 tx=0).stop 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 shutdown_connections 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa44c162980 0x7fa44c15efc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44c0d31f0 0x7fa44c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 --2- 172.21.15.46:0/3625115230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa44c0d2030 0x7fa44c0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.403+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 >> 172.21.15.46:0/3625115230 conn(0x7fa44c09e420 msgr2=0x7fa44c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:38.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.404+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 shutdown_connections 2024-04-26T23:42:38.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:38.404+0000 7fa450e00640 1 -- 172.21.15.46:0/3625115230 wait complete. 2024-04-26T23:42:38.429 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:42:38.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:38.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:42:39.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:39.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:40.846 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:42:40.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:42:40.847 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:42:40.885 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:42:40.886 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:42:40.886 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:40.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:42:40.939 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:42:40.939 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:40.939 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:40.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:42:40.986 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:42:40.986 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:42:40.986 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:41.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:42:41.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.257+0000 7f81d5c00640 1 Processor -- start 2024-04-26T23:42:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.257+0000 7f81d5c00640 1 -- start start 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.258+0000 7f81d5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81d00b4000 0x7f81d00d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f81d00d1f60 0x7f81d00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f81d00d2990 con 0x7f81d00b4000 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f81d00d11f0 con 0x7f81d00d1f60 2024-04-26T23:42:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81d00d5f20 con 0x7f81d015f4e0 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81d00b4000 0x7f81d00d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81d00b4000 0x7f81d00d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53044/0 (socket says 172.21.15.46:53044) 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81c7400640 1 -- 172.21.15.46:0/690929641 learned_addr learned my addr 172.21.15.46:0/690929641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81cfe00640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.259+0000 7f81d5200640 1 --2- 172.21.15.46:0/690929641 >> v2:172.21.15.142:3300/0 conn(0x7f81d00d1f60 0x7f81d00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.260+0000 7f81c7400640 1 -- 172.21.15.46:0/690929641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 msgr2=0x7f81d00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.260+0000 7f81c7400640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.260+0000 7f81c7400640 1 -- 172.21.15.46:0/690929641 >> v2:172.21.15.142:3300/0 conn(0x7f81d00d1f60 msgr2=0x7f81d00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.260+0000 7f81c7400640 1 --2- 172.21.15.46:0/690929641 >> v2:172.21.15.142:3300/0 conn(0x7f81d00d1f60 0x7f81d00cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.260+0000 7f81c7400640 1 -- 172.21.15.46:0/690929641 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d00d0f10 con 0x7f81d00b4000 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.261+0000 7f81c7400640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d00d1a20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f81d00bb0c0 tx=0x7f81b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476f54001e2b97df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.261+0000 7f81cf400640 1 -- 172.21.15.46:0/690929641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8037070 con 0x7f81d00b4000 2024-04-26T23:42:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.261+0000 7f81cf400640 1 -- 172.21.15.46:0/690929641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8032440 con 0x7f81d00b4000 2024-04-26T23:42:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.261+0000 7f81cf400640 1 -- 172.21.15.46:0/690929641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8032810 con 0x7f81d00b4000 2024-04-26T23:42:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 -- 172.21.15.46:0/690929641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 msgr2=0x7f81d00d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d00d1a20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f81d00bb0c0 tx=0x7f81b802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 -- 172.21.15.46:0/690929641 shutdown_connections 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 --2- 172.21.15.46:0/690929641 >> v2:172.21.15.142:3300/0 conn(0x7f81d00d1f60 0x7f81d00cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d00d1a20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 --2- 172.21.15.46:0/690929641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 -- 172.21.15.46:0/690929641 >> 172.21.15.46:0/690929641 conn(0x7f81d00d5010 msgr2=0x7f81d00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 -- 172.21.15.46:0/690929641 shutdown_connections 2024-04-26T23:42:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.262+0000 7f81d5c00640 1 -- 172.21.15.46:0/690929641 wait complete. 2024-04-26T23:42:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.264+0000 7f81d5c00640 1 Processor -- start 2024-04-26T23:42:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.264+0000 7f81d5c00640 1 -- start start 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00d1f60 0x7f81d01f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d01f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81d015f8c0 con 0x7f81d00b4000 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81d00b1d60 con 0x7f81d00d1f60 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81d5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81d00d0210 con 0x7f81d015f4e0 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53046/0 (socket says 172.21.15.46:53046) 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.265+0000 7f81cfe00640 1 -- 172.21.15.46:0/1434140718 learned_addr learned my addr 172.21.15.46:0/1434140718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 -- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 msgr2=0x7f81d01f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d01f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 -- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00d1f60 msgr2=0x7f81d01f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00d1f60 0x7f81d01f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 -- 172.21.15.46:0/1434140718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d01f6240 con 0x7f81d00b4000 2024-04-26T23:42:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81cfe00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f81c0007e40 tx=0x7f81c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0014620 con 0x7f81d00b4000 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c0019070 con 0x7f81d00b4000 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00149e0 con 0x7f81d00b4000 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.266+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d01f6520 con 0x7f81d00b4000 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.267+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d01f6c60 con 0x7f81d00b4000 2024-04-26T23:42:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.268+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d00693b0 con 0x7f81d00b4000 2024-04-26T23:42:41.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.269+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81c0014b80 con 0x7f81d00b4000 2024-04-26T23:42:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.270+0000 7f81ccc00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 0x7f81ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.270+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f81c008c600 con 0x7f81d00b4000 2024-04-26T23:42:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.270+0000 7f81c7400640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 0x7f81ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.271+0000 7f81c7400640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 0x7f81ac06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034000 tx=0x7f81b802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.275+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81c005a800 con 0x7f81d00b4000 2024-04-26T23:42:41.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.520+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81d01f7080 con 0x7f81d00b4000 2024-04-26T23:42:41.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.521+0000 7f81ccc00640 1 -- 172.21.15.46:0/1434140718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5182 (secure 0 0 0) 0x7f81c005a100 con 0x7f81d00b4000 2024-04-26T23:42:41.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:41.525 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:37.177552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:41.525 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-04-26T23:42:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.523+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 msgr2=0x7f81ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.523+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 0x7f81ac06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034000 tx=0x7f81b802c000 comp rx=0 tx=0).stop 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.523+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 msgr2=0x7f81d0151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.523+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f81c0007e40 tx=0x7f81c000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 shutdown_connections 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f81ac06c920 0x7f81ac06ede0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d015f4e0 0x7f81d01f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00d1f60 0x7f81d01f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 --2- 172.21.15.46:0/1434140718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d00b4000 0x7f81d0151570 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 >> 172.21.15.46:0/1434140718 conn(0x7f81d00d5010 msgr2=0x7f81d015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:41.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 shutdown_connections 2024-04-26T23:42:41.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.524+0000 7f81d5c00640 1 -- 172.21.15.46:0/1434140718 wait complete. 2024-04-26T23:42:41.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.749+0000 7f3636000640 1 Processor -- start 2024-04-26T23:42:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.749+0000 7f3636000640 1 -- start start 2024-04-26T23:42:41.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.751+0000 7f3636000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f36300cc690 0x7f36300cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.751+0000 7f3636000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3630072bb0 con 0x7f36301454a0 2024-04-26T23:42:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.751+0000 7f3636000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3630072e90 con 0x7f36300c6a50 2024-04-26T23:42:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.751+0000 7f3636000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3630073170 con 0x7f36300cc690 2024-04-26T23:42:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.752+0000 7f3634c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f36301454a0 0x7f3630073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45662/0 (socket says 172.21.15.46:45662) 2024-04-26T23:42:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.752+0000 7f3634c00640 1 -- 172.21.15.46:0/1892297845 learned_addr learned my addr 172.21.15.46:0/1892297845 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.752+0000 7f362fe00640 1 --2- 172.21.15.46:0/1892297845 >> v2:172.21.15.142:3300/0 conn(0x7f36300cc690 0x7f36300cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.752+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277961638 0 0) 0x7f3630072bb0 con 0x7f36301454a0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.752+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3618003520 con 0x7f36301454a0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087939139 0 0) 0x7f3618003520 con 0x7f36301454a0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 >> v2:172.21.15.142:3300/0 conn(0x7f36300cc690 msgr2=0x7f36300cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 --2- 172.21.15.46:0/1892297845 >> v2:172.21.15.142:3300/0 conn(0x7f36300cc690 0x7f36300cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 >> v1:172.21.15.142:6789/0 conn(0x7f36300c6a50 legacy=0x7f36300cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3630073450 con 0x7f36301454a0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3624001670 con 0x7f36301454a0 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 >> v1:172.21.15.46:6789/0 conn(0x7f36301454a0 legacy=0x7f3630073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3618003a00 0x7f3618003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618005210 0x7f361800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3630073170 con 0x7f3618004800 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f361800c140 con 0x7f3618005210 2024-04-26T23:42:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f361800c340 con 0x7f3618003a00 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.753+0000 7f362fe00640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f3634c00640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3618003a00 0x7f3618003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f362fe00640 1 -- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3618003a00 msgr2=0x7f3618003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f362fe00640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3618003a00 0x7f3618003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f362fe00640 1 -- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618005210 msgr2=0x7f361800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f3635600640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618005210 0x7f361800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f362fe00640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618005210 0x7f361800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f362fe00640 1 -- 172.21.15.46:0/1892297845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3618003520 con 0x7f3618004800 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f3635600640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618005210 0x7f361800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.754+0000 7f3635600640 1 -- 172.21.15.46:0/1892297845 reap_dead start 2024-04-26T23:42:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.755+0000 7f362fe00640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 0x7f3618004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f361c00ba00 tx=0x7f3618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281c082af04909df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.756+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c030070 con 0x7f3618004800 2024-04-26T23:42:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.756+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c02adb0 con 0x7f3618004800 2024-04-26T23:42:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.756+0000 7f362f400640 1 -- 172.21.15.46:0/1892297845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c00b600 con 0x7f3618004800 2024-04-26T23:42:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 -- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 msgr2=0x7f3618004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 0x7f3618004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f361c00ba00 tx=0x7f3618005830 comp rx=0 tx=0).stop 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 -- 172.21.15.46:0/1892297845 shutdown_connections 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 --2- 172.21.15.46:0/1892297845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 -- 172.21.15.46:0/1892297845 >> 172.21.15.46:0/1892297845 conn(0x7f36300b26d0 msgr2=0x7f36300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 -- 172.21.15.46:0/1892297845 shutdown_connections 2024-04-26T23:42:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.757+0000 7f3636000640 1 -- 172.21.15.46:0/1892297845 wait complete. 2024-04-26T23:42:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.759+0000 7f3636000640 1 Processor -- start 2024-04-26T23:42:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.759+0000 7f3636000640 1 -- start start 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618004800 0x7f36301415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36300b00e0 0x7f36301f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3630145a00 con 0x7f36301f7f70 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3630145880 con 0x7f3618004800 2024-04-26T23:42:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.760+0000 7f3636000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f36300c6ea0 con 0x7f36300b00e0 2024-04-26T23:42:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53068/0 (socket says 172.21.15.46:53068) 2024-04-26T23:42:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 -- 172.21.15.46:0/604983808 learned_addr learned my addr 172.21.15.46:0/604983808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3634c00640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618004800 0x7f36301415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f362fe00640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36300b00e0 0x7f36301f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 -- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36300b00e0 msgr2=0x7f36301f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36300b00e0 0x7f36301f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 -- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618004800 msgr2=0x7f36301415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618004800 0x7f36301415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.761+0000 7f3635600640 1 -- 172.21.15.46:0/604983808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36301f8850 con 0x7f36301f7f70 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f3635600640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f36300b05e0 tx=0x7f362800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628018070 con 0x7f36301f7f70 2024-04-26T23:42:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36301f8b30 con 0x7f36301f7f70 2024-04-26T23:42:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3628003d80 con 0x7f36301f7f70 2024-04-26T23:42:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280052b0 con 0x7f36301f7f70 2024-04-26T23:42:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.762+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301f5860 con 0x7f36301f7f70 2024-04-26T23:42:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.763+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36301f4940 con 0x7f36301f7f70 2024-04-26T23:42:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.765+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3628005450 con 0x7f36301f7f70 2024-04-26T23:42:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.766+0000 7f362cc00640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 0x7f361006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.766+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f362808b060 con 0x7f36301f7f70 2024-04-26T23:42:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.766+0000 7f3634c00640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 0x7f361006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.768+0000 7f3634c00640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 0x7f361006ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f3624009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.770+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36301f4940 con 0x7f36301f7f70 2024-04-26T23:42:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.977+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36300693b0 con 0x7f36301f7f70 2024-04-26T23:42:41.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.981+0000 7f362cc00640 1 -- 172.21.15.46:0/604983808 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16906 (secure 0 0 0) 0x7f3628059280 con 0x7f36301f7f70 2024-04-26T23:42:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:41.986 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:58.143214+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 msgr2=0x7f361006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 0x7f361006ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f3624009040 comp rx=0 tx=0).stop 2024-04-26T23:42:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 msgr2=0x7f36300cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f36300b05e0 tx=0x7f362800d960 comp rx=0 tx=0).stop 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 shutdown_connections 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f361006c920 0x7f361006ede0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36301f7f70 0x7f36300cb0d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36300b00e0 0x7f36301f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 --2- 172.21.15.46:0/604983808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3618004800 0x7f36301415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 >> 172.21.15.46:0/604983808 conn(0x7f36300b26d0 msgr2=0x7f3630160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 shutdown_connections 2024-04-26T23:42:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:41.984+0000 7f3636000640 1 -- 172.21.15.46:0/604983808 wait complete. 2024-04-26T23:42:42.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:41:54.469953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:41:55.605157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:42:42.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.204+0000 7f4188a00640 1 Processor -- start 2024-04-26T23:42:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.204+0000 7f4188a00640 1 -- start start 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4188a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f418415f210 0x7f41840cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4188a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840cd610 0x7f41840cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4188a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41840af570 con 0x7f41840ae8f0 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4188a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f41840cdfb0 con 0x7f418415f210 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4188a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41840af850 con 0x7f41840cd610 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4182800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f418415f210 0x7f41840cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4182800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f418415f210 0x7f41840cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59674/0 (socket says 172.21.15.46:59674) 2024-04-26T23:42:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.206+0000 7f4182800640 1 -- 172.21.15.46:0/89844529 learned_addr learned my addr 172.21.15.46:0/89844529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.207+0000 7f4179e00640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840cd610 0x7f41840cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.207+0000 7f4182800640 1 -- 172.21.15.46:0/89844529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840cd610 msgr2=0x7f41840cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.207+0000 7f4182800640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840cd610 0x7f41840cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.207+0000 7f4182800640 1 -- 172.21.15.46:0/89844529 >> v1:172.21.15.142:6789/0 conn(0x7f41840ae8f0 legacy=0x7f41840aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.207+0000 7f4182800640 1 -- 172.21.15.46:0/89844529 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41840cb680 con 0x7f418415f210 2024-04-26T23:42:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.208+0000 7f4182800640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840cc160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4174009870 tx=0x7f417402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8180a3804306636 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.208+0000 7f4181e00640 1 -- 172.21.15.46:0/89844529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174037070 con 0x7f418415f210 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.208+0000 7f4181e00640 1 -- 172.21.15.46:0/89844529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4174032440 con 0x7f418415f210 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.208+0000 7f4181e00640 1 -- 172.21.15.46:0/89844529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174032810 con 0x7f418415f210 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 -- 172.21.15.46:0/89844529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 msgr2=0x7f41840cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840cc160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4174009870 tx=0x7f417402a200 comp rx=0 tx=0).stop 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 -- 172.21.15.46:0/89844529 shutdown_connections 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840cd610 0x7f41840cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 --2- 172.21.15.46:0/89844529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840cc160 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 -- 172.21.15.46:0/89844529 >> 172.21.15.46:0/89844529 conn(0x7f41840b27b0 msgr2=0x7f41840b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 -- 172.21.15.46:0/89844529 shutdown_connections 2024-04-26T23:42:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.209+0000 7f4188a00640 1 -- 172.21.15.46:0/89844529 wait complete. 2024-04-26T23:42:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.211+0000 7f4188a00640 1 Processor -- start 2024-04-26T23:42:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.211+0000 7f4188a00640 1 -- start start 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840ae8f0 0x7f41841e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41840cd610 0x7f41840c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f418415f5f0 con 0x7f418415f210 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41840cbc40 con 0x7f41840cd610 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4188a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4184071e80 con 0x7f41840ae8f0 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4183200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4183200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59684/0 (socket says 172.21.15.46:59684) 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4183200640 1 -- 172.21.15.46:0/2543168580 learned_addr learned my addr 172.21.15.46:0/2543168580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4182800640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840ae8f0 0x7f41841e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.212+0000 7f4179e00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41840cd610 0x7f41840c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 -- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840ae8f0 msgr2=0x7f41841e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840ae8f0 0x7f41841e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 -- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41840cd610 msgr2=0x7f41840c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41840cd610 0x7f41840c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 -- 172.21.15.46:0/2543168580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41840caad0 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4183200640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c7b0 tx=0x7f417c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c017070 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c003da0 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41840ca230 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.213+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c00ce60 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.214+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841f4c40 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.215+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41840693b0 con 0x7f418415f210 2024-04-26T23:42:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.216+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f417c01a070 con 0x7f418415f210 2024-04-26T23:42:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.217+0000 7f417b400640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.217+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f417c08b320 con 0x7f418415f210 2024-04-26T23:42:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.217+0000 7f4182800640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.218+0000 7f4182800640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4174034000 tx=0x7f417402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.221+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f417c059530 con 0x7f418415f210 2024-04-26T23:42:42.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.418+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41841f4e10 con 0x7f418415f210 2024-04-26T23:42:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.419+0000 7f417b400640 1 -- 172.21.15.46:0/2543168580 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f417c058e30 con 0x7f418415f210 2024-04-26T23:42:42.423 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:42:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 msgr2=0x7f416006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4174034000 tx=0x7f417402e000 comp rx=0 tx=0).stop 2024-04-26T23:42:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 msgr2=0x7f41840c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c7b0 tx=0x7f417c00cc70 comp rx=0 tx=0).stop 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 shutdown_connections 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f418415f210 0x7f41840c5f50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41840cd610 0x7f41840c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 --2- 172.21.15.46:0/2543168580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41840ae8f0 0x7f41841e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 >> 172.21.15.46:0/2543168580 conn(0x7f41840b27b0 msgr2=0x7f418415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 shutdown_connections 2024-04-26T23:42:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.422+0000 7f4188a00640 1 -- 172.21.15.46:0/2543168580 wait complete. 2024-04-26T23:42:42.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:42:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.643+0000 7fb311600640 1 Processor -- start 2024-04-26T23:42:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.643+0000 7fb311600640 1 -- start start 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.644+0000 7fb311600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb30c15e220 0x7fb30c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.644+0000 7fb311600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb30c0d07e0 con 0x7fb30c15d730 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.644+0000 7fb311600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb30c06eb90 con 0x7fb30c0b53b0 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.644+0000 7fb311600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb30c0b4310 con 0x7fb30c15e220 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.645+0000 7fb30aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb30c15d730 0x7fb30c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37160/0 (socket says 172.21.15.46:37160) 2024-04-26T23:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.645+0000 7fb30aa00640 1 -- 172.21.15.46:0/3167338021 learned_addr learned my addr 172.21.15.46:0/3167338021 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.645+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 862460493 0 0) 0x7fb30c0d07e0 con 0x7fb30c15d730 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.645+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2f0003520 con 0x7fb30c15d730 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.645+0000 7fb30b400640 1 --2- 172.21.15.46:0/3167338021 >> v2:172.21.15.142:3300/0 conn(0x7fb30c15e220 0x7fb30c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2918384350 0 0) 0x7fb2f0003520 con 0x7fb30c15d730 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 >> v2:172.21.15.142:3300/0 conn(0x7fb30c15e220 msgr2=0x7fb30c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 --2- 172.21.15.46:0/3167338021 >> v2:172.21.15.142:3300/0 conn(0x7fb30c15e220 0x7fb30c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 >> v1:172.21.15.142:6789/0 conn(0x7fb30c0b53b0 legacy=0x7fb30c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c0b45f0 con 0x7fb30c15d730 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb2fc0019f0 con 0x7fb30c15d730 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 >> v1:172.21.15.46:6789/0 conn(0x7fb30c15d730 legacy=0x7fb30c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2f0003a00 0x7fb2f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 0x7fb2f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0b4310 con 0x7fb2f0005210 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f000c140 con 0x7fb2f0004800 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f000c340 con 0x7fb2f0003a00 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb30b400640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 0x7fb2f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb30a000640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.646+0000 7fb30aa00640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2f0003a00 0x7fb2f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.647+0000 7fb30b400640 1 -- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2f0003a00 msgr2=0x7fb2f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.647+0000 7fb30b400640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2f0003a00 0x7fb2f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.647+0000 7fb30b400640 1 -- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0004800 msgr2=0x7fb2f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.647+0000 7fb30b400640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.647+0000 7fb30b400640 1 -- 172.21.15.46:0/3167338021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0003520 con 0x7fb2f0005210 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.648+0000 7fb30b400640 1 -- 172.21.15.46:0/3167338021 reap_dead start 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.648+0000 7fb30b400640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 0x7fb2f000b9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb30400ba00 tx=0x7fb2f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23782dfe4046ed1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.648+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304030070 con 0x7fb2f0005210 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.648+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30402ade0 con 0x7fb2f0005210 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.648+0000 7fb309600640 1 -- 172.21.15.46:0/3167338021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30400b600 con 0x7fb2f0005210 2024-04-26T23:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 -- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 msgr2=0x7fb2f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 0x7fb2f000b9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb30400ba00 tx=0x7fb2f0005830 comp rx=0 tx=0).stop 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 -- 172.21.15.46:0/3167338021 shutdown_connections 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 --2- 172.21.15.46:0/3167338021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2f0005210 0x7fb2f000b9c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 -- 172.21.15.46:0/3167338021 >> 172.21.15.46:0/3167338021 conn(0x7fb30c06c980 msgr2=0x7fb30c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 -- 172.21.15.46:0/3167338021 shutdown_connections 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.649+0000 7fb311600640 1 -- 172.21.15.46:0/3167338021 wait complete. 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.651+0000 7fb311600640 1 Processor -- start 2024-04-26T23:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.651+0000 7fb311600640 1 -- start start 2024-04-26T23:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 0x7fb30c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c144e90 0x7fb30c145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0b4e90 con 0x7fb30c1408a0 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0b4bb0 con 0x7fb2f0005210 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb311600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c0b3ec0 con 0x7fb30c144e90 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb30a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb30a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59702/0 (socket says 172.21.15.46:59702) 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb30a000640 1 -- 172.21.15.46:0/3321352706 learned_addr learned my addr 172.21.15.46:0/3321352706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb30aa00640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 0x7fb30c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.652+0000 7fb30a000640 1 -- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c144e90 msgr2=0x7fb30c145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30a000640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c144e90 0x7fb30c145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30a000640 1 -- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 msgr2=0x7fb30c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30a000640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 0x7fb30c0c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30a000640 1 -- 172.21.15.46:0/3321352706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c145bc0 con 0x7fb30c1408a0 2024-04-26T23:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30aa00640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 0x7fb30c0c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb30a000640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400e460 tx=0x7fb2f400e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40135f0 con 0x7fb30c1408a0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f4019070 con 0x7fb30c1408a0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40139d0 con 0x7fb30c1408a0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c0c5dc0 con 0x7fb30c1408a0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.653+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c0c6380 con 0x7fb30c1408a0 2024-04-26T23:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.655+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c15d460 con 0x7fb30c1408a0 2024-04-26T23:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.655+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2f4013b70 con 0x7fb30c1408a0 2024-04-26T23:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.656+0000 7fb302a00640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 0x7fb2e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.657+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb2f408bc20 con 0x7fb30c1408a0 2024-04-26T23:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.657+0000 7fb30aa00640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 0x7fb2e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.658+0000 7fb30aa00640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 0x7fb2e806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0013d0 tx=0x7fb2fc001340 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.661+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2f4023020 con 0x7fb30c1408a0 2024-04-26T23:42:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.870+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb30c069530 con 0x7fb30c1408a0 2024-04-26T23:42:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.871+0000 7fb302a00640 1 -- 172.21.15.46:0/3321352706 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7fb2f40597d0 con 0x7fb30c1408a0 2024-04-26T23:42:42.876 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:42:42.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 msgr2=0x7fb2e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 0x7fb2e806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0013d0 tx=0x7fb2fc001340 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 msgr2=0x7fb30c0c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400e460 tx=0x7fb2f400e920 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 shutdown_connections 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2e806c8f0 0x7fb2e806edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c144e90 0x7fb30c145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c1408a0 0x7fb30c0c8ac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 --2- 172.21.15.46:0/3321352706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f0005210 0x7fb30c0c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 >> 172.21.15.46:0/3321352706 conn(0x7fb30c06c980 msgr2=0x7fb30c0af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 shutdown_connections 2024-04-26T23:42:42.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:42.874+0000 7fb311600640 1 -- 172.21.15.46:0/3321352706 wait complete. 2024-04-26T23:42:42.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:42:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.102+0000 7f18aca00640 1 Processor -- start 2024-04-26T23:42:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.102+0000 7f18aca00640 1 -- start start 2024-04-26T23:42:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.104+0000 7f18aca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 0x7f18a806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.104+0000 7f18aca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.104+0000 7f18aca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a806f200 con 0x7f18a806c890 2024-04-26T23:42:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.104+0000 7f18aca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f18a8163d00 con 0x7f18a815f1f0 2024-04-26T23:42:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.104+0000 7f18aca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18a806ccf0 con 0x7f18a8163440 2024-04-26T23:42:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.105+0000 7f18a6800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.105+0000 7f18a5e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 0x7f18a806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.105+0000 7f18a6800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:33868/0 (socket says 172.21.15.46:33868) 2024-04-26T23:42:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.105+0000 7f18a6800640 1 -- 172.21.15.46:0/4163942899 learned_addr learned my addr 172.21.15.46:0/4163942899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.106+0000 7f18a6800640 1 -- 172.21.15.46:0/4163942899 >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 msgr2=0x7f18a806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.106+0000 7f18a6800640 1 --2- 172.21.15.46:0/4163942899 >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 0x7f18a806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.106+0000 7f18a6800640 1 -- 172.21.15.46:0/4163942899 >> v1:172.21.15.142:6789/0 conn(0x7f18a806c890 legacy=0x7f18a8162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.106+0000 7f18a6800640 1 -- 172.21.15.46:0/4163942899 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a8163a20 con 0x7f18a8163440 2024-04-26T23:42:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.106+0000 7f18a5e00640 1 --2- 172.21.15.46:0/4163942899 >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 0x7f18a806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:42:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.107+0000 7f18a6800640 1 --2- 172.21.15.46:0/4163942899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd37f96e56dd6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.108+0000 7f18a4a00640 1 -- 172.21.15.46:0/4163942899 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0037070 con 0x7f18a8163440 2024-04-26T23:42:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.108+0000 7f18a4a00640 1 -- 172.21.15.46:0/4163942899 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a0032440 con 0x7f18a8163440 2024-04-26T23:42:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.108+0000 7f18a4a00640 1 -- 172.21.15.46:0/4163942899 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0032810 con 0x7f18a8163440 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.108+0000 7f18aca00640 1 -- 172.21.15.46:0/4163942899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 msgr2=0x7f18a806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.108+0000 7f18aca00640 1 --2- 172.21.15.46:0/4163942899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009870 tx=0x7f18a002a250 comp rx=0 tx=0).stop 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 -- 172.21.15.46:0/4163942899 shutdown_connections 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 --2- 172.21.15.46:0/4163942899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a8163440 0x7f18a806fca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 --2- 172.21.15.46:0/4163942899 >> v2:172.21.15.142:3300/0 conn(0x7f18a815f1f0 0x7f18a806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 -- 172.21.15.46:0/4163942899 >> 172.21.15.46:0/4163942899 conn(0x7f18a806da90 msgr2=0x7f18a80d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 -- 172.21.15.46:0/4163942899 shutdown_connections 2024-04-26T23:42:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.109+0000 7f18aca00640 1 -- 172.21.15.46:0/4163942899 wait complete. 2024-04-26T23:42:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.111+0000 7f18aca00640 1 Processor -- start 2024-04-26T23:42:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.111+0000 7f18aca00640 1 -- start start 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 0x7f18a80b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 0x7f18a80c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18a815f5d0 con 0x7f18a806c890 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a806f810 con 0x7f18a8163440 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18aca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18a80701e0 con 0x7f18a815f1f0 2024-04-26T23:42:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59718/0 (socket says 172.21.15.46:59718) 2024-04-26T23:42:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a5e00640 1 -- 172.21.15.46:0/1050027505 learned_addr learned my addr 172.21.15.46:0/1050027505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a6800640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 0x7f18a80c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a5e00640 1 -- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 msgr2=0x7f18a80b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.113+0000 7f18a5400640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 0x7f18a80b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5e00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 0x7f18a80b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5e00640 1 -- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 msgr2=0x7f18a80c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5e00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 0x7f18a80c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5e00640 1 -- 172.21.15.46:0/1050027505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a80c69d0 con 0x7f18a806c890 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a6800640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 0x7f18a80c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5400640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 0x7f18a80b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18a5e00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f189800efc0 tx=0x7f189800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898018020 con 0x7f18a806c890 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898014680 con 0x7f18a806c890 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898014a50 con 0x7f18a806c890 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a80c6cb0 con 0x7f18a806c890 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.114+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a80c7270 con 0x7f18a806c890 2024-04-26T23:42:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.116+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a8069530 con 0x7f18a806c890 2024-04-26T23:42:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.117+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1898005d10 con 0x7f18a806c890 2024-04-26T23:42:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.118+0000 7f189e000640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.118+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f189808cc30 con 0x7f18a806c890 2024-04-26T23:42:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.118+0000 7f18a5400640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.120+0000 7f18a5400640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f18a80b0c20 tx=0x7f1890009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.122+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1898022030 con 0x7f18a806c890 2024-04-26T23:42:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.317+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18a80d2910 con 0x7f18a806c890 2024-04-26T23:42:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.318+0000 7f189e000640 1 -- 172.21.15.46:0/1050027505 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f1898010040 con 0x7f18a806c890 2024-04-26T23:42:43.322 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.320+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 msgr2=0x7f188406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.320+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f18a80b0c20 tx=0x7f1890009040 comp rx=0 tx=0).stop 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.320+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 msgr2=0x7f18a814b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.320+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f189800efc0 tx=0x7f189800c490 comp rx=0 tx=0).stop 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 shutdown_connections 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a8163440 0x7f18a80c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815f1f0 0x7f18a80b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 --2- 172.21.15.46:0/1050027505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a806c890 0x7f18a814b830 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 >> 172.21.15.46:0/1050027505 conn(0x7f18a806da90 msgr2=0x7f18a80af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:43.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 shutdown_connections 2024-04-26T23:42:43.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.321+0000 7f18aca00640 1 -- 172.21.15.46:0/1050027505 wait complete. 2024-04-26T23:42:43.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.544+0000 7fdc75400640 1 Processor -- start 2024-04-26T23:42:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.544+0000 7fdc75400640 1 -- start start 2024-04-26T23:42:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc7006e510 0x7fdc7006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc70160bd0 0x7fdc70161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc7015d540 0x7fdc7015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7006ee30 con 0x7fdc7015d540 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdc70161940 con 0x7fdc70160bd0 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc75400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc70071f80 con 0x7fdc7006e510 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc6f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc7015d540 0x7fdc7015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc6f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc7015d540 0x7fdc7015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59722/0 (socket says 172.21.15.46:59722) 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc6f200640 1 -- 172.21.15.46:0/3903936976 learned_addr learned my addr 172.21.15.46:0/3903936976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc6de00640 1 --2- 172.21.15.46:0/3903936976 >> v2:172.21.15.142:3300/0 conn(0x7fdc70160bd0 0x7fdc70161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.546+0000 7fdc6e800640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc7006e510 0x7fdc7006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.547+0000 7fdc6f200640 1 -- 172.21.15.46:0/3903936976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc7006e510 msgr2=0x7fdc7006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.547+0000 7fdc6f200640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc7006e510 0x7fdc7006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.547+0000 7fdc6f200640 1 -- 172.21.15.46:0/3903936976 >> v2:172.21.15.142:3300/0 conn(0x7fdc70160bd0 msgr2=0x7fdc70161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.547+0000 7fdc6f200640 1 --2- 172.21.15.46:0/3903936976 >> v2:172.21.15.142:3300/0 conn(0x7fdc70160bd0 0x7fdc70161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.547+0000 7fdc6f200640 1 -- 172.21.15.46:0/3903936976 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc70072260 con 0x7fdc7015d540 2024-04-26T23:42:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.548+0000 7fdc6f200640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc7015d9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdc700b6270 tx=0x7fdc6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10685aaa249257d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.548+0000 7fdc6d400640 1 -- 172.21.15.46:0/3903936976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68037070 con 0x7fdc7015d540 2024-04-26T23:42:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.548+0000 7fdc6d400640 1 -- 172.21.15.46:0/3903936976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc68032440 con 0x7fdc7015d540 2024-04-26T23:42:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.548+0000 7fdc6d400640 1 -- 172.21.15.46:0/3903936976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68032810 con 0x7fdc7015d540 2024-04-26T23:42:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 -- 172.21.15.46:0/3903936976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 msgr2=0x7fdc7015d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc7015d9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdc700b6270 tx=0x7fdc6802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 -- 172.21.15.46:0/3903936976 shutdown_connections 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc7015d9d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 --2- 172.21.15.46:0/3903936976 >> v2:172.21.15.142:3300/0 conn(0x7fdc70160bd0 0x7fdc70161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 --2- 172.21.15.46:0/3903936976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc7006e510 0x7fdc7006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 -- 172.21.15.46:0/3903936976 >> 172.21.15.46:0/3903936976 conn(0x7fdc7006c530 msgr2=0x7fdc7006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 -- 172.21.15.46:0/3903936976 shutdown_connections 2024-04-26T23:42:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.549+0000 7fdc75400640 1 -- 172.21.15.46:0/3903936976 wait complete. 2024-04-26T23:42:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.551+0000 7fdc75400640 1 Processor -- start 2024-04-26T23:42:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.551+0000 7fdc75400640 1 -- start start 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 0x7fdc700af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc70160bd0 0x7fdc701588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70161570 con 0x7fdc7015d540 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70072820 con 0x7fdc7006e510 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc75400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc701616f0 con 0x7fdc70160bd0 2024-04-26T23:42:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59728/0 (socket says 172.21.15.46:59728) 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.552+0000 7fdc6de00640 1 -- 172.21.15.46:0/2294313675 learned_addr learned my addr 172.21.15.46:0/2294313675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6e800640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 0x7fdc700af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 -- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc70160bd0 msgr2=0x7fdc701588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc70160bd0 0x7fdc701588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 -- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 msgr2=0x7fdc700af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 0x7fdc700af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 -- 172.21.15.46:0/2294313675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7013cad0 con 0x7fdc7015d540 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6e800640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 0x7fdc700af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc6de00640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007ef0 tx=0x7fdc6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014620 con 0x7fdc7015d540 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.553+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc60019070 con 0x7fdc7015d540 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.554+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014a00 con 0x7fdc7015d540 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.554+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7013cd70 con 0x7fdc7015d540 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.554+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7013d330 con 0x7fdc7015d540 2024-04-26T23:42:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.555+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc70069530 con 0x7fdc7015d540 2024-04-26T23:42:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.556+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc60014ba0 con 0x7fdc7015d540 2024-04-26T23:42:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.557+0000 7fdc66a00640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 0x7fdc4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.557+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fdc6008c660 con 0x7fdc7015d540 2024-04-26T23:42:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.557+0000 7fdc6e800640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 0x7fdc4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.559+0000 7fdc6e800640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 0x7fdc4c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc58006fd0 tx=0x7fdc58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.561+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc6005a8f0 con 0x7fdc7015d540 2024-04-26T23:42:43.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.798+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc700b0200 con 0x7fdc7015d540 2024-04-26T23:42:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.800+0000 7fdc66a00640 1 -- 172.21.15.46:0/2294313675 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5182 (secure 0 0 0) 0x7fdc6005a1f0 con 0x7fdc7015d540 2024-04-26T23:42:43.804 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:43.804 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:37.177552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-04-26T23:42:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.802+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 msgr2=0x7fdc4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.802+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 0x7fdc4c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc58006fd0 tx=0x7fdc58008040 comp rx=0 tx=0).stop 2024-04-26T23:42:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.802+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 msgr2=0x7fdc701583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.802+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007ef0 tx=0x7fdc6000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 shutdown_connections 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc4c06c8f0 0x7fdc4c06edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc70160bd0 0x7fdc701588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7015d540 0x7fdc701583a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 --2- 172.21.15.46:0/2294313675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc7006e510 0x7fdc700af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 >> 172.21.15.46:0/2294313675 conn(0x7fdc7006c530 msgr2=0x7fdc7006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 shutdown_connections 2024-04-26T23:42:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:43.803+0000 7fdc75400640 1 -- 172.21.15.46:0/2294313675 wait complete. 2024-04-26T23:42:43.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.026+0000 7fad11400640 1 Processor -- start 2024-04-26T23:42:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.027+0000 7fad11400640 1 -- start start 2024-04-26T23:42:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c0cc690 0x7fad0c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad11400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad11400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad0c072bb0 con 0x7fad0c1454a0 2024-04-26T23:42:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad11400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c072e90 con 0x7fad0c0cc690 2024-04-26T23:42:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad11400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c073170 con 0x7fad0c0c6a50 2024-04-26T23:42:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c0cc690 0x7fad0c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c0cc690 0x7fad0c0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59738/0 (socket says 172.21.15.46:59738) 2024-04-26T23:42:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad0be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fad0c1454a0 0x7fad0c073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37162/0 (socket says 172.21.15.46:37162) 2024-04-26T23:42:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.029+0000 7fad03e00640 1 -- 172.21.15.46:0/627090782 learned_addr learned my addr 172.21.15.46:0/627090782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad03e00640 1 -- 172.21.15.46:0/627090782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 msgr2=0x7fad0c0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad10a00640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad03e00640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c0c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad03e00640 1 -- 172.21.15.46:0/627090782 >> v1:172.21.15.46:6789/0 conn(0x7fad0c1454a0 legacy=0x7fad0c073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad03e00640 1 -- 172.21.15.46:0/627090782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c073450 con 0x7fad0c0cc690 2024-04-26T23:42:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.030+0000 7fad10a00640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.031+0000 7fad03e00640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc690 0x7fad0c0cdc30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420d79e57d46f731 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.031+0000 7fad0b400640 1 -- 172.21.15.46:0/627090782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4037070 con 0x7fad0c0cc690 2024-04-26T23:42:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.031+0000 7fad0b400640 1 -- 172.21.15.46:0/627090782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf4032440 con 0x7fad0c0cc690 2024-04-26T23:42:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.031+0000 7fad0b400640 1 -- 172.21.15.46:0/627090782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4032810 con 0x7fad0c0cc690 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 -- 172.21.15.46:0/627090782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc690 msgr2=0x7fad0c0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc690 0x7fad0c0cdc30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf402a250 comp rx=0 tx=0).stop 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 -- 172.21.15.46:0/627090782 shutdown_connections 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 --2- 172.21.15.46:0/627090782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc690 0x7fad0c0cdc30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 -- 172.21.15.46:0/627090782 >> 172.21.15.46:0/627090782 conn(0x7fad0c0b26d0 msgr2=0x7fad0c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 -- 172.21.15.46:0/627090782 shutdown_connections 2024-04-26T23:42:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.032+0000 7fad11400640 1 -- 172.21.15.46:0/627090782 wait complete. 2024-04-26T23:42:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.034+0000 7fad11400640 1 Processor -- start 2024-04-26T23:42:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.034+0000 7fad11400640 1 -- start start 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c0cc690 0x7fad0c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c145880 con 0x7fad0c1454a0 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c072880 con 0x7fad0c0cc690 2024-04-26T23:42:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad11400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c145a00 con 0x7fad0c0c6a50 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad10a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad10a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59750/0 (socket says 172.21.15.46:59750) 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.035+0000 7fad10a00640 1 -- 172.21.15.46:0/1718353791 learned_addr learned my addr 172.21.15.46:0/1718353791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad0be00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad03e00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c0cc690 0x7fad0c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 -- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 msgr2=0x7fad0c13c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c13c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 -- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c0cc690 msgr2=0x7fad0c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c0cc690 0x7fad0c0b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 -- 172.21.15.46:0/1718353791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c15cc30 con 0x7fad0c1454a0 2024-04-26T23:42:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.036+0000 7fad10a00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fad0400cef0 tx=0x7fad0400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.037+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0401a070 con 0x7fad0c1454a0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.037+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0400d860 con 0x7fad0c1454a0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.037+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400dc50 con 0x7fad0c1454a0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.037+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c15cf10 con 0x7fad0c1454a0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.037+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c15d4d0 con 0x7fad0c1454a0 2024-04-26T23:42:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.038+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c0693b0 con 0x7fad0c1454a0 2024-04-26T23:42:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.039+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad0401d070 con 0x7fad0c1454a0 2024-04-26T23:42:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.040+0000 7fad08c00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 0x7face806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.040+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fad0408c5c0 con 0x7fad0c1454a0 2024-04-26T23:42:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.040+0000 7fad0be00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 0x7face806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.042+0000 7fad0be00640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 0x7face806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fad0c144e90 tx=0x7facfc007ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.044+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad0405a910 con 0x7fad0c1454a0 2024-04-26T23:42:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.261+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad0c0cefa0 con 0x7fad0c1454a0 2024-04-26T23:42:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.264+0000 7fad08c00640 1 -- 172.21.15.46:0/1718353791 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16906 (secure 0 0 0) 0x7fad0405a210 con 0x7fad0c1454a0 2024-04-26T23:42:44.269 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:44.269 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:58.143214+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 msgr2=0x7face806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 0x7face806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fad0c144e90 tx=0x7facfc007ce0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 msgr2=0x7fad0c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fad0400cef0 tx=0x7fad0400a710 comp rx=0 tx=0).stop 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 shutdown_connections 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7face806c8f0 0x7face806edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c1454a0 0x7fad0c0b3d00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c0cc690 0x7fad0c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.267+0000 7fad11400640 1 --2- 172.21.15.46:0/1718353791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c0c6a50 0x7fad0c13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.268+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 >> 172.21.15.46:0/1718353791 conn(0x7fad0c0b26d0 msgr2=0x7fad0c0aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.268+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 shutdown_connections 2024-04-26T23:42:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.268+0000 7fad11400640 1 -- 172.21.15.46:0/1718353791 wait complete. 2024-04-26T23:42:44.291 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:42:44.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.498+0000 7fe207200640 1 Processor -- start 2024-04-26T23:42:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.498+0000 7fe207200640 1 -- start start 2024-04-26T23:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2081454a0 0x7fe2080cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe208073970 0x7fe2080cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2080c6a50 0x7fe2080c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe208072bb0 con 0x7fe208073970 2024-04-26T23:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe208072e90 con 0x7fe2081454a0 2024-04-26T23:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe207200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe208073170 con 0x7fe2080c6a50 2024-04-26T23:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe205400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe208073970 0x7fe2080cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe205400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe208073970 0x7fe2080cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59762/0 (socket says 172.21.15.46:59762) 2024-04-26T23:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.500+0000 7fe205400640 1 -- 172.21.15.46:0/2004086912 learned_addr learned my addr 172.21.15.46:0/2004086912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205400640 1 -- 172.21.15.46:0/2004086912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2080c6a50 msgr2=0x7fe2080c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205400640 1 --2- 172.21.15.46:0/2004086912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2080c6a50 0x7fe2080c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205400640 1 -- 172.21.15.46:0/2004086912 >> v2:172.21.15.142:3300/0 conn(0x7fe2081454a0 msgr2=0x7fe2080cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205e00640 1 --2- 172.21.15.46:0/2004086912 >> v2:172.21.15.142:3300/0 conn(0x7fe2081454a0 0x7fe2080cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205400640 1 --2- 172.21.15.46:0/2004086912 >> v2:172.21.15.142:3300/0 conn(0x7fe2081454a0 0x7fe2080cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.501+0000 7fe205400640 1 -- 172.21.15.46:0/2004086912 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe208073450 con 0x7fe208073970 2024-04-26T23:42:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.502+0000 7fe205400640 1 --2- 172.21.15.46:0/2004086912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe208073970 0x7fe2080cdc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2080d1ba0 tx=0x7fe1f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87838a237bf056b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.502+0000 7fe204a00640 1 -- 172.21.15.46:0/2004086912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8037070 con 0x7fe208073970 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe204a00640 1 -- 172.21.15.46:0/2004086912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f8032440 con 0x7fe208073970 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe204a00640 1 -- 172.21.15.46:0/2004086912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8032810 con 0x7fe208073970 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 -- 172.21.15.46:0/2004086912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe208073970 msgr2=0x7fe2080cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 --2- 172.21.15.46:0/2004086912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe208073970 0x7fe2080cdc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2080d1ba0 tx=0x7fe1f802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 -- 172.21.15.46:0/2004086912 shutdown_connections 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 --2- 172.21.15.46:0/2004086912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2080c6a50 0x7fe2080c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 --2- 172.21.15.46:0/2004086912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe208073970 0x7fe2080cdc30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 --2- 172.21.15.46:0/2004086912 >> v2:172.21.15.142:3300/0 conn(0x7fe2081454a0 0x7fe2080cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 -- 172.21.15.46:0/2004086912 >> 172.21.15.46:0/2004086912 conn(0x7fe2080b26d0 msgr2=0x7fe2080b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 -- 172.21.15.46:0/2004086912 shutdown_connections 2024-04-26T23:42:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.503+0000 7fe207200640 1 -- 172.21.15.46:0/2004086912 wait complete. 2024-04-26T23:42:44.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.505+0000 7fe207200640 1 Processor -- start 2024-04-26T23:42:44.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.505+0000 7fe207200640 1 -- start start 2024-04-26T23:42:44.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe208073970 0x7fe2080b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 0x7fe20815fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe208073e00 con 0x7fe2080c6a50 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe208072880 con 0x7fe2081454a0 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.506+0000 7fe207200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe208145880 con 0x7fe208073970 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59776/0 (socket says 172.21.15.46:59776) 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 -- 172.21.15.46:0/2775742845 learned_addr learned my addr 172.21.15.46:0/2775742845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205e00640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe208073970 0x7fe2080b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe206800640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 0x7fe20815fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 -- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe208073970 msgr2=0x7fe2080b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe208073970 0x7fe2080b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 -- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 msgr2=0x7fe20815fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 0x7fe20815fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe205400640 1 -- 172.21.15.46:0/2775742845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe208070190 con 0x7fe2080c6a50 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.507+0000 7fe206800640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 0x7fe20815fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe205400640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8000c00 tx=0x7fe1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8041020 con 0x7fe2080c6a50 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f8030820 con 0x7fe2080c6a50 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8037040 con 0x7fe2080c6a50 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20806f580 con 0x7fe2080c6a50 2024-04-26T23:42:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.508+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20806fb40 con 0x7fe2080c6a50 2024-04-26T23:42:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.509+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2080693b0 con 0x7fe2080c6a50 2024-04-26T23:42:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.510+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1f8044020 con 0x7fe2080c6a50 2024-04-26T23:42:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.511+0000 7fe1fe000640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 0x7fe1e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.511+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe1f80ac5a0 con 0x7fe2080c6a50 2024-04-26T23:42:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.512+0000 7fe205e00640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 0x7fe1e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.513+0000 7fe205e00640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 0x7fe1e406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b7c0 tx=0x7fe1f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.516+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1f804b030 con 0x7fe2080c6a50 2024-04-26T23:42:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.761+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2080c6730 con 0x7fe2080c6a50 2024-04-26T23:42:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.763+0000 7fe1fe000640 1 -- 172.21.15.46:0/2775742845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5182 (secure 0 0 0) 0x7fe1f807a830 con 0x7fe2080c6a50 2024-04-26T23:42:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:44.767 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:37.177552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 msgr2=0x7fe1e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 0x7fe1e406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b7c0 tx=0x7fe1f0008040 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 msgr2=0x7fe20815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8000c00 tx=0x7fe1f80046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 shutdown_connections 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe1e406c920 0x7fe1e406ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2081454a0 0x7fe20815fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080c6a50 0x7fe20815f8f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 --2- 172.21.15.46:0/2775742845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe208073970 0x7fe2080b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.765+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 >> 172.21.15.46:0/2775742845 conn(0x7fe2080b26d0 msgr2=0x7fe2080c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.766+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 shutdown_connections 2024-04-26T23:42:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.766+0000 7fe207200640 1 -- 172.21.15.46:0/2775742845 wait complete. 2024-04-26T23:42:44.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.983+0000 7f40cfe00640 1 Processor -- start 2024-04-26T23:42:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.983+0000 7f40cfe00640 1 -- start start 2024-04-26T23:42:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40cfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40d015f740 0x7f40d0073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015fcb0 0x7f40d00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40cfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d00cbcb0 con 0x7f40d00c6d50 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40cfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f40d00cba60 con 0x7f40d015f740 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40d00cbf90 con 0x7f40d015fcb0 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40d015f740 0x7f40d0073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40d015f740 0x7f40d0073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59782/0 (socket says 172.21.15.46:59782) 2024-04-26T23:42:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.984+0000 7f40c7e00640 1 -- 172.21.15.46:0/4209784785 learned_addr learned my addr 172.21.15.46:0/4209784785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.985+0000 7f40c7e00640 1 -- 172.21.15.46:0/4209784785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015fcb0 msgr2=0x7f40d00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.985+0000 7f40c7e00640 1 --2- 172.21.15.46:0/4209784785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015fcb0 0x7f40d00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.985+0000 7f40c7e00640 1 -- 172.21.15.46:0/4209784785 >> v1:172.21.15.142:6789/0 conn(0x7f40d00c6d50 legacy=0x7f40d015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.985+0000 7f40c7e00640 1 -- 172.21.15.46:0/4209784785 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d0072cd0 con 0x7f40d015f740 2024-04-26T23:42:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.986+0000 7f40c7e00640 1 --2- 172.21.15.46:0/4209784785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d015f740 0x7f40d0073be0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62eaff34d8875062 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.986+0000 7f40ce000640 1 -- 172.21.15.46:0/4209784785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8037070 con 0x7f40d015f740 2024-04-26T23:42:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.986+0000 7f40ce000640 1 -- 172.21.15.46:0/4209784785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b8032440 con 0x7f40d015f740 2024-04-26T23:42:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.986+0000 7f40ce000640 1 -- 172.21.15.46:0/4209784785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8032810 con 0x7f40d015f740 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 -- 172.21.15.46:0/4209784785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d015f740 msgr2=0x7f40d0073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 --2- 172.21.15.46:0/4209784785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d015f740 0x7f40d0073be0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802a180 comp rx=0 tx=0).stop 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 -- 172.21.15.46:0/4209784785 shutdown_connections 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 --2- 172.21.15.46:0/4209784785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015fcb0 0x7f40d00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 --2- 172.21.15.46:0/4209784785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d015f740 0x7f40d0073be0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 -- 172.21.15.46:0/4209784785 >> 172.21.15.46:0/4209784785 conn(0x7f40d00b26d0 msgr2=0x7f40d00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 -- 172.21.15.46:0/4209784785 shutdown_connections 2024-04-26T23:42:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.987+0000 7f40cfe00640 1 -- 172.21.15.46:0/4209784785 wait complete. 2024-04-26T23:42:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.989+0000 7f40cfe00640 1 Processor -- start 2024-04-26T23:42:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.989+0000 7f40cfe00640 1 -- start start 2024-04-26T23:42:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015f740 0x7f40d0147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d015fcb0 0x7f40d015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40d00c72b0 con 0x7f40d00c6d50 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40d00c7130 con 0x7f40d015fcb0 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40d00cda60 con 0x7f40d015f740 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59796/0 (socket says 172.21.15.46:59796) 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 -- 172.21.15.46:0/1186348195 learned_addr learned my addr 172.21.15.46:0/1186348195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40c7e00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015f740 0x7f40d0147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 -- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015f740 msgr2=0x7f40d0147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cf400640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d015fcb0 0x7f40d015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015f740 0x7f40d0147170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 -- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d015fcb0 msgr2=0x7f40d015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d015fcb0 0x7f40d015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.990+0000 7f40cea00640 1 -- 172.21.15.46:0/1186348195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d01476b0 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40cea00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f40c000b410 tx=0x7f40c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0015020 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c0011830 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0011c20 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d00b3ca0 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.991+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d0149e60 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.992+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d00693b0 con 0x7f40d00c6d50 2024-04-26T23:42:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.993+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40c0018020 con 0x7f40d00c6d50 2024-04-26T23:42:44.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.994+0000 7f40c6a00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 0x7f40ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.995+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f40c008bc10 con 0x7f40d00c6d50 2024-04-26T23:42:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.995+0000 7f40c7e00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 0x7f40ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.996+0000 7f40c7e00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 0x7f40ac06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f40b8034000 tx=0x7f40b802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:44.999+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40c001f020 con 0x7f40d00c6d50 2024-04-26T23:42:45.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.250+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40d00ce740 con 0x7f40d00c6d50 2024-04-26T23:42:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.251+0000 7f40c6a00640 1 -- 172.21.15.46:0/1186348195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5182 (secure 0 0 0) 0x7f40c00172c0 con 0x7f40d00c6d50 2024-04-26T23:42:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:45.256 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:41:56.611187+0000","modified":"2024-04-26T23:42:37.177552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5793},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5793":{"gid":5793,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6834/970201181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":970201181},{"type":"v1","addr":"172.21.15.46:6834","nonce":970201181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5793,"qdb_cluster":[5793]},"id":3}]} 2024-04-26T23:42:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 msgr2=0x7f40ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 0x7f40ac06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f40b8034000 tx=0x7f40b802e000 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 msgr2=0x7f40d01f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f40c000b410 tx=0x7f40c000b8d0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 shutdown_connections 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f40ac06c8f0 0x7f40ac06edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d015fcb0 0x7f40d015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d015f740 0x7f40d0147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 --2- 172.21.15.46:0/1186348195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40d00c6d50 0x7f40d01f8fd0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 >> 172.21.15.46:0/1186348195 conn(0x7f40d00b26d0 msgr2=0x7f40d00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.254+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 shutdown_connections 2024-04-26T23:42:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.255+0000 7f40cfe00640 1 -- 172.21.15.46:0/1186348195 wait complete. 2024-04-26T23:42:45.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.477+0000 7f4619800640 1 Processor -- start 2024-04-26T23:42:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.478+0000 7f4619800640 1 -- start start 2024-04-26T23:42:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4619800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f461415f4e0 0x7f46140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4619800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f46140d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4619800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140d11f0 con 0x7f46140d1f60 2024-04-26T23:42:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4619800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f46140d2990 con 0x7f461415f4e0 2024-04-26T23:42:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4619800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46140d5f20 con 0x7f46140b4000 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4613e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f461415f4e0 0x7f46140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4613e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f461415f4e0 0x7f46140d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59812/0 (socket says 172.21.15.46:59812) 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4618e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f46140d1f60 0x7f46140cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37174/0 (socket says 172.21.15.46:37174) 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.479+0000 7f4613e00640 1 -- 172.21.15.46:0/603590339 learned_addr learned my addr 172.21.15.46:0/603590339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f4613e00640 1 -- 172.21.15.46:0/603590339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 msgr2=0x7f46140d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f460b400640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f46140d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f4613e00640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f46140d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f4613e00640 1 -- 172.21.15.46:0/603590339 >> v1:172.21.15.46:6789/0 conn(0x7f46140d1f60 legacy=0x7f46140cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f4613400640 1 -- 172.21.15.46:0/603590339 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770191861 0 0) 0x7f46140d11f0 con 0x7f46140d1f60 2024-04-26T23:42:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.480+0000 7f4613e00640 1 -- 172.21.15.46:0/603590339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140d0f10 con 0x7f461415f4e0 2024-04-26T23:42:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.481+0000 7f4613e00640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140d2df0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4604009e50 tx=0x7f460402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743f477f7be2a8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.481+0000 7f4613400640 1 -- 172.21.15.46:0/603590339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604036070 con 0x7f461415f4e0 2024-04-26T23:42:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.481+0000 7f4613400640 1 -- 172.21.15.46:0/603590339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604031600 con 0x7f461415f4e0 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.481+0000 7f4613400640 1 -- 172.21.15.46:0/603590339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040319d0 con 0x7f461415f4e0 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 -- 172.21.15.46:0/603590339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 msgr2=0x7f46140d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140d2df0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4604009e50 tx=0x7f460402a710 comp rx=0 tx=0).stop 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 -- 172.21.15.46:0/603590339 shutdown_connections 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f46140d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 --2- 172.21.15.46:0/603590339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140d2df0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 -- 172.21.15.46:0/603590339 >> 172.21.15.46:0/603590339 conn(0x7f46140d5010 msgr2=0x7f46140d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 -- 172.21.15.46:0/603590339 shutdown_connections 2024-04-26T23:42:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.482+0000 7f4619800640 1 -- 172.21.15.46:0/603590339 wait complete. 2024-04-26T23:42:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.484+0000 7f4619800640 1 Processor -- start 2024-04-26T23:42:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.484+0000 7f4619800640 1 -- start start 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f461415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1f60 0x7f4614158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f461415f8c0 con 0x7f461415f4e0 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b1d60 con 0x7f46140d1f60 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4619800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46140b3290 con 0x7f46140b4000 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59826/0 (socket says 172.21.15.46:59826) 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 -- 172.21.15.46:0/2383776275 learned_addr learned my addr 172.21.15.46:0/2383776275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4613e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f461415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 -- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 msgr2=0x7f461415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f461415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.485+0000 7f4618e00640 1 -- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1f60 msgr2=0x7f4614158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4618e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1f60 0x7f4614158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4618e00640 1 -- 172.21.15.46:0/2383776275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140b97c0 con 0x7f461415f4e0 2024-04-26T23:42:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4618e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c450 tx=0x7f460c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c018070 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c007500 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c004a20 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46140b9aa0 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.486+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4614159140 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.487+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46140693b0 con 0x7f461415f4e0 2024-04-26T23:42:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.488+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f460c01b070 con 0x7f461415f4e0 2024-04-26T23:42:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.489+0000 7f4610c00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.489+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f460c08aed0 con 0x7f461415f4e0 2024-04-26T23:42:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.490+0000 7f4613e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.491+0000 7f4613e00640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4604033000 tx=0x7f4604009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.493+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f460c059200 con 0x7f461415f4e0 2024-04-26T23:42:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.692+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46140cb970 con 0x7f461415f4e0 2024-04-26T23:42:45.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.694+0000 7f4610c00640 1 -- 172.21.15.46:0/2383776275 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16906 (secure 0 0 0) 0x7f460c058b00 con 0x7f461415f4e0 2024-04-26T23:42:45.699 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:45.699 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:41:58.143214+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:41:54.469953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T23:41:55.605157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 msgr2=0x7f45f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4604033000 tx=0x7f4604009000 comp rx=0 tx=0).stop 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 msgr2=0x7f46140b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c450 tx=0x7f460c00c910 comp rx=0 tx=0).stop 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 shutdown_connections 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f461415f4e0 0x7f46140b63c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1f60 0x7f4614158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 --2- 172.21.15.46:0/2383776275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140b4000 0x7f461415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.697+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 >> 172.21.15.46:0/2383776275 conn(0x7f46140d5010 msgr2=0x7f46141611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.698+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 shutdown_connections 2024-04-26T23:42:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.698+0000 7f4619800640 1 -- 172.21.15.46:0/2383776275 wait complete. 2024-04-26T23:42:45.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:42:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.920+0000 7f92b3e00640 1 Processor -- start 2024-04-26T23:42:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.920+0000 7f92b3e00640 1 -- start start 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.921+0000 7f92b3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b41454a0 0x7f92b40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.921+0000 7f92b3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 0x7f92b40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.921+0000 7f92b3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4072bb0 con 0x7f92b4073970 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.921+0000 7f92b3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f92b40c73e0 con 0x7f92b41454a0 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.921+0000 7f92b3e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f92b4072e90 con 0x7f92b40c6a50 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b41454a0 0x7f92b40cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b41454a0 0x7f92b40cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59838/0 (socket says 172.21.15.46:59838) 2024-04-26T23:42:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 -- 172.21.15.46:0/3483489977 learned_addr learned my addr 172.21.15.46:0/3483489977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 -- 172.21.15.46:0/3483489977 >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 msgr2=0x7f92b40c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b3400640 1 --2- 172.21.15.46:0/3483489977 >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 0x7f92b40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 --2- 172.21.15.46:0/3483489977 >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 0x7f92b40c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 -- 172.21.15.46:0/3483489977 >> v1:172.21.15.142:6789/0 conn(0x7f92b4073970 legacy=0x7f92b40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b2a00640 1 -- 172.21.15.46:0/3483489977 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4073170 con 0x7f92b41454a0 2024-04-26T23:42:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.922+0000 7f92b3400640 1 --2- 172.21.15.46:0/3483489977 >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 0x7f92b40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.923+0000 7f92b2a00640 1 --2- 172.21.15.46:0/3483489977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b41454a0 0x7f92b40cc690 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f92b415f760 tx=0x7f92a402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf3989a85aad4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b1600640 1 -- 172.21.15.46:0/3483489977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4036070 con 0x7f92b41454a0 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b1600640 1 -- 172.21.15.46:0/3483489977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a4031600 con 0x7f92b41454a0 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b1600640 1 -- 172.21.15.46:0/3483489977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40319d0 con 0x7f92b41454a0 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 -- 172.21.15.46:0/3483489977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b41454a0 msgr2=0x7f92b40cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 --2- 172.21.15.46:0/3483489977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b41454a0 0x7f92b40cc690 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f92b415f760 tx=0x7f92a402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 -- 172.21.15.46:0/3483489977 shutdown_connections 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 --2- 172.21.15.46:0/3483489977 >> v2:172.21.15.142:3300/0 conn(0x7f92b40c6a50 0x7f92b40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 --2- 172.21.15.46:0/3483489977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b41454a0 0x7f92b40cc690 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.924+0000 7f92b3e00640 1 -- 172.21.15.46:0/3483489977 >> 172.21.15.46:0/3483489977 conn(0x7f92b40b26d0 msgr2=0x7f92b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.925+0000 7f92b3e00640 1 -- 172.21.15.46:0/3483489977 shutdown_connections 2024-04-26T23:42:45.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.925+0000 7f92b3e00640 1 -- 172.21.15.46:0/3483489977 wait complete. 2024-04-26T23:42:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.926+0000 7f92b3e00640 1 Processor -- start 2024-04-26T23:42:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.926+0000 7f92b3e00640 1 -- start start 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 0x7f92b40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92b41454a0 0x7f92b40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4073e00 con 0x7f92b4073970 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4072880 con 0x7f92b40c6a50 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.927+0000 7f92b3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92b4145880 con 0x7f92b41454a0 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59846/0 (socket says 172.21.15.46:59846) 2024-04-26T23:42:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 -- 172.21.15.46:0/2983738079 learned_addr learned my addr 172.21.15.46:0/2983738079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2000640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 0x7f92b40b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 -- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92b41454a0 msgr2=0x7f92b40b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b3400640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92b41454a0 0x7f92b40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92b41454a0 0x7f92b40b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 -- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 msgr2=0x7f92b40b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 0x7f92b40b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 -- 172.21.15.46:0/2983738079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4139800 con 0x7f92b4073970 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2000640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 0x7f92b40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.928+0000 7f92b2a00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009f40 tx=0x7f92a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.929+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4036050 con 0x7f92b4073970 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.929+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a4042070 con 0x7f92b4073970 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.929+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b40b5780 con 0x7f92b4073970 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.929+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4009510 con 0x7f92b4073970 2024-04-26T23:42:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.929+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b4139ae0 con 0x7f92b4073970 2024-04-26T23:42:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.930+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b40693b0 con 0x7f92b4073970 2024-04-26T23:42:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.931+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92a4007c50 con 0x7f92b4073970 2024-04-26T23:42:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.932+0000 7f92aaa00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 0x7f929006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.932+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f92a40acd80 con 0x7f92b4073970 2024-04-26T23:42:45.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.933+0000 7f92b2000640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 0x7f929006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:45.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.934+0000 7f92b2000640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 0x7f929006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f92b4135200 tx=0x7f929c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:45.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:45.937+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92a407af80 con 0x7f92b4073970 2024-04-26T23:42:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.182+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f92b40b5550 con 0x7f92b4073970 2024-04-26T23:42:46.187 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:42:46.182+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:42:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.186+0000 7f92aaa00640 1 -- 172.21.15.46:0/2983738079 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f92a407a880 con 0x7f92b4073970 2024-04-26T23:42:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:42:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 msgr2=0x7f929006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 0x7f929006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f92b4135200 tx=0x7f929c009040 comp rx=0 tx=0).stop 2024-04-26T23:42:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 msgr2=0x7f92b40b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009f40 tx=0x7f92a4004a40 comp rx=0 tx=0).stop 2024-04-26T23:42:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 shutdown_connections 2024-04-26T23:42:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f929006c8f0 0x7f929006edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92b41454a0 0x7f92b40b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.189+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b40c6a50 0x7f92b40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.190+0000 7f92b3e00640 1 --2- 172.21.15.46:0/2983738079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4073970 0x7f92b40b6260 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.190+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 >> 172.21.15.46:0/2983738079 conn(0x7f92b40b26d0 msgr2=0x7f92b415f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.190+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 shutdown_connections 2024-04-26T23:42:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.190+0000 7f92b3e00640 1 -- 172.21.15.46:0/2983738079 wait complete. 2024-04-26T23:42:46.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:42:46.295 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:42:46.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.392+0000 7f785d800640 1 Processor -- start 2024-04-26T23:42:46.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.392+0000 7f785d800640 1 -- start start 2024-04-26T23:42:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.393+0000 7f785d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858145750 0x7f785806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.393+0000 7f785d800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f785806f400 0x7f78580b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.393+0000 7f785d800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78580ce0c0 con 0x7f78580b5120 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.393+0000 7f785d800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f785806f860 con 0x7f7858145750 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.393+0000 7f785d800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f78580b5940 con 0x7f785806f400 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858145750 0x7f785806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f785ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78580b5120 0x7f78580d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37202/0 (socket says 172.21.15.46:37202) 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858145750 0x7f785806eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59878/0 (socket says 172.21.15.46:59878) 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f785ce00640 1 -- 172.21.15.46:0/1987815982 learned_addr learned my addr 172.21.15.46:0/1987815982 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857400640 1 --2- 172.21.15.46:0/1987815982 >> v2:172.21.15.142:3300/0 conn(0x7f785806f400 0x7f78580b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 -- 172.21.15.46:0/1987815982 >> v2:172.21.15.142:3300/0 conn(0x7f785806f400 msgr2=0x7f78580b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 --2- 172.21.15.46:0/1987815982 >> v2:172.21.15.142:3300/0 conn(0x7f785806f400 0x7f78580b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 -- 172.21.15.46:0/1987815982 >> v1:172.21.15.46:6789/0 conn(0x7f78580b5120 legacy=0x7f78580d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7857e00640 1 -- 172.21.15.46:0/1987815982 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78580ce330 con 0x7f7858145750 2024-04-26T23:42:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.394+0000 7f7856a00640 1 -- 172.21.15.46:0/1987815982 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3788794664 0 0) 0x7f78580ce0c0 con 0x7f78580b5120 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.395+0000 7f7857e00640 1 --2- 172.21.15.46:0/1987815982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785806eec0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f78580b4210 tx=0x7f784802c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9e307544868fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f7856a00640 1 -- 172.21.15.46:0/1987815982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848032770 con 0x7f7858145750 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f7856a00640 1 -- 172.21.15.46:0/1987815982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848038070 con 0x7f7858145750 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f7856a00640 1 -- 172.21.15.46:0/1987815982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848032b30 con 0x7f7858145750 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 -- 172.21.15.46:0/1987815982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 msgr2=0x7f785806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 --2- 172.21.15.46:0/1987815982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785806eec0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f78580b4210 tx=0x7f784802c870 comp rx=0 tx=0).stop 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 -- 172.21.15.46:0/1987815982 shutdown_connections 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 --2- 172.21.15.46:0/1987815982 >> v2:172.21.15.142:3300/0 conn(0x7f785806f400 0x7f78580b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 --2- 172.21.15.46:0/1987815982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785806eec0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 -- 172.21.15.46:0/1987815982 >> 172.21.15.46:0/1987815982 conn(0x7f785806c350 msgr2=0x7f785806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 -- 172.21.15.46:0/1987815982 shutdown_connections 2024-04-26T23:42:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.396+0000 7f785d800640 1 -- 172.21.15.46:0/1987815982 wait complete. 2024-04-26T23:42:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.398+0000 7f785d800640 1 Processor -- start 2024-04-26T23:42:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.398+0000 7f785d800640 1 -- start start 2024-04-26T23:42:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785806f400 0x7f785815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78580b5120 0x7f785813d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78580b5590 con 0x7f7858145750 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78580b5c20 con 0x7f785806f400 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785d800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7858142e00 con 0x7f78580b5120 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59882/0 (socket says 172.21.15.46:59882) 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.399+0000 7f785ce00640 1 -- 172.21.15.46:0/2597309618 learned_addr learned my addr 172.21.15.46:0/2597309618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f7857e00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785806f400 0x7f785815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 -- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78580b5120 msgr2=0x7f785813d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78580b5120 0x7f785813d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 -- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785806f400 msgr2=0x7f785815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785806f400 0x7f785815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 -- 172.21.15.46:0/2597309618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581430e0 con 0x7f7858145750 2024-04-26T23:42:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785ce00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f785000c7a0 tx=0x7f785000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850018070 con 0x7f7858145750 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785000a640 con 0x7f7858145750 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000aab0 con 0x7f7858145750 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581433c0 con 0x7f7858145750 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.400+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7858143980 con 0x7f7858145750 2024-04-26T23:42:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.401+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580b6cc0 con 0x7f7858145750 2024-04-26T23:42:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.402+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f785000ac50 con 0x7f7858145750 2024-04-26T23:42:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.404+0000 7f784fe00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 0x7f783406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.404+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f785008b460 con 0x7f7858145750 2024-04-26T23:42:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.404+0000 7f7857e00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 0x7f783406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.406+0000 7f7857e00640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 0x7f783406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7848037000 tx=0x7f7848032000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.408+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78580b6cc0 con 0x7f7858145750 2024-04-26T23:42:46.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:46.654+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f78580693b0 con 0x7f7858145750 2024-04-26T23:42:47.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.189+0000 7f784fe00640 1 -- 172.21.15.46:0/2597309618 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f7850059610 con 0x7f7858145750 2024-04-26T23:42:47.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 msgr2=0x7f783406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 0x7f783406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7848037000 tx=0x7f7848032000 comp rx=0 tx=0).stop 2024-04-26T23:42:47.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 msgr2=0x7f785813df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f785000c7a0 tx=0x7f785000cc60 comp rx=0 tx=0).stop 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 shutdown_connections 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f783406c8f0 0x7f783406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858145750 0x7f785813df20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78580b5120 0x7f785813d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 --2- 172.21.15.46:0/2597309618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785806f400 0x7f785815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.193+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 >> 172.21.15.46:0/2597309618 conn(0x7f785806c350 msgr2=0x7f785806e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:47.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.194+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 shutdown_connections 2024-04-26T23:42:47.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.194+0000 7f785d800640 1 -- 172.21.15.46:0/2597309618 wait complete. 2024-04-26T23:42:47.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:42:47.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.434+0000 7f60aea00640 1 Processor -- start 2024-04-26T23:42:47.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.435+0000 7f60aea00640 1 -- start start 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a81469c0 0x7f60a815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60a80b35d0 0x7f60a80727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a8146da0 con 0x7f60a80cf560 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f60a80d0ac0 con 0x7f60a80b35d0 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60a80cfeb0 con 0x7f60a81469c0 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60aca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60a80b35d0 0x7f60a80727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60a7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60a80cf560 0x7f60a80b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37216/0 (socket says 172.21.15.46:37216) 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.436+0000 7f60a7400640 1 -- 172.21.15.46:0/4275201020 learned_addr learned my addr 172.21.15.46:0/4275201020 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60a7e00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a81469c0 0x7f60a815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60aca00640 1 -- 172.21.15.46:0/4275201020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a81469c0 msgr2=0x7f60a815d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60aca00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a81469c0 0x7f60a815d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60aca00640 1 -- 172.21.15.46:0/4275201020 >> v1:172.21.15.46:6789/0 conn(0x7f60a80cf560 legacy=0x7f60a80b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60aca00640 1 -- 172.21.15.46:0/4275201020 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a80d0190 con 0x7f60a80b35d0 2024-04-26T23:42:47.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.437+0000 7f60a6a00640 1 -- 172.21.15.46:0/4275201020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417061700 0 0) 0x7f60a8146da0 con 0x7f60a80cf560 2024-04-26T23:42:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.438+0000 7f60aca00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a80727d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097a0 tx=0x7f60a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2954a29841d70de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.438+0000 7f60a6a00640 1 -- 172.21.15.46:0/4275201020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0037070 con 0x7f60a80b35d0 2024-04-26T23:42:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.438+0000 7f60a6a00640 1 -- 172.21.15.46:0/4275201020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a0032440 con 0x7f60a80b35d0 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.438+0000 7f60a6a00640 1 -- 172.21.15.46:0/4275201020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0032810 con 0x7f60a80b35d0 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 -- 172.21.15.46:0/4275201020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 msgr2=0x7f60a80727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a80727d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097a0 tx=0x7f60a002a180 comp rx=0 tx=0).stop 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 -- 172.21.15.46:0/4275201020 shutdown_connections 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a80727d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 --2- 172.21.15.46:0/4275201020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a81469c0 0x7f60a815d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 -- 172.21.15.46:0/4275201020 >> 172.21.15.46:0/4275201020 conn(0x7f60a806c770 msgr2=0x7f60a806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 -- 172.21.15.46:0/4275201020 shutdown_connections 2024-04-26T23:42:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.439+0000 7f60aea00640 1 -- 172.21.15.46:0/4275201020 wait complete. 2024-04-26T23:42:47.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.441+0000 7f60aea00640 1 Processor -- start 2024-04-26T23:42:47.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.441+0000 7f60aea00640 1 -- start start 2024-04-26T23:42:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 0x7f60a813c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 0x7f60a813f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80cf9e0 con 0x7f60a80b35d0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80d0750 con 0x7f60a81469c0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.442+0000 7f60aea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60a80cfb60 con 0x7f60a80cf560 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59892/0 (socket says 172.21.15.46:59892) 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 -- 172.21.15.46:0/3933234115 learned_addr learned my addr 172.21.15.46:0/3933234115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7400640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 0x7f60a813c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 -- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 msgr2=0x7f60a813c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60aca00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 0x7f60a813f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 0x7f60a813c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 -- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 msgr2=0x7f60a813f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 0x7f60a813f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7e00640 1 -- 172.21.15.46:0/3933234115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a813a0b0 con 0x7f60a80b35d0 2024-04-26T23:42:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60aca00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 0x7f60a813f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.443+0000 7f60a7400640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 0x7f60a813c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f60a7e00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f609000d5f0 tx=0x7f609000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900125f0 con 0x7f60a80b35d0 2024-04-26T23:42:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6090019070 con 0x7f60a80b35d0 2024-04-26T23:42:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900129d0 con 0x7f60a80b35d0 2024-04-26T23:42:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a813a390 con 0x7f60a80b35d0 2024-04-26T23:42:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.444+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a813a950 con 0x7f60a80b35d0 2024-04-26T23:42:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.445+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8069530 con 0x7f60a80b35d0 2024-04-26T23:42:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.447+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6090012b70 con 0x7f60a80b35d0 2024-04-26T23:42:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.448+0000 7f609fe00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 0x7f608406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.449+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f609008c190 con 0x7f60a80b35d0 2024-04-26T23:42:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.449+0000 7f60a7400640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 0x7f608406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.451+0000 7f60a7400640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 0x7f608406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f60a80b1420 tx=0x7f6098009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.452+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f609005a340 con 0x7f60a80b35d0 2024-04-26T23:42:47.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:47.669+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60a8073bd0 con 0x7f60a80b35d0 2024-04-26T23:42:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.469+0000 7f609fe00640 1 -- 172.21.15.46:0/3933234115 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) ==== 181+0+0 (secure 0 0 0) 0x7f6090059c40 con 0x7f60a80b35d0 2024-04-26T23:42:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:42:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 msgr2=0x7f608406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 0x7f608406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f60a80b1420 tx=0x7f6098009040 comp rx=0 tx=0).stop 2024-04-26T23:42:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 msgr2=0x7f60a815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f609000d5f0 tx=0x7f609000dab0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 shutdown_connections 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f608406c8f0 0x7f608406edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a81469c0 0x7f60a813f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60a80cf560 0x7f60a813c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 --2- 172.21.15.46:0/3933234115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a80b35d0 0x7f60a815ad70 secure :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f609000d5f0 tx=0x7f609000dab0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.473+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 >> 172.21.15.46:0/3933234115 conn(0x7f60a806c770 msgr2=0x7f60a80b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.477+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 shutdown_connections 2024-04-26T23:42:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.477+0000 7f60aea00640 1 -- 172.21.15.46:0/3933234115 wait complete. 2024-04-26T23:42:48.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:42:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.711+0000 7fdf4f200640 1 Processor -- start 2024-04-26T23:42:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.712+0000 7fdf4f200640 1 -- start start 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.713+0000 7fdf4f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf500d3cc0 0x7fdf500cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.713+0000 7fdf4f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf500d2080 0x7fdf500b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.713+0000 7fdf4f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf500d01a0 con 0x7fdf501627f0 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.713+0000 7fdf4f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdf500d0480 con 0x7fdf500d3cc0 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.713+0000 7fdf4f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf500d0760 con 0x7fdf500d2080 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf500d3cc0 0x7fdf500cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf500d3cc0 0x7fdf500cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59900/0 (socket says 172.21.15.46:59900) 2024-04-26T23:42:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 -- 172.21.15.46:0/2487749628 learned_addr learned my addr 172.21.15.46:0/2487749628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 -- 172.21.15.46:0/2487749628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf500d2080 msgr2=0x7fdf500b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 --2- 172.21.15.46:0/2487749628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf500d2080 0x7fdf500b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 -- 172.21.15.46:0/2487749628 >> v1:172.21.15.142:6789/0 conn(0x7fdf501627f0 legacy=0x7fdf500d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.714+0000 7fdf4d400640 1 -- 172.21.15.46:0/2487749628 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500d0a40 con 0x7fdf500d3cc0 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.715+0000 7fdf4d400640 1 --2- 172.21.15.46:0/2487749628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cfbc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df6908a753d59fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.715+0000 7fdf4ca00640 1 -- 172.21.15.46:0/2487749628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c037070 con 0x7fdf500d3cc0 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4ca00640 1 -- 172.21.15.46:0/2487749628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c032440 con 0x7fdf500d3cc0 2024-04-26T23:42:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4ca00640 1 -- 172.21.15.46:0/2487749628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c032810 con 0x7fdf500d3cc0 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 -- 172.21.15.46:0/2487749628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d3cc0 msgr2=0x7fdf500cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 --2- 172.21.15.46:0/2487749628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cfbc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02a290 comp rx=0 tx=0).stop 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 -- 172.21.15.46:0/2487749628 shutdown_connections 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 --2- 172.21.15.46:0/2487749628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf500d2080 0x7fdf500b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 --2- 172.21.15.46:0/2487749628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cfbc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 -- 172.21.15.46:0/2487749628 >> 172.21.15.46:0/2487749628 conn(0x7fdf5009e380 msgr2=0x7fdf500d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 -- 172.21.15.46:0/2487749628 shutdown_connections 2024-04-26T23:42:48.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.716+0000 7fdf4f200640 1 -- 172.21.15.46:0/2487749628 wait complete. 2024-04-26T23:42:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.718+0000 7fdf4f200640 1 Processor -- start 2024-04-26T23:42:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.718+0000 7fdf4f200640 1 -- start start 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 0x7fdf5014d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50162bd0 con 0x7fdf500d2080 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf500d2530 con 0x7fdf500d3cc0 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.719+0000 7fdf4f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf500b4a90 con 0x7fdf501627f0 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59912/0 (socket says 172.21.15.46:59912) 2024-04-26T23:42:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 -- 172.21.15.46:0/1491005226 learned_addr learned my addr 172.21.15.46:0/1491005226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4d400640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 -- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 msgr2=0x7fdf5014d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4e800640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 0x7fdf5014d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 0x7fdf5014d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 -- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 msgr2=0x7fdf500cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 -- 172.21.15.46:0/1491005226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500b82b0 con 0x7fdf500d2080 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4d400640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4e800640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 0x7fdf5014d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.720+0000 7fdf4de00640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b3e0 tx=0x7fdf4000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.721+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40015020 con 0x7fdf500d2080 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.721+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf40011850 con 0x7fdf500d2080 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.721+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40011c40 con 0x7fdf500d2080 2024-04-26T23:42:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.721+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf500b8590 con 0x7fdf500d2080 2024-04-26T23:42:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.721+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf500b8b50 con 0x7fdf500d2080 2024-04-26T23:42:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.722+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf500693b0 con 0x7fdf500d2080 2024-04-26T23:42:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.723+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf40018020 con 0x7fdf500d2080 2024-04-26T23:42:48.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.724+0000 7fdf46000640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 0x7fdf3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:48.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.724+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5749+0+0 (secure 0 0 0) 0x7fdf4008bc70 con 0x7fdf500d2080 2024-04-26T23:42:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.725+0000 7fdf4d400640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 0x7fdf3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.726+0000 7fdf4d400640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 0x7fdf3006edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c034000 tx=0x7fdf3c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.729+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf4001f020 con 0x7fdf500d2080 2024-04-26T23:42:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:48.942+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf500d3390 con 0x7fdf500d2080 2024-04-26T23:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.472+0000 7fdf46000640 1 -- 172.21.15.46:0/1491005226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) ==== 169+0+0 (secure 0 0 0) 0x7fdf40059f40 con 0x7fdf500d2080 2024-04-26T23:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.477+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 msgr2=0x7fdf3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.477+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 0x7fdf3006edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c034000 tx=0x7fdf3c02e000 comp rx=0 tx=0).stop 2024-04-26T23:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.477+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 msgr2=0x7fdf500b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.477+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b3e0 tx=0x7fdf4000b8a0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 shutdown_connections 2024-04-26T23:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdf3006c8f0 0x7fdf3006edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf501627f0 0x7fdf5014d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf500d3cc0 0x7fdf500cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 --2- 172.21.15.46:0/1491005226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf500d2080 0x7fdf500b7bf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 >> 172.21.15.46:0/1491005226 conn(0x7fdf5009e380 msgr2=0x7fdf50161ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 shutdown_connections 2024-04-26T23:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.481+0000 7fdf4f200640 1 -- 172.21.15.46:0/1491005226 wait complete. 2024-04-26T23:42:49.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:42:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.697+0000 7f6901400640 1 Processor -- start 2024-04-26T23:42:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.697+0000 7f6901400640 1 -- start start 2024-04-26T23:42:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.699+0000 7f6901400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 0x7f68fc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.699+0000 7f6901400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 0x7f68fc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.699+0000 7f6901400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc06eb90 con 0x7f68fc0b53b0 2024-04-26T23:42:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.699+0000 7f6901400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f68fc0b4310 con 0x7f68fc15d730 2024-04-26T23:42:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.699+0000 7f6901400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0b45f0 con 0x7f68fc15e220 2024-04-26T23:42:49.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.700+0000 7f68fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 0x7f68fc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.700+0000 7f68f3e00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f68fc0b53b0 0x7f68fc15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:59246/0 (socket says 172.21.15.46:59246) 2024-04-26T23:42:49.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.700+0000 7f68f3e00640 1 -- 172.21.15.46:0/1165152801 learned_addr learned my addr 172.21.15.46:0/1165152801 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:49.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fa800640 1 --2- 172.21.15.46:0/1165152801 >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 0x7f68fc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fb200640 1 -- 172.21.15.46:0/1165152801 >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 msgr2=0x7f68fc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:49.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fb200640 1 --2- 172.21.15.46:0/1165152801 >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 0x7f68fc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fb200640 1 -- 172.21.15.46:0/1165152801 >> v1:172.21.15.142:6789/0 conn(0x7f68fc0b53b0 legacy=0x7f68fc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fb200640 1 -- 172.21.15.46:0/1165152801 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0b48d0 con 0x7f68fc15e220 2024-04-26T23:42:49.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.701+0000 7f68fa800640 1 --2- 172.21.15.46:0/1165152801 >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 0x7f68fc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:49.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.704+0000 7f68fb200640 1 --2- 172.21.15.46:0/1165152801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 0x7f68fc06dc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009d90 tx=0x7f68f402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90bd1bd69c25a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.705+0000 7f68f9e00640 1 -- 172.21.15.46:0/1165152801 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4007740 con 0x7f68fc15e220 2024-04-26T23:42:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.705+0000 7f68f9e00640 1 -- 172.21.15.46:0/1165152801 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f40078e0 con 0x7f68fc15e220 2024-04-26T23:42:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.705+0000 7f68f9e00640 1 -- 172.21.15.46:0/1165152801 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4007cb0 con 0x7f68fc15e220 2024-04-26T23:42:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 -- 172.21.15.46:0/1165152801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 msgr2=0x7f68fc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 --2- 172.21.15.46:0/1165152801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 0x7f68fc06dc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009d90 tx=0x7f68f402a660 comp rx=0 tx=0).stop 2024-04-26T23:42:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 -- 172.21.15.46:0/1165152801 shutdown_connections 2024-04-26T23:42:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 --2- 172.21.15.46:0/1165152801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15e220 0x7f68fc06dc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 --2- 172.21.15.46:0/1165152801 >> v2:172.21.15.142:3300/0 conn(0x7f68fc15d730 0x7f68fc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 -- 172.21.15.46:0/1165152801 >> 172.21.15.46:0/1165152801 conn(0x7f68fc06c980 msgr2=0x7f68fc0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 -- 172.21.15.46:0/1165152801 shutdown_connections 2024-04-26T23:42:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.706+0000 7f6901400640 1 -- 172.21.15.46:0/1165152801 wait complete. 2024-04-26T23:42:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.708+0000 7f6901400640 1 Processor -- start 2024-04-26T23:42:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.708+0000 7f6901400640 1 -- start start 2024-04-26T23:42:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 0x7f68fc0c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 0x7f68fc15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0b3ec0 con 0x7f68fc0b53b0 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0b4e90 con 0x7f68fc15e220 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f6901400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0b4040 con 0x7f68fc15d730 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59926/0 (socket says 172.21.15.46:59926) 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 -- 172.21.15.46:0/3829352576 learned_addr learned my addr 172.21.15.46:0/3829352576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fb200640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 0x7f68fc15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68f3e00640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 0x7f68fc0c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 -- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 msgr2=0x7f68fc0c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 0x7f68fc0c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 -- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 msgr2=0x7f68fc15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 0x7f68fc15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fa800640 1 -- 172.21.15.46:0/3829352576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1f4e80 con 0x7f68fc0b53b0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68fb200640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 0x7f68fc15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.709+0000 7f68f3e00640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 0x7f68fc0c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f68fa800640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e760 tx=0x7f68ec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00f040 con 0x7f68fc0b53b0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ec004230 con 0x7f68fc0b53b0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc0c6060 con 0x7f68fc0b53b0 2024-04-26T23:42:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00ee70 con 0x7f68fc0b53b0 2024-04-26T23:42:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.710+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1f53d0 con 0x7f68fc0b53b0 2024-04-26T23:42:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.711+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc069530 con 0x7f68fc0b53b0 2024-04-26T23:42:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.712+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68ec00c6d0 con 0x7f68fc0b53b0 2024-04-26T23:42:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.713+0000 7f68f2a00640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.713+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f68ec08b7e0 con 0x7f68fc0b53b0 2024-04-26T23:42:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.714+0000 7f68f3e00640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.716+0000 7f68f3e00640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 0x7f68d806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0733c0 tx=0x7f68e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.718+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68fc0c6060 con 0x7f68fc0b53b0 2024-04-26T23:42:49.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.921+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f68fc073110 con 0x7f68fc0b53b0 2024-04-26T23:42:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.928+0000 7f68f2a00640 1 -- 172.21.15.46:0/3829352576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f68ec017020 con 0x7f68fc0b53b0 2024-04-26T23:42:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 msgr2=0x7f68d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 0x7f68d806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0733c0 tx=0x7f68e4009040 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 msgr2=0x7f68fc1f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e760 tx=0x7f68ec00ec20 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 shutdown_connections 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc15e220 0x7f68fc15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68fc15d730 0x7f68fc0c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 --2- 172.21.15.46:0/3829352576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0b53b0 0x7f68fc1f8f90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 >> 172.21.15.46:0/3829352576 conn(0x7f68fc06c980 msgr2=0x7f68fc072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 shutdown_connections 2024-04-26T23:42:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:49.932+0000 7f6901400640 1 -- 172.21.15.46:0/3829352576 wait complete. 2024-04-26T23:42:49.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-26T23:42:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.160+0000 7fb518e00640 1 Processor -- start 2024-04-26T23:42:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.160+0000 7fb518e00640 1 -- start start 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.161+0000 7fb518e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5140d1990 0x7fb5140d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.161+0000 7fb518e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 0x7fb5140b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.161+0000 7fb518e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5140d22b0 con 0x7fb5140d0c10 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.161+0000 7fb518e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140d0780 con 0x7fb5140d1990 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.161+0000 7fb518e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140d5da0 con 0x7fb5141639c0 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5140d1990 0x7fb5140d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb512a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5140d0c10 0x7fb5140d5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37220/0 (socket says 172.21.15.46:37220) 2024-04-26T23:42:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb512a00640 1 -- 172.21.15.46:0/2721909732 learned_addr learned my addr 172.21.15.46:0/2721909732 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513400640 1 -- 172.21.15.46:0/2721909732 >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 msgr2=0x7fb5140b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513e00640 1 --2- 172.21.15.46:0/2721909732 >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 0x7fb5140b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb512000640 1 -- 172.21.15.46:0/2721909732 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638070342 0 0) 0x7fb5140d22b0 con 0x7fb5140d0c10 2024-04-26T23:42:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513400640 1 --2- 172.21.15.46:0/2721909732 >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 0x7fb5140b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513400640 1 -- 172.21.15.46:0/2721909732 >> v1:172.21.15.46:6789/0 conn(0x7fb5140d0c10 legacy=0x7fb5140d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513400640 1 -- 172.21.15.46:0/2721909732 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140cfe50 con 0x7fb5140d1990 2024-04-26T23:42:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.162+0000 7fb513e00640 1 --2- 172.21.15.46:0/2721909732 >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 0x7fb5140b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.163+0000 7fb513400640 1 --2- 172.21.15.46:0/2721909732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5140d1990 0x7fb5140d1d70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb51408d7c0 tx=0x7fb50402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167ddcc2e859f217 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb512000640 1 -- 172.21.15.46:0/2721909732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504007740 con 0x7fb5140d1990 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb512000640 1 -- 172.21.15.46:0/2721909732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5040078e0 con 0x7fb5140d1990 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb512000640 1 -- 172.21.15.46:0/2721909732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504007cb0 con 0x7fb5140d1990 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 -- 172.21.15.46:0/2721909732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5140d1990 msgr2=0x7fb5140d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 --2- 172.21.15.46:0/2721909732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5140d1990 0x7fb5140d1d70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb51408d7c0 tx=0x7fb50402a680 comp rx=0 tx=0).stop 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 -- 172.21.15.46:0/2721909732 shutdown_connections 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 --2- 172.21.15.46:0/2721909732 >> v2:172.21.15.142:3300/0 conn(0x7fb5141639c0 0x7fb5140b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 --2- 172.21.15.46:0/2721909732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5140d1990 0x7fb5140d1d70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 -- 172.21.15.46:0/2721909732 >> 172.21.15.46:0/2721909732 conn(0x7fb51409e2a0 msgr2=0x7fb5140d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 -- 172.21.15.46:0/2721909732 shutdown_connections 2024-04-26T23:42:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.164+0000 7fb518e00640 1 -- 172.21.15.46:0/2721909732 wait complete. 2024-04-26T23:42:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.166+0000 7fb518e00640 1 Processor -- start 2024-04-26T23:42:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.166+0000 7fb518e00640 1 -- start start 2024-04-26T23:42:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5140d0c10 0x7fb5141e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5140cf080 0x7fb5140ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415f530 con 0x7fb5141639c0 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140d0410 con 0x7fb5140d0c10 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.167+0000 7fb518e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140cfa90 con 0x7fb5140cf080 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59944/0 (socket says 172.21.15.46:59944) 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 -- 172.21.15.46:0/3221278327 learned_addr learned my addr 172.21.15.46:0/3221278327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb513400640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5140d0c10 0x7fb5141e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb513e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5140cf080 0x7fb5140ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 -- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5140cf080 msgr2=0x7fb5140ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5140cf080 0x7fb5140ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 -- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5140d0c10 msgr2=0x7fb5141e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5140d0c10 0x7fb5141e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.168+0000 7fb512a00640 1 -- 172.21.15.46:0/3221278327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5141ea080 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb512a00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c4d0 tx=0x7fb4fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc018070 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc007500 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004a20 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5141ea360 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.169+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140c8ab0 con 0x7fb5141639c0 2024-04-26T23:42:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.170+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140693b0 con 0x7fb5141639c0 2024-04-26T23:42:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.171+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4fc01b070 con 0x7fb5141639c0 2024-04-26T23:42:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.172+0000 7fb50b400640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.172+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fb4fc08ad30 con 0x7fb5141639c0 2024-04-26T23:42:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.173+0000 7fb513400640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.174+0000 7fb513400640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 0x7fb4f006edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.176+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4fc0592c0 con 0x7fb5141639c0 2024-04-26T23:42:50.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.464+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) -- 0x7fb5140d1770 con 0x7fb5141639c0 2024-04-26T23:42:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.488+0000 7fb50b400640 1 -- 172.21.15.46:0/3221278327 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v274) ==== 143+0+0 (secure 0 0 0) 0x7fb4fc058bc0 con 0x7fb5141639c0 2024-04-26T23:42:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 msgr2=0x7fb4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 0x7fb4f006edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402f000 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 msgr2=0x7fb5140cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c4d0 tx=0x7fb4fc00c990 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 shutdown_connections 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5140cf080 0x7fb5140ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5141639c0 0x7fb5140cdb40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 --2- 172.21.15.46:0/3221278327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5140d0c10 0x7fb5141e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 >> 172.21.15.46:0/3221278327 conn(0x7fb51409e2a0 msgr2=0x7fb5140d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 shutdown_connections 2024-04-26T23:42:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.491+0000 7fb518e00640 1 -- 172.21.15.46:0/3221278327 wait complete. 2024-04-26T23:42:50.515 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-26T23:42:50.515 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) That a directory is merged during scrub. 2024-04-26T23:42:50.515 INFO:tasks.cephfs_test_runner:test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:42:50.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:50.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.712+0000 7f81b6400640 1 Processor -- start 2024-04-26T23:42:50.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.713+0000 7f81b6400640 1 -- start start 2024-04-26T23:42:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81b00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b00d11f0 con 0x7f81b015f4e0 2024-04-26T23:42:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b6400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b00d5f20 con 0x7f81b00b4000 2024-04-26T23:42:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81b00d0ce0 con 0x7f81b00d2130 2024-04-26T23:42:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b5000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f81b015f4e0 0x7f81b00d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37236/0 (socket says 172.21.15.46:37236) 2024-04-26T23:42:50.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.714+0000 7f81b5000640 1 -- 172.21.15.46:0/2880929523 learned_addr learned my addr 172.21.15.46:0/2880929523 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905612094 0 0) 0x7f81b00d11f0 con 0x7f81b015f4e0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8194003520 con 0x7f81b015f4e0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81afe00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81b00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1799916517 0 0) 0x7f8194003520 con 0x7f81b015f4e0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 msgr2=0x7f81b00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81b00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 >> v1:172.21.15.142:6789/0 conn(0x7f81b00b4000 legacy=0x7f81b00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00d0f10 con 0x7f81b015f4e0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f81a00019f0 con 0x7f81b015f4e0 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 >> v1:172.21.15.46:6789/0 conn(0x7f81b015f4e0 legacy=0x7f81b00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003ff0 0x7f8194004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194004d40 0x7f81940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f819400bb50 con 0x7f8194003ff0 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f819400bcd0 con 0x7f8194004d40 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f819400bed0 con 0x7f81b00d2130 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.715+0000 7f81afe00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003ff0 0x7f8194004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81b5000640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81afe00640 1 -- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 msgr2=0x7f81940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81afe00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81940039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81b5a00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194004d40 0x7f81940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81afe00640 1 -- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194004d40 msgr2=0x7f81940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81afe00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194004d40 0x7f81940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81afe00640 1 -- 172.21.15.46:0/2880929523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8194003520 con 0x7f8194003ff0 2024-04-26T23:42:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.716+0000 7f81b5000640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00d2130 0x7f81940039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.717+0000 7f81afe00640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003ff0 0x7f8194004800 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f819800ba00 tx=0x7f8194005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d7ff8736339566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.717+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198030070 con 0x7f8194003ff0 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.717+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819802ada0 con 0x7f8194003ff0 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.717+0000 7f81af400640 1 -- 172.21.15.46:0/2880929523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b600 con 0x7f8194003ff0 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 -- 172.21.15.46:0/2880929523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003ff0 msgr2=0x7f8194004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 --2- 172.21.15.46:0/2880929523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003ff0 0x7f8194004800 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f819800ba00 tx=0x7f8194005830 comp rx=0 tx=0).stop 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b5a00640 1 -- 172.21.15.46:0/2880929523 reap_dead start 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 -- 172.21.15.46:0/2880929523 shutdown_connections 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 -- 172.21.15.46:0/2880929523 >> 172.21.15.46:0/2880929523 conn(0x7f81b00d5010 msgr2=0x7f81b00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 -- 172.21.15.46:0/2880929523 shutdown_connections 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.718+0000 7f81b6400640 1 -- 172.21.15.46:0/2880929523 wait complete. 2024-04-26T23:42:50.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.720+0000 7f81b6400640 1 Processor -- start 2024-04-26T23:42:50.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.720+0000 7f81b6400640 1 -- start start 2024-04-26T23:42:50.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194003ff0 0x7f81b01f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00cd130 0x7f81b00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00b1d60 con 0x7f81b00b6ae0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00d2990 con 0x7f8194003ff0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81b6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f81b015f8c0 con 0x7f81b00cd130 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59962/0 (socket says 172.21.15.46:59962) 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 -- 172.21.15.46:0/1962728026 learned_addr learned my addr 172.21.15.46:0/1962728026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 -- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00cd130 msgr2=0x7f81b00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00cd130 0x7f81b00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 -- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194003ff0 msgr2=0x7f81b01f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194003ff0 0x7f81b01f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.721+0000 7f81afe00640 1 -- 172.21.15.46:0/1962728026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01f5c90 con 0x7f81b00b6ae0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81afe00640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8198036040 tx=0x7f8198004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198043030 con 0x7f81b00b6ae0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819802aba0 con 0x7f81b00b6ae0 2024-04-26T23:42:50.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b600 con 0x7f81b00b6ae0 2024-04-26T23:42:50.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01f6a80 con 0x7f81b00b6ae0 2024-04-26T23:42:50.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.722+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b00cdff0 con 0x7f81b00b6ae0 2024-04-26T23:42:50.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.723+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b00693b0 con 0x7f81b00b6ae0 2024-04-26T23:42:50.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.724+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8198046070 con 0x7f81b00b6ae0 2024-04-26T23:42:50.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.725+0000 7f81acc00640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 0x7f818c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:50.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.725+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f81980ac480 con 0x7f81b00b6ae0 2024-04-26T23:42:50.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.726+0000 7f81b5000640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 0x7f818c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:50.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.727+0000 7f81b5000640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 0x7f818c06ede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81a0001350 tx=0x7f81a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.729+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81b01f6a80 con 0x7f81b00b6ae0 2024-04-26T23:42:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.936+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81b00d33d0 con 0x7f81b00b6ae0 2024-04-26T23:42:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.937+0000 7f81acc00640 1 -- 172.21.15.46:0/1962728026 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13255 (secure 0 0 0) 0x7f81b01f5c90 con 0x7f81b00b6ae0 2024-04-26T23:42:50.941 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:50.942 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:42:49.470104+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/970201181":"2024-04-27T23:42:46.184113+0000","172.21.15.46:6832/970201181":"2024-04-27T23:42:46.184113+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 msgr2=0x7f818c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 0x7f818c06ede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81a0001350 tx=0x7f81a0009040 comp rx=0 tx=0).stop 2024-04-26T23:42:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 msgr2=0x7f81b00b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8198036040 tx=0x7f8198004730 comp rx=0 tx=0).stop 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 shutdown_connections 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f818c06c920 0x7f818c06ede0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81b00cd130 0x7f81b00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b00b6ae0 0x7f81b00b6f40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 --2- 172.21.15.46:0/1962728026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8194003ff0 0x7f81b01f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 >> 172.21.15.46:0/1962728026 conn(0x7f81b00d5010 msgr2=0x7f81b00d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 shutdown_connections 2024-04-26T23:42:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:50.940+0000 7f81b6400640 1 -- 172.21.15.46:0/1962728026 wait complete. 2024-04-26T23:42:50.963 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:42:50.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:42:51.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.170+0000 7fa341600640 1 Processor -- start 2024-04-26T23:42:51.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.170+0000 7fa341600640 1 -- start start 2024-04-26T23:42:51.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.171+0000 7fa341600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 0x7fa33c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.171+0000 7fa341600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.171+0000 7fa341600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c0ce7d0 con 0x7fa33c0cdb30 2024-04-26T23:42:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.171+0000 7fa341600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa33c0ceab0 con 0x7fa33c15cec0 2024-04-26T23:42:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.171+0000 7fa341600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0ced90 con 0x7fa33c073d00 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.172+0000 7fa33be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 0x7fa33c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.172+0000 7fa340c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.172+0000 7fa340c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:34080/0 (socket says 172.21.15.46:34080) 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.172+0000 7fa33be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 0x7fa33c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:42088/0 (socket says 172.21.15.46:42088) 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.172+0000 7fa340c00640 1 -- 172.21.15.46:0/2972808182 learned_addr learned my addr 172.21.15.46:0/2972808182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.173+0000 7fa340c00640 1 -- 172.21.15.46:0/2972808182 >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 msgr2=0x7fa33c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.173+0000 7fa340c00640 1 --2- 172.21.15.46:0/2972808182 >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 0x7fa33c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.173+0000 7fa340c00640 1 -- 172.21.15.46:0/2972808182 >> v1:172.21.15.142:6789/0 conn(0x7fa33c0cdb30 legacy=0x7fa33c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.173+0000 7fa340c00640 1 -- 172.21.15.46:0/2972808182 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c0cf070 con 0x7fa33c073d00 2024-04-26T23:42:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.175+0000 7fa340c00640 1 --2- 172.21.15.46:0/2972808182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa33402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d88c5d586d7405 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.175+0000 7fa33aa00640 1 -- 172.21.15.46:0/2972808182 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037070 con 0x7fa33c073d00 2024-04-26T23:42:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.175+0000 7fa33aa00640 1 -- 172.21.15.46:0/2972808182 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa334032440 con 0x7fa33c073d00 2024-04-26T23:42:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.175+0000 7fa33aa00640 1 -- 172.21.15.46:0/2972808182 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334032810 con 0x7fa33c073d00 2024-04-26T23:42:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 -- 172.21.15.46:0/2972808182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 msgr2=0x7fa33c0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 --2- 172.21.15.46:0/2972808182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa33402a250 comp rx=0 tx=0).stop 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 -- 172.21.15.46:0/2972808182 shutdown_connections 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 --2- 172.21.15.46:0/2972808182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0b3b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 --2- 172.21.15.46:0/2972808182 >> v2:172.21.15.142:3300/0 conn(0x7fa33c15cec0 0x7fa33c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 -- 172.21.15.46:0/2972808182 >> 172.21.15.46:0/2972808182 conn(0x7fa33c0b27b0 msgr2=0x7fa33c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 -- 172.21.15.46:0/2972808182 shutdown_connections 2024-04-26T23:42:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.176+0000 7fa341600640 1 -- 172.21.15.46:0/2972808182 wait complete. 2024-04-26T23:42:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.178+0000 7fa341600640 1 Processor -- start 2024-04-26T23:42:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.179+0000 7fa341600640 1 -- start start 2024-04-26T23:42:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c0cdb30 0x7fa33c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0b4080 con 0x7fa33c15cec0 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c15d2a0 con 0x7fa33c0cdb30 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa341600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0b4200 con 0x7fa33c073d00 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa340c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa340c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59964/0 (socket says 172.21.15.46:59964) 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa340c00640 1 -- 172.21.15.46:0/4069274763 learned_addr learned my addr 172.21.15.46:0/4069274763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.180+0000 7fa33be00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa33b400640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c0cdb30 0x7fa33c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 -- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 msgr2=0x7fa33c0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 -- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c0cdb30 msgr2=0x7fa33c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c0cdb30 0x7fa33c15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 -- 172.21.15.46:0/4069274763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c0c5100 con 0x7fa33c15cec0 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa33be00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa340c00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.181+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334041020 con 0x7fa33c15cec0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.182+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33403d640 con 0x7fa33c15cec0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.182+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037040 con 0x7fa33c15cec0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.182+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c0c53e0 con 0x7fa33c15cec0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.182+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c0c59a0 con 0x7fa33c15cec0 2024-04-26T23:42:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.183+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c0693b0 con 0x7fa33c15cec0 2024-04-26T23:42:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.184+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa334044020 con 0x7fa33c15cec0 2024-04-26T23:42:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.185+0000 7fa333e00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.185+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fa3340ac120 con 0x7fa33c15cec0 2024-04-26T23:42:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.186+0000 7fa33be00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.187+0000 7fa33be00640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 0x7fa31806ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009af0 tx=0x7fa32c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.190+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa33404b030 con 0x7fa33c15cec0 2024-04-26T23:42:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.397+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa33c0c4200 con 0x7fa33c15cec0 2024-04-26T23:42:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.398+0000 7fa333e00640 1 -- 172.21.15.46:0/4069274763 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7fa33407a620 con 0x7fa33c15cec0 2024-04-26T23:42:51.403 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 msgr2=0x7fa31806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 0x7fa31806ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009af0 tx=0x7fa32c008040 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 msgr2=0x7fa33c15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa3340046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 shutdown_connections 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c15cec0 0x7fa33c15f560 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c0cdb30 0x7fa33c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 --2- 172.21.15.46:0/4069274763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa33c073d00 0x7fa33c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 >> 172.21.15.46:0/4069274763 conn(0x7fa33c0b27b0 msgr2=0x7fa33c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.401+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 shutdown_connections 2024-04-26T23:42:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.402+0000 7fa341600640 1 -- 172.21.15.46:0/4069274763 wait complete. 2024-04-26T23:42:51.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags' 2024-04-26T23:42:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.639+0000 7f4561200640 1 Processor -- start 2024-04-26T23:42:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.639+0000 7f4561200640 1 -- start start 2024-04-26T23:42:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4561200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f455c06e510 0x7f455c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4561200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f455c160bd0 0x7f455c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4561200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c06ee30 con 0x7f455c15d540 2024-04-26T23:42:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4561200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0742d0 con 0x7f455c06e510 2024-04-26T23:42:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4561200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f455c161940 con 0x7f455c160bd0 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4559e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f455c06e510 0x7f455c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4559e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f455c06e510 0x7f455c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59980/0 (socket says 172.21.15.46:59980) 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.641+0000 7f4559e00640 1 -- 172.21.15.46:0/2749786804 learned_addr learned my addr 172.21.15.46:0/2749786804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.642+0000 7f4559400640 1 --2- 172.21.15.46:0/2749786804 >> v2:172.21.15.142:3300/0 conn(0x7f455c160bd0 0x7f455c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.642+0000 7f4559e00640 1 -- 172.21.15.46:0/2749786804 >> v2:172.21.15.142:3300/0 conn(0x7f455c160bd0 msgr2=0x7f455c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.642+0000 7f4559e00640 1 --2- 172.21.15.46:0/2749786804 >> v2:172.21.15.142:3300/0 conn(0x7f455c160bd0 0x7f455c161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.642+0000 7f4559e00640 1 -- 172.21.15.46:0/2749786804 >> v1:172.21.15.142:6789/0 conn(0x7f455c15d540 legacy=0x7f455c15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.642+0000 7f4559e00640 1 -- 172.21.15.46:0/2749786804 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c071f80 con 0x7f455c06e510 2024-04-26T23:42:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.643+0000 7f4559e00640 1 --2- 172.21.15.46:0/2749786804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c06e8f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f454c009870 tx=0x7f454c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb368c1307760a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.643+0000 7f4558a00640 1 -- 172.21.15.46:0/2749786804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c037070 con 0x7f455c06e510 2024-04-26T23:42:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.643+0000 7f4558a00640 1 -- 172.21.15.46:0/2749786804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c032440 con 0x7f455c06e510 2024-04-26T23:42:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.643+0000 7f4558a00640 1 -- 172.21.15.46:0/2749786804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c032810 con 0x7f455c06e510 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 -- 172.21.15.46:0/2749786804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 msgr2=0x7f455c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 --2- 172.21.15.46:0/2749786804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c06e8f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f454c009870 tx=0x7f454c02a270 comp rx=0 tx=0).stop 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 -- 172.21.15.46:0/2749786804 shutdown_connections 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 --2- 172.21.15.46:0/2749786804 >> v2:172.21.15.142:3300/0 conn(0x7f455c160bd0 0x7f455c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 --2- 172.21.15.46:0/2749786804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c06e8f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 -- 172.21.15.46:0/2749786804 >> 172.21.15.46:0/2749786804 conn(0x7f455c06c530 msgr2=0x7f455c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 -- 172.21.15.46:0/2749786804 shutdown_connections 2024-04-26T23:42:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.644+0000 7f4561200640 1 -- 172.21.15.46:0/2749786804 wait complete. 2024-04-26T23:42:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.646+0000 7f4561200640 1 Processor -- start 2024-04-26T23:42:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.646+0000 7f4561200640 1 -- start start 2024-04-26T23:42:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f455c15d540 0x7f455c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f455c160bd0 0x7f455c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f455c072820 con 0x7f455c06e510 2024-04-26T23:42:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f455c072540 con 0x7f455c160bd0 2024-04-26T23:42:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4561200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f455c1615a0 con 0x7f455c15d540 2024-04-26T23:42:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59982/0 (socket says 172.21.15.46:59982) 2024-04-26T23:42:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 -- 172.21.15.46:0/4207410323 learned_addr learned my addr 172.21.15.46:0/4207410323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559400640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f455c15d540 0x7f455c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 -- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f455c15d540 msgr2=0x7f455c0aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f455c15d540 0x7f455c0aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 -- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f455c160bd0 msgr2=0x7f455c0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f455c160bd0 0x7f455c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.647+0000 7f4559e00640 1 -- 172.21.15.46:0/4207410323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0c69b0 con 0x7f455c06e510 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4559e00640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c037040 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c004230 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c030620 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c0af7b0 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.648+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c0afd70 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.649+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c0b0430 con 0x7f455c06e510 2024-04-26T23:42:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.650+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f454c0307c0 con 0x7f455c06e510 2024-04-26T23:42:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.651+0000 7f4552000640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 0x7f453c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.651+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f454c0ac900 con 0x7f455c06e510 2024-04-26T23:42:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.652+0000 7f4559400640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 0x7f453c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.653+0000 7f4559400640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 0x7f453c06ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f455c069cc0 tx=0x7f4548008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.656+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f455c0b0430 con 0x7f455c06e510 2024-04-26T23:42:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:51.936+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]} v 0) -- 0x7f455c06fb00 con 0x7f455c06e510 2024-04-26T23:42:52.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.501+0000 7f4552000640 1 -- 172.21.15.46:0/4207410323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]}]=0 v276) ==== 154+0+0 (secure 0 0 0) 0x7f454c07ae00 con 0x7f455c06e510 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 msgr2=0x7f453c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 0x7f453c06ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f455c069cc0 tx=0x7f4548008040 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 msgr2=0x7f455c15c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 shutdown_connections 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f453c06c920 0x7f453c06ede0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f455c160bd0 0x7f455c0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f455c15d540 0x7f455c0aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 --2- 172.21.15.46:0/4207410323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c06e510 0x7f455c15c530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.504+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 >> 172.21.15.46:0/4207410323 conn(0x7f455c06c530 msgr2=0x7f455c160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.505+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 shutdown_connections 2024-04-26T23:42:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.505+0000 7f4561200640 1 -- 172.21.15.46:0/4207410323 wait complete. 2024-04-26T23:42:52.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:42:52.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.749+0000 7f124f200640 1 Processor -- start 2024-04-26T23:42:52.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.749+0000 7f124f200640 1 -- start start 2024-04-26T23:42:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.751+0000 7f124f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12500b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.751+0000 7f124f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12500d07d0 con 0x7f12500b5fb0 2024-04-26T23:42:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.751+0000 7f124f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12500d0ab0 con 0x7f1250162aa0 2024-04-26T23:42:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.751+0000 7f124f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12500d0d90 con 0x7f12501626c0 2024-04-26T23:42:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.752+0000 7f124d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f12500b5fb0 0x7f12500d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57218/0 (socket says 172.21.15.46:57218) 2024-04-26T23:42:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.752+0000 7f124d400640 1 -- 172.21.15.46:0/3246545791 learned_addr learned my addr 172.21.15.46:0/3246545791 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.752+0000 7f124de00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12500b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.752+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1667826866 0 0) 0x7f12500d07d0 con 0x7f12500b5fb0 2024-04-26T23:42:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.752+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1234003520 con 0x7f12500b5fb0 2024-04-26T23:42:52.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 531110287 0 0) 0x7f1234003520 con 0x7f12500b5fb0 2024-04-26T23:42:52.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 msgr2=0x7f12500b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12500b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 >> v1:172.21.15.142:6789/0 conn(0x7f1250162aa0 legacy=0x7f12500d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500d2c80 con 0x7f12500b5fb0 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f123c005090 con 0x7f12500b5fb0 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 >> v1:172.21.15.46:6789/0 conn(0x7f12500b5fb0 legacy=0x7f12500d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12340039e0 0x7f1234003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1234004800 0x7f1234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12500d0d90 con 0x7f12340039e0 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f123400ba70 con 0x7f1234004800 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f123400bc70 con 0x7f12501626c0 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.753+0000 7f124de00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12340039e0 0x7f1234003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124d400640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1234004800 0x7f1234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124e800640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124de00640 1 -- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 msgr2=0x7f12340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124de00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12501626c0 0x7f12340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124de00640 1 -- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1234004800 msgr2=0x7f1234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124de00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1234004800 0x7f1234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.754+0000 7f124de00640 1 -- 172.21.15.46:0/3246545791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1234003520 con 0x7f12340039e0 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.755+0000 7f124de00640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12340039e0 0x7f1234003e00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1240009a90 tx=0x7f1234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41de2003b4c0988 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.755+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240036070 con 0x7f12340039e0 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.755+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240031400 con 0x7f12340039e0 2024-04-26T23:42:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.755+0000 7f124ca00640 1 -- 172.21.15.46:0/3246545791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240009690 con 0x7f12340039e0 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 -- 172.21.15.46:0/3246545791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12340039e0 msgr2=0x7f1234003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 --2- 172.21.15.46:0/3246545791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12340039e0 0x7f1234003e00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1240009a90 tx=0x7f1234005830 comp rx=0 tx=0).stop 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124e800640 1 -- 172.21.15.46:0/3246545791 reap_dead start 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 -- 172.21.15.46:0/3246545791 shutdown_connections 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 -- 172.21.15.46:0/3246545791 >> 172.21.15.46:0/3246545791 conn(0x7f125009e2a0 msgr2=0x7f12500d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 -- 172.21.15.46:0/3246545791 shutdown_connections 2024-04-26T23:42:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.756+0000 7f124f200640 1 -- 172.21.15.46:0/3246545791 wait complete. 2024-04-26T23:42:52.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.758+0000 7f124f200640 1 Processor -- start 2024-04-26T23:42:52.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.758+0000 7f124f200640 1 -- start start 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12340039e0 0x7f12501f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f125006d350 0x7f125006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12500b47c0 con 0x7f12501f85e0 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12500b4940 con 0x7f12340039e0 2024-04-26T23:42:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12500b4ac0 con 0x7f125006d350 2024-04-26T23:42:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38872/0 (socket says 172.21.15.46:38872) 2024-04-26T23:42:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124e800640 1 -- 172.21.15.46:0/1329757178 learned_addr learned my addr 172.21.15.46:0/1329757178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.759+0000 7f124d400640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f125006d350 0x7f125006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 -- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f125006d350 msgr2=0x7f125006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f125006d350 0x7f125006d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 -- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12340039e0 msgr2=0x7f12501f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12340039e0 0x7f12501f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 -- 172.21.15.46:0/1329757178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12501f9310 con 0x7f12501f85e0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124e800640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1250162e80 tx=0x7f124800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248003970 con 0x7f12501f85e0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1248018070 con 0x7f12501f85e0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248003d50 con 0x7f12501f85e0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.760+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12501f52b0 con 0x7f12501f85e0 2024-04-26T23:42:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.761+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12501f4f80 con 0x7f12501f85e0 2024-04-26T23:42:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.762+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125006dcf0 con 0x7f12501f85e0 2024-04-26T23:42:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.763+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12480050d0 con 0x7f12501f85e0 2024-04-26T23:42:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.763+0000 7f1246000640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 0x7f123006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.764+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f124808b710 con 0x7f12501f85e0 2024-04-26T23:42:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.764+0000 7f124de00640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 0x7f123006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.765+0000 7f124de00640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 0x7f123006ede0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f124002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.768+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1248059ca0 con 0x7f12501f85e0 2024-04-26T23:42:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.973+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f12500693b0 con 0x7f12501f85e0 2024-04-26T23:42:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.981+0000 7f1246000640 1 -- 172.21.15.46:0/1329757178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f124801be40 con 0x7f12501f85e0 2024-04-26T23:42:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.983+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 msgr2=0x7f123006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.983+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 0x7f123006ede0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f124002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.983+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 msgr2=0x7f12501f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.983+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1250162e80 tx=0x7f124800ec10 comp rx=0 tx=0).stop 2024-04-26T23:42:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 shutdown_connections 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f123006c920 0x7f123006ede0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501f85e0 0x7f12501f8a90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f125006d350 0x7f125006d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 --2- 172.21.15.46:0/1329757178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12340039e0 0x7f12501f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 >> 172.21.15.46:0/1329757178 conn(0x7f125009e2a0 msgr2=0x7f12500b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 shutdown_connections 2024-04-26T23:42:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:52.984+0000 7f124f200640 1 -- 172.21.15.46:0/1329757178 wait complete. 2024-04-26T23:42:53.006 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:42:53.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:42:53.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:42:53.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:42:53.223 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:53.223 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:53.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:42:53.284 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:42:53.284 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:53.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:42:53.343 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:42:53.343 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:53.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:42:53.403 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:42:53.403 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:42:53.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:42:53.461 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:42:53.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:42:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.675+0000 7f9245800640 1 Processor -- start 2024-04-26T23:42:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.676+0000 7f9245800640 1 -- start start 2024-04-26T23:42:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92401466b0 0x7f924006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 0x7f92400cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f9240160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9240073ad0 con 0x7f92401466b0 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9240146a90 con 0x7f92400720e0 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f9245800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9240073d70 con 0x7f9240160910 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f923fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92401466b0 0x7f924006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.677+0000 7f923fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92401466b0 0x7f924006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38882/0 (socket says 172.21.15.46:38882) 2024-04-26T23:42:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 -- 172.21.15.46:0/151500296 learned_addr learned my addr 172.21.15.46:0/151500296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f9244e00640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f9240160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 -- 172.21.15.46:0/151500296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 msgr2=0x7f9240160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f9240160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 -- 172.21.15.46:0/151500296 >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 msgr2=0x7f92400cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f9237e00640 1 --2- 172.21.15.46:0/151500296 >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 0x7f92400cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 --2- 172.21.15.46:0/151500296 >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 0x7f92400cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f923fe00640 1 -- 172.21.15.46:0/151500296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9240074050 con 0x7f92401466b0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.678+0000 7f9237e00640 1 --2- 172.21.15.46:0/151500296 >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 0x7f92400cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.679+0000 7f923fe00640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f924006de80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9230009870 tx=0x7f923002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cfa36db0a9aee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.679+0000 7f923f400640 1 -- 172.21.15.46:0/151500296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230037070 con 0x7f92401466b0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.679+0000 7f923f400640 1 -- 172.21.15.46:0/151500296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9230032440 con 0x7f92401466b0 2024-04-26T23:42:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f923f400640 1 -- 172.21.15.46:0/151500296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230032810 con 0x7f92401466b0 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 -- 172.21.15.46:0/151500296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 msgr2=0x7f924006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f924006de80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9230009870 tx=0x7f923002a210 comp rx=0 tx=0).stop 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 -- 172.21.15.46:0/151500296 shutdown_connections 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f9240160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 --2- 172.21.15.46:0/151500296 >> v2:172.21.15.142:3300/0 conn(0x7f92400720e0 0x7f92400cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 --2- 172.21.15.46:0/151500296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f924006de80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 -- 172.21.15.46:0/151500296 >> 172.21.15.46:0/151500296 conn(0x7f92400b26d0 msgr2=0x7f92400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 -- 172.21.15.46:0/151500296 shutdown_connections 2024-04-26T23:42:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.680+0000 7f9245800640 1 -- 172.21.15.46:0/151500296 wait complete. 2024-04-26T23:42:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.682+0000 7f9245800640 1 Processor -- start 2024-04-26T23:42:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.682+0000 7f9245800640 1 -- start start 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92400720e0 0x7f92400b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f924013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f924006e3c0 con 0x7f92401466b0 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92400cb470 con 0x7f92400720e0 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9245800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92400cdda0 con 0x7f9240160910 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9237e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9237e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38892/0 (socket says 172.21.15.46:38892) 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.683+0000 7f9237e00640 1 -- 172.21.15.46:0/2563443270 learned_addr learned my addr 172.21.15.46:0/2563443270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f923fe00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92400720e0 0x7f92400b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9244e00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f924013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 -- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 msgr2=0x7f924013e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f924013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 -- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92400720e0 msgr2=0x7f92400b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92400720e0 0x7f92400b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 -- 172.21.15.46:0/2563443270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92400b3e80 con 0x7f92401466b0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.684+0000 7f9237e00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f922800c7f0 tx=0x7f922800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.685+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228015070 con 0x7f92401466b0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.685+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9228007820 con 0x7f92401466b0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.685+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228007bd0 con 0x7f92401466b0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.685+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92400b4160 con 0x7f92401466b0 2024-04-26T23:42:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.685+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92400b4720 con 0x7f92401466b0 2024-04-26T23:42:53.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.686+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92400d06a0 con 0x7f92401466b0 2024-04-26T23:42:53.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.687+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9228018070 con 0x7f92401466b0 2024-04-26T23:42:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.689+0000 7f923cc00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.689+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f922808bb50 con 0x7f92401466b0 2024-04-26T23:42:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.689+0000 7f923fe00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.690+0000 7f923fe00640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 0x7f921c06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9230034000 tx=0x7f923002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.693+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92400d06a0 con 0x7f92401466b0 2024-04-26T23:42:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.936+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92400693b0 con 0x7f92401466b0 2024-04-26T23:42:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.937+0000 7f923cc00640 1 -- 172.21.15.46:0/2563443270 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+3872 (secure 0 0 0) 0x7f922805a0f0 con 0x7f92401466b0 2024-04-26T23:42:53.941 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:53.941 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2207152103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2207152103},{"type":"v1","addr":"172.21.15.46:6835","nonce":2207152103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/857002077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":857002077},{"type":"v1","addr":"172.21.15.142:6835","nonce":857002077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3462108139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3462108139},{"type":"v1","addr":"172.21.15.142:6837","nonce":3462108139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2238514593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2238514593},{"type":"v1","addr":"172.21.15.46:6834","nonce":2238514593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[]} 2024-04-26T23:42:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 msgr2=0x7f921c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 0x7f921c06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9230034000 tx=0x7f923002e000 comp rx=0 tx=0).stop 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 msgr2=0x7f92400b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f922800c7f0 tx=0x7f922800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 shutdown_connections 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9240160910 0x7f924013e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92401466b0 0x7f92400b3710 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 --2- 172.21.15.46:0/2563443270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92400720e0 0x7f92400b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.939+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 >> 172.21.15.46:0/2563443270 conn(0x7f92400b26d0 msgr2=0x7f924006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.940+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 shutdown_connections 2024-04-26T23:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:53.940+0000 7f9245800640 1 -- 172.21.15.46:0/2563443270 wait complete. 2024-04-26T23:42:53.963 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:42:53.963 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:42:53.963 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:42:53.963 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:42:53.963 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:42:53.963 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:42:53.964 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:42:53.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:42:53.964 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:42:53.964 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:42:53.964 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:42:53.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:42:53.964 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:42:53.964 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:42:53.965 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:42:53.965 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:42:53.980 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:42:53.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:42:53.983 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:42:53.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:42:53.986 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:42:53.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:42:53.990 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:42:53.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:42:54.028 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:42:54.028 DEBUG:teuthology.parallel:result is None 2024-04-26T23:42:54.032 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:42:54.032 DEBUG:teuthology.parallel:result is None 2024-04-26T23:42:54.037 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:42:54.037 DEBUG:teuthology.parallel:result is None 2024-04-26T23:42:54.042 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:42:54.043 DEBUG:teuthology.parallel:result is None 2024-04-26T23:42:54.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:42:54.177 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:42:54.177 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:42:54.187 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:42:54.196 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:42:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.221+0000 7f944ba00640 1 Processor -- start 2024-04-26T23:42:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.222+0000 7f944ba00640 1 -- start start 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f944ba00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f944406e510 0x7f944406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f944ba00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9444160bd0 0x7f9444161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f944ba00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944406ee30 con 0x7f944415d540 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f944ba00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f94440742d0 con 0x7f944406e510 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f944ba00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9444161940 con 0x7f9444160bd0 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f9449000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f944406e510 0x7f944406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f9449000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f944406e510 0x7f944406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38948/0 (socket says 172.21.15.46:38948) 2024-04-26T23:42:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.223+0000 7f9449000640 1 -- 172.21.15.46:0/409477483 learned_addr learned my addr 172.21.15.46:0/409477483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.224+0000 7f9443e00640 1 --2- 172.21.15.46:0/409477483 >> v2:172.21.15.142:3300/0 conn(0x7f9444160bd0 0x7f9444161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.224+0000 7f9449000640 1 -- 172.21.15.46:0/409477483 >> v2:172.21.15.142:3300/0 conn(0x7f9444160bd0 msgr2=0x7f9444161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.224+0000 7f9449000640 1 --2- 172.21.15.46:0/409477483 >> v2:172.21.15.142:3300/0 conn(0x7f9444160bd0 0x7f9444161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.224+0000 7f9449000640 1 -- 172.21.15.46:0/409477483 >> v1:172.21.15.142:6789/0 conn(0x7f944415d540 legacy=0x7f944415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.224+0000 7f9449000640 1 -- 172.21.15.46:0/409477483 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9444071f80 con 0x7f944406e510 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.225+0000 7f9449000640 1 --2- 172.21.15.46:0/409477483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944406e510 0x7f944406e8f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9438009870 tx=0x7f943802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84276f1a617b6af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.225+0000 7f9443400640 1 -- 172.21.15.46:0/409477483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438037070 con 0x7f944406e510 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.225+0000 7f9443400640 1 -- 172.21.15.46:0/409477483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9438032440 con 0x7f944406e510 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.225+0000 7f9443400640 1 -- 172.21.15.46:0/409477483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438032810 con 0x7f944406e510 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 -- 172.21.15.46:0/409477483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944406e510 msgr2=0x7f944406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 --2- 172.21.15.46:0/409477483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944406e510 0x7f944406e8f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9438009870 tx=0x7f943802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 -- 172.21.15.46:0/409477483 shutdown_connections 2024-04-26T23:42:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 --2- 172.21.15.46:0/409477483 >> v2:172.21.15.142:3300/0 conn(0x7f9444160bd0 0x7f9444161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 --2- 172.21.15.46:0/409477483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944406e510 0x7f944406e8f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 -- 172.21.15.46:0/409477483 >> 172.21.15.46:0/409477483 conn(0x7f944406c530 msgr2=0x7f944406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 -- 172.21.15.46:0/409477483 shutdown_connections 2024-04-26T23:42:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.226+0000 7f944ba00640 1 -- 172.21.15.46:0/409477483 wait complete. 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.228+0000 7f944ba00640 1 Processor -- start 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.228+0000 7f944ba00640 1 -- start start 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944406e510 0x7f94441f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9444160bd0 0x7f94441f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9444072820 con 0x7f944415d540 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9444072540 con 0x7f9444160bd0 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f944ba00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f94441615a0 con 0x7f944406e510 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38950/0 (socket says 172.21.15.46:38950) 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 -- 172.21.15.46:0/2303661790 learned_addr learned my addr 172.21.15.46:0/2303661790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9449a00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9444160bd0 0x7f94441f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9449000640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944406e510 0x7f94441f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 -- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944406e510 msgr2=0x7f94441f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944406e510 0x7f94441f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 -- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9444160bd0 msgr2=0x7f94441f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9444160bd0 0x7f94441f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.229+0000 7f9443e00640 1 -- 172.21.15.46:0/2303661790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94441f5500 con 0x7f944415d540 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f9443e00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f943000d620 tx=0x7f943000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943000b620 con 0x7f944415d540 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9430019070 con 0x7f944415d540 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943000ba00 con 0x7f944415d540 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94441f57e0 con 0x7f944415d540 2024-04-26T23:42:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.230+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441f5180 con 0x7f944415d540 2024-04-26T23:42:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.231+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9444069530 con 0x7f944415d540 2024-04-26T23:42:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.232+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f943000bba0 con 0x7f944415d540 2024-04-26T23:42:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.233+0000 7f9440c00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 0x7f942406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.233+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f943008bc90 con 0x7f944415d540 2024-04-26T23:42:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.234+0000 7f9449000640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 0x7f942406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.234+0000 7f9449000640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 0x7f942406ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9438034000 tx=0x7f943802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.238+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9430059bd0 con 0x7f944415d540 2024-04-26T23:42:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.454+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f94440b2f90 con 0x7f944415d540 2024-04-26T23:42:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.524+0000 7f9440c00640 1 -- 172.21.15.46:0/2303661790 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) ==== 97+0+0 (secure 0 0 0) 0x7f943005ded0 con 0x7f944415d540 2024-04-26T23:42:54.529 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 msgr2=0x7f942406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 0x7f942406ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9438034000 tx=0x7f943802e000 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 msgr2=0x7f94441f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f943000d620 tx=0x7f943000dae0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 shutdown_connections 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942406c920 0x7f942406ede0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9444160bd0 0x7f94441f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944415d540 0x7f94441f8720 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.528+0000 7f944ba00640 1 --2- 172.21.15.46:0/2303661790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944406e510 0x7f94441f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.529+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 >> 172.21.15.46:0/2303661790 conn(0x7f944406c530 msgr2=0x7f94440b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.529+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 shutdown_connections 2024-04-26T23:42:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.529+0000 7f944ba00640 1 -- 172.21.15.46:0/2303661790 wait complete. 2024-04-26T23:42:54.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:42:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.745+0000 7f1fa3200640 1 Processor -- start 2024-04-26T23:42:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.745+0000 7f1fa3200640 1 -- start start 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f9c0b48a0 0x7f1f9c162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c1635b0 con 0x7f1f9c1631d0 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c0af0c0 con 0x7f1f9c159950 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c0af3a0 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f9c0b48a0 0x7f1f9c162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1f9c1631d0 0x7f1f9c06e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57242/0 (socket says 172.21.15.46:57242) 2024-04-26T23:42:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.747+0000 7f1fa2800640 1 -- 172.21.15.46:0/950191173 learned_addr learned my addr 172.21.15.46:0/950191173 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.748+0000 7f1fa1400640 1 -- 172.21.15.46:0/950191173 >> v1:172.21.15.142:6789/0 conn(0x7f1f9c159950 legacy=0x7f1f9c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.748+0000 7f1fa1400640 1 -- 172.21.15.46:0/950191173 >> v1:172.21.15.46:6789/0 conn(0x7f1f9c1631d0 legacy=0x7f1f9c06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.748+0000 7f1fa0a00640 1 -- 172.21.15.46:0/950191173 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426569517 0 0) 0x7f1f9c1635b0 con 0x7f1f9c1631d0 2024-04-26T23:42:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.748+0000 7f1fa1400640 1 -- 172.21.15.46:0/950191173 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c0af680 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa1400640 1 --2- 172.21.15.46:0/950191173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b48a0 0x7f1f9c162c90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0097a0 tx=0x7f1f8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a15bbd5096e89b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa0a00640 1 -- 172.21.15.46:0/950191173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c037070 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa0a00640 1 -- 172.21.15.46:0/950191173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c032440 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa0a00640 1 -- 172.21.15.46:0/950191173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c032810 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa3200640 1 -- 172.21.15.46:0/950191173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b48a0 msgr2=0x7f1f9c162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.749+0000 7f1fa3200640 1 --2- 172.21.15.46:0/950191173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b48a0 0x7f1f9c162c90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0097a0 tx=0x7f1f8c02a230 comp rx=0 tx=0).stop 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.750+0000 7f1fa3200640 1 -- 172.21.15.46:0/950191173 shutdown_connections 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.750+0000 7f1fa3200640 1 --2- 172.21.15.46:0/950191173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b48a0 0x7f1f9c162c90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.750+0000 7f1fa3200640 1 -- 172.21.15.46:0/950191173 >> 172.21.15.46:0/950191173 conn(0x7f1f9c0b2820 msgr2=0x7f1f9c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.750+0000 7f1fa3200640 1 -- 172.21.15.46:0/950191173 shutdown_connections 2024-04-26T23:42:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.750+0000 7f1fa3200640 1 -- 172.21.15.46:0/950191173 wait complete. 2024-04-26T23:42:54.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.751+0000 7f1fa3200640 1 Processor -- start 2024-04-26T23:42:54.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.751+0000 7f1fa3200640 1 -- start start 2024-04-26T23:42:54.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f9c0b48a0 0x7f1f9c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 0x7f1f9c162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0b0160 con 0x7f1f9c1631d0 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0afc40 con 0x7f1f9c141e60 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.752+0000 7f1fa3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c06eb30 con 0x7f1f9c0b48a0 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38978/0 (socket says 172.21.15.46:38978) 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 -- 172.21.15.46:0/842240826 learned_addr learned my addr 172.21.15.46:0/842240826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa2800640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 0x7f1f9c162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 -- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f9c0b48a0 msgr2=0x7f1f9c1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f9c0b48a0 0x7f1f9c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 -- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 msgr2=0x7f1f9c162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 0x7f1f9c162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 -- 172.21.15.46:0/842240826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c162d10 con 0x7f1f9c1631d0 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa2800640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 0x7f1f9c162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:54.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.753+0000 7f1fa1400640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0098c0 tx=0x7f1f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.754+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c040020 con 0x7f1f9c1631d0 2024-04-26T23:42:54.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.754+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c162a30 con 0x7f1f9c1631d0 2024-04-26T23:42:54.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.754+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c038040 con 0x7f1f9c1631d0 2024-04-26T23:42:54.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.754+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c037070 con 0x7f1f9c1631d0 2024-04-26T23:42:54.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.754+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c153900 con 0x7f1f9c1631d0 2024-04-26T23:42:54.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.755+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c0693b0 con 0x7f1f9c1631d0 2024-04-26T23:42:54.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.756+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f8c043020 con 0x7f1f9c1631d0 2024-04-26T23:42:54.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.757+0000 7f1f96000640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 0x7f1f8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:54.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.757+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f1f8c0abd00 con 0x7f1f9c1631d0 2024-04-26T23:42:54.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.758+0000 7f1fa1e00640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 0x7f1f8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:54.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.759+0000 7f1fa1e00640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 0x7f1f8006ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000f60 tx=0x7f1f900079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:54.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:54.762+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f8c04a070 con 0x7f1f9c1631d0 2024-04-26T23:42:55.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.044+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1f9c15ef40 con 0x7f1f9c1631d0 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.045+0000 7f1f96000640 1 -- 172.21.15.46:0/842240826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f1f8c032e20 con 0x7f1f9c1631d0 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:42:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.051 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.053 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:42:55.054 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:42:55.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:42:55.060 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:42:55.061 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.047+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 msgr2=0x7f1f8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.047+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 0x7f1f8006ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000f60 tx=0x7f1f900079b0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.047+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 msgr2=0x7f1f9c161ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.047+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0098c0 tx=0x7f1f8c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 shutdown_connections 2024-04-26T23:42:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f8006c920 0x7f1f8006ede0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f9c141e60 0x7f1f9c162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c1631d0 0x7f1f9c161ec0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 --2- 172.21.15.46:0/842240826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f9c0b48a0 0x7f1f9c1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 >> 172.21.15.46:0/842240826 conn(0x7f1f9c0b2820 msgr2=0x7f1f9c0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 shutdown_connections 2024-04-26T23:42:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.048+0000 7f1fa3200640 1 -- 172.21.15.46:0/842240826 wait complete. 2024-04-26T23:42:55.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.279+0000 7f43abe00640 1 Processor -- start 2024-04-26T23:42:55.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.279+0000 7f43abe00640 1 -- start start 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.280+0000 7f43abe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43ac15f1d0 0x7f43ac0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.280+0000 7f43abe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43ac0cd770 0x7f43ac0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.280+0000 7f43abe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43ac0b03c0 con 0x7f43ac15edf0 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.280+0000 7f43abe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f43ac0cf5c0 con 0x7f43ac15f1d0 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.280+0000 7f43abe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f43ac0cb680 con 0x7f43ac0cd770 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43aaa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f43ac15edf0 0x7f43ac0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57246/0 (socket says 172.21.15.46:57246) 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43aaa00640 1 -- 172.21.15.46:0/822642421 learned_addr learned my addr 172.21.15.46:0/822642421 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43a2000640 1 --2- 172.21.15.46:0/822642421 >> v2:172.21.15.46:3300/0 conn(0x7f43ac15f1d0 0x7f43ac0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43ab400640 1 --2- 172.21.15.46:0/822642421 >> v2:172.21.15.142:3300/0 conn(0x7f43ac0cd770 0x7f43ac0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43aa000640 1 -- 172.21.15.46:0/822642421 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414125270 0 0) 0x7f43ac0b03c0 con 0x7f43ac15edf0 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43aa000640 1 -- 172.21.15.46:0/822642421 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4390003520 con 0x7f43ac15edf0 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43a2000640 1 -- 172.21.15.46:0/822642421 >> v2:172.21.15.142:3300/0 conn(0x7f43ac0cd770 msgr2=0x7f43ac0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43a2000640 1 --2- 172.21.15.46:0/822642421 >> v2:172.21.15.142:3300/0 conn(0x7f43ac0cd770 0x7f43ac0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43a2000640 1 -- 172.21.15.46:0/822642421 >> v1:172.21.15.46:6789/0 conn(0x7f43ac15edf0 legacy=0x7f43ac0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.281+0000 7f43a2000640 1 -- 172.21.15.46:0/822642421 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac0cb960 con 0x7f43ac15f1d0 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.282+0000 7f43a2000640 1 --2- 172.21.15.46:0/822642421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac15f1d0 0x7f43ac0cf080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f43ac070350 tx=0x7f439402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711a18450bf80f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43aa000640 1 -- 172.21.15.46:0/822642421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394037070 con 0x7f43ac15f1d0 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43aa000640 1 -- 172.21.15.46:0/822642421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394032440 con 0x7f43ac15f1d0 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43aa000640 1 -- 172.21.15.46:0/822642421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394032810 con 0x7f43ac15f1d0 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 -- 172.21.15.46:0/822642421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac15f1d0 msgr2=0x7f43ac0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 --2- 172.21.15.46:0/822642421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac15f1d0 0x7f43ac0cf080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f43ac070350 tx=0x7f439402a250 comp rx=0 tx=0).stop 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 -- 172.21.15.46:0/822642421 shutdown_connections 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 --2- 172.21.15.46:0/822642421 >> v2:172.21.15.142:3300/0 conn(0x7f43ac0cd770 0x7f43ac0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 --2- 172.21.15.46:0/822642421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac15f1d0 0x7f43ac0cf080 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 -- 172.21.15.46:0/822642421 >> 172.21.15.46:0/822642421 conn(0x7f43ac0b26d0 msgr2=0x7f43ac0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.283+0000 7f43abe00640 1 -- 172.21.15.46:0/822642421 shutdown_connections 2024-04-26T23:42:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.284+0000 7f43abe00640 1 -- 172.21.15.46:0/822642421 wait complete. 2024-04-26T23:42:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.285+0000 7f43abe00640 1 Processor -- start 2024-04-26T23:42:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.286+0000 7f43abe00640 1 -- start start 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ac15edf0 0x7f43ac147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43ac149820 0x7f43ac15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0cc200 con 0x7f43ac0cd770 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0cbf20 con 0x7f43ac15edf0 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43abe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0cb1a0 con 0x7f43ac149820 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38998/0 (socket says 172.21.15.46:38998) 2024-04-26T23:42:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 -- 172.21.15.46:0/1654763431 learned_addr learned my addr 172.21.15.46:0/1654763431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 -- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43ac149820 msgr2=0x7f43ac15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43ac149820 0x7f43ac15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 -- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ac15edf0 msgr2=0x7f43ac147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ac15edf0 0x7f43ac147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.287+0000 7f43aaa00640 1 -- 172.21.15.46:0/1654763431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac149dc0 con 0x7f43ac0cd770 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43aaa00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f439c001460 tx=0x7f439c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c017070 con 0x7f43ac0cd770 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c00ace0 con 0x7f43ac0cd770 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac14a0a0 con 0x7f43ac0cd770 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c012620 con 0x7f43ac0cd770 2024-04-26T23:42:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.288+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac14a660 con 0x7f43ac0cd770 2024-04-26T23:42:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.289+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac0c7970 con 0x7f43ac0cd770 2024-04-26T23:42:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.290+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439c01a070 con 0x7f43ac0cd770 2024-04-26T23:42:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.291+0000 7f43a3400640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 0x7f438806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.291+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f439c08b8b0 con 0x7f43ac0cd770 2024-04-26T23:42:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.292+0000 7f43a2000640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 0x7f438806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.293+0000 7f43a2000640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 0x7f438806ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4394034000 tx=0x7f439402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.296+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f439c059e00 con 0x7f43ac0cd770 2024-04-26T23:42:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.504+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43ac0693b0 con 0x7f43ac0cd770 2024-04-26T23:42:55.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.506+0000 7f43a3400640 1 -- 172.21.15.46:0/1654763431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13128 (secure 0 0 0) 0x7f439c059700 con 0x7f43ac0cd770 2024-04-26T23:42:55.511 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:55.511 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:42:54.539068+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 msgr2=0x7f438806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 0x7f438806ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4394034000 tx=0x7f439402c000 comp rx=0 tx=0).stop 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 msgr2=0x7f43ac1470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f439c001460 tx=0x7f439c00a430 comp rx=0 tx=0).stop 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 shutdown_connections 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f438806c920 0x7f438806ede0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43ac149820 0x7f43ac15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ac15edf0 0x7f43ac147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 --2- 172.21.15.46:0/1654763431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43ac0cd770 0x7f43ac1470e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.509+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 >> 172.21.15.46:0/1654763431 conn(0x7f43ac0b26d0 msgr2=0x7f43ac0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.510+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 shutdown_connections 2024-04-26T23:42:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.510+0000 7f43abe00640 1 -- 172.21.15.46:0/1654763431 wait complete. 2024-04-26T23:42:55.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:42:55.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:42:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.732+0000 7fbcbe200640 1 Processor -- start 2024-04-26T23:42:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.732+0000 7fbcbe200640 1 -- start start 2024-04-26T23:42:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbcb81640d0 0x7fbcb81644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb80cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb80cb4f0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb81649f0 con 0x7fbcb81640d0 2024-04-26T23:42:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.733+0000 7fbcbe200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb8071e80 con 0x7fbcb8073af0 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39002/0 (socket says 172.21.15.46:39002) 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 -- 172.21.15.46:0/3408440277 learned_addr learned my addr 172.21.15.46:0/3408440277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcb7e00640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb80cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbce00640 1 --2- 172.21.15.46:0/3408440277 >> v2:172.21.15.142:3300/0 conn(0x7fbcb81640d0 0x7fbcb81644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 -- 172.21.15.46:0/3408440277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 msgr2=0x7fbcb80cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb80cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 -- 172.21.15.46:0/3408440277 >> v2:172.21.15.142:3300/0 conn(0x7fbcb81640d0 msgr2=0x7fbcb81644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 --2- 172.21.15.46:0/3408440277 >> v2:172.21.15.142:3300/0 conn(0x7fbcb81640d0 0x7fbcb81644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.734+0000 7fbcbd800640 1 -- 172.21.15.46:0/3408440277 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb8072160 con 0x7fbcb80cbec0 2024-04-26T23:42:55.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.736+0000 7fbcbd800640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80b6d70 tx=0x7fbcb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b547e096315d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.736+0000 7fbcb7400640 1 -- 172.21.15.46:0/3408440277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0036070 con 0x7fbcb80cbec0 2024-04-26T23:42:55.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.736+0000 7fbcb7400640 1 -- 172.21.15.46:0/3408440277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb0031600 con 0x7fbcb80cbec0 2024-04-26T23:42:55.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.736+0000 7fbcb7400640 1 -- 172.21.15.46:0/3408440277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00319d0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 -- 172.21.15.46:0/3408440277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 msgr2=0x7fbcb80ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80b6d70 tx=0x7fbcb002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 -- 172.21.15.46:0/3408440277 shutdown_connections 2024-04-26T23:42:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80ce7d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 --2- 172.21.15.46:0/3408440277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb80cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 --2- 172.21.15.46:0/3408440277 >> v2:172.21.15.142:3300/0 conn(0x7fbcb81640d0 0x7fbcb81644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 -- 172.21.15.46:0/3408440277 >> 172.21.15.46:0/3408440277 conn(0x7fbcb80b27b0 msgr2=0x7fbcb80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 -- 172.21.15.46:0/3408440277 shutdown_connections 2024-04-26T23:42:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.737+0000 7fbcbe200640 1 -- 172.21.15.46:0/3408440277 wait complete. 2024-04-26T23:42:55.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.740+0000 7fbcbe200640 1 Processor -- start 2024-04-26T23:42:55.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.740+0000 7fbcbe200640 1 -- start start 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb8153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb81640d0 0x7fbcb80c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb80ced10 con 0x7fbcb80cbec0 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8073f70 con 0x7fbcb81640d0 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.741+0000 7fbcbe200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb80cee90 con 0x7fbcb8073af0 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39004/0 (socket says 172.21.15.46:39004) 2024-04-26T23:42:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 -- 172.21.15.46:0/1864986416 learned_addr learned my addr 172.21.15.46:0/1864986416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 -- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 msgr2=0x7fbcb8153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcbd800640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb81640d0 0x7fbcb80c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb8153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 -- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb81640d0 msgr2=0x7fbcb80c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb81640d0 0x7fbcb80c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 -- 172.21.15.46:0/1864986416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80b5d60 con 0x7fbcb80cbec0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.742+0000 7fbcb7e00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d800 tx=0x7fbca400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.743+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4018030 con 0x7fbcb80cbec0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.743+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca40096a0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.743+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4009870 con 0x7fbcb80cbec0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.743+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb80b6040 con 0x7fbcb80cbec0 2024-04-26T23:42:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.743+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb8153590 con 0x7fbcb80cbec0 2024-04-26T23:42:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.744+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb80c4eb0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.745+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbca401b070 con 0x7fbcb80cbec0 2024-04-26T23:42:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.747+0000 7fbcb4c00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 0x7fbc9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.747+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7fbca408bcc0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.747+0000 7fbcbce00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 0x7fbc9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.749+0000 7fbcbce00640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 0x7fbc9806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0098f0 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.752+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbca4092080 con 0x7fbcb80cbec0 2024-04-26T23:42:55.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.961+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcb80693b0 con 0x7fbcb80cbec0 2024-04-26T23:42:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.963+0000 7fbcb4c00640 1 -- 172.21.15.46:0/1864986416 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7fbca405a210 con 0x7fbcb80cbec0 2024-04-26T23:42:55.967 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.965+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 msgr2=0x7fbc9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.965+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 0x7fbc9806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0098f0 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.965+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 msgr2=0x7fbcb80b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.965+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d800 tx=0x7fbca400dcc0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 shutdown_connections 2024-04-26T23:42:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbc9806c8f0 0x7fbc9806edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb81640d0 0x7fbcb80c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb80cbec0 0x7fbcb80b52c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 --2- 172.21.15.46:0/1864986416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb8073af0 0x7fbcb8153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 >> 172.21.15.46:0/1864986416 conn(0x7fbcb80b27b0 msgr2=0x7fbcb80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 shutdown_connections 2024-04-26T23:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:55.966+0000 7fbcbe200640 1 -- 172.21.15.46:0/1864986416 wait complete. 2024-04-26T23:42:55.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:42:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.191+0000 7f0db5e00640 1 Processor -- start 2024-04-26T23:42:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.191+0000 7f0db5e00640 1 -- start start 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0db0073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0db0072190 0x7f0db0166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db006f1d0 con 0x7f0db00cf450 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0db006f470 con 0x7f0db0072190 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0db0072930 con 0x7f0db00cff40 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db4a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0db00cf450 0x7f0db00cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57260/0 (socket says 172.21.15.46:57260) 2024-04-26T23:42:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db4a00640 1 -- 172.21.15.46:0/2226046650 learned_addr learned my addr 172.21.15.46:0/2226046650 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0dafe00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0db0073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0db5400640 1 --2- 172.21.15.46:0/2226046650 >> v2:172.21.15.142:3300/0 conn(0x7f0db0072190 0x7f0db0166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2748619012 0 0) 0x7f0db006f1d0 con 0x7f0db00cf450 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d98003520 con 0x7f0db00cf450 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 20192228 0 0) 0x7f0d98003520 con 0x7f0db00cf450 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 msgr2=0x7f0db0073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0db0073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 >> v2:172.21.15.142:3300/0 conn(0x7f0db0072190 msgr2=0x7f0db0166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 --2- 172.21.15.46:0/2226046650 >> v2:172.21.15.142:3300/0 conn(0x7f0db0072190 0x7f0db0166b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db0072c10 con 0x7f0db00cf450 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0da4001670 con 0x7f0db00cf450 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 >> v1:172.21.15.46:6789/0 conn(0x7f0db00cf450 legacy=0x7f0db00cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0d980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d98003f80 0x7f0d98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98004d40 0x7f0d980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0db006f470 con 0x7f0d98003f80 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0072930 con 0x7f0d98004d40 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.193+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9800b9c0 con 0x7f0db00cff40 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d98003f80 0x7f0d98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0db4a00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0d980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 -- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 msgr2=0x7f0d980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0d980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 -- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98004d40 msgr2=0x7f0d980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98004d40 0x7f0d980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0dafe00640 1 -- 172.21.15.46:0/2226046650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98003520 con 0x7f0d98003f80 2024-04-26T23:42:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.194+0000 7f0db4a00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0db00cff40 0x7f0d980039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.195+0000 7f0dafe00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d98003f80 0x7f0d98004800 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009af0 tx=0x7f0d98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03d9b1e78323449 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.195+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c036070 con 0x7f0d98003f80 2024-04-26T23:42:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.195+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c031400 con 0x7f0d98003f80 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.195+0000 7f0daf400640 1 -- 172.21.15.46:0/2226046650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c009690 con 0x7f0d98003f80 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 -- 172.21.15.46:0/2226046650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d98003f80 msgr2=0x7f0d98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 --2- 172.21.15.46:0/2226046650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d98003f80 0x7f0d98004800 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009af0 tx=0x7f0d98005830 comp rx=0 tx=0).stop 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5400640 1 -- 172.21.15.46:0/2226046650 reap_dead start 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 -- 172.21.15.46:0/2226046650 shutdown_connections 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 -- 172.21.15.46:0/2226046650 >> 172.21.15.46:0/2226046650 conn(0x7f0db00b2820 msgr2=0x7f0db00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 -- 172.21.15.46:0/2226046650 shutdown_connections 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.196+0000 7f0db5e00640 1 -- 172.21.15.46:0/2226046650 wait complete. 2024-04-26T23:42:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.198+0000 7f0db5e00640 1 Processor -- start 2024-04-26T23:42:56.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.198+0000 7f0db5e00640 1 -- start start 2024-04-26T23:42:56.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d98003f80 0x7f0db01e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db01e0a70 0x7f0db01f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00734b0 con 0x7f0db01e0ef0 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00731d0 con 0x7f0db01e0a70 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0db0072600 con 0x7f0d98003f80 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.199+0000 7f0db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39028/0 (socket says 172.21.15.46:39028) 2024-04-26T23:42:56.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 -- 172.21.15.46:0/3462543885 learned_addr learned my addr 172.21.15.46:0/3462543885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 -- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d98003f80 msgr2=0x7f0db01e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d98003f80 0x7f0db01e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 -- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db01e0a70 msgr2=0x7f0db01f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db01e0a70 0x7f0db01f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 -- 172.21.15.46:0/3462543885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db01f8f00 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.200+0000 7f0db5400640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0db00768e0 tx=0x7f0da800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.201+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800bae0 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.201+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da801a070 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.201+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8014400 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.201+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db01dff90 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.201+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db01e0550 con 0x7f0db01e0ef0 2024-04-26T23:42:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.202+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db01e0750 con 0x7f0db01e0ef0 2024-04-26T23:42:56.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.203+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0da8005d10 con 0x7f0db01e0ef0 2024-04-26T23:42:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.204+0000 7f0dacc00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 0x7f0d9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.204+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f0da808bc30 con 0x7f0db01e0ef0 2024-04-26T23:42:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.205+0000 7f0db4a00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 0x7f0d9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.207+0000 7f0db4a00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 0x7f0d9006ede0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001860 tx=0x7f0da4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.209+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0db01e0750 con 0x7f0db01e0ef0 2024-04-26T23:42:56.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.415+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0db00693b0 con 0x7f0db01e0ef0 2024-04-26T23:42:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.416+0000 7f0dacc00640 1 -- 172.21.15.46:0/3462543885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) ==== 84+0+65 (secure 0 0 0) 0x7f0da8024020 con 0x7f0db01e0ef0 2024-04-26T23:42:56.420 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:42:56.420 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:42:56.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:42:56.421 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:42:56.421 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:42:56.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:42:56.421 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:42:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 msgr2=0x7f0d9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 0x7f0d9006ede0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001860 tx=0x7f0da4009040 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 msgr2=0x7f0db01f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0db00768e0 tx=0x7f0da800b480 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 shutdown_connections 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d9006c920 0x7f0d9006ede0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0db01e0ef0 0x7f0db01f87f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db01e0a70 0x7f0db01f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 --2- 172.21.15.46:0/3462543885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d98003f80 0x7f0db01e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 >> 172.21.15.46:0/3462543885 conn(0x7f0db00b2820 msgr2=0x7f0db01e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 shutdown_connections 2024-04-26T23:42:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.419+0000 7f0db5e00640 1 -- 172.21.15.46:0/3462543885 wait complete. 2024-04-26T23:42:56.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:42:56.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:42:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.649+0000 7fc3e4a00640 1 Processor -- start 2024-04-26T23:42:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.649+0000 7fc3e4a00640 1 -- start start 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.650+0000 7fc3e4a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3e00b2240 0x7fc3e006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.650+0000 7fc3e4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3e00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3e4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e00d3c10 con 0x7fc3e00d1b10 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3e4a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e00b2620 con 0x7fc3e00b2240 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3e4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e00cd4e0 con 0x7fc3e00cfd80 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3dde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3e00d1b10 0x7fc3e00cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57272/0 (socket says 172.21.15.46:57272) 2024-04-26T23:42:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3dde00640 1 -- 172.21.15.46:0/3553960841 learned_addr learned my addr 172.21.15.46:0/3553960841 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144592087 0 0) 0x7fc3e00d3c10 con 0x7fc3e00d1b10 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c4003520 con 0x7fc3e00d1b10 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3df200640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3e00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.651+0000 7fc3de800640 1 --2- 172.21.15.46:0/3553960841 >> v2:172.21.15.142:3300/0 conn(0x7fc3e00b2240 0x7fc3e006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3709650233 0 0) 0x7fc3c4003520 con 0x7fc3e00d1b10 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 msgr2=0x7fc3e00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3e00d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 >> v2:172.21.15.142:3300/0 conn(0x7fc3e00b2240 msgr2=0x7fc3e006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 --2- 172.21.15.46:0/3553960841 >> v2:172.21.15.142:3300/0 conn(0x7fc3e00b2240 0x7fc3e006d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e00cd7c0 con 0x7fc3e00d1b10 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc3c8001670 con 0x7fc3e00d1b10 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 >> v1:172.21.15.46:6789/0 conn(0x7fc3e00d1b10 legacy=0x7fc3e00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3c40039b0 0x7fc3c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3c4004800 0x7fc3c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00b2620 con 0x7fc3c40039b0 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00cd4e0 con 0x7fc3c4004800 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c400baf0 con 0x7fc3e00cfd80 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3de800640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3c40039b0 0x7fc3c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.652+0000 7fc3dde00640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3c4004800 0x7fc3c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3df200640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3de800640 1 -- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 msgr2=0x7fc3c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3de800640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3e00cfd80 0x7fc3c40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3de800640 1 -- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3c4004800 msgr2=0x7fc3c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3de800640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3c4004800 0x7fc3c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.653+0000 7fc3de800640 1 -- 172.21.15.46:0/3553960841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c4003520 con 0x7fc3c40039b0 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.654+0000 7fc3de800640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3c40039b0 0x7fc3c4003dd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ba00 tx=0x7fc3c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c43d1517fc1e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.654+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0030070 con 0x7fc3c40039b0 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.654+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d002adb0 con 0x7fc3c40039b0 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.654+0000 7fc3dd400640 1 -- 172.21.15.46:0/3553960841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000b600 con 0x7fc3c40039b0 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 -- 172.21.15.46:0/3553960841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3c40039b0 msgr2=0x7fc3c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/3553960841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3c40039b0 0x7fc3c4003dd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ba00 tx=0x7fc3c4005830 comp rx=0 tx=0).stop 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3df200640 1 -- 172.21.15.46:0/3553960841 reap_dead start 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 -- 172.21.15.46:0/3553960841 shutdown_connections 2024-04-26T23:42:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 -- 172.21.15.46:0/3553960841 >> 172.21.15.46:0/3553960841 conn(0x7fc3e00d2df0 msgr2=0x7fc3e00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 -- 172.21.15.46:0/3553960841 shutdown_connections 2024-04-26T23:42:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.655+0000 7fc3e4a00640 1 -- 172.21.15.46:0/3553960841 wait complete. 2024-04-26T23:42:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.657+0000 7fc3e4a00640 1 Processor -- start 2024-04-26T23:42:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.657+0000 7fc3e4a00640 1 -- start start 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 0x7fc3e0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e00c8260 0x7fc3e00cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00ce060 con 0x7fc3e00cca20 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00cdd80 con 0x7fc3e00c8260 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e00d1fc0 con 0x7fc3c40039b0 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39046/0 (socket says 172.21.15.46:39046) 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 -- 172.21.15.46:0/2288187395 learned_addr learned my addr 172.21.15.46:0/2288187395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3de800640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 0x7fc3e0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 -- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 msgr2=0x7fc3e0154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 0x7fc3e0154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 -- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e00c8260 msgr2=0x7fc3e00cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e00c8260 0x7fc3e00cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 -- 172.21.15.46:0/2288187395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e00c71c0 con 0x7fc3e00cca20 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3de800640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 0x7fc3e0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3dde00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e820 tx=0x7fc3c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800c870 con 0x7fc3e00cca20 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c8018070 con 0x7fc3e00cca20 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e00c74a0 con 0x7fc3e00cca20 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800cc50 con 0x7fc3e00cca20 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.659+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e00c7a60 con 0x7fc3e00cca20 2024-04-26T23:42:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.661+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e00693b0 con 0x7fc3e00cca20 2024-04-26T23:42:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.662+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3c801d460 con 0x7fc3e00cca20 2024-04-26T23:42:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.663+0000 7fc3d6a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 0x7fc3bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.663+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7fc3c808b920 con 0x7fc3e00cca20 2024-04-26T23:42:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.663+0000 7fc3de800640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 0x7fc3bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.665+0000 7fc3de800640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 0x7fc3bc06ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40057e0 tx=0x7fc3d0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:56.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.667+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3c8059950 con 0x7fc3e00cca20 2024-04-26T23:42:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:56.878+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3e00708c0 con 0x7fc3e00cca20 2024-04-26T23:42:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.557+0000 7fc3d6a00640 1 -- 172.21.15.46:0/2288187395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) ==== 138+0+0 (secure 0 0 0) 0x7fc3c805dc50 con 0x7fc3e00cca20 2024-04-26T23:42:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:42:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.564+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 msgr2=0x7fc3bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.564+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 0x7fc3bc06ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40057e0 tx=0x7fc3d0030000 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.564+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 msgr2=0x7fc3e00c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.564+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e820 tx=0x7fc3c800ece0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 shutdown_connections 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3bc06c920 0x7fc3bc06ede0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e00c8260 0x7fc3e00cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3e00cca20 0x7fc3e00c6850 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 --2- 172.21.15.46:0/2288187395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3c40039b0 0x7fc3e0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 >> 172.21.15.46:0/2288187395 conn(0x7fc3e00d2df0 msgr2=0x7fc3e00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:57.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.566+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 shutdown_connections 2024-04-26T23:42:57.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.567+0000 7fc3e4a00640 1 -- 172.21.15.46:0/2288187395 wait complete. 2024-04-26T23:42:57.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:42:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.786+0000 7f594e200640 1 Processor -- start 2024-04-26T23:42:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.786+0000 7f594e200640 1 -- start start 2024-04-26T23:42:57.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f594e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59480c9450 0x7f59480d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f594e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f59480d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f594e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59480c9830 con 0x7f59480d19c0 2024-04-26T23:42:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f594e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f59480c69f0 con 0x7f59480c9450 2024-04-26T23:42:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f594e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59480cec10 con 0x7f59480ce4c0 2024-04-26T23:42:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f5947e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f59480d19c0 0x7f594815caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57278/0 (socket says 172.21.15.46:57278) 2024-04-26T23:42:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.788+0000 7f5947e00640 1 -- 172.21.15.46:0/3938345020 learned_addr learned my addr 172.21.15.46:0/3938345020 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:42:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5947400640 1 --2- 172.21.15.46:0/3938345020 >> v2:172.21.15.142:3300/0 conn(0x7f59480c9450 0x7f59480d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946a00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f59480d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820310415 0 0) 0x7f59480c9830 con 0x7f59480d19c0 2024-04-26T23:42:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f592c003520 con 0x7f59480d19c0 2024-04-26T23:42:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702069363 0 0) 0x7f592c003520 con 0x7f59480d19c0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 msgr2=0x7f59480d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f59480d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 >> v2:172.21.15.142:3300/0 conn(0x7f59480c9450 msgr2=0x7f59480d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 --2- 172.21.15.46:0/3938345020 >> v2:172.21.15.142:3300/0 conn(0x7f59480c9450 0x7f59480d39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.789+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59480c6c40 con 0x7f59480d19c0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f59400019f0 con 0x7f59480d19c0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 >> v1:172.21.15.46:6789/0 conn(0x7f59480d19c0 legacy=0x7f594815caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f592c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c003f80 0x7f592c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c004d40 0x7f592c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59480c69f0 con 0x7f592c004d40 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59480cec10 con 0x7f592c003f80 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f592c00ba70 con 0x7f59480ce4c0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5947e00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c004d40 0x7f592c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5947400640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f592c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.790+0000 7f5946a00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c003f80 0x7f592c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947e00640 1 -- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 msgr2=0x7f592c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947e00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f592c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947e00640 1 -- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c003f80 msgr2=0x7f592c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947e00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c003f80 0x7f592c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947e00640 1 -- 172.21.15.46:0/3938345020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c003520 con 0x7f592c004d40 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5947400640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59480ce4c0 0x7f592c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:42:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.791+0000 7f5946a00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c003f80 0x7f592c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:57.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.792+0000 7f5947e00640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c004d40 0x7f592c0051c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f594000a7d0 tx=0x7f592c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6215ff911c24bfc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:57.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.792+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594000a3f0 con 0x7f592c004d40 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.792+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594002f070 con 0x7f592c004d40 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.792+0000 7f5946000640 1 -- 172.21.15.46:0/3938345020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594002bc90 con 0x7f592c004d40 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 -- 172.21.15.46:0/3938345020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c004d40 msgr2=0x7f592c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 --2- 172.21.15.46:0/3938345020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c004d40 0x7f592c0051c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f594000a7d0 tx=0x7f592c005830 comp rx=0 tx=0).stop 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f5947e00640 1 -- 172.21.15.46:0/3938345020 reap_dead start 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 -- 172.21.15.46:0/3938345020 shutdown_connections 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 -- 172.21.15.46:0/3938345020 >> 172.21.15.46:0/3938345020 conn(0x7f59480d2ea0 msgr2=0x7f59480d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 -- 172.21.15.46:0/3938345020 shutdown_connections 2024-04-26T23:42:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.793+0000 7f594e200640 1 -- 172.21.15.46:0/3938345020 wait complete. 2024-04-26T23:42:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.795+0000 7f594e200640 1 Processor -- start 2024-04-26T23:42:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.795+0000 7f594e200640 1 -- start start 2024-04-26T23:42:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c004d40 0x7f594809fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f594806deb0 0x7f594815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59480ce920 con 0x7f594809d970 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59480c7200 con 0x7f592c004d40 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f594e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59480d1480 con 0x7f594806deb0 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f5947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f5947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39078/0 (socket says 172.21.15.46:39078) 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.796+0000 7f5947e00640 1 -- 172.21.15.46:0/3190062509 learned_addr learned my addr 172.21.15.46:0/3190062509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5946a00640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f594806deb0 0x7f594815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947400640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c004d40 0x7f594809fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 -- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f594806deb0 msgr2=0x7f594815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f594806deb0 0x7f594815f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 -- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c004d40 msgr2=0x7f594809fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:57.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c004d40 0x7f594809fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:57.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 -- 172.21.15.46:0/3190062509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594806e390 con 0x7f594809d970 2024-04-26T23:42:57.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f5947e00640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f594000a7d0 tx=0x7f5940001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940045070 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5940034470 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594806e670 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.797+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594000a3f0 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.798+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594806ec30 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.799+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5948069530 con 0x7f594809d970 2024-04-26T23:42:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.800+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5940048070 con 0x7f594809d970 2024-04-26T23:42:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.801+0000 7f593f400640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 0x7f592806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.801+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5619+0+0 (secure 0 0 0) 0x7f59400ab810 con 0x7f594809d970 2024-04-26T23:42:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.802+0000 7f5947400640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 0x7f592806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:57.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.803+0000 7f5947400640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 0x7f592806edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59380099b0 tx=0x7f5938008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:57.806+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5940079bf0 con 0x7f594809d970 2024-04-26T23:42:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.010+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f59480d2c80 con 0x7f594809d970 2024-04-26T23:42:58.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.572+0000 7f593f400640 1 -- 172.21.15.46:0/3190062509 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) ==== 170+0+0 (secure 0 0 0) 0x7f59400794f0 con 0x7f594809d970 2024-04-26T23:42:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 msgr2=0x7f592806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 0x7f592806edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59380099b0 tx=0x7f5938008040 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 msgr2=0x7f594809de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f594000a7d0 tx=0x7f5940001cf0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 shutdown_connections 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f592806c8f0 0x7f592806edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594809d970 0x7f594809de00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f594806deb0 0x7f594815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 --2- 172.21.15.46:0/3190062509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592c004d40 0x7f594809fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.577+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 >> 172.21.15.46:0/3190062509 conn(0x7f59480d2ea0 msgr2=0x7f59480ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.578+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 shutdown_connections 2024-04-26T23:42:58.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.578+0000 7f594e200640 1 -- 172.21.15.46:0/3190062509 wait complete. 2024-04-26T23:42:58.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:42:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.787+0000 7f9fa3200640 1 Processor -- start 2024-04-26T23:42:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.787+0000 7f9fa3200640 1 -- start start 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.788+0000 7f9fa3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.788+0000 7f9fa3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f9c0ae8f0 0x7f9f9c0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.788+0000 7f9fa3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9c0af570 con 0x7f9f9c15f210 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.788+0000 7f9fa3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0af850 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.788+0000 7f9fa3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0cb680 con 0x7f9f9c0ae8f0 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39090/0 (socket says 172.21.15.46:39090) 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 -- 172.21.15.46:0/2568037233 learned_addr learned my addr 172.21.15.46:0/2568037233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1e00640 1 --1- 172.21.15.46:0/2568037233 >> v1:172.21.15.46:6789/0 conn(0x7f9f9c15f210 0x7f9f9c0cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57290/0 (socket says 172.21.15.46:57290) 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa2800640 1 --2- 172.21.15.46:0/2568037233 >> v2:172.21.15.142:3300/0 conn(0x7f9f9c0ae8f0 0x7f9f9c0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 -- 172.21.15.46:0/2568037233 >> v2:172.21.15.142:3300/0 conn(0x7f9f9c0ae8f0 msgr2=0x7f9f9c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 --2- 172.21.15.46:0/2568037233 >> v2:172.21.15.142:3300/0 conn(0x7f9f9c0ae8f0 0x7f9f9c0aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 -- 172.21.15.46:0/2568037233 >> v1:172.21.15.46:6789/0 conn(0x7f9f9c15f210 legacy=0x7f9f9c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.789+0000 7f9fa1400640 1 -- 172.21.15.46:0/2568037233 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c0cb960 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.791+0000 7f9fa1400640 1 --2- 172.21.15.46:0/2568037233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0cb1e0 tx=0x7f9f8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e444f5b74b8df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.791+0000 7f9fa0a00640 1 -- 172.21.15.46:0/2568037233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c037070 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.791+0000 7f9fa0a00640 1 -- 172.21.15.46:0/2568037233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c032440 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.791+0000 7f9fa0a00640 1 -- 172.21.15.46:0/2568037233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c032810 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 -- 172.21.15.46:0/2568037233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 msgr2=0x7f9f9c0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2568037233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0cb1e0 tx=0x7f9f8c02a270 comp rx=0 tx=0).stop 2024-04-26T23:42:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 -- 172.21.15.46:0/2568037233 shutdown_connections 2024-04-26T23:42:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2568037233 >> v2:172.21.15.142:3300/0 conn(0x7f9f9c0ae8f0 0x7f9f9c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2568037233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0cc160 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 -- 172.21.15.46:0/2568037233 >> 172.21.15.46:0/2568037233 conn(0x7f9f9c0b27b0 msgr2=0x7f9f9c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 -- 172.21.15.46:0/2568037233 shutdown_connections 2024-04-26T23:42:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.792+0000 7f9fa3200640 1 -- 172.21.15.46:0/2568037233 wait complete. 2024-04-26T23:42:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.794+0000 7f9fa3200640 1 Processor -- start 2024-04-26T23:42:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.794+0000 7f9fa3200640 1 -- start start 2024-04-26T23:42:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f9c0ae8f0 0x7f9f9c150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 0x7f9f9c0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c15f5f0 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c0af240 con 0x7f9f9c15f210 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c071e80 con 0x7f9f9c0ae8f0 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39098/0 (socket says 172.21.15.46:39098) 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 -- 172.21.15.46:0/2874612968 learned_addr learned my addr 172.21.15.46:0/2874612968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 -- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f9c0ae8f0 msgr2=0x7f9f9c150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1e00640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f9c0ae8f0 0x7f9f9c150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f9c0ae8f0 0x7f9f9c150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 -- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 msgr2=0x7f9f9c0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa2800640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 0x7f9f9c0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 0x7f9f9c0b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.795+0000 7f9fa1400640 1 -- 172.21.15.46:0/2874612968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c0b3940 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9fa2800640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 0x7f9f9c0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9fa1400640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c02a2b0 tx=0x7f9f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c041020 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0306c0 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c037040 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c0b3c20 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.796+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c0b41e0 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.797+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9c0693b0 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.798+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f8c044020 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.799+0000 7f9f96000640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 0x7f9f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.799+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6001+0+0 (secure 0 0 0) 0x7f9f8c0ac550 con 0x7f9f9c0cdd20 2024-04-26T23:42:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.800+0000 7f9fa1e00640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 0x7f9f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.801+0000 7f9fa1e00640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 0x7f9f8006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c1619a0 tx=0x7f9f98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:58.805+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f8c04b030 con 0x7f9f9c0cdd20 2024-04-26T23:42:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.048+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9f9c0cc590 con 0x7f9f9c0cdd20 2024-04-26T23:42:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.576+0000 7f9f96000640 1 -- 172.21.15.46:0/2874612968 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v37) ==== 173+0+0 (secure 0 0 0) 0x7f9f8c07a7b0 con 0x7f9f9c0cdd20 2024-04-26T23:42:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 9 and data pool 10 2024-04-26T23:42:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.581+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 msgr2=0x7f9f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.581+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 0x7f9f8006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c1619a0 tx=0x7f9f98009040 comp rx=0 tx=0).stop 2024-04-26T23:42:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.581+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 msgr2=0x7f9f9c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.581+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c02a2b0 tx=0x7f9f8c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 shutdown_connections 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f8006c8f0 0x7f9f8006edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f9c15f210 0x7f9f9c0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f9c0cdd20 0x7f9f9c0b5560 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 --2- 172.21.15.46:0/2874612968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f9c0ae8f0 0x7f9f9c150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.583+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 >> 172.21.15.46:0/2874612968 conn(0x7f9f9c0b27b0 msgr2=0x7f9f9c0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.584+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 shutdown_connections 2024-04-26T23:42:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.585+0000 7f9fa3200640 1 -- 172.21.15.46:0/2874612968 wait complete. 2024-04-26T23:42:59.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:42:59.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.795+0000 7f2108e00640 1 Processor -- start 2024-04-26T23:42:59.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.795+0000 7f2108e00640 1 -- start start 2024-04-26T23:42:59.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2108e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2104162a60 0x7f21040d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2108e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21040d5a90 0x7f21040b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2108e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21040b65e0 con 0x7f21040b4c50 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2108e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f21040d12e0 con 0x7f21040d5a90 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2108e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21040d05f0 con 0x7f2104162a60 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.796+0000 7f2102a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21040d5a90 0x7f21040b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21040d5a90 0x7f21040b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39108/0 (socket says 172.21.15.46:39108) 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 -- 172.21.15.46:0/4065274260 learned_addr learned my addr 172.21.15.46:0/4065274260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 -- 172.21.15.46:0/4065274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2104162a60 msgr2=0x7f21040d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 --2- 172.21.15.46:0/4065274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2104162a60 0x7f21040d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 -- 172.21.15.46:0/4065274260 >> v1:172.21.15.142:6789/0 conn(0x7f21040b4c50 legacy=0x7f21040d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:42:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.797+0000 7f2102a00640 1 -- 172.21.15.46:0/4065274260 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21040d1530 con 0x7f21040d5a90 2024-04-26T23:42:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.798+0000 7f2102a00640 1 --2- 172.21.15.46:0/4065274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21040d5a90 0x7f21040b4710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009870 tx=0x7f20f002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7e0c92a6e25f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.798+0000 7f2102000640 1 -- 172.21.15.46:0/4065274260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0037070 con 0x7f21040d5a90 2024-04-26T23:42:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.798+0000 7f2102000640 1 -- 172.21.15.46:0/4065274260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f002ae00 con 0x7f21040d5a90 2024-04-26T23:42:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.798+0000 7f2102000640 1 -- 172.21.15.46:0/4065274260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0032660 con 0x7f21040d5a90 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 -- 172.21.15.46:0/4065274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21040d5a90 msgr2=0x7f21040b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 --2- 172.21.15.46:0/4065274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21040d5a90 0x7f21040b4710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009870 tx=0x7f20f002a0f0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 -- 172.21.15.46:0/4065274260 shutdown_connections 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 --2- 172.21.15.46:0/4065274260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21040d5a90 0x7f21040b4710 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 --2- 172.21.15.46:0/4065274260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2104162a60 0x7f21040d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 -- 172.21.15.46:0/4065274260 >> 172.21.15.46:0/4065274260 conn(0x7f21040d4fa0 msgr2=0x7f21040d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 -- 172.21.15.46:0/4065274260 shutdown_connections 2024-04-26T23:42:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.799+0000 7f2108e00640 1 -- 172.21.15.46:0/4065274260 wait complete. 2024-04-26T23:42:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.801+0000 7f2108e00640 1 Processor -- start 2024-04-26T23:42:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.801+0000 7f2108e00640 1 -- start start 2024-04-26T23:42:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040b4c50 0x7f2104151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040d5a90 0x7f21041e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21040b5280 con 0x7f2104162a60 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21040b5100 con 0x7f21040b4c50 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2108e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21040b5400 con 0x7f21040d5a90 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2103e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2103e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39112/0 (socket says 172.21.15.46:39112) 2024-04-26T23:42:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.802+0000 7f2103e00640 1 -- 172.21.15.46:0/2630020727 learned_addr learned my addr 172.21.15.46:0/2630020727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:42:59.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 -- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040d5a90 msgr2=0x7f21041e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040d5a90 0x7f21041e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 -- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040b4c50 msgr2=0x7f2104151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040b4c50 0x7f2104151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 -- 172.21.15.46:0/2630020727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21040c8c70 con 0x7f2104162a60 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f2103e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b440 tx=0x7f20fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc015020 con 0x7f2104162a60 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc011830 con 0x7f2104162a60 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.803+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc011c20 con 0x7f2104162a60 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.804+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21040c8f50 con 0x7f2104162a60 2024-04-26T23:42:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.804+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21040c9510 con 0x7f2104162a60 2024-04-26T23:42:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.805+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2104069ac0 con 0x7f2104162a60 2024-04-26T23:42:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.806+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20fc018020 con 0x7f2104162a60 2024-04-26T23:42:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.807+0000 7f20fb400640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 0x7f20e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.807+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f20fc08b820 con 0x7f2104162a60 2024-04-26T23:42:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.808+0000 7f2103400640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 0x7f20e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.809+0000 7f2103400640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 0x7f20e406edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f20f400ab90 tx=0x7f20f4009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:42:59.812+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20fc01f030 con 0x7f2104162a60 2024-04-26T23:43:00.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.013+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21040d02b0 con 0x7f2104162a60 2024-04-26T23:43:00.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.016+0000 7f20fb400640 1 -- 172.21.15.46:0/2630020727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16861 (secure 0 0 0) 0x7f20fc0172c0 con 0x7f2104162a60 2024-04-26T23:43:00.019 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:00.020 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:42:59.049860+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.018+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 msgr2=0x7f20e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.018+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 0x7f20e406edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f20f400ab90 tx=0x7f20f4009250 comp rx=0 tx=0).stop 2024-04-26T23:43:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.018+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 msgr2=0x7f21041e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.018+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b440 tx=0x7f20fc00b900 comp rx=0 tx=0).stop 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 shutdown_connections 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f20e406c8f0 0x7f20e406edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2104162a60 0x7f21041e9360 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040d5a90 0x7f21041e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 --2- 172.21.15.46:0/2630020727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040b4c50 0x7f2104151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 >> 172.21.15.46:0/2630020727 conn(0x7f21040d4fa0 msgr2=0x7f2104161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 shutdown_connections 2024-04-26T23:43:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.019+0000 7f2108e00640 1 -- 172.21.15.46:0/2630020727 wait complete. 2024-04-26T23:43:00.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.247+0000 7f2378a00640 1 Processor -- start 2024-04-26T23:43:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.247+0000 7f2378a00640 1 -- start start 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.248+0000 7f2378a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23740720e0 0x7f23740cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2378a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2374073ad0 con 0x7f2374160910 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2378a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2374146a90 con 0x7f23741466b0 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2378a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2374073d70 con 0x7f23740720e0 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23740720e0 0x7f23740cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23740720e0 0x7f23740cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39120/0 (socket says 172.21.15.46:39120) 2024-04-26T23:43:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 -- 172.21.15.46:0/2437567436 learned_addr learned my addr 172.21.15.46:0/2437567436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 -- 172.21.15.46:0/2437567436 >> v1:172.21.15.142:6789/0 conn(0x7f23741466b0 legacy=0x7f237406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 -- 172.21.15.46:0/2437567436 >> v1:172.21.15.46:6789/0 conn(0x7f2374160910 legacy=0x7f23740cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.249+0000 7f2372a00640 1 -- 172.21.15.46:0/2437567436 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2374074050 con 0x7f23740720e0 2024-04-26T23:43:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2372a00640 1 --2- 172.21.15.46:0/2437567436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740720e0 0x7f23740cd860 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f23600098a0 tx=0x7f236002a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ac243d044effb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2372000640 1 -- 172.21.15.46:0/2437567436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360037070 con 0x7f23740720e0 2024-04-26T23:43:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2372000640 1 -- 172.21.15.46:0/2437567436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2360032440 con 0x7f23740720e0 2024-04-26T23:43:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2372000640 1 -- 172.21.15.46:0/2437567436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360032810 con 0x7f23740720e0 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2378a00640 1 -- 172.21.15.46:0/2437567436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740720e0 msgr2=0x7f23740cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.251+0000 7f2378a00640 1 --2- 172.21.15.46:0/2437567436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740720e0 0x7f23740cd860 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f23600098a0 tx=0x7f236002a2e0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.252+0000 7f2378a00640 1 -- 172.21.15.46:0/2437567436 shutdown_connections 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.252+0000 7f2378a00640 1 --2- 172.21.15.46:0/2437567436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740720e0 0x7f23740cd860 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.252+0000 7f2378a00640 1 -- 172.21.15.46:0/2437567436 >> 172.21.15.46:0/2437567436 conn(0x7f23740b26d0 msgr2=0x7f23740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.252+0000 7f2378a00640 1 -- 172.21.15.46:0/2437567436 shutdown_connections 2024-04-26T23:43:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.252+0000 7f2378a00640 1 -- 172.21.15.46:0/2437567436 wait complete. 2024-04-26T23:43:00.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.253+0000 7f2378a00640 1 Processor -- start 2024-04-26T23:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.254+0000 7f2378a00640 1 -- start start 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.254+0000 7f2378a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23740720e0 0x7f23740aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2378a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2378a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2374160910 0x7f23741f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2378a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23740cdda0 con 0x7f23741466b0 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2378a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f23740cb470 con 0x7f23740720e0 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2378a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23740cc0a0 con 0x7f2374160910 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39132/0 (socket says 172.21.15.46:39132) 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 -- 172.21.15.46:0/1915718720 learned_addr learned my addr 172.21.15.46:0/1915718720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2373400640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23740720e0 0x7f23740aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 -- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2374160910 msgr2=0x7f23741f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2374160910 0x7f23741f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 -- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23740720e0 msgr2=0x7f23740aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23740720e0 0x7f23740aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.255+0000 7f2372a00640 1 -- 172.21.15.46:0/1915718720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237415fde0 con 0x7f23741466b0 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f2372a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2360000c00 tx=0x7f23600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360037040 con 0x7f23741466b0 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2360004230 con 0x7f23741466b0 2024-04-26T23:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360003610 con 0x7f23741466b0 2024-04-26T23:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23740c8d60 con 0x7f23741466b0 2024-04-26T23:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.256+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23740c9320 con 0x7f23741466b0 2024-04-26T23:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.257+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23740693b0 con 0x7f23741466b0 2024-04-26T23:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.258+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23600037b0 con 0x7f23741466b0 2024-04-26T23:43:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.259+0000 7f236b400640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 0x7f235406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.259+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f23600ac9b0 con 0x7f23741466b0 2024-04-26T23:43:00.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.260+0000 7f2373400640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 0x7f235406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.261+0000 7f2373400640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 0x7f235406ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2364003d90 tx=0x7f2364009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.264+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f236007ac40 con 0x7f23741466b0 2024-04-26T23:43:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.465+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23740c7180 con 0x7f23741466b0 2024-04-26T23:43:00.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.468+0000 7f236b400640 1 -- 172.21.15.46:0/1915718720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16861 (secure 0 0 0) 0x7f236007a540 con 0x7f23741466b0 2024-04-26T23:43:00.472 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:00.472 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:42:59.049860+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 msgr2=0x7f235406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 0x7f235406ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2364003d90 tx=0x7f2364009040 comp rx=0 tx=0).stop 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 msgr2=0x7f23740b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2360000c00 tx=0x7f23600046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 shutdown_connections 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f235406c920 0x7f235406ede0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2374160910 0x7f23741f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23741466b0 0x7f23740b5dc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 --2- 172.21.15.46:0/1915718720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23740720e0 0x7f23740aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.471+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 >> 172.21.15.46:0/1915718720 conn(0x7f23740b26d0 msgr2=0x7f23740726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.472+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 shutdown_connections 2024-04-26T23:43:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.472+0000 7f2378a00640 1 -- 172.21.15.46:0/1915718720 wait complete. 2024-04-26T23:43:00.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:43:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.708+0000 7f409f000640 1 Processor -- start 2024-04-26T23:43:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.708+0000 7f409f000640 1 -- start start 2024-04-26T23:43:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4098160130 0x7f4098160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40980cdc60 0x7f4098072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40980cbe60 con 0x7f409815cc40 2024-04-26T23:43:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f40980cca70 con 0x7f4098160130 2024-04-26T23:43:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f40980cc140 con 0x7f40980cdc60 2024-04-26T23:43:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4098160130 0x7f4098160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f409815cc40 0x7f40980cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57316/0 (socket says 172.21.15.46:57316) 2024-04-26T23:43:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.710+0000 7f409d200640 1 -- 172.21.15.46:0/1885730258 learned_addr learned my addr 172.21.15.46:0/1885730258 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f409e600640 1 --2- 172.21.15.46:0/1885730258 >> v2:172.21.15.142:3300/0 conn(0x7f40980cdc60 0x7f4098072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f409dc00640 1 -- 172.21.15.46:0/1885730258 >> v2:172.21.15.142:3300/0 conn(0x7f40980cdc60 msgr2=0x7f4098072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f409dc00640 1 --2- 172.21.15.46:0/1885730258 >> v2:172.21.15.142:3300/0 conn(0x7f40980cdc60 0x7f4098072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f409dc00640 1 -- 172.21.15.46:0/1885730258 >> v1:172.21.15.46:6789/0 conn(0x7f409815cc40 legacy=0x7f40980cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f409dc00640 1 -- 172.21.15.46:0/1885730258 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40980cc420 con 0x7f4098160130 2024-04-26T23:43:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.711+0000 7f4097e00640 1 -- 172.21.15.46:0/1885730258 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302939988 0 0) 0x7f40980cbe60 con 0x7f409815cc40 2024-04-26T23:43:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.712+0000 7f409dc00640 1 --2- 172.21.15.46:0/1885730258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4098160130 0x7f4098160510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f40980c46d0 tx=0x7f408c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eff7ad791d106f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.712+0000 7f4097e00640 1 -- 172.21.15.46:0/1885730258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c007740 con 0x7f4098160130 2024-04-26T23:43:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.712+0000 7f4097e00640 1 -- 172.21.15.46:0/1885730258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c0078e0 con 0x7f4098160130 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.712+0000 7f4097e00640 1 -- 172.21.15.46:0/1885730258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c007cb0 con 0x7f4098160130 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 -- 172.21.15.46:0/1885730258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4098160130 msgr2=0x7f4098160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 --2- 172.21.15.46:0/1885730258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4098160130 0x7f4098160510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f40980c46d0 tx=0x7f408c02a5f0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 -- 172.21.15.46:0/1885730258 shutdown_connections 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 --2- 172.21.15.46:0/1885730258 >> v2:172.21.15.142:3300/0 conn(0x7f40980cdc60 0x7f4098072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 --2- 172.21.15.46:0/1885730258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4098160130 0x7f4098160510 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 -- 172.21.15.46:0/1885730258 >> 172.21.15.46:0/1885730258 conn(0x7f40980b27b0 msgr2=0x7f40980b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 -- 172.21.15.46:0/1885730258 shutdown_connections 2024-04-26T23:43:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.713+0000 7f409f000640 1 -- 172.21.15.46:0/1885730258 wait complete. 2024-04-26T23:43:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.715+0000 7f409f000640 1 Processor -- start 2024-04-26T23:43:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.715+0000 7f409f000640 1 -- start start 2024-04-26T23:43:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 0x7f40981f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4098160130 0x7f409815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f409815d220 con 0x7f40980cdc60 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f409815d0a0 con 0x7f409815cc40 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4098072760 con 0x7f4098160130 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39152/0 (socket says 172.21.15.46:39152) 2024-04-26T23:43:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.716+0000 7f409dc00640 1 -- 172.21.15.46:0/3940489265 learned_addr learned my addr 172.21.15.46:0/3940489265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 -- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4098160130 msgr2=0x7f409815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4098160130 0x7f409815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 -- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 msgr2=0x7f40981f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409d200640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 0x7f40981f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 0x7f40981f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 -- 172.21.15.46:0/3940489265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40981f8db0 con 0x7f40980cdc60 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409d200640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 0x7f40981f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409dc00640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f408c009e80 tx=0x7f408c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c03f070 con 0x7f40980cdc60 2024-04-26T23:43:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c031c40 con 0x7f40980cdc60 2024-04-26T23:43:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c044460 con 0x7f40980cdc60 2024-04-26T23:43:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40981f9090 con 0x7f40980cdc60 2024-04-26T23:43:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.717+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40980c6870 con 0x7f40980cdc60 2024-04-26T23:43:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.719+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40980693b0 con 0x7f40980cdc60 2024-04-26T23:43:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.720+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f408c03b410 con 0x7f40980cdc60 2024-04-26T23:43:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.720+0000 7f4095600640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 0x7f407806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.721+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f408c0ad5b0 con 0x7f40980cdc60 2024-04-26T23:43:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.721+0000 7f409d200640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 0x7f407806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.722+0000 7f409d200640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 0x7f407806edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f40980c7070 tx=0x7f4084009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.725+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f408c07b8e0 con 0x7f40980cdc60 2024-04-26T23:43:00.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:00.972+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f40980cbb10 con 0x7f40980cdc60 2024-04-26T23:43:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.621+0000 7f4095600640 1 -- 172.21.15.46:0/3940489265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) ==== 118+0+0 (secure 0 0 0) 0x7f408c07b1e0 con 0x7f40980cdc60 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.625+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 msgr2=0x7f407806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.625+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 0x7f407806edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f40980c7070 tx=0x7f4084009040 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.625+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 msgr2=0x7f40980c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.625+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f408c009e80 tx=0x7f408c004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 shutdown_connections 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f407806c8f0 0x7f407806edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4098160130 0x7f409815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f409815cc40 0x7f40981f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 --2- 172.21.15.46:0/3940489265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40980cdc60 0x7f40980c6330 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 >> 172.21.15.46:0/3940489265 conn(0x7f40980b27b0 msgr2=0x7f409815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:01.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 shutdown_connections 2024-04-26T23:43:01.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.626+0000 7f409f000640 1 -- 172.21.15.46:0/3940489265 wait complete. 2024-04-26T23:43:01.652 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:43:01.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:43:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.867+0000 7f69e5e00640 1 Processor -- start 2024-04-26T23:43:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.867+0000 7f69e5e00640 1 -- start start 2024-04-26T23:43:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69e5e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69e00d04c0 0x7f69e006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69e5e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00b4b30 con 0x7f69e00b44f0 2024-04-26T23:43:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69e5e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00b4e10 con 0x7f69e006f2e0 2024-04-26T23:43:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69e5e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f69e00b48f0 con 0x7f69e00d04c0 2024-04-26T23:43:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69df400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f69e00b44f0 0x7f69e00ce470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45216/0 (socket says 172.21.15.46:45216) 2024-04-26T23:43:01.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.869+0000 7f69df400640 1 -- 172.21.15.46:0/2877560608 learned_addr learned my addr 172.21.15.46:0/2877560608 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:01.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.870+0000 7f69dea00640 1 --2- 172.21.15.46:0/2877560608 >> v2:172.21.15.142:3300/0 conn(0x7f69e00d04c0 0x7f69e006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:01.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.870+0000 7f69dd600640 1 -- 172.21.15.46:0/2877560608 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198804325 0 0) 0x7f69e00b4b30 con 0x7f69e00b44f0 2024-04-26T23:43:01.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.870+0000 7f69dd600640 1 -- 172.21.15.46:0/2877560608 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c4003520 con 0x7f69e00b44f0 2024-04-26T23:43:01.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.870+0000 7f69dea00640 1 -- 172.21.15.46:0/2877560608 >> v1:172.21.15.142:6789/0 conn(0x7f69e006f2e0 legacy=0x7f69e00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.871+0000 7f69dea00640 1 -- 172.21.15.46:0/2877560608 >> v1:172.21.15.46:6789/0 conn(0x7f69e00b44f0 legacy=0x7f69e00ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.871+0000 7f69dea00640 1 -- 172.21.15.46:0/2877560608 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00b50f0 con 0x7f69e00d04c0 2024-04-26T23:43:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.872+0000 7f69dea00640 1 --2- 172.21.15.46:0/2877560608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00d04c0 0x7f69e006eda0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69d00097a0 tx=0x7f69d002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b00b1b4a9246144 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.872+0000 7f69dd600640 1 -- 172.21.15.46:0/2877560608 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0037070 con 0x7f69e00d04c0 2024-04-26T23:43:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.872+0000 7f69dd600640 1 -- 172.21.15.46:0/2877560608 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d0032440 con 0x7f69e00d04c0 2024-04-26T23:43:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.872+0000 7f69dd600640 1 -- 172.21.15.46:0/2877560608 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0032810 con 0x7f69e00d04c0 2024-04-26T23:43:01.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 -- 172.21.15.46:0/2877560608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00d04c0 msgr2=0x7f69e006eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2877560608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00d04c0 0x7f69e006eda0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69d00097a0 tx=0x7f69d002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 -- 172.21.15.46:0/2877560608 shutdown_connections 2024-04-26T23:43:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2877560608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00d04c0 0x7f69e006eda0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 -- 172.21.15.46:0/2877560608 >> 172.21.15.46:0/2877560608 conn(0x7f69e006c980 msgr2=0x7f69e00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 -- 172.21.15.46:0/2877560608 shutdown_connections 2024-04-26T23:43:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.873+0000 7f69e5e00640 1 -- 172.21.15.46:0/2877560608 wait complete. 2024-04-26T23:43:01.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.876+0000 7f69e5e00640 1 Processor -- start 2024-04-26T23:43:01.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.876+0000 7f69e5e00640 1 -- start start 2024-04-26T23:43:01.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:01.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00b44f0 0x7f69e013f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:01.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 0x7f69e00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00ceb80 con 0x7f69e006f2e0 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00b56b0 con 0x7f69e00b44f0 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69e5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f69e00ced00 con 0x7f69e00d04c0 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36722/0 (socket says 172.21.15.46:36722) 2024-04-26T23:43:01.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.877+0000 7f69dea00640 1 -- 172.21.15.46:0/2312323951 learned_addr learned my addr 172.21.15.46:0/2312323951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69df400640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 0x7f69e00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 -- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 msgr2=0x7f69e00b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 0x7f69e00b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 -- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00b44f0 msgr2=0x7f69e013f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00b44f0 0x7f69e013f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 -- 172.21.15.46:0/2312323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00b0580 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69df400640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 0x7f69e00b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69dea00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f69d0035040 tx=0x7f69d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.878+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0037040 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.879+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d0030500 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.879+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e00ab9d0 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.879+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0030910 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.879+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e00abf90 con 0x7f69e006f2e0 2024-04-26T23:43:01.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.880+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e0069530 con 0x7f69e006f2e0 2024-04-26T23:43:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.881+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69d0044020 con 0x7f69e006f2e0 2024-04-26T23:43:01.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.882+0000 7f69d7400640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:01.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.882+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f69d00ac010 con 0x7f69e006f2e0 2024-04-26T23:43:01.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.883+0000 7f69de000640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:01.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.884+0000 7f69de000640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 0x7f69c006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc0079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:01.886+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69d007a220 con 0x7f69e006f2e0 2024-04-26T23:43:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.128+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f69e006eed0 con 0x7f69e006f2e0 2024-04-26T23:43:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.634+0000 7f69d7400640 1 -- 172.21.15.46:0/2312323951 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f69d0079b20 con 0x7f69e006f2e0 2024-04-26T23:43:02.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 msgr2=0x7f69c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 0x7f69c006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc0079d0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 msgr2=0x7f69e00ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f69d0035040 tx=0x7f69d00046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 shutdown_connections 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69e00d04c0 0x7f69e00b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e00b44f0 0x7f69e013f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 --2- 172.21.15.46:0/2312323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69e006f2e0 0x7f69e00ab380 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 >> 172.21.15.46:0/2312323951 conn(0x7f69e006c980 msgr2=0x7f69e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 shutdown_connections 2024-04-26T23:43:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.638+0000 7f69e5e00640 1 -- 172.21.15.46:0/2312323951 wait complete. 2024-04-26T23:43:02.661 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:43:02.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.870+0000 7f8015c00640 1 Processor -- start 2024-04-26T23:43:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.871+0000 7f8015c00640 1 -- start start 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.872+0000 7f8015c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80100d1520 0x7f80100d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.872+0000 7f8015c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f80100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.872+0000 7f8015c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d1e40 con 0x7f801015d510 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.872+0000 7f8015c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f80100d1220 con 0x7f80100d1520 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.872+0000 7f8015c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80100ce560 con 0x7f80100d3af0 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80100d1520 0x7f80100d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80100d1520 0x7f80100d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36730/0 (socket says 172.21.15.46:36730) 2024-04-26T23:43:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 -- 172.21.15.46:0/2964463382 learned_addr learned my addr 172.21.15.46:0/2964463382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800f400640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f80100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 -- 172.21.15.46:0/2964463382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 msgr2=0x7f80100706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f80100706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 -- 172.21.15.46:0/2964463382 >> v1:172.21.15.142:6789/0 conn(0x7f801015d510 legacy=0x7f80100d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800ea00640 1 -- 172.21.15.46:0/2964463382 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80100ce7b0 con 0x7f80100d1520 2024-04-26T23:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.873+0000 7f800f400640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f80100706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.874+0000 7f800ea00640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100d1900 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8000009870 tx=0x7f800002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7439a23d1aa697 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.875+0000 7f800d600640 1 -- 172.21.15.46:0/2964463382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000037070 con 0x7f80100d1520 2024-04-26T23:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.875+0000 7f800d600640 1 -- 172.21.15.46:0/2964463382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000032440 con 0x7f80100d1520 2024-04-26T23:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.875+0000 7f800d600640 1 -- 172.21.15.46:0/2964463382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000032810 con 0x7f80100d1520 2024-04-26T23:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 -- 172.21.15.46:0/2964463382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 msgr2=0x7f80100d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100d1900 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8000009870 tx=0x7f800002a200 comp rx=0 tx=0).stop 2024-04-26T23:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 -- 172.21.15.46:0/2964463382 shutdown_connections 2024-04-26T23:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f80100706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 --2- 172.21.15.46:0/2964463382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100d1900 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 -- 172.21.15.46:0/2964463382 >> 172.21.15.46:0/2964463382 conn(0x7f80100d2f40 msgr2=0x7f80100d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 -- 172.21.15.46:0/2964463382 shutdown_connections 2024-04-26T23:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.876+0000 7f8015c00640 1 -- 172.21.15.46:0/2964463382 wait complete. 2024-04-26T23:43:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.878+0000 7f8015c00640 1 Processor -- start 2024-04-26T23:43:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.878+0000 7f8015c00640 1 -- start start 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f8010155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801015d510 0x7f80100b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80100ced70 con 0x7f80100d1520 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8010070230 con 0x7f801015d510 2024-04-26T23:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f8015c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8010070c00 con 0x7f80100d3af0 2024-04-26T23:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36734/0 (socket says 172.21.15.46:36734) 2024-04-26T23:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 -- 172.21.15.46:0/4013789908 learned_addr learned my addr 172.21.15.46:0/4013789908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800e000640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f8010155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 -- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 msgr2=0x7f8010155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f8010155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 -- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801015d510 msgr2=0x7f80100b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801015d510 0x7f80100b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.879+0000 7f800ea00640 1 -- 172.21.15.46:0/4013789908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801015c840 con 0x7f80100d1520 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f800ea00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8000009870 tx=0x7f80000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000041020 con 0x7f80100d1520 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000030800 con 0x7f80100d1520 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000037040 con 0x7f80100d1520 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80100b5870 con 0x7f80100d1520 2024-04-26T23:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.880+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80100cb3e0 con 0x7f80100d1520 2024-04-26T23:43:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.882+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010069530 con 0x7f80100d1520 2024-04-26T23:43:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.883+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8000044020 con 0x7f80100d1520 2024-04-26T23:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.884+0000 7f8006a00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.884+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f80000ac670 con 0x7f80100d1520 2024-04-26T23:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.885+0000 7f800e000640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.886+0000 7f800e000640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 0x7f7ff006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006fd0 tx=0x7f7ffc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:02.889+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f800004b030 con 0x7f80100d1520 2024-04-26T23:43:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.135+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80100cf470 con 0x7f80100d1520 2024-04-26T23:43:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.137+0000 7f8006a00640 1 -- 172.21.15.46:0/4013789908 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5188 (secure 0 0 0) 0x7f801015c840 con 0x7f80100d1520 2024-04-26T23:43:03.140 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:03.140 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3235774034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3235774034},{"type":"v1","addr":"172.21.15.142:6837","nonce":3235774034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6579,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3040630531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3040630531},{"type":"v1","addr":"172.21.15.46:6833","nonce":3040630531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6582,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1818830260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1818830260},{"type":"v1","addr":"172.21.15.46:6835","nonce":1818830260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:02.129282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2927770154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2927770154},{"type":"v1","addr":"172.21.15.142:6835","nonce":2927770154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":4}]} 2024-04-26T23:43:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 41 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.139+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 msgr2=0x7f7ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.139+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 0x7f7ff006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006fd0 tx=0x7f7ffc008040 comp rx=0 tx=0).stop 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.139+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 msgr2=0x7f80100b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.139+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8000009870 tx=0x7f80000046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 shutdown_connections 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801015d510 0x7f80100b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100d3af0 0x7f8010155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 --2- 172.21.15.46:0/4013789908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80100d1520 0x7f80100b5290 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 >> 172.21.15.46:0/4013789908 conn(0x7f80100d2f40 msgr2=0x7f80100afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 shutdown_connections 2024-04-26T23:43:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.140+0000 7f8015c00640 1 -- 172.21.15.46:0/4013789908 wait complete. 2024-04-26T23:43:03.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.372+0000 7f924be00640 1 Processor -- start 2024-04-26T23:43:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.372+0000 7f924be00640 1 -- start start 2024-04-26T23:43:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f924c162a60 0x7f924c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f924c0d5a90 0x7f924c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c0b4c50 0x7f924c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0d12e0 con 0x7f924c162a60 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0d05f0 con 0x7f924c0d5a90 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.373+0000 7f924be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f924c0b65e0 con 0x7f924c0b4c50 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f924c162a60 0x7f924c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f924c162a60 0x7f924c0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36736/0 (socket says 172.21.15.46:36736) 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 -- 172.21.15.46:0/2124042716 learned_addr learned my addr 172.21.15.46:0/2124042716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f9243e00640 1 --2- 172.21.15.46:0/2124042716 >> v2:172.21.15.142:3300/0 conn(0x7f924c0d5a90 0x7f924c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924b400640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c0b4c50 0x7f924c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 -- 172.21.15.46:0/2124042716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c0b4c50 msgr2=0x7f924c0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c0b4c50 0x7f924c0d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 -- 172.21.15.46:0/2124042716 >> v2:172.21.15.142:3300/0 conn(0x7f924c0d5a90 msgr2=0x7f924c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 --2- 172.21.15.46:0/2124042716 >> v2:172.21.15.142:3300/0 conn(0x7f924c0d5a90 0x7f924c0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.374+0000 7f924aa00640 1 -- 172.21.15.46:0/2124042716 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0d1530 con 0x7f924c162a60 2024-04-26T23:43:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.376+0000 7f924aa00640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c162a60 0x7f924c0d0a80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f924c06f5e0 tx=0x7f923c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532a58d935b65747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.376+0000 7f924a000640 1 -- 172.21.15.46:0/2124042716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c037070 con 0x7f924c162a60 2024-04-26T23:43:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.376+0000 7f924a000640 1 -- 172.21.15.46:0/2124042716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923c02ae00 con 0x7f924c162a60 2024-04-26T23:43:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.376+0000 7f924a000640 1 -- 172.21.15.46:0/2124042716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c032660 con 0x7f924c162a60 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 -- 172.21.15.46:0/2124042716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c162a60 msgr2=0x7f924c0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c162a60 0x7f924c0d0a80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f924c06f5e0 tx=0x7f923c02a0b0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 -- 172.21.15.46:0/2124042716 shutdown_connections 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c0b4c50 0x7f924c0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 --2- 172.21.15.46:0/2124042716 >> v2:172.21.15.142:3300/0 conn(0x7f924c0d5a90 0x7f924c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 --2- 172.21.15.46:0/2124042716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c162a60 0x7f924c0d0a80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 -- 172.21.15.46:0/2124042716 >> 172.21.15.46:0/2124042716 conn(0x7f924c0d4fa0 msgr2=0x7f924c0d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 -- 172.21.15.46:0/2124042716 shutdown_connections 2024-04-26T23:43:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.377+0000 7f924be00640 1 -- 172.21.15.46:0/2124042716 wait complete. 2024-04-26T23:43:03.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.379+0000 7f924be00640 1 Processor -- start 2024-04-26T23:43:03.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.379+0000 7f924be00640 1 -- start start 2024-04-26T23:43:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 0x7f924c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 0x7f924c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0b5240 con 0x7f924c0d5a90 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0b50c0 con 0x7f924c0b4c50 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f924c0b53c0 con 0x7f924c162a60 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36738/0 (socket says 172.21.15.46:36738) 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 -- 172.21.15.46:0/3199656586 learned_addr learned my addr 172.21.15.46:0/3199656586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924aa00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 0x7f924c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924b400640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 0x7f924c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 -- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 msgr2=0x7f924c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 0x7f924c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 -- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 msgr2=0x7f924c0b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 0x7f924c0b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f9243e00640 1 -- 172.21.15.46:0/3199656586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0b92a0 con 0x7f924c0d5a90 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924aa00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 0x7f924c0b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.380+0000 7f924b400640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 0x7f924c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f9243e00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f923400d830 tx=0x7f923400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923400b6c0 con 0x7f924c0d5a90 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234019070 con 0x7f924c0d5a90 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923400ba80 con 0x7f924c0d5a90 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c0b9580 con 0x7f924c0d5a90 2024-04-26T23:43:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.381+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c0b9b40 con 0x7f924c0d5a90 2024-04-26T23:43:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.382+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c069ac0 con 0x7f924c0d5a90 2024-04-26T23:43:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.383+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f923401e460 con 0x7f924c0d5a90 2024-04-26T23:43:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.384+0000 7f9242a00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.384+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f923408c340 con 0x7f924c0d5a90 2024-04-26T23:43:03.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.385+0000 7f924aa00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.386+0000 7f924aa00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f923c034000 tx=0x7f923c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.389+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f923405a5e0 con 0x7f924c0d5a90 2024-04-26T23:43:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.595+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f924c0bae20 con 0x7f924c0d5a90 2024-04-26T23:43:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.597+0000 7f9242a00640 1 -- 172.21.15.46:0/3199656586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16848 (secure 0 0 0) 0x7f9234059ee0 con 0x7f924c0d5a90 2024-04-26T23:43:03.601 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:03.601 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:00.597270+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 msgr2=0x7f922806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f923c034000 tx=0x7f923c02e000 comp rx=0 tx=0).stop 2024-04-26T23:43:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 msgr2=0x7f924c0ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f923400d830 tx=0x7f923400dcf0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 shutdown_connections 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924c162a60 0x7f924c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924c0d5a90 0x7f924c0ce300 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 --2- 172.21.15.46:0/3199656586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c0b4c50 0x7f924c0b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 >> 172.21.15.46:0/3199656586 conn(0x7f924c0d4fa0 msgr2=0x7f924c0d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 shutdown_connections 2024-04-26T23:43:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.600+0000 7f924be00640 1 -- 172.21.15.46:0/3199656586 wait complete. 2024-04-26T23:43:03.623 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:43:03.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:43:03.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.808+0000 7f3880a00640 1 Processor -- start 2024-04-26T23:43:03.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.808+0000 7f3880a00640 1 -- start start 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.809+0000 7f3880a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f387c1466b0 0x7f387c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.809+0000 7f3880a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f387c0720e0 0x7f387c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3880a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387c073ad0 con 0x7f387c160910 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3880a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f387c146a90 con 0x7f387c0720e0 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3880a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f387c073d70 con 0x7f387c1466b0 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3873e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f387c0720e0 0x7f387c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3873e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f387c0720e0 0x7f387c0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36744/0 (socket says 172.21.15.46:36744) 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f3873e00640 1 -- 172.21.15.46:0/2875040843 learned_addr learned my addr 172.21.15.46:0/2875040843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.810+0000 7f387a800640 1 --2- 172.21.15.46:0/2875040843 >> v2:172.21.15.142:3300/0 conn(0x7f387c1466b0 0x7f387c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.811+0000 7f3873e00640 1 -- 172.21.15.46:0/2875040843 >> v2:172.21.15.142:3300/0 conn(0x7f387c1466b0 msgr2=0x7f387c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.811+0000 7f3873e00640 1 --2- 172.21.15.46:0/2875040843 >> v2:172.21.15.142:3300/0 conn(0x7f387c1466b0 0x7f387c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.811+0000 7f3873e00640 1 -- 172.21.15.46:0/2875040843 >> v1:172.21.15.142:6789/0 conn(0x7f387c160910 legacy=0x7f387c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.811+0000 7f3873e00640 1 -- 172.21.15.46:0/2875040843 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c074050 con 0x7f387c0720e0 2024-04-26T23:43:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.812+0000 7f3873e00640 1 --2- 172.21.15.46:0/2875040843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c0720e0 0x7f387c0cd860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3864009870 tx=0x7f386402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86bd96ddf73e457 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.812+0000 7f3879e00640 1 -- 172.21.15.46:0/2875040843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864037070 con 0x7f387c0720e0 2024-04-26T23:43:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.812+0000 7f3879e00640 1 -- 172.21.15.46:0/2875040843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3864032440 con 0x7f387c0720e0 2024-04-26T23:43:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.812+0000 7f3879e00640 1 -- 172.21.15.46:0/2875040843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864032810 con 0x7f387c0720e0 2024-04-26T23:43:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 -- 172.21.15.46:0/2875040843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c0720e0 msgr2=0x7f387c0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 --2- 172.21.15.46:0/2875040843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c0720e0 0x7f387c0cd860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3864009870 tx=0x7f386402a270 comp rx=0 tx=0).stop 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 -- 172.21.15.46:0/2875040843 shutdown_connections 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 --2- 172.21.15.46:0/2875040843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c0720e0 0x7f387c0cd860 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 --2- 172.21.15.46:0/2875040843 >> v2:172.21.15.142:3300/0 conn(0x7f387c1466b0 0x7f387c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 -- 172.21.15.46:0/2875040843 >> 172.21.15.46:0/2875040843 conn(0x7f387c0b26d0 msgr2=0x7f387c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 -- 172.21.15.46:0/2875040843 shutdown_connections 2024-04-26T23:43:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.813+0000 7f3880a00640 1 -- 172.21.15.46:0/2875040843 wait complete. 2024-04-26T23:43:03.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.815+0000 7f3880a00640 1 Processor -- start 2024-04-26T23:43:03.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.815+0000 7f3880a00640 1 -- start start 2024-04-26T23:43:03.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 0x7f387c15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f387c1466b0 0x7f387c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f387c06e3c0 con 0x7f387c160910 2024-04-26T23:43:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f387c0cb470 con 0x7f387c0720e0 2024-04-26T23:43:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f3880a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f387c0cdda0 con 0x7f387c1466b0 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f387b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.816+0000 7f387b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36752/0 (socket says 172.21.15.46:36752) 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 -- 172.21.15.46:0/4124071478 learned_addr learned my addr 172.21.15.46:0/4124071478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387a800640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 0x7f387c15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f3873e00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f387c1466b0 0x7f387c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 -- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f387c1466b0 msgr2=0x7f387c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f387c1466b0 0x7f387c0cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 -- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 msgr2=0x7f387c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 0x7f387c15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 -- 172.21.15.46:0/4124071478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c0d0330 con 0x7f387c160910 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387a800640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 0x7f387c15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.817+0000 7f387b200640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f387400b970 tx=0x7f387400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.818+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874019070 con 0x7f387c160910 2024-04-26T23:43:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.818+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387400cb40 con 0x7f387c160910 2024-04-26T23:43:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.818+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874014440 con 0x7f387c160910 2024-04-26T23:43:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.818+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c0c7cf0 con 0x7f387c160910 2024-04-26T23:43:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.818+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c15fd50 con 0x7f387c160910 2024-04-26T23:43:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.819+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c0693b0 con 0x7f387c160910 2024-04-26T23:43:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.820+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f387401c070 con 0x7f387c160910 2024-04-26T23:43:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.821+0000 7f3872a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 0x7f385806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.822+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f387408ba30 con 0x7f387c160910 2024-04-26T23:43:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.822+0000 7f387a800640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 0x7f385806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.823+0000 7f387a800640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 0x7f385806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:03.826+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3874059c30 con 0x7f387c160910 2024-04-26T23:43:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.025+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f387c0ce3d0 con 0x7f385806c8f0 2024-04-26T23:43:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.045+0000 7f3872a00640 1 -- 172.21.15.46:0/4124071478 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251969 (secure 0 0 0) 0x7f387c0ce3d0 con 0x7f385806c8f0 2024-04-26T23:43:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 msgr2=0x7f385806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 0x7f385806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).stop 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 msgr2=0x7f387c140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f387400b970 tx=0x7f387400be30 comp rx=0 tx=0).stop 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 shutdown_connections 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f385806c8f0 0x7f385806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f387c160910 0x7f387c140e50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f387c1466b0 0x7f387c0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 --2- 172.21.15.46:0/4124071478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f387c0720e0 0x7f387c15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.051+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 >> 172.21.15.46:0/4124071478 conn(0x7f387c0b26d0 msgr2=0x7f387c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.052+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 shutdown_connections 2024-04-26T23:43:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.052+0000 7f3880a00640 1 -- 172.21.15.46:0/4124071478 wait complete. 2024-04-26T23:43:04.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-04-26T23:43:04.018332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310820,"kb_used_data":7284,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754663900,"statfs":{"total":773094113280,"available":772775833600,"internally_reserved":0,"allocated":7458816,"data_stored":3984136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":310693425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002164"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615557+0000","last_change":"2024-04-26T23:42:59.629619+0000","last_active":"2024-04-26T23:43:00.615557+0000","last_peered":"2024-04-26T23:43:00.615557+0000","last_clean":"2024-04-26T23:43:00.615557+0000","last_became_active":"2024-04-26T23:42:59.628945+0000","last_became_peered":"2024-04-26T23:42:59.628945+0000","last_unstale":"2024-04-26T23:43:00.615557+0000","last_undegraded":"2024-04-26T23:43:00.615557+0000","last_fullsized":"2024-04-26T23:43:00.615557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615489+0000","last_change":"2024-04-26T23:42:59.590650+0000","last_active":"2024-04-26T23:43:00.615489+0000","last_peered":"2024-04-26T23:43:00.615489+0000","last_clean":"2024-04-26T23:43:00.615489+0000","last_became_active":"2024-04-26T23:42:59.590342+0000","last_became_peered":"2024-04-26T23:42:59.590342+0000","last_unstale":"2024-04-26T23:43:00.615489+0000","last_undegraded":"2024-04-26T23:43:00.615489+0000","last_fullsized":"2024-04-26T23:43:00.615489+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:24.931821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626045+0000","last_change":"2024-04-26T23:42:59.622242+0000","last_active":"2024-04-26T23:42:59.626045+0000","last_peered":"2024-04-26T23:42:59.626045+0000","last_clean":"2024-04-26T23:42:59.626045+0000","last_became_active":"2024-04-26T23:42:59.621949+0000","last_became_peered":"2024-04-26T23:42:59.621949+0000","last_unstale":"2024-04-26T23:42:59.626045+0000","last_undegraded":"2024-04-26T23:42:59.626045+0000","last_fullsized":"2024-04-26T23:42:59.626045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:40:55.318346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612444+0000","last_change":"2024-04-26T23:42:59.614699+0000","last_active":"2024-04-26T23:43:00.612444+0000","last_peered":"2024-04-26T23:43:00.612444+0000","last_clean":"2024-04-26T23:43:00.612444+0000","last_became_active":"2024-04-26T23:42:59.604746+0000","last_became_peered":"2024-04-26T23:42:59.604746+0000","last_unstale":"2024-04-26T23:43:00.612444+0000","last_undegraded":"2024-04-26T23:43:00.612444+0000","last_fullsized":"2024-04-26T23:43:00.612444+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:16.305731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.624512+0000","last_change":"2024-04-26T23:42:59.621842+0000","last_active":"2024-04-26T23:42:59.624512+0000","last_peered":"2024-04-26T23:42:59.624512+0000","last_clean":"2024-04-26T23:42:59.624512+0000","last_became_active":"2024-04-26T23:42:59.621528+0000","last_became_peered":"2024-04-26T23:42:59.621528+0000","last_unstale":"2024-04-26T23:42:59.624512+0000","last_undegraded":"2024-04-26T23:42:59.624512+0000","last_fullsized":"2024-04-26T23:42:59.624512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:02.124799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611107+0000","last_change":"2024-04-26T23:42:59.607177+0000","last_active":"2024-04-26T23:43:00.611107+0000","last_peered":"2024-04-26T23:43:00.611107+0000","last_clean":"2024-04-26T23:43:00.611107+0000","last_became_active":"2024-04-26T23:42:59.605285+0000","last_became_peered":"2024-04-26T23:42:59.605285+0000","last_unstale":"2024-04-26T23:43:00.611107+0000","last_undegraded":"2024-04-26T23:43:00.611107+0000","last_fullsized":"2024-04-26T23:43:00.611107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:46.648241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.614794+0000","last_change":"2024-04-26T23:42:59.611948+0000","last_active":"2024-04-26T23:42:59.614794+0000","last_peered":"2024-04-26T23:42:59.614794+0000","last_clean":"2024-04-26T23:42:59.614794+0000","last_became_active":"2024-04-26T23:42:59.611648+0000","last_became_peered":"2024-04-26T23:42:59.611648+0000","last_unstale":"2024-04-26T23:42:59.614794+0000","last_undegraded":"2024-04-26T23:42:59.614794+0000","last_fullsized":"2024-04-26T23:42:59.614794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:25.005086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628123+0000","last_change":"2024-04-26T23:42:59.625671+0000","last_active":"2024-04-26T23:42:59.628123+0000","last_peered":"2024-04-26T23:42:59.628123+0000","last_clean":"2024-04-26T23:42:59.628123+0000","last_became_active":"2024-04-26T23:42:59.624126+0000","last_became_peered":"2024-04-26T23:42:59.624126+0000","last_unstale":"2024-04-26T23:42:59.628123+0000","last_undegraded":"2024-04-26T23:42:59.628123+0000","last_fullsized":"2024-04-26T23:42:59.628123+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:24.931821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630949+0000","last_change":"2024-04-26T23:42:59.626720+0000","last_active":"2024-04-26T23:42:59.630949+0000","last_peered":"2024-04-26T23:42:59.630949+0000","last_clean":"2024-04-26T23:42:59.630949+0000","last_became_active":"2024-04-26T23:42:59.626305+0000","last_became_peered":"2024-04-26T23:42:59.626305+0000","last_unstale":"2024-04-26T23:42:59.630949+0000","last_undegraded":"2024-04-26T23:42:59.630949+0000","last_fullsized":"2024-04-26T23:42:59.630949+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:50.975185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.611404+0000","last_change":"2024-04-26T23:42:59.608120+0000","last_active":"2024-04-26T23:42:59.611404+0000","last_peered":"2024-04-26T23:42:59.611404+0000","last_clean":"2024-04-26T23:42:59.611404+0000","last_became_active":"2024-04-26T23:42:59.607806+0000","last_became_peered":"2024-04-26T23:42:59.607806+0000","last_unstale":"2024-04-26T23:42:59.611404+0000","last_undegraded":"2024-04-26T23:42:59.611404+0000","last_fullsized":"2024-04-26T23:42:59.611404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612060+0000","last_change":"2024-04-26T23:42:59.602912+0000","last_active":"2024-04-26T23:43:00.612060+0000","last_peered":"2024-04-26T23:43:00.612060+0000","last_clean":"2024-04-26T23:43:00.612060+0000","last_became_active":"2024-04-26T23:42:59.602556+0000","last_became_peered":"2024-04-26T23:42:59.602556+0000","last_unstale":"2024-04-26T23:43:00.612060+0000","last_undegraded":"2024-04-26T23:43:00.612060+0000","last_fullsized":"2024-04-26T23:43:00.612060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.688133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.633438+0000","last_change":"2024-04-26T23:42:59.628492+0000","last_active":"2024-04-26T23:42:59.633438+0000","last_peered":"2024-04-26T23:42:59.633438+0000","last_clean":"2024-04-26T23:42:59.633438+0000","last_became_active":"2024-04-26T23:42:59.626107+0000","last_became_peered":"2024-04-26T23:42:59.626107+0000","last_unstale":"2024-04-26T23:42:59.633438+0000","last_undegraded":"2024-04-26T23:42:59.633438+0000","last_fullsized":"2024-04-26T23:42:59.633438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:30.938963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627695+0000","last_change":"2024-04-26T23:42:59.620161+0000","last_active":"2024-04-26T23:42:59.627695+0000","last_peered":"2024-04-26T23:42:59.627695+0000","last_clean":"2024-04-26T23:42:59.627695+0000","last_became_active":"2024-04-26T23:42:59.619887+0000","last_became_peered":"2024-04-26T23:42:59.619887+0000","last_unstale":"2024-04-26T23:42:59.627695+0000","last_undegraded":"2024-04-26T23:42:59.627695+0000","last_fullsized":"2024-04-26T23:42:59.627695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:05.730493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618269+0000","last_change":"2024-04-26T23:42:59.616732+0000","last_active":"2024-04-26T23:42:59.618269+0000","last_peered":"2024-04-26T23:42:59.618269+0000","last_clean":"2024-04-26T23:42:59.618269+0000","last_became_active":"2024-04-26T23:42:59.616359+0000","last_became_peered":"2024-04-26T23:42:59.616359+0000","last_unstale":"2024-04-26T23:42:59.618269+0000","last_undegraded":"2024-04-26T23:42:59.618269+0000","last_fullsized":"2024-04-26T23:42:59.618269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:30.938963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.611893+0000","last_change":"2024-04-26T23:42:59.609257+0000","last_active":"2024-04-26T23:42:59.611893+0000","last_peered":"2024-04-26T23:42:59.611893+0000","last_clean":"2024-04-26T23:42:59.611893+0000","last_became_active":"2024-04-26T23:42:59.608758+0000","last_became_peered":"2024-04-26T23:42:59.608758+0000","last_unstale":"2024-04-26T23:42:59.611893+0000","last_undegraded":"2024-04-26T23:42:59.611893+0000","last_fullsized":"2024-04-26T23:42:59.611893+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627694+0000","last_change":"2024-04-26T23:42:59.626578+0000","last_active":"2024-04-26T23:42:59.627694+0000","last_peered":"2024-04-26T23:42:59.627694+0000","last_clean":"2024-04-26T23:42:59.627694+0000","last_became_active":"2024-04-26T23:42:59.626271+0000","last_became_peered":"2024-04-26T23:42:59.626271+0000","last_unstale":"2024-04-26T23:42:59.627694+0000","last_undegraded":"2024-04-26T23:42:59.627694+0000","last_fullsized":"2024-04-26T23:42:59.627694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:33.549888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615574+0000","last_change":"2024-04-26T23:42:59.590358+0000","last_active":"2024-04-26T23:43:00.615574+0000","last_peered":"2024-04-26T23:43:00.615574+0000","last_clean":"2024-04-26T23:43:00.615574+0000","last_became_active":"2024-04-26T23:42:59.590109+0000","last_became_peered":"2024-04-26T23:42:59.590109+0000","last_unstale":"2024-04-26T23:43:00.615574+0000","last_undegraded":"2024-04-26T23:43:00.615574+0000","last_fullsized":"2024-04-26T23:43:00.615574+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613104+0000","last_change":"2024-04-26T23:42:59.594648+0000","last_active":"2024-04-26T23:43:00.613104+0000","last_peered":"2024-04-26T23:43:00.613104+0000","last_clean":"2024-04-26T23:43:00.613104+0000","last_became_active":"2024-04-26T23:42:59.594300+0000","last_became_peered":"2024-04-26T23:42:59.594300+0000","last_unstale":"2024-04-26T23:43:00.613104+0000","last_undegraded":"2024-04-26T23:43:00.613104+0000","last_fullsized":"2024-04-26T23:43:00.613104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:52.333954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615741+0000","last_change":"2024-04-26T23:42:59.613180+0000","last_active":"2024-04-26T23:42:59.615741+0000","last_peered":"2024-04-26T23:42:59.615741+0000","last_clean":"2024-04-26T23:42:59.615741+0000","last_became_active":"2024-04-26T23:42:59.612873+0000","last_became_peered":"2024-04-26T23:42:59.612873+0000","last_unstale":"2024-04-26T23:42:59.615741+0000","last_undegraded":"2024-04-26T23:42:59.615741+0000","last_fullsized":"2024-04-26T23:42:59.615741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.440991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.614858+0000","last_change":"2024-04-26T23:42:59.612855+0000","last_active":"2024-04-26T23:42:59.614858+0000","last_peered":"2024-04-26T23:42:59.614858+0000","last_clean":"2024-04-26T23:42:59.614858+0000","last_became_active":"2024-04-26T23:42:59.611349+0000","last_became_peered":"2024-04-26T23:42:59.611349+0000","last_unstale":"2024-04-26T23:42:59.614858+0000","last_undegraded":"2024-04-26T23:42:59.614858+0000","last_fullsized":"2024-04-26T23:42:59.614858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:38.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627673+0000","last_change":"2024-04-26T23:42:59.618805+0000","last_active":"2024-04-26T23:42:59.627673+0000","last_peered":"2024-04-26T23:42:59.627673+0000","last_clean":"2024-04-26T23:42:59.627673+0000","last_became_active":"2024-04-26T23:42:59.618461+0000","last_became_peered":"2024-04-26T23:42:59.618461+0000","last_unstale":"2024-04-26T23:42:59.627673+0000","last_undegraded":"2024-04-26T23:42:59.627673+0000","last_fullsized":"2024-04-26T23:42:59.627673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:12.727185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.613087+0000","last_change":"2024-04-26T23:42:59.611086+0000","last_active":"2024-04-26T23:42:59.613087+0000","last_peered":"2024-04-26T23:42:59.613087+0000","last_clean":"2024-04-26T23:42:59.613087+0000","last_became_active":"2024-04-26T23:42:59.610671+0000","last_became_peered":"2024-04-26T23:42:59.610671+0000","last_unstale":"2024-04-26T23:42:59.613087+0000","last_undegraded":"2024-04-26T23:42:59.613087+0000","last_fullsized":"2024-04-26T23:42:59.613087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.612664+0000","last_change":"2024-04-26T23:42:59.607190+0000","last_active":"2024-04-26T23:42:59.612664+0000","last_peered":"2024-04-26T23:42:59.612664+0000","last_clean":"2024-04-26T23:42:59.612664+0000","last_became_active":"2024-04-26T23:42:59.606900+0000","last_became_peered":"2024-04-26T23:42:59.606900+0000","last_unstale":"2024-04-26T23:42:59.612664+0000","last_undegraded":"2024-04-26T23:42:59.612664+0000","last_fullsized":"2024-04-26T23:42:59.612664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:35.274152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615525+0000","last_change":"2024-04-26T23:42:59.613563+0000","last_active":"2024-04-26T23:42:59.615525+0000","last_peered":"2024-04-26T23:42:59.615525+0000","last_clean":"2024-04-26T23:42:59.615525+0000","last_became_active":"2024-04-26T23:42:59.609456+0000","last_became_peered":"2024-04-26T23:42:59.609456+0000","last_unstale":"2024-04-26T23:42:59.615525+0000","last_undegraded":"2024-04-26T23:42:59.615525+0000","last_fullsized":"2024-04-26T23:42:59.615525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:52.435789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.633361+0000","last_change":"2024-04-26T23:42:59.628284+0000","last_active":"2024-04-26T23:42:59.633361+0000","last_peered":"2024-04-26T23:42:59.633361+0000","last_clean":"2024-04-26T23:42:59.633361+0000","last_became_active":"2024-04-26T23:42:59.627890+0000","last_became_peered":"2024-04-26T23:42:59.627890+0000","last_unstale":"2024-04-26T23:42:59.633361+0000","last_undegraded":"2024-04-26T23:42:59.633361+0000","last_fullsized":"2024-04-26T23:42:59.633361+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:52.435789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628030+0000","last_change":"2024-04-26T23:42:59.624443+0000","last_active":"2024-04-26T23:42:59.628030+0000","last_peered":"2024-04-26T23:42:59.628030+0000","last_clean":"2024-04-26T23:42:59.628030+0000","last_became_active":"2024-04-26T23:42:59.624057+0000","last_became_peered":"2024-04-26T23:42:59.624057+0000","last_unstale":"2024-04-26T23:42:59.628030+0000","last_undegraded":"2024-04-26T23:42:59.628030+0000","last_fullsized":"2024-04-26T23:42:59.628030+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:10.597772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612453+0000","last_change":"2024-04-26T23:42:59.609728+0000","last_active":"2024-04-26T23:43:00.612453+0000","last_peered":"2024-04-26T23:43:00.612453+0000","last_clean":"2024-04-26T23:43:00.612453+0000","last_became_active":"2024-04-26T23:42:59.608975+0000","last_became_peered":"2024-04-26T23:42:59.608975+0000","last_unstale":"2024-04-26T23:43:00.612453+0000","last_undegraded":"2024-04-26T23:43:00.612453+0000","last_fullsized":"2024-04-26T23:43:00.612453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:54.994380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614535+0000","last_change":"2024-04-26T23:42:59.623360+0000","last_active":"2024-04-26T23:43:00.614535+0000","last_peered":"2024-04-26T23:43:00.614535+0000","last_clean":"2024-04-26T23:43:00.614535+0000","last_became_active":"2024-04-26T23:42:59.619656+0000","last_became_peered":"2024-04-26T23:42:59.619656+0000","last_unstale":"2024-04-26T23:43:00.614535+0000","last_undegraded":"2024-04-26T23:43:00.614535+0000","last_fullsized":"2024-04-26T23:43:00.614535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:18.640000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626081+0000","last_change":"2024-04-26T23:42:59.622365+0000","last_active":"2024-04-26T23:42:59.626081+0000","last_peered":"2024-04-26T23:42:59.626081+0000","last_clean":"2024-04-26T23:42:59.626081+0000","last_became_active":"2024-04-26T23:42:59.622088+0000","last_became_peered":"2024-04-26T23:42:59.622088+0000","last_unstale":"2024-04-26T23:42:59.626081+0000","last_undegraded":"2024-04-26T23:42:59.626081+0000","last_fullsized":"2024-04-26T23:42:59.626081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:19.236153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628066+0000","last_change":"2024-04-26T23:42:59.625694+0000","last_active":"2024-04-26T23:42:59.628066+0000","last_peered":"2024-04-26T23:42:59.628066+0000","last_clean":"2024-04-26T23:42:59.628066+0000","last_became_active":"2024-04-26T23:42:59.623369+0000","last_became_peered":"2024-04-26T23:42:59.623369+0000","last_unstale":"2024-04-26T23:42:59.628066+0000","last_undegraded":"2024-04-26T23:42:59.628066+0000","last_fullsized":"2024-04-26T23:42:59.628066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611791+0000","last_change":"2024-04-26T23:42:59.609045+0000","last_active":"2024-04-26T23:43:00.611791+0000","last_peered":"2024-04-26T23:43:00.611791+0000","last_clean":"2024-04-26T23:43:00.611791+0000","last_became_active":"2024-04-26T23:42:59.608757+0000","last_became_peered":"2024-04-26T23:42:59.608757+0000","last_unstale":"2024-04-26T23:43:00.611791+0000","last_undegraded":"2024-04-26T23:43:00.611791+0000","last_fullsized":"2024-04-26T23:43:00.611791+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:32.654201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611306+0000","last_change":"2024-04-26T23:42:59.614563+0000","last_active":"2024-04-26T23:43:00.611306+0000","last_peered":"2024-04-26T23:43:00.611306+0000","last_clean":"2024-04-26T23:43:00.611306+0000","last_became_active":"2024-04-26T23:42:59.614242+0000","last_became_peered":"2024-04-26T23:42:59.614242+0000","last_unstale":"2024-04-26T23:43:00.611306+0000","last_undegraded":"2024-04-26T23:43:00.611306+0000","last_fullsized":"2024-04-26T23:43:00.611306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:35.274152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.613288+0000","last_change":"2024-04-26T23:42:59.608227+0000","last_active":"2024-04-26T23:42:59.613288+0000","last_peered":"2024-04-26T23:42:59.613288+0000","last_clean":"2024-04-26T23:42:59.613288+0000","last_became_active":"2024-04-26T23:42:59.607903+0000","last_became_peered":"2024-04-26T23:42:59.607903+0000","last_unstale":"2024-04-26T23:42:59.613288+0000","last_undegraded":"2024-04-26T23:42:59.613288+0000","last_fullsized":"2024-04-26T23:42:59.613288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:26.062442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615682+0000","last_change":"2024-04-26T23:42:58.606513+0000","last_active":"2024-04-26T23:43:00.615682+0000","last_peered":"2024-04-26T23:43:00.615682+0000","last_clean":"2024-04-26T23:43:00.615682+0000","last_became_active":"2024-04-26T23:42:58.604348+0000","last_became_peered":"2024-04-26T23:42:58.604348+0000","last_unstale":"2024-04-26T23:43:00.615682+0000","last_undegraded":"2024-04-26T23:43:00.615682+0000","last_fullsized":"2024-04-26T23:43:00.615682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:20.638855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612256+0000","last_change":"2024-04-26T23:42:59.614493+0000","last_active":"2024-04-26T23:43:00.612256+0000","last_peered":"2024-04-26T23:43:00.612256+0000","last_clean":"2024-04-26T23:43:00.612256+0000","last_became_active":"2024-04-26T23:42:59.613947+0000","last_became_peered":"2024-04-26T23:42:59.613947+0000","last_unstale":"2024-04-26T23:43:00.612256+0000","last_undegraded":"2024-04-26T23:43:00.612256+0000","last_fullsized":"2024-04-26T23:43:00.612256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:50.158295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613214+0000","last_change":"2024-04-26T23:42:58.599947+0000","last_active":"2024-04-26T23:43:00.613214+0000","last_peered":"2024-04-26T23:43:00.613214+0000","last_clean":"2024-04-26T23:43:00.613214+0000","last_became_active":"2024-04-26T23:42:58.599675+0000","last_became_peered":"2024-04-26T23:42:58.599675+0000","last_unstale":"2024-04-26T23:43:00.613214+0000","last_undegraded":"2024-04-26T23:43:00.613214+0000","last_fullsized":"2024-04-26T23:43:00.613214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:49.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612864+0000","last_change":"2024-04-26T23:42:59.615384+0000","last_active":"2024-04-26T23:43:00.612864+0000","last_peered":"2024-04-26T23:43:00.612864+0000","last_clean":"2024-04-26T23:43:00.612864+0000","last_became_active":"2024-04-26T23:42:59.614994+0000","last_became_peered":"2024-04-26T23:42:59.614994+0000","last_unstale":"2024-04-26T23:43:00.612864+0000","last_undegraded":"2024-04-26T23:43:00.612864+0000","last_fullsized":"2024-04-26T23:43:00.612864+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:38.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612288+0000","last_change":"2024-04-26T23:42:58.596571+0000","last_active":"2024-04-26T23:43:00.612288+0000","last_peered":"2024-04-26T23:43:00.612288+0000","last_clean":"2024-04-26T23:43:00.612288+0000","last_became_active":"2024-04-26T23:42:58.596068+0000","last_became_peered":"2024-04-26T23:42:58.596068+0000","last_unstale":"2024-04-26T23:43:00.612288+0000","last_undegraded":"2024-04-26T23:43:00.612288+0000","last_fullsized":"2024-04-26T23:43:00.612288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:18.219188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628258+0000","last_change":"2024-04-26T23:42:59.625271+0000","last_active":"2024-04-26T23:42:59.628258+0000","last_peered":"2024-04-26T23:42:59.628258+0000","last_clean":"2024-04-26T23:42:59.628258+0000","last_became_active":"2024-04-26T23:42:59.624971+0000","last_became_peered":"2024-04-26T23:42:59.624971+0000","last_unstale":"2024-04-26T23:42:59.628258+0000","last_undegraded":"2024-04-26T23:42:59.628258+0000","last_fullsized":"2024-04-26T23:42:59.628258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:30.058041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611847+0000","last_change":"2024-04-26T23:42:58.612316+0000","last_active":"2024-04-26T23:43:00.611847+0000","last_peered":"2024-04-26T23:43:00.611847+0000","last_clean":"2024-04-26T23:43:00.611847+0000","last_became_active":"2024-04-26T23:42:58.610775+0000","last_became_peered":"2024-04-26T23:42:58.610775+0000","last_unstale":"2024-04-26T23:43:00.611847+0000","last_undegraded":"2024-04-26T23:43:00.611847+0000","last_fullsized":"2024-04-26T23:43:00.611847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:40:54.301381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.609933+0000","last_change":"2024-04-26T23:42:59.605268+0000","last_active":"2024-04-26T23:42:59.609933+0000","last_peered":"2024-04-26T23:42:59.609933+0000","last_clean":"2024-04-26T23:42:59.609933+0000","last_became_active":"2024-04-26T23:42:59.604815+0000","last_became_peered":"2024-04-26T23:42:59.604815+0000","last_unstale":"2024-04-26T23:42:59.609933+0000","last_undegraded":"2024-04-26T23:42:59.609933+0000","last_fullsized":"2024-04-26T23:42:59.609933+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:12.870982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.620180+0000","last_change":"2024-04-26T23:42:58.611569+0000","last_active":"2024-04-26T23:42:59.620180+0000","last_peered":"2024-04-26T23:42:59.620180+0000","last_clean":"2024-04-26T23:42:59.620180+0000","last_became_active":"2024-04-26T23:42:58.611291+0000","last_became_peered":"2024-04-26T23:42:58.611291+0000","last_unstale":"2024-04-26T23:42:59.620180+0000","last_undegraded":"2024-04-26T23:42:59.620180+0000","last_fullsized":"2024-04-26T23:42:59.620180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:53.524565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613890+0000","last_change":"2024-04-26T23:42:59.591178+0000","last_active":"2024-04-26T23:43:00.613890+0000","last_peered":"2024-04-26T23:43:00.613890+0000","last_clean":"2024-04-26T23:43:00.613890+0000","last_became_active":"2024-04-26T23:42:59.590926+0000","last_became_peered":"2024-04-26T23:42:59.590926+0000","last_unstale":"2024-04-26T23:43:00.613890+0000","last_undegraded":"2024-04-26T23:43:00.613890+0000","last_fullsized":"2024-04-26T23:43:00.613890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:12.727185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.610903+0000","last_change":"2024-04-26T23:42:58.606402+0000","last_active":"2024-04-26T23:42:59.610903+0000","last_peered":"2024-04-26T23:42:59.610903+0000","last_clean":"2024-04-26T23:42:59.610903+0000","last_became_active":"2024-04-26T23:42:58.606056+0000","last_became_peered":"2024-04-26T23:42:58.606056+0000","last_unstale":"2024-04-26T23:42:59.610903+0000","last_undegraded":"2024-04-26T23:42:59.610903+0000","last_fullsized":"2024-04-26T23:42:59.610903+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.629456+0000","last_change":"2024-04-26T23:42:59.624941+0000","last_active":"2024-04-26T23:42:59.629456+0000","last_peered":"2024-04-26T23:42:59.629456+0000","last_clean":"2024-04-26T23:42:59.629456+0000","last_became_active":"2024-04-26T23:42:59.624657+0000","last_became_peered":"2024-04-26T23:42:59.624657+0000","last_unstale":"2024-04-26T23:42:59.629456+0000","last_undegraded":"2024-04-26T23:42:59.629456+0000","last_fullsized":"2024-04-26T23:42:59.629456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:11.421304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598846+0000","last_change":"2024-04-26T23:42:58.611062+0000","last_active":"2024-04-26T23:42:59.598846+0000","last_peered":"2024-04-26T23:42:59.598846+0000","last_clean":"2024-04-26T23:42:59.598846+0000","last_became_active":"2024-04-26T23:42:58.610764+0000","last_became_peered":"2024-04-26T23:42:58.610764+0000","last_unstale":"2024-04-26T23:42:59.598846+0000","last_undegraded":"2024-04-26T23:42:59.598846+0000","last_fullsized":"2024-04-26T23:42:59.598846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.599564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.621650+0000","last_change":"2024-04-26T23:42:59.621084+0000","last_active":"2024-04-26T23:42:59.621650+0000","last_peered":"2024-04-26T23:42:59.621650+0000","last_clean":"2024-04-26T23:42:59.621650+0000","last_became_active":"2024-04-26T23:42:59.620804+0000","last_became_peered":"2024-04-26T23:42:59.620804+0000","last_unstale":"2024-04-26T23:42:59.621650+0000","last_undegraded":"2024-04-26T23:42:59.621650+0000","last_fullsized":"2024-04-26T23:42:59.621650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:21.200215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600180+0000","last_change":"2024-04-26T23:42:58.609071+0000","last_active":"2024-04-26T23:42:59.600180+0000","last_peered":"2024-04-26T23:42:59.600180+0000","last_clean":"2024-04-26T23:42:59.600180+0000","last_became_active":"2024-04-26T23:42:58.608791+0000","last_became_peered":"2024-04-26T23:42:58.608791+0000","last_unstale":"2024-04-26T23:42:59.600180+0000","last_undegraded":"2024-04-26T23:42:59.600180+0000","last_fullsized":"2024-04-26T23:42:59.600180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:49.038563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614935+0000","last_change":"2024-04-26T23:42:59.619920+0000","last_active":"2024-04-26T23:43:00.614935+0000","last_peered":"2024-04-26T23:43:00.614935+0000","last_clean":"2024-04-26T23:43:00.614935+0000","last_became_active":"2024-04-26T23:42:59.616159+0000","last_became_peered":"2024-04-26T23:42:59.616159+0000","last_unstale":"2024-04-26T23:43:00.614935+0000","last_undegraded":"2024-04-26T23:43:00.614935+0000","last_fullsized":"2024-04-26T23:43:00.614935+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:10.597772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600854+0000","last_change":"2024-04-26T23:42:58.610278+0000","last_active":"2024-04-26T23:42:59.600854+0000","last_peered":"2024-04-26T23:42:59.600854+0000","last_clean":"2024-04-26T23:42:59.600854+0000","last_became_active":"2024-04-26T23:42:58.609983+0000","last_became_peered":"2024-04-26T23:42:58.609983+0000","last_unstale":"2024-04-26T23:42:59.600854+0000","last_undegraded":"2024-04-26T23:42:59.600854+0000","last_fullsized":"2024-04-26T23:42:59.600854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:23.914857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630909+0000","last_change":"2024-04-26T23:42:59.629828+0000","last_active":"2024-04-26T23:42:59.630909+0000","last_peered":"2024-04-26T23:42:59.630909+0000","last_clean":"2024-04-26T23:42:59.630909+0000","last_became_active":"2024-04-26T23:42:59.629161+0000","last_became_peered":"2024-04-26T23:42:59.629161+0000","last_unstale":"2024-04-26T23:42:59.630909+0000","last_undegraded":"2024-04-26T23:42:59.630909+0000","last_fullsized":"2024-04-26T23:42:59.630909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:27.182114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613722+0000","last_change":"2024-04-26T23:42:58.606650+0000","last_active":"2024-04-26T23:43:00.613722+0000","last_peered":"2024-04-26T23:43:00.613722+0000","last_clean":"2024-04-26T23:43:00.613722+0000","last_became_active":"2024-04-26T23:42:58.606368+0000","last_became_peered":"2024-04-26T23:42:58.606368+0000","last_unstale":"2024-04-26T23:43:00.613722+0000","last_undegraded":"2024-04-26T23:43:00.613722+0000","last_fullsized":"2024-04-26T23:43:00.613722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:10.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.631008+0000","last_change":"2024-04-26T23:42:59.629570+0000","last_active":"2024-04-26T23:42:59.631008+0000","last_peered":"2024-04-26T23:42:59.631008+0000","last_clean":"2024-04-26T23:42:59.631008+0000","last_became_active":"2024-04-26T23:42:59.629136+0000","last_became_peered":"2024-04-26T23:42:59.629136+0000","last_unstale":"2024-04-26T23:42:59.631008+0000","last_undegraded":"2024-04-26T23:42:59.631008+0000","last_fullsized":"2024-04-26T23:42:59.631008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:02.969615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618202+0000","last_change":"2024-04-26T23:42:58.608490+0000","last_active":"2024-04-26T23:42:59.618202+0000","last_peered":"2024-04-26T23:42:59.618202+0000","last_clean":"2024-04-26T23:42:59.618202+0000","last_became_active":"2024-04-26T23:42:58.608101+0000","last_became_peered":"2024-04-26T23:42:58.608101+0000","last_unstale":"2024-04-26T23:42:59.618202+0000","last_undegraded":"2024-04-26T23:42:59.618202+0000","last_fullsized":"2024-04-26T23:42:59.618202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:09.367069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612282+0000","last_change":"2024-04-26T23:42:59.609120+0000","last_active":"2024-04-26T23:43:00.612282+0000","last_peered":"2024-04-26T23:43:00.612282+0000","last_clean":"2024-04-26T23:43:00.612282+0000","last_became_active":"2024-04-26T23:42:59.608813+0000","last_became_peered":"2024-04-26T23:42:59.608813+0000","last_unstale":"2024-04-26T23:43:00.612282+0000","last_undegraded":"2024-04-26T23:43:00.612282+0000","last_fullsized":"2024-04-26T23:43:00.612282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:44.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615129+0000","last_change":"2024-04-26T23:42:58.607907+0000","last_active":"2024-04-26T23:43:00.615129+0000","last_peered":"2024-04-26T23:43:00.615129+0000","last_clean":"2024-04-26T23:43:00.615129+0000","last_became_active":"2024-04-26T23:42:58.607315+0000","last_became_peered":"2024-04-26T23:42:58.607315+0000","last_unstale":"2024-04-26T23:43:00.615129+0000","last_undegraded":"2024-04-26T23:43:00.615129+0000","last_fullsized":"2024-04-26T23:43:00.615129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:10.616509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611719+0000","last_change":"2024-04-26T23:42:59.614865+0000","last_active":"2024-04-26T23:43:00.611719+0000","last_peered":"2024-04-26T23:43:00.611719+0000","last_clean":"2024-04-26T23:43:00.611719+0000","last_became_active":"2024-04-26T23:42:59.610118+0000","last_became_peered":"2024-04-26T23:42:59.610118+0000","last_unstale":"2024-04-26T23:43:00.611719+0000","last_undegraded":"2024-04-26T23:43:00.611719+0000","last_fullsized":"2024-04-26T23:43:00.611719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:11.633474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.616303+0000","last_change":"2024-04-26T23:42:58.611341+0000","last_active":"2024-04-26T23:42:59.616303+0000","last_peered":"2024-04-26T23:42:59.616303+0000","last_clean":"2024-04-26T23:42:59.616303+0000","last_became_active":"2024-04-26T23:42:58.610934+0000","last_became_peered":"2024-04-26T23:42:58.610934+0000","last_unstale":"2024-04-26T23:42:59.616303+0000","last_undegraded":"2024-04-26T23:42:59.616303+0000","last_fullsized":"2024-04-26T23:42:59.616303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:17.623035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630957+0000","last_change":"2024-04-26T23:42:59.626823+0000","last_active":"2024-04-26T23:42:59.630957+0000","last_peered":"2024-04-26T23:42:59.630957+0000","last_clean":"2024-04-26T23:42:59.630957+0000","last_became_active":"2024-04-26T23:42:59.624280+0000","last_became_peered":"2024-04-26T23:42:59.624280+0000","last_unstale":"2024-04-26T23:42:59.630957+0000","last_undegraded":"2024-04-26T23:42:59.630957+0000","last_fullsized":"2024-04-26T23:42:59.630957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598768+0000","last_change":"2024-04-26T23:42:58.600727+0000","last_active":"2024-04-26T23:42:59.598768+0000","last_peered":"2024-04-26T23:42:59.598768+0000","last_clean":"2024-04-26T23:42:59.598768+0000","last_became_active":"2024-04-26T23:42:58.600418+0000","last_became_peered":"2024-04-26T23:42:58.600418+0000","last_unstale":"2024-04-26T23:42:59.598768+0000","last_undegraded":"2024-04-26T23:42:59.598768+0000","last_fullsized":"2024-04-26T23:42:59.598768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:53.977415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615785+0000","last_change":"2024-04-26T23:42:59.613504+0000","last_active":"2024-04-26T23:42:59.615785+0000","last_peered":"2024-04-26T23:42:59.615785+0000","last_clean":"2024-04-26T23:42:59.615785+0000","last_became_active":"2024-04-26T23:42:59.613221+0000","last_became_peered":"2024-04-26T23:42:59.613221+0000","last_unstale":"2024-04-26T23:42:59.615785+0000","last_undegraded":"2024-04-26T23:42:59.615785+0000","last_fullsized":"2024-04-26T23:42:59.615785+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:46.855644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.593851+0000","last_change":"2024-04-26T23:42:58.604571+0000","last_active":"2024-04-26T23:42:59.593851+0000","last_peered":"2024-04-26T23:42:59.593851+0000","last_clean":"2024-04-26T23:42:59.593851+0000","last_became_active":"2024-04-26T23:42:58.603976+0000","last_became_peered":"2024-04-26T23:42:58.603976+0000","last_unstale":"2024-04-26T23:42:59.593851+0000","last_undegraded":"2024-04-26T23:42:59.593851+0000","last_fullsized":"2024-04-26T23:42:59.593851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:04.713528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614304+0000","last_change":"2024-04-26T23:42:59.623013+0000","last_active":"2024-04-26T23:43:00.614304+0000","last_peered":"2024-04-26T23:43:00.614304+0000","last_clean":"2024-04-26T23:43:00.614304+0000","last_became_active":"2024-04-26T23:42:59.613790+0000","last_became_peered":"2024-04-26T23:42:59.613790+0000","last_unstale":"2024-04-26T23:43:00.614304+0000","last_undegraded":"2024-04-26T23:43:00.614304+0000","last_fullsized":"2024-04-26T23:43:00.614304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:05.730493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611848+0000","last_change":"2024-04-26T23:42:58.605522+0000","last_active":"2024-04-26T23:43:00.611848+0000","last_peered":"2024-04-26T23:43:00.611848+0000","last_clean":"2024-04-26T23:43:00.611848+0000","last_became_active":"2024-04-26T23:42:58.605144+0000","last_became_peered":"2024-04-26T23:42:58.605144+0000","last_unstale":"2024-04-26T23:43:00.611848+0000","last_undegraded":"2024-04-26T23:43:00.611848+0000","last_fullsized":"2024-04-26T23:43:00.611848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:25.045477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626063+0000","last_change":"2024-04-26T23:42:59.621635+0000","last_active":"2024-04-26T23:42:59.626063+0000","last_peered":"2024-04-26T23:42:59.626063+0000","last_clean":"2024-04-26T23:42:59.626063+0000","last_became_active":"2024-04-26T23:42:59.621353+0000","last_became_peered":"2024-04-26T23:42:59.621353+0000","last_unstale":"2024-04-26T23:42:59.626063+0000","last_undegraded":"2024-04-26T23:42:59.626063+0000","last_fullsized":"2024-04-26T23:42:59.626063+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:20.038491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627708+0000","last_change":"2024-04-26T23:42:58.610642+0000","last_active":"2024-04-26T23:42:59.627708+0000","last_peered":"2024-04-26T23:42:59.627708+0000","last_clean":"2024-04-26T23:42:59.627708+0000","last_became_active":"2024-04-26T23:42:58.608796+0000","last_became_peered":"2024-04-26T23:42:58.608796+0000","last_unstale":"2024-04-26T23:42:59.627708+0000","last_undegraded":"2024-04-26T23:42:59.627708+0000","last_fullsized":"2024-04-26T23:42:59.627708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.833833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.623089+0000","last_change":"2024-04-26T23:42:59.620411+0000","last_active":"2024-04-26T23:42:59.623089+0000","last_peered":"2024-04-26T23:42:59.623089+0000","last_clean":"2024-04-26T23:42:59.623089+0000","last_became_active":"2024-04-26T23:42:59.620101+0000","last_became_peered":"2024-04-26T23:42:59.620101+0000","last_unstale":"2024-04-26T23:42:59.623089+0000","last_undegraded":"2024-04-26T23:42:59.623089+0000","last_fullsized":"2024-04-26T23:42:59.623089+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:39.011966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626407+0000","last_change":"2024-04-26T23:42:58.607059+0000","last_active":"2024-04-26T23:42:59.626407+0000","last_peered":"2024-04-26T23:42:59.626407+0000","last_clean":"2024-04-26T23:42:59.626407+0000","last_became_active":"2024-04-26T23:42:58.606743+0000","last_became_peered":"2024-04-26T23:42:58.606743+0000","last_unstale":"2024-04-26T23:42:59.626407+0000","last_undegraded":"2024-04-26T23:42:59.626407+0000","last_fullsized":"2024-04-26T23:42:59.626407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:57.710121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613467+0000","last_change":"2024-04-26T23:42:59.620161+0000","last_active":"2024-04-26T23:43:00.613467+0000","last_peered":"2024-04-26T23:43:00.613467+0000","last_clean":"2024-04-26T23:43:00.613467+0000","last_became_active":"2024-04-26T23:42:59.616033+0000","last_became_peered":"2024-04-26T23:42:59.616033+0000","last_unstale":"2024-04-26T23:43:00.613467+0000","last_undegraded":"2024-04-26T23:43:00.613467+0000","last_fullsized":"2024-04-26T23:43:00.613467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:51.850798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.623821+0000","last_change":"2024-04-26T23:42:58.610035+0000","last_active":"2024-04-26T23:42:59.623821+0000","last_peered":"2024-04-26T23:42:59.623821+0000","last_clean":"2024-04-26T23:42:59.623821+0000","last_became_active":"2024-04-26T23:42:58.609629+0000","last_became_peered":"2024-04-26T23:42:58.609629+0000","last_unstale":"2024-04-26T23:42:59.623821+0000","last_undegraded":"2024-04-26T23:42:59.623821+0000","last_fullsized":"2024-04-26T23:42:59.623821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:11.710220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.622060+0000","last_change":"2024-04-26T23:42:59.618327+0000","last_active":"2024-04-26T23:42:59.622060+0000","last_peered":"2024-04-26T23:42:59.622060+0000","last_clean":"2024-04-26T23:42:59.622060+0000","last_became_active":"2024-04-26T23:42:59.617748+0000","last_became_peered":"2024-04-26T23:42:59.617748+0000","last_unstale":"2024-04-26T23:42:59.622060+0000","last_undegraded":"2024-04-26T23:42:59.622060+0000","last_fullsized":"2024-04-26T23:42:59.622060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:54.215313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598567+0000","last_change":"2024-04-26T23:42:58.608607+0000","last_active":"2024-04-26T23:42:59.598567+0000","last_peered":"2024-04-26T23:42:59.598567+0000","last_clean":"2024-04-26T23:42:59.598567+0000","last_became_active":"2024-04-26T23:42:58.608319+0000","last_became_peered":"2024-04-26T23:42:58.608319+0000","last_unstale":"2024-04-26T23:42:59.598567+0000","last_undegraded":"2024-04-26T23:42:59.598567+0000","last_fullsized":"2024-04-26T23:42:59.598567+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:11.535549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.610968+0000","last_change":"2024-04-26T23:42:59.613359+0000","last_active":"2024-04-26T23:43:00.610968+0000","last_peered":"2024-04-26T23:43:00.610968+0000","last_clean":"2024-04-26T23:43:00.610968+0000","last_became_active":"2024-04-26T23:42:59.612927+0000","last_became_peered":"2024-04-26T23:42:59.612927+0000","last_unstale":"2024-04-26T23:43:00.610968+0000","last_undegraded":"2024-04-26T23:43:00.610968+0000","last_fullsized":"2024-04-26T23:43:00.610968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:07.616529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611853+0000","last_change":"2024-04-26T23:42:58.610654+0000","last_active":"2024-04-26T23:43:00.611853+0000","last_peered":"2024-04-26T23:43:00.611853+0000","last_clean":"2024-04-26T23:43:00.611853+0000","last_became_active":"2024-04-26T23:42:58.607702+0000","last_became_peered":"2024-04-26T23:42:58.607702+0000","last_unstale":"2024-04-26T23:43:00.611853+0000","last_undegraded":"2024-04-26T23:43:00.611853+0000","last_fullsized":"2024-04-26T23:43:00.611853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:19.021526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627979+0000","last_change":"2024-04-26T23:42:59.624418+0000","last_active":"2024-04-26T23:42:59.627979+0000","last_peered":"2024-04-26T23:42:59.627979+0000","last_clean":"2024-04-26T23:42:59.627979+0000","last_became_active":"2024-04-26T23:42:59.623652+0000","last_became_peered":"2024-04-26T23:42:59.623652+0000","last_unstale":"2024-04-26T23:42:59.627979+0000","last_undegraded":"2024-04-26T23:42:59.627979+0000","last_fullsized":"2024-04-26T23:42:59.627979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:18.640000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598943+0000","last_change":"2024-04-26T23:42:58.604994+0000","last_active":"2024-04-26T23:42:59.598943+0000","last_peered":"2024-04-26T23:42:59.598943+0000","last_clean":"2024-04-26T23:42:59.598943+0000","last_became_active":"2024-04-26T23:42:58.601695+0000","last_became_peered":"2024-04-26T23:42:58.601695+0000","last_unstale":"2024-04-26T23:42:59.598943+0000","last_undegraded":"2024-04-26T23:42:59.598943+0000","last_fullsized":"2024-04-26T23:42:59.598943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:15.288766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.624291+0000","last_change":"2024-04-26T23:42:59.621722+0000","last_active":"2024-04-26T23:42:59.624291+0000","last_peered":"2024-04-26T23:42:59.624291+0000","last_clean":"2024-04-26T23:42:59.624291+0000","last_became_active":"2024-04-26T23:42:59.621418+0000","last_became_peered":"2024-04-26T23:42:59.621418+0000","last_unstale":"2024-04-26T23:42:59.624291+0000","last_undegraded":"2024-04-26T23:42:59.624291+0000","last_fullsized":"2024-04-26T23:42:59.624291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:36.266079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.610618+0000","last_change":"2024-04-26T23:42:58.600812+0000","last_active":"2024-04-26T23:43:00.610618+0000","last_peered":"2024-04-26T23:43:00.610618+0000","last_clean":"2024-04-26T23:43:00.610618+0000","last_became_active":"2024-04-26T23:42:58.600475+0000","last_became_peered":"2024-04-26T23:42:58.600475+0000","last_unstale":"2024-04-26T23:43:00.610618+0000","last_undegraded":"2024-04-26T23:43:00.610618+0000","last_fullsized":"2024-04-26T23:43:00.610618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:29.921998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611663+0000","last_change":"2024-04-26T23:42:59.595024+0000","last_active":"2024-04-26T23:43:00.611663+0000","last_peered":"2024-04-26T23:43:00.611663+0000","last_clean":"2024-04-26T23:43:00.611663+0000","last_became_active":"2024-04-26T23:42:59.594731+0000","last_became_peered":"2024-04-26T23:42:59.594731+0000","last_unstale":"2024-04-26T23:43:00.611663+0000","last_undegraded":"2024-04-26T23:43:00.611663+0000","last_fullsized":"2024-04-26T23:43:00.611663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627568+0000","last_change":"2024-04-26T23:42:58.603016+0000","last_active":"2024-04-26T23:42:59.627568+0000","last_peered":"2024-04-26T23:42:59.627568+0000","last_clean":"2024-04-26T23:42:59.627568+0000","last_became_active":"2024-04-26T23:42:58.601309+0000","last_became_peered":"2024-04-26T23:42:58.601309+0000","last_unstale":"2024-04-26T23:42:59.627568+0000","last_undegraded":"2024-04-26T23:42:59.627568+0000","last_fullsized":"2024-04-26T23:42:59.627568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:09.367069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.622082+0000","last_change":"2024-04-26T23:42:59.621139+0000","last_active":"2024-04-26T23:42:59.622082+0000","last_peered":"2024-04-26T23:42:59.622082+0000","last_clean":"2024-04-26T23:42:59.622082+0000","last_became_active":"2024-04-26T23:42:59.620811+0000","last_became_peered":"2024-04-26T23:42:59.620811+0000","last_unstale":"2024-04-26T23:42:59.622082+0000","last_undegraded":"2024-04-26T23:42:59.622082+0000","last_fullsized":"2024-04-26T23:42:59.622082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:33.937004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600087+0000","last_change":"2024-04-26T23:42:58.605201+0000","last_active":"2024-04-26T23:42:59.600087+0000","last_peered":"2024-04-26T23:42:59.600087+0000","last_clean":"2024-04-26T23:42:59.600087+0000","last_became_active":"2024-04-26T23:42:58.604930+0000","last_became_peered":"2024-04-26T23:42:58.604930+0000","last_unstale":"2024-04-26T23:42:59.600087+0000","last_undegraded":"2024-04-26T23:42:59.600087+0000","last_fullsized":"2024-04-26T23:42:59.600087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:51.316989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627943+0000","last_change":"2024-04-26T23:42:59.623628+0000","last_active":"2024-04-26T23:42:59.627943+0000","last_peered":"2024-04-26T23:42:59.627943+0000","last_clean":"2024-04-26T23:42:59.627943+0000","last_became_active":"2024-04-26T23:42:59.623270+0000","last_became_peered":"2024-04-26T23:42:59.623270+0000","last_unstale":"2024-04-26T23:42:59.627943+0000","last_undegraded":"2024-04-26T23:42:59.627943+0000","last_fullsized":"2024-04-26T23:42:59.627943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.727086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612191+0000","last_change":"2024-04-26T23:42:58.596340+0000","last_active":"2024-04-26T23:43:00.612191+0000","last_peered":"2024-04-26T23:43:00.612191+0000","last_clean":"2024-04-26T23:43:00.612191+0000","last_became_active":"2024-04-26T23:42:58.595712+0000","last_became_peered":"2024-04-26T23:42:58.595712+0000","last_unstale":"2024-04-26T23:43:00.612191+0000","last_undegraded":"2024-04-26T23:43:00.612191+0000","last_fullsized":"2024-04-26T23:43:00.612191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:29.041076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618987+0000","last_change":"2024-04-26T23:42:59.616399+0000","last_active":"2024-04-26T23:42:59.618987+0000","last_peered":"2024-04-26T23:42:59.618987+0000","last_clean":"2024-04-26T23:42:59.618987+0000","last_became_active":"2024-04-26T23:42:59.616087+0000","last_became_peered":"2024-04-26T23:42:59.616087+0000","last_unstale":"2024-04-26T23:42:59.618987+0000","last_undegraded":"2024-04-26T23:42:59.618987+0000","last_fullsized":"2024-04-26T23:42:59.618987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:08.577984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615103+0000","last_change":"2024-04-26T23:42:58.607312+0000","last_active":"2024-04-26T23:43:00.615103+0000","last_peered":"2024-04-26T23:43:00.615103+0000","last_clean":"2024-04-26T23:43:00.615103+0000","last_became_active":"2024-04-26T23:42:58.607036+0000","last_became_peered":"2024-04-26T23:42:58.607036+0000","last_unstale":"2024-04-26T23:43:00.615103+0000","last_undegraded":"2024-04-26T23:43:00.615103+0000","last_fullsized":"2024-04-26T23:43:00.615103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:31.637236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.612623+0000","last_change":"2024-04-26T23:42:59.608287+0000","last_active":"2024-04-26T23:42:59.612623+0000","last_peered":"2024-04-26T23:42:59.612623+0000","last_clean":"2024-04-26T23:42:59.612623+0000","last_became_active":"2024-04-26T23:42:59.607985+0000","last_became_peered":"2024-04-26T23:42:59.607985+0000","last_unstale":"2024-04-26T23:42:59.612623+0000","last_undegraded":"2024-04-26T23:42:59.612623+0000","last_fullsized":"2024-04-26T23:42:59.612623+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.864190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.595765+0000","last_change":"2024-04-26T23:42:58.604935+0000","last_active":"2024-04-26T23:42:59.595765+0000","last_peered":"2024-04-26T23:42:59.595765+0000","last_clean":"2024-04-26T23:42:59.595765+0000","last_became_active":"2024-04-26T23:42:58.604434+0000","last_became_peered":"2024-04-26T23:42:58.604434+0000","last_unstale":"2024-04-26T23:42:59.595765+0000","last_undegraded":"2024-04-26T23:42:59.595765+0000","last_fullsized":"2024-04-26T23:42:59.595765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:09.580807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612002+0000","last_change":"2024-04-26T23:42:59.604178+0000","last_active":"2024-04-26T23:43:00.612002+0000","last_peered":"2024-04-26T23:43:00.612002+0000","last_clean":"2024-04-26T23:43:00.612002+0000","last_became_active":"2024-04-26T23:42:59.603858+0000","last_became_peered":"2024-04-26T23:42:59.603858+0000","last_unstale":"2024-04-26T23:43:00.612002+0000","last_undegraded":"2024-04-26T23:43:00.612002+0000","last_fullsized":"2024-04-26T23:43:00.612002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:21.655820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598744+0000","last_change":"2024-04-26T23:42:58.604612+0000","last_active":"2024-04-26T23:42:59.598744+0000","last_peered":"2024-04-26T23:42:59.598744+0000","last_clean":"2024-04-26T23:42:59.598744+0000","last_became_active":"2024-04-26T23:42:58.604304+0000","last_became_peered":"2024-04-26T23:42:58.604304+0000","last_unstale":"2024-04-26T23:42:59.598744+0000","last_undegraded":"2024-04-26T23:42:59.598744+0000","last_fullsized":"2024-04-26T23:42:59.598744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:11.159842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614474+0000","last_change":"2024-04-26T23:42:59.591881+0000","last_active":"2024-04-26T23:43:00.614474+0000","last_peered":"2024-04-26T23:43:00.614474+0000","last_clean":"2024-04-26T23:43:00.614474+0000","last_became_active":"2024-04-26T23:42:59.591616+0000","last_became_peered":"2024-04-26T23:42:59.591616+0000","last_unstale":"2024-04-26T23:43:00.614474+0000","last_undegraded":"2024-04-26T23:43:00.614474+0000","last_fullsized":"2024-04-26T23:43:00.614474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.727086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.599029+0000","last_change":"2024-04-26T23:42:58.609539+0000","last_active":"2024-04-26T23:42:59.599029+0000","last_peered":"2024-04-26T23:42:59.599029+0000","last_clean":"2024-04-26T23:42:59.599029+0000","last_became_active":"2024-04-26T23:42:58.609219+0000","last_became_peered":"2024-04-26T23:42:58.609219+0000","last_unstale":"2024-04-26T23:42:59.599029+0000","last_undegraded":"2024-04-26T23:42:59.599029+0000","last_fullsized":"2024-04-26T23:42:59.599029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:45.631276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630592+0000","last_change":"2024-04-26T23:42:59.629437+0000","last_active":"2024-04-26T23:42:59.630592+0000","last_peered":"2024-04-26T23:42:59.630592+0000","last_clean":"2024-04-26T23:42:59.630592+0000","last_became_active":"2024-04-26T23:42:59.629079+0000","last_became_peered":"2024-04-26T23:42:59.629079+0000","last_unstale":"2024-04-26T23:42:59.630592+0000","last_undegraded":"2024-04-26T23:42:59.630592+0000","last_fullsized":"2024-04-26T23:42:59.630592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:50.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613835+0000","last_change":"2024-04-26T23:42:58.606748+0000","last_active":"2024-04-26T23:43:00.613835+0000","last_peered":"2024-04-26T23:43:00.613835+0000","last_clean":"2024-04-26T23:43:00.613835+0000","last_became_active":"2024-04-26T23:42:58.604715+0000","last_became_peered":"2024-04-26T23:42:58.604715+0000","last_unstale":"2024-04-26T23:43:00.613835+0000","last_undegraded":"2024-04-26T23:43:00.613835+0000","last_fullsized":"2024-04-26T23:43:00.613835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:01.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615119+0000","last_change":"2024-04-26T23:42:59.611305+0000","last_active":"2024-04-26T23:42:59.615119+0000","last_peered":"2024-04-26T23:42:59.615119+0000","last_clean":"2024-04-26T23:42:59.615119+0000","last_became_active":"2024-04-26T23:42:59.611027+0000","last_became_peered":"2024-04-26T23:42:59.611027+0000","last_unstale":"2024-04-26T23:42:59.615119+0000","last_undegraded":"2024-04-26T23:42:59.615119+0000","last_fullsized":"2024-04-26T23:42:59.615119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:16.502140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.596547+0000","last_change":"2024-04-26T23:42:58.612065+0000","last_active":"2024-04-26T23:42:59.596547+0000","last_peered":"2024-04-26T23:42:59.596547+0000","last_clean":"2024-04-26T23:42:59.596547+0000","last_became_active":"2024-04-26T23:42:58.611794+0000","last_became_peered":"2024-04-26T23:42:58.611794+0000","last_unstale":"2024-04-26T23:42:59.596547+0000","last_undegraded":"2024-04-26T23:42:59.596547+0000","last_fullsized":"2024-04-26T23:42:59.596547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:58.672680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":250,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.592380+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:42:59.592380+0000","last_peered":"2024-04-26T23:42:59.592380+0000","last_clean":"2024-04-26T23:42:59.592380+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:42:59.592380+0000","last_undegraded":"2024-04-26T23:42:59.592380+0000","last_fullsized":"2024-04-26T23:42:59.592380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":940130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":349020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":349008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":786432,"data_stored":350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40256,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331584,"statfs":{"total":96636764160,"available":96595542016,"internally_reserved":0,"allocated":774144,"data_stored":348986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40260,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331580,"statfs":{"total":96636764160,"available":96595537920,"internally_reserved":0,"allocated":774144,"data_stored":348986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":351362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":946046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:43:04.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:43:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.324+0000 7ff28b200640 1 Processor -- start 2024-04-26T23:43:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.324+0000 7ff28b200640 1 -- start start 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff28c14c2e0 0x7ff28c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c0cb0f0 0x7ff28c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c0cd9f0 0x7ff28c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0cde60 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0cb8b0 con 0x7ff28c14c2e0 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c14c6c0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c0cd9f0 0x7ff28c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c0cd9f0 0x7ff28c0c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36754/0 (socket says 172.21.15.46:36754) 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.326+0000 7ff28a800640 1 -- 172.21.15.46:0/1989625541 learned_addr learned my addr 172.21.15.46:0/1989625541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff289400640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c0cb0f0 0x7ff28c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff289e00640 1 --2- 172.21.15.46:0/1989625541 >> v2:172.21.15.142:3300/0 conn(0x7ff28c14c2e0 0x7ff28c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff28a800640 1 -- 172.21.15.46:0/1989625541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c0cb0f0 msgr2=0x7ff28c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff28a800640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c0cb0f0 0x7ff28c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff28a800640 1 -- 172.21.15.46:0/1989625541 >> v2:172.21.15.142:3300/0 conn(0x7ff28c14c2e0 msgr2=0x7ff28c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff28a800640 1 --2- 172.21.15.46:0/1989625541 >> v2:172.21.15.142:3300/0 conn(0x7ff28c14c2e0 0x7ff28c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.327+0000 7ff28a800640 1 -- 172.21.15.46:0/1989625541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c167e10 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.328+0000 7ff28a800640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c0c5940 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0cd480 tx=0x7ff28402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101f9cafd651ad11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.328+0000 7ff288a00640 1 -- 172.21.15.46:0/1989625541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284036070 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.328+0000 7ff288a00640 1 -- 172.21.15.46:0/1989625541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff284031600 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.328+0000 7ff288a00640 1 -- 172.21.15.46:0/1989625541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840319d0 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 -- 172.21.15.46:0/1989625541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cd9f0 msgr2=0x7ff28c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c0c5940 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0cd480 tx=0x7ff28402a610 comp rx=0 tx=0).stop 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 -- 172.21.15.46:0/1989625541 shutdown_connections 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c0c5940 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 --2- 172.21.15.46:0/1989625541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c0cb0f0 0x7ff28c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 --2- 172.21.15.46:0/1989625541 >> v2:172.21.15.142:3300/0 conn(0x7ff28c14c2e0 0x7ff28c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 -- 172.21.15.46:0/1989625541 >> 172.21.15.46:0/1989625541 conn(0x7ff28c0b26d0 msgr2=0x7ff28c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 -- 172.21.15.46:0/1989625541 shutdown_connections 2024-04-26T23:43:04.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.329+0000 7ff28b200640 1 -- 172.21.15.46:0/1989625541 wait complete. 2024-04-26T23:43:04.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.331+0000 7ff28b200640 1 Processor -- start 2024-04-26T23:43:04.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.331+0000 7ff28b200640 1 -- start start 2024-04-26T23:43:04.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c14c2e0 0x7ff28c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c167610 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c167330 con 0x7ff28c0cd9f0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff28b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c1678f0 con 0x7ff28c14c2e0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff289e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff289e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36762/0 (socket says 172.21.15.46:36762) 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.332+0000 7ff289e00640 1 -- 172.21.15.46:0/3624858610 learned_addr learned my addr 172.21.15.46:0/3624858610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289400640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 -- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c14c2e0 msgr2=0x7ff28c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff28a800640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c14c2e0 0x7ff28c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c14c2e0 0x7ff28c06de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 -- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 msgr2=0x7ff28c072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 -- 172.21.15.46:0/3624858610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c073b40 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289400640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff289e00640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d830 tx=0x7ff27c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00b6c0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c019070 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.333+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00ba80 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.334+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c14ace0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.334+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c149f20 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.335+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c0693b0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.336+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff27c01e460 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.337+0000 7ff282000640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.337+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff27c08c2b0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.338+0000 7ff289400640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.339+0000 7ff289400640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 0x7ff26c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.342+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff27c05a5e0 con 0x7ff28c0cb0f0 2024-04-26T23:43:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.540+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff28c1634d0 con 0x7ff26c06c8f0 2024-04-26T23:43:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.559+0000 7ff282000640 1 -- 172.21.15.46:0/3624858610 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251969 (secure 0 0 0) 0x7ff28c1634d0 con 0x7ff26c06c8f0 2024-04-26T23:43:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:43:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 msgr2=0x7ff26c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 0x7ff26c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278008040 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 msgr2=0x7ff28c1f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d830 tx=0x7ff27c00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 shutdown_connections 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff28c14c2e0 0x7ff28c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff28c0cd9f0 0x7ff28c072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 --2- 172.21.15.46:0/3624858610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0cb0f0 0x7ff28c1f8da0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 >> 172.21.15.46:0/3624858610 conn(0x7ff28c0b26d0 msgr2=0x7ff28c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 shutdown_connections 2024-04-26T23:43:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.565+0000 7ff28b200640 1 -- 172.21.15.46:0/3624858610 wait complete. 2024-04-26T23:43:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-04-26T23:43:04.018332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310820,"kb_used_data":7284,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754663900,"statfs":{"total":773094113280,"available":772775833600,"internally_reserved":0,"allocated":7458816,"data_stored":3984136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":310693425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002164"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615557+0000","last_change":"2024-04-26T23:42:59.629619+0000","last_active":"2024-04-26T23:43:00.615557+0000","last_peered":"2024-04-26T23:43:00.615557+0000","last_clean":"2024-04-26T23:43:00.615557+0000","last_became_active":"2024-04-26T23:42:59.628945+0000","last_became_peered":"2024-04-26T23:42:59.628945+0000","last_unstale":"2024-04-26T23:43:00.615557+0000","last_undegraded":"2024-04-26T23:43:00.615557+0000","last_fullsized":"2024-04-26T23:43:00.615557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615489+0000","last_change":"2024-04-26T23:42:59.590650+0000","last_active":"2024-04-26T23:43:00.615489+0000","last_peered":"2024-04-26T23:43:00.615489+0000","last_clean":"2024-04-26T23:43:00.615489+0000","last_became_active":"2024-04-26T23:42:59.590342+0000","last_became_peered":"2024-04-26T23:42:59.590342+0000","last_unstale":"2024-04-26T23:43:00.615489+0000","last_undegraded":"2024-04-26T23:43:00.615489+0000","last_fullsized":"2024-04-26T23:43:00.615489+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:24.931821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626045+0000","last_change":"2024-04-26T23:42:59.622242+0000","last_active":"2024-04-26T23:42:59.626045+0000","last_peered":"2024-04-26T23:42:59.626045+0000","last_clean":"2024-04-26T23:42:59.626045+0000","last_became_active":"2024-04-26T23:42:59.621949+0000","last_became_peered":"2024-04-26T23:42:59.621949+0000","last_unstale":"2024-04-26T23:42:59.626045+0000","last_undegraded":"2024-04-26T23:42:59.626045+0000","last_fullsized":"2024-04-26T23:42:59.626045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:40:55.318346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612444+0000","last_change":"2024-04-26T23:42:59.614699+0000","last_active":"2024-04-26T23:43:00.612444+0000","last_peered":"2024-04-26T23:43:00.612444+0000","last_clean":"2024-04-26T23:43:00.612444+0000","last_became_active":"2024-04-26T23:42:59.604746+0000","last_became_peered":"2024-04-26T23:42:59.604746+0000","last_unstale":"2024-04-26T23:43:00.612444+0000","last_undegraded":"2024-04-26T23:43:00.612444+0000","last_fullsized":"2024-04-26T23:43:00.612444+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:16.305731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.624512+0000","last_change":"2024-04-26T23:42:59.621842+0000","last_active":"2024-04-26T23:42:59.624512+0000","last_peered":"2024-04-26T23:42:59.624512+0000","last_clean":"2024-04-26T23:42:59.624512+0000","last_became_active":"2024-04-26T23:42:59.621528+0000","last_became_peered":"2024-04-26T23:42:59.621528+0000","last_unstale":"2024-04-26T23:42:59.624512+0000","last_undegraded":"2024-04-26T23:42:59.624512+0000","last_fullsized":"2024-04-26T23:42:59.624512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:02.124799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611107+0000","last_change":"2024-04-26T23:42:59.607177+0000","last_active":"2024-04-26T23:43:00.611107+0000","last_peered":"2024-04-26T23:43:00.611107+0000","last_clean":"2024-04-26T23:43:00.611107+0000","last_became_active":"2024-04-26T23:42:59.605285+0000","last_became_peered":"2024-04-26T23:42:59.605285+0000","last_unstale":"2024-04-26T23:43:00.611107+0000","last_undegraded":"2024-04-26T23:43:00.611107+0000","last_fullsized":"2024-04-26T23:43:00.611107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:46.648241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.614794+0000","last_change":"2024-04-26T23:42:59.611948+0000","last_active":"2024-04-26T23:42:59.614794+0000","last_peered":"2024-04-26T23:42:59.614794+0000","last_clean":"2024-04-26T23:42:59.614794+0000","last_became_active":"2024-04-26T23:42:59.611648+0000","last_became_peered":"2024-04-26T23:42:59.611648+0000","last_unstale":"2024-04-26T23:42:59.614794+0000","last_undegraded":"2024-04-26T23:42:59.614794+0000","last_fullsized":"2024-04-26T23:42:59.614794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:25.005086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628123+0000","last_change":"2024-04-26T23:42:59.625671+0000","last_active":"2024-04-26T23:42:59.628123+0000","last_peered":"2024-04-26T23:42:59.628123+0000","last_clean":"2024-04-26T23:42:59.628123+0000","last_became_active":"2024-04-26T23:42:59.624126+0000","last_became_peered":"2024-04-26T23:42:59.624126+0000","last_unstale":"2024-04-26T23:42:59.628123+0000","last_undegraded":"2024-04-26T23:42:59.628123+0000","last_fullsized":"2024-04-26T23:42:59.628123+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:24.931821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630949+0000","last_change":"2024-04-26T23:42:59.626720+0000","last_active":"2024-04-26T23:42:59.630949+0000","last_peered":"2024-04-26T23:42:59.630949+0000","last_clean":"2024-04-26T23:42:59.630949+0000","last_became_active":"2024-04-26T23:42:59.626305+0000","last_became_peered":"2024-04-26T23:42:59.626305+0000","last_unstale":"2024-04-26T23:42:59.630949+0000","last_undegraded":"2024-04-26T23:42:59.630949+0000","last_fullsized":"2024-04-26T23:42:59.630949+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:50.975185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.611404+0000","last_change":"2024-04-26T23:42:59.608120+0000","last_active":"2024-04-26T23:42:59.611404+0000","last_peered":"2024-04-26T23:42:59.611404+0000","last_clean":"2024-04-26T23:42:59.611404+0000","last_became_active":"2024-04-26T23:42:59.607806+0000","last_became_peered":"2024-04-26T23:42:59.607806+0000","last_unstale":"2024-04-26T23:42:59.611404+0000","last_undegraded":"2024-04-26T23:42:59.611404+0000","last_fullsized":"2024-04-26T23:42:59.611404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612060+0000","last_change":"2024-04-26T23:42:59.602912+0000","last_active":"2024-04-26T23:43:00.612060+0000","last_peered":"2024-04-26T23:43:00.612060+0000","last_clean":"2024-04-26T23:43:00.612060+0000","last_became_active":"2024-04-26T23:42:59.602556+0000","last_became_peered":"2024-04-26T23:42:59.602556+0000","last_unstale":"2024-04-26T23:43:00.612060+0000","last_undegraded":"2024-04-26T23:43:00.612060+0000","last_fullsized":"2024-04-26T23:43:00.612060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.688133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.633438+0000","last_change":"2024-04-26T23:42:59.628492+0000","last_active":"2024-04-26T23:42:59.633438+0000","last_peered":"2024-04-26T23:42:59.633438+0000","last_clean":"2024-04-26T23:42:59.633438+0000","last_became_active":"2024-04-26T23:42:59.626107+0000","last_became_peered":"2024-04-26T23:42:59.626107+0000","last_unstale":"2024-04-26T23:42:59.633438+0000","last_undegraded":"2024-04-26T23:42:59.633438+0000","last_fullsized":"2024-04-26T23:42:59.633438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:30.938963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627695+0000","last_change":"2024-04-26T23:42:59.620161+0000","last_active":"2024-04-26T23:42:59.627695+0000","last_peered":"2024-04-26T23:42:59.627695+0000","last_clean":"2024-04-26T23:42:59.627695+0000","last_became_active":"2024-04-26T23:42:59.619887+0000","last_became_peered":"2024-04-26T23:42:59.619887+0000","last_unstale":"2024-04-26T23:42:59.627695+0000","last_undegraded":"2024-04-26T23:42:59.627695+0000","last_fullsized":"2024-04-26T23:42:59.627695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:05.730493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618269+0000","last_change":"2024-04-26T23:42:59.616732+0000","last_active":"2024-04-26T23:42:59.618269+0000","last_peered":"2024-04-26T23:42:59.618269+0000","last_clean":"2024-04-26T23:42:59.618269+0000","last_became_active":"2024-04-26T23:42:59.616359+0000","last_became_peered":"2024-04-26T23:42:59.616359+0000","last_unstale":"2024-04-26T23:42:59.618269+0000","last_undegraded":"2024-04-26T23:42:59.618269+0000","last_fullsized":"2024-04-26T23:42:59.618269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:30.938963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.611893+0000","last_change":"2024-04-26T23:42:59.609257+0000","last_active":"2024-04-26T23:42:59.611893+0000","last_peered":"2024-04-26T23:42:59.611893+0000","last_clean":"2024-04-26T23:42:59.611893+0000","last_became_active":"2024-04-26T23:42:59.608758+0000","last_became_peered":"2024-04-26T23:42:59.608758+0000","last_unstale":"2024-04-26T23:42:59.611893+0000","last_undegraded":"2024-04-26T23:42:59.611893+0000","last_fullsized":"2024-04-26T23:42:59.611893+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627694+0000","last_change":"2024-04-26T23:42:59.626578+0000","last_active":"2024-04-26T23:42:59.627694+0000","last_peered":"2024-04-26T23:42:59.627694+0000","last_clean":"2024-04-26T23:42:59.627694+0000","last_became_active":"2024-04-26T23:42:59.626271+0000","last_became_peered":"2024-04-26T23:42:59.626271+0000","last_unstale":"2024-04-26T23:42:59.627694+0000","last_undegraded":"2024-04-26T23:42:59.627694+0000","last_fullsized":"2024-04-26T23:42:59.627694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:33.549888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615574+0000","last_change":"2024-04-26T23:42:59.590358+0000","last_active":"2024-04-26T23:43:00.615574+0000","last_peered":"2024-04-26T23:43:00.615574+0000","last_clean":"2024-04-26T23:43:00.615574+0000","last_became_active":"2024-04-26T23:42:59.590109+0000","last_became_peered":"2024-04-26T23:42:59.590109+0000","last_unstale":"2024-04-26T23:43:00.615574+0000","last_undegraded":"2024-04-26T23:43:00.615574+0000","last_fullsized":"2024-04-26T23:43:00.615574+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613104+0000","last_change":"2024-04-26T23:42:59.594648+0000","last_active":"2024-04-26T23:43:00.613104+0000","last_peered":"2024-04-26T23:43:00.613104+0000","last_clean":"2024-04-26T23:43:00.613104+0000","last_became_active":"2024-04-26T23:42:59.594300+0000","last_became_peered":"2024-04-26T23:42:59.594300+0000","last_unstale":"2024-04-26T23:43:00.613104+0000","last_undegraded":"2024-04-26T23:43:00.613104+0000","last_fullsized":"2024-04-26T23:43:00.613104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:52.333954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615741+0000","last_change":"2024-04-26T23:42:59.613180+0000","last_active":"2024-04-26T23:42:59.615741+0000","last_peered":"2024-04-26T23:42:59.615741+0000","last_clean":"2024-04-26T23:42:59.615741+0000","last_became_active":"2024-04-26T23:42:59.612873+0000","last_became_peered":"2024-04-26T23:42:59.612873+0000","last_unstale":"2024-04-26T23:42:59.615741+0000","last_undegraded":"2024-04-26T23:42:59.615741+0000","last_fullsized":"2024-04-26T23:42:59.615741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.440991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.614858+0000","last_change":"2024-04-26T23:42:59.612855+0000","last_active":"2024-04-26T23:42:59.614858+0000","last_peered":"2024-04-26T23:42:59.614858+0000","last_clean":"2024-04-26T23:42:59.614858+0000","last_became_active":"2024-04-26T23:42:59.611349+0000","last_became_peered":"2024-04-26T23:42:59.611349+0000","last_unstale":"2024-04-26T23:42:59.614858+0000","last_undegraded":"2024-04-26T23:42:59.614858+0000","last_fullsized":"2024-04-26T23:42:59.614858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:38.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627673+0000","last_change":"2024-04-26T23:42:59.618805+0000","last_active":"2024-04-26T23:42:59.627673+0000","last_peered":"2024-04-26T23:42:59.627673+0000","last_clean":"2024-04-26T23:42:59.627673+0000","last_became_active":"2024-04-26T23:42:59.618461+0000","last_became_peered":"2024-04-26T23:42:59.618461+0000","last_unstale":"2024-04-26T23:42:59.627673+0000","last_undegraded":"2024-04-26T23:42:59.627673+0000","last_fullsized":"2024-04-26T23:42:59.627673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:12.727185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.613087+0000","last_change":"2024-04-26T23:42:59.611086+0000","last_active":"2024-04-26T23:42:59.613087+0000","last_peered":"2024-04-26T23:42:59.613087+0000","last_clean":"2024-04-26T23:42:59.613087+0000","last_became_active":"2024-04-26T23:42:59.610671+0000","last_became_peered":"2024-04-26T23:42:59.610671+0000","last_unstale":"2024-04-26T23:42:59.613087+0000","last_undegraded":"2024-04-26T23:42:59.613087+0000","last_fullsized":"2024-04-26T23:42:59.613087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.612664+0000","last_change":"2024-04-26T23:42:59.607190+0000","last_active":"2024-04-26T23:42:59.612664+0000","last_peered":"2024-04-26T23:42:59.612664+0000","last_clean":"2024-04-26T23:42:59.612664+0000","last_became_active":"2024-04-26T23:42:59.606900+0000","last_became_peered":"2024-04-26T23:42:59.606900+0000","last_unstale":"2024-04-26T23:42:59.612664+0000","last_undegraded":"2024-04-26T23:42:59.612664+0000","last_fullsized":"2024-04-26T23:42:59.612664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:35.274152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615525+0000","last_change":"2024-04-26T23:42:59.613563+0000","last_active":"2024-04-26T23:42:59.615525+0000","last_peered":"2024-04-26T23:42:59.615525+0000","last_clean":"2024-04-26T23:42:59.615525+0000","last_became_active":"2024-04-26T23:42:59.609456+0000","last_became_peered":"2024-04-26T23:42:59.609456+0000","last_unstale":"2024-04-26T23:42:59.615525+0000","last_undegraded":"2024-04-26T23:42:59.615525+0000","last_fullsized":"2024-04-26T23:42:59.615525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:52.435789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.633361+0000","last_change":"2024-04-26T23:42:59.628284+0000","last_active":"2024-04-26T23:42:59.633361+0000","last_peered":"2024-04-26T23:42:59.633361+0000","last_clean":"2024-04-26T23:42:59.633361+0000","last_became_active":"2024-04-26T23:42:59.627890+0000","last_became_peered":"2024-04-26T23:42:59.627890+0000","last_unstale":"2024-04-26T23:42:59.633361+0000","last_undegraded":"2024-04-26T23:42:59.633361+0000","last_fullsized":"2024-04-26T23:42:59.633361+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:52.435789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628030+0000","last_change":"2024-04-26T23:42:59.624443+0000","last_active":"2024-04-26T23:42:59.628030+0000","last_peered":"2024-04-26T23:42:59.628030+0000","last_clean":"2024-04-26T23:42:59.628030+0000","last_became_active":"2024-04-26T23:42:59.624057+0000","last_became_peered":"2024-04-26T23:42:59.624057+0000","last_unstale":"2024-04-26T23:42:59.628030+0000","last_undegraded":"2024-04-26T23:42:59.628030+0000","last_fullsized":"2024-04-26T23:42:59.628030+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:10.597772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612453+0000","last_change":"2024-04-26T23:42:59.609728+0000","last_active":"2024-04-26T23:43:00.612453+0000","last_peered":"2024-04-26T23:43:00.612453+0000","last_clean":"2024-04-26T23:43:00.612453+0000","last_became_active":"2024-04-26T23:42:59.608975+0000","last_became_peered":"2024-04-26T23:42:59.608975+0000","last_unstale":"2024-04-26T23:43:00.612453+0000","last_undegraded":"2024-04-26T23:43:00.612453+0000","last_fullsized":"2024-04-26T23:43:00.612453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:54.994380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614535+0000","last_change":"2024-04-26T23:42:59.623360+0000","last_active":"2024-04-26T23:43:00.614535+0000","last_peered":"2024-04-26T23:43:00.614535+0000","last_clean":"2024-04-26T23:43:00.614535+0000","last_became_active":"2024-04-26T23:42:59.619656+0000","last_became_peered":"2024-04-26T23:42:59.619656+0000","last_unstale":"2024-04-26T23:43:00.614535+0000","last_undegraded":"2024-04-26T23:43:00.614535+0000","last_fullsized":"2024-04-26T23:43:00.614535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:18.640000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626081+0000","last_change":"2024-04-26T23:42:59.622365+0000","last_active":"2024-04-26T23:42:59.626081+0000","last_peered":"2024-04-26T23:42:59.626081+0000","last_clean":"2024-04-26T23:42:59.626081+0000","last_became_active":"2024-04-26T23:42:59.622088+0000","last_became_peered":"2024-04-26T23:42:59.622088+0000","last_unstale":"2024-04-26T23:42:59.626081+0000","last_undegraded":"2024-04-26T23:42:59.626081+0000","last_fullsized":"2024-04-26T23:42:59.626081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:19.236153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628066+0000","last_change":"2024-04-26T23:42:59.625694+0000","last_active":"2024-04-26T23:42:59.628066+0000","last_peered":"2024-04-26T23:42:59.628066+0000","last_clean":"2024-04-26T23:42:59.628066+0000","last_became_active":"2024-04-26T23:42:59.623369+0000","last_became_peered":"2024-04-26T23:42:59.623369+0000","last_unstale":"2024-04-26T23:42:59.628066+0000","last_undegraded":"2024-04-26T23:42:59.628066+0000","last_fullsized":"2024-04-26T23:42:59.628066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:17.633227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611791+0000","last_change":"2024-04-26T23:42:59.609045+0000","last_active":"2024-04-26T23:43:00.611791+0000","last_peered":"2024-04-26T23:43:00.611791+0000","last_clean":"2024-04-26T23:43:00.611791+0000","last_became_active":"2024-04-26T23:42:59.608757+0000","last_became_peered":"2024-04-26T23:42:59.608757+0000","last_unstale":"2024-04-26T23:43:00.611791+0000","last_undegraded":"2024-04-26T23:43:00.611791+0000","last_fullsized":"2024-04-26T23:43:00.611791+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:32.654201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611306+0000","last_change":"2024-04-26T23:42:59.614563+0000","last_active":"2024-04-26T23:43:00.611306+0000","last_peered":"2024-04-26T23:43:00.611306+0000","last_clean":"2024-04-26T23:43:00.611306+0000","last_became_active":"2024-04-26T23:42:59.614242+0000","last_became_peered":"2024-04-26T23:42:59.614242+0000","last_unstale":"2024-04-26T23:43:00.611306+0000","last_undegraded":"2024-04-26T23:43:00.611306+0000","last_fullsized":"2024-04-26T23:43:00.611306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:35.274152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.613288+0000","last_change":"2024-04-26T23:42:59.608227+0000","last_active":"2024-04-26T23:42:59.613288+0000","last_peered":"2024-04-26T23:42:59.613288+0000","last_clean":"2024-04-26T23:42:59.613288+0000","last_became_active":"2024-04-26T23:42:59.607903+0000","last_became_peered":"2024-04-26T23:42:59.607903+0000","last_unstale":"2024-04-26T23:42:59.613288+0000","last_undegraded":"2024-04-26T23:42:59.613288+0000","last_fullsized":"2024-04-26T23:42:59.613288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:26.062442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615682+0000","last_change":"2024-04-26T23:42:58.606513+0000","last_active":"2024-04-26T23:43:00.615682+0000","last_peered":"2024-04-26T23:43:00.615682+0000","last_clean":"2024-04-26T23:43:00.615682+0000","last_became_active":"2024-04-26T23:42:58.604348+0000","last_became_peered":"2024-04-26T23:42:58.604348+0000","last_unstale":"2024-04-26T23:43:00.615682+0000","last_undegraded":"2024-04-26T23:43:00.615682+0000","last_fullsized":"2024-04-26T23:43:00.615682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:20.638855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612256+0000","last_change":"2024-04-26T23:42:59.614493+0000","last_active":"2024-04-26T23:43:00.612256+0000","last_peered":"2024-04-26T23:43:00.612256+0000","last_clean":"2024-04-26T23:43:00.612256+0000","last_became_active":"2024-04-26T23:42:59.613947+0000","last_became_peered":"2024-04-26T23:42:59.613947+0000","last_unstale":"2024-04-26T23:43:00.612256+0000","last_undegraded":"2024-04-26T23:43:00.612256+0000","last_fullsized":"2024-04-26T23:43:00.612256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:50.158295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613214+0000","last_change":"2024-04-26T23:42:58.599947+0000","last_active":"2024-04-26T23:43:00.613214+0000","last_peered":"2024-04-26T23:43:00.613214+0000","last_clean":"2024-04-26T23:43:00.613214+0000","last_became_active":"2024-04-26T23:42:58.599675+0000","last_became_peered":"2024-04-26T23:42:58.599675+0000","last_unstale":"2024-04-26T23:43:00.613214+0000","last_undegraded":"2024-04-26T23:43:00.613214+0000","last_fullsized":"2024-04-26T23:43:00.613214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:49.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612864+0000","last_change":"2024-04-26T23:42:59.615384+0000","last_active":"2024-04-26T23:43:00.612864+0000","last_peered":"2024-04-26T23:43:00.612864+0000","last_clean":"2024-04-26T23:43:00.612864+0000","last_became_active":"2024-04-26T23:42:59.614994+0000","last_became_peered":"2024-04-26T23:42:59.614994+0000","last_unstale":"2024-04-26T23:43:00.612864+0000","last_undegraded":"2024-04-26T23:43:00.612864+0000","last_fullsized":"2024-04-26T23:43:00.612864+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:38.248082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612288+0000","last_change":"2024-04-26T23:42:58.596571+0000","last_active":"2024-04-26T23:43:00.612288+0000","last_peered":"2024-04-26T23:43:00.612288+0000","last_clean":"2024-04-26T23:43:00.612288+0000","last_became_active":"2024-04-26T23:42:58.596068+0000","last_became_peered":"2024-04-26T23:42:58.596068+0000","last_unstale":"2024-04-26T23:43:00.612288+0000","last_undegraded":"2024-04-26T23:43:00.612288+0000","last_fullsized":"2024-04-26T23:43:00.612288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:18.219188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.628258+0000","last_change":"2024-04-26T23:42:59.625271+0000","last_active":"2024-04-26T23:42:59.628258+0000","last_peered":"2024-04-26T23:42:59.628258+0000","last_clean":"2024-04-26T23:42:59.628258+0000","last_became_active":"2024-04-26T23:42:59.624971+0000","last_became_peered":"2024-04-26T23:42:59.624971+0000","last_unstale":"2024-04-26T23:42:59.628258+0000","last_undegraded":"2024-04-26T23:42:59.628258+0000","last_fullsized":"2024-04-26T23:42:59.628258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:30.058041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611847+0000","last_change":"2024-04-26T23:42:58.612316+0000","last_active":"2024-04-26T23:43:00.611847+0000","last_peered":"2024-04-26T23:43:00.611847+0000","last_clean":"2024-04-26T23:43:00.611847+0000","last_became_active":"2024-04-26T23:42:58.610775+0000","last_became_peered":"2024-04-26T23:42:58.610775+0000","last_unstale":"2024-04-26T23:43:00.611847+0000","last_undegraded":"2024-04-26T23:43:00.611847+0000","last_fullsized":"2024-04-26T23:43:00.611847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:40:54.301381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.609933+0000","last_change":"2024-04-26T23:42:59.605268+0000","last_active":"2024-04-26T23:42:59.609933+0000","last_peered":"2024-04-26T23:42:59.609933+0000","last_clean":"2024-04-26T23:42:59.609933+0000","last_became_active":"2024-04-26T23:42:59.604815+0000","last_became_peered":"2024-04-26T23:42:59.604815+0000","last_unstale":"2024-04-26T23:42:59.609933+0000","last_undegraded":"2024-04-26T23:42:59.609933+0000","last_fullsized":"2024-04-26T23:42:59.609933+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:12.870982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.620180+0000","last_change":"2024-04-26T23:42:58.611569+0000","last_active":"2024-04-26T23:42:59.620180+0000","last_peered":"2024-04-26T23:42:59.620180+0000","last_clean":"2024-04-26T23:42:59.620180+0000","last_became_active":"2024-04-26T23:42:58.611291+0000","last_became_peered":"2024-04-26T23:42:58.611291+0000","last_unstale":"2024-04-26T23:42:59.620180+0000","last_undegraded":"2024-04-26T23:42:59.620180+0000","last_fullsized":"2024-04-26T23:42:59.620180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:53.524565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613890+0000","last_change":"2024-04-26T23:42:59.591178+0000","last_active":"2024-04-26T23:43:00.613890+0000","last_peered":"2024-04-26T23:43:00.613890+0000","last_clean":"2024-04-26T23:43:00.613890+0000","last_became_active":"2024-04-26T23:42:59.590926+0000","last_became_peered":"2024-04-26T23:42:59.590926+0000","last_unstale":"2024-04-26T23:43:00.613890+0000","last_undegraded":"2024-04-26T23:43:00.613890+0000","last_fullsized":"2024-04-26T23:43:00.613890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:12.727185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.610903+0000","last_change":"2024-04-26T23:42:58.606402+0000","last_active":"2024-04-26T23:42:59.610903+0000","last_peered":"2024-04-26T23:42:59.610903+0000","last_clean":"2024-04-26T23:42:59.610903+0000","last_became_active":"2024-04-26T23:42:58.606056+0000","last_became_peered":"2024-04-26T23:42:58.606056+0000","last_unstale":"2024-04-26T23:42:59.610903+0000","last_undegraded":"2024-04-26T23:42:59.610903+0000","last_fullsized":"2024-04-26T23:42:59.610903+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.629456+0000","last_change":"2024-04-26T23:42:59.624941+0000","last_active":"2024-04-26T23:42:59.629456+0000","last_peered":"2024-04-26T23:42:59.629456+0000","last_clean":"2024-04-26T23:42:59.629456+0000","last_became_active":"2024-04-26T23:42:59.624657+0000","last_became_peered":"2024-04-26T23:42:59.624657+0000","last_unstale":"2024-04-26T23:42:59.629456+0000","last_undegraded":"2024-04-26T23:42:59.629456+0000","last_fullsized":"2024-04-26T23:42:59.629456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:11.421304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598846+0000","last_change":"2024-04-26T23:42:58.611062+0000","last_active":"2024-04-26T23:42:59.598846+0000","last_peered":"2024-04-26T23:42:59.598846+0000","last_clean":"2024-04-26T23:42:59.598846+0000","last_became_active":"2024-04-26T23:42:58.610764+0000","last_became_peered":"2024-04-26T23:42:58.610764+0000","last_unstale":"2024-04-26T23:42:59.598846+0000","last_undegraded":"2024-04-26T23:42:59.598846+0000","last_fullsized":"2024-04-26T23:42:59.598846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.599564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.621650+0000","last_change":"2024-04-26T23:42:59.621084+0000","last_active":"2024-04-26T23:42:59.621650+0000","last_peered":"2024-04-26T23:42:59.621650+0000","last_clean":"2024-04-26T23:42:59.621650+0000","last_became_active":"2024-04-26T23:42:59.620804+0000","last_became_peered":"2024-04-26T23:42:59.620804+0000","last_unstale":"2024-04-26T23:42:59.621650+0000","last_undegraded":"2024-04-26T23:42:59.621650+0000","last_fullsized":"2024-04-26T23:42:59.621650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:21.200215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600180+0000","last_change":"2024-04-26T23:42:58.609071+0000","last_active":"2024-04-26T23:42:59.600180+0000","last_peered":"2024-04-26T23:42:59.600180+0000","last_clean":"2024-04-26T23:42:59.600180+0000","last_became_active":"2024-04-26T23:42:58.608791+0000","last_became_peered":"2024-04-26T23:42:58.608791+0000","last_unstale":"2024-04-26T23:42:59.600180+0000","last_undegraded":"2024-04-26T23:42:59.600180+0000","last_fullsized":"2024-04-26T23:42:59.600180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:49.038563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614935+0000","last_change":"2024-04-26T23:42:59.619920+0000","last_active":"2024-04-26T23:43:00.614935+0000","last_peered":"2024-04-26T23:43:00.614935+0000","last_clean":"2024-04-26T23:43:00.614935+0000","last_became_active":"2024-04-26T23:42:59.616159+0000","last_became_peered":"2024-04-26T23:42:59.616159+0000","last_unstale":"2024-04-26T23:43:00.614935+0000","last_undegraded":"2024-04-26T23:43:00.614935+0000","last_fullsized":"2024-04-26T23:43:00.614935+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:10.597772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600854+0000","last_change":"2024-04-26T23:42:58.610278+0000","last_active":"2024-04-26T23:42:59.600854+0000","last_peered":"2024-04-26T23:42:59.600854+0000","last_clean":"2024-04-26T23:42:59.600854+0000","last_became_active":"2024-04-26T23:42:58.609983+0000","last_became_peered":"2024-04-26T23:42:58.609983+0000","last_unstale":"2024-04-26T23:42:59.600854+0000","last_undegraded":"2024-04-26T23:42:59.600854+0000","last_fullsized":"2024-04-26T23:42:59.600854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:23.914857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630909+0000","last_change":"2024-04-26T23:42:59.629828+0000","last_active":"2024-04-26T23:42:59.630909+0000","last_peered":"2024-04-26T23:42:59.630909+0000","last_clean":"2024-04-26T23:42:59.630909+0000","last_became_active":"2024-04-26T23:42:59.629161+0000","last_became_peered":"2024-04-26T23:42:59.629161+0000","last_unstale":"2024-04-26T23:42:59.630909+0000","last_undegraded":"2024-04-26T23:42:59.630909+0000","last_fullsized":"2024-04-26T23:42:59.630909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:27.182114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613722+0000","last_change":"2024-04-26T23:42:58.606650+0000","last_active":"2024-04-26T23:43:00.613722+0000","last_peered":"2024-04-26T23:43:00.613722+0000","last_clean":"2024-04-26T23:43:00.613722+0000","last_became_active":"2024-04-26T23:42:58.606368+0000","last_became_peered":"2024-04-26T23:42:58.606368+0000","last_unstale":"2024-04-26T23:43:00.613722+0000","last_undegraded":"2024-04-26T23:43:00.613722+0000","last_fullsized":"2024-04-26T23:43:00.613722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:10.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.631008+0000","last_change":"2024-04-26T23:42:59.629570+0000","last_active":"2024-04-26T23:42:59.631008+0000","last_peered":"2024-04-26T23:42:59.631008+0000","last_clean":"2024-04-26T23:42:59.631008+0000","last_became_active":"2024-04-26T23:42:59.629136+0000","last_became_peered":"2024-04-26T23:42:59.629136+0000","last_unstale":"2024-04-26T23:42:59.631008+0000","last_undegraded":"2024-04-26T23:42:59.631008+0000","last_fullsized":"2024-04-26T23:42:59.631008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:02.969615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618202+0000","last_change":"2024-04-26T23:42:58.608490+0000","last_active":"2024-04-26T23:42:59.618202+0000","last_peered":"2024-04-26T23:42:59.618202+0000","last_clean":"2024-04-26T23:42:59.618202+0000","last_became_active":"2024-04-26T23:42:58.608101+0000","last_became_peered":"2024-04-26T23:42:58.608101+0000","last_unstale":"2024-04-26T23:42:59.618202+0000","last_undegraded":"2024-04-26T23:42:59.618202+0000","last_fullsized":"2024-04-26T23:42:59.618202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:09.367069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612282+0000","last_change":"2024-04-26T23:42:59.609120+0000","last_active":"2024-04-26T23:43:00.612282+0000","last_peered":"2024-04-26T23:43:00.612282+0000","last_clean":"2024-04-26T23:43:00.612282+0000","last_became_active":"2024-04-26T23:42:59.608813+0000","last_became_peered":"2024-04-26T23:42:59.608813+0000","last_unstale":"2024-04-26T23:43:00.612282+0000","last_undegraded":"2024-04-26T23:43:00.612282+0000","last_fullsized":"2024-04-26T23:43:00.612282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:44.484537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615129+0000","last_change":"2024-04-26T23:42:58.607907+0000","last_active":"2024-04-26T23:43:00.615129+0000","last_peered":"2024-04-26T23:43:00.615129+0000","last_clean":"2024-04-26T23:43:00.615129+0000","last_became_active":"2024-04-26T23:42:58.607315+0000","last_became_peered":"2024-04-26T23:42:58.607315+0000","last_unstale":"2024-04-26T23:43:00.615129+0000","last_undegraded":"2024-04-26T23:43:00.615129+0000","last_fullsized":"2024-04-26T23:43:00.615129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:10.616509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611719+0000","last_change":"2024-04-26T23:42:59.614865+0000","last_active":"2024-04-26T23:43:00.611719+0000","last_peered":"2024-04-26T23:43:00.611719+0000","last_clean":"2024-04-26T23:43:00.611719+0000","last_became_active":"2024-04-26T23:42:59.610118+0000","last_became_peered":"2024-04-26T23:42:59.610118+0000","last_unstale":"2024-04-26T23:43:00.611719+0000","last_undegraded":"2024-04-26T23:43:00.611719+0000","last_fullsized":"2024-04-26T23:43:00.611719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:11.633474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.616303+0000","last_change":"2024-04-26T23:42:58.611341+0000","last_active":"2024-04-26T23:42:59.616303+0000","last_peered":"2024-04-26T23:42:59.616303+0000","last_clean":"2024-04-26T23:42:59.616303+0000","last_became_active":"2024-04-26T23:42:58.610934+0000","last_became_peered":"2024-04-26T23:42:58.610934+0000","last_unstale":"2024-04-26T23:42:59.616303+0000","last_undegraded":"2024-04-26T23:42:59.616303+0000","last_fullsized":"2024-04-26T23:42:59.616303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:17.623035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630957+0000","last_change":"2024-04-26T23:42:59.626823+0000","last_active":"2024-04-26T23:42:59.630957+0000","last_peered":"2024-04-26T23:42:59.630957+0000","last_clean":"2024-04-26T23:42:59.630957+0000","last_became_active":"2024-04-26T23:42:59.624280+0000","last_became_peered":"2024-04-26T23:42:59.624280+0000","last_unstale":"2024-04-26T23:42:59.630957+0000","last_undegraded":"2024-04-26T23:42:59.630957+0000","last_fullsized":"2024-04-26T23:42:59.630957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598768+0000","last_change":"2024-04-26T23:42:58.600727+0000","last_active":"2024-04-26T23:42:59.598768+0000","last_peered":"2024-04-26T23:42:59.598768+0000","last_clean":"2024-04-26T23:42:59.598768+0000","last_became_active":"2024-04-26T23:42:58.600418+0000","last_became_peered":"2024-04-26T23:42:58.600418+0000","last_unstale":"2024-04-26T23:42:59.598768+0000","last_undegraded":"2024-04-26T23:42:59.598768+0000","last_fullsized":"2024-04-26T23:42:59.598768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:53.977415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615785+0000","last_change":"2024-04-26T23:42:59.613504+0000","last_active":"2024-04-26T23:42:59.615785+0000","last_peered":"2024-04-26T23:42:59.615785+0000","last_clean":"2024-04-26T23:42:59.615785+0000","last_became_active":"2024-04-26T23:42:59.613221+0000","last_became_peered":"2024-04-26T23:42:59.613221+0000","last_unstale":"2024-04-26T23:42:59.615785+0000","last_undegraded":"2024-04-26T23:42:59.615785+0000","last_fullsized":"2024-04-26T23:42:59.615785+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:46.855644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.593851+0000","last_change":"2024-04-26T23:42:58.604571+0000","last_active":"2024-04-26T23:42:59.593851+0000","last_peered":"2024-04-26T23:42:59.593851+0000","last_clean":"2024-04-26T23:42:59.593851+0000","last_became_active":"2024-04-26T23:42:58.603976+0000","last_became_peered":"2024-04-26T23:42:58.603976+0000","last_unstale":"2024-04-26T23:42:59.593851+0000","last_undegraded":"2024-04-26T23:42:59.593851+0000","last_fullsized":"2024-04-26T23:42:59.593851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:04.713528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614304+0000","last_change":"2024-04-26T23:42:59.623013+0000","last_active":"2024-04-26T23:43:00.614304+0000","last_peered":"2024-04-26T23:43:00.614304+0000","last_clean":"2024-04-26T23:43:00.614304+0000","last_became_active":"2024-04-26T23:42:59.613790+0000","last_became_peered":"2024-04-26T23:42:59.613790+0000","last_unstale":"2024-04-26T23:43:00.614304+0000","last_undegraded":"2024-04-26T23:43:00.614304+0000","last_fullsized":"2024-04-26T23:43:00.614304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:05.730493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611848+0000","last_change":"2024-04-26T23:42:58.605522+0000","last_active":"2024-04-26T23:43:00.611848+0000","last_peered":"2024-04-26T23:43:00.611848+0000","last_clean":"2024-04-26T23:43:00.611848+0000","last_became_active":"2024-04-26T23:42:58.605144+0000","last_became_peered":"2024-04-26T23:42:58.605144+0000","last_unstale":"2024-04-26T23:43:00.611848+0000","last_undegraded":"2024-04-26T23:43:00.611848+0000","last_fullsized":"2024-04-26T23:43:00.611848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:25.045477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626063+0000","last_change":"2024-04-26T23:42:59.621635+0000","last_active":"2024-04-26T23:42:59.626063+0000","last_peered":"2024-04-26T23:42:59.626063+0000","last_clean":"2024-04-26T23:42:59.626063+0000","last_became_active":"2024-04-26T23:42:59.621353+0000","last_became_peered":"2024-04-26T23:42:59.621353+0000","last_unstale":"2024-04-26T23:42:59.626063+0000","last_undegraded":"2024-04-26T23:42:59.626063+0000","last_fullsized":"2024-04-26T23:42:59.626063+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:20.038491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627708+0000","last_change":"2024-04-26T23:42:58.610642+0000","last_active":"2024-04-26T23:42:59.627708+0000","last_peered":"2024-04-26T23:42:59.627708+0000","last_clean":"2024-04-26T23:42:59.627708+0000","last_became_active":"2024-04-26T23:42:58.608796+0000","last_became_peered":"2024-04-26T23:42:58.608796+0000","last_unstale":"2024-04-26T23:42:59.627708+0000","last_undegraded":"2024-04-26T23:42:59.627708+0000","last_fullsized":"2024-04-26T23:42:59.627708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.833833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.623089+0000","last_change":"2024-04-26T23:42:59.620411+0000","last_active":"2024-04-26T23:42:59.623089+0000","last_peered":"2024-04-26T23:42:59.623089+0000","last_clean":"2024-04-26T23:42:59.623089+0000","last_became_active":"2024-04-26T23:42:59.620101+0000","last_became_peered":"2024-04-26T23:42:59.620101+0000","last_unstale":"2024-04-26T23:42:59.623089+0000","last_undegraded":"2024-04-26T23:42:59.623089+0000","last_fullsized":"2024-04-26T23:42:59.623089+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:39.011966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.626407+0000","last_change":"2024-04-26T23:42:58.607059+0000","last_active":"2024-04-26T23:42:59.626407+0000","last_peered":"2024-04-26T23:42:59.626407+0000","last_clean":"2024-04-26T23:42:59.626407+0000","last_became_active":"2024-04-26T23:42:58.606743+0000","last_became_peered":"2024-04-26T23:42:58.606743+0000","last_unstale":"2024-04-26T23:42:59.626407+0000","last_undegraded":"2024-04-26T23:42:59.626407+0000","last_fullsized":"2024-04-26T23:42:59.626407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:57.710121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613467+0000","last_change":"2024-04-26T23:42:59.620161+0000","last_active":"2024-04-26T23:43:00.613467+0000","last_peered":"2024-04-26T23:43:00.613467+0000","last_clean":"2024-04-26T23:43:00.613467+0000","last_became_active":"2024-04-26T23:42:59.616033+0000","last_became_peered":"2024-04-26T23:42:59.616033+0000","last_unstale":"2024-04-26T23:43:00.613467+0000","last_undegraded":"2024-04-26T23:43:00.613467+0000","last_fullsized":"2024-04-26T23:43:00.613467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:51.850798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.623821+0000","last_change":"2024-04-26T23:42:58.610035+0000","last_active":"2024-04-26T23:42:59.623821+0000","last_peered":"2024-04-26T23:42:59.623821+0000","last_clean":"2024-04-26T23:42:59.623821+0000","last_became_active":"2024-04-26T23:42:58.609629+0000","last_became_peered":"2024-04-26T23:42:58.609629+0000","last_unstale":"2024-04-26T23:42:59.623821+0000","last_undegraded":"2024-04-26T23:42:59.623821+0000","last_fullsized":"2024-04-26T23:42:59.623821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:11.710220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.622060+0000","last_change":"2024-04-26T23:42:59.618327+0000","last_active":"2024-04-26T23:42:59.622060+0000","last_peered":"2024-04-26T23:42:59.622060+0000","last_clean":"2024-04-26T23:42:59.622060+0000","last_became_active":"2024-04-26T23:42:59.617748+0000","last_became_peered":"2024-04-26T23:42:59.617748+0000","last_unstale":"2024-04-26T23:42:59.622060+0000","last_undegraded":"2024-04-26T23:42:59.622060+0000","last_fullsized":"2024-04-26T23:42:59.622060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:54.215313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598567+0000","last_change":"2024-04-26T23:42:58.608607+0000","last_active":"2024-04-26T23:42:59.598567+0000","last_peered":"2024-04-26T23:42:59.598567+0000","last_clean":"2024-04-26T23:42:59.598567+0000","last_became_active":"2024-04-26T23:42:58.608319+0000","last_became_peered":"2024-04-26T23:42:58.608319+0000","last_unstale":"2024-04-26T23:42:59.598567+0000","last_undegraded":"2024-04-26T23:42:59.598567+0000","last_fullsized":"2024-04-26T23:42:59.598567+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:11.535549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.610968+0000","last_change":"2024-04-26T23:42:59.613359+0000","last_active":"2024-04-26T23:43:00.610968+0000","last_peered":"2024-04-26T23:43:00.610968+0000","last_clean":"2024-04-26T23:43:00.610968+0000","last_became_active":"2024-04-26T23:42:59.612927+0000","last_became_peered":"2024-04-26T23:42:59.612927+0000","last_unstale":"2024-04-26T23:43:00.610968+0000","last_undegraded":"2024-04-26T23:43:00.610968+0000","last_fullsized":"2024-04-26T23:43:00.610968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:07.616529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611853+0000","last_change":"2024-04-26T23:42:58.610654+0000","last_active":"2024-04-26T23:43:00.611853+0000","last_peered":"2024-04-26T23:43:00.611853+0000","last_clean":"2024-04-26T23:43:00.611853+0000","last_became_active":"2024-04-26T23:42:58.607702+0000","last_became_peered":"2024-04-26T23:42:58.607702+0000","last_unstale":"2024-04-26T23:43:00.611853+0000","last_undegraded":"2024-04-26T23:43:00.611853+0000","last_fullsized":"2024-04-26T23:43:00.611853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:19.021526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627979+0000","last_change":"2024-04-26T23:42:59.624418+0000","last_active":"2024-04-26T23:42:59.627979+0000","last_peered":"2024-04-26T23:42:59.627979+0000","last_clean":"2024-04-26T23:42:59.627979+0000","last_became_active":"2024-04-26T23:42:59.623652+0000","last_became_peered":"2024-04-26T23:42:59.623652+0000","last_unstale":"2024-04-26T23:42:59.627979+0000","last_undegraded":"2024-04-26T23:42:59.627979+0000","last_fullsized":"2024-04-26T23:42:59.627979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:18.640000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598943+0000","last_change":"2024-04-26T23:42:58.604994+0000","last_active":"2024-04-26T23:42:59.598943+0000","last_peered":"2024-04-26T23:42:59.598943+0000","last_clean":"2024-04-26T23:42:59.598943+0000","last_became_active":"2024-04-26T23:42:58.601695+0000","last_became_peered":"2024-04-26T23:42:58.601695+0000","last_unstale":"2024-04-26T23:42:59.598943+0000","last_undegraded":"2024-04-26T23:42:59.598943+0000","last_fullsized":"2024-04-26T23:42:59.598943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:15.288766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.624291+0000","last_change":"2024-04-26T23:42:59.621722+0000","last_active":"2024-04-26T23:42:59.624291+0000","last_peered":"2024-04-26T23:42:59.624291+0000","last_clean":"2024-04-26T23:42:59.624291+0000","last_became_active":"2024-04-26T23:42:59.621418+0000","last_became_peered":"2024-04-26T23:42:59.621418+0000","last_unstale":"2024-04-26T23:42:59.624291+0000","last_undegraded":"2024-04-26T23:42:59.624291+0000","last_fullsized":"2024-04-26T23:42:59.624291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:36.266079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.610618+0000","last_change":"2024-04-26T23:42:58.600812+0000","last_active":"2024-04-26T23:43:00.610618+0000","last_peered":"2024-04-26T23:43:00.610618+0000","last_clean":"2024-04-26T23:43:00.610618+0000","last_became_active":"2024-04-26T23:42:58.600475+0000","last_became_peered":"2024-04-26T23:42:58.600475+0000","last_unstale":"2024-04-26T23:43:00.610618+0000","last_undegraded":"2024-04-26T23:43:00.610618+0000","last_fullsized":"2024-04-26T23:43:00.610618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:29.921998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.611663+0000","last_change":"2024-04-26T23:42:59.595024+0000","last_active":"2024-04-26T23:43:00.611663+0000","last_peered":"2024-04-26T23:43:00.611663+0000","last_clean":"2024-04-26T23:43:00.611663+0000","last_became_active":"2024-04-26T23:42:59.594731+0000","last_became_peered":"2024-04-26T23:42:59.594731+0000","last_unstale":"2024-04-26T23:43:00.611663+0000","last_undegraded":"2024-04-26T23:43:00.611663+0000","last_fullsized":"2024-04-26T23:43:00.611663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627568+0000","last_change":"2024-04-26T23:42:58.603016+0000","last_active":"2024-04-26T23:42:59.627568+0000","last_peered":"2024-04-26T23:42:59.627568+0000","last_clean":"2024-04-26T23:42:59.627568+0000","last_became_active":"2024-04-26T23:42:58.601309+0000","last_became_peered":"2024-04-26T23:42:58.601309+0000","last_unstale":"2024-04-26T23:42:59.627568+0000","last_undegraded":"2024-04-26T23:42:59.627568+0000","last_fullsized":"2024-04-26T23:42:59.627568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:09.367069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.622082+0000","last_change":"2024-04-26T23:42:59.621139+0000","last_active":"2024-04-26T23:42:59.622082+0000","last_peered":"2024-04-26T23:42:59.622082+0000","last_clean":"2024-04-26T23:42:59.622082+0000","last_became_active":"2024-04-26T23:42:59.620811+0000","last_became_peered":"2024-04-26T23:42:59.620811+0000","last_unstale":"2024-04-26T23:42:59.622082+0000","last_undegraded":"2024-04-26T23:42:59.622082+0000","last_fullsized":"2024-04-26T23:42:59.622082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:33.937004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.600087+0000","last_change":"2024-04-26T23:42:58.605201+0000","last_active":"2024-04-26T23:42:59.600087+0000","last_peered":"2024-04-26T23:42:59.600087+0000","last_clean":"2024-04-26T23:42:59.600087+0000","last_became_active":"2024-04-26T23:42:58.604930+0000","last_became_peered":"2024-04-26T23:42:58.604930+0000","last_unstale":"2024-04-26T23:42:59.600087+0000","last_undegraded":"2024-04-26T23:42:59.600087+0000","last_fullsized":"2024-04-26T23:42:59.600087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:51.316989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.627943+0000","last_change":"2024-04-26T23:42:59.623628+0000","last_active":"2024-04-26T23:42:59.627943+0000","last_peered":"2024-04-26T23:42:59.627943+0000","last_clean":"2024-04-26T23:42:59.627943+0000","last_became_active":"2024-04-26T23:42:59.623270+0000","last_became_peered":"2024-04-26T23:42:59.623270+0000","last_unstale":"2024-04-26T23:42:59.627943+0000","last_undegraded":"2024-04-26T23:42:59.627943+0000","last_fullsized":"2024-04-26T23:42:59.627943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.727086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612191+0000","last_change":"2024-04-26T23:42:58.596340+0000","last_active":"2024-04-26T23:43:00.612191+0000","last_peered":"2024-04-26T23:43:00.612191+0000","last_clean":"2024-04-26T23:43:00.612191+0000","last_became_active":"2024-04-26T23:42:58.595712+0000","last_became_peered":"2024-04-26T23:42:58.595712+0000","last_unstale":"2024-04-26T23:43:00.612191+0000","last_undegraded":"2024-04-26T23:43:00.612191+0000","last_fullsized":"2024-04-26T23:43:00.612191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:29.041076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.618987+0000","last_change":"2024-04-26T23:42:59.616399+0000","last_active":"2024-04-26T23:42:59.618987+0000","last_peered":"2024-04-26T23:42:59.618987+0000","last_clean":"2024-04-26T23:42:59.618987+0000","last_became_active":"2024-04-26T23:42:59.616087+0000","last_became_peered":"2024-04-26T23:42:59.616087+0000","last_unstale":"2024-04-26T23:42:59.618987+0000","last_undegraded":"2024-04-26T23:42:59.618987+0000","last_fullsized":"2024-04-26T23:42:59.618987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:08.577984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.615103+0000","last_change":"2024-04-26T23:42:58.607312+0000","last_active":"2024-04-26T23:43:00.615103+0000","last_peered":"2024-04-26T23:43:00.615103+0000","last_clean":"2024-04-26T23:43:00.615103+0000","last_became_active":"2024-04-26T23:42:58.607036+0000","last_became_peered":"2024-04-26T23:42:58.607036+0000","last_unstale":"2024-04-26T23:43:00.615103+0000","last_undegraded":"2024-04-26T23:43:00.615103+0000","last_fullsized":"2024-04-26T23:43:00.615103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:31.637236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.612623+0000","last_change":"2024-04-26T23:42:59.608287+0000","last_active":"2024-04-26T23:42:59.612623+0000","last_peered":"2024-04-26T23:42:59.612623+0000","last_clean":"2024-04-26T23:42:59.612623+0000","last_became_active":"2024-04-26T23:42:59.607985+0000","last_became_peered":"2024-04-26T23:42:59.607985+0000","last_unstale":"2024-04-26T23:42:59.612623+0000","last_undegraded":"2024-04-26T23:42:59.612623+0000","last_fullsized":"2024-04-26T23:42:59.612623+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.864190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.595765+0000","last_change":"2024-04-26T23:42:58.604935+0000","last_active":"2024-04-26T23:42:59.595765+0000","last_peered":"2024-04-26T23:42:59.595765+0000","last_clean":"2024-04-26T23:42:59.595765+0000","last_became_active":"2024-04-26T23:42:58.604434+0000","last_became_peered":"2024-04-26T23:42:58.604434+0000","last_unstale":"2024-04-26T23:42:59.595765+0000","last_undegraded":"2024-04-26T23:42:59.595765+0000","last_fullsized":"2024-04-26T23:42:59.595765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:09.580807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.612002+0000","last_change":"2024-04-26T23:42:59.604178+0000","last_active":"2024-04-26T23:43:00.612002+0000","last_peered":"2024-04-26T23:43:00.612002+0000","last_clean":"2024-04-26T23:43:00.612002+0000","last_became_active":"2024-04-26T23:42:59.603858+0000","last_became_peered":"2024-04-26T23:42:59.603858+0000","last_unstale":"2024-04-26T23:43:00.612002+0000","last_undegraded":"2024-04-26T23:43:00.612002+0000","last_fullsized":"2024-04-26T23:43:00.612002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:21.655820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.598744+0000","last_change":"2024-04-26T23:42:58.604612+0000","last_active":"2024-04-26T23:42:59.598744+0000","last_peered":"2024-04-26T23:42:59.598744+0000","last_clean":"2024-04-26T23:42:59.598744+0000","last_became_active":"2024-04-26T23:42:58.604304+0000","last_became_peered":"2024-04-26T23:42:58.604304+0000","last_unstale":"2024-04-26T23:42:59.598744+0000","last_undegraded":"2024-04-26T23:42:59.598744+0000","last_fullsized":"2024-04-26T23:42:59.598744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:11.159842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.614474+0000","last_change":"2024-04-26T23:42:59.591881+0000","last_active":"2024-04-26T23:43:00.614474+0000","last_peered":"2024-04-26T23:43:00.614474+0000","last_clean":"2024-04-26T23:43:00.614474+0000","last_became_active":"2024-04-26T23:42:59.591616+0000","last_became_peered":"2024-04-26T23:42:59.591616+0000","last_unstale":"2024-04-26T23:43:00.614474+0000","last_undegraded":"2024-04-26T23:43:00.614474+0000","last_fullsized":"2024-04-26T23:43:00.614474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:58.727086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.599029+0000","last_change":"2024-04-26T23:42:58.609539+0000","last_active":"2024-04-26T23:42:59.599029+0000","last_peered":"2024-04-26T23:42:59.599029+0000","last_clean":"2024-04-26T23:42:59.599029+0000","last_became_active":"2024-04-26T23:42:58.609219+0000","last_became_peered":"2024-04-26T23:42:58.609219+0000","last_unstale":"2024-04-26T23:42:59.599029+0000","last_undegraded":"2024-04-26T23:42:59.599029+0000","last_fullsized":"2024-04-26T23:42:59.599029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:45.631276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.630592+0000","last_change":"2024-04-26T23:42:59.629437+0000","last_active":"2024-04-26T23:42:59.630592+0000","last_peered":"2024-04-26T23:42:59.630592+0000","last_clean":"2024-04-26T23:42:59.630592+0000","last_became_active":"2024-04-26T23:42:59.629079+0000","last_became_peered":"2024-04-26T23:42:59.629079+0000","last_unstale":"2024-04-26T23:42:59.630592+0000","last_undegraded":"2024-04-26T23:42:59.630592+0000","last_fullsized":"2024-04-26T23:42:59.630592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:50.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T23:43:00.613835+0000","last_change":"2024-04-26T23:42:58.606748+0000","last_active":"2024-04-26T23:43:00.613835+0000","last_peered":"2024-04-26T23:43:00.613835+0000","last_clean":"2024-04-26T23:43:00.613835+0000","last_became_active":"2024-04-26T23:42:58.604715+0000","last_became_peered":"2024-04-26T23:42:58.604715+0000","last_unstale":"2024-04-26T23:43:00.613835+0000","last_undegraded":"2024-04-26T23:43:00.613835+0000","last_fullsized":"2024-04-26T23:43:00.613835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:01.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.615119+0000","last_change":"2024-04-26T23:42:59.611305+0000","last_active":"2024-04-26T23:42:59.615119+0000","last_peered":"2024-04-26T23:42:59.615119+0000","last_clean":"2024-04-26T23:42:59.615119+0000","last_became_active":"2024-04-26T23:42:59.611027+0000","last_became_peered":"2024-04-26T23:42:59.611027+0000","last_unstale":"2024-04-26T23:42:59.615119+0000","last_undegraded":"2024-04-26T23:42:59.615119+0000","last_fullsized":"2024-04-26T23:42:59.615119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:58.558454+0000","last_clean_scrub_stamp":"2024-04-26T23:42:58.558454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:16.502140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.596547+0000","last_change":"2024-04-26T23:42:58.612065+0000","last_active":"2024-04-26T23:42:59.596547+0000","last_peered":"2024-04-26T23:42:59.596547+0000","last_clean":"2024-04-26T23:42:59.596547+0000","last_became_active":"2024-04-26T23:42:58.611794+0000","last_became_peered":"2024-04-26T23:42:58.611794+0000","last_unstale":"2024-04-26T23:42:59.596547+0000","last_undegraded":"2024-04-26T23:42:59.596547+0000","last_fullsized":"2024-04-26T23:42:59.596547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:42:57.541489+0000","last_clean_scrub_stamp":"2024-04-26T23:42:57.541489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:58.672680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":250,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T23:42:59.592380+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:42:59.592380+0000","last_peered":"2024-04-26T23:42:59.592380+0000","last_clean":"2024-04-26T23:42:59.592380+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:42:59.592380+0000","last_undegraded":"2024-04-26T23:42:59.592380+0000","last_fullsized":"2024-04-26T23:42:59.592380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":940130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":349020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":349008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":786432,"data_stored":350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40256,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331584,"statfs":{"total":96636764160,"available":96595542016,"internally_reserved":0,"allocated":774144,"data_stored":348986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40260,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331580,"statfs":{"total":96636764160,"available":96595537920,"internally_reserved":0,"allocated":774144,"data_stored":348986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":351362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":946046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:43:04.591 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:43:04.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:43:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.851+0000 7f79a4e00640 1 Processor -- start 2024-04-26T23:43:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.851+0000 7f79a4e00640 1 -- start start 2024-04-26T23:43:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f79a4e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f79a00d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f79a4e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79a00d3b90 0x7f79a00d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f79a4e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a00b46f0 con 0x7f79a00d5b90 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f79a4e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79a00d11d0 con 0x7f79a00d3b90 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f79a4e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79a00b44a0 con 0x7f79a0162aa0 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f799d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f79a00d5b90 0x7f79a00b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45220/0 (socket says 172.21.15.46:45220) 2024-04-26T23:43:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.853+0000 7f799d400640 1 -- 172.21.15.46:0/2108955876 learned_addr learned my addr 172.21.15.46:0/2108955876 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799de00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f79a00d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799e800640 1 --2- 172.21.15.46:0/2108955876 >> v2:172.21.15.142:3300/0 conn(0x7f79a00d3b90 0x7f79a00d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169716554 0 0) 0x7f79a00b46f0 con 0x7f79a00d5b90 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7984003520 con 0x7f79a00d5b90 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 453072111 0 0) 0x7f7984003520 con 0x7f79a00d5b90 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 msgr2=0x7f79a00d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f79a00d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 >> v2:172.21.15.142:3300/0 conn(0x7f79a00d3b90 msgr2=0x7f79a00d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 --2- 172.21.15.46:0/2108955876 >> v2:172.21.15.142:3300/0 conn(0x7f79a00d3b90 0x7f79a00d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a00b49d0 con 0x7f79a00d5b90 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7990001670 con 0x7f79a00d5b90 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 >> v1:172.21.15.46:6789/0 conn(0x7f79a00d5b90 legacy=0x7f79a00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f7984004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79a00d11d0 con 0x7f79840039b0 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79a00b44a0 con 0x7f7984004d40 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.854+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f798400baa0 con 0x7f79a0162aa0 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799d400640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f7984004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799e800640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 -- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 msgr2=0x7f7984004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a0162aa0 0x7f7984004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 -- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984004d40 msgr2=0x7f79840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799de00640 1 -- 172.21.15.46:0/2108955876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7984003520 con 0x7f79840039b0 2024-04-26T23:43:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.855+0000 7f799e800640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.856+0000 7f799de00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7988009ac0 tx=0x7f7984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3bea105c1bf029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988036070 con 0x7f79840039b0 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7988031400 con 0x7f79840039b0 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f799ca00640 1 -- 172.21.15.46:0/2108955876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988009690 con 0x7f79840039b0 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 -- 172.21.15.46:0/2108955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79840039b0 msgr2=0x7f7984003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 --2- 172.21.15.46:0/2108955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7988009ac0 tx=0x7f7984005830 comp rx=0 tx=0).stop 2024-04-26T23:43:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f799e800640 1 -- 172.21.15.46:0/2108955876 reap_dead start 2024-04-26T23:43:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 -- 172.21.15.46:0/2108955876 shutdown_connections 2024-04-26T23:43:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 -- 172.21.15.46:0/2108955876 >> 172.21.15.46:0/2108955876 conn(0x7f79a00d5070 msgr2=0x7f79a00d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 -- 172.21.15.46:0/2108955876 shutdown_connections 2024-04-26T23:43:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.857+0000 7f79a4e00640 1 -- 172.21.15.46:0/2108955876 wait complete. 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.859+0000 7f79a4e00640 1 Processor -- start 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.859+0000 7f79a4e00640 1 -- start start 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79840039b0 0x7f79a00cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a014efb0 0x7f79a0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79a00d3fd0 con 0x7f79a00c9bf0 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79a00b5020 con 0x7f79a014efb0 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.860+0000 7f79a4e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79a0162e80 con 0x7f79840039b0 2024-04-26T23:43:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36778/0 (socket says 172.21.15.46:36778) 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 -- 172.21.15.46:0/4060572234 learned_addr learned my addr 172.21.15.46:0/4060572234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799de00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79840039b0 0x7f79a00cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 -- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79840039b0 msgr2=0x7f79a00cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79840039b0 0x7f79a00cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 -- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a014efb0 msgr2=0x7f79a0152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a014efb0 0x7f79a0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 -- 172.21.15.46:0/4060572234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a0152760 con 0x7f79a00c9bf0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.861+0000 7f799e800640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b4450 tx=0x7f7998009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.862+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998019070 con 0x7f79a00c9bf0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.862+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7998007d40 con 0x7f79a00c9bf0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.862+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998014640 con 0x7f79a00c9bf0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.862+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a00cde20 con 0x7f79a00c9bf0 2024-04-26T23:43:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.862+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a00ce0e0 con 0x7f79a00c9bf0 2024-04-26T23:43:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.863+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a0069530 con 0x7f79a00c9bf0 2024-04-26T23:43:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.864+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f799801c070 con 0x7f79a00c9bf0 2024-04-26T23:43:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.865+0000 7f7996000640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 0x7f797c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.865+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f799808b7f0 con 0x7f79a00c9bf0 2024-04-26T23:43:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.866+0000 7f799de00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 0x7f797c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.867+0000 7f799de00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 0x7f797c06ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7988033000 tx=0x7f7988008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:04.870+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79a0152760 con 0x7f79a00c9bf0 2024-04-26T23:43:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.147+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f79a0069e60 con 0x7f79a00c9bf0 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.152+0000 7f7996000640 1 -- 172.21.15.46:0/4060572234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f79980599f0 con 0x7f79a00c9bf0 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:43:05.155 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:43:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 msgr2=0x7f797c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 0x7f797c06ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7988033000 tx=0x7f7988008030 comp rx=0 tx=0).stop 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 msgr2=0x7f79a01f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b4450 tx=0x7f7998009d40 comp rx=0 tx=0).stop 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 shutdown_connections 2024-04-26T23:43:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797c06c920 0x7f797c06ede0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79a00c9bf0 0x7f79a01f85a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a014efb0 0x7f79a0152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 --2- 172.21.15.46:0/4060572234 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79840039b0 0x7f79a00cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 >> 172.21.15.46:0/4060572234 conn(0x7f79a00d5070 msgr2=0x7f79a00b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 shutdown_connections 2024-04-26T23:43:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.155+0000 7f79a4e00640 1 -- 172.21.15.46:0/4060572234 wait complete. 2024-04-26T23:43:05.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:05.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.383+0000 7f4bbfe00640 1 Processor -- start 2024-04-26T23:43:05.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.383+0000 7f4bbfe00640 1 -- start start 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc015edf0 0x7f4bc00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bc015f1d0 0x7f4bc00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc00b03c0 con 0x7f4bc00cb1a0 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc00cf5c0 con 0x7f4bc015f1d0 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc00cb680 con 0x7f4bc015edf0 2024-04-26T23:43:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbe000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bc015f1d0 0x7f4bc00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbe000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bc015f1d0 0x7f4bc00cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36790/0 (socket says 172.21.15.46:36790) 2024-04-26T23:43:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbe000640 1 -- 172.21.15.46:0/6538541 learned_addr learned my addr 172.21.15.46:0/6538541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.385+0000 7f4bbea00640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc015edf0 0x7f4bc00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.386+0000 7f4bbe000640 1 -- 172.21.15.46:0/6538541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc015edf0 msgr2=0x7f4bc00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.386+0000 7f4bbe000640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc015edf0 0x7f4bc00cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.386+0000 7f4bbe000640 1 -- 172.21.15.46:0/6538541 >> v1:172.21.15.142:6789/0 conn(0x7f4bc00cb1a0 legacy=0x7f4bc00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.386+0000 7f4bbe000640 1 -- 172.21.15.46:0/6538541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc00cb960 con 0x7f4bc015f1d0 2024-04-26T23:43:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.387+0000 7f4bbe000640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc015f1d0 0x7f4bc00cd060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b34e0 tx=0x7f4bac02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac75dee22b22d16b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.388+0000 7f4bbd600640 1 -- 172.21.15.46:0/6538541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac037070 con 0x7f4bc015f1d0 2024-04-26T23:43:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.388+0000 7f4bbd600640 1 -- 172.21.15.46:0/6538541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac032440 con 0x7f4bc015f1d0 2024-04-26T23:43:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.388+0000 7f4bbd600640 1 -- 172.21.15.46:0/6538541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac032810 con 0x7f4bc015f1d0 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 -- 172.21.15.46:0/6538541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc015f1d0 msgr2=0x7f4bc00cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc015f1d0 0x7f4bc00cd060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b34e0 tx=0x7f4bac02a290 comp rx=0 tx=0).stop 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 -- 172.21.15.46:0/6538541 shutdown_connections 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc015f1d0 0x7f4bc00cd060 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/6538541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc015edf0 0x7f4bc00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 -- 172.21.15.46:0/6538541 >> 172.21.15.46:0/6538541 conn(0x7f4bc00b26d0 msgr2=0x7f4bc00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:05.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 -- 172.21.15.46:0/6538541 shutdown_connections 2024-04-26T23:43:05.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.389+0000 7f4bbfe00640 1 -- 172.21.15.46:0/6538541 wait complete. 2024-04-26T23:43:05.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.391+0000 7f4bbfe00640 1 Processor -- start 2024-04-26T23:43:05.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.392+0000 7f4bbfe00640 1 -- start start 2024-04-26T23:43:05.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc00cb1a0 0x7f4bc014c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 0x7f4bc00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc00cc200 con 0x7f4bc00c89b0 2024-04-26T23:43:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc00cbf20 con 0x7f4bc015edf0 2024-04-26T23:43:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc00c6950 con 0x7f4bc00cb1a0 2024-04-26T23:43:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36800/0 (socket says 172.21.15.46:36800) 2024-04-26T23:43:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.393+0000 7f4bbf400640 1 -- 172.21.15.46:0/1815393561 learned_addr learned my addr 172.21.15.46:0/1815393561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbea00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc00cb1a0 0x7f4bc014c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbe000640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 0x7f4bc00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 -- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc00cb1a0 msgr2=0x7f4bc014c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc00cb1a0 0x7f4bc014c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 -- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 msgr2=0x7f4bc00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 0x7f4bc00721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 -- 172.21.15.46:0/1815393561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc0072a10 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbe000640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 0x7f4bc00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbf400640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c700 tx=0x7f4bb800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8016020 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb800a9f0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800ade0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.394+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc0072cf0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.395+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc00732b0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.396+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc015c830 con 0x7f4bc00c89b0 2024-04-26T23:43:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.397+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bb8019020 con 0x7f4bc00c89b0 2024-04-26T23:43:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.398+0000 7f4bb7400640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 0x7f4ba006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.398+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f4bb808bfa0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.398+0000 7f4bbea00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 0x7f4ba006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.400+0000 7f4bbea00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 0x7f4ba006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.403+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bc015c830 con 0x7f4bc00c89b0 2024-04-26T23:43:05.544 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-04-26T23:43:05.541+0000 7fe394e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:43:05.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.655+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bc00693b0 con 0x7f4bc00c89b0 2024-04-26T23:43:05.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.656+0000 7f4bb7400640 1 -- 172.21.15.46:0/1815393561 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5188 (secure 0 0 0) 0x7f4bb8020020 con 0x7f4bc00c89b0 2024-04-26T23:43:05.659 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:05.659 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3235774034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3235774034},{"type":"v1","addr":"172.21.15.142:6837","nonce":3235774034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6579,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3040630531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3040630531},{"type":"v1","addr":"172.21.15.46:6833","nonce":3040630531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6582,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1818830260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1818830260},{"type":"v1","addr":"172.21.15.46:6835","nonce":1818830260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:02.129282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2927770154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2927770154},{"type":"v1","addr":"172.21.15.142:6835","nonce":2927770154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":4}]} 2024-04-26T23:43:05.659 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 41 2024-04-26T23:43:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 msgr2=0x7f4ba006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 0x7f4ba006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0008040 comp rx=0 tx=0).stop 2024-04-26T23:43:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 msgr2=0x7f4bc00c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c700 tx=0x7f4bb800cbc0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 shutdown_connections 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ba006c920 0x7f4ba006ede0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc00c89b0 0x7f4bc00c9390 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc015edf0 0x7f4bc00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/1815393561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bc00cb1a0 0x7f4bc014c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 >> 172.21.15.46:0/1815393561 conn(0x7f4bc00b26d0 msgr2=0x7f4bc00aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 shutdown_connections 2024-04-26T23:43:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:05.659+0000 7f4bbfe00640 1 -- 172.21.15.46:0/1815393561 wait complete. 2024-04-26T23:43:06.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:42:59.049825+0000', 'modified': '2024-04-26T23:43:02.129282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6591}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6591': {'gid': 6591, 'name': 'd', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6835/2927770154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2927770154}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2927770154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6591, 'qdb_cluster': [6591]} 2024-04-26T23:43:06.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:43:06.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T23:43:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.305+0000 7f2077e00640 1 Processor -- start 2024-04-26T23:43:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.305+0000 7f2077e00640 1 -- start start 2024-04-26T23:43:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2077e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f207815d040 0x7f20780cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2077e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20780d37e0 0x7f20780b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2077e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20780d0b40 con 0x7f20780b2b60 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2077e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f20780cd840 con 0x7f20780d37e0 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2077e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f20780d0db0 con 0x7f207815d040 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2076000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20780d37e0 0x7f20780b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2076000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20780d37e0 0x7f20780b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36810/0 (socket says 172.21.15.46:36810) 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.307+0000 7f2076000640 1 -- 172.21.15.46:0/2124218014 learned_addr learned my addr 172.21.15.46:0/2124218014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.308+0000 7f2076000640 1 -- 172.21.15.46:0/2124218014 >> v2:172.21.15.142:3300/0 conn(0x7f207815d040 msgr2=0x7f20780cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.308+0000 7f2076000640 1 --2- 172.21.15.46:0/2124218014 >> v2:172.21.15.142:3300/0 conn(0x7f207815d040 0x7f20780cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.308+0000 7f2076000640 1 -- 172.21.15.46:0/2124218014 >> v1:172.21.15.142:6789/0 conn(0x7f20780b2b60 legacy=0x7f20780ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.308+0000 7f2076000640 1 -- 172.21.15.46:0/2124218014 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780cdae0 con 0x7f20780d37e0 2024-04-26T23:43:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.309+0000 7f2076000640 1 --2- 172.21.15.46:0/2124218014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780d37e0 0x7f20780b2620 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2078073420 tx=0x7f206402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccf6909d6b7dd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.309+0000 7f2075600640 1 -- 172.21.15.46:0/2124218014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064037070 con 0x7f20780d37e0 2024-04-26T23:43:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2075600640 1 -- 172.21.15.46:0/2124218014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064032440 con 0x7f20780d37e0 2024-04-26T23:43:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2075600640 1 -- 172.21.15.46:0/2124218014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064032810 con 0x7f20780d37e0 2024-04-26T23:43:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 -- 172.21.15.46:0/2124218014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780d37e0 msgr2=0x7f20780b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 --2- 172.21.15.46:0/2124218014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780d37e0 0x7f20780b2620 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2078073420 tx=0x7f206402a230 comp rx=0 tx=0).stop 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 -- 172.21.15.46:0/2124218014 shutdown_connections 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 --2- 172.21.15.46:0/2124218014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780d37e0 0x7f20780b2620 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 --2- 172.21.15.46:0/2124218014 >> v2:172.21.15.142:3300/0 conn(0x7f207815d040 0x7f20780cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 -- 172.21.15.46:0/2124218014 >> 172.21.15.46:0/2124218014 conn(0x7f20780d2cc0 msgr2=0x7f20780d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 -- 172.21.15.46:0/2124218014 shutdown_connections 2024-04-26T23:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.310+0000 7f2077e00640 1 -- 172.21.15.46:0/2124218014 wait complete. 2024-04-26T23:43:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.312+0000 7f2077e00640 1 Processor -- start 2024-04-26T23:43:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.312+0000 7f2077e00640 1 -- start start 2024-04-26T23:43:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20780d37e0 0x7f2078157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f207815d040 0x7f2078150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20780b2270 con 0x7f20780b2b60 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f207815d420 con 0x7f207815d040 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.313+0000 7f2077e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20780b23f0 con 0x7f20780d37e0 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36814/0 (socket says 172.21.15.46:36814) 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 -- 172.21.15.46:0/3418206304 learned_addr learned my addr 172.21.15.46:0/3418206304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076000640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20780d37e0 0x7f2078157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2077400640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f207815d040 0x7f2078150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 -- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20780d37e0 msgr2=0x7f2078157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20780d37e0 0x7f2078157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 -- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f207815d040 msgr2=0x7f2078150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f207815d040 0x7f2078150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.314+0000 7f2076a00640 1 -- 172.21.15.46:0/3418206304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2078155670 con 0x7f20780b2b60 2024-04-26T23:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f2076a00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f206800b410 tx=0x7f206800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f206ea00640 1 -- 172.21.15.46:0/3418206304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068015020 con 0x7f20780b2b60 2024-04-26T23:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f206ea00640 1 -- 172.21.15.46:0/3418206304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2068011850 con 0x7f20780b2b60 2024-04-26T23:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f206ea00640 1 -- 172.21.15.46:0/3418206304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068011c40 con 0x7f20780b2b60 2024-04-26T23:43:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20781558e0 con 0x7f20780b2b60 2024-04-26T23:43:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.315+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2078155ea0 con 0x7f20780b2b60 2024-04-26T23:43:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.318+0000 7f206ea00640 1 -- 172.21.15.46:0/3418206304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2068018020 con 0x7f20780b2b60 2024-04-26T23:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.319+0000 7f206ea00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 0x7f205806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.319+0000 7f206ea00640 1 -- 172.21.15.46:0/3418206304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f206808bc80 con 0x7f20780b2b60 2024-04-26T23:43:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.320+0000 7f2080785740 1 Processor -- start 2024-04-26T23:43:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.320+0000 7f2080785740 1 -- start start 2024-04-26T23:43:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.320+0000 7f2076000640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 0x7f205806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f59910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a613ebb600 con 0x55a613f93e80 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a613e58100 con 0x55a613f82270 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a613eb75c0 con 0x55a613f577b0 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36826/0 (socket says 172.21.15.46:36826) 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 -- 172.21.15.46:0/1387835478 learned_addr learned my addr 172.21.15.46:0/1387835478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076a00640 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f59910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 -- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 msgr2=0x55a613f59910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f59910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 -- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 msgr2=0x55a613f826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2077400640 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f826f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.321+0000 7f2076000640 1 -- 172.21.15.46:0/1387835478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a613bcd2d0 con 0x55a613f93e80 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.322+0000 7f2076000640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 0x7f205806edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2064034000 tx=0x7f206402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.322+0000 7f2076000640 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f206402a270 tx=0x7f2064043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b3307c759c6689 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.322+0000 7f2063400640 1 -- 172.21.15.46:0/1387835478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206404c070 con 0x55a613f93e80 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.322+0000 7f2063400640 1 -- 172.21.15.46:0/1387835478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20640329d0 con 0x55a613f93e80 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.322+0000 7f2063400640 1 -- 172.21.15.46:0/1387835478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064037040 con 0x55a613f93e80 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 -- 172.21.15.46:0/1387835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 msgr2=0x55a613f57b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f206402a270 tx=0x7f2064043000 comp rx=0 tx=0).stop 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 -- 172.21.15.46:0/1387835478 shutdown_connections 2024-04-26T23:43:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f57b90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 --2- 172.21.15.46:0/1387835478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f59910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 -- 172.21.15.46:0/1387835478 >> 172.21.15.46:0/1387835478 conn(0x55a613eebd10 msgr2=0x55a613f9d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 -- 172.21.15.46:0/1387835478 shutdown_connections 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.323+0000 7f2080785740 1 -- 172.21.15.46:0/1387835478 wait complete. 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.324+0000 7f2080785740 1 -- start start 2024-04-26T23:43:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.324+0000 7f2080785740 1 Processor -- start 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f97a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a613e42f70 con 0x55a613f93e80 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a613f7fdc0 con 0x55a613f82270 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a613fad400 con 0x55a613f577b0 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2080785740 10 client.0 _ll_register_callbacks cb 0x55a613eb1480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36840/0 (socket says 172.21.15.46:36840) 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.325+0000 7f2077400640 1 -- 172.21.15.46:0/4287831865 learned_addr learned my addr 172.21.15.46:0/4287831865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2076a00640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 -- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 msgr2=0x55a613f7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 -- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 msgr2=0x55a613f97a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 -- 172.21.15.46:0/4287831865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a613c976b0 con 0x55a613f93e80 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2076a00640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2077400640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f207000f4a0 tx=0x7f207000f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8045eb225b8aacf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2057e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207000d070 con 0x55a613f93e80 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.326+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f2070003be0 con 0x55a613f93e80 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20640329d0 con 0x55a613f93e80 2024-04-26T23:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2070020050 con 0x55a613f93e80 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 10 client.6729 fetch_fsmap learned FSMap version 41 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55a613faceb0 con 0x55a613f93e80 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20640329d0 con 0x55a613f93e80 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7f2070023020 con 0x55a613f93e80 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 10 client.6729 fetch_fsmap finished waiting for FSMap version 41 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 10 client.6729 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 20 client.6729 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613fb06e0 0x55a613fb0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 4 client.6729 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.327+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 --> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a613ec9210 con 0x55a613fb06e0 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.328+0000 7f2076a00640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613fb06e0 0x55a613fb0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.329+0000 7f2076a00640 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613fb06e0 0x55a613fb0ac0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74352daccde7b6b server_cookie=c97eb9301768770c in_seq=0 out_seq=0 2024-04-26T23:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.329+0000 7f2057e00640 10 client.6729 ms_handle_connect on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.331+0000 7f2057e00640 1 -- 172.21.15.46:0/4287831865 <== mds.0 v2:172.21.15.142:6834/2927770154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a613ec9210 con 0x55a613fb06e0 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.331+0000 7f2057e00640 10 client.6729 handle_command_reply: tid=0 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.331+0000 7f2080785740 1 client.6729 shutdown 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.331+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 msgr2=0x55a613f97f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.331+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f207000f4a0 tx=0x7f207000f960 comp rx=0 tx=0).stop 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 shutdown_connections 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613fb06e0 0x55a613fb0ac0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613f97f60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f97a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 --2- 172.21.15.46:0/4287831865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613f7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 >> 172.21.15.46:0/4287831865 conn(0x55a613eebd10 msgr2=0x55a613f7ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2057e00640 0 client.6729 ms_handle_reset on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:06.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 shutdown_connections 2024-04-26T23:43:06.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 1 -- 172.21.15.46:0/4287831865 wait complete. 2024-04-26T23:43:06.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.332+0000 7f2080785740 20 client.6729 trim_cache size 0 max 16384 2024-04-26T23:43:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.352+0000 7f2080785740 1 Processor -- start 2024-04-26T23:43:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.352+0000 7f2080785740 1 -- start start 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 0x55a613fafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f93e80 0x55a613f82650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2078200e70 con 0x55a613f577b0 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20780c8ad0 con 0x55a613f93e80 2024-04-26T23:43:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20780c9120 con 0x55a613f82270 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36842/0 (socket says 172.21.15.46:36842) 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.353+0000 7f2076a00640 1 -- 172.21.15.46:0/1250303555 learned_addr learned my addr 172.21.15.46:0/1250303555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076000640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 0x55a613fafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2077400640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f93e80 0x55a613f82650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 -- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 msgr2=0x55a613fafbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 0x55a613fafbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 -- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f93e80 msgr2=0x55a613f82650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f93e80 0x55a613f82650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 -- 172.21.15.46:0/1250303555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a613f55b10 con 0x55a613f577b0 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076000640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 0x55a613fafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:06.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.354+0000 7f2076a00640 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2068059510 tx=0x7f2068060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644b126f6823d034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2062a00640 1 -- 172.21.15.46:0/1250303555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068087070 con 0x55a613f577b0 2024-04-26T23:43:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2062a00640 1 -- 172.21.15.46:0/1250303555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2068011de0 con 0x55a613f577b0 2024-04-26T23:43:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2062a00640 1 -- 172.21.15.46:0/1250303555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206808a090 con 0x55a613f577b0 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 -- 172.21.15.46:0/1250303555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 msgr2=0x55a613f853c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2068059510 tx=0x7f2068060040 comp rx=0 tx=0).stop 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 -- 172.21.15.46:0/1250303555 shutdown_connections 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f93e80 0x55a613f82650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f82270 0x55a613fafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 --2- 172.21.15.46:0/1250303555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f577b0 0x55a613f853c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 -- 172.21.15.46:0/1250303555 >> 172.21.15.46:0/1250303555 conn(0x55a613eebd10 msgr2=0x55a613f837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 -- 172.21.15.46:0/1250303555 shutdown_connections 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.355+0000 7f2080785740 1 -- 172.21.15.46:0/1250303555 wait complete. 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.356+0000 7f2080785740 1 -- start start 2024-04-26T23:43:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.356+0000 7f2080785740 1 Processor -- start 2024-04-26T23:43:06.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613fab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a613f83660 con 0x55a613f93e80 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a613fad190 con 0x55a613f82270 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a613f968c0 con 0x55a613f577b0 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.357+0000 7f2080785740 10 client.0 _ll_register_callbacks cb 0x7f2070004c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36854/0 (socket says 172.21.15.46:36854) 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 -- 172.21.15.46:0/3735212448 learned_addr learned my addr 172.21.15.46:0/3735212448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2076a00640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613fab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 -- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 msgr2=0x55a613fab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613fab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 -- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 msgr2=0x55a613f9fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 -- 172.21.15.46:0/3735212448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a613f84d60 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2077400640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x55a613f975f0 tx=0x7f2070003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d4ce3d1d0fe216 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.358+0000 7f2057e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207000d040 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f207002d3a0 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2068011de0 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2070034050 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 10 client.6735 fetch_fsmap learned FSMap version 41 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55a613f83030 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2068011de0 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7f2070037020 con 0x55a613f93e80 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 10 client.6735 fetch_fsmap finished waiting for FSMap version 41 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 10 client.6735 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 20 client.6735 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613f92c00 0x55a613f92fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 4 client.6735 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:43:06.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.359+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 --> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] -- command(tid 0: {"prefix": "status"}) -- 0x55a613ec3080 con 0x55a613f92c00 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.360+0000 7f2076a00640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613f92c00 0x55a613f92fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.361+0000 7f2076a00640 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613f92c00 0x55a613f92fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb76d66e80185b84 server_cookie=8dfdc815e8c1758d in_seq=0 out_seq=0 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.361+0000 7f2057e00640 10 client.6735 ms_handle_connect on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.361+0000 7f2057e00640 1 -- 172.21.15.46:0/3735212448 <== mds.0 v2:172.21.15.142:6834/2927770154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a613ec3080 con 0x55a613f92c00 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.361+0000 7f2057e00640 10 client.6735 handle_command_reply: tid=0 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 client.6735 shutdown 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 msgr2=0x55a613fa03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x55a613f975f0 tx=0x7f2070003ae0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 shutdown_connections 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x55a613f92c00 0x55a613f92fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a613f93e80 0x55a613fa03b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a613f82270 0x55a613f9fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 --2- 172.21.15.46:0/3735212448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a613f577b0 0x55a613fab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2057e00640 0 client.6735 ms_handle_reset on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 >> 172.21.15.46:0/3735212448 conn(0x55a613eebd10 msgr2=0x55a613f7fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 shutdown_connections 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 1 -- 172.21.15.46:0/3735212448 wait complete. 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.362+0000 7f2080785740 20 client.6735 trim_cache size 0 max 16384 2024-04-26T23:43:06.367 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6591, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7830739900000001, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 41, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 65, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 65, 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.173882770000001 2024-04-26T23:43:06.368 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 msgr2=0x7f205806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 0x7f205806edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2064034000 tx=0x7f206402e000 comp rx=0 tx=0).stop 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 msgr2=0x7f207806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f206800b410 tx=0x7f206800b8d0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 shutdown_connections 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205806c8f0 0x7f205806edb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f207815d040 0x7f2078150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20780d37e0 0x7f2078157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 --2- 172.21.15.46:0/3418206304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20780b2b60 0x7f207806da90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 >> 172.21.15.46:0/3418206304 conn(0x7f20780d2cc0 msgr2=0x7f207815c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 shutdown_connections 2024-04-26T23:43:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.363+0000 7f2077e00640 1 -- 172.21.15.46:0/3418206304 wait complete. 2024-04-26T23:43:06.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.590+0000 7f30f7200640 1 Processor -- start 2024-04-26T23:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.590+0000 7f30f7200640 1 -- start start 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30f8159950 0x7f30f80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30f81631d0 0x7f30f815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f30f8163640 con 0x7f30f81631d0 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f30f80af0c0 con 0x7f30f8159950 2024-04-26T23:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30f80af3a0 con 0x7f30f80ae8e0 2024-04-26T23:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30f81631d0 0x7f30f815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30f81631d0 0x7f30f815f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36864/0 (socket says 172.21.15.46:36864) 2024-04-26T23:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.592+0000 7f30f6800640 1 -- 172.21.15.46:0/2812121610 learned_addr learned my addr 172.21.15.46:0/2812121610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f5e00640 1 --2- 172.21.15.46:0/2812121610 >> v2:172.21.15.142:3300/0 conn(0x7f30f8159950 0x7f30f80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f5400640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f6800640 1 -- 172.21.15.46:0/2812121610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 msgr2=0x7f30f8162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f6800640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f6800640 1 -- 172.21.15.46:0/2812121610 >> v2:172.21.15.142:3300/0 conn(0x7f30f8159950 msgr2=0x7f30f80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f6800640 1 --2- 172.21.15.46:0/2812121610 >> v2:172.21.15.142:3300/0 conn(0x7f30f8159950 0x7f30f80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.593+0000 7f30f6800640 1 -- 172.21.15.46:0/2812121610 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80af680 con 0x7f30f81631d0 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.594+0000 7f30f6800640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f81631d0 0x7f30f815f6a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f30f806dda0 tx=0x7f30f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99869e974413c751 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.594+0000 7f30f4a00640 1 -- 172.21.15.46:0/2812121610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0037070 con 0x7f30f81631d0 2024-04-26T23:43:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.594+0000 7f30f4a00640 1 -- 172.21.15.46:0/2812121610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f0032440 con 0x7f30f81631d0 2024-04-26T23:43:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.594+0000 7f30f4a00640 1 -- 172.21.15.46:0/2812121610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0032810 con 0x7f30f81631d0 2024-04-26T23:43:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 -- 172.21.15.46:0/2812121610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f81631d0 msgr2=0x7f30f815f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f81631d0 0x7f30f815f6a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f30f806dda0 tx=0x7f30f002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 -- 172.21.15.46:0/2812121610 shutdown_connections 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f81631d0 0x7f30f815f6a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 --2- 172.21.15.46:0/2812121610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 --2- 172.21.15.46:0/2812121610 >> v2:172.21.15.142:3300/0 conn(0x7f30f8159950 0x7f30f80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 -- 172.21.15.46:0/2812121610 >> 172.21.15.46:0/2812121610 conn(0x7f30f80b2820 msgr2=0x7f30f80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 -- 172.21.15.46:0/2812121610 shutdown_connections 2024-04-26T23:43:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.595+0000 7f30f7200640 1 -- 172.21.15.46:0/2812121610 wait complete. 2024-04-26T23:43:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.597+0000 7f30f7200640 1 Processor -- start 2024-04-26T23:43:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.597+0000 7f30f7200640 1 -- start start 2024-04-26T23:43:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30f81631d0 0x7f30f814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80aed70 con 0x7f30f814bbb0 2024-04-26T23:43:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80afc40 con 0x7f30f81631d0 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.598+0000 7f30f7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30f80b0160 con 0x7f30f80ae8e0 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36866/0 (socket says 172.21.15.46:36866) 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 -- 172.21.15.46:0/1951739677 learned_addr learned my addr 172.21.15.46:0/1951739677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f5e00640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 -- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 msgr2=0x7f30f8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 -- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30f81631d0 msgr2=0x7f30f814b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30f81631d0 0x7f30f814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 -- 172.21.15.46:0/1951739677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f815a900 con 0x7f30f814bbb0 2024-04-26T23:43:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f5e00640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.599+0000 7f30f6800640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f30f002a220 tx=0x7f30f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.600+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0037040 con 0x7f30f814bbb0 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.600+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f0042070 con 0x7f30f814bbb0 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.600+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0030900 con 0x7f30f814bbb0 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.600+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f815abe0 con 0x7f30f814bbb0 2024-04-26T23:43:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.600+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f814f610 con 0x7f30f814bbb0 2024-04-26T23:43:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.601+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f80693b0 con 0x7f30f814bbb0 2024-04-26T23:43:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.602+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30f0030aa0 con 0x7f30f814bbb0 2024-04-26T23:43:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.603+0000 7f30ee000640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 0x7f30d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.603+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f30f00ac570 con 0x7f30f814bbb0 2024-04-26T23:43:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.604+0000 7f30f5e00640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 0x7f30d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.606+0000 7f30f5e00640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 0x7f30d806ede0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f30e8006fd0 tx=0x7f30e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:06.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.609+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30f007a1b0 con 0x7f30f814bbb0 2024-04-26T23:43:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.859+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30f80b35e0 con 0x7f30f814bbb0 2024-04-26T23:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.860+0000 7f30ee000640 1 -- 172.21.15.46:0/1951739677 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f30f007e4b0 con 0x7f30f814bbb0 2024-04-26T23:43:06.863 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:43:06.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.862+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 msgr2=0x7f30d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.862+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 0x7f30d806ede0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f30e8006fd0 tx=0x7f30e8008040 comp rx=0 tx=0).stop 2024-04-26T23:43:06.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.862+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 msgr2=0x7f30f815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:06.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.862+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f30f002a220 tx=0x7f30f00046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 shutdown_connections 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30d806c920 0x7f30d806ede0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f814bbb0 0x7f30f815c270 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30f81631d0 0x7f30f814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 --2- 172.21.15.46:0/1951739677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30f80ae8e0 0x7f30f8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 >> 172.21.15.46:0/1951739677 conn(0x7f30f80b2820 msgr2=0x7f30f80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 shutdown_connections 2024-04-26T23:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:06.863+0000 7f30f7200640 1 -- 172.21.15.46:0/1951739677 wait complete. 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:43:06.886 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:06.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:06.887 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:43:06.942 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:43:06.942 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:43:06.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:43:06.942 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42234sec preferred_lft 42234sec 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:43:06.943 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:43:06.944 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:06.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:06.945 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:43:07.028 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:43:07.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:07.033 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:43:07.088 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:43:07.089 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:43:07.089 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:43:07.089 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:43:07.089 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:43:07.090 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:43:07.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:43:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:07.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:07.315 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:43:07.366 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:43:07.366 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:43:07.367 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:07.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:07.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:43:07.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:43:07.458 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:07.515 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:07.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:43:07.515 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:07.567 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:43:07.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:07.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:07.695 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:07.695 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:07.696 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:07.719 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:07.719 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:07.719 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:07.719 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:43:07.719 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:43:07.747 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:43:07.748 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:43:07.749 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:43:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:43:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:43:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:43:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:43:07.753 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:43:07.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:07.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:07.828 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:43:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:43:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:07.972 INFO:teuthology.orchestra.run.smithi046.stdout:client6744 2024-04-26T23:43:07.973 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:07.973 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:08.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_size 20 2024-04-26T23:43:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.244+0000 7f4ead800640 1 Processor -- start 2024-04-26T23:43:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.244+0000 7f4ead800640 1 -- start start 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.245+0000 7f4ead800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ea815f210 0x7f4ea80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.245+0000 7f4ead800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 0x7f4ea80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ead800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea80af570 con 0x7f4ea80ae8f0 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ead800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea80cdfb0 con 0x7f4ea815f210 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ead800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea80af850 con 0x7f4ea80cd610 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ea815f210 0x7f4ea80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ea815f210 0x7f4ea80cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36872/0 (socket says 172.21.15.46:36872) 2024-04-26T23:43:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea7e00640 1 -- 172.21.15.46:0/2887866385 learned_addr learned my addr 172.21.15.46:0/2887866385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea7e00640 1 -- 172.21.15.46:0/2887866385 >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 msgr2=0x7f4ea80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea7400640 1 --2- 172.21.15.46:0/2887866385 >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 0x7f4ea80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.247+0000 7f4ea7e00640 1 --2- 172.21.15.46:0/2887866385 >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 0x7f4ea80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.247+0000 7f4ea7e00640 1 -- 172.21.15.46:0/2887866385 >> v1:172.21.15.46:6789/0 conn(0x7f4ea80ae8f0 legacy=0x7f4ea80aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.246+0000 7f4ea6a00640 1 -- 172.21.15.46:0/2887866385 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819231791 0 0) 0x7f4ea80af570 con 0x7f4ea80ae8f0 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.247+0000 7f4ea7e00640 1 -- 172.21.15.46:0/2887866385 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80cb680 con 0x7f4ea815f210 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.247+0000 7f4ea7400640 1 --2- 172.21.15.46:0/2887866385 >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 0x7f4ea80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.248+0000 7f4ea7e00640 1 --2- 172.21.15.46:0/2887866385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea80cc160 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80b3710 tx=0x7f4e9802a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f73291925526032 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.248+0000 7f4ea6a00640 1 -- 172.21.15.46:0/2887866385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98037070 con 0x7f4ea815f210 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.248+0000 7f4ea6a00640 1 -- 172.21.15.46:0/2887866385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98032440 con 0x7f4ea815f210 2024-04-26T23:43:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.248+0000 7f4ea6a00640 1 -- 172.21.15.46:0/2887866385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98032810 con 0x7f4ea815f210 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 -- 172.21.15.46:0/2887866385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 msgr2=0x7f4ea80cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 --2- 172.21.15.46:0/2887866385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea80cc160 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80b3710 tx=0x7f4e9802a280 comp rx=0 tx=0).stop 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 -- 172.21.15.46:0/2887866385 shutdown_connections 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 --2- 172.21.15.46:0/2887866385 >> v2:172.21.15.142:3300/0 conn(0x7f4ea80cd610 0x7f4ea80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 --2- 172.21.15.46:0/2887866385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea80cc160 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 -- 172.21.15.46:0/2887866385 >> 172.21.15.46:0/2887866385 conn(0x7f4ea80b27b0 msgr2=0x7f4ea80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 -- 172.21.15.46:0/2887866385 shutdown_connections 2024-04-26T23:43:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.249+0000 7f4ead800640 1 -- 172.21.15.46:0/2887866385 wait complete. 2024-04-26T23:43:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.251+0000 7f4ead800640 1 Processor -- start 2024-04-26T23:43:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.251+0000 7f4ead800640 1 -- start start 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea80ae8f0 0x7f4ea80c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea80cd610 0x7f4ea81e4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea815f5f0 con 0x7f4ea815f210 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea80cbc40 con 0x7f4ea80ae8f0 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ead800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea8071e80 con 0x7f4ea80cd610 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36878/0 (socket says 172.21.15.46:36878) 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 -- 172.21.15.46:0/2682826194 learned_addr learned my addr 172.21.15.46:0/2682826194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ea7e00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea80ae8f0 0x7f4ea80c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4ea7400640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea80cd610 0x7f4ea81e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 -- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea80cd610 msgr2=0x7f4ea81e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea80cd610 0x7f4ea81e4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 -- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea80ae8f0 msgr2=0x7f4ea80c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea80ae8f0 0x7f4ea80c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.252+0000 7f4eace00640 1 -- 172.21.15.46:0/2682826194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81e4a50 con 0x7f4ea815f210 2024-04-26T23:43:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4eace00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c780 tx=0x7f4ea000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0003da0 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea000ce30 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0005290 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea80c8360 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.253+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea80c8920 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.254+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea80693b0 con 0x7f4ea815f210 2024-04-26T23:43:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.255+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ea0005430 con 0x7f4ea815f210 2024-04-26T23:43:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.257+0000 7f4e9fe00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.257+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f4ea008bca0 con 0x7f4ea815f210 2024-04-26T23:43:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.257+0000 7f4ea7e00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.259+0000 7f4ea7e00640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 0x7f4e8806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e98034000 tx=0x7f4e9802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.261+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ea0059ea0 con 0x7f4ea815f210 2024-04-26T23:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.466+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f4ea80cf840 con 0x7f4ea815f210 2024-04-26T23:43:08.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.472+0000 7f4e9fe00640 1 -- 172.21.15.46:0/2682826194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7f4ea00597a0 con 0x7f4ea815f210 2024-04-26T23:43:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 msgr2=0x7f4e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 0x7f4e8806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e98034000 tx=0x7f4e9802e000 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 msgr2=0x7f4ea814f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c780 tx=0x7f4ea000cc40 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 shutdown_connections 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ea815f210 0x7f4ea814f080 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea80cd610 0x7f4ea81e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 --2- 172.21.15.46:0/2682826194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea80ae8f0 0x7f4ea80c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 >> 172.21.15.46:0/2682826194 conn(0x7f4ea80b27b0 msgr2=0x7f4ea80ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 shutdown_connections 2024-04-26T23:43:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.475+0000 7f4ead800640 1 -- 172.21.15.46:0/2682826194 wait complete. 2024-04-26T23:43:08.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_merge_size 5 2024-04-26T23:43:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.703+0000 7ff9c1400640 1 Processor -- start 2024-04-26T23:43:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.704+0000 7ff9c1400640 1 -- start start 2024-04-26T23:43:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.705+0000 7ff9c1400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff9bc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.705+0000 7ff9c1400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9bc162aa0 0x7ff9bc0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.705+0000 7ff9c1400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9bc0d07d0 con 0x7ff9bc1626c0 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.705+0000 7ff9c1400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff9bc0d0ab0 con 0x7ff9bc162aa0 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.705+0000 7ff9c1400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d0d90 con 0x7ff9bc0b5fb0 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9bbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9bc1626c0 0x7ff9bc0d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45248/0 (socket says 172.21.15.46:45248) 2024-04-26T23:43:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9bbe00640 1 -- 172.21.15.46:0/3616529631 learned_addr learned my addr 172.21.15.46:0/3616529631 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9bb400640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff9bc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829459296 0 0) 0x7ff9bc0d07d0 con 0x7ff9bc1626c0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> v2:172.21.15.142:3300/0 conn(0x7ff9bc162aa0 0x7ff9bc0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff99c003520 con 0x7ff9bc1626c0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 283812866 0 0) 0x7ff99c003520 con 0x7ff9bc1626c0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 msgr2=0x7ff9bc0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff9bc0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 >> v2:172.21.15.142:3300/0 conn(0x7ff9bc162aa0 msgr2=0x7ff9bc0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 --2- 172.21.15.46:0/3616529631 >> v2:172.21.15.142:3300/0 conn(0x7ff9bc162aa0 0x7ff9bc0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.706+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc0d2c80 con 0x7ff9bc1626c0 2024-04-26T23:43:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff9ac001670 con 0x7ff9bc1626c0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 >> v1:172.21.15.46:6789/0 conn(0x7ff9bc1626c0 legacy=0x7ff9bc0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff99c0039b0 0x7ff99c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff99c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff99c004d40 0x7ff99c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d0ab0 con 0x7ff99c004d40 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d0d90 con 0x7ff99c0039b0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c00ba70 con 0x7ff9bc0b5fb0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> v2:172.21.15.142:3300/0 conn(0x7ff9bc162aa0 0x7ff9bc0d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff99c004d40 0x7ff99c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9bbe00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff99c0039b0 0x7ff99c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.707+0000 7ff9bb400640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff99c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.708+0000 7ff9c0a00640 1 -- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 msgr2=0x7ff99c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.708+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0b5fb0 0x7ff99c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.708+0000 7ff9c0a00640 1 -- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff99c0039b0 msgr2=0x7ff99c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.708+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff99c0039b0 0x7ff99c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.708+0000 7ff9c0a00640 1 -- 172.21.15.46:0/3616529631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c003520 con 0x7ff99c004d40 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.709+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff99c004d40 0x7ff99c0051c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc162f10 tx=0x7ff99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510b99415cc82b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400f040 con 0x7ff99c004d40 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b402ad00 con 0x7ff99c004d40 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9baa00640 1 -- 172.21.15.46:0/3616529631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4004270 con 0x7ff99c004d40 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 -- 172.21.15.46:0/3616529631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff99c004d40 msgr2=0x7ff99c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3616529631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff99c004d40 0x7ff99c0051c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc162f10 tx=0x7ff99c005830 comp rx=0 tx=0).stop 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c0a00640 1 -- 172.21.15.46:0/3616529631 reap_dead start 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 -- 172.21.15.46:0/3616529631 shutdown_connections 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 -- 172.21.15.46:0/3616529631 >> 172.21.15.46:0/3616529631 conn(0x7ff9bc09e2a0 msgr2=0x7ff9bc0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 -- 172.21.15.46:0/3616529631 shutdown_connections 2024-04-26T23:43:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.710+0000 7ff9c1400640 1 -- 172.21.15.46:0/3616529631 wait complete. 2024-04-26T23:43:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.712+0000 7ff9c1400640 1 Processor -- start 2024-04-26T23:43:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.712+0000 7ff9c1400640 1 -- start start 2024-04-26T23:43:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9bc1550a0 0x7ff9bc0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0cce60 0x7ff9bc1614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0b4b50 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0b49d0 con 0x7ff9bc1550a0 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.713+0000 7ff9c1400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d1e40 con 0x7ff9bc0cce60 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36898/0 (socket says 172.21.15.46:36898) 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 -- 172.21.15.46:0/3705611451 learned_addr learned my addr 172.21.15.46:0/3705611451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bbe00640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9bc1550a0 0x7ff9bc0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9c0a00640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0cce60 0x7ff9bc1614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 -- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0cce60 msgr2=0x7ff9bc1614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0cce60 0x7ff9bc1614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 -- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9bc1550a0 msgr2=0x7ff9bc0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9bc1550a0 0x7ff9bc0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.714+0000 7ff9bb400640 1 -- 172.21.15.46:0/3705611451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc0cc0a0 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9bb400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4007f00 tx=0x7ff9a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a401a020 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a4016470 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4016840 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc0cc380 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.715+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc0cb700 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.716+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9bc0693b0 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.718+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9a401d020 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.718+0000 7ff9b3e00640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.719+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff9a408cc90 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.719+0000 7ff9bbe00640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:08.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.720+0000 7ff9bbe00640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 0x7ff99806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac001eb0 tx=0x7ff9ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.724+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9a4024020 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.927+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7ff9bc0d25b0 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.932+0000 7ff9b3e00640 1 -- 172.21.15.46:0/3705611451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v16)=0 v16) ==== 116+0+0 (secure 0 0 0) 0x7ff9a405ae90 con 0x7ff9bc0cdc30 2024-04-26T23:43:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 msgr2=0x7ff99806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 0x7ff99806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac001eb0 tx=0x7ff9ac009040 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 msgr2=0x7ff9bc160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4007f00 tx=0x7ff9a40115a0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 shutdown_connections 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9bc0cce60 0x7ff9bc1614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc0cdc30 0x7ff9bc160fa0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 --2- 172.21.15.46:0/3705611451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9bc1550a0 0x7ff9bc0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 >> 172.21.15.46:0/3705611451 conn(0x7ff9bc09e2a0 msgr2=0x7ff9bc06de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 shutdown_connections 2024-04-26T23:43:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:08.936+0000 7ff9c1400640 1 -- 172.21.15.46:0/3705611451 wait complete. 2024-04-26T23:43:08.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_bits 1 2024-04-26T23:43:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.172+0000 7ff303200640 1 Processor -- start 2024-04-26T23:43:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.172+0000 7ff303200640 1 -- start start 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.173+0000 7ff303200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff304160130 0x7ff304160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.173+0000 7ff303200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff30415cc40 0x7ff3040cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.173+0000 7ff303200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3040cbe60 con 0x7ff30415d0a0 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.173+0000 7ff303200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff3040cca70 con 0x7ff304160130 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.173+0000 7ff303200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff3040cc140 con 0x7ff30415cc40 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff304160130 0x7ff304160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff304160130 0x7ff304160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36906/0 (socket says 172.21.15.46:36906) 2024-04-26T23:43:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 -- 172.21.15.46:0/2049671588 learned_addr learned my addr 172.21.15.46:0/2049671588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 -- 172.21.15.46:0/2049671588 >> v2:172.21.15.142:3300/0 conn(0x7ff30415cc40 msgr2=0x7ff3040cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 --2- 172.21.15.46:0/2049671588 >> v2:172.21.15.142:3300/0 conn(0x7ff30415cc40 0x7ff3040cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 -- 172.21.15.46:0/2049671588 >> v1:172.21.15.142:6789/0 conn(0x7ff30415d0a0 legacy=0x7ff3040cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.174+0000 7ff301e00640 1 -- 172.21.15.46:0/2049671588 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3040cc420 con 0x7ff304160130 2024-04-26T23:43:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.175+0000 7ff301e00640 1 --2- 172.21.15.46:0/2049671588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304160130 0x7ff304160510 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ce780 tx=0x7ff2f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3794e94044682a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff301400640 1 -- 172.21.15.46:0/2049671588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4037070 con 0x7ff304160130 2024-04-26T23:43:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff301400640 1 -- 172.21.15.46:0/2049671588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4032440 con 0x7ff304160130 2024-04-26T23:43:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff301400640 1 -- 172.21.15.46:0/2049671588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4032810 con 0x7ff304160130 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 -- 172.21.15.46:0/2049671588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304160130 msgr2=0x7ff304160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 --2- 172.21.15.46:0/2049671588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304160130 0x7ff304160510 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ce780 tx=0x7ff2f402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 -- 172.21.15.46:0/2049671588 shutdown_connections 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 --2- 172.21.15.46:0/2049671588 >> v2:172.21.15.142:3300/0 conn(0x7ff30415cc40 0x7ff3040cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 --2- 172.21.15.46:0/2049671588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304160130 0x7ff304160510 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.176+0000 7ff303200640 1 -- 172.21.15.46:0/2049671588 >> 172.21.15.46:0/2049671588 conn(0x7ff3040b27b0 msgr2=0x7ff3040b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.177+0000 7ff303200640 1 -- 172.21.15.46:0/2049671588 shutdown_connections 2024-04-26T23:43:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.177+0000 7ff303200640 1 -- 172.21.15.46:0/2049671588 wait complete. 2024-04-26T23:43:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.178+0000 7ff303200640 1 Processor -- start 2024-04-26T23:43:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.178+0000 7ff303200640 1 -- start start 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30415d0a0 0x7ff30414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 0x7ff3040c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3040733b0 con 0x7ff30415cc40 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff304073230 con 0x7ff304160130 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.179+0000 7ff303200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff304073530 con 0x7ff30415d0a0 2024-04-26T23:43:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36922/0 (socket says 172.21.15.46:36922) 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 -- 172.21.15.46:0/2814632888 learned_addr learned my addr 172.21.15.46:0/2814632888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff302800640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 0x7ff3040c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 -- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30415d0a0 msgr2=0x7ff30414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30415d0a0 0x7ff30414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 -- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 msgr2=0x7ff3040c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 0x7ff3040c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff301e00640 1 -- 172.21.15.46:0/2814632888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3040c7660 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.180+0000 7ff302800640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 0x7ff3040c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff301e00640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4041070 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f403c890 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3040c7380 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4037040 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.181+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3040c8050 con 0x7ff30415cc40 2024-04-26T23:43:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.182+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3040693b0 con 0x7ff30415cc40 2024-04-26T23:43:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.183+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2f4044070 con 0x7ff30415cc40 2024-04-26T23:43:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.184+0000 7ff2faa00640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 0x7ff2e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.185+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff2f40ac9c0 con 0x7ff30415cc40 2024-04-26T23:43:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.185+0000 7ff2f9400640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 0x7ff2e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.187+0000 7ff2f9400640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 0x7ff2e006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3040c5850 tx=0x7ff2ec006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.189+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2f407ad70 con 0x7ff30415cc40 2024-04-26T23:43:09.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:09.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.406+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7ff3040cbb10 con 0x7ff30415cc40 2024-04-26T23:43:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.415+0000 7ff2faa00640 1 -- 172.21.15.46:0/2814632888 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7ff2f407a670 con 0x7ff30415cc40 2024-04-26T23:43:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 msgr2=0x7ff2e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 0x7ff2e006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3040c5850 tx=0x7ff2ec006c90 comp rx=0 tx=0).stop 2024-04-26T23:43:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 msgr2=0x7ff3041f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 shutdown_connections 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2e006c8f0 0x7ff2e006edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304160130 0x7ff3040c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30415d0a0 0x7ff30414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 --2- 172.21.15.46:0/2814632888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30415cc40 0x7ff3041f4aa0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 >> 172.21.15.46:0/2814632888 conn(0x7ff3040b27b0 msgr2=0x7ff30415add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 shutdown_connections 2024-04-26T23:43:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.418+0000 7ff303200640 1 -- 172.21.15.46:0/2814632888 wait complete. 2024-04-26T23:43:09.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:09.441 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir").st_ino) 2024-04-26T23:43:09.508 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:43:09.656 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-26T23:43:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:09.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.925+0000 7f6ddb200640 1 Processor -- start 2024-04-26T23:43:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.925+0000 7f6ddb200640 1 -- start start 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.926+0000 7f6ddb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc0d2130 0x7f6ddc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.926+0000 7f6ddb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ddc0b4000 0x7f6ddc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.926+0000 7f6ddb200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0d11f0 con 0x7f6ddc15f4e0 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.926+0000 7f6ddb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0d0ce0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.926+0000 7f6ddb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d2990 con 0x7f6ddc0d2130 2024-04-26T23:43:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.927+0000 7f6dda800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ddc0b4000 0x7f6ddc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.927+0000 7f6dda800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ddc0b4000 0x7f6ddc0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36930/0 (socket says 172.21.15.46:36930) 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.927+0000 7f6dda800640 1 -- 172.21.15.46:0/4045951352 learned_addr learned my addr 172.21.15.46:0/4045951352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.927+0000 7f6dd1400640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc0d2130 0x7f6ddc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.928+0000 7f6dda800640 1 -- 172.21.15.46:0/4045951352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc0d2130 msgr2=0x7f6ddc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.928+0000 7f6dda800640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc0d2130 0x7f6ddc0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.928+0000 7f6dda800640 1 -- 172.21.15.46:0/4045951352 >> v1:172.21.15.142:6789/0 conn(0x7f6ddc15f4e0 legacy=0x7f6ddc0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.928+0000 7f6dda800640 1 -- 172.21.15.46:0/4045951352 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0d5f20 con 0x7f6ddc0b4000 2024-04-26T23:43:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.929+0000 7f6dda800640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc0b2280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d9f7d650d72e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.929+0000 7f6dd9400640 1 -- 172.21.15.46:0/4045951352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4037070 con 0x7f6ddc0b4000 2024-04-26T23:43:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.929+0000 7f6dd9400640 1 -- 172.21.15.46:0/4045951352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4032440 con 0x7f6ddc0b4000 2024-04-26T23:43:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.929+0000 7f6dd9400640 1 -- 172.21.15.46:0/4045951352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4032810 con 0x7f6ddc0b4000 2024-04-26T23:43:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 -- 172.21.15.46:0/4045951352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 msgr2=0x7f6ddc0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc0b2280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd402a270 comp rx=0 tx=0).stop 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 -- 172.21.15.46:0/4045951352 shutdown_connections 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc0b2280 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 --2- 172.21.15.46:0/4045951352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc0d2130 0x7f6ddc0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 -- 172.21.15.46:0/4045951352 >> 172.21.15.46:0/4045951352 conn(0x7f6ddc0d5010 msgr2=0x7f6ddc0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 -- 172.21.15.46:0/4045951352 shutdown_connections 2024-04-26T23:43:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.930+0000 7f6ddb200640 1 -- 172.21.15.46:0/4045951352 wait complete. 2024-04-26T23:43:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.932+0000 7f6ddb200640 1 Processor -- start 2024-04-26T23:43:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.933+0000 7f6ddb200640 1 -- start start 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.933+0000 7f6ddb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.933+0000 7f6ddb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ddc0d2130 0x7f6ddc0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6ddb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc15f4e0 0x7f6ddc0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6ddb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0b1d60 con 0x7f6ddc0b4000 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6ddb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d5c40 con 0x7f6ddc0d2130 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6ddb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc15f8c0 con 0x7f6ddc15f4e0 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36942/0 (socket says 172.21.15.46:36942) 2024-04-26T23:43:09.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 -- 172.21.15.46:0/3540409042 learned_addr learned my addr 172.21.15.46:0/3540409042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd1400640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ddc0d2130 0x7f6ddc0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 -- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc15f4e0 msgr2=0x7f6ddc0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dda800640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc15f4e0 0x7f6ddc0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc15f4e0 0x7f6ddc0ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 -- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ddc0d2130 msgr2=0x7f6ddc0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ddc0d2130 0x7f6ddc0ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.934+0000 7f6dd9e00640 1 -- 172.21.15.46:0/3540409042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc160900 con 0x7f6ddc0b4000 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6dd9e00640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c730 tx=0x7f6dcc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc016020 con 0x7f6ddc0b4000 2024-04-26T23:43:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc006d50 con 0x7f6ddc0b4000 2024-04-26T23:43:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc003bb0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc160be0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.935+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1611a0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.936+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc1614e0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.938+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dcc00a850 con 0x7f6ddc0b4000 2024-04-26T23:43:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.939+0000 7f6dd2a00640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.939+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f6dcc08b9d0 con 0x7f6ddc0b4000 2024-04-26T23:43:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.939+0000 7f6dd1400640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:09.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.941+0000 7f6dd1400640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 0x7f6db806edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc161dc0 tx=0x7f6dc4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:09.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:09.943+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ddc1614e0 con 0x7f6ddc0b4000 2024-04-26T23:43:10.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.189+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ddc0693b0 con 0x7f6ddc0b4000 2024-04-26T23:43:10.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.190+0000 7f6dd2a00640 1 -- 172.21.15.46:0/3540409042 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5188 (secure 0 0 0) 0x7f6dcc020020 con 0x7f6ddc0b4000 2024-04-26T23:43:10.193 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:10.193 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3235774034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3235774034},{"type":"v1","addr":"172.21.15.142:6837","nonce":3235774034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6579,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3040630531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3040630531},{"type":"v1","addr":"172.21.15.46:6833","nonce":3040630531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6582,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1818830260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1818830260},{"type":"v1","addr":"172.21.15.46:6835","nonce":1818830260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:02.129282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2927770154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2927770154},{"type":"v1","addr":"172.21.15.142:6835","nonce":2927770154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":4}]} 2024-04-26T23:43:10.193 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 41 2024-04-26T23:43:10.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.192+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 msgr2=0x7f6db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:10.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.192+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 0x7f6db806edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc161dc0 tx=0x7f6dc4008040 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 msgr2=0x7f6ddc1620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c730 tx=0x7f6dcc00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 shutdown_connections 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ddc15f4e0 0x7f6ddc0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ddc0d2130 0x7f6ddc0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 --2- 172.21.15.46:0/3540409042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0b4000 0x7f6ddc1620a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 >> 172.21.15.46:0/3540409042 conn(0x7f6ddc0d5010 msgr2=0x7f6ddc0b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 shutdown_connections 2024-04-26T23:43:10.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10.193+0000 7f6ddb200640 1 -- 172.21.15.46:0/3540409042 wait complete. 2024-04-26T23:43:10.214 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump inode 1099511627776 2024-04-26T23:43:10.440 INFO:teuthology.orchestra.run.smithi142.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:43:09.496295+0000","btime":"2024-04-26T23:43:09.496295+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:43:09.496295+0000","atime":"2024-04-26T23:43:09.496295+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","type":"iauth","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","type":"ixattr","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6744,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6744,"want_loner":6744,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:43:10.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:43:09.496295+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "iauth", "xlock_by": {} }, "backtrace_version": 2, "btime": "2024-04-26T23:43:09.496295+0000", "change_attr": 0, "client_caps": [ { "client_id": 6744, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T23:43:09.496295+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6744, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:43:09.496295+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 6744, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ixattr", "xlock_by": {} }, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:43:10.441 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:10.441 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:10.499 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:43:10.499 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:43:10.499 DEBUG:teuthology.orchestra.run.smithi046:> import uuid 2024-04-26T23:43:10.499 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:10.499 DEBUG:teuthology.orchestra.run.smithi046:> n = 40 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = False 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> if 0 > 0: 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = True 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file" 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> dpath = os.path.dirname(path) 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> fnameprefix = os.path.basename(path) 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> os.makedirs(dpath, exist_ok=True) 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> for i in range(n): 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T23:43:10.500 DEBUG:teuthology.orchestra.run.smithi046:> f.write(f"{i}") 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> f.flush() 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(f.fileno()) 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.unlink(fpath) 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> if create_hard_links: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> for j in range(0): 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> finally: 2024-04-26T23:43:10.501 DEBUG:teuthology.orchestra.run.smithi046:> os.close(dirfd) 2024-04-26T23:43:10.502 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:43:10.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:10.845 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:10.846 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:10.947 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:10.947 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:11.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.223+0000 7ff751400640 1 Processor -- start 2024-04-26T23:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.223+0000 7ff751400640 1 -- start start 2024-04-26T23:43:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.224+0000 7ff751400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff74c06ca60 0x7ff74c162d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.224+0000 7ff751400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c1632d0 0x7ff74c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff751400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74c163760 con 0x7ff74c0ccb20 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff751400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff74c163cf0 con 0x7ff74c06ca60 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff751400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c0d19a0 con 0x7ff74c1632d0 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff74c06ca60 0x7ff74c162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff74c06ca60 0x7ff74c162d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36956/0 (socket says 172.21.15.46:36956) 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 -- 172.21.15.46:0/3965036858 learned_addr learned my addr 172.21.15.46:0/3965036858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 -- 172.21.15.46:0/3965036858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c1632d0 msgr2=0x7ff74c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 --2- 172.21.15.46:0/3965036858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c1632d0 0x7ff74c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 -- 172.21.15.46:0/3965036858 >> v1:172.21.15.142:6789/0 conn(0x7ff74c0ccb20 legacy=0x7ff74c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.225+0000 7ff743400640 1 -- 172.21.15.46:0/3965036858 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c163a10 con 0x7ff74c06ca60 2024-04-26T23:43:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.227+0000 7ff743400640 1 --2- 172.21.15.46:0/3965036858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c162d90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff734009870 tx=0x7ff73402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88ce76d332b5cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.227+0000 7ff74b400640 1 -- 172.21.15.46:0/3965036858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734037070 con 0x7ff74c06ca60 2024-04-26T23:43:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.227+0000 7ff74b400640 1 -- 172.21.15.46:0/3965036858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff734032440 con 0x7ff74c06ca60 2024-04-26T23:43:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.227+0000 7ff74b400640 1 -- 172.21.15.46:0/3965036858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734032810 con 0x7ff74c06ca60 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 -- 172.21.15.46:0/3965036858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 msgr2=0x7ff74c162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 --2- 172.21.15.46:0/3965036858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c162d90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff734009870 tx=0x7ff73402a210 comp rx=0 tx=0).stop 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 -- 172.21.15.46:0/3965036858 shutdown_connections 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 --2- 172.21.15.46:0/3965036858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c1632d0 0x7ff74c15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 --2- 172.21.15.46:0/3965036858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c162d90 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 -- 172.21.15.46:0/3965036858 >> 172.21.15.46:0/3965036858 conn(0x7ff74c0d1c30 msgr2=0x7ff74c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 -- 172.21.15.46:0/3965036858 shutdown_connections 2024-04-26T23:43:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.228+0000 7ff751400640 1 -- 172.21.15.46:0/3965036858 wait complete. 2024-04-26T23:43:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.230+0000 7ff751400640 1 Processor -- start 2024-04-26T23:43:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.230+0000 7ff751400640 1 -- start start 2024-04-26T23:43:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c0ccb20 0x7ff74c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 0x7ff74c072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c0d0470 con 0x7ff74c06ca60 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c0d0190 con 0x7ff74c1632d0 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff751400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c0ccf00 con 0x7ff74c0ccb20 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff74be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff74be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36962/0 (socket says 172.21.15.46:36962) 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.231+0000 7ff74be00640 1 -- 172.21.15.46:0/2775866713 learned_addr learned my addr 172.21.15.46:0/2775866713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 -- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c0ccb20 msgr2=0x7ff74c0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c0ccb20 0x7ff74c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 -- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 msgr2=0x7ff74c072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff750a00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 0x7ff74c072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 0x7ff74c072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 -- 172.21.15.46:0/2775866713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c1f5340 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff750a00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 0x7ff74c072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff74be00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c430 tx=0x7ff73c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011790 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c017070 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c1f5620 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.232+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011bb0 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.233+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1f5be0 con 0x7ff74c06ca60 2024-04-26T23:43:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.234+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c0693b0 con 0x7ff74c06ca60 2024-04-26T23:43:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.235+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff73c01c460 con 0x7ff74c06ca60 2024-04-26T23:43:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.236+0000 7ff748c00640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 0x7ff72806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.236+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff73c08bc80 con 0x7ff74c06ca60 2024-04-26T23:43:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.237+0000 7ff743400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 0x7ff72806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.238+0000 7ff743400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 0x7ff72806ede0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.241+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff73c059e80 con 0x7ff74c06ca60 2024-04-26T23:43:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.451+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff74c0d2b40 con 0x7ff74c06ca60 2024-04-26T23:43:11.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.454+0000 7ff748c00640 1 -- 172.21.15.46:0/2775866713 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16848 (secure 0 0 0) 0x7ff73c059780 con 0x7ff74c06ca60 2024-04-26T23:43:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:11.458 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:00.597270+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.456+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 msgr2=0x7ff72806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.456+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 0x7ff72806ede0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402c000 comp rx=0 tx=0).stop 2024-04-26T23:43:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.456+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 msgr2=0x7ff74c155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.456+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c430 tx=0x7ff73c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 shutdown_connections 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff72806c920 0x7ff72806ede0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74c1632d0 0x7ff74c072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff74c0ccb20 0x7ff74c0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 --2- 172.21.15.46:0/2775866713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74c06ca60 0x7ff74c155100 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 >> 172.21.15.46:0/2775866713 conn(0x7ff74c0d1c30 msgr2=0x7ff74c0ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 shutdown_connections 2024-04-26T23:43:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.457+0000 7ff751400640 1 -- 172.21.15.46:0/2775866713 wait complete. 2024-04-26T23:43:11.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:42:56.897784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:42:58.028020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:43:11.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:43:11.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.692+0000 7fe92be00640 1 Processor -- start 2024-04-26T23:43:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.692+0000 7fe92be00640 1 -- start start 2024-04-26T23:43:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe92c073d00 0x7fe92c0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c0ce7d0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c0cdf90 con 0x7fe92c073d00 2024-04-26T23:43:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c0ceab0 con 0x7fe92c15cec0 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe92c073d00 0x7fe92c0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe92c073d00 0x7fe92c0b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36970/0 (socket says 172.21.15.46:36970) 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe92b400640 1 -- 172.21.15.46:0/1206985370 learned_addr learned my addr 172.21.15.46:0/1206985370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.694+0000 7fe923e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe92c0cdb30 0x7fe92c0735f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45258/0 (socket says 172.21.15.46:45258) 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92aa00640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92b400640 1 -- 172.21.15.46:0/1206985370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 msgr2=0x7fe92c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92b400640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92b400640 1 -- 172.21.15.46:0/1206985370 >> v1:172.21.15.46:6789/0 conn(0x7fe92c0cdb30 legacy=0x7fe92c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92a000640 1 -- 172.21.15.46:0/1206985370 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098509449 0 0) 0x7fe92c0ce7d0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.695+0000 7fe92b400640 1 -- 172.21.15.46:0/1206985370 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0ced90 con 0x7fe92c073d00 2024-04-26T23:43:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.696+0000 7fe92b400640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c073d00 0x7fe92c0b3b40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe924009e50 tx=0x7fe92402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968a5a58d03c0eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.696+0000 7fe92a000640 1 -- 172.21.15.46:0/1206985370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924036070 con 0x7fe92c073d00 2024-04-26T23:43:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.696+0000 7fe92a000640 1 -- 172.21.15.46:0/1206985370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924031600 con 0x7fe92c073d00 2024-04-26T23:43:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.696+0000 7fe92a000640 1 -- 172.21.15.46:0/1206985370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240319d0 con 0x7fe92c073d00 2024-04-26T23:43:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 -- 172.21.15.46:0/1206985370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c073d00 msgr2=0x7fe92c0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c073d00 0x7fe92c0b3b40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe924009e50 tx=0x7fe92402a710 comp rx=0 tx=0).stop 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 -- 172.21.15.46:0/1206985370 shutdown_connections 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c073d00 0x7fe92c0b3b40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 --2- 172.21.15.46:0/1206985370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 -- 172.21.15.46:0/1206985370 >> 172.21.15.46:0/1206985370 conn(0x7fe92c0b27b0 msgr2=0x7fe92c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 -- 172.21.15.46:0/1206985370 shutdown_connections 2024-04-26T23:43:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.697+0000 7fe92be00640 1 -- 172.21.15.46:0/1206985370 wait complete. 2024-04-26T23:43:11.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.699+0000 7fe92be00640 1 Processor -- start 2024-04-26T23:43:11.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.699+0000 7fe92be00640 1 -- start start 2024-04-26T23:43:11.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 0x7fe92c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c15d2a0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c0cf350 con 0x7fe92c073d00 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe92be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c0b4080 con 0x7fe92c15cec0 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe923e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe923e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36982/0 (socket says 172.21.15.46:36982) 2024-04-26T23:43:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.700+0000 7fe923e00640 1 -- 172.21.15.46:0/250665947 learned_addr learned my addr 172.21.15.46:0/250665947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe92b400640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe92aa00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 0x7fe92c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 -- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 msgr2=0x7fe92c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 -- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 msgr2=0x7fe92c15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 0x7fe92c15fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 -- 172.21.15.46:0/250665947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0c8340 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe92aa00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 0x7fe92c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923e00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c4e0 tx=0x7fe91400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.701+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914018070 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.702+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9140074c0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.702+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914004a20 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.702+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c1f5700 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.702+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1f5cc0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.703+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c0693b0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.704+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe91401b070 con 0x7fe92c0cdb30 2024-04-26T23:43:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.705+0000 7fe923400640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 0x7fe90806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.705+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fe91408b080 con 0x7fe92c0cdb30 2024-04-26T23:43:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.706+0000 7fe92aa00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 0x7fe90806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.707+0000 7fe92aa00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 0x7fe90806ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe91c006fd0 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:11.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.709+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9140593d0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.913+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe92c15c6c0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.914+0000 7fe923400640 1 -- 172.21.15.46:0/250665947 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7fe914058cd0 con 0x7fe92c0cdb30 2024-04-26T23:43:11.917 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:43:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 msgr2=0x7fe90806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 0x7fe90806ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe91c006fd0 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 msgr2=0x7fe92c1f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c4e0 tx=0x7fe91400c9a0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 shutdown_connections 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90806c920 0x7fe90806ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe92c15cec0 0x7fe92c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c0cdb30 0x7fe92c1f4e40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 --2- 172.21.15.46:0/250665947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c073d00 0x7fe92c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 >> 172.21.15.46:0/250665947 conn(0x7fe92c0b27b0 msgr2=0x7fe92c0cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 shutdown_connections 2024-04-26T23:43:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:11.917+0000 7fe92be00640 1 -- 172.21.15.46:0/250665947 wait complete. 2024-04-26T23:43:11.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:43:12.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.156+0000 7faeb6a00640 1 Processor -- start 2024-04-26T23:43:12.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.156+0000 7faeb6a00640 1 -- start start 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb00cdd00 0x7faeb00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 0x7faeb00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb00ce7d0 con 0x7faeb015cec0 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faeb00ceab0 con 0x7faeb00cdd00 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faeb00ced90 con 0x7faeb0073b30 2024-04-26T23:43:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb00cdd00 0x7faeb00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb00cdd00 0x7faeb00735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40128/0 (socket says 172.21.15.46:40128) 2024-04-26T23:43:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb4c00640 1 -- 172.21.15.46:0/2705078656 learned_addr learned my addr 172.21.15.46:0/2705078656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.159+0000 7faeb4c00640 1 -- 172.21.15.46:0/2705078656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 msgr2=0x7faeb00b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.158+0000 7faeb6000640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 0x7faeb00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.159+0000 7faeb4c00640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 0x7faeb00b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.159+0000 7faeb4c00640 1 -- 172.21.15.46:0/2705078656 >> v1:172.21.15.142:6789/0 conn(0x7faeb015cec0 legacy=0x7faeb00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.159+0000 7faeb4c00640 1 -- 172.21.15.46:0/2705078656 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb00cf070 con 0x7faeb00cdd00 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.159+0000 7faeb6000640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 0x7faeb00b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.160+0000 7faeb4c00640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb00cdd00 0x7faeb00735f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faeb00cd4f0 tx=0x7fae9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964f49f605c6a8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.160+0000 7faeafe00640 1 -- 172.21.15.46:0/2705078656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c037070 con 0x7faeb00cdd00 2024-04-26T23:43:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.160+0000 7faeafe00640 1 -- 172.21.15.46:0/2705078656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9c032440 con 0x7faeb00cdd00 2024-04-26T23:43:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.160+0000 7faeafe00640 1 -- 172.21.15.46:0/2705078656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c032810 con 0x7faeb00cdd00 2024-04-26T23:43:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 -- 172.21.15.46:0/2705078656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb00cdd00 msgr2=0x7faeb00735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb00cdd00 0x7faeb00735f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faeb00cd4f0 tx=0x7fae9c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 -- 172.21.15.46:0/2705078656 shutdown_connections 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb0073b30 0x7faeb00b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2705078656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb00cdd00 0x7faeb00735f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 -- 172.21.15.46:0/2705078656 >> 172.21.15.46:0/2705078656 conn(0x7faeb00b27b0 msgr2=0x7faeb00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 -- 172.21.15.46:0/2705078656 shutdown_connections 2024-04-26T23:43:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.161+0000 7faeb6a00640 1 -- 172.21.15.46:0/2705078656 wait complete. 2024-04-26T23:43:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.163+0000 7faeb6a00640 1 Processor -- start 2024-04-26T23:43:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.164+0000 7faeb6a00640 1 -- start start 2024-04-26T23:43:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb00cdd00 0x7faeb015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb015cec0 0x7faeb0072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0073f70 con 0x7faeb0073b30 2024-04-26T23:43:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faeb015d2a0 con 0x7faeb015cec0 2024-04-26T23:43:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb6a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faeb00b4080 con 0x7faeb00cdd00 2024-04-26T23:43:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40144/0 (socket says 172.21.15.46:40144) 2024-04-26T23:43:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.165+0000 7faeb5600640 1 -- 172.21.15.46:0/2321434784 learned_addr learned my addr 172.21.15.46:0/2321434784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb4c00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb00cdd00 0x7faeb015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 -- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb00cdd00 msgr2=0x7faeb015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb00cdd00 0x7faeb015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb6000640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb015cec0 0x7faeb0072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 -- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb015cec0 msgr2=0x7faeb0072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb015cec0 0x7faeb0072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 -- 172.21.15.46:0/2321434784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb00cb7f0 con 0x7faeb0073b30 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faeb5600640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faea400b420 tx=0x7faea400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.166+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4011830 con 0x7faeb0073b30 2024-04-26T23:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.167+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea4016070 con 0x7faeb0073b30 2024-04-26T23:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.167+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4011c10 con 0x7faeb0073b30 2024-04-26T23:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.167+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb00cbad0 con 0x7faeb0073b30 2024-04-26T23:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.167+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb00cc090 con 0x7faeb0073b30 2024-04-26T23:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.168+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb00693b0 con 0x7faeb0073b30 2024-04-26T23:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.169+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faea401b460 con 0x7faeb0073b30 2024-04-26T23:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.170+0000 7faead600640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 0x7fae9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.170+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7faea408b8f0 con 0x7faeb0073b30 2024-04-26T23:43:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.171+0000 7faeb4c00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 0x7fae9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.172+0000 7faeb4c00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 0x7fae9006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fae9c034000 tx=0x7fae9c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.175+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faea4020020 con 0x7faeb0073b30 2024-04-26T23:43:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.380+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faeb0160680 con 0x7faeb0073b30 2024-04-26T23:43:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.381+0000 7faead600640 1 -- 172.21.15.46:0/2321434784 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7faea4059ab0 con 0x7faeb0073b30 2024-04-26T23:43:12.385 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:43:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 msgr2=0x7fae9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 0x7fae9006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fae9c034000 tx=0x7fae9c02e000 comp rx=0 tx=0).stop 2024-04-26T23:43:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 msgr2=0x7faeb00ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faea400b420 tx=0x7faea400b8e0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 shutdown_connections 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae9006c8f0 0x7fae9006edb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb015cec0 0x7faeb0072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faeb00cdd00 0x7faeb015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 --2- 172.21.15.46:0/2321434784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb0073b30 0x7faeb00ccc50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 >> 172.21.15.46:0/2321434784 conn(0x7faeb00b27b0 msgr2=0x7faeb00cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 shutdown_connections 2024-04-26T23:43:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.384+0000 7faeb6a00640 1 -- 172.21.15.46:0/2321434784 wait complete. 2024-04-26T23:43:12.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:43:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.627+0000 7f098f200640 1 Processor -- start 2024-04-26T23:43:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.627+0000 7f098f200640 1 -- start start 2024-04-26T23:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09900ce670 0x7f09900ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0990146b20 con 0x7f0990146740 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0990161a00 con 0x7f09900ce670 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f099015e3f0 con 0x7f099015df90 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09900ce670 0x7f09900ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09900ce670 0x7f09900ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40160/0 (socket says 172.21.15.46:40160) 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.629+0000 7f098e800640 1 -- 172.21.15.46:0/1958563252 learned_addr learned my addr 172.21.15.46:0/1958563252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.630+0000 7f098d400640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.630+0000 7f098e800640 1 -- 172.21.15.46:0/1958563252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 msgr2=0x7f09900d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.630+0000 7f098e800640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.630+0000 7f098e800640 1 -- 172.21.15.46:0/1958563252 >> v1:172.21.15.142:6789/0 conn(0x7f0990146740 legacy=0x7f09901612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.630+0000 7f098e800640 1 -- 172.21.15.46:0/1958563252 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900ccf30 con 0x7f09900ce670 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.631+0000 7f098e800640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09900ce670 0x7f09900ceaa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0988009e50 tx=0x7f098802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaa1411057039ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.631+0000 7f098ca00640 1 -- 172.21.15.46:0/1958563252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988036070 con 0x7f09900ce670 2024-04-26T23:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.631+0000 7f098ca00640 1 -- 172.21.15.46:0/1958563252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0988031600 con 0x7f09900ce670 2024-04-26T23:43:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.631+0000 7f098ca00640 1 -- 172.21.15.46:0/1958563252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880319d0 con 0x7f09900ce670 2024-04-26T23:43:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 -- 172.21.15.46:0/1958563252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09900ce670 msgr2=0x7f09900ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09900ce670 0x7f09900ceaa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0988009e50 tx=0x7f098802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 -- 172.21.15.46:0/1958563252 shutdown_connections 2024-04-26T23:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09900ce670 0x7f09900ceaa0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 --2- 172.21.15.46:0/1958563252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 -- 172.21.15.46:0/1958563252 >> 172.21.15.46:0/1958563252 conn(0x7f09900b26d0 msgr2=0x7f09900b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 -- 172.21.15.46:0/1958563252 shutdown_connections 2024-04-26T23:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.632+0000 7f098f200640 1 -- 172.21.15.46:0/1958563252 wait complete. 2024-04-26T23:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.634+0000 7f098f200640 1 Processor -- start 2024-04-26T23:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.634+0000 7f098f200640 1 -- start start 2024-04-26T23:43:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 0x7f099013f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09900cefe0 con 0x7f0990146740 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0990160dd0 con 0x7f09900ce670 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09900c8910 con 0x7f099015df90 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40162/0 (socket says 172.21.15.46:40162) 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.635+0000 7f098d400640 1 -- 172.21.15.46:0/1864940927 learned_addr learned my addr 172.21.15.46:0/1864940927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098de00640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 0x7f099013f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 -- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 msgr2=0x7f09900b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 -- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 msgr2=0x7f099013f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 0x7f099013f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 -- 172.21.15.46:0/1864940927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900c80b0 con 0x7f0990146740 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098de00640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 0x7f099013f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098d400640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f097c00c7f0 tx=0x7f097c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c015070 con 0x7f0990146740 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c004220 con 0x7f0990146740 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09900c8350 con 0x7f0990146740 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.636+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c007620 con 0x7f0990146740 2024-04-26T23:43:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.637+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0990143eb0 con 0x7f0990146740 2024-04-26T23:43:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.638+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09900693b0 con 0x7f0990146740 2024-04-26T23:43:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.639+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f097c0079d0 con 0x7f0990146740 2024-04-26T23:43:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.640+0000 7f0986000640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.640+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f097c08bf70 con 0x7f0990146740 2024-04-26T23:43:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.640+0000 7f098de00640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.642+0000 7f098de00640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 0x7f097006edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0980003eb0 tx=0x7f0980009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.644+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f097c05a2a0 con 0x7f0990146740 2024-04-26T23:43:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.848+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f099006f400 con 0x7f0990146740 2024-04-26T23:43:12.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.849+0000 7f0986000640 1 -- 172.21.15.46:0/1864940927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7f097c059ba0 con 0x7f0990146740 2024-04-26T23:43:12.852 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 msgr2=0x7f097006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 0x7f097006edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0980003eb0 tx=0x7f0980009040 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 msgr2=0x7f09900c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f097c00c7f0 tx=0x7f097c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 shutdown_connections 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f099015df90 0x7f09900b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0990146740 0x7f09900c7b70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 --2- 172.21.15.46:0/1864940927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09900ce670 0x7f099013f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 >> 172.21.15.46:0/1864940927 conn(0x7f09900b26d0 msgr2=0x7f09900af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 shutdown_connections 2024-04-26T23:43:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:12.852+0000 7f098f200640 1 -- 172.21.15.46:0/1864940927 wait complete. 2024-04-26T23:43:12.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:43:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.085+0000 7fbb9ca00640 1 Processor -- start 2024-04-26T23:43:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.085+0000 7fbb9ca00640 1 -- start start 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb9ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbb980720e0 0x7fbb980cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb9ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb98073ad0 con 0x7fbb981466b0 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb9ca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb98146a90 con 0x7fbb98160910 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb9ca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbb98073d70 con 0x7fbb980720e0 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb97400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb981466b0 0x7fbb9806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38820/0 (socket says 172.21.15.46:38820) 2024-04-26T23:43:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.087+0000 7fbb97400640 1 -- 172.21.15.46:0/3269747854 learned_addr learned my addr 172.21.15.46:0/3269747854 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96a00640 1 --2- 172.21.15.46:0/3269747854 >> v2:172.21.15.142:3300/0 conn(0x7fbb980720e0 0x7fbb980cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355287335 0 0) 0x7fbb98073ad0 con 0x7fbb981466b0 2024-04-26T23:43:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb7c003520 con 0x7fbb981466b0 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1258349850 0 0) 0x7fbb7c003520 con 0x7fbb981466b0 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 >> v2:172.21.15.142:3300/0 conn(0x7fbb980720e0 msgr2=0x7fbb980cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 --2- 172.21.15.46:0/3269747854 >> v2:172.21.15.142:3300/0 conn(0x7fbb980720e0 0x7fbb980cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 >> v1:172.21.15.142:6789/0 conn(0x7fbb98160910 legacy=0x7fbb980cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb98074050 con 0x7fbb981466b0 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fbb88001670 con 0x7fbb981466b0 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 >> v1:172.21.15.46:6789/0 conn(0x7fbb981466b0 legacy=0x7fbb9806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 0x7fbb7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c0051c0 0x7fbb7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98073d70 con 0x7fbb7c0039e0 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c00c090 con 0x7fbb7c0051c0 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.088+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c00c290 con 0x7fbb7c004800 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 0x7fbb7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb96a00640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 -- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c004800 msgr2=0x7fbb7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 -- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c0051c0 msgr2=0x7fbb7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c0051c0 0x7fbb7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97400640 1 -- 172.21.15.46:0/3269747854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c003520 con 0x7fbb7c0039e0 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb97e00640 1 -- 172.21.15.46:0/3269747854 reap_dead start 2024-04-26T23:43:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.089+0000 7fbb96a00640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.090+0000 7fbb97400640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 0x7fbb7c003e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c210 tx=0x7fbb7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f12ef6cac006b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.090+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88007d30 con 0x7fbb7c0039e0 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.090+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88005af0 con 0x7fbb7c0039e0 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb96000640 1 -- 172.21.15.46:0/3269747854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88007070 con 0x7fbb7c0039e0 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 -- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 msgr2=0x7fbb7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 0x7fbb7c003e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c210 tx=0x7fbb7c005830 comp rx=0 tx=0).stop 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 -- 172.21.15.46:0/3269747854 shutdown_connections 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/3269747854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7c0039e0 0x7fbb7c003e00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 -- 172.21.15.46:0/3269747854 >> 172.21.15.46:0/3269747854 conn(0x7fbb980b26d0 msgr2=0x7fbb980b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 -- 172.21.15.46:0/3269747854 shutdown_connections 2024-04-26T23:43:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.091+0000 7fbb9ca00640 1 -- 172.21.15.46:0/3269747854 wait complete. 2024-04-26T23:43:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.093+0000 7fbb9ca00640 1 Processor -- start 2024-04-26T23:43:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.093+0000 7fbb9ca00640 1 -- start start 2024-04-26T23:43:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0039e0 0x7fbb981448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb980b0c80 0x7fbb980b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98160d60 con 0x7fbb980c9eb0 2024-04-26T23:43:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb980cb470 con 0x7fbb980b0c80 2024-04-26T23:43:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb98160ee0 con 0x7fbb7c0039e0 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.094+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40182/0 (socket says 172.21.15.46:40182) 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 -- 172.21.15.46:0/1445999290 learned_addr learned my addr 172.21.15.46:0/1445999290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 -- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0039e0 msgr2=0x7fbb981448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0039e0 0x7fbb981448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 -- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb980b0c80 msgr2=0x7fbb980b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb980b0c80 0x7fbb980b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 -- 172.21.15.46:0/1445999290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb980ca3e0 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.095+0000 7fbb96a00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400f3b0 tx=0x7fbb8400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.096+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84003990 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.096+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb84003b30 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.096+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb980ca6c0 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.096+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84004020 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.096+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb980ae980 con 0x7fbb980c9eb0 2024-04-26T23:43:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.097+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb98073510 con 0x7fbb980c9eb0 2024-04-26T23:43:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.098+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb84005d10 con 0x7fbb980c9eb0 2024-04-26T23:43:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.099+0000 7fbb8f400640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 0x7fbb7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.099+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fbb8408c710 con 0x7fbb980c9eb0 2024-04-26T23:43:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.100+0000 7fbb97400640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 0x7fbb7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.101+0000 7fbb97400640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 0x7fbb7806edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb88033000 tx=0x7fbb88009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.103+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb98073510 con 0x7fbb980c9eb0 2024-04-26T23:43:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.323+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb980c7180 con 0x7fbb980c9eb0 2024-04-26T23:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.323+0000 7fbb8f400640 1 -- 172.21.15.46:0/1445999290 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7fbb84024020 con 0x7fbb980c9eb0 2024-04-26T23:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 msgr2=0x7fbb7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 0x7fbb7806edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb88033000 tx=0x7fbb88009030 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 msgr2=0x7fbb98144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400f3b0 tx=0x7fbb8400f870 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 shutdown_connections 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7806c8f0 0x7fbb7806edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb980b0c80 0x7fbb980b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb980c9eb0 0x7fbb98144e20 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 --2- 172.21.15.46:0/1445999290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0039e0 0x7fbb981448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 >> 172.21.15.46:0/1445999290 conn(0x7fbb980b26d0 msgr2=0x7fbb980b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 shutdown_connections 2024-04-26T23:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.326+0000 7fbb9ca00640 1 -- 172.21.15.46:0/1445999290 wait complete. 2024-04-26T23:43:13.349 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:43:13.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:13.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:13.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:43:13.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:13.556 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:43:13.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-26T23:43:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.833+0000 7fc64a800640 1 Processor -- start 2024-04-26T23:43:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.833+0000 7fc64a800640 1 -- start start 2024-04-26T23:43:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc64a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc644145750 0x7fc64406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc64a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc64406f400 0x7fc6440b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc64a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440ce0c0 con 0x7fc6440b5120 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc64a800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc64406f860 con 0x7fc64406f400 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc64a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc6440b5940 con 0x7fc644145750 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc648a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc64406f400 0x7fc6440b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc648a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc64406f400 0x7fc6440b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40196/0 (socket says 172.21.15.46:40196) 2024-04-26T23:43:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc649e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6440b5120 0x7fc6440d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38828/0 (socket says 172.21.15.46:38828) 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.835+0000 7fc648a00640 1 -- 172.21.15.46:0/3806809414 learned_addr learned my addr 172.21.15.46:0/3806809414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.836+0000 7fc648a00640 1 -- 172.21.15.46:0/3806809414 >> v2:172.21.15.142:3300/0 conn(0x7fc644145750 msgr2=0x7fc64406eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.836+0000 7fc648a00640 1 --2- 172.21.15.46:0/3806809414 >> v2:172.21.15.142:3300/0 conn(0x7fc644145750 0x7fc64406eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.836+0000 7fc648a00640 1 -- 172.21.15.46:0/3806809414 >> v1:172.21.15.46:6789/0 conn(0x7fc6440b5120 legacy=0x7fc6440d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.836+0000 7fc648a00640 1 -- 172.21.15.46:0/3806809414 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6440ce330 con 0x7fc64406f400 2024-04-26T23:43:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.836+0000 7fc643e00640 1 -- 172.21.15.46:0/3806809414 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301520851 0 0) 0x7fc6440ce0c0 con 0x7fc6440b5120 2024-04-26T23:43:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.837+0000 7fc648a00640 1 --2- 172.21.15.46:0/3806809414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc64406f400 0x7fc6440b4be0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc6440d1b30 tx=0x7fc63402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c661a5af5005cd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.837+0000 7fc643e00640 1 -- 172.21.15.46:0/3806809414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634037070 con 0x7fc64406f400 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.837+0000 7fc643e00640 1 -- 172.21.15.46:0/3806809414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc634032440 con 0x7fc64406f400 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.837+0000 7fc643e00640 1 -- 172.21.15.46:0/3806809414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340327f0 con 0x7fc64406f400 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 -- 172.21.15.46:0/3806809414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc64406f400 msgr2=0x7fc6440b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 --2- 172.21.15.46:0/3806809414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc64406f400 0x7fc6440b4be0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc6440d1b30 tx=0x7fc63402a290 comp rx=0 tx=0).stop 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 -- 172.21.15.46:0/3806809414 shutdown_connections 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 --2- 172.21.15.46:0/3806809414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc64406f400 0x7fc6440b4be0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 --2- 172.21.15.46:0/3806809414 >> v2:172.21.15.142:3300/0 conn(0x7fc644145750 0x7fc64406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 -- 172.21.15.46:0/3806809414 >> 172.21.15.46:0/3806809414 conn(0x7fc64406c350 msgr2=0x7fc64406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 -- 172.21.15.46:0/3806809414 shutdown_connections 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.838+0000 7fc64a800640 1 -- 172.21.15.46:0/3806809414 wait complete. 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.840+0000 7fc64a800640 1 Processor -- start 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.840+0000 7fc64a800640 1 -- start start 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64406f400 0x7fc6440cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644145750 0x7fc6440c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440b5590 con 0x7fc6440b5120 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440b5c20 con 0x7fc64406f400 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc64a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc644139330 con 0x7fc644145750 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40202/0 (socket says 172.21.15.46:40202) 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.841+0000 7fc648a00640 1 -- 172.21.15.46:0/1001749167 learned_addr learned my addr 172.21.15.46:0/1001749167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 -- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644145750 msgr2=0x7fc6440c8a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644145750 0x7fc6440c8a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 -- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64406f400 msgr2=0x7fc6440cfe80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64406f400 0x7fc6440cfe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 -- 172.21.15.46:0/1001749167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6440ca3c0 con 0x7fc6440b5120 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc648a00640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc634000c00 tx=0x7fc6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc641600640 1 -- 172.21.15.46:0/1001749167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634041070 con 0x7fc6440b5120 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc641600640 1 -- 172.21.15.46:0/1001749167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63403c870 con 0x7fc6440b5120 2024-04-26T23:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc641600640 1 -- 172.21.15.46:0/1001749167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634037040 con 0x7fc6440b5120 2024-04-26T23:43:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.842+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc644139550 con 0x7fc6440b5120 2024-04-26T23:43:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.843+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc644139b10 con 0x7fc6440b5120 2024-04-26T23:43:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.845+0000 7fc641600640 1 -- 172.21.15.46:0/1001749167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc634044070 con 0x7fc6440b5120 2024-04-26T23:43:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.846+0000 7fc641600640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 0x7fc62806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.846+0000 7fc641600640 1 -- 172.21.15.46:0/1001749167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fc6340ac980 con 0x7fc6440b5120 2024-04-26T23:43:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.847+0000 7fc64ec19740 1 Processor -- start 2024-04-26T23:43:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.847+0000 7fc649400640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 0x7fc62806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.847+0000 7fc64ec19740 1 -- start start 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561cdd8b2030 con 0x561cdd9a23f0 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cdd8cd7f0 con 0x561cdd9e9fb0 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561cdd8887e0 con 0x561cdd9cc710 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40210/0 (socket says 172.21.15.46:40210) 2024-04-26T23:43:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc649400640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 0x7fc62806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc63800abb0 tx=0x7fc638009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.848+0000 7fc649e00640 1 -- 172.21.15.46:0/1327934244 learned_addr learned my addr 172.21.15.46:0/1327934244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc648a00640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649400640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 -- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 msgr2=0x561cdd9ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 -- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 msgr2=0x561cdd9ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 -- 172.21.15.46:0/1327934244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cdd894050 con 0x561cdd9a23f0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649400640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.849+0000 7fc649e00640 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0015f0 tx=0x7fc63c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2186c113a75e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc631c00640 1 -- 172.21.15.46:0/1327934244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0074e0 con 0x561cdd9a23f0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc631c00640 1 -- 172.21.15.46:0/1327934244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c001d00 con 0x561cdd9a23f0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc631c00640 1 -- 172.21.15.46:0/1327934244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c004a20 con 0x561cdd9a23f0 2024-04-26T23:43:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 -- 172.21.15.46:0/1327934244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 msgr2=0x561cdd9e8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0015f0 tx=0x7fc63c001ab0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 -- 172.21.15.46:0/1327934244 shutdown_connections 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9a23f0 0x561cdd9e8670 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 --2- 172.21.15.46:0/1327934244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.850+0000 7fc64ec19740 1 -- 172.21.15.46:0/1327934244 >> 172.21.15.46:0/1327934244 conn(0x561cdd99ed20 msgr2=0x561cdd9e7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.851+0000 7fc64ec19740 1 -- 172.21.15.46:0/1327934244 shutdown_connections 2024-04-26T23:43:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.851+0000 7fc64ec19740 1 -- 172.21.15.46:0/1327934244 wait complete. 2024-04-26T23:43:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.851+0000 7fc64ec19740 1 -- start start 2024-04-26T23:43:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.851+0000 7fc64ec19740 1 Processor -- start 2024-04-26T23:43:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.852+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9cc710 0x561cdd9fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9e9fb0 0x561cdd9eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561cdd90a200 con 0x561cdd9c8460 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cdd904ee0 con 0x561cdd9cc710 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561cdd911e70 con 0x561cdd9e9fb0 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc64ec19740 10 client.0 _ll_register_callbacks cb 0x561cdd8fb270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40220/0 (socket says 172.21.15.46:40220) 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 -- 172.21.15.46:0/2200358782 learned_addr learned my addr 172.21.15.46:0/2200358782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc648a00640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9e9fb0 0x561cdd9eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649400640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9cc710 0x561cdd9fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 -- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9e9fb0 msgr2=0x561cdd9eaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9e9fb0 0x561cdd9eaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 -- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9cc710 msgr2=0x561cdd9fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9cc710 0x561cdd9fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.853+0000 7fc649e00640 1 -- 172.21.15.46:0/2200358782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cdd6e17d0 con 0x561cdd9c8460 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc649e00640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000f60 tx=0x7fc63c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eded15f098c62f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc626000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c026060 con 0x561cdd9c8460 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fc63c021cc0 con 0x561cdd9c8460 2024-04-26T23:43:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63c001d00 con 0x561cdd9c8460 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc63c00c6c0 con 0x561cdd9c8460 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc64ec19740 10 client.6819 fetch_fsmap learned FSMap version 41 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.854+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x561cdd6173f0 con 0x561cdd9c8460 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c001d00 con 0x561cdd9c8460 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7fc63c00c860 con 0x561cdd9c8460 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 10 client.6819 fetch_fsmap finished waiting for FSMap version 41 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 10 client.6819 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 20 client.6819 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdd9cb790 0x561cdd9cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 4 client.6819 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.855+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 --> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cdd9137e0 con 0x561cdd9cb790 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.856+0000 7fc649400640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdd9cb790 0x561cdd9cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.857+0000 7fc649400640 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdd9cb790 0x561cdd9cbba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1682cf3bf1dbf416 server_cookie=a414563e56055d6a in_seq=0 out_seq=0 2024-04-26T23:43:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.857+0000 7fc626000640 10 client.6819 ms_handle_connect on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc626000640 1 -- 172.21.15.46:0/2200358782 <== mds.0 v2:172.21.15.142:6834/2927770154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cdd9137e0 con 0x561cdd9cb790 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc626000640 10 client.6819 handle_command_reply: tid=0 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 client.6819 shutdown 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 msgr2=0x561cdd9dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000f60 tx=0x7fc63c00f000 comp rx=0 tx=0).stop 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 shutdown_connections 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdd9cb790 0x561cdd9cbba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9dfed0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9e9fb0 0x561cdd9eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 --2- 172.21.15.46:0/2200358782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9cc710 0x561cdd9fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc626000640 0 client.6819 ms_handle_reset on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 >> 172.21.15.46:0/2200358782 conn(0x561cdd9a23f0 msgr2=0x561cdd99b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 shutdown_connections 2024-04-26T23:43:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 1 -- 172.21.15.46:0/2200358782 wait complete. 2024-04-26T23:43:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.859+0000 7fc64ec19740 20 client.6819 trim_cache size 0 max 16384 2024-04-26T23:43:13.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.876+0000 7fc64ec19740 1 Processor -- start 2024-04-26T23:43:13.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.876+0000 7fc64ec19740 1 -- start start 2024-04-26T23:43:13.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 0x561cdd99bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc644201be0 con 0x561cdd9cc710 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc644200bc0 con 0x561cdd9e9fb0 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6441fff80 con 0x561cdd9c8460 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc648a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40230/0 (socket says 172.21.15.46:40230) 2024-04-26T23:43:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.877+0000 7fc648a00640 1 -- 172.21.15.46:0/4094328402 learned_addr learned my addr 172.21.15.46:0/4094328402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc649400640 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 0x561cdd99bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 -- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 msgr2=0x561cdd99bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 0x561cdd99bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 -- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 msgr2=0x561cdd9dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 -- 172.21.15.46:0/4094328402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cdd9cbf10 con 0x561cdd9cc710 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc649400640 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 0x561cdd99bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc648a00640 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc6340810b0 tx=0x7fc634009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a6357ea58851f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc631200640 1 -- 172.21.15.46:0/4094328402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63404b040 con 0x561cdd9cc710 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc631200640 1 -- 172.21.15.46:0/4094328402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63407e9e0 con 0x561cdd9cc710 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.878+0000 7fc631200640 1 -- 172.21.15.46:0/4094328402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634083210 con 0x561cdd9cc710 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 -- 172.21.15.46:0/4094328402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 msgr2=0x561cdd9df4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc6340810b0 tx=0x7fc634009490 comp rx=0 tx=0).stop 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 -- 172.21.15.46:0/4094328402 shutdown_connections 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdd9dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9cc710 0x561cdd9df4e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 --2- 172.21.15.46:0/4094328402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9c8460 0x561cdd99bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 -- 172.21.15.46:0/4094328402 >> 172.21.15.46:0/4094328402 conn(0x561cdd9a23f0 msgr2=0x561cdd9982e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 -- 172.21.15.46:0/4094328402 shutdown_connections 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.879+0000 7fc64ec19740 1 -- 172.21.15.46:0/4094328402 wait complete. 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.880+0000 7fc64ec19740 1 -- start start 2024-04-26T23:43:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.880+0000 7fc64ec19740 1 Processor -- start 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdda60510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561cdd912fd0 con 0x561cdd9c8460 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc644200570 con 0x561cdd9e9fb0 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561cdd9c5950 con 0x561cdd9cc710 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc64ec19740 10 client.0 _ll_register_callbacks cb 0x7fc63c001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc649400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc649400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40236/0 (socket says 172.21.15.46:40236) 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.881+0000 7fc649400640 1 -- 172.21.15.46:0/211891266 learned_addr learned my addr 172.21.15.46:0/211891266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649e00640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdda60510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 -- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 msgr2=0x561cdd9a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 -- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 msgr2=0x561cdda60510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdda60510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 -- 172.21.15.46:0/211891266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cdd5d70f0 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc649400640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x561cdd9eb8c0 tx=0x7fc638004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3279698ab81b5a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc626000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638026630 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fc6380267d0 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.882+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63407e9e0 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc638035050 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 10 client.6825 fetch_fsmap learned FSMap version 41 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x561cdd8cda30 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63407e9e0 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7fc638038020 con 0x561cdd9c8460 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 10 client.6825 fetch_fsmap finished waiting for FSMap version 41 2024-04-26T23:43:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 10 client.6825 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 20 client.6825 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdda60a50 0x561cdd9dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 4 client.6825 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.883+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 --> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cdd9e1f90 con 0x561cdda60a50 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.885+0000 7fc648a00640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdda60a50 0x561cdd9dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.885+0000 7fc648a00640 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdda60a50 0x561cdd9dceb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e7cbb6b240427e server_cookie=a45a294425306f6e in_seq=0 out_seq=0 2024-04-26T23:43:13.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.885+0000 7fc626000640 10 client.6825 ms_handle_connect on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc626000640 1 -- 172.21.15.46:0/211891266 <== mds.0 v2:172.21.15.142:6834/2927770154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cdd9e1f90 con 0x561cdda60a50 2024-04-26T23:43:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc626000640 10 client.6825 handle_command_reply: tid=0 2024-04-26T23:43:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc64ec19740 1 client.6825 shutdown 2024-04-26T23:43:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 msgr2=0x561cdd9cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x561cdd9eb8c0 tx=0x7fc638004a30 comp rx=0 tx=0).stop 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 shutdown_connections 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.911+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:6834/2927770154,v1:172.21.15.142:6835/2927770154] conn(0x561cdda60a50 0x561cdd9dceb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cdd9e9fb0 0x561cdda60510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561cdd9cc710 0x561cdd9a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 --2- 172.21.15.46:0/211891266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cdd9c8460 0x561cdd9cf730 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc626000640 0 client.6825 ms_handle_reset on v2:172.21.15.142:6834/2927770154 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 >> 172.21.15.46:0/211891266 conn(0x561cdd9a23f0 msgr2=0x561cdd9a38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 shutdown_connections 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 1 -- 172.21.15.46:0/211891266 wait complete. 2024-04-26T23:43:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.912+0000 7fc64ec19740 20 client.6825 trim_cache size 0 max 16384 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 msgr2=0x7fc62806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 0x7fc62806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc63800abb0 tx=0x7fc638009250 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 msgr2=0x7fc644138df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc634000c00 tx=0x7fc6340046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 shutdown_connections 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc62806c920 0x7fc62806ede0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc644145750 0x7fc6440c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440b5120 0x7fc644138df0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 --2- 172.21.15.46:0/1001749167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc64406f400 0x7fc6440cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 >> 172.21.15.46:0/1001749167 conn(0x7fc64406c350 msgr2=0x7fc6440ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 shutdown_connections 2024-04-26T23:43:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:13.913+0000 7fc64a800640 1 -- 172.21.15.46:0/1001749167 wait complete. 2024-04-26T23:43:13.936 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:43:13.936 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:13.936 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:43:13.936 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:13.936 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:43:13.937 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:13.937 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:43:13.937 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:13.956 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:43:13.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-26T23:43:13.961 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:43:13.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-26T23:43:13.964 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:43:13.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-26T23:43:13.964 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:43:13.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-26T23:43:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.148+0000 7efdea800640 1 Processor -- start 2024-04-26T23:43:14.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.148+0000 7efdea800640 1 -- start start 2024-04-26T23:43:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.149+0000 7efdea800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efde415c2d0 0x7efde4073d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.149+0000 7efdea800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efde40cff60 0x7efde4073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efdea800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40ce460 con 0x7efde4073570 2024-04-26T23:43:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efdea800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7efde406eee0 con 0x7efde40cff60 2024-04-26T23:43:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efdea800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efde40cdea0 con 0x7efde415c2d0 2024-04-26T23:43:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efde2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efde40cff60 0x7efde4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efde2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efde40cff60 0x7efde4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40238/0 (socket says 172.21.15.46:40238) 2024-04-26T23:43:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efde2a00640 1 -- 172.21.15.46:0/2192045015 learned_addr learned my addr 172.21.15.46:0/2192045015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.150+0000 7efde3400640 1 --2- 172.21.15.46:0/2192045015 >> v2:172.21.15.142:3300/0 conn(0x7efde415c2d0 0x7efde4073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.151+0000 7efde2a00640 1 -- 172.21.15.46:0/2192045015 >> v2:172.21.15.142:3300/0 conn(0x7efde415c2d0 msgr2=0x7efde4073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.151+0000 7efde2a00640 1 --2- 172.21.15.46:0/2192045015 >> v2:172.21.15.142:3300/0 conn(0x7efde415c2d0 0x7efde4073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.151+0000 7efde2a00640 1 -- 172.21.15.46:0/2192045015 >> v1:172.21.15.142:6789/0 conn(0x7efde4073570 legacy=0x7efde40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.151+0000 7efde2a00640 1 -- 172.21.15.46:0/2192045015 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde40ce180 con 0x7efde40cff60 2024-04-26T23:43:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.152+0000 7efde2a00640 1 --2- 172.21.15.46:0/2192045015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde40cff60 0x7efde4073030 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efde406eb40 tx=0x7efdd002c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f50d7f3cbf979d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.152+0000 7efde2000640 1 -- 172.21.15.46:0/2192045015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0039070 con 0x7efde40cff60 2024-04-26T23:43:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.152+0000 7efde2000640 1 -- 172.21.15.46:0/2192045015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0034640 con 0x7efde40cff60 2024-04-26T23:43:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.152+0000 7efde2000640 1 -- 172.21.15.46:0/2192045015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0034a10 con 0x7efde40cff60 2024-04-26T23:43:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 -- 172.21.15.46:0/2192045015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde40cff60 msgr2=0x7efde4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 --2- 172.21.15.46:0/2192045015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde40cff60 0x7efde4073030 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efde406eb40 tx=0x7efdd002c320 comp rx=0 tx=0).stop 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 -- 172.21.15.46:0/2192045015 shutdown_connections 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 --2- 172.21.15.46:0/2192045015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde40cff60 0x7efde4073030 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 --2- 172.21.15.46:0/2192045015 >> v2:172.21.15.142:3300/0 conn(0x7efde415c2d0 0x7efde4073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 -- 172.21.15.46:0/2192045015 >> 172.21.15.46:0/2192045015 conn(0x7efde40d1e20 msgr2=0x7efde40d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.153+0000 7efdea800640 1 -- 172.21.15.46:0/2192045015 shutdown_connections 2024-04-26T23:43:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.154+0000 7efdea800640 1 -- 172.21.15.46:0/2192045015 wait complete. 2024-04-26T23:43:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.156+0000 7efdea800640 1 Processor -- start 2024-04-26T23:43:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.157+0000 7efdea800640 1 -- start start 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 0x7efde40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efde40cff60 0x7efde40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efde40706b0 con 0x7efde415c2d0 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efde40742b0 con 0x7efde4073570 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.158+0000 7efdea800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efde4070830 con 0x7efde40cff60 2024-04-26T23:43:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40252/0 (socket says 172.21.15.46:40252) 2024-04-26T23:43:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 -- 172.21.15.46:0/2821537533 learned_addr learned my addr 172.21.15.46:0/2821537533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3400640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 0x7efde40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde2a00640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efde40cff60 0x7efde40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 -- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efde40cff60 msgr2=0x7efde40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efde40cff60 0x7efde40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 -- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 msgr2=0x7efde40c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 0x7efde40c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3e00640 1 -- 172.21.15.46:0/2821537533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde40c9090 con 0x7efde415c2d0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.159+0000 7efde3400640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 0x7efde40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.160+0000 7efde3e00640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c780 tx=0x7efddc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.160+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc004b60 con 0x7efde415c2d0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.160+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc004d00 con 0x7efde415c2d0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.160+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00aa90 con 0x7efde415c2d0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.160+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde40c9370 con 0x7efde415c2d0 2024-04-26T23:43:14.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.161+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde40c9930 con 0x7efde415c2d0 2024-04-26T23:43:14.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.162+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde40693b0 con 0x7efde415c2d0 2024-04-26T23:43:14.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.164+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efddc00ac30 con 0x7efde415c2d0 2024-04-26T23:43:14.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.166+0000 7efddb400640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 0x7efdc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.166+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7efddc08bf90 con 0x7efde415c2d0 2024-04-26T23:43:14.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.166+0000 7efde3400640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 0x7efdc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.168+0000 7efde3400640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 0x7efdc406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efdd4006fd0 tx=0x7efdd4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.170+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efddc020070 con 0x7efde415c2d0 2024-04-26T23:43:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.186+0000 7fe587200640 1 Processor -- start 2024-04-26T23:43:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.186+0000 7fe587200640 1 -- start start 2024-04-26T23:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe587200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880d0740 0x7fe5880d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe587200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe588163320 0x7fe588163790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe587200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5880b7e60 con 0x7fe5880d1910 2024-04-26T23:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe587200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe5880b8440 con 0x7fe5880d0740 2024-04-26T23:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe587200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5880d0c70 con 0x7fe588163320 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe585400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880d0740 0x7fe5880d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe585400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880d0740 0x7fe5880d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40258/0 (socket says 172.21.15.46:40258) 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.188+0000 7fe585400640 1 -- 172.21.15.46:0/2080250911 learned_addr learned my addr 172.21.15.46:0/2080250911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.189+0000 7fe585400640 1 -- 172.21.15.46:0/2080250911 >> v2:172.21.15.142:3300/0 conn(0x7fe588163320 msgr2=0x7fe588163790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.189+0000 7fe586800640 1 --2- 172.21.15.46:0/2080250911 >> v2:172.21.15.142:3300/0 conn(0x7fe588163320 0x7fe588163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.189+0000 7fe585400640 1 --2- 172.21.15.46:0/2080250911 >> v2:172.21.15.142:3300/0 conn(0x7fe588163320 0x7fe588163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.189+0000 7fe585400640 1 -- 172.21.15.46:0/2080250911 >> v1:172.21.15.142:6789/0 conn(0x7fe5880d1910 legacy=0x7fe5880d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.189+0000 7fe585400640 1 -- 172.21.15.46:0/2080250911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5880b80d0 con 0x7fe5880d0740 2024-04-26T23:43:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.190+0000 7fe585400640 1 --2- 172.21.15.46:0/2080250911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d0740 0x7fe5880d2400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe58806a750 tx=0x7fe57402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f6d9d2b4d4b8dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.190+0000 7fe584a00640 1 -- 172.21.15.46:0/2080250911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574037070 con 0x7fe5880d0740 2024-04-26T23:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.190+0000 7fe584a00640 1 -- 172.21.15.46:0/2080250911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574032440 con 0x7fe5880d0740 2024-04-26T23:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.190+0000 7fe584a00640 1 -- 172.21.15.46:0/2080250911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574032810 con 0x7fe5880d0740 2024-04-26T23:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 -- 172.21.15.46:0/2080250911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d0740 msgr2=0x7fe5880d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 --2- 172.21.15.46:0/2080250911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d0740 0x7fe5880d2400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe58806a750 tx=0x7fe57402a230 comp rx=0 tx=0).stop 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 -- 172.21.15.46:0/2080250911 shutdown_connections 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 --2- 172.21.15.46:0/2080250911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d0740 0x7fe5880d2400 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 --2- 172.21.15.46:0/2080250911 >> v2:172.21.15.142:3300/0 conn(0x7fe588163320 0x7fe588163790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 -- 172.21.15.46:0/2080250911 >> 172.21.15.46:0/2080250911 conn(0x7fe5880d4f70 msgr2=0x7fe5880d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 -- 172.21.15.46:0/2080250911 shutdown_connections 2024-04-26T23:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.191+0000 7fe587200640 1 -- 172.21.15.46:0/2080250911 wait complete. 2024-04-26T23:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.193+0000 7f3d6f400640 1 Processor -- start 2024-04-26T23:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.193+0000 7f3d6f400640 1 -- start start 2024-04-26T23:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.193+0000 7fe587200640 1 Processor -- start 2024-04-26T23:43:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.193+0000 7fe587200640 1 -- start start 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 0x7fe5881f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 0x7fe5880a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe588163e50 con 0x7fe5880d1910 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe588163cd0 con 0x7fe5880d0740 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.194+0000 7fe587200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe588163fd0 con 0x7fe588163320 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40270/0 (socket says 172.21.15.46:40270) 2024-04-26T23:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 -- 172.21.15.46:0/3357438508 learned_addr learned my addr 172.21.15.46:0/3357438508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7f3d6f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d680b3300 0x7f3d680cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7f3d6f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 0x7f3d680cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 -- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 msgr2=0x7fe5880a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7f3d6f400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d680d3ca0 con 0x7f3d680d1d90 2024-04-26T23:43:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7f3d6f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680d3a50 con 0x7f3d680b3300 2024-04-26T23:43:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7f3d6f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680ce700 con 0x7f3d680c7a90 2024-04-26T23:43:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe586800640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 0x7fe5880a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585e00640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 0x7fe5881f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 0x7fe5880a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 -- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 msgr2=0x7fe5881f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 0x7fe5881f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.195+0000 7fe585400640 1 -- 172.21.15.46:0/3357438508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5880cb160 con 0x7fe5880d1910 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7fe585e00640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 0x7fe5881f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7fe586800640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 0x7fe5880a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d680b3300 0x7f3d680cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d680b3300 0x7f3d680cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40284/0 (socket says 172.21.15.46:40284) 2024-04-26T23:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 -- 172.21.15.46:0/4137719958 learned_addr learned my addr 172.21.15.46:0/4137719958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7fe585400640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6d400640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 0x7f3d680cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 -- 172.21.15.46:0/4137719958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 msgr2=0x7f3d680cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 0x7f3d680cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 -- 172.21.15.46:0/4137719958 >> v1:172.21.15.142:6789/0 conn(0x7f3d680d1d90 legacy=0x7f3d680d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6ca00640 1 -- 172.21.15.46:0/4137719958 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680ce950 con 0x7f3d680b3300 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.196+0000 7f3d6d400640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 0x7f3d680cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.197+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574041020 con 0x7fe5880d1910 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.197+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574030970 con 0x7fe5880d1910 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.197+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5880cb440 con 0x7fe5880d1910 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.197+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574037040 con 0x7fe5880d1910 2024-04-26T23:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.197+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5880cba00 con 0x7fe5880d1910 2024-04-26T23:43:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.198+0000 7f3d6ca00640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d680cf710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009ad0 tx=0x7f3d5c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cf7e519da09a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.198+0000 7f3d67400640 1 -- 172.21.15.46:0/4137719958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c036070 con 0x7f3d680b3300 2024-04-26T23:43:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7f3d67400640 1 -- 172.21.15.46:0/4137719958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c031600 con 0x7f3d680b3300 2024-04-26T23:43:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7f3d67400640 1 -- 172.21.15.46:0/4137719958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0319d0 con 0x7f3d680b3300 2024-04-26T23:43:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7fb4a4e00640 1 Processor -- start 2024-04-26T23:43:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7fb4a4e00640 1 -- start start 2024-04-26T23:43:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5880b7b90 con 0x7fe5880d1910 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7f3d6f400640 1 -- 172.21.15.46:0/4137719958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 msgr2=0x7f3d680cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.199+0000 7f3d6f400640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d680cf710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009ad0 tx=0x7f3d5c02a560 comp rx=0 tx=0).stop 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 -- 172.21.15.46:0/4137719958 shutdown_connections 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680c7a90 0x7f3d680cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 --2- 172.21.15.46:0/4137719958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d680cf710 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 -- 172.21.15.46:0/4137719958 >> 172.21.15.46:0/4137719958 conn(0x7f3d680d2f30 msgr2=0x7f3d680d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 -- 172.21.15.46:0/4137719958 shutdown_connections 2024-04-26T23:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7f3d6f400640 1 -- 172.21.15.46:0/4137719958 wait complete. 2024-04-26T23:43:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.200+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe574044020 con 0x7fe5880d1910 2024-04-26T23:43:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.201+0000 7fb4a4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4a00b5490 0x7fb4a0163050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.201+0000 7fb4a4e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb4a00d21c0 0x7fb4a00d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.201+0000 7fb4a4e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a0163590 con 0x7fb4a00b7050 2024-04-26T23:43:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.201+0000 7fb4a4e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a00d2b30 con 0x7fb4a00b5490 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.201+0000 7fb4a4e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a0163860 con 0x7fb4a00d21c0 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4a00b5490 0x7fb4a0163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4a00b5490 0x7fb4a0163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40288/0 (socket says 172.21.15.46:40288) 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 -- 172.21.15.46:0/10207529 learned_addr learned my addr 172.21.15.46:0/10207529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fe57e000640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 0x7fe56806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fe5740ac740 con 0x7fe5880d1910 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 -- 172.21.15.46:0/10207529 >> v2:172.21.15.142:3300/0 conn(0x7fb4a00d21c0 msgr2=0x7fb4a00d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 --2- 172.21.15.46:0/10207529 >> v2:172.21.15.142:3300/0 conn(0x7fb4a00d21c0 0x7fb4a00d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 -- 172.21.15.46:0/10207529 >> v1:172.21.15.142:6789/0 conn(0x7fb4a00b7050 legacy=0x7fb4a0160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.202+0000 7fb49de00640 1 -- 172.21.15.46:0/10207529 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a0163b40 con 0x7fb4a00b5490 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.203+0000 7fe585e00640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 0x7fe56806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.204+0000 7fb49de00640 1 --2- 172.21.15.46:0/10207529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b5490 0x7fb4a0163050 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00734b0 tx=0x7fb49002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9e4321cc2d8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.204+0000 7f3d6f400640 1 Processor -- start 2024-04-26T23:43:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.204+0000 7fe585e00640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 0x7fe56806ee30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe57800a830 tx=0x7fe578008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.204+0000 7f3d6f400640 1 -- start start 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680c7a90 0x7f3d68151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680d1d90 0x7f3d681522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680b3d40 con 0x7f3d680b3300 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680b3bc0 con 0x7f3d680c7a90 2024-04-26T23:43:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.205+0000 7f3d6f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680b36e0 con 0x7f3d680d1d90 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40296/0 (socket says 172.21.15.46:40296) 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 -- 172.21.15.46:0/1992272557 learned_addr learned my addr 172.21.15.46:0/1992272557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 -- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680d1d90 msgr2=0x7f3d681522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680d1d90 0x7f3d681522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 -- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680c7a90 msgr2=0x7f3d68151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680c7a90 0x7f3d68151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.206+0000 7f3d6ca00640 1 -- 172.21.15.46:0/1992272557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680ccb00 con 0x7f3d680b3300 2024-04-26T23:43:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.207+0000 7f3d6ca00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000c00 tx=0x7f3d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c040070 con 0x7f3d680b3300 2024-04-26T23:43:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c003610 con 0x7f3d680b3300 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c036040 con 0x7f3d680b3300 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d680ccde0 con 0x7f3d680b3300 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7fb49ca00640 1 -- 172.21.15.46:0/10207529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490037070 con 0x7fb4a00b5490 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7fb49ca00640 1 -- 172.21.15.46:0/10207529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb490032440 con 0x7fb4a00b5490 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7fb49ca00640 1 -- 172.21.15.46:0/10207529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490032810 con 0x7fb4a00b5490 2024-04-26T23:43:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5880b7b90 con 0x7fe5880d1910 2024-04-26T23:43:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.210+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681e6350 con 0x7f3d680b3300 2024-04-26T23:43:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.211+0000 7fb4a4e00640 1 -- 172.21.15.46:0/10207529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b5490 msgr2=0x7fb4a0163050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.211+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/10207529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b5490 0x7fb4a0163050 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00734b0 tx=0x7fb49002a230 comp rx=0 tx=0).stop 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 -- 172.21.15.46:0/10207529 shutdown_connections 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/10207529 >> v2:172.21.15.142:3300/0 conn(0x7fb4a00d21c0 0x7fb4a00d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/10207529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b5490 0x7fb4a0163050 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 -- 172.21.15.46:0/10207529 >> 172.21.15.46:0/10207529 conn(0x7fb4a006d850 msgr2=0x7fb4a006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 -- 172.21.15.46:0/10207529 shutdown_connections 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7fb4a4e00640 1 -- 172.21.15.46:0/10207529 wait complete. 2024-04-26T23:43:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.212+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d68069ac0 con 0x7f3d680b3300 2024-04-26T23:43:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.214+0000 7fb4a4e00640 1 Processor -- start 2024-04-26T23:43:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.214+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5c043070 con 0x7f3d680b3300 2024-04-26T23:43:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.214+0000 7fb4a4e00640 1 -- start start 2024-04-26T23:43:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7f3d64c00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4a00b5490 0x7fb4a01425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f3d5c0ac860 con 0x7f3d680b3300 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a00d21c0 0x7fb4a009d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0163e20 con 0x7fb4a00b7050 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a015fe50 con 0x7fb4a00d21c0 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb4a4e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a00b5870 con 0x7fb4a00b5490 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb49d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb49d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40298/0 (socket says 172.21.15.46:40298) 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7fb49d400640 1 -- 172.21.15.46:0/3613800365 learned_addr learned my addr 172.21.15.46:0/3613800365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.216+0000 7f3d67e00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 -- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4a00b5490 msgr2=0x7fb4a01425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4a00b5490 0x7fb4a01425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 -- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a00d21c0 msgr2=0x7fb4a009d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a00d21c0 0x7fb4a009d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 -- 172.21.15.46:0/3613800365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a009bb10 con 0x7fb4a00b7050 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.217+0000 7fb49d400640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c400 tx=0x7fb48c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.218+0000 7f3d67e00640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d54000f60 tx=0x7f3d540079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.224+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c017070 con 0x7fb4a00b7050 2024-04-26T23:43:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.224+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb48c0035f0 con 0x7fb4a00b7050 2024-04-26T23:43:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.224+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a009bdb0 con 0x7fb4a00b7050 2024-04-26T23:43:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.224+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c003a20 con 0x7fb4a00b7050 2024-04-26T23:43:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.224+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a009c370 con 0x7fb4a00b7050 2024-04-26T23:43:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.225+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d5c07aaf0 con 0x7f3d680b3300 2024-04-26T23:43:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.226+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a009c570 con 0x7fb4a00b7050 2024-04-26T23:43:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.229+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb48c01a070 con 0x7fb4a00b7050 2024-04-26T23:43:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.231+0000 7fb496000640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 0x7fb48006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.232+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fb48c08b880 con 0x7fb4a00b7050 2024-04-26T23:43:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.232+0000 7fb49de00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 0x7fb48006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.233+0000 7fb49de00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 0x7fb48006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb490034000 tx=0x7fb49002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.237+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4a009c570 con 0x7fb4a00b7050 2024-04-26T23:43:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.477+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7efde40b0c10 con 0x7efde415c2d0 2024-04-26T23:43:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.496+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7fe5880693b0 con 0x7fe5880d1910 2024-04-26T23:43:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.511+0000 7efddb400640 1 -- 172.21.15.46:0/2821537533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5174 v42) ==== 95+0+0 (secure 0 0 0) 0x7efddc05a220 con 0x7efde415c2d0 2024-04-26T23:43:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 5174 2024-04-26T23:43:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.511+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7f3d680d2b00 con 0x7f3d680b3300 2024-04-26T23:43:14.516 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:43:14.513+0000 7f074f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.513+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 msgr2=0x7efdc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.513+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 0x7efdc406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efdd4006fd0 tx=0x7efdd4008040 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 msgr2=0x7efde40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c780 tx=0x7efddc00cc40 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 shutdown_connections 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efdc406c8f0 0x7efdc406edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde415c2d0 0x7efde40b2440 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efde40cff60 0x7efde40b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 --2- 172.21.15.46:0/2821537533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efde4073570 0x7efde40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 >> 172.21.15.46:0/2821537533 conn(0x7efde40d1e20 msgr2=0x7efde415b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 shutdown_connections 2024-04-26T23:43:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.514+0000 7efdea800640 1 -- 172.21.15.46:0/2821537533 wait complete. 2024-04-26T23:43:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.518+0000 7fe57e000640 1 -- 172.21.15.46:0/3357438508 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6591 v43) ==== 95+0+0 (secure 0 0 0) 0x7fe57404b030 con 0x7fe5880d1910 2024-04-26T23:43:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 6591 2024-04-26T23:43:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7fb4a0142e80 con 0x7fb4a00b7050 2024-04-26T23:43:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 msgr2=0x7fe56806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 0x7fe56806ee30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe57800a830 tx=0x7fe578008040 comp rx=0 tx=0).stop 2024-04-26T23:43:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 msgr2=0x7fe5881562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe5740046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 shutdown_connections 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe56806c970 0x7fe56806ee30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe588163320 0x7fe5880a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880d1910 0x7fe5881562d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 --2- 172.21.15.46:0/3357438508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5880d0740 0x7fe5881f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.522+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 >> 172.21.15.46:0/3357438508 conn(0x7fe5880d4f70 msgr2=0x7fe58806d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.523+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 shutdown_connections 2024-04-26T23:43:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.523+0000 7fe587200640 1 -- 172.21.15.46:0/3357438508 wait complete. 2024-04-26T23:43:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.527+0000 7f3d64c00640 1 -- 172.21.15.46:0/1992272557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6579 v44) ==== 95+0+0 (secure 0 0 0) 0x7f3d5c07a3f0 con 0x7f3d680b3300 2024-04-26T23:43:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 6579 2024-04-26T23:43:14.534 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:43:14.530+0000 7f074f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:43:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.530+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 msgr2=0x7f3d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.530+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d54000f60 tx=0x7f3d540079b0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 msgr2=0x7f3d681e88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000c00 tx=0x7f3d5c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 shutdown_connections 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680d1d90 0x7f3d681522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680c7a90 0x7f3d68151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 --2- 172.21.15.46:0/1992272557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d680b3300 0x7f3d681e88f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 >> 172.21.15.46:0/1992272557 conn(0x7f3d680d2f30 msgr2=0x7f3d68071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 shutdown_connections 2024-04-26T23:43:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.531+0000 7f3d6f400640 1 -- 172.21.15.46:0/1992272557 wait complete. 2024-04-26T23:43:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.535+0000 7fb496000640 1 -- 172.21.15.46:0/3613800365 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6582 v45) ==== 95+0+0 (secure 0 0 0) 0x7fb48c0594b0 con 0x7fb4a00b7050 2024-04-26T23:43:14.539 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:43:14.539 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:43:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 6582 2024-04-26T23:43:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 msgr2=0x7fb48006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 0x7fb48006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb490034000 tx=0x7fb49002c000 comp rx=0 tx=0).stop 2024-04-26T23:43:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 msgr2=0x7fb4a009c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c400 tx=0x7fb48c00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 shutdown_connections 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb48006c8f0 0x7fb48006edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a00d21c0 0x7fb4a009d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a00b7050 0x7fb4a009c890 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 --2- 172.21.15.46:0/3613800365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4a00b5490 0x7fb4a01425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 >> 172.21.15.46:0/3613800365 conn(0x7fb4a006d850 msgr2=0x7fb4a0159000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 shutdown_connections 2024-04-26T23:43:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.539+0000 7fb4a4e00640 1 -- 172.21.15.46:0/3613800365 wait complete. 2024-04-26T23:43:14.546 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:43:14.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:43:14.554 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:43:14.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:43:14.555 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:43:14.555 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:14.559 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:43:14.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:43:14.560 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:43:14.560 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:14.608 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:43:14.609 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:14.614 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:43:14.614 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:14.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:14.705 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:43:14.705 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:43:14.751 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:43:14.751 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:43:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.784+0000 7f42f7400640 1 Processor -- start 2024-04-26T23:43:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.784+0000 7f42f7400640 1 -- start start 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.785+0000 7f42f7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f42f0160bd0 0x7f42f0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.785+0000 7f42f7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42f00732c0 0x7f42f0073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f7400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f0161940 con 0x7f42f006e510 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f7400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f42f0071f80 con 0x7f42f00732c0 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f7400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f42f0072260 con 0x7f42f0160bd0 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f4a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42f006e510 0x7f42f006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38844/0 (socket says 172.21.15.46:38844) 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f4a00640 1 -- 172.21.15.46:0/2715196741 learned_addr learned my addr 172.21.15.46:0/2715196741 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f5400640 1 --2- 172.21.15.46:0/2715196741 >> v2:172.21.15.46:3300/0 conn(0x7f42f00732c0 0x7f42f0073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42ef400640 1 -- 172.21.15.46:0/2715196741 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287330835 0 0) 0x7f42f0161940 con 0x7f42f006e510 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f5400640 1 -- 172.21.15.46:0/2715196741 >> v2:172.21.15.142:3300/0 conn(0x7f42f0160bd0 msgr2=0x7f42f0161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f5400640 1 --2- 172.21.15.46:0/2715196741 >> v2:172.21.15.142:3300/0 conn(0x7f42f0160bd0 0x7f42f0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f5400640 1 -- 172.21.15.46:0/2715196741 >> v1:172.21.15.46:6789/0 conn(0x7f42f006e510 legacy=0x7f42f006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.786+0000 7f42f5400640 1 -- 172.21.15.46:0/2715196741 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f0072540 con 0x7f42f00732c0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.787+0000 7f42f5400640 1 --2- 172.21.15.46:0/2715196741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f0073730 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009d90 tx=0x7f42e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c7bf3a8d064 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42ef400640 1 -- 172.21.15.46:0/2715196741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8007740 con 0x7f42f00732c0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42ef400640 1 -- 172.21.15.46:0/2715196741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e80078e0 con 0x7f42f00732c0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42ef400640 1 -- 172.21.15.46:0/2715196741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8007cb0 con 0x7f42f00732c0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 -- 172.21.15.46:0/2715196741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 msgr2=0x7f42f0073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 --2- 172.21.15.46:0/2715196741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f0073730 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009d90 tx=0x7f42e802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 -- 172.21.15.46:0/2715196741 shutdown_connections 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 --2- 172.21.15.46:0/2715196741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f0073730 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 --2- 172.21.15.46:0/2715196741 >> v2:172.21.15.142:3300/0 conn(0x7f42f0160bd0 0x7f42f0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 -- 172.21.15.46:0/2715196741 >> 172.21.15.46:0/2715196741 conn(0x7f42f006c530 msgr2=0x7f42f006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 -- 172.21.15.46:0/2715196741 shutdown_connections 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.788+0000 7f42f7400640 1 -- 172.21.15.46:0/2715196741 wait complete. 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.790+0000 7f42f7400640 1 Processor -- start 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.790+0000 7f42f7400640 1 -- start start 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42f006e510 0x7f42f015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42f0160bd0 0x7f42f01e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0161720 con 0x7f42f00732c0 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01615a0 con 0x7f42f0160bd0 2024-04-26T23:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42f7400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42f0072d40 con 0x7f42f006e510 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.791+0000 7f42efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40338/0 (socket says 172.21.15.46:40338) 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 -- 172.21.15.46:0/2708764168 learned_addr learned my addr 172.21.15.46:0/2708764168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 -- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42f006e510 msgr2=0x7f42f015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42f006e510 0x7f42f015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 -- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42f0160bd0 msgr2=0x7f42f01e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42f0160bd0 0x7f42f01e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 -- 172.21.15.46:0/2708764168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f01e69f0 con 0x7f42f00732c0 2024-04-26T23:43:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42efe00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00b4a0 tx=0x7f42dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc014070 con 0x7f42f00732c0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.792+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc010610 con 0x7f42f00732c0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.793+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc010a00 con 0x7f42f00732c0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.793+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f01e8980 con 0x7f42f00732c0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.793+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f01e6c60 con 0x7f42f00732c0 2024-04-26T23:43:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.794+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f01e6e60 con 0x7f42f00732c0 2024-04-26T23:43:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.795+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42dc017070 con 0x7f42f00732c0 2024-04-26T23:43:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.796+0000 7f42ecc00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.796+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f42dc08b7e0 con 0x7f42f00732c0 2024-04-26T23:43:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.796+0000 7f42f4a00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.797+0000 7f42f4a00640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f42e4007fd0 tx=0x7f42e4007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:14.800+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42f01e6e60 con 0x7f42f00732c0 2024-04-26T23:43:15.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.041+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42f006fb00 con 0x7f42f00732c0 2024-04-26T23:43:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.042+0000 7f42ecc00640 1 -- 172.21.15.46:0/2708764168 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+1868 (secure 0 0 0) 0x7f42dc0599d0 con 0x7f42f00732c0 2024-04-26T23:43:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:15.045 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:14.531007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T23:43:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-04-26T23:43:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.044+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 msgr2=0x7f42d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.044+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f42e4007fd0 tx=0x7f42e4007960 comp rx=0 tx=0).stop 2024-04-26T23:43:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.044+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 msgr2=0x7f42f00b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.044+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00b4a0 tx=0x7f42dc00b960 comp rx=0 tx=0).stop 2024-04-26T23:43:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 shutdown_connections 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42f0160bd0 0x7f42f01e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f00732c0 0x7f42f00b54c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 --2- 172.21.15.46:0/2708764168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42f006e510 0x7f42f015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 >> 172.21.15.46:0/2708764168 conn(0x7f42f006c530 msgr2=0x7f42f015b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 shutdown_connections 2024-04-26T23:43:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:15.045+0000 7f42f7400640 1 -- 172.21.15.46:0/2708764168 wait complete. 2024-04-26T23:43:15.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:42:59.049825+0000', 'modified': '2024-04-26T23:43:14.531007+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:43:15.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-26T23:43:16.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.690+0000 7fd3f0c00640 1 Processor -- start 2024-04-26T23:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.691+0000 7fd3f0c00640 1 -- start start 2024-04-26T23:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.692+0000 7fd3f0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.692+0000 7fd3f0c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd3ec0d2130 0x7fd3ec0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.692+0000 7fd3f0c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ec0d11f0 con 0x7fd3ec0b4000 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.692+0000 7fd3f0c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ec0d0ce0 con 0x7fd3ec0d2130 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.692+0000 7fd3f0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ec0d2990 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.693+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.693+0000 7fd3ebe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd3ec0d2130 0x7fd3ec0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.693+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:35674/0 (socket says 172.21.15.46:35674) 2024-04-26T23:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.693+0000 7fd3eb400640 1 -- 172.21.15.46:0/2005399310 learned_addr learned my addr 172.21.15.46:0/2005399310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.694+0000 7fd3eb400640 1 -- 172.21.15.46:0/2005399310 >> v2:172.21.15.142:3300/0 conn(0x7fd3ec0d2130 msgr2=0x7fd3ec0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.694+0000 7fd3eb400640 1 --2- 172.21.15.46:0/2005399310 >> v2:172.21.15.142:3300/0 conn(0x7fd3ec0d2130 0x7fd3ec0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.694+0000 7fd3eb400640 1 -- 172.21.15.46:0/2005399310 >> v1:172.21.15.142:6789/0 conn(0x7fd3ec0b4000 legacy=0x7fd3ec0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.694+0000 7fd3eb400640 1 -- 172.21.15.46:0/2005399310 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec0d5f20 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.695+0000 7fd3eb400640 1 --2- 172.21.15.46:0/2005399310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009870 tx=0x7fd3dc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08be98800f5d1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3eaa00640 1 -- 172.21.15.46:0/2005399310 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc037070 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3eaa00640 1 -- 172.21.15.46:0/2005399310 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3dc032440 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3eaa00640 1 -- 172.21.15.46:0/2005399310 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc032810 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2005399310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 msgr2=0x7fd3ec0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2005399310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009870 tx=0x7fd3dc02a270 comp rx=0 tx=0).stop 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2005399310 shutdown_connections 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2005399310 >> v2:172.21.15.142:3300/0 conn(0x7fd3ec0d2130 0x7fd3ec0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2005399310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec15f4e0 0x7fd3ec0d2df0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.696+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2005399310 >> 172.21.15.46:0/2005399310 conn(0x7fd3ec0d5010 msgr2=0x7fd3ec0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.697+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2005399310 shutdown_connections 2024-04-26T23:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.697+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2005399310 wait complete. 2024-04-26T23:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.699+0000 7fd3f0c00640 1 Processor -- start 2024-04-26T23:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.699+0000 7fd3f0c00640 1 -- start start 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3ec0b4000 0x7fd3ec0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec0d2130 0x7fd3ec0b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec0b1d60 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec0d5c40 con 0x7fd3ec0b4000 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3f0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ec15f8c0 con 0x7fd3ec0d2130 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40352/0 (socket says 172.21.15.46:40352) 2024-04-26T23:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.700+0000 7fd3ebe00640 1 -- 172.21.15.46:0/2542132751 learned_addr learned my addr 172.21.15.46:0/2542132751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3eb400640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3ec0b4000 0x7fd3ec0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3e3e00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec0d2130 0x7fd3ec0b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 -- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec0d2130 msgr2=0x7fd3ec0b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec0d2130 0x7fd3ec0b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 -- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3ec0b4000 msgr2=0x7fd3ec0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3ec0b4000 0x7fd3ec0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 -- 172.21.15.46:0/2542132751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec0cc280 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.701+0000 7fd3ebe00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400efc0 tx=0x7fd3e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.702+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4018020 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.702+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e4014480 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.702+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4014870 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.702+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec0cc560 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.702+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec0ccb20 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.703+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ec0693b0 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.704+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3e4005d10 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.705+0000 7fd3e3400640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 0x7fd3c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.705+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fd3e408c930 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.706+0000 7fd3eb400640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 0x7fd3c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.707+0000 7fd3eb400640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 0x7fd3c806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.710+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3e4022070 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.952+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3ec0d33d0 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.953+0000 7fd3e3400640 1 -- 172.21.15.46:0/2542132751 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5185 (secure 0 0 0) 0x7fd3e4010040 con 0x7fd3ec15f4e0 2024-04-26T23:43:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:16.957 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:15.571311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T23:43:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-04-26T23:43:16.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 msgr2=0x7fd3c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 0x7fd3c806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 msgr2=0x7fd3ec0ba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400efc0 tx=0x7fd3e400c490 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 shutdown_connections 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd3c806c8f0 0x7fd3c806edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ec15f4e0 0x7fd3ec0ba240 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3ec0d2130 0x7fd3ec0b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 --2- 172.21.15.46:0/2542132751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3ec0b4000 0x7fd3ec0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.956+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 >> 172.21.15.46:0/2542132751 conn(0x7fd3ec0d5010 msgr2=0x7fd3ec161b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.957+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 shutdown_connections 2024-04-26T23:43:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:16.957+0000 7fd3f0c00640 1 -- 172.21.15.46:0/2542132751 wait complete. 2024-04-26T23:43:17.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:42:59.049825+0000', 'modified': '2024-04-26T23:43:15.571311+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.46:6835/2393654178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2393654178}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2393654178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:43:17.384 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6876:up:reconnect 2024-04-26T23:43:18.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.600+0000 7f022fe00640 1 Processor -- start 2024-04-26T23:43:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.600+0000 7f022fe00640 1 -- start start 2024-04-26T23:43:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02301466b0 0x7f023006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 0x7f02300cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0230073ad0 con 0x7f0230160910 2024-04-26T23:43:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0230146a90 con 0x7f02301466b0 2024-04-26T23:43:18.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0230073d70 con 0x7f02300720e0 2024-04-26T23:43:18.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02301466b0 0x7f023006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0230160910 0x7f02300cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38854/0 (socket says 172.21.15.46:38854) 2024-04-26T23:43:18.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022f400640 1 -- 172.21.15.46:0/1583969895 learned_addr learned my addr 172.21.15.46:0/1583969895 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:18.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02301466b0 0x7f023006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40364/0 (socket says 172.21.15.46:40364) 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f0226000640 1 --2- 172.21.15.46:0/1583969895 >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 0x7f02300cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.602+0000 7f022ea00640 1 -- 172.21.15.46:0/1583969895 >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 msgr2=0x7f02300cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.603+0000 7f022ea00640 1 --2- 172.21.15.46:0/1583969895 >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 0x7f02300cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.603+0000 7f022ea00640 1 -- 172.21.15.46:0/1583969895 >> v1:172.21.15.46:6789/0 conn(0x7f0230160910 legacy=0x7f02300cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.603+0000 7f022ea00640 1 -- 172.21.15.46:0/1583969895 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230074050 con 0x7f02301466b0 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.603+0000 7f022e000640 1 -- 172.21.15.46:0/1583969895 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100462250 0 0) 0x7f0230073ad0 con 0x7f0230160910 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.603+0000 7f0226000640 1 --2- 172.21.15.46:0/1583969895 >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 0x7f02300cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.604+0000 7f022ea00640 1 --2- 172.21.15.46:0/1583969895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02301466b0 0x7f023006de80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f02200097a0 tx=0x7f022002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce01818051ca4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.604+0000 7f022e000640 1 -- 172.21.15.46:0/1583969895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220037070 con 0x7f02301466b0 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.604+0000 7f022e000640 1 -- 172.21.15.46:0/1583969895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0220032440 con 0x7f02301466b0 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.604+0000 7f022e000640 1 -- 172.21.15.46:0/1583969895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220032810 con 0x7f02301466b0 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 -- 172.21.15.46:0/1583969895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02301466b0 msgr2=0x7f023006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 --2- 172.21.15.46:0/1583969895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02301466b0 0x7f023006de80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f02200097a0 tx=0x7f022002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 -- 172.21.15.46:0/1583969895 shutdown_connections 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 --2- 172.21.15.46:0/1583969895 >> v2:172.21.15.142:3300/0 conn(0x7f02300720e0 0x7f02300cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 --2- 172.21.15.46:0/1583969895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02301466b0 0x7f023006de80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 -- 172.21.15.46:0/1583969895 >> 172.21.15.46:0/1583969895 conn(0x7f02300b26d0 msgr2=0x7f02300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 -- 172.21.15.46:0/1583969895 shutdown_connections 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.605+0000 7f022fe00640 1 -- 172.21.15.46:0/1583969895 wait complete. 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.607+0000 7f022fe00640 1 Processor -- start 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.607+0000 7f022fe00640 1 -- start start 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02300720e0 0x7f0230159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02301466b0 0x7f02300b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f023006e3c0 con 0x7f0230160910 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02300cb470 con 0x7f02301466b0 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02300cdda0 con 0x7f02300720e0 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40374/0 (socket says 172.21.15.46:40374) 2024-04-26T23:43:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022f400640 1 -- 172.21.15.46:0/76963160 learned_addr learned my addr 172.21.15.46:0/76963160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.608+0000 7f022ea00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02300720e0 0x7f0230159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f0226000640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02301466b0 0x7f02300b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 -- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02300720e0 msgr2=0x7f0230159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02300720e0 0x7f0230159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 -- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02301466b0 msgr2=0x7f02300b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02301466b0 0x7f02300b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 -- 172.21.15.46:0/76963160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02300b5c40 con 0x7f0230160910 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022f400640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f022800c440 tx=0x7f022800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280074e0 con 0x7f0230160910 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280047e0 con 0x7f0230160910 2024-04-26T23:43:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228004bd0 con 0x7f0230160910 2024-04-26T23:43:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.609+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02300b5f20 con 0x7f0230160910 2024-04-26T23:43:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.610+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02300b3d70 con 0x7f0230160910 2024-04-26T23:43:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.611+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02300693b0 con 0x7f0230160910 2024-04-26T23:43:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.612+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0228011070 con 0x7f0230160910 2024-04-26T23:43:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.613+0000 7f0227400640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 0x7f020c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.614+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f022808b850 con 0x7f0230160910 2024-04-26T23:43:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.614+0000 7f022ea00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 0x7f020c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:18.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.616+0000 7f022ea00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 0x7f020c06edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0220034000 tx=0x7f022002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:18.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.618+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0228059ad0 con 0x7f0230160910 2024-04-26T23:43:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.864+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f023006e770 con 0x7f0230160910 2024-04-26T23:43:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.865+0000 7f0227400640 1 -- 172.21.15.46:0/76963160 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f02280593d0 con 0x7f0230160910 2024-04-26T23:43:18.869 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:18.869 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.868+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 msgr2=0x7f020c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.868+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 0x7f020c06edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0220034000 tx=0x7f022002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.868+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 msgr2=0x7f02300b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.868+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f022800c440 tx=0x7f022800c900 comp rx=0 tx=0).stop 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 shutdown_connections 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f020c06c8f0 0x7f020c06edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0230160910 0x7f02300b51c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02301466b0 0x7f02300b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 --2- 172.21.15.46:0/76963160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02300720e0 0x7f0230159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 >> 172.21.15.46:0/76963160 conn(0x7f02300b26d0 msgr2=0x7f02300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 shutdown_connections 2024-04-26T23:43:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:18.869+0000 7f022fe00640 1 -- 172.21.15.46:0/76963160 wait complete. 2024-04-26T23:43:19.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:42:59.049825+0000', 'modified': '2024-04-26T23:43:18.573173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2393654178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2393654178}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2393654178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6876, 'qdb_cluster': [6876]} 2024-04-26T23:43:19.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:43:19.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:43:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.509+0000 7fc70b200640 1 Processor -- start 2024-04-26T23:43:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.509+0000 7fc70b200640 1 -- start start 2024-04-26T23:43:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc70b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 0x7fc70c0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc70b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c1633c0 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc70b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c1636a0 con 0x7fc70c15f1f0 2024-04-26T23:43:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc70b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c163980 con 0x7fc70c162e20 2024-04-26T23:43:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc709400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc70c0ce1b0 0x7fc70c0cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38860/0 (socket says 172.21.15.46:38860) 2024-04-26T23:43:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.511+0000 7fc709400640 1 -- 172.21.15.46:0/3168879773 learned_addr learned my addr 172.21.15.46:0/3168879773 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc70a800640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 0x7fc70c0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073086121 0 0) 0x7fc70c1633c0 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f0003520 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574856213 0 0) 0x7fc6f0003520 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 msgr2=0x7fc70c0c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 0x7fc70c0c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 >> v1:172.21.15.142:6789/0 conn(0x7fc70c15f1f0 legacy=0x7fc70c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c163c60 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.512+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc6f8001670 con 0x7fc70c0ce1b0 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 >> v1:172.21.15.46:6789/0 conn(0x7fc70c0ce1b0 legacy=0x7fc70c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f00039e0 0x7fc6f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 0x7fc6f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc6f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c163980 con 0x7fc6f0004d40 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f000bb50 con 0x7fc6f00039e0 2024-04-26T23:43:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f000bd50 con 0x7fc70c162e20 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc6f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 -- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 msgr2=0x7fc6f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc709e00640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f00039e0 0x7fc6f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c162e20 0x7fc6f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 -- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f00039e0 msgr2=0x7fc6f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f00039e0 0x7fc6f0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.513+0000 7fc70a800640 1 -- 172.21.15.46:0/3168879773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f0003520 con 0x7fc6f0004d40 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.514+0000 7fc70a800640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc6f00051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc704009fc0 tx=0x7fc6f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480a24ca03c350e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704009c10 con 0x7fc6f0004d40 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704036070 con 0x7fc6f0004d40 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc708a00640 1 -- 172.21.15.46:0/3168879773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70402d070 con 0x7fc6f0004d40 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70b200640 1 -- 172.21.15.46:0/3168879773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 msgr2=0x7fc6f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70b200640 1 --2- 172.21.15.46:0/3168879773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc6f00051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc704009fc0 tx=0x7fc6f0005830 comp rx=0 tx=0).stop 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70a800640 1 -- 172.21.15.46:0/3168879773 reap_dead start 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70b200640 1 -- 172.21.15.46:0/3168879773 shutdown_connections 2024-04-26T23:43:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70b200640 1 -- 172.21.15.46:0/3168879773 >> 172.21.15.46:0/3168879773 conn(0x7fc70c0d1c70 msgr2=0x7fc70c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.515+0000 7fc70b200640 1 -- 172.21.15.46:0/3168879773 shutdown_connections 2024-04-26T23:43:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.516+0000 7fc70b200640 1 -- 172.21.15.46:0/3168879773 wait complete. 2024-04-26T23:43:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.517+0000 7fc70b200640 1 Processor -- start 2024-04-26T23:43:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.517+0000 7fc70b200640 1 -- start start 2024-04-26T23:43:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70c161fa0 0x7fc70c0af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c0b2a30 0x7fc70c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0c6ec0 con 0x7fc6f0004d40 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0c6d40 con 0x7fc70c161fa0 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.518+0000 7fc70b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c0c7040 con 0x7fc70c0b2a30 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40380/0 (socket says 172.21.15.46:40380) 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 -- 172.21.15.46:0/1308889278 learned_addr learned my addr 172.21.15.46:0/1308889278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 -- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c0b2a30 msgr2=0x7fc70c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c0b2a30 0x7fc70c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 -- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70c161fa0 msgr2=0x7fc70c0af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709400640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70c161fa0 0x7fc70c0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70c161fa0 0x7fc70c0af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.519+0000 7fc709e00640 1 -- 172.21.15.46:0/1308889278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c0afb30 con 0x7fc6f0004d40 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc709e00640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00d4c0 tx=0x7fc6fc00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc702000640 1 -- 172.21.15.46:0/1308889278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc005480 con 0x7fc6f0004d40 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc702000640 1 -- 172.21.15.46:0/1308889278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc017070 con 0x7fc6f0004d40 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc702000640 1 -- 172.21.15.46:0/1308889278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc003810 con 0x7fc6f0004d40 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c0afe10 con 0x7fc6f0004d40 2024-04-26T23:43:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.520+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c0b03d0 con 0x7fc6f0004d40 2024-04-26T23:43:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.522+0000 7fc702000640 1 -- 172.21.15.46:0/1308889278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6fc0039b0 con 0x7fc6f0004d40 2024-04-26T23:43:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.524+0000 7fc702000640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 0x7fc6ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.524+0000 7fc702000640 1 -- 172.21.15.46:0/1308889278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fc6fc08bc00 con 0x7fc6f0004d40 2024-04-26T23:43:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.524+0000 7fc709400640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 0x7fc6ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.525+0000 7fc713a4f740 1 Processor -- start 2024-04-26T23:43:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.525+0000 7fc713a4f740 1 -- start start 2024-04-26T23:43:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.525+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326aeb7b0 0x559326aed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc713a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559326a4f600 con 0x559326b27e80 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5593269ec100 con 0x559326aeb7b0 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559326a4b5c0 con 0x559326b16270 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40396/0 (socket says 172.21.15.46:40396) 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 -- 172.21.15.46:0/2999070580 learned_addr learned my addr 172.21.15.46:0/2999070580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 0x7fc6ec06ede0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000fb0 tx=0x7fc6f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709e00640 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326aeb7b0 0x559326aed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc70a800640 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 -- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 msgr2=0x559326b166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 -- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326aeb7b0 msgr2=0x559326aed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.526+0000 7fc709400640 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326aeb7b0 0x559326aed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.527+0000 7fc709400640 1 -- 172.21.15.46:0/2999070580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593267612d0 con 0x559326b27e80 2024-04-26T23:43:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.527+0000 7fc709400640 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006c10 tx=0x7fc6f8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5006d10a9ef3f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.527+0000 7fc6f6600640 1 -- 172.21.15.46:0/2999070580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8006e40 con 0x559326b27e80 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.527+0000 7fc6f6600640 1 -- 172.21.15.46:0/2999070580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f80047e0 con 0x559326b27e80 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.527+0000 7fc6f6600640 1 -- 172.21.15.46:0/2999070580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004b90 con 0x559326b27e80 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 -- 172.21.15.46:0/2999070580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 msgr2=0x559326aebb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006c10 tx=0x7fc6f8010000 comp rx=0 tx=0).stop 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 -- 172.21.15.46:0/2999070580 shutdown_connections 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326aebb90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 --2- 172.21.15.46:0/2999070580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326aeb7b0 0x559326aed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 -- 172.21.15.46:0/2999070580 >> 172.21.15.46:0/2999070580 conn(0x559326a7fd10 msgr2=0x559326b31270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 -- 172.21.15.46:0/2999070580 shutdown_connections 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.528+0000 7fc713a4f740 1 -- 172.21.15.46:0/2999070580 wait complete. 2024-04-26T23:43:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.529+0000 7fc713a4f740 1 -- start start 2024-04-26T23:43:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.529+0000 7fc713a4f740 1 Processor -- start 2024-04-26T23:43:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593269d6f70 con 0x559326aeb7b0 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559326b13dc0 con 0x559326b27e80 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559326b336b0 con 0x559326b16270 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.530+0000 7fc713a4f740 10 client.0 _ll_register_callbacks cb 0x559326a45480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40410/0 (socket says 172.21.15.46:40410) 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 -- 172.21.15.46:0/806024770 learned_addr learned my addr 172.21.15.46:0/806024770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc70a800640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709400640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 -- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 msgr2=0x559326b328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b328b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 -- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 msgr2=0x559326b32df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b32df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 -- 172.21.15.46:0/806024770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932682b6b0 con 0x559326aeb7b0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc70a800640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc709e00640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0595e0 tx=0x7fc6fc059e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1add4cbc023cbf8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc6eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.531+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc003ca0 con 0x559326aeb7b0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fc6fc05eb90 con 0x559326aeb7b0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f80047e0 con 0x559326aeb7b0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc6fc076050 con 0x559326aeb7b0 2024-04-26T23:43:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 10 client.6909 fetch_fsmap learned FSMap version 50 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x559326b0cae0 con 0x559326aeb7b0 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f80047e0 con 0x559326aeb7b0 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc6fc058aa0 con 0x559326aeb7b0 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 10 client.6909 fetch_fsmap finished waiting for FSMap version 50 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 10 client.6909 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 20 client.6909 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b4cec0 0x559326b4f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 4 client.6909 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.532+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 --> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559326a5d210 con 0x559326b4cec0 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.533+0000 7fc709400640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b4cec0 0x559326b4f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.533+0000 7fc709400640 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b4cec0 0x559326b4f2b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21925e919ab4704b server_cookie=27d8525b36ad0b79 in_seq=0 out_seq=0 2024-04-26T23:43:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.533+0000 7fc6eaa00640 10 client.6909 ms_handle_connect on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.534+0000 7fc6eaa00640 1 -- 172.21.15.46:0/806024770 <== mds.0 v2:172.21.15.46:6833/2393654178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559326a5d210 con 0x559326b4cec0 2024-04-26T23:43:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.534+0000 7fc6eaa00640 10 client.6909 handle_command_reply: tid=0 2024-04-26T23:43:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.534+0000 7fc713a4f740 1 client.6909 shutdown 2024-04-26T23:43:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.534+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 msgr2=0x559326b08390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.534+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0595e0 tx=0x7fc6fc059e90 comp rx=0 tx=0).stop 2024-04-26T23:43:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 shutdown_connections 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b4cec0 0x559326b4f2b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326b16270 0x559326b328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 --2- 172.21.15.46:0/806024770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326aeb7b0 0x559326b08390 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 >> 172.21.15.46:0/806024770 conn(0x559326a7fd10 msgr2=0x559326b141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc6eaa00640 0 client.6909 ms_handle_reset on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 shutdown_connections 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 1 -- 172.21.15.46:0/806024770 wait complete. 2024-04-26T23:43:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.535+0000 7fc713a4f740 20 client.6909 trim_cache size 0 max 16384 2024-04-26T23:43:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.555+0000 7fc713a4f740 1 Processor -- start 2024-04-26T23:43:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.555+0000 7fc713a4f740 1 -- start start 2024-04-26T23:43:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326b43af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b12410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c2014e0 con 0x559326b16270 2024-04-26T23:43:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c200840 con 0x559326b27e80 2024-04-26T23:43:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c200e60 con 0x559326aeb7b0 2024-04-26T23:43:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc709400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc709400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40418/0 (socket says 172.21.15.46:40418) 2024-04-26T23:43:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.556+0000 7fc709400640 1 -- 172.21.15.46:0/1072443927 learned_addr learned my addr 172.21.15.46:0/1072443927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 -- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 msgr2=0x559326b43af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326b43af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 -- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 msgr2=0x559326b12410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b12410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 -- 172.21.15.46:0/1072443927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559326b44c60 con 0x559326b16270 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc709400640 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800db70 tx=0x7fc6f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5929cb03a8f91d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc6f5c00640 1 -- 172.21.15.46:0/1072443927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800dea0 con 0x559326b16270 2024-04-26T23:43:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc6f5c00640 1 -- 172.21.15.46:0/1072443927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f8018030 con 0x559326b16270 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.557+0000 7fc6f5c00640 1 -- 172.21.15.46:0/1072443927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004c00 con 0x559326b16270 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 -- 172.21.15.46:0/1072443927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 msgr2=0x559326b44030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800db70 tx=0x7fc6f8014000 comp rx=0 tx=0).stop 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 -- 172.21.15.46:0/1072443927 shutdown_connections 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b27e80 0x559326b12410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b16270 0x559326b44030 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 --2- 172.21.15.46:0/1072443927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326b43af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 -- 172.21.15.46:0/1072443927 >> 172.21.15.46:0/1072443927 conn(0x559326a7fd10 msgr2=0x559326b141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 -- 172.21.15.46:0/1072443927 shutdown_connections 2024-04-26T23:43:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.558+0000 7fc713a4f740 1 -- 172.21.15.46:0/1072443927 wait complete. 2024-04-26T23:43:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.559+0000 7fc713a4f740 1 -- start start 2024-04-26T23:43:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.559+0000 7fc713a4f740 1 Processor -- start 2024-04-26T23:43:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326ba87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b16270 0x559326b54400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559326b354a0 con 0x559326b27e80 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559326b333f0 con 0x559326b16270 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559326b44ae0 con 0x559326aeb7b0 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc713a4f740 10 client.0 _ll_register_callbacks cb 0x7fc6fc001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc70a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc70a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40420/0 (socket says 172.21.15.46:40420) 2024-04-26T23:43:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc70a800640 1 -- 172.21.15.46:0/3190583137 learned_addr learned my addr 172.21.15.46:0/3190583137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc709e00640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326ba87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.560+0000 7fc709400640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b16270 0x559326b54400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 -- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 msgr2=0x559326ba87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326ba87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 -- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b16270 msgr2=0x559326b54400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b16270 0x559326b54400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 -- 172.21.15.46:0/3190583137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559326b0c120 con 0x559326b27e80 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc70a800640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e230 tx=0x7fc704009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bf66df58fbec65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc6eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004450 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fc7040045f0 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.561+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f8018030 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc704009cd0 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 10 client.6915 fetch_fsmap learned FSMap version 50 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x559326bab0e0 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f8018030 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc704051020 con 0x559326b27e80 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 10 client.6915 fetch_fsmap finished waiting for FSMap version 50 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 10 client.6915 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 20 client.6915 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b532d0 0x559326b536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 4 client.6915 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 --> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] -- command(tid 0: {"prefix": "status"}) -- 0x559326a57080 con 0x559326b532d0 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.562+0000 7fc709e00640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b532d0 0x559326b536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc709e00640 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b532d0 0x559326b536b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9430b7fbc43d8665 server_cookie=9b930ad0370c3ce5 in_seq=0 out_seq=0 2024-04-26T23:43:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc6eaa00640 10 client.6915 ms_handle_connect on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc6eaa00640 1 -- 172.21.15.46:0/3190583137 <== mds.0 v2:172.21.15.46:6833/2393654178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559326a57080 con 0x559326b532d0 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc6eaa00640 10 client.6915 handle_command_reply: tid=0 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc713a4f740 1 client.6915 shutdown 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 msgr2=0x559326b54940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.563+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e230 tx=0x7fc704009950 comp rx=0 tx=0).stop 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 shutdown_connections 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559326b27e80 0x559326b54940 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x559326b532d0 0x559326b536b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559326b16270 0x559326b54400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 --2- 172.21.15.46:0/3190583137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559326aeb7b0 0x559326ba87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc6eaa00640 0 client.6915 ms_handle_reset on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 >> 172.21.15.46:0/3190583137 conn(0x559326a7fd10 msgr2=0x559326b0b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 shutdown_connections 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 1 -- 172.21.15.46:0/3190583137 wait complete. 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.564+0000 7fc713a4f740 20 client.6915 trim_cache size 0 max 16384 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6876, 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:43:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.0054819820000001, 2024-04-26T23:43:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 50, 2024-04-26T23:43:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 69, 2024-04-26T23:43:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 69, 2024-04-26T23:43:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 4.8090236539999998 2024-04-26T23:43:19.571 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 msgr2=0x7fc6ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 0x7fc6ec06ede0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000fb0 tx=0x7fc6f8009040 comp rx=0 tx=0).stop 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 msgr2=0x7fc70c153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00d4c0 tx=0x7fc6fc00d980 comp rx=0 tx=0).stop 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 shutdown_connections 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6ec06c920 0x7fc6ec06ede0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70c0b2a30 0x7fc70c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70c161fa0 0x7fc70c0af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 --2- 172.21.15.46:0/1308889278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6f0004d40 0x7fc70c153940 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 >> 172.21.15.46:0/1308889278 conn(0x7fc70c0d1c70 msgr2=0x7fc70c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 shutdown_connections 2024-04-26T23:43:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.565+0000 7fc70b200640 1 -- 172.21.15.46:0/1308889278 wait complete. 2024-04-26T23:43:19.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_size 100 2024-04-26T23:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.796+0000 7f1b66a00640 1 Processor -- start 2024-04-26T23:43:19.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.796+0000 7f1b66a00640 1 -- start start 2024-04-26T23:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b66a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b6015cec0 0x7f1b600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b66a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b60073d00 0x7f1b600b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b66a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b600ce7d0 con 0x7f1b600cdb30 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b66a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b600ceab0 con 0x7f1b6015cec0 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b66a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b600ced90 con 0x7f1b60073d00 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b65600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b6015cec0 0x7f1b600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b65600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b6015cec0 0x7f1b600cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40422/0 (socket says 172.21.15.46:40422) 2024-04-26T23:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.798+0000 7f1b65600640 1 -- 172.21.15.46:0/2292777234 learned_addr learned my addr 172.21.15.46:0/2292777234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.799+0000 7f1b65600640 1 -- 172.21.15.46:0/2292777234 >> v2:172.21.15.142:3300/0 conn(0x7f1b60073d00 msgr2=0x7f1b600b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.799+0000 7f1b65600640 1 --2- 172.21.15.46:0/2292777234 >> v2:172.21.15.142:3300/0 conn(0x7f1b60073d00 0x7f1b600b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.799+0000 7f1b65600640 1 -- 172.21.15.46:0/2292777234 >> v1:172.21.15.142:6789/0 conn(0x7f1b600cdb30 legacy=0x7f1b600735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.799+0000 7f1b65600640 1 -- 172.21.15.46:0/2292777234 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b600cf070 con 0x7f1b6015cec0 2024-04-26T23:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.800+0000 7f1b65600640 1 --2- 172.21.15.46:0/2292777234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6015cec0 0x7f1b600cd5f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1b600cd4f0 tx=0x7f1b5402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81d256762c8434 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.800+0000 7f1b5fe00640 1 -- 172.21.15.46:0/2292777234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54037070 con 0x7f1b6015cec0 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.800+0000 7f1b5fe00640 1 -- 172.21.15.46:0/2292777234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b54032440 con 0x7f1b6015cec0 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.800+0000 7f1b5fe00640 1 -- 172.21.15.46:0/2292777234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54032810 con 0x7f1b6015cec0 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 -- 172.21.15.46:0/2292777234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6015cec0 msgr2=0x7f1b600cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2292777234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6015cec0 0x7f1b600cd5f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1b600cd4f0 tx=0x7f1b5402a230 comp rx=0 tx=0).stop 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 -- 172.21.15.46:0/2292777234 shutdown_connections 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2292777234 >> v2:172.21.15.142:3300/0 conn(0x7f1b60073d00 0x7f1b600b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2292777234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6015cec0 0x7f1b600cd5f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 -- 172.21.15.46:0/2292777234 >> 172.21.15.46:0/2292777234 conn(0x7f1b600b27b0 msgr2=0x7f1b600b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 -- 172.21.15.46:0/2292777234 shutdown_connections 2024-04-26T23:43:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.801+0000 7f1b66a00640 1 -- 172.21.15.46:0/2292777234 wait complete. 2024-04-26T23:43:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.803+0000 7f1b66a00640 1 Processor -- start 2024-04-26T23:43:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.803+0000 7f1b66a00640 1 -- start start 2024-04-26T23:43:19.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 0x7f1b600c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 0x7f1b600ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b600b4080 con 0x7f1b60073d00 2024-04-26T23:43:19.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6015d2a0 con 0x7f1b6015cec0 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b66a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b600b4200 con 0x7f1b600cdb30 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b65600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b65600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40434/0 (socket says 172.21.15.46:40434) 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b65600640 1 -- 172.21.15.46:0/2376827273 learned_addr learned my addr 172.21.15.46:0/2376827273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.804+0000 7f1b64c00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 0x7f1b600c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 -- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 msgr2=0x7f1b600c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b66000640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 0x7f1b600ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 0x7f1b600c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 -- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 msgr2=0x7f1b600ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 0x7f1b600ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 -- 172.21.15.46:0/2376827273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b600c7470 con 0x7f1b60073d00 2024-04-26T23:43:19.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b64c00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 0x7f1b600c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:19.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b66000640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 0x7f1b600ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:19.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b65600640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1b5402a270 tx=0x7f1b540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54041070 con 0x7f1b60073d00 2024-04-26T23:43:19.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5403c870 con 0x7f1b60073d00 2024-04-26T23:43:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54037040 con 0x7f1b60073d00 2024-04-26T23:43:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.805+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b601f50f0 con 0x7f1b60073d00 2024-04-26T23:43:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.806+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601f4da0 con 0x7f1b60073d00 2024-04-26T23:43:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.807+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b600693b0 con 0x7f1b60073d00 2024-04-26T23:43:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.808+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b54044070 con 0x7f1b60073d00 2024-04-26T23:43:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.809+0000 7f1b5d600640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 0x7f1b4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.809+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f1b540ac930 con 0x7f1b60073d00 2024-04-26T23:43:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.809+0000 7f1b64c00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 0x7f1b4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.811+0000 7f1b64c00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 0x7f1b4006edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007fd0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:19.813+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b5407ab10 con 0x7f1b60073d00 2024-04-26T23:43:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.017+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f1b600ccec0 con 0x7f1b60073d00 2024-04-26T23:43:20.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.023+0000 7f1b5d600640 1 -- 172.21.15.46:0/2376827273 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v18)=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f1b5407a410 con 0x7f1b60073d00 2024-04-26T23:43:20.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 msgr2=0x7f1b4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 0x7f1b4006edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007fd0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-04-26T23:43:20.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 msgr2=0x7f1b600c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1b5402a270 tx=0x7f1b540046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 shutdown_connections 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4006c8f0 0x7f1b4006edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b6015cec0 0x7f1b600ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b600cdb30 0x7f1b600c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 --2- 172.21.15.46:0/2376827273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b60073d00 0x7f1b600c69f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.026+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 >> 172.21.15.46:0/2376827273 conn(0x7f1b600b27b0 msgr2=0x7f1b600b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.027+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 shutdown_connections 2024-04-26T23:43:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.027+0000 7f1b66a00640 1 -- 172.21.15.46:0/2376827273 wait complete. 2024-04-26T23:43:20.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_merge_size 30 2024-04-26T23:43:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.261+0000 7f79e6000640 1 Processor -- start 2024-04-26T23:43:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.261+0000 7f79e6000640 1 -- start start 2024-04-26T23:43:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79e015f210 0x7f79e00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e6000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79e00cd610 0x7f79e00cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e6000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e00af570 con 0x7f79e00ae8f0 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f79e00cdfb0 con 0x7f79e015f210 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e6000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79e00af850 con 0x7f79e00cd610 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79e015f210 0x7f79e00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79e015f210 0x7f79e00cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40438/0 (socket says 172.21.15.46:40438) 2024-04-26T23:43:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.263+0000 7f79e4c00640 1 -- 172.21.15.46:0/23602435 learned_addr learned my addr 172.21.15.46:0/23602435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.264+0000 7f79dfe00640 1 --2- 172.21.15.46:0/23602435 >> v2:172.21.15.142:3300/0 conn(0x7f79e00cd610 0x7f79e00cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.264+0000 7f79e4c00640 1 -- 172.21.15.46:0/23602435 >> v2:172.21.15.142:3300/0 conn(0x7f79e00cd610 msgr2=0x7f79e00cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.264+0000 7f79e4c00640 1 --2- 172.21.15.46:0/23602435 >> v2:172.21.15.142:3300/0 conn(0x7f79e00cd610 0x7f79e00cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.264+0000 7f79e4c00640 1 -- 172.21.15.46:0/23602435 >> v1:172.21.15.142:6789/0 conn(0x7f79e00ae8f0 legacy=0x7f79e00aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.264+0000 7f79e4c00640 1 -- 172.21.15.46:0/23602435 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e00cb680 con 0x7f79e015f210 2024-04-26T23:43:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.265+0000 7f79e4c00640 1 --2- 172.21.15.46:0/23602435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e015f210 0x7f79e00cc160 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79e00b3710 tx=0x7f79d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5a1e517b934466 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.265+0000 7f79df400640 1 -- 172.21.15.46:0/23602435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4037070 con 0x7f79e015f210 2024-04-26T23:43:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.265+0000 7f79df400640 1 -- 172.21.15.46:0/23602435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d4032440 con 0x7f79e015f210 2024-04-26T23:43:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.265+0000 7f79df400640 1 -- 172.21.15.46:0/23602435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4032810 con 0x7f79e015f210 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 -- 172.21.15.46:0/23602435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e015f210 msgr2=0x7f79e00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 --2- 172.21.15.46:0/23602435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e015f210 0x7f79e00cc160 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79e00b3710 tx=0x7f79d402a230 comp rx=0 tx=0).stop 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 -- 172.21.15.46:0/23602435 shutdown_connections 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 --2- 172.21.15.46:0/23602435 >> v2:172.21.15.142:3300/0 conn(0x7f79e00cd610 0x7f79e00cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 --2- 172.21.15.46:0/23602435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e015f210 0x7f79e00cc160 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 -- 172.21.15.46:0/23602435 >> 172.21.15.46:0/23602435 conn(0x7f79e00b27b0 msgr2=0x7f79e00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 -- 172.21.15.46:0/23602435 shutdown_connections 2024-04-26T23:43:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.266+0000 7f79e6000640 1 -- 172.21.15.46:0/23602435 wait complete. 2024-04-26T23:43:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.268+0000 7f79e6000640 1 Processor -- start 2024-04-26T23:43:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.268+0000 7f79e6000640 1 -- start start 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e00cd610 0x7f79e01f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e015f210 0x7f79e01f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79e015f5f0 con 0x7f79e00ae8f0 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79e00cbc40 con 0x7f79e015f210 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79e0071e80 con 0x7f79e00cd610 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.269+0000 7f79e4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40440/0 (socket says 172.21.15.46:40440) 2024-04-26T23:43:20.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 -- 172.21.15.46:0/4180525985 learned_addr learned my addr 172.21.15.46:0/4180525985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 -- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e00cd610 msgr2=0x7f79e01f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e00cd610 0x7f79e01f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 -- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e015f210 msgr2=0x7f79e01f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e015f210 0x7f79e01f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 -- 172.21.15.46:0/4180525985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e01f5640 con 0x7f79e00ae8f0 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.270+0000 7f79e4c00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79d4000c00 tx=0x7f79d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.271+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4041020 con 0x7f79e00ae8f0 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.271+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d40306c0 con 0x7f79e00ae8f0 2024-04-26T23:43:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.271+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e01f5910 con 0x7f79e00ae8f0 2024-04-26T23:43:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.271+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4037040 con 0x7f79e00ae8f0 2024-04-26T23:43:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.271+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01f5ed0 con 0x7f79e00ae8f0 2024-04-26T23:43:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.272+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e00693b0 con 0x7f79e00ae8f0 2024-04-26T23:43:20.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.273+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79d4044020 con 0x7f79e00ae8f0 2024-04-26T23:43:20.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.274+0000 7f79dcc00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 0x7f79c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.274+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f79d40ac330 con 0x7f79e00ae8f0 2024-04-26T23:43:20.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.275+0000 7f79dfe00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 0x7f79c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.276+0000 7f79dfe00640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 0x7f79c006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79e00c7380 tx=0x7f79cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.278+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79d404b070 con 0x7f79e00ae8f0 2024-04-26T23:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.484+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f79e0073d20 con 0x7f79e00ae8f0 2024-04-26T23:43:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.491+0000 7f79dcc00640 1 -- 172.21.15.46:0/4180525985 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v19)=0 v19) ==== 117+0+0 (secure 0 0 0) 0x7f79d407a480 con 0x7f79e00ae8f0 2024-04-26T23:43:20.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 msgr2=0x7f79c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 0x7f79c006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79e00c7380 tx=0x7f79cc008040 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 msgr2=0x7f79e00c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79d4000c00 tx=0x7f79d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 shutdown_connections 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79c006c8f0 0x7f79c006edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e015f210 0x7f79e01f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e00cd610 0x7f79e01f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 --2- 172.21.15.46:0/4180525985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e00ae8f0 0x7f79e00c6c40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 >> 172.21.15.46:0/4180525985 conn(0x7f79e00b27b0 msgr2=0x7f79e015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 shutdown_connections 2024-04-26T23:43:20.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.494+0000 7f79e6000640 1 -- 172.21.15.46:0/4180525985 wait complete. 2024-04-26T23:43:20.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:20.566 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:43:20.562+0000 7fe441400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:43:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.721+0000 7f6475e00640 1 Processor -- start 2024-04-26T23:43:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.721+0000 7f6475e00640 1 -- start start 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.722+0000 7f6475e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f647006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.722+0000 7f6475e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647006ee30 con 0x7f6470160bd0 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.722+0000 7f6475e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6470161940 con 0x7f64700732c0 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.722+0000 7f6475e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6470071f80 con 0x7f647006e510 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.723+0000 7f6467e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6470160bd0 0x7f6470161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38868/0 (socket says 172.21.15.46:38868) 2024-04-26T23:43:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.723+0000 7f6467e00640 1 -- 172.21.15.46:0/2383980263 learned_addr learned my addr 172.21.15.46:0/2383980263 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.723+0000 7f646ea00640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f647006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.723+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701469750 0 0) 0x7f647006ee30 con 0x7f6470160bd0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.723+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6454003520 con 0x7f6470160bd0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3780672982 0 0) 0x7f6454003520 con 0x7f6470160bd0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 msgr2=0x7f647006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f647006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 >> v1:172.21.15.142:6789/0 conn(0x7f64700732c0 legacy=0x7f6470073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6470072260 con 0x7f6470160bd0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6458001670 con 0x7f6470160bd0 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 >> v1:172.21.15.46:6789/0 conn(0x7f6470160bd0 legacy=0x7f6470161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f64540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454003f60 0x7f6454004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6454004d40 0x7f64540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6470071f80 con 0x7f6454004d40 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f645400baa0 con 0x7f6454003f60 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64540100b0 con 0x7f647006e510 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646f400640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6454004d40 0x7f64540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f646ea00640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f64540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.724+0000 7f6467e00640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454003f60 0x7f6454004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.725+0000 7f646f400640 1 -- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 msgr2=0x7f64540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.725+0000 7f646f400640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647006e510 0x7f64540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.725+0000 7f646f400640 1 -- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454003f60 msgr2=0x7f6454004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.725+0000 7f646f400640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454003f60 0x7f6454004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.725+0000 7f646f400640 1 -- 172.21.15.46:0/2383980263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6454003520 con 0x7f6454004d40 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.726+0000 7f646f400640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6454004d40 0x7f64540051c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6468001e50 tx=0x7f6454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938df13d497c972d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.726+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468001a70 con 0x7f6454004d40 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.726+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468003970 con 0x7f6454004d40 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.726+0000 7f646e000640 1 -- 172.21.15.46:0/2383980263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468003d60 con 0x7f6454004d40 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 -- 172.21.15.46:0/2383980263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6454004d40 msgr2=0x7f64540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 --2- 172.21.15.46:0/2383980263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6454004d40 0x7f64540051c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6468001e50 tx=0x7f6454005830 comp rx=0 tx=0).stop 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f646f400640 1 -- 172.21.15.46:0/2383980263 reap_dead start 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 -- 172.21.15.46:0/2383980263 shutdown_connections 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 -- 172.21.15.46:0/2383980263 >> 172.21.15.46:0/2383980263 conn(0x7f647006c530 msgr2=0x7f647006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 -- 172.21.15.46:0/2383980263 shutdown_connections 2024-04-26T23:43:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.727+0000 7f6475e00640 1 -- 172.21.15.46:0/2383980263 wait complete. 2024-04-26T23:43:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.729+0000 7f6475e00640 1 Processor -- start 2024-04-26T23:43:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.729+0000 7f6475e00640 1 -- start start 2024-04-26T23:43:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454004d40 0x7f647015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64701386f0 0x7f6470138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6470072d40 con 0x7f64701416c0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6470072820 con 0x7f6454004d40 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6475e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6470072ec0 con 0x7f64701386f0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40470/0 (socket says 172.21.15.46:40470) 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 -- 172.21.15.46:0/3095549994 learned_addr learned my addr 172.21.15.46:0/3095549994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f646ea00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454004d40 0x7f647015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f646f400640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64701386f0 0x7f6470138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 -- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64701386f0 msgr2=0x7f6470138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64701386f0 0x7f6470138b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 -- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454004d40 msgr2=0x7f647015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454004d40 0x7f647015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.730+0000 7f6467e00640 1 -- 172.21.15.46:0/3095549994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64700b20e0 con 0x7f64701416c0 2024-04-26T23:43:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6467e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f645800e870 tx=0x7f645800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800c8d0 con 0x7f64701416c0 2024-04-26T23:43:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458018070 con 0x7f64701416c0 2024-04-26T23:43:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800ccb0 con 0x7f64701416c0 2024-04-26T23:43:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64700b23c0 con 0x7f64701416c0 2024-04-26T23:43:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.731+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64700b2980 con 0x7f64701416c0 2024-04-26T23:43:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.733+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64700b2b80 con 0x7f64701416c0 2024-04-26T23:43:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.734+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f645801d460 con 0x7f64701416c0 2024-04-26T23:43:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.735+0000 7f6467400640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 0x7f644c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.735+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f645808bd30 con 0x7f64701416c0 2024-04-26T23:43:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.735+0000 7f646ea00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 0x7f644c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.736+0000 7f646ea00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 0x7f644c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6460009a10 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.739+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64700b2b80 con 0x7f64701416c0 2024-04-26T23:43:20.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.987+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6470069530 con 0x7f64701416c0 2024-04-26T23:43:20.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.989+0000 7f6467400640 1 -- 172.21.15.46:0/3095549994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f6458059f20 con 0x7f64701416c0 2024-04-26T23:43:20.992 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:20.992 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:20.992 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.991+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 msgr2=0x7f644c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.991+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 0x7f644c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6460009a10 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.991+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 msgr2=0x7f6470141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.991+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f645800e870 tx=0x7f645800ed30 comp rx=0 tx=0).stop 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 shutdown_connections 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644c06c8f0 0x7f644c06edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64701386f0 0x7f6470138b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64701416c0 0x7f6470141b40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 --2- 172.21.15.46:0/3095549994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6454004d40 0x7f647015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 >> 172.21.15.46:0/3095549994 conn(0x7f647006c530 msgr2=0x7f647006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 shutdown_connections 2024-04-26T23:43:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:20.992+0000 7f6475e00640 1 -- 172.21.15.46:0/3095549994 wait complete. 2024-04-26T23:43:21.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:43:21.288 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:43:10.815281+0000","btime":"2024-04-26T23:43:09.496295+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:43:10.815281+0000","atime":"2024-04-26T23:43:09.496295+0000","time_warp_seq":0,"change_attr":31,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:43:10.815281+0000","num_files":40,"num_subdirs":0,"change_attr":31},"rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"accounted_rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"version":244,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[{"frag":"*","children":1}]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{},"nref":0,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:43:21.289 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:43:09.496295+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:43:09.496295+0000", "change_attr": 31, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:43:10.815281+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [ { "children": 1, "frag": "*" } ] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 31, "mtime": "2024-04-26T23:43:10.815281+0000", "num_files": 40, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:43:10.815281+0000", "nestlock": {}, "nlink": 1, "nref": 0, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": {}, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 244, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:43:21.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:21.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start /testdir recursive 2024-04-26T23:43:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.502+0000 7f14d6400640 1 Processor -- start 2024-04-26T23:43:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.502+0000 7f14d6400640 1 -- start start 2024-04-26T23:43:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.504+0000 7f14d6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d00b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.504+0000 7f14d6400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14d0072650 0x7f14d006e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.504+0000 7f14d6400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d0146a20 con 0x7f14d0072ad0 2024-04-26T23:43:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.504+0000 7f14d6400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f14d0164a50 con 0x7f14d0072650 2024-04-26T23:43:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.504+0000 7f14d6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14d00b37e0 con 0x7f14d00b2ec0 2024-04-26T23:43:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14d0072650 0x7f14d006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cf400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d00b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14d0072650 0x7f14d006e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:53472/0 (socket says 172.21.15.46:53472) 2024-04-26T23:43:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 -- 172.21.15.46:0/3783997657 learned_addr learned my addr 172.21.15.46:0/3783997657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 -- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 msgr2=0x7f14d00b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 --2- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d00b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 -- 172.21.15.46:0/3783997657 >> v1:172.21.15.142:6789/0 conn(0x7f14d0072ad0 legacy=0x7f14d006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.505+0000 7f14cea00640 1 -- 172.21.15.46:0/3783997657 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d0163e40 con 0x7f14d0072650 2024-04-26T23:43:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.507+0000 7f14cea00640 1 --2- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d006e200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524a0d80ae5203a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14ce000640 1 -- 172.21.15.46:0/3783997657 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037070 con 0x7f14d0072650 2024-04-26T23:43:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14ce000640 1 -- 172.21.15.46:0/3783997657 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc032440 con 0x7f14d0072650 2024-04-26T23:43:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14ce000640 1 -- 172.21.15.46:0/3783997657 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc032810 con 0x7f14d0072650 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 -- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 msgr2=0x7f14d006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 --2- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d006e200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc02a250 comp rx=0 tx=0).stop 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 -- 172.21.15.46:0/3783997657 shutdown_connections 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 --2- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d006e200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 --2- 172.21.15.46:0/3783997657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d00b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.508+0000 7f14d6400640 1 -- 172.21.15.46:0/3783997657 >> 172.21.15.46:0/3783997657 conn(0x7f14d006c770 msgr2=0x7f14d006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.509+0000 7f14d6400640 1 -- 172.21.15.46:0/3783997657 shutdown_connections 2024-04-26T23:43:21.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.509+0000 7f14d6400640 1 -- 172.21.15.46:0/3783997657 wait complete. 2024-04-26T23:43:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.510+0000 7f14d6400640 1 Processor -- start 2024-04-26T23:43:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.510+0000 7f14d6400640 1 -- start start 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d0161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01646e0 con 0x7f14d0072ad0 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0164400 con 0x7f14d0072650 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.511+0000 7f14d6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14d0072170 con 0x7f14d00b2ec0 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40478/0 (socket says 172.21.15.46:40478) 2024-04-26T23:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 -- 172.21.15.46:0/3823196365 learned_addr learned my addr 172.21.15.46:0/3823196365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cf400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cfe00640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d0161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 -- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 msgr2=0x7f14d0161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d0161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 -- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 msgr2=0x7f14d015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cea00640 1 -- 172.21.15.46:0/3823196365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01fb6c0 con 0x7f14d0072ad0 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.512+0000 7f14cfe00640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d0161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14cea00640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009990 tx=0x7f14bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14c7400640 1 -- 172.21.15.46:0/3823196365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037040 con 0x7f14d0072ad0 2024-04-26T23:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14c7400640 1 -- 172.21.15.46:0/3823196365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc042070 con 0x7f14d0072ad0 2024-04-26T23:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14c7400640 1 -- 172.21.15.46:0/3823196365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc030940 con 0x7f14d0072ad0 2024-04-26T23:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01fb9a0 con 0x7f14d0072ad0 2024-04-26T23:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.513+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01fbf60 con 0x7f14d0072ad0 2024-04-26T23:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.515+0000 7f14c7400640 1 -- 172.21.15.46:0/3823196365 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14bc030ae0 con 0x7f14d0072ad0 2024-04-26T23:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.516+0000 7f14c7400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 0x7f14b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.517+0000 7f14c7400640 1 -- 172.21.15.46:0/3823196365 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f14bc0ac630 con 0x7f14d0072ad0 2024-04-26T23:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.517+0000 7f14cf400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 0x7f14b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.517+0000 7f14d937b740 1 Processor -- start 2024-04-26T23:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.517+0000 7f14d937b740 1 -- start start 2024-04-26T23:43:21.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.518+0000 7f14cf400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 0x7f14b006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ad0 tx=0x7f14c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.518+0000 7f14d937b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.518+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 0x55ecb71dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc0d0 0x55ecb71b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14d937b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecb7115db0 con 0x55ecb71f96f0 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ecb70b2000 con 0x55ecb71dc810 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ecb7111d70 con 0x55ecb71dc0d0 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40480/0 (socket says 172.21.15.46:40480) 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 -- 172.21.15.46:0/1129156554 learned_addr learned my addr 172.21.15.46:0/1129156554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cea00640 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 0x55ecb71dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 -- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc0d0 msgr2=0x55ecb71b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc0d0 0x55ecb71b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 -- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 msgr2=0x55ecb71dbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 0x55ecb71dbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cf400640 1 -- 172.21.15.46:0/1129156554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb6ef3a20 con 0x55ecb71f96f0 2024-04-26T23:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.519+0000 7f14cea00640 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 0x55ecb71dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14cf400640 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f14c0011e10 tx=0x7f14c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8678900a90c9c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14bbe00640 1 -- 172.21.15.46:0/1129156554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0011e40 con 0x55ecb71f96f0 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14bbe00640 1 -- 172.21.15.46:0/1129156554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0026070 con 0x55ecb71f96f0 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14bbe00640 1 -- 172.21.15.46:0/1129156554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0021660 con 0x55ecb71f96f0 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14d937b740 1 -- 172.21.15.46:0/1129156554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 msgr2=0x55ecb71ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.520+0000 7f14d937b740 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f14c0011e10 tx=0x7f14c0014000 comp rx=0 tx=0).stop 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 -- 172.21.15.46:0/1129156554 shutdown_connections 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc0d0 0x55ecb71b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71dc810 0x55ecb71dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 --2- 172.21.15.46:0/1129156554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71ac310 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 -- 172.21.15.46:0/1129156554 >> 172.21.15.46:0/1129156554 conn(0x55ecb71ae580 msgr2=0x55ecb71f6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 -- 172.21.15.46:0/1129156554 shutdown_connections 2024-04-26T23:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 -- 172.21.15.46:0/1129156554 wait complete. 2024-04-26T23:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.521+0000 7f14d937b740 1 -- start start 2024-04-26T23:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.522+0000 7f14d937b740 1 Processor -- start 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 0x55ecb71f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb71f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecb71b04a0 con 0x55ecb71f96f0 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ecb709cc50 con 0x55ecb71ee210 2024-04-26T23:43:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ecb71af890 con 0x55ecb71dc810 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14d937b740 10 client.0 _ll_register_callbacks cb 0x55ecb710cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40488/0 (socket says 172.21.15.46:40488) 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 -- 172.21.15.46:0/2706197696 learned_addr learned my addr 172.21.15.46:0/2706197696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cf400640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 0x55ecb71f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cfe00640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb71f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 -- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 msgr2=0x55ecb71f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 0x55ecb71f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 -- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 msgr2=0x55ecb71f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb71f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cea00640 1 -- 172.21.15.46:0/2706197696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb6de7310 con 0x55ecb71f96f0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cf400640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 0x55ecb71f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.523+0000 7f14cfe00640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb71f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14cea00640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3c3947d0b8d4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc07e430 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f14bc078380 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0026070 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f14bc082c60 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14d937b740 10 client.6942 fetch_fsmap learned FSMap version 50 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55ecb6f541b0 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0026070 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f14bc0aace0 con 0x55ecb71f96f0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.524+0000 7f14d937b740 10 client.6942 fetch_fsmap finished waiting for FSMap version 50 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14d937b740 10 client.6942 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14d937b740 20 client.6942 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71ddfd0 0x55ecb71de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14d937b740 4 client.6942 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 --> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb71239c0 con 0x55ecb71ddfd0 2024-04-26T23:43:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14cf400640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71ddfd0 0x55ecb71de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14cf400640 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71ddfd0 0x55ecb71de3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b852b57b9b804a server_cookie=de04241bcbb03851 in_seq=0 out_seq=0 2024-04-26T23:43:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.525+0000 7f14b8c00640 10 client.6942 ms_handle_connect on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.526+0000 7f14b8c00640 1 -- 172.21.15.46:0/2706197696 <== mds.0 v2:172.21.15.46:6833/2393654178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecb71239c0 con 0x55ecb71ddfd0 2024-04-26T23:43:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.526+0000 7f14b8c00640 10 client.6942 handle_command_reply: tid=0 2024-04-26T23:43:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.526+0000 7f14d937b740 1 client.6942 shutdown 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.526+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 msgr2=0x55ecb71cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.526+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc032500 comp rx=0 tx=0).stop 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 shutdown_connections 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71ddfd0 0x55ecb71de3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb71f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71cc9d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 --2- 172.21.15.46:0/2706197696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71dc810 0x55ecb71f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14b8c00640 0 client.6942 ms_handle_reset on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 >> 172.21.15.46:0/2706197696 conn(0x55ecb71dc0d0 msgr2=0x55ecb71ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 shutdown_connections 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 1 -- 172.21.15.46:0/2706197696 wait complete. 2024-04-26T23:43:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.527+0000 7f14d937b740 20 client.6942 trim_cache size 0 max 16384 2024-04-26T23:43:21.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.547+0000 7f14d937b740 1 Processor -- start 2024-04-26T23:43:21.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.547+0000 7f14d937b740 1 -- start start 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.547+0000 7f14d937b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb7209b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f96f0 0x55ecb71ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14d937b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0206380 con 0x55ecb71dc810 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d02053a0 con 0x55ecb71ee210 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14d0204730 con 0x55ecb71f96f0 2024-04-26T23:43:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40492/0 (socket says 172.21.15.46:40492) 2024-04-26T23:43:21.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 -- 172.21.15.46:0/2119813964 learned_addr learned my addr 172.21.15.46:0/2119813964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 -- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f96f0 msgr2=0x55ecb71ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f96f0 0x55ecb71ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 -- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 msgr2=0x55ecb7209b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb7209b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.548+0000 7f14cf400640 1 -- 172.21.15.46:0/2119813964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb6dc0490 con 0x55ecb71dc810 2024-04-26T23:43:21.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14cf400640 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f14c0023040 tx=0x7f14c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d210fac16c755d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14bb400640 1 -- 172.21.15.46:0/2119813964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c003b070 con 0x55ecb71dc810 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14bb400640 1 -- 172.21.15.46:0/2119813964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0026040 con 0x55ecb71dc810 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14bb400640 1 -- 172.21.15.46:0/2119813964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0036dd0 con 0x55ecb71dc810 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 -- 172.21.15.46:0/2119813964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 msgr2=0x55ecb71df670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f14c0023040 tx=0x7f14c0016000 comp rx=0 tx=0).stop 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 -- 172.21.15.46:0/2119813964 shutdown_connections 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f96f0 0x55ecb71ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb7209b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 --2- 172.21.15.46:0/2119813964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71dc810 0x55ecb71df670 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.549+0000 7f14d937b740 1 -- 172.21.15.46:0/2119813964 >> 172.21.15.46:0/2119813964 conn(0x55ecb71dc0d0 msgr2=0x55ecb721a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.550+0000 7f14d937b740 1 -- 172.21.15.46:0/2119813964 shutdown_connections 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.550+0000 7f14d937b740 1 -- 172.21.15.46:0/2119813964 wait complete. 2024-04-26T23:43:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.550+0000 7f14d937b740 1 -- start start 2024-04-26T23:43:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.550+0000 7f14d937b740 1 Processor -- start 2024-04-26T23:43:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb72568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 0x55ecb71de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecb71ddd10 con 0x55ecb71f96f0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0204d50 con 0x55ecb71ee210 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.551+0000 7f14d937b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ecb71ac620 con 0x55ecb71f7c50 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14d937b740 10 client.0 _ll_register_callbacks cb 0x7f14bc0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40494/0 (socket says 172.21.15.46:40494) 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 -- 172.21.15.46:0/1554485852 learned_addr learned my addr 172.21.15.46:0/1554485852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cf400640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb72568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cfe00640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 0x55ecb71de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 -- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 msgr2=0x55ecb71de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 0x55ecb71de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 -- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 msgr2=0x55ecb72568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb72568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cea00640 1 -- 172.21.15.46:0/1554485852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb71f5da0 con 0x55ecb71f96f0 2024-04-26T23:43:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cf400640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb72568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.552+0000 7f14cfe00640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 0x55ecb71de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14cea00640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0785d0 tx=0x7f14bc032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efefae1f017935e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc097030 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f14bc078380 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0026040 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f14bc0a0050 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 10 client.6948 fetch_fsmap learned FSMap version 50 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55ecb71f55a0 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0026040 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f14bc0a3020 con 0x55ecb71f96f0 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 10 client.6948 fetch_fsmap finished waiting for FSMap version 50 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 10 client.6948 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 20 client.6948 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71dcd00 0x55ecb71dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 4 client.6948 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/testdir", "scrubops": ["recursive"]}] 2024-04-26T23:43:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.553+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 --> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] -- command(tid 0: {"prefix": "scrub start", "path": "/testdir", "scrubops": ["recursive"]}) -- 0x55ecb711d830 con 0x55ecb71dcd00 2024-04-26T23:43:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.554+0000 7f14cf400640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71dcd00 0x55ecb71dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.554+0000 7f14cf400640 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71dcd00 0x55ecb71dd110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2ffc25d3e912cc server_cookie=55217cc9be8ace50 in_seq=0 out_seq=0 2024-04-26T23:43:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.554+0000 7f14b8c00640 10 client.6948 ms_handle_connect on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14b8c00640 1 -- 172.21.15.46:0/1554485852 <== mds.0 v2:172.21.15.46:6833/2393654178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecb711d830 con 0x55ecb71dcd00 2024-04-26T23:43:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14b8c00640 10 client.6948 handle_command_reply: tid=0 2024-04-26T23:43:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 client.6948 shutdown 2024-04-26T23:43:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 msgr2=0x55ecb71de3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0785d0 tx=0x7f14bc032f90 comp rx=0 tx=0).stop 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 shutdown_connections 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ecb71f7c50 0x55ecb71de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb71f96f0 0x55ecb71de3b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.46:6833/2393654178,v1:172.21.15.46:6835/2393654178] conn(0x55ecb71dcd00 0x55ecb71dd110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 --2- 172.21.15.46:0/1554485852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ecb71ee210 0x55ecb72568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 >> 172.21.15.46:0/1554485852 conn(0x55ecb71dc0d0 msgr2=0x55ecb72097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.555+0000 7f14b8c00640 0 client.6948 ms_handle_reset on v2:172.21.15.46:6833/2393654178 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 shutdown_connections 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d937b740 1 -- 172.21.15.46:0/1554485852 wait complete. 2024-04-26T23:43:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d937b740 20 client.6948 trim_cache size 0 max 16384 2024-04-26T23:43:21.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:21.563 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:43:21.563 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "34cacdb2-1615-409d-ab43-9177bc072840", 2024-04-26T23:43:21.563 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:43:21.563 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 msgr2=0x7f14b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 0x7f14b006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ad0 tx=0x7f14c0008040 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 msgr2=0x7f14d0162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.556+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009990 tx=0x7f14bc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 shutdown_connections 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f14b006c8f0 0x7f14b006edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d00b2ec0 0x7f14d0161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d0072ad0 0x7f14d0162a10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 --2- 172.21.15.46:0/3823196365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0072650 0x7f14d015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 >> 172.21.15.46:0/3823196365 conn(0x7f14d006c770 msgr2=0x7f14d00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 shutdown_connections 2024-04-26T23:43:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.557+0000 7f14d6400640 1 -- 172.21.15.46:0/3823196365 wait complete. 2024-04-26T23:43:21.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.766+0000 7f5d44c00640 1 Processor -- start 2024-04-26T23:43:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.766+0000 7f5d44c00640 1 -- start start 2024-04-26T23:43:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d44c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 0x7f5d4006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d44c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d400732c0 0x7f5d40073710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d44c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4006ee30 con 0x7f5d40160bd0 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d44c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5d40161940 con 0x7f5d400732c0 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d44c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5d40071f80 con 0x7f5d4006e510 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d3e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d400732c0 0x7f5d40073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d3e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d400732c0 0x7f5d40073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40504/0 (socket says 172.21.15.46:40504) 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d3e800640 1 -- 172.21.15.46:0/1383736243 learned_addr learned my addr 172.21.15.46:0/1383736243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d3d400640 1 --1- 172.21.15.46:0/1383736243 >> v1:172.21.15.46:6789/0 conn(0x7f5d40160bd0 0x7f5d40161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38882/0 (socket says 172.21.15.46:38882) 2024-04-26T23:43:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.768+0000 7f5d3de00640 1 --2- 172.21.15.46:0/1383736243 >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 0x7f5d4006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.769+0000 7f5d3e800640 1 -- 172.21.15.46:0/1383736243 >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 msgr2=0x7f5d4006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.769+0000 7f5d3e800640 1 --2- 172.21.15.46:0/1383736243 >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 0x7f5d4006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.769+0000 7f5d3e800640 1 -- 172.21.15.46:0/1383736243 >> v1:172.21.15.46:6789/0 conn(0x7f5d40160bd0 legacy=0x7f5d40161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.769+0000 7f5d3e800640 1 -- 172.21.15.46:0/1383736243 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d40072260 con 0x7f5d400732c0 2024-04-26T23:43:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.769+0000 7f5d3de00640 1 --2- 172.21.15.46:0/1383736243 >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 0x7f5d4006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.770+0000 7f5d3e800640 1 --2- 172.21.15.46:0/1383736243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d400732c0 0x7f5d40073710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009e50 tx=0x7f5d3802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a624d1cade1e72a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.770+0000 7f5d3ca00640 1 -- 172.21.15.46:0/1383736243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38036070 con 0x7f5d400732c0 2024-04-26T23:43:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.770+0000 7f5d3ca00640 1 -- 172.21.15.46:0/1383736243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38031600 con 0x7f5d400732c0 2024-04-26T23:43:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.770+0000 7f5d3ca00640 1 -- 172.21.15.46:0/1383736243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380319d0 con 0x7f5d400732c0 2024-04-26T23:43:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 -- 172.21.15.46:0/1383736243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d400732c0 msgr2=0x7f5d40073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 --2- 172.21.15.46:0/1383736243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d400732c0 0x7f5d40073710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009e50 tx=0x7f5d3802a760 comp rx=0 tx=0).stop 2024-04-26T23:43:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 -- 172.21.15.46:0/1383736243 shutdown_connections 2024-04-26T23:43:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 --2- 172.21.15.46:0/1383736243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d400732c0 0x7f5d40073710 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 --2- 172.21.15.46:0/1383736243 >> v2:172.21.15.142:3300/0 conn(0x7f5d4006e510 0x7f5d4006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 -- 172.21.15.46:0/1383736243 >> 172.21.15.46:0/1383736243 conn(0x7f5d4006c530 msgr2=0x7f5d4006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 -- 172.21.15.46:0/1383736243 shutdown_connections 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.771+0000 7f5d44c00640 1 -- 172.21.15.46:0/1383736243 wait complete. 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.773+0000 7f5d44c00640 1 Processor -- start 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.773+0000 7f5d44c00640 1 -- start start 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d4006e510 0x7f5d401536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d400732c0 0x7f5d401397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40072d40 con 0x7f5d40160bd0 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40072820 con 0x7f5d400732c0 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d44c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5d40072ec0 con 0x7f5d4006e510 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40520/0 (socket says 172.21.15.46:40520) 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.774+0000 7f5d3e800640 1 -- 172.21.15.46:0/362724097 learned_addr learned my addr 172.21.15.46:0/362724097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 -- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d4006e510 msgr2=0x7f5d401536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d4006e510 0x7f5d401536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 -- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d400732c0 msgr2=0x7f5d401397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3d400640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d400732c0 0x7f5d401397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d400732c0 0x7f5d401397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 -- 172.21.15.46:0/362724097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d400b2630 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d3e800640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5d38034040 tx=0x7f5d38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38036050 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38042070 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38009510 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.775+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d400b2910 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.776+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d400b2ed0 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.777+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d40069530 con 0x7f5d40160bd0 2024-04-26T23:43:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.778+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d38007bd0 con 0x7f5d40160bd0 2024-04-26T23:43:21.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.779+0000 7f5d36000640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.780+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f5d380acbc0 con 0x7f5d40160bd0 2024-04-26T23:43:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.780+0000 7f5d3de00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.781+0000 7f5d3de00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000bbe0 tx=0x7f5d30006d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:21.784+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d38045360 con 0x7f5d40160bd0 2024-04-26T23:43:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.018+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d40069ac0 con 0x7f5d40160bd0 2024-04-26T23:43:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.019+0000 7f5d36000640 1 -- 172.21.15.46:0/362724097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f5d3807ae40 con 0x7f5d40160bd0 2024-04-26T23:43:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:22.024 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 msgr2=0x7f5d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000bbe0 tx=0x7f5d30006d50 comp rx=0 tx=0).stop 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 msgr2=0x7f5d400b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5d38034040 tx=0x7f5d38004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 shutdown_connections 2024-04-26T23:43:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d40160bd0 0x7f5d400b1de0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d400732c0 0x7f5d401397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 --2- 172.21.15.46:0/362724097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d4006e510 0x7f5d401536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 >> 172.21.15.46:0/362724097 conn(0x7f5d4006c530 msgr2=0x7f5d4015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.021+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 shutdown_connections 2024-04-26T23:43:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:22.022+0000 7f5d44c00640 1 -- 172.21.15.46:0/362724097 wait complete. 2024-04-26T23:43:22.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:43:22.259 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:43:10.815281+0000","btime":"2024-04-26T23:43:09.496295+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:43:10.815281+0000","atime":"2024-04-26T23:43:09.496295+0000","time_warp_seq":0,"change_attr":31,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:43:10.815281+0000","num_files":40,"num_subdirs":0,"change_attr":31},"rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"accounted_rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"version":244,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[{"frag":"*","children":1}]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":2,"authpin":0,"scrubqueue":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:43:22.260 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:43:09.496295+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:43:09.496295+0000", "change_attr": 31, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:43:10.815281+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [ { "children": 1, "frag": "*" } ] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 31, "mtime": "2024-04-26T23:43:10.815281+0000", "num_files": 40, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:43:10.815281+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "authpin": 0, "dirfrag": 2, "scrubqueue": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 244, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:43:22.260 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-04-26T23:43:27.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:27.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.483+0000 7f9247e00640 1 Processor -- start 2024-04-26T23:43:27.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.484+0000 7f9247e00640 1 -- start start 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.485+0000 7f9247e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480d2130 0x7f92480d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.485+0000 7f9247e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92480b4000 0x7f92480b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.485+0000 7f9247e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92480d11f0 con 0x7f924815f4e0 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.485+0000 7f9247e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f92480d0ce0 con 0x7f92480b4000 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.485+0000 7f9247e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92480d2990 con 0x7f92480d2130 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9246a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f924815f4e0 0x7f92480d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60532/0 (socket says 172.21.15.46:60532) 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9246a00640 1 -- 172.21.15.46:0/3337740693 learned_addr learned my addr 172.21.15.46:0/3337740693 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:27.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9247400640 1 --2- 172.21.15.46:0/3337740693 >> v2:172.21.15.46:3300/0 conn(0x7f92480b4000 0x7f92480b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9246000640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480d2130 0x7f92480d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9245600640 1 -- 172.21.15.46:0/3337740693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091601079 0 0) 0x7f92480d11f0 con 0x7f924815f4e0 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9247400640 1 -- 172.21.15.46:0/3337740693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480d2130 msgr2=0x7f92480d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9247400640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480d2130 0x7f92480d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9247400640 1 -- 172.21.15.46:0/3337740693 >> v1:172.21.15.46:6789/0 conn(0x7f924815f4e0 legacy=0x7f92480d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.486+0000 7f9247400640 1 -- 172.21.15.46:0/3337740693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480d5f20 con 0x7f92480b4000 2024-04-26T23:43:27.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.488+0000 7f9247400640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92480b4000 0x7f92480b2280 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9240009d90 tx=0x7f924002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e3237204cd5518 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:27.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.488+0000 7f9245600640 1 -- 172.21.15.46:0/3337740693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007740 con 0x7f92480b4000 2024-04-26T23:43:27.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.488+0000 7f9245600640 1 -- 172.21.15.46:0/3337740693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92400078e0 con 0x7f92480b4000 2024-04-26T23:43:27.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.488+0000 7f9245600640 1 -- 172.21.15.46:0/3337740693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007cb0 con 0x7f92480b4000 2024-04-26T23:43:27.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 -- 172.21.15.46:0/3337740693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92480b4000 msgr2=0x7f92480b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92480b4000 0x7f92480b2280 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9240009d90 tx=0x7f924002a660 comp rx=0 tx=0).stop 2024-04-26T23:43:27.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 -- 172.21.15.46:0/3337740693 shutdown_connections 2024-04-26T23:43:27.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92480b4000 0x7f92480b2280 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 --2- 172.21.15.46:0/3337740693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480d2130 0x7f92480d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 -- 172.21.15.46:0/3337740693 >> 172.21.15.46:0/3337740693 conn(0x7f92480d5010 msgr2=0x7f92480d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:27.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 -- 172.21.15.46:0/3337740693 shutdown_connections 2024-04-26T23:43:27.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.489+0000 7f9247e00640 1 -- 172.21.15.46:0/3337740693 wait complete. 2024-04-26T23:43:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.491+0000 7f9247e00640 1 Processor -- start 2024-04-26T23:43:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.491+0000 7f9247e00640 1 -- start start 2024-04-26T23:43:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480b4000 0x7f92480cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92480d2130 0x7f924806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92480b1d60 con 0x7f924815f4e0 2024-04-26T23:43:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92480d5c40 con 0x7f92480d2130 2024-04-26T23:43:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f924815f8c0 con 0x7f92480b4000 2024-04-26T23:43:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45768/0 (socket says 172.21.15.46:45768) 2024-04-26T23:43:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.492+0000 7f9247400640 1 -- 172.21.15.46:0/3014334009 learned_addr learned my addr 172.21.15.46:0/3014334009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9246000640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92480d2130 0x7f924806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9246a00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480b4000 0x7f92480cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 -- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480b4000 msgr2=0x7f92480cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480b4000 0x7f92480cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 -- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92480d2130 msgr2=0x7f924806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92480d2130 0x7f924806deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 -- 172.21.15.46:0/3014334009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480b97e0 con 0x7f924815f4e0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.493+0000 7f9247400640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f924002ac50 tx=0x7f9240004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.494+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240040070 con 0x7f924815f4e0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.494+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9240031b40 con 0x7f924815f4e0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.494+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240045460 con 0x7f924815f4e0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.494+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92480b9ac0 con 0x7f924815f4e0 2024-04-26T23:43:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.494+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92480cc620 con 0x7f924815f4e0 2024-04-26T23:43:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.495+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92480693b0 con 0x7f924815f4e0 2024-04-26T23:43:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.496+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9240039040 con 0x7f924815f4e0 2024-04-26T23:43:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.497+0000 7f923ea00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.498+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f92400ad860 con 0x7f924815f4e0 2024-04-26T23:43:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.498+0000 7f9246a00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.499+0000 7f9246a00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 0x7f922406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9238000f60 tx=0x7f9238007d00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.502+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f924007bad0 con 0x7f924815f4e0 2024-04-26T23:43:27.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.750+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92480d46d0 con 0x7f924815f4e0 2024-04-26T23:43:27.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.752+0000 7f923ea00640 1 -- 172.21.15.46:0/3014334009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f9240043e40 con 0x7f924815f4e0 2024-04-26T23:43:27.755 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:27.755 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:27.756 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 msgr2=0x7f922406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 0x7f922406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9238000f60 tx=0x7f9238007d00 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 msgr2=0x7f924814dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f924002ac50 tx=0x7f9240004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 shutdown_connections 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924815f4e0 0x7f924814dc50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92480d2130 0x7f924806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 --2- 172.21.15.46:0/3014334009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92480b4000 0x7f92480cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 >> 172.21.15.46:0/3014334009 conn(0x7f92480d5010 msgr2=0x7f92480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:27.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 shutdown_connections 2024-04-26T23:43:27.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:27.755+0000 7f9247e00640 1 -- 172.21.15.46:0/3014334009 wait complete. 2024-04-26T23:43:27.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:43:27.997 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:43:10.815281+0000","btime":"2024-04-26T23:43:09.496295+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:43:10.815281+0000","atime":"2024-04-26T23:43:09.496295+0000","time_warp_seq":0,"change_attr":31,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:43:10.815281+0000","num_files":40,"num_subdirs":0,"change_attr":31},"rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"accounted_rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:43:10.819281+0000"},"version":247,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"2024-04-26T23:43:21.558551+0000","last_scrub_version":244,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"lock":0,"dirfrag":1,"dirtyrstat":1,"dirty":1,"authpin":0,"scrubqueue":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyrstat"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:43:27.997 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:43:09.496295+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:43:09.496295+0000", "change_attr": 31, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:43:10.815281+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 31, "mtime": "2024-04-26T23:43:10.815281+0000", "num_files": 40, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "2024-04-26T23:43:21.558551+0000", "last_scrub_version": 244, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:43:10.815281+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "authpin": 0, "dirfrag": 1, "dirty": 1, "dirtyrstat": 1, "lock": 0, "scrubqueue": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 70, "rctime": "2024-04-26T23:43:10.819281+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyrstat" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 247, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:43:27.998 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T23:43:27.998 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:43:27.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:43:28.094 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:43:28.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:43:28.206 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:43:28.206 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:28.206 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:28.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:43:28.268 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:43:28.268 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:43:28.268 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:28.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:43:28.325 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:43:28.325 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:28.325 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:28.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:43:28.385 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:43:28.385 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:43:28.386 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:28.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:43:28.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.653+0000 7fcc07e00640 1 Processor -- start 2024-04-26T23:43:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.654+0000 7fcc07e00640 1 -- start start 2024-04-26T23:43:28.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.655+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.655+0000 7fcc07e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc0015c5c0 0x7fcc0015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.655+0000 7fcc07e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc000cf950 con 0x7fcc00145770 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.655+0000 7fcc07e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcc000d0c50 con 0x7fcc0015c5c0 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.655+0000 7fcc07e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0015cf90 con 0x7fcc00073570 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.656+0000 7fcc04a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.656+0000 7fcc05e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc0015c5c0 0x7fcc0015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.656+0000 7fcc04a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60214/0 (socket says 172.21.15.46:60214) 2024-04-26T23:43:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.656+0000 7fcc04a00640 1 -- 172.21.15.46:0/917379548 learned_addr learned my addr 172.21.15.46:0/917379548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.657+0000 7fcc04a00640 1 -- 172.21.15.46:0/917379548 >> v2:172.21.15.142:3300/0 conn(0x7fcc0015c5c0 msgr2=0x7fcc0015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.657+0000 7fcc04a00640 1 --2- 172.21.15.46:0/917379548 >> v2:172.21.15.142:3300/0 conn(0x7fcc0015c5c0 0x7fcc0015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.657+0000 7fcc04a00640 1 -- 172.21.15.46:0/917379548 >> v1:172.21.15.142:6789/0 conn(0x7fcc00145770 legacy=0x7fcc00072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.657+0000 7fcc04a00640 1 -- 172.21.15.46:0/917379548 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc000cfc30 con 0x7fcc00073570 2024-04-26T23:43:28.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.658+0000 7fcc04a00640 1 --2- 172.21.15.46:0/917379548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009870 tx=0x7fcbec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde8833c2061591a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.659+0000 7fcbffe00640 1 -- 172.21.15.46:0/917379548 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec037070 con 0x7fcc00073570 2024-04-26T23:43:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.659+0000 7fcbffe00640 1 -- 172.21.15.46:0/917379548 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec032440 con 0x7fcc00073570 2024-04-26T23:43:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.659+0000 7fcbffe00640 1 -- 172.21.15.46:0/917379548 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec032810 con 0x7fcc00073570 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 -- 172.21.15.46:0/917379548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 msgr2=0x7fcc000d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 --2- 172.21.15.46:0/917379548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009870 tx=0x7fcbec02a270 comp rx=0 tx=0).stop 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 -- 172.21.15.46:0/917379548 shutdown_connections 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 --2- 172.21.15.46:0/917379548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc000d0710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 --2- 172.21.15.46:0/917379548 >> v2:172.21.15.142:3300/0 conn(0x7fcc0015c5c0 0x7fcc0015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 -- 172.21.15.46:0/917379548 >> 172.21.15.46:0/917379548 conn(0x7fcc0006c530 msgr2=0x7fcc0006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 -- 172.21.15.46:0/917379548 shutdown_connections 2024-04-26T23:43:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.660+0000 7fcc07e00640 1 -- 172.21.15.46:0/917379548 wait complete. 2024-04-26T23:43:28.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.662+0000 7fcc07e00640 1 Processor -- start 2024-04-26T23:43:28.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.663+0000 7fcc07e00640 1 -- start start 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc00144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc00145770 0x7fcc000aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc000d01f0 con 0x7fcc0015c5c0 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0015d490 con 0x7fcc00145770 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc07e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc000cf5b0 con 0x7fcc00073570 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45784/0 (socket says 172.21.15.46:45784) 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc05e00640 1 -- 172.21.15.46:0/2547062985 learned_addr learned my addr 172.21.15.46:0/2547062985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.664+0000 7fcc05400640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc00144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc04a00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc00145770 0x7fcc000aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 -- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 msgr2=0x7fcc00144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc00144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 -- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc00145770 msgr2=0x7fcc000aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc00145770 0x7fcc000aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 -- 172.21.15.46:0/2547062985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc000ab420 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcc05e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e820 tx=0x7fcbf800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.665+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8017070 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.666+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf800ee70 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.666+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc000ab6c0 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.666+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800cb60 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.666+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc00141c60 con 0x7fcc0015c5c0 2024-04-26T23:43:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.667+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc00069530 con 0x7fcc0015c5c0 2024-04-26T23:43:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.668+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbf801a070 con 0x7fcc0015c5c0 2024-04-26T23:43:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.669+0000 7fcbfd600640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 0x7fcbe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.669+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fcbf808c120 con 0x7fcc0015c5c0 2024-04-26T23:43:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.669+0000 7fcc05400640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 0x7fcbe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:28.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.671+0000 7fcc05400640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 0x7fcbe006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0001d40 tx=0x7fcbf0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.673+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbf805a1f0 con 0x7fcc0015c5c0 2024-04-26T23:43:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.940+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc000b3c10 con 0x7fcc0015c5c0 2024-04-26T23:43:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.941+0000 7fcbfd600640 1 -- 172.21.15.46:0/2547062985 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7fcbf8059af0 con 0x7fcc0015c5c0 2024-04-26T23:43:28.945 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:28.945 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 msgr2=0x7fcbe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 0x7fcbe006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0001d40 tx=0x7fcbf0009040 comp rx=0 tx=0).stop 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 msgr2=0x7fcc000abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e820 tx=0x7fcbf800ec20 comp rx=0 tx=0).stop 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 shutdown_connections 2024-04-26T23:43:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe006c8f0 0x7fcbe006edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0015c5c0 0x7fcc000abff0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc00145770 0x7fcc000aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 --2- 172.21.15.46:0/2547062985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc00073570 0x7fcc00144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 >> 172.21.15.46:0/2547062985 conn(0x7fcc0006c530 msgr2=0x7fcc000b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 shutdown_connections 2024-04-26T23:43:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:28.945+0000 7fcc07e00640 1 -- 172.21.15.46:0/2547062985 wait complete. 2024-04-26T23:43:28.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.174+0000 7f3fcd400640 1 Processor -- start 2024-04-26T23:43:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.174+0000 7f3fcd400640 1 -- start start 2024-04-26T23:43:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.176+0000 7f3fcd400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3fc806e510 0x7f3fc806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.176+0000 7f3fcd400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.176+0000 7f3fcd400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc806ee30 con 0x7f3fc8160bd0 2024-04-26T23:43:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.176+0000 7f3fcd400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc8161940 con 0x7f3fc806e510 2024-04-26T23:43:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.176+0000 7f3fcd400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc8071f80 con 0x7f3fc80732c0 2024-04-26T23:43:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.177+0000 7f3fc7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.177+0000 7f3fc7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60224/0 (socket says 172.21.15.46:60224) 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.177+0000 7f3fc7200640 1 -- 172.21.15.46:0/1278382409 learned_addr learned my addr 172.21.15.46:0/1278382409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.177+0000 7f3fc6800640 1 --2- 172.21.15.46:0/1278382409 >> v2:172.21.15.142:3300/0 conn(0x7f3fc806e510 0x7f3fc806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.178+0000 7f3fc7200640 1 -- 172.21.15.46:0/1278382409 >> v2:172.21.15.142:3300/0 conn(0x7f3fc806e510 msgr2=0x7f3fc806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.178+0000 7f3fc7200640 1 --2- 172.21.15.46:0/1278382409 >> v2:172.21.15.142:3300/0 conn(0x7f3fc806e510 0x7f3fc806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.178+0000 7f3fc7200640 1 -- 172.21.15.46:0/1278382409 >> v1:172.21.15.142:6789/0 conn(0x7f3fc8160bd0 legacy=0x7f3fc8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.178+0000 7f3fc7200640 1 -- 172.21.15.46:0/1278382409 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc8072260 con 0x7f3fc80732c0 2024-04-26T23:43:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.179+0000 7f3fc7200640 1 --2- 172.21.15.46:0/1278382409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009e80 tx=0x7f3fc002a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245452a3a6aa3653 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.180+0000 7f3fc5400640 1 -- 172.21.15.46:0/1278382409 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0036070 con 0x7f3fc80732c0 2024-04-26T23:43:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.180+0000 7f3fc5400640 1 -- 172.21.15.46:0/1278382409 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc0031600 con 0x7f3fc80732c0 2024-04-26T23:43:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.180+0000 7f3fc5400640 1 -- 172.21.15.46:0/1278382409 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00319d0 con 0x7f3fc80732c0 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.180+0000 7f3fcd400640 1 -- 172.21.15.46:0/1278382409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 msgr2=0x7f3fc8073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.180+0000 7f3fcd400640 1 --2- 172.21.15.46:0/1278382409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009e80 tx=0x7f3fc002a750 comp rx=0 tx=0).stop 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 -- 172.21.15.46:0/1278382409 shutdown_connections 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 --2- 172.21.15.46:0/1278382409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc8073750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 --2- 172.21.15.46:0/1278382409 >> v2:172.21.15.142:3300/0 conn(0x7f3fc806e510 0x7f3fc806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 -- 172.21.15.46:0/1278382409 >> 172.21.15.46:0/1278382409 conn(0x7f3fc806c530 msgr2=0x7f3fc806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 -- 172.21.15.46:0/1278382409 shutdown_connections 2024-04-26T23:43:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.181+0000 7f3fcd400640 1 -- 172.21.15.46:0/1278382409 wait complete. 2024-04-26T23:43:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.183+0000 7f3fcd400640 1 Processor -- start 2024-04-26T23:43:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.183+0000 7f3fcd400640 1 -- start start 2024-04-26T23:43:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc8160bd0 0x7f3fc8145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8072d40 con 0x7f3fc806e510 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8072820 con 0x7f3fc8160bd0 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fcd400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc8072ec0 con 0x7f3fc80732c0 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45786/0 (socket says 172.21.15.46:45786) 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fc6800640 1 -- 172.21.15.46:0/516383599 learned_addr learned my addr 172.21.15.46:0/516383599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fc7200640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc8160bd0 0x7f3fc8145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.184+0000 7f3fc5e00640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 -- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 msgr2=0x7f3fc81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc81571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 -- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc8160bd0 msgr2=0x7f3fc8145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc8160bd0 0x7f3fc8145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 -- 172.21.15.46:0/516383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc81346e0 con 0x7f3fc806e510 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc5e00640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fc6800640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e870 tx=0x7f3fb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8017070 con 0x7f3fc806e510 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb800c940 con 0x7f3fc806e510 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800cd30 con 0x7f3fc806e510 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.185+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc81349c0 con 0x7f3fc806e510 2024-04-26T23:43:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.186+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc8150ff0 con 0x7f3fc806e510 2024-04-26T23:43:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.187+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc8069530 con 0x7f3fc806e510 2024-04-26T23:43:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.188+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fb801a070 con 0x7f3fc806e510 2024-04-26T23:43:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.189+0000 7f3fbea00640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 0x7f3fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.189+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f3fb808bff0 con 0x7f3fc806e510 2024-04-26T23:43:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.190+0000 7f3fc5e00640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 0x7f3fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.191+0000 7f3fc5e00640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 0x7f3fa806edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fc81f88a0 tx=0x7f3fb4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.194+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fb805a1d0 con 0x7f3fc806e510 2024-04-26T23:43:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.402+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fc806fb00 con 0x7f3fc806e510 2024-04-26T23:43:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.405+0000 7f3fbea00640 1 -- 172.21.15.46:0/516383599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17109 (secure 0 0 0) 0x7f3fb8059ad0 con 0x7f3fc806e510 2024-04-26T23:43:29.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:29.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:14.531022+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.46:6834/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.142:6835/2927770154":"2024-04-27T23:43:14.513761+0000","172.21.15.142:6834/2927770154":"2024-04-27T23:43:14.513761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.407+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 msgr2=0x7f3fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.407+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 0x7f3fa806edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fc81f88a0 tx=0x7f3fb4009040 comp rx=0 tx=0).stop 2024-04-26T23:43:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.407+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 msgr2=0x7f3fc8156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.407+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e870 tx=0x7f3fb800ed30 comp rx=0 tx=0).stop 2024-04-26T23:43:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 shutdown_connections 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3fa806c8f0 0x7f3fa806edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc8160bd0 0x7f3fc8145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fc80732c0 0x7f3fc81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 --2- 172.21.15.46:0/516383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc806e510 0x7f3fc8156ca0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 >> 172.21.15.46:0/516383599 conn(0x7f3fc806c530 msgr2=0x7f3fc80b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 shutdown_connections 2024-04-26T23:43:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.408+0000 7f3fcd400640 1 -- 172.21.15.46:0/516383599 wait complete. 2024-04-26T23:43:29.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:42:56.897784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:42:58.028020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:43:29.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:43:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.646+0000 7f40c2800640 1 Processor -- start 2024-04-26T23:43:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.646+0000 7f40c2800640 1 -- start start 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc15f300 0x7f40bc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40bc06c890 0x7f40bc0af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc0cccf0 con 0x7f40bc15ef20 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f40bc0ccfd0 con 0x7f40bc06c890 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc0aea20 con 0x7f40bc15f300 2024-04-26T23:43:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f40bc15ef20 0x7f40bc0cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60542/0 (socket says 172.21.15.46:60542) 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c1400640 1 -- 172.21.15.46:0/3354355979 learned_addr learned my addr 172.21.15.46:0/3354355979 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.648+0000 7f40c1e00640 1 --2- 172.21.15.46:0/3354355979 >> v2:172.21.15.46:3300/0 conn(0x7f40bc06c890 0x7f40bc0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.649+0000 7f40c0a00640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc15f300 0x7f40bc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.649+0000 7f40c1e00640 1 -- 172.21.15.46:0/3354355979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc15f300 msgr2=0x7f40bc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.649+0000 7f40c1e00640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc15f300 0x7f40bc06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.649+0000 7f40c1e00640 1 -- 172.21.15.46:0/3354355979 >> v1:172.21.15.46:6789/0 conn(0x7f40bc15ef20 legacy=0x7f40bc0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.649+0000 7f40c1e00640 1 -- 172.21.15.46:0/3354355979 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc0aed00 con 0x7f40bc06c890 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.650+0000 7f40c1e00640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc06c890 0x7f40bc0af500 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009d90 tx=0x7f40b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12c0d26e179d743 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.650+0000 7f40bbe00640 1 -- 172.21.15.46:0/3354355979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4031600 con 0x7f40bc06c890 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.650+0000 7f40bbe00640 1 -- 172.21.15.46:0/3354355979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b40317a0 con 0x7f40bc06c890 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.650+0000 7f40bbe00640 1 -- 172.21.15.46:0/3354355979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4031b70 con 0x7f40bc06c890 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 -- 172.21.15.46:0/3354355979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc06c890 msgr2=0x7f40bc0af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc06c890 0x7f40bc0af500 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009d90 tx=0x7f40b402a5d0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 -- 172.21.15.46:0/3354355979 shutdown_connections 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc06c890 0x7f40bc0af500 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 --2- 172.21.15.46:0/3354355979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc15f300 0x7f40bc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 -- 172.21.15.46:0/3354355979 >> 172.21.15.46:0/3354355979 conn(0x7f40bc0d1c50 msgr2=0x7f40bc0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 -- 172.21.15.46:0/3354355979 shutdown_connections 2024-04-26T23:43:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.651+0000 7f40c2800640 1 -- 172.21.15.46:0/3354355979 wait complete. 2024-04-26T23:43:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.653+0000 7f40c2800640 1 Processor -- start 2024-04-26T23:43:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.653+0000 7f40c2800640 1 -- start start 2024-04-26T23:43:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc06c890 0x7f40bc1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40bc15f300 0x7f40bc15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc163b30 con 0x7f40bc15ef20 2024-04-26T23:43:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc0cc9b0 con 0x7f40bc15f300 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc163cb0 con 0x7f40bc06c890 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.654+0000 7f40c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45814/0 (socket says 172.21.15.46:45814) 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 -- 172.21.15.46:0/1579472469 learned_addr learned my addr 172.21.15.46:0/1579472469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c1400640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc06c890 0x7f40bc1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 -- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc06c890 msgr2=0x7f40bc1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc06c890 0x7f40bc1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 -- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40bc15f300 msgr2=0x7f40bc15c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40bc15f300 0x7f40bc15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 -- 172.21.15.46:0/1579472469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc15f820 con 0x7f40bc15ef20 2024-04-26T23:43:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.655+0000 7f40c0a00640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.656+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8015070 con 0x7f40bc15ef20 2024-04-26T23:43:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.656+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8004220 con 0x7f40bc15ef20 2024-04-26T23:43:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.656+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8007620 con 0x7f40bc15ef20 2024-04-26T23:43:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.656+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc15fb00 con 0x7f40bc15ef20 2024-04-26T23:43:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.656+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1600c0 con 0x7f40bc15ef20 2024-04-26T23:43:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.657+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40bc069ac0 con 0x7f40bc15ef20 2024-04-26T23:43:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.658+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40a80079d0 con 0x7f40bc15ef20 2024-04-26T23:43:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.659+0000 7f40b9600640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.659+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f40a808bda0 con 0x7f40bc15ef20 2024-04-26T23:43:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.660+0000 7f40c1400640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.661+0000 7f40c1400640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 0x7f409c06ede0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000f60 tx=0x7f40b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:29.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.664+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40a8059f80 con 0x7f40bc15ef20 2024-04-26T23:43:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.872+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40bc161f40 con 0x7f40bc15ef20 2024-04-26T23:43:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.873+0000 7f40b9600640 1 -- 172.21.15.46:0/1579472469 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f40a8059880 con 0x7f40bc15ef20 2024-04-26T23:43:29.876 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 msgr2=0x7f409c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 0x7f409c06ede0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000f60 tx=0x7f40b0009040 comp rx=0 tx=0).stop 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 msgr2=0x7f40bc0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 shutdown_connections 2024-04-26T23:43:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40bc15f300 0x7f40bc15c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40bc15ef20 0x7f40bc0caea0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 --2- 172.21.15.46:0/1579472469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40bc06c890 0x7f40bc1617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.875+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 >> 172.21.15.46:0/1579472469 conn(0x7f40bc0d1c50 msgr2=0x7f40bc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.876+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 shutdown_connections 2024-04-26T23:43:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:29.876+0000 7f40c2800640 1 -- 172.21.15.46:0/1579472469 wait complete. 2024-04-26T23:43:29.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:43:30.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.106+0000 7f6214400640 1 Processor -- start 2024-04-26T23:43:30.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.106+0000 7f6214400640 1 -- start start 2024-04-26T23:43:30.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.107+0000 7f6214400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 0x7f620c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.107+0000 7f6214400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f620c06c890 0x7f620c162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.107+0000 7f6214400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f620c06f200 con 0x7f620c163270 2024-04-26T23:43:30.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.107+0000 7f6214400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f620c163d00 con 0x7f620c06c890 2024-04-26T23:43:30.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.107+0000 7f6214400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f620c06ccf0 con 0x7f620c15f1f0 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f620c06c890 0x7f620c162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6212400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f620c163270 0x7f620c0730b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60544/0 (socket says 172.21.15.46:60544) 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6212400640 1 -- 172.21.15.46:0/1601565903 learned_addr learned my addr 172.21.15.46:0/1601565903 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211000640 1 -- 172.21.15.46:0/1601565903 >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 msgr2=0x7f620c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211a00640 1 --2- 172.21.15.46:0/1601565903 >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 0x7f620c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211000640 1 --2- 172.21.15.46:0/1601565903 >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 0x7f620c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211000640 1 -- 172.21.15.46:0/1601565903 >> v1:172.21.15.46:6789/0 conn(0x7f620c163270 legacy=0x7f620c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211000640 1 -- 172.21.15.46:0/1601565903 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c163a20 con 0x7f620c06c890 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f620be00640 1 -- 172.21.15.46:0/1601565903 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2260193079 0 0) 0x7f620c06f200 con 0x7f620c163270 2024-04-26T23:43:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.108+0000 7f6211a00640 1 --2- 172.21.15.46:0/1601565903 >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 0x7f620c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.109+0000 7f6211000640 1 --2- 172.21.15.46:0/1601565903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c06c890 0x7f620c162d30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f620c0cf930 tx=0x7f620002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789d9190ff0b0ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f620be00640 1 -- 172.21.15.46:0/1601565903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200007740 con 0x7f620c06c890 2024-04-26T23:43:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f620be00640 1 -- 172.21.15.46:0/1601565903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62000078e0 con 0x7f620c06c890 2024-04-26T23:43:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f620be00640 1 -- 172.21.15.46:0/1601565903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200007cb0 con 0x7f620c06c890 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 -- 172.21.15.46:0/1601565903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c06c890 msgr2=0x7f620c162d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 --2- 172.21.15.46:0/1601565903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c06c890 0x7f620c162d30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f620c0cf930 tx=0x7f620002a680 comp rx=0 tx=0).stop 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 -- 172.21.15.46:0/1601565903 shutdown_connections 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 --2- 172.21.15.46:0/1601565903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c06c890 0x7f620c162d30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 --2- 172.21.15.46:0/1601565903 >> v2:172.21.15.142:3300/0 conn(0x7f620c15f1f0 0x7f620c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 -- 172.21.15.46:0/1601565903 >> 172.21.15.46:0/1601565903 conn(0x7f620c06da90 msgr2=0x7f620c0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 -- 172.21.15.46:0/1601565903 shutdown_connections 2024-04-26T23:43:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.110+0000 7f6214400640 1 -- 172.21.15.46:0/1601565903 wait complete. 2024-04-26T23:43:30.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.112+0000 7f6214400640 1 Processor -- start 2024-04-26T23:43:30.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.112+0000 7f6214400640 1 -- start start 2024-04-26T23:43:30.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f620c163270 0x7f620c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f620c0c8e80 0x7f620c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f620c15f5d0 con 0x7f620c15f1f0 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f620c06f810 con 0x7f620c0c8e80 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.113+0000 7f6214400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f620c163700 con 0x7f620c163270 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45836/0 (socket says 172.21.15.46:45836) 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 -- 172.21.15.46:0/1378005241 learned_addr learned my addr 172.21.15.46:0/1378005241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 -- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f620c163270 msgr2=0x7f620c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f620c163270 0x7f620c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 -- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f620c0c8e80 msgr2=0x7f620c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f620c0c8e80 0x7f620c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.114+0000 7f6211a00640 1 -- 172.21.15.46:0/1378005241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c0ca630 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6211a00640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c820 tx=0x7f61fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc015070 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc007a30 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc007e00 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c0ca910 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.115+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c0caed0 con 0x7f620c15f1f0 2024-04-26T23:43:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.116+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c069530 con 0x7f620c15f1f0 2024-04-26T23:43:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.117+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61fc018070 con 0x7f620c15f1f0 2024-04-26T23:43:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.118+0000 7f6209600640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.119+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f61fc08be10 con 0x7f620c15f1f0 2024-04-26T23:43:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.119+0000 7f6211000640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.121+0000 7f6211000640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 0x7f61f006edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6200034000 tx=0x7f620002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.123+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61fc08f030 con 0x7f620c15f1f0 2024-04-26T23:43:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.326+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f620c15e5d0 con 0x7f620c15f1f0 2024-04-26T23:43:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.327+0000 7f6209600640 1 -- 172.21.15.46:0/1378005241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7f61fc059f60 con 0x7f620c15f1f0 2024-04-26T23:43:30.331 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:43:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 msgr2=0x7f61f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 0x7f61f006edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6200034000 tx=0x7f620002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 msgr2=0x7f620c0c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c820 tx=0x7f61fc00cc20 comp rx=0 tx=0).stop 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 shutdown_connections 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f620c0c8e80 0x7f620c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f620c163270 0x7f620c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 --2- 172.21.15.46:0/1378005241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c15f1f0 0x7f620c0c9d60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 >> 172.21.15.46:0/1378005241 conn(0x7f620c06da90 msgr2=0x7f620c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 shutdown_connections 2024-04-26T23:43:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.331+0000 7f6214400640 1 -- 172.21.15.46:0/1378005241 wait complete. 2024-04-26T23:43:30.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:43:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.567+0000 7f0a64400640 1 Processor -- start 2024-04-26T23:43:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.567+0000 7f0a64400640 1 -- start start 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a64400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c15cf40 0x7f0a5c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a64400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a5c0af530 0x7f0a5c06c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a64400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5c160c60 con 0x7f0a5c06f060 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a64400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5c160f00 con 0x7f0a5c0af530 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a64400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c1611e0 con 0x7f0a5c15cf40 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a62400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a5c0af530 0x7f0a5c06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a62400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a5c0af530 0x7f0a5c06c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45846/0 (socket says 172.21.15.46:45846) 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a62400640 1 -- 172.21.15.46:0/1267358820 learned_addr learned my addr 172.21.15.46:0/1267358820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.569+0000 7f0a61a00640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c15cf40 0x7f0a5c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.570+0000 7f0a62400640 1 -- 172.21.15.46:0/1267358820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c15cf40 msgr2=0x7f0a5c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.570+0000 7f0a62400640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c15cf40 0x7f0a5c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.570+0000 7f0a62400640 1 -- 172.21.15.46:0/1267358820 >> v1:172.21.15.142:6789/0 conn(0x7f0a5c06f060 legacy=0x7f0a5c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.570+0000 7f0a62400640 1 -- 172.21.15.46:0/1267358820 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c1614c0 con 0x7f0a5c0af530 2024-04-26T23:43:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.571+0000 7f0a62400640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c06c890 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12c1e240a4bb83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.571+0000 7f0a5be00640 1 -- 172.21.15.46:0/1267358820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54037070 con 0x7f0a5c0af530 2024-04-26T23:43:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.571+0000 7f0a5be00640 1 -- 172.21.15.46:0/1267358820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a54032440 con 0x7f0a5c0af530 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.571+0000 7f0a5be00640 1 -- 172.21.15.46:0/1267358820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54032810 con 0x7f0a5c0af530 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 -- 172.21.15.46:0/1267358820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 msgr2=0x7f0a5c06c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c06c890 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 -- 172.21.15.46:0/1267358820 shutdown_connections 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c06c890 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 --2- 172.21.15.46:0/1267358820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c15cf40 0x7f0a5c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 -- 172.21.15.46:0/1267358820 >> 172.21.15.46:0/1267358820 conn(0x7f0a5c06da90 msgr2=0x7f0a5c0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 -- 172.21.15.46:0/1267358820 shutdown_connections 2024-04-26T23:43:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.572+0000 7f0a64400640 1 -- 172.21.15.46:0/1267358820 wait complete. 2024-04-26T23:43:30.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.574+0000 7f0a64400640 1 Processor -- start 2024-04-26T23:43:30.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.574+0000 7f0a64400640 1 -- start start 2024-04-26T23:43:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c06f060 0x7f0a5c1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 0x7f0a5c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0cc840 con 0x7f0a5c0af530 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c15d320 con 0x7f0a5c15cf40 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a64400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c0cc9c0 con 0x7f0a5c06f060 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45852/0 (socket says 172.21.15.46:45852) 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61000640 1 -- 172.21.15.46:0/2574521022 learned_addr learned my addr 172.21.15.46:0/2574521022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61a00640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c06f060 0x7f0a5c1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a62400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 0x7f0a5c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61000640 1 -- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c06f060 msgr2=0x7f0a5c1f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.575+0000 7f0a61000640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c06f060 0x7f0a5c1f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a61000640 1 -- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 msgr2=0x7f0a5c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a61000640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 0x7f0a5c15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a61000640 1 -- 172.21.15.46:0/2574521022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c152790 con 0x7f0a5c0af530 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a62400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 0x7f0a5c15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a61000640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800c440 tx=0x7f0a48001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480052d0 con 0x7f0a5c0af530 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a48005470 con 0x7f0a5c0af530 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48007260 con 0x7f0a5c0af530 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c15d9e0 con 0x7f0a5c0af530 2024-04-26T23:43:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.576+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c15df60 con 0x7f0a5c0af530 2024-04-26T23:43:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.577+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c069530 con 0x7f0a5c0af530 2024-04-26T23:43:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.578+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a48007400 con 0x7f0a5c0af530 2024-04-26T23:43:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.579+0000 7f0a59600640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 0x7f0a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.579+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f0a4808b380 con 0x7f0a5c0af530 2024-04-26T23:43:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.580+0000 7f0a61a00640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 0x7f0a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.581+0000 7f0a61a00640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 0x7f0a3c06edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c15f460 tx=0x7f0a500039b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.584+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a5c152790 con 0x7f0a5c0af530 2024-04-26T23:43:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.789+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a5c0d2960 con 0x7f0a5c0af530 2024-04-26T23:43:30.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.790+0000 7f0a59600640 1 -- 172.21.15.46:0/2574521022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7f0a480594d0 con 0x7f0a5c0af530 2024-04-26T23:43:30.794 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:43:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 msgr2=0x7f0a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 0x7f0a3c06edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c15f460 tx=0x7f0a500039b0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 msgr2=0x7f0a5c152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800c440 tx=0x7f0a48001e70 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 shutdown_connections 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a3c06c8f0 0x7f0a3c06edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a5c15cf40 0x7f0a5c15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5c0af530 0x7f0a5c152250 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 --2- 172.21.15.46:0/2574521022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a5c06f060 0x7f0a5c1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 >> 172.21.15.46:0/2574521022 conn(0x7f0a5c06da90 msgr2=0x7f0a5c0d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 shutdown_connections 2024-04-26T23:43:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:30.793+0000 7f0a64400640 1 -- 172.21.15.46:0/2574521022 wait complete. 2024-04-26T23:43:30.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.018+0000 7f505c000640 1 Processor -- start 2024-04-26T23:43:31.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.018+0000 7f505c000640 1 -- start start 2024-04-26T23:43:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 0x7f505406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5054160bd0 0x7f5054161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f505415d540 0x7f505415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f505406ee30 con 0x7f5054160bd0 2024-04-26T23:43:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5054161940 con 0x7f505415d540 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f505c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5054071f80 con 0x7f505406e510 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f5058c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5054160bd0 0x7f5054161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f5058c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5054160bd0 0x7f5054161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45860/0 (socket says 172.21.15.46:45860) 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.020+0000 7f5058c00640 1 -- 172.21.15.46:0/1237831485 learned_addr learned my addr 172.21.15.46:0/1237831485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5059600640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 0x7f505406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f505a000640 1 --2- 172.21.15.46:0/1237831485 >> v2:172.21.15.142:3300/0 conn(0x7f505415d540 0x7f505415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5058c00640 1 -- 172.21.15.46:0/1237831485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 msgr2=0x7f505406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5058c00640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 0x7f505406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5058c00640 1 -- 172.21.15.46:0/1237831485 >> v2:172.21.15.142:3300/0 conn(0x7f505415d540 msgr2=0x7f505415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5058c00640 1 --2- 172.21.15.46:0/1237831485 >> v2:172.21.15.142:3300/0 conn(0x7f505415d540 0x7f505415d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5058c00640 1 -- 172.21.15.46:0/1237831485 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5054072260 con 0x7f5054160bd0 2024-04-26T23:43:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.021+0000 7f5059600640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 0x7f505406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.022+0000 7f5058c00640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5054160bd0 0x7f5054161030 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f50540ce520 tx=0x7f504002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d789260ffc229e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.023+0000 7f5053e00640 1 -- 172.21.15.46:0/1237831485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040037070 con 0x7f5054160bd0 2024-04-26T23:43:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.023+0000 7f5053e00640 1 -- 172.21.15.46:0/1237831485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040032440 con 0x7f5054160bd0 2024-04-26T23:43:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.023+0000 7f5053e00640 1 -- 172.21.15.46:0/1237831485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040032810 con 0x7f5054160bd0 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 -- 172.21.15.46:0/1237831485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5054160bd0 msgr2=0x7f5054161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5054160bd0 0x7f5054161030 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f50540ce520 tx=0x7f504002a1d0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 -- 172.21.15.46:0/1237831485 shutdown_connections 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 --2- 172.21.15.46:0/1237831485 >> v2:172.21.15.142:3300/0 conn(0x7f505415d540 0x7f505415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5054160bd0 0x7f5054161030 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 --2- 172.21.15.46:0/1237831485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f505406e510 0x7f505406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 -- 172.21.15.46:0/1237831485 >> 172.21.15.46:0/1237831485 conn(0x7f505406c530 msgr2=0x7f505406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:31.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 -- 172.21.15.46:0/1237831485 shutdown_connections 2024-04-26T23:43:31.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.024+0000 7f505c000640 1 -- 172.21.15.46:0/1237831485 wait complete. 2024-04-26T23:43:31.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.026+0000 7f505c000640 1 Processor -- start 2024-04-26T23:43:31.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.026+0000 7f505c000640 1 -- start start 2024-04-26T23:43:31.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 0x7f50541585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 0x7f50541440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5054161570 con 0x7f505415d540 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5054072820 con 0x7f505406e510 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f505c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f50541616f0 con 0x7f5054160bd0 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f5058c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f5058c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45864/0 (socket says 172.21.15.46:45864) 2024-04-26T23:43:31.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.027+0000 7f5058c00640 1 -- 172.21.15.46:0/2772350106 learned_addr learned my addr 172.21.15.46:0/2772350106 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5059600640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 0x7f50541585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f505a000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 0x7f50541440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 -- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 msgr2=0x7f50541440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 0x7f50541440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 -- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 msgr2=0x7f50541585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 0x7f50541585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 -- 172.21.15.46:0/2772350106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50540b5bf0 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5059600640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 0x7f50541585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f505a000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 0x7f50541440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.028+0000 7f5058c00640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f50400090b0 tx=0x7f50400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.029+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040037050 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.029+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040043070 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.029+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040030c70 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.029+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5054144d70 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.029+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50540b2c60 con 0x7f505415d540 2024-04-26T23:43:31.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.030+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50540b2e60 con 0x7f505415d540 2024-04-26T23:43:31.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.031+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5040048460 con 0x7f505415d540 2024-04-26T23:43:31.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.032+0000 7f5051600640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 0x7f503406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.033+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f50400acc20 con 0x7f505415d540 2024-04-26T23:43:31.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.033+0000 7f5059600640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 0x7f503406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.034+0000 7f5059600640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 0x7f503406edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f505406eb40 tx=0x7f5048008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.037+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f50540b2e60 con 0x7f505415d540 2024-04-26T23:43:31.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.283+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5054069530 con 0x7f505415d540 2024-04-26T23:43:31.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.285+0000 7f5051600640 1 -- 172.21.15.46:0/2772350106 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f50540b5bf0 con 0x7f505415d540 2024-04-26T23:43:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:31.288 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:31.289 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 msgr2=0x7f503406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 0x7f503406edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f505406eb40 tx=0x7f5048008040 comp rx=0 tx=0).stop 2024-04-26T23:43:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 msgr2=0x7f5054145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f50400090b0 tx=0x7f50400046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 shutdown_connections 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f503406c8f0 0x7f503406edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5054160bd0 0x7f50541440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f505415d540 0x7f5054145bd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 --2- 172.21.15.46:0/2772350106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f505406e510 0x7f50541585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 >> 172.21.15.46:0/2772350106 conn(0x7f505406c530 msgr2=0x7f505406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 shutdown_connections 2024-04-26T23:43:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.288+0000 7f505c000640 1 -- 172.21.15.46:0/2772350106 wait complete. 2024-04-26T23:43:31.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.527+0000 7fc0ef000640 1 Processor -- start 2024-04-26T23:43:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.527+0000 7fc0ef000640 1 -- start start 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 0x7fc0e806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0e80720e0 0x7fc0e80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 0x7fc0e8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e8073ad0 con 0x7fc0e80720e0 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e8146a90 con 0x7fc0e81466b0 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.528+0000 7fc0ef000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e8073d70 con 0x7fc0e8160910 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0e80720e0 0x7fc0e80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0e80720e0 0x7fc0e80cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45872/0 (socket says 172.21.15.46:45872) 2024-04-26T23:43:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 -- 172.21.15.46:0/1983338521 learned_addr learned my addr 172.21.15.46:0/1983338521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1983338521 >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 0x7fc0e806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ee600640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 0x7fc0e8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 -- 172.21.15.46:0/1983338521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 msgr2=0x7fc0e8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 0x7fc0e8160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 -- 172.21.15.46:0/1983338521 >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 msgr2=0x7fc0e806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1983338521 >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 0x7fc0e806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ed200640 1 -- 172.21.15.46:0/1983338521 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e8074050 con 0x7fc0e80720e0 2024-04-26T23:43:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0ee600640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 0x7fc0e8160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.529+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1983338521 >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 0x7fc0e806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.530+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e80720e0 0x7fc0e80cd860 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009870 tx=0x7fc0d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91415fd27118cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.531+0000 7fc0e7e00640 1 -- 172.21.15.46:0/1983338521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4037070 con 0x7fc0e80720e0 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.531+0000 7fc0e7e00640 1 -- 172.21.15.46:0/1983338521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d4032440 con 0x7fc0e80720e0 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.531+0000 7fc0e7e00640 1 -- 172.21.15.46:0/1983338521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4032810 con 0x7fc0e80720e0 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.531+0000 7fc0ef000640 1 -- 172.21.15.46:0/1983338521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e80720e0 msgr2=0x7fc0e80cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.531+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e80720e0 0x7fc0e80cd860 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009870 tx=0x7fc0d402a210 comp rx=0 tx=0).stop 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 -- 172.21.15.46:0/1983338521 shutdown_connections 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e8160910 0x7fc0e8160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1983338521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e80720e0 0x7fc0e80cd860 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1983338521 >> v2:172.21.15.142:3300/0 conn(0x7fc0e81466b0 0x7fc0e806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 -- 172.21.15.46:0/1983338521 >> 172.21.15.46:0/1983338521 conn(0x7fc0e80b26d0 msgr2=0x7fc0e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 -- 172.21.15.46:0/1983338521 shutdown_connections 2024-04-26T23:43:31.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.532+0000 7fc0ef000640 1 -- 172.21.15.46:0/1983338521 wait complete. 2024-04-26T23:43:31.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.533+0000 7fc0ef000640 1 Processor -- start 2024-04-26T23:43:31.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 -- start start 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 0x7fc0e813f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e8160910 0x7fc0e80c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e806e3c0 con 0x7fc0e81466b0 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e80cb470 con 0x7fc0e8160910 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.534+0000 7fc0ef000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e80cdda0 con 0x7fc0e80720e0 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45882/0 (socket says 172.21.15.46:45882) 2024-04-26T23:43:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 -- 172.21.15.46:0/1869099190 learned_addr learned my addr 172.21.15.46:0/1869099190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 0x7fc0e813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ee600640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e8160910 0x7fc0e80c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 -- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 msgr2=0x7fc0e813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 0x7fc0e813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 -- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e8160910 msgr2=0x7fc0e80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e8160910 0x7fc0e80c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0ed200640 1 -- 172.21.15.46:0/1869099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e80aeb30 con 0x7fc0e81466b0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.535+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 0x7fc0e813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0ed200640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4037040 con 0x7fc0e81466b0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d4042070 con 0x7fc0e81466b0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4030bd0 con 0x7fc0e81466b0 2024-04-26T23:43:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e80aee10 con 0x7fc0e81466b0 2024-04-26T23:43:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.536+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e80c7e20 con 0x7fc0e81466b0 2024-04-26T23:43:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.537+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e80cae40 con 0x7fc0e81466b0 2024-04-26T23:43:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.538+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0d4047460 con 0x7fc0e81466b0 2024-04-26T23:43:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.539+0000 7fc0e5600640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 0x7fc0c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.539+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fc0d40ac970 con 0x7fc0e81466b0 2024-04-26T23:43:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.540+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 0x7fc0c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.541+0000 7fc0edc00640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 0x7fc0c806edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006fd0 tx=0x7fc0dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.545+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0e80cae40 con 0x7fc0e81466b0 2024-04-26T23:43:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.757+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0e8143170 con 0x7fc0e81466b0 2024-04-26T23:43:31.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.759+0000 7fc0e5600640 1 -- 172.21.15.46:0/1869099190 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17109 (secure 0 0 0) 0x7fc0e8143170 con 0x7fc0e81466b0 2024-04-26T23:43:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:31.763 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:14.531022+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.46:6834/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.142:6835/2927770154":"2024-04-27T23:43:14.513761+0000","172.21.15.142:6834/2927770154":"2024-04-27T23:43:14.513761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 msgr2=0x7fc0c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 0x7fc0c806edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006fd0 tx=0x7fc0dc008040 comp rx=0 tx=0).stop 2024-04-26T23:43:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 msgr2=0x7fc0e80c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 shutdown_connections 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc0c806c8f0 0x7fc0c806edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e8160910 0x7fc0e80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0e81466b0 0x7fc0e80c95f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 --2- 172.21.15.46:0/1869099190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0e80720e0 0x7fc0e813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 >> 172.21.15.46:0/1869099190 conn(0x7fc0e80b26d0 msgr2=0x7fc0e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 shutdown_connections 2024-04-26T23:43:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.762+0000 7fc0ef000640 1 -- 172.21.15.46:0/1869099190 wait complete. 2024-04-26T23:43:31.787 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:43:31.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:31.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.992+0000 7f792e000640 1 Processor -- start 2024-04-26T23:43:31.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.992+0000 7f792e000640 1 -- start start 2024-04-26T23:43:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79281466b0 0x7f792806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f79280cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7928073ad0 con 0x7f7928160910 2024-04-26T23:43:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792e000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7928146a90 con 0x7f79281466b0 2024-04-26T23:43:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7928073d70 con 0x7f79280720e0 2024-04-26T23:43:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79281466b0 0x7f792806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79281466b0 0x7f792806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42792/0 (socket says 172.21.15.46:42792) 2024-04-26T23:43:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.994+0000 7f792cc00640 1 -- 172.21.15.46:0/1743149161 learned_addr learned my addr 172.21.15.46:0/1743149161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.995+0000 7f7927e00640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f79280cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.995+0000 7f792cc00640 1 -- 172.21.15.46:0/1743149161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 msgr2=0x7f79280cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.995+0000 7f792cc00640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f79280cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.995+0000 7f792cc00640 1 -- 172.21.15.46:0/1743149161 >> v1:172.21.15.142:6789/0 conn(0x7f7928160910 legacy=0x7f79280cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.995+0000 7f792cc00640 1 -- 172.21.15.46:0/1743149161 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928074050 con 0x7f79281466b0 2024-04-26T23:43:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.996+0000 7f792cc00640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79281466b0 0x7f792806de80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f791c009870 tx=0x7f791c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6745ec8ea3e2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.996+0000 7f7927400640 1 -- 172.21.15.46:0/1743149161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c037070 con 0x7f79281466b0 2024-04-26T23:43:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.996+0000 7f7927400640 1 -- 172.21.15.46:0/1743149161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c032440 con 0x7f79281466b0 2024-04-26T23:43:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.996+0000 7f7927400640 1 -- 172.21.15.46:0/1743149161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c032810 con 0x7f79281466b0 2024-04-26T23:43:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 -- 172.21.15.46:0/1743149161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79281466b0 msgr2=0x7f792806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79281466b0 0x7f792806de80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f791c009870 tx=0x7f791c02a210 comp rx=0 tx=0).stop 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 -- 172.21.15.46:0/1743149161 shutdown_connections 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f79280cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 --2- 172.21.15.46:0/1743149161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79281466b0 0x7f792806de80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 -- 172.21.15.46:0/1743149161 >> 172.21.15.46:0/1743149161 conn(0x7f79280b26d0 msgr2=0x7f79280b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 -- 172.21.15.46:0/1743149161 shutdown_connections 2024-04-26T23:43:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.997+0000 7f792e000640 1 -- 172.21.15.46:0/1743149161 wait complete. 2024-04-26T23:43:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.999+0000 7f792e000640 1 Processor -- start 2024-04-26T23:43:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:31.999+0000 7f792e000640 1 -- start start 2024-04-26T23:43:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f792814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79281466b0 0x7f79281eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f792806e3c0 con 0x7f7928160910 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79280cb470 con 0x7f79281466b0 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79280cdda0 con 0x7f79280720e0 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42804/0 (socket says 172.21.15.46:42804) 2024-04-26T23:43:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.000+0000 7f792d600640 1 -- 172.21.15.46:0/4027291252 learned_addr learned my addr 172.21.15.46:0/4027291252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 -- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 msgr2=0x7f792814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f792814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 -- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79281466b0 msgr2=0x7f79281eb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79281466b0 0x7f79281eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 -- 172.21.15.46:0/4027291252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79281eb9f0 con 0x7f7928160910 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792d600640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f792000b970 tx=0x7f792000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920019070 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792000cb40 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.001+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79280af170 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.002+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920014440 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.002+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79280af730 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.003+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79280693b0 con 0x7f7928160910 2024-04-26T23:43:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.004+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f792001c070 con 0x7f7928160910 2024-04-26T23:43:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.004+0000 7f7924c00640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 0x7f790806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.005+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f792008baa0 con 0x7f7928160910 2024-04-26T23:43:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.005+0000 7f792cc00640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 0x7f790806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.006+0000 7f792cc00640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 0x7f790806ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f791c034000 tx=0x7f791c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.009+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7920059bf0 con 0x7f7928160910 2024-04-26T23:43:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.256+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79280c8740 con 0x7f7928160910 2024-04-26T23:43:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.257+0000 7f7924c00640 1 -- 172.21.15.46:0/4027291252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f79200594f0 con 0x7f7928160910 2024-04-26T23:43:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 msgr2=0x7f790806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 0x7f790806ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f791c034000 tx=0x7f791c02c000 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 msgr2=0x7f79281e8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f792000b970 tx=0x7f792000be30 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 shutdown_connections 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790806c920 0x7f790806ede0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7928160910 0x7f79281e8770 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79281466b0 0x7f79281eb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 --2- 172.21.15.46:0/4027291252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79280720e0 0x7f792814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 >> 172.21.15.46:0/4027291252 conn(0x7f79280b26d0 msgr2=0x7f792806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 shutdown_connections 2024-04-26T23:43:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.260+0000 7f792e000640 1 -- 172.21.15.46:0/4027291252 wait complete. 2024-04-26T23:43:32.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.492+0000 7f719c400640 1 Processor -- start 2024-04-26T23:43:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.492+0000 7f719c400640 1 -- start start 2024-04-26T23:43:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f719c400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f719406e510 0x7f719406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f719c400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719406ee30 con 0x7f7194160bd0 2024-04-26T23:43:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f719c400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7194161940 con 0x7f71940732c0 2024-04-26T23:43:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f719c400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7194071f80 con 0x7f719406e510 2024-04-26T23:43:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f7199000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7194160bd0 0x7f7194161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55822/0 (socket says 172.21.15.46:55822) 2024-04-26T23:43:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f7199000640 1 -- 172.21.15.46:0/2369415736 learned_addr learned my addr 172.21.15.46:0/2369415736 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.494+0000 7f7199a00640 1 --2- 172.21.15.46:0/2369415736 >> v2:172.21.15.46:3300/0 conn(0x7f719406e510 0x7f719406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.495+0000 7f7199a00640 1 -- 172.21.15.46:0/2369415736 >> v1:172.21.15.142:6789/0 conn(0x7f71940732c0 legacy=0x7f7194073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.495+0000 7f7199a00640 1 -- 172.21.15.46:0/2369415736 >> v1:172.21.15.46:6789/0 conn(0x7f7194160bd0 legacy=0x7f7194161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.495+0000 7f7199a00640 1 -- 172.21.15.46:0/2369415736 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194072260 con 0x7f719406e510 2024-04-26T23:43:32.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.496+0000 7f7199a00640 1 --2- 172.21.15.46:0/2369415736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f719406e8f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f71880097d0 tx=0x7f718802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc46e103b1a4e579 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.496+0000 7f7193e00640 1 -- 172.21.15.46:0/2369415736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188037070 con 0x7f719406e510 2024-04-26T23:43:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.496+0000 7f7193e00640 1 -- 172.21.15.46:0/2369415736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7188032440 con 0x7f719406e510 2024-04-26T23:43:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.496+0000 7f7193e00640 1 -- 172.21.15.46:0/2369415736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188032810 con 0x7f719406e510 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 -- 172.21.15.46:0/2369415736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 msgr2=0x7f719406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 --2- 172.21.15.46:0/2369415736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f719406e8f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f71880097d0 tx=0x7f718802a210 comp rx=0 tx=0).stop 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 -- 172.21.15.46:0/2369415736 shutdown_connections 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 --2- 172.21.15.46:0/2369415736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f719406e8f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 -- 172.21.15.46:0/2369415736 >> 172.21.15.46:0/2369415736 conn(0x7f719406c530 msgr2=0x7f719406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 -- 172.21.15.46:0/2369415736 shutdown_connections 2024-04-26T23:43:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.497+0000 7f719c400640 1 -- 172.21.15.46:0/2369415736 wait complete. 2024-04-26T23:43:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.499+0000 7f719c400640 1 Processor -- start 2024-04-26T23:43:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.499+0000 7f719c400640 1 -- start start 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71940732c0 0x7f7194159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7194160bd0 0x7f7194159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7194072d40 con 0x7f719406e510 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7194072820 con 0x7f7194160bd0 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f719c400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7194072ec0 con 0x7f71940732c0 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f7199a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f7199a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42826/0 (socket says 172.21.15.46:42826) 2024-04-26T23:43:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.500+0000 7f7199a00640 1 -- 172.21.15.46:0/191559961 learned_addr learned my addr 172.21.15.46:0/191559961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 -- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71940732c0 msgr2=0x7f7194159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71940732c0 0x7f7194159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 -- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7194160bd0 msgr2=0x7f7194159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7194160bd0 0x7f7194159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 -- 172.21.15.46:0/191559961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194157800 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7199a00640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f71880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188037040 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7188004230 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7194157ae0 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.501+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188032e20 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.502+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7194154fc0 con 0x7f719406e510 2024-04-26T23:43:32.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.503+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7194069530 con 0x7f719406e510 2024-04-26T23:43:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.504+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7188030770 con 0x7f719406e510 2024-04-26T23:43:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.505+0000 7f7191600640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 0x7f717406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.505+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f71880acc70 con 0x7f719406e510 2024-04-26T23:43:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.505+0000 7f7199000640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 0x7f717406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.506+0000 7f7199000640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 0x7f717406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7180000f60 tx=0x7f7180007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.509+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f718807ae50 con 0x7f719406e510 2024-04-26T23:43:32.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.758+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71941593f0 con 0x7f719406e510 2024-04-26T23:43:32.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.760+0000 7f7191600640 1 -- 172.21.15.46:0/191559961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5189 (secure 0 0 0) 0x7f718807a750 con 0x7f719406e510 2024-04-26T23:43:32.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:32.763 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:42:59.049825+0000","modified":"2024-04-26T23:43:18.573173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2393654178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2393654178},{"type":"v1","addr":"172.21.15.46:6835","nonce":2393654178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":4}]} 2024-04-26T23:43:32.763 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 msgr2=0x7f717406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 0x7f717406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7180000f60 tx=0x7f7180007a80 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 msgr2=0x7f71941f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f71880046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 shutdown_connections 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f717406c8f0 0x7f717406edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7194160bd0 0x7f7194159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71940732c0 0x7f7194159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 --2- 172.21.15.46:0/191559961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719406e510 0x7f71941f4cb0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.762+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 >> 172.21.15.46:0/191559961 conn(0x7f719406c530 msgr2=0x7f719415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.763+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 shutdown_connections 2024-04-26T23:43:32.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:32.763+0000 7f719c400640 1 -- 172.21.15.46:0/191559961 wait complete. 2024-04-26T23:43:32.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:33.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.001+0000 7fa7b3000640 1 Processor -- start 2024-04-26T23:43:33.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.001+0000 7fa7b3000640 1 -- start start 2024-04-26T23:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa7ac160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac073ad0 con 0x7fa7ac1466b0 2024-04-26T23:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac146a90 con 0x7fa7ac0720e0 2024-04-26T23:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac073d70 con 0x7fa7ac160910 2024-04-26T23:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b1c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7ac1466b0 0x7fa7ac06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55838/0 (socket says 172.21.15.46:55838) 2024-04-26T23:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.003+0000 7fa7b1c00640 1 -- 172.21.15.46:0/133296526 learned_addr learned my addr 172.21.15.46:0/133296526 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7b2600640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa7ac160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313896424 0 0) 0x7fa7ac073ad0 con 0x7fa7ac1466b0 2024-04-26T23:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa790003520 con 0x7fa7ac1466b0 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1662833889 0 0) 0x7fa790003520 con 0x7fa7ac1466b0 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 msgr2=0x7fa7ac160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa7ac160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 >> v1:172.21.15.142:6789/0 conn(0x7fa7ac0720e0 legacy=0x7fa7ac0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac074050 con 0x7fa7ac1466b0 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa79c001670 con 0x7fa7ac1466b0 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.004+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 >> v1:172.21.15.46:6789/0 conn(0x7fa7ac1466b0 legacy=0x7fa7ac06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7900039e0 0x7fa790003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa790004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa790004d40 0x7fa7900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac073d70 con 0x7fa790004d40 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa79000b9c0 con 0x7fa7900039e0 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa79000bbc0 con 0x7fa7ac160910 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa790004d40 0x7fa7900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b1c00640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7900039e0 0x7fa790003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b1200640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa790004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 -- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 msgr2=0x7fa790004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa790004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 -- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7900039e0 msgr2=0x7fa790003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7900039e0 0x7fa790003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b2600640 1 -- 172.21.15.46:0/133296526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa790003520 con 0x7fa790004d40 2024-04-26T23:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.005+0000 7fa7b1200640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac160910 0x7fa790004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.006+0000 7fa7b2600640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa790004d40 0x7fa7900051c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fc0 tx=0x7fa790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5347da75d790e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4009c10 con 0x7fa790004d40 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4036070 con 0x7fa790004d40 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7abe00640 1 -- 172.21.15.46:0/133296526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a402d070 con 0x7fa790004d40 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 -- 172.21.15.46:0/133296526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa790004d40 msgr2=0x7fa7900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 --2- 172.21.15.46:0/133296526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa790004d40 0x7fa7900051c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fc0 tx=0x7fa790005830 comp rx=0 tx=0).stop 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b2600640 1 -- 172.21.15.46:0/133296526 reap_dead start 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 -- 172.21.15.46:0/133296526 shutdown_connections 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 -- 172.21.15.46:0/133296526 >> 172.21.15.46:0/133296526 conn(0x7fa7ac0b26d0 msgr2=0x7fa7ac0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 -- 172.21.15.46:0/133296526 shutdown_connections 2024-04-26T23:43:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.007+0000 7fa7b3000640 1 -- 172.21.15.46:0/133296526 wait complete. 2024-04-26T23:43:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.009+0000 7fa7b3000640 1 Processor -- start 2024-04-26T23:43:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.009+0000 7fa7b3000640 1 -- start start 2024-04-26T23:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.010+0000 7fa7b3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa790004d40 0x7fa7ac0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.010+0000 7fa7b3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac13bae0 0x7fa7ac15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.010+0000 7fa7b3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac160da0 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac0cb470 con 0x7fa790004d40 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac160f20 con 0x7fa7ac13bae0 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42842/0 (socket says 172.21.15.46:42842) 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 -- 172.21.15.46:0/3645744364 learned_addr learned my addr 172.21.15.46:0/3645744364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b1c00640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa790004d40 0x7fa7ac0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b1200640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac13bae0 0x7fa7ac15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 -- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac13bae0 msgr2=0x7fa7ac15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac13bae0 0x7fa7ac15f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 -- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa790004d40 msgr2=0x7fa7ac0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa790004d40 0x7fa7ac0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.011+0000 7fa7b2600640 1 -- 172.21.15.46:0/3645744364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac0b5600 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7b2600640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402af10 tx=0x7fa7a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4009c10 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4045430 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4045820 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac0b38b0 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.012+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac0b3e70 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.014+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac0693b0 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.015+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7a4005d10 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.016+0000 7fa7a9600640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.016+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fa7a40ad330 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.017+0000 7fa7b1c00640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.018+0000 7fa7b1c00640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 0x7fa78c06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa79c000c00 tx=0x7fa79c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.021+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7a407b630 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.230+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7ac160290 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.232+0000 7fa7a9600640 1 -- 172.21.15.46:0/3645744364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17109 (secure 0 0 0) 0x7fa7a407af30 con 0x7fa7ac0b4c90 2024-04-26T23:43:33.236 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:33.236 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:14.531022+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:42:56.897784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T23:42:58.028020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.46:6834/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.142:6835/2927770154":"2024-04-27T23:43:14.513761+0000","172.21.15.142:6834/2927770154":"2024-04-27T23:43:14.513761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 msgr2=0x7fa78c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 0x7fa78c06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa79c000c00 tx=0x7fa79c009040 comp rx=0 tx=0).stop 2024-04-26T23:43:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 msgr2=0x7fa7ac15e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402af10 tx=0x7fa7a4009750 comp rx=0 tx=0).stop 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 shutdown_connections 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7ac0b4c90 0x7fa7ac15e350 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7ac13bae0 0x7fa7ac15f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 --2- 172.21.15.46:0/3645744364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa790004d40 0x7fa7ac0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 >> 172.21.15.46:0/3645744364 conn(0x7fa7ac0b26d0 msgr2=0x7fa7ac145390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 shutdown_connections 2024-04-26T23:43:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.235+0000 7fa7b3000640 1 -- 172.21.15.46:0/3645744364 wait complete. 2024-04-26T23:43:33.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:43:33.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.470+0000 7f1795400640 1 Processor -- start 2024-04-26T23:43:33.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.470+0000 7f1795400640 1 -- start start 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.471+0000 7f1795400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f17901604c0 0x7f1790073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.471+0000 7f1795400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17900b43f0 con 0x7f179006ec20 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.471+0000 7f1795400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17900b46d0 con 0x7f1790142280 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.471+0000 7f1795400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f17900b49b0 con 0x7f17901604c0 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.472+0000 7f178de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f179006ec20 0x7f17900734a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55840/0 (socket says 172.21.15.46:55840) 2024-04-26T23:43:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.472+0000 7f178de00640 1 -- 172.21.15.46:0/3297275410 learned_addr learned my addr 172.21.15.46:0/3297275410 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.472+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3640603436 0 0) 0x7f17900b43f0 con 0x7f179006ec20 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.472+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> v2:172.21.15.142:3300/0 conn(0x7f17901604c0 0x7f1790073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.472+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1774003520 con 0x7f179006ec20 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1028168843 0 0) 0x7f1774003520 con 0x7f179006ec20 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 >> v2:172.21.15.142:3300/0 conn(0x7f17901604c0 msgr2=0x7f1790073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 --2- 172.21.15.46:0/3297275410 >> v2:172.21.15.142:3300/0 conn(0x7f17901604c0 0x7f1790073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 >> v1:172.21.15.142:6789/0 conn(0x7f1790142280 legacy=0x7f179006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900b4c90 con 0x7f179006ec20 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1778001670 con 0x7f179006ec20 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> v2:172.21.15.142:3300/0 conn(0x7f17901604c0 0x7f1790073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 >> v1:172.21.15.46:6789/0 conn(0x7f179006ec20 legacy=0x7f17900734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1774003a00 0x7f1774003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774004800 0x7f1774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 0x7f177400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17900b49b0 con 0x7f1774005210 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f177400c140 con 0x7f1774004800 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f177400c340 con 0x7f1774003a00 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 0x7f177400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.473+0000 7f178de00640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774004800 0x7f1774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 -- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1774003a00 msgr2=0x7f1774003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1774003a00 0x7f1774003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 -- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774004800 msgr2=0x7f1774004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774004800 0x7f1774004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 -- 172.21.15.46:0/3297275410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774003520 con 0x7f1774005210 2024-04-26T23:43:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.474+0000 7f178f200640 1 -- 172.21.15.46:0/3297275410 reap_dead start 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f178f200640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 0x7f177400b9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1788006fd0 tx=0x7f1774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff1203cff33e13b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178800f040 con 0x7f1774005210 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178802ad00 con 0x7f1774005210 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f178d400640 1 -- 172.21.15.46:0/3297275410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004260 con 0x7f1774005210 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f1795400640 1 -- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 msgr2=0x7f177400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.475+0000 7f1795400640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 0x7f177400b9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1788006fd0 tx=0x7f1774005830 comp rx=0 tx=0).stop 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.476+0000 7f1795400640 1 -- 172.21.15.46:0/3297275410 shutdown_connections 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.476+0000 7f1795400640 1 --2- 172.21.15.46:0/3297275410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1774005210 0x7f177400b9c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.476+0000 7f1795400640 1 -- 172.21.15.46:0/3297275410 >> 172.21.15.46:0/3297275410 conn(0x7f179006c530 msgr2=0x7f179006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.476+0000 7f1795400640 1 -- 172.21.15.46:0/3297275410 shutdown_connections 2024-04-26T23:43:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.476+0000 7f1795400640 1 -- 172.21.15.46:0/3297275410 wait complete. 2024-04-26T23:43:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.477+0000 7f1795400640 1 Processor -- start 2024-04-26T23:43:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.478+0000 7f1795400640 1 -- start start 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.478+0000 7f1795400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 0x7f1790140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f1795400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f1795400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17901f8410 0x7f17901f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f1795400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1790142a20 con 0x7f1790161ce0 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f1795400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17901428a0 con 0x7f1774005210 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f1795400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f17900b3e80 con 0x7f17901f8410 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42868/0 (socket says 172.21.15.46:42868) 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 -- 172.21.15.46:0/584103363 learned_addr learned my addr 172.21.15.46:0/584103363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178e800640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 0x7f1790140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178f200640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17901f8410 0x7f17901f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 -- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17901f8410 msgr2=0x7f17901f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17901f8410 0x7f17901f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 -- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 msgr2=0x7f1790140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 0x7f1790140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178de00640 1 -- 172.21.15.46:0/584103363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901f4e00 con 0x7f1790161ce0 2024-04-26T23:43:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.479+0000 7f178e800640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 0x7f1790140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f178de00640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f177800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800c6c0 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778018070 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901f50e0 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800ca80 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.480+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901f56a0 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.481+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17901f4b80 con 0x7f1790161ce0 2024-04-26T23:43:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.482+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1778013420 con 0x7f1790161ce0 2024-04-26T23:43:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.484+0000 7f1786a00640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 0x7f176c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.484+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f177808bb60 con 0x7f1790161ce0 2024-04-26T23:43:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.484+0000 7f178e800640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 0x7f176c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.485+0000 7f178e800640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 0x7f176c06ede0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1780004510 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.488+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17901f4b80 con 0x7f1790161ce0 2024-04-26T23:43:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.737+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1790069530 con 0x7f1790161ce0 2024-04-26T23:43:33.742 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:43:33.739+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:43:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.742+0000 7f1786a00640 1 -- 172.21.15.46:0/584103363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f1778059cb0 con 0x7f1790161ce0 2024-04-26T23:43:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 msgr2=0x7f176c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 0x7f176c06ede0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1780004510 tx=0x7f1780009040 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 msgr2=0x7f1790162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f177800eb90 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 shutdown_connections 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f176c06c920 0x7f176c06ede0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17901f8410 0x7f17901f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790161ce0 0x7f1790162160 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 --2- 172.21.15.46:0/584103363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1774005210 0x7f1790140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.745+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 >> 172.21.15.46:0/584103363 conn(0x7f179006c530 msgr2=0x7f1790073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.746+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 shutdown_connections 2024-04-26T23:43:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.746+0000 7f1795400640 1 -- 172.21.15.46:0/584103363 wait complete. 2024-04-26T23:43:33.762 INFO:tasks.ceph.mds.a.smithi046.stderr: -2238> 2024-04-26T23:43:20.562+0000 7fe441400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:43:33.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:43:33.820 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.957+0000 7f79d8000640 1 Processor -- start 2024-04-26T23:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.957+0000 7f79d8000640 1 -- start start 2024-04-26T23:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.959+0000 7f79d8000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79d015dd60 0x7f79d0163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.959+0000 7f79d8000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79d00b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.959+0000 7f79d8000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d0072880 con 0x7f79d00723e0 2024-04-26T23:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.959+0000 7f79d8000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79d015e140 con 0x7f79d015dd60 2024-04-26T23:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.959+0000 7f79d8000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79d0072b40 con 0x7f79d0071f80 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79d6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f79d00723e0 0x7f79d0167200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55844/0 (socket says 172.21.15.46:55844) 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79d6000640 1 -- 172.21.15.46:0/3795935012 learned_addr learned my addr 172.21.15.46:0/3795935012 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79d5600640 1 --2- 172.21.15.46:0/3795935012 >> v2:172.21.15.142:3300/0 conn(0x7f79d015dd60 0x7f79d0163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79d4c00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79d00b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832178833 0 0) 0x7f79d0072880 con 0x7f79d00723e0 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.960+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79b8003520 con 0x7f79d00723e0 2024-04-26T23:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2645251740 0 0) 0x7f79b8003520 con 0x7f79d00723e0 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 msgr2=0x7f79d00b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79d00b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 >> v2:172.21.15.142:3300/0 conn(0x7f79d015dd60 msgr2=0x7f79d0163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 --2- 172.21.15.46:0/3795935012 >> v2:172.21.15.142:3300/0 conn(0x7f79d015dd60 0x7f79d0163bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d0072e20 con 0x7f79d00723e0 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f79c80019f0 con 0x7f79d00723e0 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 >> v1:172.21.15.46:6789/0 conn(0x7f79d00723e0 legacy=0x7f79d0167200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79b80039b0 0x7f79b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b8004d40 0x7f79b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0072b40 con 0x7f79b80039b0 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79b800b9c0 con 0x7f79b8004d40 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79b800bb40 con 0x7f79d0071f80 2024-04-26T23:43:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79d5600640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79b80039b0 0x7f79b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.961+0000 7f79d4c00640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d5600640 1 -- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 msgr2=0x7f79b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d5600640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79d0071f80 0x7f79b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d5600640 1 -- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b8004d40 msgr2=0x7f79b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d6000640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b8004d40 0x7f79b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d5600640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b8004d40 0x7f79b80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.962+0000 7f79d5600640 1 -- 172.21.15.46:0/3795935012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b8003520 con 0x7f79b80039b0 2024-04-26T23:43:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.963+0000 7f79d5600640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79b80039b0 0x7f79b8003dd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009ac0 tx=0x7f79b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d02d99818479e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.963+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4036070 con 0x7f79b80039b0 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.963+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c4031400 con 0x7f79b80039b0 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.963+0000 7f79cfe00640 1 -- 172.21.15.46:0/3795935012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4009690 con 0x7f79b80039b0 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 -- 172.21.15.46:0/3795935012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79b80039b0 msgr2=0x7f79b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 --2- 172.21.15.46:0/3795935012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79b80039b0 0x7f79b8003dd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009ac0 tx=0x7f79b8005830 comp rx=0 tx=0).stop 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d6000640 1 -- 172.21.15.46:0/3795935012 reap_dead start 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 -- 172.21.15.46:0/3795935012 shutdown_connections 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 -- 172.21.15.46:0/3795935012 >> 172.21.15.46:0/3795935012 conn(0x7f79d006c980 msgr2=0x7f79d00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 -- 172.21.15.46:0/3795935012 shutdown_connections 2024-04-26T23:43:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.964+0000 7f79d8000640 1 -- 172.21.15.46:0/3795935012 wait complete. 2024-04-26T23:43:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.966+0000 7f79d8000640 1 Processor -- start 2024-04-26T23:43:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.966+0000 7f79d8000640 1 -- start start 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b80039b0 0x7f79d0150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 0x7f79d01fd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79d006eca0 con 0x7f79d0160150 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00733e0 con 0x7f79d01fcbb0 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d8000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79d006ee20 con 0x7f79b80039b0 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42902/0 (socket says 172.21.15.46:42902) 2024-04-26T23:43:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d4c00640 1 -- 172.21.15.46:0/1516989870 learned_addr learned my addr 172.21.15.46:0/1516989870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d5600640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b80039b0 0x7f79d0150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.967+0000 7f79d6000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 0x7f79d01fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 -- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b80039b0 msgr2=0x7f79d0150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b80039b0 0x7f79d0150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 -- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 msgr2=0x7f79d01fd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 0x7f79d01fd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 -- 172.21.15.46:0/1516989870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00affa0 con 0x7f79d0160150 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d6000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 0x7f79d01fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d4c00640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f79bc011600 tx=0x7f79bc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc015870 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79bc015a10 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc015dc0 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d01fd7c0 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.968+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d00b0500 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.970+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d0160b10 con 0x7f79d0160150 2024-04-26T23:43:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.970+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79bc00f450 con 0x7f79d0160150 2024-04-26T23:43:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.972+0000 7f79cd600640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 0x7f79b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.972+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6359+0+0 (secure 0 0 0) 0x7f79bc08c6b0 con 0x7f79d0160150 2024-04-26T23:43:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.973+0000 7f79d5600640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 0x7f79b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.974+0000 7f79d5600640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 0x7f79b006ede0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f79c4033000 tx=0x7f79c4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:33.977+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79d00affa0 con 0x7f79d0160150 2024-04-26T23:43:34.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.226+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f79d0069530 con 0x7f79d0160150 2024-04-26T23:43:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.753+0000 7f79cd600640 1 -- 172.21.15.46:0/1516989870 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f79bc019030 con 0x7f79d0160150 2024-04-26T23:43:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.756+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 msgr2=0x7f79b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.756+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 0x7f79b006ede0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f79c4033000 tx=0x7f79c4008030 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.756+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 msgr2=0x7f79d01605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.756+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f79bc011600 tx=0x7f79bc011ac0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 shutdown_connections 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79b006c920 0x7f79b006ede0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d01fcbb0 0x7f79d01fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0160150 0x7f79d01605d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 --2- 172.21.15.46:0/1516989870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b80039b0 0x7f79d0150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 >> 172.21.15.46:0/1516989870 conn(0x7f79d006c980 msgr2=0x7f79d00b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 shutdown_connections 2024-04-26T23:43:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.757+0000 7f79d8000640 1 -- 172.21.15.46:0/1516989870 wait complete. 2024-04-26T23:43:34.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:43:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.988+0000 7f3127a00640 1 Processor -- start 2024-04-26T23:43:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.988+0000 7f3127a00640 1 -- start start 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.990+0000 7f3127a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.990+0000 7f3127a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f31200cf560 0x7f31200b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.990+0000 7f3127a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3120146da0 con 0x7f31200cf9e0 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.990+0000 7f3127a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f31200d0ac0 con 0x7f31200cf560 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.990+0000 7f3127a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f31200cfeb0 con 0x7f31201469c0 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.991+0000 7f3125000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.991+0000 7f3125000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:51956/0 (socket says 172.21.15.46:51956) 2024-04-26T23:43:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.991+0000 7f3125000640 1 -- 172.21.15.46:0/2115008486 learned_addr learned my addr 172.21.15.46:0/2115008486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.991+0000 7f311fe00640 1 --2- 172.21.15.46:0/2115008486 >> v2:172.21.15.142:3300/0 conn(0x7f31200cf560 0x7f31200b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.992+0000 7f3125000640 1 -- 172.21.15.46:0/2115008486 >> v2:172.21.15.142:3300/0 conn(0x7f31200cf560 msgr2=0x7f31200b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.992+0000 7f3125000640 1 --2- 172.21.15.46:0/2115008486 >> v2:172.21.15.142:3300/0 conn(0x7f31200cf560 0x7f31200b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.992+0000 7f3125000640 1 -- 172.21.15.46:0/2115008486 >> v1:172.21.15.142:6789/0 conn(0x7f31200cf9e0 legacy=0x7f312006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.992+0000 7f3125000640 1 -- 172.21.15.46:0/2115008486 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200d0190 con 0x7f31201469c0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.993+0000 7f3125000640 1 --2- 172.21.15.46:0/2115008486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3110009870 tx=0x7f311002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1947ca4f0b47852 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f311f400640 1 -- 172.21.15.46:0/2115008486 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110037070 con 0x7f31201469c0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f311f400640 1 -- 172.21.15.46:0/2115008486 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110032440 con 0x7f31201469c0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f311f400640 1 -- 172.21.15.46:0/2115008486 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110032810 con 0x7f31201469c0 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 -- 172.21.15.46:0/2115008486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 msgr2=0x7f312015d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 --2- 172.21.15.46:0/2115008486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3110009870 tx=0x7f311002a250 comp rx=0 tx=0).stop 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 -- 172.21.15.46:0/2115008486 shutdown_connections 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 --2- 172.21.15.46:0/2115008486 >> v2:172.21.15.142:3300/0 conn(0x7f31200cf560 0x7f31200b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 --2- 172.21.15.46:0/2115008486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31201469c0 0x7f312015d6d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 -- 172.21.15.46:0/2115008486 >> 172.21.15.46:0/2115008486 conn(0x7f312006c770 msgr2=0x7f312006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 -- 172.21.15.46:0/2115008486 shutdown_connections 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.994+0000 7f3127a00640 1 -- 172.21.15.46:0/2115008486 wait complete. 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.996+0000 7f3127a00640 1 Processor -- start 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.996+0000 7f3127a00640 1 -- start start 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31200cf560 0x7f31200b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31201469c0 0x7f31200aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31200737e0 con 0x7f31200cf9e0 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31200d0750 con 0x7f31201469c0 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.997+0000 7f3127a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3120073960 con 0x7f31200cf560 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42912/0 (socket says 172.21.15.46:42912) 2024-04-26T23:43:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 -- 172.21.15.46:0/4242627742 learned_addr learned my addr 172.21.15.46:0/4242627742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f3125000640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31200cf560 0x7f31200b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f3125a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31201469c0 0x7f31200aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 -- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31200cf560 msgr2=0x7f31200b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31200cf560 0x7f31200b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 -- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31201469c0 msgr2=0x7f31200aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31201469c0 0x7f31200aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.998+0000 7f311fe00640 1 -- 172.21.15.46:0/4242627742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200b4e80 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f311fe00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f310800e870 tx=0x7f310800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108017070 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310800c940 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31200b3ea0 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310800cd30 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:34.999+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31200b5d90 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.001+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3120069530 con 0x7f31200cf9e0 2024-04-26T23:43:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.002+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f310801a070 con 0x7f31200cf9e0 2024-04-26T23:43:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.003+0000 7f311cc00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.003+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6359+0+0 (secure 0 0 0) 0x7f310808c480 con 0x7f31200cf9e0 2024-04-26T23:43:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.004+0000 7f3125000640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.005+0000 7f3125000640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 0x7f30fc06edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3110034000 tx=0x7f311002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.007+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3108059fb0 con 0x7f31200cf9e0 2024-04-26T23:43:35.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:35.214+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f312006f700 con 0x7f31200cf9e0 2024-04-26T23:43:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.121+0000 7f311cc00640 1 -- 172.21.15.46:0/4242627742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) ==== 181+0+0 (secure 0 0 0) 0x7f310805e2b0 con 0x7f31200cf9e0 2024-04-26T23:43:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.127+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 msgr2=0x7f30fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.127+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 0x7f30fc06edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3110034000 tx=0x7f311002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.127+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 msgr2=0x7f31200ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.127+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f310800e870 tx=0x7f310800ed30 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 shutdown_connections 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31201469c0 0x7f31200aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31200cf9e0 0x7f31200ae8e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 --2- 172.21.15.46:0/4242627742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31200cf560 0x7f31200b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 >> 172.21.15.46:0/4242627742 conn(0x7f312006c770 msgr2=0x7f312006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 shutdown_connections 2024-04-26T23:43:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.128+0000 7f3127a00640 1 -- 172.21.15.46:0/4242627742 wait complete. 2024-04-26T23:43:36.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:43:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.353+0000 7f1219200640 1 Processor -- start 2024-04-26T23:43:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.353+0000 7f1219200640 1 -- start start 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12141469c0 0x7f121415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f12140cf560 0x7f12140b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12140cfa10 0x7f12140b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f12140d0ac0 con 0x7f12141469c0 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1214146da0 con 0x7f12140cf560 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.354+0000 7f1219200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12140cfeb0 con 0x7f12140cfa10 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12141469c0 0x7f121415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12141469c0 0x7f121415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42922/0 (socket says 172.21.15.46:42922) 2024-04-26T23:43:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 -- 172.21.15.46:0/2175453805 learned_addr learned my addr 172.21.15.46:0/2175453805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 -- 172.21.15.46:0/2175453805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12140cfa10 msgr2=0x7f12140b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 --2- 172.21.15.46:0/2175453805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12140cfa10 0x7f12140b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 -- 172.21.15.46:0/2175453805 >> v2:172.21.15.142:3300/0 conn(0x7f12140cf560 msgr2=0x7f12140b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 --2- 172.21.15.46:0/2175453805 >> v2:172.21.15.142:3300/0 conn(0x7f12140cf560 0x7f12140b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.355+0000 7f1211e00640 1 -- 172.21.15.46:0/2175453805 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140d0190 con 0x7f12141469c0 2024-04-26T23:43:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.356+0000 7f1211e00640 1 --2- 172.21.15.46:0/2175453805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141469c0 0x7f121415d6d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f12140b6330 tx=0x7f120402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86649396509ac0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.356+0000 7f1210a00640 1 -- 172.21.15.46:0/2175453805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204037070 con 0x7f12141469c0 2024-04-26T23:43:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1210a00640 1 -- 172.21.15.46:0/2175453805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204032440 con 0x7f12141469c0 2024-04-26T23:43:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1210a00640 1 -- 172.21.15.46:0/2175453805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204032810 con 0x7f12141469c0 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 -- 172.21.15.46:0/2175453805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141469c0 msgr2=0x7f121415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 --2- 172.21.15.46:0/2175453805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141469c0 0x7f121415d6d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f12140b6330 tx=0x7f120402a1d0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 -- 172.21.15.46:0/2175453805 shutdown_connections 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 --2- 172.21.15.46:0/2175453805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12140cfa10 0x7f12140b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 --2- 172.21.15.46:0/2175453805 >> v2:172.21.15.142:3300/0 conn(0x7f12140cf560 0x7f12140b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 --2- 172.21.15.46:0/2175453805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141469c0 0x7f121415d6d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 -- 172.21.15.46:0/2175453805 >> 172.21.15.46:0/2175453805 conn(0x7f121406c770 msgr2=0x7f121406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.357+0000 7f1219200640 1 -- 172.21.15.46:0/2175453805 shutdown_connections 2024-04-26T23:43:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.358+0000 7f1219200640 1 -- 172.21.15.46:0/2175453805 wait complete. 2024-04-26T23:43:36.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.359+0000 7f1219200640 1 Processor -- start 2024-04-26T23:43:36.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.359+0000 7f1219200640 1 -- start start 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12140cfa10 0x7f12140aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12141469c0 0x7f12140d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f121406e200 con 0x7f12140cf560 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12140d0750 con 0x7f12140cfa10 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.360+0000 7f1219200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f121406e380 con 0x7f12141469c0 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42928/0 (socket says 172.21.15.46:42928) 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 -- 172.21.15.46:0/1532882533 learned_addr learned my addr 172.21.15.46:0/1532882533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1212800640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12141469c0 0x7f12140d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:36.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211400640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12140cfa10 0x7f12140aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 -- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12141469c0 msgr2=0x7f12140d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12141469c0 0x7f12140d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 -- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12140cfa10 msgr2=0x7f12140aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12140cfa10 0x7f12140aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.361+0000 7f1211e00640 1 -- 172.21.15.46:0/1532882533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140ab8a0 con 0x7f12140cf560 2024-04-26T23:43:36.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f1211e00640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f12040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041070 con 0x7f12140cf560 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120403c870 con 0x7f12140cf560 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204037040 con 0x7f12140cf560 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12140abb80 con 0x7f12140cf560 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.362+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12140ac140 con 0x7f12140cf560 2024-04-26T23:43:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.363+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214069530 con 0x7f12140cf560 2024-04-26T23:43:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.364+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1204044070 con 0x7f12140cf560 2024-04-26T23:43:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.365+0000 7f120a000640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 0x7f11f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.365+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5921+0+0 (secure 0 0 0) 0x7f12040ac810 con 0x7f12140cf560 2024-04-26T23:43:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.366+0000 7f1211400640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 0x7f11f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:36.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.367+0000 7f1211400640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 0x7f11f006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1214069fa0 tx=0x7f11fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:36.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.370+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f120407ac70 con 0x7f12140cf560 2024-04-26T23:43:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:36.574+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1214160560 con 0x7f12140cf560 2024-04-26T23:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.132+0000 7f120a000640 1 -- 172.21.15.46:0/1532882533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) ==== 169+0+0 (secure 0 0 0) 0x7f120407a570 con 0x7f12140cf560 2024-04-26T23:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 msgr2=0x7f11f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 0x7f11f006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1214069fa0 tx=0x7f11fc008040 comp rx=0 tx=0).stop 2024-04-26T23:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 msgr2=0x7f121413d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f12040046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 shutdown_connections 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11f006c8f0 0x7f11f006edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12141469c0 0x7f12140d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12140cfa10 0x7f12140aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 --2- 172.21.15.46:0/1532882533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140cf560 0x7f121413d9c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 >> 172.21.15.46:0/1532882533 conn(0x7f121406c770 msgr2=0x7f12141457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.136+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 shutdown_connections 2024-04-26T23:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.137+0000 7f1219200640 1 -- 172.21.15.46:0/1532882533 wait complete. 2024-04-26T23:43:37.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_merge_size 2024-04-26T23:43:37.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.356+0000 7fc41ae00640 1 Processor -- start 2024-04-26T23:43:37.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.357+0000 7fc41ae00640 1 -- start start 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc41ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc41415f840 0x7fc4140d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc41ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc41ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4140d4090 con 0x7fc4140d2320 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc41ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140d3880 con 0x7fc41415f840 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc41ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4140d27b0 con 0x7fc4140d1980 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.358+0000 7fc419a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc41415f840 0x7fc4140d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc41a400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc4140d2320 0x7fc4140cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55860/0 (socket says 172.21.15.46:55860) 2024-04-26T23:43:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc41a400640 1 -- 172.21.15.46:0/1061128402 learned_addr learned my addr 172.21.15.46:0/1061128402 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc419000640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc419a00640 1 -- 172.21.15.46:0/1061128402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 msgr2=0x7fc4140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc419a00640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc419a00640 1 -- 172.21.15.46:0/1061128402 >> v1:172.21.15.46:6789/0 conn(0x7fc4140d2320 legacy=0x7fc4140cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc413e00640 1 -- 172.21.15.46:0/1061128402 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1789137730 0 0) 0x7fc40c001490 con 0x7fc4140d2320 2024-04-26T23:43:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.359+0000 7fc419a00640 1 -- 172.21.15.46:0/1061128402 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140b4020 con 0x7fc41415f840 2024-04-26T23:43:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.360+0000 7fc419a00640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41415f840 0x7fc4140d2e70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc4040097a0 tx=0x7fc40402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa281081b016567 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc413e00640 1 -- 172.21.15.46:0/1061128402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404037070 con 0x7fc41415f840 2024-04-26T23:43:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc413e00640 1 -- 172.21.15.46:0/1061128402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404032440 con 0x7fc41415f840 2024-04-26T23:43:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc413e00640 1 -- 172.21.15.46:0/1061128402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404032810 con 0x7fc41415f840 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 -- 172.21.15.46:0/1061128402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41415f840 msgr2=0x7fc4140d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41415f840 0x7fc4140d2e70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc4040097a0 tx=0x7fc40402a180 comp rx=0 tx=0).stop 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 -- 172.21.15.46:0/1061128402 shutdown_connections 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 --2- 172.21.15.46:0/1061128402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41415f840 0x7fc4140d2e70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 -- 172.21.15.46:0/1061128402 >> 172.21.15.46:0/1061128402 conn(0x7fc4140d4f70 msgr2=0x7fc4140d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 -- 172.21.15.46:0/1061128402 shutdown_connections 2024-04-26T23:43:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.361+0000 7fc41ae00640 1 -- 172.21.15.46:0/1061128402 wait complete. 2024-04-26T23:43:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.363+0000 7fc41ae00640 1 Processor -- start 2024-04-26T23:43:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.363+0000 7fc41ae00640 1 -- start start 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 0x7fc4140cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140b27d0 con 0x7fc4140d2320 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140d3db0 con 0x7fc41415f840 2024-04-26T23:43:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.364+0000 7fc41ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4140d33b0 con 0x7fc4140d1980 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42948/0 (socket says 172.21.15.46:42948) 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 -- 172.21.15.46:0/164339017 learned_addr learned my addr 172.21.15.46:0/164339017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419a00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc41a400640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 0x7fc4140cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 -- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 msgr2=0x7fc4140ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 -- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 msgr2=0x7fc4140cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 0x7fc4140cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419000640 1 -- 172.21.15.46:0/164339017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140baba0 con 0x7fc4140d2320 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc419a00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.365+0000 7fc41a400640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 0x7fc4140cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc419000640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc40000d800 tx=0x7fc40000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000b6c0 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc400019070 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000ba80 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4141f52e0 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.366+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141f58a0 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.367+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4140693b0 con 0x7fc4140d2320 2024-04-26T23:43:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.368+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc40000bc20 con 0x7fc4140d2320 2024-04-26T23:43:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.369+0000 7fc411600640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 0x7fc3f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.369+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fc40008bf90 con 0x7fc4140d2320 2024-04-26T23:43:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.370+0000 7fc419a00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 0x7fc3f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.371+0000 7fc419a00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 0x7fc3f406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc404034000 tx=0x7fc40402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.374+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc40001c360 con 0x7fc4140d2320 2024-04-26T23:43:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.582+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"} v 0) -- 0x7fc41415c9c0 con 0x7fc4140d2320 2024-04-26T23:43:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.588+0000 7fc411600640 1 -- 172.21.15.46:0/164339017 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"}]=0 v20) ==== 101+0+0 (secure 0 0 0) 0x7fc40005a470 con 0x7fc4140d2320 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 msgr2=0x7fc3f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 0x7fc3f406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc404034000 tx=0x7fc40402e000 comp rx=0 tx=0).stop 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 msgr2=0x7fc41415c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc40000d800 tx=0x7fc40000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 shutdown_connections 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f406c8f0 0x7fc3f406edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41415f840 0x7fc4140cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4140d2320 0x7fc41415c480 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 --2- 172.21.15.46:0/164339017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4140d1980 0x7fc4140ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.591+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 >> 172.21.15.46:0/164339017 conn(0x7fc4140d4f70 msgr2=0x7fc41415d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.592+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 shutdown_connections 2024-04-26T23:43:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.592+0000 7fc41ae00640 1 -- 172.21.15.46:0/164339017 wait complete. 2024-04-26T23:43:37.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_bits 2024-04-26T23:43:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.825+0000 7f432ac00640 1 Processor -- start 2024-04-26T23:43:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.826+0000 7f432ac00640 1 -- start start 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f432ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f432415f840 0x7f43240d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f432ac00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43240d24f0 0x7f43240b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f432ac00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43240d4090 con 0x7f43240d1980 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f432ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f43240d3880 con 0x7f432415f840 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f432ac00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f43240b4020 con 0x7f43240d24f0 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f4329800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f432415f840 0x7f43240d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f4329800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f432415f840 0x7f43240d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42958/0 (socket says 172.21.15.46:42958) 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.827+0000 7f4329800640 1 -- 172.21.15.46:0/1417628229 learned_addr learned my addr 172.21.15.46:0/1417628229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.828+0000 7f4329800640 1 -- 172.21.15.46:0/1417628229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43240d24f0 msgr2=0x7f43240b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.828+0000 7f4329800640 1 --2- 172.21.15.46:0/1417628229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43240d24f0 0x7f43240b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.828+0000 7f4329800640 1 -- 172.21.15.46:0/1417628229 >> v1:172.21.15.142:6789/0 conn(0x7f43240d1980 legacy=0x7f43240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.828+0000 7f4329800640 1 -- 172.21.15.46:0/1417628229 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240d3ad0 con 0x7f432415f840 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.829+0000 7f4329800640 1 --2- 172.21.15.46:0/1417628229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432415f840 0x7f43240d2e70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f431402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68297046506ff952 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.829+0000 7f4323e00640 1 -- 172.21.15.46:0/1417628229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314037070 con 0x7f432415f840 2024-04-26T23:43:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.829+0000 7f4323e00640 1 -- 172.21.15.46:0/1417628229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4314032440 con 0x7f432415f840 2024-04-26T23:43:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.829+0000 7f4323e00640 1 -- 172.21.15.46:0/1417628229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314032810 con 0x7f432415f840 2024-04-26T23:43:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 -- 172.21.15.46:0/1417628229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432415f840 msgr2=0x7f43240d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 --2- 172.21.15.46:0/1417628229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432415f840 0x7f43240d2e70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f431402a210 comp rx=0 tx=0).stop 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 -- 172.21.15.46:0/1417628229 shutdown_connections 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 --2- 172.21.15.46:0/1417628229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43240d24f0 0x7f43240b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 --2- 172.21.15.46:0/1417628229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432415f840 0x7f43240d2e70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 -- 172.21.15.46:0/1417628229 >> 172.21.15.46:0/1417628229 conn(0x7f43240d4f70 msgr2=0x7f43240d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 -- 172.21.15.46:0/1417628229 shutdown_connections 2024-04-26T23:43:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.830+0000 7f432ac00640 1 -- 172.21.15.46:0/1417628229 wait complete. 2024-04-26T23:43:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.832+0000 7f432ac00640 1 Processor -- start 2024-04-26T23:43:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.832+0000 7f432ac00640 1 -- start start 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 0x7f43241616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 0x7f43240cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43240d33b0 con 0x7f43240d1980 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43240b27d0 con 0x7f43240d24f0 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432ac00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f43240b3230 con 0x7f432415f840 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f4329800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f4329800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42966/0 (socket says 172.21.15.46:42966) 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f4329800640 1 -- 172.21.15.46:0/2989811594 learned_addr learned my addr 172.21.15.46:0/2989811594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f4328e00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 0x7f43241616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f4329800640 1 -- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 msgr2=0x7f43240cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.833+0000 7f432a200640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 0x7f43240cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4329800640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 0x7f43240cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4329800640 1 -- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 msgr2=0x7f43241616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4329800640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 0x7f43241616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4329800640 1 -- 172.21.15.46:0/2989811594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43241f9dd0 con 0x7f43240d1980 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4328e00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 0x7f43241616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f432a200640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 0x7f43240cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4329800640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f43140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314041070 con 0x7f43240d1980 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431403c870 con 0x7f43240d1980 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314037040 con 0x7f43240d1980 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.834+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43240c8910 con 0x7f43240d1980 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.835+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4324161c20 con 0x7f43240d1980 2024-04-26T23:43:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.836+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43240693b0 con 0x7f43240d1980 2024-04-26T23:43:37.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.837+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4314044070 con 0x7f43240d1980 2024-04-26T23:43:37.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.838+0000 7f4321600640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 0x7f430406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:37.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.838+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f43140ac590 con 0x7f43240d1980 2024-04-26T23:43:37.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.838+0000 7f4328e00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 0x7f430406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:37.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.840+0000 7f4328e00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 0x7f430406ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43240cf180 tx=0x7f4310009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:37.842+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43240d4090 con 0x7f43240d1980 2024-04-26T23:43:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.054+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"} v 0) -- 0x7f432415cca0 con 0x7f43240d1980 2024-04-26T23:43:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.059+0000 7f4321600640 1 -- 172.21.15.46:0/2989811594 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"}]=0 v21) ==== 101+0+0 (secure 0 0 0) 0x7f431407aa70 con 0x7f43240d1980 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 msgr2=0x7f430406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 0x7f430406ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43240cf180 tx=0x7f4310009040 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 msgr2=0x7f43240c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f43140046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 shutdown_connections 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f430406c920 0x7f430406ede0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f432415f840 0x7f43240cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43240d24f0 0x7f43241616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 --2- 172.21.15.46:0/2989811594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43240d1980 0x7f43240c93f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 >> 172.21.15.46:0/2989811594 conn(0x7f43240d4f70 msgr2=0x7f43240d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 shutdown_connections 2024-04-26T23:43:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.062+0000 7f432ac00640 1 -- 172.21.15.46:0/2989811594 wait complete. 2024-04-26T23:43:38.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:43:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.293+0000 7f0c32c00640 1 Processor -- start 2024-04-26T23:43:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.293+0000 7f0c32c00640 1 -- start start 2024-04-26T23:43:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c2c15d730 0x7f0c2c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0c2c0b53b0 0x7f0c2c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 0x7f0c2c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2c06eb90 con 0x7f0c2c15d730 2024-04-26T23:43:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2c0b4310 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c32c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c0b45f0 con 0x7f0c2c15e050 2024-04-26T23:43:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c2be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c2c15d730 0x7f0c2c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c2be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c2c15d730 0x7f0c2c0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42976/0 (socket says 172.21.15.46:42976) 2024-04-26T23:43:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.295+0000 7f0c2be00640 1 -- 172.21.15.46:0/548734872 learned_addr learned my addr 172.21.15.46:0/548734872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c30c00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 0x7f0c2c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2be00640 1 -- 172.21.15.46:0/548734872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 msgr2=0x7f0c2c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2b400640 1 --2- 172.21.15.46:0/548734872 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c0b53b0 0x7f0c2c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2be00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 0x7f0c2c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2be00640 1 -- 172.21.15.46:0/548734872 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c0b53b0 msgr2=0x7f0c2c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2be00640 1 --2- 172.21.15.46:0/548734872 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c0b53b0 0x7f0c2c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c2be00640 1 -- 172.21.15.46:0/548734872 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c0b48d0 con 0x7f0c2c15d730 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.296+0000 7f0c30c00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 0x7f0c2c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.297+0000 7f0c2be00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c15d730 0x7f0c2c0d00d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06ffa0 tx=0x7f0c1c02a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67aa6a99abd10f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.297+0000 7f0c2aa00640 1 -- 172.21.15.46:0/548734872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c036070 con 0x7f0c2c15d730 2024-04-26T23:43:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.297+0000 7f0c2aa00640 1 -- 172.21.15.46:0/548734872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c031600 con 0x7f0c2c15d730 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.297+0000 7f0c2aa00640 1 -- 172.21.15.46:0/548734872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0319d0 con 0x7f0c2c15d730 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 -- 172.21.15.46:0/548734872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c15d730 msgr2=0x7f0c2c0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c15d730 0x7f0c2c0d00d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06ffa0 tx=0x7f0c1c02a690 comp rx=0 tx=0).stop 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 -- 172.21.15.46:0/548734872 shutdown_connections 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15e050 0x7f0c2c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 --2- 172.21.15.46:0/548734872 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c0b53b0 0x7f0c2c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 --2- 172.21.15.46:0/548734872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c15d730 0x7f0c2c0d00d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 -- 172.21.15.46:0/548734872 >> 172.21.15.46:0/548734872 conn(0x7f0c2c06c980 msgr2=0x7f0c2c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 -- 172.21.15.46:0/548734872 shutdown_connections 2024-04-26T23:43:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.298+0000 7f0c32c00640 1 -- 172.21.15.46:0/548734872 wait complete. 2024-04-26T23:43:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.300+0000 7f0c32c00640 1 Processor -- start 2024-04-26T23:43:38.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.301+0000 7f0c32c00640 1 -- start start 2024-04-26T23:43:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 0x7f0c2c145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 0x7f0c2c145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c15e4a0 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c0b4e90 con 0x7f0c2c15e050 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c32c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c0b3ec0 con 0x7f0c2c15d730 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42980/0 (socket says 172.21.15.46:42980) 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.302+0000 7f0c2be00640 1 -- 172.21.15.46:0/3501594208 learned_addr learned my addr 172.21.15.46:0/3501594208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c30c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 0x7f0c2c145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2b400640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 0x7f0c2c145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 -- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 msgr2=0x7f0c2c145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 0x7f0c2c145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 -- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 msgr2=0x7f0c2c145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 0x7f0c2c145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 -- 172.21.15.46:0/3501594208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c140810 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c30c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 0x7f0c2c145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2b400640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 0x7f0c2c145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.303+0000 7f0c2be00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000f80 tx=0x7f0c1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.304+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c036050 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.304+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c042070 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.304+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c009510 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.304+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c140af0 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.304+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c143230 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.305+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2c069530 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.306+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c1c007c70 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.307+0000 7f0c23e00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.307+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f0c1c0acad0 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.308+0000 7f0c2b400640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.309+0000 7f0c2b400640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 0x7f0c0806edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c076420 tx=0x7f0c14008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.313+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0c1c07b050 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.509+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0c2c15ccd0 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.513+0000 7f0c23e00640 1 -- 172.21.15.46:0/3501594208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f0c1c07a950 con 0x7f0c2c0b53b0 2024-04-26T23:43:38.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 msgr2=0x7f0c0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 0x7f0c0806edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c076420 tx=0x7f0c14008040 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 msgr2=0x7f0c2c1f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000f80 tx=0x7f0c1c004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 shutdown_connections 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c15e050 0x7f0c2c145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c15d730 0x7f0c2c145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 --2- 172.21.15.46:0/3501594208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2c0b53b0 0x7f0c2c1f2b00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.516+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 >> 172.21.15.46:0/3501594208 conn(0x7f0c2c06c980 msgr2=0x7f0c2c0b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.517+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 shutdown_connections 2024-04-26T23:43:38.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.517+0000 7f0c32c00640 1 -- 172.21.15.46:0/3501594208 wait complete. 2024-04-26T23:43:38.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_size 2024-04-26T23:43:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.750+0000 7f70fe800640 1 Processor -- start 2024-04-26T23:43:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.750+0000 7f70fe800640 1 -- start start 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.751+0000 7f70fe800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70f80b64c0 0x7f70f80d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70fe800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80b46f0 con 0x7f70f80d3d60 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70fe800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80b49d0 con 0x7f70f8162aa0 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70fe800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80b44a0 con 0x7f70f80b64c0 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70f80b64c0 0x7f70f80d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70f80d3d60 0x7f70f80cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55868/0 (socket says 172.21.15.46:55868) 2024-04-26T23:43:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f6a00640 1 -- 172.21.15.46:0/301086419 learned_addr learned my addr 172.21.15.46:0/301086419 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f7e00640 1 --2- 172.21.15.46:0/301086419 >> v2:172.21.15.46:3300/0 conn(0x7f70f80b64c0 0x7f70f80d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42990/0 (socket says 172.21.15.46:42990) 2024-04-26T23:43:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f7e00640 1 -- 172.21.15.46:0/301086419 >> v1:172.21.15.142:6789/0 conn(0x7f70f8162aa0 legacy=0x7f70f80d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f7e00640 1 -- 172.21.15.46:0/301086419 >> v1:172.21.15.46:6789/0 conn(0x7f70f80d3d60 legacy=0x7f70f80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.752+0000 7f70f7e00640 1 -- 172.21.15.46:0/301086419 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80b4cb0 con 0x7f70f80b64c0 2024-04-26T23:43:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.753+0000 7f70f7e00640 1 --2- 172.21.15.46:0/301086419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80b64c0 0x7f70f80d0390 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009d90 tx=0x7f70f002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98153332cf7e0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.754+0000 7f70f6000640 1 -- 172.21.15.46:0/301086419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0031600 con 0x7f70f80b64c0 2024-04-26T23:43:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.754+0000 7f70f6000640 1 -- 172.21.15.46:0/301086419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f00317a0 con 0x7f70f80b64c0 2024-04-26T23:43:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.754+0000 7f70f6000640 1 -- 172.21.15.46:0/301086419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0031b70 con 0x7f70f80b64c0 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.754+0000 7f70fe800640 1 -- 172.21.15.46:0/301086419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80b64c0 msgr2=0x7f70f80d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.754+0000 7f70fe800640 1 --2- 172.21.15.46:0/301086419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80b64c0 0x7f70f80d0390 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009d90 tx=0x7f70f002a610 comp rx=0 tx=0).stop 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.755+0000 7f70fe800640 1 -- 172.21.15.46:0/301086419 shutdown_connections 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.755+0000 7f70fe800640 1 --2- 172.21.15.46:0/301086419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80b64c0 0x7f70f80d0390 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.755+0000 7f70fe800640 1 -- 172.21.15.46:0/301086419 >> 172.21.15.46:0/301086419 conn(0x7f70f80d5070 msgr2=0x7f70f80d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.755+0000 7f70fe800640 1 -- 172.21.15.46:0/301086419 shutdown_connections 2024-04-26T23:43:38.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.755+0000 7f70fe800640 1 -- 172.21.15.46:0/301086419 wait complete. 2024-04-26T23:43:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.756+0000 7f70fe800640 1 Processor -- start 2024-04-26T23:43:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.757+0000 7f70fe800640 1 -- start start 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f80b64c0 0x7f70f80cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70f8162aa0 0x7f70f80ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80d30f0 con 0x7f70f80d3d60 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80b5020 con 0x7f70f80b64c0 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70fe800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80d5b90 con 0x7f70f8162aa0 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42994/0 (socket says 172.21.15.46:42994) 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 -- 172.21.15.46:0/301046328 learned_addr learned my addr 172.21.15.46:0/301046328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f7400640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f80b64c0 0x7f70f80cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f7e00640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70f8162aa0 0x7f70f80ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 -- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70f8162aa0 msgr2=0x7f70f80ba5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70f8162aa0 0x7f70f80ba5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.758+0000 7f70f6a00640 1 -- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f80b64c0 msgr2=0x7f70f80cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70f6a00640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f80b64c0 0x7f70f80cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70f6a00640 1 -- 172.21.15.46:0/301046328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80c9bb0 con 0x7f70f80d3d60 2024-04-26T23:43:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70f6a00640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c400 tx=0x7f70e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0017070 con 0x7f70f80d3d60 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e0005ce0 con 0x7f70f80d3d60 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00072f0 con 0x7f70f80d3d60 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f80b7c70 con 0x7f70f80d3d60 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.759+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f80b8230 con 0x7f70f80d3d60 2024-04-26T23:43:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.761+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8069530 con 0x7f70f80d3d60 2024-04-26T23:43:38.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.762+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70e001a070 con 0x7f70f80d3d60 2024-04-26T23:43:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.763+0000 7f70ef400640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 0x7f70d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.763+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f70e008af10 con 0x7f70f80d3d60 2024-04-26T23:43:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.763+0000 7f70f7400640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 0x7f70d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.765+0000 7f70f7400640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 0x7f70d406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70e8006fd0 tx=0x7f70e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.767+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70f80c9bb0 con 0x7f70f80d3d60 2024-04-26T23:43:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.971+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"} v 0) -- 0x7f70f80b3880 con 0x7f70f80d3d60 2024-04-26T23:43:38.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.979+0000 7f70ef400640 1 -- 172.21.15.46:0/301046328 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"}]=0 v23) ==== 101+0+0 (secure 0 0 0) 0x7f70e0059360 con 0x7f70f80d3d60 2024-04-26T23:43:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 msgr2=0x7f70d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 0x7f70d406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70e8006fd0 tx=0x7f70e8008040 comp rx=0 tx=0).stop 2024-04-26T23:43:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 msgr2=0x7f70f81578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c400 tx=0x7f70e000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 shutdown_connections 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70d406c8f0 0x7f70d406edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70f8162aa0 0x7f70f80ba5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f80d3d60 0x7f70f81578c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 --2- 172.21.15.46:0/301046328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f80b64c0 0x7f70f80cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 >> 172.21.15.46:0/301046328 conn(0x7f70f80d5070 msgr2=0x7f70f8161b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 shutdown_connections 2024-04-26T23:43:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:38.982+0000 7f70fe800640 1 -- 172.21.15.46:0/301046328 wait complete. 2024-04-26T23:43:39.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags' 2024-04-26T23:43:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.209+0000 7fe54ce00640 1 Processor -- start 2024-04-26T23:43:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.209+0000 7fe54ce00640 1 -- start start 2024-04-26T23:43:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe54ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5480b48a0 0x7fe548162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe54ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5481631d0 0x7fe54806e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe54ce00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5481635b0 con 0x7fe548159950 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe54ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe5480af0c0 con 0x7fe5480b48a0 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe54ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5480af3a0 con 0x7fe5481631d0 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe546a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5480b48a0 0x7fe548162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe546a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5480b48a0 0x7fe548162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42996/0 (socket says 172.21.15.46:42996) 2024-04-26T23:43:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.211+0000 7fe546a00640 1 -- 172.21.15.46:0/406527686 learned_addr learned my addr 172.21.15.46:0/406527686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.212+0000 7fe546a00640 1 -- 172.21.15.46:0/406527686 >> v2:172.21.15.142:3300/0 conn(0x7fe5481631d0 msgr2=0x7fe54806e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.212+0000 7fe546a00640 1 --2- 172.21.15.46:0/406527686 >> v2:172.21.15.142:3300/0 conn(0x7fe5481631d0 0x7fe54806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.212+0000 7fe546a00640 1 -- 172.21.15.46:0/406527686 >> v1:172.21.15.142:6789/0 conn(0x7fe548159950 legacy=0x7fe5480ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.212+0000 7fe546a00640 1 -- 172.21.15.46:0/406527686 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5480af680 con 0x7fe5480b48a0 2024-04-26T23:43:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.213+0000 7fe546a00640 1 --2- 172.21.15.46:0/406527686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548162c90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe548071af0 tx=0x7fe53402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70911cc44bac9a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.213+0000 7fe546000640 1 -- 172.21.15.46:0/406527686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534037070 con 0x7fe5480b48a0 2024-04-26T23:43:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.213+0000 7fe546000640 1 -- 172.21.15.46:0/406527686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe534032440 con 0x7fe5480b48a0 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.213+0000 7fe546000640 1 -- 172.21.15.46:0/406527686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534032810 con 0x7fe5480b48a0 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 -- 172.21.15.46:0/406527686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 msgr2=0x7fe548162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 --2- 172.21.15.46:0/406527686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548162c90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe548071af0 tx=0x7fe53402a230 comp rx=0 tx=0).stop 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 -- 172.21.15.46:0/406527686 shutdown_connections 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 --2- 172.21.15.46:0/406527686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548162c90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 --2- 172.21.15.46:0/406527686 >> v2:172.21.15.142:3300/0 conn(0x7fe5481631d0 0x7fe54806e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 -- 172.21.15.46:0/406527686 >> 172.21.15.46:0/406527686 conn(0x7fe5480b2820 msgr2=0x7fe5480b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 -- 172.21.15.46:0/406527686 shutdown_connections 2024-04-26T23:43:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.214+0000 7fe54ce00640 1 -- 172.21.15.46:0/406527686 wait complete. 2024-04-26T23:43:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.216+0000 7fe54ce00640 1 Processor -- start 2024-04-26T23:43:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.216+0000 7fe54ce00640 1 -- start start 2024-04-26T23:43:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5481631d0 0x7fe548145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe548160400 0x7fe548160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5480b0160 con 0x7fe5480b48a0 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5480afc40 con 0x7fe548160400 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe54ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe54806e960 con 0x7fe5481631d0 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe547400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe547400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43010/0 (socket says 172.21.15.46:43010) 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe547400640 1 -- 172.21.15.46:0/451558312 learned_addr learned my addr 172.21.15.46:0/451558312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.217+0000 7fe547e00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe548160400 0x7fe548160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 -- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5481631d0 msgr2=0x7fe548145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe546a00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5481631d0 0x7fe548145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5481631d0 0x7fe548145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 -- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe548160400 msgr2=0x7fe548160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe548160400 0x7fe548160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 -- 172.21.15.46:0/451558312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54815c8d0 con 0x7fe5480b48a0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe547400640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c440 tx=0x7fe53800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538017070 con 0x7fe5480b48a0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5380035f0 con 0x7fe5480b48a0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380039e0 con 0x7fe5480b48a0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe548145a80 con 0x7fe5480b48a0 2024-04-26T23:43:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.218+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54815c5c0 con 0x7fe5480b48a0 2024-04-26T23:43:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.220+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548069ac0 con 0x7fe5480b48a0 2024-04-26T23:43:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.221+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe538003b80 con 0x7fe5480b48a0 2024-04-26T23:43:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.221+0000 7fe53f400640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.222+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fe53808ae60 con 0x7fe5480b48a0 2024-04-26T23:43:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.222+0000 7fe546a00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.223+0000 7fe546a00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe534034000 tx=0x7fe53402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:39.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.226+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe548069ac0 con 0x7fe5480b48a0 2024-04-26T23:43:39.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:39.357 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:39.496+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]} v 0) -- 0x7fe5480693b0 con 0x7fe5480b48a0 2024-04-26T23:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.148+0000 7fe53f400640 1 -- 172.21.15.46:0/451558312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]}]=0 v320) ==== 151+0+0 (secure 0 0 0) 0x7fe5380592b0 con 0x7fe5480b48a0 2024-04-26T23:43:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 msgr2=0x7fe52806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe534034000 tx=0x7fe53402e000 comp rx=0 tx=0).stop 2024-04-26T23:43:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 msgr2=0x7fe548159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c440 tx=0x7fe53800c900 comp rx=0 tx=0).stop 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 shutdown_connections 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe548160400 0x7fe548160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5481631d0 0x7fe548145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 --2- 172.21.15.46:0/451558312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5480b48a0 0x7fe548159d60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.151+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 >> 172.21.15.46:0/451558312 conn(0x7fe5480b2820 msgr2=0x7fe54815eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.152+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 shutdown_connections 2024-04-26T23:43:40.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.152+0000 7fe54ce00640 1 -- 172.21.15.46:0/451558312 wait complete. 2024-04-26T23:43:40.181 INFO:tasks.cephfs_test_runner:That a directory is merged during scrub. ... ok 2024-04-26T23:43:40.182 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:43:40.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.391+0000 7fc320c00640 1 Processor -- start 2024-04-26T23:43:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.391+0000 7fc320c00640 1 -- start start 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.393+0000 7fc320c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc31c1466b0 0x7fc31c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.393+0000 7fc320c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc31c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.393+0000 7fc320c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc31c073ad0 con 0x7fc31c0720e0 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.393+0000 7fc320c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc31c146a90 con 0x7fc31c1466b0 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.393+0000 7fc320c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c073d70 con 0x7fc31c160910 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc313e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc31c0720e0 0x7fc31c0cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55880/0 (socket says 172.21.15.46:55880) 2024-04-26T23:43:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc313e00640 1 -- 172.21.15.46:0/440709698 learned_addr learned my addr 172.21.15.46:0/440709698 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc31b400640 1 --2- 172.21.15.46:0/440709698 >> v2:172.21.15.142:3300/0 conn(0x7fc31c1466b0 0x7fc31c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc31be00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc31c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367252259 0 0) 0x7fc31c073ad0 con 0x7fc31c0720e0 2024-04-26T23:43:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.394+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2fc003520 con 0x7fc31c0720e0 2024-04-26T23:43:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266672978 0 0) 0x7fc2fc003520 con 0x7fc31c0720e0 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 msgr2=0x7fc31c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc31c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 >> v2:172.21.15.142:3300/0 conn(0x7fc31c1466b0 msgr2=0x7fc31c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 --2- 172.21.15.46:0/440709698 >> v2:172.21.15.142:3300/0 conn(0x7fc31c1466b0 0x7fc31c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c074050 con 0x7fc31c0720e0 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc304001670 con 0x7fc31c0720e0 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 >> v1:172.21.15.46:6789/0 conn(0x7fc31c0720e0 legacy=0x7fc31c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc2fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc003f80 0x7fc2fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc2fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c146a90 con 0x7fc2fc004d40 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c073d70 con 0x7fc2fc003f80 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc00b9c0 con 0x7fc31c160910 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc2fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31b400640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc2fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 -- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 msgr2=0x7fc2fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c160910 0x7fc2fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 -- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc003f80 msgr2=0x7fc2fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc003f80 0x7fc2fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.395+0000 7fc31be00640 1 -- 172.21.15.46:0/440709698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc003520 con 0x7fc2fc004d40 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc31be00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc2fc0051c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e170 tx=0x7fc2fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb71ae12daf2d7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314034020 con 0x7fc2fc004d40 2024-04-26T23:43:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3140305f0 con 0x7fc2fc004d40 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc31aa00640 1 -- 172.21.15.46:0/440709698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314009c10 con 0x7fc2fc004d40 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc320c00640 1 -- 172.21.15.46:0/440709698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 msgr2=0x7fc2fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc320c00640 1 --2- 172.21.15.46:0/440709698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc2fc0051c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e170 tx=0x7fc2fc005830 comp rx=0 tx=0).stop 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc31be00640 1 -- 172.21.15.46:0/440709698 reap_dead start 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.397+0000 7fc320c00640 1 -- 172.21.15.46:0/440709698 shutdown_connections 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.398+0000 7fc320c00640 1 -- 172.21.15.46:0/440709698 >> 172.21.15.46:0/440709698 conn(0x7fc31c0b26d0 msgr2=0x7fc31c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.398+0000 7fc320c00640 1 -- 172.21.15.46:0/440709698 shutdown_connections 2024-04-26T23:43:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.398+0000 7fc320c00640 1 -- 172.21.15.46:0/440709698 wait complete. 2024-04-26T23:43:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.399+0000 7fc320c00640 1 Processor -- start 2024-04-26T23:43:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 -- start start 2024-04-26T23:43:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c06e750 0x7fc31c06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc31c0aa4d0 0x7fc31c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c06e3c0 con 0x7fc2fc004d40 2024-04-26T23:43:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c0cb470 con 0x7fc31c0aa4d0 2024-04-26T23:43:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.400+0000 7fc320c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c160da0 con 0x7fc31c06e750 2024-04-26T23:43:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43020/0 (socket says 172.21.15.46:43020) 2024-04-26T23:43:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 -- 172.21.15.46:0/1708779732 learned_addr learned my addr 172.21.15.46:0/1708779732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 -- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c06e750 msgr2=0x7fc31c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c06e750 0x7fc31c06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 -- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc31c0aa4d0 msgr2=0x7fc31c0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc31c0aa4d0 0x7fc31c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.401+0000 7fc31b400640 1 -- 172.21.15.46:0/1708779732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c155a90 con 0x7fc2fc004d40 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc31b400640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0116d0 tx=0x7fc30c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00f940 con 0x7fc2fc004d40 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c00fae0 con 0x7fc2fc004d40 2024-04-26T23:43:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00fe90 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c0ab1c0 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.402+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c0ab780 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.403+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c0693b0 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.405+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc30c015460 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.406+0000 7fc313400640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 0x7fc2f806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.406+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fc30c08c4b0 con 0x7fc2fc004d40 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.407+0000 7fc313e00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 0x7fc2f806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.408+0000 7fc313e00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 0x7fc2f806ef80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0b51d0 tx=0x7fc3040015b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.411+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc30c019030 con 0x7fc2fc004d40 2024-04-26T23:43:40.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.617+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc31c145df0 con 0x7fc2fc004d40 2024-04-26T23:43:40.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.619+0000 7fc313400640 1 -- 172.21.15.46:0/1708779732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+13520 (secure 0 0 0) 0x7fc30c08ae20 con 0x7fc2fc004d40 2024-04-26T23:43:40.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:40.623 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":72,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:37.122346+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2393654178":"2024-04-27T23:43:33.739094+0000","172.21.15.46:6833/2393654178":"2024-04-27T23:43:33.739094+0000","172.21.15.46:6835/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.46:6834/1818830260":"2024-04-27T23:43:14.530995+0000","172.21.15.142:6835/2927770154":"2024-04-27T23:43:14.513761+0000","172.21.15.142:6834/2927770154":"2024-04-27T23:43:14.513761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 msgr2=0x7fc2f806ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 0x7fc2f806ef80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0b51d0 tx=0x7fc3040015b0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 msgr2=0x7fc31c155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0116d0 tx=0x7fc30c011b90 comp rx=0 tx=0).stop 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 shutdown_connections 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2f806cac0 0x7fc2f806ef80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc31c0aa4d0 0x7fc31c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31c06e750 0x7fc31c06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 --2- 172.21.15.46:0/1708779732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc004d40 0x7fc31c155550 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.621+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 >> 172.21.15.46:0/1708779732 conn(0x7fc31c0b26d0 msgr2=0x7fc31c06f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.622+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 shutdown_connections 2024-04-26T23:43:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.622+0000 7fc320c00640 1 -- 172.21.15.46:0/1708779732 wait complete. 2024-04-26T23:43:40.645 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:43:40.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:43:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.862+0000 7f8ad1400640 1 Processor -- start 2024-04-26T23:43:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.862+0000 7f8ad1400640 1 -- start start 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.863+0000 7f8ad1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8acc0b4b50 0x7f8acc073d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.863+0000 7f8ad1400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc0ce7d0 con 0x7f8acc15cec0 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.863+0000 7f8ad1400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc0ceab0 con 0x7f8acc0cdd00 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.863+0000 7f8ad1400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc0ced90 con 0x7f8acc0b4b50 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8ad0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8acc0b4b50 0x7f8acc073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8acbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8acc15cec0 0x7f8acc0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55896/0 (socket says 172.21.15.46:55896) 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8ad0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8acc0b4b50 0x7f8acc073d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43036/0 (socket says 172.21.15.46:43036) 2024-04-26T23:43:40.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8acbe00640 1 -- 172.21.15.46:0/1416286994 learned_addr learned my addr 172.21.15.46:0/1416286994 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8acb400640 1 -- 172.21.15.46:0/1416286994 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3582426881 0 0) 0x7f8acc0ce7d0 con 0x7f8acc15cec0 2024-04-26T23:43:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1416286994 >> v1:172.21.15.142:6789/0 conn(0x7f8acc0cdd00 legacy=0x7f8acc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1416286994 >> v1:172.21.15.46:6789/0 conn(0x7f8acc15cec0 legacy=0x7f8acc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.864+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1416286994 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0cf070 con 0x7f8acc0b4b50 2024-04-26T23:43:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.866+0000 7f8ad0a00640 1 --2- 172.21.15.46:0/1416286994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc0b4b50 0x7f8acc073d00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24548357bfe0a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.866+0000 7f8acb400640 1 -- 172.21.15.46:0/1416286994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007740 con 0x7f8acc0b4b50 2024-04-26T23:43:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.866+0000 7f8acb400640 1 -- 172.21.15.46:0/1416286994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac40078e0 con 0x7f8acc0b4b50 2024-04-26T23:43:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.866+0000 7f8acb400640 1 -- 172.21.15.46:0/1416286994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007cb0 con 0x7f8acc0b4b50 2024-04-26T23:43:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 -- 172.21.15.46:0/1416286994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc0b4b50 msgr2=0x7f8acc073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1416286994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc0b4b50 0x7f8acc073d00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 -- 172.21.15.46:0/1416286994 shutdown_connections 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1416286994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc0b4b50 0x7f8acc073d00 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 -- 172.21.15.46:0/1416286994 >> 172.21.15.46:0/1416286994 conn(0x7f8acc0b27b0 msgr2=0x7f8acc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 -- 172.21.15.46:0/1416286994 shutdown_connections 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.867+0000 7f8ad1400640 1 -- 172.21.15.46:0/1416286994 wait complete. 2024-04-26T23:43:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.869+0000 7f8ad1400640 1 Processor -- start 2024-04-26T23:43:40.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.869+0000 7f8ad1400640 1 -- start start 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8acc0b4b50 0x7f8acc157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8acc0cdd00 0x7f8acc0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc167b00 con 0x7f8acc15cec0 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc15d2a0 con 0x7f8acc0b4b50 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad1400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc167c80 con 0x7f8acc0cdd00 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43038/0 (socket says 172.21.15.46:43038) 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1381687220 learned_addr learned my addr 172.21.15.46:0/1381687220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:40.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8acbe00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8acc0b4b50 0x7f8acc157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ac3400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8acc0cdd00 0x7f8acc0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8acc0cdd00 msgr2=0x7f8acc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.870+0000 7f8ad0a00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8acc0cdd00 0x7f8acc0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8acc0b4b50 msgr2=0x7f8acc157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad0a00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8acc0b4b50 0x7f8acc157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad0a00640 1 -- 172.21.15.46:0/1381687220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0aa610 con 0x7f8acc15cec0 2024-04-26T23:43:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad0a00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4040070 con 0x7f8acc15cec0 2024-04-26T23:43:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4031be0 con 0x7f8acc15cec0 2024-04-26T23:43:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac403c420 con 0x7f8acc15cec0 2024-04-26T23:43:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc0aa8f0 con 0x7f8acc15cec0 2024-04-26T23:43:40.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.871+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc0aaeb0 con 0x7f8acc15cec0 2024-04-26T23:43:40.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.873+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc0ab3b0 con 0x7f8acc15cec0 2024-04-26T23:43:40.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.874+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ac4039040 con 0x7f8acc15cec0 2024-04-26T23:43:40.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.875+0000 7f8ac8c00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.875+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f8ac40acfa0 con 0x7f8acc15cec0 2024-04-26T23:43:40.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.875+0000 7f8acbe00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.877+0000 7f8acbe00640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 0x7f8aa806edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8abc000f60 tx=0x7f8abc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:40.879+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8acc0ab3b0 con 0x7f8acc15cec0 2024-04-26T23:43:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.088+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8acc0693b0 con 0x7f8acc15cec0 2024-04-26T23:43:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.089+0000 7f8ac8c00640 1 -- 172.21.15.46:0/1381687220 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8ac4038350 con 0x7f8acc15cec0 2024-04-26T23:43:41.095 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.091+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 msgr2=0x7f8aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 0x7f8aa806edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8abc000f60 tx=0x7f8abc009040 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 msgr2=0x7f8acc0b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac4004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 shutdown_connections 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8acc15cec0 0x7f8acc0b0d30 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8acc0cdd00 0x7f8acc0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 --2- 172.21.15.46:0/1381687220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8acc0b4b50 0x7f8acc157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 >> 172.21.15.46:0/1381687220 conn(0x7f8acc0b27b0 msgr2=0x7f8acc0730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 shutdown_connections 2024-04-26T23:43:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.092+0000 7f8ad1400640 1 -- 172.21.15.46:0/1381687220 wait complete. 2024-04-26T23:43:41.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-26T23:43:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.322+0000 7fe9df200640 1 Processor -- start 2024-04-26T23:43:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.323+0000 7fe9df200640 1 -- start start 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.324+0000 7fe9df200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9e00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.324+0000 7fe9df200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e00b41b0 con 0x7fe9e006f8f0 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.324+0000 7fe9df200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e00b4490 con 0x7fe9e015bf50 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.324+0000 7fe9df200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e015c330 con 0x7fe9e0073340 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9dd400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9e006f8f0 0x7fe9e0072c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55898/0 (socket says 172.21.15.46:55898) 2024-04-26T23:43:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9dd400640 1 -- 172.21.15.46:0/3650435200 learned_addr learned my addr 172.21.15.46:0/3650435200 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810612204 0 0) 0x7fe9e00b41b0 con 0x7fe9e006f8f0 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9c0003520 con 0x7fe9e006f8f0 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9de800640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9e00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.325+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1422033415 0 0) 0x7fe9c0003520 con 0x7fe9e006f8f0 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 msgr2=0x7fe9e00b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9e00b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 >> v1:172.21.15.142:6789/0 conn(0x7fe9e015bf50 legacy=0x7fe9e00ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e00b4770 con 0x7fe9e006f8f0 2024-04-26T23:43:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe9c80019f0 con 0x7fe9e006f8f0 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 >> v1:172.21.15.46:6789/0 conn(0x7fe9e006f8f0 legacy=0x7fe9e0072c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9c00039e0 0x7fe9c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c0004800 0x7fe9c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9c0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e015c330 con 0x7fe9c00039e0 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c000dd90 con 0x7fe9c0004800 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c000df90 con 0x7fe9e0073340 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dde00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9c00039e0 0x7fe9c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9de800640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9c0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.326+0000 7fe9dd400640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c0004800 0x7fe9c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9dde00640 1 -- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 msgr2=0x7fe9c0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9dde00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9c0005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9dde00640 1 -- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c0004800 msgr2=0x7fe9c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9dde00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c0004800 0x7fe9c0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9dde00640 1 -- 172.21.15.46:0/3650435200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0003520 con 0x7fe9c00039e0 2024-04-26T23:43:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.327+0000 7fe9de800640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e0073340 0x7fe9c0005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.328+0000 7fe9dde00640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9c00039e0 0x7fe9c0003e00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001750 tx=0x7fe9c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474c21409109c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.328+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0001370 con 0x7fe9c00039e0 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.328+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d00052d0 con 0x7fe9c00039e0 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.328+0000 7fe9dca00640 1 -- 172.21.15.46:0/3650435200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d002abe0 con 0x7fe9c00039e0 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 -- 172.21.15.46:0/3650435200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9c00039e0 msgr2=0x7fe9c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 --2- 172.21.15.46:0/3650435200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9c00039e0 0x7fe9c0003e00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001750 tx=0x7fe9c0005830 comp rx=0 tx=0).stop 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9de800640 1 -- 172.21.15.46:0/3650435200 reap_dead start 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 -- 172.21.15.46:0/3650435200 shutdown_connections 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 -- 172.21.15.46:0/3650435200 >> 172.21.15.46:0/3650435200 conn(0x7fe9e00b2820 msgr2=0x7fe9e00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 -- 172.21.15.46:0/3650435200 shutdown_connections 2024-04-26T23:43:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.329+0000 7fe9df200640 1 -- 172.21.15.46:0/3650435200 wait complete. 2024-04-26T23:43:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.331+0000 7fe9df200640 1 Processor -- start 2024-04-26T23:43:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.331+0000 7fe9df200640 1 -- start start 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 0x7fe9e0149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e014fd00 0x7fe9e00c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e00b3b20 con 0x7fe9e01f7f70 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e00739f0 con 0x7fe9c00039e0 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9df200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e00b3ca0 con 0x7fe9e014fd00 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43058/0 (socket says 172.21.15.46:43058) 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.332+0000 7fe9dd400640 1 -- 172.21.15.46:0/1275608591 learned_addr learned my addr 172.21.15.46:0/1275608591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dde00640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 0x7fe9e0149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9de800640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e014fd00 0x7fe9e00c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 -- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e014fd00 msgr2=0x7fe9e00c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e014fd00 0x7fe9e00c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 -- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 msgr2=0x7fe9e0149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 0x7fe9e0149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 -- 172.21.15.46:0/1275608591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e01f8400 con 0x7fe9e01f7f70 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dde00640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 0x7fe9e0149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9dd400640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8002050 tx=0x7fe9c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8019070 con 0x7fe9e01f7f70 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c8004b60 con 0x7fe9e01f7f70 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80070d0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.333+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e01f86e0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.334+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e01f8ca0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.335+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e00693b0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.336+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9c801c070 con 0x7fe9e01f7f70 2024-04-26T23:43:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.337+0000 7fe9d6000640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 0x7fe9bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.337+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fe9c808bbd0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.337+0000 7fe9dde00640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 0x7fe9bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.338+0000 7fe9dde00640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 0x7fe9bc06edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0032000 tx=0x7fe9d002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.341+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9c805a0b0 con 0x7fe9e01f7f70 2024-04-26T23:43:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:41.619+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) -- 0x7fe9e015d0e0 con 0x7fe9e01f7f70 2024-04-26T23:43:42.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.161+0000 7fe9d6000640 1 -- 172.21.15.46:0/1275608591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v322) ==== 146+0+0 (secure 0 0 0) 0x7fe9c80599b0 con 0x7fe9e01f7f70 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 msgr2=0x7fe9bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 0x7fe9bc06edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0032000 tx=0x7fe9d002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 msgr2=0x7fe9e014a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8002050 tx=0x7fe9c800b4a0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 shutdown_connections 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9bc06c8f0 0x7fe9bc06edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9e014fd00 0x7fe9e00c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e01f7f70 0x7fe9e014a230 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 --2- 172.21.15.46:0/1275608591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c00039e0 0x7fe9e0149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 >> 172.21.15.46:0/1275608591 conn(0x7fe9e00b2820 msgr2=0x7fe9e0072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:42.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 shutdown_connections 2024-04-26T23:43:42.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.165+0000 7fe9df200640 1 -- 172.21.15.46:0/1275608591 wait complete. 2024-04-26T23:43:42.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:43:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.409+0000 7fae10a00640 1 Processor -- start 2024-04-26T23:43:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.410+0000 7fae10a00640 1 -- start start 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.411+0000 7fae10a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fae0c06f4b0 0x7fae0c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.411+0000 7fae10a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae0c0b4320 0x7fae0c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.411+0000 7fae10a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0c0b4b30 con 0x7fae0c0d04c0 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.411+0000 7fae10a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fae0c0b4e10 con 0x7fae0c0b4320 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.411+0000 7fae10a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fae0c0b50f0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae09e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fae0c0d04c0 0x7fae0c06eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55218/0 (socket says 172.21.15.46:55218) 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae09e00640 1 -- 172.21.15.46:0/97800739 learned_addr learned my addr 172.21.15.46:0/97800739 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:42.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae0a800640 1 --2- 172.21.15.46:0/97800739 >> v2:172.21.15.46:3300/0 conn(0x7fae0c0b4320 0x7fae0c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:42.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae0a800640 1 -- 172.21.15.46:0/97800739 >> v2:172.21.15.142:3300/0 conn(0x7fae0c06f4b0 msgr2=0x7fae0c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:42.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae0a800640 1 --2- 172.21.15.46:0/97800739 >> v2:172.21.15.142:3300/0 conn(0x7fae0c06f4b0 0x7fae0c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae0a800640 1 -- 172.21.15.46:0/97800739 >> v1:172.21.15.46:6789/0 conn(0x7fae0c0d04c0 legacy=0x7fae0c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.412+0000 7fae0a800640 1 -- 172.21.15.46:0/97800739 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c0b53d0 con 0x7fae0c0b4320 2024-04-26T23:43:42.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.413+0000 7fae0a800640 1 --2- 172.21.15.46:0/97800739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c0b4320 0x7fae0c0ce470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b62a0 tx=0x7fae0402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64feff6daea2eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:42.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae09400640 1 -- 172.21.15.46:0/97800739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04007740 con 0x7fae0c0b4320 2024-04-26T23:43:42.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae09400640 1 -- 172.21.15.46:0/97800739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae040078e0 con 0x7fae0c0b4320 2024-04-26T23:43:42.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae09400640 1 -- 172.21.15.46:0/97800739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04007cb0 con 0x7fae0c0b4320 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 -- 172.21.15.46:0/97800739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c0b4320 msgr2=0x7fae0c0ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 --2- 172.21.15.46:0/97800739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c0b4320 0x7fae0c0ce470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b62a0 tx=0x7fae0402a5f0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 -- 172.21.15.46:0/97800739 shutdown_connections 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 --2- 172.21.15.46:0/97800739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c0b4320 0x7fae0c0ce470 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 --2- 172.21.15.46:0/97800739 >> v2:172.21.15.142:3300/0 conn(0x7fae0c06f4b0 0x7fae0c0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.414+0000 7fae10a00640 1 -- 172.21.15.46:0/97800739 >> 172.21.15.46:0/97800739 conn(0x7fae0c06c980 msgr2=0x7fae0c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.415+0000 7fae10a00640 1 -- 172.21.15.46:0/97800739 shutdown_connections 2024-04-26T23:43:42.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.415+0000 7fae10a00640 1 -- 172.21.15.46:0/97800739 wait complete. 2024-04-26T23:43:42.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.416+0000 7fae10a00640 1 Processor -- start 2024-04-26T23:43:42.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.417+0000 7fae10a00640 1 -- start start 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae0c0b4320 0x7fae0c1381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 0x7fae0c1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0b4910 con 0x7fae0c06f4b0 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0b4790 con 0x7fae0c0b4320 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae10a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c0ce9b0 con 0x7fae0c0d04c0 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37978/0 (socket says 172.21.15.46:37978) 2024-04-26T23:43:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae09e00640 1 -- 172.21.15.46:0/2544131425 learned_addr learned my addr 172.21.15.46:0/2544131425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae03e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae0c0b4320 0x7fae0c1381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae0a800640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 0x7fae0c1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.418+0000 7fae09e00640 1 -- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 msgr2=0x7fae0c1f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae09e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 0x7fae0c1f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae09e00640 1 -- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae0c0b4320 msgr2=0x7fae0c1381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae09e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae0c0b4320 0x7fae0c1381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae09e00640 1 -- 172.21.15.46:0/2544131425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c15cbe0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae0a800640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 0x7fae0c1f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae09e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c3c0 tx=0x7fadfc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc017070 con 0x7fae0c06f4b0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc005ce0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c15c900 con 0x7fae0c06f4b0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0072f0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.419+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c155610 con 0x7fae0c06f4b0 2024-04-26T23:43:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.421+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c069530 con 0x7fae0c06f4b0 2024-04-26T23:43:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.422+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadfc01a070 con 0x7fae0c06f4b0 2024-04-26T23:43:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.422+0000 7fae02000640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 0x7fade806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.423+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fadfc08ae80 con 0x7fae0c06f4b0 2024-04-26T23:43:42.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.423+0000 7fae03e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 0x7fade806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.425+0000 7fae03e00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 0x7fade806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fae0c13b080 tx=0x7fadf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:42.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.428+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadfc0592d0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.637+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fae0c13f430 con 0x7fae0c06f4b0 2024-04-26T23:43:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.644+0000 7fae02000640 1 -- 172.21.15.46:0/2544131425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fadfc058bd0 con 0x7fae0c06f4b0 2024-04-26T23:43:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 msgr2=0x7fade806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 0x7fade806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fae0c13b080 tx=0x7fadf4008040 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 msgr2=0x7fae0c155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c3c0 tx=0x7fadfc00c880 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 shutdown_connections 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fade806c8f0 0x7fade806edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae0c0d04c0 0x7fae0c1f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae0c0b4320 0x7fae0c1381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 --2- 172.21.15.46:0/2544131425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae0c06f4b0 0x7fae0c155e10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 >> 172.21.15.46:0/2544131425 conn(0x7fae0c06c980 msgr2=0x7fae0c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 shutdown_connections 2024-04-26T23:43:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:42.648+0000 7fae10a00640 1 -- 172.21.15.46:0/2544131425 wait complete. 2024-04-26T23:43:42.672 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:43:42.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:43:42.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:43:42.891 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:43:42.891 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:42.891 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:42.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:43:42.953 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:43:42.953 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:43.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:43:43.010 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:43.010 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:43.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:43:43.071 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:43:43.071 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:43.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:43:43.129 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:43:43.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.345+0000 7fa3f3e00640 1 Processor -- start 2024-04-26T23:43:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.345+0000 7fa3f3e00640 1 -- start start 2024-04-26T23:43:43.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f414c6c0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f40cd260 con 0x7fa3f40cd9f0 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f40cb8b0 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37990/0 (socket says 172.21.15.46:37990) 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f2000640 1 -- 172.21.15.46:0/1291747698 learned_addr learned my addr 172.21.15.46:0/1291747698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.347+0000 7fa3f2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3f414c2e0 0x7fa3f40cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55222/0 (socket says 172.21.15.46:55222) 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.348+0000 7fa3f2000640 1 -- 172.21.15.46:0/1291747698 >> v1:172.21.15.142:6789/0 conn(0x7fa3f40cd9f0 legacy=0x7fa3f40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.348+0000 7fa3f2000640 1 -- 172.21.15.46:0/1291747698 >> v1:172.21.15.46:6789/0 conn(0x7fa3f414c2e0 legacy=0x7fa3f40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.348+0000 7fa3f2000640 1 -- 172.21.15.46:0/1291747698 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f4167e10 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.349+0000 7fa3f2000640 1 --2- 172.21.15.46:0/1291747698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009870 tx=0x7fa3e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aded7d7a4ddeb482 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.349+0000 7fa3f1600640 1 -- 172.21.15.46:0/1291747698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0037070 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.349+0000 7fa3f1600640 1 -- 172.21.15.46:0/1291747698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e0032440 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.349+0000 7fa3f1600640 1 -- 172.21.15.46:0/1291747698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0032810 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 -- 172.21.15.46:0/1291747698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f40ccdd0 msgr2=0x7fa3f40cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/1291747698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009870 tx=0x7fa3e002a270 comp rx=0 tx=0).stop 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 -- 172.21.15.46:0/1291747698 shutdown_connections 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/1291747698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cd4b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 -- 172.21.15.46:0/1291747698 >> 172.21.15.46:0/1291747698 conn(0x7fa3f40b26d0 msgr2=0x7fa3f40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 -- 172.21.15.46:0/1291747698 shutdown_connections 2024-04-26T23:43:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.350+0000 7fa3f3e00640 1 -- 172.21.15.46:0/1291747698 wait complete. 2024-04-26T23:43:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.352+0000 7fa3f3e00640 1 Processor -- start 2024-04-26T23:43:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.352+0000 7fa3f3e00640 1 -- start start 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.353+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.353+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3f40cd9f0 0x7fa3f40b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4167330 con 0x7fa3f414c2e0 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40cde20 con 0x7fa3f40ccdd0 2024-04-26T23:43:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f4167610 con 0x7fa3f40cd9f0 2024-04-26T23:43:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37996/0 (socket says 172.21.15.46:37996) 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f3400640 1 -- 172.21.15.46:0/363604838 learned_addr learned my addr 172.21.15.46:0/363604838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f2a00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.354+0000 7fa3f2000640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3f40cd9f0 0x7fa3f40b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 -- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3f40cd9f0 msgr2=0x7fa3f40b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3f40cd9f0 0x7fa3f40b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 -- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 msgr2=0x7fa3f40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 -- 172.21.15.46:0/363604838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41fd080 con 0x7fa3f414c2e0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f2a00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3400640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c730 tx=0x7fa3ec00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec016020 con 0x7fa3f414c2e0 2024-04-26T23:43:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ec006d50 con 0x7fa3f414c2e0 2024-04-26T23:43:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003bb0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.355+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41fd360 con 0x7fa3f414c2e0 2024-04-26T23:43:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.356+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41fd920 con 0x7fa3f414c2e0 2024-04-26T23:43:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.357+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f40693b0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.358+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3ec0050d0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.359+0000 7fa3eaa00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.360+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fa3ec08b6c0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.360+0000 7fa3f2a00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.361+0000 7fa3f2a00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 0x7fa3d406edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007fd0 tx=0x7fa3e4007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.366+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3ec020020 con 0x7fa3f414c2e0 2024-04-26T23:43:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.610+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f406fe50 con 0x7fa3f414c2e0 2024-04-26T23:43:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.611+0000 7fa3eaa00640 1 -- 172.21.15.46:0/363604838 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+3875 (secure 0 0 0) 0x7fa3ec059ba0 con 0x7fa3f414c2e0 2024-04-26T23:43:43.615 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3197602523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3197602523},{"type":"v1","addr":"172.21.15.142:6837","nonce":3197602523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1019371491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1019371491},{"type":"v1","addr":"172.21.15.142:6836","nonce":1019371491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3292035285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3292035285},{"type":"v1","addr":"172.21.15.46:6834","nonce":3292035285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":7044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3135497724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3135497724},{"type":"v1","addr":"172.21.15.46:6835","nonce":3135497724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[]} 2024-04-26T23:43:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 52 2024-04-26T23:43:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.613+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 msgr2=0x7fa3d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.613+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 0x7fa3d406edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007fd0 tx=0x7fa3e4007d40 comp rx=0 tx=0).stop 2024-04-26T23:43:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.613+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 msgr2=0x7fa3f40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.613+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c730 tx=0x7fa3ec00cb30 comp rx=0 tx=0).stop 2024-04-26T23:43:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 shutdown_connections 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3f414c2e0 0x7fa3f40b6170 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3f40cd9f0 0x7fa3f40b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 --2- 172.21.15.46:0/363604838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f40ccdd0 0x7fa3f40cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 >> 172.21.15.46:0/363604838 conn(0x7fa3f40b26d0 msgr2=0x7fa3f40b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 shutdown_connections 2024-04-26T23:43:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.614+0000 7fa3f3e00640 1 -- 172.21.15.46:0/363604838 wait complete. 2024-04-26T23:43:43.638 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:43:43.638 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:43:43.638 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:43:43.638 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:43.639 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:43:43.639 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:43:43.639 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:43:43.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:43.639 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:43:43.639 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:43:43.639 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:43:43.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:43.640 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:43:43.640 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:43:43.640 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:43:43.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:43:43.656 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:43:43.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:43:43.658 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:43:43.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:43:43.660 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:43:43.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:43:43.661 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:43:43.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:43:43.703 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:43:43.703 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:43.706 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:43:43.706 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:43.709 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:43:43.709 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:43.709 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:43:43.710 DEBUG:teuthology.parallel:result is None 2024-04-26T23:43:43.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:43:43.854 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:43:43.855 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:43:43.859 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:43:43.860 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:43:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.886+0000 7f3da8000640 1 Processor -- start 2024-04-26T23:43:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.886+0000 7f3da8000640 1 -- start start 2024-04-26T23:43:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da8000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3da0160bd0 0x7f3da0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da8000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da00732c0 0x7f3da0073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da8000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da0161940 con 0x7f3da006e510 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da8000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3da0071f80 con 0x7f3da0160bd0 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da8000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3da0072260 con 0x7f3da00732c0 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.887+0000 7f3da4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3da0160bd0 0x7f3da0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3da0160bd0 0x7f3da0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38032/0 (socket says 172.21.15.46:38032) 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da5600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3da006e510 0x7f3da006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55244/0 (socket says 172.21.15.46:55244) 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 -- 172.21.15.46:0/2586207173 learned_addr learned my addr 172.21.15.46:0/2586207173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da6000640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da00732c0 0x7f3da0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 -- 172.21.15.46:0/2586207173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da00732c0 msgr2=0x7f3da0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da00732c0 0x7f3da0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 -- 172.21.15.46:0/2586207173 >> v1:172.21.15.46:6789/0 conn(0x7f3da006e510 legacy=0x7f3da006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.888+0000 7f3da4c00640 1 -- 172.21.15.46:0/2586207173 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0072540 con 0x7f3da0160bd0 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.889+0000 7f3da4c00640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da0160bd0 0x7f3da0161060 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db83a443d405dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.889+0000 7f3d9fe00640 1 -- 172.21.15.46:0/2586207173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c037070 con 0x7f3da0160bd0 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.889+0000 7f3d9fe00640 1 -- 172.21.15.46:0/2586207173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c032440 con 0x7f3da0160bd0 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.889+0000 7f3d9fe00640 1 -- 172.21.15.46:0/2586207173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c032810 con 0x7f3da0160bd0 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 -- 172.21.15.46:0/2586207173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da0160bd0 msgr2=0x7f3da0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da0160bd0 0x7f3da0161060 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c02a2a0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 -- 172.21.15.46:0/2586207173 shutdown_connections 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da00732c0 0x7f3da0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 --2- 172.21.15.46:0/2586207173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da0160bd0 0x7f3da0161060 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 -- 172.21.15.46:0/2586207173 >> 172.21.15.46:0/2586207173 conn(0x7f3da006c530 msgr2=0x7f3da006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 -- 172.21.15.46:0/2586207173 shutdown_connections 2024-04-26T23:43:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.890+0000 7f3da8000640 1 -- 172.21.15.46:0/2586207173 wait complete. 2024-04-26T23:43:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.892+0000 7f3da8000640 1 Processor -- start 2024-04-26T23:43:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.892+0000 7f3da8000640 1 -- start start 2024-04-26T23:43:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da006e510 0x7f3da013ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 0x7f3da0142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0161720 con 0x7f3da00732c0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01615a0 con 0x7f3da0160bd0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da8000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3da0072d40 con 0x7f3da006e510 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.893+0000 7f3da4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38048/0 (socket says 172.21.15.46:38048) 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 -- 172.21.15.46:0/4141494766 learned_addr learned my addr 172.21.15.46:0/4141494766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da5600640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da006e510 0x7f3da013ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da6000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 0x7f3da0142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 -- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da006e510 msgr2=0x7f3da013ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da006e510 0x7f3da013ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 -- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 msgr2=0x7f3da0142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 0x7f3da0142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 -- 172.21.15.46:0/4141494766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01404f0 con 0x7f3da00732c0 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da6000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 0x7f3da0142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.894+0000 7f3da4c00640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0090b0 tx=0x7f3d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.895+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c037040 con 0x7f3da00732c0 2024-04-26T23:43:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.895+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c042070 con 0x7f3da00732c0 2024-04-26T23:43:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.895+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0308e0 con 0x7f3da00732c0 2024-04-26T23:43:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.895+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da013ef10 con 0x7f3da00732c0 2024-04-26T23:43:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.895+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da013f7b0 con 0x7f3da00732c0 2024-04-26T23:43:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.896+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da0069530 con 0x7f3da00732c0 2024-04-26T23:43:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.897+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d8c030a80 con 0x7f3da00732c0 2024-04-26T23:43:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.899+0000 7f3d9d600640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 0x7f3d8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.899+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f3d8c0ac380 con 0x7f3da00732c0 2024-04-26T23:43:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.899+0000 7f3da5600640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 0x7f3d8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.900+0000 7f3da5600640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 0x7f3d8006edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3da006ca10 tx=0x7f3d94007ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:43.903+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d8c07a860 con 0x7f3da00732c0 2024-04-26T23:43:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.121+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3da00cdfe0 con 0x7f3da00732c0 2024-04-26T23:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.186+0000 7f3d9d600640 1 -- 172.21.15.46:0/4141494766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7f3d8c07a160 con 0x7f3da00732c0 2024-04-26T23:43:44.191 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:43:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 msgr2=0x7f3d8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 0x7f3d8006edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3da006ca10 tx=0x7f3d94007ce0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 msgr2=0x7f3da013e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0090b0 tx=0x7f3d8c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 shutdown_connections 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d8006c8f0 0x7f3d8006edb0 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3da006ca10 tx=0x7f3d94007ce0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0160bd0 0x7f3da0142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da00732c0 0x7f3da013e6c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 --2- 172.21.15.46:0/4141494766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da006e510 0x7f3da013ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.189+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 >> 172.21.15.46:0/4141494766 conn(0x7f3da006c530 msgr2=0x7f3da0073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.190+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 shutdown_connections 2024-04-26T23:43:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.190+0000 7f3da8000640 1 -- 172.21.15.46:0/4141494766 wait complete. 2024-04-26T23:43:44.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:43:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.407+0000 7f005b200640 1 Processor -- start 2024-04-26T23:43:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.407+0000 7f005b200640 1 -- start start 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f005b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f005c0cf450 0x7f005c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f005b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 0x7f005c0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f005b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f005c06f470 con 0x7f005c0cfd70 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f005b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f005c072930 con 0x7f005c0cf450 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f005b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f005c072c10 con 0x7f005c072190 2024-04-26T23:43:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f0059e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f005c0cf450 0x7f005c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f0059e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f005c0cf450 0x7f005c073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38052/0 (socket says 172.21.15.46:38052) 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.409+0000 7f0059e00640 1 -- 172.21.15.46:0/758220783 learned_addr learned my addr 172.21.15.46:0/758220783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0051400640 1 --2- 172.21.15.46:0/758220783 >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 0x7f005c0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0059e00640 1 -- 172.21.15.46:0/758220783 >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 msgr2=0x7f005c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0059e00640 1 --2- 172.21.15.46:0/758220783 >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 0x7f005c0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0059e00640 1 -- 172.21.15.46:0/758220783 >> v1:172.21.15.46:6789/0 conn(0x7f005c0cfd70 legacy=0x7f005c06e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0059e00640 1 -- 172.21.15.46:0/758220783 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c072ef0 con 0x7f005c0cf450 2024-04-26T23:43:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.410+0000 7f0051400640 1 --2- 172.21.15.46:0/758220783 >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 0x7f005c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.411+0000 7f0059e00640 1 --2- 172.21.15.46:0/758220783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c073db0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f005c0703e0 tx=0x7f004c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2db9852b27600f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.411+0000 7f0059400640 1 -- 172.21.15.46:0/758220783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c037070 con 0x7f005c0cf450 2024-04-26T23:43:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f0059400640 1 -- 172.21.15.46:0/758220783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004c032440 con 0x7f005c0cf450 2024-04-26T23:43:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f0059400640 1 -- 172.21.15.46:0/758220783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c032810 con 0x7f005c0cf450 2024-04-26T23:43:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 -- 172.21.15.46:0/758220783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 msgr2=0x7f005c073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 --2- 172.21.15.46:0/758220783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c073db0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f005c0703e0 tx=0x7f004c02a270 comp rx=0 tx=0).stop 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 -- 172.21.15.46:0/758220783 shutdown_connections 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 --2- 172.21.15.46:0/758220783 >> v2:172.21.15.142:3300/0 conn(0x7f005c072190 0x7f005c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 --2- 172.21.15.46:0/758220783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c073db0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 -- 172.21.15.46:0/758220783 >> 172.21.15.46:0/758220783 conn(0x7f005c0b2820 msgr2=0x7f005c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 -- 172.21.15.46:0/758220783 shutdown_connections 2024-04-26T23:43:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.412+0000 7f005b200640 1 -- 172.21.15.46:0/758220783 wait complete. 2024-04-26T23:43:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.414+0000 7f005b200640 1 Processor -- start 2024-04-26T23:43:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.414+0000 7f005b200640 1 -- start start 2024-04-26T23:43:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c072190 0x7f005c1456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c0cfd70 0x7f005c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f005c072620 con 0x7f005c0cf450 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f005c0734b0 con 0x7f005c0cfd70 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f005b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f005c0d0180 con 0x7f005c072190 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f0051400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f0051400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38068/0 (socket says 172.21.15.46:38068) 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.415+0000 7f0051400640 1 -- 172.21.15.46:0/2600207961 learned_addr learned my addr 172.21.15.46:0/2600207961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0059e00640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c072190 0x7f005c1456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 -- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c072190 msgr2=0x7f005c1456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c072190 0x7f005c1456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f005a800640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c0cfd70 0x7f005c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 -- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c0cfd70 msgr2=0x7f005c0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c0cfd70 0x7f005c0cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 -- 172.21.15.46:0/2600207961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c1e6300 con 0x7f005c0cf450 2024-04-26T23:43:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.416+0000 7f0051400640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f004400c7f0 tx=0x7f004400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.417+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044015070 con 0x7f005c0cf450 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.417+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0044007820 con 0x7f005c0cf450 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.417+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044007bd0 con 0x7f005c0cf450 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.417+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c1e65e0 con 0x7f005c0cf450 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.417+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1e6ba0 con 0x7f005c0cf450 2024-04-26T23:43:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.418+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c0693b0 con 0x7f005c0cf450 2024-04-26T23:43:44.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.419+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0044018070 con 0x7f005c0cf450 2024-04-26T23:43:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.420+0000 7f0052a00640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 0x7f003806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.420+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f004408bb40 con 0x7f005c0cf450 2024-04-26T23:43:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.421+0000 7f0059e00640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 0x7f003806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.422+0000 7f0059e00640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 0x7f003806ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f004c034000 tx=0x7f004c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.425+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f004405a090 con 0x7f005c0cf450 2024-04-26T23:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.702+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f005c073e50 con 0x7f005c0cf450 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.704+0000 7f0052a00640 1 -- 172.21.15.46:0/2600207961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f0044059990 con 0x7f005c0cf450 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:43:44.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:43:44.711 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:43:44.713 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:43:44.715 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:43:44.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:43:44.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:43:44.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:43:44.721 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 msgr2=0x7f003806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 0x7f003806ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f004c034000 tx=0x7f004c02e000 comp rx=0 tx=0).stop 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 msgr2=0x7f005c1e5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f004400c7f0 tx=0x7f004400ccb0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 shutdown_connections 2024-04-26T23:43:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.706+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003806c920 0x7f003806ede0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c0cfd70 0x7f005c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cf450 0x7f005c1e5c70 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 --2- 172.21.15.46:0/2600207961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c072190 0x7f005c1456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 >> 172.21.15.46:0/2600207961 conn(0x7f005c0b2820 msgr2=0x7f005c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 shutdown_connections 2024-04-26T23:43:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.707+0000 7f005b200640 1 -- 172.21.15.46:0/2600207961 wait complete. 2024-04-26T23:43:44.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.925+0000 7f5c86c00640 1 Processor -- start 2024-04-26T23:43:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.925+0000 7f5c86c00640 1 -- start start 2024-04-26T23:43:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.927+0000 7f5c86c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c801466b0 0x7f5c8006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.927+0000 7f5c86c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c80160910 0x7f5c80160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.927+0000 7f5c86c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c80073ad0 con 0x7f5c800720e0 2024-04-26T23:43:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.927+0000 7f5c86c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5c801612a0 con 0x7f5c801466b0 2024-04-26T23:43:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.927+0000 7f5c86c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5c80146a90 con 0x7f5c80160910 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c801466b0 0x7f5c8006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c801466b0 0x7f5c8006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38080/0 (socket says 172.21.15.46:38080) 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 -- 172.21.15.46:0/1717925770 learned_addr learned my addr 172.21.15.46:0/1717925770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c86200640 1 --2- 172.21.15.46:0/1717925770 >> v2:172.21.15.142:3300/0 conn(0x7f5c80160910 0x7f5c80160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 -- 172.21.15.46:0/1717925770 >> v2:172.21.15.142:3300/0 conn(0x7f5c80160910 msgr2=0x7f5c80160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 --2- 172.21.15.46:0/1717925770 >> v2:172.21.15.142:3300/0 conn(0x7f5c80160910 0x7f5c80160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 -- 172.21.15.46:0/1717925770 >> v1:172.21.15.142:6789/0 conn(0x7f5c800720e0 legacy=0x7f5c800cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.928+0000 7f5c85800640 1 -- 172.21.15.46:0/1717925770 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c80073d70 con 0x7f5c801466b0 2024-04-26T23:43:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.930+0000 7f5c85800640 1 --2- 172.21.15.46:0/1717925770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c801466b0 0x7f5c8006de80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009870 tx=0x7f5c7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea3afb95e408f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.930+0000 7f5c7fe00640 1 -- 172.21.15.46:0/1717925770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74037070 con 0x7f5c801466b0 2024-04-26T23:43:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.930+0000 7f5c7fe00640 1 -- 172.21.15.46:0/1717925770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c74032440 con 0x7f5c801466b0 2024-04-26T23:43:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.930+0000 7f5c7fe00640 1 -- 172.21.15.46:0/1717925770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74032810 con 0x7f5c801466b0 2024-04-26T23:43:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 -- 172.21.15.46:0/1717925770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c801466b0 msgr2=0x7f5c8006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 --2- 172.21.15.46:0/1717925770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c801466b0 0x7f5c8006de80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009870 tx=0x7f5c7402a270 comp rx=0 tx=0).stop 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 -- 172.21.15.46:0/1717925770 shutdown_connections 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 --2- 172.21.15.46:0/1717925770 >> v2:172.21.15.142:3300/0 conn(0x7f5c80160910 0x7f5c80160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 --2- 172.21.15.46:0/1717925770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c801466b0 0x7f5c8006de80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 -- 172.21.15.46:0/1717925770 >> 172.21.15.46:0/1717925770 conn(0x7f5c800b26d0 msgr2=0x7f5c800b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 -- 172.21.15.46:0/1717925770 shutdown_connections 2024-04-26T23:43:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.931+0000 7f5c86c00640 1 -- 172.21.15.46:0/1717925770 wait complete. 2024-04-26T23:43:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.933+0000 7f5c86c00640 1 Processor -- start 2024-04-26T23:43:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.933+0000 7f5c86c00640 1 -- start start 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 0x7f5c800af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 0x7f5c801f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800cb470 con 0x7f5c80160910 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800cb190 con 0x7f5c801466b0 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.934+0000 7f5c86c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8006e3c0 con 0x7f5c800720e0 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38088/0 (socket says 172.21.15.46:38088) 2024-04-26T23:43:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 -- 172.21.15.46:0/3085482836 learned_addr learned my addr 172.21.15.46:0/3085482836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c84e00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 0x7f5c801f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c85800640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 0x7f5c800af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 -- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 msgr2=0x7f5c800af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 0x7f5c800af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 -- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 msgr2=0x7f5c801f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 0x7f5c801f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 -- 172.21.15.46:0/3085482836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c800afa60 con 0x7f5c80160910 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c85800640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 0x7f5c800af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c84e00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 0x7f5c801f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.935+0000 7f5c86200640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e60 tx=0x7f5c7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.936+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78014420 con 0x7f5c80160910 2024-04-26T23:43:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.936+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c78019070 con 0x7f5c80160910 2024-04-26T23:43:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.936+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78014820 con 0x7f5c80160910 2024-04-26T23:43:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.936+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c800afd40 con 0x7f5c80160910 2024-04-26T23:43:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.936+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c800b0300 con 0x7f5c80160910 2024-04-26T23:43:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.937+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c800693b0 con 0x7f5c80160910 2024-04-26T23:43:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.938+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c780149c0 con 0x7f5c80160910 2024-04-26T23:43:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.939+0000 7f5c7d600640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 0x7f5c6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.939+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f5c7808c260 con 0x7f5c80160910 2024-04-26T23:43:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.940+0000 7f5c85800640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 0x7f5c6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.941+0000 7f5c85800640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 0x7f5c6006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034000 tx=0x7f5c7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:44.943+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c7805a7b0 con 0x7f5c80160910 2024-04-26T23:43:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.151+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c80151dc0 con 0x7f5c80160910 2024-04-26T23:43:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.153+0000 7f5c7d600640 1 -- 172.21.15.46:0/3085482836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13129 (secure 0 0 0) 0x7f5c7805a0b0 con 0x7f5c80160910 2024-04-26T23:43:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:45.158 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:44.200181+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 msgr2=0x7f5c6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 0x7f5c6006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034000 tx=0x7f5c7402e000 comp rx=0 tx=0).stop 2024-04-26T23:43:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 msgr2=0x7f5c800cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e60 tx=0x7f5c7800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 shutdown_connections 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c6006c920 0x7f5c6006ede0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80160910 0x7f5c800cce30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801466b0 0x7f5c801f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 --2- 172.21.15.46:0/3085482836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c800720e0 0x7f5c800af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.156+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 >> 172.21.15.46:0/3085482836 conn(0x7f5c800b26d0 msgr2=0x7f5c800ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.157+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 shutdown_connections 2024-04-26T23:43:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.157+0000 7f5c86c00640 1 -- 172.21.15.46:0/3085482836 wait complete. 2024-04-26T23:43:45.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:43:45.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:43:45.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.377+0000 7f7b01e00640 1 Processor -- start 2024-04-26T23:43:45.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.378+0000 7f7b01e00640 1 -- start start 2024-04-26T23:43:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 0x7f7afc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7afc0b5fb0 0x7f7afc0d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7afc0d07d0 con 0x7f7afc162aa0 2024-04-26T23:43:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7afc0d0ab0 con 0x7f7afc0b5fb0 2024-04-26T23:43:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7afc0d0d90 con 0x7f7afc1626c0 2024-04-26T23:43:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7afc162aa0 0x7f7afc0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55264/0 (socket says 172.21.15.46:55264) 2024-04-26T23:43:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.379+0000 7f7b01400640 1 -- 172.21.15.46:0/3773324592 learned_addr learned my addr 172.21.15.46:0/3773324592 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:45.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afbe00640 1 --2- 172.21.15.46:0/3773324592 >> v2:172.21.15.142:3300/0 conn(0x7f7afc0b5fb0 0x7f7afc0d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7b00a00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 0x7f7afc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1348863453 0 0) 0x7f7afc0d07d0 con 0x7f7afc162aa0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ae4003520 con 0x7f7afc162aa0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3846241935 0 0) 0x7f7ae4003520 con 0x7f7afc162aa0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 msgr2=0x7f7afc0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 0x7f7afc0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 >> v2:172.21.15.142:3300/0 conn(0x7f7afc0b5fb0 msgr2=0x7f7afc0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 --2- 172.21.15.46:0/3773324592 >> v2:172.21.15.142:3300/0 conn(0x7f7afc0b5fb0 0x7f7afc0d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc0d2c80 con 0x7f7afc162aa0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7af40019f0 con 0x7f7afc162aa0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 >> v1:172.21.15.46:6789/0 conn(0x7f7afc162aa0 legacy=0x7f7afc0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae40039b0 0x7f7ae4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae4004800 0x7f7ae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 0x7f7ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0d0ab0 con 0x7f7ae40039b0 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0d0d90 con 0x7f7ae4004800 2024-04-26T23:43:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.380+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae400b9c0 con 0x7f7afc1626c0 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae40039b0 0x7f7ae4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 -- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 msgr2=0x7f7ae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc1626c0 0x7f7ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7afbe00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae4004800 0x7f7ae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 -- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae4004800 msgr2=0x7f7ae4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae4004800 0x7f7ae4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.381+0000 7f7b00a00640 1 -- 172.21.15.46:0/3773324592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae4003520 con 0x7f7ae40039b0 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.382+0000 7f7b00a00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae40039b0 0x7f7ae4003dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7af000ba30 tx=0x7f7ae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0634ef12e1e605 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.382+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0030070 con 0x7f7ae40039b0 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.382+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af002ae10 con 0x7f7ae40039b0 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.382+0000 7f7afb400640 1 -- 172.21.15.46:0/3773324592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af000b600 con 0x7f7ae40039b0 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 -- 172.21.15.46:0/3773324592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae40039b0 msgr2=0x7f7ae4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 --2- 172.21.15.46:0/3773324592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae40039b0 0x7f7ae4003dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7af000ba30 tx=0x7f7ae4005830 comp rx=0 tx=0).stop 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01400640 1 -- 172.21.15.46:0/3773324592 reap_dead start 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 -- 172.21.15.46:0/3773324592 shutdown_connections 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 -- 172.21.15.46:0/3773324592 >> 172.21.15.46:0/3773324592 conn(0x7f7afc09e2a0 msgr2=0x7f7afc0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 -- 172.21.15.46:0/3773324592 shutdown_connections 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.383+0000 7f7b01e00640 1 -- 172.21.15.46:0/3773324592 wait complete. 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.385+0000 7f7b01e00640 1 Processor -- start 2024-04-26T23:43:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.385+0000 7f7b01e00640 1 -- start start 2024-04-26T23:43:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40039b0 0x7f7afc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc14a7f0 0x7f7afc14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0d2190 con 0x7f7afc1f5810 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0d2010 con 0x7f7ae40039b0 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7b01e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7afc0b47c0 con 0x7f7afc14a7f0 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38102/0 (socket says 172.21.15.46:38102) 2024-04-26T23:43:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.386+0000 7f7afbe00640 1 -- 172.21.15.46:0/1611734720 learned_addr learned my addr 172.21.15.46:0/1611734720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7b01400640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc14a7f0 0x7f7afc14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 -- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc14a7f0 msgr2=0x7f7afc14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc14a7f0 0x7f7afc14ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 -- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40039b0 msgr2=0x7f7afc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40039b0 0x7f7afc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 -- 172.21.15.46:0/1611734720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc15dc70 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7afbe00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80108d0 tx=0x7f7ae8010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800f040 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae800eb00 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801c400 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7afc15df50 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.387+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7afc15e510 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.389+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7afc0693b0 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.390+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae801c5a0 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.390+0000 7f7af8c00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 0x7f7adc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.391+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f7ae808bfd0 con 0x7f7afc1f5810 2024-04-26T23:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.391+0000 7f7b00a00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 0x7f7adc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.392+0000 7f7b00a00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 0x7f7adc06edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7af000ba00 tx=0x7f7af002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.395+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ae8018030 con 0x7f7afc1f5810 2024-04-26T23:43:45.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.597+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7afc0d25b0 con 0x7f7afc1f5810 2024-04-26T23:43:45.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.598+0000 7f7af8c00640 1 -- 172.21.15.46:0/1611734720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f7ae8059f70 con 0x7f7afc1f5810 2024-04-26T23:43:45.603 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 msgr2=0x7f7adc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 0x7f7adc06edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7af000ba00 tx=0x7f7af002c000 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 msgr2=0x7f7afc14a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80108d0 tx=0x7f7ae8010d90 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 shutdown_connections 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7adc06c8f0 0x7f7adc06edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7afc14a7f0 0x7f7afc14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7afc1f5810 0x7f7afc14a2b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 --2- 172.21.15.46:0/1611734720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40039b0 0x7f7afc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 >> 172.21.15.46:0/1611734720 conn(0x7f7afc09e2a0 msgr2=0x7f7afc0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 shutdown_connections 2024-04-26T23:43:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.601+0000 7f7b01e00640 1 -- 172.21.15.46:0/1611734720 wait complete. 2024-04-26T23:43:45.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:43:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.832+0000 7f2e42800640 1 Processor -- start 2024-04-26T23:43:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.833+0000 7f2e42800640 1 -- start start 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.834+0000 7f2e42800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c072600 0x7f2e3c06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.834+0000 7f2e42800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.834+0000 7f2e42800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3c06eff0 con 0x7f2e3c14cf10 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.834+0000 7f2e42800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3c06f2d0 con 0x7f2e3c1651a0 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.834+0000 7f2e42800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c06f5b0 con 0x7f2e3c072600 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38110/0 (socket says 172.21.15.46:38110) 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 -- 172.21.15.46:0/3129279545 learned_addr learned my addr 172.21.15.46:0/3129279545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e40a00640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c072600 0x7f2e3c06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 -- 172.21.15.46:0/3129279545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c072600 msgr2=0x7f2e3c06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c072600 0x7f2e3c06e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 -- 172.21.15.46:0/3129279545 >> v1:172.21.15.142:6789/0 conn(0x7f2e3c14cf10 legacy=0x7f2e3c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.835+0000 7f2e41e00640 1 -- 172.21.15.46:0/3129279545 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c06f890 con 0x7f2e3c1651a0 2024-04-26T23:43:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.836+0000 7f2e41e00640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009e50 tx=0x7f2e3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76620053de80d5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e3be00640 1 -- 172.21.15.46:0/3129279545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34036070 con 0x7f2e3c1651a0 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e3be00640 1 -- 172.21.15.46:0/3129279545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34031600 con 0x7f2e3c1651a0 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e3be00640 1 -- 172.21.15.46:0/3129279545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340319d0 con 0x7f2e3c1651a0 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 -- 172.21.15.46:0/3129279545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c1651a0 msgr2=0x7f2e3c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009e50 tx=0x7f2e3402a710 comp rx=0 tx=0).stop 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 -- 172.21.15.46:0/3129279545 shutdown_connections 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0b3f20 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 --2- 172.21.15.46:0/3129279545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c072600 0x7f2e3c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 -- 172.21.15.46:0/3129279545 >> 172.21.15.46:0/3129279545 conn(0x7f2e3c0b2820 msgr2=0x7f2e3c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 -- 172.21.15.46:0/3129279545 shutdown_connections 2024-04-26T23:43:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.837+0000 7f2e42800640 1 -- 172.21.15.46:0/3129279545 wait complete. 2024-04-26T23:43:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.839+0000 7f2e42800640 1 Processor -- start 2024-04-26T23:43:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.839+0000 7f2e42800640 1 -- start start 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c14cf10 0x7f2e3c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b4460 con 0x7f2e3c072600 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c163bd0 con 0x7f2e3c1651a0 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.840+0000 7f2e42800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b45e0 con 0x7f2e3c14cf10 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38116/0 (socket says 172.21.15.46:38116) 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 -- 172.21.15.46:0/3611548159 learned_addr learned my addr 172.21.15.46:0/3611548159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e40a00640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c14cf10 0x7f2e3c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 -- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c14cf10 msgr2=0x7f2e3c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41e00640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c14cf10 0x7f2e3c0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 -- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c1651a0 msgr2=0x7f2e3c0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 -- 172.21.15.46:0/3611548159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c1fd3b0 con 0x7f2e3c072600 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.841+0000 7f2e41400640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c430 tx=0x7f2e3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.842+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300052d0 con 0x7f2e3c072600 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.842+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e30005490 con 0x7f2e3c072600 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.842+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30007260 con 0x7f2e3c072600 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.842+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c1634e0 con 0x7f2e3c072600 2024-04-26T23:43:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.842+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1fcb10 con 0x7f2e3c072600 2024-04-26T23:43:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.843+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c0693b0 con 0x7f2e3c072600 2024-04-26T23:43:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.844+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e30007400 con 0x7f2e3c072600 2024-04-26T23:43:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.846+0000 7f2e39600640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 0x7f2e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.846+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f2e3008af00 con 0x7f2e3c072600 2024-04-26T23:43:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.847+0000 7f2e40a00640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 0x7f2e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.848+0000 7f2e40a00640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 0x7f2e2006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b2780 tx=0x7f2e2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:45.851+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e300594e0 con 0x7f2e3c072600 2024-04-26T23:43:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.057+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2e3c0b5ac0 con 0x7f2e3c072600 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.058+0000 7f2e39600640 1 -- 172.21.15.46:0/3611548159 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f2e30058de0 con 0x7f2e3c072600 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:43:46.063 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 msgr2=0x7f2e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 0x7f2e2006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b2780 tx=0x7f2e2c008040 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 msgr2=0x7f2e3c162fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c430 tx=0x7f2e3000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 shutdown_connections 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2e2006c8f0 0x7f2e2006edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c1651a0 0x7f2e3c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e3c14cf10 0x7f2e3c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 --2- 172.21.15.46:0/3611548159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e3c072600 0x7f2e3c162fa0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 >> 172.21.15.46:0/3611548159 conn(0x7f2e3c0b2820 msgr2=0x7f2e3c1667d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 shutdown_connections 2024-04-26T23:43:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.061+0000 7f2e42800640 1 -- 172.21.15.46:0/3611548159 wait complete. 2024-04-26T23:43:46.085 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:43:46.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:43:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.306+0000 7f71cf200640 1 Processor -- start 2024-04-26T23:43:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.306+0000 7f71cf200640 1 -- start start 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.307+0000 7f71cf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71d01627f0 0x7f71d00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.307+0000 7f71cf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.307+0000 7f71cf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d00d01a0 con 0x7f71d00d2080 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.307+0000 7f71cf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f71d00d0480 con 0x7f71d01627f0 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.307+0000 7f71cf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f71d00d0760 con 0x7f71d00d3cc0 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71d01627f0 0x7f71d00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71d01627f0 0x7f71d00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38132/0 (socket says 172.21.15.46:38132) 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 -- 172.21.15.46:0/1227000431 learned_addr learned my addr 172.21.15.46:0/1227000431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cd400640 1 --1- 172.21.15.46:0/1227000431 >> v1:172.21.15.46:6789/0 conn(0x7f71d00d2080 0x7f71d00d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55270/0 (socket says 172.21.15.46:55270) 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71ce800640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 -- 172.21.15.46:0/1227000431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 msgr2=0x7f71d00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 -- 172.21.15.46:0/1227000431 >> v1:172.21.15.46:6789/0 conn(0x7f71d00d2080 legacy=0x7f71d00d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cca00640 1 -- 172.21.15.46:0/1227000431 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160630260 0 0) 0x7f71d00d01a0 con 0x7f71d00d2080 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71cde00640 1 -- 172.21.15.46:0/1227000431 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d00d0a40 con 0x7f71d01627f0 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.308+0000 7f71ce800640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.310+0000 7f71cde00640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00cfbc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009870 tx=0x7f71b802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed7922a3a1e6e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.310+0000 7f71cca00640 1 -- 172.21.15.46:0/1227000431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8037070 con 0x7f71d01627f0 2024-04-26T23:43:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.310+0000 7f71cca00640 1 -- 172.21.15.46:0/1227000431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b8032640 con 0x7f71d01627f0 2024-04-26T23:43:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.310+0000 7f71cca00640 1 -- 172.21.15.46:0/1227000431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8032a10 con 0x7f71d01627f0 2024-04-26T23:43:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 -- 172.21.15.46:0/1227000431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 msgr2=0x7f71d00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00cfbc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009870 tx=0x7f71b802a2e0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 -- 172.21.15.46:0/1227000431 shutdown_connections 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 --2- 172.21.15.46:0/1227000431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00cfbc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 -- 172.21.15.46:0/1227000431 >> 172.21.15.46:0/1227000431 conn(0x7f71d009e380 msgr2=0x7f71d00d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 -- 172.21.15.46:0/1227000431 shutdown_connections 2024-04-26T23:43:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.311+0000 7f71cf200640 1 -- 172.21.15.46:0/1227000431 wait complete. 2024-04-26T23:43:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.313+0000 7f71cf200640 1 Processor -- start 2024-04-26T23:43:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.313+0000 7f71cf200640 1 -- start start 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 0x7f71d01614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0162bd0 con 0x7f71d01627f0 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71d00d24f0 con 0x7f71d00d2080 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f71d00b4a90 con 0x7f71d00d3cc0 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38144/0 (socket says 172.21.15.46:38144) 2024-04-26T23:43:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71ce800640 1 -- 172.21.15.46:0/2526150402 learned_addr learned my addr 172.21.15.46:0/2526150402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cde00640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 0x7f71d01614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.314+0000 7f71cd400640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 -- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 msgr2=0x7f71d00b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 -- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 msgr2=0x7f71d01614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 0x7f71d01614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 -- 172.21.15.46:0/2526150402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d00b8520 con 0x7f71d01627f0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71cde00640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 0x7f71d01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71ce800640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f71c800c7f0 tx=0x7f71c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8015070 con 0x7f71d01627f0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c8004240 con 0x7f71d01627f0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8007620 con 0x7f71d01627f0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.315+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d00b8800 con 0x7f71d01627f0 2024-04-26T23:43:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.316+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d00b8dc0 con 0x7f71d01627f0 2024-04-26T23:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.317+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d00d1550 con 0x7f71d01627f0 2024-04-26T23:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.318+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c80079d0 con 0x7f71d01627f0 2024-04-26T23:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.318+0000 7f71c6000640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 0x7f71ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.319+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f71c808bb00 con 0x7f71d01627f0 2024-04-26T23:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.319+0000 7f71cde00640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 0x7f71ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.320+0000 7f71cde00640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 0x7f71ac06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71b8034000 tx=0x7f71b802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.323+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71d00d1550 con 0x7f71d01627f0 2024-04-26T23:43:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:46.529+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f71d00693b0 con 0x7f71d01627f0 2024-04-26T23:43:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.215+0000 7f71c6000640 1 -- 172.21.15.46:0/2526150402 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f71c805a050 con 0x7f71d01627f0 2024-04-26T23:43:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:43:47.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 msgr2=0x7f71ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 0x7f71ac06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71b8034000 tx=0x7f71b802e000 comp rx=0 tx=0).stop 2024-04-26T23:43:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 msgr2=0x7f71d00ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f71c800c7f0 tx=0x7f71c800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 shutdown_connections 2024-04-26T23:43:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f71ac06c8f0 0x7f71ac06edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71d01627f0 0x7f71d00ba7b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71d00d3cc0 0x7f71d00b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.219+0000 7f71cf200640 1 --2- 172.21.15.46:0/2526150402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d00d2080 0x7f71d01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.220+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 >> 172.21.15.46:0/2526150402 conn(0x7f71d009e380 msgr2=0x7f71d00b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.220+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 shutdown_connections 2024-04-26T23:43:47.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.220+0000 7f71cf200640 1 -- 172.21.15.46:0/2526150402 wait complete. 2024-04-26T23:43:47.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:43:47.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.461+0000 7f5aee800640 1 Processor -- start 2024-04-26T23:43:47.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.462+0000 7f5aee800640 1 -- start start 2024-04-26T23:43:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.463+0000 7f5aee800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 0x7f5ae80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.463+0000 7f5aee800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae80d07d0 con 0x7f5ae81626c0 2024-04-26T23:43:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.463+0000 7f5aee800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae80d0ab0 con 0x7f5ae8162aa0 2024-04-26T23:43:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.463+0000 7f5aee800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae80d0d90 con 0x7f5ae80b5fb0 2024-04-26T23:43:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.464+0000 7f5aed400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ae81626c0 0x7f5ae80d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55272/0 (socket says 172.21.15.46:55272) 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.464+0000 7f5aed400640 1 -- 172.21.15.46:0/4058712050 learned_addr learned my addr 172.21.15.46:0/4058712050 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.464+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277055887 0 0) 0x7f5ae80d07d0 con 0x7f5ae81626c0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.464+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5acc003520 con 0x7f5ae81626c0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.464+0000 7f5aeca00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 0x7f5ae80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1930552387 0 0) 0x7f5acc003520 con 0x7f5ae81626c0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 msgr2=0x7f5ae80b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 0x7f5ae80b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 >> v1:172.21.15.142:6789/0 conn(0x7f5ae8162aa0 legacy=0x7f5ae80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae80d2c80 con 0x7f5ae81626c0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5ad8001670 con 0x7f5ae81626c0 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 >> v1:172.21.15.46:6789/0 conn(0x7f5ae81626c0 legacy=0x7f5ae80d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5acc0039e0 0x7f5acc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5acc004800 0x7f5acc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 0x7f5acc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae80d0d90 con 0x7f5acc004800 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc00bb80 con 0x7f5acc0039e0 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc00bd80 con 0x7f5ae80b5fb0 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5aeca00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5acc004800 0x7f5acc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.465+0000 7f5aed400640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5acc0039e0 0x7f5acc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.466+0000 7f5aeca00640 1 -- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 msgr2=0x7f5acc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.466+0000 7f5aeca00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ae80b5fb0 0x7f5acc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.466+0000 7f5aeca00640 1 -- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5acc0039e0 msgr2=0x7f5acc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.466+0000 7f5aeca00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5acc0039e0 0x7f5acc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.466+0000 7f5aeca00640 1 -- 172.21.15.46:0/4058712050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc003520 con 0x7f5acc004800 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.467+0000 7f5aeca00640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5acc004800 0x7f5acc004c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400ba10 tx=0x7f5acc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd54d7b9251f9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.467+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400b5e0 con 0x7f5acc004800 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.467+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad402ae00 con 0x7f5acc004800 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.467+0000 7f5ae7e00640 1 -- 172.21.15.46:0/4058712050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4005bb0 con 0x7f5acc004800 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 -- 172.21.15.46:0/4058712050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5acc004800 msgr2=0x7f5acc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 --2- 172.21.15.46:0/4058712050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5acc004800 0x7f5acc004c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400ba10 tx=0x7f5acc005830 comp rx=0 tx=0).stop 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aede00640 1 -- 172.21.15.46:0/4058712050 reap_dead start 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 -- 172.21.15.46:0/4058712050 shutdown_connections 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 -- 172.21.15.46:0/4058712050 >> 172.21.15.46:0/4058712050 conn(0x7f5ae809e2a0 msgr2=0x7f5ae80d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 -- 172.21.15.46:0/4058712050 shutdown_connections 2024-04-26T23:43:47.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.468+0000 7f5aee800640 1 -- 172.21.15.46:0/4058712050 wait complete. 2024-04-26T23:43:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.470+0000 7f5aee800640 1 Processor -- start 2024-04-26T23:43:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.470+0000 7f5aee800640 1 -- start start 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5acc004800 0x7f5ae80c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae8150310 0x7f5ae80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae80b47c0 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae80d2220 con 0x7f5ae8150310 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aee800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae80b4940 con 0x7f5acc004800 2024-04-26T23:43:47.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38176/0 (socket says 172.21.15.46:38176) 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.471+0000 7f5aeca00640 1 -- 172.21.15.46:0/364749977 learned_addr learned my addr 172.21.15.46:0/364749977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aed400640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5acc004800 0x7f5ae80c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 -- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5acc004800 msgr2=0x7f5ae80c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5acc004800 0x7f5ae80c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aede00640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae8150310 0x7f5ae80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 -- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae8150310 msgr2=0x7f5ae80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae8150310 0x7f5ae80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 -- 172.21.15.46:0/364749977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae80c9100 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aeca00640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5ad402a8f0 tx=0x7f5ad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad402ac20 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4005990 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae80c93e0 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.472+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4005d40 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.473+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae80c99a0 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.474+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae80693b0 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.475+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ad4003610 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.476+0000 7f5ae5600640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 0x7f5ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.476+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5619+0+0 (secure 0 0 0) 0x7f5ad40abf70 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.477+0000 7f5aed400640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 0x7f5ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.478+0000 7f5aed400640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 0x7f5ac806edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8001eb0 tx=0x7f5ad8000f50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:47.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.481+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ad407a360 con 0x7f5ae80ca1a0 2024-04-26T23:43:47.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:47.691+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5ae80d25b0 con 0x7f5ae80ca1a0 2024-04-26T23:43:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.232+0000 7f5ae5600640 1 -- 172.21.15.46:0/364749977 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f5ad4079c60 con 0x7f5ae80ca1a0 2024-04-26T23:43:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:43:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 msgr2=0x7f5ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 0x7f5ac806edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8001eb0 tx=0x7f5ad8000f50 comp rx=0 tx=0).stop 2024-04-26T23:43:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 msgr2=0x7f5ae80cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5ad402a8f0 tx=0x7f5ad4004730 comp rx=0 tx=0).stop 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 shutdown_connections 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5ac806c8f0 0x7f5ac806edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae8150310 0x7f5ae80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80ca1a0 0x7f5ae80cd480 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 --2- 172.21.15.46:0/364749977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5acc004800 0x7f5ae80c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.236+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 >> 172.21.15.46:0/364749977 conn(0x7f5ae809e2a0 msgr2=0x7f5ae80d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.238+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 shutdown_connections 2024-04-26T23:43:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.238+0000 7f5aee800640 1 -- 172.21.15.46:0/364749977 wait complete. 2024-04-26T23:43:48.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:43:48.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.455+0000 7f2d27e00640 1 Processor -- start 2024-04-26T23:43:48.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.455+0000 7f2d27e00640 1 -- start start 2024-04-26T23:43:48.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d27e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d201597e0 0x7f2d200ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d27e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 0x7f2d200b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d27e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d20073120 con 0x7f2d20071f10 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d27e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2d20159bc0 con 0x7f2d201597e0 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d27e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2d20072b60 con 0x7f2d200b5730 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d25400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d201597e0 0x7f2d200ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d25400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d201597e0 0x7f2d200ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38186/0 (socket says 172.21.15.46:38186) 2024-04-26T23:43:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.457+0000 7f2d25400640 1 -- 172.21.15.46:0/3404705785 learned_addr learned my addr 172.21.15.46:0/3404705785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d24a00640 1 --2- 172.21.15.46:0/3404705785 >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 0x7f2d200b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d25400640 1 -- 172.21.15.46:0/3404705785 >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 msgr2=0x7f2d200b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d25400640 1 --2- 172.21.15.46:0/3404705785 >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 0x7f2d200b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d25400640 1 -- 172.21.15.46:0/3404705785 >> v1:172.21.15.46:6789/0 conn(0x7f2d20071f10 legacy=0x7f2d20072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d25400640 1 -- 172.21.15.46:0/3404705785 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20072e40 con 0x7f2d201597e0 2024-04-26T23:43:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.458+0000 7f2d24a00640 1 --2- 172.21.15.46:0/3404705785 >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 0x7f2d200b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.459+0000 7f2d25400640 1 --2- 172.21.15.46:0/3404705785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d201597e0 0x7f2d200ae8e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2d200743c0 tx=0x7f2d1402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d048813150b440 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.459+0000 7f2d1fe00640 1 -- 172.21.15.46:0/3404705785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14037070 con 0x7f2d201597e0 2024-04-26T23:43:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.459+0000 7f2d1fe00640 1 -- 172.21.15.46:0/3404705785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1402ae50 con 0x7f2d201597e0 2024-04-26T23:43:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.459+0000 7f2d1fe00640 1 -- 172.21.15.46:0/3404705785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14032860 con 0x7f2d201597e0 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 -- 172.21.15.46:0/3404705785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d201597e0 msgr2=0x7f2d200ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 --2- 172.21.15.46:0/3404705785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d201597e0 0x7f2d200ae8e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2d200743c0 tx=0x7f2d1402a300 comp rx=0 tx=0).stop 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 -- 172.21.15.46:0/3404705785 shutdown_connections 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 --2- 172.21.15.46:0/3404705785 >> v2:172.21.15.142:3300/0 conn(0x7f2d200b5730 0x7f2d200b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 --2- 172.21.15.46:0/3404705785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d201597e0 0x7f2d200ae8e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 -- 172.21.15.46:0/3404705785 >> 172.21.15.46:0/3404705785 conn(0x7f2d2006c980 msgr2=0x7f2d200b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 -- 172.21.15.46:0/3404705785 shutdown_connections 2024-04-26T23:43:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.460+0000 7f2d27e00640 1 -- 172.21.15.46:0/3404705785 wait complete. 2024-04-26T23:43:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.462+0000 7f2d27e00640 1 Processor -- start 2024-04-26T23:43:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.462+0000 7f2d27e00640 1 -- start start 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d20071f10 0x7f2d201531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 0x7f2d201527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d200aee20 con 0x7f2d200b5730 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d200b43b0 con 0x7f2d201597e0 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d27e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2d200aefa0 con 0x7f2d20071f10 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.463+0000 7f2d24a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38200/0 (socket says 172.21.15.46:38200) 2024-04-26T23:43:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 -- 172.21.15.46:0/1754364555 learned_addr learned my addr 172.21.15.46:0/1754364555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 -- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d20071f10 msgr2=0x7f2d201531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d25400640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d20071f10 0x7f2d201531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d25e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 0x7f2d201527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d20071f10 0x7f2d201531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 -- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 msgr2=0x7f2d201527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 0x7f2d201527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 -- 172.21.15.46:0/1754364555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20152d20 con 0x7f2d200b5730 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d25e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 0x7f2d201527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.464+0000 7f2d24a00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c7f0 tx=0x7f2d0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.465+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c015070 con 0x7f2d200b5730 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.465+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c007a30 con 0x7f2d200b5730 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.465+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d20154110 con 0x7f2d200b5730 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.465+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c007e00 con 0x7f2d200b5730 2024-04-26T23:43:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.465+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d200cda30 con 0x7f2d200b5730 2024-04-26T23:43:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.466+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d20068c80 con 0x7f2d200b5730 2024-04-26T23:43:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.467+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d0c018070 con 0x7f2d200b5730 2024-04-26T23:43:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.468+0000 7f2d1d600640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 0x7f2d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.469+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6001+0+0 (secure 0 0 0) 0x7f2d0c08bc70 con 0x7f2d200b5730 2024-04-26T23:43:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.469+0000 7f2d25400640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 0x7f2d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:48.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.470+0000 7f2d25400640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 0x7f2d0006edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2d14034000 tx=0x7f2d1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.473+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d20068c80 con 0x7f2d200b5730 2024-04-26T23:43:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:48.719+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2d20069530 con 0x7f2d200b5730 2024-04-26T23:43:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.237+0000 7f2d1d600640 1 -- 172.21.15.46:0/1754364555 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v55) ==== 174+0+0 (secure 0 0 0) 0x7f2d0c059f60 con 0x7f2d200b5730 2024-04-26T23:43:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 11 and data pool 12 2024-04-26T23:43:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.243+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 msgr2=0x7f2d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.243+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 0x7f2d0006edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2d14034000 tx=0x7f2d1402e000 comp rx=0 tx=0).stop 2024-04-26T23:43:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.243+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 msgr2=0x7f2d201539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.243+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c7f0 tx=0x7f2d0c00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 shutdown_connections 2024-04-26T23:43:49.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2d0006c8f0 0x7f2d0006edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d201597e0 0x7f2d201527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d200b5730 0x7f2d201539a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 --2- 172.21.15.46:0/1754364555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d20071f10 0x7f2d201531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.244+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 >> 172.21.15.46:0/1754364555 conn(0x7f2d2006c980 msgr2=0x7f2d200af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:49.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.248+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 shutdown_connections 2024-04-26T23:43:49.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.248+0000 7f2d27e00640 1 -- 172.21.15.46:0/1754364555 wait complete. 2024-04-26T23:43:49.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.458+0000 7f48d6400640 1 Processor -- start 2024-04-26T23:43:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.458+0000 7f48d6400640 1 -- start start 2024-04-26T23:43:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48d00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d00d11f0 con 0x7f48d015f4e0 2024-04-26T23:43:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d6400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d00d5f20 con 0x7f48d00d2130 2024-04-26T23:43:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48d00d0ce0 con 0x7f48d00b4000 2024-04-26T23:43:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d5000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f48d015f4e0 0x7f48d00d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55296/0 (socket says 172.21.15.46:55296) 2024-04-26T23:43:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.460+0000 7f48d5000640 1 -- 172.21.15.46:0/2269334143 learned_addr learned my addr 172.21.15.46:0/2269334143 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48d5a00640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48d00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111344298 0 0) 0x7f48d00d11f0 con 0x7f48d015f4e0 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48b8003520 con 0x7f48d015f4e0 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4031473574 0 0) 0x7f48b8003520 con 0x7f48d015f4e0 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 msgr2=0x7f48d00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48d00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 >> v1:172.21.15.142:6789/0 conn(0x7f48d00d2130 legacy=0x7f48d00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d00d0f10 con 0x7f48d015f4e0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f48c4001670 con 0x7f48d015f4e0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 >> v1:172.21.15.46:6789/0 conn(0x7f48d015f4e0 legacy=0x7f48d00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b80039e0 0x7f48b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48b8004d40 0x7f48b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48b800ba70 con 0x7f48b80039e0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48b800bc20 con 0x7f48b8004d40 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48b800be20 con 0x7f48d00b4000 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.461+0000 7f48d5000640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b80039e0 0x7f48b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48cfe00640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5a00640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48b8004d40 0x7f48b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5000640 1 -- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 msgr2=0x7f48b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5000640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48d00b4000 0x7f48b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5000640 1 -- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48b8004d40 msgr2=0x7f48b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5000640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48b8004d40 0x7f48b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5000640 1 -- 172.21.15.46:0/2269334143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b8003520 con 0x7f48b80039e0 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.462+0000 7f48d5a00640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48b8004d40 0x7f48b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.463+0000 7f48d5000640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b80039e0 0x7f48b8003e00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c210 tx=0x7f48b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e17d772c6263be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.463+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4007cf0 con 0x7f48b80039e0 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.463+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c4005af0 con 0x7f48b80039e0 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.463+0000 7f48cf400640 1 -- 172.21.15.46:0/2269334143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4007070 con 0x7f48b80039e0 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 -- 172.21.15.46:0/2269334143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b80039e0 msgr2=0x7f48b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 --2- 172.21.15.46:0/2269334143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b80039e0 0x7f48b8003e00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c210 tx=0x7f48b8005830 comp rx=0 tx=0).stop 2024-04-26T23:43:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d5a00640 1 -- 172.21.15.46:0/2269334143 reap_dead start 2024-04-26T23:43:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 -- 172.21.15.46:0/2269334143 shutdown_connections 2024-04-26T23:43:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 -- 172.21.15.46:0/2269334143 >> 172.21.15.46:0/2269334143 conn(0x7f48d00d5010 msgr2=0x7f48d00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 -- 172.21.15.46:0/2269334143 shutdown_connections 2024-04-26T23:43:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.464+0000 7f48d6400640 1 -- 172.21.15.46:0/2269334143 wait complete. 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.466+0000 7f48d6400640 1 Processor -- start 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.466+0000 7f48d6400640 1 -- start start 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48b80039e0 0x7f48d015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d015dd30 0x7f48d01e8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48d00b1d60 con 0x7f48d01e90f0 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48d00d2990 con 0x7f48d015dd30 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48d015f8c0 con 0x7f48b80039e0 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38224/0 (socket says 172.21.15.46:38224) 2024-04-26T23:43:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.467+0000 7f48d5a00640 1 -- 172.21.15.46:0/1003633 learned_addr learned my addr 172.21.15.46:0/1003633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48cfe00640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d015dd30 0x7f48d01e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5000640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48b80039e0 0x7f48d015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 -- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48b80039e0 msgr2=0x7f48d015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48b80039e0 0x7f48d015d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 -- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d015dd30 msgr2=0x7f48d01e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d015dd30 0x7f48d01e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 -- 172.21.15.46:0/1003633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d00cdd70 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d5a00640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f48d008d650 tx=0x7f48c800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c801c070 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c8017400 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c80177f0 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.468+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d00ce050 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.469+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d00ce610 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.470+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d00693b0 con 0x7f48d01e90f0 2024-04-26T23:43:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.471+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48c801f070 con 0x7f48d01e90f0 2024-04-26T23:43:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.472+0000 7f48ccc00640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.472+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f48c808bc80 con 0x7f48d01e90f0 2024-04-26T23:43:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.472+0000 7f48d5000640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.473+0000 7f48d5000640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 0x7f48b006edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48c4032000 tx=0x7f48c4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.477+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48c8012070 con 0x7f48d01e90f0 2024-04-26T23:43:49.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.681+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48d00d33d0 con 0x7f48d01e90f0 2024-04-26T23:43:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.683+0000 7f48ccc00640 1 -- 172.21.15.46:0/1003633 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16922 (secure 0 0 0) 0x7f48c8059f10 con 0x7f48d01e90f0 2024-04-26T23:43:49.688 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:49.688 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:48.721917+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 msgr2=0x7f48b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 0x7f48b006edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48c4032000 tx=0x7f48c4009030 comp rx=0 tx=0).stop 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 msgr2=0x7f48d01e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f48d008d650 tx=0x7f48c800f4a0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 shutdown_connections 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d01e90f0 0x7f48d01e9570 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d015dd30 0x7f48d01e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 --2- 172.21.15.46:0/1003633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48b80039e0 0x7f48d015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 >> 172.21.15.46:0/1003633 conn(0x7f48d00d5010 msgr2=0x7f48d00d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 shutdown_connections 2024-04-26T23:43:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.686+0000 7f48d6400640 1 -- 172.21.15.46:0/1003633 wait complete. 2024-04-26T23:43:49.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:49.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.919+0000 7f6833000640 1 Processor -- start 2024-04-26T23:43:49.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.920+0000 7f6833000640 1 -- start start 2024-04-26T23:43:49.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f682c14c2e0 0x7f682c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cb0f0 0x7f682c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f682c0cd9f0 0x7f682c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f682c0cde60 con 0x7f682c0cd9f0 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f682c0cb8b0 con 0x7f682c14c2e0 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.921+0000 7f6833000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f682c14c6c0 con 0x7f682c0cb0f0 2024-04-26T23:43:49.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f682c0cd9f0 0x7f682c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f682c0cd9f0 0x7f682c0c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38232/0 (socket says 172.21.15.46:38232) 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 -- 172.21.15.46:0/1087133226 learned_addr learned my addr 172.21.15.46:0/1087133226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6831c00640 1 --2- 172.21.15.46:0/1087133226 >> v2:172.21.15.142:3300/0 conn(0x7f682c14c2e0 0x7f682c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6831200640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cb0f0 0x7f682c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 -- 172.21.15.46:0/1087133226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cb0f0 msgr2=0x7f682c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cb0f0 0x7f682c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 -- 172.21.15.46:0/1087133226 >> v2:172.21.15.142:3300/0 conn(0x7f682c14c2e0 msgr2=0x7f682c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 --2- 172.21.15.46:0/1087133226 >> v2:172.21.15.142:3300/0 conn(0x7f682c14c2e0 0x7f682c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.922+0000 7f6832600640 1 -- 172.21.15.46:0/1087133226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c167e10 con 0x7f682c0cd9f0 2024-04-26T23:43:49.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.924+0000 7f6832600640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c0cd9f0 0x7f682c0c5940 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f682c0cd480 tx=0x7f682402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3267fe5dcaae2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.924+0000 7f682be00640 1 -- 172.21.15.46:0/1087133226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824036070 con 0x7f682c0cd9f0 2024-04-26T23:43:49.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.924+0000 7f682be00640 1 -- 172.21.15.46:0/1087133226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824031600 con 0x7f682c0cd9f0 2024-04-26T23:43:49.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.924+0000 7f682be00640 1 -- 172.21.15.46:0/1087133226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68240319d0 con 0x7f682c0cd9f0 2024-04-26T23:43:49.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 -- 172.21.15.46:0/1087133226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c0cd9f0 msgr2=0x7f682c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c0cd9f0 0x7f682c0c5940 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f682c0cd480 tx=0x7f682402a610 comp rx=0 tx=0).stop 2024-04-26T23:43:49.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 -- 172.21.15.46:0/1087133226 shutdown_connections 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c0cd9f0 0x7f682c0c5940 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 --2- 172.21.15.46:0/1087133226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cb0f0 0x7f682c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 --2- 172.21.15.46:0/1087133226 >> v2:172.21.15.142:3300/0 conn(0x7f682c14c2e0 0x7f682c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 -- 172.21.15.46:0/1087133226 >> 172.21.15.46:0/1087133226 conn(0x7f682c0b26d0 msgr2=0x7f682c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 -- 172.21.15.46:0/1087133226 shutdown_connections 2024-04-26T23:43:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.925+0000 7f6833000640 1 -- 172.21.15.46:0/1087133226 wait complete. 2024-04-26T23:43:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.927+0000 7f6833000640 1 Processor -- start 2024-04-26T23:43:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.927+0000 7f6833000640 1 -- start start 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0cb0f0 0x7f682c1f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cd9f0 0x7f682c072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f682c167610 con 0x7f682c14c2e0 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f682c167330 con 0x7f682c0cb0f0 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6833000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f682c1678f0 con 0x7f682c0cd9f0 2024-04-26T23:43:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6832600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6832600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38234/0 (socket says 172.21.15.46:38234) 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6832600640 1 -- 172.21.15.46:0/2076668112 learned_addr learned my addr 172.21.15.46:0/2076668112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6831c00640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0cb0f0 0x7f682c1f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.928+0000 7f6831200640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cd9f0 0x7f682c072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 -- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cd9f0 msgr2=0x7f682c072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cd9f0 0x7f682c072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 -- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0cb0f0 msgr2=0x7f682c1f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0cb0f0 0x7f682c1f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 -- 172.21.15.46:0/2076668112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c073b40 con 0x7f682c14c2e0 2024-04-26T23:43:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6832600640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f40 tx=0x7f6824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682403f020 con 0x7f682c14c2e0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824037040 con 0x7f682c14c2e0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824036070 con 0x7f682c14c2e0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c14ace0 con 0x7f682c14c2e0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.929+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c149f20 con 0x7f682c14c2e0 2024-04-26T23:43:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.930+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c0693b0 con 0x7f682c14c2e0 2024-04-26T23:43:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.932+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6824042020 con 0x7f682c14c2e0 2024-04-26T23:43:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.933+0000 7f6829600640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 0x7f680c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.933+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f68240abff0 con 0x7f682c14c2e0 2024-04-26T23:43:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.933+0000 7f6831c00640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 0x7f680c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.934+0000 7f6831c00640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 0x7f680c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6820009a50 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:49.937+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6824049030 con 0x7f682c14c2e0 2024-04-26T23:43:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.141+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f682c1634d0 con 0x7f682c14c2e0 2024-04-26T23:43:50.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.145+0000 7f6829600640 1 -- 172.21.15.46:0/2076668112 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16922 (secure 0 0 0) 0x7f682407a2e0 con 0x7f682c14c2e0 2024-04-26T23:43:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:50.150 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:48.721917+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.147+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 msgr2=0x7f680c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.147+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 0x7f680c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6820009a50 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-04-26T23:43:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.147+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 msgr2=0x7f682c06de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.147+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f40 tx=0x7f6824004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 shutdown_connections 2024-04-26T23:43:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f680c06c8f0 0x7f680c06edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f682c14c2e0 0x7f682c06de60 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f682c0cd9f0 0x7f682c072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 --2- 172.21.15.46:0/2076668112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0cb0f0 0x7f682c1f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 >> 172.21.15.46:0/2076668112 conn(0x7f682c0b26d0 msgr2=0x7f682c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 shutdown_connections 2024-04-26T23:43:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.148+0000 7f6833000640 1 -- 172.21.15.46:0/2076668112 wait complete. 2024-04-26T23:43:50.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:43:50.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.388+0000 7f9736e00640 1 Processor -- start 2024-04-26T23:43:50.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.388+0000 7f9736e00640 1 -- start start 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.389+0000 7f9736e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f973015cec0 0x7f97300cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.389+0000 7f9736e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 0x7f97300735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9736e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97300ce7d0 con 0x7f9730073b30 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9736e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f97300ceab0 con 0x7f973015cec0 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9736e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f97300ced90 con 0x7f97300cdb30 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f973015cec0 0x7f97300cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9736400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9730073b30 0x7f9730071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55300/0 (socket says 172.21.15.46:55300) 2024-04-26T23:43:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9736400640 1 -- 172.21.15.46:0/4113085139 learned_addr learned my addr 172.21.15.46:0/4113085139 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735000640 1 --2- 172.21.15.46:0/4113085139 >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 0x7f97300735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735a00640 1 -- 172.21.15.46:0/4113085139 >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 msgr2=0x7f97300735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735a00640 1 --2- 172.21.15.46:0/4113085139 >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 0x7f97300735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735a00640 1 -- 172.21.15.46:0/4113085139 >> v1:172.21.15.46:6789/0 conn(0x7f9730073b30 legacy=0x7f9730071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735a00640 1 -- 172.21.15.46:0/4113085139 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97300cf070 con 0x7f973015cec0 2024-04-26T23:43:50.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f9735000640 1 --2- 172.21.15.46:0/4113085139 >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 0x7f97300735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:50.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.390+0000 7f972fe00640 1 -- 172.21.15.46:0/4113085139 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289352872 0 0) 0x7f97300ce7d0 con 0x7f9730073b30 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.391+0000 7f9735a00640 1 --2- 172.21.15.46:0/4113085139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973015cec0 0x7f97300cd5f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9730074300 tx=0x7f972402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5895977a2814a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f972fe00640 1 -- 172.21.15.46:0/4113085139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724037070 con 0x7f973015cec0 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f972fe00640 1 -- 172.21.15.46:0/4113085139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9724032440 con 0x7f973015cec0 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f972fe00640 1 -- 172.21.15.46:0/4113085139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724032810 con 0x7f973015cec0 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 -- 172.21.15.46:0/4113085139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973015cec0 msgr2=0x7f97300cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 --2- 172.21.15.46:0/4113085139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973015cec0 0x7f97300cd5f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9730074300 tx=0x7f972402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 -- 172.21.15.46:0/4113085139 shutdown_connections 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 --2- 172.21.15.46:0/4113085139 >> v2:172.21.15.142:3300/0 conn(0x7f97300cdb30 0x7f97300735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 --2- 172.21.15.46:0/4113085139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973015cec0 0x7f97300cd5f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.392+0000 7f9736e00640 1 -- 172.21.15.46:0/4113085139 >> 172.21.15.46:0/4113085139 conn(0x7f97300b27b0 msgr2=0x7f97300b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.393+0000 7f9736e00640 1 -- 172.21.15.46:0/4113085139 shutdown_connections 2024-04-26T23:43:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.393+0000 7f9736e00640 1 -- 172.21.15.46:0/4113085139 wait complete. 2024-04-26T23:43:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.395+0000 7f9736e00640 1 Processor -- start 2024-04-26T23:43:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.395+0000 7f9736e00640 1 -- start start 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97300cdb30 0x7f973015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f973015cec0 0x7f97300cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9730073f70 con 0x7f9730073b30 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f973015d2a0 con 0x7f97300cdb30 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97300b4b50 con 0x7f973015cec0 2024-04-26T23:43:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38248/0 (socket says 172.21.15.46:38248) 2024-04-26T23:43:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 -- 172.21.15.46:0/2964516945 learned_addr learned my addr 172.21.15.46:0/2964516945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9736400640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f973015cec0 0x7f97300cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 -- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f973015cec0 msgr2=0x7f97300cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735000640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97300cdb30 0x7f973015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f973015cec0 0x7f97300cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 -- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97300cdb30 msgr2=0x7f973015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97300cdb30 0x7f973015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.396+0000 7f9735a00640 1 -- 172.21.15.46:0/2964516945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9730155e70 con 0x7f9730073b30 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f9735a00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f9724009890 tx=0x7f97240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724037040 con 0x7f9730073b30 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9724042070 con 0x7f9730073b30 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724030a50 con 0x7f9730073b30 2024-04-26T23:43:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9730156150 con 0x7f9730073b30 2024-04-26T23:43:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.397+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9730156710 con 0x7f9730073b30 2024-04-26T23:43:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.398+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97300b6090 con 0x7f9730073b30 2024-04-26T23:43:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.399+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9724030bf0 con 0x7f9730073b30 2024-04-26T23:43:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.400+0000 7f972d600640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.400+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f97240ac890 con 0x7f9730073b30 2024-04-26T23:43:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.401+0000 7f9735000640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.403+0000 7f9735000640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 0x7f971006edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97300ae9a0 tx=0x7f971c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.405+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97300b6090 con 0x7f9730073b30 2024-04-26T23:43:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:50.655+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f97300693b0 con 0x7f9730073b30 2024-04-26T23:43:51.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.278+0000 7f972d600640 1 -- 172.21.15.46:0/2964516945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) ==== 118+0+0 (secure 0 0 0) 0x7f972407aa90 con 0x7f9730073b30 2024-04-26T23:43:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 msgr2=0x7f971006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 0x7f971006edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97300ae9a0 tx=0x7f971c008040 comp rx=0 tx=0).stop 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 msgr2=0x7f9730154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f9724009890 tx=0x7f97240046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 shutdown_connections 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f973015cec0 0x7f97300cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97300cdb30 0x7f973015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 --2- 172.21.15.46:0/2964516945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730073b30 0x7f9730154160 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.281+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 >> 172.21.15.46:0/2964516945 conn(0x7f97300b27b0 msgr2=0x7f97300b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.282+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 shutdown_connections 2024-04-26T23:43:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.282+0000 7f9736e00640 1 -- 172.21.15.46:0/2964516945 wait complete. 2024-04-26T23:43:51.308 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:43:51.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:43:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.522+0000 7f5084c00640 1 Processor -- start 2024-04-26T23:43:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.522+0000 7f5084c00640 1 -- start start 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.524+0000 7f5084c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5080072620 0x7f50800b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.524+0000 7f5084c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5080073120 con 0x7f50800b5730 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.524+0000 7f5084c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5080159bc0 con 0x7f50801597e0 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.524+0000 7f5084c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f50800aeff0 con 0x7f5080072620 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.524+0000 7f507e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5080072620 0x7f50800b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5080072620 0x7f50800b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38252/0 (socket says 172.21.15.46:38252) 2024-04-26T23:43:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507e800640 1 -- 172.21.15.46:0/2342816919 learned_addr learned my addr 172.21.15.46:0/2342816919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507e800640 1 -- 172.21.15.46:0/2342816919 >> v1:172.21.15.142:6789/0 conn(0x7f50801597e0 legacy=0x7f50800ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507e800640 1 -- 172.21.15.46:0/2342816919 >> v1:172.21.15.46:6789/0 conn(0x7f50800b5730 legacy=0x7f5080071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507ca00640 1 -- 172.21.15.46:0/2342816919 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416967718 0 0) 0x7f5080073120 con 0x7f50800b5730 2024-04-26T23:43:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.525+0000 7f507e800640 1 -- 172.21.15.46:0/2342816919 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5080072b60 con 0x7f5080072620 2024-04-26T23:43:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.526+0000 7f507e800640 1 --2- 172.21.15.46:0/2342816919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50800b5bc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5078009e50 tx=0x7f507802a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadcc2699b4f8f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.526+0000 7f507ca00640 1 -- 172.21.15.46:0/2342816919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078033610 con 0x7f5080072620 2024-04-26T23:43:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f507ca00640 1 -- 172.21.15.46:0/2342816919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507802f070 con 0x7f5080072620 2024-04-26T23:43:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f507ca00640 1 -- 172.21.15.46:0/2342816919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780339d0 con 0x7f5080072620 2024-04-26T23:43:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 -- 172.21.15.46:0/2342816919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 msgr2=0x7f50800b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 --2- 172.21.15.46:0/2342816919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50800b5bc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5078009e50 tx=0x7f507802a810 comp rx=0 tx=0).stop 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 -- 172.21.15.46:0/2342816919 shutdown_connections 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 --2- 172.21.15.46:0/2342816919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50800b5bc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 -- 172.21.15.46:0/2342816919 >> 172.21.15.46:0/2342816919 conn(0x7f508006c980 msgr2=0x7f50800b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 -- 172.21.15.46:0/2342816919 shutdown_connections 2024-04-26T23:43:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.527+0000 7f5084c00640 1 -- 172.21.15.46:0/2342816919 wait complete. 2024-04-26T23:43:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.529+0000 7f5084c00640 1 Processor -- start 2024-04-26T23:43:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.529+0000 7f5084c00640 1 -- start start 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 0x7f50800ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 0x7f50800d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50800b43b0 con 0x7f5080072620 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50800b40d0 con 0x7f50801597e0 2024-04-26T23:43:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f5084c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f50800b48d0 con 0x7f50800b5730 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f507de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f507de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38264/0 (socket says 172.21.15.46:38264) 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.530+0000 7f507de00640 1 -- 172.21.15.46:0/1358466707 learned_addr learned my addr 172.21.15.46:0/1358466707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507e800640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 0x7f50800d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 -- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 msgr2=0x7f50800ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507d400640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 0x7f50800ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 0x7f50800ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 -- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 msgr2=0x7f50800d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 0x7f50800d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 -- 172.21.15.46:0/1358466707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5080146cf0 con 0x7f5080072620 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507e800640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 0x7f50800d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507d400640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 0x7f50800ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.531+0000 7f507de00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f507000b420 tx=0x7f507000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.532+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070011830 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.532+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070016070 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.532+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070011c10 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.532+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50800cd6b0 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.532+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50800cdc70 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.533+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5080159020 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.534+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f507001b460 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.535+0000 7f5076000640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.535+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f507008b7e0 con 0x7f5080072620 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.536+0000 7f507d400640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.537+0000 7f507d400640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 0x7f506006edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f60 tx=0x7f506c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.539+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5080159020 con 0x7f5080072620 2024-04-26T23:43:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:51.785+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5080069530 con 0x7f5080072620 2024-04-26T23:43:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.281+0000 7f5076000640 1 -- 172.21.15.46:0/1358466707 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f5070020020 con 0x7f5080072620 2024-04-26T23:43:52.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.284+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 msgr2=0x7f506006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.284+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 0x7f506006edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f60 tx=0x7f506c009040 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.284+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 msgr2=0x7f50801467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.284+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f507000b420 tx=0x7f507000b8e0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 shutdown_connections 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50801597e0 0x7f50800d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50800b5730 0x7f50800ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 --2- 172.21.15.46:0/1358466707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5080072620 0x7f50801467b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 >> 172.21.15.46:0/1358466707 conn(0x7f508006c980 msgr2=0x7f50800b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 shutdown_connections 2024-04-26T23:43:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.285+0000 7f5084c00640 1 -- 172.21.15.46:0/1358466707 wait complete. 2024-04-26T23:43:52.315 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:43:52.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:52.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.520+0000 7f4f6e400640 1 Processor -- start 2024-04-26T23:43:52.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.520+0000 7f4f6e400640 1 -- start start 2024-04-26T23:43:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f6e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f6e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f680b4000 0x7f4f680d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f6e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f680d11f0 con 0x7f4f680d1f60 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f6e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4f680d2990 con 0x7f4f680b4000 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f6e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f680d5f20 con 0x7f4f6815f4e0 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.522+0000 7f4f67e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f680b4000 0x7f4f680d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f6da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4f680d1f60 0x7f4f680cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47378/0 (socket says 172.21.15.46:47378) 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f67e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f680b4000 0x7f4f680d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46992/0 (socket says 172.21.15.46:46992) 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f6da00640 1 -- 172.21.15.46:0/3755414532 learned_addr learned my addr 172.21.15.46:0/3755414532 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f6d000640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f67e00640 1 -- 172.21.15.46:0/3755414532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 msgr2=0x7f4f680d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f67e00640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f67e00640 1 -- 172.21.15.46:0/3755414532 >> v1:172.21.15.46:6789/0 conn(0x7f4f680d1f60 legacy=0x7f4f680cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.523+0000 7f4f67e00640 1 -- 172.21.15.46:0/3755414532 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f680d0f10 con 0x7f4f680b4000 2024-04-26T23:43:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.524+0000 7f4f67e00640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680b4000 0x7f4f680d1a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb12c2d44d222e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.524+0000 7f4f67400640 1 -- 172.21.15.46:0/3755414532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50037070 con 0x7f4f680b4000 2024-04-26T23:43:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f67400640 1 -- 172.21.15.46:0/3755414532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50032440 con 0x7f4f680b4000 2024-04-26T23:43:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f67400640 1 -- 172.21.15.46:0/3755414532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50032810 con 0x7f4f680b4000 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 -- 172.21.15.46:0/3755414532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680b4000 msgr2=0x7f4f680d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680b4000 0x7f4f680d1a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002a180 comp rx=0 tx=0).stop 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 -- 172.21.15.46:0/3755414532 shutdown_connections 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680b4000 0x7f4f680d1a20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 --2- 172.21.15.46:0/3755414532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 -- 172.21.15.46:0/3755414532 >> 172.21.15.46:0/3755414532 conn(0x7f4f680d5010 msgr2=0x7f4f680d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 -- 172.21.15.46:0/3755414532 shutdown_connections 2024-04-26T23:43:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.525+0000 7f4f6e400640 1 -- 172.21.15.46:0/3755414532 wait complete. 2024-04-26T23:43:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.527+0000 7f4f6e400640 1 Processor -- start 2024-04-26T23:43:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.527+0000 7f4f6e400640 1 -- start start 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f680b4000 0x7f4f680d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680b9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6815f8c0 con 0x7f4f680d1f60 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680b1d60 con 0x7f4f680b4000 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f6e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f680b3290 con 0x7f4f6815f4e0 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47002/0 (socket says 172.21.15.46:47002) 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.528+0000 7f4f67e00640 1 -- 172.21.15.46:0/870479558 learned_addr learned my addr 172.21.15.46:0/870479558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 -- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 msgr2=0x7f4f680b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 -- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f680b4000 msgr2=0x7f4f680d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f680b4000 0x7f4f680d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 -- 172.21.15.46:0/870479558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f680b9e30 con 0x7f4f680d1f60 2024-04-26T23:43:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.529+0000 7f4f67e00640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4f50035040 tx=0x7f4f500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.530+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50037040 con 0x7f4f680d1f60 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.530+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50030770 con 0x7f4f680d1f60 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.530+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50030b80 con 0x7f4f680d1f60 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.530+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f680ba110 con 0x7f4f680d1f60 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.530+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f680cc330 con 0x7f4f680d1f60 2024-04-26T23:43:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.531+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f680693b0 con 0x7f4f680d1f60 2024-04-26T23:43:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.532+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f50044020 con 0x7f4f680d1f60 2024-04-26T23:43:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.533+0000 7f4f64c00640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 0x7f4f4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.533+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f4f500ac230 con 0x7f4f680d1f60 2024-04-26T23:43:52.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.534+0000 7f4f6d000640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 0x7f4f4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:52.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.535+0000 7f4f6d000640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 0x7f4f4406edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f580070b0 tx=0x7f4f58007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.538+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f50032e20 con 0x7f4f680d1f60 2024-04-26T23:43:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.793+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f680d33d0 con 0x7f4f680d1f60 2024-04-26T23:43:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.794+0000 7f4f64c00640 1 -- 172.21.15.46:0/870479558 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7f4f5007a4c0 con 0x7f4f680d1f60 2024-04-26T23:43:52.799 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:52.799 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2819521829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2819521829},{"type":"v1","addr":"172.21.15.46:6834","nonce":2819521829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1317531531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1317531531},{"type":"v1","addr":"172.21.15.46:6835","nonce":1317531531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/634149566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":634149566},{"type":"v1","addr":"172.21.15.142:6836","nonce":634149566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:43:51.787417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2319300947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2319300947},{"type":"v1","addr":"172.21.15.142:6837","nonce":2319300947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7164,"qdb_cluster":[7164]},"id":5}]} 2024-04-26T23:43:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2024-04-26T23:43:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 msgr2=0x7f4f4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 0x7f4f4406edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f580070b0 tx=0x7f4f58007040 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 msgr2=0x7f4f68154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4f50035040 tx=0x7f4f500046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 shutdown_connections 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f4406c8f0 0x7f4f4406edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6815f4e0 0x7f4f680b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f680d1f60 0x7f4f68154510 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 --2- 172.21.15.46:0/870479558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f680b4000 0x7f4f680d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 >> 172.21.15.46:0/870479558 conn(0x7f4f680d5010 msgr2=0x7f4f6806e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 shutdown_connections 2024-04-26T23:43:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:52.798+0000 7f4f6e400640 1 -- 172.21.15.46:0/870479558 wait complete. 2024-04-26T23:43:52.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.028+0000 7feece400640 1 Processor -- start 2024-04-26T23:43:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.029+0000 7feece400640 1 -- start start 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.030+0000 7feece400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feec806e510 0x7feec806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.030+0000 7feece400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 0x7feec8161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.030+0000 7feece400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec806ee30 con 0x7feec815d540 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.030+0000 7feece400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7feec80742d0 con 0x7feec806e510 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.030+0000 7feece400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feec8161940 con 0x7feec8160bd0 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.031+0000 7feec7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feec806e510 0x7feec806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.031+0000 7feec7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feec806e510 0x7feec806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47012/0 (socket says 172.21.15.46:47012) 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.031+0000 7feec7400640 1 -- 172.21.15.46:0/1996218706 learned_addr learned my addr 172.21.15.46:0/1996218706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.031+0000 7feebfe00640 1 --2- 172.21.15.46:0/1996218706 >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 0x7feec8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.032+0000 7feec7400640 1 -- 172.21.15.46:0/1996218706 >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 msgr2=0x7feec8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.032+0000 7feec7400640 1 --2- 172.21.15.46:0/1996218706 >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 0x7feec8161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.032+0000 7feec7400640 1 -- 172.21.15.46:0/1996218706 >> v1:172.21.15.142:6789/0 conn(0x7feec815d540 legacy=0x7feec815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.032+0000 7feec7400640 1 -- 172.21.15.46:0/1996218706 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec8071f80 con 0x7feec806e510 2024-04-26T23:43:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.032+0000 7feebfe00640 1 --2- 172.21.15.46:0/1996218706 >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 0x7feec8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.033+0000 7feec7400640 1 --2- 172.21.15.46:0/1996218706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec806e510 0x7feec806e8f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d4ebec4647415e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feec6a00640 1 -- 172.21.15.46:0/1996218706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8037070 con 0x7feec806e510 2024-04-26T23:43:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feec6a00640 1 -- 172.21.15.46:0/1996218706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8032440 con 0x7feec806e510 2024-04-26T23:43:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feec6a00640 1 -- 172.21.15.46:0/1996218706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8032810 con 0x7feec806e510 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 -- 172.21.15.46:0/1996218706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec806e510 msgr2=0x7feec806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 --2- 172.21.15.46:0/1996218706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec806e510 0x7feec806e8f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 -- 172.21.15.46:0/1996218706 shutdown_connections 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 --2- 172.21.15.46:0/1996218706 >> v2:172.21.15.142:3300/0 conn(0x7feec8160bd0 0x7feec8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 --2- 172.21.15.46:0/1996218706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec806e510 0x7feec806e8f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 -- 172.21.15.46:0/1996218706 >> 172.21.15.46:0/1996218706 conn(0x7feec806c530 msgr2=0x7feec806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.034+0000 7feece400640 1 -- 172.21.15.46:0/1996218706 shutdown_connections 2024-04-26T23:43:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.035+0000 7feece400640 1 -- 172.21.15.46:0/1996218706 wait complete. 2024-04-26T23:43:53.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.036+0000 7feece400640 1 Processor -- start 2024-04-26T23:43:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.036+0000 7feece400640 1 -- start start 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feec806e510 0x7feec815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feec8160bd0 0x7feec80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feec8072820 con 0x7feec815d540 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feec8072540 con 0x7feec806e510 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.037+0000 7feece400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feec81615a0 con 0x7feec8160bd0 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47020/0 (socket says 172.21.15.46:47020) 2024-04-26T23:43:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 -- 172.21.15.46:0/2296759115 learned_addr learned my addr 172.21.15.46:0/2296759115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 -- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feec8160bd0 msgr2=0x7feec80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feec7400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feec806e510 0x7feec815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feec8160bd0 0x7feec80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 -- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feec806e510 msgr2=0x7feec815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feec806e510 0x7feec815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 -- 172.21.15.46:0/2296759115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec815b8a0 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.038+0000 7feebfe00640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7feeb000d7d0 tx=0x7feeb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.039+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0004030 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.039+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb00041d0 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.039+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00096a0 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.039+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec8144880 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.039+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81e7ae0 con 0x7feec815d540 2024-04-26T23:43:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.040+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8069530 con 0x7feec815d540 2024-04-26T23:43:53.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.042+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeb0009840 con 0x7feec815d540 2024-04-26T23:43:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.042+0000 7feebf400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 0x7feea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.043+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7feeb008bf70 con 0x7feec815d540 2024-04-26T23:43:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.043+0000 7feec7400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 0x7feea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.044+0000 7feec7400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 0x7feea406edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feeb8034000 tx=0x7feeb802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.047+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feeb005a210 con 0x7feec815d540 2024-04-26T23:43:53.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.258+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feec806fb00 con 0x7feec815d540 2024-04-26T23:43:53.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.261+0000 7feebf400640 1 -- 172.21.15.46:0/2296759115 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16909 (secure 0 0 0) 0x7feeb0059b10 con 0x7feec815d540 2024-04-26T23:43:53.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:53.266 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:50.237954+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 msgr2=0x7feea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 0x7feea406edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feeb8034000 tx=0x7feeb802e000 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 msgr2=0x7feec81456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7feeb000d7d0 tx=0x7feeb000dc90 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 shutdown_connections 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feea406c8f0 0x7feea406edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feec8160bd0 0x7feec80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec815d540 0x7feec81456a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 --2- 172.21.15.46:0/2296759115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feec806e510 0x7feec815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 >> 172.21.15.46:0/2296759115 conn(0x7feec806c530 msgr2=0x7feec80b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.264+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 shutdown_connections 2024-04-26T23:43:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.265+0000 7feece400640 1 -- 172.21.15.46:0/2296759115 wait complete. 2024-04-26T23:43:53.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:43:53.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:43:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.497+0000 7f6045000640 1 Processor -- start 2024-04-26T23:43:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.497+0000 7f6045000640 1 -- start start 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.498+0000 7f6045000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60400d3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.498+0000 7f6045000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60400b33a0 0x7f60400cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.498+0000 7f6045000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60401647e0 con 0x7f60400b2fc0 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.498+0000 7f6045000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f60400b46b0 con 0x7f60400b33a0 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.498+0000 7f6045000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6040164a80 con 0x7f60400c6450 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60400b2fc0 0x7f60400c5d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47384/0 (socket says 172.21.15.46:47384) 2024-04-26T23:43:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603f400640 1 -- 172.21.15.46:0/1928863436 learned_addr learned my addr 172.21.15.46:0/1928863436 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603fe00640 1 --2- 172.21.15.46:0/1928863436 >> v2:172.21.15.142:3300/0 conn(0x7f60400b33a0 0x7f60400cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603ea00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60400d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491065047 0 0) 0x7f60401647e0 con 0x7f60400b2fc0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.499+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6020003520 con 0x7f60400b2fc0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791433506 0 0) 0x7f6020003520 con 0x7f60400b2fc0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 msgr2=0x7f60400d3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60400d3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 >> v2:172.21.15.142:3300/0 conn(0x7f60400b33a0 msgr2=0x7f60400cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 --2- 172.21.15.46:0/1928863436 >> v2:172.21.15.142:3300/0 conn(0x7f60400b33a0 0x7f60400cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604006d460 con 0x7f60400b2fc0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6030001670 con 0x7f60400b2fc0 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 >> v1:172.21.15.46:6789/0 conn(0x7f60400b2fc0 legacy=0x7f60400c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6020003f30 0x7f6020004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6020004d40 0x7f60200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60400b46b0 con 0x7f6020003f30 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6040164a80 con 0x7f6020004d40 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f602000b9c0 con 0x7f60400c6450 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.500+0000 7f603ea00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6020003f30 0x7f6020004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603fe00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6020004d40 0x7f60200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603f400640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603ea00640 1 -- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 msgr2=0x7f60200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603ea00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603ea00640 1 -- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6020004d40 msgr2=0x7f60200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603ea00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6020004d40 0x7f60200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603ea00640 1 -- 172.21.15.46:0/1928863436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6020003520 con 0x7f6020003f30 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603f400640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60400c6450 0x7f60200039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.501+0000 7f603fe00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6020004d40 0x7f60200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.502+0000 7f603ea00640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6020003f30 0x7f6020004800 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fc0 tx=0x7f6020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57d883b133a306 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.502+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028009c10 con 0x7f6020003f30 2024-04-26T23:43:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.502+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6028036070 con 0x7f6020003f30 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.502+0000 7f603e000640 1 -- 172.21.15.46:0/1928863436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028031850 con 0x7f6020003f30 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 -- 172.21.15.46:0/1928863436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6020003f30 msgr2=0x7f6020004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 --2- 172.21.15.46:0/1928863436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6020003f30 0x7f6020004800 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fc0 tx=0x7f6020005830 comp rx=0 tx=0).stop 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f603fe00640 1 -- 172.21.15.46:0/1928863436 reap_dead start 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 -- 172.21.15.46:0/1928863436 shutdown_connections 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 -- 172.21.15.46:0/1928863436 >> 172.21.15.46:0/1928863436 conn(0x7f60400d2d70 msgr2=0x7f60400d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 -- 172.21.15.46:0/1928863436 shutdown_connections 2024-04-26T23:43:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.503+0000 7f6045000640 1 -- 172.21.15.46:0/1928863436 wait complete. 2024-04-26T23:43:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.505+0000 7f6045000640 1 Processor -- start 2024-04-26T23:43:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.505+0000 7f6045000640 1 -- start start 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6020003f30 0x7f6040162830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f604015af80 0x7f604015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60400cd3c0 con 0x7f60400b0c70 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f604006da20 con 0x7f604015af80 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f6045000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60400b3dd0 con 0x7f6020003f30 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f603fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f603fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47052/0 (socket says 172.21.15.46:47052) 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.506+0000 7f603fe00640 1 -- 172.21.15.46:0/1492327232 learned_addr learned my addr 172.21.15.46:0/1492327232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603ea00640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f604015af80 0x7f604015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603f400640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6020003f30 0x7f6040162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 -- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6020003f30 msgr2=0x7f6040162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6020003f30 0x7f6040162830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 -- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f604015af80 msgr2=0x7f604015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f604015af80 0x7f604015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 -- 172.21.15.46:0/1492327232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604015b940 con 0x7f60400b0c70 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f603fe00640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f60400b37b0 tx=0x7f6038010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800f040 con 0x7f60400b0c70 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603801c070 con 0x7f60400b0c70 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6040162d70 con 0x7f60400b0c70 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.507+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800eaf0 con 0x7f60400b0c70 2024-04-26T23:43:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.508+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60400b19e0 con 0x7f60400b0c70 2024-04-26T23:43:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.509+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60400ca6d0 con 0x7f60400b0c70 2024-04-26T23:43:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.510+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f603800ec90 con 0x7f60400b0c70 2024-04-26T23:43:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.511+0000 7f6037400640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 0x7f601c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.511+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f603808b860 con 0x7f60400b0c70 2024-04-26T23:43:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.511+0000 7f603f400640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 0x7f601c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.513+0000 7f603f400640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 0x7f601c06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6030007990 tx=0x7f6030009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.515+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60400ca6d0 con 0x7f60400b0c70 2024-04-26T23:43:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.714+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6040069ac0 con 0x7f601c06c920 2024-04-26T23:43:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.734+0000 7f6037400640 1 -- 172.21.15.46:0/1492327232 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252048 (secure 0 0 0) 0x7f6040069ac0 con 0x7f601c06c920 2024-04-26T23:43:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:53.743 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.738+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 msgr2=0x7f601c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.738+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 0x7f601c06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6030007990 tx=0x7f6030009040 comp rx=0 tx=0).stop 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.738+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 msgr2=0x7f60400b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.738+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f60400b37b0 tx=0x7f6038010c20 comp rx=0 tx=0).stop 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 shutdown_connections 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601c06c920 0x7f601c06ede0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400b0c70 0x7f60400b10f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f604015af80 0x7f604015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 --2- 172.21.15.46:0/1492327232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6020003f30 0x7f6040162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:53.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 >> 172.21.15.46:0/1492327232 conn(0x7f60400d2d70 msgr2=0x7f60400d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:53.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 shutdown_connections 2024-04-26T23:43:53.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:53.739+0000 7f6045000640 1 -- 172.21.15.46:0/1492327232 wait complete. 2024-04-26T23:43:53.762 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-04-26T23:43:52.036167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":312380,"kb_used_data":8824,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754662340,"statfs":{"total":773094113280,"available":772774236160,"internally_reserved":0,"allocated":9035776,"data_stored":4756692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":310693425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.817024"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401538+0000","last_change":"2024-04-26T23:43:49.281640+0000","last_active":"2024-04-26T23:43:50.401538+0000","last_peered":"2024-04-26T23:43:50.401538+0000","last_clean":"2024-04-26T23:43:50.401538+0000","last_became_active":"2024-04-26T23:43:49.281328+0000","last_became_peered":"2024-04-26T23:43:49.281328+0000","last_unstale":"2024-04-26T23:43:50.401538+0000","last_undegraded":"2024-04-26T23:43:50.401538+0000","last_fullsized":"2024-04-26T23:43:50.401538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245873+0000","last_change":"2024-04-26T23:43:49.276811+0000","last_active":"2024-04-26T23:43:50.245873+0000","last_peered":"2024-04-26T23:43:50.245873+0000","last_clean":"2024-04-26T23:43:50.245873+0000","last_became_active":"2024-04-26T23:43:49.271168+0000","last_became_peered":"2024-04-26T23:43:49.271168+0000","last_unstale":"2024-04-26T23:43:50.245873+0000","last_undegraded":"2024-04-26T23:43:50.245873+0000","last_fullsized":"2024-04-26T23:43:50.245873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:21.981490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.289989+0000","last_change":"2024-04-26T23:43:49.282871+0000","last_active":"2024-04-26T23:43:49.289989+0000","last_peered":"2024-04-26T23:43:49.289989+0000","last_clean":"2024-04-26T23:43:49.289989+0000","last_became_active":"2024-04-26T23:43:49.282474+0000","last_became_peered":"2024-04-26T23:43:49.282474+0000","last_unstale":"2024-04-26T23:43:49.289989+0000","last_undegraded":"2024-04-26T23:43:49.289989+0000","last_fullsized":"2024-04-26T23:43:49.289989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:57.480439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.269859+0000","last_change":"2024-04-26T23:43:49.264517+0000","last_active":"2024-04-26T23:43:49.269859+0000","last_peered":"2024-04-26T23:43:49.269859+0000","last_clean":"2024-04-26T23:43:49.269859+0000","last_became_active":"2024-04-26T23:43:49.264246+0000","last_became_peered":"2024-04-26T23:43:49.264246+0000","last_unstale":"2024-04-26T23:43:49.269859+0000","last_undegraded":"2024-04-26T23:43:49.269859+0000","last_fullsized":"2024-04-26T23:43:49.269859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:48.686834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380412+0000","last_change":"2024-04-26T23:43:49.285506+0000","last_active":"2024-04-26T23:43:50.380412+0000","last_peered":"2024-04-26T23:43:50.380412+0000","last_clean":"2024-04-26T23:43:50.380412+0000","last_became_active":"2024-04-26T23:43:49.285205+0000","last_became_peered":"2024-04-26T23:43:49.285205+0000","last_unstale":"2024-04-26T23:43:50.380412+0000","last_undegraded":"2024-04-26T23:43:50.380412+0000","last_fullsized":"2024-04-26T23:43:50.380412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:00.043535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248647+0000","last_change":"2024-04-26T23:43:49.283827+0000","last_active":"2024-04-26T23:43:50.248647+0000","last_peered":"2024-04-26T23:43:50.248647+0000","last_clean":"2024-04-26T23:43:50.248647+0000","last_became_active":"2024-04-26T23:43:49.283313+0000","last_became_peered":"2024-04-26T23:43:49.283313+0000","last_unstale":"2024-04-26T23:43:50.248647+0000","last_undegraded":"2024-04-26T23:43:50.248647+0000","last_fullsized":"2024-04-26T23:43:50.248647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.283065+0000","last_change":"2024-04-26T23:43:49.277515+0000","last_active":"2024-04-26T23:43:49.283065+0000","last_peered":"2024-04-26T23:43:49.283065+0000","last_clean":"2024-04-26T23:43:49.283065+0000","last_became_active":"2024-04-26T23:43:49.277142+0000","last_became_peered":"2024-04-26T23:43:49.277142+0000","last_unstale":"2024-04-26T23:43:49.283065+0000","last_undegraded":"2024-04-26T23:43:49.283065+0000","last_fullsized":"2024-04-26T23:43:49.283065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:27.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380028+0000","last_change":"2024-04-26T23:43:49.278317+0000","last_active":"2024-04-26T23:43:50.380028+0000","last_peered":"2024-04-26T23:43:50.380028+0000","last_clean":"2024-04-26T23:43:50.380028+0000","last_became_active":"2024-04-26T23:43:49.278029+0000","last_became_peered":"2024-04-26T23:43:49.278029+0000","last_unstale":"2024-04-26T23:43:50.380028+0000","last_undegraded":"2024-04-26T23:43:50.380028+0000","last_fullsized":"2024-04-26T23:43:50.380028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:44.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248371+0000","last_change":"2024-04-26T23:43:49.276323+0000","last_active":"2024-04-26T23:43:50.248371+0000","last_peered":"2024-04-26T23:43:50.248371+0000","last_clean":"2024-04-26T23:43:50.248371+0000","last_became_active":"2024-04-26T23:43:49.276029+0000","last_became_peered":"2024-04-26T23:43:49.276029+0000","last_unstale":"2024-04-26T23:43:50.248371+0000","last_undegraded":"2024-04-26T23:43:50.248371+0000","last_fullsized":"2024-04-26T23:43:50.248371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:57.480439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.264948+0000","last_change":"2024-04-26T23:43:49.260059+0000","last_active":"2024-04-26T23:43:49.264948+0000","last_peered":"2024-04-26T23:43:49.264948+0000","last_clean":"2024-04-26T23:43:49.264948+0000","last_became_active":"2024-04-26T23:43:49.259786+0000","last_became_peered":"2024-04-26T23:43:49.259786+0000","last_unstale":"2024-04-26T23:43:49.264948+0000","last_undegraded":"2024-04-26T23:43:49.264948+0000","last_fullsized":"2024-04-26T23:43:49.264948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:06.143662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245898+0000","last_change":"2024-04-26T23:43:49.273639+0000","last_active":"2024-04-26T23:43:50.245898+0000","last_peered":"2024-04-26T23:43:50.245898+0000","last_clean":"2024-04-26T23:43:50.245898+0000","last_became_active":"2024-04-26T23:43:49.273373+0000","last_became_peered":"2024-04-26T23:43:49.273373+0000","last_unstale":"2024-04-26T23:43:50.245898+0000","last_undegraded":"2024-04-26T23:43:50.245898+0000","last_fullsized":"2024-04-26T23:43:50.245898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:35.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.263453+0000","last_change":"2024-04-26T23:43:49.258202+0000","last_active":"2024-04-26T23:43:49.263453+0000","last_peered":"2024-04-26T23:43:49.263453+0000","last_clean":"2024-04-26T23:43:49.263453+0000","last_became_active":"2024-04-26T23:43:49.257848+0000","last_became_peered":"2024-04-26T23:43:49.257848+0000","last_unstale":"2024-04-26T23:43:49.263453+0000","last_undegraded":"2024-04-26T23:43:49.263453+0000","last_fullsized":"2024-04-26T23:43:49.263453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:52.149622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380533+0000","last_change":"2024-04-26T23:43:49.277903+0000","last_active":"2024-04-26T23:43:50.380533+0000","last_peered":"2024-04-26T23:43:50.380533+0000","last_clean":"2024-04-26T23:43:50.380533+0000","last_became_active":"2024-04-26T23:43:49.277420+0000","last_became_peered":"2024-04-26T23:43:49.277420+0000","last_unstale":"2024-04-26T23:43:50.380533+0000","last_undegraded":"2024-04-26T23:43:50.380533+0000","last_fullsized":"2024-04-26T23:43:50.380533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:02.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248720+0000","last_change":"2024-04-26T23:43:49.265154+0000","last_active":"2024-04-26T23:43:50.248720+0000","last_peered":"2024-04-26T23:43:50.248720+0000","last_clean":"2024-04-26T23:43:50.248720+0000","last_became_active":"2024-04-26T23:43:49.264872+0000","last_became_peered":"2024-04-26T23:43:49.264872+0000","last_unstale":"2024-04-26T23:43:50.248720+0000","last_undegraded":"2024-04-26T23:43:50.248720+0000","last_fullsized":"2024-04-26T23:43:50.248720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246235+0000","last_change":"2024-04-26T23:43:49.281969+0000","last_active":"2024-04-26T23:43:50.246235+0000","last_peered":"2024-04-26T23:43:50.246235+0000","last_clean":"2024-04-26T23:43:50.246235+0000","last_became_active":"2024-04-26T23:43:49.281660+0000","last_became_peered":"2024-04-26T23:43:49.281660+0000","last_unstale":"2024-04-26T23:43:50.246235+0000","last_undegraded":"2024-04-26T23:43:50.246235+0000","last_fullsized":"2024-04-26T23:43:50.246235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:40.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401115+0000","last_change":"2024-04-26T23:43:49.267035+0000","last_active":"2024-04-26T23:43:50.401115+0000","last_peered":"2024-04-26T23:43:50.401115+0000","last_clean":"2024-04-26T23:43:50.401115+0000","last_became_active":"2024-04-26T23:43:49.266694+0000","last_became_peered":"2024-04-26T23:43:49.266694+0000","last_unstale":"2024-04-26T23:43:50.401115+0000","last_undegraded":"2024-04-26T23:43:50.401115+0000","last_fullsized":"2024-04-26T23:43:50.401115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381054+0000","last_change":"2024-04-26T23:43:49.277132+0000","last_active":"2024-04-26T23:43:50.381054+0000","last_peered":"2024-04-26T23:43:50.381054+0000","last_clean":"2024-04-26T23:43:50.381054+0000","last_became_active":"2024-04-26T23:43:49.272417+0000","last_became_peered":"2024-04-26T23:43:49.272417+0000","last_unstale":"2024-04-26T23:43:50.381054+0000","last_undegraded":"2024-04-26T23:43:50.381054+0000","last_fullsized":"2024-04-26T23:43:50.381054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:48.386587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291189+0000","last_change":"2024-04-26T23:43:49.289141+0000","last_active":"2024-04-26T23:43:49.291189+0000","last_peered":"2024-04-26T23:43:49.291189+0000","last_clean":"2024-04-26T23:43:49.291189+0000","last_became_active":"2024-04-26T23:43:49.287044+0000","last_became_peered":"2024-04-26T23:43:49.287044+0000","last_unstale":"2024-04-26T23:43:49.291189+0000","last_undegraded":"2024-04-26T23:43:49.291189+0000","last_fullsized":"2024-04-26T23:43:49.291189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:17.305998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249523+0000","last_change":"2024-04-26T23:43:49.270776+0000","last_active":"2024-04-26T23:43:50.249523+0000","last_peered":"2024-04-26T23:43:50.249523+0000","last_clean":"2024-04-26T23:43:50.249523+0000","last_became_active":"2024-04-26T23:43:49.270390+0000","last_became_peered":"2024-04-26T23:43:49.270390+0000","last_unstale":"2024-04-26T23:43:50.249523+0000","last_undegraded":"2024-04-26T23:43:50.249523+0000","last_fullsized":"2024-04-26T23:43:50.249523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381105+0000","last_change":"2024-04-26T23:43:49.279449+0000","last_active":"2024-04-26T23:43:50.381105+0000","last_peered":"2024-04-26T23:43:50.381105+0000","last_clean":"2024-04-26T23:43:50.381105+0000","last_became_active":"2024-04-26T23:43:49.279150+0000","last_became_peered":"2024-04-26T23:43:49.279150+0000","last_unstale":"2024-04-26T23:43:50.381105+0000","last_undegraded":"2024-04-26T23:43:50.381105+0000","last_fullsized":"2024-04-26T23:43:50.381105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:55.389994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247042+0000","last_change":"2024-04-26T23:43:49.270369+0000","last_active":"2024-04-26T23:43:50.247042+0000","last_peered":"2024-04-26T23:43:50.247042+0000","last_clean":"2024-04-26T23:43:50.247042+0000","last_became_active":"2024-04-26T23:43:49.270048+0000","last_became_peered":"2024-04-26T23:43:49.270048+0000","last_unstale":"2024-04-26T23:43:50.247042+0000","last_undegraded":"2024-04-26T23:43:50.247042+0000","last_fullsized":"2024-04-26T23:43:50.247042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:10.859716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380370+0000","last_change":"2024-04-26T23:43:49.279930+0000","last_active":"2024-04-26T23:43:50.380370+0000","last_peered":"2024-04-26T23:43:50.380370+0000","last_clean":"2024-04-26T23:43:50.380370+0000","last_became_active":"2024-04-26T23:43:49.277904+0000","last_became_peered":"2024-04-26T23:43:49.277904+0000","last_unstale":"2024-04-26T23:43:50.380370+0000","last_undegraded":"2024-04-26T23:43:50.380370+0000","last_fullsized":"2024-04-26T23:43:50.380370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:41.993455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246915+0000","last_change":"2024-04-26T23:43:49.266390+0000","last_active":"2024-04-26T23:43:50.246915+0000","last_peered":"2024-04-26T23:43:50.246915+0000","last_clean":"2024-04-26T23:43:50.246915+0000","last_became_active":"2024-04-26T23:43:49.266063+0000","last_became_peered":"2024-04-26T23:43:49.266063+0000","last_unstale":"2024-04-26T23:43:50.246915+0000","last_undegraded":"2024-04-26T23:43:50.246915+0000","last_fullsized":"2024-04-26T23:43:50.246915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:20.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245215+0000","last_change":"2024-04-26T23:43:49.282643+0000","last_active":"2024-04-26T23:43:50.245215+0000","last_peered":"2024-04-26T23:43:50.245215+0000","last_clean":"2024-04-26T23:43:50.245215+0000","last_became_active":"2024-04-26T23:43:49.282010+0000","last_became_peered":"2024-04-26T23:43:49.282010+0000","last_unstale":"2024-04-26T23:43:50.245215+0000","last_undegraded":"2024-04-26T23:43:50.245215+0000","last_fullsized":"2024-04-26T23:43:50.245215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245377+0000","last_change":"2024-04-26T23:43:49.267041+0000","last_active":"2024-04-26T23:43:50.245377+0000","last_peered":"2024-04-26T23:43:50.245377+0000","last_clean":"2024-04-26T23:43:50.245377+0000","last_became_active":"2024-04-26T23:43:49.266054+0000","last_became_peered":"2024-04-26T23:43:49.266054+0000","last_unstale":"2024-04-26T23:43:50.245377+0000","last_undegraded":"2024-04-26T23:43:50.245377+0000","last_fullsized":"2024-04-26T23:43:50.245377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.264922+0000","last_change":"2024-04-26T23:43:49.259445+0000","last_active":"2024-04-26T23:43:49.264922+0000","last_peered":"2024-04-26T23:43:49.264922+0000","last_clean":"2024-04-26T23:43:49.264922+0000","last_became_active":"2024-04-26T23:43:49.257127+0000","last_became_peered":"2024-04-26T23:43:49.257127+0000","last_unstale":"2024-04-26T23:43:49.264922+0000","last_undegraded":"2024-04-26T23:43:49.264922+0000","last_fullsized":"2024-04-26T23:43:49.264922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:43:03.059431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401101+0000","last_change":"2024-04-26T23:43:49.273281+0000","last_active":"2024-04-26T23:43:50.401101+0000","last_peered":"2024-04-26T23:43:50.401101+0000","last_clean":"2024-04-26T23:43:50.401101+0000","last_became_active":"2024-04-26T23:43:49.272814+0000","last_became_peered":"2024-04-26T23:43:49.272814+0000","last_unstale":"2024-04-26T23:43:50.401101+0000","last_undegraded":"2024-04-26T23:43:50.401101+0000","last_fullsized":"2024-04-26T23:43:50.401101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381067+0000","last_change":"2024-04-26T23:43:49.277373+0000","last_active":"2024-04-26T23:43:50.381067+0000","last_peered":"2024-04-26T23:43:50.381067+0000","last_clean":"2024-04-26T23:43:50.381067+0000","last_became_active":"2024-04-26T23:43:49.277041+0000","last_became_peered":"2024-04-26T23:43:49.277041+0000","last_unstale":"2024-04-26T23:43:50.381067+0000","last_undegraded":"2024-04-26T23:43:50.381067+0000","last_fullsized":"2024-04-26T23:43:50.381067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:08.299501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.244878+0000","last_change":"2024-04-26T23:43:49.260106+0000","last_active":"2024-04-26T23:43:50.244878+0000","last_peered":"2024-04-26T23:43:50.244878+0000","last_clean":"2024-04-26T23:43:50.244878+0000","last_became_active":"2024-04-26T23:43:49.258993+0000","last_became_peered":"2024-04-26T23:43:49.258993+0000","last_unstale":"2024-04-26T23:43:50.244878+0000","last_undegraded":"2024-04-26T23:43:50.244878+0000","last_fullsized":"2024-04-26T23:43:50.244878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:50.053540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400663+0000","last_change":"2024-04-26T23:43:49.266824+0000","last_active":"2024-04-26T23:43:50.400663+0000","last_peered":"2024-04-26T23:43:50.400663+0000","last_clean":"2024-04-26T23:43:50.400663+0000","last_became_active":"2024-04-26T23:43:49.266432+0000","last_became_peered":"2024-04-26T23:43:49.266432+0000","last_unstale":"2024-04-26T23:43:50.400663+0000","last_undegraded":"2024-04-26T23:43:50.400663+0000","last_fullsized":"2024-04-26T23:43:50.400663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291163+0000","last_change":"2024-04-26T23:43:49.274213+0000","last_active":"2024-04-26T23:43:49.291163+0000","last_peered":"2024-04-26T23:43:49.291163+0000","last_clean":"2024-04-26T23:43:49.291163+0000","last_became_active":"2024-04-26T23:43:49.273801+0000","last_became_peered":"2024-04-26T23:43:49.273801+0000","last_unstale":"2024-04-26T23:43:49.291163+0000","last_undegraded":"2024-04-26T23:43:49.291163+0000","last_fullsized":"2024-04-26T23:43:49.291163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400907+0000","last_change":"2024-04-26T23:43:49.278712+0000","last_active":"2024-04-26T23:43:50.400907+0000","last_peered":"2024-04-26T23:43:50.400907+0000","last_clean":"2024-04-26T23:43:50.400907+0000","last_became_active":"2024-04-26T23:43:49.278072+0000","last_became_peered":"2024-04-26T23:43:49.278072+0000","last_unstale":"2024-04-26T23:43:50.400907+0000","last_undegraded":"2024-04-26T23:43:50.400907+0000","last_fullsized":"2024-04-26T23:43:50.400907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.289910+0000","last_change":"2024-04-26T23:43:49.285984+0000","last_active":"2024-04-26T23:43:49.289910+0000","last_peered":"2024-04-26T23:43:49.289910+0000","last_clean":"2024-04-26T23:43:49.289910+0000","last_became_active":"2024-04-26T23:43:49.285690+0000","last_became_peered":"2024-04-26T23:43:49.285690+0000","last_unstale":"2024-04-26T23:43:49.289910+0000","last_undegraded":"2024-04-26T23:43:49.289910+0000","last_fullsized":"2024-04-26T23:43:49.289910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:18.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380998+0000","last_change":"2024-04-26T23:43:48.278182+0000","last_active":"2024-04-26T23:43:50.380998+0000","last_peered":"2024-04-26T23:43:50.380998+0000","last_clean":"2024-04-26T23:43:50.380998+0000","last_became_active":"2024-04-26T23:43:48.277955+0000","last_became_peered":"2024-04-26T23:43:48.277955+0000","last_unstale":"2024-04-26T23:43:50.380998+0000","last_undegraded":"2024-04-26T23:43:50.380998+0000","last_fullsized":"2024-04-26T23:43:50.380998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:35.498033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245406+0000","last_change":"2024-04-26T23:43:49.259791+0000","last_active":"2024-04-26T23:43:50.245406+0000","last_peered":"2024-04-26T23:43:50.245406+0000","last_clean":"2024-04-26T23:43:50.245406+0000","last_became_active":"2024-04-26T23:43:49.258639+0000","last_became_peered":"2024-04-26T23:43:49.258639+0000","last_unstale":"2024-04-26T23:43:50.245406+0000","last_undegraded":"2024-04-26T23:43:50.245406+0000","last_fullsized":"2024-04-26T23:43:50.245406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401306+0000","last_change":"2024-04-26T23:43:48.267027+0000","last_active":"2024-04-26T23:43:50.401306+0000","last_peered":"2024-04-26T23:43:50.401306+0000","last_clean":"2024-04-26T23:43:50.401306+0000","last_became_active":"2024-04-26T23:43:48.266726+0000","last_became_peered":"2024-04-26T23:43:48.266726+0000","last_unstale":"2024-04-26T23:43:50.401306+0000","last_undegraded":"2024-04-26T23:43:50.401306+0000","last_fullsized":"2024-04-26T23:43:50.401306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:53.216357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247363+0000","last_change":"2024-04-26T23:43:49.276183+0000","last_active":"2024-04-26T23:43:50.247363+0000","last_peered":"2024-04-26T23:43:50.247363+0000","last_clean":"2024-04-26T23:43:50.247363+0000","last_became_active":"2024-04-26T23:43:49.275898+0000","last_became_peered":"2024-04-26T23:43:49.275898+0000","last_unstale":"2024-04-26T23:43:50.247363+0000","last_undegraded":"2024-04-26T23:43:50.247363+0000","last_fullsized":"2024-04-26T23:43:50.247363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401113+0000","last_change":"2024-04-26T23:43:48.262249+0000","last_active":"2024-04-26T23:43:50.401113+0000","last_peered":"2024-04-26T23:43:50.401113+0000","last_clean":"2024-04-26T23:43:50.401113+0000","last_became_active":"2024-04-26T23:43:48.261963+0000","last_became_peered":"2024-04-26T23:43:48.261963+0000","last_unstale":"2024-04-26T23:43:50.401113+0000","last_undegraded":"2024-04-26T23:43:50.401113+0000","last_fullsized":"2024-04-26T23:43:50.401113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:24.385420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247422+0000","last_change":"2024-04-26T23:43:49.290218+0000","last_active":"2024-04-26T23:43:50.247422+0000","last_peered":"2024-04-26T23:43:50.247422+0000","last_clean":"2024-04-26T23:43:50.247422+0000","last_became_active":"2024-04-26T23:43:49.289919+0000","last_became_peered":"2024-04-26T23:43:49.289919+0000","last_unstale":"2024-04-26T23:43:50.247422+0000","last_undegraded":"2024-04-26T23:43:50.247422+0000","last_fullsized":"2024-04-26T23:43:50.247422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:45.654839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282712+0000","last_change":"2024-04-26T23:43:48.269395+0000","last_active":"2024-04-26T23:43:49.282712+0000","last_peered":"2024-04-26T23:43:49.282712+0000","last_clean":"2024-04-26T23:43:49.282712+0000","last_became_active":"2024-04-26T23:43:48.268281+0000","last_became_peered":"2024-04-26T23:43:48.268281+0000","last_unstale":"2024-04-26T23:43:49.282712+0000","last_undegraded":"2024-04-26T23:43:49.282712+0000","last_fullsized":"2024-04-26T23:43:49.282712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:01.194903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245377+0000","last_change":"2024-04-26T23:43:49.276454+0000","last_active":"2024-04-26T23:43:50.245377+0000","last_peered":"2024-04-26T23:43:50.245377+0000","last_clean":"2024-04-26T23:43:50.245377+0000","last_became_active":"2024-04-26T23:43:49.276167+0000","last_became_peered":"2024-04-26T23:43:49.276167+0000","last_unstale":"2024-04-26T23:43:50.245377+0000","last_undegraded":"2024-04-26T23:43:50.245377+0000","last_fullsized":"2024-04-26T23:43:50.245377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247802+0000","last_change":"2024-04-26T23:43:48.261837+0000","last_active":"2024-04-26T23:43:50.247802+0000","last_peered":"2024-04-26T23:43:50.247802+0000","last_clean":"2024-04-26T23:43:50.247802+0000","last_became_active":"2024-04-26T23:43:48.261551+0000","last_became_peered":"2024-04-26T23:43:48.261551+0000","last_unstale":"2024-04-26T23:43:50.247802+0000","last_undegraded":"2024-04-26T23:43:50.247802+0000","last_fullsized":"2024-04-26T23:43:50.247802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:18.700430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246064+0000","last_change":"2024-04-26T23:43:49.271363+0000","last_active":"2024-04-26T23:43:50.246064+0000","last_peered":"2024-04-26T23:43:50.246064+0000","last_clean":"2024-04-26T23:43:50.246064+0000","last_became_active":"2024-04-26T23:43:49.270409+0000","last_became_peered":"2024-04-26T23:43:49.270409+0000","last_unstale":"2024-04-26T23:43:50.246064+0000","last_undegraded":"2024-04-26T23:43:50.246064+0000","last_fullsized":"2024-04-26T23:43:50.246064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:13.511140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288607+0000","last_change":"2024-04-26T23:43:48.272655+0000","last_active":"2024-04-26T23:43:49.288607+0000","last_peered":"2024-04-26T23:43:49.288607+0000","last_clean":"2024-04-26T23:43:49.288607+0000","last_became_active":"2024-04-26T23:43:48.272213+0000","last_became_peered":"2024-04-26T23:43:48.272213+0000","last_unstale":"2024-04-26T23:43:49.288607+0000","last_undegraded":"2024-04-26T23:43:49.288607+0000","last_fullsized":"2024-04-26T23:43:49.288607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:13.642533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247588+0000","last_change":"2024-04-26T23:43:49.271469+0000","last_active":"2024-04-26T23:43:50.247588+0000","last_peered":"2024-04-26T23:43:50.247588+0000","last_clean":"2024-04-26T23:43:50.247588+0000","last_became_active":"2024-04-26T23:43:49.266320+0000","last_became_peered":"2024-04-26T23:43:49.266320+0000","last_unstale":"2024-04-26T23:43:50.247588+0000","last_undegraded":"2024-04-26T23:43:50.247588+0000","last_fullsized":"2024-04-26T23:43:50.247588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245637+0000","last_change":"2024-04-26T23:43:48.257801+0000","last_active":"2024-04-26T23:43:50.245637+0000","last_peered":"2024-04-26T23:43:50.245637+0000","last_clean":"2024-04-26T23:43:50.245637+0000","last_became_active":"2024-04-26T23:43:48.257201+0000","last_became_peered":"2024-04-26T23:43:48.257201+0000","last_unstale":"2024-04-26T23:43:50.245637+0000","last_undegraded":"2024-04-26T23:43:50.245637+0000","last_fullsized":"2024-04-26T23:43:50.245637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:05.126550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288139+0000","last_change":"2024-04-26T23:43:49.286057+0000","last_active":"2024-04-26T23:43:49.288139+0000","last_peered":"2024-04-26T23:43:49.288139+0000","last_clean":"2024-04-26T23:43:49.288139+0000","last_became_active":"2024-04-26T23:43:49.285728+0000","last_became_peered":"2024-04-26T23:43:49.285728+0000","last_unstale":"2024-04-26T23:43:49.288139+0000","last_undegraded":"2024-04-26T23:43:49.288139+0000","last_fullsized":"2024-04-26T23:43:49.288139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380142+0000","last_change":"2024-04-26T23:43:48.270525+0000","last_active":"2024-04-26T23:43:50.380142+0000","last_peered":"2024-04-26T23:43:50.380142+0000","last_clean":"2024-04-26T23:43:50.380142+0000","last_became_active":"2024-04-26T23:43:48.269325+0000","last_became_peered":"2024-04-26T23:43:48.269325+0000","last_unstale":"2024-04-26T23:43:50.380142+0000","last_undegraded":"2024-04-26T23:43:50.380142+0000","last_fullsized":"2024-04-26T23:43:50.380142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:05.144529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245740+0000","last_change":"2024-04-26T23:43:49.268453+0000","last_active":"2024-04-26T23:43:50.245740+0000","last_peered":"2024-04-26T23:43:50.245740+0000","last_clean":"2024-04-26T23:43:50.245740+0000","last_became_active":"2024-04-26T23:43:49.268164+0000","last_became_peered":"2024-04-26T23:43:49.268164+0000","last_unstale":"2024-04-26T23:43:50.245740+0000","last_undegraded":"2024-04-26T23:43:50.245740+0000","last_fullsized":"2024-04-26T23:43:50.245740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:05:17.580086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401578+0000","last_change":"2024-04-26T23:43:48.267082+0000","last_active":"2024-04-26T23:43:50.401578+0000","last_peered":"2024-04-26T23:43:50.401578+0000","last_clean":"2024-04-26T23:43:50.401578+0000","last_became_active":"2024-04-26T23:43:48.266766+0000","last_became_peered":"2024-04-26T23:43:48.266766+0000","last_unstale":"2024-04-26T23:43:50.401578+0000","last_undegraded":"2024-04-26T23:43:50.401578+0000","last_fullsized":"2024-04-26T23:43:50.401578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.027181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.263928+0000","last_change":"2024-04-26T23:43:49.258867+0000","last_active":"2024-04-26T23:43:49.263928+0000","last_peered":"2024-04-26T23:43:49.263928+0000","last_clean":"2024-04-26T23:43:49.263928+0000","last_became_active":"2024-04-26T23:43:49.258563+0000","last_became_peered":"2024-04-26T23:43:49.258563+0000","last_unstale":"2024-04-26T23:43:49.263928+0000","last_undegraded":"2024-04-26T23:43:49.263928+0000","last_fullsized":"2024-04-26T23:43:49.263928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:35.068481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.253140+0000","last_change":"2024-04-26T23:43:48.258153+0000","last_active":"2024-04-26T23:43:49.253140+0000","last_peered":"2024-04-26T23:43:49.253140+0000","last_clean":"2024-04-26T23:43:49.253140+0000","last_became_active":"2024-04-26T23:43:48.257819+0000","last_became_peered":"2024-04-26T23:43:48.257819+0000","last_unstale":"2024-04-26T23:43:49.253140+0000","last_undegraded":"2024-04-26T23:43:49.253140+0000","last_fullsized":"2024-04-26T23:43:49.253140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:24.844178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248954+0000","last_change":"2024-04-26T23:43:49.285079+0000","last_active":"2024-04-26T23:43:50.248954+0000","last_peered":"2024-04-26T23:43:50.248954+0000","last_clean":"2024-04-26T23:43:50.248954+0000","last_became_active":"2024-04-26T23:43:49.284741+0000","last_became_peered":"2024-04-26T23:43:49.284741+0000","last_unstale":"2024-04-26T23:43:50.248954+0000","last_undegraded":"2024-04-26T23:43:50.248954+0000","last_fullsized":"2024-04-26T23:43:50.248954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380866+0000","last_change":"2024-04-26T23:43:48.274029+0000","last_active":"2024-04-26T23:43:50.380866+0000","last_peered":"2024-04-26T23:43:50.380866+0000","last_clean":"2024-04-26T23:43:50.380866+0000","last_became_active":"2024-04-26T23:43:48.273749+0000","last_became_peered":"2024-04-26T23:43:48.273749+0000","last_unstale":"2024-04-26T23:43:50.380866+0000","last_undegraded":"2024-04-26T23:43:50.380866+0000","last_fullsized":"2024-04-26T23:43:50.380866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:57.220373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282075+0000","last_change":"2024-04-26T23:43:49.267651+0000","last_active":"2024-04-26T23:43:49.282075+0000","last_peered":"2024-04-26T23:43:49.282075+0000","last_clean":"2024-04-26T23:43:49.282075+0000","last_became_active":"2024-04-26T23:43:49.266997+0000","last_became_peered":"2024-04-26T23:43:49.266997+0000","last_unstale":"2024-04-26T23:43:49.282075+0000","last_undegraded":"2024-04-26T23:43:49.282075+0000","last_fullsized":"2024-04-26T23:43:49.282075+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.290578+0000","last_change":"2024-04-26T23:43:48.272658+0000","last_active":"2024-04-26T23:43:49.290578+0000","last_peered":"2024-04-26T23:43:49.290578+0000","last_clean":"2024-04-26T23:43:49.290578+0000","last_became_active":"2024-04-26T23:43:48.272297+0000","last_became_peered":"2024-04-26T23:43:48.272297+0000","last_unstale":"2024-04-26T23:43:49.290578+0000","last_undegraded":"2024-04-26T23:43:49.290578+0000","last_fullsized":"2024-04-26T23:43:49.290578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:00.819196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.272712+0000","last_change":"2024-04-26T23:43:49.269214+0000","last_active":"2024-04-26T23:43:49.272712+0000","last_peered":"2024-04-26T23:43:49.272712+0000","last_clean":"2024-04-26T23:43:49.272712+0000","last_became_active":"2024-04-26T23:43:49.268915+0000","last_became_peered":"2024-04-26T23:43:49.268915+0000","last_unstale":"2024-04-26T23:43:49.272712+0000","last_undegraded":"2024-04-26T23:43:49.272712+0000","last_fullsized":"2024-04-26T23:43:49.272712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:12.117486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.286288+0000","last_change":"2024-04-26T23:43:48.262515+0000","last_active":"2024-04-26T23:43:49.286288+0000","last_peered":"2024-04-26T23:43:49.286288+0000","last_clean":"2024-04-26T23:43:49.286288+0000","last_became_active":"2024-04-26T23:43:48.262236+0000","last_became_peered":"2024-04-26T23:43:48.262236+0000","last_unstale":"2024-04-26T23:43:49.286288+0000","last_undegraded":"2024-04-26T23:43:49.286288+0000","last_fullsized":"2024-04-26T23:43:49.286288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:50.001320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.287022+0000","last_change":"2024-04-26T23:43:49.283662+0000","last_active":"2024-04-26T23:43:49.287022+0000","last_peered":"2024-04-26T23:43:49.287022+0000","last_clean":"2024-04-26T23:43:49.287022+0000","last_became_active":"2024-04-26T23:43:49.283285+0000","last_became_peered":"2024-04-26T23:43:49.283285+0000","last_unstale":"2024-04-26T23:43:49.287022+0000","last_undegraded":"2024-04-26T23:43:49.287022+0000","last_fullsized":"2024-04-26T23:43:49.287022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400538+0000","last_change":"2024-04-26T23:43:48.273533+0000","last_active":"2024-04-26T23:43:50.400538+0000","last_peered":"2024-04-26T23:43:50.400538+0000","last_clean":"2024-04-26T23:43:50.400538+0000","last_became_active":"2024-04-26T23:43:48.272771+0000","last_became_peered":"2024-04-26T23:43:48.272771+0000","last_unstale":"2024-04-26T23:43:50.400538+0000","last_undegraded":"2024-04-26T23:43:50.400538+0000","last_fullsized":"2024-04-26T23:43:50.400538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:11.100374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.273112+0000","last_change":"2024-04-26T23:43:49.268115+0000","last_active":"2024-04-26T23:43:49.273112+0000","last_peered":"2024-04-26T23:43:49.273112+0000","last_clean":"2024-04-26T23:43:49.273112+0000","last_became_active":"2024-04-26T23:43:49.267811+0000","last_became_peered":"2024-04-26T23:43:49.267811+0000","last_unstale":"2024-04-26T23:43:49.273112+0000","last_undegraded":"2024-04-26T23:43:49.273112+0000","last_fullsized":"2024-04-26T23:43:49.273112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:24.933653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380243+0000","last_change":"2024-04-26T23:43:48.272786+0000","last_active":"2024-04-26T23:43:50.380243+0000","last_peered":"2024-04-26T23:43:50.380243+0000","last_clean":"2024-04-26T23:43:50.380243+0000","last_became_active":"2024-04-26T23:43:48.272517+0000","last_became_peered":"2024-04-26T23:43:48.272517+0000","last_unstale":"2024-04-26T23:43:50.380243+0000","last_undegraded":"2024-04-26T23:43:50.380243+0000","last_fullsized":"2024-04-26T23:43:50.380243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:50.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282903+0000","last_change":"2024-04-26T23:43:49.277153+0000","last_active":"2024-04-26T23:43:49.282903+0000","last_peered":"2024-04-26T23:43:49.282903+0000","last_clean":"2024-04-26T23:43:49.282903+0000","last_became_active":"2024-04-26T23:43:49.275809+0000","last_became_peered":"2024-04-26T23:43:49.275809+0000","last_unstale":"2024-04-26T23:43:49.282903+0000","last_undegraded":"2024-04-26T23:43:49.282903+0000","last_fullsized":"2024-04-26T23:43:49.282903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:20.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245603+0000","last_change":"2024-04-26T23:43:48.268648+0000","last_active":"2024-04-26T23:43:50.245603+0000","last_peered":"2024-04-26T23:43:50.245603+0000","last_clean":"2024-04-26T23:43:50.245603+0000","last_became_active":"2024-04-26T23:43:48.268378+0000","last_became_peered":"2024-04-26T23:43:48.268378+0000","last_unstale":"2024-04-26T23:43:50.245603+0000","last_undegraded":"2024-04-26T23:43:50.245603+0000","last_fullsized":"2024-04-26T23:43:50.245603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:43:02.042319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247918+0000","last_change":"2024-04-26T23:43:49.281360+0000","last_active":"2024-04-26T23:43:50.247918+0000","last_peered":"2024-04-26T23:43:50.247918+0000","last_clean":"2024-04-26T23:43:50.247918+0000","last_became_active":"2024-04-26T23:43:49.276490+0000","last_became_peered":"2024-04-26T23:43:49.276490+0000","last_unstale":"2024-04-26T23:43:50.247918+0000","last_undegraded":"2024-04-26T23:43:50.247918+0000","last_fullsized":"2024-04-26T23:43:50.247918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401729+0000","last_change":"2024-04-26T23:43:48.268775+0000","last_active":"2024-04-26T23:43:50.401729+0000","last_peered":"2024-04-26T23:43:50.401729+0000","last_clean":"2024-04-26T23:43:50.401729+0000","last_became_active":"2024-04-26T23:43:48.268103+0000","last_became_peered":"2024-04-26T23:43:48.268103+0000","last_unstale":"2024-04-26T23:43:50.401729+0000","last_undegraded":"2024-04-26T23:43:50.401729+0000","last_fullsized":"2024-04-26T23:43:50.401729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:11.911808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380414+0000","last_change":"2024-04-26T23:43:49.272024+0000","last_active":"2024-04-26T23:43:50.380414+0000","last_peered":"2024-04-26T23:43:50.380414+0000","last_clean":"2024-04-26T23:43:50.380414+0000","last_became_active":"2024-04-26T23:43:49.271718+0000","last_became_peered":"2024-04-26T23:43:49.271718+0000","last_unstale":"2024-04-26T23:43:50.380414+0000","last_undegraded":"2024-04-26T23:43:50.380414+0000","last_fullsized":"2024-04-26T23:43:50.380414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:41.510299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248237+0000","last_change":"2024-04-26T23:43:48.277185+0000","last_active":"2024-04-26T23:43:50.248237+0000","last_peered":"2024-04-26T23:43:50.248237+0000","last_clean":"2024-04-26T23:43:50.248237+0000","last_became_active":"2024-04-26T23:43:48.276850+0000","last_became_peered":"2024-04-26T23:43:48.276850+0000","last_unstale":"2024-04-26T23:43:50.248237+0000","last_undegraded":"2024-04-26T23:43:50.248237+0000","last_fullsized":"2024-04-26T23:43:50.248237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:07.878542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380168+0000","last_change":"2024-04-26T23:43:49.278338+0000","last_active":"2024-04-26T23:43:50.380168+0000","last_peered":"2024-04-26T23:43:50.380168+0000","last_clean":"2024-04-26T23:43:50.380168+0000","last_became_active":"2024-04-26T23:43:49.277925+0000","last_became_peered":"2024-04-26T23:43:49.277925+0000","last_unstale":"2024-04-26T23:43:50.380168+0000","last_undegraded":"2024-04-26T23:43:50.380168+0000","last_fullsized":"2024-04-26T23:43:50.380168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:05.965232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380846+0000","last_change":"2024-04-26T23:43:48.263020+0000","last_active":"2024-04-26T23:43:50.380846+0000","last_peered":"2024-04-26T23:43:50.380846+0000","last_clean":"2024-04-26T23:43:50.380846+0000","last_became_active":"2024-04-26T23:43:48.262636+0000","last_became_peered":"2024-04-26T23:43:48.262636+0000","last_unstale":"2024-04-26T23:43:50.380846+0000","last_undegraded":"2024-04-26T23:43:50.380846+0000","last_fullsized":"2024-04-26T23:43:50.380846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:22:42.857702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249232+0000","last_change":"2024-04-26T23:43:49.275183+0000","last_active":"2024-04-26T23:43:50.249232+0000","last_peered":"2024-04-26T23:43:50.249232+0000","last_clean":"2024-04-26T23:43:50.249232+0000","last_became_active":"2024-04-26T23:43:49.274846+0000","last_became_peered":"2024-04-26T23:43:49.274846+0000","last_unstale":"2024-04-26T23:43:50.249232+0000","last_undegraded":"2024-04-26T23:43:50.249232+0000","last_fullsized":"2024-04-26T23:43:50.249232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.267076+0000","last_change":"2024-04-26T23:43:48.267385+0000","last_active":"2024-04-26T23:43:49.267076+0000","last_peered":"2024-04-26T23:43:49.267076+0000","last_clean":"2024-04-26T23:43:49.267076+0000","last_became_active":"2024-04-26T23:43:48.267110+0000","last_became_peered":"2024-04-26T23:43:48.267110+0000","last_unstale":"2024-04-26T23:43:49.267076+0000","last_undegraded":"2024-04-26T23:43:49.267076+0000","last_fullsized":"2024-04-26T23:43:49.267076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:09.842604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245621+0000","last_change":"2024-04-26T23:43:49.269004+0000","last_active":"2024-04-26T23:43:50.245621+0000","last_peered":"2024-04-26T23:43:50.245621+0000","last_clean":"2024-04-26T23:43:50.245621+0000","last_became_active":"2024-04-26T23:43:49.268622+0000","last_became_peered":"2024-04-26T23:43:49.268622+0000","last_unstale":"2024-04-26T23:43:50.245621+0000","last_undegraded":"2024-04-26T23:43:50.245621+0000","last_fullsized":"2024-04-26T23:43:50.245621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248895+0000","last_change":"2024-04-26T23:43:48.276629+0000","last_active":"2024-04-26T23:43:50.248895+0000","last_peered":"2024-04-26T23:43:50.248895+0000","last_clean":"2024-04-26T23:43:50.248895+0000","last_became_active":"2024-04-26T23:43:48.276310+0000","last_became_peered":"2024-04-26T23:43:48.276310+0000","last_unstale":"2024-04-26T23:43:50.248895+0000","last_undegraded":"2024-04-26T23:43:50.248895+0000","last_fullsized":"2024-04-26T23:43:50.248895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:32.905965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245985+0000","last_change":"2024-04-26T23:43:49.254506+0000","last_active":"2024-04-26T23:43:50.245985+0000","last_peered":"2024-04-26T23:43:50.245985+0000","last_clean":"2024-04-26T23:43:50.245985+0000","last_became_active":"2024-04-26T23:43:49.254150+0000","last_became_peered":"2024-04-26T23:43:49.254150+0000","last_unstale":"2024-04-26T23:43:50.245985+0000","last_undegraded":"2024-04-26T23:43:50.245985+0000","last_fullsized":"2024-04-26T23:43:50.245985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:20.380543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248216+0000","last_change":"2024-04-26T23:43:48.271236+0000","last_active":"2024-04-26T23:43:50.248216+0000","last_peered":"2024-04-26T23:43:50.248216+0000","last_clean":"2024-04-26T23:43:50.248216+0000","last_became_active":"2024-04-26T23:43:48.270973+0000","last_became_peered":"2024-04-26T23:43:48.270973+0000","last_unstale":"2024-04-26T23:43:50.248216+0000","last_undegraded":"2024-04-26T23:43:50.248216+0000","last_fullsized":"2024-04-26T23:43:50.248216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:43.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.270290+0000","last_change":"2024-04-26T23:43:49.266112+0000","last_active":"2024-04-26T23:43:49.270290+0000","last_peered":"2024-04-26T23:43:49.270290+0000","last_clean":"2024-04-26T23:43:49.270290+0000","last_became_active":"2024-04-26T23:43:49.265831+0000","last_became_peered":"2024-04-26T23:43:49.265831+0000","last_unstale":"2024-04-26T23:43:49.270290+0000","last_undegraded":"2024-04-26T23:43:49.270290+0000","last_fullsized":"2024-04-26T23:43:49.270290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:36.307742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249346+0000","last_change":"2024-04-26T23:43:48.271994+0000","last_active":"2024-04-26T23:43:50.249346+0000","last_peered":"2024-04-26T23:43:50.249346+0000","last_clean":"2024-04-26T23:43:50.249346+0000","last_became_active":"2024-04-26T23:43:48.271637+0000","last_became_peered":"2024-04-26T23:43:48.271637+0000","last_unstale":"2024-04-26T23:43:50.249346+0000","last_undegraded":"2024-04-26T23:43:50.249346+0000","last_fullsized":"2024-04-26T23:43:50.249346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.027181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246917+0000","last_change":"2024-04-26T23:43:49.257865+0000","last_active":"2024-04-26T23:43:50.246917+0000","last_peered":"2024-04-26T23:43:50.246917+0000","last_clean":"2024-04-26T23:43:50.246917+0000","last_became_active":"2024-04-26T23:43:49.257549+0000","last_became_peered":"2024-04-26T23:43:49.257549+0000","last_unstale":"2024-04-26T23:43:50.246917+0000","last_undegraded":"2024-04-26T23:43:50.246917+0000","last_fullsized":"2024-04-26T23:43:50.246917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:27.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.292624+0000","last_change":"2024-04-26T23:43:48.267933+0000","last_active":"2024-04-26T23:43:49.292624+0000","last_peered":"2024-04-26T23:43:49.292624+0000","last_clean":"2024-04-26T23:43:49.292624+0000","last_became_active":"2024-04-26T23:43:48.267580+0000","last_became_peered":"2024-04-26T23:43:48.267580+0000","last_unstale":"2024-04-26T23:43:49.292624+0000","last_undegraded":"2024-04-26T23:43:49.292624+0000","last_fullsized":"2024-04-26T23:43:49.292624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:06.059243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.283088+0000","last_change":"2024-04-26T23:43:49.278318+0000","last_active":"2024-04-26T23:43:49.283088+0000","last_peered":"2024-04-26T23:43:49.283088+0000","last_clean":"2024-04-26T23:43:49.283088+0000","last_became_active":"2024-04-26T23:43:49.276396+0000","last_became_peered":"2024-04-26T23:43:49.276396+0000","last_unstale":"2024-04-26T23:43:49.283088+0000","last_undegraded":"2024-04-26T23:43:49.283088+0000","last_fullsized":"2024-04-26T23:43:49.283088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:10.859716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245992+0000","last_change":"2024-04-26T23:43:48.266508+0000","last_active":"2024-04-26T23:43:50.245992+0000","last_peered":"2024-04-26T23:43:50.245992+0000","last_clean":"2024-04-26T23:43:50.245992+0000","last_became_active":"2024-04-26T23:43:48.266199+0000","last_became_peered":"2024-04-26T23:43:48.266199+0000","last_unstale":"2024-04-26T23:43:50.245992+0000","last_undegraded":"2024-04-26T23:43:50.245992+0000","last_fullsized":"2024-04-26T23:43:50.245992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.669722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247533+0000","last_change":"2024-04-26T23:43:49.278365+0000","last_active":"2024-04-26T23:43:50.247533+0000","last_peered":"2024-04-26T23:43:50.247533+0000","last_clean":"2024-04-26T23:43:50.247533+0000","last_became_active":"2024-04-26T23:43:49.278100+0000","last_became_peered":"2024-04-26T23:43:49.278100+0000","last_unstale":"2024-04-26T23:43:50.247533+0000","last_undegraded":"2024-04-26T23:43:50.247533+0000","last_fullsized":"2024-04-26T23:43:50.247533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401478+0000","last_change":"2024-04-26T23:43:48.259534+0000","last_active":"2024-04-26T23:43:50.401478+0000","last_peered":"2024-04-26T23:43:50.401478+0000","last_clean":"2024-04-26T23:43:50.401478+0000","last_became_active":"2024-04-26T23:43:48.259223+0000","last_became_peered":"2024-04-26T23:43:48.259223+0000","last_unstale":"2024-04-26T23:43:50.401478+0000","last_undegraded":"2024-04-26T23:43:50.401478+0000","last_fullsized":"2024-04-26T23:43:50.401478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:43.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245146+0000","last_change":"2024-04-26T23:43:49.274564+0000","last_active":"2024-04-26T23:43:50.245146+0000","last_peered":"2024-04-26T23:43:50.245146+0000","last_clean":"2024-04-26T23:43:50.245146+0000","last_became_active":"2024-04-26T23:43:49.273784+0000","last_became_peered":"2024-04-26T23:43:49.273784+0000","last_unstale":"2024-04-26T23:43:50.245146+0000","last_undegraded":"2024-04-26T23:43:50.245146+0000","last_fullsized":"2024-04-26T23:43:50.245146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400868+0000","last_change":"2024-04-26T23:43:48.270164+0000","last_active":"2024-04-26T23:43:50.400868+0000","last_peered":"2024-04-26T23:43:50.400868+0000","last_clean":"2024-04-26T23:43:50.400868+0000","last_became_active":"2024-04-26T23:43:48.269610+0000","last_became_peered":"2024-04-26T23:43:48.269610+0000","last_unstale":"2024-04-26T23:43:50.400868+0000","last_undegraded":"2024-04-26T23:43:50.400868+0000","last_fullsized":"2024-04-26T23:43:50.400868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:32.905965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.287058+0000","last_change":"2024-04-26T23:43:49.284103+0000","last_active":"2024-04-26T23:43:49.287058+0000","last_peered":"2024-04-26T23:43:49.287058+0000","last_clean":"2024-04-26T23:43:49.287058+0000","last_became_active":"2024-04-26T23:43:49.283704+0000","last_became_peered":"2024-04-26T23:43:49.283704+0000","last_unstale":"2024-04-26T23:43:49.287058+0000","last_undegraded":"2024-04-26T23:43:49.287058+0000","last_fullsized":"2024-04-26T23:43:49.287058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:45.654839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.254450+0000","last_change":"2024-04-26T23:43:48.267306+0000","last_active":"2024-04-26T23:43:49.254450+0000","last_peered":"2024-04-26T23:43:49.254450+0000","last_clean":"2024-04-26T23:43:49.254450+0000","last_became_active":"2024-04-26T23:43:48.267031+0000","last_became_peered":"2024-04-26T23:43:48.267031+0000","last_unstale":"2024-04-26T23:43:49.254450+0000","last_undegraded":"2024-04-26T23:43:49.254450+0000","last_fullsized":"2024-04-26T23:43:49.254450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:43.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288120+0000","last_change":"2024-04-26T23:43:49.285770+0000","last_active":"2024-04-26T23:43:49.288120+0000","last_peered":"2024-04-26T23:43:49.288120+0000","last_clean":"2024-04-26T23:43:49.288120+0000","last_became_active":"2024-04-26T23:43:49.285440+0000","last_became_peered":"2024-04-26T23:43:49.285440+0000","last_unstale":"2024-04-26T23:43:49.288120+0000","last_undegraded":"2024-04-26T23:43:49.288120+0000","last_fullsized":"2024-04-26T23:43:49.288120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380471+0000","last_change":"2024-04-26T23:43:48.277021+0000","last_active":"2024-04-26T23:43:50.380471+0000","last_peered":"2024-04-26T23:43:50.380471+0000","last_clean":"2024-04-26T23:43:50.380471+0000","last_became_active":"2024-04-26T23:43:48.276741+0000","last_became_peered":"2024-04-26T23:43:48.276741+0000","last_unstale":"2024-04-26T23:43:50.380471+0000","last_undegraded":"2024-04-26T23:43:50.380471+0000","last_fullsized":"2024-04-26T23:43:50.380471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:27.654355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291552+0000","last_change":"2024-04-26T23:43:49.289735+0000","last_active":"2024-04-26T23:43:49.291552+0000","last_peered":"2024-04-26T23:43:49.291552+0000","last_clean":"2024-04-26T23:43:49.291552+0000","last_became_active":"2024-04-26T23:43:49.289127+0000","last_became_peered":"2024-04-26T23:43:49.289127+0000","last_unstale":"2024-04-26T23:43:49.291552+0000","last_undegraded":"2024-04-26T23:43:49.291552+0000","last_fullsized":"2024-04-26T23:43:49.291552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:57.649051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.256240+0000","last_change":"2024-04-26T23:43:48.275725+0000","last_active":"2024-04-26T23:43:49.256240+0000","last_peered":"2024-04-26T23:43:49.256240+0000","last_clean":"2024-04-26T23:43:49.256240+0000","last_became_active":"2024-04-26T23:43:48.275410+0000","last_became_peered":"2024-04-26T23:43:48.275410+0000","last_unstale":"2024-04-26T23:43:49.256240+0000","last_undegraded":"2024-04-26T23:43:49.256240+0000","last_fullsized":"2024-04-26T23:43:49.256240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:56.258918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291873+0000","last_change":"2024-04-26T23:43:49.282195+0000","last_active":"2024-04-26T23:43:49.291873+0000","last_peered":"2024-04-26T23:43:49.291873+0000","last_clean":"2024-04-26T23:43:49.291873+0000","last_became_active":"2024-04-26T23:43:49.280767+0000","last_became_peered":"2024-04-26T23:43:49.280767+0000","last_unstale":"2024-04-26T23:43:49.291873+0000","last_undegraded":"2024-04-26T23:43:49.291873+0000","last_fullsized":"2024-04-26T23:43:49.291873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.269796+0000","last_change":"2024-04-26T23:43:48.276508+0000","last_active":"2024-04-26T23:43:49.269796+0000","last_peered":"2024-04-26T23:43:49.269796+0000","last_clean":"2024-04-26T23:43:49.269796+0000","last_became_active":"2024-04-26T23:43:48.276231+0000","last_became_peered":"2024-04-26T23:43:48.276231+0000","last_unstale":"2024-04-26T23:43:49.269796+0000","last_undegraded":"2024-04-26T23:43:49.269796+0000","last_fullsized":"2024-04-26T23:43:49.269796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:44.637728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401416+0000","last_change":"2024-04-26T23:43:49.284165+0000","last_active":"2024-04-26T23:43:50.401416+0000","last_peered":"2024-04-26T23:43:50.401416+0000","last_clean":"2024-04-26T23:43:50.401416+0000","last_became_active":"2024-04-26T23:43:49.283732+0000","last_became_peered":"2024-04-26T23:43:49.283732+0000","last_unstale":"2024-04-26T23:43:50.401416+0000","last_undegraded":"2024-04-26T23:43:50.401416+0000","last_fullsized":"2024-04-26T23:43:50.401416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249238+0000","last_change":"2024-04-26T23:43:48.275599+0000","last_active":"2024-04-26T23:43:50.249238+0000","last_peered":"2024-04-26T23:43:50.249238+0000","last_clean":"2024-04-26T23:43:50.249238+0000","last_became_active":"2024-04-26T23:43:48.275285+0000","last_became_peered":"2024-04-26T23:43:48.275285+0000","last_unstale":"2024-04-26T23:43:50.249238+0000","last_undegraded":"2024-04-26T23:43:50.249238+0000","last_fullsized":"2024-04-26T23:43:50.249238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:11.911808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401056+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:43:50.401056+0000","last_peered":"2024-04-26T23:43:50.401056+0000","last_clean":"2024-04-26T23:43:50.401056+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:43:50.401056+0000","last_undegraded":"2024-04-26T23:43:50.401056+0000","last_fullsized":"2024-04-26T23:43:50.401056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1576960,"data_stored":1040128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":978944,"data_stored":444658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":970752,"data_stored":443080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":451454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":983040,"data_stored":450320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":978944,"data_stored":443788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":974848,"data_stored":443158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1572864,"data_stored":1040106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:43:53.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:43:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.010+0000 7f8a68e00640 1 Processor -- start 2024-04-26T23:43:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.010+0000 7f8a68e00640 1 -- start start 2024-04-26T23:43:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.012+0000 7f8a68e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8a6406e510 0x7f8a6406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.012+0000 7f8a68e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a6406ee30 con 0x7f8a64160bd0 2024-04-26T23:43:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.012+0000 7f8a68e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a64161940 con 0x7f8a640732c0 2024-04-26T23:43:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.012+0000 7f8a68e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8a64071f80 con 0x7f8a6406e510 2024-04-26T23:43:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.013+0000 7f8a59400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8a64160bd0 0x7f8a64161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47400/0 (socket says 172.21.15.46:47400) 2024-04-26T23:43:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.013+0000 7f8a59400640 1 -- 172.21.15.46:0/2931545620 learned_addr learned my addr 172.21.15.46:0/2931545620 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.013+0000 7f8a61e00640 1 --2- 172.21.15.46:0/2931545620 >> v2:172.21.15.142:3300/0 conn(0x7f8a6406e510 0x7f8a6406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.013+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982200978 0 0) 0x7f8a6406ee30 con 0x7f8a64160bd0 2024-04-26T23:43:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.013+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a48003520 con 0x7f8a64160bd0 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1680314885 0 0) 0x7f8a48003520 con 0x7f8a64160bd0 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 >> v2:172.21.15.142:3300/0 conn(0x7f8a6406e510 msgr2=0x7f8a6406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 --2- 172.21.15.46:0/2931545620 >> v2:172.21.15.142:3300/0 conn(0x7f8a6406e510 0x7f8a6406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 >> v1:172.21.15.142:6789/0 conn(0x7f8a640732c0 legacy=0x7f8a64073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a64072260 con 0x7f8a64160bd0 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8a4c001670 con 0x7f8a64160bd0 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61e00640 1 --2- 172.21.15.46:0/2931545620 >> v2:172.21.15.142:3300/0 conn(0x7f8a6406e510 0x7f8a6406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 >> v1:172.21.15.46:6789/0 conn(0x7f8a64160bd0 legacy=0x7f8a64161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a48003a00 0x7f8a48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48004800 0x7f8a48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64071f80 con 0x7f8a480051c0 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4800c0c0 con 0x7f8a48004800 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4800c2c0 con 0x7f8a48003a00 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.014+0000 7f8a62800640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a59400640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48004800 0x7f8a48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a61e00640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a48003a00 0x7f8a48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a62800640 1 -- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a48003a00 msgr2=0x7f8a48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a62800640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a48003a00 0x7f8a48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a62800640 1 -- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48004800 msgr2=0x7f8a48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a62800640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48004800 0x7f8a48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.015+0000 7f8a62800640 1 -- 172.21.15.46:0/2931545620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a48003520 con 0x7f8a480051c0 2024-04-26T23:43:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.016+0000 7f8a62800640 1 -- 172.21.15.46:0/2931545620 reap_dead start 2024-04-26T23:43:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.016+0000 7f8a62800640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a4800b9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001e50 tx=0x7f8a48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1eb2cd42624821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.016+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c001a70 con 0x7f8a480051c0 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.016+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c003970 con 0x7f8a480051c0 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.016+0000 7f8a61400640 1 -- 172.21.15.46:0/2931545620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c003d60 con 0x7f8a480051c0 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 -- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 msgr2=0x7f8a4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a4800b9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001e50 tx=0x7f8a48005830 comp rx=0 tx=0).stop 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 -- 172.21.15.46:0/2931545620 shutdown_connections 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 --2- 172.21.15.46:0/2931545620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a4800b9c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 -- 172.21.15.46:0/2931545620 >> 172.21.15.46:0/2931545620 conn(0x7f8a6406c530 msgr2=0x7f8a6406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 -- 172.21.15.46:0/2931545620 shutdown_connections 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.017+0000 7f8a68e00640 1 -- 172.21.15.46:0/2931545620 wait complete. 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.019+0000 7f8a68e00640 1 Processor -- start 2024-04-26T23:43:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.019+0000 7f8a68e00640 1 -- start start 2024-04-26T23:43:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a64143ce0 0x7f8a640b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6413dbf0 0x7f8a6413e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64072d40 con 0x7f8a480051c0 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64072820 con 0x7f8a6413dbf0 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a68e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8a64072ec0 con 0x7f8a64143ce0 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.020+0000 7f8a61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47068/0 (socket says 172.21.15.46:47068) 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 -- 172.21.15.46:0/945539601 learned_addr learned my addr 172.21.15.46:0/945539601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 -- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a64143ce0 msgr2=0x7f8a640b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a64143ce0 0x7f8a640b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 -- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6413dbf0 msgr2=0x7f8a6413e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a62800640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6413dbf0 0x7f8a6413e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6413dbf0 0x7f8a6413e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 -- 172.21.15.46:0/945539601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a64144230 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a61e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8a54011720 tx=0x7f8a54011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400f9c0 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5400fb60 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.021+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54015460 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.022+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a64144510 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.022+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a64144ad0 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.023+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a64069530 con 0x7f8a480051c0 2024-04-26T23:43:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.024+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a54015600 con 0x7f8a480051c0 2024-04-26T23:43:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.025+0000 7f8a5aa00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.025+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f8a5408c800 con 0x7f8a480051c0 2024-04-26T23:43:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.026+0000 7f8a59400640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.027+0000 7f8a59400640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8a640b3b60 tx=0x7f8a4c000ef0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.030+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a54019030 con 0x7f8a480051c0 2024-04-26T23:43:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.229+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a64146330 con 0x7f8a4006c8f0 2024-04-26T23:43:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.248+0000 7f8a5aa00640 1 -- 172.21.15.46:0/945539601 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252048 (secure 0 0 0) 0x7f8a64146330 con 0x7f8a4006c8f0 2024-04-26T23:43:54.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:54.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:43:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 msgr2=0x7f8a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8a640b3b60 tx=0x7f8a4c000ef0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 msgr2=0x7f8a6415c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8a54011720 tx=0x7f8a54011be0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 shutdown_connections 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6413dbf0 0x7f8a6413e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a64143ce0 0x7f8a640b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 --2- 172.21.15.46:0/945539601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a480051c0 0x7f8a6415c4e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 >> 172.21.15.46:0/945539601 conn(0x7f8a6406c530 msgr2=0x7f8a640b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 shutdown_connections 2024-04-26T23:43:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.255+0000 7f8a68e00640 1 -- 172.21.15.46:0/945539601 wait complete. 2024-04-26T23:43:54.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-26T23:43:54.037038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":312380,"kb_used_data":8824,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754662340,"statfs":{"total":773094113280,"available":772774236160,"internally_reserved":0,"allocated":9035776,"data_stored":4756692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":310693425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002122"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401538+0000","last_change":"2024-04-26T23:43:49.281640+0000","last_active":"2024-04-26T23:43:50.401538+0000","last_peered":"2024-04-26T23:43:50.401538+0000","last_clean":"2024-04-26T23:43:50.401538+0000","last_became_active":"2024-04-26T23:43:49.281328+0000","last_became_peered":"2024-04-26T23:43:49.281328+0000","last_unstale":"2024-04-26T23:43:50.401538+0000","last_undegraded":"2024-04-26T23:43:50.401538+0000","last_fullsized":"2024-04-26T23:43:50.401538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245873+0000","last_change":"2024-04-26T23:43:49.276811+0000","last_active":"2024-04-26T23:43:50.245873+0000","last_peered":"2024-04-26T23:43:50.245873+0000","last_clean":"2024-04-26T23:43:50.245873+0000","last_became_active":"2024-04-26T23:43:49.271168+0000","last_became_peered":"2024-04-26T23:43:49.271168+0000","last_unstale":"2024-04-26T23:43:50.245873+0000","last_undegraded":"2024-04-26T23:43:50.245873+0000","last_fullsized":"2024-04-26T23:43:50.245873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:21.981490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.289989+0000","last_change":"2024-04-26T23:43:49.282871+0000","last_active":"2024-04-26T23:43:49.289989+0000","last_peered":"2024-04-26T23:43:49.289989+0000","last_clean":"2024-04-26T23:43:49.289989+0000","last_became_active":"2024-04-26T23:43:49.282474+0000","last_became_peered":"2024-04-26T23:43:49.282474+0000","last_unstale":"2024-04-26T23:43:49.289989+0000","last_undegraded":"2024-04-26T23:43:49.289989+0000","last_fullsized":"2024-04-26T23:43:49.289989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:57.480439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.269859+0000","last_change":"2024-04-26T23:43:49.264517+0000","last_active":"2024-04-26T23:43:49.269859+0000","last_peered":"2024-04-26T23:43:49.269859+0000","last_clean":"2024-04-26T23:43:49.269859+0000","last_became_active":"2024-04-26T23:43:49.264246+0000","last_became_peered":"2024-04-26T23:43:49.264246+0000","last_unstale":"2024-04-26T23:43:49.269859+0000","last_undegraded":"2024-04-26T23:43:49.269859+0000","last_fullsized":"2024-04-26T23:43:49.269859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:48.686834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380412+0000","last_change":"2024-04-26T23:43:49.285506+0000","last_active":"2024-04-26T23:43:50.380412+0000","last_peered":"2024-04-26T23:43:50.380412+0000","last_clean":"2024-04-26T23:43:50.380412+0000","last_became_active":"2024-04-26T23:43:49.285205+0000","last_became_peered":"2024-04-26T23:43:49.285205+0000","last_unstale":"2024-04-26T23:43:50.380412+0000","last_undegraded":"2024-04-26T23:43:50.380412+0000","last_fullsized":"2024-04-26T23:43:50.380412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:00.043535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248647+0000","last_change":"2024-04-26T23:43:49.283827+0000","last_active":"2024-04-26T23:43:50.248647+0000","last_peered":"2024-04-26T23:43:50.248647+0000","last_clean":"2024-04-26T23:43:50.248647+0000","last_became_active":"2024-04-26T23:43:49.283313+0000","last_became_peered":"2024-04-26T23:43:49.283313+0000","last_unstale":"2024-04-26T23:43:50.248647+0000","last_undegraded":"2024-04-26T23:43:50.248647+0000","last_fullsized":"2024-04-26T23:43:50.248647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.283065+0000","last_change":"2024-04-26T23:43:49.277515+0000","last_active":"2024-04-26T23:43:49.283065+0000","last_peered":"2024-04-26T23:43:49.283065+0000","last_clean":"2024-04-26T23:43:49.283065+0000","last_became_active":"2024-04-26T23:43:49.277142+0000","last_became_peered":"2024-04-26T23:43:49.277142+0000","last_unstale":"2024-04-26T23:43:49.283065+0000","last_undegraded":"2024-04-26T23:43:49.283065+0000","last_fullsized":"2024-04-26T23:43:49.283065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:27.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380028+0000","last_change":"2024-04-26T23:43:49.278317+0000","last_active":"2024-04-26T23:43:50.380028+0000","last_peered":"2024-04-26T23:43:50.380028+0000","last_clean":"2024-04-26T23:43:50.380028+0000","last_became_active":"2024-04-26T23:43:49.278029+0000","last_became_peered":"2024-04-26T23:43:49.278029+0000","last_unstale":"2024-04-26T23:43:50.380028+0000","last_undegraded":"2024-04-26T23:43:50.380028+0000","last_fullsized":"2024-04-26T23:43:50.380028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:44.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248371+0000","last_change":"2024-04-26T23:43:49.276323+0000","last_active":"2024-04-26T23:43:50.248371+0000","last_peered":"2024-04-26T23:43:50.248371+0000","last_clean":"2024-04-26T23:43:50.248371+0000","last_became_active":"2024-04-26T23:43:49.276029+0000","last_became_peered":"2024-04-26T23:43:49.276029+0000","last_unstale":"2024-04-26T23:43:50.248371+0000","last_undegraded":"2024-04-26T23:43:50.248371+0000","last_fullsized":"2024-04-26T23:43:50.248371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:57.480439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.264948+0000","last_change":"2024-04-26T23:43:49.260059+0000","last_active":"2024-04-26T23:43:49.264948+0000","last_peered":"2024-04-26T23:43:49.264948+0000","last_clean":"2024-04-26T23:43:49.264948+0000","last_became_active":"2024-04-26T23:43:49.259786+0000","last_became_peered":"2024-04-26T23:43:49.259786+0000","last_unstale":"2024-04-26T23:43:49.264948+0000","last_undegraded":"2024-04-26T23:43:49.264948+0000","last_fullsized":"2024-04-26T23:43:49.264948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:06.143662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245898+0000","last_change":"2024-04-26T23:43:49.273639+0000","last_active":"2024-04-26T23:43:50.245898+0000","last_peered":"2024-04-26T23:43:50.245898+0000","last_clean":"2024-04-26T23:43:50.245898+0000","last_became_active":"2024-04-26T23:43:49.273373+0000","last_became_peered":"2024-04-26T23:43:49.273373+0000","last_unstale":"2024-04-26T23:43:50.245898+0000","last_undegraded":"2024-04-26T23:43:50.245898+0000","last_fullsized":"2024-04-26T23:43:50.245898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:35.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.263453+0000","last_change":"2024-04-26T23:43:49.258202+0000","last_active":"2024-04-26T23:43:49.263453+0000","last_peered":"2024-04-26T23:43:49.263453+0000","last_clean":"2024-04-26T23:43:49.263453+0000","last_became_active":"2024-04-26T23:43:49.257848+0000","last_became_peered":"2024-04-26T23:43:49.257848+0000","last_unstale":"2024-04-26T23:43:49.263453+0000","last_undegraded":"2024-04-26T23:43:49.263453+0000","last_fullsized":"2024-04-26T23:43:49.263453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:52.149622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380533+0000","last_change":"2024-04-26T23:43:49.277903+0000","last_active":"2024-04-26T23:43:50.380533+0000","last_peered":"2024-04-26T23:43:50.380533+0000","last_clean":"2024-04-26T23:43:50.380533+0000","last_became_active":"2024-04-26T23:43:49.277420+0000","last_became_peered":"2024-04-26T23:43:49.277420+0000","last_unstale":"2024-04-26T23:43:50.380533+0000","last_undegraded":"2024-04-26T23:43:50.380533+0000","last_fullsized":"2024-04-26T23:43:50.380533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:02.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248720+0000","last_change":"2024-04-26T23:43:49.265154+0000","last_active":"2024-04-26T23:43:50.248720+0000","last_peered":"2024-04-26T23:43:50.248720+0000","last_clean":"2024-04-26T23:43:50.248720+0000","last_became_active":"2024-04-26T23:43:49.264872+0000","last_became_peered":"2024-04-26T23:43:49.264872+0000","last_unstale":"2024-04-26T23:43:50.248720+0000","last_undegraded":"2024-04-26T23:43:50.248720+0000","last_fullsized":"2024-04-26T23:43:50.248720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246235+0000","last_change":"2024-04-26T23:43:49.281969+0000","last_active":"2024-04-26T23:43:50.246235+0000","last_peered":"2024-04-26T23:43:50.246235+0000","last_clean":"2024-04-26T23:43:50.246235+0000","last_became_active":"2024-04-26T23:43:49.281660+0000","last_became_peered":"2024-04-26T23:43:49.281660+0000","last_unstale":"2024-04-26T23:43:50.246235+0000","last_undegraded":"2024-04-26T23:43:50.246235+0000","last_fullsized":"2024-04-26T23:43:50.246235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:40.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401115+0000","last_change":"2024-04-26T23:43:49.267035+0000","last_active":"2024-04-26T23:43:50.401115+0000","last_peered":"2024-04-26T23:43:50.401115+0000","last_clean":"2024-04-26T23:43:50.401115+0000","last_became_active":"2024-04-26T23:43:49.266694+0000","last_became_peered":"2024-04-26T23:43:49.266694+0000","last_unstale":"2024-04-26T23:43:50.401115+0000","last_undegraded":"2024-04-26T23:43:50.401115+0000","last_fullsized":"2024-04-26T23:43:50.401115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381054+0000","last_change":"2024-04-26T23:43:49.277132+0000","last_active":"2024-04-26T23:43:50.381054+0000","last_peered":"2024-04-26T23:43:50.381054+0000","last_clean":"2024-04-26T23:43:50.381054+0000","last_became_active":"2024-04-26T23:43:49.272417+0000","last_became_peered":"2024-04-26T23:43:49.272417+0000","last_unstale":"2024-04-26T23:43:50.381054+0000","last_undegraded":"2024-04-26T23:43:50.381054+0000","last_fullsized":"2024-04-26T23:43:50.381054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:48.386587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291189+0000","last_change":"2024-04-26T23:43:49.289141+0000","last_active":"2024-04-26T23:43:49.291189+0000","last_peered":"2024-04-26T23:43:49.291189+0000","last_clean":"2024-04-26T23:43:49.291189+0000","last_became_active":"2024-04-26T23:43:49.287044+0000","last_became_peered":"2024-04-26T23:43:49.287044+0000","last_unstale":"2024-04-26T23:43:49.291189+0000","last_undegraded":"2024-04-26T23:43:49.291189+0000","last_fullsized":"2024-04-26T23:43:49.291189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:17.305998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249523+0000","last_change":"2024-04-26T23:43:49.270776+0000","last_active":"2024-04-26T23:43:50.249523+0000","last_peered":"2024-04-26T23:43:50.249523+0000","last_clean":"2024-04-26T23:43:50.249523+0000","last_became_active":"2024-04-26T23:43:49.270390+0000","last_became_peered":"2024-04-26T23:43:49.270390+0000","last_unstale":"2024-04-26T23:43:50.249523+0000","last_undegraded":"2024-04-26T23:43:50.249523+0000","last_fullsized":"2024-04-26T23:43:50.249523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381105+0000","last_change":"2024-04-26T23:43:49.279449+0000","last_active":"2024-04-26T23:43:50.381105+0000","last_peered":"2024-04-26T23:43:50.381105+0000","last_clean":"2024-04-26T23:43:50.381105+0000","last_became_active":"2024-04-26T23:43:49.279150+0000","last_became_peered":"2024-04-26T23:43:49.279150+0000","last_unstale":"2024-04-26T23:43:50.381105+0000","last_undegraded":"2024-04-26T23:43:50.381105+0000","last_fullsized":"2024-04-26T23:43:50.381105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:55.389994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247042+0000","last_change":"2024-04-26T23:43:49.270369+0000","last_active":"2024-04-26T23:43:50.247042+0000","last_peered":"2024-04-26T23:43:50.247042+0000","last_clean":"2024-04-26T23:43:50.247042+0000","last_became_active":"2024-04-26T23:43:49.270048+0000","last_became_peered":"2024-04-26T23:43:49.270048+0000","last_unstale":"2024-04-26T23:43:50.247042+0000","last_undegraded":"2024-04-26T23:43:50.247042+0000","last_fullsized":"2024-04-26T23:43:50.247042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:10.859716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380370+0000","last_change":"2024-04-26T23:43:49.279930+0000","last_active":"2024-04-26T23:43:50.380370+0000","last_peered":"2024-04-26T23:43:50.380370+0000","last_clean":"2024-04-26T23:43:50.380370+0000","last_became_active":"2024-04-26T23:43:49.277904+0000","last_became_peered":"2024-04-26T23:43:49.277904+0000","last_unstale":"2024-04-26T23:43:50.380370+0000","last_undegraded":"2024-04-26T23:43:50.380370+0000","last_fullsized":"2024-04-26T23:43:50.380370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:41.993455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246915+0000","last_change":"2024-04-26T23:43:49.266390+0000","last_active":"2024-04-26T23:43:50.246915+0000","last_peered":"2024-04-26T23:43:50.246915+0000","last_clean":"2024-04-26T23:43:50.246915+0000","last_became_active":"2024-04-26T23:43:49.266063+0000","last_became_peered":"2024-04-26T23:43:49.266063+0000","last_unstale":"2024-04-26T23:43:50.246915+0000","last_undegraded":"2024-04-26T23:43:50.246915+0000","last_fullsized":"2024-04-26T23:43:50.246915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:20.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245215+0000","last_change":"2024-04-26T23:43:49.282643+0000","last_active":"2024-04-26T23:43:50.245215+0000","last_peered":"2024-04-26T23:43:50.245215+0000","last_clean":"2024-04-26T23:43:50.245215+0000","last_became_active":"2024-04-26T23:43:49.282010+0000","last_became_peered":"2024-04-26T23:43:49.282010+0000","last_unstale":"2024-04-26T23:43:50.245215+0000","last_undegraded":"2024-04-26T23:43:50.245215+0000","last_fullsized":"2024-04-26T23:43:50.245215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245377+0000","last_change":"2024-04-26T23:43:49.267041+0000","last_active":"2024-04-26T23:43:50.245377+0000","last_peered":"2024-04-26T23:43:50.245377+0000","last_clean":"2024-04-26T23:43:50.245377+0000","last_became_active":"2024-04-26T23:43:49.266054+0000","last_became_peered":"2024-04-26T23:43:49.266054+0000","last_unstale":"2024-04-26T23:43:50.245377+0000","last_undegraded":"2024-04-26T23:43:50.245377+0000","last_fullsized":"2024-04-26T23:43:50.245377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.264922+0000","last_change":"2024-04-26T23:43:49.259445+0000","last_active":"2024-04-26T23:43:49.264922+0000","last_peered":"2024-04-26T23:43:49.264922+0000","last_clean":"2024-04-26T23:43:49.264922+0000","last_became_active":"2024-04-26T23:43:49.257127+0000","last_became_peered":"2024-04-26T23:43:49.257127+0000","last_unstale":"2024-04-26T23:43:49.264922+0000","last_undegraded":"2024-04-26T23:43:49.264922+0000","last_fullsized":"2024-04-26T23:43:49.264922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:43:03.059431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401101+0000","last_change":"2024-04-26T23:43:49.273281+0000","last_active":"2024-04-26T23:43:50.401101+0000","last_peered":"2024-04-26T23:43:50.401101+0000","last_clean":"2024-04-26T23:43:50.401101+0000","last_became_active":"2024-04-26T23:43:49.272814+0000","last_became_peered":"2024-04-26T23:43:49.272814+0000","last_unstale":"2024-04-26T23:43:50.401101+0000","last_undegraded":"2024-04-26T23:43:50.401101+0000","last_fullsized":"2024-04-26T23:43:50.401101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:25.858003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.381067+0000","last_change":"2024-04-26T23:43:49.277373+0000","last_active":"2024-04-26T23:43:50.381067+0000","last_peered":"2024-04-26T23:43:50.381067+0000","last_clean":"2024-04-26T23:43:50.381067+0000","last_became_active":"2024-04-26T23:43:49.277041+0000","last_became_peered":"2024-04-26T23:43:49.277041+0000","last_unstale":"2024-04-26T23:43:50.381067+0000","last_undegraded":"2024-04-26T23:43:50.381067+0000","last_fullsized":"2024-04-26T23:43:50.381067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:08.299501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.244878+0000","last_change":"2024-04-26T23:43:49.260106+0000","last_active":"2024-04-26T23:43:50.244878+0000","last_peered":"2024-04-26T23:43:50.244878+0000","last_clean":"2024-04-26T23:43:50.244878+0000","last_became_active":"2024-04-26T23:43:49.258993+0000","last_became_peered":"2024-04-26T23:43:49.258993+0000","last_unstale":"2024-04-26T23:43:50.244878+0000","last_undegraded":"2024-04-26T23:43:50.244878+0000","last_fullsized":"2024-04-26T23:43:50.244878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:50.053540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400663+0000","last_change":"2024-04-26T23:43:49.266824+0000","last_active":"2024-04-26T23:43:50.400663+0000","last_peered":"2024-04-26T23:43:50.400663+0000","last_clean":"2024-04-26T23:43:50.400663+0000","last_became_active":"2024-04-26T23:43:49.266432+0000","last_became_peered":"2024-04-26T23:43:49.266432+0000","last_unstale":"2024-04-26T23:43:50.400663+0000","last_undegraded":"2024-04-26T23:43:50.400663+0000","last_fullsized":"2024-04-26T23:43:50.400663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291163+0000","last_change":"2024-04-26T23:43:49.274213+0000","last_active":"2024-04-26T23:43:49.291163+0000","last_peered":"2024-04-26T23:43:49.291163+0000","last_clean":"2024-04-26T23:43:49.291163+0000","last_became_active":"2024-04-26T23:43:49.273801+0000","last_became_peered":"2024-04-26T23:43:49.273801+0000","last_unstale":"2024-04-26T23:43:49.291163+0000","last_undegraded":"2024-04-26T23:43:49.291163+0000","last_fullsized":"2024-04-26T23:43:49.291163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400907+0000","last_change":"2024-04-26T23:43:49.278712+0000","last_active":"2024-04-26T23:43:50.400907+0000","last_peered":"2024-04-26T23:43:50.400907+0000","last_clean":"2024-04-26T23:43:50.400907+0000","last_became_active":"2024-04-26T23:43:49.278072+0000","last_became_peered":"2024-04-26T23:43:49.278072+0000","last_unstale":"2024-04-26T23:43:50.400907+0000","last_undegraded":"2024-04-26T23:43:50.400907+0000","last_fullsized":"2024-04-26T23:43:50.400907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:45.060306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.289910+0000","last_change":"2024-04-26T23:43:49.285984+0000","last_active":"2024-04-26T23:43:49.289910+0000","last_peered":"2024-04-26T23:43:49.289910+0000","last_clean":"2024-04-26T23:43:49.289910+0000","last_became_active":"2024-04-26T23:43:49.285690+0000","last_became_peered":"2024-04-26T23:43:49.285690+0000","last_unstale":"2024-04-26T23:43:49.289910+0000","last_undegraded":"2024-04-26T23:43:49.289910+0000","last_fullsized":"2024-04-26T23:43:49.289910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:18.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380998+0000","last_change":"2024-04-26T23:43:48.278182+0000","last_active":"2024-04-26T23:43:50.380998+0000","last_peered":"2024-04-26T23:43:50.380998+0000","last_clean":"2024-04-26T23:43:50.380998+0000","last_became_active":"2024-04-26T23:43:48.277955+0000","last_became_peered":"2024-04-26T23:43:48.277955+0000","last_unstale":"2024-04-26T23:43:50.380998+0000","last_undegraded":"2024-04-26T23:43:50.380998+0000","last_fullsized":"2024-04-26T23:43:50.380998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:35.498033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245406+0000","last_change":"2024-04-26T23:43:49.259791+0000","last_active":"2024-04-26T23:43:50.245406+0000","last_peered":"2024-04-26T23:43:50.245406+0000","last_clean":"2024-04-26T23:43:50.245406+0000","last_became_active":"2024-04-26T23:43:49.258639+0000","last_became_peered":"2024-04-26T23:43:49.258639+0000","last_unstale":"2024-04-26T23:43:50.245406+0000","last_undegraded":"2024-04-26T23:43:50.245406+0000","last_fullsized":"2024-04-26T23:43:50.245406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401306+0000","last_change":"2024-04-26T23:43:48.267027+0000","last_active":"2024-04-26T23:43:50.401306+0000","last_peered":"2024-04-26T23:43:50.401306+0000","last_clean":"2024-04-26T23:43:50.401306+0000","last_became_active":"2024-04-26T23:43:48.266726+0000","last_became_peered":"2024-04-26T23:43:48.266726+0000","last_unstale":"2024-04-26T23:43:50.401306+0000","last_undegraded":"2024-04-26T23:43:50.401306+0000","last_fullsized":"2024-04-26T23:43:50.401306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:53.216357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247363+0000","last_change":"2024-04-26T23:43:49.276183+0000","last_active":"2024-04-26T23:43:50.247363+0000","last_peered":"2024-04-26T23:43:50.247363+0000","last_clean":"2024-04-26T23:43:50.247363+0000","last_became_active":"2024-04-26T23:43:49.275898+0000","last_became_peered":"2024-04-26T23:43:49.275898+0000","last_unstale":"2024-04-26T23:43:50.247363+0000","last_undegraded":"2024-04-26T23:43:50.247363+0000","last_fullsized":"2024-04-26T23:43:50.247363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401113+0000","last_change":"2024-04-26T23:43:48.262249+0000","last_active":"2024-04-26T23:43:50.401113+0000","last_peered":"2024-04-26T23:43:50.401113+0000","last_clean":"2024-04-26T23:43:50.401113+0000","last_became_active":"2024-04-26T23:43:48.261963+0000","last_became_peered":"2024-04-26T23:43:48.261963+0000","last_unstale":"2024-04-26T23:43:50.401113+0000","last_undegraded":"2024-04-26T23:43:50.401113+0000","last_fullsized":"2024-04-26T23:43:50.401113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:24.385420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247422+0000","last_change":"2024-04-26T23:43:49.290218+0000","last_active":"2024-04-26T23:43:50.247422+0000","last_peered":"2024-04-26T23:43:50.247422+0000","last_clean":"2024-04-26T23:43:50.247422+0000","last_became_active":"2024-04-26T23:43:49.289919+0000","last_became_peered":"2024-04-26T23:43:49.289919+0000","last_unstale":"2024-04-26T23:43:50.247422+0000","last_undegraded":"2024-04-26T23:43:50.247422+0000","last_fullsized":"2024-04-26T23:43:50.247422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:45.654839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282712+0000","last_change":"2024-04-26T23:43:48.269395+0000","last_active":"2024-04-26T23:43:49.282712+0000","last_peered":"2024-04-26T23:43:49.282712+0000","last_clean":"2024-04-26T23:43:49.282712+0000","last_became_active":"2024-04-26T23:43:48.268281+0000","last_became_peered":"2024-04-26T23:43:48.268281+0000","last_unstale":"2024-04-26T23:43:49.282712+0000","last_undegraded":"2024-04-26T23:43:49.282712+0000","last_fullsized":"2024-04-26T23:43:49.282712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:01.194903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245377+0000","last_change":"2024-04-26T23:43:49.276454+0000","last_active":"2024-04-26T23:43:50.245377+0000","last_peered":"2024-04-26T23:43:50.245377+0000","last_clean":"2024-04-26T23:43:50.245377+0000","last_became_active":"2024-04-26T23:43:49.276167+0000","last_became_peered":"2024-04-26T23:43:49.276167+0000","last_unstale":"2024-04-26T23:43:50.245377+0000","last_undegraded":"2024-04-26T23:43:50.245377+0000","last_fullsized":"2024-04-26T23:43:50.245377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247802+0000","last_change":"2024-04-26T23:43:48.261837+0000","last_active":"2024-04-26T23:43:50.247802+0000","last_peered":"2024-04-26T23:43:50.247802+0000","last_clean":"2024-04-26T23:43:50.247802+0000","last_became_active":"2024-04-26T23:43:48.261551+0000","last_became_peered":"2024-04-26T23:43:48.261551+0000","last_unstale":"2024-04-26T23:43:50.247802+0000","last_undegraded":"2024-04-26T23:43:50.247802+0000","last_fullsized":"2024-04-26T23:43:50.247802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:18.700430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246064+0000","last_change":"2024-04-26T23:43:49.271363+0000","last_active":"2024-04-26T23:43:50.246064+0000","last_peered":"2024-04-26T23:43:50.246064+0000","last_clean":"2024-04-26T23:43:50.246064+0000","last_became_active":"2024-04-26T23:43:49.270409+0000","last_became_peered":"2024-04-26T23:43:49.270409+0000","last_unstale":"2024-04-26T23:43:50.246064+0000","last_undegraded":"2024-04-26T23:43:50.246064+0000","last_fullsized":"2024-04-26T23:43:50.246064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:13.511140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288607+0000","last_change":"2024-04-26T23:43:48.272655+0000","last_active":"2024-04-26T23:43:49.288607+0000","last_peered":"2024-04-26T23:43:49.288607+0000","last_clean":"2024-04-26T23:43:49.288607+0000","last_became_active":"2024-04-26T23:43:48.272213+0000","last_became_peered":"2024-04-26T23:43:48.272213+0000","last_unstale":"2024-04-26T23:43:49.288607+0000","last_undegraded":"2024-04-26T23:43:49.288607+0000","last_fullsized":"2024-04-26T23:43:49.288607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:13.642533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247588+0000","last_change":"2024-04-26T23:43:49.271469+0000","last_active":"2024-04-26T23:43:50.247588+0000","last_peered":"2024-04-26T23:43:50.247588+0000","last_clean":"2024-04-26T23:43:50.247588+0000","last_became_active":"2024-04-26T23:43:49.266320+0000","last_became_peered":"2024-04-26T23:43:49.266320+0000","last_unstale":"2024-04-26T23:43:50.247588+0000","last_undegraded":"2024-04-26T23:43:50.247588+0000","last_fullsized":"2024-04-26T23:43:50.247588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245637+0000","last_change":"2024-04-26T23:43:48.257801+0000","last_active":"2024-04-26T23:43:50.245637+0000","last_peered":"2024-04-26T23:43:50.245637+0000","last_clean":"2024-04-26T23:43:50.245637+0000","last_became_active":"2024-04-26T23:43:48.257201+0000","last_became_peered":"2024-04-26T23:43:48.257201+0000","last_unstale":"2024-04-26T23:43:50.245637+0000","last_undegraded":"2024-04-26T23:43:50.245637+0000","last_fullsized":"2024-04-26T23:43:50.245637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:05.126550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288139+0000","last_change":"2024-04-26T23:43:49.286057+0000","last_active":"2024-04-26T23:43:49.288139+0000","last_peered":"2024-04-26T23:43:49.288139+0000","last_clean":"2024-04-26T23:43:49.288139+0000","last_became_active":"2024-04-26T23:43:49.285728+0000","last_became_peered":"2024-04-26T23:43:49.285728+0000","last_unstale":"2024-04-26T23:43:49.288139+0000","last_undegraded":"2024-04-26T23:43:49.288139+0000","last_fullsized":"2024-04-26T23:43:49.288139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380142+0000","last_change":"2024-04-26T23:43:48.270525+0000","last_active":"2024-04-26T23:43:50.380142+0000","last_peered":"2024-04-26T23:43:50.380142+0000","last_clean":"2024-04-26T23:43:50.380142+0000","last_became_active":"2024-04-26T23:43:48.269325+0000","last_became_peered":"2024-04-26T23:43:48.269325+0000","last_unstale":"2024-04-26T23:43:50.380142+0000","last_undegraded":"2024-04-26T23:43:50.380142+0000","last_fullsized":"2024-04-26T23:43:50.380142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:05.144529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245740+0000","last_change":"2024-04-26T23:43:49.268453+0000","last_active":"2024-04-26T23:43:50.245740+0000","last_peered":"2024-04-26T23:43:50.245740+0000","last_clean":"2024-04-26T23:43:50.245740+0000","last_became_active":"2024-04-26T23:43:49.268164+0000","last_became_peered":"2024-04-26T23:43:49.268164+0000","last_unstale":"2024-04-26T23:43:50.245740+0000","last_undegraded":"2024-04-26T23:43:50.245740+0000","last_fullsized":"2024-04-26T23:43:50.245740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:05:17.580086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401578+0000","last_change":"2024-04-26T23:43:48.267082+0000","last_active":"2024-04-26T23:43:50.401578+0000","last_peered":"2024-04-26T23:43:50.401578+0000","last_clean":"2024-04-26T23:43:50.401578+0000","last_became_active":"2024-04-26T23:43:48.266766+0000","last_became_peered":"2024-04-26T23:43:48.266766+0000","last_unstale":"2024-04-26T23:43:50.401578+0000","last_undegraded":"2024-04-26T23:43:50.401578+0000","last_fullsized":"2024-04-26T23:43:50.401578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.027181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.263928+0000","last_change":"2024-04-26T23:43:49.258867+0000","last_active":"2024-04-26T23:43:49.263928+0000","last_peered":"2024-04-26T23:43:49.263928+0000","last_clean":"2024-04-26T23:43:49.263928+0000","last_became_active":"2024-04-26T23:43:49.258563+0000","last_became_peered":"2024-04-26T23:43:49.258563+0000","last_unstale":"2024-04-26T23:43:49.263928+0000","last_undegraded":"2024-04-26T23:43:49.263928+0000","last_fullsized":"2024-04-26T23:43:49.263928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:35.068481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.253140+0000","last_change":"2024-04-26T23:43:48.258153+0000","last_active":"2024-04-26T23:43:49.253140+0000","last_peered":"2024-04-26T23:43:49.253140+0000","last_clean":"2024-04-26T23:43:49.253140+0000","last_became_active":"2024-04-26T23:43:48.257819+0000","last_became_peered":"2024-04-26T23:43:48.257819+0000","last_unstale":"2024-04-26T23:43:49.253140+0000","last_undegraded":"2024-04-26T23:43:49.253140+0000","last_fullsized":"2024-04-26T23:43:49.253140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:24.844178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248954+0000","last_change":"2024-04-26T23:43:49.285079+0000","last_active":"2024-04-26T23:43:50.248954+0000","last_peered":"2024-04-26T23:43:50.248954+0000","last_clean":"2024-04-26T23:43:50.248954+0000","last_became_active":"2024-04-26T23:43:49.284741+0000","last_became_peered":"2024-04-26T23:43:49.284741+0000","last_unstale":"2024-04-26T23:43:50.248954+0000","last_undegraded":"2024-04-26T23:43:50.248954+0000","last_fullsized":"2024-04-26T23:43:50.248954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:35.457309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380866+0000","last_change":"2024-04-26T23:43:48.274029+0000","last_active":"2024-04-26T23:43:50.380866+0000","last_peered":"2024-04-26T23:43:50.380866+0000","last_clean":"2024-04-26T23:43:50.380866+0000","last_became_active":"2024-04-26T23:43:48.273749+0000","last_became_peered":"2024-04-26T23:43:48.273749+0000","last_unstale":"2024-04-26T23:43:50.380866+0000","last_undegraded":"2024-04-26T23:43:50.380866+0000","last_fullsized":"2024-04-26T23:43:50.380866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:57.220373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282075+0000","last_change":"2024-04-26T23:43:49.267651+0000","last_active":"2024-04-26T23:43:49.282075+0000","last_peered":"2024-04-26T23:43:49.282075+0000","last_clean":"2024-04-26T23:43:49.282075+0000","last_became_active":"2024-04-26T23:43:49.266997+0000","last_became_peered":"2024-04-26T23:43:49.266997+0000","last_unstale":"2024-04-26T23:43:49.282075+0000","last_undegraded":"2024-04-26T23:43:49.282075+0000","last_fullsized":"2024-04-26T23:43:49.282075+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.290578+0000","last_change":"2024-04-26T23:43:48.272658+0000","last_active":"2024-04-26T23:43:49.290578+0000","last_peered":"2024-04-26T23:43:49.290578+0000","last_clean":"2024-04-26T23:43:49.290578+0000","last_became_active":"2024-04-26T23:43:48.272297+0000","last_became_peered":"2024-04-26T23:43:48.272297+0000","last_unstale":"2024-04-26T23:43:49.290578+0000","last_undegraded":"2024-04-26T23:43:49.290578+0000","last_fullsized":"2024-04-26T23:43:49.290578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:00.819196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.272712+0000","last_change":"2024-04-26T23:43:49.269214+0000","last_active":"2024-04-26T23:43:49.272712+0000","last_peered":"2024-04-26T23:43:49.272712+0000","last_clean":"2024-04-26T23:43:49.272712+0000","last_became_active":"2024-04-26T23:43:49.268915+0000","last_became_peered":"2024-04-26T23:43:49.268915+0000","last_unstale":"2024-04-26T23:43:49.272712+0000","last_undegraded":"2024-04-26T23:43:49.272712+0000","last_fullsized":"2024-04-26T23:43:49.272712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:12.117486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.286288+0000","last_change":"2024-04-26T23:43:48.262515+0000","last_active":"2024-04-26T23:43:49.286288+0000","last_peered":"2024-04-26T23:43:49.286288+0000","last_clean":"2024-04-26T23:43:49.286288+0000","last_became_active":"2024-04-26T23:43:48.262236+0000","last_became_peered":"2024-04-26T23:43:48.262236+0000","last_unstale":"2024-04-26T23:43:49.286288+0000","last_undegraded":"2024-04-26T23:43:49.286288+0000","last_fullsized":"2024-04-26T23:43:49.286288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:50.001320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.287022+0000","last_change":"2024-04-26T23:43:49.283662+0000","last_active":"2024-04-26T23:43:49.287022+0000","last_peered":"2024-04-26T23:43:49.287022+0000","last_clean":"2024-04-26T23:43:49.287022+0000","last_became_active":"2024-04-26T23:43:49.283285+0000","last_became_peered":"2024-04-26T23:43:49.283285+0000","last_unstale":"2024-04-26T23:43:49.287022+0000","last_undegraded":"2024-04-26T23:43:49.287022+0000","last_fullsized":"2024-04-26T23:43:49.287022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:16.841615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400538+0000","last_change":"2024-04-26T23:43:48.273533+0000","last_active":"2024-04-26T23:43:50.400538+0000","last_peered":"2024-04-26T23:43:50.400538+0000","last_clean":"2024-04-26T23:43:50.400538+0000","last_became_active":"2024-04-26T23:43:48.272771+0000","last_became_peered":"2024-04-26T23:43:48.272771+0000","last_unstale":"2024-04-26T23:43:50.400538+0000","last_undegraded":"2024-04-26T23:43:50.400538+0000","last_fullsized":"2024-04-26T23:43:50.400538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:11.100374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.273112+0000","last_change":"2024-04-26T23:43:49.268115+0000","last_active":"2024-04-26T23:43:49.273112+0000","last_peered":"2024-04-26T23:43:49.273112+0000","last_clean":"2024-04-26T23:43:49.273112+0000","last_became_active":"2024-04-26T23:43:49.267811+0000","last_became_peered":"2024-04-26T23:43:49.267811+0000","last_unstale":"2024-04-26T23:43:49.273112+0000","last_undegraded":"2024-04-26T23:43:49.273112+0000","last_fullsized":"2024-04-26T23:43:49.273112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:24.933653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380243+0000","last_change":"2024-04-26T23:43:48.272786+0000","last_active":"2024-04-26T23:43:50.380243+0000","last_peered":"2024-04-26T23:43:50.380243+0000","last_clean":"2024-04-26T23:43:50.380243+0000","last_became_active":"2024-04-26T23:43:48.272517+0000","last_became_peered":"2024-04-26T23:43:48.272517+0000","last_unstale":"2024-04-26T23:43:50.380243+0000","last_undegraded":"2024-04-26T23:43:50.380243+0000","last_fullsized":"2024-04-26T23:43:50.380243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:50.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.282903+0000","last_change":"2024-04-26T23:43:49.277153+0000","last_active":"2024-04-26T23:43:49.282903+0000","last_peered":"2024-04-26T23:43:49.282903+0000","last_clean":"2024-04-26T23:43:49.282903+0000","last_became_active":"2024-04-26T23:43:49.275809+0000","last_became_peered":"2024-04-26T23:43:49.275809+0000","last_unstale":"2024-04-26T23:43:49.282903+0000","last_undegraded":"2024-04-26T23:43:49.282903+0000","last_fullsized":"2024-04-26T23:43:49.282903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:20.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245603+0000","last_change":"2024-04-26T23:43:48.268648+0000","last_active":"2024-04-26T23:43:50.245603+0000","last_peered":"2024-04-26T23:43:50.245603+0000","last_clean":"2024-04-26T23:43:50.245603+0000","last_became_active":"2024-04-26T23:43:48.268378+0000","last_became_peered":"2024-04-26T23:43:48.268378+0000","last_unstale":"2024-04-26T23:43:50.245603+0000","last_undegraded":"2024-04-26T23:43:50.245603+0000","last_fullsized":"2024-04-26T23:43:50.245603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:43:02.042319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247918+0000","last_change":"2024-04-26T23:43:49.281360+0000","last_active":"2024-04-26T23:43:50.247918+0000","last_peered":"2024-04-26T23:43:50.247918+0000","last_clean":"2024-04-26T23:43:50.247918+0000","last_became_active":"2024-04-26T23:43:49.276490+0000","last_became_peered":"2024-04-26T23:43:49.276490+0000","last_unstale":"2024-04-26T23:43:50.247918+0000","last_undegraded":"2024-04-26T23:43:50.247918+0000","last_fullsized":"2024-04-26T23:43:50.247918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401729+0000","last_change":"2024-04-26T23:43:48.268775+0000","last_active":"2024-04-26T23:43:50.401729+0000","last_peered":"2024-04-26T23:43:50.401729+0000","last_clean":"2024-04-26T23:43:50.401729+0000","last_became_active":"2024-04-26T23:43:48.268103+0000","last_became_peered":"2024-04-26T23:43:48.268103+0000","last_unstale":"2024-04-26T23:43:50.401729+0000","last_undegraded":"2024-04-26T23:43:50.401729+0000","last_fullsized":"2024-04-26T23:43:50.401729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:11.911808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380414+0000","last_change":"2024-04-26T23:43:49.272024+0000","last_active":"2024-04-26T23:43:50.380414+0000","last_peered":"2024-04-26T23:43:50.380414+0000","last_clean":"2024-04-26T23:43:50.380414+0000","last_became_active":"2024-04-26T23:43:49.271718+0000","last_became_peered":"2024-04-26T23:43:49.271718+0000","last_unstale":"2024-04-26T23:43:50.380414+0000","last_undegraded":"2024-04-26T23:43:50.380414+0000","last_fullsized":"2024-04-26T23:43:50.380414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:41.510299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248237+0000","last_change":"2024-04-26T23:43:48.277185+0000","last_active":"2024-04-26T23:43:50.248237+0000","last_peered":"2024-04-26T23:43:50.248237+0000","last_clean":"2024-04-26T23:43:50.248237+0000","last_became_active":"2024-04-26T23:43:48.276850+0000","last_became_peered":"2024-04-26T23:43:48.276850+0000","last_unstale":"2024-04-26T23:43:50.248237+0000","last_undegraded":"2024-04-26T23:43:50.248237+0000","last_fullsized":"2024-04-26T23:43:50.248237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:07.878542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380168+0000","last_change":"2024-04-26T23:43:49.278338+0000","last_active":"2024-04-26T23:43:50.380168+0000","last_peered":"2024-04-26T23:43:50.380168+0000","last_clean":"2024-04-26T23:43:50.380168+0000","last_became_active":"2024-04-26T23:43:49.277925+0000","last_became_peered":"2024-04-26T23:43:49.277925+0000","last_unstale":"2024-04-26T23:43:50.380168+0000","last_undegraded":"2024-04-26T23:43:50.380168+0000","last_fullsized":"2024-04-26T23:43:50.380168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:05.965232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380846+0000","last_change":"2024-04-26T23:43:48.263020+0000","last_active":"2024-04-26T23:43:50.380846+0000","last_peered":"2024-04-26T23:43:50.380846+0000","last_clean":"2024-04-26T23:43:50.380846+0000","last_became_active":"2024-04-26T23:43:48.262636+0000","last_became_peered":"2024-04-26T23:43:48.262636+0000","last_unstale":"2024-04-26T23:43:50.380846+0000","last_undegraded":"2024-04-26T23:43:50.380846+0000","last_fullsized":"2024-04-26T23:43:50.380846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:22:42.857702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249232+0000","last_change":"2024-04-26T23:43:49.275183+0000","last_active":"2024-04-26T23:43:50.249232+0000","last_peered":"2024-04-26T23:43:50.249232+0000","last_clean":"2024-04-26T23:43:50.249232+0000","last_became_active":"2024-04-26T23:43:49.274846+0000","last_became_peered":"2024-04-26T23:43:49.274846+0000","last_unstale":"2024-04-26T23:43:50.249232+0000","last_undegraded":"2024-04-26T23:43:50.249232+0000","last_fullsized":"2024-04-26T23:43:50.249232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.267076+0000","last_change":"2024-04-26T23:43:48.267385+0000","last_active":"2024-04-26T23:43:49.267076+0000","last_peered":"2024-04-26T23:43:49.267076+0000","last_clean":"2024-04-26T23:43:49.267076+0000","last_became_active":"2024-04-26T23:43:48.267110+0000","last_became_peered":"2024-04-26T23:43:48.267110+0000","last_unstale":"2024-04-26T23:43:49.267076+0000","last_undegraded":"2024-04-26T23:43:49.267076+0000","last_fullsized":"2024-04-26T23:43:49.267076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:09.842604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245621+0000","last_change":"2024-04-26T23:43:49.269004+0000","last_active":"2024-04-26T23:43:50.245621+0000","last_peered":"2024-04-26T23:43:50.245621+0000","last_clean":"2024-04-26T23:43:50.245621+0000","last_became_active":"2024-04-26T23:43:49.268622+0000","last_became_peered":"2024-04-26T23:43:49.268622+0000","last_unstale":"2024-04-26T23:43:50.245621+0000","last_undegraded":"2024-04-26T23:43:50.245621+0000","last_fullsized":"2024-04-26T23:43:50.245621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:55.539821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248895+0000","last_change":"2024-04-26T23:43:48.276629+0000","last_active":"2024-04-26T23:43:50.248895+0000","last_peered":"2024-04-26T23:43:50.248895+0000","last_clean":"2024-04-26T23:43:50.248895+0000","last_became_active":"2024-04-26T23:43:48.276310+0000","last_became_peered":"2024-04-26T23:43:48.276310+0000","last_unstale":"2024-04-26T23:43:50.248895+0000","last_undegraded":"2024-04-26T23:43:50.248895+0000","last_fullsized":"2024-04-26T23:43:50.248895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:32.905965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245985+0000","last_change":"2024-04-26T23:43:49.254506+0000","last_active":"2024-04-26T23:43:50.245985+0000","last_peered":"2024-04-26T23:43:50.245985+0000","last_clean":"2024-04-26T23:43:50.245985+0000","last_became_active":"2024-04-26T23:43:49.254150+0000","last_became_peered":"2024-04-26T23:43:49.254150+0000","last_unstale":"2024-04-26T23:43:50.245985+0000","last_undegraded":"2024-04-26T23:43:50.245985+0000","last_fullsized":"2024-04-26T23:43:50.245985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:20.380543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.248216+0000","last_change":"2024-04-26T23:43:48.271236+0000","last_active":"2024-04-26T23:43:50.248216+0000","last_peered":"2024-04-26T23:43:50.248216+0000","last_clean":"2024-04-26T23:43:50.248216+0000","last_became_active":"2024-04-26T23:43:48.270973+0000","last_became_peered":"2024-04-26T23:43:48.270973+0000","last_unstale":"2024-04-26T23:43:50.248216+0000","last_undegraded":"2024-04-26T23:43:50.248216+0000","last_fullsized":"2024-04-26T23:43:50.248216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:43.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.270290+0000","last_change":"2024-04-26T23:43:49.266112+0000","last_active":"2024-04-26T23:43:49.270290+0000","last_peered":"2024-04-26T23:43:49.270290+0000","last_clean":"2024-04-26T23:43:49.270290+0000","last_became_active":"2024-04-26T23:43:49.265831+0000","last_became_peered":"2024-04-26T23:43:49.265831+0000","last_unstale":"2024-04-26T23:43:49.270290+0000","last_undegraded":"2024-04-26T23:43:49.270290+0000","last_fullsized":"2024-04-26T23:43:49.270290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:36.307742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249346+0000","last_change":"2024-04-26T23:43:48.271994+0000","last_active":"2024-04-26T23:43:50.249346+0000","last_peered":"2024-04-26T23:43:50.249346+0000","last_clean":"2024-04-26T23:43:50.249346+0000","last_became_active":"2024-04-26T23:43:48.271637+0000","last_became_peered":"2024-04-26T23:43:48.271637+0000","last_unstale":"2024-04-26T23:43:50.249346+0000","last_undegraded":"2024-04-26T23:43:50.249346+0000","last_fullsized":"2024-04-26T23:43:50.249346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.027181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.246917+0000","last_change":"2024-04-26T23:43:49.257865+0000","last_active":"2024-04-26T23:43:50.246917+0000","last_peered":"2024-04-26T23:43:50.246917+0000","last_clean":"2024-04-26T23:43:50.246917+0000","last_became_active":"2024-04-26T23:43:49.257549+0000","last_became_peered":"2024-04-26T23:43:49.257549+0000","last_unstale":"2024-04-26T23:43:50.246917+0000","last_undegraded":"2024-04-26T23:43:50.246917+0000","last_fullsized":"2024-04-26T23:43:50.246917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:27.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.292624+0000","last_change":"2024-04-26T23:43:48.267933+0000","last_active":"2024-04-26T23:43:49.292624+0000","last_peered":"2024-04-26T23:43:49.292624+0000","last_clean":"2024-04-26T23:43:49.292624+0000","last_became_active":"2024-04-26T23:43:48.267580+0000","last_became_peered":"2024-04-26T23:43:48.267580+0000","last_unstale":"2024-04-26T23:43:49.292624+0000","last_undegraded":"2024-04-26T23:43:49.292624+0000","last_fullsized":"2024-04-26T23:43:49.292624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:06.059243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.283088+0000","last_change":"2024-04-26T23:43:49.278318+0000","last_active":"2024-04-26T23:43:49.283088+0000","last_peered":"2024-04-26T23:43:49.283088+0000","last_clean":"2024-04-26T23:43:49.283088+0000","last_became_active":"2024-04-26T23:43:49.276396+0000","last_became_peered":"2024-04-26T23:43:49.276396+0000","last_unstale":"2024-04-26T23:43:49.283088+0000","last_undegraded":"2024-04-26T23:43:49.283088+0000","last_fullsized":"2024-04-26T23:43:49.283088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:10.859716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245992+0000","last_change":"2024-04-26T23:43:48.266508+0000","last_active":"2024-04-26T23:43:50.245992+0000","last_peered":"2024-04-26T23:43:50.245992+0000","last_clean":"2024-04-26T23:43:50.245992+0000","last_became_active":"2024-04-26T23:43:48.266199+0000","last_became_peered":"2024-04-26T23:43:48.266199+0000","last_unstale":"2024-04-26T23:43:50.245992+0000","last_undegraded":"2024-04-26T23:43:50.245992+0000","last_fullsized":"2024-04-26T23:43:50.245992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.669722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.247533+0000","last_change":"2024-04-26T23:43:49.278365+0000","last_active":"2024-04-26T23:43:50.247533+0000","last_peered":"2024-04-26T23:43:50.247533+0000","last_clean":"2024-04-26T23:43:50.247533+0000","last_became_active":"2024-04-26T23:43:49.278100+0000","last_became_peered":"2024-04-26T23:43:49.278100+0000","last_unstale":"2024-04-26T23:43:50.247533+0000","last_undegraded":"2024-04-26T23:43:50.247533+0000","last_fullsized":"2024-04-26T23:43:50.247533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401478+0000","last_change":"2024-04-26T23:43:48.259534+0000","last_active":"2024-04-26T23:43:50.401478+0000","last_peered":"2024-04-26T23:43:50.401478+0000","last_clean":"2024-04-26T23:43:50.401478+0000","last_became_active":"2024-04-26T23:43:48.259223+0000","last_became_peered":"2024-04-26T23:43:48.259223+0000","last_unstale":"2024-04-26T23:43:50.401478+0000","last_undegraded":"2024-04-26T23:43:50.401478+0000","last_fullsized":"2024-04-26T23:43:50.401478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:43.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.245146+0000","last_change":"2024-04-26T23:43:49.274564+0000","last_active":"2024-04-26T23:43:50.245146+0000","last_peered":"2024-04-26T23:43:50.245146+0000","last_clean":"2024-04-26T23:43:50.245146+0000","last_became_active":"2024-04-26T23:43:49.273784+0000","last_became_peered":"2024-04-26T23:43:49.273784+0000","last_unstale":"2024-04-26T23:43:50.245146+0000","last_undegraded":"2024-04-26T23:43:50.245146+0000","last_fullsized":"2024-04-26T23:43:50.245146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:25.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.400868+0000","last_change":"2024-04-26T23:43:48.270164+0000","last_active":"2024-04-26T23:43:50.400868+0000","last_peered":"2024-04-26T23:43:50.400868+0000","last_clean":"2024-04-26T23:43:50.400868+0000","last_became_active":"2024-04-26T23:43:48.269610+0000","last_became_peered":"2024-04-26T23:43:48.269610+0000","last_unstale":"2024-04-26T23:43:50.400868+0000","last_undegraded":"2024-04-26T23:43:50.400868+0000","last_fullsized":"2024-04-26T23:43:50.400868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:32.905965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.287058+0000","last_change":"2024-04-26T23:43:49.284103+0000","last_active":"2024-04-26T23:43:49.287058+0000","last_peered":"2024-04-26T23:43:49.287058+0000","last_clean":"2024-04-26T23:43:49.287058+0000","last_became_active":"2024-04-26T23:43:49.283704+0000","last_became_peered":"2024-04-26T23:43:49.283704+0000","last_unstale":"2024-04-26T23:43:49.287058+0000","last_undegraded":"2024-04-26T23:43:49.287058+0000","last_fullsized":"2024-04-26T23:43:49.287058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:45.654839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.254450+0000","last_change":"2024-04-26T23:43:48.267306+0000","last_active":"2024-04-26T23:43:49.254450+0000","last_peered":"2024-04-26T23:43:49.254450+0000","last_clean":"2024-04-26T23:43:49.254450+0000","last_became_active":"2024-04-26T23:43:48.267031+0000","last_became_peered":"2024-04-26T23:43:48.267031+0000","last_unstale":"2024-04-26T23:43:49.254450+0000","last_undegraded":"2024-04-26T23:43:49.254450+0000","last_fullsized":"2024-04-26T23:43:49.254450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:43.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.288120+0000","last_change":"2024-04-26T23:43:49.285770+0000","last_active":"2024-04-26T23:43:49.288120+0000","last_peered":"2024-04-26T23:43:49.288120+0000","last_clean":"2024-04-26T23:43:49.288120+0000","last_became_active":"2024-04-26T23:43:49.285440+0000","last_became_peered":"2024-04-26T23:43:49.285440+0000","last_unstale":"2024-04-26T23:43:49.288120+0000","last_undegraded":"2024-04-26T23:43:49.288120+0000","last_fullsized":"2024-04-26T23:43:49.288120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:04.635216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.380471+0000","last_change":"2024-04-26T23:43:48.277021+0000","last_active":"2024-04-26T23:43:50.380471+0000","last_peered":"2024-04-26T23:43:50.380471+0000","last_clean":"2024-04-26T23:43:50.380471+0000","last_became_active":"2024-04-26T23:43:48.276741+0000","last_became_peered":"2024-04-26T23:43:48.276741+0000","last_unstale":"2024-04-26T23:43:50.380471+0000","last_undegraded":"2024-04-26T23:43:50.380471+0000","last_fullsized":"2024-04-26T23:43:50.380471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:27.654355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291552+0000","last_change":"2024-04-26T23:43:49.289735+0000","last_active":"2024-04-26T23:43:49.291552+0000","last_peered":"2024-04-26T23:43:49.291552+0000","last_clean":"2024-04-26T23:43:49.291552+0000","last_became_active":"2024-04-26T23:43:49.289127+0000","last_became_peered":"2024-04-26T23:43:49.289127+0000","last_unstale":"2024-04-26T23:43:49.291552+0000","last_undegraded":"2024-04-26T23:43:49.291552+0000","last_fullsized":"2024-04-26T23:43:49.291552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:57.649051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.256240+0000","last_change":"2024-04-26T23:43:48.275725+0000","last_active":"2024-04-26T23:43:49.256240+0000","last_peered":"2024-04-26T23:43:49.256240+0000","last_clean":"2024-04-26T23:43:49.256240+0000","last_became_active":"2024-04-26T23:43:48.275410+0000","last_became_peered":"2024-04-26T23:43:48.275410+0000","last_unstale":"2024-04-26T23:43:49.256240+0000","last_undegraded":"2024-04-26T23:43:49.256240+0000","last_fullsized":"2024-04-26T23:43:49.256240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:56.258918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.291873+0000","last_change":"2024-04-26T23:43:49.282195+0000","last_active":"2024-04-26T23:43:49.291873+0000","last_peered":"2024-04-26T23:43:49.291873+0000","last_clean":"2024-04-26T23:43:49.291873+0000","last_became_active":"2024-04-26T23:43:49.280767+0000","last_became_peered":"2024-04-26T23:43:49.280767+0000","last_unstale":"2024-04-26T23:43:49.291873+0000","last_undegraded":"2024-04-26T23:43:49.291873+0000","last_fullsized":"2024-04-26T23:43:49.291873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:10.133959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-26T23:43:49.269796+0000","last_change":"2024-04-26T23:43:48.276508+0000","last_active":"2024-04-26T23:43:49.269796+0000","last_peered":"2024-04-26T23:43:49.269796+0000","last_clean":"2024-04-26T23:43:49.269796+0000","last_became_active":"2024-04-26T23:43:48.276231+0000","last_became_peered":"2024-04-26T23:43:48.276231+0000","last_unstale":"2024-04-26T23:43:49.269796+0000","last_undegraded":"2024-04-26T23:43:49.269796+0000","last_fullsized":"2024-04-26T23:43:49.269796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:44.637728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401416+0000","last_change":"2024-04-26T23:43:49.284165+0000","last_active":"2024-04-26T23:43:50.401416+0000","last_peered":"2024-04-26T23:43:50.401416+0000","last_clean":"2024-04-26T23:43:50.401416+0000","last_became_active":"2024-04-26T23:43:49.283732+0000","last_became_peered":"2024-04-26T23:43:49.283732+0000","last_unstale":"2024-04-26T23:43:50.401416+0000","last_undegraded":"2024-04-26T23:43:50.401416+0000","last_fullsized":"2024-04-26T23:43:50.401416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:48.217955+0000","last_clean_scrub_stamp":"2024-04-26T23:43:48.217955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:29.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.249238+0000","last_change":"2024-04-26T23:43:48.275599+0000","last_active":"2024-04-26T23:43:50.249238+0000","last_peered":"2024-04-26T23:43:50.249238+0000","last_clean":"2024-04-26T23:43:50.249238+0000","last_became_active":"2024-04-26T23:43:48.275285+0000","last_became_peered":"2024-04-26T23:43:48.275285+0000","last_unstale":"2024-04-26T23:43:50.249238+0000","last_undegraded":"2024-04-26T23:43:50.249238+0000","last_fullsized":"2024-04-26T23:43:50.249238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:43:47.200843+0000","last_clean_scrub_stamp":"2024-04-26T23:43:47.200843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:11.911808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T23:43:50.401056+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:43:50.401056+0000","last_peered":"2024-04-26T23:43:50.401056+0000","last_clean":"2024-04-26T23:43:50.401056+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:43:50.401056+0000","last_undegraded":"2024-04-26T23:43:50.401056+0000","last_fullsized":"2024-04-26T23:43:50.401056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1576960,"data_stored":1040128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":978944,"data_stored":444658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":970752,"data_stored":443080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":451454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":983040,"data_stored":450320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":978944,"data_stored":443788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":974848,"data_stored":443158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1572864,"data_stored":1040106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:43:54.283 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:43:54.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:43:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.545+0000 7fd2eb400640 1 Processor -- start 2024-04-26T23:43:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.545+0000 7fd2eb400640 1 -- start start 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2eb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2e40b53b0 0x7fd2e415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2eb400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 0x7fd2e406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2eb400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2e40d07e0 con 0x7fd2e415d730 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2eb400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd2e406eb90 con 0x7fd2e40b53b0 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2eb400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd2e40b4310 con 0x7fd2e415e050 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2e40b53b0 0x7fd2e415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2e40b53b0 0x7fd2e415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47080/0 (socket says 172.21.15.46:47080) 2024-04-26T23:43:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.547+0000 7fd2e3e00640 1 -- 172.21.15.46:0/3547170314 learned_addr learned my addr 172.21.15.46:0/3547170314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e9400640 1 --2- 172.21.15.46:0/3547170314 >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 0x7fd2e406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e3e00640 1 -- 172.21.15.46:0/3547170314 >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 msgr2=0x7fd2e406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e3e00640 1 --2- 172.21.15.46:0/3547170314 >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 0x7fd2e406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e3e00640 1 -- 172.21.15.46:0/3547170314 >> v1:172.21.15.46:6789/0 conn(0x7fd2e415d730 legacy=0x7fd2e40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e3e00640 1 -- 172.21.15.46:0/3547170314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e40b45f0 con 0x7fd2e40b53b0 2024-04-26T23:43:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.548+0000 7fd2e9400640 1 --2- 172.21.15.46:0/3547170314 >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 0x7fd2e406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.549+0000 7fd2e3e00640 1 --2- 172.21.15.46:0/3547170314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e415db10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4074180 tx=0x7fd2d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e88c76431c5279 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.549+0000 7fd2e3400640 1 -- 172.21.15.46:0/3547170314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0037070 con 0x7fd2e40b53b0 2024-04-26T23:43:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2e3400640 1 -- 172.21.15.46:0/3547170314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d0032440 con 0x7fd2e40b53b0 2024-04-26T23:43:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2e3400640 1 -- 172.21.15.46:0/3547170314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0032810 con 0x7fd2e40b53b0 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 -- 172.21.15.46:0/3547170314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 msgr2=0x7fd2e415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 --2- 172.21.15.46:0/3547170314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e415db10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4074180 tx=0x7fd2d002a270 comp rx=0 tx=0).stop 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 -- 172.21.15.46:0/3547170314 shutdown_connections 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 --2- 172.21.15.46:0/3547170314 >> v2:172.21.15.142:3300/0 conn(0x7fd2e415e050 0x7fd2e406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 --2- 172.21.15.46:0/3547170314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e415db10 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 -- 172.21.15.46:0/3547170314 >> 172.21.15.46:0/3547170314 conn(0x7fd2e406c980 msgr2=0x7fd2e40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.550+0000 7fd2eb400640 1 -- 172.21.15.46:0/3547170314 shutdown_connections 2024-04-26T23:43:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.551+0000 7fd2eb400640 1 -- 172.21.15.46:0/3547170314 wait complete. 2024-04-26T23:43:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.552+0000 7fd2eb400640 1 Processor -- start 2024-04-26T23:43:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.552+0000 7fd2eb400640 1 -- start start 2024-04-26T23:43:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2e415d730 0x7fd2e40c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2e415e050 0x7fd2e40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e40b4e90 con 0x7fd2e40b53b0 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e40b4bb0 con 0x7fd2e415e050 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.553+0000 7fd2eb400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e415e460 con 0x7fd2e415d730 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47084/0 (socket says 172.21.15.46:47084) 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 -- 172.21.15.46:0/4082241919 learned_addr learned my addr 172.21.15.46:0/4082241919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e9400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2e415e050 0x7fd2e40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e3e00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2e415d730 0x7fd2e40c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 -- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2e415d730 msgr2=0x7fd2e40c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2e415d730 0x7fd2e40c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 -- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2e415e050 msgr2=0x7fd2e40b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2e415e050 0x7fd2e40b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.554+0000 7fd2e8a00640 1 -- 172.21.15.46:0/4082241919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e40c6d80 con 0x7fd2e40b53b0 2024-04-26T23:43:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2e8a00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c430 tx=0x7fd2d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80052d0 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d8005490 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e40c7060 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8007260 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.555+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e40c7340 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.556+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4069530 con 0x7fd2e40b53b0 2024-04-26T23:43:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.557+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2d8007400 con 0x7fd2e40b53b0 2024-04-26T23:43:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.559+0000 7fd2e0c00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 0x7fd2c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.559+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd2d808b240 con 0x7fd2e40b53b0 2024-04-26T23:43:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.560+0000 7fd2e3e00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 0x7fd2c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.561+0000 7fd2e3e00640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 0x7fd2c406edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0034000 tx=0x7fd2d002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.563+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2d80594d0 con 0x7fd2e40b53b0 2024-04-26T23:43:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.842+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd2e41441e0 con 0x7fd2e40b53b0 2024-04-26T23:43:54.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.846+0000 7fd2e0c00640 1 -- 172.21.15.46:0/4082241919 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fd2d8058dd0 con 0x7fd2e40b53b0 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:43:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:43:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 msgr2=0x7fd2c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 0x7fd2c406edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0034000 tx=0x7fd2d002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 msgr2=0x7fd2e40c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c430 tx=0x7fd2d800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 shutdown_connections 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd2c406c8f0 0x7fd2c406edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2e415e050 0x7fd2e40b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2e415d730 0x7fd2e40c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 --2- 172.21.15.46:0/4082241919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e40b53b0 0x7fd2e40c6710 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 >> 172.21.15.46:0/4082241919 conn(0x7fd2e406c980 msgr2=0x7fd2e40af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 shutdown_connections 2024-04-26T23:43:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:54.850+0000 7fd2eb400640 1 -- 172.21.15.46:0/4082241919 wait complete. 2024-04-26T23:43:54.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:43:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.069+0000 7efc94400640 1 Processor -- start 2024-04-26T23:43:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.069+0000 7efc94400640 1 -- start start 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc94400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c160bd0 0x7efc8c161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc94400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc8c0732c0 0x7efc8c073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc94400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c161940 con 0x7efc8c06e510 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc94400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c071f80 con 0x7efc8c0732c0 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc94400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c072260 con 0x7efc8c160bd0 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc92400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc8c0732c0 0x7efc8c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc91a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efc8c06e510 0x7efc8c06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47416/0 (socket says 172.21.15.46:47416) 2024-04-26T23:43:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.071+0000 7efc91a00640 1 -- 172.21.15.46:0/659173884 learned_addr learned my addr 172.21.15.46:0/659173884 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc91000640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c160bd0 0x7efc8c161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc92400640 1 -- 172.21.15.46:0/659173884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c160bd0 msgr2=0x7efc8c161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc92400640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c160bd0 0x7efc8c161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc92400640 1 -- 172.21.15.46:0/659173884 >> v1:172.21.15.46:6789/0 conn(0x7efc8c06e510 legacy=0x7efc8c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc92400640 1 -- 172.21.15.46:0/659173884 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c072540 con 0x7efc8c0732c0 2024-04-26T23:43:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.072+0000 7efc8be00640 1 -- 172.21.15.46:0/659173884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436267483 0 0) 0x7efc8c161940 con 0x7efc8c06e510 2024-04-26T23:43:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.073+0000 7efc92400640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c0732c0 0x7efc8c073750 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efc84009d90 tx=0x7efc8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb8474c79c42418 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.073+0000 7efc8be00640 1 -- 172.21.15.46:0/659173884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84007740 con 0x7efc8c0732c0 2024-04-26T23:43:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.073+0000 7efc8be00640 1 -- 172.21.15.46:0/659173884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840078e0 con 0x7efc8c0732c0 2024-04-26T23:43:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.073+0000 7efc8be00640 1 -- 172.21.15.46:0/659173884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84007cb0 con 0x7efc8c0732c0 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 -- 172.21.15.46:0/659173884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c0732c0 msgr2=0x7efc8c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c0732c0 0x7efc8c073750 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efc84009d90 tx=0x7efc8402a660 comp rx=0 tx=0).stop 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 -- 172.21.15.46:0/659173884 shutdown_connections 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c0732c0 0x7efc8c073750 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 --2- 172.21.15.46:0/659173884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c160bd0 0x7efc8c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 -- 172.21.15.46:0/659173884 >> 172.21.15.46:0/659173884 conn(0x7efc8c06c530 msgr2=0x7efc8c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 -- 172.21.15.46:0/659173884 shutdown_connections 2024-04-26T23:43:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.074+0000 7efc94400640 1 -- 172.21.15.46:0/659173884 wait complete. 2024-04-26T23:43:55.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.076+0000 7efc94400640 1 Processor -- start 2024-04-26T23:43:55.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.076+0000 7efc94400640 1 -- start start 2024-04-26T23:43:55.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc8c06e510 0x7efc8c0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c0732c0 0x7efc8c145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c1616f0 con 0x7efc8c160bd0 2024-04-26T23:43:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c161570 con 0x7efc8c06e510 2024-04-26T23:43:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc94400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c072d40 con 0x7efc8c0732c0 2024-04-26T23:43:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc92400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc92400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47096/0 (socket says 172.21.15.46:47096) 2024-04-26T23:43:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.077+0000 7efc92400640 1 -- 172.21.15.46:0/3678927692 learned_addr learned my addr 172.21.15.46:0/3678927692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc91000640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c0732c0 0x7efc8c145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc91a00640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc8c06e510 0x7efc8c0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 -- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c0732c0 msgr2=0x7efc8c145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c0732c0 0x7efc8c145960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 -- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc8c06e510 msgr2=0x7efc8c0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc8c06e510 0x7efc8c0b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 -- 172.21.15.46:0/3678927692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c0b2de0 con 0x7efc8c160bd0 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc92400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efc8402ac50 tx=0x7efc84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84040070 con 0x7efc8c160bd0 2024-04-26T23:43:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84031c90 con 0x7efc8c160bd0 2024-04-26T23:43:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84045680 con 0x7efc8c160bd0 2024-04-26T23:43:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.078+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8c1526e0 con 0x7efc8c160bd0 2024-04-26T23:43:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.079+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8c152980 con 0x7efc8c160bd0 2024-04-26T23:43:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.080+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8c069530 con 0x7efc8c160bd0 2024-04-26T23:43:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.081+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc84039040 con 0x7efc8c160bd0 2024-04-26T23:43:55.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.082+0000 7efc89600640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 0x7efc6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.082+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7efc840ad680 con 0x7efc8c160bd0 2024-04-26T23:43:55.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.082+0000 7efc91a00640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 0x7efc6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.083+0000 7efc91a00640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 0x7efc6c06edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc80000f60 tx=0x7efc80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.086+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc8407b9d0 con 0x7efc8c160bd0 2024-04-26T23:43:55.206 INFO:tasks.ceph.mds.b.smithi142.stderr:2024-04-26T23:43:55.202+0000 7fd58b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:43:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.329+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc8c0cf130 con 0x7efc8c160bd0 2024-04-26T23:43:55.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.330+0000 7efc89600640 1 -- 172.21.15.46:0/3678927692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7efc84043e40 con 0x7efc8c160bd0 2024-04-26T23:43:55.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:55.335 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2819521829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2819521829},{"type":"v1","addr":"172.21.15.46:6834","nonce":2819521829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1317531531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1317531531},{"type":"v1","addr":"172.21.15.46:6835","nonce":1317531531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/634149566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":634149566},{"type":"v1","addr":"172.21.15.142:6836","nonce":634149566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:43:51.787417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2319300947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2319300947},{"type":"v1","addr":"172.21.15.142:6837","nonce":2319300947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7164,"qdb_cluster":[7164]},"id":5}]} 2024-04-26T23:43:55.335 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2024-04-26T23:43:55.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.332+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 msgr2=0x7efc6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.332+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 0x7efc6c06edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc80000f60 tx=0x7efc80009040 comp rx=0 tx=0).stop 2024-04-26T23:43:55.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.332+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 msgr2=0x7efc8c13c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efc8402ac50 tx=0x7efc84004a40 comp rx=0 tx=0).stop 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 shutdown_connections 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc6c06c8f0 0x7efc6c06edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8c160bd0 0x7efc8c13c4e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc8c0732c0 0x7efc8c145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 --2- 172.21.15.46:0/3678927692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc8c06e510 0x7efc8c0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 >> 172.21.15.46:0/3678927692 conn(0x7efc8c06c530 msgr2=0x7efc8c15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 shutdown_connections 2024-04-26T23:43:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.333+0000 7efc94400640 1 -- 172.21.15.46:0/3678927692 wait complete. 2024-04-26T23:43:55.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:43:48.721859+0000', 'modified': '2024-04-26T23:43:51.787417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7164': {'gid': 7164, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2319300947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6835', 'nonce': 2319300947}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2319300947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7164, 'qdb_cluster': [7164]} 2024-04-26T23:43:55.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:43:55.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:43:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.983+0000 7feed5c00640 1 Processor -- start 2024-04-26T23:43:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.983+0000 7feed5c00640 1 -- start start 2024-04-26T23:43:55.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.985+0000 7feed5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feed0160910 0x7feed0160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.985+0000 7feed5c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed0073ad0 con 0x7feed01466b0 2024-04-26T23:43:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feed5c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed0146a90 con 0x7feed00720e0 2024-04-26T23:43:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feed5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7feed01612a0 con 0x7feed0160910 2024-04-26T23:43:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feed5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feed0160910 0x7feed0160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feecfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feed01466b0 0x7feed006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47426/0 (socket says 172.21.15.46:47426) 2024-04-26T23:43:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feecfe00640 1 -- 172.21.15.46:0/3820751050 learned_addr learned my addr 172.21.15.46:0/3820751050 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.986+0000 7feecf400640 1 -- 172.21.15.46:0/3820751050 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4174000010 0 0) 0x7feed0073ad0 con 0x7feed01466b0 2024-04-26T23:43:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.987+0000 7feecf400640 1 -- 172.21.15.46:0/3820751050 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb4003520 con 0x7feed01466b0 2024-04-26T23:43:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.987+0000 7feed5200640 1 -- 172.21.15.46:0/3820751050 >> v1:172.21.15.142:6789/0 conn(0x7feed00720e0 legacy=0x7feed00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.987+0000 7feed5200640 1 -- 172.21.15.46:0/3820751050 >> v1:172.21.15.46:6789/0 conn(0x7feed01466b0 legacy=0x7feed006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.987+0000 7feed5200640 1 -- 172.21.15.46:0/3820751050 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed0073d70 con 0x7feed0160910 2024-04-26T23:43:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feed5200640 1 --2- 172.21.15.46:0/3820751050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed0160910 0x7feed0160d60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feec80097d0 tx=0x7feec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57c7589a6221d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feecf400640 1 -- 172.21.15.46:0/3820751050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8037070 con 0x7feed0160910 2024-04-26T23:43:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feecf400640 1 -- 172.21.15.46:0/3820751050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8032440 con 0x7feed0160910 2024-04-26T23:43:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feecf400640 1 -- 172.21.15.46:0/3820751050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8032810 con 0x7feed0160910 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feed5c00640 1 -- 172.21.15.46:0/3820751050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed0160910 msgr2=0x7feed0160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.988+0000 7feed5c00640 1 --2- 172.21.15.46:0/3820751050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed0160910 0x7feed0160d60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feec80097d0 tx=0x7feec802a270 comp rx=0 tx=0).stop 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.989+0000 7feed5c00640 1 -- 172.21.15.46:0/3820751050 shutdown_connections 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.989+0000 7feed5c00640 1 --2- 172.21.15.46:0/3820751050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed0160910 0x7feed0160d60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.989+0000 7feed5c00640 1 -- 172.21.15.46:0/3820751050 >> 172.21.15.46:0/3820751050 conn(0x7feed00b26d0 msgr2=0x7feed00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.989+0000 7feed5c00640 1 -- 172.21.15.46:0/3820751050 shutdown_connections 2024-04-26T23:43:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.989+0000 7feed5c00640 1 -- 172.21.15.46:0/3820751050 wait complete. 2024-04-26T23:43:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.991+0000 7feed5c00640 1 Processor -- start 2024-04-26T23:43:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.991+0000 7feed5c00640 1 -- start start 2024-04-26T23:43:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 0x7feed00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed0160910 0x7feed00aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feed00cb470 con 0x7feed00720e0 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feed00cb190 con 0x7feed01466b0 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feed5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feed00cb990 con 0x7feed0160910 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47128/0 (socket says 172.21.15.46:47128) 2024-04-26T23:43:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.992+0000 7feecfe00640 1 -- 172.21.15.46:0/1349971741 learned_addr learned my addr 172.21.15.46:0/1349971741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feec7400640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 0x7feed00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 -- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed0160910 msgr2=0x7feed00aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed0160910 0x7feed00aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 -- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 msgr2=0x7feed00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 0x7feed00b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 -- 172.21.15.46:0/1349971741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00b4ce0 con 0x7feed00720e0 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feec7400640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 0x7feed00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feecfe00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feec000cbc0 tx=0x7feec000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feeccc00640 1 -- 172.21.15.46:0/1349971741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0017070 con 0x7feed00720e0 2024-04-26T23:43:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feeccc00640 1 -- 172.21.15.46:0/1349971741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec000ad40 con 0x7feed00720e0 2024-04-26T23:43:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed00b4f80 con 0x7feed00720e0 2024-04-26T23:43:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.993+0000 7feeccc00640 1 -- 172.21.15.46:0/1349971741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0012600 con 0x7feed00720e0 2024-04-26T23:43:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.994+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed00b5540 con 0x7feed00720e0 2024-04-26T23:43:56.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.996+0000 7feeccc00640 1 -- 172.21.15.46:0/1349971741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feec001a070 con 0x7feed00720e0 2024-04-26T23:43:56.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.997+0000 7feeccc00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.997+0000 7feeccc00640 1 -- 172.21.15.46:0/1349971741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7feec008bb00 con 0x7feed00720e0 2024-04-26T23:43:56.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.997+0000 7feeda1b3740 1 Processor -- start 2024-04-26T23:43:56.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feec7400640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 -- start start 2024-04-26T23:43:56.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56276012d600 con 0x562760205e80 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5627600ca100 con 0x5627601f4270 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.998+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5627601295c0 con 0x5627601c97b0 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47138/0 (socket says 172.21.15.46:47138) 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 -- 172.21.15.46:0/143029892 learned_addr learned my addr 172.21.15.46:0/143029892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 0x7feeac06edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7feed00b6b90 tx=0x7feeb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feecfe00640 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feed5200640 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 -- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 msgr2=0x5627601cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 -- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 msgr2=0x5627601f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:55.999+0000 7feec7400640 1 -- 172.21.15.46:0/143029892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275fe3f2d0 con 0x562760205e80 2024-04-26T23:43:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.000+0000 7feec7400640 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feeb800da90 tx=0x7feeb8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247b2fd3955406f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.000+0000 7feec4c00640 1 -- 172.21.15.46:0/143029892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8024070 con 0x562760205e80 2024-04-26T23:43:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.000+0000 7feec4c00640 1 -- 172.21.15.46:0/143029892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8014db0 con 0x562760205e80 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.000+0000 7feec4c00640 1 -- 172.21.15.46:0/143029892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb801f620 con 0x562760205e80 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 -- 172.21.15.46:0/143029892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 msgr2=0x5627601c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feeb800da90 tx=0x7feeb8010000 comp rx=0 tx=0).stop 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 -- 172.21.15.46:0/143029892 shutdown_connections 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x5627601c9b90 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 --2- 172.21.15.46:0/143029892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 -- 172.21.15.46:0/143029892 >> 172.21.15.46:0/143029892 conn(0x56276015dd10 msgr2=0x56276020f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 -- 172.21.15.46:0/143029892 shutdown_connections 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.001+0000 7feeda1b3740 1 -- 172.21.15.46:0/143029892 wait complete. 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.002+0000 7feeda1b3740 1 -- start start 2024-04-26T23:43:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.002+0000 7feeda1b3740 1 Processor -- start 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601c97b0 0x5627601ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562760205e80 0x5627601eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5627600b4f70 con 0x5627601f4270 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5627601f1dc0 con 0x5627601c97b0 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56276021f240 con 0x562760205e80 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feeda1b3740 10 client.0 _ll_register_callbacks cb 0x562760123480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47152/0 (socket says 172.21.15.46:47152) 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feec7400640 1 -- 172.21.15.46:0/2350525818 learned_addr learned my addr 172.21.15.46:0/2350525818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.003+0000 7feecfe00640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601c97b0 0x5627601ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feed5200640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562760205e80 0x5627601eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 -- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562760205e80 msgr2=0x5627601eca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562760205e80 0x5627601eca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 -- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601c97b0 msgr2=0x5627601ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601c97b0 0x5627601ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 -- 172.21.15.46:0/2350525818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275ff096b0 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feec7400640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000f60 tx=0x7feeb8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919310f55632e855 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feebd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8030060 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb8014db0 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.004+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7feeb80067d0 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7feeb8036050 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 10 client.7314 fetch_fsmap learned FSMap version 59 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x56276021ef80 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8014db0 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7feeb8039020 con 0x5627601f4270 2024-04-26T23:43:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 10 client.7314 fetch_fsmap finished waiting for FSMap version 59 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 10 client.7314 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 20 client.7314 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601f59f0 0x56276022c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 4 client.7314 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.005+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 --> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56276013b210 con 0x5627601f59f0 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.006+0000 7feecfe00640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601f59f0 0x56276022c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.007+0000 7feecfe00640 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601f59f0 0x56276022c7a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2a30c66f2a6cb8 server_cookie=9eb8a246dc6babe0 in_seq=0 out_seq=0 2024-04-26T23:43:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.007+0000 7feebd600640 10 client.7314 ms_handle_connect on v2:172.21.15.142:6835/2319300947 2024-04-26T23:43:56.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.009+0000 7feebd600640 1 -- 172.21.15.46:0/2350525818 <== mds.0 v2:172.21.15.142:6835/2319300947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56276013b210 con 0x5627601f59f0 2024-04-26T23:43:56.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.009+0000 7feebd600640 10 client.7314 handle_command_reply: tid=0 2024-04-26T23:43:56.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.009+0000 7feeda1b3740 1 client.7314 shutdown 2024-04-26T23:43:56.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 msgr2=0x5627601eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000f60 tx=0x7feeb8000ed0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 shutdown_connections 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601f59f0 0x56276022c7a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562760205e80 0x5627601eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601f4270 0x5627601eeb80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 --2- 172.21.15.46:0/2350525818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601c97b0 0x5627601ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feebd600640 0 client.7314 ms_handle_reset on v2:172.21.15.142:6835/2319300947 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 >> 172.21.15.46:0/2350525818 conn(0x56276015dd10 msgr2=0x5627601f1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 shutdown_connections 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 1 -- 172.21.15.46:0/2350525818 wait complete. 2024-04-26T23:43:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.010+0000 7feeda1b3740 20 client.7314 trim_cache size 0 max 16384 2024-04-26T23:43:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.030+0000 7feeda1b3740 1 Processor -- start 2024-04-26T23:43:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.030+0000 7feeda1b3740 1 -- start start 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feed0154530 con 0x562760205e80 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feed0152940 con 0x5627601f4270 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feed0153ee0 con 0x5627601c97b0 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.031+0000 7feed5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47168/0 (socket says 172.21.15.46:47168) 2024-04-26T23:43:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 -- 172.21.15.46:0/986037573 learned_addr learned my addr 172.21.15.46:0/986037573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feecfe00640 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 -- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 msgr2=0x5627601f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 -- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 msgr2=0x5627601f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 -- 172.21.15.46:0/986037573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562760221750 con 0x562760205e80 2024-04-26T23:43:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.032+0000 7feed5200640 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7feec802a860 tx=0x7feec802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68644748fd78a96d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feebfe00640 1 -- 172.21.15.46:0/986037573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8043070 con 0x562760205e80 2024-04-26T23:43:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feebfe00640 1 -- 172.21.15.46:0/986037573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec803c640 con 0x562760205e80 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feebfe00640 1 -- 172.21.15.46:0/986037573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8037040 con 0x562760205e80 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 -- 172.21.15.46:0/986037573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 msgr2=0x56276021d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7feec802a860 tx=0x7feec802c070 comp rx=0 tx=0).stop 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 -- 172.21.15.46:0/986037573 shutdown_connections 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562760205e80 0x56276021d990 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627601f4270 0x5627601f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 --2- 172.21.15.46:0/986037573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601c97b0 0x5627601f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 -- 172.21.15.46:0/986037573 >> 172.21.15.46:0/986037573 conn(0x56276015dd10 msgr2=0x5627601ea300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 -- 172.21.15.46:0/986037573 shutdown_connections 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.033+0000 7feeda1b3740 1 -- 172.21.15.46:0/986037573 wait complete. 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.034+0000 7feeda1b3740 1 -- start start 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.034+0000 7feeda1b3740 1 Processor -- start 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601f4270 0x5627602053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562760205e80 0x562760205920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5627601f5870 con 0x5627601c97b0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56276020ff40 con 0x562760205e80 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5627601ecf70 con 0x5627601f4270 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feeda1b3740 10 client.0 _ll_register_callbacks cb 0x7feeb800df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47178/0 (socket says 172.21.15.46:47178) 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.035+0000 7feecfe00640 1 -- 172.21.15.46:0/3193351943 learned_addr learned my addr 172.21.15.46:0/3193351943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 -- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601f4270 msgr2=0x5627602053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601f4270 0x5627602053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 -- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562760205e80 msgr2=0x562760205920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562760205e80 0x562760205920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 -- 172.21.15.46:0/3193351943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562760204c00 con 0x5627601c97b0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feecfe00640 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7feec0001560 tx=0x7feec00014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f641d876e0e9a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feebd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:43:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0075020 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7feec0061f50 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.036+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec803c640 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7feec0004b60 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 10 client.7320 fetch_fsmap learned FSMap version 59 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5627601f1290 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec803c640 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7feec005eb40 con 0x5627601c97b0 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 10 client.7320 fetch_fsmap finished waiting for FSMap version 59 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 10 client.7320 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 20 client.7320 populate_metadata read hostname 'smithi046' 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601c7f70 0x5627601c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 4 client.7320 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.037+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 --> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] -- command(tid 0: {"prefix": "status"}) -- 0x562760135080 con 0x5627601c7f70 2024-04-26T23:43:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.038+0000 7feec7400640 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601c7f70 0x5627601c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feec7400640 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601c7f70 0x5627601c8350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50ee09c53407e2f server_cookie=c2bb5b2629d02338 in_seq=0 out_seq=0 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feebd600640 10 client.7320 ms_handle_connect on v2:172.21.15.142:6835/2319300947 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feebd600640 1 -- 172.21.15.46:0/3193351943 <== mds.0 v2:172.21.15.142:6835/2319300947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562760135080 con 0x5627601c7f70 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feebd600640 10 client.7320 handle_command_reply: tid=0 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 client.7320 shutdown 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 msgr2=0x56276026f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7feec0001560 tx=0x7feec00014d0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 shutdown_connections 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:6835/2319300947,v1:172.21.15.142:6837/2319300947] conn(0x5627601c7f70 0x5627601c8350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562760205e80 0x562760205920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5627601f4270 0x5627602053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feebd600640 0 client.7320 ms_handle_reset on v2:172.21.15.142:6835/2319300947 2024-04-26T23:43:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 --2- 172.21.15.46:0/3193351943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627601c97b0 0x56276026f690 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.039+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 >> 172.21.15.46:0/3193351943 conn(0x56276015dd10 msgr2=0x562760231a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 shutdown_connections 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feeda1b3740 1 -- 172.21.15.46:0/3193351943 wait complete. 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feeda1b3740 20 client.7320 trim_cache size 0 max 16384 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:43:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7164, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7999390660000003, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 59, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.178095621000001 2024-04-26T23:43:56.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 msgr2=0x7feeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 0x7feeac06edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7feed00b6b90 tx=0x7feeb8009040 comp rx=0 tx=0).stop 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 msgr2=0x7feed00b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feec000cbc0 tx=0x7feec000a430 comp rx=0 tx=0).stop 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 shutdown_connections 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed0160910 0x7feed00aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed01466b0 0x7feed00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 --2- 172.21.15.46:0/1349971741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed00720e0 0x7feed00b1500 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.040+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 >> 172.21.15.46:0/1349971741 conn(0x7feed00b26d0 msgr2=0x7feed00cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.041+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 shutdown_connections 2024-04-26T23:43:56.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.041+0000 7feed5c00640 1 -- 172.21.15.46:0/1349971741 wait complete. 2024-04-26T23:43:56.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:43:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.270+0000 7f6a5ae00640 1 Processor -- start 2024-04-26T23:43:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.270+0000 7f6a5ae00640 1 -- start start 2024-04-26T23:43:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a5ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a5415f200 0x7f6a540b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a5ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a540b6400 con 0x7f6a540af2f0 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a5ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a540b5100 con 0x7f6a5406da70 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a5ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6a540b4eb0 con 0x7f6a5415f200 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a59a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a5415f200 0x7f6a540b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a59a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a5415f200 0x7f6a540b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47190/0 (socket says 172.21.15.46:47190) 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a59a00640 1 -- 172.21.15.46:0/1339420475 learned_addr learned my addr 172.21.15.46:0/1339420475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.272+0000 7f6a59000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a540af2f0 0x7f6a5406d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47430/0 (socket says 172.21.15.46:47430) 2024-04-26T23:43:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.273+0000 7f6a59a00640 1 -- 172.21.15.46:0/1339420475 >> v1:172.21.15.142:6789/0 conn(0x7f6a5406da70 legacy=0x7f6a5406e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.273+0000 7f6a59a00640 1 -- 172.21.15.46:0/1339420475 >> v1:172.21.15.46:6789/0 conn(0x7f6a540af2f0 legacy=0x7f6a5406d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.273+0000 7f6a59a00640 1 -- 172.21.15.46:0/1339420475 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a540b53e0 con 0x7f6a5415f200 2024-04-26T23:43:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.274+0000 7f6a59a00640 1 --2- 172.21.15.46:0/1339420475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5415f200 0x7f6a540b5ec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009870 tx=0x7f6a4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409c9523cd958fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.275+0000 7f6a53e00640 1 -- 172.21.15.46:0/1339420475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48037070 con 0x7f6a5415f200 2024-04-26T23:43:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.275+0000 7f6a53e00640 1 -- 172.21.15.46:0/1339420475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48032440 con 0x7f6a5415f200 2024-04-26T23:43:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.275+0000 7f6a53e00640 1 -- 172.21.15.46:0/1339420475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48032810 con 0x7f6a5415f200 2024-04-26T23:43:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.275+0000 7f6a5ae00640 1 -- 172.21.15.46:0/1339420475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5415f200 msgr2=0x7f6a540b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.275+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/1339420475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5415f200 0x7f6a540b5ec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009870 tx=0x7f6a4802a270 comp rx=0 tx=0).stop 2024-04-26T23:43:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.276+0000 7f6a5ae00640 1 -- 172.21.15.46:0/1339420475 shutdown_connections 2024-04-26T23:43:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.276+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/1339420475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5415f200 0x7f6a540b5ec0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.276+0000 7f6a5ae00640 1 -- 172.21.15.46:0/1339420475 >> 172.21.15.46:0/1339420475 conn(0x7f6a540727a0 msgr2=0x7f6a54072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.276+0000 7f6a5ae00640 1 -- 172.21.15.46:0/1339420475 shutdown_connections 2024-04-26T23:43:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.276+0000 7f6a5ae00640 1 -- 172.21.15.46:0/1339420475 wait complete. 2024-04-26T23:43:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.277+0000 7f6a5ae00640 1 Processor -- start 2024-04-26T23:43:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 -- start start 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a540af2f0 0x7f6a54155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a5415f200 0x7f6a541556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a540aeff0 con 0x7f6a5406da70 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a540b59a0 con 0x7f6a5415f200 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.278+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a540af750 con 0x7f6a540af2f0 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47192/0 (socket says 172.21.15.46:47192) 2024-04-26T23:43:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 -- 172.21.15.46:0/2219139849 learned_addr learned my addr 172.21.15.46:0/2219139849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 -- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a540af2f0 msgr2=0x7f6a54155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a540af2f0 0x7f6a54155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 -- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a5415f200 msgr2=0x7f6a541556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a5415f200 0x7f6a541556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 -- 172.21.15.46:0/2219139849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a54161ce0 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.279+0000 7f6a59a00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6a48000c00 tx=0x7f6a480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.280+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48037040 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.280+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48042070 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.280+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48030bd0 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.280+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a540ce510 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.280+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a540cead0 con 0x7f6a5406da70 2024-04-26T23:43:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.281+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a54069ac0 con 0x7f6a5406da70 2024-04-26T23:43:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.282+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a48047460 con 0x7f6a5406da70 2024-04-26T23:43:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.284+0000 7f6a51600640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 0x7f6a3406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.284+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6a480ac890 con 0x7f6a5406da70 2024-04-26T23:43:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.284+0000 7f6a59000640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 0x7f6a3406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.286+0000 7f6a59000640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 0x7f6a3406ee00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a540ce370 tx=0x7f6a400079c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.288+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a48045360 con 0x7f6a5406da70 2024-04-26T23:43:56.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.535+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a54162a30 con 0x7f6a5406da70 2024-04-26T23:43:56.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.536+0000 7f6a51600640 1 -- 172.21.15.46:0/2219139849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7f6a4807aba0 con 0x7f6a5406da70 2024-04-26T23:43:56.541 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:43:56.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 msgr2=0x7f6a3406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 0x7f6a3406ee00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a540ce370 tx=0x7f6a400079c0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 msgr2=0x7f6a5414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6a48000c00 tx=0x7f6a480046c0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 shutdown_connections 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a3406c940 0x7f6a3406ee00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a5415f200 0x7f6a541556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a540af2f0 0x7f6a54155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 --2- 172.21.15.46:0/2219139849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5406da70 0x7f6a5414a140 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 >> 172.21.15.46:0/2219139849 conn(0x7f6a540727a0 msgr2=0x7f6a540b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 shutdown_connections 2024-04-26T23:43:56.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56.539+0000 7f6a5ae00640 1 -- 172.21.15.46:0/2219139849 wait complete. 2024-04-26T23:43:56.563 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:43:56.564 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:43:56.565 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:56.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:56.565 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:43:56.619 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:43:56.619 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:43:56.619 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:43:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:43:56.621 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:43:56.622 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:43:56.623 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:43:56.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:56.624 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:43:56.706 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:43:56.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:56.710 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:43:56.766 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:43:56.766 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:43:56.766 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:43:56.766 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:43:56.766 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:43:56.766 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:43:56.767 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:43:56.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:43:56.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:43:56.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:43:56.767 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:43:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:43:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:56.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:56.992 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:43:57.044 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:43:57.044 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:43:57.045 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:57.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:57.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:43:57.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:43:57.134 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:57.189 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:43:57.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:43:57.189 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:57.245 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:43:57.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:57.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:57.374 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:57.374 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:57.374 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:57.396 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:57.396 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:57.396 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:57.397 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:43:57.397 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:43:57.425 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:43:57.425 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:43:57.425 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:43:57.426 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:43:57.426 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:43:57.426 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:43:57.426 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:43:57.426 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:43:57.429 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:43:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:43:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:43:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:43:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:43:57.434 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:43:57.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:57.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:43:57.517 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:43:57.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:43:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:57.668 INFO:teuthology.orchestra.run.smithi046.stdout:client7332 2024-04-26T23:43:57.668 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:57.668 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:57.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:57.721 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-26T23:43:57.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:43:57.786 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-26T23:43:57.847 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:43:57.996 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-26T23:43:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:43:58.010 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:58.010 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:58.111 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:43:58.111 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:43:58.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:43:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.376+0000 7fdd5ca00640 1 Processor -- start 2024-04-26T23:43:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.376+0000 7fdd5ca00640 1 -- start start 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.378+0000 7fdd5ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd580720e0 0x7fdd580cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.378+0000 7fdd5ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd58073ad0 con 0x7fdd58160910 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.378+0000 7fdd5ca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd58146a90 con 0x7fdd581466b0 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.378+0000 7fdd5ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdd58073d70 con 0x7fdd580720e0 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd580720e0 0x7fdd580cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd580720e0 0x7fdd580cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47208/0 (socket says 172.21.15.46:47208) 2024-04-26T23:43:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 -- 172.21.15.46:0/437706799 learned_addr learned my addr 172.21.15.46:0/437706799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 -- 172.21.15.46:0/437706799 >> v1:172.21.15.142:6789/0 conn(0x7fdd581466b0 legacy=0x7fdd5806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 -- 172.21.15.46:0/437706799 >> v1:172.21.15.46:6789/0 conn(0x7fdd58160910 legacy=0x7fdd580cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.379+0000 7fdd55e00640 1 -- 172.21.15.46:0/437706799 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd58074050 con 0x7fdd580720e0 2024-04-26T23:43:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.380+0000 7fdd55e00640 1 --2- 172.21.15.46:0/437706799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580720e0 0x7fdd580cd860 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009e80 tx=0x7fdd4002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beba5f9880cd7c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.380+0000 7fdd55400640 1 -- 172.21.15.46:0/437706799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40036070 con 0x7fdd580720e0 2024-04-26T23:43:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd55400640 1 -- 172.21.15.46:0/437706799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd40031600 con 0x7fdd580720e0 2024-04-26T23:43:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd55400640 1 -- 172.21.15.46:0/437706799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400319d0 con 0x7fdd580720e0 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 -- 172.21.15.46:0/437706799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580720e0 msgr2=0x7fdd580cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/437706799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580720e0 0x7fdd580cd860 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009e80 tx=0x7fdd4002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 -- 172.21.15.46:0/437706799 shutdown_connections 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/437706799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580720e0 0x7fdd580cd860 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 -- 172.21.15.46:0/437706799 >> 172.21.15.46:0/437706799 conn(0x7fdd580b26d0 msgr2=0x7fdd580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 -- 172.21.15.46:0/437706799 shutdown_connections 2024-04-26T23:43:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.381+0000 7fdd5ca00640 1 -- 172.21.15.46:0/437706799 wait complete. 2024-04-26T23:43:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.383+0000 7fdd5ca00640 1 Processor -- start 2024-04-26T23:43:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.383+0000 7fdd5ca00640 1 -- start start 2024-04-26T23:43:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580720e0 0x7fdd580aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581466b0 0x7fdd580b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580cdda0 con 0x7fdd58160910 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580cb470 con 0x7fdd580720e0 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.384+0000 7fdd5ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdd580cc0a0 con 0x7fdd581466b0 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47218/0 (socket says 172.21.15.46:47218) 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 -- 172.21.15.46:0/4187646102 learned_addr learned my addr 172.21.15.46:0/4187646102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd55e00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581466b0 0x7fdd580b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd56800640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580720e0 0x7fdd580aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 -- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581466b0 msgr2=0x7fdd580b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581466b0 0x7fdd580b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 -- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580720e0 msgr2=0x7fdd580aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580720e0 0x7fdd580aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.385+0000 7fdd57200640 1 -- 172.21.15.46:0/4187646102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5815fde0 con 0x7fdd58160910 2024-04-26T23:43:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd57200640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c390 tx=0x7fdd5000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50017070 con 0x7fdd58160910 2024-04-26T23:43:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd500052d0 con 0x7fdd58160910 2024-04-26T23:43:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003810 con 0x7fdd58160910 2024-04-26T23:43:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd580c8d60 con 0x7fdd58160910 2024-04-26T23:43:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.386+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd580c9320 con 0x7fdd58160910 2024-04-26T23:43:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.387+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580693b0 con 0x7fdd58160910 2024-04-26T23:43:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.388+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd5001a070 con 0x7fdd58160910 2024-04-26T23:43:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.389+0000 7fdd4ea00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 0x7fdd3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.389+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fdd5008afc0 con 0x7fdd58160910 2024-04-26T23:43:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.390+0000 7fdd56800640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 0x7fdd3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.391+0000 7fdd56800640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 0x7fdd3406ede0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd48004ea0 tx=0x7fdd48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.393+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd50059250 con 0x7fdd58160910 2024-04-26T23:43:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.598+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd580c7180 con 0x7fdd58160910 2024-04-26T23:43:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.600+0000 7fdd4ea00640 1 -- 172.21.15.46:0/4187646102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16909 (secure 0 0 0) 0x7fdd50058b50 con 0x7fdd58160910 2024-04-26T23:43:58.605 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:43:58.605 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:43:50.237954+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 msgr2=0x7fdd3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 0x7fdd3406ede0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd48004ea0 tx=0x7fdd48009040 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 msgr2=0x7fdd581f81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c390 tx=0x7fdd5000c850 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 shutdown_connections 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3406c920 0x7fdd3406ede0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58160910 0x7fdd581f81b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581466b0 0x7fdd580b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 --2- 172.21.15.46:0/4187646102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580720e0 0x7fdd580aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 >> 172.21.15.46:0/4187646102 conn(0x7fdd580b26d0 msgr2=0x7fdd580726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 shutdown_connections 2024-04-26T23:43:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.603+0000 7fdd5ca00640 1 -- 172.21.15.46:0/4187646102 wait complete. 2024-04-26T23:43:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:43:46.543830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:43:47.711290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:43:58.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:43:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.830+0000 7f6fbe200640 1 Processor -- start 2024-04-26T23:43:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.830+0000 7f6fbe200640 1 -- start start 2024-04-26T23:43:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbe200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fb80cdd00 0x7f6fb80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbe200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb80ce7d0 con 0x7f6fb815cec0 2024-04-26T23:43:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbe200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb80ceab0 con 0x7f6fb8073b30 2024-04-26T23:43:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbe200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb80ced90 con 0x7f6fb80cdd00 2024-04-26T23:43:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fb815cec0 0x7f6fb80cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47448/0 (socket says 172.21.15.46:47448) 2024-04-26T23:43:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.832+0000 7f6fbce00640 1 -- 172.21.15.46:0/2517886967 learned_addr learned my addr 172.21.15.46:0/2517886967 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 460646558 0 0) 0x7f6fb80ce7d0 con 0x7f6fb815cec0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa0003520 con 0x7f6fb815cec0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/2517886967 >> v2:172.21.15.142:3300/0 conn(0x7f6fb80cdd00 0x7f6fb80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182964621 0 0) 0x7f6fa0003520 con 0x7f6fb815cec0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 >> v2:172.21.15.142:3300/0 conn(0x7f6fb80cdd00 msgr2=0x7f6fb80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 --2- 172.21.15.46:0/2517886967 >> v2:172.21.15.142:3300/0 conn(0x7f6fb80cdd00 0x7f6fb80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 >> v1:172.21.15.142:6789/0 conn(0x7f6fb8073b30 legacy=0x7f6fb80b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb80cf070 con 0x7f6fb815cec0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6fac001670 con 0x7f6fb815cec0 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 >> v1:172.21.15.46:6789/0 conn(0x7f6fb815cec0 legacy=0x7f6fb80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa0003a00 0x7f6fa0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa0005210 0x7f6fa000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb80ced90 con 0x7f6fa0004800 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa000c160 con 0x7f6fa0005210 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.833+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa000c360 con 0x7f6fa0003a00 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fbce00640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa0003a00 0x7f6fa0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fbd800640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa0005210 0x7f6fa000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 -- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa0003a00 msgr2=0x7f6fa0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa0003a00 0x7f6fa0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 -- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa0005210 msgr2=0x7f6fa000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa0005210 0x7f6fa000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fb7e00640 1 -- 172.21.15.46:0/2517886967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa0003520 con 0x7f6fa0004800 2024-04-26T23:43:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fbd800640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa0005210 0x7f6fa000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.834+0000 7f6fbd800640 1 -- 172.21.15.46:0/2517886967 reap_dead start 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.836+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fa0004c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ba30 tx=0x7f6fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad6bec145b2a337 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.836+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4030070 con 0x7f6fa0004800 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.836+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa402ade0 con 0x7f6fa0004800 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.836+0000 7f6fb7400640 1 -- 172.21.15.46:0/2517886967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400b600 con 0x7f6fa0004800 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 -- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 msgr2=0x7f6fa0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fa0004c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ba30 tx=0x7f6fa0005830 comp rx=0 tx=0).stop 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 -- 172.21.15.46:0/2517886967 shutdown_connections 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 --2- 172.21.15.46:0/2517886967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fa0004c80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 -- 172.21.15.46:0/2517886967 >> 172.21.15.46:0/2517886967 conn(0x7f6fb80b27b0 msgr2=0x7f6fb80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 -- 172.21.15.46:0/2517886967 shutdown_connections 2024-04-26T23:43:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.837+0000 7f6fbe200640 1 -- 172.21.15.46:0/2517886967 wait complete. 2024-04-26T23:43:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.839+0000 7f6fbe200640 1 Processor -- start 2024-04-26T23:43:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.839+0000 7f6fbe200640 1 -- start start 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 0x7f6fb815f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb815d6c0 0x7f6fb815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8073f70 con 0x7f6fa0004800 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb815d2a0 con 0x7f6fb815f4f0 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbe200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb80735f0 con 0x7f6fb815d6c0 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47234/0 (socket says 172.21.15.46:47234) 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.840+0000 7f6fbce00640 1 -- 172.21.15.46:0/3616134904 learned_addr learned my addr 172.21.15.46:0/3616134904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 0x7f6fb815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbd800640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb815d6c0 0x7f6fb815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 -- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb815d6c0 msgr2=0x7f6fb815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb815d6c0 0x7f6fb815db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 -- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 msgr2=0x7f6fb815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 0x7f6fb815f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 -- 172.21.15.46:0/3616134904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb815e3f0 con 0x7f6fa0004800 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 0x7f6fb815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbce00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e700 tx=0x7f6fac00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ee10 con 0x7f6fa0004800 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac018070 con 0x7f6fa0004800 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb815e6d0 con 0x7f6fa0004800 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00caf0 con 0x7f6fa0004800 2024-04-26T23:43:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.841+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb815ec90 con 0x7f6fa0004800 2024-04-26T23:43:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.843+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb80693b0 con 0x7f6fa0004800 2024-04-26T23:43:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.844+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fac00cc90 con 0x7f6fa0004800 2024-04-26T23:43:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.845+0000 7f6fb4c00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 0x7f6f9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.845+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6fac08b920 con 0x7f6fa0004800 2024-04-26T23:43:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.845+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 0x7f6f9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.846+0000 7f6fb7e00640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 0x7f6f9806ede0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ba00 tx=0x7f6fa4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:58.850+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fac059b20 con 0x7f6fa0004800 2024-04-26T23:43:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.055+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6fb80764d0 con 0x7f6fa0004800 2024-04-26T23:43:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.056+0000 7f6fb4c00640 1 -- 172.21.15.46:0/3616134904 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f6fac059420 con 0x7f6fa0004800 2024-04-26T23:43:59.060 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.058+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 msgr2=0x7f6f9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.058+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 0x7f6f9806ede0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ba00 tx=0x7f6fa4030000 comp rx=0 tx=0).stop 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.058+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 msgr2=0x7f6fb81502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.058+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e700 tx=0x7f6fac00ebc0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 shutdown_connections 2024-04-26T23:43:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9806c920 0x7f6f9806ede0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb815d6c0 0x7f6fb815db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb815f4f0 0x7f6fb815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 --2- 172.21.15.46:0/3616134904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fa0004800 0x7f6fb81502b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 >> 172.21.15.46:0/3616134904 conn(0x7f6fb80b27b0 msgr2=0x7f6fb80c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 shutdown_connections 2024-04-26T23:43:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.059+0000 7f6fbe200640 1 -- 172.21.15.46:0/3616134904 wait complete. 2024-04-26T23:43:59.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:43:59.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.276+0000 7fde5fe00640 1 Processor -- start 2024-04-26T23:43:59.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.276+0000 7fde5fe00640 1 -- start start 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.277+0000 7fde5fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde6015cec0 0x7fde600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde600ce7d0 con 0x7fde60073d00 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde600ceab0 con 0x7fde600cdb30 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fde600ced90 con 0x7fde6015cec0 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde6015cec0 0x7fde600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fde60073d00 0x7fde600b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47450/0 (socket says 172.21.15.46:47450) 2024-04-26T23:43:59.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5f400640 1 -- 172.21.15.46:0/3485627666 learned_addr learned my addr 172.21.15.46:0/3485627666 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:59.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5ea00640 1 -- 172.21.15.46:0/3485627666 >> v1:172.21.15.142:6789/0 conn(0x7fde600cdb30 legacy=0x7fde600735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:59.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5ea00640 1 -- 172.21.15.46:0/3485627666 >> v1:172.21.15.46:6789/0 conn(0x7fde60073d00 legacy=0x7fde600b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.279+0000 7fde5ea00640 1 -- 172.21.15.46:0/3485627666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde600cf070 con 0x7fde6015cec0 2024-04-26T23:43:59.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.278+0000 7fde5e000640 1 -- 172.21.15.46:0/3485627666 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655047348 0 0) 0x7fde600ce7d0 con 0x7fde60073d00 2024-04-26T23:43:59.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5ea00640 1 --2- 172.21.15.46:0/3485627666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde600cd5f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fde500097a0 tx=0x7fde5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d9040c6bcde40c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5e000640 1 -- 172.21.15.46:0/3485627666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50037070 con 0x7fde6015cec0 2024-04-26T23:43:59.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5e000640 1 -- 172.21.15.46:0/3485627666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde50032440 con 0x7fde6015cec0 2024-04-26T23:43:59.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5e000640 1 -- 172.21.15.46:0/3485627666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50032810 con 0x7fde6015cec0 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5fe00640 1 -- 172.21.15.46:0/3485627666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 msgr2=0x7fde600cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.280+0000 7fde5fe00640 1 --2- 172.21.15.46:0/3485627666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde600cd5f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fde500097a0 tx=0x7fde5002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.281+0000 7fde5fe00640 1 -- 172.21.15.46:0/3485627666 shutdown_connections 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.281+0000 7fde5fe00640 1 --2- 172.21.15.46:0/3485627666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde600cd5f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.281+0000 7fde5fe00640 1 -- 172.21.15.46:0/3485627666 >> 172.21.15.46:0/3485627666 conn(0x7fde600b27b0 msgr2=0x7fde600b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.281+0000 7fde5fe00640 1 -- 172.21.15.46:0/3485627666 shutdown_connections 2024-04-26T23:43:59.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.281+0000 7fde5fe00640 1 -- 172.21.15.46:0/3485627666 wait complete. 2024-04-26T23:43:59.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.282+0000 7fde5fe00640 1 Processor -- start 2024-04-26T23:43:59.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 -- start start 2024-04-26T23:43:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 0x7fde6015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600cdb30 0x7fde6015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde600b4250 con 0x7fde6015cec0 2024-04-26T23:43:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde6015d2a0 con 0x7fde60073d00 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.283+0000 7fde5fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde600b43d0 con 0x7fde600cdb30 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47260/0 (socket says 172.21.15.46:47260) 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 -- 172.21.15.46:0/231704441 learned_addr learned my addr 172.21.15.46:0/231704441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5ea00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 0x7fde6015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde57e00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600cdb30 0x7fde6015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 -- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600cdb30 msgr2=0x7fde6015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600cdb30 0x7fde6015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 -- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 msgr2=0x7fde6015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 0x7fde6015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5f400640 1 -- 172.21.15.46:0/231704441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde601545f0 con 0x7fde6015cec0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.284+0000 7fde5ea00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 0x7fde6015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde5f400640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c800 tx=0x7fde5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58018070 con 0x7fde6015cec0 2024-04-26T23:43:59.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde58004d80 con 0x7fde6015cec0 2024-04-26T23:43:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5800a880 con 0x7fde6015cec0 2024-04-26T23:43:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6015db90 con 0x7fde6015cec0 2024-04-26T23:43:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.285+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6015e150 con 0x7fde6015cec0 2024-04-26T23:43:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.286+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde600693b0 con 0x7fde6015cec0 2024-04-26T23:43:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.288+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde5800aa20 con 0x7fde6015cec0 2024-04-26T23:43:59.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.289+0000 7fde56a00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.290+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fde5808b210 con 0x7fde6015cec0 2024-04-26T23:43:59.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.290+0000 7fde5ea00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.291+0000 7fde5ea00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 0x7fde3c06edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fde50034000 tx=0x7fde5002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.294+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde58059600 con 0x7fde6015cec0 2024-04-26T23:43:59.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.505+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde600cf8f0 con 0x7fde6015cec0 2024-04-26T23:43:59.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.506+0000 7fde56a00640 1 -- 172.21.15.46:0/231704441 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fde58058f00 con 0x7fde6015cec0 2024-04-26T23:43:59.511 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:43:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.509+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 msgr2=0x7fde3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.509+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 0x7fde3c06edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fde50034000 tx=0x7fde5002e000 comp rx=0 tx=0).stop 2024-04-26T23:43:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.509+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 msgr2=0x7fde601540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.509+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c800 tx=0x7fde5800ccc0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 shutdown_connections 2024-04-26T23:43:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6015cec0 0x7fde601540b0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600cdb30 0x7fde6015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 --2- 172.21.15.46:0/231704441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60073d00 0x7fde6015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 >> 172.21.15.46:0/231704441 conn(0x7fde600b27b0 msgr2=0x7fde60073420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 shutdown_connections 2024-04-26T23:43:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.510+0000 7fde5fe00640 1 -- 172.21.15.46:0/231704441 wait complete. 2024-04-26T23:43:59.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:43:59.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.738+0000 7f900be00640 1 Processor -- start 2024-04-26T23:43:59.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.738+0000 7f900be00640 1 -- start start 2024-04-26T23:43:59.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.739+0000 7f900be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f900c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.739+0000 7f900be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c0d0630 con 0x7f900c072360 2024-04-26T23:43:59.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f900be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c0c64f0 con 0x7f900c14c9e0 2024-04-26T23:43:59.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f900be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f900c0d0910 con 0x7f900c0c75a0 2024-04-26T23:43:59.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f900b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f900c072360 0x7f900c166fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47460/0 (socket says 172.21.15.46:47460) 2024-04-26T23:43:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f900b400640 1 -- 172.21.15.46:0/3599967741 learned_addr learned my addr 172.21.15.46:0/3599967741 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:43:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f9003e00640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f900c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.740+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941481134 0 0) 0x7f900c0d0630 con 0x7f900c072360 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ff0003520 con 0x7f900c072360 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1054985272 0 0) 0x7f8ff0003520 con 0x7f900c072360 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 msgr2=0x7f900c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f900c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 >> v1:172.21.15.142:6789/0 conn(0x7f900c14c9e0 legacy=0x7f900c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c14d030 con 0x7f900c072360 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f90040019f0 con 0x7f900c072360 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 >> v1:172.21.15.46:6789/0 conn(0x7f900c072360 legacy=0x7f900c166fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f8ff00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0003f20 0x7f8ff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff0004d40 0x7f8ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000baa0 con 0x7f8ff0004d40 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000bc50 con 0x7f8ff0003f20 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.741+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff000be50 con 0x7f900c0c75a0 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff0004d40 0x7f8ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900aa00640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f8ff00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f9003e00640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0003f20 0x7f8ff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 -- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 msgr2=0x7f8ff00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c0c75a0 0x7f8ff00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 -- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0003f20 msgr2=0x7f8ff0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0003f20 0x7f8ff0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f900b400640 1 -- 172.21.15.46:0/3599967741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0003520 con 0x7f8ff0004d40 2024-04-26T23:43:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.742+0000 7f9003e00640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0003f20 0x7f8ff0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:43:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.743+0000 7f900b400640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff0004d40 0x7f8ff00051c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f900400a920 tx=0x7f8ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dc7b23d2c27b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.743+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900400a4c0 con 0x7f8ff0004d40 2024-04-26T23:43:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f900402f070 con 0x7f8ff0004d40 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900a000640 1 -- 172.21.15.46:0/3599967741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900402bd20 con 0x7f8ff0004d40 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 -- 172.21.15.46:0/3599967741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff0004d40 msgr2=0x7f8ff00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 --2- 172.21.15.46:0/3599967741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff0004d40 0x7f8ff00051c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f900400a920 tx=0x7f8ff0005830 comp rx=0 tx=0).stop 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900b400640 1 -- 172.21.15.46:0/3599967741 reap_dead start 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 -- 172.21.15.46:0/3599967741 shutdown_connections 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 -- 172.21.15.46:0/3599967741 >> 172.21.15.46:0/3599967741 conn(0x7f900c0b26d0 msgr2=0x7f900c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 -- 172.21.15.46:0/3599967741 shutdown_connections 2024-04-26T23:43:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.744+0000 7f900be00640 1 -- 172.21.15.46:0/3599967741 wait complete. 2024-04-26T23:43:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.746+0000 7f900be00640 1 Processor -- start 2024-04-26T23:43:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.746+0000 7f900be00640 1 -- start start 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c15fd40 0x7f900c1fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c0ca530 0x7f900c0c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0727f0 con 0x7f900c0c8a70 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0c6860 con 0x7f900c0ca530 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.747+0000 7f900be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f900c072970 con 0x7f900c15fd40 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47274/0 (socket says 172.21.15.46:47274) 2024-04-26T23:43:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 -- 172.21.15.46:0/1113196233 learned_addr learned my addr 172.21.15.46:0/1113196233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900aa00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c15fd40 0x7f900c1fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f9003e00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c0ca530 0x7f900c0c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 -- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c15fd40 msgr2=0x7f900c1fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c15fd40 0x7f900c1fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 -- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c0ca530 msgr2=0x7f900c0c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c0ca530 0x7f900c0c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.748+0000 7f900b400640 1 -- 172.21.15.46:0/1113196233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c140890 con 0x7f900c0c8a70 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f900b400640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f900400aa10 tx=0x7f900402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900400a4c0 con 0x7f900c0c8a70 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9004034470 con 0x7f900c0c8a70 2024-04-26T23:43:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004034840 con 0x7f900c0c8a70 2024-04-26T23:43:59.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c140b70 con 0x7f900c0c8a70 2024-04-26T23:43:59.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.749+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c0cc730 con 0x7f900c0c8a70 2024-04-26T23:43:59.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.750+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c0693b0 con 0x7f900c0c8a70 2024-04-26T23:43:59.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.751+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90040349e0 con 0x7f900c0c8a70 2024-04-26T23:43:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.752+0000 7f9002a00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 0x7f8fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.752+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f90040abca0 con 0x7f900c0c8a70 2024-04-26T23:43:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.753+0000 7f900aa00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 0x7f8fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.754+0000 7f900aa00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 0x7f8fe806edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc003640 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:59.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.757+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90040798f0 con 0x7f900c0c8a70 2024-04-26T23:43:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.965+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f900c06f400 con 0x7f900c0c8a70 2024-04-26T23:43:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.966+0000 7f9002a00640 1 -- 172.21.15.46:0/1113196233 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f900407dbf0 con 0x7f900c0c8a70 2024-04-26T23:43:59.970 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:43:59.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 msgr2=0x7f8fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 0x7f8fe806edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc003640 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 msgr2=0x7f900c1f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f900400aa10 tx=0x7f900402d070 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 shutdown_connections 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fe806c8f0 0x7f8fe806edb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f900c0c8a70 0x7f900c1f8f60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c0ca530 0x7f900c0c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 --2- 172.21.15.46:0/1113196233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f900c15fd40 0x7f900c1fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.968+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 >> 172.21.15.46:0/1113196233 conn(0x7f900c0b26d0 msgr2=0x7f900c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.969+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 shutdown_connections 2024-04-26T23:43:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:43:59.969+0000 7f900be00640 1 -- 172.21.15.46:0/1113196233 wait complete. 2024-04-26T23:43:59.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:44:00.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.198+0000 7fce33800640 1 Processor -- start 2024-04-26T23:44:00.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.198+0000 7fce33800640 1 -- start start 2024-04-26T23:44:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce33800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce2c160bd0 0x7fce2c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce33800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c161940 con 0x7fce2c06e510 2024-04-26T23:44:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce33800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c071f80 con 0x7fce2c0732c0 2024-04-26T23:44:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce33800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c072260 con 0x7fce2c160bd0 2024-04-26T23:44:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce30e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce2c06e510 0x7fce2c06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47470/0 (socket says 172.21.15.46:47470) 2024-04-26T23:44:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.200+0000 7fce30e00640 1 -- 172.21.15.46:0/1215869748 learned_addr learned my addr 172.21.15.46:0/1215869748 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2be00640 1 --2- 172.21.15.46:0/1215869748 >> v2:172.21.15.142:3300/0 conn(0x7fce2c160bd0 0x7fce2c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969628260 0 0) 0x7fce2c161940 con 0x7fce2c06e510 2024-04-26T23:44:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce14003520 con 0x7fce2c06e510 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1951125012 0 0) 0x7fce14003520 con 0x7fce2c06e510 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 >> v2:172.21.15.142:3300/0 conn(0x7fce2c160bd0 msgr2=0x7fce2c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 --2- 172.21.15.46:0/1215869748 >> v2:172.21.15.142:3300/0 conn(0x7fce2c160bd0 0x7fce2c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 >> v1:172.21.15.142:6789/0 conn(0x7fce2c0732c0 legacy=0x7fce2c073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c072540 con 0x7fce2c06e510 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.201+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fce20001670 con 0x7fce2c06e510 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 >> v1:172.21.15.46:6789/0 conn(0x7fce2c06e510 legacy=0x7fce2c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce140039e0 0x7fce14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce14004800 0x7fce14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c072260 con 0x7fce140051c0 2024-04-26T23:44:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce1400c0f0 con 0x7fce140039e0 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce1400c2f0 con 0x7fce14004800 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 -- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce14004800 msgr2=0x7fce14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce14004800 0x7fce14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 -- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce140039e0 msgr2=0x7fce14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce140039e0 0x7fce14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.202+0000 7fce31800640 1 -- 172.21.15.46:0/1215869748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c072540 con 0x7fce140051c0 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.203+0000 7fce31800640 1 -- 172.21.15.46:0/1215869748 reap_dead start 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.203+0000 7fce31800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce1400b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fce24001e20 tx=0x7fce14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20acb743d3b528d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.204+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24001a70 con 0x7fce140051c0 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.204+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2402ae40 con 0x7fce140051c0 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.204+0000 7fce2b400640 1 -- 172.21.15.46:0/1215869748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24003bb0 con 0x7fce140051c0 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.204+0000 7fce33800640 1 -- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 msgr2=0x7fce1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.204+0000 7fce33800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce1400b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fce24001e20 tx=0x7fce14005830 comp rx=0 tx=0).stop 2024-04-26T23:44:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.205+0000 7fce33800640 1 -- 172.21.15.46:0/1215869748 shutdown_connections 2024-04-26T23:44:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.205+0000 7fce33800640 1 --2- 172.21.15.46:0/1215869748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce1400b9c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.205+0000 7fce33800640 1 -- 172.21.15.46:0/1215869748 >> 172.21.15.46:0/1215869748 conn(0x7fce2c06c530 msgr2=0x7fce2c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.205+0000 7fce33800640 1 -- 172.21.15.46:0/1215869748 shutdown_connections 2024-04-26T23:44:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.205+0000 7fce33800640 1 -- 172.21.15.46:0/1215869748 wait complete. 2024-04-26T23:44:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.206+0000 7fce33800640 1 Processor -- start 2024-04-26T23:44:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.207+0000 7fce33800640 1 -- start start 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0b1820 0x7fce2c160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 0x7fce2c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c072ec0 con 0x7fce140051c0 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c072d40 con 0x7fce2c0b1820 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce33800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c146b10 con 0x7fce2c15f4d0 2024-04-26T23:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47304/0 (socket says 172.21.15.46:47304) 2024-04-26T23:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 -- 172.21.15.46:0/1947138989 learned_addr learned my addr 172.21.15.46:0/1947138989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce31800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 0x7fce2c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce2be00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0b1820 0x7fce2c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 -- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 msgr2=0x7fce2c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 0x7fce2c15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 -- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0b1820 msgr2=0x7fce2c160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0b1820 0x7fce2c160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.208+0000 7fce30e00640 1 -- 172.21.15.46:0/1947138989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c0b1d70 con 0x7fce140051c0 2024-04-26T23:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce31800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 0x7fce2c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce30e00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c800 tx=0x7fce2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20018070 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce200074e0 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20004a00 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c0b2050 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.209+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c0b2610 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.210+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c069530 con 0x7fce140051c0 2024-04-26T23:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.211+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce20004ba0 con 0x7fce140051c0 2024-04-26T23:44:00.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.212+0000 7fce28c00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.212+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fce2008b390 con 0x7fce140051c0 2024-04-26T23:44:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.213+0000 7fce2be00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.214+0000 7fce2be00640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fce18006a90 tx=0x7fce18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.217+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce20059620 con 0x7fce140051c0 2024-04-26T23:44:00.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.435+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce2c06fb00 con 0x7fce140051c0 2024-04-26T23:44:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.435+0000 7fce28c00640 1 -- 172.21.15.46:0/1947138989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7fce20058f20 con 0x7fce140051c0 2024-04-26T23:44:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 msgr2=0x7fce0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fce18006a90 tx=0x7fce18008040 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 msgr2=0x7fce2c15bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c800 tx=0x7fce2000ccc0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 shutdown_connections 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce0c06c8f0 0x7fce0c06edb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce2c15f4d0 0x7fce2c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce2c0b1820 0x7fce2c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 --2- 172.21.15.46:0/1947138989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140051c0 0x7fce2c15bc80 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 >> 172.21.15.46:0/1947138989 conn(0x7fce2c06c530 msgr2=0x7fce2c0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 shutdown_connections 2024-04-26T23:44:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.438+0000 7fce33800640 1 -- 172.21.15.46:0/1947138989 wait complete. 2024-04-26T23:44:00.462 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:44:00.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:00.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:00.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:44:00.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:00.614 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:44:00.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.875+0000 7f6fe3200640 1 Processor -- start 2024-04-26T23:44:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.875+0000 7f6fe3200640 1 -- start start 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fe415f4e0 0x7f6fe40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40b4000 0x7f6fe40d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe40d2990 con 0x7f6fe40d1f60 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe40d11f0 con 0x7f6fe415f4e0 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.876+0000 7f6fe3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40d5f20 con 0x7f6fe40b4000 2024-04-26T23:44:00.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47318/0 (socket says 172.21.15.46:47318) 2024-04-26T23:44:00.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 -- 172.21.15.46:0/1254736545 learned_addr learned my addr 172.21.15.46:0/1254736545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:00.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe1400640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40b4000 0x7f6fe40d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe1e00640 1 --2- 172.21.15.46:0/1254736545 >> v2:172.21.15.142:3300/0 conn(0x7f6fe415f4e0 0x7f6fe40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 -- 172.21.15.46:0/1254736545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40b4000 msgr2=0x7f6fe40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40b4000 0x7f6fe40d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 -- 172.21.15.46:0/1254736545 >> v2:172.21.15.142:3300/0 conn(0x7f6fe415f4e0 msgr2=0x7f6fe40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 --2- 172.21.15.46:0/1254736545 >> v2:172.21.15.142:3300/0 conn(0x7f6fe415f4e0 0x7f6fe40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.877+0000 7f6fe2800640 1 -- 172.21.15.46:0/1254736545 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40d0f10 con 0x7f6fe40d1f60 2024-04-26T23:44:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.879+0000 7f6fe2800640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408d8b0 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd16ac10029d5fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.879+0000 7f6fe0a00640 1 -- 172.21.15.46:0/1254736545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc037070 con 0x7f6fe40d1f60 2024-04-26T23:44:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.879+0000 7f6fe0a00640 1 -- 172.21.15.46:0/1254736545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc032440 con 0x7f6fe40d1f60 2024-04-26T23:44:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.879+0000 7f6fe0a00640 1 -- 172.21.15.46:0/1254736545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc032810 con 0x7f6fe40d1f60 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 -- 172.21.15.46:0/1254736545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe40d1f60 msgr2=0x7f6fe40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408d8b0 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 -- 172.21.15.46:0/1254736545 shutdown_connections 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe40d1f60 0x7f6fe40cfcd0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 --2- 172.21.15.46:0/1254736545 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40b4000 0x7f6fe40d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 --2- 172.21.15.46:0/1254736545 >> v2:172.21.15.142:3300/0 conn(0x7f6fe415f4e0 0x7f6fe40d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 -- 172.21.15.46:0/1254736545 >> 172.21.15.46:0/1254736545 conn(0x7f6fe40d5010 msgr2=0x7f6fe40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 -- 172.21.15.46:0/1254736545 shutdown_connections 2024-04-26T23:44:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.880+0000 7f6fe3200640 1 -- 172.21.15.46:0/1254736545 wait complete. 2024-04-26T23:44:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.882+0000 7f6fe3200640 1 Processor -- start 2024-04-26T23:44:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.882+0000 7f6fe3200640 1 -- start start 2024-04-26T23:44:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe40b4000 0x7f6fe40b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 0x7f6fe4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe415f8c0 con 0x7f6fe415f4e0 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40b1d60 con 0x7f6fe40b4000 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40d0210 con 0x7f6fe40d1f60 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47328/0 (socket says 172.21.15.46:47328) 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 -- 172.21.15.46:0/2287865850 learned_addr learned my addr 172.21.15.46:0/2287865850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe1e00640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe40b4000 0x7f6fe40b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe1400640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 0x7f6fe4154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 -- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 msgr2=0x7f6fe4154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 0x7f6fe4154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 -- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe40b4000 msgr2=0x7f6fe40b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe40b4000 0x7f6fe40b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe2800640 1 -- 172.21.15.46:0/2287865850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40cd360 con 0x7f6fe415f4e0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.883+0000 7f6fe1400640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 0x7f6fe4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fe2800640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0090b0 tx=0x7f6fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc037040 con 0x7f6fe415f4e0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc042070 con 0x7f6fe415f4e0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc030920 con 0x7f6fe415f4e0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe40ba710 con 0x7f6fe415f4e0 2024-04-26T23:44:00.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.884+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe414ecb0 con 0x7f6fe415f4e0 2024-04-26T23:44:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.886+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe40693b0 con 0x7f6fe415f4e0 2024-04-26T23:44:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.887+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fdc030ac0 con 0x7f6fe415f4e0 2024-04-26T23:44:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.888+0000 7f6fda000640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.888+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6fdc0ac640 con 0x7f6fe415f4e0 2024-04-26T23:44:00.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.888+0000 7f6fe1e00640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.890+0000 7f6fe1e00640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 0x7f6fc406edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40099c0 tx=0x7f6fd4003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:00.893+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fe40cd360 con 0x7f6fe415f4e0 2024-04-26T23:44:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.137+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fe40d33d0 con 0x7f6fe415f4e0 2024-04-26T23:44:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.138+0000 7f6fda000640 1 -- 172.21.15.46:0/2287865850 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7f6fdc07a280 con 0x7f6fe415f4e0 2024-04-26T23:44:01.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2819521829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2819521829},{"type":"v1","addr":"172.21.15.46:6834","nonce":2819521829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1317531531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1317531531},{"type":"v1","addr":"172.21.15.46:6835","nonce":1317531531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/634149566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":634149566},{"type":"v1","addr":"172.21.15.142:6836","nonce":634149566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:43:51.787417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2319300947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2319300947},{"type":"v1","addr":"172.21.15.142:6837","nonce":2319300947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7164,"qdb_cluster":[7164]},"id":5}]} 2024-04-26T23:44:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2024-04-26T23:44:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 msgr2=0x7f6fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 0x7f6fc406edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40099c0 tx=0x7f6fd4003990 comp rx=0 tx=0).stop 2024-04-26T23:44:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 msgr2=0x7f6fe41545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0090b0 tx=0x7f6fdc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 shutdown_connections 2024-04-26T23:44:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe415f4e0 0x7f6fe41545d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fe40d1f60 0x7f6fe4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 --2- 172.21.15.46:0/2287865850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe40b4000 0x7f6fe40b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 >> 172.21.15.46:0/2287865850 conn(0x7f6fe40d5010 msgr2=0x7f6fe415e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.141+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 shutdown_connections 2024-04-26T23:44:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.142+0000 7f6fe3200640 1 -- 172.21.15.46:0/2287865850 wait complete. 2024-04-26T23:44:01.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-26T23:44:01.415 INFO:teuthology.orchestra.run.smithi142.stdout:{"message":"","return_code":0} 2024-04-26T23:44:01.416 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T23:44:01.416 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:44:01.416 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:01.417 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:44:01.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:01.417 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:44:01.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:01.417 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:44:01.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:01.435 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:44:01.435 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:01.439 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:44:01.439 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:01.441 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:44:01.441 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:01.442 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:44:01.442 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:01.442 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-26T23:44:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.605+0000 7f6bae490f40 1 Processor -- start 2024-04-26T23:44:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.605+0000 7f6bae490f40 1 -- start start 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90002c80 0x5629579e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 0x5629579ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629578bad20 con 0x5629579ec130 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5629578bed60 con 0x5629579ebaa0 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bae490f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5629579ed7d0 con 0x7f6b90002c80 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bac000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bac000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47358/0 (socket says 172.21.15.46:47358) 2024-04-26T23:44:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.606+0000 7f6bac000640 1 -- 172.21.15.46:0/776112231 learned_addr learned my addr 172.21.15.46:0/776112231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 -- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90002c80 msgr2=0x5629579e3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bab600640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90002c80 0x5629579e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6baac00640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 0x5629579ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90002c80 0x5629579e3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 -- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 msgr2=0x5629579ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 0x5629579ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 -- 172.21.15.46:0/776112231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562957945070 con 0x5629579ec130 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6baac00640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 0x5629579ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6bac000640 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x5629578ff380 tx=0x7f6ba4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762fc8b49d86a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.607+0000 7f6ba9800640 1 -- 172.21.15.46:0/776112231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4014410 con 0x5629579ec130 2024-04-26T23:44:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6ba9800640 1 -- 172.21.15.46:0/776112231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba40145b0 con 0x5629579ec130 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6ba9800640 1 -- 172.21.15.46:0/776112231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4014780 con 0x5629579ec130 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 -- 172.21.15.46:0/776112231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 msgr2=0x562957945990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x5629578ff380 tx=0x7f6ba4005d80 comp rx=0 tx=0).stop 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 -- 172.21.15.46:0/776112231 shutdown_connections 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x562957945990 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629579ebaa0 0x5629579ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 --2- 172.21.15.46:0/776112231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90002c80 0x5629579e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 -- 172.21.15.46:0/776112231 >> 172.21.15.46:0/776112231 conn(0x562957918a00 msgr2=0x562957960310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 -- 172.21.15.46:0/776112231 shutdown_connections 2024-04-26T23:44:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.608+0000 7f6bae490f40 1 -- 172.21.15.46:0/776112231 wait complete. 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.609+0000 7f6bae490f40 1 Processor -- start 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.609+0000 7f6bae490f40 1 -- start start 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629579ebaa0 0x5629579ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x5629579ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90002c80 0x5629579e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629579ef400 con 0x5629579ec130 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5629579427f0 con 0x7f6b90002c80 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.610+0000 7f6bae490f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5629579edaf0 con 0x5629579ebaa0 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x5629579ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x5629579ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47360/0 (socket says 172.21.15.46:47360) 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 -- 172.21.15.46:0/3807325297 learned_addr learned my addr 172.21.15.46:0/3807325297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6bab600640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629579ebaa0 0x5629579ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6bac000640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90002c80 0x5629579e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 -- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629579ebaa0 msgr2=0x5629579ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5629579ebaa0 0x5629579ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 -- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90002c80 msgr2=0x5629579e2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90002c80 0x5629579e2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6baac00640 1 -- 172.21.15.46:0/3807325297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562957a7cce0 con 0x5629579ec130 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.611+0000 7f6bac000640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90002c80 0x5629579e2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6baac00640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x5629579ee610 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6b98045f70 tx=0x7f6b98001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6ba3e00640 1 -- 172.21.15.46:0/3807325297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98053460 con 0x5629579ec130 2024-04-26T23:44:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6ba3e00640 1 -- 172.21.15.46:0/3807325297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98053620 con 0x5629579ec130 2024-04-26T23:44:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6ba3e00640 1 -- 172.21.15.46:0/3807325297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98053a10 con 0x5629579ec130 2024-04-26T23:44:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562957a7cfc0 con 0x5629579ec130 2024-04-26T23:44:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.612+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629579e8dd0 con 0x5629579ec130 2024-04-26T23:44:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.614+0000 7f6ba3e00640 1 -- 172.21.15.46:0/3807325297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b98053bb0 con 0x5629579ec130 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.615+0000 7f6ba3e00640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8c06b300 0x7f6b8c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.615+0000 7f6ba3e00640 1 -- 172.21.15.46:0/3807325297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6b980ce130 con 0x5629579ec130 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.615+0000 7f6bae490f40 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x5629579eeb50 0x5629579f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.615+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 --> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) -- 0x5629579e95d0 con 0x5629579eeb50 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.616+0000 7f6bab600640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8c06b300 0x7f6b8c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.616+0000 7f6bac000640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x5629579eeb50 0x5629579f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.616+0000 7f6bac000640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x5629579eeb50 0x5629579f1000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.617+0000 7f6bab600640 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8c06b300 0x7f6b8c06d7c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c001200 tx=0x7f6b9c001de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bac000640 1 -- 172.21.15.46:0/3807325297 <== osd.0 v2:172.21.15.46:6800/3755207586 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v80'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ba4014760 con 0x5629579eeb50 2024-04-26T23:44:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x5629579eeb50 msgr2=0x5629579f1000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x5629579eeb50 0x5629579f1000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8c06b300 msgr2=0x7f6b8c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b8c06b300 0x7f6b8c06d7c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c001200 tx=0x7f6b9c001de0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 msgr2=0x5629579ee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 --2- 172.21.15.46:0/3807325297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629579ec130 0x5629579ee610 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6b98045f70 tx=0x7f6b98001230 comp rx=0 tx=0).stop 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bac000640 1 -- 172.21.15.46:0/3807325297 reap_dead start 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 shutdown_connections 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 >> 172.21.15.46:0/3807325297 conn(0x562957918a00 msgr2=0x562957918e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 shutdown_connections 2024-04-26T23:44:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.619+0000 7f6bae490f40 1 -- 172.21.15.46:0/3807325297 wait complete. 2024-04-26T23:44:01.628 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-26T23:44:01.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-26T23:44:01.629 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-26T23:44:01.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-26T23:44:01.629 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-26T23:44:01.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-26T23:44:01.630 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-26T23:44:01.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-26T23:44:01.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.802+0000 7f8051800640 1 Processor -- start 2024-04-26T23:44:01.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.802+0000 7f8051800640 1 -- start start 2024-04-26T23:44:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7fde12200640 1 Processor -- start 2024-04-26T23:44:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f11fb600640 1 Processor -- start 2024-04-26T23:44:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7fde12200640 1 -- start start 2024-04-26T23:44:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f11fb600640 1 -- start start 2024-04-26T23:44:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f8051800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f804c1606d0 0x7f804c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f8051800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0cf260 con 0x7f804c0b1b80 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f8051800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0d17a0 con 0x7f804c0cdcf0 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.804+0000 7f8051800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f804c0cf540 con 0x7f804c1606d0 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f804b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f804c0b1b80 0x7f804c0cfcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47486/0 (socket says 172.21.15.46:47486) 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f804b400640 1 -- 172.21.15.46:0/3192882643 learned_addr learned my addr 172.21.15.46:0/3192882643 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f804be00640 1 --2- 172.21.15.46:0/3192882643 >> v2:172.21.15.142:3300/0 conn(0x7f804c1606d0 0x7f804c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282589612 0 0) 0x7f804c0cf260 con 0x7f804c0b1b80 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8030003520 con 0x7f804c0b1b80 2024-04-26T23:44:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7f11fb600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f4163720 0x7f11f40c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.805+0000 7fde12200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c0d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11fb600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11f406f590 0x7f11f40c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11fb600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f4167350 con 0x7f11f406fae0 2024-04-26T23:44:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11fb600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f11f4163b00 con 0x7f11f4163720 2024-04-26T23:44:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11fb600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f11f40cd590 con 0x7f11f406f590 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3930123929 0 0) 0x7f8030003520 con 0x7f804c0b1b80 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 >> v2:172.21.15.142:3300/0 conn(0x7f804c1606d0 msgr2=0x7f804c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 --2- 172.21.15.46:0/3192882643 >> v2:172.21.15.142:3300/0 conn(0x7f804c1606d0 0x7f804c0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 >> v1:172.21.15.142:6789/0 conn(0x7f804c0cdcf0 legacy=0x7f804c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde12200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde12200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 0x7fde0c162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde12200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0d11b0 con 0x7fde0c0d1fd0 2024-04-26T23:44:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde12200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0d2e80 con 0x7fde0c0d0d20 2024-04-26T23:44:01.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde12200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c0d3140 con 0x7fde0c15f3b0 2024-04-26T23:44:01.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0d1a80 con 0x7f804c0b1b80 2024-04-26T23:44:01.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde0aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde0aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47366/0 (socket says 172.21.15.46:47366) 2024-04-26T23:44:01.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11f8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f4163720 0x7f11f40c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7fde0aa00640 1 -- 172.21.15.46:0/3793366124 learned_addr learned my addr 172.21.15.46:0/3793366124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8038001670 con 0x7f804c0b1b80 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11f8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f4163720 0x7f11f40c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47368/0 (socket says 172.21.15.46:47368) 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f11f8c00640 1 -- 172.21.15.46:0/2190220326 learned_addr learned my addr 172.21.15.46:0/2190220326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 >> v1:172.21.15.46:6789/0 conn(0x7f804c0b1b80 legacy=0x7f804c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8030003a00 0x7f8030003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 0x7f8030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80300052e0 0x7f803000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0cf540 con 0x7f8030004800 2024-04-26T23:44:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f803000c1e0 con 0x7f8030003a00 2024-04-26T23:44:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.806+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f803000c3e0 con 0x7f80300052e0 2024-04-26T23:44:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 0x7f8030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f11f9600640 1 --2- 172.21.15.46:0/2190220326 >> v2:172.21.15.142:3300/0 conn(0x7f11f406f590 0x7f11f40c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f8050e00640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80300052e0 0x7f803000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0b400640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c0d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0aa00640 1 -- 172.21.15.46:0/3793366124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 msgr2=0x7fde0c0d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0aa00640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c0d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f11f8c00640 1 -- 172.21.15.46:0/2190220326 >> v2:172.21.15.142:3300/0 conn(0x7f11f406f590 msgr2=0x7f11f40c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0aa00640 1 -- 172.21.15.46:0/3793366124 >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 msgr2=0x7fde0c162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0be00640 1 --2- 172.21.15.46:0/3793366124 >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 0x7fde0c162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0aa00640 1 --2- 172.21.15.46:0/3793366124 >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 0x7fde0c162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0aa00640 1 -- 172.21.15.46:0/3793366124 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c0d3420 con 0x7fde0c0d1fd0 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7fde0be00640 1 --2- 172.21.15.46:0/3793366124 >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 0x7fde0c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f11f8c00640 1 --2- 172.21.15.46:0/2190220326 >> v2:172.21.15.142:3300/0 conn(0x7f11f406f590 0x7f11f40c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f11f8c00640 1 -- 172.21.15.46:0/2190220326 >> v1:172.21.15.142:6789/0 conn(0x7f11f406fae0 legacy=0x7f11f40ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f11f8c00640 1 -- 172.21.15.46:0/2190220326 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41675f0 con 0x7f11f4163720 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 -- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80300052e0 msgr2=0x7f803000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80300052e0 0x7f803000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 -- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8030003a00 msgr2=0x7f8030003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8030003a00 0x7f8030003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f804b400640 1 -- 172.21.15.46:0/3192882643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8030003520 con 0x7f8030004800 2024-04-26T23:44:01.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.807+0000 7f8050e00640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80300052e0 0x7f803000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:44:01.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.808+0000 7f8050e00640 1 -- 172.21.15.46:0/3192882643 reap_dead start 2024-04-26T23:44:01.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.808+0000 7f9db6000640 1 Processor -- start 2024-04-26T23:44:01.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.808+0000 7f9db6000640 1 -- start start 2024-04-26T23:44:01.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.808+0000 7f11f8c00640 1 --2- 172.21.15.46:0/2190220326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f4163720 0x7f11f40c3d00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ce810 tx=0x7f11e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6af289070683f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.808+0000 7fde0aa00640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fde0c08e580 tx=0x7fddf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3cf9571b39f69e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f11f3400640 1 -- 172.21.15.46:0/2190220326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8037070 con 0x7f11f4163720 2024-04-26T23:44:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f11f3400640 1 -- 172.21.15.46:0/2190220326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e8032440 con 0x7f11f4163720 2024-04-26T23:44:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f11f3400640 1 -- 172.21.15.46:0/2190220326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8032810 con 0x7f11f4163720 2024-04-26T23:44:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f804b400640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 0x7f8030004c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f803800c160 tx=0x7f8030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e86b808c3203bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7fde0a000640 1 -- 172.21.15.46:0/3793366124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8037070 con 0x7fde0c0d1fd0 2024-04-26T23:44:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7fde0a000640 1 -- 172.21.15.46:0/3793366124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf8032440 con 0x7fde0c0d1fd0 2024-04-26T23:44:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7fde0a000640 1 -- 172.21.15.46:0/3793366124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8032810 con 0x7fde0c0d1fd0 2024-04-26T23:44:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038007c80 con 0x7f8030004800 2024-04-26T23:44:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038005af0 con 0x7f8030004800 2024-04-26T23:44:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f804aa00640 1 -- 172.21.15.46:0/3192882643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038007070 con 0x7f8030004800 2024-04-26T23:44:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.809+0000 7f11fb600640 1 -- 172.21.15.46:0/2190220326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f4163720 msgr2=0x7f11f40c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 --2- 172.21.15.46:0/2190220326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f4163720 0x7f11f40c3d00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ce810 tx=0x7f11e802a230 comp rx=0 tx=0).stop 2024-04-26T23:44:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db6000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9db0071db0 0x7f9db0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 -- 172.21.15.46:0/2190220326 shutdown_connections 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 --2- 172.21.15.46:0/2190220326 >> v2:172.21.15.142:3300/0 conn(0x7f11f406f590 0x7f11f40c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 --2- 172.21.15.46:0/2190220326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f4163720 0x7f11f40c3d00 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 -- 172.21.15.46:0/2190220326 >> 172.21.15.46:0/2190220326 conn(0x7f11f40d0d10 msgr2=0x7f11f40d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 0x7f9db00726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db6000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db00c7a00 con 0x7f9db00736b0 2024-04-26T23:44:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9db0072c10 con 0x7f9db00cc840 2024-04-26T23:44:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db6000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9db006dd40 con 0x7f9db0071db0 2024-04-26T23:44:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 -- 172.21.15.46:0/2190220326 shutdown_connections 2024-04-26T23:44:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 -- 172.21.15.46:0/3793366124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d1fd0 msgr2=0x7fde0c0d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fde0c08e580 tx=0x7fddf802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9dafe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9db00736b0 0x7f9db015d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47502/0 (socket says 172.21.15.46:47502) 2024-04-26T23:44:01.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9dafe00640 1 -- 172.21.15.46:0/3326601445 learned_addr learned my addr 172.21.15.46:0/3326601445 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:01.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f8051800640 1 -- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 msgr2=0x7f8030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 -- 172.21.15.46:0/3793366124 shutdown_connections 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 --2- 172.21.15.46:0/3793366124 >> v2:172.21.15.142:3300/0 conn(0x7fde0c0d0d20 0x7fde0c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d1fd0 0x7fde0c0d07e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 --2- 172.21.15.46:0/3793366124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7fde12200640 1 -- 172.21.15.46:0/3793366124 >> 172.21.15.46:0/3793366124 conn(0x7fde0c09e450 msgr2=0x7fde0c0d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f8051800640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 0x7f8030004c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f803800c160 tx=0x7f8030005830 comp rx=0 tx=0).stop 2024-04-26T23:44:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f11fb600640 1 -- 172.21.15.46:0/2190220326 wait complete. 2024-04-26T23:44:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db4c00640 1 --2- 172.21.15.46:0/3326601445 >> v2:172.21.15.142:3300/0 conn(0x7f9db0071db0 0x7f9db0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.810+0000 7f9db5600640 1 --2- 172.21.15.46:0/3326601445 >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 0x7f9db00726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7fde12200640 1 -- 172.21.15.46:0/3793366124 shutdown_connections 2024-04-26T23:44:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f8051800640 1 -- 172.21.15.46:0/3192882643 shutdown_connections 2024-04-26T23:44:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7fde12200640 1 -- 172.21.15.46:0/3793366124 wait complete. 2024-04-26T23:44:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f8051800640 1 --2- 172.21.15.46:0/3192882643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8030004800 0x7f8030004c80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f8051800640 1 -- 172.21.15.46:0/3192882643 >> 172.21.15.46:0/3192882643 conn(0x7f804c0d2d60 msgr2=0x7f804c0d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9daf400640 1 -- 172.21.15.46:0/3326601445 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2516971238 0 0) 0x7f9db00c7a00 con 0x7f9db00736b0 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9daf400640 1 -- 172.21.15.46:0/3326601445 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d98003520 con 0x7f9db00736b0 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9db4c00640 1 -- 172.21.15.46:0/3326601445 >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 msgr2=0x7f9db00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9db4c00640 1 --2- 172.21.15.46:0/3326601445 >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 0x7f9db00726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9db4c00640 1 -- 172.21.15.46:0/3326601445 >> v1:172.21.15.46:6789/0 conn(0x7f9db00736b0 legacy=0x7f9db015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9db4c00640 1 -- 172.21.15.46:0/3326601445 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db006df90 con 0x7f9db0071db0 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.811+0000 7f9db5600640 1 --2- 172.21.15.46:0/3326601445 >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 0x7f9db00726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.812+0000 7f8051800640 1 -- 172.21.15.46:0/3192882643 shutdown_connections 2024-04-26T23:44:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.812+0000 7f8051800640 1 -- 172.21.15.46:0/3192882643 wait complete. 2024-04-26T23:44:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f11fb600640 1 Processor -- start 2024-04-26T23:44:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f11fb600640 1 -- start start 2024-04-26T23:44:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f9db4c00640 1 --2- 172.21.15.46:0/3326601445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db0071db0 0x7f9db0072190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9db006d7e0 tx=0x7f9da402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ae96ff31bd6fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7fde12200640 1 Processor -- start 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f9daf400640 1 -- 172.21.15.46:0/3326601445 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4037070 con 0x7f9db0071db0 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f9daf400640 1 -- 172.21.15.46:0/3326601445 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da4032440 con 0x7f9db0071db0 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7f9daf400640 1 -- 172.21.15.46:0/3326601445 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4032810 con 0x7f9db0071db0 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.813+0000 7fde12200640 1 -- start start 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11f406fae0 0x7f11f40c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11f4163720 0x7f11f41518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4167e90 con 0x7f11f406f590 2024-04-26T23:44:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c1615d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 0x7fde0c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4167bb0 con 0x7f11f4163720 2024-04-26T23:44:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11fb600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11f40d07a0 con 0x7f11f406fae0 2024-04-26T23:44:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15f790 con 0x7fde0c0d0d20 2024-04-26T23:44:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0d39e0 con 0x7fde0c0b7780 2024-04-26T23:44:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7fde12200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c153960 con 0x7fde0c15f3b0 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 -- 172.21.15.46:0/3326601445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db0071db0 msgr2=0x7f9db0072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 --2- 172.21.15.46:0/3326601445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db0071db0 0x7f9db0072190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9db006d7e0 tx=0x7f9da402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 -- 172.21.15.46:0/3326601445 shutdown_connections 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 --2- 172.21.15.46:0/3326601445 >> v2:172.21.15.46:3300/0 conn(0x7f9db00cc840 0x7f9db00726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 --2- 172.21.15.46:0/3326601445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db0071db0 0x7f9db0072190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f9db6000640 1 -- 172.21.15.46:0/3326601445 >> 172.21.15.46:0/3326601445 conn(0x7f9db00d0b20 msgr2=0x7f9db00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f8051800640 1 Processor -- start 2024-04-26T23:44:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.814+0000 7f11f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47402/0 (socket says 172.21.15.46:47402) 2024-04-26T23:44:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 -- 172.21.15.46:0/1970631560 learned_addr learned my addr 172.21.15.46:0/1970631560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47400/0 (socket says 172.21.15.46:47400) 2024-04-26T23:44:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 -- 172.21.15.46:0/3803753777 learned_addr learned my addr 172.21.15.46:0/3803753777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f9db6000640 1 -- 172.21.15.46:0/3326601445 shutdown_connections 2024-04-26T23:44:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f8051800640 1 -- start start 2024-04-26T23:44:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0aa00640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c1615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f3e00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11f406fae0 0x7f11f40c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 -- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11f406fae0 msgr2=0x7f11f40c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11f406fae0 0x7f11f40c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 -- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11f4163720 msgr2=0x7f11f41518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11f4163720 0x7f11f41518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f11f8c00640 1 -- 172.21.15.46:0/3803753777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f4164b00 con 0x7f11f406f590 2024-04-26T23:44:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0be00640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 0x7fde0c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 -- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 msgr2=0x7fde0c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c1615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 -- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 msgr2=0x7fde0c0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 0x7fde0c0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7fde0b400640 1 -- 172.21.15.46:0/1970631560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c0c8c40 con 0x7fde0c0d0d20 2024-04-26T23:44:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7fde0be00640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 0x7fde0c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.815+0000 7f9db6000640 1 -- 172.21.15.46:0/3326601445 wait complete. 2024-04-26T23:44:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7fde0b400640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d800 tx=0x7fddfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11f8c00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009960 tx=0x7f11e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8041020 con 0x7f11f406f590 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11fb600640 1 -- 172.21.15.46:0/3803753777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f4164de0 con 0x7f11f406f590 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e8030780 con 0x7f11f406f590 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8037040 con 0x7f11f406f590 2024-04-26T23:44:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f11fb600640 1 -- 172.21.15.46:0/3803753777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41653a0 con 0x7f11f406f590 2024-04-26T23:44:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f8051800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 0x7f804c1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.816+0000 7f8051800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804c158d50 0x7f804c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b660 con 0x7fde0c0d0d20 2024-04-26T23:44:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc019070 con 0x7fde0c0d0d20 2024-04-26T23:44:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00ba20 con 0x7fde0c0d0d20 2024-04-26T23:44:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8051800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8050e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f9db6000640 1 Processor -- start 2024-04-26T23:44:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c0b8ce0 con 0x7fde0c0d0d20 2024-04-26T23:44:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c0ba9a0 con 0x7fde0c0d0d20 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f804be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 0x7f804c1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8051800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0d1d60 con 0x7f804c15afe0 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8051800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0afaf0 con 0x7f804c158d50 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8051800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0d1ee0 con 0x7f8030004800 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f804be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 0x7f804c1f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:55418/0 (socket says 172.21.15.46:55418) 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f804be00640 1 -- 172.21.15.46:0/4140623973 learned_addr learned my addr 172.21.15.46:0/4140623973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f8050e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47404/0 (socket says 172.21.15.46:47404) 2024-04-26T23:44:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.817+0000 7f804b400640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804c158d50 0x7f804c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f9db6000640 1 -- start start 2024-04-26T23:44:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f40693b0 con 0x7f11f406f590 2024-04-26T23:44:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f8050e00640 1 -- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 msgr2=0x7f804c1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f8050e00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 0x7f804c1f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f8050e00640 1 -- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804c158d50 msgr2=0x7f804c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f8050e00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804c158d50 0x7f804c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.818+0000 7f8050e00640 1 -- 172.21.15.46:0/4140623973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c15b9a0 con 0x7f804c15afe0 2024-04-26T23:44:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.819+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c0693b0 con 0x7fde0c0d0d20 2024-04-26T23:44:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.819+0000 7f8050e00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f804c073510 tx=0x7f804400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.819+0000 7f9db6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9db0071db0 0x7f9db00ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 0x7f9db014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00cb310 con 0x7f9db00736b0 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9db0073ef0 con 0x7f9db00cc840 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9db015d770 con 0x7f9db0071db0 2024-04-26T23:44:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddfc00bbc0 con 0x7fde0c0d0d20 2024-04-26T23:44:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 0x7f9db014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9dafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db4c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9db0071db0 0x7f9db00ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9dafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47408/0 (socket says 172.21.15.46:47408) 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9db5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 0x7f9db014d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:32818/0 (socket says 172.21.15.46:32818) 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.820+0000 7f9dafe00640 1 -- 172.21.15.46:0/3524593660 learned_addr learned my addr 172.21.15.46:0/3524593660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 -- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9db0071db0 msgr2=0x7f9db00ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9db0071db0 0x7f9db00ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 -- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 msgr2=0x7f9db014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 0x7f9db014d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 -- 172.21.15.46:0/3524593660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db00b4690 con 0x7f9db00736b0 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7f9dafe00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c430 tx=0x7f9d9c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.821+0000 7fde03400640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 0x7fddec06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.822+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fddfc08bff0 con 0x7fde0c0d0d20 2024-04-26T23:44:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.822+0000 7fde0aa00640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 0x7fddec06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.822+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11e8044020 con 0x7f11f406f590 2024-04-26T23:44:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.823+0000 7fde0aa00640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 0x7fddec06ee30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.823+0000 7f11f0c00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 0x7f11d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.823+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f11e80ac920 con 0x7f11f406f590 2024-04-26T23:44:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.824+0000 7f11f3e00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 0x7f11d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.824+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044003b80 con 0x7f804c15afe0 2024-04-26T23:44:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c017070 con 0x7f9db00736b0 2024-04-26T23:44:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db00b4970 con 0x7f9db00736b0 2024-04-26T23:44:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c15bc50 con 0x7f804c15afe0 2024-04-26T23:44:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c0b6e50 con 0x7f804c15afe0 2024-04-26T23:44:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f11f3e00640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 0x7f11d406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006fd0 tx=0x7f11e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db00c9260 con 0x7f9db00736b0 2024-04-26T23:44:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.825+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11e804b030 con 0x7f11f406f590 2024-04-26T23:44:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.826+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c0d2060 con 0x7f804c15afe0 2024-04-26T23:44:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.826+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db00c9460 con 0x7f9db00736b0 2024-04-26T23:44:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.829+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8044018070 con 0x7f804c15afe0 2024-04-26T23:44:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.829+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440050d0 con 0x7f804c15afe0 2024-04-26T23:44:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.829+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c005ce0 con 0x7f9db00736b0 2024-04-26T23:44:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.829+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c021060 con 0x7f9db00736b0 2024-04-26T23:44:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.830+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddfc05a360 con 0x7fde0c0d0d20 2024-04-26T23:44:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.832+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9c01a070 con 0x7f9db00736b0 2024-04-26T23:44:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.833+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8044005270 con 0x7f804c15afe0 2024-04-26T23:44:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.833+0000 7f9dacc00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 0x7f9d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.833+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f9d9c08b870 con 0x7f9db00736b0 2024-04-26T23:44:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.834+0000 7f8043e00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 0x7f802c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.834+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f804408c420 con 0x7f804c15afe0 2024-04-26T23:44:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.834+0000 7f804be00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 0x7f802c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.834+0000 7f9db4c00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 0x7f9d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.835+0000 7f804be00640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 0x7f802c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f803c009c40 tx=0x7f803c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.836+0000 7f9db4c00640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 0x7f9d9006edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9da4034000 tx=0x7f9da402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.837+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f804c0d2060 con 0x7f804c15afe0 2024-04-26T23:44:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:01.841+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9db00c9460 con 0x7f9db00736b0 2024-04-26T23:44:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.109+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7f11f4151e10 con 0x7f11f406f590 2024-04-26T23:44:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.117+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f9db00b6aa0 con 0x7f9db00736b0 2024-04-26T23:44:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.212+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f804c0693b0 con 0x7f804c15afe0 2024-04-26T23:44:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.213+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fde0c0d27b0 con 0x7fde0c0d0d20 2024-04-26T23:44:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.361+0000 7f11f0c00640 1 -- 172.21.15.46:0/3803753777 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7158 v60) ==== 95+0+0 (secure 0 0 0) 0x7f11e807ad50 con 0x7f11f406f590 2024-04-26T23:44:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 7158 2024-04-26T23:44:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 msgr2=0x7f11d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 0x7f11d406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006fd0 tx=0x7f11e0008040 comp rx=0 tx=0).stop 2024-04-26T23:44:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 msgr2=0x7f11f4164440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009960 tx=0x7f11e80046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 shutdown_connections 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d406c8f0 0x7f11d406edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11f4163720 0x7f11f41518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11f406fae0 0x7f11f40c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 --2- 172.21.15.46:0/3803753777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f406f590 0x7f11f4164440 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.364+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 >> 172.21.15.46:0/3803753777 conn(0x7f11f40d0d10 msgr2=0x7f11f40b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:02.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.365+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 shutdown_connections 2024-04-26T23:44:02.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.365+0000 7f11e6000640 1 -- 172.21.15.46:0/3803753777 wait complete. 2024-04-26T23:44:02.376 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:44:02.371+0000 7f074f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:44:02.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.378+0000 7f9dacc00640 1 -- 172.21.15.46:0/3524593660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 7161 v61) ==== 95+0+0 (secure 0 0 0) 0x7f9d9c059560 con 0x7f9db00736b0 2024-04-26T23:44:02.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.378+0000 7f8043e00640 1 -- 172.21.15.46:0/4140623973 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7164 v61) ==== 95+0+0 (secure 0 0 0) 0x7f804405a620 con 0x7f804c15afe0 2024-04-26T23:44:02.383 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 7161 2024-04-26T23:44:02.383 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 7164 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.382+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 msgr2=0x7f9d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.382+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 0x7f9d9006edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9da4034000 tx=0x7f9da402e000 comp rx=0 tx=0).stop 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 msgr2=0x7f9db01f9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c430 tx=0x7f9d9c00c830 comp rx=0 tx=0).stop 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 msgr2=0x7f802c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 0x7f802c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f803c009c40 tx=0x7f803c009190 comp rx=0 tx=0).stop 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 msgr2=0x7f804c15b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f804c073510 tx=0x7f804400ead0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 shutdown_connections 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9d9006c8f0 0x7f9d9006edb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db00cc840 0x7f9db014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db00736b0 0x7f9db01f9020 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 --2- 172.21.15.46:0/3524593660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9db0071db0 0x7f9db00ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 >> 172.21.15.46:0/3524593660 conn(0x7f9db00d0b20 msgr2=0x7f9db00ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 shutdown_connections 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f802c06c8f0 0x7f802c06edb0 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f803c009c40 tx=0x7f803c009190 comp rx=0 tx=0).stop 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f804c15afe0 0x7f804c15b460 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804c158d50 0x7f804c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 --2- 172.21.15.46:0/4140623973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8030004800 0x7f804c1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.383+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 >> 172.21.15.46:0/4140623973 conn(0x7f804c0d2d60 msgr2=0x7f804c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.384+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 shutdown_connections 2024-04-26T23:44:02.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.384+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 shutdown_connections 2024-04-26T23:44:02.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.384+0000 7f9db6000640 1 -- 172.21.15.46:0/3524593660 wait complete. 2024-04-26T23:44:02.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:02.384+0000 7f8051800640 1 -- 172.21.15.46:0/4140623973 wait complete. 2024-04-26T23:44:02.391 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:44:02.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:44:02.402 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:44:02.402 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:02.412 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:44:02.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:44:02.413 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:44:02.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:44:02.420 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:44:02.420 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:02.425 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:44:02.426 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:02.554 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:44:02.569 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:44:02.570 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:44:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.382+0000 7fde03400640 1 -- 172.21.15.46:0/1970631560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 5351 v62) ==== 95+0+0 (secure 0 0 0) 0x7fddfc059c60 con 0x7fde0c0d0d20 2024-04-26T23:44:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 5351 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.385+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 msgr2=0x7fddec06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.385+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 0x7fddec06ee30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.385+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 msgr2=0x7fde0c161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.385+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d800 tx=0x7fddfc00dcc0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 shutdown_connections 2024-04-26T23:44:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fddec06c970 0x7fddec06ee30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c0b7780 0x7fde0c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde0c15f3b0 0x7fde0c1615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 --2- 172.21.15.46:0/1970631560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde0c0d0d20 0x7fde0c161090 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 >> 172.21.15.46:0/1970631560 conn(0x7fde0c09e450 msgr2=0x7fde0c15d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 shutdown_connections 2024-04-26T23:44:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.386+0000 7fde12200640 1 -- 172.21.15.46:0/1970631560 wait complete. 2024-04-26T23:44:03.415 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:44:03.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:44:03.419 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:44:03.420 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:03.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:03.564 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:44:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.580+0000 7f1fc6200640 1 Processor -- start 2024-04-26T23:44:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.580+0000 7f1fc6200640 1 -- start start 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fc0073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fc6200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc00b43f0 con 0x7f1fc0142280 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fc6200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc00b46d0 con 0x7f1fc006ec20 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fc6200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00b49b0 con 0x7f1fc01604c0 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fbf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1fc0142280 0x7f1fc006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58118/0 (socket says 172.21.15.46:58118) 2024-04-26T23:44:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fbf400640 1 -- 172.21.15.46:0/1273832794 learned_addr learned my addr 172.21.15.46:0/1273832794 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672150222 0 0) 0x7f1fc00b43f0 con 0x7f1fc0142280 2024-04-26T23:44:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fbfe00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fc0073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.582+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa4003520 con 0x7f1fc0142280 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3178779728 0 0) 0x7f1fa4003520 con 0x7f1fc0142280 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 msgr2=0x7f1fc0073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fc0073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 >> v1:172.21.15.142:6789/0 conn(0x7f1fc006ec20 legacy=0x7f1fc00734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00b4c90 con 0x7f1fc0142280 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1fb0001690 con 0x7f1fc0142280 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 >> v1:172.21.15.46:6789/0 conn(0x7f1fc0142280 legacy=0x7f1fc006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40039e0 0x7f1fa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fa4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00b49b0 con 0x7f1fa4004d40 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa400bad0 con 0x7f1fa40039e0 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa400bcd0 con 0x7f1fc01604c0 2024-04-26T23:44:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.583+0000 7f1fbfe00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbf400640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40039e0 0x7f1fa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbea00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fa4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbfe00640 1 -- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 msgr2=0x7f1fa4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbfe00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc01604c0 0x7f1fa4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbfe00640 1 -- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40039e0 msgr2=0x7f1fa4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbfe00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40039e0 0x7f1fa4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbfe00640 1 -- 172.21.15.46:0/1273832794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4003520 con 0x7f1fa4004d40 2024-04-26T23:44:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.584+0000 7f1fbf400640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40039e0 0x7f1fa4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fbfe00640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fa40051c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f90 tx=0x7f1fa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b55c083084d6f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8009c10 con 0x7f1fa4004d40 2024-04-26T23:44:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb8036070 con 0x7f1fa4004d40 2024-04-26T23:44:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fbe000640 1 -- 172.21.15.46:0/1273832794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8031850 con 0x7f1fa4004d40 2024-04-26T23:44:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fc6200640 1 -- 172.21.15.46:0/1273832794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 msgr2=0x7f1fa40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fc6200640 1 --2- 172.21.15.46:0/1273832794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fa40051c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f90 tx=0x7f1fa4005830 comp rx=0 tx=0).stop 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fbfe00640 1 -- 172.21.15.46:0/1273832794 reap_dead start 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fc6200640 1 -- 172.21.15.46:0/1273832794 shutdown_connections 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fc6200640 1 -- 172.21.15.46:0/1273832794 >> 172.21.15.46:0/1273832794 conn(0x7f1fc006c530 msgr2=0x7f1fc006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.585+0000 7f1fc6200640 1 -- 172.21.15.46:0/1273832794 shutdown_connections 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.586+0000 7f1fc6200640 1 -- 172.21.15.46:0/1273832794 wait complete. 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.587+0000 7f1fc6200640 1 Processor -- start 2024-04-26T23:44:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.587+0000 7f1fc6200640 1 -- start start 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc00b1e30 0x7f1fc00b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fc015b810 0x7f1fc015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0142a20 con 0x7f1fa4004d40 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01428a0 con 0x7f1fc015b810 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fc6200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00b3e80 con 0x7f1fc00b1e30 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46016/0 (socket says 172.21.15.46:46016) 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.588+0000 7f1fbf400640 1 -- 172.21.15.46:0/904758162 learned_addr learned my addr 172.21.15.46:0/904758162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 -- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc00b1e30 msgr2=0x7f1fc00b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc00b1e30 0x7f1fc00b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 -- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fc015b810 msgr2=0x7f1fc015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fc015b810 0x7f1fc015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 -- 172.21.15.46:0/904758162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc015c1d0 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fbf400640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000d7f0 tx=0x7f1fb000dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0019070 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb00074e0 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00049e0 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc015f680 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.589+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc015fc40 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.590+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc0069530 con 0x7f1fa4004d40 2024-04-26T23:44:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.591+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fb0004b80 con 0x7f1fa4004d40 2024-04-26T23:44:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.592+0000 7f1fb7400640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 0x7f1fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.592+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f1fb008b830 con 0x7f1fa4004d40 2024-04-26T23:44:03.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.593+0000 7f1fbea00640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 0x7f1fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.594+0000 7f1fbea00640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 0x7f1fa006edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.597+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fb00599e0 con 0x7f1fa4004d40 2024-04-26T23:44:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.837+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fc00cd570 con 0x7f1fa4004d40 2024-04-26T23:44:03.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.838+0000 7f1fb7400640 1 -- 172.21.15.46:0/904758162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+4355 (secure 0 0 0) 0x7f1fb00592e0 con 0x7f1fa4004d40 2024-04-26T23:44:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:03.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:03.388245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-26T23:44:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 63 2024-04-26T23:44:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 msgr2=0x7f1fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 0x7f1fa006edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac009040 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 msgr2=0x7f1fc013c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000d7f0 tx=0x7f1fb000dcb0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 shutdown_connections 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fa006c8f0 0x7f1fa006edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fc015b810 0x7f1fc015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fc00b1e30 0x7f1fc00b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 --2- 172.21.15.46:0/904758162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa4004d40 0x7f1fc013c3b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 >> 172.21.15.46:0/904758162 conn(0x7f1fc006c530 msgr2=0x7f1fc01452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 shutdown_connections 2024-04-26T23:44:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:03.841+0000 7f1fc6200640 1 -- 172.21.15.46:0/904758162 wait complete. 2024-04-26T23:44:04.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:43:48.721859+0000', 'modified': '2024-04-26T23:44:03.388245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7416}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7416': {'gid': 7416, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6833/357252295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 357252295}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 357252295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:44:04.266 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7416:up:replay 2024-04-26T23:44:05.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.477+0000 7fddb9200640 1 Processor -- start 2024-04-26T23:44:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.477+0000 7fddb9200640 1 -- start start 2024-04-26T23:44:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb415edf0 0x7fddb40cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb40b03c0 con 0x7fddb40cd770 2024-04-26T23:44:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb9200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb40cb680 con 0x7fddb415f1d0 2024-04-26T23:44:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40cf5c0 con 0x7fddb415edf0 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb415edf0 0x7fddb40cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fddb40cd770 0x7fddb40c5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58120/0 (socket says 172.21.15.46:58120) 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.479+0000 7fddb3e00640 1 -- 172.21.15.46:0/2925471344 learned_addr learned my addr 172.21.15.46:0/2925471344 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.480+0000 7fddb2000640 1 -- 172.21.15.46:0/2925471344 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879193893 0 0) 0x7fddb40b03c0 con 0x7fddb40cd770 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.480+0000 7fddb3400640 1 -- 172.21.15.46:0/2925471344 >> v1:172.21.15.142:6789/0 conn(0x7fddb415f1d0 legacy=0x7fddb40cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.480+0000 7fddb3400640 1 -- 172.21.15.46:0/2925471344 >> v1:172.21.15.46:6789/0 conn(0x7fddb40cd770 legacy=0x7fddb40c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.480+0000 7fddb3400640 1 -- 172.21.15.46:0/2925471344 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40cb960 con 0x7fddb415edf0 2024-04-26T23:44:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.482+0000 7fddb3400640 1 --2- 172.21.15.46:0/2925471344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb415edf0 0x7fddb40cf080 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009d90 tx=0x7fdda402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83386a634cc1bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.482+0000 7fddb2000640 1 -- 172.21.15.46:0/2925471344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037070 con 0x7fddb415edf0 2024-04-26T23:44:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.482+0000 7fddb2000640 1 -- 172.21.15.46:0/2925471344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda4007740 con 0x7fddb415edf0 2024-04-26T23:44:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.482+0000 7fddb2000640 1 -- 172.21.15.46:0/2925471344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4007b10 con 0x7fddb415edf0 2024-04-26T23:44:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 -- 172.21.15.46:0/2925471344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb415edf0 msgr2=0x7fddb40cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 --2- 172.21.15.46:0/2925471344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb415edf0 0x7fddb40cf080 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009d90 tx=0x7fdda402a780 comp rx=0 tx=0).stop 2024-04-26T23:44:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 -- 172.21.15.46:0/2925471344 shutdown_connections 2024-04-26T23:44:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 --2- 172.21.15.46:0/2925471344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb415edf0 0x7fddb40cf080 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 -- 172.21.15.46:0/2925471344 >> 172.21.15.46:0/2925471344 conn(0x7fddb40b26d0 msgr2=0x7fddb40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 -- 172.21.15.46:0/2925471344 shutdown_connections 2024-04-26T23:44:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.483+0000 7fddb9200640 1 -- 172.21.15.46:0/2925471344 wait complete. 2024-04-26T23:44:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.485+0000 7fddb9200640 1 Processor -- start 2024-04-26T23:44:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.485+0000 7fddb9200640 1 -- start start 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40cd770 0x7fddb40c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb415edf0 0x7fddb4151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cc200 con 0x7fddb41530a0 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cbce0 con 0x7fddb40cd770 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40cb1a0 con 0x7fddb415edf0 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46032/0 (socket says 172.21.15.46:46032) 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.486+0000 7fddb3e00640 1 -- 172.21.15.46:0/634891966 learned_addr learned my addr 172.21.15.46:0/634891966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 -- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb415edf0 msgr2=0x7fddb4151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb415edf0 0x7fddb4151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 -- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40cd770 msgr2=0x7fddb40c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40cd770 0x7fddb40c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 -- 172.21.15.46:0/634891966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4151ec0 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb3e00640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c450 tx=0x7fddac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac017070 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac005d00 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac007310 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.487+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb4159df0 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.488+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb415a3b0 con 0x7fddb41530a0 2024-04-26T23:44:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.489+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb40693b0 con 0x7fddb41530a0 2024-04-26T23:44:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.490+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddac01a070 con 0x7fddb41530a0 2024-04-26T23:44:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.491+0000 7fddab400640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.491+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fddac08b260 con 0x7fddb41530a0 2024-04-26T23:44:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.491+0000 7fddb3400640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.493+0000 7fddb3400640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdda4034000 tx=0x7fdda402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.496+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddac059410 con 0x7fddb41530a0 2024-04-26T23:44:05.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.748+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddb40d1610 con 0x7fddb41530a0 2024-04-26T23:44:05.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.749+0000 7fddab400640 1 -- 172.21.15.46:0/634891966 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5180 (secure 0 0 0) 0x7fddac058d10 con 0x7fddb41530a0 2024-04-26T23:44:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:05.755 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:04.410769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-26T23:44:05.755 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 65 2024-04-26T23:44:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 msgr2=0x7fdd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdda4034000 tx=0x7fdda402c000 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 msgr2=0x7fddb4153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c450 tx=0x7fddac00c910 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 shutdown_connections 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41530a0 0x7fddb4153550 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb415edf0 0x7fddb4151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 --2- 172.21.15.46:0/634891966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40cd770 0x7fddb40c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.752+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 >> 172.21.15.46:0/634891966 conn(0x7fddb40b26d0 msgr2=0x7fddb40b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.753+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 shutdown_connections 2024-04-26T23:44:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:05.753+0000 7fddb9200640 1 -- 172.21.15.46:0/634891966 wait complete. 2024-04-26T23:44:06.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:43:48.721859+0000', 'modified': '2024-04-26T23:44:04.410769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7416}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7416': {'gid': 7416, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6833/357252295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 357252295}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 357252295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:44:06.179 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7416:up:rejoin 2024-04-26T23:44:07.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:07.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.395+0000 7fc265800640 1 Processor -- start 2024-04-26T23:44:07.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.395+0000 7fc265800640 1 -- start start 2024-04-26T23:44:07.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc265800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2601466b0 0x7fc26006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc265800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc260160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc265800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc260073ad0 con 0x7fc2600720e0 2024-04-26T23:44:07.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc265800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc260146a90 con 0x7fc2601466b0 2024-04-26T23:44:07.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc265800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc260073d70 con 0x7fc260160910 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.397+0000 7fc25fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2601466b0 0x7fc26006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2601466b0 0x7fc26006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46036/0 (socket says 172.21.15.46:46036) 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 -- 172.21.15.46:0/2095447066 learned_addr learned my addr 172.21.15.46:0/2095447066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc264e00640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc260160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 -- 172.21.15.46:0/2095447066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 msgr2=0x7fc260160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc260160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 -- 172.21.15.46:0/2095447066 >> v1:172.21.15.142:6789/0 conn(0x7fc2600720e0 legacy=0x7fc2600cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:07.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.398+0000 7fc25fe00640 1 -- 172.21.15.46:0/2095447066 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc260074050 con 0x7fc2601466b0 2024-04-26T23:44:07.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.400+0000 7fc25fe00640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26006de80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc2500098a0 tx=0x7fc25002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebac93b3ef7b91e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:07.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.400+0000 7fc25ea00640 1 -- 172.21.15.46:0/2095447066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250037070 con 0x7fc2601466b0 2024-04-26T23:44:07.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.400+0000 7fc25ea00640 1 -- 172.21.15.46:0/2095447066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc250032440 con 0x7fc2601466b0 2024-04-26T23:44:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.400+0000 7fc25ea00640 1 -- 172.21.15.46:0/2095447066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250032810 con 0x7fc2601466b0 2024-04-26T23:44:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 -- 172.21.15.46:0/2095447066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 msgr2=0x7fc26006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26006de80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc2500098a0 tx=0x7fc25002a250 comp rx=0 tx=0).stop 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 -- 172.21.15.46:0/2095447066 shutdown_connections 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc260160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 --2- 172.21.15.46:0/2095447066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26006de80 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 -- 172.21.15.46:0/2095447066 >> 172.21.15.46:0/2095447066 conn(0x7fc2600b26d0 msgr2=0x7fc2600b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 -- 172.21.15.46:0/2095447066 shutdown_connections 2024-04-26T23:44:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.401+0000 7fc265800640 1 -- 172.21.15.46:0/2095447066 wait complete. 2024-04-26T23:44:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.403+0000 7fc265800640 1 Processor -- start 2024-04-26T23:44:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.403+0000 7fc265800640 1 -- start start 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2600720e0 0x7fc2600b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc26015f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc26006e3c0 con 0x7fc2601466b0 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600cb470 con 0x7fc2600720e0 2024-04-26T23:44:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc265800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc2600cb990 con 0x7fc260160910 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc25f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc25f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46044/0 (socket says 172.21.15.46:46044) 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.404+0000 7fc25f400640 1 -- 172.21.15.46:0/2535643669 learned_addr learned my addr 172.21.15.46:0/2535643669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25fe00640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2600720e0 0x7fc2600b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc264e00640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc26015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 -- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 msgr2=0x7fc26015f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc26015f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 -- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2600720e0 msgr2=0x7fc2600b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2600720e0 0x7fc2600b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 -- 172.21.15.46:0/2535643669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2600b4d30 con 0x7fc2601466b0 2024-04-26T23:44:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.405+0000 7fc25f400640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c430 tx=0x7fc24c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.406+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c011790 con 0x7fc2601466b0 2024-04-26T23:44:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.406+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24c017070 con 0x7fc2601466b0 2024-04-26T23:44:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.406+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c011bb0 con 0x7fc2601466b0 2024-04-26T23:44:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.406+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2600b5010 con 0x7fc2601466b0 2024-04-26T23:44:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.406+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2600b55d0 con 0x7fc2601466b0 2024-04-26T23:44:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.407+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2600693b0 con 0x7fc2601466b0 2024-04-26T23:44:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.408+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc24c01c460 con 0x7fc2601466b0 2024-04-26T23:44:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.409+0000 7fc257e00640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 0x7fc24006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.409+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fc24c08bbc0 con 0x7fc2601466b0 2024-04-26T23:44:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.409+0000 7fc25fe00640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 0x7fc24006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.411+0000 7fc25fe00640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 0x7fc24006ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc250034000 tx=0x7fc25002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.414+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc24c059e00 con 0x7fc2601466b0 2024-04-26T23:44:07.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.660+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2600c7180 con 0x7fc2601466b0 2024-04-26T23:44:07.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.661+0000 7fc257e00640 1 -- 172.21.15.46:0/2535643669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7fc24c059700 con 0x7fc2601466b0 2024-04-26T23:44:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:07.666 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:06.419460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7416,"qdb_cluster":[7416]},"id":5}]} 2024-04-26T23:44:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 msgr2=0x7fc24006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 0x7fc24006ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc250034000 tx=0x7fc25002e000 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 msgr2=0x7fc26013d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c430 tx=0x7fc24c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 shutdown_connections 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc24006c920 0x7fc24006ede0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc260160910 0x7fc26015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2601466b0 0x7fc26013d510 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 --2- 172.21.15.46:0/2535643669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2600720e0 0x7fc2600b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 >> 172.21.15.46:0/2535643669 conn(0x7fc2600b26d0 msgr2=0x7fc2600c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 shutdown_connections 2024-04-26T23:44:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:07.664+0000 7fc265800640 1 -- 172.21.15.46:0/2535643669 wait complete. 2024-04-26T23:44:08.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:43:48.721859+0000', 'modified': '2024-04-26T23:44:06.419460+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7416}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7416': {'gid': 7416, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6833/357252295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 357252295}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 357252295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7416, 'qdb_cluster': [7416]} 2024-04-26T23:44:08.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:44:08.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:44:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.308+0000 7f2657200640 1 Processor -- start 2024-04-26T23:44:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.308+0000 7f2657200640 1 -- start start 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.309+0000 7f2657200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26581454a0 0x7f26580cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.309+0000 7f2657200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f26580cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.309+0000 7f2657200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2658072bb0 con 0x7f26580c6a50 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.309+0000 7f2657200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2658072e90 con 0x7f26581454a0 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.309+0000 7f2657200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2658073170 con 0x7f2658073970 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2656800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26580c6a50 0x7f26580cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58134/0 (socket says 172.21.15.46:58134) 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2656800640 1 -- 172.21.15.46:0/999475237 learned_addr learned my addr 172.21.15.46:0/999475237 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2655e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26581454a0 0x7f26580cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2655400640 1 -- 172.21.15.46:0/999475237 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449721860 0 0) 0x7f2658072bb0 con 0x7f26580c6a50 2024-04-26T23:44:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2655400640 1 -- 172.21.15.46:0/999475237 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f263c003520 con 0x7f26580c6a50 2024-04-26T23:44:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f264fe00640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f26580cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.310+0000 7f2655e00640 1 -- 172.21.15.46:0/999475237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 msgr2=0x7f26580cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.311+0000 7f2655e00640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f26580cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.311+0000 7f2655e00640 1 -- 172.21.15.46:0/999475237 >> v1:172.21.15.46:6789/0 conn(0x7f26580c6a50 legacy=0x7f26580cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.311+0000 7f2655e00640 1 -- 172.21.15.46:0/999475237 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658073450 con 0x7f26581454a0 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.311+0000 7f264fe00640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f26580cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.312+0000 7f2655e00640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580cc690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c464c340550d8a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.312+0000 7f2655400640 1 -- 172.21.15.46:0/999475237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648037070 con 0x7f26581454a0 2024-04-26T23:44:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.312+0000 7f2655400640 1 -- 172.21.15.46:0/999475237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648032440 con 0x7f26581454a0 2024-04-26T23:44:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.312+0000 7f2655400640 1 -- 172.21.15.46:0/999475237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648032810 con 0x7f26581454a0 2024-04-26T23:44:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 -- 172.21.15.46:0/999475237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 msgr2=0x7f26580cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580cc690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a180 comp rx=0 tx=0).stop 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 -- 172.21.15.46:0/999475237 shutdown_connections 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f26580cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 --2- 172.21.15.46:0/999475237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580cc690 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 -- 172.21.15.46:0/999475237 >> 172.21.15.46:0/999475237 conn(0x7f26580b26d0 msgr2=0x7f26580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 -- 172.21.15.46:0/999475237 shutdown_connections 2024-04-26T23:44:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.313+0000 7f2657200640 1 -- 172.21.15.46:0/999475237 wait complete. 2024-04-26T23:44:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.315+0000 7f2657200640 1 Processor -- start 2024-04-26T23:44:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.315+0000 7f2657200640 1 -- start start 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f265813c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26580c6a50 0x7f26580af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2658073dd0 con 0x7f26581454a0 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2658072880 con 0x7f26580c6a50 2024-04-26T23:44:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2657200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26580b0e60 con 0x7f2658073970 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46056/0 (socket says 172.21.15.46:46056) 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2656800640 1 -- 172.21.15.46:0/4226530327 learned_addr learned my addr 172.21.15.46:0/4226530327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.316+0000 7f2655e00640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f265813c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 -- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 msgr2=0x7f265813c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f265813c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 -- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26580c6a50 msgr2=0x7f26580af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26580c6a50 0x7f26580af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 -- 172.21.15.46:0/4226530327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580b0250 con 0x7f26581454a0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2656800640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb40 tx=0x7f265000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f264e000640 1 -- 172.21.15.46:0/4226530327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650017070 con 0x7f26581454a0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f264e000640 1 -- 172.21.15.46:0/4226530327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265000aca0 con 0x7f26581454a0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26580b0530 con 0x7f26581454a0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.317+0000 7f264e000640 1 -- 172.21.15.46:0/4226530327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650012620 con 0x7f26581454a0 2024-04-26T23:44:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.318+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26580b0af0 con 0x7f26581454a0 2024-04-26T23:44:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.321+0000 7f264e000640 1 -- 172.21.15.46:0/4226530327 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f265001a070 con 0x7f26581454a0 2024-04-26T23:44:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.322+0000 7f264e000640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.322+0000 7f264e000640 1 -- 172.21.15.46:0/4226530327 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f265008bc10 con 0x7f26581454a0 2024-04-26T23:44:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.322+0000 7f265fc02740 1 Processor -- start 2024-04-26T23:44:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.322+0000 7f265fc02740 1 -- start start 2024-04-26T23:44:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.322+0000 7f2655e00640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581be9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581be7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560581b4b600 con 0x560581c12270 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560581ae8100 con 0x560581be77b0 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560581b475c0 con 0x560581c23e80 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.323+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46068/0 (socket says 172.21.15.46:46068) 2024-04-26T23:44:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 -- 172.21.15.46:0/318631161 learned_addr learned my addr 172.21.15.46:0/318631161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2655e00640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f264fe00640 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581be7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 -- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 msgr2=0x560581be7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2655e00640 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581be9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581be7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 -- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 msgr2=0x560581be9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581be9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 -- 172.21.15.46:0/318631161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058185d2d0 con 0x560581c12270 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2656800640 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2650061d70 tx=0x7f26500596c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e801cc4ffc2d989 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.324+0000 7f2646a00640 1 -- 172.21.15.46:0/318631161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500622a0 con 0x560581c12270 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f2646a00640 1 -- 172.21.15.46:0/318631161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650071070 con 0x560581c12270 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f2646a00640 1 -- 172.21.15.46:0/318631161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650055d30 con 0x560581c12270 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 -- 172.21.15.46:0/318631161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 msgr2=0x560581c126f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2650061d70 tx=0x7f26500596c0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 -- 172.21.15.46:0/318631161 shutdown_connections 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c12270 0x560581c126f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581be7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 --2- 172.21.15.46:0/318631161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581be9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 -- 172.21.15.46:0/318631161 >> 172.21.15.46:0/318631161 conn(0x560581b7bd10 msgr2=0x560581c2d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 -- 172.21.15.46:0/318631161 shutdown_connections 2024-04-26T23:44:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.325+0000 7f265fc02740 1 -- 172.21.15.46:0/318631161 wait complete. 2024-04-26T23:44:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.326+0000 7f265fc02740 1 -- start start 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.326+0000 7f265fc02740 1 Processor -- start 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.327+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581bff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.327+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c12270 0x560581c12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.327+0000 7f265fc02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.327+0000 7f265fc02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560581ad2f70 con 0x560581c23e80 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560581c0fdc0 con 0x560581be77b0 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560581c3dc70 con 0x560581c12270 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f265fc02740 10 client.0 _ll_register_callbacks cb 0x560581b41480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46070/0 (socket says 172.21.15.46:46070) 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 -- 172.21.15.46:0/576244192 learned_addr learned my addr 172.21.15.46:0/576244192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f264fe00640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c12270 0x560581c12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2655e00640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581bff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 -- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c12270 msgr2=0x560581c12670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c12270 0x560581c12670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 -- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 msgr2=0x560581bff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581bff7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2656800640 1 -- 172.21.15.46:0/576244192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605819276b0 con 0x560581c23e80 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.328+0000 7f2655e00640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581bff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f2656800640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2650059650 tx=0x7f2650001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37750018e713dec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f263b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650004b60 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f2650004d00 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2650071070 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2650058b10 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f265fc02740 10 client.7467 fetch_fsmap learned FSMap version 66 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x560581c14500 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.329+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650071070 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f2650058cb0 con 0x560581c23e80 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 10 client.7467 fetch_fsmap finished waiting for FSMap version 66 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 10 client.7467 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 20 client.7467 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581c3d250 0x560581c24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 4 client.7467 mds_command: new command op to 7416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:44:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 --> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560581b59210 con 0x560581c3d250 2024-04-26T23:44:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.330+0000 7f2655e00640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581c3d250 0x560581c24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.331+0000 7f2655e00640 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581c3d250 0x560581c24260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75445301dc6591d server_cookie=c4e9ee24a07ede86 in_seq=0 out_seq=0 2024-04-26T23:44:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.331+0000 7f263b400640 10 client.7467 ms_handle_connect on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.331+0000 7f263b400640 1 -- 172.21.15.46:0/576244192 <== mds.0 v2:172.21.15.46:6832/357252295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560581b59210 con 0x560581c3d250 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.331+0000 7f263b400640 10 client.7467 handle_command_reply: tid=0 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 client.7467 shutdown 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 msgr2=0x560581c13ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2650059650 tx=0x7f2650001250 comp rx=0 tx=0).stop 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 shutdown_connections 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581c3d250 0x560581c24260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c13ed0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c12270 0x560581c12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 --2- 172.21.15.46:0/576244192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581be77b0 0x560581bff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f263b400640 0 client.7467 ms_handle_reset on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 >> 172.21.15.46:0/576244192 conn(0x560581b7bd10 msgr2=0x560581c101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 shutdown_connections 2024-04-26T23:44:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 1 -- 172.21.15.46:0/576244192 wait complete. 2024-04-26T23:44:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.332+0000 7f265fc02740 20 client.7467 trim_cache size 0 max 16384 2024-04-26T23:44:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.352+0000 7f265fc02740 1 Processor -- start 2024-04-26T23:44:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.352+0000 7f265fc02740 1 -- start start 2024-04-26T23:44:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581be8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581c26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2658200a40 con 0x560581be77b0 2024-04-26T23:44:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26580b5ce0 con 0x560581c12270 2024-04-26T23:44:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.353+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26582004c0 con 0x560581c23e80 2024-04-26T23:44:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46086/0 (socket says 172.21.15.46:46086) 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 -- 172.21.15.46:0/2576641152 learned_addr learned my addr 172.21.15.46:0/2576641152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2656800640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581c26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f264fe00640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581be8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 -- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 msgr2=0x560581c26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581c26840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 -- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 msgr2=0x560581be8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581be8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f2655e00640 1 -- 172.21.15.46:0/2576641152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560581bdd410 con 0x560581be77b0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.354+0000 7f264fe00640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581be8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.355+0000 7f2655e00640 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f264802a770 tx=0x7f264802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3420990f3fbfe5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.355+0000 7f2646000640 1 -- 172.21.15.46:0/2576641152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648037040 con 0x560581be77b0 2024-04-26T23:44:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.355+0000 7f2646000640 1 -- 172.21.15.46:0/2576641152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264804f070 con 0x560581be77b0 2024-04-26T23:44:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.355+0000 7f2646000640 1 -- 172.21.15.46:0/2576641152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264804a830 con 0x560581be77b0 2024-04-26T23:44:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 -- 172.21.15.46:0/2576641152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 msgr2=0x560581c0a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f264802a770 tx=0x7f264802c000 comp rx=0 tx=0).stop 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 -- 172.21.15.46:0/2576641152 shutdown_connections 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581c23e80 0x560581c26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581be8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 --2- 172.21.15.46:0/2576641152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581be77b0 0x560581c0a600 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 -- 172.21.15.46:0/2576641152 >> 172.21.15.46:0/2576641152 conn(0x560581b7bd10 msgr2=0x560581bdddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 -- 172.21.15.46:0/2576641152 shutdown_connections 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.356+0000 7f265fc02740 1 -- 172.21.15.46:0/2576641152 wait complete. 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.357+0000 7f265fc02740 1 -- start start 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.357+0000 7f265fc02740 1 Processor -- start 2024-04-26T23:44:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581be77b0 0x560581bfe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581c30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560581ad2f70 con 0x560581c23e80 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560581c0fdc0 con 0x560581c12270 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560581c0f030 con 0x560581be77b0 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f265fc02740 10 client.0 _ll_register_callbacks cb 0x7f26500592b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f2656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46102/0 (socket says 172.21.15.46:46102) 2024-04-26T23:44:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.358+0000 7f2656800640 1 -- 172.21.15.46:0/2924960007 learned_addr learned my addr 172.21.15.46:0/2924960007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 -- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581be77b0 msgr2=0x560581bfe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2655e00640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581be77b0 0x560581bfe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581be77b0 0x560581bfe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 -- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 msgr2=0x560581c30d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581c30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 -- 172.21.15.46:0/2924960007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560581c26d80 con 0x560581c23e80 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f2656800640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb10 tx=0x7f26500014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104f9f457584ecf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f263b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500622a0 con 0x560581c23e80 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f2650055640 con 0x560581c23e80 2024-04-26T23:44:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264804f070 con 0x560581c23e80 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.359+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2650004b60 con 0x560581c23e80 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 10 client.7473 fetch_fsmap learned FSMap version 66 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x560581c26d80 con 0x560581c23e80 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264804f070 con 0x560581c23e80 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f2650058580 con 0x560581c23e80 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 10 client.7473 fetch_fsmap finished waiting for FSMap version 66 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 10 client.7473 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 20 client.7473 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581ca7570 0x560581c22c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 4 client.7473 mds_command: new command op to 7416 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:44:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 --> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] -- command(tid 0: {"prefix": "status"}) -- 0x560581b53080 con 0x560581ca7570 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.360+0000 7f2655e00640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581ca7570 0x560581c22c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.361+0000 7f2655e00640 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581ca7570 0x560581c22c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe69663979ec269 server_cookie=fac34620e958fbe5 in_seq=0 out_seq=0 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.361+0000 7f263b400640 10 client.7473 ms_handle_connect on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.361+0000 7f263b400640 1 -- 172.21.15.46:0/2924960007 <== mds.0 v2:172.21.15.46:6832/357252295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560581b53080 con 0x560581ca7570 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.361+0000 7f263b400640 10 client.7473 handle_command_reply: tid=0 2024-04-26T23:44:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.361+0000 7f265fc02740 1 client.7473 shutdown 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 msgr2=0x560581c15a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb10 tx=0x7f26500014d0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 shutdown_connections 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x560581ca7570 0x560581c22c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560581c23e80 0x560581c15a40 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560581c12270 0x560581c30d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 --2- 172.21.15.46:0/2924960007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560581be77b0 0x560581bfe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f263b400640 0 client.7473 ms_handle_reset on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 >> 172.21.15.46:0/2924960007 conn(0x560581b7bd10 msgr2=0x560581bdddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 shutdown_connections 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 1 -- 172.21.15.46:0/2924960007 wait complete. 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.362+0000 7f265fc02740 20 client.7473 trim_cache size 0 max 16384 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:44:08.372 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7416, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.9663088799999997, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 66, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.8077360650000003 2024-04-26T23:44:08.373 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:44:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 msgr2=0x7f263406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 msgr2=0x7f26580c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb40 tx=0x7f265000a060 comp rx=0 tx=0).stop 2024-04-26T23:44:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 shutdown_connections 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26581454a0 0x7f26580c96a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26580c6a50 0x7f26580af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 --2- 172.21.15.46:0/4226530327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2658073970 0x7f265813c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 >> 172.21.15.46:0/4226530327 conn(0x7f26580b26d0 msgr2=0x7f265815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 shutdown_connections 2024-04-26T23:44:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.363+0000 7f2657200640 1 -- 172.21.15.46:0/4226530327 wait complete. 2024-04-26T23:44:08.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:44:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.584+0000 7f30ea400640 1 Processor -- start 2024-04-26T23:44:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.585+0000 7f30ea400640 1 -- start start 2024-04-26T23:44:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.586+0000 7f30ea400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30e40d1990 0x7f30e40d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.586+0000 7f30ea400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e40d22b0 con 0x7f30e41639c0 2024-04-26T23:44:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.586+0000 7f30ea400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e40cfe50 con 0x7f30e40d0c10 2024-04-26T23:44:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.586+0000 7f30ea400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f30e40d0780 con 0x7f30e40d1990 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.587+0000 7f30e9a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f30e41639c0 0x7f30e40b7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58140/0 (socket says 172.21.15.46:58140) 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.587+0000 7f30e9a00640 1 -- 172.21.15.46:0/3140933877 learned_addr learned my addr 172.21.15.46:0/3140933877 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.587+0000 7f30e9000640 1 --2- 172.21.15.46:0/3140933877 >> v2:172.21.15.46:3300/0 conn(0x7f30e40d1990 0x7f30e40d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.587+0000 7f30e3400640 1 -- 172.21.15.46:0/3140933877 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13314275 0 0) 0x7f30e40d22b0 con 0x7f30e41639c0 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.587+0000 7f30e3400640 1 -- 172.21.15.46:0/3140933877 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30c8003520 con 0x7f30e41639c0 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.588+0000 7f30e9000640 1 -- 172.21.15.46:0/3140933877 >> v1:172.21.15.142:6789/0 conn(0x7f30e40d0c10 legacy=0x7f30e40d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.588+0000 7f30e9000640 1 -- 172.21.15.46:0/3140933877 >> v1:172.21.15.46:6789/0 conn(0x7f30e41639c0 legacy=0x7f30e40b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.588+0000 7f30e9000640 1 -- 172.21.15.46:0/3140933877 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e40d0130 con 0x7f30e40d1990 2024-04-26T23:44:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.589+0000 7f30e9000640 1 --2- 172.21.15.46:0/3140933877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d1990 0x7f30e40d1d70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738cbb70ea9ac06f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.589+0000 7f30e3400640 1 -- 172.21.15.46:0/3140933877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4007740 con 0x7f30e40d1990 2024-04-26T23:44:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.589+0000 7f30e3400640 1 -- 172.21.15.46:0/3140933877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d40078e0 con 0x7f30e40d1990 2024-04-26T23:44:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.589+0000 7f30e3400640 1 -- 172.21.15.46:0/3140933877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4007cb0 con 0x7f30e40d1990 2024-04-26T23:44:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 -- 172.21.15.46:0/3140933877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d1990 msgr2=0x7f30e40d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 --2- 172.21.15.46:0/3140933877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d1990 0x7f30e40d1d70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 -- 172.21.15.46:0/3140933877 shutdown_connections 2024-04-26T23:44:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 --2- 172.21.15.46:0/3140933877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d1990 0x7f30e40d1d70 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 -- 172.21.15.46:0/3140933877 >> 172.21.15.46:0/3140933877 conn(0x7f30e409e2a0 msgr2=0x7f30e40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 -- 172.21.15.46:0/3140933877 shutdown_connections 2024-04-26T23:44:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.590+0000 7f30ea400640 1 -- 172.21.15.46:0/3140933877 wait complete. 2024-04-26T23:44:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.592+0000 7f30ea400640 1 Processor -- start 2024-04-26T23:44:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.592+0000 7f30ea400640 1 -- start start 2024-04-26T23:44:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e40d1990 0x7f30e41eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e41639c0 0x7f30e41eb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30e415f530 con 0x7f30e40d0c10 2024-04-26T23:44:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30e40d5da0 con 0x7f30e40d1990 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30ea400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30e40cfa90 con 0x7f30e41639c0 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30e9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30e9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46122/0 (socket says 172.21.15.46:46122) 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.593+0000 7f30e9000640 1 -- 172.21.15.46:0/1467041660 learned_addr learned my addr 172.21.15.46:0/1467041660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 -- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e41639c0 msgr2=0x7f30e41eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e41639c0 0x7f30e41eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 -- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e40d1990 msgr2=0x7f30e41eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e40d1990 0x7f30e41eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 -- 172.21.15.46:0/1467041660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e40b7c80 con 0x7f30e40d0c10 2024-04-26T23:44:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e9000640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009eb0 tx=0x7f30d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4040070 con 0x7f30e40d0c10 2024-04-26T23:44:08.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d4031c40 con 0x7f30e40d0c10 2024-04-26T23:44:08.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e40cce30 con 0x7f30e40d0c10 2024-04-26T23:44:08.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.594+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4045460 con 0x7f30e40d0c10 2024-04-26T23:44:08.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.595+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e40b8200 con 0x7f30e40d0c10 2024-04-26T23:44:08.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.596+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e40693b0 con 0x7f30e40d0c10 2024-04-26T23:44:08.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.597+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30d4039040 con 0x7f30e40d0c10 2024-04-26T23:44:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.598+0000 7f30e0c00640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 0x7f30c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.598+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f30d40ad670 con 0x7f30e40d0c10 2024-04-26T23:44:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.598+0000 7f30e3e00640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 0x7f30c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:08.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.600+0000 7f30e3e00640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 0x7f30c406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000f60 tx=0x7f30d00079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.603+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30d407b8b0 con 0x7f30e40d0c10 2024-04-26T23:44:08.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.849+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30e40d1770 con 0x7f30e40d0c10 2024-04-26T23:44:08.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.850+0000 7f30e0c00640 1 -- 172.21.15.46:0/1467041660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f30d407b1b0 con 0x7f30e40d0c10 2024-04-26T23:44:08.854 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:44:08.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.852+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 msgr2=0x7f30c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.852+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 0x7f30c406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000f60 tx=0x7f30d00079d0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.852+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 msgr2=0x7f30e415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.852+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009eb0 tx=0x7f30d4004a40 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 shutdown_connections 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f30c406c8f0 0x7f30c406edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e41639c0 0x7f30e41eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e40d1990 0x7f30e41eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 --2- 172.21.15.46:0/1467041660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e40d0c10 0x7f30e415c7b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 >> 172.21.15.46:0/1467041660 conn(0x7f30e409e2a0 msgr2=0x7f30e40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 shutdown_connections 2024-04-26T23:44:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:08.853+0000 7f30ea400640 1 -- 172.21.15.46:0/1467041660 wait complete. 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:44:08.876 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:08.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:08.877 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:44:08.930 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:44:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.932 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:44:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:08.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:08.934 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:44:09.019 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:44:09.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:09.023 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:44:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:44:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:44:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:44:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:44:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:44:09.080 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:44:09.080 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:44:09.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:44:09.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:44:09.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:44:09.081 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:44:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:44:09.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:09.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:09.301 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:44:09.349 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:44:09.349 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:44:09.350 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:09.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:09.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:44:09.403 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:44:09.398+0000 7f76aa000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:44:09.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:44:09.430 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:09.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:09.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:09.486 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:09.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:44:09.487 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:09.579 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:44:09.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:09.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:09.709 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:09.709 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:09.709 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:44:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:44:09.759 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:44:09.761 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:44:09.761 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:44:09.761 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:44:09.761 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:44:09.761 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:44:09.766 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:44:09.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:09.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:09.843 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:44:09.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:44:09.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:client7485 2024-04-26T23:44:09.966 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:09.966 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:10.017 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-26T23:44:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-26T23:44:10.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:44:10.070 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-26T23:44:10.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start /scrub_repair_path repair 2024-04-26T23:44:10.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.283+0000 7fe5ef200640 1 Processor -- start 2024-04-26T23:44:10.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.283+0000 7fe5ef200640 1 -- start start 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5f0146740 0x7fe5f00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f00d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f00cdc10 0x7fe5f00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f00b48f0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f00b3ce0 con 0x7fe5f0146740 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ef200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f00b3fc0 con 0x7fe5f0146b20 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.285+0000 7fe5ee800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f00cdc10 0x7fe5f00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f00cdc10 0x7fe5f00d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46126/0 (socket says 172.21.15.46:46126) 2024-04-26T23:44:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 -- 172.21.15.46:0/1998049108 learned_addr learned my addr 172.21.15.46:0/1998049108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 -- 172.21.15.46:0/1998049108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 msgr2=0x7fe5f00d10e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 --2- 172.21.15.46:0/1998049108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f00d10e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 -- 172.21.15.46:0/1998049108 >> v2:172.21.15.142:3300/0 conn(0x7fe5f0146740 msgr2=0x7fe5f00cd6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 --2- 172.21.15.46:0/1998049108 >> v2:172.21.15.142:3300/0 conn(0x7fe5f0146740 0x7fe5f00cd6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.286+0000 7fe5ee800640 1 -- 172.21.15.46:0/1998049108 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f00b42a0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.287+0000 7fe5ee800640 1 --2- 172.21.15.46:0/1998049108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f00d1620 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00b5280 tx=0x7fe5e802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a260dcf4811eb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.287+0000 7fe5eca00640 1 -- 172.21.15.46:0/1998049108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8036070 con 0x7fe5f00cdc10 2024-04-26T23:44:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5eca00640 1 -- 172.21.15.46:0/1998049108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8031600 con 0x7fe5f00cdc10 2024-04-26T23:44:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5eca00640 1 -- 172.21.15.46:0/1998049108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e802ae60 con 0x7fe5f00cdc10 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 -- 172.21.15.46:0/1998049108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 msgr2=0x7fe5f00d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1998049108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f00d1620 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00b5280 tx=0x7fe5e802a640 comp rx=0 tx=0).stop 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 -- 172.21.15.46:0/1998049108 shutdown_connections 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1998049108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f00d1620 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1998049108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f00d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1998049108 >> v2:172.21.15.142:3300/0 conn(0x7fe5f0146740 0x7fe5f00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 -- 172.21.15.46:0/1998049108 >> 172.21.15.46:0/1998049108 conn(0x7fe5f006c770 msgr2=0x7fe5f006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.288+0000 7fe5ef200640 1 -- 172.21.15.46:0/1998049108 shutdown_connections 2024-04-26T23:44:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.289+0000 7fe5ef200640 1 -- 172.21.15.46:0/1998049108 wait complete. 2024-04-26T23:44:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.290+0000 7fe5ef200640 1 Processor -- start 2024-04-26T23:44:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.290+0000 7fe5ef200640 1 -- start start 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5f0146740 0x7fe5f00b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0160c20 con 0x7fe5f00cdc10 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f015e560 con 0x7fe5f0146740 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.291+0000 7fe5ef200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f0160da0 con 0x7fe5f0146b20 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46140/0 (socket says 172.21.15.46:46140) 2024-04-26T23:44:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 -- 172.21.15.46:0/1075090665 learned_addr learned my addr 172.21.15.46:0/1075090665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 -- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 msgr2=0x7fe5f013fe80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:44:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f013fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 -- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5f0146740 msgr2=0x7fe5f00b22f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5f0146740 0x7fe5f00b22f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.292+0000 7fe5ede00640 1 -- 172.21.15.46:0/1075090665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f01409f0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5ede00640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b450 tx=0x7fe5e000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5e6000640 1 -- 172.21.15.46:0/1075090665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0015020 con 0x7fe5f00cdc10 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5e6000640 1 -- 172.21.15.46:0/1075090665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e00118b0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5e6000640 1 -- 172.21.15.46:0/1075090665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011ca0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f0140cd0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.293+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f015aaa0 con 0x7fe5f00cdc10 2024-04-26T23:44:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.296+0000 7fe5e6000640 1 -- 172.21.15.46:0/1075090665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5e0018020 con 0x7fe5f00cdc10 2024-04-26T23:44:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.297+0000 7fe5e6000640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.297+0000 7fe5e6000640 1 -- 172.21.15.46:0/1075090665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fe5e008bf60 con 0x7fe5f00cdc10 2024-04-26T23:44:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.298+0000 7fe5f7a43740 1 Processor -- start 2024-04-26T23:44:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.298+0000 7fe5f7a43740 1 -- start start 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3d4500 0x56199e3d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e38c940 0x56199e3d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56199e2f0d80 con 0x56199e3b6c60 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56199e28d0d0 con 0x56199e38c940 2024-04-26T23:44:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56199e2ecd40 con 0x56199e3d4500 2024-04-26T23:44:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46154/0 (socket says 172.21.15.46:46154) 2024-04-26T23:44:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 -- 172.21.15.46:0/3803864822 learned_addr learned my addr 172.21.15.46:0/3803864822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 -- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3d4500 msgr2=0x56199e3d4a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3d4500 0x56199e3d4a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 -- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e38c940 msgr2=0x56199e3d2bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e38c940 0x56199e3d2bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.299+0000 7fe5ede00640 1 -- 172.21.15.46:0/3803864822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199e0ceaf0 con 0x56199e3b6c60 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.300+0000 7fe5ede00640 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0062150 tx=0x7fe5e005a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434da2ac2eb15c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.300+0000 7fe5ed400640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.300+0000 7fe5da600640 1 -- 172.21.15.46:0/3803864822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0062680 con 0x56199e3b6c60 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.300+0000 7fe5da600640 1 -- 172.21.15.46:0/3803864822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e0057da0 con 0x56199e3b6c60 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.300+0000 7fe5da600640 1 -- 172.21.15.46:0/3803864822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0058ef0 con 0x56199e3b6c60 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 -- 172.21.15.46:0/3803864822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 msgr2=0x56199e3b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0062150 tx=0x7fe5e005a2e0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 -- 172.21.15.46:0/3803864822 shutdown_connections 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e38c940 0x56199e3d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3d4500 0x56199e3d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/3803864822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3b6c60 0x56199e3b7040 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 -- 172.21.15.46:0/3803864822 >> 172.21.15.46:0/3803864822 conn(0x56199e389270 msgr2=0x56199e3d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 -- 172.21.15.46:0/3803864822 shutdown_connections 2024-04-26T23:44:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.301+0000 7fe5f7a43740 1 -- 172.21.15.46:0/3803864822 wait complete. 2024-04-26T23:44:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.302+0000 7fe5f7a43740 1 -- start start 2024-04-26T23:44:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.302+0000 7fe5f7a43740 1 Processor -- start 2024-04-26T23:44:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e3d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56199e38d6e0 con 0x56199e3d4500 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56199e277bb0 con 0x56199e3e1e10 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56199e3871a0 con 0x56199e3b6c60 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5f7a43740 10 client.0 _ll_register_callbacks cb 0x56199e298f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46160/0 (socket says 172.21.15.46:46160) 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.303+0000 7fe5ed400640 1 -- 172.21.15.46:0/219052296 learned_addr learned my addr 172.21.15.46:0/219052296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 -- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 msgr2=0x56199e3d0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e3d0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 -- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 msgr2=0x56199e3ba3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3ba3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 -- 172.21.15.46:0/219052296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56199dfc23e0 con 0x56199e3d4500 2024-04-26T23:44:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5ed400640 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc014e50 tx=0x7fe5dc012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2da7e4a61cc1a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc014250 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fe5dc013400 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e0057da0 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe5dc005960 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5f7a43740 10 client.7497 fetch_fsmap learned FSMap version 66 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.304+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x56199e12f280 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e0057da0 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fe5dc01b030 con 0x56199e3d4500 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 10 client.7497 fetch_fsmap finished waiting for FSMap version 66 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 10 client.7497 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 20 client.7497 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3bab00 0x56199e3b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 4 client.7497 mds_command: new command op to 7416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:44:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 --> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56199e2fe990 con 0x56199e3bab00 2024-04-26T23:44:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.305+0000 7fe5ede00640 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3bab00 0x56199e3b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.306+0000 7fe5ede00640 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3bab00 0x56199e3b41f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db443cf94cf0be06 server_cookie=9345a018330f3331 in_seq=0 out_seq=0 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.306+0000 7fe5cea00640 10 client.7497 ms_handle_connect on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5cea00640 1 -- 172.21.15.46:0/219052296 <== mds.0 v2:172.21.15.46:6832/357252295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56199e2fe990 con 0x56199e3bab00 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5cea00640 10 client.7497 handle_command_reply: tid=0 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 client.7497 shutdown 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 msgr2=0x56199e3b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc014e50 tx=0x7fe5dc012760 comp rx=0 tx=0).stop 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 shutdown_connections 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3bab00 0x56199e3b41f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b8ce0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/219052296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e3d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5cea00640 0 client.7497 ms_handle_reset on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 >> 172.21.15.46:0/219052296 conn(0x56199e38c940 msgr2=0x56199e3d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 shutdown_connections 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 1 -- 172.21.15.46:0/219052296 wait complete. 2024-04-26T23:44:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.307+0000 7fe5f7a43740 20 client.7497 trim_cache size 0 max 16384 2024-04-26T23:44:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.327+0000 7fe5f7a43740 1 Processor -- start 2024-04-26T23:44:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.327+0000 7fe5f7a43740 1 -- start start 2024-04-26T23:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3b6c60 0x56199e3f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3e1e10 0x56199e3c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01fda60 con 0x56199e3d4500 2024-04-26T23:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01fcaa0 con 0x56199e3b6c60 2024-04-26T23:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f01fbed0 con 0x56199e3e1e10 2024-04-26T23:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46164/0 (socket says 172.21.15.46:46164) 2024-04-26T23:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.328+0000 7fe5ed400640 1 -- 172.21.15.46:0/4120194722 learned_addr learned my addr 172.21.15.46:0/4120194722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 -- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3e1e10 msgr2=0x56199e3c9390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3e1e10 0x56199e3c9390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 -- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3b6c60 msgr2=0x56199e3f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3b6c60 0x56199e3f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 -- 172.21.15.46:0/4120194722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199df9b560 con 0x56199e3d4500 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5ed400640 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005e10 tx=0x7fe5dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9290ce347799381 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.329+0000 7fe5d9c00640 1 -- 172.21.15.46:0/4120194722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc031070 con 0x56199e3d4500 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5d9c00640 1 -- 172.21.15.46:0/4120194722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5dc021070 con 0x56199e3d4500 2024-04-26T23:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5d9c00640 1 -- 172.21.15.46:0/4120194722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0368a0 con 0x56199e3d4500 2024-04-26T23:44:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 -- 172.21.15.46:0/4120194722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 msgr2=0x56199e3b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005e10 tx=0x7fe5dc017000 comp rx=0 tx=0).stop 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 -- 172.21.15.46:0/4120194722 shutdown_connections 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3e1e10 0x56199e3c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e3b41d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/4120194722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3b6c60 0x56199e3f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 -- 172.21.15.46:0/4120194722 >> 172.21.15.46:0/4120194722 conn(0x56199e38c940 msgr2=0x56199e38be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 -- 172.21.15.46:0/4120194722 shutdown_connections 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.330+0000 7fe5f7a43740 1 -- 172.21.15.46:0/4120194722 wait complete. 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.331+0000 7fe5f7a43740 1 -- start start 2024-04-26T23:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.331+0000 7fe5f7a43740 1 Processor -- start 2024-04-26T23:44:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e38cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3c7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56199e3c98d0 con 0x56199e3d4500 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01fc450 con 0x56199e3e1e10 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56199e3d3220 con 0x56199e3b6c60 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5f7a43740 10 client.0 _ll_register_callbacks cb 0x7fe5dc006db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.332+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46172/0 (socket says 172.21.15.46:46172) 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 -- 172.21.15.46:0/860419544 learned_addr learned my addr 172.21.15.46:0/860419544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 -- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 msgr2=0x56199e38cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e38cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 -- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 msgr2=0x56199e3c7400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3c7400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 -- 172.21.15.46:0/860419544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56199e3c9510 con 0x56199e3d4500 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5ed400640 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc016010 tx=0x7fe5dc00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480fae8edadcb134 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.333+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc031070 con 0x56199e3d4500 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc021070 con 0x56199e3d4500 2024-04-26T23:44:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fe5dc003a00 con 0x56199e3d4500 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe5dc046050 con 0x56199e3d4500 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 10 client.7503 fetch_fsmap learned FSMap version 66 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x56199e38b500 con 0x56199e3d4500 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5dc021070 con 0x56199e3d4500 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fe5dc049020 con 0x56199e3d4500 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 10 client.7503 fetch_fsmap finished waiting for FSMap version 66 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 10 client.7503 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 20 client.7503 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3e12f0 0x56199e3c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 4 client.7503 mds_command: new command op to 7416 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.334+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 --> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) -- 0x56199e2f8800 con 0x56199e3e12f0 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.335+0000 7fe5ede00640 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3e12f0 0x56199e3c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.335+0000 7fe5ede00640 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3e12f0 0x56199e3c7940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56e031c1d66f0f9 server_cookie=40e31439f4fe8e46 in_seq=0 out_seq=0 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.335+0000 7fe5cea00640 10 client.7503 ms_handle_connect on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5cea00640 1 -- 172.21.15.46:0/860419544 <== mds.0 v2:172.21.15.46:6832/357252295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56199e2f8800 con 0x56199e3e12f0 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5cea00640 10 client.7503 handle_command_reply: tid=0 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 client.7503 shutdown 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 msgr2=0x56199e38d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc016010 tx=0x7fe5dc00e000 comp rx=0 tx=0).stop 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 shutdown_connections 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:6832/357252295,v1:172.21.15.46:6833/357252295] conn(0x56199e3e12f0 0x56199e3c7940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56199e3e1e10 0x56199e3c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56199e3d4500 0x56199e38d260 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 --2- 172.21.15.46:0/860419544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56199e3b6c60 0x56199e38cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 >> 172.21.15.46:0/860419544 conn(0x56199e38c940 msgr2=0x56199e3e4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5cea00640 0 client.7503 ms_handle_reset on v2:172.21.15.46:6832/357252295 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 shutdown_connections 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 1 -- 172.21.15.46:0/860419544 wait complete. 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.336+0000 7fe5f7a43740 20 client.7503 trim_cache size 0 max 16384 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "879f2f80-da16-4b18-a833-b2838108f1c9", 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:44:10.343 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:44:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5d006c8f0 msgr2=0x7fe5d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5d006c8f0 0x7fe5d006edb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x56199e389f50 tx=0x7fe5dc006b60 comp rx=0 tx=0).stop 2024-04-26T23:44:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 msgr2=0x7fe5f013f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b450 tx=0x7fe5e000b910 comp rx=0 tx=0).stop 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 shutdown_connections 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5f0146b20 0x7fe5f013fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5f0146740 0x7fe5f00b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 --2- 172.21.15.46:0/1075090665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00cdc10 0x7fe5f013f7d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 >> 172.21.15.46:0/1075090665 conn(0x7fe5f006c770 msgr2=0x7fe5f00ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.337+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 shutdown_connections 2024-04-26T23:44:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:10.338+0000 7fe5ef200640 1 -- 172.21.15.46:0/1075090665 wait complete. 2024-04-26T23:44:10.359 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '879f2f80-da16-4b18-a833-b2838108f1c9', 'mode': 'asynchronous'}' 2024-04-26T23:44:20.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:20.360 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-26T23:44:20.427 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:44:20.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:44:20.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:44:20.643 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:44:20.643 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:44:20.644 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:20.644 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:20.702 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:20.703 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:20.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:20.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.964+0000 7fa4e5a00640 1 Processor -- start 2024-04-26T23:44:20.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.965+0000 7fa4e5a00640 1 -- start start 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4e0162a60 0x7fa4e00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e00d5a90 0x7fa4e00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 0x7fa4e00d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e00d12e0 con 0x7fa4e0162a60 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e00d05f0 con 0x7fa4e00b4c50 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.966+0000 7fa4e5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e00b65e0 con 0x7fa4e00d5a90 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4e0162a60 0x7fa4e00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4e0162a60 0x7fa4e00d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50940/0 (socket says 172.21.15.46:50940) 2024-04-26T23:44:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 -- 172.21.15.46:0/2456242387 learned_addr learned my addr 172.21.15.46:0/2456242387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 -- 172.21.15.46:0/2456242387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e00d5a90 msgr2=0x7fa4e00b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4e5000640 1 --2- 172.21.15.46:0/2456242387 >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 0x7fa4e00d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/2456242387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e00d5a90 0x7fa4e00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 -- 172.21.15.46:0/2456242387 >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 msgr2=0x7fa4e00d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/2456242387 >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 0x7fa4e00d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4dfe00640 1 -- 172.21.15.46:0/2456242387 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00d1530 con 0x7fa4e0162a60 2024-04-26T23:44:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.967+0000 7fa4e5000640 1 --2- 172.21.15.46:0/2456242387 >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 0x7fa4e00d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.968+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/2456242387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e0162a60 0x7fa4e00d0a80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cf8d0 tx=0x7fa4d002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145b9664406f3d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4dea00640 1 -- 172.21.15.46:0/2456242387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0037070 con 0x7fa4e0162a60 2024-04-26T23:44:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4dea00640 1 -- 172.21.15.46:0/2456242387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d002ae00 con 0x7fa4e0162a60 2024-04-26T23:44:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4dea00640 1 -- 172.21.15.46:0/2456242387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0032660 con 0x7fa4e0162a60 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 -- 172.21.15.46:0/2456242387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e0162a60 msgr2=0x7fa4e00d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/2456242387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e0162a60 0x7fa4e00d0a80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cf8d0 tx=0x7fa4d002a0b0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 -- 172.21.15.46:0/2456242387 shutdown_connections 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/2456242387 >> v2:172.21.15.142:3300/0 conn(0x7fa4e00b4c50 0x7fa4e00d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/2456242387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e00d5a90 0x7fa4e00b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/2456242387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e0162a60 0x7fa4e00d0a80 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.969+0000 7fa4e5a00640 1 -- 172.21.15.46:0/2456242387 >> 172.21.15.46:0/2456242387 conn(0x7fa4e00d4fa0 msgr2=0x7fa4e00d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.970+0000 7fa4e5a00640 1 -- 172.21.15.46:0/2456242387 shutdown_connections 2024-04-26T23:44:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.970+0000 7fa4e5a00640 1 -- 172.21.15.46:0/2456242387 wait complete. 2024-04-26T23:44:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.971+0000 7fa4e5a00640 1 Processor -- start 2024-04-26T23:44:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.972+0000 7fa4e5a00640 1 -- start start 2024-04-26T23:44:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.972+0000 7fa4e5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4e5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4e00d5a90 0x7fa4e00b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4e5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e0162a60 0x7fa4e00ba860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4e5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b5280 con 0x7fa4e00b4c50 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4e5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b5100 con 0x7fa4e00d5a90 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4e5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e00b5400 con 0x7fa4e0162a60 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50948/0 (socket says 172.21.15.46:50948) 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 -- 172.21.15.46:0/1518317862 learned_addr learned my addr 172.21.15.46:0/1518317862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 -- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e0162a60 msgr2=0x7fa4e00ba860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e0162a60 0x7fa4e00ba860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 -- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4e00d5a90 msgr2=0x7fa4e00b9050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4e00d5a90 0x7fa4e00b9050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.973+0000 7fa4dfe00640 1 -- 172.21.15.46:0/1518317862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00b97c0 con 0x7fa4e00b4c50 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4dfe00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0000c00 tx=0x7fa4d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0042070 con 0x7fa4e00b4c50 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d003e670 con 0x7fa4e00b4c50 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e00b9aa0 con 0x7fa4e00b4c50 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0037040 con 0x7fa4e00b4c50 2024-04-26T23:44:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.974+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e00ba060 con 0x7fa4e00b4c50 2024-04-26T23:44:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.975+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e0069ac0 con 0x7fa4e00b4c50 2024-04-26T23:44:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.976+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4d003a040 con 0x7fa4e00b4c50 2024-04-26T23:44:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.977+0000 7fa4d7e00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 0x7fa4bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.977+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fa4d002ae00 con 0x7fa4e00b4c50 2024-04-26T23:44:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.979+0000 7fa4df400640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 0x7fa4bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:20.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.980+0000 7fa4df400640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 0x7fa4bc06edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cf650 tx=0x7fa4c8009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:20.982+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4d007a2c0 con 0x7fa4e00b4c50 2024-04-26T23:44:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.186+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4e00d0250 con 0x7fa4e00b4c50 2024-04-26T23:44:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.189+0000 7fa4d7e00640 1 -- 172.21.15.46:0/1518317862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+17040 (secure 0 0 0) 0x7fa4d0079bc0 con 0x7fa4e00b4c50 2024-04-26T23:44:21.194 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:21.194 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:02.386476+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6837/2319300947":"2024-04-27T23:44:02.372699+0000","172.21.15.142:6835/2319300947":"2024-04-27T23:44:02.372699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 msgr2=0x7fa4bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 0x7fa4bc06edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cf650 tx=0x7fa4c8009250 comp rx=0 tx=0).stop 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 msgr2=0x7fa4e00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0000c00 tx=0x7fa4d00046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 shutdown_connections 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa4bc06c8f0 0x7fa4bc06edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4e0162a60 0x7fa4e00ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4e00d5a90 0x7fa4e00b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 --2- 172.21.15.46:0/1518317862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4e00b4c50 0x7fa4e00cd200 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.192+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 >> 172.21.15.46:0/1518317862 conn(0x7fa4e00d4fa0 msgr2=0x7fa4e00b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.193+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 shutdown_connections 2024-04-26T23:44:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.193+0000 7fa4e5a00640 1 -- 172.21.15.46:0/1518317862 wait complete. 2024-04-26T23:44:21.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:43:46.543830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:43:47.711290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:44:21.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:44:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.428+0000 7fd08ce00640 1 Processor -- start 2024-04-26T23:44:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.428+0000 7fd08ce00640 1 -- start start 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.429+0000 7fd08ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd0881640d0 0x7fd0881644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.429+0000 7fd08ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 0x7fd0880cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.429+0000 7fd08ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0881649f0 con 0x7fd088073af0 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.429+0000 7fd08ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd0880cb4f0 con 0x7fd0881640d0 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.429+0000 7fd08ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd088071e80 con 0x7fd0880ce7d0 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.430+0000 7fd07fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd088073af0 0x7fd0880cb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55752/0 (socket says 172.21.15.46:55752) 2024-04-26T23:44:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.430+0000 7fd07fe00640 1 -- 172.21.15.46:0/4159641693 learned_addr learned my addr 172.21.15.46:0/4159641693 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.430+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789090504 0 0) 0x7fd0881649f0 con 0x7fd088073af0 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.430+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd06c003520 con 0x7fd088073af0 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083182327 0 0) 0x7fd06c003520 con 0x7fd088073af0 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 msgr2=0x7fd0880cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 0x7fd0880cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 >> v2:172.21.15.142:3300/0 conn(0x7fd0881640d0 msgr2=0x7fd0881644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 --2- 172.21.15.46:0/4159641693 >> v2:172.21.15.142:3300/0 conn(0x7fd0881640d0 0x7fd0881644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd088072160 con 0x7fd088073af0 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd070001670 con 0x7fd088073af0 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 >> v1:172.21.15.46:6789/0 conn(0x7fd088073af0 legacy=0x7fd0880cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd06c0039b0 0x7fd06c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 0x7fd06c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd06c004e60 0x7fd06c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880cb4f0 con 0x7fd06c0039b0 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd088071e80 con 0x7fd06c004e60 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c005590 con 0x7fd0880ce7d0 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.431+0000 7fd087400640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd06c0039b0 0x7fd06c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.432+0000 7fd087400640 1 -- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 msgr2=0x7fd06c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.432+0000 7fd087400640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd0880ce7d0 0x7fd06c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.432+0000 7fd087400640 1 -- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd06c004e60 msgr2=0x7fd06c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.432+0000 7fd087400640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd06c004e60 0x7fd06c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.432+0000 7fd087400640 1 -- 172.21.15.46:0/4159641693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c003520 con 0x7fd06c0039b0 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.433+0000 7fd087400640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd06c0039b0 0x7fd06c003dd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd078009860 tx=0x7fd06c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f377539570477596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.433+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078037070 con 0x7fd06c0039b0 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.433+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07802aaa0 con 0x7fd06c0039b0 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.433+0000 7fd086a00640 1 -- 172.21.15.46:0/4159641693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078009460 con 0x7fd06c0039b0 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 -- 172.21.15.46:0/4159641693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd06c0039b0 msgr2=0x7fd06c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 --2- 172.21.15.46:0/4159641693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd06c0039b0 0x7fd06c003dd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd078009860 tx=0x7fd06c005830 comp rx=0 tx=0).stop 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd087e00640 1 -- 172.21.15.46:0/4159641693 reap_dead start 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 -- 172.21.15.46:0/4159641693 shutdown_connections 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 -- 172.21.15.46:0/4159641693 >> 172.21.15.46:0/4159641693 conn(0x7fd0880b27b0 msgr2=0x7fd0880b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 -- 172.21.15.46:0/4159641693 shutdown_connections 2024-04-26T23:44:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.434+0000 7fd08ce00640 1 -- 172.21.15.46:0/4159641693 wait complete. 2024-04-26T23:44:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.436+0000 7fd08ce00640 1 Processor -- start 2024-04-26T23:44:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.436+0000 7fd08ce00640 1 -- start start 2024-04-26T23:44:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd06c0039b0 0x7fd08815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08815d310 0x7fd08814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880cc090 con 0x7fd08814e670 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd088073f70 con 0x7fd08815d310 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd08ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd0880cc210 con 0x7fd06c0039b0 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd087e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.437+0000 7fd087e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50968/0 (socket says 172.21.15.46:50968) 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 -- 172.21.15.46:0/3361828399 learned_addr learned my addr 172.21.15.46:0/3361828399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087400640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd06c0039b0 0x7fd08815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd07fe00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08815d310 0x7fd08814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 -- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd06c0039b0 msgr2=0x7fd08815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd06c0039b0 0x7fd08815cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 -- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08815d310 msgr2=0x7fd08814e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08815d310 0x7fd08814e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 -- 172.21.15.46:0/3361828399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0880c9c50 con 0x7fd08814e670 2024-04-26T23:44:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.438+0000 7fd087e00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd08806d380 tx=0x7fd08000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.439+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080017070 con 0x7fd08814e670 2024-04-26T23:44:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.439+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08000a770 con 0x7fd08814e670 2024-04-26T23:44:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.439+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000ab20 con 0x7fd08814e670 2024-04-26T23:44:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.439+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0880c9f30 con 0x7fd08814e670 2024-04-26T23:44:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.439+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0880ca4f0 con 0x7fd08814e670 2024-04-26T23:44:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.440+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0880693b0 con 0x7fd08814e670 2024-04-26T23:44:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.441+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd08001a070 con 0x7fd08814e670 2024-04-26T23:44:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.442+0000 7fd07f400640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 0x7fd06406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.443+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fd08008b2b0 con 0x7fd08814e670 2024-04-26T23:44:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.443+0000 7fd087400640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 0x7fd06406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.444+0000 7fd087400640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 0x7fd06406ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd078034000 tx=0x7fd07802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.447+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd080058ea0 con 0x7fd08814e670 2024-04-26T23:44:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.654+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0880ccd90 con 0x7fd08814e670 2024-04-26T23:44:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.655+0000 7fd07f400640 1 -- 172.21.15.46:0/3361828399 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd08005d1a0 con 0x7fd08814e670 2024-04-26T23:44:21.660 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 msgr2=0x7fd06406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 0x7fd06406ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd078034000 tx=0x7fd07802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 msgr2=0x7fd0880c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd08806d380 tx=0x7fd08000cae0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 shutdown_connections 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd06406c920 0x7fd06406ede0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08814e670 0x7fd0880c8ee0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08815d310 0x7fd08814e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 --2- 172.21.15.46:0/3361828399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd06c0039b0 0x7fd08815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.658+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 >> 172.21.15.46:0/3361828399 conn(0x7fd0880b27b0 msgr2=0x7fd0880b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.659+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 shutdown_connections 2024-04-26T23:44:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.659+0000 7fd08ce00640 1 -- 172.21.15.46:0/3361828399 wait complete. 2024-04-26T23:44:21.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:44:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.894+0000 7f4fa1c00640 1 Processor -- start 2024-04-26T23:44:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.894+0000 7f4fa1c00640 1 -- start start 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4fa1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4fa1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 0x7f4f9c0d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4fa1c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9c0d07d0 con 0x7f4f9c162aa0 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4fa1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4f9c0d0ab0 con 0x7f4f9c1626c0 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4fa1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9c0d0d90 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.895+0000 7f4f9be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44008/0 (socket says 172.21.15.46:44008) 2024-04-26T23:44:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 -- 172.21.15.46:0/1679531677 learned_addr learned my addr 172.21.15.46:0/1679531677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 -- 172.21.15.46:0/1679531677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 msgr2=0x7f4f9c0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9b400640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 0x7f4f9c0d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 0x7f4f9c0d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 -- 172.21.15.46:0/1679531677 >> v1:172.21.15.142:6789/0 conn(0x7f4f9c162aa0 legacy=0x7f4f9c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9be00640 1 -- 172.21.15.46:0/1679531677 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c0d2c80 con 0x7f4f9c1626c0 2024-04-26T23:44:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.896+0000 7f4f9b400640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 0x7f4f9c0d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.897+0000 7f4f9be00640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1420793e6f6e5ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.897+0000 7f4f9aa00640 1 -- 172.21.15.46:0/1679531677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c037070 con 0x7f4f9c1626c0 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4f9aa00640 1 -- 172.21.15.46:0/1679531677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c032440 con 0x7f4f9c1626c0 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4f9aa00640 1 -- 172.21.15.46:0/1679531677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c032810 con 0x7f4f9c1626c0 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 -- 172.21.15.46:0/1679531677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1626c0 msgr2=0x7f4f9c0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c02a200 comp rx=0 tx=0).stop 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 -- 172.21.15.46:0/1679531677 shutdown_connections 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c0b5fb0 0x7f4f9c0d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/1679531677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0b47c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 -- 172.21.15.46:0/1679531677 >> 172.21.15.46:0/1679531677 conn(0x7f4f9c09e2a0 msgr2=0x7f4f9c0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 -- 172.21.15.46:0/1679531677 shutdown_connections 2024-04-26T23:44:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.898+0000 7f4fa1c00640 1 -- 172.21.15.46:0/1679531677 wait complete. 2024-04-26T23:44:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.900+0000 7f4fa1c00640 1 Processor -- start 2024-04-26T23:44:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.900+0000 7f4fa1c00640 1 -- start start 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 0x7f4f9c0b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0d2190 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0d2010 con 0x7f4f9c1626c0 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.901+0000 7f4fa1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9c0d31a0 con 0x7f4f9c162aa0 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44014/0 (socket says 172.21.15.46:44014) 2024-04-26T23:44:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 -- 172.21.15.46:0/3012312995 learned_addr learned my addr 172.21.15.46:0/3012312995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4fa1200640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 0x7f4f9c0b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9b400640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 -- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 msgr2=0x7f4f9c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 0x7f4f9c0b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 -- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f9c1626c0 msgr2=0x7f4f9c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4f9be00640 1 -- 172.21.15.46:0/3012312995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c0cec20 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.902+0000 7f4fa1200640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 0x7f4f9c0b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4f9be00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c041020 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c030970 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c037040 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c1f83c0 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.903+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c1f8980 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.904+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9c0693b0 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.905+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f8c044020 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.906+0000 7f4f93e00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.906+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f4f8c0ac770 con 0x7f4f9c0b5fb0 2024-04-26T23:44:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.907+0000 7f4f9b400640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.908+0000 7f4f9b400640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c1618b0 tx=0x7f4f84008520 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:21.911+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f8c04b030 con 0x7f4f9c0b5fb0 2024-04-26T23:44:22.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.116+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f9c0d25b0 con 0x7f4f9c0b5fb0 2024-04-26T23:44:22.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.116+0000 7f4f93e00640 1 -- 172.21.15.46:0/3012312995 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f4f8c07a920 con 0x7f4f9c0b5fb0 2024-04-26T23:44:22.121 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 msgr2=0x7f4f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c1618b0 tx=0x7f4f84008520 comp rx=0 tx=0).stop 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 msgr2=0x7f4f9c1f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 shutdown_connections 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f9c162aa0 0x7f4f9c0b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f9c1626c0 0x7f4f9c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 --2- 172.21.15.46:0/3012312995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c0b5fb0 0x7f4f9c1f4470 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 >> 172.21.15.46:0/3012312995 conn(0x7f4f9c09e2a0 msgr2=0x7f4f9c0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.119+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 shutdown_connections 2024-04-26T23:44:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.120+0000 7f4fa1c00640 1 -- 172.21.15.46:0/3012312995 wait complete. 2024-04-26T23:44:22.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:44:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.357+0000 7fa415e00640 1 Processor -- start 2024-04-26T23:44:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.358+0000 7fa415e00640 1 -- start start 2024-04-26T23:44:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.359+0000 7fa415e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4100b53b0 0x7fa41015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.359+0000 7fa415e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 0x7fa41006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.359+0000 7fa415e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4100d07e0 con 0x7fa41015d730 2024-04-26T23:44:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.359+0000 7fa415e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa41006eb90 con 0x7fa4100b53b0 2024-04-26T23:44:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.359+0000 7fa415e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa4100b4310 con 0x7fa41015e050 2024-04-26T23:44:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4100b53b0 0x7fa41015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa41015d730 0x7fa4100d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41534/0 (socket says 172.21.15.46:41534) 2024-04-26T23:44:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40ea00640 1 -- 172.21.15.46:0/8640476 learned_addr learned my addr 172.21.15.46:0/8640476 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40f400640 1 --2- 172.21.15.46:0/8640476 >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 0x7fa41006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40e000640 1 -- 172.21.15.46:0/8640476 >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 msgr2=0x7fa41006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40e000640 1 --2- 172.21.15.46:0/8640476 >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 0x7fa41006dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40e000640 1 -- 172.21.15.46:0/8640476 >> v1:172.21.15.46:6789/0 conn(0x7fa41015d730 legacy=0x7fa4100d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40e000640 1 -- 172.21.15.46:0/8640476 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100b45f0 con 0x7fa4100b53b0 2024-04-26T23:44:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.360+0000 7fa40f400640 1 --2- 172.21.15.46:0/8640476 >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 0x7fa41006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.361+0000 7fa40e000640 1 --2- 172.21.15.46:0/8640476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015db10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa410074180 tx=0x7fa3f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f126c6948ceaab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa40d600640 1 -- 172.21.15.46:0/8640476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8037070 con 0x7fa4100b53b0 2024-04-26T23:44:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa40d600640 1 -- 172.21.15.46:0/8640476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f8032440 con 0x7fa4100b53b0 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa40d600640 1 -- 172.21.15.46:0/8640476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8032810 con 0x7fa4100b53b0 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 -- 172.21.15.46:0/8640476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 msgr2=0x7fa41015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 --2- 172.21.15.46:0/8640476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015db10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa410074180 tx=0x7fa3f802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 -- 172.21.15.46:0/8640476 shutdown_connections 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 --2- 172.21.15.46:0/8640476 >> v2:172.21.15.142:3300/0 conn(0x7fa41015e050 0x7fa41006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 --2- 172.21.15.46:0/8640476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015db10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.362+0000 7fa415e00640 1 -- 172.21.15.46:0/8640476 >> 172.21.15.46:0/8640476 conn(0x7fa41006c980 msgr2=0x7fa4100b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.363+0000 7fa415e00640 1 -- 172.21.15.46:0/8640476 shutdown_connections 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.363+0000 7fa415e00640 1 -- 172.21.15.46:0/8640476 wait complete. 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.364+0000 7fa415e00640 1 Processor -- start 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.364+0000 7fa415e00640 1 -- start start 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa41015d730 0x7fa41013ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa41015e050 0x7fa4100735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b4e90 con 0x7fa4100b53b0 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b4bb0 con 0x7fa41015e050 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.365+0000 7fa415e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa41015e460 con 0x7fa41015d730 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44034/0 (socket says 172.21.15.46:44034) 2024-04-26T23:44:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 -- 172.21.15.46:0/2401459064 learned_addr learned my addr 172.21.15.46:0/2401459064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40e000640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa41015d730 0x7fa41013ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 -- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa41015d730 msgr2=0x7fa41013ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40f400640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa41015e050 0x7fa4100735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa41015d730 0x7fa41013ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 -- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa41015e050 msgr2=0x7fa4100735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa41015e050 0x7fa4100735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.366+0000 7fa40ea00640 1 -- 172.21.15.46:0/2401459064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410142e60 con 0x7fa4100b53b0 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa40ea00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa40000cb10 tx=0x7fa40000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400017070 con 0x7fa4100b53b0 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40000ac70 con 0x7fa4100b53b0 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400012620 con 0x7fa4100b53b0 2024-04-26T23:44:22.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41013b1d0 con 0x7fa4100b53b0 2024-04-26T23:44:22.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.367+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41013b790 con 0x7fa4100b53b0 2024-04-26T23:44:22.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.368+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410069530 con 0x7fa4100b53b0 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.369+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40001a070 con 0x7fa4100b53b0 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.370+0000 7fa407400640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 0x7fa3ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.370+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fa40008bb30 con 0x7fa4100b53b0 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.371+0000 7fa40e000640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 0x7fa3ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.372+0000 7fa40e000640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 0x7fa3ec06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8034000 tx=0x7fa3f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.375+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa400059d70 con 0x7fa4100b53b0 2024-04-26T23:44:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.580+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa410076550 con 0x7fa4100b53b0 2024-04-26T23:44:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.581+0000 7fa407400640 1 -- 172.21.15.46:0/2401459064 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fa400059670 con 0x7fa4100b53b0 2024-04-26T23:44:22.585 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:44:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 msgr2=0x7fa3ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 0x7fa3ec06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8034000 tx=0x7fa3f802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 msgr2=0x7fa41015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa40000cb10 tx=0x7fa40000a060 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 shutdown_connections 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa3ec06c8f0 0x7fa3ec06edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa41015e050 0x7fa4100735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa41015d730 0x7fa41013ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 --2- 172.21.15.46:0/2401459064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4100b53b0 0x7fa41015aa10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 >> 172.21.15.46:0/2401459064 conn(0x7fa41006c980 msgr2=0x7fa4100b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 shutdown_connections 2024-04-26T23:44:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.584+0000 7fa415e00640 1 -- 172.21.15.46:0/2401459064 wait complete. 2024-04-26T23:44:22.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:44:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.821+0000 7f792f200640 1 Processor -- start 2024-04-26T23:44:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.821+0000 7f792f200640 1 -- start start 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79300720e0 0x7f79300cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7930160910 0x7f7930160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7930073ad0 con 0x7f79300720e0 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7930146a90 con 0x7f7930160910 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7930073d70 con 0x7f79301466b0 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79300720e0 0x7f79300cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79300720e0 0x7f79300cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44038/0 (socket says 172.21.15.46:44038) 2024-04-26T23:44:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.823+0000 7f792d400640 1 -- 172.21.15.46:0/870441666 learned_addr learned my addr 172.21.15.46:0/870441666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.824+0000 7f792d400640 1 -- 172.21.15.46:0/870441666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 msgr2=0x7f793006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.824+0000 7f792d400640 1 --2- 172.21.15.46:0/870441666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.824+0000 7f792d400640 1 -- 172.21.15.46:0/870441666 >> v2:172.21.15.142:3300/0 conn(0x7f7930160910 msgr2=0x7f7930160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.824+0000 7f792d400640 1 --2- 172.21.15.46:0/870441666 >> v2:172.21.15.142:3300/0 conn(0x7f7930160910 0x7f7930160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.824+0000 7f792d400640 1 -- 172.21.15.46:0/870441666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7930074050 con 0x7f79300720e0 2024-04-26T23:44:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.825+0000 7f792d400640 1 --2- 172.21.15.46:0/870441666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300cd860 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f79300cc920 tx=0x7f791802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63127d53997ba281 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.825+0000 7f792ca00640 1 -- 172.21.15.46:0/870441666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918037070 con 0x7f79300720e0 2024-04-26T23:44:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.825+0000 7f792ca00640 1 -- 172.21.15.46:0/870441666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7918032440 con 0x7f79300720e0 2024-04-26T23:44:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.825+0000 7f792ca00640 1 -- 172.21.15.46:0/870441666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918032810 con 0x7f79300720e0 2024-04-26T23:44:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 -- 172.21.15.46:0/870441666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 msgr2=0x7f79300cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 --2- 172.21.15.46:0/870441666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300cd860 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f79300cc920 tx=0x7f791802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 -- 172.21.15.46:0/870441666 shutdown_connections 2024-04-26T23:44:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 --2- 172.21.15.46:0/870441666 >> v2:172.21.15.142:3300/0 conn(0x7f7930160910 0x7f7930160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 --2- 172.21.15.46:0/870441666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300cd860 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 --2- 172.21.15.46:0/870441666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 -- 172.21.15.46:0/870441666 >> 172.21.15.46:0/870441666 conn(0x7f79300b26d0 msgr2=0x7f79300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 -- 172.21.15.46:0/870441666 shutdown_connections 2024-04-26T23:44:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.826+0000 7f792f200640 1 -- 172.21.15.46:0/870441666 wait complete. 2024-04-26T23:44:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.828+0000 7f792f200640 1 Processor -- start 2024-04-26T23:44:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.828+0000 7f792f200640 1 -- start start 2024-04-26T23:44:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7930160910 0x7f79300ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f793006e3c0 con 0x7f79300720e0 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79300cb470 con 0x7f7930160910 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79300cdda0 con 0x7f79301466b0 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44040/0 (socket says 172.21.15.46:44040) 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.829+0000 7f792de00640 1 -- 172.21.15.46:0/2101744091 learned_addr learned my addr 172.21.15.46:0/2101744091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792e800640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7930160910 0x7f79300ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 -- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 msgr2=0x7f793015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 -- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7930160910 msgr2=0x7f79300ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7930160910 0x7f79300ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 -- 172.21.15.46:0/2101744091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300caa40 con 0x7f79300720e0 2024-04-26T23:44:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792de00640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f792000b410 tx=0x7f792000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920015020 con 0x7f79300720e0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7920011850 con 0x7f79300720e0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920011c40 con 0x7f79300720e0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.830+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79300c8b80 con 0x7f79300720e0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.831+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79300c9140 con 0x7f79300720e0 2024-04-26T23:44:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.832+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79300693b0 con 0x7f79300720e0 2024-04-26T23:44:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.833+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7920018020 con 0x7f79300720e0 2024-04-26T23:44:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.834+0000 7f7926000640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 0x7f790c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.834+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f792008bbc0 con 0x7f79300720e0 2024-04-26T23:44:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.834+0000 7f792d400640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 0x7f790c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.836+0000 7f792d400640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 0x7f790c06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7918034000 tx=0x7f791802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:22.839+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f792001f020 con 0x7f79300720e0 2024-04-26T23:44:23.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.060+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f79300c7180 con 0x7f79300720e0 2024-04-26T23:44:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.061+0000 7f7926000640 1 -- 172.21.15.46:0/2101744091 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) ==== 80+0+124 (secure 0 0 0) 0x7f79200172c0 con 0x7f79300720e0 2024-04-26T23:44:23.065 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.142:6837/2319300947 2024-04-27T23:44:02.372699+0000 2024-04-26T23:44:23.066 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.142:6835/2319300947 2024-04-27T23:44:02.372699+0000 2024-04-26T23:44:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-26T23:44:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 msgr2=0x7f790c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 0x7f790c06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7918034000 tx=0x7f791802c000 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 msgr2=0x7f79300c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f792000b410 tx=0x7f792000b8d0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 shutdown_connections 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f790c06c8f0 0x7f790c06edb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7930160910 0x7f79300ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79301466b0 0x7f793015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 --2- 172.21.15.46:0/2101744091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79300720e0 0x7f79300c8530 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 >> 172.21.15.46:0/2101744091 conn(0x7f79300b26d0 msgr2=0x7f7930142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 shutdown_connections 2024-04-26T23:44:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.064+0000 7f792f200640 1 -- 172.21.15.46:0/2101744091 wait complete. 2024-04-26T23:44:23.089 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:44:23.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:23.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:23.190 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:44:23.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:23.190 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:44:23.241 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:44:23.242 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:44:23.242 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:23.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:44:23.301 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:44:23.301 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:23.301 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:23.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:44:23.359 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:44:23.359 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:44:23.359 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:23.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:44:23.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:23.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.633+0000 7f05dee00640 1 Processor -- start 2024-04-26T23:44:23.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.634+0000 7f05dee00640 1 -- start start 2024-04-26T23:44:23.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.635+0000 7f05dee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.635+0000 7f05dee00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05d806c890 0x7f05d8162d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.635+0000 7f05dee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d8163760 con 0x7f05d81632d0 2024-04-26T23:44:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.635+0000 7f05dee00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f05d8163cf0 con 0x7f05d806c890 2024-04-26T23:44:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.635+0000 7f05dee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05d806ccf0 con 0x7f05d80ccb20 2024-04-26T23:44:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.636+0000 7f05dda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.636+0000 7f05dda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:47896/0 (socket says 172.21.15.46:47896) 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.636+0000 7f05dda00640 1 -- 172.21.15.46:0/1364185655 learned_addr learned my addr 172.21.15.46:0/1364185655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.636+0000 7f05dd000640 1 --2- 172.21.15.46:0/1364185655 >> v2:172.21.15.142:3300/0 conn(0x7f05d806c890 0x7f05d8162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.637+0000 7f05dda00640 1 -- 172.21.15.46:0/1364185655 >> v2:172.21.15.142:3300/0 conn(0x7f05d806c890 msgr2=0x7f05d8162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.637+0000 7f05dda00640 1 --2- 172.21.15.46:0/1364185655 >> v2:172.21.15.142:3300/0 conn(0x7f05d806c890 0x7f05d8162d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.637+0000 7f05dda00640 1 -- 172.21.15.46:0/1364185655 >> v1:172.21.15.142:6789/0 conn(0x7f05d81632d0 legacy=0x7f05d815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.637+0000 7f05dda00640 1 -- 172.21.15.46:0/1364185655 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d80d19a0 con 0x7f05d80ccb20 2024-04-26T23:44:23.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.638+0000 7f05dda00640 1 --2- 172.21.15.46:0/1364185655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009870 tx=0x7f05cc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a4dc7b6c7ba365 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:23.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.639+0000 7f05d7e00640 1 -- 172.21.15.46:0/1364185655 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc037070 con 0x7f05d80ccb20 2024-04-26T23:44:23.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.639+0000 7f05d7e00640 1 -- 172.21.15.46:0/1364185655 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05cc032440 con 0x7f05d80ccb20 2024-04-26T23:44:23.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.639+0000 7f05d7e00640 1 -- 172.21.15.46:0/1364185655 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc032810 con 0x7f05d80ccb20 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 -- 172.21.15.46:0/1364185655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 msgr2=0x7f05d806c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 --2- 172.21.15.46:0/1364185655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009870 tx=0x7f05cc02a250 comp rx=0 tx=0).stop 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 -- 172.21.15.46:0/1364185655 shutdown_connections 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 --2- 172.21.15.46:0/1364185655 >> v2:172.21.15.142:3300/0 conn(0x7f05d806c890 0x7f05d8162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 --2- 172.21.15.46:0/1364185655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d806c350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 -- 172.21.15.46:0/1364185655 >> 172.21.15.46:0/1364185655 conn(0x7f05d80d1c30 msgr2=0x7f05d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.640+0000 7f05dee00640 1 -- 172.21.15.46:0/1364185655 shutdown_connections 2024-04-26T23:44:23.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.641+0000 7f05dee00640 1 -- 172.21.15.46:0/1364185655 wait complete. 2024-04-26T23:44:23.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.643+0000 7f05dee00640 1 Processor -- start 2024-04-26T23:44:23.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.643+0000 7f05dee00640 1 -- start start 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 0x7f05d80b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80d0190 con 0x7f05d806c890 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80cfeb0 con 0x7f05d81632d0 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05d80d0470 con 0x7f05d80ccb20 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.644+0000 7f05dda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44054/0 (socket says 172.21.15.46:44054) 2024-04-26T23:44:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 -- 172.21.15.46:0/3736948004 learned_addr learned my addr 172.21.15.46:0/3736948004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:23.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dd000640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 -- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 msgr2=0x7f05d80705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d80705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 -- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 msgr2=0x7f05d80b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05de400640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 0x7f05d80b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 0x7f05d80b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 -- 172.21.15.46:0/3736948004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d80b5830 con 0x7f05d806c890 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dd000640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d80705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05de400640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 0x7f05d80b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.645+0000 7f05dda00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009990 tx=0x7f05cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.646+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc037040 con 0x7f05d806c890 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.646+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05cc042070 con 0x7f05d806c890 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.646+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc030940 con 0x7f05d806c890 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.646+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d80b5b10 con 0x7f05d806c890 2024-04-26T23:44:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.646+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d80b60d0 con 0x7f05d806c890 2024-04-26T23:44:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.647+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d80693b0 con 0x7f05d806c890 2024-04-26T23:44:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.648+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05cc030ae0 con 0x7f05d806c890 2024-04-26T23:44:23.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.649+0000 7f05d5600640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:23.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.649+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f05cc0ac5f0 con 0x7f05d806c890 2024-04-26T23:44:23.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.650+0000 7f05dd000640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.651+0000 7f05dd000640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 0x7f05b806edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f05d80c8cb0 tx=0x7f05c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.653+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05cc07a7a0 con 0x7f05d806c890 2024-04-26T23:44:23.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.902+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d8160930 con 0x7f05d806c890 2024-04-26T23:44:23.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.903+0000 7f05d5600640 1 -- 172.21.15.46:0/3736948004 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7f05cc07a0a0 con 0x7f05d806c890 2024-04-26T23:44:23.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:23.909 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:06.419460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7416,"qdb_cluster":[7416]},"id":5}]} 2024-04-26T23:44:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-04-26T23:44:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 msgr2=0x7f05b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 0x7f05b806edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f05d80c8cb0 tx=0x7f05c4008040 comp rx=0 tx=0).stop 2024-04-26T23:44:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 msgr2=0x7f05d8070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009990 tx=0x7f05cc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 shutdown_connections 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05d81632d0 0x7f05d80b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05d80ccb20 0x7f05d80705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 --2- 172.21.15.46:0/3736948004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d806c890 0x7f05d8070060 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 >> 172.21.15.46:0/3736948004 conn(0x7f05d80d1c30 msgr2=0x7f05d80b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 shutdown_connections 2024-04-26T23:44:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:23.907+0000 7f05dee00640 1 -- 172.21.15.46:0/3736948004 wait complete. 2024-04-26T23:44:23.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.140+0000 7ffaefa00640 1 Processor -- start 2024-04-26T23:44:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.141+0000 7ffaefa00640 1 -- start start 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffae806e510 0x7ffae806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 0x7ffae8161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 0x7ffae815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffae806ee30 con 0x7ffae806e510 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffae8161940 con 0x7ffae815d540 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.142+0000 7ffaefa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffae8071f80 con 0x7ffae8160bd0 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffae806e510 0x7ffae806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffae806e510 0x7ffae806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44066/0 (socket says 172.21.15.46:44066) 2024-04-26T23:44:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 -- 172.21.15.46:0/4224726371 learned_addr learned my addr 172.21.15.46:0/4224726371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffae7e00640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 0x7ffae8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 -- 172.21.15.46:0/4224726371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 msgr2=0x7ffae8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaeda00640 1 --2- 172.21.15.46:0/4224726371 >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 0x7ffae815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 0x7ffae8161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 -- 172.21.15.46:0/4224726371 >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 msgr2=0x7ffae815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 --2- 172.21.15.46:0/4224726371 >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 0x7ffae815d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaed000640 1 -- 172.21.15.46:0/4224726371 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae8072260 con 0x7ffae806e510 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffae7e00640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 0x7ffae8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.143+0000 7ffaeda00640 1 --2- 172.21.15.46:0/4224726371 >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 0x7ffae815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.144+0000 7ffaed000640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae806e510 0x7ffae806e8f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffae80ce520 tx=0x7ffad802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece634f447a7efaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.145+0000 7ffae7400640 1 -- 172.21.15.46:0/4224726371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8037070 con 0x7ffae806e510 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.145+0000 7ffae7400640 1 -- 172.21.15.46:0/4224726371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad8032440 con 0x7ffae806e510 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.145+0000 7ffae7400640 1 -- 172.21.15.46:0/4224726371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8032810 con 0x7ffae806e510 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 -- 172.21.15.46:0/4224726371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae806e510 msgr2=0x7ffae806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae806e510 0x7ffae806e8f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffae80ce520 tx=0x7ffad802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 -- 172.21.15.46:0/4224726371 shutdown_connections 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 --2- 172.21.15.46:0/4224726371 >> v2:172.21.15.142:3300/0 conn(0x7ffae815d540 0x7ffae815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae8160bd0 0x7ffae8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 --2- 172.21.15.46:0/4224726371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae806e510 0x7ffae806e8f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 -- 172.21.15.46:0/4224726371 >> 172.21.15.46:0/4224726371 conn(0x7ffae806c530 msgr2=0x7ffae806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 -- 172.21.15.46:0/4224726371 shutdown_connections 2024-04-26T23:44:24.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.146+0000 7ffaefa00640 1 -- 172.21.15.46:0/4224726371 wait complete. 2024-04-26T23:44:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.148+0000 7ffaefa00640 1 Processor -- start 2024-04-26T23:44:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.148+0000 7ffaefa00640 1 -- start start 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae806e510 0x7ffae80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 0x7ffae815f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8161570 con 0x7ffae8160bd0 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8072820 con 0x7ffae815d540 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaefa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffae81616f0 con 0x7ffae806e510 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaeda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaeda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44082/0 (socket says 172.21.15.46:44082) 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.149+0000 7ffaeda00640 1 -- 172.21.15.46:0/3126889150 learned_addr learned my addr 172.21.15.46:0/3126889150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffae7e00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 0x7ffae815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaed000640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae806e510 0x7ffae80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 -- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae806e510 msgr2=0x7ffae80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae806e510 0x7ffae80b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 -- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 msgr2=0x7ffae815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 0x7ffae815f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 -- 172.21.15.46:0/3126889150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80b3830 con 0x7ffae8160bd0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffae7e00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 0x7ffae815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaeda00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffae000afd0 tx=0x7ffae000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0015020 con 0x7ffae8160bd0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0004440 con 0x7ffae8160bd0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0003850 con 0x7ffae8160bd0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.150+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae80b22b0 con 0x7ffae8160bd0 2024-04-26T23:44:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.151+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae80b2af0 con 0x7ffae8160bd0 2024-04-26T23:44:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.152+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae8069530 con 0x7ffae8160bd0 2024-04-26T23:44:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.153+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffae0018020 con 0x7ffae8160bd0 2024-04-26T23:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.154+0000 7ffae4c00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.155+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7ffae008bda0 con 0x7ffae8160bd0 2024-04-26T23:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.155+0000 7ffaed000640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.156+0000 7ffaed000640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 0x7ffac806edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffad8034000 tx=0x7ffad802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.160+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffae001f020 con 0x7ffae8160bd0 2024-04-26T23:44:24.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.364+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffae806fb00 con 0x7ffae8160bd0 2024-04-26T23:44:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.366+0000 7ffae4c00640 1 -- 172.21.15.46:0/3126889150 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+17040 (secure 0 0 0) 0x7ffae80b22b0 con 0x7ffae8160bd0 2024-04-26T23:44:24.371 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:24.371 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:02.386476+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6837/2319300947":"2024-04-27T23:44:02.372699+0000","172.21.15.142:6835/2319300947":"2024-04-27T23:44:02.372699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 msgr2=0x7ffac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 0x7ffac806edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffad8034000 tx=0x7ffad802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 msgr2=0x7ffae8145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffae000afd0 tx=0x7ffae000ca20 comp rx=0 tx=0).stop 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 shutdown_connections 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffae8160bd0 0x7ffae8145c30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffae815d540 0x7ffae815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 --2- 172.21.15.46:0/3126889150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffae806e510 0x7ffae80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 >> 172.21.15.46:0/3126889150 conn(0x7ffae806c530 msgr2=0x7ffae815d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 shutdown_connections 2024-04-26T23:44:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.369+0000 7ffaefa00640 1 -- 172.21.15.46:0/3126889150 wait complete. 2024-04-26T23:44:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:43:46.543830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:43:47.711290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:44:24.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:44:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.608+0000 7fd16fe00640 1 Processor -- start 2024-04-26T23:44:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.608+0000 7fd16fe00640 1 -- start start 2024-04-26T23:44:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.609+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.609+0000 7fd16fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 0x7fd1700d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.609+0000 7fd16fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1700d07d0 con 0x7fd170162aa0 2024-04-26T23:44:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.609+0000 7fd16fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd1700d0ab0 con 0x7fd1700b5fb0 2024-04-26T23:44:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.609+0000 7fd16fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1700d0d90 con 0x7fd1701626c0 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd170162aa0 0x7fd1700d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41536/0 (socket says 172.21.15.46:41536) 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16f400640 1 -- 172.21.15.46:0/2387185959 learned_addr learned my addr 172.21.15.46:0/2387185959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16e000640 1 --2- 172.21.15.46:0/2387185959 >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 0x7fd1700d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16ea00640 1 --2- 172.21.15.46:0/2387185959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16d600640 1 -- 172.21.15.46:0/2387185959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617195746 0 0) 0x7fd1700d07d0 con 0x7fd170162aa0 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.610+0000 7fd16d600640 1 -- 172.21.15.46:0/2387185959 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd154003520 con 0x7fd170162aa0 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.611+0000 7fd16ea00640 1 -- 172.21.15.46:0/2387185959 >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 msgr2=0x7fd1700d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.611+0000 7fd16ea00640 1 --2- 172.21.15.46:0/2387185959 >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 0x7fd1700d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.611+0000 7fd16ea00640 1 -- 172.21.15.46:0/2387185959 >> v1:172.21.15.46:6789/0 conn(0x7fd170162aa0 legacy=0x7fd1700d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.611+0000 7fd16ea00640 1 -- 172.21.15.46:0/2387185959 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1700d2c80 con 0x7fd1701626c0 2024-04-26T23:44:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.611+0000 7fd16e000640 1 --2- 172.21.15.46:0/2387185959 >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 0x7fd1700d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:44:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.612+0000 7fd16ea00640 1 --2- 172.21.15.46:0/2387185959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700b47c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd160009d90 tx=0x7fd16002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc063ad40a11879 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.612+0000 7fd16d600640 1 -- 172.21.15.46:0/2387185959 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160007740 con 0x7fd1701626c0 2024-04-26T23:44:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.612+0000 7fd16d600640 1 -- 172.21.15.46:0/2387185959 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1600078e0 con 0x7fd1701626c0 2024-04-26T23:44:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.612+0000 7fd16d600640 1 -- 172.21.15.46:0/2387185959 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160007cb0 con 0x7fd1701626c0 2024-04-26T23:44:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 -- 172.21.15.46:0/2387185959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 msgr2=0x7fd1700b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 --2- 172.21.15.46:0/2387185959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700b47c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd160009d90 tx=0x7fd16002a660 comp rx=0 tx=0).stop 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 -- 172.21.15.46:0/2387185959 shutdown_connections 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 --2- 172.21.15.46:0/2387185959 >> v2:172.21.15.142:3300/0 conn(0x7fd1700b5fb0 0x7fd1700d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 --2- 172.21.15.46:0/2387185959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700b47c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 -- 172.21.15.46:0/2387185959 >> 172.21.15.46:0/2387185959 conn(0x7fd17009e2a0 msgr2=0x7fd1700d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 -- 172.21.15.46:0/2387185959 shutdown_connections 2024-04-26T23:44:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.613+0000 7fd16fe00640 1 -- 172.21.15.46:0/2387185959 wait complete. 2024-04-26T23:44:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.615+0000 7fd16fe00640 1 Processor -- start 2024-04-26T23:44:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.615+0000 7fd16fe00640 1 -- start start 2024-04-26T23:44:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1700b5fb0 0x7fd1700a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1700d2190 con 0x7fd170162aa0 2024-04-26T23:44:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1700d2010 con 0x7fd1700b5fb0 2024-04-26T23:44:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.616+0000 7fd16fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1700d31a0 con 0x7fd1701626c0 2024-04-26T23:44:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44088/0 (socket says 172.21.15.46:44088) 2024-04-26T23:44:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 -- 172.21.15.46:0/383205013 learned_addr learned my addr 172.21.15.46:0/383205013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16e000640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16ea00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1700b5fb0 0x7fd1700a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 -- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 msgr2=0x7fd1700cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 -- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1700b5fb0 msgr2=0x7fd1700a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1700b5fb0 0x7fd1700a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 -- 172.21.15.46:0/383205013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1700a0a70 con 0x7fd170162aa0 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.617+0000 7fd16f400640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c3c0 tx=0x7fd16800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.618+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168017070 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.618+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd168005d00 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.618+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168007310 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.618+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17009ee20 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.618+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17009f3e0 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.619+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17009f5e0 con 0x7fd170162aa0 2024-04-26T23:44:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.620+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd16801a070 con 0x7fd170162aa0 2024-04-26T23:44:24.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.621+0000 7fd166a00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 0x7fd15006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:24.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.621+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7fd16808b0d0 con 0x7fd170162aa0 2024-04-26T23:44:24.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.622+0000 7fd16ea00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 0x7fd15006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:24.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.623+0000 7fd16ea00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 0x7fd15006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd160034000 tx=0x7fd16002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.626+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd17009f5e0 con 0x7fd170162aa0 2024-04-26T23:44:24.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.828+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1700693b0 con 0x7fd170162aa0 2024-04-26T23:44:24.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.829+0000 7fd166a00640 1 -- 172.21.15.46:0/383205013 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd168007e20 con 0x7fd170162aa0 2024-04-26T23:44:24.834 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 msgr2=0x7fd15006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 0x7fd15006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd160034000 tx=0x7fd16002f000 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 msgr2=0x7fd1700d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c3c0 tx=0x7fd16800c880 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 shutdown_connections 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd15006c8f0 0x7fd15006edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd170162aa0 0x7fd1700d0040 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1701626c0 0x7fd1700cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 --2- 172.21.15.46:0/383205013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1700b5fb0 0x7fd1700a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 >> 172.21.15.46:0/383205013 conn(0x7fd17009e2a0 msgr2=0x7fd1700cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 shutdown_connections 2024-04-26T23:44:24.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:24.832+0000 7fd16fe00640 1 -- 172.21.15.46:0/383205013 wait complete. 2024-04-26T23:44:24.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:44:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.067+0000 7f6a94a00640 1 Processor -- start 2024-04-26T23:44:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.068+0000 7f6a94a00640 1 -- start start 2024-04-26T23:44:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a94a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a901626c0 0x7f6a900b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a94a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 0x7f6a900d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a94a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a900d07d0 con 0x7f6a90162aa0 2024-04-26T23:44:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a94a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6a900d0ab0 con 0x7f6a901626c0 2024-04-26T23:44:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a94a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a900d0d90 con 0x7f6a900b5fb0 2024-04-26T23:44:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a8fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a90162aa0 0x7f6a900d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41550/0 (socket says 172.21.15.46:41550) 2024-04-26T23:44:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.069+0000 7f6a8fe00640 1 -- 172.21.15.46:0/69122518 learned_addr learned my addr 172.21.15.46:0/69122518 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8ea00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 0x7f6a900d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8f400640 1 --2- 172.21.15.46:0/69122518 >> v2:172.21.15.142:3300/0 conn(0x7f6a901626c0 0x7f6a900b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063264863 0 0) 0x7f6a900d07d0 con 0x7f6a90162aa0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a74003520 con 0x7f6a90162aa0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4057658651 0 0) 0x7f6a74003520 con 0x7f6a90162aa0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 msgr2=0x7f6a900d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 0x7f6a900d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 >> v2:172.21.15.142:3300/0 conn(0x7f6a901626c0 msgr2=0x7f6a900b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 --2- 172.21.15.46:0/69122518 >> v2:172.21.15.142:3300/0 conn(0x7f6a901626c0 0x7f6a900b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a900d2c80 con 0x7f6a90162aa0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6a880019f0 con 0x7f6a90162aa0 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 >> v1:172.21.15.46:6789/0 conn(0x7f6a90162aa0 legacy=0x7f6a900d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a740039b0 0x7f6a74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 0x7f6a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a74004d40 0x7f6a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900d0ab0 con 0x7f6a74004d40 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900d0d90 con 0x7f6a740039b0 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.070+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7400b9c0 con 0x7f6a900b5fb0 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a74004d40 0x7f6a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8f400640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a740039b0 0x7f6a74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 -- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 msgr2=0x7f6a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900b5fb0 0x7f6a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 -- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a740039b0 msgr2=0x7f6a74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a740039b0 0x7f6a74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8fe00640 1 -- 172.21.15.46:0/69122518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74003520 con 0x7f6a74004d40 2024-04-26T23:44:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.071+0000 7f6a8f400640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a740039b0 0x7f6a74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.072+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a74004d40 0x7f6a740051c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800a8a0 tx=0x7f6a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96ddef552dcdedc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.072+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800a4c0 con 0x7f6a74004d40 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.072+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8802f070 con 0x7f6a74004d40 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a8e000640 1 -- 172.21.15.46:0/69122518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8802bcf0 con 0x7f6a74004d40 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 -- 172.21.15.46:0/69122518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a74004d40 msgr2=0x7f6a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 --2- 172.21.15.46:0/69122518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a74004d40 0x7f6a740051c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800a8a0 tx=0x7f6a74005830 comp rx=0 tx=0).stop 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a8fe00640 1 -- 172.21.15.46:0/69122518 reap_dead start 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 -- 172.21.15.46:0/69122518 shutdown_connections 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 -- 172.21.15.46:0/69122518 >> 172.21.15.46:0/69122518 conn(0x7f6a9009e2a0 msgr2=0x7f6a900d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 -- 172.21.15.46:0/69122518 shutdown_connections 2024-04-26T23:44:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.073+0000 7f6a94a00640 1 -- 172.21.15.46:0/69122518 wait complete. 2024-04-26T23:44:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.075+0000 7f6a94a00640 1 Processor -- start 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.075+0000 7f6a94a00640 1 -- start start 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 0x7f6a900c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 0x7f6a900c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900d5850 con 0x7f6a90158380 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900d2220 con 0x7f6a900cf030 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a94a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a900d59d0 con 0x7f6a900cec50 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44098/0 (socket says 172.21.15.46:44098) 2024-04-26T23:44:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.076+0000 7f6a8fe00640 1 -- 172.21.15.46:0/3616092066 learned_addr learned my addr 172.21.15.46:0/3616092066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8f400640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 0x7f6a900c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8ea00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 0x7f6a900c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 -- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 msgr2=0x7f6a900c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 0x7f6a900c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 -- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 msgr2=0x7f6a900c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 0x7f6a900c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 -- 172.21.15.46:0/3616092066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a900ca600 con 0x7f6a90158380 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8f400640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 0x7f6a900c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8ea00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 0x7f6a900c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a8fe00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800a9c0 tx=0x7f6a88001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.077+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800a4c0 con 0x7f6a90158380 2024-04-26T23:44:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.078+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88034470 con 0x7f6a90158380 2024-04-26T23:44:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.078+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88002010 con 0x7f6a90158380 2024-04-26T23:44:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.078+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a900ca8e0 con 0x7f6a90158380 2024-04-26T23:44:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.078+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a900caea0 con 0x7f6a90158380 2024-04-26T23:44:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.079+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a900693b0 con 0x7f6a90158380 2024-04-26T23:44:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.080+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a88034860 con 0x7f6a90158380 2024-04-26T23:44:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.081+0000 7f6a87400640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 0x7f6a7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.081+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f6a880abc50 con 0x7f6a90158380 2024-04-26T23:44:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.082+0000 7f6a8f400640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 0x7f6a7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.083+0000 7f6a8f400640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 0x7f6a7006ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009b20 tx=0x7f6a80008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.086+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a880b1020 con 0x7f6a90158380 2024-04-26T23:44:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.290+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a900d25b0 con 0x7f6a90158380 2024-04-26T23:44:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.291+0000 7f6a87400640 1 -- 172.21.15.46:0/3616092066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f6a88046060 con 0x7f6a90158380 2024-04-26T23:44:25.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:44:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.293+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 msgr2=0x7f6a7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.293+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 0x7f6a7006ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009b20 tx=0x7f6a80008040 comp rx=0 tx=0).stop 2024-04-26T23:44:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.293+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 msgr2=0x7f6a90158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.293+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800a9c0 tx=0x7f6a88001e30 comp rx=0 tx=0).stop 2024-04-26T23:44:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 shutdown_connections 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a7006c920 0x7f6a7006ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a90158380 0x7f6a90158800 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900cf030 0x7f6a900c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 --2- 172.21.15.46:0/3616092066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a900cec50 0x7f6a900c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 >> 172.21.15.46:0/3616092066 conn(0x7f6a9009e2a0 msgr2=0x7f6a9006dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 shutdown_connections 2024-04-26T23:44:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.294+0000 7f6a94a00640 1 -- 172.21.15.46:0/3616092066 wait complete. 2024-04-26T23:44:25.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:44:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.526+0000 7f37ad800640 1 Processor -- start 2024-04-26T23:44:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.527+0000 7f37ad800640 1 -- start start 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.528+0000 7f37ad800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37a815f840 0x7f37a80d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.528+0000 7f37ad800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37a80d1980 0x7f37a80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.528+0000 7f37ad800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a80d4090 con 0x7f37a80d2320 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.528+0000 7f37ad800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f37a80d3880 con 0x7f37a815f840 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.528+0000 7f37ad800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37a80d2790 con 0x7f37a80d1980 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37a815f840 0x7f37a80d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37ace00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f37a80d2320 0x7f37a80cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41552/0 (socket says 172.21.15.46:41552) 2024-04-26T23:44:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37ace00640 1 -- 172.21.15.46:0/500502083 learned_addr learned my addr 172.21.15.46:0/500502083 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7400640 1 --2- 172.21.15.46:0/500502083 >> v2:172.21.15.142:3300/0 conn(0x7f37a80d1980 0x7f37a80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7e00640 1 -- 172.21.15.46:0/500502083 >> v2:172.21.15.142:3300/0 conn(0x7f37a80d1980 msgr2=0x7f37a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7e00640 1 --2- 172.21.15.46:0/500502083 >> v2:172.21.15.142:3300/0 conn(0x7f37a80d1980 0x7f37a80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7e00640 1 -- 172.21.15.46:0/500502083 >> v1:172.21.15.46:6789/0 conn(0x7f37a80d2320 legacy=0x7f37a80cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a7e00640 1 -- 172.21.15.46:0/500502083 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a80b4020 con 0x7f37a815f840 2024-04-26T23:44:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.529+0000 7f37a6a00640 1 -- 172.21.15.46:0/500502083 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314627353 0 0) 0x7f37a0001110 con 0x7f37a80d2320 2024-04-26T23:44:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.530+0000 7f37a7e00640 1 --2- 172.21.15.46:0/500502083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a815f840 0x7f37a80d2e70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f37a808d000 tx=0x7f379802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161c1b6e4629d1c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37a6a00640 1 -- 172.21.15.46:0/500502083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798037070 con 0x7f37a815f840 2024-04-26T23:44:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37a6a00640 1 -- 172.21.15.46:0/500502083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3798032440 con 0x7f37a815f840 2024-04-26T23:44:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37a6a00640 1 -- 172.21.15.46:0/500502083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798032810 con 0x7f37a815f840 2024-04-26T23:44:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 -- 172.21.15.46:0/500502083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a815f840 msgr2=0x7f37a80d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 --2- 172.21.15.46:0/500502083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a815f840 0x7f37a80d2e70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f37a808d000 tx=0x7f379802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 -- 172.21.15.46:0/500502083 shutdown_connections 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 --2- 172.21.15.46:0/500502083 >> v2:172.21.15.142:3300/0 conn(0x7f37a80d1980 0x7f37a80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 --2- 172.21.15.46:0/500502083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a815f840 0x7f37a80d2e70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.531+0000 7f37ad800640 1 -- 172.21.15.46:0/500502083 >> 172.21.15.46:0/500502083 conn(0x7f37a80d4f70 msgr2=0x7f37a80d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.532+0000 7f37ad800640 1 -- 172.21.15.46:0/500502083 shutdown_connections 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.532+0000 7f37ad800640 1 -- 172.21.15.46:0/500502083 wait complete. 2024-04-26T23:44:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.533+0000 7f37ad800640 1 Processor -- start 2024-04-26T23:44:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.534+0000 7f37ad800640 1 -- start start 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 0x7f37a80b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37a815f840 0x7f37a81f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80b27d0 con 0x7f37a80d2320 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80d3db0 con 0x7f37a80d1980 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37ad800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37a80d33b0 con 0x7f37a815f840 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44108/0 (socket says 172.21.15.46:44108) 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 -- 172.21.15.46:0/323987284 learned_addr learned my addr 172.21.15.46:0/323987284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7e00640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 0x7f37a80b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 -- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37a815f840 msgr2=0x7f37a81f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37a815f840 0x7f37a81f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 -- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 msgr2=0x7f37a80b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 0x7f37a80b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7400640 1 -- 172.21.15.46:0/323987284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a80ce580 con 0x7f37a80d2320 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.535+0000 7f37a7e00640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 0x7f37a80b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f37a7400640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f379000ccd0 tx=0x7f379000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790003990 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3790003b30 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004020 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a80ce820 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.536+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a80cede0 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.537+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a80693b0 con 0x7f37a80d2320 2024-04-26T23:44:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.538+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37900041c0 con 0x7f37a80d2320 2024-04-26T23:44:25.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.539+0000 7f379fe00640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 0x7f378406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.539+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f379008c200 con 0x7f37a80d2320 2024-04-26T23:44:25.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.540+0000 7f37a7e00640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 0x7f378406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.541+0000 7f37a7e00640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 0x7f378406edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3798034000 tx=0x7f379802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.544+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3790014030 con 0x7f37a80d2320 2024-04-26T23:44:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.746+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37a81f8130 con 0x7f37a80d2320 2024-04-26T23:44:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.747+0000 7f379fe00640 1 -- 172.21.15.46:0/323987284 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f379005a440 con 0x7f37a80d2320 2024-04-26T23:44:25.752 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 msgr2=0x7f378406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 0x7f378406edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3798034000 tx=0x7f379802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 msgr2=0x7f37a81f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f379000ccd0 tx=0x7f379000a5c0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 shutdown_connections 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f378406c8f0 0x7f378406edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37a815f840 0x7f37a81f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a80d2320 0x7f37a81f76b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 --2- 172.21.15.46:0/323987284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37a80d1980 0x7f37a80b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 >> 172.21.15.46:0/323987284 conn(0x7f37a80d4f70 msgr2=0x7f37a80b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 shutdown_connections 2024-04-26T23:44:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.750+0000 7f37ad800640 1 -- 172.21.15.46:0/323987284 wait complete. 2024-04-26T23:44:25.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:25.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.972+0000 7fab10a00640 1 Processor -- start 2024-04-26T23:44:25.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.972+0000 7fab10a00640 1 -- start start 2024-04-26T23:44:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab10a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab0c0cc690 0x7fab0c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab10a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c0c6a50 0x7fab0c0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab10a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0c072bb0 con 0x7fab0c1454a0 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab10a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fab0c072e90 con 0x7fab0c0cc690 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab10a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c073170 con 0x7fab0c0c6a50 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab0c0cc690 0x7fab0c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab0c0cc690 0x7fab0c0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44118/0 (socket says 172.21.15.46:44118) 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.974+0000 7fab03e00640 1 -- 172.21.15.46:0/3485206412 learned_addr learned my addr 172.21.15.46:0/3485206412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.975+0000 7fab03e00640 1 -- 172.21.15.46:0/3485206412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c0c6a50 msgr2=0x7fab0c0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.975+0000 7fab03e00640 1 --2- 172.21.15.46:0/3485206412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c0c6a50 0x7fab0c0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.975+0000 7fab03e00640 1 -- 172.21.15.46:0/3485206412 >> v1:172.21.15.142:6789/0 conn(0x7fab0c1454a0 legacy=0x7fab0c073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.975+0000 7fab03e00640 1 -- 172.21.15.46:0/3485206412 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c073450 con 0x7fab0c0cc690 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.976+0000 7fab03e00640 1 --2- 172.21.15.46:0/3485206412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0cc690 0x7fab0c0cdc30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009870 tx=0x7faaf402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a0e56443e62644 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.976+0000 7fab09e00640 1 -- 172.21.15.46:0/3485206412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4037070 con 0x7fab0c0cc690 2024-04-26T23:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.976+0000 7fab09e00640 1 -- 172.21.15.46:0/3485206412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf4032440 con 0x7fab0c0cc690 2024-04-26T23:44:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.976+0000 7fab09e00640 1 -- 172.21.15.46:0/3485206412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4032810 con 0x7fab0c0cc690 2024-04-26T23:44:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 -- 172.21.15.46:0/3485206412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0cc690 msgr2=0x7fab0c0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 --2- 172.21.15.46:0/3485206412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0cc690 0x7fab0c0cdc30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009870 tx=0x7faaf402a210 comp rx=0 tx=0).stop 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 -- 172.21.15.46:0/3485206412 shutdown_connections 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 --2- 172.21.15.46:0/3485206412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c0c6a50 0x7fab0c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 --2- 172.21.15.46:0/3485206412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0cc690 0x7fab0c0cdc30 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 -- 172.21.15.46:0/3485206412 >> 172.21.15.46:0/3485206412 conn(0x7fab0c0b26d0 msgr2=0x7fab0c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 -- 172.21.15.46:0/3485206412 shutdown_connections 2024-04-26T23:44:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.977+0000 7fab10a00640 1 -- 172.21.15.46:0/3485206412 wait complete. 2024-04-26T23:44:25.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.979+0000 7fab10a00640 1 Processor -- start 2024-04-26T23:44:25.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.980+0000 7fab10a00640 1 -- start start 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 0x7fab0c15ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c1454a0 0x7fab0c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c145880 con 0x7fab0c0c6a50 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c072880 con 0x7fab0c0cc690 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab10a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c145a00 con 0x7fab0c1454a0 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44126/0 (socket says 172.21.15.46:44126) 2024-04-26T23:44:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.981+0000 7fab0a800640 1 -- 172.21.15.46:0/3363428543 learned_addr learned my addr 172.21.15.46:0/3363428543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab03e00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 0x7fab0c15ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 -- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c1454a0 msgr2=0x7fab0c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0b200640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c1454a0 0x7fab0c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c1454a0 0x7fab0c15dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 -- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 msgr2=0x7fab0c15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 0x7fab0c15ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 -- 172.21.15.46:0/3363428543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c15ccd0 con 0x7fab0c0c6a50 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab03e00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 0x7fab0c15ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab0a800640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b3e0 tx=0x7faafc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc015020 con 0x7fab0c0c6a50 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc011850 con 0x7fab0c0c6a50 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc011c40 con 0x7fab0c0c6a50 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.982+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c15d810 con 0x7fab0c0c6a50 2024-04-26T23:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.983+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c15d250 con 0x7fab0c0c6a50 2024-04-26T23:44:25.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.984+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c0693b0 con 0x7fab0c0c6a50 2024-04-26T23:44:25.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.985+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faafc018020 con 0x7fab0c0c6a50 2024-04-26T23:44:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.986+0000 7fab02a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.986+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7faafc08bd40 con 0x7fab0c0c6a50 2024-04-26T23:44:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.986+0000 7fab03e00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.988+0000 7fab03e00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 0x7faae806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faaf4034000 tx=0x7faaf402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:25.990+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faafc01f020 con 0x7fab0c0c6a50 2024-04-26T23:44:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.235+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab0c15e7f0 con 0x7fab0c0c6a50 2024-04-26T23:44:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.236+0000 7fab02a00640 1 -- 172.21.15.46:0/3363428543 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7faafc059ef0 con 0x7fab0c0c6a50 2024-04-26T23:44:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:26.241 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:06.419460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7416,"qdb_cluster":[7416]},"id":5}]} 2024-04-26T23:44:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 msgr2=0x7faae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 0x7faae806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faaf4034000 tx=0x7faaf402e000 comp rx=0 tx=0).stop 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 msgr2=0x7fab0c15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b3e0 tx=0x7faafc00b8a0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 shutdown_connections 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab0c1454a0 0x7fab0c15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab0c0cc690 0x7fab0c15ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.239+0000 7fab10a00640 1 --2- 172.21.15.46:0/3363428543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c0c6a50 0x7fab0c15b870 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.240+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 >> 172.21.15.46:0/3363428543 conn(0x7fab0c0b26d0 msgr2=0x7fab0c15fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.240+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 shutdown_connections 2024-04-26T23:44:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.240+0000 7fab10a00640 1 -- 172.21.15.46:0/3363428543 wait complete. 2024-04-26T23:44:26.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.472+0000 7f7a0f200640 1 Processor -- start 2024-04-26T23:44:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.472+0000 7f7a0f200640 1 -- start start 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.473+0000 7f7a0f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 0x7f7a100cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.473+0000 7f7a0f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7a100d0be0 0x7f7a100ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.473+0000 7f7a0f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a100d3c30 con 0x7f7a1015d080 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.473+0000 7f7a0f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7a100cdb40 con 0x7f7a100d0be0 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.473+0000 7f7a0f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a100cdde0 con 0x7f7a100d1a60 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.474+0000 7f7a0de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7a1015d080 0x7f7a100d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41568/0 (socket says 172.21.15.46:41568) 2024-04-26T23:44:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.474+0000 7f7a0de00640 1 -- 172.21.15.46:0/1960065780 learned_addr learned my addr 172.21.15.46:0/1960065780 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.474+0000 7f7a0d400640 1 --2- 172.21.15.46:0/1960065780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 0x7f7a100cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.474+0000 7f7a0e800640 1 --2- 172.21.15.46:0/1960065780 >> v2:172.21.15.142:3300/0 conn(0x7f7a100d0be0 0x7f7a100ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038405871 0 0) 0x7f7a100d3c30 con 0x7f7a1015d080 2024-04-26T23:44:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f0003520 con 0x7f7a1015d080 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0d400640 1 -- 172.21.15.46:0/1960065780 >> v2:172.21.15.142:3300/0 conn(0x7f7a100d0be0 msgr2=0x7f7a100ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0d400640 1 --2- 172.21.15.46:0/1960065780 >> v2:172.21.15.142:3300/0 conn(0x7f7a100d0be0 0x7f7a100ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0d400640 1 -- 172.21.15.46:0/1960065780 >> v1:172.21.15.46:6789/0 conn(0x7f7a1015d080 legacy=0x7f7a100d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4060584601 0 0) 0x7f79f0003520 con 0x7f7a1015d080 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.475+0000 7f7a0d400640 1 -- 172.21.15.46:0/1960065780 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a100ce0c0 con 0x7f7a100d1a60 2024-04-26T23:44:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.476+0000 7f7a0d400640 1 --2- 172.21.15.46:0/1960065780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 0x7f7a100cfdb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009dc0 tx=0x7f79f802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ac9c1bb9c7c8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.477+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00001670 con 0x7f7a100d1a60 2024-04-26T23:44:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.477+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f8007540 con 0x7f7a100d1a60 2024-04-26T23:44:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.477+0000 7f7a0ca00640 1 -- 172.21.15.46:0/1960065780 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8007710 con 0x7f7a100d1a60 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.477+0000 7f7a0f200640 1 -- 172.21.15.46:0/1960065780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 msgr2=0x7f7a100cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.477+0000 7f7a0f200640 1 --2- 172.21.15.46:0/1960065780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 0x7f7a100cfdb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009dc0 tx=0x7f79f802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 -- 172.21.15.46:0/1960065780 shutdown_connections 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 --2- 172.21.15.46:0/1960065780 >> v2:172.21.15.142:3300/0 conn(0x7f7a100d0be0 0x7f7a100ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 --2- 172.21.15.46:0/1960065780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a100d1a60 0x7f7a100cfdb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 -- 172.21.15.46:0/1960065780 >> 172.21.15.46:0/1960065780 conn(0x7f7a100d2d70 msgr2=0x7f7a100d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 -- 172.21.15.46:0/1960065780 shutdown_connections 2024-04-26T23:44:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.478+0000 7f7a0f200640 1 -- 172.21.15.46:0/1960065780 wait complete. 2024-04-26T23:44:26.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.480+0000 7f7a0f200640 1 Processor -- start 2024-04-26T23:44:26.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.480+0000 7f7a0f200640 1 -- start start 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a100d1a60 0x7f7a100cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a1015d080 0x7f7a100c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a100d3a10 con 0x7f7a100d0be0 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a100d3890 con 0x7f7a100d1a60 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1015d460 con 0x7f7a1015d080 2024-04-26T23:44:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44134/0 (socket says 172.21.15.46:44134) 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.481+0000 7f7a0de00640 1 -- 172.21.15.46:0/58750822 learned_addr learned my addr 172.21.15.46:0/58750822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0e800640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a1015d080 0x7f7a100c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0d400640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a100d1a60 0x7f7a100cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 -- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a1015d080 msgr2=0x7f7a100c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a1015d080 0x7f7a100c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 -- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a100d1a60 msgr2=0x7f7a100cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a100d1a60 0x7f7a100cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 -- 172.21.15.46:0/58750822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1014fb40 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.482+0000 7f7a0de00640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7a00001380 tx=0x7f7a0000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.483+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00017070 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.483+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a00005ad0 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.483+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00007070 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.483+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a100cb520 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.483+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a100cbae0 con 0x7f7a100d0be0 2024-04-26T23:44:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.484+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10069ac0 con 0x7f7a100d0be0 2024-04-26T23:44:26.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.485+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a0001a070 con 0x7f7a100d0be0 2024-04-26T23:44:26.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.486+0000 7f7a06000640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 0x7f79ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.486+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f7a0008b290 con 0x7f7a100d0be0 2024-04-26T23:44:26.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.487+0000 7f7a0d400640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 0x7f79ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.488+0000 7f7a0d400640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 0x7f79ec06ee00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.491+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a000594c0 con 0x7f7a100d0be0 2024-04-26T23:44:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.696+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a1006f9a0 con 0x7f7a100d0be0 2024-04-26T23:44:26.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.698+0000 7f7a06000640 1 -- 172.21.15.46:0/58750822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+17040 (secure 0 0 0) 0x7f7a00058dc0 con 0x7f7a100d0be0 2024-04-26T23:44:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:26.703 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:02.386476+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6837/2319300947":"2024-04-27T23:44:02.372699+0000","172.21.15.142:6835/2319300947":"2024-04-27T23:44:02.372699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:26.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.700+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 msgr2=0x7f79ec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.700+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 0x7f79ec06ee00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f802f000 comp rx=0 tx=0).stop 2024-04-26T23:44:26.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.700+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 msgr2=0x7f7a100cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.700+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7a00001380 tx=0x7f7a0000cbe0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 shutdown_connections 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f79ec06c940 0x7f79ec06ee00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a1015d080 0x7f7a100c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a100d1a60 0x7f7a100cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 --2- 172.21.15.46:0/58750822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a100d0be0 0x7f7a100cae90 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 >> 172.21.15.46:0/58750822 conn(0x7f7a100d2d70 msgr2=0x7f7a1006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 shutdown_connections 2024-04-26T23:44:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.701+0000 7f7a0f200640 1 -- 172.21.15.46:0/58750822 wait complete. 2024-04-26T23:44:26.724 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:44:26.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.934+0000 7f953be00640 1 Processor -- start 2024-04-26T23:44:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.934+0000 7f953be00640 1 -- start start 2024-04-26T23:44:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.936+0000 7f953be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f953c0d2130 0x7f953c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.936+0000 7f953be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.936+0000 7f953be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953c0d11f0 con 0x7f953c15f4e0 2024-04-26T23:44:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.936+0000 7f953be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f953c0d0ce0 con 0x7f953c0d2130 2024-04-26T23:44:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.936+0000 7f953be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f953c0d2990 con 0x7f953c0b4000 2024-04-26T23:44:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953aa00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f953c15f4e0 0x7f953c0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:41498/0 (socket says 172.21.15.46:41498) 2024-04-26T23:44:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953aa00640 1 -- 172.21.15.46:0/1200963006 learned_addr learned my addr 172.21.15.46:0/1200963006 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953a000640 1 --2- 172.21.15.46:0/1200963006 >> v2:172.21.15.142:3300/0 conn(0x7f953c0d2130 0x7f953c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953b400640 1 -- 172.21.15.46:0/1200963006 >> v2:172.21.15.142:3300/0 conn(0x7f953c0d2130 msgr2=0x7f953c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.937+0000 7f953b400640 1 --2- 172.21.15.46:0/1200963006 >> v2:172.21.15.142:3300/0 conn(0x7f953c0d2130 0x7f953c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.938+0000 7f953b400640 1 -- 172.21.15.46:0/1200963006 >> v1:172.21.15.142:6789/0 conn(0x7f953c15f4e0 legacy=0x7f953c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.938+0000 7f953b400640 1 -- 172.21.15.46:0/1200963006 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c0d5f20 con 0x7f953c0b4000 2024-04-26T23:44:26.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.939+0000 7f953b400640 1 --2- 172.21.15.46:0/1200963006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0b2280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9534009d90 tx=0x7f953402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5a33d4d09ac158 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.939+0000 7f9539600640 1 -- 172.21.15.46:0/1200963006 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534007740 con 0x7f953c0b4000 2024-04-26T23:44:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f9539600640 1 -- 172.21.15.46:0/1200963006 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95340078e0 con 0x7f953c0b4000 2024-04-26T23:44:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f9539600640 1 -- 172.21.15.46:0/1200963006 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534007cb0 con 0x7f953c0b4000 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 -- 172.21.15.46:0/1200963006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 msgr2=0x7f953c0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 --2- 172.21.15.46:0/1200963006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0b2280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9534009d90 tx=0x7f953402a660 comp rx=0 tx=0).stop 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 -- 172.21.15.46:0/1200963006 shutdown_connections 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 --2- 172.21.15.46:0/1200963006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0b2280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 --2- 172.21.15.46:0/1200963006 >> v2:172.21.15.142:3300/0 conn(0x7f953c0d2130 0x7f953c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.940+0000 7f953be00640 1 -- 172.21.15.46:0/1200963006 >> 172.21.15.46:0/1200963006 conn(0x7f953c0d5010 msgr2=0x7f953c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.941+0000 7f953be00640 1 -- 172.21.15.46:0/1200963006 shutdown_connections 2024-04-26T23:44:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.941+0000 7f953be00640 1 -- 172.21.15.46:0/1200963006 wait complete. 2024-04-26T23:44:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.943+0000 7f953be00640 1 Processor -- start 2024-04-26T23:44:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.943+0000 7f953be00640 1 -- start start 2024-04-26T23:44:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c0d2130 0x7f953c1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f953c0b1d60 con 0x7f953c15f4e0 2024-04-26T23:44:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f953c0d5c40 con 0x7f953c0d2130 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.944+0000 7f953be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f953c15f8c0 con 0x7f953c0b4000 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44136/0 (socket says 172.21.15.46:44136) 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 -- 172.21.15.46:0/1629651889 learned_addr learned my addr 172.21.15.46:0/1629651889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953aa00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953a000640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c0d2130 0x7f953c1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 -- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 msgr2=0x7f953c0a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 -- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c0d2130 msgr2=0x7f953c1f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c0d2130 0x7f953c1f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.945+0000 7f953b400640 1 -- 172.21.15.46:0/1629651889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c0a1ce0 con 0x7f953c15f4e0 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f953b400640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f953402ac50 tx=0x7f9534004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534040070 con 0x7f953c15f4e0 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9534031b40 con 0x7f953c15f4e0 2024-04-26T23:44:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534045460 con 0x7f953c15f4e0 2024-04-26T23:44:26.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c0a1fc0 con 0x7f953c15f4e0 2024-04-26T23:44:26.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.946+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c0a2580 con 0x7f953c15f4e0 2024-04-26T23:44:26.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.948+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c0693b0 con 0x7f953c15f4e0 2024-04-26T23:44:26.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.949+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9534039040 con 0x7f953c15f4e0 2024-04-26T23:44:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.950+0000 7f9532a00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 0x7f951c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.950+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f95340ad510 con 0x7f953c15f4e0 2024-04-26T23:44:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.950+0000 7f953aa00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 0x7f951c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.952+0000 7f953aa00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 0x7f951c06edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f953c0bacf0 tx=0x7f952c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:26.954+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f953407b750 con 0x7f953c15f4e0 2024-04-26T23:44:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.200+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f953c0d46d0 con 0x7f953c15f4e0 2024-04-26T23:44:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.202+0000 7f9532a00640 1 -- 172.21.15.46:0/1629651889 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7f953407b050 con 0x7f953c15f4e0 2024-04-26T23:44:27.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:27.207 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:06.419460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7416,"qdb_cluster":[7416]},"id":5}]} 2024-04-26T23:44:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-04-26T23:44:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 msgr2=0x7f951c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 0x7f951c06edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f953c0bacf0 tx=0x7f952c009040 comp rx=0 tx=0).stop 2024-04-26T23:44:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 msgr2=0x7f953c1f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f953402ac50 tx=0x7f9534004a40 comp rx=0 tx=0).stop 2024-04-26T23:44:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 shutdown_connections 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f951c06c8f0 0x7f951c06edb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c15f4e0 0x7f953c1f9390 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c0d2130 0x7f953c1f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 --2- 172.21.15.46:0/1629651889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f953c0b4000 0x7f953c0a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 >> 172.21.15.46:0/1629651889 conn(0x7f953c0d5010 msgr2=0x7f953c0b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.206+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 shutdown_connections 2024-04-26T23:44:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.207+0000 7f953be00640 1 -- 172.21.15.46:0/1629651889 wait complete. 2024-04-26T23:44:27.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.432+0000 7fef11200640 1 Processor -- start 2024-04-26T23:44:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.432+0000 7fef11200640 1 -- start start 2024-04-26T23:44:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef11200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef0c06e510 0x7fef0c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef11200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef0c0732c0 0x7fef0c073710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef11200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0c06ee30 con 0x7fef0c160bd0 2024-04-26T23:44:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef11200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c161940 con 0x7fef0c06e510 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef11200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c071f80 con 0x7fef0c0732c0 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef09e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef0c06e510 0x7fef0c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef09e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef0c06e510 0x7fef0c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44152/0 (socket says 172.21.15.46:44152) 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.434+0000 7fef09e00640 1 -- 172.21.15.46:0/3146711488 learned_addr learned my addr 172.21.15.46:0/3146711488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.435+0000 7fef09e00640 1 -- 172.21.15.46:0/3146711488 >> v2:172.21.15.142:3300/0 conn(0x7fef0c0732c0 msgr2=0x7fef0c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.435+0000 7fef09e00640 1 --2- 172.21.15.46:0/3146711488 >> v2:172.21.15.142:3300/0 conn(0x7fef0c0732c0 0x7fef0c073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.435+0000 7fef09e00640 1 -- 172.21.15.46:0/3146711488 >> v1:172.21.15.142:6789/0 conn(0x7fef0c160bd0 legacy=0x7fef0c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.435+0000 7fef09e00640 1 -- 172.21.15.46:0/3146711488 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c072260 con 0x7fef0c06e510 2024-04-26T23:44:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.436+0000 7fef09e00640 1 --2- 172.21.15.46:0/3146711488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c06e510 0x7fef0c06e8f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf43d94ba1ea35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.436+0000 7fef08a00640 1 -- 172.21.15.46:0/3146711488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc037070 con 0x7fef0c06e510 2024-04-26T23:44:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.436+0000 7fef08a00640 1 -- 172.21.15.46:0/3146711488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc032440 con 0x7fef0c06e510 2024-04-26T23:44:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.436+0000 7fef08a00640 1 -- 172.21.15.46:0/3146711488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc032810 con 0x7fef0c06e510 2024-04-26T23:44:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 -- 172.21.15.46:0/3146711488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c06e510 msgr2=0x7fef0c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 --2- 172.21.15.46:0/3146711488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c06e510 0x7fef0c06e8f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02a270 comp rx=0 tx=0).stop 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 -- 172.21.15.46:0/3146711488 shutdown_connections 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 --2- 172.21.15.46:0/3146711488 >> v2:172.21.15.142:3300/0 conn(0x7fef0c0732c0 0x7fef0c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 --2- 172.21.15.46:0/3146711488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c06e510 0x7fef0c06e8f0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 -- 172.21.15.46:0/3146711488 >> 172.21.15.46:0/3146711488 conn(0x7fef0c06c530 msgr2=0x7fef0c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 -- 172.21.15.46:0/3146711488 shutdown_connections 2024-04-26T23:44:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.437+0000 7fef11200640 1 -- 172.21.15.46:0/3146711488 wait complete. 2024-04-26T23:44:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.439+0000 7fef11200640 1 Processor -- start 2024-04-26T23:44:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.439+0000 7fef11200640 1 -- start start 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef0c06e510 0x7fef0c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 0x7fef0c0ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c072d40 con 0x7fef0c0732c0 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c072820 con 0x7fef0c160bd0 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef11200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c072ec0 con 0x7fef0c06e510 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44156/0 (socket says 172.21.15.46:44156) 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef09400640 1 -- 172.21.15.46:0/3126339289 learned_addr learned my addr 172.21.15.46:0/3126339289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef09e00640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef0c06e510 0x7fef0c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.441+0000 7fef0a800640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 0x7fef0c0ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 -- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef0c06e510 msgr2=0x7fef0c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef0c06e510 0x7fef0c1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 -- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 msgr2=0x7fef0c0ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 0x7fef0c0ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 -- 172.21.15.46:0/3126339289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0ad060 con 0x7fef0c0732c0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef0a800640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 0x7fef0c0ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef09400640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7feef800c410 tx=0x7feef800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8017070 con 0x7fef0c0732c0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef80035f0 con 0x7fef0c0732c0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8003a20 con 0x7fef0c0732c0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c0ad340 con 0x7fef0c0732c0 2024-04-26T23:44:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.442+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c0ad900 con 0x7fef0c0732c0 2024-04-26T23:44:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.444+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c069530 con 0x7fef0c0732c0 2024-04-26T23:44:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.445+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feef801a070 con 0x7fef0c0732c0 2024-04-26T23:44:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.446+0000 7fef02000640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.446+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7feef808b120 con 0x7fef0c0732c0 2024-04-26T23:44:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.446+0000 7fef09e00640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.447+0000 7fef09e00640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 0x7feeec06edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feefc034000 tx=0x7feefc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.452+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feef8059400 con 0x7fef0c0732c0 2024-04-26T23:44:27.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.699+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef0c06fb00 con 0x7fef0c0732c0 2024-04-26T23:44:27.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.700+0000 7fef02000640 1 -- 172.21.15.46:0/3126339289 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7feef8058d00 con 0x7fef0c0732c0 2024-04-26T23:44:27.705 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:43:48.721859+0000","modified":"2024-04-26T23:44:06.419460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7416":{"gid":7416,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6833/357252295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":357252295},{"type":"v1","addr":"172.21.15.46:6833","nonce":357252295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7416,"qdb_cluster":[7416]},"id":5}]} 2024-04-26T23:44:27.705 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-04-26T23:44:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 msgr2=0x7feeec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 0x7feeec06edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feefc034000 tx=0x7feefc02e000 comp rx=0 tx=0).stop 2024-04-26T23:44:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 msgr2=0x7fef0c1f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7feef800c410 tx=0x7feef800c8d0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 shutdown_connections 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef0c160bd0 0x7fef0c0ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0732c0 0x7fef0c1f2e70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 --2- 172.21.15.46:0/3126339289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef0c06e510 0x7fef0c1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 >> 172.21.15.46:0/3126339289 conn(0x7fef0c06c530 msgr2=0x7fef0c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 shutdown_connections 2024-04-26T23:44:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.704+0000 7fef11200640 1 -- 172.21.15.46:0/3126339289 wait complete. 2024-04-26T23:44:27.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.938+0000 7f7107600640 1 Processor -- start 2024-04-26T23:44:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.938+0000 7f7107600640 1 -- start start 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.939+0000 7f7107600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71000d3550 0x7f710006e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.939+0000 7f7107600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710006d7c0 con 0x7f710015f100 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.939+0000 7f7107600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710006daa0 con 0x7f71000d04e0 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.939+0000 7f7107600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f710006eac0 con 0x7f71000d3550 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7105600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71000d3550 0x7f710006e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7104c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f710015f100 0x7f7100070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41584/0 (socket says 172.21.15.46:41584) 2024-04-26T23:44:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7104c00640 1 -- 172.21.15.46:0/698496586 learned_addr learned my addr 172.21.15.46:0/698496586 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7105600640 1 -- 172.21.15.46:0/698496586 >> v1:172.21.15.142:6789/0 conn(0x7f71000d04e0 legacy=0x7f71000d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7105600640 1 -- 172.21.15.46:0/698496586 >> v1:172.21.15.46:6789/0 conn(0x7f710015f100 legacy=0x7f7100070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f7105600640 1 -- 172.21.15.46:0/698496586 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710006dd80 con 0x7f71000d3550 2024-04-26T23:44:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.940+0000 7f70ff400640 1 -- 172.21.15.46:0/698496586 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632683517 0 0) 0x7f710006d7c0 con 0x7f710015f100 2024-04-26T23:44:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7105600640 1 --2- 172.21.15.46:0/698496586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71000d3550 0x7f710006e580 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32378f3a725b0d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f70ff400640 1 -- 172.21.15.46:0/698496586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8037070 con 0x7f71000d3550 2024-04-26T23:44:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f70ff400640 1 -- 172.21.15.46:0/698496586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f8032640 con 0x7f71000d3550 2024-04-26T23:44:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f70ff400640 1 -- 172.21.15.46:0/698496586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8032a10 con 0x7f71000d3550 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7107600640 1 -- 172.21.15.46:0/698496586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71000d3550 msgr2=0x7f710006e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7107600640 1 --2- 172.21.15.46:0/698496586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71000d3550 0x7f710006e580 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f802a2c0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7107600640 1 -- 172.21.15.46:0/698496586 shutdown_connections 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7107600640 1 --2- 172.21.15.46:0/698496586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71000d3550 0x7f710006e580 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.942+0000 7f7107600640 1 -- 172.21.15.46:0/698496586 >> 172.21.15.46:0/698496586 conn(0x7f710009ba70 msgr2=0x7f710009be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.943+0000 7f7107600640 1 -- 172.21.15.46:0/698496586 shutdown_connections 2024-04-26T23:44:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.943+0000 7f7107600640 1 -- 172.21.15.46:0/698496586 wait complete. 2024-04-26T23:44:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.944+0000 7f7107600640 1 Processor -- start 2024-04-26T23:44:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.945+0000 7f7107600640 1 -- start start 2024-04-26T23:44:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 0x7f71000b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71000d3550 0x7f71001f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71000715d0 con 0x7f710015f100 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7100071450 con 0x7f71000d04e0 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7107600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f710015f4e0 con 0x7f71000d3550 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44170/0 (socket says 172.21.15.46:44170) 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 -- 172.21.15.46:0/774398740 learned_addr learned my addr 172.21.15.46:0/774398740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7104c00640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 0x7f71000b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 -- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71000d3550 msgr2=0x7f71001f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71000d3550 0x7f71001f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 -- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 msgr2=0x7f71000b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 0x7f71000b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7105600640 1 -- 172.21.15.46:0/774398740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71001f8930 con 0x7f710015f100 2024-04-26T23:44:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.946+0000 7f7104c00640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 0x7f71000b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f7105600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8041020 con 0x7f710015f100 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f80308f0 con 0x7f710015f100 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71001f8bd0 con 0x7f710015f100 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8037040 con 0x7f710015f100 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.947+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001f9190 con 0x7f710015f100 2024-04-26T23:44:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.948+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710006c840 con 0x7f710015f100 2024-04-26T23:44:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.949+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70f8044020 con 0x7f710015f100 2024-04-26T23:44:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.950+0000 7f70fcc00640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.950+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f70f80ac3d0 con 0x7f710015f100 2024-04-26T23:44:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.951+0000 7f7104c00640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.952+0000 7f7104c00640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 0x7f70dc06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000f60 tx=0x7f70f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:27.955+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f710006c840 con 0x7f710015f100 2024-04-26T23:44:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.163+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f710006f450 con 0x7f710015f100 2024-04-26T23:44:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.165+0000 7f70fcc00640 1 -- 172.21.15.46:0/774398740 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+17040 (secure 0 0 0) 0x7f70f804b070 con 0x7f710015f100 2024-04-26T23:44:28.171 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:28.171 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:02.386476+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:43:46.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T23:43:47.711290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6837/2319300947":"2024-04-27T23:44:02.372699+0000","172.21.15.142:6835/2319300947":"2024-04-27T23:44:02.372699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 msgr2=0x7f70dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 0x7f70dc06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000f60 tx=0x7f70f0009040 comp rx=0 tx=0).stop 2024-04-26T23:44:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 msgr2=0x7f71000b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f80046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 shutdown_connections 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f710015f100 0x7f71000b4190 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71000d3550 0x7f71001f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 --2- 172.21.15.46:0/774398740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71000d04e0 0x7f71000b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 >> 172.21.15.46:0/774398740 conn(0x7f710009ba70 msgr2=0x7f710009bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 shutdown_connections 2024-04-26T23:44:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.169+0000 7f7107600640 1 -- 172.21.15.46:0/774398740 wait complete. 2024-04-26T23:44:28.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:44:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.402+0000 7f4c1b000640 1 Processor -- start 2024-04-26T23:44:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.402+0000 7f4c1b000640 1 -- start start 2024-04-26T23:44:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c140ccdd0 0x7f4c140cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1414c6c0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4c140cde60 con 0x7f4c140ccdd0 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4c140cb8b0 con 0x7f4c140cd9f0 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c19200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c140ccdd0 0x7f4c140cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c19c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4c1414c2e0 0x7f4c140cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41588/0 (socket says 172.21.15.46:41588) 2024-04-26T23:44:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c19c00640 1 -- 172.21.15.46:0/300521938 learned_addr learned my addr 172.21.15.46:0/300521938 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.403+0000 7f4c1a600640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c19200640 1 -- 172.21.15.46:0/300521938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 msgr2=0x7f4c140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c19200640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c19200640 1 -- 172.21.15.46:0/300521938 >> v1:172.21.15.46:6789/0 conn(0x7f4c1414c2e0 legacy=0x7f4c140cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c19200640 1 -- 172.21.15.46:0/300521938 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140cd260 con 0x7f4c140ccdd0 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c13e00640 1 -- 172.21.15.46:0/300521938 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141057513 0 0) 0x7f4c1414c6c0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.404+0000 7f4c1a600640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c19200640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c140ccdd0 0x7f4c140cd4b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097a0 tx=0x7f4c0002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6579c358a455031 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c13e00640 1 -- 172.21.15.46:0/300521938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00037070 con 0x7f4c140ccdd0 2024-04-26T23:44:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c13e00640 1 -- 172.21.15.46:0/300521938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0002ae50 con 0x7f4c140ccdd0 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c13e00640 1 -- 172.21.15.46:0/300521938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00032660 con 0x7f4c140ccdd0 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 -- 172.21.15.46:0/300521938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c140ccdd0 msgr2=0x7f4c140cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c140ccdd0 0x7f4c140cd4b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097a0 tx=0x7f4c0002a100 comp rx=0 tx=0).stop 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 -- 172.21.15.46:0/300521938 shutdown_connections 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 --2- 172.21.15.46:0/300521938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c140ccdd0 0x7f4c140cd4b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 -- 172.21.15.46:0/300521938 >> 172.21.15.46:0/300521938 conn(0x7f4c140b26d0 msgr2=0x7f4c140b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 -- 172.21.15.46:0/300521938 shutdown_connections 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.406+0000 7f4c1b000640 1 -- 172.21.15.46:0/300521938 wait complete. 2024-04-26T23:44:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.408+0000 7f4c1b000640 1 Processor -- start 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.408+0000 7f4c1b000640 1 -- start start 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c140ccdd0 0x7f4c1414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14167330 con 0x7f4c1414c2e0 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14167050 con 0x7f4c140ccdd0 2024-04-26T23:44:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4c14167610 con 0x7f4c140cd9f0 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44198/0 (socket says 172.21.15.46:44198) 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.409+0000 7f4c1a600640 1 -- 172.21.15.46:0/500605451 learned_addr learned my addr 172.21.15.46:0/500605451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c19c00640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c140ccdd0 0x7f4c1414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c19200640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 -- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 msgr2=0x7f4c140730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 -- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c140ccdd0 msgr2=0x7f4c1414aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c140ccdd0 0x7f4c1414aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 -- 172.21.15.46:0/500605451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1414b200 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.410+0000 7f4c1a600640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00cef0 tx=0x7f4c0c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.411+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c01a070 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.411+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c00d860 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.411+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00dc50 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.411+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1414b4e0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.411+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1414baa0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.412+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c140693b0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.414+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c0c01d070 con 0x7f4c1414c2e0 2024-04-26T23:44:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.414+0000 7f4c11600640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 0x7f4bf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.414+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f4c0c08c660 con 0x7f4c1414c2e0 2024-04-26T23:44:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.416+0000 7f4c19c00640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 0x7f4bf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.416+0000 7f4c19c00640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 0x7f4bf406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000f60 tx=0x7f4c08007b90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.420+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c0c05a8d0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.671+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4c140b2440 con 0x7f4c1414c2e0 2024-04-26T23:44:28.677 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:44:28.672+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:44:28.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.676+0000 7f4c11600640 1 -- 172.21.15.46:0/500605451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7f4c0c05a1d0 con 0x7f4c1414c2e0 2024-04-26T23:44:28.680 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:44:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.679+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 msgr2=0x7f4bf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.679+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 0x7f4bf406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000f60 tx=0x7f4c08007b90 comp rx=0 tx=0).stop 2024-04-26T23:44:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.679+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 msgr2=0x7f4c14154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.679+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00cef0 tx=0x7f4c0c00a710 comp rx=0 tx=0).stop 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 shutdown_connections 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4bf406c920 0x7f4bf406ede0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c1414c2e0 0x7f4c14154a20 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c140cd9f0 0x7f4c140730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 --2- 172.21.15.46:0/500605451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c140ccdd0 0x7f4c1414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 >> 172.21.15.46:0/500605451 conn(0x7f4c140b26d0 msgr2=0x7f4c140b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:28.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.680+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 shutdown_connections 2024-04-26T23:44:28.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.681+0000 7f4c1b000640 1 -- 172.21.15.46:0/500605451 wait complete. 2024-04-26T23:44:28.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:44:28.707 INFO:tasks.ceph.mds.c.smithi046.stderr: -2650> 2024-04-26T23:44:09.398+0000 7f76aa000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:44:28.798 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:44:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.896+0000 7efd7c000640 1 Processor -- start 2024-04-26T23:44:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.896+0000 7efd7c000640 1 -- start start 2024-04-26T23:44:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd7c000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd74072620 0x7efd740b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd7c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd74073120 con 0x7efd740b5730 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd7c000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd74159bc0 con 0x7efd741597e0 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd7c000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efd740aeff0 con 0x7efd74072620 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd78c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efd740b5730 0x7efd74071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41600/0 (socket says 172.21.15.46:41600) 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd78c00640 1 -- 172.21.15.46:0/3469318629 learned_addr learned my addr 172.21.15.46:0/3469318629 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd7a000640 1 --2- 172.21.15.46:0/3469318629 >> v2:172.21.15.142:3300/0 conn(0x7efd74072620 0x7efd740b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.898+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465407323 0 0) 0x7efd74073120 con 0x7efd740b5730 2024-04-26T23:44:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd5c003520 con 0x7efd740b5730 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089966708 0 0) 0x7efd5c003520 con 0x7efd740b5730 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 >> v2:172.21.15.142:3300/0 conn(0x7efd74072620 msgr2=0x7efd740b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 --2- 172.21.15.46:0/3469318629 >> v2:172.21.15.142:3300/0 conn(0x7efd74072620 0x7efd740b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 >> v1:172.21.15.142:6789/0 conn(0x7efd741597e0 legacy=0x7efd740ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74072b60 con 0x7efd740b5730 2024-04-26T23:44:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7efd60001670 con 0x7efd740b5730 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 >> v1:172.21.15.46:6789/0 conn(0x7efd740b5730 legacy=0x7efd74071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 0x7efd5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd740aeff0 con 0x7efd5c003a00 2024-04-26T23:44:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c00c0a0 con 0x7efd5c0051c0 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.899+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c00c2a0 con 0x7efd5c004800 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 0x7efd5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd7a000640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd78c00640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 -- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd5c004800 msgr2=0x7efd5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 -- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c0051c0 msgr2=0x7efd5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd79600640 1 -- 172.21.15.46:0/3469318629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c003520 con 0x7efd5c003a00 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.900+0000 7efd7a000640 1 -- 172.21.15.46:0/3469318629 reap_dead start 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.902+0000 7efd79600640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 0x7efd5c003e20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7efd68001750 tx=0x7efd5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a817bdfcbd17da19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.902+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68001370 con 0x7efd5c003a00 2024-04-26T23:44:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.902+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd680052d0 con 0x7efd5c003a00 2024-04-26T23:44:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.902+0000 7efd73e00640 1 -- 172.21.15.46:0/3469318629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6802ab40 con 0x7efd5c003a00 2024-04-26T23:44:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 -- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 msgr2=0x7efd5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 0x7efd5c003e20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7efd68001750 tx=0x7efd5c005830 comp rx=0 tx=0).stop 2024-04-26T23:44:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 -- 172.21.15.46:0/3469318629 shutdown_connections 2024-04-26T23:44:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 --2- 172.21.15.46:0/3469318629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5c003a00 0x7efd5c003e20 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 -- 172.21.15.46:0/3469318629 >> 172.21.15.46:0/3469318629 conn(0x7efd7406c980 msgr2=0x7efd740b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 -- 172.21.15.46:0/3469318629 shutdown_connections 2024-04-26T23:44:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.903+0000 7efd7c000640 1 -- 172.21.15.46:0/3469318629 wait complete. 2024-04-26T23:44:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.905+0000 7efd7c000640 1 Processor -- start 2024-04-26T23:44:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.905+0000 7efd7c000640 1 -- start start 2024-04-26T23:44:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c003a00 0x7efd740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd741563a0 0x7efd74152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd740b43b0 con 0x7efd741eb490 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd740b40d0 con 0x7efd5c003a00 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.906+0000 7efd7c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efd740b5bc0 con 0x7efd741563a0 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44222/0 (socket says 172.21.15.46:44222) 2024-04-26T23:44:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 -- 172.21.15.46:0/3456095659 learned_addr learned my addr 172.21.15.46:0/3456095659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 -- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd741563a0 msgr2=0x7efd74152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd741563a0 0x7efd74152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 -- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c003a00 msgr2=0x7efd740cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c003a00 0x7efd740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 -- 172.21.15.46:0/3456095659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd740ce450 con 0x7efd741eb490 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.907+0000 7efd7a000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7efd740b06b0 tx=0x7efd6c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.908+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c016620 con 0x7efd741eb490 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.908+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c01b070 con 0x7efd741eb490 2024-04-26T23:44:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.908+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c016a20 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.908+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd740ce730 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.908+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd740cecf0 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.909+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd74069530 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.910+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd6c005d10 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.912+0000 7efd71600640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 0x7efd5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.912+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6273+0+0 (secure 0 0 0) 0x7efd6c08c870 con 0x7efd741eb490 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.912+0000 7efd79600640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 0x7efd5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.913+0000 7efd79600640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 0x7efd5406ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd68032000 tx=0x7efd6802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:28.917+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd6c025020 con 0x7efd741eb490 2024-04-26T23:44:29.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.160+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efd741586d0 con 0x7efd741eb490 2024-04-26T23:44:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.680+0000 7efd71600640 1 -- 172.21.15.46:0/3456095659 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) ==== 104+0+0 (secure 0 0 0) 0x7efd6c05aaf0 con 0x7efd741eb490 2024-04-26T23:44:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.684+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 msgr2=0x7efd5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.684+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 0x7efd5406ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd68032000 tx=0x7efd6802c000 comp rx=0 tx=0).stop 2024-04-26T23:44:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 msgr2=0x7efd741eb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7efd740b06b0 tx=0x7efd6c00f520 comp rx=0 tx=0).stop 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 shutdown_connections 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efd5406c920 0x7efd5406ede0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd741eb490 0x7efd741eb910 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd741563a0 0x7efd74152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 --2- 172.21.15.46:0/3456095659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5c003a00 0x7efd740cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 >> 172.21.15.46:0/3456095659 conn(0x7efd7406c980 msgr2=0x7efd740af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 shutdown_connections 2024-04-26T23:44:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.685+0000 7efd7c000640 1 -- 172.21.15.46:0/3456095659 wait complete. 2024-04-26T23:44:29.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:44:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.920+0000 7f4864c00640 1 Processor -- start 2024-04-26T23:44:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.920+0000 7f4864c00640 1 -- start start 2024-04-26T23:44:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.921+0000 7f4864c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f486015f6c0 0x7f48600d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.921+0000 7f4864c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48600d5820 0x7f48600d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.921+0000 7f4864c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48600d0fe0 con 0x7f48600d36f0 2024-04-26T23:44:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.921+0000 7f4864c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f48600d5d00 con 0x7f48600d5820 2024-04-26T23:44:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.921+0000 7f4864c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f48600cffc0 con 0x7f486015f6c0 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48600d5820 0x7f48600d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f48600d36f0 0x7f48600b1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41610/0 (socket says 172.21.15.46:41610) 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485e800640 1 -- 172.21.15.46:0/3933486182 learned_addr learned my addr 172.21.15.46:0/3933486182 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485de00640 1 --2- 172.21.15.46:0/3933486182 >> v2:172.21.15.142:3300/0 conn(0x7f486015f6c0 0x7f48600d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485d400640 1 -- 172.21.15.46:0/3933486182 >> v2:172.21.15.142:3300/0 conn(0x7f486015f6c0 msgr2=0x7f48600d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.922+0000 7f485d400640 1 --2- 172.21.15.46:0/3933486182 >> v2:172.21.15.142:3300/0 conn(0x7f486015f6c0 0x7f48600d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.923+0000 7f485d400640 1 -- 172.21.15.46:0/3933486182 >> v1:172.21.15.46:6789/0 conn(0x7f48600d36f0 legacy=0x7f48600b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.923+0000 7f485d400640 1 -- 172.21.15.46:0/3933486182 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600d02a0 con 0x7f48600d5820 2024-04-26T23:44:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f485d400640 1 --2- 172.21.15.46:0/3933486182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d5820 0x7f48600d0aa0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f486008df70 tx=0x7f484802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e5d157e690fd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f485ca00640 1 -- 172.21.15.46:0/3933486182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848037070 con 0x7f48600d5820 2024-04-26T23:44:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f485ca00640 1 -- 172.21.15.46:0/3933486182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4848032440 con 0x7f48600d5820 2024-04-26T23:44:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f485ca00640 1 -- 172.21.15.46:0/3933486182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848032810 con 0x7f48600d5820 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f4864c00640 1 -- 172.21.15.46:0/3933486182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d5820 msgr2=0x7f48600d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.924+0000 7f4864c00640 1 --2- 172.21.15.46:0/3933486182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d5820 0x7f48600d0aa0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f486008df70 tx=0x7f484802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 -- 172.21.15.46:0/3933486182 shutdown_connections 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 --2- 172.21.15.46:0/3933486182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d5820 0x7f48600d0aa0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 --2- 172.21.15.46:0/3933486182 >> v2:172.21.15.142:3300/0 conn(0x7f486015f6c0 0x7f48600d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 -- 172.21.15.46:0/3933486182 >> 172.21.15.46:0/3933486182 conn(0x7f486009e450 msgr2=0x7f48600d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 -- 172.21.15.46:0/3933486182 shutdown_connections 2024-04-26T23:44:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.925+0000 7f4864c00640 1 -- 172.21.15.46:0/3933486182 wait complete. 2024-04-26T23:44:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.927+0000 7f4864c00640 1 Processor -- start 2024-04-26T23:44:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.927+0000 7f4864c00640 1 -- start start 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48600d5820 0x7f48601f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f486015f6c0 0x7f486015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48600cfda0 con 0x7f48600d36f0 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48600cfc20 con 0x7f486015f6c0 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f4864c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48600d30b0 con 0x7f48600d5820 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44234/0 (socket says 172.21.15.46:44234) 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 -- 172.21.15.46:0/1214920147 learned_addr learned my addr 172.21.15.46:0/1214920147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485e800640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f486015f6c0 0x7f486015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 -- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48600d5820 msgr2=0x7f48601f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48600d5820 0x7f48601f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 -- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f486015f6c0 msgr2=0x7f486015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.928+0000 7f485de00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f486015f6c0 0x7f486015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f485de00640 1 -- 172.21.15.46:0/1214920147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600cd230 con 0x7f48600d36f0 2024-04-26T23:44:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f485de00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4850007f20 tx=0x7f485000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850014620 con 0x7f48600d36f0 2024-04-26T23:44:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4850019070 con 0x7f48600d36f0 2024-04-26T23:44:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48600cd510 con 0x7f48600d36f0 2024-04-26T23:44:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850014a00 con 0x7f48600d36f0 2024-04-26T23:44:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.929+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48600cddb0 con 0x7f48600d36f0 2024-04-26T23:44:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.930+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48600cd6e0 con 0x7f48600d36f0 2024-04-26T23:44:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.932+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4850014ba0 con 0x7f48600d36f0 2024-04-26T23:44:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.932+0000 7f4856000640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.933+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6273+0+0 (secure 0 0 0) 0x7f485008c750 con 0x7f48600d36f0 2024-04-26T23:44:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.933+0000 7f485d400640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.934+0000 7f485d400640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 0x7f483c06edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4848034000 tx=0x7f484802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:29.937+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f485005a8a0 con 0x7f48600d36f0 2024-04-26T23:44:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.142+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4860069530 con 0x7f48600d36f0 2024-04-26T23:44:30.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.691+0000 7f4856000640 1 -- 172.21.15.46:0/1214920147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7f485005a1a0 con 0x7f48600d36f0 2024-04-26T23:44:30.695 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.694+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 msgr2=0x7f483c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.694+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 0x7f483c06edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4848034000 tx=0x7f484802c000 comp rx=0 tx=0).stop 2024-04-26T23:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.694+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 msgr2=0x7f48600cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.694+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4850007f20 tx=0x7f485000e570 comp rx=0 tx=0).stop 2024-04-26T23:44:30.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 shutdown_connections 2024-04-26T23:44:30.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f486015f6c0 0x7f486015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48600d5820 0x7f48601f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 --2- 172.21.15.46:0/1214920147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48600d36f0 0x7f48600cc930 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 >> 172.21.15.46:0/1214920147 conn(0x7f486009e450 msgr2=0x7f48600b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.695+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 shutdown_connections 2024-04-26T23:44:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.696+0000 7f4864c00640 1 -- 172.21.15.46:0/1214920147 wait complete. 2024-04-26T23:44:30.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:44:30.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.922+0000 7fcd2a600640 1 Processor -- start 2024-04-26T23:44:30.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.922+0000 7fcd2a600640 1 -- start start 2024-04-26T23:44:30.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.923+0000 7fcd2a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd241627f0 0x7fcd240cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.923+0000 7fcd2a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd240d2080 0x7fcd240d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.923+0000 7fcd2a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd240d01a0 con 0x7fcd240d3af0 2024-04-26T23:44:30.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.923+0000 7fcd2a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcd240d0480 con 0x7fcd241627f0 2024-04-26T23:44:30.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.923+0000 7fcd2a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcd240d0760 con 0x7fcd240d2080 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd240d3af0 0x7fcd24162f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41612/0 (socket says 172.21.15.46:41612) 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29c00640 1 -- 172.21.15.46:0/3465841273 learned_addr learned my addr 172.21.15.46:0/3465841273 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29200640 1 --2- 172.21.15.46:0/3465841273 >> v2:172.21.15.46:3300/0 conn(0x7fcd241627f0 0x7fcd240cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd23400640 1 -- 172.21.15.46:0/3465841273 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921521121 0 0) 0x7fcd240d01a0 con 0x7fcd240d3af0 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd23e00640 1 --2- 172.21.15.46:0/3465841273 >> v2:172.21.15.142:3300/0 conn(0x7fcd240d2080 0x7fcd240d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd23400640 1 -- 172.21.15.46:0/3465841273 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd08003520 con 0x7fcd240d3af0 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29200640 1 -- 172.21.15.46:0/3465841273 >> v2:172.21.15.142:3300/0 conn(0x7fcd240d2080 msgr2=0x7fcd240d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29200640 1 --2- 172.21.15.46:0/3465841273 >> v2:172.21.15.142:3300/0 conn(0x7fcd240d2080 0x7fcd240d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.924+0000 7fcd29200640 1 -- 172.21.15.46:0/3465841273 >> v1:172.21.15.46:6789/0 conn(0x7fcd240d3af0 legacy=0x7fcd24162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.925+0000 7fcd29200640 1 -- 172.21.15.46:0/3465841273 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd240d0a40 con 0x7fcd241627f0 2024-04-26T23:44:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd29200640 1 --2- 172.21.15.46:0/3465841273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd240cfbc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fcd2408f8a0 tx=0x7fcd0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb76234c243a2db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd23400640 1 -- 172.21.15.46:0/3465841273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c037070 con 0x7fcd241627f0 2024-04-26T23:44:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd23400640 1 -- 172.21.15.46:0/3465841273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c032440 con 0x7fcd241627f0 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd23400640 1 -- 172.21.15.46:0/3465841273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c032810 con 0x7fcd241627f0 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd2a600640 1 -- 172.21.15.46:0/3465841273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 msgr2=0x7fcd240cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.926+0000 7fcd2a600640 1 --2- 172.21.15.46:0/3465841273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd240cfbc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fcd2408f8a0 tx=0x7fcd0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 -- 172.21.15.46:0/3465841273 shutdown_connections 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 --2- 172.21.15.46:0/3465841273 >> v2:172.21.15.142:3300/0 conn(0x7fcd240d2080 0x7fcd240d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 --2- 172.21.15.46:0/3465841273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd240cfbc0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 -- 172.21.15.46:0/3465841273 >> 172.21.15.46:0/3465841273 conn(0x7fcd2409e380 msgr2=0x7fcd240d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 -- 172.21.15.46:0/3465841273 shutdown_connections 2024-04-26T23:44:30.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.927+0000 7fcd2a600640 1 -- 172.21.15.46:0/3465841273 wait complete. 2024-04-26T23:44:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.929+0000 7fcd2a600640 1 Processor -- start 2024-04-26T23:44:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.929+0000 7fcd2a600640 1 -- start start 2024-04-26T23:44:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd240d2080 0x7fcd2415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd240d3af0 0x7fcd240ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24162bd0 con 0x7fcd241627f0 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240d24f0 con 0x7fcd240d3af0 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd2a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd240b5560 con 0x7fcd240d2080 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd29c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd29c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44248/0 (socket says 172.21.15.46:44248) 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd29c00640 1 -- 172.21.15.46:0/967120495 learned_addr learned my addr 172.21.15.46:0/967120495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd29200640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd240d2080 0x7fcd2415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.930+0000 7fcd23e00640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd240d3af0 0x7fcd240ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 -- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd240d2080 msgr2=0x7fcd2415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd240d2080 0x7fcd2415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 -- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd240d3af0 msgr2=0x7fcd240ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd240d3af0 0x7fcd240ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 -- 172.21.15.46:0/967120495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd240b7f40 con 0x7fcd241627f0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd29c00640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c460 tx=0x7fcd1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c017070 con 0x7fcd241627f0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c005ce0 con 0x7fcd241627f0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd240b8220 con 0x7fcd241627f0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0072b0 con 0x7fcd241627f0 2024-04-26T23:44:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.931+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd240b87e0 con 0x7fcd241627f0 2024-04-26T23:44:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.933+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd240693b0 con 0x7fcd241627f0 2024-04-26T23:44:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.934+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd1c01a070 con 0x7fcd241627f0 2024-04-26T23:44:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.935+0000 7fcd20c00640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.935+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5835+0+0 (secure 0 0 0) 0x7fcd1c08aed0 con 0x7fcd241627f0 2024-04-26T23:44:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.935+0000 7fcd29200640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.937+0000 7fcd29200640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 0x7fcd0006edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c034000 tx=0x7fcd0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:30.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:30.940+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd1c058c20 con 0x7fcd241627f0 2024-04-26T23:44:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.143+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcd240ce1d0 con 0x7fcd241627f0 2024-04-26T23:44:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.706+0000 7fcd20c00640 1 -- 172.21.15.46:0/967120495 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7fcd1c05cf20 con 0x7fcd241627f0 2024-04-26T23:44:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:44:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.709+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 msgr2=0x7fcd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.709+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 0x7fcd0006edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c034000 tx=0x7fcd0c02e000 comp rx=0 tx=0).stop 2024-04-26T23:44:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.709+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 msgr2=0x7fcd241f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.709+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c460 tx=0x7fcd1c00c920 comp rx=0 tx=0).stop 2024-04-26T23:44:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 shutdown_connections 2024-04-26T23:44:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd241627f0 0x7fcd241f5000 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd240d3af0 0x7fcd240ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 --2- 172.21.15.46:0/967120495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd240d2080 0x7fcd2415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.710+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 >> 172.21.15.46:0/967120495 conn(0x7fcd2409e380 msgr2=0x7fcd240b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:31.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.711+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 shutdown_connections 2024-04-26T23:44:31.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.713+0000 7fcd2a600640 1 -- 172.21.15.46:0/967120495 wait complete. 2024-04-26T23:44:31.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:44:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.935+0000 7fb932600640 1 Processor -- start 2024-04-26T23:44:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.936+0000 7fb932600640 1 -- start start 2024-04-26T23:44:31.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.937+0000 7fb932600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 0x7fb92c16a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.937+0000 7fb932600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92c0d4090 con 0x7fb92c15f840 2024-04-26T23:44:31.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.937+0000 7fb932600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92c0b4020 con 0x7fb92c0cfb60 2024-04-26T23:44:31.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.937+0000 7fb932600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c0d3880 con 0x7fb92c0d06d0 2024-04-26T23:44:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.938+0000 7fb931200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb92c15f840 0x7fb92c0d2e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50846/0 (socket says 172.21.15.46:50846) 2024-04-26T23:44:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.938+0000 7fb931200640 1 -- 172.21.15.46:0/4271880300 learned_addr learned my addr 172.21.15.46:0/4271880300 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.938+0000 7fb931c00640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 0x7fb92c16a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.938+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046429115 0 0) 0x7fb920001110 con 0x7fb92c15f840 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.938+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb914003520 con 0x7fb92c15f840 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2164304598 0 0) 0x7fb914003520 con 0x7fb92c15f840 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 msgr2=0x7fb92c16a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 0x7fb92c16a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 >> v1:172.21.15.142:6789/0 conn(0x7fb92c0cfb60 legacy=0x7fb92c0cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c0d3ad0 con 0x7fb92c15f840 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb920001840 con 0x7fb92c15f840 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 >> v1:172.21.15.46:6789/0 conn(0x7fb92c15f840 legacy=0x7fb92c0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9140039e0 0x7fb914003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 0x7fb914004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb9140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0d3880 con 0x7fb914004d40 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb91400baa0 con 0x7fb9140039e0 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb91400bca0 con 0x7fb92c0d06d0 2024-04-26T23:44:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931c00640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb9140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931200640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9140039e0 0x7fb914003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931c00640 1 -- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 msgr2=0x7fb914004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931c00640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c0d06d0 0x7fb914004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931c00640 1 -- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9140039e0 msgr2=0x7fb914003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.939+0000 7fb931c00640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9140039e0 0x7fb914003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.940+0000 7fb931c00640 1 -- 172.21.15.46:0/4271880300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb914003520 con 0x7fb914004d40 2024-04-26T23:44:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.940+0000 7fb931200640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9140039e0 0x7fb914003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb931c00640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb9140051c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb924009f90 tx=0x7fb914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1df5155f12feca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924009c10 con 0x7fb914004d40 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924036070 con 0x7fb914004d40 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb92b400640 1 -- 172.21.15.46:0/4271880300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924031880 con 0x7fb914004d40 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb932600640 1 -- 172.21.15.46:0/4271880300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 msgr2=0x7fb9140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb932600640 1 --2- 172.21.15.46:0/4271880300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb9140051c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb924009f90 tx=0x7fb914005830 comp rx=0 tx=0).stop 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.941+0000 7fb931c00640 1 -- 172.21.15.46:0/4271880300 reap_dead start 2024-04-26T23:44:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.942+0000 7fb932600640 1 -- 172.21.15.46:0/4271880300 shutdown_connections 2024-04-26T23:44:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.942+0000 7fb932600640 1 -- 172.21.15.46:0/4271880300 >> 172.21.15.46:0/4271880300 conn(0x7fb92c0d4f70 msgr2=0x7fb92c0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.942+0000 7fb932600640 1 -- 172.21.15.46:0/4271880300 shutdown_connections 2024-04-26T23:44:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.942+0000 7fb932600640 1 -- 172.21.15.46:0/4271880300 wait complete. 2024-04-26T23:44:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.943+0000 7fb932600640 1 Processor -- start 2024-04-26T23:44:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 -- start start 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 0x7fb92c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb92c1598c0 0x7fb92c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0b3d00 con 0x7fb914004d40 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0b27d0 con 0x7fb92c1598c0 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.944+0000 7fb932600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c0b3e80 con 0x7fb92c1f8570 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39964/0 (socket says 172.21.15.46:39964) 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 -- 172.21.15.46:0/318808257 learned_addr learned my addr 172.21.15.46:0/318808257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931c00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb92c1598c0 0x7fb92c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb92be00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 0x7fb92c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 -- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 msgr2=0x7fb92c15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 0x7fb92c15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 -- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb92c1598c0 msgr2=0x7fb92c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb92c1598c0 0x7fb92c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb931200640 1 -- 172.21.15.46:0/318808257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c1f8d50 con 0x7fb914004d40 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.945+0000 7fb92be00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 0x7fb92c15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb931200640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb92000d8e0 tx=0x7fb92000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920019070 con 0x7fb914004d40 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9200074e0 con 0x7fb914004d40 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c1f9030 con 0x7fb914004d40 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920004a00 con 0x7fb914004d40 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.946+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c1615e0 con 0x7fb914004d40 2024-04-26T23:44:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.947+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c0693b0 con 0x7fb914004d40 2024-04-26T23:44:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.949+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb92001c070 con 0x7fb914004d40 2024-04-26T23:44:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.949+0000 7fb928c00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 0x7fb90c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.950+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fb92008b770 con 0x7fb914004d40 2024-04-26T23:44:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.950+0000 7fb92be00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 0x7fb90c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.952+0000 7fb92be00640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 0x7fb90c06ede0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb92c161320 tx=0x7fb918009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:31.954+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb92c0d4090 con 0x7fb914004d40 2024-04-26T23:44:32.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.158+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb92c0d5d10 con 0x7fb914004d40 2024-04-26T23:44:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.165+0000 7fb928c00640 1 -- 172.21.15.46:0/318808257 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fb920059b90 con 0x7fb914004d40 2024-04-26T23:44:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.167+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 msgr2=0x7fb90c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.167+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 0x7fb90c06ede0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb92c161320 tx=0x7fb918009040 comp rx=0 tx=0).stop 2024-04-26T23:44:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.167+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 msgr2=0x7fb92c0ba830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.167+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb92000d8e0 tx=0x7fb92000dda0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 shutdown_connections 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90c06c920 0x7fb90c06ede0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb92c1598c0 0x7fb92c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92c1f8570 0x7fb92c15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 --2- 172.21.15.46:0/318808257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb914004d40 0x7fb92c0ba830 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 >> 172.21.15.46:0/318808257 conn(0x7fb92c0d4f70 msgr2=0x7fb92c0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 shutdown_connections 2024-04-26T23:44:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.168+0000 7fb932600640 1 -- 172.21.15.46:0/318808257 wait complete. 2024-04-26T23:44:32.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-26T23:44:32.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.402+0000 7fb0aa600640 1 Processor -- start 2024-04-26T23:44:32.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.402+0000 7fb0aa600640 1 -- start start 2024-04-26T23:44:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0aa600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0a406f8f0 0x7fb0a40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0aa600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a40b41b0 con 0x7fb0a415bf50 2024-04-26T23:44:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0aa600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a40b4490 con 0x7fb0a4072c30 2024-04-26T23:44:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0aa600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a415c330 con 0x7fb0a406f8f0 2024-04-26T23:44:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0a9200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb0a415bf50 0x7fb0a40ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50852/0 (socket says 172.21.15.46:50852) 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.403+0000 7fb0a9200640 1 -- 172.21.15.46:0/2952698574 learned_addr learned my addr 172.21.15.46:0/2952698574 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3e00640 1 --2- 172.21.15.46:0/2952698574 >> v2:172.21.15.46:3300/0 conn(0x7fb0a406f8f0 0x7fb0a40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3400640 1 -- 172.21.15.46:0/2952698574 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189491991 0 0) 0x7fb0a40b41b0 con 0x7fb0a415bf50 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3400640 1 -- 172.21.15.46:0/2952698574 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb08c003520 con 0x7fb0a415bf50 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3e00640 1 -- 172.21.15.46:0/2952698574 >> v1:172.21.15.142:6789/0 conn(0x7fb0a4072c30 legacy=0x7fb0a40b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3e00640 1 -- 172.21.15.46:0/2952698574 >> v1:172.21.15.46:6789/0 conn(0x7fb0a415bf50 legacy=0x7fb0a40ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.404+0000 7fb0a3e00640 1 -- 172.21.15.46:0/2952698574 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a40b4770 con 0x7fb0a406f8f0 2024-04-26T23:44:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.406+0000 7fb0a3e00640 1 --2- 172.21.15.46:0/2952698574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a406f8f0 0x7fb0a40b3b20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097a0 tx=0x7fb09002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befffde434ede11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.406+0000 7fb0a3400640 1 -- 172.21.15.46:0/2952698574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090037070 con 0x7fb0a406f8f0 2024-04-26T23:44:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.406+0000 7fb0a3400640 1 -- 172.21.15.46:0/2952698574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb090032440 con 0x7fb0a406f8f0 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.406+0000 7fb0a3400640 1 -- 172.21.15.46:0/2952698574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090032810 con 0x7fb0a406f8f0 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 -- 172.21.15.46:0/2952698574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a406f8f0 msgr2=0x7fb0a40b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 --2- 172.21.15.46:0/2952698574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a406f8f0 0x7fb0a40b3b20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097a0 tx=0x7fb09002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 -- 172.21.15.46:0/2952698574 shutdown_connections 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 --2- 172.21.15.46:0/2952698574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a406f8f0 0x7fb0a40b3b20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 -- 172.21.15.46:0/2952698574 >> 172.21.15.46:0/2952698574 conn(0x7fb0a40b2820 msgr2=0x7fb0a40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 -- 172.21.15.46:0/2952698574 shutdown_connections 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.407+0000 7fb0aa600640 1 -- 172.21.15.46:0/2952698574 wait complete. 2024-04-26T23:44:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.409+0000 7fb0aa600640 1 Processor -- start 2024-04-26T23:44:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.409+0000 7fb0aa600640 1 -- start start 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 0x7fb0a4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0a4072c30 0x7fb0a4146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 0x7fb0a41548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a40b5960 con 0x7fb0a415bf50 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a40739f0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.410+0000 7fb0aa600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a40b5ae0 con 0x7fb0a4072c30 2024-04-26T23:44:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.411+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 0x7fb0a41548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.411+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 0x7fb0a41548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39992/0 (socket says 172.21.15.46:39992) 2024-04-26T23:44:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.411+0000 7fb0a9c00640 1 -- 172.21.15.46:0/659798423 learned_addr learned my addr 172.21.15.46:0/659798423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.412+0000 7fb0a9200640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 0x7fb0a4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.413+0000 7fb0a9200640 1 -- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0a4072c30 msgr2=0x7fb0a4146460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.413+0000 7fb0a9200640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0a4072c30 0x7fb0a4146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.413+0000 7fb0a9200640 1 -- 172.21.15.46:0/659798423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 msgr2=0x7fb0a41548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.413+0000 7fb0a9200640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 0x7fb0a41548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.413+0000 7fb0a9200640 1 -- 172.21.15.46:0/659798423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a41585d0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.414+0000 7fb0a9200640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 0x7fb0a4157e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c390 tx=0x7fb098001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.418+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098017070 con 0x7fb0a406f8f0 2024-04-26T23:44:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.418+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb098005ce0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.418+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0980072d0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.418+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a41588b0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.418+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a4158e70 con 0x7fb0a406f8f0 2024-04-26T23:44:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.420+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a40693b0 con 0x7fb0a406f8f0 2024-04-26T23:44:32.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.424+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb09801a070 con 0x7fb0a406f8f0 2024-04-26T23:44:32.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.425+0000 7fb0a0c00640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 0x7fb08406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:32.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.425+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fb09808b180 con 0x7fb0a406f8f0 2024-04-26T23:44:32.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.426+0000 7fb0a3e00640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 0x7fb08406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.428+0000 7fb0a3e00640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 0x7fb08406ee00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb090034000 tx=0x7fb09002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.447+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb098059060 con 0x7fb0a406f8f0 2024-04-26T23:44:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:32.724+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) -- 0x7fb0a40b00a0 con 0x7fb0a406f8f0 2024-04-26T23:44:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.726+0000 7fb0a0c00640 1 -- 172.21.15.46:0/659798423 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v366) ==== 143+0+0 (secure 0 0 0) 0x7fb09805d360 con 0x7fb0a406f8f0 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 msgr2=0x7fb08406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 0x7fb08406ee00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb090034000 tx=0x7fb09002c000 comp rx=0 tx=0).stop 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 msgr2=0x7fb0a4157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 0x7fb0a4157e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c390 tx=0x7fb098001e70 comp rx=0 tx=0).stop 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 shutdown_connections 2024-04-26T23:44:33.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb08406c940 0x7fb08406ee00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0a415bf50 0x7fb0a41548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0a4072c30 0x7fb0a4146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 --2- 172.21.15.46:0/659798423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0a406f8f0 0x7fb0a4157e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.729+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 >> 172.21.15.46:0/659798423 conn(0x7fb0a40b2820 msgr2=0x7fb0a40b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.730+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 shutdown_connections 2024-04-26T23:44:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.730+0000 7fb0aa600640 1 -- 172.21.15.46:0/659798423 wait complete. 2024-04-26T23:44:33.761 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-26T23:44:33.762 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_split_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) That a directory is split during scrub. 2024-04-26T23:44:33.762 INFO:tasks.cephfs_test_runner:test_scrub_split_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:44:33.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.971+0000 7fe4c2e00640 1 Processor -- start 2024-04-26T23:44:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.972+0000 7fe4c2e00640 1 -- start start 2024-04-26T23:44:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.973+0000 7fe4c2e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 0x7fe4bc070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.973+0000 7fe4c2e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4bc0ce070 0x7fe4bc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.973+0000 7fe4c2e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4bc15f980 con 0x7fe4bc0d1bc0 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.973+0000 7fe4c2e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe4bc0cdbe0 con 0x7fe4bc0ce070 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.973+0000 7fe4c2e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc0d20c0 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.974+0000 7fe4bbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 0x7fe4bc070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.974+0000 7fe4c0e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4bc0ce070 0x7fe4bc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.974+0000 7fe4bbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 0x7fe4bc070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60314/0 (socket says 172.21.15.46:60314) 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.974+0000 7fe4bbe00640 1 -- 172.21.15.46:0/410729781 learned_addr learned my addr 172.21.15.46:0/410729781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.975+0000 7fe4c0e00640 1 -- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 msgr2=0x7fe4bc070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.975+0000 7fe4c0e00640 1 --2- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 0x7fe4bc070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.975+0000 7fe4c0e00640 1 -- 172.21.15.46:0/410729781 >> v1:172.21.15.142:6789/0 conn(0x7fe4bc0d1bc0 legacy=0x7fe4bc070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.975+0000 7fe4c0e00640 1 -- 172.21.15.46:0/410729781 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc0d23a0 con 0x7fe4bc0ce070 2024-04-26T23:44:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.977+0000 7fe4c0e00640 1 --2- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0ce070 0x7fe4bc0ce520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009d90 tx=0x7fe4b402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3efc9cbb7bd795 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.977+0000 7fe4bb400640 1 -- 172.21.15.46:0/410729781 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4007740 con 0x7fe4bc0ce070 2024-04-26T23:44:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.977+0000 7fe4bb400640 1 -- 172.21.15.46:0/410729781 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b40078e0 con 0x7fe4bc0ce070 2024-04-26T23:44:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.977+0000 7fe4bb400640 1 -- 172.21.15.46:0/410729781 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4007cb0 con 0x7fe4bc0ce070 2024-04-26T23:44:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 -- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0ce070 msgr2=0x7fe4bc0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0ce070 0x7fe4bc0ce520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009d90 tx=0x7fe4b402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 -- 172.21.15.46:0/410729781 shutdown_connections 2024-04-26T23:44:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0ce070 0x7fe4bc0ce520 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/410729781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc15f5a0 0x7fe4bc070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 -- 172.21.15.46:0/410729781 >> 172.21.15.46:0/410729781 conn(0x7fe4bc0d2f40 msgr2=0x7fe4bc0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 -- 172.21.15.46:0/410729781 shutdown_connections 2024-04-26T23:44:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.978+0000 7fe4c2e00640 1 -- 172.21.15.46:0/410729781 wait complete. 2024-04-26T23:44:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.980+0000 7fe4c2e00640 1 Processor -- start 2024-04-26T23:44:33.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.981+0000 7fe4c2e00640 1 -- start start 2024-04-26T23:44:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc0ce070 0x7fe4bc15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 0x7fe4bc0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc0d07d0 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc0cd870 con 0x7fe4bc0d1bc0 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c2e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc071090 con 0x7fe4bc0ce070 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39998/0 (socket says 172.21.15.46:39998) 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4c0e00640 1 -- 172.21.15.46:0/372632673 learned_addr learned my addr 172.21.15.46:0/372632673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4bbe00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc0ce070 0x7fe4bc15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.982+0000 7fe4b3400640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 0x7fe4bc0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 -- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc0ce070 msgr2=0x7fe4bc15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc0ce070 0x7fe4bc15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 -- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 msgr2=0x7fe4bc0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 0x7fe4bc0b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 -- 172.21.15.46:0/372632673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc0b81b0 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4b3400640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 0x7fe4bc0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4c0e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009eb0 tx=0x7fe4b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.983+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4040070 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.984+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4031bc0 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.984+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4009450 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.984+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc0b8490 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.984+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc0b8a50 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.985+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4bc069530 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.986+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4b4039040 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.987+0000 7fe4b8c00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 0x7fe49806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.987+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fe4b40ad380 con 0x7fe4bc15f5a0 2024-04-26T23:44:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.988+0000 7fe4bbe00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 0x7fe49806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.989+0000 7fe4bbe00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 0x7fe49806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009ac0 tx=0x7fe4ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:33.992+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4b403ae60 con 0x7fe4bc15f5a0 2024-04-26T23:44:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.197+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4bc0764e0 con 0x7fe4bc15f5a0 2024-04-26T23:44:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.199+0000 7fe4b8c00640 1 -- 172.21.15.46:0/372632673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+13388 (secure 0 0 0) 0x7fe4b407b830 con 0x7fe4bc15f5a0 2024-04-26T23:44:34.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:34.203 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:31.691835+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/357252295":"2024-04-27T23:44:28.673922+0000","172.21.15.142:6837/2319300947":"2024-04-27T23:44:02.372699+0000","172.21.15.46:6833/357252295":"2024-04-27T23:44:28.673922+0000","172.21.15.142:6835/2319300947":"2024-04-27T23:44:02.372699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 msgr2=0x7fe49806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 0x7fe49806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009ac0 tx=0x7fe4ac008040 comp rx=0 tx=0).stop 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 msgr2=0x7fe4bc1f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009eb0 tx=0x7fe4b4004a40 comp rx=0 tx=0).stop 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 shutdown_connections 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe49806c8f0 0x7fe49806edb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15f5a0 0x7fe4bc1f83c0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4bc0d1bc0 0x7fe4bc0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 --2- 172.21.15.46:0/372632673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4bc0ce070 0x7fe4bc15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 >> 172.21.15.46:0/372632673 conn(0x7fe4bc0d2f40 msgr2=0x7fe4bc0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 shutdown_connections 2024-04-26T23:44:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.202+0000 7fe4c2e00640 1 -- 172.21.15.46:0/372632673 wait complete. 2024-04-26T23:44:34.225 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:44:34.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:44:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.436+0000 7fe72f200640 1 Processor -- start 2024-04-26T23:44:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.436+0000 7fe72f200640 1 -- start start 2024-04-26T23:44:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.437+0000 7fe72f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7280735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe728073b30 0x7fe7280b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7280ce7d0 con 0x7fe72815cec0 2024-04-26T23:44:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe7280ceab0 con 0x7fe728073b30 2024-04-26T23:44:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe7280ced90 con 0x7fe7280cdd00 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe72815cec0 0x7fe7280cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50860/0 (socket says 172.21.15.46:50860) 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.438+0000 7fe72de00640 1 -- 172.21.15.46:0/2773294652 learned_addr learned my addr 172.21.15.46:0/2773294652 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72e800640 1 --2- 172.21.15.46:0/2773294652 >> v2:172.21.15.142:3300/0 conn(0x7fe728073b30 0x7fe7280b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72d400640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7280735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607510854 0 0) 0x7fe7280ce7d0 con 0x7fe72815cec0 2024-04-26T23:44:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe714003520 con 0x7fe72815cec0 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 78431254 0 0) 0x7fe714003520 con 0x7fe72815cec0 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 msgr2=0x7fe7280735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7280735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 >> v2:172.21.15.142:3300/0 conn(0x7fe728073b30 msgr2=0x7fe7280b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 --2- 172.21.15.46:0/2773294652 >> v2:172.21.15.142:3300/0 conn(0x7fe728073b30 0x7fe7280b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7280cf070 con 0x7fe72815cec0 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe7200019f0 con 0x7fe72815cec0 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 >> v1:172.21.15.46:6789/0 conn(0x7fe72815cec0 legacy=0x7fe7280cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7140039b0 0x7fe714003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe714004800 0x7fe714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7280ceab0 con 0x7fe714004800 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7280ced90 con 0x7fe7140039b0 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.439+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe71400b9c0 con 0x7fe7280cdd00 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe714004800 0x7fe714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72de00640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7140039b0 0x7fe714003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72e800640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 -- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 msgr2=0x7fe7140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7280cdd00 0x7fe7140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 -- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7140039b0 msgr2=0x7fe714003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7140039b0 0x7fe714003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.440+0000 7fe72d400640 1 -- 172.21.15.46:0/2773294652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe714003520 con 0x7fe714004800 2024-04-26T23:44:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.441+0000 7fe72d400640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe714004800 0x7fe714004c80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe718009ac0 tx=0x7fe714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56423b50d3baec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.441+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718036070 con 0x7fe714004800 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe718031400 con 0x7fe714004800 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72ca00640 1 -- 172.21.15.46:0/2773294652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718009690 con 0x7fe714004800 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 -- 172.21.15.46:0/2773294652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe714004800 msgr2=0x7fe714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 --2- 172.21.15.46:0/2773294652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe714004800 0x7fe714004c80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe718009ac0 tx=0x7fe714005830 comp rx=0 tx=0).stop 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72e800640 1 -- 172.21.15.46:0/2773294652 reap_dead start 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 -- 172.21.15.46:0/2773294652 shutdown_connections 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 -- 172.21.15.46:0/2773294652 >> 172.21.15.46:0/2773294652 conn(0x7fe7280b27b0 msgr2=0x7fe7280b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 -- 172.21.15.46:0/2773294652 shutdown_connections 2024-04-26T23:44:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.442+0000 7fe72f200640 1 -- 172.21.15.46:0/2773294652 wait complete. 2024-04-26T23:44:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.444+0000 7fe72f200640 1 Processor -- start 2024-04-26T23:44:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.444+0000 7fe72f200640 1 -- start start 2024-04-26T23:44:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe714004800 0x7fe7280c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7281f8b00 0x7fe72815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe728073f70 con 0x7fe7281f7f60 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe72815d2a0 con 0x7fe7281f8b00 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.445+0000 7fe72f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe7280735f0 con 0x7fe714004800 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40014/0 (socket says 172.21.15.46:40014) 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 -- 172.21.15.46:0/3761763051 learned_addr learned my addr 172.21.15.46:0/3761763051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72de00640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe714004800 0x7fe7280c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72d400640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7281f8b00 0x7fe72815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 -- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe714004800 msgr2=0x7fe7280c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe714004800 0x7fe7280c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 -- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7281f8b00 msgr2=0x7fe72815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7281f8b00 0x7fe72815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.446+0000 7fe72e800640 1 -- 172.21.15.46:0/3761763051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72815f7c0 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe72e800640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe7280cfe50 tx=0x7fe724011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724019020 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724003610 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724003a40 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72815fa30 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.447+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72815fff0 con 0x7fe7281f7f60 2024-04-26T23:44:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.448+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7280693b0 con 0x7fe7281f7f60 2024-04-26T23:44:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.449+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe72401c020 con 0x7fe7281f7f60 2024-04-26T23:44:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.450+0000 7fe71e000640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 0x7fe70c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.450+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fe72408bd10 con 0x7fe7281f7f60 2024-04-26T23:44:34.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.451+0000 7fe72de00640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 0x7fe70c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.452+0000 7fe72de00640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 0x7fe70c06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe7200013d0 tx=0x7fe720009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.455+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe724091020 con 0x7fe7281f7f60 2024-04-26T23:44:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.668+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7280cfb30 con 0x7fe7281f7f60 2024-04-26T23:44:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.670+0000 7fe71e000640 1 -- 172.21.15.46:0/3761763051 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fe724023030 con 0x7fe7281f7f60 2024-04-26T23:44:34.673 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 msgr2=0x7fe70c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 0x7fe70c06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe7200013d0 tx=0x7fe720009040 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 msgr2=0x7fe7281f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe7280cfe50 tx=0x7fe724011920 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 shutdown_connections 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe70c06c8f0 0x7fe70c06edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7281f7f60 0x7fe7281f83e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7281f8b00 0x7fe72815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 --2- 172.21.15.46:0/3761763051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe714004800 0x7fe7280c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 >> 172.21.15.46:0/3761763051 conn(0x7fe7280b27b0 msgr2=0x7fe7280b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 shutdown_connections 2024-04-26T23:44:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.672+0000 7fe72f200640 1 -- 172.21.15.46:0/3761763051 wait complete. 2024-04-26T23:44:34.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags' 2024-04-26T23:44:34.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.905+0000 7f0dee200640 1 Processor -- start 2024-04-26T23:44:34.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.906+0000 7f0dee200640 1 -- start start 2024-04-26T23:44:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.907+0000 7f0dee200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0de81633a0 0x7f0de816a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.907+0000 7f0dee200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de806f430 con 0x7f0de8159950 2024-04-26T23:44:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.907+0000 7f0dee200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de80af0c0 con 0x7f0de80b48a0 2024-04-26T23:44:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.907+0000 7f0dee200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0de80af3a0 con 0x7f0de81633a0 2024-04-26T23:44:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.908+0000 7f0dece00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0de8159950 0x7f0de80ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50862/0 (socket says 172.21.15.46:50862) 2024-04-26T23:44:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.908+0000 7f0dece00640 1 -- 172.21.15.46:0/3178165774 learned_addr learned my addr 172.21.15.46:0/3178165774 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.908+0000 7f0ded800640 1 --2- 172.21.15.46:0/3178165774 >> v2:172.21.15.142:3300/0 conn(0x7f0de81633a0 0x7f0de816a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.908+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283538033 0 0) 0x7f0de806f430 con 0x7f0de8159950 2024-04-26T23:44:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.908+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dcc003520 con 0x7f0de8159950 2024-04-26T23:44:34.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2587020753 0 0) 0x7f0dcc003520 con 0x7f0de8159950 2024-04-26T23:44:34.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 >> v2:172.21.15.142:3300/0 conn(0x7f0de81633a0 msgr2=0x7f0de816a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 --2- 172.21.15.46:0/3178165774 >> v2:172.21.15.142:3300/0 conn(0x7f0de81633a0 0x7f0de816a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 >> v1:172.21.15.142:6789/0 conn(0x7f0de80b48a0 legacy=0x7f0de8162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de80af680 con 0x7f0de8159950 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0dd8001670 con 0x7f0de8159950 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 >> v1:172.21.15.46:6789/0 conn(0x7f0de8159950 legacy=0x7f0de80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0dcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc004800 0x7f0dcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0051c0 0x7f0dcc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0de80af3a0 con 0x7f0dcc0039b0 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc00c060 con 0x7f0dcc0051c0 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc00c260 con 0x7f0dcc004800 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.909+0000 7f0dece00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0dcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0ded800640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0051c0 0x7f0dcc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0ddfe00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc004800 0x7f0dcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0dece00640 1 -- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc004800 msgr2=0x7f0dcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0dece00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc004800 0x7f0dcc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0dece00640 1 -- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0051c0 msgr2=0x7f0dcc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0dece00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0051c0 0x7f0dcc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0dece00640 1 -- 172.21.15.46:0/3178165774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc003520 con 0x7f0dcc0039b0 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0ded800640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0051c0 0x7f0dcc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.910+0000 7f0ded800640 1 -- 172.21.15.46:0/3178165774 reap_dead start 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.911+0000 7f0dece00640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0dcc003dd0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c240 tx=0x7f0dcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d066f900ef54bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.911+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8007cf0 con 0x7f0dcc0039b0 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.911+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd8005af0 con 0x7f0dcc0039b0 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.911+0000 7f0de7e00640 1 -- 172.21.15.46:0/3178165774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8007070 con 0x7f0dcc0039b0 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 -- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 msgr2=0x7f0dcc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0dcc003dd0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c240 tx=0x7f0dcc005830 comp rx=0 tx=0).stop 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 -- 172.21.15.46:0/3178165774 shutdown_connections 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 --2- 172.21.15.46:0/3178165774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0dcc003dd0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 -- 172.21.15.46:0/3178165774 >> 172.21.15.46:0/3178165774 conn(0x7f0de80b2820 msgr2=0x7f0de80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 -- 172.21.15.46:0/3178165774 shutdown_connections 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.912+0000 7f0dee200640 1 -- 172.21.15.46:0/3178165774 wait complete. 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.914+0000 7f0dee200640 1 Processor -- start 2024-04-26T23:44:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.914+0000 7f0dee200640 1 -- start start 2024-04-26T23:44:34.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0de8145880 0x7f0de8162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0de81545b0 0x7f0de8154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0de80b0160 con 0x7f0dcc0039b0 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0de80afc40 con 0x7f0de81545b0 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dee200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0de8162c90 con 0x7f0de8145880 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 -- 172.21.15.46:0/1304928276 learned_addr learned my addr 172.21.15.46:0/1304928276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0ddfe00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0de8145880 0x7f0de8162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0ded800640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0de81545b0 0x7f0de8154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 -- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0de8145880 msgr2=0x7f0de8162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0de8145880 0x7f0de8162610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 -- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0de81545b0 msgr2=0x7f0de8154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.915+0000 7f0dece00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0de81545b0 0x7f0de8154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0dece00640 1 -- 172.21.15.46:0/1304928276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de8160a50 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0dece00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8001970 tx=0x7f0de80af290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8043070 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd8001630 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de8160d30 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8037040 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.916+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de8161810 con 0x7f0dcc0039b0 2024-04-26T23:44:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.917+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de80693b0 con 0x7f0dcc0039b0 2024-04-26T23:44:34.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.918+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dd8046070 con 0x7f0dcc0039b0 2024-04-26T23:44:34.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.919+0000 7f0de5600640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:34.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.920+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7f0dd80aae40 con 0x7f0dcc0039b0 2024-04-26T23:44:34.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.920+0000 7f0ddfe00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:34.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.921+0000 7f0ddfe00640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 0x7f0dc406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:34.924+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0dd80792e0 con 0x7f0dcc0039b0 2024-04-26T23:44:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.215+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags"]} v 0) -- 0x7f0de80b35e0 con 0x7f0dcc0039b0 2024-04-26T23:44:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.733+0000 7f0de5600640 1 -- 172.21.15.46:0/1304928276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags"]}]=0 v368) ==== 154+0+0 (secure 0 0 0) 0x7f0dd8078be0 con 0x7f0dcc0039b0 2024-04-26T23:44:35.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.736+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 msgr2=0x7f0dc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.736+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 0x7f0dc406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0009040 comp rx=0 tx=0).stop 2024-04-26T23:44:35.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.736+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 msgr2=0x7f0de81620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.736+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8001970 tx=0x7f0de80af290 comp rx=0 tx=0).stop 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 shutdown_connections 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0de81545b0 0x7f0de8154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0de8145880 0x7f0de8162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 --2- 172.21.15.46:0/1304928276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0039b0 0x7f0de81620d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 >> 172.21.15.46:0/1304928276 conn(0x7f0de80b2820 msgr2=0x7f0de806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 shutdown_connections 2024-04-26T23:44:35.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.737+0000 7f0dee200640 1 -- 172.21.15.46:0/1304928276 wait complete. 2024-04-26T23:44:35.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:44:35.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.973+0000 7fd5e9000640 1 Processor -- start 2024-04-26T23:44:35.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.973+0000 7fd5e9000640 1 -- start start 2024-04-26T23:44:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5e41644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e41649f0 con 0x7fd5e4073af0 2024-04-26T23:44:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e4071e80 con 0x7fd5e40ce7d0 2024-04-26T23:44:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40cb4f0 con 0x7fd5e41640d0 2024-04-26T23:44:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd5e4073af0 0x7fd5e40cb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50876/0 (socket says 172.21.15.46:50876) 2024-04-26T23:44:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.975+0000 7fd5e2a00640 1 -- 172.21.15.46:0/2737509663 learned_addr learned my addr 172.21.15.46:0/2737509663 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e3400640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5e41644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1424645066 0 0) 0x7fd5e41649f0 con 0x7fd5e4073af0 2024-04-26T23:44:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c8003520 con 0x7fd5e4073af0 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2808537456 0 0) 0x7fd5c8003520 con 0x7fd5e4073af0 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 msgr2=0x7fd5e41644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5e41644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 >> v1:172.21.15.142:6789/0 conn(0x7fd5e40ce7d0 legacy=0x7fd5e40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e4072160 con 0x7fd5e4073af0 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd5d0001670 con 0x7fd5e4073af0 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 >> v1:172.21.15.46:6789/0 conn(0x7fd5e4073af0 legacy=0x7fd5e40cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80039e0 0x7fd5c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.976+0000 7fd5e2000640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c8004d40 0x7fd5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40cb4f0 con 0x7fd5c8004d40 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c800b9c0 con 0x7fd5c80039e0 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c800bbc0 con 0x7fd5e41640d0 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c8004d40 0x7fd5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e3400640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80039e0 0x7fd5c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.977+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 msgr2=0x7fd5c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.978+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e41640d0 0x7fd5c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.978+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80039e0 msgr2=0x7fd5c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.978+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80039e0 0x7fd5c8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.978+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2737509663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c8003520 con 0x7fd5c8004d40 2024-04-26T23:44:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.978+0000 7fd5e3400640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80039e0 0x7fd5c8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:44:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.979+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c8004d40 0x7fd5c80051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc002010 tx=0x7fd5c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad8f3f2278cad53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.979+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc036070 con 0x7fd5c8004d40 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.979+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc031600 con 0x7fd5c8004d40 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.979+0000 7fd5e2000640 1 -- 172.21.15.46:0/2737509663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc001be0 con 0x7fd5c8004d40 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 -- 172.21.15.46:0/2737509663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c8004d40 msgr2=0x7fd5c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2737509663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c8004d40 0x7fd5c80051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc002010 tx=0x7fd5c8005830 comp rx=0 tx=0).stop 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2737509663 reap_dead start 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 -- 172.21.15.46:0/2737509663 shutdown_connections 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 -- 172.21.15.46:0/2737509663 >> 172.21.15.46:0/2737509663 conn(0x7fd5e40b27b0 msgr2=0x7fd5e40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 -- 172.21.15.46:0/2737509663 shutdown_connections 2024-04-26T23:44:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.980+0000 7fd5e9000640 1 -- 172.21.15.46:0/2737509663 wait complete. 2024-04-26T23:44:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.982+0000 7fd5e9000640 1 Processor -- start 2024-04-26T23:44:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.982+0000 7fd5e9000640 1 -- start start 2024-04-26T23:44:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e41528c0 0x7fd5e4152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40c8130 0x7fd5e40c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40cc090 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4073f70 con 0x7fd5e41528c0 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40cc210 con 0x7fd5e40c8130 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40056/0 (socket says 172.21.15.46:40056) 2024-04-26T23:44:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.983+0000 7fd5e2a00640 1 -- 172.21.15.46:0/884594011 learned_addr learned my addr 172.21.15.46:0/884594011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40c8130 0x7fd5e40c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 -- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40c8130 msgr2=0x7fd5e40c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40c8130 0x7fd5e40c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e3400640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e41528c0 0x7fd5e4152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 -- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e41528c0 msgr2=0x7fd5e4152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e41528c0 0x7fd5e4152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 -- 172.21.15.46:0/884594011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40c7550 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e8a0 tx=0x7fd5d000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0019070 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.984+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d000c8d0 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.985+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000cca0 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.985+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e40b5580 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.985+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e40b5b40 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.986+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e40693b0 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.987+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5d001c070 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.989+0000 7fd5db400640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 0x7fd5c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.989+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fd5d008b4d0 con 0x7fd5e40c7cd0 2024-04-26T23:44:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.989+0000 7fd5e3400640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 0x7fd5c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.991+0000 7fd5e3400640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 0x7fd5c406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009930 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:35.993+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5d00598f0 con 0x7fd5e40c7cd0 2024-04-26T23:44:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.197+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd5e4164e00 con 0x7fd5e40c7cd0 2024-04-26T23:44:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.202+0000 7fd5db400640 1 -- 172.21.15.46:0/884594011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fd5d00591f0 con 0x7fd5e40c7cd0 2024-04-26T23:44:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 msgr2=0x7fd5c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 0x7fd5c406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009930 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 msgr2=0x7fd5e40c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e8a0 tx=0x7fd5d000ed60 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 shutdown_connections 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c406c8f0 0x7fd5c406edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40c8130 0x7fd5e40c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40c7cd0 0x7fd5e40c6ad0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 --2- 172.21.15.46:0/884594011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e41528c0 0x7fd5e4152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 >> 172.21.15.46:0/884594011 conn(0x7fd5e40b27b0 msgr2=0x7fd5e41662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 shutdown_connections 2024-04-26T23:44:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.205+0000 7fd5e9000640 1 -- 172.21.15.46:0/884594011 wait complete. 2024-04-26T23:44:36.229 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:44:36.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:44:36.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:44:36.449 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:44:36.449 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:36.449 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:36.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:44:36.511 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:44:36.511 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:36.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:44:36.567 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:36.567 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:36.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:44:36.625 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:44:36.626 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:36.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:44:36.681 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:44:36.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.896+0000 7fd673200640 1 Processor -- start 2024-04-26T23:44:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.897+0000 7fd673200640 1 -- start start 2024-04-26T23:44:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd673200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740d5a90 0x7fd6740d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd673200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6740b4e20 0x7fd6740d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd673200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6740b65e0 con 0x7fd674162a60 2024-04-26T23:44:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd673200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd6740d12e0 con 0x7fd6740b4e20 2024-04-26T23:44:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd673200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740d05f0 con 0x7fd6740d5a90 2024-04-26T23:44:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd671e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd674162a60 0x7fd6740b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50880/0 (socket says 172.21.15.46:50880) 2024-04-26T23:44:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.898+0000 7fd671e00640 1 -- 172.21.15.46:0/713138350 learned_addr learned my addr 172.21.15.46:0/713138350 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308742657 0 0) 0x7fd6740b65e0 con 0x7fd674162a60 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd671400640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740d5a90 0x7fd6740d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd672800640 1 --2- 172.21.15.46:0/713138350 >> v2:172.21.15.142:3300/0 conn(0x7fd6740b4e20 0x7fd6740d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd658003520 con 0x7fd674162a60 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2314831173 0 0) 0x7fd658003520 con 0x7fd674162a60 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740d5a90 msgr2=0x7fd6740d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740d5a90 0x7fd6740d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 >> v2:172.21.15.142:3300/0 conn(0x7fd6740b4e20 msgr2=0x7fd6740d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 --2- 172.21.15.46:0/713138350 >> v2:172.21.15.142:3300/0 conn(0x7fd6740b4e20 0x7fd6740d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6740d1530 con 0x7fd674162a60 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd664001670 con 0x7fd674162a60 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 >> v1:172.21.15.46:6789/0 conn(0x7fd674162a60 legacy=0x7fd6740b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6580039b0 0x7fd658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6580051c0 0x7fd65800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740d12e0 con 0x7fd658004800 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740d05f0 con 0x7fd6580051c0 2024-04-26T23:44:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.899+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd65800c140 con 0x7fd6580039b0 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671e00640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6580039b0 0x7fd658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd672800640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6580051c0 0x7fd65800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 -- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6580039b0 msgr2=0x7fd658003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6580039b0 0x7fd658003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 -- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6580051c0 msgr2=0x7fd65800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6580051c0 0x7fd65800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd671400640 1 -- 172.21.15.46:0/713138350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd658003520 con 0x7fd658004800 2024-04-26T23:44:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.900+0000 7fd672800640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6580051c0 0x7fd65800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.901+0000 7fd671400640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd658004c80 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd67408d8b0 tx=0x7fd658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f816d8f5e7a469b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.901+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660030070 con 0x7fd658004800 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.901+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66002ac90 con 0x7fd658004800 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.901+0000 7fd670a00640 1 -- 172.21.15.46:0/713138350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000b550 con 0x7fd658004800 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 -- 172.21.15.46:0/713138350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 msgr2=0x7fd658004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 --2- 172.21.15.46:0/713138350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd658004c80 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd67408d8b0 tx=0x7fd658005830 comp rx=0 tx=0).stop 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd672800640 1 -- 172.21.15.46:0/713138350 reap_dead start 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 -- 172.21.15.46:0/713138350 shutdown_connections 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 -- 172.21.15.46:0/713138350 >> 172.21.15.46:0/713138350 conn(0x7fd6740d4fa0 msgr2=0x7fd6740d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 -- 172.21.15.46:0/713138350 shutdown_connections 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.902+0000 7fd673200640 1 -- 172.21.15.46:0/713138350 wait complete. 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.904+0000 7fd673200640 1 Processor -- start 2024-04-26T23:44:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.904+0000 7fd673200640 1 -- start start 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 0x7fd674150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6740cb150 0x7fd67406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740b52d0 con 0x7fd658004800 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740d0a80 con 0x7fd6740cb150 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd673200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd674150980 con 0x7fd6740cd360 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd671e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd671e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40072/0 (socket says 172.21.15.46:40072) 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.905+0000 7fd671e00640 1 -- 172.21.15.46:0/306896608 learned_addr learned my addr 172.21.15.46:0/306896608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 -- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 msgr2=0x7fd674150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671400640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 0x7fd674150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 0x7fd674150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd672800640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6740cb150 0x7fd67406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 -- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6740cb150 msgr2=0x7fd67406d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6740cb150 0x7fd67406d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 -- 172.21.15.46:0/306896608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6740cd860 con 0x7fd658004800 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671400640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 0x7fd674150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd671e00640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd66400e780 tx=0x7fd66400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400ee90 con 0x7fd658004800 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664018070 con 0x7fd658004800 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400cab0 con 0x7fd658004800 2024-04-26T23:44:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.906+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6740cdb40 con 0x7fd658004800 2024-04-26T23:44:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.907+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6740ce100 con 0x7fd658004800 2024-04-26T23:44:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.908+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd674069ac0 con 0x7fd658004800 2024-04-26T23:44:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.909+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd66400cc50 con 0x7fd658004800 2024-04-26T23:44:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.910+0000 7fd66a000640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 0x7fd65406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.910+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fd66408b910 con 0x7fd658004800 2024-04-26T23:44:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.910+0000 7fd671400640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 0x7fd65406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.911+0000 7fd671400640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 0x7fd65406ef50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd67406f5e0 tx=0x7fd660030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:36.914+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd664059e60 con 0x7fd658004800 2024-04-26T23:44:37.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.158+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd67406cec0 con 0x7fd658004800 2024-04-26T23:44:37.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.159+0000 7fd66a000640 1 -- 172.21.15.46:0/306896608 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3875 (secure 0 0 0) 0x7fd664059760 con 0x7fd658004800 2024-04-26T23:44:37.163 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:37.163 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2764876140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2764876140},{"type":"v1","addr":"172.21.15.142:6836","nonce":2764876140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3945885857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3945885857},{"type":"v1","addr":"172.21.15.142:6837","nonce":3945885857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7431,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3349996320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3349996320},{"type":"v1","addr":"172.21.15.46:6835","nonce":3349996320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4061434395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4061434395},{"type":"v1","addr":"172.21.15.46:6833","nonce":4061434395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[]} 2024-04-26T23:44:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 68 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 msgr2=0x7fd65406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 0x7fd65406ef50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd67406f5e0 tx=0x7fd660030000 comp rx=0 tx=0).stop 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 msgr2=0x7fd6740ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd66400e780 tx=0x7fd66400ec40 comp rx=0 tx=0).stop 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 shutdown_connections 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd65406ca90 0x7fd65406ef50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6740cb150 0x7fd67406d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6740cd360 0x7fd674150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 --2- 172.21.15.46:0/306896608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd658004800 0x7fd6740ce900 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 >> 172.21.15.46:0/306896608 conn(0x7fd6740d4fa0 msgr2=0x7fd6740d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 shutdown_connections 2024-04-26T23:44:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.162+0000 7fd673200640 1 -- 172.21.15.46:0/306896608 wait complete. 2024-04-26T23:44:37.185 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:44:37.186 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:44:37.186 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:44:37.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:37.186 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:44:37.186 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:44:37.186 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:44:37.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:37.187 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:44:37.187 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:44:37.187 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:44:37.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:37.187 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:44:37.187 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:44:37.188 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:44:37.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:37.202 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:44:37.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:44:37.203 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:44:37.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:44:37.211 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:44:37.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:44:37.213 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:44:37.214 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:44:37.249 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:44:37.249 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:37.255 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:44:37.255 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:37.258 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:44:37.258 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:37.262 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:44:37.263 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:37.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:44:37.397 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:44:37.398 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:44:37.414 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:44:37.415 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:44:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.432+0000 7fd6eae00640 1 Processor -- start 2024-04-26T23:44:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.432+0000 7fd6eae00640 1 -- start start 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.433+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6e4167e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.433+0000 7fd6eae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e40ce7b0 con 0x7fd6e415d510 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.433+0000 7fd6eae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e40cea90 con 0x7fd6e40d1520 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.433+0000 7fd6eae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e40d1220 con 0x7fd6e40d3af0 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.433+0000 7fd6e3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6e415d510 0x7fd6e40d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50894/0 (socket says 172.21.15.46:50894) 2024-04-26T23:44:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e3400640 1 -- 172.21.15.46:0/2988796599 learned_addr learned my addr 172.21.15.46:0/2988796599 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955104154 0 0) 0x7fd6e40ce7b0 con 0x7fd6e415d510 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6c8003520 con 0x7fd6e415d510 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e8e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6e4167e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3972404919 0 0) 0x7fd6c8003520 con 0x7fd6e415d510 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 msgr2=0x7fd6e4167e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6e4167e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 >> v1:172.21.15.142:6789/0 conn(0x7fd6e40d1520 legacy=0x7fd6e40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e40ced70 con 0x7fd6e415d510 2024-04-26T23:44:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd6d0001670 con 0x7fd6e415d510 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 >> v1:172.21.15.46:6789/0 conn(0x7fd6e415d510 legacy=0x7fd6e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.434+0000 7fd6e2a00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e2a00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8004d40 0x7fd6c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40d1220 con 0x7fd6c80039e0 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800ba70 con 0x7fd6c8004d40 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c800bc70 con 0x7fd6e40d3af0 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3400640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e8e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8004d40 0x7fd6c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 -- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 msgr2=0x7fd6c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40d3af0 0x7fd6c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 -- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8004d40 msgr2=0x7fd6c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8004d40 0x7fd6c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.435+0000 7fd6e3e00640 1 -- 172.21.15.46:0/2988796599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c8003520 con 0x7fd6c80039e0 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.436+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6c8003e00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001720 tx=0x7fd6c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58738454ae801e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4001370 con 0x7fd6c80039e0 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d40052d0 con 0x7fd6c80039e0 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6e2a00640 1 -- 172.21.15.46:0/2988796599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d402abc0 con 0x7fd6c80039e0 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 -- 172.21.15.46:0/2988796599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 msgr2=0x7fd6c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 --2- 172.21.15.46:0/2988796599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6c8003e00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001720 tx=0x7fd6c8005830 comp rx=0 tx=0).stop 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6e8e00640 1 -- 172.21.15.46:0/2988796599 reap_dead start 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 -- 172.21.15.46:0/2988796599 shutdown_connections 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 -- 172.21.15.46:0/2988796599 >> 172.21.15.46:0/2988796599 conn(0x7fd6e40d2f40 msgr2=0x7fd6e40d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 -- 172.21.15.46:0/2988796599 shutdown_connections 2024-04-26T23:44:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.437+0000 7fd6eae00640 1 -- 172.21.15.46:0/2988796599 wait complete. 2024-04-26T23:44:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.439+0000 7fd6eae00640 1 Processor -- start 2024-04-26T23:44:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.439+0000 7fd6eae00640 1 -- start start 2024-04-26T23:44:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 0x7fd6e414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 0x7fd6e40b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40ce560 con 0x7fd6c80039e0 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4071960 con 0x7fd6e415ff90 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6eae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e4070230 con 0x7fd6e40b7630 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40118/0 (socket says 172.21.15.46:40118) 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.440+0000 7fd6e3e00640 1 -- 172.21.15.46:0/4098353485 learned_addr learned my addr 172.21.15.46:0/4098353485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e8e00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 0x7fd6e40b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3400640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 0x7fd6e414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 -- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 msgr2=0x7fd6e40b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 0x7fd6e40b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 -- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 msgr2=0x7fd6e414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 0x7fd6e414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 -- 172.21.15.46:0/4098353485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e414da30 con 0x7fd6c80039e0 2024-04-26T23:44:37.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e8e00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 0x7fd6e40b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:37.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3400640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 0x7fd6e414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.441+0000 7fd6e3e00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4008fd0 tx=0x7fd6d4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.442+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4037040 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.442+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d4005480 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.442+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4007a30 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.442+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e4160410 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.442+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e40cbae0 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.443+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4069530 con 0x7fd6c80039e0 2024-04-26T23:44:37.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.444+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6d4044020 con 0x7fd6c80039e0 2024-04-26T23:44:37.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.445+0000 7fd6dbe00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 0x7fd6c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.445+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5421+0+0 (secure 0 0 0) 0x7fd6d40ab740 con 0x7fd6c80039e0 2024-04-26T23:44:37.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.445+0000 7fd6e3400640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 0x7fd6c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.446+0000 7fd6e3400640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 0x7fd6c406ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4154be0 tx=0x7fd6d0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:37.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.449+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6d4079bf0 con 0x7fd6c80039e0 2024-04-26T23:44:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.661+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd6e40d0ee0 con 0x7fd6c80039e0 2024-04-26T23:44:37.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.764+0000 7fd6dbe00640 1 -- 172.21.15.46:0/4098353485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) ==== 97+0+0 (secure 0 0 0) 0x7fd6d40794f0 con 0x7fd6c80039e0 2024-04-26T23:44:37.768 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:44:37.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.767+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 msgr2=0x7fd6c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.767+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 0x7fd6c406ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4154be0 tx=0x7fd6d0009040 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.767+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 msgr2=0x7fd6e4147b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.767+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4008fd0 tx=0x7fd6d4001120 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 shutdown_connections 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd6c406c920 0x7fd6c406ede0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6e40b7630 0x7fd6e40b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e415ff90 0x7fd6e414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 --2- 172.21.15.46:0/4098353485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80039e0 0x7fd6e4147b40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 >> 172.21.15.46:0/4098353485 conn(0x7fd6e40d2f40 msgr2=0x7fd6e4158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 shutdown_connections 2024-04-26T23:44:37.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.768+0000 7fd6eae00640 1 -- 172.21.15.46:0/4098353485 wait complete. 2024-04-26T23:44:37.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:44:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.984+0000 7f1b17a00640 1 Processor -- start 2024-04-26T23:44:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.985+0000 7f1b17a00640 1 -- start start 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.986+0000 7f1b17a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b10162980 0x7f1b100d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.986+0000 7f1b17a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.986+0000 7f1b17a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1015fd30 con 0x7f1b100d31f0 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.986+0000 7f1b17a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b10162d60 con 0x7f1b10162980 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.986+0000 7f1b17a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b100d1050 con 0x7f1b100d2200 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b10162980 0x7f1b100d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b10162980 0x7f1b100d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40130/0 (socket says 172.21.15.46:40130) 2024-04-26T23:44:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 -- 172.21.15.46:0/3088232580 learned_addr learned my addr 172.21.15.46:0/3088232580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15a00640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 -- 172.21.15.46:0/3088232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 msgr2=0x7f1b1015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 -- 172.21.15.46:0/3088232580 >> v1:172.21.15.142:6789/0 conn(0x7f1b100d31f0 legacy=0x7f1b100d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15000640 1 -- 172.21.15.46:0/3088232580 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b10160010 con 0x7f1b10162980 2024-04-26T23:44:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.987+0000 7f1b15a00640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.988+0000 7f1b15000640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10162980 0x7f1b100d2cb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009870 tx=0x7f1b0002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd59f82506346d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b0fe00640 1 -- 172.21.15.46:0/3088232580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00037070 con 0x7f1b10162980 2024-04-26T23:44:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b0fe00640 1 -- 172.21.15.46:0/3088232580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b00032440 con 0x7f1b10162980 2024-04-26T23:44:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b0fe00640 1 -- 172.21.15.46:0/3088232580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00032810 con 0x7f1b10162980 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 -- 172.21.15.46:0/3088232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10162980 msgr2=0x7f1b100d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10162980 0x7f1b100d2cb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009870 tx=0x7f1b0002a200 comp rx=0 tx=0).stop 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 -- 172.21.15.46:0/3088232580 shutdown_connections 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 --2- 172.21.15.46:0/3088232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10162980 0x7f1b100d2cb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.989+0000 7f1b17a00640 1 -- 172.21.15.46:0/3088232580 >> 172.21.15.46:0/3088232580 conn(0x7f1b1009e420 msgr2=0x7f1b100d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.990+0000 7f1b17a00640 1 -- 172.21.15.46:0/3088232580 shutdown_connections 2024-04-26T23:44:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.990+0000 7f1b17a00640 1 -- 172.21.15.46:0/3088232580 wait complete. 2024-04-26T23:44:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.991+0000 7f1b17a00640 1 Processor -- start 2024-04-26T23:44:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.991+0000 7f1b17a00640 1 -- start start 2024-04-26T23:44:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 0x7f1b100bab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101608b0 con 0x7f1b100d31f0 2024-04-26T23:44:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101605d0 con 0x7f1b10162980 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.992+0000 7f1b17a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b10160d50 con 0x7f1b100d2200 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40140/0 (socket says 172.21.15.46:40140) 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 -- 172.21.15.46:0/2914742534 learned_addr learned my addr 172.21.15.46:0/2914742534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b15000640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b15a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 0x7f1b100bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 -- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 msgr2=0x7f1b1015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 -- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 msgr2=0x7f1b100bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 0x7f1b100bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b07e00640 1 -- 172.21.15.46:0/2914742534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b100cada0 con 0x7f1b100d31f0 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.993+0000 7f1b15a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 0x7f1b100bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b07e00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c430 tx=0x7f1af800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011790 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af8017070 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011bb0 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b100cb080 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.994+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b100cb640 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.995+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b10069530 con 0x7f1b100d31f0 2024-04-26T23:44:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.996+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1af801c460 con 0x7f1b100d31f0 2024-04-26T23:44:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.997+0000 7f1b0d600640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.997+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5249+0+0 (secure 0 0 0) 0x7f1af808b8c0 con 0x7f1b100d31f0 2024-04-26T23:44:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.998+0000 7f1b15000640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:37.999+0000 7f1b15000640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 0x7f1aec06edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1b00034000 tx=0x7f1b0002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.001+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1af8059ea0 con 0x7f1b100d31f0 2024-04-26T23:44:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.288+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b1015b800 con 0x7f1b100d31f0 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.289+0000 7f1b0d600640 1 -- 172.21.15.46:0/2914742534 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f1af80597a0 con 0x7f1b100d31f0 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:44:38.293 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.296 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.298 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:44:38.300 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.301 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:44:38.302 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:44:38.305 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 msgr2=0x7f1aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 0x7f1aec06edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1b00034000 tx=0x7f1b0002e000 comp rx=0 tx=0).stop 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 msgr2=0x7f1b100ba5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c430 tx=0x7f1af800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 shutdown_connections 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b10162980 0x7f1b100bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b100d31f0 0x7f1b100ba5e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 --2- 172.21.15.46:0/2914742534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b100d2200 0x7f1b1015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 >> 172.21.15.46:0/2914742534 conn(0x7f1b1009e420 msgr2=0x7f1b1015f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.292+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 shutdown_connections 2024-04-26T23:44:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.293+0000 7f1b17a00640 1 -- 172.21.15.46:0/2914742534 wait complete. 2024-04-26T23:44:38.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:38.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.524+0000 7f0b1d400640 1 Processor -- start 2024-04-26T23:44:38.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.524+0000 7f0b1d400640 1 -- start start 2024-04-26T23:44:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b1d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b18070030 0x7f0b180b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b1d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b1815f900 0x7f0b1815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b1d400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1815c4f0 con 0x7f0b1815ed60 2024-04-26T23:44:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b1d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0b180b4090 con 0x7f0b1815f900 2024-04-26T23:44:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b1d400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0b180b4370 con 0x7f0b18070030 2024-04-26T23:44:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b15e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0b1815ed60 0x7f0b1815f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50900/0 (socket says 172.21.15.46:50900) 2024-04-26T23:44:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b15e00640 1 -- 172.21.15.46:0/3985154890 learned_addr learned my addr 172.21.15.46:0/3985154890 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.526+0000 7f0b17200640 1 --2- 172.21.15.46:0/3985154890 >> v2:172.21.15.46:3300/0 conn(0x7f0b1815f900 0x7f0b1815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b16800640 1 --2- 172.21.15.46:0/3985154890 >> v2:172.21.15.142:3300/0 conn(0x7f0b18070030 0x7f0b180b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b15400640 1 -- 172.21.15.46:0/3985154890 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4279872015 0 0) 0x7f0b1815c4f0 con 0x7f0b1815ed60 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b17200640 1 -- 172.21.15.46:0/3985154890 >> v2:172.21.15.142:3300/0 conn(0x7f0b18070030 msgr2=0x7f0b180b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b17200640 1 --2- 172.21.15.46:0/3985154890 >> v2:172.21.15.142:3300/0 conn(0x7f0b18070030 0x7f0b180b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b17200640 1 -- 172.21.15.46:0/3985154890 >> v1:172.21.15.46:6789/0 conn(0x7f0b1815ed60 legacy=0x7f0b1815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.527+0000 7f0b17200640 1 -- 172.21.15.46:0/3985154890 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b180b4650 con 0x7f0b1815f900 2024-04-26T23:44:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.528+0000 7f0b17200640 1 --2- 172.21.15.46:0/3985154890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815f900 0x7f0b1815aba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0b1806d250 tx=0x7f0b1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e81e06c6795de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.528+0000 7f0b15400640 1 -- 172.21.15.46:0/3985154890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007740 con 0x7f0b1815f900 2024-04-26T23:44:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.528+0000 7f0b15400640 1 -- 172.21.15.46:0/3985154890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b100078e0 con 0x7f0b1815f900 2024-04-26T23:44:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.528+0000 7f0b15400640 1 -- 172.21.15.46:0/3985154890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007cb0 con 0x7f0b1815f900 2024-04-26T23:44:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 -- 172.21.15.46:0/3985154890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815f900 msgr2=0x7f0b1815aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 --2- 172.21.15.46:0/3985154890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815f900 0x7f0b1815aba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0b1806d250 tx=0x7f0b1002a680 comp rx=0 tx=0).stop 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 -- 172.21.15.46:0/3985154890 shutdown_connections 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 --2- 172.21.15.46:0/3985154890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815f900 0x7f0b1815aba0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 --2- 172.21.15.46:0/3985154890 >> v2:172.21.15.142:3300/0 conn(0x7f0b18070030 0x7f0b180b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 -- 172.21.15.46:0/3985154890 >> 172.21.15.46:0/3985154890 conn(0x7f0b1806c530 msgr2=0x7f0b1806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 -- 172.21.15.46:0/3985154890 shutdown_connections 2024-04-26T23:44:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.529+0000 7f0b1d400640 1 -- 172.21.15.46:0/3985154890 wait complete. 2024-04-26T23:44:38.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.531+0000 7f0b1d400640 1 Processor -- start 2024-04-26T23:44:38.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.531+0000 7f0b1d400640 1 -- start start 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b18070030 0x7f0b18153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1815f900 0x7f0b181647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180d0880 con 0x7f0b1815ed60 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180d0700 con 0x7f0b1815f900 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b1d400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180d0a00 con 0x7f0b18070030 2024-04-26T23:44:38.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40162/0 (socket says 172.21.15.46:40162) 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b15e00640 1 -- 172.21.15.46:0/834826179 learned_addr learned my addr 172.21.15.46:0/834826179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b16800640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b18070030 0x7f0b18153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.532+0000 7f0b17200640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1815f900 0x7f0b181647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 -- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b18070030 msgr2=0x7f0b18153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b18070030 0x7f0b18153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 -- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1815f900 msgr2=0x7f0b181647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1815f900 0x7f0b181647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 -- 172.21.15.46:0/834826179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b18153590 con 0x7f0b1815ed60 2024-04-26T23:44:38.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b15e00640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400c4d0 tx=0x7f0b0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04018070 con 0x7f0b1815ed60 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b04007500 con 0x7f0b1815ed60 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04004a20 con 0x7f0b1815ed60 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b18158fd0 con 0x7f0b1815ed60 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.533+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b18159590 con 0x7f0b1815ed60 2024-04-26T23:44:38.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.534+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b18069530 con 0x7f0b1815ed60 2024-04-26T23:44:38.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.535+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b0401b070 con 0x7f0b1815ed60 2024-04-26T23:44:38.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.537+0000 7f0b0ea00640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.537+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5249+0+0 (secure 0 0 0) 0x7f0b0408ad70 con 0x7f0b1815ed60 2024-04-26T23:44:38.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.538+0000 7f0b16800640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.539+0000 7f0b16800640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 0x7f0af806edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.542+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b040592c0 con 0x7f0b1815ed60 2024-04-26T23:44:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.747+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b1815f3e0 con 0x7f0b1815ed60 2024-04-26T23:44:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.749+0000 7f0b0ea00640 1 -- 172.21.15.46:0/834826179 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+13129 (secure 0 0 0) 0x7f0b04058bc0 con 0x7f0b1815ed60 2024-04-26T23:44:38.753 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:38.753 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:37.776254+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.751+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 msgr2=0x7f0af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.751+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 0x7f0af806edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08008040 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 msgr2=0x7f0b18164280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400c4d0 tx=0x7f0b0400c990 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 shutdown_connections 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1815f900 0x7f0b181647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1815ed60 0x7f0b18164280 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 --2- 172.21.15.46:0/834826179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b18070030 0x7f0b18153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 >> 172.21.15.46:0/834826179 conn(0x7f0b1806c530 msgr2=0x7f0b180cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 shutdown_connections 2024-04-26T23:44:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.752+0000 7f0b1d400640 1 -- 172.21.15.46:0/834826179 wait complete. 2024-04-26T23:44:38.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:44:38.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:44:38.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.978+0000 7fecf2600640 1 Processor -- start 2024-04-26T23:44:38.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.978+0000 7fecf2600640 1 -- start start 2024-04-26T23:44:38.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf2600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 0x7fecec0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf2600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecec073b30 0x7fecec0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf2600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecec0ce7d0 con 0x7fecec15cec0 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf2600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fecec0ceab0 con 0x7fecec073b30 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf2600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecec0ced90 con 0x7fecec0cdd00 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecec073b30 0x7fecec0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf1200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecec15cec0 0x7fecec0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50914/0 (socket says 172.21.15.46:50914) 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.980+0000 7fecf1200640 1 -- 172.21.15.46:0/3417619123 learned_addr learned my addr 172.21.15.46:0/3417619123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecebe00640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 0x7fecec0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecf1c00640 1 -- 172.21.15.46:0/3417619123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 msgr2=0x7fecec0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecf1c00640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 0x7fecec0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecf1c00640 1 -- 172.21.15.46:0/3417619123 >> v1:172.21.15.46:6789/0 conn(0x7fecec15cec0 legacy=0x7fecec0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecf1c00640 1 -- 172.21.15.46:0/3417619123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec0cf070 con 0x7fecec073b30 2024-04-26T23:44:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.981+0000 7fecebe00640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 0x7fecec0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.982+0000 7fecf1c00640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec073b30 0x7fecec0b3b40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e26dd6e9abdade server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.982+0000 7feceb400640 1 -- 172.21.15.46:0/3417619123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4037070 con 0x7fecec073b30 2024-04-26T23:44:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.982+0000 7feceb400640 1 -- 172.21.15.46:0/3417619123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece4032440 con 0x7fecec073b30 2024-04-26T23:44:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.982+0000 7feceb400640 1 -- 172.21.15.46:0/3417619123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4032810 con 0x7fecec073b30 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 -- 172.21.15.46:0/3417619123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec073b30 msgr2=0x7fecec0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec073b30 0x7fecec0b3b40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402a180 comp rx=0 tx=0).stop 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 -- 172.21.15.46:0/3417619123 shutdown_connections 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec073b30 0x7fecec0b3b40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 --2- 172.21.15.46:0/3417619123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec0cdd00 0x7fecec0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 -- 172.21.15.46:0/3417619123 >> 172.21.15.46:0/3417619123 conn(0x7fecec0b27b0 msgr2=0x7fecec0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 -- 172.21.15.46:0/3417619123 shutdown_connections 2024-04-26T23:44:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.983+0000 7fecf2600640 1 -- 172.21.15.46:0/3417619123 wait complete. 2024-04-26T23:44:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.985+0000 7fecf2600640 1 Processor -- start 2024-04-26T23:44:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.985+0000 7fecf2600640 1 -- start start 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 0x7fecec159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec15cec0 0x7fecec0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecec073f70 con 0x7fecec0cdd00 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecec15d2a0 con 0x7fecec15cec0 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecf2600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecec0b4080 con 0x7fecec073b30 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.986+0000 7fecebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40184/0 (socket says 172.21.15.46:40184) 2024-04-26T23:44:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 -- 172.21.15.46:0/2920659773 learned_addr learned my addr 172.21.15.46:0/2920659773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecf1200640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 0x7fecec159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 -- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 msgr2=0x7fecec159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecf1c00640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec15cec0 0x7fecec0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 0x7fecec159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 -- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec15cec0 msgr2=0x7fecec0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec15cec0 0x7fecec0c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 -- 172.21.15.46:0/2920659773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec0c7e90 con 0x7fecec0cdd00 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecf1200640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 0x7fecec159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.987+0000 7fecebe00640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fecd400cf90 tx=0x7fecd400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.988+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd401a070 con 0x7fecec0cdd00 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.988+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd400d840 con 0x7fecec0cdd00 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.988+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd400dc10 con 0x7fecec0cdd00 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.988+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec0c8170 con 0x7fecec0cdd00 2024-04-26T23:44:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.988+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec0c8730 con 0x7fecec0cdd00 2024-04-26T23:44:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.989+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec0693b0 con 0x7fecec0cdd00 2024-04-26T23:44:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.990+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd401d070 con 0x7fecec0cdd00 2024-04-26T23:44:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.991+0000 7fece8c00640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 0x7fecc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.991+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5249+0+0 (secure 0 0 0) 0x7fecd408c160 con 0x7fecec0cdd00 2024-04-26T23:44:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.992+0000 7fecf1200640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 0x7fecc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.993+0000 7fecf1200640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 0x7fecc806edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fecec15c210 tx=0x7fecdc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:38.996+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecd405a100 con 0x7fecec0cdd00 2024-04-26T23:44:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.197+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecec0730b0 con 0x7fecec0cdd00 2024-04-26T23:44:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.198+0000 7fece8c00640 1 -- 172.21.15.46:0/2920659773 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fecd405e400 con 0x7fecec0cdd00 2024-04-26T23:44:39.201 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 msgr2=0x7fecc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 0x7fecc806edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fecec15c210 tx=0x7fecdc009040 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 msgr2=0x7fecec072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fecd400cf90 tx=0x7fecd400a7d0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 shutdown_connections 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fecc806c8f0 0x7fecc806edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec15cec0 0x7fecec0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec0cdd00 0x7fecec072610 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 --2- 172.21.15.46:0/2920659773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec073b30 0x7fecec159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 >> 172.21.15.46:0/2920659773 conn(0x7fecec0b27b0 msgr2=0x7fecec0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 shutdown_connections 2024-04-26T23:44:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.200+0000 7fecf2600640 1 -- 172.21.15.46:0/2920659773 wait complete. 2024-04-26T23:44:39.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:44:39.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.421+0000 7fda46e00640 1 Processor -- start 2024-04-26T23:44:39.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.421+0000 7fda46e00640 1 -- start start 2024-04-26T23:44:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.422+0000 7fda46e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda400d0c10 0x7fda400d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.422+0000 7fda46e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.422+0000 7fda46e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda400cfe50 con 0x7fda400d1990 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.422+0000 7fda46e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fda400d0780 con 0x7fda400d0c10 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.422+0000 7fda46e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda400d0130 con 0x7fda401639c0 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.423+0000 7fda46400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.423+0000 7fda46400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:60538/0 (socket says 172.21.15.46:60538) 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.423+0000 7fda46400640 1 -- 172.21.15.46:0/707353126 learned_addr learned my addr 172.21.15.46:0/707353126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.423+0000 7fda45000640 1 --2- 172.21.15.46:0/707353126 >> v2:172.21.15.142:3300/0 conn(0x7fda400d0c10 0x7fda400d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.424+0000 7fda46400640 1 -- 172.21.15.46:0/707353126 >> v2:172.21.15.142:3300/0 conn(0x7fda400d0c10 msgr2=0x7fda400d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.424+0000 7fda46400640 1 --2- 172.21.15.46:0/707353126 >> v2:172.21.15.142:3300/0 conn(0x7fda400d0c10 0x7fda400d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.424+0000 7fda46400640 1 -- 172.21.15.46:0/707353126 >> v1:172.21.15.142:6789/0 conn(0x7fda400d1990 legacy=0x7fda400d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.424+0000 7fda46400640 1 -- 172.21.15.46:0/707353126 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400d0410 con 0x7fda401639c0 2024-04-26T23:44:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.425+0000 7fda46400640 1 --2- 172.21.15.46:0/707353126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda38009870 tx=0x7fda3802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a04092cb5dd1ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.425+0000 7fda3fe00640 1 -- 172.21.15.46:0/707353126 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38037070 con 0x7fda401639c0 2024-04-26T23:44:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda3fe00640 1 -- 172.21.15.46:0/707353126 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda38032440 con 0x7fda401639c0 2024-04-26T23:44:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda3fe00640 1 -- 172.21.15.46:0/707353126 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38032810 con 0x7fda401639c0 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 -- 172.21.15.46:0/707353126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 msgr2=0x7fda400b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 --2- 172.21.15.46:0/707353126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda38009870 tx=0x7fda3802a250 comp rx=0 tx=0).stop 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 -- 172.21.15.46:0/707353126 shutdown_connections 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 --2- 172.21.15.46:0/707353126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda401639c0 0x7fda400b7dd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 --2- 172.21.15.46:0/707353126 >> v2:172.21.15.142:3300/0 conn(0x7fda400d0c10 0x7fda400d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 -- 172.21.15.46:0/707353126 >> 172.21.15.46:0/707353126 conn(0x7fda4009e2a0 msgr2=0x7fda400d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 -- 172.21.15.46:0/707353126 shutdown_connections 2024-04-26T23:44:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.426+0000 7fda46e00640 1 -- 172.21.15.46:0/707353126 wait complete. 2024-04-26T23:44:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.428+0000 7fda46e00640 1 Processor -- start 2024-04-26T23:44:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.428+0000 7fda46e00640 1 -- start start 2024-04-26T23:44:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda400d1990 0x7fda401f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda401639c0 0x7fda401572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda400cfc10 con 0x7fda400d0c10 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda400cfa90 con 0x7fda401639c0 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda46e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda400d5bd0 con 0x7fda400d1990 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda45a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda45a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40188/0 (socket says 172.21.15.46:40188) 2024-04-26T23:44:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.429+0000 7fda45a00640 1 -- 172.21.15.46:0/2188236270 learned_addr learned my addr 172.21.15.46:0/2188236270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 -- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda400d1990 msgr2=0x7fda401f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45000640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda400d1990 0x7fda401f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda400d1990 0x7fda401f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda46400640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda401639c0 0x7fda401572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 -- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda401639c0 msgr2=0x7fda401572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda401639c0 0x7fda401572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 -- 172.21.15.46:0/2188236270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400cb9c0 con 0x7fda400d0c10 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda45a00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c790 tx=0x7fda3400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.430+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34016020 con 0x7fda400d0c10 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.431+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34006d50 con 0x7fda400d0c10 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.431+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003bb0 con 0x7fda400d0c10 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.431+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401f6490 con 0x7fda400d0c10 2024-04-26T23:44:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.431+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda400c8f70 con 0x7fda400d0c10 2024-04-26T23:44:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.432+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda400693b0 con 0x7fda400d0c10 2024-04-26T23:44:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.433+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda3400a850 con 0x7fda400d0c10 2024-04-26T23:44:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.434+0000 7fda3d600640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.434+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5249+0+0 (secure 0 0 0) 0x7fda3408b650 con 0x7fda400d0c10 2024-04-26T23:44:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.434+0000 7fda45000640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.435+0000 7fda45000640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 0x7fda2006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda40157af0 tx=0x7fda2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.439+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda34020020 con 0x7fda400d0c10 2024-04-26T23:44:39.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:39.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.644+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fda400d1770 con 0x7fda400d0c10 2024-04-26T23:44:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.644+0000 7fda3d600640 1 -- 172.21.15.46:0/2188236270 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) ==== 84+0+65 (secure 0 0 0) 0x7fda401f6490 con 0x7fda400d0c10 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:44:39.648 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:44:39.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 msgr2=0x7fda2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 0x7fda2006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda40157af0 tx=0x7fda2c008040 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 msgr2=0x7fda401f6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c790 tx=0x7fda3400cc50 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 shutdown_connections 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda401639c0 0x7fda401572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda400d1990 0x7fda401f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 --2- 172.21.15.46:0/2188236270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda400d0c10 0x7fda401f6690 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 >> 172.21.15.46:0/2188236270 conn(0x7fda4009e2a0 msgr2=0x7fda400d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 shutdown_connections 2024-04-26T23:44:39.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.647+0000 7fda46e00640 1 -- 172.21.15.46:0/2188236270 wait complete. 2024-04-26T23:44:39.669 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:44:39.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:44:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.919+0000 7fcf6d600640 1 Processor -- start 2024-04-26T23:44:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.919+0000 7fcf6d600640 1 -- start start 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.920+0000 7fcf6d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf681469c0 0x7fcf6815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.920+0000 7fcf6d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf68146da0 con 0x7fcf680cfa10 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.920+0000 7fcf6d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680cfeb0 con 0x7fcf680cf560 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.920+0000 7fcf6d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcf680d0ac0 con 0x7fcf681469c0 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf66a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf681469c0 0x7fcf6815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf67400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf680cfa10 0x7fcf680727d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50926/0 (socket says 172.21.15.46:50926) 2024-04-26T23:44:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf67400640 1 -- 172.21.15.46:0/3398754914 learned_addr learned my addr 172.21.15.46:0/3398754914 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf66a00640 1 -- 172.21.15.46:0/3398754914 >> v1:172.21.15.142:6789/0 conn(0x7fcf680cf560 legacy=0x7fcf680b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf66a00640 1 -- 172.21.15.46:0/3398754914 >> v1:172.21.15.46:6789/0 conn(0x7fcf680cfa10 legacy=0x7fcf680727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf66a00640 1 -- 172.21.15.46:0/3398754914 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680d0190 con 0x7fcf681469c0 2024-04-26T23:44:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.921+0000 7fcf65600640 1 -- 172.21.15.46:0/3398754914 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811788502 0 0) 0x7fcf68146da0 con 0x7fcf680cfa10 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.922+0000 7fcf66a00640 1 --2- 172.21.15.46:0/3398754914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf681469c0 0x7fcf6815d6d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097a0 tx=0x7fcf5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b639a84c2dcc93b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf65600640 1 -- 172.21.15.46:0/3398754914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58037070 con 0x7fcf681469c0 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf65600640 1 -- 172.21.15.46:0/3398754914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58032440 con 0x7fcf681469c0 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf65600640 1 -- 172.21.15.46:0/3398754914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58032810 con 0x7fcf681469c0 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 -- 172.21.15.46:0/3398754914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf681469c0 msgr2=0x7fcf6815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 --2- 172.21.15.46:0/3398754914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf681469c0 0x7fcf6815d6d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097a0 tx=0x7fcf5802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 -- 172.21.15.46:0/3398754914 shutdown_connections 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 --2- 172.21.15.46:0/3398754914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf681469c0 0x7fcf6815d6d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 -- 172.21.15.46:0/3398754914 >> 172.21.15.46:0/3398754914 conn(0x7fcf6806c770 msgr2=0x7fcf6806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 -- 172.21.15.46:0/3398754914 shutdown_connections 2024-04-26T23:44:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.923+0000 7fcf6d600640 1 -- 172.21.15.46:0/3398754914 wait complete. 2024-04-26T23:44:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.925+0000 7fcf6d600640 1 Processor -- start 2024-04-26T23:44:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.925+0000 7fcf6d600640 1 -- start start 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680cf560 0x7fcf68153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf681469c0 0x7fcf680abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680b35d0 con 0x7fcf680cfa10 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680d0750 con 0x7fcf681469c0 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf6d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680b3750 con 0x7fcf680cf560 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf66000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf66000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40202/0 (socket says 172.21.15.46:40202) 2024-04-26T23:44:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.926+0000 7fcf66000640 1 -- 172.21.15.46:0/1166650358 learned_addr learned my addr 172.21.15.46:0/1166650358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf67400640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf681469c0 0x7fcf680abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 -- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680cf560 msgr2=0x7fcf68153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680cf560 0x7fcf68153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 -- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf681469c0 msgr2=0x7fcf680abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf681469c0 0x7fcf680abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 -- 172.21.15.46:0/1166650358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680aa820 con 0x7fcf680cfa10 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf66000640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b3f0 tx=0x7fcf5400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.927+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54015020 con 0x7fcf680cfa10 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.928+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf54011830 con 0x7fcf680cfa10 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.928+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54011c20 con 0x7fcf680cfa10 2024-04-26T23:44:39.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.928+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf680aab00 con 0x7fcf680cfa10 2024-04-26T23:44:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.928+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf680ab0c0 con 0x7fcf680cfa10 2024-04-26T23:44:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.929+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68069530 con 0x7fcf680cfa10 2024-04-26T23:44:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.930+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf54018020 con 0x7fcf680cfa10 2024-04-26T23:44:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.931+0000 7fcf5ea00640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 0x7fcf480731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.931+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5249+0+0 (secure 0 0 0) 0x7fcf5408b5a0 con 0x7fcf680cfa10 2024-04-26T23:44:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.932+0000 7fcf66a00640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 0x7fcf480731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.933+0000 7fcf66a00640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 0x7fcf480731c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:39.936+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf5401f030 con 0x7fcf680cfa10 2024-04-26T23:44:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.146+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcf681461e0 con 0x7fcf680cfa10 2024-04-26T23:44:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.796+0000 7fcf5ea00640 1 -- 172.21.15.46:0/1166650358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) ==== 138+0+0 (secure 0 0 0) 0x7fcf540172c0 con 0x7fcf680cfa10 2024-04-26T23:44:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 msgr2=0x7fcf480731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 0x7fcf480731c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802c000 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 msgr2=0x7fcf681f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b3f0 tx=0x7fcf5400b8b0 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 shutdown_connections 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf48070d00 0x7fcf480731c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf681469c0 0x7fcf680abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf680cfa10 0x7fcf681f7240 secure :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b3f0 tx=0x7fcf5400b8b0 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.802+0000 7fcf6d600640 1 --2- 172.21.15.46:0/1166650358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680cf560 0x7fcf68153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.803+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 >> 172.21.15.46:0/1166650358 conn(0x7fcf6806c770 msgr2=0x7fcf68145730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.803+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 shutdown_connections 2024-04-26T23:44:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:40.804+0000 7fcf6d600640 1 -- 172.21.15.46:0/1166650358 wait complete. 2024-04-26T23:44:40.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:44:41.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.042+0000 7f7352e00640 1 Processor -- start 2024-04-26T23:44:41.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.042+0000 7f7352e00640 1 -- start start 2024-04-26T23:44:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.044+0000 7f7352e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c1597e0 0x7f734c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.044+0000 7f7352e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f734c072620 0x7f734c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.044+0000 7f7352e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734c073120 con 0x7f734c0b5730 2024-04-26T23:44:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.044+0000 7f7352e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0b60d0 con 0x7f734c072620 2024-04-26T23:44:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.044+0000 7f7352e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f734c159bc0 con 0x7f734c1597e0 2024-04-26T23:44:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.045+0000 7f7343400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f734c0b5730 0x7f734c071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50936/0 (socket says 172.21.15.46:50936) 2024-04-26T23:44:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.045+0000 7f7350e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f734c072620 0x7f734c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.045+0000 7f7343400640 1 -- 172.21.15.46:0/751530621 learned_addr learned my addr 172.21.15.46:0/751530621 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.045+0000 7f734be00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c1597e0 0x7f734c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f734b400640 1 -- 172.21.15.46:0/751530621 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445162406 0 0) 0x7f734c073120 con 0x7f734c0b5730 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f734b400640 1 -- 172.21.15.46:0/751530621 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7330003520 con 0x7f734c0b5730 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f7350e00640 1 -- 172.21.15.46:0/751530621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c1597e0 msgr2=0x7f734c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f7350e00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c1597e0 0x7f734c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f7350e00640 1 -- 172.21.15.46:0/751530621 >> v1:172.21.15.46:6789/0 conn(0x7f734c0b5730 legacy=0x7f734c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.046+0000 7f7350e00640 1 -- 172.21.15.46:0/751530621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c072b60 con 0x7f734c072620 2024-04-26T23:44:41.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.047+0000 7f7350e00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c072620 0x7f734c0b5b90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f73440097a0 tx=0x7f734402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbadb98b6653290 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:41.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.047+0000 7f734b400640 1 -- 172.21.15.46:0/751530621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344037070 con 0x7f734c072620 2024-04-26T23:44:41.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.047+0000 7f734b400640 1 -- 172.21.15.46:0/751530621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7344032440 con 0x7f734c072620 2024-04-26T23:44:41.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.047+0000 7f734b400640 1 -- 172.21.15.46:0/751530621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344032810 con 0x7f734c072620 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 -- 172.21.15.46:0/751530621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c072620 msgr2=0x7f734c0b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c072620 0x7f734c0b5b90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f73440097a0 tx=0x7f734402a210 comp rx=0 tx=0).stop 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 -- 172.21.15.46:0/751530621 shutdown_connections 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c072620 0x7f734c0b5b90 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 --2- 172.21.15.46:0/751530621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c1597e0 0x7f734c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 -- 172.21.15.46:0/751530621 >> 172.21.15.46:0/751530621 conn(0x7f734c06c980 msgr2=0x7f734c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 -- 172.21.15.46:0/751530621 shutdown_connections 2024-04-26T23:44:41.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.048+0000 7f7352e00640 1 -- 172.21.15.46:0/751530621 wait complete. 2024-04-26T23:44:41.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.050+0000 7f7352e00640 1 Processor -- start 2024-04-26T23:44:41.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.051+0000 7f7352e00640 1 -- start start 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 0x7f734c153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f734c0b5730 0x7f734c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0b43b0 con 0x7f734c1597e0 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0b40d0 con 0x7f734c0b5730 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7352e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0aee20 con 0x7f734c072620 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7350e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7350e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40220/0 (socket says 172.21.15.46:40220) 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7350e00640 1 -- 172.21.15.46:0/948466000 learned_addr learned my addr 172.21.15.46:0/948466000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f734be00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 0x7f734c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.052+0000 7f7343400640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f734c0b5730 0x7f734c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 -- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 msgr2=0x7f734c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 0x7f734c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 -- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f734c0b5730 msgr2=0x7f734c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f734c0b5730 0x7f734c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 -- 172.21.15.46:0/948466000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c0d0890 con 0x7f734c1597e0 2024-04-26T23:44:41.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f734be00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 0x7f734c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:41.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7350e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f734402a800 tx=0x7f73440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:41.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344037050 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7344043070 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734403cb10 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c153a00 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.053+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c153fc0 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.055+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c0736d0 con 0x7f734c1597e0 2024-04-26T23:44:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.056+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f734403ccb0 con 0x7f734c1597e0 2024-04-26T23:44:41.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.058+0000 7f7348c00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 0x7f732806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.058+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5619+0+0 (secure 0 0 0) 0x7f73440ac7a0 con 0x7f734c1597e0 2024-04-26T23:44:41.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.058+0000 7f734be00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 0x7f732806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.060+0000 7f734be00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 0x7f732806edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f733c007440 tx=0x7f733c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:41.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.063+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f734c0736d0 con 0x7f734c1597e0 2024-04-26T23:44:41.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.269+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f734c0b2d40 con 0x7f734c1597e0 2024-04-26T23:44:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.803+0000 7f7348c00640 1 -- 172.21.15.46:0/948466000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) ==== 170+0+0 (secure 0 0 0) 0x7f7344046360 con 0x7f734c1597e0 2024-04-26T23:44:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:44:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.806+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 msgr2=0x7f732806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.806+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 0x7f732806edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f733c007440 tx=0x7f733c00b040 comp rx=0 tx=0).stop 2024-04-26T23:44:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.806+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 msgr2=0x7f734c150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.806+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f734402a800 tx=0x7f73440046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 shutdown_connections 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f732806c8f0 0x7f732806edb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c1597e0 0x7f734c150850 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f734c0b5730 0x7f734c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 --2- 172.21.15.46:0/948466000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f734c072620 0x7f734c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.808+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 >> 172.21.15.46:0/948466000 conn(0x7f734c06c980 msgr2=0x7f734c0afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.809+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 shutdown_connections 2024-04-26T23:44:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:41.809+0000 7f7352e00640 1 -- 172.21.15.46:0/948466000 wait complete. 2024-04-26T23:44:41.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:44:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.031+0000 7fc7ff000640 1 Processor -- start 2024-04-26T23:44:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.031+0000 7fc7ff000640 1 -- start start 2024-04-26T23:44:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7f8160130 0x7fc7f8160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 0x7fc7f80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f80cca70 con 0x7fc7f8160130 2024-04-26T23:44:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f80cbe60 con 0x7fc7f815d0a0 2024-04-26T23:44:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7ff000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80cc140 con 0x7fc7f815cc40 2024-04-26T23:44:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7fdc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7f8160130 0x7fc7f8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7fdc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7f8160130 0x7fc7f8160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60044/0 (socket says 172.21.15.46:60044) 2024-04-26T23:44:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.033+0000 7fc7fdc00640 1 -- 172.21.15.46:0/2236709537 learned_addr learned my addr 172.21.15.46:0/2236709537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fd200640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fe600640 1 --2- 172.21.15.46:0/2236709537 >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 0x7fc7f80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fdc00640 1 -- 172.21.15.46:0/2236709537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 msgr2=0x7fc7f80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fdc00640 1 -- 172.21.15.46:0/2236709537 >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 msgr2=0x7fc7f80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/2236709537 >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 0x7fc7f80cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fdc00640 1 -- 172.21.15.46:0/2236709537 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80cc420 con 0x7fc7f8160130 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fd200640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.034+0000 7fc7fe600640 1 --2- 172.21.15.46:0/2236709537 >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 0x7fc7f80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.035+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f8160510 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8071c40 tx=0x7fc7ec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1d3dabc1331c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.035+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2236709537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec037070 con 0x7fc7f8160130 2024-04-26T23:44:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.035+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2236709537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec032440 con 0x7fc7f8160130 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.035+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2236709537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec032810 con 0x7fc7f8160130 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 -- 172.21.15.46:0/2236709537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 msgr2=0x7fc7f8160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f8160510 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8071c40 tx=0x7fc7ec02a140 comp rx=0 tx=0).stop 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 -- 172.21.15.46:0/2236709537 shutdown_connections 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 --2- 172.21.15.46:0/2236709537 >> v2:172.21.15.142:3300/0 conn(0x7fc7f815d0a0 0x7fc7f80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 --2- 172.21.15.46:0/2236709537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f8160510 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 -- 172.21.15.46:0/2236709537 >> 172.21.15.46:0/2236709537 conn(0x7fc7f80b27b0 msgr2=0x7fc7f80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 -- 172.21.15.46:0/2236709537 shutdown_connections 2024-04-26T23:44:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.036+0000 7fc7ff000640 1 -- 172.21.15.46:0/2236709537 wait complete. 2024-04-26T23:44:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.038+0000 7fc7ff000640 1 Processor -- start 2024-04-26T23:44:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.039+0000 7fc7ff000640 1 -- start start 2024-04-26T23:44:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7f815d0a0 0x7fc7f815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80cfa60 con 0x7fc7f8160130 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80cf8e0 con 0x7fc7f815d0a0 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7ff000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80cfbe0 con 0x7fc7f815cc40 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60050/0 (socket says 172.21.15.46:60050) 2024-04-26T23:44:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.040+0000 7fc7fe600640 1 -- 172.21.15.46:0/3829208337 learned_addr learned my addr 172.21.15.46:0/3829208337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fd200640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7f815d0a0 0x7fc7f815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 -- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 msgr2=0x7fc7f80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 -- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7f815d0a0 msgr2=0x7fc7f815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7f815d0a0 0x7fc7f815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 -- 172.21.15.46:0/3829208337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80ca5e0 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7fe600640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000ce40 tx=0x7fc7f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.041+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f001a070 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.042+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f000d840 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.042+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000dc30 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.042+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f80ca8c0 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.042+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f8147130 con 0x7fc7f8160130 2024-04-26T23:44:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.043+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8147330 con 0x7fc7f8160130 2024-04-26T23:44:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.045+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7f001d070 con 0x7fc7f8160130 2024-04-26T23:44:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.047+0000 7fc7f5600640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 0x7fc7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.047+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6001+0+0 (secure 0 0 0) 0x7fc7f008c4d0 con 0x7fc7f8160130 2024-04-26T23:44:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.048+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 0x7fc7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:42.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.049+0000 7fc7fdc00640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 0x7fc7d806edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec034000 tx=0x7fc7ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.052+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7f8147330 con 0x7fc7f8160130 2024-04-26T23:44:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.304+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc7f8146f20 con 0x7fc7f8160130 2024-04-26T23:44:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.826+0000 7fc7f5600640 1 -- 172.21.15.46:0/3829208337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v71) ==== 174+0+0 (secure 0 0 0) 0x7fc7f005a730 con 0x7fc7f8160130 2024-04-26T23:44:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 13 and data pool 14 2024-04-26T23:44:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.830+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 msgr2=0x7fc7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.830+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 0x7fc7d806edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec034000 tx=0x7fc7ec02e000 comp rx=0 tx=0).stop 2024-04-26T23:44:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.830+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 msgr2=0x7fc7f815b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.830+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000ce40 tx=0x7fc7f000a710 comp rx=0 tx=0).stop 2024-04-26T23:44:42.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 shutdown_connections 2024-04-26T23:44:42.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc7d806c8f0 0x7fc7d806edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f8160130 0x7fc7f815b310 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7f815d0a0 0x7fc7f815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 --2- 172.21.15.46:0/3829208337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7f815cc40 0x7fc7f80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.831+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 >> 172.21.15.46:0/3829208337 conn(0x7fc7f80b27b0 msgr2=0x7fc7f80730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.833+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 shutdown_connections 2024-04-26T23:44:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:42.833+0000 7fc7ff000640 1 -- 172.21.15.46:0/3829208337 wait complete. 2024-04-26T23:44:42.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.049+0000 7f0368800640 1 Processor -- start 2024-04-26T23:44:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.049+0000 7f0368800640 1 -- start start 2024-04-26T23:44:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0368800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0368800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03600732c0 0x7f0360073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0368800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0360161940 con 0x7f036006e510 2024-04-26T23:44:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0368800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0360071f80 con 0x7f03600732c0 2024-04-26T23:44:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0368800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0360072260 con 0x7f0360160bd0 2024-04-26T23:44:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0365400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.051+0000 7f0365400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:33258/0 (socket says 172.21.15.46:33258) 2024-04-26T23:44:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0365400640 1 -- 172.21.15.46:0/3059550127 learned_addr learned my addr 172.21.15.46:0/3059550127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0366800640 1 --2- 172.21.15.46:0/3059550127 >> v2:172.21.15.142:3300/0 conn(0x7f03600732c0 0x7f0360073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0365400640 1 -- 172.21.15.46:0/3059550127 >> v2:172.21.15.142:3300/0 conn(0x7f03600732c0 msgr2=0x7f0360073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0365400640 1 --2- 172.21.15.46:0/3059550127 >> v2:172.21.15.142:3300/0 conn(0x7f03600732c0 0x7f0360073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0365400640 1 -- 172.21.15.46:0/3059550127 >> v1:172.21.15.142:6789/0 conn(0x7f036006e510 legacy=0x7f036006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.052+0000 7f0365400640 1 -- 172.21.15.46:0/3059550127 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360072540 con 0x7f0360160bd0 2024-04-26T23:44:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.054+0000 7f0365400640 1 --2- 172.21.15.46:0/3059550127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0350009870 tx=0x7f035002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3647f4e0831c655f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.054+0000 7f0364a00640 1 -- 172.21.15.46:0/3059550127 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350037070 con 0x7f0360160bd0 2024-04-26T23:44:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.054+0000 7f0364a00640 1 -- 172.21.15.46:0/3059550127 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0350032440 con 0x7f0360160bd0 2024-04-26T23:44:43.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.054+0000 7f0364a00640 1 -- 172.21.15.46:0/3059550127 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350032810 con 0x7f0360160bd0 2024-04-26T23:44:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 -- 172.21.15.46:0/3059550127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 msgr2=0x7f0360161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 --2- 172.21.15.46:0/3059550127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0350009870 tx=0x7f035002a270 comp rx=0 tx=0).stop 2024-04-26T23:44:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 -- 172.21.15.46:0/3059550127 shutdown_connections 2024-04-26T23:44:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 --2- 172.21.15.46:0/3059550127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360160bd0 0x7f0360161030 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 --2- 172.21.15.46:0/3059550127 >> v2:172.21.15.142:3300/0 conn(0x7f03600732c0 0x7f0360073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 -- 172.21.15.46:0/3059550127 >> 172.21.15.46:0/3059550127 conn(0x7f036006c530 msgr2=0x7f036006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 -- 172.21.15.46:0/3059550127 shutdown_connections 2024-04-26T23:44:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.055+0000 7f0368800640 1 -- 172.21.15.46:0/3059550127 wait complete. 2024-04-26T23:44:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.057+0000 7f0368800640 1 Processor -- start 2024-04-26T23:44:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.057+0000 7f0368800640 1 -- start start 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f036006e510 0x7f03601f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 0x7f03600b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03601616f0 con 0x7f0360160bd0 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0360161570 con 0x7f036006e510 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0368800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0360072d40 con 0x7f03600732c0 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0366800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0366800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60058/0 (socket says 172.21.15.46:60058) 2024-04-26T23:44:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0366800640 1 -- 172.21.15.46:0/2516918028 learned_addr learned my addr 172.21.15.46:0/2516918028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.058+0000 7f0365e00640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f036006e510 0x7f03601f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0365400640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 0x7f03600b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 -- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 msgr2=0x7f03600b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 0x7f03600b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 -- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f036006e510 msgr2=0x7f03601f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f036006e510 0x7f03601f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 -- 172.21.15.46:0/2516918028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360158c90 con 0x7f0360160bd0 2024-04-26T23:44:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0365400640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 0x7f03600b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0366800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f035c00efc0 tx=0x7f035c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c018020 con 0x7f0360160bd0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c014480 con 0x7f0360160bd0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c014870 con 0x7f0360160bd0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.059+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0360158f70 con 0x7f0360160bd0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.060+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0360159530 con 0x7f0360160bd0 2024-04-26T23:44:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.061+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360069530 con 0x7f0360160bd0 2024-04-26T23:44:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.062+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f035c005d10 con 0x7f0360160bd0 2024-04-26T23:44:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.063+0000 7f035a000640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.063+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f035c08c8e0 con 0x7f0360160bd0 2024-04-26T23:44:43.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.064+0000 7f0365e00640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.065+0000 7f0365e00640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0354000f60 tx=0x7f0354009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.069+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f035c022070 con 0x7f0360160bd0 2024-04-26T23:44:43.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.278+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f036006fb00 con 0x7f0360160bd0 2024-04-26T23:44:43.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.279+0000 7f035a000640 1 -- 172.21.15.46:0/2516918028 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16862 (secure 0 0 0) 0x7f035c010040 con 0x7f0360160bd0 2024-04-26T23:44:43.283 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:43.283 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:42.305245+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.282+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 msgr2=0x7f034406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.282+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0354000f60 tx=0x7f0354009040 comp rx=0 tx=0).stop 2024-04-26T23:44:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.282+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 msgr2=0x7f036015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.282+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f035c00efc0 tx=0x7f035c00c490 comp rx=0 tx=0).stop 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 shutdown_connections 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160bd0 0x7f036015b420 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03600732c0 0x7f03600b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 --2- 172.21.15.46:0/2516918028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f036006e510 0x7f03601f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 >> 172.21.15.46:0/2516918028 conn(0x7f036006c530 msgr2=0x7f0360141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 shutdown_connections 2024-04-26T23:44:43.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.283+0000 7f0368800640 1 -- 172.21.15.46:0/2516918028 wait complete. 2024-04-26T23:44:43.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:43.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.515+0000 7f2b7a600640 1 Processor -- start 2024-04-26T23:44:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.515+0000 7f2b7a600640 1 -- start start 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b7415edf0 0x7f2b740cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 0x7f2b740cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2b740cb1a0 0x7f2b740cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740cf5c0 con 0x7f2b7415edf0 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740b03c0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b7a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740cb680 con 0x7f2b7415f1d0 2024-04-26T23:44:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b79200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b7415edf0 0x7f2b740cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b79200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b7415edf0 0x7f2b740cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60060/0 (socket says 172.21.15.46:60060) 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.517+0000 7f2b79200640 1 -- 172.21.15.46:0/2021764285 learned_addr learned my addr 172.21.15.46:0/2021764285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b73e00640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 0x7f2b740cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79c00640 1 --2- 172.21.15.46:0/2021764285 >> v2:172.21.15.142:3300/0 conn(0x7f2b740cb1a0 0x7f2b740cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79200640 1 -- 172.21.15.46:0/2021764285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 msgr2=0x7f2b740cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79200640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 0x7f2b740cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79200640 1 -- 172.21.15.46:0/2021764285 >> v2:172.21.15.142:3300/0 conn(0x7f2b740cb1a0 msgr2=0x7f2b740cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79200640 1 --2- 172.21.15.46:0/2021764285 >> v2:172.21.15.142:3300/0 conn(0x7f2b740cb1a0 0x7f2b740cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b79200640 1 -- 172.21.15.46:0/2021764285 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740cb960 con 0x7f2b7415edf0 2024-04-26T23:44:43.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.518+0000 7f2b73e00640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 0x7f2b740cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.519+0000 7f2b79200640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7415edf0 0x7f2b740cf080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2b740b36c0 tx=0x7f2b6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae8b5cdfc50baa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.519+0000 7f2b73400640 1 -- 172.21.15.46:0/2021764285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68037070 con 0x7f2b7415edf0 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.519+0000 7f2b73400640 1 -- 172.21.15.46:0/2021764285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68032440 con 0x7f2b7415edf0 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.519+0000 7f2b73400640 1 -- 172.21.15.46:0/2021764285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68032810 con 0x7f2b7415edf0 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 -- 172.21.15.46:0/2021764285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7415edf0 msgr2=0x7f2b740cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7415edf0 0x7f2b740cf080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2b740b36c0 tx=0x7f2b6802a270 comp rx=0 tx=0).stop 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 -- 172.21.15.46:0/2021764285 shutdown_connections 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 --2- 172.21.15.46:0/2021764285 >> v2:172.21.15.142:3300/0 conn(0x7f2b740cb1a0 0x7f2b740cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415f1d0 0x7f2b740cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 --2- 172.21.15.46:0/2021764285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7415edf0 0x7f2b740cf080 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 -- 172.21.15.46:0/2021764285 >> 172.21.15.46:0/2021764285 conn(0x7f2b740b26d0 msgr2=0x7f2b740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 -- 172.21.15.46:0/2021764285 shutdown_connections 2024-04-26T23:44:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.520+0000 7f2b7a600640 1 -- 172.21.15.46:0/2021764285 wait complete. 2024-04-26T23:44:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.522+0000 7f2b7a600640 1 Processor -- start 2024-04-26T23:44:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.522+0000 7f2b7a600640 1 -- start start 2024-04-26T23:44:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415edf0 0x7f2b740b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b741f4410 0x7f2b740aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cc200 con 0x7f2b740cb1a0 2024-04-26T23:44:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cbf20 con 0x7f2b741f4410 2024-04-26T23:44:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.523+0000 7f2b7a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740ae8e0 con 0x7f2b7415edf0 2024-04-26T23:44:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60064/0 (socket says 172.21.15.46:60064) 2024-04-26T23:44:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 -- 172.21.15.46:0/1413425288 learned_addr learned my addr 172.21.15.46:0/1413425288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b73e00640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415edf0 0x7f2b740b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79c00640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b741f4410 0x7f2b740aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 -- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415edf0 msgr2=0x7f2b740b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415edf0 0x7f2b740b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 -- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b741f4410 msgr2=0x7f2b740aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b741f4410 0x7f2b740aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.524+0000 7f2b79200640 1 -- 172.21.15.46:0/1413425288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7415da30 con 0x7f2b740cb1a0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b79200640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2b6802a2b0 tx=0x7f2b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68037040 con 0x7f2b740cb1a0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68042070 con 0x7f2b740cb1a0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68030bb0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7415dd10 con 0x7f2b740cb1a0 2024-04-26T23:44:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.525+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7415e2d0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.526+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b740693b0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.528+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b68047460 con 0x7f2b740cb1a0 2024-04-26T23:44:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.528+0000 7f2b70c00640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.529+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f2b680ac9d0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.529+0000 7f2b73e00640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.530+0000 7f2b73e00640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b741626e0 tx=0x7f2b60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.533+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b6807abd0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.739+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b740ccf50 con 0x7f2b740cb1a0 2024-04-26T23:44:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.741+0000 7f2b70c00640 1 -- 172.21.15.46:0/1413425288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16862 (secure 0 0 0) 0x7f2b6807a4d0 con 0x7f2b740cb1a0 2024-04-26T23:44:43.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:42.305245+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 msgr2=0x7f2b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b741626e0 tx=0x7f2b60008040 comp rx=0 tx=0).stop 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 msgr2=0x7f2b7415d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2b6802a2b0 tx=0x7f2b680046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 shutdown_connections 2024-04-26T23:44:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b741f4410 0x7f2b740aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b7415edf0 0x7f2b740b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 --2- 172.21.15.46:0/1413425288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b740cb1a0 0x7f2b7415d390 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 >> 172.21.15.46:0/1413425288 conn(0x7f2b740b26d0 msgr2=0x7f2b740b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 shutdown_connections 2024-04-26T23:44:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.744+0000 7f2b7a600640 1 -- 172.21.15.46:0/1413425288 wait complete. 2024-04-26T23:44:43.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:44:43.780 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:44:43.777+0000 7fc430600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:44:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.980+0000 7f0e16400640 1 Processor -- start 2024-04-26T23:44:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.980+0000 7f0e16400640 1 -- start start 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.981+0000 7f0e16400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e1015f280 0x7f0e100d3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.981+0000 7f0e16400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0e10162ec0 0x7f0e101632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.981+0000 7f0e16400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e100d2ce0 con 0x7f0e10163bb0 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.981+0000 7f0e16400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0e100d3fe0 con 0x7f0e1015f280 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.981+0000 7f0e16400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0e100d0da0 con 0x7f0e10162ec0 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e1015f280 0x7f0e100d3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e0fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e10163bb0 0x7f0e100d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55038/0 (socket says 172.21.15.46:55038) 2024-04-26T23:44:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e0fe00640 1 -- 172.21.15.46:0/473595280 learned_addr learned my addr 172.21.15.46:0/473595280 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15a00640 1 --2- 172.21.15.46:0/473595280 >> v2:172.21.15.142:3300/0 conn(0x7f0e10162ec0 0x7f0e101632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15000640 1 -- 172.21.15.46:0/473595280 >> v2:172.21.15.142:3300/0 conn(0x7f0e10162ec0 msgr2=0x7f0e101632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15000640 1 --2- 172.21.15.46:0/473595280 >> v2:172.21.15.142:3300/0 conn(0x7f0e10162ec0 0x7f0e101632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15000640 1 -- 172.21.15.46:0/473595280 >> v1:172.21.15.46:6789/0 conn(0x7f0e10163bb0 legacy=0x7f0e100d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.982+0000 7f0e15000640 1 -- 172.21.15.46:0/473595280 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e100d2fc0 con 0x7f0e1015f280 2024-04-26T23:44:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.983+0000 7f0e15000640 1 --2- 172.21.15.46:0/473595280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1015f280 0x7f0e100d3aa0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cf950 tx=0x7f0e0002a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3c845df73e209a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e0f400640 1 -- 172.21.15.46:0/473595280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00037070 con 0x7f0e1015f280 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e0f400640 1 -- 172.21.15.46:0/473595280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0002aca0 con 0x7f0e1015f280 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e0f400640 1 -- 172.21.15.46:0/473595280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00032660 con 0x7f0e1015f280 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 -- 172.21.15.46:0/473595280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1015f280 msgr2=0x7f0e100d3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 --2- 172.21.15.46:0/473595280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1015f280 0x7f0e100d3aa0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cf950 tx=0x7f0e0002a010 comp rx=0 tx=0).stop 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 -- 172.21.15.46:0/473595280 shutdown_connections 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 --2- 172.21.15.46:0/473595280 >> v2:172.21.15.142:3300/0 conn(0x7f0e10162ec0 0x7f0e101632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 --2- 172.21.15.46:0/473595280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1015f280 0x7f0e100d3aa0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.984+0000 7f0e16400640 1 -- 172.21.15.46:0/473595280 >> 172.21.15.46:0/473595280 conn(0x7f0e100d4fe0 msgr2=0x7f0e100d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.985+0000 7f0e16400640 1 -- 172.21.15.46:0/473595280 shutdown_connections 2024-04-26T23:44:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.985+0000 7f0e16400640 1 -- 172.21.15.46:0/473595280 wait complete. 2024-04-26T23:44:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.986+0000 7f0e16400640 1 Processor -- start 2024-04-26T23:44:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.987+0000 7f0e16400640 1 -- start start 2024-04-26T23:44:43.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.987+0000 7f0e16400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1015f280 0x7f0e101f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e16400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e10162ec0 0x7f0e101f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e16400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e16400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100d3610 con 0x7f0e10163bb0 2024-04-26T23:44:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e16400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100d0fe0 con 0x7f0e1015f280 2024-04-26T23:44:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e16400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e10163830 con 0x7f0e10162ec0 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60080/0 (socket says 172.21.15.46:60080) 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15a00640 1 -- 172.21.15.46:0/1394512293 learned_addr learned my addr 172.21.15.46:0/1394512293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e10162ec0 0x7f0e101f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15000640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1015f280 0x7f0e101f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15a00640 1 -- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e10162ec0 msgr2=0x7f0e101f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.988+0000 7f0e15a00640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e10162ec0 0x7f0e101f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e15a00640 1 -- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1015f280 msgr2=0x7f0e101f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e15a00640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1015f280 0x7f0e101f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e15a00640 1 -- 172.21.15.46:0/1394512293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1014e7a0 con 0x7f0e10163bb0 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e15a00640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007e40 tx=0x7f0e0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08014620 con 0x7f0e10163bb0 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e08019070 con 0x7f0e10163bb0 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1014ea80 con 0x7f0e10163bb0 2024-04-26T23:44:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08014a20 con 0x7f0e10163bb0 2024-04-26T23:44:43.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.989+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101f5280 con 0x7f0e10163bb0 2024-04-26T23:44:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.991+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e100693b0 con 0x7f0e10163bb0 2024-04-26T23:44:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.992+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e08014bc0 con 0x7f0e10163bb0 2024-04-26T23:44:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.993+0000 7f0e0cc00640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 0x7f0dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.993+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f0e0808c600 con 0x7f0e10163bb0 2024-04-26T23:44:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.994+0000 7f0e15000640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 0x7f0dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.995+0000 7f0e15000640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 0x7f0dec06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0e00034000 tx=0x7f0e0002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:43.998+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e0805a890 con 0x7f0e10163bb0 2024-04-26T23:44:44.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.251+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0e10068e80 con 0x7f0e10163bb0 2024-04-26T23:44:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.859+0000 7f0e0cc00640 1 -- 172.21.15.46:0/1394512293 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f0e0805a190 con 0x7f0e10163bb0 2024-04-26T23:44:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 msgr2=0x7f0dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 0x7f0dec06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0e00034000 tx=0x7f0e0002e000 comp rx=0 tx=0).stop 2024-04-26T23:44:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 msgr2=0x7f0e1015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007e40 tx=0x7f0e0800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 shutdown_connections 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dec06c8f0 0x7f0dec06edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e10163bb0 0x7f0e1015d550 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e10162ec0 0x7f0e101f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 --2- 172.21.15.46:0/1394512293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1015f280 0x7f0e101f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 >> 172.21.15.46:0/1394512293 conn(0x7f0e100d4fe0 msgr2=0x7f0e100b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 shutdown_connections 2024-04-26T23:44:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:44.863+0000 7f0e16400640 1 -- 172.21.15.46:0/1394512293 wait complete. 2024-04-26T23:44:44.891 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:44:44.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:44:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.158+0000 7f8092800640 1 Processor -- start 2024-04-26T23:44:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.159+0000 7f8092800640 1 -- start start 2024-04-26T23:44:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.160+0000 7f8092800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f808c15dd60 0x7f808c163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.160+0000 7f8092800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f808c0723e0 0x7f808c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.160+0000 7f8092800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c072880 con 0x7f808c071f80 2024-04-26T23:44:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.160+0000 7f8092800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f808c15e140 con 0x7f808c0723e0 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.160+0000 7f8092800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f808c072b40 con 0x7f808c15dd60 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f808c0723e0 0x7f808c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f808c0723e0 0x7f808c06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60086/0 (socket says 172.21.15.46:60086) 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 -- 172.21.15.46:0/805421911 learned_addr learned my addr 172.21.15.46:0/805421911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 -- 172.21.15.46:0/805421911 >> v2:172.21.15.142:3300/0 conn(0x7f808c15dd60 msgr2=0x7f808c163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 --2- 172.21.15.46:0/805421911 >> v2:172.21.15.142:3300/0 conn(0x7f808c15dd60 0x7f808c163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 -- 172.21.15.46:0/805421911 >> v1:172.21.15.142:6789/0 conn(0x7f808c071f80 legacy=0x7f808c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.161+0000 7f808be00640 1 -- 172.21.15.46:0/805421911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c072e20 con 0x7f808c0723e0 2024-04-26T23:44:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.162+0000 7f808be00640 1 --2- 172.21.15.46:0/805421911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c06dc90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f808c0703d0 tx=0x7f808402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2805db9e9114856 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f808a000640 1 -- 172.21.15.46:0/805421911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084036070 con 0x7f808c0723e0 2024-04-26T23:44:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f808a000640 1 -- 172.21.15.46:0/805421911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8084031600 con 0x7f808c0723e0 2024-04-26T23:44:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f808a000640 1 -- 172.21.15.46:0/805421911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840319d0 con 0x7f808c0723e0 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 -- 172.21.15.46:0/805421911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 msgr2=0x7f808c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 --2- 172.21.15.46:0/805421911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c06dc90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f808c0703d0 tx=0x7f808402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 -- 172.21.15.46:0/805421911 shutdown_connections 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 --2- 172.21.15.46:0/805421911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c06dc90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 --2- 172.21.15.46:0/805421911 >> v2:172.21.15.142:3300/0 conn(0x7f808c15dd60 0x7f808c163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 -- 172.21.15.46:0/805421911 >> 172.21.15.46:0/805421911 conn(0x7f808c06c980 msgr2=0x7f808c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 -- 172.21.15.46:0/805421911 shutdown_connections 2024-04-26T23:44:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.163+0000 7f8092800640 1 -- 172.21.15.46:0/805421911 wait complete. 2024-04-26T23:44:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.165+0000 7f8092800640 1 Processor -- start 2024-04-26T23:44:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.165+0000 7f8092800640 1 -- start start 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808c071f80 0x7f808c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f808c15dd60 0x7f808c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0b42c0 con 0x7f808c0723e0 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0733e0 con 0x7f808c071f80 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f8092800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0b4440 con 0x7f808c15dd60 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.166+0000 7f808aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60092/0 (socket says 172.21.15.46:60092) 2024-04-26T23:44:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 -- 172.21.15.46:0/2261050607 learned_addr learned my addr 172.21.15.46:0/2261050607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 -- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f808c15dd60 msgr2=0x7f808c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f808c15dd60 0x7f808c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 -- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808c071f80 msgr2=0x7f808c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808b400640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808c071f80 0x7f808c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808c071f80 0x7f808c155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 -- 172.21.15.46:0/2261050607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c165fd0 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f808aa00640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f807800c410 tx=0x7f807800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.167+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078017070 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.168+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80780035f0 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.168+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078003a20 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.168+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c167610 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.168+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c154f80 con 0x7f808c0723e0 2024-04-26T23:44:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.169+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c069530 con 0x7f808c0723e0 2024-04-26T23:44:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.170+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8078003bc0 con 0x7f808c0723e0 2024-04-26T23:44:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.171+0000 7f8083400640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.171+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f807808af20 con 0x7f808c0723e0 2024-04-26T23:44:45.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.172+0000 7f808b400640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.173+0000 7f808b400640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 0x7f806c06edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a770 tx=0x7f807c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.175+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8078059250 con 0x7f808c0723e0 2024-04-26T23:44:45.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.427+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f808c073e40 con 0x7f808c0723e0 2024-04-26T23:44:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.880+0000 7f8083400640 1 -- 172.21.15.46:0/2261050607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7f8078058b50 con 0x7f808c0723e0 2024-04-26T23:44:45.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 msgr2=0x7f806c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 0x7f806c06edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a770 tx=0x7f807c008040 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 msgr2=0x7f808c15f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f807800c410 tx=0x7f807800c8d0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 shutdown_connections 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f808c15dd60 0x7f808c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c0723e0 0x7f808c15f660 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 --2- 172.21.15.46:0/2261050607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808c071f80 0x7f808c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.884+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 >> 172.21.15.46:0/2261050607 conn(0x7f808c06c980 msgr2=0x7f808c0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.885+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 shutdown_connections 2024-04-26T23:44:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:45.885+0000 7f8092800640 1 -- 172.21.15.46:0/2261050607 wait complete. 2024-04-26T23:44:45.913 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:44:45.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:46.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.117+0000 7f1770c00640 1 Processor -- start 2024-04-26T23:44:46.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.117+0000 7f1770c00640 1 -- start start 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.118+0000 7f1770c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f176c0b3710 0x7f176c072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.118+0000 7f1770c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c0d1380 con 0x7f176c068090 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.118+0000 7f1770c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c06cf00 con 0x7f176c15cf30 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.118+0000 7f1770c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f176c0d16b0 con 0x7f176c0b3710 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.119+0000 7f176be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f176c0b3710 0x7f176c072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.119+0000 7f176be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f176c0b3710 0x7f176c072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60102/0 (socket says 172.21.15.46:60102) 2024-04-26T23:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.119+0000 7f176be00640 1 -- 172.21.15.46:0/3627821057 learned_addr learned my addr 172.21.15.46:0/3627821057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.119+0000 7f176be00640 1 -- 172.21.15.46:0/3627821057 >> v1:172.21.15.142:6789/0 conn(0x7f176c15cf30 legacy=0x7f176c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.119+0000 7f176be00640 1 -- 172.21.15.46:0/3627821057 >> v1:172.21.15.46:6789/0 conn(0x7f176c068090 legacy=0x7f176c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.120+0000 7f176be00640 1 -- 172.21.15.46:0/3627821057 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c06d1e0 con 0x7f176c0b3710 2024-04-26T23:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.120+0000 7f176aa00640 1 -- 172.21.15.46:0/3627821057 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491449565 0 0) 0x7f176c0d1380 con 0x7f176c068090 2024-04-26T23:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.121+0000 7f176be00640 1 --2- 172.21.15.46:0/3627821057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c0b3710 0x7f176c072c10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1764009870 tx=0x7f176402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878b0e0ad663c51c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.121+0000 7f176aa00640 1 -- 172.21.15.46:0/3627821057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764037070 con 0x7f176c0b3710 2024-04-26T23:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.121+0000 7f176aa00640 1 -- 172.21.15.46:0/3627821057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764032440 con 0x7f176c0b3710 2024-04-26T23:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.121+0000 7f176aa00640 1 -- 172.21.15.46:0/3627821057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764032810 con 0x7f176c0b3710 2024-04-26T23:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 -- 172.21.15.46:0/3627821057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c0b3710 msgr2=0x7f176c072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 --2- 172.21.15.46:0/3627821057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c0b3710 0x7f176c072c10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1764009870 tx=0x7f176402a270 comp rx=0 tx=0).stop 2024-04-26T23:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 -- 172.21.15.46:0/3627821057 shutdown_connections 2024-04-26T23:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 --2- 172.21.15.46:0/3627821057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c0b3710 0x7f176c072c10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 -- 172.21.15.46:0/3627821057 >> 172.21.15.46:0/3627821057 conn(0x7f176c0b4f00 msgr2=0x7f176c0b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 -- 172.21.15.46:0/3627821057 shutdown_connections 2024-04-26T23:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.122+0000 7f1770c00640 1 -- 172.21.15.46:0/3627821057 wait complete. 2024-04-26T23:44:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.123+0000 7f1770c00640 1 Processor -- start 2024-04-26T23:44:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.124+0000 7f1770c00640 1 -- start start 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 0x7f176c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 0x7f176c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f176c15d490 con 0x7f176c068090 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f176c15d310 con 0x7f176c0b3710 2024-04-26T23:44:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1770c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f176c0b1ab0 con 0x7f176c15cf30 2024-04-26T23:44:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60106/0 (socket says 172.21.15.46:60106) 2024-04-26T23:44:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176b400640 1 -- 172.21.15.46:0/802289244 learned_addr learned my addr 172.21.15.46:0/802289244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f1762a00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 0x7f176c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176be00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 0x7f176c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176b400640 1 -- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 msgr2=0x7f176c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.125+0000 7f176b400640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 0x7f176c15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f176b400640 1 -- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 msgr2=0x7f176c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f176b400640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 0x7f176c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f176b400640 1 -- 172.21.15.46:0/802289244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c15adb0 con 0x7f176c068090 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1762a00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 0x7f176c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f176be00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 0x7f176c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f176b400640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c760 tx=0x7f175c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c003b70 con 0x7f176c068090 2024-04-26T23:44:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175c003d30 con 0x7f176c068090 2024-04-26T23:44:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c15b090 con 0x7f176c068090 2024-04-26T23:44:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c00aa90 con 0x7f176c068090 2024-04-26T23:44:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.126+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c15b650 con 0x7f176c068090 2024-04-26T23:44:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.128+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c076370 con 0x7f176c068090 2024-04-26T23:44:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.129+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f175c00ac30 con 0x7f176c068090 2024-04-26T23:44:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.130+0000 7f1763e00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 0x7f174806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.130+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f175c08bc10 con 0x7f176c068090 2024-04-26T23:44:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.130+0000 7f1762a00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 0x7f174806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.132+0000 7f1762a00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 0x7f174806edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f176c070790 tx=0x7f1754009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.134+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f175c059eb0 con 0x7f176c068090 2024-04-26T23:44:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.377+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f176c0cddf0 con 0x7f176c068090 2024-04-26T23:44:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.378+0000 7f1763e00640 1 -- 172.21.15.46:0/802289244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7f175c060080 con 0x7f176c068090 2024-04-26T23:44:46.381 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:46.382 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/932747087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":932747087},{"type":"v1","addr":"172.21.15.142:6835","nonce":932747087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2454420217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2454420217},{"type":"v1","addr":"172.21.15.142:6837","nonce":2454420217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3409172203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3409172203},{"type":"v1","addr":"172.21.15.46:6834","nonce":3409172203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:44:45.428792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7695":{"gid":7695,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2670265715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2670265715},{"type":"v1","addr":"172.21.15.46:6835","nonce":2670265715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7695,"qdb_cluster":[7695]},"id":6}]} 2024-04-26T23:44:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 74 2024-04-26T23:44:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.380+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 msgr2=0x7f174806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.380+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 0x7f174806edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f176c070790 tx=0x7f1754009040 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.380+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 msgr2=0x7f176c15be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.380+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c760 tx=0x7f175c00cc20 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 shutdown_connections 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f174806c8f0 0x7f174806edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f176c15cf30 0x7f176c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f176c0b3710 0x7f176c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 --2- 172.21.15.46:0/802289244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c068090 0x7f176c15be50 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 >> 172.21.15.46:0/802289244 conn(0x7f176c0b4f00 msgr2=0x7f176c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 shutdown_connections 2024-04-26T23:44:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.381+0000 7f1770c00640 1 -- 172.21.15.46:0/802289244 wait complete. 2024-04-26T23:44:46.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.611+0000 7f2577e00640 1 Processor -- start 2024-04-26T23:44:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.611+0000 7f2577e00640 1 -- start start 2024-04-26T23:44:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f25780d2130 0x7f25780d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25780d11f0 con 0x7f25780b4000 2024-04-26T23:44:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25780d5f20 con 0x7f257815f4e0 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f25780d0ce0 con 0x7f25780d2130 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f25780b4000 0x7f25780b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55048/0 (socket says 172.21.15.46:55048) 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f2577400640 1 -- 172.21.15.46:0/1610124699 learned_addr learned my addr 172.21.15.46:0/1610124699 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.613+0000 7f256fe00640 1 --2- 172.21.15.46:0/1610124699 >> v2:172.21.15.142:3300/0 conn(0x7f25780d2130 0x7f25780d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180547494 0 0) 0x7f25780d11f0 con 0x7f25780b4000 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f255c003520 con 0x7f25780b4000 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2524641041 0 0) 0x7f255c003520 con 0x7f25780b4000 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 >> v2:172.21.15.142:3300/0 conn(0x7f25780d2130 msgr2=0x7f25780d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 --2- 172.21.15.46:0/1610124699 >> v2:172.21.15.142:3300/0 conn(0x7f25780d2130 0x7f25780d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 >> v1:172.21.15.142:6789/0 conn(0x7f257815f4e0 legacy=0x7f25780d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25780d0f10 con 0x7f25780b4000 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f25700019f0 con 0x7f25780b4000 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 >> v1:172.21.15.46:6789/0 conn(0x7f25780b4000 legacy=0x7f25780b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f255c0039b0 0x7f255c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f255c004800 0x7f255c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f255c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f255c00c030 con 0x7f255c0051c0 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f255c00c1e0 con 0x7f255c0039b0 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f255c00c3e0 con 0x7f255c004800 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.614+0000 7f2577400640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f255c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2576a00640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f255c0039b0 0x7f255c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2577400640 1 -- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f255c004800 msgr2=0x7f255c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2577400640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f255c004800 0x7f255c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2577400640 1 -- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f255c0039b0 msgr2=0x7f255c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2577400640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f255c0039b0 0x7f255c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.615+0000 7f2577400640 1 -- 172.21.15.46:0/1610124699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c003520 con 0x7f255c0051c0 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.616+0000 7f2577400640 1 -- 172.21.15.46:0/1610124699 reap_dead start 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.616+0000 7f2577400640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f255c00b9c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f257000a860 tx=0x7f255c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245330eb12150d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.616+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000a480 con 0x7f255c0051c0 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.616+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257002f070 con 0x7f255c0051c0 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.616+0000 7f2576000640 1 -- 172.21.15.46:0/1610124699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257002bca0 con 0x7f255c0051c0 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 -- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 msgr2=0x7f255c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f255c00b9c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f257000a860 tx=0x7f255c005830 comp rx=0 tx=0).stop 2024-04-26T23:44:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 -- 172.21.15.46:0/1610124699 shutdown_connections 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 --2- 172.21.15.46:0/1610124699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f255c00b9c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 -- 172.21.15.46:0/1610124699 >> 172.21.15.46:0/1610124699 conn(0x7f25780d5010 msgr2=0x7f25780d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 -- 172.21.15.46:0/1610124699 shutdown_connections 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.617+0000 7f2577e00640 1 -- 172.21.15.46:0/1610124699 wait complete. 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.619+0000 7f2577e00640 1 Processor -- start 2024-04-26T23:44:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.619+0000 7f2577e00640 1 -- start start 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25780a1c90 0x7f257814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 0x7f257814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25780b1d60 con 0x7f255c0051c0 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25780d2990 con 0x7f25780a1c90 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2577e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f257815f8c0 con 0x7f25780a20f0 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2576a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2576a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60130/0 (socket says 172.21.15.46:60130) 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f2576a00640 1 -- 172.21.15.46:0/968225756 learned_addr learned my addr 172.21.15.46:0/968225756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.620+0000 7f256fe00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25780a1c90 0x7f257814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2577400640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 0x7f257814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 -- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 msgr2=0x7f257814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 0x7f257814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 -- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25780a1c90 msgr2=0x7f257814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25780a1c90 0x7f257814ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 -- 172.21.15.46:0/968225756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257809ff80 con 0x7f255c0051c0 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2577400640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 0x7f257814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2576a00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f256800e4d0 tx=0x7f256800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680052e0 con 0x7f255c0051c0 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2568018070 con 0x7f255c0051c0 2024-04-26T23:44:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568003810 con 0x7f255c0051c0 2024-04-26T23:44:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.621+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25780a0260 con 0x7f255c0051c0 2024-04-26T23:44:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.622+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25780a0820 con 0x7f255c0051c0 2024-04-26T23:44:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.623+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25780693b0 con 0x7f255c0051c0 2024-04-26T23:44:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.624+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25680039b0 con 0x7f255c0051c0 2024-04-26T23:44:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.625+0000 7f256f400640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.625+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f256808ba30 con 0x7f255c0051c0 2024-04-26T23:44:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.625+0000 7f256fe00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.626+0000 7f256fe00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 0x7f255406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2578153030 tx=0x7f2560008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.629+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2568059c30 con 0x7f255c0051c0 2024-04-26T23:44:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.832+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25780d33d0 con 0x7f255c0051c0 2024-04-26T23:44:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.834+0000 7f256f400640 1 -- 172.21.15.46:0/968225756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16849 (secure 0 0 0) 0x7f2568059530 con 0x7f255c0051c0 2024-04-26T23:44:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:46.838 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:43.845091+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 msgr2=0x7f255406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 0x7f255406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2578153030 tx=0x7f2560008040 comp rx=0 tx=0).stop 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 msgr2=0x7f25780a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f256800e4d0 tx=0x7f256800e990 comp rx=0 tx=0).stop 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 shutdown_connections 2024-04-26T23:44:46.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25780a20f0 0x7f257814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25780a1c90 0x7f257814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 --2- 172.21.15.46:0/968225756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255c0051c0 0x7f25780a1020 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 >> 172.21.15.46:0/968225756 conn(0x7f25780d5010 msgr2=0x7f25780d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 shutdown_connections 2024-04-26T23:44:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:46.837+0000 7f2577e00640 1 -- 172.21.15.46:0/968225756 wait complete. 2024-04-26T23:44:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:44:46.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:44:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.052+0000 7f991aa00640 1 Processor -- start 2024-04-26T23:44:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.052+0000 7f991aa00640 1 -- start start 2024-04-26T23:44:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f991aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f991415f1d0 0x7f99140cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f991aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140b03c0 con 0x7f991415edf0 2024-04-26T23:44:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f991aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140cb680 con 0x7f99140cd770 2024-04-26T23:44:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f991aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f99140cf5c0 con 0x7f991415f1d0 2024-04-26T23:44:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f9919600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f991415edf0 0x7f99140cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55062/0 (socket says 172.21.15.46:55062) 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f9919600640 1 -- 172.21.15.46:0/3438856977 learned_addr learned my addr 172.21.15.46:0/3438856977 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052948865 0 0) 0x7f99140b03c0 con 0x7f991415edf0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.054+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98fc003520 con 0x7f991415edf0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9918c00640 1 --2- 172.21.15.46:0/3438856977 >> v2:172.21.15.142:3300/0 conn(0x7f991415f1d0 0x7f99140cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4135252836 0 0) 0x7f98fc003520 con 0x7f991415edf0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 >> v2:172.21.15.142:3300/0 conn(0x7f991415f1d0 msgr2=0x7f99140cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 --2- 172.21.15.46:0/3438856977 >> v2:172.21.15.142:3300/0 conn(0x7f991415f1d0 0x7f99140cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 >> v1:172.21.15.142:6789/0 conn(0x7f99140cd770 legacy=0x7f99140c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140cb960 con 0x7f991415edf0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9908001690 con 0x7f991415edf0 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 >> v1:172.21.15.46:6789/0 conn(0x7f991415edf0 legacy=0x7f99140cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc003a00 0x7f98fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc004800 0x7f98fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f98fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99140cf5c0 con 0x7f98fc0052e0 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc00c220 con 0x7f98fc003a00 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc00c420 con 0x7f98fc004800 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.055+0000 7f991a000640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f98fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f9919600640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc003a00 0x7f98fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f9918c00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc004800 0x7f98fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f991a000640 1 -- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc004800 msgr2=0x7f98fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f991a000640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc004800 0x7f98fc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f991a000640 1 -- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc003a00 msgr2=0x7f98fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f991a000640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc003a00 0x7f98fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f991a000640 1 -- 172.21.15.46:0/3438856977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc003520 con 0x7f98fc0052e0 2024-04-26T23:44:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.056+0000 7f9919600640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc003a00 0x7f98fc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.057+0000 7f991a000640 1 -- 172.21.15.46:0/3438856977 reap_dead start 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.057+0000 7f991a000640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f98fc00b9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f99140cb130 tx=0x7f98fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502f9549874b451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.057+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c004080 con 0x7f98fc0052e0 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.057+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c006d70 con 0x7f98fc0052e0 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.057+0000 7f9913e00640 1 -- 172.21.15.46:0/3438856977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c02ad70 con 0x7f98fc0052e0 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 -- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 msgr2=0x7f98fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f98fc00b9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f99140cb130 tx=0x7f98fc005830 comp rx=0 tx=0).stop 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 -- 172.21.15.46:0/3438856977 shutdown_connections 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 --2- 172.21.15.46:0/3438856977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f98fc00b9c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 -- 172.21.15.46:0/3438856977 >> 172.21.15.46:0/3438856977 conn(0x7f99140b26d0 msgr2=0x7f99140b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 -- 172.21.15.46:0/3438856977 shutdown_connections 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.058+0000 7f991aa00640 1 -- 172.21.15.46:0/3438856977 wait complete. 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.059+0000 7f991aa00640 1 Processor -- start 2024-04-26T23:44:47.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.060+0000 7f991aa00640 1 -- start start 2024-04-26T23:44:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.060+0000 7f991aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.060+0000 7f991aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141f5e00 0x7f991406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f991aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f991406fb30 0x7f99141f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f991aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99140cc200 con 0x7f98fc0052e0 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f991aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99140cbf20 con 0x7f99141f5e00 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f991aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99140cb1a0 con 0x7f991406fb30 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60156/0 (socket says 172.21.15.46:60156) 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 -- 172.21.15.46:0/1918887968 learned_addr learned my addr 172.21.15.46:0/1918887968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9918c00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141f5e00 0x7f991406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 -- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f991406fb30 msgr2=0x7f99141f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f991406fb30 0x7f99141f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 -- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141f5e00 msgr2=0x7f991406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141f5e00 0x7f991406f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 -- 172.21.15.46:0/1918887968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991406eb40 con 0x7f98fc0052e0 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.061+0000 7f9919600640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f990800d770 tx=0x7f990800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.062+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908019070 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.062+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99080074e0 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.062+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908004a20 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.062+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141f5890 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.062+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9914158f00 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.063+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991406ff50 con 0x7f98fc0052e0 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.064+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9908004bc0 con 0x7f98fc0052e0 2024-04-26T23:44:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.065+0000 7f9911600640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 0x7f98f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.065+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f990808b530 con 0x7f98fc0052e0 2024-04-26T23:44:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.065+0000 7f9918c00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 0x7f98f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.067+0000 7f9918c00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 0x7f98f406ede0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9900006fd0 tx=0x7f9900008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.069+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f991406eb40 con 0x7f98fc0052e0 2024-04-26T23:44:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.267+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99140693b0 con 0x7f98f406c920 2024-04-26T23:44:47.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.287+0000 7f9911600640 1 -- 172.21.15.46:0/1918887968 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252075 (secure 0 0 0) 0x7f99140693b0 con 0x7f98f406c920 2024-04-26T23:44:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:44:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 msgr2=0x7f98f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 0x7f98f406ede0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9900006fd0 tx=0x7f9900008040 comp rx=0 tx=0).stop 2024-04-26T23:44:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 msgr2=0x7f99141f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f990800d770 tx=0x7f990800dc30 comp rx=0 tx=0).stop 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 shutdown_connections 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f98f406c920 0x7f98f406ede0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f991406fb30 0x7f99141f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141f5e00 0x7f991406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 --2- 172.21.15.46:0/1918887968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98fc0052e0 0x7f99141f7ae0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 >> 172.21.15.46:0/1918887968 conn(0x7f99140b26d0 msgr2=0x7f99140c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 shutdown_connections 2024-04-26T23:44:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.292+0000 7f991aa00640 1 -- 172.21.15.46:0/1918887968 wait complete. 2024-04-26T23:44:47.314 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-26T23:44:46.055989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318088,"kb_used_data":10376,"kb_used_omap":12,"kb_used_meta":307571,"kb_avail":754656632,"statfs":{"total":773094113280,"available":772768391168,"internally_reserved":0,"allocated":10625024,"data_stored":5530564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":314953265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.257184"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309821+0000","last_change":"2024-04-26T23:44:42.872520+0000","last_active":"2024-04-26T23:44:44.309821+0000","last_peered":"2024-04-26T23:44:44.309821+0000","last_clean":"2024-04-26T23:44:44.309821+0000","last_became_active":"2024-04-26T23:44:42.872226+0000","last_became_peered":"2024-04-26T23:44:42.872226+0000","last_unstale":"2024-04-26T23:44:44.309821+0000","last_undegraded":"2024-04-26T23:44:44.309821+0000","last_fullsized":"2024-04-26T23:44:44.309821+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:28.591442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317146+0000","last_change":"2024-04-26T23:44:42.863475+0000","last_active":"2024-04-26T23:44:44.317146+0000","last_peered":"2024-04-26T23:44:44.317146+0000","last_clean":"2024-04-26T23:44:44.317146+0000","last_became_active":"2024-04-26T23:44:42.863105+0000","last_became_peered":"2024-04-26T23:44:42.863105+0000","last_unstale":"2024-04-26T23:44:44.317146+0000","last_undegraded":"2024-04-26T23:44:44.317146+0000","last_fullsized":"2024-04-26T23:44:44.317146+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:11.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308904+0000","last_change":"2024-04-26T23:44:42.859580+0000","last_active":"2024-04-26T23:44:44.308904+0000","last_peered":"2024-04-26T23:44:44.308904+0000","last_clean":"2024-04-26T23:44:44.308904+0000","last_became_active":"2024-04-26T23:44:42.857639+0000","last_became_peered":"2024-04-26T23:44:42.857639+0000","last_unstale":"2024-04-26T23:44:44.308904+0000","last_undegraded":"2024-04-26T23:44:44.308904+0000","last_fullsized":"2024-04-26T23:44:44.308904+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:42.558949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863808+0000","last_change":"2024-04-26T23:44:42.871232+0000","last_active":"2024-04-26T23:44:43.863808+0000","last_peered":"2024-04-26T23:44:43.863808+0000","last_clean":"2024-04-26T23:44:43.863808+0000","last_became_active":"2024-04-26T23:44:42.869991+0000","last_became_peered":"2024-04-26T23:44:42.869991+0000","last_unstale":"2024-04-26T23:44:43.863808+0000","last_undegraded":"2024-04-26T23:44:43.863808+0000","last_fullsized":"2024-04-26T23:44:43.863808+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.831699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862915+0000","last_change":"2024-04-26T23:44:42.868688+0000","last_active":"2024-04-26T23:44:43.862915+0000","last_peered":"2024-04-26T23:44:43.862915+0000","last_clean":"2024-04-26T23:44:43.862915+0000","last_became_active":"2024-04-26T23:44:42.868238+0000","last_became_peered":"2024-04-26T23:44:42.868238+0000","last_unstale":"2024-04-26T23:44:43.862915+0000","last_undegraded":"2024-04-26T23:44:43.862915+0000","last_fullsized":"2024-04-26T23:44:43.862915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861273+0000","last_change":"2024-04-26T23:44:42.875391+0000","last_active":"2024-04-26T23:44:43.861273+0000","last_peered":"2024-04-26T23:44:43.861273+0000","last_clean":"2024-04-26T23:44:43.861273+0000","last_became_active":"2024-04-26T23:44:42.875103+0000","last_became_peered":"2024-04-26T23:44:42.875103+0000","last_unstale":"2024-04-26T23:44:43.861273+0000","last_undegraded":"2024-04-26T23:44:43.861273+0000","last_fullsized":"2024-04-26T23:44:43.861273+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:14.704015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616686+0000","last_change":"2024-04-26T23:44:42.871842+0000","last_active":"2024-04-26T23:44:44.616686+0000","last_peered":"2024-04-26T23:44:44.616686+0000","last_clean":"2024-04-26T23:44:44.616686+0000","last_became_active":"2024-04-26T23:44:42.871467+0000","last_became_peered":"2024-04-26T23:44:42.871467+0000","last_unstale":"2024-04-26T23:44:44.616686+0000","last_undegraded":"2024-04-26T23:44:44.616686+0000","last_fullsized":"2024-04-26T23:44:44.616686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.644330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862443+0000","last_change":"2024-04-26T23:44:42.879136+0000","last_active":"2024-04-26T23:44:43.862443+0000","last_peered":"2024-04-26T23:44:43.862443+0000","last_clean":"2024-04-26T23:44:43.862443+0000","last_became_active":"2024-04-26T23:44:42.870794+0000","last_became_peered":"2024-04-26T23:44:42.870794+0000","last_unstale":"2024-04-26T23:44:43.862443+0000","last_undegraded":"2024-04-26T23:44:43.862443+0000","last_fullsized":"2024-04-26T23:44:43.862443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863534+0000","last_change":"2024-04-26T23:44:42.854698+0000","last_active":"2024-04-26T23:44:43.863534+0000","last_peered":"2024-04-26T23:44:43.863534+0000","last_clean":"2024-04-26T23:44:43.863534+0000","last_became_active":"2024-04-26T23:44:42.854294+0000","last_became_peered":"2024-04-26T23:44:42.854294+0000","last_unstale":"2024-04-26T23:44:43.863534+0000","last_undegraded":"2024-04-26T23:44:43.863534+0000","last_fullsized":"2024-04-26T23:44:43.863534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:13.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790681+0000","last_change":"2024-04-26T23:44:42.876750+0000","last_active":"2024-04-26T23:44:44.790681+0000","last_peered":"2024-04-26T23:44:44.790681+0000","last_clean":"2024-04-26T23:44:44.790681+0000","last_became_active":"2024-04-26T23:44:42.876408+0000","last_became_peered":"2024-04-26T23:44:42.876408+0000","last_unstale":"2024-04-26T23:44:44.790681+0000","last_undegraded":"2024-04-26T23:44:44.790681+0000","last_fullsized":"2024-04-26T23:44:44.790681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:22.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863178+0000","last_change":"2024-04-26T23:44:42.855897+0000","last_active":"2024-04-26T23:44:43.863178+0000","last_peered":"2024-04-26T23:44:43.863178+0000","last_clean":"2024-04-26T23:44:43.863178+0000","last_became_active":"2024-04-26T23:44:42.855619+0000","last_became_peered":"2024-04-26T23:44:42.855619+0000","last_unstale":"2024-04-26T23:44:43.863178+0000","last_undegraded":"2024-04-26T23:44:43.863178+0000","last_fullsized":"2024-04-26T23:44:43.863178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:49.259655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789972+0000","last_change":"2024-04-26T23:44:42.860344+0000","last_active":"2024-04-26T23:44:44.789972+0000","last_peered":"2024-04-26T23:44:44.789972+0000","last_clean":"2024-04-26T23:44:44.789972+0000","last_became_active":"2024-04-26T23:44:42.860075+0000","last_became_peered":"2024-04-26T23:44:42.860075+0000","last_unstale":"2024-04-26T23:44:44.789972+0000","last_undegraded":"2024-04-26T23:44:44.789972+0000","last_fullsized":"2024-04-26T23:44:44.789972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:12.382214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863269+0000","last_change":"2024-04-26T23:44:42.868000+0000","last_active":"2024-04-26T23:44:43.863269+0000","last_peered":"2024-04-26T23:44:43.863269+0000","last_clean":"2024-04-26T23:44:43.863269+0000","last_became_active":"2024-04-26T23:44:42.867321+0000","last_became_peered":"2024-04-26T23:44:42.867321+0000","last_unstale":"2024-04-26T23:44:43.863269+0000","last_undegraded":"2024-04-26T23:44:43.863269+0000","last_fullsized":"2024-04-26T23:44:43.863269+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:35.816205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862180+0000","last_change":"2024-04-26T23:44:42.872461+0000","last_active":"2024-04-26T23:44:43.862180+0000","last_peered":"2024-04-26T23:44:43.862180+0000","last_clean":"2024-04-26T23:44:43.862180+0000","last_became_active":"2024-04-26T23:44:42.872072+0000","last_became_peered":"2024-04-26T23:44:42.872072+0000","last_unstale":"2024-04-26T23:44:43.862180+0000","last_undegraded":"2024-04-26T23:44:43.862180+0000","last_fullsized":"2024-04-26T23:44:43.862180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:13.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862972+0000","last_change":"2024-04-26T23:44:42.863448+0000","last_active":"2024-04-26T23:44:43.862972+0000","last_peered":"2024-04-26T23:44:43.862972+0000","last_clean":"2024-04-26T23:44:43.862972+0000","last_became_active":"2024-04-26T23:44:42.854980+0000","last_became_peered":"2024-04-26T23:44:42.854980+0000","last_unstale":"2024-04-26T23:44:43.862972+0000","last_undegraded":"2024-04-26T23:44:43.862972+0000","last_fullsized":"2024-04-26T23:44:43.862972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:49.259655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.859914+0000","last_change":"2024-04-26T23:44:42.857036+0000","last_active":"2024-04-26T23:44:43.859914+0000","last_peered":"2024-04-26T23:44:43.859914+0000","last_clean":"2024-04-26T23:44:43.859914+0000","last_became_active":"2024-04-26T23:44:42.856742+0000","last_became_peered":"2024-04-26T23:44:42.856742+0000","last_unstale":"2024-04-26T23:44:43.859914+0000","last_undegraded":"2024-04-26T23:44:43.859914+0000","last_fullsized":"2024-04-26T23:44:43.859914+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:30.868226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861389+0000","last_change":"2024-04-26T23:44:42.854684+0000","last_active":"2024-04-26T23:44:43.861389+0000","last_peered":"2024-04-26T23:44:43.861389+0000","last_clean":"2024-04-26T23:44:43.861389+0000","last_became_active":"2024-04-26T23:44:42.854486+0000","last_became_peered":"2024-04-26T23:44:42.854486+0000","last_unstale":"2024-04-26T23:44:43.861389+0000","last_undegraded":"2024-04-26T23:44:43.861389+0000","last_fullsized":"2024-04-26T23:44:43.861389+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:45.526881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309182+0000","last_change":"2024-04-26T23:44:42.860568+0000","last_active":"2024-04-26T23:44:44.309182+0000","last_peered":"2024-04-26T23:44:44.309182+0000","last_clean":"2024-04-26T23:44:44.309182+0000","last_became_active":"2024-04-26T23:44:42.860231+0000","last_became_peered":"2024-04-26T23:44:42.860231+0000","last_unstale":"2024-04-26T23:44:44.309182+0000","last_undegraded":"2024-04-26T23:44:44.309182+0000","last_fullsized":"2024-04-26T23:44:44.309182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.647805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863369+0000","last_change":"2024-04-26T23:44:42.869489+0000","last_active":"2024-04-26T23:44:43.863369+0000","last_peered":"2024-04-26T23:44:43.863369+0000","last_clean":"2024-04-26T23:44:43.863369+0000","last_became_active":"2024-04-26T23:44:42.869116+0000","last_became_peered":"2024-04-26T23:44:42.869116+0000","last_unstale":"2024-04-26T23:44:43.863369+0000","last_undegraded":"2024-04-26T23:44:43.863369+0000","last_fullsized":"2024-04-26T23:44:43.863369+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:54.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863874+0000","last_change":"2024-04-26T23:44:42.864540+0000","last_active":"2024-04-26T23:44:43.863874+0000","last_peered":"2024-04-26T23:44:43.863874+0000","last_clean":"2024-04-26T23:44:43.863874+0000","last_became_active":"2024-04-26T23:44:42.863046+0000","last_became_peered":"2024-04-26T23:44:42.863046+0000","last_unstale":"2024-04-26T23:44:43.863874+0000","last_undegraded":"2024-04-26T23:44:43.863874+0000","last_fullsized":"2024-04-26T23:44:43.863874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:31.593390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862450+0000","last_change":"2024-04-26T23:44:42.865765+0000","last_active":"2024-04-26T23:44:43.862450+0000","last_peered":"2024-04-26T23:44:43.862450+0000","last_clean":"2024-04-26T23:44:43.862450+0000","last_became_active":"2024-04-26T23:44:42.865460+0000","last_became_peered":"2024-04-26T23:44:42.865460+0000","last_unstale":"2024-04-26T23:44:43.862450+0000","last_undegraded":"2024-04-26T23:44:43.862450+0000","last_fullsized":"2024-04-26T23:44:43.862450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:07.089428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862574+0000","last_change":"2024-04-26T23:44:42.866704+0000","last_active":"2024-04-26T23:44:43.862574+0000","last_peered":"2024-04-26T23:44:43.862574+0000","last_clean":"2024-04-26T23:44:43.862574+0000","last_became_active":"2024-04-26T23:44:42.866401+0000","last_became_peered":"2024-04-26T23:44:42.866401+0000","last_unstale":"2024-04-26T23:44:43.862574+0000","last_undegraded":"2024-04-26T23:44:43.862574+0000","last_fullsized":"2024-04-26T23:44:43.862574+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607410+0000","last_change":"2024-04-26T23:44:42.871487+0000","last_active":"2024-04-26T23:44:44.607410+0000","last_peered":"2024-04-26T23:44:44.607410+0000","last_clean":"2024-04-26T23:44:44.607410+0000","last_became_active":"2024-04-26T23:44:42.871172+0000","last_became_peered":"2024-04-26T23:44:42.871172+0000","last_unstale":"2024-04-26T23:44:44.607410+0000","last_undegraded":"2024-04-26T23:44:44.607410+0000","last_fullsized":"2024-04-26T23:44:44.607410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308918+0000","last_change":"2024-04-26T23:44:42.861593+0000","last_active":"2024-04-26T23:44:44.308918+0000","last_peered":"2024-04-26T23:44:44.308918+0000","last_clean":"2024-04-26T23:44:44.308918+0000","last_became_active":"2024-04-26T23:44:42.857745+0000","last_became_peered":"2024-04-26T23:44:42.857745+0000","last_unstale":"2024-04-26T23:44:44.308918+0000","last_undegraded":"2024-04-26T23:44:44.308918+0000","last_fullsized":"2024-04-26T23:44:44.308918+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:59.229331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317286+0000","last_change":"2024-04-26T23:44:42.871284+0000","last_active":"2024-04-26T23:44:44.317286+0000","last_peered":"2024-04-26T23:44:44.317286+0000","last_clean":"2024-04-26T23:44:44.317286+0000","last_became_active":"2024-04-26T23:44:42.870979+0000","last_became_peered":"2024-04-26T23:44:42.870979+0000","last_unstale":"2024-04-26T23:44:44.317286+0000","last_undegraded":"2024-04-26T23:44:44.317286+0000","last_fullsized":"2024-04-26T23:44:44.317286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.141723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863682+0000","last_change":"2024-04-26T23:44:42.872617+0000","last_active":"2024-04-26T23:44:43.863682+0000","last_peered":"2024-04-26T23:44:43.863682+0000","last_clean":"2024-04-26T23:44:43.863682+0000","last_became_active":"2024-04-26T23:44:42.872098+0000","last_became_peered":"2024-04-26T23:44:42.872098+0000","last_unstale":"2024-04-26T23:44:43.863682+0000","last_undegraded":"2024-04-26T23:44:43.863682+0000","last_fullsized":"2024-04-26T23:44:43.863682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.831699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790227+0000","last_change":"2024-04-26T23:44:42.871024+0000","last_active":"2024-04-26T23:44:44.790227+0000","last_peered":"2024-04-26T23:44:44.790227+0000","last_clean":"2024-04-26T23:44:44.790227+0000","last_became_active":"2024-04-26T23:44:42.870749+0000","last_became_peered":"2024-04-26T23:44:42.870749+0000","last_unstale":"2024-04-26T23:44:44.790227+0000","last_undegraded":"2024-04-26T23:44:44.790227+0000","last_fullsized":"2024-04-26T23:44:44.790227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:51.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.864129+0000","last_change":"2024-04-26T23:44:42.865396+0000","last_active":"2024-04-26T23:44:43.864129+0000","last_peered":"2024-04-26T23:44:43.864129+0000","last_clean":"2024-04-26T23:44:43.864129+0000","last_became_active":"2024-04-26T23:44:42.865046+0000","last_became_peered":"2024-04-26T23:44:42.865046+0000","last_unstale":"2024-04-26T23:44:43.864129+0000","last_undegraded":"2024-04-26T23:44:43.864129+0000","last_fullsized":"2024-04-26T23:44:43.864129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:09.246470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617100+0000","last_change":"2024-04-26T23:44:42.880347+0000","last_active":"2024-04-26T23:44:44.617100+0000","last_peered":"2024-04-26T23:44:44.617100+0000","last_clean":"2024-04-26T23:44:44.617100+0000","last_became_active":"2024-04-26T23:44:42.879981+0000","last_became_peered":"2024-04-26T23:44:42.879981+0000","last_unstale":"2024-04-26T23:44:44.617100+0000","last_undegraded":"2024-04-26T23:44:44.617100+0000","last_fullsized":"2024-04-26T23:44:44.617100+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:27.590839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862912+0000","last_change":"2024-04-26T23:44:42.853333+0000","last_active":"2024-04-26T23:44:43.862912+0000","last_peered":"2024-04-26T23:44:43.862912+0000","last_clean":"2024-04-26T23:44:43.862912+0000","last_became_active":"2024-04-26T23:44:42.853024+0000","last_became_peered":"2024-04-26T23:44:42.853024+0000","last_unstale":"2024-04-26T23:44:43.862912+0000","last_undegraded":"2024-04-26T23:44:43.862912+0000","last_fullsized":"2024-04-26T23:44:43.862912+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316728+0000","last_change":"2024-04-26T23:44:42.868712+0000","last_active":"2024-04-26T23:44:44.316728+0000","last_peered":"2024-04-26T23:44:44.316728+0000","last_clean":"2024-04-26T23:44:44.316728+0000","last_became_active":"2024-04-26T23:44:42.868285+0000","last_became_peered":"2024-04-26T23:44:42.868285+0000","last_unstale":"2024-04-26T23:44:44.316728+0000","last_undegraded":"2024-04-26T23:44:44.316728+0000","last_fullsized":"2024-04-26T23:44:44.316728+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:36.299392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.859870+0000","last_change":"2024-04-26T23:44:42.850843+0000","last_active":"2024-04-26T23:44:43.859870+0000","last_peered":"2024-04-26T23:44:43.859870+0000","last_clean":"2024-04-26T23:44:43.859870+0000","last_became_active":"2024-04-26T23:44:42.850535+0000","last_became_peered":"2024-04-26T23:44:42.850535+0000","last_unstale":"2024-04-26T23:44:43.859870+0000","last_undegraded":"2024-04-26T23:44:43.859870+0000","last_fullsized":"2024-04-26T23:44:43.859870+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:38:23.729676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860796+0000","last_change":"2024-04-26T23:44:42.858443+0000","last_active":"2024-04-26T23:44:43.860796+0000","last_peered":"2024-04-26T23:44:43.860796+0000","last_clean":"2024-04-26T23:44:43.860796+0000","last_became_active":"2024-04-26T23:44:42.858083+0000","last_became_peered":"2024-04-26T23:44:42.858083+0000","last_unstale":"2024-04-26T23:44:43.860796+0000","last_undegraded":"2024-04-26T23:44:43.860796+0000","last_fullsized":"2024-04-26T23:44:43.860796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:30.903702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607853+0000","last_change":"2024-04-26T23:44:41.841523+0000","last_active":"2024-04-26T23:44:44.607853+0000","last_peered":"2024-04-26T23:44:44.607853+0000","last_clean":"2024-04-26T23:44:44.607853+0000","last_became_active":"2024-04-26T23:44:41.841245+0000","last_became_peered":"2024-04-26T23:44:41.841245+0000","last_unstale":"2024-04-26T23:44:44.607853+0000","last_undegraded":"2024-04-26T23:44:44.607853+0000","last_fullsized":"2024-04-26T23:44:44.607853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:27.611799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861876+0000","last_change":"2024-04-26T23:44:42.872181+0000","last_active":"2024-04-26T23:44:43.861876+0000","last_peered":"2024-04-26T23:44:43.861876+0000","last_clean":"2024-04-26T23:44:43.861876+0000","last_became_active":"2024-04-26T23:44:42.871882+0000","last_became_peered":"2024-04-26T23:44:42.871882+0000","last_unstale":"2024-04-26T23:44:43.861876+0000","last_undegraded":"2024-04-26T23:44:43.861876+0000","last_fullsized":"2024-04-26T23:44:43.861876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:06.813953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863832+0000","last_change":"2024-04-26T23:44:41.841247+0000","last_active":"2024-04-26T23:44:43.863832+0000","last_peered":"2024-04-26T23:44:43.863832+0000","last_clean":"2024-04-26T23:44:43.863832+0000","last_became_active":"2024-04-26T23:44:41.840948+0000","last_became_peered":"2024-04-26T23:44:41.840948+0000","last_unstale":"2024-04-26T23:44:43.863832+0000","last_undegraded":"2024-04-26T23:44:43.863832+0000","last_fullsized":"2024-04-26T23:44:43.863832+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:57.025808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617002+0000","last_change":"2024-04-26T23:44:42.879903+0000","last_active":"2024-04-26T23:44:44.617002+0000","last_peered":"2024-04-26T23:44:44.617002+0000","last_clean":"2024-04-26T23:44:44.617002+0000","last_became_active":"2024-04-26T23:44:42.879443+0000","last_became_peered":"2024-04-26T23:44:42.879443+0000","last_unstale":"2024-04-26T23:44:44.617002+0000","last_undegraded":"2024-04-26T23:44:44.617002+0000","last_fullsized":"2024-04-26T23:44:44.617002+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:00.754030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617245+0000","last_change":"2024-04-26T23:44:41.844262+0000","last_active":"2024-04-26T23:44:44.617245+0000","last_peered":"2024-04-26T23:44:44.617245+0000","last_clean":"2024-04-26T23:44:44.617245+0000","last_became_active":"2024-04-26T23:44:41.843883+0000","last_became_peered":"2024-04-26T23:44:41.843883+0000","last_unstale":"2024-04-26T23:44:44.617245+0000","last_undegraded":"2024-04-26T23:44:44.617245+0000","last_fullsized":"2024-04-26T23:44:44.617245+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:02.069303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861219+0000","last_change":"2024-04-26T23:44:42.870657+0000","last_active":"2024-04-26T23:44:43.861219+0000","last_peered":"2024-04-26T23:44:43.861219+0000","last_clean":"2024-04-26T23:44:43.861219+0000","last_became_active":"2024-04-26T23:44:42.870357+0000","last_became_peered":"2024-04-26T23:44:42.870357+0000","last_unstale":"2024-04-26T23:44:43.861219+0000","last_undegraded":"2024-04-26T23:44:43.861219+0000","last_fullsized":"2024-04-26T23:44:43.861219+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:31.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607852+0000","last_change":"2024-04-26T23:44:41.846659+0000","last_active":"2024-04-26T23:44:44.607852+0000","last_peered":"2024-04-26T23:44:44.607852+0000","last_clean":"2024-04-26T23:44:44.607852+0000","last_became_active":"2024-04-26T23:44:41.846217+0000","last_became_peered":"2024-04-26T23:44:41.846217+0000","last_unstale":"2024-04-26T23:44:44.607852+0000","last_undegraded":"2024-04-26T23:44:44.607852+0000","last_fullsized":"2024-04-26T23:44:44.607852+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:18.279295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862499+0000","last_change":"2024-04-26T23:44:42.871507+0000","last_active":"2024-04-26T23:44:43.862499+0000","last_peered":"2024-04-26T23:44:43.862499+0000","last_clean":"2024-04-26T23:44:43.862499+0000","last_became_active":"2024-04-26T23:44:42.871185+0000","last_became_peered":"2024-04-26T23:44:42.871185+0000","last_unstale":"2024-04-26T23:44:43.862499+0000","last_undegraded":"2024-04-26T23:44:43.862499+0000","last_fullsized":"2024-04-26T23:44:43.862499+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863081+0000","last_change":"2024-04-26T23:44:41.832254+0000","last_active":"2024-04-26T23:44:43.863081+0000","last_peered":"2024-04-26T23:44:43.863081+0000","last_clean":"2024-04-26T23:44:43.863081+0000","last_became_active":"2024-04-26T23:44:41.831862+0000","last_became_peered":"2024-04-26T23:44:41.831862+0000","last_unstale":"2024-04-26T23:44:43.863081+0000","last_undegraded":"2024-04-26T23:44:43.863081+0000","last_fullsized":"2024-04-26T23:44:43.863081+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:26.487332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607530+0000","last_change":"2024-04-26T23:44:42.875992+0000","last_active":"2024-04-26T23:44:44.607530+0000","last_peered":"2024-04-26T23:44:44.607530+0000","last_clean":"2024-04-26T23:44:44.607530+0000","last_became_active":"2024-04-26T23:44:42.875405+0000","last_became_peered":"2024-04-26T23:44:42.875405+0000","last_unstale":"2024-04-26T23:44:44.607530+0000","last_undegraded":"2024-04-26T23:44:44.607530+0000","last_fullsized":"2024-04-26T23:44:44.607530+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863617+0000","last_change":"2024-04-26T23:44:41.837359+0000","last_active":"2024-04-26T23:44:43.863617+0000","last_peered":"2024-04-26T23:44:43.863617+0000","last_clean":"2024-04-26T23:44:43.863617+0000","last_became_active":"2024-04-26T23:44:41.837041+0000","last_became_peered":"2024-04-26T23:44:41.837041+0000","last_unstale":"2024-04-26T23:44:43.863617+0000","last_undegraded":"2024-04-26T23:44:43.863617+0000","last_fullsized":"2024-04-26T23:44:43.863617+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.617796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860603+0000","last_change":"2024-04-26T23:44:42.875213+0000","last_active":"2024-04-26T23:44:43.860603+0000","last_peered":"2024-04-26T23:44:43.860603+0000","last_clean":"2024-04-26T23:44:43.860603+0000","last_became_active":"2024-04-26T23:44:42.874888+0000","last_became_peered":"2024-04-26T23:44:42.874888+0000","last_unstale":"2024-04-26T23:44:43.860603+0000","last_undegraded":"2024-04-26T23:44:43.860603+0000","last_fullsized":"2024-04-26T23:44:43.860603+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:55.092598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309111+0000","last_change":"2024-04-26T23:44:41.830584+0000","last_active":"2024-04-26T23:44:44.309111+0000","last_peered":"2024-04-26T23:44:44.309111+0000","last_clean":"2024-04-26T23:44:44.309111+0000","last_became_active":"2024-04-26T23:44:41.830257+0000","last_became_peered":"2024-04-26T23:44:41.830257+0000","last_unstale":"2024-04-26T23:44:44.309111+0000","last_undegraded":"2024-04-26T23:44:44.309111+0000","last_fullsized":"2024-04-26T23:44:44.309111+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.943401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863357+0000","last_change":"2024-04-26T23:44:42.869744+0000","last_active":"2024-04-26T23:44:43.863357+0000","last_peered":"2024-04-26T23:44:43.863357+0000","last_clean":"2024-04-26T23:44:43.863357+0000","last_became_active":"2024-04-26T23:44:42.867745+0000","last_became_peered":"2024-04-26T23:44:42.867745+0000","last_unstale":"2024-04-26T23:44:43.863357+0000","last_undegraded":"2024-04-26T23:44:43.863357+0000","last_fullsized":"2024-04-26T23:44:43.863357+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:54.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616748+0000","last_change":"2024-04-26T23:44:41.842153+0000","last_active":"2024-04-26T23:44:44.616748+0000","last_peered":"2024-04-26T23:44:44.616748+0000","last_clean":"2024-04-26T23:44:44.616748+0000","last_became_active":"2024-04-26T23:44:41.841780+0000","last_became_peered":"2024-04-26T23:44:41.841780+0000","last_unstale":"2024-04-26T23:44:44.616748+0000","last_undegraded":"2024-04-26T23:44:44.616748+0000","last_fullsized":"2024-04-26T23:44:44.616748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:03.904868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863224+0000","last_change":"2024-04-26T23:44:42.854464+0000","last_active":"2024-04-26T23:44:43.863224+0000","last_peered":"2024-04-26T23:44:43.863224+0000","last_clean":"2024-04-26T23:44:43.863224+0000","last_became_active":"2024-04-26T23:44:42.853680+0000","last_became_peered":"2024-04-26T23:44:42.853680+0000","last_unstale":"2024-04-26T23:44:43.863224+0000","last_undegraded":"2024-04-26T23:44:43.863224+0000","last_fullsized":"2024-04-26T23:44:43.863224+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:35.816205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317363+0000","last_change":"2024-04-26T23:44:41.845695+0000","last_active":"2024-04-26T23:44:44.317363+0000","last_peered":"2024-04-26T23:44:44.317363+0000","last_clean":"2024-04-26T23:44:44.317363+0000","last_became_active":"2024-04-26T23:44:41.845355+0000","last_became_peered":"2024-04-26T23:44:41.845355+0000","last_unstale":"2024-04-26T23:44:44.317363+0000","last_undegraded":"2024-04-26T23:44:44.317363+0000","last_fullsized":"2024-04-26T23:44:44.317363+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:08.232437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617491+0000","last_change":"2024-04-26T23:44:42.867907+0000","last_active":"2024-04-26T23:44:44.617491+0000","last_peered":"2024-04-26T23:44:44.617491+0000","last_clean":"2024-04-26T23:44:44.617491+0000","last_became_active":"2024-04-26T23:44:42.867586+0000","last_became_peered":"2024-04-26T23:44:42.867586+0000","last_unstale":"2024-04-26T23:44:44.617491+0000","last_undegraded":"2024-04-26T23:44:44.617491+0000","last_fullsized":"2024-04-26T23:44:44.617491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:36.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309349+0000","last_change":"2024-04-26T23:44:41.835268+0000","last_active":"2024-04-26T23:44:44.309349+0000","last_peered":"2024-04-26T23:44:44.309349+0000","last_clean":"2024-04-26T23:44:44.309349+0000","last_became_active":"2024-04-26T23:44:41.834973+0000","last_became_peered":"2024-04-26T23:44:41.834973+0000","last_unstale":"2024-04-26T23:44:44.309349+0000","last_undegraded":"2024-04-26T23:44:44.309349+0000","last_fullsized":"2024-04-26T23:44:44.309349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:09.664813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309635+0000","last_change":"2024-04-26T23:44:42.854706+0000","last_active":"2024-04-26T23:44:44.309635+0000","last_peered":"2024-04-26T23:44:44.309635+0000","last_clean":"2024-04-26T23:44:44.309635+0000","last_became_active":"2024-04-26T23:44:42.852315+0000","last_became_peered":"2024-04-26T23:44:42.852315+0000","last_unstale":"2024-04-26T23:44:44.309635+0000","last_undegraded":"2024-04-26T23:44:44.309635+0000","last_fullsized":"2024-04-26T23:44:44.309635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:49.880815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863459+0000","last_change":"2024-04-26T23:44:41.839936+0000","last_active":"2024-04-26T23:44:43.863459+0000","last_peered":"2024-04-26T23:44:43.863459+0000","last_clean":"2024-04-26T23:44:43.863459+0000","last_became_active":"2024-04-26T23:44:41.836483+0000","last_became_peered":"2024-04-26T23:44:41.836483+0000","last_unstale":"2024-04-26T23:44:43.863459+0000","last_undegraded":"2024-04-26T23:44:43.863459+0000","last_fullsized":"2024-04-26T23:44:43.863459+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:40.608548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863701+0000","last_change":"2024-04-26T23:44:42.858355+0000","last_active":"2024-04-26T23:44:43.863701+0000","last_peered":"2024-04-26T23:44:43.863701+0000","last_clean":"2024-04-26T23:44:43.863701+0000","last_became_active":"2024-04-26T23:44:42.857610+0000","last_became_peered":"2024-04-26T23:44:42.857610+0000","last_unstale":"2024-04-26T23:44:43.863701+0000","last_undegraded":"2024-04-26T23:44:43.863701+0000","last_fullsized":"2024-04-26T23:44:43.863701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:13.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861113+0000","last_change":"2024-04-26T23:44:41.840943+0000","last_active":"2024-04-26T23:44:43.861113+0000","last_peered":"2024-04-26T23:44:43.861113+0000","last_clean":"2024-04-26T23:44:43.861113+0000","last_became_active":"2024-04-26T23:44:41.840305+0000","last_became_peered":"2024-04-26T23:44:41.840305+0000","last_unstale":"2024-04-26T23:44:43.861113+0000","last_undegraded":"2024-04-26T23:44:43.861113+0000","last_fullsized":"2024-04-26T23:44:43.861113+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:35.285359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862664+0000","last_change":"2024-04-26T23:44:42.867729+0000","last_active":"2024-04-26T23:44:43.862664+0000","last_peered":"2024-04-26T23:44:43.862664+0000","last_clean":"2024-04-26T23:44:43.862664+0000","last_became_active":"2024-04-26T23:44:42.867141+0000","last_became_peered":"2024-04-26T23:44:42.867141+0000","last_unstale":"2024-04-26T23:44:43.862664+0000","last_undegraded":"2024-04-26T23:44:43.862664+0000","last_fullsized":"2024-04-26T23:44:43.862664+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616777+0000","last_change":"2024-04-26T23:44:41.845091+0000","last_active":"2024-04-26T23:44:44.616777+0000","last_peered":"2024-04-26T23:44:44.616777+0000","last_clean":"2024-04-26T23:44:44.616777+0000","last_became_active":"2024-04-26T23:44:41.844728+0000","last_became_peered":"2024-04-26T23:44:41.844728+0000","last_unstale":"2024-04-26T23:44:44.616777+0000","last_undegraded":"2024-04-26T23:44:44.616777+0000","last_fullsized":"2024-04-26T23:44:44.616777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:35.285359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862141+0000","last_change":"2024-04-26T23:44:42.859718+0000","last_active":"2024-04-26T23:44:43.862141+0000","last_peered":"2024-04-26T23:44:43.862141+0000","last_clean":"2024-04-26T23:44:43.862141+0000","last_became_active":"2024-04-26T23:44:42.859232+0000","last_became_peered":"2024-04-26T23:44:42.859232+0000","last_unstale":"2024-04-26T23:44:43.862141+0000","last_undegraded":"2024-04-26T23:44:43.862141+0000","last_fullsized":"2024-04-26T23:44:43.862141+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:15.559779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860676+0000","last_change":"2024-04-26T23:44:41.835160+0000","last_active":"2024-04-26T23:44:43.860676+0000","last_peered":"2024-04-26T23:44:43.860676+0000","last_clean":"2024-04-26T23:44:43.860676+0000","last_became_active":"2024-04-26T23:44:41.834795+0000","last_became_peered":"2024-04-26T23:44:41.834795+0000","last_unstale":"2024-04-26T23:44:43.860676+0000","last_undegraded":"2024-04-26T23:44:43.860676+0000","last_fullsized":"2024-04-26T23:44:43.860676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:33.478880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790479+0000","last_change":"2024-04-26T23:44:42.872468+0000","last_active":"2024-04-26T23:44:44.790479+0000","last_peered":"2024-04-26T23:44:44.790479+0000","last_clean":"2024-04-26T23:44:44.790479+0000","last_became_active":"2024-04-26T23:44:42.872245+0000","last_became_peered":"2024-04-26T23:44:42.872245+0000","last_unstale":"2024-04-26T23:44:44.790479+0000","last_undegraded":"2024-04-26T23:44:44.790479+0000","last_fullsized":"2024-04-26T23:44:44.790479+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:32.463740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863033+0000","last_change":"2024-04-26T23:44:41.842232+0000","last_active":"2024-04-26T23:44:43.863033+0000","last_peered":"2024-04-26T23:44:43.863033+0000","last_clean":"2024-04-26T23:44:43.863033+0000","last_became_active":"2024-04-26T23:44:41.841916+0000","last_became_peered":"2024-04-26T23:44:41.841916+0000","last_unstale":"2024-04-26T23:44:43.863033+0000","last_undegraded":"2024-04-26T23:44:43.863033+0000","last_fullsized":"2024-04-26T23:44:43.863033+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:56.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789978+0000","last_change":"2024-04-26T23:44:42.871143+0000","last_active":"2024-04-26T23:44:44.789978+0000","last_peered":"2024-04-26T23:44:44.789978+0000","last_clean":"2024-04-26T23:44:44.789978+0000","last_became_active":"2024-04-26T23:44:42.868768+0000","last_became_peered":"2024-04-26T23:44:42.868768+0000","last_unstale":"2024-04-26T23:44:44.789978+0000","last_undegraded":"2024-04-26T23:44:44.789978+0000","last_fullsized":"2024-04-26T23:44:44.789978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.884286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862577+0000","last_change":"2024-04-26T23:44:41.831009+0000","last_active":"2024-04-26T23:44:43.862577+0000","last_peered":"2024-04-26T23:44:43.862577+0000","last_clean":"2024-04-26T23:44:43.862577+0000","last_became_active":"2024-04-26T23:44:41.830543+0000","last_became_peered":"2024-04-26T23:44:41.830543+0000","last_unstale":"2024-04-26T23:44:43.862577+0000","last_undegraded":"2024-04-26T23:44:43.862577+0000","last_fullsized":"2024-04-26T23:44:43.862577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:27.611799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861105+0000","last_change":"2024-04-26T23:44:42.859660+0000","last_active":"2024-04-26T23:44:43.861105+0000","last_peered":"2024-04-26T23:44:43.861105+0000","last_clean":"2024-04-26T23:44:43.861105+0000","last_became_active":"2024-04-26T23:44:42.859102+0000","last_became_peered":"2024-04-26T23:44:42.859102+0000","last_unstale":"2024-04-26T23:44:43.861105+0000","last_undegraded":"2024-04-26T23:44:43.861105+0000","last_fullsized":"2024-04-26T23:44:43.861105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:16.644278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.608139+0000","last_change":"2024-04-26T23:44:41.844834+0000","last_active":"2024-04-26T23:44:44.608139+0000","last_peered":"2024-04-26T23:44:44.608139+0000","last_clean":"2024-04-26T23:44:44.608139+0000","last_became_active":"2024-04-26T23:44:41.844491+0000","last_became_peered":"2024-04-26T23:44:41.844491+0000","last_unstale":"2024-04-26T23:44:44.608139+0000","last_undegraded":"2024-04-26T23:44:44.608139+0000","last_fullsized":"2024-04-26T23:44:44.608139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.617796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607665+0000","last_change":"2024-04-26T23:44:42.870254+0000","last_active":"2024-04-26T23:44:44.607665+0000","last_peered":"2024-04-26T23:44:44.607665+0000","last_clean":"2024-04-26T23:44:44.607665+0000","last_became_active":"2024-04-26T23:44:42.869761+0000","last_became_peered":"2024-04-26T23:44:42.869761+0000","last_unstale":"2024-04-26T23:44:44.607665+0000","last_undegraded":"2024-04-26T23:44:44.607665+0000","last_fullsized":"2024-04-26T23:44:44.607665+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616841+0000","last_change":"2024-04-26T23:44:41.847401+0000","last_active":"2024-04-26T23:44:44.616841+0000","last_peered":"2024-04-26T23:44:44.616841+0000","last_clean":"2024-04-26T23:44:44.616841+0000","last_became_active":"2024-04-26T23:44:41.847059+0000","last_became_peered":"2024-04-26T23:44:41.847059+0000","last_unstale":"2024-04-26T23:44:44.616841+0000","last_undegraded":"2024-04-26T23:44:44.616841+0000","last_fullsized":"2024-04-26T23:44:44.616841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:32.127690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863765+0000","last_change":"2024-04-26T23:44:42.857294+0000","last_active":"2024-04-26T23:44:43.863765+0000","last_peered":"2024-04-26T23:44:43.863765+0000","last_clean":"2024-04-26T23:44:43.863765+0000","last_became_active":"2024-04-26T23:44:42.856969+0000","last_became_peered":"2024-04-26T23:44:42.856969+0000","last_unstale":"2024-04-26T23:44:43.863765+0000","last_undegraded":"2024-04-26T23:44:43.863765+0000","last_fullsized":"2024-04-26T23:44:43.863765+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:31.593390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863567+0000","last_change":"2024-04-26T23:44:41.836981+0000","last_active":"2024-04-26T23:44:43.863567+0000","last_peered":"2024-04-26T23:44:43.863567+0000","last_clean":"2024-04-26T23:44:43.863567+0000","last_became_active":"2024-04-26T23:44:41.836100+0000","last_became_peered":"2024-04-26T23:44:41.836100+0000","last_unstale":"2024-04-26T23:44:43.863567+0000","last_undegraded":"2024-04-26T23:44:43.863567+0000","last_fullsized":"2024-04-26T23:44:43.863567+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:05.493175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860202+0000","last_change":"2024-04-26T23:44:42.881995+0000","last_active":"2024-04-26T23:44:43.860202+0000","last_peered":"2024-04-26T23:44:43.860202+0000","last_clean":"2024-04-26T23:44:43.860202+0000","last_became_active":"2024-04-26T23:44:42.881622+0000","last_became_peered":"2024-04-26T23:44:42.881622+0000","last_unstale":"2024-04-26T23:44:43.860202+0000","last_undegraded":"2024-04-26T23:44:43.860202+0000","last_fullsized":"2024-04-26T23:44:43.860202+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:52.347119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316266+0000","last_change":"2024-04-26T23:44:41.837673+0000","last_active":"2024-04-26T23:44:44.316266+0000","last_peered":"2024-04-26T23:44:44.316266+0000","last_clean":"2024-04-26T23:44:44.316266+0000","last_became_active":"2024-04-26T23:44:41.837377+0000","last_became_peered":"2024-04-26T23:44:41.837377+0000","last_unstale":"2024-04-26T23:44:44.316266+0000","last_undegraded":"2024-04-26T23:44:44.316266+0000","last_fullsized":"2024-04-26T23:44:44.316266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:12.145152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316992+0000","last_change":"2024-04-26T23:44:42.852172+0000","last_active":"2024-04-26T23:44:44.316992+0000","last_peered":"2024-04-26T23:44:44.316992+0000","last_clean":"2024-04-26T23:44:44.316992+0000","last_became_active":"2024-04-26T23:44:42.851791+0000","last_became_peered":"2024-04-26T23:44:42.851791+0000","last_unstale":"2024-04-26T23:44:44.316992+0000","last_undegraded":"2024-04-26T23:44:44.316992+0000","last_fullsized":"2024-04-26T23:44:44.316992+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:04.918901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789935+0000","last_change":"2024-04-26T23:44:41.833443+0000","last_active":"2024-04-26T23:44:44.789935+0000","last_peered":"2024-04-26T23:44:44.789935+0000","last_clean":"2024-04-26T23:44:44.789935+0000","last_became_active":"2024-04-26T23:44:41.833147+0000","last_became_peered":"2024-04-26T23:44:41.833147+0000","last_unstale":"2024-04-26T23:44:44.789935+0000","last_undegraded":"2024-04-26T23:44:44.789935+0000","last_fullsized":"2024-04-26T23:44:44.789935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:12.969965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316519+0000","last_change":"2024-04-26T23:44:42.869187+0000","last_active":"2024-04-26T23:44:44.316519+0000","last_peered":"2024-04-26T23:44:44.316519+0000","last_clean":"2024-04-26T23:44:44.316519+0000","last_became_active":"2024-04-26T23:44:42.868830+0000","last_became_peered":"2024-04-26T23:44:42.868830+0000","last_unstale":"2024-04-26T23:44:44.316519+0000","last_undegraded":"2024-04-26T23:44:44.316519+0000","last_fullsized":"2024-04-26T23:44:44.316519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:34.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316891+0000","last_change":"2024-04-26T23:44:41.838044+0000","last_active":"2024-04-26T23:44:44.316891+0000","last_peered":"2024-04-26T23:44:44.316891+0000","last_clean":"2024-04-26T23:44:44.316891+0000","last_became_active":"2024-04-26T23:44:41.837752+0000","last_became_peered":"2024-04-26T23:44:41.837752+0000","last_unstale":"2024-04-26T23:44:44.316891+0000","last_undegraded":"2024-04-26T23:44:44.316891+0000","last_fullsized":"2024-04-26T23:44:44.316891+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:30.579357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861873+0000","last_change":"2024-04-26T23:44:42.858527+0000","last_active":"2024-04-26T23:44:43.861873+0000","last_peered":"2024-04-26T23:44:43.861873+0000","last_clean":"2024-04-26T23:44:43.861873+0000","last_became_active":"2024-04-26T23:44:42.858137+0000","last_became_peered":"2024-04-26T23:44:42.858137+0000","last_unstale":"2024-04-26T23:44:43.861873+0000","last_undegraded":"2024-04-26T23:44:43.861873+0000","last_fullsized":"2024-04-26T23:44:43.861873+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:22.542824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789362+0000","last_change":"2024-04-26T23:44:41.845605+0000","last_active":"2024-04-26T23:44:44.789362+0000","last_peered":"2024-04-26T23:44:44.789362+0000","last_clean":"2024-04-26T23:44:44.789362+0000","last_became_active":"2024-04-26T23:44:41.845281+0000","last_became_peered":"2024-04-26T23:44:41.845281+0000","last_unstale":"2024-04-26T23:44:44.789362+0000","last_undegraded":"2024-04-26T23:44:44.789362+0000","last_fullsized":"2024-04-26T23:44:44.789362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:17.497909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607410+0000","last_change":"2024-04-26T23:44:42.871307+0000","last_active":"2024-04-26T23:44:44.607410+0000","last_peered":"2024-04-26T23:44:44.607410+0000","last_clean":"2024-04-26T23:44:44.607410+0000","last_became_active":"2024-04-26T23:44:42.870755+0000","last_became_peered":"2024-04-26T23:44:42.870755+0000","last_unstale":"2024-04-26T23:44:44.607410+0000","last_undegraded":"2024-04-26T23:44:44.607410+0000","last_fullsized":"2024-04-26T23:44:44.607410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:07.089428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617180+0000","last_change":"2024-04-26T23:44:41.846189+0000","last_active":"2024-04-26T23:44:44.617180+0000","last_peered":"2024-04-26T23:44:44.617180+0000","last_clean":"2024-04-26T23:44:44.617180+0000","last_became_active":"2024-04-26T23:44:41.845926+0000","last_became_peered":"2024-04-26T23:44:41.845926+0000","last_unstale":"2024-04-26T23:44:44.617180+0000","last_undegraded":"2024-04-26T23:44:44.617180+0000","last_fullsized":"2024-04-26T23:44:44.617180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:33.478880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861850+0000","last_change":"2024-04-26T23:44:42.867595+0000","last_active":"2024-04-26T23:44:43.861850+0000","last_peered":"2024-04-26T23:44:43.861850+0000","last_clean":"2024-04-26T23:44:43.861850+0000","last_became_active":"2024-04-26T23:44:42.859827+0000","last_became_peered":"2024-04-26T23:44:42.859827+0000","last_unstale":"2024-04-26T23:44:43.861850+0000","last_undegraded":"2024-04-26T23:44:43.861850+0000","last_fullsized":"2024-04-26T23:44:43.861850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:53.492153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863884+0000","last_change":"2024-04-26T23:44:41.843679+0000","last_active":"2024-04-26T23:44:43.863884+0000","last_peered":"2024-04-26T23:44:43.863884+0000","last_clean":"2024-04-26T23:44:43.863884+0000","last_became_active":"2024-04-26T23:44:41.843157+0000","last_became_peered":"2024-04-26T23:44:41.843157+0000","last_unstale":"2024-04-26T23:44:43.863884+0000","last_undegraded":"2024-04-26T23:44:43.863884+0000","last_fullsized":"2024-04-26T23:44:43.863884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:27.409179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861851+0000","last_change":"2024-04-26T23:44:42.866472+0000","last_active":"2024-04-26T23:44:43.861851+0000","last_peered":"2024-04-26T23:44:43.861851+0000","last_clean":"2024-04-26T23:44:43.861851+0000","last_became_active":"2024-04-26T23:44:42.865740+0000","last_became_peered":"2024-04-26T23:44:42.865740+0000","last_unstale":"2024-04-26T23:44:43.861851+0000","last_undegraded":"2024-04-26T23:44:43.861851+0000","last_fullsized":"2024-04-26T23:44:43.861851+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.634829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863652+0000","last_change":"2024-04-26T23:44:41.836746+0000","last_active":"2024-04-26T23:44:43.863652+0000","last_peered":"2024-04-26T23:44:43.863652+0000","last_clean":"2024-04-26T23:44:43.863652+0000","last_became_active":"2024-04-26T23:44:41.835371+0000","last_became_peered":"2024-04-26T23:44:41.835371+0000","last_unstale":"2024-04-26T23:44:43.863652+0000","last_undegraded":"2024-04-26T23:44:43.863652+0000","last_fullsized":"2024-04-26T23:44:43.863652+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:23.924236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317432+0000","last_change":"2024-04-26T23:44:42.862002+0000","last_active":"2024-04-26T23:44:44.317432+0000","last_peered":"2024-04-26T23:44:44.317432+0000","last_clean":"2024-04-26T23:44:44.317432+0000","last_became_active":"2024-04-26T23:44:42.861727+0000","last_became_peered":"2024-04-26T23:44:42.861727+0000","last_unstale":"2024-04-26T23:44:44.317432+0000","last_undegraded":"2024-04-26T23:44:44.317432+0000","last_fullsized":"2024-04-26T23:44:44.317432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:42.558949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607582+0000","last_change":"2024-04-26T23:44:41.840373+0000","last_active":"2024-04-26T23:44:44.607582+0000","last_peered":"2024-04-26T23:44:44.607582+0000","last_clean":"2024-04-26T23:44:44.607582+0000","last_became_active":"2024-04-26T23:44:41.840079+0000","last_became_peered":"2024-04-26T23:44:41.840079+0000","last_unstale":"2024-04-26T23:44:44.607582+0000","last_undegraded":"2024-04-26T23:44:44.607582+0000","last_fullsized":"2024-04-26T23:44:44.607582+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:56.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789589+0000","last_change":"2024-04-26T23:44:42.881502+0000","last_active":"2024-04-26T23:44:44.789589+0000","last_peered":"2024-04-26T23:44:44.789589+0000","last_clean":"2024-04-26T23:44:44.789589+0000","last_became_active":"2024-04-26T23:44:42.881040+0000","last_became_peered":"2024-04-26T23:44:42.881040+0000","last_unstale":"2024-04-26T23:44:44.789589+0000","last_undegraded":"2024-04-26T23:44:44.789589+0000","last_fullsized":"2024-04-26T23:44:44.789589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:51.058728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861157+0000","last_change":"2024-04-26T23:44:41.836712+0000","last_active":"2024-04-26T23:44:43.861157+0000","last_peered":"2024-04-26T23:44:43.861157+0000","last_clean":"2024-04-26T23:44:43.861157+0000","last_became_active":"2024-04-26T23:44:41.836329+0000","last_became_peered":"2024-04-26T23:44:41.836329+0000","last_unstale":"2024-04-26T23:44:43.861157+0000","last_undegraded":"2024-04-26T23:44:43.861157+0000","last_fullsized":"2024-04-26T23:44:43.861157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:02.069303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308877+0000","last_change":"2024-04-26T23:44:42.869332+0000","last_active":"2024-04-26T23:44:44.308877+0000","last_peered":"2024-04-26T23:44:44.308877+0000","last_clean":"2024-04-26T23:44:44.308877+0000","last_became_active":"2024-04-26T23:44:42.868837+0000","last_became_peered":"2024-04-26T23:44:42.868837+0000","last_unstale":"2024-04-26T23:44:44.308877+0000","last_undegraded":"2024-04-26T23:44:44.308877+0000","last_fullsized":"2024-04-26T23:44:44.308877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.141723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316465+0000","last_change":"2024-04-26T23:44:41.842867+0000","last_active":"2024-04-26T23:44:44.316465+0000","last_peered":"2024-04-26T23:44:44.316465+0000","last_clean":"2024-04-26T23:44:44.316465+0000","last_became_active":"2024-04-26T23:44:41.842529+0000","last_became_peered":"2024-04-26T23:44:41.842529+0000","last_unstale":"2024-04-26T23:44:44.316465+0000","last_undegraded":"2024-04-26T23:44:44.316465+0000","last_fullsized":"2024-04-26T23:44:44.316465+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:40.817666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607200+0000","last_change":"2024-04-26T23:44:42.873462+0000","last_active":"2024-04-26T23:44:44.607200+0000","last_peered":"2024-04-26T23:44:44.607200+0000","last_clean":"2024-04-26T23:44:44.607200+0000","last_became_active":"2024-04-26T23:44:42.873095+0000","last_became_peered":"2024-04-26T23:44:42.873095+0000","last_unstale":"2024-04-26T23:44:44.607200+0000","last_undegraded":"2024-04-26T23:44:44.607200+0000","last_fullsized":"2024-04-26T23:44:44.607200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:13.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607566+0000","last_change":"2024-04-26T23:44:41.839198+0000","last_active":"2024-04-26T23:44:44.607566+0000","last_peered":"2024-04-26T23:44:44.607566+0000","last_clean":"2024-04-26T23:44:44.607566+0000","last_became_active":"2024-04-26T23:44:41.838683+0000","last_became_peered":"2024-04-26T23:44:41.838683+0000","last_unstale":"2024-04-26T23:44:44.607566+0000","last_undegraded":"2024-04-26T23:44:44.607566+0000","last_fullsized":"2024-04-26T23:44:44.607566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:37.624561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860850+0000","last_change":"2024-04-26T23:44:42.856089+0000","last_active":"2024-04-26T23:44:43.860850+0000","last_peered":"2024-04-26T23:44:43.860850+0000","last_clean":"2024-04-26T23:44:43.860850+0000","last_became_active":"2024-04-26T23:44:42.855682+0000","last_became_peered":"2024-04-26T23:44:42.855682+0000","last_unstale":"2024-04-26T23:44:43.860850+0000","last_undegraded":"2024-04-26T23:44:43.860850+0000","last_fullsized":"2024-04-26T23:44:43.860850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:42.144285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616852+0000","last_change":"2024-04-26T23:44:41.838757+0000","last_active":"2024-04-26T23:44:44.616852+0000","last_peered":"2024-04-26T23:44:44.616852+0000","last_clean":"2024-04-26T23:44:44.616852+0000","last_became_active":"2024-04-26T23:44:41.838423+0000","last_became_peered":"2024-04-26T23:44:41.838423+0000","last_unstale":"2024-04-26T23:44:44.616852+0000","last_undegraded":"2024-04-26T23:44:44.616852+0000","last_fullsized":"2024-04-26T23:44:44.616852+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:10.975944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617345+0000","last_change":"2024-04-26T23:44:42.871295+0000","last_active":"2024-04-26T23:44:44.617345+0000","last_peered":"2024-04-26T23:44:44.617345+0000","last_clean":"2024-04-26T23:44:44.617345+0000","last_became_active":"2024-04-26T23:44:42.870709+0000","last_became_peered":"2024-04-26T23:44:42.870709+0000","last_unstale":"2024-04-26T23:44:44.617345+0000","last_undegraded":"2024-04-26T23:44:44.617345+0000","last_fullsized":"2024-04-26T23:44:44.617345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:16.266265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863952+0000","last_change":"2024-04-26T23:44:41.843191+0000","last_active":"2024-04-26T23:44:43.863952+0000","last_peered":"2024-04-26T23:44:43.863952+0000","last_clean":"2024-04-26T23:44:43.863952+0000","last_became_active":"2024-04-26T23:44:41.842727+0000","last_became_peered":"2024-04-26T23:44:41.842727+0000","last_unstale":"2024-04-26T23:44:43.863952+0000","last_undegraded":"2024-04-26T23:44:43.863952+0000","last_fullsized":"2024-04-26T23:44:43.863952+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:18.279295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":305,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617196+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:44:44.617196+0000","last_peered":"2024-04-26T23:44:44.617196+0000","last_clean":"2024-04-26T23:44:44.617196+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:44:44.617196+0000","last_undegraded":"2024-04-26T23:44:44.617196+0000","last_fullsized":"2024-04-26T23:44:44.617196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1773568,"data_stored":1133550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40664,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331176,"statfs":{"total":96636764160,"available":96595124224,"internally_reserved":0,"allocated":1191936,"data_stored":545342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40660,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331180,"statfs":{"total":96636764160,"available":96595128320,"internally_reserved":0,"allocated":1179648,"data_stored":543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40644,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331196,"statfs":{"total":96636764160,"available":96595144704,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1175552,"data_stored":543196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41256,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1789952,"data_stored":1135812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:44:47.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:44:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.568+0000 7f01aa400640 1 Processor -- start 2024-04-26T23:44:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.569+0000 7f01aa400640 1 -- start start 2024-04-26T23:44:47.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.570+0000 7f01aa400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a406ec20 0x7f01a40734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01aa400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a40b3e80 0x7f01a40739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01aa400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a40b43f0 con 0x7f01a4142280 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01aa400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f01a40b46d0 con 0x7f01a40b3e80 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01aa400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01a40b49b0 con 0x7f01a406ec20 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01a3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a40b3e80 0x7f01a40739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01a3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a40b3e80 0x7f01a40739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60160/0 (socket says 172.21.15.46:60160) 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01a3e00640 1 -- 172.21.15.46:0/4039633866 learned_addr learned my addr 172.21.15.46:0/4039633866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01a2a00640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a406ec20 0x7f01a40734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.571+0000 7f01a3e00640 1 -- 172.21.15.46:0/4039633866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a406ec20 msgr2=0x7f01a40734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.572+0000 7f01a3e00640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a406ec20 0x7f01a40734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.572+0000 7f01a3e00640 1 -- 172.21.15.46:0/4039633866 >> v1:172.21.15.142:6789/0 conn(0x7f01a4142280 legacy=0x7f01a406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.572+0000 7f01a3e00640 1 -- 172.21.15.46:0/4039633866 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a40b4c90 con 0x7f01a40b3e80 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.573+0000 7f01a3e00640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b3e80 0x7f01a40739e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f019c009870 tx=0x7f019c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80b2f178c59cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.573+0000 7f01a2000640 1 -- 172.21.15.46:0/4039633866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c037070 con 0x7f01a40b3e80 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.573+0000 7f01a2000640 1 -- 172.21.15.46:0/4039633866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c032440 con 0x7f01a40b3e80 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.573+0000 7f01a2000640 1 -- 172.21.15.46:0/4039633866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c032810 con 0x7f01a40b3e80 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 -- 172.21.15.46:0/4039633866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b3e80 msgr2=0x7f01a40739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b3e80 0x7f01a40739e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f019c009870 tx=0x7f019c02a250 comp rx=0 tx=0).stop 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 -- 172.21.15.46:0/4039633866 shutdown_connections 2024-04-26T23:44:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b3e80 0x7f01a40739e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 --2- 172.21.15.46:0/4039633866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a406ec20 0x7f01a40734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 -- 172.21.15.46:0/4039633866 >> 172.21.15.46:0/4039633866 conn(0x7f01a406c530 msgr2=0x7f01a406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 -- 172.21.15.46:0/4039633866 shutdown_connections 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.574+0000 7f01aa400640 1 -- 172.21.15.46:0/4039633866 wait complete. 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.576+0000 7f01aa400640 1 Processor -- start 2024-04-26T23:44:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.576+0000 7f01aa400640 1 -- start start 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 0x7f01a4140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a4142280 0x7f01a41f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4142a20 con 0x7f01a406ec20 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01a41428a0 con 0x7f01a4142280 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01aa400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01a4073f20 con 0x7f01a40b3e80 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60172/0 (socket says 172.21.15.46:60172) 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 -- 172.21.15.46:0/3440444295 learned_addr learned my addr 172.21.15.46:0/3440444295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a2a00640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 0x7f01a4140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a4142280 0x7f01a41f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 -- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 msgr2=0x7f01a4140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 0x7f01a4140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 -- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a4142280 msgr2=0x7f01a41f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.577+0000 7f01a3400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a4142280 0x7f01a41f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f01a3400640 1 -- 172.21.15.46:0/3440444295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a415e4c0 con 0x7f01a406ec20 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f01a2a00640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 0x7f01a4140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f01a3400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f019400c3c0 tx=0x7f019400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940052d0 con 0x7f01a406ec20 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194005490 con 0x7f01a406ec20 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007260 con 0x7f01a406ec20 2024-04-26T23:44:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a415e7a0 con 0x7f01a406ec20 2024-04-26T23:44:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.578+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a415ed60 con 0x7f01a406ec20 2024-04-26T23:44:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.580+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a4069530 con 0x7f01a406ec20 2024-04-26T23:44:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.581+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0194007400 con 0x7f01a406ec20 2024-04-26T23:44:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.582+0000 7f019b400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.582+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f019408b150 con 0x7f01a406ec20 2024-04-26T23:44:47.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.583+0000 7f01a2a00640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:47.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.584+0000 7f01a2a00640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f01a41f8760 tx=0x7f018c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.587+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0194059480 con 0x7f01a406ec20 2024-04-26T23:44:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.787+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01a4144bd0 con 0x7f018006c8f0 2024-04-26T23:44:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.806+0000 7f019b400640 1 -- 172.21.15.46:0/3440444295 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252075 (secure 0 0 0) 0x7f018c0588c0 con 0x7f018006c8f0 2024-04-26T23:44:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:44:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 msgr2=0x7f018006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f01a41f8760 tx=0x7f018c00b040 comp rx=0 tx=0).stop 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 msgr2=0x7f01a40ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f019400c3c0 tx=0x7f019400c880 comp rx=0 tx=0).stop 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 shutdown_connections 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a4142280 0x7f01a41f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40b3e80 0x7f01a4140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 --2- 172.21.15.46:0/3440444295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a406ec20 0x7f01a40ce270 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.811+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 >> 172.21.15.46:0/3440444295 conn(0x7f01a406c530 msgr2=0x7f01a40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.812+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 shutdown_connections 2024-04-26T23:44:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:47.812+0000 7f01aa400640 1 -- 172.21.15.46:0/3440444295 wait complete. 2024-04-26T23:44:47.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-26T23:44:46.055989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318088,"kb_used_data":10376,"kb_used_omap":12,"kb_used_meta":307571,"kb_avail":754656632,"statfs":{"total":773094113280,"available":772768391168,"internally_reserved":0,"allocated":10625024,"data_stored":5530564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":314953265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.257184"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309821+0000","last_change":"2024-04-26T23:44:42.872520+0000","last_active":"2024-04-26T23:44:44.309821+0000","last_peered":"2024-04-26T23:44:44.309821+0000","last_clean":"2024-04-26T23:44:44.309821+0000","last_became_active":"2024-04-26T23:44:42.872226+0000","last_became_peered":"2024-04-26T23:44:42.872226+0000","last_unstale":"2024-04-26T23:44:44.309821+0000","last_undegraded":"2024-04-26T23:44:44.309821+0000","last_fullsized":"2024-04-26T23:44:44.309821+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:28.591442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317146+0000","last_change":"2024-04-26T23:44:42.863475+0000","last_active":"2024-04-26T23:44:44.317146+0000","last_peered":"2024-04-26T23:44:44.317146+0000","last_clean":"2024-04-26T23:44:44.317146+0000","last_became_active":"2024-04-26T23:44:42.863105+0000","last_became_peered":"2024-04-26T23:44:42.863105+0000","last_unstale":"2024-04-26T23:44:44.317146+0000","last_undegraded":"2024-04-26T23:44:44.317146+0000","last_fullsized":"2024-04-26T23:44:44.317146+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:11.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308904+0000","last_change":"2024-04-26T23:44:42.859580+0000","last_active":"2024-04-26T23:44:44.308904+0000","last_peered":"2024-04-26T23:44:44.308904+0000","last_clean":"2024-04-26T23:44:44.308904+0000","last_became_active":"2024-04-26T23:44:42.857639+0000","last_became_peered":"2024-04-26T23:44:42.857639+0000","last_unstale":"2024-04-26T23:44:44.308904+0000","last_undegraded":"2024-04-26T23:44:44.308904+0000","last_fullsized":"2024-04-26T23:44:44.308904+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:42.558949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863808+0000","last_change":"2024-04-26T23:44:42.871232+0000","last_active":"2024-04-26T23:44:43.863808+0000","last_peered":"2024-04-26T23:44:43.863808+0000","last_clean":"2024-04-26T23:44:43.863808+0000","last_became_active":"2024-04-26T23:44:42.869991+0000","last_became_peered":"2024-04-26T23:44:42.869991+0000","last_unstale":"2024-04-26T23:44:43.863808+0000","last_undegraded":"2024-04-26T23:44:43.863808+0000","last_fullsized":"2024-04-26T23:44:43.863808+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.831699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862915+0000","last_change":"2024-04-26T23:44:42.868688+0000","last_active":"2024-04-26T23:44:43.862915+0000","last_peered":"2024-04-26T23:44:43.862915+0000","last_clean":"2024-04-26T23:44:43.862915+0000","last_became_active":"2024-04-26T23:44:42.868238+0000","last_became_peered":"2024-04-26T23:44:42.868238+0000","last_unstale":"2024-04-26T23:44:43.862915+0000","last_undegraded":"2024-04-26T23:44:43.862915+0000","last_fullsized":"2024-04-26T23:44:43.862915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861273+0000","last_change":"2024-04-26T23:44:42.875391+0000","last_active":"2024-04-26T23:44:43.861273+0000","last_peered":"2024-04-26T23:44:43.861273+0000","last_clean":"2024-04-26T23:44:43.861273+0000","last_became_active":"2024-04-26T23:44:42.875103+0000","last_became_peered":"2024-04-26T23:44:42.875103+0000","last_unstale":"2024-04-26T23:44:43.861273+0000","last_undegraded":"2024-04-26T23:44:43.861273+0000","last_fullsized":"2024-04-26T23:44:43.861273+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:14.704015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616686+0000","last_change":"2024-04-26T23:44:42.871842+0000","last_active":"2024-04-26T23:44:44.616686+0000","last_peered":"2024-04-26T23:44:44.616686+0000","last_clean":"2024-04-26T23:44:44.616686+0000","last_became_active":"2024-04-26T23:44:42.871467+0000","last_became_peered":"2024-04-26T23:44:42.871467+0000","last_unstale":"2024-04-26T23:44:44.616686+0000","last_undegraded":"2024-04-26T23:44:44.616686+0000","last_fullsized":"2024-04-26T23:44:44.616686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.644330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862443+0000","last_change":"2024-04-26T23:44:42.879136+0000","last_active":"2024-04-26T23:44:43.862443+0000","last_peered":"2024-04-26T23:44:43.862443+0000","last_clean":"2024-04-26T23:44:43.862443+0000","last_became_active":"2024-04-26T23:44:42.870794+0000","last_became_peered":"2024-04-26T23:44:42.870794+0000","last_unstale":"2024-04-26T23:44:43.862443+0000","last_undegraded":"2024-04-26T23:44:43.862443+0000","last_fullsized":"2024-04-26T23:44:43.862443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863534+0000","last_change":"2024-04-26T23:44:42.854698+0000","last_active":"2024-04-26T23:44:43.863534+0000","last_peered":"2024-04-26T23:44:43.863534+0000","last_clean":"2024-04-26T23:44:43.863534+0000","last_became_active":"2024-04-26T23:44:42.854294+0000","last_became_peered":"2024-04-26T23:44:42.854294+0000","last_unstale":"2024-04-26T23:44:43.863534+0000","last_undegraded":"2024-04-26T23:44:43.863534+0000","last_fullsized":"2024-04-26T23:44:43.863534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:13.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790681+0000","last_change":"2024-04-26T23:44:42.876750+0000","last_active":"2024-04-26T23:44:44.790681+0000","last_peered":"2024-04-26T23:44:44.790681+0000","last_clean":"2024-04-26T23:44:44.790681+0000","last_became_active":"2024-04-26T23:44:42.876408+0000","last_became_peered":"2024-04-26T23:44:42.876408+0000","last_unstale":"2024-04-26T23:44:44.790681+0000","last_undegraded":"2024-04-26T23:44:44.790681+0000","last_fullsized":"2024-04-26T23:44:44.790681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:22.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863178+0000","last_change":"2024-04-26T23:44:42.855897+0000","last_active":"2024-04-26T23:44:43.863178+0000","last_peered":"2024-04-26T23:44:43.863178+0000","last_clean":"2024-04-26T23:44:43.863178+0000","last_became_active":"2024-04-26T23:44:42.855619+0000","last_became_peered":"2024-04-26T23:44:42.855619+0000","last_unstale":"2024-04-26T23:44:43.863178+0000","last_undegraded":"2024-04-26T23:44:43.863178+0000","last_fullsized":"2024-04-26T23:44:43.863178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:49.259655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789972+0000","last_change":"2024-04-26T23:44:42.860344+0000","last_active":"2024-04-26T23:44:44.789972+0000","last_peered":"2024-04-26T23:44:44.789972+0000","last_clean":"2024-04-26T23:44:44.789972+0000","last_became_active":"2024-04-26T23:44:42.860075+0000","last_became_peered":"2024-04-26T23:44:42.860075+0000","last_unstale":"2024-04-26T23:44:44.789972+0000","last_undegraded":"2024-04-26T23:44:44.789972+0000","last_fullsized":"2024-04-26T23:44:44.789972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:12.382214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863269+0000","last_change":"2024-04-26T23:44:42.868000+0000","last_active":"2024-04-26T23:44:43.863269+0000","last_peered":"2024-04-26T23:44:43.863269+0000","last_clean":"2024-04-26T23:44:43.863269+0000","last_became_active":"2024-04-26T23:44:42.867321+0000","last_became_peered":"2024-04-26T23:44:42.867321+0000","last_unstale":"2024-04-26T23:44:43.863269+0000","last_undegraded":"2024-04-26T23:44:43.863269+0000","last_fullsized":"2024-04-26T23:44:43.863269+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:35.816205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862180+0000","last_change":"2024-04-26T23:44:42.872461+0000","last_active":"2024-04-26T23:44:43.862180+0000","last_peered":"2024-04-26T23:44:43.862180+0000","last_clean":"2024-04-26T23:44:43.862180+0000","last_became_active":"2024-04-26T23:44:42.872072+0000","last_became_peered":"2024-04-26T23:44:42.872072+0000","last_unstale":"2024-04-26T23:44:43.862180+0000","last_undegraded":"2024-04-26T23:44:43.862180+0000","last_fullsized":"2024-04-26T23:44:43.862180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:13.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862972+0000","last_change":"2024-04-26T23:44:42.863448+0000","last_active":"2024-04-26T23:44:43.862972+0000","last_peered":"2024-04-26T23:44:43.862972+0000","last_clean":"2024-04-26T23:44:43.862972+0000","last_became_active":"2024-04-26T23:44:42.854980+0000","last_became_peered":"2024-04-26T23:44:42.854980+0000","last_unstale":"2024-04-26T23:44:43.862972+0000","last_undegraded":"2024-04-26T23:44:43.862972+0000","last_fullsized":"2024-04-26T23:44:43.862972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:49.259655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.859914+0000","last_change":"2024-04-26T23:44:42.857036+0000","last_active":"2024-04-26T23:44:43.859914+0000","last_peered":"2024-04-26T23:44:43.859914+0000","last_clean":"2024-04-26T23:44:43.859914+0000","last_became_active":"2024-04-26T23:44:42.856742+0000","last_became_peered":"2024-04-26T23:44:42.856742+0000","last_unstale":"2024-04-26T23:44:43.859914+0000","last_undegraded":"2024-04-26T23:44:43.859914+0000","last_fullsized":"2024-04-26T23:44:43.859914+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:30.868226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861389+0000","last_change":"2024-04-26T23:44:42.854684+0000","last_active":"2024-04-26T23:44:43.861389+0000","last_peered":"2024-04-26T23:44:43.861389+0000","last_clean":"2024-04-26T23:44:43.861389+0000","last_became_active":"2024-04-26T23:44:42.854486+0000","last_became_peered":"2024-04-26T23:44:42.854486+0000","last_unstale":"2024-04-26T23:44:43.861389+0000","last_undegraded":"2024-04-26T23:44:43.861389+0000","last_fullsized":"2024-04-26T23:44:43.861389+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:45.526881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309182+0000","last_change":"2024-04-26T23:44:42.860568+0000","last_active":"2024-04-26T23:44:44.309182+0000","last_peered":"2024-04-26T23:44:44.309182+0000","last_clean":"2024-04-26T23:44:44.309182+0000","last_became_active":"2024-04-26T23:44:42.860231+0000","last_became_peered":"2024-04-26T23:44:42.860231+0000","last_unstale":"2024-04-26T23:44:44.309182+0000","last_undegraded":"2024-04-26T23:44:44.309182+0000","last_fullsized":"2024-04-26T23:44:44.309182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.647805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863369+0000","last_change":"2024-04-26T23:44:42.869489+0000","last_active":"2024-04-26T23:44:43.863369+0000","last_peered":"2024-04-26T23:44:43.863369+0000","last_clean":"2024-04-26T23:44:43.863369+0000","last_became_active":"2024-04-26T23:44:42.869116+0000","last_became_peered":"2024-04-26T23:44:42.869116+0000","last_unstale":"2024-04-26T23:44:43.863369+0000","last_undegraded":"2024-04-26T23:44:43.863369+0000","last_fullsized":"2024-04-26T23:44:43.863369+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:54.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863874+0000","last_change":"2024-04-26T23:44:42.864540+0000","last_active":"2024-04-26T23:44:43.863874+0000","last_peered":"2024-04-26T23:44:43.863874+0000","last_clean":"2024-04-26T23:44:43.863874+0000","last_became_active":"2024-04-26T23:44:42.863046+0000","last_became_peered":"2024-04-26T23:44:42.863046+0000","last_unstale":"2024-04-26T23:44:43.863874+0000","last_undegraded":"2024-04-26T23:44:43.863874+0000","last_fullsized":"2024-04-26T23:44:43.863874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:31.593390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862450+0000","last_change":"2024-04-26T23:44:42.865765+0000","last_active":"2024-04-26T23:44:43.862450+0000","last_peered":"2024-04-26T23:44:43.862450+0000","last_clean":"2024-04-26T23:44:43.862450+0000","last_became_active":"2024-04-26T23:44:42.865460+0000","last_became_peered":"2024-04-26T23:44:42.865460+0000","last_unstale":"2024-04-26T23:44:43.862450+0000","last_undegraded":"2024-04-26T23:44:43.862450+0000","last_fullsized":"2024-04-26T23:44:43.862450+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:07.089428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862574+0000","last_change":"2024-04-26T23:44:42.866704+0000","last_active":"2024-04-26T23:44:43.862574+0000","last_peered":"2024-04-26T23:44:43.862574+0000","last_clean":"2024-04-26T23:44:43.862574+0000","last_became_active":"2024-04-26T23:44:42.866401+0000","last_became_peered":"2024-04-26T23:44:42.866401+0000","last_unstale":"2024-04-26T23:44:43.862574+0000","last_undegraded":"2024-04-26T23:44:43.862574+0000","last_fullsized":"2024-04-26T23:44:43.862574+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607410+0000","last_change":"2024-04-26T23:44:42.871487+0000","last_active":"2024-04-26T23:44:44.607410+0000","last_peered":"2024-04-26T23:44:44.607410+0000","last_clean":"2024-04-26T23:44:44.607410+0000","last_became_active":"2024-04-26T23:44:42.871172+0000","last_became_peered":"2024-04-26T23:44:42.871172+0000","last_unstale":"2024-04-26T23:44:44.607410+0000","last_undegraded":"2024-04-26T23:44:44.607410+0000","last_fullsized":"2024-04-26T23:44:44.607410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:34.418179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308918+0000","last_change":"2024-04-26T23:44:42.861593+0000","last_active":"2024-04-26T23:44:44.308918+0000","last_peered":"2024-04-26T23:44:44.308918+0000","last_clean":"2024-04-26T23:44:44.308918+0000","last_became_active":"2024-04-26T23:44:42.857745+0000","last_became_peered":"2024-04-26T23:44:42.857745+0000","last_unstale":"2024-04-26T23:44:44.308918+0000","last_undegraded":"2024-04-26T23:44:44.308918+0000","last_fullsized":"2024-04-26T23:44:44.308918+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:59.229331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317286+0000","last_change":"2024-04-26T23:44:42.871284+0000","last_active":"2024-04-26T23:44:44.317286+0000","last_peered":"2024-04-26T23:44:44.317286+0000","last_clean":"2024-04-26T23:44:44.317286+0000","last_became_active":"2024-04-26T23:44:42.870979+0000","last_became_peered":"2024-04-26T23:44:42.870979+0000","last_unstale":"2024-04-26T23:44:44.317286+0000","last_undegraded":"2024-04-26T23:44:44.317286+0000","last_fullsized":"2024-04-26T23:44:44.317286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.141723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863682+0000","last_change":"2024-04-26T23:44:42.872617+0000","last_active":"2024-04-26T23:44:43.863682+0000","last_peered":"2024-04-26T23:44:43.863682+0000","last_clean":"2024-04-26T23:44:43.863682+0000","last_became_active":"2024-04-26T23:44:42.872098+0000","last_became_peered":"2024-04-26T23:44:42.872098+0000","last_unstale":"2024-04-26T23:44:43.863682+0000","last_undegraded":"2024-04-26T23:44:43.863682+0000","last_fullsized":"2024-04-26T23:44:43.863682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.831699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790227+0000","last_change":"2024-04-26T23:44:42.871024+0000","last_active":"2024-04-26T23:44:44.790227+0000","last_peered":"2024-04-26T23:44:44.790227+0000","last_clean":"2024-04-26T23:44:44.790227+0000","last_became_active":"2024-04-26T23:44:42.870749+0000","last_became_peered":"2024-04-26T23:44:42.870749+0000","last_unstale":"2024-04-26T23:44:44.790227+0000","last_undegraded":"2024-04-26T23:44:44.790227+0000","last_fullsized":"2024-04-26T23:44:44.790227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:51.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.864129+0000","last_change":"2024-04-26T23:44:42.865396+0000","last_active":"2024-04-26T23:44:43.864129+0000","last_peered":"2024-04-26T23:44:43.864129+0000","last_clean":"2024-04-26T23:44:43.864129+0000","last_became_active":"2024-04-26T23:44:42.865046+0000","last_became_peered":"2024-04-26T23:44:42.865046+0000","last_unstale":"2024-04-26T23:44:43.864129+0000","last_undegraded":"2024-04-26T23:44:43.864129+0000","last_fullsized":"2024-04-26T23:44:43.864129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:09.246470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617100+0000","last_change":"2024-04-26T23:44:42.880347+0000","last_active":"2024-04-26T23:44:44.617100+0000","last_peered":"2024-04-26T23:44:44.617100+0000","last_clean":"2024-04-26T23:44:44.617100+0000","last_became_active":"2024-04-26T23:44:42.879981+0000","last_became_peered":"2024-04-26T23:44:42.879981+0000","last_unstale":"2024-04-26T23:44:44.617100+0000","last_undegraded":"2024-04-26T23:44:44.617100+0000","last_fullsized":"2024-04-26T23:44:44.617100+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:27.590839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862912+0000","last_change":"2024-04-26T23:44:42.853333+0000","last_active":"2024-04-26T23:44:43.862912+0000","last_peered":"2024-04-26T23:44:43.862912+0000","last_clean":"2024-04-26T23:44:43.862912+0000","last_became_active":"2024-04-26T23:44:42.853024+0000","last_became_peered":"2024-04-26T23:44:42.853024+0000","last_unstale":"2024-04-26T23:44:43.862912+0000","last_undegraded":"2024-04-26T23:44:43.862912+0000","last_fullsized":"2024-04-26T23:44:43.862912+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316728+0000","last_change":"2024-04-26T23:44:42.868712+0000","last_active":"2024-04-26T23:44:44.316728+0000","last_peered":"2024-04-26T23:44:44.316728+0000","last_clean":"2024-04-26T23:44:44.316728+0000","last_became_active":"2024-04-26T23:44:42.868285+0000","last_became_peered":"2024-04-26T23:44:42.868285+0000","last_unstale":"2024-04-26T23:44:44.316728+0000","last_undegraded":"2024-04-26T23:44:44.316728+0000","last_fullsized":"2024-04-26T23:44:44.316728+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:36.299392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.859870+0000","last_change":"2024-04-26T23:44:42.850843+0000","last_active":"2024-04-26T23:44:43.859870+0000","last_peered":"2024-04-26T23:44:43.859870+0000","last_clean":"2024-04-26T23:44:43.859870+0000","last_became_active":"2024-04-26T23:44:42.850535+0000","last_became_peered":"2024-04-26T23:44:42.850535+0000","last_unstale":"2024-04-26T23:44:43.859870+0000","last_undegraded":"2024-04-26T23:44:43.859870+0000","last_fullsized":"2024-04-26T23:44:43.859870+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:38:23.729676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860796+0000","last_change":"2024-04-26T23:44:42.858443+0000","last_active":"2024-04-26T23:44:43.860796+0000","last_peered":"2024-04-26T23:44:43.860796+0000","last_clean":"2024-04-26T23:44:43.860796+0000","last_became_active":"2024-04-26T23:44:42.858083+0000","last_became_peered":"2024-04-26T23:44:42.858083+0000","last_unstale":"2024-04-26T23:44:43.860796+0000","last_undegraded":"2024-04-26T23:44:43.860796+0000","last_fullsized":"2024-04-26T23:44:43.860796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:30.903702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607853+0000","last_change":"2024-04-26T23:44:41.841523+0000","last_active":"2024-04-26T23:44:44.607853+0000","last_peered":"2024-04-26T23:44:44.607853+0000","last_clean":"2024-04-26T23:44:44.607853+0000","last_became_active":"2024-04-26T23:44:41.841245+0000","last_became_peered":"2024-04-26T23:44:41.841245+0000","last_unstale":"2024-04-26T23:44:44.607853+0000","last_undegraded":"2024-04-26T23:44:44.607853+0000","last_fullsized":"2024-04-26T23:44:44.607853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:27.611799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861876+0000","last_change":"2024-04-26T23:44:42.872181+0000","last_active":"2024-04-26T23:44:43.861876+0000","last_peered":"2024-04-26T23:44:43.861876+0000","last_clean":"2024-04-26T23:44:43.861876+0000","last_became_active":"2024-04-26T23:44:42.871882+0000","last_became_peered":"2024-04-26T23:44:42.871882+0000","last_unstale":"2024-04-26T23:44:43.861876+0000","last_undegraded":"2024-04-26T23:44:43.861876+0000","last_fullsized":"2024-04-26T23:44:43.861876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:06.813953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863832+0000","last_change":"2024-04-26T23:44:41.841247+0000","last_active":"2024-04-26T23:44:43.863832+0000","last_peered":"2024-04-26T23:44:43.863832+0000","last_clean":"2024-04-26T23:44:43.863832+0000","last_became_active":"2024-04-26T23:44:41.840948+0000","last_became_peered":"2024-04-26T23:44:41.840948+0000","last_unstale":"2024-04-26T23:44:43.863832+0000","last_undegraded":"2024-04-26T23:44:43.863832+0000","last_fullsized":"2024-04-26T23:44:43.863832+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:57.025808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617002+0000","last_change":"2024-04-26T23:44:42.879903+0000","last_active":"2024-04-26T23:44:44.617002+0000","last_peered":"2024-04-26T23:44:44.617002+0000","last_clean":"2024-04-26T23:44:44.617002+0000","last_became_active":"2024-04-26T23:44:42.879443+0000","last_became_peered":"2024-04-26T23:44:42.879443+0000","last_unstale":"2024-04-26T23:44:44.617002+0000","last_undegraded":"2024-04-26T23:44:44.617002+0000","last_fullsized":"2024-04-26T23:44:44.617002+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:00.754030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617245+0000","last_change":"2024-04-26T23:44:41.844262+0000","last_active":"2024-04-26T23:44:44.617245+0000","last_peered":"2024-04-26T23:44:44.617245+0000","last_clean":"2024-04-26T23:44:44.617245+0000","last_became_active":"2024-04-26T23:44:41.843883+0000","last_became_peered":"2024-04-26T23:44:41.843883+0000","last_unstale":"2024-04-26T23:44:44.617245+0000","last_undegraded":"2024-04-26T23:44:44.617245+0000","last_fullsized":"2024-04-26T23:44:44.617245+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:02.069303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861219+0000","last_change":"2024-04-26T23:44:42.870657+0000","last_active":"2024-04-26T23:44:43.861219+0000","last_peered":"2024-04-26T23:44:43.861219+0000","last_clean":"2024-04-26T23:44:43.861219+0000","last_became_active":"2024-04-26T23:44:42.870357+0000","last_became_peered":"2024-04-26T23:44:42.870357+0000","last_unstale":"2024-04-26T23:44:43.861219+0000","last_undegraded":"2024-04-26T23:44:43.861219+0000","last_fullsized":"2024-04-26T23:44:43.861219+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:31.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607852+0000","last_change":"2024-04-26T23:44:41.846659+0000","last_active":"2024-04-26T23:44:44.607852+0000","last_peered":"2024-04-26T23:44:44.607852+0000","last_clean":"2024-04-26T23:44:44.607852+0000","last_became_active":"2024-04-26T23:44:41.846217+0000","last_became_peered":"2024-04-26T23:44:41.846217+0000","last_unstale":"2024-04-26T23:44:44.607852+0000","last_undegraded":"2024-04-26T23:44:44.607852+0000","last_fullsized":"2024-04-26T23:44:44.607852+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:18.279295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862499+0000","last_change":"2024-04-26T23:44:42.871507+0000","last_active":"2024-04-26T23:44:43.862499+0000","last_peered":"2024-04-26T23:44:43.862499+0000","last_clean":"2024-04-26T23:44:43.862499+0000","last_became_active":"2024-04-26T23:44:42.871185+0000","last_became_peered":"2024-04-26T23:44:42.871185+0000","last_unstale":"2024-04-26T23:44:43.862499+0000","last_undegraded":"2024-04-26T23:44:43.862499+0000","last_fullsized":"2024-04-26T23:44:43.862499+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863081+0000","last_change":"2024-04-26T23:44:41.832254+0000","last_active":"2024-04-26T23:44:43.863081+0000","last_peered":"2024-04-26T23:44:43.863081+0000","last_clean":"2024-04-26T23:44:43.863081+0000","last_became_active":"2024-04-26T23:44:41.831862+0000","last_became_peered":"2024-04-26T23:44:41.831862+0000","last_unstale":"2024-04-26T23:44:43.863081+0000","last_undegraded":"2024-04-26T23:44:43.863081+0000","last_fullsized":"2024-04-26T23:44:43.863081+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:26.487332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607530+0000","last_change":"2024-04-26T23:44:42.875992+0000","last_active":"2024-04-26T23:44:44.607530+0000","last_peered":"2024-04-26T23:44:44.607530+0000","last_clean":"2024-04-26T23:44:44.607530+0000","last_became_active":"2024-04-26T23:44:42.875405+0000","last_became_peered":"2024-04-26T23:44:42.875405+0000","last_unstale":"2024-04-26T23:44:44.607530+0000","last_undegraded":"2024-04-26T23:44:44.607530+0000","last_fullsized":"2024-04-26T23:44:44.607530+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:17.890498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863617+0000","last_change":"2024-04-26T23:44:41.837359+0000","last_active":"2024-04-26T23:44:43.863617+0000","last_peered":"2024-04-26T23:44:43.863617+0000","last_clean":"2024-04-26T23:44:43.863617+0000","last_became_active":"2024-04-26T23:44:41.837041+0000","last_became_peered":"2024-04-26T23:44:41.837041+0000","last_unstale":"2024-04-26T23:44:43.863617+0000","last_undegraded":"2024-04-26T23:44:43.863617+0000","last_fullsized":"2024-04-26T23:44:43.863617+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.617796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860603+0000","last_change":"2024-04-26T23:44:42.875213+0000","last_active":"2024-04-26T23:44:43.860603+0000","last_peered":"2024-04-26T23:44:43.860603+0000","last_clean":"2024-04-26T23:44:43.860603+0000","last_became_active":"2024-04-26T23:44:42.874888+0000","last_became_peered":"2024-04-26T23:44:42.874888+0000","last_unstale":"2024-04-26T23:44:43.860603+0000","last_undegraded":"2024-04-26T23:44:43.860603+0000","last_fullsized":"2024-04-26T23:44:43.860603+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:55.092598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309111+0000","last_change":"2024-04-26T23:44:41.830584+0000","last_active":"2024-04-26T23:44:44.309111+0000","last_peered":"2024-04-26T23:44:44.309111+0000","last_clean":"2024-04-26T23:44:44.309111+0000","last_became_active":"2024-04-26T23:44:41.830257+0000","last_became_peered":"2024-04-26T23:44:41.830257+0000","last_unstale":"2024-04-26T23:44:44.309111+0000","last_undegraded":"2024-04-26T23:44:44.309111+0000","last_fullsized":"2024-04-26T23:44:44.309111+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.943401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863357+0000","last_change":"2024-04-26T23:44:42.869744+0000","last_active":"2024-04-26T23:44:43.863357+0000","last_peered":"2024-04-26T23:44:43.863357+0000","last_clean":"2024-04-26T23:44:43.863357+0000","last_became_active":"2024-04-26T23:44:42.867745+0000","last_became_peered":"2024-04-26T23:44:42.867745+0000","last_unstale":"2024-04-26T23:44:43.863357+0000","last_undegraded":"2024-04-26T23:44:43.863357+0000","last_fullsized":"2024-04-26T23:44:43.863357+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:54.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616748+0000","last_change":"2024-04-26T23:44:41.842153+0000","last_active":"2024-04-26T23:44:44.616748+0000","last_peered":"2024-04-26T23:44:44.616748+0000","last_clean":"2024-04-26T23:44:44.616748+0000","last_became_active":"2024-04-26T23:44:41.841780+0000","last_became_peered":"2024-04-26T23:44:41.841780+0000","last_unstale":"2024-04-26T23:44:44.616748+0000","last_undegraded":"2024-04-26T23:44:44.616748+0000","last_fullsized":"2024-04-26T23:44:44.616748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:03.904868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863224+0000","last_change":"2024-04-26T23:44:42.854464+0000","last_active":"2024-04-26T23:44:43.863224+0000","last_peered":"2024-04-26T23:44:43.863224+0000","last_clean":"2024-04-26T23:44:43.863224+0000","last_became_active":"2024-04-26T23:44:42.853680+0000","last_became_peered":"2024-04-26T23:44:42.853680+0000","last_unstale":"2024-04-26T23:44:43.863224+0000","last_undegraded":"2024-04-26T23:44:43.863224+0000","last_fullsized":"2024-04-26T23:44:43.863224+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:35.816205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317363+0000","last_change":"2024-04-26T23:44:41.845695+0000","last_active":"2024-04-26T23:44:44.317363+0000","last_peered":"2024-04-26T23:44:44.317363+0000","last_clean":"2024-04-26T23:44:44.317363+0000","last_became_active":"2024-04-26T23:44:41.845355+0000","last_became_peered":"2024-04-26T23:44:41.845355+0000","last_unstale":"2024-04-26T23:44:44.317363+0000","last_undegraded":"2024-04-26T23:44:44.317363+0000","last_fullsized":"2024-04-26T23:44:44.317363+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:08.232437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617491+0000","last_change":"2024-04-26T23:44:42.867907+0000","last_active":"2024-04-26T23:44:44.617491+0000","last_peered":"2024-04-26T23:44:44.617491+0000","last_clean":"2024-04-26T23:44:44.617491+0000","last_became_active":"2024-04-26T23:44:42.867586+0000","last_became_peered":"2024-04-26T23:44:42.867586+0000","last_unstale":"2024-04-26T23:44:44.617491+0000","last_undegraded":"2024-04-26T23:44:44.617491+0000","last_fullsized":"2024-04-26T23:44:44.617491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:36.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309349+0000","last_change":"2024-04-26T23:44:41.835268+0000","last_active":"2024-04-26T23:44:44.309349+0000","last_peered":"2024-04-26T23:44:44.309349+0000","last_clean":"2024-04-26T23:44:44.309349+0000","last_became_active":"2024-04-26T23:44:41.834973+0000","last_became_peered":"2024-04-26T23:44:41.834973+0000","last_unstale":"2024-04-26T23:44:44.309349+0000","last_undegraded":"2024-04-26T23:44:44.309349+0000","last_fullsized":"2024-04-26T23:44:44.309349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:09.664813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.309635+0000","last_change":"2024-04-26T23:44:42.854706+0000","last_active":"2024-04-26T23:44:44.309635+0000","last_peered":"2024-04-26T23:44:44.309635+0000","last_clean":"2024-04-26T23:44:44.309635+0000","last_became_active":"2024-04-26T23:44:42.852315+0000","last_became_peered":"2024-04-26T23:44:42.852315+0000","last_unstale":"2024-04-26T23:44:44.309635+0000","last_undegraded":"2024-04-26T23:44:44.309635+0000","last_fullsized":"2024-04-26T23:44:44.309635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:49.880815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863459+0000","last_change":"2024-04-26T23:44:41.839936+0000","last_active":"2024-04-26T23:44:43.863459+0000","last_peered":"2024-04-26T23:44:43.863459+0000","last_clean":"2024-04-26T23:44:43.863459+0000","last_became_active":"2024-04-26T23:44:41.836483+0000","last_became_peered":"2024-04-26T23:44:41.836483+0000","last_unstale":"2024-04-26T23:44:43.863459+0000","last_undegraded":"2024-04-26T23:44:43.863459+0000","last_fullsized":"2024-04-26T23:44:43.863459+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:40.608548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863701+0000","last_change":"2024-04-26T23:44:42.858355+0000","last_active":"2024-04-26T23:44:43.863701+0000","last_peered":"2024-04-26T23:44:43.863701+0000","last_clean":"2024-04-26T23:44:43.863701+0000","last_became_active":"2024-04-26T23:44:42.857610+0000","last_became_peered":"2024-04-26T23:44:42.857610+0000","last_unstale":"2024-04-26T23:44:43.863701+0000","last_undegraded":"2024-04-26T23:44:43.863701+0000","last_fullsized":"2024-04-26T23:44:43.863701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:13.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861113+0000","last_change":"2024-04-26T23:44:41.840943+0000","last_active":"2024-04-26T23:44:43.861113+0000","last_peered":"2024-04-26T23:44:43.861113+0000","last_clean":"2024-04-26T23:44:43.861113+0000","last_became_active":"2024-04-26T23:44:41.840305+0000","last_became_peered":"2024-04-26T23:44:41.840305+0000","last_unstale":"2024-04-26T23:44:43.861113+0000","last_undegraded":"2024-04-26T23:44:43.861113+0000","last_fullsized":"2024-04-26T23:44:43.861113+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:35.285359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862664+0000","last_change":"2024-04-26T23:44:42.867729+0000","last_active":"2024-04-26T23:44:43.862664+0000","last_peered":"2024-04-26T23:44:43.862664+0000","last_clean":"2024-04-26T23:44:43.862664+0000","last_became_active":"2024-04-26T23:44:42.867141+0000","last_became_peered":"2024-04-26T23:44:42.867141+0000","last_unstale":"2024-04-26T23:44:43.862664+0000","last_undegraded":"2024-04-26T23:44:43.862664+0000","last_fullsized":"2024-04-26T23:44:43.862664+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616777+0000","last_change":"2024-04-26T23:44:41.845091+0000","last_active":"2024-04-26T23:44:44.616777+0000","last_peered":"2024-04-26T23:44:44.616777+0000","last_clean":"2024-04-26T23:44:44.616777+0000","last_became_active":"2024-04-26T23:44:41.844728+0000","last_became_peered":"2024-04-26T23:44:41.844728+0000","last_unstale":"2024-04-26T23:44:44.616777+0000","last_undegraded":"2024-04-26T23:44:44.616777+0000","last_fullsized":"2024-04-26T23:44:44.616777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:35.285359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862141+0000","last_change":"2024-04-26T23:44:42.859718+0000","last_active":"2024-04-26T23:44:43.862141+0000","last_peered":"2024-04-26T23:44:43.862141+0000","last_clean":"2024-04-26T23:44:43.862141+0000","last_became_active":"2024-04-26T23:44:42.859232+0000","last_became_peered":"2024-04-26T23:44:42.859232+0000","last_unstale":"2024-04-26T23:44:43.862141+0000","last_undegraded":"2024-04-26T23:44:43.862141+0000","last_fullsized":"2024-04-26T23:44:43.862141+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:15.559779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860676+0000","last_change":"2024-04-26T23:44:41.835160+0000","last_active":"2024-04-26T23:44:43.860676+0000","last_peered":"2024-04-26T23:44:43.860676+0000","last_clean":"2024-04-26T23:44:43.860676+0000","last_became_active":"2024-04-26T23:44:41.834795+0000","last_became_peered":"2024-04-26T23:44:41.834795+0000","last_unstale":"2024-04-26T23:44:43.860676+0000","last_undegraded":"2024-04-26T23:44:43.860676+0000","last_fullsized":"2024-04-26T23:44:43.860676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:33.478880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.790479+0000","last_change":"2024-04-26T23:44:42.872468+0000","last_active":"2024-04-26T23:44:44.790479+0000","last_peered":"2024-04-26T23:44:44.790479+0000","last_clean":"2024-04-26T23:44:44.790479+0000","last_became_active":"2024-04-26T23:44:42.872245+0000","last_became_peered":"2024-04-26T23:44:42.872245+0000","last_unstale":"2024-04-26T23:44:44.790479+0000","last_undegraded":"2024-04-26T23:44:44.790479+0000","last_fullsized":"2024-04-26T23:44:44.790479+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:32.463740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863033+0000","last_change":"2024-04-26T23:44:41.842232+0000","last_active":"2024-04-26T23:44:43.863033+0000","last_peered":"2024-04-26T23:44:43.863033+0000","last_clean":"2024-04-26T23:44:43.863033+0000","last_became_active":"2024-04-26T23:44:41.841916+0000","last_became_peered":"2024-04-26T23:44:41.841916+0000","last_unstale":"2024-04-26T23:44:43.863033+0000","last_undegraded":"2024-04-26T23:44:43.863033+0000","last_fullsized":"2024-04-26T23:44:43.863033+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:56.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789978+0000","last_change":"2024-04-26T23:44:42.871143+0000","last_active":"2024-04-26T23:44:44.789978+0000","last_peered":"2024-04-26T23:44:44.789978+0000","last_clean":"2024-04-26T23:44:44.789978+0000","last_became_active":"2024-04-26T23:44:42.868768+0000","last_became_peered":"2024-04-26T23:44:42.868768+0000","last_unstale":"2024-04-26T23:44:44.789978+0000","last_undegraded":"2024-04-26T23:44:44.789978+0000","last_fullsized":"2024-04-26T23:44:44.789978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.884286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.862577+0000","last_change":"2024-04-26T23:44:41.831009+0000","last_active":"2024-04-26T23:44:43.862577+0000","last_peered":"2024-04-26T23:44:43.862577+0000","last_clean":"2024-04-26T23:44:43.862577+0000","last_became_active":"2024-04-26T23:44:41.830543+0000","last_became_peered":"2024-04-26T23:44:41.830543+0000","last_unstale":"2024-04-26T23:44:43.862577+0000","last_undegraded":"2024-04-26T23:44:43.862577+0000","last_fullsized":"2024-04-26T23:44:43.862577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:27.611799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861105+0000","last_change":"2024-04-26T23:44:42.859660+0000","last_active":"2024-04-26T23:44:43.861105+0000","last_peered":"2024-04-26T23:44:43.861105+0000","last_clean":"2024-04-26T23:44:43.861105+0000","last_became_active":"2024-04-26T23:44:42.859102+0000","last_became_peered":"2024-04-26T23:44:42.859102+0000","last_unstale":"2024-04-26T23:44:43.861105+0000","last_undegraded":"2024-04-26T23:44:43.861105+0000","last_fullsized":"2024-04-26T23:44:43.861105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:16.644278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.608139+0000","last_change":"2024-04-26T23:44:41.844834+0000","last_active":"2024-04-26T23:44:44.608139+0000","last_peered":"2024-04-26T23:44:44.608139+0000","last_clean":"2024-04-26T23:44:44.608139+0000","last_became_active":"2024-04-26T23:44:41.844491+0000","last_became_peered":"2024-04-26T23:44:41.844491+0000","last_unstale":"2024-04-26T23:44:44.608139+0000","last_undegraded":"2024-04-26T23:44:44.608139+0000","last_fullsized":"2024-04-26T23:44:44.608139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.617796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607665+0000","last_change":"2024-04-26T23:44:42.870254+0000","last_active":"2024-04-26T23:44:44.607665+0000","last_peered":"2024-04-26T23:44:44.607665+0000","last_clean":"2024-04-26T23:44:44.607665+0000","last_became_active":"2024-04-26T23:44:42.869761+0000","last_became_peered":"2024-04-26T23:44:42.869761+0000","last_unstale":"2024-04-26T23:44:44.607665+0000","last_undegraded":"2024-04-26T23:44:44.607665+0000","last_fullsized":"2024-04-26T23:44:44.607665+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:53.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616841+0000","last_change":"2024-04-26T23:44:41.847401+0000","last_active":"2024-04-26T23:44:44.616841+0000","last_peered":"2024-04-26T23:44:44.616841+0000","last_clean":"2024-04-26T23:44:44.616841+0000","last_became_active":"2024-04-26T23:44:41.847059+0000","last_became_peered":"2024-04-26T23:44:41.847059+0000","last_unstale":"2024-04-26T23:44:44.616841+0000","last_undegraded":"2024-04-26T23:44:44.616841+0000","last_fullsized":"2024-04-26T23:44:44.616841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:32.127690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863765+0000","last_change":"2024-04-26T23:44:42.857294+0000","last_active":"2024-04-26T23:44:43.863765+0000","last_peered":"2024-04-26T23:44:43.863765+0000","last_clean":"2024-04-26T23:44:43.863765+0000","last_became_active":"2024-04-26T23:44:42.856969+0000","last_became_peered":"2024-04-26T23:44:42.856969+0000","last_unstale":"2024-04-26T23:44:43.863765+0000","last_undegraded":"2024-04-26T23:44:43.863765+0000","last_fullsized":"2024-04-26T23:44:43.863765+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:31.593390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863567+0000","last_change":"2024-04-26T23:44:41.836981+0000","last_active":"2024-04-26T23:44:43.863567+0000","last_peered":"2024-04-26T23:44:43.863567+0000","last_clean":"2024-04-26T23:44:43.863567+0000","last_became_active":"2024-04-26T23:44:41.836100+0000","last_became_peered":"2024-04-26T23:44:41.836100+0000","last_unstale":"2024-04-26T23:44:43.863567+0000","last_undegraded":"2024-04-26T23:44:43.863567+0000","last_fullsized":"2024-04-26T23:44:43.863567+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:05.493175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860202+0000","last_change":"2024-04-26T23:44:42.881995+0000","last_active":"2024-04-26T23:44:43.860202+0000","last_peered":"2024-04-26T23:44:43.860202+0000","last_clean":"2024-04-26T23:44:43.860202+0000","last_became_active":"2024-04-26T23:44:42.881622+0000","last_became_peered":"2024-04-26T23:44:42.881622+0000","last_unstale":"2024-04-26T23:44:43.860202+0000","last_undegraded":"2024-04-26T23:44:43.860202+0000","last_fullsized":"2024-04-26T23:44:43.860202+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:52.347119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316266+0000","last_change":"2024-04-26T23:44:41.837673+0000","last_active":"2024-04-26T23:44:44.316266+0000","last_peered":"2024-04-26T23:44:44.316266+0000","last_clean":"2024-04-26T23:44:44.316266+0000","last_became_active":"2024-04-26T23:44:41.837377+0000","last_became_peered":"2024-04-26T23:44:41.837377+0000","last_unstale":"2024-04-26T23:44:44.316266+0000","last_undegraded":"2024-04-26T23:44:44.316266+0000","last_fullsized":"2024-04-26T23:44:44.316266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:12.145152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316992+0000","last_change":"2024-04-26T23:44:42.852172+0000","last_active":"2024-04-26T23:44:44.316992+0000","last_peered":"2024-04-26T23:44:44.316992+0000","last_clean":"2024-04-26T23:44:44.316992+0000","last_became_active":"2024-04-26T23:44:42.851791+0000","last_became_peered":"2024-04-26T23:44:42.851791+0000","last_unstale":"2024-04-26T23:44:44.316992+0000","last_undegraded":"2024-04-26T23:44:44.316992+0000","last_fullsized":"2024-04-26T23:44:44.316992+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:04.918901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789935+0000","last_change":"2024-04-26T23:44:41.833443+0000","last_active":"2024-04-26T23:44:44.789935+0000","last_peered":"2024-04-26T23:44:44.789935+0000","last_clean":"2024-04-26T23:44:44.789935+0000","last_became_active":"2024-04-26T23:44:41.833147+0000","last_became_peered":"2024-04-26T23:44:41.833147+0000","last_unstale":"2024-04-26T23:44:44.789935+0000","last_undegraded":"2024-04-26T23:44:44.789935+0000","last_fullsized":"2024-04-26T23:44:44.789935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:12.969965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316519+0000","last_change":"2024-04-26T23:44:42.869187+0000","last_active":"2024-04-26T23:44:44.316519+0000","last_peered":"2024-04-26T23:44:44.316519+0000","last_clean":"2024-04-26T23:44:44.316519+0000","last_became_active":"2024-04-26T23:44:42.868830+0000","last_became_peered":"2024-04-26T23:44:42.868830+0000","last_unstale":"2024-04-26T23:44:44.316519+0000","last_undegraded":"2024-04-26T23:44:44.316519+0000","last_fullsized":"2024-04-26T23:44:44.316519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:34.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316891+0000","last_change":"2024-04-26T23:44:41.838044+0000","last_active":"2024-04-26T23:44:44.316891+0000","last_peered":"2024-04-26T23:44:44.316891+0000","last_clean":"2024-04-26T23:44:44.316891+0000","last_became_active":"2024-04-26T23:44:41.837752+0000","last_became_peered":"2024-04-26T23:44:41.837752+0000","last_unstale":"2024-04-26T23:44:44.316891+0000","last_undegraded":"2024-04-26T23:44:44.316891+0000","last_fullsized":"2024-04-26T23:44:44.316891+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:48:30.579357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861873+0000","last_change":"2024-04-26T23:44:42.858527+0000","last_active":"2024-04-26T23:44:43.861873+0000","last_peered":"2024-04-26T23:44:43.861873+0000","last_clean":"2024-04-26T23:44:43.861873+0000","last_became_active":"2024-04-26T23:44:42.858137+0000","last_became_peered":"2024-04-26T23:44:42.858137+0000","last_unstale":"2024-04-26T23:44:43.861873+0000","last_undegraded":"2024-04-26T23:44:43.861873+0000","last_fullsized":"2024-04-26T23:44:43.861873+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:22.542824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789362+0000","last_change":"2024-04-26T23:44:41.845605+0000","last_active":"2024-04-26T23:44:44.789362+0000","last_peered":"2024-04-26T23:44:44.789362+0000","last_clean":"2024-04-26T23:44:44.789362+0000","last_became_active":"2024-04-26T23:44:41.845281+0000","last_became_peered":"2024-04-26T23:44:41.845281+0000","last_unstale":"2024-04-26T23:44:44.789362+0000","last_undegraded":"2024-04-26T23:44:44.789362+0000","last_fullsized":"2024-04-26T23:44:44.789362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:17.497909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607410+0000","last_change":"2024-04-26T23:44:42.871307+0000","last_active":"2024-04-26T23:44:44.607410+0000","last_peered":"2024-04-26T23:44:44.607410+0000","last_clean":"2024-04-26T23:44:44.607410+0000","last_became_active":"2024-04-26T23:44:42.870755+0000","last_became_peered":"2024-04-26T23:44:42.870755+0000","last_unstale":"2024-04-26T23:44:44.607410+0000","last_undegraded":"2024-04-26T23:44:44.607410+0000","last_fullsized":"2024-04-26T23:44:44.607410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:07.089428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617180+0000","last_change":"2024-04-26T23:44:41.846189+0000","last_active":"2024-04-26T23:44:44.617180+0000","last_peered":"2024-04-26T23:44:44.617180+0000","last_clean":"2024-04-26T23:44:44.617180+0000","last_became_active":"2024-04-26T23:44:41.845926+0000","last_became_peered":"2024-04-26T23:44:41.845926+0000","last_unstale":"2024-04-26T23:44:44.617180+0000","last_undegraded":"2024-04-26T23:44:44.617180+0000","last_fullsized":"2024-04-26T23:44:44.617180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:33.478880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861850+0000","last_change":"2024-04-26T23:44:42.867595+0000","last_active":"2024-04-26T23:44:43.861850+0000","last_peered":"2024-04-26T23:44:43.861850+0000","last_clean":"2024-04-26T23:44:43.861850+0000","last_became_active":"2024-04-26T23:44:42.859827+0000","last_became_peered":"2024-04-26T23:44:42.859827+0000","last_unstale":"2024-04-26T23:44:43.861850+0000","last_undegraded":"2024-04-26T23:44:43.861850+0000","last_fullsized":"2024-04-26T23:44:43.861850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:53.492153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863884+0000","last_change":"2024-04-26T23:44:41.843679+0000","last_active":"2024-04-26T23:44:43.863884+0000","last_peered":"2024-04-26T23:44:43.863884+0000","last_clean":"2024-04-26T23:44:43.863884+0000","last_became_active":"2024-04-26T23:44:41.843157+0000","last_became_peered":"2024-04-26T23:44:41.843157+0000","last_unstale":"2024-04-26T23:44:43.863884+0000","last_undegraded":"2024-04-26T23:44:43.863884+0000","last_fullsized":"2024-04-26T23:44:43.863884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:27.409179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861851+0000","last_change":"2024-04-26T23:44:42.866472+0000","last_active":"2024-04-26T23:44:43.861851+0000","last_peered":"2024-04-26T23:44:43.861851+0000","last_clean":"2024-04-26T23:44:43.861851+0000","last_became_active":"2024-04-26T23:44:42.865740+0000","last_became_peered":"2024-04-26T23:44:42.865740+0000","last_unstale":"2024-04-26T23:44:43.861851+0000","last_undegraded":"2024-04-26T23:44:43.861851+0000","last_fullsized":"2024-04-26T23:44:43.861851+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.634829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863652+0000","last_change":"2024-04-26T23:44:41.836746+0000","last_active":"2024-04-26T23:44:43.863652+0000","last_peered":"2024-04-26T23:44:43.863652+0000","last_clean":"2024-04-26T23:44:43.863652+0000","last_became_active":"2024-04-26T23:44:41.835371+0000","last_became_peered":"2024-04-26T23:44:41.835371+0000","last_unstale":"2024-04-26T23:44:43.863652+0000","last_undegraded":"2024-04-26T23:44:43.863652+0000","last_fullsized":"2024-04-26T23:44:43.863652+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:23.924236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.317432+0000","last_change":"2024-04-26T23:44:42.862002+0000","last_active":"2024-04-26T23:44:44.317432+0000","last_peered":"2024-04-26T23:44:44.317432+0000","last_clean":"2024-04-26T23:44:44.317432+0000","last_became_active":"2024-04-26T23:44:42.861727+0000","last_became_peered":"2024-04-26T23:44:42.861727+0000","last_unstale":"2024-04-26T23:44:44.317432+0000","last_undegraded":"2024-04-26T23:44:44.317432+0000","last_fullsized":"2024-04-26T23:44:44.317432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:42.558949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607582+0000","last_change":"2024-04-26T23:44:41.840373+0000","last_active":"2024-04-26T23:44:44.607582+0000","last_peered":"2024-04-26T23:44:44.607582+0000","last_clean":"2024-04-26T23:44:44.607582+0000","last_became_active":"2024-04-26T23:44:41.840079+0000","last_became_peered":"2024-04-26T23:44:41.840079+0000","last_unstale":"2024-04-26T23:44:44.607582+0000","last_undegraded":"2024-04-26T23:44:44.607582+0000","last_fullsized":"2024-04-26T23:44:44.607582+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:56.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.789589+0000","last_change":"2024-04-26T23:44:42.881502+0000","last_active":"2024-04-26T23:44:44.789589+0000","last_peered":"2024-04-26T23:44:44.789589+0000","last_clean":"2024-04-26T23:44:44.789589+0000","last_became_active":"2024-04-26T23:44:42.881040+0000","last_became_peered":"2024-04-26T23:44:42.881040+0000","last_unstale":"2024-04-26T23:44:44.789589+0000","last_undegraded":"2024-04-26T23:44:44.789589+0000","last_fullsized":"2024-04-26T23:44:44.789589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:51.058728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.861157+0000","last_change":"2024-04-26T23:44:41.836712+0000","last_active":"2024-04-26T23:44:43.861157+0000","last_peered":"2024-04-26T23:44:43.861157+0000","last_clean":"2024-04-26T23:44:43.861157+0000","last_became_active":"2024-04-26T23:44:41.836329+0000","last_became_peered":"2024-04-26T23:44:41.836329+0000","last_unstale":"2024-04-26T23:44:43.861157+0000","last_undegraded":"2024-04-26T23:44:43.861157+0000","last_fullsized":"2024-04-26T23:44:43.861157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:02.069303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.308877+0000","last_change":"2024-04-26T23:44:42.869332+0000","last_active":"2024-04-26T23:44:44.308877+0000","last_peered":"2024-04-26T23:44:44.308877+0000","last_clean":"2024-04-26T23:44:44.308877+0000","last_became_active":"2024-04-26T23:44:42.868837+0000","last_became_peered":"2024-04-26T23:44:42.868837+0000","last_unstale":"2024-04-26T23:44:44.308877+0000","last_undegraded":"2024-04-26T23:44:44.308877+0000","last_fullsized":"2024-04-26T23:44:44.308877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.141723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.316465+0000","last_change":"2024-04-26T23:44:41.842867+0000","last_active":"2024-04-26T23:44:44.316465+0000","last_peered":"2024-04-26T23:44:44.316465+0000","last_clean":"2024-04-26T23:44:44.316465+0000","last_became_active":"2024-04-26T23:44:41.842529+0000","last_became_peered":"2024-04-26T23:44:41.842529+0000","last_unstale":"2024-04-26T23:44:44.316465+0000","last_undegraded":"2024-04-26T23:44:44.316465+0000","last_fullsized":"2024-04-26T23:44:44.316465+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:40.817666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607200+0000","last_change":"2024-04-26T23:44:42.873462+0000","last_active":"2024-04-26T23:44:44.607200+0000","last_peered":"2024-04-26T23:44:44.607200+0000","last_clean":"2024-04-26T23:44:44.607200+0000","last_became_active":"2024-04-26T23:44:42.873095+0000","last_became_peered":"2024-04-26T23:44:42.873095+0000","last_unstale":"2024-04-26T23:44:44.607200+0000","last_undegraded":"2024-04-26T23:44:44.607200+0000","last_fullsized":"2024-04-26T23:44:44.607200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:13.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.607566+0000","last_change":"2024-04-26T23:44:41.839198+0000","last_active":"2024-04-26T23:44:44.607566+0000","last_peered":"2024-04-26T23:44:44.607566+0000","last_clean":"2024-04-26T23:44:44.607566+0000","last_became_active":"2024-04-26T23:44:41.838683+0000","last_became_peered":"2024-04-26T23:44:41.838683+0000","last_unstale":"2024-04-26T23:44:44.607566+0000","last_undegraded":"2024-04-26T23:44:44.607566+0000","last_fullsized":"2024-04-26T23:44:44.607566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:37.624561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.860850+0000","last_change":"2024-04-26T23:44:42.856089+0000","last_active":"2024-04-26T23:44:43.860850+0000","last_peered":"2024-04-26T23:44:43.860850+0000","last_clean":"2024-04-26T23:44:43.860850+0000","last_became_active":"2024-04-26T23:44:42.855682+0000","last_became_peered":"2024-04-26T23:44:42.855682+0000","last_unstale":"2024-04-26T23:44:43.860850+0000","last_undegraded":"2024-04-26T23:44:43.860850+0000","last_fullsized":"2024-04-26T23:44:43.860850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:42.144285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.616852+0000","last_change":"2024-04-26T23:44:41.838757+0000","last_active":"2024-04-26T23:44:44.616852+0000","last_peered":"2024-04-26T23:44:44.616852+0000","last_clean":"2024-04-26T23:44:44.616852+0000","last_became_active":"2024-04-26T23:44:41.838423+0000","last_became_peered":"2024-04-26T23:44:41.838423+0000","last_unstale":"2024-04-26T23:44:44.616852+0000","last_undegraded":"2024-04-26T23:44:44.616852+0000","last_fullsized":"2024-04-26T23:44:44.616852+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:39:10.975944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617345+0000","last_change":"2024-04-26T23:44:42.871295+0000","last_active":"2024-04-26T23:44:44.617345+0000","last_peered":"2024-04-26T23:44:44.617345+0000","last_clean":"2024-04-26T23:44:44.617345+0000","last_became_active":"2024-04-26T23:44:42.870709+0000","last_became_peered":"2024-04-26T23:44:42.870709+0000","last_unstale":"2024-04-26T23:44:44.617345+0000","last_undegraded":"2024-04-26T23:44:44.617345+0000","last_fullsized":"2024-04-26T23:44:44.617345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:41.796244+0000","last_clean_scrub_stamp":"2024-04-26T23:44:41.796244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:16.266265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:43.863952+0000","last_change":"2024-04-26T23:44:41.843191+0000","last_active":"2024-04-26T23:44:43.863952+0000","last_peered":"2024-04-26T23:44:43.863952+0000","last_clean":"2024-04-26T23:44:43.863952+0000","last_became_active":"2024-04-26T23:44:41.842727+0000","last_became_peered":"2024-04-26T23:44:41.842727+0000","last_unstale":"2024-04-26T23:44:43.863952+0000","last_undegraded":"2024-04-26T23:44:43.863952+0000","last_fullsized":"2024-04-26T23:44:43.863952+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:44:40.782211+0000","last_clean_scrub_stamp":"2024-04-26T23:44:40.782211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:18.279295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":305,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T23:44:44.617196+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:44:44.617196+0000","last_peered":"2024-04-26T23:44:44.617196+0000","last_clean":"2024-04-26T23:44:44.617196+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:44:44.617196+0000","last_undegraded":"2024-04-26T23:44:44.617196+0000","last_fullsized":"2024-04-26T23:44:44.617196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1773568,"data_stored":1133550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40664,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331176,"statfs":{"total":96636764160,"available":96595124224,"internally_reserved":0,"allocated":1191936,"data_stored":545342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40660,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331180,"statfs":{"total":96636764160,"available":96595128320,"internally_reserved":0,"allocated":1179648,"data_stored":543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40644,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331196,"statfs":{"total":96636764160,"available":96595144704,"internally_reserved":0,"allocated":1171456,"data_stored":543126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1175552,"data_stored":543196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41256,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1789952,"data_stored":1135812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:44:47.840 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:44:47.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:44:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.098+0000 7fb2e9c00640 1 Processor -- start 2024-04-26T23:44:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.098+0000 7fb2e9c00640 1 -- start start 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2e406f8f0 0x7fb2e406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e406c890 0x7fb2e4162c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e4163160 0x7fb2e40b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e41424a0 con 0x7fb2e406c890 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e415fed0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e415f5c0 con 0x7fb2e4163160 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e406c890 0x7fb2e4162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e406c890 0x7fb2e4162c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60184/0 (socket says 172.21.15.46:60184) 2024-04-26T23:44:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e3400640 1 -- 172.21.15.46:0/4045779905 learned_addr learned my addr 172.21.15.46:0/4045779905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.101+0000 7fb2e3e00640 1 --2- 172.21.15.46:0/4045779905 >> v2:172.21.15.142:3300/0 conn(0x7fb2e406f8f0 0x7fb2e406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e9200640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e4163160 0x7fb2e40b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e3400640 1 -- 172.21.15.46:0/4045779905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e4163160 msgr2=0x7fb2e40b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e3400640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e4163160 0x7fb2e40b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e3400640 1 -- 172.21.15.46:0/4045779905 >> v2:172.21.15.142:3300/0 conn(0x7fb2e406f8f0 msgr2=0x7fb2e406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e3400640 1 --2- 172.21.15.46:0/4045779905 >> v2:172.21.15.142:3300/0 conn(0x7fb2e406f8f0 0x7fb2e406c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.102+0000 7fb2e3400640 1 -- 172.21.15.46:0/4045779905 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e4163990 con 0x7fb2e406c890 2024-04-26T23:44:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.103+0000 7fb2e3400640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406c890 0x7fb2e4162c20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406e9f0 tx=0x7fb2d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1db49aca0452ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.103+0000 7fb2e2a00640 1 -- 172.21.15.46:0/4045779905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0037070 con 0x7fb2e406c890 2024-04-26T23:44:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.103+0000 7fb2e2a00640 1 -- 172.21.15.46:0/4045779905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0032440 con 0x7fb2e406c890 2024-04-26T23:44:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.103+0000 7fb2e2a00640 1 -- 172.21.15.46:0/4045779905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0032810 con 0x7fb2e406c890 2024-04-26T23:44:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 -- 172.21.15.46:0/4045779905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406c890 msgr2=0x7fb2e4162c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406c890 0x7fb2e4162c20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406e9f0 tx=0x7fb2d002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 -- 172.21.15.46:0/4045779905 shutdown_connections 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e4163160 0x7fb2e40b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/4045779905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406c890 0x7fb2e4162c20 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/4045779905 >> v2:172.21.15.142:3300/0 conn(0x7fb2e406f8f0 0x7fb2e406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 -- 172.21.15.46:0/4045779905 >> 172.21.15.46:0/4045779905 conn(0x7fb2e40b3930 msgr2=0x7fb2e40b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 -- 172.21.15.46:0/4045779905 shutdown_connections 2024-04-26T23:44:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.104+0000 7fb2e9c00640 1 -- 172.21.15.46:0/4045779905 wait complete. 2024-04-26T23:44:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.106+0000 7fb2e9c00640 1 Processor -- start 2024-04-26T23:44:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.106+0000 7fb2e9c00640 1 -- start start 2024-04-26T23:44:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 0x7fb2e40aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2e4163160 0x7fb2e4141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41635f0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e415fb60 con 0x7fb2e4163160 2024-04-26T23:44:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.107+0000 7fb2e9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e4163770 con 0x7fb2e406c890 2024-04-26T23:44:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60198/0 (socket says 172.21.15.46:60198) 2024-04-26T23:44:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 -- 172.21.15.46:0/357107804 learned_addr learned my addr 172.21.15.46:0/357107804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e9200640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2e4163160 0x7fb2e4141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3e00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 0x7fb2e40aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 -- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 msgr2=0x7fb2e40aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 0x7fb2e40aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 -- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2e4163160 msgr2=0x7fb2e4141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2e4163160 0x7fb2e4141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3400640 1 -- 172.21.15.46:0/357107804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e4151f60 con 0x7fb2e406f8f0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.108+0000 7fb2e3e00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 0x7fb2e40aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2e3400640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb2d002a220 tx=0x7fb2d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0041020 con 0x7fb2e406f8f0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d00306c0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0037040 con 0x7fb2e406f8f0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e4153260 con 0x7fb2e406f8f0 2024-04-26T23:44:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.109+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41537e0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.110+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e4069ac0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.111+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2d0044020 con 0x7fb2e406f8f0 2024-04-26T23:44:48.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.112+0000 7fb2dbe00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 0x7fb2c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.112+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fb2d00ac570 con 0x7fb2e406f8f0 2024-04-26T23:44:48.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.113+0000 7fb2e3e00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 0x7fb2c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.114+0000 7fb2e3e00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 0x7fb2c406edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4006fd0 tx=0x7fb2d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.116+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2d004b030 con 0x7fb2e406f8f0 2024-04-26T23:44:48.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.406+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb2e4162dc0 con 0x7fb2e406f8f0 2024-04-26T23:44:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.411+0000 7fb2dbe00640 1 -- 172.21.15.46:0/357107804 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fb2d007a800 con 0x7fb2e406f8f0 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:44:48.415 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:44:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 msgr2=0x7fb2c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 0x7fb2c406edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4006fd0 tx=0x7fb2d4008040 comp rx=0 tx=0).stop 2024-04-26T23:44:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 msgr2=0x7fb2e4141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb2d002a220 tx=0x7fb2d00046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 shutdown_connections 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2c406c8f0 0x7fb2c406edb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2e4163160 0x7fb2e4141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e406f8f0 0x7fb2e4141590 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 --2- 172.21.15.46:0/357107804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2e406c890 0x7fb2e40aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 >> 172.21.15.46:0/357107804 conn(0x7fb2e40b3930 msgr2=0x7fb2e4155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 shutdown_connections 2024-04-26T23:44:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.415+0000 7fb2e9c00640 1 -- 172.21.15.46:0/357107804 wait complete. 2024-04-26T23:44:48.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.647+0000 7f7426a00640 1 Processor -- start 2024-04-26T23:44:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.647+0000 7f7426a00640 1 -- start start 2024-04-26T23:44:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74200720e0 0x7f74200cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7420073ad0 con 0x7f7420160910 2024-04-26T23:44:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7420146a90 con 0x7f74201466b0 2024-04-26T23:44:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7420073d70 con 0x7f74200720e0 2024-04-26T23:44:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7420160910 0x7f74200cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55076/0 (socket says 172.21.15.46:55076) 2024-04-26T23:44:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7426000640 1 -- 172.21.15.46:0/3800060755 learned_addr learned my addr 172.21.15.46:0/3800060755 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.649+0000 7f7424c00640 1 --2- 172.21.15.46:0/3800060755 >> v2:172.21.15.46:3300/0 conn(0x7f74200720e0 0x7f74200cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.650+0000 7f741fe00640 1 -- 172.21.15.46:0/3800060755 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069429781 0 0) 0x7f7420073ad0 con 0x7f7420160910 2024-04-26T23:44:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.650+0000 7f741fe00640 1 -- 172.21.15.46:0/3800060755 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7408003520 con 0x7f7420160910 2024-04-26T23:44:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.650+0000 7f7424c00640 1 -- 172.21.15.46:0/3800060755 >> v1:172.21.15.142:6789/0 conn(0x7f74201466b0 legacy=0x7f742006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.650+0000 7f7424c00640 1 -- 172.21.15.46:0/3800060755 >> v1:172.21.15.46:6789/0 conn(0x7f7420160910 legacy=0x7f74200cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.650+0000 7f7424c00640 1 -- 172.21.15.46:0/3800060755 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7420074050 con 0x7f74200720e0 2024-04-26T23:44:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.651+0000 7f7424c00640 1 --2- 172.21.15.46:0/3800060755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74200720e0 0x7f74200cd860 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097d0 tx=0x7f740c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0290f63ceedad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.651+0000 7f741fe00640 1 -- 172.21.15.46:0/3800060755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c037070 con 0x7f74200720e0 2024-04-26T23:44:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.651+0000 7f741fe00640 1 -- 172.21.15.46:0/3800060755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c032440 con 0x7f74200720e0 2024-04-26T23:44:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.651+0000 7f741fe00640 1 -- 172.21.15.46:0/3800060755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0327f0 con 0x7f74200720e0 2024-04-26T23:44:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 -- 172.21.15.46:0/3800060755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74200720e0 msgr2=0x7f74200cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 --2- 172.21.15.46:0/3800060755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74200720e0 0x7f74200cd860 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097d0 tx=0x7f740c02a270 comp rx=0 tx=0).stop 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 -- 172.21.15.46:0/3800060755 shutdown_connections 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 --2- 172.21.15.46:0/3800060755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74200720e0 0x7f74200cd860 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 -- 172.21.15.46:0/3800060755 >> 172.21.15.46:0/3800060755 conn(0x7f74200b26d0 msgr2=0x7f74200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 -- 172.21.15.46:0/3800060755 shutdown_connections 2024-04-26T23:44:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.652+0000 7f7426a00640 1 -- 172.21.15.46:0/3800060755 wait complete. 2024-04-26T23:44:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.654+0000 7f7426a00640 1 Processor -- start 2024-04-26T23:44:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.654+0000 7f7426a00640 1 -- start start 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.655+0000 7f7426a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74200720e0 0x7f7420141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.655+0000 7f7426a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 0x7f74200b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74200cdda0 con 0x7f7420160910 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74200cb470 con 0x7f74201466b0 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74200cc0a0 con 0x7f74200720e0 2024-04-26T23:44:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60212/0 (socket says 172.21.15.46:60212) 2024-04-26T23:44:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 -- 172.21.15.46:0/1807000778 learned_addr learned my addr 172.21.15.46:0/1807000778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7424c00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 0x7f74200b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 -- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74200720e0 msgr2=0x7f7420141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74200720e0 0x7f7420141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 -- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 msgr2=0x7f74200b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 0x7f74200b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7426000640 1 -- 172.21.15.46:0/1807000778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74200c8930 con 0x7f7420160910 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.656+0000 7f7424c00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 0x7f74200b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f7426000640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f741800cad0 tx=0x7f741800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418017070 con 0x7f7420160910 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741800ac30 con 0x7f7420160910 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418012400 con 0x7f7420160910 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74200c8c10 con 0x7f7420160910 2024-04-26T23:44:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.657+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74200c91d0 con 0x7f7420160910 2024-04-26T23:44:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.658+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74200693b0 con 0x7f7420160910 2024-04-26T23:44:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.659+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f741801a070 con 0x7f7420160910 2024-04-26T23:44:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.660+0000 7f741d600640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 0x7f740006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.660+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f741808bb10 con 0x7f7420160910 2024-04-26T23:44:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.661+0000 7f7425600640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 0x7f740006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.662+0000 7f7425600640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 0x7f740006ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7414000f60 tx=0x7f74140079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.665+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7418059d10 con 0x7f7420160910 2024-04-26T23:44:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.919+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74200cd3f0 con 0x7f7420160910 2024-04-26T23:44:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.920+0000 7f741d600640 1 -- 172.21.15.46:0/1807000778 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7f7418059610 con 0x7f7420160910 2024-04-26T23:44:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:48.924 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/932747087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":932747087},{"type":"v1","addr":"172.21.15.142:6835","nonce":932747087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2454420217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2454420217},{"type":"v1","addr":"172.21.15.142:6837","nonce":2454420217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3409172203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3409172203},{"type":"v1","addr":"172.21.15.46:6834","nonce":3409172203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:44:45.428792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7695":{"gid":7695,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2670265715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2670265715},{"type":"v1","addr":"172.21.15.46:6835","nonce":2670265715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7695,"qdb_cluster":[7695]},"id":6}]} 2024-04-26T23:44:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 74 2024-04-26T23:44:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 msgr2=0x7f740006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 0x7f740006ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7414000f60 tx=0x7f74140079b0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 msgr2=0x7f74200b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f741800cad0 tx=0x7f741800cf90 comp rx=0 tx=0).stop 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 shutdown_connections 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f740006c920 0x7f740006ede0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420160910 0x7f74200b0b20 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74201466b0 0x7f74200b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 --2- 172.21.15.46:0/1807000778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74200720e0 0x7f7420141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 >> 172.21.15.46:0/1807000778 conn(0x7f74200b26d0 msgr2=0x7f742006f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 shutdown_connections 2024-04-26T23:44:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:48.924+0000 7f7426a00640 1 -- 172.21.15.46:0/1807000778 wait complete. 2024-04-26T23:44:49.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:44:42.305207+0000', 'modified': '2024-04-26T23:44:45.428792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7695}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7695': {'gid': 7695, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/2670265715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2670265715}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2670265715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7695, 'qdb_cluster': [7695]} 2024-04-26T23:44:49.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:44:49.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:44:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.574+0000 7f83d7200640 1 Processor -- start 2024-04-26T23:44:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.574+0000 7f83d7200640 1 -- start start 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.576+0000 7f83d7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83d80720e0 0x7f83d80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.576+0000 7f83d7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83d8160910 0x7f83d8160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.576+0000 7f83d7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d8073ad0 con 0x7f83d81466b0 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.576+0000 7f83d7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f83d81612a0 con 0x7f83d8160910 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.576+0000 7f83d7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f83d8146a90 con 0x7f83d80720e0 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83d8160910 0x7f83d8160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83d8160910 0x7f83d8160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60214/0 (socket says 172.21.15.46:60214) 2024-04-26T23:44:49.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 -- 172.21.15.46:0/3511035200 learned_addr learned my addr 172.21.15.46:0/3511035200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83cfe00640 1 --2- 172.21.15.46:0/3511035200 >> v2:172.21.15.142:3300/0 conn(0x7f83d80720e0 0x7f83d80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 -- 172.21.15.46:0/3511035200 >> v2:172.21.15.142:3300/0 conn(0x7f83d80720e0 msgr2=0x7f83d80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 --2- 172.21.15.46:0/3511035200 >> v2:172.21.15.142:3300/0 conn(0x7f83d80720e0 0x7f83d80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 -- 172.21.15.46:0/3511035200 >> v1:172.21.15.142:6789/0 conn(0x7f83d81466b0 legacy=0x7f83d806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.577+0000 7f83d6800640 1 -- 172.21.15.46:0/3511035200 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d8073d70 con 0x7f83d8160910 2024-04-26T23:44:49.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.578+0000 7f83d6800640 1 --2- 172.21.15.46:0/3511035200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d8160910 0x7f83d8160d60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009e50 tx=0x7f83d002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c8617cabf6872 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d5400640 1 -- 172.21.15.46:0/3511035200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0036070 con 0x7f83d8160910 2024-04-26T23:44:49.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d5400640 1 -- 172.21.15.46:0/3511035200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0031600 con 0x7f83d8160910 2024-04-26T23:44:49.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d5400640 1 -- 172.21.15.46:0/3511035200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00319d0 con 0x7f83d8160910 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 -- 172.21.15.46:0/3511035200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d8160910 msgr2=0x7f83d8160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 --2- 172.21.15.46:0/3511035200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d8160910 0x7f83d8160d60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009e50 tx=0x7f83d002a710 comp rx=0 tx=0).stop 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 -- 172.21.15.46:0/3511035200 shutdown_connections 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 --2- 172.21.15.46:0/3511035200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d8160910 0x7f83d8160d60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 --2- 172.21.15.46:0/3511035200 >> v2:172.21.15.142:3300/0 conn(0x7f83d80720e0 0x7f83d80cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.579+0000 7f83d7200640 1 -- 172.21.15.46:0/3511035200 >> 172.21.15.46:0/3511035200 conn(0x7f83d80b26d0 msgr2=0x7f83d80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.580+0000 7f83d7200640 1 -- 172.21.15.46:0/3511035200 shutdown_connections 2024-04-26T23:44:49.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.580+0000 7f83d7200640 1 -- 172.21.15.46:0/3511035200 wait complete. 2024-04-26T23:44:49.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.581+0000 7f83d7200640 1 Processor -- start 2024-04-26T23:44:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.581+0000 7f83d7200640 1 -- start start 2024-04-26T23:44:49.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83d81466b0 0x7f83d81e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83d8160910 0x7f83d81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80cb470 con 0x7f83d80720e0 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80cb190 con 0x7f83d8160910 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.582+0000 7f83d7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83d80cdda0 con 0x7f83d81466b0 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60226/0 (socket says 172.21.15.46:60226) 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 -- 172.21.15.46:0/2625179325 learned_addr learned my addr 172.21.15.46:0/2625179325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 -- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83d81466b0 msgr2=0x7f83d81e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83d81466b0 0x7f83d81e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 -- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83d8160910 msgr2=0x7f83d81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83d8160910 0x7f83d81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 -- 172.21.15.46:0/2625179325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d80ce7a0 con 0x7f83d80720e0 2024-04-26T23:44:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.583+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b420 tx=0x7f83c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.584+0000 7f83ce000640 1 -- 172.21.15.46:0/2625179325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011830 con 0x7f83d80720e0 2024-04-26T23:44:49.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.584+0000 7f83ce000640 1 -- 172.21.15.46:0/2625179325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c8016070 con 0x7f83d80720e0 2024-04-26T23:44:49.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.584+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d80cea80 con 0x7f83d80720e0 2024-04-26T23:44:49.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.584+0000 7f83ce000640 1 -- 172.21.15.46:0/2625179325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011c10 con 0x7f83d80720e0 2024-04-26T23:44:49.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.584+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d80cf040 con 0x7f83d80720e0 2024-04-26T23:44:49.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.586+0000 7f83ce000640 1 -- 172.21.15.46:0/2625179325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83c801b460 con 0x7f83d80720e0 2024-04-26T23:44:49.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.587+0000 7f83ce000640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 0x7f83b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.587+0000 7f83ce000640 1 -- 172.21.15.46:0/2625179325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f83c808b820 con 0x7f83d80720e0 2024-04-26T23:44:49.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.588+0000 7f83dfc4a740 1 Processor -- start 2024-04-26T23:44:49.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.588+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 0x7f83b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.588+0000 7f83dfc4a740 1 -- start start 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252ab3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ab1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e252a15600 con 0x55e252adc270 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e2529b2100 con 0x55e252ab17b0 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e252a115c0 con 0x55e252aede80 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60242/0 (socket says 172.21.15.46:60242) 2024-04-26T23:44:49.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 -- 172.21.15.46:0/2514371188 learned_addr learned my addr 172.21.15.46:0/2514371188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 0x7f83b406edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f83c0006fd0 tx=0x7f83c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252ab3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ab1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 -- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 msgr2=0x55e252ab1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ab1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 -- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 msgr2=0x55e252ab3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252ab3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d6800640 1 -- 172.21.15.46:0/2514371188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2527272d0 con 0x55e252adc270 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.589+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252ab3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.590+0000 7f83d6800640 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f83d0034040 tx=0x7f83d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d790e495768fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.590+0000 7f83c6a00640 1 -- 172.21.15.46:0/2514371188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0036050 con 0x55e252adc270 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.590+0000 7f83c6a00640 1 -- 172.21.15.46:0/2514371188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0042070 con 0x55e252adc270 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.590+0000 7f83c6a00640 1 -- 172.21.15.46:0/2514371188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d003d8b0 con 0x55e252adc270 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2514371188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 msgr2=0x55e252adc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f83d0034040 tx=0x7f83d002c000 comp rx=0 tx=0).stop 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2514371188 shutdown_connections 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252adc6f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ab1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2514371188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252ab3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2514371188 >> 172.21.15.46:0/2514371188 conn(0x55e252a45d10 msgr2=0x55e252af7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2514371188 shutdown_connections 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.591+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2514371188 wait complete. 2024-04-26T23:44:49.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.592+0000 7f83dfc4a740 1 -- start start 2024-04-26T23:44:49.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.592+0000 7f83dfc4a740 1 Processor -- start 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252ab17b0 0x55e252afa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ad8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e25299cf70 con 0x55e252adc270 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e252ad9dc0 con 0x55e252aede80 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e252ac9a00 con 0x55e252ab17b0 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.593+0000 7f83dfc4a740 10 client.0 _ll_register_callbacks cb 0x55e252a0b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60256/0 (socket says 172.21.15.46:60256) 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 -- 172.21.15.46:0/2250936436 learned_addr learned my addr 172.21.15.46:0/2250936436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252ab17b0 0x55e252afa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 -- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252ab17b0 msgr2=0x55e252afa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252ab17b0 0x55e252afa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83d6800640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ad8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 -- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 msgr2=0x55e252ad8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ad8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 -- 172.21.15.46:0/2250936436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2527f16b0 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83cfe00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83d80b2630 tx=0x7f83c0006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444f97a3799bcc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.594+0000 7f83bb400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0014e20 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f83c0020460 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d0042070 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f83c0005990 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 10 client.7845 fetch_fsmap learned FSMap version 74 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55e252ac9740 con 0x55e252adc270 2024-04-26T23:44:49.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0042070 con 0x55e252adc270 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f83c0030080 con 0x55e252adc270 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 10 client.7845 fetch_fsmap finished waiting for FSMap version 74 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 10 client.7845 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 20 client.7845 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af8600 0x55e252af89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 4 client.7845 mds_command: new command op to 7695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.595+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 --> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e252a23210 con 0x55e252af8600 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.596+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af8600 0x55e252af89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.596+0000 7f83d5e00640 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af8600 0x55e252af89e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db935673ddd7cb40 server_cookie=f95bb7d30397ede8 in_seq=0 out_seq=0 2024-04-26T23:44:49.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.596+0000 7f83bb400640 10 client.7845 ms_handle_connect on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.597+0000 7f83bb400640 1 -- 172.21.15.46:0/2250936436 <== mds.0 v2:172.21.15.46:6833/2670265715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e252a23210 con 0x55e252af8600 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.597+0000 7f83bb400640 10 client.7845 handle_command_reply: tid=0 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.597+0000 7f83dfc4a740 1 client.7845 shutdown 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.597+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 msgr2=0x55e252ad8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.597+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83d80b2630 tx=0x7f83c0006e10 comp rx=0 tx=0).stop 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 shutdown_connections 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af8600 0x55e252af89e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ad8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad8a10 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/2250936436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252ab17b0 0x55e252afa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 >> 172.21.15.46:0/2250936436 conn(0x55e252a45d10 msgr2=0x55e252ada1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83bb400640 0 client.7845 ms_handle_reset on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 shutdown_connections 2024-04-26T23:44:49.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 1 -- 172.21.15.46:0/2250936436 wait complete. 2024-04-26T23:44:49.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.598+0000 7f83dfc4a740 20 client.7845 trim_cache size 0 max 16384 2024-04-26T23:44:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.618+0000 7f83dfc4a740 1 Processor -- start 2024-04-26T23:44:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.618+0000 7f83dfc4a740 1 -- start start 2024-04-26T23:44:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252b06b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ad6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8200d40 con 0x55e252adc270 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80d03d0 con 0x55e252ab17b0 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.619+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83d82006f0 con 0x55e252aede80 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60262/0 (socket says 172.21.15.46:60262) 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 -- 172.21.15.46:0/828596428 learned_addr learned my addr 172.21.15.46:0/828596428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83d5e00640 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252b06b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 -- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 msgr2=0x55e252ad6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ad6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 -- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 msgr2=0x55e252b06b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252b06b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 -- 172.21.15.46:0/828596428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e252aafb10 con 0x55e252adc270 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.620+0000 7f83cfe00640 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83c0014ce0 tx=0x7f83c001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b87d4ee775b4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83c6000640 1 -- 172.21.15.46:0/828596428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0014070 con 0x55e252adc270 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83c6000640 1 -- 172.21.15.46:0/828596428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0025070 con 0x55e252adc270 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83c6000640 1 -- 172.21.15.46:0/828596428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0020460 con 0x55e252adc270 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 -- 172.21.15.46:0/828596428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 msgr2=0x55e252ad6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83c0014ce0 tx=0x7f83c001d000 comp rx=0 tx=0).stop 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 -- 172.21.15.46:0/828596428 shutdown_connections 2024-04-26T23:44:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252aede80 0x55e252ad6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252adc270 0x55e252ad6250 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/828596428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252ab17b0 0x55e252b06b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.621+0000 7f83dfc4a740 1 -- 172.21.15.46:0/828596428 >> 172.21.15.46:0/828596428 conn(0x55e252a45d10 msgr2=0x55e252af8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.622+0000 7f83dfc4a740 1 -- 172.21.15.46:0/828596428 shutdown_connections 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.622+0000 7f83dfc4a740 1 -- 172.21.15.46:0/828596428 wait complete. 2024-04-26T23:44:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.622+0000 7f83dfc4a740 1 -- start start 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.622+0000 7f83dfc4a740 1 Processor -- start 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252adc270 0x55e252b709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ab0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e252ac8d00 con 0x55e252ab17b0 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e252ac9290 con 0x55e252aede80 2024-04-26T23:44:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.623+0000 7f83dfc4a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e252afac50 con 0x55e252adc270 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83dfc4a740 10 client.0 _ll_register_callbacks cb 0x7f83c000b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60278/0 (socket says 172.21.15.46:60278) 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 -- 172.21.15.46:0/939272869 learned_addr learned my addr 172.21.15.46:0/939272869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 -- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252adc270 msgr2=0x55e252b709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252adc270 0x55e252b709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 -- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 msgr2=0x55e252ab0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ab0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 -- 172.21.15.46:0/939272869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e252af8f40 con 0x55e252ab17b0 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83d5e00640 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83c8061a60 tx=0x7f83c8055780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f86f1e22aca3b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.624+0000 7f83bb400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8071030 con 0x55e252ab17b0 2024-04-26T23:44:49.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f83c8055a20 con 0x55e252ab17b0 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c0025070 con 0x55e252ab17b0 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f83c8061f90 con 0x55e252ab17b0 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 10 client.7851 fetch_fsmap learned FSMap version 74 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55e252b1a160 con 0x55e252ab17b0 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0025070 con 0x55e252ab17b0 2024-04-26T23:44:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f83c8071b90 con 0x55e252ab17b0 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 10 client.7851 fetch_fsmap finished waiting for FSMap version 74 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 10 client.7851 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 20 client.7851 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af9520 0x55e252af9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 4 client.7851 mds_command: new command op to 7695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.625+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 --> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] -- command(tid 0: {"prefix": "status"}) -- 0x55e252afb3b0 con 0x55e252af9520 2024-04-26T23:44:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.626+0000 7f83cfe00640 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af9520 0x55e252af9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.626+0000 7f83cfe00640 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af9520 0x55e252af9900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a137d56a2c5ba133 server_cookie=9672683e886c836e in_seq=0 out_seq=0 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.626+0000 7f83bb400640 10 client.7851 ms_handle_connect on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83bb400640 1 -- 172.21.15.46:0/939272869 <== mds.0 v2:172.21.15.46:6833/2670265715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e252afb3b0 con 0x55e252af9520 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83bb400640 10 client.7851 handle_command_reply: tid=0 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 client.7851 shutdown 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 msgr2=0x55e252b19b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83c8061a60 tx=0x7f83c8055780 comp rx=0 tx=0).stop 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 shutdown_connections 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x55e252af9520 0x55e252af9900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e252aede80 0x55e252ab0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e252adc270 0x55e252b709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 --2- 172.21.15.46:0/939272869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e252ab17b0 0x55e252b19b20 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83bb400640 0 client.7851 ms_handle_reset on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 >> 172.21.15.46:0/939272869 conn(0x55e252a45d10 msgr2=0x55e252ad8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 shutdown_connections 2024-04-26T23:44:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 1 -- 172.21.15.46:0/939272869 wait complete. 2024-04-26T23:44:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.627+0000 7f83dfc4a740 20 client.7851 trim_cache size 0 max 16384 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7695, 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:44:49.647 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7979089009999996, 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 74, 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 95, 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.227840928000001 2024-04-26T23:44:49.648 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 msgr2=0x7f83b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 0x7f83b406edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f83c0006fd0 tx=0x7f83c0008040 comp rx=0 tx=0).stop 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 msgr2=0x7f83d8141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b420 tx=0x7f83c800b8e0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 shutdown_connections 2024-04-26T23:44:49.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83b406c8f0 0x7f83b406edb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83d8160910 0x7f83d81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83d81466b0 0x7f83d81e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 --2- 172.21.15.46:0/2625179325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d80720e0 0x7f83d8141130 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.628+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 >> 172.21.15.46:0/2625179325 conn(0x7f83d80b26d0 msgr2=0x7f83d8142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.629+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 shutdown_connections 2024-04-26T23:44:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.629+0000 7f83d7200640 1 -- 172.21.15.46:0/2625179325 wait complete. 2024-04-26T23:44:49.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:44:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.856+0000 7f4b2b200640 1 Processor -- start 2024-04-26T23:44:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.857+0000 7f4b2b200640 1 -- start start 2024-04-26T23:44:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4b2c0d1990 0x7f4b2c0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c0d0780 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c0d22b0 con 0x7f4b2c0d1990 2024-04-26T23:44:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b2b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c0cfe50 con 0x7f4b2c1639c0 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b29400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b29400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60288/0 (socket says 172.21.15.46:60288) 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.858+0000 7f4b29400640 1 -- 172.21.15.46:0/3332422422 learned_addr learned my addr 172.21.15.46:0/3332422422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29e00640 1 --2- 172.21.15.46:0/3332422422 >> v2:172.21.15.142:3300/0 conn(0x7f4b2c0d1990 0x7f4b2c0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b2a800640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29400640 1 -- 172.21.15.46:0/3332422422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 msgr2=0x7f4b2c0d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29400640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29400640 1 -- 172.21.15.46:0/3332422422 >> v2:172.21.15.142:3300/0 conn(0x7f4b2c0d1990 msgr2=0x7f4b2c0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29400640 1 --2- 172.21.15.46:0/3332422422 >> v2:172.21.15.142:3300/0 conn(0x7f4b2c0d1990 0x7f4b2c0d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.859+0000 7f4b29400640 1 -- 172.21.15.46:0/3332422422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c0d0130 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.860+0000 7f4b29400640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c08d560 tx=0x7f4b1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f3f7c90852d5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.860+0000 7f4b28a00640 1 -- 172.21.15.46:0/3332422422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14037070 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.860+0000 7f4b28a00640 1 -- 172.21.15.46:0/3332422422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14032440 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.860+0000 7f4b28a00640 1 -- 172.21.15.46:0/3332422422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14032810 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 -- 172.21.15.46:0/3332422422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d0c10 msgr2=0x7f4b2c0d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c08d560 tx=0x7f4b1402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 -- 172.21.15.46:0/3332422422 shutdown_connections 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 --2- 172.21.15.46:0/3332422422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c0d5690 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 --2- 172.21.15.46:0/3332422422 >> v2:172.21.15.142:3300/0 conn(0x7f4b2c0d1990 0x7f4b2c0d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 -- 172.21.15.46:0/3332422422 >> 172.21.15.46:0/3332422422 conn(0x7f4b2c09e2a0 msgr2=0x7f4b2c0d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 -- 172.21.15.46:0/3332422422 shutdown_connections 2024-04-26T23:44:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.861+0000 7f4b2b200640 1 -- 172.21.15.46:0/3332422422 wait complete. 2024-04-26T23:44:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.863+0000 7f4b2b200640 1 Processor -- start 2024-04-26T23:44:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.863+0000 7f4b2b200640 1 -- start start 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c1f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c0cfa90 con 0x7f4b2c0d1990 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c15f530 con 0x7f4b2c0d0c10 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b2b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c0cfc10 con 0x7f4b2c1639c0 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b29400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b29400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60294/0 (socket says 172.21.15.46:60294) 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.864+0000 7f4b29400640 1 -- 172.21.15.46:0/1188856489 learned_addr learned my addr 172.21.15.46:0/1188856489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29e00640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c1f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b2a800640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 -- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 msgr2=0x7f4b2c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 -- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 msgr2=0x7f4b2c1f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c1f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 -- 172.21.15.46:0/1188856489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c0ced50 con 0x7f4b2c0d1990 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29e00640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c1f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b2a800640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b29400640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4b1402a220 tx=0x7f4b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14041020 con 0x7f4b2c0d1990 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b140306c0 con 0x7f4b2c0d1990 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c0cf030 con 0x7f4b2c0d1990 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.865+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14037040 con 0x7f4b2c0d1990 2024-04-26T23:44:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.866+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1e92f0 con 0x7f4b2c0d1990 2024-04-26T23:44:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.867+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c0693b0 con 0x7f4b2c0d1990 2024-04-26T23:44:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.868+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b14044020 con 0x7f4b2c0d1990 2024-04-26T23:44:49.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.869+0000 7f4b22000640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 0x7f4b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:49.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.869+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f4b140ac540 con 0x7f4b2c0d1990 2024-04-26T23:44:49.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.869+0000 7f4b29e00640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 0x7f4b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.870+0000 7f4b29e00640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 0x7f4b0806edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0bab50 tx=0x7f4b1c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:49.874+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b1404b030 con 0x7f4b2c0d1990 2024-04-26T23:44:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.121+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4b2c0d1770 con 0x7f4b2c0d1990 2024-04-26T23:44:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.122+0000 7f4b22000640 1 -- 172.21.15.46:0/1188856489 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f4b1407a7d0 con 0x7f4b2c0d1990 2024-04-26T23:44:50.126 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:44:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 msgr2=0x7f4b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 0x7f4b0806edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0bab50 tx=0x7f4b1c00b040 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 msgr2=0x7f4b2c1e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4b1402a220 tx=0x7f4b140046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 shutdown_connections 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b0806c8f0 0x7f4b0806edb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b2c1639c0 0x7f4b2c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c0d1990 0x7f4b2c1e8db0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 --2- 172.21.15.46:0/1188856489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c0d0c10 0x7f4b2c1f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 >> 172.21.15.46:0/1188856489 conn(0x7f4b2c09e2a0 msgr2=0x7f4b2c0d5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 shutdown_connections 2024-04-26T23:44:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50.125+0000 7f4b2b200640 1 -- 172.21.15.46:0/1188856489 wait complete. 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:44:50.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:50.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.150 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:44:50.204 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:44:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:44:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:44:50.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.209 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:44:50.287 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:44:50.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.291 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:44:50.345 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:44:50.345 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:44:50.345 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:44:50.345 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:44:50.345 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:44:50.345 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:44:50.345 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:44:50.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:44:50.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:44:50.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:44:50.346 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:44:50.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:44:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:50.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.568 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:44:50.619 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:44:50.619 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:44:50.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:50.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:44:50.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:44:50.708 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:50.764 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:44:50.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:44:50.765 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:50.820 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:44:50.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:50.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:50.957 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:50.958 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:50.958 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:50.980 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:50.980 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:50.980 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:50.980 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:44:50.980 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:44:51.006 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:44:51.006 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:44:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:44:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:44:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:44:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:44:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:44:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:44:51.013 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:44:51.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:51.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:51.096 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:44:51.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:44:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:51.242 INFO:teuthology.orchestra.run.smithi046.stdout:client7860 2024-04-26T23:44:51.243 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:51.243 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:51.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:51.298 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir").st_ino) 2024-04-26T23:44:51.366 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:44:51.499 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-26T23:44:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:51.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:51.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.768+0000 7fe345a00640 1 Processor -- start 2024-04-26T23:44:51.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.768+0000 7fe345a00640 1 -- start start 2024-04-26T23:44:51.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe34015edf0 0x7fe3400cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe34015f1d0 0x7fe3400cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3400b03c0 con 0x7fe3400cb1a0 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe3400cf5c0 con 0x7fe34015f1d0 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe3400cb680 con 0x7fe34015edf0 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe33f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe34015f1d0 0x7fe3400cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe3400cb1a0 0x7fe3400ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55092/0 (socket says 172.21.15.46:55092) 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe33f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe34015f1d0 0x7fe3400cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60298/0 (socket says 172.21.15.46:60298) 2024-04-26T23:44:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.770+0000 7fe345000640 1 -- 172.21.15.46:0/2772364621 learned_addr learned my addr 172.21.15.46:0/2772364621 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33fe00640 1 --2- 172.21.15.46:0/2772364621 >> v2:172.21.15.142:3300/0 conn(0x7fe34015edf0 0x7fe3400cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33f400640 1 -- 172.21.15.46:0/2772364621 >> v2:172.21.15.142:3300/0 conn(0x7fe34015edf0 msgr2=0x7fe3400cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33f400640 1 --2- 172.21.15.46:0/2772364621 >> v2:172.21.15.142:3300/0 conn(0x7fe34015edf0 0x7fe3400cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33f400640 1 -- 172.21.15.46:0/2772364621 >> v1:172.21.15.46:6789/0 conn(0x7fe3400cb1a0 legacy=0x7fe3400ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33f400640 1 -- 172.21.15.46:0/2772364621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400cb960 con 0x7fe34015f1d0 2024-04-26T23:44:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.771+0000 7fe33ea00640 1 -- 172.21.15.46:0/2772364621 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 400764700 0 0) 0x7fe3400b03c0 con 0x7fe3400cb1a0 2024-04-26T23:44:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.772+0000 7fe33f400640 1 --2- 172.21.15.46:0/2772364621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34015f1d0 0x7fe3400cd060 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe3400d0a00 tx=0x7fe32c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed35c3a714befdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.772+0000 7fe33ea00640 1 -- 172.21.15.46:0/2772364621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c037070 con 0x7fe34015f1d0 2024-04-26T23:44:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.772+0000 7fe33ea00640 1 -- 172.21.15.46:0/2772364621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c032440 con 0x7fe34015f1d0 2024-04-26T23:44:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.772+0000 7fe33ea00640 1 -- 172.21.15.46:0/2772364621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c032810 con 0x7fe34015f1d0 2024-04-26T23:44:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 -- 172.21.15.46:0/2772364621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34015f1d0 msgr2=0x7fe3400cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 --2- 172.21.15.46:0/2772364621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34015f1d0 0x7fe3400cd060 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe3400d0a00 tx=0x7fe32c02a250 comp rx=0 tx=0).stop 2024-04-26T23:44:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 -- 172.21.15.46:0/2772364621 shutdown_connections 2024-04-26T23:44:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 --2- 172.21.15.46:0/2772364621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34015f1d0 0x7fe3400cd060 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 --2- 172.21.15.46:0/2772364621 >> v2:172.21.15.142:3300/0 conn(0x7fe34015edf0 0x7fe3400cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 -- 172.21.15.46:0/2772364621 >> 172.21.15.46:0/2772364621 conn(0x7fe3400b26d0 msgr2=0x7fe3400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 -- 172.21.15.46:0/2772364621 shutdown_connections 2024-04-26T23:44:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.773+0000 7fe345a00640 1 -- 172.21.15.46:0/2772364621 wait complete. 2024-04-26T23:44:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.775+0000 7fe345a00640 1 Processor -- start 2024-04-26T23:44:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.775+0000 7fe345a00640 1 -- start start 2024-04-26T23:44:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3400cb1a0 0x7fe34015c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34015edf0 0x7fe34014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400cc200 con 0x7fe34014b800 2024-04-26T23:44:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400cbf20 con 0x7fe34015edf0 2024-04-26T23:44:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.776+0000 7fe345a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe3400c6950 con 0x7fe3400cb1a0 2024-04-26T23:44:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60310/0 (socket says 172.21.15.46:60310) 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 -- 172.21.15.46:0/2393030524 learned_addr learned my addr 172.21.15.46:0/2393030524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe33f400640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34015edf0 0x7fe34014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe33fe00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3400cb1a0 0x7fe34015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 -- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3400cb1a0 msgr2=0x7fe34015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3400cb1a0 0x7fe34015c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 -- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34015edf0 msgr2=0x7fe34014bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34015edf0 0x7fe34014bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 -- 172.21.15.46:0/2393030524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400b4b40 con 0x7fe34014b800 2024-04-26T23:44:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.777+0000 7fe345000640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe33800caf0 tx=0x7fe33800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.778+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33800ac40 con 0x7fe34014b800 2024-04-26T23:44:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.778+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33800ae00 con 0x7fe34014b800 2024-04-26T23:44:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.778+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338012640 con 0x7fe34014b800 2024-04-26T23:44:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.778+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3400b4e20 con 0x7fe34014b800 2024-04-26T23:44:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.778+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3400b56b0 con 0x7fe34014b800 2024-04-26T23:44:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.779+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3400b0160 con 0x7fe34014b800 2024-04-26T23:44:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.780+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3380128a0 con 0x7fe34014b800 2024-04-26T23:44:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.781+0000 7fe337e00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 0x7fe32006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.781+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fe33808c390 con 0x7fe34014b800 2024-04-26T23:44:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.782+0000 7fe33fe00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 0x7fe32006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.783+0000 7fe33fe00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 0x7fe32006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe330006fd0 tx=0x7fe330008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:51.786+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3400b0160 con 0x7fe34014b800 2024-04-26T23:44:52.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.032+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3400cfdf0 con 0x7fe34014b800 2024-04-26T23:44:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.033+0000 7fe337e00640 1 -- 172.21.15.46:0/2393030524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7fe3400cfdf0 con 0x7fe34014b800 2024-04-26T23:44:52.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:52.037 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/932747087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":932747087},{"type":"v1","addr":"172.21.15.142:6835","nonce":932747087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2454420217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2454420217},{"type":"v1","addr":"172.21.15.142:6837","nonce":2454420217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3409172203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3409172203},{"type":"v1","addr":"172.21.15.46:6834","nonce":3409172203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:44:45.428792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7695":{"gid":7695,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2670265715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2670265715},{"type":"v1","addr":"172.21.15.46:6835","nonce":2670265715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7695,"qdb_cluster":[7695]},"id":6}]} 2024-04-26T23:44:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 74 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 msgr2=0x7fe32006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 0x7fe32006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe330006fd0 tx=0x7fe330008040 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 msgr2=0x7fe340072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe33800caf0 tx=0x7fe33800a060 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 shutdown_connections 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe32006c8f0 0x7fe32006edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34014b800 0x7fe340072d90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34015edf0 0x7fe34014bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 --2- 172.21.15.46:0/2393030524 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3400cb1a0 0x7fe34015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 >> 172.21.15.46:0/2393030524 conn(0x7fe3400b26d0 msgr2=0x7fe3400af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 shutdown_connections 2024-04-26T23:44:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.036+0000 7fe345a00640 1 -- 172.21.15.46:0/2393030524 wait complete. 2024-04-26T23:44:52.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:44:52.289 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:44:51.353182+0000","btime":"2024-04-26T23:44:51.353182+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:44:51.353182+0000","atime":"2024-04-26T23:44:51.353182+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","type":"iauth","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","type":"ixattr","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":7860,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":7860,"want_loner":7860,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:44:52.289 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:44:51.353182+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "iauth", "xlock_by": {} }, "backtrace_version": 2, "btime": "2024-04-26T23:44:51.353182+0000", "change_attr": 0, "client_caps": [ { "client_id": 7860, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T23:44:51.353182+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 7860, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:44:51.353182+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 7860, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ixattr", "xlock_by": {} }, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:44:52.290 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:52.290 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:52.344 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:44:52.344 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> import uuid 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> n = 21 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = False 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> if 0 > 0: 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = True 2024-04-26T23:44:52.345 DEBUG:teuthology.orchestra.run.smithi046:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file" 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> dpath = os.path.dirname(path) 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> fnameprefix = os.path.basename(path) 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> os.makedirs(dpath, exist_ok=True) 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T23:44:52.346 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> for i in range(n): 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> f.write(f"{i}") 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> f.flush() 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(f.fileno()) 2024-04-26T23:44:52.347 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> os.unlink(fpath) 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> if create_hard_links: 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> for j in range(0): 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:44:52.348 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:44:52.349 DEBUG:teuthology.orchestra.run.smithi046:> finally: 2024-04-26T23:44:52.349 DEBUG:teuthology.orchestra.run.smithi046:> os.close(dirfd) 2024-04-26T23:44:52.349 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:44:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:44:52.608 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:52.608 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:52.709 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:44:52.710 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:44:52.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:44:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.981+0000 7f461a400640 1 Processor -- start 2024-04-26T23:44:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.981+0000 7f461a400640 1 -- start start 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f461a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 0x7f46140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f461a400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140d11f0 con 0x7f46140b4000 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f461a400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140d5f20 con 0x7f461415f4e0 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f461a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46140d0ce0 con 0x7f46140d2130 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f4619a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f46140b4000 0x7f46140b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55692/0 (socket says 172.21.15.46:55692) 2024-04-26T23:44:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.983+0000 7f4619a00640 1 -- 172.21.15.46:0/3801596815 learned_addr learned my addr 172.21.15.46:0/3801596815 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268830604 0 0) 0x7f46140d11f0 con 0x7f46140b4000 2024-04-26T23:44:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f460be00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 0x7f46140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45f8003520 con 0x7f46140b4000 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3088657292 0 0) 0x7f45f8003520 con 0x7f46140b4000 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 msgr2=0x7f46140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 0x7f46140d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 >> v1:172.21.15.142:6789/0 conn(0x7f461415f4e0 legacy=0x7f46140d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140d0f10 con 0x7f46140b4000 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f460c0019f0 con 0x7f46140b4000 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 >> v1:172.21.15.46:6789/0 conn(0x7f46140b4000 legacy=0x7f46140b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f80039e0 0x7f45f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f45f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 0x7f45f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45f800ba70 con 0x7f45f8004800 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.984+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45f800bbf0 con 0x7f45f80039e0 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45f800bdf0 con 0x7f46140d2130 2024-04-26T23:44:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f45f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 -- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 msgr2=0x7f45f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46140d2130 0x7f45f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 -- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f80039e0 msgr2=0x7f45f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f80039e0 0x7f45f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.985+0000 7f460be00640 1 -- 172.21.15.46:0/3801596815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f8003520 con 0x7f45f8004800 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f460be00640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f45f8004c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009a90 tx=0x7f45f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecdbb9a2e79b201 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc036070 con 0x7f45f8004800 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45fc031400 con 0x7f45f8004800 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f4613e00640 1 -- 172.21.15.46:0/3801596815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc009690 con 0x7f45f8004800 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f461a400640 1 -- 172.21.15.46:0/3801596815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 msgr2=0x7f45f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f461a400640 1 --2- 172.21.15.46:0/3801596815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f45f8004c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009a90 tx=0x7f45f8005830 comp rx=0 tx=0).stop 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.987+0000 7f4619a00640 1 -- 172.21.15.46:0/3801596815 reap_dead start 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.988+0000 7f461a400640 1 -- 172.21.15.46:0/3801596815 shutdown_connections 2024-04-26T23:44:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.988+0000 7f461a400640 1 -- 172.21.15.46:0/3801596815 >> 172.21.15.46:0/3801596815 conn(0x7f46140d5010 msgr2=0x7f46140d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:52.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.988+0000 7f461a400640 1 -- 172.21.15.46:0/3801596815 shutdown_connections 2024-04-26T23:44:52.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.988+0000 7f461a400640 1 -- 172.21.15.46:0/3801596815 wait complete. 2024-04-26T23:44:52.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.990+0000 7f461a400640 1 Processor -- start 2024-04-26T23:44:52.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.990+0000 7f461a400640 1 -- start start 2024-04-26T23:44:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f461406dd50 0x7f461406e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 0x7f46140cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b1d60 con 0x7f45f8004800 2024-04-26T23:44:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46140d2990 con 0x7f46140cc290 2024-04-26T23:44:52.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f461a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f461415f8c0 con 0x7f461406dd50 2024-04-26T23:44:52.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:52.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45990/0 (socket says 172.21.15.46:45990) 2024-04-26T23:44:52.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 -- 172.21.15.46:0/496639603 learned_addr learned my addr 172.21.15.46:0/496639603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619a00640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 0x7f46140cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f460be00640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f461406dd50 0x7f461406e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 -- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f461406dd50 msgr2=0x7f461406e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f461406dd50 0x7f461406e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 -- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 msgr2=0x7f46140cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 0x7f46140cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.991+0000 7f4619000640 1 -- 172.21.15.46:0/496639603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461415d780 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f4619a00640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 0x7f46140cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f4619000640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f460400c560 tx=0x7f460400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040052d0 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604012630 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040054b0 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46140caea0 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.992+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46140cb460 con 0x7f45f8004800 2024-04-26T23:44:52.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.994+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46140b5580 con 0x7f45f8004800 2024-04-26T23:44:52.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.995+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4604012a20 con 0x7f45f8004800 2024-04-26T23:44:52.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.995+0000 7f4611600640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:52.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.996+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f460408b960 con 0x7f45f8004800 2024-04-26T23:44:52.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.996+0000 7f460be00640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:52.997+0000 7f460be00640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f45fc033000 tx=0x7f45fc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.000+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46140b5580 con 0x7f45f8004800 2024-04-26T23:44:53.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.204+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46140693b0 con 0x7f45f8004800 2024-04-26T23:44:53.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.207+0000 7f4611600640 1 -- 172.21.15.46:0/496639603 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16849 (secure 0 0 0) 0x7f4604059bf0 con 0x7f45f8004800 2024-04-26T23:44:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:53.211 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:43.845091+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:44:53.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 msgr2=0x7f45f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f45fc033000 tx=0x7f45fc008030 comp rx=0 tx=0).stop 2024-04-26T23:44:53.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 msgr2=0x7f46140ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f460400c560 tx=0x7f460400ca20 comp rx=0 tx=0).stop 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 shutdown_connections 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f006c8f0 0x7f45f006edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140cc290 0x7f46140cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f461406dd50 0x7f461406e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 --2- 172.21.15.46:0/496639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f8004800 0x7f46140ce010 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 >> 172.21.15.46:0/496639603 conn(0x7f46140d5010 msgr2=0x7f46140d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 shutdown_connections 2024-04-26T23:44:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.210+0000 7f461a400640 1 -- 172.21.15.46:0/496639603 wait complete. 2024-04-26T23:44:53.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:44:40.164396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:44:41.287502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:44:53.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:44:53.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.451+0000 7f7c85600640 1 Processor -- start 2024-04-26T23:44:53.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.452+0000 7f7c85600640 1 -- start start 2024-04-26T23:44:53.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.453+0000 7f7c85600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7c8015d730 0x7f7c800d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.453+0000 7f7c85600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 0x7f7c8006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.453+0000 7f7c85600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8006eb90 con 0x7f7c800b53b0 2024-04-26T23:44:53.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.453+0000 7f7c85600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7c800b4310 con 0x7f7c8015d730 2024-04-26T23:44:53.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.453+0000 7f7c85600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c800b45f0 con 0x7f7c8015e220 2024-04-26T23:44:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.454+0000 7f7c7f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 0x7f7c8006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.454+0000 7f7c7f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 0x7f7c8006dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:56744/0 (socket says 172.21.15.46:56744) 2024-04-26T23:44:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.454+0000 7f7c7f400640 1 -- 172.21.15.46:0/2279924954 learned_addr learned my addr 172.21.15.46:0/2279924954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.454+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2279924954 >> v2:172.21.15.142:3300/0 conn(0x7f7c8015d730 0x7f7c800d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.455+0000 7f7c7ea00640 1 -- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 msgr2=0x7f7c8006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.455+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 0x7f7c8006dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.455+0000 7f7c7ea00640 1 -- 172.21.15.46:0/2279924954 >> v1:172.21.15.142:6789/0 conn(0x7f7c800b53b0 legacy=0x7f7c8015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.455+0000 7f7c7ea00640 1 -- 172.21.15.46:0/2279924954 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c800b48d0 con 0x7f7c8015d730 2024-04-26T23:44:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.456+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015d730 0x7f7c800d00d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b2409c869da5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c7d600640 1 -- 172.21.15.46:0/2279924954 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70037070 con 0x7f7c8015d730 2024-04-26T23:44:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c7d600640 1 -- 172.21.15.46:0/2279924954 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c70032440 con 0x7f7c8015d730 2024-04-26T23:44:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c7d600640 1 -- 172.21.15.46:0/2279924954 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70032810 con 0x7f7c8015d730 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 -- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015d730 msgr2=0x7f7c800d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 --2- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015d730 0x7f7c800d00d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002a180 comp rx=0 tx=0).stop 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 -- 172.21.15.46:0/2279924954 shutdown_connections 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 --2- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c8015e220 0x7f7c8006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 --2- 172.21.15.46:0/2279924954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015d730 0x7f7c800d00d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.457+0000 7f7c85600640 1 -- 172.21.15.46:0/2279924954 >> 172.21.15.46:0/2279924954 conn(0x7f7c8006c980 msgr2=0x7f7c800b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.458+0000 7f7c85600640 1 -- 172.21.15.46:0/2279924954 shutdown_connections 2024-04-26T23:44:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.458+0000 7f7c85600640 1 -- 172.21.15.46:0/2279924954 wait complete. 2024-04-26T23:44:53.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.459+0000 7f7c85600640 1 Processor -- start 2024-04-26T23:44:53.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.460+0000 7f7c85600640 1 -- start start 2024-04-26T23:44:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.460+0000 7f7c85600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 0x7f7c801f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.460+0000 7f7c85600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c85600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015e220 0x7f7c801f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c85600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800b3ec0 con 0x7f7c8015d730 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c85600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800b4e90 con 0x7f7c8015e220 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c85600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c800b4040 con 0x7f7c800b53b0 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46002/0 (socket says 172.21.15.46:46002) 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 -- 172.21.15.46:0/2535856816 learned_addr learned my addr 172.21.15.46:0/2535856816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 0x7f7c801f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 -- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 msgr2=0x7f7c801f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 0x7f7c801f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 -- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015e220 msgr2=0x7f7c801f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015e220 0x7f7c801f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7e000640 1 -- 172.21.15.46:0/2535856816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c801ea4b0 con 0x7f7c8015d730 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.461+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 0x7f7c801f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c7e000640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c3c0 tx=0x7f7c6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c680052d0 con 0x7f7c8015d730 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c68005490 con 0x7f7c8015d730 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c801ea790 con 0x7f7c8015d730 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68007260 con 0x7f7c8015d730 2024-04-26T23:44:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.462+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801ead50 con 0x7f7c8015d730 2024-04-26T23:44:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.463+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c801eb240 con 0x7f7c8015d730 2024-04-26T23:44:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.464+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c68007400 con 0x7f7c8015d730 2024-04-26T23:44:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.465+0000 7f7c76a00640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 0x7f7c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.465+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f7c6808b100 con 0x7f7c8015d730 2024-04-26T23:44:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.466+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 0x7f7c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.467+0000 7f7c7ea00640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 0x7f7c5c06edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c70034000 tx=0x7f7c7002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.470+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c68059390 con 0x7f7c8015d730 2024-04-26T23:44:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.676+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c80069530 con 0x7f7c8015d730 2024-04-26T23:44:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.678+0000 7f7c76a00640 1 -- 172.21.15.46:0/2535856816 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f7c68058c90 con 0x7f7c8015d730 2024-04-26T23:44:53.681 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:44:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 msgr2=0x7f7c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 0x7f7c5c06edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c70034000 tx=0x7f7c7002e000 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 msgr2=0x7f7c80145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c3c0 tx=0x7f7c6800c880 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 shutdown_connections 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7c5c06c8f0 0x7f7c5c06edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c8015e220 0x7f7c801f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c8015d730 0x7f7c80145fe0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 --2- 172.21.15.46:0/2535856816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c800b53b0 0x7f7c801f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 >> 172.21.15.46:0/2535856816 conn(0x7f7c8006c980 msgr2=0x7f7c80072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 shutdown_connections 2024-04-26T23:44:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.681+0000 7f7c85600640 1 -- 172.21.15.46:0/2535856816 wait complete. 2024-04-26T23:44:53.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:44:53.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.918+0000 7f3d7d000640 1 Processor -- start 2024-04-26T23:44:53.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.918+0000 7f3d7d000640 1 -- start start 2024-04-26T23:44:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.919+0000 7f3d7d000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d780cdd20 0x7f3d780cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.919+0000 7f3d7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d780ae8f0 0x7f3d780aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d7d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d780af570 con 0x7f3d7815f210 2024-04-26T23:44:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d7d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3d780af850 con 0x7f3d780ae8f0 2024-04-26T23:44:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d7d000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3d780cb680 con 0x7f3d780cdd20 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d780ae8f0 0x7f3d780aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d780ae8f0 0x7f3d780aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46008/0 (socket says 172.21.15.46:46008) 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 -- 172.21.15.46:0/724488928 learned_addr learned my addr 172.21.15.46:0/724488928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 -- 172.21.15.46:0/724488928 >> v2:172.21.15.142:3300/0 conn(0x7f3d780cdd20 msgr2=0x7f3d780cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 --2- 172.21.15.46:0/724488928 >> v2:172.21.15.142:3300/0 conn(0x7f3d780cdd20 0x7f3d780cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 -- 172.21.15.46:0/724488928 >> v1:172.21.15.142:6789/0 conn(0x7f3d7815f210 legacy=0x7f3d780cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.920+0000 7f3d77e00640 1 -- 172.21.15.46:0/724488928 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d780cb960 con 0x7f3d780ae8f0 2024-04-26T23:44:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d77e00640 1 --2- 172.21.15.46:0/724488928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d780ae8f0 0x7f3d780aed00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3d780b3710 tx=0x7f3d7002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf5c54e2d08181 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d76000640 1 -- 172.21.15.46:0/724488928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70036070 con 0x7f3d780ae8f0 2024-04-26T23:44:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d76000640 1 -- 172.21.15.46:0/724488928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d70031600 con 0x7f3d780ae8f0 2024-04-26T23:44:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d76000640 1 -- 172.21.15.46:0/724488928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700319d0 con 0x7f3d780ae8f0 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d7d000640 1 -- 172.21.15.46:0/724488928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d780ae8f0 msgr2=0x7f3d780aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.922+0000 7f3d7d000640 1 --2- 172.21.15.46:0/724488928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d780ae8f0 0x7f3d780aed00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3d780b3710 tx=0x7f3d7002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 -- 172.21.15.46:0/724488928 shutdown_connections 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 --2- 172.21.15.46:0/724488928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d780ae8f0 0x7f3d780aed00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 --2- 172.21.15.46:0/724488928 >> v2:172.21.15.142:3300/0 conn(0x7f3d780cdd20 0x7f3d780cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 -- 172.21.15.46:0/724488928 >> 172.21.15.46:0/724488928 conn(0x7f3d780b27b0 msgr2=0x7f3d780b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 -- 172.21.15.46:0/724488928 shutdown_connections 2024-04-26T23:44:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.923+0000 7f3d7d000640 1 -- 172.21.15.46:0/724488928 wait complete. 2024-04-26T23:44:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.924+0000 7f3d7d000640 1 Processor -- start 2024-04-26T23:44:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.924+0000 7f3d7d000640 1 -- start start 2024-04-26T23:44:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 0x7f3d7806e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d780cdd20 0x7f3d780cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7815f5f0 con 0x7f3d7815f210 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d780af240 con 0x7f3d780ae8f0 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.925+0000 7f3d7d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d78071e80 con 0x7f3d780cdd20 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46018/0 (socket says 172.21.15.46:46018) 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 -- 172.21.15.46:0/813019280 learned_addr learned my addr 172.21.15.46:0/813019280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77400640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 0x7f3d7806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d76a00640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d780cdd20 0x7f3d780cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 -- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d780cdd20 msgr2=0x7f3d780cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d780cdd20 0x7f3d780cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 -- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 msgr2=0x7f3d7806e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 0x7f3d7806e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77e00640 1 -- 172.21.15.46:0/813019280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d78154bf0 con 0x7f3d7815f210 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.926+0000 7f3d77400640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 0x7f3d7806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d77e00640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3d70034040 tx=0x7f3d70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70041020 con 0x7f3d7815f210 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d700077a0 con 0x7f3d7815f210 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70036040 con 0x7f3d7815f210 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7814ec60 con 0x7f3d7815f210 2024-04-26T23:44:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.927+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7814f4a0 con 0x7f3d7815f210 2024-04-26T23:44:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.928+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d780693b0 con 0x7f3d7815f210 2024-04-26T23:44:53.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.929+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d70044020 con 0x7f3d7815f210 2024-04-26T23:44:53.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.930+0000 7f3d6f400640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 0x7f3d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:53.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.930+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f3d700acae0 con 0x7f3d7815f210 2024-04-26T23:44:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.931+0000 7f3d77400640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 0x7f3d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.932+0000 7f3d77400640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 0x7f3d5806edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800a810 tx=0x7f3d68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:53.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:53.935+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d7004b030 con 0x7f3d7815f210 2024-04-26T23:44:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.137+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3d780cda20 con 0x7f3d7815f210 2024-04-26T23:44:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.138+0000 7f3d6f400640 1 -- 172.21.15.46:0/813019280 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f3d7814ec60 con 0x7f3d7815f210 2024-04-26T23:44:54.142 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:44:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 msgr2=0x7f3d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 0x7f3d5806edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800a810 tx=0x7f3d68008040 comp rx=0 tx=0).stop 2024-04-26T23:44:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 msgr2=0x7f3d780c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3d70034040 tx=0x7f3d70004a40 comp rx=0 tx=0).stop 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 shutdown_connections 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d5806c8f0 0x7f3d5806edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7815f210 0x7f3d780c8c30 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d780cdd20 0x7f3d780cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 --2- 172.21.15.46:0/813019280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d780ae8f0 0x7f3d7806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 >> 172.21.15.46:0/813019280 conn(0x7f3d780b27b0 msgr2=0x7f3d780b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 shutdown_connections 2024-04-26T23:44:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.141+0000 7f3d7d000640 1 -- 172.21.15.46:0/813019280 wait complete. 2024-04-26T23:44:54.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:44:54.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.367+0000 7f4383000640 1 Processor -- start 2024-04-26T23:44:54.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.368+0000 7f4383000640 1 -- start start 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.369+0000 7f4383000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f437c0d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.369+0000 7f4383000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c0d4090 con 0x7f437c0d1980 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.369+0000 7f4383000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c0b4020 con 0x7f437c0d24f0 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.369+0000 7f4383000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0d3880 con 0x7f437c15f840 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.370+0000 7f4381200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f437c0d1980 0x7f437c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55696/0 (socket says 172.21.15.46:55696) 2024-04-26T23:44:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.370+0000 7f4381200640 1 -- 172.21.15.46:0/387221595 learned_addr learned my addr 172.21.15.46:0/387221595 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.370+0000 7f4381c00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f437c0d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.370+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4083703195 0 0) 0x7f4368001110 con 0x7f437c0d1980 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.370+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003520 con 0x7f437c0d1980 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046513282 0 0) 0x7f4364003520 con 0x7f437c0d1980 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 msgr2=0x7f437c0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f437c0d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 >> v1:172.21.15.142:6789/0 conn(0x7f437c0d24f0 legacy=0x7f437c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0d3ad0 con 0x7f437c0d1980 2024-04-26T23:44:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4368001840 con 0x7f437c0d1980 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 >> v1:172.21.15.46:6789/0 conn(0x7f437c0d1980 legacy=0x7f437c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f43640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f4364004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4364004d40 0x7f43640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0d3880 con 0x7f4364003f60 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f436400baa0 con 0x7f4364004d40 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f436400bca0 con 0x7f437c15f840 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.371+0000 7f4381200640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f4364004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381c00640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f43640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381200640 1 -- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 msgr2=0x7f43640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381200640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c15f840 0x7f43640039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381200640 1 -- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4364004d40 msgr2=0x7f43640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381200640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4364004d40 0x7f43640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.372+0000 7f4381200640 1 -- 172.21.15.46:0/387221595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364003520 con 0x7f4364003f60 2024-04-26T23:44:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.373+0000 7f4381200640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f4364004800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f436800c4c0 tx=0x7f4364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b320f457a87671 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.373+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436800c0e0 con 0x7f4364003f60 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.373+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368034070 con 0x7f4364003f60 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.373+0000 7f437be00640 1 -- 172.21.15.46:0/387221595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368005d70 con 0x7f4364003f60 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 -- 172.21.15.46:0/387221595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 msgr2=0x7f4364004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 --2- 172.21.15.46:0/387221595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f4364004800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f436800c4c0 tx=0x7f4364005830 comp rx=0 tx=0).stop 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4382600640 1 -- 172.21.15.46:0/387221595 reap_dead start 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 -- 172.21.15.46:0/387221595 shutdown_connections 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 -- 172.21.15.46:0/387221595 >> 172.21.15.46:0/387221595 conn(0x7f437c0d4f70 msgr2=0x7f437c0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 -- 172.21.15.46:0/387221595 shutdown_connections 2024-04-26T23:44:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.374+0000 7f4383000640 1 -- 172.21.15.46:0/387221595 wait complete. 2024-04-26T23:44:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.376+0000 7f4383000640 1 Processor -- start 2024-04-26T23:44:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.376+0000 7f4383000640 1 -- start start 2024-04-26T23:44:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437c0b7dc0 0x7f437c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 0x7f437c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0d2e70 con 0x7f4364003f60 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0b27d0 con 0x7f437c0b7dc0 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4383000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0d2ff0 con 0x7f437c0cd960 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4381c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4381c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46046/0 (socket says 172.21.15.46:46046) 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4381c00640 1 -- 172.21.15.46:0/2376859533 learned_addr learned my addr 172.21.15.46:0/2376859533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4381200640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437c0b7dc0 0x7f437c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4382600640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 0x7f437c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.377+0000 7f4381c00640 1 -- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 msgr2=0x7f437c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4381c00640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 0x7f437c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4381c00640 1 -- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437c0b7dc0 msgr2=0x7f437c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4381c00640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437c0b7dc0 0x7f437c0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4381c00640 1 -- 172.21.15.46:0/2376859533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0cb110 con 0x7f4364003f60 2024-04-26T23:44:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4382600640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 0x7f437c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4381c00640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f437000f350 tx=0x7f437000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437001c070 con 0x7f4364003f60 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4370017620 con 0x7f4364003f60 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370017a10 con 0x7f4364003f60 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c0ce680 con 0x7f4364003f60 2024-04-26T23:44:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.378+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c0cec40 con 0x7f4364003f60 2024-04-26T23:44:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.380+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c0693b0 con 0x7f4364003f60 2024-04-26T23:44:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.381+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f437001f070 con 0x7f4364003f60 2024-04-26T23:44:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.382+0000 7f4379600640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 0x7f435c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.382+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f437008bff0 con 0x7f4364003f60 2024-04-26T23:44:54.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.383+0000 7f4381200640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 0x7f435c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.384+0000 7f4381200640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 0x7f435c06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f436800c490 tx=0x7f436802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.387+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4370012070 con 0x7f4364003f60 2024-04-26T23:44:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.595+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f437c0d5d10 con 0x7f4364003f60 2024-04-26T23:44:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.596+0000 7f4379600640 1 -- 172.21.15.46:0/2376859533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f4370059c80 con 0x7f4364003f60 2024-04-26T23:44:54.600 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 msgr2=0x7f435c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 0x7f435c06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f436800c490 tx=0x7f436802e000 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 msgr2=0x7f437c0ba2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f437000f350 tx=0x7f437000f810 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 shutdown_connections 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f435c06c8f0 0x7f435c06edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437c0cd960 0x7f437c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437c0b7dc0 0x7f437c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 --2- 172.21.15.46:0/2376859533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364003f60 0x7f437c0ba2c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.599+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 >> 172.21.15.46:0/2376859533 conn(0x7f437c0d4f70 msgr2=0x7f437c0b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.600+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 shutdown_connections 2024-04-26T23:44:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.600+0000 7f4383000640 1 -- 172.21.15.46:0/2376859533 wait complete. 2024-04-26T23:44:54.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.833+0000 7f47d6000640 1 Processor -- start 2024-04-26T23:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.833+0000 7f47d6000640 1 -- start start 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.834+0000 7f47d6000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f47d00b4000 0x7f47d00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.834+0000 7f47d6000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00d11f0 con 0x7f47d015f4e0 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.834+0000 7f47d6000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00d5f20 con 0x7f47d00d2130 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.834+0000 7f47d6000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f47d00d0ce0 con 0x7f47d00b4000 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.835+0000 7f47d4c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f47d015f4e0 0x7f47d00d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55706/0 (socket says 172.21.15.46:55706) 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.835+0000 7f47d4c00640 1 -- 172.21.15.46:0/2222459621 learned_addr learned my addr 172.21.15.46:0/2222459621 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.835+0000 7f47d5600640 1 --2- 172.21.15.46:0/2222459621 >> v2:172.21.15.142:3300/0 conn(0x7f47d00b4000 0x7f47d00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.835+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330901426 0 0) 0x7f47d00d11f0 con 0x7f47d015f4e0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b4003520 con 0x7f47d015f4e0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895037179 0 0) 0x7f47d00d5f20 con 0x7f47d00d2130 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47d00d11f0 con 0x7f47d00d2130 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2143264173 0 0) 0x7f47b4003520 con 0x7f47d015f4e0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 >> v2:172.21.15.142:3300/0 conn(0x7f47d00b4000 msgr2=0x7f47d00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 --2- 172.21.15.46:0/2222459621 >> v2:172.21.15.142:3300/0 conn(0x7f47d00b4000 0x7f47d00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 >> v1:172.21.15.142:6789/0 conn(0x7f47d00d2130 legacy=0x7f47d00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00d0f10 con 0x7f47d015f4e0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f47c0001670 con 0x7f47d015f4e0 2024-04-26T23:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 >> v1:172.21.15.46:6789/0 conn(0x7f47d015f4e0 legacy=0x7f47d00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003bc0 0x7f47b4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47b400c090 con 0x7f47b40051c0 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47b400c240 con 0x7f47b4004800 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.836+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47b400c440 con 0x7f47b4003bc0 2024-04-26T23:44:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d4c00640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003bc0 0x7f47b4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47c7e00640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 -- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003bc0 msgr2=0x7f47b4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003bc0 0x7f47b4003fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 -- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 msgr2=0x7f47b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.837+0000 7f47d5600640 1 -- 172.21.15.46:0/2222459621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b4003520 con 0x7f47b40051c0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.838+0000 7f47d5600640 1 -- 172.21.15.46:0/2222459621 reap_dead start 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.838+0000 7f47d5600640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fc0 tx=0x7f47b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d47aae4c363b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.838+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8035020 con 0x7f47b40051c0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.838+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c8003bd0 con 0x7f47b40051c0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.838+0000 7f47cfe00640 1 -- 172.21.15.46:0/2222459621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8009c10 con 0x7f47b40051c0 2024-04-26T23:44:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 -- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 msgr2=0x7f47b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fc0 tx=0x7f47b4005830 comp rx=0 tx=0).stop 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 -- 172.21.15.46:0/2222459621 shutdown_connections 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 --2- 172.21.15.46:0/2222459621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 -- 172.21.15.46:0/2222459621 >> 172.21.15.46:0/2222459621 conn(0x7f47d00d5010 msgr2=0x7f47d00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 -- 172.21.15.46:0/2222459621 shutdown_connections 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.839+0000 7f47d6000640 1 -- 172.21.15.46:0/2222459621 wait complete. 2024-04-26T23:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.841+0000 7f47d6000640 1 Processor -- start 2024-04-26T23:44:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.841+0000 7f47d6000640 1 -- start start 2024-04-26T23:44:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d00c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00cd270 0x7f47d00cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00b1d60 con 0x7f47d0160790 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d2990 con 0x7f47b40051c0 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47d6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47d015f8c0 con 0x7f47d00cd270 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.842+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46064/0 (socket says 172.21.15.46:46064) 2024-04-26T23:44:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 -- 172.21.15.46:0/886227688 learned_addr learned my addr 172.21.15.46:0/886227688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47d4c00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d00c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47d5600640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00cd270 0x7f47d00cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 -- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00cd270 msgr2=0x7f47d00cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00cd270 0x7f47d00cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 -- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 msgr2=0x7f47d00c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d00c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 -- 172.21.15.46:0/886227688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00cc210 con 0x7f47d0160790 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47d4c00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d00c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.843+0000 7f47c7e00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d7d0 tx=0x7f47b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.844+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8016070 con 0x7f47d0160790 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.844+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b800b8b0 con 0x7f47d0160790 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.844+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800bc80 con 0x7f47d0160790 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.844+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d00cc4b0 con 0x7f47d0160790 2024-04-26T23:44:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.844+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d00cca70 con 0x7f47d0160790 2024-04-26T23:44:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.845+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d00693b0 con 0x7f47d0160790 2024-04-26T23:44:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.846+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b8019070 con 0x7f47d0160790 2024-04-26T23:44:54.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.847+0000 7f47cd600640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:54.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.847+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7f47b808bc40 con 0x7f47d0160790 2024-04-26T23:44:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.848+0000 7f47d4c00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.849+0000 7f47d4c00640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001200 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:54.851+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47b800be20 con 0x7f47d0160790 2024-04-26T23:44:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.070+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47d00c8720 con 0x7f47d0160790 2024-04-26T23:44:55.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.071+0000 7f47cd600640 1 -- 172.21.15.46:0/886227688 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) ==== 80+0+0 (secure 0 0 0) 0x7f47b8059ed0 con 0x7f47d0160790 2024-04-26T23:44:55.075 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 msgr2=0x7f47ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001200 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 msgr2=0x7f47d0160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d7d0 tx=0x7f47b800dc90 comp rx=0 tx=0).stop 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 shutdown_connections 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00cd270 0x7f47d00cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d0160790 0x7f47d0160c10 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 --2- 172.21.15.46:0/886227688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d00c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 >> 172.21.15.46:0/886227688 conn(0x7f47d00d5010 msgr2=0x7f47d00b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 shutdown_connections 2024-04-26T23:44:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.074+0000 7f47d6000640 1 -- 172.21.15.46:0/886227688 wait complete. 2024-04-26T23:44:55.096 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:44:55.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:55.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:44:55.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:44:55.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:44:55.318 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:44:55.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-04-26T23:44:55.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.584+0000 7fc659c00640 1 Processor -- start 2024-04-26T23:44:55.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.584+0000 7fc659c00640 1 -- start start 2024-04-26T23:44:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.585+0000 7fc659c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc65406f4b0 0x7fc6540b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc659c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6540ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc659c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6540b4b30 con 0x7fc6540d04c0 2024-04-26T23:44:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc659c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc6540b4e10 con 0x7fc65406f4b0 2024-04-26T23:44:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc659c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6540b50f0 con 0x7fc6540b4320 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc652a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6540d04c0 0x7fc65406eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55720/0 (socket says 172.21.15.46:55720) 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc652a00640 1 -- 172.21.15.46:0/2413496229 learned_addr learned my addr 172.21.15.46:0/2413496229 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc653400640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6540ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.586+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> v2:172.21.15.142:3300/0 conn(0x7fc65406f4b0 0x7fc6540b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68925521 0 0) 0x7fc6540b4b30 con 0x7fc6540d04c0 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc638003520 con 0x7fc6540d04c0 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3525471258 0 0) 0x7fc638003520 con 0x7fc6540d04c0 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 msgr2=0x7fc6540ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6540ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 >> v2:172.21.15.142:3300/0 conn(0x7fc65406f4b0 msgr2=0x7fc6540b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 --2- 172.21.15.46:0/2413496229 >> v2:172.21.15.142:3300/0 conn(0x7fc65406f4b0 0x7fc6540b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6540b53d0 con 0x7fc6540d04c0 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> v2:172.21.15.142:3300/0 conn(0x7fc65406f4b0 0x7fc6540b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc644001690 con 0x7fc6540d04c0 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 >> v1:172.21.15.46:6789/0 conn(0x7fc6540d04c0 legacy=0x7fc65406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc638003f30 0x7fc638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638004d40 0x7fc6380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540b4e10 con 0x7fc638003f30 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540b50f0 con 0x7fc638004d40 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.587+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc63800dcd0 con 0x7fc6540b4320 2024-04-26T23:44:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc638003f30 0x7fc638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc653400640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638004d40 0x7fc6380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652a00640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 -- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 msgr2=0x7fc6380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 -- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638004d40 msgr2=0x7fc6380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638004d40 0x7fc6380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652000640 1 -- 172.21.15.46:0/2413496229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc638003520 con 0x7fc638003f30 2024-04-26T23:44:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.588+0000 7fc652a00640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6540b4320 0x7fc6380039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.589+0000 7fc652000640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc638003f30 0x7fc638004800 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098d0 tx=0x7fc638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4c3cca6526eb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00f040 con 0x7fc638003f30 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c02ad50 con 0x7fc638003f30 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc651600640 1 -- 172.21.15.46:0/2413496229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c004250 con 0x7fc638003f30 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc659c00640 1 -- 172.21.15.46:0/2413496229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc638003f30 msgr2=0x7fc638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc659c00640 1 --2- 172.21.15.46:0/2413496229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc638003f30 0x7fc638004800 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098d0 tx=0x7fc638005830 comp rx=0 tx=0).stop 2024-04-26T23:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc653400640 1 -- 172.21.15.46:0/2413496229 reap_dead start 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc659c00640 1 -- 172.21.15.46:0/2413496229 shutdown_connections 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.590+0000 7fc659c00640 1 -- 172.21.15.46:0/2413496229 >> 172.21.15.46:0/2413496229 conn(0x7fc65406c980 msgr2=0x7fc6540b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.591+0000 7fc659c00640 1 -- 172.21.15.46:0/2413496229 shutdown_connections 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.591+0000 7fc659c00640 1 -- 172.21.15.46:0/2413496229 wait complete. 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.593+0000 7fc659c00640 1 Processor -- start 2024-04-26T23:44:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.593+0000 7fc659c00640 1 -- start start 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638003f30 0x7fc6541f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc65413c980 0x7fc65415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540b4910 con 0x7fc65415e8b0 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540b4790 con 0x7fc638003f30 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc659c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc65406dc90 con 0x7fc65413c980 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc653400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc653400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46080/0 (socket says 172.21.15.46:46080) 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc653400640 1 -- 172.21.15.46:0/373639565 learned_addr learned my addr 172.21.15.46:0/373639565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc652000640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc65413c980 0x7fc65415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.594+0000 7fc652a00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638003f30 0x7fc6541f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 -- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc65413c980 msgr2=0x7fc65415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc65413c980 0x7fc65415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 -- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638003f30 msgr2=0x7fc6541f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638003f30 0x7fc6541f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 -- 172.21.15.46:0/373639565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65413c040 con 0x7fc65415e8b0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc653400640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc65406faf0 tx=0x7fc64c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc64aa00640 1 -- 172.21.15.46:0/373639565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c011e10 con 0x7fc65415e8b0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc64aa00640 1 -- 172.21.15.46:0/373639565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c01b070 con 0x7fc65415e8b0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc64aa00640 1 -- 172.21.15.46:0/373639565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c003990 con 0x7fc65415e8b0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.595+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65413b4c0 con 0x7fc65415e8b0 2024-04-26T23:44:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.596+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65413ba80 con 0x7fc65415e8b0 2024-04-26T23:44:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.598+0000 7fc64aa00640 1 -- 172.21.15.46:0/373639565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc64c005d10 con 0x7fc65415e8b0 2024-04-26T23:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.599+0000 7fc64aa00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.599+0000 7fc64aa00640 1 -- 172.21.15.46:0/373639565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fc64c08c0b0 con 0x7fc65415e8b0 2024-04-26T23:44:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.599+0000 7fc65c9eb740 1 Processor -- start 2024-04-26T23:44:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.600+0000 7fc65c9eb740 1 -- start start 2024-04-26T23:44:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.600+0000 7fc652a00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366af0 0x560b2233f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2233d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b2224f030 con 0x560b22366ed0 2024-04-26T23:44:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b2226a7f0 con 0x560b22369890 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560b222257e0 con 0x560b22366af0 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc652a00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 0x7fc63006edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc644001880 tx=0x7fc644009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc652000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc652000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46088/0 (socket says 172.21.15.46:46088) 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc652000640 1 -- 172.21.15.46:0/4010001836 learned_addr learned my addr 172.21.15.46:0/4010001836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc652a00640 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366af0 0x560b2233f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.601+0000 7fc653400640 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2233d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 -- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366af0 msgr2=0x560b2233f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366af0 0x560b2233f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 -- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 msgr2=0x560b2233d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2233d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 -- 172.21.15.46:0/4010001836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22231050 con 0x560b22366ed0 2024-04-26T23:44:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc652000640 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098d0 tx=0x7fc63c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484aa78b99522193 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc643400640 1 -- 172.21.15.46:0/4010001836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c042070 con 0x560b22366ed0 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc643400640 1 -- 172.21.15.46:0/4010001836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c02ac10 con 0x560b22366ed0 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.602+0000 7fc643400640 1 -- 172.21.15.46:0/4010001836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c004250 con 0x560b22366ed0 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 -- 172.21.15.46:0/4010001836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 msgr2=0x560b22369350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098d0 tx=0x7fc63c02c000 comp rx=0 tx=0).stop 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 -- 172.21.15.46:0/4010001836 shutdown_connections 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2233d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22366ed0 0x560b22369350 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/4010001836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366af0 0x560b2233f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 -- 172.21.15.46:0/4010001836 >> 172.21.15.46:0/4010001836 conn(0x560b2233bb50 msgr2=0x560b22384350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.603+0000 7fc65c9eb740 1 -- 172.21.15.46:0/4010001836 shutdown_connections 2024-04-26T23:44:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.604+0000 7fc65c9eb740 1 -- 172.21.15.46:0/4010001836 wait complete. 2024-04-26T23:44:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.604+0000 7fc65c9eb740 1 -- start start 2024-04-26T23:44:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.604+0000 7fc65c9eb740 1 Processor -- start 2024-04-26T23:44:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.605+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22350bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.605+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.605+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b223653a0 0x560b2237b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.605+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b222a7200 con 0x560b22369890 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b222a1ee0 con 0x560b223653a0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560b222aee70 con 0x560b22366ed0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc65c9eb740 10 client.0 _ll_register_callbacks cb 0x560b22298270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46090/0 (socket says 172.21.15.46:46090) 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 -- 172.21.15.46:0/9057099 learned_addr learned my addr 172.21.15.46:0/9057099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652a00640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22350bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 -- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 msgr2=0x560b22350bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22350bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc653400640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b223653a0 0x560b2237b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 -- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b223653a0 msgr2=0x560b2237b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b223653a0 0x560b2237b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652000640 1 -- 172.21.15.46:0/9057099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2207e7d0 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.606+0000 7fc652a00640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22350bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc652000640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098a0 tx=0x7fc63c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8426993e73959aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc637e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c042040 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fc63c003a20 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63c02ac10 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc63c003c40 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc65c9eb740 10 client.7908 fetch_fsmap learned FSMap version 74 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.607+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x560b21fb43f0 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c02ac10 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fc63c005990 con 0x560b22369890 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 10 client.7908 fetch_fsmap finished waiting for FSMap version 74 2024-04-26T23:44:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 10 client.7908 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-26T23:44:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 20 client.7908 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22383a00 0x560b22383e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 4 client.7908 mds_command: new command op to 7695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:44:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 --> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b222b07e0 con 0x560b22383a00 2024-04-26T23:44:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.608+0000 7fc652a00640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22383a00 0x560b22383e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.609+0000 7fc652a00640 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22383a00 0x560b22383e60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5186d4a00fc8b079 server_cookie=dde268d77f1c08e1 in_seq=0 out_seq=0 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.609+0000 7fc637e00640 10 client.7908 ms_handle_connect on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.609+0000 7fc637e00640 1 -- 172.21.15.46:0/9057099 <== mds.0 v2:172.21.15.46:6833/2670265715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b222b07e0 con 0x560b22383a00 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.609+0000 7fc637e00640 10 client.7908 handle_command_reply: tid=0 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.609+0000 7fc65c9eb740 1 client.7908 shutdown 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 msgr2=0x560b2237b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098a0 tx=0x7fc63c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 shutdown_connections 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22383a00 0x560b22383e60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b223653a0 0x560b2237b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b22369890 0x560b2237b3c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/9057099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22350bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 >> 172.21.15.46:0/9057099 conn(0x560b22366af0 msgr2=0x560b223878d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc637e00640 0 client.7908 ms_handle_reset on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 shutdown_connections 2024-04-26T23:44:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 1 -- 172.21.15.46:0/9057099 wait complete. 2024-04-26T23:44:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.610+0000 7fc65c9eb740 20 client.7908 trim_cache size 0 max 16384 2024-04-26T23:44:55.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.626+0000 7fc65c9eb740 1 Processor -- start 2024-04-26T23:44:55.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 -- start start 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22384540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b22384a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b2236b100 con 0x560b223653a0 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541fe180 con 0x560b22369890 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.627+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6541fd1c0 con 0x560b22366ed0 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46104/0 (socket says 172.21.15.46:46104) 2024-04-26T23:44:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 -- 172.21.15.46:0/1400146654 learned_addr learned my addr 172.21.15.46:0/1400146654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652000640 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22384540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 -- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 msgr2=0x560b22384540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22384540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 -- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 msgr2=0x560b22384a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc653400640 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b22384a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b22384a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 -- 172.21.15.46:0/1400146654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22340f30 con 0x560b223653a0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.628+0000 7fc652a00640 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc64400ffc0 tx=0x7fc644012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983232d3ae931f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc642a00640 1 -- 172.21.15.46:0/1400146654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64402c070 con 0x560b223653a0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc642a00640 1 -- 172.21.15.46:0/1400146654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6440138f0 con 0x560b223653a0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc642a00640 1 -- 172.21.15.46:0/1400146654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644013ca0 con 0x560b223653a0 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 -- 172.21.15.46:0/1400146654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 msgr2=0x560b22386910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc64400ffc0 tx=0x7fc644012000 comp rx=0 tx=0).stop 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 -- 172.21.15.46:0/1400146654 shutdown_connections 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b22384a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22384540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/1400146654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b22386910 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.629+0000 7fc65c9eb740 1 -- 172.21.15.46:0/1400146654 >> 172.21.15.46:0/1400146654 conn(0x560b22366af0 msgr2=0x560b223509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.630+0000 7fc65c9eb740 1 -- 172.21.15.46:0/1400146654 shutdown_connections 2024-04-26T23:44:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.630+0000 7fc65c9eb740 1 -- 172.21.15.46:0/1400146654 wait complete. 2024-04-26T23:44:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.630+0000 7fc65c9eb740 1 -- start start 2024-04-26T23:44:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.630+0000 7fc65c9eb740 1 Processor -- start 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22408c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2234af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541fcb70 con 0x560b223653a0 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541fc520 con 0x560b22369890 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560b222affd0 con 0x560b22366ed0 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.631+0000 7fc65c9eb740 10 client.0 _ll_register_callbacks cb 0x7fc63800dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46118/0 (socket says 172.21.15.46:46118) 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 -- 172.21.15.46:0/2490299706 learned_addr learned my addr 172.21.15.46:0/2490299706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652000640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22408c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 -- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 msgr2=0x560b22408c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22408c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 -- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 msgr2=0x560b2234af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2234af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.632+0000 7fc652a00640 1 -- 172.21.15.46:0/2490299706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b21f740f0 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc652a00640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc644001eb0 tx=0x7fc644001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d017d9725b2dd88a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc637e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644035060 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fc6440262e0 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6440138f0 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc64403b050 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc65c9eb740 10 client.7914 fetch_fsmap learned FSMap version 74 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x560b2226aa30 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.633+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6440138f0 con 0x560b223653a0 2024-04-26T23:44:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fc64403e020 con 0x560b223653a0 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 10 client.7914 fetch_fsmap finished waiting for FSMap version 74 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 10 client.7914 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 20 client.7914 populate_metadata read hostname 'smithi046' 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22364510 0x560b2234b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 4 client.7914 mds_command: new command op to 7695 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 --> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b222aa650 con 0x560b22364510 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc652000640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22364510 0x560b2234b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc652000640 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22364510 0x560b2234b480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1731d72ab5838ce0 server_cookie=2059d5f55274dc4d in_seq=0 out_seq=0 2024-04-26T23:44:55.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.634+0000 7fc637e00640 10 client.7914 ms_handle_connect on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:55.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.656+0000 7fc637e00640 1 -- 172.21.15.46:0/2490299706 <== mds.0 v2:172.21.15.46:6833/2670265715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b222aa650 con 0x560b22364510 2024-04-26T23:44:55.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.656+0000 7fc637e00640 10 client.7914 handle_command_reply: tid=0 2024-04-26T23:44:55.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 client.7914 shutdown 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 msgr2=0x560b223843d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc644001eb0 tx=0x7fc644001ad0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 shutdown_connections 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:6833/2670265715,v1:172.21.15.46:6835/2670265715] conn(0x560b22364510 0x560b2234b480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b22369890 0x560b2234af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560b22366ed0 0x560b22408c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 --2- 172.21.15.46:0/2490299706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b223653a0 0x560b223843d0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc637e00640 0 client.7914 ms_handle_reset on v2:172.21.15.46:6833/2670265715 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 >> 172.21.15.46:0/2490299706 conn(0x560b22366af0 msgr2=0x560b22387450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 shutdown_connections 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 1 -- 172.21.15.46:0/2490299706 wait complete. 2024-04-26T23:44:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.657+0000 7fc65c9eb740 20 client.7914 trim_cache size 0 max 16384 2024-04-26T23:44:55.662 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:44:55.662 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-04-26T23:44:55.662 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:44:55.662 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 msgr2=0x7fc63006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 0x7fc63006edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc644001880 tx=0x7fc644009040 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 msgr2=0x7fc6540ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc65406faf0 tx=0x7fc64c0115a0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 shutdown_connections 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc65415e8b0 0x7fc6540ce960 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc65413c980 0x7fc65415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 --2- 172.21.15.46:0/373639565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638003f30 0x7fc6541f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 >> 172.21.15.46:0/373639565 conn(0x7fc65406c980 msgr2=0x7fc654143910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 shutdown_connections 2024-04-26T23:44:55.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.658+0000 7fc659c00640 1 -- 172.21.15.46:0/373639565 wait complete. 2024-04-26T23:44:55.681 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:44:55.681 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:55.681 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:44:55.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:55.682 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:44:55.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:55.682 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:44:55.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:44:55.701 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:44:55.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-26T23:44:55.705 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:44:55.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-26T23:44:55.709 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:44:55.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-26T23:44:55.714 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:44:55.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-26T23:44:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.930+0000 7ff1ae400640 1 Processor -- start 2024-04-26T23:44:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.931+0000 7ff1ae400640 1 -- start start 2024-04-26T23:44:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.931+0000 7fe559200640 1 Processor -- start 2024-04-26T23:44:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7fe559200640 1 -- start start 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ae400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 0x7ff1a80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ae400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 0x7ff1a81637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ae400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a80b7e60 con 0x7ff1a80d1910 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ae400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a80b8440 con 0x7ff1a80d0740 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ae400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a80d0c70 con 0x7ff1a8163320 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ad000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1a80d1910 0x7ff1a80d1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55730/0 (socket says 172.21.15.46:55730) 2024-04-26T23:44:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.932+0000 7ff1ad000640 1 -- 172.21.15.46:0/1611453347 learned_addr learned my addr 172.21.15.46:0/1611453347 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7ff1ada00640 1 --2- 172.21.15.46:0/1611453347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 0x7ff1a81637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7ff1a7e00640 1 --2- 172.21.15.46:0/1611453347 >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 0x7ff1a80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7ff1a7400640 1 -- 172.21.15.46:0/1611453347 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403198017 0 0) 0x7ff1a80b7e60 con 0x7ff1a80d1910 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7ff1a7400640 1 -- 172.21.15.46:0/1611453347 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff190003520 con 0x7ff1a80d1910 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7fe559200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5540d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7fe559200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540d2e80 con 0x7fe5540d0d20 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.933+0000 7fe559200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540d3140 con 0x7fe55415f3b0 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7fe559200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540d3420 con 0x7fe5540d21a0 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7ff1ada00640 1 -- 172.21.15.46:0/1611453347 >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 msgr2=0x7ff1a80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7ff1ada00640 1 --2- 172.21.15.46:0/1611453347 >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 0x7ff1a80d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7ff1ada00640 1 -- 172.21.15.46:0/1611453347 >> v1:172.21.15.46:6789/0 conn(0x7ff1a80d1910 legacy=0x7ff1a80d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7ff1ada00640 1 -- 172.21.15.46:0/1611453347 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a80b80d0 con 0x7ff1a8163320 2024-04-26T23:44:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7fe552800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe5540d0d20 0x7fe5541692d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55736/0 (socket says 172.21.15.46:55736) 2024-04-26T23:44:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7fe552800640 1 -- 172.21.15.46:0/3318369017 learned_addr learned my addr 172.21.15.46:0/3318369017 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7ff1a7e00640 1 --2- 172.21.15.46:0/1611453347 >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 0x7ff1a80d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:44:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7fe551400640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5540d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.934+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667727076 0 0) 0x7fe5540d2e80 con 0x7fe5540d0d20 2024-04-26T23:44:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe538003520 con 0x7fe5540d0d20 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3942598848 0 0) 0x7fe538003520 con 0x7fe5540d0d20 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 msgr2=0x7fe5540d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5540d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 >> v1:172.21.15.142:6789/0 conn(0x7fe55415f3b0 legacy=0x7fe5540d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540d3700 con 0x7fe5540d0d20 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe54c0019f0 con 0x7fe5540d0d20 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 >> v1:172.21.15.46:6789/0 conn(0x7fe5540d0d20 legacy=0x7fe5541692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5380039e0 0x7fe538003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe538004800 0x7fe538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d3420 con 0x7fe5380039e0 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe53800ba70 con 0x7fe538004800 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe53800bc70 con 0x7fe5540d21a0 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.935+0000 7fe551e00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5380039e0 0x7fe538003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551400640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe538004800 0x7fe538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7ff1ada00640 1 --2- 172.21.15.46:0/1611453347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 0x7ff1a81637d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009d90 tx=0x7ff1a002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9809eb4706be88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe552800640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551e00640 1 -- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 msgr2=0x7fe5380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551e00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551e00640 1 -- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe538004800 msgr2=0x7fe538004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551e00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe538004800 0x7fe538004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe551e00640 1 -- 172.21.15.46:0/3318369017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540d3700 con 0x7fe5380039e0 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.936+0000 7fe552800640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540d21a0 0x7fe5380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7ff1a7400640 1 -- 172.21.15.46:0/1611453347 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0007740 con 0x7ff1a8163320 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7ff1a7400640 1 -- 172.21.15.46:0/1611453347 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a00078e0 con 0x7ff1a8163320 2024-04-26T23:44:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7ff1a7400640 1 -- 172.21.15.46:0/1611453347 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0007c90 con 0x7ff1a8163320 2024-04-26T23:44:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7fe551e00640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5380039e0 0x7fe538003e00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe544001720 tx=0x7fe538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f943b4b6330f7213 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7ff1ae400640 1 -- 172.21.15.46:0/1611453347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 msgr2=0x7ff1a81637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7ff1ae400640 1 --2- 172.21.15.46:0/1611453347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 0x7ff1a81637d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009d90 tx=0x7ff1a002a660 comp rx=0 tx=0).stop 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.937+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544001370 con 0x7fe5380039e0 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5440052d0 con 0x7fe5380039e0 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7fe550a00640 1 -- 172.21.15.46:0/3318369017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54402ab90 con 0x7fe5380039e0 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 -- 172.21.15.46:0/1611453347 shutdown_connections 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 --2- 172.21.15.46:0/1611453347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a8163320 0x7ff1a81637d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 --2- 172.21.15.46:0/1611453347 >> v2:172.21.15.142:3300/0 conn(0x7ff1a80d0740 0x7ff1a80d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 -- 172.21.15.46:0/1611453347 >> 172.21.15.46:0/1611453347 conn(0x7ff1a80d4f70 msgr2=0x7ff1a80d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 -- 172.21.15.46:0/1611453347 shutdown_connections 2024-04-26T23:44:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7ff1ae400640 1 -- 172.21.15.46:0/1611453347 wait complete. 2024-04-26T23:44:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7fe559200640 1 -- 172.21.15.46:0/3318369017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5380039e0 msgr2=0x7fe538003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7fe559200640 1 --2- 172.21.15.46:0/3318369017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5380039e0 0x7fe538003e00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe544001720 tx=0x7fe538005830 comp rx=0 tx=0).stop 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.938+0000 7fe552800640 1 -- 172.21.15.46:0/3318369017 reap_dead start 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.939+0000 7fe559200640 1 -- 172.21.15.46:0/3318369017 shutdown_connections 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.939+0000 7fe559200640 1 -- 172.21.15.46:0/3318369017 >> 172.21.15.46:0/3318369017 conn(0x7fe55409e450 msgr2=0x7fe5540d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.939+0000 7fe559200640 1 -- 172.21.15.46:0/3318369017 shutdown_connections 2024-04-26T23:44:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.939+0000 7fe559200640 1 -- 172.21.15.46:0/3318369017 wait complete. 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.940+0000 7ff1ae400640 1 Processor -- start 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.940+0000 7ff1ae400640 1 -- start start 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 0x7ff1a80ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a80d1910 0x7ff1a815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8163e90 con 0x7ff1a8163320 2024-04-26T23:44:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8163d10 con 0x7ff1a80d0740 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ae400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a8164010 con 0x7ff1a80d1910 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7ff1ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.941+0000 7fe559200640 1 Processor -- start 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 0x7ff1a80ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 0x7ff1a80ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:34142/0 (socket says 172.21.15.46:34142) 2024-04-26T23:44:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ad000640 1 -- 172.21.15.46:0/419583336 learned_addr learned my addr 172.21.15.46:0/419583336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1a7e00640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a80d1910 0x7ff1a815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7fe559200640 1 -- start start 2024-04-26T23:44:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46124/0 (socket says 172.21.15.46:46124) 2024-04-26T23:44:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 -- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a80d1910 msgr2=0x7ff1a815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a80d1910 0x7ff1a815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 -- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 msgr2=0x7ff1a80ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 0x7ff1a80ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7ff1ada00640 1 -- 172.21.15.46:0/419583336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a80ceaa0 con 0x7ff1a8163320 2024-04-26T23:44:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.942+0000 7f49ea400640 1 Processor -- start 2024-04-26T23:44:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7f49ea400640 1 -- start start 2024-04-26T23:44:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5380039e0 0x7fe5541537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7ff1ada00640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff1a808e1b0 tx=0x7ff1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 0x7fe5540b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe55415f790 con 0x7fe554153fb0 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d3f00 con 0x7fe5380039e0 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe559200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540b97d0 con 0x7fe5540b8290 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe551400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe551400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46132/0 (socket says 172.21.15.46:46132) 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe551400640 1 -- 172.21.15.46:0/4263795987 learned_addr learned my addr 172.21.15.46:0/4263795987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7fe552800640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 0x7fe5540b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.943+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0009450 con 0x7ff1a8163320 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a0004240 con 0x7ff1a8163320 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004610 con 0x7ff1a8163320 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a80cd590 con 0x7ff1a8163320 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551e00640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5380039e0 0x7fe5541537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f05afe00640 1 Processor -- start 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a80cdb50 con 0x7ff1a8163320 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551400640 1 -- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 msgr2=0x7fe5540b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551400640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 0x7fe5540b9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551400640 1 -- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5380039e0 msgr2=0x7fe5541537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551400640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5380039e0 0x7fe5541537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7fe551400640 1 -- 172.21.15.46:0/4263795987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540b8730 con 0x7fe554153fb0 2024-04-26T23:44:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7fe552800640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 0x7fe5540b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f49ea400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e40d21a0 0x7f49e40d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f49ea400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e41692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f49ea400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e40d2e80 con 0x7f49e415f3b0 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f49ea400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f49e40d11b0 con 0x7f49e40d21a0 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f49ea400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49e40d3140 con 0x7f49e40d0d20 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e40d21a0 0x7f49e40d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e40d21a0 0x7f49e40d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46142/0 (socket says 172.21.15.46:46142) 2024-04-26T23:44:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 -- 172.21.15.46:0/2151769201 learned_addr learned my addr 172.21.15.46:0/2151769201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.944+0000 7f05afe00640 1 -- start start 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f05afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05a815f3b0 0x7f05a80d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f05afe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d1fd0 0x7f05a80d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f05afe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a80d2e80 con 0x7f05a80d0d20 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f05afe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f05a80d11b0 con 0x7f05a815f3b0 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f05afe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05a80d3140 con 0x7f05a80d1fd0 2024-04-26T23:44:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 -- 172.21.15.46:0/2151769201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 msgr2=0x7f49e41692d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 --2- 172.21.15.46:0/2151769201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e41692d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 -- 172.21.15.46:0/2151769201 >> v1:172.21.15.142:6789/0 conn(0x7f49e415f3b0 legacy=0x7f49e40d1a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:44:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.945+0000 7f49e2a00640 1 -- 172.21.15.46:0/2151769201 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e40d3420 con 0x7f49e40d21a0 2024-04-26T23:44:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7f05aca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d1fd0 0x7f05a80d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7fe551400640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe540011600 tx=0x7fe540011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a80693b0 con 0x7ff1a8163320 2024-04-26T23:44:55.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7f05ade00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f05a80d0d20 0x7f05a8162fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:60468/0 (socket says 172.21.15.46:60468) 2024-04-26T23:44:55.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7f05ade00640 1 -- 172.21.15.46:0/912440451 learned_addr learned my addr 172.21.15.46:0/912440451 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:55.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.946+0000 7f05ad400640 1 --2- 172.21.15.46:0/912440451 >> v2:172.21.15.46:3300/0 conn(0x7f05a815f3b0 0x7f05a80d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54000f920 con 0x7fe554153fb0 2024-04-26T23:44:55.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120467873 0 0) 0x7f05a80d2e80 con 0x7f05a80d0d20 2024-04-26T23:44:55.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0590003520 con 0x7f05a80d0d20 2024-04-26T23:44:55.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54000fac0 con 0x7fe554153fb0 2024-04-26T23:44:55.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05ad400640 1 -- 172.21.15.46:0/912440451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d1fd0 msgr2=0x7f05a80d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:55.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2960908412 0 0) 0x7f0590003520 con 0x7f05a80d0d20 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54000fe70 con 0x7fe554153fb0 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5540b8a10 con 0x7fe554153fb0 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5540b8fd0 con 0x7fe554153fb0 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05ad400640 1 --2- 172.21.15.46:0/912440451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d1fd0 0x7f05a80d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05ad400640 1 -- 172.21.15.46:0/912440451 >> v1:172.21.15.142:6789/0 conn(0x7f05a80d0d20 legacy=0x7f05a8162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f05ad400640 1 -- 172.21.15.46:0/912440451 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a80d3420 con 0x7f05a815f3b0 2024-04-26T23:44:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.947+0000 7f49e2a00640 1 --2- 172.21.15.46:0/2151769201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e40d21a0 0x7f49e40d07e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e038777eb57bfe32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.948+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1a0039040 con 0x7ff1a8163320 2024-04-26T23:44:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.949+0000 7f05ad400640 1 --2- 172.21.15.46:0/912440451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a815f3b0 0x7f05a80d1a90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f059c0097a0 tx=0x7f059c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf3e71843c576a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.949+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5540693b0 con 0x7fe554153fb0 2024-04-26T23:44:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.949+0000 7ff1a4c00640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 0x7ff18806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.949+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7ff1a00acf60 con 0x7ff1a8163320 2024-04-26T23:44:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.950+0000 7ff1ad000640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 0x7ff18806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.950+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe540015460 con 0x7fe554153fb0 2024-04-26T23:44:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.951+0000 7ff1ad000640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 0x7ff18806ee80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005880 tx=0x7ff19c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.951+0000 7fe54a000640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 0x7fe53406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.952+0000 7fe551e00640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 0x7fe53406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.953+0000 7fe551e00640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 0x7fe53406ee00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe544032000 tx=0x7fe54402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.954+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6101+0+0 (secure 0 0 0) 0x7fe54008a8c0 con 0x7fe554153fb0 2024-04-26T23:44:55.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.954+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c037070 con 0x7f05a815f3b0 2024-04-26T23:44:55.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.954+0000 7f49e2000640 1 -- 172.21.15.46:0/2151769201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0037070 con 0x7f49e40d21a0 2024-04-26T23:44:55.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.955+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1a0040030 con 0x7ff1a8163320 2024-04-26T23:44:55.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.959+0000 7f49e2000640 1 -- 172.21.15.46:0/2151769201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0032440 con 0x7f49e40d21a0 2024-04-26T23:44:55.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.959+0000 7f49e2000640 1 -- 172.21.15.46:0/2151769201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0032810 con 0x7f49e40d21a0 2024-04-26T23:44:55.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe540019030 con 0x7fe554153fb0 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 -- 172.21.15.46:0/2151769201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e40d21a0 msgr2=0x7f49e40d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 --2- 172.21.15.46:0/2151769201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e40d21a0 0x7f49e40d07e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d002a230 comp rx=0 tx=0).stop 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 -- 172.21.15.46:0/2151769201 shutdown_connections 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 --2- 172.21.15.46:0/2151769201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e41692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 --2- 172.21.15.46:0/2151769201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e40d21a0 0x7f49e40d07e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 -- 172.21.15.46:0/2151769201 >> 172.21.15.46:0/2151769201 conn(0x7f49e409e450 msgr2=0x7f49e40d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.960+0000 7f49ea400640 1 -- 172.21.15.46:0/2151769201 shutdown_connections 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.961+0000 7f49ea400640 1 -- 172.21.15.46:0/2151769201 wait complete. 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.962+0000 7f49ea400640 1 Processor -- start 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.963+0000 7f49ea400640 1 -- start start 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40d21a0 0x7f49e40b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49e415f790 con 0x7f49e415f3b0 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40d39e0 con 0x7f49e40d21a0 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49ea400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49e40ba710 con 0x7f49e40d0d20 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46156/0 (socket says 172.21.15.46:46156) 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49e3e00640 1 -- 172.21.15.46:0/1884035253 learned_addr learned my addr 172.21.15.46:0/1884035253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49e3400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.964+0000 7f49e2a00640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40d21a0 0x7f49e40b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 -- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 msgr2=0x7f49e414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 -- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40d21a0 msgr2=0x7f49e40b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40d21a0 0x7f49e40b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 -- 172.21.15.46:0/1884035253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e40b5e80 con 0x7f49e415f3b0 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:44:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.965+0000 7f49e3e00640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b990 tx=0x7f49dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc019070 con 0x7f49e415f3b0 2024-04-26T23:44:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c02aba0 con 0x7f05a815f3b0 2024-04-26T23:44:55.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f05a7e00640 1 -- 172.21.15.46:0/912440451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00019f0 con 0x7f05a815f3b0 2024-04-26T23:44:55.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc00cba0 con 0x7f49e415f3b0 2024-04-26T23:44:55.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc014440 con 0x7f49e415f3b0 2024-04-26T23:44:55.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e40b72d0 con 0x7f49e415f3b0 2024-04-26T23:44:55.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.968+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e40b7890 con 0x7f49e415f3b0 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.969+0000 7f05afe00640 1 -- 172.21.15.46:0/912440451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a815f3b0 msgr2=0x7f05a80d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.969+0000 7f05afe00640 1 --2- 172.21.15.46:0/912440451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a815f3b0 0x7f05a80d1a90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f059c0097a0 tx=0x7f059c02a110 comp rx=0 tx=0).stop 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.969+0000 7f05afe00640 1 -- 172.21.15.46:0/912440451 shutdown_connections 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.969+0000 7f05afe00640 1 --2- 172.21.15.46:0/912440451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d1fd0 0x7f05a80d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.970+0000 7f05afe00640 1 --2- 172.21.15.46:0/912440451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a815f3b0 0x7f05a80d1a90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.970+0000 7f05afe00640 1 -- 172.21.15.46:0/912440451 >> 172.21.15.46:0/912440451 conn(0x7f05a809e450 msgr2=0x7f05a80d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.970+0000 7f05afe00640 1 -- 172.21.15.46:0/912440451 shutdown_connections 2024-04-26T23:44:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.970+0000 7f05afe00640 1 -- 172.21.15.46:0/912440451 wait complete. 2024-04-26T23:44:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.970+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e40693b0 con 0x7f49e415f3b0 2024-04-26T23:44:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.971+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49dc01c070 con 0x7f49e415f3b0 2024-04-26T23:44:55.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.972+0000 7f49db400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 0x7f49c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.972+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6217+0+0 (secure 0 0 0) 0x7f49dc08bb90 con 0x7f49e415f3b0 2024-04-26T23:44:55.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.973+0000 7f05afe00640 1 Processor -- start 2024-04-26T23:44:55.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.973+0000 7f05afe00640 1 -- start start 2024-04-26T23:44:55.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.973+0000 7f49e3400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 0x7f49c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.974+0000 7f05afe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d0d20 0x7f05a8148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f49e3400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 0x7f49c406edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006fd0 tx=0x7f49d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05afe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a815f3b0 0x7f05a81f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05afe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05a815f790 con 0x7f05a80d1fd0 2024-04-26T23:44:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05afe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80d39e0 con 0x7f05a815f3b0 2024-04-26T23:44:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05afe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05a80b6d70 con 0x7f05a80d0d20 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46170/0 (socket says 172.21.15.46:46170) 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05aca00640 1 -- 172.21.15.46:0/870983317 learned_addr learned my addr 172.21.15.46:0/870983317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05ad400640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d0d20 0x7f05a8148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.975+0000 7f05ade00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a815f3b0 0x7f05a81f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 -- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d0d20 msgr2=0x7f05a8148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d0d20 0x7f05a8148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 -- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a815f3b0 msgr2=0x7f05a81f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a815f3b0 0x7f05a81f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 -- 172.21.15.46:0/870983317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81492e0 con 0x7f05a80d1fd0 2024-04-26T23:44:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.976+0000 7f05aca00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f059400e870 tx=0x7f059400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594017070 con 0x7f05a80d1fd0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059400c940 con 0x7f05a80d1fd0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059400cd30 con 0x7f05a80d1fd0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a80b6160 con 0x7f05a80d1fd0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a80b6720 con 0x7f05a80d1fd0 2024-04-26T23:44:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.983+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49dc059d20 con 0x7f49e415f3b0 2024-04-26T23:44:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.985+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584005290 con 0x7f05a80d1fd0 2024-04-26T23:44:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.987+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f059401a070 con 0x7f05a80d1fd0 2024-04-26T23:44:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.988+0000 7f05a5600640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.989+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6217+0+0 (secure 0 0 0) 0x7f059408bb50 con 0x7f05a80d1fd0 2024-04-26T23:44:55.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.989+0000 7f05ad400640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.991+0000 7f05ad400640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 0x7f058806ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f059c034000 tx=0x7f059c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:56.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:55.998+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05a81492e0 con 0x7f05a80d1fd0 2024-04-26T23:44:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.246+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7ff1a80b9bb0 con 0x7ff1a8163320 2024-04-26T23:44:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.255+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fe5540d2e80 con 0x7fe554153fb0 2024-04-26T23:44:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.268+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f49e40d29f0 con 0x7f49e415f3b0 2024-04-26T23:44:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.279+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f0584005580 con 0x7f05a80d1fd0 2024-04-26T23:44:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.988+0000 7ff1a4c00640 1 -- 172.21.15.46:0/419583336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7692 v75) ==== 95+0+0 (secure 0 0 0) 0x7ff1a007ab00 con 0x7ff1a8163320 2024-04-26T23:44:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 7692 2024-04-26T23:44:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.991+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 msgr2=0x7ff18806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.991+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 0x7ff18806ee80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005880 tx=0x7ff19c009040 comp rx=0 tx=0).stop 2024-04-26T23:44:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.991+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 msgr2=0x7ff1a80c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.991+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff1a808e1b0 tx=0x7ff1a0004a40 comp rx=0 tx=0).stop 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 shutdown_connections 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff18806c9c0 0x7ff18806ee80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a8163320 0x7ff1a80c8ec0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1a80d1910 0x7ff1a815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 --2- 172.21.15.46:0/419583336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1a80d0740 0x7ff1a80ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 >> 172.21.15.46:0/419583336 conn(0x7ff1a80d4f70 msgr2=0x7ff1a80b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 shutdown_connections 2024-04-26T23:44:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.992+0000 7ff1ae400640 1 -- 172.21.15.46:0/419583336 wait complete. 2024-04-26T23:44:56.996 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:44:56.993+0000 7f074f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:44:57.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:56.996+0000 7fe54a000640 1 -- 172.21.15.46:0/4263795987 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7695 v76) ==== 95+0+0 (secure 0 0 0) 0x7fe54005a0f0 con 0x7fe554153fb0 2024-04-26T23:44:57.000 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 7695 2024-04-26T23:44:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 msgr2=0x7fe53406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 0x7fe53406ee00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe544032000 tx=0x7fe54402e000 comp rx=0 tx=0).stop 2024-04-26T23:44:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 msgr2=0x7fe5540b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe540011600 tx=0x7fe540011ac0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 shutdown_connections 2024-04-26T23:44:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe53406c940 0x7fe53406ee00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5540b8290 0x7fe5540b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe554153fb0 0x7fe5540b7d50 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 --2- 172.21.15.46:0/4263795987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5380039e0 0x7fe5541537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.000+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 >> 172.21.15.46:0/4263795987 conn(0x7fe55409e450 msgr2=0x7fe55415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.001+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 shutdown_connections 2024-04-26T23:44:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.002+0000 7fe559200640 1 -- 172.21.15.46:0/4263795987 wait complete. 2024-04-26T23:44:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.011+0000 7f49db400640 1 -- 172.21.15.46:0/1884035253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5537 v77) ==== 95+0+0 (secure 0 0 0) 0x7f49dc059620 con 0x7f49e415f3b0 2024-04-26T23:44:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 5537 2024-04-26T23:44:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.016+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 msgr2=0x7f49c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.016+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 0x7f49c406edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006fd0 tx=0x7f49d4008040 comp rx=0 tx=0).stop 2024-04-26T23:44:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.016+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 msgr2=0x7f49e41625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.016+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b990 tx=0x7f49dc00be50 comp rx=0 tx=0).stop 2024-04-26T23:44:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 shutdown_connections 2024-04-26T23:44:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49c406c8f0 0x7f49c406edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e415f3b0 0x7f49e41625a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40d21a0 0x7f49e40b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 --2- 172.21.15.46:0/1884035253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49e40d0d20 0x7f49e414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.017+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 >> 172.21.15.46:0/1884035253 conn(0x7f49e409e450 msgr2=0x7f49e40d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.019+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 shutdown_connections 2024-04-26T23:44:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.019+0000 7f49ea400640 1 -- 172.21.15.46:0/1884035253 wait complete. 2024-04-26T23:44:57.024 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:44:57.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:44:57.029 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:44:57.029 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:57.033 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:44:57.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:44:57.042 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:44:57.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:44:57.050 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:44:57.050 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:57.086 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:44:57.086 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:57.171 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:44:57.205 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:44:57.226 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:44:57.990 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:44:57.986+0000 7f074f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:44:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.992+0000 7f05a5600640 1 -- 172.21.15.46:0/870983317 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5534 v78) ==== 95+0+0 (secure 0 0 0) 0x7f0594059d70 con 0x7f05a80d1fd0 2024-04-26T23:44:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:failed mds gid 5534 2024-04-26T23:44:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 msgr2=0x7f058806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 0x7f058806ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f059c034000 tx=0x7f059c02e000 comp rx=0 tx=0).stop 2024-04-26T23:44:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 msgr2=0x7f05a815fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f059400e870 tx=0x7f059400ed30 comp rx=0 tx=0).stop 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 shutdown_connections 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a815f3b0 0x7f05a81f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05a80d1fd0 0x7f05a815fd10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 --2- 172.21.15.46:0/870983317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a80d0d20 0x7f05a8148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.996+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 >> 172.21.15.46:0/870983317 conn(0x7f05a809e450 msgr2=0x7f05a80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.997+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 shutdown_connections 2024-04-26T23:44:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:57.997+0000 7f05afe00640 1 -- 172.21.15.46:0/870983317 wait complete. 2024-04-26T23:44:58.026 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:44:58.026 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:44:58.036 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:44:58.036 DEBUG:teuthology.parallel:result is None 2024-04-26T23:44:58.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:44:58.193 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.261+0000 7f832f200640 1 Processor -- start 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.261+0000 7f832f200640 1 -- start start 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.262+0000 7f832f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f833015d080 0x7f83300cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.262+0000 7f832f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f83300ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.262+0000 7f832f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83300d3c30 con 0x7f83300d0be0 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.262+0000 7f832f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f83300cdb40 con 0x7f833015d080 2024-04-26T23:44:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.262+0000 7f832f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83300cdde0 con 0x7f83300d1a60 2024-04-26T23:44:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.263+0000 7f832de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f833015d080 0x7f83300cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.263+0000 7f832d400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f83300d0be0 0x7f83300d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:60484/0 (socket says 172.21.15.46:60484) 2024-04-26T23:44:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.263+0000 7f832d400640 1 -- 172.21.15.46:0/2901729892 learned_addr learned my addr 172.21.15.46:0/2901729892 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:44:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.263+0000 7f832de00640 1 --2- 172.21.15.46:0/2901729892 >> v2:172.21.15.142:3300/0 conn(0x7f833015d080 0x7f83300cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:34206/0 (socket says 172.21.15.46:34206) 2024-04-26T23:44:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.263+0000 7f832e800640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f83300ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.264+0000 7f832de00640 1 -- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 msgr2=0x7f83300ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.264+0000 7f832de00640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f83300ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.264+0000 7f832de00640 1 -- 172.21.15.46:0/2901729892 >> v1:172.21.15.142:6789/0 conn(0x7f83300d0be0 legacy=0x7f83300d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.264+0000 7f832de00640 1 -- 172.21.15.46:0/2901729892 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83300ce0c0 con 0x7f833015d080 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.265+0000 7f832de00640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83300cfdb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f83200097a0 tx=0x7f832002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8a4541956b393f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.265+0000 7f832ca00640 1 -- 172.21.15.46:0/2901729892 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320037070 con 0x7f833015d080 2024-04-26T23:44:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.265+0000 7f832ca00640 1 -- 172.21.15.46:0/2901729892 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8320032440 con 0x7f833015d080 2024-04-26T23:44:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.265+0000 7f832ca00640 1 -- 172.21.15.46:0/2901729892 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320032810 con 0x7f833015d080 2024-04-26T23:44:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 -- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 msgr2=0x7f83300cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83300cfdb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f83200097a0 tx=0x7f832002a210 comp rx=0 tx=0).stop 2024-04-26T23:44:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 -- 172.21.15.46:0/2901729892 shutdown_connections 2024-04-26T23:44:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f83300ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 --2- 172.21.15.46:0/2901729892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83300cfdb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 -- 172.21.15.46:0/2901729892 >> 172.21.15.46:0/2901729892 conn(0x7f83300d2d70 msgr2=0x7f83300d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 -- 172.21.15.46:0/2901729892 shutdown_connections 2024-04-26T23:44:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.266+0000 7f832f200640 1 -- 172.21.15.46:0/2901729892 wait complete. 2024-04-26T23:44:58.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.268+0000 7f832f200640 1 Processor -- start 2024-04-26T23:44:58.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.268+0000 7f832f200640 1 -- start start 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f8330159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83301595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83300d3a10 con 0x7f83300d0be0 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83300d3890 con 0x7f833015d080 2024-04-26T23:44:58.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f833015d460 con 0x7f83300d1a60 2024-04-26T23:44:58.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46216/0 (socket says 172.21.15.46:46216) 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.269+0000 7f832de00640 1 -- 172.21.15.46:0/2320460729 learned_addr learned my addr 172.21.15.46:0/2320460729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832e800640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83301595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832d400640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f8330159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 -- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 msgr2=0x7f8330159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f8330159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 -- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 msgr2=0x7f83301595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83301595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 -- 172.21.15.46:0/2320460729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83301f5270 con 0x7f83300d0be0 2024-04-26T23:44:58.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.270+0000 7f832de00640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f832002a800 tx=0x7f83200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.271+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320037050 con 0x7f83300d0be0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.271+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8320043070 con 0x7f83300d0be0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.271+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832003cb30 con 0x7f83300d0be0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.271+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83301f5510 con 0x7f83300d0be0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.271+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83301f5ad0 con 0x7f83300d0be0 2024-04-26T23:44:58.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.272+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8330069ac0 con 0x7f83300d0be0 2024-04-26T23:44:58.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.273+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f832003ccd0 con 0x7f83300d0be0 2024-04-26T23:44:58.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.275+0000 7f8326000640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 0x7f831006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.275+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f83200acb00 con 0x7f83300d0be0 2024-04-26T23:44:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.276+0000 7f832d400640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 0x7f831006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.277+0000 7f832d400640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 0x7f831006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f833015c590 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.279+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f832007ad00 con 0x7f83300d0be0 2024-04-26T23:44:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.523+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83300cef80 con 0x7f83300d0be0 2024-04-26T23:44:58.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.524+0000 7f8326000640 1 -- 172.21.15.46:0/2320460729 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+4350 (secure 0 0 0) 0x7f832007a600 con 0x7f83300d0be0 2024-04-26T23:44:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:44:58.528 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:44:57.998897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T23:44:58.528 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 79 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.527+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 msgr2=0x7f831006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.527+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 0x7f831006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f833015c590 tx=0x7f831c009040 comp rx=0 tx=0).stop 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.527+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 msgr2=0x7f833015b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.527+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f832002a800 tx=0x7f83200046c0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 shutdown_connections 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831006c8f0 0x7f831006edb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833015d080 0x7f83301595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83300d1a60 0x7f8330159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 --2- 172.21.15.46:0/2320460729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83300d0be0 0x7f833015b340 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 >> 172.21.15.46:0/2320460729 conn(0x7f83300d2d70 msgr2=0x7f83300700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 shutdown_connections 2024-04-26T23:44:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:44:58.528+0000 7f832f200640 1 -- 172.21.15.46:0/2320460729 wait complete. 2024-04-26T23:44:58.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:44:42.305207+0000', 'modified': '2024-04-26T23:44:57.998897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/2779258602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2779258602}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2779258602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:44:58.952 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7959:up:replay 2024-04-26T23:44:59.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:00.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.171+0000 7f0f4e800640 1 Processor -- start 2024-04-26T23:45:00.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.171+0000 7f0f4e800640 1 -- start start 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 0x7f0f480d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4e800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f480cfe00 con 0x7f0f480d5a30 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4815f870 con 0x7f0f480d0e60 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f480d09d0 con 0x7f0f480d1920 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f480d5a30 0x7f0f480b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55760/0 (socket says 172.21.15.46:55760) 2024-04-26T23:45:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.173+0000 7f0f4de00640 1 -- 172.21.15.46:0/2296119724 learned_addr learned my addr 172.21.15.46:0/2296119724 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.174+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 0x7f0f480d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.174+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260030517 0 0) 0x7f0f480cfe00 con 0x7f0f480d5a30 2024-04-26T23:45:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.174+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f30003520 con 0x7f0f480d5a30 2024-04-26T23:45:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1979307070 0 0) 0x7f0f30003520 con 0x7f0f480d5a30 2024-04-26T23:45:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 msgr2=0x7f0f480d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 0x7f0f480d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 >> v1:172.21.15.142:6789/0 conn(0x7f0f480d0e60 legacy=0x7f0f480d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f480d00a0 con 0x7f0f480d5a30 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0f400019f0 con 0x7f0f480d5a30 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 >> v1:172.21.15.46:6789/0 conn(0x7f0f480d5a30 legacy=0x7f0f480b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300039e0 0x7f0f30003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f30004800 0x7f0f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 0x7f0f300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480d09d0 con 0x7f0f300039e0 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000ba70 con 0x7f0f30004800 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3000bc20 con 0x7f0f480d1920 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300039e0 0x7f0f30003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f4d400640 1 -- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 msgr2=0x7f0f300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480d1920 0x7f0f300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f4d400640 1 -- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f30004800 msgr2=0x7f0f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.175+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f30004800 0x7f0f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.176+0000 7f0f4d400640 1 -- 172.21.15.46:0/2296119724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f30003520 con 0x7f0f300039e0 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300039e0 0x7f0f30003dc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009a90 tx=0x7f0f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a97ab872ca9c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c036070 con 0x7f0f300039e0 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c031400 con 0x7f0f300039e0 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f47e00640 1 -- 172.21.15.46:0/2296119724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c009690 con 0x7f0f300039e0 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f4e800640 1 -- 172.21.15.46:0/2296119724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300039e0 msgr2=0x7f0f30003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2296119724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300039e0 0x7f0f30003dc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009a90 tx=0x7f0f30005830 comp rx=0 tx=0).stop 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f4de00640 1 -- 172.21.15.46:0/2296119724 reap_dead start 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.177+0000 7f0f4e800640 1 -- 172.21.15.46:0/2296119724 shutdown_connections 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.178+0000 7f0f4e800640 1 -- 172.21.15.46:0/2296119724 >> 172.21.15.46:0/2296119724 conn(0x7f0f480d4f10 msgr2=0x7f0f480d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.178+0000 7f0f4e800640 1 -- 172.21.15.46:0/2296119724 shutdown_connections 2024-04-26T23:45:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.178+0000 7f0f4e800640 1 -- 172.21.15.46:0/2296119724 wait complete. 2024-04-26T23:45:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.179+0000 7f0f4e800640 1 Processor -- start 2024-04-26T23:45:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.180+0000 7f0f4e800640 1 -- start start 2024-04-26T23:45:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.180+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f300039e0 0x7f0f481f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480c9450 0x7f0f481ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480d2cf0 con 0x7f0f481eadf0 2024-04-26T23:45:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480d0660 con 0x7f0f300039e0 2024-04-26T23:45:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f480d2e70 con 0x7f0f480c9450 2024-04-26T23:45:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46244/0 (socket says 172.21.15.46:46244) 2024-04-26T23:45:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 -- 172.21.15.46:0/2733566858 learned_addr learned my addr 172.21.15.46:0/2733566858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f300039e0 0x7f0f481f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4ca00640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480c9450 0x7f0f481ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 -- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480c9450 msgr2=0x7f0f481ea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480c9450 0x7f0f481ea8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 -- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f300039e0 msgr2=0x7f0f481f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.181+0000 7f0f4de00640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f300039e0 0x7f0f481f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f4de00640 1 -- 172.21.15.46:0/2733566858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f480b7b30 con 0x7f0f481eadf0 2024-04-26T23:45:00.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f4de00640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0f480d4e70 tx=0x7f0f4000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000a5b0 con 0x7f0f481eadf0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4000a770 con 0x7f0f481eadf0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000ab60 con 0x7f0f481eadf0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f480b7e10 con 0x7f0f481eadf0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.182+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f480b83d0 con 0x7f0f481eadf0 2024-04-26T23:45:00.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.183+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4808d980 con 0x7f0f481eadf0 2024-04-26T23:45:00.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.185+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f4000ad00 con 0x7f0f481eadf0 2024-04-26T23:45:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.186+0000 7f0f45600640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 0x7f0f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.186+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f0f4008b490 con 0x7f0f481eadf0 2024-04-26T23:45:00.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.187+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 0x7f0f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.188+0000 7f0f4d400640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 0x7f0f2806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c033000 tx=0x7f0f3c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.191+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f4808d980 con 0x7f0f481eadf0 2024-04-26T23:45:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.437+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f480693b0 con 0x7f0f481eadf0 2024-04-26T23:45:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.439+0000 7f0f45600640 1 -- 172.21.15.46:0/2733566858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5178 (secure 0 0 0) 0x7f0f4001a030 con 0x7f0f481eadf0 2024-04-26T23:45:00.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:00.443 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:44:59.019803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T23:45:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 81 2024-04-26T23:45:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 msgr2=0x7f0f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 0x7f0f2806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c033000 tx=0x7f0f3c008030 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 msgr2=0x7f0f481f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0f480d4e70 tx=0x7f0f4000cf00 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 shutdown_connections 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f2806c8f0 0x7f0f2806edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f481eadf0 0x7f0f481f8b00 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f480c9450 0x7f0f481ea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 --2- 172.21.15.46:0/2733566858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f300039e0 0x7f0f481f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 >> 172.21.15.46:0/2733566858 conn(0x7f0f480d4f10 msgr2=0x7f0f480b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 shutdown_connections 2024-04-26T23:45:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:00.442+0000 7f0f4e800640 1 -- 172.21.15.46:0/2733566858 wait complete. 2024-04-26T23:45:00.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:44:42.305207+0000', 'modified': '2024-04-26T23:44:59.019803+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/2779258602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2779258602}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2779258602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:45:00.869 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7959:up:rejoin 2024-04-26T23:45:01.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.088+0000 7f6e27600640 1 Processor -- start 2024-04-26T23:45:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.088+0000 7f6e27600640 1 -- start start 2024-04-26T23:45:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.089+0000 7f6e27600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e2006e780 0x7f6e200b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.089+0000 7f6e27600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e200b4940 con 0x7f6e200b43c0 2024-04-26T23:45:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.089+0000 7f6e27600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e200b4c20 con 0x7f6e2014d200 2024-04-26T23:45:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.089+0000 7f6e27600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6e200b4f00 con 0x7f6e2006e780 2024-04-26T23:45:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e2006e780 0x7f6e200b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e2006e780 0x7f6e200b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38030/0 (socket says 172.21.15.46:38030) 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 -- 172.21.15.46:0/2838889122 learned_addr learned my addr 172.21.15.46:0/2838889122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e1fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6e200b43c0 0x7f6e2014caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35808/0 (socket says 172.21.15.46:35808) 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 -- 172.21.15.46:0/2838889122 >> v1:172.21.15.142:6789/0 conn(0x7f6e2014d200 legacy=0x7f6e20072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 -- 172.21.15.46:0/2838889122 >> v1:172.21.15.46:6789/0 conn(0x7f6e200b43c0 legacy=0x7f6e2014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.090+0000 7f6e24c00640 1 -- 172.21.15.46:0/2838889122 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e200b51e0 con 0x7f6e2006e780 2024-04-26T23:45:02.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e24c00640 1 --2- 172.21.15.46:0/2838889122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e200b3e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009870 tx=0x7f6e1002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25331f03e9a66af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:02.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e1f400640 1 -- 172.21.15.46:0/2838889122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10037070 con 0x7f6e2006e780 2024-04-26T23:45:02.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e1f400640 1 -- 172.21.15.46:0/2838889122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e10032440 con 0x7f6e2006e780 2024-04-26T23:45:02.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e1f400640 1 -- 172.21.15.46:0/2838889122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10032810 con 0x7f6e2006e780 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e27600640 1 -- 172.21.15.46:0/2838889122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 msgr2=0x7f6e200b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.092+0000 7f6e27600640 1 --2- 172.21.15.46:0/2838889122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e200b3e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009870 tx=0x7f6e1002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.093+0000 7f6e27600640 1 -- 172.21.15.46:0/2838889122 shutdown_connections 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.093+0000 7f6e27600640 1 --2- 172.21.15.46:0/2838889122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e200b3e80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.093+0000 7f6e27600640 1 -- 172.21.15.46:0/2838889122 >> 172.21.15.46:0/2838889122 conn(0x7f6e2006c530 msgr2=0x7f6e2006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.093+0000 7f6e27600640 1 -- 172.21.15.46:0/2838889122 shutdown_connections 2024-04-26T23:45:02.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.093+0000 7f6e27600640 1 -- 172.21.15.46:0/2838889122 wait complete. 2024-04-26T23:45:02.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.094+0000 7f6e27600640 1 Processor -- start 2024-04-26T23:45:02.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 -- start start 2024-04-26T23:45:02.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 0x7f6e2015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e2014d200 0x7f6e20160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2006ece0 con 0x7f6e2006e780 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2006eb60 con 0x7f6e200b43c0 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.095+0000 7f6e27600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2006ee60 con 0x7f6e2014d200 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38036/0 (socket says 172.21.15.46:38036) 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 -- 172.21.15.46:0/271249704 learned_addr learned my addr 172.21.15.46:0/271249704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e1fe00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 0x7f6e2015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e25600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e2014d200 0x7f6e20160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 -- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e2014d200 msgr2=0x7f6e20160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e2014d200 0x7f6e20160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 -- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 msgr2=0x7f6e2015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 0x7f6e2015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e24c00640 1 -- 172.21.15.46:0/271249704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e20160f80 con 0x7f6e2006e780 2024-04-26T23:45:02.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.096+0000 7f6e1fe00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 0x7f6e2015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e24c00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009870 tx=0x7f6e100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10041070 con 0x7f6e2006e780 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1003c870 con 0x7f6e2006e780 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10037040 con 0x7f6e2006e780 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2015f9a0 con 0x7f6e2006e780 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.097+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2015ff60 con 0x7f6e2006e780 2024-04-26T23:45:02.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.099+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e20069530 con 0x7f6e2006e780 2024-04-26T23:45:02.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.099+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e10044070 con 0x7f6e2006e780 2024-04-26T23:45:02.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.100+0000 7f6e1cc00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 0x7f6dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:02.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.100+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f6e100ac940 con 0x7f6e2006e780 2024-04-26T23:45:02.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.101+0000 7f6e1fe00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 0x7f6dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:02.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.102+0000 7f6e1fe00640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 0x7f6dfc06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e08007fd0 tx=0x7f6e08009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.105+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e1007ac70 con 0x7f6e2006e780 2024-04-26T23:45:02.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.350+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2014b3a0 con 0x7f6e2006e780 2024-04-26T23:45:02.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.351+0000 7f6e1cc00640 1 -- 172.21.15.46:0/271249704 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f6e1007a570 con 0x7f6e2006e780 2024-04-26T23:45:02.355 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:02.355 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:02.355 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 msgr2=0x7f6dfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 0x7f6dfc06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e08007fd0 tx=0x7f6e08009040 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 msgr2=0x7f6e201f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009870 tx=0x7f6e100046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 shutdown_connections 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dfc06c8f0 0x7f6dfc06edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e2014d200 0x7f6e20160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e200b43c0 0x7f6e2015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 --2- 172.21.15.46:0/271249704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e2006e780 0x7f6e201f9c80 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 >> 172.21.15.46:0/271249704 conn(0x7f6e2006c530 msgr2=0x7f6e20161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 shutdown_connections 2024-04-26T23:45:02.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:02.354+0000 7f6e27600640 1 -- 172.21.15.46:0/271249704 wait complete. 2024-04-26T23:45:02.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:44:42.305207+0000', 'modified': '2024-04-26T23:45:01.027471+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2779258602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2779258602}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2779258602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7959, 'qdb_cluster': [7959]} 2024-04-26T23:45:02.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:45:02.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:45:03.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.000+0000 7f8487e00640 1 Processor -- start 2024-04-26T23:45:03.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.000+0000 7f8487e00640 1 -- start start 2024-04-26T23:45:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84881466b0 0x7f848806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84880720e0 0x7f84880cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8488073ad0 con 0x7f8488160910 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8488146a90 con 0x7f84881466b0 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8488073d70 con 0x7f84880720e0 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8486a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84881466b0 0x7f848806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8488160910 0x7f84880cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35812/0 (socket says 172.21.15.46:35812) 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8487400640 1 -- 172.21.15.46:0/1776925617 learned_addr learned my addr 172.21.15.46:0/1776925617 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.002+0000 7f8486a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84881466b0 0x7f848806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38040/0 (socket says 172.21.15.46:38040) 2024-04-26T23:45:03.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.003+0000 7f8486000640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84880720e0 0x7f84880cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.003+0000 7f8486a00640 1 -- 172.21.15.46:0/1776925617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84880720e0 msgr2=0x7f84880cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.003+0000 7f8486a00640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84880720e0 0x7f84880cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.003+0000 7f8486a00640 1 -- 172.21.15.46:0/1776925617 >> v1:172.21.15.46:6789/0 conn(0x7f8488160910 legacy=0x7f84880cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.003+0000 7f8486a00640 1 -- 172.21.15.46:0/1776925617 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8488074050 con 0x7f84881466b0 2024-04-26T23:45:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.004+0000 7f8486a00640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84881466b0 0x7f848806de80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f847802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309be834d4e4f37b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.004+0000 7f8485600640 1 -- 172.21.15.46:0/1776925617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478037070 con 0x7f84881466b0 2024-04-26T23:45:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.004+0000 7f8485600640 1 -- 172.21.15.46:0/1776925617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8478032440 con 0x7f84881466b0 2024-04-26T23:45:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.004+0000 7f8485600640 1 -- 172.21.15.46:0/1776925617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478032810 con 0x7f84881466b0 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 -- 172.21.15.46:0/1776925617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84881466b0 msgr2=0x7f848806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84881466b0 0x7f848806de80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f847802a180 comp rx=0 tx=0).stop 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 -- 172.21.15.46:0/1776925617 shutdown_connections 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84880720e0 0x7f84880cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 --2- 172.21.15.46:0/1776925617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84881466b0 0x7f848806de80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.005+0000 7f8487e00640 1 -- 172.21.15.46:0/1776925617 >> 172.21.15.46:0/1776925617 conn(0x7f84880b26d0 msgr2=0x7f84880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.006+0000 7f8487e00640 1 -- 172.21.15.46:0/1776925617 shutdown_connections 2024-04-26T23:45:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.006+0000 7f8487e00640 1 -- 172.21.15.46:0/1776925617 wait complete. 2024-04-26T23:45:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.007+0000 7f8487e00640 1 Processor -- start 2024-04-26T23:45:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.008+0000 7f8487e00640 1 -- start start 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.008+0000 7f8487e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.008+0000 7f8487e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84881466b0 0x7f84880cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8487e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8488160910 0x7f84880cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8487e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f848806e3c0 con 0x7f84880720e0 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8487e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84880cb470 con 0x7f84881466b0 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8487e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cdda0 con 0x7f8488160910 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38044/0 (socket says 172.21.15.46:38044) 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 -- 172.21.15.46:0/3711564507 learned_addr learned my addr 172.21.15.46:0/3711564507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 -- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8488160910 msgr2=0x7f84880cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8488160910 0x7f84880cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 -- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84881466b0 msgr2=0x7f84880cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84881466b0 0x7f84880cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.009+0000 7f8486a00640 1 -- 172.21.15.46:0/3711564507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84880b4850 con 0x7f84880720e0 2024-04-26T23:45:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f8486a00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f84780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f847ea00640 1 -- 172.21.15.46:0/3711564507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478041070 con 0x7f84880720e0 2024-04-26T23:45:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f847ea00640 1 -- 172.21.15.46:0/3711564507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847803c950 con 0x7f84880720e0 2024-04-26T23:45:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84880b37b0 con 0x7f84880720e0 2024-04-26T23:45:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f847ea00640 1 -- 172.21.15.46:0/3711564507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478037040 con 0x7f84880720e0 2024-04-26T23:45:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.010+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84880b3d70 con 0x7f84880720e0 2024-04-26T23:45:03.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.013+0000 7f847ea00640 1 -- 172.21.15.46:0/3711564507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8478044070 con 0x7f84880720e0 2024-04-26T23:45:03.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.014+0000 7f847ea00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.014+0000 7f847ea00640 1 -- 172.21.15.46:0/3711564507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f84780ac940 con 0x7f84880720e0 2024-04-26T23:45:03.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.014+0000 7f8490869740 1 Processor -- start 2024-04-26T23:45:03.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.014+0000 7f8490869740 1 -- start start 2024-04-26T23:45:03.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.014+0000 7f8486000640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415dee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 0x557415da6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557415d0b600 con 0x557415dcf5a0 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557415ca8100 con 0x557415dee900 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.015+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557415d075c0 con 0x557415da9950 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38060/0 (socket says 172.21.15.46:38060) 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 -- 172.21.15.46:0/1395311007 learned_addr learned my addr 172.21.15.46:0/1395311007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486000640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 0x7f846806edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8474006fd0 tx=0x7f8474008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 -- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 msgr2=0x557415dee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486000640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415dee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8487400640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 0x557415da6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415dee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 -- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 msgr2=0x557415da6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 0x557415da6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 -- 172.21.15.46:0/1395311007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557415a1d2d0 con 0x557415dcf5a0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8487400640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 0x557415da6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.016+0000 7f8486a00640 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f847807a020 tx=0x7f8478079c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b6ca0d8e3eaec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8473400640 1 -- 172.21.15.46:0/1395311007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847804b040 con 0x557415dcf5a0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8473400640 1 -- 172.21.15.46:0/1395311007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847807e6d0 con 0x557415dcf5a0 2024-04-26T23:45:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8473400640 1 -- 172.21.15.46:0/1395311007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478082f00 con 0x557415dcf5a0 2024-04-26T23:45:03.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 -- 172.21.15.46:0/1395311007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 msgr2=0x557415dcf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f847807a020 tx=0x7f8478079c80 comp rx=0 tx=0).stop 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 -- 172.21.15.46:0/1395311007 shutdown_connections 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dee900 0x557415da6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415dee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 --2- 172.21.15.46:0/1395311007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcf980 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.017+0000 7f8490869740 1 -- 172.21.15.46:0/1395311007 >> 172.21.15.46:0/1395311007 conn(0x557415d3bd10 msgr2=0x557415ded270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.018+0000 7f8490869740 1 -- 172.21.15.46:0/1395311007 shutdown_connections 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.018+0000 7f8490869740 1 -- 172.21.15.46:0/1395311007 wait complete. 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.018+0000 7f8490869740 1 -- start start 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.019+0000 7f8490869740 1 Processor -- start 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.019+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415da9950 0x557415da7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 0x557415dc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557415d0b600 con 0x557415dcf5a0 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557415c92f70 con 0x557415da9950 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557415da3db0 con 0x557415dd0470 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8490869740 10 client.0 _ll_register_callbacks cb 0x557415d01480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38062/0 (socket says 172.21.15.46:38062) 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 -- 172.21.15.46:0/3066685579 learned_addr learned my addr 172.21.15.46:0/3066685579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8487400640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 0x557415dc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486a00640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415da9950 0x557415da7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 -- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 msgr2=0x557415dc9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 0x557415dc9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 -- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415da9950 msgr2=0x557415da7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415da9950 0x557415da7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8486000640 1 -- 172.21.15.46:0/3066685579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557415ae76b0 con 0x557415dcf5a0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.020+0000 7f8487400640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 0x557415dc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8486000640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f847400dde0 tx=0x7f84740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7ca173cca7749e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8467e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847401a050 con 0x557415dcf5a0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f847400de10 con 0x557415dcf5a0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847807e6d0 con 0x557415dcf5a0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8474005990 con 0x557415dcf5a0 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8490869740 10 client.7998 fetch_fsmap learned FSMap version 82 2024-04-26T23:45:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x557415da6cf0 con 0x557415dcf5a0 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.021+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847807e6d0 con 0x557415dcf5a0 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f847402f080 con 0x557415dcf5a0 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 10 client.7998 fetch_fsmap finished waiting for FSMap version 82 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 10 client.7998 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 20 client.7998 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dd34a0 0x557415def0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 4 client.7998 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 --> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557415d19210 con 0x557415dd34a0 2024-04-26T23:45:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.022+0000 7f8486a00640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dd34a0 0x557415def0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.023+0000 7f8486a00640 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dd34a0 0x557415def0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d98539b20d12ddf server_cookie=f394979af28befcb in_seq=0 out_seq=0 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.023+0000 7f8467e00640 10 client.7998 ms_handle_connect on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8467e00640 1 -- 172.21.15.46:0/3066685579 <== mds.0 v2:172.21.15.46:6834/2779258602 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557415d19210 con 0x557415dd34a0 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8467e00640 10 client.7998 handle_command_reply: tid=0 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 client.7998 shutdown 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 msgr2=0x557415dcff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f847400dde0 tx=0x7f84740046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 shutdown_connections 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dd34a0 0x557415def0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415dd0470 0x557415dc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dcff30 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 --2- 172.21.15.46:0/3066685579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415da9950 0x557415da7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8467e00640 0 client.7998 ms_handle_reset on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 >> 172.21.15.46:0/3066685579 conn(0x557415d3bd10 msgr2=0x557415df1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 shutdown_connections 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 1 -- 172.21.15.46:0/3066685579 wait complete. 2024-04-26T23:45:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.024+0000 7f8490869740 20 client.7998 trim_cache size 0 max 16384 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.046+0000 7f8490869740 1 Processor -- start 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.046+0000 7f8490869740 1 -- start start 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415e00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84880d0520 con 0x557415dcf5a0 2024-04-26T23:45:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84880cf880 con 0x557415dd0470 2024-04-26T23:45:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cfed0 con 0x557415da9950 2024-04-26T23:45:03.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38078/0 (socket says 172.21.15.46:38078) 2024-04-26T23:45:03.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.047+0000 7f8486000640 1 -- 172.21.15.46:0/3654844873 learned_addr learned my addr 172.21.15.46:0/3654844873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486a00640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 -- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 msgr2=0x557415e150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 -- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 msgr2=0x557415e00150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8487400640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415e00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415e00150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 -- 172.21.15.46:0/3654844873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557415da5b10 con 0x557415dcf5a0 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8487400640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415e00150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.048+0000 7f8486000640 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f8474014dd0 tx=0x7f847401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28269ab35c52fd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8472a00640 1 -- 172.21.15.46:0/3654844873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847401a070 con 0x557415dcf5a0 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8472a00640 1 -- 172.21.15.46:0/3654844873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8474024070 con 0x557415dcf5a0 2024-04-26T23:45:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8472a00640 1 -- 172.21.15.46:0/3654844873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84740127f0 con 0x557415dcf5a0 2024-04-26T23:45:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 -- 172.21.15.46:0/3654844873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 msgr2=0x557415dffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f8474014dd0 tx=0x7f847401e000 comp rx=0 tx=0).stop 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 -- 172.21.15.46:0/3654844873 shutdown_connections 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415e00150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415dffc10 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 --2- 172.21.15.46:0/3654844873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 -- 172.21.15.46:0/3654844873 >> 172.21.15.46:0/3654844873 conn(0x557415d3bd10 msgr2=0x557415da5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.049+0000 7f8490869740 1 -- 172.21.15.46:0/3654844873 shutdown_connections 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.050+0000 7f8490869740 1 -- 172.21.15.46:0/3654844873 wait complete. 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.050+0000 7f8490869740 1 -- start start 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.050+0000 7f8490869740 1 Processor -- start 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e13c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415dca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557415dd3320 con 0x557415dcf5a0 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557415dd1b00 con 0x557415dd0470 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557415dd3970 con 0x557415da9950 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.051+0000 7f8490869740 10 client.0 _ll_register_callbacks cb 0x7f847400b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38092/0 (socket says 172.21.15.46:38092) 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 -- 172.21.15.46:0/1222378480 learned_addr learned my addr 172.21.15.46:0/1222378480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486a00640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e13c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 -- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 msgr2=0x557415e13c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e13c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 -- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 msgr2=0x557415dca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8487400640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415dca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415dca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 -- 172.21.15.46:0/1222378480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557415d9df80 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8487400640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415dca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.052+0000 7f8486000640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f847400dde0 tx=0x7f8474015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28039abae9391766 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847401a070 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f8474012e30 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8474024070 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8474045050 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8490869740 10 client.8004 fetch_fsmap learned FSMap version 82 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x557415e53c10 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8474024070 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f8474048020 con 0x557415dcf5a0 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8490869740 10 client.8004 fetch_fsmap finished waiting for FSMap version 82 2024-04-26T23:45:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.053+0000 7f8490869740 10 client.8004 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8490869740 20 client.8004 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dcadd0 0x557415dd43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8490869740 4 client.8004 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 --> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] -- command(tid 0: {"prefix": "status"}) -- 0x557415dd2dc0 con 0x557415dcadd0 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8486a00640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dcadd0 0x557415dd43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8486a00640 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dcadd0 0x557415dd43b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5435b45bd2fa94c server_cookie=9a9cd61ef2ea2c19 in_seq=0 out_seq=0 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.054+0000 7f8467e00640 10 client.8004 ms_handle_connect on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8467e00640 1 -- 172.21.15.46:0/1222378480 <== mds.0 v2:172.21.15.46:6834/2779258602 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x557415dd2dc0 con 0x557415dcadd0 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8467e00640 10 client.8004 handle_command_reply: tid=0 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 client.8004 shutdown 2024-04-26T23:45:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 msgr2=0x557415e141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f847400dde0 tx=0x7f8474015000 comp rx=0 tx=0).stop 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 shutdown_connections 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557415dd0470 0x557415dca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557415dcf5a0 0x557415e141b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x557415dcadd0 0x557415dd43b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 --2- 172.21.15.46:0/1222378480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557415da9950 0x557415e13c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8467e00640 0 client.8004 ms_handle_reset on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 >> 172.21.15.46:0/1222378480 conn(0x557415d3bd10 msgr2=0x557415da8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 shutdown_connections 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 1 -- 172.21.15.46:0/1222378480 wait complete. 2024-04-26T23:45:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.055+0000 7f8490869740 20 client.8004 trim_cache size 0 max 16384 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7959, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.0500654090000001, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 82, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 100, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 100, 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.8288283549999997 2024-04-26T23:45:03.066 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:45:03.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 msgr2=0x7f846806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 0x7f846806edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8474006fd0 tx=0x7f8474008040 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 msgr2=0x7f84880b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f84780046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 shutdown_connections 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8488160910 0x7f84880cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84881466b0 0x7f84880cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 --2- 172.21.15.46:0/3711564507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84880720e0 0x7f84880b5060 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 >> 172.21.15.46:0/3711564507 conn(0x7f84880b26d0 msgr2=0x7f8488160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 shutdown_connections 2024-04-26T23:45:03.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.056+0000 7f8487e00640 1 -- 172.21.15.46:0/3711564507 wait complete. 2024-04-26T23:45:03.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_size 20 2024-04-26T23:45:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.267+0000 7f4320400640 1 Processor -- start 2024-04-26T23:45:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.267+0000 7f4320400640 1 -- start start 2024-04-26T23:45:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f4320400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43180b4cd0 0x7f431815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f4320400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43180d2790 0x7f43180b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f4320400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431815d420 con 0x7f4318073020 2024-04-26T23:45:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f4320400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f43180b55f0 con 0x7f43180d2790 2024-04-26T23:45:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f4320400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f43180b59a0 con 0x7f43180b4cd0 2024-04-26T23:45:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f431d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43180d2790 0x7f43180b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f431e400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4318073020 0x7f4318073450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35824/0 (socket says 172.21.15.46:35824) 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f431d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43180d2790 0x7f43180b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38098/0 (socket says 172.21.15.46:38098) 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.269+0000 7f431e400640 1 -- 172.21.15.46:0/829201294 learned_addr learned my addr 172.21.15.46:0/829201294 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.270+0000 7f431da00640 1 --2- 172.21.15.46:0/829201294 >> v2:172.21.15.142:3300/0 conn(0x7f43180b4cd0 0x7f431815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.270+0000 7f431d000640 1 -- 172.21.15.46:0/829201294 >> v2:172.21.15.142:3300/0 conn(0x7f43180b4cd0 msgr2=0x7f431815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.270+0000 7f431d000640 1 --2- 172.21.15.46:0/829201294 >> v2:172.21.15.142:3300/0 conn(0x7f43180b4cd0 0x7f431815cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.270+0000 7f431d000640 1 -- 172.21.15.46:0/829201294 >> v1:172.21.15.46:6789/0 conn(0x7f4318073020 legacy=0x7f4318073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.270+0000 7f431d000640 1 -- 172.21.15.46:0/829201294 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43180b5c80 con 0x7f43180d2790 2024-04-26T23:45:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.271+0000 7f431d000640 1 --2- 172.21.15.46:0/829201294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f43180b50b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f43180d2da0 tx=0x7f430402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ddb8a95b9677a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4317e00640 1 -- 172.21.15.46:0/829201294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304037070 con 0x7f43180d2790 2024-04-26T23:45:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4317e00640 1 -- 172.21.15.46:0/829201294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4304032440 con 0x7f43180d2790 2024-04-26T23:45:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4317e00640 1 -- 172.21.15.46:0/829201294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430402ae60 con 0x7f43180d2790 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4320400640 1 -- 172.21.15.46:0/829201294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 msgr2=0x7f43180b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4320400640 1 --2- 172.21.15.46:0/829201294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f43180b50b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f43180d2da0 tx=0x7f430402a110 comp rx=0 tx=0).stop 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4320400640 1 -- 172.21.15.46:0/829201294 shutdown_connections 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4320400640 1 --2- 172.21.15.46:0/829201294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f43180b50b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.272+0000 7f4320400640 1 --2- 172.21.15.46:0/829201294 >> v2:172.21.15.142:3300/0 conn(0x7f43180b4cd0 0x7f431815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.273+0000 7f4320400640 1 -- 172.21.15.46:0/829201294 >> 172.21.15.46:0/829201294 conn(0x7f431806d640 msgr2=0x7f431806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.273+0000 7f4320400640 1 -- 172.21.15.46:0/829201294 shutdown_connections 2024-04-26T23:45:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.273+0000 7f4320400640 1 -- 172.21.15.46:0/829201294 wait complete. 2024-04-26T23:45:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.274+0000 7f4320400640 1 Processor -- start 2024-04-26T23:45:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.275+0000 7f4320400640 1 -- start start 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.275+0000 7f4320400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4318073020 0x7f431814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f4320400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43180b4cd0 0x7f43181533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f4320400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f4320400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43180b6520 con 0x7f43180d2790 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f4320400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43180b6240 con 0x7f4318073020 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f4320400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4318159d70 con 0x7f43180b4cd0 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f431e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f431e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38114/0 (socket says 172.21.15.46:38114) 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f431e400640 1 -- 172.21.15.46:0/2971637505 learned_addr learned my addr 172.21.15.46:0/2971637505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f431da00640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4318073020 0x7f431814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.276+0000 7f431d000640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43180b4cd0 0x7f43181533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 -- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43180b4cd0 msgr2=0x7f43181533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43180b4cd0 0x7f43181533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 -- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4318073020 msgr2=0x7f431814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4318073020 0x7f431814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 -- 172.21.15.46:0/2971637505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43180d1580 con 0x7f43180d2790 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f431e400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f431000cab0 tx=0x7f431000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310017070 con 0x7f43180d2790 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431000ac30 con 0x7f43180d2790 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43180d1860 con 0x7f43180d2790 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310012400 con 0x7f43180d2790 2024-04-26T23:45:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.277+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431815a860 con 0x7f43180d2790 2024-04-26T23:45:03.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.279+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4318069530 con 0x7f43180d2790 2024-04-26T23:45:03.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.280+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431001a070 con 0x7f43180d2790 2024-04-26T23:45:03.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.280+0000 7f4315600640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 0x7f42f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.281+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f431008ba40 con 0x7f43180d2790 2024-04-26T23:45:03.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.281+0000 7f431da00640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 0x7f42f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.282+0000 7f431da00640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 0x7f42f806edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f430c006fd0 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.285+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f431815d420 con 0x7f43180d2790 2024-04-26T23:45:03.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.488+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f431806dd00 con 0x7f43180d2790 2024-04-26T23:45:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.494+0000 7f4315600640 1 -- 172.21.15.46:0/2971637505 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v27)=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f4310059560 con 0x7f43180d2790 2024-04-26T23:45:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 msgr2=0x7f42f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 0x7f42f806edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f430c006fd0 tx=0x7f430c008040 comp rx=0 tx=0).stop 2024-04-26T23:45:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 msgr2=0x7f4318153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f431000cab0 tx=0x7f431000cf70 comp rx=0 tx=0).stop 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 shutdown_connections 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.497+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42f806c8f0 0x7f42f806edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43180d2790 0x7f4318153930 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43180b4cd0 0x7f43181533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 --2- 172.21.15.46:0/2971637505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4318073020 0x7f431814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 >> 172.21.15.46:0/2971637505 conn(0x7f431806d640 msgr2=0x7f431806f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 shutdown_connections 2024-04-26T23:45:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.498+0000 7f4320400640 1 -- 172.21.15.46:0/2971637505 wait complete. 2024-04-26T23:45:03.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_merge_size 5 2024-04-26T23:45:03.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.725+0000 7f5090a00640 1 Processor -- start 2024-04-26T23:45:03.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.725+0000 7f5090a00640 1 -- start start 2024-04-26T23:45:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5090a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c15f1e0 0x7f508c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5090a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508c070100 0x7f508c070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5090a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508c0afbc0 con 0x7f508c06e0e0 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5090a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f508c070aa0 con 0x7f508c070100 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5090a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f508c0b07d0 con 0x7f508c15f1e0 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5089e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508c070100 0x7f508c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5089e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f508c070100 0x7f508c070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38122/0 (socket says 172.21.15.46:38122) 2024-04-26T23:45:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f5089e00640 1 -- 172.21.15.46:0/692320125 learned_addr learned my addr 172.21.15.46:0/692320125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.727+0000 7f508a800640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c15f1e0 0x7f508c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.728+0000 7f5089e00640 1 -- 172.21.15.46:0/692320125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c15f1e0 msgr2=0x7f508c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.728+0000 7f5089e00640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c15f1e0 0x7f508c072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.728+0000 7f5089e00640 1 -- 172.21.15.46:0/692320125 >> v1:172.21.15.46:6789/0 conn(0x7f508c06e0e0 legacy=0x7f508c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.728+0000 7f5089e00640 1 -- 172.21.15.46:0/692320125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c0afea0 con 0x7f508c070100 2024-04-26T23:45:03.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.729+0000 7f5089e00640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c070100 0x7f508c070560 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f507c009870 tx=0x7f507c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8002257e19d3f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.729+0000 7f5089400640 1 -- 172.21.15.46:0/692320125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c037070 con 0x7f508c070100 2024-04-26T23:45:03.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.729+0000 7f5089400640 1 -- 172.21.15.46:0/692320125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c032440 con 0x7f508c070100 2024-04-26T23:45:03.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.729+0000 7f5089400640 1 -- 172.21.15.46:0/692320125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c032810 con 0x7f508c070100 2024-04-26T23:45:03.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 -- 172.21.15.46:0/692320125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c070100 msgr2=0x7f508c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c070100 0x7f508c070560 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f507c009870 tx=0x7f507c02a270 comp rx=0 tx=0).stop 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 -- 172.21.15.46:0/692320125 shutdown_connections 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c070100 0x7f508c070560 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 --2- 172.21.15.46:0/692320125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c15f1e0 0x7f508c072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 -- 172.21.15.46:0/692320125 >> 172.21.15.46:0/692320125 conn(0x7f508c0b2c60 msgr2=0x7f508c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 -- 172.21.15.46:0/692320125 shutdown_connections 2024-04-26T23:45:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.730+0000 7f5090a00640 1 -- 172.21.15.46:0/692320125 wait complete. 2024-04-26T23:45:03.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.732+0000 7f5090a00640 1 Processor -- start 2024-04-26T23:45:03.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.732+0000 7f5090a00640 1 -- start start 2024-04-26T23:45:03.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c06e0e0 0x7f508c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c070100 0x7f508c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f508c15f5c0 con 0x7f508c15f1e0 2024-04-26T23:45:03.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0b0460 con 0x7f508c06e0e0 2024-04-26T23:45:03.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f5090a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f508c15caa0 con 0x7f508c070100 2024-04-26T23:45:03.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f508b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f508b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38128/0 (socket says 172.21.15.46:38128) 2024-04-26T23:45:03.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.733+0000 7f508b200640 1 -- 172.21.15.46:0/3391088747 learned_addr learned my addr 172.21.15.46:0/3391088747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:03.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f5089e00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c070100 0x7f508c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508a800640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c06e0e0 0x7f508c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 -- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c070100 msgr2=0x7f508c148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c070100 0x7f508c148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 -- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c06e0e0 msgr2=0x7f508c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c06e0e0 0x7f508c15c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 -- 172.21.15.46:0/3391088747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c15cd80 con 0x7f508c15f1e0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f508b200640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f508400c7a0 tx=0x7f508400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084003b70 con 0x7f508c15f1e0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5084003d30 con 0x7f508c15f1e0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400aa90 con 0x7f508c15f1e0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.734+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c15d060 con 0x7f508c15f1e0 2024-04-26T23:45:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.735+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c15d620 con 0x7f508c15f1e0 2024-04-26T23:45:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.736+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c0693b0 con 0x7f508c15f1e0 2024-04-26T23:45:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.737+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f508400ac30 con 0x7f508c15f1e0 2024-04-26T23:45:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.738+0000 7f5082a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 0x7f506806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.738+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f508408bcd0 con 0x7f508c15f1e0 2024-04-26T23:45:03.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.739+0000 7f508a800640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 0x7f506806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.740+0000 7f508a800640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 0x7f506806ede0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.743+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5084059ee0 con 0x7f508c15f1e0 2024-04-26T23:45:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.948+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f508c06f6f0 con 0x7f508c15f1e0 2024-04-26T23:45:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.954+0000 7f5082a00640 1 -- 172.21.15.46:0/3391088747 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v28)=0 v28) ==== 116+0+0 (secure 0 0 0) 0x7f50840597e0 con 0x7f508c15f1e0 2024-04-26T23:45:03.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 msgr2=0x7f506806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 0x7f506806ede0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 msgr2=0x7f508c161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f508400c7a0 tx=0x7f508400cc60 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 shutdown_connections 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f506806c920 0x7f506806ede0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f508c15f1e0 0x7f508c161b40 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c070100 0x7f508c148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 --2- 172.21.15.46:0/3391088747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c06e0e0 0x7f508c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 >> 172.21.15.46:0/3391088747 conn(0x7f508c0b2c60 msgr2=0x7f508c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 shutdown_connections 2024-04-26T23:45:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:03.957+0000 7f5090a00640 1 -- 172.21.15.46:0/3391088747 wait complete. 2024-04-26T23:45:03.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_bits 1 2024-04-26T23:45:04.013 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:45:04.009+0000 7fd1c5a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:45:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.194+0000 7f1811800640 1 Processor -- start 2024-04-26T23:45:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.194+0000 7f1811800640 1 -- start start 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.195+0000 7f1811800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f180c0cdd00 0x7f180c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.195+0000 7f1811800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c0ce7d0 con 0x7f180c073b30 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.195+0000 7f1811800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c0ceab0 con 0x7f180c15cec0 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.195+0000 7f1811800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f180c0ced90 con 0x7f180c0cdd00 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f1810e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f180c073b30 0x7f180c0b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35834/0 (socket says 172.21.15.46:35834) 2024-04-26T23:45:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f1810e00640 1 -- 172.21.15.46:0/900231539 learned_addr learned my addr 172.21.15.46:0/900231539 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3495135335 0 0) 0x7f180c0ce7d0 con 0x7f180c073b30 2024-04-26T23:45:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f1803e00640 1 --2- 172.21.15.46:0/900231539 >> v2:172.21.15.142:3300/0 conn(0x7f180c0cdd00 0x7f180c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f0003520 con 0x7f180c073b30 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2507036049 0 0) 0x7f17f0003520 con 0x7f180c073b30 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 >> v2:172.21.15.142:3300/0 conn(0x7f180c0cdd00 msgr2=0x7f180c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 --2- 172.21.15.46:0/900231539 >> v2:172.21.15.142:3300/0 conn(0x7f180c0cdd00 0x7f180c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.196+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 >> v1:172.21.15.142:6789/0 conn(0x7f180c15cec0 legacy=0x7f180c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c0cf070 con 0x7f180c073b30 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f18040019f0 con 0x7f180c073b30 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 >> v1:172.21.15.46:6789/0 conn(0x7f180c073b30 legacy=0x7f180c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17f0003a00 0x7f17f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f0004800 0x7f17f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 0x7f17f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f180c0ced90 con 0x7f17f00051c0 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17f000c0c0 con 0x7f17f0004800 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f17f000c2c0 con 0x7f17f0003a00 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f1810e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 0x7f17f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f1810e00640 1 -- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17f0003a00 msgr2=0x7f17f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f180be00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17f0003a00 0x7f17f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.197+0000 7f1803e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f0004800 0x7f17f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1810e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17f0003a00 0x7f17f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1810e00640 1 -- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f0004800 msgr2=0x7f17f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1810e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f0004800 0x7f17f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1810e00640 1 -- 172.21.15.46:0/900231539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f0003520 con 0x7f17f00051c0 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f180be00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17f0003a00 0x7f17f0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1803e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f0004800 0x7f17f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.198+0000 7f1810e00640 1 -- 172.21.15.46:0/900231539 reap_dead start 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.199+0000 7f1810e00640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 0x7f17f000b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f180400a770 tx=0x7f17f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42236230397a3c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.199+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400a390 con 0x7f17f00051c0 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.199+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180402f070 con 0x7f17f00051c0 2024-04-26T23:45:04.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.199+0000 7f180b400640 1 -- 172.21.15.46:0/900231539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180402bbe0 con 0x7f17f00051c0 2024-04-26T23:45:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 -- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 msgr2=0x7f17f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 0x7f17f000b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f180400a770 tx=0x7f17f0005830 comp rx=0 tx=0).stop 2024-04-26T23:45:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 -- 172.21.15.46:0/900231539 shutdown_connections 2024-04-26T23:45:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 --2- 172.21.15.46:0/900231539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f00051c0 0x7f17f000b9c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 -- 172.21.15.46:0/900231539 >> 172.21.15.46:0/900231539 conn(0x7f180c0b27b0 msgr2=0x7f180c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 -- 172.21.15.46:0/900231539 shutdown_connections 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.200+0000 7f1811800640 1 -- 172.21.15.46:0/900231539 wait complete. 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.202+0000 7f1811800640 1 Processor -- start 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.202+0000 7f1811800640 1 -- start start 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 0x7f180c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f180c155ca0 0x7f180c1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f180c073f70 con 0x7f180c0c5c70 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f180c15d2a0 con 0x7f17f00051c0 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1811800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f180c0735f0 con 0x7f180c155ca0 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1810e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1810e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38140/0 (socket says 172.21.15.46:38140) 2024-04-26T23:45:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.203+0000 7f1810e00640 1 -- 172.21.15.46:0/2787861147 learned_addr learned my addr 172.21.15.46:0/2787861147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1803e00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f180c155ca0 0x7f180c1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f180be00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 0x7f180c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 -- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f180c155ca0 msgr2=0x7f180c1f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f180c155ca0 0x7f180c1f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 -- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 msgr2=0x7f180c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 0x7f180c1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 -- 172.21.15.46:0/2787861147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1f4f00 con 0x7f180c0c5c70 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f180be00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 0x7f180c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1810e00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f180402b430 tx=0x7f180c0ce9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400a390 con 0x7f180c0c5c70 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804001250 con 0x7f180c0c5c70 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804001ee0 con 0x7f180c0c5c70 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.204+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c0c8ea0 con 0x7f180c0c5c70 2024-04-26T23:45:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.205+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1f5480 con 0x7f180c0c5c70 2024-04-26T23:45:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.206+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c0b12c0 con 0x7f180c0c5c70 2024-04-26T23:45:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.207+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1804034470 con 0x7f180c0c5c70 2024-04-26T23:45:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.208+0000 7f1808c00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 0x7f17e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.208+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f18040abc40 con 0x7f180c0c5c70 2024-04-26T23:45:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.209+0000 7f180be00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 0x7f17e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.210+0000 7f180be00640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 0x7f17e806ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f180c0cf8e0 tx=0x7f17fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.213+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f180c0b12c0 con 0x7f180c0c5c70 2024-04-26T23:45:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.418+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f180c0693b0 con 0x7f180c0c5c70 2024-04-26T23:45:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.425+0000 7f1808c00640 1 -- 172.21.15.46:0/2787861147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v29)=0 v29) ==== 116+0+0 (secure 0 0 0) 0x7f1804079db0 con 0x7f180c0c5c70 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.428+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 msgr2=0x7f17e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.428+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 0x7f17e806ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f180c0cf8e0 tx=0x7f17fc009040 comp rx=0 tx=0).stop 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.428+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 msgr2=0x7f180c0c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.428+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f180402b430 tx=0x7f180c0ce9a0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 shutdown_connections 2024-04-26T23:45:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f17e806c920 0x7f17e806ede0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f180c0c5c70 0x7f180c0c93a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f180c155ca0 0x7f180c1f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 --2- 172.21.15.46:0/2787861147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17f00051c0 0x7f180c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 >> 172.21.15.46:0/2787861147 conn(0x7f180c0b27b0 msgr2=0x7f180c0b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 shutdown_connections 2024-04-26T23:45:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.429+0000 7f1811800640 1 -- 172.21.15.46:0/2787861147 wait complete. 2024-04-26T23:45:04.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.707+0000 7f7d16400640 1 Processor -- start 2024-04-26T23:45:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.707+0000 7f7d16400640 1 -- start start 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.709+0000 7f7d16400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d10155b20 0x7f7d1015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.709+0000 7f7d16400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d10155f00 0x7f7d100b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.709+0000 7f7d16400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d100b4000 con 0x7f7d1006eda0 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.709+0000 7f7d16400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7d100b6200 con 0x7f7d10155b20 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.709+0000 7f7d16400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7d100b42c0 con 0x7f7d10155f00 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d10155b20 0x7f7d1015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d10155b20 0x7f7d1015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38152/0 (socket says 172.21.15.46:38152) 2024-04-26T23:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 -- 172.21.15.46:0/2347527068 learned_addr learned my addr 172.21.15.46:0/2347527068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0ea00640 1 --2- 172.21.15.46:0/2347527068 >> v2:172.21.15.142:3300/0 conn(0x7f7d10155f00 0x7f7d100b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 -- 172.21.15.46:0/2347527068 >> v2:172.21.15.142:3300/0 conn(0x7f7d10155f00 msgr2=0x7f7d100b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 --2- 172.21.15.46:0/2347527068 >> v2:172.21.15.142:3300/0 conn(0x7f7d10155f00 0x7f7d100b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 -- 172.21.15.46:0/2347527068 >> v1:172.21.15.46:6789/0 conn(0x7f7d1006eda0 legacy=0x7f7d1015f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.710+0000 7f7d0f400640 1 -- 172.21.15.46:0/2347527068 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d100b45a0 con 0x7f7d10155b20 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.711+0000 7f7d0f400640 1 --2- 172.21.15.46:0/2347527068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d10155b20 0x7f7d1015c820 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7d1006fdd0 tx=0x7f7d0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740ec6781bd685fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d0e000640 1 -- 172.21.15.46:0/2347527068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00037070 con 0x7f7d10155b20 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d0e000640 1 -- 172.21.15.46:0/2347527068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d00032440 con 0x7f7d10155b20 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d0e000640 1 -- 172.21.15.46:0/2347527068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00032810 con 0x7f7d10155b20 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 -- 172.21.15.46:0/2347527068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d10155b20 msgr2=0x7f7d1015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 --2- 172.21.15.46:0/2347527068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d10155b20 0x7f7d1015c820 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7d1006fdd0 tx=0x7f7d0002a270 comp rx=0 tx=0).stop 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 -- 172.21.15.46:0/2347527068 shutdown_connections 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 --2- 172.21.15.46:0/2347527068 >> v2:172.21.15.142:3300/0 conn(0x7f7d10155f00 0x7f7d100b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 --2- 172.21.15.46:0/2347527068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d10155b20 0x7f7d1015c820 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.712+0000 7f7d16400640 1 -- 172.21.15.46:0/2347527068 >> 172.21.15.46:0/2347527068 conn(0x7f7d1006c530 msgr2=0x7f7d1006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.713+0000 7f7d16400640 1 -- 172.21.15.46:0/2347527068 shutdown_connections 2024-04-26T23:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.713+0000 7f7d16400640 1 -- 172.21.15.46:0/2347527068 wait complete. 2024-04-26T23:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.714+0000 7f7d16400640 1 Processor -- start 2024-04-26T23:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.714+0000 7f7d16400640 1 -- start start 2024-04-26T23:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d10155b20 0x7f7d1015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d10155f00 0x7f7d1015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d100b4da0 con 0x7f7d1006eda0 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d100cfd40 con 0x7f7d10155f00 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.715+0000 7f7d16400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1014f4f0 con 0x7f7d10155b20 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38164/0 (socket says 172.21.15.46:38164) 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 -- 172.21.15.46:0/667621988 learned_addr learned my addr 172.21.15.46:0/667621988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 -- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d10155b20 msgr2=0x7f7d1015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d10155b20 0x7f7d1015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 -- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d10155f00 msgr2=0x7f7d1015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d10155f00 0x7f7d1015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 -- 172.21.15.46:0/667621988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1014dca0 con 0x7f7d1006eda0 2024-04-26T23:45:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.716+0000 7f7d0f400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7d0002a2b0 tx=0x7f7d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.717+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00041020 con 0x7f7d1006eda0 2024-04-26T23:45:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.717+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d000306e0 con 0x7f7d1006eda0 2024-04-26T23:45:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.717+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d101e5560 con 0x7f7d1006eda0 2024-04-26T23:45:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.717+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00037040 con 0x7f7d1006eda0 2024-04-26T23:45:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.717+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101e5b20 con 0x7f7d1006eda0 2024-04-26T23:45:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.718+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10069530 con 0x7f7d1006eda0 2024-04-26T23:45:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.719+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d00044020 con 0x7f7d1006eda0 2024-04-26T23:45:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.720+0000 7f7d07400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.720+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f7d000ac690 con 0x7f7d1006eda0 2024-04-26T23:45:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.721+0000 7f7d0ea00640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.722+0000 7f7d0ea00640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 0x7f7cec06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80070b0 tx=0x7f7cf8007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:04.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.727+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d0004b030 con 0x7f7d1006eda0 2024-04-26T23:45:04.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.972+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d10074320 con 0x7f7d1006eda0 2024-04-26T23:45:04.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.973+0000 7f7d07400640 1 -- 172.21.15.46:0/667621988 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f7d0007a800 con 0x7f7d1006eda0 2024-04-26T23:45:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:04.977 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:04.977 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 msgr2=0x7f7cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 0x7f7cec06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80070b0 tx=0x7f7cf8007040 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 msgr2=0x7f7d101417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7d0002a2b0 tx=0x7f7d000046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 shutdown_connections 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d10155f00 0x7f7d1015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d10155b20 0x7f7d1015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 --2- 172.21.15.46:0/667621988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1006eda0 0x7f7d101417f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 >> 172.21.15.46:0/667621988 conn(0x7f7d1006c530 msgr2=0x7f7d10154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 shutdown_connections 2024-04-26T23:45:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:04.976+0000 7f7d16400640 1 -- 172.21.15.46:0/667621988 wait complete. 2024-04-26T23:45:05.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:45:05.234 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:44:52.577168+0000","btime":"2024-04-26T23:44:51.353182+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:44:52.577168+0000","atime":"2024-04-26T23:44:51.353182+0000","time_warp_seq":0,"change_attr":21,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:44:52.577168+0000","num_files":21,"num_subdirs":0,"change_attr":21},"rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"accounted_rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"version":128,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{},"nref":0,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:45:05.235 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:44:51.353182+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:44:51.353182+0000", "change_attr": 21, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:44:52.577168+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 21, "mtime": "2024-04-26T23:44:52.577168+0000", "num_files": 21, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:44:52.577168+0000", "nestlock": {}, "nlink": 1, "nref": 0, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": {}, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 128, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:45:05.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:05.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /testdir recursive 2024-04-26T23:45:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.454+0000 7f7488a00640 1 Processor -- start 2024-04-26T23:45:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.455+0000 7f7488a00640 1 -- start start 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.456+0000 7f7488a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74840d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.456+0000 7f7488a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74840d5d00 con 0x7f748415f6c0 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.456+0000 7f7488a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74840cffc0 con 0x7f74840d5820 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.456+0000 7f7488a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74840d02a0 con 0x7f74840d3e00 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.457+0000 7f7481e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f748415f6c0 0x7f74840d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35844/0 (socket says 172.21.15.46:35844) 2024-04-26T23:45:05.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.457+0000 7f7481e00640 1 -- 172.21.15.46:0/4023445484 learned_addr learned my addr 172.21.15.46:0/4023445484 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.457+0000 7f7482800640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74840d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.457+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622566623 0 0) 0x7f74840d5d00 con 0x7f748415f6c0 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.457+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7468003520 con 0x7f748415f6c0 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1015932859 0 0) 0x7f7468003520 con 0x7f748415f6c0 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 msgr2=0x7f74840d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74840d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 >> v1:172.21.15.142:6789/0 conn(0x7f74840d5820 legacy=0x7f74840d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74840d0580 con 0x7f748415f6c0 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7474001670 con 0x7f748415f6c0 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 >> v1:172.21.15.46:6789/0 conn(0x7f748415f6c0 legacy=0x7f74840d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74680039e0 0x7f7468003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468004800 0x7f7468004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74840d02a0 con 0x7f74680039e0 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f746800baa0 con 0x7f7468004800 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f746800bca0 con 0x7f74840d3e00 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.458+0000 7f7481e00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74680039e0 0x7f7468003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7482800640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481e00640 1 -- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 msgr2=0x7f74680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481400640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468004800 0x7f7468004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481e00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74840d3e00 0x7f74680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481e00640 1 -- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468004800 msgr2=0x7f7468004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481e00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468004800 0x7f7468004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.459+0000 7f7481e00640 1 -- 172.21.15.46:0/4023445484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7468003520 con 0x7f74680039e0 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.460+0000 7f7481e00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74680039e0 0x7f7468003e00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f7468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72165d7b6c4ef578 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.460+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474007d30 con 0x7f74680039e0 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.460+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474005af0 con 0x7f74680039e0 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.460+0000 7f7480a00640 1 -- 172.21.15.46:0/4023445484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474007070 con 0x7f74680039e0 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 -- 172.21.15.46:0/4023445484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74680039e0 msgr2=0x7f7468003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 --2- 172.21.15.46:0/4023445484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74680039e0 0x7f7468003e00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f7468005830 comp rx=0 tx=0).stop 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7482800640 1 -- 172.21.15.46:0/4023445484 reap_dead start 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 -- 172.21.15.46:0/4023445484 shutdown_connections 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 -- 172.21.15.46:0/4023445484 >> 172.21.15.46:0/4023445484 conn(0x7f748409e450 msgr2=0x7f74840d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 -- 172.21.15.46:0/4023445484 shutdown_connections 2024-04-26T23:45:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.461+0000 7f7488a00640 1 -- 172.21.15.46:0/4023445484 wait complete. 2024-04-26T23:45:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.463+0000 7f7488a00640 1 Processor -- start 2024-04-26T23:45:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.463+0000 7f7488a00640 1 -- start start 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74680039e0 0x7f74841614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 0x7f748415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74840d3230 con 0x7f7484150bc0 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74840d30b0 con 0x7f74680039e0 2024-04-26T23:45:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7488a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74840d2100 con 0x7f748415a980 2024-04-26T23:45:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38188/0 (socket says 172.21.15.46:38188) 2024-04-26T23:45:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7481400640 1 -- 172.21.15.46:0/853924415 learned_addr learned my addr 172.21.15.46:0/853924415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7482800640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 0x7f748415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7481400640 1 -- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 msgr2=0x7f748415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.464+0000 7f7481e00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74680039e0 0x7f74841614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7481400640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 0x7f748415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7481400640 1 -- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74680039e0 msgr2=0x7f74841614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7481400640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74680039e0 0x7f74841614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7481400640 1 -- 172.21.15.46:0/853924415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7484161a30 con 0x7f7484150bc0 2024-04-26T23:45:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7482800640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 0x7f748415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7481400640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f747000d570 tx=0x7f747000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f747a000640 1 -- 172.21.15.46:0/853924415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470003610 con 0x7f7484150bc0 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f747a000640 1 -- 172.21.15.46:0/853924415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7470017070 con 0x7f7484150bc0 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f747a000640 1 -- 172.21.15.46:0/853924415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74700039f0 con 0x7f7484150bc0 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74840cd8c0 con 0x7f7484150bc0 2024-04-26T23:45:05.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.465+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74840cde80 con 0x7f7484150bc0 2024-04-26T23:45:05.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.468+0000 7f747a000640 1 -- 172.21.15.46:0/853924415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7470003b90 con 0x7f7484150bc0 2024-04-26T23:45:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.469+0000 7f747a000640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 0x7f746406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.469+0000 7f747a000640 1 -- 172.21.15.46:0/853924415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f747008bc00 con 0x7f7484150bc0 2024-04-26T23:45:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.470+0000 7f748b85c740 1 Processor -- start 2024-04-26T23:45:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.470+0000 7f7481e00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 0x7f746406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.470+0000 7f748b85c740 1 -- start start 2024-04-26T23:45:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 0x563f0e895b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8960d0 0x563f0e86aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f0e7cfdb0 con 0x563f0e8b36f0 2024-04-26T23:45:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f0e76c000 con 0x563f0e896810 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563f0e7cbd70 con 0x563f0e8960d0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f7481e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f7481e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38194/0 (socket says 172.21.15.46:38194) 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.471+0000 7f7481e00640 1 -- 172.21.15.46:0/1903476108 learned_addr learned my addr 172.21.15.46:0/1903476108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7482800640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8960d0 0x563f0e86aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 0x7f746406ee00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7474032000 tx=0x7f747402d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481400640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 0x563f0e895b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 -- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8960d0 msgr2=0x563f0e86aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8960d0 0x563f0e86aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 -- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 msgr2=0x563f0e895b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 0x563f0e895b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 -- 172.21.15.46:0/1903476108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0e5ada20 con 0x563f0e8b36f0 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481400640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 0x563f0e895b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f7481e00640 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f747402add0 tx=0x7f747402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0556eef3083207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f746e600640 1 -- 172.21.15.46:0/1903476108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474042070 con 0x563f0e8b36f0 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f746e600640 1 -- 172.21.15.46:0/1903476108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474001480 con 0x563f0e8b36f0 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.472+0000 7f746e600640 1 -- 172.21.15.46:0/1903476108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474001630 con 0x563f0e8b36f0 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 -- 172.21.15.46:0/1903476108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 msgr2=0x563f0e866310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f747402add0 tx=0x7f747402e000 comp rx=0 tx=0).stop 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 -- 172.21.15.46:0/1903476108 shutdown_connections 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8960d0 0x563f0e86aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e896810 0x563f0e895b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 --2- 172.21.15.46:0/1903476108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e866310 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 -- 172.21.15.46:0/1903476108 >> 172.21.15.46:0/1903476108 conn(0x563f0e868580 msgr2=0x563f0e8b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 -- 172.21.15.46:0/1903476108 shutdown_connections 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.473+0000 7f748b85c740 1 -- 172.21.15.46:0/1903476108 wait complete. 2024-04-26T23:45:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.474+0000 7f748b85c740 1 -- start start 2024-04-26T23:45:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.474+0000 7f748b85c740 1 Processor -- start 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8a7850 0x563f0e883bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f0e86a4a0 con 0x563f0e896810 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f0e756c50 con 0x563f0e8b36f0 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563f0e869890 con 0x563f0e8a7850 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.475+0000 7f748b85c740 10 client.0 _ll_register_callbacks cb 0x563f0e7c6fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:05.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38206/0 (socket says 172.21.15.46:38206) 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 -- 172.21.15.46:0/2930622020 learned_addr learned my addr 172.21.15.46:0/2930622020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 -- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8a7850 msgr2=0x563f0e883bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7482800640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8a7850 0x563f0e883bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481400640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8a7850 0x563f0e883bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 -- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 msgr2=0x563f0e8a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 -- 172.21.15.46:0/2930622020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0e4a1310 con 0x563f0e896810 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7481e00640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f747402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e9df894b01bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.476+0000 7f7462a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474042040 con 0x563f0e896810 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f7474052480 con 0x563f0e896810 2024-04-26T23:45:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7474001480 con 0x563f0e896810 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f747405f050 con 0x563f0e896810 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 10 client.8049 fetch_fsmap learned FSMap version 82 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563f0e60e1b0 con 0x563f0e896810 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474001480 con 0x563f0e896810 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f7474062020 con 0x563f0e896810 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 10 client.8049 fetch_fsmap finished waiting for FSMap version 82 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 10 client.8049 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 20 client.8049 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.477+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8a7c30 0x563f0e86a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.478+0000 7f748b85c740 4 client.8049 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.478+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 --> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f0e7dd9c0 con 0x563f0e8a7c30 2024-04-26T23:45:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.478+0000 7f7481400640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8a7c30 0x563f0e86a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.478+0000 7f7481400640 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8a7c30 0x563f0e86a930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90d24c6b7a14188 server_cookie=664daed952ef46e8 in_seq=0 out_seq=0 2024-04-26T23:45:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.478+0000 7f7462a00640 10 client.8049 ms_handle_connect on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f7462a00640 1 -- 172.21.15.46:0/2930622020 <== mds.0 v2:172.21.15.46:6834/2779258602 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f0e7dd9c0 con 0x563f0e8a7c30 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f7462a00640 10 client.8049 handle_command_reply: tid=0 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 client.8049 shutdown 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 msgr2=0x563f0e8b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f747402c000 comp rx=0 tx=0).stop 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 shutdown_connections 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8a7c30 0x563f0e86a930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e8a7850 0x563f0e883bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930622020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e896810 0x563f0e8b0f90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f7462a00640 0 client.8049 ms_handle_reset on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.479+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 >> 172.21.15.46:0/2930622020 conn(0x563f0e8960d0 msgr2=0x563f0e865970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.480+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 shutdown_connections 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.480+0000 7f748b85c740 1 -- 172.21.15.46:0/2930622020 wait complete. 2024-04-26T23:45:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.480+0000 7f748b85c740 20 client.8049 trim_cache size 0 max 16384 2024-04-26T23:45:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.500+0000 7f748b85c740 1 Processor -- start 2024-04-26T23:45:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.500+0000 7f748b85c740 1 -- start start 2024-04-26T23:45:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e8b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f0e8673c0 con 0x563f0e8a7850 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7484202a20 con 0x563f0e8b36f0 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7484201a40 con 0x563f0e896810 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f7481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f7481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38222/0 (socket says 172.21.15.46:38222) 2024-04-26T23:45:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.501+0000 7f7481400640 1 -- 172.21.15.46:0/2930717877 learned_addr learned my addr 172.21.15.46:0/2930717877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481e00640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e8b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7482800640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 -- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 msgr2=0x563f0e8b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e8b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 -- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 msgr2=0x563f0e8ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 -- 172.21.15.46:0/2930717877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0e47a490 con 0x563f0e8a7850 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7482800640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.502+0000 7f7481400640 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8c99123efc37ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f746dc00640 1 -- 172.21.15.46:0/2930717877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470088070 con 0x563f0e8a7850 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f746dc00640 1 -- 172.21.15.46:0/2930717877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747005db60 con 0x563f0e8a7850 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f746dc00640 1 -- 172.21.15.46:0/2930717877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470062390 con 0x563f0e8a7850 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 -- 172.21.15.46:0/2930717877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 msgr2=0x563f0e892410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470060040 comp rx=0 tx=0).stop 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 -- 172.21.15.46:0/2930717877 shutdown_connections 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8b36f0 0x563f0e8ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8a7850 0x563f0e892410 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 --2- 172.21.15.46:0/2930717877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e8b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.503+0000 7f748b85c740 1 -- 172.21.15.46:0/2930717877 >> 172.21.15.46:0/2930717877 conn(0x563f0e8960d0 msgr2=0x563f0e869aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.504+0000 7f748b85c740 1 -- 172.21.15.46:0/2930717877 shutdown_connections 2024-04-26T23:45:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.504+0000 7f748b85c740 1 -- 172.21.15.46:0/2930717877 wait complete. 2024-04-26T23:45:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.504+0000 7f748b85c740 1 -- start start 2024-04-26T23:45:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.504+0000 7f748b85c740 1 Processor -- start 2024-04-26T23:45:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e926f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8a7850 0x563f0e8ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74842013f0 con 0x563f0e8b36f0 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7484200dd0 con 0x563f0e8a7850 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563f0e8673c0 con 0x563f0e896810 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.505+0000 7f748b85c740 10 client.0 _ll_register_callbacks cb 0x7f74680045c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38226/0 (socket says 172.21.15.46:38226) 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 -- 172.21.15.46:0/3010063878 learned_addr learned my addr 172.21.15.46:0/3010063878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7481e00640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e926f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7481400640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8a7850 0x563f0e8ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 -- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 msgr2=0x563f0e926f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e926f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 -- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8a7850 msgr2=0x563f0e8ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8a7850 0x563f0e8ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.506+0000 7f7482800640 1 -- 172.21.15.46:0/3010063878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0e8cf7e0 con 0x563f0e8b36f0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7482800640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f748408dfc0 tx=0x7f747c0125f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d196b93ab6696c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7462a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c01e020 con 0x563f0e8b36f0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f747c017420 con 0x563f0e8b36f0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747005db60 con 0x563f0e8b36f0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f747c012e50 con 0x563f0e8b36f0 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f748b85c740 10 client.8055 fetch_fsmap learned FSMap version 82 2024-04-26T23:45:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563f0e866660 con 0x563f0e8b36f0 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.507+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747005db60 con 0x563f0e8b36f0 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f747c02b080 con 0x563f0e8b36f0 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 10 client.8055 fetch_fsmap finished waiting for FSMap version 82 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 10 client.8055 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 20 client.8055 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8aed50 0x563f0e929cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 4 client.8055 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/testdir", "scrubops": ["recursive"]}] 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 --> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] -- command(tid 0: {"prefix": "scrub start", "path": "/testdir", "scrubops": ["recursive"]}) -- 0x563f0e7d7830 con 0x563f0e8aed50 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f7481e00640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8aed50 0x563f0e929cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f7481e00640 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8aed50 0x563f0e929cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96aa41c4acfdb0 server_cookie=858a426659f3b801 in_seq=0 out_seq=0 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.508+0000 7f7462a00640 10 client.8055 ms_handle_connect on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f7462a00640 1 -- 172.21.15.46:0/3010063878 <== mds.0 v2:172.21.15.46:6834/2779258602 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f0e7d7830 con 0x563f0e8aed50 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f7462a00640 10 client.8055 handle_command_reply: tid=0 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 client.8055 shutdown 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 msgr2=0x563f0e8caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f748408dfc0 tx=0x7f747c0125f0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 shutdown_connections 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0e8b36f0 0x563f0e8caca0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f0e8a7850 0x563f0e8ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.46:6834/2779258602,v1:172.21.15.46:6835/2779258602] conn(0x563f0e8aed50 0x563f0e929cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 --2- 172.21.15.46:0/3010063878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563f0e896810 0x563f0e926f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 >> 172.21.15.46:0/3010063878 conn(0x563f0e8960d0 msgr2=0x563f0e892460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.509+0000 7f7462a00640 0 client.8055 ms_handle_reset on v2:172.21.15.46:6834/2779258602 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.510+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 shutdown_connections 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.510+0000 7f748b85c740 1 -- 172.21.15.46:0/3010063878 wait complete. 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.510+0000 7f748b85c740 20 client.8055 trim_cache size 0 max 16384 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "40c921bf-d815-4f45-8b2a-a5dded4a1590", 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:45:05.516 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 msgr2=0x7f746406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 0x7f746406ee00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7474032000 tx=0x7f747402d000 comp rx=0 tx=0).stop 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 msgr2=0x7f748415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f747000d570 tx=0x7f747000da30 comp rx=0 tx=0).stop 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 shutdown_connections 2024-04-26T23:45:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f746406c940 0x7f746406ee00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f748415a980 0x7f748415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484150bc0 0x7f748415a030 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 --2- 172.21.15.46:0/853924415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74680039e0 0x7f74841614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 >> 172.21.15.46:0/853924415 conn(0x7f748409e450 msgr2=0x7f74840b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 shutdown_connections 2024-04-26T23:45:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.511+0000 7f7488a00640 1 -- 172.21.15.46:0/853924415 wait complete. 2024-04-26T23:45:05.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.726+0000 7f1a93e00640 1 Processor -- start 2024-04-26T23:45:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.726+0000 7f1a93e00640 1 -- start start 2024-04-26T23:45:05.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1a941466b0 0x7f1a9406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a940cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a94073ad0 con 0x7f1a94160910 2024-04-26T23:45:05.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1a94146a90 con 0x7f1a941466b0 2024-04-26T23:45:05.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1a94073d70 con 0x7f1a940720e0 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1a94160910 0x7f1a940cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35858/0 (socket says 172.21.15.46:35858) 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.728+0000 7f1a93400640 1 -- 172.21.15.46:0/3274306062 learned_addr learned my addr 172.21.15.46:0/3274306062 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a92a00640 1 --2- 172.21.15.46:0/3274306062 >> v2:172.21.15.142:3300/0 conn(0x7f1a941466b0 0x7f1a9406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a92000640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a940cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512043551 0 0) 0x7f1a94073ad0 con 0x7f1a94160910 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a78003520 con 0x7f1a94160910 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 176702945 0 0) 0x7f1a78003520 con 0x7f1a94160910 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 msgr2=0x7f1a940cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a940cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 >> v2:172.21.15.142:3300/0 conn(0x7f1a941466b0 msgr2=0x7f1a9406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 --2- 172.21.15.46:0/3274306062 >> v2:172.21.15.142:3300/0 conn(0x7f1a941466b0 0x7f1a9406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a94074050 con 0x7f1a94160910 2024-04-26T23:45:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1a8c0019f0 con 0x7f1a94160910 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 >> v1:172.21.15.46:6789/0 conn(0x7f1a94160910 legacy=0x7f1a940cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780039b0 0x7f1a78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a78004d40 0x7f1a780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94146a90 con 0x7f1a78004d40 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94073d70 con 0x7f1a780039b0 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.729+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7800b9c0 con 0x7f1a940720e0 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a78004d40 0x7f1a780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a92000640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a92a00640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780039b0 0x7f1a78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 -- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 msgr2=0x7f1a78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a940720e0 0x7f1a78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 -- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780039b0 msgr2=0x7f1a78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780039b0 0x7f1a78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.730+0000 7f1a93400640 1 -- 172.21.15.46:0/3274306062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a78003520 con 0x7f1a78004d40 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.731+0000 7f1a93400640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a78004d40 0x7f1a780051c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00a8d0 tx=0x7f1a78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb64d772a59fffb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00a4c0 con 0x7f1a78004d40 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c02f070 con 0x7f1a78004d40 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a91600640 1 -- 172.21.15.46:0/3274306062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c02bd00 con 0x7f1a78004d40 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a93e00640 1 -- 172.21.15.46:0/3274306062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a78004d40 msgr2=0x7f1a780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a93e00640 1 --2- 172.21.15.46:0/3274306062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a78004d40 0x7f1a780051c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00a8d0 tx=0x7f1a78005830 comp rx=0 tx=0).stop 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a93400640 1 -- 172.21.15.46:0/3274306062 reap_dead start 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a93e00640 1 -- 172.21.15.46:0/3274306062 shutdown_connections 2024-04-26T23:45:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.732+0000 7f1a93e00640 1 -- 172.21.15.46:0/3274306062 >> 172.21.15.46:0/3274306062 conn(0x7f1a940b26d0 msgr2=0x7f1a940b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.733+0000 7f1a93e00640 1 -- 172.21.15.46:0/3274306062 shutdown_connections 2024-04-26T23:45:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.733+0000 7f1a93e00640 1 -- 172.21.15.46:0/3274306062 wait complete. 2024-04-26T23:45:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.734+0000 7f1a93e00640 1 Processor -- start 2024-04-26T23:45:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.735+0000 7f1a93e00640 1 -- start start 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.735+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a78004d40 0x7f1a9413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a94143cf0 0x7f1a94144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9406e3c0 con 0x7f1a940ced10 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a940cb470 con 0x7f1a94143cf0 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1a940cc0a0 con 0x7f1a78004d40 2024-04-26T23:45:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38242/0 (socket says 172.21.15.46:38242) 2024-04-26T23:45:05.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92000640 1 -- 172.21.15.46:0/1831750490 learned_addr learned my addr 172.21.15.46:0/1831750490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92a00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a78004d40 0x7f1a9413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a93400640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a94143cf0 0x7f1a94144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92000640 1 -- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a78004d40 msgr2=0x7f1a9413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.736+0000 7f1a92000640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a78004d40 0x7f1a9413f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a92000640 1 -- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a94143cf0 msgr2=0x7f1a94144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a92000640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a94143cf0 0x7f1a94144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a92000640 1 -- 172.21.15.46:0/1831750490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a941446b0 con 0x7f1a940ced10 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a92000640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f1a800119a0 tx=0x7f1a80011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000faf0 con 0x7f1a940ced10 2024-04-26T23:45:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8000fc90 con 0x7f1a940ced10 2024-04-26T23:45:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80014440 con 0x7f1a940ced10 2024-04-26T23:45:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a94144970 con 0x7f1a940ced10 2024-04-26T23:45:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.737+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a94144f30 con 0x7f1a940ced10 2024-04-26T23:45:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.738+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a940cf6d0 con 0x7f1a940ced10 2024-04-26T23:45:05.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.740+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a80012040 con 0x7f1a940ced10 2024-04-26T23:45:05.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.740+0000 7f1a8b400640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:05.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.741+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f1a8008c4f0 con 0x7f1a940ced10 2024-04-26T23:45:05.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.741+0000 7f1a92a00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.742+0000 7f1a92a00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 0x7f1a7406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009a10 tx=0x7f1a84008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:05.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.745+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a80019030 con 0x7f1a940ced10 2024-04-26T23:45:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.993+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a94145170 con 0x7f1a940ced10 2024-04-26T23:45:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.995+0000 7f1a8b400640 1 -- 172.21.15.46:0/1831750490 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f1a94145170 con 0x7f1a940ced10 2024-04-26T23:45:05.998 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:05.998 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 msgr2=0x7f1a7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 0x7f1a7406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009a10 tx=0x7f1a84008040 comp rx=0 tx=0).stop 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 msgr2=0x7f1a940cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f1a800119a0 tx=0x7f1a80011e60 comp rx=0 tx=0).stop 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 shutdown_connections 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a94143cf0 0x7f1a94144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940ced10 0x7f1a940cf190 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 --2- 172.21.15.46:0/1831750490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a78004d40 0x7f1a9413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.997+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 >> 172.21.15.46:0/1831750490 conn(0x7f1a940b26d0 msgr2=0x7f1a9406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.998+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 shutdown_connections 2024-04-26T23:45:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:05.998+0000 7f1a93e00640 1 -- 172.21.15.46:0/1831750490 wait complete. 2024-04-26T23:45:06.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:45:06.246 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:44:52.577168+0000","btime":"2024-04-26T23:44:51.353182+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:44:52.577168+0000","atime":"2024-04-26T23:44:51.353182+0000","time_warp_seq":0,"change_attr":21,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:44:52.577168+0000","num_files":21,"num_subdirs":0,"change_attr":21},"rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"accounted_rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"version":128,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"authpin":0,"scrubqueue":0},"nref":1,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:45:06.247 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:44:51.353182+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:44:51.353182+0000", "change_attr": 21, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:44:52.577168+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 21, "mtime": "2024-04-26T23:44:52.577168+0000", "num_files": 21, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:44:52.577168+0000", "nestlock": {}, "nlink": 1, "nref": 1, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "authpin": 0, "dirfrag": 1, "scrubqueue": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 128, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:45:06.247 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-04-26T23:45:09.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:09.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:11.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.461+0000 7f2739600640 1 Processor -- start 2024-04-26T23:45:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.462+0000 7f2739600640 1 -- start start 2024-04-26T23:45:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.463+0000 7f2739600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f273415edf0 0x7f27340cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.463+0000 7f2739600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f273415f1d0 0x7f27340cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.463+0000 7f2739600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27340b03c0 con 0x7f27340cb1a0 2024-04-26T23:45:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.463+0000 7f2739600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f27340cf5c0 con 0x7f273415edf0 2024-04-26T23:45:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.463+0000 7f2739600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f27340cb680 con 0x7f273415f1d0 2024-04-26T23:45:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f2733e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f273415edf0 0x7f27340cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f2733e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f273415edf0 0x7f27340cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38252/0 (socket says 172.21.15.46:38252) 2024-04-26T23:45:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f2733e00640 1 -- 172.21.15.46:0/3171079976 learned_addr learned my addr 172.21.15.46:0/3171079976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f272b400640 1 --2- 172.21.15.46:0/3171079976 >> v2:172.21.15.142:3300/0 conn(0x7f273415f1d0 0x7f27340cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f2733e00640 1 -- 172.21.15.46:0/3171079976 >> v2:172.21.15.142:3300/0 conn(0x7f273415f1d0 msgr2=0x7f27340cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.464+0000 7f2733e00640 1 --2- 172.21.15.46:0/3171079976 >> v2:172.21.15.142:3300/0 conn(0x7f273415f1d0 0x7f27340cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.465+0000 7f2733e00640 1 -- 172.21.15.46:0/3171079976 >> v1:172.21.15.142:6789/0 conn(0x7f27340cb1a0 legacy=0x7f27340ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.465+0000 7f2733e00640 1 -- 172.21.15.46:0/3171079976 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340cb960 con 0x7f273415edf0 2024-04-26T23:45:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.466+0000 7f2733e00640 1 --2- 172.21.15.46:0/3171079976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273415edf0 0x7f27340cf080 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f27340d0a00 tx=0x7f272402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecbc9bdcf253e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.466+0000 7f2733400640 1 -- 172.21.15.46:0/3171079976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724037070 con 0x7f273415edf0 2024-04-26T23:45:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.466+0000 7f2733400640 1 -- 172.21.15.46:0/3171079976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2724032440 con 0x7f273415edf0 2024-04-26T23:45:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.466+0000 7f2733400640 1 -- 172.21.15.46:0/3171079976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724032810 con 0x7f273415edf0 2024-04-26T23:45:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 -- 172.21.15.46:0/3171079976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273415edf0 msgr2=0x7f27340cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 --2- 172.21.15.46:0/3171079976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273415edf0 0x7f27340cf080 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f27340d0a00 tx=0x7f272402a2c0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 -- 172.21.15.46:0/3171079976 shutdown_connections 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 --2- 172.21.15.46:0/3171079976 >> v2:172.21.15.142:3300/0 conn(0x7f273415f1d0 0x7f27340cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 --2- 172.21.15.46:0/3171079976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273415edf0 0x7f27340cf080 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 -- 172.21.15.46:0/3171079976 >> 172.21.15.46:0/3171079976 conn(0x7f27340b26d0 msgr2=0x7f27340b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 -- 172.21.15.46:0/3171079976 shutdown_connections 2024-04-26T23:45:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.467+0000 7f2739600640 1 -- 172.21.15.46:0/3171079976 wait complete. 2024-04-26T23:45:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.469+0000 7f2739600640 1 Processor -- start 2024-04-26T23:45:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.469+0000 7f2739600640 1 -- start start 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27340cb1a0 0x7f27340c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f273415edf0 0x7f273415fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27340cc200 con 0x7f27341513a0 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27340cbf20 con 0x7f27340cb1a0 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2739600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27340c6950 con 0x7f273415edf0 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2738c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2738c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38268/0 (socket says 172.21.15.46:38268) 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.470+0000 7f2738c00640 1 -- 172.21.15.46:0/4041592324 learned_addr learned my addr 172.21.15.46:0/4041592324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2733e00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27340cb1a0 0x7f27340c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f272b400640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f273415edf0 0x7f273415fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 -- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f273415edf0 msgr2=0x7f273415fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f273415edf0 0x7f273415fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 -- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27340cb1a0 msgr2=0x7f27340c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27340cb1a0 0x7f27340c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 -- 172.21.15.46:0/4041592324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340c83c0 con 0x7f27341513a0 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2738c00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c6d0 tx=0x7f272c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c016020 con 0x7f27341513a0 2024-04-26T23:45:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.471+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c00a840 con 0x7f27341513a0 2024-04-26T23:45:11.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.472+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c00ac70 con 0x7f27341513a0 2024-04-26T23:45:11.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.472+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27340c86a0 con 0x7f27341513a0 2024-04-26T23:45:11.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.472+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273415d540 con 0x7f27341513a0 2024-04-26T23:45:11.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.473+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273415d740 con 0x7f27341513a0 2024-04-26T23:45:11.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.474+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f272c019020 con 0x7f27341513a0 2024-04-26T23:45:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.475+0000 7f2730c00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 0x7f271006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.475+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f272c08c360 con 0x7f27341513a0 2024-04-26T23:45:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.476+0000 7f2733e00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 0x7f271006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.477+0000 7f2733e00640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 0x7f271006edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2724034000 tx=0x7f272402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:11.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.480+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f273415d740 con 0x7f27341513a0 2024-04-26T23:45:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.724+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27340af9e0 con 0x7f27341513a0 2024-04-26T23:45:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.726+0000 7f2730c00640 1 -- 172.21.15.46:0/4041592324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f272c020020 con 0x7f27341513a0 2024-04-26T23:45:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:11.731 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.728+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 msgr2=0x7f271006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.728+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 0x7f271006edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2724034000 tx=0x7f272402e000 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.728+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 msgr2=0x7f27340c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c6d0 tx=0x7f272c00cb90 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 shutdown_connections 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f271006c8f0 0x7f271006edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27341513a0 0x7f27340c95c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f273415edf0 0x7f273415fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 --2- 172.21.15.46:0/4041592324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27340cb1a0 0x7f27340c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 >> 172.21.15.46:0/4041592324 conn(0x7f27340b26d0 msgr2=0x7f27340b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 shutdown_connections 2024-04-26T23:45:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:11.729+0000 7f2739600640 1 -- 172.21.15.46:0/4041592324 wait complete. 2024-04-26T23:45:11.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1099511627776 2024-04-26T23:45:11.987 INFO:teuthology.orchestra.run.smithi046.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2024-04-26T23:44:52.577168+0000","btime":"2024-04-26T23:44:51.353182+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T23:44:52.577168+0000","atime":"2024-04-26T23:44:51.353182+0000","time_warp_seq":0,"change_attr":21,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T23:44:52.577168+0000","num_files":21,"num_subdirs":0,"change_attr":21},"rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"accounted_rstat":{"version":0,"rbytes":32,"rfiles":21,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T23:44:52.580168+0000"},"version":131,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"2024-04-26T23:45:05.510918+0000","last_scrub_version":128,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[{"frag":"*","children":1}]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"lock":0,"dirfrag":2,"dirtyrstat":1,"dirty":1,"authpin":0,"scrubqueue":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyrstat"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2024-04-26T23:45:11.988 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T23:44:51.353182+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T23:44:51.353182+0000", "change_attr": 21, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T23:44:52.577168+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [ { "children": 1, "frag": "*" } ] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 21, "mtime": "2024-04-26T23:44:52.577168+0000", "num_files": 21, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "2024-04-26T23:45:05.510918+0000", "last_scrub_version": 128, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T23:44:52.577168+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "authpin": 0, "dirfrag": 2, "dirty": 1, "dirtyrstat": 1, "lock": 0, "scrubqueue": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 32, "rctime": "2024-04-26T23:44:52.580168+0000", "rfiles": 21, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyrstat" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 131, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } 2024-04-26T23:45:11.988 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T23:45:11.988 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:45:11.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:45:12.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:45:12.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:45:12.215 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:45:12.215 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:12.215 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:12.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:45:12.275 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:45:12.276 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:45:12.276 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:12.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:45:12.334 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:45:12.334 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:12.334 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:12.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:45:12.396 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:45:12.396 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:45:12.396 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:12.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:45:12.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.669+0000 7fc6a0a00640 1 Processor -- start 2024-04-26T23:45:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.670+0000 7fc6a0a00640 1 -- start start 2024-04-26T23:45:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc69c15f4e0 0x7fc69c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 0x7fc69c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 0x7fc69c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c0d2990 con 0x7fc69c15f4e0 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c0d11f0 con 0x7fc69c0d1f60 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc6a0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0d5f20 con 0x7fc69c0b4000 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc69a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc69c15f4e0 0x7fc69c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc69a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc69c15f4e0 0x7fc69c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36672/0 (socket says 172.21.15.46:36672) 2024-04-26T23:45:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.671+0000 7fc69a800640 1 -- 172.21.15.46:0/2574200321 learned_addr learned my addr 172.21.15.46:0/2574200321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc699e00640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 0x7fc69c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69b200640 1 --2- 172.21.15.46:0/2574200321 >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 0x7fc69c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69a800640 1 -- 172.21.15.46:0/2574200321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 msgr2=0x7fc69c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69a800640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 0x7fc69c0d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69a800640 1 -- 172.21.15.46:0/2574200321 >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 msgr2=0x7fc69c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69a800640 1 --2- 172.21.15.46:0/2574200321 >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 0x7fc69c0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69a800640 1 -- 172.21.15.46:0/2574200321 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0d0f10 con 0x7fc69c15f4e0 2024-04-26T23:45:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc69b200640 1 --2- 172.21.15.46:0/2574200321 >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 0x7fc69c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.672+0000 7fc699e00640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 0x7fc69c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.673+0000 7fc69a800640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c15f4e0 0x7fc69c0d2df0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0bb0c0 tx=0x7fc68402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a086734133e2fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.673+0000 7fc699400640 1 -- 172.21.15.46:0/2574200321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684037070 con 0x7fc69c15f4e0 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc699400640 1 -- 172.21.15.46:0/2574200321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc684032440 con 0x7fc69c15f4e0 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc699400640 1 -- 172.21.15.46:0/2574200321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684032810 con 0x7fc69c15f4e0 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 -- 172.21.15.46:0/2574200321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c15f4e0 msgr2=0x7fc69c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c15f4e0 0x7fc69c0d2df0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0bb0c0 tx=0x7fc68402a1d0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 -- 172.21.15.46:0/2574200321 shutdown_connections 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/2574200321 >> v2:172.21.15.142:3300/0 conn(0x7fc69c0d1f60 0x7fc69c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0b4000 0x7fc69c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/2574200321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c15f4e0 0x7fc69c0d2df0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 -- 172.21.15.46:0/2574200321 >> 172.21.15.46:0/2574200321 conn(0x7fc69c0d5010 msgr2=0x7fc69c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 -- 172.21.15.46:0/2574200321 shutdown_connections 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.674+0000 7fc6a0a00640 1 -- 172.21.15.46:0/2574200321 wait complete. 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.676+0000 7fc6a0a00640 1 Processor -- start 2024-04-26T23:45:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.676+0000 7fc6a0a00640 1 -- start start 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0d1f60 0x7fc69c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c15f4e0 0x7fc69c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c15f8c0 con 0x7fc69c0b4000 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0b1d60 con 0x7fc69c15f4e0 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc6a0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0d0210 con 0x7fc69c0d1f60 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc69a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc69a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36684/0 (socket says 172.21.15.46:36684) 2024-04-26T23:45:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.677+0000 7fc69a800640 1 -- 172.21.15.46:0/3629689392 learned_addr learned my addr 172.21.15.46:0/3629689392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:12.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc699e00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0d1f60 0x7fc69c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69b200640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c15f4e0 0x7fc69c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 -- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0d1f60 msgr2=0x7fc69c0b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0d1f60 0x7fc69c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 -- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c15f4e0 msgr2=0x7fc69c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c15f4e0 0x7fc69c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 -- 172.21.15.46:0/3629689392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0b5540 con 0x7fc69c0b4000 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.678+0000 7fc69a800640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc68402a210 tx=0x7fc6840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.679+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684037040 con 0x7fc69c0b4000 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.679+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc684042070 con 0x7fc69c0b4000 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.679+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684030bb0 con 0x7fc69c0b4000 2024-04-26T23:45:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.679+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c15bce0 con 0x7fc69c0b4000 2024-04-26T23:45:12.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.679+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0b9430 con 0x7fc69c0b4000 2024-04-26T23:45:12.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.680+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c0b9220 con 0x7fc69c0b4000 2024-04-26T23:45:12.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.681+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc684047460 con 0x7fc69c0b4000 2024-04-26T23:45:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.682+0000 7fc692a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.682+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7fc6840acac0 con 0x7fc69c0b4000 2024-04-26T23:45:12.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.683+0000 7fc699e00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:12.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.684+0000 7fc699e00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 0x7fc67806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006fd0 tx=0x7fc68c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:12.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.686+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc684045360 con 0x7fc69c0b4000 2024-04-26T23:45:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.931+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69c0693b0 con 0x7fc69c0b4000 2024-04-26T23:45:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.932+0000 7fc692a00640 1 -- 172.21.15.46:0/3629689392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7fc68407aba0 con 0x7fc69c0b4000 2024-04-26T23:45:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:12.937 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 msgr2=0x7fc67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 0x7fc67806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006fd0 tx=0x7fc68c008040 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 msgr2=0x7fc69c14f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc68402a210 tx=0x7fc6840046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 shutdown_connections 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c15f4e0 0x7fc69c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc69c0d1f60 0x7fc69c0b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 --2- 172.21.15.46:0/3629689392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc69c0b4000 0x7fc69c14f2a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 >> 172.21.15.46:0/3629689392 conn(0x7fc69c0d5010 msgr2=0x7fc69c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 shutdown_connections 2024-04-26T23:45:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:12.935+0000 7fc6a0a00640 1 -- 172.21.15.46:0/3629689392 wait complete. 2024-04-26T23:45:12.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.169+0000 7f3baec00640 1 Processor -- start 2024-04-26T23:45:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.169+0000 7f3baec00640 1 -- start start 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3baec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3ba80d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3baec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba80cf950 con 0x7f3ba815ccd0 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3baec00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba80cfc30 con 0x7f3ba8072e60 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3baec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba80d0c50 con 0x7f3ba8145770 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3bacc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ba815ccd0 0x7f3ba80b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51836/0 (socket says 172.21.15.46:51836) 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3bacc00640 1 -- 172.21.15.46:0/2623931345 learned_addr learned my addr 172.21.15.46:0/2623931345 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.171+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3ba80d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982665492 0 0) 0x7f3ba80cf950 con 0x7f3ba815ccd0 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b88003520 con 0x7f3ba815ccd0 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183037954 0 0) 0x7f3ba80cfc30 con 0x7f3ba8072e60 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba80cf950 con 0x7f3ba8072e60 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2351160603 0 0) 0x7f3b88003520 con 0x7f3ba815ccd0 2024-04-26T23:45:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 msgr2=0x7f3ba80d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3ba80d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 >> v1:172.21.15.142:6789/0 conn(0x7f3ba8072e60 legacy=0x7f3ba815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba80cff10 con 0x7f3ba815ccd0 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3ba0001670 con 0x7f3ba815ccd0 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 >> v1:172.21.15.46:6789/0 conn(0x7f3ba815ccd0 legacy=0x7f3ba80b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3b88003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3b88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b88004d40 0x7f3b880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba80d0c50 con 0x7f3b88004120 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8800bb00 con 0x7f3b88004d40 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.172+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8800bd00 con 0x7f3ba8145770 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3b88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3b88003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 -- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 msgr2=0x7f3b88003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3b88003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 -- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b88004d40 msgr2=0x7f3b880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b88004d40 0x7f3b880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7400640 1 -- 172.21.15.46:0/2623931345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b88003520 con 0x7f3b88004120 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.173+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba8145770 0x7f3b88003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.174+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3b88004800 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b90001fd0 tx=0x7f3b88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c645e7dd15e8bab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.174+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90001bf0 con 0x7f3b88004120 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.174+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b90003b70 con 0x7f3b88004120 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3ba6a00640 1 -- 172.21.15.46:0/2623931345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900050d0 con 0x7f3b88004120 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 -- 172.21.15.46:0/2623931345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 msgr2=0x7f3b88004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 --2- 172.21.15.46:0/2623931345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3b88004800 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b90001fd0 tx=0x7f3b88005830 comp rx=0 tx=0).stop 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3bacc00640 1 -- 172.21.15.46:0/2623931345 reap_dead start 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 -- 172.21.15.46:0/2623931345 shutdown_connections 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 -- 172.21.15.46:0/2623931345 >> 172.21.15.46:0/2623931345 conn(0x7f3ba806c530 msgr2=0x7f3ba806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 -- 172.21.15.46:0/2623931345 shutdown_connections 2024-04-26T23:45:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.175+0000 7f3baec00640 1 -- 172.21.15.46:0/2623931345 wait complete. 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.177+0000 7f3baec00640 1 Processor -- start 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.177+0000 7f3baec00640 1 -- start start 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba8142680 0x7f3ba80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba813d030 0x7f3ba813a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba80cf5b0 con 0x7f3b88004120 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba815d490 con 0x7f3ba8142680 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.178+0000 7f3baec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba80cf730 con 0x7f3ba813d030 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36700/0 (socket says 172.21.15.46:36700) 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 -- 172.21.15.46:0/2243504824 learned_addr learned my addr 172.21.15.46:0/2243504824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba8142680 0x7f3ba80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3bacc00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba813d030 0x7f3ba813a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 -- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba813d030 msgr2=0x7f3ba813a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba813d030 0x7f3ba813a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 -- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba8142680 msgr2=0x7f3ba80b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba8142680 0x7f3ba80b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.179+0000 7f3ba7e00640 1 -- 172.21.15.46:0/2243504824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba813c630 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3b980108e0 tx=0x7f3b98010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800f040 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9800ec20 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801c400 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba8142b00 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.180+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba80b09a0 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.181+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba8069530 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.182+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b9801c5a0 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.183+0000 7f3b9fe00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 0x7f3b8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.183+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f3b9808c4f0 con 0x7f3b88004120 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.184+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 0x7f3b8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.185+0000 7f3ba7400640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 0x7f3b8406ede0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3b90032000 tx=0x7f3b9002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.188+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b98018030 con 0x7f3b88004120 2024-04-26T23:45:13.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.397+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ba80b6060 con 0x7f3b88004120 2024-04-26T23:45:13.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.401+0000 7f3b9fe00640 1 -- 172.21.15.46:0/2243504824 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+17303 (secure 0 0 0) 0x7f3b98059f80 con 0x7f3b88004120 2024-04-26T23:45:13.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:13.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:57.986395+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.142:6834/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.46:6835/2670265715":"2024-04-27T23:44:56.993227+0000","172.21.15.46:6833/2670265715":"2024-04-27T23:44:56.993227+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 msgr2=0x7f3b8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 0x7f3b8406ede0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3b90032000 tx=0x7f3b9002f000 comp rx=0 tx=0).stop 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 msgr2=0x7f3ba813bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3b980108e0 tx=0x7f3b98010da0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 shutdown_connections 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8406c920 0x7f3b8406ede0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba813d030 0x7f3ba813a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba8142680 0x7f3ba80b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 --2- 172.21.15.46:0/2243504824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b88004120 0x7f3ba813bc70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 >> 172.21.15.46:0/2243504824 conn(0x7f3ba806c530 msgr2=0x7f3ba806e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.403+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 shutdown_connections 2024-04-26T23:45:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.404+0000 7f3baec00640 1 -- 172.21.15.46:0/2243504824 wait complete. 2024-04-26T23:45:13.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:44:40.164396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:44:41.287502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:45:13.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:45:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.633+0000 7f52a7200640 1 Processor -- start 2024-04-26T23:45:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.634+0000 7f52a7200640 1 -- start start 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.635+0000 7f52a7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f52a80ce7d0 0x7f52a80cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a8071e80 con 0x7f52a8073af0 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a8072160 con 0x7f52a81640d0 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f52a80cb4f0 con 0x7f52a80ce7d0 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52a8073af0 0x7f52a80cb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51838/0 (socket says 172.21.15.46:51838) 2024-04-26T23:45:13.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a5400640 1 -- 172.21.15.46:0/1842766936 learned_addr learned my addr 172.21.15.46:0/1842766936 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:13.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577216526 0 0) 0x7f52a8071e80 con 0x7f52a8073af0 2024-04-26T23:45:13.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a6800640 1 --2- 172.21.15.46:0/1842766936 >> v2:172.21.15.142:3300/0 conn(0x7f52a80ce7d0 0x7f52a80cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.636+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5288003520 con 0x7f52a8073af0 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2332877953 0 0) 0x7f5288003520 con 0x7f52a8073af0 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 >> v2:172.21.15.142:3300/0 conn(0x7f52a80ce7d0 msgr2=0x7f52a80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 --2- 172.21.15.46:0/1842766936 >> v2:172.21.15.142:3300/0 conn(0x7f52a80ce7d0 0x7f52a80cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 >> v1:172.21.15.142:6789/0 conn(0x7f52a81640d0 legacy=0x7f52a81644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a8072440 con 0x7f52a8073af0 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f52900019f0 con 0x7f52a8073af0 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 >> v1:172.21.15.46:6789/0 conn(0x7f52a8073af0 legacy=0x7f52a80cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5288003a00 0x7f5288003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5288005210 0x7f528800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80cb4f0 con 0x7f5288004800 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f528800c160 con 0x7f5288003a00 2024-04-26T23:45:13.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f528800c360 con 0x7f5288005210 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.637+0000 7f52a5400640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5e00640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5288003a00 0x7f5288003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a6800640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5288005210 0x7f528800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5400640 1 -- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5288005210 msgr2=0x7f528800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5400640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5288005210 0x7f528800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5400640 1 -- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5288003a00 msgr2=0x7f5288003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5400640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5288003a00 0x7f5288003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a5400640 1 -- 172.21.15.46:0/1842766936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5288003520 con 0x7f5288004800 2024-04-26T23:45:13.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.638+0000 7f52a6800640 1 -- 172.21.15.46:0/1842766936 reap_dead start 2024-04-26T23:45:13.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.639+0000 7f52a5400640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f5288004c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f529000a7b0 tx=0x7f5288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff4f639d6e31d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.639+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000a3d0 con 0x7f5288004800 2024-04-26T23:45:13.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.639+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529002f070 con 0x7f5288004800 2024-04-26T23:45:13.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.639+0000 7f52a4a00640 1 -- 172.21.15.46:0/1842766936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529002bc80 con 0x7f5288004800 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 -- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 msgr2=0x7f5288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f5288004c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f529000a7b0 tx=0x7f5288005830 comp rx=0 tx=0).stop 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 -- 172.21.15.46:0/1842766936 shutdown_connections 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 --2- 172.21.15.46:0/1842766936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 -- 172.21.15.46:0/1842766936 >> 172.21.15.46:0/1842766936 conn(0x7f52a80b27b0 msgr2=0x7f52a80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 -- 172.21.15.46:0/1842766936 shutdown_connections 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.640+0000 7f52a7200640 1 -- 172.21.15.46:0/1842766936 wait complete. 2024-04-26T23:45:13.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.642+0000 7f52a7200640 1 Processor -- start 2024-04-26T23:45:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.642+0000 7f52a7200640 1 -- start start 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52a80c46c0 0x7f52a8162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 0x7f52a80b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80cedc0 con 0x7f5288004800 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80cec40 con 0x7f52a80b4e90 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f52a80cef40 con 0x7f52a80c46c0 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36714/0 (socket says 172.21.15.46:36714) 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a5e00640 1 -- 172.21.15.46:0/2447740968 learned_addr learned my addr 172.21.15.46:0/2447740968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.643+0000 7f52a5400640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52a80c46c0 0x7f52a8162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a6800640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 0x7f52a80b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 -- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52a80c46c0 msgr2=0x7f52a8162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52a80c46c0 0x7f52a8162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 -- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 msgr2=0x7f52a80b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 0x7f52a80b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 -- 172.21.15.46:0/2447740968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a80c4b50 con 0x7f5288004800 2024-04-26T23:45:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a6800640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 0x7f52a80b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a5e00640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f529800e580 tx=0x7f529800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980052e0 con 0x7f5288004800 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298018070 con 0x7f5288004800 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298003810 con 0x7f5288004800 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a80c4e30 con 0x7f5288004800 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.644+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a80c53f0 con 0x7f5288004800 2024-04-26T23:45:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.646+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a80693b0 con 0x7f5288004800 2024-04-26T23:45:13.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.647+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52980039b0 con 0x7f5288004800 2024-04-26T23:45:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.648+0000 7f529e000640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 0x7f528406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.648+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f529808bb80 con 0x7f5288004800 2024-04-26T23:45:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.648+0000 7f52a5400640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 0x7f528406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.649+0000 7f52a5400640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 0x7f528406ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f529000a780 tx=0x7f5290009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.652+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5298059c70 con 0x7f5288004800 2024-04-26T23:45:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.863+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52a80ccd90 con 0x7f5288004800 2024-04-26T23:45:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.864+0000 7f529e000640 1 -- 172.21.15.46:0/2447740968 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f5298059570 con 0x7f5288004800 2024-04-26T23:45:13.869 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:45:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 msgr2=0x7f528406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 0x7f528406ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f529000a780 tx=0x7f5290009030 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 msgr2=0x7f52a80c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f529800e580 tx=0x7f529800ea40 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 shutdown_connections 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f528406c920 0x7f528406ede0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80b4e90 0x7f52a80b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52a80c46c0 0x7f52a8162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 --2- 172.21.15.46:0/2447740968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288004800 0x7f52a80c3760 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 >> 172.21.15.46:0/2447740968 conn(0x7f52a80b27b0 msgr2=0x7f52a80b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 shutdown_connections 2024-04-26T23:45:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:13.867+0000 7f52a7200640 1 -- 172.21.15.46:0/2447740968 wait complete. 2024-04-26T23:45:13.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:45:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.097+0000 7f6a1cc00640 1 Processor -- start 2024-04-26T23:45:14.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.098+0000 7f6a1cc00640 1 -- start start 2024-04-26T23:45:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a181626c0 0x7f6a180b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a180d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a18162aa0 0x7f6a180d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6a180d07d0 con 0x7f6a18162aa0 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6a180d0ab0 con 0x7f6a181626c0 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a1cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180d0d90 con 0x7f6a180b5fb0 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a17e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a18162aa0 0x7f6a180d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a17e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a18162aa0 0x7f6a180d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36724/0 (socket says 172.21.15.46:36724) 2024-04-26T23:45:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.099+0000 7f6a17e00640 1 -- 172.21.15.46:0/2620116529 learned_addr learned my addr 172.21.15.46:0/2620116529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17400640 1 --2- 172.21.15.46:0/2620116529 >> v2:172.21.15.142:3300/0 conn(0x7f6a181626c0 0x7f6a180b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a180d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17e00640 1 -- 172.21.15.46:0/2620116529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 msgr2=0x7f6a180d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17e00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a180d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17e00640 1 -- 172.21.15.46:0/2620116529 >> v2:172.21.15.142:3300/0 conn(0x7f6a181626c0 msgr2=0x7f6a180b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17e00640 1 --2- 172.21.15.46:0/2620116529 >> v2:172.21.15.142:3300/0 conn(0x7f6a181626c0 0x7f6a180b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.100+0000 7f6a17e00640 1 -- 172.21.15.46:0/2620116529 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a180d2c80 con 0x7f6a18162aa0 2024-04-26T23:45:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.101+0000 7f6a17e00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180d1900 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6a1808f2e0 tx=0x7f6a1002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbdff05fe6f9e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.101+0000 7f6a16a00640 1 -- 172.21.15.46:0/2620116529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10036070 con 0x7f6a18162aa0 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.101+0000 7f6a16a00640 1 -- 172.21.15.46:0/2620116529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a10031600 con 0x7f6a18162aa0 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.101+0000 7f6a16a00640 1 -- 172.21.15.46:0/2620116529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100319d0 con 0x7f6a18162aa0 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 -- 172.21.15.46:0/2620116529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 msgr2=0x7f6a180d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180d1900 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6a1808f2e0 tx=0x7f6a1002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 -- 172.21.15.46:0/2620116529 shutdown_connections 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180d1900 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/2620116529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a180d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/2620116529 >> v2:172.21.15.142:3300/0 conn(0x7f6a181626c0 0x7f6a180b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 -- 172.21.15.46:0/2620116529 >> 172.21.15.46:0/2620116529 conn(0x7f6a1809e2a0 msgr2=0x7f6a180d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 -- 172.21.15.46:0/2620116529 shutdown_connections 2024-04-26T23:45:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.102+0000 7f6a1cc00640 1 -- 172.21.15.46:0/2620116529 wait complete. 2024-04-26T23:45:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.104+0000 7f6a1cc00640 1 Processor -- start 2024-04-26T23:45:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.104+0000 7f6a1cc00640 1 -- start start 2024-04-26T23:45:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a1815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a181626c0 0x7f6a181f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180d1fc0 con 0x7f6a18162aa0 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180d1e40 con 0x7f6a181626c0 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a1cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180d2140 con 0x7f6a180b5fb0 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36730/0 (socket says 172.21.15.46:36730) 2024-04-26T23:45:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a17e00640 1 -- 172.21.15.46:0/3901464879 learned_addr learned my addr 172.21.15.46:0/3901464879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:14.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.105+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a181626c0 0x7f6a181f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17400640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a1815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 -- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 msgr2=0x7f6a1815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a1815e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 -- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a181626c0 msgr2=0x7f6a181f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a181626c0 0x7f6a181f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 -- 172.21.15.46:0/3901464879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a181f5ad0 con 0x7f6a18162aa0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a17e00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6a10034040 tx=0x7f6a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10036050 con 0x7f6a18162aa0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a10042070 con 0x7f6a18162aa0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10009510 con 0x7f6a18162aa0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.106+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a181f5db0 con 0x7f6a18162aa0 2024-04-26T23:45:14.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.107+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a180b8640 con 0x7f6a18162aa0 2024-04-26T23:45:14.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.108+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a180693b0 con 0x7f6a18162aa0 2024-04-26T23:45:14.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.109+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a10007bd0 con 0x7f6a18162aa0 2024-04-26T23:45:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.110+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 0x7f69f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.110+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f6a100acbe0 con 0x7f6a18162aa0 2024-04-26T23:45:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.110+0000 7f6a17400640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 0x7f69f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.111+0000 7f6a17400640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 0x7f69f406ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006fd0 tx=0x7f6a08008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.115+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a1007adf0 con 0x7f6a18162aa0 2024-04-26T23:45:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.326+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a180b3b90 con 0x7f6a18162aa0 2024-04-26T23:45:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.327+0000 7f6a0f400640 1 -- 172.21.15.46:0/3901464879 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f6a1007a6f0 con 0x7f6a18162aa0 2024-04-26T23:45:14.332 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:45:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.329+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 msgr2=0x7f69f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.329+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 0x7f69f406ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006fd0 tx=0x7f6a08008040 comp rx=0 tx=0).stop 2024-04-26T23:45:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.329+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 msgr2=0x7f6a180b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.329+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6a10034040 tx=0x7f6a10004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 shutdown_connections 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f69f406c920 0x7f69f406ede0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a18162aa0 0x7f6a180b8b60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a181626c0 0x7f6a181f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 --2- 172.21.15.46:0/3901464879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a180b5fb0 0x7f6a1815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 >> 172.21.15.46:0/3901464879 conn(0x7f6a1809e2a0 msgr2=0x7f6a180d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 shutdown_connections 2024-04-26T23:45:14.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.330+0000 7f6a1cc00640 1 -- 172.21.15.46:0/3901464879 wait complete. 2024-04-26T23:45:14.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:45:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.567+0000 7f26d0000640 1 Processor -- start 2024-04-26T23:45:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.568+0000 7f26d0000640 1 -- start start 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.569+0000 7f26d0000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f26c80d04c0 0x7f26c806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.569+0000 7f26d0000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26c80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26d0000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c80b4b30 con 0x7f26c80b4320 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26d0000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f26c806f740 con 0x7f26c80d04c0 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26d0000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b4e10 con 0x7f26c806f2e0 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26ce000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26c80b4320 0x7f26c8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51840/0 (socket says 172.21.15.46:51840) 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26ce000640 1 -- 172.21.15.46:0/3632049953 learned_addr learned my addr 172.21.15.46:0/3632049953 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26cd600640 1 --2- 172.21.15.46:0/3632049953 >> v2:172.21.15.142:3300/0 conn(0x7f26c80d04c0 0x7f26c806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26c80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.570+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143908238 0 0) 0x7f26c80b4b30 con 0x7f26c80b4320 2024-04-26T23:45:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26b0005730 con 0x7f26c80b4320 2024-04-26T23:45:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2695001814 0 0) 0x7f26b0005730 con 0x7f26c80b4320 2024-04-26T23:45:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 msgr2=0x7f26c80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26c80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 >> v2:172.21.15.142:3300/0 conn(0x7f26c80d04c0 msgr2=0x7f26c806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 --2- 172.21.15.46:0/3632049953 >> v2:172.21.15.142:3300/0 conn(0x7f26c80d04c0 0x7f26c806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80b50f0 con 0x7f26c80b4320 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f26c00019f0 con 0x7f26c80b4320 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 >> v1:172.21.15.46:6789/0 conn(0x7f26c80b4320 legacy=0x7f26c8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b0005bc0 0x7f26b0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26b0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26b00070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b4e10 con 0x7f26b0006520 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26b00077e0 con 0x7f26b0005bc0 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26b000ffd0 con 0x7f26c806f2e0 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.571+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26b0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26cd600640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b0005bc0 0x7f26b0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ce000640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26b00070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ccc00640 1 -- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 msgr2=0x7f26b00070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c806f2e0 0x7f26b00070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ccc00640 1 -- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b0005bc0 msgr2=0x7f26b0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b0005bc0 0x7f26b0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.572+0000 7f26ccc00640 1 -- 172.21.15.46:0/3632049953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b0005730 con 0x7f26b0006520 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.573+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26b0006a10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009af0 tx=0x7f26b0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9e46d542038224 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.573+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8036070 con 0x7f26b0006520 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.573+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8031400 con 0x7f26b0006520 2024-04-26T23:45:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.573+0000 7f26c7e00640 1 -- 172.21.15.46:0/3632049953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8009690 con 0x7f26b0006520 2024-04-26T23:45:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 -- 172.21.15.46:0/3632049953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 msgr2=0x7f26b0006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 --2- 172.21.15.46:0/3632049953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26b0006a10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009af0 tx=0x7f26b0007a40 comp rx=0 tx=0).stop 2024-04-26T23:45:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26ce000640 1 -- 172.21.15.46:0/3632049953 reap_dead start 2024-04-26T23:45:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 -- 172.21.15.46:0/3632049953 shutdown_connections 2024-04-26T23:45:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 -- 172.21.15.46:0/3632049953 >> 172.21.15.46:0/3632049953 conn(0x7f26c806c980 msgr2=0x7f26c80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 -- 172.21.15.46:0/3632049953 shutdown_connections 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.574+0000 7f26d0000640 1 -- 172.21.15.46:0/3632049953 wait complete. 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.576+0000 7f26d0000640 1 Processor -- start 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.576+0000 7f26d0000640 1 -- start start 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26c813c530 0x7f26c813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c815c5d0 0x7f26c815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b4770 con 0x7f26b0006520 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b56b0 con 0x7f26c813c530 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26d0000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b48f0 con 0x7f26c815c5d0 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26cd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26cd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36750/0 (socket says 172.21.15.46:36750) 2024-04-26T23:45:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.577+0000 7f26cd600640 1 -- 172.21.15.46:0/3550567323 learned_addr learned my addr 172.21.15.46:0/3550567323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26c813c530 0x7f26c813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26ce000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c815c5d0 0x7f26c815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 -- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c815c5d0 msgr2=0x7f26c815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c815c5d0 0x7f26c815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 -- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26c813c530 msgr2=0x7f26c813c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26c813c530 0x7f26c813c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 -- 172.21.15.46:0/3550567323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c81f4440 con 0x7f26b0006520 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26cd600640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f26bc011700 tx=0x7f26bc011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.578+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00f960 con 0x7f26b0006520 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.579+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc00fb00 con 0x7f26b0006520 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.579+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc015460 con 0x7f26b0006520 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.579+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c815d1b0 con 0x7f26b0006520 2024-04-26T23:45:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.579+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c815d770 con 0x7f26b0006520 2024-04-26T23:45:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.580+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c8069530 con 0x7f26b0006520 2024-04-26T23:45:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.581+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26bc015600 con 0x7f26b0006520 2024-04-26T23:45:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.582+0000 7f26c5600640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 0x7f26ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.582+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f26bc08c3f0 con 0x7f26b0006520 2024-04-26T23:45:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.583+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 0x7f26ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.584+0000 7f26ccc00640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 0x7f26ac06ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26b8033000 tx=0x7f26b8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.587+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26bc019030 con 0x7f26b0006520 2024-04-26T23:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.792+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26c813d2d0 con 0x7f26b0006520 2024-04-26T23:45:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.793+0000 7f26c5600640 1 -- 172.21.15.46:0/3550567323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f26bc059e80 con 0x7f26b0006520 2024-04-26T23:45:14.798 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:45:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 msgr2=0x7f26ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 0x7f26ac06ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26b8033000 tx=0x7f26b8008030 comp rx=0 tx=0).stop 2024-04-26T23:45:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 msgr2=0x7f26c81f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f26bc011700 tx=0x7f26bc011bc0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 shutdown_connections 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26ac06c920 0x7f26ac06ede0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26c815c5d0 0x7f26c815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26c813c530 0x7f26c813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 --2- 172.21.15.46:0/3550567323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b0006520 0x7f26c81f3f00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 >> 172.21.15.46:0/3550567323 conn(0x7f26c806c980 msgr2=0x7f26c80b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 shutdown_connections 2024-04-26T23:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:14.796+0000 7f26d0000640 1 -- 172.21.15.46:0/3550567323 wait complete. 2024-04-26T23:45:14.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.035+0000 7f08bf200640 1 Processor -- start 2024-04-26T23:45:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.036+0000 7f08bf200640 1 -- start start 2024-04-26T23:45:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.037+0000 7f08bf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08b815f4e0 0x7f08b80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.037+0000 7f08bf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b80d11f0 con 0x7f08b80d2130 2024-04-26T23:45:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.037+0000 7f08bf200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b80d5f20 con 0x7f08b80b4000 2024-04-26T23:45:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.037+0000 7f08bf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f08b80d0ce0 con 0x7f08b815f4e0 2024-04-26T23:45:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.037+0000 7f08be800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08b80d2130 0x7f08b80b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51842/0 (socket says 172.21.15.46:51842) 2024-04-26T23:45:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.038+0000 7f08be800640 1 -- 172.21.15.46:0/1365139758 learned_addr learned my addr 172.21.15.46:0/1365139758 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.038+0000 7f08bde00640 1 --2- 172.21.15.46:0/1365139758 >> v2:172.21.15.142:3300/0 conn(0x7f08b815f4e0 0x7f08b80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.038+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990043876 0 0) 0x7f08b80d11f0 con 0x7f08b80d2130 2024-04-26T23:45:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.038+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08a0003520 con 0x7f08b80d2130 2024-04-26T23:45:15.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.038+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652452780 0 0) 0x7f08a0003520 con 0x7f08b80d2130 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 >> v2:172.21.15.142:3300/0 conn(0x7f08b815f4e0 msgr2=0x7f08b80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 --2- 172.21.15.46:0/1365139758 >> v2:172.21.15.142:3300/0 conn(0x7f08b815f4e0 0x7f08b80d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 >> v1:172.21.15.142:6789/0 conn(0x7f08b80b4000 legacy=0x7f08b80d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b80d0f10 con 0x7f08b80d2130 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f08b40019f0 con 0x7f08b80d2130 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 >> v1:172.21.15.46:6789/0 conn(0x7f08b80d2130 legacy=0x7f08b80b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08a0003a00 0x7f08a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 0x7f08a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a00051c0 0x7f08a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08a000c090 con 0x7f08a0004800 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08a000c240 con 0x7f08a00051c0 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08a000c440 con 0x7f08a0003a00 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bd400640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 0x7f08a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.039+0000 7f08bde00640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08a0003a00 0x7f08a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08bd400640 1 -- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08a0003a00 msgr2=0x7f08a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08bd400640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08a0003a00 0x7f08a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08bd400640 1 -- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a00051c0 msgr2=0x7f08a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08be800640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a00051c0 0x7f08a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08bd400640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a00051c0 0x7f08a000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08bd400640 1 -- 172.21.15.46:0/1365139758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a0003520 con 0x7f08a0004800 2024-04-26T23:45:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.040+0000 7f08be800640 1 -- 172.21.15.46:0/1365139758 reap_dead start 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bd400640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 0x7f08a0004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08a8001750 tx=0x7f08a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d6350f451298ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8001370 con 0x7f08a0004800 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a80052d0 con 0x7f08a0004800 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bca00640 1 -- 172.21.15.46:0/1365139758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a802abc0 con 0x7f08a0004800 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bf200640 1 -- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 msgr2=0x7f08a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.041+0000 7f08bf200640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 0x7f08a0004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08a8001750 tx=0x7f08a0005830 comp rx=0 tx=0).stop 2024-04-26T23:45:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.042+0000 7f08bf200640 1 -- 172.21.15.46:0/1365139758 shutdown_connections 2024-04-26T23:45:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.042+0000 7f08bf200640 1 --2- 172.21.15.46:0/1365139758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a0004800 0x7f08a0004c80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.042+0000 7f08bf200640 1 -- 172.21.15.46:0/1365139758 >> 172.21.15.46:0/1365139758 conn(0x7f08b80d5010 msgr2=0x7f08b80d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.042+0000 7f08bf200640 1 -- 172.21.15.46:0/1365139758 shutdown_connections 2024-04-26T23:45:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.042+0000 7f08bf200640 1 -- 172.21.15.46:0/1365139758 wait complete. 2024-04-26T23:45:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.043+0000 7f08bf200640 1 Processor -- start 2024-04-26T23:45:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.044+0000 7f08bf200640 1 -- start start 2024-04-26T23:45:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0004800 0x7f08b814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b80ccd20 0x7f08b814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08b80b1d60 con 0x7f08b814e510 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08b80d2990 con 0x7f08a0004800 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08b815f8c0 con 0x7f08b80ccd20 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36776/0 (socket says 172.21.15.46:36776) 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 -- 172.21.15.46:0/4142999263 learned_addr learned my addr 172.21.15.46:0/4142999263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bd400640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b80ccd20 0x7f08b814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08bde00640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0004800 0x7f08b814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 -- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b80ccd20 msgr2=0x7f08b814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b80ccd20 0x7f08b814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.045+0000 7f08be800640 1 -- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0004800 msgr2=0x7f08b814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08be800640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0004800 0x7f08b814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08be800640 1 -- 172.21.15.46:0/4142999263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b80cd580 con 0x7f08b814e510 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08be800640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f08b808f170 tx=0x7f08b400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b400b7a0 con 0x7f08b814e510 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b400b940 con 0x7f08b814e510 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b400bd30 con 0x7f08b814e510 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b80cd860 con 0x7f08b814e510 2024-04-26T23:45:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.046+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b80cd1a0 con 0x7f08b814e510 2024-04-26T23:45:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.048+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b80693b0 con 0x7f08b814e510 2024-04-26T23:45:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.049+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08b401b020 con 0x7f08b814e510 2024-04-26T23:45:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.050+0000 7f08b2000640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 0x7f089c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.050+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f08b408be50 con 0x7f08b814e510 2024-04-26T23:45:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.051+0000 7f08bde00640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 0x7f089c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.052+0000 7f08bde00640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 0x7f089c06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009a90 tx=0x7f08ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.055+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08b4059f40 con 0x7f08b814e510 2024-04-26T23:45:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.306+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b80cdd80 con 0x7f08b814e510 2024-04-26T23:45:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.307+0000 7f08b2000640 1 -- 172.21.15.46:0/4142999263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f08b4059840 con 0x7f08b814e510 2024-04-26T23:45:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:15.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 msgr2=0x7f089c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 0x7f089c06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009a90 tx=0x7f08ac008040 comp rx=0 tx=0).stop 2024-04-26T23:45:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 msgr2=0x7f08b814e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f08b808f170 tx=0x7f08b400df60 comp rx=0 tx=0).stop 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 shutdown_connections 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f089c06c8f0 0x7f089c06edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08b814e510 0x7f08b814e990 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b80ccd20 0x7f08b814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 --2- 172.21.15.46:0/4142999263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0004800 0x7f08b814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 >> 172.21.15.46:0/4142999263 conn(0x7f08b80d5010 msgr2=0x7f08b815b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 shutdown_connections 2024-04-26T23:45:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.310+0000 7f08bf200640 1 -- 172.21.15.46:0/4142999263 wait complete. 2024-04-26T23:45:15.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.548+0000 7f60ef200640 1 Processor -- start 2024-04-26T23:45:15.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.548+0000 7f60ef200640 1 -- start start 2024-04-26T23:45:15.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ef200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e81466b0 0x7f60e806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ef200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e8073ad0 con 0x7f60e80720e0 2024-04-26T23:45:15.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ef200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e8146a90 con 0x7f60e8160910 2024-04-26T23:45:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ef200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f60e8073d70 con 0x7f60e81466b0 2024-04-26T23:45:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ede00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e81466b0 0x7f60e806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ede00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e81466b0 0x7f60e806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36792/0 (socket says 172.21.15.46:36792) 2024-04-26T23:45:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.550+0000 7f60ede00640 1 -- 172.21.15.46:0/3627149931 learned_addr learned my addr 172.21.15.46:0/3627149931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.551+0000 7f60ede00640 1 -- 172.21.15.46:0/3627149931 >> v1:172.21.15.142:6789/0 conn(0x7f60e8160910 legacy=0x7f60e80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.551+0000 7f60ede00640 1 -- 172.21.15.46:0/3627149931 >> v1:172.21.15.46:6789/0 conn(0x7f60e80720e0 legacy=0x7f60e80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.551+0000 7f60ede00640 1 -- 172.21.15.46:0/3627149931 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e8074050 con 0x7f60e81466b0 2024-04-26T23:45:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.552+0000 7f60ede00640 1 --2- 172.21.15.46:0/3627149931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e81466b0 0x7f60e806de80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba539c1ba5104069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.552+0000 7f60eca00640 1 -- 172.21.15.46:0/3627149931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc037070 con 0x7f60e81466b0 2024-04-26T23:45:15.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.552+0000 7f60eca00640 1 -- 172.21.15.46:0/3627149931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc032440 con 0x7f60e81466b0 2024-04-26T23:45:15.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.552+0000 7f60eca00640 1 -- 172.21.15.46:0/3627149931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc032810 con 0x7f60e81466b0 2024-04-26T23:45:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 -- 172.21.15.46:0/3627149931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e81466b0 msgr2=0x7f60e806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 --2- 172.21.15.46:0/3627149931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e81466b0 0x7f60e806de80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc02a2e0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 -- 172.21.15.46:0/3627149931 shutdown_connections 2024-04-26T23:45:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 --2- 172.21.15.46:0/3627149931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e81466b0 0x7f60e806de80 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 -- 172.21.15.46:0/3627149931 >> 172.21.15.46:0/3627149931 conn(0x7f60e80b26d0 msgr2=0x7f60e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 -- 172.21.15.46:0/3627149931 shutdown_connections 2024-04-26T23:45:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.553+0000 7f60ef200640 1 -- 172.21.15.46:0/3627149931 wait complete. 2024-04-26T23:45:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.555+0000 7f60ef200640 1 Processor -- start 2024-04-26T23:45:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.555+0000 7f60ef200640 1 -- start start 2024-04-26T23:45:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 0x7f60e813f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e8160910 0x7f60e813f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60e806e3c0 con 0x7f60e80720e0 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80cb470 con 0x7f60e81466b0 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ef200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60e80cc0a0 con 0x7f60e8160910 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36800/0 (socket says 172.21.15.46:36800) 2024-04-26T23:45:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.556+0000 7f60ede00640 1 -- 172.21.15.46:0/4190361218 learned_addr learned my addr 172.21.15.46:0/4190361218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ee800640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e8160910 0x7f60e813f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ed400640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 0x7f60e813f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 -- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e8160910 msgr2=0x7f60e813f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e8160910 0x7f60e813f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 -- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 msgr2=0x7f60e813f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 0x7f60e813f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 -- 172.21.15.46:0/4190361218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e80c9720 con 0x7f60e80720e0 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ed400640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 0x7f60e813f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.557+0000 7f60ede00640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.558+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc037040 con 0x7f60e80720e0 2024-04-26T23:45:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.558+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc004230 con 0x7f60e80720e0 2024-04-26T23:45:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.558+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc003610 con 0x7f60e80720e0 2024-04-26T23:45:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.558+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e814b0a0 con 0x7f60e80720e0 2024-04-26T23:45:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.558+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e814b660 con 0x7f60e80720e0 2024-04-26T23:45:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.559+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e80693b0 con 0x7f60e80720e0 2024-04-26T23:45:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.561+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60dc0037b0 con 0x7f60e80720e0 2024-04-26T23:45:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.562+0000 7f60e2000640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 0x7f60cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.562+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f60dc0acc80 con 0x7f60e80720e0 2024-04-26T23:45:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.562+0000 7f60ed400640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 0x7f60cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.563+0000 7f60ed400640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 0x7f60cc06edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f60e80d0f50 tx=0x7f60d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.566+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60dc07acd0 con 0x7f60e80720e0 2024-04-26T23:45:15.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.772+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60e806e820 con 0x7f60e80720e0 2024-04-26T23:45:15.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.774+0000 7f60e2000640 1 -- 172.21.15.46:0/4190361218 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+17303 (secure 0 0 0) 0x7f60dc07a5d0 con 0x7f60e80720e0 2024-04-26T23:45:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:15.779 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:57.986395+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.142:6834/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.46:6835/2670265715":"2024-04-27T23:44:56.993227+0000","172.21.15.46:6833/2670265715":"2024-04-27T23:44:56.993227+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 msgr2=0x7f60cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 0x7f60cc06edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f60e80d0f50 tx=0x7f60d8009040 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 msgr2=0x7f60e8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098a0 tx=0x7f60dc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 shutdown_connections 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60cc06c8f0 0x7f60cc06edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e8160910 0x7f60e813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e81466b0 0x7f60e813f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 --2- 172.21.15.46:0/4190361218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e80720e0 0x7f60e8141250 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 >> 172.21.15.46:0/4190361218 conn(0x7f60e80b26d0 msgr2=0x7f60e80726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 shutdown_connections 2024-04-26T23:45:15.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:15.777+0000 7f60ef200640 1 -- 172.21.15.46:0/4190361218 wait complete. 2024-04-26T23:45:15.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:45:15.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.006+0000 7f029be00640 1 Processor -- start 2024-04-26T23:45:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.007+0000 7f029be00640 1 -- start start 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f029c1466b0 0x7f029c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f029c160910 0x7f029c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029c073ad0 con 0x7f029c0720e0 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f029c1612a0 con 0x7f029c160910 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f029c146a90 con 0x7f029c1466b0 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f029c160910 0x7f029c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f029c160910 0x7f029c160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36810/0 (socket says 172.21.15.46:36810) 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.008+0000 7f029b400640 1 -- 172.21.15.46:0/1425060919 learned_addr learned my addr 172.21.15.46:0/1425060919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.009+0000 7f029aa00640 1 --2- 172.21.15.46:0/1425060919 >> v2:172.21.15.142:3300/0 conn(0x7f029c1466b0 0x7f029c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.009+0000 7f029b400640 1 -- 172.21.15.46:0/1425060919 >> v2:172.21.15.142:3300/0 conn(0x7f029c1466b0 msgr2=0x7f029c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.009+0000 7f029b400640 1 --2- 172.21.15.46:0/1425060919 >> v2:172.21.15.142:3300/0 conn(0x7f029c1466b0 0x7f029c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.009+0000 7f029b400640 1 -- 172.21.15.46:0/1425060919 >> v1:172.21.15.46:6789/0 conn(0x7f029c0720e0 legacy=0x7f029c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.009+0000 7f029b400640 1 -- 172.21.15.46:0/1425060919 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c073d70 con 0x7f029c160910 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.010+0000 7f029b400640 1 --2- 172.21.15.46:0/1425060919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c160d60 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0294009e50 tx=0x7f029402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7f260da13cfaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.010+0000 7f029a000640 1 -- 172.21.15.46:0/1425060919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294036070 con 0x7f029c160910 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.010+0000 7f029a000640 1 -- 172.21.15.46:0/1425060919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0294031600 con 0x7f029c160910 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.010+0000 7f029a000640 1 -- 172.21.15.46:0/1425060919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940319d0 con 0x7f029c160910 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 -- 172.21.15.46:0/1425060919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 msgr2=0x7f029c160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 --2- 172.21.15.46:0/1425060919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c160d60 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0294009e50 tx=0x7f029402a760 comp rx=0 tx=0).stop 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 -- 172.21.15.46:0/1425060919 shutdown_connections 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 --2- 172.21.15.46:0/1425060919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c160d60 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 --2- 172.21.15.46:0/1425060919 >> v2:172.21.15.142:3300/0 conn(0x7f029c1466b0 0x7f029c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.011+0000 7f029be00640 1 -- 172.21.15.46:0/1425060919 >> 172.21.15.46:0/1425060919 conn(0x7f029c0b26d0 msgr2=0x7f029c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.012+0000 7f029be00640 1 -- 172.21.15.46:0/1425060919 shutdown_connections 2024-04-26T23:45:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.012+0000 7f029be00640 1 -- 172.21.15.46:0/1425060919 wait complete. 2024-04-26T23:45:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.013+0000 7f029be00640 1 Processor -- start 2024-04-26T23:45:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 -- start start 2024-04-26T23:45:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f029c0720e0 0x7f029c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f029c1466b0 0x7f029c1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f029c0cb470 con 0x7f029c160910 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f029c0cb190 con 0x7f029c0720e0 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.014+0000 7f029be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f029c06e3c0 con 0x7f029c1466b0 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36826/0 (socket says 172.21.15.46:36826) 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 -- 172.21.15.46:0/916781857 learned_addr learned my addr 172.21.15.46:0/916781857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029aa00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f029c0720e0 0x7f029c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f0293e00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f029c1466b0 0x7f029c1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 -- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f029c1466b0 msgr2=0x7f029c1521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f029c1466b0 0x7f029c1521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 -- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f029c0720e0 msgr2=0x7f029c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f029c0720e0 0x7f029c0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.015+0000 7f029b400640 1 -- 172.21.15.46:0/916781857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c139620 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f029b400640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0294009e50 tx=0x7f0294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294036050 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0294042070 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029c139900 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294009510 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.016+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029c0b56b0 con 0x7f029c160910 2024-04-26T23:45:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.017+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029c0693b0 con 0x7f029c160910 2024-04-26T23:45:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.018+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0294007bd0 con 0x7f029c160910 2024-04-26T23:45:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.019+0000 7f0292a00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.019+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f02940acbe0 con 0x7f029c160910 2024-04-26T23:45:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.020+0000 7f029aa00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.021+0000 7f029aa00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 0x7f027806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.024+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0294045360 con 0x7f029c160910 2024-04-26T23:45:16.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.268+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f029c0c7180 con 0x7f029c160910 2024-04-26T23:45:16.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.269+0000 7f0292a00640 1 -- 172.21.15.46:0/916781857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7f029407ad60 con 0x7f029c160910 2024-04-26T23:45:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:16.273 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:16.274 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:16.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 msgr2=0x7f027806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 0x7f027806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c008040 comp rx=0 tx=0).stop 2024-04-26T23:45:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 msgr2=0x7f029c138720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0294009e50 tx=0x7f0294004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 shutdown_connections 2024-04-26T23:45:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f029c160910 0x7f029c138720 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f029c1466b0 0x7f029c1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 --2- 172.21.15.46:0/916781857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f029c0720e0 0x7f029c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.271+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 >> 172.21.15.46:0/916781857 conn(0x7f029c0b26d0 msgr2=0x7f029c06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.272+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 shutdown_connections 2024-04-26T23:45:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.272+0000 7f029be00640 1 -- 172.21.15.46:0/916781857 wait complete. 2024-04-26T23:45:16.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.502+0000 7fda7d000640 1 Processor -- start 2024-04-26T23:45:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.502+0000 7fda7d000640 1 -- start start 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.504+0000 7fda7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda780cc690 0x7fda780cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.504+0000 7fda7d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda78072bb0 con 0x7fda780c6a50 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.504+0000 7fda7d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda78072e90 con 0x7fda781454a0 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.504+0000 7fda7d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fda78073170 con 0x7fda780cc690 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda780cc690 0x7fda780cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda780cc690 0x7fda780cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36828/0 (socket says 172.21.15.46:36828) 2024-04-26T23:45:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 -- 172.21.15.46:0/4268229039 learned_addr learned my addr 172.21.15.46:0/4268229039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 -- 172.21.15.46:0/4268229039 >> v1:172.21.15.142:6789/0 conn(0x7fda781454a0 legacy=0x7fda78073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 -- 172.21.15.46:0/4268229039 >> v1:172.21.15.46:6789/0 conn(0x7fda780c6a50 legacy=0x7fda780cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.505+0000 7fda6fe00640 1 -- 172.21.15.46:0/4268229039 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda78073450 con 0x7fda780cc690 2024-04-26T23:45:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.506+0000 7fda6fe00640 1 --2- 172.21.15.46:0/4268229039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780cc690 0x7fda780cdc30 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f30b5e01127ed0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.507+0000 7fda76a00640 1 -- 172.21.15.46:0/4268229039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60038070 con 0x7fda780cc690 2024-04-26T23:45:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.507+0000 7fda76a00640 1 -- 172.21.15.46:0/4268229039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60031710 con 0x7fda780cc690 2024-04-26T23:45:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.507+0000 7fda76a00640 1 -- 172.21.15.46:0/4268229039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60031ae0 con 0x7fda780cc690 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.507+0000 7fda7d000640 1 -- 172.21.15.46:0/4268229039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780cc690 msgr2=0x7fda780cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.507+0000 7fda7d000640 1 --2- 172.21.15.46:0/4268229039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780cc690 0x7fda780cdc30 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002c4b0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.508+0000 7fda7d000640 1 -- 172.21.15.46:0/4268229039 shutdown_connections 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.508+0000 7fda7d000640 1 --2- 172.21.15.46:0/4268229039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780cc690 0x7fda780cdc30 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.508+0000 7fda7d000640 1 -- 172.21.15.46:0/4268229039 >> 172.21.15.46:0/4268229039 conn(0x7fda780b26d0 msgr2=0x7fda780b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.508+0000 7fda7d000640 1 -- 172.21.15.46:0/4268229039 shutdown_connections 2024-04-26T23:45:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.508+0000 7fda7d000640 1 -- 172.21.15.46:0/4268229039 wait complete. 2024-04-26T23:45:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.509+0000 7fda7d000640 1 Processor -- start 2024-04-26T23:45:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.510+0000 7fda7d000640 1 -- start start 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.510+0000 7fda7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.510+0000 7fda7d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 0x7fda781f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda7d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda781454a0 0x7fda78140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda7d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda78145880 con 0x7fda780c6a50 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda7d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda78072880 con 0x7fda781454a0 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda7d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda78145a00 con 0x7fda780cc690 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36832/0 (socket says 172.21.15.46:36832) 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 -- 172.21.15.46:0/1941611907 learned_addr learned my addr 172.21.15.46:0/1941611907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77e00640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda781454a0 0x7fda78140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda6fe00640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 0x7fda781f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 -- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 msgr2=0x7fda781f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 0x7fda781f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 -- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda781454a0 msgr2=0x7fda78140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda781454a0 0x7fda78140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda77400640 1 -- 172.21.15.46:0/1941611907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda781f1a30 con 0x7fda780c6a50 2024-04-26T23:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.511+0000 7fda6fe00640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 0x7fda781f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda77400640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b3f0 tx=0x7fda6800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68015020 con 0x7fda780c6a50 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda68011830 con 0x7fda780c6a50 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68011c20 con 0x7fda780c6a50 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7806ed10 con 0x7fda780c6a50 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.512+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7813e460 con 0x7fda780c6a50 2024-04-26T23:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.513+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda780693b0 con 0x7fda780c6a50 2024-04-26T23:45:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.515+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda68018020 con 0x7fda780c6a50 2024-04-26T23:45:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.516+0000 7fda6f400640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.516+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7fda6808b9c0 con 0x7fda780c6a50 2024-04-26T23:45:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.516+0000 7fda6fe00640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.517+0000 7fda6fe00640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fda60035000 tx=0x7fda60008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.520+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda6801f030 con 0x7fda780c6a50 2024-04-26T23:45:16.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.765+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda780cefa0 con 0x7fda780c6a50 2024-04-26T23:45:16.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.766+0000 7fda6f400640 1 -- 172.21.15.46:0/1941611907 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5185 (secure 0 0 0) 0x7fda781f1a30 con 0x7fda780c6a50 2024-04-26T23:45:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:16.771 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:44:42.305207+0000","modified":"2024-04-26T23:45:01.027471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2779258602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2779258602},{"type":"v1","addr":"172.21.15.46:6835","nonce":2779258602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7959,"qdb_cluster":[7959]},"id":6}]} 2024-04-26T23:45:16.771 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 msgr2=0x7fda5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fda60035000 tx=0x7fda60008030 comp rx=0 tx=0).stop 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 msgr2=0x7fda7815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b3f0 tx=0x7fda6800b8b0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 shutdown_connections 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda781454a0 0x7fda78140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda780cc690 0x7fda781f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 --2- 172.21.15.46:0/1941611907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda780c6a50 0x7fda7815b660 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 >> 172.21.15.46:0/1941611907 conn(0x7fda780b26d0 msgr2=0x7fda780b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 shutdown_connections 2024-04-26T23:45:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.769+0000 7fda7d000640 1 -- 172.21.15.46:0/1941611907 wait complete. 2024-04-26T23:45:16.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.995+0000 7fa48ec00640 1 Processor -- start 2024-04-26T23:45:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.995+0000 7fa48ec00640 1 -- start start 2024-04-26T23:45:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.996+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa48806d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.996+0000 7fa48ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4880b76a0 0x7fa4880b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.996+0000 7fa48ec00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4880d21d0 con 0x7fa4880d0f50 2024-04-26T23:45:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa4880d1ef0 con 0x7fa4880b76a0 2024-04-26T23:45:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa4880b6bc0 con 0x7fa4880d1a40 2024-04-26T23:45:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4880b76a0 0x7fa4880b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4880b76a0 0x7fa4880b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36836/0 (socket says 172.21.15.46:36836) 2024-04-26T23:45:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48e200640 1 -- 172.21.15.46:0/1964321629 learned_addr learned my addr 172.21.15.46:0/1964321629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.997+0000 7fa48ce00640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa48806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.998+0000 7fa48e200640 1 -- 172.21.15.46:0/1964321629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 msgr2=0x7fa48806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.998+0000 7fa48e200640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa48806d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.998+0000 7fa48e200640 1 -- 172.21.15.46:0/1964321629 >> v1:172.21.15.142:6789/0 conn(0x7fa4880d0f50 legacy=0x7fa4880d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.998+0000 7fa48e200640 1 -- 172.21.15.46:0/1964321629 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4880b6ea0 con 0x7fa4880b76a0 2024-04-26T23:45:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.999+0000 7fa48e200640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880b7b00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa480009870 tx=0x7fa48002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a29611e7cdf478 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.999+0000 7fa487e00640 1 -- 172.21.15.46:0/1964321629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480037070 con 0x7fa4880b76a0 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.999+0000 7fa487e00640 1 -- 172.21.15.46:0/1964321629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480032440 con 0x7fa4880b76a0 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:16.999+0000 7fa487e00640 1 -- 172.21.15.46:0/1964321629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480032810 con 0x7fa4880b76a0 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 -- 172.21.15.46:0/1964321629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 msgr2=0x7fa4880b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880b7b00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa480009870 tx=0x7fa48002a180 comp rx=0 tx=0).stop 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 -- 172.21.15.46:0/1964321629 shutdown_connections 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880b7b00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 --2- 172.21.15.46:0/1964321629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa48806d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 -- 172.21.15.46:0/1964321629 >> 172.21.15.46:0/1964321629 conn(0x7fa4880d2cc0 msgr2=0x7fa4880d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 -- 172.21.15.46:0/1964321629 shutdown_connections 2024-04-26T23:45:17.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.000+0000 7fa48ec00640 1 -- 172.21.15.46:0/1964321629 wait complete. 2024-04-26T23:45:17.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.002+0000 7fa48ec00640 1 Processor -- start 2024-04-26T23:45:17.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.002+0000 7fa48ec00640 1 -- start start 2024-04-26T23:45:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4880d0f50 0x7fa4880cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa4880cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa48815f7c0 con 0x7fa4880b76a0 2024-04-26T23:45:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa48815f640 con 0x7fa4880d0f50 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa4880b3ff0 con 0x7fa4880d1a40 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36848/0 (socket says 172.21.15.46:36848) 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.003+0000 7fa48d800640 1 -- 172.21.15.46:0/3858476256 learned_addr learned my addr 172.21.15.46:0/3858476256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 -- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 msgr2=0x7fa4880cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa4880cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 -- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4880d0f50 msgr2=0x7fa4880cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4880d0f50 0x7fa4880cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 -- 172.21.15.46:0/3858476256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4880d0220 con 0x7fa4880b76a0 2024-04-26T23:45:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48d800640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa47800ba80 tx=0x7fa47800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cc00 con 0x7fa4880b76a0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47800cda0 con 0x7fa4880b76a0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478003bb0 con 0x7fa4880b76a0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.004+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4880cbe60 con 0x7fa4880b76a0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.005+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4880cc420 con 0x7fa4880b76a0 2024-04-26T23:45:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.006+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa488069ac0 con 0x7fa4880b76a0 2024-04-26T23:45:17.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.007+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa478014440 con 0x7fa4880b76a0 2024-04-26T23:45:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.008+0000 7fa485600640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.008+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7fa47808bd30 con 0x7fa4880b76a0 2024-04-26T23:45:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.009+0000 7fa48ce00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.010+0000 7fa48ce00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa4880cbd90 tx=0x7fa474008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.013+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa478059eb0 con 0x7fa4880b76a0 2024-04-26T23:45:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.220+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4880c65a0 con 0x7fa4880b76a0 2024-04-26T23:45:17.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.223+0000 7fa485600640 1 -- 172.21.15.46:0/3858476256 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+17303 (secure 0 0 0) 0x7fa4780597b0 con 0x7fa4880b76a0 2024-04-26T23:45:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:17.228 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:44:57.986395+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:44:40.164396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T23:44:41.287502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.142:6834/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.46:6835/2670265715":"2024-04-27T23:44:56.993227+0000","172.21.15.46:6833/2670265715":"2024-04-27T23:44:56.993227+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 msgr2=0x7fa46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa4880cbd90 tx=0x7fa474008040 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 msgr2=0x7fa4880cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa47800ba80 tx=0x7fa47800bf40 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 shutdown_connections 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880d1a40 0x7fa4880cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4880d0f50 0x7fa4880cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 --2- 172.21.15.46:0/3858476256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880b76a0 0x7fa4880cb850 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.226+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 >> 172.21.15.46:0/3858476256 conn(0x7fa4880d2cc0 msgr2=0x7fa48815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:17.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.227+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 shutdown_connections 2024-04-26T23:45:17.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.227+0000 7fa48ec00640 1 -- 172.21.15.46:0/3858476256 wait complete. 2024-04-26T23:45:17.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:45:17.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.455+0000 7f75c8a00640 1 Processor -- start 2024-04-26T23:45:17.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.456+0000 7f75c8a00640 1 -- start start 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.457+0000 7f75c8a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c406e510 0x7f75c406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.457+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75c4073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.457+0000 7f75c8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c406ee30 con 0x7f75c4160bd0 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.457+0000 7f75c8a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f75c4161940 con 0x7f75c406e510 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.457+0000 7f75c8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75c4071f80 con 0x7f75c40732c0 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75c4160bd0 0x7f75c4161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51886/0 (socket says 172.21.15.46:51886) 2024-04-26T23:45:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c1400640 1 -- 172.21.15.46:0/3667345120 learned_addr learned my addr 172.21.15.46:0/3667345120 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c1e00640 1 --2- 172.21.15.46:0/3667345120 >> v2:172.21.15.142:3300/0 conn(0x7f75c406e510 0x7f75c406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c2800640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75c4073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160020967 0 0) 0x7f75c406ee30 con 0x7f75c4160bd0 2024-04-26T23:45:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.458+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a8003520 con 0x7f75c4160bd0 2024-04-26T23:45:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3797706761 0 0) 0x7f75a8003520 con 0x7f75c4160bd0 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 msgr2=0x7f75c4073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75c4073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 >> v2:172.21.15.142:3300/0 conn(0x7f75c406e510 msgr2=0x7f75c406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 --2- 172.21.15.46:0/3667345120 >> v2:172.21.15.142:3300/0 conn(0x7f75c406e510 0x7f75c406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c4072260 con 0x7f75c4160bd0 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f75ac001670 con 0x7f75c4160bd0 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 >> v1:172.21.15.46:6789/0 conn(0x7f75c4160bd0 legacy=0x7f75c4161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4161940 con 0x7f75a8004d40 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4071f80 con 0x7f75a8003f30 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75a800b9c0 con 0x7f75c40732c0 2024-04-26T23:45:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c2800640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c1e00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.459+0000 7f75c1400640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c2800640 1 -- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 msgr2=0x7f75a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c2800640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c2800640 1 -- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a8003f30 msgr2=0x7f75a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c2800640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c2800640 1 -- 172.21.15.46:0/3667345120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8003520 con 0x7f75a8004d40 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c1e00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c40732c0 0x7f75a80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.460+0000 7f75c1400640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c2800640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009fc0 tx=0x7f75a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c17664bd82f6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc009c10 con 0x7f75a8004d40 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc036070 con 0x7f75a8004d40 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c0a00640 1 -- 172.21.15.46:0/3667345120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc031850 con 0x7f75a8004d40 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c8a00640 1 -- 172.21.15.46:0/3667345120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 msgr2=0x7f75a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.461+0000 7f75c8a00640 1 --2- 172.21.15.46:0/3667345120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009fc0 tx=0x7f75a8005830 comp rx=0 tx=0).stop 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.462+0000 7f75c2800640 1 -- 172.21.15.46:0/3667345120 reap_dead start 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.462+0000 7f75c8a00640 1 -- 172.21.15.46:0/3667345120 shutdown_connections 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.462+0000 7f75c8a00640 1 -- 172.21.15.46:0/3667345120 >> 172.21.15.46:0/3667345120 conn(0x7f75c406c530 msgr2=0x7f75c406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.462+0000 7f75c8a00640 1 -- 172.21.15.46:0/3667345120 shutdown_connections 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.462+0000 7f75c8a00640 1 -- 172.21.15.46:0/3667345120 wait complete. 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.463+0000 7f75c8a00640 1 Processor -- start 2024-04-26T23:45:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 -- start start 2024-04-26T23:45:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c41524d0 0x7f75c413a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c413ab80 0x7f75c413c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4072d40 con 0x7f75a8004d40 2024-04-26T23:45:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4072820 con 0x7f75c41524d0 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.464+0000 7f75c8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75c4072ec0 con 0x7f75c413ab80 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36866/0 (socket says 172.21.15.46:36866) 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 -- 172.21.15.46:0/2918345685 learned_addr learned my addr 172.21.15.46:0/2918345685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 -- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c413ab80 msgr2=0x7f75c413c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c413ab80 0x7f75c413c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 -- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c41524d0 msgr2=0x7f75c413a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c41524d0 0x7f75c413a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 -- 172.21.15.46:0/2918345685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c4139ba0 con 0x7f75a8004d40 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.465+0000 7f75c1e00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f75b40109d0 tx=0x7f75b4010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.466+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400f040 con 0x7f75a8004d40 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.466+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b400eba0 con 0x7f75a8004d40 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.466+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b401c400 con 0x7f75a8004d40 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.466+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c4152950 con 0x7f75a8004d40 2024-04-26T23:45:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.466+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c41434c0 con 0x7f75a8004d40 2024-04-26T23:45:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.467+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c413afa0 con 0x7f75a8004d40 2024-04-26T23:45:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.468+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75b401c5a0 con 0x7f75a8004d40 2024-04-26T23:45:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.469+0000 7f75ba000640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 0x7f75a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.469+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6389+0+0 (secure 0 0 0) 0x7f75b408c440 con 0x7f75a8004d40 2024-04-26T23:45:17.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.470+0000 7f75c1400640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 0x7f75a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.471+0000 7f75c1400640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 0x7f75a006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75c41f6550 tx=0x7f75ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.473+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75b4018030 con 0x7f75a8004d40 2024-04-26T23:45:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.717+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75c406fb00 con 0x7f75a8004d40 2024-04-26T23:45:17.723 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:45:17.719+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:45:17.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.723+0000 7f75ba000640 1 -- 172.21.15.46:0/2918345685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f75b405a650 con 0x7f75a8004d40 2024-04-26T23:45:17.727 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 msgr2=0x7f75a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 0x7f75a006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75c41f6550 tx=0x7f75ac009040 comp rx=0 tx=0).stop 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 msgr2=0x7f75c415c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f75b40109d0 tx=0x7f75b4010e90 comp rx=0 tx=0).stop 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 shutdown_connections 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f75a006c920 0x7f75a006ede0 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75c41f6550 tx=0x7f75ac009040 comp rx=0 tx=0).stop 2024-04-26T23:45:17.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c413ab80 0x7f75c413c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c41524d0 0x7f75c413a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 --2- 172.21.15.46:0/2918345685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8004d40 0x7f75c415c4f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.727+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 >> 172.21.15.46:0/2918345685 conn(0x7f75c406c530 msgr2=0x7f75c40b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.729+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 shutdown_connections 2024-04-26T23:45:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.729+0000 7f75c8a00640 1 -- 172.21.15.46:0/2918345685 wait complete. 2024-04-26T23:45:17.738 INFO:tasks.ceph.mds.a.smithi046.stderr: -1656> 2024-04-26T23:45:04.009+0000 7fd1c5a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:45:17.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:45:17.787 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:45:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.928+0000 7fc8c5c00640 1 Processor -- start 2024-04-26T23:45:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.928+0000 7fc8c5c00640 1 -- start start 2024-04-26T23:45:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.930+0000 7fc8c5c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8c01562b0 0x7fc8c0142070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.930+0000 7fc8c5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8c0072f70 0x7fc8c0073380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.930+0000 7fc8c5c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c00738c0 con 0x7fc8c015c370 2024-04-26T23:45:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.930+0000 7fc8c5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c013f1f0 con 0x7fc8c0072f70 2024-04-26T23:45:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.930+0000 7fc8c5c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c015cba0 con 0x7fc8c01562b0 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8bf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc8c015c370 0x7fc8c0160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51898/0 (socket says 172.21.15.46:51898) 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8bf400640 1 -- 172.21.15.46:0/2872838145 learned_addr learned my addr 172.21.15.46:0/2872838145 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8c5200640 1 --2- 172.21.15.46:0/2872838145 >> v2:172.21.15.46:3300/0 conn(0x7fc8c0072f70 0x7fc8c0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8bfe00640 1 --2- 172.21.15.46:0/2872838145 >> v2:172.21.15.142:3300/0 conn(0x7fc8c01562b0 0x7fc8c0142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8c5200640 1 -- 172.21.15.46:0/2872838145 >> v2:172.21.15.142:3300/0 conn(0x7fc8c01562b0 msgr2=0x7fc8c0142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8c5200640 1 --2- 172.21.15.46:0/2872838145 >> v2:172.21.15.142:3300/0 conn(0x7fc8c01562b0 0x7fc8c0142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8c5200640 1 -- 172.21.15.46:0/2872838145 >> v1:172.21.15.46:6789/0 conn(0x7fc8c015c370 legacy=0x7fc8c0160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8bea00640 1 -- 172.21.15.46:0/2872838145 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515069479 0 0) 0x7fc8ac001110 con 0x7fc8c015c370 2024-04-26T23:45:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.931+0000 7fc8c5200640 1 -- 172.21.15.46:0/2872838145 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c015ce80 con 0x7fc8c0072f70 2024-04-26T23:45:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.932+0000 7fc8c5200640 1 --2- 172.21.15.46:0/2872838145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c0072f70 0x7fc8c0073380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00cf7a0 tx=0x7fc8b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8932ccd2278102fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8bea00640 1 -- 172.21.15.46:0/2872838145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007740 con 0x7fc8c0072f70 2024-04-26T23:45:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8bea00640 1 -- 172.21.15.46:0/2872838145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b80078e0 con 0x7fc8c0072f70 2024-04-26T23:45:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8bea00640 1 -- 172.21.15.46:0/2872838145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007cb0 con 0x7fc8c0072f70 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 -- 172.21.15.46:0/2872838145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c0072f70 msgr2=0x7fc8c0073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/2872838145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c0072f70 0x7fc8c0073380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00cf7a0 tx=0x7fc8b802a680 comp rx=0 tx=0).stop 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 -- 172.21.15.46:0/2872838145 shutdown_connections 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/2872838145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c0072f70 0x7fc8c0073380 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/2872838145 >> v2:172.21.15.142:3300/0 conn(0x7fc8c01562b0 0x7fc8c0142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 -- 172.21.15.46:0/2872838145 >> 172.21.15.46:0/2872838145 conn(0x7fc8c00d1c90 msgr2=0x7fc8c00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.933+0000 7fc8c5c00640 1 -- 172.21.15.46:0/2872838145 shutdown_connections 2024-04-26T23:45:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.934+0000 7fc8c5c00640 1 -- 172.21.15.46:0/2872838145 wait complete. 2024-04-26T23:45:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.935+0000 7fc8c5c00640 1 Processor -- start 2024-04-26T23:45:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.935+0000 7fc8c5c00640 1 -- start start 2024-04-26T23:45:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8c0072f70 0x7fc8c01f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 0x7fc8c00c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01607a0 con 0x7fc8c015c370 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c015fb70 con 0x7fc8c0072f70 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c015c820 con 0x7fc8c01562b0 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.936+0000 7fc8c5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36896/0 (socket says 172.21.15.46:36896) 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 -- 172.21.15.46:0/394087329 learned_addr learned my addr 172.21.15.46:0/394087329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8bfe00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8c0072f70 0x7fc8c01f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8bf400640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 0x7fc8c00c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 -- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 msgr2=0x7fc8c00c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 0x7fc8c00c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 -- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8c0072f70 msgr2=0x7fc8c01f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8c0072f70 0x7fc8c01f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8c5200640 1 -- 172.21.15.46:0/394087329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c013b0e0 con 0x7fc8c015c370 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.937+0000 7fc8bf400640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 0x7fc8c00c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8c5200640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009e80 tx=0x7fc8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b803f070 con 0x7fc8c015c370 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8031c40 con 0x7fc8c015c370 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c013b3c0 con 0x7fc8c015c370 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8009450 con 0x7fc8c015c370 2024-04-26T23:45:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.938+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c013b980 con 0x7fc8c015c370 2024-04-26T23:45:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.939+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c0069ac0 con 0x7fc8c015c370 2024-04-26T23:45:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.940+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b8003b90 con 0x7fc8c015c370 2024-04-26T23:45:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.942+0000 7fc8b7e00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 0x7fc8a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.942+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6475+0+0 (secure 0 0 0) 0x7fc8b80acff0 con 0x7fc8c015c370 2024-04-26T23:45:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.942+0000 7fc8bfe00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 0x7fc8a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.944+0000 7fc8bfe00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 0x7fc8a006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0006fd0 tx=0x7fc8b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:17.946+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8b807b0c0 con 0x7fc8c015c370 2024-04-26T23:45:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.187+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc8c0074180 con 0x7fc8c015c370 2024-04-26T23:45:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.726+0000 7fc8b7e00640 1 -- 172.21.15.46:0/394087329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) ==== 104+0+0 (secure 0 0 0) 0x7fc8b807a9c0 con 0x7fc8c015c370 2024-04-26T23:45:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 msgr2=0x7fc8a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 0x7fc8a006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0006fd0 tx=0x7fc8b0008040 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 msgr2=0x7fc8c00c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009e80 tx=0x7fc8b8004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 shutdown_connections 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc8a006c8f0 0x7fc8a006edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c015c370 0x7fc8c00c81c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8c01562b0 0x7fc8c00c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 --2- 172.21.15.46:0/394087329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8c0072f70 0x7fc8c01f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 >> 172.21.15.46:0/394087329 conn(0x7fc8c00d1c90 msgr2=0x7fc8c015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 shutdown_connections 2024-04-26T23:45:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:18.730+0000 7fc8c5c00640 1 -- 172.21.15.46:0/394087329 wait complete. 2024-04-26T23:45:18.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:45:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.021+0000 7fafca200640 1 Processor -- start 2024-04-26T23:45:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.022+0000 7fafca200640 1 -- start start 2024-04-26T23:45:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.023+0000 7fafca200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fafc40b4400 0x7fafc406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.023+0000 7fafca200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafc406e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.023+0000 7fafca200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc40b5690 con 0x7fafc414cfa0 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafca200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fafc406e5c0 con 0x7fafc40b4400 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafca200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fafc4071eb0 con 0x7fafc406dc30 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc8e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fafc414cfa0 0x7fafc4163ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51904/0 (socket says 172.21.15.46:51904) 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc8e00640 1 -- 172.21.15.46:0/1893305538 learned_addr learned my addr 172.21.15.46:0/1893305538 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766024019 0 0) 0x7fafc40b5690 con 0x7fafc414cfa0 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc3e00640 1 --2- 172.21.15.46:0/1893305538 >> v2:172.21.15.142:3300/0 conn(0x7fafc40b4400 0x7fafc406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc9800640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafc406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.024+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafac003520 con 0x7fafc414cfa0 2024-04-26T23:45:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344757495 0 0) 0x7fafac003520 con 0x7fafc414cfa0 2024-04-26T23:45:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 msgr2=0x7fafc406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafc406e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 >> v2:172.21.15.142:3300/0 conn(0x7fafc40b4400 msgr2=0x7fafc406fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 --2- 172.21.15.46:0/1893305538 >> v2:172.21.15.142:3300/0 conn(0x7fafc40b4400 0x7fafc406fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4072190 con 0x7fafc414cfa0 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fafb8001690 con 0x7fafc414cfa0 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3e00640 1 --2- 172.21.15.46:0/1893305538 >> v2:172.21.15.142:3300/0 conn(0x7fafc40b4400 0x7fafc406fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:45:19.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 >> v1:172.21.15.46:6789/0 conn(0x7fafc414cfa0 legacy=0x7fafc4163ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafac0039b0 0x7fafac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafac004800 0x7fafac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fafc406e5c0 con 0x7fafac0039b0 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4071eb0 con 0x7fafac004800 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fafac00ba40 con 0x7fafc406dc30 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.025+0000 7fafc8e00640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafac0039b0 0x7fafac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc9800640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc8e00640 1 -- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 msgr2=0x7fafac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc8e00640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafc406dc30 0x7fafac0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc8e00640 1 -- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafac004800 msgr2=0x7fafac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc8e00640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafac004800 0x7fafac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.026+0000 7fafc8e00640 1 -- 172.21.15.46:0/1893305538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac003520 con 0x7fafac0039b0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafc8e00640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafac0039b0 0x7fafac003dd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fafc40b3f10 tx=0x7fafac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3537c26aea9dbb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8007ca0 con 0x7fafac0039b0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb8005ad0 con 0x7fafac0039b0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafc3400640 1 -- 172.21.15.46:0/1893305538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8007070 con 0x7fafac0039b0 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafca200640 1 -- 172.21.15.46:0/1893305538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafac0039b0 msgr2=0x7fafac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.027+0000 7fafca200640 1 --2- 172.21.15.46:0/1893305538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafac0039b0 0x7fafac003dd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fafc40b3f10 tx=0x7fafac005830 comp rx=0 tx=0).stop 2024-04-26T23:45:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.028+0000 7fafc9800640 1 -- 172.21.15.46:0/1893305538 reap_dead start 2024-04-26T23:45:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.028+0000 7fafca200640 1 -- 172.21.15.46:0/1893305538 shutdown_connections 2024-04-26T23:45:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.028+0000 7fafca200640 1 -- 172.21.15.46:0/1893305538 >> 172.21.15.46:0/1893305538 conn(0x7fafc406c980 msgr2=0x7fafc40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.028+0000 7fafca200640 1 -- 172.21.15.46:0/1893305538 shutdown_connections 2024-04-26T23:45:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.028+0000 7fafca200640 1 -- 172.21.15.46:0/1893305538 wait complete. 2024-04-26T23:45:19.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.030+0000 7fafca200640 1 Processor -- start 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.030+0000 7fafca200640 1 -- start start 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafac0039b0 0x7fafc41f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc4144b10 0x7fafc4165480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fafc414d380 con 0x7fafc41659c0 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4072750 con 0x7fafc4144b10 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafca200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fafc40b5320 con 0x7fafac0039b0 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafc9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafc9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36916/0 (socket says 172.21.15.46:36916) 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafc9800640 1 -- 172.21.15.46:0/641078431 learned_addr learned my addr 172.21.15.46:0/641078431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafc8e00640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafac0039b0 0x7fafc41f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.031+0000 7fafc3e00640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc4144b10 0x7fafc4165480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 -- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafac0039b0 msgr2=0x7fafc41f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafac0039b0 0x7fafc41f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 -- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc4144b10 msgr2=0x7fafc4165480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc4144b10 0x7fafc4165480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 -- 172.21.15.46:0/641078431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4145880 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc9800640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fafc4073b30 tx=0x7fafbc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc01c070 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc017400 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc4145b60 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0177f0 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.032+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc4146120 con 0x7fafc41659c0 2024-04-26T23:45:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.033+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4165e40 con 0x7fafc41659c0 2024-04-26T23:45:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.034+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafbc005d10 con 0x7fafc41659c0 2024-04-26T23:45:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.035+0000 7fafc0c00640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 0x7fafa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.035+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6475+0+0 (secure 0 0 0) 0x7fafbc08c350 con 0x7fafc41659c0 2024-04-26T23:45:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.036+0000 7fafc8e00640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 0x7fafa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.037+0000 7fafc8e00640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 0x7fafa406ede0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fafb8032000 tx=0x7fafb8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:19.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.040+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafbc012070 con 0x7fafc41659c0 2024-04-26T23:45:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.245+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fafc40d03b0 con 0x7fafc41659c0 2024-04-26T23:45:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.737+0000 7fafc0c00640 1 -- 172.21.15.46:0/641078431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fafbc05a3e0 con 0x7fafc41659c0 2024-04-26T23:45:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:45:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 msgr2=0x7fafa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 0x7fafa406ede0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fafb8032000 tx=0x7fafb8009030 comp rx=0 tx=0).stop 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 msgr2=0x7fafc41f9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fafc4073b30 tx=0x7fafbc00f4a0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 shutdown_connections 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fafa406c920 0x7fafa406ede0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafc41659c0 0x7fafc41f9310 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc4144b10 0x7fafc4165480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 --2- 172.21.15.46:0/641078431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafac0039b0 0x7fafc41f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.741+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 >> 172.21.15.46:0/641078431 conn(0x7fafc406c980 msgr2=0x7fafc40aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.742+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 shutdown_connections 2024-04-26T23:45:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.742+0000 7fafca200640 1 -- 172.21.15.46:0/641078431 wait complete. 2024-04-26T23:45:19.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:45:19.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.979+0000 7f0729800640 1 Processor -- start 2024-04-26T23:45:19.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.979+0000 7f0729800640 1 -- start start 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0729800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0724146b20 0x7f07240d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0729800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07240d1620 0x7f07240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0729800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07240b3ce0 con 0x7f0724146740 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0729800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f07240b48f0 con 0x7f0724146b20 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0729800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07240b3fc0 con 0x7f07240d1620 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0723e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0724146740 0x7f07240cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51910/0 (socket says 172.21.15.46:51910) 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.980+0000 7f0723e00640 1 -- 172.21.15.46:0/3120271489 learned_addr learned my addr 172.21.15.46:0/3120271489 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0723400640 1 --2- 172.21.15.46:0/3120271489 >> v2:172.21.15.46:3300/0 conn(0x7f0724146b20 0x7f07240d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0728e00640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07240d1620 0x7f07240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0722a00640 1 -- 172.21.15.46:0/3120271489 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897697227 0 0) 0x7f07240b3ce0 con 0x7f0724146740 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0722a00640 1 -- 172.21.15.46:0/3120271489 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0708003520 con 0x7f0724146740 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0723400640 1 -- 172.21.15.46:0/3120271489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07240d1620 msgr2=0x7f07240cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0723400640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07240d1620 0x7f07240cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0723400640 1 -- 172.21.15.46:0/3120271489 >> v1:172.21.15.46:6789/0 conn(0x7f0724146740 legacy=0x7f07240cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.981+0000 7f0723400640 1 -- 172.21.15.46:0/3120271489 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07240b42a0 con 0x7f0724146b20 2024-04-26T23:45:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.982+0000 7f0723400640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0724146b20 0x7f07240d10e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f071002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7509d2fcdbd0346e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.982+0000 7f0722a00640 1 -- 172.21.15.46:0/3120271489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710037070 con 0x7f0724146b20 2024-04-26T23:45:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0722a00640 1 -- 172.21.15.46:0/3120271489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071002ae50 con 0x7f0724146b20 2024-04-26T23:45:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0722a00640 1 -- 172.21.15.46:0/3120271489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710032660 con 0x7f0724146b20 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 -- 172.21.15.46:0/3120271489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0724146b20 msgr2=0x7f07240d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0724146b20 0x7f07240d10e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f071002a100 comp rx=0 tx=0).stop 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 -- 172.21.15.46:0/3120271489 shutdown_connections 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07240d1620 0x7f07240cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 --2- 172.21.15.46:0/3120271489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0724146b20 0x7f07240d10e0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 -- 172.21.15.46:0/3120271489 >> 172.21.15.46:0/3120271489 conn(0x7f072406c770 msgr2=0x7f072406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 -- 172.21.15.46:0/3120271489 shutdown_connections 2024-04-26T23:45:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.983+0000 7f0729800640 1 -- 172.21.15.46:0/3120271489 wait complete. 2024-04-26T23:45:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.985+0000 7f0729800640 1 Processor -- start 2024-04-26T23:45:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.985+0000 7f0729800640 1 -- start start 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0724146740 0x7f072415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 0x7f07240733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f072415e560 con 0x7f07240d1620 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07240d1a30 con 0x7f0724146b20 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.986+0000 7f0729800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07240cdde0 con 0x7f0724146740 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36936/0 (socket says 172.21.15.46:36936) 2024-04-26T23:45:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 -- 172.21.15.46:0/665819049 learned_addr learned my addr 172.21.15.46:0/665819049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:19.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0728e00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 0x7f07240733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723400640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0724146740 0x7f072415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 -- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0724146740 msgr2=0x7f072415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0724146740 0x7f072415c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 -- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 msgr2=0x7f07240733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 0x7f07240733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 -- 172.21.15.46:0/665819049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0724133120 con 0x7f07240d1620 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0728e00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 0x7f07240733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.987+0000 7f0723e00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f071400c460 tx=0x7f071400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.988+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714017070 con 0x7f07240d1620 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.988+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0714005ce0 con 0x7f07240d1620 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.988+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072415cd10 con 0x7f07240d1620 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.988+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714007290 con 0x7f07240d1620 2024-04-26T23:45:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.988+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07240b0c00 con 0x7f07240d1620 2024-04-26T23:45:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.989+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07240693b0 con 0x7f07240d1620 2024-04-26T23:45:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.990+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f071401a070 con 0x7f07240d1620 2024-04-26T23:45:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.991+0000 7f071be00640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.991+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6037+0+0 (secure 0 0 0) 0x7f071408aff0 con 0x7f07240d1620 2024-04-26T23:45:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.992+0000 7f0723400640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.993+0000 7f0723400640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0710034000 tx=0x7f071002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:19.995+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0724133120 con 0x7f07240d1620 2024-04-26T23:45:20.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.204+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07240b5e60 con 0x7f07240d1620 2024-04-26T23:45:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.751+0000 7f071be00640 1 -- 172.21.15.46:0/665819049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f0714059320 con 0x7f07240d1620 2024-04-26T23:45:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:45:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.757+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 msgr2=0x7f070406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.757+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0710034000 tx=0x7f071002e000 comp rx=0 tx=0).stop 2024-04-26T23:45:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.757+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 msgr2=0x7f072415b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.757+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f071400c460 tx=0x7f071400c920 comp rx=0 tx=0).stop 2024-04-26T23:45:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 shutdown_connections 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0724146b20 0x7f07240733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0724146740 0x7f072415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 --2- 172.21.15.46:0/665819049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07240d1620 0x7f072415b350 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.759+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 >> 172.21.15.46:0/665819049 conn(0x7f072406c770 msgr2=0x7f0724145a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.760+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 shutdown_connections 2024-04-26T23:45:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.760+0000 7f0729800640 1 -- 172.21.15.46:0/665819049 wait complete. 2024-04-26T23:45:20.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_merge_size 2024-04-26T23:45:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.992+0000 7f5eac400640 1 Processor -- start 2024-04-26T23:45:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.992+0000 7f5eac400640 1 -- start start 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5eac400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ea40b53b0 0x7f5ea415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5eac400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5eac400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea40d07e0 con 0x7f5ea415d730 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5eac400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea406eb90 con 0x7f5ea40b53b0 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5eac400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea40b4310 con 0x7f5ea415e050 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5ea9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ea40b53b0 0x7f5ea415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5ea9a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ea415d730 0x7f5ea40d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51916/0 (socket says 172.21.15.46:51916) 2024-04-26T23:45:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.994+0000 7f5ea9a00640 1 -- 172.21.15.46:0/845372073 learned_addr learned my addr 172.21.15.46:0/845372073 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5eaa400640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5ea9000640 1 -- 172.21.15.46:0/845372073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 msgr2=0x7f5ea406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5ea9000640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5ea9000640 1 -- 172.21.15.46:0/845372073 >> v1:172.21.15.46:6789/0 conn(0x7f5ea415d730 legacy=0x7f5ea40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5ea9000640 1 -- 172.21.15.46:0/845372073 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea40b45f0 con 0x7f5ea40b53b0 2024-04-26T23:45:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.995+0000 7f5eaa400640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.996+0000 7f5ea9000640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea40b53b0 0x7f5ea415db10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5e900097a0 tx=0x7f5e9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2e54bfb0c32d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.996+0000 7f5ea3e00640 1 -- 172.21.15.46:0/845372073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90037070 con 0x7f5ea40b53b0 2024-04-26T23:45:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.996+0000 7f5ea3e00640 1 -- 172.21.15.46:0/845372073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e90032440 con 0x7f5ea40b53b0 2024-04-26T23:45:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.996+0000 7f5ea3e00640 1 -- 172.21.15.46:0/845372073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90032810 con 0x7f5ea40b53b0 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 -- 172.21.15.46:0/845372073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea40b53b0 msgr2=0x7f5ea415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea40b53b0 0x7f5ea415db10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5e900097a0 tx=0x7f5e9002a180 comp rx=0 tx=0).stop 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 -- 172.21.15.46:0/845372073 shutdown_connections 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 --2- 172.21.15.46:0/845372073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea40b53b0 0x7f5ea415db10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 -- 172.21.15.46:0/845372073 >> 172.21.15.46:0/845372073 conn(0x7f5ea406c980 msgr2=0x7f5ea40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 -- 172.21.15.46:0/845372073 shutdown_connections 2024-04-26T23:45:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.997+0000 7f5eac400640 1 -- 172.21.15.46:0/845372073 wait complete. 2024-04-26T23:45:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.999+0000 7f5eac400640 1 Processor -- start 2024-04-26T23:45:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:20.999+0000 7f5eac400640 1 -- start start 2024-04-26T23:45:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 0x7f5ea4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea41424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40b4e90 con 0x7f5ea415d730 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40b4bb0 con 0x7f5ea40b53b0 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.001+0000 7f5eac400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea415e4a0 con 0x7f5ea415e050 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.002+0000 7f5ea9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.002+0000 7f5ea9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36958/0 (socket says 172.21.15.46:36958) 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.002+0000 7f5ea9000640 1 -- 172.21.15.46:0/3803148362 learned_addr learned my addr 172.21.15.46:0/3803148362 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.002+0000 7f5eaa400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea41424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.002+0000 7f5ea9a00640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 0x7f5ea4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9000640 1 -- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 msgr2=0x7f5ea41424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9000640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea41424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9000640 1 -- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 msgr2=0x7f5ea4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9000640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 0x7f5ea4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9000640 1 -- 172.21.15.46:0/3803148362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea413a080 con 0x7f5ea415d730 2024-04-26T23:45:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5eaa400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea41424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.003+0000 7f5ea9a00640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 0x7f5ea4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.004+0000 7f5ea9000640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5e90035040 tx=0x7f5e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.005+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90037040 con 0x7f5ea415d730 2024-04-26T23:45:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.005+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e90030770 con 0x7f5ea415d730 2024-04-26T23:45:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.005+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90030b80 con 0x7f5ea415d730 2024-04-26T23:45:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.005+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea413a360 con 0x7f5ea415d730 2024-04-26T23:45:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.005+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea4142e60 con 0x7f5ea415d730 2024-04-26T23:45:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.007+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea4069530 con 0x7f5ea415d730 2024-04-26T23:45:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.009+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e90044020 con 0x7f5ea415d730 2024-04-26T23:45:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.011+0000 7f5ea1600640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 0x7f5e8406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.011+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f5e900ac0b0 con 0x7f5ea415d730 2024-04-26T23:45:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.012+0000 7f5ea9a00640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 0x7f5e8406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.014+0000 7f5ea9a00640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 0x7f5e8406ee80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40d06b0 tx=0x7f5e98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.017+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e90032e20 con 0x7f5ea415d730 2024-04-26T23:45:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.222+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"} v 0) -- 0x7f5ea4076550 con 0x7f5ea415d730 2024-04-26T23:45:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.230+0000 7f5ea1600640 1 -- 172.21.15.46:0/3803148362 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"}]=0 v30) ==== 101+0+0 (secure 0 0 0) 0x7f5e9007a580 con 0x7f5ea415d730 2024-04-26T23:45:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 msgr2=0x7f5e8406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 0x7f5e8406ee80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40d06b0 tx=0x7f5e98009040 comp rx=0 tx=0).stop 2024-04-26T23:45:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 msgr2=0x7f5ea4073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5e90035040 tx=0x7f5e900046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 shutdown_connections 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5e8406c9c0 0x7f5e8406ee80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ea415e050 0x7f5ea41424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ea415d730 0x7f5ea4073940 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.233+0000 7f5eac400640 1 --2- 172.21.15.46:0/3803148362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea40b53b0 0x7f5ea4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.234+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 >> 172.21.15.46:0/3803148362 conn(0x7f5ea406c980 msgr2=0x7f5ea40b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.234+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 shutdown_connections 2024-04-26T23:45:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.234+0000 7f5eac400640 1 -- 172.21.15.46:0/3803148362 wait complete. 2024-04-26T23:45:21.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_bits 2024-04-26T23:45:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.468+0000 7fc5bfe00640 1 Processor -- start 2024-04-26T23:45:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.468+0000 7fc5bfe00640 1 -- start start 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.469+0000 7fc5bfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc5c00ca8c0 0x7fc5c00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.469+0000 7fc5bfe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c00d3250 con 0x7fc5c015f280 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.469+0000 7fc5bfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c00d2280 con 0x7fc5c00d3bd0 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.469+0000 7fc5bfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c00c8af0 con 0x7fc5c00ca8c0 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.470+0000 7fc5bea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc5c015f280 0x7fc5c00d34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51928/0 (socket says 172.21.15.46:51928) 2024-04-26T23:45:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.470+0000 7fc5bea00640 1 -- 172.21.15.46:0/3082152843 learned_addr learned my addr 172.21.15.46:0/3082152843 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:21.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.470+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951950995 0 0) 0x7fc5c00d3250 con 0x7fc5c015f280 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.470+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a4003520 con 0x7fc5c015f280 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.470+0000 7fc5bf400640 1 --2- 172.21.15.46:0/3082152843 >> v2:172.21.15.142:3300/0 conn(0x7fc5c00ca8c0 0x7fc5c00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851002357 0 0) 0x7fc5a4003520 con 0x7fc5c015f280 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 >> v2:172.21.15.142:3300/0 conn(0x7fc5c00ca8c0 msgr2=0x7fc5c00d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 --2- 172.21.15.46:0/3082152843 >> v2:172.21.15.142:3300/0 conn(0x7fc5c00ca8c0 0x7fc5c00d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 >> v1:172.21.15.142:6789/0 conn(0x7fc5c00d3bd0 legacy=0x7fc5c00d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c00c8dd0 con 0x7fc5c015f280 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc5b0001670 con 0x7fc5c015f280 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 >> v1:172.21.15.46:6789/0 conn(0x7fc5c015f280 legacy=0x7fc5c00d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 0x7fc5a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4005210 0x7fc5a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00c8af0 con 0x7fc5a4003a00 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a400c160 con 0x7fc5a4004800 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a400c360 con 0x7fc5a4005210 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.471+0000 7fc5bea00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 0x7fc5a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5be000640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bf400640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4005210 0x7fc5a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bea00640 1 -- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4005210 msgr2=0x7fc5a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bea00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4005210 0x7fc5a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bea00640 1 -- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a4004800 msgr2=0x7fc5a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bea00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bea00640 1 -- 172.21.15.46:0/3082152843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a4003520 con 0x7fc5a4003a00 2024-04-26T23:45:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5be000640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:45:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.472+0000 7fc5bf400640 1 -- 172.21.15.46:0/3082152843 reap_dead start 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.473+0000 7fc5bea00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 0x7fc5a4003e20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c190 tx=0x7fc5a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f414dd34bb3ddcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.473+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0007c60 con 0x7fc5a4003a00 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.473+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0005ad0 con 0x7fc5a4003a00 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.473+0000 7fc5bd600640 1 -- 172.21.15.46:0/3082152843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0007070 con 0x7fc5a4003a00 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 -- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 msgr2=0x7fc5a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 0x7fc5a4003e20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c190 tx=0x7fc5a4005830 comp rx=0 tx=0).stop 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 -- 172.21.15.46:0/3082152843 shutdown_connections 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/3082152843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5a4003a00 0x7fc5a4003e20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 -- 172.21.15.46:0/3082152843 >> 172.21.15.46:0/3082152843 conn(0x7fc5c00d4f00 msgr2=0x7fc5c00d5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 -- 172.21.15.46:0/3082152843 shutdown_connections 2024-04-26T23:45:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.474+0000 7fc5bfe00640 1 -- 172.21.15.46:0/3082152843 wait complete. 2024-04-26T23:45:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.476+0000 7fc5bfe00640 1 Processor -- start 2024-04-26T23:45:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.476+0000 7fc5bfe00640 1 -- start start 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4003a00 0x7fc5c00cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5c0149d50 0x7fc5c014a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00d0d40 con 0x7fc5c0161640 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00c9390 con 0x7fc5c0149d50 2024-04-26T23:45:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.477+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c00d0ec0 con 0x7fc5a4003a00 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36978/0 (socket says 172.21.15.46:36978) 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 -- 172.21.15.46:0/930503471 learned_addr learned my addr 172.21.15.46:0/930503471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5be000640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5c0149d50 0x7fc5c014a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bea00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4003a00 0x7fc5c00cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 -- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4003a00 msgr2=0x7fc5c00cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4003a00 0x7fc5c00cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 -- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5c0149d50 msgr2=0x7fc5c014a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5c0149d50 0x7fc5c014a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.478+0000 7fc5bf400640 1 -- 172.21.15.46:0/930503471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c014a710 con 0x7fc5c0161640 2024-04-26T23:45:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5bf400640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b800f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8004440 con 0x7fc5c0161640 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b8019070 con 0x7fc5c0161640 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8003850 con 0x7fc5c0161640 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c0158330 con 0x7fc5c0161640 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.479+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c0155650 con 0x7fc5c0161640 2024-04-26T23:45:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.480+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c00693b0 con 0x7fc5c0161640 2024-04-26T23:45:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.481+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5b8012070 con 0x7fc5c0161640 2024-04-26T23:45:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.482+0000 7fc5b6a00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 0x7fc5a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.482+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7fc5b808b320 con 0x7fc5c0161640 2024-04-26T23:45:21.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.483+0000 7fc5bea00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 0x7fc5a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.484+0000 7fc5bea00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 0x7fc5a006ede0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c160 tx=0x7fc5b002d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.487+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5b8059810 con 0x7fc5c0161640 2024-04-26T23:45:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.693+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"} v 0) -- 0x7fc5c00ba770 con 0x7fc5c0161640 2024-04-26T23:45:21.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.700+0000 7fc5b6a00640 1 -- 172.21.15.46:0/930503471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"}]=0 v31) ==== 101+0+0 (secure 0 0 0) 0x7fc5b8059110 con 0x7fc5c0161640 2024-04-26T23:45:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 msgr2=0x7fc5a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 0x7fc5a006ede0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c160 tx=0x7fc5b002d000 comp rx=0 tx=0).stop 2024-04-26T23:45:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 msgr2=0x7fc5c0161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b800f860 comp rx=0 tx=0).stop 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 shutdown_connections 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc5a006c920 0x7fc5a006ede0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5c0161640 0x7fc5c0161ac0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5c0149d50 0x7fc5c014a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 --2- 172.21.15.46:0/930503471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a4003a00 0x7fc5c00cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 >> 172.21.15.46:0/930503471 conn(0x7fc5c00d4f00 msgr2=0x7fc5c006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 shutdown_connections 2024-04-26T23:45:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.703+0000 7fc5bfe00640 1 -- 172.21.15.46:0/930503471 wait complete. 2024-04-26T23:45:21.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:45:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.940+0000 7ffaaf200640 1 Processor -- start 2024-04-26T23:45:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.941+0000 7ffaaf200640 1 -- start start 2024-04-26T23:45:21.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.942+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffab00d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.942+0000 7ffaaf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffab00d2ce0 0x7ffab00d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.942+0000 7ffaaf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab00d3250 con 0x7ffab015f280 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.942+0000 7ffaaf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffab00c8af0 con 0x7ffab00d2ce0 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.942+0000 7ffaaf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffab00c8dd0 con 0x7ffab00d3bd0 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaade00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffab015f280 0x7ffab00d34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46826/0 (socket says 172.21.15.46:46826) 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaade00640 1 -- 172.21.15.46:0/4217903835 learned_addr learned my addr 172.21.15.46:0/4217903835 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaad400640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffab00d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaae800640 1 --2- 172.21.15.46:0/4217903835 >> v2:172.21.15.142:3300/0 conn(0x7ffab00d2ce0 0x7ffab00d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696163233 0 0) 0x7ffab00d3250 con 0x7ffab015f280 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa94003520 con 0x7ffab015f280 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2579474208 0 0) 0x7ffa94003520 con 0x7ffab015f280 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 msgr2=0x7ffab00d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffab00d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 >> v2:172.21.15.142:3300/0 conn(0x7ffab00d2ce0 msgr2=0x7ffab00d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 --2- 172.21.15.46:0/4217903835 >> v2:172.21.15.142:3300/0 conn(0x7ffab00d2ce0 0x7ffab00d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab00c90b0 con 0x7ffab015f280 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ffaa0001670 con 0x7ffab015f280 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 >> v1:172.21.15.46:6789/0 conn(0x7ffab015f280 legacy=0x7ffab00d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.943+0000 7ffaaca00640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffa940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaaca00640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa94004030 0x7ffa94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaaca00640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffa940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffab00c8af0 con 0x7ffa94004d40 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffab00c8dd0 con 0x7ffa94004030 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa9400bb20 con 0x7ffab00d3bd0 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffa940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaade00640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffa940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaad400640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa94004030 0x7ffa94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 -- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 msgr2=0x7ffa940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab00d3bd0 0x7ffa940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 -- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa94004030 msgr2=0x7ffa94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa94004030 0x7ffa94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaae800640 1 -- 172.21.15.46:0/4217903835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa94003520 con 0x7ffa94004d40 2024-04-26T23:45:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.944+0000 7ffaad400640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa94004030 0x7ffa94004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.945+0000 7ffaae800640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffa940051c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007360 tx=0x7ffa94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cffda564ec6510 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8009be0 con 0x7ffa94004d40 2024-04-26T23:45:21.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa802e620 con 0x7ffa94004d40 2024-04-26T23:45:21.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaca00640 1 -- 172.21.15.46:0/4217903835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa802e9f0 con 0x7ffa94004d40 2024-04-26T23:45:21.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 -- 172.21.15.46:0/4217903835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 msgr2=0x7ffa940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 --2- 172.21.15.46:0/4217903835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffa940051c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007360 tx=0x7ffa94005830 comp rx=0 tx=0).stop 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaae800640 1 -- 172.21.15.46:0/4217903835 reap_dead start 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 -- 172.21.15.46:0/4217903835 shutdown_connections 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 -- 172.21.15.46:0/4217903835 >> 172.21.15.46:0/4217903835 conn(0x7ffab00d4f00 msgr2=0x7ffab00d5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 -- 172.21.15.46:0/4217903835 shutdown_connections 2024-04-26T23:45:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.946+0000 7ffaaf200640 1 -- 172.21.15.46:0/4217903835 wait complete. 2024-04-26T23:45:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.948+0000 7ffaaf200640 1 Processor -- start 2024-04-26T23:45:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.948+0000 7ffaaf200640 1 -- start start 2024-04-26T23:45:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 0x7ffab015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 0x7ffab009f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffab00d0ec0 con 0x7ffa94004d40 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffab00d0d40 con 0x7ffab009ed40 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.949+0000 7ffaaf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffab015f660 con 0x7ffab015d190 2024-04-26T23:45:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54968/0 (socket says 172.21.15.46:54968) 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 -- 172.21.15.46:0/1930075496 learned_addr learned my addr 172.21.15.46:0/1930075496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaae800640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 0x7ffab009f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaad400640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 0x7ffab015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 -- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 msgr2=0x7ffab015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 0x7ffab015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 -- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 msgr2=0x7ffab009f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 0x7ffab009f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaade00640 1 -- 172.21.15.46:0/1930075496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab00a2420 con 0x7ffa94004d40 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaae800640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 0x7ffab009f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.950+0000 7ffaad400640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 0x7ffab015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaade00640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e800 tx=0x7ffaa000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000c870 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa0018070 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000cc50 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab00a2700 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.951+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab00ce990 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.952+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab00693b0 con 0x7ffa94004d40 2024-04-26T23:45:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.953+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffaa0013420 con 0x7ffa94004d40 2024-04-26T23:45:21.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.955+0000 7ffaa6000640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 0x7ffa9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:21.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.955+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7ffaa008b5d0 con 0x7ffa94004d40 2024-04-26T23:45:21.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.955+0000 7ffaad400640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 0x7ffa9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:21.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.956+0000 7ffaad400640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 0x7ffa9006ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006fd0 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:21.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:21.961+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffaa0059b50 con 0x7ffa94004d40 2024-04-26T23:45:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.164+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ffab00d57b0 con 0x7ffa94004d40 2024-04-26T23:45:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.170+0000 7ffaa6000640 1 -- 172.21.15.46:0/1930075496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7ffaa0059450 con 0x7ffa94004d40 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 msgr2=0x7ffa9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 0x7ffa9006ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006fd0 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 msgr2=0x7ffab0154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e800 tx=0x7ffaa000ecc0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 shutdown_connections 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa9006c920 0x7ffa9006ede0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab009ed40 0x7ffab009f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015d190 0x7ffab015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1930075496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa94004d40 0x7ffab0154400 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 >> 172.21.15.46:0/1930075496 conn(0x7ffab00d4f00 msgr2=0x7ffab00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 shutdown_connections 2024-04-26T23:45:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.173+0000 7ffaaf200640 1 -- 172.21.15.46:0/1930075496 wait complete. 2024-04-26T23:45:22.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_size 2024-04-26T23:45:22.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.393+0000 7f8a43400640 1 Processor -- start 2024-04-26T23:45:22.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.393+0000 7f8a43400640 1 -- start start 2024-04-26T23:45:22.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a43400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8a3c0b53b0 0x7f8a3c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a43400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 0x7f8a3c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a43400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c0d07e0 con 0x7f8a3c15d730 2024-04-26T23:45:22.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a43400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3c06eb90 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a43400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c0b4310 con 0x7f8a3c15e050 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a40a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8a3c15d730 0x7f8a3c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:50092/0 (socket says 172.21.15.46:50092) 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a40a00640 1 -- 172.21.15.46:0/207948055 learned_addr learned my addr 172.21.15.46:0/207948055 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.395+0000 7f8a41400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 0x7f8a3c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a3be00640 1 --2- 172.21.15.46:0/207948055 >> v2:172.21.15.142:3300/0 conn(0x7f8a3c0b53b0 0x7f8a3c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a3be00640 1 -- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 msgr2=0x7f8a3c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a3be00640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 0x7f8a3c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a3be00640 1 -- 172.21.15.46:0/207948055 >> v1:172.21.15.142:6789/0 conn(0x7f8a3c15d730 legacy=0x7f8a3c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a3be00640 1 -- 172.21.15.46:0/207948055 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c0b45f0 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.396+0000 7f8a41400640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 0x7f8a3c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.398+0000 7f8a3be00640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c15db10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a280097a0 tx=0x7f8a2802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee56f8b766a6802 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.398+0000 7f8a3b400640 1 -- 172.21.15.46:0/207948055 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28037070 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.398+0000 7f8a3b400640 1 -- 172.21.15.46:0/207948055 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a28032440 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.398+0000 7f8a3b400640 1 -- 172.21.15.46:0/207948055 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28032810 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 -- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 msgr2=0x7f8a3c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c15db10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a280097a0 tx=0x7f8a2802a180 comp rx=0 tx=0).stop 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 -- 172.21.15.46:0/207948055 shutdown_connections 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15e050 0x7f8a3c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 --2- 172.21.15.46:0/207948055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c15db10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 -- 172.21.15.46:0/207948055 >> 172.21.15.46:0/207948055 conn(0x7f8a3c06c980 msgr2=0x7f8a3c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 -- 172.21.15.46:0/207948055 shutdown_connections 2024-04-26T23:45:22.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.399+0000 7f8a43400640 1 -- 172.21.15.46:0/207948055 wait complete. 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.401+0000 7f8a43400640 1 Processor -- start 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.401+0000 7f8a43400640 1 -- start start 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 0x7f8a3c1e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0b4e90 con 0x7f8a3c15e050 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0b4bb0 con 0x7f8a3c0b53b0 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a43400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c15e4a0 con 0x7f8a3c15d730 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54982/0 (socket says 172.21.15.46:54982) 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.402+0000 7f8a41400640 1 -- 172.21.15.46:0/1164048383 learned_addr learned my addr 172.21.15.46:0/1164048383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a3be00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 0x7f8a3c1e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a40a00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 -- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 msgr2=0x7f8a3c1e48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 0x7f8a3c1e48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 -- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 msgr2=0x7f8a3c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c13f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 -- 172.21.15.46:0/1164048383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1e3570 con 0x7f8a3c15e050 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a3be00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 0x7f8a3c1e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a41400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400efc0 tx=0x7f8a34009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34018030 con 0x7f8a3c15e050 2024-04-26T23:45:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3400cde0 con 0x7f8a3c15e050 2024-04-26T23:45:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34003bb0 con 0x7f8a3c15e050 2024-04-26T23:45:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.403+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1e3830 con 0x7f8a3c15e050 2024-04-26T23:45:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.404+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1e3df0 con 0x7f8a3c15e050 2024-04-26T23:45:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.405+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c069530 con 0x7f8a3c15e050 2024-04-26T23:45:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.406+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a34005d10 con 0x7f8a3c15e050 2024-04-26T23:45:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.407+0000 7f8a38c00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 0x7f8a1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.407+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f8a3408c560 con 0x7f8a3c15e050 2024-04-26T23:45:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.407+0000 7f8a40a00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 0x7f8a1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.408+0000 7f8a40a00640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 0x7f8a1c06edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f60 tx=0x7f8a30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.411+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a34022070 con 0x7f8a3c15e050 2024-04-26T23:45:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.618+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"} v 0) -- 0x7f8a3c0d03e0 con 0x7f8a3c15e050 2024-04-26T23:45:22.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.623+0000 7f8a38c00640 1 -- 172.21.15.46:0/1164048383 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"}]=0 v33) ==== 101+0+0 (secure 0 0 0) 0x7f8a34010040 con 0x7f8a3c15e050 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 msgr2=0x7f8a1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 0x7f8a1c06edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f60 tx=0x7f8a30009040 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 msgr2=0x7f8a3c1e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400efc0 tx=0x7f8a34009f90 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 shutdown_connections 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8a1c06c8f0 0x7f8a1c06edb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a3c15e050 0x7f8a3c1e55c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a3c15d730 0x7f8a3c1e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 --2- 172.21.15.46:0/1164048383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c0b53b0 0x7f8a3c13f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 >> 172.21.15.46:0/1164048383 conn(0x7f8a3c06c980 msgr2=0x7f8a3c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.626+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 shutdown_connections 2024-04-26T23:45:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.627+0000 7f8a43400640 1 -- 172.21.15.46:0/1164048383 wait complete. 2024-04-26T23:45:22.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags' 2024-04-26T23:45:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.857+0000 7f0053200640 1 Processor -- start 2024-04-26T23:45:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.857+0000 7f0053200640 1 -- start start 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0053200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f004c06c350 0x7f004c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0053200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f004c0cf860 0x7f004c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0053200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c0d1ab0 con 0x7f004c0d1350 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0053200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f004c06cc70 con 0x7f004c0cf860 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0053200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f004c0d17d0 con 0x7f004c06c350 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0052800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f004c0cf860 0x7f004c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0052800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f004c0cf860 0x7f004c0cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54990/0 (socket says 172.21.15.46:54990) 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.859+0000 7f0052800640 1 -- 172.21.15.46:0/3204403462 learned_addr learned my addr 172.21.15.46:0/3204403462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.860+0000 7f0051e00640 1 --2- 172.21.15.46:0/3204403462 >> v2:172.21.15.142:3300/0 conn(0x7f004c06c350 0x7f004c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.860+0000 7f0052800640 1 -- 172.21.15.46:0/3204403462 >> v2:172.21.15.142:3300/0 conn(0x7f004c06c350 msgr2=0x7f004c06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.860+0000 7f0052800640 1 --2- 172.21.15.46:0/3204403462 >> v2:172.21.15.142:3300/0 conn(0x7f004c06c350 0x7f004c06c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.860+0000 7f0052800640 1 -- 172.21.15.46:0/3204403462 >> v1:172.21.15.142:6789/0 conn(0x7f004c0d1350 legacy=0x7f004c06fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.860+0000 7f0052800640 1 -- 172.21.15.46:0/3204403462 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c06f0e0 con 0x7f004c0cf860 2024-04-26T23:45:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.861+0000 7f0052800640 1 --2- 172.21.15.46:0/3204403462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c0cc2e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f004c071a40 tx=0x7f004802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324da0a3c3a5124e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.861+0000 7f0050a00640 1 -- 172.21.15.46:0/3204403462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048037070 con 0x7f004c0cf860 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.861+0000 7f0050a00640 1 -- 172.21.15.46:0/3204403462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0048032440 con 0x7f004c0cf860 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.861+0000 7f0050a00640 1 -- 172.21.15.46:0/3204403462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048032810 con 0x7f004c0cf860 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 -- 172.21.15.46:0/3204403462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 msgr2=0x7f004c0cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 --2- 172.21.15.46:0/3204403462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c0cc2e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f004c071a40 tx=0x7f004802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 -- 172.21.15.46:0/3204403462 shutdown_connections 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 --2- 172.21.15.46:0/3204403462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c0cc2e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 --2- 172.21.15.46:0/3204403462 >> v2:172.21.15.142:3300/0 conn(0x7f004c06c350 0x7f004c06c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 -- 172.21.15.46:0/3204403462 >> 172.21.15.46:0/3204403462 conn(0x7f004c0d1c90 msgr2=0x7f004c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 -- 172.21.15.46:0/3204403462 shutdown_connections 2024-04-26T23:45:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.862+0000 7f0053200640 1 -- 172.21.15.46:0/3204403462 wait complete. 2024-04-26T23:45:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.864+0000 7f0053200640 1 Processor -- start 2024-04-26T23:45:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.864+0000 7f0053200640 1 -- start start 2024-04-26T23:45:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f004c06c350 0x7f004c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f004c0d1350 0x7f004c1595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f004c070c10 con 0x7f004c0cf860 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f004c06f6a0 con 0x7f004c0d1350 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0053200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f004c146060 con 0x7f004c06c350 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0051400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0051400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55000/0 (socket says 172.21.15.46:55000) 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.865+0000 7f0051400640 1 -- 172.21.15.46:0/2176066836 learned_addr learned my addr 172.21.15.46:0/2176066836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051e00640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f004c06c350 0x7f004c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0052800640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f004c0d1350 0x7f004c1595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 -- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f004c06c350 msgr2=0x7f004c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f004c06c350 0x7f004c1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 -- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f004c0d1350 msgr2=0x7f004c1595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f004c0d1350 0x7f004c1595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 -- 172.21.15.46:0/2176066836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c15f330 con 0x7f004c0cf860 2024-04-26T23:45:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.866+0000 7f0051400640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f003c00b3f0 tx=0x7f003c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.867+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c015020 con 0x7f004c0cf860 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.867+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c011830 con 0x7f004c0cf860 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.867+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c011c20 con 0x7f004c0cf860 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.867+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c15f610 con 0x7f004c0cf860 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.867+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c15fbd0 con 0x7f004c0cf860 2024-04-26T23:45:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.869+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c069ac0 con 0x7f004c0cf860 2024-04-26T23:45:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.869+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f003c018020 con 0x7f004c0cf860 2024-04-26T23:45:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.870+0000 7f0042000640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.871+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f003c08b770 con 0x7f004c0cf860 2024-04-26T23:45:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.871+0000 7f0051e00640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.872+0000 7f0051e00640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 0x7f003006edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f004c1406a0 tx=0x7f00440039b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:22.875+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f003c01f030 con 0x7f004c0cf860 2024-04-26T23:45:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.156+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags"]} v 0) -- 0x7f004c0cf180 con 0x7f004c0cf860 2024-04-26T23:45:23.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.766+0000 7f0042000640 1 -- 172.21.15.46:0/2176066836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_split_dirfrags"]}]=0 v414) ==== 151+0+0 (secure 0 0 0) 0x7f003c0172c0 con 0x7f004c0cf860 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 msgr2=0x7f003006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 0x7f003006edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f004c1406a0 tx=0x7f00440039b0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 msgr2=0x7f004c159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f003c00b3f0 tx=0x7f003c00b8b0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 shutdown_connections 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f004c0d1350 0x7f004c1595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004c0cf860 0x7f004c159060 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 --2- 172.21.15.46:0/2176066836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f004c06c350 0x7f004c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 >> 172.21.15.46:0/2176066836 conn(0x7f004c0d1c90 msgr2=0x7f004c144660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.770+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 shutdown_connections 2024-04-26T23:45:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:23.771+0000 7f0053200640 1 -- 172.21.15.46:0/2176066836 wait complete. 2024-04-26T23:45:23.801 INFO:tasks.cephfs_test_runner:That a directory is split during scrub. ... ok 2024-04-26T23:45:23.801 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-26T23:45:23.801 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-26T23:45:23.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.007+0000 7fcd34a00640 1 Processor -- start 2024-04-26T23:45:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.007+0000 7fcd34a00640 1 -- start start 2024-04-26T23:45:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd34a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd301466b0 0x7fcd3006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd34a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd300720e0 0x7fcd300cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd34a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd30073ad0 con 0x7fcd30160910 2024-04-26T23:45:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd34a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcd30146a90 con 0x7fcd300720e0 2024-04-26T23:45:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd34a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcd30073d70 con 0x7fcd301466b0 2024-04-26T23:45:24.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd26a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd300720e0 0x7fcd300cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd26a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd300720e0 0x7fcd300cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55012/0 (socket says 172.21.15.46:55012) 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.009+0000 7fcd26a00640 1 -- 172.21.15.46:0/3023262732 learned_addr learned my addr 172.21.15.46:0/3023262732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.010+0000 7fcd26a00640 1 -- 172.21.15.46:0/3023262732 >> v2:172.21.15.142:3300/0 conn(0x7fcd301466b0 msgr2=0x7fcd3006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.010+0000 7fcd2f400640 1 --2- 172.21.15.46:0/3023262732 >> v2:172.21.15.142:3300/0 conn(0x7fcd301466b0 0x7fcd3006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.010+0000 7fcd26a00640 1 --2- 172.21.15.46:0/3023262732 >> v2:172.21.15.142:3300/0 conn(0x7fcd301466b0 0x7fcd3006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.010+0000 7fcd26a00640 1 -- 172.21.15.46:0/3023262732 >> v1:172.21.15.142:6789/0 conn(0x7fcd30160910 legacy=0x7fcd300cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.010+0000 7fcd26a00640 1 -- 172.21.15.46:0/3023262732 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd30074050 con 0x7fcd300720e0 2024-04-26T23:45:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.011+0000 7fcd26a00640 1 --2- 172.21.15.46:0/3023262732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd300720e0 0x7fcd300cd860 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e50 tx=0x7fcd1802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31195947aa041ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.011+0000 7fcd2ea00640 1 -- 172.21.15.46:0/3023262732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18036070 con 0x7fcd300720e0 2024-04-26T23:45:24.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd2ea00640 1 -- 172.21.15.46:0/3023262732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18031600 con 0x7fcd300720e0 2024-04-26T23:45:24.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd2ea00640 1 -- 172.21.15.46:0/3023262732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180319d0 con 0x7fcd300720e0 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 -- 172.21.15.46:0/3023262732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd300720e0 msgr2=0x7fcd300cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3023262732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd300720e0 0x7fcd300cd860 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e50 tx=0x7fcd1802a710 comp rx=0 tx=0).stop 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 -- 172.21.15.46:0/3023262732 shutdown_connections 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3023262732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd300720e0 0x7fcd300cd860 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3023262732 >> v2:172.21.15.142:3300/0 conn(0x7fcd301466b0 0x7fcd3006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.012+0000 7fcd34a00640 1 -- 172.21.15.46:0/3023262732 >> 172.21.15.46:0/3023262732 conn(0x7fcd300b26d0 msgr2=0x7fcd300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:24.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.013+0000 7fcd34a00640 1 -- 172.21.15.46:0/3023262732 shutdown_connections 2024-04-26T23:45:24.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.013+0000 7fcd34a00640 1 -- 172.21.15.46:0/3023262732 wait complete. 2024-04-26T23:45:24.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.014+0000 7fcd34a00640 1 Processor -- start 2024-04-26T23:45:24.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.014+0000 7fcd34a00640 1 -- start start 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd300720e0 0x7fcd300b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 0x7fcd3015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3006e3c0 con 0x7fcd301466b0 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300cb470 con 0x7fcd30160910 2024-04-26T23:45:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.015+0000 7fcd34a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd300cdda0 con 0x7fcd300720e0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55020/0 (socket says 172.21.15.46:55020) 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 -- 172.21.15.46:0/3225958332 learned_addr learned my addr 172.21.15.46:0/3225958332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd2f400640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd300720e0 0x7fcd300b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 -- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd300720e0 msgr2=0x7fcd300b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd300720e0 0x7fcd300b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 -- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 msgr2=0x7fcd3015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd2fe00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 0x7fcd3015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 0x7fcd3015f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd26a00640 1 -- 172.21.15.46:0/3225958332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd301f8bb0 con 0x7fcd301466b0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.016+0000 7fcd2fe00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 0x7fcd3015f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd26a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fcd18000f80 tx=0x7fcd18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18036040 con 0x7fcd301466b0 2024-04-26T23:45:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18003990 con 0x7fcd301466b0 2024-04-26T23:45:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003d60 con 0x7fcd301466b0 2024-04-26T23:45:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301f8e90 con 0x7fcd301466b0 2024-04-26T23:45:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.017+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd300afa60 con 0x7fcd301466b0 2024-04-26T23:45:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.018+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd300693b0 con 0x7fcd301466b0 2024-04-26T23:45:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.019+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd18008730 con 0x7fcd301466b0 2024-04-26T23:45:24.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.020+0000 7fcd27e00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 0x7fcd0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.020+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7fcd180ace80 con 0x7fcd301466b0 2024-04-26T23:45:24.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.021+0000 7fcd2f400640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 0x7fcd0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.022+0000 7fcd2f400640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 0x7fcd0c06edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b7c0 tx=0x7fcd20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.025+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd1807b400 con 0x7fcd301466b0 2024-04-26T23:45:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.220+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd300c7180 con 0x7fcd301466b0 2024-04-26T23:45:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.222+0000 7fcd27e00640 1 -- 172.21.15.46:0/3225958332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+13519 (secure 0 0 0) 0x7fcd1807ad00 con 0x7fcd301466b0 2024-04-26T23:45:24.227 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:24.227 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:45:20.739079+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2779258602":"2024-04-27T23:45:17.719891+0000","172.21.15.46:6834/2779258602":"2024-04-27T23:45:17.719891+0000","172.21.15.142:6835/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.142:6834/932747087":"2024-04-27T23:44:57.986346+0000","172.21.15.46:6835/2670265715":"2024-04-27T23:44:56.993227+0000","172.21.15.46:6833/2670265715":"2024-04-27T23:44:56.993227+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.224+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 msgr2=0x7fcd0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.224+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 0x7fcd0c06edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b7c0 tx=0x7fcd20008040 comp rx=0 tx=0).stop 2024-04-26T23:45:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.224+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 msgr2=0x7fcd301f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.224+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fcd18000f80 tx=0x7fcd18004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 shutdown_connections 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd0c06c8f0 0x7fcd0c06edb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd30160910 0x7fcd3015f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd301466b0 0x7fcd301f8380 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 --2- 172.21.15.46:0/3225958332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd300720e0 0x7fcd300b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 >> 172.21.15.46:0/3225958332 conn(0x7fcd300b26d0 msgr2=0x7fcd300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 shutdown_connections 2024-04-26T23:45:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.225+0000 7fcd34a00640 1 -- 172.21.15.46:0/3225958332 wait complete. 2024-04-26T23:45:24.247 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:45:24.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:45:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.453+0000 7f5e03200640 1 Processor -- start 2024-04-26T23:45:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.454+0000 7f5e03200640 1 -- start start 2024-04-26T23:45:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.455+0000 7f5e03200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc0cdd20 0x7f5dfc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.455+0000 7f5e03200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dfc0ae8f0 0x7f5dfc0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.455+0000 7f5e03200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dfc0af2a0 con 0x7f5dfc15f210 2024-04-26T23:45:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.455+0000 7f5e03200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5dfc0af570 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.455+0000 7f5e03200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc0af850 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e02800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dfc0ae8f0 0x7f5dfc0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e01e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5dfc15f210 0x7f5dfc0cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46842/0 (socket says 172.21.15.46:46842) 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e01e00640 1 -- 172.21.15.46:0/767752559 learned_addr learned my addr 172.21.15.46:0/767752559 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e01400640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc0cdd20 0x7f5dfc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e02800640 1 -- 172.21.15.46:0/767752559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc0cdd20 msgr2=0x7f5dfc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e02800640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc0cdd20 0x7f5dfc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e02800640 1 -- 172.21.15.46:0/767752559 >> v1:172.21.15.46:6789/0 conn(0x7f5dfc15f210 legacy=0x7f5dfc0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.456+0000 7f5e02800640 1 -- 172.21.15.46:0/767752559 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0cb680 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.458+0000 7f5e02800640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0aed60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1ed1a74434c646 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.458+0000 7f5e00a00640 1 -- 172.21.15.46:0/767752559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007740 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.458+0000 7f5e00a00640 1 -- 172.21.15.46:0/767752559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80078e0 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.458+0000 7f5e00a00640 1 -- 172.21.15.46:0/767752559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007cb0 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 -- 172.21.15.46:0/767752559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0ae8f0 msgr2=0x7f5dfc0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0aed60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df802a660 comp rx=0 tx=0).stop 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 -- 172.21.15.46:0/767752559 shutdown_connections 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0aed60 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 --2- 172.21.15.46:0/767752559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc0cdd20 0x7f5dfc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 -- 172.21.15.46:0/767752559 >> 172.21.15.46:0/767752559 conn(0x7f5dfc0b27b0 msgr2=0x7f5dfc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 -- 172.21.15.46:0/767752559 shutdown_connections 2024-04-26T23:45:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.459+0000 7f5e03200640 1 -- 172.21.15.46:0/767752559 wait complete. 2024-04-26T23:45:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.460+0000 7f5e03200640 1 Processor -- start 2024-04-26T23:45:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.461+0000 7f5e03200640 1 -- start start 2024-04-26T23:45:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc15f210 0x7f5dfc156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc15f5f0 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0cbc40 con 0x7f5dfc0ae8f0 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e03200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc071e80 con 0x7f5dfc15f210 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55042/0 (socket says 172.21.15.46:55042) 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 -- 172.21.15.46:0/425299721 learned_addr learned my addr 172.21.15.46:0/425299721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01e00640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 -- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc15f210 msgr2=0x7f5dfc156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc15f210 0x7f5dfc156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 -- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 msgr2=0x7f5dfc0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01400640 1 -- 172.21.15.46:0/425299721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0c7e70 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.462+0000 7f5e01e00640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5e01400640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d830 tx=0x7f5dec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00b6c0 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec019070 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00ba80 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc0c8150 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.463+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1571f0 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.465+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc0693b0 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.465+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dec01e460 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.466+0000 7f5df6000640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 0x7f5de006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.466+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f5dec08c210 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.467+0000 7f5e01e00640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 0x7f5de006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.468+0000 7f5e01e00640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 0x7f5de006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5df0000f60 tx=0x7f5df0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.471+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5dec05a660 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.680+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5dfc157450 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.682+0000 7f5df6000640 1 -- 172.21.15.46:0/425299721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f5dec059f60 con 0x7f5dfc0cdd20 2024-04-26T23:45:24.686 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:45:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.684+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 msgr2=0x7f5de006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.684+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 0x7f5de006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5df0000f60 tx=0x7f5df0009040 comp rx=0 tx=0).stop 2024-04-26T23:45:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.684+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 msgr2=0x7f5dfc0c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.684+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d830 tx=0x7f5dec00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 shutdown_connections 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5de006c8f0 0x7f5de006edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dfc15f210 0x7f5dfc156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dfc0cdd20 0x7f5dfc0c8c30 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 --2- 172.21.15.46:0/425299721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dfc0ae8f0 0x7f5dfc0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 >> 172.21.15.46:0/425299721 conn(0x7f5dfc0b27b0 msgr2=0x7f5dfc15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 shutdown_connections 2024-04-26T23:45:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.685+0000 7f5e03200640 1 -- 172.21.15.46:0/425299721 wait complete. 2024-04-26T23:45:24.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-26T23:45:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.921+0000 7f0d95200640 1 Processor -- start 2024-04-26T23:45:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.921+0000 7f0d95200640 1 -- start start 2024-04-26T23:45:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d95200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0d900cff40 0x7f0d90073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d95200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d95200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d9006f1d0 con 0x7f0d900cf450 2024-04-26T23:45:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d95200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9006f470 con 0x7f0d900cff40 2024-04-26T23:45:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d95200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d90072930 con 0x7f0d90072190 2024-04-26T23:45:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.923+0000 7f0d8fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:37644/0 (socket says 172.21.15.46:37644) 2024-04-26T23:45:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 -- 172.21.15.46:0/2842856769 learned_addr learned my addr 172.21.15.46:0/2842856769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8ea00640 1 --2- 172.21.15.46:0/2842856769 >> v2:172.21.15.142:3300/0 conn(0x7f0d900cff40 0x7f0d90073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 -- 172.21.15.46:0/2842856769 >> v2:172.21.15.142:3300/0 conn(0x7f0d900cff40 msgr2=0x7f0d90073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/2842856769 >> v2:172.21.15.142:3300/0 conn(0x7f0d900cff40 0x7f0d90073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 -- 172.21.15.46:0/2842856769 >> v1:172.21.15.142:6789/0 conn(0x7f0d900cf450 legacy=0x7f0d900cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8e000640 1 -- 172.21.15.46:0/2842856769 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573130822 0 0) 0x7f0d9006f1d0 con 0x7f0d900cf450 2024-04-26T23:45:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.924+0000 7f0d8fe00640 1 -- 172.21.15.46:0/2842856769 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d90072c10 con 0x7f0d90072190 2024-04-26T23:45:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.926+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/2842856769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009e50 tx=0x7f0d8802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682a800edbd222c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.926+0000 7f0d8e000640 1 -- 172.21.15.46:0/2842856769 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88036070 con 0x7f0d90072190 2024-04-26T23:45:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.926+0000 7f0d8e000640 1 -- 172.21.15.46:0/2842856769 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d88031600 con 0x7f0d90072190 2024-04-26T23:45:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.926+0000 7f0d8e000640 1 -- 172.21.15.46:0/2842856769 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880319d0 con 0x7f0d90072190 2024-04-26T23:45:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 -- 172.21.15.46:0/2842856769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 msgr2=0x7f0d90166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 --2- 172.21.15.46:0/2842856769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009e50 tx=0x7f0d8802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 -- 172.21.15.46:0/2842856769 shutdown_connections 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 --2- 172.21.15.46:0/2842856769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d90072190 0x7f0d90166b00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 --2- 172.21.15.46:0/2842856769 >> v2:172.21.15.142:3300/0 conn(0x7f0d900cff40 0x7f0d90073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 -- 172.21.15.46:0/2842856769 >> 172.21.15.46:0/2842856769 conn(0x7f0d900b2820 msgr2=0x7f0d900b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 -- 172.21.15.46:0/2842856769 shutdown_connections 2024-04-26T23:45:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.927+0000 7f0d95200640 1 -- 172.21.15.46:0/2842856769 wait complete. 2024-04-26T23:45:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.929+0000 7f0d95200640 1 Processor -- start 2024-04-26T23:45:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.929+0000 7f0d95200640 1 -- start start 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d900cf450 0x7f0d900b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d900cff40 0x7f0d9006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d900734b0 con 0x7f0d90072190 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d900731d0 con 0x7f0d900cff40 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.930+0000 7f0d95200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d900725e0 con 0x7f0d900cf450 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55050/0 (socket says 172.21.15.46:55050) 2024-04-26T23:45:24.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 -- 172.21.15.46:0/4020421960 learned_addr learned my addr 172.21.15.46:0/4020421960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 -- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d900cf450 msgr2=0x7f0d900b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d900cf450 0x7f0d900b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 -- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d900cff40 msgr2=0x7f0d9006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d900cff40 0x7f0d9006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 -- 172.21.15.46:0/4020421960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d900aefb0 con 0x7f0d90072190 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.931+0000 7f0d8f400640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c3c0 tx=0x7f0d8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.932+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80017070 con 0x7f0d90072190 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.932+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d800035f0 con 0x7f0d90072190 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.932+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d800039e0 con 0x7f0d90072190 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.932+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d900af570 con 0x7f0d90072190 2024-04-26T23:45:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.932+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d900afb30 con 0x7f0d90072190 2024-04-26T23:45:24.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.933+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d900afd30 con 0x7f0d90072190 2024-04-26T23:45:24.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.934+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d8001a070 con 0x7f0d90072190 2024-04-26T23:45:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.935+0000 7f0d87400640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.935+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f0d8008ae60 con 0x7f0d90072190 2024-04-26T23:45:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.936+0000 7f0d8ea00640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.937+0000 7f0d8ea00640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 0x7f0d7006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006a90 tx=0x7f0d7c006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:24.940+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d900afd30 con 0x7f0d90072190 2024-04-26T23:45:25.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.217+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) -- 0x7f0d900af290 con 0x7f0d90072190 2024-04-26T23:45:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.778+0000 7f0d87400640 1 -- 172.21.15.46:0/4020421960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v416) ==== 161+0+0 (secure 0 0 0) 0x7f0d800592b0 con 0x7f0d90072190 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 msgr2=0x7f0d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 0x7f0d7006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006a90 tx=0x7f0d7c006a20 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 msgr2=0x7f0d900b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c3c0 tx=0x7f0d8000c880 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 shutdown_connections 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d900cff40 0x7f0d9006e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d900cf450 0x7f0d900b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 --2- 172.21.15.46:0/4020421960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d90072190 0x7f0d900b0050 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.783+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 >> 172.21.15.46:0/4020421960 conn(0x7f0d900b2820 msgr2=0x7f0d900b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.784+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 shutdown_connections 2024-04-26T23:45:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:25.784+0000 7f0d95200640 1 -- 172.21.15.46:0/4020421960 wait complete. 2024-04-26T23:45:25.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:45:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.015+0000 7fe896000640 1 Processor -- start 2024-04-26T23:45:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.016+0000 7fe896000640 1 -- start start 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.017+0000 7fe896000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe89015f480 0x7fe8900d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.017+0000 7fe896000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8901630a0 0x7fe890163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.017+0000 7fe896000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe890163e70 con 0x7fe8900d1470 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe896000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8900d0aa0 con 0x7fe8901630a0 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe896000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900d0170 con 0x7fe89015f480 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe8900d1470 0x7fe8900d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46848/0 (socket says 172.21.15.46:46848) 2024-04-26T23:45:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88ea00640 1 -- 172.21.15.46:0/2996283853 learned_addr learned my addr 172.21.15.46:0/2996283853 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693448706 0 0) 0x7fe890163e70 con 0x7fe8900d1470 2024-04-26T23:45:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88f400640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe89015f480 0x7fe8900d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe870003520 con 0x7fe8900d1470 2024-04-26T23:45:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.018+0000 7fe88fe00640 1 --2- 172.21.15.46:0/2996283853 >> v2:172.21.15.142:3300/0 conn(0x7fe8901630a0 0x7fe890163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36106410 0 0) 0x7fe870003520 con 0x7fe8900d1470 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe89015f480 msgr2=0x7fe8900d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe89015f480 0x7fe8900d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 >> v2:172.21.15.142:3300/0 conn(0x7fe8901630a0 msgr2=0x7fe890163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 --2- 172.21.15.46:0/2996283853 >> v2:172.21.15.142:3300/0 conn(0x7fe8901630a0 0x7fe890163550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900d0450 con 0x7fe8900d1470 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe8780019f0 con 0x7fe8900d1470 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 >> v1:172.21.15.46:6789/0 conn(0x7fe8900d1470 legacy=0x7fe8900d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8700039b0 0x7fe870003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe870004800 0x7fe870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8700051c0 0x7fe87000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900d0aa0 con 0x7fe870004800 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900d0170 con 0x7fe8700051c0 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe87000c040 con 0x7fe8700039b0 2024-04-26T23:45:26.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88ea00640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe870004800 0x7fe870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88f400640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8700039b0 0x7fe870003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.019+0000 7fe88fe00640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8700051c0 0x7fe87000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.020+0000 7fe88ea00640 1 -- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8700039b0 msgr2=0x7fe870003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.020+0000 7fe88ea00640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8700039b0 0x7fe870003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.020+0000 7fe88ea00640 1 -- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8700051c0 msgr2=0x7fe87000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.020+0000 7fe88ea00640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8700051c0 0x7fe87000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.020+0000 7fe88ea00640 1 -- 172.21.15.46:0/2996283853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe870003520 con 0x7fe870004800 2024-04-26T23:45:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.021+0000 7fe88ea00640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe870004800 0x7fe870004c80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe8900d1870 tx=0x7fe870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb10e425b7ae740 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.021+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87800a3f0 con 0x7fe870004800 2024-04-26T23:45:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.021+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87802f070 con 0x7fe870004800 2024-04-26T23:45:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.021+0000 7fe88e000640 1 -- 172.21.15.46:0/2996283853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87802bc80 con 0x7fe870004800 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 -- 172.21.15.46:0/2996283853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe870004800 msgr2=0x7fe870004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 --2- 172.21.15.46:0/2996283853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe870004800 0x7fe870004c80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe8900d1870 tx=0x7fe870005830 comp rx=0 tx=0).stop 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe88fe00640 1 -- 172.21.15.46:0/2996283853 reap_dead start 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 -- 172.21.15.46:0/2996283853 shutdown_connections 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 -- 172.21.15.46:0/2996283853 >> 172.21.15.46:0/2996283853 conn(0x7fe8900d5070 msgr2=0x7fe8900d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 -- 172.21.15.46:0/2996283853 shutdown_connections 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.022+0000 7fe896000640 1 -- 172.21.15.46:0/2996283853 wait complete. 2024-04-26T23:45:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.024+0000 7fe896000640 1 Processor -- start 2024-04-26T23:45:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.024+0000 7fe896000640 1 -- start start 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe870004800 0x7fe8900cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 0x7fe8900a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe89015f860 con 0x7fe89009f080 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900cfe60 con 0x7fe89015a6f0 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe896000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe89015f9e0 con 0x7fe870004800 2024-04-26T23:45:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55068/0 (socket says 172.21.15.46:55068) 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88fe00640 1 -- 172.21.15.46:0/3913106117 learned_addr learned my addr 172.21.15.46:0/3913106117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88ea00640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 0x7fe8900a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88f400640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe870004800 0x7fe8900cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88fe00640 1 -- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe870004800 msgr2=0x7fe8900cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.025+0000 7fe88fe00640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe870004800 0x7fe8900cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe88fe00640 1 -- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 msgr2=0x7fe8900a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe88fe00640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 0x7fe8900a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe88fe00640 1 -- 172.21.15.46:0/3913106117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900a19b0 con 0x7fe89009f080 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe88ea00640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 0x7fe8900a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe88fe00640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe88800f410 tx=0x7fe88800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888017020 con 0x7fe89009f080 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe888004460 con 0x7fe89009f080 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888003850 con 0x7fe89009f080 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8900a1c90 con 0x7fe89009f080 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.026+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8900a2250 con 0x7fe89009f080 2024-04-26T23:45:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.028+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89008d8a0 con 0x7fe89009f080 2024-04-26T23:45:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.028+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe88801a020 con 0x7fe89009f080 2024-04-26T23:45:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.029+0000 7fe887400640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 0x7fe86c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.029+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7fe88808b920 con 0x7fe89009f080 2024-04-26T23:45:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.030+0000 7fe88f400640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 0x7fe86c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.031+0000 7fe88f400640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 0x7fe86c06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe8900baa90 tx=0x7fe880008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.035+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe89008d8a0 con 0x7fe89009f080 2024-04-26T23:45:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.241+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe890069530 con 0x7fe89009f080 2024-04-26T23:45:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.248+0000 7fe887400640 1 -- 172.21.15.46:0/3913106117 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fe888021020 con 0x7fe89009f080 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 msgr2=0x7fe86c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 0x7fe86c06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe8900baa90 tx=0x7fe880008040 comp rx=0 tx=0).stop 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 msgr2=0x7fe89009f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe88800f410 tx=0x7fe88800f8d0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 shutdown_connections 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86c06c8f0 0x7fe86c06edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe89009f080 0x7fe89009f500 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe89015a6f0 0x7fe8900a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 --2- 172.21.15.46:0/3913106117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe870004800 0x7fe8900cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.251+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 >> 172.21.15.46:0/3913106117 conn(0x7fe8900d5070 msgr2=0x7fe890160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.252+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 shutdown_connections 2024-04-26T23:45:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.252+0000 7fe896000640 1 -- 172.21.15.46:0/3913106117 wait complete. 2024-04-26T23:45:26.275 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:45:26.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:45:26.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:45:26.489 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:45:26.489 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:26.490 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:26.550 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:45:26.550 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:45:26.551 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:26.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:45:26.606 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:26.606 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:26.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:45:26.665 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:45:26.665 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:26.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:45:26.722 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:45:26.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.944+0000 7f45c3e00640 1 Processor -- start 2024-04-26T23:45:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.945+0000 7f45c3e00640 1 -- start start 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.946+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45c40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.946+0000 7f45c3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c40d11f0 con 0x7f45c40b4000 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.946+0000 7f45c3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c40d5f20 con 0x7f45c415f4e0 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.946+0000 7f45c3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45c40d0ce0 con 0x7f45c40d2130 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.947+0000 7f45c3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f45c40b4000 0x7f45c40b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46850/0 (socket says 172.21.15.46:46850) 2024-04-26T23:45:26.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.947+0000 7f45c3400640 1 -- 172.21.15.46:0/1656200062 learned_addr learned my addr 172.21.15.46:0/1656200062 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.947+0000 7f45bbe00640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45c40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.947+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328947877 0 0) 0x7f45c40d11f0 con 0x7f45c40b4000 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.947+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45c40b4000 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3734911713 0 0) 0x7f45a8003520 con 0x7f45c40b4000 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 msgr2=0x7f45c40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45c40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 >> v1:172.21.15.142:6789/0 conn(0x7f45c415f4e0 legacy=0x7f45c40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c40d0f10 con 0x7f45c40b4000 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f45bc0019f0 con 0x7f45c40b4000 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 >> v1:172.21.15.46:6789/0 conn(0x7f45c40b4000 legacy=0x7f45c40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800ba70 con 0x7f45a8004d40 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800bc20 con 0x7f45a80039e0 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800be20 con 0x7f45c40d2130 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c3400640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.948+0000 7f45c2a00640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45bbe00640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45c3400640 1 -- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 msgr2=0x7f45a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45c3400640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45c40d2130 0x7f45a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45c3400640 1 -- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45a80039e0 msgr2=0x7f45a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45c3400640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.949+0000 7f45c3400640 1 -- 172.21.15.46:0/1656200062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003520 con 0x7f45a8004d40 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.950+0000 7f45c3400640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00a740 tx=0x7f45a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d89bd1c85bf65d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.950+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00a390 con 0x7f45a8004d40 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.950+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc02f070 con 0x7f45a8004d40 2024-04-26T23:45:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.950+0000 7f45c2000640 1 -- 172.21.15.46:0/1656200062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc02bbc0 con 0x7f45a8004d40 2024-04-26T23:45:26.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 -- 172.21.15.46:0/1656200062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 --2- 172.21.15.46:0/1656200062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00a740 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3400640 1 -- 172.21.15.46:0/1656200062 reap_dead start 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 -- 172.21.15.46:0/1656200062 shutdown_connections 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 -- 172.21.15.46:0/1656200062 >> 172.21.15.46:0/1656200062 conn(0x7f45c40d5010 msgr2=0x7f45c40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 -- 172.21.15.46:0/1656200062 shutdown_connections 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.951+0000 7f45c3e00640 1 -- 172.21.15.46:0/1656200062 wait complete. 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.953+0000 7f45c3e00640 1 Processor -- start 2024-04-26T23:45:26.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.953+0000 7f45c3e00640 1 -- start start 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45a8004d40 0x7f45c40cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c415d850 0x7f45c4161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40b1d60 con 0x7f45c415cc20 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40d2990 con 0x7f45c415d850 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45c415f8c0 con 0x7f45a8004d40 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.954+0000 7f45c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55100/0 (socket says 172.21.15.46:55100) 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 -- 172.21.15.46:0/2577255347 learned_addr learned my addr 172.21.15.46:0/2577255347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c2a00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45a8004d40 0x7f45c40cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45bbe00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c415d850 0x7f45c4161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 -- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45a8004d40 msgr2=0x7f45c40cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45a8004d40 0x7f45c40cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 -- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c415d850 msgr2=0x7f45c4161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c415d850 0x7f45c4161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 -- 172.21.15.46:0/2577255347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c415d5e0 con 0x7f45c415cc20 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.955+0000 7f45c3400640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f80 tx=0x7f45c40d10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.956+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc001ee0 con 0x7f45c415cc20 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.956+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc02bb70 con 0x7f45c415cc20 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.956+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c40ccff0 con 0x7f45c415cc20 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.956+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc034470 con 0x7f45c415cc20 2024-04-26T23:45:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.956+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c40cd5b0 con 0x7f45c415cc20 2024-04-26T23:45:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.957+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c40693b0 con 0x7f45c415cc20 2024-04-26T23:45:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.958+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45bc034610 con 0x7f45c415cc20 2024-04-26T23:45:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.959+0000 7f45bb400640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 0x7f45a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.959+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f45bc0abc10 con 0x7f45c415cc20 2024-04-26T23:45:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.960+0000 7f45c2a00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 0x7f45a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.961+0000 7f45c2a00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 0x7f45a006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f45b40035f0 tx=0x7f45b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:26.963+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45c415d5e0 con 0x7f45c415cc20 2024-04-26T23:45:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.210+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45c40d33d0 con 0x7f45c415cc20 2024-04-26T23:45:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.211+0000 7f45bb400640 1 -- 172.21.15.46:0/2577255347 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+3869 (secure 0 0 0) 0x7f45bc07a060 con 0x7f45c415cc20 2024-04-26T23:45:27.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:27.216 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/847706429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":847706429},{"type":"v1","addr":"172.21.15.142:6835","nonce":847706429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2209650837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2209650837},{"type":"v1","addr":"172.21.15.142:6837","nonce":2209650837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7947,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/591028231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":591028231},{"type":"v1","addr":"172.21.15.46:6833","nonce":591028231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1266794995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1266794995},{"type":"v1","addr":"172.21.15.46:6835","nonce":1266794995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[]} 2024-04-26T23:45:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 84 2024-04-26T23:45:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.213+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 msgr2=0x7f45a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.213+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 0x7f45a006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f45b40035f0 tx=0x7f45b4009040 comp rx=0 tx=0).stop 2024-04-26T23:45:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.213+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 msgr2=0x7f45c415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.213+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f80 tx=0x7f45c40d10e0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 shutdown_connections 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45a006c8f0 0x7f45a006edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c415cc20 0x7f45c415d0a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c415d850 0x7f45c4161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 --2- 172.21.15.46:0/2577255347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45a8004d40 0x7f45c40cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 >> 172.21.15.46:0/2577255347 conn(0x7f45c40d5010 msgr2=0x7f45c406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 shutdown_connections 2024-04-26T23:45:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.214+0000 7f45c3e00640 1 -- 172.21.15.46:0/2577255347 wait complete. 2024-04-26T23:45:27.238 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:45:27.238 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:45:27.239 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:45:27.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:45:27.239 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:45:27.239 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:45:27.239 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:45:27.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:45:27.239 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:45:27.239 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:45:27.240 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:45:27.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:45:27.240 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:45:27.240 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:45:27.240 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:45:27.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:45:27.256 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:45:27.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:45:27.258 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:45:27.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:45:27.265 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:45:27.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:45:27.266 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:45:27.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:45:27.303 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:45:27.304 DEBUG:teuthology.parallel:result is None 2024-04-26T23:45:27.308 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:45:27.308 DEBUG:teuthology.parallel:result is None 2024-04-26T23:45:27.312 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:45:27.312 DEBUG:teuthology.parallel:result is None 2024-04-26T23:45:27.318 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:45:27.319 DEBUG:teuthology.parallel:result is None 2024-04-26T23:45:27.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:45:27.445 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:45:27.448 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:45:27.463 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:45:27.466 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:45:27.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.489+0000 7f9ff1200640 1 Processor -- start 2024-04-26T23:45:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.489+0000 7f9ff1200640 1 -- start start 2024-04-26T23:45:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.490+0000 7f9ff1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fec15f4b0 0x7f9fec0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.490+0000 7f9ff1200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec0cfd60 con 0x7f9fec0b5140 2024-04-26T23:45:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.490+0000 7f9ff1200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec0d2910 con 0x7f9fec0d55e0 2024-04-26T23:45:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.490+0000 7f9ff1200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec15f890 con 0x7f9fec15f4b0 2024-04-26T23:45:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fec15f4b0 0x7f9fec0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9fec0b5140 0x7f9fec0d4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46870/0 (socket says 172.21.15.46:46870) 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9400640 1 -- 172.21.15.46:0/3712089570 learned_addr learned my addr 172.21.15.46:0/3712089570 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9e00640 1 -- 172.21.15.46:0/3712089570 >> v1:172.21.15.142:6789/0 conn(0x7f9fec0d55e0 legacy=0x7f9fec0d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe8a00640 1 -- 172.21.15.46:0/3712089570 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701680830 0 0) 0x7f9fd8001110 con 0x7f9fec0b5140 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9e00640 1 -- 172.21.15.46:0/3712089570 >> v1:172.21.15.46:6789/0 conn(0x7f9fec0b5140 legacy=0x7f9fec0d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.491+0000 7f9fe9e00640 1 -- 172.21.15.46:0/3712089570 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0b55d0 con 0x7f9fec15f4b0 2024-04-26T23:45:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.492+0000 7f9fe9e00640 1 --2- 172.21.15.46:0/3712089570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0d23d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0097a0 tx=0x7f9fdc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ab99ae4b7453 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.492+0000 7f9fe8a00640 1 -- 172.21.15.46:0/3712089570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc037070 con 0x7f9fec15f4b0 2024-04-26T23:45:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.492+0000 7f9fe8a00640 1 -- 172.21.15.46:0/3712089570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc032440 con 0x7f9fec15f4b0 2024-04-26T23:45:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.492+0000 7f9fe8a00640 1 -- 172.21.15.46:0/3712089570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc032810 con 0x7f9fec15f4b0 2024-04-26T23:45:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 -- 172.21.15.46:0/3712089570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 msgr2=0x7f9fec0d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3712089570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0d23d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0097a0 tx=0x7f9fdc02a230 comp rx=0 tx=0).stop 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 -- 172.21.15.46:0/3712089570 shutdown_connections 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3712089570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0d23d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 -- 172.21.15.46:0/3712089570 >> 172.21.15.46:0/3712089570 conn(0x7f9fec08eb50 msgr2=0x7f9fec09e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 -- 172.21.15.46:0/3712089570 shutdown_connections 2024-04-26T23:45:27.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.493+0000 7f9ff1200640 1 -- 172.21.15.46:0/3712089570 wait complete. 2024-04-26T23:45:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.495+0000 7f9ff1200640 1 Processor -- start 2024-04-26T23:45:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.495+0000 7f9ff1200640 1 -- start start 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fec0d55e0 0x7f9fec159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec0caf60 0x7f9fec161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0b5e70 con 0x7f9fec15f4b0 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0b5b90 con 0x7f9fec0caf60 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9ff1200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0b4680 con 0x7f9fec0d55e0 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55148/0 (socket says 172.21.15.46:55148) 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 -- 172.21.15.46:0/3246629718 learned_addr learned my addr 172.21.15.46:0/3246629718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9e00640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fec0d55e0 0x7f9fec159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fea800640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec0caf60 0x7f9fec161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 -- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fec0d55e0 msgr2=0x7f9fec159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fec0d55e0 0x7f9fec159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 -- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec0caf60 msgr2=0x7f9fec161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec0caf60 0x7f9fec161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.496+0000 7f9fe9400640 1 -- 172.21.15.46:0/3246629718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0cc800 con 0x7f9fec15f4b0 2024-04-26T23:45:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9fe9400640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c3f0 tx=0x7f9fd800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80074e0 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd80047e0 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec15a760 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8004c50 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.497+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec160880 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.498+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec09e940 con 0x7f9fec15f4b0 2024-04-26T23:45:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.499+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fd8011070 con 0x7f9fec15f4b0 2024-04-26T23:45:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.500+0000 7f9fe2000640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 0x7f9fcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.500+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5507+0+0 (secure 0 0 0) 0x7f9fd808b3f0 con 0x7f9fec15f4b0 2024-04-26T23:45:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.501+0000 7f9fe9e00640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 0x7f9fcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.502+0000 7f9fe9e00640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 0x7f9fcc06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc034000 tx=0x7f9fdc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.504+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9fec09e940 con 0x7f9fec15f4b0 2024-04-26T23:45:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.726+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9fec069530 con 0x7f9fec15f4b0 2024-04-26T23:45:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.813+0000 7f9fe2000640 1 -- 172.21.15.46:0/3246629718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) ==== 97+0+0 (secure 0 0 0) 0x7f9fec0cc800 con 0x7f9fec15f4b0 2024-04-26T23:45:27.818 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:45:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.816+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 msgr2=0x7f9fcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.816+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 0x7f9fcc06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc034000 tx=0x7f9fdc02e000 comp rx=0 tx=0).stop 2024-04-26T23:45:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.816+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 msgr2=0x7f9fec0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.816+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c3f0 tx=0x7f9fd800c8b0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 shutdown_connections 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fcc06c8f0 0x7f9fcc06edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec0caf60 0x7f9fec161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fec15f4b0 0x7f9fec0caa20 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 --2- 172.21.15.46:0/3246629718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fec0d55e0 0x7f9fec159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 >> 172.21.15.46:0/3246629718 conn(0x7f9fec08eb50 msgr2=0x7f9fec0d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 shutdown_connections 2024-04-26T23:45:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:27.817+0000 7f9ff1200640 1 -- 172.21.15.46:0/3246629718 wait complete. 2024-04-26T23:45:27.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:45:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.041+0000 7fd34be00640 1 Processor -- start 2024-04-26T23:45:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.041+0000 7fd34be00640 1 -- start start 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0cff40 0x7fd34c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c06f1d0 con 0x7fd34c0cf450 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c06f470 con 0x7fd34c072190 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c072930 con 0x7fd34c0cff40 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0cff40 0x7fd34c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0cff40 0x7fd34c073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55154/0 (socket says 172.21.15.46:55154) 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 -- 172.21.15.46:0/3463367164 learned_addr learned my addr 172.21.15.46:0/3463367164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34aa00640 1 --1- 172.21.15.46:0/3463367164 >> v1:172.21.15.46:6789/0 conn(0x7fd34c0cf450 0x7fd34c0cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46874/0 (socket says 172.21.15.46:46874) 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 -- 172.21.15.46:0/3463367164 >> v1:172.21.15.142:6789/0 conn(0x7fd34c072190 legacy=0x7fd34c166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 -- 172.21.15.46:0/3463367164 >> v1:172.21.15.46:6789/0 conn(0x7fd34c0cf450 legacy=0x7fd34c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.043+0000 7fd34a000640 1 -- 172.21.15.46:0/3463367164 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c072c10 con 0x7fd34c0cff40 2024-04-26T23:45:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.044+0000 7fd34a000640 1 --2- 172.21.15.46:0/3463367164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0cff40 0x7fd34c073db0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0703e0 tx=0x7fd33802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93c7a533e1785f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.045+0000 7fd349600640 1 -- 172.21.15.46:0/3463367164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338037070 con 0x7fd34c0cff40 2024-04-26T23:45:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.045+0000 7fd349600640 1 -- 172.21.15.46:0/3463367164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd338032440 con 0x7fd34c0cff40 2024-04-26T23:45:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.045+0000 7fd349600640 1 -- 172.21.15.46:0/3463367164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338032810 con 0x7fd34c0cff40 2024-04-26T23:45:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 -- 172.21.15.46:0/3463367164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0cff40 msgr2=0x7fd34c073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 --2- 172.21.15.46:0/3463367164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0cff40 0x7fd34c073db0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0703e0 tx=0x7fd33802a200 comp rx=0 tx=0).stop 2024-04-26T23:45:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 -- 172.21.15.46:0/3463367164 shutdown_connections 2024-04-26T23:45:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 --2- 172.21.15.46:0/3463367164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0cff40 0x7fd34c073db0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 -- 172.21.15.46:0/3463367164 >> 172.21.15.46:0/3463367164 conn(0x7fd34c0b2820 msgr2=0x7fd34c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 -- 172.21.15.46:0/3463367164 shutdown_connections 2024-04-26T23:45:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.046+0000 7fd34be00640 1 -- 172.21.15.46:0/3463367164 wait complete. 2024-04-26T23:45:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.048+0000 7fd34be00640 1 Processor -- start 2024-04-26T23:45:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.048+0000 7fd34be00640 1 -- start start 2024-04-26T23:45:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd34c0cf450 0x7fd34c1de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd34c0cff40 0x7fd34c1e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0734b0 con 0x7fd34c072190 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0731d0 con 0x7fd34c0cf450 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0725a0 con 0x7fd34c0cff40 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55160/0 (socket says 172.21.15.46:55160) 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 -- 172.21.15.46:0/3269028340 learned_addr learned my addr 172.21.15.46:0/3269028340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 -- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd34c0cff40 msgr2=0x7fd34c1e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd34c0cff40 0x7fd34c1e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 -- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd34c0cf450 msgr2=0x7fd34c1de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd34c0cf450 0x7fd34c1de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.049+0000 7fd34aa00640 1 -- 172.21.15.46:0/3269028340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c145610 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd34aa00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c400 tx=0x7fd33c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0074e0 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c007680 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004a00 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1458f0 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.050+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c0b3bf0 con 0x7fd34c072190 2024-04-26T23:45:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.051+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c0693b0 con 0x7fd34c072190 2024-04-26T23:45:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.052+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd33c01a020 con 0x7fd34c072190 2024-04-26T23:45:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.053+0000 7fd343400640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 0x7fd32c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.053+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5249+0+0 (secure 0 0 0) 0x7fd33c08b320 con 0x7fd34c072190 2024-04-26T23:45:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.054+0000 7fd34a000640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 0x7fd32c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.055+0000 7fd34a000640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 0x7fd32c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd338034000 tx=0x7fd33802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.057+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd33c0597e0 con 0x7fd34c072190 2024-04-26T23:45:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.337+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd34c1e0fa0 con 0x7fd34c072190 2024-04-26T23:45:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.338+0000 7fd343400640 1 -- 172.21.15.46:0/3269028340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd33c0590e0 con 0x7fd34c072190 2024-04-26T23:45:28.342 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:28.342 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:45:28.342 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:45:28.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:45:28.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:45:28.351 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:45:28.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:45:28.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:45:28.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:45:28.355 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:45:28.358 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:45:28.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:28.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:45:28.358 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:45:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 msgr2=0x7fd32c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 0x7fd32c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd338034000 tx=0x7fd33802c000 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 msgr2=0x7fd34c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c400 tx=0x7fd33c00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 shutdown_connections 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd32c06c8f0 0x7fd32c06edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd34c0cff40 0x7fd34c1e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd34c0cf450 0x7fd34c1de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 --2- 172.21.15.46:0/3269028340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c072190 0x7fd34c0b43f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 >> 172.21.15.46:0/3269028340 conn(0x7fd34c0b2820 msgr2=0x7fd34c13f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 shutdown_connections 2024-04-26T23:45:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.340+0000 7fd34be00640 1 -- 172.21.15.46:0/3269028340 wait complete. 2024-04-26T23:45:28.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.568+0000 7f1b04a00640 1 Processor -- start 2024-04-26T23:45:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.569+0000 7f1b04a00640 1 -- start start 2024-04-26T23:45:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.570+0000 7f1b04a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 0x7f1b000d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.570+0000 7f1b04a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.570+0000 7f1b04a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b000d11f0 con 0x7f1b0015f4e0 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.570+0000 7f1b04a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b000d0ce0 con 0x7f1b000b4000 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.570+0000 7f1b04a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b000d2990 con 0x7f1b000d2130 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.571+0000 7f1affe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.571+0000 7f1af7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 0x7f1b000d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.571+0000 7f1affe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:53622/0 (socket says 172.21.15.46:53622) 2024-04-26T23:45:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.571+0000 7f1affe00640 1 -- 172.21.15.46:0/2755086881 learned_addr learned my addr 172.21.15.46:0/2755086881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.572+0000 7f1af7e00640 1 -- 172.21.15.46:0/2755086881 >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 msgr2=0x7f1b000b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.572+0000 7f1af7e00640 1 --2- 172.21.15.46:0/2755086881 >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.572+0000 7f1af7e00640 1 -- 172.21.15.46:0/2755086881 >> v1:172.21.15.142:6789/0 conn(0x7f1b0015f4e0 legacy=0x7f1b000d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.572+0000 7f1af7e00640 1 -- 172.21.15.46:0/2755086881 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b000d5f20 con 0x7f1b000d2130 2024-04-26T23:45:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.572+0000 7f1affe00640 1 --2- 172.21.15.46:0/2755086881 >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:45:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.573+0000 7f1af7e00640 1 --2- 172.21.15.46:0/2755086881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 0x7f1b000d2df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80097a0 tx=0x7f1ae802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd2fd7356b33462 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.574+0000 7f1afea00640 1 -- 172.21.15.46:0/2755086881 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8037070 con 0x7f1b000d2130 2024-04-26T23:45:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.574+0000 7f1afea00640 1 -- 172.21.15.46:0/2755086881 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae8032440 con 0x7f1b000d2130 2024-04-26T23:45:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.574+0000 7f1afea00640 1 -- 172.21.15.46:0/2755086881 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8032810 con 0x7f1b000d2130 2024-04-26T23:45:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 -- 172.21.15.46:0/2755086881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 msgr2=0x7f1b000d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 --2- 172.21.15.46:0/2755086881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 0x7f1b000d2df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80097a0 tx=0x7f1ae802a180 comp rx=0 tx=0).stop 2024-04-26T23:45:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 -- 172.21.15.46:0/2755086881 shutdown_connections 2024-04-26T23:45:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 --2- 172.21.15.46:0/2755086881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b000d2130 0x7f1b000d2df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 --2- 172.21.15.46:0/2755086881 >> v2:172.21.15.142:3300/0 conn(0x7f1b000b4000 0x7f1b000b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 -- 172.21.15.46:0/2755086881 >> 172.21.15.46:0/2755086881 conn(0x7f1b000d5010 msgr2=0x7f1b000d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 -- 172.21.15.46:0/2755086881 shutdown_connections 2024-04-26T23:45:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.575+0000 7f1b04a00640 1 -- 172.21.15.46:0/2755086881 wait complete. 2024-04-26T23:45:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.577+0000 7f1b04a00640 1 Processor -- start 2024-04-26T23:45:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.577+0000 7f1b04a00640 1 -- start start 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b000d2130 0x7f1b00162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b0015f4e0 0x7f1b000b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b000b1d60 con 0x7f1b000b4000 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b000d5c40 con 0x7f1b000d2130 2024-04-26T23:45:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1b04a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0015f8c0 con 0x7f1b0015f4e0 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1aff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1aff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55168/0 (socket says 172.21.15.46:55168) 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1aff400640 1 -- 172.21.15.46:0/1798004204 learned_addr learned my addr 172.21.15.46:0/1798004204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1af7e00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b000d2130 0x7f1b00162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.578+0000 7f1affe00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b0015f4e0 0x7f1b000b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 -- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b0015f4e0 msgr2=0x7f1b000b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b0015f4e0 0x7f1b000b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 -- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b000d2130 msgr2=0x7f1b00162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b000d2130 0x7f1b00162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 -- 172.21.15.46:0/1798004204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0015fc00 con 0x7f1b000b4000 2024-04-26T23:45:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1aff400640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c3c0 tx=0x7f1af000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00052d0 con 0x7f1b000b4000 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af0005490 con 0x7f1b000b4000 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0015fea0 con 0x7f1b000b4000 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.579+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0007260 con 0x7f1b000b4000 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.580+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b00162860 con 0x7f1b000b4000 2024-04-26T23:45:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.581+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b000693b0 con 0x7f1b000b4000 2024-04-26T23:45:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.582+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1af0007400 con 0x7f1b000b4000 2024-04-26T23:45:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.583+0000 7f1af7400640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 0x7f1adc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.583+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5249+0+0 (secure 0 0 0) 0x7f1af008aea0 con 0x7f1b000b4000 2024-04-26T23:45:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.583+0000 7f1af7e00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 0x7f1adc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.584+0000 7f1af7e00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 0x7f1adc06edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8034000 tx=0x7f1ae802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.587+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1af0059480 con 0x7f1b000b4000 2024-04-26T23:45:28.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.792+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b000d33d0 con 0x7f1b000b4000 2024-04-26T23:45:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.794+0000 7f1af7400640 1 -- 172.21.15.46:0/1798004204 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+13130 (secure 0 0 0) 0x7f1af0058d80 con 0x7f1b000b4000 2024-04-26T23:45:28.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:28.807 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:45:27.826181+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 msgr2=0x7f1adc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 0x7f1adc06edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8034000 tx=0x7f1ae802e000 comp rx=0 tx=0).stop 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 msgr2=0x7f1b00160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c3c0 tx=0x7f1af000c880 comp rx=0 tx=0).stop 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 shutdown_connections 2024-04-26T23:45:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1adc06c8f0 0x7f1adc06edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b0015f4e0 0x7f1b000b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b000d2130 0x7f1b00162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 --2- 172.21.15.46:0/1798004204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b000b4000 0x7f1b00160980 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 >> 172.21.15.46:0/1798004204 conn(0x7f1b000d5010 msgr2=0x7f1b0006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 shutdown_connections 2024-04-26T23:45:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:28.797+0000 7f1b04a00640 1 -- 172.21.15.46:0/1798004204 wait complete. 2024-04-26T23:45:29.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:45:29.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:45:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.250+0000 7f9705200640 1 Processor -- start 2024-04-26T23:45:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.250+0000 7f9705200640 1 -- start start 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.251+0000 7f9705200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9700160130 0x7f9700160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f9705200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97000cbe60 con 0x7f97000cdc60 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f9705200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97000cc140 con 0x7f970015cc40 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f9705200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f97000cca70 con 0x7f9700160130 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9700160130 0x7f9700160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ffe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97000cdc60 0x7f9700072220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46880/0 (socket says 172.21.15.46:46880) 2024-04-26T23:45:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ffe00640 1 -- 172.21.15.46:0/277124651 learned_addr learned my addr 172.21.15.46:0/277124651 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ff400640 1 -- 172.21.15.46:0/277124651 >> v1:172.21.15.142:6789/0 conn(0x7f970015cc40 legacy=0x7f97000cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ff400640 1 -- 172.21.15.46:0/277124651 >> v1:172.21.15.46:6789/0 conn(0x7f97000cdc60 legacy=0x7f9700072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96ff400640 1 -- 172.21.15.46:0/277124651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97000cc420 con 0x7f9700160130 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.252+0000 7f96fe000640 1 -- 172.21.15.46:0/277124651 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070356099 0 0) 0x7f97000cbe60 con 0x7f97000cdc60 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.254+0000 7f96ff400640 1 --2- 172.21.15.46:0/277124651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700160510 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f96f00097a0 tx=0x7f96f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf80b4ead7fd2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.254+0000 7f96fe000640 1 -- 172.21.15.46:0/277124651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0037070 con 0x7f9700160130 2024-04-26T23:45:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.254+0000 7f96fe000640 1 -- 172.21.15.46:0/277124651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f002ae50 con 0x7f9700160130 2024-04-26T23:45:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.254+0000 7f96fe000640 1 -- 172.21.15.46:0/277124651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0032660 con 0x7f9700160130 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 -- 172.21.15.46:0/277124651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 msgr2=0x7f9700160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 --2- 172.21.15.46:0/277124651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700160510 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f96f00097a0 tx=0x7f96f002a140 comp rx=0 tx=0).stop 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 -- 172.21.15.46:0/277124651 shutdown_connections 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 --2- 172.21.15.46:0/277124651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700160510 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 -- 172.21.15.46:0/277124651 >> 172.21.15.46:0/277124651 conn(0x7f97000b27b0 msgr2=0x7f97000b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 -- 172.21.15.46:0/277124651 shutdown_connections 2024-04-26T23:45:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.255+0000 7f9705200640 1 -- 172.21.15.46:0/277124651 wait complete. 2024-04-26T23:45:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.257+0000 7f9705200640 1 Processor -- start 2024-04-26T23:45:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.257+0000 7f9705200640 1 -- start start 2024-04-26T23:45:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97000cdc60 0x7f970014f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 0x7f97000c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f970015d220 con 0x7f9700160130 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f970015d0a0 con 0x7f970015cc40 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f9705200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9700072930 con 0x7f97000cdc60 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55184/0 (socket says 172.21.15.46:55184) 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96ffe00640 1 -- 172.21.15.46:0/1874670128 learned_addr learned my addr 172.21.15.46:0/1874670128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96ffe00640 1 -- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97000cdc60 msgr2=0x7f970014f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96ff400640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97000cdc60 0x7f970014f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.258+0000 7f96fea00640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 0x7f97000c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96ffe00640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97000cdc60 0x7f970014f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96ffe00640 1 -- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 msgr2=0x7f97000c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96ffe00640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 0x7f97000c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96ffe00640 1 -- 172.21.15.46:0/1874670128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001f4b30 con 0x7f9700160130 2024-04-26T23:45:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96fea00640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 0x7f97000c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96ffe00640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c780 tx=0x7f96f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8018070 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f800ce30 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f800a840 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001f4df0 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.259+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001f53b0 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.260+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97000693b0 con 0x7f9700160130 2024-04-26T23:45:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.262+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96f801b070 con 0x7f9700160130 2024-04-26T23:45:29.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.262+0000 7f96f7400640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 0x7f96dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.263+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5249+0+0 (secure 0 0 0) 0x7f96f808ae40 con 0x7f9700160130 2024-04-26T23:45:29.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.263+0000 7f96ff400640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 0x7f96dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.265+0000 7f96ff400640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 0x7f96dc06edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.268+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96f8059430 con 0x7f9700160130 2024-04-26T23:45:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.469+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97000736f0 con 0x7f9700160130 2024-04-26T23:45:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.470+0000 7f96f7400640 1 -- 172.21.15.46:0/1874670128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f96f8058d30 con 0x7f9700160130 2024-04-26T23:45:29.475 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 msgr2=0x7f96dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 0x7f96dc06edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).stop 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 msgr2=0x7f9700153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c780 tx=0x7f96f800cc40 comp rx=0 tx=0).stop 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 shutdown_connections 2024-04-26T23:45:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96dc06c8f0 0x7f96dc06edb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700160130 0x7f9700153f20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f970015cc40 0x7f97000c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 --2- 172.21.15.46:0/1874670128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97000cdc60 0x7f970014f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 >> 172.21.15.46:0/1874670128 conn(0x7f97000b27b0 msgr2=0x7f970015d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 shutdown_connections 2024-04-26T23:45:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.473+0000 7f9705200640 1 -- 172.21.15.46:0/1874670128 wait complete. 2024-04-26T23:45:29.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:45:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.690+0000 7fd6b7200640 1 Processor -- start 2024-04-26T23:45:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.690+0000 7fd6b7200640 1 -- start start 2024-04-26T23:45:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.692+0000 7fd6b7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.692+0000 7fd6b7200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6b80b4f30 0x7fd6b8072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.692+0000 7fd6b7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b806eff0 con 0x7fd6b8071ef0 2024-04-26T23:45:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.692+0000 7fd6b7200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b806f2d0 con 0x7fd6b80b4f30 2024-04-26T23:45:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.692+0000 7fd6b7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b806f5b0 con 0x7fd6b814cf10 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.693+0000 7fd6b5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.693+0000 7fd6b6800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6b80b4f30 0x7fd6b8072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.693+0000 7fd6b5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:37812/0 (socket says 172.21.15.46:37812) 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.693+0000 7fd6b5e00640 1 -- 172.21.15.46:0/3780096014 learned_addr learned my addr 172.21.15.46:0/3780096014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.693+0000 7fd6b4a00640 1 -- 172.21.15.46:0/3780096014 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038054669 0 0) 0x7fd6b806eff0 con 0x7fd6b8071ef0 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.694+0000 7fd6b5e00640 1 -- 172.21.15.46:0/3780096014 >> v2:172.21.15.142:3300/0 conn(0x7fd6b80b4f30 msgr2=0x7fd6b8072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.694+0000 7fd6b5e00640 1 --2- 172.21.15.46:0/3780096014 >> v2:172.21.15.142:3300/0 conn(0x7fd6b80b4f30 0x7fd6b8072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.694+0000 7fd6b5e00640 1 -- 172.21.15.46:0/3780096014 >> v1:172.21.15.142:6789/0 conn(0x7fd6b8071ef0 legacy=0x7fd6b8164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.694+0000 7fd6b5e00640 1 -- 172.21.15.46:0/3780096014 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b806f890 con 0x7fd6b814cf10 2024-04-26T23:45:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.695+0000 7fd6b5e00640 1 --2- 172.21.15.46:0/3780096014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009870 tx=0x7fd6a802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbad419728a8ee1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.695+0000 7fd6b4a00640 1 -- 172.21.15.46:0/3780096014 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8037070 con 0x7fd6b814cf10 2024-04-26T23:45:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.695+0000 7fd6b4a00640 1 -- 172.21.15.46:0/3780096014 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8032440 con 0x7fd6b814cf10 2024-04-26T23:45:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.695+0000 7fd6b4a00640 1 -- 172.21.15.46:0/3780096014 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032810 con 0x7fd6b814cf10 2024-04-26T23:45:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 -- 172.21.15.46:0/3780096014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 msgr2=0x7fd6b806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 --2- 172.21.15.46:0/3780096014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009870 tx=0x7fd6a802a250 comp rx=0 tx=0).stop 2024-04-26T23:45:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 -- 172.21.15.46:0/3780096014 shutdown_connections 2024-04-26T23:45:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 --2- 172.21.15.46:0/3780096014 >> v2:172.21.15.142:3300/0 conn(0x7fd6b80b4f30 0x7fd6b8072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 --2- 172.21.15.46:0/3780096014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b814cf10 0x7fd6b806e960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 -- 172.21.15.46:0/3780096014 >> 172.21.15.46:0/3780096014 conn(0x7fd6b80b2820 msgr2=0x7fd6b80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 -- 172.21.15.46:0/3780096014 shutdown_connections 2024-04-26T23:45:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.696+0000 7fd6b7200640 1 -- 172.21.15.46:0/3780096014 wait complete. 2024-04-26T23:45:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.698+0000 7fd6b7200640 1 Processor -- start 2024-04-26T23:45:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.698+0000 7fd6b7200640 1 -- start start 2024-04-26T23:45:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b8071ef0 0x7fd6b8162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 0x7fd6b80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81648a0 con 0x7fd6b814cf10 2024-04-26T23:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8163bd0 con 0x7fd6b80b4f30 2024-04-26T23:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8164a20 con 0x7fd6b8071ef0 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.699+0000 7fd6b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55186/0 (socket says 172.21.15.46:55186) 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 -- 172.21.15.46:0/17515569 learned_addr learned my addr 172.21.15.46:0/17515569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b5e00640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b8071ef0 0x7fd6b8162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b5400640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 0x7fd6b80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 -- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b8071ef0 msgr2=0x7fd6b8162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b8071ef0 0x7fd6b8162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 -- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 msgr2=0x7fd6b80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 0x7fd6b80c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 -- 172.21.15.46:0/17515569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b80ca190 con 0x7fd6b814cf10 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b5400640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 0x7fd6b80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.700+0000 7fd6b6800640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000efc0 tx=0x7fd6b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.701+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0018020 con 0x7fd6b814cf10 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.701+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0014480 con 0x7fd6b814cf10 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.701+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0014870 con 0x7fd6b814cf10 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.701+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b80aa720 con 0x7fd6b814cf10 2024-04-26T23:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.701+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b80caff0 con 0x7fd6b814cf10 2024-04-26T23:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.702+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b80693b0 con 0x7fd6b814cf10 2024-04-26T23:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.703+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6b0005d10 con 0x7fd6b814cf10 2024-04-26T23:45:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.705+0000 7fd6ae000640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.705+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5249+0+0 (secure 0 0 0) 0x7fd6b008c530 con 0x7fd6b814cf10 2024-04-26T23:45:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.705+0000 7fd6b5e00640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.706+0000 7fd6b5e00640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 0x7fd69806edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8034000 tx=0x7fd6a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.709+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6b0022070 con 0x7fd6b814cf10 2024-04-26T23:45:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.907+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6b80b2eb0 con 0x7fd6b814cf10 2024-04-26T23:45:29.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.908+0000 7fd6ae000640 1 -- 172.21.15.46:0/17515569 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) ==== 84+0+65 (secure 0 0 0) 0x7fd6b0010040 con 0x7fd6b814cf10 2024-04-26T23:45:29.912 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:45:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 msgr2=0x7fd69806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 0x7fd69806edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8034000 tx=0x7fd6a802e000 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 msgr2=0x7fd6b813eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000efc0 tx=0x7fd6b000c490 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 shutdown_connections 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6b814cf10 0x7fd6b813eb30 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80b4f30 0x7fd6b80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 --2- 172.21.15.46:0/17515569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6b8071ef0 0x7fd6b8162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 >> 172.21.15.46:0/17515569 conn(0x7fd6b80b2820 msgr2=0x7fd6b806e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 shutdown_connections 2024-04-26T23:45:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:29.910+0000 7fd6b7200640 1 -- 172.21.15.46:0/17515569 wait complete. 2024-04-26T23:45:29.932 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:45:29.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:45:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.134+0000 7f9fbe200640 1 Processor -- start 2024-04-26T23:45:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.135+0000 7f9fbe200640 1 -- start start 2024-04-26T23:45:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbe200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fb81627f0 0x7f9fb80cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbe200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 0x7f9fb80d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbe200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb80d01a0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbe200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80d0480 con 0x7f9fb81627f0 2024-04-26T23:45:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbe200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80d0760 con 0x7f9fb80d2080 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbd800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9fb80d3af0 0x7f9fb8162f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46896/0 (socket says 172.21.15.46:46896) 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbd800640 1 -- 172.21.15.46:0/184876210 learned_addr learned my addr 172.21.15.46:0/184876210 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.136+0000 7f9fbce00640 1 --2- 172.21.15.46:0/184876210 >> v2:172.21.15.46:3300/0 conn(0x7f9fb81627f0 0x7f9fb80cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fb7400640 1 -- 172.21.15.46:0/184876210 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907146807 0 0) 0x7f9fb80d01a0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fb7400640 1 -- 172.21.15.46:0/184876210 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fa0003520 con 0x7f9fb80d3af0 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/184876210 >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 0x7f9fb80d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fbce00640 1 -- 172.21.15.46:0/184876210 >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 msgr2=0x7f9fb80d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fbce00640 1 --2- 172.21.15.46:0/184876210 >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 0x7f9fb80d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fbce00640 1 -- 172.21.15.46:0/184876210 >> v1:172.21.15.46:6789/0 conn(0x7f9fb80d3af0 legacy=0x7f9fb8162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fbce00640 1 -- 172.21.15.46:0/184876210 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80d0a40 con 0x7f9fb81627f0 2024-04-26T23:45:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.137+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/184876210 >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 0x7f9fb80d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.138+0000 7f9fbce00640 1 --2- 172.21.15.46:0/184876210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb81627f0 0x7f9fb80cfbc0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9fb808f8a0 tx=0x7f9fa402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54c7a530ab69c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.138+0000 7f9fb7400640 1 -- 172.21.15.46:0/184876210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4037070 con 0x7f9fb81627f0 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.138+0000 7f9fb7400640 1 -- 172.21.15.46:0/184876210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4032440 con 0x7f9fb81627f0 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.138+0000 7f9fb7400640 1 -- 172.21.15.46:0/184876210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4032810 con 0x7f9fb81627f0 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 -- 172.21.15.46:0/184876210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb81627f0 msgr2=0x7f9fb80cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 --2- 172.21.15.46:0/184876210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb81627f0 0x7f9fb80cfbc0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9fb808f8a0 tx=0x7f9fa402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 -- 172.21.15.46:0/184876210 shutdown_connections 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 --2- 172.21.15.46:0/184876210 >> v2:172.21.15.142:3300/0 conn(0x7f9fb80d2080 0x7f9fb80d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 --2- 172.21.15.46:0/184876210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb81627f0 0x7f9fb80cfbc0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 -- 172.21.15.46:0/184876210 >> 172.21.15.46:0/184876210 conn(0x7f9fb809e380 msgr2=0x7f9fb80d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 -- 172.21.15.46:0/184876210 shutdown_connections 2024-04-26T23:45:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.139+0000 7f9fbe200640 1 -- 172.21.15.46:0/184876210 wait complete. 2024-04-26T23:45:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.141+0000 7f9fbe200640 1 Processor -- start 2024-04-26T23:45:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.141+0000 7f9fbe200640 1 -- start start 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb80d2080 0x7f9fb81f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 0x7f9fb80b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8162bd0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d24f0 con 0x7f9fb81627f0 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.142+0000 7f9fbe200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80b5560 con 0x7f9fb80d2080 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55206/0 (socket says 172.21.15.46:55206) 2024-04-26T23:45:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 -- 172.21.15.46:0/1529085635 learned_addr learned my addr 172.21.15.46:0/1529085635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fbce00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb80d2080 0x7f9fb81f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fbd800640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 0x7f9fb80b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 -- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb80d2080 msgr2=0x7f9fb81f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb80d2080 0x7f9fb81f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 -- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 msgr2=0x7f9fb80b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 0x7f9fb80b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 -- 172.21.15.46:0/1529085635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80c8dd0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fbd800640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 0x7f9fb80b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.143+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c7f0 tx=0x7f9fac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.144+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac015070 con 0x7f9fb80d3af0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.144+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fac007a30 con 0x7f9fb80d3af0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.144+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac007e00 con 0x7f9fb80d3af0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.144+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb80c90b0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.144+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb80c9670 con 0x7f9fb80d3af0 2024-04-26T23:45:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.145+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb80693b0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.146+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fac018070 con 0x7f9fb80d3af0 2024-04-26T23:45:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.147+0000 7f9fb4c00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.147+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5249+0+0 (secure 0 0 0) 0x7f9fac08b9b0 con 0x7f9fb80d3af0 2024-04-26T23:45:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.148+0000 7f9fbce00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:30.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.149+0000 7f9fbce00640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 0x7f9f9806edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4034000 tx=0x7f9fa402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.152+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9fac059f90 con 0x7f9fb80d3af0 2024-04-26T23:45:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.360+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9fb80d3390 con 0x7f9fb80d3af0 2024-04-26T23:45:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.832+0000 7f9fb4c00640 1 -- 172.21.15.46:0/1529085635 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) ==== 138+0+0 (secure 0 0 0) 0x7f9fac059890 con 0x7f9fb80d3af0 2024-04-26T23:45:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:45:30.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 msgr2=0x7f9f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 0x7f9f9806edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4034000 tx=0x7f9fa402e000 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 msgr2=0x7f9fb80ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c7f0 tx=0x7f9fac00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 shutdown_connections 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81627f0 0x7f9fb80b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb80d3af0 0x7f9fb80ca540 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 --2- 172.21.15.46:0/1529085635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb80d2080 0x7f9fb81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 >> 172.21.15.46:0/1529085635 conn(0x7f9fb809e380 msgr2=0x7f9fb80b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 shutdown_connections 2024-04-26T23:45:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:30.837+0000 7f9fbe200640 1 -- 172.21.15.46:0/1529085635 wait complete. 2024-04-26T23:45:30.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:45:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.088+0000 7f6ec5000640 1 Processor -- start 2024-04-26T23:45:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.088+0000 7f6ec5000640 1 -- start start 2024-04-26T23:45:31.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ec5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ec01626c0 0x7f6ec00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ec5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec0162aa0 0x7f6ec00d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ec5000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ec00d07d0 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ec5000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec00d0ab0 con 0x7f6ec01626c0 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ec5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec00d0d90 con 0x7f6ec0162aa0 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ebf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ec01626c0 0x7f6ec00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ebea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ec00b5fb0 0x7f6ec00d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46906/0 (socket says 172.21.15.46:46906) 2024-04-26T23:45:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.089+0000 7f6ebea00640 1 -- 172.21.15.46:0/1296499422 learned_addr learned my addr 172.21.15.46:0/1296499422 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebf400640 1 -- 172.21.15.46:0/1296499422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec0162aa0 msgr2=0x7f6ec00d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebfe00640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec0162aa0 0x7f6ec00d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebf400640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec0162aa0 0x7f6ec00d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebf400640 1 -- 172.21.15.46:0/1296499422 >> v1:172.21.15.46:6789/0 conn(0x7f6ec00b5fb0 legacy=0x7f6ec00d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebf400640 1 -- 172.21.15.46:0/1296499422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec00d2c80 con 0x7f6ec01626c0 2024-04-26T23:45:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.090+0000 7f6ebe000640 1 -- 172.21.15.46:0/1296499422 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505022317 0 0) 0x7f6ec00d07d0 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.091+0000 7f6ebf400640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec01626c0 0x7f6ec00b47c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009d90 tx=0x7f6eb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9725963fecc29515 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.091+0000 7f6ebe000640 1 -- 172.21.15.46:0/1296499422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0007740 con 0x7f6ec01626c0 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.091+0000 7f6ebe000640 1 -- 172.21.15.46:0/1296499422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb00078e0 con 0x7f6ec01626c0 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.091+0000 7f6ebe000640 1 -- 172.21.15.46:0/1296499422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0007cb0 con 0x7f6ec01626c0 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 -- 172.21.15.46:0/1296499422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec01626c0 msgr2=0x7f6ec00b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec01626c0 0x7f6ec00b47c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009d90 tx=0x7f6eb002a660 comp rx=0 tx=0).stop 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 -- 172.21.15.46:0/1296499422 shutdown_connections 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec0162aa0 0x7f6ec00d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1296499422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec01626c0 0x7f6ec00b47c0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 -- 172.21.15.46:0/1296499422 >> 172.21.15.46:0/1296499422 conn(0x7f6ec009e2a0 msgr2=0x7f6ec00d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 -- 172.21.15.46:0/1296499422 shutdown_connections 2024-04-26T23:45:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.092+0000 7f6ec5000640 1 -- 172.21.15.46:0/1296499422 wait complete. 2024-04-26T23:45:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.094+0000 7f6ec5000640 1 Processor -- start 2024-04-26T23:45:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.094+0000 7f6ec5000640 1 -- start start 2024-04-26T23:45:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec01626c0 0x7f6ec01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0162aa0 0x7f6ec0160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec00d2210 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec00d2090 con 0x7f6ec0162aa0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ec5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec00d31a0 con 0x7f6ec01626c0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ebf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ebf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55222/0 (socket says 172.21.15.46:55222) 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.095+0000 7f6ebf400640 1 -- 172.21.15.46:0/1472178332 learned_addr learned my addr 172.21.15.46:0/1472178332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebea00640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec01626c0 0x7f6ec01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebfe00640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0162aa0 0x7f6ec0160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 -- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec01626c0 msgr2=0x7f6ec01562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec01626c0 0x7f6ec01562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 -- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0162aa0 msgr2=0x7f6ec0160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0162aa0 0x7f6ec0160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 -- 172.21.15.46:0/1472178332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec00cd680 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.096+0000 7f6ebf400640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009d90 tx=0x7f6eb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.097+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0040070 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.097+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb0031c40 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.097+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0003990 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.097+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec00cd960 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.097+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec00cdf20 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.098+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec00693b0 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.099+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6eb0039040 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.100+0000 7f6eb7400640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 0x7f6e9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.100+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5619+0+0 (secure 0 0 0) 0x7f6eb00ad2f0 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.101+0000 7f6ebea00640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 0x7f6e9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.102+0000 7f6ebea00640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 0x7f6e9c06ede0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8001360 tx=0x7f6ea8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.106+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6eb007ba00 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.316+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6ec00d25b0 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.850+0000 7f6eb7400640 1 -- 172.21.15.46:0/1472178332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) ==== 170+0+0 (secure 0 0 0) 0x7f6eb007b300 con 0x7f6ec00b5fb0 2024-04-26T23:45:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:45:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.854+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 msgr2=0x7f6e9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.854+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 0x7f6e9c06ede0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8001360 tx=0x7f6ea8009040 comp rx=0 tx=0).stop 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.854+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 msgr2=0x7f6ec009ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.854+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009d90 tx=0x7f6eb0004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 shutdown_connections 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e9c06c920 0x7f6e9c06ede0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0162aa0 0x7f6ec0160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec01626c0 0x7f6ec01562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 --2- 172.21.15.46:0/1472178332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00b5fb0 0x7f6ec009ec10 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.855+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 >> 172.21.15.46:0/1472178332 conn(0x7f6ec009e2a0 msgr2=0x7f6ec00d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.856+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 shutdown_connections 2024-04-26T23:45:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:31.857+0000 7f6ec5000640 1 -- 172.21.15.46:0/1472178332 wait complete. 2024-04-26T23:45:31.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:45:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.076+0000 7f9ef8a00640 1 Processor -- start 2024-04-26T23:45:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.076+0000 7f9ef8a00640 1 -- start start 2024-04-26T23:45:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ef406f060 0x7f9ef40af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef4160c60 con 0x7f9ef415cf40 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef8a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef4160f00 con 0x7f9ef406ca60 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef8a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef41611e0 con 0x7f9ef406f060 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ef415cf40 0x7f9ef406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48806/0 (socket says 172.21.15.46:48806) 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef1e00640 1 -- 172.21.15.46:0/1621961687 learned_addr learned my addr 172.21.15.46:0/1621961687 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.078+0000 7f9ef2800640 1 --2- 172.21.15.46:0/1621961687 >> v2:172.21.15.46:3300/0 conn(0x7f9ef406f060 0x7f9ef40af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.079+0000 7f9ef0a00640 1 -- 172.21.15.46:0/1621961687 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808180026 0 0) 0x7f9ef4160c60 con 0x7f9ef415cf40 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.079+0000 7f9ef0a00640 1 -- 172.21.15.46:0/1621961687 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ed8003520 con 0x7f9ef415cf40 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.079+0000 7f9ef2800640 1 -- 172.21.15.46:0/1621961687 >> v1:172.21.15.142:6789/0 conn(0x7f9ef406ca60 legacy=0x7f9ef40aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.079+0000 7f9ef2800640 1 -- 172.21.15.46:0/1621961687 >> v1:172.21.15.46:6789/0 conn(0x7f9ef415cf40 legacy=0x7f9ef406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.079+0000 7f9ef2800640 1 -- 172.21.15.46:0/1621961687 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef41614c0 con 0x7f9ef406f060 2024-04-26T23:45:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.080+0000 7f9ef2800640 1 --2- 172.21.15.46:0/1621961687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef406f060 0x7f9ef40af530 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406cfc0 tx=0x7f9eec02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6c9fe16d3a6055 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.080+0000 7f9ef0a00640 1 -- 172.21.15.46:0/1621961687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec007740 con 0x7f9ef406f060 2024-04-26T23:45:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.080+0000 7f9ef0a00640 1 -- 172.21.15.46:0/1621961687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec0078e0 con 0x7f9ef406f060 2024-04-26T23:45:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.080+0000 7f9ef0a00640 1 -- 172.21.15.46:0/1621961687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec007cb0 con 0x7f9ef406f060 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 -- 172.21.15.46:0/1621961687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef406f060 msgr2=0x7f9ef40af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/1621961687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef406f060 0x7f9ef40af530 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406cfc0 tx=0x7f9eec02a560 comp rx=0 tx=0).stop 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 -- 172.21.15.46:0/1621961687 shutdown_connections 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/1621961687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef406f060 0x7f9ef40af530 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 -- 172.21.15.46:0/1621961687 >> 172.21.15.46:0/1621961687 conn(0x7f9ef406da90 msgr2=0x7f9ef40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 -- 172.21.15.46:0/1621961687 shutdown_connections 2024-04-26T23:45:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.081+0000 7f9ef8a00640 1 -- 172.21.15.46:0/1621961687 wait complete. 2024-04-26T23:45:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.083+0000 7f9ef8a00640 1 Processor -- start 2024-04-26T23:45:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.083+0000 7f9ef8a00640 1 -- start start 2024-04-26T23:45:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ef406ca60 0x7f9ef4153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ef406f060 0x7f9ef4154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4167ec0 con 0x7f9ef415cf40 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef415d320 con 0x7f9ef406ca60 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef4168040 con 0x7f9ef406f060 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48706/0 (socket says 172.21.15.46:48706) 2024-04-26T23:45:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.084+0000 7f9ef2800640 1 -- 172.21.15.46:0/3350080304 learned_addr learned my addr 172.21.15.46:0/3350080304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef1400640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ef406f060 0x7f9ef4154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 -- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ef406f060 msgr2=0x7f9ef4154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef1e00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ef406ca60 0x7f9ef4153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ef406f060 0x7f9ef4154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 -- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ef406ca60 msgr2=0x7f9ef4153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ef406ca60 0x7f9ef4153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 -- 172.21.15.46:0/3350080304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef4154b70 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef2800640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9eec02ab50 tx=0x7f9eec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec040070 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec031bc0 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef4154e50 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.085+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec009450 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.086+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef4155410 con 0x7f9ef415cf40 2024-04-26T23:45:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.087+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef4069530 con 0x7f9ef415cf40 2024-04-26T23:45:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.088+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9eec039040 con 0x7f9ef415cf40 2024-04-26T23:45:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.089+0000 7f9eea000640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 0x7f9ed406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.089+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6001+0+0 (secure 0 0 0) 0x7f9eec0ad580 con 0x7f9ef415cf40 2024-04-26T23:45:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.090+0000 7f9ef1e00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 0x7f9ed406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.091+0000 7f9ef1e00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 0x7f9ed406ede0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4160770 tx=0x7f9ee4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.094+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9eec07b7e0 con 0x7f9ef415cf40 2024-04-26T23:45:32.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.334+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9ef40d2960 con 0x7f9ef415cf40 2024-04-26T23:45:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.870+0000 7f9eea000640 1 -- 172.21.15.46:0/3350080304 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v87) ==== 174+0+0 (secure 0 0 0) 0x7f9eec07b0e0 con 0x7f9ef415cf40 2024-04-26T23:45:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 15 and data pool 16 2024-04-26T23:45:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.880+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 msgr2=0x7f9ed406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.880+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 0x7f9ed406ede0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4160770 tx=0x7f9ee4009040 comp rx=0 tx=0).stop 2024-04-26T23:45:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.880+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 msgr2=0x7f9ef41563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.880+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9eec02ab50 tx=0x7f9eec004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.880+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 shutdown_connections 2024-04-26T23:45:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.881+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9ed406c920 0x7f9ed406ede0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.881+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef415cf40 0x7f9ef41563c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.881+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ef406f060 0x7f9ef4154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.881+0000 7f9ef8a00640 1 --2- 172.21.15.46:0/3350080304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ef406ca60 0x7f9ef4153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.881+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 >> 172.21.15.46:0/3350080304 conn(0x7f9ef406da90 msgr2=0x7f9ef40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:32.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.883+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 shutdown_connections 2024-04-26T23:45:32.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:32.883+0000 7f9ef8a00640 1 -- 172.21.15.46:0/3350080304 wait complete. 2024-04-26T23:45:32.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.089+0000 7f2919200640 1 Processor -- start 2024-04-26T23:45:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.089+0000 7f2919200640 1 -- start start 2024-04-26T23:45:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2919200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 0x7f29140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2919200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29140b4000 0x7f29140d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2919200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29140d11f0 con 0x7f29140d1f60 2024-04-26T23:45:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2919200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f29140d2990 con 0x7f29140b4000 2024-04-26T23:45:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2919200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f29140d5f20 con 0x7f291415f4e0 2024-04-26T23:45:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2912a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29140b4000 0x7f29140d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2913e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29140d1f60 0x7f29140cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48812/0 (socket says 172.21.15.46:48812) 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.091+0000 7f2913e00640 1 -- 172.21.15.46:0/446848519 learned_addr learned my addr 172.21.15.46:0/446848519 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2913400640 1 --2- 172.21.15.46:0/446848519 >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 0x7f29140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2912a00640 1 -- 172.21.15.46:0/446848519 >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 msgr2=0x7f29140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2912a00640 1 --2- 172.21.15.46:0/446848519 >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 0x7f29140d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2912a00640 1 -- 172.21.15.46:0/446848519 >> v1:172.21.15.46:6789/0 conn(0x7f29140d1f60 legacy=0x7f29140cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2912a00640 1 -- 172.21.15.46:0/446848519 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29140d0f10 con 0x7f29140b4000 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2912000640 1 -- 172.21.15.46:0/446848519 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564063214 0 0) 0x7f29140d11f0 con 0x7f29140d1f60 2024-04-26T23:45:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.092+0000 7f2913400640 1 --2- 172.21.15.46:0/446848519 >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 0x7f29140d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.093+0000 7f2912a00640 1 --2- 172.21.15.46:0/446848519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29140b4000 0x7f29140d1a20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f291408d8b0 tx=0x7f28fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc68110836dc6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.093+0000 7f2912000640 1 -- 172.21.15.46:0/446848519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc037070 con 0x7f29140b4000 2024-04-26T23:45:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.093+0000 7f2912000640 1 -- 172.21.15.46:0/446848519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28fc032440 con 0x7f29140b4000 2024-04-26T23:45:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.093+0000 7f2912000640 1 -- 172.21.15.46:0/446848519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc032810 con 0x7f29140b4000 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 -- 172.21.15.46:0/446848519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29140b4000 msgr2=0x7f29140d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 --2- 172.21.15.46:0/446848519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29140b4000 0x7f29140d1a20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f291408d8b0 tx=0x7f28fc02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 -- 172.21.15.46:0/446848519 shutdown_connections 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 --2- 172.21.15.46:0/446848519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29140b4000 0x7f29140d1a20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 --2- 172.21.15.46:0/446848519 >> v2:172.21.15.142:3300/0 conn(0x7f291415f4e0 0x7f29140d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 -- 172.21.15.46:0/446848519 >> 172.21.15.46:0/446848519 conn(0x7f29140d5010 msgr2=0x7f29140d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 -- 172.21.15.46:0/446848519 shutdown_connections 2024-04-26T23:45:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.094+0000 7f2919200640 1 -- 172.21.15.46:0/446848519 wait complete. 2024-04-26T23:45:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.096+0000 7f2919200640 1 Processor -- start 2024-04-26T23:45:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.096+0000 7f2919200640 1 -- start start 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29140b4000 0x7f29140b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29140d1f60 0x7f29140b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f291415f8c0 con 0x7f291415f4e0 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29140b1d60 con 0x7f29140b4000 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2919200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29140b3290 con 0x7f29140d1f60 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.097+0000 7f2913e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48724/0 (socket says 172.21.15.46:48724) 2024-04-26T23:45:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 -- 172.21.15.46:0/2944029427 learned_addr learned my addr 172.21.15.46:0/2944029427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913400640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29140b4000 0x7f29140b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 -- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29140d1f60 msgr2=0x7f29140b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2912a00640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29140d1f60 0x7f29140b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29140d1f60 0x7f29140b93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 -- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29140b4000 msgr2=0x7f29140b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29140b4000 0x7f29140b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 -- 172.21.15.46:0/2944029427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291414cbe0 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.098+0000 7f2913e00640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c7a0 tx=0x7f290c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.099+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c018070 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.099+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c00a640 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.099+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291414cec0 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.099+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00aab0 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.099+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291414d480 con 0x7f291415f4e0 2024-04-26T23:45:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.100+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29140693b0 con 0x7f291415f4e0 2024-04-26T23:45:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.101+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f290c01b070 con 0x7f291415f4e0 2024-04-26T23:45:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.102+0000 7f290b400640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 0x7f28f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.102+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6101+0+0 (secure 0 0 0) 0x7f290c08b310 con 0x7f291415f4e0 2024-04-26T23:45:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.103+0000 7f2913400640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 0x7f28f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.104+0000 7f2913400640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 0x7f28f006edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f290400b840 tx=0x7f2904008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.107+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f290c0595b0 con 0x7f291415f4e0 2024-04-26T23:45:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.311+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29140d33d0 con 0x7f291415f4e0 2024-04-26T23:45:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.313+0000 7f290b400640 1 -- 172.21.15.46:0/2944029427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16861 (secure 0 0 0) 0x7f290c058eb0 con 0x7f291415f4e0 2024-04-26T23:45:33.318 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:45:32.336385+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.315+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 msgr2=0x7f28f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.315+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 0x7f28f006edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f290400b840 tx=0x7f2904008040 comp rx=0 tx=0).stop 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.315+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 msgr2=0x7f29141614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.315+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c7a0 tx=0x7f290c00cc60 comp rx=0 tx=0).stop 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 shutdown_connections 2024-04-26T23:45:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28f006c8f0 0x7f28f006edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291415f4e0 0x7f29141614a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29140d1f60 0x7f29140b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 --2- 172.21.15.46:0/2944029427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29140b4000 0x7f29140b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 >> 172.21.15.46:0/2944029427 conn(0x7f29140d5010 msgr2=0x7f291406e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 shutdown_connections 2024-04-26T23:45:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.316+0000 7f2919200640 1 -- 172.21.15.46:0/2944029427 wait complete. 2024-04-26T23:45:33.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:33.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.545+0000 7f88c5200640 1 Processor -- start 2024-04-26T23:45:33.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.545+0000 7f88c5200640 1 -- start start 2024-04-26T23:45:33.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88c5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88c006ec20 0x7f88c00734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88c5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c00739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88c5200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c00b43f0 con 0x7f88c0142280 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88c5200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f88c00b46d0 con 0x7f88c006ec20 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88c5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88c00b49b0 con 0x7f88c00b3e80 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88bd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88c006ec20 0x7f88c00734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88bde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f88c0142280 0x7f88c006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48814/0 (socket says 172.21.15.46:48814) 2024-04-26T23:45:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.547+0000 7f88bde00640 1 -- 172.21.15.46:0/1517996314 learned_addr learned my addr 172.21.15.46:0/1517996314 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:33.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.548+0000 7f88be800640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c00739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.548+0000 7f88bd400640 1 -- 172.21.15.46:0/1517996314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 msgr2=0x7f88c00739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:33.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.548+0000 7f88bd400640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c00739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.548+0000 7f88bd400640 1 -- 172.21.15.46:0/1517996314 >> v1:172.21.15.46:6789/0 conn(0x7f88c0142280 legacy=0x7f88c006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.548+0000 7f88bd400640 1 -- 172.21.15.46:0/1517996314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c00b4c90 con 0x7f88c006ec20 2024-04-26T23:45:33.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.549+0000 7f88bd400640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00734a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0097a0 tx=0x7f88ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac90757fcdf7abf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.549+0000 7f88bca00640 1 -- 172.21.15.46:0/1517996314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac037070 con 0x7f88c006ec20 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.549+0000 7f88bca00640 1 -- 172.21.15.46:0/1517996314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac032440 con 0x7f88c006ec20 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.549+0000 7f88bca00640 1 -- 172.21.15.46:0/1517996314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac032810 con 0x7f88c006ec20 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 -- 172.21.15.46:0/1517996314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 msgr2=0x7f88c00734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00734a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0097a0 tx=0x7f88ac02a180 comp rx=0 tx=0).stop 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 -- 172.21.15.46:0/1517996314 shutdown_connections 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c00739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 --2- 172.21.15.46:0/1517996314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00734a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 -- 172.21.15.46:0/1517996314 >> 172.21.15.46:0/1517996314 conn(0x7f88c006c530 msgr2=0x7f88c006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 -- 172.21.15.46:0/1517996314 shutdown_connections 2024-04-26T23:45:33.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.550+0000 7f88c5200640 1 -- 172.21.15.46:0/1517996314 wait complete. 2024-04-26T23:45:33.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.552+0000 7f88c5200640 1 Processor -- start 2024-04-26T23:45:33.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.552+0000 7f88c5200640 1 -- start start 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c013eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0142280 0x7f88c00af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0142a20 con 0x7f88c006ec20 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01428a0 con 0x7f88c0142280 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88c5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88c0073f20 con 0x7f88c00b3e80 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48732/0 (socket says 172.21.15.46:48732) 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 -- 172.21.15.46:0/756705205 learned_addr learned my addr 172.21.15.46:0/756705205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 -- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 msgr2=0x7f88c013eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c013eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 -- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0142280 msgr2=0x7f88c00af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0142280 0x7f88c00af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.553+0000 7f88bde00640 1 -- 172.21.15.46:0/756705205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c015e0c0 con 0x7f88c006ec20 2024-04-26T23:45:33.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88bde00640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c3c0 tx=0x7f88b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0017070 con 0x7f88c006ec20 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b0005d00 con 0x7f88c006ec20 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0007310 con 0x7f88c006ec20 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c015e3a0 con 0x7f88c006ec20 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.554+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c015ee70 con 0x7f88c006ec20 2024-04-26T23:45:33.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.555+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c0069530 con 0x7f88c006ec20 2024-04-26T23:45:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.557+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88b001a070 con 0x7f88c006ec20 2024-04-26T23:45:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.558+0000 7f88b6000640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.558+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6101+0+0 (secure 0 0 0) 0x7f88b008b060 con 0x7f88c006ec20 2024-04-26T23:45:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.558+0000 7f88bd400640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.559+0000 7f88bd400640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 0x7f88a006edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f88ac034000 tx=0x7f88ac02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:33.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.562+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88b0059300 con 0x7f88c006ec20 2024-04-26T23:45:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.780+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88c0144b80 con 0x7f88c006ec20 2024-04-26T23:45:33.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.782+0000 7f88b6000640 1 -- 172.21.15.46:0/756705205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16861 (secure 0 0 0) 0x7f88b0058c00 con 0x7f88c006ec20 2024-04-26T23:45:33.787 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:33.787 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:45:32.336385+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 msgr2=0x7f88a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 0x7f88a006edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f88ac034000 tx=0x7f88ac02c000 comp rx=0 tx=0).stop 2024-04-26T23:45:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 msgr2=0x7f88c00aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c3c0 tx=0x7f88b000c880 comp rx=0 tx=0).stop 2024-04-26T23:45:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 shutdown_connections 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0142280 0x7f88c00af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88c00b3e80 0x7f88c013eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 --2- 172.21.15.46:0/756705205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88c006ec20 0x7f88c00aeca0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.785+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 >> 172.21.15.46:0/756705205 conn(0x7f88c006c530 msgr2=0x7f88c01622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.786+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 shutdown_connections 2024-04-26T23:45:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:33.786+0000 7f88c5200640 1 -- 172.21.15.46:0/756705205 wait complete. 2024-04-26T23:45:33.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:45:33.827 INFO:tasks.ceph.mds.b.smithi142.stderr:2024-04-26T23:45:33.823+0000 7fac10a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.005+0000 7f60e9200640 1 Processor -- start 2024-04-26T23:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.006+0000 7f60e9200640 1 -- start start 2024-04-26T23:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.007+0000 7f60e9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e415d730 0x7f60e40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.007+0000 7f60e9200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60e40b53b0 0x7f60e415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.007+0000 7f60e9200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e406eb90 con 0x7f60e415e050 2024-04-26T23:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.007+0000 7f60e9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40b4310 con 0x7f60e415d730 2024-04-26T23:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.007+0000 7f60e9200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40b45f0 con 0x7f60e40b53b0 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.008+0000 7f60e1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e415d730 0x7f60e40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.008+0000 7f60e1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60e415d730 0x7f60e40d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48746/0 (socket says 172.21.15.46:48746) 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.008+0000 7f60e1e00640 1 -- 172.21.15.46:0/3449168822 learned_addr learned my addr 172.21.15.46:0/3449168822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.008+0000 7f60e1400640 1 --2- 172.21.15.46:0/3449168822 >> v2:172.21.15.142:3300/0 conn(0x7f60e40b53b0 0x7f60e415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.009+0000 7f60e1e00640 1 -- 172.21.15.46:0/3449168822 >> v2:172.21.15.142:3300/0 conn(0x7f60e40b53b0 msgr2=0x7f60e415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.009+0000 7f60e1e00640 1 --2- 172.21.15.46:0/3449168822 >> v2:172.21.15.142:3300/0 conn(0x7f60e40b53b0 0x7f60e415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.009+0000 7f60e1e00640 1 -- 172.21.15.46:0/3449168822 >> v1:172.21.15.142:6789/0 conn(0x7f60e415e050 legacy=0x7f60e406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.009+0000 7f60e1e00640 1 -- 172.21.15.46:0/3449168822 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e40b48d0 con 0x7f60e415d730 2024-04-26T23:45:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.010+0000 7f60e1e00640 1 --2- 172.21.15.46:0/3449168822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e415d730 0x7f60e40d00d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f60e406ffa0 tx=0x7f60d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca473d1aecdfdac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.010+0000 7f60e0a00640 1 -- 172.21.15.46:0/3449168822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60e415d730 2024-04-26T23:45:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.010+0000 7f60e0a00640 1 -- 172.21.15.46:0/3449168822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d4032440 con 0x7f60e415d730 2024-04-26T23:45:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.010+0000 7f60e0a00640 1 -- 172.21.15.46:0/3449168822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032810 con 0x7f60e415d730 2024-04-26T23:45:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 -- 172.21.15.46:0/3449168822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e415d730 msgr2=0x7f60e40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 --2- 172.21.15.46:0/3449168822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e415d730 0x7f60e40d00d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f60e406ffa0 tx=0x7f60d402a230 comp rx=0 tx=0).stop 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 -- 172.21.15.46:0/3449168822 shutdown_connections 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 --2- 172.21.15.46:0/3449168822 >> v2:172.21.15.142:3300/0 conn(0x7f60e40b53b0 0x7f60e415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 --2- 172.21.15.46:0/3449168822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e415d730 0x7f60e40d00d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 -- 172.21.15.46:0/3449168822 >> 172.21.15.46:0/3449168822 conn(0x7f60e406c980 msgr2=0x7f60e40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 -- 172.21.15.46:0/3449168822 shutdown_connections 2024-04-26T23:45:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.011+0000 7f60e9200640 1 -- 172.21.15.46:0/3449168822 wait complete. 2024-04-26T23:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.013+0000 7f60e9200640 1 Processor -- start 2024-04-26T23:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.013+0000 7f60e9200640 1 -- start start 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e415d730 0x7f60e40afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e415e050 0x7f60e41f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60e415e460 con 0x7f60e40b53b0 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40b4e90 con 0x7f60e415d730 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60e40b3ec0 con 0x7f60e415e050 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48750/0 (socket says 172.21.15.46:48750) 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 -- 172.21.15.46:0/1686971350 learned_addr learned my addr 172.21.15.46:0/1686971350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 -- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e415e050 msgr2=0x7f60e41f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e415e050 0x7f60e41f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 -- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e415d730 msgr2=0x7f60e40afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e415d730 0x7f60e40afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.015+0000 7f60e1e00640 1 -- 172.21.15.46:0/1686971350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e41f35f0 con 0x7f60e40b53b0 2024-04-26T23:45:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60e1e00640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a270 tx=0x7f60d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037040 con 0x7f60e40b53b0 2024-04-26T23:45:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d4042070 con 0x7f60e40b53b0 2024-04-26T23:45:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4030860 con 0x7f60e40b53b0 2024-04-26T23:45:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e41f38d0 con 0x7f60e40b53b0 2024-04-26T23:45:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.016+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41438f0 con 0x7f60e40b53b0 2024-04-26T23:45:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.017+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4069530 con 0x7f60e40b53b0 2024-04-26T23:45:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.018+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60d4030a00 con 0x7f60e40b53b0 2024-04-26T23:45:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.019+0000 7f60da000640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 0x7f60c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.020+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f60d40ac5d0 con 0x7f60e40b53b0 2024-04-26T23:45:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.020+0000 7f60e1400640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 0x7f60c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.022+0000 7f60e1400640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 0x7f60c406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006a90 tx=0x7f60d0006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.024+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60d407a860 con 0x7f60e40b53b0 2024-04-26T23:45:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.267+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f60e415b3e0 con 0x7f60e40b53b0 2024-04-26T23:45:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.887+0000 7f60da000640 1 -- 172.21.15.46:0/1686971350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) ==== 118+0+0 (secure 0 0 0) 0x7f60d407a160 con 0x7f60e40b53b0 2024-04-26T23:45:34.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.890+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 msgr2=0x7f60c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:34.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.890+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 0x7f60c406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006a90 tx=0x7f60d0006a20 comp rx=0 tx=0).stop 2024-04-26T23:45:34.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.890+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 msgr2=0x7f60e413f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.890+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a270 tx=0x7f60d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 shutdown_connections 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c406c8f0 0x7f60c406edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60e415e050 0x7f60e41f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60e415d730 0x7f60e40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 --2- 172.21.15.46:0/1686971350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60e40b53b0 0x7f60e413f2d0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 >> 172.21.15.46:0/1686971350 conn(0x7f60e406c980 msgr2=0x7f60e40b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 shutdown_connections 2024-04-26T23:45:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:34.891+0000 7f60e9200640 1 -- 172.21.15.46:0/1686971350 wait complete. 2024-04-26T23:45:34.921 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:45:34.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:45:35.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.139+0000 7ff98da00640 1 Processor -- start 2024-04-26T23:45:35.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.139+0000 7ff98da00640 1 -- start start 2024-04-26T23:45:35.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.140+0000 7ff98da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9880cdd00 0x7ff9880735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.140+0000 7ff98da00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff988073b30 0x7ff9880b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.140+0000 7ff98da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9880ce7d0 con 0x7ff98815cec0 2024-04-26T23:45:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.140+0000 7ff98da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff9880ceab0 con 0x7ff9880cdd00 2024-04-26T23:45:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.140+0000 7ff98da00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff9880ced90 con 0x7ff988073b30 2024-04-26T23:45:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff987e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff98815cec0 0x7ff9880cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48826/0 (socket says 172.21.15.46:48826) 2024-04-26T23:45:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff987e00640 1 -- 172.21.15.46:0/1317773975 learned_addr learned my addr 172.21.15.46:0/1317773975 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff987400640 1 --2- 172.21.15.46:0/1317773975 >> v2:172.21.15.46:3300/0 conn(0x7ff9880cdd00 0x7ff9880735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff98d000640 1 --2- 172.21.15.46:0/1317773975 >> v2:172.21.15.142:3300/0 conn(0x7ff988073b30 0x7ff9880b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff986a00640 1 -- 172.21.15.46:0/1317773975 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 255008046 0 0) 0x7ff9880ce7d0 con 0x7ff98815cec0 2024-04-26T23:45:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.141+0000 7ff986a00640 1 -- 172.21.15.46:0/1317773975 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff96c003520 con 0x7ff98815cec0 2024-04-26T23:45:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.142+0000 7ff987400640 1 -- 172.21.15.46:0/1317773975 >> v2:172.21.15.142:3300/0 conn(0x7ff988073b30 msgr2=0x7ff9880b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.142+0000 7ff987400640 1 --2- 172.21.15.46:0/1317773975 >> v2:172.21.15.142:3300/0 conn(0x7ff988073b30 0x7ff9880b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.142+0000 7ff987400640 1 -- 172.21.15.46:0/1317773975 >> v1:172.21.15.46:6789/0 conn(0x7ff98815cec0 legacy=0x7ff9880cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.142+0000 7ff987400640 1 -- 172.21.15.46:0/1317773975 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9880cf070 con 0x7ff9880cdd00 2024-04-26T23:45:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff987400640 1 --2- 172.21.15.46:0/1317773975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9880cdd00 0x7ff9880735f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff9880c7c50 tx=0x7ff97002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8d1afa93b7449 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff986a00640 1 -- 172.21.15.46:0/1317773975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970037070 con 0x7ff9880cdd00 2024-04-26T23:45:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff986a00640 1 -- 172.21.15.46:0/1317773975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff970032440 con 0x7ff9880cdd00 2024-04-26T23:45:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff986a00640 1 -- 172.21.15.46:0/1317773975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970032810 con 0x7ff9880cdd00 2024-04-26T23:45:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff98da00640 1 -- 172.21.15.46:0/1317773975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9880cdd00 msgr2=0x7ff9880735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.143+0000 7ff98da00640 1 --2- 172.21.15.46:0/1317773975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9880cdd00 0x7ff9880735f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff9880c7c50 tx=0x7ff97002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 -- 172.21.15.46:0/1317773975 shutdown_connections 2024-04-26T23:45:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 --2- 172.21.15.46:0/1317773975 >> v2:172.21.15.142:3300/0 conn(0x7ff988073b30 0x7ff9880b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 --2- 172.21.15.46:0/1317773975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9880cdd00 0x7ff9880735f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 -- 172.21.15.46:0/1317773975 >> 172.21.15.46:0/1317773975 conn(0x7ff9880b27b0 msgr2=0x7ff9880b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 -- 172.21.15.46:0/1317773975 shutdown_connections 2024-04-26T23:45:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.144+0000 7ff98da00640 1 -- 172.21.15.46:0/1317773975 wait complete. 2024-04-26T23:45:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.145+0000 7ff98da00640 1 Processor -- start 2024-04-26T23:45:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.146+0000 7ff98da00640 1 -- start start 2024-04-26T23:45:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988073b30 0x7ff98814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9880cdd00 0x7ff9880ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff988073f70 con 0x7ff98815cec0 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff98815d2a0 con 0x7ff988073b30 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff9880b4080 con 0x7ff9880cdd00 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48764/0 (socket says 172.21.15.46:48764) 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 -- 172.21.15.46:0/819544514 learned_addr learned my addr 172.21.15.46:0/819544514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff987e00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988073b30 0x7ff98814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff987400640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9880cdd00 0x7ff9880ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 -- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9880cdd00 msgr2=0x7ff9880ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9880cdd00 0x7ff9880ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 -- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988073b30 msgr2=0x7ff98814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988073b30 0x7ff98814f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.147+0000 7ff98d000640 1 -- 172.21.15.46:0/819544514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9880cbb00 con 0x7ff98815cec0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff98d000640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff980007d90 tx=0x7ff98000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980014420 con 0x7ff98815cec0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980019070 con 0x7ff98815cec0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9880cbde0 con 0x7ff98815cec0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980014820 con 0x7ff98815cec0 2024-04-26T23:45:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.148+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9880cc3a0 con 0x7ff98815cec0 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.149+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9880693b0 con 0x7ff98815cec0 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.150+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9800149c0 con 0x7ff98815cec0 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.151+0000 7ff97fe00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.151+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7ff98008c5b0 con 0x7ff98815cec0 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.152+0000 7ff987e00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:35.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.153+0000 7ff987e00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 0x7ff96406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff978005880 tx=0x7ff978009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.155+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff98005a7b0 con 0x7ff98815cec0 2024-04-26T23:45:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.402+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff98815c430 con 0x7ff98815cec0 2024-04-26T23:45:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.908+0000 7ff97fe00640 1 -- 172.21.15.46:0/819544514 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) ==== 122+0+0 (secure 0 0 0) 0x7ff98005a0b0 con 0x7ff98815cec0 2024-04-26T23:45:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 msgr2=0x7ff96406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 0x7ff96406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff978005880 tx=0x7ff978009040 comp rx=0 tx=0).stop 2024-04-26T23:45:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 msgr2=0x7ff9880ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff980007d90 tx=0x7ff98000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 shutdown_connections 2024-04-26T23:45:35.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98815cec0 0x7ff9880ae8f0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9880cdd00 0x7ff9880ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 --2- 172.21.15.46:0/819544514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988073b30 0x7ff98814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.912+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 >> 172.21.15.46:0/819544514 conn(0x7ff9880b27b0 msgr2=0x7ff9880c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.913+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 shutdown_connections 2024-04-26T23:45:35.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:35.913+0000 7ff98da00640 1 -- 172.21.15.46:0/819544514 wait complete. 2024-04-26T23:45:35.943 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:45:35.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.156+0000 7fdcb7200640 1 Processor -- start 2024-04-26T23:45:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.156+0000 7fdcb7200640 1 -- start start 2024-04-26T23:45:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.157+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdcb80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.157+0000 7fdcb7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb8073ad0 con 0x7fdcb8160910 2024-04-26T23:45:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.157+0000 7fdcb7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb8146a90 con 0x7fdcb81466b0 2024-04-26T23:45:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.157+0000 7fdcb7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb8073d70 con 0x7fdcb80720e0 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.158+0000 7fdcb6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdcb8160910 0x7fdcb80cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48838/0 (socket says 172.21.15.46:48838) 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.158+0000 7fdcb6800640 1 -- 172.21.15.46:0/2693172676 learned_addr learned my addr 172.21.15.46:0/2693172676 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.158+0000 7fdcafe00640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdcb80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.158+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358034877 0 0) 0x7fdcb8073ad0 con 0x7fdcb8160910 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.158+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc9c003520 con 0x7fdcb8160910 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3238642541 0 0) 0x7fdc9c003520 con 0x7fdcb8160910 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 msgr2=0x7fdcb80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:36.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdcb80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 >> v1:172.21.15.142:6789/0 conn(0x7fdcb81466b0 legacy=0x7fdcb806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb8074050 con 0x7fdcb8160910 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fdcb00019f0 con 0x7fdcb8160910 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 >> v1:172.21.15.46:6789/0 conn(0x7fdcb8160910 legacy=0x7fdcb80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdc9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc9c003fb0 0x7fdc9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc9c004d40 0x7fdc9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8073d70 con 0x7fdc9c004d40 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c00bb00 con 0x7fdc9c003fb0 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9c00bd00 con 0x7fdcb80720e0 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.159+0000 7fdcb6800640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc9c004d40 0x7fdc9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcafe00640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc9c003fb0 0x7fdc9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb5e00640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdc9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb6800640 1 -- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 msgr2=0x7fdc9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb6800640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80720e0 0x7fdc9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb6800640 1 -- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc9c003fb0 msgr2=0x7fdc9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb6800640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc9c003fb0 0x7fdc9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.160+0000 7fdcb6800640 1 -- 172.21.15.46:0/2693172676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c003520 con 0x7fdc9c004d40 2024-04-26T23:45:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.161+0000 7fdcb6800640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc9c004d40 0x7fdc9c0051c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000a7b0 tx=0x7fdc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83a879e94540911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.161+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000a3d0 con 0x7fdc9c004d40 2024-04-26T23:45:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.161+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb002f070 con 0x7fdc9c004d40 2024-04-26T23:45:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.161+0000 7fdcb5400640 1 -- 172.21.15.46:0/2693172676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb002bc80 con 0x7fdc9c004d40 2024-04-26T23:45:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 -- 172.21.15.46:0/2693172676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc9c004d40 msgr2=0x7fdc9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2693172676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc9c004d40 0x7fdc9c0051c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000a7b0 tx=0x7fdc9c005830 comp rx=0 tx=0).stop 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb6800640 1 -- 172.21.15.46:0/2693172676 reap_dead start 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 -- 172.21.15.46:0/2693172676 shutdown_connections 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 -- 172.21.15.46:0/2693172676 >> 172.21.15.46:0/2693172676 conn(0x7fdcb80b26d0 msgr2=0x7fdcb80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 -- 172.21.15.46:0/2693172676 shutdown_connections 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.162+0000 7fdcb7200640 1 -- 172.21.15.46:0/2693172676 wait complete. 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.164+0000 7fdcb7200640 1 Processor -- start 2024-04-26T23:45:36.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.164+0000 7fdcb7200640 1 -- start start 2024-04-26T23:45:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80b5340 0x7fdcb8144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb8144b80 0x7fdcb81f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb80cc0a0 con 0x7fdcb81415d0 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb80cb470 con 0x7fdcb8144b80 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb80cc220 con 0x7fdcb80b5340 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48774/0 (socket says 172.21.15.46:48774) 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.165+0000 7fdcb5e00640 1 -- 172.21.15.46:0/2189975068 learned_addr learned my addr 172.21.15.46:0/2189975068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcafe00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80b5340 0x7fdcb8144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 -- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80b5340 msgr2=0x7fdcb8144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb6800640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb8144b80 0x7fdcb81f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80b5340 0x7fdcb8144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 -- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb8144b80 msgr2=0x7fdcb81f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb8144b80 0x7fdcb81f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 -- 172.21.15.46:0/2189975068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb81419b0 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb5e00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e5b0 tx=0x7fdca800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80052e0 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca8018070 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003810 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.166+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb80b5840 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.167+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb80b5e00 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.168+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8144f90 con 0x7fdcb81415d0 2024-04-26T23:45:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.169+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdca80039b0 con 0x7fdcb81415d0 2024-04-26T23:45:36.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.170+0000 7fdcae000640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 0x7fdc9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.170+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7fdca808b7a0 con 0x7fdcb81415d0 2024-04-26T23:45:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.171+0000 7fdcafe00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 0x7fdc9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.172+0000 7fdcafe00640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 0x7fdc9406ede0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8070150 tx=0x7fdca0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.174+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcb8144f90 con 0x7fdcb81415d0 2024-04-26T23:45:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.423+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcb80c7180 con 0x7fdcb81415d0 2024-04-26T23:45:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.424+0000 7fdcae000640 1 -- 172.21.15.46:0/2189975068 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5183 (secure 0 0 0) 0x7fdca8059ad0 con 0x7fdcb81415d0 2024-04-26T23:45:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:36.430 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:35.404532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:45:36.430 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 90 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 msgr2=0x7fdc9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 0x7fdc9406ede0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8070150 tx=0x7fdca0008040 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 msgr2=0x7fdcb806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e5b0 tx=0x7fdca800ea70 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 shutdown_connections 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdc9406c920 0x7fdc9406ede0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb8144b80 0x7fdcb81f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb80b5340 0x7fdcb8144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.427+0000 7fdcb7200640 1 --2- 172.21.15.46:0/2189975068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb81415d0 0x7fdcb806e5c0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.428+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 >> 172.21.15.46:0/2189975068 conn(0x7fdcb80b26d0 msgr2=0x7fdcb80ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.428+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 shutdown_connections 2024-04-26T23:45:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.428+0000 7fdcb7200640 1 -- 172.21.15.46:0/2189975068 wait complete. 2024-04-26T23:45:36.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:45:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.656+0000 7fc37aa00640 1 Processor -- start 2024-04-26T23:45:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.656+0000 7fc37aa00640 1 -- start start 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc37aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc37406ec20 0x7fc3740734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc37aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3740b3e80 0x7fc3740739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc37aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3740b43f0 con 0x7fc374142280 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc37aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc3740b46d0 con 0x7fc3740b3e80 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc37aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc3740b49b0 con 0x7fc37406ec20 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc378a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3740b3e80 0x7fc3740739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc378a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3740b3e80 0x7fc3740739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48784/0 (socket says 172.21.15.46:48784) 2024-04-26T23:45:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc378a00640 1 -- 172.21.15.46:0/2575668746 learned_addr learned my addr 172.21.15.46:0/2575668746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.658+0000 7fc373400640 1 --2- 172.21.15.46:0/2575668746 >> v2:172.21.15.142:3300/0 conn(0x7fc37406ec20 0x7fc3740734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.659+0000 7fc378a00640 1 -- 172.21.15.46:0/2575668746 >> v2:172.21.15.142:3300/0 conn(0x7fc37406ec20 msgr2=0x7fc3740734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.659+0000 7fc378a00640 1 --2- 172.21.15.46:0/2575668746 >> v2:172.21.15.142:3300/0 conn(0x7fc37406ec20 0x7fc3740734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.659+0000 7fc378a00640 1 -- 172.21.15.46:0/2575668746 >> v1:172.21.15.142:6789/0 conn(0x7fc374142280 legacy=0x7fc37406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.659+0000 7fc378a00640 1 -- 172.21.15.46:0/2575668746 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740b4c90 con 0x7fc3740b3e80 2024-04-26T23:45:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.660+0000 7fc378a00640 1 --2- 172.21.15.46:0/2575668746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc3740739e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc37406f7c0 tx=0x7fc36c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab10c91089c04cee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.660+0000 7fc372a00640 1 -- 172.21.15.46:0/2575668746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c037070 con 0x7fc3740b3e80 2024-04-26T23:45:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.660+0000 7fc372a00640 1 -- 172.21.15.46:0/2575668746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c032440 con 0x7fc3740b3e80 2024-04-26T23:45:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.660+0000 7fc372a00640 1 -- 172.21.15.46:0/2575668746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c032810 con 0x7fc3740b3e80 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 -- 172.21.15.46:0/2575668746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 msgr2=0x7fc3740739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2575668746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc3740739e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc37406f7c0 tx=0x7fc36c02a230 comp rx=0 tx=0).stop 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 -- 172.21.15.46:0/2575668746 shutdown_connections 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2575668746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc3740739e0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2575668746 >> v2:172.21.15.142:3300/0 conn(0x7fc37406ec20 0x7fc3740734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 -- 172.21.15.46:0/2575668746 >> 172.21.15.46:0/2575668746 conn(0x7fc37406c530 msgr2=0x7fc37406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 -- 172.21.15.46:0/2575668746 shutdown_connections 2024-04-26T23:45:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.661+0000 7fc37aa00640 1 -- 172.21.15.46:0/2575668746 wait complete. 2024-04-26T23:45:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.663+0000 7fc37aa00640 1 Processor -- start 2024-04-26T23:45:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.663+0000 7fc37aa00640 1 -- start start 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 0x7fc3741613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 0x7fc37415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc374142a20 con 0x7fc3740b3e80 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741428a0 con 0x7fc374142280 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc37aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc374073f20 con 0x7fc37406ec20 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc373400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc373400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48796/0 (socket says 172.21.15.46:48796) 2024-04-26T23:45:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.664+0000 7fc373400640 1 -- 172.21.15.46:0/2481149501 learned_addr learned my addr 172.21.15.46:0/2481149501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:36.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373e00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 0x7fc3741613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc378a00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 0x7fc37415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 -- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 msgr2=0x7fc3741613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 0x7fc3741613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 -- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 msgr2=0x7fc37415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 0x7fc37415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 -- 172.21.15.46:0/2481149501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740c67f0 con 0x7fc3740b3e80 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc378a00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 0x7fc37415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373e00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 0x7fc3741613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc373400640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00d830 tx=0x7fc35c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.665+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00b6c0 con 0x7fc3740b3e80 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.666+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35c019070 con 0x7fc3740b3e80 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.666+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00ba80 con 0x7fc3740b3e80 2024-04-26T23:45:36.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.666+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3740c6ad0 con 0x7fc3740b3e80 2024-04-26T23:45:36.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.666+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3740ced10 con 0x7fc3740b3e80 2024-04-26T23:45:36.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.667+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc374069530 con 0x7fc3740b3e80 2024-04-26T23:45:36.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.668+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc35c01e460 con 0x7fc3740b3e80 2024-04-26T23:45:36.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.669+0000 7fc36be00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 0x7fc35006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:36.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.669+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7fc35c08c460 con 0x7fc3740b3e80 2024-04-26T23:45:36.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.670+0000 7fc373e00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 0x7fc35006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:36.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.671+0000 7fc373e00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 0x7fc35006edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc364007e90 tx=0x7fc3640073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:36.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.674+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc35c05a660 con 0x7fc3740b3e80 2024-04-26T23:45:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.881+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc374144a60 con 0x7fc3740b3e80 2024-04-26T23:45:36.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.884+0000 7fc36be00640 1 -- 172.21.15.46:0/2481149501 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16848 (secure 0 0 0) 0x7fc35c059f60 con 0x7fc3740b3e80 2024-04-26T23:45:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:36.889 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":111,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:45:33.870499+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 msgr2=0x7fc35006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 0x7fc35006edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc364007e90 tx=0x7fc3640073a0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 msgr2=0x7fc37413b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00d830 tx=0x7fc35c00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 shutdown_connections 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc35006c8f0 0x7fc35006edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc374142280 0x7fc37415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3740b3e80 0x7fc37413b760 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 --2- 172.21.15.46:0/2481149501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37406ec20 0x7fc3741613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.887+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 >> 172.21.15.46:0/2481149501 conn(0x7fc37406c530 msgr2=0x7fc37406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.888+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 shutdown_connections 2024-04-26T23:45:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:36.888+0000 7fc37aa00640 1 -- 172.21.15.46:0/2481149501 wait complete. 2024-04-26T23:45:36.912 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:45:36.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:45:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.123+0000 7f9058c00640 1 Processor -- start 2024-04-26T23:45:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.123+0000 7f9058c00640 1 -- start start 2024-04-26T23:45:37.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9058c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90541454a0 0x7f90540cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9058c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9054073970 0x7f90540cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9058c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9054072bb0 con 0x7f90540c6a50 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9058c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9054072e90 con 0x7f9054073970 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9058c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9054073170 con 0x7f90541454a0 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9052a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9054073970 0x7f90540cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9053e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f90540c6a50 0x7f90540cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48846/0 (socket says 172.21.15.46:48846) 2024-04-26T23:45:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9053e00640 1 -- 172.21.15.46:0/2380460843 learned_addr learned my addr 172.21.15.46:0/2380460843 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.125+0000 7f9053400640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90541454a0 0x7f90540cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.126+0000 7f9052a00640 1 -- 172.21.15.46:0/2380460843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90541454a0 msgr2=0x7f90540cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.126+0000 7f9052a00640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90541454a0 0x7f90540cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.126+0000 7f9052a00640 1 -- 172.21.15.46:0/2380460843 >> v1:172.21.15.46:6789/0 conn(0x7f90540c6a50 legacy=0x7f90540cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.126+0000 7f9052a00640 1 -- 172.21.15.46:0/2380460843 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9054073450 con 0x7f9054073970 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.126+0000 7f9052000640 1 -- 172.21.15.46:0/2380460843 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268150110 0 0) 0x7f9054072bb0 con 0x7f90540c6a50 2024-04-26T23:45:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.127+0000 7f9052a00640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054073970 0x7f90540cdc30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9040009d90 tx=0x7f904002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de0c512a8059c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.127+0000 7f9052000640 1 -- 172.21.15.46:0/2380460843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040007740 con 0x7f9054073970 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.127+0000 7f9052000640 1 -- 172.21.15.46:0/2380460843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400078e0 con 0x7f9054073970 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.127+0000 7f9052000640 1 -- 172.21.15.46:0/2380460843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040007cb0 con 0x7f9054073970 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 -- 172.21.15.46:0/2380460843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054073970 msgr2=0x7f90540cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054073970 0x7f90540cdc30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9040009d90 tx=0x7f904002a660 comp rx=0 tx=0).stop 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 -- 172.21.15.46:0/2380460843 shutdown_connections 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054073970 0x7f90540cdc30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 --2- 172.21.15.46:0/2380460843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90541454a0 0x7f90540cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 -- 172.21.15.46:0/2380460843 >> 172.21.15.46:0/2380460843 conn(0x7f90540b26d0 msgr2=0x7f90540b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 -- 172.21.15.46:0/2380460843 shutdown_connections 2024-04-26T23:45:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.128+0000 7f9058c00640 1 -- 172.21.15.46:0/2380460843 wait complete. 2024-04-26T23:45:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.130+0000 7f9058c00640 1 Processor -- start 2024-04-26T23:45:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.130+0000 7f9058c00640 1 -- start start 2024-04-26T23:45:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9054073970 0x7f90541f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90541454a0 0x7f905415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9054073dd0 con 0x7f90540c6a50 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9054072880 con 0x7f90541454a0 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9058c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90541f24a0 con 0x7f9054073970 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9052a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9052a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48812/0 (socket says 172.21.15.46:48812) 2024-04-26T23:45:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.131+0000 7f9052a00640 1 -- 172.21.15.46:0/780788174 learned_addr learned my addr 172.21.15.46:0/780788174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9053400640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9054073970 0x7f90541f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 -- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9054073970 msgr2=0x7f90541f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9054073970 0x7f90541f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 -- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90541454a0 msgr2=0x7f905415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90541454a0 0x7f905415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 -- 172.21.15.46:0/780788174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90540b02f0 con 0x7f90540c6a50 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9052a00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9040009d90 tx=0x7f9040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040040070 con 0x7f90540c6a50 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9040031c40 con 0x7f90540c6a50 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904003c420 con 0x7f90540c6a50 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.132+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90540af250 con 0x7f90540c6a50 2024-04-26T23:45:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.133+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90540af810 con 0x7f90540c6a50 2024-04-26T23:45:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.134+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90540693b0 con 0x7f90540c6a50 2024-04-26T23:45:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.135+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9040039040 con 0x7f90540c6a50 2024-04-26T23:45:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.136+0000 7f904b400640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.136+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f90400ad2d0 con 0x7f90540c6a50 2024-04-26T23:45:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.136+0000 7f9053400640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.138+0000 7f9053400640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 0x7f903406edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9044006fd0 tx=0x7f9044008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.141+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f904007b560 con 0x7f90540c6a50 2024-04-26T23:45:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.343+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90540cefa0 con 0x7f903406c8f0 2024-04-26T23:45:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.362+0000 7f904b400640 1 -- 172.21.15.46:0/780788174 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f90540cefa0 con 0x7f903406c8f0 2024-04-26T23:45:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:45:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 msgr2=0x7f903406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 0x7f903406edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9044006fd0 tx=0x7f9044008040 comp rx=0 tx=0).stop 2024-04-26T23:45:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 msgr2=0x7f90540ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9040009d90 tx=0x7f9040004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 shutdown_connections 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90541454a0 0x7f905415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90540c6a50 0x7f90540ae8e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 --2- 172.21.15.46:0/780788174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9054073970 0x7f90541f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 >> 172.21.15.46:0/780788174 conn(0x7f90540b26d0 msgr2=0x7f90540b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 shutdown_connections 2024-04-26T23:45:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.369+0000 7f9058c00640 1 -- 172.21.15.46:0/780788174 wait complete. 2024-04-26T23:45:37.393 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T23:45:36.074213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323840,"kb_used_data":11976,"kb_used_omap":12,"kb_used_meta":311731,"kb_avail":754650880,"statfs":{"total":773094113280,"available":772762501120,"internally_reserved":0,"allocated":12263424,"data_stored":6341612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":319213105},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.237669"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876317+0000","last_change":"2024-04-26T23:45:32.891792+0000","last_active":"2024-04-26T23:45:33.876317+0000","last_peered":"2024-04-26T23:45:33.876317+0000","last_clean":"2024-04-26T23:45:33.876317+0000","last_became_active":"2024-04-26T23:45:32.887973+0000","last_became_peered":"2024-04-26T23:45:32.887973+0000","last_unstale":"2024-04-26T23:45:33.876317+0000","last_undegraded":"2024-04-26T23:45:33.876317+0000","last_fullsized":"2024-04-26T23:45:33.876317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:38.717777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303755+0000","last_change":"2024-04-26T23:45:32.892543+0000","last_active":"2024-04-26T23:45:34.303755+0000","last_peered":"2024-04-26T23:45:34.303755+0000","last_clean":"2024-04-26T23:45:34.303755+0000","last_became_active":"2024-04-26T23:45:32.892251+0000","last_became_peered":"2024-04-26T23:45:32.892251+0000","last_unstale":"2024-04-26T23:45:34.303755+0000","last_undegraded":"2024-04-26T23:45:34.303755+0000","last_fullsized":"2024-04-26T23:45:34.303755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:32.182610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876622+0000","last_change":"2024-04-26T23:45:32.893090+0000","last_active":"2024-04-26T23:45:33.876622+0000","last_peered":"2024-04-26T23:45:33.876622+0000","last_clean":"2024-04-26T23:45:33.876622+0000","last_became_active":"2024-04-26T23:45:32.891297+0000","last_became_peered":"2024-04-26T23:45:32.891297+0000","last_unstale":"2024-04-26T23:45:33.876622+0000","last_undegraded":"2024-04-26T23:45:33.876622+0000","last_fullsized":"2024-04-26T23:45:33.876622+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:27.620869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222123+0000","last_change":"2024-04-26T23:45:32.892911+0000","last_active":"2024-04-26T23:45:34.222123+0000","last_peered":"2024-04-26T23:45:34.222123+0000","last_clean":"2024-04-26T23:45:34.222123+0000","last_became_active":"2024-04-26T23:45:32.891887+0000","last_became_peered":"2024-04-26T23:45:32.891887+0000","last_unstale":"2024-04-26T23:45:34.222123+0000","last_undegraded":"2024-04-26T23:45:34.222123+0000","last_fullsized":"2024-04-26T23:45:34.222123+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:13.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880459+0000","last_change":"2024-04-26T23:45:32.896396+0000","last_active":"2024-04-26T23:45:33.880459+0000","last_peered":"2024-04-26T23:45:33.880459+0000","last_clean":"2024-04-26T23:45:33.880459+0000","last_became_active":"2024-04-26T23:45:32.896068+0000","last_became_peered":"2024-04-26T23:45:32.896068+0000","last_unstale":"2024-04-26T23:45:33.880459+0000","last_undegraded":"2024-04-26T23:45:33.880459+0000","last_fullsized":"2024-04-26T23:45:33.880459+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303214+0000","last_change":"2024-04-26T23:45:32.894808+0000","last_active":"2024-04-26T23:45:34.303214+0000","last_peered":"2024-04-26T23:45:34.303214+0000","last_clean":"2024-04-26T23:45:34.303214+0000","last_became_active":"2024-04-26T23:45:32.894523+0000","last_became_peered":"2024-04-26T23:45:32.894523+0000","last_unstale":"2024-04-26T23:45:34.303214+0000","last_undegraded":"2024-04-26T23:45:34.303214+0000","last_fullsized":"2024-04-26T23:45:34.303214+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:45.130923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302472+0000","last_change":"2024-04-26T23:45:32.894223+0000","last_active":"2024-04-26T23:45:34.302472+0000","last_peered":"2024-04-26T23:45:34.302472+0000","last_clean":"2024-04-26T23:45:34.302472+0000","last_became_active":"2024-04-26T23:45:32.893948+0000","last_became_peered":"2024-04-26T23:45:32.893948+0000","last_unstale":"2024-04-26T23:45:34.302472+0000","last_undegraded":"2024-04-26T23:45:34.302472+0000","last_fullsized":"2024-04-26T23:45:34.302472+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:20.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300761+0000","last_change":"2024-04-26T23:45:32.899983+0000","last_active":"2024-04-26T23:45:34.300761+0000","last_peered":"2024-04-26T23:45:34.300761+0000","last_clean":"2024-04-26T23:45:34.300761+0000","last_became_active":"2024-04-26T23:45:32.899657+0000","last_became_peered":"2024-04-26T23:45:32.899657+0000","last_unstale":"2024-04-26T23:45:34.300761+0000","last_undegraded":"2024-04-26T23:45:34.300761+0000","last_fullsized":"2024-04-26T23:45:34.300761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:03.197511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304397+0000","last_change":"2024-04-26T23:45:32.892658+0000","last_active":"2024-04-26T23:45:34.304397+0000","last_peered":"2024-04-26T23:45:34.304397+0000","last_clean":"2024-04-26T23:45:34.304397+0000","last_became_active":"2024-04-26T23:45:32.892327+0000","last_became_peered":"2024-04-26T23:45:32.892327+0000","last_unstale":"2024-04-26T23:45:34.304397+0000","last_undegraded":"2024-04-26T23:45:34.304397+0000","last_fullsized":"2024-04-26T23:45:34.304397+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880887+0000","last_change":"2024-04-26T23:45:32.899150+0000","last_active":"2024-04-26T23:45:33.880887+0000","last_peered":"2024-04-26T23:45:33.880887+0000","last_clean":"2024-04-26T23:45:33.880887+0000","last_became_active":"2024-04-26T23:45:32.898602+0000","last_became_peered":"2024-04-26T23:45:32.898602+0000","last_unstale":"2024-04-26T23:45:33.880887+0000","last_undegraded":"2024-04-26T23:45:33.880887+0000","last_fullsized":"2024-04-26T23:45:33.880887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:43.530478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895255+0000","last_change":"2024-04-26T23:45:32.898242+0000","last_active":"2024-04-26T23:45:33.895255+0000","last_peered":"2024-04-26T23:45:33.895255+0000","last_clean":"2024-04-26T23:45:33.895255+0000","last_became_active":"2024-04-26T23:45:32.897681+0000","last_became_peered":"2024-04-26T23:45:32.897681+0000","last_unstale":"2024-04-26T23:45:33.895255+0000","last_undegraded":"2024-04-26T23:45:33.895255+0000","last_fullsized":"2024-04-26T23:45:33.895255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:28.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881386+0000","last_change":"2024-04-26T23:45:32.887503+0000","last_active":"2024-04-26T23:45:33.881386+0000","last_peered":"2024-04-26T23:45:33.881386+0000","last_clean":"2024-04-26T23:45:33.881386+0000","last_became_active":"2024-04-26T23:45:32.887250+0000","last_became_peered":"2024-04-26T23:45:32.887250+0000","last_unstale":"2024-04-26T23:45:33.881386+0000","last_undegraded":"2024-04-26T23:45:33.881386+0000","last_fullsized":"2024-04-26T23:45:33.881386+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:27.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221547+0000","last_change":"2024-04-26T23:45:32.887310+0000","last_active":"2024-04-26T23:45:34.221547+0000","last_peered":"2024-04-26T23:45:34.221547+0000","last_clean":"2024-04-26T23:45:34.221547+0000","last_became_active":"2024-04-26T23:45:32.886989+0000","last_became_peered":"2024-04-26T23:45:32.886989+0000","last_unstale":"2024-04-26T23:45:34.221547+0000","last_undegraded":"2024-04-26T23:45:34.221547+0000","last_fullsized":"2024-04-26T23:45:34.221547+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:28.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303408+0000","last_change":"2024-04-26T23:45:32.886030+0000","last_active":"2024-04-26T23:45:34.303408+0000","last_peered":"2024-04-26T23:45:34.303408+0000","last_clean":"2024-04-26T23:45:34.303408+0000","last_became_active":"2024-04-26T23:45:32.885693+0000","last_became_peered":"2024-04-26T23:45:32.885693+0000","last_unstale":"2024-04-26T23:45:34.303408+0000","last_undegraded":"2024-04-26T23:45:34.303408+0000","last_fullsized":"2024-04-26T23:45:34.303408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:08.801629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302317+0000","last_change":"2024-04-26T23:45:32.878481+0000","last_active":"2024-04-26T23:45:34.302317+0000","last_peered":"2024-04-26T23:45:34.302317+0000","last_clean":"2024-04-26T23:45:34.302317+0000","last_became_active":"2024-04-26T23:45:32.878177+0000","last_became_peered":"2024-04-26T23:45:32.878177+0000","last_unstale":"2024-04-26T23:45:34.302317+0000","last_undegraded":"2024-04-26T23:45:34.302317+0000","last_fullsized":"2024-04-26T23:45:34.302317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:45.140341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302228+0000","last_change":"2024-04-26T23:45:32.886120+0000","last_active":"2024-04-26T23:45:34.302228+0000","last_peered":"2024-04-26T23:45:34.302228+0000","last_clean":"2024-04-26T23:45:34.302228+0000","last_became_active":"2024-04-26T23:45:32.885809+0000","last_became_peered":"2024-04-26T23:45:32.885809+0000","last_unstale":"2024-04-26T23:45:34.302228+0000","last_undegraded":"2024-04-26T23:45:34.302228+0000","last_fullsized":"2024-04-26T23:45:34.302228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:08.526634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876824+0000","last_change":"2024-04-26T23:45:32.887630+0000","last_active":"2024-04-26T23:45:33.876824+0000","last_peered":"2024-04-26T23:45:33.876824+0000","last_clean":"2024-04-26T23:45:33.876824+0000","last_became_active":"2024-04-26T23:45:32.887097+0000","last_became_peered":"2024-04-26T23:45:32.887097+0000","last_unstale":"2024-04-26T23:45:33.876824+0000","last_undegraded":"2024-04-26T23:45:33.876824+0000","last_fullsized":"2024-04-26T23:45:33.876824+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:39.859921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303808+0000","last_change":"2024-04-26T23:45:32.897689+0000","last_active":"2024-04-26T23:45:34.303808+0000","last_peered":"2024-04-26T23:45:34.303808+0000","last_clean":"2024-04-26T23:45:34.303808+0000","last_became_active":"2024-04-26T23:45:32.896867+0000","last_became_peered":"2024-04-26T23:45:32.896867+0000","last_unstale":"2024-04-26T23:45:34.303808+0000","last_undegraded":"2024-04-26T23:45:34.303808+0000","last_fullsized":"2024-04-26T23:45:34.303808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.581149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876661+0000","last_change":"2024-04-26T23:45:32.900248+0000","last_active":"2024-04-26T23:45:33.876661+0000","last_peered":"2024-04-26T23:45:33.876661+0000","last_clean":"2024-04-26T23:45:33.876661+0000","last_became_active":"2024-04-26T23:45:32.899026+0000","last_became_peered":"2024-04-26T23:45:32.899026+0000","last_unstale":"2024-04-26T23:45:33.876661+0000","last_undegraded":"2024-04-26T23:45:33.876661+0000","last_fullsized":"2024-04-26T23:45:33.876661+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:46.600183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881808+0000","last_change":"2024-04-26T23:45:32.891886+0000","last_active":"2024-04-26T23:45:33.881808+0000","last_peered":"2024-04-26T23:45:33.881808+0000","last_clean":"2024-04-26T23:45:33.881808+0000","last_became_active":"2024-04-26T23:45:32.890365+0000","last_became_peered":"2024-04-26T23:45:32.890365+0000","last_unstale":"2024-04-26T23:45:33.881808+0000","last_undegraded":"2024-04-26T23:45:33.881808+0000","last_fullsized":"2024-04-26T23:45:33.881808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:43.530478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301912+0000","last_change":"2024-04-26T23:45:32.888704+0000","last_active":"2024-04-26T23:45:34.301912+0000","last_peered":"2024-04-26T23:45:34.301912+0000","last_clean":"2024-04-26T23:45:34.301912+0000","last_became_active":"2024-04-26T23:45:32.888385+0000","last_became_peered":"2024-04-26T23:45:32.888385+0000","last_unstale":"2024-04-26T23:45:34.301912+0000","last_undegraded":"2024-04-26T23:45:34.301912+0000","last_fullsized":"2024-04-26T23:45:34.301912+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:10.622727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880728+0000","last_change":"2024-04-26T23:45:32.899405+0000","last_active":"2024-04-26T23:45:33.880728+0000","last_peered":"2024-04-26T23:45:33.880728+0000","last_clean":"2024-04-26T23:45:33.880728+0000","last_became_active":"2024-04-26T23:45:32.898872+0000","last_became_peered":"2024-04-26T23:45:32.898872+0000","last_unstale":"2024-04-26T23:45:33.880728+0000","last_undegraded":"2024-04-26T23:45:33.880728+0000","last_fullsized":"2024-04-26T23:45:33.880728+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876167+0000","last_change":"2024-04-26T23:45:32.895597+0000","last_active":"2024-04-26T23:45:33.876167+0000","last_peered":"2024-04-26T23:45:33.876167+0000","last_clean":"2024-04-26T23:45:33.876167+0000","last_became_active":"2024-04-26T23:45:32.887956+0000","last_became_peered":"2024-04-26T23:45:32.887956+0000","last_unstale":"2024-04-26T23:45:33.876167+0000","last_undegraded":"2024-04-26T23:45:33.876167+0000","last_fullsized":"2024-04-26T23:45:33.876167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:11.919345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880223+0000","last_change":"2024-04-26T23:45:32.902007+0000","last_active":"2024-04-26T23:45:33.880223+0000","last_peered":"2024-04-26T23:45:33.880223+0000","last_clean":"2024-04-26T23:45:33.880223+0000","last_became_active":"2024-04-26T23:45:32.901708+0000","last_became_peered":"2024-04-26T23:45:32.901708+0000","last_unstale":"2024-04-26T23:45:33.880223+0000","last_undegraded":"2024-04-26T23:45:33.880223+0000","last_fullsized":"2024-04-26T23:45:33.880223+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.581149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881911+0000","last_change":"2024-04-26T23:45:32.897039+0000","last_active":"2024-04-26T23:45:33.881911+0000","last_peered":"2024-04-26T23:45:33.881911+0000","last_clean":"2024-04-26T23:45:33.881911+0000","last_became_active":"2024-04-26T23:45:32.896772+0000","last_became_peered":"2024-04-26T23:45:32.896772+0000","last_unstale":"2024-04-26T23:45:33.881911+0000","last_undegraded":"2024-04-26T23:45:33.881911+0000","last_fullsized":"2024-04-26T23:45:33.881911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:56.852278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.882410+0000","last_change":"2024-04-26T23:45:32.896863+0000","last_active":"2024-04-26T23:45:33.882410+0000","last_peered":"2024-04-26T23:45:33.882410+0000","last_clean":"2024-04-26T23:45:33.882410+0000","last_became_active":"2024-04-26T23:45:32.896605+0000","last_became_peered":"2024-04-26T23:45:32.896605+0000","last_unstale":"2024-04-26T23:45:33.882410+0000","last_undegraded":"2024-04-26T23:45:33.882410+0000","last_fullsized":"2024-04-26T23:45:33.882410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:20.369804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895102+0000","last_change":"2024-04-26T23:45:32.882004+0000","last_active":"2024-04-26T23:45:33.895102+0000","last_peered":"2024-04-26T23:45:33.895102+0000","last_clean":"2024-04-26T23:45:33.895102+0000","last_became_active":"2024-04-26T23:45:32.881696+0000","last_became_peered":"2024-04-26T23:45:32.881696+0000","last_unstale":"2024-04-26T23:45:33.895102+0000","last_undegraded":"2024-04-26T23:45:33.895102+0000","last_fullsized":"2024-04-26T23:45:33.895102+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:07.996624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303223+0000","last_change":"2024-04-26T23:45:32.894243+0000","last_active":"2024-04-26T23:45:34.303223+0000","last_peered":"2024-04-26T23:45:34.303223+0000","last_clean":"2024-04-26T23:45:34.303223+0000","last_became_active":"2024-04-26T23:45:32.892442+0000","last_became_peered":"2024-04-26T23:45:32.892442+0000","last_unstale":"2024-04-26T23:45:34.303223+0000","last_undegraded":"2024-04-26T23:45:34.303223+0000","last_fullsized":"2024-04-26T23:45:34.303223+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:20.942027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301903+0000","last_change":"2024-04-26T23:45:32.893890+0000","last_active":"2024-04-26T23:45:34.301903+0000","last_peered":"2024-04-26T23:45:34.301903+0000","last_clean":"2024-04-26T23:45:34.301903+0000","last_became_active":"2024-04-26T23:45:32.892216+0000","last_became_peered":"2024-04-26T23:45:32.892216+0000","last_unstale":"2024-04-26T23:45:34.301903+0000","last_undegraded":"2024-04-26T23:45:34.301903+0000","last_fullsized":"2024-04-26T23:45:34.301903+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:53.121661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880956+0000","last_change":"2024-04-26T23:45:32.899713+0000","last_active":"2024-04-26T23:45:33.880956+0000","last_peered":"2024-04-26T23:45:33.880956+0000","last_clean":"2024-04-26T23:45:33.880956+0000","last_became_active":"2024-04-26T23:45:32.899433+0000","last_became_peered":"2024-04-26T23:45:32.899433+0000","last_unstale":"2024-04-26T23:45:33.880956+0000","last_undegraded":"2024-04-26T23:45:33.880956+0000","last_fullsized":"2024-04-26T23:45:33.880956+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302752+0000","last_change":"2024-04-26T23:45:32.910663+0000","last_active":"2024-04-26T23:45:34.302752+0000","last_peered":"2024-04-26T23:45:34.302752+0000","last_clean":"2024-04-26T23:45:34.302752+0000","last_became_active":"2024-04-26T23:45:32.910310+0000","last_became_peered":"2024-04-26T23:45:32.910310+0000","last_unstale":"2024-04-26T23:45:34.302752+0000","last_undegraded":"2024-04-26T23:45:34.302752+0000","last_fullsized":"2024-04-26T23:45:34.302752+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:42.385444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301029+0000","last_change":"2024-04-26T23:45:32.895307+0000","last_active":"2024-04-26T23:45:34.301029+0000","last_peered":"2024-04-26T23:45:34.301029+0000","last_clean":"2024-04-26T23:45:34.301029+0000","last_became_active":"2024-04-26T23:45:32.895010+0000","last_became_peered":"2024-04-26T23:45:32.895010+0000","last_unstale":"2024-04-26T23:45:34.301029+0000","last_undegraded":"2024-04-26T23:45:34.301029+0000","last_fullsized":"2024-04-26T23:45:34.301029+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:31.870024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877412+0000","last_change":"2024-04-26T23:45:32.897165+0000","last_active":"2024-04-26T23:45:33.877412+0000","last_peered":"2024-04-26T23:45:33.877412+0000","last_clean":"2024-04-26T23:45:33.877412+0000","last_became_active":"2024-04-26T23:45:32.896446+0000","last_became_peered":"2024-04-26T23:45:32.896446+0000","last_unstale":"2024-04-26T23:45:33.877412+0000","last_undegraded":"2024-04-26T23:45:33.877412+0000","last_fullsized":"2024-04-26T23:45:33.877412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:30.757390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222139+0000","last_change":"2024-04-26T23:45:31.876231+0000","last_active":"2024-04-26T23:45:34.222139+0000","last_peered":"2024-04-26T23:45:34.222139+0000","last_clean":"2024-04-26T23:45:34.222139+0000","last_became_active":"2024-04-26T23:45:31.875441+0000","last_became_peered":"2024-04-26T23:45:31.875441+0000","last_unstale":"2024-04-26T23:45:34.222139+0000","last_undegraded":"2024-04-26T23:45:34.222139+0000","last_fullsized":"2024-04-26T23:45:34.222139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:19.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301694+0000","last_change":"2024-04-26T23:45:32.897608+0000","last_active":"2024-04-26T23:45:34.301694+0000","last_peered":"2024-04-26T23:45:34.301694+0000","last_clean":"2024-04-26T23:45:34.301694+0000","last_became_active":"2024-04-26T23:45:32.897291+0000","last_became_peered":"2024-04-26T23:45:32.897291+0000","last_unstale":"2024-04-26T23:45:34.301694+0000","last_undegraded":"2024-04-26T23:45:34.301694+0000","last_fullsized":"2024-04-26T23:45:34.301694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:00.717172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301389+0000","last_change":"2024-04-26T23:45:31.889214+0000","last_active":"2024-04-26T23:45:34.301389+0000","last_peered":"2024-04-26T23:45:34.301389+0000","last_clean":"2024-04-26T23:45:34.301389+0000","last_became_active":"2024-04-26T23:45:31.888934+0000","last_became_peered":"2024-04-26T23:45:31.888934+0000","last_unstale":"2024-04-26T23:45:34.301389+0000","last_undegraded":"2024-04-26T23:45:34.301389+0000","last_fullsized":"2024-04-26T23:45:34.301389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:27.660581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881352+0000","last_change":"2024-04-26T23:45:32.897239+0000","last_active":"2024-04-26T23:45:33.881352+0000","last_peered":"2024-04-26T23:45:33.881352+0000","last_clean":"2024-04-26T23:45:33.881352+0000","last_became_active":"2024-04-26T23:45:32.896145+0000","last_became_peered":"2024-04-26T23:45:32.896145+0000","last_unstale":"2024-04-26T23:45:33.881352+0000","last_undegraded":"2024-04-26T23:45:33.881352+0000","last_fullsized":"2024-04-26T23:45:33.881352+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881927+0000","last_change":"2024-04-26T23:45:31.888514+0000","last_active":"2024-04-26T23:45:33.881927+0000","last_peered":"2024-04-26T23:45:33.881927+0000","last_clean":"2024-04-26T23:45:33.881927+0000","last_became_active":"2024-04-26T23:45:31.887133+0000","last_became_peered":"2024-04-26T23:45:31.887133+0000","last_unstale":"2024-04-26T23:45:33.881927+0000","last_undegraded":"2024-04-26T23:45:33.881927+0000","last_fullsized":"2024-04-26T23:45:33.881927+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:05.288252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895306+0000","last_change":"2024-04-26T23:45:32.894946+0000","last_active":"2024-04-26T23:45:33.895306+0000","last_peered":"2024-04-26T23:45:33.895306+0000","last_clean":"2024-04-26T23:45:33.895306+0000","last_became_active":"2024-04-26T23:45:32.891643+0000","last_became_peered":"2024-04-26T23:45:32.891643+0000","last_unstale":"2024-04-26T23:45:33.895306+0000","last_undegraded":"2024-04-26T23:45:33.895306+0000","last_fullsized":"2024-04-26T23:45:33.895306+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:54.791803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880559+0000","last_change":"2024-04-26T23:45:31.886258+0000","last_active":"2024-04-26T23:45:33.880559+0000","last_peered":"2024-04-26T23:45:33.880559+0000","last_clean":"2024-04-26T23:45:33.880559+0000","last_became_active":"2024-04-26T23:45:31.884112+0000","last_became_peered":"2024-04-26T23:45:31.884112+0000","last_unstale":"2024-04-26T23:45:33.880559+0000","last_undegraded":"2024-04-26T23:45:33.880559+0000","last_fullsized":"2024-04-26T23:45:33.880559+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:05.288252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222110+0000","last_change":"2024-04-26T23:45:32.900605+0000","last_active":"2024-04-26T23:45:34.222110+0000","last_peered":"2024-04-26T23:45:34.222110+0000","last_clean":"2024-04-26T23:45:34.222110+0000","last_became_active":"2024-04-26T23:45:32.900332+0000","last_became_peered":"2024-04-26T23:45:32.900332+0000","last_unstale":"2024-04-26T23:45:34.222110+0000","last_undegraded":"2024-04-26T23:45:34.222110+0000","last_fullsized":"2024-04-26T23:45:34.222110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:10.670319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304385+0000","last_change":"2024-04-26T23:45:31.883629+0000","last_active":"2024-04-26T23:45:34.304385+0000","last_peered":"2024-04-26T23:45:34.304385+0000","last_clean":"2024-04-26T23:45:34.304385+0000","last_became_active":"2024-04-26T23:45:31.883331+0000","last_became_peered":"2024-04-26T23:45:31.883331+0000","last_unstale":"2024-04-26T23:45:34.304385+0000","last_undegraded":"2024-04-26T23:45:34.304385+0000","last_fullsized":"2024-04-26T23:45:34.304385+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877565+0000","last_change":"2024-04-26T23:45:32.909760+0000","last_active":"2024-04-26T23:45:33.877565+0000","last_peered":"2024-04-26T23:45:33.877565+0000","last_clean":"2024-04-26T23:45:33.877565+0000","last_became_active":"2024-04-26T23:45:32.909145+0000","last_became_peered":"2024-04-26T23:45:32.909145+0000","last_unstale":"2024-04-26T23:45:33.877565+0000","last_undegraded":"2024-04-26T23:45:33.877565+0000","last_fullsized":"2024-04-26T23:45:33.877565+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:03.444283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.894720+0000","last_change":"2024-04-26T23:45:31.888291+0000","last_active":"2024-04-26T23:45:33.894720+0000","last_peered":"2024-04-26T23:45:33.894720+0000","last_clean":"2024-04-26T23:45:33.894720+0000","last_became_active":"2024-04-26T23:45:31.887997+0000","last_became_peered":"2024-04-26T23:45:31.887997+0000","last_unstale":"2024-04-26T23:45:33.894720+0000","last_undegraded":"2024-04-26T23:45:33.894720+0000","last_fullsized":"2024-04-26T23:45:33.894720+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:34.548868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304298+0000","last_change":"2024-04-26T23:45:32.898917+0000","last_active":"2024-04-26T23:45:34.304298+0000","last_peered":"2024-04-26T23:45:34.304298+0000","last_clean":"2024-04-26T23:45:34.304298+0000","last_became_active":"2024-04-26T23:45:32.898479+0000","last_became_peered":"2024-04-26T23:45:32.898479+0000","last_unstale":"2024-04-26T23:45:34.304298+0000","last_undegraded":"2024-04-26T23:45:34.304298+0000","last_fullsized":"2024-04-26T23:45:34.304298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:06.682604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881043+0000","last_change":"2024-04-26T23:45:31.891534+0000","last_active":"2024-04-26T23:45:33.881043+0000","last_peered":"2024-04-26T23:45:33.881043+0000","last_clean":"2024-04-26T23:45:33.881043+0000","last_became_active":"2024-04-26T23:45:31.890799+0000","last_became_peered":"2024-04-26T23:45:31.890799+0000","last_unstale":"2024-04-26T23:45:33.881043+0000","last_undegraded":"2024-04-26T23:45:33.881043+0000","last_fullsized":"2024-04-26T23:45:33.881043+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:28.756536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222481+0000","last_change":"2024-04-26T23:45:32.902497+0000","last_active":"2024-04-26T23:45:34.222481+0000","last_peered":"2024-04-26T23:45:34.222481+0000","last_clean":"2024-04-26T23:45:34.222481+0000","last_became_active":"2024-04-26T23:45:32.902207+0000","last_became_peered":"2024-04-26T23:45:32.902207+0000","last_unstale":"2024-04-26T23:45:34.222481+0000","last_undegraded":"2024-04-26T23:45:34.222481+0000","last_fullsized":"2024-04-26T23:45:34.222481+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:08.526634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221476+0000","last_change":"2024-04-26T23:45:31.880250+0000","last_active":"2024-04-26T23:45:34.221476+0000","last_peered":"2024-04-26T23:45:34.221476+0000","last_clean":"2024-04-26T23:45:34.221476+0000","last_became_active":"2024-04-26T23:45:31.879929+0000","last_became_peered":"2024-04-26T23:45:31.879929+0000","last_unstale":"2024-04-26T23:45:34.221476+0000","last_undegraded":"2024-04-26T23:45:34.221476+0000","last_fullsized":"2024-04-26T23:45:34.221476+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:55.835940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877007+0000","last_change":"2024-04-26T23:45:32.897567+0000","last_active":"2024-04-26T23:45:33.877007+0000","last_peered":"2024-04-26T23:45:33.877007+0000","last_clean":"2024-04-26T23:45:33.877007+0000","last_became_active":"2024-04-26T23:45:32.896900+0000","last_became_peered":"2024-04-26T23:45:32.896900+0000","last_unstale":"2024-04-26T23:45:33.877007+0000","last_undegraded":"2024-04-26T23:45:33.877007+0000","last_fullsized":"2024-04-26T23:45:33.877007+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:24.608533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.882044+0000","last_change":"2024-04-26T23:45:31.891676+0000","last_active":"2024-04-26T23:45:33.882044+0000","last_peered":"2024-04-26T23:45:33.882044+0000","last_clean":"2024-04-26T23:45:33.882044+0000","last_became_active":"2024-04-26T23:45:31.890789+0000","last_became_peered":"2024-04-26T23:45:31.890789+0000","last_unstale":"2024-04-26T23:45:33.882044+0000","last_undegraded":"2024-04-26T23:45:33.882044+0000","last_fullsized":"2024-04-26T23:45:33.882044+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:16.612826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880336+0000","last_change":"2024-04-26T23:45:32.893518+0000","last_active":"2024-04-26T23:45:33.880336+0000","last_peered":"2024-04-26T23:45:33.880336+0000","last_clean":"2024-04-26T23:45:33.880336+0000","last_became_active":"2024-04-26T23:45:32.891197+0000","last_became_peered":"2024-04-26T23:45:32.891197+0000","last_unstale":"2024-04-26T23:45:33.880336+0000","last_undegraded":"2024-04-26T23:45:33.880336+0000","last_fullsized":"2024-04-26T23:45:33.880336+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:06.682604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876647+0000","last_change":"2024-04-26T23:45:31.881147+0000","last_active":"2024-04-26T23:45:33.876647+0000","last_peered":"2024-04-26T23:45:33.876647+0000","last_clean":"2024-04-26T23:45:33.876647+0000","last_became_active":"2024-04-26T23:45:31.880853+0000","last_became_peered":"2024-04-26T23:45:31.880853+0000","last_unstale":"2024-04-26T23:45:33.876647+0000","last_undegraded":"2024-04-26T23:45:33.876647+0000","last_fullsized":"2024-04-26T23:45:33.876647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:56.857293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301309+0000","last_change":"2024-04-26T23:45:32.902092+0000","last_active":"2024-04-26T23:45:34.301309+0000","last_peered":"2024-04-26T23:45:34.301309+0000","last_clean":"2024-04-26T23:45:34.301309+0000","last_became_active":"2024-04-26T23:45:32.901136+0000","last_became_peered":"2024-04-26T23:45:32.901136+0000","last_unstale":"2024-04-26T23:45:34.301309+0000","last_undegraded":"2024-04-26T23:45:34.301309+0000","last_fullsized":"2024-04-26T23:45:34.301309+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:59.284795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304525+0000","last_change":"2024-04-26T23:45:31.880787+0000","last_active":"2024-04-26T23:45:34.304525+0000","last_peered":"2024-04-26T23:45:34.304525+0000","last_clean":"2024-04-26T23:45:34.304525+0000","last_became_active":"2024-04-26T23:45:31.880484+0000","last_became_peered":"2024-04-26T23:45:31.880484+0000","last_unstale":"2024-04-26T23:45:34.304525+0000","last_undegraded":"2024-04-26T23:45:34.304525+0000","last_fullsized":"2024-04-26T23:45:34.304525+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:06.980286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880669+0000","last_change":"2024-04-26T23:45:32.902394+0000","last_active":"2024-04-26T23:45:33.880669+0000","last_peered":"2024-04-26T23:45:33.880669+0000","last_clean":"2024-04-26T23:45:33.880669+0000","last_became_active":"2024-04-26T23:45:32.891001+0000","last_became_peered":"2024-04-26T23:45:32.891001+0000","last_unstale":"2024-04-26T23:45:33.880669+0000","last_undegraded":"2024-04-26T23:45:33.880669+0000","last_fullsized":"2024-04-26T23:45:33.880669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303369+0000","last_change":"2024-04-26T23:45:31.878767+0000","last_active":"2024-04-26T23:45:34.303369+0000","last_peered":"2024-04-26T23:45:34.303369+0000","last_clean":"2024-04-26T23:45:34.303369+0000","last_became_active":"2024-04-26T23:45:31.877274+0000","last_became_peered":"2024-04-26T23:45:31.877274+0000","last_unstale":"2024-04-26T23:45:34.303369+0000","last_undegraded":"2024-04-26T23:45:34.303369+0000","last_fullsized":"2024-04-26T23:45:34.303369+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:31.580936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895387+0000","last_change":"2024-04-26T23:45:32.886485+0000","last_active":"2024-04-26T23:45:33.895387+0000","last_peered":"2024-04-26T23:45:33.895387+0000","last_clean":"2024-04-26T23:45:33.895387+0000","last_became_active":"2024-04-26T23:45:32.886095+0000","last_became_peered":"2024-04-26T23:45:32.886095+0000","last_unstale":"2024-04-26T23:45:33.895387+0000","last_undegraded":"2024-04-26T23:45:33.895387+0000","last_fullsized":"2024-04-26T23:45:33.895387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:22.732038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880729+0000","last_change":"2024-04-26T23:45:31.885731+0000","last_active":"2024-04-26T23:45:33.880729+0000","last_peered":"2024-04-26T23:45:33.880729+0000","last_clean":"2024-04-26T23:45:33.880729+0000","last_became_active":"2024-04-26T23:45:31.884231+0000","last_became_peered":"2024-04-26T23:45:31.884231+0000","last_unstale":"2024-04-26T23:45:33.880729+0000","last_undegraded":"2024-04-26T23:45:33.880729+0000","last_fullsized":"2024-04-26T23:45:33.880729+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:16.612826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304195+0000","last_change":"2024-04-26T23:45:32.891589+0000","last_active":"2024-04-26T23:45:34.304195+0000","last_peered":"2024-04-26T23:45:34.304195+0000","last_clean":"2024-04-26T23:45:34.304195+0000","last_became_active":"2024-04-26T23:45:32.891305+0000","last_became_peered":"2024-04-26T23:45:32.891305+0000","last_unstale":"2024-04-26T23:45:34.304195+0000","last_undegraded":"2024-04-26T23:45:34.304195+0000","last_fullsized":"2024-04-26T23:45:34.304195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221622+0000","last_change":"2024-04-26T23:45:31.873384+0000","last_active":"2024-04-26T23:45:34.221622+0000","last_peered":"2024-04-26T23:45:34.221622+0000","last_clean":"2024-04-26T23:45:34.221622+0000","last_became_active":"2024-04-26T23:45:31.873051+0000","last_became_peered":"2024-04-26T23:45:31.873051+0000","last_unstale":"2024-04-26T23:45:34.221622+0000","last_undegraded":"2024-04-26T23:45:34.221622+0000","last_fullsized":"2024-04-26T23:45:34.221622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:42.514140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301723+0000","last_change":"2024-04-26T23:45:32.891702+0000","last_active":"2024-04-26T23:45:34.301723+0000","last_peered":"2024-04-26T23:45:34.301723+0000","last_clean":"2024-04-26T23:45:34.301723+0000","last_became_active":"2024-04-26T23:45:32.891381+0000","last_became_peered":"2024-04-26T23:45:32.891381+0000","last_unstale":"2024-04-26T23:45:34.301723+0000","last_undegraded":"2024-04-26T23:45:34.301723+0000","last_fullsized":"2024-04-26T23:45:34.301723+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:37.701650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301002+0000","last_change":"2024-04-26T23:45:31.886925+0000","last_active":"2024-04-26T23:45:34.301002+0000","last_peered":"2024-04-26T23:45:34.301002+0000","last_clean":"2024-04-26T23:45:34.301002+0000","last_became_active":"2024-04-26T23:45:31.886582+0000","last_became_peered":"2024-04-26T23:45:31.886582+0000","last_unstale":"2024-04-26T23:45:34.301002+0000","last_undegraded":"2024-04-26T23:45:34.301002+0000","last_fullsized":"2024-04-26T23:45:34.301002+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:46.533241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876689+0000","last_change":"2024-04-26T23:45:32.897716+0000","last_active":"2024-04-26T23:45:33.876689+0000","last_peered":"2024-04-26T23:45:33.876689+0000","last_clean":"2024-04-26T23:45:33.876689+0000","last_became_active":"2024-04-26T23:45:32.888321+0000","last_became_peered":"2024-04-26T23:45:32.888321+0000","last_unstale":"2024-04-26T23:45:33.876689+0000","last_undegraded":"2024-04-26T23:45:33.876689+0000","last_fullsized":"2024-04-26T23:45:33.876689+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:10.438148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":22,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880501+0000","last_change":"2024-04-26T23:45:31.882816+0000","last_active":"2024-04-26T23:45:33.880501+0000","last_peered":"2024-04-26T23:45:33.880501+0000","last_clean":"2024-04-26T23:45:33.880501+0000","last_became_active":"2024-04-26T23:45:31.882476+0000","last_became_peered":"2024-04-26T23:45:31.882476+0000","last_unstale":"2024-04-26T23:45:33.880501+0000","last_undegraded":"2024-04-26T23:45:33.880501+0000","last_fullsized":"2024-04-26T23:45:33.880501+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:49.776017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301197+0000","last_change":"2024-04-26T23:45:32.912788+0000","last_active":"2024-04-26T23:45:34.301197+0000","last_peered":"2024-04-26T23:45:34.301197+0000","last_clean":"2024-04-26T23:45:34.301197+0000","last_became_active":"2024-04-26T23:45:32.910911+0000","last_became_peered":"2024-04-26T23:45:32.910911+0000","last_unstale":"2024-04-26T23:45:34.301197+0000","last_undegraded":"2024-04-26T23:45:34.301197+0000","last_fullsized":"2024-04-26T23:45:34.301197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:21.631715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304006+0000","last_change":"2024-04-26T23:45:31.879138+0000","last_active":"2024-04-26T23:45:34.304006+0000","last_peered":"2024-04-26T23:45:34.304006+0000","last_clean":"2024-04-26T23:45:34.304006+0000","last_became_active":"2024-04-26T23:45:31.878751+0000","last_became_peered":"2024-04-26T23:45:31.878751+0000","last_unstale":"2024-04-26T23:45:34.304006+0000","last_undegraded":"2024-04-26T23:45:34.304006+0000","last_fullsized":"2024-04-26T23:45:34.304006+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:48.251318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304437+0000","last_change":"2024-04-26T23:45:32.910970+0000","last_active":"2024-04-26T23:45:34.304437+0000","last_peered":"2024-04-26T23:45:34.304437+0000","last_clean":"2024-04-26T23:45:34.304437+0000","last_became_active":"2024-04-26T23:45:32.910721+0000","last_became_peered":"2024-04-26T23:45:32.910721+0000","last_unstale":"2024-04-26T23:45:34.304437+0000","last_undegraded":"2024-04-26T23:45:34.304437+0000","last_fullsized":"2024-04-26T23:45:34.304437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304792+0000","last_change":"2024-04-26T23:45:31.889679+0000","last_active":"2024-04-26T23:45:34.304792+0000","last_peered":"2024-04-26T23:45:34.304792+0000","last_clean":"2024-04-26T23:45:34.304792+0000","last_became_active":"2024-04-26T23:45:31.889351+0000","last_became_peered":"2024-04-26T23:45:31.889351+0000","last_unstale":"2024-04-26T23:45:34.304792+0000","last_undegraded":"2024-04-26T23:45:34.304792+0000","last_fullsized":"2024-04-26T23:45:34.304792+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:01.214289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880805+0000","last_change":"2024-04-26T23:45:32.893279+0000","last_active":"2024-04-26T23:45:33.880805+0000","last_peered":"2024-04-26T23:45:33.880805+0000","last_clean":"2024-04-26T23:45:33.880805+0000","last_became_active":"2024-04-26T23:45:32.891083+0000","last_became_peered":"2024-04-26T23:45:32.891083+0000","last_unstale":"2024-04-26T23:45:33.880805+0000","last_undegraded":"2024-04-26T23:45:33.880805+0000","last_fullsized":"2024-04-26T23:45:33.880805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:27.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895177+0000","last_change":"2024-04-26T23:45:31.888365+0000","last_active":"2024-04-26T23:45:33.895177+0000","last_peered":"2024-04-26T23:45:33.895177+0000","last_clean":"2024-04-26T23:45:33.895177+0000","last_became_active":"2024-04-26T23:45:31.887719+0000","last_became_peered":"2024-04-26T23:45:31.887719+0000","last_unstale":"2024-04-26T23:45:33.895177+0000","last_undegraded":"2024-04-26T23:45:33.895177+0000","last_fullsized":"2024-04-26T23:45:33.895177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:26.656816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877300+0000","last_change":"2024-04-26T23:45:32.888898+0000","last_active":"2024-04-26T23:45:33.877300+0000","last_peered":"2024-04-26T23:45:33.877300+0000","last_clean":"2024-04-26T23:45:33.877300+0000","last_became_active":"2024-04-26T23:45:32.888556+0000","last_became_peered":"2024-04-26T23:45:32.888556+0000","last_unstale":"2024-04-26T23:45:33.877300+0000","last_undegraded":"2024-04-26T23:45:33.877300+0000","last_fullsized":"2024-04-26T23:45:33.877300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:09.557437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301946+0000","last_change":"2024-04-26T23:45:31.890213+0000","last_active":"2024-04-26T23:45:34.301946+0000","last_peered":"2024-04-26T23:45:34.301946+0000","last_clean":"2024-04-26T23:45:34.301946+0000","last_became_active":"2024-04-26T23:45:31.889263+0000","last_became_peered":"2024-04-26T23:45:31.889263+0000","last_unstale":"2024-04-26T23:45:34.301946+0000","last_undegraded":"2024-04-26T23:45:34.301946+0000","last_fullsized":"2024-04-26T23:45:34.301946+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:32.653816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877577+0000","last_change":"2024-04-26T23:45:32.909517+0000","last_active":"2024-04-26T23:45:33.877577+0000","last_peered":"2024-04-26T23:45:33.877577+0000","last_clean":"2024-04-26T23:45:33.877577+0000","last_became_active":"2024-04-26T23:45:32.903973+0000","last_became_peered":"2024-04-26T23:45:32.903973+0000","last_unstale":"2024-04-26T23:45:33.877577+0000","last_undegraded":"2024-04-26T23:45:33.877577+0000","last_fullsized":"2024-04-26T23:45:33.877577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:51.472436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300814+0000","last_change":"2024-04-26T23:45:31.886802+0000","last_active":"2024-04-26T23:45:34.300814+0000","last_peered":"2024-04-26T23:45:34.300814+0000","last_clean":"2024-04-26T23:45:34.300814+0000","last_became_active":"2024-04-26T23:45:31.885469+0000","last_became_peered":"2024-04-26T23:45:31.885469+0000","last_unstale":"2024-04-26T23:45:34.300814+0000","last_undegraded":"2024-04-26T23:45:34.300814+0000","last_fullsized":"2024-04-26T23:45:34.300814+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:08.458279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222003+0000","last_change":"2024-04-26T23:45:32.897460+0000","last_active":"2024-04-26T23:45:34.222003+0000","last_peered":"2024-04-26T23:45:34.222003+0000","last_clean":"2024-04-26T23:45:34.222003+0000","last_became_active":"2024-04-26T23:45:32.896744+0000","last_became_peered":"2024-04-26T23:45:32.896744+0000","last_unstale":"2024-04-26T23:45:34.222003+0000","last_undegraded":"2024-04-26T23:45:34.222003+0000","last_fullsized":"2024-04-26T23:45:34.222003+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:22.732038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303370+0000","last_change":"2024-04-26T23:45:31.878549+0000","last_active":"2024-04-26T23:45:34.303370+0000","last_peered":"2024-04-26T23:45:34.303370+0000","last_clean":"2024-04-26T23:45:34.303370+0000","last_became_active":"2024-04-26T23:45:31.878260+0000","last_became_peered":"2024-04-26T23:45:31.878260+0000","last_unstale":"2024-04-26T23:45:34.303370+0000","last_undegraded":"2024-04-26T23:45:34.303370+0000","last_fullsized":"2024-04-26T23:45:34.303370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:01.011964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301596+0000","last_change":"2024-04-26T23:45:32.896520+0000","last_active":"2024-04-26T23:45:34.301596+0000","last_peered":"2024-04-26T23:45:34.301596+0000","last_clean":"2024-04-26T23:45:34.301596+0000","last_became_active":"2024-04-26T23:45:32.896191+0000","last_became_peered":"2024-04-26T23:45:32.896191+0000","last_unstale":"2024-04-26T23:45:34.301596+0000","last_undegraded":"2024-04-26T23:45:34.301596+0000","last_fullsized":"2024-04-26T23:45:34.301596+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:23.995760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301108+0000","last_change":"2024-04-26T23:45:31.890747+0000","last_active":"2024-04-26T23:45:34.301108+0000","last_peered":"2024-04-26T23:45:34.301108+0000","last_clean":"2024-04-26T23:45:34.301108+0000","last_became_active":"2024-04-26T23:45:31.890454+0000","last_became_peered":"2024-04-26T23:45:31.890454+0000","last_unstale":"2024-04-26T23:45:34.301108+0000","last_undegraded":"2024-04-26T23:45:34.301108+0000","last_fullsized":"2024-04-26T23:45:34.301108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:12.324313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876562+0000","last_change":"2024-04-26T23:45:32.891583+0000","last_active":"2024-04-26T23:45:33.876562+0000","last_peered":"2024-04-26T23:45:33.876562+0000","last_clean":"2024-04-26T23:45:33.876562+0000","last_became_active":"2024-04-26T23:45:32.889398+0000","last_became_peered":"2024-04-26T23:45:32.889398+0000","last_unstale":"2024-04-26T23:45:33.876562+0000","last_undegraded":"2024-04-26T23:45:33.876562+0000","last_fullsized":"2024-04-26T23:45:33.876562+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:09.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881082+0000","last_change":"2024-04-26T23:45:31.886764+0000","last_active":"2024-04-26T23:45:33.881082+0000","last_peered":"2024-04-26T23:45:33.881082+0000","last_clean":"2024-04-26T23:45:33.881082+0000","last_became_active":"2024-04-26T23:45:31.884462+0000","last_became_peered":"2024-04-26T23:45:31.884462+0000","last_unstale":"2024-04-26T23:45:33.881082+0000","last_undegraded":"2024-04-26T23:45:33.881082+0000","last_fullsized":"2024-04-26T23:45:33.881082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:49.776017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880890+0000","last_change":"2024-04-26T23:45:32.888133+0000","last_active":"2024-04-26T23:45:33.880890+0000","last_peered":"2024-04-26T23:45:33.880890+0000","last_clean":"2024-04-26T23:45:33.880890+0000","last_became_active":"2024-04-26T23:45:32.887839+0000","last_became_peered":"2024-04-26T23:45:32.887839+0000","last_unstale":"2024-04-26T23:45:33.880890+0000","last_undegraded":"2024-04-26T23:45:33.880890+0000","last_fullsized":"2024-04-26T23:45:33.880890+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876756+0000","last_change":"2024-04-26T23:45:31.885140+0000","last_active":"2024-04-26T23:45:33.876756+0000","last_peered":"2024-04-26T23:45:33.876756+0000","last_clean":"2024-04-26T23:45:33.876756+0000","last_became_active":"2024-04-26T23:45:31.884819+0000","last_became_peered":"2024-04-26T23:45:31.884819+0000","last_unstale":"2024-04-26T23:45:33.876756+0000","last_undegraded":"2024-04-26T23:45:33.876756+0000","last_fullsized":"2024-04-26T23:45:33.876756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:15.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221786+0000","last_change":"2024-04-26T23:45:32.896746+0000","last_active":"2024-04-26T23:45:34.221786+0000","last_peered":"2024-04-26T23:45:34.221786+0000","last_clean":"2024-04-26T23:45:34.221786+0000","last_became_active":"2024-04-26T23:45:32.894779+0000","last_became_peered":"2024-04-26T23:45:32.894779+0000","last_unstale":"2024-04-26T23:45:34.221786+0000","last_undegraded":"2024-04-26T23:45:34.221786+0000","last_fullsized":"2024-04-26T23:45:34.221786+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:54.791803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303998+0000","last_change":"2024-04-26T23:45:31.878288+0000","last_active":"2024-04-26T23:45:34.303998+0000","last_peered":"2024-04-26T23:45:34.303998+0000","last_clean":"2024-04-26T23:45:34.303998+0000","last_became_active":"2024-04-26T23:45:31.876834+0000","last_became_peered":"2024-04-26T23:45:31.876834+0000","last_unstale":"2024-04-26T23:45:34.303998+0000","last_undegraded":"2024-04-26T23:45:34.303998+0000","last_fullsized":"2024-04-26T23:45:34.303998+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:03.669792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895230+0000","last_change":"2024-04-26T23:45:32.896803+0000","last_active":"2024-04-26T23:45:33.895230+0000","last_peered":"2024-04-26T23:45:33.895230+0000","last_clean":"2024-04-26T23:45:33.895230+0000","last_became_active":"2024-04-26T23:45:32.895374+0000","last_became_peered":"2024-04-26T23:45:32.895374+0000","last_unstale":"2024-04-26T23:45:33.895230+0000","last_undegraded":"2024-04-26T23:45:33.895230+0000","last_fullsized":"2024-04-26T23:45:33.895230+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:02.230627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880592+0000","last_change":"2024-04-26T23:45:31.883746+0000","last_active":"2024-04-26T23:45:33.880592+0000","last_peered":"2024-04-26T23:45:33.880592+0000","last_clean":"2024-04-26T23:45:33.880592+0000","last_became_active":"2024-04-26T23:45:31.882969+0000","last_became_peered":"2024-04-26T23:45:31.882969+0000","last_unstale":"2024-04-26T23:45:33.880592+0000","last_undegraded":"2024-04-26T23:45:33.880592+0000","last_fullsized":"2024-04-26T23:45:33.880592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:25.870271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301344+0000","last_change":"2024-04-26T23:45:32.901524+0000","last_active":"2024-04-26T23:45:34.301344+0000","last_peered":"2024-04-26T23:45:34.301344+0000","last_clean":"2024-04-26T23:45:34.301344+0000","last_became_active":"2024-04-26T23:45:32.901182+0000","last_became_peered":"2024-04-26T23:45:32.901182+0000","last_unstale":"2024-04-26T23:45:34.301344+0000","last_undegraded":"2024-04-26T23:45:34.301344+0000","last_fullsized":"2024-04-26T23:45:34.301344+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.575166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880775+0000","last_change":"2024-04-26T23:45:31.883532+0000","last_active":"2024-04-26T23:45:33.880775+0000","last_peered":"2024-04-26T23:45:33.880775+0000","last_clean":"2024-04-26T23:45:33.880775+0000","last_became_active":"2024-04-26T23:45:31.883090+0000","last_became_peered":"2024-04-26T23:45:31.883090+0000","last_unstale":"2024-04-26T23:45:33.880775+0000","last_undegraded":"2024-04-26T23:45:33.880775+0000","last_fullsized":"2024-04-26T23:45:33.880775+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:25.870271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301523+0000","last_change":"2024-04-26T23:45:32.897526+0000","last_active":"2024-04-26T23:45:34.301523+0000","last_peered":"2024-04-26T23:45:34.301523+0000","last_clean":"2024-04-26T23:45:34.301523+0000","last_became_active":"2024-04-26T23:45:32.897234+0000","last_became_peered":"2024-04-26T23:45:32.897234+0000","last_unstale":"2024-04-26T23:45:34.301523+0000","last_undegraded":"2024-04-26T23:45:34.301523+0000","last_fullsized":"2024-04-26T23:45:34.301523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:21.458052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300998+0000","last_change":"2024-04-26T23:45:31.881587+0000","last_active":"2024-04-26T23:45:34.300998+0000","last_peered":"2024-04-26T23:45:34.300998+0000","last_clean":"2024-04-26T23:45:34.300998+0000","last_became_active":"2024-04-26T23:45:31.881296+0000","last_became_peered":"2024-04-26T23:45:31.881296+0000","last_unstale":"2024-04-26T23:45:34.300998+0000","last_undegraded":"2024-04-26T23:45:34.300998+0000","last_fullsized":"2024-04-26T23:45:34.300998+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:54:08.315315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.305117+0000","last_change":"2024-04-26T23:45:32.890084+0000","last_active":"2024-04-26T23:45:34.305117+0000","last_peered":"2024-04-26T23:45:34.305117+0000","last_clean":"2024-04-26T23:45:34.305117+0000","last_became_active":"2024-04-26T23:45:32.889450+0000","last_became_peered":"2024-04-26T23:45:32.889450+0000","last_unstale":"2024-04-26T23:45:34.305117+0000","last_undegraded":"2024-04-26T23:45:34.305117+0000","last_fullsized":"2024-04-26T23:45:34.305117+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.464920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303475+0000","last_change":"2024-04-26T23:45:31.874536+0000","last_active":"2024-04-26T23:45:34.303475+0000","last_peered":"2024-04-26T23:45:34.303475+0000","last_clean":"2024-04-26T23:45:34.303475+0000","last_became_active":"2024-04-26T23:45:31.874213+0000","last_became_peered":"2024-04-26T23:45:31.874213+0000","last_unstale":"2024-04-26T23:45:34.303475+0000","last_undegraded":"2024-04-26T23:45:34.303475+0000","last_fullsized":"2024-04-26T23:45:34.303475+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:22.163710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895676+0000","last_change":"2024-04-26T23:45:32.880264+0000","last_active":"2024-04-26T23:45:33.895676+0000","last_peered":"2024-04-26T23:45:33.895676+0000","last_clean":"2024-04-26T23:45:33.895676+0000","last_became_active":"2024-04-26T23:45:32.879965+0000","last_became_peered":"2024-04-26T23:45:32.879965+0000","last_unstale":"2024-04-26T23:45:33.895676+0000","last_undegraded":"2024-04-26T23:45:33.895676+0000","last_fullsized":"2024-04-26T23:45:33.895676+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:10.670319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301255+0000","last_change":"2024-04-26T23:45:31.882674+0000","last_active":"2024-04-26T23:45:34.301255+0000","last_peered":"2024-04-26T23:45:34.301255+0000","last_clean":"2024-04-26T23:45:34.301255+0000","last_became_active":"2024-04-26T23:45:31.881987+0000","last_became_peered":"2024-04-26T23:45:31.881987+0000","last_unstale":"2024-04-26T23:45:34.301255+0000","last_undegraded":"2024-04-26T23:45:34.301255+0000","last_fullsized":"2024-04-26T23:45:34.301255+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:38.140097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.894663+0000","last_change":"2024-04-26T23:45:32.886402+0000","last_active":"2024-04-26T23:45:33.894663+0000","last_peered":"2024-04-26T23:45:33.894663+0000","last_clean":"2024-04-26T23:45:33.894663+0000","last_became_active":"2024-04-26T23:45:32.886141+0000","last_became_peered":"2024-04-26T23:45:32.886141+0000","last_unstale":"2024-04-26T23:45:33.894663+0000","last_undegraded":"2024-04-26T23:45:33.894663+0000","last_fullsized":"2024-04-26T23:45:33.894663+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:20.737661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301742+0000","last_change":"2024-04-26T23:45:31.886926+0000","last_active":"2024-04-26T23:45:34.301742+0000","last_peered":"2024-04-26T23:45:34.301742+0000","last_clean":"2024-04-26T23:45:34.301742+0000","last_became_active":"2024-04-26T23:45:31.886626+0000","last_became_peered":"2024-04-26T23:45:31.886626+0000","last_unstale":"2024-04-26T23:45:34.301742+0000","last_undegraded":"2024-04-26T23:45:34.301742+0000","last_fullsized":"2024-04-26T23:45:34.301742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:18.057584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":334,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301943+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:45:34.301943+0000","last_peered":"2024-04-26T23:45:34.301943+0000","last_clean":"2024-04-26T23:45:34.301943+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:45:34.301943+0000","last_undegraded":"2024-04-26T23:45:34.301943+0000","last_fullsized":"2024-04-26T23:45:34.301943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":1235035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40868,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330972,"statfs":{"total":96636764160,"available":96594915328,"internally_reserved":0,"allocated":1396736,"data_stored":646723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":644507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1384448,"data_stored":644599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":645089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":644541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1380352,"data_stored":644541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45604,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326236,"statfs":{"total":96636764160,"available":96590065664,"internally_reserved":0,"allocated":1986560,"data_stored":1236577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:45:37.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:45:37.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.652+0000 7f48c8e00640 1 Processor -- start 2024-04-26T23:45:37.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.652+0000 7f48c8e00640 1 -- start start 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.653+0000 7f48c8e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40d0f50 0x7f48c406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.653+0000 7f48c8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48c40b86b0 0x7f48c40d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.653+0000 7f48c8e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c40d21d0 con 0x7f48c40d1330 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c8e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f48c40d1ef0 con 0x7f48c40b86b0 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c8e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48c40b6bc0 con 0x7f48c40d0f50 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48c40b86b0 0x7f48c40d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48c40b86b0 0x7f48c40d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48814/0 (socket says 172.21.15.46:48814) 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 -- 172.21.15.46:0/936960414 learned_addr learned my addr 172.21.15.46:0/936960414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3400640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40d0f50 0x7f48c406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 -- 172.21.15.46:0/936960414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40d0f50 msgr2=0x7f48c406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40d0f50 0x7f48c406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 -- 172.21.15.46:0/936960414 >> v1:172.21.15.142:6789/0 conn(0x7f48c40d1330 legacy=0x7f48c40b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:37.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.654+0000 7f48c3e00640 1 -- 172.21.15.46:0/936960414 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40b6ea0 con 0x7f48c40b86b0 2024-04-26T23:45:37.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c3e00640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40b86b0 0x7f48c40d17b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009e50 tx=0x7f48bc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8ba9138be6ec6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c2a00640 1 -- 172.21.15.46:0/936960414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc036070 con 0x7f48c40b86b0 2024-04-26T23:45:37.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c2a00640 1 -- 172.21.15.46:0/936960414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc031600 con 0x7f48c40b86b0 2024-04-26T23:45:37.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c2a00640 1 -- 172.21.15.46:0/936960414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0319d0 con 0x7f48c40b86b0 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c8e00640 1 -- 172.21.15.46:0/936960414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40b86b0 msgr2=0x7f48c40d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.656+0000 7f48c8e00640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40b86b0 0x7f48c40d17b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009e50 tx=0x7f48bc02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 -- 172.21.15.46:0/936960414 shutdown_connections 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40b86b0 0x7f48c40d17b0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 --2- 172.21.15.46:0/936960414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40d0f50 0x7f48c406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 -- 172.21.15.46:0/936960414 >> 172.21.15.46:0/936960414 conn(0x7f48c40d2cc0 msgr2=0x7f48c40d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 -- 172.21.15.46:0/936960414 shutdown_connections 2024-04-26T23:45:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.657+0000 7f48c8e00640 1 -- 172.21.15.46:0/936960414 wait complete. 2024-04-26T23:45:37.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.658+0000 7f48c8e00640 1 Processor -- start 2024-04-26T23:45:37.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 -- start start 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 0x7f48c40cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48c40d0f50 0x7f48c40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48c415f7c0 con 0x7f48c40d1330 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48c415f640 con 0x7f48c40d0f50 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.659+0000 7f48c8e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48c40b3ff0 con 0x7f48c40b86b0 2024-04-26T23:45:37.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48828/0 (socket says 172.21.15.46:48828) 2024-04-26T23:45:37.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 -- 172.21.15.46:0/4136026053 learned_addr learned my addr 172.21.15.46:0/4136026053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3400640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 0x7f48c40cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48baa00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48c40d0f50 0x7f48c40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 -- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 msgr2=0x7f48c40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 0x7f48c40cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 -- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48c40d0f50 msgr2=0x7f48c40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48c40d0f50 0x7f48c40ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3e00640 1 -- 172.21.15.46:0/4136026053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40afc00 con 0x7f48c40d1330 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.660+0000 7f48c3400640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 0x7f48c40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48c3e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f48bc034040 tx=0x7f48bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc036050 con 0x7f48c40d1330 2024-04-26T23:45:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc042070 con 0x7f48c40d1330 2024-04-26T23:45:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc009510 con 0x7f48c40d1330 2024-04-26T23:45:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c40afee0 con 0x7f48c40d1330 2024-04-26T23:45:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.661+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c40b1550 con 0x7f48c40d1330 2024-04-26T23:45:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.663+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c4069ac0 con 0x7f48c40d1330 2024-04-26T23:45:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.664+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48bc007bd0 con 0x7f48c40d1330 2024-04-26T23:45:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.665+0000 7f48bbe00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 0x7f48a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.665+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f48bc0accc0 con 0x7f48c40d1330 2024-04-26T23:45:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.665+0000 7f48c3400640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 0x7f48a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.666+0000 7f48c3400640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 0x7f48a006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.669+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48bc0b3080 con 0x7f48c40d1330 2024-04-26T23:45:37.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.875+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48c406dc50 con 0x7f48a006c8f0 2024-04-26T23:45:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.891+0000 7f48bbe00640 1 -- 172.21.15.46:0/4136026053 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f48c406dc50 con 0x7f48a006c8f0 2024-04-26T23:45:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:45:37.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 msgr2=0x7f48a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 0x7f48a006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 msgr2=0x7f48c40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f48bc034040 tx=0x7f48bc004a40 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 shutdown_connections 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48a006c8f0 0x7f48a006edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48c40d1330 0x7f48c40ce880 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48c40d0f50 0x7f48c40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 --2- 172.21.15.46:0/4136026053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48c40b86b0 0x7f48c40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 >> 172.21.15.46:0/4136026053 conn(0x7f48c40d2cc0 msgr2=0x7f48c40b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 shutdown_connections 2024-04-26T23:45:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:37.898+0000 7f48c8e00640 1 -- 172.21.15.46:0/4136026053 wait complete. 2024-04-26T23:45:37.925 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T23:45:36.074213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323840,"kb_used_data":11976,"kb_used_omap":12,"kb_used_meta":311731,"kb_avail":754650880,"statfs":{"total":773094113280,"available":772762501120,"internally_reserved":0,"allocated":12263424,"data_stored":6341612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":319213105},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.237669"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876317+0000","last_change":"2024-04-26T23:45:32.891792+0000","last_active":"2024-04-26T23:45:33.876317+0000","last_peered":"2024-04-26T23:45:33.876317+0000","last_clean":"2024-04-26T23:45:33.876317+0000","last_became_active":"2024-04-26T23:45:32.887973+0000","last_became_peered":"2024-04-26T23:45:32.887973+0000","last_unstale":"2024-04-26T23:45:33.876317+0000","last_undegraded":"2024-04-26T23:45:33.876317+0000","last_fullsized":"2024-04-26T23:45:33.876317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:38.717777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303755+0000","last_change":"2024-04-26T23:45:32.892543+0000","last_active":"2024-04-26T23:45:34.303755+0000","last_peered":"2024-04-26T23:45:34.303755+0000","last_clean":"2024-04-26T23:45:34.303755+0000","last_became_active":"2024-04-26T23:45:32.892251+0000","last_became_peered":"2024-04-26T23:45:32.892251+0000","last_unstale":"2024-04-26T23:45:34.303755+0000","last_undegraded":"2024-04-26T23:45:34.303755+0000","last_fullsized":"2024-04-26T23:45:34.303755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:32.182610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876622+0000","last_change":"2024-04-26T23:45:32.893090+0000","last_active":"2024-04-26T23:45:33.876622+0000","last_peered":"2024-04-26T23:45:33.876622+0000","last_clean":"2024-04-26T23:45:33.876622+0000","last_became_active":"2024-04-26T23:45:32.891297+0000","last_became_peered":"2024-04-26T23:45:32.891297+0000","last_unstale":"2024-04-26T23:45:33.876622+0000","last_undegraded":"2024-04-26T23:45:33.876622+0000","last_fullsized":"2024-04-26T23:45:33.876622+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:27.620869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222123+0000","last_change":"2024-04-26T23:45:32.892911+0000","last_active":"2024-04-26T23:45:34.222123+0000","last_peered":"2024-04-26T23:45:34.222123+0000","last_clean":"2024-04-26T23:45:34.222123+0000","last_became_active":"2024-04-26T23:45:32.891887+0000","last_became_peered":"2024-04-26T23:45:32.891887+0000","last_unstale":"2024-04-26T23:45:34.222123+0000","last_undegraded":"2024-04-26T23:45:34.222123+0000","last_fullsized":"2024-04-26T23:45:34.222123+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:13.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880459+0000","last_change":"2024-04-26T23:45:32.896396+0000","last_active":"2024-04-26T23:45:33.880459+0000","last_peered":"2024-04-26T23:45:33.880459+0000","last_clean":"2024-04-26T23:45:33.880459+0000","last_became_active":"2024-04-26T23:45:32.896068+0000","last_became_peered":"2024-04-26T23:45:32.896068+0000","last_unstale":"2024-04-26T23:45:33.880459+0000","last_undegraded":"2024-04-26T23:45:33.880459+0000","last_fullsized":"2024-04-26T23:45:33.880459+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303214+0000","last_change":"2024-04-26T23:45:32.894808+0000","last_active":"2024-04-26T23:45:34.303214+0000","last_peered":"2024-04-26T23:45:34.303214+0000","last_clean":"2024-04-26T23:45:34.303214+0000","last_became_active":"2024-04-26T23:45:32.894523+0000","last_became_peered":"2024-04-26T23:45:32.894523+0000","last_unstale":"2024-04-26T23:45:34.303214+0000","last_undegraded":"2024-04-26T23:45:34.303214+0000","last_fullsized":"2024-04-26T23:45:34.303214+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:45.130923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302472+0000","last_change":"2024-04-26T23:45:32.894223+0000","last_active":"2024-04-26T23:45:34.302472+0000","last_peered":"2024-04-26T23:45:34.302472+0000","last_clean":"2024-04-26T23:45:34.302472+0000","last_became_active":"2024-04-26T23:45:32.893948+0000","last_became_peered":"2024-04-26T23:45:32.893948+0000","last_unstale":"2024-04-26T23:45:34.302472+0000","last_undegraded":"2024-04-26T23:45:34.302472+0000","last_fullsized":"2024-04-26T23:45:34.302472+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:20.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300761+0000","last_change":"2024-04-26T23:45:32.899983+0000","last_active":"2024-04-26T23:45:34.300761+0000","last_peered":"2024-04-26T23:45:34.300761+0000","last_clean":"2024-04-26T23:45:34.300761+0000","last_became_active":"2024-04-26T23:45:32.899657+0000","last_became_peered":"2024-04-26T23:45:32.899657+0000","last_unstale":"2024-04-26T23:45:34.300761+0000","last_undegraded":"2024-04-26T23:45:34.300761+0000","last_fullsized":"2024-04-26T23:45:34.300761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:03.197511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304397+0000","last_change":"2024-04-26T23:45:32.892658+0000","last_active":"2024-04-26T23:45:34.304397+0000","last_peered":"2024-04-26T23:45:34.304397+0000","last_clean":"2024-04-26T23:45:34.304397+0000","last_became_active":"2024-04-26T23:45:32.892327+0000","last_became_peered":"2024-04-26T23:45:32.892327+0000","last_unstale":"2024-04-26T23:45:34.304397+0000","last_undegraded":"2024-04-26T23:45:34.304397+0000","last_fullsized":"2024-04-26T23:45:34.304397+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880887+0000","last_change":"2024-04-26T23:45:32.899150+0000","last_active":"2024-04-26T23:45:33.880887+0000","last_peered":"2024-04-26T23:45:33.880887+0000","last_clean":"2024-04-26T23:45:33.880887+0000","last_became_active":"2024-04-26T23:45:32.898602+0000","last_became_peered":"2024-04-26T23:45:32.898602+0000","last_unstale":"2024-04-26T23:45:33.880887+0000","last_undegraded":"2024-04-26T23:45:33.880887+0000","last_fullsized":"2024-04-26T23:45:33.880887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:43.530478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895255+0000","last_change":"2024-04-26T23:45:32.898242+0000","last_active":"2024-04-26T23:45:33.895255+0000","last_peered":"2024-04-26T23:45:33.895255+0000","last_clean":"2024-04-26T23:45:33.895255+0000","last_became_active":"2024-04-26T23:45:32.897681+0000","last_became_peered":"2024-04-26T23:45:32.897681+0000","last_unstale":"2024-04-26T23:45:33.895255+0000","last_undegraded":"2024-04-26T23:45:33.895255+0000","last_fullsized":"2024-04-26T23:45:33.895255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:28.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881386+0000","last_change":"2024-04-26T23:45:32.887503+0000","last_active":"2024-04-26T23:45:33.881386+0000","last_peered":"2024-04-26T23:45:33.881386+0000","last_clean":"2024-04-26T23:45:33.881386+0000","last_became_active":"2024-04-26T23:45:32.887250+0000","last_became_peered":"2024-04-26T23:45:32.887250+0000","last_unstale":"2024-04-26T23:45:33.881386+0000","last_undegraded":"2024-04-26T23:45:33.881386+0000","last_fullsized":"2024-04-26T23:45:33.881386+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:27.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221547+0000","last_change":"2024-04-26T23:45:32.887310+0000","last_active":"2024-04-26T23:45:34.221547+0000","last_peered":"2024-04-26T23:45:34.221547+0000","last_clean":"2024-04-26T23:45:34.221547+0000","last_became_active":"2024-04-26T23:45:32.886989+0000","last_became_peered":"2024-04-26T23:45:32.886989+0000","last_unstale":"2024-04-26T23:45:34.221547+0000","last_undegraded":"2024-04-26T23:45:34.221547+0000","last_fullsized":"2024-04-26T23:45:34.221547+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:28.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303408+0000","last_change":"2024-04-26T23:45:32.886030+0000","last_active":"2024-04-26T23:45:34.303408+0000","last_peered":"2024-04-26T23:45:34.303408+0000","last_clean":"2024-04-26T23:45:34.303408+0000","last_became_active":"2024-04-26T23:45:32.885693+0000","last_became_peered":"2024-04-26T23:45:32.885693+0000","last_unstale":"2024-04-26T23:45:34.303408+0000","last_undegraded":"2024-04-26T23:45:34.303408+0000","last_fullsized":"2024-04-26T23:45:34.303408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:08.801629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302317+0000","last_change":"2024-04-26T23:45:32.878481+0000","last_active":"2024-04-26T23:45:34.302317+0000","last_peered":"2024-04-26T23:45:34.302317+0000","last_clean":"2024-04-26T23:45:34.302317+0000","last_became_active":"2024-04-26T23:45:32.878177+0000","last_became_peered":"2024-04-26T23:45:32.878177+0000","last_unstale":"2024-04-26T23:45:34.302317+0000","last_undegraded":"2024-04-26T23:45:34.302317+0000","last_fullsized":"2024-04-26T23:45:34.302317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:45.140341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302228+0000","last_change":"2024-04-26T23:45:32.886120+0000","last_active":"2024-04-26T23:45:34.302228+0000","last_peered":"2024-04-26T23:45:34.302228+0000","last_clean":"2024-04-26T23:45:34.302228+0000","last_became_active":"2024-04-26T23:45:32.885809+0000","last_became_peered":"2024-04-26T23:45:32.885809+0000","last_unstale":"2024-04-26T23:45:34.302228+0000","last_undegraded":"2024-04-26T23:45:34.302228+0000","last_fullsized":"2024-04-26T23:45:34.302228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:08.526634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876824+0000","last_change":"2024-04-26T23:45:32.887630+0000","last_active":"2024-04-26T23:45:33.876824+0000","last_peered":"2024-04-26T23:45:33.876824+0000","last_clean":"2024-04-26T23:45:33.876824+0000","last_became_active":"2024-04-26T23:45:32.887097+0000","last_became_peered":"2024-04-26T23:45:32.887097+0000","last_unstale":"2024-04-26T23:45:33.876824+0000","last_undegraded":"2024-04-26T23:45:33.876824+0000","last_fullsized":"2024-04-26T23:45:33.876824+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:39.859921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303808+0000","last_change":"2024-04-26T23:45:32.897689+0000","last_active":"2024-04-26T23:45:34.303808+0000","last_peered":"2024-04-26T23:45:34.303808+0000","last_clean":"2024-04-26T23:45:34.303808+0000","last_became_active":"2024-04-26T23:45:32.896867+0000","last_became_peered":"2024-04-26T23:45:32.896867+0000","last_unstale":"2024-04-26T23:45:34.303808+0000","last_undegraded":"2024-04-26T23:45:34.303808+0000","last_fullsized":"2024-04-26T23:45:34.303808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.581149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876661+0000","last_change":"2024-04-26T23:45:32.900248+0000","last_active":"2024-04-26T23:45:33.876661+0000","last_peered":"2024-04-26T23:45:33.876661+0000","last_clean":"2024-04-26T23:45:33.876661+0000","last_became_active":"2024-04-26T23:45:32.899026+0000","last_became_peered":"2024-04-26T23:45:32.899026+0000","last_unstale":"2024-04-26T23:45:33.876661+0000","last_undegraded":"2024-04-26T23:45:33.876661+0000","last_fullsized":"2024-04-26T23:45:33.876661+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:46.600183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881808+0000","last_change":"2024-04-26T23:45:32.891886+0000","last_active":"2024-04-26T23:45:33.881808+0000","last_peered":"2024-04-26T23:45:33.881808+0000","last_clean":"2024-04-26T23:45:33.881808+0000","last_became_active":"2024-04-26T23:45:32.890365+0000","last_became_peered":"2024-04-26T23:45:32.890365+0000","last_unstale":"2024-04-26T23:45:33.881808+0000","last_undegraded":"2024-04-26T23:45:33.881808+0000","last_fullsized":"2024-04-26T23:45:33.881808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:43.530478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301912+0000","last_change":"2024-04-26T23:45:32.888704+0000","last_active":"2024-04-26T23:45:34.301912+0000","last_peered":"2024-04-26T23:45:34.301912+0000","last_clean":"2024-04-26T23:45:34.301912+0000","last_became_active":"2024-04-26T23:45:32.888385+0000","last_became_peered":"2024-04-26T23:45:32.888385+0000","last_unstale":"2024-04-26T23:45:34.301912+0000","last_undegraded":"2024-04-26T23:45:34.301912+0000","last_fullsized":"2024-04-26T23:45:34.301912+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:10.622727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880728+0000","last_change":"2024-04-26T23:45:32.899405+0000","last_active":"2024-04-26T23:45:33.880728+0000","last_peered":"2024-04-26T23:45:33.880728+0000","last_clean":"2024-04-26T23:45:33.880728+0000","last_became_active":"2024-04-26T23:45:32.898872+0000","last_became_peered":"2024-04-26T23:45:32.898872+0000","last_unstale":"2024-04-26T23:45:33.880728+0000","last_undegraded":"2024-04-26T23:45:33.880728+0000","last_fullsized":"2024-04-26T23:45:33.880728+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876167+0000","last_change":"2024-04-26T23:45:32.895597+0000","last_active":"2024-04-26T23:45:33.876167+0000","last_peered":"2024-04-26T23:45:33.876167+0000","last_clean":"2024-04-26T23:45:33.876167+0000","last_became_active":"2024-04-26T23:45:32.887956+0000","last_became_peered":"2024-04-26T23:45:32.887956+0000","last_unstale":"2024-04-26T23:45:33.876167+0000","last_undegraded":"2024-04-26T23:45:33.876167+0000","last_fullsized":"2024-04-26T23:45:33.876167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:11.919345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880223+0000","last_change":"2024-04-26T23:45:32.902007+0000","last_active":"2024-04-26T23:45:33.880223+0000","last_peered":"2024-04-26T23:45:33.880223+0000","last_clean":"2024-04-26T23:45:33.880223+0000","last_became_active":"2024-04-26T23:45:32.901708+0000","last_became_peered":"2024-04-26T23:45:32.901708+0000","last_unstale":"2024-04-26T23:45:33.880223+0000","last_undegraded":"2024-04-26T23:45:33.880223+0000","last_fullsized":"2024-04-26T23:45:33.880223+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.581149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881911+0000","last_change":"2024-04-26T23:45:32.897039+0000","last_active":"2024-04-26T23:45:33.881911+0000","last_peered":"2024-04-26T23:45:33.881911+0000","last_clean":"2024-04-26T23:45:33.881911+0000","last_became_active":"2024-04-26T23:45:32.896772+0000","last_became_peered":"2024-04-26T23:45:32.896772+0000","last_unstale":"2024-04-26T23:45:33.881911+0000","last_undegraded":"2024-04-26T23:45:33.881911+0000","last_fullsized":"2024-04-26T23:45:33.881911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:56.852278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.882410+0000","last_change":"2024-04-26T23:45:32.896863+0000","last_active":"2024-04-26T23:45:33.882410+0000","last_peered":"2024-04-26T23:45:33.882410+0000","last_clean":"2024-04-26T23:45:33.882410+0000","last_became_active":"2024-04-26T23:45:32.896605+0000","last_became_peered":"2024-04-26T23:45:32.896605+0000","last_unstale":"2024-04-26T23:45:33.882410+0000","last_undegraded":"2024-04-26T23:45:33.882410+0000","last_fullsized":"2024-04-26T23:45:33.882410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:20.369804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895102+0000","last_change":"2024-04-26T23:45:32.882004+0000","last_active":"2024-04-26T23:45:33.895102+0000","last_peered":"2024-04-26T23:45:33.895102+0000","last_clean":"2024-04-26T23:45:33.895102+0000","last_became_active":"2024-04-26T23:45:32.881696+0000","last_became_peered":"2024-04-26T23:45:32.881696+0000","last_unstale":"2024-04-26T23:45:33.895102+0000","last_undegraded":"2024-04-26T23:45:33.895102+0000","last_fullsized":"2024-04-26T23:45:33.895102+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:07.996624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303223+0000","last_change":"2024-04-26T23:45:32.894243+0000","last_active":"2024-04-26T23:45:34.303223+0000","last_peered":"2024-04-26T23:45:34.303223+0000","last_clean":"2024-04-26T23:45:34.303223+0000","last_became_active":"2024-04-26T23:45:32.892442+0000","last_became_peered":"2024-04-26T23:45:32.892442+0000","last_unstale":"2024-04-26T23:45:34.303223+0000","last_undegraded":"2024-04-26T23:45:34.303223+0000","last_fullsized":"2024-04-26T23:45:34.303223+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:20.942027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301903+0000","last_change":"2024-04-26T23:45:32.893890+0000","last_active":"2024-04-26T23:45:34.301903+0000","last_peered":"2024-04-26T23:45:34.301903+0000","last_clean":"2024-04-26T23:45:34.301903+0000","last_became_active":"2024-04-26T23:45:32.892216+0000","last_became_peered":"2024-04-26T23:45:32.892216+0000","last_unstale":"2024-04-26T23:45:34.301903+0000","last_undegraded":"2024-04-26T23:45:34.301903+0000","last_fullsized":"2024-04-26T23:45:34.301903+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:53.121661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880956+0000","last_change":"2024-04-26T23:45:32.899713+0000","last_active":"2024-04-26T23:45:33.880956+0000","last_peered":"2024-04-26T23:45:33.880956+0000","last_clean":"2024-04-26T23:45:33.880956+0000","last_became_active":"2024-04-26T23:45:32.899433+0000","last_became_peered":"2024-04-26T23:45:32.899433+0000","last_unstale":"2024-04-26T23:45:33.880956+0000","last_undegraded":"2024-04-26T23:45:33.880956+0000","last_fullsized":"2024-04-26T23:45:33.880956+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.302752+0000","last_change":"2024-04-26T23:45:32.910663+0000","last_active":"2024-04-26T23:45:34.302752+0000","last_peered":"2024-04-26T23:45:34.302752+0000","last_clean":"2024-04-26T23:45:34.302752+0000","last_became_active":"2024-04-26T23:45:32.910310+0000","last_became_peered":"2024-04-26T23:45:32.910310+0000","last_unstale":"2024-04-26T23:45:34.302752+0000","last_undegraded":"2024-04-26T23:45:34.302752+0000","last_fullsized":"2024-04-26T23:45:34.302752+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:42.385444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301029+0000","last_change":"2024-04-26T23:45:32.895307+0000","last_active":"2024-04-26T23:45:34.301029+0000","last_peered":"2024-04-26T23:45:34.301029+0000","last_clean":"2024-04-26T23:45:34.301029+0000","last_became_active":"2024-04-26T23:45:32.895010+0000","last_became_peered":"2024-04-26T23:45:32.895010+0000","last_unstale":"2024-04-26T23:45:34.301029+0000","last_undegraded":"2024-04-26T23:45:34.301029+0000","last_fullsized":"2024-04-26T23:45:34.301029+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:31.870024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877412+0000","last_change":"2024-04-26T23:45:32.897165+0000","last_active":"2024-04-26T23:45:33.877412+0000","last_peered":"2024-04-26T23:45:33.877412+0000","last_clean":"2024-04-26T23:45:33.877412+0000","last_became_active":"2024-04-26T23:45:32.896446+0000","last_became_peered":"2024-04-26T23:45:32.896446+0000","last_unstale":"2024-04-26T23:45:33.877412+0000","last_undegraded":"2024-04-26T23:45:33.877412+0000","last_fullsized":"2024-04-26T23:45:33.877412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:30.757390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222139+0000","last_change":"2024-04-26T23:45:31.876231+0000","last_active":"2024-04-26T23:45:34.222139+0000","last_peered":"2024-04-26T23:45:34.222139+0000","last_clean":"2024-04-26T23:45:34.222139+0000","last_became_active":"2024-04-26T23:45:31.875441+0000","last_became_peered":"2024-04-26T23:45:31.875441+0000","last_unstale":"2024-04-26T23:45:34.222139+0000","last_undegraded":"2024-04-26T23:45:34.222139+0000","last_fullsized":"2024-04-26T23:45:34.222139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:19.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301694+0000","last_change":"2024-04-26T23:45:32.897608+0000","last_active":"2024-04-26T23:45:34.301694+0000","last_peered":"2024-04-26T23:45:34.301694+0000","last_clean":"2024-04-26T23:45:34.301694+0000","last_became_active":"2024-04-26T23:45:32.897291+0000","last_became_peered":"2024-04-26T23:45:32.897291+0000","last_unstale":"2024-04-26T23:45:34.301694+0000","last_undegraded":"2024-04-26T23:45:34.301694+0000","last_fullsized":"2024-04-26T23:45:34.301694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:00.717172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301389+0000","last_change":"2024-04-26T23:45:31.889214+0000","last_active":"2024-04-26T23:45:34.301389+0000","last_peered":"2024-04-26T23:45:34.301389+0000","last_clean":"2024-04-26T23:45:34.301389+0000","last_became_active":"2024-04-26T23:45:31.888934+0000","last_became_peered":"2024-04-26T23:45:31.888934+0000","last_unstale":"2024-04-26T23:45:34.301389+0000","last_undegraded":"2024-04-26T23:45:34.301389+0000","last_fullsized":"2024-04-26T23:45:34.301389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:27.660581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881352+0000","last_change":"2024-04-26T23:45:32.897239+0000","last_active":"2024-04-26T23:45:33.881352+0000","last_peered":"2024-04-26T23:45:33.881352+0000","last_clean":"2024-04-26T23:45:33.881352+0000","last_became_active":"2024-04-26T23:45:32.896145+0000","last_became_peered":"2024-04-26T23:45:32.896145+0000","last_unstale":"2024-04-26T23:45:33.881352+0000","last_undegraded":"2024-04-26T23:45:33.881352+0000","last_fullsized":"2024-04-26T23:45:33.881352+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881927+0000","last_change":"2024-04-26T23:45:31.888514+0000","last_active":"2024-04-26T23:45:33.881927+0000","last_peered":"2024-04-26T23:45:33.881927+0000","last_clean":"2024-04-26T23:45:33.881927+0000","last_became_active":"2024-04-26T23:45:31.887133+0000","last_became_peered":"2024-04-26T23:45:31.887133+0000","last_unstale":"2024-04-26T23:45:33.881927+0000","last_undegraded":"2024-04-26T23:45:33.881927+0000","last_fullsized":"2024-04-26T23:45:33.881927+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:05.288252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895306+0000","last_change":"2024-04-26T23:45:32.894946+0000","last_active":"2024-04-26T23:45:33.895306+0000","last_peered":"2024-04-26T23:45:33.895306+0000","last_clean":"2024-04-26T23:45:33.895306+0000","last_became_active":"2024-04-26T23:45:32.891643+0000","last_became_peered":"2024-04-26T23:45:32.891643+0000","last_unstale":"2024-04-26T23:45:33.895306+0000","last_undegraded":"2024-04-26T23:45:33.895306+0000","last_fullsized":"2024-04-26T23:45:33.895306+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:54.791803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880559+0000","last_change":"2024-04-26T23:45:31.886258+0000","last_active":"2024-04-26T23:45:33.880559+0000","last_peered":"2024-04-26T23:45:33.880559+0000","last_clean":"2024-04-26T23:45:33.880559+0000","last_became_active":"2024-04-26T23:45:31.884112+0000","last_became_peered":"2024-04-26T23:45:31.884112+0000","last_unstale":"2024-04-26T23:45:33.880559+0000","last_undegraded":"2024-04-26T23:45:33.880559+0000","last_fullsized":"2024-04-26T23:45:33.880559+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:05.288252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222110+0000","last_change":"2024-04-26T23:45:32.900605+0000","last_active":"2024-04-26T23:45:34.222110+0000","last_peered":"2024-04-26T23:45:34.222110+0000","last_clean":"2024-04-26T23:45:34.222110+0000","last_became_active":"2024-04-26T23:45:32.900332+0000","last_became_peered":"2024-04-26T23:45:32.900332+0000","last_unstale":"2024-04-26T23:45:34.222110+0000","last_undegraded":"2024-04-26T23:45:34.222110+0000","last_fullsized":"2024-04-26T23:45:34.222110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:10.670319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304385+0000","last_change":"2024-04-26T23:45:31.883629+0000","last_active":"2024-04-26T23:45:34.304385+0000","last_peered":"2024-04-26T23:45:34.304385+0000","last_clean":"2024-04-26T23:45:34.304385+0000","last_became_active":"2024-04-26T23:45:31.883331+0000","last_became_peered":"2024-04-26T23:45:31.883331+0000","last_unstale":"2024-04-26T23:45:34.304385+0000","last_undegraded":"2024-04-26T23:45:34.304385+0000","last_fullsized":"2024-04-26T23:45:34.304385+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877565+0000","last_change":"2024-04-26T23:45:32.909760+0000","last_active":"2024-04-26T23:45:33.877565+0000","last_peered":"2024-04-26T23:45:33.877565+0000","last_clean":"2024-04-26T23:45:33.877565+0000","last_became_active":"2024-04-26T23:45:32.909145+0000","last_became_peered":"2024-04-26T23:45:32.909145+0000","last_unstale":"2024-04-26T23:45:33.877565+0000","last_undegraded":"2024-04-26T23:45:33.877565+0000","last_fullsized":"2024-04-26T23:45:33.877565+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:03.444283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.894720+0000","last_change":"2024-04-26T23:45:31.888291+0000","last_active":"2024-04-26T23:45:33.894720+0000","last_peered":"2024-04-26T23:45:33.894720+0000","last_clean":"2024-04-26T23:45:33.894720+0000","last_became_active":"2024-04-26T23:45:31.887997+0000","last_became_peered":"2024-04-26T23:45:31.887997+0000","last_unstale":"2024-04-26T23:45:33.894720+0000","last_undegraded":"2024-04-26T23:45:33.894720+0000","last_fullsized":"2024-04-26T23:45:33.894720+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:34.548868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304298+0000","last_change":"2024-04-26T23:45:32.898917+0000","last_active":"2024-04-26T23:45:34.304298+0000","last_peered":"2024-04-26T23:45:34.304298+0000","last_clean":"2024-04-26T23:45:34.304298+0000","last_became_active":"2024-04-26T23:45:32.898479+0000","last_became_peered":"2024-04-26T23:45:32.898479+0000","last_unstale":"2024-04-26T23:45:34.304298+0000","last_undegraded":"2024-04-26T23:45:34.304298+0000","last_fullsized":"2024-04-26T23:45:34.304298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:06.682604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881043+0000","last_change":"2024-04-26T23:45:31.891534+0000","last_active":"2024-04-26T23:45:33.881043+0000","last_peered":"2024-04-26T23:45:33.881043+0000","last_clean":"2024-04-26T23:45:33.881043+0000","last_became_active":"2024-04-26T23:45:31.890799+0000","last_became_peered":"2024-04-26T23:45:31.890799+0000","last_unstale":"2024-04-26T23:45:33.881043+0000","last_undegraded":"2024-04-26T23:45:33.881043+0000","last_fullsized":"2024-04-26T23:45:33.881043+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:28.756536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222481+0000","last_change":"2024-04-26T23:45:32.902497+0000","last_active":"2024-04-26T23:45:34.222481+0000","last_peered":"2024-04-26T23:45:34.222481+0000","last_clean":"2024-04-26T23:45:34.222481+0000","last_became_active":"2024-04-26T23:45:32.902207+0000","last_became_peered":"2024-04-26T23:45:32.902207+0000","last_unstale":"2024-04-26T23:45:34.222481+0000","last_undegraded":"2024-04-26T23:45:34.222481+0000","last_fullsized":"2024-04-26T23:45:34.222481+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:08.526634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221476+0000","last_change":"2024-04-26T23:45:31.880250+0000","last_active":"2024-04-26T23:45:34.221476+0000","last_peered":"2024-04-26T23:45:34.221476+0000","last_clean":"2024-04-26T23:45:34.221476+0000","last_became_active":"2024-04-26T23:45:31.879929+0000","last_became_peered":"2024-04-26T23:45:31.879929+0000","last_unstale":"2024-04-26T23:45:34.221476+0000","last_undegraded":"2024-04-26T23:45:34.221476+0000","last_fullsized":"2024-04-26T23:45:34.221476+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:55.835940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877007+0000","last_change":"2024-04-26T23:45:32.897567+0000","last_active":"2024-04-26T23:45:33.877007+0000","last_peered":"2024-04-26T23:45:33.877007+0000","last_clean":"2024-04-26T23:45:33.877007+0000","last_became_active":"2024-04-26T23:45:32.896900+0000","last_became_peered":"2024-04-26T23:45:32.896900+0000","last_unstale":"2024-04-26T23:45:33.877007+0000","last_undegraded":"2024-04-26T23:45:33.877007+0000","last_fullsized":"2024-04-26T23:45:33.877007+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:24.608533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.882044+0000","last_change":"2024-04-26T23:45:31.891676+0000","last_active":"2024-04-26T23:45:33.882044+0000","last_peered":"2024-04-26T23:45:33.882044+0000","last_clean":"2024-04-26T23:45:33.882044+0000","last_became_active":"2024-04-26T23:45:31.890789+0000","last_became_peered":"2024-04-26T23:45:31.890789+0000","last_unstale":"2024-04-26T23:45:33.882044+0000","last_undegraded":"2024-04-26T23:45:33.882044+0000","last_fullsized":"2024-04-26T23:45:33.882044+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:16.612826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880336+0000","last_change":"2024-04-26T23:45:32.893518+0000","last_active":"2024-04-26T23:45:33.880336+0000","last_peered":"2024-04-26T23:45:33.880336+0000","last_clean":"2024-04-26T23:45:33.880336+0000","last_became_active":"2024-04-26T23:45:32.891197+0000","last_became_peered":"2024-04-26T23:45:32.891197+0000","last_unstale":"2024-04-26T23:45:33.880336+0000","last_undegraded":"2024-04-26T23:45:33.880336+0000","last_fullsized":"2024-04-26T23:45:33.880336+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:06.682604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876647+0000","last_change":"2024-04-26T23:45:31.881147+0000","last_active":"2024-04-26T23:45:33.876647+0000","last_peered":"2024-04-26T23:45:33.876647+0000","last_clean":"2024-04-26T23:45:33.876647+0000","last_became_active":"2024-04-26T23:45:31.880853+0000","last_became_peered":"2024-04-26T23:45:31.880853+0000","last_unstale":"2024-04-26T23:45:33.876647+0000","last_undegraded":"2024-04-26T23:45:33.876647+0000","last_fullsized":"2024-04-26T23:45:33.876647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:56.857293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301309+0000","last_change":"2024-04-26T23:45:32.902092+0000","last_active":"2024-04-26T23:45:34.301309+0000","last_peered":"2024-04-26T23:45:34.301309+0000","last_clean":"2024-04-26T23:45:34.301309+0000","last_became_active":"2024-04-26T23:45:32.901136+0000","last_became_peered":"2024-04-26T23:45:32.901136+0000","last_unstale":"2024-04-26T23:45:34.301309+0000","last_undegraded":"2024-04-26T23:45:34.301309+0000","last_fullsized":"2024-04-26T23:45:34.301309+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:59.284795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304525+0000","last_change":"2024-04-26T23:45:31.880787+0000","last_active":"2024-04-26T23:45:34.304525+0000","last_peered":"2024-04-26T23:45:34.304525+0000","last_clean":"2024-04-26T23:45:34.304525+0000","last_became_active":"2024-04-26T23:45:31.880484+0000","last_became_peered":"2024-04-26T23:45:31.880484+0000","last_unstale":"2024-04-26T23:45:34.304525+0000","last_undegraded":"2024-04-26T23:45:34.304525+0000","last_fullsized":"2024-04-26T23:45:34.304525+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:06.980286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880669+0000","last_change":"2024-04-26T23:45:32.902394+0000","last_active":"2024-04-26T23:45:33.880669+0000","last_peered":"2024-04-26T23:45:33.880669+0000","last_clean":"2024-04-26T23:45:33.880669+0000","last_became_active":"2024-04-26T23:45:32.891001+0000","last_became_peered":"2024-04-26T23:45:32.891001+0000","last_unstale":"2024-04-26T23:45:33.880669+0000","last_undegraded":"2024-04-26T23:45:33.880669+0000","last_fullsized":"2024-04-26T23:45:33.880669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:04.742340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303369+0000","last_change":"2024-04-26T23:45:31.878767+0000","last_active":"2024-04-26T23:45:34.303369+0000","last_peered":"2024-04-26T23:45:34.303369+0000","last_clean":"2024-04-26T23:45:34.303369+0000","last_became_active":"2024-04-26T23:45:31.877274+0000","last_became_peered":"2024-04-26T23:45:31.877274+0000","last_unstale":"2024-04-26T23:45:34.303369+0000","last_undegraded":"2024-04-26T23:45:34.303369+0000","last_fullsized":"2024-04-26T23:45:34.303369+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:31.580936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895387+0000","last_change":"2024-04-26T23:45:32.886485+0000","last_active":"2024-04-26T23:45:33.895387+0000","last_peered":"2024-04-26T23:45:33.895387+0000","last_clean":"2024-04-26T23:45:33.895387+0000","last_became_active":"2024-04-26T23:45:32.886095+0000","last_became_peered":"2024-04-26T23:45:32.886095+0000","last_unstale":"2024-04-26T23:45:33.895387+0000","last_undegraded":"2024-04-26T23:45:33.895387+0000","last_fullsized":"2024-04-26T23:45:33.895387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:22.732038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880729+0000","last_change":"2024-04-26T23:45:31.885731+0000","last_active":"2024-04-26T23:45:33.880729+0000","last_peered":"2024-04-26T23:45:33.880729+0000","last_clean":"2024-04-26T23:45:33.880729+0000","last_became_active":"2024-04-26T23:45:31.884231+0000","last_became_peered":"2024-04-26T23:45:31.884231+0000","last_unstale":"2024-04-26T23:45:33.880729+0000","last_undegraded":"2024-04-26T23:45:33.880729+0000","last_fullsized":"2024-04-26T23:45:33.880729+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:16.612826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304195+0000","last_change":"2024-04-26T23:45:32.891589+0000","last_active":"2024-04-26T23:45:34.304195+0000","last_peered":"2024-04-26T23:45:34.304195+0000","last_clean":"2024-04-26T23:45:34.304195+0000","last_became_active":"2024-04-26T23:45:32.891305+0000","last_became_peered":"2024-04-26T23:45:32.891305+0000","last_unstale":"2024-04-26T23:45:34.304195+0000","last_undegraded":"2024-04-26T23:45:34.304195+0000","last_fullsized":"2024-04-26T23:45:34.304195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221622+0000","last_change":"2024-04-26T23:45:31.873384+0000","last_active":"2024-04-26T23:45:34.221622+0000","last_peered":"2024-04-26T23:45:34.221622+0000","last_clean":"2024-04-26T23:45:34.221622+0000","last_became_active":"2024-04-26T23:45:31.873051+0000","last_became_peered":"2024-04-26T23:45:31.873051+0000","last_unstale":"2024-04-26T23:45:34.221622+0000","last_undegraded":"2024-04-26T23:45:34.221622+0000","last_fullsized":"2024-04-26T23:45:34.221622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:42.514140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301723+0000","last_change":"2024-04-26T23:45:32.891702+0000","last_active":"2024-04-26T23:45:34.301723+0000","last_peered":"2024-04-26T23:45:34.301723+0000","last_clean":"2024-04-26T23:45:34.301723+0000","last_became_active":"2024-04-26T23:45:32.891381+0000","last_became_peered":"2024-04-26T23:45:32.891381+0000","last_unstale":"2024-04-26T23:45:34.301723+0000","last_undegraded":"2024-04-26T23:45:34.301723+0000","last_fullsized":"2024-04-26T23:45:34.301723+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:37.701650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301002+0000","last_change":"2024-04-26T23:45:31.886925+0000","last_active":"2024-04-26T23:45:34.301002+0000","last_peered":"2024-04-26T23:45:34.301002+0000","last_clean":"2024-04-26T23:45:34.301002+0000","last_became_active":"2024-04-26T23:45:31.886582+0000","last_became_peered":"2024-04-26T23:45:31.886582+0000","last_unstale":"2024-04-26T23:45:34.301002+0000","last_undegraded":"2024-04-26T23:45:34.301002+0000","last_fullsized":"2024-04-26T23:45:34.301002+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:46.533241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876689+0000","last_change":"2024-04-26T23:45:32.897716+0000","last_active":"2024-04-26T23:45:33.876689+0000","last_peered":"2024-04-26T23:45:33.876689+0000","last_clean":"2024-04-26T23:45:33.876689+0000","last_became_active":"2024-04-26T23:45:32.888321+0000","last_became_peered":"2024-04-26T23:45:32.888321+0000","last_unstale":"2024-04-26T23:45:33.876689+0000","last_undegraded":"2024-04-26T23:45:33.876689+0000","last_fullsized":"2024-04-26T23:45:33.876689+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:10.438148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":22,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880501+0000","last_change":"2024-04-26T23:45:31.882816+0000","last_active":"2024-04-26T23:45:33.880501+0000","last_peered":"2024-04-26T23:45:33.880501+0000","last_clean":"2024-04-26T23:45:33.880501+0000","last_became_active":"2024-04-26T23:45:31.882476+0000","last_became_peered":"2024-04-26T23:45:31.882476+0000","last_unstale":"2024-04-26T23:45:33.880501+0000","last_undegraded":"2024-04-26T23:45:33.880501+0000","last_fullsized":"2024-04-26T23:45:33.880501+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:49.776017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301197+0000","last_change":"2024-04-26T23:45:32.912788+0000","last_active":"2024-04-26T23:45:34.301197+0000","last_peered":"2024-04-26T23:45:34.301197+0000","last_clean":"2024-04-26T23:45:34.301197+0000","last_became_active":"2024-04-26T23:45:32.910911+0000","last_became_peered":"2024-04-26T23:45:32.910911+0000","last_unstale":"2024-04-26T23:45:34.301197+0000","last_undegraded":"2024-04-26T23:45:34.301197+0000","last_fullsized":"2024-04-26T23:45:34.301197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:21.631715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304006+0000","last_change":"2024-04-26T23:45:31.879138+0000","last_active":"2024-04-26T23:45:34.304006+0000","last_peered":"2024-04-26T23:45:34.304006+0000","last_clean":"2024-04-26T23:45:34.304006+0000","last_became_active":"2024-04-26T23:45:31.878751+0000","last_became_peered":"2024-04-26T23:45:31.878751+0000","last_unstale":"2024-04-26T23:45:34.304006+0000","last_undegraded":"2024-04-26T23:45:34.304006+0000","last_fullsized":"2024-04-26T23:45:34.304006+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:48.251318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304437+0000","last_change":"2024-04-26T23:45:32.910970+0000","last_active":"2024-04-26T23:45:34.304437+0000","last_peered":"2024-04-26T23:45:34.304437+0000","last_clean":"2024-04-26T23:45:34.304437+0000","last_became_active":"2024-04-26T23:45:32.910721+0000","last_became_peered":"2024-04-26T23:45:32.910721+0000","last_unstale":"2024-04-26T23:45:34.304437+0000","last_undegraded":"2024-04-26T23:45:34.304437+0000","last_fullsized":"2024-04-26T23:45:34.304437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:35.565206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.304792+0000","last_change":"2024-04-26T23:45:31.889679+0000","last_active":"2024-04-26T23:45:34.304792+0000","last_peered":"2024-04-26T23:45:34.304792+0000","last_clean":"2024-04-26T23:45:34.304792+0000","last_became_active":"2024-04-26T23:45:31.889351+0000","last_became_peered":"2024-04-26T23:45:31.889351+0000","last_unstale":"2024-04-26T23:45:34.304792+0000","last_undegraded":"2024-04-26T23:45:34.304792+0000","last_fullsized":"2024-04-26T23:45:34.304792+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:01.214289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880805+0000","last_change":"2024-04-26T23:45:32.893279+0000","last_active":"2024-04-26T23:45:33.880805+0000","last_peered":"2024-04-26T23:45:33.880805+0000","last_clean":"2024-04-26T23:45:33.880805+0000","last_became_active":"2024-04-26T23:45:32.891083+0000","last_became_peered":"2024-04-26T23:45:32.891083+0000","last_unstale":"2024-04-26T23:45:33.880805+0000","last_undegraded":"2024-04-26T23:45:33.880805+0000","last_fullsized":"2024-04-26T23:45:33.880805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:27.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895177+0000","last_change":"2024-04-26T23:45:31.888365+0000","last_active":"2024-04-26T23:45:33.895177+0000","last_peered":"2024-04-26T23:45:33.895177+0000","last_clean":"2024-04-26T23:45:33.895177+0000","last_became_active":"2024-04-26T23:45:31.887719+0000","last_became_peered":"2024-04-26T23:45:31.887719+0000","last_unstale":"2024-04-26T23:45:33.895177+0000","last_undegraded":"2024-04-26T23:45:33.895177+0000","last_fullsized":"2024-04-26T23:45:33.895177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:26.656816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877300+0000","last_change":"2024-04-26T23:45:32.888898+0000","last_active":"2024-04-26T23:45:33.877300+0000","last_peered":"2024-04-26T23:45:33.877300+0000","last_clean":"2024-04-26T23:45:33.877300+0000","last_became_active":"2024-04-26T23:45:32.888556+0000","last_became_peered":"2024-04-26T23:45:32.888556+0000","last_unstale":"2024-04-26T23:45:33.877300+0000","last_undegraded":"2024-04-26T23:45:33.877300+0000","last_fullsized":"2024-04-26T23:45:33.877300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:09.557437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301946+0000","last_change":"2024-04-26T23:45:31.890213+0000","last_active":"2024-04-26T23:45:34.301946+0000","last_peered":"2024-04-26T23:45:34.301946+0000","last_clean":"2024-04-26T23:45:34.301946+0000","last_became_active":"2024-04-26T23:45:31.889263+0000","last_became_peered":"2024-04-26T23:45:31.889263+0000","last_unstale":"2024-04-26T23:45:34.301946+0000","last_undegraded":"2024-04-26T23:45:34.301946+0000","last_fullsized":"2024-04-26T23:45:34.301946+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:32.653816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.877577+0000","last_change":"2024-04-26T23:45:32.909517+0000","last_active":"2024-04-26T23:45:33.877577+0000","last_peered":"2024-04-26T23:45:33.877577+0000","last_clean":"2024-04-26T23:45:33.877577+0000","last_became_active":"2024-04-26T23:45:32.903973+0000","last_became_peered":"2024-04-26T23:45:32.903973+0000","last_unstale":"2024-04-26T23:45:33.877577+0000","last_undegraded":"2024-04-26T23:45:33.877577+0000","last_fullsized":"2024-04-26T23:45:33.877577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:51.472436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300814+0000","last_change":"2024-04-26T23:45:31.886802+0000","last_active":"2024-04-26T23:45:34.300814+0000","last_peered":"2024-04-26T23:45:34.300814+0000","last_clean":"2024-04-26T23:45:34.300814+0000","last_became_active":"2024-04-26T23:45:31.885469+0000","last_became_peered":"2024-04-26T23:45:31.885469+0000","last_unstale":"2024-04-26T23:45:34.300814+0000","last_undegraded":"2024-04-26T23:45:34.300814+0000","last_fullsized":"2024-04-26T23:45:34.300814+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:08.458279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.222003+0000","last_change":"2024-04-26T23:45:32.897460+0000","last_active":"2024-04-26T23:45:34.222003+0000","last_peered":"2024-04-26T23:45:34.222003+0000","last_clean":"2024-04-26T23:45:34.222003+0000","last_became_active":"2024-04-26T23:45:32.896744+0000","last_became_peered":"2024-04-26T23:45:32.896744+0000","last_unstale":"2024-04-26T23:45:34.222003+0000","last_undegraded":"2024-04-26T23:45:34.222003+0000","last_fullsized":"2024-04-26T23:45:34.222003+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:22.732038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303370+0000","last_change":"2024-04-26T23:45:31.878549+0000","last_active":"2024-04-26T23:45:34.303370+0000","last_peered":"2024-04-26T23:45:34.303370+0000","last_clean":"2024-04-26T23:45:34.303370+0000","last_became_active":"2024-04-26T23:45:31.878260+0000","last_became_peered":"2024-04-26T23:45:31.878260+0000","last_unstale":"2024-04-26T23:45:34.303370+0000","last_undegraded":"2024-04-26T23:45:34.303370+0000","last_fullsized":"2024-04-26T23:45:34.303370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:01.011964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301596+0000","last_change":"2024-04-26T23:45:32.896520+0000","last_active":"2024-04-26T23:45:34.301596+0000","last_peered":"2024-04-26T23:45:34.301596+0000","last_clean":"2024-04-26T23:45:34.301596+0000","last_became_active":"2024-04-26T23:45:32.896191+0000","last_became_peered":"2024-04-26T23:45:32.896191+0000","last_unstale":"2024-04-26T23:45:34.301596+0000","last_undegraded":"2024-04-26T23:45:34.301596+0000","last_fullsized":"2024-04-26T23:45:34.301596+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:23.995760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301108+0000","last_change":"2024-04-26T23:45:31.890747+0000","last_active":"2024-04-26T23:45:34.301108+0000","last_peered":"2024-04-26T23:45:34.301108+0000","last_clean":"2024-04-26T23:45:34.301108+0000","last_became_active":"2024-04-26T23:45:31.890454+0000","last_became_peered":"2024-04-26T23:45:31.890454+0000","last_unstale":"2024-04-26T23:45:34.301108+0000","last_undegraded":"2024-04-26T23:45:34.301108+0000","last_fullsized":"2024-04-26T23:45:34.301108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:12.324313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876562+0000","last_change":"2024-04-26T23:45:32.891583+0000","last_active":"2024-04-26T23:45:33.876562+0000","last_peered":"2024-04-26T23:45:33.876562+0000","last_clean":"2024-04-26T23:45:33.876562+0000","last_became_active":"2024-04-26T23:45:32.889398+0000","last_became_peered":"2024-04-26T23:45:32.889398+0000","last_unstale":"2024-04-26T23:45:33.876562+0000","last_undegraded":"2024-04-26T23:45:33.876562+0000","last_fullsized":"2024-04-26T23:45:33.876562+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:09.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.881082+0000","last_change":"2024-04-26T23:45:31.886764+0000","last_active":"2024-04-26T23:45:33.881082+0000","last_peered":"2024-04-26T23:45:33.881082+0000","last_clean":"2024-04-26T23:45:33.881082+0000","last_became_active":"2024-04-26T23:45:31.884462+0000","last_became_peered":"2024-04-26T23:45:31.884462+0000","last_unstale":"2024-04-26T23:45:33.881082+0000","last_undegraded":"2024-04-26T23:45:33.881082+0000","last_fullsized":"2024-04-26T23:45:33.881082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:49.776017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880890+0000","last_change":"2024-04-26T23:45:32.888133+0000","last_active":"2024-04-26T23:45:33.880890+0000","last_peered":"2024-04-26T23:45:33.880890+0000","last_clean":"2024-04-26T23:45:33.880890+0000","last_became_active":"2024-04-26T23:45:32.887839+0000","last_became_peered":"2024-04-26T23:45:32.887839+0000","last_unstale":"2024-04-26T23:45:33.880890+0000","last_undegraded":"2024-04-26T23:45:33.880890+0000","last_fullsized":"2024-04-26T23:45:33.880890+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:21.806073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.876756+0000","last_change":"2024-04-26T23:45:31.885140+0000","last_active":"2024-04-26T23:45:33.876756+0000","last_peered":"2024-04-26T23:45:33.876756+0000","last_clean":"2024-04-26T23:45:33.876756+0000","last_became_active":"2024-04-26T23:45:31.884819+0000","last_became_peered":"2024-04-26T23:45:31.884819+0000","last_unstale":"2024-04-26T23:45:33.876756+0000","last_undegraded":"2024-04-26T23:45:33.876756+0000","last_fullsized":"2024-04-26T23:45:33.876756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:15.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.221786+0000","last_change":"2024-04-26T23:45:32.896746+0000","last_active":"2024-04-26T23:45:34.221786+0000","last_peered":"2024-04-26T23:45:34.221786+0000","last_clean":"2024-04-26T23:45:34.221786+0000","last_became_active":"2024-04-26T23:45:32.894779+0000","last_became_peered":"2024-04-26T23:45:32.894779+0000","last_unstale":"2024-04-26T23:45:34.221786+0000","last_undegraded":"2024-04-26T23:45:34.221786+0000","last_fullsized":"2024-04-26T23:45:34.221786+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:54.791803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303998+0000","last_change":"2024-04-26T23:45:31.878288+0000","last_active":"2024-04-26T23:45:34.303998+0000","last_peered":"2024-04-26T23:45:34.303998+0000","last_clean":"2024-04-26T23:45:34.303998+0000","last_became_active":"2024-04-26T23:45:31.876834+0000","last_became_peered":"2024-04-26T23:45:31.876834+0000","last_unstale":"2024-04-26T23:45:34.303998+0000","last_undegraded":"2024-04-26T23:45:34.303998+0000","last_fullsized":"2024-04-26T23:45:34.303998+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:03.669792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895230+0000","last_change":"2024-04-26T23:45:32.896803+0000","last_active":"2024-04-26T23:45:33.895230+0000","last_peered":"2024-04-26T23:45:33.895230+0000","last_clean":"2024-04-26T23:45:33.895230+0000","last_became_active":"2024-04-26T23:45:32.895374+0000","last_became_peered":"2024-04-26T23:45:32.895374+0000","last_unstale":"2024-04-26T23:45:33.895230+0000","last_undegraded":"2024-04-26T23:45:33.895230+0000","last_fullsized":"2024-04-26T23:45:33.895230+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:02.230627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880592+0000","last_change":"2024-04-26T23:45:31.883746+0000","last_active":"2024-04-26T23:45:33.880592+0000","last_peered":"2024-04-26T23:45:33.880592+0000","last_clean":"2024-04-26T23:45:33.880592+0000","last_became_active":"2024-04-26T23:45:31.882969+0000","last_became_peered":"2024-04-26T23:45:31.882969+0000","last_unstale":"2024-04-26T23:45:33.880592+0000","last_undegraded":"2024-04-26T23:45:33.880592+0000","last_fullsized":"2024-04-26T23:45:33.880592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:25.870271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301344+0000","last_change":"2024-04-26T23:45:32.901524+0000","last_active":"2024-04-26T23:45:34.301344+0000","last_peered":"2024-04-26T23:45:34.301344+0000","last_clean":"2024-04-26T23:45:34.301344+0000","last_became_active":"2024-04-26T23:45:32.901182+0000","last_became_peered":"2024-04-26T23:45:32.901182+0000","last_unstale":"2024-04-26T23:45:34.301344+0000","last_undegraded":"2024-04-26T23:45:34.301344+0000","last_fullsized":"2024-04-26T23:45:34.301344+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.575166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.880775+0000","last_change":"2024-04-26T23:45:31.883532+0000","last_active":"2024-04-26T23:45:33.880775+0000","last_peered":"2024-04-26T23:45:33.880775+0000","last_clean":"2024-04-26T23:45:33.880775+0000","last_became_active":"2024-04-26T23:45:31.883090+0000","last_became_peered":"2024-04-26T23:45:31.883090+0000","last_unstale":"2024-04-26T23:45:33.880775+0000","last_undegraded":"2024-04-26T23:45:33.880775+0000","last_fullsized":"2024-04-26T23:45:33.880775+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:25.870271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301523+0000","last_change":"2024-04-26T23:45:32.897526+0000","last_active":"2024-04-26T23:45:34.301523+0000","last_peered":"2024-04-26T23:45:34.301523+0000","last_clean":"2024-04-26T23:45:34.301523+0000","last_became_active":"2024-04-26T23:45:32.897234+0000","last_became_peered":"2024-04-26T23:45:32.897234+0000","last_unstale":"2024-04-26T23:45:34.301523+0000","last_undegraded":"2024-04-26T23:45:34.301523+0000","last_fullsized":"2024-04-26T23:45:34.301523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:21.458052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.300998+0000","last_change":"2024-04-26T23:45:31.881587+0000","last_active":"2024-04-26T23:45:34.300998+0000","last_peered":"2024-04-26T23:45:34.300998+0000","last_clean":"2024-04-26T23:45:34.300998+0000","last_became_active":"2024-04-26T23:45:31.881296+0000","last_became_peered":"2024-04-26T23:45:31.881296+0000","last_unstale":"2024-04-26T23:45:34.300998+0000","last_undegraded":"2024-04-26T23:45:34.300998+0000","last_fullsized":"2024-04-26T23:45:34.300998+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:54:08.315315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.305117+0000","last_change":"2024-04-26T23:45:32.890084+0000","last_active":"2024-04-26T23:45:34.305117+0000","last_peered":"2024-04-26T23:45:34.305117+0000","last_clean":"2024-04-26T23:45:34.305117+0000","last_became_active":"2024-04-26T23:45:32.889450+0000","last_became_peered":"2024-04-26T23:45:32.889450+0000","last_unstale":"2024-04-26T23:45:34.305117+0000","last_undegraded":"2024-04-26T23:45:34.305117+0000","last_fullsized":"2024-04-26T23:45:34.305117+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:06.464920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.303475+0000","last_change":"2024-04-26T23:45:31.874536+0000","last_active":"2024-04-26T23:45:34.303475+0000","last_peered":"2024-04-26T23:45:34.303475+0000","last_clean":"2024-04-26T23:45:34.303475+0000","last_became_active":"2024-04-26T23:45:31.874213+0000","last_became_peered":"2024-04-26T23:45:31.874213+0000","last_unstale":"2024-04-26T23:45:34.303475+0000","last_undegraded":"2024-04-26T23:45:34.303475+0000","last_fullsized":"2024-04-26T23:45:34.303475+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:22.163710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.895676+0000","last_change":"2024-04-26T23:45:32.880264+0000","last_active":"2024-04-26T23:45:33.895676+0000","last_peered":"2024-04-26T23:45:33.895676+0000","last_clean":"2024-04-26T23:45:33.895676+0000","last_became_active":"2024-04-26T23:45:32.879965+0000","last_became_peered":"2024-04-26T23:45:32.879965+0000","last_unstale":"2024-04-26T23:45:33.895676+0000","last_undegraded":"2024-04-26T23:45:33.895676+0000","last_fullsized":"2024-04-26T23:45:33.895676+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:10.670319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301255+0000","last_change":"2024-04-26T23:45:31.882674+0000","last_active":"2024-04-26T23:45:34.301255+0000","last_peered":"2024-04-26T23:45:34.301255+0000","last_clean":"2024-04-26T23:45:34.301255+0000","last_became_active":"2024-04-26T23:45:31.881987+0000","last_became_peered":"2024-04-26T23:45:31.881987+0000","last_unstale":"2024-04-26T23:45:34.301255+0000","last_undegraded":"2024-04-26T23:45:34.301255+0000","last_fullsized":"2024-04-26T23:45:34.301255+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:38.140097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:33.894663+0000","last_change":"2024-04-26T23:45:32.886402+0000","last_active":"2024-04-26T23:45:33.894663+0000","last_peered":"2024-04-26T23:45:33.894663+0000","last_clean":"2024-04-26T23:45:33.894663+0000","last_became_active":"2024-04-26T23:45:32.886141+0000","last_became_peered":"2024-04-26T23:45:32.886141+0000","last_unstale":"2024-04-26T23:45:33.894663+0000","last_undegraded":"2024-04-26T23:45:33.894663+0000","last_fullsized":"2024-04-26T23:45:33.894663+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:31.834569+0000","last_clean_scrub_stamp":"2024-04-26T23:45:31.834569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:20.737661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301742+0000","last_change":"2024-04-26T23:45:31.886926+0000","last_active":"2024-04-26T23:45:34.301742+0000","last_peered":"2024-04-26T23:45:34.301742+0000","last_clean":"2024-04-26T23:45:34.301742+0000","last_became_active":"2024-04-26T23:45:31.886626+0000","last_became_peered":"2024-04-26T23:45:31.886626+0000","last_unstale":"2024-04-26T23:45:34.301742+0000","last_undegraded":"2024-04-26T23:45:34.301742+0000","last_fullsized":"2024-04-26T23:45:34.301742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:45:30.818231+0000","last_clean_scrub_stamp":"2024-04-26T23:45:30.818231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:18.057584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":334,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-26T23:45:34.301943+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:45:34.301943+0000","last_peered":"2024-04-26T23:45:34.301943+0000","last_clean":"2024-04-26T23:45:34.301943+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:45:34.301943+0000","last_undegraded":"2024-04-26T23:45:34.301943+0000","last_fullsized":"2024-04-26T23:45:34.301943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":1235035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40868,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330972,"statfs":{"total":96636764160,"available":96594915328,"internally_reserved":0,"allocated":1396736,"data_stored":646723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":644507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1384448,"data_stored":644599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":645089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1380352,"data_stored":644541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1380352,"data_stored":644541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45604,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326236,"statfs":{"total":96636764160,"available":96590065664,"internally_reserved":0,"allocated":1986560,"data_stored":1236577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:45:37.932 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:45:37.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:45:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.194+0000 7f216d600640 1 Processor -- start 2024-04-26T23:45:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.194+0000 7f216d600640 1 -- start start 2024-04-26T23:45:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f216d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f216806ca60 0x7f216816a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f216d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21680cccf0 con 0x7f216815f300 2024-04-26T23:45:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f216d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21680aea20 con 0x7f216815ef20 2024-04-26T23:45:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f216d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f21680ccfd0 con 0x7f216806ca60 2024-04-26T23:45:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f2167400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f216815f300 0x7f216806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48858/0 (socket says 172.21.15.46:48858) 2024-04-26T23:45:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f2167400640 1 -- 172.21.15.46:0/3922603426 learned_addr learned my addr 172.21.15.46:0/3922603426 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.196+0000 7f216cc00640 1 --2- 172.21.15.46:0/3922603426 >> v2:172.21.15.142:3300/0 conn(0x7f216806ca60 0x7f216816a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935701224 0 0) 0x7f21680cccf0 con 0x7f216815f300 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f214c003520 con 0x7f216815f300 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3677022727 0 0) 0x7f214c003520 con 0x7f216815f300 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 >> v2:172.21.15.142:3300/0 conn(0x7f216806ca60 msgr2=0x7f216816a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 --2- 172.21.15.46:0/3922603426 >> v2:172.21.15.142:3300/0 conn(0x7f216806ca60 0x7f216816a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 >> v1:172.21.15.142:6789/0 conn(0x7f216815ef20 legacy=0x7f21680cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680aed00 con 0x7f216815f300 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2150001670 con 0x7f216815f300 2024-04-26T23:45:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 >> v1:172.21.15.46:6789/0 conn(0x7f216815f300 legacy=0x7f216806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c0039b0 0x7f214c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 0x7f214c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f214c0051c0 0x7f214c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21680ccfd0 con 0x7f214c004800 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f214c00c160 con 0x7f214c0051c0 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f214c00c360 con 0x7f214c0039b0 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.197+0000 7f2167400640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 0x7f214c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f216cc00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f214c0051c0 0x7f214c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167e00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c0039b0 0x7f214c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167400640 1 -- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c0039b0 msgr2=0x7f214c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167400640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c0039b0 0x7f214c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167400640 1 -- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f214c0051c0 msgr2=0x7f214c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167400640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f214c0051c0 0x7f214c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167400640 1 -- 172.21.15.46:0/3922603426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c003520 con 0x7f214c004800 2024-04-26T23:45:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f2167e00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c0039b0 0x7f214c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f216cc00640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f214c0051c0 0x7f214c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:38.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.198+0000 7f216cc00640 1 -- 172.21.15.46:0/3922603426 reap_dead start 2024-04-26T23:45:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.199+0000 7f2167400640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 0x7f214c004c80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f215000c210 tx=0x7f214c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a4e2c2b9b5122f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.199+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150007d10 con 0x7f214c004800 2024-04-26T23:45:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.199+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21500058d0 con 0x7f214c004800 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.199+0000 7f2166a00640 1 -- 172.21.15.46:0/3922603426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150005d60 con 0x7f214c004800 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 -- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 msgr2=0x7f214c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 0x7f214c004c80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f215000c210 tx=0x7f214c005830 comp rx=0 tx=0).stop 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 -- 172.21.15.46:0/3922603426 shutdown_connections 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 --2- 172.21.15.46:0/3922603426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214c004800 0x7f214c004c80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 -- 172.21.15.46:0/3922603426 >> 172.21.15.46:0/3922603426 conn(0x7f21680d1c50 msgr2=0x7f21680d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 -- 172.21.15.46:0/3922603426 shutdown_connections 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.200+0000 7f216d600640 1 -- 172.21.15.46:0/3922603426 wait complete. 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.202+0000 7f216d600640 1 Processor -- start 2024-04-26T23:45:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.202+0000 7f216d600640 1 -- start start 2024-04-26T23:45:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c004800 0x7f216815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 0x7f21680c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2168163cb0 con 0x7f216815bf00 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2168163b30 con 0x7f21680c6fe0 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f216806c350 con 0x7f214c004800 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.203+0000 7f216cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48834/0 (socket says 172.21.15.46:48834) 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 -- 172.21.15.46:0/2602731595 learned_addr learned my addr 172.21.15.46:0/2602731595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f2167400640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 0x7f21680c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f2167e00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c004800 0x7f216815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 -- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c004800 msgr2=0x7f216815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c004800 0x7f216815b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 -- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 msgr2=0x7f21680c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 0x7f21680c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 -- 172.21.15.46:0/2602731595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216815bd10 con 0x7f216815bf00 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f2167400640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 0x7f21680c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.204+0000 7f216cc00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2160010790 tx=0x7f2160010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.205+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000f040 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.205+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216001b070 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.205+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160004430 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.205+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21680c5cc0 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.205+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21680c6220 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.206+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2168069ac0 con 0x7f216815bf00 2024-04-26T23:45:38.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.207+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f216000e670 con 0x7f216815bf00 2024-04-26T23:45:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.208+0000 7f215fe00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 0x7f214406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.208+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f216008bd50 con 0x7f216815bf00 2024-04-26T23:45:38.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.209+0000 7f2167e00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 0x7f214406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.210+0000 7f2167e00640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 0x7f214406edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f216806b1d0 tx=0x7f2158009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.213+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21680c5cc0 con 0x7f216815bf00 2024-04-26T23:45:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.501+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f21680d2ac0 con 0x7f216815bf00 2024-04-26T23:45:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.505+0000 7f215fe00640 1 -- 172.21.15.46:0/2602731595 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f2160059ec0 con 0x7f216815bf00 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:45:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 msgr2=0x7f214406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 0x7f214406edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f216806b1d0 tx=0x7f2158009040 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 msgr2=0x7f21680c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2160010790 tx=0x7f2160010c50 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 shutdown_connections 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214406c8f0 0x7f214406edb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216815bf00 0x7f21680c7b20 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680c6fe0 0x7f21680c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 --2- 172.21.15.46:0/2602731595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f214c004800 0x7f216815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 >> 172.21.15.46:0/2602731595 conn(0x7f21680d1c50 msgr2=0x7f216815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 shutdown_connections 2024-04-26T23:45:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.508+0000 7f216d600640 1 -- 172.21.15.46:0/2602731595 wait complete. 2024-04-26T23:45:38.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:45:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.740+0000 7f45b5200640 1 Processor -- start 2024-04-26T23:45:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.741+0000 7f45b5200640 1 -- start start 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.742+0000 7f45b5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 0x7f45b00cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.742+0000 7f45b5200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.742+0000 7f45b5200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b014c6c0 con 0x7f45b014c2e0 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.742+0000 7f45b5200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f45b00cde60 con 0x7f45b00cd9f0 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.742+0000 7f45b5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45b00cb8b0 con 0x7f45b00ccdd0 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.743+0000 7f45afe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.743+0000 7f45a6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 0x7f45b00cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.743+0000 7f45afe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:48836/0 (socket says 172.21.15.46:48836) 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.743+0000 7f45afe00640 1 -- 172.21.15.46:0/1963294304 learned_addr learned my addr 172.21.15.46:0/1963294304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.744+0000 7f45a6a00640 1 -- 172.21.15.46:0/1963294304 >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 msgr2=0x7f45b00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.744+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1963294304 >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.744+0000 7f45a6a00640 1 -- 172.21.15.46:0/1963294304 >> v1:172.21.15.142:6789/0 conn(0x7f45b014c2e0 legacy=0x7f45b00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.744+0000 7f45a6a00640 1 -- 172.21.15.46:0/1963294304 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b0167e10 con 0x7f45b00ccdd0 2024-04-26T23:45:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.744+0000 7f45afe00640 1 --2- 172.21.15.46:0/1963294304 >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:45:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.745+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1963294304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 0x7f45b00cd4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45980097a0 tx=0x7f459802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51be4cc1824a8ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45aea00640 1 -- 172.21.15.46:0/1963294304 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598037070 con 0x7f45b00ccdd0 2024-04-26T23:45:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45aea00640 1 -- 172.21.15.46:0/1963294304 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f459802ae40 con 0x7f45b00ccdd0 2024-04-26T23:45:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45aea00640 1 -- 172.21.15.46:0/1963294304 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598032660 con 0x7f45b00ccdd0 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 -- 172.21.15.46:0/1963294304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 msgr2=0x7f45b00cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 --2- 172.21.15.46:0/1963294304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 0x7f45b00cd4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45980097a0 tx=0x7f459802a0f0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 -- 172.21.15.46:0/1963294304 shutdown_connections 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 --2- 172.21.15.46:0/1963294304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00ccdd0 0x7f45b00cd4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 --2- 172.21.15.46:0/1963294304 >> v2:172.21.15.142:3300/0 conn(0x7f45b00cd9f0 0x7f45b00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.746+0000 7f45b5200640 1 -- 172.21.15.46:0/1963294304 >> 172.21.15.46:0/1963294304 conn(0x7f45b00b26d0 msgr2=0x7f45b00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.747+0000 7f45b5200640 1 -- 172.21.15.46:0/1963294304 shutdown_connections 2024-04-26T23:45:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.747+0000 7f45b5200640 1 -- 172.21.15.46:0/1963294304 wait complete. 2024-04-26T23:45:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.748+0000 7f45b5200640 1 Processor -- start 2024-04-26T23:45:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.748+0000 7f45b5200640 1 -- start start 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00cd9f0 0x7f45b00c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b014c2e0 0x7f45b00ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0167610 con 0x7f45b00ccdd0 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0167330 con 0x7f45b014c2e0 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.749+0000 7f45b5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f45b01678f0 con 0x7f45b00cd9f0 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48844/0 (socket says 172.21.15.46:48844) 2024-04-26T23:45:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 -- 172.21.15.46:0/1345146971 learned_addr learned my addr 172.21.15.46:0/1345146971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00cd9f0 0x7f45b00c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45afe00640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b014c2e0 0x7f45b00ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 -- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00cd9f0 msgr2=0x7f45b00c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00cd9f0 0x7f45b00c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 -- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b014c2e0 msgr2=0x7f45b00ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b014c2e0 0x7f45b00ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.750+0000 7f45af400640 1 -- 172.21.15.46:0/1345146971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b00b3b70 con 0x7f45b00ccdd0 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45af400640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f45a000c450 tx=0x7f45a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00052d0 con 0x7f45b00ccdd0 2024-04-26T23:45:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0005470 con 0x7f45b00ccdd0 2024-04-26T23:45:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0012870 con 0x7f45b00ccdd0 2024-04-26T23:45:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b00b3e50 con 0x7f45b00ccdd0 2024-04-26T23:45:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.751+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b00b4410 con 0x7f45b00ccdd0 2024-04-26T23:45:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.753+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b00693b0 con 0x7f45b00ccdd0 2024-04-26T23:45:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.753+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45a0012a10 con 0x7f45b00ccdd0 2024-04-26T23:45:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.755+0000 7f45a7e00640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.755+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f45a008b920 con 0x7f45b00ccdd0 2024-04-26T23:45:38.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.756+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.757+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 0x7f458c06edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4598034000 tx=0x7f459802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:38.760+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45a0059bb0 con 0x7f45b00ccdd0 2024-04-26T23:45:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.004+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45b00b49c0 con 0x7f45b00ccdd0 2024-04-26T23:45:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.006+0000 7f45a7e00640 1 -- 172.21.15.46:0/1345146971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5183 (secure 0 0 0) 0x7f45b00b49c0 con 0x7f45b00ccdd0 2024-04-26T23:45:39.010 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:45:39.010 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:35.404532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:45:39.011 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 90 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 msgr2=0x7f458c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 0x7f458c06edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4598034000 tx=0x7f459802e000 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 msgr2=0x7f45b0147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f45a000c450 tx=0x7f45a000c910 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 shutdown_connections 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b014c2e0 0x7f45b00ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45b00cd9f0 0x7f45b00c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 --2- 172.21.15.46:0/1345146971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00ccdd0 0x7f45b0147d30 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 >> 172.21.15.46:0/1345146971 conn(0x7f45b00b26d0 msgr2=0x7f45b006f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.008+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 shutdown_connections 2024-04-26T23:45:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.009+0000 7f45b5200640 1 -- 172.21.15.46:0/1345146971 wait complete. 2024-04-26T23:45:39.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:45:32.336330+0000', 'modified': '2024-04-26T23:45:35.404532+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8253': {'gid': 8253, 'name': 'b', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2078483956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2078483956}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2078483956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8253, 'qdb_cluster': [8253]} 2024-04-26T23:45:39.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:45:39.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:45:39.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.647+0000 7fb2e1a00640 1 Processor -- start 2024-04-26T23:45:39.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.648+0000 7fb2e1a00640 1 -- start start 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.649+0000 7fb2e1a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2dc145770 0x7fb2dc0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.649+0000 7fb2e1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2dc072e60 0x7fb2dc0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.649+0000 7fb2e1a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2dc0cf950 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.649+0000 7fb2e1a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb2dc0d0c50 con 0x7fb2dc072e60 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.649+0000 7fb2e1a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb2dc15d490 con 0x7fb2dc145770 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2db400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb2dc15c5c0 0x7fb2dc0b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48866/0 (socket says 172.21.15.46:48866) 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2dc072e60 0x7fb2dc0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2db400640 1 -- 172.21.15.46:0/2769513474 learned_addr learned my addr 172.21.15.46:0/2769513474 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:39.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2daa00640 1 --2- 172.21.15.46:0/2769513474 >> v2:172.21.15.142:3300/0 conn(0x7fb2dc145770 0x7fb2dc0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2d3e00640 1 -- 172.21.15.46:0/2769513474 >> v2:172.21.15.142:3300/0 conn(0x7fb2dc145770 msgr2=0x7fb2dc0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/2769513474 >> v2:172.21.15.142:3300/0 conn(0x7fb2dc145770 0x7fb2dc0d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2d3e00640 1 -- 172.21.15.46:0/2769513474 >> v1:172.21.15.46:6789/0 conn(0x7fb2dc15c5c0 legacy=0x7fb2dc0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.650+0000 7fb2d3e00640 1 -- 172.21.15.46:0/2769513474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc0cfc30 con 0x7fb2dc072e60 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.651+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/2769513474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc072e60 0x7fb2dc0732c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06d2c0 tx=0x7fb2c402c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66868222f77b3887 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2da000640 1 -- 172.21.15.46:0/2769513474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4039070 con 0x7fb2dc072e60 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2da000640 1 -- 172.21.15.46:0/2769513474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c4034640 con 0x7fb2dc072e60 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2da000640 1 -- 172.21.15.46:0/2769513474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4034a10 con 0x7fb2dc072e60 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 -- 172.21.15.46:0/2769513474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc072e60 msgr2=0x7fb2dc0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/2769513474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc072e60 0x7fb2dc0732c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06d2c0 tx=0x7fb2c402c3b0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 -- 172.21.15.46:0/2769513474 shutdown_connections 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/2769513474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc072e60 0x7fb2dc0732c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/2769513474 >> v2:172.21.15.142:3300/0 conn(0x7fb2dc145770 0x7fb2dc0d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 -- 172.21.15.46:0/2769513474 >> 172.21.15.46:0/2769513474 conn(0x7fb2dc06c530 msgr2=0x7fb2dc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.652+0000 7fb2e1a00640 1 -- 172.21.15.46:0/2769513474 shutdown_connections 2024-04-26T23:45:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.653+0000 7fb2e1a00640 1 -- 172.21.15.46:0/2769513474 wait complete. 2024-04-26T23:45:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.654+0000 7fb2e1a00640 1 Processor -- start 2024-04-26T23:45:39.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.655+0000 7fb2e1a00640 1 -- start start 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.655+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2dc072e60 0x7fb2dc144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.655+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 0x7fb2dc1eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0d01f0 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc073800 con 0x7fb2dc072e60 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc0cf5b0 con 0x7fb2dc145770 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48854/0 (socket says 172.21.15.46:48854) 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 -- 172.21.15.46:0/1242190051 learned_addr learned my addr 172.21.15.46:0/1242190051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2dc072e60 0x7fb2dc144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 0x7fb2dc1eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 -- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 msgr2=0x7fb2dc1eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 0x7fb2dc1eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 -- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2dc072e60 msgr2=0x7fb2dc144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2dc072e60 0x7fb2dc144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2db400640 1 -- 172.21.15.46:0/1242190051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc1f60f0 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.656+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 0x7fb2dc1eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2db400640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400cab0 tx=0x7fb2d400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2d2a00640 1 -- 172.21.15.46:0/1242190051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4017070 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2d2a00640 1 -- 172.21.15.46:0/1242190051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d400ac30 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc0b0a70 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2d2a00640 1 -- 172.21.15.46:0/1242190051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4012400 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.657+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc0b1030 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.661+0000 7fb2d2a00640 1 -- 172.21.15.46:0/1242190051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2d401a070 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.661+0000 7fb2d2a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.662+0000 7fb2d2a00640 1 -- 172.21.15.46:0/1242190051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7fb2d408b9b0 con 0x7fb2dc15c5c0 2024-04-26T23:45:39.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.662+0000 7fb2e4784740 1 Processor -- start 2024-04-26T23:45:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.662+0000 7fb2e4784740 1 -- start start 2024-04-26T23:45:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.662+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e20c5e0 0x56435e20e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e237520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56435e170600 con 0x56435e248cb0 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56435e10d100 con 0x56435e2370a0 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56435e16c5c0 con 0x56435e20c5e0 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48862/0 (socket says 172.21.15.46:48862) 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2d3e00640 1 -- 172.21.15.46:0/3073489434 learned_addr learned my addr 172.21.15.46:0/3073489434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.663+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 0x7fb2b806edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 -- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e20c5e0 msgr2=0x56435e20e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e20c5e0 0x56435e20e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 -- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 msgr2=0x56435e237520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e237520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 -- 172.21.15.46:0/3073489434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435de822d0 con 0x56435e248cb0 2024-04-26T23:45:39.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613d864c9783a39a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2cb400640 1 -- 172.21.15.46:0/3073489434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4039040 con 0x56435e248cb0 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2cb400640 1 -- 172.21.15.46:0/3073489434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c4034640 con 0x56435e248cb0 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.664+0000 7fb2cb400640 1 -- 172.21.15.46:0/3073489434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40349f0 con 0x56435e248cb0 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 -- 172.21.15.46:0/3073489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 msgr2=0x56435e20c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c402e000 comp rx=0 tx=0).stop 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 -- 172.21.15.46:0/3073489434 shutdown_connections 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e237520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e20c9c0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 --2- 172.21.15.46:0/3073489434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e20c5e0 0x56435e20e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 -- 172.21.15.46:0/3073489434 >> 172.21.15.46:0/3073489434 conn(0x56435e1a0d10 msgr2=0x56435e2520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 -- 172.21.15.46:0/3073489434 shutdown_connections 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.665+0000 7fb2e4784740 1 -- 172.21.15.46:0/3073489434 wait complete. 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.666+0000 7fb2e4784740 1 -- start start 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.666+0000 7fb2e4784740 1 Processor -- start 2024-04-26T23:45:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e24a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 0x56435e233620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56435e0f7f70 con 0x56435e20c5e0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56435e234bf0 con 0x56435e2370a0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56435e263000 con 0x56435e248cb0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.667+0000 7fb2e4784740 10 client.0 _ll_register_callbacks cb 0x56435e166480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48870/0 (socket says 172.21.15.46:48870) 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 -- 172.21.15.46:0/1136855662 learned_addr learned my addr 172.21.15.46:0/1136855662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2db400640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 0x56435e233620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 -- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 msgr2=0x56435e233620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e24a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 0x56435e233620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 -- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 msgr2=0x56435e24a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e24a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 -- 172.21.15.46:0/1136855662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435df4c6b0 con 0x56435e20c5e0 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2db400640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 0x56435e233620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00ffe0 tx=0x7fb2cc014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6624f764741519 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.668+0000 7fb2bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc025060 con 0x56435e20c5e0 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7fb2cc015820 con 0x56435e20c5e0 2024-04-26T23:45:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c4034640 con 0x56435e20c5e0 2024-04-26T23:45:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc02b050 con 0x56435e20c5e0 2024-04-26T23:45:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 10 client.8397 fetch_fsmap learned FSMap version 90 2024-04-26T23:45:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56435e2377d0 con 0x56435e20c5e0 2024-04-26T23:45:39.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c4034640 con 0x56435e20c5e0 2024-04-26T23:45:39.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fb2cc02e020 con 0x56435e20c5e0 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 10 client.8397 fetch_fsmap finished waiting for FSMap version 90 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 10 client.8397 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.669+0000 7fb2e4784740 20 client.8397 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.670+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e238300 0x56435e261f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.670+0000 7fb2e4784740 4 client.8397 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:45:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.670+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56435e17e210 con 0x56435e238300 2024-04-26T23:45:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.670+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e238300 0x56435e261f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.671+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e238300 0x56435e261f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812e216a727136b0 server_cookie=f65342cd4f4446b5 in_seq=0 out_seq=0 2024-04-26T23:45:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.671+0000 7fb2bfe00640 10 client.8397 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:45:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1136855662 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56435e17e210 con 0x56435e238300 2024-04-26T23:45:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2bfe00640 10 client.8397 handle_command_reply: tid=0 2024-04-26T23:45:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 client.8397 shutdown 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 msgr2=0x56435e226910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00ffe0 tx=0x7fb2cc014830 comp rx=0 tx=0).stop 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 shutdown_connections 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e238300 0x56435e261f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e248cb0 0x56435e233620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e24a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1136855662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e20c5e0 0x56435e226910 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2bfe00640 0 client.8397 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.673+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 >> 172.21.15.46:0/1136855662 conn(0x56435e1a0d10 msgr2=0x56435e234ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.674+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 shutdown_connections 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.674+0000 7fb2e4784740 1 -- 172.21.15.46:0/1136855662 wait complete. 2024-04-26T23:45:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.674+0000 7fb2e4784740 20 client.8397 trim_cache size 0 max 16384 2024-04-26T23:45:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.693+0000 7fb2e4784740 1 Processor -- start 2024-04-26T23:45:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 -- start start 2024-04-26T23:45:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e261f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e237a30 0x56435e252640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc200e90 con 0x56435e248cb0 2024-04-26T23:45:39.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1f96f0 con 0x56435e2370a0 2024-04-26T23:45:39.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.694+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc1425d0 con 0x56435e237a30 2024-04-26T23:45:39.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48874/0 (socket says 172.21.15.46:48874) 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1803047454 learned_addr learned my addr 172.21.15.46:0/1803047454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2db400640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e237a30 0x56435e252640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e237a30 msgr2=0x56435e252640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e261f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e237a30 0x56435e252640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 msgr2=0x56435e261f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e261f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1803047454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435e20df80 con 0x56435e248cb0 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.695+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e261f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009860 tx=0x7fb2c4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535b5ecf59eef295 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2caa00640 1 -- 172.21.15.46:0/1803047454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4039040 con 0x56435e248cb0 2024-04-26T23:45:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2caa00640 1 -- 172.21.15.46:0/1803047454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40044a0 con 0x56435e248cb0 2024-04-26T23:45:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2caa00640 1 -- 172.21.15.46:0/1803047454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c404d3b0 con 0x56435e248cb0 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 -- 172.21.15.46:0/1803047454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 msgr2=0x56435e224690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009860 tx=0x7fb2c4008030 comp rx=0 tx=0).stop 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 -- 172.21.15.46:0/1803047454 shutdown_connections 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e237a30 0x56435e252640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e248cb0 0x56435e224690 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1803047454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e2370a0 0x56435e261f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.696+0000 7fb2e4784740 1 -- 172.21.15.46:0/1803047454 >> 172.21.15.46:0/1803047454 conn(0x56435e1a0d10 msgr2=0x56435e226720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.697+0000 7fb2e4784740 1 -- 172.21.15.46:0/1803047454 shutdown_connections 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.697+0000 7fb2e4784740 1 -- 172.21.15.46:0/1803047454 wait complete. 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.697+0000 7fb2e4784740 1 -- start start 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.697+0000 7fb2e4784740 1 Processor -- start 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e2370a0 0x56435e2cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 0x56435e2625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56435e238180 con 0x56435e237a30 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56435e262e00 con 0x56435e248cb0 2024-04-26T23:45:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.698+0000 7fb2e4784740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56435e2387a0 con 0x56435e2370a0 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2e4784740 10 client.0 _ll_register_callbacks cb 0x7fb2cc00b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48876/0 (socket says 172.21.15.46:48876) 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1362745323 learned_addr learned my addr 172.21.15.46:0/1362745323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e2370a0 msgr2=0x56435e2cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e2370a0 0x56435e2cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e2370a0 0x56435e2cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 msgr2=0x56435e2625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2db400640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 0x56435e2625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 0x56435e2625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2d3e00640 1 -- 172.21.15.46:0/1362745323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435e2d0360 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.699+0000 7fb2db400640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 0x56435e2625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2d3e00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009890 tx=0x7fb2c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a349ee27083f80f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4039040 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7fb2c405ac70 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c40044a0 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb2c405ae70 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2e4784740 10 client.8403 fetch_fsmap learned FSMap version 90 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56435e2cf880 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.700+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40044a0 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fb2c404d5f0 con 0x56435e237a30 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 10 client.8403 fetch_fsmap finished waiting for FSMap version 90 2024-04-26T23:45:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 10 client.8403 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:45:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 20 client.8403 populate_metadata read hostname 'smithi046' 2024-04-26T23:45:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e225260 0x56435e225640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 4 client.8403 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:45:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "status"}) -- 0x56435e239e60 con 0x56435e225260 2024-04-26T23:45:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.701+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e225260 0x56435e225640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.702+0000 7fb2daa00640 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e225260 0x56435e225640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3c41185cbcc976 server_cookie=dc6e529eac2eb18f in_seq=0 out_seq=0 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.702+0000 7fb2bfe00640 10 client.8403 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2bfe00640 1 -- 172.21.15.46:0/1362745323 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56435e239e60 con 0x56435e225260 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2bfe00640 10 client.8403 handle_command_reply: tid=0 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 client.8403 shutdown 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 msgr2=0x56435e262060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009890 tx=0x7fb2c4004730 comp rx=0 tx=0).stop 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 shutdown_connections 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x56435e225260 0x56435e225640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56435e248cb0 0x56435e2625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56435e237a30 0x56435e262060 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 --2- 172.21.15.46:0/1362745323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56435e2370a0 0x56435e2cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 >> 172.21.15.46:0/1362745323 conn(0x56435e1a0d10 msgr2=0x56435e24a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2bfe00640 0 client.8403 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:45:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 shutdown_connections 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.703+0000 7fb2e4784740 1 -- 172.21.15.46:0/1362745323 wait complete. 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e4784740 20 client.8403 trim_cache size 0 max 16384 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8253, 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:45:39.713 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.8286985580000001, 2024-04-26T23:45:39.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 90, 2024-04-26T23:45:39.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 111, 2024-04-26T23:45:39.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 111, 2024-04-26T23:45:39.714 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.236648847 2024-04-26T23:45:39.714 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 msgr2=0x7fb2b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 0x7fb2b806edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 msgr2=0x7fb2dc1eb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400cab0 tx=0x7fb2d400cf70 comp rx=0 tx=0).stop 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 shutdown_connections 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2dc15c5c0 0x7fb2dc1eb0c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2dc145770 0x7fb2dc1eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 --2- 172.21.15.46:0/1242190051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2dc072e60 0x7fb2dc144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 >> 172.21.15.46:0/1242190051 conn(0x7fb2dc06c530 msgr2=0x7fb2dc0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.704+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 shutdown_connections 2024-04-26T23:45:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.705+0000 7fb2e1a00640 1 -- 172.21.15.46:0/1242190051 wait complete. 2024-04-26T23:45:39.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:45:39.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:39.881 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.926+0000 7f7cd3200640 1 Processor -- start 2024-04-26T23:45:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.926+0000 7f7cd3200640 1 -- start start 2024-04-26T23:45:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.927+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc06da70 0x7f7ccc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.927+0000 7f7cd3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ccc0cde60 0x7f7ccc0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.927+0000 7f7cd3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ccc15d300 con 0x7f7ccc15cf20 2024-04-26T23:45:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.927+0000 7f7cd3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7ccc0ce2f0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.927+0000 7f7cd3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc0d06a0 con 0x7f7ccc06da70 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ccc15cf20 0x7f7ccc06d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48874/0 (socket says 172.21.15.46:48874) 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd1e00640 1 -- 172.21.15.46:0/2088058781 learned_addr learned my addr 172.21.15.46:0/2088058781 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd2800640 1 --2- 172.21.15.46:0/2088058781 >> v2:172.21.15.46:3300/0 conn(0x7f7ccc0cde60 0x7f7ccc0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd1400640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc06da70 0x7f7ccc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd0a00640 1 -- 172.21.15.46:0/2088058781 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 127903390 0 0) 0x7f7ccc15d300 con 0x7f7ccc15cf20 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd0a00640 1 -- 172.21.15.46:0/2088058781 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb4003520 con 0x7f7ccc15cf20 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd2800640 1 -- 172.21.15.46:0/2088058781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc06da70 msgr2=0x7f7ccc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd2800640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc06da70 0x7f7ccc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd2800640 1 -- 172.21.15.46:0/2088058781 >> v1:172.21.15.46:6789/0 conn(0x7f7ccc15cf20 legacy=0x7f7ccc06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.928+0000 7f7cd2800640 1 -- 172.21.15.46:0/2088058781 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc0d0980 con 0x7f7ccc0cde60 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.930+0000 7f7cd2800640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc0b4840 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009d90 tx=0x7f7cc802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b636e42cc2b8cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.930+0000 7f7cd0a00640 1 -- 172.21.15.46:0/2088058781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007740 con 0x7f7ccc0cde60 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.930+0000 7f7cd0a00640 1 -- 172.21.15.46:0/2088058781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc80078e0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.930+0000 7f7cd0a00640 1 -- 172.21.15.46:0/2088058781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007c90 con 0x7f7ccc0cde60 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 -- 172.21.15.46:0/2088058781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 msgr2=0x7f7ccc0b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc0b4840 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009d90 tx=0x7f7cc802a660 comp rx=0 tx=0).stop 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 -- 172.21.15.46:0/2088058781 shutdown_connections 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc0b4840 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 --2- 172.21.15.46:0/2088058781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc06da70 0x7f7ccc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 -- 172.21.15.46:0/2088058781 >> 172.21.15.46:0/2088058781 conn(0x7f7ccc0727a0 msgr2=0x7f7ccc072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 -- 172.21.15.46:0/2088058781 shutdown_connections 2024-04-26T23:45:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.931+0000 7f7cd3200640 1 -- 172.21.15.46:0/2088058781 wait complete. 2024-04-26T23:45:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.933+0000 7f7cd3200640 1 Processor -- start 2024-04-26T23:45:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.933+0000 7f7cd3200640 1 -- start start 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ccc06da70 0x7f7ccc155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc15cf20 0x7f7ccc14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc0b4d80 con 0x7f7ccc0cde60 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc0ce5b0 con 0x7f7ccc06da70 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc0b4f00 con 0x7f7ccc15cf20 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48888/0 (socket says 172.21.15.46:48888) 2024-04-26T23:45:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.934+0000 7f7cd1400640 1 -- 172.21.15.46:0/3105350253 learned_addr learned my addr 172.21.15.46:0/3105350253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd2800640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc15cf20 0x7f7ccc14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1e00640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ccc06da70 0x7f7ccc155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 -- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc15cf20 msgr2=0x7f7ccc14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc15cf20 0x7f7ccc14d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 -- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ccc06da70 msgr2=0x7f7ccc155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ccc06da70 0x7f7ccc155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 -- 172.21.15.46:0/3105350253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc157c10 con 0x7f7ccc0cde60 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.935+0000 7f7cd1400640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00e9e0 tx=0x7f7cbc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.936+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc017070 con 0x7f7ccc0cde60 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.936+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc00c9d0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.936+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00cdc0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.936+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc157ef0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.936+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc1584b0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.937+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc069ac0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.941+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cbc01a070 con 0x7f7ccc0cde60 2024-04-26T23:45:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.942+0000 7f7cc6000640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.942+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f7cbc08c4e0 con 0x7f7ccc0cde60 2024-04-26T23:45:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.942+0000 7f7cd1e00640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.943+0000 7f7cd1e00640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:39.945+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7cbc05a860 con 0x7f7ccc0cde60 2024-04-26T23:45:40.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.198+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ccc0cd690 con 0x7f7ccc0cde60 2024-04-26T23:45:40.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.198+0000 7f7cc6000640 1 -- 172.21.15.46:0/3105350253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7f7cbc05a160 con 0x7f7ccc0cde60 2024-04-26T23:45:40.203 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:45:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 msgr2=0x7f7cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).stop 2024-04-26T23:45:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 msgr2=0x7f7ccc06fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00e9e0 tx=0x7f7cbc00eea0 comp rx=0 tx=0).stop 2024-04-26T23:45:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 shutdown_connections 2024-04-26T23:45:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ccc15cf20 0x7f7ccc14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ccc0cde60 0x7f7ccc06fed0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 --2- 172.21.15.46:0/3105350253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ccc06da70 0x7f7ccc155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.201+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 >> 172.21.15.46:0/3105350253 conn(0x7f7ccc0727a0 msgr2=0x7f7ccc06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.202+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 shutdown_connections 2024-04-26T23:45:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40.202+0000 7f7cd3200640 1 -- 172.21.15.46:0/3105350253 wait complete. 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:45:40.226 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:45:40.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.227 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:45:40.277 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:45:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:45:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:45:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.278 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:45:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:45:40.280 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:45:40.280 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:45:40.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.281 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:45:40.362 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:45:40.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.367 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:45:40.423 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:45:40.424 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:45:40.424 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:45:40.424 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:45:40.424 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:45:40.424 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:45:40.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:45:40.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:45:40.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.644 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:45:40.696 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:45:40.696 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:45:40.696 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:45:40.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:45:40.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:45:40.776 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:45:40.829 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:45:40.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:45:40.829 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:45:40.883 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:45:40.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:40.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:45:41.001 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:45:41.002 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:45:41.002 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:45:41.023 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:45:41.023 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:45:41.023 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:45:41.023 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:45:41.023 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:45:41.054 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:45:41.056 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:45:41.056 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:45:41.056 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:45:41.056 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:45:41.056 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:45:41.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:45:41.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:41.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:45:41.138 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:45:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:45:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:45:41.289 INFO:teuthology.orchestra.run.smithi046.stdout:client5777 2024-04-26T23:45:41.290 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:41.290 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:41.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-26T23:45:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.558+0000 7f9a1e800640 1 Processor -- start 2024-04-26T23:45:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.559+0000 7f9a1e800640 1 -- start start 2024-04-26T23:45:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.560+0000 7f9a1e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a181627f0 0x7f9a180cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.560+0000 7f9a1e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.560+0000 7f9a1e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a180d01a0 con 0x7f9a180d3af0 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.560+0000 7f9a1e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9a180d0480 con 0x7f9a181627f0 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.560+0000 7f9a1e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180d0760 con 0x7f9a180d2080 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.561+0000 7f9a1d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a181627f0 0x7f9a180cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.561+0000 7f9a1ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.561+0000 7f9a1ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:43722/0 (socket says 172.21.15.46:43722) 2024-04-26T23:45:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.561+0000 7f9a1ca00640 1 -- 172.21.15.46:0/2606688535 learned_addr learned my addr 172.21.15.46:0/2606688535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.562+0000 7f9a1ca00640 1 -- 172.21.15.46:0/2606688535 >> v2:172.21.15.142:3300/0 conn(0x7f9a181627f0 msgr2=0x7f9a180cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:45:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.562+0000 7f9a1ca00640 1 --2- 172.21.15.46:0/2606688535 >> v2:172.21.15.142:3300/0 conn(0x7f9a181627f0 0x7f9a180cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.562+0000 7f9a1ca00640 1 -- 172.21.15.46:0/2606688535 >> v1:172.21.15.142:6789/0 conn(0x7f9a180d3af0 legacy=0x7f9a18162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.562+0000 7f9a1ca00640 1 -- 172.21.15.46:0/2606688535 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a180d0a40 con 0x7f9a180d2080 2024-04-26T23:45:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.562+0000 7f9a17e00640 1 -- 172.21.15.46:0/2606688535 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729148616 0 0) 0x7f9a180d01a0 con 0x7f9a180d3af0 2024-04-26T23:45:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.564+0000 7f9a1ca00640 1 --2- 172.21.15.46:0/2606688535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009e50 tx=0x7f9a0402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc54c7c872c0bb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.564+0000 7f9a17e00640 1 -- 172.21.15.46:0/2606688535 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04036070 con 0x7f9a180d2080 2024-04-26T23:45:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.564+0000 7f9a17e00640 1 -- 172.21.15.46:0/2606688535 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a04031600 con 0x7f9a180d2080 2024-04-26T23:45:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.564+0000 7f9a17e00640 1 -- 172.21.15.46:0/2606688535 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040319d0 con 0x7f9a180d2080 2024-04-26T23:45:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 -- 172.21.15.46:0/2606688535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 msgr2=0x7f9a180d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 --2- 172.21.15.46:0/2606688535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009e50 tx=0x7f9a0402a7a0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 -- 172.21.15.46:0/2606688535 shutdown_connections 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 --2- 172.21.15.46:0/2606688535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a180d2080 0x7f9a180d35b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 --2- 172.21.15.46:0/2606688535 >> v2:172.21.15.142:3300/0 conn(0x7f9a181627f0 0x7f9a180cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 -- 172.21.15.46:0/2606688535 >> 172.21.15.46:0/2606688535 conn(0x7f9a1809e380 msgr2=0x7f9a180d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 -- 172.21.15.46:0/2606688535 shutdown_connections 2024-04-26T23:45:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.565+0000 7f9a1e800640 1 -- 172.21.15.46:0/2606688535 wait complete. 2024-04-26T23:45:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.567+0000 7f9a1e800640 1 Processor -- start 2024-04-26T23:45:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.567+0000 7f9a1e800640 1 -- start start 2024-04-26T23:45:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a180d3af0 0x7f9a18160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a181627f0 0x7f9a18161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18162bd0 con 0x7f9a180d2080 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180d24f0 con 0x7f9a180d3af0 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180b5560 con 0x7f9a181627f0 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48892/0 (socket says 172.21.15.46:48892) 2024-04-26T23:45:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.568+0000 7f9a1d400640 1 -- 172.21.15.46:0/273123962 learned_addr learned my addr 172.21.15.46:0/273123962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:45:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1ca00640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a180d3af0 0x7f9a18160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 -- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a181627f0 msgr2=0x7f9a18161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a181627f0 0x7f9a18161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 -- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a180d3af0 msgr2=0x7f9a18160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a180d3af0 0x7f9a18160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 -- 172.21.15.46:0/273123962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1815b9a0 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1d400640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00e870 tx=0x7f9a0c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c017070 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00c940 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a180b7d20 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.569+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00cd30 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.570+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a180b82e0 con 0x7f9a180d2080 2024-04-26T23:45:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.571+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a180693b0 con 0x7f9a180d2080 2024-04-26T23:45:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.572+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a0c01a070 con 0x7f9a180d2080 2024-04-26T23:45:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.573+0000 7f9a15600640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.573+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6101+0+0 (secure 0 0 0) 0x7f9a0c08bb10 con 0x7f9a180d2080 2024-04-26T23:45:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.574+0000 7f9a1ca00640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.575+0000 7f9a1ca00640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a04033000 tx=0x7f9a04009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.577+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a0c059e40 con 0x7f9a180d2080 2024-04-26T23:45:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.822+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9a180d5490 con 0x7f9a180d2080 2024-04-26T23:45:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.949+0000 7f9a15600640 1 -- 172.21.15.46:0/273123962 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v91) ==== 167+0+0 (secure 0 0 0) 0x7f9a0c059740 con 0x7f9a180d2080 2024-04-26T23:45:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:enabled new snapshots 2024-04-26T23:45:41.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 msgr2=0x7f99f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a04033000 tx=0x7f9a04009000 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 msgr2=0x7f9a1815b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00e870 tx=0x7f9a0c00ed30 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 shutdown_connections 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a181627f0 0x7f9a18161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a180d3af0 0x7f9a18160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 --2- 172.21.15.46:0/273123962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a180d2080 0x7f9a1815b460 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 >> 172.21.15.46:0/273123962 conn(0x7f9a1809e380 msgr2=0x7f9a1815f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 shutdown_connections 2024-04-26T23:45:41.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:45:41.952+0000 7f9a1e800640 1 -- 172.21.15.46:0/273123962 wait complete. 2024-04-26T23:45:41.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:45:41.978 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-26T23:45:42.044 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:45:42.045 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:45:42.098 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> import uuid 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> n = 2000 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = False 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> if 3 > 0: 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = True 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> dpath = os.path.dirname(path) 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> fnameprefix = os.path.basename(path) 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> os.makedirs(dpath, exist_ok=True) 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:45:42.099 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> for i in range(n): 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> f.write(f"{i}") 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> f.flush() 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(f.fileno()) 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> os.unlink(fpath) 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> if create_hard_links: 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> for j in range(3): 2024-04-26T23:45:42.100 DEBUG:teuthology.orchestra.run.smithi046:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T23:45:42.101 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:45:42.101 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:45:42.101 DEBUG:teuthology.orchestra.run.smithi046:> finally: 2024-04-26T23:45:42.101 DEBUG:teuthology.orchestra.run.smithi046:> os.close(dirfd) 2024-04-26T23:45:42.101 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:46:10.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:10.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:40.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:40.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:46:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:46:49.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:46:49.126 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-26T23:46:49.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:46:49.270 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-26T23:47:10.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:10.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:40.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:40.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:52.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:47:52.056 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-26T23:47:52.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 perf dump 2024-04-26T23:47:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.319+0000 7f07c2200640 1 Processor -- start 2024-04-26T23:47:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.319+0000 7f07c2200640 1 -- start start 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.320+0000 7f07c2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc0cd2a0 0x7f07bc0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.320+0000 7f07c2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07bc06f130 0x7f07bc06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07c2200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc15d460 con 0x7f07bc06c350 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07c2200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f07bc15d720 con 0x7f07bc06f130 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07c2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc15da00 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07bc06f130 0x7f07bc06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07bc06f130 0x7f07bc06f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41006/0 (socket says 172.21.15.46:41006) 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 -- 172.21.15.46:0/3096917358 learned_addr learned my addr 172.21.15.46:0/3096917358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 -- 172.21.15.46:0/3096917358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc0cd2a0 msgr2=0x7f07bc0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 --2- 172.21.15.46:0/3096917358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc0cd2a0 0x7f07bc0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 -- 172.21.15.46:0/3096917358 >> v1:172.21.15.142:6789/0 conn(0x7f07bc06c350 legacy=0x7f07bc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.321+0000 7f07bbe00640 1 -- 172.21.15.46:0/3096917358 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc15dce0 con 0x7f07bc06f130 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.323+0000 7f07bbe00640 1 --2- 172.21.15.46:0/3096917358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc06f130 0x7f07bc06f5b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d654785f2e071672 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.323+0000 7f07bb400640 1 -- 172.21.15.46:0/3096917358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8037070 con 0x7f07bc06f130 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.323+0000 7f07bb400640 1 -- 172.21.15.46:0/3096917358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8032440 con 0x7f07bc06f130 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.323+0000 7f07bb400640 1 -- 172.21.15.46:0/3096917358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8032810 con 0x7f07bc06f130 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 -- 172.21.15.46:0/3096917358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc06f130 msgr2=0x7f07bc06f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 --2- 172.21.15.46:0/3096917358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc06f130 0x7f07bc06f5b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a802a210 comp rx=0 tx=0).stop 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 -- 172.21.15.46:0/3096917358 shutdown_connections 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 --2- 172.21.15.46:0/3096917358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc06f130 0x7f07bc06f5b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 --2- 172.21.15.46:0/3096917358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc0cd2a0 0x7f07bc0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 -- 172.21.15.46:0/3096917358 >> 172.21.15.46:0/3096917358 conn(0x7f07bc0d1c10 msgr2=0x7f07bc0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 -- 172.21.15.46:0/3096917358 shutdown_connections 2024-04-26T23:47:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.324+0000 7f07c2200640 1 -- 172.21.15.46:0/3096917358 wait complete. 2024-04-26T23:47:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.326+0000 7f07c2200640 1 Processor -- start 2024-04-26T23:47:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.327+0000 7f07c2200640 1 -- start start 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc06c350 0x7f07bc0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07bc06f130 0x7f07bc0c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc06c7e0 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc15e2a0 con 0x7f07bc06f130 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c2200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc06c960 con 0x7f07bc06c350 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41020/0 (socket says 172.21.15.46:41020) 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c1800640 1 -- 172.21.15.46:0/3418300953 learned_addr learned my addr 172.21.15.46:0/3418300953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07c0e00640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc06c350 0x7f07bc0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.328+0000 7f07bbe00640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07bc06f130 0x7f07bc0c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 -- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc06c350 msgr2=0x7f07bc0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc06c350 0x7f07bc0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 -- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07bc06f130 msgr2=0x7f07bc0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07bc06f130 0x7f07bc0c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 -- 172.21.15.46:0/3418300953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc0b27a0 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07c1800640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c7b0 tx=0x7f07b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.329+0000 7f07b8c00640 1 -- 172.21.15.46:0/3418300953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004b60 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.330+0000 7f07b8c00640 1 -- 172.21.15.46:0/3418300953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b4004d00 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.330+0000 7f07b8c00640 1 -- 172.21.15.46:0/3418300953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b400aa90 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.330+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc14f070 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.330+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc147e50 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.332+0000 7f07b8c00640 1 -- 172.21.15.46:0/3418300953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07b400ac30 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.334+0000 7f07b8c00640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 0x7f079c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.334+0000 7f07b8c00640 1 -- 172.21.15.46:0/3418300953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f07b408bf80 con 0x7f07bc0cd2a0 2024-04-26T23:47:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.334+0000 7f07c6575740 1 Processor -- start 2024-04-26T23:47:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.335+0000 7f07c0e00640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 0x7f079c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.335+0000 7f07c6575740 1 -- start start 2024-04-26T23:47:52.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c63120a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 0x55c6312260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c631143900 con 0x55c6312279e0 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6310e01d0 con 0x55c6311dfe20 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c63113f8c0 con 0x55c63120a140 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c0e00640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 0x7f079c06ede0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b810 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41036/0 (socket says 172.21.15.46:41036) 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 -- 172.21.15.46:0/564232580 learned_addr learned my addr 172.21.15.46:0/564232580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c0e00640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c63120a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07c1800640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 0x55c6312260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 -- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 msgr2=0x55c63120a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c63120a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 -- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 msgr2=0x55c6312260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 0x55c6312260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.336+0000 7f07bbe00640 1 -- 172.21.15.46:0/564232580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c630f1f7d0 con 0x55c6312279e0 2024-04-26T23:47:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.337+0000 7f07c1800640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 0x55c6312260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.337+0000 7f07bbe00640 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcc981fe2ffcf4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.337+0000 7f07b1600640 1 -- 172.21.15.46:0/564232580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8043070 con 0x55c6312279e0 2024-04-26T23:47:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.337+0000 7f07b1600640 1 -- 172.21.15.46:0/564232580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8032740 con 0x55c6312279e0 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.337+0000 7f07b1600640 1 -- 172.21.15.46:0/564232580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8037040 con 0x55c6312279e0 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 -- 172.21.15.46:0/564232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 msgr2=0x55c631227f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a802c000 comp rx=0 tx=0).stop 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 -- 172.21.15.46:0/564232580 shutdown_connections 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6311dfe20 0x55c6312260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c631227f30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 --2- 172.21.15.46:0/564232580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c63120a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 -- 172.21.15.46:0/564232580 >> 172.21.15.46:0/564232580 conn(0x55c6311dc750 msgr2=0x55c631224f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 -- 172.21.15.46:0/564232580 shutdown_connections 2024-04-26T23:47:52.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.338+0000 7f07c6575740 1 -- 172.21.15.46:0/564232580 wait complete. 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.339+0000 7f07c6575740 1 -- start start 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.339+0000 7f07c6575740 1 Processor -- start 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c6311de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631206a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c6311e0bc0 con 0x55c6312279e0 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6310cb120 con 0x55c6312065e0 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c6311da680 con 0x55c63120a140 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07c6575740 10 client.0 _ll_register_callbacks cb 0x55c631103750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.340+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41046/0 (socket says 172.21.15.46:41046) 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 -- 172.21.15.46:0/1772447861 learned_addr learned my addr 172.21.15.46:0/1772447861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 -- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 msgr2=0x55c6311de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c6311de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 -- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 msgr2=0x55c631206a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631206a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 -- 172.21.15.46:0/1772447861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c630e150f0 con 0x55c6312279e0 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f07a80090b0 tx=0x7f07a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f451c4ae775f700 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07a6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.341+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8037040 con 0x55c6312279e0 2024-04-26T23:47:52.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f07a8032d00 con 0x55c6312279e0 2024-04-26T23:47:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a8032740 con 0x55c6312279e0 2024-04-26T23:47:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f07a8055050 con 0x55c6312279e0 2024-04-26T23:47:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 10 client.8427 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55c630f7ff60 con 0x55c6312279e0 2024-04-26T23:47:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8032740 con 0x55c6312279e0 2024-04-26T23:47:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f07a8058020 con 0x55c6312279e0 2024-04-26T23:47:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 10 client.8427 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 10 client.8427 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 20 client.8427 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c631239110 0x55c631239570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 4 client.8427 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.342+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c631151510 con 0x55c631239110 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.343+0000 7f07c0e00640 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c631239110 0x55c631239570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.344+0000 7f07c0e00640 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c631239110 0x55c631239570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a807f08ca26627a9 server_cookie=1ebbe091863a0162 in_seq=0 out_seq=0 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.344+0000 7f07a6000640 10 client.8427 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07a6000640 1 -- 172.21.15.46:0/1772447861 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c631151510 con 0x55c631239110 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07a6000640 10 client.8427 handle_command_reply: tid=0 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 client.8427 shutdown 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 msgr2=0x55c63120da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f07a80090b0 tx=0x7f07a8004730 comp rx=0 tx=0).stop 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 shutdown_connections 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c631239110 0x55c631239570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631206a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120da50 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 --2- 172.21.15.46:0/1772447861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c63120a140 0x55c6311de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07a6000640 0 client.8427 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 >> 172.21.15.46:0/1772447861 conn(0x55c6311dfe20 msgr2=0x55c6312284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.346+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 shutdown_connections 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.347+0000 7f07c6575740 1 -- 172.21.15.46:0/1772447861 wait complete. 2024-04-26T23:47:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.347+0000 7f07c6575740 20 client.8427 trim_cache size 0 max 16384 2024-04-26T23:47:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.365+0000 7f07c6575740 1 Processor -- start 2024-04-26T23:47:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.365+0000 7f07c6575740 1 -- start start 2024-04-26T23:47:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.365+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631223050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.365+0000 7f07c6575740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312279e0 0x55c6311e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c6575740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c631227300 con 0x55c63120a140 2024-04-26T23:47:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc201fa0 con 0x55c6312065e0 2024-04-26T23:47:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc14cd40 con 0x55c6312279e0 2024-04-26T23:47:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41048/0 (socket says 172.21.15.46:41048) 2024-04-26T23:47:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 -- 172.21.15.46:0/1183629714 learned_addr learned my addr 172.21.15.46:0/1183629714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 -- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312279e0 msgr2=0x55c6311e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c1800640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312279e0 0x55c6311e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312279e0 0x55c6311e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 -- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 msgr2=0x55c631223050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c0e00640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631223050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631223050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07bbe00640 1 -- 172.21.15.46:0/1183629714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c630dee1b0 con 0x55c63120a140 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.366+0000 7f07c0e00640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631223050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07bbe00640 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009630 tx=0x7f07a800b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c1e66764f654f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07b0c00640 1 -- 172.21.15.46:0/1183629714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8037040 con 0x55c63120a140 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07b0c00640 1 -- 172.21.15.46:0/1183629714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a804d070 con 0x55c63120a140 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07b0c00640 1 -- 172.21.15.46:0/1183629714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8043040 con 0x55c63120a140 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07c6575740 1 -- 172.21.15.46:0/1183629714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 msgr2=0x55c63122bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.367+0000 7f07c6575740 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009630 tx=0x7f07a800b050 comp rx=0 tx=0).stop 2024-04-26T23:47:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 -- 172.21.15.46:0/1183629714 shutdown_connections 2024-04-26T23:47:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312279e0 0x55c6311e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c63120a140 0x55c63122bcc0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 --2- 172.21.15.46:0/1183629714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6312065e0 0x55c631223050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 -- 172.21.15.46:0/1183629714 >> 172.21.15.46:0/1183629714 conn(0x55c6311dfe20 msgr2=0x55c6311de100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 -- 172.21.15.46:0/1183629714 shutdown_connections 2024-04-26T23:47:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 -- 172.21.15.46:0/1183629714 wait complete. 2024-04-26T23:47:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.368+0000 7f07c6575740 1 -- start start 2024-04-26T23:47:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.369+0000 7f07c6575740 1 Processor -- start 2024-04-26T23:47:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312065e0 0x55c63129e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63120a140 0x55c63120d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc14c6f0 con 0x55c6312279e0 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc14c0a0 con 0x55c63120a140 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c631145e10 con 0x55c6312065e0 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c6575740 10 client.0 _ll_register_callbacks cb 0x7f07a80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41054/0 (socket says 172.21.15.46:41054) 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 -- 172.21.15.46:0/300151794 learned_addr learned my addr 172.21.15.46:0/300151794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c0e00640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312065e0 0x55c63129e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07bbe00640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63120a140 0x55c63120d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 -- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312065e0 msgr2=0x55c63129e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312065e0 0x55c63129e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 -- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63120a140 msgr2=0x55c63120d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63120a140 0x55c63120d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.370+0000 7f07c1800640 1 -- 172.21.15.46:0/300151794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c631223b70 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c1800640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f07b4055f40 tx=0x7f07b4059500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3ba5052cb20361 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b405de90 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f07b4056150 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a804d070 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f07b4079050 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c6575740 10 client.8433 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55c631248d10 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a804d070 con 0x55c6312279e0 2024-04-26T23:47:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f07b4062230 con 0x55c6312279e0 2024-04-26T23:47:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c6575740 10 client.8433 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.371+0000 7f07c6575740 10 client.8433 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.372+0000 7f07c6575740 20 client.8433 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.372+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c63122b220 0x55c631233fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.372+0000 7f07c6575740 4 client.8433 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-26T23:47:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.372+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55c63114b380 con 0x55c63122b220 2024-04-26T23:47:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.372+0000 7f07c0e00640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c63122b220 0x55c631233fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.373+0000 7f07c0e00640 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c63122b220 0x55c631233fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9e9323f725b734 server_cookie=98164cb35fb46def in_seq=0 out_seq=0 2024-04-26T23:47:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.373+0000 7f07a6000640 10 client.8433 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07a6000640 1 -- 172.21.15.46:0/300151794 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17987 (crc 0 0 0) 0x55c63114b380 con 0x55c63122b220 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07a6000640 10 client.8433 handle_command_reply: tid=0 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 client.8433 shutdown 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 msgr2=0x55c63120dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f07b4055f40 tx=0x7f07b4059500 comp rx=0 tx=0).stop 2024-04-26T23:47:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 shutdown_connections 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55c63122b220 0x55c631233fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6312279e0 0x55c63120dcf0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63120a140 0x55c63120d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 --2- 172.21.15.46:0/300151794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6312065e0 0x55c63129e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07a6000640 0 client.8433 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.375+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 >> 172.21.15.46:0/300151794 conn(0x55c6311dfe20 msgr2=0x55c6311de100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 shutdown_connections 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c6575740 1 -- 172.21.15.46:0/300151794 wait complete. 2024-04-26T23:47:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c6575740 20 client.8433 trim_cache size 0 max 16384 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1206, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1207, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 514421, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 19890323, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 9, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.528730328, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.210280445, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.064865657, 2024-04-26T23:47:52.386 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.232029304, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1206, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.439345682, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000364299 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000278, 2024-04-26T23:47:52.387 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000139 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1739, 2024-04-26T23:47:52.388 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1768, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 441091, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 25372046, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.804161373, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.209282468, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.080003707, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.484321420, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1767, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.533394026, 2024-04-26T23:47:52.389 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000301864 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 139248 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 28765, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 50617, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 9999805, 2024-04-26T23:47:52.390 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 69937759, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 7, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 7.227535578, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 4.365918153, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 3.029803227, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.144998134, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 50617, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.394338382, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000086815 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 45486, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.028076445, 2024-04-26T23:47:52.391 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000617 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 124288, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 30960 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1417, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 31.858787390, 2024-04-26T23:47:52.392 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.022483265 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2400, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.826684043, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000761118 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 25725, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 25725, 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-26T23:47:52.393 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 25725, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 524.741006933, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.020398095 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 12, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 19898, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 0, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 19898, 2024-04-26T23:47:52.394 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 19897, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 55, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 55, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 2, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 46727, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 46203, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 105, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-04-26T23:47:52.395 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:47:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 904, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 4019, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 2905, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 2029, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 62, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 14122 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-26T23:47:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 2000, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 4148, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 1947, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 2200, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:47:52.398 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 24167, 2024-04-26T23:47:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 24167, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 32, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 32, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 3, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-04-26T23:47:52.400 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 94752208, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2628, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 119.968101820, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.045649962 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 2014, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 2014, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 0, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 13, 2024-04-26T23:47:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 13, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 0, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 19898, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 19899, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 1, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 55, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 8396, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 8341, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 110692, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231788 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 28626, 2024-04-26T23:47:52.402 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 25725, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 8, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2000, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 5.101454783, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002550727 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 30, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.013439196, 2024-04-26T23:47:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000447973 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6000, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 20.213922429, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003368987 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7481, 2024-04-26T23:47:52.404 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 9.102754932, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001216783 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-26T23:47:52.405 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000200043, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000200043 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.406 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.006559292, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006559292 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.044697434, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.044697434 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000297787, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000297787 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.185437877, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.023179734 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2200, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 449.676547235, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.204398430 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-26T23:47:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.019167581, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.019167581 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000653771, 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000653771 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8000, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 40.375874573, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.005046984 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:47:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 1, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 0, 2024-04-26T23:47:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 13715, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 13715, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 126, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:47:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-26T23:47:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-26T23:47:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 16386502, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 6171, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 528, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 6, 2024-04-26T23:47:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 5616, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 80, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-26T23:47:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 21397525, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 882984, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 4980, 2024-04-26T23:47:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 91044778, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 31, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 4980, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4980, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 43.391162795, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008713084 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4980, 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 5154 2024-04-26T23:47:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 4980, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 87, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 4893, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 11, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 87, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 2715, 2024-04-26T23:47:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 54, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 1947, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 33, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-26T23:47:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 22, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-26T23:47:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-26T23:47:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 113, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 3, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 8, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 8, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 236, 2024-04-26T23:47:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 49 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 32, 2024-04-26T23:47:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 32, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 1947, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 1947, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 31705, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 8522199, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 31705, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:47:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 31705, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 8522199, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:47:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 4980, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 91257001, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 4980, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 91257001, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-26T23:47:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 4980, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 4980, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 4980, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 4980, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:47:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 1947, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 212223, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 1947, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 87, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 212223, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x55be54b5c500": { 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 84, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 24167, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 90557904, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 24167, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 2628, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 90557820, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:47:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:47:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:47:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:52.436 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:47:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 msgr2=0x7f079c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 0x7f079c06ede0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b810 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2024-04-26T23:47:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 msgr2=0x7f07bc0c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.376+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c7b0 tx=0x7f07b400cc70 comp rx=0 tx=0).stop 2024-04-26T23:47:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 shutdown_connections 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079c06c920 0x7f079c06ede0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07bc0cd2a0 0x7f07bc0c6fd0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07bc06f130 0x7f07bc0c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 --2- 172.21.15.46:0/3418300953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07bc06c350 0x7f07bc0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 >> 172.21.15.46:0/3418300953 conn(0x7f07bc0d1c10 msgr2=0x7f07bc15fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 shutdown_connections 2024-04-26T23:47:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.377+0000 7f07c2200640 1 -- 172.21.15.46:0/3418300953 wait complete. 2024-04-26T23:47:52.439 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-26T23:47:52.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:47:52.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start '~mdsdir' recursive 2024-04-26T23:47:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.660+0000 7f9f2f200640 1 Processor -- start 2024-04-26T23:47:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.660+0000 7f9f2f200640 1 -- start start 2024-04-26T23:47:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30146740 0x7f9f300cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f300d17f0 0x7f9f300cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f300b3ce0 con 0x7f9f30146b20 2024-04-26T23:47:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9f300b48f0 con 0x7f9f300d17f0 2024-04-26T23:47:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f300b3fc0 con 0x7f9f30146740 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f30146b20 0x7f9f300d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53912/0 (socket says 172.21.15.46:53912) 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.662+0000 7f9f2d400640 1 -- 172.21.15.46:0/3473756758 learned_addr learned my addr 172.21.15.46:0/3473756758 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30146740 0x7f9f300cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3473756758 >> v2:172.21.15.142:3300/0 conn(0x7f9f300d17f0 0x7f9f300cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653777751 0 0) 0x7f9f300b3ce0 con 0x7f9f30146b20 2024-04-26T23:47:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f14003520 con 0x7f9f30146b20 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 853327449 0 0) 0x7f9f14003520 con 0x7f9f30146b20 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30146740 msgr2=0x7f9f300cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30146740 0x7f9f300cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 >> v2:172.21.15.142:3300/0 conn(0x7f9f300d17f0 msgr2=0x7f9f300cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 --2- 172.21.15.46:0/3473756758 >> v2:172.21.15.142:3300/0 conn(0x7f9f300d17f0 0x7f9f300cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f300b42a0 con 0x7f9f30146b20 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9f1c001670 con 0x7f9f30146b20 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30146740 0x7f9f300cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 >> v1:172.21.15.46:6789/0 conn(0x7f9f30146b20 legacy=0x7f9f300d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140039b0 0x7f9f14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.663+0000 7f9f2ca00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f14004800 0x7f9f14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2ca00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140051c0 0x7f9f1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300b48f0 con 0x7f9f140051c0 2024-04-26T23:47:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300b3fc0 con 0x7f9f14004800 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1400bf80 con 0x7f9f140039b0 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140051c0 0x7f9f1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f14004800 0x7f9f14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140039b0 0x7f9f14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 -- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140039b0 msgr2=0x7f9f14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140039b0 0x7f9f14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 -- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f14004800 msgr2=0x7f9f14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f14004800 0x7f9f14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2e800640 1 -- 172.21.15.46:0/3473756758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f14003520 con 0x7f9f140051c0 2024-04-26T23:47:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.664+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f14004800 0x7f9f14004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.665+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140051c0 0x7f9f1400b9c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009f90 tx=0x7f9f14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563dc4dd89999f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28009c10 con 0x7f9f140051c0 2024-04-26T23:47:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f28036070 con 0x7f9f140051c0 2024-04-26T23:47:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2ca00640 1 -- 172.21.15.46:0/3473756758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28031880 con 0x7f9f140051c0 2024-04-26T23:47:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2f200640 1 -- 172.21.15.46:0/3473756758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140051c0 msgr2=0x7f9f1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3473756758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140051c0 0x7f9f1400b9c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009f90 tx=0x7f9f14005830 comp rx=0 tx=0).stop 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2e800640 1 -- 172.21.15.46:0/3473756758 reap_dead start 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2f200640 1 -- 172.21.15.46:0/3473756758 shutdown_connections 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2f200640 1 -- 172.21.15.46:0/3473756758 >> 172.21.15.46:0/3473756758 conn(0x7f9f3006c770 msgr2=0x7f9f3006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.666+0000 7f9f2f200640 1 -- 172.21.15.46:0/3473756758 shutdown_connections 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.667+0000 7f9f2f200640 1 -- 172.21.15.46:0/3473756758 wait complete. 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.668+0000 7f9f2f200640 1 Processor -- start 2024-04-26T23:47:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.669+0000 7f9f2f200640 1 -- start start 2024-04-26T23:47:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.669+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140051c0 0x7f9f3015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.669+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30132e20 0x7f9f300ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300cd6d0 con 0x7f9f301332a0 2024-04-26T23:47:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3015e560 con 0x7f9f140051c0 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f300cd850 con 0x7f9f30132e20 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41078/0 (socket says 172.21.15.46:41078) 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 -- 172.21.15.46:0/3816700262 learned_addr learned my addr 172.21.15.46:0/3816700262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 -- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30132e20 msgr2=0x7f9f300ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30132e20 0x7f9f300ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 -- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140051c0 msgr2=0x7f9f3015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140051c0 0x7f9f3015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.670+0000 7f9f2e800640 1 -- 172.21.15.46:0/3816700262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3015ce50 con 0x7f9f301332a0 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9f2802aee0 tx=0x7f9f28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f26000640 1 -- 172.21.15.46:0/3816700262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28009c10 con 0x7f9f301332a0 2024-04-26T23:47:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f26000640 1 -- 172.21.15.46:0/3816700262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f28031db0 con 0x7f9f301332a0 2024-04-26T23:47:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f301431b0 con 0x7f9f301332a0 2024-04-26T23:47:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f26000640 1 -- 172.21.15.46:0/3816700262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28045670 con 0x7f9f301332a0 2024-04-26T23:47:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.671+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f300ad730 con 0x7f9f301332a0 2024-04-26T23:47:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.674+0000 7f9f26000640 1 -- 172.21.15.46:0/3816700262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f28005d10 con 0x7f9f301332a0 2024-04-26T23:47:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.675+0000 7f9f26000640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.676+0000 7f9f26000640 1 -- 172.21.15.46:0/3816700262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f9f280ad340 con 0x7f9f301332a0 2024-04-26T23:47:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.676+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.676+0000 7f9f37f9d740 1 Processor -- start 2024-04-26T23:47:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.676+0000 7f9f37f9d740 1 -- start start 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e5c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ff7d95db0 con 0x556ff7e308b0 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556ff7d32000 con 0x556ff7e58ba0 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556ff7d91d70 con 0x556ff7e594c0 2024-04-26T23:47:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.677+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f300b5280 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41092/0 (socket says 172.21.15.46:41092) 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 -- 172.21.15.46:0/3765309396 learned_addr learned my addr 172.21.15.46:0/3765309396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e5c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 -- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 msgr2=0x556ff7e5bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e5bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 -- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 msgr2=0x556ff7e5c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e5c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 -- 172.21.15.46:0/3765309396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff7b73a20 con 0x556ff7e308b0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e5c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.678+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c007bf0 tx=0x7f9f1c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507d195ac8243e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f1aa00640 1 -- 172.21.15.46:0/3765309396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00ddd0 con 0x556ff7e308b0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f1aa00640 1 -- 172.21.15.46:0/3765309396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c0070d0 con 0x556ff7e308b0 2024-04-26T23:47:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f1aa00640 1 -- 172.21.15.46:0/3765309396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c007480 con 0x556ff7e308b0 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3765309396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 msgr2=0x556ff7e58f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c007bf0 tx=0x7f9f1c00d520 comp rx=0 tx=0).stop 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3765309396 shutdown_connections 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e308b0 0x556ff7e58f80 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3765309396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e5c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3765309396 >> 172.21.15.46:0/3765309396 conn(0x556ff7e2e740 msgr2=0x556ff7e76d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3765309396 shutdown_connections 2024-04-26T23:47:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.679+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3765309396 wait complete. 2024-04-26T23:47:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.680+0000 7f9f37f9d740 1 -- start start 2024-04-26T23:47:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.680+0000 7f9f37f9d740 1 Processor -- start 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 0x556ff7e5a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ff7e2ca90 con 0x556ff7e594c0 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556ff7d1cc50 con 0x556ff7e4e840 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556ff7e2c4d0 con 0x556ff7e58ba0 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.681+0000 7f9f37f9d740 10 client.0 _ll_register_callbacks cb 0x556ff7d8cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41104/0 (socket says 172.21.15.46:41104) 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 -- 172.21.15.46:0/36308402 learned_addr learned my addr 172.21.15.46:0/36308402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2e800640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 0x556ff7e5a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2de00640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 -- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 msgr2=0x556ff7e512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 -- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 msgr2=0x556ff7e5a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 0x556ff7e5a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 -- 172.21.15.46:0/36308402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff7a67310 con 0x556ff7e594c0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2e800640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 0x556ff7e5a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2de00640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f2d400640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f22afa2ffe7f22d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.682+0000 7f9f0f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c023020 con 0x556ff7e594c0 2024-04-26T23:47:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f9f1c0047e0 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f1c0070d0 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c0049e0 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 10 client.8445 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x556ff7bd41b0 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c0070d0 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f9f1c004b80 con 0x556ff7e594c0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 10 client.8445 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 10 client.8445 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 20 client.8445 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.683+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e52120 0x556ff7e52550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.684+0000 7f9f37f9d740 4 client.8445 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.684+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff7da39c0 con 0x556ff7e52120 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.684+0000 7f9f2de00640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e52120 0x556ff7e52550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.685+0000 7f9f2de00640 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e52120 0x556ff7e52550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396ae0dcc9f6d514 server_cookie=902df1e63012fdeb in_seq=0 out_seq=0 2024-04-26T23:47:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.685+0000 7f9f0f400640 10 client.8445 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f0f400640 1 -- 172.21.15.46:0/36308402 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ff7da39c0 con 0x556ff7e52120 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f0f400640 10 client.8445 handle_command_reply: tid=0 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 client.8445 shutdown 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 msgr2=0x556ff7e59cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c000ed0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 shutdown_connections 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e52120 0x556ff7e52550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e4e840 0x556ff7e5a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e594c0 0x556ff7e59cf0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/36308402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 >> 172.21.15.46:0/36308402 conn(0x556ff7e308b0 msgr2=0x556ff7e30cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f0f400640 0 client.8445 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 shutdown_connections 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.687+0000 7f9f37f9d740 1 -- 172.21.15.46:0/36308402 wait complete. 2024-04-26T23:47:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.688+0000 7f9f37f9d740 20 client.8445 trim_cache size 0 max 16384 2024-04-26T23:47:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.707+0000 7f9f37f9d740 1 Processor -- start 2024-04-26T23:47:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.707+0000 7f9f37f9d740 1 -- start start 2024-04-26T23:47:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e50de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f301fe120 con 0x556ff7e4e840 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f30073d80 con 0x556ff7e58ba0 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f300730a0 con 0x556ff7e594c0 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41118/0 (socket says 172.21.15.46:41118) 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.708+0000 7f9f2de00640 1 -- 172.21.15.46:0/2605160261 learned_addr learned my addr 172.21.15.46:0/2605160261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 -- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 msgr2=0x556ff7e51320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 -- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 msgr2=0x556ff7e50de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e50de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 -- 172.21.15.46:0/2605160261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff7a40490 con 0x556ff7e4e840 2024-04-26T23:47:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f2de00640 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9f200093d0 tx=0x7f9f20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c32dbf48d80929 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f1a000640 1 -- 172.21.15.46:0/2605160261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20011da0 con 0x556ff7e4e840 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f1a000640 1 -- 172.21.15.46:0/2605160261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2002f070 con 0x556ff7e4e840 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.709+0000 7f9f1a000640 1 -- 172.21.15.46:0/2605160261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200228a0 con 0x556ff7e4e840 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 -- 172.21.15.46:0/2605160261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 msgr2=0x556ff7e7a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9f200093d0 tx=0x7f9f20017000 comp rx=0 tx=0).stop 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 -- 172.21.15.46:0/2605160261 shutdown_connections 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e594c0 0x556ff7e51320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e58ba0 0x556ff7e50de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/2605160261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e7a610 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 -- 172.21.15.46:0/2605160261 >> 172.21.15.46:0/2605160261 conn(0x556ff7e308b0 msgr2=0x556ff7e4f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 -- 172.21.15.46:0/2605160261 shutdown_connections 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.710+0000 7f9f37f9d740 1 -- 172.21.15.46:0/2605160261 wait complete. 2024-04-26T23:47:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.711+0000 7f9f37f9d740 1 -- start start 2024-04-26T23:47:52.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.711+0000 7f9f37f9d740 1 Processor -- start 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e50d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e594c0 0x556ff7e512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ff7e330b0 con 0x556ff7e4e840 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300736f0 con 0x556ff7e594c0 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556ff7e31f90 con 0x556ff7e58ba0 2024-04-26T23:47:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f37f9d740 10 client.0 _ll_register_callbacks cb 0x7f9f1c007a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41120/0 (socket says 172.21.15.46:41120) 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.712+0000 7f9f2de00640 1 -- 172.21.15.46:0/3306190825 learned_addr learned my addr 172.21.15.46:0/3306190825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2e800640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e594c0 0x556ff7e512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e50d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 -- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 msgr2=0x556ff7e50d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e50d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 -- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e594c0 msgr2=0x556ff7e512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e594c0 0x556ff7e512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 -- 172.21.15.46:0/3306190825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff7ef5e60 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f2de00640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x556ff7e780a0 tx=0x7f9f20009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2a7453d1cef9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.713+0000 7f9f0f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20022ab0 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f9f2000f040 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2002f070 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9f20022c50 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 10 client.8451 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x556ff7e9e6c0 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2002f070 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f9f20044020 con 0x556ff7e4e840 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 10 client.8451 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 10 client.8451 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 20 client.8451 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e56bb0 0x556ff7e56f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 4 client.8451 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-26T23:47:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.714+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) -- 0x556ff7d9d830 con 0x556ff7e56bb0 2024-04-26T23:47:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.715+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e56bb0 0x556ff7e56f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.716+0000 7f9f2d400640 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e56bb0 0x556ff7e56f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d059ae919e4450c3 server_cookie=56cabf02d69a36 in_seq=0 out_seq=0 2024-04-26T23:47:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.716+0000 7f9f0f400640 10 client.8451 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.743+0000 7f9f0f400640 1 -- 172.21.15.46:0/3306190825 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556ff7d9d830 con 0x556ff7e56bb0 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.743+0000 7f9f0f400640 10 client.8451 handle_command_reply: tid=0 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.743+0000 7f9f37f9d740 1 client.8451 shutdown 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 msgr2=0x556ff7e9daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x556ff7e780a0 tx=0x7f9f20009190 comp rx=0 tx=0).stop 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 shutdown_connections 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x556ff7e56bb0 0x556ff7e56f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ff7e594c0 0x556ff7e512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556ff7e58ba0 0x556ff7e50d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 --2- 172.21.15.46:0/3306190825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff7e4e840 0x556ff7e9daa0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 >> 172.21.15.46:0/3306190825 conn(0x556ff7e308b0 msgr2=0x556ff7e4f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f0f400640 0 client.8451 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:52.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 shutdown_connections 2024-04-26T23:47:52.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 1 -- 172.21.15.46:0/3306190825 wait complete. 2024-04-26T23:47:52.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.744+0000 7f9f37f9d740 20 client.8451 trim_cache size 0 max 16384 2024-04-26T23:47:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:47:52.749 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:47:52.750 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "bfee5ecf-1ed5-4b13-b818-c3ec93e664bb", 2024-04-26T23:47:52.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:47:52.750 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.745+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 msgr2=0x7f9f1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.745+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f300b5280 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.745+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 msgr2=0x7f9f300acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.745+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9f2802aee0 tx=0x7f9f28009750 comp rx=0 tx=0).stop 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 shutdown_connections 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f301332a0 0x7f9f300acdd0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f30132e20 0x7f9f300ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 --2- 172.21.15.46:0/3816700262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140051c0 0x7f9f3015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:52.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 >> 172.21.15.46:0/3816700262 conn(0x7f9f3006c770 msgr2=0x7f9f3015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:52.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 shutdown_connections 2024-04-26T23:47:52.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:52.746+0000 7f9f2f200640 1 -- 172.21.15.46:0/3816700262 wait complete. 2024-04-26T23:47:52.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:47:52.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-26T23:47:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.003+0000 7f3397600640 1 Processor -- start 2024-04-26T23:47:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.003+0000 7f3397600640 1 -- start start 2024-04-26T23:47:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3397600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f339015f480 0x7f33900d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3397600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33901630a0 0x7f33901634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3397600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3390163e70 con 0x7f33900d1470 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3397600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3390163a30 con 0x7f339015f480 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3397600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f33900d0aa0 con 0x7f33901630a0 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f338fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f33900d1470 0x7f33900d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53928/0 (socket says 172.21.15.46:53928) 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3394c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f339015f480 0x7f33900d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f338fe00640 1 -- 172.21.15.46:0/1463514118 learned_addr learned my addr 172.21.15.46:0/1463514118 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:47:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.005+0000 7f3395600640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33901630a0 0x7f33901634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.006+0000 7f3394c00640 1 -- 172.21.15.46:0/1463514118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33901630a0 msgr2=0x7f33901634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.006+0000 7f3394c00640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33901630a0 0x7f33901634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.006+0000 7f3394c00640 1 -- 172.21.15.46:0/1463514118 >> v1:172.21.15.46:6789/0 conn(0x7f33900d1470 legacy=0x7f33900d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.006+0000 7f3394c00640 1 -- 172.21.15.46:0/1463514118 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900d0170 con 0x7f339015f480 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.006+0000 7f338f400640 1 -- 172.21.15.46:0/1463514118 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735459164 0 0) 0x7f3390163e70 con 0x7f33900d1470 2024-04-26T23:47:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.007+0000 7f3394c00640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339015f480 0x7f33900d0f30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f33840097a0 tx=0x7f338402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a62bc4d66e7dcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.007+0000 7f338f400640 1 -- 172.21.15.46:0/1463514118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384037070 con 0x7f339015f480 2024-04-26T23:47:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.007+0000 7f338f400640 1 -- 172.21.15.46:0/1463514118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384032440 con 0x7f339015f480 2024-04-26T23:47:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.007+0000 7f338f400640 1 -- 172.21.15.46:0/1463514118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384032810 con 0x7f339015f480 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 -- 172.21.15.46:0/1463514118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339015f480 msgr2=0x7f33900d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339015f480 0x7f33900d0f30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f33840097a0 tx=0x7f338402a210 comp rx=0 tx=0).stop 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 -- 172.21.15.46:0/1463514118 shutdown_connections 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33901630a0 0x7f33901634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 --2- 172.21.15.46:0/1463514118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339015f480 0x7f33900d0f30 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 -- 172.21.15.46:0/1463514118 >> 172.21.15.46:0/1463514118 conn(0x7f33900d5070 msgr2=0x7f33900d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 -- 172.21.15.46:0/1463514118 shutdown_connections 2024-04-26T23:47:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.008+0000 7f3397600640 1 -- 172.21.15.46:0/1463514118 wait complete. 2024-04-26T23:47:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.011+0000 7f3397600640 1 Processor -- start 2024-04-26T23:47:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.011+0000 7f3397600640 1 -- start start 2024-04-26T23:47:53.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33900d1470 0x7f33900cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f339015f480 0x7f33900cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33900cfe60 con 0x7f33901630a0 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33900d0730 con 0x7f339015f480 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3397600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f339015f860 con 0x7f33900d1470 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3395600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3395600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41134/0 (socket says 172.21.15.46:41134) 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3395600640 1 -- 172.21.15.46:0/289347546 learned_addr learned my addr 172.21.15.46:0/289347546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.012+0000 7f3394c00640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33900d1470 0x7f33900cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f338fe00640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f339015f480 0x7f33900cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 -- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33900d1470 msgr2=0x7f33900cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33900d1470 0x7f33900cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 -- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f339015f480 msgr2=0x7f33900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f339015f480 0x7f33900cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 -- 172.21.15.46:0/289347546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900cdc50 con 0x7f33901630a0 2024-04-26T23:47:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.013+0000 7f3395600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f338800efc0 tx=0x7f338800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.014+0000 7f338cc00640 1 -- 172.21.15.46:0/289347546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388018020 con 0x7f33901630a0 2024-04-26T23:47:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.014+0000 7f338cc00640 1 -- 172.21.15.46:0/289347546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388014480 con 0x7f33901630a0 2024-04-26T23:47:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.014+0000 7f338cc00640 1 -- 172.21.15.46:0/289347546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388014870 con 0x7f33901630a0 2024-04-26T23:47:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.014+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33901e9760 con 0x7f33901630a0 2024-04-26T23:47:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.014+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901e9d20 con 0x7f33901630a0 2024-04-26T23:47:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.016+0000 7f338cc00640 1 -- 172.21.15.46:0/289347546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3388005d10 con 0x7f33901630a0 2024-04-26T23:47:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.017+0000 7f338cc00640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 0x7f337006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.018+0000 7f338cc00640 1 -- 172.21.15.46:0/289347546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f338808ca00 con 0x7f33901630a0 2024-04-26T23:47:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.018+0000 7f3394c00640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 0x7f337006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.019+0000 7f339a577740 1 Processor -- start 2024-04-26T23:47:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.019+0000 7f339a577740 1 -- start start 2024-04-26T23:47:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.019+0000 7f3394c00640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 0x7f337006edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3384034000 tx=0x7f338402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e96de0 0x5585f5e97330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e4f220 0x5585f5e954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5585f5d5f030 con 0x5585f5e79540 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585f5d7a7f0 con 0x5585f5e96de0 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5585f5d357e0 con 0x5585f5e4f220 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f3394c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f3394c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41142/0 (socket says 172.21.15.46:41142) 2024-04-26T23:47:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f3394c00640 1 -- 172.21.15.46:0/1800630529 learned_addr learned my addr 172.21.15.46:0/1800630529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f3395600640 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e4f220 0x5585f5e954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.020+0000 7f338fe00640 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e96de0 0x5585f5e97330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 -- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e4f220 msgr2=0x5585f5e954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e4f220 0x5585f5e954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 -- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e96de0 msgr2=0x5585f5e97330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e96de0 0x5585f5e97330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 -- 172.21.15.46:0/1800630529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5d41050 con 0x5585f5e79540 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3394c00640 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3384035040 tx=0x7f338402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7cd415529db328 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3381200640 1 -- 172.21.15.46:0/1800630529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338404e070 con 0x5585f5e79540 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3381200640 1 -- 172.21.15.46:0/1800630529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384032c30 con 0x5585f5e79540 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.021+0000 7f3381200640 1 -- 172.21.15.46:0/1800630529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384037040 con 0x5585f5e79540 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 -- 172.21.15.46:0/1800630529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 msgr2=0x5585f5e79920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3384035040 tx=0x7f338402c000 comp rx=0 tx=0).stop 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 -- 172.21.15.46:0/1800630529 shutdown_connections 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e4f220 0x5585f5e954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e96de0 0x5585f5e97330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 --2- 172.21.15.46:0/1800630529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e79540 0x5585f5e79920 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.022+0000 7f339a577740 1 -- 172.21.15.46:0/1800630529 >> 172.21.15.46:0/1800630529 conn(0x5585f5e4bb50 msgr2=0x5585f5e94350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.023+0000 7f339a577740 1 -- 172.21.15.46:0/1800630529 shutdown_connections 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.023+0000 7f339a577740 1 -- 172.21.15.46:0/1800630529 wait complete. 2024-04-26T23:47:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.023+0000 7f339a577740 1 -- start start 2024-04-26T23:47:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.024+0000 7f339a577740 1 Processor -- start 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5e66d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5e4d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5585f5db7200 con 0x5585f5e96de0 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585f5db1ee0 con 0x5585f5e7a740 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5585f5dbee70 con 0x5585f5e79540 2024-04-26T23:47:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f339a577740 10 client.0 _ll_register_callbacks cb 0x5585f5da8270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f338fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f3394c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5e66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.025+0000 7f338fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e7a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41150/0 (socket says 172.21.15.46:41150) 2024-04-26T23:47:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f338fe00640 1 -- 172.21.15.46:0/1042874304 learned_addr learned my addr 172.21.15.46:0/1042874304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3395600640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5e4d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3394c00640 1 -- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 msgr2=0x5585f5e4d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3394c00640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5e4d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3394c00640 1 -- 172.21.15.46:0/1042874304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 msgr2=0x5585f5e7a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3394c00640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e7a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.026+0000 7f3394c00640 1 -- 172.21.15.46:0/1042874304 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5b8e7d0 con 0x5585f5e79540 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f3394c00640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5e66d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3384000c00 tx=0x7f3384004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857055c48e5eacb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f3375600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3301/0 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338404e070 con 0x5585f5e79540 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mon.2 v2:172.21.15.142:3301/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f3384053cc0 con 0x5585f5e79540 2024-04-26T23:47:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384032c30 con 0x5585f5e79540 2024-04-26T23:47:53.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3384037040 con 0x5585f5e79540 2024-04-26T23:47:53.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f339a577740 10 client.5798 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:53.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.027+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5585f5ac43f0 con 0x5585f5e79540 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mon.2 v2:172.21.15.142:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384032c30 con 0x5585f5e79540 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mon.2 v2:172.21.15.142:3301/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f338403cbe0 con 0x5585f5e79540 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 10 client.5798 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 10 client.5798 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 20 client.5798 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5ea7920 0x5585f5ea7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 4 client.5798 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f5dc07e0 con 0x5585f5ea7920 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.028+0000 7f338fe00640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5ea7920 0x5585f5ea7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.029+0000 7f338fe00640 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5ea7920 0x5585f5ea7d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf5a72cfd92203b server_cookie=d8f09ae754d4ac6f in_seq=0 out_seq=0 2024-04-26T23:47:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.029+0000 7f3375600640 10 client.5798 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.030+0000 7f3375600640 1 -- 172.21.15.46:0/1042874304 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585f5dc07e0 con 0x5585f5ea7920 2024-04-26T23:47:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.030+0000 7f3375600640 10 client.5798 handle_command_reply: tid=0 2024-04-26T23:47:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 client.5798 shutdown 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 msgr2=0x5585f5e66d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5e66d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3384000c00 tx=0x7f3384004730 comp rx=0 tx=0).stop 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 shutdown_connections 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5ea7920 0x5585f5ea7d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5e4d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e7a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 --2- 172.21.15.46:0/1042874304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5e66d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f3375600640 0 client.5798 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 >> 172.21.15.46:0/1042874304 conn(0x5585f5e4f220 msgr2=0x5585f5e98160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 shutdown_connections 2024-04-26T23:47:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.031+0000 7f339a577740 1 -- 172.21.15.46:0/1042874304 wait complete. 2024-04-26T23:47:53.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.032+0000 7f339a577740 20 client.5798 trim_cache size 0 max 16384 2024-04-26T23:47:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.071+0000 7f339a577740 1 Processor -- start 2024-04-26T23:47:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.071+0000 7f339a577740 1 -- start start 2024-04-26T23:47:53.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.073+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.073+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5eae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.073+0000 7f339a577740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e4dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.074+0000 7f3394c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.074+0000 7f339a577740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3390201ce0 con 0x5585f5e96de0 2024-04-26T23:47:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.074+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3390200d00 con 0x5585f5e7a740 2024-04-26T23:47:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.074+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f33900b9620 con 0x5585f5e79540 2024-04-26T23:47:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.074+0000 7f3394c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:46172/0 (socket says 172.21.15.46:46172) 2024-04-26T23:47:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.075+0000 7f3394c00640 1 -- 172.21.15.46:0/2563244595 learned_addr learned my addr 172.21.15.46:0/2563244595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.075+0000 7f3395600640 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e4dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.076+0000 7f3394c00640 1 -- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 msgr2=0x5585f5eae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.076+0000 7f3394c00640 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5eae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.076+0000 7f3394c00640 1 -- 172.21.15.46:0/2563244595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 msgr2=0x5585f5e4dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.076+0000 7f3394c00640 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e4dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.076+0000 7f3394c00640 1 -- 172.21.15.46:0/2563244595 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5e7a570 con 0x5585f5e79540 2024-04-26T23:47:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.077+0000 7f3395600640 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e4dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:47:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.079+0000 7f3394c00640 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3384032540 tx=0x7f338400b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b70c56703c8ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.080+0000 7f3377e00640 1 -- 172.21.15.46:0/2563244595 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384060070 con 0x5585f5e79540 2024-04-26T23:47:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.080+0000 7f3377e00640 1 -- 172.21.15.46:0/2563244595 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384052540 con 0x5585f5e79540 2024-04-26T23:47:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.080+0000 7f3377e00640 1 -- 172.21.15.46:0/2563244595 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384052930 con 0x5585f5e79540 2024-04-26T23:47:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.081+0000 7f339a577740 1 -- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 msgr2=0x5585f5ea3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.081+0000 7f339a577740 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3384032540 tx=0x7f338400b050 comp rx=0 tx=0).stop 2024-04-26T23:47:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.082+0000 7f339a577740 1 -- 172.21.15.46:0/2563244595 shutdown_connections 2024-04-26T23:47:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.082+0000 7f339a577740 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5e4dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.082+0000 7f339a577740 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e7a740 0x5585f5eae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.083+0000 7f339a577740 1 --2- 172.21.15.46:0/2563244595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e79540 0x5585f5ea3aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.083+0000 7f339a577740 1 -- 172.21.15.46:0/2563244595 >> 172.21.15.46:0/2563244595 conn(0x5585f5e4f220 msgr2=0x5585f5e45130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.083+0000 7f339a577740 1 -- 172.21.15.46:0/2563244595 shutdown_connections 2024-04-26T23:47:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.084+0000 7f339a577740 1 -- 172.21.15.46:0/2563244595 wait complete. 2024-04-26T23:47:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.085+0000 7f339a577740 1 -- start start 2024-04-26T23:47:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.085+0000 7f339a577740 1 Processor -- start 2024-04-26T23:47:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.088+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 0x5585f5e951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.088+0000 7f339a577740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 0x5585f5f0d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.088+0000 7f339a577740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.089+0000 7f339a577740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5585f5dbffd0 con 0x5585f5e96de0 2024-04-26T23:47:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.089+0000 7f3394c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 0x5585f5e951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.089+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33900b9c70 con 0x5585f5e79540 2024-04-26T23:47:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f3395600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f339a577740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5585f5e7a200 con 0x5585f5e7a740 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f339a577740 10 client.0 _ll_register_callbacks cb 0x7f338404c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f3395600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41170/0 (socket says 172.21.15.46:41170) 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f3395600640 1 -- 172.21.15.46:0/3250429617 learned_addr learned my addr 172.21.15.46:0/3250429617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f338fe00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 0x5585f5f0d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.090+0000 7f3394c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 0x5585f5e951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:56472/0 (socket says 172.21.15.46:56472) 2024-04-26T23:47:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f338fe00640 1 -- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 msgr2=0x5585f5e951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f338fe00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 0x5585f5e951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f338fe00640 1 -- 172.21.15.46:0/3250429617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 msgr2=0x5585f5eb7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f338fe00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f338fe00640 1 -- 172.21.15.46:0/3250429617 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5a840f0 con 0x5585f5e7a740 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.091+0000 7f3395600640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:47:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.092+0000 7f338fe00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 0x5585f5f0d070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5585f5e7d5f0 tx=0x7f337c007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25540301c7d34c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.092+0000 7f3375600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3301/0 2024-04-26T23:47:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.093+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c023060 con 0x5585f5e7a740 2024-04-26T23:47:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.094+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mon.2 v2:172.21.15.142:3301/0 2 ==== osd_map(113..113 src has 1..113) ==== 6159+0+0 (secure 0 0 0) 0x7f337c01d3e0 con 0x5585f5e7a740 2024-04-26T23:47:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.095+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384052540 con 0x5585f5e7a740 2024-04-26T23:47:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.096+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f337c01a030 con 0x5585f5e7a740 2024-04-26T23:47:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.096+0000 7f339a577740 10 client.5804 fetch_fsmap learned FSMap version 91 2024-04-26T23:47:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.096+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5585f5d7aa30 con 0x5585f5e7a740 2024-04-26T23:47:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.097+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mon.2 v2:172.21.15.142:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384052540 con 0x5585f5e7a740 2024-04-26T23:47:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.097+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mon.2 v2:172.21.15.142:3301/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f337c01d5e0 con 0x5585f5e7a740 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 10 client.5804 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 10 client.5804 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 20 client.5804 populate_metadata read hostname 'smithi046' 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5e4dc20 0x5585f5e4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 4 client.5804 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:47:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585f5dba650 con 0x5585f5e4dc20 2024-04-26T23:47:53.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.098+0000 7f3394c00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5e4dc20 0x5585f5e4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.100+0000 7f3394c00640 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5e4dc20 0x5585f5e4e000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a3297e8b680ca3 server_cookie=9f057996c4cfd94f in_seq=0 out_seq=0 2024-04-26T23:47:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.101+0000 7f3375600640 10 client.5804 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.102+0000 7f3375600640 1 -- 172.21.15.46:0/3250429617 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5585f5dba650 con 0x5585f5e4dc20 2024-04-26T23:47:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.102+0000 7f3375600640 10 client.5804 handle_command_reply: tid=0 2024-04-26T23:47:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.102+0000 7f339a577740 1 client.5804 shutdown 2024-04-26T23:47:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.102+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 msgr2=0x5585f5f0d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.102+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 0x5585f5f0d070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5585f5e7d5f0 tx=0x7f337c007780 comp rx=0 tx=0).stop 2024-04-26T23:47:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.103+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 shutdown_connections 2024-04-26T23:47:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x5585f5e4dc20 0x5585f5e4e000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f5e96de0 0x5585f5eb7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f3375600640 0 client.5804 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:47:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5585f5e7a740 0x5585f5f0d070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 --2- 172.21.15.46:0/3250429617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585f5e79540 0x5585f5e951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 >> 172.21.15.46:0/3250429617 conn(0x5585f5e4f220 msgr2=0x5585f5ea3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.104+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 shutdown_connections 2024-04-26T23:47:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.105+0000 7f339a577740 1 -- 172.21.15.46:0/3250429617 wait complete. 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "scrub active (609 inodes in the stack)", 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bfee5ecf-1ed5-4b13-b818-c3ec93e664bb": { 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "~mds0", 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "bfee5ecf-1ed5-4b13-b818-c3ec93e664bb", 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:47:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:47:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.106+0000 7f339a577740 20 client.5804 trim_cache size 0 max 16384 2024-04-26T23:47:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.108+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 msgr2=0x7f337006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.108+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 0x7f337006edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3384034000 tx=0x7f338402e000 comp rx=0 tx=0).stop 2024-04-26T23:47:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 msgr2=0x7f33900ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f338800efc0 tx=0x7f338800c490 comp rx=0 tx=0).stop 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 shutdown_connections 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f337006c8f0 0x7f337006edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33901630a0 0x7f33900ce520 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f339015f480 0x7f33900cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 --2- 172.21.15.46:0/289347546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33900d1470 0x7f33900cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.109+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 >> 172.21.15.46:0/289347546 conn(0x7f33900d5070 msgr2=0x7f33900d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.110+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 shutdown_connections 2024-04-26T23:47:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:47:53.110+0000 7f3397600640 1 -- 172.21.15.46:0/289347546 wait complete. 2024-04-26T23:47:53.151 INFO:tasks.cephfs.filesystem:scrub status for tag:bfee5ecf-1ed5-4b13-b818-c3ec93e664bb - {'path': '~mds0', 'tag': 'bfee5ecf-1ed5-4b13-b818-c3ec93e664bb', 'options': 'recursive'} 2024-04-26T23:48:11.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:11.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:23.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:23.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-26T23:48:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.372+0000 7f461b200640 1 Processor -- start 2024-04-26T23:48:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.373+0000 7f461b200640 1 -- start start 2024-04-26T23:48:23.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.374+0000 7f461b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4614162a70 0x7f46140d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.374+0000 7f461b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46140d3830 0x7f46140d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.374+0000 7f461b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140d3400 con 0x7f46140d1ae0 2024-04-26T23:48:23.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.374+0000 7f461b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f46140d2370 con 0x7f46140d3830 2024-04-26T23:48:23.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.374+0000 7f461b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f46140d2650 con 0x7f4614162a70 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46140d3830 0x7f46140d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46140d3830 0x7f46140d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37638/0 (socket says 172.21.15.46:37638) 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 -- 172.21.15.46:0/3454290474 learned_addr learned my addr 172.21.15.46:0/3454290474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619e00640 1 --2- 172.21.15.46:0/3454290474 >> v2:172.21.15.142:3300/0 conn(0x7f4614162a70 0x7f46140d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 -- 172.21.15.46:0/3454290474 >> v2:172.21.15.142:3300/0 conn(0x7f4614162a70 msgr2=0x7f46140d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 --2- 172.21.15.46:0/3454290474 >> v2:172.21.15.142:3300/0 conn(0x7f4614162a70 0x7f46140d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 -- 172.21.15.46:0/3454290474 >> v1:172.21.15.142:6789/0 conn(0x7f46140d1ae0 legacy=0x7f46140cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.375+0000 7f4619400640 1 -- 172.21.15.46:0/3454290474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140d0df0 con 0x7f46140d3830 2024-04-26T23:48:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.376+0000 7f4619400640 1 --2- 172.21.15.46:0/3454290474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140d15a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4614074b40 tx=0x7f460402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6f9878294a37ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f4618a00640 1 -- 172.21.15.46:0/3454290474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604036070 con 0x7f46140d3830 2024-04-26T23:48:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f4618a00640 1 -- 172.21.15.46:0/3454290474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604031600 con 0x7f46140d3830 2024-04-26T23:48:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f4618a00640 1 -- 172.21.15.46:0/3454290474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040319d0 con 0x7f46140d3830 2024-04-26T23:48:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 -- 172.21.15.46:0/3454290474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 msgr2=0x7f46140d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 --2- 172.21.15.46:0/3454290474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140d15a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4614074b40 tx=0x7f460402a780 comp rx=0 tx=0).stop 2024-04-26T23:48:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 -- 172.21.15.46:0/3454290474 shutdown_connections 2024-04-26T23:48:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 --2- 172.21.15.46:0/3454290474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140d15a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 --2- 172.21.15.46:0/3454290474 >> v2:172.21.15.142:3300/0 conn(0x7f4614162a70 0x7f46140d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.377+0000 7f461b200640 1 -- 172.21.15.46:0/3454290474 >> 172.21.15.46:0/3454290474 conn(0x7f46140d4f20 msgr2=0x7f46140d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.378+0000 7f461b200640 1 -- 172.21.15.46:0/3454290474 shutdown_connections 2024-04-26T23:48:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.378+0000 7f461b200640 1 -- 172.21.15.46:0/3454290474 wait complete. 2024-04-26T23:48:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.380+0000 7f461b200640 1 Processor -- start 2024-04-26T23:48:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.380+0000 7f461b200640 1 -- start start 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 0x7f46140cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 0x7f46140cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46140d2070 con 0x7f46140d3830 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46140d1ef0 con 0x7f46140d1ae0 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.381+0000 7f461b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46140b39f0 con 0x7f4614162a70 2024-04-26T23:48:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37642/0 (socket says 172.21.15.46:37642) 2024-04-26T23:48:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 -- 172.21.15.46:0/2733245882 learned_addr learned my addr 172.21.15.46:0/2733245882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619e00640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 0x7f46140cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 -- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 msgr2=0x7f46140cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f461a800640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 0x7f46140cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 0x7f46140cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 -- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 msgr2=0x7f46140cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 0x7f46140cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619400640 1 -- 172.21.15.46:0/2733245882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140cd980 con 0x7f46140d3830 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f4619e00640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 0x7f46140cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.382+0000 7f461a800640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 0x7f46140cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f4619400640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4604034040 tx=0x7f4604004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f460e000640 1 -- 172.21.15.46:0/2733245882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604036050 con 0x7f46140d3830 2024-04-26T23:48:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f460e000640 1 -- 172.21.15.46:0/2733245882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604042070 con 0x7f46140d3830 2024-04-26T23:48:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f460e000640 1 -- 172.21.15.46:0/2733245882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604009510 con 0x7f46140d3830 2024-04-26T23:48:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46140ba510 con 0x7f46140d3830 2024-04-26T23:48:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.383+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46140cd3c0 con 0x7f46140d3830 2024-04-26T23:48:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.387+0000 7f460e000640 1 -- 172.21.15.46:0/2733245882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4604007c70 con 0x7f46140d3830 2024-04-26T23:48:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.388+0000 7f460e000640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.388+0000 7f460e000640 1 -- 172.21.15.46:0/2733245882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f46040acd80 con 0x7f46140d3830 2024-04-26T23:48:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.388+0000 7f461f799740 1 Processor -- start 2024-04-26T23:48:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.388+0000 7f4619e00640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.388+0000 7f461f799740 1 -- start start 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bdbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d370ac1030 con 0x55d370bb13f0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d370adc7f0 con 0x55d370bdb710 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d370a977e0 con 0x55d370bf8fb0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.389+0000 7f4619e00640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 0x7f45f806edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f460800b820 tx=0x7f4608008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37652/0 (socket says 172.21.15.46:37652) 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 -- 172.21.15.46:0/3340121251 learned_addr learned my addr 172.21.15.46:0/3340121251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f4619e00640 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bdbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f4619400640 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 -- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 msgr2=0x55d370bf9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bf9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 -- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 msgr2=0x55d370bdbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bdbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.390+0000 7f461a800640 1 -- 172.21.15.46:0/3340121251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d370aa3050 con 0x55d370bb13f0 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f461a800640 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4610001740 tx=0x7f4610001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9273c17b917a81f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f4602600640 1 -- 172.21.15.46:0/3340121251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610018070 con 0x55d370bb13f0 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f4602600640 1 -- 172.21.15.46:0/3340121251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461000d730 con 0x55d370bb13f0 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f4602600640 1 -- 172.21.15.46:0/3340121251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000db20 con 0x55d370bb13f0 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f461f799740 1 -- 172.21.15.46:0/3340121251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 msgr2=0x55d370bf7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.391+0000 7f461f799740 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4610001740 tx=0x7f4610001c00 comp rx=0 tx=0).stop 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 -- 172.21.15.46:0/3340121251 shutdown_connections 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bb13f0 0x55d370bf7670 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bf9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 --2- 172.21.15.46:0/3340121251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bdbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 -- 172.21.15.46:0/3340121251 >> 172.21.15.46:0/3340121251 conn(0x55d370badd20 msgr2=0x55d370bf6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 -- 172.21.15.46:0/3340121251 shutdown_connections 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 -- 172.21.15.46:0/3340121251 wait complete. 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.392+0000 7f461f799740 1 -- start start 2024-04-26T23:48:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.393+0000 7f461f799740 1 Processor -- start 2024-04-26T23:48:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bed5d0 0x55d370bf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d370b19200 con 0x55d370bf8fb0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d370b13ee0 con 0x55d370bdb710 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d370b20e70 con 0x55d370bed5d0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461f799740 10 client.0 _ll_register_callbacks cb 0x55d370b0a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37664/0 (socket says 172.21.15.46:37664) 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 -- 172.21.15.46:0/3218434861 learned_addr learned my addr 172.21.15.46:0/3218434861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f461a800640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bed5d0 0x55d370bf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 -- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bed5d0 msgr2=0x55d370bf9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bed5d0 0x55d370bf9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619e00640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 -- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 msgr2=0x55d370bccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.394+0000 7f4619400640 1 -- 172.21.15.46:0/3218434861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3708f07d0 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f4619400640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4604044ff0 tx=0x7f46040316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c9c28352ff549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f45f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460407bd70 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f4604079c90 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461000d730 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f460407a840 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f461f799740 10 client.8484 fetch_fsmap learned FSMap version 91 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.395+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d3708263f0 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461000d730 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f4604092050 con 0x55d370bf8fb0 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 10 client.8484 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 10 client.8484 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 20 client.8484 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370c060d0 0x55d370c06530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 4 client.8484 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:48:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.396+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d370b227e0 con 0x55d370c060d0 2024-04-26T23:48:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.397+0000 7f4619e00640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370c060d0 0x55d370c06530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.398+0000 7f4619e00640 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370c060d0 0x55d370c06530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12dbd8f9f3d0381 server_cookie=4ef1adb97dc37bb2 in_seq=0 out_seq=0 2024-04-26T23:48:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.398+0000 7f45f6a00640 10 client.8484 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.399+0000 7f45f6a00640 1 -- 172.21.15.46:0/3218434861 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d370b227e0 con 0x55d370c060d0 2024-04-26T23:48:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.399+0000 7f45f6a00640 10 client.8484 handle_command_reply: tid=0 2024-04-26T23:48:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.399+0000 7f461f799740 1 client.8484 shutdown 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 msgr2=0x55d370bf9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4604044ff0 tx=0x7f46040316c0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 shutdown_connections 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370c060d0 0x55d370c06530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bed5d0 0x55d370bf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bf8fb0 0x55d370bf9aa0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 --2- 172.21.15.46:0/3218434861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370bccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f45f6a00640 0 client.8484 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 >> 172.21.15.46:0/3218434861 conn(0x55d370bb13f0 msgr2=0x55d370baf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 shutdown_connections 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 1 -- 172.21.15.46:0/3218434861 wait complete. 2024-04-26T23:48:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.400+0000 7f461f799740 20 client.8484 trim_cache size 0 max 16384 2024-04-26T23:48:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.420+0000 7f461f799740 1 Processor -- start 2024-04-26T23:48:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.420+0000 7f461f799740 1 -- start start 2024-04-26T23:48:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370ba72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bd6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4614201da0 con 0x55d370bed5d0 2024-04-26T23:48:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4614200df0 con 0x55d370bdb710 2024-04-26T23:48:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f461415c980 con 0x55d370bf8fb0 2024-04-26T23:48:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37674/0 (socket says 172.21.15.46:37674) 2024-04-26T23:48:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 -- 172.21.15.46:0/628744551 learned_addr learned my addr 172.21.15.46:0/628744551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 -- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 msgr2=0x55d370bd6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bd6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 -- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 msgr2=0x55d370ba72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370ba72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.421+0000 7f4619400640 1 -- 172.21.15.46:0/628744551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3707e60f0 con 0x55d370bed5d0 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f4619400640 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4604007fd0 tx=0x7f4604049070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b5bb65136e960d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f4601c00640 1 -- 172.21.15.46:0/628744551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460407ec40 con 0x55d370bed5d0 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f4601c00640 1 -- 172.21.15.46:0/628744551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46040a9070 con 0x55d370bed5d0 2024-04-26T23:48:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f4601c00640 1 -- 172.21.15.46:0/628744551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604079e90 con 0x55d370bed5d0 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f461f799740 1 -- 172.21.15.46:0/628744551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 msgr2=0x55d370c066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.422+0000 7f461f799740 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4604007fd0 tx=0x7f4604049070 comp rx=0 tx=0).stop 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 -- 172.21.15.46:0/628744551 shutdown_connections 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bf8fb0 0x55d370bd6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c066a0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 --2- 172.21.15.46:0/628744551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bdb710 0x55d370ba72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 -- 172.21.15.46:0/628744551 >> 172.21.15.46:0/628744551 conn(0x55d370bb13f0 msgr2=0x55d370bb0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 -- 172.21.15.46:0/628744551 shutdown_connections 2024-04-26T23:48:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 -- 172.21.15.46:0/628744551 wait complete. 2024-04-26T23:48:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.423+0000 7f461f799740 1 -- start start 2024-04-26T23:48:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.424+0000 7f461f799740 1 Processor -- start 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bdb710 0x55d370c04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bf8fb0 0x55d370c18f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d370b21fd0 con 0x55d370bed5d0 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f461415cfd0 con 0x55d370bf8fb0 2024-04-26T23:48:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d370c197e0 con 0x55d370bdb710 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461f799740 10 client.0 _ll_register_callbacks cb 0x7f460402acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37686/0 (socket says 172.21.15.46:37686) 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 -- 172.21.15.46:0/1128536710 learned_addr learned my addr 172.21.15.46:0/1128536710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f461a800640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bf8fb0 0x55d370c18f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619e00640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bdb710 0x55d370c04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 -- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bdb710 msgr2=0x55d370c04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bdb710 0x55d370c04e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 -- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bf8fb0 msgr2=0x55d370c18f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bf8fb0 0x55d370c18f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.425+0000 7f4619400640 1 -- 172.21.15.46:0/1128536710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d370adca30 con 0x55d370bed5d0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f4619400640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f460407a780 tx=0x7f46040aafa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cffdc076f8e2220 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f45f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604088060 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f4604076590 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46040a9070 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f460408e050 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f461f799740 10 client.8490 fetch_fsmap learned FSMap version 91 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.426+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d370adca30 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46040a9070 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f4604076790 con 0x55d370bed5d0 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 10 client.8490 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 10 client.8490 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 20 client.8490 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370bf0a40 0x55d370c65950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 4 client.8490 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:48:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d370b1c650 con 0x55d370bf0a40 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.427+0000 7f4619e00640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370bf0a40 0x55d370c65950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.428+0000 7f4619e00640 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370bf0a40 0x55d370c65950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e5328f472119a7 server_cookie=4845485b580ecc08 in_seq=0 out_seq=0 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.428+0000 7f45f6a00640 10 client.8490 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f45f6a00640 1 -- 172.21.15.46:0/1128536710 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d370b1c650 con 0x55d370bf0a40 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f45f6a00640 10 client.8490 handle_command_reply: tid=0 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 client.8490 shutdown 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 msgr2=0x55d370c189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f460407a780 tx=0x7f46040aafa0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 shutdown_connections 2024-04-26T23:48:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x55d370bf0a40 0x55d370c65950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d370bf8fb0 0x55d370c18f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d370bed5d0 0x55d370c189e0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 --2- 172.21.15.46:0/1128536710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d370bdb710 0x55d370c04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f45f6a00640 0 client.8490 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 >> 172.21.15.46:0/1128536710 conn(0x55d370bb13f0 msgr2=0x55d370bef830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 shutdown_connections 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.429+0000 7f461f799740 1 -- 172.21.15.46:0/1128536710 wait complete. 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461f799740 20 client.8490 trim_cache size 0 max 16384 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:48:23.436 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:48:23.437 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:48:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 msgr2=0x7f45f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 0x7f45f806edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f460800b820 tx=0x7f4608008040 comp rx=0 tx=0).stop 2024-04-26T23:48:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 msgr2=0x7f46140b9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4604034040 tx=0x7f4604004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 shutdown_connections 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4614162a70 0x7f46140cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46140d3830 0x7f46140b9fd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 --2- 172.21.15.46:0/2733245882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46140d1ae0 0x7f46140cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 >> 172.21.15.46:0/2733245882 conn(0x7f46140d4f20 msgr2=0x7f46141616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 shutdown_connections 2024-04-26T23:48:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.430+0000 7f461b200640 1 -- 172.21.15.46:0/2733245882 wait complete. 2024-04-26T23:48:23.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:48:23.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 perf dump 2024-04-26T23:48:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.670+0000 7f44ca800640 1 Processor -- start 2024-04-26T23:48:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.670+0000 7f44ca800640 1 -- start start 2024-04-26T23:48:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44ca800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44c40d04c0 0x7f44c406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44ca800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c40b4b30 con 0x7f44c406f2e0 2024-04-26T23:48:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44ca800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c40b4e10 con 0x7f44c40b44f0 2024-04-26T23:48:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44ca800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f44c40b48f0 con 0x7f44c40d04c0 2024-04-26T23:48:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44c2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f44c406f2e0 0x7f44c40b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40344/0 (socket says 172.21.15.46:40344) 2024-04-26T23:48:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44c2a00640 1 -- 172.21.15.46:0/3734381276 learned_addr learned my addr 172.21.15.46:0/3734381276 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.672+0000 7f44c3400640 1 --2- 172.21.15.46:0/3734381276 >> v2:172.21.15.142:3300/0 conn(0x7f44c40d04c0 0x7f44c406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591120749 0 0) 0x7f44c40b4b30 con 0x7f44c406f2e0 2024-04-26T23:48:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44a8003520 con 0x7f44c406f2e0 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2717358235 0 0) 0x7f44a8003520 con 0x7f44c406f2e0 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 >> v2:172.21.15.142:3300/0 conn(0x7f44c40d04c0 msgr2=0x7f44c406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 --2- 172.21.15.46:0/3734381276 >> v2:172.21.15.142:3300/0 conn(0x7f44c40d04c0 0x7f44c406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 >> v1:172.21.15.142:6789/0 conn(0x7f44c40b44f0 legacy=0x7f44c40ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c40b50f0 con 0x7f44c406f2e0 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f44ac001670 con 0x7f44c406f2e0 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 >> v1:172.21.15.46:6789/0 conn(0x7f44c406f2e0 legacy=0x7f44c40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8003a00 0x7f44a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44a8005210 0x7f44a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40b48f0 con 0x7f44a8004800 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44a800c160 con 0x7f44a8005210 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.673+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f44a800c360 con 0x7f44a8003a00 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c3400640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8003a00 0x7f44a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c3e00640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44a8005210 0x7f44a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 -- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8003a00 msgr2=0x7f44a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8003a00 0x7f44a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 -- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44a8005210 msgr2=0x7f44a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44a8005210 0x7f44a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c2a00640 1 -- 172.21.15.46:0/3734381276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a8003520 con 0x7f44a8004800 2024-04-26T23:48:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c3e00640 1 -- 172.21.15.46:0/3734381276 reap_dead start 2024-04-26T23:48:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.674+0000 7f44c3400640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8003a00 0x7f44a8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.676+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c0e0 tx=0x7f44a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6a4da630780d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.676+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac007c40 con 0x7f44a8004800 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.676+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ac005af0 con 0x7f44a8004800 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.676+0000 7f44c2000640 1 -- 172.21.15.46:0/3734381276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac007070 con 0x7f44a8004800 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 -- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 msgr2=0x7f44a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c0e0 tx=0x7f44a8005830 comp rx=0 tx=0).stop 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 -- 172.21.15.46:0/3734381276 shutdown_connections 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 --2- 172.21.15.46:0/3734381276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 -- 172.21.15.46:0/3734381276 >> 172.21.15.46:0/3734381276 conn(0x7f44c406c980 msgr2=0x7f44c40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 -- 172.21.15.46:0/3734381276 shutdown_connections 2024-04-26T23:48:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.677+0000 7f44ca800640 1 -- 172.21.15.46:0/3734381276 wait complete. 2024-04-26T23:48:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.679+0000 7f44ca800640 1 Processor -- start 2024-04-26T23:48:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.679+0000 7f44ca800640 1 -- start start 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 0x7f44c40c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c40c6060 0x7f44c415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40b3de0 con 0x7f44c415b0c0 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40b56b0 con 0x7f44c40c6060 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44ca800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f44c40b3f60 con 0x7f44a8004800 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37700/0 (socket says 172.21.15.46:37700) 2024-04-26T23:48:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.680+0000 7f44c3e00640 1 -- 172.21.15.46:0/1492557151 learned_addr learned my addr 172.21.15.46:0/1492557151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3400640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 0x7f44c40c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c2a00640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c40c6060 0x7f44c415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 -- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 msgr2=0x7f44c40c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 0x7f44c40c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 -- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c40c6060 msgr2=0x7f44c415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c40c6060 0x7f44c415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 -- 172.21.15.46:0/1492557151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c415ccb0 con 0x7f44c415b0c0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3400640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 0x7f44c40c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f44bc007fd0 tx=0x7f44bc00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44bb400640 1 -- 172.21.15.46:0/1492557151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc019070 con 0x7f44c415b0c0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44bb400640 1 -- 172.21.15.46:0/1492557151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc006d60 con 0x7f44c415b0c0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.681+0000 7f44bb400640 1 -- 172.21.15.46:0/1492557151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc003b90 con 0x7f44c415b0c0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.682+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c415cf50 con 0x7f44c415b0c0 2024-04-26T23:48:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.682+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c415d510 con 0x7f44c415b0c0 2024-04-26T23:48:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.684+0000 7f44bb400640 1 -- 172.21.15.46:0/1492557151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44bc01c070 con 0x7f44c415b0c0 2024-04-26T23:48:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.685+0000 7f44bb400640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 0x7f44a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.685+0000 7f44bb400640 1 -- 172.21.15.46:0/1492557151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f44bc08b330 con 0x7f44c415b0c0 2024-04-26T23:48:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.685+0000 7f44c3400640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 0x7f44a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.686+0000 7f44cd66d740 1 Processor -- start 2024-04-26T23:48:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.686+0000 7f44cd66d740 1 -- start start 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c3400640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 0x7f44a006edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44c40cf610 tx=0x7f44b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c1a7f2900 con 0x561c1a88ec30 2024-04-26T23:48:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561c1a78f1d0 con 0x561c1a8b8f50 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561c1a7ee8c0 con 0x561c1a8d67f0 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37704/0 (socket says 172.21.15.46:37704) 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c3e00640 1 -- 172.21.15.46:0/1771000848 learned_addr learned my addr 172.21.15.46:0/1771000848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c2a00640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.687+0000 7f44c3400640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 -- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 msgr2=0x561c1a8d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 -- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 msgr2=0x561c1a8b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 -- 172.21.15.46:0/1771000848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1a5ce7d0 con 0x561c1a88ec30 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3400640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc058e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4afc6a32d77dc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44b3e00640 1 -- 172.21.15.46:0/1771000848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc061a50 con 0x561c1a88ec30 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44b3e00640 1 -- 172.21.15.46:0/1771000848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc057170 con 0x561c1a88ec30 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.688+0000 7f44b3e00640 1 -- 172.21.15.46:0/1771000848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc071180 con 0x561c1a88ec30 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 -- 172.21.15.46:0/1771000848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 msgr2=0x561c1a8d4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc058e70 comp rx=0 tx=0).stop 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 -- 172.21.15.46:0/1771000848 shutdown_connections 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a88ec30 0x561c1a8d4eb0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1771000848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 -- 172.21.15.46:0/1771000848 >> 172.21.15.46:0/1771000848 conn(0x561c1a88b560 msgr2=0x561c1a8d3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 -- 172.21.15.46:0/1771000848 shutdown_connections 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.689+0000 7f44cd66d740 1 -- 172.21.15.46:0/1771000848 wait complete. 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.690+0000 7f44cd66d740 1 -- start start 2024-04-26T23:48:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.690+0000 7f44cd66d740 1 Processor -- start 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8b8f50 0x561c1a8bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8d2890 0x561c1a8d5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c1a88f9d0 con 0x561c1a8d67f0 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561c1a77a120 con 0x561c1a8d2890 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561c1a889490 con 0x561c1a8b8f50 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44cd66d740 10 client.0 _ll_register_callbacks cb 0x561c1a7b2750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37718/0 (socket says 172.21.15.46:37718) 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.691+0000 7f44c2a00640 1 -- 172.21.15.46:0/1776151802 learned_addr learned my addr 172.21.15.46:0/1776151802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c3400640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8b8f50 0x561c1a8bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c3e00640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8d2890 0x561c1a8d5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 -- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8b8f50 msgr2=0x561c1a8bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8b8f50 0x561c1a8bb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 -- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8d2890 msgr2=0x561c1a8d5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8d2890 0x561c1a8d5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 -- 172.21.15.46:0/1776151802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1a4c40f0 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44c2a00640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c200 tx=0x7f44ac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae1cd40114dd432 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac001680 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f44ac0047e0 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.692+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44bc057170 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f44ac0049e0 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 10 client.8502 fetch_fsmap learned FSMap version 91 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x561c1a62ef60 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ac04b070 con 0x561c1a8d67f0 2024-04-26T23:48:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f44ac03d040 con 0x561c1a8d67f0 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 10 client.8502 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 10 client.8502 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 20 client.8502 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8e58a0 0x561c1a8e5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 4 client.8502 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.693+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1a800510 con 0x561c1a8e58a0 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.694+0000 7f44c3400640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8e58a0 0x561c1a8e5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.695+0000 7f44c3400640 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8e58a0 0x561c1a8e5c80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebca4d8dae2d9766 server_cookie=606f7dbd6d93bacb in_seq=0 out_seq=0 2024-04-26T23:48:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.695+0000 7f44b0c00640 10 client.8502 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44b0c00640 1 -- 172.21.15.46:0/1776151802 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c1a800510 con 0x561c1a8e58a0 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44b0c00640 10 client.8502 handle_command_reply: tid=0 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 client.8502 shutdown 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 msgr2=0x561c1a8d4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c200 tx=0x7f44ac02c000 comp rx=0 tx=0).stop 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 shutdown_connections 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8e58a0 0x561c1a8e5c80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8d2890 0x561c1a8d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d67f0 0x561c1a8d4ce0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 --2- 172.21.15.46:0/1776151802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8b8f50 0x561c1a8bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 >> 172.21.15.46:0/1776151802 conn(0x561c1a88ec30 msgr2=0x561c1a8d72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44b0c00640 0 client.8502 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 shutdown_connections 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.697+0000 7f44cd66d740 1 -- 172.21.15.46:0/1776151802 wait complete. 2024-04-26T23:48:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.698+0000 7f44cd66d740 20 client.8502 trim_cache size 0 max 16384 2024-04-26T23:48:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.715+0000 7f44cd66d740 1 Processor -- start 2024-04-26T23:48:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.715+0000 7f44cd66d740 1 -- start start 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c42027f0 con 0x561c1a8d2890 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4201810 con 0x561c1a8b8f50 2024-04-26T23:48:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f44c415ee30 con 0x561c1a8d67f0 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.716+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37732/0 (socket says 172.21.15.46:37732) 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 -- 172.21.15.46:0/2900569432 learned_addr learned my addr 172.21.15.46:0/2900569432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c3e00640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c3400640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 -- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 msgr2=0x561c1a8d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 -- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 msgr2=0x561c1a8d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 -- 172.21.15.46:0/2900569432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1a49d1b0 con 0x561c1a8d2890 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c3400640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c3e00640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.717+0000 7f44c2a00640 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02ad20 tx=0x7f44ac03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cd20a2430215c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44b3400640 1 -- 172.21.15.46:0/2900569432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac001680 con 0x561c1a8d2890 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44b3400640 1 -- 172.21.15.46:0/2900569432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ac04b040 con 0x561c1a8d2890 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44b3400640 1 -- 172.21.15.46:0/2900569432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac046610 con 0x561c1a8d2890 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 -- 172.21.15.46:0/2900569432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 msgr2=0x561c1a8ede70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02ad20 tx=0x7f44ac03e000 comp rx=0 tx=0).stop 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 -- 172.21.15.46:0/2900569432 shutdown_connections 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8ede70 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 --2- 172.21.15.46:0/2900569432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a8d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 -- 172.21.15.46:0/2900569432 >> 172.21.15.46:0/2900569432 conn(0x561c1a88ec30 msgr2=0x561c1a88f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 -- 172.21.15.46:0/2900569432 shutdown_connections 2024-04-26T23:48:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.718+0000 7f44cd66d740 1 -- 172.21.15.46:0/2900569432 wait complete. 2024-04-26T23:48:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.719+0000 7f44cd66d740 1 -- start start 2024-04-26T23:48:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.719+0000 7f44cd66d740 1 Processor -- start 2024-04-26T23:48:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a936dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c42011c0 con 0x561c1a8d2890 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561c1a88dec0 con 0x561c1a8b8f50 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561c1a94db80 con 0x561c1a8d67f0 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44cd66d740 10 client.0 _ll_register_callbacks cb 0x7f44ac02af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.720+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37744/0 (socket says 172.21.15.46:37744) 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 -- 172.21.15.46:0/3659771447 learned_addr learned my addr 172.21.15.46:0/3659771447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 -- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 msgr2=0x561c1a8d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 -- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 msgr2=0x561c1a936dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a936dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 -- 172.21.15.46:0/3659771447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1a8c96f0 con 0x561c1a8d2890 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44c2a00640 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c0e0 tx=0x7f44ac001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f67c84c4f96d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.721+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac046410 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f44ac0465b0 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44ac04b040 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f44ac0467b0 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 10 client.8508 fetch_fsmap learned FSMap version 91 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x561c1a889060 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ac04b040 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) ==== 3494+0+0 (secure 0 0 0) 0x7f44ac069020 con 0x561c1a8d2890 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 10 client.8508 fetch_fsmap finished waiting for FSMap version 91 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 10 client.8508 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 20 client.8508 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8d50b0 0x561c1a88ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 4 client.8508 mds_command: new command op to 8253 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.722+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 --> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] -- command(tid 0: {"prefix": "perf dump"}) -- 0x561c1a7fa380 con 0x561c1a8d50b0 2024-04-26T23:48:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.723+0000 7f44c3400640 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8d50b0 0x561c1a88ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.724+0000 7f44c3400640 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8d50b0 0x561c1a88ca00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aafb6b1414917d7 server_cookie=e3c0c13b789d0974 in_seq=0 out_seq=0 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.724+0000 7f44b0c00640 10 client.8508 ms_handle_connect on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.725+0000 7f44b0c00640 1 -- 172.21.15.46:0/3659771447 <== mds.0 v2:172.21.15.142:6836/2078483956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18002 (crc 0 0 0) 0x561c1a7fa380 con 0x561c1a8d50b0 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.725+0000 7f44b0c00640 10 client.8508 handle_command_reply: tid=0 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 client.8508 shutdown 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 msgr2=0x561c1a8b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c0e0 tx=0x7f44ac001dd0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 shutdown_connections 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:6836/2078483956,v1:172.21.15.142:6837/2078483956] conn(0x561c1a8d50b0 0x561c1a88ca00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561c1a8d67f0 0x561c1a8d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c1a8d2890 0x561c1a8b7d00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 --2- 172.21.15.46:0/3659771447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1a8b8f50 0x561c1a936dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 >> 172.21.15.46:0/3659771447 conn(0x561c1a88ec30 msgr2=0x561c1a88f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44b0c00640 0 client.8508 ms_handle_reset on v2:172.21.15.142:6836/2078483956 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 shutdown_connections 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 1 -- 172.21.15.46:0/3659771447 wait complete. 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.726+0000 7f44cd66d740 20 client.8508 trim_cache size 0 max 16384 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:48:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2670, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2671, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1087403, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 21897720, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 17, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.726232736, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.300303329, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.106531658, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.280256395, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2670, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.684215015, 2024-04-26T23:48:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000256260 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 10, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000001324, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000132 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3203, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 3238, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 813733, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 25864940, 2024-04-26T23:48:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.947076787, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.255328672, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.114990663, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.532054944, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3237, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.760056356, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000234802 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 150192 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 29930, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 51750, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 10302219, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 70312267, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 7, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 7.366010859, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 4.415491138, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 3.063921573, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.187960395, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 51750, 2024-04-26T23:48:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.588583916, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000088668 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 45488, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.028078492, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000617 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 128592, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 38128 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-26T23:48:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2832, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 32.471631479, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.011465971 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4326, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.242067176, 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000518277 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 25725, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 25725, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 25725, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 524.741006933, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.020398095 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 12, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 17898, 2024-04-26T23:48:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 17898, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 17897, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 2, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 2, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 2, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 46728, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 46204, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 105, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-04-26T23:48:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 2011, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 2011, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:48:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 11, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 1, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 10, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 2000, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 904, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 4019, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 2905, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 2029, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 63, 2024-04-26T23:48:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 14122 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 4148, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 3947, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 2200, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-26T23:48:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 1, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 26175, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 26175, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 35, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 35, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 4, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-04-26T23:48:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 95966264, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2633, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 120.039074484, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.045590229 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 14, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 2015, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 2001, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 13, 2024-04-26T23:48:23.746 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 13, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 0, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 17898, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 19899, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 2001, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 2, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 8396, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 8394, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 98252, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231788 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 28626, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 25725, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 10, 2024-04-26T23:48:23.747 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2000, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 5.101454783, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002550727 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 30, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.013439196, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000447973 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.748 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6000, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 20.213922429, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003368987 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7481, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 9.102754932, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001216783 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-26T23:48:23.749 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-26T23:48:23.750 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000200043, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000200043 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.006559292, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006559292 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-26T23:48:23.751 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.044697434, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.044697434 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000297787, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000297787 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-26T23:48:23.752 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.185437877, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.023179734 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2200, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 449.676547235, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.204398430 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.753 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.019167581, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.019167581 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.754 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000653771, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000653771 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.756 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8000, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 40.375874573, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.005046984 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 1, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 0, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 13715, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 13715, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 161, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:48:23.758 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-26T23:48:23.759 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-26T23:48:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 250798, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 93, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 528, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 6, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 5720, 2024-04-26T23:48:23.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 82, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 13478701, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 661627, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-26T23:48:23.762 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 9016, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 92473486, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 31, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 9016, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 9016, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 58.903863686, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006533259 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 9016, 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 9193 2024-04-26T23:48:23.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 9016, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 2106, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 6910, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 11, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 94, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 2726, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 57, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 1947, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 2033, 2024-04-26T23:48:23.764 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 2012, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 22, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-26T23:48:23.765 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 115, 2024-04-26T23:48:23.766 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 5, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 8, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 8, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 240, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 51 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-26T23:48:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 32, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 32, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 3947, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 3947, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:48:23.768 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 35798, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 9401696, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 35798, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 35798, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 9401696, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:48:23.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 9016, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 92887709, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 9016, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 92887709, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 9016, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 9016, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 9016, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 9016, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.771 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 3947, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 414223, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 3947, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 93, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 414223, 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:48:23.772 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x55be54b5c500": { 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 26175, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 91771960, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 26175, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-26T23:48:23.773 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 2633, 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 91771960, 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:23.774 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:48:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 msgr2=0x7f44a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 0x7f44a006edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44c40cf610 tx=0x7f44b4008040 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 msgr2=0x7f44c415b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f44bc007fd0 tx=0x7f44bc00ea00 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 shutdown_connections 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f44a006c8f0 0x7f44a006edb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c415b0c0 0x7f44c415b540 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c40c6060 0x7f44c415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 --2- 172.21.15.46:0/1492557151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44a8004800 0x7f44c40c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 >> 172.21.15.46:0/1492557151 conn(0x7f44c406c980 msgr2=0x7f44c4139900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 shutdown_connections 2024-04-26T23:48:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:23.727+0000 7f44ca800640 1 -- 172.21.15.46:0/1492557151 wait complete. 2024-04-26T23:48:23.777 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:48:23.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:48:23.894 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:48:24.005 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:48:24.005 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:48:24.005 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:24.006 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:24.066 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:24.066 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:24.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.335+0000 7fec03e00640 1 Processor -- start 2024-04-26T23:48:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.336+0000 7fec03e00640 1 -- start start 2024-04-26T23:48:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.337+0000 7fec03e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec040d2130 0x7fec040d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.337+0000 7fec03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec040b4000 0x7fec040b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.337+0000 7fec03e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec040d11f0 con 0x7fec0415f4e0 2024-04-26T23:48:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.337+0000 7fec03e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fec040d0ce0 con 0x7fec040b4000 2024-04-26T23:48:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.337+0000 7fec03e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fec040d2990 con 0x7fec040d2130 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec03400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec040b4000 0x7fec040b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec02a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fec0415f4e0 0x7fec040d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40350/0 (socket says 172.21.15.46:40350) 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec02a00640 1 -- 172.21.15.46:0/3535214530 learned_addr learned my addr 172.21.15.46:0/3535214530 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec02000640 1 --2- 172.21.15.46:0/3535214530 >> v2:172.21.15.142:3300/0 conn(0x7fec040d2130 0x7fec040d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec01600640 1 -- 172.21.15.46:0/3535214530 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546640230 0 0) 0x7fec040d11f0 con 0x7fec0415f4e0 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec01600640 1 -- 172.21.15.46:0/3535214530 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe8003520 con 0x7fec0415f4e0 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec03400640 1 -- 172.21.15.46:0/3535214530 >> v2:172.21.15.142:3300/0 conn(0x7fec040d2130 msgr2=0x7fec040d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec03400640 1 --2- 172.21.15.46:0/3535214530 >> v2:172.21.15.142:3300/0 conn(0x7fec040d2130 0x7fec040d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec03400640 1 -- 172.21.15.46:0/3535214530 >> v1:172.21.15.46:6789/0 conn(0x7fec0415f4e0 legacy=0x7fec040d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.338+0000 7fec03400640 1 -- 172.21.15.46:0/3535214530 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec040d5f20 con 0x7fec040b4000 2024-04-26T23:48:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.339+0000 7fec03400640 1 --2- 172.21.15.46:0/3535214530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040b4000 0x7fec040b2280 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313166f3fff7544a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec01600640 1 -- 172.21.15.46:0/3535214530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc007740 con 0x7fec040b4000 2024-04-26T23:48:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec01600640 1 -- 172.21.15.46:0/3535214530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc0078e0 con 0x7fec040b4000 2024-04-26T23:48:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec01600640 1 -- 172.21.15.46:0/3535214530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc007cb0 con 0x7fec040b4000 2024-04-26T23:48:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 -- 172.21.15.46:0/3535214530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040b4000 msgr2=0x7fec040b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 --2- 172.21.15.46:0/3535214530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040b4000 0x7fec040b2280 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 -- 172.21.15.46:0/3535214530 shutdown_connections 2024-04-26T23:48:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 --2- 172.21.15.46:0/3535214530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040b4000 0x7fec040b2280 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 --2- 172.21.15.46:0/3535214530 >> v2:172.21.15.142:3300/0 conn(0x7fec040d2130 0x7fec040d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 -- 172.21.15.46:0/3535214530 >> 172.21.15.46:0/3535214530 conn(0x7fec040d5010 msgr2=0x7fec040d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:24.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 -- 172.21.15.46:0/3535214530 shutdown_connections 2024-04-26T23:48:24.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.340+0000 7fec03e00640 1 -- 172.21.15.46:0/3535214530 wait complete. 2024-04-26T23:48:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.342+0000 7fec03e00640 1 Processor -- start 2024-04-26T23:48:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.342+0000 7fec03e00640 1 -- start start 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec040b4000 0x7fec040b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec0415f4e0 0x7fec040a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec040b1d60 con 0x7fec040d2130 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec040d5c40 con 0x7fec0415f4e0 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.343+0000 7fec03e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec0415f8c0 con 0x7fec040b4000 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37760/0 (socket says 172.21.15.46:37760) 2024-04-26T23:48:24.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 -- 172.21.15.46:0/3559571002 learned_addr learned my addr 172.21.15.46:0/3559571002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02a00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec040b4000 0x7fec040b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 -- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec040b4000 msgr2=0x7fec040b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec040b4000 0x7fec040b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 -- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec0415f4e0 msgr2=0x7fec040a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec0415f4e0 0x7fec040a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 -- 172.21.15.46:0/3559571002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec040b6670 con 0x7fec040d2130 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.344+0000 7fec02000640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7febf000d800 tx=0x7febf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.345+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000b6c0 con 0x7fec040d2130 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.345+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf0019070 con 0x7fec040d2130 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.345+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000ba80 con 0x7fec040d2130 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.345+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0409f020 con 0x7fec040d2130 2024-04-26T23:48:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.345+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec040a0cf0 con 0x7fec040d2130 2024-04-26T23:48:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.346+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec040693b0 con 0x7fec040d2130 2024-04-26T23:48:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.347+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febf000bc20 con 0x7fec040d2130 2024-04-26T23:48:24.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.348+0000 7febfaa00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.348+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7febf008c260 con 0x7fec040d2130 2024-04-26T23:48:24.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.349+0000 7fec02a00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.350+0000 7fec02a00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 0x7febe406edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fec040b3500 tx=0x7febf4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.353+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febf005a410 con 0x7fec040d2130 2024-04-26T23:48:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.558+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec040d33d0 con 0x7fec040d2130 2024-04-26T23:48:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.559+0000 7febfaa00640 1 -- 172.21.15.46:0/3559571002 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17012 (secure 0 0 0) 0x7febf0059d10 con 0x7fec040d2130 2024-04-26T23:48:24.565 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:24.565 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:03.797242+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.9","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 msgr2=0x7febe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 0x7febe406edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fec040b3500 tx=0x7febf4009040 comp rx=0 tx=0).stop 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 msgr2=0x7fec0415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7febf000d800 tx=0x7febf000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 shutdown_connections 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec0415f4e0 0x7fec040a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec040d2130 0x7fec0415dec0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 --2- 172.21.15.46:0/3559571002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec040b4000 0x7fec040b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 >> 172.21.15.46:0/3559571002 conn(0x7fec040d5010 msgr2=0x7fec04161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 shutdown_connections 2024-04-26T23:48:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.563+0000 7fec03e00640 1 -- 172.21.15.46:0/3559571002 wait complete. 2024-04-26T23:48:24.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:45:30.380235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:45:31.336100+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:48:24.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:48:24.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.799+0000 7f1b47200640 1 Processor -- start 2024-04-26T23:48:24.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.800+0000 7f1b47200640 1 -- start start 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b47200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b40160510 0x7f1b400cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b47200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 0x7f1b40072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b47200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b400cbe60 con 0x7f1b40160130 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b47200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400cca70 con 0x7f1b40160510 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b47200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4015d350 con 0x7f1b400cda90 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b45e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b40160130 0x7f1b4015cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40354/0 (socket says 172.21.15.46:40354) 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b45400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b40160510 0x7f1b400cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.801+0000 7f1b45e00640 1 -- 172.21.15.46:0/2859789685 learned_addr learned my addr 172.21.15.46:0/2859789685 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b46800640 1 --2- 172.21.15.46:0/2859789685 >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 0x7f1b40072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b45400640 1 -- 172.21.15.46:0/2859789685 >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 msgr2=0x7f1b40072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b45400640 1 --2- 172.21.15.46:0/2859789685 >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 0x7f1b40072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b45400640 1 -- 172.21.15.46:0/2859789685 >> v1:172.21.15.46:6789/0 conn(0x7f1b40160130 legacy=0x7f1b4015cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b45400640 1 -- 172.21.15.46:0/2859789685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b400cc140 con 0x7f1b40160510 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b44a00640 1 -- 172.21.15.46:0/2859789685 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1026306208 0 0) 0x7f1b400cbe60 con 0x7f1b40160130 2024-04-26T23:48:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.802+0000 7f1b46800640 1 --2- 172.21.15.46:0/2859789685 >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 0x7f1b40072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.803+0000 7f1b45400640 1 --2- 172.21.15.46:0/2859789685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b40160510 0x7f1b400cd550 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1b400cd190 tx=0x7f1b3002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9439de2c756d61f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.803+0000 7f1b44a00640 1 -- 172.21.15.46:0/2859789685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30037070 con 0x7f1b40160510 2024-04-26T23:48:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b44a00640 1 -- 172.21.15.46:0/2859789685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30032440 con 0x7f1b40160510 2024-04-26T23:48:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b44a00640 1 -- 172.21.15.46:0/2859789685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3002ae60 con 0x7f1b40160510 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 -- 172.21.15.46:0/2859789685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b40160510 msgr2=0x7f1b400cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 --2- 172.21.15.46:0/2859789685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b40160510 0x7f1b400cd550 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1b400cd190 tx=0x7f1b3002a110 comp rx=0 tx=0).stop 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 -- 172.21.15.46:0/2859789685 shutdown_connections 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 --2- 172.21.15.46:0/2859789685 >> v2:172.21.15.142:3300/0 conn(0x7f1b400cda90 0x7f1b40072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 --2- 172.21.15.46:0/2859789685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b40160510 0x7f1b400cd550 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 -- 172.21.15.46:0/2859789685 >> 172.21.15.46:0/2859789685 conn(0x7f1b400b27b0 msgr2=0x7f1b400b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 -- 172.21.15.46:0/2859789685 shutdown_connections 2024-04-26T23:48:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.804+0000 7f1b47200640 1 -- 172.21.15.46:0/2859789685 wait complete. 2024-04-26T23:48:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.806+0000 7f1b47200640 1 Processor -- start 2024-04-26T23:48:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.806+0000 7f1b47200640 1 -- start start 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 0x7f1b4015e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 0x7f1b4015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40072760 con 0x7f1b400cda90 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400cc700 con 0x7f1b40160130 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b47200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b400728e0 con 0x7f1b40160510 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37782/0 (socket says 172.21.15.46:37782) 2024-04-26T23:48:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.807+0000 7f1b45e00640 1 -- 172.21.15.46:0/1789384186 learned_addr learned my addr 172.21.15.46:0/1789384186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:24.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45400640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 0x7f1b4015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b46800640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 0x7f1b4015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 -- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 msgr2=0x7f1b4015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 0x7f1b4015eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 -- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 msgr2=0x7f1b4015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 0x7f1b4015e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 -- 172.21.15.46:0/1789384186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b400c6650 con 0x7f1b400cda90 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45400640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 0x7f1b4015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b46800640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 0x7f1b4015eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.808+0000 7f1b45e00640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c4d0 tx=0x7f1b3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.809+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34018070 con 0x7f1b400cda90 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.809+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34007500 con 0x7f1b400cda90 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.809+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34004a20 con 0x7f1b400cda90 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.809+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b400c6930 con 0x7f1b400cda90 2024-04-26T23:48:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.809+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b400c6ef0 con 0x7f1b400cda90 2024-04-26T23:48:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.810+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b400693b0 con 0x7f1b400cda90 2024-04-26T23:48:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.811+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b3401b070 con 0x7f1b400cda90 2024-04-26T23:48:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.813+0000 7f1b3a000640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.813+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f1b3408b150 con 0x7f1b400cda90 2024-04-26T23:48:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.813+0000 7f1b45400640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.815+0000 7f1b45400640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b30034000 tx=0x7f1b3002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:24.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:24.818+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b34059270 con 0x7f1b400cda90 2024-04-26T23:48:25.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.021+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b400cb4e0 con 0x7f1b400cda90 2024-04-26T23:48:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.022+0000 7f1b3a000640 1 -- 172.21.15.46:0/1789384186 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f1b34058b70 con 0x7f1b400cda90 2024-04-26T23:48:25.027 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:48:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 msgr2=0x7f1b2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b30034000 tx=0x7f1b3002e000 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 msgr2=0x7f1b400ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c4d0 tx=0x7f1b3400c990 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 shutdown_connections 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b40160510 0x7f1b4015eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b40160130 0x7f1b4015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 --2- 172.21.15.46:0/1789384186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b400cda90 0x7f1b400ca1c0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 >> 172.21.15.46:0/1789384186 conn(0x7f1b400b27b0 msgr2=0x7f1b400c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 shutdown_connections 2024-04-26T23:48:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.025+0000 7f1b47200640 1 -- 172.21.15.46:0/1789384186 wait complete. 2024-04-26T23:48:25.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:48:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.261+0000 7f3ead000640 1 Processor -- start 2024-04-26T23:48:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.261+0000 7f3ead000640 1 -- start start 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ead000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3ea815cf40 0x7f3ea806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ead000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3ea806c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ead000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea8160c60 con 0x7f3ea80aee20 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ead000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea8160f00 con 0x7f3ea815cf40 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ead000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea81611e0 con 0x7f3ea806f060 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ea6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ea80aee20 0x7f3ea80af2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40364/0 (socket says 172.21.15.46:40364) 2024-04-26T23:48:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.263+0000 7f3ea6800640 1 -- 172.21.15.46:0/754269114 learned_addr learned my addr 172.21.15.46:0/754269114 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/754269114 >> v2:172.21.15.142:3300/0 conn(0x7f3ea815cf40 0x7f3ea806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea5400640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3ea806c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851724112 0 0) 0x7f3ea8160c60 con 0x7f3ea80aee20 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e88003520 con 0x7f3ea80aee20 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2920750531 0 0) 0x7f3e88003520 con 0x7f3ea80aee20 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 msgr2=0x7f3ea806c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3ea806c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 >> v2:172.21.15.142:3300/0 conn(0x7f3ea815cf40 msgr2=0x7f3ea806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 --2- 172.21.15.46:0/754269114 >> v2:172.21.15.142:3300/0 conn(0x7f3ea815cf40 0x7f3ea806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81614c0 con 0x7f3ea80aee20 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3ea00019f0 con 0x7f3ea80aee20 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.264+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 >> v1:172.21.15.46:6789/0 conn(0x7f3ea80aee20 legacy=0x7f3ea80af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e880039b0 0x7f3e88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3e88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e88004d40 0x7f3e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8160f00 con 0x7f3e880039b0 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81611e0 con 0x7f3e88004d40 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8800b9f0 con 0x7f3ea806f060 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e880039b0 0x7f3e88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea5400640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3e88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.265+0000 7f3ea6800640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e88004d40 0x7f3e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.266+0000 7f3ea5e00640 1 -- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 msgr2=0x7f3e88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.266+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea806f060 0x7f3e88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.266+0000 7f3ea5e00640 1 -- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e88004d40 msgr2=0x7f3e880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.266+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e88004d40 0x7f3e880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.266+0000 7f3ea5e00640 1 -- 172.21.15.46:0/754269114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e88003520 con 0x7f3e880039b0 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.267+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e880039b0 0x7f3e88003dd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ac0 tx=0x7f3e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fba0e79e166566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.267+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98036070 con 0x7f3e880039b0 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.267+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e98031400 con 0x7f3e880039b0 2024-04-26T23:48:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.267+0000 7f3ea4a00640 1 -- 172.21.15.46:0/754269114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98009690 con 0x7f3e880039b0 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 -- 172.21.15.46:0/754269114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e880039b0 msgr2=0x7f3e88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 --2- 172.21.15.46:0/754269114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e880039b0 0x7f3e88003dd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ac0 tx=0x7f3e88005830 comp rx=0 tx=0).stop 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ea6800640 1 -- 172.21.15.46:0/754269114 reap_dead start 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 -- 172.21.15.46:0/754269114 shutdown_connections 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 -- 172.21.15.46:0/754269114 >> 172.21.15.46:0/754269114 conn(0x7f3ea806da90 msgr2=0x7f3ea80d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 -- 172.21.15.46:0/754269114 shutdown_connections 2024-04-26T23:48:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.268+0000 7f3ead000640 1 -- 172.21.15.46:0/754269114 wait complete. 2024-04-26T23:48:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.270+0000 7f3ead000640 1 Processor -- start 2024-04-26T23:48:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.270+0000 7f3ead000640 1 -- start start 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e880039b0 0x7f3ea814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea8156330 0x7f3ea80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea806c890 con 0x7f3ea80b2100 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea815d320 con 0x7f3e880039b0 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ead000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea806ca10 con 0x7f3ea8156330 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ea6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ea6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37802/0 (socket says 172.21.15.46:37802) 2024-04-26T23:48:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.271+0000 7f3ea6800640 1 -- 172.21.15.46:0/3375097874 learned_addr learned my addr 172.21.15.46:0/3375097874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea5400640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea8156330 0x7f3ea80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e880039b0 0x7f3ea814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 -- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea8156330 msgr2=0x7f3ea80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea8156330 0x7f3ea80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 -- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e880039b0 msgr2=0x7f3ea814e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e880039b0 0x7f3ea814e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 -- 172.21.15.46:0/3375097874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81f8550 con 0x7f3ea80b2100 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.272+0000 7f3ea6800640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80a0100 tx=0x7f3ea0007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.273+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000eb00 con 0x7f3ea80b2100 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.273+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea0018070 con 0x7f3ea80b2100 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.273+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0013420 con 0x7f3ea80b2100 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.273+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81f87f0 con 0x7f3ea80b2100 2024-04-26T23:48:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.273+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81f8db0 con 0x7f3ea80b2100 2024-04-26T23:48:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.274+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea8069530 con 0x7f3ea80b2100 2024-04-26T23:48:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.275+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ea00135c0 con 0x7f3ea80b2100 2024-04-26T23:48:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.276+0000 7f3e9e000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 0x7f3e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.276+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f3ea008c0f0 con 0x7f3ea80b2100 2024-04-26T23:48:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.277+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 0x7f3e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.278+0000 7f3ea5e00640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 0x7f3e8406edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3e98033000 tx=0x7f3e98008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.281+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ea005a2a0 con 0x7f3ea80b2100 2024-04-26T23:48:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.482+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ea8160700 con 0x7f3ea80b2100 2024-04-26T23:48:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.483+0000 7f3e9e000640 1 -- 172.21.15.46:0/3375097874 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f3ea0059ba0 con 0x7f3ea80b2100 2024-04-26T23:48:25.487 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:48:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 msgr2=0x7f3e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 0x7f3e8406edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3e98033000 tx=0x7f3e98008030 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 msgr2=0x7f3ea80b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80a0100 tx=0x7f3ea0007bc0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 shutdown_connections 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3e8406c8f0 0x7f3e8406edb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea80b2100 0x7f3ea80b25b0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ea8156330 0x7f3ea80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 --2- 172.21.15.46:0/3375097874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e880039b0 0x7f3ea814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 >> 172.21.15.46:0/3375097874 conn(0x7f3ea806da90 msgr2=0x7f3ea80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 shutdown_connections 2024-04-26T23:48:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.486+0000 7f3ead000640 1 -- 172.21.15.46:0/3375097874 wait complete. 2024-04-26T23:48:25.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:48:25.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.722+0000 7f4b22400640 1 Processor -- start 2024-04-26T23:48:25.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.723+0000 7f4b22400640 1 -- start start 2024-04-26T23:48:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b22400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152ba0 0x7f4b1c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b22400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b22400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c15f920 con 0x7f4b1c152f80 2024-04-26T23:48:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b22400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c160530 con 0x7f4b1c0cfdb0 2024-04-26T23:48:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b22400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c15fc00 con 0x7f4b1c152ba0 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37810/0 (socket says 172.21.15.46:37810) 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.724+0000 7f4b1be00640 1 -- 172.21.15.46:0/3292513172 learned_addr learned my addr 172.21.15.46:0/3292513172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.725+0000 7f4b21000640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152ba0 0x7f4b1c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.725+0000 7f4b1be00640 1 -- 172.21.15.46:0/3292513172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152ba0 msgr2=0x7f4b1c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.725+0000 7f4b1be00640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152ba0 0x7f4b1c0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.725+0000 7f4b1be00640 1 -- 172.21.15.46:0/3292513172 >> v1:172.21.15.142:6789/0 conn(0x7f4b1c152f80 legacy=0x7f4b1c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.725+0000 7f4b1be00640 1 -- 172.21.15.46:0/3292513172 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c15fee0 con 0x7f4b1c0cfdb0 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.726+0000 7f4b1be00640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009870 tx=0x7f4b0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5947fce7253af399 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.726+0000 7f4b1b400640 1 -- 172.21.15.46:0/3292513172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08037070 con 0x7f4b1c0cfdb0 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.726+0000 7f4b1b400640 1 -- 172.21.15.46:0/3292513172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b08032440 con 0x7f4b1c0cfdb0 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.726+0000 7f4b1b400640 1 -- 172.21.15.46:0/3292513172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08032810 con 0x7f4b1c0cfdb0 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 -- 172.21.15.46:0/3292513172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c0cfdb0 msgr2=0x7f4b1c06ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009870 tx=0x7f4b0802a210 comp rx=0 tx=0).stop 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 -- 172.21.15.46:0/3292513172 shutdown_connections 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c0cfdb0 0x7f4b1c06ef70 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 --2- 172.21.15.46:0/3292513172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152ba0 0x7f4b1c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 -- 172.21.15.46:0/3292513172 >> 172.21.15.46:0/3292513172 conn(0x7f4b1c0b26d0 msgr2=0x7f4b1c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 -- 172.21.15.46:0/3292513172 shutdown_connections 2024-04-26T23:48:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.727+0000 7f4b22400640 1 -- 172.21.15.46:0/3292513172 wait complete. 2024-04-26T23:48:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.729+0000 7f4b22400640 1 Processor -- start 2024-04-26T23:48:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.729+0000 7f4b22400640 1 -- start start 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b1c152ba0 0x7f4b1c14bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 0x7f4b1c071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c160220 con 0x7f4b1c072260 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c0cf590 con 0x7f4b1c152ba0 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b22400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c0ccb80 con 0x7f4b1c152f80 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.730+0000 7f4b21a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37814/0 (socket says 172.21.15.46:37814) 2024-04-26T23:48:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 -- 172.21.15.46:0/412569511 learned_addr learned my addr 172.21.15.46:0/412569511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21000640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b1c152ba0 0x7f4b1c14bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b1be00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 0x7f4b1c071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 -- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 msgr2=0x7f4b1c071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 0x7f4b1c071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 -- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b1c152ba0 msgr2=0x7f4b1c14bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b1c152ba0 0x7f4b1c14bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 -- 172.21.15.46:0/412569511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c150580 con 0x7f4b1c072260 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b1be00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 0x7f4b1c071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.731+0000 7f4b21a00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ba10 tx=0x7f4b1400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.732+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14019070 con 0x7f4b1c072260 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.732+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1400cb40 con 0x7f4b1c072260 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.732+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14014440 con 0x7f4b1c072260 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.732+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c150860 con 0x7f4b1c072260 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.732+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c150e20 con 0x7f4b1c072260 2024-04-26T23:48:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.733+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1c0693b0 con 0x7f4b1c072260 2024-04-26T23:48:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.734+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b1401c070 con 0x7f4b1c072260 2024-04-26T23:48:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.735+0000 7f4b18c00640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 0x7f4afc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.735+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f4b1408b950 con 0x7f4b1c072260 2024-04-26T23:48:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.736+0000 7f4b21000640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 0x7f4afc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.737+0000 7f4b21000640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 0x7f4afc06ede0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006fd0 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.740+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b14059b00 con 0x7f4b1c072260 2024-04-26T23:48:25.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.948+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b1c14fc20 con 0x7f4b1c072260 2024-04-26T23:48:25.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.948+0000 7f4b18c00640 1 -- 172.21.15.46:0/412569511 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f4b14059400 con 0x7f4b1c072260 2024-04-26T23:48:25.953 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 msgr2=0x7f4afc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 0x7f4afc06ede0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006fd0 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 msgr2=0x7f4b1c151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ba10 tx=0x7f4b1400bed0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 shutdown_connections 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4afc06c920 0x7f4afc06ede0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b1c072260 0x7f4b1c151620 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b1c152f80 0x7f4b1c071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 --2- 172.21.15.46:0/412569511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b1c152ba0 0x7f4b1c14bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 >> 172.21.15.46:0/412569511 conn(0x7f4b1c0b26d0 msgr2=0x7f4b1c06e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.951+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 shutdown_connections 2024-04-26T23:48:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:25.952+0000 7f4b22400640 1 -- 172.21.15.46:0/412569511 wait complete. 2024-04-26T23:48:25.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:48:26.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.173+0000 7f934a400640 1 Processor -- start 2024-04-26T23:48:26.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.174+0000 7f934a400640 1 -- start start 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.175+0000 7f934a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f93440cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.175+0000 7f934a400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93440b03c0 con 0x7f934415f1d0 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.175+0000 7f934a400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93440cb680 con 0x7f934415edf0 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.175+0000 7f934a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93440cf5c0 con 0x7f93440cd770 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.176+0000 7f9343e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f934415f1d0 0x7f93440c5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40378/0 (socket says 172.21.15.46:40378) 2024-04-26T23:48:26.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.176+0000 7f9343e00640 1 -- 172.21.15.46:0/3915080126 learned_addr learned my addr 172.21.15.46:0/3915080126 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.176+0000 7f9349a00640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f93440cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.176+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706587473 0 0) 0x7f93440b03c0 con 0x7f934415f1d0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.176+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932c003520 con 0x7f934415f1d0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1409864323 0 0) 0x7f932c003520 con 0x7f934415f1d0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 msgr2=0x7f93440cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f93440cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 >> v1:172.21.15.142:6789/0 conn(0x7f934415edf0 legacy=0x7f93440cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93440cb960 con 0x7f934415f1d0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9330001690 con 0x7f934415f1d0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 >> v1:172.21.15.46:6789/0 conn(0x7f934415f1d0 legacy=0x7f93440c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c0039e0 0x7f932c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f932c004800 0x7f932c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f932c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93440cf5c0 con 0x7f932c0039e0 2024-04-26T23:48:26.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f932c00bb30 con 0x7f932c004800 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f932c00bd30 con 0x7f93440cd770 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.177+0000 7f9349000640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c0039e0 0x7f932c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9343e00640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f932c004800 0x7f932c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349a00640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f932c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349000640 1 -- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 msgr2=0x7f932c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349000640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93440cd770 0x7f932c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349000640 1 -- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f932c004800 msgr2=0x7f932c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349000640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f932c004800 0x7f932c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.178+0000 7f9349000640 1 -- 172.21.15.46:0/3915080126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c003520 con 0x7f932c0039e0 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.179+0000 7f9349000640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c0039e0 0x7f932c003e00 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9338001720 tx=0x7f932c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45b477037f3ac27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.179+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338001370 con 0x7f932c0039e0 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.179+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93380052d0 con 0x7f932c0039e0 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.179+0000 7f9343400640 1 -- 172.21.15.46:0/3915080126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933802ab90 con 0x7f932c0039e0 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 -- 172.21.15.46:0/3915080126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c0039e0 msgr2=0x7f932c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 --2- 172.21.15.46:0/3915080126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c0039e0 0x7f932c003e00 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9338001720 tx=0x7f932c005830 comp rx=0 tx=0).stop 2024-04-26T23:48:26.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f9349a00640 1 -- 172.21.15.46:0/3915080126 reap_dead start 2024-04-26T23:48:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 -- 172.21.15.46:0/3915080126 shutdown_connections 2024-04-26T23:48:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 -- 172.21.15.46:0/3915080126 >> 172.21.15.46:0/3915080126 conn(0x7f93440b26d0 msgr2=0x7f93440b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 -- 172.21.15.46:0/3915080126 shutdown_connections 2024-04-26T23:48:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.180+0000 7f934a400640 1 -- 172.21.15.46:0/3915080126 wait complete. 2024-04-26T23:48:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.182+0000 7f934a400640 1 Processor -- start 2024-04-26T23:48:26.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.182+0000 7f934a400640 1 -- start start 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441514b0 0x7f934415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344151890 0x7f9344160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93440cc200 con 0x7f934415e420 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93440cbf20 con 0x7f93441514b0 2024-04-26T23:48:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f934a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93440cb1a0 con 0x7f9344151890 2024-04-26T23:48:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.183+0000 7f9343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37832/0 (socket says 172.21.15.46:37832) 2024-04-26T23:48:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 -- 172.21.15.46:0/4253646933 learned_addr learned my addr 172.21.15.46:0/4253646933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9349a00640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344151890 0x7f9344160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 -- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344151890 msgr2=0x7f9344160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344151890 0x7f9344160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 -- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441514b0 msgr2=0x7f934415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441514b0 0x7f934415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 -- 172.21.15.46:0/4253646933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934415d660 con 0x7f934415e420 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.184+0000 7f9343e00640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f933000e780 tx=0x7f933000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.185+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933000c6c0 con 0x7f934415e420 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.185+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330018070 con 0x7f934415e420 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.185+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933000ca60 con 0x7f934415e420 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.185+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934415d940 con 0x7f934415e420 2024-04-26T23:48:26.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.185+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934415df00 con 0x7f934415e420 2024-04-26T23:48:26.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.186+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93440c7110 con 0x7f934415e420 2024-04-26T23:48:26.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.187+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f933000cc00 con 0x7f934415e420 2024-04-26T23:48:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.188+0000 7f9340c00640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 0x7f932406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.188+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f933008bbd0 con 0x7f934415e420 2024-04-26T23:48:26.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.189+0000 7f9349000640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 0x7f932406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.190+0000 7f9349000640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 0x7f932406edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9338032000 tx=0x7f933802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.192+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93440c7110 con 0x7f934415e420 2024-04-26T23:48:26.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.411+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93440693b0 con 0x7f934415e420 2024-04-26T23:48:26.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.412+0000 7f9340c00640 1 -- 172.21.15.46:0/4253646933 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) ==== 80+0+0 (secure 0 0 0) 0x7f9330059d80 con 0x7f934415e420 2024-04-26T23:48:26.416 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:48:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.414+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 msgr2=0x7f932406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.414+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 0x7f932406edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9338032000 tx=0x7f933802c000 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 msgr2=0x7f934415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f933000e780 tx=0x7f933000ec40 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 shutdown_connections 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f932406c8f0 0x7f932406edb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344151890 0x7f9344160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934415e420 0x7f934415a560 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 --2- 172.21.15.46:0/4253646933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441514b0 0x7f934415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 >> 172.21.15.46:0/4253646933 conn(0x7f93440b26d0 msgr2=0x7f93440d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 shutdown_connections 2024-04-26T23:48:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.415+0000 7f934a400640 1 -- 172.21.15.46:0/4253646933 wait complete. 2024-04-26T23:48:26.438 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:48:26.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:26.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:48:26.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:48:26.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:26.563 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:48:26.620 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:48:26.620 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:48:26.620 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:26.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:48:26.660 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:48:26.660 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:26.660 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:26.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:48:26.700 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:48:26.700 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:48:26.700 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:26.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:48:26.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.958+0000 7fc983200640 1 Processor -- start 2024-04-26T23:48:26.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.958+0000 7fc983200640 1 -- start start 2024-04-26T23:48:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.959+0000 7fc983200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9840d37b0 0x7fc9840d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.959+0000 7fc983200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc9840b6090 0x7fc9840b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.959+0000 7fc983200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9840ce240 con 0x7fc984160650 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc983200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc9840ce4e0 con 0x7fc9840b6090 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc983200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc9840ce7c0 con 0x7fc9840d37b0 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc981e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc984160650 0x7fc9840d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40392/0 (socket says 172.21.15.46:40392) 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc981e00640 1 -- 172.21.15.46:0/2526565271 learned_addr learned my addr 172.21.15.46:0/2526565271 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc982800640 1 --2- 172.21.15.46:0/2526565271 >> v2:172.21.15.46:3300/0 conn(0x7fc9840b6090 0x7fc9840b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.960+0000 7fc981400640 1 --2- 172.21.15.46:0/2526565271 >> v2:172.21.15.142:3300/0 conn(0x7fc9840d37b0 0x7fc9840d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.961+0000 7fc982800640 1 -- 172.21.15.46:0/2526565271 >> v2:172.21.15.142:3300/0 conn(0x7fc9840d37b0 msgr2=0x7fc9840d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.961+0000 7fc982800640 1 --2- 172.21.15.46:0/2526565271 >> v2:172.21.15.142:3300/0 conn(0x7fc9840d37b0 0x7fc9840d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.961+0000 7fc982800640 1 -- 172.21.15.46:0/2526565271 >> v1:172.21.15.46:6789/0 conn(0x7fc984160650 legacy=0x7fc9840d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.961+0000 7fc982800640 1 -- 172.21.15.46:0/2526565271 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9840ceaa0 con 0x7fc9840b6090 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.961+0000 7fc980a00640 1 -- 172.21.15.46:0/2526565271 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209342385 0 0) 0x7fc9840ce240 con 0x7fc984160650 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.962+0000 7fc982800640 1 --2- 172.21.15.46:0/2526565271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9840b6090 0x7fc9840b6520 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc9840730f0 tx=0x7fc97c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885bc50060588b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.962+0000 7fc980a00640 1 -- 172.21.15.46:0/2526565271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c007740 con 0x7fc9840b6090 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.962+0000 7fc980a00640 1 -- 172.21.15.46:0/2526565271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c0078e0 con 0x7fc9840b6090 2024-04-26T23:48:26.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.962+0000 7fc980a00640 1 -- 172.21.15.46:0/2526565271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c007cb0 con 0x7fc9840b6090 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 -- 172.21.15.46:0/2526565271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9840b6090 msgr2=0x7fc9840b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 --2- 172.21.15.46:0/2526565271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9840b6090 0x7fc9840b6520 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc9840730f0 tx=0x7fc97c02a680 comp rx=0 tx=0).stop 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 -- 172.21.15.46:0/2526565271 shutdown_connections 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 --2- 172.21.15.46:0/2526565271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9840b6090 0x7fc9840b6520 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 --2- 172.21.15.46:0/2526565271 >> v2:172.21.15.142:3300/0 conn(0x7fc9840d37b0 0x7fc9840d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 -- 172.21.15.46:0/2526565271 >> 172.21.15.46:0/2526565271 conn(0x7fc9840d2cc0 msgr2=0x7fc9840d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 -- 172.21.15.46:0/2526565271 shutdown_connections 2024-04-26T23:48:26.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.963+0000 7fc983200640 1 -- 172.21.15.46:0/2526565271 wait complete. 2024-04-26T23:48:26.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.964+0000 7fc983200640 1 Processor -- start 2024-04-26T23:48:26.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.965+0000 7fc983200640 1 -- start start 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.965+0000 7fc983200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9840b6090 0x7fc98415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc983200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9840d37b0 0x7fc98409cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc983200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc983200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840d1660 con 0x7fc984160650 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc983200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840d14e0 con 0x7fc9840b6090 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc983200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9840b6a60 con 0x7fc9840d37b0 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc982800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc982800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37864/0 (socket says 172.21.15.46:37864) 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc982800640 1 -- 172.21.15.46:0/4188685635 learned_addr learned my addr 172.21.15.46:0/4188685635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc981e00640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9840b6090 0x7fc98415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.966+0000 7fc981400640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9840d37b0 0x7fc98409cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 -- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9840d37b0 msgr2=0x7fc98409cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9840d37b0 0x7fc98409cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 -- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9840b6090 msgr2=0x7fc98415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9840b6090 0x7fc98415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 -- 172.21.15.46:0/4188685635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98409d5c0 con 0x7fc984160650 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc982800640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c040070 con 0x7fc984160650 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c031bf0 con 0x7fc984160650 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c045460 con 0x7fc984160650 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98409d8a0 con 0x7fc984160650 2024-04-26T23:48:26.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.967+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98409de60 con 0x7fc984160650 2024-04-26T23:48:26.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.968+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984069ac0 con 0x7fc984160650 2024-04-26T23:48:26.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.970+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc97c039040 con 0x7fc984160650 2024-04-26T23:48:26.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.970+0000 7fc97a000640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 0x7fc96006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:26.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.971+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fc97c0ad6f0 con 0x7fc984160650 2024-04-26T23:48:26.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.971+0000 7fc981e00640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 0x7fc96006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:26.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.972+0000 7fc981e00640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 0x7fc96006edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc974000f60 tx=0x7fc974007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:26.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:26.975+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc97c07ba50 con 0x7fc984160650 2024-04-26T23:48:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.223+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc98409e060 con 0x7fc984160650 2024-04-26T23:48:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.224+0000 7fc97a000640 1 -- 172.21.15.46:0/4188685635 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5183 (secure 0 0 0) 0x7fc97c07b350 con 0x7fc984160650 2024-04-26T23:48:27.229 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:27.229 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:41.824964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:48:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 msgr2=0x7fc96006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 0x7fc96006edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc974000f60 tx=0x7fc974007c00 comp rx=0 tx=0).stop 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 msgr2=0x7fc9840cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 shutdown_connections 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc96006c8f0 0x7fc96006edb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc984160650 0x7fc9840cee90 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9840d37b0 0x7fc98409cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 --2- 172.21.15.46:0/4188685635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9840b6090 0x7fc98415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 >> 172.21.15.46:0/4188685635 conn(0x7fc9840d2cc0 msgr2=0x7fc98415f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.227+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 shutdown_connections 2024-04-26T23:48:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.228+0000 7fc983200640 1 -- 172.21.15.46:0/4188685635 wait complete. 2024-04-26T23:48:27.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.457+0000 7ff30be00640 1 Processor -- start 2024-04-26T23:48:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.458+0000 7ff30be00640 1 -- start start 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff30c0d2130 0x7ff30c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c0d11f0 con 0x7ff30c15f4e0 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c0d0ce0 con 0x7ff30c0d2130 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c0d2990 con 0x7ff30c0b4000 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff30c0d2130 0x7ff30c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff30c0d2130 0x7ff30c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37876/0 (socket says 172.21.15.46:37876) 2024-04-26T23:48:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.460+0000 7ff30a000640 1 -- 172.21.15.46:0/2196867375 learned_addr learned my addr 172.21.15.46:0/2196867375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30b400640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30a000640 1 -- 172.21.15.46:0/2196867375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 msgr2=0x7ff30c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30a000640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30a000640 1 -- 172.21.15.46:0/2196867375 >> v1:172.21.15.142:6789/0 conn(0x7ff30c15f4e0 legacy=0x7ff30c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30a000640 1 -- 172.21.15.46:0/2196867375 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c0d5f20 con 0x7ff30c0d2130 2024-04-26T23:48:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.461+0000 7ff30b400640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.462+0000 7ff30a000640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c0d2df0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4010691eec80a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.462+0000 7ff309600640 1 -- 172.21.15.46:0/2196867375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8037070 con 0x7ff30c0d2130 2024-04-26T23:48:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.462+0000 7ff309600640 1 -- 172.21.15.46:0/2196867375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f8032440 con 0x7ff30c0d2130 2024-04-26T23:48:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.462+0000 7ff309600640 1 -- 172.21.15.46:0/2196867375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8032810 con 0x7ff30c0d2130 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 -- 172.21.15.46:0/2196867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 msgr2=0x7ff30c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c0d2df0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f802a210 comp rx=0 tx=0).stop 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 -- 172.21.15.46:0/2196867375 shutdown_connections 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 --2- 172.21.15.46:0/2196867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c0d2df0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 -- 172.21.15.46:0/2196867375 >> 172.21.15.46:0/2196867375 conn(0x7ff30c0d5010 msgr2=0x7ff30c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 -- 172.21.15.46:0/2196867375 shutdown_connections 2024-04-26T23:48:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.463+0000 7ff30be00640 1 -- 172.21.15.46:0/2196867375 wait complete. 2024-04-26T23:48:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.465+0000 7ff30be00640 1 Processor -- start 2024-04-26T23:48:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.465+0000 7ff30be00640 1 -- start start 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.466+0000 7ff30be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.466+0000 7ff30be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.466+0000 7ff30be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff30c15f4e0 0x7ff30c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0b1d60 con 0x7ff30c0d2130 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0d5c40 con 0x7ff30c15f4e0 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c15f8c0 con 0x7ff30c0b4000 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37878/0 (socket says 172.21.15.46:37878) 2024-04-26T23:48:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 -- 172.21.15.46:0/1310616416 learned_addr learned my addr 172.21.15.46:0/1310616416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30aa00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30b400640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff30c15f4e0 0x7ff30c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 -- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 msgr2=0x7ff30c0b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 -- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff30c15f4e0 msgr2=0x7ff30c14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff30c15f4e0 0x7ff30c14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.467+0000 7ff30a000640 1 -- 172.21.15.46:0/1310616416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c14c590 con 0x7ff30c0d2130 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff30a000640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8041020 con 0x7ff30c0d2130 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f8030970 con 0x7ff30c0d2130 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8037040 con 0x7ff30c0d2130 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c14c870 con 0x7ff30c0d2130 2024-04-26T23:48:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.468+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c14ce30 con 0x7ff30c0d2130 2024-04-26T23:48:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.470+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c0693b0 con 0x7ff30c0d2130 2024-04-26T23:48:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.471+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2f8044020 con 0x7ff30c0d2130 2024-04-26T23:48:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.472+0000 7ff302a00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.472+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7ff2f80ac7b0 con 0x7ff30c0d2130 2024-04-26T23:48:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.473+0000 7ff30aa00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.474+0000 7ff30aa00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.477+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2f804b030 con 0x7ff30c0d2130 2024-04-26T23:48:27.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.683+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff30c0d33d0 con 0x7ff30c0d2130 2024-04-26T23:48:27.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.685+0000 7ff302a00640 1 -- 172.21.15.46:0/1310616416 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17012 (secure 0 0 0) 0x7ff2f807a970 con 0x7ff30c0d2130 2024-04-26T23:48:27.690 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:03.797242+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.9","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.687+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 msgr2=0x7ff2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.687+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.687+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 msgr2=0x7ff30c1f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.687+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f80046c0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 shutdown_connections 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff30c15f4e0 0x7ff30c14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c0d2130 0x7ff30c1f4d90 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 --2- 172.21.15.46:0/1310616416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30c0b4000 0x7ff30c0b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 >> 172.21.15.46:0/1310616416 conn(0x7ff30c0d5010 msgr2=0x7ff30c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 shutdown_connections 2024-04-26T23:48:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.688+0000 7ff30be00640 1 -- 172.21.15.46:0/1310616416 wait complete. 2024-04-26T23:48:27.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:45:30.380235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:45:31.336100+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:48:27.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:48:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.923+0000 7f95c3400640 1 Processor -- start 2024-04-26T23:48:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.923+0000 7f95c3400640 1 -- start start 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc15d730 0x7f95bc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95bc15e220 0x7f95bc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c3400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc06eb90 con 0x7f95bc0b53b0 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c3400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f95bc0b4310 con 0x7f95bc15e220 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c3400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc0b45f0 con 0x7f95bc15d730 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95bbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95bc0b53b0 0x7f95bc15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40396/0 (socket says 172.21.15.46:40396) 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95bbe00640 1 -- 172.21.15.46:0/1152946854 learned_addr learned my addr 172.21.15.46:0/1152946854 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c1400640 1 --2- 172.21.15.46:0/1152946854 >> v2:172.21.15.46:3300/0 conn(0x7f95bc15e220 0x7f95bc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.925+0000 7f95c0a00640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc15d730 0x7f95bc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95bb400640 1 -- 172.21.15.46:0/1152946854 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259881585 0 0) 0x7f95bc06eb90 con 0x7f95bc0b53b0 2024-04-26T23:48:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95bb400640 1 -- 172.21.15.46:0/1152946854 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a4003520 con 0x7f95bc0b53b0 2024-04-26T23:48:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95c1400640 1 -- 172.21.15.46:0/1152946854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc15d730 msgr2=0x7f95bc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95c1400640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc15d730 0x7f95bc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95c1400640 1 -- 172.21.15.46:0/1152946854 >> v1:172.21.15.46:6789/0 conn(0x7f95bc0b53b0 legacy=0x7f95bc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.926+0000 7f95c1400640 1 -- 172.21.15.46:0/1152946854 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0b48d0 con 0x7f95bc15e220 2024-04-26T23:48:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.927+0000 7f95c1400640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15e220 0x7f95bc06dc90 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009d90 tx=0x7f95b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fe88a17d80e7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.927+0000 7f95bb400640 1 -- 172.21.15.46:0/1152946854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007740 con 0x7f95bc15e220 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.927+0000 7f95bb400640 1 -- 172.21.15.46:0/1152946854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b40078e0 con 0x7f95bc15e220 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.927+0000 7f95bb400640 1 -- 172.21.15.46:0/1152946854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007cb0 con 0x7f95bc15e220 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 -- 172.21.15.46:0/1152946854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15e220 msgr2=0x7f95bc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15e220 0x7f95bc06dc90 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009d90 tx=0x7f95b402a660 comp rx=0 tx=0).stop 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 -- 172.21.15.46:0/1152946854 shutdown_connections 2024-04-26T23:48:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15e220 0x7f95bc06dc90 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 --2- 172.21.15.46:0/1152946854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc15d730 0x7f95bc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 -- 172.21.15.46:0/1152946854 >> 172.21.15.46:0/1152946854 conn(0x7f95bc06c980 msgr2=0x7f95bc0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 -- 172.21.15.46:0/1152946854 shutdown_connections 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.928+0000 7f95c3400640 1 -- 172.21.15.46:0/1152946854 wait complete. 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.930+0000 7f95c3400640 1 Processor -- start 2024-04-26T23:48:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.930+0000 7f95c3400640 1 -- start start 2024-04-26T23:48:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc0b53b0 0x7f95bc13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95bc15e220 0x7f95bc133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc0b3ec0 con 0x7f95bc15d730 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc0b4e90 con 0x7f95bc15e220 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c3400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc0b4040 con 0x7f95bc0b53b0 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37884/0 (socket says 172.21.15.46:37884) 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95bbe00640 1 -- 172.21.15.46:0/2866492565 learned_addr learned my addr 172.21.15.46:0/2866492565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c0a00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc0b53b0 0x7f95bc13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.931+0000 7f95c1400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95bc15e220 0x7f95bc133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 -- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc0b53b0 msgr2=0x7f95bc13ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc0b53b0 0x7f95bc13ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 -- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95bc15e220 msgr2=0x7f95bc133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95bc15e220 0x7f95bc133d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 -- 172.21.15.46:0/2866492565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0b0530 con 0x7f95bc15d730 2024-04-26T23:48:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95bbe00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f95a800c390 tx=0x7f95a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8017070 con 0x7f95bc15d730 2024-04-26T23:48:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a8005ce0 con 0x7f95bc15d730 2024-04-26T23:48:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc0aff70 con 0x7f95bc15d730 2024-04-26T23:48:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.932+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a80072f0 con 0x7f95bc15d730 2024-04-26T23:48:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.933+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc134d30 con 0x7f95bc15d730 2024-04-26T23:48:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.934+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc069530 con 0x7f95bc15d730 2024-04-26T23:48:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.935+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95a801a070 con 0x7f95bc15d730 2024-04-26T23:48:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.936+0000 7f95b8c00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 0x7f959c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.936+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f95a808b170 con 0x7f95bc15d730 2024-04-26T23:48:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.936+0000 7f95c0a00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 0x7f959c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.937+0000 7f95c0a00640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 0x7f959c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009ad0 tx=0x7f95b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:27.940+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95a8059320 con 0x7f95bc15d730 2024-04-26T23:48:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.146+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95bc0d03d0 con 0x7f95bc15d730 2024-04-26T23:48:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.147+0000 7f95b8c00640 1 -- 172.21.15.46:0/2866492565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f95a8058c20 con 0x7f95bc15d730 2024-04-26T23:48:28.151 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:48:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 msgr2=0x7f959c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 0x7f959c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009ad0 tx=0x7f95b0008040 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 msgr2=0x7f95bc0abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f95a800c390 tx=0x7f95a800c850 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 shutdown_connections 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f959c06c920 0x7f959c06ede0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95bc15e220 0x7f95bc133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95bc15d730 0x7f95bc0abc20 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 --2- 172.21.15.46:0/2866492565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95bc0b53b0 0x7f95bc13ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.149+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 >> 172.21.15.46:0/2866492565 conn(0x7f95bc06c980 msgr2=0x7f95bc146120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.150+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 shutdown_connections 2024-04-26T23:48:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.150+0000 7f95c3400640 1 -- 172.21.15.46:0/2866492565 wait complete. 2024-04-26T23:48:28.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:48:28.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.387+0000 7fbba5800640 1 Processor -- start 2024-04-26T23:48:28.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.387+0000 7fbba5800640 1 -- start start 2024-04-26T23:48:28.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbba015f210 0x7fbba00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba5800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbba00af000 0x7fbba00cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba5800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba00af570 con 0x7fbba00cd610 2024-04-26T23:48:28.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbba00cdfb0 con 0x7fbba015f210 2024-04-26T23:48:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba5800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbba00af850 con 0x7fbba00af000 2024-04-26T23:48:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbb9fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbba015f210 0x7fbba00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbb9fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbba015f210 0x7fbba00cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37898/0 (socket says 172.21.15.46:37898) 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbb9fe00640 1 -- 172.21.15.46:0/3352003685 learned_addr learned my addr 172.21.15.46:0/3352003685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbb97e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbba00cd610 0x7fbba00ae8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40408/0 (socket says 172.21.15.46:40408) 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.389+0000 7fbba4e00640 1 --2- 172.21.15.46:0/3352003685 >> v2:172.21.15.142:3300/0 conn(0x7fbba00af000 0x7fbba00cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.390+0000 7fbb9fe00640 1 -- 172.21.15.46:0/3352003685 >> v2:172.21.15.142:3300/0 conn(0x7fbba00af000 msgr2=0x7fbba00cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.390+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/3352003685 >> v2:172.21.15.142:3300/0 conn(0x7fbba00af000 0x7fbba00cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.390+0000 7fbb9fe00640 1 -- 172.21.15.46:0/3352003685 >> v1:172.21.15.46:6789/0 conn(0x7fbba00cd610 legacy=0x7fbba00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.390+0000 7fbb9fe00640 1 -- 172.21.15.46:0/3352003685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba00cb680 con 0x7fbba015f210 2024-04-26T23:48:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.390+0000 7fbb9f400640 1 -- 172.21.15.46:0/3352003685 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699587275 0 0) 0x7fbba00af570 con 0x7fbba00cd610 2024-04-26T23:48:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.391+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/3352003685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba015f210 0x7fbba00cc160 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cce80 tx=0x7fbb9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce007ec14d050c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.391+0000 7fbb9f400640 1 -- 172.21.15.46:0/3352003685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90037070 con 0x7fbba015f210 2024-04-26T23:48:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.391+0000 7fbb9f400640 1 -- 172.21.15.46:0/3352003685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb90032440 con 0x7fbba015f210 2024-04-26T23:48:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.391+0000 7fbb9f400640 1 -- 172.21.15.46:0/3352003685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90032810 con 0x7fbba015f210 2024-04-26T23:48:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 -- 172.21.15.46:0/3352003685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba015f210 msgr2=0x7fbba00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 --2- 172.21.15.46:0/3352003685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba015f210 0x7fbba00cc160 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cce80 tx=0x7fbb9002a270 comp rx=0 tx=0).stop 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 -- 172.21.15.46:0/3352003685 shutdown_connections 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 --2- 172.21.15.46:0/3352003685 >> v2:172.21.15.142:3300/0 conn(0x7fbba00af000 0x7fbba00cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 --2- 172.21.15.46:0/3352003685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba015f210 0x7fbba00cc160 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 -- 172.21.15.46:0/3352003685 >> 172.21.15.46:0/3352003685 conn(0x7fbba00b27b0 msgr2=0x7fbba00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 -- 172.21.15.46:0/3352003685 shutdown_connections 2024-04-26T23:48:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.392+0000 7fbba5800640 1 -- 172.21.15.46:0/3352003685 wait complete. 2024-04-26T23:48:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.394+0000 7fbba5800640 1 Processor -- start 2024-04-26T23:48:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.394+0000 7fbba5800640 1 -- start start 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 0x7fbba01f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 0x7fbba01f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbba015f5f0 con 0x7fbba00cd610 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbba00cbc40 con 0x7fbba015f210 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbba5800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbba0071e80 con 0x7fbba00af000 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37902/0 (socket says 172.21.15.46:37902) 2024-04-26T23:48:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.395+0000 7fbb97e00640 1 -- 172.21.15.46:0/1915884032 learned_addr learned my addr 172.21.15.46:0/1915884032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbba4e00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 0x7fbba01f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 0x7fbba01f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 -- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 msgr2=0x7fbba01f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 0x7fbba01f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 -- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 msgr2=0x7fbba01f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 0x7fbba01f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 -- 172.21.15.46:0/1915884032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba00c72c0 con 0x7fbba00cd610 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 0x7fbba01f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbba4e00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 0x7fbba01f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb97e00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c4d0 tx=0x7fbb8800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.396+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88018070 con 0x7fbba00cd610 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.397+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88007500 con 0x7fbba00cd610 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.397+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88004a20 con 0x7fbba00cd610 2024-04-26T23:48:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.397+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba01f4c70 con 0x7fbba00cd610 2024-04-26T23:48:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.397+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba01f4850 con 0x7fbba00cd610 2024-04-26T23:48:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.398+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba00693b0 con 0x7fbba00cd610 2024-04-26T23:48:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.399+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb8801b070 con 0x7fbba00cd610 2024-04-26T23:48:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.400+0000 7fbb9cc00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 0x7fbb7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.400+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fbb8808b0c0 con 0x7fbba00cd610 2024-04-26T23:48:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.401+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 0x7fbb7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.402+0000 7fbb9fe00640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 0x7fbb7c06ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbb90034000 tx=0x7fbb9002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.405+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb88059270 con 0x7fbba00cd610 2024-04-26T23:48:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.608+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbba00b2cc0 con 0x7fbba00cd610 2024-04-26T23:48:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.610+0000 7fbb9cc00640 1 -- 172.21.15.46:0/1915884032 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7fbb88058b70 con 0x7fbba00cd610 2024-04-26T23:48:28.614 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 msgr2=0x7fbb7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 0x7fbb7c06ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbb90034000 tx=0x7fbb9002e000 comp rx=0 tx=0).stop 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 msgr2=0x7fbba01f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c4d0 tx=0x7fbb8800c990 comp rx=0 tx=0).stop 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 shutdown_connections 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb7c06c920 0x7fbb7c06ede0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba015f210 0x7fbba01f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba00cd610 0x7fbba01f4310 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 --2- 172.21.15.46:0/1915884032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbba00af000 0x7fbba01f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 >> 172.21.15.46:0/1915884032 conn(0x7fbba00b27b0 msgr2=0x7fbba00ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 shutdown_connections 2024-04-26T23:48:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.612+0000 7fbba5800640 1 -- 172.21.15.46:0/1915884032 wait complete. 2024-04-26T23:48:28.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:48:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.833+0000 7f0f0fe00640 1 Processor -- start 2024-04-26T23:48:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.833+0000 7f0f0fe00640 1 -- start start 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f1015f840 0x7f0f100d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f100d1980 0x7f0f100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f100d2320 0x7f0f100cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100d3880 con 0x7f0f1015f840 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100d4090 con 0x7f0f100d2320 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100d27b0 con 0x7f0f100d1980 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f1015f840 0x7f0f100d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f1015f840 0x7f0f100d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37910/0 (socket says 172.21.15.46:37910) 2024-04-26T23:48:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.835+0000 7f0f0ea00640 1 -- 172.21.15.46:0/892295044 learned_addr learned my addr 172.21.15.46:0/892295044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0e000640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f100d1980 0x7f0f100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0f400640 1 --2- 172.21.15.46:0/892295044 >> v2:172.21.15.142:3300/0 conn(0x7f0f100d2320 0x7f0f100cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0ea00640 1 -- 172.21.15.46:0/892295044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f100d1980 msgr2=0x7f0f100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f100d1980 0x7f0f100d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0ea00640 1 -- 172.21.15.46:0/892295044 >> v2:172.21.15.142:3300/0 conn(0x7f0f100d2320 msgr2=0x7f0f100cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/892295044 >> v2:172.21.15.142:3300/0 conn(0x7f0f100d2320 0x7f0f100cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.836+0000 7f0f0ea00640 1 -- 172.21.15.46:0/892295044 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100b4020 con 0x7f0f1015f840 2024-04-26T23:48:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.837+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f1015f840 0x7f0f100d2e70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0f100d5fc0 tx=0x7f0f0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfaa4252c757367 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.837+0000 7f0f0d600640 1 -- 172.21.15.46:0/892295044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00037070 con 0x7f0f1015f840 2024-04-26T23:48:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.837+0000 7f0f0d600640 1 -- 172.21.15.46:0/892295044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f00032440 con 0x7f0f1015f840 2024-04-26T23:48:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.837+0000 7f0f0d600640 1 -- 172.21.15.46:0/892295044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00032810 con 0x7f0f1015f840 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 -- 172.21.15.46:0/892295044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f1015f840 msgr2=0x7f0f100d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f1015f840 0x7f0f100d2e70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0f100d5fc0 tx=0x7f0f0002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 -- 172.21.15.46:0/892295044 shutdown_connections 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/892295044 >> v2:172.21.15.142:3300/0 conn(0x7f0f100d2320 0x7f0f100cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f100d1980 0x7f0f100d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/892295044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f1015f840 0x7f0f100d2e70 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 -- 172.21.15.46:0/892295044 >> 172.21.15.46:0/892295044 conn(0x7f0f100d4f70 msgr2=0x7f0f100d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 -- 172.21.15.46:0/892295044 shutdown_connections 2024-04-26T23:48:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.838+0000 7f0f0fe00640 1 -- 172.21.15.46:0/892295044 wait complete. 2024-04-26T23:48:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.840+0000 7f0f0fe00640 1 Processor -- start 2024-04-26T23:48:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.840+0000 7f0f0fe00640 1 -- start start 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 0x7f0f10160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1015f840 0x7f0f101f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100b27d0 con 0x7f0f100d2320 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d3db0 con 0x7f0f100d1980 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.841+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100d33b0 con 0x7f0f1015f840 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37918/0 (socket says 172.21.15.46:37918) 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 -- 172.21.15.46:0/2903068380 learned_addr learned my addr 172.21.15.46:0/2903068380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0f400640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1015f840 0x7f0f101f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 0x7f0f10160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 -- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1015f840 msgr2=0x7f0f101f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1015f840 0x7f0f101f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 -- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 msgr2=0x7f0f10160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 0x7f0f10160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0e000640 1 -- 172.21.15.46:0/2903068380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101609a0 con 0x7f0f100d2320 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.842+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 0x7f0f10160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f0e000640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c7f0 tx=0x7f0efc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc015070 con 0x7f0f100d2320 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0efc004220 con 0x7f0f100d2320 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc003610 con 0x7f0f100d2320 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f10160c80 con 0x7f0f100d2320 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.843+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f10161240 con 0x7f0f100d2320 2024-04-26T23:48:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.844+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100693b0 con 0x7f0f100d2320 2024-04-26T23:48:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.845+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0efc0039c0 con 0x7f0f100d2320 2024-04-26T23:48:28.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.846+0000 7f0f06a00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 0x7f0ef006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:28.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.846+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7f0efc08bde0 con 0x7f0f100d2320 2024-04-26T23:48:28.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.847+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 0x7f0ef006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:28.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.848+0000 7f0f0ea00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 0x7f0ef006ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f00034000 tx=0x7f0f0002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:28.851+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0efc059f90 con 0x7f0f100d2320 2024-04-26T23:48:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.065+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f100d4c10 con 0x7f0f100d2320 2024-04-26T23:48:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.066+0000 7f0f06a00640 1 -- 172.21.15.46:0/2903068380 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f0efc059890 con 0x7f0f100d2320 2024-04-26T23:48:29.071 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 msgr2=0x7f0ef006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 0x7f0ef006ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f00034000 tx=0x7f0f0002e000 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 msgr2=0x7f0f101f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c7f0 tx=0x7f0efc00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 shutdown_connections 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ef006c920 0x7f0ef006ede0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1015f840 0x7f0f101f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f100d2320 0x7f0f101f51c0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 --2- 172.21.15.46:0/2903068380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f100d1980 0x7f0f10160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 >> 172.21.15.46:0/2903068380 conn(0x7f0f100d4f70 msgr2=0x7f0f100b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 shutdown_connections 2024-04-26T23:48:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.069+0000 7f0f0fe00640 1 -- 172.21.15.46:0/2903068380 wait complete. 2024-04-26T23:48:29.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.300+0000 7fdab3200640 1 Processor -- start 2024-04-26T23:48:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.300+0000 7fdab3200640 1 -- start start 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdab40cc690 0x7fdab40cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 0x7fdab40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab4072bb0 con 0x7fdab41454a0 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdab40c73e0 con 0x7fdab40cc690 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdab4072e90 con 0x7fdab40c6a50 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdaa9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdab40cc690 0x7fdab40cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdab41454a0 0x7fdab4073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40412/0 (socket says 172.21.15.46:40412) 2024-04-26T23:48:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.302+0000 7fdab1e00640 1 -- 172.21.15.46:0/1753656131 learned_addr learned my addr 172.21.15.46:0/1753656131 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdaa9400640 1 -- 172.21.15.46:0/1753656131 >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 msgr2=0x7fdab40c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdab2800640 1 --2- 172.21.15.46:0/1753656131 >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 0x7fdab40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdab1400640 1 -- 172.21.15.46:0/1753656131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095024285 0 0) 0x7fdab4072bb0 con 0x7fdab41454a0 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdaa9400640 1 --2- 172.21.15.46:0/1753656131 >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 0x7fdab40c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdaa9400640 1 -- 172.21.15.46:0/1753656131 >> v1:172.21.15.46:6789/0 conn(0x7fdab41454a0 legacy=0x7fdab4073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdaa9400640 1 -- 172.21.15.46:0/1753656131 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab4073170 con 0x7fdab40cc690 2024-04-26T23:48:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.303+0000 7fdab2800640 1 --2- 172.21.15.46:0/1753656131 >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 0x7fdab40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.304+0000 7fdaa9400640 1 --2- 172.21.15.46:0/1753656131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40cc690 0x7fdab40cdc30 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fdab415f760 tx=0x7fda9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192344915048a1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.304+0000 7fdab1400640 1 -- 172.21.15.46:0/1753656131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c037070 con 0x7fdab40cc690 2024-04-26T23:48:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.304+0000 7fdab1400640 1 -- 172.21.15.46:0/1753656131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c032440 con 0x7fdab40cc690 2024-04-26T23:48:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.304+0000 7fdab1400640 1 -- 172.21.15.46:0/1753656131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c032810 con 0x7fdab40cc690 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 -- 172.21.15.46:0/1753656131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40cc690 msgr2=0x7fdab40cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 --2- 172.21.15.46:0/1753656131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40cc690 0x7fdab40cdc30 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fdab415f760 tx=0x7fda9c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 -- 172.21.15.46:0/1753656131 shutdown_connections 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 --2- 172.21.15.46:0/1753656131 >> v2:172.21.15.142:3300/0 conn(0x7fdab40c6a50 0x7fdab40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 --2- 172.21.15.46:0/1753656131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40cc690 0x7fdab40cdc30 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 -- 172.21.15.46:0/1753656131 >> 172.21.15.46:0/1753656131 conn(0x7fdab40b26d0 msgr2=0x7fdab40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 -- 172.21.15.46:0/1753656131 shutdown_connections 2024-04-26T23:48:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.305+0000 7fdab3200640 1 -- 172.21.15.46:0/1753656131 wait complete. 2024-04-26T23:48:29.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.307+0000 7fdab3200640 1 Processor -- start 2024-04-26T23:48:29.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.308+0000 7fdab3200640 1 -- start start 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab40cc690 0x7fdab40c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdab41454a0 0x7fdab415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4072880 con 0x7fdab40c6a50 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40ce170 con 0x7fdab40cc690 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdab4145880 con 0x7fdab41454a0 2024-04-26T23:48:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37944/0 (socket says 172.21.15.46:37944) 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.309+0000 7fdab1e00640 1 -- 172.21.15.46:0/3056606932 learned_addr learned my addr 172.21.15.46:0/3056606932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 -- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdab41454a0 msgr2=0x7fdab415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdab41454a0 0x7fdab415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 -- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab40cc690 msgr2=0x7fdab40c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab40cc690 0x7fdab40c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 -- 172.21.15.46:0/3056606932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab41f3b50 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab1e00640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c430 tx=0x7fdaa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40074e0 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa40047e0 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4004bd0 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.310+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab41f3e30 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.311+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41f43f0 con 0x7fdab40c6a50 2024-04-26T23:48:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.312+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40693b0 con 0x7fdab40c6a50 2024-04-26T23:48:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.313+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaa4011070 con 0x7fdab40c6a50 2024-04-26T23:48:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.313+0000 7fdaaaa00640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.314+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fdaa408bab0 con 0x7fdab40c6a50 2024-04-26T23:48:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.314+0000 7fdaa9400640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.315+0000 7fdaa9400640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 0x7fda9006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fda9c034000 tx=0x7fda9c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.320+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdaa4059c60 con 0x7fdab40c6a50 2024-04-26T23:48:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.557+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab40cefa0 con 0x7fdab40c6a50 2024-04-26T23:48:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.558+0000 7fdaaaa00640 1 -- 172.21.15.46:0/3056606932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5183 (secure 0 0 0) 0x7fdaa4059560 con 0x7fdab40c6a50 2024-04-26T23:48:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:29.563 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:41.824964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:48:29.564 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 msgr2=0x7fda9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 0x7fda9006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fda9c034000 tx=0x7fda9c02c000 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 msgr2=0x7fdab40c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c430 tx=0x7fdaa400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 shutdown_connections 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdab41454a0 0x7fdab415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab40cc690 0x7fdab40c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 --2- 172.21.15.46:0/3056606932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab40c6a50 0x7fdab40c5940 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 >> 172.21.15.46:0/3056606932 conn(0x7fdab40b26d0 msgr2=0x7fdab4160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 shutdown_connections 2024-04-26T23:48:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.561+0000 7fdab3200640 1 -- 172.21.15.46:0/3056606932 wait complete. 2024-04-26T23:48:29.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.785+0000 7fa5d5600640 1 Processor -- start 2024-04-26T23:48:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.785+0000 7fa5d5600640 1 -- start start 2024-04-26T23:48:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5d00d2130 0x7fa5d00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d5600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa5d00b4000 0x7fa5d00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d5600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d00d11f0 con 0x7fa5d015f4e0 2024-04-26T23:48:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d5600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d00d0ce0 con 0x7fa5d00d2130 2024-04-26T23:48:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d5600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d00d2990 con 0x7fa5d00b4000 2024-04-26T23:48:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5cfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5d015f4e0 0x7fa5d00d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40428/0 (socket says 172.21.15.46:40428) 2024-04-26T23:48:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5cfe00640 1 -- 172.21.15.46:0/3530683573 learned_addr learned my addr 172.21.15.46:0/3530683573 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/3530683573 >> v2:172.21.15.46:3300/0 conn(0x7fa5d00d2130 0x7fa5d00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5d4c00640 1 --2- 172.21.15.46:0/3530683573 >> v2:172.21.15.142:3300/0 conn(0x7fa5d00b4000 0x7fa5d00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5cf400640 1 -- 172.21.15.46:0/3530683573 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806718888 0 0) 0x7fa5d00d11f0 con 0x7fa5d015f4e0 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.787+0000 7fa5cf400640 1 -- 172.21.15.46:0/3530683573 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b4003520 con 0x7fa5d015f4e0 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.788+0000 7fa5c7e00640 1 -- 172.21.15.46:0/3530683573 >> v2:172.21.15.142:3300/0 conn(0x7fa5d00b4000 msgr2=0x7fa5d00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.788+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/3530683573 >> v2:172.21.15.142:3300/0 conn(0x7fa5d00b4000 0x7fa5d00b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.788+0000 7fa5c7e00640 1 -- 172.21.15.46:0/3530683573 >> v1:172.21.15.46:6789/0 conn(0x7fa5d015f4e0 legacy=0x7fa5d00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.788+0000 7fa5c7e00640 1 -- 172.21.15.46:0/3530683573 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d00d5f20 con 0x7fa5d00d2130 2024-04-26T23:48:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/3530683573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00d2130 0x7fa5d00d2df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097a0 tx=0x7fa5b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1b8d6b3755558c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5cf400640 1 -- 172.21.15.46:0/3530683573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8037070 con 0x7fa5d00d2130 2024-04-26T23:48:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5cf400640 1 -- 172.21.15.46:0/3530683573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b8032440 con 0x7fa5d00d2130 2024-04-26T23:48:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5cf400640 1 -- 172.21.15.46:0/3530683573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8032810 con 0x7fa5d00d2130 2024-04-26T23:48:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5d5600640 1 -- 172.21.15.46:0/3530683573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00d2130 msgr2=0x7fa5d00d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5d5600640 1 --2- 172.21.15.46:0/3530683573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00d2130 0x7fa5d00d2df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80097a0 tx=0x7fa5b802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5d5600640 1 -- 172.21.15.46:0/3530683573 shutdown_connections 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5d5600640 1 --2- 172.21.15.46:0/3530683573 >> v2:172.21.15.142:3300/0 conn(0x7fa5d00b4000 0x7fa5d00b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.789+0000 7fa5d5600640 1 --2- 172.21.15.46:0/3530683573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00d2130 0x7fa5d00d2df0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.790+0000 7fa5d5600640 1 -- 172.21.15.46:0/3530683573 >> 172.21.15.46:0/3530683573 conn(0x7fa5d00d5010 msgr2=0x7fa5d00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.790+0000 7fa5d5600640 1 -- 172.21.15.46:0/3530683573 shutdown_connections 2024-04-26T23:48:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.790+0000 7fa5d5600640 1 -- 172.21.15.46:0/3530683573 wait complete. 2024-04-26T23:48:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.791+0000 7fa5d5600640 1 Processor -- start 2024-04-26T23:48:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.791+0000 7fa5d5600640 1 -- start start 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5d00d2130 0x7fa5d015e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5d015f4e0 0x7fa5d009ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d00b1d60 con 0x7fa5d00b4000 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d00d5c40 con 0x7fa5d015f4e0 2024-04-26T23:48:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.792+0000 7fa5d5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d015f8c0 con 0x7fa5d00d2130 2024-04-26T23:48:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37964/0 (socket says 172.21.15.46:37964) 2024-04-26T23:48:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 -- 172.21.15.46:0/1456234977 learned_addr learned my addr 172.21.15.46:0/1456234977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5d00d2130 0x7fa5d015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 -- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5d00d2130 msgr2=0x7fa5d015e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5d4c00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5d015f4e0 0x7fa5d009ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5d00d2130 0x7fa5d015e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 -- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5d015f4e0 msgr2=0x7fa5d009ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5d015f4e0 0x7fa5d009ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.793+0000 7fa5cfe00640 1 -- 172.21.15.46:0/1456234977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d00ce900 con 0x7fa5d00b4000 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5cfe00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000c430 tx=0x7fa5c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0017070 con 0x7fa5d00b4000 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c0005ce0 con 0x7fa5d00b4000 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00072f0 con 0x7fa5d00b4000 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d015e750 con 0x7fa5d00b4000 2024-04-26T23:48:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.794+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d00cee50 con 0x7fa5d00b4000 2024-04-26T23:48:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.795+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d00cf020 con 0x7fa5d00b4000 2024-04-26T23:48:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.796+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5c001a070 con 0x7fa5d00b4000 2024-04-26T23:48:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.797+0000 7fa5ccc00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 0x7fa5ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.797+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fa5c008b1a0 con 0x7fa5d00b4000 2024-04-26T23:48:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.798+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 0x7fa5ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.799+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 0x7fa5ac06ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8034000 tx=0x7fa5b802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:29.802+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5c00593f0 con 0x7fa5d00b4000 2024-04-26T23:48:30.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.007+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5d00cd2a0 con 0x7fa5d00b4000 2024-04-26T23:48:30.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.009+0000 7fa5ccc00640 1 -- 172.21.15.46:0/1456234977 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17012 (secure 0 0 0) 0x7fa5c0058cf0 con 0x7fa5d00b4000 2024-04-26T23:48:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:30.014 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:03.797242+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.9","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.011+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 msgr2=0x7fa5ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 0x7fa5ac06ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8034000 tx=0x7fa5b802e000 comp rx=0 tx=0).stop 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 msgr2=0x7fa5d01fd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000c430 tx=0x7fa5c000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 shutdown_connections 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa5ac06c920 0x7fa5ac06ede0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5d015f4e0 0x7fa5d009ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5d00d2130 0x7fa5d015e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 --2- 172.21.15.46:0/1456234977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d00b4000 0x7fa5d01fd810 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 >> 172.21.15.46:0/1456234977 conn(0x7fa5d00d5010 msgr2=0x7fa5d00b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 shutdown_connections 2024-04-26T23:48:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.012+0000 7fa5d5600640 1 -- 172.21.15.46:0/1456234977 wait complete. 2024-04-26T23:48:30.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:48:30.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.245+0000 7ffa88a00640 1 Processor -- start 2024-04-26T23:48:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.245+0000 7ffa88a00640 1 -- start start 2024-04-26T23:48:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.247+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa841466b0 0x7ffa8406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.247+0000 7ffa88a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa840720e0 0x7ffa840cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.247+0000 7ffa88a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa84073ad0 con 0x7ffa84160910 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.247+0000 7ffa88a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffa84146a90 con 0x7ffa840720e0 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.247+0000 7ffa88a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa84073d70 con 0x7ffa841466b0 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa840720e0 0x7ffa840cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa83400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa841466b0 0x7ffa8406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa840720e0 0x7ffa840cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:45054/0 (socket says 172.21.15.46:45054) 2024-04-26T23:48:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 -- 172.21.15.46:0/2410157209 learned_addr learned my addr 172.21.15.46:0/2410157209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 -- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa841466b0 msgr2=0x7ffa8406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 --2- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa841466b0 0x7ffa8406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 -- 172.21.15.46:0/2410157209 >> v1:172.21.15.142:6789/0 conn(0x7ffa84160910 legacy=0x7ffa840cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.248+0000 7ffa82a00640 1 -- 172.21.15.46:0/2410157209 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84074050 con 0x7ffa840720e0 2024-04-26T23:48:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.250+0000 7ffa82a00640 1 --2- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840720e0 0x7ffa840cd860 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e50 tx=0x7ffa6c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68133a9fec93577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.250+0000 7ffa82000640 1 -- 172.21.15.46:0/2410157209 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c036070 con 0x7ffa840720e0 2024-04-26T23:48:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.250+0000 7ffa82000640 1 -- 172.21.15.46:0/2410157209 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c031600 con 0x7ffa840720e0 2024-04-26T23:48:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.250+0000 7ffa82000640 1 -- 172.21.15.46:0/2410157209 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0319d0 con 0x7ffa840720e0 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 -- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840720e0 msgr2=0x7ffa840cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 --2- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840720e0 0x7ffa840cd860 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e50 tx=0x7ffa6c02a710 comp rx=0 tx=0).stop 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 -- 172.21.15.46:0/2410157209 shutdown_connections 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 --2- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840720e0 0x7ffa840cd860 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 --2- 172.21.15.46:0/2410157209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa841466b0 0x7ffa8406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 -- 172.21.15.46:0/2410157209 >> 172.21.15.46:0/2410157209 conn(0x7ffa840b26d0 msgr2=0x7ffa840b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 -- 172.21.15.46:0/2410157209 shutdown_connections 2024-04-26T23:48:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.251+0000 7ffa88a00640 1 -- 172.21.15.46:0/2410157209 wait complete. 2024-04-26T23:48:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.253+0000 7ffa88a00640 1 Processor -- start 2024-04-26T23:48:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.253+0000 7ffa88a00640 1 -- start start 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 0x7ffa8415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84160910 0x7ffa84142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8406e3c0 con 0x7ffa841466b0 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cb470 con 0x7ffa84160910 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa88a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840cdda0 con 0x7ffa840720e0 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.254+0000 7ffa82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37976/0 (socket says 172.21.15.46:37976) 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 -- 172.21.15.46:0/4109199454 learned_addr learned my addr 172.21.15.46:0/4109199454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa83400640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 0x7ffa8415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa83e00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84160910 0x7ffa84142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 -- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 msgr2=0x7ffa8415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 0x7ffa8415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 -- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84160910 msgr2=0x7ffa84142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84160910 0x7ffa84142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 -- 172.21.15.46:0/4109199454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa840c8a70 con 0x7ffa841466b0 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa83400640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 0x7ffa8415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.255+0000 7ffa82a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e50 tx=0x7ffa6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.256+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c041020 con 0x7ffa841466b0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.256+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c007820 con 0x7ffa841466b0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.256+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c036040 con 0x7ffa841466b0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.256+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa840c8d50 con 0x7ffa841466b0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.256+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa840c9310 con 0x7ffa841466b0 2024-04-26T23:48:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.257+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa840693b0 con 0x7ffa841466b0 2024-04-26T23:48:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.258+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa6c044020 con 0x7ffa841466b0 2024-04-26T23:48:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.259+0000 7ffa7b400640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.259+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7ffa6c0acac0 con 0x7ffa841466b0 2024-04-26T23:48:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.260+0000 7ffa83400640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.261+0000 7ffa83400640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 0x7ffa6006edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009a20 tx=0x7ffa74008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.263+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa6c04b030 con 0x7ffa841466b0 2024-04-26T23:48:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.497+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa840c6420 con 0x7ffa841466b0 2024-04-26T23:48:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.498+0000 7ffa7b400640 1 -- 172.21.15.46:0/4109199454 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5183 (secure 0 0 0) 0x7ffa6c07ad30 con 0x7ffa841466b0 2024-04-26T23:48:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:30.503 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:41.824964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:48:30.504 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-26T23:48:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 msgr2=0x7ffa6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 0x7ffa6006edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009a20 tx=0x7ffa74008040 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 msgr2=0x7ffa840c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e50 tx=0x7ffa6c004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 shutdown_connections 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84160910 0x7ffa84142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa841466b0 0x7ffa840c9b10 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 --2- 172.21.15.46:0/4109199454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840720e0 0x7ffa8415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 >> 172.21.15.46:0/4109199454 conn(0x7ffa840b26d0 msgr2=0x7ffa8406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 shutdown_connections 2024-04-26T23:48:30.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.501+0000 7ffa88a00640 1 -- 172.21.15.46:0/4109199454 wait complete. 2024-04-26T23:48:30.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.725+0000 7faf87200640 1 Processor -- start 2024-04-26T23:48:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.725+0000 7faf87200640 1 -- start start 2024-04-26T23:48:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 0x7faf8806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf880720e0 0x7faf880cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf88160910 0x7faf88160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faf88073ad0 con 0x7faf880720e0 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faf88146a90 con 0x7faf88160910 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf87200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf88073d70 con 0x7faf881466b0 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf880720e0 0x7faf880cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf880720e0 0x7faf880cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37980/0 (socket says 172.21.15.46:37980) 2024-04-26T23:48:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf85400640 1 -- 172.21.15.46:0/3374843965 learned_addr learned my addr 172.21.15.46:0/3374843965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.727+0000 7faf85e00640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 0x7faf8806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85400640 1 -- 172.21.15.46:0/3374843965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 msgr2=0x7faf8806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85400640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 0x7faf8806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf86800640 1 --2- 172.21.15.46:0/3374843965 >> v2:172.21.15.142:3300/0 conn(0x7faf88160910 0x7faf88160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85400640 1 -- 172.21.15.46:0/3374843965 >> v2:172.21.15.142:3300/0 conn(0x7faf88160910 msgr2=0x7faf88160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85400640 1 --2- 172.21.15.46:0/3374843965 >> v2:172.21.15.142:3300/0 conn(0x7faf88160910 0x7faf88160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85400640 1 -- 172.21.15.46:0/3374843965 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf88074050 con 0x7faf880720e0 2024-04-26T23:48:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.728+0000 7faf85e00640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 0x7faf8806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.729+0000 7faf85400640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf880cd860 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7faf880cc920 tx=0x7faf7402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f6e12a569738aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.729+0000 7faf84a00640 1 -- 172.21.15.46:0/3374843965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74037070 con 0x7faf880720e0 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.729+0000 7faf84a00640 1 -- 172.21.15.46:0/3374843965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74032440 con 0x7faf880720e0 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.729+0000 7faf84a00640 1 -- 172.21.15.46:0/3374843965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74032810 con 0x7faf880720e0 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 -- 172.21.15.46:0/3374843965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 msgr2=0x7faf880cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf880cd860 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7faf880cc920 tx=0x7faf7402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 -- 172.21.15.46:0/3374843965 shutdown_connections 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 --2- 172.21.15.46:0/3374843965 >> v2:172.21.15.142:3300/0 conn(0x7faf88160910 0x7faf88160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf880cd860 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 --2- 172.21.15.46:0/3374843965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf881466b0 0x7faf8806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 -- 172.21.15.46:0/3374843965 >> 172.21.15.46:0/3374843965 conn(0x7faf880b26d0 msgr2=0x7faf880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 -- 172.21.15.46:0/3374843965 shutdown_connections 2024-04-26T23:48:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.730+0000 7faf87200640 1 -- 172.21.15.46:0/3374843965 wait complete. 2024-04-26T23:48:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.732+0000 7faf87200640 1 Processor -- start 2024-04-26T23:48:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.732+0000 7faf87200640 1 -- start start 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.733+0000 7faf87200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.733+0000 7faf87200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf881466b0 0x7faf880cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf87200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf88160910 0x7faf880cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf87200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf8806e3c0 con 0x7faf880720e0 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf87200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf880cb470 con 0x7faf881466b0 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf87200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf880cdda0 con 0x7faf88160910 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37990/0 (socket says 172.21.15.46:37990) 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 -- 172.21.15.46:0/871675896 learned_addr learned my addr 172.21.15.46:0/871675896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 -- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf88160910 msgr2=0x7faf880cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf88160910 0x7faf880cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 -- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf881466b0 msgr2=0x7faf880cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf881466b0 0x7faf880cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.734+0000 7faf85e00640 1 -- 172.21.15.46:0/871675896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf880c8d60 con 0x7faf880720e0 2024-04-26T23:48:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf85e00640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c7f0 tx=0x7faf7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78015070 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf78007820 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf880c9040 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78007bf0 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.735+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf880c9b20 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.736+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf880693b0 con 0x7faf880720e0 2024-04-26T23:48:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.737+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf78018070 con 0x7faf880720e0 2024-04-26T23:48:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.738+0000 7faf7e000640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 0x7faf6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.738+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7faf7808bf70 con 0x7faf880720e0 2024-04-26T23:48:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.739+0000 7faf85400640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 0x7faf6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.740+0000 7faf85400640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 0x7faf6806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faf74034000 tx=0x7faf7402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.742+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf7805a120 con 0x7faf880720e0 2024-04-26T23:48:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.980+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf880b2440 con 0x7faf880720e0 2024-04-26T23:48:30.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.982+0000 7faf7e000640 1 -- 172.21.15.46:0/871675896 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5183 (secure 0 0 0) 0x7faf78059a20 con 0x7faf880720e0 2024-04-26T23:48:30.987 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-26T23:45:32.336330+0000","modified":"2024-04-26T23:45:41.824964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8253":{"gid":8253,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2078483956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2078483956},{"type":"v1","addr":"172.21.15.142:6837","nonce":2078483956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8253,"qdb_cluster":[8253]},"id":7}]} 2024-04-26T23:48:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-26T23:48:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 msgr2=0x7faf6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 0x7faf6806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faf74034000 tx=0x7faf7402c000 comp rx=0 tx=0).stop 2024-04-26T23:48:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 msgr2=0x7faf8815f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c7f0 tx=0x7faf7800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 shutdown_connections 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf6806c8f0 0x7faf6806edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf88160910 0x7faf880cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf881466b0 0x7faf880cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 --2- 172.21.15.46:0/871675896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf880720e0 0x7faf8815f680 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 >> 172.21.15.46:0/871675896 conn(0x7faf880b26d0 msgr2=0x7faf880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.984+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 shutdown_connections 2024-04-26T23:48:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:30.985+0000 7faf87200640 1 -- 172.21.15.46:0/871675896 wait complete. 2024-04-26T23:48:31.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.211+0000 7fb549000640 1 Processor -- start 2024-04-26T23:48:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.211+0000 7fb549000640 1 -- start start 2024-04-26T23:48:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb549000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5440c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb549000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb544072bb0 con 0x7fb5441454a0 2024-04-26T23:48:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb549000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb544072e90 con 0x7fb5440cc690 2024-04-26T23:48:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb549000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb544073170 con 0x7fb5440c6a50 2024-04-26T23:48:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb543400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5441454a0 0x7fb544073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40442/0 (socket says 172.21.15.46:40442) 2024-04-26T23:48:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb543400640 1 -- 172.21.15.46:0/3434340082 learned_addr learned my addr 172.21.15.46:0/3434340082 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.213+0000 7fb543e00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5440c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3258606585 0 0) 0x7fb544072bb0 con 0x7fb5441454a0 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb528005730 con 0x7fb5441454a0 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4041860151 0 0) 0x7fb528005730 con 0x7fb5441454a0 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 msgr2=0x7fb5440c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5440c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 >> v1:172.21.15.142:6789/0 conn(0x7fb5440cc690 legacy=0x7fb5440cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb544073450 con 0x7fb5441454a0 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb534001670 con 0x7fb5441454a0 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 >> v1:172.21.15.46:6789/0 conn(0x7fb5441454a0 legacy=0x7fb544073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528005bf0 0x7fb528006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb528006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5280073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb544073170 con 0x7fb528006a10 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb52800dc80 con 0x7fb528005bf0 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.214+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb52800de80 con 0x7fb5440c6a50 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb528006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb543400640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528005bf0 0x7fb528006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb543e00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5280073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 -- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 msgr2=0x7fb5280073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5280073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 -- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528005bf0 msgr2=0x7fb528006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528005bf0 0x7fb528006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb542a00640 1 -- 172.21.15.46:0/3434340082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb528005730 con 0x7fb528006a10 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.215+0000 7fb543e00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5440c6a50 0x7fb5280073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.216+0000 7fb542a00640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb528006e90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb530001720 tx=0x7fb528007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e3ff700d9e795a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.216+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530001370 con 0x7fb528006a10 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.216+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5300052d0 con 0x7fb528006a10 2024-04-26T23:48:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.216+0000 7fb542000640 1 -- 172.21.15.46:0/3434340082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53002ab90 con 0x7fb528006a10 2024-04-26T23:48:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 -- 172.21.15.46:0/3434340082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 msgr2=0x7fb528006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 --2- 172.21.15.46:0/3434340082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb528006e90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb530001720 tx=0x7fb528007a40 comp rx=0 tx=0).stop 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb543e00640 1 -- 172.21.15.46:0/3434340082 reap_dead start 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 -- 172.21.15.46:0/3434340082 shutdown_connections 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 -- 172.21.15.46:0/3434340082 >> 172.21.15.46:0/3434340082 conn(0x7fb5440b26d0 msgr2=0x7fb5440b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 -- 172.21.15.46:0/3434340082 shutdown_connections 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.217+0000 7fb549000640 1 -- 172.21.15.46:0/3434340082 wait complete. 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.219+0000 7fb549000640 1 Processor -- start 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.219+0000 7fb549000640 1 -- start start 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb54415c800 0x7fb54415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb54415e790 0x7fb54415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb544145a00 con 0x7fb528006a10 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb544145880 con 0x7fb54415e790 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb549000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440c6ee0 con 0x7fb54415c800 2024-04-26T23:48:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37994/0 (socket says 172.21.15.46:37994) 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.220+0000 7fb543400640 1 -- 172.21.15.46:0/1831475001 learned_addr learned my addr 172.21.15.46:0/1831475001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb542a00640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb54415c800 0x7fb54415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 -- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb54415c800 msgr2=0x7fb54415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb54415c800 0x7fb54415acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 -- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb54415e790 msgr2=0x7fb54415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543e00640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb54415e790 0x7fb54415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb54415e790 0x7fb54415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 -- 172.21.15.46:0/1831475001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54415cd50 con 0x7fb528006a10 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb543400640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb53400e760 tx=0x7fb53400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53400ee70 con 0x7fb528006a10 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb534018070 con 0x7fb528006a10 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53400cb30 con 0x7fb528006a10 2024-04-26T23:48:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.221+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54415d030 con 0x7fb528006a10 2024-04-26T23:48:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.222+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5440b4ca0 con 0x7fb528006a10 2024-04-26T23:48:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.223+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5440b4ea0 con 0x7fb528006a10 2024-04-26T23:48:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.224+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb53400ccd0 con 0x7fb528006a10 2024-04-26T23:48:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.225+0000 7fb53b400640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 0x7fb52406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.226+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fb53408bc40 con 0x7fb528006a10 2024-04-26T23:48:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.226+0000 7fb542a00640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 0x7fb52406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.227+0000 7fb542a00640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 0x7fb52406ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb530032000 tx=0x7fb53002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.230+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5440b4ea0 con 0x7fb528006a10 2024-04-26T23:48:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.438+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5440693b0 con 0x7fb528006a10 2024-04-26T23:48:31.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.441+0000 7fb53b400640 1 -- 172.21.15.46:0/1831475001 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17012 (secure 0 0 0) 0x7fb534059e00 con 0x7fb528006a10 2024-04-26T23:48:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:31.447 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:03.797242+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:45:30.380235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T23:45:31.336100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.9","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 msgr2=0x7fb52406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 0x7fb52406ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb530032000 tx=0x7fb53002e000 comp rx=0 tx=0).stop 2024-04-26T23:48:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 msgr2=0x7fb54413bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb53400e760 tx=0x7fb53400ec20 comp rx=0 tx=0).stop 2024-04-26T23:48:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 shutdown_connections 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb52406c920 0x7fb52406ede0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb54415e790 0x7fb54415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb54415c800 0x7fb54415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 --2- 172.21.15.46:0/1831475001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb528006a10 0x7fb54413bee0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.444+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 >> 172.21.15.46:0/1831475001 conn(0x7fb5440b26d0 msgr2=0x7fb5440b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.445+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 shutdown_connections 2024-04-26T23:48:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.445+0000 7fb549000640 1 -- 172.21.15.46:0/1831475001 wait complete. 2024-04-26T23:48:31.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:48:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.679+0000 7fe0e3200640 1 Processor -- start 2024-04-26T23:48:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.679+0000 7fe0e3200640 1 -- start start 2024-04-26T23:48:31.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 0x7fe0e40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe0e406c350 0x7fe0e406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e40cff50 con 0x7fe0e41670a0 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e4168070 con 0x7fe0e406c350 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e40d0560 con 0x7fe0e40c5200 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe0e41670a0 0x7fe0e4167530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40450/0 (socket says 172.21.15.46:40450) 2024-04-26T23:48:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e2800640 1 -- 172.21.15.46:0/2910548227 learned_addr learned my addr 172.21.15.46:0/2910548227 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 0x7fe0e40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.681+0000 7fe0e1400640 1 --2- 172.21.15.46:0/2910548227 >> v2:172.21.15.142:3300/0 conn(0x7fe0e406c350 0x7fe0e406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481232326 0 0) 0x7fe0e40cff50 con 0x7fe0e41670a0 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0c4003520 con 0x7fe0e41670a0 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2897539211 0 0) 0x7fe0c4003520 con 0x7fe0e41670a0 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 msgr2=0x7fe0e40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 0x7fe0e40d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 >> v2:172.21.15.142:3300/0 conn(0x7fe0e406c350 msgr2=0x7fe0e406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 --2- 172.21.15.46:0/2910548227 >> v2:172.21.15.142:3300/0 conn(0x7fe0e406c350 0x7fe0e406c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e4167d90 con 0x7fe0e41670a0 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe0dc001670 con 0x7fe0e41670a0 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 >> v1:172.21.15.46:6789/0 conn(0x7fe0e41670a0 legacy=0x7fe0e4167530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 0x7fe0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4168070 con 0x7fe0c40039b0 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e40d0560 con 0x7fe0c4004800 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c400b9c0 con 0x7fe0e40c5200 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.682+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1400640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1e00640 1 -- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 msgr2=0x7fe0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40c5200 0x7fe0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1e00640 1 -- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0c4004800 msgr2=0x7fe0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1e00640 1 -- 172.21.15.46:0/2910548227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4003520 con 0x7fe0c40039b0 2024-04-26T23:48:31.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.683+0000 7fe0e1400640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:31.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.684+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0c4003dd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009ac0 tx=0x7fe0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9feed70776d507 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.684+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4036070 con 0x7fe0c40039b0 2024-04-26T23:48:31.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.684+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d4031400 con 0x7fe0c40039b0 2024-04-26T23:48:31.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.684+0000 7fe0e0a00640 1 -- 172.21.15.46:0/2910548227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4009690 con 0x7fe0c40039b0 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 -- 172.21.15.46:0/2910548227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 msgr2=0x7fe0c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 --2- 172.21.15.46:0/2910548227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0c4003dd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009ac0 tx=0x7fe0c4005830 comp rx=0 tx=0).stop 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e2800640 1 -- 172.21.15.46:0/2910548227 reap_dead start 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 -- 172.21.15.46:0/2910548227 shutdown_connections 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 -- 172.21.15.46:0/2910548227 >> 172.21.15.46:0/2910548227 conn(0x7fe0e40d1cd0 msgr2=0x7fe0e40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 -- 172.21.15.46:0/2910548227 shutdown_connections 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.685+0000 7fe0e3200640 1 -- 172.21.15.46:0/2910548227 wait complete. 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.687+0000 7fe0e3200640 1 Processor -- start 2024-04-26T23:48:31.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.687+0000 7fe0e3200640 1 -- start start 2024-04-26T23:48:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40cbbb0 0x7fe0e415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e40cc030 0x7fe0e415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e40d09f0 con 0x7fe0c40039b0 2024-04-26T23:48:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e406ccd0 con 0x7fe0e40cc030 2024-04-26T23:48:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e40d0b70 con 0x7fe0e40cbbb0 2024-04-26T23:48:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.688+0000 7fe0e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38014/0 (socket says 172.21.15.46:38014) 2024-04-26T23:48:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 -- 172.21.15.46:0/3204985945 learned_addr learned my addr 172.21.15.46:0/3204985945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:31.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 -- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40cbbb0 msgr2=0x7fe0e415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40cbbb0 0x7fe0e415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 -- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e40cc030 msgr2=0x7fe0e415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e40cc030 0x7fe0e415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 -- 172.21.15.46:0/3204985945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e40c9510 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.689+0000 7fe0e1e00640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fe0d402abe0 tx=0x7fe0d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.690+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4040020 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.690+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d40038d0 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.690+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4036040 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.690+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e40c97f0 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.690+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e40c9db0 con 0x7fe0c40039b0 2024-04-26T23:48:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.691+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e4069ac0 con 0x7fe0c40039b0 2024-04-26T23:48:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.692+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0d4043020 con 0x7fe0c40039b0 2024-04-26T23:48:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.693+0000 7fe0da000640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 0x7fe0c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.693+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6187+0+0 (secure 0 0 0) 0x7fe0d40abf50 con 0x7fe0c40039b0 2024-04-26T23:48:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.694+0000 7fe0e1400640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 0x7fe0c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:31.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.695+0000 7fe0e1400640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 0x7fe0c006ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b76f0 tx=0x7fe0cc009750 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.698+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0d404a070 con 0x7fe0c40039b0 2024-04-26T23:48:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.943+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0e406ef00 con 0x7fe0c40039b0 2024-04-26T23:48:31.949 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:48:31.944+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:48:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.948+0000 7fe0da000640 1 -- 172.21.15.46:0/3204985945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 161+0+0 (secure 0 0 0) 0x7fe0d407a100 con 0x7fe0c40039b0 2024-04-26T23:48:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 msgr2=0x7fe0c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 0x7fe0c006ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b76f0 tx=0x7fe0cc009750 comp rx=0 tx=0).stop 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 msgr2=0x7fe0e40c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fe0d402abe0 tx=0x7fe0d4004730 comp rx=0 tx=0).stop 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 shutdown_connections 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe0c006c920 0x7fe0c006ede0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e40cc030 0x7fe0e415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0e40cbbb0 0x7fe0e415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 --2- 172.21.15.46:0/3204985945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c40039b0 0x7fe0e40c1eb0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.951+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 >> 172.21.15.46:0/3204985945 conn(0x7fe0e40d1cd0 msgr2=0x7fe0e406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.952+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 shutdown_connections 2024-04-26T23:48:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:31.953+0000 7fe0e3200640 1 -- 172.21.15.46:0/3204985945 wait complete. 2024-04-26T23:48:31.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:48:32.082 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:48:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.177+0000 7fe96c000640 1 Processor -- start 2024-04-26T23:48:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.178+0000 7fe96c000640 1 -- start start 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.179+0000 7fe96c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9640b7f30 0x7fe9640d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.179+0000 7fe96c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9640d0270 con 0x7fe9640d1b80 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.179+0000 7fe96c000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9640b5e80 con 0x7fe9640cfe90 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.179+0000 7fe96c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe9640b6160 con 0x7fe9640b7f30 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe968c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9640d1b80 0x7fe96415cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36372/0 (socket says 172.21.15.46:36372) 2024-04-26T23:48:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe968c00640 1 -- 172.21.15.46:0/2742187761 learned_addr learned my addr 172.21.15.46:0/2742187761 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:32.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe96a000640 1 --2- 172.21.15.46:0/2742187761 >> v2:172.21.15.46:3300/0 conn(0x7fe9640b7f30 0x7fe9640d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:32.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe963e00640 1 -- 172.21.15.46:0/2742187761 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257214067 0 0) 0x7fe9640d0270 con 0x7fe9640d1b80 2024-04-26T23:48:32.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe963e00640 1 -- 172.21.15.46:0/2742187761 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe948003520 con 0x7fe9640d1b80 2024-04-26T23:48:32.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe96a000640 1 -- 172.21.15.46:0/2742187761 >> v1:172.21.15.142:6789/0 conn(0x7fe9640cfe90 legacy=0x7fe9640d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe96a000640 1 -- 172.21.15.46:0/2742187761 >> v1:172.21.15.46:6789/0 conn(0x7fe9640d1b80 legacy=0x7fe96415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.180+0000 7fe96a000640 1 -- 172.21.15.46:0/2742187761 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9640b6440 con 0x7fe9640b7f30 2024-04-26T23:48:32.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.181+0000 7fe96a000640 1 --2- 172.21.15.46:0/2742187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640d39b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009d90 tx=0x7fe95c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7ad935ceaad00b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:32.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe963e00640 1 -- 172.21.15.46:0/2742187761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007740 con 0x7fe9640b7f30 2024-04-26T23:48:32.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe963e00640 1 -- 172.21.15.46:0/2742187761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95c0078e0 con 0x7fe9640b7f30 2024-04-26T23:48:32.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe963e00640 1 -- 172.21.15.46:0/2742187761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007cb0 con 0x7fe9640b7f30 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe96c000640 1 -- 172.21.15.46:0/2742187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 msgr2=0x7fe9640d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe96c000640 1 --2- 172.21.15.46:0/2742187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640d39b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009d90 tx=0x7fe95c02a660 comp rx=0 tx=0).stop 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe96c000640 1 -- 172.21.15.46:0/2742187761 shutdown_connections 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe96c000640 1 --2- 172.21.15.46:0/2742187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640d39b0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.182+0000 7fe96c000640 1 -- 172.21.15.46:0/2742187761 >> 172.21.15.46:0/2742187761 conn(0x7fe9640d2e90 msgr2=0x7fe9640d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.183+0000 7fe96c000640 1 -- 172.21.15.46:0/2742187761 shutdown_connections 2024-04-26T23:48:32.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.183+0000 7fe96c000640 1 -- 172.21.15.46:0/2742187761 wait complete. 2024-04-26T23:48:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.184+0000 7fe96c000640 1 Processor -- start 2024-04-26T23:48:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 -- start start 2024-04-26T23:48:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9640cfe90 0x7fe9640c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9640d1b80 0x7fe9641f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9640d0c70 con 0x7fe9640b7f30 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9640b6a00 con 0x7fe9640d1b80 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.185+0000 7fe96c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9640d0df0 con 0x7fe9640cfe90 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40730/0 (socket says 172.21.15.46:40730) 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 -- 172.21.15.46:0/2343349748 learned_addr learned my addr 172.21.15.46:0/2343349748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe96a000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9640d1b80 0x7fe9641f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 -- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9640cfe90 msgr2=0x7fe9640c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9640cfe90 0x7fe9640c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 -- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9640d1b80 msgr2=0x7fe9641f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9640d1b80 0x7fe9641f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.186+0000 7fe969600640 1 -- 172.21.15.46:0/2343349748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9640c8a50 con 0x7fe9640b7f30 2024-04-26T23:48:32.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe969600640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c400 tx=0x7fe95400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540052d0 con 0x7fe9640b7f30 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe954005490 con 0x7fe9640b7f30 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954007260 con 0x7fe9640b7f30 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9640c8d30 con 0x7fe9640b7f30 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.187+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641f3220 con 0x7fe9640b7f30 2024-04-26T23:48:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.188+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964069530 con 0x7fe9640b7f30 2024-04-26T23:48:32.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.189+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe954007400 con 0x7fe9640b7f30 2024-04-26T23:48:32.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.190+0000 7fe961600640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 0x7fe94406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:32.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.190+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6245+0+0 (secure 0 0 0) 0x7fe95408b290 con 0x7fe9640b7f30 2024-04-26T23:48:32.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.191+0000 7fe968c00640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 0x7fe94406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.192+0000 7fe968c00640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 0x7fe94406ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe950005880 tx=0x7fe950007950 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:32.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.195+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe954059490 con 0x7fe9640b7f30 2024-04-26T23:48:32.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.441+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe9640b4360 con 0x7fe9640b7f30 2024-04-26T23:48:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.958+0000 7fe961600640 1 -- 172.21.15.46:0/2343349748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) ==== 104+0+0 (secure 0 0 0) 0x7fe954058d90 con 0x7fe9640b7f30 2024-04-26T23:48:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 msgr2=0x7fe94406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 0x7fe94406ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe950005880 tx=0x7fe950007950 comp rx=0 tx=0).stop 2024-04-26T23:48:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 msgr2=0x7fe9640a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c400 tx=0x7fe95400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 shutdown_connections 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe94406c920 0x7fe94406ede0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9640d1b80 0x7fe9641f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9640cfe90 0x7fe9640c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 --2- 172.21.15.46:0/2343349748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640b7f30 0x7fe9640a06d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.961+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 >> 172.21.15.46:0/2343349748 conn(0x7fe9640d2e90 msgr2=0x7fe9640d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.962+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 shutdown_connections 2024-04-26T23:48:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:32.962+0000 7fe96c000640 1 -- 172.21.15.46:0/2343349748 wait complete. 2024-04-26T23:48:32.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:48:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.209+0000 7f1ae8200640 1 Processor -- start 2024-04-26T23:48:33.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.209+0000 7f1ae8200640 1 -- start start 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.210+0000 7f1ae8200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ae015b730 0x7f1ae006f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.210+0000 7f1ae8200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 0x7f1ae00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.210+0000 7f1ae8200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae006fbb0 con 0x7f1ae0070a40 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.210+0000 7f1ae8200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae006c350 con 0x7f1ae015b730 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.210+0000 7f1ae8200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae006fe90 con 0x7f1ae00ccfc0 2024-04-26T23:48:33.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ae015b730 0x7f1ae006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ae015b730 0x7f1ae006f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40740/0 (socket says 172.21.15.46:40740) 2024-04-26T23:48:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 -- 172.21.15.46:0/2992746178 learned_addr learned my addr 172.21.15.46:0/2992746178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae6200640 1 --2- 172.21.15.46:0/2992746178 >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 0x7f1ae00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 -- 172.21.15.46:0/2992746178 >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 msgr2=0x7f1ae00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 --2- 172.21.15.46:0/2992746178 >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 0x7f1ae00d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 -- 172.21.15.46:0/2992746178 >> v1:172.21.15.142:6789/0 conn(0x7f1ae0070a40 legacy=0x7f1ae00cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae5800640 1 -- 172.21.15.46:0/2992746178 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae0070170 con 0x7f1ae015b730 2024-04-26T23:48:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.211+0000 7f1ae6200640 1 --2- 172.21.15.46:0/2992746178 >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 0x7f1ae00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.212+0000 7f1ae5800640 1 --2- 172.21.15.46:0/2992746178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae006f260 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00cf9a0 tx=0x7f1ad002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91069b184d48bee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.213+0000 7f1adfe00640 1 -- 172.21.15.46:0/2992746178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0037070 con 0x7f1ae015b730 2024-04-26T23:48:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.213+0000 7f1adfe00640 1 -- 172.21.15.46:0/2992746178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad0032440 con 0x7f1ae015b730 2024-04-26T23:48:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.213+0000 7f1adfe00640 1 -- 172.21.15.46:0/2992746178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0032810 con 0x7f1ae015b730 2024-04-26T23:48:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.213+0000 7f1ae8200640 1 -- 172.21.15.46:0/2992746178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 msgr2=0x7f1ae006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.213+0000 7f1ae8200640 1 --2- 172.21.15.46:0/2992746178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae006f260 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00cf9a0 tx=0x7f1ad002a230 comp rx=0 tx=0).stop 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 -- 172.21.15.46:0/2992746178 shutdown_connections 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 --2- 172.21.15.46:0/2992746178 >> v2:172.21.15.142:3300/0 conn(0x7f1ae00ccfc0 0x7f1ae00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 --2- 172.21.15.46:0/2992746178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae006f260 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 -- 172.21.15.46:0/2992746178 >> 172.21.15.46:0/2992746178 conn(0x7f1ae00ce030 msgr2=0x7f1ae00ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 -- 172.21.15.46:0/2992746178 shutdown_connections 2024-04-26T23:48:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.214+0000 7f1ae8200640 1 -- 172.21.15.46:0/2992746178 wait complete. 2024-04-26T23:48:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.215+0000 7f1ae8200640 1 Processor -- start 2024-04-26T23:48:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.216+0000 7f1ae8200640 1 -- start start 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ae0070a40 0x7f1ae01517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ae00ccfc0 0x7f1ae00b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae015bb10 con 0x7f1ae015b730 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae006f7a0 con 0x7f1ae00ccfc0 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae8200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae0169ff0 con 0x7f1ae0070a40 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40748/0 (socket says 172.21.15.46:40748) 2024-04-26T23:48:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae6200640 1 -- 172.21.15.46:0/3836279835 learned_addr learned my addr 172.21.15.46:0/3836279835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae5800640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ae0070a40 0x7f1ae01517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.217+0000 7f1ae4e00640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ae00ccfc0 0x7f1ae00b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 -- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ae0070a40 msgr2=0x7f1ae01517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ae0070a40 0x7f1ae01517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 -- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ae00ccfc0 msgr2=0x7f1ae00b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ae00ccfc0 0x7f1ae00b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 -- 172.21.15.46:0/3836279835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae0169d10 con 0x7f1ae015b730 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae6200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800cef0 tx=0x7f1ad800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad801a070 con 0x7f1ae015b730 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad800d860 con 0x7f1ae015b730 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800dc50 con 0x7f1ae015b730 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae0151f70 con 0x7f1ae015b730 2024-04-26T23:48:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.218+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae0168f60 con 0x7f1ae015b730 2024-04-26T23:48:33.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.220+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0069530 con 0x7f1ae015b730 2024-04-26T23:48:33.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.221+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ad801d070 con 0x7f1ae015b730 2024-04-26T23:48:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.222+0000 7f1add600640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 0x7f1ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.222+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6245+0+0 (secure 0 0 0) 0x7f1ad808c610 con 0x7f1ae015b730 2024-04-26T23:48:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.222+0000 7f1ae5800640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 0x7f1ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.223+0000 7f1ae5800640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 0x7f1ac006edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:33.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.226+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ad805a8d0 con 0x7f1ae015b730 2024-04-26T23:48:33.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.426+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ae00cb970 con 0x7f1ae015b730 2024-04-26T23:48:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.961+0000 7f1add600640 1 -- 172.21.15.46:0/3836279835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f1ad805a1d0 con 0x7f1ae015b730 2024-04-26T23:48:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:48:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.965+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 msgr2=0x7f1ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.965+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 0x7f1ac006edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad002e000 comp rx=0 tx=0).stop 2024-04-26T23:48:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.965+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 msgr2=0x7f1ae00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.965+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800cef0 tx=0x7f1ad800a710 comp rx=0 tx=0).stop 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 shutdown_connections 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ac006c8f0 0x7f1ac006edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae015b730 0x7f1ae00b47d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ae00ccfc0 0x7f1ae00b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 --2- 172.21.15.46:0/3836279835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ae0070a40 0x7f1ae01517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.966+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 >> 172.21.15.46:0/3836279835 conn(0x7f1ae00ce030 msgr2=0x7f1ae00b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.967+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 shutdown_connections 2024-04-26T23:48:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:33.968+0000 7f1ae8200640 1 -- 172.21.15.46:0/3836279835 wait complete. 2024-04-26T23:48:33.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:48:34.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.206+0000 7eff06e00640 1 Processor -- start 2024-04-26T23:48:34.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.206+0000 7eff06e00640 1 -- start start 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.207+0000 7eff06e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff00146b20 0x7eff000d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff06e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff000d1620 0x7eff000cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff06e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000b3ce0 con 0x7eff00146740 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff06e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7eff000d1a70 con 0x7eff00146b20 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff06e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7eff000b48f0 con 0x7eff000d1620 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff00146b20 0x7eff000d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff00146b20 0x7eff000d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40754/0 (socket says 172.21.15.46:40754) 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 -- 172.21.15.46:0/1429967670 learned_addr learned my addr 172.21.15.46:0/1429967670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05a00640 1 --1- 172.21.15.46:0/1429967670 >> v1:172.21.15.46:6789/0 conn(0x7eff00146740 0x7eff000cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36384/0 (socket says 172.21.15.46:36384) 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff06400640 1 --2- 172.21.15.46:0/1429967670 >> v2:172.21.15.142:3300/0 conn(0x7eff000d1620 0x7eff000cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 -- 172.21.15.46:0/1429967670 >> v2:172.21.15.142:3300/0 conn(0x7eff000d1620 msgr2=0x7eff000cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 --2- 172.21.15.46:0/1429967670 >> v2:172.21.15.142:3300/0 conn(0x7eff000d1620 0x7eff000cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.208+0000 7eff05000640 1 -- 172.21.15.46:0/1429967670 >> v1:172.21.15.46:6789/0 conn(0x7eff00146740 legacy=0x7eff000cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.209+0000 7eff05000640 1 -- 172.21.15.46:0/1429967670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff000b3fc0 con 0x7eff00146b20 2024-04-26T23:48:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.209+0000 7efeffe00640 1 -- 172.21.15.46:0/1429967670 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550285892 0 0) 0x7eff000b3ce0 con 0x7eff00146740 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff05000640 1 --2- 172.21.15.46:0/1429967670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff000d10e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7eff0006fe90 tx=0x7efeec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a18f57ab0d804c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7efeffe00640 1 -- 172.21.15.46:0/1429967670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec037070 con 0x7eff00146b20 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7efeffe00640 1 -- 172.21.15.46:0/1429967670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec032440 con 0x7eff00146b20 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7efeffe00640 1 -- 172.21.15.46:0/1429967670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec032810 con 0x7eff00146b20 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 -- 172.21.15.46:0/1429967670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 msgr2=0x7eff000d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 --2- 172.21.15.46:0/1429967670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff000d10e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7eff0006fe90 tx=0x7efeec02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 -- 172.21.15.46:0/1429967670 shutdown_connections 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 --2- 172.21.15.46:0/1429967670 >> v2:172.21.15.142:3300/0 conn(0x7eff000d1620 0x7eff000cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 --2- 172.21.15.46:0/1429967670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff000d10e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.210+0000 7eff06e00640 1 -- 172.21.15.46:0/1429967670 >> 172.21.15.46:0/1429967670 conn(0x7eff0006c770 msgr2=0x7eff0006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.211+0000 7eff06e00640 1 -- 172.21.15.46:0/1429967670 shutdown_connections 2024-04-26T23:48:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.211+0000 7eff06e00640 1 -- 172.21.15.46:0/1429967670 wait complete. 2024-04-26T23:48:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.212+0000 7eff06e00640 1 Processor -- start 2024-04-26T23:48:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.212+0000 7eff06e00640 1 -- start start 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 0x7eff000aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff00146740 0x7eff00071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff0015e560 con 0x7eff00146b20 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff000b4580 con 0x7eff000d1620 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.213+0000 7eff06e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7eff000cdde0 con 0x7eff00146740 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40760/0 (socket says 172.21.15.46:40760) 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 -- 172.21.15.46:0/945658266 learned_addr learned my addr 172.21.15.46:0/945658266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff05a00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 0x7eff000aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff05000640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff00146740 0x7eff00071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 -- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff00146740 msgr2=0x7eff00071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff00146740 0x7eff00071e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 -- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 msgr2=0x7eff000aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 0x7eff000aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 -- 172.21.15.46:0/945658266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff000aee90 con 0x7eff00146b20 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff05a00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 0x7eff000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.214+0000 7eff06400640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efef800cc80 tx=0x7efef800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.215+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800ce30 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.215+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef8003990 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.215+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff000af170 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.215+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8003d60 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.215+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff000af730 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.216+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff000693b0 con 0x7eff00146b20 2024-04-26T23:48:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.217+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efef8003f00 con 0x7eff00146b20 2024-04-26T23:48:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.218+0000 7efefd600640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 0x7efee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.218+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5807+0+0 (secure 0 0 0) 0x7efef808c0c0 con 0x7eff00146b20 2024-04-26T23:48:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.219+0000 7eff05a00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 0x7efee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.220+0000 7eff05a00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 0x7efee006edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efef4000f60 tx=0x7efef4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.223+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efef8014030 con 0x7eff00146b20 2024-04-26T23:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.441+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7eff000b36f0 con 0x7eff00146b20 2024-04-26T23:48:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.965+0000 7efefd600640 1 -- 172.21.15.46:0/945658266 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7efef805a3f0 con 0x7eff00146b20 2024-04-26T23:48:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.970+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 msgr2=0x7efee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.970+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 0x7efee006edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efef4000f60 tx=0x7efef4009040 comp rx=0 tx=0).stop 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.970+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 msgr2=0x7eff0015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.970+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efef800cc80 tx=0x7efef800a5c0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 shutdown_connections 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efee006c8f0 0x7efee006edb0 secure :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efef4000f60 tx=0x7efef4009040 comp rx=0 tx=0).stop 2024-04-26T23:48:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff00146b20 0x7eff0015c810 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff00146740 0x7eff00071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 --2- 172.21.15.46:0/945658266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff000d1620 0x7eff000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.971+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 >> 172.21.15.46:0/945658266 conn(0x7eff0006c770 msgr2=0x7eff0015e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.973+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 shutdown_connections 2024-04-26T23:48:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:34.973+0000 7eff06e00640 1 -- 172.21.15.46:0/945658266 wait complete. 2024-04-26T23:48:35.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:48:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.189+0000 7f72c3200640 1 Processor -- start 2024-04-26T23:48:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.189+0000 7f72c3200640 1 -- start start 2024-04-26T23:48:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.190+0000 7f72c3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72c4146740 0x7f72c415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.190+0000 7f72c3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.190+0000 7f72c3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c4146b20 con 0x7f72c40ce670 2024-04-26T23:48:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.190+0000 7f72c3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f72c40ccf30 con 0x7f72c4146740 2024-04-26T23:48:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.190+0000 7f72c3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72c4160af0 con 0x7f72c40d0330 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72c4146740 0x7f72c415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72c4146740 0x7f72c415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40776/0 (socket says 172.21.15.46:40776) 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 -- 172.21.15.46:0/4039378666 learned_addr learned my addr 172.21.15.46:0/4039378666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 -- 172.21.15.46:0/4039378666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 msgr2=0x7f72c41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 --2- 172.21.15.46:0/4039378666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 -- 172.21.15.46:0/4039378666 >> v1:172.21.15.142:6789/0 conn(0x7f72c40ce670 legacy=0x7f72c40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.191+0000 7f72c1e00640 1 -- 172.21.15.46:0/4039378666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c4160dd0 con 0x7f72c4146740 2024-04-26T23:48:35.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.192+0000 7f72c1e00640 1 --2- 172.21.15.46:0/4039378666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415df90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dd15d082156680 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c0a00640 1 -- 172.21.15.46:0/4039378666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4037070 con 0x7f72c4146740 2024-04-26T23:48:35.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c0a00640 1 -- 172.21.15.46:0/4039378666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b4032440 con 0x7f72c4146740 2024-04-26T23:48:35.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c0a00640 1 -- 172.21.15.46:0/4039378666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4032810 con 0x7f72c4146740 2024-04-26T23:48:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 -- 172.21.15.46:0/4039378666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 msgr2=0x7f72c415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 --2- 172.21.15.46:0/4039378666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415df90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b402a210 comp rx=0 tx=0).stop 2024-04-26T23:48:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 -- 172.21.15.46:0/4039378666 shutdown_connections 2024-04-26T23:48:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 --2- 172.21.15.46:0/4039378666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c41612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 --2- 172.21.15.46:0/4039378666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415df90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 -- 172.21.15.46:0/4039378666 >> 172.21.15.46:0/4039378666 conn(0x7f72c40b26d0 msgr2=0x7f72c40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 -- 172.21.15.46:0/4039378666 shutdown_connections 2024-04-26T23:48:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.193+0000 7f72c3200640 1 -- 172.21.15.46:0/4039378666 wait complete. 2024-04-26T23:48:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.195+0000 7f72c3200640 1 Processor -- start 2024-04-26T23:48:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.195+0000 7f72c3200640 1 -- start start 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ce670 0x7f72c415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72c40cccf0 con 0x7f72c4146740 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72c40ccb70 con 0x7f72c40ce670 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.196+0000 7f72c3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72c4161830 con 0x7f72c40d0330 2024-04-26T23:48:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40788/0 (socket says 172.21.15.46:40788) 2024-04-26T23:48:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 -- 172.21.15.46:0/790815626 learned_addr learned my addr 172.21.15.46:0/790815626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c1400640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c1e00640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ce670 0x7f72c415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 -- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 msgr2=0x7f72c415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c415dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 -- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ce670 msgr2=0x7f72c415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ce670 0x7f72c415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.197+0000 7f72c2800640 1 -- 172.21.15.46:0/790815626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c415ce00 con 0x7f72c4146740 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72c2800640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c7b0 tx=0x7f72bc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc017070 con 0x7f72c4146740 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc003da0 con 0x7f72c4146740 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0052b0 con 0x7f72c4146740 2024-04-26T23:48:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c415e6d0 con 0x7f72c4146740 2024-04-26T23:48:35.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.198+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c415ec90 con 0x7f72c4146740 2024-04-26T23:48:35.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.199+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c41612f0 con 0x7f72c4146740 2024-04-26T23:48:35.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.200+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72bc01a070 con 0x7f72c4146740 2024-04-26T23:48:35.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.201+0000 7f72ba000640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 0x7f72a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.201+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f72bc08afe0 con 0x7f72c4146740 2024-04-26T23:48:35.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.202+0000 7f72c1e00640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 0x7f72a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.203+0000 7f72c1e00640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 0x7f72a006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f72b4034000 tx=0x7f72b402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.206+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72c41612f0 con 0x7f72c4146740 2024-04-26T23:48:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.410+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f72c40b6cb0 con 0x7f72c4146740 2024-04-26T23:48:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.415+0000 7f72ba000640 1 -- 172.21.15.46:0/790815626 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f72c415ce00 con 0x7f72c4146740 2024-04-26T23:48:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 msgr2=0x7f72a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 0x7f72a006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f72b4034000 tx=0x7f72b402e000 comp rx=0 tx=0).stop 2024-04-26T23:48:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 msgr2=0x7f72c415c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c7b0 tx=0x7f72bc00cc70 comp rx=0 tx=0).stop 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 shutdown_connections 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72a006c8f0 0x7f72a006edb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c4146740 0x7f72c415c8c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72c40d0330 0x7f72c415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 --2- 172.21.15.46:0/790815626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ce670 0x7f72c415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 >> 172.21.15.46:0/790815626 conn(0x7f72c40b26d0 msgr2=0x7f72c40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 shutdown_connections 2024-04-26T23:48:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.418+0000 7f72c3200640 1 -- 172.21.15.46:0/790815626 wait complete. 2024-04-26T23:48:35.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-26T23:48:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.645+0000 7ff9bea00640 1 Processor -- start 2024-04-26T23:48:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.646+0000 7ff9bea00640 1 -- start start 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9b8073db0 0x7ff9b8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b806f1d0 con 0x7ff9b80cf450 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b80741c0 con 0x7ff9b80cff40 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b80726d0 con 0x7ff9b8073db0 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bd600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9b80cf450 0x7ff9b80cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36400/0 (socket says 172.21.15.46:36400) 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9bd600640 1 -- 172.21.15.46:0/3615952860 learned_addr learned my addr 172.21.15.46:0/3615952860 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9be000640 1 --2- 172.21.15.46:0/3615952860 >> v2:172.21.15.46:3300/0 conn(0x7ff9b8073db0 0x7ff9b8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9be000640 1 -- 172.21.15.46:0/3615952860 >> v1:172.21.15.142:6789/0 conn(0x7ff9b80cff40 legacy=0x7ff9b8164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9be000640 1 -- 172.21.15.46:0/3615952860 >> v1:172.21.15.46:6789/0 conn(0x7ff9b80cf450 legacy=0x7ff9b80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9be000640 1 -- 172.21.15.46:0/3615952860 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b806f470 con 0x7ff9b8073db0 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.648+0000 7ff9b7e00640 1 -- 172.21.15.46:0/3615952860 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299103125 0 0) 0x7ff9b806f1d0 con 0x7ff9b80cf450 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9be000640 1 --2- 172.21.15.46:0/3615952860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b8073db0 0x7ff9b8072190 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe1e931f1a3b976 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9b7e00640 1 -- 172.21.15.46:0/3615952860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0007740 con 0x7ff9b8073db0 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9b7e00640 1 -- 172.21.15.46:0/3615952860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b00078e0 con 0x7ff9b8073db0 2024-04-26T23:48:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9b7e00640 1 -- 172.21.15.46:0/3615952860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0007cb0 con 0x7ff9b8073db0 2024-04-26T23:48:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9bea00640 1 -- 172.21.15.46:0/3615952860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b8073db0 msgr2=0x7ff9b8072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.650+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3615952860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b8073db0 0x7ff9b8072190 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b002a660 comp rx=0 tx=0).stop 2024-04-26T23:48:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.651+0000 7ff9bea00640 1 -- 172.21.15.46:0/3615952860 shutdown_connections 2024-04-26T23:48:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.651+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3615952860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b8073db0 0x7ff9b8072190 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.651+0000 7ff9bea00640 1 -- 172.21.15.46:0/3615952860 >> 172.21.15.46:0/3615952860 conn(0x7ff9b80b2820 msgr2=0x7ff9b80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.651+0000 7ff9bea00640 1 -- 172.21.15.46:0/3615952860 shutdown_connections 2024-04-26T23:48:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.651+0000 7ff9bea00640 1 -- 172.21.15.46:0/3615952860 wait complete. 2024-04-26T23:48:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.653+0000 7ff9bea00640 1 Processor -- start 2024-04-26T23:48:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.653+0000 7ff9bea00640 1 -- start start 2024-04-26T23:48:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 0x7ff9b813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 0x7ff9b81df380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8072ef0 con 0x7ff9b80cff40 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8072c10 con 0x7ff9b8073db0 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80731d0 con 0x7ff9b80cf450 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40814/0 (socket says 172.21.15.46:40814) 2024-04-26T23:48:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 -- 172.21.15.46:0/3958507584 learned_addr learned my addr 172.21.15.46:0/3958507584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bcc00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 0x7ff9b81df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9bd600640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 0x7ff9b813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 -- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 msgr2=0x7ff9b81df380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 0x7ff9b81df380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.654+0000 7ff9be000640 1 -- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 msgr2=0x7ff9b813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9be000640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 0x7ff9b813ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9be000640 1 -- 172.21.15.46:0/3958507584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b81df8c0 con 0x7ff9b80cff40 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9bcc00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 0x7ff9b81df380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9bd600640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 0x7ff9b813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9be000640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b003f070 con 0x7ff9b80cff40 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b0031c90 con 0x7ff9b80cff40 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0043630 con 0x7ff9b80cff40 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b8144de0 con 0x7ff9b80cff40 2024-04-26T23:48:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.655+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b81453a0 con 0x7ff9b80cff40 2024-04-26T23:48:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.656+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b80693b0 con 0x7ff9b80cff40 2024-04-26T23:48:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.658+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9b003b410 con 0x7ff9b80cff40 2024-04-26T23:48:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.658+0000 7ff9b5600640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 0x7ff99806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.659+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7ff9b00acc20 con 0x7ff9b80cff40 2024-04-26T23:48:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.659+0000 7ff9bd600640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 0x7ff99806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:35.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.660+0000 7ff9bd600640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 0x7ff99806ede0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f60 tx=0x7ff9ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.663+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9b007b1e0 con 0x7ff9b80cff40 2024-04-26T23:48:35.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.939+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) -- 0x7ff9b80cf830 con 0x7ff9b80cff40 2024-04-26T23:48:35.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.986+0000 7ff9b5600640 1 -- 172.21.15.46:0/3958507584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v530) ==== 158+0+0 (secure 0 0 0) 0x7ff9b007aae0 con 0x7ff9b80cff40 2024-04-26T23:48:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 msgr2=0x7ff99806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 0x7ff99806ede0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f60 tx=0x7ff9ac009040 comp rx=0 tx=0).stop 2024-04-26T23:48:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 msgr2=0x7ff9b81dfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b0004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 shutdown_connections 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff99806c920 0x7ff99806ede0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b80cff40 0x7ff9b81dfc60 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9b80cf450 0x7ff9b81df380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 --2- 172.21.15.46:0/3958507584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8073db0 0x7ff9b813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.988+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 >> 172.21.15.46:0/3958507584 conn(0x7ff9b80b2820 msgr2=0x7ff9b80b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.989+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 shutdown_connections 2024-04-26T23:48:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:35.989+0000 7ff9bea00640 1 -- 172.21.15.46:0/3958507584 wait complete. 2024-04-26T23:48:36.010 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-26T23:48:36.010 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-26T23:48:36.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.213+0000 7f67eaa00640 1 Processor -- start 2024-04-26T23:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.213+0000 7f67eaa00640 1 -- start start 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e415aa20 0x7f67e415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67e4160ae0 0x7f67e4160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67e406c350 0x7f67e406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f67e415d4f0 con 0x7f67e406c350 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f67e4161530 con 0x7f67e4160ae0 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67eaa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67e4161810 con 0x7f67e415aa20 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.215+0000 7f67ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67e406c350 0x7f67e406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67e406c350 0x7f67e406c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40826/0 (socket says 172.21.15.46:40826) 2024-04-26T23:48:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 -- 172.21.15.46:0/2304302959 learned_addr learned my addr 172.21.15.46:0/2304302959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67e9600640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e415aa20 0x7f67e415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67e8c00640 1 --2- 172.21.15.46:0/2304302959 >> v2:172.21.15.142:3300/0 conn(0x7f67e4160ae0 0x7f67e4160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 -- 172.21.15.46:0/2304302959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e415aa20 msgr2=0x7f67e415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e415aa20 0x7f67e415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 -- 172.21.15.46:0/2304302959 >> v2:172.21.15.142:3300/0 conn(0x7f67e4160ae0 msgr2=0x7f67e4160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 --2- 172.21.15.46:0/2304302959 >> v2:172.21.15.142:3300/0 conn(0x7f67e4160ae0 0x7f67e4160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.216+0000 7f67ea000640 1 -- 172.21.15.46:0/2304302959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e415d7b0 con 0x7f67e406c350 2024-04-26T23:48:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.217+0000 7f67ea000640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e406c350 0x7f67e406c7e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f67e406edd0 tx=0x7f67dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4d9c486f6b15d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.218+0000 7f67e3e00640 1 -- 172.21.15.46:0/2304302959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc036070 con 0x7f67e406c350 2024-04-26T23:48:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.218+0000 7f67e3e00640 1 -- 172.21.15.46:0/2304302959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc031600 con 0x7f67e406c350 2024-04-26T23:48:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.218+0000 7f67e3e00640 1 -- 172.21.15.46:0/2304302959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0319d0 con 0x7f67e406c350 2024-04-26T23:48:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.218+0000 7f67eaa00640 1 -- 172.21.15.46:0/2304302959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e406c350 msgr2=0x7f67e406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.218+0000 7f67eaa00640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e406c350 0x7f67e406c7e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f67e406edd0 tx=0x7f67dc02a610 comp rx=0 tx=0).stop 2024-04-26T23:48:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 -- 172.21.15.46:0/2304302959 shutdown_connections 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e406c350 0x7f67e406c7e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 --2- 172.21.15.46:0/2304302959 >> v2:172.21.15.142:3300/0 conn(0x7f67e4160ae0 0x7f67e4160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 --2- 172.21.15.46:0/2304302959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e415aa20 0x7f67e415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 -- 172.21.15.46:0/2304302959 >> 172.21.15.46:0/2304302959 conn(0x7f67e40d1cd0 msgr2=0x7f67e40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 -- 172.21.15.46:0/2304302959 shutdown_connections 2024-04-26T23:48:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.219+0000 7f67eaa00640 1 -- 172.21.15.46:0/2304302959 wait complete. 2024-04-26T23:48:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.220+0000 7f67eaa00640 1 Processor -- start 2024-04-26T23:48:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.220+0000 7f67eaa00640 1 -- start start 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e406c350 0x7f67e40b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67e4160ae0 0x7f67e4158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40712e0 con 0x7f67e415aa20 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67e415ae00 con 0x7f67e4160ae0 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.221+0000 7f67eaa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67e4071460 con 0x7f67e406c350 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40836/0 (socket says 172.21.15.46:40836) 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 -- 172.21.15.46:0/3083966071 learned_addr learned my addr 172.21.15.46:0/3083966071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e9600640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e406c350 0x7f67e40b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67ea000640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67e4160ae0 0x7f67e4158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 -- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e406c350 msgr2=0x7f67e40b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e406c350 0x7f67e40b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 -- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67e4160ae0 msgr2=0x7f67e4158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67e4160ae0 0x7f67e4158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.222+0000 7f67e8c00640 1 -- 172.21.15.46:0/3083966071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e4154920 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67e8c00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d830 tx=0x7f67d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000b6c0 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d0019070 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000ba80 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e4154c00 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.223+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41551c0 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.224+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e4069ac0 con 0x7f67e415aa20 2024-04-26T23:48:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.225+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67d001e460 con 0x7f67e415aa20 2024-04-26T23:48:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.226+0000 7f67e1600640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 0x7f67c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.227+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f67d008c160 con 0x7f67e415aa20 2024-04-26T23:48:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.227+0000 7f67e9600640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 0x7f67c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.228+0000 7f67e9600640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 0x7f67c406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006fd0 tx=0x7f67d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.232+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67d005a660 con 0x7f67e415aa20 2024-04-26T23:48:36.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.443+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67e40d29d0 con 0x7f67e415aa20 2024-04-26T23:48:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.445+0000 7f67e1600640 1 -- 172.21.15.46:0/3083966071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+13269 (secure 0 0 0) 0x7f67d0059f60 con 0x7f67e415aa20 2024-04-26T23:48:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:36.450 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:34.962439+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6837/2078483956":"2024-04-27T23:48:31.945325+0000","172.21.15.142:6836/2078483956":"2024-04-27T23:48:31.945325+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.447+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 msgr2=0x7f67c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.447+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 0x7f67c406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006fd0 tx=0x7f67d8008040 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.447+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 msgr2=0x7f67e41520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.447+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d830 tx=0x7f67d000dcf0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 shutdown_connections 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67c406c8f0 0x7f67c406edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67e4160ae0 0x7f67e4158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415aa20 0x7f67e41520c0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 --2- 172.21.15.46:0/3083966071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67e406c350 0x7f67e40b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 >> 172.21.15.46:0/3083966071 conn(0x7f67e40d1cd0 msgr2=0x7f67e40b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 shutdown_connections 2024-04-26T23:48:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.448+0000 7f67eaa00640 1 -- 172.21.15.46:0/3083966071 wait complete. 2024-04-26T23:48:36.471 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:48:36.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:48:36.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.674+0000 7f1b6fa00640 1 Processor -- start 2024-04-26T23:48:36.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.674+0000 7f1b6fa00640 1 -- start start 2024-04-26T23:48:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6fa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b680d3d60 0x7f1b680cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6fa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b680b64c0 0x7f1b680d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b680b46f0 con 0x7f1b68162aa0 2024-04-26T23:48:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6fa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b680b44a0 con 0x7f1b680b64c0 2024-04-26T23:48:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6fa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b680b5020 con 0x7f1b680d3d60 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b68162aa0 0x7f1b680d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36414/0 (socket says 172.21.15.46:36414) 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.675+0000 7f1b6d000640 1 -- 172.21.15.46:0/1770184699 learned_addr learned my addr 172.21.15.46:0/1770184699 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1770184699 >> v2:172.21.15.46:3300/0 conn(0x7f1b680b64c0 0x7f1b680d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b67e00640 1 --2- 172.21.15.46:0/1770184699 >> v2:172.21.15.142:3300/0 conn(0x7f1b680d3d60 0x7f1b680cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b67400640 1 -- 172.21.15.46:0/1770184699 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707576820 0 0) 0x7f1b680b46f0 con 0x7f1b68162aa0 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b6da00640 1 -- 172.21.15.46:0/1770184699 >> v2:172.21.15.142:3300/0 conn(0x7f1b680d3d60 msgr2=0x7f1b680cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1770184699 >> v2:172.21.15.142:3300/0 conn(0x7f1b680d3d60 0x7f1b680cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b6da00640 1 -- 172.21.15.46:0/1770184699 >> v1:172.21.15.46:6789/0 conn(0x7f1b68162aa0 legacy=0x7f1b680d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.676+0000 7f1b6da00640 1 -- 172.21.15.46:0/1770184699 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b680b49d0 con 0x7f1b680b64c0 2024-04-26T23:48:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.677+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1770184699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b680b64c0 0x7f1b680d01c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f1b680c89a0 tx=0x7f1b6002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1450228560d4a194 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b67400640 1 -- 172.21.15.46:0/1770184699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007740 con 0x7f1b680b64c0 2024-04-26T23:48:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b67400640 1 -- 172.21.15.46:0/1770184699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b600078e0 con 0x7f1b680b64c0 2024-04-26T23:48:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b67400640 1 -- 172.21.15.46:0/1770184699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007cb0 con 0x7f1b680b64c0 2024-04-26T23:48:36.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1770184699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b680b64c0 msgr2=0x7f1b680d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1770184699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b680b64c0 0x7f1b680d01c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f1b680c89a0 tx=0x7f1b6002a5f0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1770184699 shutdown_connections 2024-04-26T23:48:36.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1770184699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b680b64c0 0x7f1b680d01c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1770184699 >> v2:172.21.15.142:3300/0 conn(0x7f1b680d3d60 0x7f1b680cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1770184699 >> 172.21.15.46:0/1770184699 conn(0x7f1b680d5070 msgr2=0x7f1b680d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:36.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1770184699 shutdown_connections 2024-04-26T23:48:36.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.678+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1770184699 wait complete. 2024-04-26T23:48:36.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.680+0000 7f1b6fa00640 1 Processor -- start 2024-04-26T23:48:36.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.680+0000 7f1b6fa00640 1 -- start start 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b680b64c0 0x7f1b680ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b680d3d60 0x7f1b680c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680d30f0 con 0x7f1b68162aa0 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b68162e80 con 0x7f1b680b64c0 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.681+0000 7f1b6fa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b680d5b90 con 0x7f1b680d3d60 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40850/0 (socket says 172.21.15.46:40850) 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 -- 172.21.15.46:0/1215039699 learned_addr learned my addr 172.21.15.46:0/1215039699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b67e00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b680d3d60 0x7f1b680c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 -- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b680d3d60 msgr2=0x7f1b680c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6d000640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b680b64c0 0x7f1b680ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b680d3d60 0x7f1b680c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 -- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b680b64c0 msgr2=0x7f1b680ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b680b64c0 0x7f1b680ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 -- 172.21.15.46:0/1215039699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b680c9620 con 0x7f1b68162aa0 2024-04-26T23:48:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.682+0000 7f1b6da00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009e50 tx=0x7f1b60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.683+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60040070 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.683+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60031bf0 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.683+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60045460 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.683+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b680c9900 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.683+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b680ca1a0 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.684+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b68069530 con 0x7f1b68162aa0 2024-04-26T23:48:36.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.685+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b60039040 con 0x7f1b68162aa0 2024-04-26T23:48:36.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.686+0000 7f1b64c00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:36.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.686+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f1b600ad4d0 con 0x7f1b68162aa0 2024-04-26T23:48:36.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.687+0000 7f1b6d000640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:36.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.688+0000 7f1b6d000640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 0x7f1b4806edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000f60 tx=0x7f1b58009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:36.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.690+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b6007baf0 con 0x7f1b68162aa0 2024-04-26T23:48:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.896+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b680c9ad0 con 0x7f1b68162aa0 2024-04-26T23:48:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.897+0000 7f1b64c00640 1 -- 172.21.15.46:0/1215039699 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f1b60043e40 con 0x7f1b68162aa0 2024-04-26T23:48:36.902 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.899+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 msgr2=0x7f1b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 0x7f1b4806edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000f60 tx=0x7f1b58009040 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 msgr2=0x7f1b681510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009e50 tx=0x7f1b60004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 shutdown_connections 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68162aa0 0x7f1b681510a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b680d3d60 0x7f1b680c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 --2- 172.21.15.46:0/1215039699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b680b64c0 0x7f1b680ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 >> 172.21.15.46:0/1215039699 conn(0x7f1b680d5070 msgr2=0x7f1b68161400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 shutdown_connections 2024-04-26T23:48:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:36.900+0000 7f1b6fa00640 1 -- 172.21.15.46:0/1215039699 wait complete. 2024-04-26T23:48:36.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-26T23:48:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.127+0000 7f9b59e00640 1 Processor -- start 2024-04-26T23:48:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.127+0000 7f9b59e00640 1 -- start start 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.129+0000 7f9b59e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b54162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.129+0000 7f9b59e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b540af0c0 con 0x7f9b54159950 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.129+0000 7f9b59e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b540af3a0 con 0x7f9b541631d0 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.129+0000 7f9b59e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b54163620 con 0x7f9b540b48a0 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.130+0000 7f9b58a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9b54159950 0x7f9b540ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36418/0 (socket says 172.21.15.46:36418) 2024-04-26T23:48:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.130+0000 7f9b58a00640 1 -- 172.21.15.46:0/3787123912 learned_addr learned my addr 172.21.15.46:0/3787123912 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.130+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b54162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.130+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240658985 0 0) 0x7f9b540af0c0 con 0x7f9b54159950 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.130+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b3c003520 con 0x7f9b54159950 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 662839530 0 0) 0x7f9b3c003520 con 0x7f9b54159950 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 msgr2=0x7f9b54162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b54162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 >> v1:172.21.15.142:6789/0 conn(0x7f9b541631d0 legacy=0x7f9b5406e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b540af680 con 0x7f9b54159950 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9b48001670 con 0x7f9b54159950 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 >> v1:172.21.15.46:6789/0 conn(0x7f9b54159950 legacy=0x7f9b540ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0039e0 0x7f9b3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c004800 0x7f9b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54163620 con 0x7f9b3c004800 2024-04-26T23:48:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c00baa0 con 0x7f9b3c0039e0 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c00bca0 con 0x7f9b540b48a0 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c004800 0x7f9b3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.131+0000 7f9b58a00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0039e0 0x7f9b3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b59400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b53e00640 1 -- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 msgr2=0x7f9b3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b53e00640 1 -- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0039e0 msgr2=0x7f9b3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0039e0 0x7f9b3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b53e00640 1 -- 172.21.15.46:0/3787123912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c003520 con 0x7f9b3c004800 2024-04-26T23:48:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.132+0000 7f9b59400640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540b48a0 0x7f9b3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.133+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c004800 0x7f9b3c004c80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009a90 tx=0x7f9b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519f7e18b5a256e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.133+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40036070 con 0x7f9b3c004800 2024-04-26T23:48:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.133+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b40031400 con 0x7f9b3c004800 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.133+0000 7f9b53400640 1 -- 172.21.15.46:0/3787123912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40009690 con 0x7f9b3c004800 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 -- 172.21.15.46:0/3787123912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c004800 msgr2=0x7f9b3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3787123912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c004800 0x7f9b3c004c80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009a90 tx=0x7f9b3c005830 comp rx=0 tx=0).stop 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59400640 1 -- 172.21.15.46:0/3787123912 reap_dead start 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 -- 172.21.15.46:0/3787123912 shutdown_connections 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 -- 172.21.15.46:0/3787123912 >> 172.21.15.46:0/3787123912 conn(0x7f9b540b2820 msgr2=0x7f9b540b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 -- 172.21.15.46:0/3787123912 shutdown_connections 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.134+0000 7f9b59e00640 1 -- 172.21.15.46:0/3787123912 wait complete. 2024-04-26T23:48:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.136+0000 7f9b59e00640 1 Processor -- start 2024-04-26T23:48:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.136+0000 7f9b59e00640 1 -- start start 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c004800 0x7f9b5414be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54149300 0x7f9b54162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540b0160 con 0x7f9b541541c0 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540afc40 con 0x7f9b3c004800 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9b54162c90 con 0x7f9b54149300 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40864/0 (socket says 172.21.15.46:40864) 2024-04-26T23:48:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.137+0000 7f9b59400640 1 -- 172.21.15.46:0/3176670673 learned_addr learned my addr 172.21.15.46:0/3176670673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b53e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54149300 0x7f9b54162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b58a00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c004800 0x7f9b5414be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 -- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54149300 msgr2=0x7f9b54162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54149300 0x7f9b54162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 -- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c004800 msgr2=0x7f9b5414be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c004800 0x7f9b5414be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 -- 172.21.15.46:0/3176670673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b540736c0 con 0x7f9b541541c0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b59400640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b541635b0 tx=0x7f9b4c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.138+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c006d50 con 0x7f9b541541c0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.139+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c003970 con 0x7f9b541541c0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.139+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c003da0 con 0x7f9b541541c0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.139+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b540739a0 con 0x7f9b541541c0 2024-04-26T23:48:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.139+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5414b930 con 0x7f9b541541c0 2024-04-26T23:48:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.140+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b54154b80 con 0x7f9b541541c0 2024-04-26T23:48:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.141+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b4c01b020 con 0x7f9b541541c0 2024-04-26T23:48:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.142+0000 7f9b50c00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 0x7f9b3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:37.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.142+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f9b4c08b690 con 0x7f9b541541c0 2024-04-26T23:48:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.143+0000 7f9b58a00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 0x7f9b3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.144+0000 7f9b58a00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 0x7f9b3406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48000fe0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:37.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.146+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b4c059c20 con 0x7f9b541541c0 2024-04-26T23:48:37.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.424+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) -- 0x7f9b540693b0 con 0x7f9b541541c0 2024-04-26T23:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:37.998+0000 7f9b50c00640 1 -- 172.21.15.46:0/3176670673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v532) ==== 147+0+0 (secure 0 0 0) 0x7f9b4c059520 con 0x7f9b541541c0 2024-04-26T23:48:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 msgr2=0x7f9b3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 0x7f9b3406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48000fe0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 msgr2=0x7f9b54154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b541635b0 tx=0x7f9b4c00dc00 comp rx=0 tx=0).stop 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 shutdown_connections 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9b3406c920 0x7f9b3406ede0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541541c0 0x7f9b54154640 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54149300 0x7f9b54162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 --2- 172.21.15.46:0/3176670673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c004800 0x7f9b5414be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 >> 172.21.15.46:0/3176670673 conn(0x7f9b540b2820 msgr2=0x7f9b5415e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 shutdown_connections 2024-04-26T23:48:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.002+0000 7f9b59e00640 1 -- 172.21.15.46:0/3176670673 wait complete. 2024-04-26T23:48:38.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:48:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.246+0000 7f1446200640 1 Processor -- start 2024-04-26T23:48:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.246+0000 7f1446200640 1 -- start start 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1446200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 0x7f14400735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1446200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14400ce7d0 con 0x7f144015cec0 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1446200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14400ceab0 con 0x7f1440073b30 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1446200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14400ced90 con 0x7f14400cdd00 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1444e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f144015cec0 0x7f14400cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36430/0 (socket says 172.21.15.46:36430) 2024-04-26T23:48:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.248+0000 7f1444e00640 1 -- 172.21.15.46:0/1599878219 learned_addr learned my addr 172.21.15.46:0/1599878219 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143fe00640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 0x7f14400735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414574258 0 0) 0x7f14400ce7d0 con 0x7f144015cec0 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1428003520 con 0x7f144015cec0 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2855376713 0 0) 0x7f1428003520 con 0x7f144015cec0 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 msgr2=0x7f14400735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 0x7f14400735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 >> v1:172.21.15.142:6789/0 conn(0x7f1440073b30 legacy=0x7f14400b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14400cf070 con 0x7f144015cec0 2024-04-26T23:48:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1434001670 con 0x7f144015cec0 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 >> v1:172.21.15.46:6789/0 conn(0x7f144015cec0 legacy=0x7f14400cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14280039e0 0x7f1428003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 0x7f1428005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14400ced90 con 0x7f14280039e0 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f142800bb80 con 0x7f1428004800 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f142800bd80 con 0x7f14400cdd00 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.249+0000 7f1444e00640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14280039e0 0x7f1428003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.250+0000 7f1444e00640 1 -- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 msgr2=0x7f1428005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.250+0000 7f1444e00640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14400cdd00 0x7f1428005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.250+0000 7f1444e00640 1 -- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1428004800 msgr2=0x7f1428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:38.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.250+0000 7f1444e00640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.250+0000 7f1444e00640 1 -- 172.21.15.46:0/1599878219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1428003520 con 0x7f14280039e0 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.251+0000 7f1444e00640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14280039e0 0x7f1428003e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f143400c190 tx=0x7f1428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25deee18067256c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.251+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434007c60 con 0x7f14280039e0 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.251+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1434005ad0 con 0x7f14280039e0 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.251+0000 7f143f400640 1 -- 172.21.15.46:0/1599878219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434007070 con 0x7f14280039e0 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 -- 172.21.15.46:0/1599878219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14280039e0 msgr2=0x7f1428003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 --2- 172.21.15.46:0/1599878219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14280039e0 0x7f1428003e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f143400c190 tx=0x7f1428005830 comp rx=0 tx=0).stop 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1445800640 1 -- 172.21.15.46:0/1599878219 reap_dead start 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 -- 172.21.15.46:0/1599878219 shutdown_connections 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 -- 172.21.15.46:0/1599878219 >> 172.21.15.46:0/1599878219 conn(0x7f14400b27b0 msgr2=0x7f14400b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 -- 172.21.15.46:0/1599878219 shutdown_connections 2024-04-26T23:48:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.252+0000 7f1446200640 1 -- 172.21.15.46:0/1599878219 wait complete. 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.254+0000 7f1446200640 1 Processor -- start 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.254+0000 7f1446200640 1 -- start start 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14280039e0 0x7f1440072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 0x7f1440152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1440073f70 con 0x7f1440153db0 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f144015d2a0 con 0x7f14280039e0 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1446200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14400735f0 con 0x7f14401533a0 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40894/0 (socket says 172.21.15.46:40894) 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 -- 172.21.15.46:0/3498466233 learned_addr learned my addr 172.21.15.46:0/3498466233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1444e00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14280039e0 0x7f1440072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f143fe00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 0x7f1440152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 -- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 msgr2=0x7f1440152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 0x7f1440152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 -- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14280039e0 msgr2=0x7f1440072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.255+0000 7f1445800640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14280039e0 0x7f1440072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f1445800640 1 -- 172.21.15.46:0/3498466233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14401538f0 con 0x7f1440153db0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f143fe00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 0x7f1440152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f1445800640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f14400cfe50 tx=0x7f143800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438017070 con 0x7f1440153db0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1438007070 con 0x7f1440153db0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438007460 con 0x7f1440153db0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14400c6020 con 0x7f1440153db0 2024-04-26T23:48:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.256+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14400c65e0 con 0x7f1440153db0 2024-04-26T23:48:38.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.258+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14400c67e0 con 0x7f1440153db0 2024-04-26T23:48:38.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.259+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f143801a070 con 0x7f1440153db0 2024-04-26T23:48:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.259+0000 7f143cc00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 0x7f142006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.259+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f143808ac20 con 0x7f1440153db0 2024-04-26T23:48:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.260+0000 7f1444e00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 0x7f142006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:38.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.261+0000 7f1444e00640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 0x7f142006ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f143400c160 tx=0x7f1434009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.264+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14400c67e0 con 0x7f1440153db0 2024-04-26T23:48:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.468+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14400693b0 con 0x7f1440153db0 2024-04-26T23:48:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.473+0000 7f143cc00640 1 -- 172.21.15.46:0/3498466233 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f14380591c0 con 0x7f1440153db0 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 msgr2=0x7f142006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 0x7f142006ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f143400c160 tx=0x7f1434009030 comp rx=0 tx=0).stop 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 msgr2=0x7f14400c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f14400cfe50 tx=0x7f143800c980 comp rx=0 tx=0).stop 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 shutdown_connections 2024-04-26T23:48:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f142006c920 0x7f142006ede0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440153db0 0x7f14400c91b0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14401533a0 0x7f1440152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 --2- 172.21.15.46:0/3498466233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14280039e0 0x7f1440072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.476+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 >> 172.21.15.46:0/3498466233 conn(0x7f14400b27b0 msgr2=0x7f14400b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.477+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 shutdown_connections 2024-04-26T23:48:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:38.477+0000 7f1446200640 1 -- 172.21.15.46:0/3498466233 wait complete. 2024-04-26T23:48:38.500 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:48:38.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:48:38.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:48:38.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:48:38.733 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:38.734 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:38.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:48:38.796 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:48:38.796 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:38.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:48:38.859 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:38.859 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:38.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:48:38.920 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:48:38.920 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:38.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:48:38.977 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:48:38.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.194+0000 7efcc8c00640 1 Processor -- start 2024-04-26T23:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.195+0000 7efcc8c00640 1 -- start start 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.196+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efcc40d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.196+0000 7efcc8c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efcc415ccd0 0x7efcc40732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.196+0000 7efcc8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc40cf950 con 0x7efcc4072e60 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.196+0000 7efcc8c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efcc40d0c50 con 0x7efcc415ccd0 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.196+0000 7efcc8c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efcc415d490 con 0x7efcc4145770 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efcc4072e60 0x7efcc415c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36438/0 (socket says 172.21.15.46:36438) 2024-04-26T23:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc1400640 1 -- 172.21.15.46:0/4160657697 learned_addr learned my addr 172.21.15.46:0/4160657697 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810481313 0 0) 0x7efcc40cf950 con 0x7efcc4072e60 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efca8005730 con 0x7efcc4072e60 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc2800640 1 --2- 172.21.15.46:0/4160657697 >> v2:172.21.15.142:3300/0 conn(0x7efcc415ccd0 0x7efcc40732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc1e00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efcc40d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.197+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 596626990 0 0) 0x7efca8005730 con 0x7efcc4072e60 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 msgr2=0x7efcc40d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efcc40d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 >> v2:172.21.15.142:3300/0 conn(0x7efcc415ccd0 msgr2=0x7efcc40732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 --2- 172.21.15.46:0/4160657697 >> v2:172.21.15.142:3300/0 conn(0x7efcc415ccd0 0x7efcc40732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc40cfc30 con 0x7efcc4072e60 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7efcb0001670 con 0x7efcc4072e60 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc2800640 1 --2- 172.21.15.46:0/4160657697 >> v2:172.21.15.142:3300/0 conn(0x7efcc415ccd0 0x7efcc40732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 >> v1:172.21.15.46:6789/0 conn(0x7efcc4072e60 legacy=0x7efcc415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efca8005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca8006190 0x7efca8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efca8006f50 0x7efca80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efcc40d0c50 con 0x7efca8006190 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efcc415d490 con 0x7efca8006f50 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efca800dc80 con 0x7efcc4145770 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc1400640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca8006190 0x7efca8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.198+0000 7efcc1e00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efca8005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc1400640 1 -- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 msgr2=0x7efca8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc1400640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc4145770 0x7efca8005bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc1400640 1 -- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efca8006f50 msgr2=0x7efca80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc2800640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efca8006f50 0x7efca80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc1400640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efca8006f50 0x7efca80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc1400640 1 -- 172.21.15.46:0/4160657697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca8005730 con 0x7efca8006190 2024-04-26T23:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.199+0000 7efcc2800640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efca8006f50 0x7efca80073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc1400640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca8006190 0x7efca8006a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c160 tx=0x7efca8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4db5f17093ed51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0007c80 con 0x7efca8006190 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb0005ad0 con 0x7efca8006190 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc0a00640 1 -- 172.21.15.46:0/4160657697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0007070 con 0x7efca8006190 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc8c00640 1 -- 172.21.15.46:0/4160657697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca8006190 msgr2=0x7efca8006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.200+0000 7efcc8c00640 1 --2- 172.21.15.46:0/4160657697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca8006190 0x7efca8006a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c160 tx=0x7efca8007a40 comp rx=0 tx=0).stop 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.201+0000 7efcc2800640 1 -- 172.21.15.46:0/4160657697 reap_dead start 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.201+0000 7efcc8c00640 1 -- 172.21.15.46:0/4160657697 shutdown_connections 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.201+0000 7efcc8c00640 1 -- 172.21.15.46:0/4160657697 >> 172.21.15.46:0/4160657697 conn(0x7efcc406c530 msgr2=0x7efcc406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.201+0000 7efcc8c00640 1 -- 172.21.15.46:0/4160657697 shutdown_connections 2024-04-26T23:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.201+0000 7efcc8c00640 1 -- 172.21.15.46:0/4160657697 wait complete. 2024-04-26T23:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.203+0000 7efcc8c00640 1 Processor -- start 2024-04-26T23:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.203+0000 7efcc8c00640 1 -- start start 2024-04-26T23:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 0x7efcc415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc41e67b0 0x7efcc41e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efcc40d01f0 con 0x7efcc41f4f40 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4073800 con 0x7efcc41e67b0 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc8c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efcc40cf5b0 con 0x7efca8006190 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40902/0 (socket says 172.21.15.46:40902) 2024-04-26T23:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 -- 172.21.15.46:0/3976634639 learned_addr learned my addr 172.21.15.46:0/3976634639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc1e00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 0x7efcc415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 -- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 msgr2=0x7efcc415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 0x7efcc415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 -- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc41e67b0 msgr2=0x7efcc41e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc1400640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc41e67b0 0x7efcc41e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc41e67b0 0x7efcc41e6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc2800640 1 -- 172.21.15.46:0/3976634639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc41f5cc0 con 0x7efcc41f4f40 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.204+0000 7efcc1e00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 0x7efcc415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcc2800640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7efcc406e6c0 tx=0x7efcbc004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc019070 con 0x7efcc41f4f40 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc007ce0 con 0x7efcc41f4f40 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc014640 con 0x7efcc41f4f40 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc41f5fa0 con 0x7efcc41f4f40 2024-04-26T23:48:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.205+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc4142120 con 0x7efcc41f4f40 2024-04-26T23:48:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.206+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc4069530 con 0x7efcc41f4f40 2024-04-26T23:48:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.207+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcbc01c070 con 0x7efcc41f4f40 2024-04-26T23:48:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.208+0000 7efcba000640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.208+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7efcbc08b640 con 0x7efcc41f4f40 2024-04-26T23:48:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.209+0000 7efcc1e00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.210+0000 7efcc1e00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 0x7efca406ede0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009910 tx=0x7efcb4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.214+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efcbc059b40 con 0x7efcc41f4f40 2024-04-26T23:48:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.462+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcc41422f0 con 0x7efcc41f4f40 2024-04-26T23:48:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.463+0000 7efcba000640 1 -- 172.21.15.46:0/3976634639 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+3866 (secure 0 0 0) 0x7efcbc059440 con 0x7efcc41f4f40 2024-04-26T23:48:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:39.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2323807219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2323807219},{"type":"v1","addr":"172.21.15.46:6835","nonce":2323807219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/779001185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":779001185},{"type":"v1","addr":"172.21.15.142:6835","nonce":779001185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/875359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":875359483},{"type":"v1","addr":"172.21.15.142:6837","nonce":875359483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/304879465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":304879465},{"type":"v1","addr":"172.21.15.46:6833","nonce":304879465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[]} 2024-04-26T23:48:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 93 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 msgr2=0x7efca406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 0x7efca406ede0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009910 tx=0x7efcb4008040 comp rx=0 tx=0).stop 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 msgr2=0x7efcc41f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7efcc406e6c0 tx=0x7efcbc004840 comp rx=0 tx=0).stop 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 shutdown_connections 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc41f4f40 0x7efcc41f53c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc41e67b0 0x7efcc41e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 --2- 172.21.15.46:0/3976634639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efca8006190 0x7efcc415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 >> 172.21.15.46:0/3976634639 conn(0x7efcc406c530 msgr2=0x7efcc40b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 shutdown_connections 2024-04-26T23:48:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.467+0000 7efcc8c00640 1 -- 172.21.15.46:0/3976634639 wait complete. 2024-04-26T23:48:39.492 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:48:39.493 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:48:39.493 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:48:39.493 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:48:39.493 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:48:39.493 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:48:39.493 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:48:39.493 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:48:39.494 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:48:39.494 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:48:39.494 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:48:39.494 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:48:39.494 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:48:39.494 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:48:39.495 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:48:39.495 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:48:39.516 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:48:39.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:48:39.517 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:48:39.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:48:39.518 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:48:39.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:48:39.519 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:48:39.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:48:39.561 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:48:39.561 DEBUG:teuthology.parallel:result is None 2024-04-26T23:48:39.565 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:48:39.565 DEBUG:teuthology.parallel:result is None 2024-04-26T23:48:39.566 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:48:39.567 DEBUG:teuthology.parallel:result is None 2024-04-26T23:48:39.572 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:48:39.572 DEBUG:teuthology.parallel:result is None 2024-04-26T23:48:39.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:48:39.713 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:48:39.714 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:48:39.722 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:48:39.722 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:48:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.747+0000 7f653ee00640 1 Processor -- start 2024-04-26T23:48:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.747+0000 7f653ee00640 1 -- start start 2024-04-26T23:48:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f653ee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6538142280 0x7f653806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f653ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65380b43f0 con 0x7f65380b3e80 2024-04-26T23:48:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f653ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65380b46d0 con 0x7f653806ea50 2024-04-26T23:48:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f653ee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f653806eea0 con 0x7f6538142280 2024-04-26T23:48:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f6537e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6538142280 0x7f653806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f652fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f65380b3e80 0x7f6538144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36464/0 (socket says 172.21.15.46:36464) 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.749+0000 7f652fe00640 1 -- 172.21.15.46:0/1239692689 learned_addr learned my addr 172.21.15.46:0/1239692689 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.750+0000 7f6537e00640 1 -- 172.21.15.46:0/1239692689 >> v1:172.21.15.142:6789/0 conn(0x7f653806ea50 legacy=0x7f6538144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.750+0000 7f6537e00640 1 -- 172.21.15.46:0/1239692689 >> v1:172.21.15.46:6789/0 conn(0x7f65380b3e80 legacy=0x7f6538144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.750+0000 7f6537e00640 1 -- 172.21.15.46:0/1239692689 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380b49b0 con 0x7f6538142280 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.750+0000 7f6537400640 1 -- 172.21.15.46:0/1239692689 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985741304 0 0) 0x7f65380b43f0 con 0x7f65380b3e80 2024-04-26T23:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.751+0000 7f6537e00640 1 --2- 172.21.15.46:0/1239692689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6538142280 0x7f653806e510 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f65280097a0 tx=0x7f652802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df32cac18289655 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.751+0000 7f6537400640 1 -- 172.21.15.46:0/1239692689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037070 con 0x7f6538142280 2024-04-26T23:48:39.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.751+0000 7f6537400640 1 -- 172.21.15.46:0/1239692689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528032440 con 0x7f6538142280 2024-04-26T23:48:39.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.751+0000 7f6537400640 1 -- 172.21.15.46:0/1239692689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528032810 con 0x7f6538142280 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 -- 172.21.15.46:0/1239692689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6538142280 msgr2=0x7f653806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 --2- 172.21.15.46:0/1239692689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6538142280 0x7f653806e510 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f65280097a0 tx=0x7f652802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 -- 172.21.15.46:0/1239692689 shutdown_connections 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 --2- 172.21.15.46:0/1239692689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6538142280 0x7f653806e510 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 -- 172.21.15.46:0/1239692689 >> 172.21.15.46:0/1239692689 conn(0x7f653806c530 msgr2=0x7f653806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 -- 172.21.15.46:0/1239692689 shutdown_connections 2024-04-26T23:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.752+0000 7f653ee00640 1 -- 172.21.15.46:0/1239692689 wait complete. 2024-04-26T23:48:39.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.754+0000 7f653ee00640 1 Processor -- start 2024-04-26T23:48:39.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.754+0000 7f653ee00640 1 -- start start 2024-04-26T23:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 0x7f653815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 0x7f6538158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6538145660 con 0x7f65380b3e80 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65380b4f70 con 0x7f653806ea50 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65381428a0 con 0x7f6538142280 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f652fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f652fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40952/0 (socket says 172.21.15.46:40952) 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f652fe00640 1 -- 172.21.15.46:0/297681835 learned_addr learned my addr 172.21.15.46:0/297681835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f6537e00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 0x7f653815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f653ce00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 0x7f6538158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.755+0000 7f652fe00640 1 -- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 msgr2=0x7f6538158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f652fe00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 0x7f6538158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f652fe00640 1 -- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 msgr2=0x7f653815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f652fe00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 0x7f653815c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f652fe00640 1 -- 172.21.15.46:0/297681835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65381383c0 con 0x7f65380b3e80 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f6537e00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 0x7f653815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f653ce00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 0x7f6538158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f652fe00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f652000c500 tx=0x7f652000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520018070 con 0x7f65380b3e80 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6520007580 con 0x7f65380b3e80 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520004a20 con 0x7f65380b3e80 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380cdf80 con 0x7f65380b3e80 2024-04-26T23:48:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.756+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65380ce540 con 0x7f65380b3e80 2024-04-26T23:48:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.758+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6538069530 con 0x7f65380b3e80 2024-04-26T23:48:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.759+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f652001b070 con 0x7f65380b3e80 2024-04-26T23:48:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.759+0000 7f6534c00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.760+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5335+0+0 (secure 0 0 0) 0x7f652008ae10 con 0x7f65380b3e80 2024-04-26T23:48:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.760+0000 7f6537e00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.761+0000 7f6537e00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f652802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.764+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f652004fc00 con 0x7f65380b3e80 2024-04-26T23:48:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:39.986+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65380b2320 con 0x7f65380b3e80 2024-04-26T23:48:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.032+0000 7f6534c00640 1 -- 172.21.15.46:0/297681835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) ==== 97+0+0 (secure 0 0 0) 0x7f652008e030 con 0x7f65380b3e80 2024-04-26T23:48:40.038 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:48:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 msgr2=0x7f651406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f652802e000 comp rx=0 tx=0).stop 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 msgr2=0x7f65380cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f652000c500 tx=0x7f652000c9c0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 shutdown_connections 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6538142280 0x7f6538158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65380b3e80 0x7f65380cd910 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 --2- 172.21.15.46:0/297681835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653806ea50 0x7f653815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 >> 172.21.15.46:0/297681835 conn(0x7f653806c530 msgr2=0x7f6538144840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 shutdown_connections 2024-04-26T23:48:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.036+0000 7f653ee00640 1 -- 172.21.15.46:0/297681835 wait complete. 2024-04-26T23:48:40.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:48:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.263+0000 7f2a84000640 1 Processor -- start 2024-04-26T23:48:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.264+0000 7f2a84000640 1 -- start start 2024-04-26T23:48:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.265+0000 7f2a84000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a7c0d0080 0x7f2a7c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.265+0000 7f2a84000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0d05c0 con 0x7f2a7c0737a0 2024-04-26T23:48:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a84000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0afe80 con 0x7f2a7c15be90 2024-04-26T23:48:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a84000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7c0afc50 con 0x7f2a7c0d0080 2024-04-26T23:48:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a80c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a7c0737a0 0x7f2a7c0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36474/0 (socket says 172.21.15.46:36474) 2024-04-26T23:48:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a80c00640 1 -- 172.21.15.46:0/2295857582 learned_addr learned my addr 172.21.15.46:0/2295857582 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783966166 0 0) 0x7f2a7c0d05c0 con 0x7f2a7c0737a0 2024-04-26T23:48:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.266+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003520 con 0x7f2a7c0737a0 2024-04-26T23:48:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2637920582 0 0) 0x7f2a64003520 con 0x7f2a7c0737a0 2024-04-26T23:48:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 >> v2:172.21.15.142:3300/0 conn(0x7f2a7c0d0080 msgr2=0x7f2a7c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a82000640 1 --2- 172.21.15.46:0/2295857582 >> v2:172.21.15.142:3300/0 conn(0x7f2a7c0d0080 0x7f2a7c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 --2- 172.21.15.46:0/2295857582 >> v2:172.21.15.142:3300/0 conn(0x7f2a7c0d0080 0x7f2a7c0cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 >> v1:172.21.15.142:6789/0 conn(0x7f2a7c15be90 legacy=0x7f2a7c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c0b0160 con 0x7f2a7c0737a0 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2a68001670 con 0x7f2a7c0737a0 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 >> v1:172.21.15.46:6789/0 conn(0x7f2a7c0737a0 legacy=0x7f2a7c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a640039b0 0x7f2a64003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 0x7f2a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6400c1b0 con 0x7f2a640052e0 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6400c360 con 0x7f2a640039b0 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.267+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6400c560 con 0x7f2a64004800 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 0x7f2a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a81600640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a640039b0 0x7f2a64003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a80c00640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 -- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a64004800 msgr2=0x7f2a64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 -- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a640039b0 msgr2=0x7f2a64003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a640039b0 0x7f2a64003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.268+0000 7f2a82000640 1 -- 172.21.15.46:0/2295857582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a64003520 con 0x7f2a640052e0 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.269+0000 7f2a82000640 1 -- 172.21.15.46:0/2295857582 reap_dead start 2024-04-26T23:48:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.270+0000 7f2a82000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 0x7f2a6400b9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0d04f0 tx=0x7f2a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b9c3158c86bba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.270+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74038070 con 0x7f2a640052e0 2024-04-26T23:48:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.270+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7402b7d0 con 0x7f2a640052e0 2024-04-26T23:48:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.270+0000 7f2a7be00640 1 -- 172.21.15.46:0/2295857582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74007b30 con 0x7f2a640052e0 2024-04-26T23:48:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 -- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 msgr2=0x7f2a6400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 0x7f2a6400b9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0d04f0 tx=0x7f2a64005830 comp rx=0 tx=0).stop 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 -- 172.21.15.46:0/2295857582 shutdown_connections 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 --2- 172.21.15.46:0/2295857582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a640052e0 0x7f2a6400b9c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 -- 172.21.15.46:0/2295857582 >> 172.21.15.46:0/2295857582 conn(0x7f2a7c06da90 msgr2=0x7f2a7c0d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 -- 172.21.15.46:0/2295857582 shutdown_connections 2024-04-26T23:48:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.271+0000 7f2a84000640 1 -- 172.21.15.46:0/2295857582 wait complete. 2024-04-26T23:48:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.273+0000 7f2a84000640 1 Processor -- start 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.273+0000 7f2a84000640 1 -- start start 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a640052e0 0x7f2a7c0b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 0x7f2a7c1f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0d08a0 con 0x7f2a7c15ef10 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c15c480 con 0x7f2a7c1f83b0 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a84000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b04d0 con 0x7f2a640052e0 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a80c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a80c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40982/0 (socket says 172.21.15.46:40982) 2024-04-26T23:48:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.274+0000 7f2a80c00640 1 -- 172.21.15.46:0/869659367 learned_addr learned my addr 172.21.15.46:0/869659367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a81600640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a640052e0 0x7f2a7c0b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a82000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 0x7f2a7c1f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 -- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a640052e0 msgr2=0x7f2a7c0b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a640052e0 0x7f2a7c0b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 -- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 msgr2=0x7f2a7c1f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 0x7f2a7c1f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 -- 172.21.15.46:0/869659367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c14f310 con 0x7f2a7c15ef10 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a82000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 0x7f2a7c1f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a80c00640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d8e0 tx=0x7f2a6800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68019070 con 0x7f2a7c15ef10 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a680074e0 con 0x7f2a7c15ef10 2024-04-26T23:48:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680049e0 con 0x7f2a7c15ef10 2024-04-26T23:48:40.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c14f5f0 con 0x7f2a7c15ef10 2024-04-26T23:48:40.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.275+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c06cb00 con 0x7f2a7c15ef10 2024-04-26T23:48:40.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.277+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c069530 con 0x7f2a7c15ef10 2024-04-26T23:48:40.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.277+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a68004b80 con 0x7f2a7c15ef10 2024-04-26T23:48:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.278+0000 7f2a79600640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 0x7f2a5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.278+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f2a6808b390 con 0x7f2a7c15ef10 2024-04-26T23:48:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.279+0000 7f2a81600640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 0x7f2a5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.280+0000 7f2a81600640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 0x7f2a5c06ede0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0c5900 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.283+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a68001d00 con 0x7f2a7c15ef10 2024-04-26T23:48:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.560+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2a7c0d2c50 con 0x7f2a7c15ef10 2024-04-26T23:48:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.561+0000 7f2a79600640 1 -- 172.21.15.46:0/869659367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f2a68059970 con 0x7f2a7c15ef10 2024-04-26T23:48:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:48:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.570 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:48:40.573 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.575 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:48:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.578 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:48:40.579 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:48:40.580 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 msgr2=0x7f2a5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 0x7f2a5c06ede0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0c5900 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 msgr2=0x7f2a7c06c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d8e0 tx=0x7f2a6800dda0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 shutdown_connections 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a5c06c920 0x7f2a5c06ede0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c1f83b0 0x7f2a7c1f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.563+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7c15ef10 0x7f2a7c06c5c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.564+0000 7f2a84000640 1 --2- 172.21.15.46:0/869659367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a640052e0 0x7f2a7c0b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.564+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 >> 172.21.15.46:0/869659367 conn(0x7f2a7c06da90 msgr2=0x7f2a7c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:40.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.564+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 shutdown_connections 2024-04-26T23:48:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.564+0000 7f2a84000640 1 -- 172.21.15.46:0/869659367 wait complete. 2024-04-26T23:48:40.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.796+0000 7f4a6ec00640 1 Processor -- start 2024-04-26T23:48:40.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.796+0000 7f4a6ec00640 1 -- start start 2024-04-26T23:48:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a68142280 0x7f4a6806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 0x7f4a6806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a68144840 0x7f4a68144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4a680b43f0 con 0x7f4a68144840 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4a680b46d0 con 0x7f4a680b3e80 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a680b49b0 con 0x7f4a68142280 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a68144840 0x7f4a68144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a68144840 0x7f4a68144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40992/0 (socket says 172.21.15.46:40992) 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a6cc00640 1 -- 172.21.15.46:0/461146714 learned_addr learned my addr 172.21.15.46:0/461146714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.798+0000 7f4a67e00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a68142280 0x7f4a6806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a67400640 1 --2- 172.21.15.46:0/461146714 >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 0x7f4a6806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a6cc00640 1 -- 172.21.15.46:0/461146714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a68142280 msgr2=0x7f4a6806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a68142280 0x7f4a6806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a6cc00640 1 -- 172.21.15.46:0/461146714 >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 msgr2=0x7f4a6806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/461146714 >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 0x7f4a6806ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a6cc00640 1 -- 172.21.15.46:0/461146714 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a680b4c90 con 0x7f4a68144840 2024-04-26T23:48:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.799+0000 7f4a67400640 1 --2- 172.21.15.46:0/461146714 >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 0x7f4a6806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.800+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a68144cb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f4a680700f0 tx=0x7f4a6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f20264972e415eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.800+0000 7f4a66a00640 1 -- 172.21.15.46:0/461146714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60036070 con 0x7f4a68144840 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.800+0000 7f4a66a00640 1 -- 172.21.15.46:0/461146714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60031600 con 0x7f4a68144840 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.800+0000 7f4a66a00640 1 -- 172.21.15.46:0/461146714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600319d0 con 0x7f4a68144840 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 -- 172.21.15.46:0/461146714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 msgr2=0x7f4a68144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a68144cb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f4a680700f0 tx=0x7f4a6002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 -- 172.21.15.46:0/461146714 shutdown_connections 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a68144cb0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/461146714 >> v2:172.21.15.142:3300/0 conn(0x7f4a680b3e80 0x7f4a6806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/461146714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a68142280 0x7f4a6806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 -- 172.21.15.46:0/461146714 >> 172.21.15.46:0/461146714 conn(0x7f4a6806c530 msgr2=0x7f4a6806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 -- 172.21.15.46:0/461146714 shutdown_connections 2024-04-26T23:48:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.801+0000 7f4a6ec00640 1 -- 172.21.15.46:0/461146714 wait complete. 2024-04-26T23:48:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.803+0000 7f4a6ec00640 1 Processor -- start 2024-04-26T23:48:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.803+0000 7f4a6ec00640 1 -- start start 2024-04-26T23:48:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a680b3e80 0x7f4a680b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68142280 0x7f4a6813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68142a20 con 0x7f4a68144840 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681428a0 con 0x7f4a68142280 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a681451f0 con 0x7f4a680b3e80 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41000/0 (socket says 172.21.15.46:41000) 2024-04-26T23:48:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6cc00640 1 -- 172.21.15.46:0/3935642451 learned_addr learned my addr 172.21.15.46:0/3935642451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a67400640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68142280 0x7f4a6813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a67e00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a680b3e80 0x7f4a680b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6cc00640 1 -- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a680b3e80 msgr2=0x7f4a680b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.804+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a680b3e80 0x7f4a680b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6cc00640 1 -- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68142280 msgr2=0x7f4a6813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68142280 0x7f4a6813d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6cc00640 1 -- 172.21.15.46:0/3935642451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68160150 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6cc00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009f40 tx=0x7f4a60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60041030 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60031400 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60036040 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6815ef10 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.805+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6815f4d0 con 0x7f4a68144840 2024-04-26T23:48:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.807+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a680b60b0 con 0x7f4a68144840 2024-04-26T23:48:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.808+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a60044070 con 0x7f4a68144840 2024-04-26T23:48:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.808+0000 7f4a5fe00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 0x7f4a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.809+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f4a600ac2e0 con 0x7f4a68144840 2024-04-26T23:48:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.809+0000 7f4a67e00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 0x7f4a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.811+0000 7f4a67e00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 0x7f4a4806ede0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006fd0 tx=0x7f4a58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:40.813+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a680b60b0 con 0x7f4a68144840 2024-04-26T23:48:41.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.017+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a681454f0 con 0x7f4a68144840 2024-04-26T23:48:41.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.019+0000 7f4a5fe00640 1 -- 172.21.15.46:0/3935642451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+13138 (secure 0 0 0) 0x7f4a681454f0 con 0x7f4a68144840 2024-04-26T23:48:41.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:41.024 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:40.046799+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 msgr2=0x7f4a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 0x7f4a4806ede0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006fd0 tx=0x7f4a58008040 comp rx=0 tx=0).stop 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 msgr2=0x7f4a680ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009f40 tx=0x7f4a60004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 shutdown_connections 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a4806c920 0x7f4a4806ede0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a68144840 0x7f4a680ae8e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68142280 0x7f4a6813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 --2- 172.21.15.46:0/3935642451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a680b3e80 0x7f4a680b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 >> 172.21.15.46:0/3935642451 conn(0x7f4a6806c530 msgr2=0x7f4a680b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:41.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 shutdown_connections 2024-04-26T23:48:41.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.022+0000 7f4a6ec00640 1 -- 172.21.15.46:0/3935642451 wait complete. 2024-04-26T23:48:41.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:48:41.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:48:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.254+0000 7f35bfe00640 1 Processor -- start 2024-04-26T23:48:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.255+0000 7f35bfe00640 1 -- start start 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35b81597e0 0x7f35b80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80b5730 0x7f35b80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35b8071f10 0x7f35b8072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f35b80b60d0 con 0x7f35b8071f10 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f35b80728c0 con 0x7f35b81597e0 2024-04-26T23:48:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.256+0000 7f35bfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f35b8073120 con 0x7f35b80b5730 2024-04-26T23:48:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35b8071f10 0x7f35b8072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35b8071f10 0x7f35b8072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41002/0 (socket says 172.21.15.46:41002) 2024-04-26T23:48:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 -- 172.21.15.46:0/3773581024 learned_addr learned my addr 172.21.15.46:0/3773581024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 -- 172.21.15.46:0/3773581024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80b5730 msgr2=0x7f35b80b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 --2- 172.21.15.46:0/3773581024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80b5730 0x7f35b80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 -- 172.21.15.46:0/3773581024 >> v2:172.21.15.142:3300/0 conn(0x7f35b81597e0 msgr2=0x7f35b80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 --2- 172.21.15.46:0/3773581024 >> v2:172.21.15.142:3300/0 conn(0x7f35b81597e0 0x7f35b80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.257+0000 7f35bde00640 1 -- 172.21.15.46:0/3773581024 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b8159bc0 con 0x7f35b8071f10 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.258+0000 7f35bde00640 1 --2- 172.21.15.46:0/3773581024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b8071f10 0x7f35b8072380 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f35b80b0420 tx=0x7f35b002a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ccc54b96351d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35b7e00640 1 -- 172.21.15.46:0/3773581024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0036070 con 0x7f35b8071f10 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35b7e00640 1 -- 172.21.15.46:0/3773581024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b0031600 con 0x7f35b8071f10 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35b7e00640 1 -- 172.21.15.46:0/3773581024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00319d0 con 0x7f35b8071f10 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 -- 172.21.15.46:0/3773581024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b8071f10 msgr2=0x7f35b8072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 --2- 172.21.15.46:0/3773581024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b8071f10 0x7f35b8072380 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f35b80b0420 tx=0x7f35b002a720 comp rx=0 tx=0).stop 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 -- 172.21.15.46:0/3773581024 shutdown_connections 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 --2- 172.21.15.46:0/3773581024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b8071f10 0x7f35b8072380 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 --2- 172.21.15.46:0/3773581024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80b5730 0x7f35b80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 --2- 172.21.15.46:0/3773581024 >> v2:172.21.15.142:3300/0 conn(0x7f35b81597e0 0x7f35b80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 -- 172.21.15.46:0/3773581024 >> 172.21.15.46:0/3773581024 conn(0x7f35b806c980 msgr2=0x7f35b80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 -- 172.21.15.46:0/3773581024 shutdown_connections 2024-04-26T23:48:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.259+0000 7f35bfe00640 1 -- 172.21.15.46:0/3773581024 wait complete. 2024-04-26T23:48:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.261+0000 7f35bfe00640 1 Processor -- start 2024-04-26T23:48:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.261+0000 7f35bfe00640 1 -- start start 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b8071f10 0x7f35b8144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80b5730 0x7f35b81471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35b80b40d0 con 0x7f35b81597e0 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8072e40 con 0x7f35b80b5730 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.262+0000 7f35bfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f35b80b43b0 con 0x7f35b8071f10 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41004/0 (socket says 172.21.15.46:41004) 2024-04-26T23:48:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 -- 172.21.15.46:0/433292302 learned_addr learned my addr 172.21.15.46:0/433292302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 -- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b8071f10 msgr2=0x7f35b8144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b8071f10 0x7f35b8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 -- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80b5730 msgr2=0x7f35b81471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80b5730 0x7f35b81471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 -- 172.21.15.46:0/433292302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b8147700 con 0x7f35b81597e0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.263+0000 7f35bde00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009f40 tx=0x7f35b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.264+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0041020 con 0x7f35b81597e0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.264+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b00077a0 con 0x7f35b81597e0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.264+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b8154ac0 con 0x7f35b81597e0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.264+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0036040 con 0x7f35b81597e0 2024-04-26T23:48:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.264+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b8144ea0 con 0x7f35b81597e0 2024-04-26T23:48:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.265+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b8069530 con 0x7f35b81597e0 2024-04-26T23:48:41.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.266+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35b0044020 con 0x7f35b81597e0 2024-04-26T23:48:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.267+0000 7f35b5600640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 0x7f359c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.268+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f35b00ac750 con 0x7f35b81597e0 2024-04-26T23:48:41.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.268+0000 7f35bd400640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 0x7f359c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.269+0000 7f35bd400640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 0x7f359c06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009670 tx=0x7f35ac006bf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.272+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35b004b030 con 0x7f35b81597e0 2024-04-26T23:48:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.476+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35b80b2a30 con 0x7f35b81597e0 2024-04-26T23:48:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.477+0000 7f35b5600640 1 -- 172.21.15.46:0/433292302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f35b007acb0 con 0x7f35b81597e0 2024-04-26T23:48:41.482 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 msgr2=0x7f359c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 0x7f359c06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009670 tx=0x7f35ac006bf0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 msgr2=0x7f35b8147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009f40 tx=0x7f35b0004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 shutdown_connections 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f359c06c8f0 0x7f359c06edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b81597e0 0x7f35b8147c40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80b5730 0x7f35b81471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 --2- 172.21.15.46:0/433292302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b8071f10 0x7f35b8144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 >> 172.21.15.46:0/433292302 conn(0x7f35b806c980 msgr2=0x7f35b80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:41.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 shutdown_connections 2024-04-26T23:48:41.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.480+0000 7f35bfe00640 1 -- 172.21.15.46:0/433292302 wait complete. 2024-04-26T23:48:41.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:48:41.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:41.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.711+0000 7f61b9c00640 1 Processor -- start 2024-04-26T23:48:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.711+0000 7f61b9c00640 1 -- start start 2024-04-26T23:48:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61b4162980 0x7f61b40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b9c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61b40d2200 0x7f61b415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b9c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b415fd30 con 0x7f61b40d31f0 2024-04-26T23:48:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f61b4162d60 con 0x7f61b4162980 2024-04-26T23:48:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b9c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f61b40d1050 con 0x7f61b40d2200 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61b4162980 0x7f61b40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b2000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f61b40d31f0 0x7f61b40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36482/0 (socket says 172.21.15.46:36482) 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.713+0000 7f61b2000640 1 -- 172.21.15.46:0/2907643406 learned_addr learned my addr 172.21.15.46:0/2907643406 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b3400640 1 --2- 172.21.15.46:0/2907643406 >> v2:172.21.15.142:3300/0 conn(0x7f61b40d2200 0x7f61b415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b2a00640 1 -- 172.21.15.46:0/2907643406 >> v2:172.21.15.142:3300/0 conn(0x7f61b40d2200 msgr2=0x7f61b415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b2a00640 1 --2- 172.21.15.46:0/2907643406 >> v2:172.21.15.142:3300/0 conn(0x7f61b40d2200 0x7f61b415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b2a00640 1 -- 172.21.15.46:0/2907643406 >> v1:172.21.15.46:6789/0 conn(0x7f61b40d31f0 legacy=0x7f61b40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b2a00640 1 -- 172.21.15.46:0/2907643406 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b4160010 con 0x7f61b4162980 2024-04-26T23:48:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.714+0000 7f61b1600640 1 -- 172.21.15.46:0/2907643406 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195598256 0 0) 0x7f61b415fd30 con 0x7f61b40d31f0 2024-04-26T23:48:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.715+0000 7f61b2a00640 1 --2- 172.21.15.46:0/2907643406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b40d2cb0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f61b408ecd0 tx=0x7f61a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579484b950daaa8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b1600640 1 -- 172.21.15.46:0/2907643406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4037070 con 0x7f61b4162980 2024-04-26T23:48:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b1600640 1 -- 172.21.15.46:0/2907643406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a4032440 con 0x7f61b4162980 2024-04-26T23:48:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b1600640 1 -- 172.21.15.46:0/2907643406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4032810 con 0x7f61b4162980 2024-04-26T23:48:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 -- 172.21.15.46:0/2907643406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 msgr2=0x7f61b40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 --2- 172.21.15.46:0/2907643406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b40d2cb0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f61b408ecd0 tx=0x7f61a402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 -- 172.21.15.46:0/2907643406 shutdown_connections 2024-04-26T23:48:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 --2- 172.21.15.46:0/2907643406 >> v2:172.21.15.142:3300/0 conn(0x7f61b40d2200 0x7f61b415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 --2- 172.21.15.46:0/2907643406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b40d2cb0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 -- 172.21.15.46:0/2907643406 >> 172.21.15.46:0/2907643406 conn(0x7f61b409e420 msgr2=0x7f61b40d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:41.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 -- 172.21.15.46:0/2907643406 shutdown_connections 2024-04-26T23:48:41.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.716+0000 7f61b9c00640 1 -- 172.21.15.46:0/2907643406 wait complete. 2024-04-26T23:48:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.718+0000 7f61b9c00640 1 Processor -- start 2024-04-26T23:48:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.718+0000 7f61b9c00640 1 -- start start 2024-04-26T23:48:41.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 0x7f61b40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61b41608b0 con 0x7f61b4162980 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61b41605d0 con 0x7f61b40d31f0 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.719+0000 7f61b9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f61b4160d50 con 0x7f61b40d2200 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b2a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 0x7f61b40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:41616/0 (socket says 172.21.15.46:41616) 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41016/0 (socket says 172.21.15.46:41016) 2024-04-26T23:48:41.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 -- 172.21.15.46:0/86799784 learned_addr learned my addr 172.21.15.46:0/86799784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:41.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 -- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 msgr2=0x7f61b40c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 0x7f61b40c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 -- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 msgr2=0x7f61b415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.720+0000 7f61b3400640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b3400640 1 -- 172.21.15.46:0/86799784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b40cadc0 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b2a00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 0x7f61b40c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b2000640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b3400640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f61ac007d40 tx=0x7f61ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac014420 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ac019070 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac014820 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b40cb0a0 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.721+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b40cb660 con 0x7f61b4162980 2024-04-26T23:48:41.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.723+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4069530 con 0x7f61b4162980 2024-04-26T23:48:41.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.723+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61ac0149c0 con 0x7f61b4162980 2024-04-26T23:48:41.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.724+0000 7f61aaa00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:41.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.725+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f61ac08bff0 con 0x7f61b4162980 2024-04-26T23:48:41.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.725+0000 7f61b2a00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:41.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.726+0000 7f61b2a00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f61a4034000 tx=0x7f61a402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:41.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.729+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61ac059fe0 con 0x7f61b4162980 2024-04-26T23:48:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.931+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f61b40cf4a0 con 0x7f61b4162980 2024-04-26T23:48:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.932+0000 7f61aaa00640 1 -- 172.21.15.46:0/86799784 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) ==== 84+0+65 (secure 0 0 0) 0x7f61ac05e2e0 con 0x7f61b4162980 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:48:41.937 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 msgr2=0x7f619406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f61a4034000 tx=0x7f61a402e000 comp rx=0 tx=0).stop 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 msgr2=0x7f61b415b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f61ac007d40 tx=0x7f61ac00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 shutdown_connections 2024-04-26T23:48:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b4162980 0x7f61b415b6f0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61b40d31f0 0x7f61b415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 --2- 172.21.15.46:0/86799784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61b40d2200 0x7f61b40c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.934+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 >> 172.21.15.46:0/86799784 conn(0x7f61b409e420 msgr2=0x7f61b415c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.935+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 shutdown_connections 2024-04-26T23:48:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:41.935+0000 7f61b9c00640 1 -- 172.21.15.46:0/86799784 wait complete. 2024-04-26T23:48:41.958 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:48:41.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:48:42.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.166+0000 7f8e80a00640 1 Processor -- start 2024-04-26T23:48:42.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.166+0000 7f8e80a00640 1 -- start start 2024-04-26T23:48:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e80a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e7c1627f0 0x7f8e7c0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e80a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 0x7f8e7c0d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e80a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e7c0d01a0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e80a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8e7c0d0480 con 0x7f8e7c1627f0 2024-04-26T23:48:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e80a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0d0760 con 0x7f8e7c0d2080 2024-04-26T23:48:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.168+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 0x7f8e7c0d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 0x7f8e7c0d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:45392/0 (socket says 172.21.15.46:45392) 2024-04-26T23:48:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e79e00640 1 -- 172.21.15.46:0/3091316059 learned_addr learned my addr 172.21.15.46:0/3091316059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e7a800640 1 --2- 172.21.15.46:0/3091316059 >> v2:172.21.15.142:3300/0 conn(0x7f8e7c1627f0 0x7f8e7c0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e7a800640 1 -- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 msgr2=0x7f8e7c0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e7a800640 1 --2- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 0x7f8e7c0d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e7a800640 1 -- 172.21.15.46:0/3091316059 >> v1:172.21.15.142:6789/0 conn(0x7f8e7c0d3af0 legacy=0x7f8e7c162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.169+0000 7f8e7a800640 1 -- 172.21.15.46:0/3091316059 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c0d0a40 con 0x7f8e7c1627f0 2024-04-26T23:48:42.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.171+0000 7f8e7a800640 1 --2- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c1627f0 0x7f8e7c0cfbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0097a0 tx=0x7f8e6c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfd7ccf0e27d573 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:42.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.171+0000 7f8e79400640 1 -- 172.21.15.46:0/3091316059 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c037070 con 0x7f8e7c1627f0 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e79400640 1 -- 172.21.15.46:0/3091316059 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6c032440 con 0x7f8e7c1627f0 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e79400640 1 -- 172.21.15.46:0/3091316059 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c032810 con 0x7f8e7c1627f0 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 -- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c1627f0 msgr2=0x7f8e7c0cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 --2- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c1627f0 0x7f8e7c0cfbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0097a0 tx=0x7f8e6c02a210 comp rx=0 tx=0).stop 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 -- 172.21.15.46:0/3091316059 shutdown_connections 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 --2- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c0d2080 0x7f8e7c0d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 --2- 172.21.15.46:0/3091316059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c1627f0 0x7f8e7c0cfbc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.172+0000 7f8e80a00640 1 -- 172.21.15.46:0/3091316059 >> 172.21.15.46:0/3091316059 conn(0x7f8e7c09e380 msgr2=0x7f8e7c0d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:42.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.173+0000 7f8e80a00640 1 -- 172.21.15.46:0/3091316059 shutdown_connections 2024-04-26T23:48:42.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.173+0000 7f8e80a00640 1 -- 172.21.15.46:0/3091316059 wait complete. 2024-04-26T23:48:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.175+0000 7f8e80a00640 1 Processor -- start 2024-04-26T23:48:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.175+0000 7f8e80a00640 1 -- start start 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c0d2080 0x7f8e7c159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c1627f0 0x7f8e7c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c162bd0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c0d24f0 con 0x7f8e7c0d2080 2024-04-26T23:48:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.176+0000 7f8e80a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0b5560 con 0x7f8e7c1627f0 2024-04-26T23:48:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41382/0 (socket says 172.21.15.46:41382) 2024-04-26T23:48:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 -- 172.21.15.46:0/930495028 learned_addr learned my addr 172.21.15.46:0/930495028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 -- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c1627f0 msgr2=0x7f8e7c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c1627f0 0x7f8e7c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 -- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c0d2080 msgr2=0x7f8e7c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c0d2080 0x7f8e7c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 -- 172.21.15.46:0/930495028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c0cee80 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.177+0000 7f8e79e00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e8b0 tx=0x7f8e6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.178+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68017070 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.178+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6800c940 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.178+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800cd10 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.178+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c0b86f0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.178+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c0b8cb0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.179+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c0693b0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.180+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e6801a070 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.181+0000 7f8e72a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 0x7f8e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.182+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f8e6808b8f0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.182+0000 7f8e7a800640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 0x7f8e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.183+0000 7f8e7a800640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 0x7f8e5c06edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c034000 tx=0x7f8e6c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.186+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8e68059ed0 con 0x7f8e7c0d3af0 2024-04-26T23:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:42.389+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8e7c0d3390 con 0x7f8e7c0d3af0 2024-04-26T23:48:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.054+0000 7f8e72a00640 1 -- 172.21.15.46:0/930495028 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) ==== 138+0+0 (secure 0 0 0) 0x7f8e680597d0 con 0x7f8e7c0d3af0 2024-04-26T23:48:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:48:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.058+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 msgr2=0x7f8e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.058+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 0x7f8e5c06edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c034000 tx=0x7f8e6c02c000 comp rx=0 tx=0).stop 2024-04-26T23:48:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 msgr2=0x7f8e7c1e9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e8b0 tx=0x7f8e6800ed70 comp rx=0 tx=0).stop 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 shutdown_connections 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e5c06c8f0 0x7f8e5c06edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e7c1627f0 0x7f8e7c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c0d3af0 0x7f8e7c1e9b80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 --2- 172.21.15.46:0/930495028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e7c0d2080 0x7f8e7c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.059+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 >> 172.21.15.46:0/930495028 conn(0x7f8e7c09e380 msgr2=0x7f8e7c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.060+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 shutdown_connections 2024-04-26T23:48:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.060+0000 7f8e80a00640 1 -- 172.21.15.46:0/930495028 wait complete. 2024-04-26T23:48:43.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:48:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.289+0000 7ffa6e800640 1 Processor -- start 2024-04-26T23:48:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.289+0000 7ffa6e800640 1 -- start start 2024-04-26T23:48:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa681626c0 0x7ffa680b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 0x7ffa680d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa680d07d0 con 0x7ffa68162aa0 2024-04-26T23:48:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffa680d0ab0 con 0x7ffa681626c0 2024-04-26T23:48:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa680d0d90 con 0x7ffa680b5fb0 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa681626c0 0x7ffa680b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa681626c0 0x7ffa680b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41394/0 (socket says 172.21.15.46:41394) 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.291+0000 7ffa6d400640 1 -- 172.21.15.46:0/105833474 learned_addr learned my addr 172.21.15.46:0/105833474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6ca00640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 0x7ffa680d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6d400640 1 -- 172.21.15.46:0/105833474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 msgr2=0x7ffa680d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6d400640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 0x7ffa680d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6d400640 1 -- 172.21.15.46:0/105833474 >> v1:172.21.15.142:6789/0 conn(0x7ffa68162aa0 legacy=0x7ffa680d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6d400640 1 -- 172.21.15.46:0/105833474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa680d2c80 con 0x7ffa681626c0 2024-04-26T23:48:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.292+0000 7ffa6ca00640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 0x7ffa680d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:43.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.293+0000 7ffa6d400640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa681626c0 0x7ffa680b47c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009870 tx=0x7ffa5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a7c712275fc4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:43.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.293+0000 7ffa67e00640 1 -- 172.21.15.46:0/105833474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c037070 con 0x7ffa681626c0 2024-04-26T23:48:43.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.293+0000 7ffa67e00640 1 -- 172.21.15.46:0/105833474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c032440 con 0x7ffa681626c0 2024-04-26T23:48:43.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.293+0000 7ffa67e00640 1 -- 172.21.15.46:0/105833474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c032810 con 0x7ffa681626c0 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 -- 172.21.15.46:0/105833474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa681626c0 msgr2=0x7ffa680b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa681626c0 0x7ffa680b47c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009870 tx=0x7ffa5c02a210 comp rx=0 tx=0).stop 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 -- 172.21.15.46:0/105833474 shutdown_connections 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa680b5fb0 0x7ffa680d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 --2- 172.21.15.46:0/105833474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa681626c0 0x7ffa680b47c0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 -- 172.21.15.46:0/105833474 >> 172.21.15.46:0/105833474 conn(0x7ffa6809e2a0 msgr2=0x7ffa680d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 -- 172.21.15.46:0/105833474 shutdown_connections 2024-04-26T23:48:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.294+0000 7ffa6e800640 1 -- 172.21.15.46:0/105833474 wait complete. 2024-04-26T23:48:43.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.296+0000 7ffa6e800640 1 Processor -- start 2024-04-26T23:48:43.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.296+0000 7ffa6e800640 1 -- start start 2024-04-26T23:48:43.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa680b5fb0 0x7ffa68151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 0x7ffa681f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680d2190 con 0x7ffa68162aa0 2024-04-26T23:48:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680d2010 con 0x7ffa680b5fb0 2024-04-26T23:48:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.297+0000 7ffa6e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa680d31a0 con 0x7ffa681626c0 2024-04-26T23:48:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41410/0 (socket says 172.21.15.46:41410) 2024-04-26T23:48:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 -- 172.21.15.46:0/3526844955 learned_addr learned my addr 172.21.15.46:0/3526844955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6d400640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa680b5fb0 0x7ffa68151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6ca00640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 0x7ffa681f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 -- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 msgr2=0x7ffa681f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 0x7ffa681f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 -- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa680b5fb0 msgr2=0x7ffa68151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa680b5fb0 0x7ffa68151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6de00640 1 -- 172.21.15.46:0/3526844955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68152550 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.298+0000 7ffa6ca00640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 0x7ffa681f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa6de00640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b9e0 tx=0x7ffa6000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60003b70 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa60003d10 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000cb60 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa68152830 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.299+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6814fef0 con 0x7ffa68162aa0 2024-04-26T23:48:43.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.300+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa680693b0 con 0x7ffa68162aa0 2024-04-26T23:48:43.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.302+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa6000cd00 con 0x7ffa68162aa0 2024-04-26T23:48:43.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.303+0000 7ffa65600640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 0x7ffa4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:43.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.303+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5619+0+0 (secure 0 0 0) 0x7ffa6008b990 con 0x7ffa68162aa0 2024-04-26T23:48:43.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.303+0000 7ffa6d400640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 0x7ffa4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:43.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.304+0000 7ffa6d400640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 0x7ffa4806edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c034000 tx=0x7ffa5c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:43.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.308+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa60059e00 con 0x7ffa68162aa0 2024-04-26T23:48:43.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:43.519+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffa680d25b0 con 0x7ffa68162aa0 2024-04-26T23:48:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.061+0000 7ffa65600640 1 -- 172.21.15.46:0/3526844955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) ==== 170+0+0 (secure 0 0 0) 0x7ffa60059700 con 0x7ffa68162aa0 2024-04-26T23:48:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:48:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 msgr2=0x7ffa4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 0x7ffa4806edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c034000 tx=0x7ffa5c02e000 comp rx=0 tx=0).stop 2024-04-26T23:48:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 msgr2=0x7ffa680b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b9e0 tx=0x7ffa6000bea0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 shutdown_connections 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa4806c8f0 0x7ffa4806edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa68162aa0 0x7ffa680b2f30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa681626c0 0x7ffa681f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 --2- 172.21.15.46:0/3526844955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa680b5fb0 0x7ffa68151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.066+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 >> 172.21.15.46:0/3526844955 conn(0x7ffa6809e2a0 msgr2=0x7ffa680cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.067+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 shutdown_connections 2024-04-26T23:48:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.067+0000 7ffa6e800640 1 -- 172.21.15.46:0/3526844955 wait complete. 2024-04-26T23:48:44.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:48:44.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.284+0000 7fe519800640 1 Processor -- start 2024-04-26T23:48:44.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.284+0000 7fe519800640 1 -- start start 2024-04-26T23:48:44.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe519800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe514162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe519800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51406f200 con 0x7fe514163270 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe519800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe514163a20 con 0x7fe51415f1f0 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe519800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe514163d00 con 0x7fe51406ca60 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe513400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe514163270 0x7fe51406fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57164/0 (socket says 172.21.15.46:57164) 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.286+0000 7fe513400640 1 -- 172.21.15.46:0/4227074351 learned_addr learned my addr 172.21.15.46:0/4227074351 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:44.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe50be00640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe514162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030877196 0 0) 0x7fe51406f200 con 0x7fe514163270 2024-04-26T23:48:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4f8003520 con 0x7fe514163270 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1405707681 0 0) 0x7fe4f8003520 con 0x7fe514163270 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 msgr2=0x7fe514162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe514162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 >> v1:172.21.15.142:6789/0 conn(0x7fe51415f1f0 legacy=0x7fe51406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51406f4a0 con 0x7fe514163270 2024-04-26T23:48:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe50c0019f0 con 0x7fe514163270 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.287+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 >> v1:172.21.15.46:6789/0 conn(0x7fe514163270 legacy=0x7fe51406fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe4f80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f8003fb0 0x7fe4f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8004d40 0x7fe4f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe514163d00 con 0x7fe4f8004d40 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f800bb00 con 0x7fe4f8003fb0 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f800bd00 con 0x7fe51406ca60 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe513400640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8004d40 0x7fe4f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe50be00640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f8003fb0 0x7fe4f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.288+0000 7fe512a00640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe4f80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.289+0000 7fe513400640 1 -- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 msgr2=0x7fe4f80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.289+0000 7fe513400640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51406ca60 0x7fe4f80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.289+0000 7fe513400640 1 -- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f8003fb0 msgr2=0x7fe4f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.289+0000 7fe513400640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f8003fb0 0x7fe4f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.289+0000 7fe513400640 1 -- 172.21.15.46:0/4227074351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8003520 con 0x7fe4f8004d40 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.290+0000 7fe513400640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8004d40 0x7fe4f80051c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00a7b0 tx=0x7fe4f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2317af17437208f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.290+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00a3d0 con 0x7fe4f8004d40 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.290+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50c02f070 con 0x7fe4f8004d40 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.290+0000 7fe512000640 1 -- 172.21.15.46:0/4227074351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c02bc80 con 0x7fe4f8004d40 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 -- 172.21.15.46:0/4227074351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8004d40 msgr2=0x7fe4f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 --2- 172.21.15.46:0/4227074351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8004d40 0x7fe4f80051c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00a7b0 tx=0x7fe4f8005830 comp rx=0 tx=0).stop 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe513400640 1 -- 172.21.15.46:0/4227074351 reap_dead start 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 -- 172.21.15.46:0/4227074351 shutdown_connections 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 -- 172.21.15.46:0/4227074351 >> 172.21.15.46:0/4227074351 conn(0x7fe51406da90 msgr2=0x7fe5140d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 -- 172.21.15.46:0/4227074351 shutdown_connections 2024-04-26T23:48:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.291+0000 7fe519800640 1 -- 172.21.15.46:0/4227074351 wait complete. 2024-04-26T23:48:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.293+0000 7fe519800640 1 Processor -- start 2024-04-26T23:48:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.293+0000 7fe519800640 1 -- start start 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe514161260 0x7fe514161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe514162530 0x7fe51414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe514163700 con 0x7fe514161640 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe51406f810 con 0x7fe514161260 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe519800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe514162d30 con 0x7fe514162530 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe513400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe513400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41428/0 (socket says 172.21.15.46:41428) 2024-04-26T23:48:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.294+0000 7fe513400640 1 -- 172.21.15.46:0/1327268238 learned_addr learned my addr 172.21.15.46:0/1327268238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe50be00640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe514162530 0x7fe51414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe512a00640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe514161260 0x7fe514161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 -- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe514162530 msgr2=0x7fe51414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe514162530 0x7fe51414bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 -- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe514161260 msgr2=0x7fe514161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe514161260 0x7fe514161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 -- 172.21.15.46:0/1327268238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5140c5a10 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe513400640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02b470 tx=0x7fe50c001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00a3d0 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50c02bc00 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.295+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c001200 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.296+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5140c5cf0 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.296+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5140c62b0 con 0x7fe514161640 2024-04-26T23:48:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.297+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe514069530 con 0x7fe514161640 2024-04-26T23:48:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.298+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe50c03e030 con 0x7fe514161640 2024-04-26T23:48:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.299+0000 7fe50aa00640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 0x7fe4f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.299+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6001+0+0 (secure 0 0 0) 0x7fe50c0ab2d0 con 0x7fe514161640 2024-04-26T23:48:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.300+0000 7fe512a00640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 0x7fe4f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.301+0000 7fe512a00640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 0x7fe4f006ede0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe504007fd0 tx=0x7fe504009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:44.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.304+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe50c0795c0 con 0x7fe514161640 2024-04-26T23:48:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:44.552+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe51415d920 con 0x7fe514161640 2024-04-26T23:48:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.101+0000 7fe50aa00640 1 -- 172.21.15.46:0/1327268238 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v96) ==== 174+0+0 (secure 0 0 0) 0x7fe50c078ec0 con 0x7fe514161640 2024-04-26T23:48:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 17 and data pool 18 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.109+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 msgr2=0x7fe4f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.109+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 0x7fe4f006ede0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe504007fd0 tx=0x7fe504009040 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.109+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 msgr2=0x7fe51414c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.109+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02b470 tx=0x7fe50c001cf0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 shutdown_connections 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe4f006c920 0x7fe4f006ede0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe514161640 0x7fe51414c220 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe514162530 0x7fe51414bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 --2- 172.21.15.46:0/1327268238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe514161260 0x7fe514161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.110+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 >> 172.21.15.46:0/1327268238 conn(0x7fe51406da90 msgr2=0x7fe5140afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.111+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 shutdown_connections 2024-04-26T23:48:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.111+0000 7fe519800640 1 -- 172.21.15.46:0/1327268238 wait complete. 2024-04-26T23:48:45.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.328+0000 7f971b200640 1 Processor -- start 2024-04-26T23:48:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.328+0000 7f971b200640 1 -- start start 2024-04-26T23:48:45.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.329+0000 7f971b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 0x7f971c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.329+0000 7f971b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f971c0720e0 0x7f971c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f971b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c073ad0 con 0x7f971c160910 2024-04-26T23:48:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f971b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f971c146a90 con 0x7f971c0720e0 2024-04-26T23:48:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f971b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f971c073d70 con 0x7f971c1466b0 2024-04-26T23:48:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f971a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f971c160910 0x7f971c0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57178/0 (socket says 172.21.15.46:57178) 2024-04-26T23:48:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f971a800640 1 -- 172.21.15.46:0/1788658075 learned_addr learned my addr 172.21.15.46:0/1788658075 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f9719400640 1 --2- 172.21.15.46:0/1788658075 >> v2:172.21.15.142:3300/0 conn(0x7f971c0720e0 0x7f971c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.330+0000 7f9719e00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 0x7f971c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532722500 0 0) 0x7f971c073ad0 con 0x7f971c160910 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9700003520 con 0x7f971c160910 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641566336 0 0) 0x7f9700003520 con 0x7f971c160910 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 msgr2=0x7f971c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 0x7f971c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 >> v2:172.21.15.142:3300/0 conn(0x7f971c0720e0 msgr2=0x7f971c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 --2- 172.21.15.46:0/1788658075 >> v2:172.21.15.142:3300/0 conn(0x7f971c0720e0 0x7f971c0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c074050 con 0x7f971c160910 2024-04-26T23:48:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f97140019f0 con 0x7f971c160910 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 >> v1:172.21.15.46:6789/0 conn(0x7f971c160910 legacy=0x7f971c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700004800 0x7f9700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 0x7f97000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f971c146a90 con 0x7f9700004800 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f971c073d70 con 0x7f97000039b0 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f970000fdd0 con 0x7f971c1466b0 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.331+0000 7f9719400640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700004800 0x7f9700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719e00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719400640 1 -- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 msgr2=0x7f97000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719400640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c1466b0 0x7f97000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719400640 1 -- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97000039b0 msgr2=0x7f9700003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719400640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719400640 1 -- 172.21.15.46:0/1788658075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9700003520 con 0x7f9700004800 2024-04-26T23:48:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.332+0000 7f9719e00640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.333+0000 7f9719400640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700004800 0x7f9700004c80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9708009ac0 tx=0x7f9700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7f361d55771be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.333+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708036070 con 0x7f9700004800 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.333+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9708031400 con 0x7f9700004800 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.333+0000 7f9718a00640 1 -- 172.21.15.46:0/1788658075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708009690 con 0x7f9700004800 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 -- 172.21.15.46:0/1788658075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700004800 msgr2=0x7f9700004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 --2- 172.21.15.46:0/1788658075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700004800 0x7f9700004c80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9708009ac0 tx=0x7f9700005830 comp rx=0 tx=0).stop 2024-04-26T23:48:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971a800640 1 -- 172.21.15.46:0/1788658075 reap_dead start 2024-04-26T23:48:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 -- 172.21.15.46:0/1788658075 shutdown_connections 2024-04-26T23:48:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 -- 172.21.15.46:0/1788658075 >> 172.21.15.46:0/1788658075 conn(0x7f971c0b26d0 msgr2=0x7f971c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 -- 172.21.15.46:0/1788658075 shutdown_connections 2024-04-26T23:48:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.334+0000 7f971b200640 1 -- 172.21.15.46:0/1788658075 wait complete. 2024-04-26T23:48:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.336+0000 7f971b200640 1 Processor -- start 2024-04-26T23:48:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.336+0000 7f971b200640 1 -- start start 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 0x7f971c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c0cfbf0 0x7f971c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0cdda0 con 0x7f971c15e430 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0cb470 con 0x7f9700004800 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f971b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0cc0a0 con 0x7f971c0cfbf0 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f9719400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f9719400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41442/0 (socket says 172.21.15.46:41442) 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.337+0000 7f9719400640 1 -- 172.21.15.46:0/3938436178 learned_addr learned my addr 172.21.15.46:0/3938436178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719e00640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 0x7f971c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f971a800640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c0cfbf0 0x7f971c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 -- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c0cfbf0 msgr2=0x7f971c0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c0cfbf0 0x7f971c0d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 -- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 msgr2=0x7f971c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 0x7f971c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 -- 172.21.15.46:0/3938436178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c0ca2b0 con 0x7f971c15e430 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719e00640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 0x7f971c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9719400640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9708009be0 tx=0x7f9708004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708040020 con 0x7f971c15e430 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.338+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9708003a40 con 0x7f971c15e430 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.339+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c0b4e10 con 0x7f971c15e430 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.339+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708036040 con 0x7f971c15e430 2024-04-26T23:48:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.339+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c0ca520 con 0x7f971c15e430 2024-04-26T23:48:45.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.340+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c0693b0 con 0x7f971c15e430 2024-04-26T23:48:45.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.341+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9708043020 con 0x7f971c15e430 2024-04-26T23:48:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.343+0000 7f9712000640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.343+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f97080abee0 con 0x7f971c15e430 2024-04-26T23:48:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.344+0000 7f9719e00640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.345+0000 7f9719e00640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f970c0099c0 tx=0x7f970c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.347+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f970804a070 con 0x7f971c15e430 2024-04-26T23:48:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.554+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f971c0c7180 con 0x7f971c15e430 2024-04-26T23:48:45.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.557+0000 7f9712000640 1 -- 172.21.15.46:0/3938436178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16929 (secure 0 0 0) 0x7f970807a170 con 0x7f971c15e430 2024-04-26T23:48:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:45.562 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:44.554599+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 msgr2=0x7f96fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f970c0099c0 tx=0x7f970c008040 comp rx=0 tx=0).stop 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 msgr2=0x7f971c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9708009be0 tx=0x7f9708004730 comp rx=0 tx=0).stop 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 shutdown_connections 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f971c0cfbf0 0x7f971c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f971c15e430 0x7f971c13f740 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 --2- 172.21.15.46:0/3938436178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9700004800 0x7f971c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 >> 172.21.15.46:0/3938436178 conn(0x7f971c0b26d0 msgr2=0x7f971c1429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 shutdown_connections 2024-04-26T23:48:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.560+0000 7f971b200640 1 -- 172.21.15.46:0/3938436178 wait complete. 2024-04-26T23:48:45.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.791+0000 7fbf87e00640 1 Processor -- start 2024-04-26T23:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.792+0000 7fbf87e00640 1 -- start start 2024-04-26T23:48:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.793+0000 7fbf87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf8815f1d0 0x7fbf880cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.793+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf880cd770 0x7fbf880c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.793+0000 7fbf87e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf880b03c0 con 0x7fbf8815edf0 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.793+0000 7fbf87e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbf880cf5c0 con 0x7fbf8815f1d0 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.793+0000 7fbf87e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf880cb680 con 0x7fbf880cd770 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf8815f1d0 0x7fbf880cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf8815f1d0 0x7fbf880cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41458/0 (socket says 172.21.15.46:41458) 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 -- 172.21.15.46:0/3255486972 learned_addr learned my addr 172.21.15.46:0/3255486972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf87400640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf880cd770 0x7fbf880c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 -- 172.21.15.46:0/3255486972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf880cd770 msgr2=0x7fbf880c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf880cd770 0x7fbf880c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 -- 172.21.15.46:0/3255486972 >> v1:172.21.15.142:6789/0 conn(0x7fbf8815edf0 legacy=0x7fbf880cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.794+0000 7fbf86000640 1 -- 172.21.15.46:0/3255486972 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf880cb960 con 0x7fbf8815f1d0 2024-04-26T23:48:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.795+0000 7fbf86000640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815f1d0 0x7fbf880cf080 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf9e19728b7f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf85600640 1 -- 172.21.15.46:0/3255486972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74037070 con 0x7fbf8815f1d0 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf85600640 1 -- 172.21.15.46:0/3255486972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7402ae50 con 0x7fbf8815f1d0 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf85600640 1 -- 172.21.15.46:0/3255486972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74032860 con 0x7fbf8815f1d0 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 -- 172.21.15.46:0/3255486972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815f1d0 msgr2=0x7fbf880cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815f1d0 0x7fbf880cf080 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a300 comp rx=0 tx=0).stop 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 -- 172.21.15.46:0/3255486972 shutdown_connections 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf880cd770 0x7fbf880c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 --2- 172.21.15.46:0/3255486972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815f1d0 0x7fbf880cf080 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 -- 172.21.15.46:0/3255486972 >> 172.21.15.46:0/3255486972 conn(0x7fbf880b26d0 msgr2=0x7fbf880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 -- 172.21.15.46:0/3255486972 shutdown_connections 2024-04-26T23:48:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.796+0000 7fbf87e00640 1 -- 172.21.15.46:0/3255486972 wait complete. 2024-04-26T23:48:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.798+0000 7fbf87e00640 1 Processor -- start 2024-04-26T23:48:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.798+0000 7fbf87e00640 1 -- start start 2024-04-26T23:48:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf880cd770 0x7fbf880b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf8815bac0 0x7fbf880c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf880cc200 con 0x7fbf8815edf0 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf880cbf20 con 0x7fbf880cd770 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf87e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf880cb1a0 con 0x7fbf8815bac0 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.799+0000 7fbf86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41460/0 (socket says 172.21.15.46:41460) 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 -- 172.21.15.46:0/2244134531 learned_addr learned my addr 172.21.15.46:0/2244134531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86a00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf880cd770 0x7fbf880b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf87400640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf8815bac0 0x7fbf880c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 -- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf8815bac0 msgr2=0x7fbf880c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf8815bac0 0x7fbf880c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 -- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf880cd770 msgr2=0x7fbf880b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf880cd770 0x7fbf880b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 -- 172.21.15.46:0/2244134531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8815d6a0 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.800+0000 7fbf86000640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009990 tx=0x7fbf740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.801+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74041020 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.801+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7403c670 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.801+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74037040 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.801+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf880b3e50 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.801+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf880b4410 con 0x7fbf8815edf0 2024-04-26T23:48:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.802+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf880693b0 con 0x7fbf8815edf0 2024-04-26T23:48:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.803+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf74044020 con 0x7fbf8815edf0 2024-04-26T23:48:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.804+0000 7fbf7f400640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 0x7fbf6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.804+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fbf740ac360 con 0x7fbf8815edf0 2024-04-26T23:48:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.805+0000 7fbf86a00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 0x7fbf6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.806+0000 7fbf86a00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 0x7fbf6806edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf780012e0 tx=0x7fbf78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:45.808+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf7404b030 con 0x7fbf8815edf0 2024-04-26T23:48:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.016+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf880cd3a0 con 0x7fbf8815edf0 2024-04-26T23:48:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.019+0000 7fbf7f400640 1 -- 172.21.15.46:0/2244134531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16929 (secure 0 0 0) 0x7fbf7407a600 con 0x7fbf8815edf0 2024-04-26T23:48:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:46.024 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:44.554599+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.021+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 msgr2=0x7fbf6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.021+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 0x7fbf6806edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf780012e0 tx=0x7fbf78009040 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.021+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 msgr2=0x7fbf88151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.021+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009990 tx=0x7fbf740046c0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 shutdown_connections 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf6806c8f0 0x7fbf6806edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf8815bac0 0x7fbf880c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8815edf0 0x7fbf88151960 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 --2- 172.21.15.46:0/2244134531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf880cd770 0x7fbf880b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 >> 172.21.15.46:0/2244134531 conn(0x7fbf880b26d0 msgr2=0x7fbf880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 shutdown_connections 2024-04-26T23:48:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.022+0000 7fbf87e00640 1 -- 172.21.15.46:0/2244134531 wait complete. 2024-04-26T23:48:46.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:48:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.252+0000 7ff185000640 1 Processor -- start 2024-04-26T23:48:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.253+0000 7ff185000640 1 -- start start 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.254+0000 7ff185000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 0x7ff1800b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.254+0000 7ff185000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800d07d0 con 0x7ff180162aa0 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.254+0000 7ff185000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800d0ab0 con 0x7ff1800b5fb0 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.254+0000 7ff185000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800d0d90 con 0x7ff1801626c0 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.255+0000 7ff17fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff180162aa0 0x7ff1800d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57182/0 (socket says 172.21.15.46:57182) 2024-04-26T23:48:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.255+0000 7ff17fe00640 1 -- 172.21.15.46:0/1537167008 learned_addr learned my addr 172.21.15.46:0/1537167008 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.255+0000 7ff17f400640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 0x7ff1800b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.255+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138505665 0 0) 0x7ff1800d07d0 con 0x7ff180162aa0 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.255+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff164003520 con 0x7ff180162aa0 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2781627902 0 0) 0x7ff164003520 con 0x7ff180162aa0 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 msgr2=0x7ff1800b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 0x7ff1800b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 >> v1:172.21.15.142:6789/0 conn(0x7ff1800b5fb0 legacy=0x7ff1800d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800d2c80 con 0x7ff180162aa0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff1780019f0 con 0x7ff180162aa0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 >> v1:172.21.15.46:6789/0 conn(0x7ff180162aa0 legacy=0x7ff1800d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1640039e0 0x7ff164003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff164004800 0x7ff164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 0x7ff1640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800d0d90 con 0x7ff1640039e0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff16400ba70 con 0x7ff164004800 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff16400bc70 con 0x7ff1801626c0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.256+0000 7ff17f400640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1640039e0 0x7ff164003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17ea00640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff164004800 0x7ff164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17f400640 1 -- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 msgr2=0x7ff1640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17f400640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1801626c0 0x7ff1640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17f400640 1 -- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff164004800 msgr2=0x7ff164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17f400640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff164004800 0x7ff164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17f400640 1 -- 172.21.15.46:0/1537167008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164003520 con 0x7ff1640039e0 2024-04-26T23:48:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.257+0000 7ff17ea00640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff164004800 0x7ff164004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff17f400640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1640039e0 0x7ff164003e00 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff170009a90 tx=0x7ff164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53181397573324b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170036070 con 0x7ff1640039e0 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff170031400 con 0x7ff1640039e0 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff17e000640 1 -- 172.21.15.46:0/1537167008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170009690 con 0x7ff1640039e0 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff185000640 1 -- 172.21.15.46:0/1537167008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1640039e0 msgr2=0x7ff164003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.258+0000 7ff185000640 1 --2- 172.21.15.46:0/1537167008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1640039e0 0x7ff164003e00 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff170009a90 tx=0x7ff164005830 comp rx=0 tx=0).stop 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.259+0000 7ff17fe00640 1 -- 172.21.15.46:0/1537167008 reap_dead start 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.259+0000 7ff185000640 1 -- 172.21.15.46:0/1537167008 shutdown_connections 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.259+0000 7ff185000640 1 -- 172.21.15.46:0/1537167008 >> 172.21.15.46:0/1537167008 conn(0x7ff18009e2a0 msgr2=0x7ff1800d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.259+0000 7ff185000640 1 -- 172.21.15.46:0/1537167008 shutdown_connections 2024-04-26T23:48:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.259+0000 7ff185000640 1 -- 172.21.15.46:0/1537167008 wait complete. 2024-04-26T23:48:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.260+0000 7ff185000640 1 Processor -- start 2024-04-26T23:48:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.261+0000 7ff185000640 1 -- start start 2024-04-26T23:48:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1640039e0 0x7ff18015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1800ca9c0 0x7ff180159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800b47c0 con 0x7ff18015a160 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800b4940 con 0x7ff1640039e0 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff185000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800b4ac0 con 0x7ff1800ca9c0 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41470/0 (socket says 172.21.15.46:41470) 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 -- 172.21.15.46:0/2320569236 learned_addr learned my addr 172.21.15.46:0/2320569236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17ea00640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1800ca9c0 0x7ff180159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17f400640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1640039e0 0x7ff18015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 -- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1800ca9c0 msgr2=0x7ff180159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1800ca9c0 0x7ff180159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 -- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1640039e0 msgr2=0x7ff18015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1640039e0 0x7ff18015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.262+0000 7ff17fe00640 1 -- 172.21.15.46:0/2320569236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18015eb70 con 0x7ff18015a160 2024-04-26T23:48:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff17fe00640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ff180162e80 tx=0x7ff17800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17800a610 con 0x7ff18015a160 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17800a7b0 con 0x7ff18015a160 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17800ab80 con 0x7ff18015a160 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1800a2c20 con 0x7ff18015a160 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.263+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1800ccaf0 con 0x7ff18015a160 2024-04-26T23:48:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.265+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1800693b0 con 0x7ff18015a160 2024-04-26T23:48:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.266+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff178011070 con 0x7ff18015a160 2024-04-26T23:48:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.266+0000 7ff177400640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 0x7ff15c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.267+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7ff17808b2b0 con 0x7ff18015a160 2024-04-26T23:48:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.267+0000 7ff17f400640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 0x7ff15c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.268+0000 7ff17f400640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 0x7ff15c06edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff170033000 tx=0x7ff170008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.271+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff178059540 con 0x7ff18015a160 2024-04-26T23:48:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:46.517+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff1800d25b0 con 0x7ff18015a160 2024-04-26T23:48:47.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.123+0000 7ff177400640 1 -- 172.21.15.46:0/2320569236 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7ff178058e40 con 0x7ff18015a160 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 msgr2=0x7ff15c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 0x7ff15c06edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff170033000 tx=0x7ff170008030 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 msgr2=0x7ff18015bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ff180162e80 tx=0x7ff17800ccd0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 shutdown_connections 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15c06c8f0 0x7ff15c06edb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18015a160 0x7ff18015bbb0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1800ca9c0 0x7ff180159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 --2- 172.21.15.46:0/2320569236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1640039e0 0x7ff18015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 >> 172.21.15.46:0/2320569236 conn(0x7ff18009e2a0 msgr2=0x7ff1800b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 shutdown_connections 2024-04-26T23:48:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.128+0000 7ff185000640 1 -- 172.21.15.46:0/2320569236 wait complete. 2024-04-26T23:48:47.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:48:47.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:48:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.365+0000 7f490f200640 1 Processor -- start 2024-04-26T23:48:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.366+0000 7f490f200640 1 -- start start 2024-04-26T23:48:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.367+0000 7f490f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f49100b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.367+0000 7f490f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49100d07d0 con 0x7f4910162aa0 2024-04-26T23:48:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.367+0000 7f490f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49100d0ab0 con 0x7f49101626c0 2024-04-26T23:48:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.367+0000 7f490f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49100d0d90 con 0x7f49100b5fb0 2024-04-26T23:48:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.368+0000 7f490e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4910162aa0 0x7f49100d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57184/0 (socket says 172.21.15.46:57184) 2024-04-26T23:48:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.368+0000 7f490e800640 1 -- 172.21.15.46:0/631508030 learned_addr learned my addr 172.21.15.46:0/631508030 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.368+0000 7f4907e00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f49100b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.368+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507947010 0 0) 0x7f49100d07d0 con 0x7f4910162aa0 2024-04-26T23:48:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.368+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48f4003520 con 0x7f4910162aa0 2024-04-26T23:48:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2440421457 0 0) 0x7f48f4003520 con 0x7f4910162aa0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 msgr2=0x7f49100b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f49100b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 >> v1:172.21.15.142:6789/0 conn(0x7f49101626c0 legacy=0x7f49100d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100d2c80 con 0x7f4910162aa0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f49080019f0 con 0x7f4910162aa0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 >> v1:172.21.15.46:6789/0 conn(0x7f4910162aa0 legacy=0x7f49100d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f48f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f48f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4004d40 0x7f48f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49100d0d90 con 0x7f48f4003f20 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48f400ba20 con 0x7f48f4004d40 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48f400bc20 con 0x7f49100b5fb0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f4907e00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f48f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f490e800640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4004d40 0x7f48f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.369+0000 7f490de00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f48f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f4907e00640 1 -- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 msgr2=0x7f48f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f4907e00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f48f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f4907e00640 1 -- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4004d40 msgr2=0x7f48f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f4907e00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4004d40 0x7f48f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f4907e00640 1 -- 172.21.15.46:0/631508030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f4003520 con 0x7f48f4003f20 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f490de00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49100b5fb0 0x7f48f40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:47.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.370+0000 7f490e800640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4004d40 0x7f48f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.371+0000 7f4907e00640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f48f4004800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009ab0 tx=0x7f48f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb71cb95ea166f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.371+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8036070 con 0x7f48f4003f20 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.371+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f8005990 con 0x7f48f4003f20 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.371+0000 7f490d400640 1 -- 172.21.15.46:0/631508030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8009680 con 0x7f48f4003f20 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 -- 172.21.15.46:0/631508030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 msgr2=0x7f48f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 --2- 172.21.15.46:0/631508030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f48f4004800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009ab0 tx=0x7f48f4005830 comp rx=0 tx=0).stop 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490e800640 1 -- 172.21.15.46:0/631508030 reap_dead start 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 -- 172.21.15.46:0/631508030 shutdown_connections 2024-04-26T23:48:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 -- 172.21.15.46:0/631508030 >> 172.21.15.46:0/631508030 conn(0x7f491009e2a0 msgr2=0x7f49100d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 -- 172.21.15.46:0/631508030 shutdown_connections 2024-04-26T23:48:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.372+0000 7f490f200640 1 -- 172.21.15.46:0/631508030 wait complete. 2024-04-26T23:48:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.374+0000 7f490f200640 1 Processor -- start 2024-04-26T23:48:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.374+0000 7f490f200640 1 -- start start 2024-04-26T23:48:47.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f491015d190 0x7f49100d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491015d610 0x7f49100d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49100b47c0 con 0x7f48f4003f20 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49100d2220 con 0x7f491015d610 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49100b4940 con 0x7f491015d190 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41498/0 (socket says 172.21.15.46:41498) 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 -- 172.21.15.46:0/4221287285 learned_addr learned my addr 172.21.15.46:0/4221287285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490e800640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491015d610 0x7f49100d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f4907e00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f491015d190 0x7f49100d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 -- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f491015d190 msgr2=0x7f49100d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f491015d190 0x7f49100d2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 -- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491015d610 msgr2=0x7f49100d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.375+0000 7f490de00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491015d610 0x7f49100d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f490de00640 1 -- 172.21.15.46:0/4221287285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100cc180 con 0x7f48f4003f20 2024-04-26T23:48:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f490de00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f490000d820 tx=0x7f490000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000de10 con 0x7f48f4003f20 2024-04-26T23:48:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4900005280 con 0x7f48f4003f20 2024-04-26T23:48:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900007030 con 0x7f48f4003f20 2024-04-26T23:48:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49100cc440 con 0x7f48f4003f20 2024-04-26T23:48:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.376+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49100cca00 con 0x7f48f4003f20 2024-04-26T23:48:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.378+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49100693b0 con 0x7f48f4003f20 2024-04-26T23:48:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.379+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f490001a020 con 0x7f48f4003f20 2024-04-26T23:48:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.380+0000 7f4906000640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 0x7f48ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.380+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f490008b640 con 0x7f48f4003f20 2024-04-26T23:48:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.380+0000 7f4907e00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 0x7f48ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.382+0000 7f4907e00640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 0x7f48ec06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48f8033000 tx=0x7f48f8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.385+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4900059840 con 0x7f48f4003f20 2024-04-26T23:48:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:47.634+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f49100d25b0 con 0x7f48f4003f20 2024-04-26T23:48:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.135+0000 7f4906000640 1 -- 172.21.15.46:0/4221287285 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f4900059140 con 0x7f48f4003f20 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 msgr2=0x7f48ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 0x7f48ec06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48f8033000 tx=0x7f48f8008030 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 msgr2=0x7f4910154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f490000d820 tx=0x7f490000dce0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 shutdown_connections 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48ec06c8f0 0x7f48ec06edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491015d610 0x7f49100d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f491015d190 0x7f49100d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 --2- 172.21.15.46:0/4221287285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f4003f20 0x7f4910154310 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 >> 172.21.15.46:0/4221287285 conn(0x7f491009e2a0 msgr2=0x7f491006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:48.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 shutdown_connections 2024-04-26T23:48:48.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.139+0000 7f490f200640 1 -- 172.21.15.46:0/4221287285 wait complete. 2024-04-26T23:48:48.167 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:48:48.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:48.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.380+0000 7f41b4000640 1 Processor -- start 2024-04-26T23:48:48.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.380+0000 7f41b4000640 1 -- start start 2024-04-26T23:48:48.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b4000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41ac165ee0 0x7f41ac1682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b4000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ac161940 con 0x7f41ac06e510 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b4000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ac071f80 con 0x7f41ac160bd0 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b4000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f41ac0742d0 con 0x7f41ac165ee0 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b1600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f41ac06e510 0x7f41ac06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57200/0 (socket says 172.21.15.46:57200) 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b1600640 1 -- 172.21.15.46:0/1399946034 learned_addr learned my addr 172.21.15.46:0/1399946034 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.382+0000 7f41b2000640 1 --2- 172.21.15.46:0/1399946034 >> v2:172.21.15.46:3300/0 conn(0x7f41ac165ee0 0x7f41ac1682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.383+0000 7f41abe00640 1 -- 172.21.15.46:0/1399946034 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1715078462 0 0) 0x7f41ac161940 con 0x7f41ac06e510 2024-04-26T23:48:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.383+0000 7f41abe00640 1 -- 172.21.15.46:0/1399946034 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4194003520 con 0x7f41ac06e510 2024-04-26T23:48:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.383+0000 7f41b2000640 1 -- 172.21.15.46:0/1399946034 >> v1:172.21.15.142:6789/0 conn(0x7f41ac160bd0 legacy=0x7f41ac161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.383+0000 7f41b2000640 1 -- 172.21.15.46:0/1399946034 >> v1:172.21.15.46:6789/0 conn(0x7f41ac06e510 legacy=0x7f41ac06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.383+0000 7f41b2000640 1 -- 172.21.15.46:0/1399946034 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac072260 con 0x7f41ac165ee0 2024-04-26T23:48:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.384+0000 7f41b2000640 1 --2- 172.21.15.46:0/1399946034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac1682d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009dc0 tx=0x7f41a402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a50270f9f28d745 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.384+0000 7f41abe00640 1 -- 172.21.15.46:0/1399946034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4007740 con 0x7f41ac165ee0 2024-04-26T23:48:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.384+0000 7f41abe00640 1 -- 172.21.15.46:0/1399946034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a40078e0 con 0x7f41ac165ee0 2024-04-26T23:48:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.384+0000 7f41abe00640 1 -- 172.21.15.46:0/1399946034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4007c90 con 0x7f41ac165ee0 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 -- 172.21.15.46:0/1399946034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 msgr2=0x7f41ac1682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 --2- 172.21.15.46:0/1399946034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac1682d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009dc0 tx=0x7f41a402a730 comp rx=0 tx=0).stop 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 -- 172.21.15.46:0/1399946034 shutdown_connections 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 --2- 172.21.15.46:0/1399946034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac1682d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 -- 172.21.15.46:0/1399946034 >> 172.21.15.46:0/1399946034 conn(0x7f41ac06c530 msgr2=0x7f41ac06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 -- 172.21.15.46:0/1399946034 shutdown_connections 2024-04-26T23:48:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.385+0000 7f41b4000640 1 -- 172.21.15.46:0/1399946034 wait complete. 2024-04-26T23:48:48.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.386+0000 7f41b4000640 1 Processor -- start 2024-04-26T23:48:48.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.387+0000 7f41b4000640 1 -- start start 2024-04-26T23:48:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.387+0000 7f41b4000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 0x7f41ac0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.387+0000 7f41b4000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 0x7f41ac158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac072d40 con 0x7f41ac165ee0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b4000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac072820 con 0x7f41ac160bd0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b4000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41ac072ec0 con 0x7f41ac06e510 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41520/0 (socket says 172.21.15.46:41520) 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 -- 172.21.15.46:0/1213367567 learned_addr learned my addr 172.21.15.46:0/1213367567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b1600640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 0x7f41ac0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 -- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 msgr2=0x7f41ac0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b0c00640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 0x7f41ac158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 0x7f41ac0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 -- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 msgr2=0x7f41ac158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 0x7f41ac158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.388+0000 7f41b2000640 1 -- 172.21.15.46:0/1213367567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac146070 con 0x7f41ac165ee0 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41b1600640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 0x7f41ac0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41b0c00640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 0x7f41ac158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41b2000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f41a402ad20 tx=0x7f41a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a403f070 con 0x7f41ac165ee0 2024-04-26T23:48:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a4031c40 con 0x7f41ac165ee0 2024-04-26T23:48:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac0b2e80 con 0x7f41ac165ee0 2024-04-26T23:48:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4009450 con 0x7f41ac165ee0 2024-04-26T23:48:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.389+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac0b23a0 con 0x7f41ac165ee0 2024-04-26T23:48:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.391+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac069530 con 0x7f41ac165ee0 2024-04-26T23:48:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.392+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41a4003990 con 0x7f41ac165ee0 2024-04-26T23:48:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.392+0000 7f41a9600640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.393+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f41a40acd60 con 0x7f41ac165ee0 2024-04-26T23:48:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.393+0000 7f41b1600640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.394+0000 7f41b1600640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 0x7f418c06edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41a00011e0 tx=0x7f41a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.397+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41a407b000 con 0x7f41ac165ee0 2024-04-26T23:48:48.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.643+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41ac06cbf0 con 0x7f41ac165ee0 2024-04-26T23:48:48.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.644+0000 7f41a9600640 1 -- 172.21.15.46:0/1213367567 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7f41a407a900 con 0x7f41ac165ee0 2024-04-26T23:48:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:48.649 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:48:48.649 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 msgr2=0x7f418c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 0x7f418c06edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41a00011e0 tx=0x7f41a0009040 comp rx=0 tx=0).stop 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 msgr2=0x7f41ac13f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f41a402ad20 tx=0x7f41a4004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 shutdown_connections 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ac165ee0 0x7f41ac13f350 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41ac160bd0 0x7f41ac158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 --2- 172.21.15.46:0/1213367567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41ac06e510 0x7f41ac0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 >> 172.21.15.46:0/1213367567 conn(0x7f41ac06c530 msgr2=0x7f41ac0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 shutdown_connections 2024-04-26T23:48:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.647+0000 7f41b4000640 1 -- 172.21.15.46:0/1213367567 wait complete. 2024-04-26T23:48:48.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:48:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.883+0000 7fe2b7000640 1 Processor -- start 2024-04-26T23:48:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.884+0000 7fe2b7000640 1 -- start start 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b7000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2b00b4000 0x7fe2b00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b7000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b00d11f0 con 0x7fe2b015f4e0 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b7000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b00d5f20 con 0x7fe2b00d2130 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b7000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b00d0ce0 con 0x7fe2b00b4000 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe2b015f4e0 0x7fe2b00d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57202/0 (socket says 172.21.15.46:57202) 2024-04-26T23:48:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.885+0000 7fe2b5c00640 1 -- 172.21.15.46:0/1777950956 learned_addr learned my addr 172.21.15.46:0/1777950956 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.886+0000 7fe2b6600640 1 --2- 172.21.15.46:0/1777950956 >> v2:172.21.15.142:3300/0 conn(0x7fe2b00b4000 0x7fe2b00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.886+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531521297 0 0) 0x7fe2b00d11f0 con 0x7fe2b015f4e0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.886+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe298003520 con 0x7fe2b015f4e0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.886+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2781094252 0 0) 0x7fe298003520 con 0x7fe2b015f4e0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 >> v2:172.21.15.142:3300/0 conn(0x7fe2b00b4000 msgr2=0x7fe2b00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 --2- 172.21.15.46:0/1777950956 >> v2:172.21.15.142:3300/0 conn(0x7fe2b00b4000 0x7fe2b00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 >> v1:172.21.15.142:6789/0 conn(0x7fe2b00d2130 legacy=0x7fe2b00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00d0f10 con 0x7fe2b015f4e0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe2a4001670 con 0x7fe2b015f4e0 2024-04-26T23:48:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 >> v1:172.21.15.46:6789/0 conn(0x7fe2b015f4e0 legacy=0x7fe2b00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe298003a30 0x7fe298003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe29800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe29800c0c0 con 0x7fe2980051c0 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe29800c270 con 0x7fe298004800 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe29800c470 con 0x7fe298003a30 2024-04-26T23:48:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.887+0000 7fe2b6600640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe29800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b5c00640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe298003a30 0x7fe298003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b5200640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b6600640 1 -- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe298003a30 msgr2=0x7fe298003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b6600640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe298003a30 0x7fe298003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b6600640 1 -- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe298004800 msgr2=0x7fe298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b6600640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b6600640 1 -- 172.21.15.46:0/1777950956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298003520 con 0x7fe2980051c0 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.888+0000 7fe2b5200640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.889+0000 7fe2b6600640 1 -- 172.21.15.46:0/1777950956 reap_dead start 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.889+0000 7fe2b6600640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe29800b9c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fc0 tx=0x7fe298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27af215a96b19faa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.889+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8035020 con 0x7fe2980051c0 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.889+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a8003bd0 con 0x7fe2980051c0 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2afe00640 1 -- 172.21.15.46:0/1777950956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8009c10 con 0x7fe2980051c0 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 -- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 msgr2=0x7fe29800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe29800b9c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fc0 tx=0x7fe298005830 comp rx=0 tx=0).stop 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 -- 172.21.15.46:0/1777950956 shutdown_connections 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 --2- 172.21.15.46:0/1777950956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe29800b9c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 -- 172.21.15.46:0/1777950956 >> 172.21.15.46:0/1777950956 conn(0x7fe2b00d5010 msgr2=0x7fe2b00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 -- 172.21.15.46:0/1777950956 shutdown_connections 2024-04-26T23:48:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.890+0000 7fe2b7000640 1 -- 172.21.15.46:0/1777950956 wait complete. 2024-04-26T23:48:48.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.892+0000 7fe2b7000640 1 Processor -- start 2024-04-26T23:48:48.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.892+0000 7fe2b7000640 1 -- start start 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 0x7fe2b01523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b0161b60 0x7fe2b0161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00b1d60 con 0x7fe2980051c0 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00d2990 con 0x7fe2b0151f50 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.893+0000 7fe2b7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b015f8c0 con 0x7fe2b0161b60 2024-04-26T23:48:48.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41544/0 (socket says 172.21.15.46:41544) 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 -- 172.21.15.46:0/3529794835 learned_addr learned my addr 172.21.15.46:0/3529794835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5200640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 0x7fe2b01523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 -- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b0161b60 msgr2=0x7fe2b0161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b0161b60 0x7fe2b0161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 -- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 msgr2=0x7fe2b01523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 0x7fe2b01523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 -- 172.21.15.46:0/3529794835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00b7690 con 0x7fe2980051c0 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5200640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 0x7fe2b01523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.894+0000 7fe2b5c00640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400e730 tx=0x7fe2a400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.895+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400c6c0 con 0x7fe2980051c0 2024-04-26T23:48:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.895+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4018070 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.895+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400caa0 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.895+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b00b7970 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.895+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b0152910 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.896+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b00693b0 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.897+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2a400cc40 con 0x7fe2980051c0 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.898+0000 7fe2ad600640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.898+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fe2a408bae0 con 0x7fe2980051c0 2024-04-26T23:48:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.899+0000 7fe2b5200640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:48.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.900+0000 7fe2b5200640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c97a0 tx=0x7fe29c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:48.903+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2a4059da0 con 0x7fe2980051c0 2024-04-26T23:48:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.113+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b00d33d0 con 0x7fe2980051c0 2024-04-26T23:48:49.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.115+0000 7fe2ad600640 1 -- 172.21.15.46:0/3529794835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16916 (secure 0 0 0) 0x7fe2a40596a0 con 0x7fe2980051c0 2024-04-26T23:48:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:49.120 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:46.084255+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 msgr2=0x7fe29006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c97a0 tx=0x7fe29c009040 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 msgr2=0x7fe2b00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400e730 tx=0x7fe2a400ebf0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 shutdown_connections 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b0161b60 0x7fe2b0161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b0151f50 0x7fe2b01523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 --2- 172.21.15.46:0/3529794835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2980051c0 0x7fe2b00c9e60 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 >> 172.21.15.46:0/3529794835 conn(0x7fe2b00d5010 msgr2=0x7fe2b00b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 shutdown_connections 2024-04-26T23:48:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.118+0000 7fe2b7000640 1 -- 172.21.15.46:0/3529794835 wait complete. 2024-04-26T23:48:49.143 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:48:49.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:48:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.354+0000 7f6810e00640 1 Processor -- start 2024-04-26T23:48:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.354+0000 7f6810e00640 1 -- start start 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.355+0000 7f6810e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f680c1466b0 0x7f680c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.355+0000 7f6810e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680c073ad0 con 0x7f680c160910 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.355+0000 7f6810e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680c146a90 con 0x7f680c0720e0 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.355+0000 7f6810e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f680c073d70 con 0x7f680c1466b0 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.356+0000 7f680be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f680c160910 0x7f680c0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57214/0 (socket says 172.21.15.46:57214) 2024-04-26T23:48:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.356+0000 7f680be00640 1 -- 172.21.15.46:0/1407812547 learned_addr learned my addr 172.21.15.46:0/1407812547 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.356+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950433731 0 0) 0x7f680c073ad0 con 0x7f680c160910 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.356+0000 7f680b400640 1 --2- 172.21.15.46:0/1407812547 >> v2:172.21.15.142:3300/0 conn(0x7f680c1466b0 0x7f680c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.356+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67f0003520 con 0x7f680c160910 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2820101128 0 0) 0x7f67f0003520 con 0x7f680c160910 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 >> v2:172.21.15.142:3300/0 conn(0x7f680c1466b0 msgr2=0x7f680c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 --2- 172.21.15.46:0/1407812547 >> v2:172.21.15.142:3300/0 conn(0x7f680c1466b0 0x7f680c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 >> v1:172.21.15.142:6789/0 conn(0x7f680c0720e0 legacy=0x7f680c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c074050 con 0x7f680c160910 2024-04-26T23:48:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6804001670 con 0x7f680c160910 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 >> v1:172.21.15.46:6789/0 conn(0x7f680c160910 legacy=0x7f680c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 0x7f67f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67f0004800 0x7f67f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00051c0 0x7f67f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f680c073d70 con 0x7f67f00039b0 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67f000c060 con 0x7f67f00051c0 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67f000c260 con 0x7f67f0004800 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 0x7f67f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 -- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67f0004800 msgr2=0x7f67f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67f0004800 0x7f67f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 -- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00051c0 msgr2=0x7f67f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00051c0 0x7f67f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680b400640 1 -- 172.21.15.46:0/1407812547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f0003520 con 0x7f67f00039b0 2024-04-26T23:48:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.357+0000 7f680be00640 1 -- 172.21.15.46:0/1407812547 reap_dead start 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f680b400640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 0x7f67f0003dd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009ac0 tx=0x7f67f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb31b9088e44ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc036070 con 0x7f67f00039b0 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc031400 con 0x7f67f00039b0 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f680aa00640 1 -- 172.21.15.46:0/1407812547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc009690 con 0x7f67f00039b0 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f6810e00640 1 -- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 msgr2=0x7f67f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f6810e00640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 0x7f67f0003dd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009ac0 tx=0x7f67f0005830 comp rx=0 tx=0).stop 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f6810e00640 1 -- 172.21.15.46:0/1407812547 shutdown_connections 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f6810e00640 1 --2- 172.21.15.46:0/1407812547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00039b0 0x7f67f0003dd0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.359+0000 7f6810e00640 1 -- 172.21.15.46:0/1407812547 >> 172.21.15.46:0/1407812547 conn(0x7f680c0b26d0 msgr2=0x7f680c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.360+0000 7f6810e00640 1 -- 172.21.15.46:0/1407812547 shutdown_connections 2024-04-26T23:48:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.360+0000 7f6810e00640 1 -- 172.21.15.46:0/1407812547 wait complete. 2024-04-26T23:48:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.361+0000 7f6810e00640 1 Processor -- start 2024-04-26T23:48:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.362+0000 7f6810e00640 1 -- start start 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 0x7f680c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 0x7f680c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f680c160d60 con 0x7f680c144500 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f680c0cb470 con 0x7f67f00039b0 2024-04-26T23:48:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6810e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f680c160ee0 con 0x7f680c13c990 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41562/0 (socket says 172.21.15.46:41562) 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 -- 172.21.15.46:0/3505795898 learned_addr learned my addr 172.21.15.46:0/3505795898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680b400640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 0x7f680c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6802a00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 0x7f680c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 -- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 msgr2=0x7f680c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 0x7f680c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 -- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 msgr2=0x7f680c0b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 0x7f680c0b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680be00640 1 -- 172.21.15.46:0/3505795898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c15fff0 con 0x7f680c144500 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f680b400640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 0x7f680c0b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.363+0000 7f6802a00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 0x7f680c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f680be00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f680c160cf0 tx=0x7f680400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804004cb0 con 0x7f680c144500 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6804007170 con 0x7f680c144500 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804007560 con 0x7f680c144500 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c13ce10 con 0x7f680c144500 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.364+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c13d3d0 con 0x7f680c144500 2024-04-26T23:48:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.366+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c0693b0 con 0x7f680c144500 2024-04-26T23:48:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.367+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f680401c3f0 con 0x7f680c144500 2024-04-26T23:48:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.368+0000 7f6803e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 0x7f67e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.368+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f680408b370 con 0x7f680c144500 2024-04-26T23:48:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.368+0000 7f680b400640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 0x7f67e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.370+0000 7f680b400640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 0x7f67e806edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f67fc033000 tx=0x7f67fc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.372+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68040596a0 con 0x7f680c144500 2024-04-26T23:48:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.574+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f680c0c7180 con 0x7f67e806c8f0 2024-04-26T23:48:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.595+0000 7f6803e00640 1 -- 172.21.15.46:0/3505795898 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252422 (secure 0 0 0) 0x7f680c0c7180 con 0x7f67e806c8f0 2024-04-26T23:48:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:49.607 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 msgr2=0x7f67e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 0x7f67e806edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f67fc033000 tx=0x7f67fc008030 comp rx=0 tx=0).stop 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 msgr2=0x7f680c1449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f680c160cf0 tx=0x7f680400c890 comp rx=0 tx=0).stop 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 shutdown_connections 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f67e806c8f0 0x7f67e806edb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f680c144500 0x7f680c1449b0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c13c990 0x7f680c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.601+0000 7f6810e00640 1 --2- 172.21.15.46:0/3505795898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f00039b0 0x7f680c0b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.602+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 >> 172.21.15.46:0/3505795898 conn(0x7f680c0b26d0 msgr2=0x7f680c0cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:49.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.602+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 shutdown_connections 2024-04-26T23:48:49.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.602+0000 7f6810e00640 1 -- 172.21.15.46:0/3505795898 wait complete. 2024-04-26T23:48:49.628 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-04-26T23:48:48.149025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":397496,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754577224,"statfs":{"total":773094113280,"available":772687077376,"internally_reserved":0,"allocated":13705216,"data_stored":7037636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":393203249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.088883"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119240+0000","last_change":"2024-04-26T23:48:45.116061+0000","last_active":"2024-04-26T23:48:45.119240+0000","last_peered":"2024-04-26T23:48:45.119240+0000","last_clean":"2024-04-26T23:48:45.119240+0000","last_became_active":"2024-04-26T23:48:45.115774+0000","last_became_peered":"2024-04-26T23:48:45.115774+0000","last_unstale":"2024-04-26T23:48:45.119240+0000","last_undegraded":"2024-04-26T23:48:45.119240+0000","last_fullsized":"2024-04-26T23:48:45.119240+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:42.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.135511+0000","last_change":"2024-04-26T23:48:45.133919+0000","last_active":"2024-04-26T23:48:45.135511+0000","last_peered":"2024-04-26T23:48:45.135511+0000","last_clean":"2024-04-26T23:48:45.135511+0000","last_became_active":"2024-04-26T23:48:45.133520+0000","last_became_peered":"2024-04-26T23:48:45.133520+0000","last_unstale":"2024-04-26T23:48:45.135511+0000","last_undegraded":"2024-04-26T23:48:45.135511+0000","last_fullsized":"2024-04-26T23:48:45.135511+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:04.244263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131634+0000","last_change":"2024-04-26T23:48:45.128270+0000","last_active":"2024-04-26T23:48:45.131634+0000","last_peered":"2024-04-26T23:48:45.131634+0000","last_clean":"2024-04-26T23:48:45.131634+0000","last_became_active":"2024-04-26T23:48:45.127877+0000","last_became_peered":"2024-04-26T23:48:45.127877+0000","last_unstale":"2024-04-26T23:48:45.131634+0000","last_undegraded":"2024-04-26T23:48:45.131634+0000","last_fullsized":"2024-04-26T23:48:45.131634+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131153+0000","last_change":"2024-04-26T23:48:45.128503+0000","last_active":"2024-04-26T23:48:45.131153+0000","last_peered":"2024-04-26T23:48:45.131153+0000","last_clean":"2024-04-26T23:48:45.131153+0000","last_became_active":"2024-04-26T23:48:45.128169+0000","last_became_peered":"2024-04-26T23:48:45.128169+0000","last_unstale":"2024-04-26T23:48:45.131153+0000","last_undegraded":"2024-04-26T23:48:45.131153+0000","last_fullsized":"2024-04-26T23:48:45.131153+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.204584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125230+0000","last_change":"2024-04-26T23:48:45.119109+0000","last_active":"2024-04-26T23:48:45.125230+0000","last_peered":"2024-04-26T23:48:45.125230+0000","last_clean":"2024-04-26T23:48:45.125230+0000","last_became_active":"2024-04-26T23:48:45.118837+0000","last_became_peered":"2024-04-26T23:48:45.118837+0000","last_unstale":"2024-04-26T23:48:45.125230+0000","last_undegraded":"2024-04-26T23:48:45.125230+0000","last_fullsized":"2024-04-26T23:48:45.125230+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:36.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128739+0000","last_change":"2024-04-26T23:48:45.126961+0000","last_active":"2024-04-26T23:48:45.128739+0000","last_peered":"2024-04-26T23:48:45.128739+0000","last_clean":"2024-04-26T23:48:45.128739+0000","last_became_active":"2024-04-26T23:48:45.126325+0000","last_became_peered":"2024-04-26T23:48:45.126325+0000","last_unstale":"2024-04-26T23:48:45.128739+0000","last_undegraded":"2024-04-26T23:48:45.128739+0000","last_fullsized":"2024-04-26T23:48:45.128739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:55.318114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120920+0000","last_change":"2024-04-26T23:48:45.117765+0000","last_active":"2024-04-26T23:48:45.120920+0000","last_peered":"2024-04-26T23:48:45.120920+0000","last_clean":"2024-04-26T23:48:45.120920+0000","last_became_active":"2024-04-26T23:48:45.116761+0000","last_became_peered":"2024-04-26T23:48:45.116761+0000","last_unstale":"2024-04-26T23:48:45.120920+0000","last_undegraded":"2024-04-26T23:48:45.120920+0000","last_fullsized":"2024-04-26T23:48:45.120920+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:56.366552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133838+0000","last_change":"2024-04-26T23:48:45.131995+0000","last_active":"2024-04-26T23:48:45.133838+0000","last_peered":"2024-04-26T23:48:45.133838+0000","last_clean":"2024-04-26T23:48:45.133838+0000","last_became_active":"2024-04-26T23:48:45.131664+0000","last_became_peered":"2024-04-26T23:48:45.131664+0000","last_unstale":"2024-04-26T23:48:45.133838+0000","last_undegraded":"2024-04-26T23:48:45.133838+0000","last_fullsized":"2024-04-26T23:48:45.133838+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:23.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131388+0000","last_change":"2024-04-26T23:48:45.126543+0000","last_active":"2024-04-26T23:48:45.131388+0000","last_peered":"2024-04-26T23:48:45.131388+0000","last_clean":"2024-04-26T23:48:45.131388+0000","last_became_active":"2024-04-26T23:48:45.126193+0000","last_became_peered":"2024-04-26T23:48:45.126193+0000","last_unstale":"2024-04-26T23:48:45.131388+0000","last_undegraded":"2024-04-26T23:48:45.131388+0000","last_fullsized":"2024-04-26T23:48:45.131388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:31.436439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123845+0000","last_change":"2024-04-26T23:48:45.122839+0000","last_active":"2024-04-26T23:48:45.123845+0000","last_peered":"2024-04-26T23:48:45.123845+0000","last_clean":"2024-04-26T23:48:45.123845+0000","last_became_active":"2024-04-26T23:48:45.121806+0000","last_became_peered":"2024-04-26T23:48:45.121806+0000","last_unstale":"2024-04-26T23:48:45.123845+0000","last_undegraded":"2024-04-26T23:48:45.123845+0000","last_fullsized":"2024-04-26T23:48:45.123845+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:51.361411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.122474+0000","last_change":"2024-04-26T23:48:45.115829+0000","last_active":"2024-04-26T23:48:45.122474+0000","last_peered":"2024-04-26T23:48:45.122474+0000","last_clean":"2024-04-26T23:48:45.122474+0000","last_became_active":"2024-04-26T23:48:45.115536+0000","last_became_peered":"2024-04-26T23:48:45.115536+0000","last_unstale":"2024-04-26T23:48:45.122474+0000","last_undegraded":"2024-04-26T23:48:45.122474+0000","last_fullsized":"2024-04-26T23:48:45.122474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:51.361411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120728+0000","last_change":"2024-04-26T23:48:45.116511+0000","last_active":"2024-04-26T23:48:45.120728+0000","last_peered":"2024-04-26T23:48:45.120728+0000","last_clean":"2024-04-26T23:48:45.120728+0000","last_became_active":"2024-04-26T23:48:45.115910+0000","last_became_peered":"2024-04-26T23:48:45.115910+0000","last_unstale":"2024-04-26T23:48:45.120728+0000","last_undegraded":"2024-04-26T23:48:45.120728+0000","last_fullsized":"2024-04-26T23:48:45.120728+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.780714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.135123+0000","last_change":"2024-04-26T23:48:45.134014+0000","last_active":"2024-04-26T23:48:45.135123+0000","last_peered":"2024-04-26T23:48:45.135123+0000","last_clean":"2024-04-26T23:48:45.135123+0000","last_became_active":"2024-04-26T23:48:45.133703+0000","last_became_peered":"2024-04-26T23:48:45.133703+0000","last_unstale":"2024-04-26T23:48:45.135123+0000","last_undegraded":"2024-04-26T23:48:45.135123+0000","last_fullsized":"2024-04-26T23:48:45.135123+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:28.625388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.134015+0000","last_change":"2024-04-26T23:48:45.132668+0000","last_active":"2024-04-26T23:48:45.134015+0000","last_peered":"2024-04-26T23:48:45.134015+0000","last_clean":"2024-04-26T23:48:45.134015+0000","last_became_active":"2024-04-26T23:48:45.131256+0000","last_became_peered":"2024-04-26T23:48:45.131256+0000","last_unstale":"2024-04-26T23:48:45.134015+0000","last_undegraded":"2024-04-26T23:48:45.134015+0000","last_fullsized":"2024-04-26T23:48:45.134015+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:39.101779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.126557+0000","last_change":"2024-04-26T23:48:45.123018+0000","last_active":"2024-04-26T23:48:45.126557+0000","last_peered":"2024-04-26T23:48:45.126557+0000","last_clean":"2024-04-26T23:48:45.126557+0000","last_became_active":"2024-04-26T23:48:45.122406+0000","last_became_peered":"2024-04-26T23:48:45.122406+0000","last_unstale":"2024-04-26T23:48:45.126557+0000","last_undegraded":"2024-04-26T23:48:45.126557+0000","last_fullsized":"2024-04-26T23:48:45.126557+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:28.625388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.103533+0000","last_change":"2024-04-26T23:48:45.100885+0000","last_active":"2024-04-26T23:48:45.103533+0000","last_peered":"2024-04-26T23:48:45.103533+0000","last_clean":"2024-04-26T23:48:45.103533+0000","last_became_active":"2024-04-26T23:48:45.100149+0000","last_became_peered":"2024-04-26T23:48:45.100149+0000","last_unstale":"2024-04-26T23:48:45.103533+0000","last_undegraded":"2024-04-26T23:48:45.103533+0000","last_fullsized":"2024-04-26T23:48:45.103533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:22.449163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133971+0000","last_change":"2024-04-26T23:48:45.132129+0000","last_active":"2024-04-26T23:48:45.133971+0000","last_peered":"2024-04-26T23:48:45.133971+0000","last_clean":"2024-04-26T23:48:45.133971+0000","last_became_active":"2024-04-26T23:48:45.131756+0000","last_became_peered":"2024-04-26T23:48:45.131756+0000","last_unstale":"2024-04-26T23:48:45.133971+0000","last_undegraded":"2024-04-26T23:48:45.133971+0000","last_fullsized":"2024-04-26T23:48:45.133971+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:36.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133354+0000","last_change":"2024-04-26T23:48:45.131772+0000","last_active":"2024-04-26T23:48:45.133354+0000","last_peered":"2024-04-26T23:48:45.133354+0000","last_clean":"2024-04-26T23:48:45.133354+0000","last_became_active":"2024-04-26T23:48:45.130614+0000","last_became_peered":"2024-04-26T23:48:45.130614+0000","last_unstale":"2024-04-26T23:48:45.133354+0000","last_undegraded":"2024-04-26T23:48:45.133354+0000","last_fullsized":"2024-04-26T23:48:45.133354+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:31.436439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094101+0000","last_change":"2024-04-26T23:48:45.123449+0000","last_active":"2024-04-26T23:48:46.094101+0000","last_peered":"2024-04-26T23:48:46.094101+0000","last_clean":"2024-04-26T23:48:46.094101+0000","last_became_active":"2024-04-26T23:48:45.121815+0000","last_became_peered":"2024-04-26T23:48:45.121815+0000","last_unstale":"2024-04-26T23:48:46.094101+0000","last_undegraded":"2024-04-26T23:48:46.094101+0000","last_fullsized":"2024-04-26T23:48:46.094101+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:24.051850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115116+0000","last_change":"2024-04-26T23:48:45.113819+0000","last_active":"2024-04-26T23:48:45.115116+0000","last_peered":"2024-04-26T23:48:45.115116+0000","last_clean":"2024-04-26T23:48:45.115116+0000","last_became_active":"2024-04-26T23:48:45.113391+0000","last_became_peered":"2024-04-26T23:48:45.113391+0000","last_unstale":"2024-04-26T23:48:45.115116+0000","last_undegraded":"2024-04-26T23:48:45.115116+0000","last_fullsized":"2024-04-26T23:48:45.115116+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:24.209917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125911+0000","last_change":"2024-04-26T23:48:45.120284+0000","last_active":"2024-04-26T23:48:45.125911+0000","last_peered":"2024-04-26T23:48:45.125911+0000","last_clean":"2024-04-26T23:48:45.125911+0000","last_became_active":"2024-04-26T23:48:45.119682+0000","last_became_peered":"2024-04-26T23:48:45.119682+0000","last_unstale":"2024-04-26T23:48:45.125911+0000","last_undegraded":"2024-04-26T23:48:45.125911+0000","last_fullsized":"2024-04-26T23:48:45.125911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.130165+0000","last_change":"2024-04-26T23:48:45.128798+0000","last_active":"2024-04-26T23:48:45.130165+0000","last_peered":"2024-04-26T23:48:45.130165+0000","last_clean":"2024-04-26T23:48:45.130165+0000","last_became_active":"2024-04-26T23:48:45.128321+0000","last_became_peered":"2024-04-26T23:48:45.128321+0000","last_unstale":"2024-04-26T23:48:45.130165+0000","last_undegraded":"2024-04-26T23:48:45.130165+0000","last_fullsized":"2024-04-26T23:48:45.130165+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:16.431506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131349+0000","last_change":"2024-04-26T23:48:45.127189+0000","last_active":"2024-04-26T23:48:45.131349+0000","last_peered":"2024-04-26T23:48:45.131349+0000","last_clean":"2024-04-26T23:48:45.131349+0000","last_became_active":"2024-04-26T23:48:45.126885+0000","last_became_peered":"2024-04-26T23:48:45.126885+0000","last_unstale":"2024-04-26T23:48:45.131349+0000","last_undegraded":"2024-04-26T23:48:45.131349+0000","last_fullsized":"2024-04-26T23:48:45.131349+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:56.179293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133446+0000","last_change":"2024-04-26T23:48:45.129210+0000","last_active":"2024-04-26T23:48:45.133446+0000","last_peered":"2024-04-26T23:48:45.133446+0000","last_clean":"2024-04-26T23:48:45.133446+0000","last_became_active":"2024-04-26T23:48:45.128933+0000","last_became_peered":"2024-04-26T23:48:45.128933+0000","last_unstale":"2024-04-26T23:48:45.133446+0000","last_undegraded":"2024-04-26T23:48:45.133446+0000","last_fullsized":"2024-04-26T23:48:45.133446+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128457+0000","last_change":"2024-04-26T23:48:45.127404+0000","last_active":"2024-04-26T23:48:45.128457+0000","last_peered":"2024-04-26T23:48:45.128457+0000","last_clean":"2024-04-26T23:48:45.128457+0000","last_became_active":"2024-04-26T23:48:45.127075+0000","last_became_peered":"2024-04-26T23:48:45.127075+0000","last_unstale":"2024-04-26T23:48:45.128457+0000","last_undegraded":"2024-04-26T23:48:45.128457+0000","last_fullsized":"2024-04-26T23:48:45.128457+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:28.790149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115809+0000","last_change":"2024-04-26T23:48:45.114742+0000","last_active":"2024-04-26T23:48:45.115809+0000","last_peered":"2024-04-26T23:48:45.115809+0000","last_clean":"2024-04-26T23:48:45.115809+0000","last_became_active":"2024-04-26T23:48:45.114405+0000","last_became_peered":"2024-04-26T23:48:45.114405+0000","last_unstale":"2024-04-26T23:48:45.115809+0000","last_undegraded":"2024-04-26T23:48:45.115809+0000","last_fullsized":"2024-04-26T23:48:45.115809+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:06.699237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.096226+0000","last_change":"2024-04-26T23:48:45.124539+0000","last_active":"2024-04-26T23:48:46.096226+0000","last_peered":"2024-04-26T23:48:46.096226+0000","last_clean":"2024-04-26T23:48:46.096226+0000","last_became_active":"2024-04-26T23:48:45.121843+0000","last_became_peered":"2024-04-26T23:48:45.121843+0000","last_unstale":"2024-04-26T23:48:46.096226+0000","last_undegraded":"2024-04-26T23:48:46.096226+0000","last_fullsized":"2024-04-26T23:48:46.096226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:39.495048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.110952+0000","last_change":"2024-04-26T23:48:45.109679+0000","last_active":"2024-04-26T23:48:45.110952+0000","last_peered":"2024-04-26T23:48:45.110952+0000","last_clean":"2024-04-26T23:48:45.110952+0000","last_became_active":"2024-04-26T23:48:45.109384+0000","last_became_peered":"2024-04-26T23:48:45.109384+0000","last_unstale":"2024-04-26T23:48:45.110952+0000","last_undegraded":"2024-04-26T23:48:45.110952+0000","last_fullsized":"2024-04-26T23:48:45.110952+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:34.573168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.118406+0000","last_change":"2024-04-26T23:48:45.117555+0000","last_active":"2024-04-26T23:48:45.118406+0000","last_peered":"2024-04-26T23:48:45.118406+0000","last_clean":"2024-04-26T23:48:45.118406+0000","last_became_active":"2024-04-26T23:48:45.117245+0000","last_became_peered":"2024-04-26T23:48:45.117245+0000","last_unstale":"2024-04-26T23:48:45.118406+0000","last_undegraded":"2024-04-26T23:48:45.118406+0000","last_fullsized":"2024-04-26T23:48:45.118406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:39.481730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131507+0000","last_change":"2024-04-26T23:48:45.126332+0000","last_active":"2024-04-26T23:48:45.131507+0000","last_peered":"2024-04-26T23:48:45.131507+0000","last_clean":"2024-04-26T23:48:45.131507+0000","last_became_active":"2024-04-26T23:48:45.126017+0000","last_became_peered":"2024-04-26T23:48:45.126017+0000","last_unstale":"2024-04-26T23:48:45.131507+0000","last_undegraded":"2024-04-26T23:48:45.131507+0000","last_fullsized":"2024-04-26T23:48:45.131507+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.130018+0000","last_change":"2024-04-26T23:48:45.119091+0000","last_active":"2024-04-26T23:48:45.130018+0000","last_peered":"2024-04-26T23:48:45.130018+0000","last_clean":"2024-04-26T23:48:45.130018+0000","last_became_active":"2024-04-26T23:48:45.117636+0000","last_became_peered":"2024-04-26T23:48:45.117636+0000","last_unstale":"2024-04-26T23:48:45.130018+0000","last_undegraded":"2024-04-26T23:48:45.130018+0000","last_fullsized":"2024-04-26T23:48:45.130018+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:05.007579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133427+0000","last_change":"2024-04-26T23:48:45.128678+0000","last_active":"2024-04-26T23:48:45.133427+0000","last_peered":"2024-04-26T23:48:45.133427+0000","last_clean":"2024-04-26T23:48:45.133427+0000","last_became_active":"2024-04-26T23:48:45.128372+0000","last_became_peered":"2024-04-26T23:48:45.128372+0000","last_unstale":"2024-04-26T23:48:45.133427+0000","last_undegraded":"2024-04-26T23:48:45.133427+0000","last_fullsized":"2024-04-26T23:48:45.133427+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127768+0000","last_change":"2024-04-26T23:48:45.123678+0000","last_active":"2024-04-26T23:48:45.127768+0000","last_peered":"2024-04-26T23:48:45.127768+0000","last_clean":"2024-04-26T23:48:45.127768+0000","last_became_active":"2024-04-26T23:48:45.121601+0000","last_became_peered":"2024-04-26T23:48:45.121601+0000","last_unstale":"2024-04-26T23:48:45.127768+0000","last_undegraded":"2024-04-26T23:48:45.127768+0000","last_fullsized":"2024-04-26T23:48:45.127768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:42.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094020+0000","last_change":"2024-04-26T23:48:44.096929+0000","last_active":"2024-04-26T23:48:46.094020+0000","last_peered":"2024-04-26T23:48:46.094020+0000","last_clean":"2024-04-26T23:48:46.094020+0000","last_became_active":"2024-04-26T23:48:44.096639+0000","last_became_peered":"2024-04-26T23:48:44.096639+0000","last_unstale":"2024-04-26T23:48:46.094020+0000","last_undegraded":"2024-04-26T23:48:46.094020+0000","last_fullsized":"2024-04-26T23:48:46.094020+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:50.345308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.121197+0000","last_change":"2024-04-26T23:48:45.117342+0000","last_active":"2024-04-26T23:48:45.121197+0000","last_peered":"2024-04-26T23:48:45.121197+0000","last_clean":"2024-04-26T23:48:45.121197+0000","last_became_active":"2024-04-26T23:48:45.117059+0000","last_became_peered":"2024-04-26T23:48:45.117059+0000","last_unstale":"2024-04-26T23:48:45.121197+0000","last_undegraded":"2024-04-26T23:48:45.121197+0000","last_fullsized":"2024-04-26T23:48:45.121197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:29.619699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143071+0000","last_change":"2024-04-26T23:48:44.096129+0000","last_active":"2024-04-26T23:48:45.143071+0000","last_peered":"2024-04-26T23:48:45.143071+0000","last_clean":"2024-04-26T23:48:45.143071+0000","last_became_active":"2024-04-26T23:48:44.095833+0000","last_became_peered":"2024-04-26T23:48:44.095833+0000","last_unstale":"2024-04-26T23:48:45.143071+0000","last_undegraded":"2024-04-26T23:48:45.143071+0000","last_fullsized":"2024-04-26T23:48:45.143071+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:44.078864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128240+0000","last_change":"2024-04-26T23:48:45.126351+0000","last_active":"2024-04-26T23:48:45.128240+0000","last_peered":"2024-04-26T23:48:45.128240+0000","last_clean":"2024-04-26T23:48:45.128240+0000","last_became_active":"2024-04-26T23:48:45.124414+0000","last_became_peered":"2024-04-26T23:48:45.124414+0000","last_unstale":"2024-04-26T23:48:45.128240+0000","last_undegraded":"2024-04-26T23:48:45.128240+0000","last_fullsized":"2024-04-26T23:48:45.128240+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:43.255995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142149+0000","last_change":"2024-04-26T23:48:44.096582+0000","last_active":"2024-04-26T23:48:45.142149+0000","last_peered":"2024-04-26T23:48:45.142149+0000","last_clean":"2024-04-26T23:48:45.142149+0000","last_became_active":"2024-04-26T23:48:44.095305+0000","last_became_peered":"2024-04-26T23:48:44.095305+0000","last_unstale":"2024-04-26T23:48:45.142149+0000","last_undegraded":"2024-04-26T23:48:45.142149+0000","last_fullsized":"2024-04-26T23:48:45.142149+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.077797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.126257+0000","last_change":"2024-04-26T23:48:45.121316+0000","last_active":"2024-04-26T23:48:45.126257+0000","last_peered":"2024-04-26T23:48:45.126257+0000","last_clean":"2024-04-26T23:48:45.126257+0000","last_became_active":"2024-04-26T23:48:45.120974+0000","last_became_peered":"2024-04-26T23:48:45.120974+0000","last_unstale":"2024-04-26T23:48:45.126257+0000","last_undegraded":"2024-04-26T23:48:45.126257+0000","last_fullsized":"2024-04-26T23:48:45.126257+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143827+0000","last_change":"2024-04-26T23:48:44.100844+0000","last_active":"2024-04-26T23:48:45.143827+0000","last_peered":"2024-04-26T23:48:45.143827+0000","last_clean":"2024-04-26T23:48:45.143827+0000","last_became_active":"2024-04-26T23:48:44.100211+0000","last_became_peered":"2024-04-26T23:48:44.100211+0000","last_unstale":"2024-04-26T23:48:45.143827+0000","last_undegraded":"2024-04-26T23:48:45.143827+0000","last_fullsized":"2024-04-26T23:48:45.143827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:42.826400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.134546+0000","last_change":"2024-04-26T23:48:45.133141+0000","last_active":"2024-04-26T23:48:45.134546+0000","last_peered":"2024-04-26T23:48:45.134546+0000","last_clean":"2024-04-26T23:48:45.134546+0000","last_became_active":"2024-04-26T23:48:45.132513+0000","last_became_peered":"2024-04-26T23:48:45.132513+0000","last_unstale":"2024-04-26T23:48:45.134546+0000","last_undegraded":"2024-04-26T23:48:45.134546+0000","last_fullsized":"2024-04-26T23:48:45.134546+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:39.060629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141198+0000","last_change":"2024-04-26T23:48:44.102016+0000","last_active":"2024-04-26T23:48:45.141198+0000","last_peered":"2024-04-26T23:48:45.141198+0000","last_clean":"2024-04-26T23:48:45.141198+0000","last_became_active":"2024-04-26T23:48:44.101707+0000","last_became_peered":"2024-04-26T23:48:44.101707+0000","last_unstale":"2024-04-26T23:48:45.141198+0000","last_undegraded":"2024-04-26T23:48:45.141198+0000","last_fullsized":"2024-04-26T23:48:45.141198+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:18.735614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119308+0000","last_change":"2024-04-26T23:48:45.116126+0000","last_active":"2024-04-26T23:48:45.119308+0000","last_peered":"2024-04-26T23:48:45.119308+0000","last_clean":"2024-04-26T23:48:45.119308+0000","last_became_active":"2024-04-26T23:48:45.115301+0000","last_became_peered":"2024-04-26T23:48:45.115301+0000","last_unstale":"2024-04-26T23:48:45.119308+0000","last_undegraded":"2024-04-26T23:48:45.119308+0000","last_fullsized":"2024-04-26T23:48:45.119308+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:18.380171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142707+0000","last_change":"2024-04-26T23:48:44.103078+0000","last_active":"2024-04-26T23:48:45.142707+0000","last_peered":"2024-04-26T23:48:45.142707+0000","last_clean":"2024-04-26T23:48:45.142707+0000","last_became_active":"2024-04-26T23:48:44.102810+0000","last_became_peered":"2024-04-26T23:48:44.102810+0000","last_unstale":"2024-04-26T23:48:45.142707+0000","last_undegraded":"2024-04-26T23:48:45.142707+0000","last_fullsized":"2024-04-26T23:48:45.142707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:43.136945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123533+0000","last_change":"2024-04-26T23:48:45.120244+0000","last_active":"2024-04-26T23:48:45.123533+0000","last_peered":"2024-04-26T23:48:45.123533+0000","last_clean":"2024-04-26T23:48:45.123533+0000","last_became_active":"2024-04-26T23:48:45.119896+0000","last_became_peered":"2024-04-26T23:48:45.119896+0000","last_unstale":"2024-04-26T23:48:45.123533+0000","last_undegraded":"2024-04-26T23:48:45.123533+0000","last_fullsized":"2024-04-26T23:48:45.123533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:21.464098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119017+0000","last_change":"2024-04-26T23:48:44.095056+0000","last_active":"2024-04-26T23:48:45.119017+0000","last_peered":"2024-04-26T23:48:45.119017+0000","last_clean":"2024-04-26T23:48:45.119017+0000","last_became_active":"2024-04-26T23:48:44.094770+0000","last_became_peered":"2024-04-26T23:48:44.094770+0000","last_unstale":"2024-04-26T23:48:45.119017+0000","last_undegraded":"2024-04-26T23:48:45.119017+0000","last_fullsized":"2024-04-26T23:48:45.119017+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120083+0000","last_change":"2024-04-26T23:48:45.117473+0000","last_active":"2024-04-26T23:48:45.120083+0000","last_peered":"2024-04-26T23:48:45.120083+0000","last_clean":"2024-04-26T23:48:45.120083+0000","last_became_active":"2024-04-26T23:48:45.117165+0000","last_became_peered":"2024-04-26T23:48:45.117165+0000","last_unstale":"2024-04-26T23:48:45.120083+0000","last_undegraded":"2024-04-26T23:48:45.120083+0000","last_fullsized":"2024-04-26T23:48:45.120083+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:19.751717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.093955+0000","last_change":"2024-04-26T23:48:44.090397+0000","last_active":"2024-04-26T23:48:45.093955+0000","last_peered":"2024-04-26T23:48:45.093955+0000","last_clean":"2024-04-26T23:48:45.093955+0000","last_became_active":"2024-04-26T23:48:44.089163+0000","last_became_peered":"2024-04-26T23:48:44.089163+0000","last_unstale":"2024-04-26T23:48:45.093955+0000","last_undegraded":"2024-04-26T23:48:45.093955+0000","last_fullsized":"2024-04-26T23:48:45.093955+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:56.445220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094457+0000","last_change":"2024-04-26T23:48:45.116370+0000","last_active":"2024-04-26T23:48:46.094457+0000","last_peered":"2024-04-26T23:48:46.094457+0000","last_clean":"2024-04-26T23:48:46.094457+0000","last_became_active":"2024-04-26T23:48:45.116104+0000","last_became_peered":"2024-04-26T23:48:45.116104+0000","last_unstale":"2024-04-26T23:48:46.094457+0000","last_undegraded":"2024-04-26T23:48:46.094457+0000","last_fullsized":"2024-04-26T23:48:46.094457+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:05.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094279+0000","last_change":"2024-04-26T23:48:44.100501+0000","last_active":"2024-04-26T23:48:46.094279+0000","last_peered":"2024-04-26T23:48:46.094279+0000","last_clean":"2024-04-26T23:48:46.094279+0000","last_became_active":"2024-04-26T23:48:44.100094+0000","last_became_peered":"2024-04-26T23:48:44.100094+0000","last_unstale":"2024-04-26T23:48:46.094279+0000","last_undegraded":"2024-04-26T23:48:46.094279+0000","last_fullsized":"2024-04-26T23:48:46.094279+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:41.749202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.132742+0000","last_change":"2024-04-26T23:48:45.129254+0000","last_active":"2024-04-26T23:48:45.132742+0000","last_peered":"2024-04-26T23:48:45.132742+0000","last_clean":"2024-04-26T23:48:45.132742+0000","last_became_active":"2024-04-26T23:48:45.128971+0000","last_became_peered":"2024-04-26T23:48:45.128971+0000","last_unstale":"2024-04-26T23:48:45.132742+0000","last_undegraded":"2024-04-26T23:48:45.132742+0000","last_fullsized":"2024-04-26T23:48:45.132742+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115132+0000","last_change":"2024-04-26T23:48:44.098297+0000","last_active":"2024-04-26T23:48:45.115132+0000","last_peered":"2024-04-26T23:48:45.115132+0000","last_clean":"2024-04-26T23:48:45.115132+0000","last_became_active":"2024-04-26T23:48:44.097964+0000","last_became_peered":"2024-04-26T23:48:44.097964+0000","last_unstale":"2024-04-26T23:48:45.115132+0000","last_undegraded":"2024-04-26T23:48:45.115132+0000","last_fullsized":"2024-04-26T23:48:45.115132+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:04.105551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133451+0000","last_change":"2024-04-26T23:48:45.131901+0000","last_active":"2024-04-26T23:48:45.133451+0000","last_peered":"2024-04-26T23:48:45.133451+0000","last_clean":"2024-04-26T23:48:45.133451+0000","last_became_active":"2024-04-26T23:48:45.131610+0000","last_became_peered":"2024-04-26T23:48:45.131610+0000","last_unstale":"2024-04-26T23:48:45.133451+0000","last_undegraded":"2024-04-26T23:48:45.133451+0000","last_fullsized":"2024-04-26T23:48:45.133451+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:56.179293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.139459+0000","last_change":"2024-04-26T23:48:44.099342+0000","last_active":"2024-04-26T23:48:45.139459+0000","last_peered":"2024-04-26T23:48:45.139459+0000","last_clean":"2024-04-26T23:48:45.139459+0000","last_became_active":"2024-04-26T23:48:44.097981+0000","last_became_peered":"2024-04-26T23:48:44.097981+0000","last_unstale":"2024-04-26T23:48:45.139459+0000","last_undegraded":"2024-04-26T23:48:45.139459+0000","last_fullsized":"2024-04-26T23:48:45.139459+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:23.035747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128276+0000","last_change":"2024-04-26T23:48:45.117673+0000","last_active":"2024-04-26T23:48:45.128276+0000","last_peered":"2024-04-26T23:48:45.128276+0000","last_clean":"2024-04-26T23:48:45.128276+0000","last_became_active":"2024-04-26T23:48:45.117378+0000","last_became_peered":"2024-04-26T23:48:45.117378+0000","last_unstale":"2024-04-26T23:48:45.128276+0000","last_undegraded":"2024-04-26T23:48:45.128276+0000","last_fullsized":"2024-04-26T23:48:45.128276+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:21.464098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"125'6","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.145082+0000","last_change":"2024-04-26T23:48:44.099079+0000","last_active":"2024-04-26T23:48:45.145082+0000","last_peered":"2024-04-26T23:48:45.145082+0000","last_clean":"2024-04-26T23:48:45.145082+0000","last_became_active":"2024-04-26T23:48:44.098799+0000","last_became_peered":"2024-04-26T23:48:44.098799+0000","last_unstale":"2024-04-26T23:48:45.145082+0000","last_undegraded":"2024-04-26T23:48:45.145082+0000","last_fullsized":"2024-04-26T23:48:45.145082+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:27.774046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.093973+0000","last_change":"2024-04-26T23:48:45.126541+0000","last_active":"2024-04-26T23:48:46.093973+0000","last_peered":"2024-04-26T23:48:46.093973+0000","last_clean":"2024-04-26T23:48:46.093973+0000","last_became_active":"2024-04-26T23:48:45.126215+0000","last_became_peered":"2024-04-26T23:48:45.126215+0000","last_unstale":"2024-04-26T23:48:46.093973+0000","last_undegraded":"2024-04-26T23:48:46.093973+0000","last_fullsized":"2024-04-26T23:48:46.093973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:01.704598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142813+0000","last_change":"2024-04-26T23:48:44.098336+0000","last_active":"2024-04-26T23:48:45.142813+0000","last_peered":"2024-04-26T23:48:45.142813+0000","last_clean":"2024-04-26T23:48:45.142813+0000","last_became_active":"2024-04-26T23:48:44.098057+0000","last_became_peered":"2024-04-26T23:48:44.098057+0000","last_unstale":"2024-04-26T23:48:45.142813+0000","last_undegraded":"2024-04-26T23:48:45.142813+0000","last_fullsized":"2024-04-26T23:48:45.142813+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:42.239892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123098+0000","last_change":"2024-04-26T23:48:45.118315+0000","last_active":"2024-04-26T23:48:45.123098+0000","last_peered":"2024-04-26T23:48:45.123098+0000","last_clean":"2024-04-26T23:48:45.123098+0000","last_became_active":"2024-04-26T23:48:45.118011+0000","last_became_peered":"2024-04-26T23:48:45.118011+0000","last_unstale":"2024-04-26T23:48:45.123098+0000","last_undegraded":"2024-04-26T23:48:45.123098+0000","last_fullsized":"2024-04-26T23:48:45.123098+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:22.817815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119799+0000","last_change":"2024-04-26T23:48:44.099409+0000","last_active":"2024-04-26T23:48:45.119799+0000","last_peered":"2024-04-26T23:48:45.119799+0000","last_clean":"2024-04-26T23:48:45.119799+0000","last_became_active":"2024-04-26T23:48:44.099142+0000","last_became_peered":"2024-04-26T23:48:44.099142+0000","last_unstale":"2024-04-26T23:48:45.119799+0000","last_undegraded":"2024-04-26T23:48:45.119799+0000","last_fullsized":"2024-04-26T23:48:45.119799+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:06.263930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.113136+0000","last_change":"2024-04-26T23:48:45.111194+0000","last_active":"2024-04-26T23:48:45.113136+0000","last_peered":"2024-04-26T23:48:45.113136+0000","last_clean":"2024-04-26T23:48:45.113136+0000","last_became_active":"2024-04-26T23:48:45.109156+0000","last_became_peered":"2024-04-26T23:48:45.109156+0000","last_unstale":"2024-04-26T23:48:45.113136+0000","last_undegraded":"2024-04-26T23:48:45.113136+0000","last_fullsized":"2024-04-26T23:48:45.113136+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:54.300484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.116183+0000","last_change":"2024-04-26T23:48:44.096978+0000","last_active":"2024-04-26T23:48:45.116183+0000","last_peered":"2024-04-26T23:48:45.116183+0000","last_clean":"2024-04-26T23:48:45.116183+0000","last_became_active":"2024-04-26T23:48:44.096679+0000","last_became_peered":"2024-04-26T23:48:44.096679+0000","last_unstale":"2024-04-26T23:48:45.116183+0000","last_undegraded":"2024-04-26T23:48:45.116183+0000","last_fullsized":"2024-04-26T23:48:45.116183+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:03.991476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119323+0000","last_change":"2024-04-26T23:48:45.113485+0000","last_active":"2024-04-26T23:48:45.119323+0000","last_peered":"2024-04-26T23:48:45.119323+0000","last_clean":"2024-04-26T23:48:45.119323+0000","last_became_active":"2024-04-26T23:48:45.113185+0000","last_became_peered":"2024-04-26T23:48:45.113185+0000","last_unstale":"2024-04-26T23:48:45.119323+0000","last_undegraded":"2024-04-26T23:48:45.119323+0000","last_fullsized":"2024-04-26T23:48:45.119323+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:01.704598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142830+0000","last_change":"2024-04-26T23:48:44.088720+0000","last_active":"2024-04-26T23:48:45.142830+0000","last_peered":"2024-04-26T23:48:45.142830+0000","last_clean":"2024-04-26T23:48:45.142830+0000","last_became_active":"2024-04-26T23:48:44.088455+0000","last_became_peered":"2024-04-26T23:48:44.088455+0000","last_unstale":"2024-04-26T23:48:45.142830+0000","last_undegraded":"2024-04-26T23:48:45.142830+0000","last_fullsized":"2024-04-26T23:48:45.142830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:22:25.961671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125960+0000","last_change":"2024-04-26T23:48:45.120495+0000","last_active":"2024-04-26T23:48:45.125960+0000","last_peered":"2024-04-26T23:48:45.125960+0000","last_clean":"2024-04-26T23:48:45.125960+0000","last_became_active":"2024-04-26T23:48:45.120212+0000","last_became_peered":"2024-04-26T23:48:45.120212+0000","last_unstale":"2024-04-26T23:48:45.125960+0000","last_undegraded":"2024-04-26T23:48:45.125960+0000","last_fullsized":"2024-04-26T23:48:45.125960+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:39.060629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.136366+0000","last_change":"2024-04-26T23:48:44.097266+0000","last_active":"2024-04-26T23:48:45.136366+0000","last_peered":"2024-04-26T23:48:45.136366+0000","last_clean":"2024-04-26T23:48:45.136366+0000","last_became_active":"2024-04-26T23:48:44.095060+0000","last_became_peered":"2024-04-26T23:48:44.095060+0000","last_unstale":"2024-04-26T23:48:45.136366+0000","last_undegraded":"2024-04-26T23:48:45.136366+0000","last_fullsized":"2024-04-26T23:48:45.136366+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.644092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115770+0000","last_change":"2024-04-26T23:48:45.112052+0000","last_active":"2024-04-26T23:48:45.115770+0000","last_peered":"2024-04-26T23:48:45.115770+0000","last_clean":"2024-04-26T23:48:45.115770+0000","last_became_active":"2024-04-26T23:48:45.111161+0000","last_became_peered":"2024-04-26T23:48:45.111161+0000","last_unstale":"2024-04-26T23:48:45.115770+0000","last_undegraded":"2024-04-26T23:48:45.115770+0000","last_fullsized":"2024-04-26T23:48:45.115770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:36.177556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141111+0000","last_change":"2024-04-26T23:48:44.102992+0000","last_active":"2024-04-26T23:48:45.141111+0000","last_peered":"2024-04-26T23:48:45.141111+0000","last_clean":"2024-04-26T23:48:45.141111+0000","last_became_active":"2024-04-26T23:48:44.102633+0000","last_became_peered":"2024-04-26T23:48:44.102633+0000","last_unstale":"2024-04-26T23:48:45.141111+0000","last_undegraded":"2024-04-26T23:48:45.141111+0000","last_fullsized":"2024-04-26T23:48:45.141111+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:06.263930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133391+0000","last_change":"2024-04-26T23:48:45.122089+0000","last_active":"2024-04-26T23:48:45.133391+0000","last_peered":"2024-04-26T23:48:45.133391+0000","last_clean":"2024-04-26T23:48:45.133391+0000","last_became_active":"2024-04-26T23:48:45.121814+0000","last_became_peered":"2024-04-26T23:48:45.121814+0000","last_unstale":"2024-04-26T23:48:45.133391+0000","last_undegraded":"2024-04-26T23:48:45.133391+0000","last_fullsized":"2024-04-26T23:48:45.133391+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:07.280033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.109004+0000","last_change":"2024-04-26T23:48:44.104197+0000","last_active":"2024-04-26T23:48:45.109004+0000","last_peered":"2024-04-26T23:48:45.109004+0000","last_clean":"2024-04-26T23:48:45.109004+0000","last_became_active":"2024-04-26T23:48:44.103901+0000","last_became_peered":"2024-04-26T23:48:44.103901+0000","last_unstale":"2024-04-26T23:48:45.109004+0000","last_undegraded":"2024-04-26T23:48:45.109004+0000","last_fullsized":"2024-04-26T23:48:45.109004+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:42.239892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127390+0000","last_change":"2024-04-26T23:48:45.124759+0000","last_active":"2024-04-26T23:48:45.127390+0000","last_peered":"2024-04-26T23:48:45.127390+0000","last_clean":"2024-04-26T23:48:45.127390+0000","last_became_active":"2024-04-26T23:48:45.124437+0000","last_became_peered":"2024-04-26T23:48:45.124437+0000","last_unstale":"2024-04-26T23:48:45.127390+0000","last_undegraded":"2024-04-26T23:48:45.127390+0000","last_fullsized":"2024-04-26T23:48:45.127390+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:00.362640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141132+0000","last_change":"2024-04-26T23:48:44.094143+0000","last_active":"2024-04-26T23:48:45.141132+0000","last_peered":"2024-04-26T23:48:45.141132+0000","last_clean":"2024-04-26T23:48:45.141132+0000","last_became_active":"2024-04-26T23:48:44.093856+0000","last_became_peered":"2024-04-26T23:48:44.093856+0000","last_unstale":"2024-04-26T23:48:45.141132+0000","last_undegraded":"2024-04-26T23:48:45.141132+0000","last_fullsized":"2024-04-26T23:48:45.141132+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:04.991842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133395+0000","last_change":"2024-04-26T23:48:45.131287+0000","last_active":"2024-04-26T23:48:45.133395+0000","last_peered":"2024-04-26T23:48:45.133395+0000","last_clean":"2024-04-26T23:48:45.133395+0000","last_became_active":"2024-04-26T23:48:45.130847+0000","last_became_peered":"2024-04-26T23:48:45.130847+0000","last_unstale":"2024-04-26T23:48:45.133395+0000","last_undegraded":"2024-04-26T23:48:45.133395+0000","last_fullsized":"2024-04-26T23:48:45.133395+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.204584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.107989+0000","last_change":"2024-04-26T23:48:44.097109+0000","last_active":"2024-04-26T23:48:45.107989+0000","last_peered":"2024-04-26T23:48:45.107989+0000","last_clean":"2024-04-26T23:48:45.107989+0000","last_became_active":"2024-04-26T23:48:44.096806+0000","last_became_peered":"2024-04-26T23:48:44.096806+0000","last_unstale":"2024-04-26T23:48:45.107989+0000","last_undegraded":"2024-04-26T23:48:45.107989+0000","last_fullsized":"2024-04-26T23:48:45.107989+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:20.447995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133097+0000","last_change":"2024-04-26T23:48:45.131224+0000","last_active":"2024-04-26T23:48:45.133097+0000","last_peered":"2024-04-26T23:48:45.133097+0000","last_clean":"2024-04-26T23:48:45.133097+0000","last_became_active":"2024-04-26T23:48:45.130764+0000","last_became_peered":"2024-04-26T23:48:45.130764+0000","last_unstale":"2024-04-26T23:48:45.133097+0000","last_undegraded":"2024-04-26T23:48:45.133097+0000","last_fullsized":"2024-04-26T23:48:45.133097+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:16.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.116241+0000","last_change":"2024-04-26T23:48:44.096026+0000","last_active":"2024-04-26T23:48:45.116241+0000","last_peered":"2024-04-26T23:48:45.116241+0000","last_clean":"2024-04-26T23:48:45.116241+0000","last_became_active":"2024-04-26T23:48:44.094052+0000","last_became_peered":"2024-04-26T23:48:44.094052+0000","last_unstale":"2024-04-26T23:48:45.116241+0000","last_undegraded":"2024-04-26T23:48:45.116241+0000","last_fullsized":"2024-04-26T23:48:45.116241+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.077797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.129878+0000","last_change":"2024-04-26T23:48:45.120746+0000","last_active":"2024-04-26T23:48:45.129878+0000","last_peered":"2024-04-26T23:48:45.129878+0000","last_clean":"2024-04-26T23:48:45.129878+0000","last_became_active":"2024-04-26T23:48:45.120441+0000","last_became_peered":"2024-04-26T23:48:45.120441+0000","last_unstale":"2024-04-26T23:48:45.129878+0000","last_undegraded":"2024-04-26T23:48:45.129878+0000","last_fullsized":"2024-04-26T23:48:45.129878+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:43.093900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.104781+0000","last_change":"2024-04-26T23:48:44.089496+0000","last_active":"2024-04-26T23:48:45.104781+0000","last_peered":"2024-04-26T23:48:45.104781+0000","last_clean":"2024-04-26T23:48:45.104781+0000","last_became_active":"2024-04-26T23:48:44.089191+0000","last_became_peered":"2024-04-26T23:48:44.089191+0000","last_unstale":"2024-04-26T23:48:45.104781+0000","last_undegraded":"2024-04-26T23:48:45.104781+0000","last_fullsized":"2024-04-26T23:48:45.104781+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.124035+0000","last_change":"2024-04-26T23:48:45.119084+0000","last_active":"2024-04-26T23:48:45.124035+0000","last_peered":"2024-04-26T23:48:45.124035+0000","last_clean":"2024-04-26T23:48:45.124035+0000","last_became_active":"2024-04-26T23:48:45.116752+0000","last_became_peered":"2024-04-26T23:48:45.116752+0000","last_unstale":"2024-04-26T23:48:45.124035+0000","last_undegraded":"2024-04-26T23:48:45.124035+0000","last_fullsized":"2024-04-26T23:48:45.124035+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:44.711005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141187+0000","last_change":"2024-04-26T23:48:44.096039+0000","last_active":"2024-04-26T23:48:45.141187+0000","last_peered":"2024-04-26T23:48:45.141187+0000","last_clean":"2024-04-26T23:48:45.141187+0000","last_became_active":"2024-04-26T23:48:44.095777+0000","last_became_peered":"2024-04-26T23:48:44.095777+0000","last_unstale":"2024-04-26T23:48:45.141187+0000","last_undegraded":"2024-04-26T23:48:45.141187+0000","last_fullsized":"2024-04-26T23:48:45.141187+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:25.621575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128372+0000","last_change":"2024-04-26T23:48:45.127504+0000","last_active":"2024-04-26T23:48:45.128372+0000","last_peered":"2024-04-26T23:48:45.128372+0000","last_clean":"2024-04-26T23:48:45.128372+0000","last_became_active":"2024-04-26T23:48:45.127143+0000","last_became_peered":"2024-04-26T23:48:45.127143+0000","last_unstale":"2024-04-26T23:48:45.128372+0000","last_undegraded":"2024-04-26T23:48:45.128372+0000","last_fullsized":"2024-04-26T23:48:45.128372+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:18.380171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094902+0000","last_change":"2024-04-26T23:48:44.096684+0000","last_active":"2024-04-26T23:48:46.094902+0000","last_peered":"2024-04-26T23:48:46.094902+0000","last_clean":"2024-04-26T23:48:46.094902+0000","last_became_active":"2024-04-26T23:48:44.096229+0000","last_became_peered":"2024-04-26T23:48:44.096229+0000","last_unstale":"2024-04-26T23:48:46.094902+0000","last_undegraded":"2024-04-26T23:48:46.094902+0000","last_fullsized":"2024-04-26T23:48:46.094902+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.364068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119174+0000","last_change":"2024-04-26T23:48:45.115502+0000","last_active":"2024-04-26T23:48:45.119174+0000","last_peered":"2024-04-26T23:48:45.119174+0000","last_clean":"2024-04-26T23:48:45.119174+0000","last_became_active":"2024-04-26T23:48:45.115223+0000","last_became_peered":"2024-04-26T23:48:45.115223+0000","last_unstale":"2024-04-26T23:48:45.119174+0000","last_undegraded":"2024-04-26T23:48:45.119174+0000","last_fullsized":"2024-04-26T23:48:45.119174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:28.790149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.096127+0000","last_change":"2024-04-26T23:48:44.099271+0000","last_active":"2024-04-26T23:48:46.096127+0000","last_peered":"2024-04-26T23:48:46.096127+0000","last_clean":"2024-04-26T23:48:46.096127+0000","last_became_active":"2024-04-26T23:48:44.098820+0000","last_became_peered":"2024-04-26T23:48:44.098820+0000","last_unstale":"2024-04-26T23:48:46.096127+0000","last_undegraded":"2024-04-26T23:48:46.096127+0000","last_fullsized":"2024-04-26T23:48:46.096127+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:41.439599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.129482+0000","last_change":"2024-04-26T23:48:45.128911+0000","last_active":"2024-04-26T23:48:45.129482+0000","last_peered":"2024-04-26T23:48:45.129482+0000","last_clean":"2024-04-26T23:48:45.129482+0000","last_became_active":"2024-04-26T23:48:45.128595+0000","last_became_peered":"2024-04-26T23:48:45.128595+0000","last_unstale":"2024-04-26T23:48:45.129482+0000","last_undegraded":"2024-04-26T23:48:45.129482+0000","last_fullsized":"2024-04-26T23:48:45.129482+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:32.745415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.097100+0000","last_change":"2024-04-26T23:48:44.084896+0000","last_active":"2024-04-26T23:48:45.097100+0000","last_peered":"2024-04-26T23:48:45.097100+0000","last_clean":"2024-04-26T23:48:45.097100+0000","last_became_active":"2024-04-26T23:48:44.084571+0000","last_became_peered":"2024-04-26T23:48:44.084571+0000","last_unstale":"2024-04-26T23:48:45.097100+0000","last_undegraded":"2024-04-26T23:48:45.097100+0000","last_fullsized":"2024-04-26T23:48:45.097100+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:38.240384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.103562+0000","last_change":"2024-04-26T23:48:45.101182+0000","last_active":"2024-04-26T23:48:45.103562+0000","last_peered":"2024-04-26T23:48:45.103562+0000","last_clean":"2024-04-26T23:48:45.103562+0000","last_became_active":"2024-04-26T23:48:45.100534+0000","last_became_peered":"2024-04-26T23:48:45.100534+0000","last_unstale":"2024-04-26T23:48:45.103562+0000","last_undegraded":"2024-04-26T23:48:45.103562+0000","last_fullsized":"2024-04-26T23:48:45.103562+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:17.028259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.096586+0000","last_change":"2024-04-26T23:48:44.098828+0000","last_active":"2024-04-26T23:48:45.096586+0000","last_peered":"2024-04-26T23:48:45.096586+0000","last_clean":"2024-04-26T23:48:45.096586+0000","last_became_active":"2024-04-26T23:48:44.098538+0000","last_became_peered":"2024-04-26T23:48:44.098538+0000","last_unstale":"2024-04-26T23:48:45.096586+0000","last_undegraded":"2024-04-26T23:48:45.096586+0000","last_fullsized":"2024-04-26T23:48:45.096586+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:21.801712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.132473+0000","last_change":"2024-04-26T23:48:45.129776+0000","last_active":"2024-04-26T23:48:45.132473+0000","last_peered":"2024-04-26T23:48:45.132473+0000","last_clean":"2024-04-26T23:48:45.132473+0000","last_became_active":"2024-04-26T23:48:45.127982+0000","last_became_peered":"2024-04-26T23:48:45.127982+0000","last_unstale":"2024-04-26T23:48:45.132473+0000","last_undegraded":"2024-04-26T23:48:45.132473+0000","last_fullsized":"2024-04-26T23:48:45.132473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.097385+0000","last_change":"2024-04-26T23:48:44.097561+0000","last_active":"2024-04-26T23:48:45.097385+0000","last_peered":"2024-04-26T23:48:45.097385+0000","last_clean":"2024-04-26T23:48:45.097385+0000","last_became_active":"2024-04-26T23:48:44.097211+0000","last_became_peered":"2024-04-26T23:48:44.097211+0000","last_unstale":"2024-04-26T23:48:45.097385+0000","last_undegraded":"2024-04-26T23:48:45.097385+0000","last_fullsized":"2024-04-26T23:48:45.097385+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:41.749202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127474+0000","last_change":"2024-04-26T23:48:45.122074+0000","last_active":"2024-04-26T23:48:45.127474+0000","last_peered":"2024-04-26T23:48:45.127474+0000","last_clean":"2024-04-26T23:48:45.127474+0000","last_became_active":"2024-04-26T23:48:45.121814+0000","last_became_peered":"2024-04-26T23:48:45.121814+0000","last_unstale":"2024-04-26T23:48:45.127474+0000","last_undegraded":"2024-04-26T23:48:45.127474+0000","last_fullsized":"2024-04-26T23:48:45.127474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:16.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.090997+0000","last_change":"2024-04-26T23:48:44.091498+0000","last_active":"2024-04-26T23:48:45.090997+0000","last_peered":"2024-04-26T23:48:45.090997+0000","last_clean":"2024-04-26T23:48:45.090997+0000","last_became_active":"2024-04-26T23:48:44.091189+0000","last_became_peered":"2024-04-26T23:48:44.091189+0000","last_unstale":"2024-04-26T23:48:45.090997+0000","last_undegraded":"2024-04-26T23:48:45.090997+0000","last_fullsized":"2024-04-26T23:48:45.090997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:17.932219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123204+0000","last_change":"2024-04-26T23:48:45.118917+0000","last_active":"2024-04-26T23:48:45.123204+0000","last_peered":"2024-04-26T23:48:45.123204+0000","last_clean":"2024-04-26T23:48:45.123204+0000","last_became_active":"2024-04-26T23:48:45.118593+0000","last_became_peered":"2024-04-26T23:48:45.118593+0000","last_unstale":"2024-04-26T23:48:45.123204+0000","last_undegraded":"2024-04-26T23:48:45.123204+0000","last_fullsized":"2024-04-26T23:48:45.123204+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:42.765305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.104840+0000","last_change":"2024-04-26T23:48:44.094568+0000","last_active":"2024-04-26T23:48:45.104840+0000","last_peered":"2024-04-26T23:48:45.104840+0000","last_clean":"2024-04-26T23:48:45.104840+0000","last_became_active":"2024-04-26T23:48:44.094252+0000","last_became_peered":"2024-04-26T23:48:44.094252+0000","last_unstale":"2024-04-26T23:48:45.104840+0000","last_undegraded":"2024-04-26T23:48:45.104840+0000","last_fullsized":"2024-04-26T23:48:45.104840+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:03.991476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125936+0000","last_change":"2024-04-26T23:48:45.120508+0000","last_active":"2024-04-26T23:48:45.125936+0000","last_peered":"2024-04-26T23:48:45.125936+0000","last_clean":"2024-04-26T23:48:45.125936+0000","last_became_active":"2024-04-26T23:48:45.120026+0000","last_became_peered":"2024-04-26T23:48:45.120026+0000","last_unstale":"2024-04-26T23:48:45.125936+0000","last_undegraded":"2024-04-26T23:48:45.125936+0000","last_fullsized":"2024-04-26T23:48:45.125936+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"125'8","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143512+0000","last_change":"2024-04-26T23:48:44.094669+0000","last_active":"2024-04-26T23:48:45.143512+0000","last_peered":"2024-04-26T23:48:45.143512+0000","last_clean":"2024-04-26T23:48:45.143512+0000","last_became_active":"2024-04-26T23:48:44.094332+0000","last_became_peered":"2024-04-26T23:48:44.094332+0000","last_unstale":"2024-04-26T23:48:45.143512+0000","last_undegraded":"2024-04-26T23:48:45.143512+0000","last_fullsized":"2024-04-26T23:48:45.143512+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":362,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.105001+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:48:45.105001+0000","last_peered":"2024-04-26T23:48:45.105001+0000","last_clean":"2024-04-26T23:48:45.105001+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:48:45.105001+0000","last_undegraded":"2024-04-26T23:48:45.105001+0000","last_fullsized":"2024-04-26T23:48:45.105001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41616,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330224,"statfs":{"total":96636764160,"available":96594149376,"internally_reserved":0,"allocated":2166784,"data_stored":1322654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":1560576,"data_stored":732092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46600,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325240,"statfs":{"total":96636764160,"available":96589045760,"internally_reserved":0,"allocated":1564672,"data_stored":731670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50764,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321076,"statfs":{"total":96636764160,"available":96584781824,"internally_reserved":0,"allocated":1568768,"data_stored":733122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574992,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54924,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316916,"statfs":{"total":96636764160,"available":96580521984,"internally_reserved":0,"allocated":1572864,"data_stored":733156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1556480,"data_stored":731510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50756,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321084,"statfs":{"total":96636764160,"available":96584790016,"internally_reserved":0,"allocated":1560576,"data_stored":731532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2154496,"data_stored":1321900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:48:49.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:48:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.880+0000 7fb199c00640 1 Processor -- start 2024-04-26T23:48:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.880+0000 7fb199c00640 1 -- start start 2024-04-26T23:48:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.882+0000 7fb199c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1941454a0 0x7fb1940cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.882+0000 7fb199c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1940c6a50 0x7fb1940c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.882+0000 7fb199c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb194072bb0 con 0x7fb194073970 2024-04-26T23:48:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.882+0000 7fb199c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb194072e90 con 0x7fb1940c6a50 2024-04-26T23:48:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.882+0000 7fb199c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb194073170 con 0x7fb1941454a0 2024-04-26T23:48:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb199200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1940c6a50 0x7fb1940c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb18b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb194073970 0x7fb1940cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57218/0 (socket says 172.21.15.46:57218) 2024-04-26T23:48:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb18b400640 1 -- 172.21.15.46:0/2362861643 learned_addr learned my addr 172.21.15.46:0/2362861643 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb193e00640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1941454a0 0x7fb1940cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb199200640 1 -- 172.21.15.46:0/2362861643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1941454a0 msgr2=0x7fb1940cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb199200640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1941454a0 0x7fb1940cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.883+0000 7fb199200640 1 -- 172.21.15.46:0/2362861643 >> v1:172.21.15.46:6789/0 conn(0x7fb194073970 legacy=0x7fb1940cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.884+0000 7fb193400640 1 -- 172.21.15.46:0/2362861643 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682641960 0 0) 0x7fb194072bb0 con 0x7fb194073970 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.884+0000 7fb199200640 1 -- 172.21.15.46:0/2362861643 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb194073450 con 0x7fb1940c6a50 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.885+0000 7fb199200640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940c6a50 0x7fb1940c6ee0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009d90 tx=0x7fb18c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e3bf2b96abdc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.885+0000 7fb193400640 1 -- 172.21.15.46:0/2362861643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c007740 con 0x7fb1940c6a50 2024-04-26T23:48:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.885+0000 7fb193400640 1 -- 172.21.15.46:0/2362861643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18c0078e0 con 0x7fb1940c6a50 2024-04-26T23:48:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.885+0000 7fb193400640 1 -- 172.21.15.46:0/2362861643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c007cb0 con 0x7fb1940c6a50 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 -- 172.21.15.46:0/2362861643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940c6a50 msgr2=0x7fb1940c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940c6a50 0x7fb1940c6ee0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009d90 tx=0x7fb18c02a660 comp rx=0 tx=0).stop 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 -- 172.21.15.46:0/2362861643 shutdown_connections 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940c6a50 0x7fb1940c6ee0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 --2- 172.21.15.46:0/2362861643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1941454a0 0x7fb1940cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 -- 172.21.15.46:0/2362861643 >> 172.21.15.46:0/2362861643 conn(0x7fb1940b26d0 msgr2=0x7fb1940b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 -- 172.21.15.46:0/2362861643 shutdown_connections 2024-04-26T23:48:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.886+0000 7fb199c00640 1 -- 172.21.15.46:0/2362861643 wait complete. 2024-04-26T23:48:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.888+0000 7fb199c00640 1 Processor -- start 2024-04-26T23:48:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.888+0000 7fb199c00640 1 -- start start 2024-04-26T23:48:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1940c6a50 0x7fb19413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1941454a0 0x7fb1941e5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb194073dd0 con 0x7fb194073970 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb194145880 con 0x7fb1941454a0 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb1941400d0 con 0x7fb1940c6a50 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41578/0 (socket says 172.21.15.46:41578) 2024-04-26T23:48:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 -- 172.21.15.46:0/1561694672 learned_addr learned my addr 172.21.15.46:0/1561694672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb199200640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1941454a0 0x7fb1941e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb18b400640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1940c6a50 0x7fb19413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 -- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1940c6a50 msgr2=0x7fb19413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1940c6a50 0x7fb19413fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.889+0000 7fb193e00640 1 -- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1941454a0 msgr2=0x7fb1941e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb193e00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1941454a0 0x7fb1941e5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb193e00640 1 -- 172.21.15.46:0/1561694672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1941e6940 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb193e00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e870 tx=0x7fb18400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184017070 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18400c940 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18400cd30 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1941e6c20 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.890+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1941e71e0 con 0x7fb194073970 2024-04-26T23:48:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.892+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1940693b0 con 0x7fb194073970 2024-04-26T23:48:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.893+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb18401a070 con 0x7fb194073970 2024-04-26T23:48:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.893+0000 7fb190c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.894+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fb18408bf80 con 0x7fb194073970 2024-04-26T23:48:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.894+0000 7fb18b400640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.895+0000 7fb18b400640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 0x7fb17006edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb17c007fd0 tx=0x7fb17c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:49.898+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb18405a180 con 0x7fb194073970 2024-04-26T23:48:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.098+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1940cefa0 con 0x7fb17006c8f0 2024-04-26T23:48:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.116+0000 7fb190c00640 1 -- 172.21.15.46:0/1561694672 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252422 (secure 0 0 0) 0x7fb1940cefa0 con 0x7fb17006c8f0 2024-04-26T23:48:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 msgr2=0x7fb17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 0x7fb17006edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb17c007fd0 tx=0x7fb17c009040 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 msgr2=0x7fb1941e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e870 tx=0x7fb18400ed30 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 shutdown_connections 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1941454a0 0x7fb1941e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1940c6a50 0x7fb19413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 --2- 172.21.15.46:0/1561694672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb194073970 0x7fb1941e79e0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 >> 172.21.15.46:0/1561694672 conn(0x7fb1940b26d0 msgr2=0x7fb1940b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 shutdown_connections 2024-04-26T23:48:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.121+0000 7fb199c00640 1 -- 172.21.15.46:0/1561694672 wait complete. 2024-04-26T23:48:50.143 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-04-26T23:48:48.149025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":397496,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754577224,"statfs":{"total":773094113280,"available":772687077376,"internally_reserved":0,"allocated":13705216,"data_stored":7037636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":393203249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.088883"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119240+0000","last_change":"2024-04-26T23:48:45.116061+0000","last_active":"2024-04-26T23:48:45.119240+0000","last_peered":"2024-04-26T23:48:45.119240+0000","last_clean":"2024-04-26T23:48:45.119240+0000","last_became_active":"2024-04-26T23:48:45.115774+0000","last_became_peered":"2024-04-26T23:48:45.115774+0000","last_unstale":"2024-04-26T23:48:45.119240+0000","last_undegraded":"2024-04-26T23:48:45.119240+0000","last_fullsized":"2024-04-26T23:48:45.119240+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:42.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.135511+0000","last_change":"2024-04-26T23:48:45.133919+0000","last_active":"2024-04-26T23:48:45.135511+0000","last_peered":"2024-04-26T23:48:45.135511+0000","last_clean":"2024-04-26T23:48:45.135511+0000","last_became_active":"2024-04-26T23:48:45.133520+0000","last_became_peered":"2024-04-26T23:48:45.133520+0000","last_unstale":"2024-04-26T23:48:45.135511+0000","last_undegraded":"2024-04-26T23:48:45.135511+0000","last_fullsized":"2024-04-26T23:48:45.135511+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:04.244263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131634+0000","last_change":"2024-04-26T23:48:45.128270+0000","last_active":"2024-04-26T23:48:45.131634+0000","last_peered":"2024-04-26T23:48:45.131634+0000","last_clean":"2024-04-26T23:48:45.131634+0000","last_became_active":"2024-04-26T23:48:45.127877+0000","last_became_peered":"2024-04-26T23:48:45.127877+0000","last_unstale":"2024-04-26T23:48:45.131634+0000","last_undegraded":"2024-04-26T23:48:45.131634+0000","last_fullsized":"2024-04-26T23:48:45.131634+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131153+0000","last_change":"2024-04-26T23:48:45.128503+0000","last_active":"2024-04-26T23:48:45.131153+0000","last_peered":"2024-04-26T23:48:45.131153+0000","last_clean":"2024-04-26T23:48:45.131153+0000","last_became_active":"2024-04-26T23:48:45.128169+0000","last_became_peered":"2024-04-26T23:48:45.128169+0000","last_unstale":"2024-04-26T23:48:45.131153+0000","last_undegraded":"2024-04-26T23:48:45.131153+0000","last_fullsized":"2024-04-26T23:48:45.131153+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.204584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125230+0000","last_change":"2024-04-26T23:48:45.119109+0000","last_active":"2024-04-26T23:48:45.125230+0000","last_peered":"2024-04-26T23:48:45.125230+0000","last_clean":"2024-04-26T23:48:45.125230+0000","last_became_active":"2024-04-26T23:48:45.118837+0000","last_became_peered":"2024-04-26T23:48:45.118837+0000","last_unstale":"2024-04-26T23:48:45.125230+0000","last_undegraded":"2024-04-26T23:48:45.125230+0000","last_fullsized":"2024-04-26T23:48:45.125230+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:36.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128739+0000","last_change":"2024-04-26T23:48:45.126961+0000","last_active":"2024-04-26T23:48:45.128739+0000","last_peered":"2024-04-26T23:48:45.128739+0000","last_clean":"2024-04-26T23:48:45.128739+0000","last_became_active":"2024-04-26T23:48:45.126325+0000","last_became_peered":"2024-04-26T23:48:45.126325+0000","last_unstale":"2024-04-26T23:48:45.128739+0000","last_undegraded":"2024-04-26T23:48:45.128739+0000","last_fullsized":"2024-04-26T23:48:45.128739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:55.318114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120920+0000","last_change":"2024-04-26T23:48:45.117765+0000","last_active":"2024-04-26T23:48:45.120920+0000","last_peered":"2024-04-26T23:48:45.120920+0000","last_clean":"2024-04-26T23:48:45.120920+0000","last_became_active":"2024-04-26T23:48:45.116761+0000","last_became_peered":"2024-04-26T23:48:45.116761+0000","last_unstale":"2024-04-26T23:48:45.120920+0000","last_undegraded":"2024-04-26T23:48:45.120920+0000","last_fullsized":"2024-04-26T23:48:45.120920+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:56.366552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133838+0000","last_change":"2024-04-26T23:48:45.131995+0000","last_active":"2024-04-26T23:48:45.133838+0000","last_peered":"2024-04-26T23:48:45.133838+0000","last_clean":"2024-04-26T23:48:45.133838+0000","last_became_active":"2024-04-26T23:48:45.131664+0000","last_became_peered":"2024-04-26T23:48:45.131664+0000","last_unstale":"2024-04-26T23:48:45.133838+0000","last_undegraded":"2024-04-26T23:48:45.133838+0000","last_fullsized":"2024-04-26T23:48:45.133838+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:23.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131388+0000","last_change":"2024-04-26T23:48:45.126543+0000","last_active":"2024-04-26T23:48:45.131388+0000","last_peered":"2024-04-26T23:48:45.131388+0000","last_clean":"2024-04-26T23:48:45.131388+0000","last_became_active":"2024-04-26T23:48:45.126193+0000","last_became_peered":"2024-04-26T23:48:45.126193+0000","last_unstale":"2024-04-26T23:48:45.131388+0000","last_undegraded":"2024-04-26T23:48:45.131388+0000","last_fullsized":"2024-04-26T23:48:45.131388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:31.436439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123845+0000","last_change":"2024-04-26T23:48:45.122839+0000","last_active":"2024-04-26T23:48:45.123845+0000","last_peered":"2024-04-26T23:48:45.123845+0000","last_clean":"2024-04-26T23:48:45.123845+0000","last_became_active":"2024-04-26T23:48:45.121806+0000","last_became_peered":"2024-04-26T23:48:45.121806+0000","last_unstale":"2024-04-26T23:48:45.123845+0000","last_undegraded":"2024-04-26T23:48:45.123845+0000","last_fullsized":"2024-04-26T23:48:45.123845+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:51.361411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.122474+0000","last_change":"2024-04-26T23:48:45.115829+0000","last_active":"2024-04-26T23:48:45.122474+0000","last_peered":"2024-04-26T23:48:45.122474+0000","last_clean":"2024-04-26T23:48:45.122474+0000","last_became_active":"2024-04-26T23:48:45.115536+0000","last_became_peered":"2024-04-26T23:48:45.115536+0000","last_unstale":"2024-04-26T23:48:45.122474+0000","last_undegraded":"2024-04-26T23:48:45.122474+0000","last_fullsized":"2024-04-26T23:48:45.122474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:51.361411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120728+0000","last_change":"2024-04-26T23:48:45.116511+0000","last_active":"2024-04-26T23:48:45.120728+0000","last_peered":"2024-04-26T23:48:45.120728+0000","last_clean":"2024-04-26T23:48:45.120728+0000","last_became_active":"2024-04-26T23:48:45.115910+0000","last_became_peered":"2024-04-26T23:48:45.115910+0000","last_unstale":"2024-04-26T23:48:45.120728+0000","last_undegraded":"2024-04-26T23:48:45.120728+0000","last_fullsized":"2024-04-26T23:48:45.120728+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.780714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.135123+0000","last_change":"2024-04-26T23:48:45.134014+0000","last_active":"2024-04-26T23:48:45.135123+0000","last_peered":"2024-04-26T23:48:45.135123+0000","last_clean":"2024-04-26T23:48:45.135123+0000","last_became_active":"2024-04-26T23:48:45.133703+0000","last_became_peered":"2024-04-26T23:48:45.133703+0000","last_unstale":"2024-04-26T23:48:45.135123+0000","last_undegraded":"2024-04-26T23:48:45.135123+0000","last_fullsized":"2024-04-26T23:48:45.135123+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:28.625388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.134015+0000","last_change":"2024-04-26T23:48:45.132668+0000","last_active":"2024-04-26T23:48:45.134015+0000","last_peered":"2024-04-26T23:48:45.134015+0000","last_clean":"2024-04-26T23:48:45.134015+0000","last_became_active":"2024-04-26T23:48:45.131256+0000","last_became_peered":"2024-04-26T23:48:45.131256+0000","last_unstale":"2024-04-26T23:48:45.134015+0000","last_undegraded":"2024-04-26T23:48:45.134015+0000","last_fullsized":"2024-04-26T23:48:45.134015+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:39.101779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.126557+0000","last_change":"2024-04-26T23:48:45.123018+0000","last_active":"2024-04-26T23:48:45.126557+0000","last_peered":"2024-04-26T23:48:45.126557+0000","last_clean":"2024-04-26T23:48:45.126557+0000","last_became_active":"2024-04-26T23:48:45.122406+0000","last_became_peered":"2024-04-26T23:48:45.122406+0000","last_unstale":"2024-04-26T23:48:45.126557+0000","last_undegraded":"2024-04-26T23:48:45.126557+0000","last_fullsized":"2024-04-26T23:48:45.126557+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:28.625388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.103533+0000","last_change":"2024-04-26T23:48:45.100885+0000","last_active":"2024-04-26T23:48:45.103533+0000","last_peered":"2024-04-26T23:48:45.103533+0000","last_clean":"2024-04-26T23:48:45.103533+0000","last_became_active":"2024-04-26T23:48:45.100149+0000","last_became_peered":"2024-04-26T23:48:45.100149+0000","last_unstale":"2024-04-26T23:48:45.103533+0000","last_undegraded":"2024-04-26T23:48:45.103533+0000","last_fullsized":"2024-04-26T23:48:45.103533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:22.449163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133971+0000","last_change":"2024-04-26T23:48:45.132129+0000","last_active":"2024-04-26T23:48:45.133971+0000","last_peered":"2024-04-26T23:48:45.133971+0000","last_clean":"2024-04-26T23:48:45.133971+0000","last_became_active":"2024-04-26T23:48:45.131756+0000","last_became_peered":"2024-04-26T23:48:45.131756+0000","last_unstale":"2024-04-26T23:48:45.133971+0000","last_undegraded":"2024-04-26T23:48:45.133971+0000","last_fullsized":"2024-04-26T23:48:45.133971+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:36.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133354+0000","last_change":"2024-04-26T23:48:45.131772+0000","last_active":"2024-04-26T23:48:45.133354+0000","last_peered":"2024-04-26T23:48:45.133354+0000","last_clean":"2024-04-26T23:48:45.133354+0000","last_became_active":"2024-04-26T23:48:45.130614+0000","last_became_peered":"2024-04-26T23:48:45.130614+0000","last_unstale":"2024-04-26T23:48:45.133354+0000","last_undegraded":"2024-04-26T23:48:45.133354+0000","last_fullsized":"2024-04-26T23:48:45.133354+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:31.436439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094101+0000","last_change":"2024-04-26T23:48:45.123449+0000","last_active":"2024-04-26T23:48:46.094101+0000","last_peered":"2024-04-26T23:48:46.094101+0000","last_clean":"2024-04-26T23:48:46.094101+0000","last_became_active":"2024-04-26T23:48:45.121815+0000","last_became_peered":"2024-04-26T23:48:45.121815+0000","last_unstale":"2024-04-26T23:48:46.094101+0000","last_undegraded":"2024-04-26T23:48:46.094101+0000","last_fullsized":"2024-04-26T23:48:46.094101+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:24.051850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115116+0000","last_change":"2024-04-26T23:48:45.113819+0000","last_active":"2024-04-26T23:48:45.115116+0000","last_peered":"2024-04-26T23:48:45.115116+0000","last_clean":"2024-04-26T23:48:45.115116+0000","last_became_active":"2024-04-26T23:48:45.113391+0000","last_became_peered":"2024-04-26T23:48:45.113391+0000","last_unstale":"2024-04-26T23:48:45.115116+0000","last_undegraded":"2024-04-26T23:48:45.115116+0000","last_fullsized":"2024-04-26T23:48:45.115116+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:24.209917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125911+0000","last_change":"2024-04-26T23:48:45.120284+0000","last_active":"2024-04-26T23:48:45.125911+0000","last_peered":"2024-04-26T23:48:45.125911+0000","last_clean":"2024-04-26T23:48:45.125911+0000","last_became_active":"2024-04-26T23:48:45.119682+0000","last_became_peered":"2024-04-26T23:48:45.119682+0000","last_unstale":"2024-04-26T23:48:45.125911+0000","last_undegraded":"2024-04-26T23:48:45.125911+0000","last_fullsized":"2024-04-26T23:48:45.125911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.130165+0000","last_change":"2024-04-26T23:48:45.128798+0000","last_active":"2024-04-26T23:48:45.130165+0000","last_peered":"2024-04-26T23:48:45.130165+0000","last_clean":"2024-04-26T23:48:45.130165+0000","last_became_active":"2024-04-26T23:48:45.128321+0000","last_became_peered":"2024-04-26T23:48:45.128321+0000","last_unstale":"2024-04-26T23:48:45.130165+0000","last_undegraded":"2024-04-26T23:48:45.130165+0000","last_fullsized":"2024-04-26T23:48:45.130165+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:16.431506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131349+0000","last_change":"2024-04-26T23:48:45.127189+0000","last_active":"2024-04-26T23:48:45.131349+0000","last_peered":"2024-04-26T23:48:45.131349+0000","last_clean":"2024-04-26T23:48:45.131349+0000","last_became_active":"2024-04-26T23:48:45.126885+0000","last_became_peered":"2024-04-26T23:48:45.126885+0000","last_unstale":"2024-04-26T23:48:45.131349+0000","last_undegraded":"2024-04-26T23:48:45.131349+0000","last_fullsized":"2024-04-26T23:48:45.131349+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:56.179293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133446+0000","last_change":"2024-04-26T23:48:45.129210+0000","last_active":"2024-04-26T23:48:45.133446+0000","last_peered":"2024-04-26T23:48:45.133446+0000","last_clean":"2024-04-26T23:48:45.133446+0000","last_became_active":"2024-04-26T23:48:45.128933+0000","last_became_peered":"2024-04-26T23:48:45.128933+0000","last_unstale":"2024-04-26T23:48:45.133446+0000","last_undegraded":"2024-04-26T23:48:45.133446+0000","last_fullsized":"2024-04-26T23:48:45.133446+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128457+0000","last_change":"2024-04-26T23:48:45.127404+0000","last_active":"2024-04-26T23:48:45.128457+0000","last_peered":"2024-04-26T23:48:45.128457+0000","last_clean":"2024-04-26T23:48:45.128457+0000","last_became_active":"2024-04-26T23:48:45.127075+0000","last_became_peered":"2024-04-26T23:48:45.127075+0000","last_unstale":"2024-04-26T23:48:45.128457+0000","last_undegraded":"2024-04-26T23:48:45.128457+0000","last_fullsized":"2024-04-26T23:48:45.128457+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:28.790149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115809+0000","last_change":"2024-04-26T23:48:45.114742+0000","last_active":"2024-04-26T23:48:45.115809+0000","last_peered":"2024-04-26T23:48:45.115809+0000","last_clean":"2024-04-26T23:48:45.115809+0000","last_became_active":"2024-04-26T23:48:45.114405+0000","last_became_peered":"2024-04-26T23:48:45.114405+0000","last_unstale":"2024-04-26T23:48:45.115809+0000","last_undegraded":"2024-04-26T23:48:45.115809+0000","last_fullsized":"2024-04-26T23:48:45.115809+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:06.699237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.096226+0000","last_change":"2024-04-26T23:48:45.124539+0000","last_active":"2024-04-26T23:48:46.096226+0000","last_peered":"2024-04-26T23:48:46.096226+0000","last_clean":"2024-04-26T23:48:46.096226+0000","last_became_active":"2024-04-26T23:48:45.121843+0000","last_became_peered":"2024-04-26T23:48:45.121843+0000","last_unstale":"2024-04-26T23:48:46.096226+0000","last_undegraded":"2024-04-26T23:48:46.096226+0000","last_fullsized":"2024-04-26T23:48:46.096226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:39.495048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.110952+0000","last_change":"2024-04-26T23:48:45.109679+0000","last_active":"2024-04-26T23:48:45.110952+0000","last_peered":"2024-04-26T23:48:45.110952+0000","last_clean":"2024-04-26T23:48:45.110952+0000","last_became_active":"2024-04-26T23:48:45.109384+0000","last_became_peered":"2024-04-26T23:48:45.109384+0000","last_unstale":"2024-04-26T23:48:45.110952+0000","last_undegraded":"2024-04-26T23:48:45.110952+0000","last_fullsized":"2024-04-26T23:48:45.110952+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:34.573168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.118406+0000","last_change":"2024-04-26T23:48:45.117555+0000","last_active":"2024-04-26T23:48:45.118406+0000","last_peered":"2024-04-26T23:48:45.118406+0000","last_clean":"2024-04-26T23:48:45.118406+0000","last_became_active":"2024-04-26T23:48:45.117245+0000","last_became_peered":"2024-04-26T23:48:45.117245+0000","last_unstale":"2024-04-26T23:48:45.118406+0000","last_undegraded":"2024-04-26T23:48:45.118406+0000","last_fullsized":"2024-04-26T23:48:45.118406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:39.481730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.131507+0000","last_change":"2024-04-26T23:48:45.126332+0000","last_active":"2024-04-26T23:48:45.131507+0000","last_peered":"2024-04-26T23:48:45.131507+0000","last_clean":"2024-04-26T23:48:45.131507+0000","last_became_active":"2024-04-26T23:48:45.126017+0000","last_became_peered":"2024-04-26T23:48:45.126017+0000","last_unstale":"2024-04-26T23:48:45.131507+0000","last_undegraded":"2024-04-26T23:48:45.131507+0000","last_fullsized":"2024-04-26T23:48:45.131507+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.130018+0000","last_change":"2024-04-26T23:48:45.119091+0000","last_active":"2024-04-26T23:48:45.130018+0000","last_peered":"2024-04-26T23:48:45.130018+0000","last_clean":"2024-04-26T23:48:45.130018+0000","last_became_active":"2024-04-26T23:48:45.117636+0000","last_became_peered":"2024-04-26T23:48:45.117636+0000","last_unstale":"2024-04-26T23:48:45.130018+0000","last_undegraded":"2024-04-26T23:48:45.130018+0000","last_fullsized":"2024-04-26T23:48:45.130018+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:05.007579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133427+0000","last_change":"2024-04-26T23:48:45.128678+0000","last_active":"2024-04-26T23:48:45.133427+0000","last_peered":"2024-04-26T23:48:45.133427+0000","last_clean":"2024-04-26T23:48:45.133427+0000","last_became_active":"2024-04-26T23:48:45.128372+0000","last_became_peered":"2024-04-26T23:48:45.128372+0000","last_unstale":"2024-04-26T23:48:45.133427+0000","last_undegraded":"2024-04-26T23:48:45.133427+0000","last_fullsized":"2024-04-26T23:48:45.133427+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.259973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127768+0000","last_change":"2024-04-26T23:48:45.123678+0000","last_active":"2024-04-26T23:48:45.127768+0000","last_peered":"2024-04-26T23:48:45.127768+0000","last_clean":"2024-04-26T23:48:45.127768+0000","last_became_active":"2024-04-26T23:48:45.121601+0000","last_became_peered":"2024-04-26T23:48:45.121601+0000","last_unstale":"2024-04-26T23:48:45.127768+0000","last_undegraded":"2024-04-26T23:48:45.127768+0000","last_fullsized":"2024-04-26T23:48:45.127768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:42.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094020+0000","last_change":"2024-04-26T23:48:44.096929+0000","last_active":"2024-04-26T23:48:46.094020+0000","last_peered":"2024-04-26T23:48:46.094020+0000","last_clean":"2024-04-26T23:48:46.094020+0000","last_became_active":"2024-04-26T23:48:44.096639+0000","last_became_peered":"2024-04-26T23:48:44.096639+0000","last_unstale":"2024-04-26T23:48:46.094020+0000","last_undegraded":"2024-04-26T23:48:46.094020+0000","last_fullsized":"2024-04-26T23:48:46.094020+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:50.345308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.121197+0000","last_change":"2024-04-26T23:48:45.117342+0000","last_active":"2024-04-26T23:48:45.121197+0000","last_peered":"2024-04-26T23:48:45.121197+0000","last_clean":"2024-04-26T23:48:45.121197+0000","last_became_active":"2024-04-26T23:48:45.117059+0000","last_became_peered":"2024-04-26T23:48:45.117059+0000","last_unstale":"2024-04-26T23:48:45.121197+0000","last_undegraded":"2024-04-26T23:48:45.121197+0000","last_fullsized":"2024-04-26T23:48:45.121197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:29.619699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143071+0000","last_change":"2024-04-26T23:48:44.096129+0000","last_active":"2024-04-26T23:48:45.143071+0000","last_peered":"2024-04-26T23:48:45.143071+0000","last_clean":"2024-04-26T23:48:45.143071+0000","last_became_active":"2024-04-26T23:48:44.095833+0000","last_became_peered":"2024-04-26T23:48:44.095833+0000","last_unstale":"2024-04-26T23:48:45.143071+0000","last_undegraded":"2024-04-26T23:48:45.143071+0000","last_fullsized":"2024-04-26T23:48:45.143071+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:44.078864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128240+0000","last_change":"2024-04-26T23:48:45.126351+0000","last_active":"2024-04-26T23:48:45.128240+0000","last_peered":"2024-04-26T23:48:45.128240+0000","last_clean":"2024-04-26T23:48:45.128240+0000","last_became_active":"2024-04-26T23:48:45.124414+0000","last_became_peered":"2024-04-26T23:48:45.124414+0000","last_unstale":"2024-04-26T23:48:45.128240+0000","last_undegraded":"2024-04-26T23:48:45.128240+0000","last_fullsized":"2024-04-26T23:48:45.128240+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:43.255995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142149+0000","last_change":"2024-04-26T23:48:44.096582+0000","last_active":"2024-04-26T23:48:45.142149+0000","last_peered":"2024-04-26T23:48:45.142149+0000","last_clean":"2024-04-26T23:48:45.142149+0000","last_became_active":"2024-04-26T23:48:44.095305+0000","last_became_peered":"2024-04-26T23:48:44.095305+0000","last_unstale":"2024-04-26T23:48:45.142149+0000","last_undegraded":"2024-04-26T23:48:45.142149+0000","last_fullsized":"2024-04-26T23:48:45.142149+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.077797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.126257+0000","last_change":"2024-04-26T23:48:45.121316+0000","last_active":"2024-04-26T23:48:45.126257+0000","last_peered":"2024-04-26T23:48:45.126257+0000","last_clean":"2024-04-26T23:48:45.126257+0000","last_became_active":"2024-04-26T23:48:45.120974+0000","last_became_peered":"2024-04-26T23:48:45.120974+0000","last_unstale":"2024-04-26T23:48:45.126257+0000","last_undegraded":"2024-04-26T23:48:45.126257+0000","last_fullsized":"2024-04-26T23:48:45.126257+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143827+0000","last_change":"2024-04-26T23:48:44.100844+0000","last_active":"2024-04-26T23:48:45.143827+0000","last_peered":"2024-04-26T23:48:45.143827+0000","last_clean":"2024-04-26T23:48:45.143827+0000","last_became_active":"2024-04-26T23:48:44.100211+0000","last_became_peered":"2024-04-26T23:48:44.100211+0000","last_unstale":"2024-04-26T23:48:45.143827+0000","last_undegraded":"2024-04-26T23:48:45.143827+0000","last_fullsized":"2024-04-26T23:48:45.143827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:42.826400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.134546+0000","last_change":"2024-04-26T23:48:45.133141+0000","last_active":"2024-04-26T23:48:45.134546+0000","last_peered":"2024-04-26T23:48:45.134546+0000","last_clean":"2024-04-26T23:48:45.134546+0000","last_became_active":"2024-04-26T23:48:45.132513+0000","last_became_peered":"2024-04-26T23:48:45.132513+0000","last_unstale":"2024-04-26T23:48:45.134546+0000","last_undegraded":"2024-04-26T23:48:45.134546+0000","last_fullsized":"2024-04-26T23:48:45.134546+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:39.060629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141198+0000","last_change":"2024-04-26T23:48:44.102016+0000","last_active":"2024-04-26T23:48:45.141198+0000","last_peered":"2024-04-26T23:48:45.141198+0000","last_clean":"2024-04-26T23:48:45.141198+0000","last_became_active":"2024-04-26T23:48:44.101707+0000","last_became_peered":"2024-04-26T23:48:44.101707+0000","last_unstale":"2024-04-26T23:48:45.141198+0000","last_undegraded":"2024-04-26T23:48:45.141198+0000","last_fullsized":"2024-04-26T23:48:45.141198+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:18.735614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119308+0000","last_change":"2024-04-26T23:48:45.116126+0000","last_active":"2024-04-26T23:48:45.119308+0000","last_peered":"2024-04-26T23:48:45.119308+0000","last_clean":"2024-04-26T23:48:45.119308+0000","last_became_active":"2024-04-26T23:48:45.115301+0000","last_became_peered":"2024-04-26T23:48:45.115301+0000","last_unstale":"2024-04-26T23:48:45.119308+0000","last_undegraded":"2024-04-26T23:48:45.119308+0000","last_fullsized":"2024-04-26T23:48:45.119308+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:18.380171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142707+0000","last_change":"2024-04-26T23:48:44.103078+0000","last_active":"2024-04-26T23:48:45.142707+0000","last_peered":"2024-04-26T23:48:45.142707+0000","last_clean":"2024-04-26T23:48:45.142707+0000","last_became_active":"2024-04-26T23:48:44.102810+0000","last_became_peered":"2024-04-26T23:48:44.102810+0000","last_unstale":"2024-04-26T23:48:45.142707+0000","last_undegraded":"2024-04-26T23:48:45.142707+0000","last_fullsized":"2024-04-26T23:48:45.142707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:43.136945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123533+0000","last_change":"2024-04-26T23:48:45.120244+0000","last_active":"2024-04-26T23:48:45.123533+0000","last_peered":"2024-04-26T23:48:45.123533+0000","last_clean":"2024-04-26T23:48:45.123533+0000","last_became_active":"2024-04-26T23:48:45.119896+0000","last_became_peered":"2024-04-26T23:48:45.119896+0000","last_unstale":"2024-04-26T23:48:45.123533+0000","last_undegraded":"2024-04-26T23:48:45.123533+0000","last_fullsized":"2024-04-26T23:48:45.123533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:21.464098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119017+0000","last_change":"2024-04-26T23:48:44.095056+0000","last_active":"2024-04-26T23:48:45.119017+0000","last_peered":"2024-04-26T23:48:45.119017+0000","last_clean":"2024-04-26T23:48:45.119017+0000","last_became_active":"2024-04-26T23:48:44.094770+0000","last_became_peered":"2024-04-26T23:48:44.094770+0000","last_unstale":"2024-04-26T23:48:45.119017+0000","last_undegraded":"2024-04-26T23:48:45.119017+0000","last_fullsized":"2024-04-26T23:48:45.119017+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.120083+0000","last_change":"2024-04-26T23:48:45.117473+0000","last_active":"2024-04-26T23:48:45.120083+0000","last_peered":"2024-04-26T23:48:45.120083+0000","last_clean":"2024-04-26T23:48:45.120083+0000","last_became_active":"2024-04-26T23:48:45.117165+0000","last_became_peered":"2024-04-26T23:48:45.117165+0000","last_unstale":"2024-04-26T23:48:45.120083+0000","last_undegraded":"2024-04-26T23:48:45.120083+0000","last_fullsized":"2024-04-26T23:48:45.120083+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:19.751717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.093955+0000","last_change":"2024-04-26T23:48:44.090397+0000","last_active":"2024-04-26T23:48:45.093955+0000","last_peered":"2024-04-26T23:48:45.093955+0000","last_clean":"2024-04-26T23:48:45.093955+0000","last_became_active":"2024-04-26T23:48:44.089163+0000","last_became_peered":"2024-04-26T23:48:44.089163+0000","last_unstale":"2024-04-26T23:48:45.093955+0000","last_undegraded":"2024-04-26T23:48:45.093955+0000","last_fullsized":"2024-04-26T23:48:45.093955+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:56.445220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094457+0000","last_change":"2024-04-26T23:48:45.116370+0000","last_active":"2024-04-26T23:48:46.094457+0000","last_peered":"2024-04-26T23:48:46.094457+0000","last_clean":"2024-04-26T23:48:46.094457+0000","last_became_active":"2024-04-26T23:48:45.116104+0000","last_became_peered":"2024-04-26T23:48:45.116104+0000","last_unstale":"2024-04-26T23:48:46.094457+0000","last_undegraded":"2024-04-26T23:48:46.094457+0000","last_fullsized":"2024-04-26T23:48:46.094457+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:05.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094279+0000","last_change":"2024-04-26T23:48:44.100501+0000","last_active":"2024-04-26T23:48:46.094279+0000","last_peered":"2024-04-26T23:48:46.094279+0000","last_clean":"2024-04-26T23:48:46.094279+0000","last_became_active":"2024-04-26T23:48:44.100094+0000","last_became_peered":"2024-04-26T23:48:44.100094+0000","last_unstale":"2024-04-26T23:48:46.094279+0000","last_undegraded":"2024-04-26T23:48:46.094279+0000","last_fullsized":"2024-04-26T23:48:46.094279+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:41.749202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.132742+0000","last_change":"2024-04-26T23:48:45.129254+0000","last_active":"2024-04-26T23:48:45.132742+0000","last_peered":"2024-04-26T23:48:45.132742+0000","last_clean":"2024-04-26T23:48:45.132742+0000","last_became_active":"2024-04-26T23:48:45.128971+0000","last_became_peered":"2024-04-26T23:48:45.128971+0000","last_unstale":"2024-04-26T23:48:45.132742+0000","last_undegraded":"2024-04-26T23:48:45.132742+0000","last_fullsized":"2024-04-26T23:48:45.132742+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115132+0000","last_change":"2024-04-26T23:48:44.098297+0000","last_active":"2024-04-26T23:48:45.115132+0000","last_peered":"2024-04-26T23:48:45.115132+0000","last_clean":"2024-04-26T23:48:45.115132+0000","last_became_active":"2024-04-26T23:48:44.097964+0000","last_became_peered":"2024-04-26T23:48:44.097964+0000","last_unstale":"2024-04-26T23:48:45.115132+0000","last_undegraded":"2024-04-26T23:48:45.115132+0000","last_fullsized":"2024-04-26T23:48:45.115132+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:04.105551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133451+0000","last_change":"2024-04-26T23:48:45.131901+0000","last_active":"2024-04-26T23:48:45.133451+0000","last_peered":"2024-04-26T23:48:45.133451+0000","last_clean":"2024-04-26T23:48:45.133451+0000","last_became_active":"2024-04-26T23:48:45.131610+0000","last_became_peered":"2024-04-26T23:48:45.131610+0000","last_unstale":"2024-04-26T23:48:45.133451+0000","last_undegraded":"2024-04-26T23:48:45.133451+0000","last_fullsized":"2024-04-26T23:48:45.133451+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:56.179293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.139459+0000","last_change":"2024-04-26T23:48:44.099342+0000","last_active":"2024-04-26T23:48:45.139459+0000","last_peered":"2024-04-26T23:48:45.139459+0000","last_clean":"2024-04-26T23:48:45.139459+0000","last_became_active":"2024-04-26T23:48:44.097981+0000","last_became_peered":"2024-04-26T23:48:44.097981+0000","last_unstale":"2024-04-26T23:48:45.139459+0000","last_undegraded":"2024-04-26T23:48:45.139459+0000","last_fullsized":"2024-04-26T23:48:45.139459+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:23.035747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128276+0000","last_change":"2024-04-26T23:48:45.117673+0000","last_active":"2024-04-26T23:48:45.128276+0000","last_peered":"2024-04-26T23:48:45.128276+0000","last_clean":"2024-04-26T23:48:45.128276+0000","last_became_active":"2024-04-26T23:48:45.117378+0000","last_became_peered":"2024-04-26T23:48:45.117378+0000","last_unstale":"2024-04-26T23:48:45.128276+0000","last_undegraded":"2024-04-26T23:48:45.128276+0000","last_fullsized":"2024-04-26T23:48:45.128276+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:21.464098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"125'6","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.145082+0000","last_change":"2024-04-26T23:48:44.099079+0000","last_active":"2024-04-26T23:48:45.145082+0000","last_peered":"2024-04-26T23:48:45.145082+0000","last_clean":"2024-04-26T23:48:45.145082+0000","last_became_active":"2024-04-26T23:48:44.098799+0000","last_became_peered":"2024-04-26T23:48:44.098799+0000","last_unstale":"2024-04-26T23:48:45.145082+0000","last_undegraded":"2024-04-26T23:48:45.145082+0000","last_fullsized":"2024-04-26T23:48:45.145082+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:27.774046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.093973+0000","last_change":"2024-04-26T23:48:45.126541+0000","last_active":"2024-04-26T23:48:46.093973+0000","last_peered":"2024-04-26T23:48:46.093973+0000","last_clean":"2024-04-26T23:48:46.093973+0000","last_became_active":"2024-04-26T23:48:45.126215+0000","last_became_peered":"2024-04-26T23:48:45.126215+0000","last_unstale":"2024-04-26T23:48:46.093973+0000","last_undegraded":"2024-04-26T23:48:46.093973+0000","last_fullsized":"2024-04-26T23:48:46.093973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:01.704598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142813+0000","last_change":"2024-04-26T23:48:44.098336+0000","last_active":"2024-04-26T23:48:45.142813+0000","last_peered":"2024-04-26T23:48:45.142813+0000","last_clean":"2024-04-26T23:48:45.142813+0000","last_became_active":"2024-04-26T23:48:44.098057+0000","last_became_peered":"2024-04-26T23:48:44.098057+0000","last_unstale":"2024-04-26T23:48:45.142813+0000","last_undegraded":"2024-04-26T23:48:45.142813+0000","last_fullsized":"2024-04-26T23:48:45.142813+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:42.239892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123098+0000","last_change":"2024-04-26T23:48:45.118315+0000","last_active":"2024-04-26T23:48:45.123098+0000","last_peered":"2024-04-26T23:48:45.123098+0000","last_clean":"2024-04-26T23:48:45.123098+0000","last_became_active":"2024-04-26T23:48:45.118011+0000","last_became_peered":"2024-04-26T23:48:45.118011+0000","last_unstale":"2024-04-26T23:48:45.123098+0000","last_undegraded":"2024-04-26T23:48:45.123098+0000","last_fullsized":"2024-04-26T23:48:45.123098+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:22.817815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119799+0000","last_change":"2024-04-26T23:48:44.099409+0000","last_active":"2024-04-26T23:48:45.119799+0000","last_peered":"2024-04-26T23:48:45.119799+0000","last_clean":"2024-04-26T23:48:45.119799+0000","last_became_active":"2024-04-26T23:48:44.099142+0000","last_became_peered":"2024-04-26T23:48:44.099142+0000","last_unstale":"2024-04-26T23:48:45.119799+0000","last_undegraded":"2024-04-26T23:48:45.119799+0000","last_fullsized":"2024-04-26T23:48:45.119799+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:06.263930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.113136+0000","last_change":"2024-04-26T23:48:45.111194+0000","last_active":"2024-04-26T23:48:45.113136+0000","last_peered":"2024-04-26T23:48:45.113136+0000","last_clean":"2024-04-26T23:48:45.113136+0000","last_became_active":"2024-04-26T23:48:45.109156+0000","last_became_peered":"2024-04-26T23:48:45.109156+0000","last_unstale":"2024-04-26T23:48:45.113136+0000","last_undegraded":"2024-04-26T23:48:45.113136+0000","last_fullsized":"2024-04-26T23:48:45.113136+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:54.300484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.116183+0000","last_change":"2024-04-26T23:48:44.096978+0000","last_active":"2024-04-26T23:48:45.116183+0000","last_peered":"2024-04-26T23:48:45.116183+0000","last_clean":"2024-04-26T23:48:45.116183+0000","last_became_active":"2024-04-26T23:48:44.096679+0000","last_became_peered":"2024-04-26T23:48:44.096679+0000","last_unstale":"2024-04-26T23:48:45.116183+0000","last_undegraded":"2024-04-26T23:48:45.116183+0000","last_fullsized":"2024-04-26T23:48:45.116183+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:03.991476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119323+0000","last_change":"2024-04-26T23:48:45.113485+0000","last_active":"2024-04-26T23:48:45.119323+0000","last_peered":"2024-04-26T23:48:45.119323+0000","last_clean":"2024-04-26T23:48:45.119323+0000","last_became_active":"2024-04-26T23:48:45.113185+0000","last_became_peered":"2024-04-26T23:48:45.113185+0000","last_unstale":"2024-04-26T23:48:45.119323+0000","last_undegraded":"2024-04-26T23:48:45.119323+0000","last_fullsized":"2024-04-26T23:48:45.119323+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:01.704598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.142830+0000","last_change":"2024-04-26T23:48:44.088720+0000","last_active":"2024-04-26T23:48:45.142830+0000","last_peered":"2024-04-26T23:48:45.142830+0000","last_clean":"2024-04-26T23:48:45.142830+0000","last_became_active":"2024-04-26T23:48:44.088455+0000","last_became_peered":"2024-04-26T23:48:44.088455+0000","last_unstale":"2024-04-26T23:48:45.142830+0000","last_undegraded":"2024-04-26T23:48:45.142830+0000","last_fullsized":"2024-04-26T23:48:45.142830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:22:25.961671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125960+0000","last_change":"2024-04-26T23:48:45.120495+0000","last_active":"2024-04-26T23:48:45.125960+0000","last_peered":"2024-04-26T23:48:45.125960+0000","last_clean":"2024-04-26T23:48:45.125960+0000","last_became_active":"2024-04-26T23:48:45.120212+0000","last_became_peered":"2024-04-26T23:48:45.120212+0000","last_unstale":"2024-04-26T23:48:45.125960+0000","last_undegraded":"2024-04-26T23:48:45.125960+0000","last_fullsized":"2024-04-26T23:48:45.125960+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:39.060629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.136366+0000","last_change":"2024-04-26T23:48:44.097266+0000","last_active":"2024-04-26T23:48:45.136366+0000","last_peered":"2024-04-26T23:48:45.136366+0000","last_clean":"2024-04-26T23:48:45.136366+0000","last_became_active":"2024-04-26T23:48:44.095060+0000","last_became_peered":"2024-04-26T23:48:44.095060+0000","last_unstale":"2024-04-26T23:48:45.136366+0000","last_undegraded":"2024-04-26T23:48:45.136366+0000","last_fullsized":"2024-04-26T23:48:45.136366+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.644092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.115770+0000","last_change":"2024-04-26T23:48:45.112052+0000","last_active":"2024-04-26T23:48:45.115770+0000","last_peered":"2024-04-26T23:48:45.115770+0000","last_clean":"2024-04-26T23:48:45.115770+0000","last_became_active":"2024-04-26T23:48:45.111161+0000","last_became_peered":"2024-04-26T23:48:45.111161+0000","last_unstale":"2024-04-26T23:48:45.115770+0000","last_undegraded":"2024-04-26T23:48:45.115770+0000","last_fullsized":"2024-04-26T23:48:45.115770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:36.177556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141111+0000","last_change":"2024-04-26T23:48:44.102992+0000","last_active":"2024-04-26T23:48:45.141111+0000","last_peered":"2024-04-26T23:48:45.141111+0000","last_clean":"2024-04-26T23:48:45.141111+0000","last_became_active":"2024-04-26T23:48:44.102633+0000","last_became_peered":"2024-04-26T23:48:44.102633+0000","last_unstale":"2024-04-26T23:48:45.141111+0000","last_undegraded":"2024-04-26T23:48:45.141111+0000","last_fullsized":"2024-04-26T23:48:45.141111+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:06.263930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133391+0000","last_change":"2024-04-26T23:48:45.122089+0000","last_active":"2024-04-26T23:48:45.133391+0000","last_peered":"2024-04-26T23:48:45.133391+0000","last_clean":"2024-04-26T23:48:45.133391+0000","last_became_active":"2024-04-26T23:48:45.121814+0000","last_became_peered":"2024-04-26T23:48:45.121814+0000","last_unstale":"2024-04-26T23:48:45.133391+0000","last_undegraded":"2024-04-26T23:48:45.133391+0000","last_fullsized":"2024-04-26T23:48:45.133391+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:07.280033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.109004+0000","last_change":"2024-04-26T23:48:44.104197+0000","last_active":"2024-04-26T23:48:45.109004+0000","last_peered":"2024-04-26T23:48:45.109004+0000","last_clean":"2024-04-26T23:48:45.109004+0000","last_became_active":"2024-04-26T23:48:44.103901+0000","last_became_peered":"2024-04-26T23:48:44.103901+0000","last_unstale":"2024-04-26T23:48:45.109004+0000","last_undegraded":"2024-04-26T23:48:45.109004+0000","last_fullsized":"2024-04-26T23:48:45.109004+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:42.239892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127390+0000","last_change":"2024-04-26T23:48:45.124759+0000","last_active":"2024-04-26T23:48:45.127390+0000","last_peered":"2024-04-26T23:48:45.127390+0000","last_clean":"2024-04-26T23:48:45.127390+0000","last_became_active":"2024-04-26T23:48:45.124437+0000","last_became_peered":"2024-04-26T23:48:45.124437+0000","last_unstale":"2024-04-26T23:48:45.127390+0000","last_undegraded":"2024-04-26T23:48:45.127390+0000","last_fullsized":"2024-04-26T23:48:45.127390+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:00.362640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141132+0000","last_change":"2024-04-26T23:48:44.094143+0000","last_active":"2024-04-26T23:48:45.141132+0000","last_peered":"2024-04-26T23:48:45.141132+0000","last_clean":"2024-04-26T23:48:45.141132+0000","last_became_active":"2024-04-26T23:48:44.093856+0000","last_became_peered":"2024-04-26T23:48:44.093856+0000","last_unstale":"2024-04-26T23:48:45.141132+0000","last_undegraded":"2024-04-26T23:48:45.141132+0000","last_fullsized":"2024-04-26T23:48:45.141132+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:04.991842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133395+0000","last_change":"2024-04-26T23:48:45.131287+0000","last_active":"2024-04-26T23:48:45.133395+0000","last_peered":"2024-04-26T23:48:45.133395+0000","last_clean":"2024-04-26T23:48:45.133395+0000","last_became_active":"2024-04-26T23:48:45.130847+0000","last_became_peered":"2024-04-26T23:48:45.130847+0000","last_unstale":"2024-04-26T23:48:45.133395+0000","last_undegraded":"2024-04-26T23:48:45.133395+0000","last_fullsized":"2024-04-26T23:48:45.133395+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.204584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.107989+0000","last_change":"2024-04-26T23:48:44.097109+0000","last_active":"2024-04-26T23:48:45.107989+0000","last_peered":"2024-04-26T23:48:45.107989+0000","last_clean":"2024-04-26T23:48:45.107989+0000","last_became_active":"2024-04-26T23:48:44.096806+0000","last_became_peered":"2024-04-26T23:48:44.096806+0000","last_unstale":"2024-04-26T23:48:45.107989+0000","last_undegraded":"2024-04-26T23:48:45.107989+0000","last_fullsized":"2024-04-26T23:48:45.107989+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:20.447995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.133097+0000","last_change":"2024-04-26T23:48:45.131224+0000","last_active":"2024-04-26T23:48:45.133097+0000","last_peered":"2024-04-26T23:48:45.133097+0000","last_clean":"2024-04-26T23:48:45.133097+0000","last_became_active":"2024-04-26T23:48:45.130764+0000","last_became_peered":"2024-04-26T23:48:45.130764+0000","last_unstale":"2024-04-26T23:48:45.133097+0000","last_undegraded":"2024-04-26T23:48:45.133097+0000","last_fullsized":"2024-04-26T23:48:45.133097+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:16.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.116241+0000","last_change":"2024-04-26T23:48:44.096026+0000","last_active":"2024-04-26T23:48:45.116241+0000","last_peered":"2024-04-26T23:48:45.116241+0000","last_clean":"2024-04-26T23:48:45.116241+0000","last_became_active":"2024-04-26T23:48:44.094052+0000","last_became_peered":"2024-04-26T23:48:44.094052+0000","last_unstale":"2024-04-26T23:48:45.116241+0000","last_undegraded":"2024-04-26T23:48:45.116241+0000","last_fullsized":"2024-04-26T23:48:45.116241+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.077797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.129878+0000","last_change":"2024-04-26T23:48:45.120746+0000","last_active":"2024-04-26T23:48:45.129878+0000","last_peered":"2024-04-26T23:48:45.129878+0000","last_clean":"2024-04-26T23:48:45.129878+0000","last_became_active":"2024-04-26T23:48:45.120441+0000","last_became_peered":"2024-04-26T23:48:45.120441+0000","last_unstale":"2024-04-26T23:48:45.129878+0000","last_undegraded":"2024-04-26T23:48:45.129878+0000","last_fullsized":"2024-04-26T23:48:45.129878+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:43.093900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.104781+0000","last_change":"2024-04-26T23:48:44.089496+0000","last_active":"2024-04-26T23:48:45.104781+0000","last_peered":"2024-04-26T23:48:45.104781+0000","last_clean":"2024-04-26T23:48:45.104781+0000","last_became_active":"2024-04-26T23:48:44.089191+0000","last_became_peered":"2024-04-26T23:48:44.089191+0000","last_unstale":"2024-04-26T23:48:45.104781+0000","last_undegraded":"2024-04-26T23:48:45.104781+0000","last_fullsized":"2024-04-26T23:48:45.104781+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.124035+0000","last_change":"2024-04-26T23:48:45.119084+0000","last_active":"2024-04-26T23:48:45.124035+0000","last_peered":"2024-04-26T23:48:45.124035+0000","last_clean":"2024-04-26T23:48:45.124035+0000","last_became_active":"2024-04-26T23:48:45.116752+0000","last_became_peered":"2024-04-26T23:48:45.116752+0000","last_unstale":"2024-04-26T23:48:45.124035+0000","last_undegraded":"2024-04-26T23:48:45.124035+0000","last_fullsized":"2024-04-26T23:48:45.124035+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:44.711005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.141187+0000","last_change":"2024-04-26T23:48:44.096039+0000","last_active":"2024-04-26T23:48:45.141187+0000","last_peered":"2024-04-26T23:48:45.141187+0000","last_clean":"2024-04-26T23:48:45.141187+0000","last_became_active":"2024-04-26T23:48:44.095777+0000","last_became_peered":"2024-04-26T23:48:44.095777+0000","last_unstale":"2024-04-26T23:48:45.141187+0000","last_undegraded":"2024-04-26T23:48:45.141187+0000","last_fullsized":"2024-04-26T23:48:45.141187+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:25.621575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.128372+0000","last_change":"2024-04-26T23:48:45.127504+0000","last_active":"2024-04-26T23:48:45.128372+0000","last_peered":"2024-04-26T23:48:45.128372+0000","last_clean":"2024-04-26T23:48:45.128372+0000","last_became_active":"2024-04-26T23:48:45.127143+0000","last_became_peered":"2024-04-26T23:48:45.127143+0000","last_unstale":"2024-04-26T23:48:45.128372+0000","last_undegraded":"2024-04-26T23:48:45.128372+0000","last_fullsized":"2024-04-26T23:48:45.128372+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:18.380171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.094902+0000","last_change":"2024-04-26T23:48:44.096684+0000","last_active":"2024-04-26T23:48:46.094902+0000","last_peered":"2024-04-26T23:48:46.094902+0000","last_clean":"2024-04-26T23:48:46.094902+0000","last_became_active":"2024-04-26T23:48:44.096229+0000","last_became_peered":"2024-04-26T23:48:44.096229+0000","last_unstale":"2024-04-26T23:48:46.094902+0000","last_undegraded":"2024-04-26T23:48:46.094902+0000","last_fullsized":"2024-04-26T23:48:46.094902+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.364068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.119174+0000","last_change":"2024-04-26T23:48:45.115502+0000","last_active":"2024-04-26T23:48:45.119174+0000","last_peered":"2024-04-26T23:48:45.119174+0000","last_clean":"2024-04-26T23:48:45.119174+0000","last_became_active":"2024-04-26T23:48:45.115223+0000","last_became_peered":"2024-04-26T23:48:45.115223+0000","last_unstale":"2024-04-26T23:48:45.119174+0000","last_undegraded":"2024-04-26T23:48:45.119174+0000","last_fullsized":"2024-04-26T23:48:45.119174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:28.790149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-26T23:48:46.096127+0000","last_change":"2024-04-26T23:48:44.099271+0000","last_active":"2024-04-26T23:48:46.096127+0000","last_peered":"2024-04-26T23:48:46.096127+0000","last_clean":"2024-04-26T23:48:46.096127+0000","last_became_active":"2024-04-26T23:48:44.098820+0000","last_became_peered":"2024-04-26T23:48:44.098820+0000","last_unstale":"2024-04-26T23:48:46.096127+0000","last_undegraded":"2024-04-26T23:48:46.096127+0000","last_fullsized":"2024-04-26T23:48:46.096127+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:41.439599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.129482+0000","last_change":"2024-04-26T23:48:45.128911+0000","last_active":"2024-04-26T23:48:45.129482+0000","last_peered":"2024-04-26T23:48:45.129482+0000","last_clean":"2024-04-26T23:48:45.129482+0000","last_became_active":"2024-04-26T23:48:45.128595+0000","last_became_peered":"2024-04-26T23:48:45.128595+0000","last_unstale":"2024-04-26T23:48:45.129482+0000","last_undegraded":"2024-04-26T23:48:45.129482+0000","last_fullsized":"2024-04-26T23:48:45.129482+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:32.745415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.097100+0000","last_change":"2024-04-26T23:48:44.084896+0000","last_active":"2024-04-26T23:48:45.097100+0000","last_peered":"2024-04-26T23:48:45.097100+0000","last_clean":"2024-04-26T23:48:45.097100+0000","last_became_active":"2024-04-26T23:48:44.084571+0000","last_became_peered":"2024-04-26T23:48:44.084571+0000","last_unstale":"2024-04-26T23:48:45.097100+0000","last_undegraded":"2024-04-26T23:48:45.097100+0000","last_fullsized":"2024-04-26T23:48:45.097100+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:38.240384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.103562+0000","last_change":"2024-04-26T23:48:45.101182+0000","last_active":"2024-04-26T23:48:45.103562+0000","last_peered":"2024-04-26T23:48:45.103562+0000","last_clean":"2024-04-26T23:48:45.103562+0000","last_became_active":"2024-04-26T23:48:45.100534+0000","last_became_peered":"2024-04-26T23:48:45.100534+0000","last_unstale":"2024-04-26T23:48:45.103562+0000","last_undegraded":"2024-04-26T23:48:45.103562+0000","last_fullsized":"2024-04-26T23:48:45.103562+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:17.028259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.096586+0000","last_change":"2024-04-26T23:48:44.098828+0000","last_active":"2024-04-26T23:48:45.096586+0000","last_peered":"2024-04-26T23:48:45.096586+0000","last_clean":"2024-04-26T23:48:45.096586+0000","last_became_active":"2024-04-26T23:48:44.098538+0000","last_became_peered":"2024-04-26T23:48:44.098538+0000","last_unstale":"2024-04-26T23:48:45.096586+0000","last_undegraded":"2024-04-26T23:48:45.096586+0000","last_fullsized":"2024-04-26T23:48:45.096586+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:21.801712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.132473+0000","last_change":"2024-04-26T23:48:45.129776+0000","last_active":"2024-04-26T23:48:45.132473+0000","last_peered":"2024-04-26T23:48:45.132473+0000","last_clean":"2024-04-26T23:48:45.132473+0000","last_became_active":"2024-04-26T23:48:45.127982+0000","last_became_peered":"2024-04-26T23:48:45.127982+0000","last_unstale":"2024-04-26T23:48:45.132473+0000","last_undegraded":"2024-04-26T23:48:45.132473+0000","last_fullsized":"2024-04-26T23:48:45.132473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:09.043127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.097385+0000","last_change":"2024-04-26T23:48:44.097561+0000","last_active":"2024-04-26T23:48:45.097385+0000","last_peered":"2024-04-26T23:48:45.097385+0000","last_clean":"2024-04-26T23:48:45.097385+0000","last_became_active":"2024-04-26T23:48:44.097211+0000","last_became_peered":"2024-04-26T23:48:44.097211+0000","last_unstale":"2024-04-26T23:48:45.097385+0000","last_undegraded":"2024-04-26T23:48:45.097385+0000","last_fullsized":"2024-04-26T23:48:45.097385+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:41.749202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.127474+0000","last_change":"2024-04-26T23:48:45.122074+0000","last_active":"2024-04-26T23:48:45.127474+0000","last_peered":"2024-04-26T23:48:45.127474+0000","last_clean":"2024-04-26T23:48:45.127474+0000","last_became_active":"2024-04-26T23:48:45.121814+0000","last_became_peered":"2024-04-26T23:48:45.121814+0000","last_unstale":"2024-04-26T23:48:45.127474+0000","last_undegraded":"2024-04-26T23:48:45.127474+0000","last_fullsized":"2024-04-26T23:48:45.127474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:16.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.090997+0000","last_change":"2024-04-26T23:48:44.091498+0000","last_active":"2024-04-26T23:48:45.090997+0000","last_peered":"2024-04-26T23:48:45.090997+0000","last_clean":"2024-04-26T23:48:45.090997+0000","last_became_active":"2024-04-26T23:48:44.091189+0000","last_became_peered":"2024-04-26T23:48:44.091189+0000","last_unstale":"2024-04-26T23:48:45.090997+0000","last_undegraded":"2024-04-26T23:48:45.090997+0000","last_fullsized":"2024-04-26T23:48:45.090997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:17.932219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.123204+0000","last_change":"2024-04-26T23:48:45.118917+0000","last_active":"2024-04-26T23:48:45.123204+0000","last_peered":"2024-04-26T23:48:45.123204+0000","last_clean":"2024-04-26T23:48:45.123204+0000","last_became_active":"2024-04-26T23:48:45.118593+0000","last_became_peered":"2024-04-26T23:48:45.118593+0000","last_unstale":"2024-04-26T23:48:45.123204+0000","last_undegraded":"2024-04-26T23:48:45.123204+0000","last_fullsized":"2024-04-26T23:48:45.123204+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:42.765305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.104840+0000","last_change":"2024-04-26T23:48:44.094568+0000","last_active":"2024-04-26T23:48:45.104840+0000","last_peered":"2024-04-26T23:48:45.104840+0000","last_clean":"2024-04-26T23:48:45.104840+0000","last_became_active":"2024-04-26T23:48:44.094252+0000","last_became_peered":"2024-04-26T23:48:44.094252+0000","last_unstale":"2024-04-26T23:48:45.104840+0000","last_undegraded":"2024-04-26T23:48:45.104840+0000","last_fullsized":"2024-04-26T23:48:45.104840+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:03.991476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.125936+0000","last_change":"2024-04-26T23:48:45.120508+0000","last_active":"2024-04-26T23:48:45.125936+0000","last_peered":"2024-04-26T23:48:45.125936+0000","last_clean":"2024-04-26T23:48:45.125936+0000","last_became_active":"2024-04-26T23:48:45.120026+0000","last_became_peered":"2024-04-26T23:48:45.120026+0000","last_unstale":"2024-04-26T23:48:45.125936+0000","last_undegraded":"2024-04-26T23:48:45.125936+0000","last_fullsized":"2024-04-26T23:48:45.125936+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:44.055268+0000","last_clean_scrub_stamp":"2024-04-26T23:48:44.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:17.303829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"125'8","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.143512+0000","last_change":"2024-04-26T23:48:44.094669+0000","last_active":"2024-04-26T23:48:45.143512+0000","last_peered":"2024-04-26T23:48:45.143512+0000","last_clean":"2024-04-26T23:48:45.143512+0000","last_became_active":"2024-04-26T23:48:44.094332+0000","last_became_peered":"2024-04-26T23:48:44.094332+0000","last_unstale":"2024-04-26T23:48:45.143512+0000","last_undegraded":"2024-04-26T23:48:45.143512+0000","last_fullsized":"2024-04-26T23:48:45.143512+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:48:43.039165+0000","last_clean_scrub_stamp":"2024-04-26T23:48:43.039165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:38.478945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":362,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-26T23:48:45.105001+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:48:45.105001+0000","last_peered":"2024-04-26T23:48:45.105001+0000","last_clean":"2024-04-26T23:48:45.105001+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:48:45.105001+0000","last_undegraded":"2024-04-26T23:48:45.105001+0000","last_fullsized":"2024-04-26T23:48:45.105001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41616,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330224,"statfs":{"total":96636764160,"available":96594149376,"internally_reserved":0,"allocated":2166784,"data_stored":1322654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":1560576,"data_stored":732092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46600,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325240,"statfs":{"total":96636764160,"available":96589045760,"internally_reserved":0,"allocated":1564672,"data_stored":731670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50764,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321076,"statfs":{"total":96636764160,"available":96584781824,"internally_reserved":0,"allocated":1568768,"data_stored":733122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574992,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54924,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316916,"statfs":{"total":96636764160,"available":96580521984,"internally_reserved":0,"allocated":1572864,"data_stored":733156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1556480,"data_stored":731510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50756,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321084,"statfs":{"total":96636764160,"available":96584790016,"internally_reserved":0,"allocated":1560576,"data_stored":731532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2154496,"data_stored":1321900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:48:50.146 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:48:50.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:48:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.399+0000 7fcb6a800640 1 Processor -- start 2024-04-26T23:48:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.400+0000 7fcb6a800640 1 -- start start 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.401+0000 7fcb6a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb6415e220 0x7fcb6406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.401+0000 7fcb6a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb640d07e0 con 0x7fcb6415d730 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.402+0000 7fcb6a800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6406eb90 con 0x7fcb640b53b0 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.402+0000 7fcb6a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcb640b4310 con 0x7fcb6415e220 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.402+0000 7fcb63400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcb6415d730 0x7fcb640d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57230/0 (socket says 172.21.15.46:57230) 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.402+0000 7fcb63400640 1 -- 172.21.15.46:0/222285644 learned_addr learned my addr 172.21.15.46:0/222285644 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.402+0000 7fcb63e00640 1 --2- 172.21.15.46:0/222285644 >> v2:172.21.15.142:3300/0 conn(0x7fcb6415e220 0x7fcb6406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182513973 0 0) 0x7fcb640d07e0 con 0x7fcb6415d730 2024-04-26T23:48:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb48003520 con 0x7fcb6415d730 2024-04-26T23:48:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3571397362 0 0) 0x7fcb48003520 con 0x7fcb6415d730 2024-04-26T23:48:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 >> v2:172.21.15.142:3300/0 conn(0x7fcb6415e220 msgr2=0x7fcb6406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 --2- 172.21.15.46:0/222285644 >> v2:172.21.15.142:3300/0 conn(0x7fcb6415e220 0x7fcb6406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 >> v1:172.21.15.142:6789/0 conn(0x7fcb640b53b0 legacy=0x7fcb6415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640b45f0 con 0x7fcb6415d730 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcb54001670 con 0x7fcb6415d730 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 >> v1:172.21.15.46:6789/0 conn(0x7fcb6415d730 legacy=0x7fcb640d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 0x7fcb48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb480051c0 0x7fcb4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640b4310 con 0x7fcb48003a00 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4800c0c0 con 0x7fcb480051c0 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.403+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4800c2c0 con 0x7fcb48004800 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 0x7fcb48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb62a00640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 -- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb48004800 msgr2=0x7fcb48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 -- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb480051c0 msgr2=0x7fcb4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb480051c0 0x7fcb4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63400640 1 -- 172.21.15.46:0/222285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb48003520 con 0x7fcb48003a00 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb63e00640 1 -- 172.21.15.46:0/222285644 reap_dead start 2024-04-26T23:48:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.404+0000 7fcb62a00640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.405+0000 7fcb63400640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 0x7fcb48003e20 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c110 tx=0x7fcb48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e040331db0771c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007c40 con 0x7fcb48003a00 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54005af0 con 0x7fcb48003a00 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb62000640 1 -- 172.21.15.46:0/222285644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007070 con 0x7fcb48003a00 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 -- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 msgr2=0x7fcb48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 0x7fcb48003e20 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c110 tx=0x7fcb48005830 comp rx=0 tx=0).stop 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 -- 172.21.15.46:0/222285644 shutdown_connections 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 --2- 172.21.15.46:0/222285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb48003a00 0x7fcb48003e20 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 -- 172.21.15.46:0/222285644 >> 172.21.15.46:0/222285644 conn(0x7fcb6406c980 msgr2=0x7fcb640b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 -- 172.21.15.46:0/222285644 shutdown_connections 2024-04-26T23:48:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.406+0000 7fcb6a800640 1 -- 172.21.15.46:0/222285644 wait complete. 2024-04-26T23:48:50.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.408+0000 7fcb6a800640 1 Processor -- start 2024-04-26T23:48:50.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.408+0000 7fcb6a800640 1 -- start start 2024-04-26T23:48:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb48003a00 0x7fcb6415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 0x7fcb640c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640b4e90 con 0x7fcb640c6a00 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640b4bb0 con 0x7fcb48003a00 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.409+0000 7fcb6a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb640b3ec0 con 0x7fcb641460a0 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41600/0 (socket says 172.21.15.46:41600) 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 -- 172.21.15.46:0/4076665138 learned_addr learned my addr 172.21.15.46:0/4076665138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb62a00640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 0x7fcb640c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 -- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 msgr2=0x7fcb640c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63400640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb48003a00 0x7fcb6415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 0x7fcb640c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 -- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb48003a00 msgr2=0x7fcb6415aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb48003a00 0x7fcb6415aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 -- 172.21.15.46:0/4076665138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6415af80 con 0x7fcb640c6a00 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb62a00640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 0x7fcb640c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.410+0000 7fcb63e00640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fcb64074340 tx=0x7fcb5c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.411+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c018020 con 0x7fcb640c6a00 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.411+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00cbe0 con 0x7fcb640c6a00 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.411+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c014480 con 0x7fcb640c6a00 2024-04-26T23:48:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.411+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb64145000 con 0x7fcb640c6a00 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.411+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641455c0 con 0x7fcb640c6a00 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.412+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb64069530 con 0x7fcb640c6a00 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.413+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb5c005d10 con 0x7fcb640c6a00 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.414+0000 7fcb5b400640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 0x7fcb4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.414+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fcb5c08c730 con 0x7fcb640c6a00 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.415+0000 7fcb63400640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 0x7fcb4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.416+0000 7fcb63400640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 0x7fcb4406edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcb54032000 tx=0x7fcb54009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.418+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb5c022070 con 0x7fcb640c6a00 2024-04-26T23:48:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.701+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcb640aeaf0 con 0x7fcb640c6a00 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.706+0000 7fcb5b400640 1 -- 172.21.15.46:0/4076665138 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fcb5c010040 con 0x7fcb640c6a00 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:48:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:48:50.712 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:48:50.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 msgr2=0x7fcb4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 0x7fcb4406edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcb54032000 tx=0x7fcb54009030 comp rx=0 tx=0).stop 2024-04-26T23:48:50.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 msgr2=0x7fcb64072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fcb64074340 tx=0x7fcb5c00c0c0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 shutdown_connections 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb4406c8f0 0x7fcb4406edb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb640c6a00 0x7fcb64072f60 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb641460a0 0x7fcb640c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 --2- 172.21.15.46:0/4076665138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb48003a00 0x7fcb6415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 >> 172.21.15.46:0/4076665138 conn(0x7fcb6406c980 msgr2=0x7fcb64141070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 shutdown_connections 2024-04-26T23:48:50.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.709+0000 7fcb6a800640 1 -- 172.21.15.46:0/4076665138 wait complete. 2024-04-26T23:48:50.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:48:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.939+0000 7ff7bec00640 1 Processor -- start 2024-04-26T23:48:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.939+0000 7ff7bec00640 1 -- start start 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7b80720e0 0x7ff7b80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b8160910 0x7ff7b8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bec00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b8073ad0 con 0x7ff7b81466b0 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b8146a90 con 0x7ff7b80720e0 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b8073d70 con 0x7ff7b8160910 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7b80720e0 0x7ff7b80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7b80720e0 0x7ff7b80cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41614/0 (socket says 172.21.15.46:41614) 2024-04-26T23:48:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.941+0000 7ff7bce00640 1 -- 172.21.15.46:0/688867394 learned_addr learned my addr 172.21.15.46:0/688867394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.942+0000 7ff7be200640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b8160910 0x7ff7b8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.942+0000 7ff7bce00640 1 -- 172.21.15.46:0/688867394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b8160910 msgr2=0x7ff7b8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.942+0000 7ff7bce00640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b8160910 0x7ff7b8160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.942+0000 7ff7bce00640 1 -- 172.21.15.46:0/688867394 >> v1:172.21.15.142:6789/0 conn(0x7ff7b81466b0 legacy=0x7ff7b806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.942+0000 7ff7bce00640 1 -- 172.21.15.46:0/688867394 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8074050 con 0x7ff7b80720e0 2024-04-26T23:48:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.943+0000 7ff7bce00640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b80720e0 0x7ff7b80cd860 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472254b4a146293d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.943+0000 7ff7b7e00640 1 -- 172.21.15.46:0/688867394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4037070 con 0x7ff7b80720e0 2024-04-26T23:48:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.943+0000 7ff7b7e00640 1 -- 172.21.15.46:0/688867394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4032440 con 0x7ff7b80720e0 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.943+0000 7ff7b7e00640 1 -- 172.21.15.46:0/688867394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4032810 con 0x7ff7b80720e0 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 -- 172.21.15.46:0/688867394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b80720e0 msgr2=0x7ff7b80cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b80720e0 0x7ff7b80cd860 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a402a290 comp rx=0 tx=0).stop 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 -- 172.21.15.46:0/688867394 shutdown_connections 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b8160910 0x7ff7b8160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 --2- 172.21.15.46:0/688867394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b80720e0 0x7ff7b80cd860 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 -- 172.21.15.46:0/688867394 >> 172.21.15.46:0/688867394 conn(0x7ff7b80b26d0 msgr2=0x7ff7b80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 -- 172.21.15.46:0/688867394 shutdown_connections 2024-04-26T23:48:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.944+0000 7ff7bec00640 1 -- 172.21.15.46:0/688867394 wait complete. 2024-04-26T23:48:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.946+0000 7ff7bec00640 1 Processor -- start 2024-04-26T23:48:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.946+0000 7ff7bec00640 1 -- start start 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 0x7ff7b815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 0x7ff7b80b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80cdda0 con 0x7ff7b8160910 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80cb470 con 0x7ff7b80720e0 2024-04-26T23:48:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7bec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80cb990 con 0x7ff7b81466b0 2024-04-26T23:48:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7be200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.947+0000 7ff7be200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41628/0 (socket says 172.21.15.46:41628) 2024-04-26T23:48:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 -- 172.21.15.46:0/3515365624 learned_addr learned my addr 172.21.15.46:0/3515365624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7bce00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 0x7ff7b80b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 -- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 msgr2=0x7ff7b80b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7bd800640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 0x7ff7b815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 0x7ff7b80b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 -- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 msgr2=0x7ff7b815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 0x7ff7b815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 -- 172.21.15.46:0/3515365624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80b5d90 con 0x7ff7b8160910 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7bce00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 0x7ff7b80b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7bd800640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 0x7ff7b815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.948+0000 7ff7be200640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cc30 tx=0x7ff7b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.949+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ce40 con 0x7ff7b8160910 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.949+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b0003990 con 0x7ff7b8160910 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.949+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0003d40 con 0x7ff7b8160910 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.949+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b80b6070 con 0x7ff7b8160910 2024-04-26T23:48:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.949+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b80c8a60 con 0x7ff7b8160910 2024-04-26T23:48:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.950+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b80693b0 con 0x7ff7b8160910 2024-04-26T23:48:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.951+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7b0003ee0 con 0x7ff7b8160910 2024-04-26T23:48:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.953+0000 7ff7b5600640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.953+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7ff7b008c160 con 0x7ff7b8160910 2024-04-26T23:48:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.953+0000 7ff7bd800640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.954+0000 7ff7bd800640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 0x7ff79806edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac000f60 tx=0x7ff7ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:50.957+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7b0014030 con 0x7ff7b8160910 2024-04-26T23:48:51.055 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-04-26T23:48:51.049+0000 7fbb23e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:48:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.200+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b80c7180 con 0x7ff7b8160910 2024-04-26T23:48:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.201+0000 7ff7b5600640 1 -- 172.21.15.46:0/3515365624 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7ff7b005a3f0 con 0x7ff7b8160910 2024-04-26T23:48:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:48:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:48:51.207 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 msgr2=0x7ff79806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 0x7ff79806edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac000f60 tx=0x7ff7ac009040 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 msgr2=0x7ff7b80af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cc30 tx=0x7ff7b000a5c0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 shutdown_connections 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b8160910 0x7ff7b80af160 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b81466b0 0x7ff7b80b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 --2- 172.21.15.46:0/3515365624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80720e0 0x7ff7b815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 >> 172.21.15.46:0/3515365624 conn(0x7ff7b80b26d0 msgr2=0x7ff7b80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 shutdown_connections 2024-04-26T23:48:51.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.204+0000 7ff7bec00640 1 -- 172.21.15.46:0/3515365624 wait complete. 2024-04-26T23:48:51.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:48:44.554510+0000', 'modified': '2024-04-26T23:48:47.636866+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8724}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8724': {'gid': 8724, 'name': 'd', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6836/475647056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 475647056}, {'type': 'v1', 'addr': '172.21.15.142:6836', 'nonce': 475647056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8724, 'qdb_cluster': [8724]} 2024-04-26T23:48:51.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:48:51.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T23:48:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.851+0000 7fded0800640 1 Processor -- start 2024-04-26T23:48:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.851+0000 7fded0800640 1 -- start start 2024-04-26T23:48:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.853+0000 7fded0800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8160bd0 0x7fdec8161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.853+0000 7fded0800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdec80732c0 0x7fdec8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.853+0000 7fded0800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec8161940 con 0x7fdec806e510 2024-04-26T23:48:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.853+0000 7fded0800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdec8071f80 con 0x7fdec80732c0 2024-04-26T23:48:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.853+0000 7fded0800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdec8072260 con 0x7fdec8160bd0 2024-04-26T23:48:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdecde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdec806e510 0x7fdec806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57232/0 (socket says 172.21.15.46:57232) 2024-04-26T23:48:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdecde00640 1 -- 172.21.15.46:0/208571371 learned_addr learned my addr 172.21.15.46:0/208571371 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:48:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdece800640 1 --2- 172.21.15.46:0/208571371 >> v2:172.21.15.46:3300/0 conn(0x7fdec80732c0 0x7fdec8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdecd400640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8160bd0 0x7fdec8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdece800640 1 -- 172.21.15.46:0/208571371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8160bd0 msgr2=0x7fdec8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdece800640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8160bd0 0x7fdec8161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdece800640 1 -- 172.21.15.46:0/208571371 >> v1:172.21.15.46:6789/0 conn(0x7fdec806e510 legacy=0x7fdec806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdece800640 1 -- 172.21.15.46:0/208571371 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec8072540 con 0x7fdec80732c0 2024-04-26T23:48:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.854+0000 7fdecca00640 1 -- 172.21.15.46:0/208571371 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321917586 0 0) 0x7fdec8161940 con 0x7fdec806e510 2024-04-26T23:48:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fdece800640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec80732c0 0x7fdec8073750 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009d90 tx=0x7fdec402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09a4d70823ebcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fdecca00640 1 -- 172.21.15.46:0/208571371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007740 con 0x7fdec80732c0 2024-04-26T23:48:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fdecca00640 1 -- 172.21.15.46:0/208571371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec40078e0 con 0x7fdec80732c0 2024-04-26T23:48:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fdecca00640 1 -- 172.21.15.46:0/208571371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007cb0 con 0x7fdec80732c0 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fded0800640 1 -- 172.21.15.46:0/208571371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec80732c0 msgr2=0x7fdec8073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.856+0000 7fded0800640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec80732c0 0x7fdec8073750 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009d90 tx=0x7fdec402a660 comp rx=0 tx=0).stop 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 -- 172.21.15.46:0/208571371 shutdown_connections 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec80732c0 0x7fdec8073750 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 --2- 172.21.15.46:0/208571371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8160bd0 0x7fdec8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 -- 172.21.15.46:0/208571371 >> 172.21.15.46:0/208571371 conn(0x7fdec806c530 msgr2=0x7fdec806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 -- 172.21.15.46:0/208571371 shutdown_connections 2024-04-26T23:48:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.857+0000 7fded0800640 1 -- 172.21.15.46:0/208571371 wait complete. 2024-04-26T23:48:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.858+0000 7fded0800640 1 Processor -- start 2024-04-26T23:48:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 -- start start 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec806e510 0x7fdec8145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec80732c0 0x7fdec8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81616f0 con 0x7fdec8160bd0 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8161570 con 0x7fdec806e510 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.859+0000 7fded0800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdec8072d40 con 0x7fdec80732c0 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58088/0 (socket says 172.21.15.46:58088) 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 -- 172.21.15.46:0/873253398 learned_addr learned my addr 172.21.15.46:0/873253398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 -- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec80732c0 msgr2=0x7fdec8155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec80732c0 0x7fdec8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 -- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec806e510 msgr2=0x7fdec8145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec806e510 0x7fdec8145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.860+0000 7fdece800640 1 -- 172.21.15.46:0/873253398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec8145e00 con 0x7fdec8160bd0 2024-04-26T23:48:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fdece800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdec402ac50 tx=0x7fdec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fdec2000640 1 -- 172.21.15.46:0/873253398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4040070 con 0x7fdec8160bd0 2024-04-26T23:48:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fdec2000640 1 -- 172.21.15.46:0/873253398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec4031bc0 con 0x7fdec8160bd0 2024-04-26T23:48:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fdec2000640 1 -- 172.21.15.46:0/873253398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4045460 con 0x7fdec8160bd0 2024-04-26T23:48:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81f2e20 con 0x7fdec8160bd0 2024-04-26T23:48:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.861+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81f33e0 con 0x7fdec8160bd0 2024-04-26T23:48:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.864+0000 7fdec2000640 1 -- 172.21.15.46:0/873253398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec4039040 con 0x7fdec8160bd0 2024-04-26T23:48:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.865+0000 7fdec2000640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 0x7fdea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.866+0000 7fdec2000640 1 -- 172.21.15.46:0/873253398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fdec40ad730 con 0x7fdec8160bd0 2024-04-26T23:48:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.866+0000 7fded35ca740 1 Processor -- start 2024-04-26T23:48:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.866+0000 7fded35ca740 1 -- start start 2024-04-26T23:48:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.866+0000 7fdecde00640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 0x7fdea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8aaed5e0 0x563c8aaef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab180a0 0x563c8ab18520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c8aa51600 con 0x563c8ab29cb0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c8a9ee100 con 0x563c8aaed5e0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563c8aa4d5c0 con 0x563c8ab180a0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58100/0 (socket says 172.21.15.46:58100) 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 -- 172.21.15.46:0/1824109839 learned_addr learned my addr 172.21.15.46:0/1824109839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 -- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab180a0 msgr2=0x563c8ab18520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab180a0 0x563c8ab18520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 -- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8aaed5e0 msgr2=0x563c8aaef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8aaed5e0 0x563c8aaef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.867+0000 7fdecd400640 1 -- 172.21.15.46:0/1824109839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8a7632d0 con 0x563c8ab29cb0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.868+0000 7fdecde00640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 0x7fdea806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc001a10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.868+0000 7fdecd400640 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4007780 tx=0x7fdeb4007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b040d1921dfd0c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.868+0000 7fdeba600640 1 -- 172.21.15.46:0/1824109839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400d870 con 0x563c8ab29cb0 2024-04-26T23:48:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.868+0000 7fdeba600640 1 -- 172.21.15.46:0/1824109839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4018070 con 0x563c8ab29cb0 2024-04-26T23:48:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.868+0000 7fdeba600640 1 -- 172.21.15.46:0/1824109839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400dc50 con 0x563c8ab29cb0 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 -- 172.21.15.46:0/1824109839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 msgr2=0x563c8aaed9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4007780 tx=0x7fdeb4007c40 comp rx=0 tx=0).stop 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 -- 172.21.15.46:0/1824109839 shutdown_connections 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab180a0 0x563c8ab18520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab29cb0 0x563c8aaed9c0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 --2- 172.21.15.46:0/1824109839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8aaed5e0 0x563c8aaef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 -- 172.21.15.46:0/1824109839 >> 172.21.15.46:0/1824109839 conn(0x563c8aa81d10 msgr2=0x563c8ab330a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 -- 172.21.15.46:0/1824109839 shutdown_connections 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.869+0000 7fded35ca740 1 -- 172.21.15.46:0/1824109839 wait complete. 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.870+0000 7fded35ca740 1 -- start start 2024-04-26T23:48:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.870+0000 7fded35ca740 1 Processor -- start 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c8a9d8f70 con 0x563c8ab180a0 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c8ab15bf0 con 0x563c8ab29cb0 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563c8ab196f0 con 0x563c8aaed5e0 2024-04-26T23:48:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fded35ca740 10 client.0 _ll_register_callbacks cb 0x563c8aa47480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58108/0 (socket says 172.21.15.46:58108) 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fdecd400640 1 -- 172.21.15.46:0/2850758631 learned_addr learned my addr 172.21.15.46:0/2850758631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.871+0000 7fdecde00640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdece800640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 -- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 msgr2=0x563c8ab16da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab16da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 -- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 msgr2=0x563c8ab2a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab2a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 -- 172.21.15.46:0/2850758631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8a82d6b0 con 0x563c8ab180a0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdece800640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab2a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdecd400640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4015040 tx=0x7fdeb4006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd39174da97119f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdeaea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4021070 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4018070 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.872+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fdeb400da70 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fdeb402e050 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 10 client.8859 fetch_fsmap learned FSMap version 100 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x563c8ab19890 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4018070 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fdeb4031020 con 0x563c8ab180a0 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 10 client.8859 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 10 client.8859 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 20 client.8859 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8ab12da0 0x563c8ab31c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 4 client.8859 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:48:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.873+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c8aa5f210 con 0x563c8ab12da0 2024-04-26T23:48:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.874+0000 7fdecde00640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8ab12da0 0x563c8ab31c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.875+0000 7fdecde00640 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8ab12da0 0x563c8ab31c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc83bbf4677055b server_cookie=2d53e18e86abd07c in_seq=0 out_seq=0 2024-04-26T23:48:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.875+0000 7fdeaea00640 10 client.8859 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:48:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.877+0000 7fdeaea00640 1 -- 172.21.15.46:0/2850758631 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c8aa5f210 con 0x563c8ab12da0 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.877+0000 7fdeaea00640 10 client.8859 handle_command_reply: tid=0 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.877+0000 7fded35ca740 1 client.8859 shutdown 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 msgr2=0x563c8ab172e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4015040 tx=0x7fdeb4006e10 comp rx=0 tx=0).stop 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 shutdown_connections 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8ab12da0 0x563c8ab31c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab2a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab172e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 --2- 172.21.15.46:0/2850758631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab16da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fdeaea00640 0 client.8859 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 >> 172.21.15.46:0/2850758631 conn(0x563c8aa81d10 msgr2=0x563c8ab361e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 shutdown_connections 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 1 -- 172.21.15.46:0/2850758631 wait complete. 2024-04-26T23:48:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.878+0000 7fded35ca740 20 client.8859 trim_cache size 0 max 16384 2024-04-26T23:48:51.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.898+0000 7fded35ca740 1 Processor -- start 2024-04-26T23:48:51.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.898+0000 7fded35ca740 1 -- start start 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab2a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81fc820 con 0x563c8ab180a0 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdec80b5aa0 con 0x563c8ab29cb0 2024-04-26T23:48:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdec81fc230 con 0x563c8aaed5e0 2024-04-26T23:48:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.899+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58124/0 (socket says 172.21.15.46:58124) 2024-04-26T23:48:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 -- 172.21.15.46:0/1262268518 learned_addr learned my addr 172.21.15.46:0/1262268518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecde00640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab2a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdece800640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 -- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 msgr2=0x563c8ab2a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab2a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 -- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 msgr2=0x563c8ab467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 -- 172.21.15.46:0/1262268518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8aae48a0 con 0x563c8ab180a0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdece800640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.900+0000 7fdecd400640 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40076a0 tx=0x7fdeb401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e4446f2300ed31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fdeb9c00640 1 -- 172.21.15.46:0/1262268518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4037070 con 0x563c8ab180a0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fdeb9c00640 1 -- 172.21.15.46:0/1262268518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4018040 con 0x563c8ab180a0 2024-04-26T23:48:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fdeb9c00640 1 -- 172.21.15.46:0/1262268518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4021070 con 0x563c8ab180a0 2024-04-26T23:48:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 -- 172.21.15.46:0/1262268518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 msgr2=0x563c8ab5b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40076a0 tx=0x7fdeb401e000 comp rx=0 tx=0).stop 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 -- 172.21.15.46:0/1262268518 shutdown_connections 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab29cb0 0x563c8ab467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8ab180a0 0x563c8ab5b250 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 --2- 172.21.15.46:0/1262268518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8aaed5e0 0x563c8ab2a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.901+0000 7fded35ca740 1 -- 172.21.15.46:0/1262268518 >> 172.21.15.46:0/1262268518 conn(0x563c8aa81d10 msgr2=0x563c8aae3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.902+0000 7fded35ca740 1 -- 172.21.15.46:0/1262268518 shutdown_connections 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.902+0000 7fded35ca740 1 -- 172.21.15.46:0/1262268518 wait complete. 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.902+0000 7fded35ca740 1 -- start start 2024-04-26T23:48:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.902+0000 7fded35ca740 1 Processor -- start 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab180a0 0x563c8ab104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab29cb0 0x563c8ab361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c8ab2a330 con 0x563c8aaed5e0 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c8ab476f0 con 0x563c8ab180a0 2024-04-26T23:48:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563c8ab2aef0 con 0x563c8ab29cb0 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.903+0000 7fded35ca740 10 client.0 _ll_register_callbacks cb 0x7fdeb40075d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58134/0 (socket says 172.21.15.46:58134) 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 -- 172.21.15.46:0/3712626546 learned_addr learned my addr 172.21.15.46:0/3712626546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 -- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab29cb0 msgr2=0x563c8ab361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab29cb0 0x563c8ab361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 -- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab180a0 msgr2=0x563c8ab104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab180a0 0x563c8ab104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 -- 172.21.15.46:0/3712626546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8aaec6b0 con 0x563c8aaed5e0 2024-04-26T23:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdecde00640 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007330 tx=0x7fdebc0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866562a13ac93af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.904+0000 7fdeaea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc010040 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fdebc0269b0 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4018040 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fdebc026020 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 10 client.8865 fetch_fsmap learned FSMap version 100 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x563c8ab15380 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4018040 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fdebc026bb0 con 0x563c8aaed5e0 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 10 client.8865 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 10 client.8865 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 20 client.8865 populate_metadata read hostname 'smithi046' 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8abb91f0 0x563c8abb95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 4 client.8865 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:48:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.905+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "status"}) -- 0x563c8aa59080 con 0x563c8abb91f0 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.906+0000 7fdecd400640 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8abb91f0 0x563c8abb95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.907+0000 7fdecd400640 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8abb91f0 0x563c8abb95d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b809bf92308d55 server_cookie=dd73ca44c3fd3a97 in_seq=0 out_seq=0 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.908+0000 7fdeaea00640 10 client.8865 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.908+0000 7fdeaea00640 1 -- 172.21.15.46:0/3712626546 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x563c8aa59080 con 0x563c8abb91f0 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.908+0000 7fdeaea00640 10 client.8865 handle_command_reply: tid=0 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.908+0000 7fded35ca740 1 client.8865 shutdown 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 msgr2=0x563c8ab0ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007330 tx=0x7fdebc0010f0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 shutdown_connections 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x563c8abb91f0 0x563c8abb95d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fdeaea00640 0 client.8865 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563c8ab29cb0 0x563c8ab361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c8ab180a0 0x563c8ab104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 --2- 172.21.15.46:0/3712626546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c8aaed5e0 0x563c8ab0ff70 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 >> 172.21.15.46:0/3712626546 conn(0x563c8aa81d10 msgr2=0x563c8aae7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 shutdown_connections 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 1 -- 172.21.15.46:0/3712626546 wait complete. 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.909+0000 7fded35ca740 20 client.8865 trim_cache size 0 max 16384 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:48:51.915 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8724, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.8040458250000002, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 100, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 126, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 126, 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.187023347 2024-04-26T23:48:51.916 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 msgr2=0x7fdea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 0x7fdea806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc001a10 comp rx=0 tx=0).stop 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 msgr2=0x7fdec813c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdec402ac50 tx=0x7fdec4004a40 comp rx=0 tx=0).stop 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 shutdown_connections 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdea806c8f0 0x7fdea806edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec8160bd0 0x7fdec813c800 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec80732c0 0x7fdec8155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 --2- 172.21.15.46:0/873253398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec806e510 0x7fdec8145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 >> 172.21.15.46:0/873253398 conn(0x7fdec806c530 msgr2=0x7fdec815d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 shutdown_connections 2024-04-26T23:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:51.910+0000 7fded0800640 1 -- 172.21.15.46:0/873253398 wait complete. 2024-04-26T23:48:51.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:48:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.128+0000 7f32fd800640 1 Processor -- start 2024-04-26T23:48:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.129+0000 7f32fd800640 1 -- start start 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f80d1990 0x7f32f80d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f32f80d0c10 0x7f32f80d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f81639c0 0x7f32f80d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f32f80d0780 con 0x7f32f80d1990 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f32f80d22b0 con 0x7f32f80d0c10 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32fd800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32f80cfe50 con 0x7f32f81639c0 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f80d1990 0x7f32f80d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f80d1990 0x7f32f80d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58150/0 (socket says 172.21.15.46:58150) 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.130+0000 7f32f7e00640 1 -- 172.21.15.46:0/2766387300 learned_addr learned my addr 172.21.15.46:0/2766387300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.131+0000 7f32f7e00640 1 -- 172.21.15.46:0/2766387300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f81639c0 msgr2=0x7f32f80d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.131+0000 7f32f7e00640 1 --2- 172.21.15.46:0/2766387300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f81639c0 0x7f32f80d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.131+0000 7f32f7e00640 1 -- 172.21.15.46:0/2766387300 >> v2:172.21.15.142:3300/0 conn(0x7f32f80d0c10 msgr2=0x7f32f80d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.131+0000 7f32f7e00640 1 --2- 172.21.15.46:0/2766387300 >> v2:172.21.15.142:3300/0 conn(0x7f32f80d0c10 0x7f32f80d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.131+0000 7f32f7e00640 1 -- 172.21.15.46:0/2766387300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f80d0130 con 0x7f32f80d1990 2024-04-26T23:48:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.132+0000 7f32f7e00640 1 --2- 172.21.15.46:0/2766387300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d1990 0x7f32f80d1d70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f32f80c8720 tx=0x7f32e802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c04e38219d6f036 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.132+0000 7f32f7400640 1 -- 172.21.15.46:0/2766387300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037070 con 0x7f32f80d1990 2024-04-26T23:48:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.132+0000 7f32f7400640 1 -- 172.21.15.46:0/2766387300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e8032440 con 0x7f32f80d1990 2024-04-26T23:48:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.132+0000 7f32f7400640 1 -- 172.21.15.46:0/2766387300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032810 con 0x7f32f80d1990 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 -- 172.21.15.46:0/2766387300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d1990 msgr2=0x7f32f80d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 --2- 172.21.15.46:0/2766387300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d1990 0x7f32f80d1d70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f32f80c8720 tx=0x7f32e802a200 comp rx=0 tx=0).stop 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 -- 172.21.15.46:0/2766387300 shutdown_connections 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 --2- 172.21.15.46:0/2766387300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f81639c0 0x7f32f80d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 --2- 172.21.15.46:0/2766387300 >> v2:172.21.15.142:3300/0 conn(0x7f32f80d0c10 0x7f32f80d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 --2- 172.21.15.46:0/2766387300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d1990 0x7f32f80d1d70 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 -- 172.21.15.46:0/2766387300 >> 172.21.15.46:0/2766387300 conn(0x7f32f809e2a0 msgr2=0x7f32f80d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 -- 172.21.15.46:0/2766387300 shutdown_connections 2024-04-26T23:48:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.133+0000 7f32fd800640 1 -- 172.21.15.46:0/2766387300 wait complete. 2024-04-26T23:48:52.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.135+0000 7f32fd800640 1 Processor -- start 2024-04-26T23:48:52.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.135+0000 7f32fd800640 1 -- start start 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80d1990 0x7f32f80c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f81639c0 0x7f32f80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32f80cfa90 con 0x7f32f80d0c10 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32f815f530 con 0x7f32f81639c0 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32fd800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32f80cfc10 con 0x7f32f80d1990 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58154/0 (socket says 172.21.15.46:58154) 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.136+0000 7f32f7e00640 1 -- 172.21.15.46:0/1190293347 learned_addr learned my addr 172.21.15.46:0/1190293347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 -- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80d1990 msgr2=0x7f32f80c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80d1990 0x7f32f80c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 -- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f81639c0 msgr2=0x7f32f80cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f81639c0 0x7f32f80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 -- 172.21.15.46:0/1190293347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f80ccf50 con 0x7f32f80d0c10 2024-04-26T23:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f7e00640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f32e80090b0 tx=0x7f32e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.137+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8041070 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.138+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e803c870 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.138+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037040 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.138+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f80cd1d0 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.138+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f80cd790 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.139+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f80693b0 con 0x7f32f80d0c10 2024-04-26T23:48:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.140+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32e8044070 con 0x7f32f80d0c10 2024-04-26T23:48:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.141+0000 7f32f4c00640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 0x7f32d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.142+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f32e80ac9a0 con 0x7f32f80d0c10 2024-04-26T23:48:52.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.142+0000 7f32ef400640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 0x7f32d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.143+0000 7f32ef400640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 0x7f32d406edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ce030 tx=0x7f32e0006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.146+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32f80ccf50 con 0x7f32f80d0c10 2024-04-26T23:48:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.397+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f32f80cf080 con 0x7f32f80d0c10 2024-04-26T23:48:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.398+0000 7f32f4c00640 1 -- 172.21.15.46:0/1190293347 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f32e807acc0 con 0x7f32f80d0c10 2024-04-26T23:48:52.402 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:48:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 msgr2=0x7f32d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 0x7f32d406edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ce030 tx=0x7f32e0006c90 comp rx=0 tx=0).stop 2024-04-26T23:48:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 msgr2=0x7f32f81577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f32e80090b0 tx=0x7f32e80046c0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 shutdown_connections 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32d406c8f0 0x7f32d406edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f81639c0 0x7f32f80cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80d1990 0x7f32f80c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 --2- 172.21.15.46:0/1190293347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f80d0c10 0x7f32f81577d0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 >> 172.21.15.46:0/1190293347 conn(0x7f32f809e2a0 msgr2=0x7f32f815f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.400+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 shutdown_connections 2024-04-26T23:48:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52.401+0000 7f32fd800640 1 -- 172.21.15.46:0/1190293347 wait complete. 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:48:52.425 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:48:52.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:52.426 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:48:52.480 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:48:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:48:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:48:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:48:52.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:52.485 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:48:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:48:52.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:52.573 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:48:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:48:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:48:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:48:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:48:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:48:52.631 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:48:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:52.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:52.851 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:48:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:48:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:48:52.904 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:48:52.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:52.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:48:52.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:48:52.994 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:48:53.049 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:48:53.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:48:53.050 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:48:53.103 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:48:53.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:53.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:48:53.241 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:48:53.241 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:48:53.241 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:48:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:48:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:48:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:48:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:48:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:48:53.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:48:53.293 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:48:53.293 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:48:53.293 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:48:53.293 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:48:53.293 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:48:53.297 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:48:53.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:48:53.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:48:53.376 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:48:53.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:53.531 INFO:teuthology.orchestra.run.smithi046.stdout:client5942 2024-04-26T23:48:53.532 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:48:53.532 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.0 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:53.589 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389468 s, 26.9 MB/s 2024-04-26T23:48:53.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:53.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:53.844 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454017 s, 23.1 MB/s 2024-04-26T23:48:53.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:53.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:53.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401306 s, 26.1 MB/s 2024-04-26T23:48:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397614 s, 26.4 MB/s 2024-04-26T23:48:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290547 s, 36.1 MB/s 2024-04-26T23:48:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380071 s, 27.6 MB/s 2024-04-26T23:48:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402164 s, 26.1 MB/s 2024-04-26T23:48:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359279 s, 29.2 MB/s 2024-04-26T23:48:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315691 s, 33.2 MB/s 2024-04-26T23:48:54.189 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.189 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.189 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271523 s, 38.6 MB/s 2024-04-26T23:48:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278713 s, 37.6 MB/s 2024-04-26T23:48:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283825 s, 36.9 MB/s 2024-04-26T23:48:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260121 s, 40.3 MB/s 2024-04-26T23:48:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327102 s, 32.1 MB/s 2024-04-26T23:48:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270601 s, 38.7 MB/s 2024-04-26T23:48:54.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.419 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416971 s, 25.1 MB/s 2024-04-26T23:48:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455371 s, 23.0 MB/s 2024-04-26T23:48:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435709 s, 24.1 MB/s 2024-04-26T23:48:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296682 s, 35.3 MB/s 2024-04-26T23:48:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374881 s, 28.0 MB/s 2024-04-26T23:48:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320937 s, 32.7 MB/s 2024-04-26T23:48:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258969 s, 40.5 MB/s 2024-04-26T23:48:54.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.711 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024846 s, 42.2 MB/s 2024-04-26T23:48:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030212 s, 34.7 MB/s 2024-04-26T23:48:54.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.778 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213175 s, 49.2 MB/s 2024-04-26T23:48:54.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325387 s, 32.2 MB/s 2024-04-26T23:48:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255996 s, 41.0 MB/s 2024-04-26T23:48:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252891 s, 41.5 MB/s 2024-04-26T23:48:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.916 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282762 s, 37.1 MB/s 2024-04-26T23:48:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249672 s, 42.0 MB/s 2024-04-26T23:48:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232591 s, 45.1 MB/s 2024-04-26T23:48:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305138 s, 34.4 MB/s 2024-04-26T23:48:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.1 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:55.027 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:55.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027467 s, 38.2 MB/s 2024-04-26T23:48:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240306 s, 43.6 MB/s 2024-04-26T23:48:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244235 s, 42.9 MB/s 2024-04-26T23:48:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299761 s, 35.0 MB/s 2024-04-26T23:48:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254954 s, 41.1 MB/s 2024-04-26T23:48:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334306 s, 31.4 MB/s 2024-04-26T23:48:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251182 s, 41.7 MB/s 2024-04-26T23:48:55.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259891 s, 40.3 MB/s 2024-04-26T23:48:55.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257541 s, 40.7 MB/s 2024-04-26T23:48:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306589 s, 34.2 MB/s 2024-04-26T23:48:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317453 s, 33.0 MB/s 2024-04-26T23:48:55.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.558 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246042 s, 42.6 MB/s 2024-04-26T23:48:55.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.590 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262454 s, 40.0 MB/s 2024-04-26T23:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224902 s, 46.6 MB/s 2024-04-26T23:48:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305919 s, 34.3 MB/s 2024-04-26T23:48:55.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.686 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232414 s, 45.1 MB/s 2024-04-26T23:48:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245811 s, 42.7 MB/s 2024-04-26T23:48:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446435 s, 23.5 MB/s 2024-04-26T23:48:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022175 s, 47.3 MB/s 2024-04-26T23:48:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343322 s, 30.5 MB/s 2024-04-26T23:48:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276632 s, 37.9 MB/s 2024-04-26T23:48:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281916 s, 37.2 MB/s 2024-04-26T23:48:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022593 s, 46.4 MB/s 2024-04-26T23:48:55.966 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229226 s, 45.7 MB/s 2024-04-26T23:48:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236642 s, 44.3 MB/s 2024-04-26T23:48:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236979 s, 44.2 MB/s 2024-04-26T23:48:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248573 s, 42.2 MB/s 2024-04-26T23:48:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412626 s, 25.4 MB/s 2024-04-26T23:48:56.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.140 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028237 s, 37.1 MB/s 2024-04-26T23:48:56.170 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.170 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.170 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238894 s, 43.9 MB/s 2024-04-26T23:48:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241452 s, 43.4 MB/s 2024-04-26T23:48:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251308 s, 41.7 MB/s 2024-04-26T23:48:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:56.246 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:56.246 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:56.246 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.2 2024-04-26T23:48:56.246 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:56.247 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:56.247 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:56.247 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:56.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234115 s, 44.8 MB/s 2024-04-26T23:48:56.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288551 s, 36.3 MB/s 2024-04-26T23:48:56.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273101 s, 38.4 MB/s 2024-04-26T23:48:56.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.500 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244499 s, 42.9 MB/s 2024-04-26T23:48:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222512 s, 47.1 MB/s 2024-04-26T23:48:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259311 s, 40.4 MB/s 2024-04-26T23:48:56.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.595 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246733 s, 42.5 MB/s 2024-04-26T23:48:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227498 s, 46.1 MB/s 2024-04-26T23:48:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243526 s, 43.1 MB/s 2024-04-26T23:48:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207206 s, 50.6 MB/s 2024-04-26T23:48:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230835 s, 45.4 MB/s 2024-04-26T23:48:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244339 s, 42.9 MB/s 2024-04-26T23:48:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309416 s, 33.9 MB/s 2024-04-26T23:48:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308291 s, 34.0 MB/s 2024-04-26T23:48:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246579 s, 42.5 MB/s 2024-04-26T23:48:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261745 s, 40.1 MB/s 2024-04-26T23:48:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274278 s, 38.2 MB/s 2024-04-26T23:48:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218481 s, 48.0 MB/s 2024-04-26T23:48:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230144 s, 45.6 MB/s 2024-04-26T23:48:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276106 s, 38.0 MB/s 2024-04-26T23:48:57.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.043 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240888 s, 43.5 MB/s 2024-04-26T23:48:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307286 s, 34.1 MB/s 2024-04-26T23:48:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229984 s, 45.6 MB/s 2024-04-26T23:48:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267861 s, 39.1 MB/s 2024-04-26T23:48:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225695 s, 46.5 MB/s 2024-04-26T23:48:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267717 s, 39.2 MB/s 2024-04-26T23:48:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250942 s, 41.8 MB/s 2024-04-26T23:48:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023604 s, 44.4 MB/s 2024-04-26T23:48:57.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.300 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244518 s, 42.9 MB/s 2024-04-26T23:48:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221836 s, 47.3 MB/s 2024-04-26T23:48:57.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.363 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260906 s, 40.2 MB/s 2024-04-26T23:48:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283435 s, 37.0 MB/s 2024-04-26T23:48:57.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.3 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:57.412 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318675 s, 32.9 MB/s 2024-04-26T23:48:57.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269366 s, 38.9 MB/s 2024-04-26T23:48:57.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262081 s, 40.0 MB/s 2024-04-26T23:48:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225692 s, 46.5 MB/s 2024-04-26T23:48:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216045 s, 48.5 MB/s 2024-04-26T23:48:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285408 s, 36.7 MB/s 2024-04-26T23:48:57.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292533 s, 35.8 MB/s 2024-04-26T23:48:57.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275638 s, 38.0 MB/s 2024-04-26T23:48:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022512 s, 46.6 MB/s 2024-04-26T23:48:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240885 s, 43.5 MB/s 2024-04-26T23:48:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257029 s, 40.8 MB/s 2024-04-26T23:48:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264038 s, 39.7 MB/s 2024-04-26T23:48:57.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.953 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022679 s, 46.2 MB/s 2024-04-26T23:48:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217428 s, 48.2 MB/s 2024-04-26T23:48:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202886 s, 51.7 MB/s 2024-04-26T23:48:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248036 s, 42.3 MB/s 2024-04-26T23:48:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231339 s, 45.3 MB/s 2024-04-26T23:48:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216348 s, 48.5 MB/s 2024-04-26T23:48:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241313 s, 43.5 MB/s 2024-04-26T23:48:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239395 s, 43.8 MB/s 2024-04-26T23:48:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223623 s, 46.9 MB/s 2024-04-26T23:48:58.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.218 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227485 s, 46.1 MB/s 2024-04-26T23:48:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023478 s, 44.7 MB/s 2024-04-26T23:48:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170165 s, 61.6 MB/s 2024-04-26T23:48:58.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262755 s, 39.9 MB/s 2024-04-26T23:48:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022021 s, 47.6 MB/s 2024-04-26T23:48:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233462 s, 44.9 MB/s 2024-04-26T23:48:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216855 s, 48.4 MB/s 2024-04-26T23:48:58.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.419 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02261 s, 46.4 MB/s 2024-04-26T23:48:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.451 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245399 s, 42.7 MB/s 2024-04-26T23:48:58.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304813 s, 34.4 MB/s 2024-04-26T23:48:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245615 s, 42.7 MB/s 2024-04-26T23:48:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.4 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:58.531 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222992 s, 47.0 MB/s 2024-04-26T23:48:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232743 s, 45.1 MB/s 2024-04-26T23:48:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251908 s, 41.6 MB/s 2024-04-26T23:48:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248587 s, 42.2 MB/s 2024-04-26T23:48:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025233 s, 41.6 MB/s 2024-04-26T23:48:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211422 s, 49.6 MB/s 2024-04-26T23:48:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232376 s, 45.1 MB/s 2024-04-26T23:48:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258484 s, 40.6 MB/s 2024-04-26T23:48:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243665 s, 43.0 MB/s 2024-04-26T23:48:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263744 s, 39.8 MB/s 2024-04-26T23:48:58.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:58.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:58.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234166 s, 44.8 MB/s 2024-04-26T23:48:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227257 s, 46.1 MB/s 2024-04-26T23:48:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219019 s, 47.9 MB/s 2024-04-26T23:48:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207657 s, 50.5 MB/s 2024-04-26T23:48:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293852 s, 35.7 MB/s 2024-04-26T23:48:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244879 s, 42.8 MB/s 2024-04-26T23:48:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225691 s, 46.5 MB/s 2024-04-26T23:48:59.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221626 s, 47.3 MB/s 2024-04-26T23:48:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178337 s, 58.8 MB/s 2024-04-26T23:48:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232323 s, 45.1 MB/s 2024-04-26T23:48:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219548 s, 47.8 MB/s 2024-04-26T23:48:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247171 s, 42.4 MB/s 2024-04-26T23:48:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230561 s, 45.5 MB/s 2024-04-26T23:48:59.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279836 s, 37.5 MB/s 2024-04-26T23:48:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290976 s, 36.0 MB/s 2024-04-26T23:48:59.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.439 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259061 s, 40.5 MB/s 2024-04-26T23:48:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207414 s, 50.6 MB/s 2024-04-26T23:48:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021559 s, 48.6 MB/s 2024-04-26T23:48:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252785 s, 41.5 MB/s 2024-04-26T23:48:59.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219053 s, 47.9 MB/s 2024-04-26T23:48:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256272 s, 40.9 MB/s 2024-04-26T23:48:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310922 s, 33.7 MB/s 2024-04-26T23:48:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.5 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:48:59.638 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:48:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:48:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:48:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231006 s, 45.4 MB/s 2024-04-26T23:48:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024215 s, 43.3 MB/s 2024-04-26T23:48:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309343 s, 33.9 MB/s 2024-04-26T23:48:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225772 s, 46.4 MB/s 2024-04-26T23:48:59.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:48:59.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:48:59.999 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026986 s, 38.9 MB/s 2024-04-26T23:49:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247708 s, 42.3 MB/s 2024-04-26T23:49:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235289 s, 44.6 MB/s 2024-04-26T23:49:00.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203071 s, 51.6 MB/s 2024-04-26T23:49:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250091 s, 41.9 MB/s 2024-04-26T23:49:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271519 s, 38.6 MB/s 2024-04-26T23:49:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205746 s, 51.0 MB/s 2024-04-26T23:49:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286527 s, 36.6 MB/s 2024-04-26T23:49:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239884 s, 43.7 MB/s 2024-04-26T23:49:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.273 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214294 s, 48.9 MB/s 2024-04-26T23:49:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233399 s, 44.9 MB/s 2024-04-26T23:49:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210686 s, 49.8 MB/s 2024-04-26T23:49:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228391 s, 45.9 MB/s 2024-04-26T23:49:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263299 s, 39.8 MB/s 2024-04-26T23:49:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295613 s, 35.5 MB/s 2024-04-26T23:49:00.460 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.460 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.460 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216205 s, 48.5 MB/s 2024-04-26T23:49:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236216 s, 44.4 MB/s 2024-04-26T23:49:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235221 s, 44.6 MB/s 2024-04-26T23:49:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283336 s, 37.0 MB/s 2024-04-26T23:49:00.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210746 s, 49.8 MB/s 2024-04-26T23:49:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266107 s, 39.4 MB/s 2024-04-26T23:49:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238906 s, 43.9 MB/s 2024-04-26T23:49:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298683 s, 35.1 MB/s 2024-04-26T23:49:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271907 s, 38.6 MB/s 2024-04-26T23:49:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312141 s, 33.6 MB/s 2024-04-26T23:49:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211529 s, 49.6 MB/s 2024-04-26T23:49:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242883 s, 43.2 MB/s 2024-04-26T23:49:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229745 s, 45.6 MB/s 2024-04-26T23:49:00.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.6 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:00.857 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208315 s, 50.3 MB/s 2024-04-26T23:49:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022318 s, 47.0 MB/s 2024-04-26T23:49:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239306 s, 43.8 MB/s 2024-04-26T23:49:01.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.092 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273508 s, 38.3 MB/s 2024-04-26T23:49:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235818 s, 44.5 MB/s 2024-04-26T23:49:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228055 s, 46.0 MB/s 2024-04-26T23:49:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021832 s, 48.0 MB/s 2024-04-26T23:49:01.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291792 s, 35.9 MB/s 2024-04-26T23:49:01.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.249 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023791 s, 44.1 MB/s 2024-04-26T23:49:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025223 s, 41.6 MB/s 2024-04-26T23:49:01.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.318 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293971 s, 35.7 MB/s 2024-04-26T23:49:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291317 s, 36.0 MB/s 2024-04-26T23:49:01.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.384 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234905 s, 44.6 MB/s 2024-04-26T23:49:01.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291566 s, 36.0 MB/s 2024-04-26T23:49:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219363 s, 47.8 MB/s 2024-04-26T23:49:01.477 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.477 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.477 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229079 s, 45.8 MB/s 2024-04-26T23:49:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332011 s, 31.6 MB/s 2024-04-26T23:49:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229438 s, 45.7 MB/s 2024-04-26T23:49:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246932 s, 42.5 MB/s 2024-04-26T23:49:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241321 s, 43.5 MB/s 2024-04-26T23:49:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270596 s, 38.8 MB/s 2024-04-26T23:49:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252785 s, 41.5 MB/s 2024-04-26T23:49:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254353 s, 41.2 MB/s 2024-04-26T23:49:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217092 s, 48.3 MB/s 2024-04-26T23:49:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222804 s, 47.1 MB/s 2024-04-26T23:49:01.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.789 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206276 s, 50.8 MB/s 2024-04-26T23:49:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224314 s, 46.7 MB/s 2024-04-26T23:49:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021737 s, 48.2 MB/s 2024-04-26T23:49:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235367 s, 44.6 MB/s 2024-04-26T23:49:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253219 s, 41.4 MB/s 2024-04-26T23:49:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245672 s, 42.7 MB/s 2024-04-26T23:49:01.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023366 s, 44.9 MB/s 2024-04-26T23:49:01.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.7 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:01.983 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:02.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:02.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299035 s, 35.1 MB/s 2024-04-26T23:49:02.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029117 s, 36.0 MB/s 2024-04-26T23:49:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294514 s, 35.6 MB/s 2024-04-26T23:49:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247701 s, 42.3 MB/s 2024-04-26T23:49:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027849 s, 37.7 MB/s 2024-04-26T23:49:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298962 s, 35.1 MB/s 2024-04-26T23:49:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214886 s, 48.8 MB/s 2024-04-26T23:49:02.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.375 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234221 s, 44.8 MB/s 2024-04-26T23:49:02.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.400 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178498 s, 58.7 MB/s 2024-04-26T23:49:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276715 s, 37.9 MB/s 2024-04-26T23:49:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337107 s, 31.1 MB/s 2024-04-26T23:49:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254717 s, 41.2 MB/s 2024-04-26T23:49:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239534 s, 43.8 MB/s 2024-04-26T23:49:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247107 s, 42.4 MB/s 2024-04-26T23:49:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215058 s, 48.8 MB/s 2024-04-26T23:49:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262603 s, 39.9 MB/s 2024-04-26T23:49:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219276 s, 47.8 MB/s 2024-04-26T23:49:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221516 s, 47.3 MB/s 2024-04-26T23:49:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260052 s, 40.3 MB/s 2024-04-26T23:49:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02368 s, 44.3 MB/s 2024-04-26T23:49:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203716 s, 51.5 MB/s 2024-04-26T23:49:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226552 s, 46.3 MB/s 2024-04-26T23:49:02.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.853 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283892 s, 36.9 MB/s 2024-04-26T23:49:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027738 s, 37.8 MB/s 2024-04-26T23:49:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289311 s, 36.2 MB/s 2024-04-26T23:49:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027959 s, 37.5 MB/s 2024-04-26T23:49:02.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:02.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:02.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236236 s, 44.4 MB/s 2024-04-26T23:49:03.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.025 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323047 s, 32.5 MB/s 2024-04-26T23:49:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235394 s, 44.5 MB/s 2024-04-26T23:49:03.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.083 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021927 s, 47.8 MB/s 2024-04-26T23:49:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223365 s, 46.9 MB/s 2024-04-26T23:49:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242462 s, 43.2 MB/s 2024-04-26T23:49:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.8 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:03.157 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:03.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:03.305 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.305 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.305 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249978 s, 41.9 MB/s 2024-04-26T23:49:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314213 s, 33.4 MB/s 2024-04-26T23:49:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028445 s, 36.9 MB/s 2024-04-26T23:49:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216722 s, 48.4 MB/s 2024-04-26T23:49:03.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238535 s, 44.0 MB/s 2024-04-26T23:49:03.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024369 s, 43.0 MB/s 2024-04-26T23:49:03.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252087 s, 41.6 MB/s 2024-04-26T23:49:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303407 s, 34.6 MB/s 2024-04-26T23:49:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247003 s, 42.5 MB/s 2024-04-26T23:49:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231932 s, 45.2 MB/s 2024-04-26T23:49:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254828 s, 41.1 MB/s 2024-04-26T23:49:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294969 s, 35.5 MB/s 2024-04-26T23:49:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225117 s, 46.6 MB/s 2024-04-26T23:49:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301997 s, 34.7 MB/s 2024-04-26T23:49:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308613 s, 34.0 MB/s 2024-04-26T23:49:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254956 s, 41.1 MB/s 2024-04-26T23:49:03.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.860 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.860 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289689 s, 36.2 MB/s 2024-04-26T23:49:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031432 s, 33.4 MB/s 2024-04-26T23:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218722 s, 47.9 MB/s 2024-04-26T23:49:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228953 s, 45.8 MB/s 2024-04-26T23:49:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221851 s, 47.3 MB/s 2024-04-26T23:49:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321477 s, 32.6 MB/s 2024-04-26T23:49:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273305 s, 38.4 MB/s 2024-04-26T23:49:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231407 s, 45.3 MB/s 2024-04-26T23:49:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291494 s, 36.0 MB/s 2024-04-26T23:49:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281102 s, 37.3 MB/s 2024-04-26T23:49:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299908 s, 35.0 MB/s 2024-04-26T23:49:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213863 s, 49.0 MB/s 2024-04-26T23:49:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248182 s, 42.3 MB/s 2024-04-26T23:49:04.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.295 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280249 s, 37.4 MB/s 2024-04-26T23:49:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025166 s, 41.7 MB/s 2024-04-26T23:49:04.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.363 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260095 s, 40.3 MB/s 2024-04-26T23:49:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.9 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:04.376 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253525 s, 41.4 MB/s 2024-04-26T23:49:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243277 s, 43.1 MB/s 2024-04-26T23:49:04.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263589 s, 39.8 MB/s 2024-04-26T23:49:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225395 s, 46.5 MB/s 2024-04-26T23:49:04.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.654 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023706 s, 44.2 MB/s 2024-04-26T23:49:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300446 s, 34.9 MB/s 2024-04-26T23:49:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028083 s, 37.3 MB/s 2024-04-26T23:49:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240482 s, 43.6 MB/s 2024-04-26T23:49:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235504 s, 44.5 MB/s 2024-04-26T23:49:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244471 s, 42.9 MB/s 2024-04-26T23:49:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230936 s, 45.4 MB/s 2024-04-26T23:49:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257034 s, 40.8 MB/s 2024-04-26T23:49:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294525 s, 35.6 MB/s 2024-04-26T23:49:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247248 s, 42.4 MB/s 2024-04-26T23:49:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224046 s, 46.8 MB/s 2024-04-26T23:49:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264238 s, 39.7 MB/s 2024-04-26T23:49:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416935 s, 25.1 MB/s 2024-04-26T23:49:05.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.097 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257756 s, 40.7 MB/s 2024-04-26T23:49:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257163 s, 40.8 MB/s 2024-04-26T23:49:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026115 s, 40.2 MB/s 2024-04-26T23:49:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309669 s, 33.9 MB/s 2024-04-26T23:49:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022976 s, 45.6 MB/s 2024-04-26T23:49:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216323 s, 48.5 MB/s 2024-04-26T23:49:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258198 s, 40.6 MB/s 2024-04-26T23:49:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247922 s, 42.3 MB/s 2024-04-26T23:49:05.368 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.368 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.368 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329184 s, 31.9 MB/s 2024-04-26T23:49:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236688 s, 44.3 MB/s 2024-04-26T23:49:05.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.439 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331556 s, 31.6 MB/s 2024-04-26T23:49:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235333 s, 44.6 MB/s 2024-04-26T23:49:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241076 s, 43.5 MB/s 2024-04-26T23:49:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249761 s, 42.0 MB/s 2024-04-26T23:49:05.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.566 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261109 s, 40.2 MB/s 2024-04-26T23:49:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:05.579 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:05.579 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:05.579 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.10 2024-04-26T23:49:05.580 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:05.580 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:05.580 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:05.580 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294609 s, 35.6 MB/s 2024-04-26T23:49:05.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.755 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141231 s, 74.2 MB/s 2024-04-26T23:49:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296152 s, 35.4 MB/s 2024-04-26T23:49:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243123 s, 43.1 MB/s 2024-04-26T23:49:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232656 s, 45.1 MB/s 2024-04-26T23:49:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021737 s, 48.2 MB/s 2024-04-26T23:49:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229005 s, 45.8 MB/s 2024-04-26T23:49:05.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.946 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318559 s, 32.9 MB/s 2024-04-26T23:49:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242748 s, 43.2 MB/s 2024-04-26T23:49:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221922 s, 47.2 MB/s 2024-04-26T23:49:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200982 s, 52.2 MB/s 2024-04-26T23:49:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222937 s, 47.0 MB/s 2024-04-26T23:49:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224123 s, 46.8 MB/s 2024-04-26T23:49:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306599 s, 34.2 MB/s 2024-04-26T23:49:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224772 s, 46.7 MB/s 2024-04-26T23:49:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238923 s, 43.9 MB/s 2024-04-26T23:49:06.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274051 s, 38.3 MB/s 2024-04-26T23:49:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326141 s, 32.2 MB/s 2024-04-26T23:49:06.292 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.292 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.292 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260884 s, 40.2 MB/s 2024-04-26T23:49:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02619 s, 40.0 MB/s 2024-04-26T23:49:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244039 s, 43.0 MB/s 2024-04-26T23:49:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267544 s, 39.2 MB/s 2024-04-26T23:49:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282607 s, 37.1 MB/s 2024-04-26T23:49:06.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.453 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023305 s, 45.0 MB/s 2024-04-26T23:49:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242101 s, 43.3 MB/s 2024-04-26T23:49:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229472 s, 45.7 MB/s 2024-04-26T23:49:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233104 s, 45.0 MB/s 2024-04-26T23:49:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231012 s, 45.4 MB/s 2024-04-26T23:49:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223728 s, 46.9 MB/s 2024-04-26T23:49:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217077 s, 48.3 MB/s 2024-04-26T23:49:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238628 s, 43.9 MB/s 2024-04-26T23:49:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233687 s, 44.9 MB/s 2024-04-26T23:49:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:06.703 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.11 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:06.704 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265145 s, 39.5 MB/s 2024-04-26T23:49:06.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286315 s, 36.6 MB/s 2024-04-26T23:49:06.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228778 s, 45.8 MB/s 2024-04-26T23:49:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255312 s, 41.1 MB/s 2024-04-26T23:49:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300411 s, 34.9 MB/s 2024-04-26T23:49:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217521 s, 48.2 MB/s 2024-04-26T23:49:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291906 s, 35.9 MB/s 2024-04-26T23:49:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235972 s, 44.4 MB/s 2024-04-26T23:49:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234763 s, 44.7 MB/s 2024-04-26T23:49:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291528 s, 36.0 MB/s 2024-04-26T23:49:07.177 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202611 s, 51.8 MB/s 2024-04-26T23:49:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031597 s, 33.2 MB/s 2024-04-26T23:49:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208644 s, 50.3 MB/s 2024-04-26T23:49:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264291 s, 39.7 MB/s 2024-04-26T23:49:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029849 s, 35.1 MB/s 2024-04-26T23:49:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232819 s, 45.0 MB/s 2024-04-26T23:49:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250113 s, 41.9 MB/s 2024-04-26T23:49:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220944 s, 47.5 MB/s 2024-04-26T23:49:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216347 s, 48.5 MB/s 2024-04-26T23:49:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292023 s, 35.9 MB/s 2024-04-26T23:49:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225935 s, 46.4 MB/s 2024-04-26T23:49:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274245 s, 38.2 MB/s 2024-04-26T23:49:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314063 s, 33.4 MB/s 2024-04-26T23:49:07.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235525 s, 44.5 MB/s 2024-04-26T23:49:07.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220233 s, 47.6 MB/s 2024-04-26T23:49:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238389 s, 44.0 MB/s 2024-04-26T23:49:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231064 s, 45.4 MB/s 2024-04-26T23:49:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03049 s, 34.4 MB/s 2024-04-26T23:49:07.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.781 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289401 s, 36.2 MB/s 2024-04-26T23:49:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286946 s, 36.5 MB/s 2024-04-26T23:49:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216096 s, 48.5 MB/s 2024-04-26T23:49:07.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:07.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:07.875 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211946 s, 49.5 MB/s 2024-04-26T23:49:07.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:07.889 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:07.889 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:07.890 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.12 2024-04-26T23:49:07.890 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:07.890 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:07.890 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:07.890 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:08.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.039 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241142 s, 43.5 MB/s 2024-04-26T23:49:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244466 s, 42.9 MB/s 2024-04-26T23:49:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301752 s, 34.7 MB/s 2024-04-26T23:49:08.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.135 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019682 s, 53.3 MB/s 2024-04-26T23:49:08.168 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.168 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.169 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253105 s, 41.4 MB/s 2024-04-26T23:49:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223725 s, 46.9 MB/s 2024-04-26T23:49:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261269 s, 40.1 MB/s 2024-04-26T23:49:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266377 s, 39.4 MB/s 2024-04-26T23:49:08.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248258 s, 42.2 MB/s 2024-04-26T23:49:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252728 s, 41.5 MB/s 2024-04-26T23:49:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251248 s, 41.7 MB/s 2024-04-26T23:49:08.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288574 s, 36.3 MB/s 2024-04-26T23:49:08.429 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.429 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.429 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236791 s, 44.3 MB/s 2024-04-26T23:49:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257549 s, 40.7 MB/s 2024-04-26T23:49:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234077 s, 44.8 MB/s 2024-04-26T23:49:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219618 s, 47.7 MB/s 2024-04-26T23:49:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225875 s, 46.4 MB/s 2024-04-26T23:49:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218706 s, 47.9 MB/s 2024-04-26T23:49:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237172 s, 44.2 MB/s 2024-04-26T23:49:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201933 s, 51.9 MB/s 2024-04-26T23:49:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228368 s, 45.9 MB/s 2024-04-26T23:49:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310301 s, 33.8 MB/s 2024-04-26T23:49:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248913 s, 42.1 MB/s 2024-04-26T23:49:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281853 s, 37.2 MB/s 2024-04-26T23:49:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.801 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236284 s, 44.4 MB/s 2024-04-26T23:49:08.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.832 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.832 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237616 s, 44.1 MB/s 2024-04-26T23:49:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229593 s, 45.7 MB/s 2024-04-26T23:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262395 s, 40.0 MB/s 2024-04-26T23:49:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279813 s, 37.5 MB/s 2024-04-26T23:49:08.973 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305662 s, 34.3 MB/s 2024-04-26T23:49:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241012 s, 43.5 MB/s 2024-04-26T23:49:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250023 s, 41.9 MB/s 2024-04-26T23:49:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.13 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:09.052 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305064 s, 34.4 MB/s 2024-04-26T23:49:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224167 s, 46.8 MB/s 2024-04-26T23:49:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292819 s, 35.8 MB/s 2024-04-26T23:49:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254439 s, 41.2 MB/s 2024-04-26T23:49:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322451 s, 32.5 MB/s 2024-04-26T23:49:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229291 s, 45.7 MB/s 2024-04-26T23:49:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229149 s, 45.8 MB/s 2024-04-26T23:49:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258649 s, 40.5 MB/s 2024-04-26T23:49:09.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.472 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208376 s, 50.3 MB/s 2024-04-26T23:49:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249179 s, 42.1 MB/s 2024-04-26T23:49:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207882 s, 50.4 MB/s 2024-04-26T23:49:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283232 s, 37.0 MB/s 2024-04-26T23:49:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255883 s, 41.0 MB/s 2024-04-26T23:49:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237069 s, 44.2 MB/s 2024-04-26T23:49:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024165 s, 43.4 MB/s 2024-04-26T23:49:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029454 s, 35.6 MB/s 2024-04-26T23:49:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221377 s, 47.4 MB/s 2024-04-26T23:49:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268989 s, 39.0 MB/s 2024-04-26T23:49:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242358 s, 43.3 MB/s 2024-04-26T23:49:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208802 s, 50.2 MB/s 2024-04-26T23:49:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217467 s, 48.2 MB/s 2024-04-26T23:49:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273119 s, 38.4 MB/s 2024-04-26T23:49:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.913 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207854 s, 50.4 MB/s 2024-04-26T23:49:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.942 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219005 s, 47.9 MB/s 2024-04-26T23:49:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221694 s, 47.3 MB/s 2024-04-26T23:49:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196421 s, 53.4 MB/s 2024-04-26T23:49:10.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.027 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244673 s, 42.9 MB/s 2024-04-26T23:49:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287555 s, 36.5 MB/s 2024-04-26T23:49:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269752 s, 38.9 MB/s 2024-04-26T23:49:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254632 s, 41.2 MB/s 2024-04-26T23:49:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232304 s, 45.1 MB/s 2024-04-26T23:49:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258863 s, 40.5 MB/s 2024-04-26T23:49:10.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.14 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:10.203 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:10.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024139 s, 43.4 MB/s 2024-04-26T23:49:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251102 s, 41.8 MB/s 2024-04-26T23:49:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259329 s, 40.4 MB/s 2024-04-26T23:49:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230782 s, 45.4 MB/s 2024-04-26T23:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239906 s, 43.7 MB/s 2024-04-26T23:49:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247011 s, 42.5 MB/s 2024-04-26T23:49:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240152 s, 43.7 MB/s 2024-04-26T23:49:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228058 s, 46.0 MB/s 2024-04-26T23:49:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232506 s, 45.1 MB/s 2024-04-26T23:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207858 s, 50.4 MB/s 2024-04-26T23:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276196 s, 38.0 MB/s 2024-04-26T23:49:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256793 s, 40.8 MB/s 2024-04-26T23:49:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284147 s, 36.9 MB/s 2024-04-26T23:49:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025359 s, 41.3 MB/s 2024-04-26T23:49:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292302 s, 35.9 MB/s 2024-04-26T23:49:10.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.825 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242009 s, 43.3 MB/s 2024-04-26T23:49:10.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.855 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228663 s, 45.9 MB/s 2024-04-26T23:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232302 s, 45.1 MB/s 2024-04-26T23:49:10.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.914 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226551 s, 46.3 MB/s 2024-04-26T23:49:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.948 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275742 s, 38.0 MB/s 2024-04-26T23:49:10.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:10.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:10.978 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242529 s, 43.2 MB/s 2024-04-26T23:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233484 s, 44.9 MB/s 2024-04-26T23:49:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305303 s, 34.3 MB/s 2024-04-26T23:49:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304898 s, 34.4 MB/s 2024-04-26T23:49:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212477 s, 49.4 MB/s 2024-04-26T23:49:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208036 s, 50.4 MB/s 2024-04-26T23:49:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235794 s, 44.5 MB/s 2024-04-26T23:49:11.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.201 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02193 s, 47.8 MB/s 2024-04-26T23:49:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243921 s, 43.0 MB/s 2024-04-26T23:49:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181788 s, 57.7 MB/s 2024-04-26T23:49:11.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.287 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241742 s, 43.4 MB/s 2024-04-26T23:49:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214045 s, 49.0 MB/s 2024-04-26T23:49:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:11.328 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:11.328 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:11.328 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.15 2024-04-26T23:49:11.328 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:11.328 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:11.329 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:11.329 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:11.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.475 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023568 s, 44.5 MB/s 2024-04-26T23:49:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320866 s, 32.7 MB/s 2024-04-26T23:49:11.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02287 s, 45.8 MB/s 2024-04-26T23:49:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307616 s, 34.1 MB/s 2024-04-26T23:49:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250761 s, 41.8 MB/s 2024-04-26T23:49:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255607 s, 41.0 MB/s 2024-04-26T23:49:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296239 s, 35.4 MB/s 2024-04-26T23:49:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254529 s, 41.2 MB/s 2024-04-26T23:49:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263616 s, 39.8 MB/s 2024-04-26T23:49:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01854 s, 56.6 MB/s 2024-04-26T23:49:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254298 s, 41.2 MB/s 2024-04-26T23:49:11.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:11.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246523 s, 42.5 MB/s 2024-04-26T23:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208974 s, 50.2 MB/s 2024-04-26T23:49:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171558 s, 61.1 MB/s 2024-04-26T23:49:11.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183242 s, 57.2 MB/s 2024-04-26T23:49:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196519 s, 53.4 MB/s 2024-04-26T23:49:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205071 s, 51.1 MB/s 2024-04-26T23:49:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215001 s, 48.8 MB/s 2024-04-26T23:49:12.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022088 s, 47.5 MB/s 2024-04-26T23:49:12.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.047 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028851 s, 36.3 MB/s 2024-04-26T23:49:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237649 s, 44.1 MB/s 2024-04-26T23:49:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218368 s, 48.0 MB/s 2024-04-26T23:49:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223092 s, 47.0 MB/s 2024-04-26T23:49:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021687 s, 48.4 MB/s 2024-04-26T23:49:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02289 s, 45.8 MB/s 2024-04-26T23:49:12.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.224 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260317 s, 40.3 MB/s 2024-04-26T23:49:12.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269643 s, 38.9 MB/s 2024-04-26T23:49:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232805 s, 45.0 MB/s 2024-04-26T23:49:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259907 s, 40.3 MB/s 2024-04-26T23:49:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268948 s, 39.0 MB/s 2024-04-26T23:49:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242201 s, 43.3 MB/s 2024-04-26T23:49:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211885 s, 49.5 MB/s 2024-04-26T23:49:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.16 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:12.425 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:12.426 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:12.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023328 s, 44.9 MB/s 2024-04-26T23:49:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216237 s, 48.5 MB/s 2024-04-26T23:49:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316831 s, 33.1 MB/s 2024-04-26T23:49:12.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224018 s, 46.8 MB/s 2024-04-26T23:49:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262486 s, 39.9 MB/s 2024-04-26T23:49:12.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231307 s, 45.3 MB/s 2024-04-26T23:49:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304804 s, 34.4 MB/s 2024-04-26T23:49:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219606 s, 47.7 MB/s 2024-04-26T23:49:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242485 s, 43.2 MB/s 2024-04-26T23:49:12.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208407 s, 50.3 MB/s 2024-04-26T23:49:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228095 s, 46.0 MB/s 2024-04-26T23:49:12.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224066 s, 46.8 MB/s 2024-04-26T23:49:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232098 s, 45.2 MB/s 2024-04-26T23:49:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.024 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260757 s, 40.2 MB/s 2024-04-26T23:49:13.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.060 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029893 s, 35.1 MB/s 2024-04-26T23:49:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316643 s, 33.1 MB/s 2024-04-26T23:49:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258628 s, 40.5 MB/s 2024-04-26T23:49:13.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024146 s, 43.4 MB/s 2024-04-26T23:49:13.196 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.196 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.197 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283693 s, 37.0 MB/s 2024-04-26T23:49:13.226 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.226 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.226 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224152 s, 46.8 MB/s 2024-04-26T23:49:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227498 s, 46.1 MB/s 2024-04-26T23:49:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278875 s, 37.6 MB/s 2024-04-26T23:49:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222303 s, 47.2 MB/s 2024-04-26T23:49:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272745 s, 38.4 MB/s 2024-04-26T23:49:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255828 s, 41.0 MB/s 2024-04-26T23:49:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252331 s, 41.6 MB/s 2024-04-26T23:49:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198589 s, 52.8 MB/s 2024-04-26T23:49:13.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.472 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207274 s, 50.6 MB/s 2024-04-26T23:49:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257538 s, 40.7 MB/s 2024-04-26T23:49:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222664 s, 47.1 MB/s 2024-04-26T23:49:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282187 s, 37.2 MB/s 2024-04-26T23:49:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190931 s, 54.9 MB/s 2024-04-26T23:49:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.17 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:13.607 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232937 s, 45.0 MB/s 2024-04-26T23:49:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207459 s, 50.5 MB/s 2024-04-26T23:49:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180662 s, 58.0 MB/s 2024-04-26T23:49:13.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.844 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299373 s, 35.0 MB/s 2024-04-26T23:49:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240836 s, 43.5 MB/s 2024-04-26T23:49:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183107 s, 57.3 MB/s 2024-04-26T23:49:13.926 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.926 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.926 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195885 s, 53.5 MB/s 2024-04-26T23:49:13.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.959 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262487 s, 39.9 MB/s 2024-04-26T23:49:13.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:13.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:13.986 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206167 s, 50.9 MB/s 2024-04-26T23:49:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220128 s, 47.6 MB/s 2024-04-26T23:49:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212565 s, 49.3 MB/s 2024-04-26T23:49:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249022 s, 42.1 MB/s 2024-04-26T23:49:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248261 s, 42.2 MB/s 2024-04-26T23:49:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241102 s, 43.5 MB/s 2024-04-26T23:49:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300488 s, 34.9 MB/s 2024-04-26T23:49:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256751 s, 40.8 MB/s 2024-04-26T23:49:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287249 s, 36.5 MB/s 2024-04-26T23:49:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230734 s, 45.4 MB/s 2024-04-26T23:49:14.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221501 s, 47.3 MB/s 2024-04-26T23:49:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237044 s, 44.2 MB/s 2024-04-26T23:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297796 s, 35.2 MB/s 2024-04-26T23:49:14.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233505 s, 44.9 MB/s 2024-04-26T23:49:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241006 s, 43.5 MB/s 2024-04-26T23:49:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257978 s, 40.6 MB/s 2024-04-26T23:49:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233599 s, 44.9 MB/s 2024-04-26T23:49:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253901 s, 41.3 MB/s 2024-04-26T23:49:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231189 s, 45.4 MB/s 2024-04-26T23:49:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209624 s, 50.0 MB/s 2024-04-26T23:49:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259055 s, 40.5 MB/s 2024-04-26T23:49:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251936 s, 41.6 MB/s 2024-04-26T23:49:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286552 s, 36.6 MB/s 2024-04-26T23:49:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253844 s, 41.3 MB/s 2024-04-26T23:49:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.18 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:14.723 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286094 s, 36.7 MB/s 2024-04-26T23:49:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237267 s, 44.2 MB/s 2024-04-26T23:49:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274294 s, 38.2 MB/s 2024-04-26T23:49:14.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187987 s, 55.8 MB/s 2024-04-26T23:49:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209477 s, 50.1 MB/s 2024-04-26T23:49:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215087 s, 48.8 MB/s 2024-04-26T23:49:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311719 s, 33.6 MB/s 2024-04-26T23:49:15.089 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.089 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.089 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225545 s, 46.5 MB/s 2024-04-26T23:49:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206318 s, 50.8 MB/s 2024-04-26T23:49:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230595 s, 45.5 MB/s 2024-04-26T23:49:15.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282177 s, 37.2 MB/s 2024-04-26T23:49:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202744 s, 51.7 MB/s 2024-04-26T23:49:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244006 s, 43.0 MB/s 2024-04-26T23:49:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225685 s, 46.5 MB/s 2024-04-26T23:49:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200725 s, 52.2 MB/s 2024-04-26T23:49:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242388 s, 43.3 MB/s 2024-04-26T23:49:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299731 s, 35.0 MB/s 2024-04-26T23:49:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238566 s, 44.0 MB/s 2024-04-26T23:49:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231296 s, 45.3 MB/s 2024-04-26T23:49:15.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338092 s, 31.0 MB/s 2024-04-26T23:49:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261253 s, 40.1 MB/s 2024-04-26T23:49:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023999 s, 43.7 MB/s 2024-04-26T23:49:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239266 s, 43.8 MB/s 2024-04-26T23:49:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224747 s, 46.7 MB/s 2024-04-26T23:49:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238824 s, 43.9 MB/s 2024-04-26T23:49:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246886 s, 42.5 MB/s 2024-04-26T23:49:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023534 s, 44.6 MB/s 2024-04-26T23:49:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237849 s, 44.1 MB/s 2024-04-26T23:49:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232679 s, 45.1 MB/s 2024-04-26T23:49:15.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.774 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251911 s, 41.6 MB/s 2024-04-26T23:49:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225751 s, 46.4 MB/s 2024-04-26T23:49:15.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208342 s, 50.3 MB/s 2024-04-26T23:49:15.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:15.844 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:15.844 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:15.844 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.19 2024-04-26T23:49:15.844 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:15.845 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:15.845 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:15.845 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:15.988 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:15.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209349 s, 50.1 MB/s 2024-04-26T23:49:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312496 s, 33.6 MB/s 2024-04-26T23:49:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202541 s, 51.8 MB/s 2024-04-26T23:49:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252389 s, 41.5 MB/s 2024-04-26T23:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262997 s, 39.9 MB/s 2024-04-26T23:49:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229161 s, 45.8 MB/s 2024-04-26T23:49:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030003 s, 34.9 MB/s 2024-04-26T23:49:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247709 s, 42.3 MB/s 2024-04-26T23:49:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214098 s, 49.0 MB/s 2024-04-26T23:49:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.276 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256345 s, 40.9 MB/s 2024-04-26T23:49:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200729 s, 52.2 MB/s 2024-04-26T23:49:16.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.333 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023973 s, 43.7 MB/s 2024-04-26T23:49:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231168 s, 45.4 MB/s 2024-04-26T23:49:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233237 s, 45.0 MB/s 2024-04-26T23:49:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027473 s, 38.2 MB/s 2024-04-26T23:49:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322717 s, 32.5 MB/s 2024-04-26T23:49:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249216 s, 42.1 MB/s 2024-04-26T23:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231564 s, 45.3 MB/s 2024-04-26T23:49:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224301 s, 46.7 MB/s 2024-04-26T23:49:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320434 s, 32.7 MB/s 2024-04-26T23:49:16.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020779 s, 50.5 MB/s 2024-04-26T23:49:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228875 s, 45.8 MB/s 2024-04-26T23:49:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237245 s, 44.2 MB/s 2024-04-26T23:49:16.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.720 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314897 s, 33.3 MB/s 2024-04-26T23:49:16.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.757 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315196 s, 33.3 MB/s 2024-04-26T23:49:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252978 s, 41.4 MB/s 2024-04-26T23:49:16.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234213 s, 44.8 MB/s 2024-04-26T23:49:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255333 s, 41.1 MB/s 2024-04-26T23:49:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223261 s, 47.0 MB/s 2024-04-26T23:49:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317143 s, 33.1 MB/s 2024-04-26T23:49:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237296 s, 44.2 MB/s 2024-04-26T23:49:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261962 s, 40.0 MB/s 2024-04-26T23:49:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:16.995 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:16.995 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:16.996 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.20 2024-04-26T23:49:16.996 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:16.996 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:16.996 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:16.996 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:17.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288765 s, 36.3 MB/s 2024-04-26T23:49:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196448 s, 53.4 MB/s 2024-04-26T23:49:17.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252666 s, 41.5 MB/s 2024-04-26T23:49:17.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202452 s, 51.8 MB/s 2024-04-26T23:49:17.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248625 s, 42.2 MB/s 2024-04-26T23:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273032 s, 38.4 MB/s 2024-04-26T23:49:17.328 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.328 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.328 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244228 s, 42.9 MB/s 2024-04-26T23:49:17.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245136 s, 42.8 MB/s 2024-04-26T23:49:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268931 s, 39.0 MB/s 2024-04-26T23:49:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256814 s, 40.8 MB/s 2024-04-26T23:49:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238639 s, 43.9 MB/s 2024-04-26T23:49:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196526 s, 53.4 MB/s 2024-04-26T23:49:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263176 s, 39.8 MB/s 2024-04-26T23:49:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229775 s, 45.6 MB/s 2024-04-26T23:49:17.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306057 s, 34.3 MB/s 2024-04-26T23:49:17.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.615 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239563 s, 43.8 MB/s 2024-04-26T23:49:17.650 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274392 s, 38.2 MB/s 2024-04-26T23:49:17.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.680 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230768 s, 45.4 MB/s 2024-04-26T23:49:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210519 s, 49.8 MB/s 2024-04-26T23:49:17.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.740 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241251 s, 43.5 MB/s 2024-04-26T23:49:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209266 s, 50.1 MB/s 2024-04-26T23:49:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224567 s, 46.7 MB/s 2024-04-26T23:49:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185524 s, 56.5 MB/s 2024-04-26T23:49:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212836 s, 49.3 MB/s 2024-04-26T23:49:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294505 s, 35.6 MB/s 2024-04-26T23:49:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248412 s, 42.2 MB/s 2024-04-26T23:49:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228588 s, 45.9 MB/s 2024-04-26T23:49:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:17.982 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259114 s, 40.5 MB/s 2024-04-26T23:49:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263921 s, 39.7 MB/s 2024-04-26T23:49:18.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.055 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308629 s, 34.0 MB/s 2024-04-26T23:49:18.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232997 s, 45.0 MB/s 2024-04-26T23:49:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260869 s, 40.2 MB/s 2024-04-26T23:49:18.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:18.133 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.21 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:18.134 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:18.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334539 s, 31.3 MB/s 2024-04-26T23:49:18.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.332 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02614 s, 40.1 MB/s 2024-04-26T23:49:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247276 s, 42.4 MB/s 2024-04-26T23:49:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309407 s, 33.9 MB/s 2024-04-26T23:49:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230523 s, 45.5 MB/s 2024-04-26T23:49:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.473 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304951 s, 34.4 MB/s 2024-04-26T23:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024645 s, 42.5 MB/s 2024-04-26T23:49:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250878 s, 41.8 MB/s 2024-04-26T23:49:18.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230714 s, 45.4 MB/s 2024-04-26T23:49:18.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.600 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233961 s, 44.8 MB/s 2024-04-26T23:49:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246381 s, 42.6 MB/s 2024-04-26T23:49:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236729 s, 44.3 MB/s 2024-04-26T23:49:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221939 s, 47.2 MB/s 2024-04-26T23:49:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285612 s, 36.7 MB/s 2024-04-26T23:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231567 s, 45.3 MB/s 2024-04-26T23:49:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022884 s, 45.8 MB/s 2024-04-26T23:49:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273118 s, 38.4 MB/s 2024-04-26T23:49:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233371 s, 44.9 MB/s 2024-04-26T23:49:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223332 s, 47.0 MB/s 2024-04-26T23:49:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022478 s, 46.6 MB/s 2024-04-26T23:49:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293536 s, 35.7 MB/s 2024-04-26T23:49:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310333 s, 33.8 MB/s 2024-04-26T23:49:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303751 s, 34.5 MB/s 2024-04-26T23:49:19.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221169 s, 47.4 MB/s 2024-04-26T23:49:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298647 s, 35.1 MB/s 2024-04-26T23:49:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233124 s, 45.0 MB/s 2024-04-26T23:49:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223879 s, 46.8 MB/s 2024-04-26T23:49:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292724 s, 35.8 MB/s 2024-04-26T23:49:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238907 s, 43.9 MB/s 2024-04-26T23:49:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212864 s, 49.3 MB/s 2024-04-26T23:49:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223597 s, 46.9 MB/s 2024-04-26T23:49:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023432 s, 44.7 MB/s 2024-04-26T23:49:19.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.22 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:19.330 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:19.331 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320458 s, 32.7 MB/s 2024-04-26T23:49:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206922 s, 50.7 MB/s 2024-04-26T23:49:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01933 s, 54.2 MB/s 2024-04-26T23:49:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252223 s, 41.6 MB/s 2024-04-26T23:49:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233941 s, 44.8 MB/s 2024-04-26T23:49:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229957 s, 45.6 MB/s 2024-04-26T23:49:19.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021716 s, 48.3 MB/s 2024-04-26T23:49:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227985 s, 46.0 MB/s 2024-04-26T23:49:19.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216732 s, 48.4 MB/s 2024-04-26T23:49:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275511 s, 38.1 MB/s 2024-04-26T23:49:19.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260043 s, 40.3 MB/s 2024-04-26T23:49:19.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262743 s, 39.9 MB/s 2024-04-26T23:49:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199966 s, 52.4 MB/s 2024-04-26T23:49:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027697 s, 37.9 MB/s 2024-04-26T23:49:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213988 s, 49.0 MB/s 2024-04-26T23:49:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253935 s, 41.3 MB/s 2024-04-26T23:49:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234161 s, 44.8 MB/s 2024-04-26T23:49:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215284 s, 48.7 MB/s 2024-04-26T23:49:20.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.026 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237781 s, 44.1 MB/s 2024-04-26T23:49:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237224 s, 44.2 MB/s 2024-04-26T23:49:20.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.085 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216271 s, 48.5 MB/s 2024-04-26T23:49:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244391 s, 42.9 MB/s 2024-04-26T23:49:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237456 s, 44.2 MB/s 2024-04-26T23:49:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272142 s, 38.5 MB/s 2024-04-26T23:49:20.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.211 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225835 s, 46.4 MB/s 2024-04-26T23:49:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225343 s, 46.5 MB/s 2024-04-26T23:49:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222962 s, 47.0 MB/s 2024-04-26T23:49:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319493 s, 32.8 MB/s 2024-04-26T23:49:20.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319181 s, 32.9 MB/s 2024-04-26T23:49:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304117 s, 34.5 MB/s 2024-04-26T23:49:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232251 s, 45.1 MB/s 2024-04-26T23:49:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021121 s, 49.6 MB/s 2024-04-26T23:49:20.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:20.454 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:20.454 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:20.455 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.23 2024-04-26T23:49:20.455 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:20.455 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:20.455 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:20.455 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215117 s, 48.7 MB/s 2024-04-26T23:49:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235174 s, 44.6 MB/s 2024-04-26T23:49:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298366 s, 35.1 MB/s 2024-04-26T23:49:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232465 s, 45.1 MB/s 2024-04-26T23:49:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208194 s, 50.4 MB/s 2024-04-26T23:49:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243142 s, 43.1 MB/s 2024-04-26T23:49:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264025 s, 39.7 MB/s 2024-04-26T23:49:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245227 s, 42.8 MB/s 2024-04-26T23:49:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259409 s, 40.4 MB/s 2024-04-26T23:49:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024558 s, 42.7 MB/s 2024-04-26T23:49:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215686 s, 48.6 MB/s 2024-04-26T23:49:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283842 s, 36.9 MB/s 2024-04-26T23:49:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306131 s, 34.3 MB/s 2024-04-26T23:49:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206659 s, 50.7 MB/s 2024-04-26T23:49:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210483 s, 49.8 MB/s 2024-04-26T23:49:21.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.065 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223285 s, 47.0 MB/s 2024-04-26T23:49:21.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.103 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314172 s, 33.4 MB/s 2024-04-26T23:49:21.133 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237754 s, 44.1 MB/s 2024-04-26T23:49:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227114 s, 46.2 MB/s 2024-04-26T23:49:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233358 s, 44.9 MB/s 2024-04-26T23:49:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247973 s, 42.3 MB/s 2024-04-26T23:49:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209759 s, 50.0 MB/s 2024-04-26T23:49:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227474 s, 46.1 MB/s 2024-04-26T23:49:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026086 s, 40.2 MB/s 2024-04-26T23:49:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216648 s, 48.4 MB/s 2024-04-26T23:49:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249514 s, 42.0 MB/s 2024-04-26T23:49:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243877 s, 43.0 MB/s 2024-04-26T23:49:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.436 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249053 s, 42.1 MB/s 2024-04-26T23:49:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233247 s, 45.0 MB/s 2024-04-26T23:49:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238094 s, 44.0 MB/s 2024-04-26T23:49:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270336 s, 38.8 MB/s 2024-04-26T23:49:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249037 s, 42.1 MB/s 2024-04-26T23:49:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.24 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:21.574 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:21.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223829 s, 46.8 MB/s 2024-04-26T23:49:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257192 s, 40.8 MB/s 2024-04-26T23:49:21.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.785 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339777 s, 30.9 MB/s 2024-04-26T23:49:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021653 s, 48.4 MB/s 2024-04-26T23:49:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233873 s, 44.8 MB/s 2024-04-26T23:49:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228429 s, 45.9 MB/s 2024-04-26T23:49:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216572 s, 48.4 MB/s 2024-04-26T23:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024258 s, 43.2 MB/s 2024-04-26T23:49:21.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.963 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213916 s, 49.0 MB/s 2024-04-26T23:49:21.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:21.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:21.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269979 s, 38.8 MB/s 2024-04-26T23:49:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238197 s, 44.0 MB/s 2024-04-26T23:49:22.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.062 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269787 s, 38.9 MB/s 2024-04-26T23:49:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257546 s, 40.7 MB/s 2024-04-26T23:49:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247179 s, 42.4 MB/s 2024-04-26T23:49:22.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.160 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279701 s, 37.5 MB/s 2024-04-26T23:49:22.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253675 s, 41.3 MB/s 2024-04-26T23:49:22.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.222 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245006 s, 42.8 MB/s 2024-04-26T23:49:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226654 s, 46.3 MB/s 2024-04-26T23:49:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252035 s, 41.6 MB/s 2024-04-26T23:49:22.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234082 s, 44.8 MB/s 2024-04-26T23:49:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257533 s, 40.7 MB/s 2024-04-26T23:49:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283106 s, 37.0 MB/s 2024-04-26T23:49:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314659 s, 33.3 MB/s 2024-04-26T23:49:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244394 s, 42.9 MB/s 2024-04-26T23:49:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284041 s, 36.9 MB/s 2024-04-26T23:49:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233134 s, 45.0 MB/s 2024-04-26T23:49:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018127 s, 57.8 MB/s 2024-04-26T23:49:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286529 s, 36.6 MB/s 2024-04-26T23:49:22.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.606 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245274 s, 42.8 MB/s 2024-04-26T23:49:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198081 s, 52.9 MB/s 2024-04-26T23:49:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254252 s, 41.2 MB/s 2024-04-26T23:49:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249061 s, 42.1 MB/s 2024-04-26T23:49:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:22.708 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:22.708 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:22.708 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.25 2024-04-26T23:49:22.708 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:22.708 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:22.709 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:22.709 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231333 s, 45.3 MB/s 2024-04-26T23:49:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320972 s, 32.7 MB/s 2024-04-26T23:49:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.919 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184047 s, 57.0 MB/s 2024-04-26T23:49:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210256 s, 49.9 MB/s 2024-04-26T23:49:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240825 s, 43.5 MB/s 2024-04-26T23:49:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251548 s, 41.7 MB/s 2024-04-26T23:49:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314561 s, 33.3 MB/s 2024-04-26T23:49:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221857 s, 47.3 MB/s 2024-04-26T23:49:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307466 s, 34.1 MB/s 2024-04-26T23:49:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238245 s, 44.0 MB/s 2024-04-26T23:49:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257749 s, 40.7 MB/s 2024-04-26T23:49:23.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.209 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239154 s, 43.8 MB/s 2024-04-26T23:49:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236711 s, 44.3 MB/s 2024-04-26T23:49:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219325 s, 47.8 MB/s 2024-04-26T23:49:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293179 s, 35.8 MB/s 2024-04-26T23:49:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232421 s, 45.1 MB/s 2024-04-26T23:49:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296274 s, 35.4 MB/s 2024-04-26T23:49:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020888 s, 50.2 MB/s 2024-04-26T23:49:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243367 s, 43.1 MB/s 2024-04-26T23:49:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023546 s, 44.5 MB/s 2024-04-26T23:49:23.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231757 s, 45.2 MB/s 2024-04-26T23:49:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295786 s, 35.5 MB/s 2024-04-26T23:49:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219364 s, 47.8 MB/s 2024-04-26T23:49:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280555 s, 37.4 MB/s 2024-04-26T23:49:23.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.614 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202886 s, 51.7 MB/s 2024-04-26T23:49:23.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274379 s, 38.2 MB/s 2024-04-26T23:49:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223113 s, 47.0 MB/s 2024-04-26T23:49:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022756 s, 46.1 MB/s 2024-04-26T23:49:23.740 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.740 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.740 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276167 s, 38.0 MB/s 2024-04-26T23:49:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219492 s, 47.8 MB/s 2024-04-26T23:49:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217254 s, 48.3 MB/s 2024-04-26T23:49:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227189 s, 46.2 MB/s 2024-04-26T23:49:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:23.839 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:23.839 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:23.839 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.26 2024-04-26T23:49:23.839 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:23.839 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:23.840 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:23.840 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225327 s, 46.5 MB/s 2024-04-26T23:49:24.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225876 s, 46.4 MB/s 2024-04-26T23:49:24.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249413 s, 42.0 MB/s 2024-04-26T23:49:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211385 s, 49.6 MB/s 2024-04-26T23:49:24.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328372 s, 31.9 MB/s 2024-04-26T23:49:24.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260596 s, 40.2 MB/s 2024-04-26T23:49:24.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.182 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297133 s, 35.3 MB/s 2024-04-26T23:49:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229572 s, 45.7 MB/s 2024-04-26T23:49:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249995 s, 41.9 MB/s 2024-04-26T23:49:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286607 s, 36.6 MB/s 2024-04-26T23:49:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227939 s, 46.0 MB/s 2024-04-26T23:49:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241888 s, 43.3 MB/s 2024-04-26T23:49:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244968 s, 42.8 MB/s 2024-04-26T23:49:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232762 s, 45.0 MB/s 2024-04-26T23:49:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215514 s, 48.7 MB/s 2024-04-26T23:49:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315208 s, 33.3 MB/s 2024-04-26T23:49:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308279 s, 34.0 MB/s 2024-04-26T23:49:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02259 s, 46.4 MB/s 2024-04-26T23:49:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227989 s, 46.0 MB/s 2024-04-26T23:49:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226112 s, 46.4 MB/s 2024-04-26T23:49:24.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.629 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290222 s, 36.1 MB/s 2024-04-26T23:49:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027453 s, 38.2 MB/s 2024-04-26T23:49:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029374 s, 35.7 MB/s 2024-04-26T23:49:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312543 s, 33.5 MB/s 2024-04-26T23:49:24.772 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285002 s, 36.8 MB/s 2024-04-26T23:49:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283812 s, 36.9 MB/s 2024-04-26T23:49:24.834 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.834 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.834 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192493 s, 54.5 MB/s 2024-04-26T23:49:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224867 s, 46.6 MB/s 2024-04-26T23:49:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024765 s, 42.3 MB/s 2024-04-26T23:49:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294214 s, 35.6 MB/s 2024-04-26T23:49:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294448 s, 35.6 MB/s 2024-04-26T23:49:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256964 s, 40.8 MB/s 2024-04-26T23:49:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.27 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:25.012 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:25.165 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.165 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.165 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222878 s, 47.0 MB/s 2024-04-26T23:49:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021237 s, 49.4 MB/s 2024-04-26T23:49:25.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.222 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211003 s, 49.7 MB/s 2024-04-26T23:49:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238589 s, 43.9 MB/s 2024-04-26T23:49:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230028 s, 45.6 MB/s 2024-04-26T23:49:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269272 s, 38.9 MB/s 2024-04-26T23:49:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028378 s, 37.0 MB/s 2024-04-26T23:49:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264417 s, 39.7 MB/s 2024-04-26T23:49:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025086 s, 41.8 MB/s 2024-04-26T23:49:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234952 s, 44.6 MB/s 2024-04-26T23:49:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262988 s, 39.9 MB/s 2024-04-26T23:49:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258145 s, 40.6 MB/s 2024-04-26T23:49:25.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228333 s, 45.9 MB/s 2024-04-26T23:49:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229106 s, 45.8 MB/s 2024-04-26T23:49:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223587 s, 46.9 MB/s 2024-04-26T23:49:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326367 s, 32.1 MB/s 2024-04-26T23:49:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223426 s, 46.9 MB/s 2024-04-26T23:49:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249893 s, 42.0 MB/s 2024-04-26T23:49:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247005 s, 42.5 MB/s 2024-04-26T23:49:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236296 s, 44.4 MB/s 2024-04-26T23:49:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316937 s, 33.1 MB/s 2024-04-26T23:49:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242964 s, 43.2 MB/s 2024-04-26T23:49:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217294 s, 48.3 MB/s 2024-04-26T23:49:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023713 s, 44.2 MB/s 2024-04-26T23:49:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225941 s, 46.4 MB/s 2024-04-26T23:49:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213425 s, 49.1 MB/s 2024-04-26T23:49:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211694 s, 49.5 MB/s 2024-04-26T23:49:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219257 s, 47.8 MB/s 2024-04-26T23:49:26.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257306 s, 40.8 MB/s 2024-04-26T23:49:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243579 s, 43.0 MB/s 2024-04-26T23:49:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224694 s, 46.7 MB/s 2024-04-26T23:49:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260313 s, 40.3 MB/s 2024-04-26T23:49:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:26.164 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:26.164 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:26.164 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.28 2024-04-26T23:49:26.164 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:26.165 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:26.165 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:26.165 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:26.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217479 s, 48.2 MB/s 2024-04-26T23:49:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272234 s, 38.5 MB/s 2024-04-26T23:49:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239614 s, 43.8 MB/s 2024-04-26T23:49:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226297 s, 46.3 MB/s 2024-04-26T23:49:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022114 s, 47.4 MB/s 2024-04-26T23:49:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224784 s, 46.6 MB/s 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233527 s, 44.9 MB/s 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023704 s, 44.2 MB/s 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217809 s, 48.1 MB/s 2024-04-26T23:49:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217497 s, 48.2 MB/s 2024-04-26T23:49:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242113 s, 43.3 MB/s 2024-04-26T23:49:26.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021795 s, 48.1 MB/s 2024-04-26T23:49:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222693 s, 47.1 MB/s 2024-04-26T23:49:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238177 s, 44.0 MB/s 2024-04-26T23:49:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229989 s, 45.6 MB/s 2024-04-26T23:49:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216268 s, 48.5 MB/s 2024-04-26T23:49:26.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207095 s, 50.6 MB/s 2024-04-26T23:49:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.802 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169883 s, 61.7 MB/s 2024-04-26T23:49:26.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218434 s, 48.0 MB/s 2024-04-26T23:49:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281537 s, 37.2 MB/s 2024-04-26T23:49:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211037 s, 49.7 MB/s 2024-04-26T23:49:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294272 s, 35.6 MB/s 2024-04-26T23:49:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.956 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213549 s, 49.1 MB/s 2024-04-26T23:49:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241856 s, 43.4 MB/s 2024-04-26T23:49:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277738 s, 37.8 MB/s 2024-04-26T23:49:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271778 s, 38.6 MB/s 2024-04-26T23:49:27.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.087 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241575 s, 43.4 MB/s 2024-04-26T23:49:27.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240518 s, 43.6 MB/s 2024-04-26T23:49:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275519 s, 38.1 MB/s 2024-04-26T23:49:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239745 s, 43.7 MB/s 2024-04-26T23:49:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255226 s, 41.1 MB/s 2024-04-26T23:49:27.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227187 s, 46.2 MB/s 2024-04-26T23:49:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.29 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:27.257 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:27.258 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258957 s, 40.5 MB/s 2024-04-26T23:49:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246887 s, 42.5 MB/s 2024-04-26T23:49:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268098 s, 39.1 MB/s 2024-04-26T23:49:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224485 s, 46.7 MB/s 2024-04-26T23:49:27.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.538 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240061 s, 43.7 MB/s 2024-04-26T23:49:27.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02986 s, 35.1 MB/s 2024-04-26T23:49:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211461 s, 49.6 MB/s 2024-04-26T23:49:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248197 s, 42.2 MB/s 2024-04-26T23:49:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251881 s, 41.6 MB/s 2024-04-26T23:49:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221399 s, 47.4 MB/s 2024-04-26T23:49:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191026 s, 54.9 MB/s 2024-04-26T23:49:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215908 s, 48.6 MB/s 2024-04-26T23:49:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225597 s, 46.5 MB/s 2024-04-26T23:49:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209489 s, 50.1 MB/s 2024-04-26T23:49:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245836 s, 42.7 MB/s 2024-04-26T23:49:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240524 s, 43.6 MB/s 2024-04-26T23:49:27.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230256 s, 45.5 MB/s 2024-04-26T23:49:27.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208206 s, 50.4 MB/s 2024-04-26T23:49:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219942 s, 47.7 MB/s 2024-04-26T23:49:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210596 s, 49.8 MB/s 2024-04-26T23:49:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027432 s, 38.2 MB/s 2024-04-26T23:49:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310238 s, 33.8 MB/s 2024-04-26T23:49:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235365 s, 44.6 MB/s 2024-04-26T23:49:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218976 s, 47.9 MB/s 2024-04-26T23:49:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244247 s, 42.9 MB/s 2024-04-26T23:49:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239016 s, 43.9 MB/s 2024-04-26T23:49:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258219 s, 40.6 MB/s 2024-04-26T23:49:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242876 s, 43.2 MB/s 2024-04-26T23:49:28.266 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.266 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.266 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264133 s, 39.7 MB/s 2024-04-26T23:49:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233092 s, 45.0 MB/s 2024-04-26T23:49:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209823 s, 50.0 MB/s 2024-04-26T23:49:28.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229545 s, 45.7 MB/s 2024-04-26T23:49:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.30 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:28.364 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:28.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256212 s, 40.9 MB/s 2024-04-26T23:49:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239228 s, 43.8 MB/s 2024-04-26T23:49:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277925 s, 37.7 MB/s 2024-04-26T23:49:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228416 s, 45.9 MB/s 2024-04-26T23:49:28.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.637 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299268 s, 35.0 MB/s 2024-04-26T23:49:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208976 s, 50.2 MB/s 2024-04-26T23:49:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228624 s, 45.9 MB/s 2024-04-26T23:49:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264283 s, 39.7 MB/s 2024-04-26T23:49:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241789 s, 43.4 MB/s 2024-04-26T23:49:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234287 s, 44.8 MB/s 2024-04-26T23:49:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222741 s, 47.1 MB/s 2024-04-26T23:49:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253769 s, 41.3 MB/s 2024-04-26T23:49:28.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.880 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242182 s, 43.3 MB/s 2024-04-26T23:49:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212878 s, 49.3 MB/s 2024-04-26T23:49:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254957 s, 41.1 MB/s 2024-04-26T23:49:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277213 s, 37.8 MB/s 2024-04-26T23:49:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239126 s, 43.9 MB/s 2024-04-26T23:49:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251561 s, 41.7 MB/s 2024-04-26T23:49:29.067 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.067 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.067 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244437 s, 42.9 MB/s 2024-04-26T23:49:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231872 s, 45.2 MB/s 2024-04-26T23:49:29.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222227 s, 47.2 MB/s 2024-04-26T23:49:29.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217488 s, 48.2 MB/s 2024-04-26T23:49:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331874 s, 31.6 MB/s 2024-04-26T23:49:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327104 s, 32.1 MB/s 2024-04-26T23:49:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231063 s, 45.4 MB/s 2024-04-26T23:49:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210181 s, 49.9 MB/s 2024-04-26T23:49:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264577 s, 39.6 MB/s 2024-04-26T23:49:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227417 s, 46.1 MB/s 2024-04-26T23:49:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256059 s, 41.0 MB/s 2024-04-26T23:49:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265416 s, 39.5 MB/s 2024-04-26T23:49:29.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028553 s, 36.7 MB/s 2024-04-26T23:49:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249212 s, 42.1 MB/s 2024-04-26T23:49:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.31 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:49:29.498 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:49:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:49:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193208 s, 54.3 MB/s 2024-04-26T23:49:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229135 s, 45.8 MB/s 2024-04-26T23:49:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222777 s, 47.1 MB/s 2024-04-26T23:49:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224134 s, 46.8 MB/s 2024-04-26T23:49:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295002 s, 35.5 MB/s 2024-04-26T23:49:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232924 s, 45.0 MB/s 2024-04-26T23:49:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299577 s, 35.0 MB/s 2024-04-26T23:49:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310794 s, 33.7 MB/s 2024-04-26T23:49:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264384 s, 39.7 MB/s 2024-04-26T23:49:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213246 s, 49.2 MB/s 2024-04-26T23:49:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244768 s, 42.8 MB/s 2024-04-26T23:49:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239774 s, 43.7 MB/s 2024-04-26T23:49:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023332 s, 44.9 MB/s 2024-04-26T23:49:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228708 s, 45.8 MB/s 2024-04-26T23:49:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264486 s, 39.6 MB/s 2024-04-26T23:49:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296343 s, 35.4 MB/s 2024-04-26T23:49:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289998 s, 36.2 MB/s 2024-04-26T23:49:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231138 s, 45.4 MB/s 2024-04-26T23:49:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306657 s, 34.2 MB/s 2024-04-26T23:49:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255592 s, 41.0 MB/s 2024-04-26T23:49:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270961 s, 38.7 MB/s 2024-04-26T23:49:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023986 s, 43.7 MB/s 2024-04-26T23:49:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230638 s, 45.5 MB/s 2024-04-26T23:49:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031562 s, 33.2 MB/s 2024-04-26T23:49:30.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.428 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309155 s, 33.9 MB/s 2024-04-26T23:49:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322356 s, 32.5 MB/s 2024-04-26T23:49:30.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.498 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247443 s, 42.4 MB/s 2024-04-26T23:49:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228003 s, 46.0 MB/s 2024-04-26T23:49:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215637 s, 48.6 MB/s 2024-04-26T23:49:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215291 s, 48.7 MB/s 2024-04-26T23:49:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239832 s, 43.7 MB/s 2024-04-26T23:49:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:49:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:49:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024313 s, 43.1 MB/s 2024-04-26T23:49:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:49:30.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:49:30.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-26T23:49:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.878+0000 7f8fc8a00640 1 Processor -- start 2024-04-26T23:49:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.878+0000 7f8fc8a00640 1 -- start start 2024-04-26T23:49:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.880+0000 7f8fc8a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8fc40c4ea0 0x7f8fc40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.880+0000 7f8fc8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fc40cfc90 0x7f8fc406f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.880+0000 7f8fc8a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc4163a30 con 0x7f8fc4163650 2024-04-26T23:49:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.880+0000 7f8fc8a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc40d0d30 con 0x7f8fc40cfc90 2024-04-26T23:49:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.880+0000 7f8fc8a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc40d0120 con 0x7f8fc40c4ea0 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fc40cfc90 0x7f8fc406f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fc40cfc90 0x7f8fc406f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48764/0 (socket says 172.21.15.46:48764) 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 -- 172.21.15.46:0/652347578 learned_addr learned my addr 172.21.15.46:0/652347578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/652347578 >> v2:172.21.15.142:3300/0 conn(0x7f8fc40c4ea0 0x7f8fc40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 -- 172.21.15.46:0/652347578 >> v2:172.21.15.142:3300/0 conn(0x7f8fc40c4ea0 msgr2=0x7f8fc40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/652347578 >> v2:172.21.15.142:3300/0 conn(0x7f8fc40c4ea0 0x7f8fc40ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 -- 172.21.15.46:0/652347578 >> v1:172.21.15.142:6789/0 conn(0x7f8fc4163650 legacy=0x7f8fc406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.881+0000 7f8fc3e00640 1 -- 172.21.15.46:0/652347578 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc40d0400 con 0x7f8fc40cfc90 2024-04-26T23:49:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.883+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/652347578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc40cfc90 0x7f8fc406f180 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4167070 tx=0x7f8fbc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516293fc783a5691 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.883+0000 7f8fc2000640 1 -- 172.21.15.46:0/652347578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc037070 con 0x7f8fc40cfc90 2024-04-26T23:49:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.883+0000 7f8fc2000640 1 -- 172.21.15.46:0/652347578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc032440 con 0x7f8fc40cfc90 2024-04-26T23:49:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.883+0000 7f8fc2000640 1 -- 172.21.15.46:0/652347578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc032810 con 0x7f8fc40cfc90 2024-04-26T23:49:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 -- 172.21.15.46:0/652347578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc40cfc90 msgr2=0x7f8fc406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/652347578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc40cfc90 0x7f8fc406f180 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4167070 tx=0x7f8fbc02a230 comp rx=0 tx=0).stop 2024-04-26T23:49:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 -- 172.21.15.46:0/652347578 shutdown_connections 2024-04-26T23:49:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/652347578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc40cfc90 0x7f8fc406f180 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/652347578 >> v2:172.21.15.142:3300/0 conn(0x7f8fc40c4ea0 0x7f8fc40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 -- 172.21.15.46:0/652347578 >> 172.21.15.46:0/652347578 conn(0x7f8fc40d1c80 msgr2=0x7f8fc40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.884+0000 7f8fc8a00640 1 -- 172.21.15.46:0/652347578 shutdown_connections 2024-04-26T23:49:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.885+0000 7f8fc8a00640 1 -- 172.21.15.46:0/652347578 wait complete. 2024-04-26T23:49:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.887+0000 7f8fc8a00640 1 Processor -- start 2024-04-26T23:49:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.887+0000 7f8fc8a00640 1 -- start start 2024-04-26T23:49:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.888+0000 7f8fc8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 0x7f8fc4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.888+0000 7f8fc8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fc4163650 0x7f8fc4166020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.888+0000 7f8fc8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc406ca60 con 0x7f8fc4166560 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc40d09c0 con 0x7f8fc4163650 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc406cbe0 con 0x7f8fc40c4ea0 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48768/0 (socket says 172.21.15.46:48768) 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 -- 172.21.15.46:0/648081255 learned_addr learned my addr 172.21.15.46:0/648081255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fc4163650 0x7f8fc4166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3400640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 0x7f8fc4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 -- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 msgr2=0x7f8fc4154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 0x7f8fc4154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 -- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fc4163650 msgr2=0x7f8fc4166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fc4163650 0x7f8fc4166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3e00640 1 -- 172.21.15.46:0/648081255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc4164900 con 0x7f8fc4166560 2024-04-26T23:49:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.889+0000 7f8fc3400640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 0x7f8fc4154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0090b0 tx=0x7f8fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fbb400640 1 -- 172.21.15.46:0/648081255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc041020 con 0x7f8fc4166560 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fbb400640 1 -- 172.21.15.46:0/648081255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0306c0 con 0x7f8fc4166560 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fbb400640 1 -- 172.21.15.46:0/648081255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc037040 con 0x7f8fc4166560 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc4164be0 con 0x7f8fc4166560 2024-04-26T23:49:30.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.890+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41651a0 con 0x7f8fc4166560 2024-04-26T23:49:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.893+0000 7f8fbb400640 1 -- 172.21.15.46:0/648081255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fbc044020 con 0x7f8fc4166560 2024-04-26T23:49:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.894+0000 7f8fbb400640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.894+0000 7f8fbb400640 1 -- 172.21.15.46:0/648081255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f8fbc0ac540 con 0x7f8fc4166560 2024-04-26T23:49:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.895+0000 7f8fccfa2740 1 Processor -- start 2024-04-26T23:49:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.895+0000 7f8fc3400640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.895+0000 7f8fccfa2740 1 -- start start 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2ced0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ced620 0x561da2cc2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561da2c26f90 con 0x561da2d0ac40 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561da2bc31e0 con 0x561da2ced620 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561da2c22f50 con 0x561da2cedd60 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.896+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48772/0 (socket says 172.21.15.46:48772) 2024-04-26T23:49:30.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 -- 172.21.15.46:0/3522107578 learned_addr learned my addr 172.21.15.46:0/3522107578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2ced0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ced620 0x561da2cc2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 -- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 msgr2=0x561da2ced0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2ced0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 -- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ced620 msgr2=0x561da2cc2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ced620 0x561da2cc2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 -- 172.21.15.46:0/3522107578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da2a04ba0 con 0x561da2d0ac40 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2ced0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 0x7f8fa006edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007e90 tx=0x7f8fb40073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.897+0000 7f8fc3400640 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007aa0 tx=0x7f8fb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af9835c9a5a0436 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.898+0000 7f8fb3400640 1 -- 172.21.15.46:0/3522107578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003b70 con 0x561da2d0ac40 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.898+0000 7f8fb3400640 1 -- 172.21.15.46:0/3522107578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb4003d10 con 0x561da2d0ac40 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.898+0000 7f8fb3400640 1 -- 172.21.15.46:0/3522107578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004240 con 0x561da2d0ac40 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.898+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3522107578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 msgr2=0x561da2cbd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.898+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007aa0 tx=0x7f8fb4019000 comp rx=0 tx=0).stop 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3522107578 shutdown_connections 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ced620 0x561da2cc2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2ced0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3522107578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2cbd580 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3522107578 >> 172.21.15.46:0/3522107578 conn(0x561da2cbf840 msgr2=0x561da2d080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3522107578 shutdown_connections 2024-04-26T23:49:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3522107578 wait complete. 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.899+0000 7f8fccfa2740 1 -- start start 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.900+0000 7f8fccfa2740 1 Processor -- start 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d07540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2d0ac40 0x561da2cc2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561da2cc19f0 con 0x561da2ce8af0 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561da2badda0 con 0x561da2d0ac40 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561da2cc0de0 con 0x561da2cedd60 2024-04-26T23:49:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fccfa2740 10 client.0 _ll_register_callbacks cb 0x561da2bda400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48776/0 (socket says 172.21.15.46:48776) 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 -- 172.21.15.46:0/2176350473 learned_addr learned my addr 172.21.15.46:0/2176350473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3400640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d07540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2d0ac40 0x561da2cc2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 -- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 msgr2=0x561da2d07540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d07540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 -- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2d0ac40 msgr2=0x561da2cc2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2d0ac40 0x561da2cc2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.901+0000 7f8fc3e00640 1 -- 172.21.15.46:0/2176350473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da28f8490 con 0x561da2ce8af0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc035040 tx=0x7f8fbc079c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d4cfb9ffd1996e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc07a040 con 0x561da2ce8af0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb4003d10 con 0x561da2ce8af0 2024-04-26T23:49:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f8fbc0ac0b0 con 0x561da2ce8af0 2024-04-26T23:49:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc07e3f0 con 0x561da2ce8af0 2024-04-26T23:49:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fccfa2740 10 client.8883 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561da2a65330 con 0x561da2ce8af0 2024-04-26T23:49:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.902+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb4003d10 con 0x561da2ce8af0 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f8fbc03a030 con 0x561da2ce8af0 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 10 client.8883 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 10 client.8883 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 20 client.8883 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d18b70 0x561da2cee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 4 client.8883 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.903+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da2c34ba0 con 0x561da2d18b70 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.904+0000 7f8fc3400640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d18b70 0x561da2cee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.905+0000 7f8fc3400640 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d18b70 0x561da2cee250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b81d33c029f7345 server_cookie=59b4a39cddf204ff in_seq=0 out_seq=0 2024-04-26T23:49:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.905+0000 7f8fa7e00640 10 client.8883 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.907+0000 7f8fa7e00640 1 -- 172.21.15.46:0/2176350473 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561da2c34ba0 con 0x561da2d18b70 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.907+0000 7f8fa7e00640 10 client.8883 handle_command_reply: tid=0 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 client.8883 shutdown 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 msgr2=0x561da2cc3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc035040 tx=0x7f8fbc079c80 comp rx=0 tx=0).stop 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 shutdown_connections 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d18b70 0x561da2cee250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2cc3330 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2d0ac40 0x561da2cc2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/2176350473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d07540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fa7e00640 0 client.8883 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 >> 172.21.15.46:0/2176350473 conn(0x561da2ced620 msgr2=0x561da2d080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 shutdown_connections 2024-04-26T23:49:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 1 -- 172.21.15.46:0/2176350473 wait complete. 2024-04-26T23:49:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.908+0000 7f8fccfa2740 20 client.8883 trim_cache size 0 max 16384 2024-04-26T23:49:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.928+0000 7f8fccfa2740 1 Processor -- start 2024-04-26T23:49:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.928+0000 7f8fccfa2740 1 -- start start 2024-04-26T23:49:30.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.928+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 0x561da2d06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d06e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc42060f0 con 0x561da2d0ac40 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4205170 con 0x561da2ce8af0 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc40c6e70 con 0x561da2cedd60 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48788/0 (socket says 172.21.15.46:48788) 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 -- 172.21.15.46:0/3826823275 learned_addr learned my addr 172.21.15.46:0/3826823275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3400640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 0x561da2d06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 -- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 msgr2=0x561da2d06e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d06e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 -- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 msgr2=0x561da2d06900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 0x561da2d06900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3e00640 1 -- 172.21.15.46:0/3826823275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da28d1610 con 0x561da2d0ac40 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.929+0000 7f8fc3400640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 0x561da2d06900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0782f0 tx=0x7f8fbc047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909396b050c8405d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fb2a00640 1 -- 172.21.15.46:0/3826823275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc07a040 con 0x561da2d0ac40 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fb2a00640 1 -- 172.21.15.46:0/3826823275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0ac2b0 con 0x561da2d0ac40 2024-04-26T23:49:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fb2a00640 1 -- 172.21.15.46:0/3826823275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0829d0 con 0x561da2d0ac40 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3826823275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 msgr2=0x561da2d01820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.930+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0782f0 tx=0x7f8fbc047000 comp rx=0 tx=0).stop 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3826823275 shutdown_connections 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2d0ac40 0x561da2d01820 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2cedd60 0x561da2d06e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/3826823275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2ce8af0 0x561da2d06900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3826823275 >> 172.21.15.46:0/3826823275 conn(0x561da2ced620 msgr2=0x561da2cc03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3826823275 shutdown_connections 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 -- 172.21.15.46:0/3826823275 wait complete. 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.931+0000 7f8fccfa2740 1 -- start start 2024-04-26T23:49:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.932+0000 7f8fccfa2740 1 Processor -- start 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2cedd60 0x561da2d17210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2d0ac40 0x561da2d068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561da2d17de0 con 0x561da2ce8af0 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4204b20 con 0x561da2cedd60 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561da2cc14b0 con 0x561da2d0ac40 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fccfa2740 10 client.0 _ll_register_callbacks cb 0x7f8fbc006cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48796/0 (socket says 172.21.15.46:48796) 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fc3400640 1 -- 172.21.15.46:0/1433243376 learned_addr learned my addr 172.21.15.46:0/1433243376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2cedd60 0x561da2d17210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.933+0000 7f8fc3e00640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2d0ac40 0x561da2d068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 -- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2d0ac40 msgr2=0x561da2d068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2d0ac40 0x561da2d068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 -- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2cedd60 msgr2=0x561da2d17210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2cedd60 0x561da2d17210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 -- 172.21.15.46:0/1433243376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da2ce73f0 con 0x561da2ce8af0 2024-04-26T23:49:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fc3400640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400cf80 tx=0x7f8fb4004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b443b44ac0d744c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb401d650 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f8fb401d7f0 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.934+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc0ac2b0 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f8fb403d050 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 10 client.8889 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561da2d0bac0 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0ac2b0 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f8fb4040020 con 0x561da2ce8af0 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 10 client.8889 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 10 client.8889 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 20 client.8889 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d06e20 0x561da2ce9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 4 client.8889 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.935+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x561da2cbec40 con 0x561da2d06e20 2024-04-26T23:49:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.936+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d06e20 0x561da2ce9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.937+0000 7f8fc2a00640 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d06e20 0x561da2ce9a50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90e4cd40f76206a server_cookie=827eb9df11c128dc in_seq=0 out_seq=0 2024-04-26T23:49:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.937+0000 7f8fa7e00640 10 client.8889 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.943+0000 7f8fa7e00640 1 -- 172.21.15.46:0/1433243376 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561da2cbec40 con 0x561da2d06e20 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.943+0000 7f8fa7e00640 10 client.8889 handle_command_reply: tid=0 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.943+0000 7f8fccfa2740 1 client.8889 shutdown 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 msgr2=0x561da2d85140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400cf80 tx=0x7f8fb4004a00 comp rx=0 tx=0).stop 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 shutdown_connections 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x561da2d06e20 0x561da2ce9a50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561da2d0ac40 0x561da2d068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da2cedd60 0x561da2d17210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 --2- 172.21.15.46:0/1433243376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da2ce8af0 0x561da2d85140 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 >> 172.21.15.46:0/1433243376 conn(0x561da2ced620 msgr2=0x561da2d07cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fa7e00640 0 client.8889 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 shutdown_connections 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 1 -- 172.21.15.46:0/1433243376 wait complete. 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.944+0000 7f8fccfa2740 20 client.8889 trim_cache size 0 max 16384 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:49:30.949 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "854b1853-a265-4e82-a185-a41359fb0332", 2024-04-26T23:49:30.950 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:49:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:49:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 msgr2=0x7f8fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 0x7f8fa006edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007e90 tx=0x7f8fb40073a0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 msgr2=0x7f8fc4163c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0090b0 tx=0x7f8fbc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 shutdown_connections 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc4166560 0x7f8fc4163c10 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fc4163650 0x7f8fc4166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 --2- 172.21.15.46:0/648081255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fc40c4ea0 0x7f8fc4154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 >> 172.21.15.46:0/648081255 conn(0x7f8fc40d1c80 msgr2=0x7f8fc4161c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 shutdown_connections 2024-04-26T23:49:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:30.945+0000 7f8fc8a00640 1 -- 172.21.15.46:0/648081255 wait complete. 2024-04-26T23:49:30.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:49:30.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-26T23:49:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.201+0000 7fd25ee00640 1 Processor -- start 2024-04-26T23:49:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.201+0000 7fd25ee00640 1 -- start start 2024-04-26T23:49:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.202+0000 7fd25ee00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd25815d730 0x7fd2580d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.202+0000 7fd25ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25806eb90 con 0x7fd2580b53b0 2024-04-26T23:49:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ee00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd2580b4310 con 0x7fd25815d730 2024-04-26T23:49:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580b45f0 con 0x7fd25815e220 2024-04-26T23:49:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:35990/0 (socket says 172.21.15.46:35990) 2024-04-26T23:49:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd25ce00640 1 -- 172.21.15.46:0/1507408571 learned_addr learned my addr 172.21.15.46:0/1507408571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.203+0000 7fd257e00640 1 --2- 172.21.15.46:0/1507408571 >> v2:172.21.15.142:3300/0 conn(0x7fd25815d730 0x7fd2580d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.204+0000 7fd25ce00640 1 -- 172.21.15.46:0/1507408571 >> v2:172.21.15.142:3300/0 conn(0x7fd25815d730 msgr2=0x7fd2580d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.204+0000 7fd25ce00640 1 --2- 172.21.15.46:0/1507408571 >> v2:172.21.15.142:3300/0 conn(0x7fd25815d730 0x7fd2580d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.204+0000 7fd25ce00640 1 -- 172.21.15.46:0/1507408571 >> v1:172.21.15.142:6789/0 conn(0x7fd2580b53b0 legacy=0x7fd25815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.204+0000 7fd25ce00640 1 -- 172.21.15.46:0/1507408571 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2580b48d0 con 0x7fd25815e220 2024-04-26T23:49:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.205+0000 7fd25ce00640 1 --2- 172.21.15.46:0/1507408571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd250009e50 tx=0x7fd25002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c360f3d916d1e0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd257400640 1 -- 172.21.15.46:0/1507408571 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250036070 con 0x7fd25815e220 2024-04-26T23:49:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd257400640 1 -- 172.21.15.46:0/1507408571 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250031600 con 0x7fd25815e220 2024-04-26T23:49:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd257400640 1 -- 172.21.15.46:0/1507408571 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500319d0 con 0x7fd25815e220 2024-04-26T23:49:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 -- 172.21.15.46:0/1507408571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 msgr2=0x7fd25806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 --2- 172.21.15.46:0/1507408571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd250009e50 tx=0x7fd25002a710 comp rx=0 tx=0).stop 2024-04-26T23:49:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 -- 172.21.15.46:0/1507408571 shutdown_connections 2024-04-26T23:49:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 --2- 172.21.15.46:0/1507408571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25815e220 0x7fd25806dc90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 --2- 172.21.15.46:0/1507408571 >> v2:172.21.15.142:3300/0 conn(0x7fd25815d730 0x7fd2580d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.206+0000 7fd25ee00640 1 -- 172.21.15.46:0/1507408571 >> 172.21.15.46:0/1507408571 conn(0x7fd25806c980 msgr2=0x7fd2580b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.207+0000 7fd25ee00640 1 -- 172.21.15.46:0/1507408571 shutdown_connections 2024-04-26T23:49:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.207+0000 7fd25ee00640 1 -- 172.21.15.46:0/1507408571 wait complete. 2024-04-26T23:49:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.208+0000 7fd25ee00640 1 Processor -- start 2024-04-26T23:49:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.209+0000 7fd25ee00640 1 -- start start 2024-04-26T23:49:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2580b53b0 0x7fd258140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25815e220 0x7fd2580725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580b3ec0 con 0x7fd25815d730 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580b4e90 con 0x7fd25815e220 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd25ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580b4040 con 0x7fd2580b53b0 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48812/0 (socket says 172.21.15.46:48812) 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd24fe00640 1 -- 172.21.15.46:0/412280375 learned_addr learned my addr 172.21.15.46:0/412280375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.210+0000 7fd257e00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2580b53b0 0x7fd258140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd25ce00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25815e220 0x7fd2580725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 -- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2580b53b0 msgr2=0x7fd258140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2580b53b0 0x7fd258140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 -- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25815e220 msgr2=0x7fd2580725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25815e220 0x7fd2580725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 -- 172.21.15.46:0/412280375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd258141600 con 0x7fd25815d730 2024-04-26T23:49:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd24fe00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c3f0 tx=0x7fd24000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd254c00640 1 -- 172.21.15.46:0/412280375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400052d0 con 0x7fd25815d730 2024-04-26T23:49:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd254c00640 1 -- 172.21.15.46:0/412280375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240005490 con 0x7fd25815d730 2024-04-26T23:49:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd254c00640 1 -- 172.21.15.46:0/412280375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240007260 con 0x7fd25815d730 2024-04-26T23:49:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.211+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2581418e0 con 0x7fd25815d730 2024-04-26T23:49:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.212+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2580720b0 con 0x7fd25815d730 2024-04-26T23:49:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.215+0000 7fd254c00640 1 -- 172.21.15.46:0/412280375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd240007400 con 0x7fd25815d730 2024-04-26T23:49:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.216+0000 7fd254c00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.216+0000 7fd254c00640 1 -- 172.21.15.46:0/412280375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fd24008b190 con 0x7fd25815d730 2024-04-26T23:49:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.216+0000 7fd257e00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.216+0000 7fd261bdd740 1 Processor -- start 2024-04-26T23:49:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.216+0000 7fd261bdd740 1 -- start start 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.217+0000 7fd257e00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd258069fc0 tx=0x7fd248009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.217+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad33c30 0x55728ad79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd261bdd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55728ac97900 con 0x55728ad5df50 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55728ac341d0 con 0x55728ad33c30 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55728ac938c0 con 0x55728ad7b7f0 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48818/0 (socket says 172.21.15.46:48818) 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 -- 172.21.15.46:0/3213177365 learned_addr learned my addr 172.21.15.46:0/3213177365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd24fe00640 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd25ce00640 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad33c30 0x55728ad79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 -- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 msgr2=0x55728ad7bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad7bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 -- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad33c30 msgr2=0x55728ad79eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad33c30 0x55728ad79eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.218+0000 7fd257e00640 1 -- 172.21.15.46:0/3213177365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728aa737d0 con 0x55728ad5df50 2024-04-26T23:49:31.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.219+0000 7fd257e00640 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd248006a90 tx=0x7fd248014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad1207964144c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.219+0000 7fd24cc00640 1 -- 172.21.15.46:0/3213177365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24801b070 con 0x55728ad5df50 2024-04-26T23:49:31.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.219+0000 7fd24cc00640 1 -- 172.21.15.46:0/3213177365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248027070 con 0x55728ad5df50 2024-04-26T23:49:31.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.219+0000 7fd24cc00640 1 -- 172.21.15.46:0/3213177365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248003a20 con 0x55728ad5df50 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 -- 172.21.15.46:0/3213177365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 msgr2=0x55728ad5e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd248006a90 tx=0x7fd248014000 comp rx=0 tx=0).stop 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 -- 172.21.15.46:0/3213177365 shutdown_connections 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad33c30 0x55728ad79eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3213177365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad5e330 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 -- 172.21.15.46:0/3213177365 >> 172.21.15.46:0/3213177365 conn(0x55728ad30560 msgr2=0x55728ad78d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 -- 172.21.15.46:0/3213177365 shutdown_connections 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.220+0000 7fd261bdd740 1 -- 172.21.15.46:0/3213177365 wait complete. 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.221+0000 7fd261bdd740 1 -- start start 2024-04-26T23:49:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.221+0000 7fd261bdd740 1 Processor -- start 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad60f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55728ad349d0 con 0x55728ad5df50 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55728ac1f120 con 0x55728ad7b7f0 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55728ad2e490 con 0x55728ad5bf70 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd261bdd740 10 client.0 _ll_register_callbacks cb 0x55728ac57750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd257e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd257e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48830/0 (socket says 172.21.15.46:48830) 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.222+0000 7fd257e00640 1 -- 172.21.15.46:0/1324956502 learned_addr learned my addr 172.21.15.46:0/1324956502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd25ce00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad60f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd24fe00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 -- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 msgr2=0x55728ad60f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad60f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 -- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 msgr2=0x55728ad744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 -- 172.21.15.46:0/1324956502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728a9690f0 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd24fe00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd257e00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd248024040 tx=0x7fd248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6425e74a5ae6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd245600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24801b040 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fd248036c70 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.223+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248027070 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fd24803d050 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 10 client.8898 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55728aad3f60 con 0x55728ad5df50 2024-04-26T23:49:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248027070 con 0x55728ad5df50 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fd248040020 con 0x55728ad5df50 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 10 client.8898 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 10 client.8898 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 20 client.8898 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad702e0 0x55728ad70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 4 client.8898 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.224+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55728aca5510 con 0x55728ad702e0 2024-04-26T23:49:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.225+0000 7fd24fe00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad702e0 0x55728ad70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.226+0000 7fd24fe00640 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad702e0 0x55728ad70720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d409a3ae3a1087 server_cookie=f5d38bc6cfb10ad1 in_seq=0 out_seq=0 2024-04-26T23:49:31.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.226+0000 7fd245600640 10 client.8898 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd245600640 1 -- 172.21.15.46:0/1324956502 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55728aca5510 con 0x55728ad702e0 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd245600640 10 client.8898 handle_command_reply: tid=0 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 client.8898 shutdown 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 msgr2=0x55728ad73f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd248024040 tx=0x7fd248004730 comp rx=0 tx=0).stop 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 shutdown_connections 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad702e0 0x55728ad70720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad60f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 --2- 172.21.15.46:0/1324956502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728ad73f70 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd245600640 0 client.8898 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 >> 172.21.15.46:0/1324956502 conn(0x55728ad33c30 msgr2=0x55728ad7c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 shutdown_connections 2024-04-26T23:49:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 1 -- 172.21.15.46:0/1324956502 wait complete. 2024-04-26T23:49:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.228+0000 7fd261bdd740 20 client.8898 trim_cache size 0 max 16384 2024-04-26T23:49:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.247+0000 7fd261bdd740 1 Processor -- start 2024-04-26T23:49:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.247+0000 7fd261bdd740 1 -- start start 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad31100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad75060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd258206310 con 0x55728ad5df50 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd258205350 con 0x55728ad7b7f0 2024-04-26T23:49:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.248+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2581fd520 con 0x55728ad5bf70 2024-04-26T23:49:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48844/0 (socket says 172.21.15.46:48844) 2024-04-26T23:49:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 -- 172.21.15.46:0/2287559366 learned_addr learned my addr 172.21.15.46:0/2287559366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd257e00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad31100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd25ce00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad75060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 -- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 msgr2=0x55728ad31100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad31100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 -- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 msgr2=0x55728ad75060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad75060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd24fe00640 1 -- 172.21.15.46:0/2287559366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728a9421b0 con 0x55728ad5df50 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd25ce00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad75060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.249+0000 7fd257e00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad31100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd24fe00640 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd24005b010 tx=0x7fd240071040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332853feeca47c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd247e00640 1 -- 172.21.15.46:0/2287559366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400895a0 con 0x55728ad5df50 2024-04-26T23:49:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd247e00640 1 -- 172.21.15.46:0/2287559366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240057040 con 0x55728ad5df50 2024-04-26T23:49:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd247e00640 1 -- 172.21.15.46:0/2287559366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400710b0 con 0x55728ad5df50 2024-04-26T23:49:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 -- 172.21.15.46:0/2287559366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 msgr2=0x55728adecff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd24005b010 tx=0x7fd240071040 comp rx=0 tx=0).stop 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 -- 172.21.15.46:0/2287559366 shutdown_connections 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad7b7f0 0x55728ad75060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adecff0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 --2- 172.21.15.46:0/2287559366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad5bf70 0x55728ad31100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.250+0000 7fd261bdd740 1 -- 172.21.15.46:0/2287559366 >> 172.21.15.46:0/2287559366 conn(0x55728ad33c30 msgr2=0x55728ad60c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.251+0000 7fd261bdd740 1 -- 172.21.15.46:0/2287559366 shutdown_connections 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.251+0000 7fd261bdd740 1 -- 172.21.15.46:0/2287559366 wait complete. 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.251+0000 7fd261bdd740 1 -- start start 2024-04-26T23:49:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.251+0000 7fd261bdd740 1 Processor -- start 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 0x55728ad2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad74f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55728ad2e7c0 con 0x55728ad5df50 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd258204d00 con 0x55728ad5bf70 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55728aded530 con 0x55728ad7b7f0 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.252+0000 7fd261bdd740 10 client.0 _ll_register_callbacks cb 0x7fd248011db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48854/0 (socket says 172.21.15.46:48854) 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 -- 172.21.15.46:0/3263917395 learned_addr learned my addr 172.21.15.46:0/3263917395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd25ce00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad74f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd257e00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 0x55728ad2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 -- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 msgr2=0x55728ad74f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad74f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 -- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 msgr2=0x55728ad2d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 0x55728ad2d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 -- 172.21.15.46:0/3263917395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728adec1b0 con 0x55728ad5df50 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd257e00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 0x55728ad2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd24fe00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd240089460 tx=0x7fd240000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b1fa8d84aca182 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.253+0000 7fd245600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400895a0 con 0x55728ad5df50 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fd2400553b0 con 0x55728ad5df50 2024-04-26T23:49:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd240057040 con 0x55728ad5df50 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fd2400782d0 con 0x55728ad5df50 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 10 client.8904 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55728ad754c0 con 0x55728ad5df50 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240057040 con 0x55728ad5df50 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fd240058190 con 0x55728ad5df50 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 10 client.8904 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 10 client.8904 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 20 client.8904 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad79e90 0x55728ad7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 4 client.8904 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.254+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55728ad75c70 con 0x55728ad79e90 2024-04-26T23:49:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.255+0000 7fd257e00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad79e90 0x55728ad7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.256+0000 7fd257e00640 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad79e90 0x55728ad7a270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc1543207fbe3c server_cookie=9b2dcf954c79ec6f in_seq=0 out_seq=0 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.256+0000 7fd245600640 10 client.8904 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd245600640 1 -- 172.21.15.46:0/3263917395 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55728ad75c70 con 0x55728ad79e90 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd245600640 10 client.8904 handle_command_reply: tid=0 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 client.8904 shutdown 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 msgr2=0x55728adeafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd240089460 tx=0x7fd240000ed0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 shutdown_connections 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x55728ad79e90 0x55728ad7a270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55728ad7b7f0 0x55728ad74f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd245600640 0 client.8904 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55728ad5df50 0x55728adeafc0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 --2- 172.21.15.46:0/3263917395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55728ad5bf70 0x55728ad2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 >> 172.21.15.46:0/3263917395 conn(0x55728ad33c30 msgr2=0x55728ad5cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 shutdown_connections 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 1 -- 172.21.15.46:0/3263917395 wait complete. 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.257+0000 7fd261bdd740 20 client.8904 trim_cache size 0 max 16384 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:49:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:49:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 msgr2=0x7fd23406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd258069fc0 tx=0x7fd248009190 comp rx=0 tx=0).stop 2024-04-26T23:49:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 msgr2=0x7fd2581efbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c3f0 tx=0x7fd24000c8b0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 shutdown_connections 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25815e220 0x7fd2580725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25815d730 0x7fd2581efbf0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 --2- 172.21.15.46:0/412280375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2580b53b0 0x7fd258140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 >> 172.21.15.46:0/412280375 conn(0x7fd25806c980 msgr2=0x7fd2580b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 shutdown_connections 2024-04-26T23:49:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.258+0000 7fd25ee00640 1 -- 172.21.15.46:0/412280375 wait complete. 2024-04-26T23:49:31.279 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:49:31.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-26T23:49:31.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.479+0000 7f148b000640 1 Processor -- start 2024-04-26T23:49:31.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.479+0000 7f148b000640 1 -- start start 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14841595c0 0x7f1484073a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f14840d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 0x7f1484162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f14840ae8e0 con 0x7f14841595c0 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f14840aeb30 con 0x7f14840d0cc0 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f148b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14840aee10 con 0x7f1484071f90 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f1489c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14841595c0 0x7f1484073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f1489c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14841595c0 0x7f1484073a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48866/0 (socket says 172.21.15.46:48866) 2024-04-26T23:49:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.481+0000 7f1489c00640 1 -- 172.21.15.46:0/475718877 learned_addr learned my addr 172.21.15.46:0/475718877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489200640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f14840d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489c00640 1 -- 172.21.15.46:0/475718877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 msgr2=0x7f14840d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f148a600640 1 --2- 172.21.15.46:0/475718877 >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 0x7f1484162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489c00640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f14840d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489c00640 1 -- 172.21.15.46:0/475718877 >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 msgr2=0x7f1484162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489c00640 1 --2- 172.21.15.46:0/475718877 >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 0x7f1484162bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489c00640 1 -- 172.21.15.46:0/475718877 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14840af0f0 con 0x7f14841595c0 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f1489200640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f14840d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.482+0000 7f148a600640 1 --2- 172.21.15.46:0/475718877 >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 0x7f1484162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.483+0000 7f1489c00640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14841595c0 0x7f1484073a30 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f148406fe20 tx=0x7f147802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f6f4a6f894e7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.483+0000 7f1483e00640 1 -- 172.21.15.46:0/475718877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478037070 con 0x7f14841595c0 2024-04-26T23:49:31.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.483+0000 7f1483e00640 1 -- 172.21.15.46:0/475718877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1478032440 con 0x7f14841595c0 2024-04-26T23:49:31.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.483+0000 7f1483e00640 1 -- 172.21.15.46:0/475718877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478032810 con 0x7f14841595c0 2024-04-26T23:49:31.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 -- 172.21.15.46:0/475718877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14841595c0 msgr2=0x7f1484073a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14841595c0 0x7f1484073a30 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f148406fe20 tx=0x7f147802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 -- 172.21.15.46:0/475718877 shutdown_connections 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 --2- 172.21.15.46:0/475718877 >> v2:172.21.15.142:3300/0 conn(0x7f14840d0cc0 0x7f1484162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f14840d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 --2- 172.21.15.46:0/475718877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14841595c0 0x7f1484073a30 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 -- 172.21.15.46:0/475718877 >> 172.21.15.46:0/475718877 conn(0x7f148406c350 msgr2=0x7f148406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 -- 172.21.15.46:0/475718877 shutdown_connections 2024-04-26T23:49:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.484+0000 7f148b000640 1 -- 172.21.15.46:0/475718877 wait complete. 2024-04-26T23:49:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.486+0000 7f148b000640 1 Processor -- start 2024-04-26T23:49:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.486+0000 7f148b000640 1 -- start start 2024-04-26T23:49:31.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f148414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 0x7f14840cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14841599a0 con 0x7f14840d0cc0 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14840af6b0 con 0x7f14841595c0 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.487+0000 7f148b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1484073f70 con 0x7f1484071f90 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48876/0 (socket says 172.21.15.46:48876) 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 -- 172.21.15.46:0/1758002570 learned_addr learned my addr 172.21.15.46:0/1758002570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489c00640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f148414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f148a600640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 0x7f14840cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 -- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 msgr2=0x7f148414af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f148414af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 -- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 msgr2=0x7f14840cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 0x7f14840cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 -- 172.21.15.46:0/1758002570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14840ce580 con 0x7f14840d0cc0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f148a600640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 0x7f14840cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.488+0000 7f1489200640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f147000c820 tx=0x7f147000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.489+0000 7f1481600640 1 -- 172.21.15.46:0/1758002570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470015070 con 0x7f14840d0cc0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.489+0000 7f1481600640 1 -- 172.21.15.46:0/1758002570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470007a30 con 0x7f14840d0cc0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.489+0000 7f1481600640 1 -- 172.21.15.46:0/1758002570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470007e00 con 0x7f14840d0cc0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.489+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14840ce860 con 0x7f14840d0cc0 2024-04-26T23:49:31.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.489+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841f4cd0 con 0x7f14840d0cc0 2024-04-26T23:49:31.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.491+0000 7f1481600640 1 -- 172.21.15.46:0/1758002570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1470018070 con 0x7f14840d0cc0 2024-04-26T23:49:31.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.493+0000 7f1481600640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.493+0000 7f1481600640 1 -- 172.21.15.46:0/1758002570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f147008bd20 con 0x7f14840d0cc0 2024-04-26T23:49:31.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.494+0000 7f148f3ad740 1 Processor -- start 2024-04-26T23:49:31.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.494+0000 7f1489c00640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.494+0000 7f148f3ad740 1 -- start start 2024-04-26T23:49:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67903f0 0x564ef67d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ef66a0030 con 0x564ef67ba710 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ef66bb7f0 con 0x564ef67d7fb0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564ef66767e0 con 0x564ef67903f0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f1489c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f1489c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48878/0 (socket says 172.21.15.46:48878) 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f1489c00640 1 -- 172.21.15.46:0/961870343 learned_addr learned my addr 172.21.15.46:0/961870343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f1489c00640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 0x7f146406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f147802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f148a600640 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67903f0 0x564ef67d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.495+0000 7f1489200640 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 -- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67903f0 msgr2=0x564ef67d6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67903f0 0x564ef67d6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 -- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 msgr2=0x564ef67d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67d8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 -- 172.21.15.46:0/961870343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef6682050 con 0x564ef67ba710 2024-04-26T23:49:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1489c00640 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f1478009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d071c9487ae9fda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1476000640 1 -- 172.21.15.46:0/961870343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147804e070 con 0x564ef67ba710 2024-04-26T23:49:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1476000640 1 -- 172.21.15.46:0/961870343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147803e660 con 0x564ef67ba710 2024-04-26T23:49:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.496+0000 7f1476000640 1 -- 172.21.15.46:0/961870343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478037040 con 0x564ef67ba710 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 -- 172.21.15.46:0/961870343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 msgr2=0x564ef67baaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f1478009240 comp rx=0 tx=0).stop 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 -- 172.21.15.46:0/961870343 shutdown_connections 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67903f0 0x564ef67d6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 --2- 172.21.15.46:0/961870343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67ba710 0x564ef67baaf0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 -- 172.21.15.46:0/961870343 >> 172.21.15.46:0/961870343 conn(0x564ef678cd20 msgr2=0x564ef67d5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 -- 172.21.15.46:0/961870343 shutdown_connections 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.497+0000 7f148f3ad740 1 -- 172.21.15.46:0/961870343 wait complete. 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.498+0000 7f148f3ad740 1 -- start start 2024-04-26T23:49:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.498+0000 7f148f3ad740 1 Processor -- start 2024-04-26T23:49:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 0x564ef67a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ef66f8200 con 0x564ef67aa080 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ef66f2ee0 con 0x564ef67ba710 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564ef66ffe70 con 0x564ef67d7fb0 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.499+0000 7f148f3ad740 10 client.0 _ll_register_callbacks cb 0x564ef66e9270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48886/0 (socket says 172.21.15.46:48886) 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 -- 172.21.15.46:0/3552658470 learned_addr learned my addr 172.21.15.46:0/3552658470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 -- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 msgr2=0x564ef67a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f1489200640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 0x564ef67a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f1489c00640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 0x564ef67a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 -- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 msgr2=0x564ef67a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 -- 172.21.15.46:0/3552658470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef64cf7d0 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f1489200640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 0x564ef67a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f148a600640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f147c00dbb0 tx=0x7f147c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7772a9e434e57d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.500+0000 7f146aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0095a0 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f147c009740 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147803e660 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f147c022050 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f148f3ad740 10 client.8916 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x564ef64053f0 con 0x564ef67aa080 2024-04-26T23:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147803e660 con 0x564ef67aa080 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f147c025020 con 0x564ef67aa080 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.501+0000 7f148f3ad740 10 client.8916 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f148f3ad740 10 client.8916 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f148f3ad740 20 client.8916 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef67aada0 0x564ef67bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f148f3ad740 4 client.8916 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef67017e0 con 0x564ef67aada0 2024-04-26T23:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.502+0000 7f1489c00640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef67aada0 0x564ef67bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.503+0000 7f1489c00640 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef67aada0 0x564ef67bd3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ad53141fa09c10 server_cookie=8cefcd6b7371bfaf in_seq=0 out_seq=0 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.503+0000 7f146aa00640 10 client.8916 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.505+0000 7f146aa00640 1 -- 172.21.15.46:0/3552658470 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ef67017e0 con 0x564ef67aada0 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.505+0000 7f146aa00640 10 client.8916 handle_command_reply: tid=0 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.505+0000 7f148f3ad740 1 client.8916 shutdown 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 msgr2=0x564ef67a8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f147c00dbb0 tx=0x7f147c00b7e0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 shutdown_connections 2024-04-26T23:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef67aada0 0x564ef67bd3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67a8cd0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67d7fb0 0x564ef67a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 --2- 172.21.15.46:0/3552658470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f146aa00640 0 client.8916 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 >> 172.21.15.46:0/3552658470 conn(0x564ef67903f0 msgr2=0x564ef678f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 shutdown_connections 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 1 -- 172.21.15.46:0/3552658470 wait complete. 2024-04-26T23:49:31.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.506+0000 7f148f3ad740 20 client.8916 trim_cache size 0 max 16384 2024-04-26T23:49:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.525+0000 7f148f3ad740 1 Processor -- start 2024-04-26T23:49:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.525+0000 7f148f3ad740 1 -- start start 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67aa080 0x564ef67a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1484201990 con 0x564ef67d7fb0 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1484200970 con 0x564ef67ba710 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14841f8980 con 0x564ef67aa080 2024-04-26T23:49:31.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.526+0000 7f148a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48900/0 (socket says 172.21.15.46:48900) 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 -- 172.21.15.46:0/1599451611 learned_addr learned my addr 172.21.15.46:0/1599451611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f1489200640 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f1489c00640 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67aa080 0x564ef67a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 -- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67aa080 msgr2=0x564ef67a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67aa080 0x564ef67a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 -- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 msgr2=0x564ef67a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 -- 172.21.15.46:0/1599451611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef63c50f0 con 0x564ef67d7fb0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.527+0000 7f148a600640 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f147c000f80 tx=0x7f147c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6e9f5307834083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f1475600640 1 -- 172.21.15.46:0/1599451611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c009c00 con 0x564ef67d7fb0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f1475600640 1 -- 172.21.15.46:0/1599451611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0095a0 con 0x564ef67d7fb0 2024-04-26T23:49:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f1475600640 1 -- 172.21.15.46:0/1599451611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00d250 con 0x564ef67d7fb0 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 -- 172.21.15.46:0/1599451611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 msgr2=0x564ef67a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f147c000f80 tx=0x7f147c014000 comp rx=0 tx=0).stop 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 -- 172.21.15.46:0/1599451611 shutdown_connections 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67d7fb0 0x564ef67a98a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67ba710 0x564ef67a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 --2- 172.21.15.46:0/1599451611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67aa080 0x564ef67a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 -- 172.21.15.46:0/1599451611 >> 172.21.15.46:0/1599451611 conn(0x564ef67903f0 msgr2=0x564ef67d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 -- 172.21.15.46:0/1599451611 shutdown_connections 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.528+0000 7f148f3ad740 1 -- 172.21.15.46:0/1599451611 wait complete. 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.529+0000 7f148f3ad740 1 -- start start 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.529+0000 7f148f3ad740 1 Processor -- start 2024-04-26T23:49:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67ba710 0x564ef67cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ef6700fd0 con 0x564ef67aa080 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1484200330 con 0x564ef67d7fb0 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564ef6790270 con 0x564ef67ba710 2024-04-26T23:49:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.530+0000 7f148f3ad740 10 client.0 _ll_register_callbacks cb 0x7f147c0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:31.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48910/0 (socket says 172.21.15.46:48910) 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 -- 172.21.15.46:0/2859134003 learned_addr learned my addr 172.21.15.46:0/2859134003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 -- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67ba710 msgr2=0x564ef67cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67ba710 0x564ef67cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f148a600640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 -- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 msgr2=0x564ef67f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 -- 172.21.15.46:0/2859134003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef66bba30 con 0x564ef67aa080 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f148a600640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f1489c00640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1478009960 tx=0x7f1478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a42c369e14c694a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.531+0000 7f146aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478037040 con 0x564ef67aa080 2024-04-26T23:49:31.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f14780594a0 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c0095a0 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f1478063050 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 10 client.8922 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x564ef67b8f90 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0095a0 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f1478066020 con 0x564ef67aa080 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 10 client.8922 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 10 client.8922 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 20 client.8922 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef6790ba0 0x564ef67f8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 4 client.8922 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.532+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ef66fb650 con 0x564ef6790ba0 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.533+0000 7f1489200640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef6790ba0 0x564ef67f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.534+0000 7f1489200640 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef6790ba0 0x564ef67f8650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9403cb40bd09f3 server_cookie=16f5ce5b6ba6d227 in_seq=0 out_seq=0 2024-04-26T23:49:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.534+0000 7f146aa00640 10 client.8922 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f146aa00640 1 -- 172.21.15.46:0/2859134003 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x564ef66fb650 con 0x564ef6790ba0 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f146aa00640 10 client.8922 handle_command_reply: tid=0 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 client.8922 shutdown 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 msgr2=0x564ef67caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1478009960 tx=0x7f1478004730 comp rx=0 tx=0).stop 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 shutdown_connections 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x564ef6790ba0 0x564ef67f8650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ef67d7fb0 0x564ef67f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564ef67ba710 0x564ef67cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 --2- 172.21.15.46:0/2859134003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ef67aa080 0x564ef67caeb0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f146aa00640 0 client.8922 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 >> 172.21.15.46:0/2859134003 conn(0x564ef67903f0 msgr2=0x564ef67b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 shutdown_connections 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 1 -- 172.21.15.46:0/2859134003 wait complete. 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.535+0000 7f148f3ad740 20 client.8922 trim_cache size 0 max 16384 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "854b1853-a265-4e82-a185-a41359fb0332": { 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_control_test_path", 2024-04-26T23:49:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "854b1853-a265-4e82-a185-a41359fb0332", 2024-04-26T23:49:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-26T23:49:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:31.543 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:49:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 msgr2=0x7f146406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 0x7f146406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f147802e000 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 msgr2=0x7f14841f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f147000c820 tx=0x7f147000cc20 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 shutdown_connections 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14841595c0 0x7f14840cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14840d0cc0 0x7f14841f54d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 --2- 172.21.15.46:0/1758002570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1484071f90 0x7f148414af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 >> 172.21.15.46:0/1758002570 conn(0x7f148406c350 msgr2=0x7f148414f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 shutdown_connections 2024-04-26T23:49:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:31.536+0000 7f148b000640 1 -- 172.21.15.46:0/1758002570 wait complete. 2024-04-26T23:49:36.557 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:49:36.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-26T23:49:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.774+0000 7f0f70a00640 1 Processor -- start 2024-04-26T23:49:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.774+0000 7f0f70a00640 1 -- start start 2024-04-26T23:49:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.775+0000 7f0f70a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f6c0d3830 0x7f0f6c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.775+0000 7f0f70a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6c0d20c0 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.775+0000 7f0f70a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6c0d3400 con 0x7f0f6c162a70 2024-04-26T23:49:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.775+0000 7f0f70a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c0d2370 con 0x7f0f6c0d3830 2024-04-26T23:49:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.776+0000 7f0f6be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f6c0d1cb0 0x7f0f6c0b29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55672/0 (socket says 172.21.15.46:55672) 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.776+0000 7f0f6be00640 1 -- 172.21.15.46:0/2286268734 learned_addr learned my addr 172.21.15.46:0/2286268734 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.776+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2286268734 >> v2:172.21.15.142:3300/0 conn(0x7f0f6c0d3830 0x7f0f6c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.776+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495721434 0 0) 0x7f0f6c0d20c0 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.776+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f4c003520 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1232287240 0 0) 0x7f0f4c003520 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 >> v2:172.21.15.142:3300/0 conn(0x7f0f6c0d3830 msgr2=0x7f0f6c0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 --2- 172.21.15.46:0/2286268734 >> v2:172.21.15.142:3300/0 conn(0x7f0f6c0d3830 0x7f0f6c0d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 >> v1:172.21.15.142:6789/0 conn(0x7f0f6c162a70 legacy=0x7f0f6c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c0d2650 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0f64001670 con 0x7f0f6c0d1cb0 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 >> v1:172.21.15.46:6789/0 conn(0x7f0f6c0d1cb0 legacy=0x7f0f6c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f4c0051c0 0x7f0f4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d2370 con 0x7f0f4c004800 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c00c070 con 0x7f0f4c0039b0 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c00c270 con 0x7f0f4c0051c0 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.777+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6aa00640 1 -- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f4c0051c0 msgr2=0x7f0f4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f4c0051c0 0x7f0f4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6aa00640 1 -- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f4c0039b0 msgr2=0x7f0f4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6aa00640 1 -- 172.21.15.46:0/2286268734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c003520 con 0x7f0f4c004800 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.778+0000 7f0f6be00640 1 -- 172.21.15.46:0/2286268734 reap_dead start 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009fc0 tx=0x7f0f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea46c995f1c4ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54009c10 con 0x7f0f4c004800 2024-04-26T23:49:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f54036070 con 0x7f0f4c004800 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f6a000640 1 -- 172.21.15.46:0/2286268734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54031670 con 0x7f0f4c004800 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f70a00640 1 -- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 msgr2=0x7f0f4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.779+0000 7f0f70a00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009fc0 tx=0x7f0f4c005830 comp rx=0 tx=0).stop 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.780+0000 7f0f70a00640 1 -- 172.21.15.46:0/2286268734 shutdown_connections 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.780+0000 7f0f70a00640 1 --2- 172.21.15.46:0/2286268734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.780+0000 7f0f70a00640 1 -- 172.21.15.46:0/2286268734 >> 172.21.15.46:0/2286268734 conn(0x7f0f6c0d4f20 msgr2=0x7f0f6c0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.780+0000 7f0f70a00640 1 -- 172.21.15.46:0/2286268734 shutdown_connections 2024-04-26T23:49:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.780+0000 7f0f70a00640 1 -- 172.21.15.46:0/2286268734 wait complete. 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.782+0000 7f0f70a00640 1 Processor -- start 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.782+0000 7f0f70a00640 1 -- start start 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f6c0cd690 0x7f0f6c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f6c0b9cb0 0x7f0f6c0ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c162fd0 con 0x7f0f4c004800 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c162e50 con 0x7f0f6c0cd690 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f70a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d2e20 con 0x7f0f6c0b9cb0 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50826/0 (socket says 172.21.15.46:50826) 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 -- 172.21.15.46:0/1814341904 learned_addr learned my addr 172.21.15.46:0/1814341904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 -- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f6c0b9cb0 msgr2=0x7f0f6c0ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f6c0b9cb0 0x7f0f6c0ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 -- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f6c0cd690 msgr2=0x7f0f6c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f6c0cd690 0x7f0f6c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.783+0000 7f0f6b400640 1 -- 172.21.15.46:0/1814341904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c1f8770 con 0x7f0f4c004800 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f6b400640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00c500 tx=0x7f0f5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f63400640 1 -- 172.21.15.46:0/1814341904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0052d0 con 0x7f0f4c004800 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f63400640 1 -- 172.21.15.46:0/1814341904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c005470 con 0x7f0f4c004800 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f63400640 1 -- 172.21.15.46:0/1814341904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c012870 con 0x7f0f4c004800 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c1f8a50 con 0x7f0f4c004800 2024-04-26T23:49:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.784+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1f9010 con 0x7f0f4c004800 2024-04-26T23:49:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.786+0000 7f0f63400640 1 -- 172.21.15.46:0/1814341904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f5c012a10 con 0x7f0f4c004800 2024-04-26T23:49:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.787+0000 7f0f63400640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 0x7f0f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.788+0000 7f0f63400640 1 -- 172.21.15.46:0/1814341904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f0f5c08b9b0 con 0x7f0f4c004800 2024-04-26T23:49:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.788+0000 7f0f74dff740 1 Processor -- start 2024-04-26T23:49:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.788+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 0x7f0f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.788+0000 7f0f74dff740 1 -- start start 2024-04-26T23:49:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207825d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562078125030 con 0x5620782153f0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5620781407f0 con 0x56207823f710 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.789+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5620780fb7e0 con 0x56207825cfb0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 0x7f0f4806edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0f54035000 tx=0x7f0f5402d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50838/0 (socket says 172.21.15.46:50838) 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 -- 172.21.15.46:0/2159501501 learned_addr learned my addr 172.21.15.46:0/2159501501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207825d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6b400640 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 -- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 msgr2=0x56207825d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207825d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 -- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 msgr2=0x56207823faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.790+0000 7f0f6be00640 1 -- 172.21.15.46:0/2159501501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562078107050 con 0x5620782153f0 2024-04-26T23:49:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f6be00640 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f0f640011e0 tx=0x7f0f64007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830570d6a852b371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f5b400640 1 -- 172.21.15.46:0/2159501501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f640074e0 con 0x5620782153f0 2024-04-26T23:49:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f5b400640 1 -- 172.21.15.46:0/2159501501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64007e30 con 0x5620782153f0 2024-04-26T23:49:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f5b400640 1 -- 172.21.15.46:0/2159501501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64004a00 con 0x5620782153f0 2024-04-26T23:49:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f74dff740 1 -- 172.21.15.46:0/2159501501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 msgr2=0x56207825b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.791+0000 7f0f74dff740 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f0f640011e0 tx=0x7f0f64007be0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 -- 172.21.15.46:0/2159501501 shutdown_connections 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620782153f0 0x56207825b670 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207825d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 --2- 172.21.15.46:0/2159501501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 -- 172.21.15.46:0/2159501501 >> 172.21.15.46:0/2159501501 conn(0x562078211d20 msgr2=0x56207825a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 -- 172.21.15.46:0/2159501501 shutdown_connections 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 -- 172.21.15.46:0/2159501501 wait complete. 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.792+0000 7f0f74dff740 1 -- start start 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.793+0000 7f0f74dff740 1 Processor -- start 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x5620782418d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56207817d200 con 0x56207823e210 2024-04-26T23:49:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562078177ee0 con 0x56207823f710 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562078184e70 con 0x56207825cfb0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f74dff740 10 client.0 _ll_register_callbacks cb 0x56207816e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50844/0 (socket says 172.21.15.46:50844) 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 -- 172.21.15.46:0/379990581 learned_addr learned my addr 172.21.15.46:0/379990581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6b400640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x5620782418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 -- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 msgr2=0x56207823dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 -- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 msgr2=0x5620782418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x5620782418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6be00640 1 -- 172.21.15.46:0/379990581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562077f547d0 con 0x56207823e210 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.794+0000 7f0f6b400640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x5620782418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f6be00640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c08d4c0 tx=0x7f0f6400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f851bc0e1adf56e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f53e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64004cf0 con 0x56207823e210 2024-04-26T23:49:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f0f6400c690 con 0x56207823e210 2024-04-26T23:49:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f64007e30 con 0x56207823e210 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0f64027050 con 0x56207823e210 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f74dff740 10 client.8934 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.795+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x562077e8a3f0 con 0x56207823e210 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64007e30 con 0x56207823e210 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f0f6402a020 con 0x56207823e210 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 10 client.8934 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 10 client.8934 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 20 client.8934 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207825daa0 0x56207825dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 4 client.8934 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:49:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.796+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620781867e0 con 0x56207825daa0 2024-04-26T23:49:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.797+0000 7f0f6b400640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207825daa0 0x56207825dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.798+0000 7f0f6b400640 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207825daa0 0x56207825dec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d39b99e7cae3022 server_cookie=721f092bdeabe806 in_seq=0 out_seq=0 2024-04-26T23:49:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.798+0000 7f0f53e00640 10 client.8934 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.799+0000 7f0f53e00640 1 -- 172.21.15.46:0/379990581 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620781867e0 con 0x56207825daa0 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.799+0000 7f0f53e00640 10 client.8934 handle_command_reply: tid=0 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.799+0000 7f0f74dff740 1 client.8934 shutdown 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 msgr2=0x56207822a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c08d4c0 tx=0x7f0f6400f000 comp rx=0 tx=0).stop 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 shutdown_connections 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207825daa0 0x56207825dec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a780 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 --2- 172.21.15.46:0/379990581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x5620782418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f53e00640 0 client.8934 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 >> 172.21.15.46:0/379990581 conn(0x5620782153f0 msgr2=0x56207820e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 shutdown_connections 2024-04-26T23:49:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 1 -- 172.21.15.46:0/379990581 wait complete. 2024-04-26T23:49:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.800+0000 7f0f74dff740 20 client.8934 trim_cache size 0 max 16384 2024-04-26T23:49:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.819+0000 7f0f74dff740 1 Processor -- start 2024-04-26T23:49:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 -- start start 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56207823ea90 con 0x56207823e210 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562078243ac0 con 0x56207823f710 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.820+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c2024e0 con 0x56207825cfb0 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50856/0 (socket says 172.21.15.46:50856) 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 -- 172.21.15.46:0/4029478676 learned_addr learned my addr 172.21.15.46:0/4029478676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 -- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 msgr2=0x56207823db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 -- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 msgr2=0x56207823d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 -- 172.21.15.46:0/4029478676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562077e4a0f0 con 0x56207823e210 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c059240 tx=0x7f0f5c089c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e007f8c2d6a8703 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f5aa00640 1 -- 172.21.15.46:0/4029478676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c057840 con 0x56207823e210 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f5aa00640 1 -- 172.21.15.46:0/4029478676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c055bb0 con 0x56207823e210 2024-04-26T23:49:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.821+0000 7f0f5aa00640 1 -- 172.21.15.46:0/4029478676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c061e90 con 0x56207823e210 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 -- 172.21.15.46:0/4029478676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 msgr2=0x56207822a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c059240 tx=0x7f0f5c089c10 comp rx=0 tx=0).stop 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 -- 172.21.15.46:0/4029478676 shutdown_connections 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x56207823db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823f710 0x56207823d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4029478676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823e210 0x56207822a270 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 -- 172.21.15.46:0/4029478676 >> 172.21.15.46:0/4029478676 conn(0x5620782153f0 msgr2=0x562078241720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 -- 172.21.15.46:0/4029478676 shutdown_connections 2024-04-26T23:49:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.822+0000 7f0f74dff740 1 -- 172.21.15.46:0/4029478676 wait complete. 2024-04-26T23:49:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.823+0000 7f0f74dff740 1 -- start start 2024-04-26T23:49:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.823+0000 7f0f74dff740 1 Processor -- start 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823e210 0x562078241270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x562078272d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c2008d0 con 0x56207823f710 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c201520 con 0x56207823e210 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c200f20 con 0x56207825cfb0 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f74dff740 10 client.0 _ll_register_callbacks cb 0x7f0f64001d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50862/0 (socket says 172.21.15.46:50862) 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.824+0000 7f0f6aa00640 1 -- 172.21.15.46:0/4273878869 learned_addr learned my addr 172.21.15.46:0/4273878869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6be00640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x562078272d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 -- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 msgr2=0x562078272d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823e210 0x562078241270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x562078272d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 -- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823e210 msgr2=0x562078241270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823e210 0x562078241270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 -- 172.21.15.46:0/4273878869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562078140a30 con 0x56207823f710 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f6aa00640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402aed0 tx=0x7f0f54009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63016a1657e4951 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.825+0000 7f0f53e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54031dd0 con 0x56207823f710 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f0f54054450 con 0x56207823f710 2024-04-26T23:49:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f5c055bb0 con 0x56207823f710 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0f54054650 con 0x56207823f710 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 10 client.8940 fetch_fsmap learned FSMap version 100 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x562078242180 con 0x56207823f710 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c055bb0 con 0x56207823f710 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f0f54030070 con 0x56207823f710 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 10 client.8940 fetch_fsmap finished waiting for FSMap version 100 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 10 client.8940 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 20 client.8940 populate_metadata read hostname 'smithi046' 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207820e960 0x562078273240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 4 client.8940 mds_command: new command op to 8724 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:49:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.826+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 --> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562078180650 con 0x56207820e960 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.827+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207820e960 0x562078273240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.828+0000 7f0f6b400640 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207820e960 0x562078273240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388cc531619a121d server_cookie=acf28fc31db115f5 in_seq=0 out_seq=0 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.828+0000 7f0f53e00640 10 client.8940 ms_handle_connect on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f53e00640 1 -- 172.21.15.46:0/4273878869 <== mds.0 v2:172.21.15.142:6834/475647056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562078180650 con 0x56207820e960 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f53e00640 10 client.8940 handle_command_reply: tid=0 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 client.8940 shutdown 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 msgr2=0x5620782417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402aed0 tx=0x7f0f54009750 comp rx=0 tx=0).stop 2024-04-26T23:49:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 shutdown_connections 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:6834/475647056,v1:172.21.15.142:6836/475647056] conn(0x56207820e960 0x562078273240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56207825cfb0 0x562078272d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56207823f710 0x5620782417b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 --2- 172.21.15.46:0/4273878869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207823e210 0x562078241270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f53e00640 0 client.8940 ms_handle_reset on v2:172.21.15.142:6834/475647056 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 >> 172.21.15.46:0/4273878869 conn(0x5620782153f0 msgr2=0x5620782424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 shutdown_connections 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 1 -- 172.21.15.46:0/4273878869 wait complete. 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.829+0000 7f0f74dff740 20 client.8940 trim_cache size 0 max 16384 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:49:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:49:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:49:36.836 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:49:36.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 msgr2=0x7f0f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 0x7f0f4806edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0f54035000 tx=0x7f0f5402d000 comp rx=0 tx=0).stop 2024-04-26T23:49:36.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 msgr2=0x7f0f6c0c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00c500 tx=0x7f0f5c00c9c0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 shutdown_connections 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f4806c8f0 0x7f0f4806edb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f6c0b9cb0 0x7f0f6c0ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f6c0cd690 0x7f0f6c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 --2- 172.21.15.46:0/1814341904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4c004800 0x7f0f6c0c7c90 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 >> 172.21.15.46:0/1814341904 conn(0x7f0f6c0d4f20 msgr2=0x7f0f6c06e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 shutdown_connections 2024-04-26T23:49:36.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:36.830+0000 7f0f70a00640 1 -- 172.21.15.46:0/1814341904 wait complete. 2024-04-26T23:49:36.851 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:49:36.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.051+0000 7fe43ee00640 1 Processor -- start 2024-04-26T23:49:37.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.051+0000 7fe43ee00640 1 -- start start 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43ee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4381466b0 0x7fe43806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe438160910 0x7fe438160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe438073ad0 con 0x7fe4380720e0 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43ee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe438146a90 con 0x7fe4381466b0 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe438073d70 con 0x7fe438160910 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4381466b0 0x7fe43806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe4380720e0 0x7fe4380cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55680/0 (socket says 172.21.15.46:55680) 2024-04-26T23:49:37.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.053+0000 7fe43d000640 1 -- 172.21.15.46:0/4286078555 learned_addr learned my addr 172.21.15.46:0/4286078555 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:37.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.054+0000 7fe43e400640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe438160910 0x7fe438160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.054+0000 7fe43da00640 1 -- 172.21.15.46:0/4286078555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe438160910 msgr2=0x7fe438160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:37.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.054+0000 7fe43da00640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe438160910 0x7fe438160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.054+0000 7fe43da00640 1 -- 172.21.15.46:0/4286078555 >> v1:172.21.15.46:6789/0 conn(0x7fe4380720e0 legacy=0x7fe4380cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.054+0000 7fe43da00640 1 -- 172.21.15.46:0/4286078555 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe438074050 con 0x7fe4381466b0 2024-04-26T23:49:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.055+0000 7fe43da00640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4381466b0 0x7fe43806de80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097d0 tx=0x7fe42c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb0f156ecc6aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.055+0000 7fe437e00640 1 -- 172.21.15.46:0/4286078555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037070 con 0x7fe4381466b0 2024-04-26T23:49:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.055+0000 7fe437e00640 1 -- 172.21.15.46:0/4286078555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c032440 con 0x7fe4381466b0 2024-04-26T23:49:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.055+0000 7fe437e00640 1 -- 172.21.15.46:0/4286078555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c032810 con 0x7fe4381466b0 2024-04-26T23:49:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 -- 172.21.15.46:0/4286078555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4381466b0 msgr2=0x7fe43806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4381466b0 0x7fe43806de80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097d0 tx=0x7fe42c02a1c0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 -- 172.21.15.46:0/4286078555 shutdown_connections 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe438160910 0x7fe438160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 --2- 172.21.15.46:0/4286078555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4381466b0 0x7fe43806de80 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 -- 172.21.15.46:0/4286078555 >> 172.21.15.46:0/4286078555 conn(0x7fe4380b26d0 msgr2=0x7fe4380b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 -- 172.21.15.46:0/4286078555 shutdown_connections 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.056+0000 7fe43ee00640 1 -- 172.21.15.46:0/4286078555 wait complete. 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.058+0000 7fe43ee00640 1 Processor -- start 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.058+0000 7fe43ee00640 1 -- start start 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 0x7fe438142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe438160910 0x7fe438142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe43806e3c0 con 0x7fe4380720e0 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4380cb470 con 0x7fe438160910 2024-04-26T23:49:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4380cb990 con 0x7fe4381466b0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50872/0 (socket says 172.21.15.46:50872) 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.059+0000 7fe43da00640 1 -- 172.21.15.46:0/986027565 learned_addr learned my addr 172.21.15.46:0/986027565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43d000640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 0x7fe438142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43e400640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe438160910 0x7fe438142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 -- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 msgr2=0x7fe438142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 0x7fe438142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 -- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe438160910 msgr2=0x7fe438142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe438160910 0x7fe438142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 -- 172.21.15.46:0/986027565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380c88d0 con 0x7fe4380720e0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43d000640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 0x7fe438142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe43da00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098f0 tx=0x7fe42c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037040 con 0x7fe4380720e0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c030770 con 0x7fe4380720e0 2024-04-26T23:49:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.060+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c030b40 con 0x7fe4380720e0 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.061+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4380c8bb0 con 0x7fe4380720e0 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.061+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4380c9170 con 0x7fe4380720e0 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.062+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4380693b0 con 0x7fe4380720e0 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.063+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe42c044020 con 0x7fe4380720e0 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.064+0000 7fe435600640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.064+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fe42c0ac2f0 con 0x7fe4380720e0 2024-04-26T23:49:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.064+0000 7fe43d000640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.065+0000 7fe43d000640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe4381420a0 tx=0x7fe424009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.069+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe42c032e20 con 0x7fe4380720e0 2024-04-26T23:49:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.312+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4380c7180 con 0x7fe4380720e0 2024-04-26T23:49:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.314+0000 7fe435600640 1 -- 172.21.15.46:0/986027565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7fe42c07a470 con 0x7fe4380720e0 2024-04-26T23:49:37.318 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:37.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:49:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.317+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 msgr2=0x7fe41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.317+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe4381420a0 tx=0x7fe424009040 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 msgr2=0x7fe43815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098f0 tx=0x7fe42c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 shutdown_connections 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe438160910 0x7fe438142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4381466b0 0x7fe438142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 --2- 172.21.15.46:0/986027565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4380720e0 0x7fe43815f4e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 >> 172.21.15.46:0/986027565 conn(0x7fe4380b26d0 msgr2=0x7fe43806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 shutdown_connections 2024-04-26T23:49:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.318+0000 7fe43ee00640 1 -- 172.21.15.46:0/986027565 wait complete. 2024-04-26T23:49:37.341 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-26T23:49:37.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:49:37.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.548+0000 7fa951e00640 1 Processor -- start 2024-04-26T23:49:37.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.549+0000 7fa951e00640 1 -- start start 2024-04-26T23:49:37.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa951e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa94c0cf450 0x7fa94c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa951e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa94c0cff40 0x7fa94c166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa951e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c06f1d0 con 0x7fa94c072190 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa951e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c06f470 con 0x7fa94c0cf450 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa951e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c072930 con 0x7fa94c0cff40 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa950a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa94c0cf450 0x7fa94c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa950a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa94c0cf450 0x7fa94c073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50886/0 (socket says 172.21.15.46:50886) 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.550+0000 7fa950a00640 1 -- 172.21.15.46:0/2246647604 learned_addr learned my addr 172.21.15.46:0/2246647604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.551+0000 7fa950a00640 1 -- 172.21.15.46:0/2246647604 >> v2:172.21.15.142:3300/0 conn(0x7fa94c0cff40 msgr2=0x7fa94c166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.551+0000 7fa950a00640 1 --2- 172.21.15.46:0/2246647604 >> v2:172.21.15.142:3300/0 conn(0x7fa94c0cff40 0x7fa94c166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.551+0000 7fa950a00640 1 -- 172.21.15.46:0/2246647604 >> v1:172.21.15.142:6789/0 conn(0x7fa94c072190 legacy=0x7fa94c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.551+0000 7fa950a00640 1 -- 172.21.15.46:0/2246647604 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c072c10 con 0x7fa94c0cf450 2024-04-26T23:49:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.552+0000 7fa950a00640 1 --2- 172.21.15.46:0/2246647604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c073db0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa94c06f3a0 tx=0x7fa94002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643f381153fae5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.552+0000 7fa94b400640 1 -- 172.21.15.46:0/2246647604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940037070 con 0x7fa94c0cf450 2024-04-26T23:49:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.552+0000 7fa94b400640 1 -- 172.21.15.46:0/2246647604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940032440 con 0x7fa94c0cf450 2024-04-26T23:49:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.552+0000 7fa94b400640 1 -- 172.21.15.46:0/2246647604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940032810 con 0x7fa94c0cf450 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 -- 172.21.15.46:0/2246647604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 msgr2=0x7fa94c073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 --2- 172.21.15.46:0/2246647604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c073db0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa94c06f3a0 tx=0x7fa94002a210 comp rx=0 tx=0).stop 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 -- 172.21.15.46:0/2246647604 shutdown_connections 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 --2- 172.21.15.46:0/2246647604 >> v2:172.21.15.142:3300/0 conn(0x7fa94c0cff40 0x7fa94c166b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 --2- 172.21.15.46:0/2246647604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c073db0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 -- 172.21.15.46:0/2246647604 >> 172.21.15.46:0/2246647604 conn(0x7fa94c0b2820 msgr2=0x7fa94c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 -- 172.21.15.46:0/2246647604 shutdown_connections 2024-04-26T23:49:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.553+0000 7fa951e00640 1 -- 172.21.15.46:0/2246647604 wait complete. 2024-04-26T23:49:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.555+0000 7fa951e00640 1 Processor -- start 2024-04-26T23:49:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.555+0000 7fa951e00640 1 -- start start 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94c072190 0x7fa94c1de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa94c0cff40 0x7fa94c144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0734b0 con 0x7fa94c0cf450 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0731d0 con 0x7fa94c072190 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa951e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c072620 con 0x7fa94c0cff40 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa94be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa94be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50902/0 (socket says 172.21.15.46:50902) 2024-04-26T23:49:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.556+0000 7fa94be00640 1 -- 172.21.15.46:0/2411766299 learned_addr learned my addr 172.21.15.46:0/2411766299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:37.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa950a00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94c072190 0x7fa94c1de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa951400640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa94c0cff40 0x7fa94c144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 -- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa94c0cff40 msgr2=0x7fa94c144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa94c0cff40 0x7fa94c144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 -- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94c072190 msgr2=0x7fa94c1de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94c072190 0x7fa94c1de8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 -- 172.21.15.46:0/2411766299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0aea10 con 0x7fa94c0cf450 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa94be00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b410 tx=0x7fa93800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938015020 con 0x7fa94c0cf450 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938011830 con 0x7fa94c0cf450 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938011c20 con 0x7fa94c0cf450 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.557+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c0aecf0 con 0x7fa94c0cf450 2024-04-26T23:49:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.558+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c0af2b0 con 0x7fa94c0cf450 2024-04-26T23:49:37.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.559+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c0693b0 con 0x7fa94c0cf450 2024-04-26T23:49:37.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.560+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa938018020 con 0x7fa94c0cf450 2024-04-26T23:49:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.561+0000 7fa948c00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.561+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fa93808bca0 con 0x7fa94c0cf450 2024-04-26T23:49:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.562+0000 7fa950a00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:37.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.563+0000 7fa950a00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 0x7fa92c06edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa940034000 tx=0x7fa94002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.566+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa93801f020 con 0x7fa94c0cf450 2024-04-26T23:49:37.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.844+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa94c144640 con 0x7fa94c0cf450 2024-04-26T23:49:37.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.846+0000 7fa948c00640 1 -- 172.21.15.46:0/2411766299 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1028 (secure 0 0 0) 0x7fa9380172c0 con 0x7fa94c0cf450 2024-04-26T23:49:37.850 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:37.850 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076935910,"bytes_used":2559770624,"bytes_avail":770534342656,"bytes_total":773094113280,"write_bytes_sec":23933463,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":100,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8724}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-26T23:48:32.144178+0000","services":{}},"progress_events":{}} 2024-04-26T23:49:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 msgr2=0x7fa92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 0x7fa92c06edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa940034000 tx=0x7fa94002e000 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 msgr2=0x7fa94c0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b410 tx=0x7fa93800b8d0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 shutdown_connections 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa94c0cff40 0x7fa94c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c0cf450 0x7fa94c0afab0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 --2- 172.21.15.46:0/2411766299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94c072190 0x7fa94c1de8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.848+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 >> 172.21.15.46:0/2411766299 conn(0x7fa94c0b2820 msgr2=0x7fa94c0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.849+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 shutdown_connections 2024-04-26T23:49:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:37.849+0000 7fa951e00640 1 -- 172.21.15.46:0/2411766299 wait complete. 2024-04-26T23:49:37.872 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:49:37.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:49:37.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:49:38.077 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:49:38.078 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:49:38.078 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:49:38.078 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:38.139 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:49:38.140 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:38.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.416+0000 7f41f7e00640 1 Processor -- start 2024-04-26T23:49:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.416+0000 7f41f7e00640 1 -- start start 2024-04-26T23:49:38.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.417+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41f80b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.417+0000 7f41f7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f80d21d0 con 0x7f41f80d1a40 2024-04-26T23:49:38.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f80d1ef0 con 0x7f41f80d0f50 2024-04-26T23:49:38.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80b6bc0 con 0x7f41f806d670 2024-04-26T23:49:38.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f41f80d1a40 0x7f41f80b76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55692/0 (socket says 172.21.15.46:55692) 2024-04-26T23:49:38.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f6000640 1 -- 172.21.15.46:0/3833230057 learned_addr learned my addr 172.21.15.46:0/3833230057 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f7400640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41f80b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814735305 0 0) 0x7f41f80d21d0 con 0x7f41f80d1a40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.418+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41dc003520 con 0x7f41f80d1a40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3328739929 0 0) 0x7f41dc003520 con 0x7f41f80d1a40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 msgr2=0x7f41f80b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41f80b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 >> v1:172.21.15.142:6789/0 conn(0x7f41f80d0f50 legacy=0x7f41f80d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80b6ea0 con 0x7f41f80d1a40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f41e0001670 con 0x7f41f80d1a40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 >> v1:172.21.15.46:6789/0 conn(0x7f41f80d1a40 legacy=0x7f41f80b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41dc0039e0 0x7f41dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41dc004d40 0x7f41dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80b6bc0 con 0x7f41dc0039e0 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00bb50 con 0x7f41dc004d40 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc00bd50 con 0x7f41f806d670 2024-04-26T23:49:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.419+0000 7f41f6a00640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41dc0039e0 0x7f41dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6000640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6a00640 1 -- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 msgr2=0x7f41dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6a00640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f806d670 0x7f41dc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6a00640 1 -- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41dc004d40 msgr2=0x7f41dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6a00640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41dc004d40 0x7f41dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.420+0000 7f41f6a00640 1 -- 172.21.15.46:0/3833230057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc003520 con 0x7f41dc0039e0 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.421+0000 7f41f6a00640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41dc0039e0 0x7f41dc003e00 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f41e8001750 tx=0x7f41dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6f4b4b3fb6996 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.421+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8001370 con 0x7f41dc0039e0 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.421+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e80052d0 con 0x7f41dc0039e0 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.421+0000 7f41f5600640 1 -- 172.21.15.46:0/3833230057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e802abe0 con 0x7f41dc0039e0 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 -- 172.21.15.46:0/3833230057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41dc0039e0 msgr2=0x7f41dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 --2- 172.21.15.46:0/3833230057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41dc0039e0 0x7f41dc003e00 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f41e8001750 tx=0x7f41dc005830 comp rx=0 tx=0).stop 2024-04-26T23:49:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7400640 1 -- 172.21.15.46:0/3833230057 reap_dead start 2024-04-26T23:49:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 -- 172.21.15.46:0/3833230057 shutdown_connections 2024-04-26T23:49:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 -- 172.21.15.46:0/3833230057 >> 172.21.15.46:0/3833230057 conn(0x7f41f80d2cc0 msgr2=0x7f41f80d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 -- 172.21.15.46:0/3833230057 shutdown_connections 2024-04-26T23:49:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.422+0000 7f41f7e00640 1 -- 172.21.15.46:0/3833230057 wait complete. 2024-04-26T23:49:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.424+0000 7f41f7e00640 1 Processor -- start 2024-04-26T23:49:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.424+0000 7f41f7e00640 1 -- start start 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f80ceb10 0x7f41f80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80c9d90 0x7f41f80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41f815f7c0 con 0x7f41f80ca750 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41f815f640 con 0x7f41f80c9d90 2024-04-26T23:49:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.425+0000 7f41f7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80b3ff0 con 0x7f41f80ceb10 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50914/0 (socket says 172.21.15.46:50914) 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 -- 172.21.15.46:0/525595116 learned_addr learned my addr 172.21.15.46:0/525595116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f6a00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f80ceb10 0x7f41f80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f6000640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80c9d90 0x7f41f80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 -- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f80ceb10 msgr2=0x7f41f80ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f80ceb10 0x7f41f80ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 -- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80c9d90 msgr2=0x7f41f80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80c9d90 0x7f41f80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.426+0000 7f41f7400640 1 -- 172.21.15.46:0/525595116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80cf640 con 0x7f41f80ca750 2024-04-26T23:49:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41f7400640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f41f8162e10 tx=0x7f41f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0018020 con 0x7f41f80ca750 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f000bce0 con 0x7f41f80ca750 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f80cf920 con 0x7f41f80ca750 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0014640 con 0x7f41f80ca750 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.427+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f815fa70 con 0x7f41f80ca750 2024-04-26T23:49:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.429+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8069ac0 con 0x7f41f80ca750 2024-04-26T23:49:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.429+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41f0005d10 con 0x7f41f80ca750 2024-04-26T23:49:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.430+0000 7f41eea00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.430+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f41f008c3e0 con 0x7f41f80ca750 2024-04-26T23:49:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.431+0000 7f41f6a00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.432+0000 7f41f6a00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f41e8032000 tx=0x7f41e802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.435+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41f0022030 con 0x7f41f80ca750 2024-04-26T23:49:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.639+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41f806dc50 con 0x7f41f80ca750 2024-04-26T23:49:38.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.642+0000 7f41eea00640 1 -- 172.21.15.46:0/525595116 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16916 (secure 0 0 0) 0x7f41f005a670 con 0x7f41f80ca750 2024-04-26T23:49:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:38.646 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:46.084255+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:38.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 msgr2=0x7f41d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f41e8032000 tx=0x7f41e802e000 comp rx=0 tx=0).stop 2024-04-26T23:49:38.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 msgr2=0x7f41f80cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f41f8162e10 tx=0x7f41f000b480 comp rx=0 tx=0).stop 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 shutdown_connections 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f80ca750 0x7f41f80cabe0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80c9d90 0x7f41f80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.644+0000 7f41f7e00640 1 --2- 172.21.15.46:0/525595116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41f80ceb10 0x7f41f80ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.645+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 >> 172.21.15.46:0/525595116 conn(0x7f41f80d2cc0 msgr2=0x7f41f8070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.645+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 shutdown_connections 2024-04-26T23:49:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.645+0000 7f41f7e00640 1 -- 172.21.15.46:0/525595116 wait complete. 2024-04-26T23:49:38.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:48:42.408660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:48:43.535112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:49:38.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:49:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.869+0000 7f647ae00640 1 Processor -- start 2024-04-26T23:49:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.870+0000 7f647ae00640 1 -- start start 2024-04-26T23:49:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.871+0000 7f647ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f647415d730 0x7f64740d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.871+0000 7f647ae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f647415e220 0x7f647406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.871+0000 7f647ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647406eb90 con 0x7f64740b53b0 2024-04-26T23:49:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f647ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f64740b4310 con 0x7f647415d730 2024-04-26T23:49:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f647ae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f64740b45f0 con 0x7f647415e220 2024-04-26T23:49:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f647415d730 0x7f64740d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f647415d730 0x7f64740d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50920/0 (socket says 172.21.15.46:50920) 2024-04-26T23:49:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 -- 172.21.15.46:0/846094228 learned_addr learned my addr 172.21.15.46:0/846094228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6478e00640 1 --2- 172.21.15.46:0/846094228 >> v2:172.21.15.142:3300/0 conn(0x7f647415e220 0x7f647406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 -- 172.21.15.46:0/846094228 >> v2:172.21.15.142:3300/0 conn(0x7f647415e220 msgr2=0x7f647406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 --2- 172.21.15.46:0/846094228 >> v2:172.21.15.142:3300/0 conn(0x7f647415e220 0x7f647406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 -- 172.21.15.46:0/846094228 >> v1:172.21.15.142:6789/0 conn(0x7f64740b53b0 legacy=0x7f647415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.872+0000 7f6473e00640 1 -- 172.21.15.46:0/846094228 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740b48d0 con 0x7f647415d730 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.873+0000 7f6473e00640 1 --2- 172.21.15.46:0/846094228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415d730 0x7f64740d00d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6474074180 tx=0x7f646402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaba187b9456091b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.874+0000 7f6473400640 1 -- 172.21.15.46:0/846094228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464037070 con 0x7f647415d730 2024-04-26T23:49:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.874+0000 7f6473400640 1 -- 172.21.15.46:0/846094228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464032440 con 0x7f647415d730 2024-04-26T23:49:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.874+0000 7f6473400640 1 -- 172.21.15.46:0/846094228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464032810 con 0x7f647415d730 2024-04-26T23:49:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.874+0000 7f647ae00640 1 -- 172.21.15.46:0/846094228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415d730 msgr2=0x7f64740d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.874+0000 7f647ae00640 1 --2- 172.21.15.46:0/846094228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415d730 0x7f64740d00d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6474074180 tx=0x7f646402a230 comp rx=0 tx=0).stop 2024-04-26T23:49:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 -- 172.21.15.46:0/846094228 shutdown_connections 2024-04-26T23:49:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 --2- 172.21.15.46:0/846094228 >> v2:172.21.15.142:3300/0 conn(0x7f647415e220 0x7f647406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 --2- 172.21.15.46:0/846094228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415d730 0x7f64740d00d0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 -- 172.21.15.46:0/846094228 >> 172.21.15.46:0/846094228 conn(0x7f647406c980 msgr2=0x7f64740b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 -- 172.21.15.46:0/846094228 shutdown_connections 2024-04-26T23:49:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.875+0000 7f647ae00640 1 -- 172.21.15.46:0/846094228 wait complete. 2024-04-26T23:49:38.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.876+0000 7f647ae00640 1 Processor -- start 2024-04-26T23:49:38.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.877+0000 7f647ae00640 1 -- start start 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.877+0000 7f647ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64740b53b0 0x7f647415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f647ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647415d730 0x7f64741f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f647ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f647ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b3ec0 con 0x7f647415e220 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f647ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b4e90 con 0x7f64740b53b0 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f647ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f64740b4040 con 0x7f647415d730 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50934/0 (socket says 172.21.15.46:50934) 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 -- 172.21.15.46:0/2671657048 learned_addr learned my addr 172.21.15.46:0/2671657048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6473e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64740b53b0 0x7f647415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f646be00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647415d730 0x7f64741f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 -- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647415d730 msgr2=0x7f64741f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647415d730 0x7f64741f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 -- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64740b53b0 msgr2=0x7f647415a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64740b53b0 0x7f647415a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.878+0000 7f6478e00640 1 -- 172.21.15.46:0/2671657048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6474140020 con 0x7f647415e220 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f6478e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f646c007e40 tx=0x7f646c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c014620 con 0x7f647415e220 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c019070 con 0x7f647415e220 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c014a20 con 0x7f647415e220 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6474140300 con 0x7f647415e220 2024-04-26T23:49:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.879+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64741408c0 con 0x7f647415e220 2024-04-26T23:49:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.880+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6474069530 con 0x7f647415e220 2024-04-26T23:49:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.881+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f646c014bc0 con 0x7f647415e220 2024-04-26T23:49:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.882+0000 7f6470c00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 0x7f645006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.882+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f646c08c580 con 0x7f647415e220 2024-04-26T23:49:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.883+0000 7f6473e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 0x7f645006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.884+0000 7f6473e00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 0x7f645006ee00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6464034000 tx=0x7f646402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:38.887+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f646c05a890 con 0x7f647415e220 2024-04-26T23:49:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.093+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64740b1c80 con 0x7f647415e220 2024-04-26T23:49:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.094+0000 7f6470c00640 1 -- 172.21.15.46:0/2671657048 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f646c05a190 con 0x7f647415e220 2024-04-26T23:49:39.098 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 msgr2=0x7f645006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 0x7f645006ee00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6464034000 tx=0x7f646402e000 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 msgr2=0x7f64741f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f646c007e40 tx=0x7f646c00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 shutdown_connections 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f645006c940 0x7f645006ee00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647415e220 0x7f64741f6930 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f647415d730 0x7f64741f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 --2- 172.21.15.46:0/2671657048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64740b53b0 0x7f647415a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 >> 172.21.15.46:0/2671657048 conn(0x7f647406c980 msgr2=0x7f64740b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 shutdown_connections 2024-04-26T23:49:39.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.098+0000 7f647ae00640 1 -- 172.21.15.46:0/2671657048 wait complete. 2024-04-26T23:49:39.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:49:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.335+0000 7f840c000640 1 Processor -- start 2024-04-26T23:49:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.335+0000 7f840c000640 1 -- start start 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.337+0000 7f840c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 0x7f84040b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.337+0000 7f840c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84040b4b30 con 0x7f84040b4320 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.337+0000 7f840c000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84040b4e10 con 0x7f84040d04c0 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.337+0000 7f840c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b50f0 con 0x7f840406f4b0 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.337+0000 7f840a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84040b4320 0x7f84040ce470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55708/0 (socket says 172.21.15.46:55708) 2024-04-26T23:49:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f840a000640 1 -- 172.21.15.46:0/1345794739 learned_addr learned my addr 172.21.15.46:0/1345794739 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:39.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8408c00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 0x7f84040b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418564634 0 0) 0x7f84040b4b30 con 0x7f84040b4320 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83ec005730 con 0x7f84040b4320 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 470392859 0 0) 0x7f83ec005730 con 0x7f84040b4320 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 msgr2=0x7f84040b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 0x7f84040b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 >> v1:172.21.15.142:6789/0 conn(0x7f84040d04c0 legacy=0x7f840406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84040b53d0 con 0x7f84040b4320 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.338+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f83fc0019f0 con 0x7f84040b4320 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 >> v1:172.21.15.46:6789/0 conn(0x7f84040b4320 legacy=0x7f84040ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ec005bf0 0x7f83ec006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec006a10 0x7f83ec006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 0x7f83ec007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b50f0 con 0x7f83ec005bf0 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00dd60 con 0x7f83ec006a10 2024-04-26T23:49:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec00df60 con 0x7f840406f4b0 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ec005bf0 0x7f83ec006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8408c00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec006a10 0x7f83ec006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 -- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 msgr2=0x7f83ec007480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840406f4b0 0x7f83ec007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 -- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec006a10 msgr2=0x7f83ec006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec006a10 0x7f83ec006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.339+0000 7f8409600640 1 -- 172.21.15.46:0/1345794739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec005730 con 0x7f83ec005bf0 2024-04-26T23:49:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.340+0000 7f8408c00640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec006a10 0x7f83ec006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f8409600640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ec005bf0 0x7f83ec006010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001750 tx=0x7f83ec007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f90f21ccbcb356a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8001370 con 0x7f83ec005bf0 2024-04-26T23:49:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f80052d0 con 0x7f83ec005bf0 2024-04-26T23:49:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f8403e00640 1 -- 172.21.15.46:0/1345794739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f802abe0 con 0x7f83ec005bf0 2024-04-26T23:49:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f840c000640 1 -- 172.21.15.46:0/1345794739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ec005bf0 msgr2=0x7f83ec006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.341+0000 7f840c000640 1 --2- 172.21.15.46:0/1345794739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ec005bf0 0x7f83ec006010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001750 tx=0x7f83ec007a40 comp rx=0 tx=0).stop 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.342+0000 7f840a000640 1 -- 172.21.15.46:0/1345794739 reap_dead start 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.342+0000 7f840c000640 1 -- 172.21.15.46:0/1345794739 shutdown_connections 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.342+0000 7f840c000640 1 -- 172.21.15.46:0/1345794739 >> 172.21.15.46:0/1345794739 conn(0x7f840406c980 msgr2=0x7f84040b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.342+0000 7f840c000640 1 -- 172.21.15.46:0/1345794739 shutdown_connections 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.342+0000 7f840c000640 1 -- 172.21.15.46:0/1345794739 wait complete. 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.343+0000 7f840c000640 1 Processor -- start 2024-04-26T23:49:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.344+0000 7f840c000640 1 -- start start 2024-04-26T23:49:39.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec005bf0 0x7f840413e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840413d6d0 0x7f840415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b48d0 con 0x7f84041f15b0 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b4750 con 0x7f83ec005bf0 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b3de0 con 0x7f840413d6d0 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50960/0 (socket says 172.21.15.46:50960) 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 -- 172.21.15.46:0/3323459572 learned_addr learned my addr 172.21.15.46:0/3323459572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 -- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840413d6d0 msgr2=0x7f840415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840413d6d0 0x7f840415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 -- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec005bf0 msgr2=0x7f840413e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f8409600640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec005bf0 0x7f840413e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec005bf0 0x7f840413e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.345+0000 7f840a000640 1 -- 172.21.15.46:0/3323459572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840413e5b0 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f840a000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f840406faf0 tx=0x7f83fc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00a610 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc00a7d0 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00abc0 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041f0b10 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.346+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840413e830 con 0x7f84041f15b0 2024-04-26T23:49:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.347+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404069530 con 0x7f84041f15b0 2024-04-26T23:49:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.348+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83fc007070 con 0x7f84041f15b0 2024-04-26T23:49:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.349+0000 7f8401600640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.349+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f83fc08b610 con 0x7f84041f15b0 2024-04-26T23:49:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.350+0000 7f8409600640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.351+0000 7f8409600640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 0x7f83e406ede0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001750 tx=0x7f83f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.354+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f840413e5b0 con 0x7f84041f15b0 2024-04-26T23:49:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.557+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f840406dfd0 con 0x7f84041f15b0 2024-04-26T23:49:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.558+0000 7f8401600640 1 -- 172.21.15.46:0/3323459572 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7f83fc059810 con 0x7f84041f15b0 2024-04-26T23:49:39.562 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:49:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 msgr2=0x7f83e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 0x7f83e406ede0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001750 tx=0x7f83f802e000 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 msgr2=0x7f84040ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f840406faf0 tx=0x7f83fc00ccd0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 shutdown_connections 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84041f15b0 0x7f84040ce6f0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f840413d6d0 0x7f840415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 --2- 172.21.15.46:0/3323459572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83ec005bf0 0x7f840413e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.560+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 >> 172.21.15.46:0/3323459572 conn(0x7f840406c980 msgr2=0x7f84040cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.561+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 shutdown_connections 2024-04-26T23:49:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.561+0000 7f840c000640 1 -- 172.21.15.46:0/3323459572 wait complete. 2024-04-26T23:49:39.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:49:39.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.802+0000 7f652f200640 1 Processor -- start 2024-04-26T23:49:39.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.802+0000 7f652f200640 1 -- start start 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653015df90 0x7f65300d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015e420 con 0x7f6530146740 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6530146b20 con 0x7f65300ce670 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6530161a00 con 0x7f653015df90 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6530146740 0x7f65301612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55718/0 (socket says 172.21.15.46:55718) 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f652de00640 1 -- 172.21.15.46:0/439775010 learned_addr learned my addr 172.21.15.46:0/439775010 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.804+0000 7f6527e00640 1 --2- 172.21.15.46:0/439775010 >> v2:172.21.15.46:3300/0 conn(0x7f653015df90 0x7f65300d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.805+0000 7f6527e00640 1 -- 172.21.15.46:0/439775010 >> v1:172.21.15.142:6789/0 conn(0x7f65300ce670 legacy=0x7f65300cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.805+0000 7f652d400640 1 -- 172.21.15.46:0/439775010 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295189959 0 0) 0x7f6520001110 con 0x7f6530146740 2024-04-26T23:49:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.805+0000 7f6527e00640 1 -- 172.21.15.46:0/439775010 >> v1:172.21.15.46:6789/0 conn(0x7f6530146740 legacy=0x7f65301612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.805+0000 7f6527e00640 1 -- 172.21.15.46:0/439775010 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65300ccf30 con 0x7f653015df90 2024-04-26T23:49:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.806+0000 7f6527e00640 1 --2- 172.21.15.46:0/439775010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300d0330 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f65180097a0 tx=0x7f651802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6fa94919eaaf5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.806+0000 7f652d400640 1 -- 172.21.15.46:0/439775010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518037070 con 0x7f653015df90 2024-04-26T23:49:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.806+0000 7f652d400640 1 -- 172.21.15.46:0/439775010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518032440 con 0x7f653015df90 2024-04-26T23:49:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.806+0000 7f652d400640 1 -- 172.21.15.46:0/439775010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518032810 con 0x7f653015df90 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 -- 172.21.15.46:0/439775010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 msgr2=0x7f65300d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 --2- 172.21.15.46:0/439775010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300d0330 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f65180097a0 tx=0x7f651802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 -- 172.21.15.46:0/439775010 shutdown_connections 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 --2- 172.21.15.46:0/439775010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300d0330 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 -- 172.21.15.46:0/439775010 >> 172.21.15.46:0/439775010 conn(0x7f65300b26d0 msgr2=0x7f65300b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 -- 172.21.15.46:0/439775010 shutdown_connections 2024-04-26T23:49:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.807+0000 7f652f200640 1 -- 172.21.15.46:0/439775010 wait complete. 2024-04-26T23:49:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.809+0000 7f652f200640 1 Processor -- start 2024-04-26T23:49:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.809+0000 7f652f200640 1 -- start start 2024-04-26T23:49:39.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65300ce670 0x7f653013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6530146740 0x7f6530133cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65300ccb70 con 0x7f653015df90 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6530160dd0 con 0x7f6530146740 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65300cccf0 con 0x7f65300ce670 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50980/0 (socket says 172.21.15.46:50980) 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.810+0000 7f652e800640 1 -- 172.21.15.46:0/743775282 learned_addr learned my addr 172.21.15.46:0/743775282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652de00640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65300ce670 0x7f653013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f6527e00640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6530146740 0x7f6530133cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 -- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65300ce670 msgr2=0x7f653013fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65300ce670 0x7f653013fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 -- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6530146740 msgr2=0x7f6530133cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6530146740 0x7f6530133cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 -- 172.21.15.46:0/743775282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6530072a00 con 0x7f653015df90 2024-04-26T23:49:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652e800640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f652800c7b0 tx=0x7f652800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528004b60 con 0x7f653015df90 2024-04-26T23:49:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528004d00 con 0x7f653015df90 2024-04-26T23:49:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800aa90 con 0x7f653015df90 2024-04-26T23:49:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.811+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6530140230 con 0x7f653015df90 2024-04-26T23:49:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.812+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6530145bf0 con 0x7f653015df90 2024-04-26T23:49:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.813+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65300693b0 con 0x7f653015df90 2024-04-26T23:49:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.814+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f652800ac30 con 0x7f653015df90 2024-04-26T23:49:39.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.815+0000 7f6526000640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 0x7f650c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:39.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.815+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f652808c090 con 0x7f653015df90 2024-04-26T23:49:39.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.816+0000 7f652de00640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 0x7f650c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.817+0000 7f652de00640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 0x7f650c06edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6520000f60 tx=0x7f6520009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:39.820+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6528020070 con 0x7f653015df90 2024-04-26T23:49:40.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.028+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f653006f400 con 0x7f653015df90 2024-04-26T23:49:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.029+0000 7f6526000640 1 -- 172.21.15.46:0/743775282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7f652805a200 con 0x7f653015df90 2024-04-26T23:49:40.033 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 msgr2=0x7f650c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 0x7f650c06edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6520000f60 tx=0x7f6520009040 comp rx=0 tx=0).stop 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 msgr2=0x7f65300afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f652800c7b0 tx=0x7f652800cc70 comp rx=0 tx=0).stop 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 shutdown_connections 2024-04-26T23:49:40.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650c06c8f0 0x7f650c06edb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653015df90 0x7f65300afa60 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6530146740 0x7f6530133cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 --2- 172.21.15.46:0/743775282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65300ce670 0x7f653013fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 >> 172.21.15.46:0/743775282 conn(0x7f65300b26d0 msgr2=0x7f65300b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.032+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 shutdown_connections 2024-04-26T23:49:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.033+0000 7f652f200640 1 -- 172.21.15.46:0/743775282 wait complete. 2024-04-26T23:49:40.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.261+0000 7f7881a00640 1 Processor -- start 2024-04-26T23:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.262+0000 7f7881a00640 1 -- start start 2024-04-26T23:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.263+0000 7f7881a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f787c0720e0 0x7f787c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.263+0000 7f7881a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c073ad0 con 0x7f787c160910 2024-04-26T23:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.263+0000 7f7881a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c146a90 con 0x7f787c1466b0 2024-04-26T23:49:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.263+0000 7f7881a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f787c073d70 con 0x7f787c0720e0 2024-04-26T23:49:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.264+0000 7f7881000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f787c160910 0x7f787c0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55732/0 (socket says 172.21.15.46:55732) 2024-04-26T23:49:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.264+0000 7f7881000640 1 -- 172.21.15.46:0/3443944926 learned_addr learned my addr 172.21.15.46:0/3443944926 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.264+0000 7f787b400640 1 --2- 172.21.15.46:0/3443944926 >> v2:172.21.15.142:3300/0 conn(0x7f787c0720e0 0x7f787c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.265+0000 7f787aa00640 1 -- 172.21.15.46:0/3443944926 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877086694 0 0) 0x7f787c073ad0 con 0x7f787c160910 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.265+0000 7f787aa00640 1 -- 172.21.15.46:0/3443944926 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7860003520 con 0x7f787c160910 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.265+0000 7f787b400640 1 -- 172.21.15.46:0/3443944926 >> v1:172.21.15.142:6789/0 conn(0x7f787c1466b0 legacy=0x7f787c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.265+0000 7f787b400640 1 -- 172.21.15.46:0/3443944926 >> v1:172.21.15.46:6789/0 conn(0x7f787c160910 legacy=0x7f787c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.265+0000 7f787b400640 1 -- 172.21.15.46:0/3443944926 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c074050 con 0x7f787c0720e0 2024-04-26T23:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.266+0000 7f787b400640 1 --2- 172.21.15.46:0/3443944926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c0cd860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78680097d0 tx=0x7f786802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553e7e619ebdcab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f787aa00640 1 -- 172.21.15.46:0/3443944926 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868037070 con 0x7f787c0720e0 2024-04-26T23:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f787aa00640 1 -- 172.21.15.46:0/3443944926 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7868032440 con 0x7f787c0720e0 2024-04-26T23:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f787aa00640 1 -- 172.21.15.46:0/3443944926 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868032810 con 0x7f787c0720e0 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f7881a00640 1 -- 172.21.15.46:0/3443944926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 msgr2=0x7f787c0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f7881a00640 1 --2- 172.21.15.46:0/3443944926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c0cd860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78680097d0 tx=0x7f786802a250 comp rx=0 tx=0).stop 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f7881a00640 1 -- 172.21.15.46:0/3443944926 shutdown_connections 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f7881a00640 1 --2- 172.21.15.46:0/3443944926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c0cd860 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.267+0000 7f7881a00640 1 -- 172.21.15.46:0/3443944926 >> 172.21.15.46:0/3443944926 conn(0x7f787c0b26d0 msgr2=0x7f787c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.268+0000 7f7881a00640 1 -- 172.21.15.46:0/3443944926 shutdown_connections 2024-04-26T23:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.268+0000 7f7881a00640 1 -- 172.21.15.46:0/3443944926 wait complete. 2024-04-26T23:49:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.270+0000 7f7881a00640 1 Processor -- start 2024-04-26T23:49:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.270+0000 7f7881a00640 1 -- start start 2024-04-26T23:49:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c1f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f787c1466b0 0x7f787c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0cdda0 con 0x7f787c160910 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0cb470 con 0x7f787c0720e0 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.271+0000 7f7881a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f787c0cc0a0 con 0x7f787c1466b0 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.272+0000 7f7881000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.272+0000 7f7881000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50992/0 (socket says 172.21.15.46:50992) 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.272+0000 7f7881000640 1 -- 172.21.15.46:0/2979284145 learned_addr learned my addr 172.21.15.46:0/2979284145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.272+0000 7f787b400640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f787c1466b0 0x7f787c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.272+0000 7f787be00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c1f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 -- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f787c1466b0 msgr2=0x7f787c15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f787c1466b0 0x7f787c15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 -- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 msgr2=0x7f787c1f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c1f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 -- 172.21.15.46:0/2979284145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0d0650 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881000640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f787400caa0 tx=0x7f787400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874017070 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787400ac00 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874012400 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.273+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1f5520 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.274+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1f5ae0 con 0x7f787c160910 2024-04-26T23:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.275+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c0693b0 con 0x7f787c160910 2024-04-26T23:49:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.276+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f787401a070 con 0x7f787c160910 2024-04-26T23:49:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.277+0000 7f7873e00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 0x7f785c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.277+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f787408b9f0 con 0x7f787c160910 2024-04-26T23:49:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.278+0000 7f787be00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 0x7f785c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:40.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.279+0000 7f787be00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 0x7f785c06ede0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f787c160190 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.282+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7874059630 con 0x7f787c160910 2024-04-26T23:49:40.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.503+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f787c071dd0 con 0x7f787c160910 2024-04-26T23:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.504+0000 7f7873e00640 1 -- 172.21.15.46:0/2979284145 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) ==== 80+0+0 (secure 0 0 0) 0x7f787405d930 con 0x7f787c160910 2024-04-26T23:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.507+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 msgr2=0x7f785c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.507+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 0x7f785c06ede0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f787c160190 tx=0x7f786c009040 comp rx=0 tx=0).stop 2024-04-26T23:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.507+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 msgr2=0x7f787c0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.507+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f787400caa0 tx=0x7f787400cf60 comp rx=0 tx=0).stop 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 shutdown_connections 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f785c06c920 0x7f785c06ede0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f787c160910 0x7f787c0d0110 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f787c1466b0 0x7f787c15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 --2- 172.21.15.46:0/2979284145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f787c0720e0 0x7f787c1f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 >> 172.21.15.46:0/2979284145 conn(0x7f787c0b26d0 msgr2=0x7f787c0cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 shutdown_connections 2024-04-26T23:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:40.508+0000 7f7881a00640 1 -- 172.21.15.46:0/2979284145 wait complete. 2024-04-26T23:49:40.530 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:49:40.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:49:40.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:49:41.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:49:41.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:49:41.261 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:49:41.311 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:49:41.312 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:49:41.312 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:41.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:49:41.353 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:49:41.353 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:49:41.353 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:41.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:49:41.394 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:49:41.394 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:49:41.394 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:41.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:49:41.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.650+0000 7f01aa200640 1 Processor -- start 2024-04-26T23:49:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.650+0000 7f01aa200640 1 -- start start 2024-04-26T23:49:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a415f4e0 0x7f01a40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f01a40b4000 0x7f01a40d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40d1f60 0x7f01a40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f01a40d2990 con 0x7f01a415f4e0 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f01a40d11f0 con 0x7f01a40b4000 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.651+0000 7f01aa200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01a40d5f20 con 0x7f01a40d1f60 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a415f4e0 0x7f01a40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01a415f4e0 0x7f01a40d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50998/0 (socket says 172.21.15.46:50998) 2024-04-26T23:49:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 -- 172.21.15.46:0/3276337418 learned_addr learned my addr 172.21.15.46:0/3276337418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a9800640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40d1f60 0x7f01a40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3276337418 >> v2:172.21.15.142:3300/0 conn(0x7f01a40b4000 0x7f01a40d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 -- 172.21.15.46:0/3276337418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40d1f60 msgr2=0x7f01a40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40d1f60 0x7f01a40cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 -- 172.21.15.46:0/3276337418 >> v2:172.21.15.142:3300/0 conn(0x7f01a40b4000 msgr2=0x7f01a40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3276337418 >> v2:172.21.15.142:3300/0 conn(0x7f01a40b4000 0x7f01a40d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.652+0000 7f01a8e00640 1 -- 172.21.15.46:0/3276337418 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a40d0f10 con 0x7f01a415f4e0 2024-04-26T23:49:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.654+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a415f4e0 0x7f01a40d2df0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f01a408d880 tx=0x7f019402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a856877e705b9fee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.654+0000 7f01a3400640 1 -- 172.21.15.46:0/3276337418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194036070 con 0x7f01a415f4e0 2024-04-26T23:49:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.654+0000 7f01a3400640 1 -- 172.21.15.46:0/3276337418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194031600 con 0x7f01a415f4e0 2024-04-26T23:49:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.654+0000 7f01a3400640 1 -- 172.21.15.46:0/3276337418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940319d0 con 0x7f01a415f4e0 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 -- 172.21.15.46:0/3276337418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a415f4e0 msgr2=0x7f01a40d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a415f4e0 0x7f01a40d2df0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f01a408d880 tx=0x7f019402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 -- 172.21.15.46:0/3276337418 shutdown_connections 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a40d1f60 0x7f01a40cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 --2- 172.21.15.46:0/3276337418 >> v2:172.21.15.142:3300/0 conn(0x7f01a40b4000 0x7f01a40d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 --2- 172.21.15.46:0/3276337418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a415f4e0 0x7f01a40d2df0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 -- 172.21.15.46:0/3276337418 >> 172.21.15.46:0/3276337418 conn(0x7f01a40d5010 msgr2=0x7f01a40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 -- 172.21.15.46:0/3276337418 shutdown_connections 2024-04-26T23:49:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.655+0000 7f01aa200640 1 -- 172.21.15.46:0/3276337418 wait complete. 2024-04-26T23:49:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.657+0000 7f01aa200640 1 Processor -- start 2024-04-26T23:49:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.657+0000 7f01aa200640 1 -- start start 2024-04-26T23:49:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 0x7f01a41f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a415f4e0 0x7f01a40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01a415f8c0 con 0x7f01a40b4000 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01a40b1d60 con 0x7f01a40d1f60 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01aa200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f01a40d0210 con 0x7f01a415f4e0 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.658+0000 7f01a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51010/0 (socket says 172.21.15.46:51010) 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 -- 172.21.15.46:0/3895323436 learned_addr learned my addr 172.21.15.46:0/3895323436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a9800640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a415f4e0 0x7f01a40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 0x7f01a41f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 -- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a415f4e0 msgr2=0x7f01a40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a415f4e0 0x7f01a40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 -- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 msgr2=0x7f01a41f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 0x7f01a41f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a8e00640 1 -- 172.21.15.46:0/3895323436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a4158130 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.659+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 0x7f01a41f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01a8e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0194034040 tx=0x7f0194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194036050 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194042070 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194009510 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a41f94c0 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.660+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a40ce2c0 con 0x7f01a40b4000 2024-04-26T23:49:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.661+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a40693b0 con 0x7f01a40b4000 2024-04-26T23:49:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.663+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0194007c50 con 0x7f01a40b4000 2024-04-26T23:49:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.664+0000 7f01a0c00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.664+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f01940acbf0 con 0x7f01a40b4000 2024-04-26T23:49:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.665+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.666+0000 7f01a3e00640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cb720 tx=0x7f018c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.669+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f019407ae80 con 0x7f01a40b4000 2024-04-26T23:49:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.917+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a40d33d0 con 0x7f01a40b4000 2024-04-26T23:49:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.919+0000 7f01a0c00640 1 -- 172.21.15.46:0/3895323436 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7f019407a780 con 0x7f01a40b4000 2024-04-26T23:49:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:41.924 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:49:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 msgr2=0x7f018006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cb720 tx=0x7f018c008040 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 msgr2=0x7f01a4157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0194034040 tx=0x7f0194004a40 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 shutdown_connections 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f018006c8f0 0x7f018006edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a415f4e0 0x7f01a40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a40d1f60 0x7f01a41f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 --2- 172.21.15.46:0/3895323436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a40b4000 0x7f01a4157bf0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 >> 172.21.15.46:0/3895323436 conn(0x7f01a40d5010 msgr2=0x7f01a406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 shutdown_connections 2024-04-26T23:49:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:41.922+0000 7f01aa200640 1 -- 172.21.15.46:0/3895323436 wait complete. 2024-04-26T23:49:41.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:41.937 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:41.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:42.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.172+0000 7fe9f7e00640 1 Processor -- start 2024-04-26T23:49:42.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.172+0000 7fe9f7e00640 1 -- start start 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 0x7fe9f80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9f8073970 0x7fe9f80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9f8072bb0 con 0x7fe9f80c6a50 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe9f8072e90 con 0x7fe9f8073970 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe9f8073170 con 0x7fe9f81454a0 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9f8073970 0x7fe9f80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9f8073970 0x7fe9f80cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58494/0 (socket says 172.21.15.46:58494) 2024-04-26T23:49:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.174+0000 7fe9f6000640 1 -- 172.21.15.46:0/2317520747 learned_addr learned my addr 172.21.15.46:0/2317520747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6a00640 1 --2- 172.21.15.46:0/2317520747 >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 0x7fe9f80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6000640 1 -- 172.21.15.46:0/2317520747 >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 msgr2=0x7fe9f80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6000640 1 --2- 172.21.15.46:0/2317520747 >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 0x7fe9f80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6000640 1 -- 172.21.15.46:0/2317520747 >> v1:172.21.15.142:6789/0 conn(0x7fe9f80c6a50 legacy=0x7fe9f80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6000640 1 -- 172.21.15.46:0/2317520747 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f8073450 con 0x7fe9f8073970 2024-04-26T23:49:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.175+0000 7fe9f6a00640 1 --2- 172.21.15.46:0/2317520747 >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 0x7fe9f80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.176+0000 7fe9f6000640 1 --2- 172.21.15.46:0/2317520747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f8073970 0x7fe9f80cdc30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80d1ba0 tx=0x7fe9e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cc9fad3adf0649 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f5600640 1 -- 172.21.15.46:0/2317520747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4037070 con 0x7fe9f8073970 2024-04-26T23:49:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f5600640 1 -- 172.21.15.46:0/2317520747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e4032440 con 0x7fe9f8073970 2024-04-26T23:49:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f5600640 1 -- 172.21.15.46:0/2317520747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4032810 con 0x7fe9f8073970 2024-04-26T23:49:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 -- 172.21.15.46:0/2317520747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f8073970 msgr2=0x7fe9f80cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/2317520747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f8073970 0x7fe9f80cdc30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80d1ba0 tx=0x7fe9e402a230 comp rx=0 tx=0).stop 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 -- 172.21.15.46:0/2317520747 shutdown_connections 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/2317520747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f8073970 0x7fe9f80cdc30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/2317520747 >> v2:172.21.15.142:3300/0 conn(0x7fe9f81454a0 0x7fe9f80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.177+0000 7fe9f7e00640 1 -- 172.21.15.46:0/2317520747 >> 172.21.15.46:0/2317520747 conn(0x7fe9f80b26d0 msgr2=0x7fe9f80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.178+0000 7fe9f7e00640 1 -- 172.21.15.46:0/2317520747 shutdown_connections 2024-04-26T23:49:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.178+0000 7fe9f7e00640 1 -- 172.21.15.46:0/2317520747 wait complete. 2024-04-26T23:49:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.179+0000 7fe9f7e00640 1 Processor -- start 2024-04-26T23:49:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.179+0000 7fe9f7e00640 1 -- start start 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8073970 0x7fe9f813c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9f81454a0 0x7fe9f81f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8073e00 con 0x7fe9f80c6a50 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8072880 con 0x7fe9f8073970 2024-04-26T23:49:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.180+0000 7fe9f7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f8145880 con 0x7fe9f81454a0 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58498/0 (socket says 172.21.15.46:58498) 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 -- 172.21.15.46:0/471154575 learned_addr learned my addr 172.21.15.46:0/471154575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6a00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8073970 0x7fe9f813c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f7400640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9f81454a0 0x7fe9f81f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 -- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9f81454a0 msgr2=0x7fe9f81f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9f81454a0 0x7fe9f81f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 -- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8073970 msgr2=0x7fe9f813c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8073970 0x7fe9f813c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.181+0000 7fe9f6000640 1 -- 172.21.15.46:0/471154575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f80b4f40 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9f6000640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000c00 tx=0x7fe9e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4041020 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e4030970 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f80b51e0 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4037040 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.182+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f80b57a0 con 0x7fe9f80c6a50 2024-04-26T23:49:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.184+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f80693b0 con 0x7fe9f80c6a50 2024-04-26T23:49:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.185+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9e4044020 con 0x7fe9f80c6a50 2024-04-26T23:49:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.186+0000 7fe9ef400640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 0x7fe9d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.186+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fe9e40ac400 con 0x7fe9f80c6a50 2024-04-26T23:49:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.186+0000 7fe9f6a00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 0x7fe9d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.188+0000 7fe9f6a00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 0x7fe9d806edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800a950 tx=0x7fe9e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.191+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9e404b070 con 0x7fe9f80c6a50 2024-04-26T23:49:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.396+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9f8076430 con 0x7fe9f80c6a50 2024-04-26T23:49:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.398+0000 7fe9ef400640 1 -- 172.21.15.46:0/471154575 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16916 (secure 0 0 0) 0x7fe9e407a690 con 0x7fe9f80c6a50 2024-04-26T23:49:42.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:42.404 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:46.084255+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 msgr2=0x7fe9d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 0x7fe9d806edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800a950 tx=0x7fe9e8008040 comp rx=0 tx=0).stop 2024-04-26T23:49:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 msgr2=0x7fe9f8140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000c00 tx=0x7fe9e40046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 shutdown_connections 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe9d806c8f0 0x7fe9d806edb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9f81454a0 0x7fe9f81f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f80c6a50 0x7fe9f8140290 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 --2- 172.21.15.46:0/471154575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8073970 0x7fe9f813c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 >> 172.21.15.46:0/471154575 conn(0x7fe9f80b26d0 msgr2=0x7fe9f815f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 shutdown_connections 2024-04-26T23:49:42.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.401+0000 7fe9f7e00640 1 -- 172.21.15.46:0/471154575 wait complete. 2024-04-26T23:49:42.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:48:42.408660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:48:43.535112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:49:42.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:49:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.637+0000 7f32dbe00640 1 Processor -- start 2024-04-26T23:49:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.637+0000 7f32dbe00640 1 -- start start 2024-04-26T23:49:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.638+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 0x7f32dc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.638+0000 7f32dbe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f32dc162aa0 0x7f32dc0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.638+0000 7f32dbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc0d07d0 con 0x7f32dc0b5fb0 2024-04-26T23:49:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.638+0000 7f32dbe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0d0ab0 con 0x7f32dc162aa0 2024-04-26T23:49:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.638+0000 7f32dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc0d0d90 con 0x7f32dc1626c0 2024-04-26T23:49:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32da000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f32dc0b5fb0 0x7f32dc0d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36526/0 (socket says 172.21.15.46:36526) 2024-04-26T23:49:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32da000640 1 -- 172.21.15.46:0/3927489679 learned_addr learned my addr 172.21.15.46:0/3927489679 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32daa00640 1 --2- 172.21.15.46:0/3927489679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 0x7f32dc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32db400640 1 --2- 172.21.15.46:0/3927489679 >> v2:172.21.15.142:3300/0 conn(0x7f32dc162aa0 0x7f32dc0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564207859 0 0) 0x7f32dc0d07d0 con 0x7f32dc0b5fb0 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.639+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32bc003520 con 0x7f32dc0b5fb0 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.640+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2688013379 0 0) 0x7f32bc003520 con 0x7f32dc0b5fb0 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.640+0000 7f32daa00640 1 -- 172.21.15.46:0/3927489679 >> v2:172.21.15.142:3300/0 conn(0x7f32dc162aa0 msgr2=0x7f32dc0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.640+0000 7f32daa00640 1 --2- 172.21.15.46:0/3927489679 >> v2:172.21.15.142:3300/0 conn(0x7f32dc162aa0 0x7f32dc0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.640+0000 7f32daa00640 1 -- 172.21.15.46:0/3927489679 >> v1:172.21.15.46:6789/0 conn(0x7f32dc0b5fb0 legacy=0x7f32dc0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.640+0000 7f32daa00640 1 -- 172.21.15.46:0/3927489679 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0d2c80 con 0x7f32dc1626c0 2024-04-26T23:49:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.641+0000 7f32daa00640 1 --2- 172.21.15.46:0/3927489679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 0x7f32dc0b47c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3ebd9091251006 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.642+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc037070 con 0x7f32dc1626c0 2024-04-26T23:49:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.642+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32cc02aba0 con 0x7f32dc1626c0 2024-04-26T23:49:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.642+0000 7f32d9600640 1 -- 172.21.15.46:0/3927489679 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4001b70 con 0x7f32dc1626c0 2024-04-26T23:49:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.642+0000 7f32dbe00640 1 -- 172.21.15.46:0/3927489679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 msgr2=0x7f32dc0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.642+0000 7f32dbe00640 1 --2- 172.21.15.46:0/3927489679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 0x7f32dc0b47c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02a110 comp rx=0 tx=0).stop 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 -- 172.21.15.46:0/3927489679 shutdown_connections 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 --2- 172.21.15.46:0/3927489679 >> v2:172.21.15.142:3300/0 conn(0x7f32dc162aa0 0x7f32dc0d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 --2- 172.21.15.46:0/3927489679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc1626c0 0x7f32dc0b47c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 -- 172.21.15.46:0/3927489679 >> 172.21.15.46:0/3927489679 conn(0x7f32dc09e2a0 msgr2=0x7f32dc0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 -- 172.21.15.46:0/3927489679 shutdown_connections 2024-04-26T23:49:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.643+0000 7f32dbe00640 1 -- 172.21.15.46:0/3927489679 wait complete. 2024-04-26T23:49:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.645+0000 7f32dbe00640 1 Processor -- start 2024-04-26T23:49:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.645+0000 7f32dbe00640 1 -- start start 2024-04-26T23:49:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 0x7f32dc1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 0x7f32dc0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0d21d0 con 0x7f32dc1626c0 2024-04-26T23:49:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0d2050 con 0x7f32dc162aa0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc0d31a0 con 0x7f32dc0b5fb0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58512/0 (socket says 172.21.15.46:58512) 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32da000640 1 -- 172.21.15.46:0/539445227 learned_addr learned my addr 172.21.15.46:0/539445227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.646+0000 7f32daa00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 0x7f32dc1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32db400640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 0x7f32dc0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 -- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 msgr2=0x7f32dc1f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 0x7f32dc1f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 -- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 msgr2=0x7f32dc0ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 0x7f32dc0ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 -- 172.21.15.46:0/539445227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0b7d70 con 0x7f32dc1626c0 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32db400640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 0x7f32dc0ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32daa00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 0x7f32dc1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32da000640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f32c400ca70 tx=0x7f32c400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4017070 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c400a6a0 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c400aa90 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.647+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc0b8050 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.648+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc0b8610 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.649+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc0693b0 con 0x7f32dc1626c0 2024-04-26T23:49:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.650+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32c401a070 con 0x7f32dc1626c0 2024-04-26T23:49:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.650+0000 7f32d2a00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.651+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f32c408b590 con 0x7f32dc1626c0 2024-04-26T23:49:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.651+0000 7f32daa00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.652+0000 7f32daa00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 0x7f32b806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f32cc034000 tx=0x7f32cc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.655+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32c4059820 con 0x7f32dc1626c0 2024-04-26T23:49:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.859+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32dc0b5040 con 0x7f32dc1626c0 2024-04-26T23:49:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.860+0000 7f32d2a00640 1 -- 172.21.15.46:0/539445227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f32c4059120 con 0x7f32dc1626c0 2024-04-26T23:49:42.864 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 msgr2=0x7f32b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 0x7f32b806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f32cc034000 tx=0x7f32cc02e000 comp rx=0 tx=0).stop 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 msgr2=0x7f32dc0a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f32c400ca70 tx=0x7f32c400cf30 comp rx=0 tx=0).stop 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 shutdown_connections 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32b806c8f0 0x7f32b806edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32dc162aa0 0x7f32dc0ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32dc1626c0 0x7f32dc0a0310 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 --2- 172.21.15.46:0/539445227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32dc0b5fb0 0x7f32dc1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.862+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 >> 172.21.15.46:0/539445227 conn(0x7f32dc09e2a0 msgr2=0x7f32dc06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.863+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 shutdown_connections 2024-04-26T23:49:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:42.863+0000 7f32dbe00640 1 -- 172.21.15.46:0/539445227 wait complete. 2024-04-26T23:49:42.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.099+0000 7fa856600640 1 Processor -- start 2024-04-26T23:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.099+0000 7fa856600640 1 -- start start 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa856600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8500cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa856600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa8500cdb30 0x7fa8500735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa856600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8500ce7d0 con 0x7fa850073b30 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa856600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa8500cdf90 con 0x7fa8500cdb30 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa856600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa8500ceab0 con 0x7fa85015cec0 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa855c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa850073b30 0x7fa850071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36534/0 (socket says 172.21.15.46:36534) 2024-04-26T23:49:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.101+0000 7fa855c00640 1 -- 172.21.15.46:0/4266501202 learned_addr learned my addr 172.21.15.46:0/4266501202 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa847e00640 1 --2- 172.21.15.46:0/4266501202 >> v2:172.21.15.142:3300/0 conn(0x7fa8500cdb30 0x7fa8500735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa855200640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8500cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847846855 0 0) 0x7fa8500ce7d0 con 0x7fa850073b30 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa830003520 con 0x7fa850073b30 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2007477432 0 0) 0x7fa830003520 con 0x7fa850073b30 2024-04-26T23:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 msgr2=0x7fa8500cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8500cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 >> v2:172.21.15.142:3300/0 conn(0x7fa8500cdb30 msgr2=0x7fa8500735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 --2- 172.21.15.46:0/4266501202 >> v2:172.21.15.142:3300/0 conn(0x7fa8500cdb30 0x7fa8500735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8500ced90 con 0x7fa850073b30 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa848001a10 con 0x7fa850073b30 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.102+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 >> v1:172.21.15.46:6789/0 conn(0x7fa850073b30 legacy=0x7fa850071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8300039b0 0x7fa830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa830004800 0x7fa830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8500ceab0 con 0x7fa830004800 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa83000b9c0 con 0x7fa8300039b0 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa83000bb40 con 0x7fa85015cec0 2024-04-26T23:49:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa830004800 0x7fa830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 -- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 msgr2=0x7fa8300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa855c00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa855200640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8300039b0 0x7fa830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa85015cec0 0x7fa8300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 -- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8300039b0 msgr2=0x7fa830003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8300039b0 0x7fa830003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.103+0000 7fa847e00640 1 -- 172.21.15.46:0/4266501202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa830003520 con 0x7fa830004800 2024-04-26T23:49:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.104+0000 7fa847e00640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa830004800 0x7fa830004c80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa840009ac0 tx=0x7fa830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8b92fdac91dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840036070 con 0x7fa830004800 2024-04-26T23:49:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840031400 con 0x7fa830004800 2024-04-26T23:49:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa84fe00640 1 -- 172.21.15.46:0/4266501202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840009690 con 0x7fa830004800 2024-04-26T23:49:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa856600640 1 -- 172.21.15.46:0/4266501202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa830004800 msgr2=0x7fa830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa856600640 1 --2- 172.21.15.46:0/4266501202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa830004800 0x7fa830004c80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa840009ac0 tx=0x7fa830005830 comp rx=0 tx=0).stop 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa855c00640 1 -- 172.21.15.46:0/4266501202 reap_dead start 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa856600640 1 -- 172.21.15.46:0/4266501202 shutdown_connections 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.105+0000 7fa856600640 1 -- 172.21.15.46:0/4266501202 >> 172.21.15.46:0/4266501202 conn(0x7fa8500b27b0 msgr2=0x7fa8500b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.106+0000 7fa856600640 1 -- 172.21.15.46:0/4266501202 shutdown_connections 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.106+0000 7fa856600640 1 -- 172.21.15.46:0/4266501202 wait complete. 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.107+0000 7fa856600640 1 Processor -- start 2024-04-26T23:49:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.108+0000 7fa856600640 1 -- start start 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa830004800 0x7fa8500c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 0x7fa8500c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa85015d2a0 con 0x7fa8500c40e0 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8500cf350 con 0x7fa8500aa630 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa856600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa850073fa0 con 0x7fa830004800 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58516/0 (socket says 172.21.15.46:58516) 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 -- 172.21.15.46:0/773706692 learned_addr learned my addr 172.21.15.46:0/773706692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855200640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa830004800 0x7fa8500c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa847e00640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 0x7fa8500c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 -- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa830004800 msgr2=0x7fa8500c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa830004800 0x7fa8500c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 -- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 msgr2=0x7fa8500c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 0x7fa8500c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa855c00640 1 -- 172.21.15.46:0/773706692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8500c6750 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.109+0000 7fa847e00640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 0x7fa8500c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa855c00640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa8500b3830 tx=0x7fa84800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004d60 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84800bb80 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848007110 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8500c6a30 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.110+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8500c6ff0 con 0x7fa8500c40e0 2024-04-26T23:49:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.111+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8500693b0 con 0x7fa8500c40e0 2024-04-26T23:49:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.113+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa84801b020 con 0x7fa8500c40e0 2024-04-26T23:49:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.114+0000 7fa84d600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.114+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fa84808bed0 con 0x7fa8500c40e0 2024-04-26T23:49:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.115+0000 7fa855200640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.116+0000 7fa855200640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 0x7fa82c06edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa838009930 tx=0x7fa838008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.119+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa84805a160 con 0x7fa8500c40e0 2024-04-26T23:49:43.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.322+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8500cf8f0 con 0x7fa8500c40e0 2024-04-26T23:49:43.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.322+0000 7fa84d600640 1 -- 172.21.15.46:0/773706692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7fa848059a60 con 0x7fa8500c40e0 2024-04-26T23:49:43.327 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:49:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 msgr2=0x7fa82c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 0x7fa82c06edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa838009930 tx=0x7fa838008040 comp rx=0 tx=0).stop 2024-04-26T23:49:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 msgr2=0x7fa8501f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa8500b3830 tx=0x7fa84800b4a0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 shutdown_connections 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500c40e0 0x7fa8501f5dc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8500aa630 0x7fa8500c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 --2- 172.21.15.46:0/773706692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa830004800 0x7fa8500c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.325+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 >> 172.21.15.46:0/773706692 conn(0x7fa8500b27b0 msgr2=0x7fa850072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.326+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 shutdown_connections 2024-04-26T23:49:43.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.326+0000 7fa856600640 1 -- 172.21.15.46:0/773706692 wait complete. 2024-04-26T23:49:43.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:49:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.553+0000 7f9f16c00640 1 Processor -- start 2024-04-26T23:49:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.553+0000 7f9f16c00640 1 -- start start 2024-04-26T23:49:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.555+0000 7f9f16c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f100d04c0 0x7f9f1006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.555+0000 7f9f16c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.555+0000 7f9f16c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f100b4b30 con 0x7f9f100b4320 2024-04-26T23:49:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.555+0000 7f9f16c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9f1006f740 con 0x7f9f100d04c0 2024-04-26T23:49:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.555+0000 7f9f16c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f100b4e10 con 0x7f9f1006f2e0 2024-04-26T23:49:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.556+0000 7f9f07e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.556+0000 7f9f07e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:55804/0 (socket says 172.21.15.46:55804) 2024-04-26T23:49:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.556+0000 7f9f07e00640 1 -- 172.21.15.46:0/3804185396 learned_addr learned my addr 172.21.15.46:0/3804185396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.556+0000 7f9f0fe00640 1 --2- 172.21.15.46:0/3804185396 >> v2:172.21.15.142:3300/0 conn(0x7f9f100d04c0 0x7f9f1006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.557+0000 7f9f07e00640 1 -- 172.21.15.46:0/3804185396 >> v2:172.21.15.142:3300/0 conn(0x7f9f100d04c0 msgr2=0x7f9f1006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.557+0000 7f9f07e00640 1 --2- 172.21.15.46:0/3804185396 >> v2:172.21.15.142:3300/0 conn(0x7f9f100d04c0 0x7f9f1006eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.557+0000 7f9f07e00640 1 -- 172.21.15.46:0/3804185396 >> v1:172.21.15.142:6789/0 conn(0x7f9f100b4320 legacy=0x7f9f10071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.557+0000 7f9f07e00640 1 -- 172.21.15.46:0/3804185396 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f100b50f0 con 0x7f9f1006f2e0 2024-04-26T23:49:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.558+0000 7f9f07e00640 1 --2- 172.21.15.46:0/3804185396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009870 tx=0x7f9ef802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30286cff1b76b368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.559+0000 7f9f0f400640 1 -- 172.21.15.46:0/3804185396 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8037070 con 0x7f9f1006f2e0 2024-04-26T23:49:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.559+0000 7f9f0f400640 1 -- 172.21.15.46:0/3804185396 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef8032440 con 0x7f9f1006f2e0 2024-04-26T23:49:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.559+0000 7f9f0f400640 1 -- 172.21.15.46:0/3804185396 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8032810 con 0x7f9f1006f2e0 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.559+0000 7f9f16c00640 1 -- 172.21.15.46:0/3804185396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 msgr2=0x7f9f100b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.559+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3804185396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009870 tx=0x7f9ef802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 -- 172.21.15.46:0/3804185396 shutdown_connections 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3804185396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1006f2e0 0x7f9f100b3de0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3804185396 >> v2:172.21.15.142:3300/0 conn(0x7f9f100d04c0 0x7f9f1006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 -- 172.21.15.46:0/3804185396 >> 172.21.15.46:0/3804185396 conn(0x7f9f1006c980 msgr2=0x7f9f100b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 -- 172.21.15.46:0/3804185396 shutdown_connections 2024-04-26T23:49:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.560+0000 7f9f16c00640 1 -- 172.21.15.46:0/3804185396 wait complete. 2024-04-26T23:49:43.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.561+0000 7f9f16c00640 1 Processor -- start 2024-04-26T23:49:43.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.561+0000 7f9f16c00640 1 -- start start 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 0x7f9f1015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 0x7f9f10143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f100b4770 con 0x7f9f100b4320 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f100b56b0 con 0x7f9f1006f2e0 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.562+0000 7f9f16c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f100b48f0 con 0x7f9f100d04c0 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58526/0 (socket says 172.21.15.46:58526) 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 -- 172.21.15.46:0/3746360205 learned_addr learned my addr 172.21.15.46:0/3746360205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f14c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 0x7f9f10143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f0fe00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 0x7f9f1015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 -- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 msgr2=0x7f9f10143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 0x7f9f10143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 -- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 msgr2=0x7f9f1015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 0x7f9f1015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 -- 172.21.15.46:0/3746360205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f10138690 con 0x7f9f100b4320 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f14c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 0x7f9f10143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f0fe00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 0x7f9f1015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.563+0000 7f9f07e00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8000c00 tx=0x7f9ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.564+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8037040 con 0x7f9f100b4320 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.564+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef8042070 con 0x7f9f100b4320 2024-04-26T23:49:43.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.564+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8030880 con 0x7f9f100b4320 2024-04-26T23:49:43.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.564+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1015baf0 con 0x7f9f100b4320 2024-04-26T23:49:43.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.564+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1015ce10 con 0x7f9f100b4320 2024-04-26T23:49:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.565+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f10069530 con 0x7f9f100b4320 2024-04-26T23:49:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.566+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ef802ad10 con 0x7f9f100b4320 2024-04-26T23:49:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.567+0000 7f9f0cc00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 0x7f9eec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.567+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f9ef80ac2b0 con 0x7f9f100b4320 2024-04-26T23:49:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.568+0000 7f9f0fe00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 0x7f9eec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.569+0000 7f9f0fe00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 0x7f9eec06edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009ac0 tx=0x7f9f00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.571+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ef807a4b0 con 0x7f9f100b4320 2024-04-26T23:49:43.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.768+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f1013ac50 con 0x7f9f100b4320 2024-04-26T23:49:43.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.769+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3746360205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7f9ef8079db0 con 0x7f9f100b4320 2024-04-26T23:49:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:49:43.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 msgr2=0x7f9eec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 0x7f9eec06edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009ac0 tx=0x7f9f00008040 comp rx=0 tx=0).stop 2024-04-26T23:49:43.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 msgr2=0x7f9f10132140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8000c00 tx=0x7f9ef80046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 shutdown_connections 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9eec06c8f0 0x7f9eec06edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f100d04c0 0x7f9f10143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f100b4320 0x7f9f10132140 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 --2- 172.21.15.46:0/3746360205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f1006f2e0 0x7f9f1015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 >> 172.21.15.46:0/3746360205 conn(0x7f9f1006c980 msgr2=0x7f9f10071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 shutdown_connections 2024-04-26T23:49:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.771+0000 7f9f16c00640 1 -- 172.21.15.46:0/3746360205 wait complete. 2024-04-26T23:49:43.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.990+0000 7fe0ac800640 1 Processor -- start 2024-04-26T23:49:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.990+0000 7fe0ac800640 1 -- start start 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.992+0000 7fe0ac800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a415b730 0x7fe0a40b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.992+0000 7fe0ac800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a40b0d90 0x7fe0a40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.992+0000 7fe0ac800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0a415f3c0 con 0x7fe0a40b55d0 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.992+0000 7fe0ac800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a415ffd0 con 0x7fe0a40b0d90 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.992+0000 7fe0ac800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a415f6a0 con 0x7fe0a415b730 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a40b0d90 0x7fe0a40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a40b0d90 0x7fe0a40b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58530/0 (socket says 172.21.15.46:58530) 2024-04-26T23:49:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 -- 172.21.15.46:0/4273515559 learned_addr learned my addr 172.21.15.46:0/4273515559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9e00640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a415b730 0x7fe0a40b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 -- 172.21.15.46:0/4273515559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a415b730 msgr2=0x7fe0a40b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a415b730 0x7fe0a40b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 -- 172.21.15.46:0/4273515559 >> v1:172.21.15.142:6789/0 conn(0x7fe0a40b55d0 legacy=0x7fe0a406f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.993+0000 7fe0a9400640 1 -- 172.21.15.46:0/4273515559 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a415f980 con 0x7fe0a40b0d90 2024-04-26T23:49:44.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.994+0000 7fe0a9400640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a40b0d90 0x7fe0a40b5090 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe094009870 tx=0x7fe09402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14520b53db85646 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0a8a00640 1 -- 172.21.15.46:0/4273515559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094037070 con 0x7fe0a40b0d90 2024-04-26T23:49:44.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0a8a00640 1 -- 172.21.15.46:0/4273515559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe094032440 con 0x7fe0a40b0d90 2024-04-26T23:49:44.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0a8a00640 1 -- 172.21.15.46:0/4273515559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094032810 con 0x7fe0a40b0d90 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 -- 172.21.15.46:0/4273515559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a40b0d90 msgr2=0x7fe0a40b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a40b0d90 0x7fe0a40b5090 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe094009870 tx=0x7fe09402a210 comp rx=0 tx=0).stop 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 -- 172.21.15.46:0/4273515559 shutdown_connections 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a40b0d90 0x7fe0a40b5090 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 --2- 172.21.15.46:0/4273515559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a415b730 0x7fe0a40b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.995+0000 7fe0ac800640 1 -- 172.21.15.46:0/4273515559 >> 172.21.15.46:0/4273515559 conn(0x7fe0a40b38b0 msgr2=0x7fe0a40d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.996+0000 7fe0ac800640 1 -- 172.21.15.46:0/4273515559 shutdown_connections 2024-04-26T23:49:44.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.996+0000 7fe0ac800640 1 -- 172.21.15.46:0/4273515559 wait complete. 2024-04-26T23:49:44.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.997+0000 7fe0ac800640 1 Processor -- start 2024-04-26T23:49:44.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.998+0000 7fe0ac800640 1 -- start start 2024-04-26T23:49:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.998+0000 7fe0ac800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0a40b0d90 0x7fe0a415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0ac800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a40b55d0 0x7fe0a414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0ac800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0ac800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4070680 con 0x7fe0a415b730 2024-04-26T23:49:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0ac800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a415bb10 con 0x7fe0a40b0d90 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0ac800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a4070800 con 0x7fe0a40b55d0 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58534/0 (socket says 172.21.15.46:58534) 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0aa800640 1 -- 172.21.15.46:0/368533272 learned_addr learned my addr 172.21.15.46:0/368533272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0a9400640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a40b55d0 0x7fe0a414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0a9e00640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0a40b0d90 0x7fe0a415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:43.999+0000 7fe0aa800640 1 -- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a40b55d0 msgr2=0x7fe0a414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0aa800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a40b55d0 0x7fe0a414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0aa800640 1 -- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0a40b0d90 msgr2=0x7fe0a415aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0aa800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0a40b0d90 0x7fe0a415aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0aa800640 1 -- 172.21.15.46:0/368533272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a414f770 con 0x7fe0a415b730 2024-04-26T23:49:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0aa800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c7a0 tx=0x7fe0a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0016020 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a000aa50 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0012420 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a414fa10 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.000+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a414ffd0 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.002+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a4150310 con 0x7fe0a415b730 2024-04-26T23:49:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.003+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0a0019020 con 0x7fe0a415b730 2024-04-26T23:49:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.003+0000 7fe09e000640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 0x7fe08806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.004+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fe0a008b740 con 0x7fe0a415b730 2024-04-26T23:49:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.004+0000 7fe0a9e00640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 0x7fe08806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.005+0000 7fe0a9e00640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 0x7fe08806edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe098006fd0 tx=0x7fe098008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.011+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0a4150310 con 0x7fe0a415b730 2024-04-26T23:49:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.259+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a4069530 con 0x7fe0a415b730 2024-04-26T23:49:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.261+0000 7fe09e000640 1 -- 172.21.15.46:0/368533272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7fe0a0020020 con 0x7fe0a415b730 2024-04-26T23:49:44.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:44.266 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:49:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:49:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 msgr2=0x7fe08806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 0x7fe08806edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe098006fd0 tx=0x7fe098008040 comp rx=0 tx=0).stop 2024-04-26T23:49:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 msgr2=0x7fe0a415e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c7a0 tx=0x7fe0a000cc60 comp rx=0 tx=0).stop 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 shutdown_connections 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe08806c8f0 0x7fe08806edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a415b730 0x7fe0a415e220 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0a40b55d0 0x7fe0a414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 --2- 172.21.15.46:0/368533272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0a40b0d90 0x7fe0a415aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.263+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 >> 172.21.15.46:0/368533272 conn(0x7fe0a40b38b0 msgr2=0x7fe0a406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.264+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 shutdown_connections 2024-04-26T23:49:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.264+0000 7fe0ac800640 1 -- 172.21.15.46:0/368533272 wait complete. 2024-04-26T23:49:44.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.492+0000 7f0dd2000640 1 Processor -- start 2024-04-26T23:49:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.492+0000 7f0dd2000640 1 -- start start 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dd2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0dcc15f4e0 0x7f0dcc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dd2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dd2000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc0d11f0 con 0x7f0dcc0d1f60 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dd2000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc0d2990 con 0x7f0dcc0b4000 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dd2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc0d5f20 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dcbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dcbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58544/0 (socket says 172.21.15.46:58544) 2024-04-26T23:49:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.493+0000 7f0dcbe00640 1 -- 172.21.15.46:0/3765090690 learned_addr learned my addr 172.21.15.46:0/3765090690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:44.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.494+0000 7f0dcbe00640 1 -- 172.21.15.46:0/3765090690 >> v2:172.21.15.142:3300/0 conn(0x7f0dcc15f4e0 msgr2=0x7f0dcc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:44.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.494+0000 7f0dcbe00640 1 --2- 172.21.15.46:0/3765090690 >> v2:172.21.15.142:3300/0 conn(0x7f0dcc15f4e0 0x7f0dcc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.494+0000 7f0dcbe00640 1 -- 172.21.15.46:0/3765090690 >> v1:172.21.15.142:6789/0 conn(0x7f0dcc0d1f60 legacy=0x7f0dcc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:44.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.494+0000 7f0dcbe00640 1 -- 172.21.15.46:0/3765090690 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0d0f10 con 0x7f0dcc0b4000 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.495+0000 7f0dcbe00640 1 --2- 172.21.15.46:0/3765090690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc08d8b0 tx=0x7f0db802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc54f0020761c8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.495+0000 7f0dcb400640 1 -- 172.21.15.46:0/3765090690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8037070 con 0x7f0dcc0b4000 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.495+0000 7f0dcb400640 1 -- 172.21.15.46:0/3765090690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db8032440 con 0x7f0dcc0b4000 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.495+0000 7f0dcb400640 1 -- 172.21.15.46:0/3765090690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8032810 con 0x7f0dcc0b4000 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 -- 172.21.15.46:0/3765090690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0b4000 msgr2=0x7f0dcc0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3765090690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc08d8b0 tx=0x7f0db802a230 comp rx=0 tx=0).stop 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 -- 172.21.15.46:0/3765090690 shutdown_connections 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3765090690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc0d1a20 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3765090690 >> v2:172.21.15.142:3300/0 conn(0x7f0dcc15f4e0 0x7f0dcc0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 -- 172.21.15.46:0/3765090690 >> 172.21.15.46:0/3765090690 conn(0x7f0dcc0d5010 msgr2=0x7f0dcc0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 -- 172.21.15.46:0/3765090690 shutdown_connections 2024-04-26T23:49:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.496+0000 7f0dd2000640 1 -- 172.21.15.46:0/3765090690 wait complete. 2024-04-26T23:49:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.499+0000 7f0dd2000640 1 Processor -- start 2024-04-26T23:49:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.499+0000 7f0dd2000640 1 -- start start 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 0x7f0dcc1f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc15f8c0 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0b1d60 con 0x7f0dcc0b4000 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0b3290 con 0x7f0dcc0d1f60 2024-04-26T23:49:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.500+0000 7f0dd1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58560/0 (socket says 172.21.15.46:58560) 2024-04-26T23:49:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 -- 172.21.15.46:0/3434495132 learned_addr learned my addr 172.21.15.46:0/3434495132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 -- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 msgr2=0x7f0dcc1f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dcbe00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 0x7f0dcc1f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd0c00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 0x7f0dcc1f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 -- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0b4000 msgr2=0x7f0dcc161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 -- 172.21.15.46:0/3434495132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc1f6270 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dcbe00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 0x7f0dcc1f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.501+0000 7f0dd1600640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c780 tx=0x7f0dc400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.502+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4017070 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.502+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc4003da0 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.502+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40052b0 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.502+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc1f5f90 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.502+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc0cc930 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.503+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc0693b0 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.504+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dc401a070 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.505+0000 7f0dc8c00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.505+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f0dc408b380 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.506+0000 7f0dd0c00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.507+0000 7f0dd0c00640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 0x7f0dac06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0d1bd0 tx=0x7f0dc0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.510+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0dc4059610 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.718+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dcc0d33d0 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.721+0000 7f0dc8c00640 1 -- 172.21.15.46:0/3434495132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16916 (secure 0 0 0) 0x7f0dc4058f10 con 0x7f0dcc15f4e0 2024-04-26T23:49:44.727 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:44.727 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:46.084255+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.724+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 msgr2=0x7f0dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.724+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 0x7f0dac06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0d1bd0 tx=0x7f0dc0008040 comp rx=0 tx=0).stop 2024-04-26T23:49:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.724+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 msgr2=0x7f0dcc1f8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.724+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c780 tx=0x7f0dc400cc40 comp rx=0 tx=0).stop 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 shutdown_connections 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc15f4e0 0x7f0dcc1f8240 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dcc0d1f60 0x7f0dcc1f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 --2- 172.21.15.46:0/3434495132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dcc0b4000 0x7f0dcc161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 >> 172.21.15.46:0/3434495132 conn(0x7f0dcc0d5010 msgr2=0x7f0dcc15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 shutdown_connections 2024-04-26T23:49:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.725+0000 7f0dd2000640 1 -- 172.21.15.46:0/3434495132 wait complete. 2024-04-26T23:49:44.751 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:49:44.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.956+0000 7f5ffd200640 1 Processor -- start 2024-04-26T23:49:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.957+0000 7f5ffd200640 1 -- start start 2024-04-26T23:49:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.958+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5ff8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ffd200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff8073ad0 con 0x7f5ff81466b0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ffd200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff8146a90 con 0x7f5ff80720e0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ffd200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff8073d70 con 0x7f5ff8160910 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ff7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ff81466b0 0x7f5ff806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36544/0 (socket says 172.21.15.46:36544) 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ff7400640 1 -- 172.21.15.46:0/1070192905 learned_addr learned my addr 172.21.15.46:0/1070192905 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ff7e00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5ff8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432113643 0 0) 0x7f5ff8073ad0 con 0x7f5ff81466b0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.959+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fdc003520 con 0x7f5ff81466b0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2654751637 0 0) 0x7f5fdc003520 con 0x7f5ff81466b0 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 msgr2=0x7f5ff8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5ff8160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 >> v1:172.21.15.142:6789/0 conn(0x7f5ff80720e0 legacy=0x7f5ff80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff8074050 con 0x7f5ff81466b0 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5fe8001670 con 0x7f5ff81466b0 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 >> v1:172.21.15.46:6789/0 conn(0x7f5ff81466b0 legacy=0x7f5ff806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5fdc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5fdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fdc004d40 0x7f5fdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8073d70 con 0x7f5fdc003f60 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc00baa0 con 0x7f5fdc004d40 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc00bca0 con 0x7f5ff8160910 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.960+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5fdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff7e00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fdc004d40 0x7f5fdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff7400640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5fdc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff6a00640 1 -- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 msgr2=0x7f5fdc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff8160910 0x7f5fdc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff6a00640 1 -- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fdc004d40 msgr2=0x7f5fdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fdc004d40 0x7f5fdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.961+0000 7f5ff6a00640 1 -- 172.21.15.46:0/1070192905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc003520 con 0x7f5fdc003f60 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.962+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5fdc004800 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001750 tx=0x7f5fdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf19e91e9fcb633d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.962+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4001370 con 0x7f5fdc003f60 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.962+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe40052d0 con 0x7f5fdc003f60 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.962+0000 7f5ff6000640 1 -- 172.21.15.46:0/1070192905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe402abe0 con 0x7f5fdc003f60 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 -- 172.21.15.46:0/1070192905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 msgr2=0x7f5fdc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 --2- 172.21.15.46:0/1070192905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5fdc004800 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001750 tx=0x7f5fdc005830 comp rx=0 tx=0).stop 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ff7e00640 1 -- 172.21.15.46:0/1070192905 reap_dead start 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 -- 172.21.15.46:0/1070192905 shutdown_connections 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 -- 172.21.15.46:0/1070192905 >> 172.21.15.46:0/1070192905 conn(0x7f5ff80b26d0 msgr2=0x7f5ff80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 -- 172.21.15.46:0/1070192905 shutdown_connections 2024-04-26T23:49:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.963+0000 7f5ffd200640 1 -- 172.21.15.46:0/1070192905 wait complete. 2024-04-26T23:49:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.965+0000 7f5ffd200640 1 Processor -- start 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.965+0000 7f5ffd200640 1 -- start start 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 0x7f5ff81415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff80b4160 0x7f5ff80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8160da0 con 0x7f5fdc003f60 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80cb470 con 0x7f5ff8141130 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ffd200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff8160f20 con 0x7f5ff80b4160 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58582/0 (socket says 172.21.15.46:58582) 2024-04-26T23:49:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ff7400640 1 -- 172.21.15.46:0/605918830 learned_addr learned my addr 172.21.15.46:0/605918830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.966+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 0x7f5ff81415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7e00640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff80b4160 0x7f5ff80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 -- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff80b4160 msgr2=0x7f5ff80b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff80b4160 0x7f5ff80b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 -- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 msgr2=0x7f5ff81415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 0x7f5ff81415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 -- 172.21.15.46:0/605918830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff80c7c10 con 0x7f5fdc003f60 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 0x7f5ff81415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ff7400640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800d7a0 tx=0x7f5fe800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8019070 con 0x7f5fdc003f60 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe80074e0 con 0x7f5fdc003f60 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80049e0 con 0x7f5fdc003f60 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff80b4b20 con 0x7f5fdc003f60 2024-04-26T23:49:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.967+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff80c7e80 con 0x7f5fdc003f60 2024-04-26T23:49:44.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.968+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff80693b0 con 0x7f5fdc003f60 2024-04-26T23:49:44.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.970+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fe8004b80 con 0x7f5fdc003f60 2024-04-26T23:49:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.971+0000 7f5fef400640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.971+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f5fe808b710 con 0x7f5fdc003f60 2024-04-26T23:49:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.971+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.972+0000 7f5ff6a00640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 0x7f5fd806ede0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4032000 tx=0x7f5fe402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:44.975+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fe80599a0 con 0x7f5fdc003f60 2024-04-26T23:49:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.218+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ff80c7180 con 0x7f5fdc003f60 2024-04-26T23:49:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.219+0000 7f5fef400640 1 -- 172.21.15.46:0/605918830 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7f5fe80592a0 con 0x7f5fdc003f60 2024-04-26T23:49:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:45.224 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:49:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 msgr2=0x7f5fd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 0x7f5fd806ede0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4032000 tx=0x7f5fe402e000 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 msgr2=0x7f5ff80b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800d7a0 tx=0x7f5fe800dc60 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 shutdown_connections 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ff80b4160 0x7f5ff80b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ff8141130 0x7f5ff81415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 --2- 172.21.15.46:0/605918830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc003f60 0x7f5ff80b5810 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 >> 172.21.15.46:0/605918830 conn(0x7f5ff80b26d0 msgr2=0x7f5ff806eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 shutdown_connections 2024-04-26T23:49:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.222+0000 7f5ffd200640 1 -- 172.21.15.46:0/605918830 wait complete. 2024-04-26T23:49:45.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:45.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.456+0000 7f87d8800640 1 Processor -- start 2024-04-26T23:49:45.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.456+0000 7f87d8800640 1 -- start start 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d8800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87d006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d8800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d00b43f0 con 0x7f87d006ea50 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d8800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d00b46d0 con 0x7f87d00b3e80 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d8800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87d006eec0 con 0x7f87d0142280 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87d006ea50 0x7f87d0144f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36558/0 (socket says 172.21.15.46:36558) 2024-04-26T23:49:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.458+0000 7f87d6800640 1 -- 172.21.15.46:0/1475693940 learned_addr learned my addr 172.21.15.46:0/1475693940 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d5e00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87d006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962266084 0 0) 0x7f87d00b43f0 con 0x7f87d006ea50 2024-04-26T23:49:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87b8003520 con 0x7f87d006ea50 2024-04-26T23:49:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267687236 0 0) 0x7f87b8003520 con 0x7f87d006ea50 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 msgr2=0x7f87d006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87d006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 >> v1:172.21.15.142:6789/0 conn(0x7f87d00b3e80 legacy=0x7f87d0144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.459+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d00b49b0 con 0x7f87d006ea50 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f87cc0019f0 con 0x7f87d006ea50 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 >> v1:172.21.15.46:6789/0 conn(0x7f87d006ea50 legacy=0x7f87d0144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b80039e0 0x7f87b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87d006eec0 con 0x7f87b8004d40 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87b800ba70 con 0x7f87b80039e0 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87b800bc20 con 0x7f87d0142280 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d6800640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d5e00640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b80039e0 0x7f87b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d5400640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.460+0000 7f87d6800640 1 -- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 msgr2=0x7f87b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.461+0000 7f87d6800640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.461+0000 7f87d6800640 1 -- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b80039e0 msgr2=0x7f87b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.461+0000 7f87d6800640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b80039e0 0x7f87b8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.461+0000 7f87d6800640 1 -- 172.21.15.46:0/1475693940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b8003520 con 0x7f87b8004d40 2024-04-26T23:49:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.461+0000 7f87d5400640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d0142280 0x7f87b8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.462+0000 7f87d6800640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87b80051c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00a7a0 tx=0x7f87b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b303898291cadc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.462+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00a3f0 con 0x7f87b8004d40 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.462+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc02f070 con 0x7f87b8004d40 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.462+0000 7f87d4a00640 1 -- 172.21.15.46:0/1475693940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc02bc10 con 0x7f87b8004d40 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 -- 172.21.15.46:0/1475693940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 msgr2=0x7f87b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 --2- 172.21.15.46:0/1475693940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87b80051c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00a7a0 tx=0x7f87b8005830 comp rx=0 tx=0).stop 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d6800640 1 -- 172.21.15.46:0/1475693940 reap_dead start 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 -- 172.21.15.46:0/1475693940 shutdown_connections 2024-04-26T23:49:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 -- 172.21.15.46:0/1475693940 >> 172.21.15.46:0/1475693940 conn(0x7f87d006c530 msgr2=0x7f87d006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 -- 172.21.15.46:0/1475693940 shutdown_connections 2024-04-26T23:49:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.463+0000 7f87d8800640 1 -- 172.21.15.46:0/1475693940 wait complete. 2024-04-26T23:49:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.465+0000 7f87d8800640 1 Processor -- start 2024-04-26T23:49:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.465+0000 7f87d8800640 1 -- start start 2024-04-26T23:49:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87d015bdf0 0x7f87d015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 0x7f87d00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0145660 con 0x7f87b8004d40 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87d00b4f70 con 0x7f87d015bdf0 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d8800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87d01428a0 con 0x7f87d00ce700 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58590/0 (socket says 172.21.15.46:58590) 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d5e00640 1 -- 172.21.15.46:0/2462975118 learned_addr learned my addr 172.21.15.46:0/2462975118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d5400640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87d015bdf0 0x7f87d015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.467+0000 7f87d6800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 0x7f87d00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 -- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 msgr2=0x7f87d00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 0x7f87d00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 -- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87d015bdf0 msgr2=0x7f87d015c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87d015bdf0 0x7f87d015c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 -- 172.21.15.46:0/2462975118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d00cf3c0 con 0x7f87b8004d40 2024-04-26T23:49:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d6800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 0x7f87d00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d5e00640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f87c4011600 tx=0x7f87c4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c400f920 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c400fac0 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c400fe70 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.468+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d00cf6a0 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.469+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d00cfc60 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.470+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d0069530 con 0x7f87b8004d40 2024-04-26T23:49:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.471+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87c4015460 con 0x7f87b8004d40 2024-04-26T23:49:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.471+0000 7f87ca000640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 0x7f87b406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.472+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f87c408c700 con 0x7f87b8004d40 2024-04-26T23:49:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.472+0000 7f87d5400640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 0x7f87b406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.473+0000 7f87d5400640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 0x7f87b406ee00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f87d015f9b0 tx=0x7f87c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.476+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f87c4019030 con 0x7f87b8004d40 2024-04-26T23:49:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.720+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d006cd20 con 0x7f87b8004d40 2024-04-26T23:49:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.721+0000 7f87ca000640 1 -- 172.21.15.46:0/2462975118 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5188 (secure 0 0 0) 0x7f87c405a980 con 0x7f87b8004d40 2024-04-26T23:49:45.727 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:45.727 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:48:44.554510+0000","modified":"2024-04-26T23:48:47.636866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6836/475647056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":475647056},{"type":"v1","addr":"172.21.15.142:6836","nonce":475647056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8724,"qdb_cluster":[8724]},"id":8}]} 2024-04-26T23:49:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-04-26T23:49:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 msgr2=0x7f87b406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 0x7f87b406ee00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f87d015f9b0 tx=0x7f87c0009040 comp rx=0 tx=0).stop 2024-04-26T23:49:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 msgr2=0x7f87d013ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f87c4011600 tx=0x7f87c4011ac0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 shutdown_connections 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f87b406c940 0x7f87b406ee00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87d00ce700 0x7f87d00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87d015bdf0 0x7f87d015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 --2- 172.21.15.46:0/2462975118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b8004d40 0x7f87d013ff80 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.724+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 >> 172.21.15.46:0/2462975118 conn(0x7f87d006c530 msgr2=0x7f87d00b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.725+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 shutdown_connections 2024-04-26T23:49:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.725+0000 7f87d8800640 1 -- 172.21.15.46:0/2462975118 wait complete. 2024-04-26T23:49:45.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:45.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.957+0000 7f1d03400640 1 Processor -- start 2024-04-26T23:49:45.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.957+0000 7f1d03400640 1 -- start start 2024-04-26T23:49:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.959+0000 7f1d03400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1cfc142280 0x7f1cfc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.959+0000 7f1d03400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cfc0b43f0 con 0x7f1cfc06ea50 2024-04-26T23:49:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.959+0000 7f1d03400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cfc0b46d0 con 0x7f1cfc0b3e80 2024-04-26T23:49:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.959+0000 7f1d03400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1cfc06eea0 con 0x7f1cfc142280 2024-04-26T23:49:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.959+0000 7f1d01400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1cfc06ea50 0x7f1cfc144f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36572/0 (socket says 172.21.15.46:36572) 2024-04-26T23:49:45.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1d01400640 1 -- 172.21.15.46:0/118883541 learned_addr learned my addr 172.21.15.46:0/118883541 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1d00a00640 1 --2- 172.21.15.46:0/118883541 >> v2:172.21.15.142:3300/0 conn(0x7f1cfc142280 0x7f1cfc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778403131 0 0) 0x7f1cfc0b43f0 con 0x7f1cfc06ea50 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ce4003520 con 0x7f1cfc06ea50 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3480113586 0 0) 0x7f1ce4003520 con 0x7f1cfc06ea50 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 >> v2:172.21.15.142:3300/0 conn(0x7f1cfc142280 msgr2=0x7f1cfc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 --2- 172.21.15.46:0/118883541 >> v2:172.21.15.142:3300/0 conn(0x7f1cfc142280 0x7f1cfc06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 >> v1:172.21.15.142:6789/0 conn(0x7f1cfc0b3e80 legacy=0x7f1cfc144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc0b49b0 con 0x7f1cfc06ea50 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1cf40019f0 con 0x7f1cfc06ea50 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 >> v1:172.21.15.46:6789/0 conn(0x7f1cfc06ea50 legacy=0x7f1cfc144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.960+0000 7f1cfb400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ce40039b0 0x7f1ce4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfb400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce4004800 0x7f1ce4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfb400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 0x7f1ce400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc06eea0 con 0x7f1ce40051c0 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce400c060 con 0x7f1ce4004800 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce400c260 con 0x7f1ce40039b0 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 0x7f1ce400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfbe00640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce4004800 0x7f1ce4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 -- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ce40039b0 msgr2=0x7f1ce4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ce40039b0 0x7f1ce4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 -- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce4004800 msgr2=0x7f1ce4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce4004800 0x7f1ce4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1d01400640 1 -- 172.21.15.46:0/118883541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce4003520 con 0x7f1ce40051c0 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.961+0000 7f1cfbe00640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce4004800 0x7f1ce4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.962+0000 7f1d01400640 1 -- 172.21.15.46:0/118883541 reap_dead start 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.962+0000 7f1d01400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 0x7f1ce400b9c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400a860 tx=0x7f1ce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5026e391124bae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400a480 con 0x7f1ce40051c0 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf402f070 con 0x7f1ce40051c0 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1cfb400640 1 -- 172.21.15.46:0/118883541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf402bca0 con 0x7f1ce40051c0 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 -- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 msgr2=0x7f1ce400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 0x7f1ce400b9c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400a860 tx=0x7f1ce4005830 comp rx=0 tx=0).stop 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 -- 172.21.15.46:0/118883541 shutdown_connections 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 --2- 172.21.15.46:0/118883541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce40051c0 0x7f1ce400b9c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 -- 172.21.15.46:0/118883541 >> 172.21.15.46:0/118883541 conn(0x7f1cfc06c530 msgr2=0x7f1cfc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:45.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 -- 172.21.15.46:0/118883541 shutdown_connections 2024-04-26T23:49:45.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.963+0000 7f1d03400640 1 -- 172.21.15.46:0/118883541 wait complete. 2024-04-26T23:49:45.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.965+0000 7f1d03400640 1 Processor -- start 2024-04-26T23:49:45.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.965+0000 7f1d03400640 1 -- start start 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce40051c0 0x7f1cfc13dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cfc13e610 0x7f1cfc1f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc1428a0 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc0b4f70 con 0x7f1ce40051c0 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.966+0000 7f1d03400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc142a20 con 0x7f1cfc13e610 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58602/0 (socket says 172.21.15.46:58602) 2024-04-26T23:49:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 -- 172.21.15.46:0/3174243380 learned_addr learned my addr 172.21.15.46:0/3174243380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d00a00640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce40051c0 0x7f1cfc13dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1cfbe00640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cfc13e610 0x7f1cfc1f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 -- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cfc13e610 msgr2=0x7f1cfc1f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cfc13e610 0x7f1cfc1f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 -- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce40051c0 msgr2=0x7f1cfc13dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce40051c0 0x7f1cfc13dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.967+0000 7f1d01400640 1 -- 172.21.15.46:0/3174243380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc1f9570 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1d01400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1cf402bab0 tx=0x7f1cf402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400a480 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf4004d60 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40346b0 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc0cf530 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.968+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc0cfaf0 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.969+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cfc069530 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.970+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cf4034850 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.971+0000 7f1cf8c00640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.971+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f1cf40abbb0 con 0x7f1cfc1f8c30 2024-04-26T23:49:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.972+0000 7f1d00a00640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.973+0000 7f1d00a00640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 0x7f1ce006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009a50 tx=0x7f1cec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:45.975+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cf4047020 con 0x7f1cfc1f8c30 2024-04-26T23:49:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.184+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cfc069e80 con 0x7f1cfc1f8c30 2024-04-26T23:49:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.186+0000 7f1cf8c00640 1 -- 172.21.15.46:0/3174243380 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16916 (secure 0 0 0) 0x7f1cf4079e40 con 0x7f1cfc1f8c30 2024-04-26T23:49:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:46.191 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:48:46.084255+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:48:42.408660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-26T23:48:43.535112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 msgr2=0x7f1ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 0x7f1ce006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009a50 tx=0x7f1cec008040 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 msgr2=0x7f1cfc15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1cf402bab0 tx=0x7f1cf402d070 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 shutdown_connections 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cfc1f8c30 0x7f1cfc15eba0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cfc13e610 0x7f1cfc1f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 --2- 172.21.15.46:0/3174243380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ce40051c0 0x7f1cfc13dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 >> 172.21.15.46:0/3174243380 conn(0x7f1cfc06c530 msgr2=0x7f1cfc0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 shutdown_connections 2024-04-26T23:49:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.189+0000 7f1d03400640 1 -- 172.21.15.46:0/3174243380 wait complete. 2024-04-26T23:49:46.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:49:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.422+0000 7f39ee800640 1 Processor -- start 2024-04-26T23:49:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.423+0000 7f39ee800640 1 -- start start 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39e815edf0 0x7f39e80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39e815f1d0 0x7f39e80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f39e80cf5c0 con 0x7f39e815edf0 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39e80b03c0 con 0x7f39e815f1d0 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ee800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39e80cb680 con 0x7f39e80cb1a0 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39e815edf0 0x7f39e80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39e815edf0 0x7f39e80cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58612/0 (socket says 172.21.15.46:58612) 2024-04-26T23:49:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.424+0000 7f39ed400640 1 -- 172.21.15.46:0/919307498 learned_addr learned my addr 172.21.15.46:0/919307498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39eca00640 1 --2- 172.21.15.46:0/919307498 >> v2:172.21.15.142:3300/0 conn(0x7f39e815f1d0 0x7f39e80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ede00640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ed400640 1 -- 172.21.15.46:0/919307498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 msgr2=0x7f39e80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ed400640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e80cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ed400640 1 -- 172.21.15.46:0/919307498 >> v2:172.21.15.142:3300/0 conn(0x7f39e815f1d0 msgr2=0x7f39e80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ed400640 1 --2- 172.21.15.46:0/919307498 >> v2:172.21.15.142:3300/0 conn(0x7f39e815f1d0 0x7f39e80cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.425+0000 7f39ed400640 1 -- 172.21.15.46:0/919307498 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e80cb960 con 0x7f39e815edf0 2024-04-26T23:49:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.426+0000 7f39ed400640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e80cf080 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f39e80ce620 tx=0x7f39d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a76939413d9382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.426+0000 7f39e7e00640 1 -- 172.21.15.46:0/919307498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8037070 con 0x7f39e815edf0 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.426+0000 7f39e7e00640 1 -- 172.21.15.46:0/919307498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d8032440 con 0x7f39e815edf0 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.426+0000 7f39e7e00640 1 -- 172.21.15.46:0/919307498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8032810 con 0x7f39e815edf0 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 -- 172.21.15.46:0/919307498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 msgr2=0x7f39e80cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e80cf080 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f39e80ce620 tx=0x7f39d802a270 comp rx=0 tx=0).stop 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 -- 172.21.15.46:0/919307498 shutdown_connections 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 --2- 172.21.15.46:0/919307498 >> v2:172.21.15.142:3300/0 conn(0x7f39e815f1d0 0x7f39e80cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 --2- 172.21.15.46:0/919307498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e80cf080 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 -- 172.21.15.46:0/919307498 >> 172.21.15.46:0/919307498 conn(0x7f39e80b26d0 msgr2=0x7f39e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 -- 172.21.15.46:0/919307498 shutdown_connections 2024-04-26T23:49:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.427+0000 7f39ee800640 1 -- 172.21.15.46:0/919307498 wait complete. 2024-04-26T23:49:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.429+0000 7f39ee800640 1 Processor -- start 2024-04-26T23:49:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.429+0000 7f39ee800640 1 -- start start 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e8160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 0x7f39e815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39e80cc200 con 0x7f39e815edf0 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39e80cbf20 con 0x7f39e8162280 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.430+0000 7f39ee800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39e80ae8e0 con 0x7f39e80cb1a0 2024-04-26T23:49:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58618/0 (socket says 172.21.15.46:58618) 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39eca00640 1 -- 172.21.15.46:0/3530732313 learned_addr learned my addr 172.21.15.46:0/3530732313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39ede00640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 0x7f39e815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39ed400640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e8160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39eca00640 1 -- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 msgr2=0x7f39e8160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.431+0000 7f39eca00640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e8160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39eca00640 1 -- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 msgr2=0x7f39e815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39eca00640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 0x7f39e815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39eca00640 1 -- 172.21.15.46:0/3530732313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e80c7d80 con 0x7f39e815edf0 2024-04-26T23:49:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39ede00640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 0x7f39e815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:49:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39eca00640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00d5a0 tx=0x7f39dc00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc019070 con 0x7f39e815edf0 2024-04-26T23:49:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39dc004240 con 0x7f39e815edf0 2024-04-26T23:49:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e80c8060 con 0x7f39e815edf0 2024-04-26T23:49:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc003610 con 0x7f39e815edf0 2024-04-26T23:49:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.432+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e80c8620 con 0x7f39e815edf0 2024-04-26T23:49:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.434+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e80693b0 con 0x7f39e815edf0 2024-04-26T23:49:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.435+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39dc01c070 con 0x7f39e815edf0 2024-04-26T23:49:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.437+0000 7f39e5600640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.437+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f39dc08b510 con 0x7f39e815edf0 2024-04-26T23:49:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.437+0000 7f39ed400640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.439+0000 7f39ed400640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.441+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39dc059150 con 0x7f39e815edf0 2024-04-26T23:49:46.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.688+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f39e80c69b0 con 0x7f39e815edf0 2024-04-26T23:49:46.694 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:49:46.689+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:49:46.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.693+0000 7f39e5600640 1 -- 172.21.15.46:0/3530732313 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f39dc05d450 con 0x7f39e815edf0 2024-04-26T23:49:46.698 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:49:46.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 msgr2=0x7f39cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d802e000 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 msgr2=0x7f39e8161d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00d5a0 tx=0x7f39dc00da60 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 shutdown_connections 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8162280 0x7f39e815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e815edf0 0x7f39e8161d40 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 --2- 172.21.15.46:0/3530732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39e80cb1a0 0x7f39e8160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.696+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 >> 172.21.15.46:0/3530732313 conn(0x7f39e80b26d0 msgr2=0x7f39e80c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.697+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 shutdown_connections 2024-04-26T23:49:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.697+0000 7f39ee800640 1 -- 172.21.15.46:0/3530732313 wait complete. 2024-04-26T23:49:46.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:49:46.804 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:49:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.921+0000 7f60f1800640 1 Processor -- start 2024-04-26T23:49:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.921+0000 7f60f1800640 1 -- start start 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60f1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60ec06f4b0 0x7f60ec0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60f1800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60f1800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ec0b4b30 con 0x7f60ec0d04c0 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60f1800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec0b4e10 con 0x7f60ec06f4b0 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60f1800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec0b50f0 con 0x7f60ec0b4320 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60ec06f4b0 0x7f60ec0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60ec06f4b0 0x7f60ec0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58630/0 (socket says 172.21.15.46:58630) 2024-04-26T23:49:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.923+0000 7f60ea000640 1 -- 172.21.15.46:0/3083930778 learned_addr learned my addr 172.21.15.46:0/3083930778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.924+0000 7f60ea000640 1 -- 172.21.15.46:0/3083930778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 msgr2=0x7f60ec0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.924+0000 7f60ea000640 1 --2- 172.21.15.46:0/3083930778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.924+0000 7f60ea000640 1 -- 172.21.15.46:0/3083930778 >> v1:172.21.15.142:6789/0 conn(0x7f60ec0d04c0 legacy=0x7f60ec06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.924+0000 7f60ea000640 1 -- 172.21.15.46:0/3083930778 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0b53d0 con 0x7f60ec06f4b0 2024-04-26T23:49:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.925+0000 7f60ea000640 1 --2- 172.21.15.46:0/3083930778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec0b3de0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b62a0 tx=0x7f60d402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f3dcf5da91d04e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.925+0000 7f60e9600640 1 -- 172.21.15.46:0/3083930778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60ec06f4b0 2024-04-26T23:49:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.925+0000 7f60e9600640 1 -- 172.21.15.46:0/3083930778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d402ae50 con 0x7f60ec06f4b0 2024-04-26T23:49:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.925+0000 7f60e9600640 1 -- 172.21.15.46:0/3083930778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032660 con 0x7f60ec06f4b0 2024-04-26T23:49:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 -- 172.21.15.46:0/3083930778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 msgr2=0x7f60ec0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 --2- 172.21.15.46:0/3083930778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec0b3de0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b62a0 tx=0x7f60d402a140 comp rx=0 tx=0).stop 2024-04-26T23:49:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 -- 172.21.15.46:0/3083930778 shutdown_connections 2024-04-26T23:49:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 --2- 172.21.15.46:0/3083930778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 --2- 172.21.15.46:0/3083930778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec0b3de0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 -- 172.21.15.46:0/3083930778 >> 172.21.15.46:0/3083930778 conn(0x7f60ec06c980 msgr2=0x7f60ec0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 -- 172.21.15.46:0/3083930778 shutdown_connections 2024-04-26T23:49:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.926+0000 7f60f1800640 1 -- 172.21.15.46:0/3083930778 wait complete. 2024-04-26T23:49:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.928+0000 7f60f1800640 1 Processor -- start 2024-04-26T23:49:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.928+0000 7f60f1800640 1 -- start start 2024-04-26T23:49:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 0x7f60ec142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0b4910 con 0x7f60ec06f4b0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0b4790 con 0x7f60ec0d04c0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60f1800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec0ce9b0 con 0x7f60ec0b4320 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58644/0 (socket says 172.21.15.46:58644) 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60eaa00640 1 -- 172.21.15.46:0/3493223276 learned_addr learned my addr 172.21.15.46:0/3493223276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60ea000640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.929+0000 7f60eb400640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 0x7f60ec142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 -- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 msgr2=0x7f60ec1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 -- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 msgr2=0x7f60ec142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 0x7f60ec142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 -- 172.21.15.46:0/3493223276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec13f790 con 0x7f60ec06f4b0 2024-04-26T23:49:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60ea000640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eb400640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 0x7f60ec142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60eaa00640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00ba50 tx=0x7f60dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc019070 con 0x7f60ec06f4b0 2024-04-26T23:49:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc00cc20 con 0x7f60ec06f4b0 2024-04-26T23:49:46.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc014680 con 0x7f60ec06f4b0 2024-04-26T23:49:46.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec13e9d0 con 0x7f60ec06f4b0 2024-04-26T23:49:46.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.930+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec13ef90 con 0x7f60ec06f4b0 2024-04-26T23:49:46.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.932+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec069530 con 0x7f60ec06f4b0 2024-04-26T23:49:46.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.933+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60dc01c070 con 0x7f60ec06f4b0 2024-04-26T23:49:46.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.934+0000 7f60e2a00640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 0x7f60c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.934+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6187+0+0 (secure 0 0 0) 0x7f60dc08b9d0 con 0x7f60ec06f4b0 2024-04-26T23:49:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.934+0000 7f60ea000640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 0x7f60c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:46.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.936+0000 7f60ea000640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 0x7f60c806ede0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:46.938+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60dc0595c0 con 0x7f60ec06f4b0 2024-04-26T23:49:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.184+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60ec0735d0 con 0x7f60ec06f4b0 2024-04-26T23:49:47.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.697+0000 7f60e2a00640 1 -- 172.21.15.46:0/3493223276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) ==== 104+0+0 (secure 0 0 0) 0x7f60dc05d8c0 con 0x7f60ec06f4b0 2024-04-26T23:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 msgr2=0x7f60c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 0x7f60c806ede0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).stop 2024-04-26T23:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 msgr2=0x7f60ec1400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00ba50 tx=0x7f60dc00bf10 comp rx=0 tx=0).stop 2024-04-26T23:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 shutdown_connections 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f60c806c920 0x7f60c806ede0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ec0d04c0 0x7f60ec142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec0b4320 0x7f60ec1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 --2- 172.21.15.46:0/3493223276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec06f4b0 0x7f60ec1400a0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 >> 172.21.15.46:0/3493223276 conn(0x7f60ec06c980 msgr2=0x7f60ec0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 shutdown_connections 2024-04-26T23:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.701+0000 7f60f1800640 1 -- 172.21.15.46:0/3493223276 wait complete. 2024-04-26T23:49:47.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:49:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.943+0000 7fec14a00640 1 Processor -- start 2024-04-26T23:49:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.943+0000 7fec14a00640 1 -- start start 2024-04-26T23:49:47.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.945+0000 7fec14a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec101469c0 0x7fec1015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.945+0000 7fec14a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec100b35d0 0x7fec100727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.945+0000 7fec14a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec10146da0 con 0x7fec100cf560 2024-04-26T23:49:47.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.945+0000 7fec14a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fec100d0ac0 con 0x7fec100b35d0 2024-04-26T23:49:47.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.945+0000 7fec14a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec100cfeb0 con 0x7fec101469c0 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec100b35d0 0x7fec100727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec100b35d0 0x7fec100727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58660/0 (socket says 172.21.15.46:58660) 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 -- 172.21.15.46:0/3860193007 learned_addr learned my addr 172.21.15.46:0/3860193007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0de00640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec101469c0 0x7fec1015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 -- 172.21.15.46:0/3860193007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec101469c0 msgr2=0x7fec1015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec101469c0 0x7fec1015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 -- 172.21.15.46:0/3860193007 >> v1:172.21.15.142:6789/0 conn(0x7fec100cf560 legacy=0x7fec100b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.946+0000 7fec0e800640 1 -- 172.21.15.46:0/3860193007 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec100d0190 con 0x7fec100b35d0 2024-04-26T23:49:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.947+0000 7fec0e800640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec100b35d0 0x7fec100727d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec0802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a37e911d06f08c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.948+0000 7fec0ca00640 1 -- 172.21.15.46:0/3860193007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08036070 con 0x7fec100b35d0 2024-04-26T23:49:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.948+0000 7fec0ca00640 1 -- 172.21.15.46:0/3860193007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec08031600 con 0x7fec100b35d0 2024-04-26T23:49:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.948+0000 7fec0ca00640 1 -- 172.21.15.46:0/3860193007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec080319d0 con 0x7fec100b35d0 2024-04-26T23:49:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.948+0000 7fec14a00640 1 -- 172.21.15.46:0/3860193007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec100b35d0 msgr2=0x7fec100727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.948+0000 7fec14a00640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec100b35d0 0x7fec100727d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec0802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 -- 172.21.15.46:0/3860193007 shutdown_connections 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec100b35d0 0x7fec100727d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 --2- 172.21.15.46:0/3860193007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec101469c0 0x7fec1015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 -- 172.21.15.46:0/3860193007 >> 172.21.15.46:0/3860193007 conn(0x7fec1006c770 msgr2=0x7fec1006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 -- 172.21.15.46:0/3860193007 shutdown_connections 2024-04-26T23:49:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.949+0000 7fec14a00640 1 -- 172.21.15.46:0/3860193007 wait complete. 2024-04-26T23:49:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.950+0000 7fec14a00640 1 Processor -- start 2024-04-26T23:49:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.951+0000 7fec14a00640 1 -- start start 2024-04-26T23:49:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.951+0000 7fec14a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec100b35d0 0x7fec10145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec14a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec100cf560 0x7fec100abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec14a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec100cf9e0 con 0x7fec101469c0 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec14a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec100d0750 con 0x7fec100b35d0 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec14a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec100cfb60 con 0x7fec100cf560 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58668/0 (socket says 172.21.15.46:58668) 2024-04-26T23:49:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 -- 172.21.15.46:0/1073767389 learned_addr learned my addr 172.21.15.46:0/1073767389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 -- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec100cf560 msgr2=0x7fec100abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec100cf560 0x7fec100abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 -- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec100b35d0 msgr2=0x7fec10145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec100b35d0 0x7fec10145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.952+0000 7fec0e800640 1 -- 172.21.15.46:0/1073767389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec100ab8c0 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec0e800640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08036050 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec08042070 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08009510 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec100aa820 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.953+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec100ab0c0 con 0x7fec101469c0 2024-04-26T23:49:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.955+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec10069530 con 0x7fec101469c0 2024-04-26T23:49:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.955+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec08007bd0 con 0x7fec101469c0 2024-04-26T23:49:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.956+0000 7fec06000640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 0x7febf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.956+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6187+0+0 (secure 0 0 0) 0x7fec080acc40 con 0x7fec101469c0 2024-04-26T23:49:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.957+0000 7fec0de00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 0x7febf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:47.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.958+0000 7fec0de00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 0x7febf006ede0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fec00006fd0 tx=0x7fec00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:47.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:47.961+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec08045360 con 0x7fec101469c0 2024-04-26T23:49:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.171+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec10069da0 con 0x7fec101469c0 2024-04-26T23:49:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.707+0000 7fec06000640 1 -- 172.21.15.46:0/1073767389 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) ==== 181+0+0 (secure 0 0 0) 0x7fec0807ae90 con 0x7fec101469c0 2024-04-26T23:49:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:49:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.712+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 msgr2=0x7febf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.712+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 0x7febf006ede0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fec00006fd0 tx=0x7fec00008040 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.712+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 msgr2=0x7fec1013d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.712+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec08004a40 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 shutdown_connections 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febf006c920 0x7febf006ede0 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fec00006fd0 tx=0x7fec00008040 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec101469c0 0x7fec1013d6c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec100cf560 0x7fec100abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 --2- 172.21.15.46:0/1073767389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec100b35d0 0x7fec10145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.713+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 >> 172.21.15.46:0/1073767389 conn(0x7fec1006c770 msgr2=0x7fec101438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.714+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 shutdown_connections 2024-04-26T23:49:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.714+0000 7fec14a00640 1 -- 172.21.15.46:0/1073767389 wait complete. 2024-04-26T23:49:48.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:49:48.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.938+0000 7f7e3e800640 1 Processor -- start 2024-04-26T23:49:48.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.938+0000 7f7e3e800640 1 -- start start 2024-04-26T23:49:48.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.940+0000 7f7e3e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 0x7f7e380d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.940+0000 7f7e3e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 0x7f7e380b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.940+0000 7f7e3e800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3806e270 con 0x7f7e3815f130 2024-04-26T23:49:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.940+0000 7f7e3e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3806e550 con 0x7f7e380d10e0 2024-04-26T23:49:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.940+0000 7f7e3e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3806e830 con 0x7f7e380b4510 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3ca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 0x7f7e380d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 0x7f7e380b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3d400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f7e3815f130 0x7f7e38071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:33840/0 (socket says 172.21.15.46:33840) 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3d400640 1 -- 172.21.15.46:0/4189993822 learned_addr learned my addr 172.21.15.46:0/4189993822 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3de00640 1 -- 172.21.15.46:0/4189993822 >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 msgr2=0x7f7e380d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3de00640 1 --2- 172.21.15.46:0/4189993822 >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 0x7f7e380d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3de00640 1 -- 172.21.15.46:0/4189993822 >> v1:172.21.15.142:6789/0 conn(0x7f7e3815f130 legacy=0x7f7e38071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3de00640 1 -- 172.21.15.46:0/4189993822 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3806eb10 con 0x7f7e380b4510 2024-04-26T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.941+0000 7f7e3ca00640 1 --2- 172.21.15.46:0/4189993822 >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 0x7f7e380d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.943+0000 7f7e3de00640 1 --2- 172.21.15.46:0/4189993822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 0x7f7e380b4980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009d90 tx=0x7f7e3002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20ddde970941a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.943+0000 7f7e37e00640 1 -- 172.21.15.46:0/4189993822 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30007740 con 0x7f7e380b4510 2024-04-26T23:49:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.943+0000 7f7e37e00640 1 -- 172.21.15.46:0/4189993822 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e300078e0 con 0x7f7e380b4510 2024-04-26T23:49:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.943+0000 7f7e37e00640 1 -- 172.21.15.46:0/4189993822 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30007cb0 con 0x7f7e380b4510 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 -- 172.21.15.46:0/4189993822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 msgr2=0x7f7e380b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 --2- 172.21.15.46:0/4189993822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 0x7f7e380b4980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009d90 tx=0x7f7e3002a660 comp rx=0 tx=0).stop 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 -- 172.21.15.46:0/4189993822 shutdown_connections 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 --2- 172.21.15.46:0/4189993822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e380b4510 0x7f7e380b4980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 --2- 172.21.15.46:0/4189993822 >> v2:172.21.15.142:3300/0 conn(0x7f7e380d10e0 0x7f7e380d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 -- 172.21.15.46:0/4189993822 >> 172.21.15.46:0/4189993822 conn(0x7f7e3806c770 msgr2=0x7f7e3806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 -- 172.21.15.46:0/4189993822 shutdown_connections 2024-04-26T23:49:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.944+0000 7f7e3e800640 1 -- 172.21.15.46:0/4189993822 wait complete. 2024-04-26T23:49:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.946+0000 7f7e3e800640 1 Processor -- start 2024-04-26T23:49:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.946+0000 7f7e3e800640 1 -- start start 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 0x7f7e381572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e3815f130 0x7f7e38160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3815f510 con 0x7f7e380b4510 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380725a0 con 0x7f7e380d10e0 2024-04-26T23:49:48.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3806f310 con 0x7f7e3815f130 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58674/0 (socket says 172.21.15.46:58674) 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3d400640 1 -- 172.21.15.46:0/3341162772 learned_addr learned my addr 172.21.15.46:0/3341162772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3ca00640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 0x7f7e381572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.947+0000 7f7e3de00640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e3815f130 0x7f7e38160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 -- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e3815f130 msgr2=0x7f7e38160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e3815f130 0x7f7e38160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 -- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 msgr2=0x7f7e381572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 0x7f7e381572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 -- 172.21.15.46:0/3341162772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38156230 con 0x7f7e380b4510 2024-04-26T23:49:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3ca00640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 0x7f7e381572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:49:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3d400640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c4a0 tx=0x7f7e2c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c017070 con 0x7f7e380b4510 2024-04-26T23:49:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c005ce0 con 0x7f7e380b4510 2024-04-26T23:49:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c007270 con 0x7f7e380b4510 2024-04-26T23:49:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e38156510 con 0x7f7e380b4510 2024-04-26T23:49:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.948+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e38156ad0 con 0x7f7e380b4510 2024-04-26T23:49:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.950+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e380693b0 con 0x7f7e380b4510 2024-04-26T23:49:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.950+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e2c01a070 con 0x7f7e380b4510 2024-04-26T23:49:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.951+0000 7f7e35600640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.952+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5749+0+0 (secure 0 0 0) 0x7f7e2c08ae30 con 0x7f7e380b4510 2024-04-26T23:49:48.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.952+0000 7f7e3ca00640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.953+0000 7f7e3ca00640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 0x7f7e1806edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ba0 tx=0x7f7e24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:48.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:48.956+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e2c058bd0 con 0x7f7e380b4510 2024-04-26T23:49:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.164+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e38076540 con 0x7f7e380b4510 2024-04-26T23:49:49.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.719+0000 7f7e35600640 1 -- 172.21.15.46:0/3341162772 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) ==== 169+0+0 (secure 0 0 0) 0x7f7e2c05ced0 con 0x7f7e380b4510 2024-04-26T23:49:49.724 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:49:49.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.723+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 msgr2=0x7f7e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.723+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 0x7f7e1806edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ba0 tx=0x7f7e24008040 comp rx=0 tx=0).stop 2024-04-26T23:49:49.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.723+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 msgr2=0x7f7e38155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.723+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c4a0 tx=0x7f7e2c00c960 comp rx=0 tx=0).stop 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 shutdown_connections 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e3815f130 0x7f7e38160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e380d10e0 0x7f7e381572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 --2- 172.21.15.46:0/3341162772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e380b4510 0x7f7e38155880 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.724+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 >> 172.21.15.46:0/3341162772 conn(0x7f7e3806c770 msgr2=0x7f7e3815de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.725+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 shutdown_connections 2024-04-26T23:49:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.725+0000 7f7e3e800640 1 -- 172.21.15.46:0/3341162772 wait complete. 2024-04-26T23:49:49.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:49:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.953+0000 7f7433200640 1 Processor -- start 2024-04-26T23:49:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.953+0000 7f7433200640 1 -- start start 2024-04-26T23:49:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7433200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 0x7f74340d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7433200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74340d25b0 0x7f74340d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7433200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74340d0810 con 0x7f74341629c0 2024-04-26T23:49:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7433200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f74340b6430 con 0x7f74340d25b0 2024-04-26T23:49:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7433200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f74340d0a80 con 0x7f74340c9b60 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7431e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f74341629c0 0x7f74340d1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36586/0 (socket says 172.21.15.46:36586) 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7431e00640 1 -- 172.21.15.46:0/3301874448 learned_addr learned my addr 172.21.15.46:0/3301874448 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.955+0000 7f7432800640 1 --2- 172.21.15.46:0/3301874448 >> v2:172.21.15.46:3300/0 conn(0x7f74340d25b0 0x7f74340d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7431400640 1 --2- 172.21.15.46:0/3301874448 >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 0x7f74340d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7430a00640 1 -- 172.21.15.46:0/3301874448 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618328435 0 0) 0x7f74340d0810 con 0x7f74341629c0 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7430a00640 1 -- 172.21.15.46:0/3301874448 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7414003520 con 0x7f74341629c0 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7432800640 1 -- 172.21.15.46:0/3301874448 >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 msgr2=0x7f74340d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7432800640 1 --2- 172.21.15.46:0/3301874448 >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 0x7f74340d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7432800640 1 -- 172.21.15.46:0/3301874448 >> v1:172.21.15.46:6789/0 conn(0x7f74341629c0 legacy=0x7f74340d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7432800640 1 -- 172.21.15.46:0/3301874448 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74340d0d60 con 0x7f74340d25b0 2024-04-26T23:49:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.956+0000 7f7431400640 1 --2- 172.21.15.46:0/3301874448 >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 0x7f74340d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.957+0000 7f7432800640 1 --2- 172.21.15.46:0/3301874448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f74340d2d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f74340d5fe0 tx=0x7f742c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220278b6187caccb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.957+0000 7f7430a00640 1 -- 172.21.15.46:0/3301874448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c037070 con 0x7f74340d25b0 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.957+0000 7f7430a00640 1 -- 172.21.15.46:0/3301874448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c032440 con 0x7f74340d25b0 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.957+0000 7f7430a00640 1 -- 172.21.15.46:0/3301874448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c032810 con 0x7f74340d25b0 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 -- 172.21.15.46:0/3301874448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 msgr2=0x7f74340d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 --2- 172.21.15.46:0/3301874448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f74340d2d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f74340d5fe0 tx=0x7f742c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 -- 172.21.15.46:0/3301874448 shutdown_connections 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 --2- 172.21.15.46:0/3301874448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f74340d2d50 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 --2- 172.21.15.46:0/3301874448 >> v2:172.21.15.142:3300/0 conn(0x7f74340c9b60 0x7f74340d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 -- 172.21.15.46:0/3301874448 >> 172.21.15.46:0/3301874448 conn(0x7f74340d4f70 msgr2=0x7f74340d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 -- 172.21.15.46:0/3301874448 shutdown_connections 2024-04-26T23:49:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.958+0000 7f7433200640 1 -- 172.21.15.46:0/3301874448 wait complete. 2024-04-26T23:49:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.960+0000 7f7433200640 1 Processor -- start 2024-04-26T23:49:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.960+0000 7f7433200640 1 -- start start 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 0x7f743415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74341629c0 0x7f74340b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7434162da0 con 0x7f74340d25b0 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74340d1320 con 0x7f74341629c0 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7433200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7434162f20 con 0x7f74340c9b60 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7431400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7431400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58696/0 (socket says 172.21.15.46:58696) 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7431400640 1 -- 172.21.15.46:0/83542128 learned_addr learned my addr 172.21.15.46:0/83542128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:49.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.961+0000 7f7431e00640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 0x7f743415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7432800640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74341629c0 0x7f74340b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 -- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 msgr2=0x7f743415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 0x7f743415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 -- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74341629c0 msgr2=0x7f74340b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74341629c0 0x7f74340b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 -- 172.21.15.46:0/83542128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743414dcf0 con 0x7f74340d25b0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431e00640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 0x7f743415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f7431400640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f741c00cf30 tx=0x7f741c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c01a070 con 0x7f74340d25b0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c00d840 con 0x7f74340d25b0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.962+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c00dc10 con 0x7f74340d25b0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.963+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743414dfd0 con 0x7f74340d25b0 2024-04-26T23:49:49.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.963+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743414e590 con 0x7f74340d25b0 2024-04-26T23:49:49.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.964+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743408d980 con 0x7f74340d25b0 2024-04-26T23:49:49.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.965+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f741c01d070 con 0x7f74340d25b0 2024-04-26T23:49:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.966+0000 7f742a000640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 0x7f741006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.966+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f741c08c260 con 0x7f74340d25b0 2024-04-26T23:49:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.966+0000 7f7431e00640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 0x7f741006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.968+0000 7f7431e00640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 0x7f741006ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7424007fd0 tx=0x7f7424009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:49.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:49.971+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f743408d980 con 0x7f74340d25b0 2024-04-26T23:49:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.177+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f743414e790 con 0x7f74340d25b0 2024-04-26T23:49:50.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.182+0000 7f742a000640 1 -- 172.21.15.46:0/83542128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f743414e790 con 0x7f74340d25b0 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 msgr2=0x7f741006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 0x7f741006ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7424007fd0 tx=0x7f7424009040 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 msgr2=0x7f743414eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f741c00cf30 tx=0x7f741c00a7d0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 shutdown_connections 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f741006c920 0x7f741006ede0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74341629c0 0x7f74340b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74340d25b0 0x7f743414eab0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 --2- 172.21.15.46:0/83542128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74340c9b60 0x7f743415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 >> 172.21.15.46:0/83542128 conn(0x7f74340d4f70 msgr2=0x7f743406e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 shutdown_connections 2024-04-26T23:49:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.185+0000 7f7433200640 1 -- 172.21.15.46:0/83542128 wait complete. 2024-04-26T23:49:50.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-26T23:49:50.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.422+0000 7fb775a00640 1 Processor -- start 2024-04-26T23:49:50.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.422+0000 7fb775a00640 1 -- start start 2024-04-26T23:49:50.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.423+0000 7fb775a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.423+0000 7fb775a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 0x7fb77006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.423+0000 7fb775a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7700b43f0 con 0x7fb770144840 2024-04-26T23:49:50.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.423+0000 7fb775a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb7700b46d0 con 0x7fb7700b3e80 2024-04-26T23:49:50.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.423+0000 7fb775a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7700b49b0 con 0x7fb770142280 2024-04-26T23:49:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.424+0000 7fb76ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.424+0000 7fb76ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:55994/0 (socket says 172.21.15.46:55994) 2024-04-26T23:49:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.424+0000 7fb76ea00640 1 -- 172.21.15.46:0/2951612815 learned_addr learned my addr 172.21.15.46:0/2951612815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.424+0000 7fb76e000640 1 --2- 172.21.15.46:0/2951612815 >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 0x7fb77006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.425+0000 7fb76ea00640 1 -- 172.21.15.46:0/2951612815 >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 msgr2=0x7fb77006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:50.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.425+0000 7fb76ea00640 1 --2- 172.21.15.46:0/2951612815 >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 0x7fb77006ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.425+0000 7fb76ea00640 1 -- 172.21.15.46:0/2951612815 >> v1:172.21.15.142:6789/0 conn(0x7fb770144840 legacy=0x7fb770144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.425+0000 7fb76ea00640 1 -- 172.21.15.46:0/2951612815 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7700b4c90 con 0x7fb770142280 2024-04-26T23:49:50.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.425+0000 7fb76e000640 1 --2- 172.21.15.46:0/2951612815 >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 0x7fb77006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:49:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.427+0000 7fb76ea00640 1 --2- 172.21.15.46:0/2951612815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb758009870 tx=0x7fb75802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b428084c8fe4798 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.427+0000 7fb76d600640 1 -- 172.21.15.46:0/2951612815 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758037070 con 0x7fb770142280 2024-04-26T23:49:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.427+0000 7fb76d600640 1 -- 172.21.15.46:0/2951612815 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758032440 con 0x7fb770142280 2024-04-26T23:49:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.427+0000 7fb76d600640 1 -- 172.21.15.46:0/2951612815 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758032810 con 0x7fb770142280 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 -- 172.21.15.46:0/2951612815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 msgr2=0x7fb77006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 --2- 172.21.15.46:0/2951612815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb758009870 tx=0x7fb75802a250 comp rx=0 tx=0).stop 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 -- 172.21.15.46:0/2951612815 shutdown_connections 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 --2- 172.21.15.46:0/2951612815 >> v2:172.21.15.142:3300/0 conn(0x7fb7700b3e80 0x7fb77006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 --2- 172.21.15.46:0/2951612815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770142280 0x7fb77006e510 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 -- 172.21.15.46:0/2951612815 >> 172.21.15.46:0/2951612815 conn(0x7fb77006c530 msgr2=0x7fb77006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 -- 172.21.15.46:0/2951612815 shutdown_connections 2024-04-26T23:49:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.428+0000 7fb775a00640 1 -- 172.21.15.46:0/2951612815 wait complete. 2024-04-26T23:49:50.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.430+0000 7fb775a00640 1 Processor -- start 2024-04-26T23:49:50.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.430+0000 7fb775a00640 1 -- start start 2024-04-26T23:49:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7700b3e80 0x7fb77015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770144840 0x7fb77013e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb770142a20 con 0x7fb770142280 2024-04-26T23:49:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701428a0 con 0x7fb7700b3e80 2024-04-26T23:49:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.431+0000 7fb775a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7701453c0 con 0x7fb770144840 2024-04-26T23:49:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58706/0 (socket says 172.21.15.46:58706) 2024-04-26T23:49:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 -- 172.21.15.46:0/1683255124 learned_addr learned my addr 172.21.15.46:0/1683255124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76f400640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770144840 0x7fb77013e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76ea00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7700b3e80 0x7fb77015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 -- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770144840 msgr2=0x7fb77013e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770144840 0x7fb77013e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 -- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7700b3e80 msgr2=0x7fb77015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7700b3e80 0x7fb77015f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.432+0000 7fb76e000640 1 -- 172.21.15.46:0/1683255124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77015d6c0 con 0x7fb770142280 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb76e000640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fb76000ebe0 tx=0x7fb76000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760017070 con 0x7fb770142280 2024-04-26T23:49:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76000cc30 con 0x7fb770142280 2024-04-26T23:49:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760013470 con 0x7fb770142280 2024-04-26T23:49:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77015d9a0 con 0x7fb770142280 2024-04-26T23:49:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.433+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77015df60 con 0x7fb770142280 2024-04-26T23:49:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.434+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770069530 con 0x7fb770142280 2024-04-26T23:49:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.435+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb76001a070 con 0x7fb770142280 2024-04-26T23:49:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.436+0000 7fb766a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 0x7fb74c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.436+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7fb76008bd30 con 0x7fb770142280 2024-04-26T23:49:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.437+0000 7fb76ea00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 0x7fb74c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.438+0000 7fb76ea00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 0x7fb74c06ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb758034000 tx=0x7fb75802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:50.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.441+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb76005a230 con 0x7fb770142280 2024-04-26T23:49:50.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:50.720+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) -- 0x7fb770162970 con 0x7fb770142280 2024-04-26T23:49:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.725+0000 7fb766a00640 1 -- 172.21.15.46:0/1683255124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v584) ==== 144+0+0 (secure 0 0 0) 0x7fb760059b30 con 0x7fb770142280 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 msgr2=0x7fb74c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 0x7fb74c06ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb758034000 tx=0x7fb75802e000 comp rx=0 tx=0).stop 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 msgr2=0x7fb77015c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fb76000ebe0 tx=0x7fb76000c520 comp rx=0 tx=0).stop 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 shutdown_connections 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb74c06c920 0x7fb74c06ede0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb770144840 0x7fb77013e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb770142280 0x7fb77015c020 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 --2- 172.21.15.46:0/1683255124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7700b3e80 0x7fb77015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 >> 172.21.15.46:0/1683255124 conn(0x7fb77006c530 msgr2=0x7fb7700b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 shutdown_connections 2024-04-26T23:49:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.729+0000 7fb775a00640 1 -- 172.21.15.46:0/1683255124 wait complete. 2024-04-26T23:49:51.758 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-26T23:49:51.759 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-26T23:49:51.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:51.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.970+0000 7f212e800640 1 Processor -- start 2024-04-26T23:49:51.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.971+0000 7f212e800640 1 -- start start 2024-04-26T23:49:51.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.972+0000 7f212e800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2128073570 0x7f21280d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.972+0000 7f212e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212815c5c0 0x7f212815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.972+0000 7f212e800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21280cf950 con 0x7f2128145770 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.972+0000 7f212e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f21280d0c50 con 0x7f212815c5c0 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.972+0000 7f212e800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f212815cf90 con 0x7f2128073570 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212815c5c0 0x7f212815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212815c5c0 0x7f212815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35506/0 (socket says 172.21.15.46:35506) 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 -- 172.21.15.46:0/291910465 learned_addr learned my addr 172.21.15.46:0/291910465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:51.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2128145770 0x7f2128072e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39532/0 (socket says 172.21.15.46:39532) 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2126a00640 1 --2- 172.21.15.46:0/291910465 >> v2:172.21.15.142:3300/0 conn(0x7f2128073570 0x7f21280d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 -- 172.21.15.46:0/291910465 >> v2:172.21.15.142:3300/0 conn(0x7f2128073570 msgr2=0x7f21280d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 --2- 172.21.15.46:0/291910465 >> v2:172.21.15.142:3300/0 conn(0x7f2128073570 0x7f21280d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 -- 172.21.15.46:0/291910465 >> v1:172.21.15.46:6789/0 conn(0x7f2128145770 legacy=0x7f2128072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.973+0000 7f2127e00640 1 -- 172.21.15.46:0/291910465 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280cfc30 con 0x7f212815c5c0 2024-04-26T23:49:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.974+0000 7f2127e00640 1 --2- 172.21.15.46:0/291910465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212815c5c0 0x7f212815ca50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2128070030 tx=0x7f212002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd6571b4ea2459 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f2126000640 1 -- 172.21.15.46:0/291910465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120037070 con 0x7f212815c5c0 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f2126000640 1 -- 172.21.15.46:0/291910465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120032440 con 0x7f212815c5c0 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f2126000640 1 -- 172.21.15.46:0/291910465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120032810 con 0x7f212815c5c0 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 -- 172.21.15.46:0/291910465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212815c5c0 msgr2=0x7f212815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 --2- 172.21.15.46:0/291910465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212815c5c0 0x7f212815ca50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2128070030 tx=0x7f212002a270 comp rx=0 tx=0).stop 2024-04-26T23:49:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 -- 172.21.15.46:0/291910465 shutdown_connections 2024-04-26T23:49:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 --2- 172.21.15.46:0/291910465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212815c5c0 0x7f212815ca50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 --2- 172.21.15.46:0/291910465 >> v2:172.21.15.142:3300/0 conn(0x7f2128073570 0x7f21280d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 -- 172.21.15.46:0/291910465 >> 172.21.15.46:0/291910465 conn(0x7f212806c530 msgr2=0x7f212806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.975+0000 7f212e800640 1 -- 172.21.15.46:0/291910465 shutdown_connections 2024-04-26T23:49:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.976+0000 7f212e800640 1 -- 172.21.15.46:0/291910465 wait complete. 2024-04-26T23:49:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.977+0000 7f212e800640 1 Processor -- start 2024-04-26T23:49:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.977+0000 7f212e800640 1 -- start start 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 0x7f212813ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212815c5c0 0x7f2128139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21280d01f0 con 0x7f2128145770 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f212815d490 con 0x7f2128073570 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.978+0000 7f212e800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21280cf5b0 con 0x7f212815c5c0 2024-04-26T23:49:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35510/0 (socket says 172.21.15.46:35510) 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 -- 172.21.15.46:0/435556183 learned_addr learned my addr 172.21.15.46:0/435556183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2127e00640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212815c5c0 0x7f2128139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2127400640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 0x7f212813ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 -- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212815c5c0 msgr2=0x7f2128139aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212815c5c0 0x7f2128139aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 -- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 msgr2=0x7f212813ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 0x7f212813ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2126a00640 1 -- 172.21.15.46:0/435556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212813f1c0 con 0x7f2128145770 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.979+0000 7f2127400640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 0x7f212813ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f2126a00640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f211400c7f0 tx=0x7f211400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114015070 con 0x7f2128145770 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2114004220 con 0x7f2128145770 2024-04-26T23:49:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114007620 con 0x7f2128145770 2024-04-26T23:49:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21281f6e50 con 0x7f2128145770 2024-04-26T23:49:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.980+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21281f7410 con 0x7f2128145770 2024-04-26T23:49:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.981+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2128069530 con 0x7f2128145770 2024-04-26T23:49:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.982+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21140079d0 con 0x7f2128145770 2024-04-26T23:49:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.983+0000 7f211f400640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.983+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f211408bea0 con 0x7f2128145770 2024-04-26T23:49:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.984+0000 7f2127400640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.985+0000 7f2127400640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2118007fd0 tx=0x7f2118009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:51.988+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f211405a3a0 con 0x7f2128145770 2024-04-26T23:49:52.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.190+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2128069d70 con 0x7f2128145770 2024-04-26T23:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.192+0000 7f211f400640 1 -- 172.21.15.46:0/435556183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+13267 (secure 0 0 0) 0x7f2114059ca0 con 0x7f2128145770 2024-04-26T23:49:52.197 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:52.197 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":129,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:49:49.709507+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6836/475647056":"2024-04-27T23:49:46.690786+0000","172.21.15.142:6834/475647056":"2024-04-27T23:49:46.690786+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.194+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 msgr2=0x7f210806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.194+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2118007fd0 tx=0x7f2118009040 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.194+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 msgr2=0x7f21281f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.194+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f211400c7f0 tx=0x7f211400ccb0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 shutdown_connections 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212815c5c0 0x7f2128139aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2128145770 0x7f21281f7ef0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 --2- 172.21.15.46:0/435556183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128073570 0x7f212813ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 >> 172.21.15.46:0/435556183 conn(0x7f212806c530 msgr2=0x7f21280b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 shutdown_connections 2024-04-26T23:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.195+0000 7f212e800640 1 -- 172.21.15.46:0/435556183 wait complete. 2024-04-26T23:49:52.219 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:49:52.220 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:49:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.429+0000 7f22e1c00640 1 Processor -- start 2024-04-26T23:49:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.429+0000 7f22e1c00640 1 -- start start 2024-04-26T23:49:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f22dc15cec0 0x7f22dc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22dc0cdb30 0x7f22dc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc0ce7d0 con 0x7f22dc073b30 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0ceab0 con 0x7f22dc0cdb30 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0ced90 con 0x7f22dc15cec0 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22dc0cdb30 0x7f22dc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22dc073b30 0x7f22dc071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39538/0 (socket says 172.21.15.46:39538) 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22e1200640 1 -- 172.21.15.46:0/1764534978 learned_addr learned my addr 172.21.15.46:0/1764534978 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.431+0000 7f22dbe00640 1 --2- 172.21.15.46:0/1764534978 >> v2:172.21.15.142:3300/0 conn(0x7f22dc15cec0 0x7f22dc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.432+0000 7f22d3e00640 1 -- 172.21.15.46:0/1764534978 >> v2:172.21.15.142:3300/0 conn(0x7f22dc15cec0 msgr2=0x7f22dc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.432+0000 7f22d3e00640 1 --2- 172.21.15.46:0/1764534978 >> v2:172.21.15.142:3300/0 conn(0x7f22dc15cec0 0x7f22dc0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.432+0000 7f22db400640 1 -- 172.21.15.46:0/1764534978 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681019030 0 0) 0x7f22dc0ce7d0 con 0x7f22dc073b30 2024-04-26T23:49:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.432+0000 7f22d3e00640 1 -- 172.21.15.46:0/1764534978 >> v1:172.21.15.46:6789/0 conn(0x7f22dc073b30 legacy=0x7f22dc071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.432+0000 7f22d3e00640 1 -- 172.21.15.46:0/1764534978 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0cf070 con 0x7f22dc0cdb30 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.433+0000 7f22d3e00640 1 --2- 172.21.15.46:0/1764534978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc0735f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0b3830 tx=0x7f22c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7983a967818616 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.433+0000 7f22db400640 1 -- 172.21.15.46:0/1764534978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037070 con 0x7f22dc0cdb30 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.433+0000 7f22db400640 1 -- 172.21.15.46:0/1764534978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4032440 con 0x7f22dc0cdb30 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.433+0000 7f22db400640 1 -- 172.21.15.46:0/1764534978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4032810 con 0x7f22dc0cdb30 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 -- 172.21.15.46:0/1764534978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 msgr2=0x7f22dc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 --2- 172.21.15.46:0/1764534978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc0735f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0b3830 tx=0x7f22c402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 -- 172.21.15.46:0/1764534978 shutdown_connections 2024-04-26T23:49:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 --2- 172.21.15.46:0/1764534978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc0735f0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 --2- 172.21.15.46:0/1764534978 >> v2:172.21.15.142:3300/0 conn(0x7f22dc15cec0 0x7f22dc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 -- 172.21.15.46:0/1764534978 >> 172.21.15.46:0/1764534978 conn(0x7f22dc0b27b0 msgr2=0x7f22dc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 -- 172.21.15.46:0/1764534978 shutdown_connections 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.434+0000 7f22e1c00640 1 -- 172.21.15.46:0/1764534978 wait complete. 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.435+0000 7f22e1c00640 1 Processor -- start 2024-04-26T23:49:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.436+0000 7f22e1c00640 1 -- start start 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 0x7f22dc0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22dc15cec0 0x7f22dc15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc073f70 con 0x7f22dc0cdb30 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc15d2a0 con 0x7f22dc15cec0 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b4b50 con 0x7f22dc073b30 2024-04-26T23:49:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35530/0 (socket says 172.21.15.46:35530) 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22d3e00640 1 -- 172.21.15.46:0/664173531 learned_addr learned my addr 172.21.15.46:0/664173531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22dbe00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 0x7f22dc0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.437+0000 7f22e1200640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22dc15cec0 0x7f22dc15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 -- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 msgr2=0x7f22dc0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 0x7f22dc0b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 -- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22dc15cec0 msgr2=0x7f22dc15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22dc15cec0 0x7f22dc15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 -- 172.21.15.46:0/664173531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1f4e90 con 0x7f22dc0cdb30 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22dbe00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 0x7f22dc0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:49:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d3e00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f22c402a790 tx=0x7f22c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4040020 con 0x7f22dc0cdb30 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4038040 con 0x7f22dc0cdb30 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037070 con 0x7f22dc0cdb30 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.438+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1f5170 con 0x7f22dc0cdb30 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.439+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1f5730 con 0x7f22dc0cdb30 2024-04-26T23:49:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.440+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0693b0 con 0x7f22dc0cdb30 2024-04-26T23:49:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.441+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22c4043020 con 0x7f22dc0cdb30 2024-04-26T23:49:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.442+0000 7f22d8c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 0x7f22b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.442+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f22c40ac060 con 0x7f22dc0cdb30 2024-04-26T23:49:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.443+0000 7f22dbe00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 0x7f22b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.444+0000 7f22dbe00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 0x7f22b806ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.447+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22c404a030 con 0x7f22dc0cdb30 2024-04-26T23:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.650+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22dc0ca3d0 con 0x7f22dc0cdb30 2024-04-26T23:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.651+0000 7f22d8c00640 1 -- 172.21.15.46:0/664173531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) ==== 93+0+10 (secure 0 0 0) 0x7f22c4032e20 con 0x7f22dc0cdb30 2024-04-26T23:49:52.656 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 msgr2=0x7f22b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 0x7f22b806ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 msgr2=0x7f22dc1f8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f22c402a790 tx=0x7f22c40046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 shutdown_connections 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22b806c920 0x7f22b806ede0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22dc15cec0 0x7f22dc15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0cdb30 0x7f22dc1f8990 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 --2- 172.21.15.46:0/664173531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22dc073b30 0x7f22dc0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.653+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 >> 172.21.15.46:0/664173531 conn(0x7f22dc0b27b0 msgr2=0x7f22dc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.654+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 shutdown_connections 2024-04-26T23:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.654+0000 7f22e1c00640 1 -- 172.21.15.46:0/664173531 wait complete. 2024-04-26T23:49:52.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-26T23:49:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.890+0000 7fcad7e00640 1 Processor -- start 2024-04-26T23:49:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.890+0000 7fcad7e00640 1 -- start start 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcad80d1990 0x7fcad80d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcad80d0c10 0x7fcad80d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad80d22b0 con 0x7fcad81639c0 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80d0780 con 0x7fcad80d0c10 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80cfe50 con 0x7fcad80d1990 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.892+0000 7fcad6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcad80d0c10 0x7fcad80d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcad80d0c10 0x7fcad80d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35542/0 (socket says 172.21.15.46:35542) 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 -- 172.21.15.46:0/825951317 learned_addr learned my addr 172.21.15.46:0/825951317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 -- 172.21.15.46:0/825951317 >> v2:172.21.15.142:3300/0 conn(0x7fcad80d1990 msgr2=0x7fcad80d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 --2- 172.21.15.46:0/825951317 >> v2:172.21.15.142:3300/0 conn(0x7fcad80d1990 0x7fcad80d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 -- 172.21.15.46:0/825951317 >> v1:172.21.15.142:6789/0 conn(0x7fcad81639c0 legacy=0x7fcad80d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:52.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.893+0000 7fcad6000640 1 -- 172.21.15.46:0/825951317 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80d0130 con 0x7fcad80d0c10 2024-04-26T23:49:52.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.894+0000 7fcad6000640 1 --2- 172.21.15.46:0/825951317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d0c10 0x7fcad80d5690 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fcad808d560 tx=0x7fcac402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d042084afd582cca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.894+0000 7fcad5600640 1 -- 172.21.15.46:0/825951317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037070 con 0x7fcad80d0c10 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad5600640 1 -- 172.21.15.46:0/825951317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac4032440 con 0x7fcad80d0c10 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad5600640 1 -- 172.21.15.46:0/825951317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4032810 con 0x7fcad80d0c10 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 -- 172.21.15.46:0/825951317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d0c10 msgr2=0x7fcad80d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 --2- 172.21.15.46:0/825951317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d0c10 0x7fcad80d5690 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fcad808d560 tx=0x7fcac402a230 comp rx=0 tx=0).stop 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 -- 172.21.15.46:0/825951317 shutdown_connections 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 --2- 172.21.15.46:0/825951317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d0c10 0x7fcad80d5690 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 --2- 172.21.15.46:0/825951317 >> v2:172.21.15.142:3300/0 conn(0x7fcad80d1990 0x7fcad80d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 -- 172.21.15.46:0/825951317 >> 172.21.15.46:0/825951317 conn(0x7fcad809e2a0 msgr2=0x7fcad80d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:52.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 -- 172.21.15.46:0/825951317 shutdown_connections 2024-04-26T23:49:52.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.895+0000 7fcad7e00640 1 -- 172.21.15.46:0/825951317 wait complete. 2024-04-26T23:49:52.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.897+0000 7fcad7e00640 1 Processor -- start 2024-04-26T23:49:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.897+0000 7fcad7e00640 1 -- start start 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcad80d0c10 0x7fcad80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcad81639c0 0x7fcad81f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80cfa90 con 0x7fcad80d1990 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcad815f530 con 0x7fcad80d0c10 2024-04-26T23:49:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80cfc10 con 0x7fcad81639c0 2024-04-26T23:49:52.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.898+0000 7fcad6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35546/0 (socket says 172.21.15.46:35546) 2024-04-26T23:49:52.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 -- 172.21.15.46:0/2855076003 learned_addr learned my addr 172.21.15.46:0/2855076003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 -- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcad81639c0 msgr2=0x7fcad81f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcad81639c0 0x7fcad81f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 -- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcad80d0c10 msgr2=0x7fcad80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcad80d0c10 0x7fcad80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 -- 172.21.15.46:0/2855076003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80ca6b0 con 0x7fcad80d1990 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.899+0000 7fcad6000640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.900+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4041070 con 0x7fcad80d1990 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.900+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac403c870 con 0x7fcad80d1990 2024-04-26T23:49:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.900+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037040 con 0x7fcad80d1990 2024-04-26T23:49:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.900+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad80ceb10 con 0x7fcad80d1990 2024-04-26T23:49:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.900+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad80ca950 con 0x7fcad80d1990 2024-04-26T23:49:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.901+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad80693b0 con 0x7fcad80d1990 2024-04-26T23:49:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.902+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcac4044070 con 0x7fcad80d1990 2024-04-26T23:49:52.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.904+0000 7fcacea00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 0x7fcab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:52.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.904+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7fcac40ac560 con 0x7fcad80d1990 2024-04-26T23:49:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.904+0000 7fcad6a00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 0x7fcab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:52.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.905+0000 7fcad6a00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 0x7fcab806edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcad80cb840 tx=0x7fcac8006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:52.908+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcac407ab90 con 0x7fcad80d1990 2024-04-26T23:49:53.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.193+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) -- 0x7fcad80cab20 con 0x7fcad80d1990 2024-04-26T23:49:53.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.740+0000 7fcacea00640 1 -- 172.21.15.46:0/2855076003 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v586) ==== 158+0+0 (secure 0 0 0) 0x7fcac407a490 con 0x7fcad80d1990 2024-04-26T23:49:53.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 msgr2=0x7fcab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 0x7fcab806edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcad80cb840 tx=0x7fcac8006c90 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 msgr2=0x7fcad81f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 shutdown_connections 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcab806c8f0 0x7fcab806edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcad81639c0 0x7fcad81f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad80d1990 0x7fcad81f8a70 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 --2- 172.21.15.46:0/2855076003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcad80d0c10 0x7fcad80cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 >> 172.21.15.46:0/2855076003 conn(0x7fcad809e2a0 msgr2=0x7fcad806dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:53.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 shutdown_connections 2024-04-26T23:49:53.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.745+0000 7fcad7e00640 1 -- 172.21.15.46:0/2855076003 wait complete. 2024-04-26T23:49:53.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.988+0000 7f8836200640 1 Processor -- start 2024-04-26T23:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.988+0000 7f8836200640 1 -- start start 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8836200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f883015cec0 0x7f88300cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8836200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88300ce7d0 con 0x7f8830073d00 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8836200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88300ceab0 con 0x7f88300cdb30 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8836200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f88300ced90 con 0x7f883015cec0 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8834e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f883015cec0 0x7f88300cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8834e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f883015cec0 0x7f88300cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35554/0 (socket says 172.21.15.46:35554) 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8834e00640 1 -- 172.21.15.46:0/542594696 learned_addr learned my addr 172.21.15.46:0/542594696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.990+0000 7f8835800640 1 --1- 172.21.15.46:0/542594696 >> v1:172.21.15.46:6789/0 conn(0x7f8830073d00 0x7f88300b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39546/0 (socket says 172.21.15.46:39546) 2024-04-26T23:49:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.991+0000 7f8834e00640 1 -- 172.21.15.46:0/542594696 >> v1:172.21.15.142:6789/0 conn(0x7f88300cdb30 legacy=0x7f88300735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.991+0000 7f8834e00640 1 -- 172.21.15.46:0/542594696 >> v1:172.21.15.46:6789/0 conn(0x7f8830073d00 legacy=0x7f88300b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.991+0000 7f8834e00640 1 -- 172.21.15.46:0/542594696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300cf070 con 0x7f883015cec0 2024-04-26T23:49:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.991+0000 7f882fe00640 1 -- 172.21.15.46:0/542594696 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647580265 0 0) 0x7f88300ce7d0 con 0x7f8830073d00 2024-04-26T23:49:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.992+0000 7f8834e00640 1 --2- 172.21.15.46:0/542594696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300cd5f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355d5cfe339418d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.992+0000 7f882fe00640 1 -- 172.21.15.46:0/542594696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820037070 con 0x7f883015cec0 2024-04-26T23:49:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.992+0000 7f882fe00640 1 -- 172.21.15.46:0/542594696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8820032440 con 0x7f883015cec0 2024-04-26T23:49:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.992+0000 7f882fe00640 1 -- 172.21.15.46:0/542594696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820032810 con 0x7f883015cec0 2024-04-26T23:49:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 -- 172.21.15.46:0/542594696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 msgr2=0x7f88300cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 --2- 172.21.15.46:0/542594696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300cd5f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 -- 172.21.15.46:0/542594696 shutdown_connections 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 --2- 172.21.15.46:0/542594696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300cd5f0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 -- 172.21.15.46:0/542594696 >> 172.21.15.46:0/542594696 conn(0x7f88300b27b0 msgr2=0x7f88300b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 -- 172.21.15.46:0/542594696 shutdown_connections 2024-04-26T23:49:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.993+0000 7f8836200640 1 -- 172.21.15.46:0/542594696 wait complete. 2024-04-26T23:49:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.995+0000 7f8836200640 1 Processor -- start 2024-04-26T23:49:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.995+0000 7f8836200640 1 -- start start 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8830073d00 0x7f8830157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 0x7f88301589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88300b4250 con 0x7f883015cec0 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f883015d2a0 con 0x7f88300cdb30 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8836200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88300b43d0 con 0x7f8830073d00 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8835800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8835800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35556/0 (socket says 172.21.15.46:35556) 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8835800640 1 -- 172.21.15.46:0/2643353136 learned_addr learned my addr 172.21.15.46:0/2643353136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8834e00640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8830073d00 0x7f8830157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.996+0000 7f8827e00640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 0x7f88301589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 -- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8830073d00 msgr2=0x7f8830157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8830073d00 0x7f8830157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 -- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 msgr2=0x7f88301589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 0x7f88301589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 -- 172.21.15.46:0/2643353136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8830159be0 con 0x7f883015cec0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8827e00640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 0x7f88301589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8835800640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f882800c800 tx=0x7f882800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828018070 con 0x7f883015cec0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8828004d80 con 0x7f883015cec0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800a880 con 0x7f883015cec0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8830159340 con 0x7f883015cec0 2024-04-26T23:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.997+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8830159900 con 0x7f883015cec0 2024-04-26T23:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:53.999+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300693b0 con 0x7f883015cec0 2024-04-26T23:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.000+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f882800aa20 con 0x7f883015cec0 2024-04-26T23:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.001+0000 7f882d600640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 0x7f880c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.001+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f882808afc0 con 0x7f883015cec0 2024-04-26T23:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.001+0000 7f8834e00640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 0x7f880c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.003+0000 7f8834e00640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 0x7f880c06edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8820034000 tx=0x7f882002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.006+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8828059620 con 0x7f883015cec0 2024-04-26T23:49:54.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.212+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f883015c430 con 0x7f883015cec0 2024-04-26T23:49:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.219+0000 7f882d600640 1 -- 172.21.15.46:0/2643353136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f8828058f20 con 0x7f883015cec0 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 msgr2=0x7f880c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 0x7f880c06edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8820034000 tx=0x7f882002e000 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 msgr2=0x7f88300c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f882800c800 tx=0x7f882800ccc0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 shutdown_connections 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880c06c8f0 0x7f880c06edb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883015cec0 0x7f88300c5c90 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88300cdb30 0x7f88301589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 --2- 172.21.15.46:0/2643353136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8830073d00 0x7f8830157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 >> 172.21.15.46:0/2643353136 conn(0x7f88300b27b0 msgr2=0x7f8830159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 shutdown_connections 2024-04-26T23:49:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.222+0000 7f8836200640 1 -- 172.21.15.46:0/2643353136 wait complete. 2024-04-26T23:49:54.246 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:49:54.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:49:54.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:49:54.453 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:49:54.453 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:49:54.453 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:54.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:49:54.515 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:49:54.515 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:54.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:49:54.572 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:49:54.572 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:54.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:49:54.634 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:49:54.634 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:49:54.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:49:54.691 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:49:54.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:49:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.917+0000 7f597e200640 1 Processor -- start 2024-04-26T23:49:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.917+0000 7f597e200640 1 -- start start 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f597e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59780b0960 0x7f597815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f597e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59780b0df0 con 0x7f5978154130 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f597e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597806d1b0 con 0x7f5978072f80 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f597e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f597815e4e0 con 0x7f59780b0960 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f5977400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5978154130 0x7f5978072870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39562/0 (socket says 172.21.15.46:39562) 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f5977400640 1 -- 172.21.15.46:0/1420252842 learned_addr learned my addr 172.21.15.46:0/1420252842 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.919+0000 7f5977e00640 1 --2- 172.21.15.46:0/1420252842 >> v2:172.21.15.46:3300/0 conn(0x7f59780b0960 0x7f597815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.920+0000 7f5976000640 1 -- 172.21.15.46:0/1420252842 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027168722 0 0) 0x7f59780b0df0 con 0x7f5978154130 2024-04-26T23:49:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.920+0000 7f5976000640 1 -- 172.21.15.46:0/1420252842 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f595c003520 con 0x7f5978154130 2024-04-26T23:49:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.920+0000 7f5977e00640 1 -- 172.21.15.46:0/1420252842 >> v1:172.21.15.142:6789/0 conn(0x7f5978072f80 legacy=0x7f597806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.920+0000 7f5977e00640 1 -- 172.21.15.46:0/1420252842 >> v1:172.21.15.46:6789/0 conn(0x7f5978154130 legacy=0x7f5978072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.920+0000 7f5977e00640 1 -- 172.21.15.46:0/1420252842 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59780b1060 con 0x7f59780b0960 2024-04-26T23:49:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.921+0000 7f5977e00640 1 --2- 172.21.15.46:0/1420252842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780b0960 0x7f597815dfa0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f5970009d90 tx=0x7f597002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528686e8b1ed53f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.921+0000 7f5976000640 1 -- 172.21.15.46:0/1420252842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970031600 con 0x7f59780b0960 2024-04-26T23:49:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.921+0000 7f5976000640 1 -- 172.21.15.46:0/1420252842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59700317a0 con 0x7f59780b0960 2024-04-26T23:49:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.921+0000 7f5976000640 1 -- 172.21.15.46:0/1420252842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970031b70 con 0x7f59780b0960 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 -- 172.21.15.46:0/1420252842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780b0960 msgr2=0x7f597815dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 --2- 172.21.15.46:0/1420252842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780b0960 0x7f597815dfa0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f5970009d90 tx=0x7f597002a610 comp rx=0 tx=0).stop 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 -- 172.21.15.46:0/1420252842 shutdown_connections 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 --2- 172.21.15.46:0/1420252842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780b0960 0x7f597815dfa0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 -- 172.21.15.46:0/1420252842 >> 172.21.15.46:0/1420252842 conn(0x7f59780b3560 msgr2=0x7f59780b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 -- 172.21.15.46:0/1420252842 shutdown_connections 2024-04-26T23:49:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.922+0000 7f597e200640 1 -- 172.21.15.46:0/1420252842 wait complete. 2024-04-26T23:49:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.924+0000 7f597e200640 1 Processor -- start 2024-04-26T23:49:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.924+0000 7f597e200640 1 -- start start 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5978072f80 0x7f59781545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59780b0960 0x7f5978143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f597806c9b0 con 0x7f5978154130 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f597806c6d0 con 0x7f5978072f80 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f597e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f597806cc90 con 0x7f59780b0960 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f5977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f5977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35570/0 (socket says 172.21.15.46:35570) 2024-04-26T23:49:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f5977e00640 1 -- 172.21.15.46:0/1646284542 learned_addr learned my addr 172.21.15.46:0/1646284542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.925+0000 7f5977400640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5978072f80 0x7f59781545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5976a00640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59780b0960 0x7f5978143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 -- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59780b0960 msgr2=0x7f5978143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59780b0960 0x7f5978143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 -- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5978072f80 msgr2=0x7f59781545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5978072f80 0x7f59781545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 -- 172.21.15.46:0/1646284542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59780b4e60 con 0x7f5978154130 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f5977e00640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5970009d90 tx=0x7f5970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970040070 con 0x7f5978154130 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5970030b60 con 0x7f5978154130 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970009450 con 0x7f5978154130 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.926+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59780b5140 con 0x7f5978154130 2024-04-26T23:49:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.927+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59780b5700 con 0x7f5978154130 2024-04-26T23:49:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.928+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978069530 con 0x7f5978154130 2024-04-26T23:49:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.929+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5970039040 con 0x7f5978154130 2024-04-26T23:49:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.930+0000 7f596f400640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 0x7f595806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.930+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f59700acd70 con 0x7f5978154130 2024-04-26T23:49:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.931+0000 7f5977400640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 0x7f595806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.932+0000 7f5977400640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 0x7f595806edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5968005880 tx=0x7f5968009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:54.935+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f597007b270 con 0x7f5978154130 2024-04-26T23:49:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.173+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597806d430 con 0x7f5978154130 2024-04-26T23:49:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.174+0000 7f596f400640 1 -- 172.21.15.46:0/1646284542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+3877 (secure 0 0 0) 0x7f597007ab70 con 0x7f5978154130 2024-04-26T23:49:55.182 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:55.182 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":102,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3377520785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3377520785},{"type":"v1","addr":"172.21.15.46:6835","nonce":3377520785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1855123830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1855123830},{"type":"v1","addr":"172.21.15.142:6837","nonce":1855123830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1275191715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1275191715},{"type":"v1","addr":"172.21.15.142:6836","nonce":1275191715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3306507633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3306507633},{"type":"v1","addr":"172.21.15.46:6834","nonce":3306507633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 102 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.176+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 msgr2=0x7f595806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.176+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 0x7f595806edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5968005880 tx=0x7f5968009040 comp rx=0 tx=0).stop 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.176+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 msgr2=0x7f59780b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.176+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5970009d90 tx=0x7f5970004a40 comp rx=0 tx=0).stop 2024-04-26T23:49:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 shutdown_connections 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f595806c8f0 0x7f595806edb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5978154130 0x7f59780b5f00 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59780b0960 0x7f5978143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 --2- 172.21.15.46:0/1646284542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5978072f80 0x7f59781545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 >> 172.21.15.46:0/1646284542 conn(0x7f59780b3560 msgr2=0x7f5978072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 shutdown_connections 2024-04-26T23:49:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.177+0000 7f597e200640 1 -- 172.21.15.46:0/1646284542 wait complete. 2024-04-26T23:49:55.201 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:49:55.201 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:49:55.202 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:49:55.202 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:49:55.202 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:49:55.202 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:49:55.202 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:49:55.202 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:49:55.202 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:49:55.202 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:49:55.203 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:49:55.203 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:49:55.203 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:49:55.203 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:49:55.203 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:49:55.203 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:49:55.216 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:49:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:49:55.226 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:49:55.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:49:55.227 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:49:55.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:49:55.228 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:49:55.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:49:55.274 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:49:55.274 DEBUG:teuthology.parallel:result is None 2024-04-26T23:49:55.275 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:49:55.276 DEBUG:teuthology.parallel:result is None 2024-04-26T23:49:55.280 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:49:55.280 DEBUG:teuthology.parallel:result is None 2024-04-26T23:49:55.282 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:49:55.283 DEBUG:teuthology.parallel:result is None 2024-04-26T23:49:55.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:49:55.421 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:49:55.428 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:49:55.428 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:49:55.430 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:49:55.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.454+0000 7f5239600640 1 Processor -- start 2024-04-26T23:49:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.454+0000 7f5239600640 1 -- start start 2024-04-26T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f5239600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f523406e510 0x7f523406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f5239600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5234160bd0 0x7f5234161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f5239600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523406ee30 con 0x7f523415d540 2024-04-26T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f5239600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f52340742d0 con 0x7f5234160bd0 2024-04-26T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f5239600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5234161940 con 0x7f523406e510 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f522be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5234160bd0 0x7f5234161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f522be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5234160bd0 0x7f5234161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35604/0 (socket says 172.21.15.46:35604) 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.455+0000 7f522be00640 1 -- 172.21.15.46:0/1592698404 learned_addr learned my addr 172.21.15.46:0/1592698404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.456+0000 7f5232a00640 1 --2- 172.21.15.46:0/1592698404 >> v2:172.21.15.142:3300/0 conn(0x7f523406e510 0x7f523406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.456+0000 7f522be00640 1 -- 172.21.15.46:0/1592698404 >> v2:172.21.15.142:3300/0 conn(0x7f523406e510 msgr2=0x7f523406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.456+0000 7f522be00640 1 --2- 172.21.15.46:0/1592698404 >> v2:172.21.15.142:3300/0 conn(0x7f523406e510 0x7f523406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.456+0000 7f522be00640 1 -- 172.21.15.46:0/1592698404 >> v1:172.21.15.142:6789/0 conn(0x7f523415d540 legacy=0x7f523415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.456+0000 7f522be00640 1 -- 172.21.15.46:0/1592698404 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234071f80 con 0x7f5234160bd0 2024-04-26T23:49:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.457+0000 7f522be00640 1 --2- 172.21.15.46:0/1592698404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f5234161060 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f521c009870 tx=0x7f521c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48bd807c00dc77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.457+0000 7f5232000640 1 -- 172.21.15.46:0/1592698404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c037070 con 0x7f5234160bd0 2024-04-26T23:49:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.457+0000 7f5232000640 1 -- 172.21.15.46:0/1592698404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c032440 con 0x7f5234160bd0 2024-04-26T23:49:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.457+0000 7f5232000640 1 -- 172.21.15.46:0/1592698404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c032810 con 0x7f5234160bd0 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 -- 172.21.15.46:0/1592698404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 msgr2=0x7f5234161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 --2- 172.21.15.46:0/1592698404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f5234161060 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f521c009870 tx=0x7f521c02a2a0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 -- 172.21.15.46:0/1592698404 shutdown_connections 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 --2- 172.21.15.46:0/1592698404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f5234161060 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 --2- 172.21.15.46:0/1592698404 >> v2:172.21.15.142:3300/0 conn(0x7f523406e510 0x7f523406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 -- 172.21.15.46:0/1592698404 >> 172.21.15.46:0/1592698404 conn(0x7f523406c530 msgr2=0x7f523406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 -- 172.21.15.46:0/1592698404 shutdown_connections 2024-04-26T23:49:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.458+0000 7f5239600640 1 -- 172.21.15.46:0/1592698404 wait complete. 2024-04-26T23:49:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.460+0000 7f5239600640 1 Processor -- start 2024-04-26T23:49:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.460+0000 7f5239600640 1 -- start start 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523406e510 0x7f5234151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f523415d540 0x7f523414fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5234072820 con 0x7f5234160bd0 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5234072540 con 0x7f523406e510 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5239600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f52341615a0 con 0x7f523415d540 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5232a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523406e510 0x7f5234151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35608/0 (socket says 172.21.15.46:35608) 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 -- 172.21.15.46:0/2057997751 learned_addr learned my addr 172.21.15.46:0/2057997751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f522be00640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f523415d540 0x7f523414fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 -- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f523415d540 msgr2=0x7f523414fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f523415d540 0x7f523414fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 -- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523406e510 msgr2=0x7f5234151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523406e510 0x7f5234151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.461+0000 7f5233400640 1 -- 172.21.15.46:0/2057997751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234151fe0 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f5233400640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c7a0 tx=0x7f522c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c003b70 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c003d30 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00aa90 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5234150730 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.462+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5234150ce0 con 0x7f5234160bd0 2024-04-26T23:49:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.464+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234069530 con 0x7f5234160bd0 2024-04-26T23:49:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.465+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f522c00ac30 con 0x7f5234160bd0 2024-04-26T23:49:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.466+0000 7f522aa00640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.466+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5335+0+0 (secure 0 0 0) 0x7f522c08b760 con 0x7f5234160bd0 2024-04-26T23:49:55.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.466+0000 7f5232a00640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:55.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.467+0000 7f5232a00640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 0x7f521006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5224009b40 tx=0x7f5224009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.470+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f522c059740 con 0x7f5234160bd0 2024-04-26T23:49:55.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.679+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f52340cf020 con 0x7f5234160bd0 2024-04-26T23:49:55.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.772+0000 7f522aa00640 1 -- 172.21.15.46:0/2057997751 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) ==== 97+0+0 (secure 0 0 0) 0x7f522c05da40 con 0x7f5234160bd0 2024-04-26T23:49:55.778 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:49:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 msgr2=0x7f521006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 0x7f521006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5224009b40 tx=0x7f5224009190 comp rx=0 tx=0).stop 2024-04-26T23:49:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 msgr2=0x7f52341501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c7a0 tx=0x7f522c00cc60 comp rx=0 tx=0).stop 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 shutdown_connections 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5234160bd0 0x7f52341501f0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f523415d540 0x7f523414fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 --2- 172.21.15.46:0/2057997751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523406e510 0x7f5234151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.776+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 >> 172.21.15.46:0/2057997751 conn(0x7f523406c530 msgr2=0x7f523406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.777+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 shutdown_connections 2024-04-26T23:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.778+0000 7f5239600640 1 -- 172.21.15.46:0/2057997751 wait complete. 2024-04-26T23:49:55.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:49:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.999+0000 7f2515800640 1 Processor -- start 2024-04-26T23:49:56.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:55.999+0000 7f2515800640 1 -- start start 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.000+0000 7f2515800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25100d1990 0x7f25100d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.000+0000 7f2515800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25101639c0 0x7f25100b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2515800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25100d22b0 con 0x7f25100d0c10 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2515800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f25100d0780 con 0x7f25101639c0 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2515800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25100d5da0 con 0x7f25100d1990 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25101639c0 0x7f25100b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25101639c0 0x7f25100b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35618/0 (socket says 172.21.15.46:35618) 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 -- 172.21.15.46:0/1588604438 learned_addr learned my addr 172.21.15.46:0/1588604438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:56.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f250fe00640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25100d1990 0x7f25100d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 -- 172.21.15.46:0/1588604438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25100d1990 msgr2=0x7f25100d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25100d1990 0x7f25100d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.001+0000 7f2514e00640 1 -- 172.21.15.46:0/1588604438 >> v1:172.21.15.142:6789/0 conn(0x7f25100d0c10 legacy=0x7f25100d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.002+0000 7f2514e00640 1 -- 172.21.15.46:0/1588604438 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25100cfe50 con 0x7f25101639c0 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f2514e00640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25101639c0 0x7f25100b7dd0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2508009870 tx=0x7f250802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19031b86d1fc6555 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f250ea00640 1 -- 172.21.15.46:0/1588604438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508037070 con 0x7f25101639c0 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f250ea00640 1 -- 172.21.15.46:0/1588604438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508032440 con 0x7f25101639c0 2024-04-26T23:49:56.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f250ea00640 1 -- 172.21.15.46:0/1588604438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508032810 con 0x7f25101639c0 2024-04-26T23:49:56.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f2515800640 1 -- 172.21.15.46:0/1588604438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25101639c0 msgr2=0x7f25100b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.003+0000 7f2515800640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25101639c0 0x7f25100b7dd0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2508009870 tx=0x7f250802a210 comp rx=0 tx=0).stop 2024-04-26T23:49:56.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 -- 172.21.15.46:0/1588604438 shutdown_connections 2024-04-26T23:49:56.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25101639c0 0x7f25100b7dd0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 --2- 172.21.15.46:0/1588604438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25100d1990 0x7f25100d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 -- 172.21.15.46:0/1588604438 >> 172.21.15.46:0/1588604438 conn(0x7f251009e2a0 msgr2=0x7f25100d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 -- 172.21.15.46:0/1588604438 shutdown_connections 2024-04-26T23:49:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.004+0000 7f2515800640 1 -- 172.21.15.46:0/1588604438 wait complete. 2024-04-26T23:49:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.006+0000 7f2515800640 1 Processor -- start 2024-04-26T23:49:56.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.006+0000 7f2515800640 1 -- start start 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25100d1990 0x7f25100ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25101639c0 0x7f25100ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f251015f530 con 0x7f25100d0c10 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25100d0410 con 0x7f25100d1990 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f2515800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25100cfa90 con 0x7f25101639c0 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f250fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f250fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35626/0 (socket says 172.21.15.46:35626) 2024-04-26T23:49:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.007+0000 7f250fe00640 1 -- 172.21.15.46:0/1940876701 learned_addr learned my addr 172.21.15.46:0/1940876701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:56.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250f400640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25100d1990 0x7f25100ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 -- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25101639c0 msgr2=0x7f25100ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25101639c0 0x7f25100ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 -- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25100d1990 msgr2=0x7f25100ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25100d1990 0x7f25100ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 -- 172.21.15.46:0/1940876701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25100b8650 con 0x7f25100d0c10 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f250fe00640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f24f800d800 tx=0x7f24f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800b6c0 con 0x7f25100d0c10 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f8019070 con 0x7f25100d0c10 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800ba80 con 0x7f25100d0c10 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25100b8930 con 0x7f25100d0c10 2024-04-26T23:49:56.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.008+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25100b8ef0 con 0x7f25100d0c10 2024-04-26T23:49:56.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.010+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25100693b0 con 0x7f25100d0c10 2024-04-26T23:49:56.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.011+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24f800bc20 con 0x7f25100d0c10 2024-04-26T23:49:56.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.012+0000 7f2507e00640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 0x7f24ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.012+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5249+0+0 (secure 0 0 0) 0x7f24f808bf00 con 0x7f25100d0c10 2024-04-26T23:49:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.013+0000 7f250f400640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 0x7f24ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.014+0000 7f250f400640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 0x7f24ec06ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2500007fd0 tx=0x7f2500009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.017+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24f805a450 con 0x7f25100d0c10 2024-04-26T23:49:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.300+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f25100d1770 con 0x7f25100d0c10 2024-04-26T23:49:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.301+0000 7f2507e00640 1 -- 172.21.15.46:0/1940876701 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f24f8059d50 con 0x7f25100d0c10 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:49:56.306 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:49:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:49:56.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:49:56.309 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:49:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.314 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:49:56.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:49:56.317 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:49:56.318 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 msgr2=0x7f24ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 0x7f24ec06ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2500007fd0 tx=0x7f2500009040 comp rx=0 tx=0).stop 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 msgr2=0x7f2510156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f24f800d800 tx=0x7f24f800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 shutdown_connections 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f24ec06c920 0x7f24ec06ede0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25101639c0 0x7f25100ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25100d1990 0x7f25100ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 --2- 172.21.15.46:0/1940876701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25100d0c10 0x7f2510156b50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 >> 172.21.15.46:0/1940876701 conn(0x7f251009e2a0 msgr2=0x7f2510160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 shutdown_connections 2024-04-26T23:49:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.304+0000 7f2515800640 1 -- 172.21.15.46:0/1940876701 wait complete. 2024-04-26T23:49:56.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:49:56.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.520+0000 7fc076c00640 1 Processor -- start 2024-04-26T23:49:56.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.520+0000 7fc076c00640 1 -- start start 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0700cc690 0x7fc0700cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc070072bb0 con 0x7fc0700c6a50 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc070072e90 con 0x7fc0701454a0 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc070073170 con 0x7fc0700cc690 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc074e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0700cc690 0x7fc0700cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0700c6a50 0x7fc0700cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39606/0 (socket says 172.21.15.46:39606) 2024-04-26T23:49:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.522+0000 7fc076200640 1 -- 172.21.15.46:0/356990026 learned_addr learned my addr 172.21.15.46:0/356990026 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:56.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.523+0000 7fc074e00640 1 -- 172.21.15.46:0/356990026 >> v1:172.21.15.142:6789/0 conn(0x7fc0701454a0 legacy=0x7fc070073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.523+0000 7fc074e00640 1 -- 172.21.15.46:0/356990026 >> v1:172.21.15.46:6789/0 conn(0x7fc0700c6a50 legacy=0x7fc0700cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.523+0000 7fc06fe00640 1 -- 172.21.15.46:0/356990026 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909247203 0 0) 0x7fc070072bb0 con 0x7fc0700c6a50 2024-04-26T23:49:56.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.523+0000 7fc074e00640 1 -- 172.21.15.46:0/356990026 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc070073450 con 0x7fc0700cc690 2024-04-26T23:49:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.524+0000 7fc074e00640 1 --2- 172.21.15.46:0/356990026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700cc690 0x7fc0700cdc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0097a0 tx=0x7fc05c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385bde32cd4310da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.525+0000 7fc06fe00640 1 -- 172.21.15.46:0/356990026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c037070 con 0x7fc0700cc690 2024-04-26T23:49:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.525+0000 7fc06fe00640 1 -- 172.21.15.46:0/356990026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05c032440 con 0x7fc0700cc690 2024-04-26T23:49:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.525+0000 7fc06fe00640 1 -- 172.21.15.46:0/356990026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c032810 con 0x7fc0700cc690 2024-04-26T23:49:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.525+0000 7fc076c00640 1 -- 172.21.15.46:0/356990026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700cc690 msgr2=0x7fc0700cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.525+0000 7fc076c00640 1 --2- 172.21.15.46:0/356990026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700cc690 0x7fc0700cdc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0097a0 tx=0x7fc05c02a230 comp rx=0 tx=0).stop 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.526+0000 7fc076c00640 1 -- 172.21.15.46:0/356990026 shutdown_connections 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.526+0000 7fc076c00640 1 --2- 172.21.15.46:0/356990026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700cc690 0x7fc0700cdc30 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.526+0000 7fc076c00640 1 -- 172.21.15.46:0/356990026 >> 172.21.15.46:0/356990026 conn(0x7fc0700b26d0 msgr2=0x7fc0700b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.526+0000 7fc076c00640 1 -- 172.21.15.46:0/356990026 shutdown_connections 2024-04-26T23:49:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.526+0000 7fc076c00640 1 -- 172.21.15.46:0/356990026 wait complete. 2024-04-26T23:49:56.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.527+0000 7fc076c00640 1 Processor -- start 2024-04-26T23:49:56.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.527+0000 7fc076c00640 1 -- start start 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 0x7fc0700af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0701454a0 0x7fc07006e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc070145880 con 0x7fc0700c6a50 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc070072880 con 0x7fc0700cc690 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.528+0000 7fc076c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc070145a00 con 0x7fc0701454a0 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35636/0 (socket says 172.21.15.46:35636) 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 -- 172.21.15.46:0/3851026807 learned_addr learned my addr 172.21.15.46:0/3851026807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc076200640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0701454a0 0x7fc07006e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc074e00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 0x7fc0700af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 -- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0701454a0 msgr2=0x7fc07006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0701454a0 0x7fc07006e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 -- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 msgr2=0x7fc0700af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 0x7fc0700af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 -- 172.21.15.46:0/3851026807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07015bbb0 con 0x7fc0700c6a50 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc074e00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 0x7fc0700af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.529+0000 7fc075800640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c430 tx=0x7fc06400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.530+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064017070 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.530+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0640052d0 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.530+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064003830 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.530+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0700aff40 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.530+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0700b3b00 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.531+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0700693b0 con 0x7fc0700c6a50 2024-04-26T23:49:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.532+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc06401a070 con 0x7fc0700c6a50 2024-04-26T23:49:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.533+0000 7fc06d600640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.533+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5249+0+0 (secure 0 0 0) 0x7fc06408ad00 con 0x7fc0700c6a50 2024-04-26T23:49:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.534+0000 7fc074e00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.535+0000 7fc074e00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc05c034000 tx=0x7fc05c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.537+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0640592e0 con 0x7fc0700c6a50 2024-04-26T23:49:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.732+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0700b0c60 con 0x7fc0700c6a50 2024-04-26T23:49:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.733+0000 7fc06d600640 1 -- 172.21.15.46:0/3851026807 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+13138 (secure 0 0 0) 0x7fc064058be0 con 0x7fc0700c6a50 2024-04-26T23:49:56.738 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:56.738 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":133,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:49:55.786003+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 msgr2=0x7fc05006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc05c034000 tx=0x7fc05c02e000 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 msgr2=0x7fc0700af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c430 tx=0x7fc06400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 shutdown_connections 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0701454a0 0x7fc07006e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0700cc690 0x7fc0700af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 --2- 172.21.15.46:0/3851026807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0700c6a50 0x7fc0700af2a0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 >> 172.21.15.46:0/3851026807 conn(0x7fc0700b26d0 msgr2=0x7fc07015f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 shutdown_connections 2024-04-26T23:49:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.736+0000 7fc076c00640 1 -- 172.21.15.46:0/3851026807 wait complete. 2024-04-26T23:49:56.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:49:56.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:49:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.956+0000 7f7996400640 1 Processor -- start 2024-04-26T23:49:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.956+0000 7f7996400640 1 -- start start 2024-04-26T23:49:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7996400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79900cdd00 0x7f79900735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7996400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79900ce7d0 con 0x7f7990073b30 2024-04-26T23:49:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7996400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79900ceab0 con 0x7f799015cec0 2024-04-26T23:49:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7996400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79900ced90 con 0x7f79900cdd00 2024-04-26T23:49:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7995a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7990073b30 0x7f79900b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39610/0 (socket says 172.21.15.46:39610) 2024-04-26T23:49:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f7995a00640 1 -- 172.21.15.46:0/800286826 learned_addr learned my addr 172.21.15.46:0/800286826 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.958+0000 7f798fe00640 1 --2- 172.21.15.46:0/800286826 >> v2:172.21.15.142:3300/0 conn(0x7f79900cdd00 0x7f79900735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001297439 0 0) 0x7f79900ce7d0 con 0x7f7990073b30 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7978003520 con 0x7f7990073b30 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1907688330 0 0) 0x7f7978003520 con 0x7f7990073b30 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 >> v2:172.21.15.142:3300/0 conn(0x7f79900cdd00 msgr2=0x7f79900735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 --2- 172.21.15.46:0/800286826 >> v2:172.21.15.142:3300/0 conn(0x7f79900cdd00 0x7f79900735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 >> v1:172.21.15.142:6789/0 conn(0x7f799015cec0 legacy=0x7f79900cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900cf070 con 0x7f7990073b30 2024-04-26T23:49:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f79880019f0 con 0x7f7990073b30 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 >> v1:172.21.15.46:6789/0 conn(0x7f7990073b30 legacy=0x7f79900b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 0x7f7978003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978004800 0x7f7978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79780051c0 0x7f797800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79900ced90 con 0x7f7978003a00 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f797800c0c0 con 0x7f7978004800 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.959+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f797800c2c0 con 0x7f79780051c0 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 0x7f7978003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f798fe00640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978004800 0x7f7978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995a00640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79780051c0 0x7f797800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 -- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79780051c0 msgr2=0x7f797800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79780051c0 0x7f797800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 -- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978004800 msgr2=0x7f7978004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978004800 0x7f7978004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995000640 1 -- 172.21.15.46:0/800286826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7978003520 con 0x7f7978003a00 2024-04-26T23:49:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.960+0000 7f7995a00640 1 -- 172.21.15.46:0/800286826 reap_dead start 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.961+0000 7f7995000640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 0x7f7978003e20 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7984001750 tx=0x7f7978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9212c0db521e7354 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.961+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984001370 con 0x7f7978003a00 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79840052d0 con 0x7f7978003a00 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f798f400640 1 -- 172.21.15.46:0/800286826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798402abe0 con 0x7f7978003a00 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 -- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 msgr2=0x7f7978003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 0x7f7978003e20 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7984001750 tx=0x7f7978005830 comp rx=0 tx=0).stop 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 -- 172.21.15.46:0/800286826 shutdown_connections 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 --2- 172.21.15.46:0/800286826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7978003a00 0x7f7978003e20 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 -- 172.21.15.46:0/800286826 >> 172.21.15.46:0/800286826 conn(0x7f79900b27b0 msgr2=0x7f79900b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 -- 172.21.15.46:0/800286826 shutdown_connections 2024-04-26T23:49:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.962+0000 7f7996400640 1 -- 172.21.15.46:0/800286826 wait complete. 2024-04-26T23:49:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.964+0000 7f7996400640 1 Processor -- start 2024-04-26T23:49:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.964+0000 7f7996400640 1 -- start start 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 0x7f7990158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79900ab320 0x7f79900ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7990073f70 con 0x7f79900c6900 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f799015d2a0 con 0x7f7978003a00 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.965+0000 7f7996400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79900735f0 con 0x7f79900ab320 2024-04-26T23:49:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35652/0 (socket says 172.21.15.46:35652) 2024-04-26T23:49:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 -- 172.21.15.46:0/2593564529 learned_addr learned my addr 172.21.15.46:0/2593564529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f7995000640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 0x7f7990158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f7995a00640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79900ab320 0x7f79900ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 -- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79900ab320 msgr2=0x7f79900ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79900ab320 0x7f79900ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 -- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 msgr2=0x7f7990158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 0x7f7990158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f798fe00640 1 -- 172.21.15.46:0/2593564529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799015d4c0 con 0x7f79900c6900 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.966+0000 7f7995000640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 0x7f7990158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f798fe00640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f797c011510 tx=0x7f797c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c015870 con 0x7f79900c6900 2024-04-26T23:49:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f797c015a10 con 0x7f79900c6900 2024-04-26T23:49:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799015d7a0 con 0x7f79900c6900 2024-04-26T23:49:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c015dc0 con 0x7f79900c6900 2024-04-26T23:49:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.967+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799015dd60 con 0x7f79900c6900 2024-04-26T23:49:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.968+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900693b0 con 0x7f79900c6900 2024-04-26T23:49:56.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.969+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f797c013400 con 0x7f79900c6900 2024-04-26T23:49:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.970+0000 7f798cc00640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 0x7f797006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.970+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5249+0+0 (secure 0 0 0) 0x7f797c08c100 con 0x7f79900c6900 2024-04-26T23:49:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.971+0000 7f7995000640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 0x7f797006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.972+0000 7f7995000640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 0x7f797006ede0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7984032000 tx=0x7f798402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:56.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:56.975+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f797c019030 con 0x7f79900c6900 2024-04-26T23:49:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.179+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79900b2ec0 con 0x7f79900c6900 2024-04-26T23:49:57.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.180+0000 7f798cc00640 1 -- 172.21.15.46:0/2593564529 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) ==== 93+0+10 (secure 0 0 0) 0x7f797c05a780 con 0x7f79900c6900 2024-04-26T23:49:57.185 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.182+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 msgr2=0x7f797006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.182+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 0x7f797006ede0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7984032000 tx=0x7f798402e000 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 msgr2=0x7f79900c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f797c011510 tx=0x7f797c0119d0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 shutdown_connections 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f797006c920 0x7f797006ede0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79900ab320 0x7f79900ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79900c6900 0x7f79900c6d80 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 --2- 172.21.15.46:0/2593564529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7978003a00 0x7f7990158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 >> 172.21.15.46:0/2593564529 conn(0x7f79900b27b0 msgr2=0x7f79900cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 shutdown_connections 2024-04-26T23:49:57.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.183+0000 7f7996400640 1 -- 172.21.15.46:0/2593564529 wait complete. 2024-04-26T23:49:57.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:49:57.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.408+0000 7ff102c00640 1 Processor -- start 2024-04-26T23:49:57.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.409+0000 7ff102c00640 1 -- start start 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.410+0000 7ff102c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff0fc0cf450 0x7ff0fc073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.410+0000 7ff102c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc06f1d0 con 0x7ff0fc072190 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.410+0000 7ff102c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc06f470 con 0x7ff0fc0cff40 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.410+0000 7ff102c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff0fc072930 con 0x7ff0fc0cf450 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.411+0000 7ff100e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff0fc072190 0x7ff0fc0cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39626/0 (socket says 172.21.15.46:39626) 2024-04-26T23:49:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.411+0000 7ff100e00640 1 -- 172.21.15.46:0/4228430923 learned_addr learned my addr 172.21.15.46:0/4228430923 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.411+0000 7ff101800640 1 --2- 172.21.15.46:0/4228430923 >> v2:172.21.15.142:3300/0 conn(0x7ff0fc0cf450 0x7ff0fc073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.411+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463604154 0 0) 0x7ff0fc06f1d0 con 0x7ff0fc072190 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.411+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0e4003520 con 0x7ff0fc072190 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1674670499 0 0) 0x7ff0e4003520 con 0x7ff0fc072190 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 >> v2:172.21.15.142:3300/0 conn(0x7ff0fc0cf450 msgr2=0x7ff0fc073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 --2- 172.21.15.46:0/4228430923 >> v2:172.21.15.142:3300/0 conn(0x7ff0fc0cf450 0x7ff0fc073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 >> v1:172.21.15.142:6789/0 conn(0x7ff0fc0cff40 legacy=0x7ff0fc166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc072c10 con 0x7ff0fc072190 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff0e8001670 con 0x7ff0fc072190 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 >> v1:172.21.15.46:6789/0 conn(0x7ff0fc072190 legacy=0x7ff0fc0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40039b0 0x7ff0e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 0x7ff0e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc072930 con 0x7ff0e40051c0 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e400c060 con 0x7ff0e4004800 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e400c260 con 0x7ff0e40039b0 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff102200640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 0x7ff0e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff101800640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40039b0 0x7ff0e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.412+0000 7ff100e00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.413+0000 7ff102200640 1 -- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40039b0 msgr2=0x7ff0e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.413+0000 7ff102200640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40039b0 0x7ff0e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.413+0000 7ff102200640 1 -- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4004800 msgr2=0x7ff0e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.413+0000 7ff102200640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.413+0000 7ff102200640 1 -- 172.21.15.46:0/4228430923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e4003520 con 0x7ff0e40051c0 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.414+0000 7ff102200640 1 -- 172.21.15.46:0/4228430923 reap_dead start 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.414+0000 7ff102200640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 0x7ff0e400b9c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4001e20 tx=0x7ff0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0ea8f15ab3cf30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.414+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4001a70 con 0x7ff0e40051c0 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.414+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f402ae40 con 0x7ff0e40051c0 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.414+0000 7ff0fbe00640 1 -- 172.21.15.46:0/4228430923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003bb0 con 0x7ff0e40051c0 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 -- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 msgr2=0x7ff0e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 0x7ff0e400b9c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4001e20 tx=0x7ff0e4005830 comp rx=0 tx=0).stop 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 -- 172.21.15.46:0/4228430923 shutdown_connections 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 --2- 172.21.15.46:0/4228430923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e40051c0 0x7ff0e400b9c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 -- 172.21.15.46:0/4228430923 >> 172.21.15.46:0/4228430923 conn(0x7ff0fc0b2820 msgr2=0x7ff0fc0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 -- 172.21.15.46:0/4228430923 shutdown_connections 2024-04-26T23:49:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.415+0000 7ff102c00640 1 -- 172.21.15.46:0/4228430923 wait complete. 2024-04-26T23:49:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.417+0000 7ff102c00640 1 Processor -- start 2024-04-26T23:49:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.417+0000 7ff102c00640 1 -- start start 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40051c0 0x7ff0fc1e1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc143bd0 0x7ff0fc144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc0734b0 con 0x7ff0fc1daf70 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc0731d0 con 0x7ff0fc143bd0 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc072620 con 0x7ff0e40051c0 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35678/0 (socket says 172.21.15.46:35678) 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff102200640 1 -- 172.21.15.46:0/1425111989 learned_addr learned my addr 172.21.15.46:0/1425111989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff101800640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40051c0 0x7ff0fc1e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.418+0000 7ff100e00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc143bd0 0x7ff0fc144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 -- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40051c0 msgr2=0x7ff0fc1e1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40051c0 0x7ff0fc1e1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 -- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc143bd0 msgr2=0x7ff0fc144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc143bd0 0x7ff0fc144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 -- 172.21.15.46:0/1425111989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc1dc860 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102200640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4001f10 tx=0x7ff0f40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40055c0 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f4005760 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4006d90 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.419+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc144890 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.420+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1df400 con 0x7ff0fc1daf70 2024-04-26T23:49:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.421+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc0693b0 con 0x7ff0fc1daf70 2024-04-26T23:49:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.422+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0f4007030 con 0x7ff0fc1daf70 2024-04-26T23:49:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.423+0000 7ff0f9600640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 0x7ff0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.423+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5249+0+0 (secure 0 0 0) 0x7ff0f403a020 con 0x7ff0fc1daf70 2024-04-26T23:49:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.423+0000 7ff101800640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 0x7ff0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.425+0000 7ff101800640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 0x7ff0dc06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009990 tx=0x7ff0f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.427+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0fc1dc860 con 0x7ff0fc1daf70 2024-04-26T23:49:57.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.631+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff0fc0b1190 con 0x7ff0fc1daf70 2024-04-26T23:49:57.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.632+0000 7ff0f9600640 1 -- 172.21.15.46:0/1425111989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) ==== 84+0+65 (secure 0 0 0) 0x7ff0f4078f30 con 0x7ff0fc1daf70 2024-04-26T23:49:57.637 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:49:57.638 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 msgr2=0x7ff0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 0x7ff0dc06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009990 tx=0x7ff0f0008040 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 msgr2=0x7ff0fc1db3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4001f10 tx=0x7ff0f40014a0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 shutdown_connections 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff0dc06c8f0 0x7ff0dc06edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc1daf70 0x7ff0fc1db3f0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc143bd0 0x7ff0fc144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 --2- 172.21.15.46:0/1425111989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0e40051c0 0x7ff0fc1e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.635+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 >> 172.21.15.46:0/1425111989 conn(0x7ff0fc0b2820 msgr2=0x7ff0fc1423c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.636+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 shutdown_connections 2024-04-26T23:49:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.636+0000 7ff102c00640 1 -- 172.21.15.46:0/1425111989 wait complete. 2024-04-26T23:49:57.658 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:49:57.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:49:57.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.865+0000 7fe923e00640 1 Processor -- start 2024-04-26T23:49:57.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.865+0000 7fe923e00640 1 -- start start 2024-04-26T23:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe924146740 0x7fe9240cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9240d17f0 0x7fe9240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9240b3ce0 con 0x7fe924146b20 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe9240b48f0 con 0x7fe9240d17f0 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9240b3fc0 con 0x7fe924146740 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9240d17f0 0x7fe9240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9240d17f0 0x7fe9240cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35680/0 (socket says 172.21.15.46:35680) 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe91a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe924146b20 0x7fe9240d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39634/0 (socket says 172.21.15.46:39634) 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.867+0000 7fe923400640 1 -- 172.21.15.46:0/4246197721 learned_addr learned my addr 172.21.15.46:0/4246197721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe922a00640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe924146740 0x7fe9240cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe923400640 1 -- 172.21.15.46:0/4246197721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe924146740 msgr2=0x7fe9240cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe923400640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe924146740 0x7fe9240cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe923400640 1 -- 172.21.15.46:0/4246197721 >> v1:172.21.15.46:6789/0 conn(0x7fe924146b20 legacy=0x7fe9240d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe923400640 1 -- 172.21.15.46:0/4246197721 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9240b42a0 con 0x7fe9240d17f0 2024-04-26T23:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.868+0000 7fe922000640 1 -- 172.21.15.46:0/4246197721 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711823236 0 0) 0x7fe9240b3ce0 con 0x7fe924146b20 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.869+0000 7fe923400640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240d17f0 0x7fe9240cf590 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009870 tx=0x7fe91c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d807511930a0ebf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.869+0000 7fe922000640 1 -- 172.21.15.46:0/4246197721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c037070 con 0x7fe9240d17f0 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.869+0000 7fe922000640 1 -- 172.21.15.46:0/4246197721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c032440 con 0x7fe9240d17f0 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.869+0000 7fe922000640 1 -- 172.21.15.46:0/4246197721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c032810 con 0x7fe9240d17f0 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 -- 172.21.15.46:0/4246197721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240d17f0 msgr2=0x7fe9240cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240d17f0 0x7fe9240cf590 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009870 tx=0x7fe91c02a1c0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 -- 172.21.15.46:0/4246197721 shutdown_connections 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240d17f0 0x7fe9240cf590 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 --2- 172.21.15.46:0/4246197721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe924146740 0x7fe9240cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 -- 172.21.15.46:0/4246197721 >> 172.21.15.46:0/4246197721 conn(0x7fe92406c770 msgr2=0x7fe92406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 -- 172.21.15.46:0/4246197721 shutdown_connections 2024-04-26T23:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.870+0000 7fe923e00640 1 -- 172.21.15.46:0/4246197721 wait complete. 2024-04-26T23:49:57.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.872+0000 7fe923e00640 1 Processor -- start 2024-04-26T23:49:57.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.872+0000 7fe923e00640 1 -- start start 2024-04-26T23:49:57.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240d17f0 0x7fe92413e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe924146b20 0x7fe9240b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9240cdc10 con 0x7fe924146740 2024-04-26T23:49:57.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe92415e560 con 0x7fe924146b20 2024-04-26T23:49:57.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9240cdd90 con 0x7fe9240d17f0 2024-04-26T23:49:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe91a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe91a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35688/0 (socket says 172.21.15.46:35688) 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe91a000640 1 -- 172.21.15.46:0/27395331 learned_addr learned my addr 172.21.15.46:0/27395331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe923400640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe924146b20 0x7fe9240b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.873+0000 7fe922a00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240d17f0 0x7fe92413e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 -- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240d17f0 msgr2=0x7fe92413e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240d17f0 0x7fe92413e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 -- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe924146b20 msgr2=0x7fe9240b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe924146b20 0x7fe9240b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 -- 172.21.15.46:0/27395331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9240b0d60 con 0x7fe924146740 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91a000640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c890 tx=0x7fe90c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c018070 con 0x7fe924146740 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c00a640 con 0x7fe924146740 2024-04-26T23:49:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00aa30 con 0x7fe924146740 2024-04-26T23:49:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.874+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9240b0fe0 con 0x7fe924146740 2024-04-26T23:49:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.875+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9240b15a0 con 0x7fe924146740 2024-04-26T23:49:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.876+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9240b3ce0 con 0x7fe924146740 2024-04-26T23:49:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.877+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe90c00abd0 con 0x7fe924146740 2024-04-26T23:49:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.878+0000 7fe91b400640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 0x7fe90006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.878+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5249+0+0 (secure 0 0 0) 0x7fe90c08b060 con 0x7fe924146740 2024-04-26T23:49:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.878+0000 7fe922a00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 0x7fe90006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:57.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.879+0000 7fe922a00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 0x7fe90006edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe914006fd0 tx=0x7fe914008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:57.882+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9240b3ce0 con 0x7fe924146740 2024-04-26T23:49:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.088+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe9240693b0 con 0x7fe924146740 2024-04-26T23:49:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.793+0000 7fe91b400640 1 -- 172.21.15.46:0/27395331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) ==== 138+0+0 (secure 0 0 0) 0x7fe9240b0d60 con 0x7fe924146740 2024-04-26T23:49:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.797+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 msgr2=0x7fe90006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.797+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 0x7fe90006edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe914006fd0 tx=0x7fe914008040 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 msgr2=0x7fe92415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c890 tx=0x7fe90c00cd50 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 shutdown_connections 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90006c8f0 0x7fe90006edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe924146b20 0x7fe9240b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924146740 0x7fe92415c7f0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 --2- 172.21.15.46:0/27395331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240d17f0 0x7fe92413e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 >> 172.21.15.46:0/27395331 conn(0x7fe92406c770 msgr2=0x7fe92406f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.798+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 shutdown_connections 2024-04-26T23:49:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:58.800+0000 7fe923e00640 1 -- 172.21.15.46:0/27395331 wait complete. 2024-04-26T23:49:58.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:49:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.033+0000 7fb101e00640 1 Processor -- start 2024-04-26T23:49:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.033+0000 7fb101e00640 1 -- start start 2024-04-26T23:49:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.035+0000 7fb101e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb0fc0cdd20 0x7fb0fc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.035+0000 7fb101e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.035+0000 7fb101e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc0af570 con 0x7fb0fc15f210 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.035+0000 7fb101e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc0af850 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.035+0000 7fb101e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc0cb680 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.036+0000 7fb101400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.036+0000 7fb0fbe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb0fc0cdd20 0x7fb0fc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.036+0000 7fb101400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:47616/0 (socket says 172.21.15.46:47616) 2024-04-26T23:49:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.036+0000 7fb101400640 1 -- 172.21.15.46:0/3777382891 learned_addr learned my addr 172.21.15.46:0/3777382891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.036+0000 7fb100a00640 1 --1- 172.21.15.46:0/3777382891 >> v1:172.21.15.142:6789/0 conn(0x7fb0fc15f210 0x7fb0fc0cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:39800/0 (socket says 172.21.15.46:39800) 2024-04-26T23:49:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.037+0000 7fb0fbe00640 1 -- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 msgr2=0x7fb0fc0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:49:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.037+0000 7fb0fbe00640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.037+0000 7fb0fbe00640 1 -- 172.21.15.46:0/3777382891 >> v1:172.21.15.142:6789/0 conn(0x7fb0fc15f210 legacy=0x7fb0fc0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.037+0000 7fb0fbe00640 1 -- 172.21.15.46:0/3777382891 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0cb960 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.037+0000 7fb101400640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:49:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.038+0000 7fb0fbe00640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0cc160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af408398ac01a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.039+0000 7fb0fb400640 1 -- 172.21.15.46:0/3777382891 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec037070 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.039+0000 7fb0fb400640 1 -- 172.21.15.46:0/3777382891 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec032440 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.039+0000 7fb0fb400640 1 -- 172.21.15.46:0/3777382891 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec032810 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.039+0000 7fb101e00640 1 -- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 msgr2=0x7fb0fc0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.039+0000 7fb101e00640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0cc160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02a180 comp rx=0 tx=0).stop 2024-04-26T23:49:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 -- 172.21.15.46:0/3777382891 shutdown_connections 2024-04-26T23:49:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc0ae8f0 0x7fb0fc0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 --2- 172.21.15.46:0/3777382891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0cc160 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 -- 172.21.15.46:0/3777382891 >> 172.21.15.46:0/3777382891 conn(0x7fb0fc0b27b0 msgr2=0x7fb0fc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:59.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 -- 172.21.15.46:0/3777382891 shutdown_connections 2024-04-26T23:49:59.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.040+0000 7fb101e00640 1 -- 172.21.15.46:0/3777382891 wait complete. 2024-04-26T23:49:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.042+0000 7fb101e00640 1 Processor -- start 2024-04-26T23:49:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.042+0000 7fb101e00640 1 -- start start 2024-04-26T23:49:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.043+0000 7fb101e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc15f210 0x7fb0fc0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc15f5f0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc0af280 con 0x7fb0fc0cdd20 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc071e80 con 0x7fb0fc15f210 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb100a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb100a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35694/0 (socket says 172.21.15.46:35694) 2024-04-26T23:49:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb100a00640 1 -- 172.21.15.46:0/2879340087 learned_addr learned my addr 172.21.15.46:0/2879340087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb101400640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc15f210 0x7fb0fc0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.044+0000 7fb0fbe00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 -- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc15f210 msgr2=0x7fb0fc0b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc15f210 0x7fb0fc0b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 -- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 msgr2=0x7fb0fc0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 -- 172.21.15.46:0/2879340087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0c8ff0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb100a00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c3c0 tx=0x7fb0f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00074e0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc0c92d0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f0007680 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.045+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0004a00 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.046+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc0c9890 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.047+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc0693b0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.048+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0f001a020 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.049+0000 7fb0f8c00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 0x7fb0e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.049+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5619+0+0 (secure 0 0 0) 0x7fb0f008b3e0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.050+0000 7fb0fbe00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 0x7fb0e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.051+0000 7fb0fbe00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 0x7fb0e006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec034000 tx=0x7fb0ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.054+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0f0059740 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.263+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb0fc15cba0 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.812+0000 7fb0f8c00640 1 -- 172.21.15.46:0/2879340087 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) ==== 170+0+0 (secure 0 0 0) 0x7fb0f0059040 con 0x7fb0fc0ae8f0 2024-04-26T23:49:59.817 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 msgr2=0x7fb0e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 0x7fb0e006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec034000 tx=0x7fb0ec02e000 comp rx=0 tx=0).stop 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 msgr2=0x7fb0fc15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c3c0 tx=0x7fb0f000c880 comp rx=0 tx=0).stop 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 shutdown_connections 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb0e006c8f0 0x7fb0e006edb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0fc15f210 0x7fb0fc0b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc0cdd20 0x7fb0fc0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 --2- 172.21.15.46:0/2879340087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0fc0ae8f0 0x7fb0fc15b0b0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 >> 172.21.15.46:0/2879340087 conn(0x7fb0fc0b27b0 msgr2=0x7fb0fc15d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.816+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 shutdown_connections 2024-04-26T23:49:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:49:59.817+0000 7fb101e00640 1 -- 172.21.15.46:0/2879340087 wait complete. 2024-04-26T23:49:59.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:50:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.034+0000 7f9ec0800640 1 Processor -- start 2024-04-26T23:50:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.034+0000 7f9ec0800640 1 -- start start 2024-04-26T23:50:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.035+0000 7f9ec0800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9eb81597e0 0x7f9eb80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.035+0000 7f9ec0800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9eb8072620 0x7f9eb80b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ec0800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb8073120 con 0x7f9eb80b5730 2024-04-26T23:50:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ec0800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb8159bc0 con 0x7f9eb81597e0 2024-04-26T23:50:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ec0800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb8072b60 con 0x7f9eb8072620 2024-04-26T23:50:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ebde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9eb81597e0 0x7f9eb80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ebd400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9eb80b5730 0x7f9eb8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39640/0 (socket says 172.21.15.46:39640) 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ebd400640 1 -- 172.21.15.46:0/2993848206 learned_addr learned my addr 172.21.15.46:0/2993848206 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.036+0000 7f9ebe800640 1 --2- 172.21.15.46:0/2993848206 >> v2:172.21.15.142:3300/0 conn(0x7f9eb8072620 0x7f9eb80b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.037+0000 7f9ebde00640 1 -- 172.21.15.46:0/2993848206 >> v2:172.21.15.142:3300/0 conn(0x7f9eb8072620 msgr2=0x7f9eb80b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.037+0000 7f9ebde00640 1 --2- 172.21.15.46:0/2993848206 >> v2:172.21.15.142:3300/0 conn(0x7f9eb8072620 0x7f9eb80b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.037+0000 7f9ebde00640 1 -- 172.21.15.46:0/2993848206 >> v1:172.21.15.46:6789/0 conn(0x7f9eb80b5730 legacy=0x7f9eb8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.037+0000 7f9ebde00640 1 -- 172.21.15.46:0/2993848206 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb8072e40 con 0x7f9eb81597e0 2024-04-26T23:50:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.038+0000 7f9ebde00640 1 --2- 172.21.15.46:0/2993848206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb81597e0 0x7f9eb80ae8e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b3d60 tx=0x7f9eac02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d107a57b595120 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.038+0000 7f9ebca00640 1 -- 172.21.15.46:0/2993848206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac037070 con 0x7f9eb81597e0 2024-04-26T23:50:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.038+0000 7f9ebca00640 1 -- 172.21.15.46:0/2993848206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eac032440 con 0x7f9eb81597e0 2024-04-26T23:50:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.038+0000 7f9ebca00640 1 -- 172.21.15.46:0/2993848206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac032810 con 0x7f9eb81597e0 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 -- 172.21.15.46:0/2993848206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb81597e0 msgr2=0x7f9eb80ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 --2- 172.21.15.46:0/2993848206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb81597e0 0x7f9eb80ae8e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b3d60 tx=0x7f9eac02a250 comp rx=0 tx=0).stop 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 -- 172.21.15.46:0/2993848206 shutdown_connections 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 --2- 172.21.15.46:0/2993848206 >> v2:172.21.15.142:3300/0 conn(0x7f9eb8072620 0x7f9eb80b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 --2- 172.21.15.46:0/2993848206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb81597e0 0x7f9eb80ae8e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 -- 172.21.15.46:0/2993848206 >> 172.21.15.46:0/2993848206 conn(0x7f9eb806c980 msgr2=0x7f9eb80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 -- 172.21.15.46:0/2993848206 shutdown_connections 2024-04-26T23:50:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.039+0000 7f9ec0800640 1 -- 172.21.15.46:0/2993848206 wait complete. 2024-04-26T23:50:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.041+0000 7f9ec0800640 1 Processor -- start 2024-04-26T23:50:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.041+0000 7f9ec0800640 1 -- start start 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb80b5730 0x7f9eb81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9eb81597e0 0x7f9eb80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80aee20 con 0x7f9eb8072620 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80b43b0 con 0x7f9eb80b5730 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ec0800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb80aefa0 con 0x7f9eb81597e0 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35720/0 (socket says 172.21.15.46:35720) 2024-04-26T23:50:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.042+0000 7f9ebde00640 1 -- 172.21.15.46:0/1130404236 learned_addr learned my addr 172.21.15.46:0/1130404236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:00.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebe800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9eb81597e0 0x7f9eb80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebd400640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb80b5730 0x7f9eb81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 -- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9eb81597e0 msgr2=0x7f9eb80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9eb81597e0 0x7f9eb80cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 -- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb80b5730 msgr2=0x7f9eb81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb80b5730 0x7f9eb81566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 -- 172.21.15.46:0/1130404236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb8156c20 con 0x7f9eb8072620 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ebde00640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009890 tx=0x7f9eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac037040 con 0x7f9eb8072620 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eac042070 con 0x7f9eb8072620 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb81f8670 con 0x7f9eb8072620 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.043+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac030b90 con 0x7f9eb8072620 2024-04-26T23:50:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.044+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81f8c30 con 0x7f9eb8072620 2024-04-26T23:50:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.045+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb81f8e70 con 0x7f9eb8072620 2024-04-26T23:50:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.046+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9eac047460 con 0x7f9eb8072620 2024-04-26T23:50:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.047+0000 7f9eb2000640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 0x7f9e9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.048+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 6001+0+0 (secure 0 0 0) 0x7f9eac0ac8a0 con 0x7f9eb8072620 2024-04-26T23:50:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.048+0000 7f9ebd400640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 0x7f9e9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:00.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.049+0000 7f9ebd400640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 0x7f9e9c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8157420 tx=0x7f9ea8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.053+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9eb81f8e70 con 0x7f9eb8072620 2024-04-26T23:50:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.300+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9eb8076550 con 0x7f9eb8072620 2024-04-26T23:50:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.836+0000 7f9eb2000640 1 -- 172.21.15.46:0/1130404236 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v105) ==== 174+0+0 (secure 0 0 0) 0x7f9eac032e20 con 0x7f9eb8072620 2024-04-26T23:50:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 19 and data pool 20 2024-04-26T23:50:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.849+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 msgr2=0x7f9e9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.849+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 0x7f9e9c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8157420 tx=0x7f9ea8009040 comp rx=0 tx=0).stop 2024-04-26T23:50:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.849+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 msgr2=0x7f9eb81f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.849+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009890 tx=0x7f9eac0046c0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 shutdown_connections 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e9c06c8f0 0x7f9e9c06edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9eb81597e0 0x7f9eb80cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb80b5730 0x7f9eb81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 --2- 172.21.15.46:0/1130404236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb8072620 0x7f9eb81f7fe0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:00.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.855+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 >> 172.21.15.46:0/1130404236 conn(0x7f9eb806c980 msgr2=0x7f9eb806f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:00.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.856+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 shutdown_connections 2024-04-26T23:50:00.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:00.857+0000 7f9ec0800640 1 -- 172.21.15.46:0/1130404236 wait complete. 2024-04-26T23:50:00.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:50:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.069+0000 7f2306600640 1 Processor -- start 2024-04-26T23:50:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.069+0000 7f2306600640 1 -- start start 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2306600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f230006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2306600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23000b76a0 0x7f23000b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2306600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23000d21d0 con 0x7f23000d0f50 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2306600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f23000d1ef0 con 0x7f23000b76a0 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2306600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23000b6bc0 con 0x7f23000d1a40 2024-04-26T23:50:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2305200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f23000d0f50 0x7f23000d1330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39656/0 (socket says 172.21.15.46:39656) 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2305200640 1 -- 172.21.15.46:0/3451514919 learned_addr learned my addr 172.21.15.46:0/3451514919 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.070+0000 7f2305c00640 1 --2- 172.21.15.46:0/3451514919 >> v2:172.21.15.46:3300/0 conn(0x7f23000b76a0 0x7f23000b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.071+0000 7f22f7e00640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f230006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.071+0000 7f2305c00640 1 -- 172.21.15.46:0/3451514919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 msgr2=0x7f230006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.071+0000 7f2305c00640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f230006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.071+0000 7f2305c00640 1 -- 172.21.15.46:0/3451514919 >> v1:172.21.15.46:6789/0 conn(0x7f23000d0f50 legacy=0x7f23000d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.071+0000 7f2305c00640 1 -- 172.21.15.46:0/3451514919 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23000b6ea0 con 0x7f23000b76a0 2024-04-26T23:50:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.072+0000 7f2305c00640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000b76a0 0x7f23000b7b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009d90 tx=0x7f22f802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7873debe1ca842 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.072+0000 7f22ffe00640 1 -- 172.21.15.46:0/3451514919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8031600 con 0x7f23000b76a0 2024-04-26T23:50:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f22ffe00640 1 -- 172.21.15.46:0/3451514919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f80317a0 con 0x7f23000b76a0 2024-04-26T23:50:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f22ffe00640 1 -- 172.21.15.46:0/3451514919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8031b70 con 0x7f23000b76a0 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 -- 172.21.15.46:0/3451514919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000b76a0 msgr2=0x7f23000b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000b76a0 0x7f23000b7b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009d90 tx=0x7f22f802a5d0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 -- 172.21.15.46:0/3451514919 shutdown_connections 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000b76a0 0x7f23000b7b00 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 --2- 172.21.15.46:0/3451514919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f230006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 -- 172.21.15.46:0/3451514919 >> 172.21.15.46:0/3451514919 conn(0x7f23000d2cc0 msgr2=0x7f23000d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.073+0000 7f2306600640 1 -- 172.21.15.46:0/3451514919 shutdown_connections 2024-04-26T23:50:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.074+0000 7f2306600640 1 -- 172.21.15.46:0/3451514919 wait complete. 2024-04-26T23:50:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.075+0000 7f2306600640 1 Processor -- start 2024-04-26T23:50:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.075+0000 7f2306600640 1 -- start start 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23000b76a0 0x7f23000cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f2300156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f230015f7c0 con 0x7f23000d0f50 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f230015f640 con 0x7f23000b76a0 2024-04-26T23:50:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.076+0000 7f2306600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23000b3ff0 con 0x7f23000d1a40 2024-04-26T23:50:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35732/0 (socket says 172.21.15.46:35732) 2024-04-26T23:50:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 -- 172.21.15.46:0/3616824841 learned_addr learned my addr 172.21.15.46:0/3616824841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f2305c00640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f2300156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 -- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 msgr2=0x7f2300156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f2300156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 -- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23000b76a0 msgr2=0x7f23000cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23000b76a0 0x7f23000cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.077+0000 7f22f7e00640 1 -- 172.21.15.46:0/3616824841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23000caf10 con 0x7f23000d0f50 2024-04-26T23:50:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f22f7e00640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d800 tx=0x7f22e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e800b6c0 con 0x7f23000d0f50 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e8019070 con 0x7f23000d0f50 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23000cb1f0 con 0x7f23000d0f50 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e800ba80 con 0x7f23000d0f50 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.078+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23000cb7b0 con 0x7f23000d0f50 2024-04-26T23:50:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.079+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23000cbbf0 con 0x7f23000d0f50 2024-04-26T23:50:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.081+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22e800bc20 con 0x7f23000d0f50 2024-04-26T23:50:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.082+0000 7f22fd600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 0x7f22dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.082+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f22e808c2a0 con 0x7f23000d0f50 2024-04-26T23:50:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.082+0000 7f2305200640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 0x7f22dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.084+0000 7f2305200640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 0x7f22dc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22f0000f60 tx=0x7f22f0007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.086+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22e805a4a0 con 0x7f23000d0f50 2024-04-26T23:50:01.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.290+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23000cbdc0 con 0x7f23000d0f50 2024-04-26T23:50:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.293+0000 7f22fd600640 1 -- 172.21.15.46:0/3616824841 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+16877 (secure 0 0 0) 0x7f22e8059da0 con 0x7f23000d0f50 2024-04-26T23:50:01.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:01.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":136,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:00.302761+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.295+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 msgr2=0x7f22dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.295+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 0x7f22dc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22f0000f60 tx=0x7f22f0007a80 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 msgr2=0x7f23000ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d800 tx=0x7f22e800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 shutdown_connections 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f22dc06c8f0 0x7f22dc06edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23000d1a40 0x7f2300156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23000d0f50 0x7f23000ca7a0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 --2- 172.21.15.46:0/3616824841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23000b76a0 0x7f23000cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 >> 172.21.15.46:0/3616824841 conn(0x7f23000d2cc0 msgr2=0x7f2300071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:01.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 shutdown_connections 2024-04-26T23:50:01.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.296+0000 7f2306600640 1 -- 172.21.15.46:0/3616824841 wait complete. 2024-04-26T23:50:01.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:50:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.522+0000 7f6fba000640 1 Processor -- start 2024-04-26T23:50:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.522+0000 7f6fba000640 1 -- start start 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.523+0000 7f6fba000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fb40720e0 0x7f6fb40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.523+0000 7f6fba000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fb4160910 0x7f6fb4160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.523+0000 7f6fba000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb4073ad0 con 0x7f6fb41466b0 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.523+0000 7f6fba000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb41612a0 con 0x7f6fb40720e0 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.523+0000 7f6fba000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb4146a90 con 0x7f6fb4160910 2024-04-26T23:50:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fb41466b0 0x7f6fb406de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39664/0 (socket says 172.21.15.46:39664) 2024-04-26T23:50:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb8c00640 1 -- 172.21.15.46:0/1177324394 learned_addr learned my addr 172.21.15.46:0/1177324394 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1177324394 >> v2:172.21.15.46:3300/0 conn(0x7f6fb40720e0 0x7f6fb40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3400640 1 -- 172.21.15.46:0/1177324394 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521992925 0 0) 0x7f6fb4073ad0 con 0x7f6fb41466b0 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb9600640 1 --2- 172.21.15.46:0/1177324394 >> v2:172.21.15.142:3300/0 conn(0x7f6fb4160910 0x7f6fb4160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3400640 1 -- 172.21.15.46:0/1177324394 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9c003520 con 0x7f6fb41466b0 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3e00640 1 -- 172.21.15.46:0/1177324394 >> v2:172.21.15.142:3300/0 conn(0x7f6fb4160910 msgr2=0x7f6fb4160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1177324394 >> v2:172.21.15.142:3300/0 conn(0x7f6fb4160910 0x7f6fb4160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3e00640 1 -- 172.21.15.46:0/1177324394 >> v1:172.21.15.46:6789/0 conn(0x7f6fb41466b0 legacy=0x7f6fb406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.524+0000 7f6fb3e00640 1 -- 172.21.15.46:0/1177324394 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb4073d70 con 0x7f6fb40720e0 2024-04-26T23:50:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1177324394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb40cd860 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00097a0 tx=0x7f6fa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92e0cc0b7d98ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fb3400640 1 -- 172.21.15.46:0/1177324394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0037070 con 0x7f6fb40720e0 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fb3400640 1 -- 172.21.15.46:0/1177324394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa0032440 con 0x7f6fb40720e0 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fb3400640 1 -- 172.21.15.46:0/1177324394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00327f0 con 0x7f6fb40720e0 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fba000640 1 -- 172.21.15.46:0/1177324394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 msgr2=0x7f6fb40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.526+0000 7f6fba000640 1 --2- 172.21.15.46:0/1177324394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb40cd860 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00097a0 tx=0x7f6fa002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 -- 172.21.15.46:0/1177324394 shutdown_connections 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 --2- 172.21.15.46:0/1177324394 >> v2:172.21.15.142:3300/0 conn(0x7f6fb4160910 0x7f6fb4160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 --2- 172.21.15.46:0/1177324394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb40cd860 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 -- 172.21.15.46:0/1177324394 >> 172.21.15.46:0/1177324394 conn(0x7f6fb40b26d0 msgr2=0x7f6fb40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 -- 172.21.15.46:0/1177324394 shutdown_connections 2024-04-26T23:50:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.527+0000 7f6fba000640 1 -- 172.21.15.46:0/1177324394 wait complete. 2024-04-26T23:50:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.528+0000 7f6fba000640 1 Processor -- start 2024-04-26T23:50:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 -- start start 2024-04-26T23:50:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb41466b0 0x7f6fb40b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 0x7f6fb40b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40cb470 con 0x7f6fb40720e0 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40cb190 con 0x7f6fb41466b0 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.529+0000 7f6fba000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb40cdda0 con 0x7f6fb4160910 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35742/0 (socket says 172.21.15.46:35742) 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 -- 172.21.15.46:0/1599410825 learned_addr learned my addr 172.21.15.46:0/1599410825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb41466b0 0x7f6fb40b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb9600640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 0x7f6fb40b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 -- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 msgr2=0x7f6fb40b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 0x7f6fb40b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 -- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb41466b0 msgr2=0x7f6fb40b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb41466b0 0x7f6fb40b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb8c00640 1 -- 172.21.15.46:0/1599410825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb40c8790 con 0x7f6fb40720e0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.530+0000 7f6fb9600640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 0x7f6fb40b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fb8c00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c390 tx=0x7f6fa800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8017070 con 0x7f6fb40720e0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa8005ce0 con 0x7f6fb40720e0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80072f0 con 0x7f6fb40720e0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb40c8a70 con 0x7f6fb40720e0 2024-04-26T23:50:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.531+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb40c9030 con 0x7f6fb40720e0 2024-04-26T23:50:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.532+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb40693b0 con 0x7f6fb40720e0 2024-04-26T23:50:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.533+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fa801a070 con 0x7f6fb40720e0 2024-04-26T23:50:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.534+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.534+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f6fa808b110 con 0x7f6fb40720e0 2024-04-26T23:50:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.535+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.536+0000 7f6fb3e00640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0034000 tx=0x7f6fa002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.539+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fa80593a0 con 0x7f6fb40720e0 2024-04-26T23:50:01.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.740+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fb40c7180 con 0x7f6fb40720e0 2024-04-26T23:50:01.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.744+0000 7f6fb0c00640 1 -- 172.21.15.46:0/1599410825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+16877 (secure 0 0 0) 0x7f6fa8058ca0 con 0x7f6fb40720e0 2024-04-26T23:50:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:01.749 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":136,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:00.302761+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 msgr2=0x7f6f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0034000 tx=0x7f6fa002e000 comp rx=0 tx=0).stop 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 msgr2=0x7f6fb41371e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c390 tx=0x7f6fa800c850 comp rx=0 tx=0).stop 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 shutdown_connections 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb4160910 0x7f6fb40b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb41466b0 0x7f6fb40b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 --2- 172.21.15.46:0/1599410825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fb40720e0 0x7f6fb41371e0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 >> 172.21.15.46:0/1599410825 conn(0x7f6fb40b26d0 msgr2=0x7f6fb4143510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 shutdown_connections 2024-04-26T23:50:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.746+0000 7f6fba000640 1 -- 172.21.15.46:0/1599410825 wait complete. 2024-04-26T23:50:01.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:50:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.971+0000 7ffa36a00640 1 Processor -- start 2024-04-26T23:50:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.971+0000 7ffa36a00640 1 -- start start 2024-04-26T23:50:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 0x7ffa300b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa36a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa300b4320 0x7ffa300ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa36a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300b4b30 con 0x7ffa300d04c0 2024-04-26T23:50:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa36a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffa300b4e10 con 0x7ffa300b4320 2024-04-26T23:50:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa36a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300b50f0 con 0x7ffa3006f4b0 2024-04-26T23:50:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa2fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa300d04c0 0x7ffa3006eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47692/0 (socket says 172.21.15.46:47692) 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa2fe00640 1 -- 172.21.15.46:0/721025871 learned_addr learned my addr 172.21.15.46:0/721025871 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa34a00640 1 --2- 172.21.15.46:0/721025871 >> v2:172.21.15.142:3300/0 conn(0x7ffa300b4320 0x7ffa300ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.973+0000 7ffa27e00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 0x7ffa300b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347581549 0 0) 0x7ffa300b4b30 con 0x7ffa300d04c0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa14003520 con 0x7ffa300d04c0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550710663 0 0) 0x7ffa14003520 con 0x7ffa300d04c0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 msgr2=0x7ffa300b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 0x7ffa300b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 >> v2:172.21.15.142:3300/0 conn(0x7ffa300b4320 msgr2=0x7ffa300ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 --2- 172.21.15.46:0/721025871 >> v2:172.21.15.142:3300/0 conn(0x7ffa300b4320 0x7ffa300ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa300b53d0 con 0x7ffa300d04c0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ffa20001670 con 0x7ffa300d04c0 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 >> v1:172.21.15.46:6789/0 conn(0x7ffa300d04c0 legacy=0x7ffa3006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 0x7ffa140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa14003f80 0x7ffa14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa14004d40 0x7ffa140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b4e10 con 0x7ffa14003f80 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b50f0 con 0x7ffa14004d40 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.974+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1400ba70 con 0x7ffa3006f4b0 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa14003f80 0x7ffa14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 -- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 msgr2=0x7ffa140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3006f4b0 0x7ffa140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 -- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa14004d40 msgr2=0x7ffa140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa14004d40 0x7ffa140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.975+0000 7ffa27e00640 1 -- 172.21.15.46:0/721025871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14003520 con 0x7ffa14003f80 2024-04-26T23:50:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.976+0000 7ffa27e00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa14003f80 0x7ffa14004800 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009ac0 tx=0x7ffa14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51d27c485ca7764 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.976+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18036070 con 0x7ffa14003f80 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.976+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18031400 con 0x7ffa14003f80 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.976+0000 7ffa2f400640 1 -- 172.21.15.46:0/721025871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18009690 con 0x7ffa14003f80 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 -- 172.21.15.46:0/721025871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa14003f80 msgr2=0x7ffa14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 --2- 172.21.15.46:0/721025871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa14003f80 0x7ffa14004800 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009ac0 tx=0x7ffa14005830 comp rx=0 tx=0).stop 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa34a00640 1 -- 172.21.15.46:0/721025871 reap_dead start 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 -- 172.21.15.46:0/721025871 shutdown_connections 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 -- 172.21.15.46:0/721025871 >> 172.21.15.46:0/721025871 conn(0x7ffa3006c980 msgr2=0x7ffa300b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 -- 172.21.15.46:0/721025871 shutdown_connections 2024-04-26T23:50:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.977+0000 7ffa36a00640 1 -- 172.21.15.46:0/721025871 wait complete. 2024-04-26T23:50:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.979+0000 7ffa36a00640 1 Processor -- start 2024-04-26T23:50:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.979+0000 7ffa36a00640 1 -- start start 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14003f80 0x7ffa30158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa301405b0 0x7ffa30158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b48d0 con 0x7ffa3015e820 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b4750 con 0x7ffa301405b0 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa36a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300b3de0 con 0x7ffa14003f80 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53984/0 (socket says 172.21.15.46:53984) 2024-04-26T23:50:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.980+0000 7ffa34a00640 1 -- 172.21.15.46:0/2269243446 learned_addr learned my addr 172.21.15.46:0/2269243446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 -- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14003f80 msgr2=0x7ffa30158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14003f80 0x7ffa30158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 -- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa301405b0 msgr2=0x7ffa30158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa301405b0 0x7ffa30158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 -- 172.21.15.46:0/2269243446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3015f300 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa34a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b3760 tx=0x7ffa2800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.981+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800bb10 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.982+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2801a070 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.982+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3013c580 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.982+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28014400 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.982+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301591b0 con 0x7ffa3015e820 2024-04-26T23:50:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.983+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa300caa80 con 0x7ffa3015e820 2024-04-26T23:50:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.984+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa28005d10 con 0x7ffa3015e820 2024-04-26T23:50:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.985+0000 7ffa2cc00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 0x7ffa0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.985+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7ffa2808bf80 con 0x7ffa3015e820 2024-04-26T23:50:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.985+0000 7ffa2fe00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 0x7ffa0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.987+0000 7ffa2fe00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 0x7ffa0c06ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffa30140aa0 tx=0x7ffa200015c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:01.990+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa300caa80 con 0x7ffa3015e820 2024-04-26T23:50:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.222+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffa30069530 con 0x7ffa3015e820 2024-04-26T23:50:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.871+0000 7ffa2cc00640 1 -- 172.21.15.46:0/2269243446 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v108) ==== 118+0+0 (secure 0 0 0) 0x7ffa28024020 con 0x7ffa3015e820 2024-04-26T23:50:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 msgr2=0x7ffa0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 0x7ffa0c06ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffa30140aa0 tx=0x7ffa200015c0 comp rx=0 tx=0).stop 2024-04-26T23:50:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 msgr2=0x7ffa3015ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b3760 tx=0x7ffa2800b480 comp rx=0 tx=0).stop 2024-04-26T23:50:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 shutdown_connections 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa0c06c920 0x7ffa0c06ede0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa3015e820 0x7ffa3015ecb0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa301405b0 0x7ffa30158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 --2- 172.21.15.46:0/2269243446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14003f80 0x7ffa30158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.876+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 >> 172.21.15.46:0/2269243446 conn(0x7ffa3006c980 msgr2=0x7ffa300caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.877+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 shutdown_connections 2024-04-26T23:50:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:02.877+0000 7ffa36a00640 1 -- 172.21.15.46:0/2269243446 wait complete. 2024-04-26T23:50:02.906 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:50:02.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:50:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.126+0000 7f373e400640 1 Processor -- start 2024-04-26T23:50:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.126+0000 7f373e400640 1 -- start start 2024-04-26T23:50:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.127+0000 7f373e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37381626c0 0x7f37380b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.127+0000 7f373e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37380b5fb0 0x7f37380d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.127+0000 7f373e400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37380d07d0 con 0x7f3738162aa0 2024-04-26T23:50:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.127+0000 7f373e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f37380d0ab0 con 0x7f37381626c0 2024-04-26T23:50:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.127+0000 7f373e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37380d0d90 con 0x7f37380b5fb0 2024-04-26T23:50:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37381626c0 0x7f37380b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f37381626c0 0x7f37380b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53992/0 (socket says 172.21.15.46:53992) 2024-04-26T23:50:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 -- 172.21.15.46:0/3980540809 learned_addr learned my addr 172.21.15.46:0/3980540809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f3737e00640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37380b5fb0 0x7f37380d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 -- 172.21.15.46:0/3980540809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37380b5fb0 msgr2=0x7f37380d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37380b5fb0 0x7f37380d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.128+0000 7f373d000640 1 -- 172.21.15.46:0/3980540809 >> v1:172.21.15.142:6789/0 conn(0x7f3738162aa0 legacy=0x7f37380d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.129+0000 7f373d000640 1 -- 172.21.15.46:0/3980540809 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37380d2c80 con 0x7f37381626c0 2024-04-26T23:50:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.130+0000 7f373d000640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f37380b47c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f3728009870 tx=0x7f372802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47548818e03efd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.130+0000 7f3737400640 1 -- 172.21.15.46:0/3980540809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728037070 con 0x7f37381626c0 2024-04-26T23:50:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.130+0000 7f3737400640 1 -- 172.21.15.46:0/3980540809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3728032440 con 0x7f37381626c0 2024-04-26T23:50:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.130+0000 7f3737400640 1 -- 172.21.15.46:0/3980540809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728032810 con 0x7f37381626c0 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 -- 172.21.15.46:0/3980540809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 msgr2=0x7f37380b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f37380b47c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f3728009870 tx=0x7f372802a250 comp rx=0 tx=0).stop 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 -- 172.21.15.46:0/3980540809 shutdown_connections 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37380b5fb0 0x7f37380d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 --2- 172.21.15.46:0/3980540809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f37380b47c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 -- 172.21.15.46:0/3980540809 >> 172.21.15.46:0/3980540809 conn(0x7f373809e2a0 msgr2=0x7f37380d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 -- 172.21.15.46:0/3980540809 shutdown_connections 2024-04-26T23:50:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.131+0000 7f373e400640 1 -- 172.21.15.46:0/3980540809 wait complete. 2024-04-26T23:50:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.133+0000 7f373e400640 1 Processor -- start 2024-04-26T23:50:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.133+0000 7f373e400640 1 -- start start 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 0x7f37380b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3738162aa0 0x7f373815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37380d2190 con 0x7f37381626c0 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37380d2010 con 0x7f37380b5fb0 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f373e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37380d31a0 con 0x7f3738162aa0 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f3737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f3737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53994/0 (socket says 172.21.15.46:53994) 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.134+0000 7f3737e00640 1 -- 172.21.15.46:0/3017725771 learned_addr learned my addr 172.21.15.46:0/3017725771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f373d000640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 0x7f37380b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f373da00640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3738162aa0 0x7f373815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 -- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3738162aa0 msgr2=0x7f373815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3738162aa0 0x7f373815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 -- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 msgr2=0x7f37380b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 0x7f37380b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 -- 172.21.15.46:0/3017725771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373815afc0 con 0x7f37381626c0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f373d000640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 0x7f37380b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3737e00640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3724009fc0 tx=0x7f372400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724014620 con 0x7f37381626c0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724019070 con 0x7f37381626c0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724014a00 con 0x7f37381626c0 2024-04-26T23:50:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.135+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373815b2a0 con 0x7f37381626c0 2024-04-26T23:50:03.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.136+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37380ba440 con 0x7f37381626c0 2024-04-26T23:50:03.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.137+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37380693b0 con 0x7f37381626c0 2024-04-26T23:50:03.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.138+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3724014ba0 con 0x7f37381626c0 2024-04-26T23:50:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.138+0000 7f3734c00640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 0x7f371806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.139+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f372408c660 con 0x7f37381626c0 2024-04-26T23:50:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.139+0000 7f373d000640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 0x7f371806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.140+0000 7f373d000640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 0x7f371806edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3728034000 tx=0x7f372802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:03.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.143+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f372405a8f0 con 0x7f37381626c0 2024-04-26T23:50:03.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.400+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f37380cd270 con 0x7f37381626c0 2024-04-26T23:50:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.875+0000 7f3734c00640 1 -- 172.21.15.46:0/3017725771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v109) ==== 122+0+0 (secure 0 0 0) 0x7f372405a1f0 con 0x7f37381626c0 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 msgr2=0x7f371806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 0x7f371806edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3728034000 tx=0x7f372802e000 comp rx=0 tx=0).stop 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 msgr2=0x7f3738158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3724009fc0 tx=0x7f372400e570 comp rx=0 tx=0).stop 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 shutdown_connections 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f371806c8f0 0x7f371806edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3738162aa0 0x7f373815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37381626c0 0x7f3738158380 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 --2- 172.21.15.46:0/3017725771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37380b5fb0 0x7f37380b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 >> 172.21.15.46:0/3017725771 conn(0x7f373809e2a0 msgr2=0x7f3738161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 shutdown_connections 2024-04-26T23:50:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:03.879+0000 7f373e400640 1 -- 172.21.15.46:0/3017725771 wait complete. 2024-04-26T23:50:03.910 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:50:03.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:04.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.113+0000 7f53c9200640 1 Processor -- start 2024-04-26T23:50:04.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.113+0000 7f53c9200640 1 -- start start 2024-04-26T23:50:04.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.114+0000 7f53c9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53c41466b0 0x7f53c406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c4073ad0 con 0x7f53c40720e0 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c9200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c4146a90 con 0x7f53c4160910 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f53c4073d70 con 0x7f53c41466b0 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53c41466b0 0x7f53c406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53c41466b0 0x7f53c406de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54008/0 (socket says 172.21.15.46:54008) 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 -- 172.21.15.46:0/2093973414 learned_addr learned my addr 172.21.15.46:0/2093973414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 -- 172.21.15.46:0/2093973414 >> v1:172.21.15.142:6789/0 conn(0x7f53c4160910 legacy=0x7f53c40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 -- 172.21.15.46:0/2093973414 >> v1:172.21.15.46:6789/0 conn(0x7f53c40720e0 legacy=0x7f53c40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.115+0000 7f53c3400640 1 -- 172.21.15.46:0/2093973414 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c4074050 con 0x7f53c41466b0 2024-04-26T23:50:04.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.116+0000 7f53c3400640 1 --2- 172.21.15.46:0/2093973414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c41466b0 0x7f53c406de80 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9542919d35c414ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c2000640 1 -- 172.21.15.46:0/2093973414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4037070 con 0x7f53c41466b0 2024-04-26T23:50:04.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c2000640 1 -- 172.21.15.46:0/2093973414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b4032440 con 0x7f53c41466b0 2024-04-26T23:50:04.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c2000640 1 -- 172.21.15.46:0/2093973414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4032810 con 0x7f53c41466b0 2024-04-26T23:50:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 -- 172.21.15.46:0/2093973414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c41466b0 msgr2=0x7f53c406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 --2- 172.21.15.46:0/2093973414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c41466b0 0x7f53c406de80 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b402a2e0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 -- 172.21.15.46:0/2093973414 shutdown_connections 2024-04-26T23:50:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 --2- 172.21.15.46:0/2093973414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c41466b0 0x7f53c406de80 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 -- 172.21.15.46:0/2093973414 >> 172.21.15.46:0/2093973414 conn(0x7f53c40b26d0 msgr2=0x7f53c40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:04.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.117+0000 7f53c9200640 1 -- 172.21.15.46:0/2093973414 shutdown_connections 2024-04-26T23:50:04.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.118+0000 7f53c9200640 1 -- 172.21.15.46:0/2093973414 wait complete. 2024-04-26T23:50:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.119+0000 7f53c9200640 1 Processor -- start 2024-04-26T23:50:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.119+0000 7f53c9200640 1 -- start start 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c41466b0 0x7f53c413e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 0x7f53c41e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53c406e3c0 con 0x7f53c40720e0 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40cb470 con 0x7f53c4160910 2024-04-26T23:50:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.120+0000 7f53c9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f53c40cc0a0 con 0x7f53c41466b0 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54018/0 (socket says 172.21.15.46:54018) 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 -- 172.21.15.46:0/1282585645 learned_addr learned my addr 172.21.15.46:0/1282585645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c2a00640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c41466b0 0x7f53c413e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 -- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c41466b0 msgr2=0x7f53c413e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c41466b0 0x7f53c413e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 -- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 msgr2=0x7f53c41e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3e00640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 0x7f53c41e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 0x7f53c41e8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3400640 1 -- 172.21.15.46:0/1282585645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c40b4160 con 0x7f53c40720e0 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.121+0000 7f53c3e00640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 0x7f53c41e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53c3400640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4037040 con 0x7f53c40720e0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b4004230 con 0x7f53c40720e0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c40b4440 con 0x7f53c40720e0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b403c670 con 0x7f53c40720e0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.122+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c40b4a00 con 0x7f53c40720e0 2024-04-26T23:50:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.123+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c40693b0 con 0x7f53c40720e0 2024-04-26T23:50:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.124+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53b403a070 con 0x7f53c40720e0 2024-04-26T23:50:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.125+0000 7f53bb400640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 0x7f53a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.126+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f53b40ac470 con 0x7f53c40720e0 2024-04-26T23:50:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.126+0000 7f53c2a00640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 0x7f53a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.127+0000 7f53c2a00640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 0x7f53a406ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f53c40d1240 tx=0x7f53b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.130+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53b4041020 con 0x7f53c40720e0 2024-04-26T23:50:04.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.376+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53c40c7180 con 0x7f53c40720e0 2024-04-26T23:50:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.377+0000 7f53bb400640 1 -- 172.21.15.46:0/1282585645 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f53b407a700 con 0x7f53c40720e0 2024-04-26T23:50:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:04.383 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:04.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 msgr2=0x7f53a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 0x7f53a406ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f53c40d1240 tx=0x7f53b0009040 comp rx=0 tx=0).stop 2024-04-26T23:50:04.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 msgr2=0x7f53c40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b40046c0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 shutdown_connections 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f53a406c920 0x7f53a406ede0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c4160910 0x7f53c41e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c41466b0 0x7f53c413e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 --2- 172.21.15.46:0/1282585645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c40720e0 0x7f53c40b5da0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 >> 172.21.15.46:0/1282585645 conn(0x7f53c40b26d0 msgr2=0x7f53c40c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.380+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 shutdown_connections 2024-04-26T23:50:04.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.381+0000 7f53c9200640 1 -- 172.21.15.46:0/1282585645 wait complete. 2024-04-26T23:50:04.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:50:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.617+0000 7f0818e00640 1 Processor -- start 2024-04-26T23:50:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.617+0000 7f0818e00640 1 -- start start 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0818e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08140b5530 0x7f08140ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0818e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08140ce8e0 0x7f081409be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0818e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08140d0c90 con 0x7f081409ba70 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0818e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f08140b59c0 con 0x7f08140b5530 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0818e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f08140b3f60 con 0x7f08140ce8e0 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0811400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08140b5530 0x7f08140ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0811400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08140b5530 0x7f08140ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54032/0 (socket says 172.21.15.46:54032) 2024-04-26T23:50:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.619+0000 7f0811400640 1 -- 172.21.15.46:0/3393615280 learned_addr learned my addr 172.21.15.46:0/3393615280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.620+0000 7f0811400640 1 -- 172.21.15.46:0/3393615280 >> v2:172.21.15.142:3300/0 conn(0x7f08140ce8e0 msgr2=0x7f081409be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.620+0000 7f0811400640 1 --2- 172.21.15.46:0/3393615280 >> v2:172.21.15.142:3300/0 conn(0x7f08140ce8e0 0x7f081409be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.620+0000 7f0811400640 1 -- 172.21.15.46:0/3393615280 >> v1:172.21.15.46:6789/0 conn(0x7f081409ba70 legacy=0x7f08140b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.620+0000 7f0811400640 1 -- 172.21.15.46:0/3393615280 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140d0f30 con 0x7f08140b5530 2024-04-26T23:50:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.621+0000 7f0811400640 1 --2- 172.21.15.46:0/3393615280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140b5530 0x7f08140ce3a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f08140cf320 tx=0x7f080002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d41929962fa42b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.621+0000 7f0810a00640 1 -- 172.21.15.46:0/3393615280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800037070 con 0x7f08140b5530 2024-04-26T23:50:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.621+0000 7f0810a00640 1 -- 172.21.15.46:0/3393615280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800032440 con 0x7f08140b5530 2024-04-26T23:50:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.621+0000 7f0810a00640 1 -- 172.21.15.46:0/3393615280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800032810 con 0x7f08140b5530 2024-04-26T23:50:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 -- 172.21.15.46:0/3393615280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140b5530 msgr2=0x7f08140ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 --2- 172.21.15.46:0/3393615280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140b5530 0x7f08140ce3a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f08140cf320 tx=0x7f080002a270 comp rx=0 tx=0).stop 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 -- 172.21.15.46:0/3393615280 shutdown_connections 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 --2- 172.21.15.46:0/3393615280 >> v2:172.21.15.142:3300/0 conn(0x7f08140ce8e0 0x7f081409be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 --2- 172.21.15.46:0/3393615280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140b5530 0x7f08140ce3a0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 -- 172.21.15.46:0/3393615280 >> 172.21.15.46:0/3393615280 conn(0x7f08140b4a10 msgr2=0x7f08140b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 -- 172.21.15.46:0/3393615280 shutdown_connections 2024-04-26T23:50:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.622+0000 7f0818e00640 1 -- 172.21.15.46:0/3393615280 wait complete. 2024-04-26T23:50:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.624+0000 7f0818e00640 1 Processor -- start 2024-04-26T23:50:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.624+0000 7f0818e00640 1 -- start start 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081409ba70 0x7f0814159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08140b5530 0x7f08140c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08140cdf10 con 0x7f08140ce8e0 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08140d1210 con 0x7f081409ba70 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0818e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08140cecf0 con 0x7f08140b5530 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0812800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.625+0000 7f0812800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54042/0 (socket says 172.21.15.46:54042) 2024-04-26T23:50:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 -- 172.21.15.46:0/4210577093 learned_addr learned my addr 172.21.15.46:0/4210577093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0811e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081409ba70 0x7f0814159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 -- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08140b5530 msgr2=0x7f08140c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08140b5530 0x7f08140c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 -- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081409ba70 msgr2=0x7f0814159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081409ba70 0x7f0814159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 -- 172.21.15.46:0/4210577093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140b7c40 con 0x7f08140ce8e0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f0812800640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f080c00b970 tx=0x7f080c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.626+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c019070 con 0x7f08140ce8e0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.627+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c00cb60 con 0x7f08140ce8e0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.627+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c014440 con 0x7f08140ce8e0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.627+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081415a050 con 0x7f08140ce8e0 2024-04-26T23:50:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.627+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08140b8700 con 0x7f08140ce8e0 2024-04-26T23:50:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.628+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0814069530 con 0x7f08140ce8e0 2024-04-26T23:50:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.629+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f080c01c070 con 0x7f08140ce8e0 2024-04-26T23:50:04.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.630+0000 7f080a000640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 0x7f07f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:04.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.630+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f080c08ba30 con 0x7f08140ce8e0 2024-04-26T23:50:04.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.631+0000 7f0811e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 0x7f07f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:04.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.632+0000 7f0811e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 0x7f07f406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0804007fd0 tx=0x7f0804009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.634+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f080c059cc0 con 0x7f08140ce8e0 2024-04-26T23:50:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.837+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0814074590 con 0x7f08140ce8e0 2024-04-26T23:50:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.839+0000 7f080a000640 1 -- 172.21.15.46:0/4210577093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16864 (secure 0 0 0) 0x7f080c0595c0 con 0x7f08140ce8e0 2024-04-26T23:50:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:04.844 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":137,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:01.836702+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 msgr2=0x7f07f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 0x7f07f406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0804007fd0 tx=0x7f0804009040 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 msgr2=0x7f08140b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f080c00b970 tx=0x7f080c00be30 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 shutdown_connections 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f07f406c8f0 0x7f07f406edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08140ce8e0 0x7f08140b09f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08140b5530 0x7f08140c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 --2- 172.21.15.46:0/4210577093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081409ba70 0x7f0814159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 >> 172.21.15.46:0/4210577093 conn(0x7f08140b4a10 msgr2=0x7f08140afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 shutdown_connections 2024-04-26T23:50:04.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:04.842+0000 7f0818e00640 1 -- 172.21.15.46:0/4210577093 wait complete. 2024-04-26T23:50:04.866 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:50:04.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:50:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.070+0000 7fe61ce00640 1 Processor -- start 2024-04-26T23:50:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.070+0000 7fe61ce00640 1 -- start start 2024-04-26T23:50:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.071+0000 7fe61ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 0x7fe61806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.071+0000 7fe61ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe6180720e0 0x7fe6180cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.071+0000 7fe61ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe618073ad0 con 0x7fe618160910 2024-04-26T23:50:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.071+0000 7fe61ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe618146a90 con 0x7fe6180720e0 2024-04-26T23:50:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.071+0000 7fe61ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe618073d70 con 0x7fe6181466b0 2024-04-26T23:50:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe617e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe618160910 0x7fe6180cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47718/0 (socket says 172.21.15.46:47718) 2024-04-26T23:50:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe617e00640 1 -- 172.21.15.46:0/2256177895 learned_addr learned my addr 172.21.15.46:0/2256177895 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe616a00640 1 --2- 172.21.15.46:0/2256177895 >> v2:172.21.15.142:3300/0 conn(0x7fe6180720e0 0x7fe6180cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe617400640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 0x7fe61806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881376591 0 0) 0x7fe618073ad0 con 0x7fe618160910 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.072+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5fc003520 con 0x7fe618160910 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152804577 0 0) 0x7fe5fc003520 con 0x7fe618160910 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 msgr2=0x7fe61806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 0x7fe61806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 >> v2:172.21.15.142:3300/0 conn(0x7fe6180720e0 msgr2=0x7fe6180cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 --2- 172.21.15.46:0/2256177895 >> v2:172.21.15.142:3300/0 conn(0x7fe6180720e0 0x7fe6180cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618074050 con 0x7fe618160910 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe610001670 con 0x7fe618160910 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 >> v1:172.21.15.46:6789/0 conn(0x7fe618160910 legacy=0x7fe6180cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5fc0039b0 0x7fe5fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 0x7fe5fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5fc004d40 0x7fe5fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe618146a90 con 0x7fe5fc0039b0 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe618073d70 con 0x7fe5fc004d40 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc00b9c0 con 0x7fe6181466b0 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.073+0000 7fe617400640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5fc0039b0 0x7fe5fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.074+0000 7fe617400640 1 -- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 msgr2=0x7fe5fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.074+0000 7fe617400640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6181466b0 0x7fe5fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.074+0000 7fe617400640 1 -- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5fc004d40 msgr2=0x7fe5fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.074+0000 7fe617400640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5fc004d40 0x7fe5fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.074+0000 7fe617400640 1 -- 172.21.15.46:0/2256177895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc003520 con 0x7fe5fc0039b0 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe617400640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5fc0039b0 0x7fe5fc003dd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe608009fc0 tx=0x7fe5fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a705eca84fd31913 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608009c10 con 0x7fe5fc0039b0 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608036070 con 0x7fe5fc0039b0 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe616000640 1 -- 172.21.15.46:0/2256177895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60802d070 con 0x7fe5fc0039b0 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe61ce00640 1 -- 172.21.15.46:0/2256177895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5fc0039b0 msgr2=0x7fe5fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe61ce00640 1 --2- 172.21.15.46:0/2256177895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5fc0039b0 0x7fe5fc003dd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe608009fc0 tx=0x7fe5fc005830 comp rx=0 tx=0).stop 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.075+0000 7fe617e00640 1 -- 172.21.15.46:0/2256177895 reap_dead start 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.076+0000 7fe61ce00640 1 -- 172.21.15.46:0/2256177895 shutdown_connections 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.076+0000 7fe61ce00640 1 -- 172.21.15.46:0/2256177895 >> 172.21.15.46:0/2256177895 conn(0x7fe6180b26d0 msgr2=0x7fe6180b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.076+0000 7fe61ce00640 1 -- 172.21.15.46:0/2256177895 shutdown_connections 2024-04-26T23:50:05.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.076+0000 7fe61ce00640 1 -- 172.21.15.46:0/2256177895 wait complete. 2024-04-26T23:50:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.077+0000 7fe61ce00640 1 Processor -- start 2024-04-26T23:50:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.078+0000 7fe61ce00640 1 -- start start 2024-04-26T23:50:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.078+0000 7fe61ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5fc0039b0 0x7fe6181e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.078+0000 7fe61ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6180b3710 0x7fe6181f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe61ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe61ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180cdda0 con 0x7fe6180b5000 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe61ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180cb470 con 0x7fe6180b3710 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe61ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe6180cc0a0 con 0x7fe5fc0039b0 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54072/0 (socket says 172.21.15.46:54072) 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 -- 172.21.15.46:0/4228670831 learned_addr learned my addr 172.21.15.46:0/4228670831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617400640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5fc0039b0 0x7fe6181e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 -- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5fc0039b0 msgr2=0x7fe6181e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5fc0039b0 0x7fe6181e8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 -- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6180b3710 msgr2=0x7fe6181f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6180b3710 0x7fe6181f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.079+0000 7fe617e00640 1 -- 172.21.15.46:0/4228670831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6180b3f60 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe617e00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe618160cf0 tx=0x7fe61000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610004bf0 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610004d90 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610007270 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6180b4240 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.080+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6180b4800 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.081+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6180693b0 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.082+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe610007410 con 0x7fe6180b5000 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.083+0000 7fe60f400640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 0x7fe5f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.083+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fe61008b350 con 0x7fe6180b5000 2024-04-26T23:50:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.084+0000 7fe617400640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 0x7fe5f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.085+0000 7fe617400640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 0x7fe5f406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe608035000 tx=0x7fe60802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.087+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe61001a020 con 0x7fe6180b5000 2024-04-26T23:50:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.281+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe618161200 con 0x7fe5f406c8f0 2024-04-26T23:50:05.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.300+0000 7fe60f400640 1 -- 172.21.15.46:0/4228670831 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252412 (secure 0 0 0) 0x7fe618161200 con 0x7fe5f406c8f0 2024-04-26T23:50:05.310 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.304+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 msgr2=0x7fe5f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.304+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 0x7fe5f406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe608035000 tx=0x7fe60802f000 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 msgr2=0x7fe6180b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe618160cf0 tx=0x7fe61000c960 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 shutdown_connections 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe5f406c8f0 0x7fe5f406edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6180b5000 0x7fe6180b5480 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6180b3710 0x7fe6181f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 --2- 172.21.15.46:0/4228670831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5fc0039b0 0x7fe6181e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 >> 172.21.15.46:0/4228670831 conn(0x7fe6180b26d0 msgr2=0x7fe6180ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 shutdown_connections 2024-04-26T23:50:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.305+0000 7fe61ce00640 1 -- 172.21.15.46:0/4228670831 wait complete. 2024-04-26T23:50:05.328 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-04-26T23:50:04.178283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402624,"kb_used_data":14364,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754572096,"statfs":{"total":773094113280,"available":772681826304,"internally_reserved":0,"allocated":14708736,"data_stored":7522735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":397463089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.380394"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856038+0000","last_change":"2024-04-26T23:50:00.850781+0000","last_active":"2024-04-26T23:50:00.856038+0000","last_peered":"2024-04-26T23:50:00.856038+0000","last_clean":"2024-04-26T23:50:00.856038+0000","last_became_active":"2024-04-26T23:50:00.850498+0000","last_became_peered":"2024-04-26T23:50:00.850498+0000","last_unstale":"2024-04-26T23:50:00.856038+0000","last_undegraded":"2024-04-26T23:50:00.856038+0000","last_fullsized":"2024-04-26T23:50:00.856038+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:58.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856011+0000","last_change":"2024-04-26T23:50:00.852406+0000","last_active":"2024-04-26T23:50:00.856011+0000","last_peered":"2024-04-26T23:50:00.856011+0000","last_clean":"2024-04-26T23:50:00.856011+0000","last_became_active":"2024-04-26T23:50:00.851793+0000","last_became_peered":"2024-04-26T23:50:00.851793+0000","last_unstale":"2024-04-26T23:50:00.856011+0000","last_undegraded":"2024-04-26T23:50:00.856011+0000","last_fullsized":"2024-04-26T23:50:00.856011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:54.152303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.870914+0000","last_change":"2024-04-26T23:50:00.862668+0000","last_active":"2024-04-26T23:50:00.870914+0000","last_peered":"2024-04-26T23:50:00.870914+0000","last_clean":"2024-04-26T23:50:00.870914+0000","last_became_active":"2024-04-26T23:50:00.862345+0000","last_became_peered":"2024-04-26T23:50:00.862345+0000","last_unstale":"2024-04-26T23:50:00.870914+0000","last_undegraded":"2024-04-26T23:50:00.870914+0000","last_fullsized":"2024-04-26T23:50:00.870914+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:59.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856359+0000","last_change":"2024-04-26T23:50:00.850445+0000","last_active":"2024-04-26T23:50:00.856359+0000","last_peered":"2024-04-26T23:50:00.856359+0000","last_clean":"2024-04-26T23:50:00.856359+0000","last_became_active":"2024-04-26T23:50:00.850104+0000","last_became_peered":"2024-04-26T23:50:00.850104+0000","last_unstale":"2024-04-26T23:50:00.856359+0000","last_undegraded":"2024-04-26T23:50:00.856359+0000","last_fullsized":"2024-04-26T23:50:00.856359+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:11.058674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.845460+0000","last_change":"2024-04-26T23:50:00.840694+0000","last_active":"2024-04-26T23:50:00.845460+0000","last_peered":"2024-04-26T23:50:00.845460+0000","last_clean":"2024-04-26T23:50:00.845460+0000","last_became_active":"2024-04-26T23:50:00.839932+0000","last_became_peered":"2024-04-26T23:50:00.839932+0000","last_unstale":"2024-04-26T23:50:00.845460+0000","last_undegraded":"2024-04-26T23:50:00.845460+0000","last_fullsized":"2024-04-26T23:50:00.845460+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:23.124347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862067+0000","last_change":"2024-04-26T23:50:00.855874+0000","last_active":"2024-04-26T23:50:00.862067+0000","last_peered":"2024-04-26T23:50:00.862067+0000","last_clean":"2024-04-26T23:50:00.862067+0000","last_became_active":"2024-04-26T23:50:00.855506+0000","last_became_peered":"2024-04-26T23:50:00.855506+0000","last_unstale":"2024-04-26T23:50:00.862067+0000","last_undegraded":"2024-04-26T23:50:00.862067+0000","last_fullsized":"2024-04-26T23:50:00.862067+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:21.521273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869357+0000","last_change":"2024-04-26T23:50:00.862259+0000","last_active":"2024-04-26T23:50:00.869357+0000","last_peered":"2024-04-26T23:50:00.869357+0000","last_clean":"2024-04-26T23:50:00.869357+0000","last_became_active":"2024-04-26T23:50:00.861949+0000","last_became_peered":"2024-04-26T23:50:00.861949+0000","last_unstale":"2024-04-26T23:50:00.869357+0000","last_undegraded":"2024-04-26T23:50:00.869357+0000","last_fullsized":"2024-04-26T23:50:00.869357+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:20.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861079+0000","last_change":"2024-04-26T23:50:00.850822+0000","last_active":"2024-04-26T23:50:00.861079+0000","last_peered":"2024-04-26T23:50:00.861079+0000","last_clean":"2024-04-26T23:50:00.861079+0000","last_became_active":"2024-04-26T23:50:00.850496+0000","last_became_peered":"2024-04-26T23:50:00.850496+0000","last_unstale":"2024-04-26T23:50:00.861079+0000","last_undegraded":"2024-04-26T23:50:00.861079+0000","last_fullsized":"2024-04-26T23:50:00.861079+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.997046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856892+0000","last_change":"2024-04-26T23:50:00.854760+0000","last_active":"2024-04-26T23:50:00.856892+0000","last_peered":"2024-04-26T23:50:00.856892+0000","last_clean":"2024-04-26T23:50:00.856892+0000","last_became_active":"2024-04-26T23:50:00.854402+0000","last_became_peered":"2024-04-26T23:50:00.854402+0000","last_unstale":"2024-04-26T23:50:00.856892+0000","last_undegraded":"2024-04-26T23:50:00.856892+0000","last_fullsized":"2024-04-26T23:50:00.856892+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:50.313727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.837663+0000","last_change":"2024-04-26T23:50:00.834728+0000","last_active":"2024-04-26T23:50:00.837663+0000","last_peered":"2024-04-26T23:50:00.837663+0000","last_clean":"2024-04-26T23:50:00.837663+0000","last_became_active":"2024-04-26T23:50:00.833909+0000","last_became_peered":"2024-04-26T23:50:00.833909+0000","last_unstale":"2024-04-26T23:50:00.837663+0000","last_undegraded":"2024-04-26T23:50:00.837663+0000","last_fullsized":"2024-04-26T23:50:00.837663+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:23.438270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.859711+0000","last_change":"2024-04-26T23:50:00.852245+0000","last_active":"2024-04-26T23:50:00.859711+0000","last_peered":"2024-04-26T23:50:00.859711+0000","last_clean":"2024-04-26T23:50:00.859711+0000","last_became_active":"2024-04-26T23:50:00.850761+0000","last_became_peered":"2024-04-26T23:50:00.850761+0000","last_unstale":"2024-04-26T23:50:00.859711+0000","last_undegraded":"2024-04-26T23:50:00.859711+0000","last_fullsized":"2024-04-26T23:50:00.859711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:38.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860334+0000","last_change":"2024-04-26T23:50:00.857097+0000","last_active":"2024-04-26T23:50:00.860334+0000","last_peered":"2024-04-26T23:50:00.860334+0000","last_clean":"2024-04-26T23:50:00.860334+0000","last_became_active":"2024-04-26T23:50:00.853749+0000","last_became_peered":"2024-04-26T23:50:00.853749+0000","last_unstale":"2024-04-26T23:50:00.860334+0000","last_undegraded":"2024-04-26T23:50:00.860334+0000","last_fullsized":"2024-04-26T23:50:00.860334+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:33.044388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861309+0000","last_change":"2024-04-26T23:50:00.855842+0000","last_active":"2024-04-26T23:50:00.861309+0000","last_peered":"2024-04-26T23:50:00.861309+0000","last_clean":"2024-04-26T23:50:00.861309+0000","last_became_active":"2024-04-26T23:50:00.855498+0000","last_became_peered":"2024-04-26T23:50:00.855498+0000","last_unstale":"2024-04-26T23:50:00.861309+0000","last_undegraded":"2024-04-26T23:50:00.861309+0000","last_fullsized":"2024-04-26T23:50:00.861309+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:51.918115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862100+0000","last_change":"2024-04-26T23:50:00.858206+0000","last_active":"2024-04-26T23:50:00.862100+0000","last_peered":"2024-04-26T23:50:00.862100+0000","last_clean":"2024-04-26T23:50:00.862100+0000","last_became_active":"2024-04-26T23:50:00.857798+0000","last_became_peered":"2024-04-26T23:50:00.857798+0000","last_unstale":"2024-04-26T23:50:00.862100+0000","last_undegraded":"2024-04-26T23:50:00.862100+0000","last_fullsized":"2024-04-26T23:50:00.862100+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:32.671059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866567+0000","last_change":"2024-04-26T23:50:00.862637+0000","last_active":"2024-04-26T23:50:00.866567+0000","last_peered":"2024-04-26T23:50:00.866567+0000","last_clean":"2024-04-26T23:50:00.866567+0000","last_became_active":"2024-04-26T23:50:00.862333+0000","last_became_peered":"2024-04-26T23:50:00.862333+0000","last_unstale":"2024-04-26T23:50:00.866567+0000","last_undegraded":"2024-04-26T23:50:00.866567+0000","last_fullsized":"2024-04-26T23:50:00.866567+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:47.176999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.864545+0000","last_change":"2024-04-26T23:50:00.863203+0000","last_active":"2024-04-26T23:50:00.864545+0000","last_peered":"2024-04-26T23:50:00.864545+0000","last_clean":"2024-04-26T23:50:00.864545+0000","last_became_active":"2024-04-26T23:50:00.862922+0000","last_became_peered":"2024-04-26T23:50:00.862922+0000","last_unstale":"2024-04-26T23:50:00.864545+0000","last_undegraded":"2024-04-26T23:50:00.864545+0000","last_fullsized":"2024-04-26T23:50:00.864545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:16.103199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869840+0000","last_change":"2024-04-26T23:50:00.866483+0000","last_active":"2024-04-26T23:50:00.869840+0000","last_peered":"2024-04-26T23:50:00.869840+0000","last_clean":"2024-04-26T23:50:00.869840+0000","last_became_active":"2024-04-26T23:50:00.866173+0000","last_became_peered":"2024-04-26T23:50:00.866173+0000","last_unstale":"2024-04-26T23:50:00.869840+0000","last_undegraded":"2024-04-26T23:50:00.869840+0000","last_fullsized":"2024-04-26T23:50:00.869840+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:38.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860891+0000","last_change":"2024-04-26T23:50:00.849108+0000","last_active":"2024-04-26T23:50:00.860891+0000","last_peered":"2024-04-26T23:50:00.860891+0000","last_clean":"2024-04-26T23:50:00.860891+0000","last_became_active":"2024-04-26T23:50:00.848323+0000","last_became_peered":"2024-04-26T23:50:00.848323+0000","last_unstale":"2024-04-26T23:50:00.860891+0000","last_undegraded":"2024-04-26T23:50:00.860891+0000","last_fullsized":"2024-04-26T23:50:00.860891+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:50.783226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871297+0000","last_change":"2024-04-26T23:50:00.865265+0000","last_active":"2024-04-26T23:50:00.871297+0000","last_peered":"2024-04-26T23:50:00.871297+0000","last_clean":"2024-04-26T23:50:00.871297+0000","last_became_active":"2024-04-26T23:50:00.864951+0000","last_became_peered":"2024-04-26T23:50:00.864951+0000","last_unstale":"2024-04-26T23:50:00.871297+0000","last_undegraded":"2024-04-26T23:50:00.871297+0000","last_fullsized":"2024-04-26T23:50:00.871297+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:35.487834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861944+0000","last_change":"2024-04-26T23:50:00.854900+0000","last_active":"2024-04-26T23:50:00.861944+0000","last_peered":"2024-04-26T23:50:00.861944+0000","last_clean":"2024-04-26T23:50:00.861944+0000","last_became_active":"2024-04-26T23:50:00.854633+0000","last_became_peered":"2024-04-26T23:50:00.854633+0000","last_unstale":"2024-04-26T23:50:00.861944+0000","last_undegraded":"2024-04-26T23:50:00.861944+0000","last_fullsized":"2024-04-26T23:50:00.861944+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:55.501626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862541+0000","last_change":"2024-04-26T23:50:00.857815+0000","last_active":"2024-04-26T23:50:00.862541+0000","last_peered":"2024-04-26T23:50:00.862541+0000","last_clean":"2024-04-26T23:50:00.862541+0000","last_became_active":"2024-04-26T23:50:00.857119+0000","last_became_peered":"2024-04-26T23:50:00.857119+0000","last_unstale":"2024-04-26T23:50:00.862541+0000","last_undegraded":"2024-04-26T23:50:00.862541+0000","last_fullsized":"2024-04-26T23:50:00.862541+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:24.422976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862077+0000","last_change":"2024-04-26T23:50:00.855918+0000","last_active":"2024-04-26T23:50:00.862077+0000","last_peered":"2024-04-26T23:50:00.862077+0000","last_clean":"2024-04-26T23:50:00.862077+0000","last_became_active":"2024-04-26T23:50:00.855623+0000","last_became_peered":"2024-04-26T23:50:00.855623+0000","last_unstale":"2024-04-26T23:50:00.862077+0000","last_undegraded":"2024-04-26T23:50:00.862077+0000","last_fullsized":"2024-04-26T23:50:00.862077+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:25.000532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860225+0000","last_change":"2024-04-26T23:50:00.857018+0000","last_active":"2024-04-26T23:50:00.860225+0000","last_peered":"2024-04-26T23:50:00.860225+0000","last_clean":"2024-04-26T23:50:00.860225+0000","last_became_active":"2024-04-26T23:50:00.853324+0000","last_became_peered":"2024-04-26T23:50:00.853324+0000","last_unstale":"2024-04-26T23:50:00.860225+0000","last_undegraded":"2024-04-26T23:50:00.860225+0000","last_fullsized":"2024-04-26T23:50:00.860225+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:50.057083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.855283+0000","last_change":"2024-04-26T23:50:00.853767+0000","last_active":"2024-04-26T23:50:00.855283+0000","last_peered":"2024-04-26T23:50:00.855283+0000","last_clean":"2024-04-26T23:50:00.855283+0000","last_became_active":"2024-04-26T23:50:00.853330+0000","last_became_peered":"2024-04-26T23:50:00.853330+0000","last_unstale":"2024-04-26T23:50:00.855283+0000","last_undegraded":"2024-04-26T23:50:00.855283+0000","last_fullsized":"2024-04-26T23:50:00.855283+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:42.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861373+0000","last_change":"2024-04-26T23:50:00.852079+0000","last_active":"2024-04-26T23:50:00.861373+0000","last_peered":"2024-04-26T23:50:00.861373+0000","last_clean":"2024-04-26T23:50:00.861373+0000","last_became_active":"2024-04-26T23:50:00.851724+0000","last_became_peered":"2024-04-26T23:50:00.851724+0000","last_unstale":"2024-04-26T23:50:00.861373+0000","last_undegraded":"2024-04-26T23:50:00.861373+0000","last_fullsized":"2024-04-26T23:50:00.861373+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:05.242256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872760+0000","last_change":"2024-04-26T23:50:00.867853+0000","last_active":"2024-04-26T23:50:00.872760+0000","last_peered":"2024-04-26T23:50:00.872760+0000","last_clean":"2024-04-26T23:50:00.872760+0000","last_became_active":"2024-04-26T23:50:00.867574+0000","last_became_peered":"2024-04-26T23:50:00.867574+0000","last_unstale":"2024-04-26T23:50:00.872760+0000","last_undegraded":"2024-04-26T23:50:00.872760+0000","last_fullsized":"2024-04-26T23:50:00.872760+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.997046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871274+0000","last_change":"2024-04-26T23:50:00.863800+0000","last_active":"2024-04-26T23:50:00.871274+0000","last_peered":"2024-04-26T23:50:00.871274+0000","last_clean":"2024-04-26T23:50:00.871274+0000","last_became_active":"2024-04-26T23:50:00.863208+0000","last_became_peered":"2024-04-26T23:50:00.863208+0000","last_unstale":"2024-04-26T23:50:00.871274+0000","last_undegraded":"2024-04-26T23:50:00.871274+0000","last_fullsized":"2024-04-26T23:50:00.871274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:13.535156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.839230+0000","last_change":"2024-04-26T23:50:00.837314+0000","last_active":"2024-04-26T23:50:00.839230+0000","last_peered":"2024-04-26T23:50:00.839230+0000","last_clean":"2024-04-26T23:50:00.839230+0000","last_became_active":"2024-04-26T23:50:00.836583+0000","last_became_peered":"2024-04-26T23:50:00.836583+0000","last_unstale":"2024-04-26T23:50:00.839230+0000","last_undegraded":"2024-04-26T23:50:00.839230+0000","last_fullsized":"2024-04-26T23:50:00.839230+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:52.883640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868145+0000","last_change":"2024-04-26T23:50:00.860950+0000","last_active":"2024-04-26T23:50:00.868145+0000","last_peered":"2024-04-26T23:50:00.868145+0000","last_clean":"2024-04-26T23:50:00.868145+0000","last_became_active":"2024-04-26T23:50:00.856281+0000","last_became_peered":"2024-04-26T23:50:00.856281+0000","last_unstale":"2024-04-26T23:50:00.868145+0000","last_undegraded":"2024-04-26T23:50:00.868145+0000","last_fullsized":"2024-04-26T23:50:00.868145+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:13.201883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.867505+0000","last_change":"2024-04-26T23:50:00.860447+0000","last_active":"2024-04-26T23:50:00.867505+0000","last_peered":"2024-04-26T23:50:00.867505+0000","last_clean":"2024-04-26T23:50:00.867505+0000","last_became_active":"2024-04-26T23:50:00.855300+0000","last_became_peered":"2024-04-26T23:50:00.855300+0000","last_unstale":"2024-04-26T23:50:00.867505+0000","last_undegraded":"2024-04-26T23:50:00.867505+0000","last_fullsized":"2024-04-26T23:50:00.867505+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:21.748505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861680+0000","last_change":"2024-04-26T23:50:00.860019+0000","last_active":"2024-04-26T23:50:00.861680+0000","last_peered":"2024-04-26T23:50:00.861680+0000","last_clean":"2024-04-26T23:50:00.861680+0000","last_became_active":"2024-04-26T23:50:00.858688+0000","last_became_peered":"2024-04-26T23:50:00.858688+0000","last_unstale":"2024-04-26T23:50:00.861680+0000","last_undegraded":"2024-04-26T23:50:00.861680+0000","last_fullsized":"2024-04-26T23:50:00.861680+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:42.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860688+0000","last_change":"2024-04-26T23:50:00.858160+0000","last_active":"2024-04-26T23:50:00.860688+0000","last_peered":"2024-04-26T23:50:00.860688+0000","last_clean":"2024-04-26T23:50:00.860688+0000","last_became_active":"2024-04-26T23:50:00.856402+0000","last_became_peered":"2024-04-26T23:50:00.856402+0000","last_unstale":"2024-04-26T23:50:00.860688+0000","last_undegraded":"2024-04-26T23:50:00.860688+0000","last_fullsized":"2024-04-26T23:50:00.860688+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.006515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861105+0000","last_change":"2024-04-26T23:50:00.853620+0000","last_active":"2024-04-26T23:50:00.861105+0000","last_peered":"2024-04-26T23:50:00.861105+0000","last_clean":"2024-04-26T23:50:00.861105+0000","last_became_active":"2024-04-26T23:50:00.853312+0000","last_became_peered":"2024-04-26T23:50:00.853312+0000","last_unstale":"2024-04-26T23:50:00.861105+0000","last_undegraded":"2024-04-26T23:50:00.861105+0000","last_fullsized":"2024-04-26T23:50:00.861105+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:38.189722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"136'4","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.877416+0000","last_change":"2024-04-26T23:49:59.852100+0000","last_active":"2024-04-26T23:50:00.877416+0000","last_peered":"2024-04-26T23:50:00.877416+0000","last_clean":"2024-04-26T23:50:00.877416+0000","last_became_active":"2024-04-26T23:49:59.851102+0000","last_became_peered":"2024-04-26T23:49:59.851102+0000","last_unstale":"2024-04-26T23:50:00.877416+0000","last_undegraded":"2024-04-26T23:50:00.877416+0000","last_fullsized":"2024-04-26T23:50:00.877416+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:35.312757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863701+0000","last_change":"2024-04-26T23:50:00.857753+0000","last_active":"2024-04-26T23:50:00.863701+0000","last_peered":"2024-04-26T23:50:00.863701+0000","last_clean":"2024-04-26T23:50:00.863701+0000","last_became_active":"2024-04-26T23:50:00.856361+0000","last_became_peered":"2024-04-26T23:50:00.856361+0000","last_unstale":"2024-04-26T23:50:00.863701+0000","last_undegraded":"2024-04-26T23:50:00.863701+0000","last_fullsized":"2024-04-26T23:50:00.863701+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:24.783686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873205+0000","last_change":"2024-04-26T23:49:59.851695+0000","last_active":"2024-04-26T23:50:00.873205+0000","last_peered":"2024-04-26T23:50:00.873205+0000","last_clean":"2024-04-26T23:50:00.873205+0000","last_became_active":"2024-04-26T23:49:59.850292+0000","last_became_peered":"2024-04-26T23:49:59.850292+0000","last_unstale":"2024-04-26T23:50:00.873205+0000","last_undegraded":"2024-04-26T23:50:00.873205+0000","last_fullsized":"2024-04-26T23:50:00.873205+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.054946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869404+0000","last_change":"2024-04-26T23:50:00.866785+0000","last_active":"2024-04-26T23:50:00.869404+0000","last_peered":"2024-04-26T23:50:00.869404+0000","last_clean":"2024-04-26T23:50:00.869404+0000","last_became_active":"2024-04-26T23:50:00.864403+0000","last_became_peered":"2024-04-26T23:50:00.864403+0000","last_unstale":"2024-04-26T23:50:00.869404+0000","last_undegraded":"2024-04-26T23:50:00.869404+0000","last_fullsized":"2024-04-26T23:50:00.869404+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:00.767988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872713+0000","last_change":"2024-04-26T23:49:59.849888+0000","last_active":"2024-04-26T23:50:00.872713+0000","last_peered":"2024-04-26T23:50:00.872713+0000","last_clean":"2024-04-26T23:50:00.872713+0000","last_became_active":"2024-04-26T23:49:59.849581+0000","last_became_peered":"2024-04-26T23:49:59.849581+0000","last_unstale":"2024-04-26T23:50:00.872713+0000","last_undegraded":"2024-04-26T23:50:00.872713+0000","last_fullsized":"2024-04-26T23:50:00.872713+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:45.577431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866958+0000","last_change":"2024-04-26T23:50:00.863549+0000","last_active":"2024-04-26T23:50:00.866958+0000","last_peered":"2024-04-26T23:50:00.866958+0000","last_clean":"2024-04-26T23:50:00.866958+0000","last_became_active":"2024-04-26T23:50:00.863278+0000","last_became_peered":"2024-04-26T23:50:00.863278+0000","last_unstale":"2024-04-26T23:50:00.866958+0000","last_undegraded":"2024-04-26T23:50:00.866958+0000","last_fullsized":"2024-04-26T23:50:00.866958+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:11.919852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873570+0000","last_change":"2024-04-26T23:49:59.847124+0000","last_active":"2024-04-26T23:50:00.873570+0000","last_peered":"2024-04-26T23:50:00.873570+0000","last_clean":"2024-04-26T23:50:00.873570+0000","last_became_active":"2024-04-26T23:49:59.846033+0000","last_became_peered":"2024-04-26T23:49:59.846033+0000","last_unstale":"2024-04-26T23:50:00.873570+0000","last_undegraded":"2024-04-26T23:50:00.873570+0000","last_fullsized":"2024-04-26T23:50:00.873570+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:23.407994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860298+0000","last_change":"2024-04-26T23:50:00.857951+0000","last_active":"2024-04-26T23:50:00.860298+0000","last_peered":"2024-04-26T23:50:00.860298+0000","last_clean":"2024-04-26T23:50:00.860298+0000","last_became_active":"2024-04-26T23:50:00.857655+0000","last_became_peered":"2024-04-26T23:50:00.857655+0000","last_unstale":"2024-04-26T23:50:00.860298+0000","last_undegraded":"2024-04-26T23:50:00.860298+0000","last_fullsized":"2024-04-26T23:50:00.860298+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:32.768819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871360+0000","last_change":"2024-04-26T23:49:59.842532+0000","last_active":"2024-04-26T23:50:00.871360+0000","last_peered":"2024-04-26T23:50:00.871360+0000","last_clean":"2024-04-26T23:50:00.871360+0000","last_became_active":"2024-04-26T23:49:59.841078+0000","last_became_peered":"2024-04-26T23:49:59.841078+0000","last_unstale":"2024-04-26T23:50:00.871360+0000","last_undegraded":"2024-04-26T23:50:00.871360+0000","last_fullsized":"2024-04-26T23:50:00.871360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:33.673898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846417+0000","last_change":"2024-04-26T23:50:00.841353+0000","last_active":"2024-04-26T23:50:00.846417+0000","last_peered":"2024-04-26T23:50:00.846417+0000","last_clean":"2024-04-26T23:50:00.846417+0000","last_became_active":"2024-04-26T23:50:00.840069+0000","last_became_peered":"2024-04-26T23:50:00.840069+0000","last_unstale":"2024-04-26T23:50:00.846417+0000","last_undegraded":"2024-04-26T23:50:00.846417+0000","last_fullsized":"2024-04-26T23:50:00.846417+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:59.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874693+0000","last_change":"2024-04-26T23:49:59.849948+0000","last_active":"2024-04-26T23:50:00.874693+0000","last_peered":"2024-04-26T23:50:00.874693+0000","last_clean":"2024-04-26T23:50:00.874693+0000","last_became_active":"2024-04-26T23:49:59.848958+0000","last_became_peered":"2024-04-26T23:49:59.848958+0000","last_unstale":"2024-04-26T23:50:00.874693+0000","last_undegraded":"2024-04-26T23:50:00.874693+0000","last_fullsized":"2024-04-26T23:50:00.874693+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:59.753006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862100+0000","last_change":"2024-04-26T23:50:00.853433+0000","last_active":"2024-04-26T23:50:00.862100+0000","last_peered":"2024-04-26T23:50:00.862100+0000","last_clean":"2024-04-26T23:50:00.862100+0000","last_became_active":"2024-04-26T23:50:00.853146+0000","last_became_peered":"2024-04-26T23:50:00.853146+0000","last_unstale":"2024-04-26T23:50:00.862100+0000","last_undegraded":"2024-04-26T23:50:00.862100+0000","last_fullsized":"2024-04-26T23:50:00.862100+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:44.892200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.842584+0000","last_change":"2024-04-26T23:49:59.854340+0000","last_active":"2024-04-26T23:50:00.842584+0000","last_peered":"2024-04-26T23:50:00.842584+0000","last_clean":"2024-04-26T23:50:00.842584+0000","last_became_active":"2024-04-26T23:49:59.854075+0000","last_became_peered":"2024-04-26T23:49:59.854075+0000","last_unstale":"2024-04-26T23:50:00.842584+0000","last_undegraded":"2024-04-26T23:50:00.842584+0000","last_fullsized":"2024-04-26T23:50:00.842584+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:58.878624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871143+0000","last_change":"2024-04-26T23:50:00.866146+0000","last_active":"2024-04-26T23:50:00.871143+0000","last_peered":"2024-04-26T23:50:00.871143+0000","last_clean":"2024-04-26T23:50:00.871143+0000","last_became_active":"2024-04-26T23:50:00.865411+0000","last_became_peered":"2024-04-26T23:50:00.865411+0000","last_unstale":"2024-04-26T23:50:00.871143+0000","last_undegraded":"2024-04-26T23:50:00.871143+0000","last_fullsized":"2024-04-26T23:50:00.871143+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:58.298236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841290+0000","last_change":"2024-04-26T23:49:59.846640+0000","last_active":"2024-04-26T23:50:00.841290+0000","last_peered":"2024-04-26T23:50:00.841290+0000","last_clean":"2024-04-26T23:50:00.841290+0000","last_became_active":"2024-04-26T23:49:59.846298+0000","last_became_peered":"2024-04-26T23:49:59.846298+0000","last_unstale":"2024-04-26T23:50:00.841290+0000","last_undegraded":"2024-04-26T23:50:00.841290+0000","last_fullsized":"2024-04-26T23:50:00.841290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:22.494162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863783+0000","last_change":"2024-04-26T23:50:00.852780+0000","last_active":"2024-04-26T23:50:00.863783+0000","last_peered":"2024-04-26T23:50:00.863783+0000","last_clean":"2024-04-26T23:50:00.863783+0000","last_became_active":"2024-04-26T23:50:00.852335+0000","last_became_peered":"2024-04-26T23:50:00.852335+0000","last_unstale":"2024-04-26T23:50:00.863783+0000","last_undegraded":"2024-04-26T23:50:00.863783+0000","last_fullsized":"2024-04-26T23:50:00.863783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:44.530709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844638+0000","last_change":"2024-04-26T23:49:59.850292+0000","last_active":"2024-04-26T23:50:00.844638+0000","last_peered":"2024-04-26T23:50:00.844638+0000","last_clean":"2024-04-26T23:50:00.844638+0000","last_became_active":"2024-04-26T23:49:59.849943+0000","last_became_peered":"2024-04-26T23:49:59.849943+0000","last_unstale":"2024-04-26T23:50:00.844638+0000","last_undegraded":"2024-04-26T23:50:00.844638+0000","last_fullsized":"2024-04-26T23:50:00.844638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:59.436581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860862+0000","last_change":"2024-04-26T23:50:00.856722+0000","last_active":"2024-04-26T23:50:00.860862+0000","last_peered":"2024-04-26T23:50:00.860862+0000","last_clean":"2024-04-26T23:50:00.860862+0000","last_became_active":"2024-04-26T23:50:00.856319+0000","last_became_peered":"2024-04-26T23:50:00.856319+0000","last_unstale":"2024-04-26T23:50:00.860862+0000","last_undegraded":"2024-04-26T23:50:00.860862+0000","last_fullsized":"2024-04-26T23:50:00.860862+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:12.107111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846198+0000","last_change":"2024-04-26T23:49:59.844480+0000","last_active":"2024-04-26T23:50:00.846198+0000","last_peered":"2024-04-26T23:50:00.846198+0000","last_clean":"2024-04-26T23:50:00.846198+0000","last_became_active":"2024-04-26T23:49:59.839930+0000","last_became_peered":"2024-04-26T23:49:59.839930+0000","last_unstale":"2024-04-26T23:50:00.846198+0000","last_undegraded":"2024-04-26T23:50:00.846198+0000","last_fullsized":"2024-04-26T23:50:00.846198+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.005609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874546+0000","last_change":"2024-04-26T23:50:00.871446+0000","last_active":"2024-04-26T23:50:00.874546+0000","last_peered":"2024-04-26T23:50:00.874546+0000","last_clean":"2024-04-26T23:50:00.874546+0000","last_became_active":"2024-04-26T23:50:00.870647+0000","last_became_peered":"2024-04-26T23:50:00.870647+0000","last_unstale":"2024-04-26T23:50:00.874546+0000","last_undegraded":"2024-04-26T23:50:00.874546+0000","last_fullsized":"2024-04-26T23:50:00.874546+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:38.189722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"136'3","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.875487+0000","last_change":"2024-04-26T23:49:59.845504+0000","last_active":"2024-04-26T23:50:00.875487+0000","last_peered":"2024-04-26T23:50:00.875487+0000","last_clean":"2024-04-26T23:50:00.875487+0000","last_became_active":"2024-04-26T23:49:59.845182+0000","last_became_peered":"2024-04-26T23:49:59.845182+0000","last_unstale":"2024-04-26T23:50:00.875487+0000","last_undegraded":"2024-04-26T23:50:00.875487+0000","last_fullsized":"2024-04-26T23:50:00.875487+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.054946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868120+0000","last_change":"2024-04-26T23:50:00.853238+0000","last_active":"2024-04-26T23:50:00.868120+0000","last_peered":"2024-04-26T23:50:00.868120+0000","last_clean":"2024-04-26T23:50:00.868120+0000","last_became_active":"2024-04-26T23:50:00.852940+0000","last_became_peered":"2024-04-26T23:50:00.852940+0000","last_unstale":"2024-04-26T23:50:00.868120+0000","last_undegraded":"2024-04-26T23:50:00.868120+0000","last_fullsized":"2024-04-26T23:50:00.868120+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:57.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"136'6","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.876365+0000","last_change":"2024-04-26T23:49:59.852856+0000","last_active":"2024-04-26T23:50:00.876365+0000","last_peered":"2024-04-26T23:50:00.876365+0000","last_clean":"2024-04-26T23:50:00.876365+0000","last_became_active":"2024-04-26T23:49:59.852511+0000","last_became_peered":"2024-04-26T23:49:59.852511+0000","last_unstale":"2024-04-26T23:50:00.876365+0000","last_undegraded":"2024-04-26T23:50:00.876365+0000","last_fullsized":"2024-04-26T23:50:00.876365+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:59.753006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.853822+0000","last_change":"2024-04-26T23:50:00.851468+0000","last_active":"2024-04-26T23:50:00.853822+0000","last_peered":"2024-04-26T23:50:00.853822+0000","last_clean":"2024-04-26T23:50:00.853822+0000","last_became_active":"2024-04-26T23:50:00.850963+0000","last_became_peered":"2024-04-26T23:50:00.850963+0000","last_unstale":"2024-04-26T23:50:00.853822+0000","last_undegraded":"2024-04-26T23:50:00.853822+0000","last_fullsized":"2024-04-26T23:50:00.853822+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:48.065440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873372+0000","last_change":"2024-04-26T23:49:59.844293+0000","last_active":"2024-04-26T23:50:00.873372+0000","last_peered":"2024-04-26T23:50:00.873372+0000","last_clean":"2024-04-26T23:50:00.873372+0000","last_became_active":"2024-04-26T23:49:59.843778+0000","last_became_peered":"2024-04-26T23:49:59.843778+0000","last_unstale":"2024-04-26T23:50:00.873372+0000","last_undegraded":"2024-04-26T23:50:00.873372+0000","last_fullsized":"2024-04-26T23:50:00.873372+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:57.819476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.867920+0000","last_change":"2024-04-26T23:50:00.861121+0000","last_active":"2024-04-26T23:50:00.867920+0000","last_peered":"2024-04-26T23:50:00.867920+0000","last_clean":"2024-04-26T23:50:00.867920+0000","last_became_active":"2024-04-26T23:50:00.860820+0000","last_became_peered":"2024-04-26T23:50:00.860820+0000","last_unstale":"2024-04-26T23:50:00.867920+0000","last_undegraded":"2024-04-26T23:50:00.867920+0000","last_fullsized":"2024-04-26T23:50:00.867920+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.837408+0000","last_change":"2024-04-26T23:49:59.840782+0000","last_active":"2024-04-26T23:50:00.837408+0000","last_peered":"2024-04-26T23:50:00.837408+0000","last_clean":"2024-04-26T23:50:00.837408+0000","last_became_active":"2024-04-26T23:49:59.838335+0000","last_became_peered":"2024-04-26T23:49:59.838335+0000","last_unstale":"2024-04-26T23:50:00.837408+0000","last_undegraded":"2024-04-26T23:50:00.837408+0000","last_fullsized":"2024-04-26T23:50:00.837408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:38:01.427776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863653+0000","last_change":"2024-04-26T23:50:00.858082+0000","last_active":"2024-04-26T23:50:00.863653+0000","last_peered":"2024-04-26T23:50:00.863653+0000","last_clean":"2024-04-26T23:50:00.863653+0000","last_became_active":"2024-04-26T23:50:00.857764+0000","last_became_peered":"2024-04-26T23:50:00.857764+0000","last_unstale":"2024-04-26T23:50:00.863653+0000","last_undegraded":"2024-04-26T23:50:00.863653+0000","last_fullsized":"2024-04-26T23:50:00.863653+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:35.487834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"136'8","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.878384+0000","last_change":"2024-04-26T23:49:59.844977+0000","last_active":"2024-04-26T23:50:00.878384+0000","last_peered":"2024-04-26T23:50:00.878384+0000","last_clean":"2024-04-26T23:50:00.878384+0000","last_became_active":"2024-04-26T23:49:59.844701+0000","last_became_peered":"2024-04-26T23:49:59.844701+0000","last_unstale":"2024-04-26T23:50:00.878384+0000","last_undegraded":"2024-04-26T23:50:00.878384+0000","last_fullsized":"2024-04-26T23:50:00.878384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:20.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866768+0000","last_change":"2024-04-26T23:50:00.859161+0000","last_active":"2024-04-26T23:50:00.866768+0000","last_peered":"2024-04-26T23:50:00.866768+0000","last_clean":"2024-04-26T23:50:00.866768+0000","last_became_active":"2024-04-26T23:50:00.855871+0000","last_became_peered":"2024-04-26T23:50:00.855871+0000","last_unstale":"2024-04-26T23:50:00.866768+0000","last_undegraded":"2024-04-26T23:50:00.866768+0000","last_fullsized":"2024-04-26T23:50:00.866768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.467001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868291+0000","last_change":"2024-04-26T23:49:59.834451+0000","last_active":"2024-04-26T23:50:00.868291+0000","last_peered":"2024-04-26T23:50:00.868291+0000","last_clean":"2024-04-26T23:50:00.868291+0000","last_became_active":"2024-04-26T23:49:59.833528+0000","last_became_peered":"2024-04-26T23:49:59.833528+0000","last_unstale":"2024-04-26T23:50:00.868291+0000","last_undegraded":"2024-04-26T23:50:00.868291+0000","last_fullsized":"2024-04-26T23:50:00.868291+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:15.088216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872050+0000","last_change":"2024-04-26T23:50:00.869645+0000","last_active":"2024-04-26T23:50:00.872050+0000","last_peered":"2024-04-26T23:50:00.872050+0000","last_clean":"2024-04-26T23:50:00.872050+0000","last_became_active":"2024-04-26T23:50:00.867846+0000","last_became_peered":"2024-04-26T23:50:00.867846+0000","last_unstale":"2024-04-26T23:50:00.872050+0000","last_undegraded":"2024-04-26T23:50:00.872050+0000","last_fullsized":"2024-04-26T23:50:00.872050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:55.599385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.877082+0000","last_change":"2024-04-26T23:49:59.851000+0000","last_active":"2024-04-26T23:50:00.877082+0000","last_peered":"2024-04-26T23:50:00.877082+0000","last_clean":"2024-04-26T23:50:00.877082+0000","last_became_active":"2024-04-26T23:49:59.850498+0000","last_became_peered":"2024-04-26T23:49:59.850498+0000","last_unstale":"2024-04-26T23:50:00.877082+0000","last_undegraded":"2024-04-26T23:50:00.877082+0000","last_fullsized":"2024-04-26T23:50:00.877082+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:08.787493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871096+0000","last_change":"2024-04-26T23:50:00.864432+0000","last_active":"2024-04-26T23:50:00.871096+0000","last_peered":"2024-04-26T23:50:00.871096+0000","last_clean":"2024-04-26T23:50:00.871096+0000","last_became_active":"2024-04-26T23:50:00.864078+0000","last_became_peered":"2024-04-26T23:50:00.864078+0000","last_unstale":"2024-04-26T23:50:00.871096+0000","last_undegraded":"2024-04-26T23:50:00.871096+0000","last_fullsized":"2024-04-26T23:50:00.871096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:44.892200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874256+0000","last_change":"2024-04-26T23:49:59.852297+0000","last_active":"2024-04-26T23:50:00.874256+0000","last_peered":"2024-04-26T23:50:00.874256+0000","last_clean":"2024-04-26T23:50:00.874256+0000","last_became_active":"2024-04-26T23:49:59.852001+0000","last_became_peered":"2024-04-26T23:49:59.852001+0000","last_unstale":"2024-04-26T23:50:00.874256+0000","last_undegraded":"2024-04-26T23:50:00.874256+0000","last_fullsized":"2024-04-26T23:50:00.874256+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:37.174739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.855769+0000","last_change":"2024-04-26T23:50:00.855019+0000","last_active":"2024-04-26T23:50:00.855769+0000","last_peered":"2024-04-26T23:50:00.855769+0000","last_clean":"2024-04-26T23:50:00.855769+0000","last_became_active":"2024-04-26T23:50:00.854612+0000","last_became_peered":"2024-04-26T23:50:00.854612+0000","last_unstale":"2024-04-26T23:50:00.855769+0000","last_undegraded":"2024-04-26T23:50:00.855769+0000","last_fullsized":"2024-04-26T23:50:00.855769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:53.855781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.832693+0000","last_change":"2024-04-26T23:49:59.854569+0000","last_active":"2024-04-26T23:50:00.832693+0000","last_peered":"2024-04-26T23:50:00.832693+0000","last_clean":"2024-04-26T23:50:00.832693+0000","last_became_active":"2024-04-26T23:49:59.854261+0000","last_became_peered":"2024-04-26T23:49:59.854261+0000","last_unstale":"2024-04-26T23:50:00.832693+0000","last_undegraded":"2024-04-26T23:50:00.832693+0000","last_fullsized":"2024-04-26T23:50:00.832693+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:45.577431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861256+0000","last_change":"2024-04-26T23:50:00.852121+0000","last_active":"2024-04-26T23:50:00.861256+0000","last_peered":"2024-04-26T23:50:00.861256+0000","last_clean":"2024-04-26T23:50:00.861256+0000","last_became_active":"2024-04-26T23:50:00.851726+0000","last_became_peered":"2024-04-26T23:50:00.851726+0000","last_unstale":"2024-04-26T23:50:00.861256+0000","last_undegraded":"2024-04-26T23:50:00.861256+0000","last_fullsized":"2024-04-26T23:50:00.861256+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:20.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873384+0000","last_change":"2024-04-26T23:49:59.842450+0000","last_active":"2024-04-26T23:50:00.873384+0000","last_peered":"2024-04-26T23:50:00.873384+0000","last_clean":"2024-04-26T23:50:00.873384+0000","last_became_active":"2024-04-26T23:49:59.840033+0000","last_became_peered":"2024-04-26T23:49:59.840033+0000","last_unstale":"2024-04-26T23:50:00.873384+0000","last_undegraded":"2024-04-26T23:50:00.873384+0000","last_fullsized":"2024-04-26T23:50:00.873384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:27.452018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866782+0000","last_change":"2024-04-26T23:50:00.863742+0000","last_active":"2024-04-26T23:50:00.866782+0000","last_peered":"2024-04-26T23:50:00.866782+0000","last_clean":"2024-04-26T23:50:00.866782+0000","last_became_active":"2024-04-26T23:50:00.863450+0000","last_became_peered":"2024-04-26T23:50:00.863450+0000","last_unstale":"2024-04-26T23:50:00.866782+0000","last_undegraded":"2024-04-26T23:50:00.866782+0000","last_fullsized":"2024-04-26T23:50:00.866782+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:18.945144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841765+0000","last_change":"2024-04-26T23:49:59.845287+0000","last_active":"2024-04-26T23:50:00.841765+0000","last_peered":"2024-04-26T23:50:00.841765+0000","last_clean":"2024-04-26T23:50:00.841765+0000","last_became_active":"2024-04-26T23:49:59.844977+0000","last_became_peered":"2024-04-26T23:49:59.844977+0000","last_unstale":"2024-04-26T23:50:00.841765+0000","last_undegraded":"2024-04-26T23:50:00.841765+0000","last_fullsized":"2024-04-26T23:50:00.841765+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:19.733155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860918+0000","last_change":"2024-04-26T23:50:00.849374+0000","last_active":"2024-04-26T23:50:00.860918+0000","last_peered":"2024-04-26T23:50:00.860918+0000","last_clean":"2024-04-26T23:50:00.860918+0000","last_became_active":"2024-04-26T23:50:00.848765+0000","last_became_peered":"2024-04-26T23:50:00.848765+0000","last_unstale":"2024-04-26T23:50:00.860918+0000","last_undegraded":"2024-04-26T23:50:00.860918+0000","last_fullsized":"2024-04-26T23:50:00.860918+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.850576+0000","last_change":"2024-04-26T23:49:59.845887+0000","last_active":"2024-04-26T23:50:00.850576+0000","last_peered":"2024-04-26T23:50:00.850576+0000","last_clean":"2024-04-26T23:50:00.850576+0000","last_became_active":"2024-04-26T23:49:59.845575+0000","last_became_peered":"2024-04-26T23:49:59.845575+0000","last_unstale":"2024-04-26T23:50:00.850576+0000","last_undegraded":"2024-04-26T23:50:00.850576+0000","last_fullsized":"2024-04-26T23:50:00.850576+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:49.768244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.859893+0000","last_change":"2024-04-26T23:50:00.856126+0000","last_active":"2024-04-26T23:50:00.859893+0000","last_peered":"2024-04-26T23:50:00.859893+0000","last_clean":"2024-04-26T23:50:00.859893+0000","last_became_active":"2024-04-26T23:50:00.855820+0000","last_became_peered":"2024-04-26T23:50:00.855820+0000","last_unstale":"2024-04-26T23:50:00.859893+0000","last_undegraded":"2024-04-26T23:50:00.859893+0000","last_fullsized":"2024-04-26T23:50:00.859893+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:08.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841941+0000","last_change":"2024-04-26T23:49:59.848225+0000","last_active":"2024-04-26T23:50:00.841941+0000","last_peered":"2024-04-26T23:50:00.841941+0000","last_clean":"2024-04-26T23:50:00.841941+0000","last_became_active":"2024-04-26T23:49:59.847907+0000","last_became_peered":"2024-04-26T23:49:59.847907+0000","last_unstale":"2024-04-26T23:50:00.841941+0000","last_undegraded":"2024-04-26T23:50:00.841941+0000","last_fullsized":"2024-04-26T23:50:00.841941+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:12.186900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861311+0000","last_change":"2024-04-26T23:50:00.852154+0000","last_active":"2024-04-26T23:50:00.861311+0000","last_peered":"2024-04-26T23:50:00.861311+0000","last_clean":"2024-04-26T23:50:00.861311+0000","last_became_active":"2024-04-26T23:50:00.851872+0000","last_became_peered":"2024-04-26T23:50:00.851872+0000","last_unstale":"2024-04-26T23:50:00.861311+0000","last_undegraded":"2024-04-26T23:50:00.861311+0000","last_fullsized":"2024-04-26T23:50:00.861311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:13.535156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.875564+0000","last_change":"2024-04-26T23:49:59.847737+0000","last_active":"2024-04-26T23:50:00.875564+0000","last_peered":"2024-04-26T23:50:00.875564+0000","last_clean":"2024-04-26T23:50:00.875564+0000","last_became_active":"2024-04-26T23:49:59.847312+0000","last_became_peered":"2024-04-26T23:49:59.847312+0000","last_unstale":"2024-04-26T23:50:00.875564+0000","last_undegraded":"2024-04-26T23:50:00.875564+0000","last_fullsized":"2024-04-26T23:50:00.875564+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.189674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866816+0000","last_change":"2024-04-26T23:50:00.859222+0000","last_active":"2024-04-26T23:50:00.866816+0000","last_peered":"2024-04-26T23:50:00.866816+0000","last_clean":"2024-04-26T23:50:00.866816+0000","last_became_active":"2024-04-26T23:50:00.854804+0000","last_became_peered":"2024-04-26T23:50:00.854804+0000","last_unstale":"2024-04-26T23:50:00.866816+0000","last_undegraded":"2024-04-26T23:50:00.866816+0000","last_fullsized":"2024-04-26T23:50:00.866816+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:34.688881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.831270+0000","last_change":"2024-04-26T23:49:59.839499+0000","last_active":"2024-04-26T23:50:00.831270+0000","last_peered":"2024-04-26T23:50:00.831270+0000","last_clean":"2024-04-26T23:50:00.831270+0000","last_became_active":"2024-04-26T23:49:59.839173+0000","last_became_peered":"2024-04-26T23:49:59.839173+0000","last_unstale":"2024-04-26T23:50:00.831270+0000","last_undegraded":"2024-04-26T23:50:00.831270+0000","last_fullsized":"2024-04-26T23:50:00.831270+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:12.186900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873614+0000","last_change":"2024-04-26T23:50:00.870948+0000","last_active":"2024-04-26T23:50:00.873614+0000","last_peered":"2024-04-26T23:50:00.873614+0000","last_clean":"2024-04-26T23:50:00.873614+0000","last_became_active":"2024-04-26T23:50:00.870637+0000","last_became_peered":"2024-04-26T23:50:00.870637+0000","last_unstale":"2024-04-26T23:50:00.873614+0000","last_undegraded":"2024-04-26T23:50:00.873614+0000","last_fullsized":"2024-04-26T23:50:00.873614+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:30.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.830808+0000","last_change":"2024-04-26T23:49:59.848118+0000","last_active":"2024-04-26T23:50:00.830808+0000","last_peered":"2024-04-26T23:50:00.830808+0000","last_clean":"2024-04-26T23:50:00.830808+0000","last_became_active":"2024-04-26T23:49:59.847612+0000","last_became_peered":"2024-04-26T23:49:59.847612+0000","last_unstale":"2024-04-26T23:50:00.830808+0000","last_undegraded":"2024-04-26T23:50:00.830808+0000","last_fullsized":"2024-04-26T23:50:00.830808+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:20.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844661+0000","last_change":"2024-04-26T23:50:00.838771+0000","last_active":"2024-04-26T23:50:00.844661+0000","last_peered":"2024-04-26T23:50:00.844661+0000","last_clean":"2024-04-26T23:50:00.844661+0000","last_became_active":"2024-04-26T23:50:00.837807+0000","last_became_peered":"2024-04-26T23:50:00.837807+0000","last_unstale":"2024-04-26T23:50:00.844661+0000","last_undegraded":"2024-04-26T23:50:00.844661+0000","last_fullsized":"2024-04-26T23:50:00.844661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.209908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846072+0000","last_change":"2024-04-26T23:49:59.853685+0000","last_active":"2024-04-26T23:50:00.846072+0000","last_peered":"2024-04-26T23:50:00.846072+0000","last_clean":"2024-04-26T23:50:00.846072+0000","last_became_active":"2024-04-26T23:49:59.853326+0000","last_became_peered":"2024-04-26T23:49:59.853326+0000","last_unstale":"2024-04-26T23:50:00.846072+0000","last_undegraded":"2024-04-26T23:50:00.846072+0000","last_fullsized":"2024-04-26T23:50:00.846072+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:31.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862395+0000","last_change":"2024-04-26T23:50:00.856294+0000","last_active":"2024-04-26T23:50:00.862395+0000","last_peered":"2024-04-26T23:50:00.862395+0000","last_clean":"2024-04-26T23:50:00.862395+0000","last_became_active":"2024-04-26T23:50:00.853114+0000","last_became_peered":"2024-04-26T23:50:00.853114+0000","last_unstale":"2024-04-26T23:50:00.862395+0000","last_undegraded":"2024-04-26T23:50:00.862395+0000","last_fullsized":"2024-04-26T23:50:00.862395+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:45.360258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844494+0000","last_change":"2024-04-26T23:49:59.840842+0000","last_active":"2024-04-26T23:50:00.844494+0000","last_peered":"2024-04-26T23:50:00.844494+0000","last_clean":"2024-04-26T23:50:00.844494+0000","last_became_active":"2024-04-26T23:49:59.840540+0000","last_became_peered":"2024-04-26T23:49:59.840540+0000","last_unstale":"2024-04-26T23:50:00.844494+0000","last_undegraded":"2024-04-26T23:50:00.844494+0000","last_fullsized":"2024-04-26T23:50:00.844494+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:11.092128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863986+0000","last_change":"2024-04-26T23:50:00.861791+0000","last_active":"2024-04-26T23:50:00.863986+0000","last_peered":"2024-04-26T23:50:00.863986+0000","last_clean":"2024-04-26T23:50:00.863986+0000","last_became_active":"2024-04-26T23:50:00.861148+0000","last_became_peered":"2024-04-26T23:50:00.861148+0000","last_unstale":"2024-04-26T23:50:00.863986+0000","last_undegraded":"2024-04-26T23:50:00.863986+0000","last_fullsized":"2024-04-26T23:50:00.863986+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:00.451564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.849053+0000","last_change":"2024-04-26T23:49:59.846801+0000","last_active":"2024-04-26T23:50:00.849053+0000","last_peered":"2024-04-26T23:50:00.849053+0000","last_clean":"2024-04-26T23:50:00.849053+0000","last_became_active":"2024-04-26T23:49:59.846004+0000","last_became_peered":"2024-04-26T23:49:59.846004+0000","last_unstale":"2024-04-26T23:50:00.849053+0000","last_undegraded":"2024-04-26T23:50:00.849053+0000","last_fullsized":"2024-04-26T23:50:00.849053+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:09.759634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869812+0000","last_change":"2024-04-26T23:50:00.867775+0000","last_active":"2024-04-26T23:50:00.869812+0000","last_peered":"2024-04-26T23:50:00.869812+0000","last_clean":"2024-04-26T23:50:00.869812+0000","last_became_active":"2024-04-26T23:50:00.867469+0000","last_became_peered":"2024-04-26T23:50:00.867469+0000","last_unstale":"2024-04-26T23:50:00.869812+0000","last_undegraded":"2024-04-26T23:50:00.869812+0000","last_fullsized":"2024-04-26T23:50:00.869812+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:11.058674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.842757+0000","last_change":"2024-04-26T23:49:59.847495+0000","last_active":"2024-04-26T23:50:00.842757+0000","last_peered":"2024-04-26T23:50:00.842757+0000","last_clean":"2024-04-26T23:50:00.842757+0000","last_became_active":"2024-04-26T23:49:59.847203+0000","last_became_peered":"2024-04-26T23:49:59.847203+0000","last_unstale":"2024-04-26T23:50:00.842757+0000","last_undegraded":"2024-04-26T23:50:00.842757+0000","last_fullsized":"2024-04-26T23:50:00.842757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:53.982063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871120+0000","last_change":"2024-04-26T23:50:00.864807+0000","last_active":"2024-04-26T23:50:00.871120+0000","last_peered":"2024-04-26T23:50:00.871120+0000","last_clean":"2024-04-26T23:50:00.871120+0000","last_became_active":"2024-04-26T23:50:00.863417+0000","last_became_peered":"2024-04-26T23:50:00.863417+0000","last_unstale":"2024-04-26T23:50:00.871120+0000","last_undegraded":"2024-04-26T23:50:00.871120+0000","last_fullsized":"2024-04-26T23:50:00.871120+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:55.501626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.834191+0000","last_change":"2024-04-26T23:49:59.846032+0000","last_active":"2024-04-26T23:50:00.834191+0000","last_peered":"2024-04-26T23:50:00.834191+0000","last_clean":"2024-04-26T23:50:00.834191+0000","last_became_active":"2024-04-26T23:49:59.845473+0000","last_became_peered":"2024-04-26T23:49:59.845473+0000","last_unstale":"2024-04-26T23:50:00.834191+0000","last_undegraded":"2024-04-26T23:50:00.834191+0000","last_fullsized":"2024-04-26T23:50:00.834191+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:09.759634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862426+0000","last_change":"2024-04-26T23:50:00.854897+0000","last_active":"2024-04-26T23:50:00.862426+0000","last_peered":"2024-04-26T23:50:00.862426+0000","last_clean":"2024-04-26T23:50:00.862426+0000","last_became_active":"2024-04-26T23:50:00.854473+0000","last_became_peered":"2024-04-26T23:50:00.854473+0000","last_unstale":"2024-04-26T23:50:00.862426+0000","last_undegraded":"2024-04-26T23:50:00.862426+0000","last_fullsized":"2024-04-26T23:50:00.862426+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:22.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.849236+0000","last_change":"2024-04-26T23:49:59.850420+0000","last_active":"2024-04-26T23:50:00.849236+0000","last_peered":"2024-04-26T23:50:00.849236+0000","last_clean":"2024-04-26T23:50:00.849236+0000","last_became_active":"2024-04-26T23:49:59.850129+0000","last_became_peered":"2024-04-26T23:49:59.850129+0000","last_unstale":"2024-04-26T23:50:00.849236+0000","last_undegraded":"2024-04-26T23:50:00.849236+0000","last_fullsized":"2024-04-26T23:50:00.849236+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:22.109365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":381,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.845036+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:50:00.845036+0000","last_peered":"2024-04-26T23:50:00.845036+0000","last_clean":"2024-04-26T23:50:00.845036+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:50:00.845036+0000","last_undegraded":"2024-04-26T23:50:00.845036+0000","last_fullsized":"2024-04-26T23:50:00.845036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29248,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342592,"statfs":{"total":96636764160,"available":96606814208,"internally_reserved":0,"allocated":2281472,"data_stored":1382215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50876,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320964,"statfs":{"total":96636764160,"available":96584667136,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46716,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325124,"statfs":{"total":96636764160,"available":96588926976,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2301952,"data_stored":1389438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:50:05.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:50:05.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.577+0000 7f02b6e00640 1 Processor -- start 2024-04-26T23:50:05.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.577+0000 7f02b6e00640 1 -- start start 2024-04-26T23:50:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f02b00cc690 0x7f02b00cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02b00c6a50 0x7f02b00c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b0072bb0 con 0x7f02b01454a0 2024-04-26T23:50:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f02b00c73e0 con 0x7f02b00c6a50 2024-04-26T23:50:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f02b0072e90 con 0x7f02b00cc690 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02b00c6a50 0x7f02b00c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02b00c6a50 0x7f02b00c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54078/0 (socket says 172.21.15.46:54078) 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b6400640 1 -- 172.21.15.46:0/1175219518 learned_addr learned my addr 172.21.15.46:0/1175219518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.579+0000 7f02b5000640 1 --2- 172.21.15.46:0/1175219518 >> v2:172.21.15.142:3300/0 conn(0x7f02b00cc690 0x7f02b00cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.580+0000 7f02b6400640 1 -- 172.21.15.46:0/1175219518 >> v2:172.21.15.142:3300/0 conn(0x7f02b00cc690 msgr2=0x7f02b00cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.580+0000 7f02b6400640 1 --2- 172.21.15.46:0/1175219518 >> v2:172.21.15.142:3300/0 conn(0x7f02b00cc690 0x7f02b00cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.580+0000 7f02b6400640 1 -- 172.21.15.46:0/1175219518 >> v1:172.21.15.142:6789/0 conn(0x7f02b01454a0 legacy=0x7f02b0073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.580+0000 7f02b6400640 1 -- 172.21.15.46:0/1175219518 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b0073170 con 0x7f02b00c6a50 2024-04-26T23:50:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.581+0000 7f02b6400640 1 --2- 172.21.15.46:0/1175219518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00c6a50 0x7f02b00c6ea0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f02b00cb0a0 tx=0x7f02a802c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311b48e1a82f2b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.581+0000 7f02afe00640 1 -- 172.21.15.46:0/1175219518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8033610 con 0x7f02b00c6a50 2024-04-26T23:50:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.581+0000 7f02afe00640 1 -- 172.21.15.46:0/1175219518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a80337b0 con 0x7f02b00c6a50 2024-04-26T23:50:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.581+0000 7f02afe00640 1 -- 172.21.15.46:0/1175219518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8033b80 con 0x7f02b00c6a50 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 -- 172.21.15.46:0/1175219518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00c6a50 msgr2=0x7f02b00c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 --2- 172.21.15.46:0/1175219518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00c6a50 0x7f02b00c6ea0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f02b00cb0a0 tx=0x7f02a802c900 comp rx=0 tx=0).stop 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 -- 172.21.15.46:0/1175219518 shutdown_connections 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 --2- 172.21.15.46:0/1175219518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00c6a50 0x7f02b00c6ea0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 --2- 172.21.15.46:0/1175219518 >> v2:172.21.15.142:3300/0 conn(0x7f02b00cc690 0x7f02b00cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 -- 172.21.15.46:0/1175219518 >> 172.21.15.46:0/1175219518 conn(0x7f02b00b26d0 msgr2=0x7f02b00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 -- 172.21.15.46:0/1175219518 shutdown_connections 2024-04-26T23:50:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.582+0000 7f02b6e00640 1 -- 172.21.15.46:0/1175219518 wait complete. 2024-04-26T23:50:05.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.584+0000 7f02b6e00640 1 Processor -- start 2024-04-26T23:50:05.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.584+0000 7f02b6e00640 1 -- start start 2024-04-26T23:50:05.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b00c6a50 0x7f02b01f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b01454a0 0x7f02b00ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0072880 con 0x7f02b00cc690 2024-04-26T23:50:05.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02b00ce170 con 0x7f02b00c6a50 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b6e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02b0145880 con 0x7f02b01454a0 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54084/0 (socket says 172.21.15.46:54084) 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.585+0000 7f02b5000640 1 -- 172.21.15.46:0/4230128633 learned_addr learned my addr 172.21.15.46:0/4230128633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b6400640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b01454a0 0x7f02b00ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5a00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b00c6a50 0x7f02b01f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 -- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b01454a0 msgr2=0x7f02b00ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b01454a0 0x7f02b00ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 -- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b00c6a50 msgr2=0x7f02b01f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b00c6a50 0x7f02b01f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 -- 172.21.15.46:0/4230128633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b00aaa10 con 0x7f02b00cc690 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b5000640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f029c00d830 tx=0x7f029c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c00b6c0 con 0x7f02b00cc690 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c019070 con 0x7f02b00cc690 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c00ba80 con 0x7f02b00cc690 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.586+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b00aacb0 con 0x7f02b00cc690 2024-04-26T23:50:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.587+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b00ab270 con 0x7f02b00cc690 2024-04-26T23:50:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.588+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b00693b0 con 0x7f02b00cc690 2024-04-26T23:50:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.589+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f029c01e460 con 0x7f02b00cc690 2024-04-26T23:50:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.590+0000 7f02ad600640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.590+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f029c08c3d0 con 0x7f02b00cc690 2024-04-26T23:50:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.590+0000 7f02b5a00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.591+0000 7f02b5a00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 0x7f029006edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f02a4006fd0 tx=0x7f02a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.594+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f029c05a660 con 0x7f02b00cc690 2024-04-26T23:50:05.774 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:50:05.769+0000 7fc15c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:50:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.798+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02b00aea30 con 0x7f029006c8f0 2024-04-26T23:50:05.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.817+0000 7f02ad600640 1 -- 172.21.15.46:0/4230128633 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252412 (secure 0 0 0) 0x7f02b00aea30 con 0x7f029006c8f0 2024-04-26T23:50:05.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:05.826 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 msgr2=0x7f029006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 0x7f029006edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f02a4006fd0 tx=0x7f02a4008040 comp rx=0 tx=0).stop 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 msgr2=0x7f02b00aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f029c00d830 tx=0x7f029c00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 shutdown_connections 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b01454a0 0x7f02b00ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b00cc690 0x7f02b00aa4d0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 --2- 172.21.15.46:0/4230128633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b00c6a50 0x7f02b01f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 >> 172.21.15.46:0/4230128633 conn(0x7f02b00b26d0 msgr2=0x7f02b00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 shutdown_connections 2024-04-26T23:50:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:05.822+0000 7f02b6e00640 1 -- 172.21.15.46:0/4230128633 wait complete. 2024-04-26T23:50:05.845 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-04-26T23:50:04.178283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402624,"kb_used_data":14364,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754572096,"statfs":{"total":773094113280,"available":772681826304,"internally_reserved":0,"allocated":14708736,"data_stored":7522735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":397463089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.380394"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856038+0000","last_change":"2024-04-26T23:50:00.850781+0000","last_active":"2024-04-26T23:50:00.856038+0000","last_peered":"2024-04-26T23:50:00.856038+0000","last_clean":"2024-04-26T23:50:00.856038+0000","last_became_active":"2024-04-26T23:50:00.850498+0000","last_became_peered":"2024-04-26T23:50:00.850498+0000","last_unstale":"2024-04-26T23:50:00.856038+0000","last_undegraded":"2024-04-26T23:50:00.856038+0000","last_fullsized":"2024-04-26T23:50:00.856038+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:58.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856011+0000","last_change":"2024-04-26T23:50:00.852406+0000","last_active":"2024-04-26T23:50:00.856011+0000","last_peered":"2024-04-26T23:50:00.856011+0000","last_clean":"2024-04-26T23:50:00.856011+0000","last_became_active":"2024-04-26T23:50:00.851793+0000","last_became_peered":"2024-04-26T23:50:00.851793+0000","last_unstale":"2024-04-26T23:50:00.856011+0000","last_undegraded":"2024-04-26T23:50:00.856011+0000","last_fullsized":"2024-04-26T23:50:00.856011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:54.152303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.870914+0000","last_change":"2024-04-26T23:50:00.862668+0000","last_active":"2024-04-26T23:50:00.870914+0000","last_peered":"2024-04-26T23:50:00.870914+0000","last_clean":"2024-04-26T23:50:00.870914+0000","last_became_active":"2024-04-26T23:50:00.862345+0000","last_became_peered":"2024-04-26T23:50:00.862345+0000","last_unstale":"2024-04-26T23:50:00.870914+0000","last_undegraded":"2024-04-26T23:50:00.870914+0000","last_fullsized":"2024-04-26T23:50:00.870914+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:59.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856359+0000","last_change":"2024-04-26T23:50:00.850445+0000","last_active":"2024-04-26T23:50:00.856359+0000","last_peered":"2024-04-26T23:50:00.856359+0000","last_clean":"2024-04-26T23:50:00.856359+0000","last_became_active":"2024-04-26T23:50:00.850104+0000","last_became_peered":"2024-04-26T23:50:00.850104+0000","last_unstale":"2024-04-26T23:50:00.856359+0000","last_undegraded":"2024-04-26T23:50:00.856359+0000","last_fullsized":"2024-04-26T23:50:00.856359+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:11.058674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.845460+0000","last_change":"2024-04-26T23:50:00.840694+0000","last_active":"2024-04-26T23:50:00.845460+0000","last_peered":"2024-04-26T23:50:00.845460+0000","last_clean":"2024-04-26T23:50:00.845460+0000","last_became_active":"2024-04-26T23:50:00.839932+0000","last_became_peered":"2024-04-26T23:50:00.839932+0000","last_unstale":"2024-04-26T23:50:00.845460+0000","last_undegraded":"2024-04-26T23:50:00.845460+0000","last_fullsized":"2024-04-26T23:50:00.845460+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:23.124347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862067+0000","last_change":"2024-04-26T23:50:00.855874+0000","last_active":"2024-04-26T23:50:00.862067+0000","last_peered":"2024-04-26T23:50:00.862067+0000","last_clean":"2024-04-26T23:50:00.862067+0000","last_became_active":"2024-04-26T23:50:00.855506+0000","last_became_peered":"2024-04-26T23:50:00.855506+0000","last_unstale":"2024-04-26T23:50:00.862067+0000","last_undegraded":"2024-04-26T23:50:00.862067+0000","last_fullsized":"2024-04-26T23:50:00.862067+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:21.521273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869357+0000","last_change":"2024-04-26T23:50:00.862259+0000","last_active":"2024-04-26T23:50:00.869357+0000","last_peered":"2024-04-26T23:50:00.869357+0000","last_clean":"2024-04-26T23:50:00.869357+0000","last_became_active":"2024-04-26T23:50:00.861949+0000","last_became_peered":"2024-04-26T23:50:00.861949+0000","last_unstale":"2024-04-26T23:50:00.869357+0000","last_undegraded":"2024-04-26T23:50:00.869357+0000","last_fullsized":"2024-04-26T23:50:00.869357+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:20.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861079+0000","last_change":"2024-04-26T23:50:00.850822+0000","last_active":"2024-04-26T23:50:00.861079+0000","last_peered":"2024-04-26T23:50:00.861079+0000","last_clean":"2024-04-26T23:50:00.861079+0000","last_became_active":"2024-04-26T23:50:00.850496+0000","last_became_peered":"2024-04-26T23:50:00.850496+0000","last_unstale":"2024-04-26T23:50:00.861079+0000","last_undegraded":"2024-04-26T23:50:00.861079+0000","last_fullsized":"2024-04-26T23:50:00.861079+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.997046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.856892+0000","last_change":"2024-04-26T23:50:00.854760+0000","last_active":"2024-04-26T23:50:00.856892+0000","last_peered":"2024-04-26T23:50:00.856892+0000","last_clean":"2024-04-26T23:50:00.856892+0000","last_became_active":"2024-04-26T23:50:00.854402+0000","last_became_peered":"2024-04-26T23:50:00.854402+0000","last_unstale":"2024-04-26T23:50:00.856892+0000","last_undegraded":"2024-04-26T23:50:00.856892+0000","last_fullsized":"2024-04-26T23:50:00.856892+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:50.313727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.837663+0000","last_change":"2024-04-26T23:50:00.834728+0000","last_active":"2024-04-26T23:50:00.837663+0000","last_peered":"2024-04-26T23:50:00.837663+0000","last_clean":"2024-04-26T23:50:00.837663+0000","last_became_active":"2024-04-26T23:50:00.833909+0000","last_became_peered":"2024-04-26T23:50:00.833909+0000","last_unstale":"2024-04-26T23:50:00.837663+0000","last_undegraded":"2024-04-26T23:50:00.837663+0000","last_fullsized":"2024-04-26T23:50:00.837663+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:23.438270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.859711+0000","last_change":"2024-04-26T23:50:00.852245+0000","last_active":"2024-04-26T23:50:00.859711+0000","last_peered":"2024-04-26T23:50:00.859711+0000","last_clean":"2024-04-26T23:50:00.859711+0000","last_became_active":"2024-04-26T23:50:00.850761+0000","last_became_peered":"2024-04-26T23:50:00.850761+0000","last_unstale":"2024-04-26T23:50:00.859711+0000","last_undegraded":"2024-04-26T23:50:00.859711+0000","last_fullsized":"2024-04-26T23:50:00.859711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:38.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860334+0000","last_change":"2024-04-26T23:50:00.857097+0000","last_active":"2024-04-26T23:50:00.860334+0000","last_peered":"2024-04-26T23:50:00.860334+0000","last_clean":"2024-04-26T23:50:00.860334+0000","last_became_active":"2024-04-26T23:50:00.853749+0000","last_became_peered":"2024-04-26T23:50:00.853749+0000","last_unstale":"2024-04-26T23:50:00.860334+0000","last_undegraded":"2024-04-26T23:50:00.860334+0000","last_fullsized":"2024-04-26T23:50:00.860334+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:33.044388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861309+0000","last_change":"2024-04-26T23:50:00.855842+0000","last_active":"2024-04-26T23:50:00.861309+0000","last_peered":"2024-04-26T23:50:00.861309+0000","last_clean":"2024-04-26T23:50:00.861309+0000","last_became_active":"2024-04-26T23:50:00.855498+0000","last_became_peered":"2024-04-26T23:50:00.855498+0000","last_unstale":"2024-04-26T23:50:00.861309+0000","last_undegraded":"2024-04-26T23:50:00.861309+0000","last_fullsized":"2024-04-26T23:50:00.861309+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:51.918115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862100+0000","last_change":"2024-04-26T23:50:00.858206+0000","last_active":"2024-04-26T23:50:00.862100+0000","last_peered":"2024-04-26T23:50:00.862100+0000","last_clean":"2024-04-26T23:50:00.862100+0000","last_became_active":"2024-04-26T23:50:00.857798+0000","last_became_peered":"2024-04-26T23:50:00.857798+0000","last_unstale":"2024-04-26T23:50:00.862100+0000","last_undegraded":"2024-04-26T23:50:00.862100+0000","last_fullsized":"2024-04-26T23:50:00.862100+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:32.671059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866567+0000","last_change":"2024-04-26T23:50:00.862637+0000","last_active":"2024-04-26T23:50:00.866567+0000","last_peered":"2024-04-26T23:50:00.866567+0000","last_clean":"2024-04-26T23:50:00.866567+0000","last_became_active":"2024-04-26T23:50:00.862333+0000","last_became_peered":"2024-04-26T23:50:00.862333+0000","last_unstale":"2024-04-26T23:50:00.866567+0000","last_undegraded":"2024-04-26T23:50:00.866567+0000","last_fullsized":"2024-04-26T23:50:00.866567+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:47.176999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.864545+0000","last_change":"2024-04-26T23:50:00.863203+0000","last_active":"2024-04-26T23:50:00.864545+0000","last_peered":"2024-04-26T23:50:00.864545+0000","last_clean":"2024-04-26T23:50:00.864545+0000","last_became_active":"2024-04-26T23:50:00.862922+0000","last_became_peered":"2024-04-26T23:50:00.862922+0000","last_unstale":"2024-04-26T23:50:00.864545+0000","last_undegraded":"2024-04-26T23:50:00.864545+0000","last_fullsized":"2024-04-26T23:50:00.864545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:16.103199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869840+0000","last_change":"2024-04-26T23:50:00.866483+0000","last_active":"2024-04-26T23:50:00.869840+0000","last_peered":"2024-04-26T23:50:00.869840+0000","last_clean":"2024-04-26T23:50:00.869840+0000","last_became_active":"2024-04-26T23:50:00.866173+0000","last_became_peered":"2024-04-26T23:50:00.866173+0000","last_unstale":"2024-04-26T23:50:00.869840+0000","last_undegraded":"2024-04-26T23:50:00.869840+0000","last_fullsized":"2024-04-26T23:50:00.869840+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:38.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860891+0000","last_change":"2024-04-26T23:50:00.849108+0000","last_active":"2024-04-26T23:50:00.860891+0000","last_peered":"2024-04-26T23:50:00.860891+0000","last_clean":"2024-04-26T23:50:00.860891+0000","last_became_active":"2024-04-26T23:50:00.848323+0000","last_became_peered":"2024-04-26T23:50:00.848323+0000","last_unstale":"2024-04-26T23:50:00.860891+0000","last_undegraded":"2024-04-26T23:50:00.860891+0000","last_fullsized":"2024-04-26T23:50:00.860891+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:50.783226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871297+0000","last_change":"2024-04-26T23:50:00.865265+0000","last_active":"2024-04-26T23:50:00.871297+0000","last_peered":"2024-04-26T23:50:00.871297+0000","last_clean":"2024-04-26T23:50:00.871297+0000","last_became_active":"2024-04-26T23:50:00.864951+0000","last_became_peered":"2024-04-26T23:50:00.864951+0000","last_unstale":"2024-04-26T23:50:00.871297+0000","last_undegraded":"2024-04-26T23:50:00.871297+0000","last_fullsized":"2024-04-26T23:50:00.871297+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:35.487834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861944+0000","last_change":"2024-04-26T23:50:00.854900+0000","last_active":"2024-04-26T23:50:00.861944+0000","last_peered":"2024-04-26T23:50:00.861944+0000","last_clean":"2024-04-26T23:50:00.861944+0000","last_became_active":"2024-04-26T23:50:00.854633+0000","last_became_peered":"2024-04-26T23:50:00.854633+0000","last_unstale":"2024-04-26T23:50:00.861944+0000","last_undegraded":"2024-04-26T23:50:00.861944+0000","last_fullsized":"2024-04-26T23:50:00.861944+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:55.501626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862541+0000","last_change":"2024-04-26T23:50:00.857815+0000","last_active":"2024-04-26T23:50:00.862541+0000","last_peered":"2024-04-26T23:50:00.862541+0000","last_clean":"2024-04-26T23:50:00.862541+0000","last_became_active":"2024-04-26T23:50:00.857119+0000","last_became_peered":"2024-04-26T23:50:00.857119+0000","last_unstale":"2024-04-26T23:50:00.862541+0000","last_undegraded":"2024-04-26T23:50:00.862541+0000","last_fullsized":"2024-04-26T23:50:00.862541+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:24.422976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862077+0000","last_change":"2024-04-26T23:50:00.855918+0000","last_active":"2024-04-26T23:50:00.862077+0000","last_peered":"2024-04-26T23:50:00.862077+0000","last_clean":"2024-04-26T23:50:00.862077+0000","last_became_active":"2024-04-26T23:50:00.855623+0000","last_became_peered":"2024-04-26T23:50:00.855623+0000","last_unstale":"2024-04-26T23:50:00.862077+0000","last_undegraded":"2024-04-26T23:50:00.862077+0000","last_fullsized":"2024-04-26T23:50:00.862077+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:25.000532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860225+0000","last_change":"2024-04-26T23:50:00.857018+0000","last_active":"2024-04-26T23:50:00.860225+0000","last_peered":"2024-04-26T23:50:00.860225+0000","last_clean":"2024-04-26T23:50:00.860225+0000","last_became_active":"2024-04-26T23:50:00.853324+0000","last_became_peered":"2024-04-26T23:50:00.853324+0000","last_unstale":"2024-04-26T23:50:00.860225+0000","last_undegraded":"2024-04-26T23:50:00.860225+0000","last_fullsized":"2024-04-26T23:50:00.860225+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:50.057083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.855283+0000","last_change":"2024-04-26T23:50:00.853767+0000","last_active":"2024-04-26T23:50:00.855283+0000","last_peered":"2024-04-26T23:50:00.855283+0000","last_clean":"2024-04-26T23:50:00.855283+0000","last_became_active":"2024-04-26T23:50:00.853330+0000","last_became_peered":"2024-04-26T23:50:00.853330+0000","last_unstale":"2024-04-26T23:50:00.855283+0000","last_undegraded":"2024-04-26T23:50:00.855283+0000","last_fullsized":"2024-04-26T23:50:00.855283+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:42.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861373+0000","last_change":"2024-04-26T23:50:00.852079+0000","last_active":"2024-04-26T23:50:00.861373+0000","last_peered":"2024-04-26T23:50:00.861373+0000","last_clean":"2024-04-26T23:50:00.861373+0000","last_became_active":"2024-04-26T23:50:00.851724+0000","last_became_peered":"2024-04-26T23:50:00.851724+0000","last_unstale":"2024-04-26T23:50:00.861373+0000","last_undegraded":"2024-04-26T23:50:00.861373+0000","last_fullsized":"2024-04-26T23:50:00.861373+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:05.242256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872760+0000","last_change":"2024-04-26T23:50:00.867853+0000","last_active":"2024-04-26T23:50:00.872760+0000","last_peered":"2024-04-26T23:50:00.872760+0000","last_clean":"2024-04-26T23:50:00.872760+0000","last_became_active":"2024-04-26T23:50:00.867574+0000","last_became_peered":"2024-04-26T23:50:00.867574+0000","last_unstale":"2024-04-26T23:50:00.872760+0000","last_undegraded":"2024-04-26T23:50:00.872760+0000","last_fullsized":"2024-04-26T23:50:00.872760+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.997046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871274+0000","last_change":"2024-04-26T23:50:00.863800+0000","last_active":"2024-04-26T23:50:00.871274+0000","last_peered":"2024-04-26T23:50:00.871274+0000","last_clean":"2024-04-26T23:50:00.871274+0000","last_became_active":"2024-04-26T23:50:00.863208+0000","last_became_peered":"2024-04-26T23:50:00.863208+0000","last_unstale":"2024-04-26T23:50:00.871274+0000","last_undegraded":"2024-04-26T23:50:00.871274+0000","last_fullsized":"2024-04-26T23:50:00.871274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:13.535156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.839230+0000","last_change":"2024-04-26T23:50:00.837314+0000","last_active":"2024-04-26T23:50:00.839230+0000","last_peered":"2024-04-26T23:50:00.839230+0000","last_clean":"2024-04-26T23:50:00.839230+0000","last_became_active":"2024-04-26T23:50:00.836583+0000","last_became_peered":"2024-04-26T23:50:00.836583+0000","last_unstale":"2024-04-26T23:50:00.839230+0000","last_undegraded":"2024-04-26T23:50:00.839230+0000","last_fullsized":"2024-04-26T23:50:00.839230+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:52.883640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868145+0000","last_change":"2024-04-26T23:50:00.860950+0000","last_active":"2024-04-26T23:50:00.868145+0000","last_peered":"2024-04-26T23:50:00.868145+0000","last_clean":"2024-04-26T23:50:00.868145+0000","last_became_active":"2024-04-26T23:50:00.856281+0000","last_became_peered":"2024-04-26T23:50:00.856281+0000","last_unstale":"2024-04-26T23:50:00.868145+0000","last_undegraded":"2024-04-26T23:50:00.868145+0000","last_fullsized":"2024-04-26T23:50:00.868145+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:13.201883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.867505+0000","last_change":"2024-04-26T23:50:00.860447+0000","last_active":"2024-04-26T23:50:00.867505+0000","last_peered":"2024-04-26T23:50:00.867505+0000","last_clean":"2024-04-26T23:50:00.867505+0000","last_became_active":"2024-04-26T23:50:00.855300+0000","last_became_peered":"2024-04-26T23:50:00.855300+0000","last_unstale":"2024-04-26T23:50:00.867505+0000","last_undegraded":"2024-04-26T23:50:00.867505+0000","last_fullsized":"2024-04-26T23:50:00.867505+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:21.748505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861680+0000","last_change":"2024-04-26T23:50:00.860019+0000","last_active":"2024-04-26T23:50:00.861680+0000","last_peered":"2024-04-26T23:50:00.861680+0000","last_clean":"2024-04-26T23:50:00.861680+0000","last_became_active":"2024-04-26T23:50:00.858688+0000","last_became_peered":"2024-04-26T23:50:00.858688+0000","last_unstale":"2024-04-26T23:50:00.861680+0000","last_undegraded":"2024-04-26T23:50:00.861680+0000","last_fullsized":"2024-04-26T23:50:00.861680+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:42.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860688+0000","last_change":"2024-04-26T23:50:00.858160+0000","last_active":"2024-04-26T23:50:00.860688+0000","last_peered":"2024-04-26T23:50:00.860688+0000","last_clean":"2024-04-26T23:50:00.860688+0000","last_became_active":"2024-04-26T23:50:00.856402+0000","last_became_peered":"2024-04-26T23:50:00.856402+0000","last_unstale":"2024-04-26T23:50:00.860688+0000","last_undegraded":"2024-04-26T23:50:00.860688+0000","last_fullsized":"2024-04-26T23:50:00.860688+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.006515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861105+0000","last_change":"2024-04-26T23:50:00.853620+0000","last_active":"2024-04-26T23:50:00.861105+0000","last_peered":"2024-04-26T23:50:00.861105+0000","last_clean":"2024-04-26T23:50:00.861105+0000","last_became_active":"2024-04-26T23:50:00.853312+0000","last_became_peered":"2024-04-26T23:50:00.853312+0000","last_unstale":"2024-04-26T23:50:00.861105+0000","last_undegraded":"2024-04-26T23:50:00.861105+0000","last_fullsized":"2024-04-26T23:50:00.861105+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:38.189722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"136'4","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.877416+0000","last_change":"2024-04-26T23:49:59.852100+0000","last_active":"2024-04-26T23:50:00.877416+0000","last_peered":"2024-04-26T23:50:00.877416+0000","last_clean":"2024-04-26T23:50:00.877416+0000","last_became_active":"2024-04-26T23:49:59.851102+0000","last_became_peered":"2024-04-26T23:49:59.851102+0000","last_unstale":"2024-04-26T23:50:00.877416+0000","last_undegraded":"2024-04-26T23:50:00.877416+0000","last_fullsized":"2024-04-26T23:50:00.877416+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:35.312757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863701+0000","last_change":"2024-04-26T23:50:00.857753+0000","last_active":"2024-04-26T23:50:00.863701+0000","last_peered":"2024-04-26T23:50:00.863701+0000","last_clean":"2024-04-26T23:50:00.863701+0000","last_became_active":"2024-04-26T23:50:00.856361+0000","last_became_peered":"2024-04-26T23:50:00.856361+0000","last_unstale":"2024-04-26T23:50:00.863701+0000","last_undegraded":"2024-04-26T23:50:00.863701+0000","last_fullsized":"2024-04-26T23:50:00.863701+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:24.783686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873205+0000","last_change":"2024-04-26T23:49:59.851695+0000","last_active":"2024-04-26T23:50:00.873205+0000","last_peered":"2024-04-26T23:50:00.873205+0000","last_clean":"2024-04-26T23:50:00.873205+0000","last_became_active":"2024-04-26T23:49:59.850292+0000","last_became_peered":"2024-04-26T23:49:59.850292+0000","last_unstale":"2024-04-26T23:50:00.873205+0000","last_undegraded":"2024-04-26T23:50:00.873205+0000","last_fullsized":"2024-04-26T23:50:00.873205+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.054946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869404+0000","last_change":"2024-04-26T23:50:00.866785+0000","last_active":"2024-04-26T23:50:00.869404+0000","last_peered":"2024-04-26T23:50:00.869404+0000","last_clean":"2024-04-26T23:50:00.869404+0000","last_became_active":"2024-04-26T23:50:00.864403+0000","last_became_peered":"2024-04-26T23:50:00.864403+0000","last_unstale":"2024-04-26T23:50:00.869404+0000","last_undegraded":"2024-04-26T23:50:00.869404+0000","last_fullsized":"2024-04-26T23:50:00.869404+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:00.767988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872713+0000","last_change":"2024-04-26T23:49:59.849888+0000","last_active":"2024-04-26T23:50:00.872713+0000","last_peered":"2024-04-26T23:50:00.872713+0000","last_clean":"2024-04-26T23:50:00.872713+0000","last_became_active":"2024-04-26T23:49:59.849581+0000","last_became_peered":"2024-04-26T23:49:59.849581+0000","last_unstale":"2024-04-26T23:50:00.872713+0000","last_undegraded":"2024-04-26T23:50:00.872713+0000","last_fullsized":"2024-04-26T23:50:00.872713+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:45.577431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866958+0000","last_change":"2024-04-26T23:50:00.863549+0000","last_active":"2024-04-26T23:50:00.866958+0000","last_peered":"2024-04-26T23:50:00.866958+0000","last_clean":"2024-04-26T23:50:00.866958+0000","last_became_active":"2024-04-26T23:50:00.863278+0000","last_became_peered":"2024-04-26T23:50:00.863278+0000","last_unstale":"2024-04-26T23:50:00.866958+0000","last_undegraded":"2024-04-26T23:50:00.866958+0000","last_fullsized":"2024-04-26T23:50:00.866958+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:11.919852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873570+0000","last_change":"2024-04-26T23:49:59.847124+0000","last_active":"2024-04-26T23:50:00.873570+0000","last_peered":"2024-04-26T23:50:00.873570+0000","last_clean":"2024-04-26T23:50:00.873570+0000","last_became_active":"2024-04-26T23:49:59.846033+0000","last_became_peered":"2024-04-26T23:49:59.846033+0000","last_unstale":"2024-04-26T23:50:00.873570+0000","last_undegraded":"2024-04-26T23:50:00.873570+0000","last_fullsized":"2024-04-26T23:50:00.873570+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:23.407994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860298+0000","last_change":"2024-04-26T23:50:00.857951+0000","last_active":"2024-04-26T23:50:00.860298+0000","last_peered":"2024-04-26T23:50:00.860298+0000","last_clean":"2024-04-26T23:50:00.860298+0000","last_became_active":"2024-04-26T23:50:00.857655+0000","last_became_peered":"2024-04-26T23:50:00.857655+0000","last_unstale":"2024-04-26T23:50:00.860298+0000","last_undegraded":"2024-04-26T23:50:00.860298+0000","last_fullsized":"2024-04-26T23:50:00.860298+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:32.768819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871360+0000","last_change":"2024-04-26T23:49:59.842532+0000","last_active":"2024-04-26T23:50:00.871360+0000","last_peered":"2024-04-26T23:50:00.871360+0000","last_clean":"2024-04-26T23:50:00.871360+0000","last_became_active":"2024-04-26T23:49:59.841078+0000","last_became_peered":"2024-04-26T23:49:59.841078+0000","last_unstale":"2024-04-26T23:50:00.871360+0000","last_undegraded":"2024-04-26T23:50:00.871360+0000","last_fullsized":"2024-04-26T23:50:00.871360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:33.673898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846417+0000","last_change":"2024-04-26T23:50:00.841353+0000","last_active":"2024-04-26T23:50:00.846417+0000","last_peered":"2024-04-26T23:50:00.846417+0000","last_clean":"2024-04-26T23:50:00.846417+0000","last_became_active":"2024-04-26T23:50:00.840069+0000","last_became_peered":"2024-04-26T23:50:00.840069+0000","last_unstale":"2024-04-26T23:50:00.846417+0000","last_undegraded":"2024-04-26T23:50:00.846417+0000","last_fullsized":"2024-04-26T23:50:00.846417+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:59.893607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874693+0000","last_change":"2024-04-26T23:49:59.849948+0000","last_active":"2024-04-26T23:50:00.874693+0000","last_peered":"2024-04-26T23:50:00.874693+0000","last_clean":"2024-04-26T23:50:00.874693+0000","last_became_active":"2024-04-26T23:49:59.848958+0000","last_became_peered":"2024-04-26T23:49:59.848958+0000","last_unstale":"2024-04-26T23:50:00.874693+0000","last_undegraded":"2024-04-26T23:50:00.874693+0000","last_fullsized":"2024-04-26T23:50:00.874693+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:59.753006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862100+0000","last_change":"2024-04-26T23:50:00.853433+0000","last_active":"2024-04-26T23:50:00.862100+0000","last_peered":"2024-04-26T23:50:00.862100+0000","last_clean":"2024-04-26T23:50:00.862100+0000","last_became_active":"2024-04-26T23:50:00.853146+0000","last_became_peered":"2024-04-26T23:50:00.853146+0000","last_unstale":"2024-04-26T23:50:00.862100+0000","last_undegraded":"2024-04-26T23:50:00.862100+0000","last_fullsized":"2024-04-26T23:50:00.862100+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:44.892200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.842584+0000","last_change":"2024-04-26T23:49:59.854340+0000","last_active":"2024-04-26T23:50:00.842584+0000","last_peered":"2024-04-26T23:50:00.842584+0000","last_clean":"2024-04-26T23:50:00.842584+0000","last_became_active":"2024-04-26T23:49:59.854075+0000","last_became_peered":"2024-04-26T23:49:59.854075+0000","last_unstale":"2024-04-26T23:50:00.842584+0000","last_undegraded":"2024-04-26T23:50:00.842584+0000","last_fullsized":"2024-04-26T23:50:00.842584+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:58.878624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871143+0000","last_change":"2024-04-26T23:50:00.866146+0000","last_active":"2024-04-26T23:50:00.871143+0000","last_peered":"2024-04-26T23:50:00.871143+0000","last_clean":"2024-04-26T23:50:00.871143+0000","last_became_active":"2024-04-26T23:50:00.865411+0000","last_became_peered":"2024-04-26T23:50:00.865411+0000","last_unstale":"2024-04-26T23:50:00.871143+0000","last_undegraded":"2024-04-26T23:50:00.871143+0000","last_fullsized":"2024-04-26T23:50:00.871143+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:58.298236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841290+0000","last_change":"2024-04-26T23:49:59.846640+0000","last_active":"2024-04-26T23:50:00.841290+0000","last_peered":"2024-04-26T23:50:00.841290+0000","last_clean":"2024-04-26T23:50:00.841290+0000","last_became_active":"2024-04-26T23:49:59.846298+0000","last_became_peered":"2024-04-26T23:49:59.846298+0000","last_unstale":"2024-04-26T23:50:00.841290+0000","last_undegraded":"2024-04-26T23:50:00.841290+0000","last_fullsized":"2024-04-26T23:50:00.841290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:22.494162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863783+0000","last_change":"2024-04-26T23:50:00.852780+0000","last_active":"2024-04-26T23:50:00.863783+0000","last_peered":"2024-04-26T23:50:00.863783+0000","last_clean":"2024-04-26T23:50:00.863783+0000","last_became_active":"2024-04-26T23:50:00.852335+0000","last_became_peered":"2024-04-26T23:50:00.852335+0000","last_unstale":"2024-04-26T23:50:00.863783+0000","last_undegraded":"2024-04-26T23:50:00.863783+0000","last_fullsized":"2024-04-26T23:50:00.863783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:44.530709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844638+0000","last_change":"2024-04-26T23:49:59.850292+0000","last_active":"2024-04-26T23:50:00.844638+0000","last_peered":"2024-04-26T23:50:00.844638+0000","last_clean":"2024-04-26T23:50:00.844638+0000","last_became_active":"2024-04-26T23:49:59.849943+0000","last_became_peered":"2024-04-26T23:49:59.849943+0000","last_unstale":"2024-04-26T23:50:00.844638+0000","last_undegraded":"2024-04-26T23:50:00.844638+0000","last_fullsized":"2024-04-26T23:50:00.844638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:59.436581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860862+0000","last_change":"2024-04-26T23:50:00.856722+0000","last_active":"2024-04-26T23:50:00.860862+0000","last_peered":"2024-04-26T23:50:00.860862+0000","last_clean":"2024-04-26T23:50:00.860862+0000","last_became_active":"2024-04-26T23:50:00.856319+0000","last_became_peered":"2024-04-26T23:50:00.856319+0000","last_unstale":"2024-04-26T23:50:00.860862+0000","last_undegraded":"2024-04-26T23:50:00.860862+0000","last_fullsized":"2024-04-26T23:50:00.860862+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:12.107111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846198+0000","last_change":"2024-04-26T23:49:59.844480+0000","last_active":"2024-04-26T23:50:00.846198+0000","last_peered":"2024-04-26T23:50:00.846198+0000","last_clean":"2024-04-26T23:50:00.846198+0000","last_became_active":"2024-04-26T23:49:59.839930+0000","last_became_peered":"2024-04-26T23:49:59.839930+0000","last_unstale":"2024-04-26T23:50:00.846198+0000","last_undegraded":"2024-04-26T23:50:00.846198+0000","last_fullsized":"2024-04-26T23:50:00.846198+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.005609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874546+0000","last_change":"2024-04-26T23:50:00.871446+0000","last_active":"2024-04-26T23:50:00.874546+0000","last_peered":"2024-04-26T23:50:00.874546+0000","last_clean":"2024-04-26T23:50:00.874546+0000","last_became_active":"2024-04-26T23:50:00.870647+0000","last_became_peered":"2024-04-26T23:50:00.870647+0000","last_unstale":"2024-04-26T23:50:00.874546+0000","last_undegraded":"2024-04-26T23:50:00.874546+0000","last_fullsized":"2024-04-26T23:50:00.874546+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:38.189722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"136'3","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.875487+0000","last_change":"2024-04-26T23:49:59.845504+0000","last_active":"2024-04-26T23:50:00.875487+0000","last_peered":"2024-04-26T23:50:00.875487+0000","last_clean":"2024-04-26T23:50:00.875487+0000","last_became_active":"2024-04-26T23:49:59.845182+0000","last_became_peered":"2024-04-26T23:49:59.845182+0000","last_unstale":"2024-04-26T23:50:00.875487+0000","last_undegraded":"2024-04-26T23:50:00.875487+0000","last_fullsized":"2024-04-26T23:50:00.875487+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.054946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868120+0000","last_change":"2024-04-26T23:50:00.853238+0000","last_active":"2024-04-26T23:50:00.868120+0000","last_peered":"2024-04-26T23:50:00.868120+0000","last_clean":"2024-04-26T23:50:00.868120+0000","last_became_active":"2024-04-26T23:50:00.852940+0000","last_became_peered":"2024-04-26T23:50:00.852940+0000","last_unstale":"2024-04-26T23:50:00.868120+0000","last_undegraded":"2024-04-26T23:50:00.868120+0000","last_fullsized":"2024-04-26T23:50:00.868120+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:57.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"136'6","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.876365+0000","last_change":"2024-04-26T23:49:59.852856+0000","last_active":"2024-04-26T23:50:00.876365+0000","last_peered":"2024-04-26T23:50:00.876365+0000","last_clean":"2024-04-26T23:50:00.876365+0000","last_became_active":"2024-04-26T23:49:59.852511+0000","last_became_peered":"2024-04-26T23:49:59.852511+0000","last_unstale":"2024-04-26T23:50:00.876365+0000","last_undegraded":"2024-04-26T23:50:00.876365+0000","last_fullsized":"2024-04-26T23:50:00.876365+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:59.753006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.853822+0000","last_change":"2024-04-26T23:50:00.851468+0000","last_active":"2024-04-26T23:50:00.853822+0000","last_peered":"2024-04-26T23:50:00.853822+0000","last_clean":"2024-04-26T23:50:00.853822+0000","last_became_active":"2024-04-26T23:50:00.850963+0000","last_became_peered":"2024-04-26T23:50:00.850963+0000","last_unstale":"2024-04-26T23:50:00.853822+0000","last_undegraded":"2024-04-26T23:50:00.853822+0000","last_fullsized":"2024-04-26T23:50:00.853822+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:48.065440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873372+0000","last_change":"2024-04-26T23:49:59.844293+0000","last_active":"2024-04-26T23:50:00.873372+0000","last_peered":"2024-04-26T23:50:00.873372+0000","last_clean":"2024-04-26T23:50:00.873372+0000","last_became_active":"2024-04-26T23:49:59.843778+0000","last_became_peered":"2024-04-26T23:49:59.843778+0000","last_unstale":"2024-04-26T23:50:00.873372+0000","last_undegraded":"2024-04-26T23:50:00.873372+0000","last_fullsized":"2024-04-26T23:50:00.873372+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:57.819476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.867920+0000","last_change":"2024-04-26T23:50:00.861121+0000","last_active":"2024-04-26T23:50:00.867920+0000","last_peered":"2024-04-26T23:50:00.867920+0000","last_clean":"2024-04-26T23:50:00.867920+0000","last_became_active":"2024-04-26T23:50:00.860820+0000","last_became_peered":"2024-04-26T23:50:00.860820+0000","last_unstale":"2024-04-26T23:50:00.867920+0000","last_undegraded":"2024-04-26T23:50:00.867920+0000","last_fullsized":"2024-04-26T23:50:00.867920+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.837408+0000","last_change":"2024-04-26T23:49:59.840782+0000","last_active":"2024-04-26T23:50:00.837408+0000","last_peered":"2024-04-26T23:50:00.837408+0000","last_clean":"2024-04-26T23:50:00.837408+0000","last_became_active":"2024-04-26T23:49:59.838335+0000","last_became_peered":"2024-04-26T23:49:59.838335+0000","last_unstale":"2024-04-26T23:50:00.837408+0000","last_undegraded":"2024-04-26T23:50:00.837408+0000","last_fullsized":"2024-04-26T23:50:00.837408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:38:01.427776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863653+0000","last_change":"2024-04-26T23:50:00.858082+0000","last_active":"2024-04-26T23:50:00.863653+0000","last_peered":"2024-04-26T23:50:00.863653+0000","last_clean":"2024-04-26T23:50:00.863653+0000","last_became_active":"2024-04-26T23:50:00.857764+0000","last_became_peered":"2024-04-26T23:50:00.857764+0000","last_unstale":"2024-04-26T23:50:00.863653+0000","last_undegraded":"2024-04-26T23:50:00.863653+0000","last_fullsized":"2024-04-26T23:50:00.863653+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:35.487834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"136'8","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.878384+0000","last_change":"2024-04-26T23:49:59.844977+0000","last_active":"2024-04-26T23:50:00.878384+0000","last_peered":"2024-04-26T23:50:00.878384+0000","last_clean":"2024-04-26T23:50:00.878384+0000","last_became_active":"2024-04-26T23:49:59.844701+0000","last_became_peered":"2024-04-26T23:49:59.844701+0000","last_unstale":"2024-04-26T23:50:00.878384+0000","last_undegraded":"2024-04-26T23:50:00.878384+0000","last_fullsized":"2024-04-26T23:50:00.878384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:20.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866768+0000","last_change":"2024-04-26T23:50:00.859161+0000","last_active":"2024-04-26T23:50:00.866768+0000","last_peered":"2024-04-26T23:50:00.866768+0000","last_clean":"2024-04-26T23:50:00.866768+0000","last_became_active":"2024-04-26T23:50:00.855871+0000","last_became_peered":"2024-04-26T23:50:00.855871+0000","last_unstale":"2024-04-26T23:50:00.866768+0000","last_undegraded":"2024-04-26T23:50:00.866768+0000","last_fullsized":"2024-04-26T23:50:00.866768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.467001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.868291+0000","last_change":"2024-04-26T23:49:59.834451+0000","last_active":"2024-04-26T23:50:00.868291+0000","last_peered":"2024-04-26T23:50:00.868291+0000","last_clean":"2024-04-26T23:50:00.868291+0000","last_became_active":"2024-04-26T23:49:59.833528+0000","last_became_peered":"2024-04-26T23:49:59.833528+0000","last_unstale":"2024-04-26T23:50:00.868291+0000","last_undegraded":"2024-04-26T23:50:00.868291+0000","last_fullsized":"2024-04-26T23:50:00.868291+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:15.088216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.872050+0000","last_change":"2024-04-26T23:50:00.869645+0000","last_active":"2024-04-26T23:50:00.872050+0000","last_peered":"2024-04-26T23:50:00.872050+0000","last_clean":"2024-04-26T23:50:00.872050+0000","last_became_active":"2024-04-26T23:50:00.867846+0000","last_became_peered":"2024-04-26T23:50:00.867846+0000","last_unstale":"2024-04-26T23:50:00.872050+0000","last_undegraded":"2024-04-26T23:50:00.872050+0000","last_fullsized":"2024-04-26T23:50:00.872050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:55.599385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.877082+0000","last_change":"2024-04-26T23:49:59.851000+0000","last_active":"2024-04-26T23:50:00.877082+0000","last_peered":"2024-04-26T23:50:00.877082+0000","last_clean":"2024-04-26T23:50:00.877082+0000","last_became_active":"2024-04-26T23:49:59.850498+0000","last_became_peered":"2024-04-26T23:49:59.850498+0000","last_unstale":"2024-04-26T23:50:00.877082+0000","last_undegraded":"2024-04-26T23:50:00.877082+0000","last_fullsized":"2024-04-26T23:50:00.877082+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:08.787493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871096+0000","last_change":"2024-04-26T23:50:00.864432+0000","last_active":"2024-04-26T23:50:00.871096+0000","last_peered":"2024-04-26T23:50:00.871096+0000","last_clean":"2024-04-26T23:50:00.871096+0000","last_became_active":"2024-04-26T23:50:00.864078+0000","last_became_peered":"2024-04-26T23:50:00.864078+0000","last_unstale":"2024-04-26T23:50:00.871096+0000","last_undegraded":"2024-04-26T23:50:00.871096+0000","last_fullsized":"2024-04-26T23:50:00.871096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:44.892200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"136'1","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.874256+0000","last_change":"2024-04-26T23:49:59.852297+0000","last_active":"2024-04-26T23:50:00.874256+0000","last_peered":"2024-04-26T23:50:00.874256+0000","last_clean":"2024-04-26T23:50:00.874256+0000","last_became_active":"2024-04-26T23:49:59.852001+0000","last_became_peered":"2024-04-26T23:49:59.852001+0000","last_unstale":"2024-04-26T23:50:00.874256+0000","last_undegraded":"2024-04-26T23:50:00.874256+0000","last_fullsized":"2024-04-26T23:50:00.874256+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:37.174739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.855769+0000","last_change":"2024-04-26T23:50:00.855019+0000","last_active":"2024-04-26T23:50:00.855769+0000","last_peered":"2024-04-26T23:50:00.855769+0000","last_clean":"2024-04-26T23:50:00.855769+0000","last_became_active":"2024-04-26T23:50:00.854612+0000","last_became_peered":"2024-04-26T23:50:00.854612+0000","last_unstale":"2024-04-26T23:50:00.855769+0000","last_undegraded":"2024-04-26T23:50:00.855769+0000","last_fullsized":"2024-04-26T23:50:00.855769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:53.855781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.832693+0000","last_change":"2024-04-26T23:49:59.854569+0000","last_active":"2024-04-26T23:50:00.832693+0000","last_peered":"2024-04-26T23:50:00.832693+0000","last_clean":"2024-04-26T23:50:00.832693+0000","last_became_active":"2024-04-26T23:49:59.854261+0000","last_became_peered":"2024-04-26T23:49:59.854261+0000","last_unstale":"2024-04-26T23:50:00.832693+0000","last_undegraded":"2024-04-26T23:50:00.832693+0000","last_fullsized":"2024-04-26T23:50:00.832693+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:45.577431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861256+0000","last_change":"2024-04-26T23:50:00.852121+0000","last_active":"2024-04-26T23:50:00.861256+0000","last_peered":"2024-04-26T23:50:00.861256+0000","last_clean":"2024-04-26T23:50:00.861256+0000","last_became_active":"2024-04-26T23:50:00.851726+0000","last_became_peered":"2024-04-26T23:50:00.851726+0000","last_unstale":"2024-04-26T23:50:00.861256+0000","last_undegraded":"2024-04-26T23:50:00.861256+0000","last_fullsized":"2024-04-26T23:50:00.861256+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:20.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873384+0000","last_change":"2024-04-26T23:49:59.842450+0000","last_active":"2024-04-26T23:50:00.873384+0000","last_peered":"2024-04-26T23:50:00.873384+0000","last_clean":"2024-04-26T23:50:00.873384+0000","last_became_active":"2024-04-26T23:49:59.840033+0000","last_became_peered":"2024-04-26T23:49:59.840033+0000","last_unstale":"2024-04-26T23:50:00.873384+0000","last_undegraded":"2024-04-26T23:50:00.873384+0000","last_fullsized":"2024-04-26T23:50:00.873384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:27.452018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866782+0000","last_change":"2024-04-26T23:50:00.863742+0000","last_active":"2024-04-26T23:50:00.866782+0000","last_peered":"2024-04-26T23:50:00.866782+0000","last_clean":"2024-04-26T23:50:00.866782+0000","last_became_active":"2024-04-26T23:50:00.863450+0000","last_became_peered":"2024-04-26T23:50:00.863450+0000","last_unstale":"2024-04-26T23:50:00.866782+0000","last_undegraded":"2024-04-26T23:50:00.866782+0000","last_fullsized":"2024-04-26T23:50:00.866782+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:18.945144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841765+0000","last_change":"2024-04-26T23:49:59.845287+0000","last_active":"2024-04-26T23:50:00.841765+0000","last_peered":"2024-04-26T23:50:00.841765+0000","last_clean":"2024-04-26T23:50:00.841765+0000","last_became_active":"2024-04-26T23:49:59.844977+0000","last_became_peered":"2024-04-26T23:49:59.844977+0000","last_unstale":"2024-04-26T23:50:00.841765+0000","last_undegraded":"2024-04-26T23:50:00.841765+0000","last_fullsized":"2024-04-26T23:50:00.841765+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:19.733155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.860918+0000","last_change":"2024-04-26T23:50:00.849374+0000","last_active":"2024-04-26T23:50:00.860918+0000","last_peered":"2024-04-26T23:50:00.860918+0000","last_clean":"2024-04-26T23:50:00.860918+0000","last_became_active":"2024-04-26T23:50:00.848765+0000","last_became_peered":"2024-04-26T23:50:00.848765+0000","last_unstale":"2024-04-26T23:50:00.860918+0000","last_undegraded":"2024-04-26T23:50:00.860918+0000","last_fullsized":"2024-04-26T23:50:00.860918+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.850576+0000","last_change":"2024-04-26T23:49:59.845887+0000","last_active":"2024-04-26T23:50:00.850576+0000","last_peered":"2024-04-26T23:50:00.850576+0000","last_clean":"2024-04-26T23:50:00.850576+0000","last_became_active":"2024-04-26T23:49:59.845575+0000","last_became_peered":"2024-04-26T23:49:59.845575+0000","last_unstale":"2024-04-26T23:50:00.850576+0000","last_undegraded":"2024-04-26T23:50:00.850576+0000","last_fullsized":"2024-04-26T23:50:00.850576+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:49.768244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.859893+0000","last_change":"2024-04-26T23:50:00.856126+0000","last_active":"2024-04-26T23:50:00.859893+0000","last_peered":"2024-04-26T23:50:00.859893+0000","last_clean":"2024-04-26T23:50:00.859893+0000","last_became_active":"2024-04-26T23:50:00.855820+0000","last_became_peered":"2024-04-26T23:50:00.855820+0000","last_unstale":"2024-04-26T23:50:00.859893+0000","last_undegraded":"2024-04-26T23:50:00.859893+0000","last_fullsized":"2024-04-26T23:50:00.859893+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:08.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.841941+0000","last_change":"2024-04-26T23:49:59.848225+0000","last_active":"2024-04-26T23:50:00.841941+0000","last_peered":"2024-04-26T23:50:00.841941+0000","last_clean":"2024-04-26T23:50:00.841941+0000","last_became_active":"2024-04-26T23:49:59.847907+0000","last_became_peered":"2024-04-26T23:49:59.847907+0000","last_unstale":"2024-04-26T23:50:00.841941+0000","last_undegraded":"2024-04-26T23:50:00.841941+0000","last_fullsized":"2024-04-26T23:50:00.841941+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:12.186900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.861311+0000","last_change":"2024-04-26T23:50:00.852154+0000","last_active":"2024-04-26T23:50:00.861311+0000","last_peered":"2024-04-26T23:50:00.861311+0000","last_clean":"2024-04-26T23:50:00.861311+0000","last_became_active":"2024-04-26T23:50:00.851872+0000","last_became_peered":"2024-04-26T23:50:00.851872+0000","last_unstale":"2024-04-26T23:50:00.861311+0000","last_undegraded":"2024-04-26T23:50:00.861311+0000","last_fullsized":"2024-04-26T23:50:00.861311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:13.535156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"136'2","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.875564+0000","last_change":"2024-04-26T23:49:59.847737+0000","last_active":"2024-04-26T23:50:00.875564+0000","last_peered":"2024-04-26T23:50:00.875564+0000","last_clean":"2024-04-26T23:50:00.875564+0000","last_became_active":"2024-04-26T23:49:59.847312+0000","last_became_peered":"2024-04-26T23:49:59.847312+0000","last_unstale":"2024-04-26T23:50:00.875564+0000","last_undegraded":"2024-04-26T23:50:00.875564+0000","last_fullsized":"2024-04-26T23:50:00.875564+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.189674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.866816+0000","last_change":"2024-04-26T23:50:00.859222+0000","last_active":"2024-04-26T23:50:00.866816+0000","last_peered":"2024-04-26T23:50:00.866816+0000","last_clean":"2024-04-26T23:50:00.866816+0000","last_became_active":"2024-04-26T23:50:00.854804+0000","last_became_peered":"2024-04-26T23:50:00.854804+0000","last_unstale":"2024-04-26T23:50:00.866816+0000","last_undegraded":"2024-04-26T23:50:00.866816+0000","last_fullsized":"2024-04-26T23:50:00.866816+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:34.688881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.831270+0000","last_change":"2024-04-26T23:49:59.839499+0000","last_active":"2024-04-26T23:50:00.831270+0000","last_peered":"2024-04-26T23:50:00.831270+0000","last_clean":"2024-04-26T23:50:00.831270+0000","last_became_active":"2024-04-26T23:49:59.839173+0000","last_became_peered":"2024-04-26T23:49:59.839173+0000","last_unstale":"2024-04-26T23:50:00.831270+0000","last_undegraded":"2024-04-26T23:50:00.831270+0000","last_fullsized":"2024-04-26T23:50:00.831270+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:01:12.186900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.873614+0000","last_change":"2024-04-26T23:50:00.870948+0000","last_active":"2024-04-26T23:50:00.873614+0000","last_peered":"2024-04-26T23:50:00.873614+0000","last_clean":"2024-04-26T23:50:00.873614+0000","last_became_active":"2024-04-26T23:50:00.870637+0000","last_became_peered":"2024-04-26T23:50:00.870637+0000","last_unstale":"2024-04-26T23:50:00.873614+0000","last_undegraded":"2024-04-26T23:50:00.873614+0000","last_fullsized":"2024-04-26T23:50:00.873614+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:30.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.830808+0000","last_change":"2024-04-26T23:49:59.848118+0000","last_active":"2024-04-26T23:50:00.830808+0000","last_peered":"2024-04-26T23:50:00.830808+0000","last_clean":"2024-04-26T23:50:00.830808+0000","last_became_active":"2024-04-26T23:49:59.847612+0000","last_became_peered":"2024-04-26T23:49:59.847612+0000","last_unstale":"2024-04-26T23:50:00.830808+0000","last_undegraded":"2024-04-26T23:50:00.830808+0000","last_fullsized":"2024-04-26T23:50:00.830808+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:13:20.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844661+0000","last_change":"2024-04-26T23:50:00.838771+0000","last_active":"2024-04-26T23:50:00.844661+0000","last_peered":"2024-04-26T23:50:00.844661+0000","last_clean":"2024-04-26T23:50:00.844661+0000","last_became_active":"2024-04-26T23:50:00.837807+0000","last_became_peered":"2024-04-26T23:50:00.837807+0000","last_unstale":"2024-04-26T23:50:00.844661+0000","last_undegraded":"2024-04-26T23:50:00.844661+0000","last_fullsized":"2024-04-26T23:50:00.844661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:04.209908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.846072+0000","last_change":"2024-04-26T23:49:59.853685+0000","last_active":"2024-04-26T23:50:00.846072+0000","last_peered":"2024-04-26T23:50:00.846072+0000","last_clean":"2024-04-26T23:50:00.846072+0000","last_became_active":"2024-04-26T23:49:59.853326+0000","last_became_peered":"2024-04-26T23:49:59.853326+0000","last_unstale":"2024-04-26T23:50:00.846072+0000","last_undegraded":"2024-04-26T23:50:00.846072+0000","last_fullsized":"2024-04-26T23:50:00.846072+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:31.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862395+0000","last_change":"2024-04-26T23:50:00.856294+0000","last_active":"2024-04-26T23:50:00.862395+0000","last_peered":"2024-04-26T23:50:00.862395+0000","last_clean":"2024-04-26T23:50:00.862395+0000","last_became_active":"2024-04-26T23:50:00.853114+0000","last_became_peered":"2024-04-26T23:50:00.853114+0000","last_unstale":"2024-04-26T23:50:00.862395+0000","last_undegraded":"2024-04-26T23:50:00.862395+0000","last_fullsized":"2024-04-26T23:50:00.862395+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:45.360258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.844494+0000","last_change":"2024-04-26T23:49:59.840842+0000","last_active":"2024-04-26T23:50:00.844494+0000","last_peered":"2024-04-26T23:50:00.844494+0000","last_clean":"2024-04-26T23:50:00.844494+0000","last_became_active":"2024-04-26T23:49:59.840540+0000","last_became_peered":"2024-04-26T23:49:59.840540+0000","last_unstale":"2024-04-26T23:50:00.844494+0000","last_undegraded":"2024-04-26T23:50:00.844494+0000","last_fullsized":"2024-04-26T23:50:00.844494+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:11.092128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.863986+0000","last_change":"2024-04-26T23:50:00.861791+0000","last_active":"2024-04-26T23:50:00.863986+0000","last_peered":"2024-04-26T23:50:00.863986+0000","last_clean":"2024-04-26T23:50:00.863986+0000","last_became_active":"2024-04-26T23:50:00.861148+0000","last_became_peered":"2024-04-26T23:50:00.861148+0000","last_unstale":"2024-04-26T23:50:00.863986+0000","last_undegraded":"2024-04-26T23:50:00.863986+0000","last_fullsized":"2024-04-26T23:50:00.863986+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:00.451564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.849053+0000","last_change":"2024-04-26T23:49:59.846801+0000","last_active":"2024-04-26T23:50:00.849053+0000","last_peered":"2024-04-26T23:50:00.849053+0000","last_clean":"2024-04-26T23:50:00.849053+0000","last_became_active":"2024-04-26T23:49:59.846004+0000","last_became_peered":"2024-04-26T23:49:59.846004+0000","last_unstale":"2024-04-26T23:50:00.849053+0000","last_undegraded":"2024-04-26T23:50:00.849053+0000","last_fullsized":"2024-04-26T23:50:00.849053+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:09.759634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.869812+0000","last_change":"2024-04-26T23:50:00.867775+0000","last_active":"2024-04-26T23:50:00.869812+0000","last_peered":"2024-04-26T23:50:00.869812+0000","last_clean":"2024-04-26T23:50:00.869812+0000","last_became_active":"2024-04-26T23:50:00.867469+0000","last_became_peered":"2024-04-26T23:50:00.867469+0000","last_unstale":"2024-04-26T23:50:00.869812+0000","last_undegraded":"2024-04-26T23:50:00.869812+0000","last_fullsized":"2024-04-26T23:50:00.869812+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:11.058674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.842757+0000","last_change":"2024-04-26T23:49:59.847495+0000","last_active":"2024-04-26T23:50:00.842757+0000","last_peered":"2024-04-26T23:50:00.842757+0000","last_clean":"2024-04-26T23:50:00.842757+0000","last_became_active":"2024-04-26T23:49:59.847203+0000","last_became_peered":"2024-04-26T23:49:59.847203+0000","last_unstale":"2024-04-26T23:50:00.842757+0000","last_undegraded":"2024-04-26T23:50:00.842757+0000","last_fullsized":"2024-04-26T23:50:00.842757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:53.982063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.871120+0000","last_change":"2024-04-26T23:50:00.864807+0000","last_active":"2024-04-26T23:50:00.871120+0000","last_peered":"2024-04-26T23:50:00.871120+0000","last_clean":"2024-04-26T23:50:00.871120+0000","last_became_active":"2024-04-26T23:50:00.863417+0000","last_became_peered":"2024-04-26T23:50:00.863417+0000","last_unstale":"2024-04-26T23:50:00.871120+0000","last_undegraded":"2024-04-26T23:50:00.871120+0000","last_fullsized":"2024-04-26T23:50:00.871120+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:55.501626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.834191+0000","last_change":"2024-04-26T23:49:59.846032+0000","last_active":"2024-04-26T23:50:00.834191+0000","last_peered":"2024-04-26T23:50:00.834191+0000","last_clean":"2024-04-26T23:50:00.834191+0000","last_became_active":"2024-04-26T23:49:59.845473+0000","last_became_peered":"2024-04-26T23:49:59.845473+0000","last_unstale":"2024-04-26T23:50:00.834191+0000","last_undegraded":"2024-04-26T23:50:00.834191+0000","last_fullsized":"2024-04-26T23:50:00.834191+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:09.759634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.862426+0000","last_change":"2024-04-26T23:50:00.854897+0000","last_active":"2024-04-26T23:50:00.862426+0000","last_peered":"2024-04-26T23:50:00.862426+0000","last_clean":"2024-04-26T23:50:00.862426+0000","last_became_active":"2024-04-26T23:50:00.854473+0000","last_became_peered":"2024-04-26T23:50:00.854473+0000","last_unstale":"2024-04-26T23:50:00.862426+0000","last_undegraded":"2024-04-26T23:50:00.862426+0000","last_fullsized":"2024-04-26T23:50:00.862426+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:59.795827+0000","last_clean_scrub_stamp":"2024-04-26T23:49:59.795827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:22.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.849236+0000","last_change":"2024-04-26T23:49:59.850420+0000","last_active":"2024-04-26T23:50:00.849236+0000","last_peered":"2024-04-26T23:50:00.849236+0000","last_clean":"2024-04-26T23:50:00.849236+0000","last_became_active":"2024-04-26T23:49:59.850129+0000","last_became_peered":"2024-04-26T23:49:59.850129+0000","last_unstale":"2024-04-26T23:50:00.849236+0000","last_undegraded":"2024-04-26T23:50:00.849236+0000","last_fullsized":"2024-04-26T23:50:00.849236+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:49:58.780844+0000","last_clean_scrub_stamp":"2024-04-26T23:49:58.780844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:22.109365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":381,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-26T23:50:00.845036+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:50:00.845036+0000","last_peered":"2024-04-26T23:50:00.845036+0000","last_clean":"2024-04-26T23:50:00.845036+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:50:00.845036+0000","last_undegraded":"2024-04-26T23:50:00.845036+0000","last_fullsized":"2024-04-26T23:50:00.845036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29248,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342592,"statfs":{"total":96636764160,"available":96606814208,"internally_reserved":0,"allocated":2281472,"data_stored":1382215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50876,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320964,"statfs":{"total":96636764160,"available":96584667136,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46716,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325124,"statfs":{"total":96636764160,"available":96588926976,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55036,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316804,"statfs":{"total":96636764160,"available":96580407296,"internally_reserved":0,"allocated":1687552,"data_stored":791847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2301952,"data_stored":1389438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:50:05.850 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:50:05.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:50:06.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.112+0000 7f39fb000640 1 Processor -- start 2024-04-26T23:50:06.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.113+0000 7f39fb000640 1 -- start start 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.114+0000 7f39fb000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39f40cdd00 0x7f39f40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.114+0000 7f39fb000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f40ce7d0 con 0x7f39f415cec0 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.114+0000 7f39fb000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f40ceab0 con 0x7f39f4073b30 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.114+0000 7f39fb000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39f40ced90 con 0x7f39f40cdd00 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f9c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f39f415cec0 0x7f39f40cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47726/0 (socket says 172.21.15.46:47726) 2024-04-26T23:50:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f9c00640 1 -- 172.21.15.46:0/3255190478 learned_addr learned my addr 172.21.15.46:0/3255190478 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f9200640 1 --2- 172.21.15.46:0/3255190478 >> v2:172.21.15.142:3300/0 conn(0x7f39f40cdd00 0x7f39f40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379171780 0 0) 0x7f39f40ce7d0 con 0x7f39f415cec0 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39dc003520 con 0x7f39f415cec0 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688295802 0 0) 0x7f39dc003520 con 0x7f39f415cec0 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 >> v2:172.21.15.142:3300/0 conn(0x7f39f40cdd00 msgr2=0x7f39f40735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 --2- 172.21.15.46:0/3255190478 >> v2:172.21.15.142:3300/0 conn(0x7f39f40cdd00 0x7f39f40735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 >> v1:172.21.15.142:6789/0 conn(0x7f39f4073b30 legacy=0x7f39f40b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.115+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f40cf070 con 0x7f39f415cec0 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f39e8001670 con 0x7f39f415cec0 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 >> v1:172.21.15.46:6789/0 conn(0x7f39f415cec0 legacy=0x7f39f40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc004800 0x7f39dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc0051c0 0x7f39dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39f40ced90 con 0x7f39dc003a00 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc00c0c0 con 0x7f39dc004800 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc00c2c0 con 0x7f39dc0051c0 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9200640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc004800 0x7f39dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 -- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc0051c0 msgr2=0x7f39dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39dc0051c0 0x7f39dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 -- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc004800 msgr2=0x7f39dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc004800 0x7f39dc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.116+0000 7f39f9c00640 1 -- 172.21.15.46:0/3255190478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc003520 con 0x7f39dc003a00 2024-04-26T23:50:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.117+0000 7f39fa600640 1 -- 172.21.15.46:0/3255190478 reap_dead start 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.117+0000 7f39f9c00640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39dc003e20 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c0e0 tx=0x7f39dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4610da1ee07db53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8007c20 con 0x7f39dc003a00 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e8005ad0 con 0x7f39dc003a00 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39f3e00640 1 -- 172.21.15.46:0/3255190478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8007070 con 0x7f39dc003a00 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39fb000640 1 -- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 msgr2=0x7f39dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39fb000640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39dc003e20 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c0e0 tx=0x7f39dc005830 comp rx=0 tx=0).stop 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39fb000640 1 -- 172.21.15.46:0/3255190478 shutdown_connections 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39fb000640 1 --2- 172.21.15.46:0/3255190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39dc003e20 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.118+0000 7f39fb000640 1 -- 172.21.15.46:0/3255190478 >> 172.21.15.46:0/3255190478 conn(0x7f39f40b27b0 msgr2=0x7f39f40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.119+0000 7f39fb000640 1 -- 172.21.15.46:0/3255190478 shutdown_connections 2024-04-26T23:50:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.119+0000 7f39fb000640 1 -- 172.21.15.46:0/3255190478 wait complete. 2024-04-26T23:50:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.120+0000 7f39fb000640 1 Processor -- start 2024-04-26T23:50:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.121+0000 7f39fb000640 1 -- start start 2024-04-26T23:50:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f40ab8f0 0x7f39f415f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 0x7f39f41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39f4073f70 con 0x7f39dc003a00 2024-04-26T23:50:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39f415d2a0 con 0x7f39f4151c60 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fb000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39f40735f0 con 0x7f39f40ab8f0 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54108/0 (socket says 172.21.15.46:54108) 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 -- 172.21.15.46:0/1387611521 learned_addr learned my addr 172.21.15.46:0/1387611521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9200640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f40ab8f0 0x7f39f415f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fa600640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 0x7f39f41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 -- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f40ab8f0 msgr2=0x7f39f415f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f40ab8f0 0x7f39f415f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 -- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 msgr2=0x7f39f41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 0x7f39f41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39f9c00640 1 -- 172.21.15.46:0/1387611521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f40aa850 con 0x7f39dc003a00 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.122+0000 7f39fa600640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 0x7f39f41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39f9c00640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c200 tx=0x7f39f40cec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e803f070 con 0x7f39dc003a00 2024-04-26T23:50:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e8005c80 con 0x7f39dc003a00 2024-04-26T23:50:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8001670 con 0x7f39dc003a00 2024-04-26T23:50:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f40aab30 con 0x7f39dc003a00 2024-04-26T23:50:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.123+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f40ab0f0 con 0x7f39dc003a00 2024-04-26T23:50:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.125+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f40693b0 con 0x7f39dc003a00 2024-04-26T23:50:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.126+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39e8042070 con 0x7f39dc003a00 2024-04-26T23:50:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.126+0000 7f39f1600640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 0x7f39d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.127+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f39e80abad0 con 0x7f39dc003a00 2024-04-26T23:50:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.127+0000 7f39f9200640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 0x7f39d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.128+0000 7f39f9200640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 0x7f39d406ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f39f40c7750 tx=0x7f39e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.132+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39e8049030 con 0x7f39dc003a00 2024-04-26T23:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.413+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f39f40693b0 con 0x7f39dc003a00 2024-04-26T23:50:06.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.417+0000 7f39f1600640 1 -- 172.21.15.46:0/1387611521 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f39e8079d70 con 0x7f39dc003a00 2024-04-26T23:50:06.422 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:50:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:50:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:50:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:50:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:50:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:50:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 msgr2=0x7f39d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 0x7f39d406ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f39f40c7750 tx=0x7f39e0008040 comp rx=0 tx=0).stop 2024-04-26T23:50:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 msgr2=0x7f39f40c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c200 tx=0x7f39f40cec80 comp rx=0 tx=0).stop 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 shutdown_connections 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f39d406c920 0x7f39d406ede0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39f4151c60 0x7f39f41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39f40ab8f0 0x7f39f415f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 --2- 172.21.15.46:0/1387611521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39dc003a00 0x7f39f40c6920 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.420+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 >> 172.21.15.46:0/1387611521 conn(0x7f39f40b27b0 msgr2=0x7f39f40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.421+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 shutdown_connections 2024-04-26T23:50:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.421+0000 7f39fb000640 1 -- 172.21.15.46:0/1387611521 wait complete. 2024-04-26T23:50:06.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.635+0000 7f1d2a800640 1 Processor -- start 2024-04-26T23:50:06.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.635+0000 7f1d2a800640 1 -- start start 2024-04-26T23:50:06.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.636+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d2406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.636+0000 7f1d2a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d24160910 0x7f1d24160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.636+0000 7f1d2a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d24073ad0 con 0x7f1d240720e0 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.636+0000 7f1d2a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1d24146a90 con 0x7f1d24160910 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d2a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d24073d70 con 0x7f1d241466b0 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d28a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1d240720e0 0x7f1d240cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47728/0 (socket says 172.21.15.46:47728) 2024-04-26T23:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d28a00640 1 -- 172.21.15.46:0/1766170489 learned_addr learned my addr 172.21.15.46:0/1766170489 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d29400640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d2406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d29e00640 1 --2- 172.21.15.46:0/1766170489 >> v2:172.21.15.142:3300/0 conn(0x7f1d24160910 0x7f1d24160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034537155 0 0) 0x7f1d24073ad0 con 0x7f1d240720e0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.637+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d0c003520 con 0x7f1d240720e0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 131309597 0 0) 0x7f1d0c003520 con 0x7f1d240720e0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 msgr2=0x7f1d2406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d2406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 >> v2:172.21.15.142:3300/0 conn(0x7f1d24160910 msgr2=0x7f1d24160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 --2- 172.21.15.46:0/1766170489 >> v2:172.21.15.142:3300/0 conn(0x7f1d24160910 0x7f1d24160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d24074050 con 0x7f1d240720e0 2024-04-26T23:50:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1d10001670 con 0x7f1d240720e0 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 >> v1:172.21.15.46:6789/0 conn(0x7f1d240720e0 legacy=0x7f1d240cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c003f80 0x7f1d0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c004d40 0x7f1d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24146a90 con 0x7f1d0c003f80 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24073d70 con 0x7f1d0c004d40 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0c00b9c0 con 0x7f1d241466b0 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.638+0000 7f1d28a00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c003f80 0x7f1d0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d29400640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d29e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c004d40 0x7f1d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d28a00640 1 -- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 msgr2=0x7f1d0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d28a00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d241466b0 0x7f1d0c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d28a00640 1 -- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c004d40 msgr2=0x7f1d0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d28a00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c004d40 0x7f1d0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d28a00640 1 -- 172.21.15.46:0/1766170489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c003520 con 0x7f1d0c003f80 2024-04-26T23:50:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.639+0000 7f1d29e00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c004d40 0x7f1d0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.640+0000 7f1d28a00640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c003f80 0x7f1d0c004800 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c1c0 tx=0x7f1d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd490f9f1a493b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.640+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10007c80 con 0x7f1d0c003f80 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.640+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1002e070 con 0x7f1d0c003f80 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.640+0000 7f1d23e00640 1 -- 172.21.15.46:0/1766170489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10005d90 con 0x7f1d0c003f80 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 -- 172.21.15.46:0/1766170489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c003f80 msgr2=0x7f1d0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 --2- 172.21.15.46:0/1766170489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c003f80 0x7f1d0c004800 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c1c0 tx=0x7f1d0c005830 comp rx=0 tx=0).stop 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d29e00640 1 -- 172.21.15.46:0/1766170489 reap_dead start 2024-04-26T23:50:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 -- 172.21.15.46:0/1766170489 shutdown_connections 2024-04-26T23:50:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 -- 172.21.15.46:0/1766170489 >> 172.21.15.46:0/1766170489 conn(0x7f1d240b26d0 msgr2=0x7f1d240b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 -- 172.21.15.46:0/1766170489 shutdown_connections 2024-04-26T23:50:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.641+0000 7f1d2a800640 1 -- 172.21.15.46:0/1766170489 wait complete. 2024-04-26T23:50:06.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.643+0000 7f1d2a800640 1 Processor -- start 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.643+0000 7f1d2a800640 1 -- start start 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c003f80 0x7f1d24141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 0x7f1d240b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2406de80 con 0x7f1d24141cc0 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d240cb470 con 0x7f1d240b1700 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d2a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2406e000 con 0x7f1d0c003f80 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54128/0 (socket says 172.21.15.46:54128) 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.644+0000 7f1d28a00640 1 -- 172.21.15.46:0/2320454532 learned_addr learned my addr 172.21.15.46:0/2320454532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d29400640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c003f80 0x7f1d24141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d29e00640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 0x7f1d240b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 -- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c003f80 msgr2=0x7f1d24141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c003f80 0x7f1d24141780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 -- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 msgr2=0x7f1d240b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 0x7f1d240b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 -- 172.21.15.46:0/2320454532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240b1f30 con 0x7f1d24141cc0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d29e00640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 0x7f1d240b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d28a00640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c160 tx=0x7f1d10001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10044070 con 0x7f1d24141cc0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.645+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d10007480 con 0x7f1d24141cc0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.646+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10007c80 con 0x7f1d24141cc0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.646+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d240b3bd0 con 0x7f1d24141cc0 2024-04-26T23:50:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.646+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d240b4190 con 0x7f1d24141cc0 2024-04-26T23:50:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.647+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d240693b0 con 0x7f1d24141cc0 2024-04-26T23:50:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.648+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d10047070 con 0x7f1d24141cc0 2024-04-26T23:50:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.649+0000 7f1d21600640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 0x7f1d0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.649+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f1d100ab6b0 con 0x7f1d24141cc0 2024-04-26T23:50:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.649+0000 7f1d29400640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 0x7f1d0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.651+0000 7f1d29400640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 0x7f1d0406ede0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d180098b0 tx=0x7f1d18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.653+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d100798c0 con 0x7f1d24141cc0 2024-04-26T23:50:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.895+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d2406e3e0 con 0x7f1d24141cc0 2024-04-26T23:50:06.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.897+0000 7f1d21600640 1 -- 172.21.15.46:0/2320454532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f1d100791c0 con 0x7f1d24141cc0 2024-04-26T23:50:06.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:06.903 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 msgr2=0x7f1d0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 0x7f1d0406ede0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d180098b0 tx=0x7f1d18008040 comp rx=0 tx=0).stop 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 msgr2=0x7f1d240b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c160 tx=0x7f1d10001970 comp rx=0 tx=0).stop 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 shutdown_connections 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c920 0x7f1d0406ede0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d240b1700 0x7f1d240b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d24141cc0 0x7f1d240b11c0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.900+0000 7f1d2a800640 1 --2- 172.21.15.46:0/2320454532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c003f80 0x7f1d24141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.901+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 >> 172.21.15.46:0/2320454532 conn(0x7f1d240b26d0 msgr2=0x7f1d240c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.901+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 shutdown_connections 2024-04-26T23:50:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:06.901+0000 7f1d2a800640 1 -- 172.21.15.46:0/2320454532 wait complete. 2024-04-26T23:50:07.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:50:00.302683+0000', 'modified': '2024-04-26T23:50:03.402945+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9141}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9141': {'gid': 9141, 'name': 'a', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6833/2039062774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 2039062774}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 2039062774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9141, 'qdb_cluster': [9141]} 2024-04-26T23:50:07.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:50:07.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:50:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.544+0000 7f993b200640 1 Processor -- start 2024-04-26T23:50:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.544+0000 7f993b200640 1 -- start start 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f993c15f4e0 0x7f993c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0b4000 0x7f993c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f993c0d1f60 0x7f993c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0d2990 con 0x7f993c0d1f60 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0d11f0 con 0x7f993c15f4e0 2024-04-26T23:50:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.545+0000 7f993b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f993c0d5f20 con 0x7f993c0b4000 2024-04-26T23:50:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.546+0000 7f993a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f993c0d1f60 0x7f993c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.546+0000 7f993a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f993c0d1f60 0x7f993c0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54138/0 (socket says 172.21.15.46:54138) 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.546+0000 7f993a800640 1 -- 172.21.15.46:0/3888556432 learned_addr learned my addr 172.21.15.46:0/3888556432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.546+0000 7f9939400640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0b4000 0x7f993c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.546+0000 7f9939e00640 1 --2- 172.21.15.46:0/3888556432 >> v2:172.21.15.142:3300/0 conn(0x7f993c15f4e0 0x7f993c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.547+0000 7f993a800640 1 -- 172.21.15.46:0/3888556432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0b4000 msgr2=0x7f993c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.547+0000 7f993a800640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0b4000 0x7f993c0d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.547+0000 7f993a800640 1 -- 172.21.15.46:0/3888556432 >> v2:172.21.15.142:3300/0 conn(0x7f993c15f4e0 msgr2=0x7f993c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.547+0000 7f993a800640 1 --2- 172.21.15.46:0/3888556432 >> v2:172.21.15.142:3300/0 conn(0x7f993c15f4e0 0x7f993c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.547+0000 7f993a800640 1 -- 172.21.15.46:0/3888556432 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c0d0f10 con 0x7f993c0d1f60 2024-04-26T23:50:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.548+0000 7f993a800640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0d1f60 0x7f993c0cfcd0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f993c08d8b0 tx=0x7f993402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ceffdc72c5da49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.548+0000 7f9938a00640 1 -- 172.21.15.46:0/3888556432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934036070 con 0x7f993c0d1f60 2024-04-26T23:50:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.548+0000 7f9938a00640 1 -- 172.21.15.46:0/3888556432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934031600 con 0x7f993c0d1f60 2024-04-26T23:50:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.548+0000 7f9938a00640 1 -- 172.21.15.46:0/3888556432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99340319d0 con 0x7f993c0d1f60 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 -- 172.21.15.46:0/3888556432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0d1f60 msgr2=0x7f993c0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0d1f60 0x7f993c0cfcd0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f993c08d8b0 tx=0x7f993402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 -- 172.21.15.46:0/3888556432 shutdown_connections 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0d1f60 0x7f993c0cfcd0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 --2- 172.21.15.46:0/3888556432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0b4000 0x7f993c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 --2- 172.21.15.46:0/3888556432 >> v2:172.21.15.142:3300/0 conn(0x7f993c15f4e0 0x7f993c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 -- 172.21.15.46:0/3888556432 >> 172.21.15.46:0/3888556432 conn(0x7f993c0d5010 msgr2=0x7f993c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 -- 172.21.15.46:0/3888556432 shutdown_connections 2024-04-26T23:50:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.549+0000 7f993b200640 1 -- 172.21.15.46:0/3888556432 wait complete. 2024-04-26T23:50:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.551+0000 7f993b200640 1 Processor -- start 2024-04-26T23:50:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.551+0000 7f993b200640 1 -- start start 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0d1f60 0x7f993c154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c15f4e0 0x7f993c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f993c15f8c0 con 0x7f993c0b4000 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b1d60 con 0x7f993c15f4e0 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f993b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f993c0d0210 con 0x7f993c0d1f60 2024-04-26T23:50:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54152/0 (socket says 172.21.15.46:54152) 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 -- 172.21.15.46:0/3404284849 learned_addr learned my addr 172.21.15.46:0/3404284849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 -- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0d1f60 msgr2=0x7f993c154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0d1f60 0x7f993c154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 -- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c15f4e0 msgr2=0x7f993c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.552+0000 7f9939e00640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c15f4e0 0x7f993c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f9939e00640 1 -- 172.21.15.46:0/3404284849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c0cd360 con 0x7f993c0b4000 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f9939e00640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e870 tx=0x7f992c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f9932000640 1 -- 172.21.15.46:0/3404284849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c017070 con 0x7f993c0b4000 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f9932000640 1 -- 172.21.15.46:0/3404284849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c00c940 con 0x7f993c0b4000 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c0ba710 con 0x7f993c0b4000 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f9932000640 1 -- 172.21.15.46:0/3404284849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c00cd30 con 0x7f993c0b4000 2024-04-26T23:50:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.553+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c14ecb0 con 0x7f993c0b4000 2024-04-26T23:50:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.556+0000 7f9932000640 1 -- 172.21.15.46:0/3404284849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f992c01a070 con 0x7f993c0b4000 2024-04-26T23:50:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.557+0000 7f9932000640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 0x7f991806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.557+0000 7f9932000640 1 -- 172.21.15.46:0/3404284849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f992c08bb80 con 0x7f993c0b4000 2024-04-26T23:50:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.557+0000 7f9944079740 1 Processor -- start 2024-04-26T23:50:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.557+0000 7f9944079740 1 -- start start 2024-04-26T23:50:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.557+0000 7f9939400640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 0x7f991806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f19e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 0x55a46f19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a46f100600 con 0x55a46f1c7270 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a46f09d100 con 0x55a46f19c7b0 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a46f0fc5c0 con 0x55a46f1d8e80 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.558+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54154/0 (socket says 172.21.15.46:54154) 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 -- 172.21.15.46:0/3833541600 learned_addr learned my addr 172.21.15.46:0/3833541600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f9939400640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 0x7f991806edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f993c155290 tx=0x7f9924009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f9939e00640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f9939400640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 0x55a46f19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 -- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 msgr2=0x55a46f19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 0x55a46f19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 -- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 msgr2=0x55a46f19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 -- 172.21.15.46:0/3833541600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46ee122d0 con 0x55a46f1c7270 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f9939400640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 0x55a46f19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.559+0000 7f993a800640 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f993402ac90 tx=0x7f993402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d457526c107946c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f992aa00640 1 -- 172.21.15.46:0/3833541600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934036050 con 0x55a46f1c7270 2024-04-26T23:50:07.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f992aa00640 1 -- 172.21.15.46:0/3833541600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934042070 con 0x55a46f1c7270 2024-04-26T23:50:07.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f992aa00640 1 -- 172.21.15.46:0/3833541600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934009510 con 0x55a46f1c7270 2024-04-26T23:50:07.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 -- 172.21.15.46:0/3833541600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 msgr2=0x55a46f1c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f993402ac90 tx=0x7f993402c000 comp rx=0 tx=0).stop 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 -- 172.21.15.46:0/3833541600 shutdown_connections 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1c7270 0x55a46f1c76f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1d8e80 0x55a46f19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 --2- 172.21.15.46:0/3833541600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.560+0000 7f9944079740 1 -- 172.21.15.46:0/3833541600 >> 172.21.15.46:0/3833541600 conn(0x55a46f130d10 msgr2=0x55a46f1e2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.561+0000 7f9944079740 1 -- 172.21.15.46:0/3833541600 shutdown_connections 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.561+0000 7f9944079740 1 -- 172.21.15.46:0/3833541600 wait complete. 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.561+0000 7f9944079740 1 -- start start 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.562+0000 7f9944079740 1 Processor -- start 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f1c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a46f087f70 con 0x55a46f1d8e80 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a46f1c4dc0 con 0x55a46f19c7b0 2024-04-26T23:50:07.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a46f1ca2b0 con 0x55a46f1c7270 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9944079740 10 client.0 _ll_register_callbacks cb 0x55a46f0f6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54160/0 (socket says 172.21.15.46:54160) 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 -- 172.21.15.46:0/1608034312 learned_addr learned my addr 172.21.15.46:0/1608034312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9939e00640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f1c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9939400640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 -- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 msgr2=0x55a46f1dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 -- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 msgr2=0x55a46f1c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f1c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f993a800640 1 -- 172.21.15.46:0/1608034312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46eedc6b0 con 0x55a46f1d8e80 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.563+0000 7f9939400640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f993a800640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f9934009f10 tx=0x7f9934004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06210d52af9a547 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9923400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934004450 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f99340045f0 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9934042070 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f9934009af0 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9944079740 10 client.9288 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.564+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55a46f1c82d0 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934042070 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f9934008040 con 0x55a46f1d8e80 2024-04-26T23:50:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 10 client.9288 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 10 client.9288 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 20 client.9288 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1c8fa0 0x55a46f1c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 4 client.9288 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46f10e210 con 0x55a46f1c8fa0 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.565+0000 7f9939e00640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1c8fa0 0x55a46f1c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.566+0000 7f9939e00640 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1c8fa0 0x55a46f1c9380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8911b68a2f89ba08 server_cookie=c7d54a6c68c06655 in_seq=0 out_seq=0 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.566+0000 7f9923400640 10 client.9288 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9923400640 1 -- 172.21.15.46:0/1608034312 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a46f10e210 con 0x55a46f1c8fa0 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9923400640 10 client.9288 handle_command_reply: tid=0 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 client.9288 shutdown 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 msgr2=0x55a46f1dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f9934009f10 tx=0x7f9934004ab0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 shutdown_connections 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1c8fa0 0x55a46f1c9380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f1dd090 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 --2- 172.21.15.46:0/1608034312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f19c7b0 0x55a46f1c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9923400640 0 client.9288 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 >> 172.21.15.46:0/1608034312 conn(0x55a46f130d10 msgr2=0x55a46f1c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 shutdown_connections 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 1 -- 172.21.15.46:0/1608034312 wait complete. 2024-04-26T23:50:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.567+0000 7f9944079740 20 client.9288 trim_cache size 0 max 16384 2024-04-26T23:50:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.587+0000 7f9944079740 1 Processor -- start 2024-04-26T23:50:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.587+0000 7f9944079740 1 -- start start 2024-04-26T23:50:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 0x55a46f1c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f993c200e80 con 0x55a46f19c7b0 2024-04-26T23:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f993c14e7d0 con 0x55a46f1d8e80 2024-04-26T23:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f993c151890 con 0x55a46f1c7270 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54164/0 (socket says 172.21.15.46:54164) 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 -- 172.21.15.46:0/4001568137 learned_addr learned my addr 172.21.15.46:0/4001568137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 -- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 msgr2=0x55a46f1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939400640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f993a800640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 0x55a46f1c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.588+0000 7f9939e00640 1 -- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 msgr2=0x55a46f1c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f9939e00640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 0x55a46f1c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f9939e00640 1 -- 172.21.15.46:0/4001568137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46f205f50 con 0x55a46f19c7b0 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f993a800640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 0x55a46f1c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f9939e00640 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f992c059410 tx=0x7f992c059710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9f54959a5846db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f992a000640 1 -- 172.21.15.46:0/4001568137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c076070 con 0x55a46f19c7b0 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f992a000640 1 -- 172.21.15.46:0/4001568137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c0622f0 con 0x55a46f19c7b0 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.589+0000 7f992a000640 1 -- 172.21.15.46:0/4001568137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c021040 con 0x55a46f19c7b0 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- 172.21.15.46:0/4001568137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 msgr2=0x55a46f2560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f992c059410 tx=0x7f992c059710 comp rx=0 tx=0).stop 2024-04-26T23:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- 172.21.15.46:0/4001568137 shutdown_connections 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1d8e80 0x55a46f1c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f1c7270 0x55a46f1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 --2- 172.21.15.46:0/4001568137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f19c7b0 0x55a46f2560f0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- 172.21.15.46:0/4001568137 >> 172.21.15.46:0/4001568137 conn(0x55a46f130d10 msgr2=0x55a46f1e2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- 172.21.15.46:0/4001568137 shutdown_connections 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- 172.21.15.46:0/4001568137 wait complete. 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.590+0000 7f9944079740 1 -- start start 2024-04-26T23:50:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.591+0000 7f9944079740 1 Processor -- start 2024-04-26T23:50:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f19c7b0 0x55a46f19b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 0x55a46f25b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a46f1c9ff0 con 0x55a46f1d8e80 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a46f087f70 con 0x55a46f1c7270 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a46f1f5640 con 0x55a46f19c7b0 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9944079740 10 client.0 _ll_register_callbacks cb 0x7f992c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54170/0 (socket says 172.21.15.46:54170) 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 -- 172.21.15.46:0/759304491 learned_addr learned my addr 172.21.15.46:0/759304491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9939e00640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f19c7b0 0x55a46f19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9939400640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 0x55a46f25b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 -- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f19c7b0 msgr2=0x55a46f19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f19c7b0 0x55a46f19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 -- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 msgr2=0x55a46f25b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 0x55a46f25b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f993a800640 1 -- 172.21.15.46:0/759304491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46f1dab30 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.592+0000 7f9939400640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 0x55a46f25b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f993a800640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f9934009f40 tx=0x7f993403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb88ed53273ad1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9923400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934036020 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f993403d6b0 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992c0622f0 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f9934068050 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9944079740 10 client.9294 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.593+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55a46f19bd10 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c0622f0 con 0x55a46f1d8e80 2024-04-26T23:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f993406b020 con 0x55a46f1d8e80 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 10 client.9294 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 10 client.9294 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 20 client.9294 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1cb010 0x55a46f1e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 4 client.9294 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "status"}) -- 0x55a46f108080 con 0x55a46f1cb010 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.594+0000 7f9939e00640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1cb010 0x55a46f1e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9939e00640 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1cb010 0x55a46f1e5360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e93926aba96297b server_cookie=8ff194d13afe569b in_seq=0 out_seq=0 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9923400640 10 client.9294 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9923400640 1 -- 172.21.15.46:0/759304491 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55a46f108080 con 0x55a46f1cb010 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9923400640 10 client.9294 handle_command_reply: tid=0 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 client.9294 shutdown 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 msgr2=0x55a46f25b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f9934009f40 tx=0x7f993403c000 comp rx=0 tx=0).stop 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 shutdown_connections 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55a46f1cb010 0x55a46f1e5360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a46f1d8e80 0x55a46f25b890 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a46f1c7270 0x55a46f25b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 --2- 172.21.15.46:0/759304491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a46f19c7b0 0x55a46f19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9923400640 0 client.9294 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 >> 172.21.15.46:0/759304491 conn(0x55a46f130d10 msgr2=0x55a46f1dc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 shutdown_connections 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 1 -- 172.21.15.46:0/759304491 wait complete. 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.595+0000 7f9944079740 20 client.9294 trim_cache size 0 max 16384 2024-04-26T23:50:07.604 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9141, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7565563739999996, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 109, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 137, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 137, 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.174258933999999 2024-04-26T23:50:07.605 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 msgr2=0x7f991806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 0x7f991806edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f993c155290 tx=0x7f9924009500 comp rx=0 tx=0).stop 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 msgr2=0x7f993c0b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e870 tx=0x7f992c00ed30 comp rx=0 tx=0).stop 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 shutdown_connections 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f991806c8f0 0x7f991806edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c15f4e0 0x7f993c1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f993c0d1f60 0x7f993c154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 --2- 172.21.15.46:0/3404284849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b4000 0x7f993c0b70a0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 >> 172.21.15.46:0/3404284849 conn(0x7f993c0d5010 msgr2=0x7f993c15e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 shutdown_connections 2024-04-26T23:50:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.596+0000 7f993b200640 1 -- 172.21.15.46:0/3404284849 wait complete. 2024-04-26T23:50:07.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:50:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.794+0000 7f4a3ce00640 1 Processor -- start 2024-04-26T23:50:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.795+0000 7f4a3ce00640 1 -- start start 2024-04-26T23:50:07.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a3ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4a380d1990 0x7f4a380d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a3ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a380d0c10 0x7f4a380d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a3ce00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a380d22b0 con 0x7f4a381639c0 2024-04-26T23:50:07.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a3ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4a380d0780 con 0x7f4a380d0c10 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a3ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4a380cfe50 con 0x7f4a380d1990 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a2fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a380d0c10 0x7f4a380d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a2fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a380d0c10 0x7f4a380d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54182/0 (socket says 172.21.15.46:54182) 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.796+0000 7f4a2fe00640 1 -- 172.21.15.46:0/2930938554 learned_addr learned my addr 172.21.15.46:0/2930938554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.797+0000 7f4a37400640 1 --2- 172.21.15.46:0/2930938554 >> v2:172.21.15.142:3300/0 conn(0x7f4a380d1990 0x7f4a380d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.797+0000 7f4a2fe00640 1 -- 172.21.15.46:0/2930938554 >> v2:172.21.15.142:3300/0 conn(0x7f4a380d1990 msgr2=0x7f4a380d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.797+0000 7f4a2fe00640 1 --2- 172.21.15.46:0/2930938554 >> v2:172.21.15.142:3300/0 conn(0x7f4a380d1990 0x7f4a380d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.797+0000 7f4a2fe00640 1 -- 172.21.15.46:0/2930938554 >> v1:172.21.15.142:6789/0 conn(0x7f4a381639c0 legacy=0x7f4a380d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.797+0000 7f4a2fe00640 1 -- 172.21.15.46:0/2930938554 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a380d0130 con 0x7f4a380d0c10 2024-04-26T23:50:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.798+0000 7f4a2fe00640 1 --2- 172.21.15.46:0/2930938554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a380d0c10 0x7f4a380d5690 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f4a3808d560 tx=0x7f4a2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae270a9c51f1090 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.798+0000 7f4a36a00640 1 -- 172.21.15.46:0/2930938554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20037070 con 0x7f4a380d0c10 2024-04-26T23:50:07.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.798+0000 7f4a36a00640 1 -- 172.21.15.46:0/2930938554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20032440 con 0x7f4a380d0c10 2024-04-26T23:50:07.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.798+0000 7f4a36a00640 1 -- 172.21.15.46:0/2930938554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20032810 con 0x7f4a380d0c10 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2930938554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a380d0c10 msgr2=0x7f4a380d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2930938554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a380d0c10 0x7f4a380d5690 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f4a3808d560 tx=0x7f4a2002a230 comp rx=0 tx=0).stop 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2930938554 shutdown_connections 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2930938554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a380d0c10 0x7f4a380d5690 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2930938554 >> v2:172.21.15.142:3300/0 conn(0x7f4a380d1990 0x7f4a380d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2930938554 >> 172.21.15.46:0/2930938554 conn(0x7f4a3809e2a0 msgr2=0x7f4a380d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2930938554 shutdown_connections 2024-04-26T23:50:07.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.799+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2930938554 wait complete. 2024-04-26T23:50:07.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.801+0000 7f4a3ce00640 1 Processor -- start 2024-04-26T23:50:07.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.801+0000 7f4a3ce00640 1 -- start start 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a380d0c10 0x7f4a3814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a380d1990 0x7f4a3814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a380cfa90 con 0x7f4a381639c0 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3815f530 con 0x7f4a380d1990 2024-04-26T23:50:07.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.802+0000 7f4a3ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a380cfc10 con 0x7f4a380d0c10 2024-04-26T23:50:07.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54190/0 (socket says 172.21.15.46:54190) 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 -- 172.21.15.46:0/2399504287 learned_addr learned my addr 172.21.15.46:0/2399504287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a2fe00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a380d1990 0x7f4a3814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37400640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a380d0c10 0x7f4a3814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 -- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a380d0c10 msgr2=0x7f4a3814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a380d0c10 0x7f4a3814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 -- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a380d1990 msgr2=0x7f4a3814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a380d1990 0x7f4a3814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 -- 172.21.15.46:0/2399504287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3814be10 con 0x7f4a381639c0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.803+0000 7f4a37e00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c700 tx=0x7f4a3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.804+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30016020 con 0x7f4a381639c0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.804+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3000aa50 con 0x7f4a381639c0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.804+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3814c0f0 con 0x7f4a381639c0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.804+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ae40 con 0x7f4a381639c0 2024-04-26T23:50:07.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.804+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3814c6b0 con 0x7f4a381639c0 2024-04-26T23:50:07.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.805+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a380693b0 con 0x7f4a381639c0 2024-04-26T23:50:07.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.806+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a30019020 con 0x7f4a381639c0 2024-04-26T23:50:07.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.807+0000 7f4a2f400640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 0x7f4a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:07.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.807+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f4a3008bf90 con 0x7f4a381639c0 2024-04-26T23:50:07.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.808+0000 7f4a37400640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 0x7f4a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:07.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.809+0000 7f4a37400640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 0x7f4a1406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:07.812+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a30020020 con 0x7f4a381639c0 2024-04-26T23:50:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.062+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4a380ba5b0 con 0x7f4a381639c0 2024-04-26T23:50:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.063+0000 7f4a2f400640 1 -- 172.21.15.46:0/2399504287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) ==== 53+0+73 (secure 0 0 0) 0x7f4a30059bd0 con 0x7f4a381639c0 2024-04-26T23:50:08.068 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 msgr2=0x7f4a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 0x7f4a1406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 msgr2=0x7f4a381e8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c700 tx=0x7f4a3000cbc0 comp rx=0 tx=0).stop 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 shutdown_connections 2024-04-26T23:50:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a1406c8f0 0x7f4a1406edb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a381639c0 0x7f4a381e8dc0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a380d1990 0x7f4a3814afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 --2- 172.21.15.46:0/2399504287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a380d0c10 0x7f4a3814b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 >> 172.21.15.46:0/2399504287 conn(0x7f4a3809e2a0 msgr2=0x7f4a380d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 shutdown_connections 2024-04-26T23:50:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08.066+0000 7f4a3ce00640 1 -- 172.21.15.46:0/2399504287 wait complete. 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:50:08.091 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:50:08.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.092 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:50:08.147 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41813sec preferred_lft 41813sec 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:50:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:50:08.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.150 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:50:08.234 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:50:08.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.238 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:50:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:50:08.298 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:50:08.298 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:50:08.298 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:50:08.299 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:50:08.301 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:08.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:50:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:08.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.529 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:50:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:50:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:50:08.580 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:50:08.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:50:08.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:50:08.666 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:08.720 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:50:08.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:50:08.720 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:08.776 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:50:08.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:50:08.917 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:50:08.918 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:50:08.918 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:50:08.939 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:50:08.940 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:50:08.940 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:50:08.940 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:50:08.940 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:50:08.965 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:50:08.966 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:50:08.968 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:50:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:50:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:50:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:50:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:50:08.971 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:50:08.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:08.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:09.040 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:50:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:09.170 INFO:teuthology.orchestra.run.smithi046.stdout:client6140 2024-04-26T23:50:09.171 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:50:09.171 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:50:09.223 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:09.223 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.0 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:09.223 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:09.224 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.383 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301057 s, 34.8 MB/s 2024-04-26T23:50:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438957 s, 23.9 MB/s 2024-04-26T23:50:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390603 s, 26.8 MB/s 2024-04-26T23:50:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194479 s, 53.9 MB/s 2024-04-26T23:50:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182842 s, 57.3 MB/s 2024-04-26T23:50:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242801 s, 43.2 MB/s 2024-04-26T23:50:09.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240787 s, 43.5 MB/s 2024-04-26T23:50:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274158 s, 38.2 MB/s 2024-04-26T23:50:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222046 s, 47.2 MB/s 2024-04-26T23:50:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264816 s, 39.6 MB/s 2024-04-26T23:50:09.705 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.705 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.705 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235769 s, 44.5 MB/s 2024-04-26T23:50:09.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.775 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636749 s, 16.5 MB/s 2024-04-26T23:50:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175107 s, 59.9 MB/s 2024-04-26T23:50:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254244 s, 41.2 MB/s 2024-04-26T23:50:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228082 s, 46.0 MB/s 2024-04-26T23:50:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213335 s, 49.2 MB/s 2024-04-26T23:50:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212632 s, 49.3 MB/s 2024-04-26T23:50:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236456 s, 44.3 MB/s 2024-04-26T23:50:09.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218061 s, 48.1 MB/s 2024-04-26T23:50:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022312 s, 47.0 MB/s 2024-04-26T23:50:10.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.026 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267717 s, 39.2 MB/s 2024-04-26T23:50:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382675 s, 27.4 MB/s 2024-04-26T23:50:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271853 s, 38.6 MB/s 2024-04-26T23:50:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177505 s, 59.1 MB/s 2024-04-26T23:50:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323627 s, 32.4 MB/s 2024-04-26T23:50:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205801 s, 51.0 MB/s 2024-04-26T23:50:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211067 s, 49.7 MB/s 2024-04-26T23:50:10.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195205 s, 53.7 MB/s 2024-04-26T23:50:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.268 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234984 s, 44.6 MB/s 2024-04-26T23:50:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255615 s, 41.0 MB/s 2024-04-26T23:50:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207659 s, 50.5 MB/s 2024-04-26T23:50:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168174 s, 62.4 MB/s 2024-04-26T23:50:10.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:10.357 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:10.357 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:10.357 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.1 2024-04-26T23:50:10.357 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:10.357 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:10.358 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:10.358 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230594 s, 45.5 MB/s 2024-04-26T23:50:10.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.517 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207451 s, 50.5 MB/s 2024-04-26T23:50:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236925 s, 44.3 MB/s 2024-04-26T23:50:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186417 s, 56.2 MB/s 2024-04-26T23:50:10.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261983 s, 40.0 MB/s 2024-04-26T23:50:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233591 s, 44.9 MB/s 2024-04-26T23:50:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228194 s, 46.0 MB/s 2024-04-26T23:50:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243361 s, 43.1 MB/s 2024-04-26T23:50:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249588 s, 42.0 MB/s 2024-04-26T23:50:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197486 s, 53.1 MB/s 2024-04-26T23:50:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179467 s, 58.4 MB/s 2024-04-26T23:50:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278089 s, 37.7 MB/s 2024-04-26T23:50:10.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232808 s, 45.0 MB/s 2024-04-26T23:50:10.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.852 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184434 s, 56.9 MB/s 2024-04-26T23:50:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212588 s, 49.3 MB/s 2024-04-26T23:50:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213732 s, 49.1 MB/s 2024-04-26T23:50:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196995 s, 53.2 MB/s 2024-04-26T23:50:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194273 s, 54.0 MB/s 2024-04-26T23:50:10.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:10.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:10.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017806 s, 58.9 MB/s 2024-04-26T23:50:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235868 s, 44.5 MB/s 2024-04-26T23:50:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182328 s, 57.5 MB/s 2024-04-26T23:50:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200132 s, 52.4 MB/s 2024-04-26T23:50:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230906 s, 45.4 MB/s 2024-04-26T23:50:11.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.122 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273197 s, 38.4 MB/s 2024-04-26T23:50:11.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.155 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256444 s, 40.9 MB/s 2024-04-26T23:50:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314347 s, 33.4 MB/s 2024-04-26T23:50:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239009 s, 43.9 MB/s 2024-04-26T23:50:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019912 s, 52.7 MB/s 2024-04-26T23:50:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244938 s, 42.8 MB/s 2024-04-26T23:50:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205721 s, 51.0 MB/s 2024-04-26T23:50:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288254 s, 36.4 MB/s 2024-04-26T23:50:11.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022247 s, 47.1 MB/s 2024-04-26T23:50:11.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.2 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:11.385 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262505 s, 39.9 MB/s 2024-04-26T23:50:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257207 s, 40.8 MB/s 2024-04-26T23:50:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223028 s, 47.0 MB/s 2024-04-26T23:50:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239618 s, 43.8 MB/s 2024-04-26T23:50:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258761 s, 40.5 MB/s 2024-04-26T23:50:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292777 s, 35.8 MB/s 2024-04-26T23:50:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238972 s, 43.9 MB/s 2024-04-26T23:50:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239175 s, 43.8 MB/s 2024-04-26T23:50:11.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.785 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242705 s, 43.2 MB/s 2024-04-26T23:50:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256373 s, 40.9 MB/s 2024-04-26T23:50:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244993 s, 42.8 MB/s 2024-04-26T23:50:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214529 s, 48.9 MB/s 2024-04-26T23:50:11.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209781 s, 50.0 MB/s 2024-04-26T23:50:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292812 s, 35.8 MB/s 2024-04-26T23:50:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236674 s, 44.3 MB/s 2024-04-26T23:50:12.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300903 s, 34.8 MB/s 2024-04-26T23:50:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234675 s, 44.7 MB/s 2024-04-26T23:50:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218135 s, 48.1 MB/s 2024-04-26T23:50:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242912 s, 43.2 MB/s 2024-04-26T23:50:12.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.124 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022034 s, 47.6 MB/s 2024-04-26T23:50:12.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:12.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:12.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.150 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199136 s, 52.7 MB/s 2024-04-26T23:50:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182868 s, 57.3 MB/s 2024-04-26T23:50:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181558 s, 57.8 MB/s 2024-04-26T23:50:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178471 s, 58.8 MB/s 2024-04-26T23:50:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177719 s, 59.0 MB/s 2024-04-26T23:50:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201247 s, 52.1 MB/s 2024-04-26T23:50:12.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.290 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216577 s, 48.4 MB/s 2024-04-26T23:50:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204596 s, 51.3 MB/s 2024-04-26T23:50:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224303 s, 46.7 MB/s 2024-04-26T23:50:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243245 s, 43.1 MB/s 2024-04-26T23:50:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025074 s, 41.8 MB/s 2024-04-26T23:50:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238748 s, 43.9 MB/s 2024-04-26T23:50:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.3 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:12.451 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290427 s, 36.1 MB/s 2024-04-26T23:50:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252211 s, 41.6 MB/s 2024-04-26T23:50:12.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318409 s, 32.9 MB/s 2024-04-26T23:50:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197839 s, 53.0 MB/s 2024-04-26T23:50:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259457 s, 40.4 MB/s 2024-04-26T23:50:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261483 s, 40.1 MB/s 2024-04-26T23:50:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219408 s, 47.8 MB/s 2024-04-26T23:50:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214416 s, 48.9 MB/s 2024-04-26T23:50:12.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.890 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212104 s, 49.4 MB/s 2024-04-26T23:50:12.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.918 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226157 s, 46.4 MB/s 2024-04-26T23:50:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017181 s, 61.0 MB/s 2024-04-26T23:50:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214185 s, 49.0 MB/s 2024-04-26T23:50:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246197 s, 42.6 MB/s 2024-04-26T23:50:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256264 s, 40.9 MB/s 2024-04-26T23:50:13.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230913 s, 45.4 MB/s 2024-04-26T23:50:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246225 s, 42.6 MB/s 2024-04-26T23:50:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229677 s, 45.7 MB/s 2024-04-26T23:50:13.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.147 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022649 s, 46.3 MB/s 2024-04-26T23:50:13.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280199 s, 37.4 MB/s 2024-04-26T23:50:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214509 s, 48.9 MB/s 2024-04-26T23:50:13.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.237 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230879 s, 45.4 MB/s 2024-04-26T23:50:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01947 s, 53.9 MB/s 2024-04-26T23:50:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229685 s, 45.7 MB/s 2024-04-26T23:50:13.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.317 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021078 s, 49.7 MB/s 2024-04-26T23:50:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255358 s, 41.1 MB/s 2024-04-26T23:50:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216692 s, 48.4 MB/s 2024-04-26T23:50:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282247 s, 37.2 MB/s 2024-04-26T23:50:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234574 s, 44.7 MB/s 2024-04-26T23:50:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243069 s, 43.1 MB/s 2024-04-26T23:50:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271658 s, 38.6 MB/s 2024-04-26T23:50:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263556 s, 39.8 MB/s 2024-04-26T23:50:13.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.570 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313031 s, 33.5 MB/s 2024-04-26T23:50:13.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.4 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:13.582 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280017 s, 37.4 MB/s 2024-04-26T23:50:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221706 s, 47.3 MB/s 2024-04-26T23:50:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241658 s, 43.4 MB/s 2024-04-26T23:50:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232378 s, 45.1 MB/s 2024-04-26T23:50:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030269 s, 34.6 MB/s 2024-04-26T23:50:13.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.882 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233237 s, 45.0 MB/s 2024-04-26T23:50:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021313 s, 49.2 MB/s 2024-04-26T23:50:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234105 s, 44.8 MB/s 2024-04-26T23:50:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168633 s, 62.2 MB/s 2024-04-26T23:50:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211689 s, 49.5 MB/s 2024-04-26T23:50:14.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200574 s, 52.3 MB/s 2024-04-26T23:50:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293525 s, 35.7 MB/s 2024-04-26T23:50:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240905 s, 43.5 MB/s 2024-04-26T23:50:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237995 s, 44.1 MB/s 2024-04-26T23:50:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262854 s, 39.9 MB/s 2024-04-26T23:50:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267673 s, 39.2 MB/s 2024-04-26T23:50:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254861 s, 41.1 MB/s 2024-04-26T23:50:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220385 s, 47.6 MB/s 2024-04-26T23:50:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292704 s, 35.8 MB/s 2024-04-26T23:50:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256415 s, 40.9 MB/s 2024-04-26T23:50:14.337 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.337 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.337 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275681 s, 38.0 MB/s 2024-04-26T23:50:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288431 s, 36.4 MB/s 2024-04-26T23:50:14.404 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.404 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.404 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025458 s, 41.2 MB/s 2024-04-26T23:50:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205683 s, 51.0 MB/s 2024-04-26T23:50:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211817 s, 49.5 MB/s 2024-04-26T23:50:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216927 s, 48.3 MB/s 2024-04-26T23:50:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236704 s, 44.3 MB/s 2024-04-26T23:50:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255009 s, 41.1 MB/s 2024-04-26T23:50:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311009 s, 33.7 MB/s 2024-04-26T23:50:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232093 s, 45.2 MB/s 2024-04-26T23:50:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262921 s, 39.9 MB/s 2024-04-26T23:50:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023321 s, 45.0 MB/s 2024-04-26T23:50:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:14.687 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.5 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:14.688 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234129 s, 44.8 MB/s 2024-04-26T23:50:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228918 s, 45.8 MB/s 2024-04-26T23:50:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234936 s, 44.6 MB/s 2024-04-26T23:50:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275789 s, 38.0 MB/s 2024-04-26T23:50:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227922 s, 46.0 MB/s 2024-04-26T23:50:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260334 s, 40.3 MB/s 2024-04-26T23:50:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209974 s, 49.9 MB/s 2024-04-26T23:50:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216322 s, 48.5 MB/s 2024-04-26T23:50:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243819 s, 43.0 MB/s 2024-04-26T23:50:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289374 s, 36.2 MB/s 2024-04-26T23:50:15.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.135 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203362 s, 51.6 MB/s 2024-04-26T23:50:15.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221662 s, 47.3 MB/s 2024-04-26T23:50:15.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242255 s, 43.3 MB/s 2024-04-26T23:50:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225671 s, 46.5 MB/s 2024-04-26T23:50:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237633 s, 44.1 MB/s 2024-04-26T23:50:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235364 s, 44.6 MB/s 2024-04-26T23:50:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317342 s, 33.0 MB/s 2024-04-26T23:50:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244821 s, 42.8 MB/s 2024-04-26T23:50:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205855 s, 50.9 MB/s 2024-04-26T23:50:15.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.406 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239 s, 43.9 MB/s 2024-04-26T23:50:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226012 s, 46.4 MB/s 2024-04-26T23:50:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296948 s, 35.3 MB/s 2024-04-26T23:50:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233794 s, 44.9 MB/s 2024-04-26T23:50:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275791 s, 38.0 MB/s 2024-04-26T23:50:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309986 s, 33.8 MB/s 2024-04-26T23:50:15.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.596 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211199 s, 49.6 MB/s 2024-04-26T23:50:15.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210051 s, 49.9 MB/s 2024-04-26T23:50:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024264 s, 43.2 MB/s 2024-04-26T23:50:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025036 s, 41.9 MB/s 2024-04-26T23:50:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231199 s, 45.4 MB/s 2024-04-26T23:50:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236285 s, 44.4 MB/s 2024-04-26T23:50:15.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215776 s, 48.6 MB/s 2024-04-26T23:50:15.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.6 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:15.785 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:15.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.931 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225221 s, 46.6 MB/s 2024-04-26T23:50:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311708 s, 33.6 MB/s 2024-04-26T23:50:15.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:15.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:15.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207542 s, 50.5 MB/s 2024-04-26T23:50:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263414 s, 39.8 MB/s 2024-04-26T23:50:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245255 s, 42.8 MB/s 2024-04-26T23:50:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225684 s, 46.5 MB/s 2024-04-26T23:50:16.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.122 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299154 s, 35.1 MB/s 2024-04-26T23:50:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285685 s, 36.7 MB/s 2024-04-26T23:50:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216608 s, 48.4 MB/s 2024-04-26T23:50:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235524 s, 44.5 MB/s 2024-04-26T23:50:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237346 s, 44.2 MB/s 2024-04-26T23:50:16.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.275 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226633 s, 46.3 MB/s 2024-04-26T23:50:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288758 s, 36.3 MB/s 2024-04-26T23:50:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205368 s, 51.1 MB/s 2024-04-26T23:50:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209698 s, 50.0 MB/s 2024-04-26T23:50:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225551 s, 46.5 MB/s 2024-04-26T23:50:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221524 s, 47.3 MB/s 2024-04-26T23:50:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016034 s, 65.4 MB/s 2024-04-26T23:50:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226001 s, 46.4 MB/s 2024-04-26T23:50:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281375 s, 37.3 MB/s 2024-04-26T23:50:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341209 s, 30.7 MB/s 2024-04-26T23:50:16.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.579 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251312 s, 41.7 MB/s 2024-04-26T23:50:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231544 s, 45.3 MB/s 2024-04-26T23:50:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219138 s, 47.8 MB/s 2024-04-26T23:50:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208012 s, 50.4 MB/s 2024-04-26T23:50:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278072 s, 37.7 MB/s 2024-04-26T23:50:16.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.732 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241726 s, 43.4 MB/s 2024-04-26T23:50:16.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231874 s, 45.2 MB/s 2024-04-26T23:50:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229373 s, 45.7 MB/s 2024-04-26T23:50:16.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028637 s, 36.6 MB/s 2024-04-26T23:50:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028864 s, 36.3 MB/s 2024-04-26T23:50:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183495 s, 57.1 MB/s 2024-04-26T23:50:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:16.900 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:16.900 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:16.901 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.7 2024-04-26T23:50:16.901 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:16.901 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:16.901 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:16.901 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:17.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301735 s, 34.8 MB/s 2024-04-26T23:50:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297428 s, 35.3 MB/s 2024-04-26T23:50:17.119 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.119 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.119 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252713 s, 41.5 MB/s 2024-04-26T23:50:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300512 s, 34.9 MB/s 2024-04-26T23:50:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245688 s, 42.7 MB/s 2024-04-26T23:50:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216478 s, 48.4 MB/s 2024-04-26T23:50:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230512 s, 45.5 MB/s 2024-04-26T23:50:17.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203013 s, 51.7 MB/s 2024-04-26T23:50:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.301 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023976 s, 43.7 MB/s 2024-04-26T23:50:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025539 s, 41.1 MB/s 2024-04-26T23:50:17.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.360 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235836 s, 44.5 MB/s 2024-04-26T23:50:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209543 s, 50.0 MB/s 2024-04-26T23:50:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227039 s, 46.2 MB/s 2024-04-26T23:50:17.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.443 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205376 s, 51.1 MB/s 2024-04-26T23:50:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276599 s, 37.9 MB/s 2024-04-26T23:50:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259418 s, 40.4 MB/s 2024-04-26T23:50:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247057 s, 42.4 MB/s 2024-04-26T23:50:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298791 s, 35.1 MB/s 2024-04-26T23:50:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274703 s, 38.2 MB/s 2024-04-26T23:50:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259636 s, 40.4 MB/s 2024-04-26T23:50:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218195 s, 48.1 MB/s 2024-04-26T23:50:17.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.710 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347352 s, 30.2 MB/s 2024-04-26T23:50:17.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.738 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242618 s, 43.2 MB/s 2024-04-26T23:50:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240777 s, 43.5 MB/s 2024-04-26T23:50:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242576 s, 43.2 MB/s 2024-04-26T23:50:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218365 s, 48.0 MB/s 2024-04-26T23:50:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235362 s, 44.6 MB/s 2024-04-26T23:50:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301757 s, 34.7 MB/s 2024-04-26T23:50:17.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.922 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233323 s, 44.9 MB/s 2024-04-26T23:50:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225357 s, 46.5 MB/s 2024-04-26T23:50:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:17.975 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190561 s, 55.0 MB/s 2024-04-26T23:50:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202328 s, 51.8 MB/s 2024-04-26T23:50:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.8 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:18.014 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:18.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238807 s, 43.9 MB/s 2024-04-26T23:50:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247885 s, 42.3 MB/s 2024-04-26T23:50:18.227 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295635 s, 35.5 MB/s 2024-04-26T23:50:18.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.252 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193741 s, 54.1 MB/s 2024-04-26T23:50:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238504 s, 44.0 MB/s 2024-04-26T23:50:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250334 s, 41.9 MB/s 2024-04-26T23:50:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235393 s, 44.5 MB/s 2024-04-26T23:50:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189223 s, 55.4 MB/s 2024-04-26T23:50:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254377 s, 41.2 MB/s 2024-04-26T23:50:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247773 s, 42.3 MB/s 2024-04-26T23:50:18.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.464 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242825 s, 43.2 MB/s 2024-04-26T23:50:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219454 s, 47.8 MB/s 2024-04-26T23:50:18.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205053 s, 51.1 MB/s 2024-04-26T23:50:18.550 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.551 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256634 s, 40.9 MB/s 2024-04-26T23:50:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269243 s, 38.9 MB/s 2024-04-26T23:50:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231793 s, 45.2 MB/s 2024-04-26T23:50:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258898 s, 40.5 MB/s 2024-04-26T23:50:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178105 s, 58.9 MB/s 2024-04-26T23:50:18.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.700 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276887 s, 37.9 MB/s 2024-04-26T23:50:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280421 s, 37.4 MB/s 2024-04-26T23:50:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225748 s, 46.4 MB/s 2024-04-26T23:50:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197002 s, 53.2 MB/s 2024-04-26T23:50:18.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227541 s, 46.1 MB/s 2024-04-26T23:50:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231049 s, 45.4 MB/s 2024-04-26T23:50:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023807 s, 44.0 MB/s 2024-04-26T23:50:18.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264693 s, 39.6 MB/s 2024-04-26T23:50:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227346 s, 46.1 MB/s 2024-04-26T23:50:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024091 s, 43.5 MB/s 2024-04-26T23:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226662 s, 46.3 MB/s 2024-04-26T23:50:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239393 s, 43.8 MB/s 2024-04-26T23:50:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229335 s, 45.7 MB/s 2024-04-26T23:50:19.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.089 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285544 s, 36.7 MB/s 2024-04-26T23:50:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:19.100 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:19.101 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:19.101 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.9 2024-04-26T23:50:19.101 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:19.101 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:19.102 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:19.102 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300593 s, 34.9 MB/s 2024-04-26T23:50:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227495 s, 46.1 MB/s 2024-04-26T23:50:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236963 s, 44.3 MB/s 2024-04-26T23:50:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023774 s, 44.1 MB/s 2024-04-26T23:50:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027153 s, 38.6 MB/s 2024-04-26T23:50:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322697 s, 32.5 MB/s 2024-04-26T23:50:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293318 s, 35.7 MB/s 2024-04-26T23:50:19.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238597 s, 43.9 MB/s 2024-04-26T23:50:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250365 s, 41.9 MB/s 2024-04-26T23:50:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020262 s, 51.8 MB/s 2024-04-26T23:50:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295811 s, 35.4 MB/s 2024-04-26T23:50:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280893 s, 37.3 MB/s 2024-04-26T23:50:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229185 s, 45.8 MB/s 2024-04-26T23:50:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172525 s, 60.8 MB/s 2024-04-26T23:50:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213994 s, 49.0 MB/s 2024-04-26T23:50:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212247 s, 49.4 MB/s 2024-04-26T23:50:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025468 s, 41.2 MB/s 2024-04-26T23:50:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244118 s, 43.0 MB/s 2024-04-26T23:50:19.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217815 s, 48.1 MB/s 2024-04-26T23:50:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272179 s, 38.5 MB/s 2024-04-26T23:50:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220947 s, 47.5 MB/s 2024-04-26T23:50:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275438 s, 38.1 MB/s 2024-04-26T23:50:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236589 s, 44.3 MB/s 2024-04-26T23:50:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224454 s, 46.7 MB/s 2024-04-26T23:50:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256926 s, 40.8 MB/s 2024-04-26T23:50:20.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.010 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.010 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236546 s, 44.3 MB/s 2024-04-26T23:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231542 s, 45.3 MB/s 2024-04-26T23:50:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025435 s, 41.2 MB/s 2024-04-26T23:50:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303785 s, 34.5 MB/s 2024-04-26T23:50:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231679 s, 45.3 MB/s 2024-04-26T23:50:20.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210821 s, 49.7 MB/s 2024-04-26T23:50:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236484 s, 44.3 MB/s 2024-04-26T23:50:20.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.10 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:20.204 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287372 s, 36.5 MB/s 2024-04-26T23:50:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205588 s, 51.0 MB/s 2024-04-26T23:50:20.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279694 s, 37.5 MB/s 2024-04-26T23:50:20.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.437 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254879 s, 41.1 MB/s 2024-04-26T23:50:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227903 s, 46.0 MB/s 2024-04-26T23:50:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219547 s, 47.8 MB/s 2024-04-26T23:50:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154482 s, 67.9 MB/s 2024-04-26T23:50:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209056 s, 50.2 MB/s 2024-04-26T23:50:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280666 s, 37.4 MB/s 2024-04-26T23:50:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.604 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258571 s, 40.6 MB/s 2024-04-26T23:50:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241347 s, 43.4 MB/s 2024-04-26T23:50:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206789 s, 50.7 MB/s 2024-04-26T23:50:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026532 s, 39.5 MB/s 2024-04-26T23:50:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261105 s, 40.2 MB/s 2024-04-26T23:50:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296693 s, 35.3 MB/s 2024-04-26T23:50:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354141 s, 29.6 MB/s 2024-04-26T23:50:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233078 s, 45.0 MB/s 2024-04-26T23:50:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219496 s, 47.8 MB/s 2024-04-26T23:50:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209902 s, 50.0 MB/s 2024-04-26T23:50:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257113 s, 40.8 MB/s 2024-04-26T23:50:20.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.948 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222327 s, 47.2 MB/s 2024-04-26T23:50:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244756 s, 42.8 MB/s 2024-04-26T23:50:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265973 s, 39.4 MB/s 2024-04-26T23:50:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206014 s, 50.9 MB/s 2024-04-26T23:50:21.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190274 s, 55.1 MB/s 2024-04-26T23:50:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323953 s, 32.4 MB/s 2024-04-26T23:50:21.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223856 s, 46.8 MB/s 2024-04-26T23:50:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289037 s, 36.3 MB/s 2024-04-26T23:50:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206814 s, 50.7 MB/s 2024-04-26T23:50:21.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.221 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237086 s, 44.2 MB/s 2024-04-26T23:50:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243224 s, 43.1 MB/s 2024-04-26T23:50:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223068 s, 47.0 MB/s 2024-04-26T23:50:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:21.293 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:21.293 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:21.293 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.11 2024-04-26T23:50:21.293 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:21.293 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:21.294 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:21.294 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:21.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247344 s, 42.4 MB/s 2024-04-26T23:50:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294903 s, 35.6 MB/s 2024-04-26T23:50:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213104 s, 49.2 MB/s 2024-04-26T23:50:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214405 s, 48.9 MB/s 2024-04-26T23:50:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.551 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254743 s, 41.2 MB/s 2024-04-26T23:50:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255536 s, 41.0 MB/s 2024-04-26T23:50:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290686 s, 36.1 MB/s 2024-04-26T23:50:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236837 s, 44.3 MB/s 2024-04-26T23:50:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204557 s, 51.3 MB/s 2024-04-26T23:50:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233889 s, 44.8 MB/s 2024-04-26T23:50:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205582 s, 51.0 MB/s 2024-04-26T23:50:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021961 s, 47.7 MB/s 2024-04-26T23:50:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253055 s, 41.4 MB/s 2024-04-26T23:50:21.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247667 s, 42.3 MB/s 2024-04-26T23:50:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238281 s, 44.0 MB/s 2024-04-26T23:50:21.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.877 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220459 s, 47.6 MB/s 2024-04-26T23:50:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224936 s, 46.6 MB/s 2024-04-26T23:50:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302495 s, 34.7 MB/s 2024-04-26T23:50:21.972 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:21.972 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:21.972 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221686 s, 47.3 MB/s 2024-04-26T23:50:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249529 s, 42.0 MB/s 2024-04-26T23:50:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184283 s, 56.9 MB/s 2024-04-26T23:50:22.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239086 s, 43.9 MB/s 2024-04-26T23:50:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030453 s, 34.4 MB/s 2024-04-26T23:50:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225904 s, 46.4 MB/s 2024-04-26T23:50:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236344 s, 44.4 MB/s 2024-04-26T23:50:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275718 s, 38.0 MB/s 2024-04-26T23:50:22.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222015 s, 47.2 MB/s 2024-04-26T23:50:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193447 s, 54.2 MB/s 2024-04-26T23:50:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221001 s, 47.4 MB/s 2024-04-26T23:50:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216743 s, 48.4 MB/s 2024-04-26T23:50:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024242 s, 43.3 MB/s 2024-04-26T23:50:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019114 s, 54.9 MB/s 2024-04-26T23:50:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:22.362 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.12 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:22.363 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234342 s, 44.7 MB/s 2024-04-26T23:50:22.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251655 s, 41.7 MB/s 2024-04-26T23:50:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220263 s, 47.6 MB/s 2024-04-26T23:50:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222232 s, 47.2 MB/s 2024-04-26T23:50:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236634 s, 44.3 MB/s 2024-04-26T23:50:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228555 s, 45.9 MB/s 2024-04-26T23:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321602 s, 32.6 MB/s 2024-04-26T23:50:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252585 s, 41.5 MB/s 2024-04-26T23:50:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264438 s, 39.7 MB/s 2024-04-26T23:50:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220651 s, 47.5 MB/s 2024-04-26T23:50:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283979 s, 36.9 MB/s 2024-04-26T23:50:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028536 s, 36.7 MB/s 2024-04-26T23:50:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264487 s, 39.6 MB/s 2024-04-26T23:50:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239563 s, 43.8 MB/s 2024-04-26T23:50:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222167 s, 47.2 MB/s 2024-04-26T23:50:22.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:22.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:22.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240846 s, 43.5 MB/s 2024-04-26T23:50:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294024 s, 35.7 MB/s 2024-04-26T23:50:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231752 s, 45.2 MB/s 2024-04-26T23:50:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253058 s, 41.4 MB/s 2024-04-26T23:50:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237655 s, 44.1 MB/s 2024-04-26T23:50:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241851 s, 43.4 MB/s 2024-04-26T23:50:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209915 s, 50.0 MB/s 2024-04-26T23:50:23.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237785 s, 44.1 MB/s 2024-04-26T23:50:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229921 s, 45.6 MB/s 2024-04-26T23:50:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237181 s, 44.2 MB/s 2024-04-26T23:50:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230743 s, 45.4 MB/s 2024-04-26T23:50:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231647 s, 45.3 MB/s 2024-04-26T23:50:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261399 s, 40.1 MB/s 2024-04-26T23:50:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263528 s, 39.8 MB/s 2024-04-26T23:50:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209717 s, 50.0 MB/s 2024-04-26T23:50:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299776 s, 35.0 MB/s 2024-04-26T23:50:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191422 s, 54.8 MB/s 2024-04-26T23:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:23.477 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.13 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:23.478 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:23.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280208 s, 37.4 MB/s 2024-04-26T23:50:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025634 s, 40.9 MB/s 2024-04-26T23:50:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022285 s, 47.1 MB/s 2024-04-26T23:50:23.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247006 s, 42.5 MB/s 2024-04-26T23:50:23.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.745 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228266 s, 45.9 MB/s 2024-04-26T23:50:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236365 s, 44.4 MB/s 2024-04-26T23:50:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248473 s, 42.2 MB/s 2024-04-26T23:50:23.839 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.839 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.839 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227999 s, 46.0 MB/s 2024-04-26T23:50:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026417 s, 39.7 MB/s 2024-04-26T23:50:23.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279258 s, 37.5 MB/s 2024-04-26T23:50:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216775 s, 48.4 MB/s 2024-04-26T23:50:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218798 s, 47.9 MB/s 2024-04-26T23:50:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244752 s, 42.8 MB/s 2024-04-26T23:50:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221641 s, 47.3 MB/s 2024-04-26T23:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217783 s, 48.1 MB/s 2024-04-26T23:50:24.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283566 s, 37.0 MB/s 2024-04-26T23:50:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259851 s, 40.4 MB/s 2024-04-26T23:50:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173342 s, 60.5 MB/s 2024-04-26T23:50:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279532 s, 37.5 MB/s 2024-04-26T23:50:24.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229827 s, 45.6 MB/s 2024-04-26T23:50:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292094 s, 35.9 MB/s 2024-04-26T23:50:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307814 s, 34.1 MB/s 2024-04-26T23:50:24.305 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.305 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.305 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225744 s, 46.4 MB/s 2024-04-26T23:50:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243948 s, 43.0 MB/s 2024-04-26T23:50:24.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.365 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226727 s, 46.2 MB/s 2024-04-26T23:50:24.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.398 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282427 s, 37.1 MB/s 2024-04-26T23:50:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267131 s, 39.3 MB/s 2024-04-26T23:50:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225022 s, 46.6 MB/s 2024-04-26T23:50:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255519 s, 41.0 MB/s 2024-04-26T23:50:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221824 s, 47.3 MB/s 2024-04-26T23:50:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227546 s, 46.1 MB/s 2024-04-26T23:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225787 s, 46.4 MB/s 2024-04-26T23:50:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:24.590 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:24.590 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:24.590 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.14 2024-04-26T23:50:24.590 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:24.590 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:24.591 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:24.591 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024675 s, 42.5 MB/s 2024-04-26T23:50:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284635 s, 36.8 MB/s 2024-04-26T23:50:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242255 s, 43.3 MB/s 2024-04-26T23:50:24.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228536 s, 45.9 MB/s 2024-04-26T23:50:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250335 s, 41.9 MB/s 2024-04-26T23:50:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275394 s, 38.1 MB/s 2024-04-26T23:50:24.926 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245676 s, 42.7 MB/s 2024-04-26T23:50:24.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219551 s, 47.8 MB/s 2024-04-26T23:50:24.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:24.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:24.986 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259655 s, 40.4 MB/s 2024-04-26T23:50:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200531 s, 52.3 MB/s 2024-04-26T23:50:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280118 s, 37.4 MB/s 2024-04-26T23:50:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219676 s, 47.7 MB/s 2024-04-26T23:50:25.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.112 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281615 s, 37.2 MB/s 2024-04-26T23:50:25.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224766 s, 46.7 MB/s 2024-04-26T23:50:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265077 s, 39.6 MB/s 2024-04-26T23:50:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024009 s, 43.7 MB/s 2024-04-26T23:50:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232155 s, 45.2 MB/s 2024-04-26T23:50:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215389 s, 48.7 MB/s 2024-04-26T23:50:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231438 s, 45.3 MB/s 2024-04-26T23:50:25.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.338 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289762 s, 36.2 MB/s 2024-04-26T23:50:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245469 s, 42.7 MB/s 2024-04-26T23:50:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278118 s, 37.7 MB/s 2024-04-26T23:50:25.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265163 s, 39.5 MB/s 2024-04-26T23:50:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340125 s, 30.8 MB/s 2024-04-26T23:50:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232577 s, 45.1 MB/s 2024-04-26T23:50:25.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.544 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.544 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246067 s, 42.6 MB/s 2024-04-26T23:50:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03115 s, 33.7 MB/s 2024-04-26T23:50:25.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231222 s, 45.3 MB/s 2024-04-26T23:50:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258094 s, 40.6 MB/s 2024-04-26T23:50:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257851 s, 40.7 MB/s 2024-04-26T23:50:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242212 s, 43.3 MB/s 2024-04-26T23:50:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225268 s, 46.5 MB/s 2024-04-26T23:50:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:25.749 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:25.749 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:25.749 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.15 2024-04-26T23:50:25.750 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:25.750 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:25.750 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:25.750 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204792 s, 51.2 MB/s 2024-04-26T23:50:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232045 s, 45.2 MB/s 2024-04-26T23:50:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171667 s, 61.1 MB/s 2024-04-26T23:50:25.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:25.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:25.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261302 s, 40.1 MB/s 2024-04-26T23:50:26.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216735 s, 48.4 MB/s 2024-04-26T23:50:26.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293391 s, 35.7 MB/s 2024-04-26T23:50:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225371 s, 46.5 MB/s 2024-04-26T23:50:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185826 s, 56.4 MB/s 2024-04-26T23:50:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023082 s, 45.4 MB/s 2024-04-26T23:50:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232076 s, 45.2 MB/s 2024-04-26T23:50:26.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283739 s, 37.0 MB/s 2024-04-26T23:50:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212184 s, 49.4 MB/s 2024-04-26T23:50:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02167 s, 48.4 MB/s 2024-04-26T23:50:26.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.272 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248059 s, 42.3 MB/s 2024-04-26T23:50:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210335 s, 49.9 MB/s 2024-04-26T23:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250634 s, 41.8 MB/s 2024-04-26T23:50:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271077 s, 38.7 MB/s 2024-04-26T23:50:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218492 s, 48.0 MB/s 2024-04-26T23:50:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241169 s, 43.5 MB/s 2024-04-26T23:50:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238969 s, 43.9 MB/s 2024-04-26T23:50:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200622 s, 52.3 MB/s 2024-04-26T23:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309126 s, 33.9 MB/s 2024-04-26T23:50:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246994 s, 42.5 MB/s 2024-04-26T23:50:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026221 s, 40.0 MB/s 2024-04-26T23:50:26.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.604 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230239 s, 45.5 MB/s 2024-04-26T23:50:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224142 s, 46.8 MB/s 2024-04-26T23:50:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240756 s, 43.6 MB/s 2024-04-26T23:50:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027672 s, 37.9 MB/s 2024-04-26T23:50:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275309 s, 38.1 MB/s 2024-04-26T23:50:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224344 s, 46.7 MB/s 2024-04-26T23:50:26.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.786 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236243 s, 44.4 MB/s 2024-04-26T23:50:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254052 s, 41.3 MB/s 2024-04-26T23:50:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.16 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:26.831 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:26.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:26.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:26.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025 s, 41.9 MB/s 2024-04-26T23:50:27.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.003 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205707 s, 51.0 MB/s 2024-04-26T23:50:27.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.034 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237126 s, 44.2 MB/s 2024-04-26T23:50:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223186 s, 47.0 MB/s 2024-04-26T23:50:27.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.097 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276153 s, 38.0 MB/s 2024-04-26T23:50:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236311 s, 44.4 MB/s 2024-04-26T23:50:27.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.153 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217444 s, 48.2 MB/s 2024-04-26T23:50:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282744 s, 37.1 MB/s 2024-04-26T23:50:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222831 s, 47.1 MB/s 2024-04-26T23:50:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244652 s, 42.9 MB/s 2024-04-26T23:50:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233103 s, 45.0 MB/s 2024-04-26T23:50:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226891 s, 46.2 MB/s 2024-04-26T23:50:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229923 s, 45.6 MB/s 2024-04-26T23:50:27.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.363 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244513 s, 42.9 MB/s 2024-04-26T23:50:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025652 s, 40.9 MB/s 2024-04-26T23:50:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228742 s, 45.8 MB/s 2024-04-26T23:50:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224677 s, 46.7 MB/s 2024-04-26T23:50:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243731 s, 43.0 MB/s 2024-04-26T23:50:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223393 s, 46.9 MB/s 2024-04-26T23:50:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290777 s, 36.1 MB/s 2024-04-26T23:50:27.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.587 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292292 s, 35.9 MB/s 2024-04-26T23:50:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242598 s, 43.2 MB/s 2024-04-26T23:50:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285384 s, 36.7 MB/s 2024-04-26T23:50:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267667 s, 39.2 MB/s 2024-04-26T23:50:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275699 s, 38.0 MB/s 2024-04-26T23:50:27.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.750 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227569 s, 46.1 MB/s 2024-04-26T23:50:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227791 s, 46.0 MB/s 2024-04-26T23:50:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217571 s, 48.2 MB/s 2024-04-26T23:50:27.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.843 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288206 s, 36.4 MB/s 2024-04-26T23:50:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253229 s, 41.4 MB/s 2024-04-26T23:50:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260801 s, 40.2 MB/s 2024-04-26T23:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233776 s, 44.9 MB/s 2024-04-26T23:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:27.948 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.17 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:27.949 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268506 s, 39.1 MB/s 2024-04-26T23:50:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254037 s, 41.3 MB/s 2024-04-26T23:50:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253077 s, 41.4 MB/s 2024-04-26T23:50:28.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206898 s, 50.7 MB/s 2024-04-26T23:50:28.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.210 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244525 s, 42.9 MB/s 2024-04-26T23:50:28.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339151 s, 30.9 MB/s 2024-04-26T23:50:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024847 s, 42.2 MB/s 2024-04-26T23:50:28.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273498 s, 38.3 MB/s 2024-04-26T23:50:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291297 s, 36.0 MB/s 2024-04-26T23:50:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.383 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269776 s, 38.9 MB/s 2024-04-26T23:50:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249323 s, 42.1 MB/s 2024-04-26T23:50:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212941 s, 49.2 MB/s 2024-04-26T23:50:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222681 s, 47.1 MB/s 2024-04-26T23:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242554 s, 43.2 MB/s 2024-04-26T23:50:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211651 s, 49.5 MB/s 2024-04-26T23:50:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227557 s, 46.1 MB/s 2024-04-26T23:50:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340409 s, 30.8 MB/s 2024-04-26T23:50:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234744 s, 44.7 MB/s 2024-04-26T23:50:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206753 s, 50.7 MB/s 2024-04-26T23:50:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221854 s, 47.3 MB/s 2024-04-26T23:50:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219235 s, 47.8 MB/s 2024-04-26T23:50:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232246 s, 45.1 MB/s 2024-04-26T23:50:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202107 s, 51.9 MB/s 2024-04-26T23:50:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027438 s, 38.2 MB/s 2024-04-26T23:50:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280572 s, 37.4 MB/s 2024-04-26T23:50:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191049 s, 54.9 MB/s 2024-04-26T23:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264793 s, 39.6 MB/s 2024-04-26T23:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238576 s, 44.0 MB/s 2024-04-26T23:50:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245372 s, 42.7 MB/s 2024-04-26T23:50:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236137 s, 44.4 MB/s 2024-04-26T23:50:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250958 s, 41.8 MB/s 2024-04-26T23:50:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024959 s, 42.0 MB/s 2024-04-26T23:50:29.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:29.049 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:29.049 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:29.049 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.18 2024-04-26T23:50:29.049 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:29.050 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:29.050 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:29.050 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211431 s, 49.6 MB/s 2024-04-26T23:50:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277537 s, 37.8 MB/s 2024-04-26T23:50:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287984 s, 36.4 MB/s 2024-04-26T23:50:29.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.299 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339132 s, 30.9 MB/s 2024-04-26T23:50:29.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299963 s, 35.0 MB/s 2024-04-26T23:50:29.367 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.367 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.367 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265246 s, 39.5 MB/s 2024-04-26T23:50:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.394 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218768 s, 47.9 MB/s 2024-04-26T23:50:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283511 s, 37.0 MB/s 2024-04-26T23:50:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236225 s, 44.4 MB/s 2024-04-26T23:50:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202394 s, 51.8 MB/s 2024-04-26T23:50:29.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.517 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259913 s, 40.3 MB/s 2024-04-26T23:50:29.545 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227965 s, 46.0 MB/s 2024-04-26T23:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246298 s, 42.6 MB/s 2024-04-26T23:50:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168694 s, 62.2 MB/s 2024-04-26T23:50:29.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.629 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230043 s, 45.6 MB/s 2024-04-26T23:50:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285762 s, 36.7 MB/s 2024-04-26T23:50:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240925 s, 43.5 MB/s 2024-04-26T23:50:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255533 s, 41.0 MB/s 2024-04-26T23:50:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205707 s, 51.0 MB/s 2024-04-26T23:50:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258225 s, 40.6 MB/s 2024-04-26T23:50:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.809 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227085 s, 46.2 MB/s 2024-04-26T23:50:29.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253113 s, 41.4 MB/s 2024-04-26T23:50:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245464 s, 42.7 MB/s 2024-04-26T23:50:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257569 s, 40.7 MB/s 2024-04-26T23:50:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216501 s, 48.4 MB/s 2024-04-26T23:50:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282049 s, 37.2 MB/s 2024-04-26T23:50:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261677 s, 40.1 MB/s 2024-04-26T23:50:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225359 s, 46.5 MB/s 2024-04-26T23:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263286 s, 39.8 MB/s 2024-04-26T23:50:30.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253369 s, 41.4 MB/s 2024-04-26T23:50:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246119 s, 42.6 MB/s 2024-04-26T23:50:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026054 s, 40.2 MB/s 2024-04-26T23:50:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:30.160 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:30.160 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:30.160 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.19 2024-04-26T23:50:30.160 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:30.160 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:30.161 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:30.161 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268124 s, 39.1 MB/s 2024-04-26T23:50:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238079 s, 44.0 MB/s 2024-04-26T23:50:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244949 s, 42.8 MB/s 2024-04-26T23:50:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023365 s, 44.9 MB/s 2024-04-26T23:50:30.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249977 s, 41.9 MB/s 2024-04-26T23:50:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374817 s, 28.0 MB/s 2024-04-26T23:50:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022786 s, 46.0 MB/s 2024-04-26T23:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229111 s, 45.8 MB/s 2024-04-26T23:50:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020149 s, 52.0 MB/s 2024-04-26T23:50:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019126 s, 54.8 MB/s 2024-04-26T23:50:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244192 s, 42.9 MB/s 2024-04-26T23:50:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237949 s, 44.1 MB/s 2024-04-26T23:50:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255727 s, 41.0 MB/s 2024-04-26T23:50:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264789 s, 39.6 MB/s 2024-04-26T23:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024864 s, 42.2 MB/s 2024-04-26T23:50:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221106 s, 47.4 MB/s 2024-04-26T23:50:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282275 s, 37.1 MB/s 2024-04-26T23:50:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270217 s, 38.8 MB/s 2024-04-26T23:50:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244051 s, 43.0 MB/s 2024-04-26T23:50:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215399 s, 48.7 MB/s 2024-04-26T23:50:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278481 s, 37.7 MB/s 2024-04-26T23:50:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215872 s, 48.6 MB/s 2024-04-26T23:50:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299964 s, 35.0 MB/s 2024-04-26T23:50:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227661 s, 46.1 MB/s 2024-04-26T23:50:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264305 s, 39.7 MB/s 2024-04-26T23:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241173 s, 43.5 MB/s 2024-04-26T23:50:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241468 s, 43.4 MB/s 2024-04-26T23:50:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276766 s, 37.9 MB/s 2024-04-26T23:50:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244443 s, 42.9 MB/s 2024-04-26T23:50:31.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.201 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211193 s, 49.7 MB/s 2024-04-26T23:50:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205152 s, 51.1 MB/s 2024-04-26T23:50:31.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288526 s, 36.3 MB/s 2024-04-26T23:50:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.20 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:31.275 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263019 s, 39.9 MB/s 2024-04-26T23:50:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206067 s, 50.9 MB/s 2024-04-26T23:50:31.477 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.477 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.477 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199205 s, 52.6 MB/s 2024-04-26T23:50:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023794 s, 44.1 MB/s 2024-04-26T23:50:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206424 s, 50.8 MB/s 2024-04-26T23:50:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308795 s, 34.0 MB/s 2024-04-26T23:50:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269843 s, 38.9 MB/s 2024-04-26T23:50:31.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02439 s, 43.0 MB/s 2024-04-26T23:50:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260547 s, 40.2 MB/s 2024-04-26T23:50:31.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280982 s, 37.3 MB/s 2024-04-26T23:50:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292349 s, 35.9 MB/s 2024-04-26T23:50:31.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220557 s, 47.5 MB/s 2024-04-26T23:50:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245979 s, 42.6 MB/s 2024-04-26T23:50:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244162 s, 42.9 MB/s 2024-04-26T23:50:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293463 s, 35.7 MB/s 2024-04-26T23:50:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229381 s, 45.7 MB/s 2024-04-26T23:50:31.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.922 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247129 s, 42.4 MB/s 2024-04-26T23:50:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204211 s, 51.3 MB/s 2024-04-26T23:50:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228953 s, 45.8 MB/s 2024-04-26T23:50:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294694 s, 35.6 MB/s 2024-04-26T23:50:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028207 s, 37.2 MB/s 2024-04-26T23:50:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.077 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.077 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239477 s, 43.8 MB/s 2024-04-26T23:50:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020296 s, 51.7 MB/s 2024-04-26T23:50:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239503 s, 43.8 MB/s 2024-04-26T23:50:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210459 s, 49.8 MB/s 2024-04-26T23:50:32.190 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.190 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024801 s, 42.3 MB/s 2024-04-26T23:50:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247637 s, 42.3 MB/s 2024-04-26T23:50:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237412 s, 44.2 MB/s 2024-04-26T23:50:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204081 s, 51.4 MB/s 2024-04-26T23:50:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243362 s, 43.1 MB/s 2024-04-26T23:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025418 s, 41.3 MB/s 2024-04-26T23:50:32.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308709 s, 34.0 MB/s 2024-04-26T23:50:32.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:32.387 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:32.387 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:32.387 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.21 2024-04-26T23:50:32.387 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:32.388 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:32.388 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:32.388 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242756 s, 43.2 MB/s 2024-04-26T23:50:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221781 s, 47.3 MB/s 2024-04-26T23:50:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275966 s, 38.0 MB/s 2024-04-26T23:50:32.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.621 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223098 s, 47.0 MB/s 2024-04-26T23:50:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230774 s, 45.4 MB/s 2024-04-26T23:50:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218905 s, 47.9 MB/s 2024-04-26T23:50:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247448 s, 42.4 MB/s 2024-04-26T23:50:32.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272637 s, 38.5 MB/s 2024-04-26T23:50:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243025 s, 43.1 MB/s 2024-04-26T23:50:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276556 s, 37.9 MB/s 2024-04-26T23:50:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296824 s, 35.3 MB/s 2024-04-26T23:50:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288544 s, 36.3 MB/s 2024-04-26T23:50:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215543 s, 48.6 MB/s 2024-04-26T23:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247087 s, 42.4 MB/s 2024-04-26T23:50:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289855 s, 36.2 MB/s 2024-04-26T23:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261827 s, 40.0 MB/s 2024-04-26T23:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219764 s, 47.7 MB/s 2024-04-26T23:50:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220782 s, 47.5 MB/s 2024-04-26T23:50:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241313 s, 43.5 MB/s 2024-04-26T23:50:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235627 s, 44.5 MB/s 2024-04-26T23:50:33.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199384 s, 52.6 MB/s 2024-04-26T23:50:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246891 s, 42.5 MB/s 2024-04-26T23:50:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329518 s, 31.8 MB/s 2024-04-26T23:50:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272541 s, 38.5 MB/s 2024-04-26T23:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243572 s, 43.0 MB/s 2024-04-26T23:50:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233596 s, 44.9 MB/s 2024-04-26T23:50:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258344 s, 40.6 MB/s 2024-04-26T23:50:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241938 s, 43.3 MB/s 2024-04-26T23:50:33.391 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.392 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206181 s, 50.9 MB/s 2024-04-26T23:50:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280627 s, 37.4 MB/s 2024-04-26T23:50:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281044 s, 37.3 MB/s 2024-04-26T23:50:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197746 s, 53.0 MB/s 2024-04-26T23:50:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:33.497 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:33.497 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:33.498 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.22 2024-04-26T23:50:33.498 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:33.498 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:33.498 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:33.498 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357945 s, 29.3 MB/s 2024-04-26T23:50:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229522 s, 45.7 MB/s 2024-04-26T23:50:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244895 s, 42.8 MB/s 2024-04-26T23:50:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274506 s, 38.2 MB/s 2024-04-26T23:50:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221378 s, 47.4 MB/s 2024-04-26T23:50:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030698 s, 34.2 MB/s 2024-04-26T23:50:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283535 s, 37.0 MB/s 2024-04-26T23:50:33.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249161 s, 42.1 MB/s 2024-04-26T23:50:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232417 s, 45.1 MB/s 2024-04-26T23:50:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202535 s, 51.8 MB/s 2024-04-26T23:50:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022999 s, 45.6 MB/s 2024-04-26T23:50:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210882 s, 49.7 MB/s 2024-04-26T23:50:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241605 s, 43.4 MB/s 2024-04-26T23:50:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282563 s, 37.1 MB/s 2024-04-26T23:50:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021164 s, 49.5 MB/s 2024-04-26T23:50:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.109 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248296 s, 42.2 MB/s 2024-04-26T23:50:34.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260762 s, 40.2 MB/s 2024-04-26T23:50:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239851 s, 43.7 MB/s 2024-04-26T23:50:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225679 s, 46.5 MB/s 2024-04-26T23:50:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280827 s, 37.3 MB/s 2024-04-26T23:50:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223466 s, 46.9 MB/s 2024-04-26T23:50:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240428 s, 43.6 MB/s 2024-04-26T23:50:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251324 s, 41.7 MB/s 2024-04-26T23:50:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241831 s, 43.4 MB/s 2024-04-26T23:50:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02076 s, 50.5 MB/s 2024-04-26T23:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307382 s, 34.1 MB/s 2024-04-26T23:50:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.445 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213412 s, 49.1 MB/s 2024-04-26T23:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224816 s, 46.6 MB/s 2024-04-26T23:50:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227449 s, 46.1 MB/s 2024-04-26T23:50:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229154 s, 45.8 MB/s 2024-04-26T23:50:34.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225094 s, 46.6 MB/s 2024-04-26T23:50:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262404 s, 40.0 MB/s 2024-04-26T23:50:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:34.600 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.23 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:34.601 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:34.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:34.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.750 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265485 s, 39.5 MB/s 2024-04-26T23:50:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302309 s, 34.7 MB/s 2024-04-26T23:50:34.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.819 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.819 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276634 s, 37.9 MB/s 2024-04-26T23:50:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222213 s, 47.2 MB/s 2024-04-26T23:50:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283429 s, 37.0 MB/s 2024-04-26T23:50:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255178 s, 41.1 MB/s 2024-04-26T23:50:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.944 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.944 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222948 s, 47.0 MB/s 2024-04-26T23:50:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250352 s, 41.9 MB/s 2024-04-26T23:50:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022671 s, 46.3 MB/s 2024-04-26T23:50:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230784 s, 45.4 MB/s 2024-04-26T23:50:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246814 s, 42.5 MB/s 2024-04-26T23:50:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233929 s, 44.8 MB/s 2024-04-26T23:50:35.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208444 s, 50.3 MB/s 2024-04-26T23:50:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269028 s, 39.0 MB/s 2024-04-26T23:50:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339793 s, 30.9 MB/s 2024-04-26T23:50:35.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307451 s, 34.1 MB/s 2024-04-26T23:50:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259266 s, 40.4 MB/s 2024-04-26T23:50:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305376 s, 34.3 MB/s 2024-04-26T23:50:35.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254724 s, 41.2 MB/s 2024-04-26T23:50:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304114 s, 34.5 MB/s 2024-04-26T23:50:35.396 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.396 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.396 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206886 s, 50.7 MB/s 2024-04-26T23:50:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271097 s, 38.7 MB/s 2024-04-26T23:50:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219506 s, 47.8 MB/s 2024-04-26T23:50:35.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223513 s, 46.9 MB/s 2024-04-26T23:50:35.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.510 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177663 s, 59.0 MB/s 2024-04-26T23:50:35.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.544 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259938 s, 40.3 MB/s 2024-04-26T23:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264803 s, 39.6 MB/s 2024-04-26T23:50:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234902 s, 44.6 MB/s 2024-04-26T23:50:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241578 s, 43.4 MB/s 2024-04-26T23:50:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268195 s, 39.1 MB/s 2024-04-26T23:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280557 s, 37.4 MB/s 2024-04-26T23:50:35.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.735 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266423 s, 39.4 MB/s 2024-04-26T23:50:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.24 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:35.747 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:35.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.895 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230037 s, 45.6 MB/s 2024-04-26T23:50:35.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238971 s, 43.9 MB/s 2024-04-26T23:50:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231948 s, 45.2 MB/s 2024-04-26T23:50:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:35.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274173 s, 38.2 MB/s 2024-04-26T23:50:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.019 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251851 s, 41.6 MB/s 2024-04-26T23:50:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335552 s, 31.2 MB/s 2024-04-26T23:50:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248517 s, 42.2 MB/s 2024-04-26T23:50:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284421 s, 36.9 MB/s 2024-04-26T23:50:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268297 s, 39.1 MB/s 2024-04-26T23:50:36.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.188 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238885 s, 43.9 MB/s 2024-04-26T23:50:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021851 s, 48.0 MB/s 2024-04-26T23:50:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255029 s, 41.1 MB/s 2024-04-26T23:50:36.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232234 s, 45.2 MB/s 2024-04-26T23:50:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244787 s, 42.8 MB/s 2024-04-26T23:50:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256186 s, 40.9 MB/s 2024-04-26T23:50:36.368 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220546 s, 47.5 MB/s 2024-04-26T23:50:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284426 s, 36.9 MB/s 2024-04-26T23:50:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279606 s, 37.5 MB/s 2024-04-26T23:50:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235421 s, 44.5 MB/s 2024-04-26T23:50:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022088 s, 47.5 MB/s 2024-04-26T23:50:36.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.521 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202666 s, 51.7 MB/s 2024-04-26T23:50:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02145 s, 48.9 MB/s 2024-04-26T23:50:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220203 s, 47.6 MB/s 2024-04-26T23:50:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341158 s, 30.7 MB/s 2024-04-26T23:50:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285742 s, 36.7 MB/s 2024-04-26T23:50:36.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.682 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243405 s, 43.1 MB/s 2024-04-26T23:50:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252685 s, 41.5 MB/s 2024-04-26T23:50:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236212 s, 44.4 MB/s 2024-04-26T23:50:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229805 s, 45.6 MB/s 2024-04-26T23:50:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237541 s, 44.1 MB/s 2024-04-26T23:50:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211574 s, 49.6 MB/s 2024-04-26T23:50:36.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:36.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:36.858 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258442 s, 40.6 MB/s 2024-04-26T23:50:36.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:36.870 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:36.870 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:36.870 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.25 2024-04-26T23:50:36.870 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:36.870 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:36.871 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:36.871 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.022 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283838 s, 36.9 MB/s 2024-04-26T23:50:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248273 s, 42.2 MB/s 2024-04-26T23:50:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280348 s, 37.4 MB/s 2024-04-26T23:50:37.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233646 s, 44.9 MB/s 2024-04-26T23:50:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230072 s, 45.6 MB/s 2024-04-26T23:50:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239924 s, 43.7 MB/s 2024-04-26T23:50:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208126 s, 50.4 MB/s 2024-04-26T23:50:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279931 s, 37.5 MB/s 2024-04-26T23:50:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199016 s, 52.7 MB/s 2024-04-26T23:50:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206859 s, 50.7 MB/s 2024-04-26T23:50:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021887 s, 47.9 MB/s 2024-04-26T23:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226358 s, 46.3 MB/s 2024-04-26T23:50:37.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026145 s, 40.1 MB/s 2024-04-26T23:50:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229319 s, 45.7 MB/s 2024-04-26T23:50:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025113 s, 41.8 MB/s 2024-04-26T23:50:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226252 s, 46.3 MB/s 2024-04-26T23:50:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268811 s, 39.0 MB/s 2024-04-26T23:50:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228314 s, 45.9 MB/s 2024-04-26T23:50:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248816 s, 42.1 MB/s 2024-04-26T23:50:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210574 s, 49.8 MB/s 2024-04-26T23:50:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270006 s, 38.8 MB/s 2024-04-26T23:50:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285746 s, 36.7 MB/s 2024-04-26T23:50:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236179 s, 44.4 MB/s 2024-04-26T23:50:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249283 s, 42.1 MB/s 2024-04-26T23:50:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233608 s, 44.9 MB/s 2024-04-26T23:50:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281895 s, 37.2 MB/s 2024-04-26T23:50:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308537 s, 34.0 MB/s 2024-04-26T23:50:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262483 s, 39.9 MB/s 2024-04-26T23:50:37.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.886 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338374 s, 31.0 MB/s 2024-04-26T23:50:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232433 s, 45.1 MB/s 2024-04-26T23:50:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208058 s, 50.4 MB/s 2024-04-26T23:50:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029081 s, 36.1 MB/s 2024-04-26T23:50:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:37.989 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.26 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:37.990 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240527 s, 43.6 MB/s 2024-04-26T23:50:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247944 s, 42.3 MB/s 2024-04-26T23:50:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024028 s, 43.6 MB/s 2024-04-26T23:50:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232099 s, 45.2 MB/s 2024-04-26T23:50:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222672 s, 47.1 MB/s 2024-04-26T23:50:38.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.295 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033312 s, 31.5 MB/s 2024-04-26T23:50:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230465 s, 45.5 MB/s 2024-04-26T23:50:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250645 s, 41.8 MB/s 2024-04-26T23:50:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238277 s, 44.0 MB/s 2024-04-26T23:50:38.410 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020626 s, 50.8 MB/s 2024-04-26T23:50:38.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.448 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030395 s, 34.5 MB/s 2024-04-26T23:50:38.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.484 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303446 s, 34.6 MB/s 2024-04-26T23:50:38.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.515 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252606 s, 41.5 MB/s 2024-04-26T23:50:38.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.540 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192777 s, 54.4 MB/s 2024-04-26T23:50:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.566 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206055 s, 50.9 MB/s 2024-04-26T23:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281177 s, 37.3 MB/s 2024-04-26T23:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209233 s, 50.1 MB/s 2024-04-26T23:50:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307841 s, 34.1 MB/s 2024-04-26T23:50:38.694 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263771 s, 39.8 MB/s 2024-04-26T23:50:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228722 s, 45.8 MB/s 2024-04-26T23:50:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250963 s, 41.8 MB/s 2024-04-26T23:50:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283253 s, 37.0 MB/s 2024-04-26T23:50:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240993 s, 43.5 MB/s 2024-04-26T23:50:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238337 s, 44.0 MB/s 2024-04-26T23:50:38.881 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.881 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.881 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248036 s, 42.3 MB/s 2024-04-26T23:50:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211769 s, 49.5 MB/s 2024-04-26T23:50:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215791 s, 48.6 MB/s 2024-04-26T23:50:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319379 s, 32.8 MB/s 2024-04-26T23:50:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232657 s, 45.1 MB/s 2024-04-26T23:50:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285751 s, 36.7 MB/s 2024-04-26T23:50:39.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.064 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198884 s, 52.7 MB/s 2024-04-26T23:50:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289816 s, 36.2 MB/s 2024-04-26T23:50:39.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:39.111 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:39.111 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:39.111 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.27 2024-04-26T23:50:39.112 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:39.112 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:39.112 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:39.112 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233731 s, 44.9 MB/s 2024-04-26T23:50:39.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.287 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280154 s, 37.4 MB/s 2024-04-26T23:50:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240677 s, 43.6 MB/s 2024-04-26T23:50:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254604 s, 41.2 MB/s 2024-04-26T23:50:39.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237418 s, 44.2 MB/s 2024-04-26T23:50:39.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256896 s, 40.8 MB/s 2024-04-26T23:50:39.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276086 s, 38.0 MB/s 2024-04-26T23:50:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241548 s, 43.4 MB/s 2024-04-26T23:50:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02558 s, 41.0 MB/s 2024-04-26T23:50:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244156 s, 42.9 MB/s 2024-04-26T23:50:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214514 s, 48.9 MB/s 2024-04-26T23:50:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253588 s, 41.3 MB/s 2024-04-26T23:50:39.622 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222125 s, 47.2 MB/s 2024-04-26T23:50:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285548 s, 36.7 MB/s 2024-04-26T23:50:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309193 s, 33.9 MB/s 2024-04-26T23:50:39.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.725 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230739 s, 45.4 MB/s 2024-04-26T23:50:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227907 s, 46.0 MB/s 2024-04-26T23:50:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199707 s, 52.5 MB/s 2024-04-26T23:50:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023536 s, 44.6 MB/s 2024-04-26T23:50:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222646 s, 47.1 MB/s 2024-04-26T23:50:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336149 s, 31.2 MB/s 2024-04-26T23:50:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244355 s, 42.9 MB/s 2024-04-26T23:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183429 s, 57.2 MB/s 2024-04-26T23:50:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212455 s, 49.4 MB/s 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250301 s, 41.9 MB/s 2024-04-26T23:50:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022374 s, 46.9 MB/s 2024-04-26T23:50:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224494 s, 46.7 MB/s 2024-04-26T23:50:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215245 s, 48.7 MB/s 2024-04-26T23:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240576 s, 43.6 MB/s 2024-04-26T23:50:40.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.139 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241761 s, 43.4 MB/s 2024-04-26T23:50:40.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.171 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255277 s, 41.1 MB/s 2024-04-26T23:50:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299665 s, 35.0 MB/s 2024-04-26T23:50:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:40.219 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:40.219 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:40.220 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.28 2024-04-26T23:50:40.220 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:40.220 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:40.220 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:40.220 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:40.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214222 s, 48.9 MB/s 2024-04-26T23:50:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317757 s, 33.0 MB/s 2024-04-26T23:50:40.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.439 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272833 s, 38.4 MB/s 2024-04-26T23:50:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216544 s, 48.4 MB/s 2024-04-26T23:50:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180456 s, 58.1 MB/s 2024-04-26T23:50:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214301 s, 48.9 MB/s 2024-04-26T23:50:40.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205781 s, 51.0 MB/s 2024-04-26T23:50:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239887 s, 43.7 MB/s 2024-04-26T23:50:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025619 s, 40.9 MB/s 2024-04-26T23:50:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243213 s, 43.1 MB/s 2024-04-26T23:50:40.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.674 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028115 s, 37.3 MB/s 2024-04-26T23:50:40.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.700 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201451 s, 52.1 MB/s 2024-04-26T23:50:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318306 s, 32.9 MB/s 2024-04-26T23:50:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252327 s, 41.6 MB/s 2024-04-26T23:50:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225823 s, 46.4 MB/s 2024-04-26T23:50:40.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.833 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263362 s, 39.8 MB/s 2024-04-26T23:50:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291262 s, 36.0 MB/s 2024-04-26T23:50:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216414 s, 48.5 MB/s 2024-04-26T23:50:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230618 s, 45.5 MB/s 2024-04-26T23:50:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288639 s, 36.3 MB/s 2024-04-26T23:50:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265142 s, 39.5 MB/s 2024-04-26T23:50:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274957 s, 38.1 MB/s 2024-04-26T23:50:41.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.064 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267348 s, 39.2 MB/s 2024-04-26T23:50:41.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.096 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253133 s, 41.4 MB/s 2024-04-26T23:50:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.130 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263726 s, 39.8 MB/s 2024-04-26T23:50:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201951 s, 51.9 MB/s 2024-04-26T23:50:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227982 s, 46.0 MB/s 2024-04-26T23:50:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254316 s, 41.2 MB/s 2024-04-26T23:50:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250115 s, 41.9 MB/s 2024-04-26T23:50:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02538 s, 41.3 MB/s 2024-04-26T23:50:41.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025689 s, 40.8 MB/s 2024-04-26T23:50:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308291 s, 34.0 MB/s 2024-04-26T23:50:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:41.363 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.29 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:41.364 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:41.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:41.511 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022077 s, 47.5 MB/s 2024-04-26T23:50:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251544 s, 41.7 MB/s 2024-04-26T23:50:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246768 s, 42.5 MB/s 2024-04-26T23:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228581 s, 45.9 MB/s 2024-04-26T23:50:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323984 s, 32.4 MB/s 2024-04-26T23:50:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268953 s, 39.0 MB/s 2024-04-26T23:50:41.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233962 s, 44.8 MB/s 2024-04-26T23:50:41.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262535 s, 39.9 MB/s 2024-04-26T23:50:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331176 s, 31.7 MB/s 2024-04-26T23:50:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251602 s, 41.7 MB/s 2024-04-26T23:50:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263146 s, 39.8 MB/s 2024-04-26T23:50:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.870 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257541 s, 40.7 MB/s 2024-04-26T23:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223035 s, 47.0 MB/s 2024-04-26T23:50:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288055 s, 36.4 MB/s 2024-04-26T23:50:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240001 s, 43.7 MB/s 2024-04-26T23:50:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234471 s, 44.7 MB/s 2024-04-26T23:50:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184603 s, 56.8 MB/s 2024-04-26T23:50:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218642 s, 48.0 MB/s 2024-04-26T23:50:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289597 s, 36.2 MB/s 2024-04-26T23:50:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021825 s, 48.0 MB/s 2024-04-26T23:50:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239576 s, 43.8 MB/s 2024-04-26T23:50:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285866 s, 36.7 MB/s 2024-04-26T23:50:42.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257721 s, 40.7 MB/s 2024-04-26T23:50:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.237 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260352 s, 40.3 MB/s 2024-04-26T23:50:42.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:42.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312194 s, 33.6 MB/s 2024-04-26T23:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205123 s, 51.1 MB/s 2024-04-26T23:50:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184854 s, 56.7 MB/s 2024-04-26T23:50:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193744 s, 54.1 MB/s 2024-04-26T23:50:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199512 s, 52.6 MB/s 2024-04-26T23:50:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208211 s, 50.4 MB/s 2024-04-26T23:50:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212357 s, 49.4 MB/s 2024-04-26T23:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284626 s, 36.8 MB/s 2024-04-26T23:50:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.30 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:42.470 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243241 s, 43.1 MB/s 2024-04-26T23:50:42.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.702 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329554 s, 31.8 MB/s 2024-04-26T23:50:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259995 s, 40.3 MB/s 2024-04-26T23:50:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247208 s, 42.4 MB/s 2024-04-26T23:50:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289823 s, 36.2 MB/s 2024-04-26T23:50:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257452 s, 40.7 MB/s 2024-04-26T23:50:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030054 s, 34.9 MB/s 2024-04-26T23:50:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022108 s, 47.4 MB/s 2024-04-26T23:50:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228455 s, 45.9 MB/s 2024-04-26T23:50:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230739 s, 45.4 MB/s 2024-04-26T23:50:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245214 s, 42.8 MB/s 2024-04-26T23:50:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195237 s, 53.7 MB/s 2024-04-26T23:50:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203537 s, 51.5 MB/s 2024-04-26T23:50:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222831 s, 47.1 MB/s 2024-04-26T23:50:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251651 s, 41.7 MB/s 2024-04-26T23:50:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216937 s, 48.3 MB/s 2024-04-26T23:50:43.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246176 s, 42.6 MB/s 2024-04-26T23:50:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230921 s, 45.4 MB/s 2024-04-26T23:50:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208299 s, 50.3 MB/s 2024-04-26T23:50:43.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235943 s, 44.4 MB/s 2024-04-26T23:50:43.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220682 s, 47.5 MB/s 2024-04-26T23:50:43.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271888 s, 38.6 MB/s 2024-04-26T23:50:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305836 s, 34.3 MB/s 2024-04-26T23:50:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252096 s, 41.6 MB/s 2024-04-26T23:50:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216855 s, 48.4 MB/s 2024-04-26T23:50:43.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.419 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.419 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243968 s, 43.0 MB/s 2024-04-26T23:50:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033528 s, 31.3 MB/s 2024-04-26T23:50:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234106 s, 44.8 MB/s 2024-04-26T23:50:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.521 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274219 s, 38.2 MB/s 2024-04-26T23:50:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286395 s, 36.6 MB/s 2024-04-26T23:50:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232327 s, 45.1 MB/s 2024-04-26T23:50:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295507 s, 35.5 MB/s 2024-04-26T23:50:43.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.31 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:50:43.633 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:50:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:50:43.780 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.781 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025401 s, 41.3 MB/s 2024-04-26T23:50:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324008 s, 32.4 MB/s 2024-04-26T23:50:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307044 s, 34.2 MB/s 2024-04-26T23:50:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314376 s, 33.4 MB/s 2024-04-26T23:50:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229617 s, 45.7 MB/s 2024-04-26T23:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313601 s, 33.4 MB/s 2024-04-26T23:50:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197208 s, 53.2 MB/s 2024-04-26T23:50:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033442 s, 31.4 MB/s 2024-04-26T23:50:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243029 s, 43.1 MB/s 2024-04-26T23:50:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296975 s, 35.3 MB/s 2024-04-26T23:50:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239225 s, 43.8 MB/s 2024-04-26T23:50:44.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.146 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226198 s, 46.4 MB/s 2024-04-26T23:50:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225416 s, 46.5 MB/s 2024-04-26T23:50:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031441 s, 33.4 MB/s 2024-04-26T23:50:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226485 s, 46.3 MB/s 2024-04-26T23:50:44.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240251 s, 43.6 MB/s 2024-04-26T23:50:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185719 s, 56.5 MB/s 2024-04-26T23:50:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221889 s, 47.3 MB/s 2024-04-26T23:50:44.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215143 s, 48.7 MB/s 2024-04-26T23:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321905 s, 32.6 MB/s 2024-04-26T23:50:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241263 s, 43.5 MB/s 2024-04-26T23:50:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275504 s, 38.1 MB/s 2024-04-26T23:50:44.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.482 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255427 s, 41.1 MB/s 2024-04-26T23:50:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288857 s, 36.3 MB/s 2024-04-26T23:50:44.545 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.545 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.545 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209564 s, 50.0 MB/s 2024-04-26T23:50:44.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.574 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237814 s, 44.1 MB/s 2024-04-26T23:50:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181808 s, 57.7 MB/s 2024-04-26T23:50:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225108 s, 46.6 MB/s 2024-04-26T23:50:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024768 s, 42.3 MB/s 2024-04-26T23:50:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021897 s, 47.9 MB/s 2024-04-26T23:50:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260944 s, 40.2 MB/s 2024-04-26T23:50:44.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:50:44.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:50:44.751 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302946 s, 34.6 MB/s 2024-04-26T23:50:44.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:50:44.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:44.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-26T23:50:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.975+0000 7f18ab800640 1 Processor -- start 2024-04-26T23:50:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.975+0000 7f18ab800640 1 -- start start 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.976+0000 7f18ab800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a4160540 0x7f18a40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.976+0000 7f18ab800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18a40b3400 0x7f18a4072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.976+0000 7f18ab800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a415cfc0 con 0x7f18a4159970 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.976+0000 7f18ab800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f18a40b3850 con 0x7f18a40b3400 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.976+0000 7f18ab800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18a415faf0 con 0x7f18a4160540 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18a40b3400 0x7f18a4072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18a40b3400 0x7f18a4072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51864/0 (socket says 172.21.15.46:51864) 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 -- 172.21.15.46:0/1671449968 learned_addr learned my addr 172.21.15.46:0/1671449968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a4160540 0x7f18a40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 -- 172.21.15.46:0/1671449968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a4160540 msgr2=0x7f18a40b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a4160540 0x7f18a40b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 -- 172.21.15.46:0/1671449968 >> v1:172.21.15.142:6789/0 conn(0x7f18a4159970 legacy=0x7f18a415c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:44.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.977+0000 7f18a9800640 1 -- 172.21.15.46:0/1671449968 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a415d2a0 con 0x7f18a40b3400 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.978+0000 7f18a9800640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a40b3400 0x7f18a4072020 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f189c009e50 tx=0x7f189c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e94bfc7954e0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18a3400640 1 -- 172.21.15.46:0/1671449968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c036070 con 0x7f18a40b3400 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18a3400640 1 -- 172.21.15.46:0/1671449968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c031600 con 0x7f18a40b3400 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18a3400640 1 -- 172.21.15.46:0/1671449968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0319d0 con 0x7f18a40b3400 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 -- 172.21.15.46:0/1671449968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a40b3400 msgr2=0x7f18a4072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a40b3400 0x7f18a4072020 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f189c009e50 tx=0x7f189c02a630 comp rx=0 tx=0).stop 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 -- 172.21.15.46:0/1671449968 shutdown_connections 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a40b3400 0x7f18a4072020 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 --2- 172.21.15.46:0/1671449968 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a4160540 0x7f18a40b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 -- 172.21.15.46:0/1671449968 >> 172.21.15.46:0/1671449968 conn(0x7f18a406c770 msgr2=0x7f18a406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.979+0000 7f18ab800640 1 -- 172.21.15.46:0/1671449968 shutdown_connections 2024-04-26T23:50:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.980+0000 7f18ab800640 1 -- 172.21.15.46:0/1671449968 wait complete. 2024-04-26T23:50:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.981+0000 7f18ab800640 1 Processor -- start 2024-04-26T23:50:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.981+0000 7f18ab800640 1 -- start start 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a40b3400 0x7f18a414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 0x7f18a40aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4160020 con 0x7f18a4159970 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41609a0 con 0x7f18a4160540 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.982+0000 7f18ab800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18a40b5220 con 0x7f18a40b3400 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51876/0 (socket says 172.21.15.46:51876) 2024-04-26T23:50:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 -- 172.21.15.46:0/1562112307 learned_addr learned my addr 172.21.15.46:0/1562112307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a40b3400 0x7f18a414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a9800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 0x7f18a40aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 -- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a40b3400 msgr2=0x7f18a414f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a40b3400 0x7f18a414f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 -- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 msgr2=0x7f18a40aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 0x7f18a40aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a3e00640 1 -- 172.21.15.46:0/1562112307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a40c5e40 con 0x7f18a4159970 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.983+0000 7f18a9800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 0x7f18a40aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f189000d800 tx=0x7f189000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18a0c00640 1 -- 172.21.15.46:0/1562112307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000b6c0 con 0x7f18a4159970 2024-04-26T23:50:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18a0c00640 1 -- 172.21.15.46:0/1562112307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890019070 con 0x7f18a4159970 2024-04-26T23:50:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18a0c00640 1 -- 172.21.15.46:0/1562112307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000ba80 con 0x7f18a4159970 2024-04-26T23:50:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a40c6120 con 0x7f18a4159970 2024-04-26T23:50:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.984+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a40c66e0 con 0x7f18a4159970 2024-04-26T23:50:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.986+0000 7f18a0c00640 1 -- 172.21.15.46:0/1562112307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f189000bc20 con 0x7f18a4159970 2024-04-26T23:50:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.987+0000 7f18a0c00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.987+0000 7f18a0c00640 1 -- 172.21.15.46:0/1562112307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f189008c1e0 con 0x7f18a4159970 2024-04-26T23:50:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.988+0000 7f18ae7e4740 1 Processor -- start 2024-04-26T23:50:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.988+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.988+0000 7f18ae7e4740 1 -- start start 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c76f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b317bb0f90 con 0x55b317c77450 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b317b4d1e0 con 0x55b317c77b90 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b317bacf50 con 0x55b317c94a70 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51880/0 (socket says 172.21.15.46:51880) 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18a9800640 1 -- 172.21.15.46:0/3595834763 learned_addr learned my addr 172.21.15.46:0/3595834763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.989+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1898006fd0 tx=0x7f1898008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a3e00640 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c76f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a8e00640 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 -- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 msgr2=0x55b317c47400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c47400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 -- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 msgr2=0x55b317c76f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c76f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 -- 172.21.15.46:0/3595834763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31798eba0 con 0x55b317c77450 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f18a9800640 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f189c02ac20 tx=0x7f189c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaa8f669caf9631 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.990+0000 7f1895600640 1 -- 172.21.15.46:0/3595834763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c036040 con 0x55b317c77450 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f1895600640 1 -- 172.21.15.46:0/3595834763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c042070 con 0x55b317c77450 2024-04-26T23:50:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f1895600640 1 -- 172.21.15.46:0/3595834763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c03c890 con 0x55b317c77450 2024-04-26T23:50:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 -- 172.21.15.46:0/3595834763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 msgr2=0x55b317c4c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f189c02ac20 tx=0x7f189c02e000 comp rx=0 tx=0).stop 2024-04-26T23:50:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 -- 172.21.15.46:0/3595834763 shutdown_connections 2024-04-26T23:50:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77450 0x55b317c4c260 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c76f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/3595834763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 -- 172.21.15.46:0/3595834763 >> 172.21.15.46:0/3595834763 conn(0x55b317c49670 msgr2=0x55b317c91f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 -- 172.21.15.46:0/3595834763 shutdown_connections 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.991+0000 7f18ae7e4740 1 -- 172.21.15.46:0/3595834763 wait complete. 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.992+0000 7f18ae7e4740 1 -- start start 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.992+0000 7f18ae7e4740 1 Processor -- start 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317ca6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b317c4b820 con 0x55b317c94a70 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b317b37da0 con 0x55b317c77b90 2024-04-26T23:50:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.993+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b317c4ac10 con 0x55b317c905e0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18ae7e4740 10 client.0 _ll_register_callbacks cb 0x55b317b64400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51890/0 (socket says 172.21.15.46:51890) 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 -- 172.21.15.46:0/216171551 learned_addr learned my addr 172.21.15.46:0/216171551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a9800640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 -- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 msgr2=0x55b317c795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a8e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317ca6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 -- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 msgr2=0x55b317ca6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317ca6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 -- 172.21.15.46:0/216171551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b317882490 con 0x55b317c94a70 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a9800640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.994+0000 7f18a3e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f1890059d00 tx=0x7f189005a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915b40b0b67d6529 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890088060 con 0x55b317c94a70 2024-04-26T23:50:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f18900563e0 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189c042070 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f1890072070 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f18ae7e4740 10 client.9312 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55b3179ef330 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c042070 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f1890013070 con 0x55b317c94a70 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f18ae7e4740 10 client.9312 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.995+0000 7f18ae7e4740 10 client.9312 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18ae7e4740 20 client.9312 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c90a60 0x55b317c92f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18ae7e4740 4 client.9312 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b317bbeba0 con 0x55b317c90a60 2024-04-26T23:50:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18a8e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c90a60 0x55b317c92f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f18a8e00640 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c90a60 0x55b317c92f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9eb334c793fb519 server_cookie=84ff5fd51fde0689 in_seq=0 out_seq=0 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.996+0000 7f188a000640 10 client.9312 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.997+0000 7f188a000640 1 -- 172.21.15.46:0/216171551 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b317bbeba0 con 0x55b317c90a60 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.997+0000 7f188a000640 10 client.9312 handle_command_reply: tid=0 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.997+0000 7f18ae7e4740 1 client.9312 shutdown 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 msgr2=0x55b317c6eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f1890059d00 tx=0x7f189005a530 comp rx=0 tx=0).stop 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 shutdown_connections 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c90a60 0x55b317c92f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c6eb70 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/216171551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317ca6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f188a000640 0 client.9312 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 >> 172.21.15.46:0/216171551 conn(0x55b317c77450 msgr2=0x55b317c95330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 shutdown_connections 2024-04-26T23:50:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 1 -- 172.21.15.46:0/216171551 wait complete. 2024-04-26T23:50:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:44.998+0000 7f18ae7e4740 20 client.9312 trim_cache size 0 max 16384 2024-04-26T23:50:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.018+0000 7f18ae7e4740 1 Processor -- start 2024-04-26T23:50:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.018+0000 7f18ae7e4740 1 -- start start 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c90e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4202010 con 0x55b317c94a70 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4201030 con 0x55b317c77b90 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18a415e730 con 0x55b317c905e0 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51904/0 (socket says 172.21.15.46:51904) 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.019+0000 7f18a9800640 1 -- 172.21.15.46:0/1475642829 learned_addr learned my addr 172.21.15.46:0/1475642829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 -- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 msgr2=0x55b317c8faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c90e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 -- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 msgr2=0x55b317c90e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c90e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 -- 172.21.15.46:0/1475642829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31785b610 con 0x55b317c94a70 2024-04-26T23:50:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f18a9800640 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f189c02a670 tx=0x7f189c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b22c422026c7217 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f1894c00640 1 -- 172.21.15.46:0/1475642829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c04a070 con 0x55b317c94a70 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f1894c00640 1 -- 172.21.15.46:0/1475642829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c042050 con 0x55b317c94a70 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.020+0000 7f1894c00640 1 -- 172.21.15.46:0/1475642829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c036040 con 0x55b317c94a70 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1475642829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 msgr2=0x55b317c8ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f189c02a670 tx=0x7f189c02f000 comp rx=0 tx=0).stop 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1475642829 shutdown_connections 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1475642829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c77b90 0x55b317c90e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1475642829 >> 172.21.15.46:0/1475642829 conn(0x55b317c77450 msgr2=0x55b317c4e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1475642829 shutdown_connections 2024-04-26T23:50:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.021+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1475642829 wait complete. 2024-04-26T23:50:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.022+0000 7f18ae7e4740 1 -- start start 2024-04-26T23:50:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.022+0000 7f18ae7e4740 1 Processor -- start 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18a415ed80 con 0x55b317c77b90 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b317c79800 con 0x55b317c905e0 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b317cb6260 con 0x55b317c94a70 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18ae7e4740 10 client.0 _ll_register_callbacks cb 0x7f189008c0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51910/0 (socket says 172.21.15.46:51910) 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.023+0000 7f18a8e00640 1 -- 172.21.15.46:0/1493106948 learned_addr learned my addr 172.21.15.46:0/1493106948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 -- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 msgr2=0x55b317c8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a9800640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 -- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c905e0 msgr2=0x55b317c8faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 -- 172.21.15.46:0/1493106948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b317c92560 con 0x55b317c77b90 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18a8e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f189800d530 tx=0x7f1898004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18750e72f83dafe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f188a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189801d040 con 0x55b317c77b90 2024-04-26T23:50:45.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f1898014b40 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.024+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189c042050 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f1898034050 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 10 client.9318 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55b317c92f60 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c042050 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f1898037020 con 0x55b317c77b90 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 10 client.9318 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 10 client.9318 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 20 client.9318 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c42a60 0x55b317c6b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 4 client.9318 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-26T23:50:45.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x55b317bb8a10 con 0x55b317c42a60 2024-04-26T23:50:45.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.025+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c42a60 0x55b317c6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.026+0000 7f18a3e00640 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c42a60 0x55b317c6b180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f453cdd40290c809 server_cookie=291f951a0c6e14ff in_seq=0 out_seq=0 2024-04-26T23:50:45.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.026+0000 7f188a000640 10 client.9318 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f188a000640 1 -- 172.21.15.46:0/1493106948 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b317bb8a10 con 0x55b317c42a60 2024-04-26T23:50:45.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f188a000640 10 client.9318 handle_command_reply: tid=0 2024-04-26T23:50:45.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 client.9318 shutdown 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 msgr2=0x55b317d0bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f189800d530 tx=0x7f1898004680 comp rx=0 tx=0).stop 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 shutdown_connections 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x55b317c42a60 0x55b317c6b180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b317c94a70 0x55b317c8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b317c905e0 0x55b317c8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 --2- 172.21.15.46:0/1493106948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b317c77b90 0x55b317d0bf00 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f188a000640 0 client.9318 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 >> 172.21.15.46:0/1493106948 conn(0x55b317c77450 msgr2=0x55b317c4ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 shutdown_connections 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 1 -- 172.21.15.46:0/1493106948 wait complete. 2024-04-26T23:50:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.029+0000 7f18ae7e4740 20 client.9318 trim_cache size 0 max 16384 2024-04-26T23:50:45.036 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:45.036 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:50:45.036 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "800e156f-f5e1-4511-8e02-c893e6a23a6d", 2024-04-26T23:50:45.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:50:45.036 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.030+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 msgr2=0x7f188406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.030+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1898006fd0 tx=0x7f1898008040 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.030+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 msgr2=0x7f18a40aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.030+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f189000d800 tx=0x7f189000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 shutdown_connections 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f188406c8f0 0x7f188406edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4160540 0x7f18a40aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a4159970 0x7f18a40aea50 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 --2- 172.21.15.46:0/1562112307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a40b3400 0x7f18a414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 >> 172.21.15.46:0/1562112307 conn(0x7f18a406c770 msgr2=0x7f18a4158740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 shutdown_connections 2024-04-26T23:50:45.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.031+0000 7f18ab800640 1 -- 172.21.15.46:0/1562112307 wait complete. 2024-04-26T23:50:45.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:45.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-26T23:50:45.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.280+0000 7f2d0e000640 1 Processor -- start 2024-04-26T23:50:45.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.280+0000 7f2d0e000640 1 -- start start 2024-04-26T23:50:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0e000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2d08162b90 0x7f2d080d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d080d0890 con 0x7f2d080b9280 2024-04-26T23:50:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d080d0b70 con 0x7f2d081627b0 2024-04-26T23:50:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0e000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2d080d0640 con 0x7f2d08162b90 2024-04-26T23:50:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0d600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d080b9280 0x7f2d080d2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35926/0 (socket says 172.21.15.46:35926) 2024-04-26T23:50:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.282+0000 7f2d0d600640 1 -- 172.21.15.46:0/1652757774 learned_addr learned my addr 172.21.15.46:0/1652757774 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2cffe00640 1 --2- 172.21.15.46:0/1652757774 >> v2:172.21.15.142:3300/0 conn(0x7f2d08162b90 0x7f2d080d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74432184 0 0) 0x7f2d080d0890 con 0x7f2d080b9280 2024-04-26T23:50:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cec003520 con 0x7f2d080b9280 2024-04-26T23:50:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1510778277 0 0) 0x7f2cec003520 con 0x7f2d080b9280 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 >> v2:172.21.15.142:3300/0 conn(0x7f2d08162b90 msgr2=0x7f2d080d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 --2- 172.21.15.46:0/1652757774 >> v2:172.21.15.142:3300/0 conn(0x7f2d08162b90 0x7f2d080d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 >> v1:172.21.15.142:6789/0 conn(0x7f2d081627b0 legacy=0x7f2d080d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d080d0e50 con 0x7f2d080b9280 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2d000019f0 con 0x7f2d080b9280 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 >> v1:172.21.15.46:6789/0 conn(0x7f2d080b9280 legacy=0x7f2d080d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2cec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cec004800 0x7f2cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cec0052e0 0x7f2cec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080d0640 con 0x7f2cec0039b0 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec00c1b0 con 0x7f2cec004800 2024-04-26T23:50:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.283+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec00c3b0 con 0x7f2cec0052e0 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2cec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0d600640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cec0052e0 0x7f2cec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2cffe00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cec004800 0x7f2cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 -- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cec0052e0 msgr2=0x7f2cec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cec0052e0 0x7f2cec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 -- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cec004800 msgr2=0x7f2cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cec004800 0x7f2cec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0cc00640 1 -- 172.21.15.46:0/1652757774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec003520 con 0x7f2cec0039b0 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2cffe00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cec004800 0x7f2cec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0d600640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cec0052e0 0x7f2cec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.284+0000 7f2d0d600640 1 -- 172.21.15.46:0/1652757774 reap_dead start 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.285+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2cec003dd0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001720 tx=0x7f2cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a350c5678a79a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8001370 con 0x7f2cec0039b0 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf80052d0 con 0x7f2cec0039b0 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d07e00640 1 -- 172.21.15.46:0/1652757774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf802ab70 con 0x7f2cec0039b0 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 -- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 msgr2=0x7f2cec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2cec003dd0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001720 tx=0x7f2cec005830 comp rx=0 tx=0).stop 2024-04-26T23:50:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 -- 172.21.15.46:0/1652757774 shutdown_connections 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 --2- 172.21.15.46:0/1652757774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2cec003dd0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 -- 172.21.15.46:0/1652757774 >> 172.21.15.46:0/1652757774 conn(0x7f2d080d4f90 msgr2=0x7f2d080d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 -- 172.21.15.46:0/1652757774 shutdown_connections 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.286+0000 7f2d0e000640 1 -- 172.21.15.46:0/1652757774 wait complete. 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.288+0000 7f2d0e000640 1 Processor -- start 2024-04-26T23:50:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.288+0000 7f2d0e000640 1 -- start start 2024-04-26T23:50:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d08150590 0x7f2d081509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08153a10 0x7f2d08153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080d1980 con 0x7f2cec0039b0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080b7de0 con 0x7f2d08153a10 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080d1b00 con 0x7f2d08150590 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51926/0 (socket says 172.21.15.46:51926) 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.289+0000 7f2d0cc00640 1 -- 172.21.15.46:0/3865436159 learned_addr learned my addr 172.21.15.46:0/3865436159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 -- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d08150590 msgr2=0x7f2d081509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d08150590 0x7f2d081509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 -- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08153a10 msgr2=0x7f2d08153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08153a10 0x7f2d08153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 -- 172.21.15.46:0/3865436159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d080cdec0 con 0x7f2cec0039b0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000c00 tx=0x7f2cf8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.290+0000 7f2d05600640 1 -- 172.21.15.46:0/3865436159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf803f030 con 0x7f2cec0039b0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.291+0000 7f2d05600640 1 -- 172.21.15.46:0/3865436159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005480 con 0x7f2cec0039b0 2024-04-26T23:50:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.291+0000 7f2d05600640 1 -- 172.21.15.46:0/3865436159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8037040 con 0x7f2cec0039b0 2024-04-26T23:50:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.291+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d080ce1a0 con 0x7f2cec0039b0 2024-04-26T23:50:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.291+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d080ce760 con 0x7f2cec0039b0 2024-04-26T23:50:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.293+0000 7f2d05600640 1 -- 172.21.15.46:0/3865436159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf8042070 con 0x7f2cec0039b0 2024-04-26T23:50:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.294+0000 7f2d05600640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 0x7f2ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.294+0000 7f2d05600640 1 -- 172.21.15.46:0/3865436159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2cf80ab770 con 0x7f2cec0039b0 2024-04-26T23:50:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.295+0000 7f2cffe00640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 0x7f2ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.295+0000 7f2d1256d740 1 Processor -- start 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.295+0000 7f2d1256d740 1 -- start start 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2cffe00640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 0x7f2ce406edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2d080a2f10 tx=0x7f2cf0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b8f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b911f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56216b82d900 con 0x56216b8c9e20 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56216b7ca1d0 con 0x56216b9119e0 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56216b8298c0 con 0x56216b8f4140 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51942/0 (socket says 172.21.15.46:51942) 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d0d600640 1 -- 172.21.15.46:0/348282468 learned_addr learned my addr 172.21.15.46:0/348282468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2cffe00640 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b911f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.296+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b8f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 -- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 msgr2=0x56216b8f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b8f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 -- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 msgr2=0x56216b911f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b911f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 -- 172.21.15.46:0/348282468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216b6097d0 con 0x56216b8c9e20 2024-04-26T23:50:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2d0d600640 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2d0808f300 tx=0x7f2d0000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30573bc49ccd892f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2cfd200640 1 -- 172.21.15.46:0/348282468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004b60 con 0x56216b8c9e20 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2cfd200640 1 -- 172.21.15.46:0/348282468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00004d00 con 0x56216b8c9e20 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.297+0000 7f2cfd200640 1 -- 172.21.15.46:0/348282468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007350 con 0x56216b8c9e20 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 -- 172.21.15.46:0/348282468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 msgr2=0x56216b9100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2d0808f300 tx=0x7f2d0000d520 comp rx=0 tx=0).stop 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 -- 172.21.15.46:0/348282468 shutdown_connections 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8c9e20 0x56216b9100a0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b911f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 --2- 172.21.15.46:0/348282468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b8f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 -- 172.21.15.46:0/348282468 >> 172.21.15.46:0/348282468 conn(0x56216b8c6750 msgr2=0x56216b90ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 -- 172.21.15.46:0/348282468 shutdown_connections 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.298+0000 7f2d1256d740 1 -- 172.21.15.46:0/348282468 wait complete. 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.299+0000 7f2d1256d740 1 -- start start 2024-04-26T23:50:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.299+0000 7f2d1256d740 1 Processor -- start 2024-04-26T23:50:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b9119e0 0x56216b8f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56216b8cabc0 con 0x56216b8f4140 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56216b7b5120 con 0x56216b907990 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56216b8c4680 con 0x56216b9119e0 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d1256d740 10 client.0 _ll_register_callbacks cb 0x56216b7ed750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.300+0000 7f2d0cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51946/0 (socket says 172.21.15.46:51946) 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 -- 172.21.15.46:0/2011785680 learned_addr learned my addr 172.21.15.46:0/2011785680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 -- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b9119e0 msgr2=0x56216b8f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b9119e0 0x56216b8f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b9119e0 0x56216b8f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 -- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 msgr2=0x56216b907e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 -- 172.21.15.46:0/2011785680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216b4ff0f0 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8078f70 tx=0x7f2cf80433b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56580e5e04cfa056 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2cf5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8081bc0 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.301+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2cf80a5c70 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d00004d00 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f2cf8079270 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 10 client.9330 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56216b669f60 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00004d00 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f2cf804c060 con 0x56216b8f4140 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 10 client.9330 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 10 client.9330 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 20 client.9330 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b906a20 0x56216b906e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 4 client.9330 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.302+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216b83b510 con 0x56216b906a20 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.303+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b906a20 0x56216b906e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.303+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b906a20 0x56216b906e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea434bc4273cf516 server_cookie=b3e5e26ea354c529 in_seq=0 out_seq=0 2024-04-26T23:50:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.303+0000 7f2cf5600640 10 client.9330 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2cf5600640 1 -- 172.21.15.46:0/2011785680 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56216b83b510 con 0x56216b906a20 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2cf5600640 10 client.9330 handle_command_reply: tid=0 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 client.9330 shutdown 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 msgr2=0x56216b8f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8078f70 tx=0x7f2cf80433b0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 shutdown_connections 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b906a20 0x56216b906e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b9119e0 0x56216b8f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2011785680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b8f4140 0x56216b8f7960 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2cf5600640 0 client.9330 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.304+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 >> 172.21.15.46:0/2011785680 conn(0x56216b8c9e20 msgr2=0x56216b9124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.305+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 shutdown_connections 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.305+0000 7f2d1256d740 1 -- 172.21.15.46:0/2011785680 wait complete. 2024-04-26T23:50:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.305+0000 7f2d1256d740 20 client.9330 trim_cache size 0 max 16384 2024-04-26T23:50:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.324+0000 7f2d1256d740 1 Processor -- start 2024-04-26T23:50:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.324+0000 7f2d1256d740 1 -- start start 2024-04-26T23:50:45.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08202510 con 0x56216b9119e0 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08201560 con 0x56216b907990 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080cb990 con 0x56216b8f4140 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51956/0 (socket says 172.21.15.46:51956) 2024-04-26T23:50:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.325+0000 7f2d0d600640 1 -- 172.21.15.46:0/789631451 learned_addr learned my addr 172.21.15.46:0/789631451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 -- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 msgr2=0x56216b90e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 -- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 msgr2=0x56216b907e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2cffe00640 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 -- 172.21.15.46:0/789631451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216b4d81b0 con 0x56216b9119e0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2d0d600640 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2d000017f0 tx=0x7f2d00010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8438d0f94b9234d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2cf7e00640 1 -- 172.21.15.46:0/789631451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004d60 con 0x56216b9119e0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2cf7e00640 1 -- 172.21.15.46:0/789631451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00007070 con 0x56216b9119e0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.326+0000 7f2cf7e00640 1 -- 172.21.15.46:0/789631451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007460 con 0x56216b9119e0 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 -- 172.21.15.46:0/789631451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 msgr2=0x56216b910410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2d000017f0 tx=0x7f2d00010000 comp rx=0 tx=0).stop 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 -- 172.21.15.46:0/789631451 shutdown_connections 2024-04-26T23:50:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b9119e0 0x56216b910410 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b907990 0x56216b907e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 --2- 172.21.15.46:0/789631451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 -- 172.21.15.46:0/789631451 >> 172.21.15.46:0/789631451 conn(0x56216b8c9e20 msgr2=0x56216b8f0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 -- 172.21.15.46:0/789631451 shutdown_connections 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.327+0000 7f2d1256d740 1 -- 172.21.15.46:0/789631451 wait complete. 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.328+0000 7f2d1256d740 1 -- start start 2024-04-26T23:50:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.328+0000 7f2d1256d740 1 Processor -- start 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b90c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56216b8f3740 con 0x56216b907990 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081f8e40 con 0x56216b9119e0 2024-04-26T23:50:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56216b8c9b70 con 0x56216b8f4140 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2d1256d740 10 client.0 _ll_register_callbacks cb 0x7f2cf8079040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51958/0 (socket says 172.21.15.46:51958) 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.329+0000 7f2cffe00640 1 -- 172.21.15.46:0/2941434312 learned_addr learned my addr 172.21.15.46:0/2941434312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 -- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 msgr2=0x56216b90f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 -- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 msgr2=0x56216b90c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b90c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 -- 172.21.15.46:0/2941434312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216b8f7300 con 0x56216b907990 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cffe00640 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x56216b7b5200 tx=0x7f2cf0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5721108e1b08d304 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cf5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf002f060 con 0x56216b907990 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2cf0004030 con 0x56216b907990 2024-04-26T23:50:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.330+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d00007070 con 0x56216b907990 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f2cf0035050 con 0x56216b907990 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 10 client.9336 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56216b90f890 con 0x56216b907990 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00007070 con 0x56216b907990 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f2cf0038020 con 0x56216b907990 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 10 client.9336 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 10 client.9336 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 20 client.9336 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b971680 0x56216b910410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 4 client.9336 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x56216b91fc60 con 0x56216b971680 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.331+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b971680 0x56216b910410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d0cc00640 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b971680 0x56216b910410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222aa0a4b0874365 server_cookie=d15f57d5cf854d1d in_seq=0 out_seq=0 2024-04-26T23:50:45.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2cf5600640 10 client.9336 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2cf5600640 1 -- 172.21.15.46:0/2941434312 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56216b91fc60 con 0x56216b971680 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2cf5600640 10 client.9336 handle_command_reply: tid=0 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d1256d740 1 client.9336 shutdown 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 msgr2=0x56216b8f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x56216b7b5200 tx=0x7f2cf0009190 comp rx=0 tx=0).stop 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 shutdown_connections 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.332+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56216b971680 0x56216b910410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216b9119e0 0x56216b90c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56216b907990 0x56216b8f6060 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 --2- 172.21.15.46:0/2941434312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56216b8f4140 0x56216b90f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2cf5600640 0 client.9336 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 >> 172.21.15.46:0/2941434312 conn(0x56216b8c9e20 msgr2=0x56216b911190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 shutdown_connections 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 1 -- 172.21.15.46:0/2941434312 wait complete. 2024-04-26T23:50:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d1256d740 20 client.9336 trim_cache size 0 max 16384 2024-04-26T23:50:45.340 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:50:45.340 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 msgr2=0x7f2ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 0x7f2ce406edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2d080a2f10 tx=0x7f2cf0008040 comp rx=0 tx=0).stop 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 msgr2=0x7f2d0809ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.333+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000c00 tx=0x7f2cf8001120 comp rx=0 tx=0).stop 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 shutdown_connections 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ce406c8f0 0x7f2ce406edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08153a10 0x7f2d08153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d08150590 0x7f2d081509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 --2- 172.21.15.46:0/3865436159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cec0039b0 0x7f2d0809ecd0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 >> 172.21.15.46:0/3865436159 conn(0x7f2d080d4f90 msgr2=0x7f2d0806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 shutdown_connections 2024-04-26T23:50:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.334+0000 7f2d0e000640 1 -- 172.21.15.46:0/3865436159 wait complete. 2024-04-26T23:50:45.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:45.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-26T23:50:45.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.559+0000 7f80b0c00640 1 Processor -- start 2024-04-26T23:50:45.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.559+0000 7f80b0c00640 1 -- start start 2024-04-26T23:50:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.560+0000 7f80b0c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 0x7f80ac0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.560+0000 7f80b0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80ac0cec40 0x7f80ac06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.560+0000 7f80b0c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ac068130 con 0x7f80ac0b0620 2024-04-26T23:50:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80b0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f80ac0b6290 con 0x7f80ac0cec40 2024-04-26T23:50:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80b0c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f80ac0b18d0 con 0x7f80ac0ce2a0 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80ac0cec40 0x7f80ac06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80ac0cec40 0x7f80ac06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51970/0 (socket says 172.21.15.46:51970) 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 -- 172.21.15.46:0/3200877382 learned_addr learned my addr 172.21.15.46:0/3200877382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80ab400640 1 --2- 172.21.15.46:0/3200877382 >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 0x7f80ac0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 -- 172.21.15.46:0/3200877382 >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 msgr2=0x7f80ac0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 --2- 172.21.15.46:0/3200877382 >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 0x7f80ac0b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.561+0000 7f80aaa00640 1 -- 172.21.15.46:0/3200877382 >> v1:172.21.15.142:6789/0 conn(0x7f80ac0b0620 legacy=0x7f80ac0b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.562+0000 7f80aaa00640 1 -- 172.21.15.46:0/3200877382 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac0683a0 con 0x7f80ac0cec40 2024-04-26T23:50:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.562+0000 7f80ab400640 1 --2- 172.21.15.46:0/3200877382 >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 0x7f80ac0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.563+0000 7f80aaa00640 1 --2- 172.21.15.46:0/3200877382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0cec40 0x7f80ac06c350 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f80ac073540 tx=0x7f809802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09d643897cbaac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.563+0000 7f80aa000640 1 -- 172.21.15.46:0/3200877382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098037070 con 0x7f80ac0cec40 2024-04-26T23:50:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.563+0000 7f80aa000640 1 -- 172.21.15.46:0/3200877382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8098032440 con 0x7f80ac0cec40 2024-04-26T23:50:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.563+0000 7f80aa000640 1 -- 172.21.15.46:0/3200877382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098032810 con 0x7f80ac0cec40 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 -- 172.21.15.46:0/3200877382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0cec40 msgr2=0x7f80ac06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3200877382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0cec40 0x7f80ac06c350 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f80ac073540 tx=0x7f809802a230 comp rx=0 tx=0).stop 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 -- 172.21.15.46:0/3200877382 shutdown_connections 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3200877382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0cec40 0x7f80ac06c350 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3200877382 >> v2:172.21.15.142:3300/0 conn(0x7f80ac0ce2a0 0x7f80ac0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 -- 172.21.15.46:0/3200877382 >> 172.21.15.46:0/3200877382 conn(0x7f80ac0b3000 msgr2=0x7f80ac0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 -- 172.21.15.46:0/3200877382 shutdown_connections 2024-04-26T23:50:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.564+0000 7f80b0c00640 1 -- 172.21.15.46:0/3200877382 wait complete. 2024-04-26T23:50:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.566+0000 7f80b0c00640 1 Processor -- start 2024-04-26T23:50:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.566+0000 7f80b0c00640 1 -- start start 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ac0ce2a0 0x7f80ac15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 0x7f80ac09a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac163970 con 0x7f80ac0b0620 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac163da0 con 0x7f80ac0ce2a0 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80b0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80ac163af0 con 0x7f80ac0cec40 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51974/0 (socket says 172.21.15.46:51974) 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80ab400640 1 -- 172.21.15.46:0/2574456262 learned_addr learned my addr 172.21.15.46:0/2574456262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.567+0000 7f80aaa00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ac0ce2a0 0x7f80ac15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80abe00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 0x7f80ac09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 -- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 msgr2=0x7f80ac09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 0x7f80ac09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 -- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ac0ce2a0 msgr2=0x7f80ac15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ac0ce2a0 0x7f80ac15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 -- 172.21.15.46:0/2574456262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac15ec70 con 0x7f80ac0b0620 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80abe00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 0x7f80ac09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80ab400640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c7f0 tx=0x7f809c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80a3400640 1 -- 172.21.15.46:0/2574456262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c015070 con 0x7f80ac0b0620 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80a3400640 1 -- 172.21.15.46:0/2574456262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c007a30 con 0x7f80ac0b0620 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80a3400640 1 -- 172.21.15.46:0/2574456262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c007e00 con 0x7f80ac0b0620 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ac15ef10 con 0x7f80ac0b0620 2024-04-26T23:50:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.568+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ac15f4d0 con 0x7f80ac0b0620 2024-04-26T23:50:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.571+0000 7f80a3400640 1 -- 172.21.15.46:0/2574456262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f809c018070 con 0x7f80ac0b0620 2024-04-26T23:50:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.572+0000 7f80a3400640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 0x7f808c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.572+0000 7f80a3400640 1 -- 172.21.15.46:0/2574456262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f809c08bcf0 con 0x7f80ac0b0620 2024-04-26T23:50:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.572+0000 7f80b4f72740 1 Processor -- start 2024-04-26T23:50:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.572+0000 7f80b4f72740 1 -- start start 2024-04-26T23:50:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.572+0000 7f80aaa00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 0x7f808c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x560804406500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 0x560804404670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5608042ce030 con 0x5608043e8710 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608042e97f0 con 0x5608043be3f0 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.573+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5608042a47e0 con 0x560804405fb0 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51988/0 (socket says 172.21.15.46:51988) 2024-04-26T23:50:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 -- 172.21.15.46:0/2263862406 learned_addr learned my addr 172.21.15.46:0/2263862406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80aaa00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 0x7f808c06ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8098034000 tx=0x7f809802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80abe00640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 0x560804404670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80aaa00640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x560804406500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 -- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 msgr2=0x560804406500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x560804406500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 -- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 msgr2=0x560804404670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 0x560804404670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 -- 172.21.15.46:0/2263862406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608042b0050 con 0x5608043e8710 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80abe00640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 0x560804404670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.574+0000 7f80ab400640 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f809c055c50 tx=0x7f809c0598b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbde122b3180f518 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f8097400640 1 -- 172.21.15.46:0/2263862406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c01f070 con 0x5608043e8710 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f8097400640 1 -- 172.21.15.46:0/2263862406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c0600c0 con 0x5608043e8710 2024-04-26T23:50:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f8097400640 1 -- 172.21.15.46:0/2263862406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c055e60 con 0x5608043e8710 2024-04-26T23:50:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 -- 172.21.15.46:0/2263862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 msgr2=0x5608043e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f809c055c50 tx=0x7f809c0598b0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 -- 172.21.15.46:0/2263862406 shutdown_connections 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043be3f0 0x560804404670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x560804406500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2263862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043e8af0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.575+0000 7f80b4f72740 1 -- 172.21.15.46:0/2263862406 >> 172.21.15.46:0/2263862406 conn(0x5608043bad20 msgr2=0x560804403520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.576+0000 7f80b4f72740 1 -- 172.21.15.46:0/2263862406 shutdown_connections 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.576+0000 7f80b4f72740 1 -- 172.21.15.46:0/2263862406 wait complete. 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.576+0000 7f80b4f72740 1 -- start start 2024-04-26T23:50:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.577+0000 7f80b4f72740 1 Processor -- start 2024-04-26T23:50:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x5608043e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043cf0a0 0x5608043e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560804326200 con 0x5608043e8710 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560804320ee0 con 0x5608043cf0a0 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56080432de70 con 0x560804405fb0 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80b4f72740 10 client.0 _ll_register_callbacks cb 0x560804317270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51998/0 (socket says 172.21.15.46:51998) 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 -- 172.21.15.46:0/289480592 learned_addr learned my addr 172.21.15.46:0/289480592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 -- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 msgr2=0x5608043e0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80aaa00640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x5608043e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x5608043e0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 -- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043cf0a0 msgr2=0x5608043e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043cf0a0 0x5608043e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80ab400640 1 -- 172.21.15.46:0/289480592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608040fd7d0 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.578+0000 7f80aaa00640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x5608043e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f80ab400640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f809c060270 tx=0x7f809c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb1503d4552029f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f808be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c089070 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f809c058e40 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809c0600c0 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f809c00ce40 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f80b4f72740 10 client.9348 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5608040333f0 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.579+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c0600c0 con 0x5608043e8710 2024-04-26T23:50:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f809c071030 con 0x5608043e8710 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 10 client.9348 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 10 client.9348 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 20 client.9348 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x5608043e5ea0 0x5608043e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 4 client.9348 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56080432f7e0 con 0x5608043e5ea0 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.580+0000 7f80aaa00640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x5608043e5ea0 0x5608043e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.581+0000 7f80aaa00640 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x5608043e5ea0 0x5608043e2210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46eecbab3340afc9 server_cookie=d4998f8c88dbcead in_seq=0 out_seq=0 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.581+0000 7f808be00640 10 client.9348 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.581+0000 7f808be00640 1 -- 172.21.15.46:0/289480592 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56080432f7e0 con 0x5608043e5ea0 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.581+0000 7f808be00640 10 client.9348 handle_command_reply: tid=0 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.581+0000 7f80b4f72740 1 client.9348 shutdown 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 msgr2=0x5608043cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f809c060270 tx=0x7f809c000bc0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 shutdown_connections 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x5608043e5ea0 0x5608043e2210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043cf0a0 0x5608043e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560804405fb0 0x5608043e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 --2- 172.21.15.46:0/289480592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043e8710 0x5608043cf630 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f808be00640 0 client.9348 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 >> 172.21.15.46:0/289480592 conn(0x5608043be3f0 msgr2=0x560804406aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 shutdown_connections 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 1 -- 172.21.15.46:0/289480592 wait complete. 2024-04-26T23:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.582+0000 7f80b4f72740 20 client.9348 trim_cache size 0 max 16384 2024-04-26T23:50:45.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.601+0000 7f80b4f72740 1 Processor -- start 2024-04-26T23:50:45.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 -- start start 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043cf0a0 0x5608043de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043e8710 0x5608043f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac201bf0 con 0x560804405fb0 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac200c60 con 0x5608043e8710 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.602+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80ac161310 con 0x5608043cf0a0 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52006/0 (socket says 172.21.15.46:52006) 2024-04-26T23:50:45.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 -- 172.21.15.46:0/2548893574 learned_addr learned my addr 172.21.15.46:0/2548893574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 -- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043cf0a0 msgr2=0x5608043de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80aaa00640 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043e8710 0x5608043f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043cf0a0 0x5608043de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 -- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043e8710 msgr2=0x5608043f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043e8710 0x5608043f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.603+0000 7f80abe00640 1 -- 172.21.15.46:0/2548893574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560803ff30f0 con 0x560804405fb0 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f80abe00640 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005380 tx=0x7f80a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bd7038a1326731 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f8096a00640 1 -- 172.21.15.46:0/2548893574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4006d50 con 0x560804405fb0 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f8096a00640 1 -- 172.21.15.46:0/2548893574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a400c070 con 0x560804405fb0 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f8096a00640 1 -- 172.21.15.46:0/2548893574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a400ecb0 con 0x560804405fb0 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f80b4f72740 1 -- 172.21.15.46:0/2548893574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 msgr2=0x5608043f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.604+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005380 tx=0x7f80a4005840 comp rx=0 tx=0).stop 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 -- 172.21.15.46:0/2548893574 shutdown_connections 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560804405fb0 0x5608043f93f0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608043e8710 0x5608043f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 --2- 172.21.15.46:0/2548893574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043cf0a0 0x5608043de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 -- 172.21.15.46:0/2548893574 >> 172.21.15.46:0/2548893574 conn(0x5608043be3f0 msgr2=0x5608043dedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 -- 172.21.15.46:0/2548893574 shutdown_connections 2024-04-26T23:50:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 -- 172.21.15.46:0/2548893574 wait complete. 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.605+0000 7f80b4f72740 1 -- start start 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.606+0000 7f80b4f72740 1 Processor -- start 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.606+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.606+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043e8710 0x5608043e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.606+0000 7f80b4f72740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560804405fb0 0x5608043e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80b4f72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56080432efd0 con 0x5608043cf0a0 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac2006c0 con 0x560804405fb0 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80b4f72740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5608043e4a80 con 0x5608043e8710 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80b4f72740 10 client.0 _ll_register_callbacks cb 0x7f809c059ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52020/0 (socket says 172.21.15.46:52020) 2024-04-26T23:50:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 -- 172.21.15.46:0/4104831730 learned_addr learned my addr 172.21.15.46:0/4104831730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 -- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043e8710 msgr2=0x5608043e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043e8710 0x5608043e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 -- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560804405fb0 msgr2=0x5608043e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560804405fb0 0x5608043e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 -- 172.21.15.46:0/4104831730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608042e9a30 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f80ab400640 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f809c055c50 tx=0x7f809c089ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ccd2ab3d051db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.607+0000 7f808be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c067060 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f809c058e40 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a400c070 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f809c06d050 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 10 client.9354 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5608042e9a30 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a400c070 con 0x5608043cf0a0 2024-04-26T23:50:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f809c071070 con 0x5608043cf0a0 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 10 client.9354 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 10 client.9354 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 20 client.9354 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56080447d000 0x5608043e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 4 client.9354 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.608+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560804329650 con 0x56080447d000 2024-04-26T23:50:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.609+0000 7f80aaa00640 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56080447d000 0x5608043e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.609+0000 7f80aaa00640 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56080447d000 0x5608043e0260 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec38222d01eebb2 server_cookie=f447329e8b7f8a05 in_seq=0 out_seq=0 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.609+0000 7f808be00640 10 client.9354 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f808be00640 1 -- 172.21.15.46:0/4104831730 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+69 (crc 0 0 0) 0x560804329650 con 0x56080447d000 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f808be00640 10 client.9354 handle_command_reply: tid=0 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 client.9354 shutdown 2024-04-26T23:50:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 msgr2=0x560804411b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f809c055c50 tx=0x7f809c089ec0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 shutdown_connections 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x56080447d000 0x5608043e0260 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560804405fb0 0x5608043e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5608043e8710 0x5608043e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f808be00640 0 client.9354 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 --2- 172.21.15.46:0/4104831730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608043cf0a0 0x560804411b30 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 >> 172.21.15.46:0/4104831730 conn(0x5608043be3f0 msgr2=0x5608044069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 shutdown_connections 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 1 -- 172.21.15.46:0/4104831730 wait complete. 2024-04-26T23:50:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.610+0000 7f80b4f72740 20 client.9354 trim_cache size 0 max 16384 2024-04-26T23:50:45.617 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:45.617 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-26T23:50:45.617 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:50:45.617 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 msgr2=0x7f808c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 0x7f808c06ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8098034000 tx=0x7f809802e000 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 msgr2=0x7f80ac15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c7f0 tx=0x7f809c00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 shutdown_connections 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f808c06c920 0x7f808c06ede0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ac0cec40 0x7f80ac09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ac0ce2a0 0x7f80ac15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 --2- 172.21.15.46:0/2574456262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac0b0620 0x7f80ac15b940 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 >> 172.21.15.46:0/2574456262 conn(0x7f80ac0b3000 msgr2=0x7f80ac0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 shutdown_connections 2024-04-26T23:50:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.611+0000 7f80b0c00640 1 -- 172.21.15.46:0/2574456262 wait complete. 2024-04-26T23:50:45.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.836+0000 7fa9f4400640 1 Processor -- start 2024-04-26T23:50:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.836+0000 7fa9f4400640 1 -- start start 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f4400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9ec160bd0 0x7fa9ec161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f4400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9ec0732c0 0x7fa9ec073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f4400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ec161940 con 0x7fa9ec06e510 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f4400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec071f80 con 0x7fa9ec160bd0 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f4400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec072260 con 0x7fa9ec0732c0 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.838+0000 7fa9f1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9ec160bd0 0x7fa9ec161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9ec160bd0 0x7fa9ec161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52034/0 (socket says 172.21.15.46:52034) 2024-04-26T23:50:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 -- 172.21.15.46:0/1065623947 learned_addr learned my addr 172.21.15.46:0/1065623947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f2400640 1 --2- 172.21.15.46:0/1065623947 >> v2:172.21.15.142:3300/0 conn(0x7fa9ec0732c0 0x7fa9ec073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 -- 172.21.15.46:0/1065623947 >> v2:172.21.15.142:3300/0 conn(0x7fa9ec0732c0 msgr2=0x7fa9ec073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 --2- 172.21.15.46:0/1065623947 >> v2:172.21.15.142:3300/0 conn(0x7fa9ec0732c0 0x7fa9ec073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 -- 172.21.15.46:0/1065623947 >> v1:172.21.15.142:6789/0 conn(0x7fa9ec06e510 legacy=0x7fa9ec06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.839+0000 7fa9f1000640 1 -- 172.21.15.46:0/1065623947 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec072540 con 0x7fa9ec160bd0 2024-04-26T23:50:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.840+0000 7fa9f1000640 1 --2- 172.21.15.46:0/1065623947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec160bd0 0x7fa9ec161060 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea8fad4f5374783 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9ebe00640 1 -- 172.21.15.46:0/1065623947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037070 con 0x7fa9ec160bd0 2024-04-26T23:50:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9ebe00640 1 -- 172.21.15.46:0/1065623947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d8032440 con 0x7fa9ec160bd0 2024-04-26T23:50:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9ebe00640 1 -- 172.21.15.46:0/1065623947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8032810 con 0x7fa9ec160bd0 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 -- 172.21.15.46:0/1065623947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec160bd0 msgr2=0x7fa9ec161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 --2- 172.21.15.46:0/1065623947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec160bd0 0x7fa9ec161060 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d802a250 comp rx=0 tx=0).stop 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 -- 172.21.15.46:0/1065623947 shutdown_connections 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 --2- 172.21.15.46:0/1065623947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec160bd0 0x7fa9ec161060 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 --2- 172.21.15.46:0/1065623947 >> v2:172.21.15.142:3300/0 conn(0x7fa9ec0732c0 0x7fa9ec073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 -- 172.21.15.46:0/1065623947 >> 172.21.15.46:0/1065623947 conn(0x7fa9ec06c530 msgr2=0x7fa9ec06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.841+0000 7fa9f4400640 1 -- 172.21.15.46:0/1065623947 shutdown_connections 2024-04-26T23:50:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.842+0000 7fa9f4400640 1 -- 172.21.15.46:0/1065623947 wait complete. 2024-04-26T23:50:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.843+0000 7fa9f4400640 1 Processor -- start 2024-04-26T23:50:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.844+0000 7fa9f4400640 1 -- start start 2024-04-26T23:50:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.844+0000 7fa9f4400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 0x7fa9ec158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.844+0000 7fa9f4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f4400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9ec160bd0 0x7fa9ec0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec161720 con 0x7fa9ec0732c0 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f4400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1615a0 con 0x7fa9ec06e510 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f4400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec072d40 con 0x7fa9ec160bd0 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52036/0 (socket says 172.21.15.46:52036) 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 -- 172.21.15.46:0/2648284366 learned_addr learned my addr 172.21.15.46:0/2648284366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1a00640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 0x7fa9ec158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 -- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9ec160bd0 msgr2=0x7fa9ec0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9ec160bd0 0x7fa9ec0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 -- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 msgr2=0x7fa9ec158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 0x7fa9ec158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1000640 1 -- 172.21.15.46:0/2648284366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec158690 con 0x7fa9ec0732c0 2024-04-26T23:50:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.845+0000 7fa9f1a00640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 0x7fa9ec158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9f1000640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa9d802a290 tx=0x7fa9d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037040 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d8004250 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec0b2440 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8003610 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.846+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec0b2a00 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.847+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ec069530 con 0x7fa9ec0732c0 2024-04-26T23:50:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.848+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9d80037b0 con 0x7fa9ec0732c0 2024-04-26T23:50:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.849+0000 7fa9e9600640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.849+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fa9d80acab0 con 0x7fa9ec0732c0 2024-04-26T23:50:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.850+0000 7fa9f1a00640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.851+0000 7fa9f1a00640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000a830 tx=0x7fa9e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:45.854+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9d807acb0 con 0x7fa9ec0732c0 2024-04-26T23:50:46.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.100+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9ec06ec70 con 0x7fa9ec0732c0 2024-04-26T23:50:46.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.100+0000 7fa9e9600640 1 -- 172.21.15.46:0/2648284366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7fa9d807a5b0 con 0x7fa9ec0732c0 2024-04-26T23:50:46.104 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:46.105 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 msgr2=0x7fa9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000a830 tx=0x7fa9e0008040 comp rx=0 tx=0).stop 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 msgr2=0x7fa9ec0b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa9d802a290 tx=0x7fa9d80046c0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 shutdown_connections 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9ec160bd0 0x7fa9ec0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9ec0732c0 0x7fa9ec0b18f0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 --2- 172.21.15.46:0/2648284366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec06e510 0x7fa9ec158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.103+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 >> 172.21.15.46:0/2648284366 conn(0x7fa9ec06c530 msgr2=0x7fa9ec15c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.104+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 shutdown_connections 2024-04-26T23:50:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.104+0000 7fa9f4400640 1 -- 172.21.15.46:0/2648284366 wait complete. 2024-04-26T23:50:46.127 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:46.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.337+0000 7f7481000640 1 Processor -- start 2024-04-26T23:50:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.338+0000 7f7481000640 1 -- start start 2024-04-26T23:50:46.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.339+0000 7f7481000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f747c1604c0 0x7f747c073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:46.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.339+0000 7f7481000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0b43f0 con 0x7f747c142280 2024-04-26T23:50:46.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f7481000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0b46d0 con 0x7f747c06ec20 2024-04-26T23:50:46.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f7481000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f747c0b49b0 con 0x7f747c1604c0 2024-04-26T23:50:46.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f747a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f747c1604c0 0x7f747c073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:46.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f7479e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f747c142280 0x7f747c06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35930/0 (socket says 172.21.15.46:35930) 2024-04-26T23:50:46.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f7479e00640 1 -- 172.21.15.46:0/668267114 learned_addr learned my addr 172.21.15.46:0/668267114 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:46.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f747a800640 1 -- 172.21.15.46:0/668267114 >> v1:172.21.15.142:6789/0 conn(0x7f747c06ec20 legacy=0x7f747c0734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:46.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.341+0000 7f747a800640 1 -- 172.21.15.46:0/668267114 >> v1:172.21.15.46:6789/0 conn(0x7f747c142280 legacy=0x7f747c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.341+0000 7f747a800640 1 -- 172.21.15.46:0/668267114 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c0b4c90 con 0x7f747c1604c0 2024-04-26T23:50:46.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.340+0000 7f7479400640 1 -- 172.21.15.46:0/668267114 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741333011 0 0) 0x7f747c0b43f0 con 0x7f747c142280 2024-04-26T23:50:46.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f747a800640 1 --2- 172.21.15.46:0/668267114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c073bb0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f7474009d90 tx=0x7f747402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98194f1aeb0538d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f7479400640 1 -- 172.21.15.46:0/668267114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474007740 con 0x7f747c1604c0 2024-04-26T23:50:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f7479400640 1 -- 172.21.15.46:0/668267114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74740078e0 con 0x7f747c1604c0 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f7479400640 1 -- 172.21.15.46:0/668267114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474007cb0 con 0x7f747c1604c0 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f7481000640 1 -- 172.21.15.46:0/668267114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 msgr2=0x7f747c073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.342+0000 7f7481000640 1 --2- 172.21.15.46:0/668267114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c073bb0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f7474009d90 tx=0x7f747402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.343+0000 7f7481000640 1 -- 172.21.15.46:0/668267114 shutdown_connections 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.343+0000 7f7481000640 1 --2- 172.21.15.46:0/668267114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c073bb0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.343+0000 7f7481000640 1 -- 172.21.15.46:0/668267114 >> 172.21.15.46:0/668267114 conn(0x7f747c06c530 msgr2=0x7f747c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.343+0000 7f7481000640 1 -- 172.21.15.46:0/668267114 shutdown_connections 2024-04-26T23:50:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.343+0000 7f7481000640 1 -- 172.21.15.46:0/668267114 wait complete. 2024-04-26T23:50:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.345+0000 7f7481000640 1 Processor -- start 2024-04-26T23:50:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.345+0000 7f7481000640 1 -- start start 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f747c06ec20 0x7f747c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 0x7f747c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f747c142a20 con 0x7f747c1604c0 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1428a0 con 0x7f747c142280 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7481000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f747c0b3e80 con 0x7f747c06ec20 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52058/0 (socket says 172.21.15.46:52058) 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 -- 172.21.15.46:0/875830790 learned_addr learned my addr 172.21.15.46:0/875830790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7479e00640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f747c06ec20 0x7f747c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7473e00640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 0x7f747c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 -- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f747c06ec20 msgr2=0x7f747c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f747c06ec20 0x7f747c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 -- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 msgr2=0x7f747c1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 0x7f747c1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f747a800640 1 -- 172.21.15.46:0/875830790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c15cbb0 con 0x7f747c1604c0 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.346+0000 7f7473e00640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 0x7f747c1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f747a800640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7474009eb0 tx=0x7f7474004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474040070 con 0x7f747c1604c0 2024-04-26T23:50:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474031c90 con 0x7f747c1604c0 2024-04-26T23:50:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474045680 con 0x7f747c1604c0 2024-04-26T23:50:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c1f5350 con 0x7f747c1604c0 2024-04-26T23:50:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.347+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c1f5910 con 0x7f747c1604c0 2024-04-26T23:50:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.349+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747c069530 con 0x7f747c1604c0 2024-04-26T23:50:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.349+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7474039040 con 0x7f747c1604c0 2024-04-26T23:50:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.351+0000 7f7472000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.351+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f74740ad6c0 con 0x7f747c1604c0 2024-04-26T23:50:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.352+0000 7f7479e00640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.353+0000 7f7479e00640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 0x7f745806edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f746c000f60 tx=0x7f746c007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.356+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f747407bb30 con 0x7f747c1604c0 2024-04-26T23:50:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.633+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f747c161c20 con 0x7f747c1604c0 2024-04-26T23:50:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.634+0000 7f7472000640 1 -- 172.21.15.46:0/875830790 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1189 (secure 0 0 0) 0x7f7474043e40 con 0x7f747c1604c0 2024-04-26T23:50:46.637 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:46.637 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":838,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661005,"bytes_used":2591629312,"bytes_avail":770502483968,"bytes_total":773094113280,"write_bytes_sec":34237870,"read_op_per_sec":0,"write_op_per_sec":36},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-26T23:49:42.171148+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 msgr2=0x7f745806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 0x7f745806edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f746c000f60 tx=0x7f746c007b50 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 msgr2=0x7f747c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7474009eb0 tx=0x7f7474004a40 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 shutdown_connections 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f747c1604c0 0x7f747c0b59a0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f747c142280 0x7f747c1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 --2- 172.21.15.46:0/875830790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f747c06ec20 0x7f747c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 >> 172.21.15.46:0/875830790 conn(0x7f747c06c530 msgr2=0x7f747c1609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 shutdown_connections 2024-04-26T23:50:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:46.636+0000 7f7481000640 1 -- 172.21.15.46:0/875830790 wait complete. 2024-04-26T23:50:46.656 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:50:47.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.880+0000 7f9451c00640 1 Processor -- start 2024-04-26T23:50:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.880+0000 7f9451c00640 1 -- start start 2024-04-26T23:50:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 0x7f944c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f944c073970 0x7f944c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944c072bb0 con 0x7f944c0c6a50 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f944c072e90 con 0x7f944c073970 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f944c073170 con 0x7f944c1454a0 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f944c0c6a50 0x7f944c0cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35942/0 (socket says 172.21.15.46:35942) 2024-04-26T23:50:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.882+0000 7f9451200640 1 -- 172.21.15.46:0/979210361 learned_addr learned my addr 172.21.15.46:0/979210361 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944be00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 0x7f944c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944b400640 1 --2- 172.21.15.46:0/979210361 >> v2:172.21.15.142:3300/0 conn(0x7f944c073970 0x7f944c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443032837 0 0) 0x7f944c072bb0 con 0x7f944c0c6a50 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9430003520 con 0x7f944c0c6a50 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1497612177 0 0) 0x7f9430003520 con 0x7f944c0c6a50 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 msgr2=0x7f944c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 0x7f944c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 >> v2:172.21.15.142:3300/0 conn(0x7f944c073970 msgr2=0x7f944c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 --2- 172.21.15.46:0/979210361 >> v2:172.21.15.142:3300/0 conn(0x7f944c073970 0x7f944c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c073450 con 0x7f944c0c6a50 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f94440019f0 con 0x7f944c0c6a50 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 >> v1:172.21.15.46:6789/0 conn(0x7f944c0c6a50 legacy=0x7f944c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 0x7f94300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9430003f30 0x7f9430004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9430004d40 0x7f94300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f944c072e90 con 0x7f9430004d40 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f944c073170 con 0x7f9430003f30 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.883+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f943000dc20 con 0x7f944c1454a0 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9430004d40 0x7f94300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 -- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 msgr2=0x7f94300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1454a0 0x7f94300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 -- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9430003f30 msgr2=0x7f9430004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9430003f30 0x7f9430004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.884+0000 7f9451200640 1 -- 172.21.15.46:0/979210361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9430003520 con 0x7f9430004d40 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.885+0000 7f9451200640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9430004d40 0x7f94300051c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f944400a880 tx=0x7f9430005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c412359e7eb6f4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:47.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.885+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400a4a0 con 0x7f9430004d40 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.885+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944402f070 con 0x7f9430004d40 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.885+0000 7f944aa00640 1 -- 172.21.15.46:0/979210361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944402bd40 con 0x7f9430004d40 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 -- 172.21.15.46:0/979210361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9430004d40 msgr2=0x7f94300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 --2- 172.21.15.46:0/979210361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9430004d40 0x7f94300051c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f944400a880 tx=0x7f9430005830 comp rx=0 tx=0).stop 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451200640 1 -- 172.21.15.46:0/979210361 reap_dead start 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 -- 172.21.15.46:0/979210361 shutdown_connections 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 -- 172.21.15.46:0/979210361 >> 172.21.15.46:0/979210361 conn(0x7f944c0b26d0 msgr2=0x7f944c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 -- 172.21.15.46:0/979210361 shutdown_connections 2024-04-26T23:50:47.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.886+0000 7f9451c00640 1 -- 172.21.15.46:0/979210361 wait complete. 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.888+0000 7f9451c00640 1 Processor -- start 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.888+0000 7f9451c00640 1 -- start start 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 0x7f944c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1f5780 0x7f944c0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f944c073dd0 con 0x7f944c15edd0 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f944c072880 con 0x7f944c15b190 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f9451c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f944c1f8810 con 0x7f944c1f5780 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f944b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f944b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52068/0 (socket says 172.21.15.46:52068) 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.889+0000 7f944b400640 1 -- 172.21.15.46:0/1688969837 learned_addr learned my addr 172.21.15.46:0/1688969837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944be00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 0x7f944c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 -- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1f5780 msgr2=0x7f944c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f9451200640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1f5780 0x7f944c0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1f5780 0x7f944c0b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 -- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 msgr2=0x7f944c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 0x7f944c15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 -- 172.21.15.46:0/1688969837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1f5f70 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944be00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 0x7f944c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f944b400640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f943800d800 tx=0x7f943800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.890+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943800b660 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.891+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9438019070 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.891+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943800ba40 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.891+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1f8a60 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.891+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c15b570 con 0x7f944c15edd0 2024-04-26T23:50:47.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.892+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c0693b0 con 0x7f944c15edd0 2024-04-26T23:50:47.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.893+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f943800bbe0 con 0x7f944c15edd0 2024-04-26T23:50:47.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.894+0000 7f9443e00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.894+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f943808c0a0 con 0x7f944c15edd0 2024-04-26T23:50:47.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.895+0000 7f944be00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.896+0000 7f944be00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f943c0099e0 tx=0x7f943c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:47.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:47.898+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f943805a330 con 0x7f944c15edd0 2024-04-26T23:50:48.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.148+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f944c0cefa0 con 0x7f944c15edd0 2024-04-26T23:50:48.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.149+0000 7f9443e00640 1 -- 172.21.15.46:0/1688969837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f9438059c30 con 0x7f944c15edd0 2024-04-26T23:50:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:48.153 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:48.153 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 msgr2=0x7f942c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f943c0099e0 tx=0x7f943c008040 comp rx=0 tx=0).stop 2024-04-26T23:50:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 msgr2=0x7f944c1f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f943800d800 tx=0x7f943800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 shutdown_connections 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f944c1f5780 0x7f944c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944c15edd0 0x7f944c1f5240 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 --2- 172.21.15.46:0/1688969837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f944c15b190 0x7f944c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 >> 172.21.15.46:0/1688969837 conn(0x7f944c0b26d0 msgr2=0x7f944c0c6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 shutdown_connections 2024-04-26T23:50:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.152+0000 7f9451c00640 1 -- 172.21.15.46:0/1688969837 wait complete. 2024-04-26T23:50:48.173 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:48.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.384+0000 7fecca800640 1 Processor -- start 2024-04-26T23:50:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.385+0000 7fecca800640 1 -- start start 2024-04-26T23:50:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.386+0000 7fecca800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 0x7fecc4073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.386+0000 7fecca800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc40b43f0 con 0x7fecc406ec20 2024-04-26T23:50:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.386+0000 7fecca800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc40b46d0 con 0x7fecc4142280 2024-04-26T23:50:48.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.386+0000 7fecca800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecc40b49b0 con 0x7fecc41604c0 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.387+0000 7fecc2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecc406ec20 0x7fecc40734a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35956/0 (socket says 172.21.15.46:35956) 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.387+0000 7fecc2a00640 1 -- 172.21.15.46:0/3388390535 learned_addr learned my addr 172.21.15.46:0/3388390535 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.387+0000 7fecc3e00640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 0x7fecc4073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.387+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736180065 0 0) 0x7fecc40b43f0 con 0x7fecc406ec20 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.387+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feca8003520 con 0x7fecc406ec20 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1797715110 0 0) 0x7feca8003520 con 0x7fecc406ec20 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 msgr2=0x7fecc4073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 0x7fecc4073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 >> v1:172.21.15.142:6789/0 conn(0x7fecc4142280 legacy=0x7fecc406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc40b4c90 con 0x7fecc406ec20 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fecb0001690 con 0x7fecc406ec20 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 >> v1:172.21.15.46:6789/0 conn(0x7fecc406ec20 legacy=0x7fecc40734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feca80039e0 0x7feca8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca8004800 0x7feca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 0x7feca80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecc40b49b0 con 0x7feca80039e0 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feca800baa0 con 0x7feca8004800 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feca800bca0 con 0x7fecc41604c0 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.388+0000 7fecc3400640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feca80039e0 0x7feca8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc2a00640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca8004800 0x7feca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc3400640 1 -- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 msgr2=0x7feca80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc3400640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc41604c0 0x7feca80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc3400640 1 -- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca8004800 msgr2=0x7feca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc3400640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca8004800 0x7feca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc3400640 1 -- 172.21.15.46:0/3388390535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca8003520 con 0x7feca80039e0 2024-04-26T23:50:48.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.389+0000 7fecc2a00640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca8004800 0x7feca8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.390+0000 7fecc3400640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feca80039e0 0x7feca8003dc0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fecb4001f30 tx=0x7feca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0dc00d8340cde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.390+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4001b30 con 0x7feca80039e0 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.390+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb4031620 con 0x7feca80039e0 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.390+0000 7fecc2000640 1 -- 172.21.15.46:0/3388390535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4031a50 con 0x7feca80039e0 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 -- 172.21.15.46:0/3388390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feca80039e0 msgr2=0x7feca8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 --2- 172.21.15.46:0/3388390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feca80039e0 0x7feca8003dc0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fecb4001f30 tx=0x7feca8005830 comp rx=0 tx=0).stop 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecc3e00640 1 -- 172.21.15.46:0/3388390535 reap_dead start 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 -- 172.21.15.46:0/3388390535 shutdown_connections 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 -- 172.21.15.46:0/3388390535 >> 172.21.15.46:0/3388390535 conn(0x7fecc406c530 msgr2=0x7fecc406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 -- 172.21.15.46:0/3388390535 shutdown_connections 2024-04-26T23:50:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.391+0000 7fecca800640 1 -- 172.21.15.46:0/3388390535 wait complete. 2024-04-26T23:50:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.393+0000 7fecca800640 1 Processor -- start 2024-04-26T23:50:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.393+0000 7fecca800640 1 -- start start 2024-04-26T23:50:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca80039e0 0x7fecc4139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 0x7fecc41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4142a20 con 0x7fecc415a900 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41428a0 con 0x7feca80039e0 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecca800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecc40b3e80 con 0x7fecc413fd30 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52086/0 (socket says 172.21.15.46:52086) 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecc2a00640 1 -- 172.21.15.46:0/3408477848 learned_addr learned my addr 172.21.15.46:0/3408477848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecc3400640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca80039e0 0x7fecc4139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.394+0000 7fecc3e00640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 0x7fecc41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 -- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 msgr2=0x7fecc41401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 0x7fecc41401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 -- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca80039e0 msgr2=0x7fecc4139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca80039e0 0x7fecc4139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 -- 172.21.15.46:0/3408477848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc4139f40 con 0x7fecc415a900 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc3e00640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 0x7fecc41401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecc2a00640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d940 tx=0x7fecb000de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0019070 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb00074e0 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0004a20 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc415bdc0 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.395+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc415c380 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.397+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc4069530 con 0x7fecc415a900 2024-04-26T23:50:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.398+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecb001c070 con 0x7fecc415a900 2024-04-26T23:50:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.399+0000 7fecbb400640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.399+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fecb008b860 con 0x7fecc415a900 2024-04-26T23:50:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.400+0000 7fecc3400640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.401+0000 7fecc3400640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 0x7feca406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.404+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecc4139f40 con 0x7fecc415a900 2024-04-26T23:50:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.689+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fecc4141920 con 0x7fecc415a900 2024-04-26T23:50:48.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.690+0000 7fecbb400640 1 -- 172.21.15.46:0/3408477848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1189 (secure 0 0 0) 0x7fecb0059a60 con 0x7fecc415a900 2024-04-26T23:50:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:48.693 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":840,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661005,"bytes_used":2591629312,"bytes_avail":770502483968,"bytes_total":773094113280,"write_bytes_sec":21892128,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-26T23:49:42.171148+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 msgr2=0x7feca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 0x7feca406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402f000 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 msgr2=0x7fecc415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d940 tx=0x7fecb000de00 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 shutdown_connections 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc413fd30 0x7fecc41401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecc415a900 0x7fecc415ad80 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 --2- 172.21.15.46:0/3408477848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca80039e0 0x7fecc4139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 >> 172.21.15.46:0/3408477848 conn(0x7fecc406c530 msgr2=0x7fecc4143370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 shutdown_connections 2024-04-26T23:50:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:48.693+0000 7fecca800640 1 -- 172.21.15.46:0/3408477848 wait complete. 2024-04-26T23:50:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:50:49.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.933+0000 7f37f9800640 1 Processor -- start 2024-04-26T23:50:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.934+0000 7f37f9800640 1 -- start start 2024-04-26T23:50:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 0x7f37f40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f9800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37f40b4000 0x7f37f40b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f40d11f0 con 0x7f37f415f4e0 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f9800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37f40d0ce0 con 0x7f37f40b4000 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37f40d2990 con 0x7f37f40d2130 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f37f415f4e0 0x7f37f40d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35966/0 (socket says 172.21.15.46:35966) 2024-04-26T23:50:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.935+0000 7f37f3e00640 1 -- 172.21.15.46:0/635183471 learned_addr learned my addr 172.21.15.46:0/635183471 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967764544 0 0) 0x7f37f40d11f0 con 0x7f37f415f4e0 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d4003520 con 0x7f37f415f4e0 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f3400640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 0x7f37f40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f8e00640 1 --2- 172.21.15.46:0/635183471 >> v2:172.21.15.142:3300/0 conn(0x7f37f40b4000 0x7f37f40b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 111588532 0 0) 0x7f37d4003520 con 0x7f37f415f4e0 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 msgr2=0x7f37f40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 0x7f37f40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 >> v2:172.21.15.142:3300/0 conn(0x7f37f40b4000 msgr2=0x7f37f40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 --2- 172.21.15.46:0/635183471 >> v2:172.21.15.142:3300/0 conn(0x7f37f40b4000 0x7f37f40b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f40d5f20 con 0x7f37f415f4e0 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f37e4001670 con 0x7f37f415f4e0 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 >> v1:172.21.15.46:6789/0 conn(0x7f37f415f4e0 legacy=0x7f37f40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40039b0 0x7f37d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37d4004800 0x7f37d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 0x7f37d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40d2990 con 0x7f37d4004800 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37d400b9f0 con 0x7f37d40039b0 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37d400bbf0 con 0x7f37f40d2130 2024-04-26T23:50:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.936+0000 7f37f3400640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37d4004800 0x7f37d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3e00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40039b0 0x7f37d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3400640 1 -- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 msgr2=0x7f37d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3400640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f40d2130 0x7f37d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3400640 1 -- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40039b0 msgr2=0x7f37d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3400640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40039b0 0x7f37d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3400640 1 -- 172.21.15.46:0/635183471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d4003520 con 0x7f37d4004800 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.937+0000 7f37f3e00640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40039b0 0x7f37d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.938+0000 7f37f3400640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37d4004800 0x7f37d4004c80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007330 tx=0x7f37d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc29bd4bc007fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.938+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc033070 con 0x7f37d4004800 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.938+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc02f660 con 0x7f37d4004800 2024-04-26T23:50:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.938+0000 7f37f2a00640 1 -- 172.21.15.46:0/635183471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc009be0 con 0x7f37d4004800 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 -- 172.21.15.46:0/635183471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37d4004800 msgr2=0x7f37d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 --2- 172.21.15.46:0/635183471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37d4004800 0x7f37d4004c80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007330 tx=0x7f37d4005830 comp rx=0 tx=0).stop 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f8e00640 1 -- 172.21.15.46:0/635183471 reap_dead start 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 -- 172.21.15.46:0/635183471 shutdown_connections 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 -- 172.21.15.46:0/635183471 >> 172.21.15.46:0/635183471 conn(0x7f37f40d5010 msgr2=0x7f37f40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 -- 172.21.15.46:0/635183471 shutdown_connections 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.939+0000 7f37f9800640 1 -- 172.21.15.46:0/635183471 wait complete. 2024-04-26T23:50:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.941+0000 7f37f9800640 1 Processor -- start 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.941+0000 7f37f9800640 1 -- start start 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4004800 0x7f37f41567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f4161ab0 0x7f37f4161f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40b1d60 con 0x7f37f4158550 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40d5c40 con 0x7f37d4004800 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37f415f8c0 con 0x7f37f4161ab0 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52112/0 (socket says 172.21.15.46:52112) 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f3400640 1 -- 172.21.15.46:0/1555385077 learned_addr learned my addr 172.21.15.46:0/1555385077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.942+0000 7f37f3e00640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4004800 0x7f37f41567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 -- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f4161ab0 msgr2=0x7f37f4161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f8e00640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f4161ab0 0x7f37f4161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f4161ab0 0x7f37f4161f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 -- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4004800 msgr2=0x7f37f41567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4004800 0x7f37f41567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 -- 172.21.15.46:0/1555385077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f415a610 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f3400640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007420 tx=0x7f37dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc033040 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc02f460 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc009be0 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.943+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f415a8f0 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.944+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f415aeb0 con 0x7f37f4158550 2024-04-26T23:50:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.945+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f40693b0 con 0x7f37f4158550 2024-04-26T23:50:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.946+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37dc036070 con 0x7f37f4158550 2024-04-26T23:50:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.947+0000 7f37ebe00640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.947+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f37dc0ac8d0 con 0x7f37f4158550 2024-04-26T23:50:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.947+0000 7f37f3e00640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.948+0000 7f37f3e00640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005bd0 tx=0x7f37e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:49.952+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37dc07ab60 con 0x7f37f4158550 2024-04-26T23:50:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.195+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f40d33d0 con 0x7f37f4158550 2024-04-26T23:50:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.197+0000 7f37ebe00640 1 -- 172.21.15.46:0/1555385077 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f37dc07a460 con 0x7f37f4158550 2024-04-26T23:50:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:50.200 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.199+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 msgr2=0x7f37d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.199+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005bd0 tx=0x7f37e4009040 comp rx=0 tx=0).stop 2024-04-26T23:50:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.199+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 msgr2=0x7f37f41589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.199+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007420 tx=0x7f37dc004ab0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 shutdown_connections 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37f4161ab0 0x7f37f4161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f4158550 0x7f37f41589d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 --2- 172.21.15.46:0/1555385077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4004800 0x7f37f41567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 >> 172.21.15.46:0/1555385077 conn(0x7f37f40d5010 msgr2=0x7f37f40d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 shutdown_connections 2024-04-26T23:50:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.200+0000 7f37f9800640 1 -- 172.21.15.46:0/1555385077 wait complete. 2024-04-26T23:50:50.224 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:50.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.425+0000 7f1a7fe00640 1 Processor -- start 2024-04-26T23:50:50.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.426+0000 7f1a7fe00640 1 -- start start 2024-04-26T23:50:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.427+0000 7f1a7fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a800cf450 0x7f1a80073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1a800cff40 0x7f1a80166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8006f1d0 con 0x7f1a80072190 2024-04-26T23:50:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8006f470 con 0x7f1a800cf450 2024-04-26T23:50:50.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1a80072930 con 0x7f1a800cff40 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a800cf450 0x7f1a80073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a800cf450 0x7f1a80073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52118/0 (socket says 172.21.15.46:52118) 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 -- 172.21.15.46:0/3737429047 learned_addr learned my addr 172.21.15.46:0/3737429047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a77e00640 1 --1- 172.21.15.46:0/3737429047 >> v1:172.21.15.46:6789/0 conn(0x7f1a80072190 0x7f1a800cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35978/0 (socket says 172.21.15.46:35978) 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7f400640 1 --2- 172.21.15.46:0/3737429047 >> v2:172.21.15.142:3300/0 conn(0x7f1a800cff40 0x7f1a80166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 -- 172.21.15.46:0/3737429047 >> v2:172.21.15.142:3300/0 conn(0x7f1a800cff40 msgr2=0x7f1a80166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 --2- 172.21.15.46:0/3737429047 >> v2:172.21.15.142:3300/0 conn(0x7f1a800cff40 0x7f1a80166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 -- 172.21.15.46:0/3737429047 >> v1:172.21.15.46:6789/0 conn(0x7f1a80072190 legacy=0x7f1a800cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.428+0000 7f1a7ea00640 1 -- 172.21.15.46:0/3737429047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80072c10 con 0x7f1a800cf450 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.430+0000 7f1a7ea00640 1 --2- 172.21.15.46:0/3737429047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a80073db0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006f3a0 tx=0x7f1a7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ca41e40e571e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.430+0000 7f1a7e000640 1 -- 172.21.15.46:0/3737429047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037070 con 0x7f1a800cf450 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.430+0000 7f1a7e000640 1 -- 172.21.15.46:0/3737429047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70032440 con 0x7f1a800cf450 2024-04-26T23:50:50.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.430+0000 7f1a7e000640 1 -- 172.21.15.46:0/3737429047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70032810 con 0x7f1a800cf450 2024-04-26T23:50:50.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 -- 172.21.15.46:0/3737429047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 msgr2=0x7f1a80073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/3737429047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a80073db0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006f3a0 tx=0x7f1a7002a270 comp rx=0 tx=0).stop 2024-04-26T23:50:50.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 -- 172.21.15.46:0/3737429047 shutdown_connections 2024-04-26T23:50:50.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/3737429047 >> v2:172.21.15.142:3300/0 conn(0x7f1a800cff40 0x7f1a80166b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/3737429047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a80073db0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 -- 172.21.15.46:0/3737429047 >> 172.21.15.46:0/3737429047 conn(0x7f1a800b2820 msgr2=0x7f1a800b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:50.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 -- 172.21.15.46:0/3737429047 shutdown_connections 2024-04-26T23:50:50.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.431+0000 7f1a7fe00640 1 -- 172.21.15.46:0/3737429047 wait complete. 2024-04-26T23:50:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.433+0000 7f1a7fe00640 1 Processor -- start 2024-04-26T23:50:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.433+0000 7f1a7fe00640 1 -- start start 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a80072190 0x7f1a801e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a800cff40 0x7f1a801e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800734b0 con 0x7f1a800cf450 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800731d0 con 0x7f1a800cff40 2024-04-26T23:50:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a7fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1a80072620 con 0x7f1a80072190 2024-04-26T23:50:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.434+0000 7f1a77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52122/0 (socket says 172.21.15.46:52122) 2024-04-26T23:50:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 -- 172.21.15.46:0/2431055551 learned_addr learned my addr 172.21.15.46:0/2431055551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a7ea00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a80072190 0x7f1a801e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a7f400640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a800cff40 0x7f1a801e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 -- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a80072190 msgr2=0x7f1a801e20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a80072190 0x7f1a801e20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 -- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a800cff40 msgr2=0x7f1a801e3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a800cff40 0x7f1a801e3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.435+0000 7f1a77e00640 1 -- 172.21.15.46:0/2431055551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801dfe60 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a77e00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c3c0 tx=0x7f1a6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680074e0 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68007680 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801f4e50 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68004a00 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.436+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801f5410 con 0x7f1a800cf450 2024-04-26T23:50:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.438+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a800693b0 con 0x7f1a800cf450 2024-04-26T23:50:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.440+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a6801a020 con 0x7f1a800cf450 2024-04-26T23:50:50.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.441+0000 7f1a76a00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 0x7f1a5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.441+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f1a6808b540 con 0x7f1a800cf450 2024-04-26T23:50:50.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.442+0000 7f1a7ea00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 0x7f1a5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.443+0000 7f1a7ea00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 0x7f1a5c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a70034000 tx=0x7f1a7002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.450+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a68059740 con 0x7f1a800cf450 2024-04-26T23:50:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.731+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1a80073ef0 con 0x7f1a800cf450 2024-04-26T23:50:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.732+0000 7f1a76a00640 1 -- 172.21.15.46:0/2431055551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1189 (secure 0 0 0) 0x7f1a68059040 con 0x7f1a800cf450 2024-04-26T23:50:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:50.735 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":842,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661005,"bytes_used":2591629312,"bytes_avail":770502483968,"bytes_total":773094113280,"write_bytes_sec":21892911,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-26T23:49:42.171148+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.734+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 msgr2=0x7f1a5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.734+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 0x7f1a5c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a70034000 tx=0x7f1a7002e000 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.734+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 msgr2=0x7f1a801df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.734+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c3c0 tx=0x7f1a6800c880 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 shutdown_connections 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1a5c06c8f0 0x7f1a5c06edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a800cff40 0x7f1a801e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800cf450 0x7f1a801df6c0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 --2- 172.21.15.46:0/2431055551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a80072190 0x7f1a801e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 >> 172.21.15.46:0/2431055551 conn(0x7f1a800b2820 msgr2=0x7f1a800afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 shutdown_connections 2024-04-26T23:50:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:50.735+0000 7f1a7fe00640 1 -- 172.21.15.46:0/2431055551 wait complete. 2024-04-26T23:50:50.757 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:50:51.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:51.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.973+0000 7fc1af200640 1 Processor -- start 2024-04-26T23:50:51.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.974+0000 7fc1af200640 1 -- start start 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.975+0000 7fc1af200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1b01454a0 0x7fc1b00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.975+0000 7fc1af200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b0073970 0x7fc1b00cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.975+0000 7fc1af200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1b0072bb0 con 0x7fc1b00c6a50 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.975+0000 7fc1af200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b0072e90 con 0x7fc1b01454a0 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.975+0000 7fc1af200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b0073170 con 0x7fc1b0073970 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1b01454a0 0x7fc1b00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:51.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1b01454a0 0x7fc1b00cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59480/0 (socket says 172.21.15.46:59480) 2024-04-26T23:50:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 -- 172.21.15.46:0/1570753799 learned_addr learned my addr 172.21.15.46:0/1570753799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ad400640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b0073970 0x7fc1b00cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:51.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 -- 172.21.15.46:0/1570753799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b0073970 msgr2=0x7fc1b00cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b0073970 0x7fc1b00cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 -- 172.21.15.46:0/1570753799 >> v1:172.21.15.142:6789/0 conn(0x7fc1b00c6a50 legacy=0x7fc1b00cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:51.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.976+0000 7fc1ade00640 1 -- 172.21.15.46:0/1570753799 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b0073450 con 0x7fc1b01454a0 2024-04-26T23:50:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.978+0000 7fc1ade00640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b01454a0 0x7fc1b00cc690 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009870 tx=0x7fc1a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31866822184dc10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.978+0000 7fc1aca00640 1 -- 172.21.15.46:0/1570753799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0037070 con 0x7fc1b01454a0 2024-04-26T23:50:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.978+0000 7fc1aca00640 1 -- 172.21.15.46:0/1570753799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a0032440 con 0x7fc1b01454a0 2024-04-26T23:50:51.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.978+0000 7fc1aca00640 1 -- 172.21.15.46:0/1570753799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0032810 con 0x7fc1b01454a0 2024-04-26T23:50:51.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 -- 172.21.15.46:0/1570753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b01454a0 msgr2=0x7fc1b00cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:51.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b01454a0 0x7fc1b00cc690 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009870 tx=0x7fc1a002a210 comp rx=0 tx=0).stop 2024-04-26T23:50:51.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 -- 172.21.15.46:0/1570753799 shutdown_connections 2024-04-26T23:50:51.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b0073970 0x7fc1b00cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 --2- 172.21.15.46:0/1570753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b01454a0 0x7fc1b00cc690 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 -- 172.21.15.46:0/1570753799 >> 172.21.15.46:0/1570753799 conn(0x7fc1b00b26d0 msgr2=0x7fc1b00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 -- 172.21.15.46:0/1570753799 shutdown_connections 2024-04-26T23:50:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.979+0000 7fc1af200640 1 -- 172.21.15.46:0/1570753799 wait complete. 2024-04-26T23:50:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.981+0000 7fc1af200640 1 Processor -- start 2024-04-26T23:50:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.981+0000 7fc1af200640 1 -- start start 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b0073970 0x7fc1b013f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b01454a0 0x7fc1b0143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0073dd0 con 0x7fc1b00c6a50 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0072880 con 0x7fc1b0073970 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1af200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b013eb30 con 0x7fc1b01454a0 2024-04-26T23:50:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1ad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1ad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59494/0 (socket says 172.21.15.46:59494) 2024-04-26T23:50:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.982+0000 7fc1ad400640 1 -- 172.21.15.46:0/3922194718 learned_addr learned my addr 172.21.15.46:0/3922194718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ade00640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b0073970 0x7fc1b013f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 -- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b01454a0 msgr2=0x7fc1b0143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b01454a0 0x7fc1b0143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 -- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b0073970 msgr2=0x7fc1b013f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b0073970 0x7fc1b013f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 -- 172.21.15.46:0/3922194718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b015dd40 con 0x7fc1b00c6a50 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1ad400640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc19800d800 tx=0x7fc19800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800b6c0 con 0x7fc1b00c6a50 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc198019070 con 0x7fc1b00c6a50 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b013ed90 con 0x7fc1b00c6a50 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.983+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800ba80 con 0x7fc1b00c6a50 2024-04-26T23:50:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.984+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b013f350 con 0x7fc1b00c6a50 2024-04-26T23:50:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.985+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b00693b0 con 0x7fc1b00c6a50 2024-04-26T23:50:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.986+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc19800bc20 con 0x7fc1b00c6a50 2024-04-26T23:50:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.987+0000 7fc1a6000640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 0x7fc18c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.987+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fc19808c290 con 0x7fc1b00c6a50 2024-04-26T23:50:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.987+0000 7fc1ade00640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 0x7fc18c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.988+0000 7fc1ade00640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 0x7fc18c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0034000 tx=0x7fc1a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:51.991+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc19801c360 con 0x7fc1b00c6a50 2024-04-26T23:50:52.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.242+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1b00d0120 con 0x7fc1b00c6a50 2024-04-26T23:50:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.243+0000 7fc1a6000640 1 -- 172.21.15.46:0/3922194718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7fc19805a490 con 0x7fc1b00c6a50 2024-04-26T23:50:52.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:52.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 msgr2=0x7fc18c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 0x7fc18c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0034000 tx=0x7fc1a002e000 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 msgr2=0x7fc1b01f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc19800d800 tx=0x7fc19800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 shutdown_connections 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc18c06c8f0 0x7fc18c06edb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1b01454a0 0x7fc1b0143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b00c6a50 0x7fc1b01f8130 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 --2- 172.21.15.46:0/3922194718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b0073970 0x7fc1b013f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.246+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 >> 172.21.15.46:0/3922194718 conn(0x7fc1b00b26d0 msgr2=0x7fc1b006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.247+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 shutdown_connections 2024-04-26T23:50:52.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.247+0000 7fc1af200640 1 -- 172.21.15.46:0/3922194718 wait complete. 2024-04-26T23:50:52.270 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:52.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.464+0000 7f54e6800640 1 Processor -- start 2024-04-26T23:50:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.464+0000 7f54e6800640 1 -- start start 2024-04-26T23:50:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.465+0000 7f54e6800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e0159950 0x7f54e00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.465+0000 7f54e6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54e00ae8e0 0x7f54e0162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.465+0000 7f54e6800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e006f430 con 0x7f54e01631d0 2024-04-26T23:50:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.465+0000 7f54e6800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f54e0163620 con 0x7f54e00ae8e0 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.465+0000 7f54e6800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f54e00af0c0 con 0x7f54e0159950 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54e00ae8e0 0x7f54e0162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f54e01631d0 0x7f54e015f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52878/0 (socket says 172.21.15.46:52878) 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54e00ae8e0 0x7f54e0162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59504/0 (socket says 172.21.15.46:59504) 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e5e00640 1 -- 172.21.15.46:0/1582399461 learned_addr learned my addr 172.21.15.46:0/1582399461 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e5400640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e0159950 0x7f54e00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 -- 172.21.15.46:0/1582399461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e0159950 msgr2=0x7f54e00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e0159950 0x7f54e00b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 -- 172.21.15.46:0/1582399461 >> v1:172.21.15.46:6789/0 conn(0x7f54e01631d0 legacy=0x7f54e015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54e4a00640 1 -- 172.21.15.46:0/1582399461 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e00af3a0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.466+0000 7f54dfe00640 1 -- 172.21.15.46:0/1582399461 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440739739 0 0) 0x7f54e006f430 con 0x7f54e01631d0 2024-04-26T23:50:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.467+0000 7f54e4a00640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0162c90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0097a0 tx=0x7f54cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0bc01814b625f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54dfe00640 1 -- 172.21.15.46:0/1582399461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc037070 con 0x7f54e00ae8e0 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54dfe00640 1 -- 172.21.15.46:0/1582399461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc032440 con 0x7f54e00ae8e0 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54dfe00640 1 -- 172.21.15.46:0/1582399461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc032810 con 0x7f54e00ae8e0 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54e6800640 1 -- 172.21.15.46:0/1582399461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 msgr2=0x7f54e0162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54e6800640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0162c90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0097a0 tx=0x7f54cc02a180 comp rx=0 tx=0).stop 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54e6800640 1 -- 172.21.15.46:0/1582399461 shutdown_connections 2024-04-26T23:50:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54e6800640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0162c90 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.468+0000 7f54e6800640 1 --2- 172.21.15.46:0/1582399461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e0159950 0x7f54e00b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.469+0000 7f54e6800640 1 -- 172.21.15.46:0/1582399461 >> 172.21.15.46:0/1582399461 conn(0x7f54e00b2820 msgr2=0x7f54e00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.469+0000 7f54e6800640 1 -- 172.21.15.46:0/1582399461 shutdown_connections 2024-04-26T23:50:52.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.469+0000 7f54e6800640 1 -- 172.21.15.46:0/1582399461 wait complete. 2024-04-26T23:50:52.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.470+0000 7f54e6800640 1 Processor -- start 2024-04-26T23:50:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.470+0000 7f54e6800640 1 -- start start 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 0x7f54e015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 0x7f54e00c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00afc40 con 0x7f54e00ae8e0 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00af960 con 0x7f54e0155dc0 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.471+0000 7f54e6800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f54e00b0160 con 0x7f54e01631d0 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59506/0 (socket says 172.21.15.46:59506) 2024-04-26T23:50:52.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 -- 172.21.15.46:0/651409005 learned_addr learned my addr 172.21.15.46:0/651409005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5e00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 0x7f54e00c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e4a00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 0x7f54e015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 -- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 msgr2=0x7f54e015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 0x7f54e015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 -- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 msgr2=0x7f54e00c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 0x7f54e00c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5400640 1 -- 172.21.15.46:0/651409005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e0148c60 con 0x7f54e00ae8e0 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e5e00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 0x7f54e00c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.472+0000 7f54e4a00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 0x7f54e015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54e5400640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f54d4007ef0 tx=0x7f54d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4014620 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d4019070 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e015fe10 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4014a00 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.473+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01603d0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.474+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e00693b0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.475+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54d4014ba0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.477+0000 7f54dd600640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 0x7f54c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.477+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f54d408c6f0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.477+0000 7f54e4a00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 0x7f54c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.478+0000 7f54e4a00640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 0x7f54c006edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54cc034000 tx=0x7f54cc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.481+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54e0148c60 con 0x7f54e00ae8e0 2024-04-26T23:50:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.769+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f54e0159170 con 0x7f54e00ae8e0 2024-04-26T23:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.770+0000 7f54dd600640 1 -- 172.21.15.46:0/651409005 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1599 (secure 0 0 0) 0x7f54d405a8f0 con 0x7f54e00ae8e0 2024-04-26T23:50:52.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:52.774 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":844,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076975739,"bytes_used":2592260096,"bytes_avail":770501853184,"bytes_total":773094113280,"write_bytes_sec":21919617,"read_op_per_sec":0,"write_op_per_sec":23},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-26T23:50:52.198351+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 msgr2=0x7f54c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 0x7f54c006edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54cc034000 tx=0x7f54cc02e000 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 msgr2=0x7f54e0148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f54d4007ef0 tx=0x7f54d400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 shutdown_connections 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f54c006c8f0 0x7f54c006edb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54e0155dc0 0x7f54e00c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54e01631d0 0x7f54e015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 --2- 172.21.15.46:0/651409005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54e00ae8e0 0x7f54e0148720 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 >> 172.21.15.46:0/651409005 conn(0x7f54e00b2820 msgr2=0x7f54e00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 shutdown_connections 2024-04-26T23:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.773+0000 7f54e6800640 1 -- 172.21.15.46:0/651409005 wait complete. 2024-04-26T23:50:52.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-26T23:50:52.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:52.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub resume 2024-04-26T23:50:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.998+0000 7f5f82e00640 1 Processor -- start 2024-04-26T23:50:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:52.999+0000 7f5f82e00640 1 -- start start 2024-04-26T23:50:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.000+0000 7f5f82e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f7c0d0f90 0x7f5f7c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.000+0000 7f5f82e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7c06ec30 con 0x7f5f7c073c00 2024-04-26T23:50:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.000+0000 7f5f82e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7c0cf3f0 con 0x7f5f7c145c10 2024-04-26T23:50:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.000+0000 7f5f82e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7c145ff0 con 0x7f5f7c0d0f90 2024-04-26T23:50:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.001+0000 7f5f81000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5f7c073c00 0x7f5f7c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52892/0 (socket says 172.21.15.46:52892) 2024-04-26T23:50:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.001+0000 7f5f81000640 1 -- 172.21.15.46:0/2218704296 learned_addr learned my addr 172.21.15.46:0/2218704296 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.001+0000 7f5f82400640 1 --2- 172.21.15.46:0/2218704296 >> v2:172.21.15.142:3300/0 conn(0x7f5f7c0d0f90 0x7f5f7c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.001+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298973963 0 0) 0x7f5f7c06ec30 con 0x7f5f7c073c00 2024-04-26T23:50:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.001+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f64003520 con 0x7f5f7c073c00 2024-04-26T23:50:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3093542081 0 0) 0x7f5f64003520 con 0x7f5f7c073c00 2024-04-26T23:50:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 >> v2:172.21.15.142:3300/0 conn(0x7f5f7c0d0f90 msgr2=0x7f5f7c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 --2- 172.21.15.46:0/2218704296 >> v2:172.21.15.142:3300/0 conn(0x7f5f7c0d0f90 0x7f5f7c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 >> v1:172.21.15.142:6789/0 conn(0x7f5f7c145c10 legacy=0x7f5f7c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c0cf6d0 con 0x7f5f7c073c00 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5f68001670 con 0x7f5f7c073c00 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 >> v1:172.21.15.46:6789/0 conn(0x7f5f7c073c00 legacy=0x7f5f7c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f64005210 0x7f5f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c145ff0 con 0x7f5f64003a00 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6400c160 con 0x7f5f64004800 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6400c360 con 0x7f5f64005210 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.002+0000 7f5f81a00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81000640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f82400640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f64005210 0x7f5f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81a00640 1 -- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f64005210 msgr2=0x7f5f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81a00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f64005210 0x7f5f6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81a00640 1 -- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64004800 msgr2=0x7f5f64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81a00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f81a00640 1 -- 172.21.15.46:0/2218704296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f64003520 con 0x7f5f64003a00 2024-04-26T23:50:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.003+0000 7f5f82400640 1 -- 172.21.15.46:0/2218704296 reap_dead start 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.004+0000 7f5f81a00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001720 tx=0x7f5f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef831ec554bcb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.004+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70001370 con 0x7f5f64003a00 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.004+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f700052d0 con 0x7f5f64003a00 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.004+0000 7f5f7be00640 1 -- 172.21.15.46:0/2218704296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7002abc0 con 0x7f5f64003a00 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 -- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 msgr2=0x7f5f64003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001720 tx=0x7f5f64005830 comp rx=0 tx=0).stop 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 -- 172.21.15.46:0/2218704296 shutdown_connections 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 --2- 172.21.15.46:0/2218704296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 -- 172.21.15.46:0/2218704296 >> 172.21.15.46:0/2218704296 conn(0x7f5f7c06c350 msgr2=0x7f5f7c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 -- 172.21.15.46:0/2218704296 shutdown_connections 2024-04-26T23:50:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.005+0000 7f5f82e00640 1 -- 172.21.15.46:0/2218704296 wait complete. 2024-04-26T23:50:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.007+0000 7f5f82e00640 1 Processor -- start 2024-04-26T23:50:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.007+0000 7f5f82e00640 1 -- start start 2024-04-26T23:50:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 0x7f5f7c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f7c15b960 0x7f5f7c15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c0cf0a0 con 0x7f5f64003a00 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c074090 con 0x7f5f7c0b3b80 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f82e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c1468b0 con 0x7f5f7c15b960 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f81a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f81a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59518/0 (socket says 172.21.15.46:59518) 2024-04-26T23:50:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.008+0000 7f5f81a00640 1 -- 172.21.15.46:0/1838481931 learned_addr learned my addr 172.21.15.46:0/1838481931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81000640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 0x7f5f7c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 -- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f7c15b960 msgr2=0x7f5f7c15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f7c15b960 0x7f5f7c15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 -- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 msgr2=0x7f5f7c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 0x7f5f7c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 -- 172.21.15.46:0/1838481931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c144bf0 con 0x7f5f64003a00 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81000640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 0x7f5f7c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001810 tx=0x7f5f70001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f79600640 1 -- 172.21.15.46:0/1838481931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7003f020 con 0x7f5f64003a00 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.009+0000 7f5f79600640 1 -- 172.21.15.46:0/1838481931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7002be50 con 0x7f5f64003a00 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.010+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c0b4600 con 0x7f5f64003a00 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.010+0000 7f5f79600640 1 -- 172.21.15.46:0/1838481931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70001370 con 0x7f5f64003a00 2024-04-26T23:50:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.010+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c0b4bc0 con 0x7f5f64003a00 2024-04-26T23:50:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.012+0000 7f5f79600640 1 -- 172.21.15.46:0/1838481931 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f70042020 con 0x7f5f64003a00 2024-04-26T23:50:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.014+0000 7f5f79600640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 0x7f5f5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.014+0000 7f5f79600640 1 -- 172.21.15.46:0/1838481931 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f5f700abae0 con 0x7f5f64003a00 2024-04-26T23:50:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.014+0000 7f5f871d9740 1 Processor -- start 2024-04-26T23:50:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.014+0000 7f5f81000640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 0x7f5f5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.014+0000 7f5f871d9740 1 -- start start 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555b70f9e030 con 0x555b7108e3f0 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555b70fb97f0 con 0x555b710b8710 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555b70f747e0 con 0x555b710d5fb0 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.015+0000 7f5f81000640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 0x7f5f5c06ede0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5f68001eb0 tx=0x7f5f68001b60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59534/0 (socket says 172.21.15.46:59534) 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 -- 172.21.15.46:0/838488696 learned_addr learned my addr 172.21.15.46:0/838488696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f81000640 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f81a00640 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 -- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 msgr2=0x555b710d6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 -- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 msgr2=0x555b710b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.016+0000 7f5f82400640 1 -- 172.21.15.46:0/838488696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b70f80050 con 0x555b7108e3f0 2024-04-26T23:50:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.017+0000 7f5f82400640 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c070000 tx=0x7f5f740075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15f70753c25b7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.017+0000 7f5f6e000640 1 -- 172.21.15.46:0/838488696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74019020 con 0x555b7108e3f0 2024-04-26T23:50:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.017+0000 7f5f6e000640 1 -- 172.21.15.46:0/838488696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f74015660 con 0x555b7108e3f0 2024-04-26T23:50:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.017+0000 7f5f6e000640 1 -- 172.21.15.46:0/838488696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74015a50 con 0x555b7108e3f0 2024-04-26T23:50:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 -- 172.21.15.46:0/838488696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 msgr2=0x555b710d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c070000 tx=0x7f5f740075c0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 -- 172.21.15.46:0/838488696 shutdown_connections 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b7108e3f0 0x555b710d4670 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 --2- 172.21.15.46:0/838488696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 -- 172.21.15.46:0/838488696 >> 172.21.15.46:0/838488696 conn(0x555b7108ad20 msgr2=0x555b710d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 -- 172.21.15.46:0/838488696 shutdown_connections 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.018+0000 7f5f871d9740 1 -- 172.21.15.46:0/838488696 wait complete. 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.019+0000 7f5f871d9740 1 -- start start 2024-04-26T23:50:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.019+0000 7f5f871d9740 1 Processor -- start 2024-04-26T23:50:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b710d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555b70ff6200 con 0x555b710d5fb0 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555b70ff0ee0 con 0x555b710b8710 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555b70ffde70 con 0x555b710b4a40 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f871d9740 10 client.0 _ll_register_callbacks cb 0x555b70fe7270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59548/0 (socket says 172.21.15.46:59548) 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f81000640 1 -- 172.21.15.46:0/1164726302 learned_addr learned my addr 172.21.15.46:0/1164726302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.020+0000 7f5f81000640 1 -- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 msgr2=0x555b710d6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81000640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b710d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81000640 1 -- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 msgr2=0x555b710bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81000640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81000640 1 -- 172.21.15.46:0/1164726302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b70dcd7d0 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f81000640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800fea0 tx=0x7f5f68009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38585b6b80532cab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f62a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68028060 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f5f68023c20 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f74015660 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f5f68010040 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f871d9740 10 client.9423 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.021+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x555b70d033f0 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f74015660 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f5f68013070 con 0x555b710d5fb0 2024-04-26T23:50:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 10 client.9423 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 10 client.9423 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 20 client.9423 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b710e33b0 0x555b710d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 4 client.9423 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b70fff7e0 con 0x555b710e33b0 2024-04-26T23:50:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.022+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b710e33b0 0x555b710d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.023+0000 7f5f81a00640 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b710e33b0 0x555b710d44a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4da0d9c5c1755f server_cookie=52300aa5d75760e3 in_seq=0 out_seq=0 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.023+0000 7f5f62a00640 10 client.9423 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.023+0000 7f5f62a00640 1 -- 172.21.15.46:0/1164726302 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b70fff7e0 con 0x555b710e33b0 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.023+0000 7f5f62a00640 10 client.9423 handle_command_reply: tid=0 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 client.9423 shutdown 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 msgr2=0x555b710d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800fea0 tx=0x7f5f68009070 comp rx=0 tx=0).stop 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 shutdown_connections 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b710e33b0 0x555b710d44a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b710d6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710d5fb0 0x555b710d6aa0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 --2- 172.21.15.46:0/1164726302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b8710 0x555b710bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 >> 172.21.15.46:0/1164726302 conn(0x555b7108e3f0 msgr2=0x555b7108d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f62a00640 0 client.9423 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 shutdown_connections 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 1 -- 172.21.15.46:0/1164726302 wait complete. 2024-04-26T23:50:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.024+0000 7f5f871d9740 20 client.9423 trim_cache size 0 max 16384 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.043+0000 7f5f871d9740 1 Processor -- start 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.043+0000 7f5f871d9740 1 -- start start 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b4a40 0x555b710bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c2024e0 con 0x555b710b8710 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c201500 con 0x555b710b4a40 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.044+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c200890 con 0x555b710d5fb0 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59554/0 (socket says 172.21.15.46:59554) 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 -- 172.21.15.46:0/864367156 learned_addr learned my addr 172.21.15.46:0/864367156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81a00640 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b4a40 0x555b710bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 -- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 msgr2=0x555b710d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 -- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b4a40 msgr2=0x555b710bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b4a40 0x555b710bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 -- 172.21.15.46:0/864367156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b70cc30f0 con 0x555b710b8710 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.045+0000 7f5f81000640 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5f68017fc0 tx=0x7f5f68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4303608a198ad90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f6d600640 1 -- 172.21.15.46:0/864367156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68023460 con 0x555b710b8710 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f6d600640 1 -- 172.21.15.46:0/864367156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68023600 con 0x555b710b8710 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f6d600640 1 -- 172.21.15.46:0/864367156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680237b0 con 0x555b710b8710 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 -- 172.21.15.46:0/864367156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 msgr2=0x555b710d3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5f68017fc0 tx=0x7f5f68013000 comp rx=0 tx=0).stop 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 -- 172.21.15.46:0/864367156 shutdown_connections 2024-04-26T23:50:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710d5fb0 0x555b710d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b710d3520 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 --2- 172.21.15.46:0/864367156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710b4a40 0x555b710bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 -- 172.21.15.46:0/864367156 >> 172.21.15.46:0/864367156 conn(0x555b7108e3f0 msgr2=0x555b7108d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.046+0000 7f5f871d9740 1 -- 172.21.15.46:0/864367156 shutdown_connections 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.047+0000 7f5f871d9740 1 -- 172.21.15.46:0/864367156 wait complete. 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.047+0000 7f5f871d9740 1 -- start start 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.047+0000 7f5f871d9740 1 Processor -- start 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b71087dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710d5fb0 0x555b710ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555b70ffefd0 con 0x555b710b8710 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c200ee0 con 0x555b710d5fb0 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x555b71089f60 con 0x555b710b4a40 2024-04-26T23:50:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.048+0000 7f5f871d9740 10 client.0 _ll_register_callbacks cb 0x7f5f68005da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59570/0 (socket says 172.21.15.46:59570) 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 -- 172.21.15.46:0/3968831807 learned_addr learned my addr 172.21.15.46:0/3968831807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 -- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 msgr2=0x555b71087dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b71087dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 -- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710d5fb0 msgr2=0x555b710ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710d5fb0 0x555b710ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 -- 172.21.15.46:0/3968831807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b70fb9a30 con 0x555b710b8710 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f81000640 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800fea0 tx=0x7f5f7c0cf5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ef357967840478 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.049+0000 7f5f62a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6803b060 con 0x555b710b8710 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f5f68014430 con 0x555b710b8710 2024-04-26T23:50:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f68023600 con 0x555b710b8710 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f5f68014630 con 0x555b710b8710 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 10 client.9429 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x555b710b30c0 con 0x555b710b8710 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68023600 con 0x555b710b8710 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f5f68013040 con 0x555b710b8710 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 10 client.9429 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 10 client.9429 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 20 client.9429 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b7108f860 0x555b710aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 4 client.9429 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-26T23:50:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.050+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x555b70ff9650 con 0x555b7108f860 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.051+0000 7f5f81a00640 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b7108f860 0x555b710aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.051+0000 7f5f81a00640 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b7108f860 0x555b710aedb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762824cbb270c86a server_cookie=6797ffd539dba513 in_seq=0 out_seq=0 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.051+0000 7f5f62a00640 10 client.9429 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f62a00640 1 -- 172.21.15.46:0/3968831807 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555b70ff9650 con 0x555b7108f860 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f62a00640 10 client.9429 handle_command_reply: tid=0 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 client.9429 shutdown 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 msgr2=0x555b71088300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800fea0 tx=0x7f5f7c0cf5c0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 shutdown_connections 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x555b7108f860 0x555b710aedb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b710d5fb0 0x555b710ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b710b8710 0x555b71088300 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 --2- 172.21.15.46:0/3968831807 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555b710b4a40 0x555b71087dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 >> 172.21.15.46:0/3968831807 conn(0x555b7108e3f0 msgr2=0x555b7108e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f62a00640 0 client.9429 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 shutdown_connections 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 1 -- 172.21.15.46:0/3968831807 wait complete. 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.052+0000 7f5f871d9740 20 client.9429 trim_cache size 0 max 16384 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:50:53.061 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 msgr2=0x7f5f5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 0x7f5f5c06ede0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5f68001eb0 tx=0x7f5f68001b60 comp rx=0 tx=0).stop 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 msgr2=0x7f5f7c1446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001810 tx=0x7f5f70001120 comp rx=0 tx=0).stop 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 shutdown_connections 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f5c06c920 0x7f5f5c06ede0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f7c15b960 0x7f5f7c15bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7c0b3b80 0x7f5f7c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 --2- 172.21.15.46:0/1838481931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f64003a00 0x7f5f7c1446b0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 >> 172.21.15.46:0/1838481931 conn(0x7f5f7c06c350 msgr2=0x7f5f7c15e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 shutdown_connections 2024-04-26T23:50:53.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.053+0000 7f5f82e00640 1 -- 172.21.15.46:0/1838481931 wait complete. 2024-04-26T23:50:53.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:53.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-26T23:50:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.279+0000 7f2cb7000640 1 Processor -- start 2024-04-26T23:50:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.280+0000 7f2cb7000640 1 -- start start 2024-04-26T23:50:53.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb7000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2cb00d3830 0x7f2cb00d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb7000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00d20c0 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb7000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00d3400 con 0x7f2cb0162a70 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb7000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb00d2370 con 0x7f2cb00d3830 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2cb00d3830 0x7f2cb00d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2cb00d3830 0x7f2cb00d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59578/0 (socket says 172.21.15.46:59578) 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb5200640 1 -- 172.21.15.46:0/622732641 learned_addr learned my addr 172.21.15.46:0/622732641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.281+0000 7f2cb6600640 1 --1- 172.21.15.46:0/622732641 >> v1:172.21.15.46:6789/0 conn(0x7f2cb00d1cb0 0x7f2cb00b29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52904/0 (socket says 172.21.15.46:52904) 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.282+0000 7f2cb5200640 1 -- 172.21.15.46:0/622732641 >> v1:172.21.15.142:6789/0 conn(0x7f2cb0162a70 legacy=0x7f2cb00d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.282+0000 7f2cb5200640 1 -- 172.21.15.46:0/622732641 >> v1:172.21.15.46:6789/0 conn(0x7f2cb00d1cb0 legacy=0x7f2cb00b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.282+0000 7f2cb5200640 1 -- 172.21.15.46:0/622732641 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00d2650 con 0x7f2cb00d3830 2024-04-26T23:50:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.283+0000 7f2cb5200640 1 --2- 172.21.15.46:0/622732641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d3830 0x7f2cb00d2e20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009870 tx=0x7f2c9c02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d7488c125c597c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.283+0000 7f2cafe00640 1 -- 172.21.15.46:0/622732641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c037070 con 0x7f2cb00d3830 2024-04-26T23:50:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.283+0000 7f2cafe00640 1 -- 172.21.15.46:0/622732641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c032640 con 0x7f2cb00d3830 2024-04-26T23:50:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.283+0000 7f2cafe00640 1 -- 172.21.15.46:0/622732641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c032a10 con 0x7f2cb00d3830 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 -- 172.21.15.46:0/622732641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d3830 msgr2=0x7f2cb00d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 --2- 172.21.15.46:0/622732641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d3830 0x7f2cb00d2e20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009870 tx=0x7f2c9c02a3c0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 -- 172.21.15.46:0/622732641 shutdown_connections 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 --2- 172.21.15.46:0/622732641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d3830 0x7f2cb00d2e20 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 -- 172.21.15.46:0/622732641 >> 172.21.15.46:0/622732641 conn(0x7f2cb00d4f20 msgr2=0x7f2cb00d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 -- 172.21.15.46:0/622732641 shutdown_connections 2024-04-26T23:50:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.284+0000 7f2cb7000640 1 -- 172.21.15.46:0/622732641 wait complete. 2024-04-26T23:50:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.286+0000 7f2cb7000640 1 Processor -- start 2024-04-26T23:50:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.286+0000 7f2cb7000640 1 -- start start 2024-04-26T23:50:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cb00d3830 0x7f2cb0155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 0x7f2cb00cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0162fd0 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0162e50 con 0x7f2cb00d3830 2024-04-26T23:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00b30f0 con 0x7f2cb0162a70 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59590/0 (socket says 172.21.15.46:59590) 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.287+0000 7f2cb5c00640 1 -- 172.21.15.46:0/2863052736 learned_addr learned my addr 172.21.15.46:0/2863052736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5200640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cb00d3830 0x7f2cb0155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb6600640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 0x7f2cb00cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 -- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 msgr2=0x7f2cb00cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 0x7f2cb00cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 -- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cb00d3830 msgr2=0x7f2cb0155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cb00d3830 0x7f2cb0155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 -- 172.21.15.46:0/2863052736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00cd0e0 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb6600640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 0x7f2cb00cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.288+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c3c0 tx=0x7f2ca000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.289+0000 7f2cad600640 1 -- 172.21.15.46:0/2863052736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00052d0 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.289+0000 7f2cad600640 1 -- 172.21.15.46:0/2863052736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0005490 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.289+0000 7f2cad600640 1 -- 172.21.15.46:0/2863052736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0007260 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.289+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb00cd350 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.289+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb00cd910 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.291+0000 7f2cad600640 1 -- 172.21.15.46:0/2863052736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ca0007400 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.293+0000 7f2cad600640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 0x7f2c9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.293+0000 7f2cad600640 1 -- 172.21.15.46:0/2863052736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2ca008b280 con 0x7f2cb00d1cb0 2024-04-26T23:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.293+0000 7f2cb5200640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 0x7f2c9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.293+0000 7f2cbb3fc740 1 Processor -- start 2024-04-26T23:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 -- start start 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026afcfb0 0x560026afd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cb5200640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 0x7f2c9006edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 0x560026afb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600269c5030 con 0x560026adf710 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5600269e07f0 con 0x560026ab53f0 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.294+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56002699b7e0 con 0x560026afcfb0 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59594/0 (socket says 172.21.15.46:59594) 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 -- 172.21.15.46:0/3184253294 learned_addr learned my addr 172.21.15.46:0/3184253294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb6600640 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 0x560026afb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 -- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026afcfb0 msgr2=0x560026afd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026afcfb0 0x560026afd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 -- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 msgr2=0x560026afb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 0x560026afb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 -- 172.21.15.46:0/3184253294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600269a7050 con 0x560026adf710 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb6600640 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 0x560026afb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.295+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0007810 tx=0x7f2ca000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87346ca668c2f1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2ca6000640 1 -- 172.21.15.46:0/3184253294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0061960 con 0x560026adf710 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2ca6000640 1 -- 172.21.15.46:0/3184253294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0057080 con 0x560026adf710 2024-04-26T23:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2ca6000640 1 -- 172.21.15.46:0/3184253294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0071180 con 0x560026adf710 2024-04-26T23:50:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/3184253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 msgr2=0x560026adfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0007810 tx=0x7f2ca000a000 comp rx=0 tx=0).stop 2024-04-26T23:50:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/3184253294 shutdown_connections 2024-04-26T23:50:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026ab53f0 0x560026afb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.296+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026afcfb0 0x560026afd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.297+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/3184253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026adfaf0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.297+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/3184253294 >> 172.21.15.46:0/3184253294 conn(0x560026ab1d20 msgr2=0x560026afa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.297+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/3184253294 shutdown_connections 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.297+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/3184253294 wait complete. 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.297+0000 7f2cbb3fc740 1 -- start start 2024-04-26T23:50:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.298+0000 7f2cbb3fc740 1 Processor -- start 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026afcfb0 0x560026acf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560026a1d200 con 0x560026adf710 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560026a17ee0 con 0x560026afcfb0 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560026a24e70 con 0x560026b0e4d0 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cbb3fc740 10 client.0 _ll_register_callbacks cb 0x560026a0e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59598/0 (socket says 172.21.15.46:59598) 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 -- 172.21.15.46:0/302890924 learned_addr learned my addr 172.21.15.46:0/302890924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5200640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026afcfb0 0x560026acf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 -- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 msgr2=0x560026acfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 -- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026afcfb0 msgr2=0x560026acf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026afcfb0 0x560026acf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.299+0000 7f2cb5c00640 1 -- 172.21.15.46:0/302890924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600267f47d0 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0058a80 tx=0x7f2ca0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87145e1e780f266b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0089290 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2ca0071180 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0057080 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f2ca007e050 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2cbb3fc740 10 client.9444 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56002672a3f0 con 0x560026adf710 2024-04-26T23:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0057080 con 0x560026adf710 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.300+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f2ca0081020 con 0x560026adf710 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 10 client.9444 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 10 client.9444 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 20 client.9444 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026afbc40 0x560026acd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 4 client.9444 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560026a267e0 con 0x560026afbc40 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.301+0000 7f2cb5200640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026afbc40 0x560026acd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.302+0000 7f2cb5200640 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026afbc40 0x560026acd510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481e90b77ce50c1e server_cookie=b10fcebbc55eb928 in_seq=0 out_seq=0 2024-04-26T23:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.302+0000 7f2c9aa00640 10 client.9444 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.302+0000 7f2c9aa00640 1 -- 172.21.15.46:0/302890924 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560026a267e0 con 0x560026afbc40 2024-04-26T23:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.302+0000 7f2c9aa00640 10 client.9444 handle_command_reply: tid=0 2024-04-26T23:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.302+0000 7f2cbb3fc740 1 client.9444 shutdown 2024-04-26T23:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 msgr2=0x560026accd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0058a80 tx=0x7f2ca0000bc0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 shutdown_connections 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026afbc40 0x560026acd510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026afcfb0 0x560026acf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2c9aa00640 0 client.9444 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/302890924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026adf710 0x560026accd70 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 >> 172.21.15.46:0/302890924 conn(0x560026ab53f0 msgr2=0x560026aae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 shutdown_connections 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/302890924 wait complete. 2024-04-26T23:50:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.303+0000 7f2cbb3fc740 20 client.9444 trim_cache size 0 max 16384 2024-04-26T23:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.324+0000 7f2cbb3fc740 1 Processor -- start 2024-04-26T23:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.324+0000 7f2cbb3fc740 1 -- start start 2024-04-26T23:50:53.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026ab59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026afaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0201eb0 con 0x560026afcfb0 2024-04-26T23:50:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0200f10 con 0x560026adf710 2024-04-26T23:50:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.325+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb01f59b0 con 0x560026b0e4d0 2024-04-26T23:50:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59600/0 (socket says 172.21.15.46:59600) 2024-04-26T23:50:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 -- 172.21.15.46:0/946921562 learned_addr learned my addr 172.21.15.46:0/946921562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026ab59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb6600640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026afaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 -- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 msgr2=0x560026afaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026afaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 -- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 msgr2=0x560026ab59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026ab59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 -- 172.21.15.46:0/946921562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600266ea0f0 con 0x560026afcfb0 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026ab59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.326+0000 7f2cb5200640 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d429445871fd2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.327+0000 7f2ca5600640 1 -- 172.21.15.46:0/946921562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c052070 con 0x560026afcfb0 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.327+0000 7f2ca5600640 1 -- 172.21.15.46:0/946921562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c032b30 con 0x560026afcfb0 2024-04-26T23:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.327+0000 7f2ca5600640 1 -- 172.21.15.46:0/946921562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c037040 con 0x560026afcfb0 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.327+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/946921562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 msgr2=0x560026afa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.327+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c008030 comp rx=0 tx=0).stop 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/946921562 shutdown_connections 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026afaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026afa520 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/946921562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026ab59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/946921562 >> 172.21.15.46:0/946921562 conn(0x560026ab53f0 msgr2=0x560026af5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/946921562 shutdown_connections 2024-04-26T23:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/946921562 wait complete. 2024-04-26T23:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.328+0000 7f2cbb3fc740 1 -- start start 2024-04-26T23:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.329+0000 7f2cbb3fc740 1 Processor -- start 2024-04-26T23:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026b0ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560026a25fd0 con 0x560026afcfb0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0200a10 con 0x560026adf710 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560026af3000 con 0x560026b0e4d0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cbb3fc740 10 client.0 _ll_register_callbacks cb 0x7f2ca008b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59606/0 (socket says 172.21.15.46:59606) 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 -- 172.21.15.46:0/365947374 learned_addr learned my addr 172.21.15.46:0/365947374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026b0ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 -- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 msgr2=0x560026acb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 -- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 msgr2=0x560026b0ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026b0ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5200640 1 -- 172.21.15.46:0/365947374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600269e0a30 con 0x560026afcfb0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.330+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026b0ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2cb5200640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6149d0b0e5baa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2c9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c052070 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f2c9c003610 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c9c032b30 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f2c9c037040 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2cbb3fc740 10 client.9450 fetch_fsmap learned FSMap version 109 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x560026ae2810 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.331+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c032b30 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f2c9c003810 con 0x560026afcfb0 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 10 client.9450 fetch_fsmap finished waiting for FSMap version 109 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 10 client.9450 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 20 client.9450 populate_metadata read hostname 'smithi046' 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026b73060 0x560026acb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 4 client.9450 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 --> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560026a20650 con 0x560026b73060 2024-04-26T23:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026b73060 0x560026acb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.332+0000 7f2cb5c00640 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026b73060 0x560026acb8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe1beec3bf7ac00 server_cookie=c6592f7d79184eb9 in_seq=0 out_seq=0 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2c9aa00640 10 client.9450 ms_handle_connect on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2c9aa00640 1 -- 172.21.15.46:0/365947374 <== mds.0 v2:172.21.15.46:6832/2039062774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560026a20650 con 0x560026b73060 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2c9aa00640 10 client.9450 handle_command_reply: tid=0 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 client.9450 shutdown 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 msgr2=0x560026b0cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c004730 comp rx=0 tx=0).stop 2024-04-26T23:50:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 shutdown_connections 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:6832/2039062774,v1:172.21.15.46:6833/2039062774] conn(0x560026b73060 0x560026acb8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560026b0e4d0 0x560026acb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560026afcfb0 0x560026b0cf50 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 --2- 172.21.15.46:0/365947374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560026adf710 0x560026b0ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2c9aa00640 0 client.9450 ms_handle_reset on v2:172.21.15.46:6832/2039062774 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 >> 172.21.15.46:0/365947374 conn(0x560026ab53f0 msgr2=0x560026acf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 shutdown_connections 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 1 -- 172.21.15.46:0/365947374 wait complete. 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.333+0000 7f2cbb3fc740 20 client.9450 trim_cache size 0 max 16384 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:50:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:50:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:50:53.341 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:50:53.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 msgr2=0x7f2c9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 0x7f2c9006edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c02e000 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 msgr2=0x7f2cb00b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c3c0 tx=0x7f2ca000c880 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 shutdown_connections 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c9006c8f0 0x7f2c9006edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cb0162a70 0x7f2cb00cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cb00d3830 0x7f2cb0155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 --2- 172.21.15.46:0/2863052736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00d1cb0 0x7f2cb00b3ea0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 >> 172.21.15.46:0/2863052736 conn(0x7f2cb00d4f20 msgr2=0x7f2cb0160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 shutdown_connections 2024-04-26T23:50:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.334+0000 7f2cb7000640 1 -- 172.21.15.46:0/2863052736 wait complete. 2024-04-26T23:50:53.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:53.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.558+0000 7f898ce00640 1 Processor -- start 2024-04-26T23:50:53.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.559+0000 7f898ce00640 1 -- start start 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f898ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 0x7f8988070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f898ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89880ce070 0x7f89880ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f898ce00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898815f980 con 0x7f898815f5a0 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f898ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f89880cdbe0 con 0x7f89880ce070 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f898ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f89880d20c0 con 0x7f89880d1bc0 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.560+0000 7f8986800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89880ce070 0x7f89880ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89880ce070 0x7f89880ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59614/0 (socket says 172.21.15.46:59614) 2024-04-26T23:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 -- 172.21.15.46:0/3199103254 learned_addr learned my addr 172.21.15.46:0/3199103254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8985400640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 0x7f8988070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 -- 172.21.15.46:0/3199103254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 msgr2=0x7f8988070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 0x7f8988070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 -- 172.21.15.46:0/3199103254 >> v1:172.21.15.142:6789/0 conn(0x7f898815f5a0 legacy=0x7f8988070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8986800640 1 -- 172.21.15.46:0/3199103254 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89880d23a0 con 0x7f89880ce070 2024-04-26T23:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.561+0000 7f8985400640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 0x7f8988070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.562+0000 7f8986800640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89880ce520 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f8980009870 tx=0x7f898002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb4264f5bb01d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.563+0000 7f8984a00640 1 -- 172.21.15.46:0/3199103254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980037070 con 0x7f89880ce070 2024-04-26T23:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.563+0000 7f8984a00640 1 -- 172.21.15.46:0/3199103254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8980032440 con 0x7f89880ce070 2024-04-26T23:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.563+0000 7f8984a00640 1 -- 172.21.15.46:0/3199103254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980032810 con 0x7f89880ce070 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.563+0000 7f898ce00640 1 -- 172.21.15.46:0/3199103254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 msgr2=0x7f89880ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.563+0000 7f898ce00640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89880ce520 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f8980009870 tx=0x7f898002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 -- 172.21.15.46:0/3199103254 shutdown_connections 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89880ce520 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 --2- 172.21.15.46:0/3199103254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89880d1bc0 0x7f8988070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 -- 172.21.15.46:0/3199103254 >> 172.21.15.46:0/3199103254 conn(0x7f89880d2f40 msgr2=0x7f89880d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 -- 172.21.15.46:0/3199103254 shutdown_connections 2024-04-26T23:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.564+0000 7f898ce00640 1 -- 172.21.15.46:0/3199103254 wait complete. 2024-04-26T23:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.566+0000 7f898ce00640 1 Processor -- start 2024-04-26T23:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.566+0000 7f898ce00640 1 -- start start 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89880d1bc0 0x7f89881f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f898815f5a0 0x7f89880b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89880d07d0 con 0x7f89880ce070 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89880cd870 con 0x7f89880d1bc0 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f898ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8988071090 con 0x7f898815f5a0 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f8985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f8985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59628/0 (socket says 172.21.15.46:59628) 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f8985e00640 1 -- 172.21.15.46:0/4046211525 learned_addr learned my addr 172.21.15.46:0/4046211525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.567+0000 7f8985400640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89880d1bc0 0x7f89881f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8986800640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f898815f5a0 0x7f89880b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 -- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f898815f5a0 msgr2=0x7f89880b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f898815f5a0 0x7f89880b74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 -- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89880d1bc0 msgr2=0x7f89881f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89880d1bc0 0x7f89881f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 -- 172.21.15.46:0/4046211525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898815fff0 con 0x7f89880ce070 2024-04-26T23:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f8985e00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f897800c7c0 tx=0x7f897800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978016020 con 0x7f89880ce070 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897800aa50 con 0x7f89880ce070 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897800ae40 con 0x7f89880ce070 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.568+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89880b3ef0 con 0x7f89880ce070 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.569+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89880b44b0 con 0x7f89880ce070 2024-04-26T23:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.570+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8988069530 con 0x7f89880ce070 2024-04-26T23:50:53.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.571+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8978019020 con 0x7f89880ce070 2024-04-26T23:50:53.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.572+0000 7f897e000640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 0x7f896806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.572+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f897808be20 con 0x7f89880ce070 2024-04-26T23:50:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.573+0000 7f8985400640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 0x7f896806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.574+0000 7f8985400640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 0x7f896806edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89881f8c00 tx=0x7f8974008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.577+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8978020030 con 0x7f89880ce070 2024-04-26T23:50:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.823+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89881f5c70 con 0x7f89880ce070 2024-04-26T23:50:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.824+0000 7f897e000640 1 -- 172.21.15.46:0/4046211525 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f897805a020 con 0x7f89880ce070 2024-04-26T23:50:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:53.828 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 msgr2=0x7f896806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 0x7f896806edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89881f8c00 tx=0x7f8974008040 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 msgr2=0x7f89881628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f897800c7c0 tx=0x7f897800cc80 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 shutdown_connections 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f896806c8f0 0x7f896806edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f898815f5a0 0x7f89880b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89880d1bc0 0x7f89881f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 --2- 172.21.15.46:0/4046211525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89880ce070 0x7f89881628a0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 >> 172.21.15.46:0/4046211525 conn(0x7f89880d2f40 msgr2=0x7f89880cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 shutdown_connections 2024-04-26T23:50:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:53.827+0000 7f898ce00640 1 -- 172.21.15.46:0/4046211525 wait complete. 2024-04-26T23:50:53.850 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:53.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.067+0000 7fd633e00640 1 Processor -- start 2024-04-26T23:50:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.067+0000 7fd633e00640 1 -- start start 2024-04-26T23:50:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd633e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd634159950 0x7fd6340b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd633e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6341633a0 0x7fd63406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd633e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6340af0c0 con 0x7fd6340ae8e0 2024-04-26T23:50:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd633e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd6340af3a0 con 0x7fd634159950 2024-04-26T23:50:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd633e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd6340af680 con 0x7fd6341633a0 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd62be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6340ae8e0 0x7fd634162c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52910/0 (socket says 172.21.15.46:52910) 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd62be00640 1 -- 172.21.15.46:0/1726718755 learned_addr learned my addr 172.21.15.46:0/1726718755 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.069+0000 7fd632a00640 1 --2- 172.21.15.46:0/1726718755 >> v2:172.21.15.46:3300/0 conn(0x7fd634159950 0x7fd6340b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.070+0000 7fd633400640 1 --2- 172.21.15.46:0/1726718755 >> v2:172.21.15.142:3300/0 conn(0x7fd6341633a0 0x7fd63406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.070+0000 7fd632a00640 1 -- 172.21.15.46:0/1726718755 >> v2:172.21.15.142:3300/0 conn(0x7fd6341633a0 msgr2=0x7fd63406e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.070+0000 7fd632a00640 1 --2- 172.21.15.46:0/1726718755 >> v2:172.21.15.142:3300/0 conn(0x7fd6341633a0 0x7fd63406e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.070+0000 7fd632a00640 1 -- 172.21.15.46:0/1726718755 >> v1:172.21.15.46:6789/0 conn(0x7fd6340ae8e0 legacy=0x7fd634162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.070+0000 7fd632a00640 1 -- 172.21.15.46:0/1726718755 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6340af960 con 0x7fd634159950 2024-04-26T23:50:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.071+0000 7fd632a00640 1 --2- 172.21.15.46:0/1726718755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd634159950 0x7fd6340b48a0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd634071b20 tx=0x7fd62402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97a6bbd4287e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.071+0000 7fd632000640 1 -- 172.21.15.46:0/1726718755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624037070 con 0x7fd634159950 2024-04-26T23:50:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.071+0000 7fd632000640 1 -- 172.21.15.46:0/1726718755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd624032440 con 0x7fd634159950 2024-04-26T23:50:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.071+0000 7fd632000640 1 -- 172.21.15.46:0/1726718755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624032810 con 0x7fd634159950 2024-04-26T23:50:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 -- 172.21.15.46:0/1726718755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd634159950 msgr2=0x7fd6340b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 --2- 172.21.15.46:0/1726718755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd634159950 0x7fd6340b48a0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd634071b20 tx=0x7fd62402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 -- 172.21.15.46:0/1726718755 shutdown_connections 2024-04-26T23:50:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 --2- 172.21.15.46:0/1726718755 >> v2:172.21.15.142:3300/0 conn(0x7fd6341633a0 0x7fd63406e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 --2- 172.21.15.46:0/1726718755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd634159950 0x7fd6340b48a0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 -- 172.21.15.46:0/1726718755 >> 172.21.15.46:0/1726718755 conn(0x7fd6340b2820 msgr2=0x7fd6340b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 -- 172.21.15.46:0/1726718755 shutdown_connections 2024-04-26T23:50:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.072+0000 7fd633e00640 1 -- 172.21.15.46:0/1726718755 wait complete. 2024-04-26T23:50:54.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.074+0000 7fd633e00640 1 Processor -- start 2024-04-26T23:50:54.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.074+0000 7fd633e00640 1 -- start start 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6340ae8e0 0x7fd6340d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6341f7bf0 0x7fd634072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6340b0160 con 0x7fd6341633a0 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6340aed70 con 0x7fd6340ae8e0 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd63406e960 con 0x7fd6341f7bf0 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd62be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd62be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59636/0 (socket says 172.21.15.46:59636) 2024-04-26T23:50:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd62be00640 1 -- 172.21.15.46:0/1036962523 learned_addr learned my addr 172.21.15.46:0/1036962523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd633400640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6341f7bf0 0x7fd634072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.075+0000 7fd632a00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6340ae8e0 0x7fd6340d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 -- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6341f7bf0 msgr2=0x7fd634072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6341f7bf0 0x7fd634072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 -- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6340ae8e0 msgr2=0x7fd6340d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6340ae8e0 0x7fd6340d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 -- 172.21.15.46:0/1036962523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6340ce9a0 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62be00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c400 tx=0x7fd61c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c017070 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c005ce0 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0072f0 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.076+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6340cec40 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.077+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6340cf200 con 0x7fd6341633a0 2024-04-26T23:50:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.078+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6340693b0 con 0x7fd6341633a0 2024-04-26T23:50:54.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.079+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd61c01a070 con 0x7fd6341633a0 2024-04-26T23:50:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.080+0000 7fd62aa00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 0x7fd61006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.080+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fd61c08b0e0 con 0x7fd6341633a0 2024-04-26T23:50:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.080+0000 7fd632a00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 0x7fd61006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:54.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.082+0000 7fd632a00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 0x7fd61006edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd624034000 tx=0x7fd62402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.084+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd61c058c90 con 0x7fd6341633a0 2024-04-26T23:50:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.370+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd6340b35e0 con 0x7fd6341633a0 2024-04-26T23:50:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.371+0000 7fd62aa00640 1 -- 172.21.15.46:0/1036962523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fd61c05cf90 con 0x7fd6341633a0 2024-04-26T23:50:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:54.375 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":845,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076975739,"bytes_used":2592260096,"bytes_avail":770501853184,"bytes_total":773094113280,"write_bytes_sec":9314816,"read_op_per_sec":0,"write_op_per_sec":10},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-26T23:50:52.198351+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 msgr2=0x7fd61006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 0x7fd61006edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd624034000 tx=0x7fd62402e000 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 msgr2=0x7fd6340ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c400 tx=0x7fd61c00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 shutdown_connections 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd61006c8f0 0x7fd61006edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6341f7bf0 0x7fd634072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6341633a0 0x7fd6340ce460 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 --2- 172.21.15.46:0/1036962523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6340ae8e0 0x7fd6340d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 >> 172.21.15.46:0/1036962523 conn(0x7fd6340b2820 msgr2=0x7fd63406ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 shutdown_connections 2024-04-26T23:50:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:54.374+0000 7fd633e00640 1 -- 172.21.15.46:0/1036962523 wait complete. 2024-04-26T23:50:54.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-26T23:50:55.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.602+0000 7f66e7e00640 1 Processor -- start 2024-04-26T23:50:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.602+0000 7f66e7e00640 1 -- start start 2024-04-26T23:50:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66e80720e0 0x7f66e80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e8073ad0 con 0x7f66e81466b0 2024-04-26T23:50:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e8146a90 con 0x7f66e8160910 2024-04-26T23:50:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f66e8073d70 con 0x7f66e80720e0 2024-04-26T23:50:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66e81466b0 0x7f66e806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52912/0 (socket says 172.21.15.46:52912) 2024-04-26T23:50:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.604+0000 7f66e6a00640 1 -- 172.21.15.46:0/2411145883 learned_addr learned my addr 172.21.15.46:0/2411145883 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e6000640 1 --2- 172.21.15.46:0/2411145883 >> v2:172.21.15.142:3300/0 conn(0x7f66e80720e0 0x7f66e80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757986820 0 0) 0x7f66e8073ad0 con 0x7f66e81466b0 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66cc003520 con 0x7f66e81466b0 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3265134136 0 0) 0x7f66cc003520 con 0x7f66e81466b0 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 >> v2:172.21.15.142:3300/0 conn(0x7f66e80720e0 msgr2=0x7f66e80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 --2- 172.21.15.46:0/2411145883 >> v2:172.21.15.142:3300/0 conn(0x7f66e80720e0 0x7f66e80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 >> v1:172.21.15.142:6789/0 conn(0x7f66e8160910 legacy=0x7f66e80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e8074050 con 0x7f66e81466b0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f66d8001670 con 0x7f66e81466b0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.605+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 >> v1:172.21.15.46:6789/0 conn(0x7f66e81466b0 legacy=0x7f66e806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66cc0039e0 0x7f66cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66cc004800 0x7f66cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8073d70 con 0x7f66cc0051c0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc00c090 con 0x7f66cc0039e0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66cc00c290 con 0x7f66cc004800 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e6a00640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66cc0039e0 0x7f66cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e6000640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66cc004800 0x7f66cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 -- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66cc004800 msgr2=0x7f66cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66cc004800 0x7f66cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 -- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66cc0039e0 msgr2=0x7f66cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66cc0039e0 0x7f66cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e7400640 1 -- 172.21.15.46:0/2411145883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc003520 con 0x7f66cc0051c0 2024-04-26T23:50:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.606+0000 7f66e6000640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66cc004800 0x7f66cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.607+0000 7f66e7400640 1 -- 172.21.15.46:0/2411145883 reap_dead start 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e7400640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66cc00b9c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001e50 tx=0x7f66cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432c9a55bddfa8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0001a70 con 0x7f66cc0051c0 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0003970 con 0x7f66cc0051c0 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e5600640 1 -- 172.21.15.46:0/2411145883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0003d60 con 0x7f66cc0051c0 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e7e00640 1 -- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 msgr2=0x7f66cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.608+0000 7f66e7e00640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66cc00b9c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001e50 tx=0x7f66cc005830 comp rx=0 tx=0).stop 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.609+0000 7f66e7e00640 1 -- 172.21.15.46:0/2411145883 shutdown_connections 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.609+0000 7f66e7e00640 1 --2- 172.21.15.46:0/2411145883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66cc00b9c0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.609+0000 7f66e7e00640 1 -- 172.21.15.46:0/2411145883 >> 172.21.15.46:0/2411145883 conn(0x7f66e80b26d0 msgr2=0x7f66e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.609+0000 7f66e7e00640 1 -- 172.21.15.46:0/2411145883 shutdown_connections 2024-04-26T23:50:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.609+0000 7f66e7e00640 1 -- 172.21.15.46:0/2411145883 wait complete. 2024-04-26T23:50:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.610+0000 7f66e7e00640 1 Processor -- start 2024-04-26T23:50:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.611+0000 7f66e7e00640 1 -- start start 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.611+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.611+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e815ebc0 0x7f66e813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.611+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66e813e0b0 0x7f66e815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8160d60 con 0x7f66cc0051c0 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66e80cb470 con 0x7f66e815ebc0 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66e8160ee0 con 0x7f66e813e0b0 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59666/0 (socket says 172.21.15.46:59666) 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 -- 172.21.15.46:0/964999816 learned_addr learned my addr 172.21.15.46:0/964999816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 -- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66e813e0b0 msgr2=0x7f66e815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66e813e0b0 0x7f66e815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 -- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e815ebc0 msgr2=0x7f66e813db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6000640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e815ebc0 0x7f66e813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e815ebc0 0x7f66e813db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.612+0000 7f66e6a00640 1 -- 172.21.15.46:0/964999816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e815de20 con 0x7f66cc0051c0 2024-04-26T23:50:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66e6a00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e6c0 tx=0x7f66d800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d800c660 con 0x7f66cc0051c0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d8018070 con 0x7f66cc0051c0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d800ca60 con 0x7f66cc0051c0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e815e100 con 0x7f66cc0051c0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.613+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e815e6c0 con 0x7f66cc0051c0 2024-04-26T23:50:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.614+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e813e500 con 0x7f66cc0051c0 2024-04-26T23:50:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.615+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66d800cc00 con 0x7f66cc0051c0 2024-04-26T23:50:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.616+0000 7f66dea00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 0x7f66c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.616+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f66d808bb40 con 0x7f66cc0051c0 2024-04-26T23:50:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.617+0000 7f66e6000640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 0x7f66c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.618+0000 7f66e6000640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 0x7f66c806edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f66e81409f0 tx=0x7f66d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.621+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66d8059dd0 con 0x7f66cc0051c0 2024-04-26T23:50:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.868+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66e80693b0 con 0x7f66cc0051c0 2024-04-26T23:50:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.869+0000 7f66dea00640 1 -- 172.21.15.46:0/964999816 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f66d80596d0 con 0x7f66cc0051c0 2024-04-26T23:50:55.873 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:55.873 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:50:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 msgr2=0x7f66c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 0x7f66c806edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f66e81409f0 tx=0x7f66d4008040 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 msgr2=0x7f66e80ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e6c0 tx=0x7f66d800eb80 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 shutdown_connections 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66c806c8f0 0x7f66c806edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66e813e0b0 0x7f66e815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e815ebc0 0x7f66e813db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 --2- 172.21.15.46:0/964999816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66cc0051c0 0x7f66e80ca660 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 >> 172.21.15.46:0/964999816 conn(0x7f66e80b26d0 msgr2=0x7f66e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 shutdown_connections 2024-04-26T23:50:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:55.872+0000 7f66e7e00640 1 -- 172.21.15.46:0/964999816 wait complete. 2024-04-26T23:50:55.895 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:50:55.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:50:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.091+0000 7f4eed400640 1 Processor -- start 2024-04-26T23:50:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.091+0000 7f4eed400640 1 -- start start 2024-04-26T23:50:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4eed400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee815f4b0 0x7f4ee80d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4eed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ee80d55e0 0x7f4ee80d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4eed400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee80cfd60 con 0x7f4ee80b5140 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4eed400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee815f890 con 0x7f4ee80d55e0 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4eed400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee80d2910 con 0x7f4ee815f4b0 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4ee5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4ee80b5140 0x7f4ee80d4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52914/0 (socket says 172.21.15.46:52914) 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4ee5e00640 1 -- 172.21.15.46:0/118677969 learned_addr learned my addr 172.21.15.46:0/118677969 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4ee7200640 1 --2- 172.21.15.46:0/118677969 >> v2:172.21.15.46:3300/0 conn(0x7f4ee80d55e0 0x7f4ee80d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.093+0000 7f4ee6800640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee815f4b0 0x7f4ee80d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee5400640 1 -- 172.21.15.46:0/118677969 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417750118 0 0) 0x7f4ed4001110 con 0x7f4ee80b5140 2024-04-26T23:50:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee5400640 1 -- 172.21.15.46:0/118677969 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ecc003520 con 0x7f4ee80b5140 2024-04-26T23:50:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee7200640 1 -- 172.21.15.46:0/118677969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee815f4b0 msgr2=0x7f4ee80d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee7200640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee815f4b0 0x7f4ee80d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee7200640 1 -- 172.21.15.46:0/118677969 >> v1:172.21.15.46:6789/0 conn(0x7f4ee80b5140 legacy=0x7f4ee80d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.094+0000 7f4ee7200640 1 -- 172.21.15.46:0/118677969 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee80b55d0 con 0x7f4ee80d55e0 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.095+0000 7f4ee7200640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee80d55e0 0x7f4ee80d2f50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009d90 tx=0x7f4ee002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f01b599d69c1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.095+0000 7f4ee5400640 1 -- 172.21.15.46:0/118677969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0007740 con 0x7f4ee80d55e0 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.095+0000 7f4ee5400640 1 -- 172.21.15.46:0/118677969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee00078e0 con 0x7f4ee80d55e0 2024-04-26T23:50:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.095+0000 7f4ee5400640 1 -- 172.21.15.46:0/118677969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0007cb0 con 0x7f4ee80d55e0 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 -- 172.21.15.46:0/118677969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee80d55e0 msgr2=0x7f4ee80d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee80d55e0 0x7f4ee80d2f50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009d90 tx=0x7f4ee002a660 comp rx=0 tx=0).stop 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 -- 172.21.15.46:0/118677969 shutdown_connections 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee80d55e0 0x7f4ee80d2f50 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 --2- 172.21.15.46:0/118677969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee815f4b0 0x7f4ee80d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 -- 172.21.15.46:0/118677969 >> 172.21.15.46:0/118677969 conn(0x7f4ee808eb50 msgr2=0x7f4ee809e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 -- 172.21.15.46:0/118677969 shutdown_connections 2024-04-26T23:50:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.096+0000 7f4eed400640 1 -- 172.21.15.46:0/118677969 wait complete. 2024-04-26T23:50:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.098+0000 7f4eed400640 1 Processor -- start 2024-04-26T23:50:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.098+0000 7f4eed400640 1 -- start start 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ee80d55e0 0x7f4ee814ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 0x7f4ee81f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80b5e70 con 0x7f4ee815f4b0 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80b5b90 con 0x7f4ee80d55e0 2024-04-26T23:50:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4eed400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee80b4680 con 0x7f4ee8152350 2024-04-26T23:50:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59690/0 (socket says 172.21.15.46:59690) 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.099+0000 7f4ee5e00640 1 -- 172.21.15.46:0/3205090412 learned_addr learned my addr 172.21.15.46:0/3205090412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee6800640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ee80d55e0 0x7f4ee814ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee7200640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 0x7f4ee81f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 -- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 msgr2=0x7f4ee81f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 0x7f4ee81f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 -- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ee80d55e0 msgr2=0x7f4ee814ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ee80d55e0 0x7f4ee814ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 -- 172.21.15.46:0/3205090412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee814c010 con 0x7f4ee815f4b0 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee7200640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 0x7f4ee81f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.100+0000 7f4ee5e00640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c3a0 tx=0x7f4ed400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.101+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4017070 con 0x7f4ee815f4b0 2024-04-26T23:50:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.101+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed4005ce0 con 0x7f4ee815f4b0 2024-04-26T23:50:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.101+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee8162880 con 0x7f4ee815f4b0 2024-04-26T23:50:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.101+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4007310 con 0x7f4ee815f4b0 2024-04-26T23:50:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.101+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81625a0 con 0x7f4ee815f4b0 2024-04-26T23:50:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.102+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8069530 con 0x7f4ee815f4b0 2024-04-26T23:50:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.103+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ed401a070 con 0x7f4ee815f4b0 2024-04-26T23:50:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.104+0000 7f4edea00640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 0x7f4ec806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.104+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f4ed408b0f0 con 0x7f4ee815f4b0 2024-04-26T23:50:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.105+0000 7f4ee6800640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 0x7f4ec806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.106+0000 7f4ee6800640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 0x7f4ec806ee00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ad0 tx=0x7f4ed8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.109+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ee80cfd60 con 0x7f4ee815f4b0 2024-04-26T23:50:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.394+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4ee80b4bd0 con 0x7f4ee815f4b0 2024-04-26T23:50:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.395+0000 7f4edea00640 1 -- 172.21.15.46:0/3205090412 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1575 (secure 0 0 0) 0x7f4ed4059400 con 0x7f4ee815f4b0 2024-04-26T23:50:56.399 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:56.399 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":847,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077282244,"bytes_used":2592866304,"bytes_avail":770501246976,"bytes_total":773094113280,"write_bytes_sec":9341019,"read_op_per_sec":0,"write_op_per_sec":10},"fsmap":{"epoch":109,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-26T23:50:56.199965+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:50:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 msgr2=0x7f4ec806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 0x7f4ec806ee00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ad0 tx=0x7f4ed8008040 comp rx=0 tx=0).stop 2024-04-26T23:50:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 msgr2=0x7f4ee806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c3a0 tx=0x7f4ed400c860 comp rx=0 tx=0).stop 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 shutdown_connections 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4ec806c940 0x7f4ec806ee00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ee8152350 0x7f4ee81f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee815f4b0 0x7f4ee806e360 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 --2- 172.21.15.46:0/3205090412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ee80d55e0 0x7f4ee814ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 >> 172.21.15.46:0/3205090412 conn(0x7f4ee808eb50 msgr2=0x7f4ee815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.398+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 shutdown_connections 2024-04-26T23:50:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.399+0000 7f4eed400640 1 -- 172.21.15.46:0/3205090412 wait complete. 2024-04-26T23:50:56.420 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:50:56.420 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:50:56.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:50:56.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:50:56.638 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:50:56.638 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:50:56.638 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:50:56.638 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:50:56.699 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:50:56.699 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:50:56.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:50:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.975+0000 7f035b000640 1 Processor -- start 2024-04-26T23:50:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.975+0000 7f035b000640 1 -- start start 2024-04-26T23:50:56.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035b000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f035415f4e0 0x7f03540d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03540d2130 0x7f03540b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03540d11f0 con 0x7f03540b4000 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f03540d0ce0 con 0x7f03540d2130 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035b000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f03540d2990 con 0x7f035415f4e0 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f0359200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03540b4000 0x7f03540d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52918/0 (socket says 172.21.15.46:52918) 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f0359200640 1 -- 172.21.15.46:0/2502799086 learned_addr learned my addr 172.21.15.46:0/2502799086 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.976+0000 7f035a600640 1 --2- 172.21.15.46:0/2502799086 >> v2:172.21.15.46:3300/0 conn(0x7f03540d2130 0x7f03540b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f0359c00640 1 --2- 172.21.15.46:0/2502799086 >> v2:172.21.15.142:3300/0 conn(0x7f035415f4e0 0x7f03540d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f0353e00640 1 -- 172.21.15.46:0/2502799086 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126439652 0 0) 0x7f03540d11f0 con 0x7f03540b4000 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f0353e00640 1 -- 172.21.15.46:0/2502799086 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0338003520 con 0x7f03540b4000 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f035a600640 1 -- 172.21.15.46:0/2502799086 >> v2:172.21.15.142:3300/0 conn(0x7f035415f4e0 msgr2=0x7f03540d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f035a600640 1 --2- 172.21.15.46:0/2502799086 >> v2:172.21.15.142:3300/0 conn(0x7f035415f4e0 0x7f03540d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f035a600640 1 -- 172.21.15.46:0/2502799086 >> v1:172.21.15.46:6789/0 conn(0x7f03540b4000 legacy=0x7f03540d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.977+0000 7f035a600640 1 -- 172.21.15.46:0/2502799086 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03540d5f20 con 0x7f03540d2130 2024-04-26T23:50:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.978+0000 7f035a600640 1 --2- 172.21.15.46:0/2502799086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540d2130 0x7f03540b2280 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f035408d8b0 tx=0x7f034c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161d877c00ddf62e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.978+0000 7f0353e00640 1 -- 172.21.15.46:0/2502799086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c037070 con 0x7f03540d2130 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f0353e00640 1 -- 172.21.15.46:0/2502799086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c032440 con 0x7f03540d2130 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f0353e00640 1 -- 172.21.15.46:0/2502799086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c0327f0 con 0x7f03540d2130 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 -- 172.21.15.46:0/2502799086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540d2130 msgr2=0x7f03540b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 --2- 172.21.15.46:0/2502799086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540d2130 0x7f03540b2280 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f035408d8b0 tx=0x7f034c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 -- 172.21.15.46:0/2502799086 shutdown_connections 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 --2- 172.21.15.46:0/2502799086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540d2130 0x7f03540b2280 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 --2- 172.21.15.46:0/2502799086 >> v2:172.21.15.142:3300/0 conn(0x7f035415f4e0 0x7f03540d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 -- 172.21.15.46:0/2502799086 >> 172.21.15.46:0/2502799086 conn(0x7f03540d5010 msgr2=0x7f03540d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 -- 172.21.15.46:0/2502799086 shutdown_connections 2024-04-26T23:50:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.979+0000 7f035b000640 1 -- 172.21.15.46:0/2502799086 wait complete. 2024-04-26T23:50:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.981+0000 7f035b000640 1 Processor -- start 2024-04-26T23:50:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.981+0000 7f035b000640 1 -- start start 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03540b4000 0x7f0354150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03540d2130 0x7f03540ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03540b1d60 con 0x7f035415f4e0 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03540d5c40 con 0x7f03540d2130 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f035415f8c0 con 0x7f03540b4000 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59708/0 (socket says 172.21.15.46:59708) 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.982+0000 7f035a600640 1 -- 172.21.15.46:0/548153869 learned_addr learned my addr 172.21.15.46:0/548153869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f0359c00640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03540b4000 0x7f0354150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f0359200640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03540d2130 0x7f03540ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 -- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03540b4000 msgr2=0x7f0354150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03540b4000 0x7f0354150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 -- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03540d2130 msgr2=0x7f03540ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03540d2130 0x7f03540ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 -- 172.21.15.46:0/548153869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035415e050 con 0x7f035415f4e0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.983+0000 7f035a600640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f034c035040 tx=0x7f034c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.984+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c037040 con 0x7f035415f4e0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.984+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c042070 con 0x7f035415f4e0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.984+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c030840 con 0x7f035415f4e0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.984+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03540cc260 con 0x7f035415f4e0 2024-04-26T23:50:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.984+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03540cc820 con 0x7f035415f4e0 2024-04-26T23:50:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.985+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03540693b0 con 0x7f035415f4e0 2024-04-26T23:50:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.986+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f034c0309e0 con 0x7f035415f4e0 2024-04-26T23:50:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.987+0000 7f0351600640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 0x7f033406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.987+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f034c0ac780 con 0x7f035415f4e0 2024-04-26T23:50:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.988+0000 7f0359c00640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 0x7f033406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.989+0000 7f0359c00640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 0x7f033406edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f03440099c0 tx=0x7f0344003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:56.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:56.991+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f034c045360 con 0x7f035415f4e0 2024-04-26T23:50:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.197+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03540cc9f0 con 0x7f035415f4e0 2024-04-26T23:50:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.200+0000 7f0351600640 1 -- 172.21.15.46:0/548153869 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16864 (secure 0 0 0) 0x7f034c07a980 con 0x7f035415f4e0 2024-04-26T23:50:57.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:50:57.204 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":137,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:01.836702+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:57.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 msgr2=0x7f033406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 0x7f033406edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f03440099c0 tx=0x7f0344003990 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 msgr2=0x7f035415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f034c035040 tx=0x7f034c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 shutdown_connections 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f033406c8f0 0x7f033406edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035415f4e0 0x7f035415d690 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03540d2130 0x7f03540ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 --2- 172.21.15.46:0/548153869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03540b4000 0x7f0354150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 >> 172.21.15.46:0/548153869 conn(0x7f03540d5010 msgr2=0x7f03540b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 shutdown_connections 2024-04-26T23:50:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.203+0000 7f035b000640 1 -- 172.21.15.46:0/548153869 wait complete. 2024-04-26T23:50:57.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:49:58.107194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:49:59.279006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:50:57.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:50:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.436+0000 7fa63dc00640 1 Processor -- start 2024-04-26T23:50:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.436+0000 7fa63dc00640 1 -- start start 2024-04-26T23:50:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.437+0000 7fa63dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa6380cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa63dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6380af570 con 0x7fa63815f210 2024-04-26T23:50:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa63dc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6380af850 con 0x7fa6380ae8f0 2024-04-26T23:50:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa63dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380cb680 con 0x7fa6380cdd20 2024-04-26T23:50:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa637e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa63815f210 0x7fa6380cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-04-26T23:50:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa637e00640 1 -- 172.21.15.46:0/4164176460 learned_addr learned my addr 172.21.15.46:0/4164176460 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.438+0000 7fa62fe00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa6380cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734408572 0 0) 0x7fa6380af570 con 0x7fa63815f210 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa61c003520 con 0x7fa63815f210 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2669644685 0 0) 0x7fa61c003520 con 0x7fa63815f210 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 msgr2=0x7fa6380cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa6380cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 >> v1:172.21.15.142:6789/0 conn(0x7fa6380ae8f0 legacy=0x7fa6380aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380cb960 con 0x7fa63815f210 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa628001670 con 0x7fa63815f210 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa62fe00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa6380cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 >> v1:172.21.15.46:6789/0 conn(0x7fa63815f210 legacy=0x7fa6380cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61c0039e0 0x7fa61c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.439+0000 7fa637400640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa61c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa637400640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa61c004d40 0x7fa61c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380cb680 con 0x7fa61c0039e0 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c00b9c0 con 0x7fa61c004d40 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c00bb70 con 0x7fa6380cdd20 2024-04-26T23:50:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa637e00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61c0039e0 0x7fa61c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa62fe00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa61c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.440+0000 7fa63d200640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa61c004d40 0x7fa61c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.441+0000 7fa637e00640 1 -- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 msgr2=0x7fa61c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.441+0000 7fa637e00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6380cdd20 0x7fa61c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.441+0000 7fa637e00640 1 -- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa61c004d40 msgr2=0x7fa61c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.441+0000 7fa637e00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa61c004d40 0x7fa61c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.441+0000 7fa637e00640 1 -- 172.21.15.46:0/4164176460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c003520 con 0x7fa61c0039e0 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.442+0000 7fa637e00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61c0039e0 0x7fa61c003dc0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa62800c290 tx=0x7fa61c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa07c7d441855388 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.442+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628007d50 con 0x7fa61c0039e0 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.442+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628005af0 con 0x7fa61c0039e0 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.442+0000 7fa637400640 1 -- 172.21.15.46:0/4164176460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628007070 con 0x7fa61c0039e0 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 -- 172.21.15.46:0/4164176460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61c0039e0 msgr2=0x7fa61c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 --2- 172.21.15.46:0/4164176460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61c0039e0 0x7fa61c003dc0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa62800c290 tx=0x7fa61c005830 comp rx=0 tx=0).stop 2024-04-26T23:50:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63d200640 1 -- 172.21.15.46:0/4164176460 reap_dead start 2024-04-26T23:50:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 -- 172.21.15.46:0/4164176460 shutdown_connections 2024-04-26T23:50:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 -- 172.21.15.46:0/4164176460 >> 172.21.15.46:0/4164176460 conn(0x7fa6380b27b0 msgr2=0x7fa6380b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 -- 172.21.15.46:0/4164176460 shutdown_connections 2024-04-26T23:50:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.443+0000 7fa63dc00640 1 -- 172.21.15.46:0/4164176460 wait complete. 2024-04-26T23:50:57.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.445+0000 7fa63dc00640 1 Processor -- start 2024-04-26T23:50:57.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.445+0000 7fa63dc00640 1 -- start start 2024-04-26T23:50:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa61c0039e0 0x7fa6380c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6380c6500 0x7fa6381f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380cc160 con 0x7fa6381f4e60 2024-04-26T23:50:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa63815f5f0 con 0x7fa6380c6500 2024-04-26T23:50:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380cc2e0 con 0x7fa61c0039e0 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.446+0000 7fa63d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59726/0 (socket says 172.21.15.46:59726) 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 -- 172.21.15.46:0/1006003600 learned_addr learned my addr 172.21.15.46:0/1006003600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa637e00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa61c0039e0 0x7fa6380c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa62fe00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6380c6500 0x7fa6381f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 -- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa61c0039e0 msgr2=0x7fa6380c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa61c0039e0 0x7fa6380c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 -- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6380c6500 msgr2=0x7fa6381f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6380c6500 0x7fa6381f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 -- 172.21.15.46:0/1006003600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6381e4760 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.447+0000 7fa63d200640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa6380cfcb0 tx=0x7fa63000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.448+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630019070 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.448+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa630006d60 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.448+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630003b90 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.448+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6380c6b30 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.448+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381f5d90 con 0x7fa6381f4e60 2024-04-26T23:50:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.449+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6380693b0 con 0x7fa6381f4e60 2024-04-26T23:50:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.450+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6300050d0 con 0x7fa6381f4e60 2024-04-26T23:50:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.451+0000 7fa634c00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.451+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fa63008af60 con 0x7fa6381f4e60 2024-04-26T23:50:57.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.452+0000 7fa637e00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.453+0000 7fa637e00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa628033000 tx=0x7fa62802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.456+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa630059160 con 0x7fa6381f4e60 2024-04-26T23:50:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.660+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa638071d80 con 0x7fa6381f4e60 2024-04-26T23:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.661+0000 7fa634c00640 1 -- 172.21.15.46:0/1006003600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fa630058a60 con 0x7fa6381f4e60 2024-04-26T23:50:57.665 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 msgr2=0x7fa61406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa628033000 tx=0x7fa62802e000 comp rx=0 tx=0).stop 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 msgr2=0x7fa6381f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa6380cfcb0 tx=0x7fa63000fd40 comp rx=0 tx=0).stop 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 shutdown_connections 2024-04-26T23:50:57.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.664+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6381f4e60 0x7fa6381f52e0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6380c6500 0x7fa6381f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 --2- 172.21.15.46:0/1006003600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa61c0039e0 0x7fa6380c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 >> 172.21.15.46:0/1006003600 conn(0x7fa6380b27b0 msgr2=0x7fa6380cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 shutdown_connections 2024-04-26T23:50:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.665+0000 7fa63dc00640 1 -- 172.21.15.46:0/1006003600 wait complete. 2024-04-26T23:50:57.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:50:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.900+0000 7fb58ce00640 1 Processor -- start 2024-04-26T23:50:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.900+0000 7fb58ce00640 1 -- start start 2024-04-26T23:50:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.901+0000 7fb58ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb5880b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.901+0000 7fb58ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb588162aa0 0x7fb5880d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.901+0000 7fb58ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5880d07d0 con 0x7fb5881626c0 2024-04-26T23:50:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.901+0000 7fb58ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb5880d0ab0 con 0x7fb588162aa0 2024-04-26T23:50:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.901+0000 7fb58ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5880d0d90 con 0x7fb5880b5fb0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb587400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5881626c0 0x7fb5880d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52932/0 (socket says 172.21.15.46:52932) 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb587400640 1 -- 172.21.15.46:0/2985383599 learned_addr learned my addr 172.21.15.46:0/2985383599 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb587e00640 1 --2- 172.21.15.46:0/2985383599 >> v2:172.21.15.142:3300/0 conn(0x7fb588162aa0 0x7fb5880d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb5880b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667515357 0 0) 0x7fb5880d07d0 con 0x7fb5881626c0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.902+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb56c003520 con 0x7fb5881626c0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1431010870 0 0) 0x7fb56c003520 con 0x7fb5881626c0 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 msgr2=0x7fb5880b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb5880b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 >> v2:172.21.15.142:3300/0 conn(0x7fb588162aa0 msgr2=0x7fb5880d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 --2- 172.21.15.46:0/2985383599 >> v2:172.21.15.142:3300/0 conn(0x7fb588162aa0 0x7fb5880d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880d2c80 con 0x7fb5881626c0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb578001670 con 0x7fb5881626c0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb5880b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 >> v1:172.21.15.46:6789/0 conn(0x7fb5881626c0 legacy=0x7fb5880d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c0039b0 0x7fb56c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb56c004800 0x7fb56c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb56c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880d0ab0 con 0x7fb56c004800 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880d0d90 con 0x7fb56c0039b0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb56c00ba10 con 0x7fb5880b5fb0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb56c004800 0x7fb56c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb587400640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c0039b0 0x7fb56c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb587e00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb56c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.903+0000 7fb586a00640 1 -- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 msgr2=0x7fb56c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb56c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb586a00640 1 -- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c0039b0 msgr2=0x7fb56c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c0039b0 0x7fb56c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb586a00640 1 -- 172.21.15.46:0/2985383599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c003520 con 0x7fb56c004800 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb587400640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c0039b0 0x7fb56c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.904+0000 7fb587e00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5880b5fb0 0x7fb56c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb586a00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb56c004800 0x7fb56c004c80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098e0 tx=0x7fb56c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f614a33db20640 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570037070 con 0x7fb56c004800 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57002ace0 con 0x7fb56c004800 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb586000640 1 -- 172.21.15.46:0/2985383599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000f040 con 0x7fb56c004800 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb58ce00640 1 -- 172.21.15.46:0/2985383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb56c004800 msgr2=0x7fb56c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb58ce00640 1 --2- 172.21.15.46:0/2985383599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb56c004800 0x7fb56c004c80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098e0 tx=0x7fb56c005830 comp rx=0 tx=0).stop 2024-04-26T23:50:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.905+0000 7fb587e00640 1 -- 172.21.15.46:0/2985383599 reap_dead start 2024-04-26T23:50:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.906+0000 7fb58ce00640 1 -- 172.21.15.46:0/2985383599 shutdown_connections 2024-04-26T23:50:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.906+0000 7fb58ce00640 1 -- 172.21.15.46:0/2985383599 >> 172.21.15.46:0/2985383599 conn(0x7fb58809e2a0 msgr2=0x7fb5880d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.906+0000 7fb58ce00640 1 -- 172.21.15.46:0/2985383599 shutdown_connections 2024-04-26T23:50:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.906+0000 7fb58ce00640 1 -- 172.21.15.46:0/2985383599 wait complete. 2024-04-26T23:50:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.908+0000 7fb58ce00640 1 Processor -- start 2024-04-26T23:50:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.908+0000 7fb58ce00640 1 -- start start 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 0x7fb58814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 0x7fb588150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880b4b50 con 0x7fb58806e5b0 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880b49d0 con 0x7fb58806e150 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb58ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5880d1e40 con 0x7fb56c004800 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb587e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb587e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59740/0 (socket says 172.21.15.46:59740) 2024-04-26T23:50:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb587e00640 1 -- 172.21.15.46:0/4051237214 learned_addr learned my addr 172.21.15.46:0/4051237214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.909+0000 7fb586a00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 0x7fb588150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587400640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 0x7fb58814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 -- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 msgr2=0x7fb58814f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 0x7fb58814f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 -- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 msgr2=0x7fb588150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 0x7fb588150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 -- 172.21.15.46:0/4051237214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58814f780 con 0x7fb58806e5b0 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb586a00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 0x7fb588150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587400640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 0x7fb58814f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:57.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb587e00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb58808f2b0 tx=0x7fb58000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580016620 con 0x7fb58806e5b0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58001b070 con 0x7fb58806e5b0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580016a20 con 0x7fb58806e5b0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58814fa60 con 0x7fb58806e5b0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.910+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb588150020 con 0x7fb58806e5b0 2024-04-26T23:50:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.912+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880693b0 con 0x7fb58806e5b0 2024-04-26T23:50:57.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.913+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb580005d10 con 0x7fb58806e5b0 2024-04-26T23:50:57.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.914+0000 7fb57f400640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:57.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.914+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fb58008c8e0 con 0x7fb58806e5b0 2024-04-26T23:50:57.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.914+0000 7fb587400640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.916+0000 7fb587400640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb578007910 tx=0x7fb578009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:57.919+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb580025020 con 0x7fb58806e5b0 2024-04-26T23:50:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.123+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb5880d1280 con 0x7fb58806e5b0 2024-04-26T23:50:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.124+0000 7fb57f400640 1 -- 172.21.15.46:0/4051237214 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) ==== 104+0+11 (secure 0 0 0) 0x7fb58005ab80 con 0x7fb58806e5b0 2024-04-26T23:50:58.127 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 msgr2=0x7fb56406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb578007910 tx=0x7fb578009040 comp rx=0 tx=0).stop 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 msgr2=0x7fb5880b8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb58808f2b0 tx=0x7fb58000f520 comp rx=0 tx=0).stop 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 shutdown_connections 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e5b0 0x7fb5880b8870 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb58806e150 0x7fb588150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.126+0000 7fb58ce00640 1 --2- 172.21.15.46:0/4051237214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb56c004800 0x7fb58814f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.127+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 >> 172.21.15.46:0/4051237214 conn(0x7fb58809e2a0 msgr2=0x7fb58806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.127+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 shutdown_connections 2024-04-26T23:50:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.127+0000 7fb58ce00640 1 -- 172.21.15.46:0/4051237214 wait complete. 2024-04-26T23:50:58.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:50:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.377+0000 7f9a15800640 1 Processor -- start 2024-04-26T23:50:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.378+0000 7f9a15800640 1 -- start start 2024-04-26T23:50:58.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a15800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 0x7f9a100d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a15800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a100d0c70 0x7f9a100d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a15800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a100d0270 con 0x7f9a100d19b0 2024-04-26T23:50:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a15800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9a100d1de0 con 0x7f9a100d0c70 2024-04-26T23:50:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a15800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9a100b5e80 con 0x7f9a100cfe90 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.379+0000 7f9a07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a100d0c70 0x7f9a100d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a0f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a100d19b0 0x7f9a1015cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52946/0 (socket says 172.21.15.46:52946) 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a100d0c70 0x7f9a100d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59756/0 (socket says 172.21.15.46:59756) 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a0f400640 1 -- 172.21.15.46:0/2595387463 learned_addr learned my addr 172.21.15.46:0/2595387463 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a0ea00640 1 --2- 172.21.15.46:0/2595387463 >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 0x7f9a100d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a07e00640 1 -- 172.21.15.46:0/2595387463 >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 msgr2=0x7f9a100d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a07e00640 1 --2- 172.21.15.46:0/2595387463 >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 0x7f9a100d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a07e00640 1 -- 172.21.15.46:0/2595387463 >> v1:172.21.15.46:6789/0 conn(0x7f9a100d19b0 legacy=0x7f9a1015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a07e00640 1 -- 172.21.15.46:0/2595387463 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a100b6160 con 0x7f9a100d0c70 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a0e000640 1 -- 172.21.15.46:0/2595387463 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094143107 0 0) 0x7f9a100d0270 con 0x7f9a100d19b0 2024-04-26T23:50:58.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.380+0000 7f9a0ea00640 1 --2- 172.21.15.46:0/2595387463 >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 0x7f9a100d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:50:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.382+0000 7f9a07e00640 1 --2- 172.21.15.46:0/2595387463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100d0c70 0x7f9a100d1470 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9a100732a0 tx=0x7f99f802e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f965fdda4866029e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.382+0000 7f9a0e000640 1 -- 172.21.15.46:0/2595387463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f803a070 con 0x7f9a100d0c70 2024-04-26T23:50:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.382+0000 7f9a0e000640 1 -- 172.21.15.46:0/2595387463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8033710 con 0x7f9a100d0c70 2024-04-26T23:50:58.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.382+0000 7f9a0e000640 1 -- 172.21.15.46:0/2595387463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8033ae0 con 0x7f9a100d0c70 2024-04-26T23:50:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 -- 172.21.15.46:0/2595387463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100d0c70 msgr2=0x7f9a100d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 --2- 172.21.15.46:0/2595387463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100d0c70 0x7f9a100d1470 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9a100732a0 tx=0x7f99f802e5b0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 -- 172.21.15.46:0/2595387463 shutdown_connections 2024-04-26T23:50:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 --2- 172.21.15.46:0/2595387463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100d0c70 0x7f9a100d1470 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 --2- 172.21.15.46:0/2595387463 >> v2:172.21.15.142:3300/0 conn(0x7f9a100cfe90 0x7f9a100d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 -- 172.21.15.46:0/2595387463 >> 172.21.15.46:0/2595387463 conn(0x7f9a100d2e90 msgr2=0x7f9a100d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 -- 172.21.15.46:0/2595387463 shutdown_connections 2024-04-26T23:50:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.383+0000 7f9a15800640 1 -- 172.21.15.46:0/2595387463 wait complete. 2024-04-26T23:50:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.386+0000 7f9a15800640 1 Processor -- start 2024-04-26T23:50:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.386+0000 7f9a15800640 1 -- start start 2024-04-26T23:50:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 0x7f9a100ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 0x7f9a1015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a100b6a00 con 0x7f9a100cfe90 2024-04-26T23:50:58.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a100b6720 con 0x7f9a100d19b0 2024-04-26T23:50:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a15800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1015d1f0 con 0x7f9a100d0c70 2024-04-26T23:50:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a0ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a0ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59770/0 (socket says 172.21.15.46:59770) 2024-04-26T23:50:58.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a0ea00640 1 -- 172.21.15.46:0/287335499 learned_addr learned my addr 172.21.15.46:0/287335499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a0f400640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 0x7f9a1015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.388+0000 7f9a07e00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 0x7f9a100ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 -- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 msgr2=0x7f9a100ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 0x7f9a100ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 -- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 msgr2=0x7f9a1015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 0x7f9a1015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 -- 172.21.15.46:0/287335499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10157830 con 0x7f9a100cfe90 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0f400640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 0x7f9a1015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a07e00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 0x7f9a100ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a0ea00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e7e0 tx=0x7f9a0000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000f040 con 0x7f9a100cfe90 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00004030 con 0x7f9a100cfe90 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a100c6c60 con 0x7f9a100cfe90 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.389+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00004400 con 0x7f9a100cfe90 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.390+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a100cb930 con 0x7f9a100cfe90 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.391+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a10069530 con 0x7f9a100cfe90 2024-04-26T23:50:58.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.392+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a0000c6d0 con 0x7f9a100cfe90 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.394+0000 7f9a06a00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 0x7f99ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.394+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f9a0008ba80 con 0x7f9a100cfe90 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.394+0000 7f9a07e00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 0x7f99ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.396+0000 7f9a07e00640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 0x7f99ec06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99f8037000 tx=0x7f99f8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.398+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a0008de20 con 0x7f9a100cfe90 2024-04-26T23:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.601+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9a100d1250 con 0x7f9a100cfe90 2024-04-26T23:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.602+0000 7f9a06a00640 1 -- 172.21.15.46:0/287335499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) ==== 100+0+11 (secure 0 0 0) 0x7f9a10157830 con 0x7f9a100cfe90 2024-04-26T23:50:58.606 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:50:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 msgr2=0x7f99ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 0x7f99ec06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99f8037000 tx=0x7f99f8008030 comp rx=0 tx=0).stop 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 msgr2=0x7f9a10156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e7e0 tx=0x7f9a0000eca0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 shutdown_connections 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f99ec06c8f0 0x7f99ec06edb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a100d19b0 0x7f9a1015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a100d0c70 0x7f9a100ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 --2- 172.21.15.46:0/287335499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a100cfe90 0x7f9a10156e90 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 >> 172.21.15.46:0/287335499 conn(0x7f9a100d2e90 msgr2=0x7f9a100ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 shutdown_connections 2024-04-26T23:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.605+0000 7f9a15800640 1 -- 172.21.15.46:0/287335499 wait complete. 2024-04-26T23:50:58.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:50:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.835+0000 7ff843200640 1 Processor -- start 2024-04-26T23:50:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.836+0000 7ff843200640 1 -- start start 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.837+0000 7ff843200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff84415f4e0 0x7ff8440d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.837+0000 7ff843200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8440b4000 0x7ff8440d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.837+0000 7ff843200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8440d11f0 con 0x7ff8440d1f60 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.837+0000 7ff843200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff8440d2990 con 0x7ff8440b4000 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.837+0000 7ff843200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff8440d5f20 con 0x7ff84415f4e0 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8440b4000 0x7ff8440d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8440b4000 0x7ff8440d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59786/0 (socket says 172.21.15.46:59786) 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 -- 172.21.15.46:0/3938961581 learned_addr learned my addr 172.21.15.46:0/3938961581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841e00640 1 --2- 172.21.15.46:0/3938961581 >> v2:172.21.15.142:3300/0 conn(0x7ff84415f4e0 0x7ff8440d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 -- 172.21.15.46:0/3938961581 >> v2:172.21.15.142:3300/0 conn(0x7ff84415f4e0 msgr2=0x7ff8440d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:50:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 --2- 172.21.15.46:0/3938961581 >> v2:172.21.15.142:3300/0 conn(0x7ff84415f4e0 0x7ff8440d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 -- 172.21.15.46:0/3938961581 >> v1:172.21.15.142:6789/0 conn(0x7ff8440d1f60 legacy=0x7ff8440cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.838+0000 7ff841400640 1 -- 172.21.15.46:0/3938961581 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8440d0f10 con 0x7ff8440b4000 2024-04-26T23:50:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.840+0000 7ff841400640 1 --2- 172.21.15.46:0/3938961581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440b4000 0x7ff8440d1a20 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff84408d8b0 tx=0x7ff82c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c7a2ab1df5e2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.840+0000 7ff840a00640 1 -- 172.21.15.46:0/3938961581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c037070 con 0x7ff8440b4000 2024-04-26T23:50:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.840+0000 7ff840a00640 1 -- 172.21.15.46:0/3938961581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c032440 con 0x7ff8440b4000 2024-04-26T23:50:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.840+0000 7ff840a00640 1 -- 172.21.15.46:0/3938961581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c032810 con 0x7ff8440b4000 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 -- 172.21.15.46:0/3938961581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440b4000 msgr2=0x7ff8440d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 --2- 172.21.15.46:0/3938961581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440b4000 0x7ff8440d1a20 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff84408d8b0 tx=0x7ff82c02a270 comp rx=0 tx=0).stop 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 -- 172.21.15.46:0/3938961581 shutdown_connections 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 --2- 172.21.15.46:0/3938961581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440b4000 0x7ff8440d1a20 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 --2- 172.21.15.46:0/3938961581 >> v2:172.21.15.142:3300/0 conn(0x7ff84415f4e0 0x7ff8440d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 -- 172.21.15.46:0/3938961581 >> 172.21.15.46:0/3938961581 conn(0x7ff8440d5010 msgr2=0x7ff8440d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 -- 172.21.15.46:0/3938961581 shutdown_connections 2024-04-26T23:50:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.841+0000 7ff843200640 1 -- 172.21.15.46:0/3938961581 wait complete. 2024-04-26T23:50:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.843+0000 7ff843200640 1 Processor -- start 2024-04-26T23:50:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.843+0000 7ff843200640 1 -- start start 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8440b4000 0x7ff8440b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff84415f4e0 0x7ff8441f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff84415f8c0 con 0x7ff8440d1f60 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8440b1d60 con 0x7ff8440b4000 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff843200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8440b3290 con 0x7ff84415f4e0 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff841400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff841400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59790/0 (socket says 172.21.15.46:59790) 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.844+0000 7ff841400640 1 -- 172.21.15.46:0/458925693 learned_addr learned my addr 172.21.15.46:0/458925693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841e00640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8440b4000 0x7ff8440b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff842800640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff84415f4e0 0x7ff8441f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 -- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff84415f4e0 msgr2=0x7ff8441f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff84415f4e0 0x7ff8441f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 -- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8440b4000 msgr2=0x7ff8440b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8440b4000 0x7ff8440b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 -- 172.21.15.46:0/458925693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84409ef50 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff841400640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff82c02a2b0 tx=0x7ff82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.845+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c037040 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.846+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c042070 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.846+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8440ba3a0 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.846+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c030880 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.846+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8440cca30 con 0x7ff8440d1f60 2024-04-26T23:50:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.847+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8440d46d0 con 0x7ff8440d1f60 2024-04-26T23:50:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.848+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff82c030a20 con 0x7ff8440d1f60 2024-04-26T23:50:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.849+0000 7ff83a000640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 0x7ff82006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.849+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7ff82c0ac620 con 0x7ff8440d1f60 2024-04-26T23:50:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.849+0000 7ff841e00640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 0x7ff82006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.850+0000 7ff841e00640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 0x7ff82006edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ceec0 tx=0x7ff83400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:58.854+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff82c07a8b0 con 0x7ff8440d1f60 2024-04-26T23:50:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.076+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8440693b0 con 0x7ff8440d1f60 2024-04-26T23:50:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.077+0000 7ff83a000640 1 -- 172.21.15.46:0/458925693 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v137) ==== 80+0+0 (secure 0 0 0) 0x7ff82c07a1b0 con 0x7ff8440d1f60 2024-04-26T23:50:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 msgr2=0x7ff82006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 0x7ff82006edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ceec0 tx=0x7ff83400b040 comp rx=0 tx=0).stop 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 msgr2=0x7ff8440b9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff82c02a2b0 tx=0x7ff82c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 shutdown_connections 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff82006c8f0 0x7ff82006edb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff84415f4e0 0x7ff8441f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8440d1f60 0x7ff8440b9d00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 --2- 172.21.15.46:0/458925693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8440b4000 0x7ff8440b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 >> 172.21.15.46:0/458925693 conn(0x7ff8440d5010 msgr2=0x7ff84406e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 shutdown_connections 2024-04-26T23:50:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:50:59.080+0000 7ff843200640 1 -- 172.21.15.46:0/458925693 wait complete. 2024-04-26T23:50:59.104 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:50:59.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:59.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:50:59.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:50:59.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:50:59.997 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:51:00.051 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:51:00.051 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:51:00.051 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:00.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:51:00.092 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:51:00.092 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:51:00.092 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:00.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:51:00.135 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:51:00.135 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:51:00.135 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:00.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:51:00.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:00.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.395+0000 7faa62a00640 1 Processor -- start 2024-04-26T23:51:00.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.395+0000 7faa62a00640 1 -- start start 2024-04-26T23:51:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.396+0000 7faa62a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faa5c0d2130 0x7faa5c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.396+0000 7faa62a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.396+0000 7faa62a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa5c0d11f0 con 0x7faa5c15f4e0 2024-04-26T23:51:00.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.396+0000 7faa62a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faa5c0d0ce0 con 0x7faa5c0d2130 2024-04-26T23:51:00.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.396+0000 7faa62a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c0d2990 con 0x7faa5c0b4000 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faa5c0d2130 0x7faa5c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faa5c0d2130 0x7faa5c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59798/0 (socket says 172.21.15.46:59798) 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 -- 172.21.15.46:0/508182149 learned_addr learned my addr 172.21.15.46:0/508182149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa62000640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 -- 172.21.15.46:0/508182149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 msgr2=0x7faa5c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 -- 172.21.15.46:0/508182149 >> v1:172.21.15.142:6789/0 conn(0x7faa5c15f4e0 legacy=0x7faa5c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.397+0000 7faa60c00640 1 -- 172.21.15.46:0/508182149 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c0d5f20 con 0x7faa5c0d2130 2024-04-26T23:51:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.398+0000 7faa60c00640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c0d2df0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596f4c2d4e28affd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.399+0000 7faa5be00640 1 -- 172.21.15.46:0/508182149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48037070 con 0x7faa5c0d2130 2024-04-26T23:51:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.399+0000 7faa5be00640 1 -- 172.21.15.46:0/508182149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa48032440 con 0x7faa5c0d2130 2024-04-26T23:51:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.399+0000 7faa5be00640 1 -- 172.21.15.46:0/508182149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48032810 con 0x7faa5c0d2130 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.399+0000 7faa62a00640 1 -- 172.21.15.46:0/508182149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 msgr2=0x7faa5c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.399+0000 7faa62a00640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c0d2df0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802a210 comp rx=0 tx=0).stop 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 -- 172.21.15.46:0/508182149 shutdown_connections 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 --2- 172.21.15.46:0/508182149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c0d2df0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 -- 172.21.15.46:0/508182149 >> 172.21.15.46:0/508182149 conn(0x7faa5c0d5010 msgr2=0x7faa5c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 -- 172.21.15.46:0/508182149 shutdown_connections 2024-04-26T23:51:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.400+0000 7faa62a00640 1 -- 172.21.15.46:0/508182149 wait complete. 2024-04-26T23:51:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.401+0000 7faa62a00640 1 Processor -- start 2024-04-26T23:51:00.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.402+0000 7faa62a00640 1 -- start start 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0ba820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa5c15f4e0 0x7faa5c0b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c0b1d60 con 0x7faa5c0d2130 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c0d5c40 con 0x7faa5c15f4e0 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c15f8c0 con 0x7faa5c0b4000 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59802/0 (socket says 172.21.15.46:59802) 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 -- 172.21.15.46:0/1495317878 learned_addr learned my addr 172.21.15.46:0/1495317878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa61600640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa62000640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa5c15f4e0 0x7faa5c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 -- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 msgr2=0x7faa5c0ba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0ba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 -- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa5c15f4e0 msgr2=0x7faa5c0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa5c15f4e0 0x7faa5c0b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.403+0000 7faa60c00640 1 -- 172.21.15.46:0/1495317878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c1f56c0 con 0x7faa5c0d2130 2024-04-26T23:51:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa60c00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48041020 con 0x7faa5c0d2130 2024-04-26T23:51:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa480306c0 con 0x7faa5c0d2130 2024-04-26T23:51:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48037040 con 0x7faa5c0d2130 2024-04-26T23:51:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c0b8880 con 0x7faa5c0d2130 2024-04-26T23:51:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.404+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c0b8e40 con 0x7faa5c0d2130 2024-04-26T23:51:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.406+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa5c1f5890 con 0x7faa5c0d2130 2024-04-26T23:51:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.407+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa48044020 con 0x7faa5c0d2130 2024-04-26T23:51:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.408+0000 7faa59600640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 0x7faa3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.408+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7faa480ac500 con 0x7faa5c0d2130 2024-04-26T23:51:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.408+0000 7faa61600640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 0x7faa3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.409+0000 7faa61600640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 0x7faa3c06edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faa5000a750 tx=0x7faa50008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.412+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa4804b030 con 0x7faa5c0d2130 2024-04-26T23:51:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.663+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa5c0693b0 con 0x7faa5c0d2130 2024-04-26T23:51:00.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.664+0000 7faa59600640 1 -- 172.21.15.46:0/1495317878 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7faa4807a700 con 0x7faa5c0d2130 2024-04-26T23:51:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:00.668 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:51:00.668 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:51:00.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.667+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 msgr2=0x7faa3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.667+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 0x7faa3c06edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faa5000a750 tx=0x7faa50008040 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 msgr2=0x7faa5c1f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa480046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 shutdown_connections 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa3c06c8f0 0x7faa3c06edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa5c15f4e0 0x7faa5c0b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa5c0d2130 0x7faa5c1f6880 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 --2- 172.21.15.46:0/1495317878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa5c0b4000 0x7faa5c0ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 >> 172.21.15.46:0/1495317878 conn(0x7faa5c0d5010 msgr2=0x7faa5c15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 shutdown_connections 2024-04-26T23:51:00.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.668+0000 7faa62a00640 1 -- 172.21.15.46:0/1495317878 wait complete. 2024-04-26T23:51:00.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.902+0000 7f75aca00640 1 Processor -- start 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.902+0000 7f75aca00640 1 -- start start 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.903+0000 7f75aca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75a80d1f80 0x7f75a815f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.903+0000 7f75aca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a80d1050 con 0x7f75a8162980 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.903+0000 7f75aca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a815fd30 con 0x7f75a80d1af0 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75aca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f75a8162d60 con 0x7f75a80d1f80 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75a5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75a8162980 0x7f75a80d2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52956/0 (socket says 172.21.15.46:52956) 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75a5e00640 1 -- 172.21.15.46:0/1859859641 learned_addr learned my addr 172.21.15.46:0/1859859641 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75a6800640 1 --2- 172.21.15.46:0/1859859641 >> v2:172.21.15.46:3300/0 conn(0x7f75a80d1f80 0x7f75a815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75a5400640 1 -- 172.21.15.46:0/1859859641 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728217147 0 0) 0x7f75a80d1050 con 0x7f75a8162980 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.904+0000 7f75a5400640 1 -- 172.21.15.46:0/1859859641 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f758c003520 con 0x7f75a8162980 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.905+0000 7f75a6800640 1 -- 172.21.15.46:0/1859859641 >> v1:172.21.15.142:6789/0 conn(0x7f75a80d1af0 legacy=0x7f75a815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.905+0000 7f75a6800640 1 -- 172.21.15.46:0/1859859641 >> v1:172.21.15.46:6789/0 conn(0x7f75a8162980 legacy=0x7f75a80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.905+0000 7f75a6800640 1 -- 172.21.15.46:0/1859859641 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8160010 con 0x7f75a80d1f80 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75a6800640 1 --2- 172.21.15.46:0/1859859641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a815f440 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009d90 tx=0x7f75a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896db256dbfa6aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75a5400640 1 -- 172.21.15.46:0/1859859641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007740 con 0x7f75a80d1f80 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75a5400640 1 -- 172.21.15.46:0/1859859641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a00078e0 con 0x7f75a80d1f80 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75a5400640 1 -- 172.21.15.46:0/1859859641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007cb0 con 0x7f75a80d1f80 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75aca00640 1 -- 172.21.15.46:0/1859859641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 msgr2=0x7f75a815f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.906+0000 7f75aca00640 1 --2- 172.21.15.46:0/1859859641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a815f440 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009d90 tx=0x7f75a002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.907+0000 7f75aca00640 1 -- 172.21.15.46:0/1859859641 shutdown_connections 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.907+0000 7f75aca00640 1 --2- 172.21.15.46:0/1859859641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a815f440 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.907+0000 7f75aca00640 1 -- 172.21.15.46:0/1859859641 >> 172.21.15.46:0/1859859641 conn(0x7f75a809e420 msgr2=0x7f75a80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:00.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.907+0000 7f75aca00640 1 -- 172.21.15.46:0/1859859641 shutdown_connections 2024-04-26T23:51:00.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.907+0000 7f75aca00640 1 -- 172.21.15.46:0/1859859641 wait complete. 2024-04-26T23:51:00.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.908+0000 7f75aca00640 1 Processor -- start 2024-04-26T23:51:00.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.909+0000 7f75aca00640 1 -- start start 2024-04-26T23:51:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.909+0000 7f75aca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a80d1af0 0x7f75a80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.909+0000 7f75aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f75aca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a8162980 0x7f75a814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f75aca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75a81608b0 con 0x7f75a80d1f80 2024-04-26T23:51:00.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f75aca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75a81605d0 con 0x7f75a80d1af0 2024-04-26T23:51:00.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f75aca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75a80d23b0 con 0x7f75a8162980 2024-04-26T23:51:00.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59824/0 (socket says 172.21.15.46:59824) 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 -- 172.21.15.46:0/4143715112 learned_addr learned my addr 172.21.15.46:0/4143715112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 -- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a8162980 msgr2=0x7f75a814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a8162980 0x7f75a814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 -- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a80d1af0 msgr2=0x7f75a80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f75a5e00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a80d1af0 0x7f75a80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a80d1af0 0x7f75a80ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.910+0000 7f759fe00640 1 -- 172.21.15.46:0/4143715112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80b8240 con 0x7f75a80d1f80 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f759fe00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590017070 con 0x7f75a80d1f80 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75900052d0 con 0x7f75a80d1f80 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590003830 con 0x7f75a80d1f80 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a80b84e0 con 0x7f75a80d1f80 2024-04-26T23:51:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.911+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a80b8aa0 con 0x7f75a80d1f80 2024-04-26T23:51:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.912+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8069530 con 0x7f75a80d1f80 2024-04-26T23:51:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.913+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f759001a070 con 0x7f75a80d1f80 2024-04-26T23:51:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.914+0000 7f759e000640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.914+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f759008b0c0 con 0x7f75a80d1f80 2024-04-26T23:51:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.915+0000 7f75a5e00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.916+0000 7f75a5e00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 0x7f758406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f75980011e0 tx=0x7f7598009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:00.919+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75900592c0 con 0x7f75a80d1f80 2024-04-26T23:51:01.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.125+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75a80d57f0 con 0x7f75a80d1f80 2024-04-26T23:51:01.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.127+0000 7f759e000640 1 -- 172.21.15.46:0/4143715112 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16864 (secure 0 0 0) 0x7f7590058bc0 con 0x7f75a80d1f80 2024-04-26T23:51:01.131 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:01.131 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":137,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:01.836702+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 msgr2=0x7f758406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 0x7f758406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f75980011e0 tx=0x7f7598009040 comp rx=0 tx=0).stop 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 msgr2=0x7f75a80b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 shutdown_connections 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a8162980 0x7f75a814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a80d1f80 0x7f75a80b7d00 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 --2- 172.21.15.46:0/4143715112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a80d1af0 0x7f75a80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.129+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 >> 172.21.15.46:0/4143715112 conn(0x7f75a809e420 msgr2=0x7f75a81594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.130+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 shutdown_connections 2024-04-26T23:51:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.130+0000 7f75aca00640 1 -- 172.21.15.46:0/4143715112 wait complete. 2024-04-26T23:51:01.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:49:58.107194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:49:59.279006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:51:01.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.356+0000 7fd8b9800640 1 Processor -- start 2024-04-26T23:51:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.356+0000 7fd8b9800640 1 -- start start 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.357+0000 7fd8b9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 0x7fd8b4160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.357+0000 7fd8b9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b415cc40 0x7fd8b40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.357+0000 7fd8b9800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 0x7fd8b40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b9800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40cca70 con 0x7fd8b415cc40 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b9800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40cbe60 con 0x7fd8b415d0a0 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b40cc140 con 0x7fd8b4160130 2024-04-26T23:51:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b415cc40 0x7fd8b40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b415cc40 0x7fd8b40cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59830/0 (socket says 172.21.15.46:59830) 2024-04-26T23:51:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 -- 172.21.15.46:0/663661200 learned_addr learned my addr 172.21.15.46:0/663661200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 0x7fd8b4160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b8e00640 1 --2- 172.21.15.46:0/663661200 >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 0x7fd8b40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 -- 172.21.15.46:0/663661200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 msgr2=0x7fd8b4160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 0x7fd8b4160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 -- 172.21.15.46:0/663661200 >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 msgr2=0x7fd8b40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 --2- 172.21.15.46:0/663661200 >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 0x7fd8b40cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8ab400640 1 -- 172.21.15.46:0/663661200 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40cc420 con 0x7fd8b415cc40 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 0x7fd8b4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.358+0000 7fd8b8e00640 1 --2- 172.21.15.46:0/663661200 >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 0x7fd8b40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.359+0000 7fd8ab400640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b415cc40 0x7fd8b40cd550 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40d0ad0 tx=0x7fd89c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12aaf15fdeee3a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.360+0000 7fd8b3400640 1 -- 172.21.15.46:0/663661200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c037070 con 0x7fd8b415cc40 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.360+0000 7fd8b3400640 1 -- 172.21.15.46:0/663661200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c032440 con 0x7fd8b415cc40 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.360+0000 7fd8b3400640 1 -- 172.21.15.46:0/663661200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c032810 con 0x7fd8b415cc40 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.360+0000 7fd8b9800640 1 -- 172.21.15.46:0/663661200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b415cc40 msgr2=0x7fd8b40cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.360+0000 7fd8b9800640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b415cc40 0x7fd8b40cd550 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40d0ad0 tx=0x7fd89c02a140 comp rx=0 tx=0).stop 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 -- 172.21.15.46:0/663661200 shutdown_connections 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 --2- 172.21.15.46:0/663661200 >> v2:172.21.15.142:3300/0 conn(0x7fd8b415d0a0 0x7fd8b40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b415cc40 0x7fd8b40cd550 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 --2- 172.21.15.46:0/663661200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b4160130 0x7fd8b4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 -- 172.21.15.46:0/663661200 >> 172.21.15.46:0/663661200 conn(0x7fd8b40b27b0 msgr2=0x7fd8b40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 -- 172.21.15.46:0/663661200 shutdown_connections 2024-04-26T23:51:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.361+0000 7fd8b9800640 1 -- 172.21.15.46:0/663661200 wait complete. 2024-04-26T23:51:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.363+0000 7fd8b9800640 1 Processor -- start 2024-04-26T23:51:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.363+0000 7fd8b9800640 1 -- start start 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b415d0a0 0x7fd8b4149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 0x7fd8b40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b40723a0 con 0x7fd8b4160130 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4072220 con 0x7fd8b40c7490 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b4072520 con 0x7fd8b415d0a0 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59834/0 (socket says 172.21.15.46:59834) 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8ab400640 1 -- 172.21.15.46:0/2739905412 learned_addr learned my addr 172.21.15.46:0/2739905412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b415d0a0 0x7fd8b4149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8b8e00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 0x7fd8b40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8ab400640 1 -- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b415d0a0 msgr2=0x7fd8b4149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.364+0000 7fd8ab400640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b415d0a0 0x7fd8b4149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8ab400640 1 -- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 msgr2=0x7fd8b40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8ab400640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 0x7fd8b40cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8ab400640 1 -- 172.21.15.46:0/2739905412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40c5eb0 con 0x7fd8b4160130 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8b8e00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 0x7fd8b40cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8ab400640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd89c02a6b0 tx=0x7fd89c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c037040 con 0x7fd8b4160130 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c042070 con 0x7fd8b4160130 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b40c6190 con 0x7fd8b4160130 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.365+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c030bd0 con 0x7fd8b4160130 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.366+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b40c6750 con 0x7fd8b4160130 2024-04-26T23:51:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.367+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b40693b0 con 0x7fd8b4160130 2024-04-26T23:51:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.368+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd89c047460 con 0x7fd8b4160130 2024-04-26T23:51:01.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.370+0000 7fd8b0c00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.370+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fd89c0ac8e0 con 0x7fd8b4160130 2024-04-26T23:51:01.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.370+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.372+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 0x7fd89006edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007130 tx=0x7fd8a4007060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.375+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd89c045360 con 0x7fd8b4160130 2024-04-26T23:51:01.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.578+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8b4156660 con 0x7fd8b4160130 2024-04-26T23:51:01.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.579+0000 7fd8b0c00640 1 -- 172.21.15.46:0/2739905412 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fd89c07aae0 con 0x7fd8b4160130 2024-04-26T23:51:01.582 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.581+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 msgr2=0x7fd89006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.581+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 0x7fd89006edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007130 tx=0x7fd8a4007060 comp rx=0 tx=0).stop 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.581+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 msgr2=0x7fd8b40c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.581+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd89c02a6b0 tx=0x7fd89c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 shutdown_connections 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd89006c8f0 0x7fd89006edb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8b40c7490 0x7fd8b40cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4160130 0x7fd8b40c6f50 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 --2- 172.21.15.46:0/2739905412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8b415d0a0 0x7fd8b4149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 >> 172.21.15.46:0/2739905412 conn(0x7fd8b40b27b0 msgr2=0x7fd8b415e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 shutdown_connections 2024-04-26T23:51:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.582+0000 7fd8b9800640 1 -- 172.21.15.46:0/2739905412 wait complete. 2024-04-26T23:51:01.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:51:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.816+0000 7f9f39400640 1 Processor -- start 2024-04-26T23:51:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.816+0000 7f9f39400640 1 -- start start 2024-04-26T23:51:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f39400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 0x7f9f340b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f39400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f340b5fb0 0x7f9f340d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f39400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f340d07d0 con 0x7f9f34162aa0 2024-04-26T23:51:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f39400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9f340d0ab0 con 0x7f9f340b5fb0 2024-04-26T23:51:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f39400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f340d0d90 con 0x7f9f341626c0 2024-04-26T23:51:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f38a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f34162aa0 0x7f9f340d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52962/0 (socket says 172.21.15.46:52962) 2024-04-26T23:51:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.818+0000 7f9f38a00640 1 -- 172.21.15.46:0/2628757088 learned_addr learned my addr 172.21.15.46:0/2628757088 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 0x7f9f340b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f2be00640 1 --2- 172.21.15.46:0/2628757088 >> v2:172.21.15.142:3300/0 conn(0x7f9f340b5fb0 0x7f9f340d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556030841 0 0) 0x7f9f340d07d0 con 0x7f9f34162aa0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f18003520 con 0x7f9f34162aa0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1278058084 0 0) 0x7f9f18003520 con 0x7f9f34162aa0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 msgr2=0x7f9f340b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 0x7f9f340b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 >> v2:172.21.15.142:3300/0 conn(0x7f9f340b5fb0 msgr2=0x7f9f340d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 --2- 172.21.15.46:0/2628757088 >> v2:172.21.15.142:3300/0 conn(0x7f9f340b5fb0 0x7f9f340d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f340d2c80 con 0x7f9f34162aa0 2024-04-26T23:51:01.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9f2c0019f0 con 0x7f9f34162aa0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 >> v1:172.21.15.46:6789/0 conn(0x7f9f34162aa0 legacy=0x7f9f340d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f180039b0 0x7f9f18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 0x7f9f18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f18004d40 0x7f9f180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f340d0ab0 con 0x7f9f180039b0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f340d0d90 con 0x7f9f18004d40 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.819+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1800b9c0 con 0x7f9f341626c0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f180039b0 0x7f9f18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 -- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 msgr2=0x7f9f18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f341626c0 0x7f9f18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 -- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f18004d40 msgr2=0x7f9f180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f38a00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f18004d40 0x7f9f180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f18004d40 0x7f9f180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f33e00640 1 -- 172.21.15.46:0/2628757088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18003520 con 0x7f9f180039b0 2024-04-26T23:51:01.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.820+0000 7f9f38a00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f18004d40 0x7f9f180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:01.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.821+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f180039b0 0x7f9f18003dd0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009ac0 tx=0x7f9f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f7992a2cf6822a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.822+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24036070 con 0x7f9f180039b0 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.822+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24031400 con 0x7f9f180039b0 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.822+0000 7f9f33400640 1 -- 172.21.15.46:0/2628757088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009690 con 0x7f9f180039b0 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.822+0000 7f9f39400640 1 -- 172.21.15.46:0/2628757088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f180039b0 msgr2=0x7f9f18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.822+0000 7f9f39400640 1 --2- 172.21.15.46:0/2628757088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f180039b0 0x7f9f18003dd0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009ac0 tx=0x7f9f18005830 comp rx=0 tx=0).stop 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.823+0000 7f9f38a00640 1 -- 172.21.15.46:0/2628757088 reap_dead start 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.823+0000 7f9f39400640 1 -- 172.21.15.46:0/2628757088 shutdown_connections 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.823+0000 7f9f39400640 1 -- 172.21.15.46:0/2628757088 >> 172.21.15.46:0/2628757088 conn(0x7f9f3409e2a0 msgr2=0x7f9f340d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.823+0000 7f9f39400640 1 -- 172.21.15.46:0/2628757088 shutdown_connections 2024-04-26T23:51:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.823+0000 7f9f39400640 1 -- 172.21.15.46:0/2628757088 wait complete. 2024-04-26T23:51:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.825+0000 7f9f39400640 1 Processor -- start 2024-04-26T23:51:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.826+0000 7f9f39400640 1 -- start start 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f180039b0 0x7f9f3414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f340cc000 0x7f9f340cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f340d2190 con 0x7f9f340ce050 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f340d2010 con 0x7f9f340cc000 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f39400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9f340b47c0 con 0x7f9f180039b0 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f38a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f38a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59858/0 (socket says 172.21.15.46:59858) 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f38a00640 1 -- 172.21.15.46:0/4068760167 learned_addr learned my addr 172.21.15.46:0/4068760167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.827+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f180039b0 0x7f9f3414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f2be00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f340cc000 0x7f9f340cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 -- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f180039b0 msgr2=0x7f9f3414ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f180039b0 0x7f9f3414ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 -- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f340cc000 msgr2=0x7f9f340cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f340cc000 0x7f9f340cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 -- 172.21.15.46:0/4068760167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34158a80 con 0x7f9f340ce050 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f38a00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9f340bafb0 tx=0x7f9f2c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00bb80 con 0x7f9f340ce050 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00bd20 con 0x7f9f340ce050 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c007290 con 0x7f9f340ce050 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.828+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f340ced80 con 0x7f9f340ce050 2024-04-26T23:51:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.829+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f34158d20 con 0x7f9f340ce050 2024-04-26T23:51:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.830+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f340693b0 con 0x7f9f340ce050 2024-04-26T23:51:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.831+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f2c01b020 con 0x7f9f340ce050 2024-04-26T23:51:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.831+0000 7f9f30c00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.832+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f9f2c08bfa0 con 0x7f9f340ce050 2024-04-26T23:51:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.832+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.834+0000 7f9f33e00640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9f24033000 tx=0x7f9f24008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:01.836+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f2c05a1a0 con 0x7f9f340ce050 2024-04-26T23:51:02.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.039+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f3406d020 con 0x7f9f340ce050 2024-04-26T23:51:02.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.040+0000 7f9f30c00640 1 -- 172.21.15.46:0/4068760167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) ==== 104+0+11 (secure 0 0 0) 0x7f9f2c059aa0 con 0x7f9f340ce050 2024-04-26T23:51:02.044 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:51:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 msgr2=0x7f9f1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9f24033000 tx=0x7f9f24008030 comp rx=0 tx=0).stop 2024-04-26T23:51:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 msgr2=0x7f9f340ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9f340bafb0 tx=0x7f9f2c00b4a0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 shutdown_connections 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9f1006c920 0x7f9f1006ede0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f340ce050 0x7f9f340ce4d0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f340cc000 0x7f9f340cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 --2- 172.21.15.46:0/4068760167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f180039b0 0x7f9f3414ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 >> 172.21.15.46:0/4068760167 conn(0x7f9f3409e2a0 msgr2=0x7f9f340b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 shutdown_connections 2024-04-26T23:51:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.043+0000 7f9f39400640 1 -- 172.21.15.46:0/4068760167 wait complete. 2024-04-26T23:51:02.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:51:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.276+0000 7fa9bf200640 1 Processor -- start 2024-04-26T23:51:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.277+0000 7fa9bf200640 1 -- start start 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9c00d0330 0x7fa9c01612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9c00ce670 0x7fa9c00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c0146b20 con 0x7fa9c00d0330 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c00ccf30 con 0x7fa9c00ce670 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.278+0000 7fa9bf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c0160af0 con 0x7fa9c0146740 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9c00d0330 0x7fa9c01612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9c00d0330 0x7fa9c01612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33200/0 (socket says 172.21.15.46:33200) 2024-04-26T23:51:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 -- 172.21.15.46:0/4165219337 learned_addr learned my addr 172.21.15.46:0/4165219337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:02.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9bde00640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 -- 172.21.15.46:0/4165219337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 msgr2=0x7fa9c015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:02.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 -- 172.21.15.46:0/4165219337 >> v2:172.21.15.142:3300/0 conn(0x7fa9c00ce670 msgr2=0x7fa9c00ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:02.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/4165219337 >> v2:172.21.15.142:3300/0 conn(0x7fa9c00ce670 0x7fa9c00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.279+0000 7fa9b7e00640 1 -- 172.21.15.46:0/4165219337 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c0160dd0 con 0x7fa9c00d0330 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.280+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00d0330 0x7fa9c01612f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006f090 tx=0x7fa9a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4261b052ad787a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.281+0000 7fa9bd400640 1 -- 172.21.15.46:0/4165219337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8037070 con 0x7fa9c00d0330 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.281+0000 7fa9bd400640 1 -- 172.21.15.46:0/4165219337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a8032440 con 0x7fa9c00d0330 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.281+0000 7fa9bd400640 1 -- 172.21.15.46:0/4165219337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8032810 con 0x7fa9c00d0330 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.281+0000 7fa9bf200640 1 -- 172.21.15.46:0/4165219337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00d0330 msgr2=0x7fa9c01612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.281+0000 7fa9bf200640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00d0330 0x7fa9c01612f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006f090 tx=0x7fa9a802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 -- 172.21.15.46:0/4165219337 shutdown_connections 2024-04-26T23:51:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 --2- 172.21.15.46:0/4165219337 >> v2:172.21.15.142:3300/0 conn(0x7fa9c00ce670 0x7fa9c00ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00d0330 0x7fa9c01612f0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 --2- 172.21.15.46:0/4165219337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 -- 172.21.15.46:0/4165219337 >> 172.21.15.46:0/4165219337 conn(0x7fa9c00b26d0 msgr2=0x7fa9c00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 -- 172.21.15.46:0/4165219337 shutdown_connections 2024-04-26T23:51:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.282+0000 7fa9bf200640 1 -- 172.21.15.46:0/4165219337 wait complete. 2024-04-26T23:51:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.284+0000 7fa9bf200640 1 Processor -- start 2024-04-26T23:51:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.284+0000 7fa9bf200640 1 -- start start 2024-04-26T23:51:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 0x7fa9c01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00cefe0 con 0x7fa9c00ce670 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00ccb70 con 0x7fa9c00d0330 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c00c8a40 con 0x7fa9c0146740 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.285+0000 7fa9bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33204/0 (socket says 172.21.15.46:33204) 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 -- 172.21.15.46:0/2414730251 learned_addr learned my addr 172.21.15.46:0/2414730251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9be800640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 0x7fa9c01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 -- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 msgr2=0x7fa9c015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 -- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 msgr2=0x7fa9c01f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 0x7fa9c01f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 -- 172.21.15.46:0/2414730251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c00c7c10 con 0x7fa9c00ce670 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 0x7fa9c01f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.286+0000 7fa9bde00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c7f0 tx=0x7fa9b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.287+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0015070 con 0x7fa9c00ce670 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.287+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b0004240 con 0x7fa9c00ce670 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.287+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0007620 con 0x7fa9c00ce670 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.287+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c00c7ef0 con 0x7fa9c00ce670 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.287+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c015f940 con 0x7fa9c00ce670 2024-04-26T23:51:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.288+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c00693b0 con 0x7fa9c00ce670 2024-04-26T23:51:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.289+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9b00079d0 con 0x7fa9c00ce670 2024-04-26T23:51:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.290+0000 7fa9b6000640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 0x7fa99c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.290+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7fa9b008c060 con 0x7fa9c00ce670 2024-04-26T23:51:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.291+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 0x7fa99c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.292+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 0x7fa99c06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034000 tx=0x7fa9a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.295+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9b005a390 con 0x7fa9c00ce670 2024-04-26T23:51:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.501+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9c006f400 con 0x7fa9c00ce670 2024-04-26T23:51:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.502+0000 7fa9b6000640 1 -- 172.21.15.46:0/2414730251 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) ==== 100+0+11 (secure 0 0 0) 0x7fa9b0059c90 con 0x7fa9c00ce670 2024-04-26T23:51:02.505 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 msgr2=0x7fa99c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 0x7fa99c06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034000 tx=0x7fa9a802e000 comp rx=0 tx=0).stop 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 msgr2=0x7fa9c00c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c7f0 tx=0x7fa9b000ccb0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 shutdown_connections 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa99c06c8f0 0x7fa99c06edb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9c0146740 0x7fa9c015fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c00d0330 0x7fa9c01f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 --2- 172.21.15.46:0/2414730251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c00ce670 0x7fa9c00c8500 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 >> 172.21.15.46:0/2414730251 conn(0x7fa9c00b26d0 msgr2=0x7fa9c006e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 shutdown_connections 2024-04-26T23:51:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.505+0000 7fa9bf200640 1 -- 172.21.15.46:0/2414730251 wait complete. 2024-04-26T23:51:02.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.738+0000 7f6e01c00640 1 Processor -- start 2024-04-26T23:51:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.738+0000 7f6e01c00640 1 -- start start 2024-04-26T23:51:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 0x7f6dfc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6dfc144840 0x7f6dfc144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6dfc0b43f0 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6dfc0b46d0 con 0x7f6dfc144840 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6e01c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc0b49b0 con 0x7f6dfc142280 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.740+0000 7f6df2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33206/0 (socket says 172.21.15.46:33206) 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 -- 172.21.15.46:0/1592361922 learned_addr learned my addr 172.21.15.46:0/1592361922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 0x7f6dfc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 -- 172.21.15.46:0/1592361922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 msgr2=0x7f6dfc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 0x7f6dfc06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 -- 172.21.15.46:0/1592361922 >> v2:172.21.15.142:3300/0 conn(0x7f6dfc144840 msgr2=0x7f6dfc144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 --2- 172.21.15.46:0/1592361922 >> v2:172.21.15.142:3300/0 conn(0x7f6dfc144840 0x7f6dfc144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6df2000640 1 -- 172.21.15.46:0/1592361922 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc0b4c90 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.741+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 0x7f6dfc06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.742+0000 7f6df2000640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc070170 tx=0x7f6de402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54265923f2ecf80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.742+0000 7f6dfa000640 1 -- 172.21.15.46:0/1592361922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4037070 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6dfa000640 1 -- 172.21.15.46:0/1592361922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de4032440 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6dfa000640 1 -- 172.21.15.46:0/1592361922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4032810 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 -- 172.21.15.46:0/1592361922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc0b3e80 msgr2=0x7f6dfc06ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc070170 tx=0x7f6de402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 -- 172.21.15.46:0/1592361922 shutdown_connections 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 --2- 172.21.15.46:0/1592361922 >> v2:172.21.15.142:3300/0 conn(0x7f6dfc144840 0x7f6dfc144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc0b3e80 0x7f6dfc06ea50 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 --2- 172.21.15.46:0/1592361922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc142280 0x7f6dfc06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.743+0000 7f6e01c00640 1 -- 172.21.15.46:0/1592361922 >> 172.21.15.46:0/1592361922 conn(0x7f6dfc06c530 msgr2=0x7f6dfc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.744+0000 7f6e01c00640 1 -- 172.21.15.46:0/1592361922 shutdown_connections 2024-04-26T23:51:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.744+0000 7f6e01c00640 1 -- 172.21.15.46:0/1592361922 wait complete. 2024-04-26T23:51:02.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.745+0000 7f6e01c00640 1 Processor -- start 2024-04-26T23:51:02.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.745+0000 7f6e01c00640 1 -- start start 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.746+0000 7f6e01c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b3e80 0x7f6dfc141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.746+0000 7f6e01c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 0x7f6dfc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.746+0000 7f6e01c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6e01c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc142a20 con 0x7f6dfc144840 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6e01c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1428a0 con 0x7f6dfc142280 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6e01c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc1451f0 con 0x7f6dfc0b3e80 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33208/0 (socket says 172.21.15.46:33208) 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 -- 172.21.15.46:0/3988084166 learned_addr learned my addr 172.21.15.46:0/3988084166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b3e80 0x7f6dfc141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6df2000640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 0x7f6dfc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 -- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b3e80 msgr2=0x7f6dfc141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b3e80 0x7f6dfc141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 -- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 msgr2=0x7f6dfc15ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 0x7f6dfc15ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6dfb400640 1 -- 172.21.15.46:0/3988084166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc1f56e0 con 0x7f6dfc144840 2024-04-26T23:51:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.747+0000 7f6df2000640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 0x7f6dfc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6dfb400640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b410 tx=0x7f6df400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4015020 con 0x7f6dfc144840 2024-04-26T23:51:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4011850 con 0x7f6dfc144840 2024-04-26T23:51:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc15f7c0 con 0x7f6dfc144840 2024-04-26T23:51:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4011c40 con 0x7f6dfc144840 2024-04-26T23:51:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.748+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc0b25b0 con 0x7f6dfc144840 2024-04-26T23:51:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.749+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc069530 con 0x7f6dfc144840 2024-04-26T23:51:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.750+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6df4018020 con 0x7f6dfc144840 2024-04-26T23:51:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.751+0000 7f6df3400640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 0x7f6dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.751+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f6df408bc10 con 0x7f6dfc144840 2024-04-26T23:51:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.752+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 0x7f6dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.753+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 0x7f6dd806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0075b0 tx=0x7f6dec007540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:02.756+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6df401f020 con 0x7f6dfc144840 2024-04-26T23:51:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.000+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dfc1f87c0 con 0x7f6dfc144840 2024-04-26T23:51:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.001+0000 7f6df3400640 1 -- 172.21.15.46:0/3988084166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f6df40172c0 con 0x7f6dfc144840 2024-04-26T23:51:03.005 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:03.005 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:51:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 msgr2=0x7f6dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 0x7f6dd806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0075b0 tx=0x7f6dec007540 comp rx=0 tx=0).stop 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 msgr2=0x7f6dfc1f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b410 tx=0x7f6df400b8d0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 shutdown_connections 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6dd806c8f0 0x7f6dd806edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dfc144840 0x7f6dfc1f51a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc142280 0x7f6dfc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 --2- 172.21.15.46:0/3988084166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b3e80 0x7f6dfc141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.004+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 >> 172.21.15.46:0/3988084166 conn(0x7f6dfc06c530 msgr2=0x7f6dfc073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.005+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 shutdown_connections 2024-04-26T23:51:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.005+0000 7f6e01c00640 1 -- 172.21.15.46:0/3988084166 wait complete. 2024-04-26T23:51:03.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.236+0000 7ffaa3000640 1 Processor -- start 2024-04-26T23:51:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.236+0000 7ffaa3000640 1 -- start start 2024-04-26T23:51:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.237+0000 7ffaa3000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa9c0d2130 0x7ffa9c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.237+0000 7ffaa3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa9c0d11f0 con 0x7ffa9c0b4000 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.238+0000 7ffaa3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa9c0d5f20 con 0x7ffa9c15f4e0 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.238+0000 7ffaa3000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffa9c0d0ce0 con 0x7ffa9c0d2130 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.238+0000 7ffaa2600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa9c0b4000 0x7ffa9c0b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60028/0 (socket says 172.21.15.46:60028) 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.238+0000 7ffaa2600640 1 -- 172.21.15.46:0/306801743 learned_addr learned my addr 172.21.15.46:0/306801743 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.238+0000 7ffaa1200640 1 --2- 172.21.15.46:0/306801743 >> v2:172.21.15.142:3300/0 conn(0x7ffa9c0d2130 0x7ffa9c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034167393 0 0) 0x7ffa9c0d11f0 con 0x7ffa9c0b4000 2024-04-26T23:51:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa84003520 con 0x7ffa9c0b4000 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1731323111 0 0) 0x7ffa84003520 con 0x7ffa9c0b4000 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 >> v2:172.21.15.142:3300/0 conn(0x7ffa9c0d2130 msgr2=0x7ffa9c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 --2- 172.21.15.46:0/306801743 >> v2:172.21.15.142:3300/0 conn(0x7ffa9c0d2130 0x7ffa9c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 >> v1:172.21.15.142:6789/0 conn(0x7ffa9c15f4e0 legacy=0x7ffa9c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c0d0f10 con 0x7ffa9c0b4000 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ffa940019f0 con 0x7ffa9c0b4000 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 >> v1:172.21.15.46:6789/0 conn(0x7ffa9c0b4000 legacy=0x7ffa9c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840039b0 0x7ffa84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 0x7ffa84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840051c0 0x7ffa8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8400c030 con 0x7ffa84004800 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8400c1e0 con 0x7ffa840051c0 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8400c3e0 con 0x7ffa840039b0 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.239+0000 7ffaa1200640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 0x7ffa84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa2600640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840051c0 0x7ffa8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa1200640 1 -- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840039b0 msgr2=0x7ffa84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa1200640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa840039b0 0x7ffa84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa1200640 1 -- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840051c0 msgr2=0x7ffa8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa1200640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa840051c0 0x7ffa8400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa1200640 1 -- 172.21.15.46:0/306801743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84003520 con 0x7ffa84004800 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.240+0000 7ffaa2600640 1 -- 172.21.15.46:0/306801743 reap_dead start 2024-04-26T23:51:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.241+0000 7ffaa1200640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 0x7ffa84004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009ac0 tx=0x7ffa84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c00e5d22dde9bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.241+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88036070 con 0x7ffa84004800 2024-04-26T23:51:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.241+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88031400 con 0x7ffa84004800 2024-04-26T23:51:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.241+0000 7ffa9be00640 1 -- 172.21.15.46:0/306801743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88009690 con 0x7ffa84004800 2024-04-26T23:51:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 -- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 msgr2=0x7ffa84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 0x7ffa84004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009ac0 tx=0x7ffa84005830 comp rx=0 tx=0).stop 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 -- 172.21.15.46:0/306801743 shutdown_connections 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 --2- 172.21.15.46:0/306801743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa84004800 0x7ffa84004c80 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 -- 172.21.15.46:0/306801743 >> 172.21.15.46:0/306801743 conn(0x7ffa9c0d5010 msgr2=0x7ffa9c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 -- 172.21.15.46:0/306801743 shutdown_connections 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.242+0000 7ffaa3000640 1 -- 172.21.15.46:0/306801743 wait complete. 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.244+0000 7ffaa3000640 1 Processor -- start 2024-04-26T23:51:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.244+0000 7ffaa3000640 1 -- start start 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84004800 0x7ffa9c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c0c8970 0x7ffa9c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c0b1d60 con 0x7ffa9c157350 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c0d2990 con 0x7ffa84004800 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa9c15f8c0 con 0x7ffa9c0c8970 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33224/0 (socket says 172.21.15.46:33224) 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa1200640 1 -- 172.21.15.46:0/2247650011 learned_addr learned my addr 172.21.15.46:0/2247650011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa1c00640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84004800 0x7ffa9c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.245+0000 7ffaa2600640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c0c8970 0x7ffa9c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 -- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c0c8970 msgr2=0x7ffa9c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c0c8970 0x7ffa9c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 -- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84004800 msgr2=0x7ffa9c0cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84004800 0x7ffa9c0cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 -- 172.21.15.46:0/2247650011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c160b90 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa1200640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802ab90 tx=0x7ffa88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8803f020 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88037040 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88036070 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.246+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c0cc940 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.247+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c0cb600 con 0x7ffa9c157350 2024-04-26T23:51:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.248+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9c0693b0 con 0x7ffa9c157350 2024-04-26T23:51:03.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.249+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa88042020 con 0x7ffa9c157350 2024-04-26T23:51:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.250+0000 7ffa99600640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 0x7ffa7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.250+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7ffa880abdb0 con 0x7ffa9c157350 2024-04-26T23:51:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.251+0000 7ffaa1c00640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 0x7ffa7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.252+0000 7ffaa1c00640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 0x7ffa7c06edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007fd0 tx=0x7ffa90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.255+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa88049030 con 0x7ffa9c157350 2024-04-26T23:51:03.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.464+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa9c0d33d0 con 0x7ffa9c157350 2024-04-26T23:51:03.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.466+0000 7ffa99600640 1 -- 172.21.15.46:0/2247650011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16864 (secure 0 0 0) 0x7ffa8807a140 con 0x7ffa9c157350 2024-04-26T23:51:03.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:03.470 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":137,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:01.836702+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 msgr2=0x7ffa7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 0x7ffa7c06edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007fd0 tx=0x7ffa90009040 comp rx=0 tx=0).stop 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 msgr2=0x7ffa9c156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802ab90 tx=0x7ffa88004730 comp rx=0 tx=0).stop 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 shutdown_connections 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa7c06c8f0 0x7ffa7c06edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c0c8970 0x7ffa9c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa9c157350 0x7ffa9c156ba0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 --2- 172.21.15.46:0/2247650011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa84004800 0x7ffa9c0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.468+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 >> 172.21.15.46:0/2247650011 conn(0x7ffa9c0d5010 msgr2=0x7ffa9c0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.469+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 shutdown_connections 2024-04-26T23:51:03.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.469+0000 7ffaa3000640 1 -- 172.21.15.46:0/2247650011 wait complete. 2024-04-26T23:51:03.493 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:51:03.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.691+0000 7f7bf3e00640 1 Processor -- start 2024-04-26T23:51:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.691+0000 7f7bf3e00640 1 -- start start 2024-04-26T23:51:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.693+0000 7f7bf3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7bec06e510 0x7f7bec06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.693+0000 7f7bf3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec06ee30 con 0x7f7bec0732c0 2024-04-26T23:51:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.693+0000 7f7bf3e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec161940 con 0x7f7bec160bd0 2024-04-26T23:51:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.693+0000 7f7bf3e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7bec071f80 con 0x7f7bec06e510 2024-04-26T23:51:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.693+0000 7f7bf1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7bec0732c0 0x7f7bec073710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60044/0 (socket says 172.21.15.46:60044) 2024-04-26T23:51:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.694+0000 7f7bf1e00640 1 -- 172.21.15.46:0/2592013258 learned_addr learned my addr 172.21.15.46:0/2592013258 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.694+0000 7f7bf1400640 1 --2- 172.21.15.46:0/2592013258 >> v2:172.21.15.142:3300/0 conn(0x7f7bec06e510 0x7f7bec06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.694+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196576602 0 0) 0x7f7bec06ee30 con 0x7f7bec0732c0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.694+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bd4003520 con 0x7f7bec0732c0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3226676877 0 0) 0x7f7bd4003520 con 0x7f7bec0732c0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 >> v2:172.21.15.142:3300/0 conn(0x7f7bec06e510 msgr2=0x7f7bec06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 --2- 172.21.15.46:0/2592013258 >> v2:172.21.15.142:3300/0 conn(0x7f7bec06e510 0x7f7bec06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 >> v1:172.21.15.142:6789/0 conn(0x7f7bec160bd0 legacy=0x7f7bec161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec072260 con 0x7f7bec0732c0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7be40019f0 con 0x7f7bec0732c0 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 >> v1:172.21.15.46:6789/0 conn(0x7f7bec0732c0 legacy=0x7f7bec073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd40039b0 0x7f7bd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd4004800 0x7f7bd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 0x7f7bd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec071f80 con 0x7f7bd40051c0 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd400c060 con 0x7f7bd40039b0 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd400c260 con 0x7f7bd4004800 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bf1e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 0x7f7bd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bf1400640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd40039b0 0x7f7bd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.695+0000 7f7bf0a00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd4004800 0x7f7bd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.696+0000 7f7bf1e00640 1 -- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd4004800 msgr2=0x7f7bd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.696+0000 7f7bf1e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd4004800 0x7f7bd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.696+0000 7f7bf1e00640 1 -- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd40039b0 msgr2=0x7f7bd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.696+0000 7f7bf1e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd40039b0 0x7f7bd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.696+0000 7f7bf1e00640 1 -- 172.21.15.46:0/2592013258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd4003520 con 0x7f7bd40051c0 2024-04-26T23:51:03.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.697+0000 7f7bf1e00640 1 -- 172.21.15.46:0/2592013258 reap_dead start 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.697+0000 7f7bf1e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 0x7f7bd400b9c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7be400a840 tx=0x7f7bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54026ce986aa07b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.697+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be400a460 con 0x7f7bd40051c0 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.697+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be402f070 con 0x7f7bd40051c0 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.697+0000 7f7bebe00640 1 -- 172.21.15.46:0/2592013258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be402bcb0 con 0x7f7bd40051c0 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 -- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 msgr2=0x7f7bd400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 0x7f7bd400b9c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7be400a840 tx=0x7f7bd4005830 comp rx=0 tx=0).stop 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 -- 172.21.15.46:0/2592013258 shutdown_connections 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/2592013258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd40051c0 0x7f7bd400b9c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 -- 172.21.15.46:0/2592013258 >> 172.21.15.46:0/2592013258 conn(0x7f7bec06c530 msgr2=0x7f7bec06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 -- 172.21.15.46:0/2592013258 shutdown_connections 2024-04-26T23:51:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.698+0000 7f7bf3e00640 1 -- 172.21.15.46:0/2592013258 wait complete. 2024-04-26T23:51:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.700+0000 7f7bf3e00640 1 Processor -- start 2024-04-26T23:51:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.700+0000 7f7bf3e00640 1 -- start start 2024-04-26T23:51:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd40051c0 0x7f7bec13f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bec1f5510 0x7f7bec0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec072d40 con 0x7f7bec140e80 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec072820 con 0x7f7bec1f5510 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec072ec0 con 0x7f7bd40051c0 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33244/0 (socket says 172.21.15.46:33244) 2024-04-26T23:51:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.701+0000 7f7bf0a00640 1 -- 172.21.15.46:0/786866532 learned_addr learned my addr 172.21.15.46:0/786866532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf1400640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd40051c0 0x7f7bec13f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf1e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bec1f5510 0x7f7bec0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 -- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd40051c0 msgr2=0x7f7bec13f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd40051c0 0x7f7bec13f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 -- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bec1f5510 msgr2=0x7f7bec0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bec1f5510 0x7f7bec0b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 -- 172.21.15.46:0/786866532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec1f9080 con 0x7f7bec140e80 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf0a00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e480 tx=0x7f7bd800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80052e0 con 0x7f7bec140e80 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8018070 con 0x7f7bec140e80 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003830 con 0x7f7bec140e80 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.702+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec1f9360 con 0x7f7bec140e80 2024-04-26T23:51:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.703+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec1f9920 con 0x7f7bec140e80 2024-04-26T23:51:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.704+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bec1f9d00 con 0x7f7bec140e80 2024-04-26T23:51:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.705+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bd80039d0 con 0x7f7bec140e80 2024-04-26T23:51:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.706+0000 7f7be9600640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 0x7f7bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.706+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f7bd808b9b0 con 0x7f7bec140e80 2024-04-26T23:51:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.707+0000 7f7bf1400640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 0x7f7bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.708+0000 7f7bf1400640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 0x7f7bcc06edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7be00098d0 tx=0x7f7be0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.711+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bec1f9d00 con 0x7f7bec140e80 2024-04-26T23:51:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.965+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bec069530 con 0x7f7bec140e80 2024-04-26T23:51:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.966+0000 7f7be9600640 1 -- 172.21.15.46:0/786866532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f7bd8059c40 con 0x7f7bec140e80 2024-04-26T23:51:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:03.970 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:51:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 msgr2=0x7f7bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 0x7f7bcc06edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7be00098d0 tx=0x7f7be0008040 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 msgr2=0x7f7bec1f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e480 tx=0x7f7bd800e940 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 shutdown_connections 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bcc06c8f0 0x7f7bcc06edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bec1f5510 0x7f7bec0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bec140e80 0x7f7bec1f4fd0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 --2- 172.21.15.46:0/786866532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bd40051c0 0x7f7bec13f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 >> 172.21.15.46:0/786866532 conn(0x7f7bec06c530 msgr2=0x7f7bec06e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 shutdown_connections 2024-04-26T23:51:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:03.969+0000 7f7bf3e00640 1 -- 172.21.15.46:0/786866532 wait complete. 2024-04-26T23:51:03.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.183+0000 7f732da00640 1 Processor -- start 2024-04-26T23:51:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.183+0000 7f732da00640 1 -- start start 2024-04-26T23:51:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.184+0000 7f732da00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f732806e780 0x7f73280b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.184+0000 7f732da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73280b43c0 0x7f732814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.184+0000 7f732da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73280b4940 con 0x7f732806eb60 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.184+0000 7f732da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f73280b4c20 con 0x7f73280b43c0 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.184+0000 7f732da00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f73280b4f00 con 0x7f732806e780 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f7327400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f732806eb60 0x7f7328167130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60050/0 (socket says 172.21.15.46:60050) 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f7327400640 1 -- 172.21.15.46:0/3973510012 learned_addr learned my addr 172.21.15.46:0/3973510012 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f731fe00640 1 --2- 172.21.15.46:0/3973510012 >> v2:172.21.15.46:3300/0 conn(0x7f73280b43c0 0x7f732814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f7326a00640 1 --2- 172.21.15.46:0/3973510012 >> v2:172.21.15.142:3300/0 conn(0x7f732806e780 0x7f73280b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f7326000640 1 -- 172.21.15.46:0/3973510012 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 571896553 0 0) 0x7f73280b4940 con 0x7f732806eb60 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f7326000640 1 -- 172.21.15.46:0/3973510012 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f730c003520 con 0x7f732806eb60 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f731fe00640 1 -- 172.21.15.46:0/3973510012 >> v2:172.21.15.142:3300/0 conn(0x7f732806e780 msgr2=0x7f73280b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f731fe00640 1 --2- 172.21.15.46:0/3973510012 >> v2:172.21.15.142:3300/0 conn(0x7f732806e780 0x7f73280b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f731fe00640 1 -- 172.21.15.46:0/3973510012 >> v1:172.21.15.46:6789/0 conn(0x7f732806eb60 legacy=0x7f7328167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.185+0000 7f731fe00640 1 -- 172.21.15.46:0/3973510012 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73280b51e0 con 0x7f73280b43c0 2024-04-26T23:51:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.187+0000 7f731fe00640 1 --2- 172.21.15.46:0/3973510012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73280b43c0 0x7f732814caf0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7328070070 tx=0x7f731002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da798b985994d42c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.187+0000 7f7326000640 1 -- 172.21.15.46:0/3973510012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310037070 con 0x7f73280b43c0 2024-04-26T23:51:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.187+0000 7f7326000640 1 -- 172.21.15.46:0/3973510012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310032440 con 0x7f73280b43c0 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.187+0000 7f7326000640 1 -- 172.21.15.46:0/3973510012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310032810 con 0x7f73280b43c0 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 -- 172.21.15.46:0/3973510012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73280b43c0 msgr2=0x7f732814caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 --2- 172.21.15.46:0/3973510012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73280b43c0 0x7f732814caf0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7328070070 tx=0x7f731002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 -- 172.21.15.46:0/3973510012 shutdown_connections 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 --2- 172.21.15.46:0/3973510012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73280b43c0 0x7f732814caf0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 --2- 172.21.15.46:0/3973510012 >> v2:172.21.15.142:3300/0 conn(0x7f732806e780 0x7f73280b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 -- 172.21.15.46:0/3973510012 >> 172.21.15.46:0/3973510012 conn(0x7f732806c530 msgr2=0x7f732806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 -- 172.21.15.46:0/3973510012 shutdown_connections 2024-04-26T23:51:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.188+0000 7f732da00640 1 -- 172.21.15.46:0/3973510012 wait complete. 2024-04-26T23:51:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.190+0000 7f732da00640 1 Processor -- start 2024-04-26T23:51:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.190+0000 7f732da00640 1 -- start start 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f732806eb60 0x7f7328136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73280b43c0 0x7f73281395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73280734f0 con 0x7f732806e780 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7328073370 con 0x7f73280b43c0 2024-04-26T23:51:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f732da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7328073670 con 0x7f732806eb60 2024-04-26T23:51:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33264/0 (socket says 172.21.15.46:33264) 2024-04-26T23:51:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 -- 172.21.15.46:0/2294612055 learned_addr learned my addr 172.21.15.46:0/2294612055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 -- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f732806eb60 msgr2=0x7f7328136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f732806eb60 0x7f7328136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 -- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73280b43c0 msgr2=0x7f73281395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.191+0000 7f7326a00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73280b43c0 0x7f73281395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f7326a00640 1 -- 172.21.15.46:0/2294612055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732815de40 con 0x7f732806e780 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f7326a00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f731800d830 tx=0x7f731800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800b6c0 con 0x7f732806e780 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7318019070 con 0x7f732806e780 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800ba80 con 0x7f732806e780 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732815f850 con 0x7f732806e780 2024-04-26T23:51:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.192+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732815fe10 con 0x7f732806e780 2024-04-26T23:51:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.193+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7328069530 con 0x7f732806e780 2024-04-26T23:51:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.194+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f731801e460 con 0x7f732806e780 2024-04-26T23:51:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.195+0000 7f731ea00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 0x7f730406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.195+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f731808c460 con 0x7f732806e780 2024-04-26T23:51:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.196+0000 7f731fe00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 0x7f730406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.197+0000 7f731fe00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 0x7f730406edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.200+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f731805a660 con 0x7f732806e780 2024-04-26T23:51:04.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.446+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7328163680 con 0x7f732806e780 2024-04-26T23:51:04.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.448+0000 7f731ea00640 1 -- 172.21.15.46:0/2294612055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5189 (secure 0 0 0) 0x7f7318059f60 con 0x7f732806e780 2024-04-26T23:51:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:04.451 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:50:00.302683+0000","modified":"2024-04-26T23:50:03.402945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/2039062774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2039062774},{"type":"v1","addr":"172.21.15.46:6833","nonce":2039062774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-04-26T23:51:04.451 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-26T23:51:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.450+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 msgr2=0x7f730406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.450+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 0x7f730406edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002c000 comp rx=0 tx=0).stop 2024-04-26T23:51:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 msgr2=0x7f732815d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f731800d830 tx=0x7f731800dcf0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 shutdown_connections 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f730406c8f0 0x7f730406edb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73280b43c0 0x7f73281395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f732806eb60 0x7f7328136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 --2- 172.21.15.46:0/2294612055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732806e780 0x7f732815d440 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 >> 172.21.15.46:0/2294612055 conn(0x7f732806c530 msgr2=0x7f7328072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 shutdown_connections 2024-04-26T23:51:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.451+0000 7f732da00640 1 -- 172.21.15.46:0/2294612055 wait complete. 2024-04-26T23:51:04.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.685+0000 7f1baea00640 1 Processor -- start 2024-04-26T23:51:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.685+0000 7f1baea00640 1 -- start start 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1baea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ba8145770 0x7f1ba80d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1baea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba80cf950 con 0x7f1ba8072e60 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1baea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba80cfc30 con 0x7f1ba815ccd0 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1baea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba80d0c50 con 0x7f1ba8145770 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1b9f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ba8072e60 0x7f1ba815c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60054/0 (socket says 172.21.15.46:60054) 2024-04-26T23:51:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1b9f400640 1 -- 172.21.15.46:0/923922194 learned_addr learned my addr 172.21.15.46:0/923922194 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.687+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/923922194 >> v2:172.21.15.142:3300/0 conn(0x7f1ba8145770 0x7f1ba80d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3764019360 0 0) 0x7f1ba80cf950 con 0x7f1ba8072e60 2024-04-26T23:51:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b8c005730 con 0x7f1ba8072e60 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1585112508 0 0) 0x7f1b8c005730 con 0x7f1ba8072e60 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 >> v2:172.21.15.142:3300/0 conn(0x7f1ba8145770 msgr2=0x7f1ba80d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 --2- 172.21.15.46:0/923922194 >> v2:172.21.15.142:3300/0 conn(0x7f1ba8145770 0x7f1ba80d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 >> v1:172.21.15.142:6789/0 conn(0x7f1ba815ccd0 legacy=0x7f1ba80b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba80cff10 con 0x7f1ba8072e60 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1b90001670 con 0x7f1ba8072e60 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 >> v1:172.21.15.46:6789/0 conn(0x7f1ba8072e60 legacy=0x7f1ba815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b8c005bc0 0x7f1b8c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c006a10 0x7f1b8c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 0x7f1b8c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80d0c50 con 0x7f1b8c0073d0 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c00e270 con 0x7f1b8c005bc0 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.688+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c00e470 con 0x7f1b8c006a10 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 0x7f1b8c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b8c005bc0 0x7f1b8c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1b9f400640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c006a10 0x7f1b8c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 -- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c006a10 msgr2=0x7f1b8c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c006a10 0x7f1b8c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 -- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b8c005bc0 msgr2=0x7f1b8c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b8c005bc0 0x7f1b8c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1baca00640 1 -- 172.21.15.46:0/923922194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c005730 con 0x7f1b8c0073d0 2024-04-26T23:51:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.689+0000 7f1b9f400640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c006a10 0x7f1b8c006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.690+0000 7f1baca00640 1 -- 172.21.15.46:0/923922194 reap_dead start 2024-04-26T23:51:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.690+0000 7f1baca00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 0x7f1b8c00dbd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0001e20 tx=0x7f1b8c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c056cdd60493c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.690+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0001a70 con 0x7f1b8c0073d0 2024-04-26T23:51:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba002ae30 con 0x7f1b8c0073d0 2024-04-26T23:51:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1ba7400640 1 -- 172.21.15.46:0/923922194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003b70 con 0x7f1b8c0073d0 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 -- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 msgr2=0x7f1b8c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 0x7f1b8c00dbd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0001e20 tx=0x7f1b8c007a40 comp rx=0 tx=0).stop 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 -- 172.21.15.46:0/923922194 shutdown_connections 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 --2- 172.21.15.46:0/923922194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c0073d0 0x7f1b8c00dbd0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 -- 172.21.15.46:0/923922194 >> 172.21.15.46:0/923922194 conn(0x7f1ba806c530 msgr2=0x7f1ba806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 -- 172.21.15.46:0/923922194 shutdown_connections 2024-04-26T23:51:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.691+0000 7f1baea00640 1 -- 172.21.15.46:0/923922194 wait complete. 2024-04-26T23:51:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.693+0000 7f1baea00640 1 Processor -- start 2024-04-26T23:51:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.693+0000 7f1baea00640 1 -- start start 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 0x7f1ba813ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba813f470 0x7f1ba81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80cf5b0 con 0x7f1ba813d570 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba815d490 con 0x7f1ba813f470 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba80cf730 con 0x7f1b8c0073d0 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33282/0 (socket says 172.21.15.46:33282) 2024-04-26T23:51:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.694+0000 7f1baca00640 1 -- 172.21.15.46:0/1078117297 learned_addr learned my addr 172.21.15.46:0/1078117297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 0x7f1ba813ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1b9f400640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba813f470 0x7f1ba81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 -- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 msgr2=0x7f1ba813ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 0x7f1ba813ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 -- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba813f470 msgr2=0x7f1ba81f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba813f470 0x7f1ba81f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 -- 172.21.15.46:0/1078117297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba81f5950 con 0x7f1ba813d570 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 0x7f1ba813ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.695+0000 7f1baca00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1ba002a8e0 tx=0x7f1ba00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.696+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0007db0 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.696+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba0001250 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.696+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00055f0 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.696+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba81f5c30 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.696+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba80aec60 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.697+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba8069530 con 0x7f1ba813d570 2024-04-26T23:51:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.698+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ba003d470 con 0x7f1ba813d570 2024-04-26T23:51:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.699+0000 7f1ba4c00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 0x7f1b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.699+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f1ba00aac80 con 0x7f1ba813d570 2024-04-26T23:51:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.700+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 0x7f1b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.701+0000 7f1ba7e00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 0x7f1b8406edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009980 tx=0x7f1b98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.703+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ba0044070 con 0x7f1ba813d570 2024-04-26T23:51:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.914+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ba8145c70 con 0x7f1ba813d570 2024-04-26T23:51:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.916+0000 7f1ba4c00640 1 -- 172.21.15.46:0/1078117297 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16864 (secure 0 0 0) 0x7f1ba0078f40 con 0x7f1ba813d570 2024-04-26T23:51:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:04.920 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":137,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:50:01.836702+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:49:58.107194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-26T23:49:59.279006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.919+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 msgr2=0x7f1b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.919+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 0x7f1b8406edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009980 tx=0x7f1b98008040 comp rx=0 tx=0).stop 2024-04-26T23:51:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.919+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 msgr2=0x7f1ba81f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.919+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1ba002a8e0 tx=0x7f1ba00014a0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 shutdown_connections 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b8406c8f0 0x7f1b8406edb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ba813d570 0x7f1ba81f52d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba813f470 0x7f1ba81f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 --2- 172.21.15.46:0/1078117297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b8c0073d0 0x7f1ba813ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 >> 172.21.15.46:0/1078117297 conn(0x7f1ba806c530 msgr2=0x7f1ba80b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 shutdown_connections 2024-04-26T23:51:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:04.920+0000 7f1baea00640 1 -- 172.21.15.46:0/1078117297 wait complete. 2024-04-26T23:51:04.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:51:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.160+0000 7f66c7000640 1 Processor -- start 2024-04-26T23:51:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.160+0000 7f66c7000640 1 -- start start 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.161+0000 7f66c7000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 0x7f66c00d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.161+0000 7f66c7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 0x7f66c00d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.161+0000 7f66c7000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66c00d5410 0x7f66c0162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.161+0000 7f66c7000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f66c015f890 con 0x7f66c00d5410 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c7000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f66c00cfd60 con 0x7f66c015f4b0 2024-04-26T23:51:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00d2910 con 0x7f66c00b5140 2024-04-26T23:51:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66c00d5410 0x7f66c0162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66c00d5410 0x7f66c0162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33284/0 (socket says 172.21.15.46:33284) 2024-04-26T23:51:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 -- 172.21.15.46:0/4064088733 learned_addr learned my addr 172.21.15.46:0/4064088733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66bfe00640 1 --2- 172.21.15.46:0/4064088733 >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 0x7f66c00d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66bf400640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 0x7f66c00d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 -- 172.21.15.46:0/4064088733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 msgr2=0x7f66c00d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 0x7f66c00d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 -- 172.21.15.46:0/4064088733 >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 msgr2=0x7f66c00d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 --2- 172.21.15.46:0/4064088733 >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 0x7f66c00d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.162+0000 7f66c5000640 1 -- 172.21.15.46:0/4064088733 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c00d58a0 con 0x7f66c00d5410 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.163+0000 7f66bf400640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 0x7f66c00d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.163+0000 7f66bfe00640 1 --2- 172.21.15.46:0/4064088733 >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 0x7f66c00d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.164+0000 7f66c5000640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c0162f10 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f66c00d2b30 tx=0x7f66b802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dba999ad48dc95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.164+0000 7f66bea00640 1 -- 172.21.15.46:0/4064088733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8037070 con 0x7f66c00d5410 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.164+0000 7f66bea00640 1 -- 172.21.15.46:0/4064088733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8032440 con 0x7f66c00d5410 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.164+0000 7f66bea00640 1 -- 172.21.15.46:0/4064088733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8032810 con 0x7f66c00d5410 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 -- 172.21.15.46:0/4064088733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 msgr2=0x7f66c0162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c0162f10 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f66c00d2b30 tx=0x7f66b802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 -- 172.21.15.46:0/4064088733 shutdown_connections 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c0162f10 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 --2- 172.21.15.46:0/4064088733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b5140 0x7f66c00d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 --2- 172.21.15.46:0/4064088733 >> v2:172.21.15.142:3300/0 conn(0x7f66c015f4b0 0x7f66c00d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 -- 172.21.15.46:0/4064088733 >> 172.21.15.46:0/4064088733 conn(0x7f66c008eb50 msgr2=0x7f66c009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 -- 172.21.15.46:0/4064088733 shutdown_connections 2024-04-26T23:51:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.165+0000 7f66c7000640 1 -- 172.21.15.46:0/4064088733 wait complete. 2024-04-26T23:51:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.167+0000 7f66c7000640 1 Processor -- start 2024-04-26T23:51:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.167+0000 7f66c7000640 1 -- start start 2024-04-26T23:51:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 0x7f66c00b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 0x7f66c0161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00b5b90 con 0x7f66c00d5410 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00b58b0 con 0x7f66c015f4b0 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66c7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00b5e70 con 0x7f66c00b9bf0 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33290/0 (socket says 172.21.15.46:33290) 2024-04-26T23:51:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.168+0000 7f66bfe00640 1 -- 172.21.15.46:0/1036862650 learned_addr learned my addr 172.21.15.46:0/1036862650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bf400640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 0x7f66c00b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66c5000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 0x7f66c0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 -- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 msgr2=0x7f66c0161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 0x7f66c0161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 -- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 msgr2=0x7f66c00b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 0x7f66c00b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 -- 172.21.15.46:0/1036862650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c00b8020 con 0x7f66c00d5410 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66c5000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 0x7f66c0161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bf400640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 0x7f66c00b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.169+0000 7f66bfe00640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ce30 tx=0x7f66b000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.170+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000f890 con 0x7f66c00d5410 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.170+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b000fa30 con 0x7f66c00d5410 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.170+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000fe00 con 0x7f66c00d5410 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.170+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c00b8300 con 0x7f66c00d5410 2024-04-26T23:51:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.170+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c00b88c0 con 0x7f66c00d5410 2024-04-26T23:51:05.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.171+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c0069530 con 0x7f66c00d5410 2024-04-26T23:51:05.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.172+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66b0018400 con 0x7f66c00d5410 2024-04-26T23:51:05.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.173+0000 7f66b7e00640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 0x7f66a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.173+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6101+0+0 (secure 0 0 0) 0x7f66b008cac0 con 0x7f66c00d5410 2024-04-26T23:51:05.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.174+0000 7f66bf400640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 0x7f66a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.175+0000 7f66bf400640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 0x7f66a006ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f66c00bad80 tx=0x7f66ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.177+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66b0014030 con 0x7f66c00d5410 2024-04-26T23:51:05.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.422+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66c00b4bd0 con 0x7f66c00d5410 2024-04-26T23:51:05.427 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:51:05.424+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:51:05.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.428+0000 7f66b7e00640 1 -- 172.21.15.46:0/1036862650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v110) ==== 161+0+0 (secure 0 0 0) 0x7f66b000ae20 con 0x7f66c00d5410 2024-04-26T23:51:05.431 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 msgr2=0x7f66a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 0x7f66a006ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f66c00bad80 tx=0x7f66ac008040 comp rx=0 tx=0).stop 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 msgr2=0x7f66c015e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ce30 tx=0x7f66b000a650 comp rx=0 tx=0).stop 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 shutdown_connections 2024-04-26T23:51:05.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f66a006c920 0x7f66a006ede0 secure :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f66c00bad80 tx=0x7f66ac008040 comp rx=0 tx=0).stop 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66c00b9bf0 0x7f66c0161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66c015f4b0 0x7f66c00b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 --2- 172.21.15.46:0/1036862650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c00d5410 0x7f66c015e160 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 >> 172.21.15.46:0/1036862650 conn(0x7f66c008eb50 msgr2=0x7f66c006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.431+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 shutdown_connections 2024-04-26T23:51:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.432+0000 7f66c7000640 1 -- 172.21.15.46:0/1036862650 wait complete. 2024-04-26T23:51:05.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:51:05.509 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:51:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.627+0000 7f0ab1a00640 1 Processor -- start 2024-04-26T23:51:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.627+0000 7f0ab1a00640 1 -- start start 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0aac0d0a50 0x7f0aac0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac163bb0 0x7f0aac163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0aac166fe0 0x7f0aac1673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac06eca0 con 0x7f0aac166fe0 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac164920 con 0x7f0aac0d0a50 2024-04-26T23:51:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac0d1370 con 0x7f0aac163bb0 2024-04-26T23:51:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0aac166fe0 0x7f0aac1673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0aac166fe0 0x7f0aac1673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33306/0 (socket says 172.21.15.46:33306) 2024-04-26T23:51:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 -- 172.21.15.46:0/3336638308 learned_addr learned my addr 172.21.15.46:0/3336638308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 -- 172.21.15.46:0/3336638308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac163bb0 msgr2=0x7f0aac163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 --2- 172.21.15.46:0/3336638308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac163bb0 0x7f0aac163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 -- 172.21.15.46:0/3336638308 >> v2:172.21.15.142:3300/0 conn(0x7f0aac0d0a50 msgr2=0x7f0aac0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:05.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 --2- 172.21.15.46:0/3336638308 >> v2:172.21.15.142:3300/0 conn(0x7f0aac0d0a50 0x7f0aac0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.629+0000 7f0ab1000640 1 -- 172.21.15.46:0/3336638308 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac06dee0 con 0x7f0aac166fe0 2024-04-26T23:51:05.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.630+0000 7f0ab1000640 1 --2- 172.21.15.46:0/3336638308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac166fe0 0x7f0aac1673f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0742d0 tx=0x7f0aa402a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7fb0c4085ebbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0aaaa00640 1 -- 172.21.15.46:0/3336638308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4036070 con 0x7f0aac166fe0 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0aaaa00640 1 -- 172.21.15.46:0/3336638308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa4031600 con 0x7f0aac166fe0 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0aaaa00640 1 -- 172.21.15.46:0/3336638308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40319d0 con 0x7f0aac166fe0 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3336638308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac166fe0 msgr2=0x7f0aac1673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3336638308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac166fe0 0x7f0aac1673f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0742d0 tx=0x7f0aa402a600 comp rx=0 tx=0).stop 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3336638308 shutdown_connections 2024-04-26T23:51:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3336638308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac166fe0 0x7f0aac1673f0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3336638308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac163bb0 0x7f0aac163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3336638308 >> v2:172.21.15.142:3300/0 conn(0x7f0aac0d0a50 0x7f0aac0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.631+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3336638308 >> 172.21.15.46:0/3336638308 conn(0x7f0aac06c350 msgr2=0x7f0aac06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.632+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3336638308 shutdown_connections 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.632+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3336638308 wait complete. 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.633+0000 7f0ab1a00640 1 Processor -- start 2024-04-26T23:51:05.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.633+0000 7f0ab1a00640 1 -- start start 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aac0d0a50 0x7f0aac15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac166fe0 0x7f0aac156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac06e780 con 0x7f0aac163bb0 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac06e4a0 con 0x7f0aac0d0a50 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.634+0000 7f0ab1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac164530 con 0x7f0aac166fe0 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33312/0 (socket says 172.21.15.46:33312) 2024-04-26T23:51:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 -- 172.21.15.46:0/3596917773 learned_addr learned my addr 172.21.15.46:0/3596917773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aabe00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aac0d0a50 0x7f0aac15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0ab1000640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac166fe0 0x7f0aac156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 -- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac166fe0 msgr2=0x7f0aac156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac166fe0 0x7f0aac156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 -- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aac0d0a50 msgr2=0x7f0aac15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aac0d0a50 0x7f0aac15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 -- 172.21.15.46:0/3596917773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac162f60 con 0x7f0aac163bb0 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.635+0000 7f0aab400640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b410 tx=0x7f0a9400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.636+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94015020 con 0x7f0aac163bb0 2024-04-26T23:51:05.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.636+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a94011850 con 0x7f0aac163bb0 2024-04-26T23:51:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.636+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94011c40 con 0x7f0aac163bb0 2024-04-26T23:51:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.636+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aac0723d0 con 0x7f0aac163bb0 2024-04-26T23:51:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.636+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aac155930 con 0x7f0aac163bb0 2024-04-26T23:51:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.637+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aac0693b0 con 0x7f0aac163bb0 2024-04-26T23:51:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.638+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a94018020 con 0x7f0aac163bb0 2024-04-26T23:51:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.639+0000 7f0aa3e00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 0x7f0a8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.639+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6187+0+0 (secure 0 0 0) 0x7f0a9408bc20 con 0x7f0aac163bb0 2024-04-26T23:51:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.639+0000 7f0aabe00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 0x7f0a8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.641+0000 7f0aabe00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 0x7f0a8806ede0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00a770 tx=0x7f0a9c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.643+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a9401f020 con 0x7f0aac163bb0 2024-04-26T23:51:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:05.879+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0aac06f7d0 con 0x7f0aac163bb0 2024-04-26T23:51:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.431+0000 7f0aa3e00640 1 -- 172.21.15.46:0/3596917773 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v111) ==== 104+0+0 (secure 0 0 0) 0x7f0a940172c0 con 0x7f0aac163bb0 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 msgr2=0x7f0a8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 0x7f0a8806ede0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00a770 tx=0x7f0a9c008040 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 msgr2=0x7f0aac151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b410 tx=0x7f0a9400b8d0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 shutdown_connections 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a8806c920 0x7f0a8806ede0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aac166fe0 0x7f0aac156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac163bb0 0x7f0aac151ac0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 --2- 172.21.15.46:0/3596917773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aac0d0a50 0x7f0aac15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.435+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 >> 172.21.15.46:0/3596917773 conn(0x7f0aac06c350 msgr2=0x7f0aac15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.436+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 shutdown_connections 2024-04-26T23:51:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.436+0000 7f0ab1a00640 1 -- 172.21.15.46:0/3596917773 wait complete. 2024-04-26T23:51:06.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:51:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.680+0000 7f0f36c00640 1 Processor -- start 2024-04-26T23:51:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.680+0000 7f0f36c00640 1 -- start start 2024-04-26T23:51:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.681+0000 7f0f36c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f3015f5d0 0x7f0f300d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f36c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d0320 0x7f0f300b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f36c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3015f9b0 con 0x7f0f300d3c20 2024-04-26T23:51:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f36c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0f300d4080 con 0x7f0f3015f5d0 2024-04-26T23:51:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f36c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300d07b0 con 0x7f0f300d0320 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f2fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f3015f5d0 0x7f0f300d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f34c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d0320 0x7f0f300b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f2fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f3015f5d0 0x7f0f300d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:44256/0 (socket says 172.21.15.46:44256) 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.682+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3059055823 learned_addr learned my addr 172.21.15.46:0/3059055823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.683+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d0320 msgr2=0x7f0f300b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.683+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d0320 0x7f0f300b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.683+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3059055823 >> v1:172.21.15.142:6789/0 conn(0x7f0f300d3c20 legacy=0x7f0f300cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.683+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3059055823 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300d0fc0 con 0x7f0f3015f5d0 2024-04-26T23:51:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.684+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300d36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d7d22f50f1121f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.685+0000 7f0f2ea00640 1 -- 172.21.15.46:0/3059055823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037070 con 0x7f0f3015f5d0 2024-04-26T23:51:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.685+0000 7f0f2ea00640 1 -- 172.21.15.46:0/3059055823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20032440 con 0x7f0f3015f5d0 2024-04-26T23:51:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.685+0000 7f0f2ea00640 1 -- 172.21.15.46:0/3059055823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20032810 con 0x7f0f3015f5d0 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 -- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 msgr2=0x7f0f300d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300d36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a250 comp rx=0 tx=0).stop 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 -- 172.21.15.46:0/3059055823 shutdown_connections 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d0320 0x7f0f300b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3059055823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300d36e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 -- 172.21.15.46:0/3059055823 >> 172.21.15.46:0/3059055823 conn(0x7f0f3009e460 msgr2=0x7f0f300d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 -- 172.21.15.46:0/3059055823 shutdown_connections 2024-04-26T23:51:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.686+0000 7f0f36c00640 1 -- 172.21.15.46:0/3059055823 wait complete. 2024-04-26T23:51:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.689+0000 7f0f36c00640 1 Processor -- start 2024-04-26T23:51:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.689+0000 7f0f36c00640 1 -- start start 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d3c20 0x7f0f300b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300ba260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300b1df0 con 0x7f0f300d0320 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300d0ce0 con 0x7f0f3015f5d0 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f36c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300b20d0 con 0x7f0f300d3c20 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.690+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33326/0 (socket says 172.21.15.46:33326) 2024-04-26T23:51:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3711036961 learned_addr learned my addr 172.21.15.46:0/3711036961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d3c20 msgr2=0x7f0f300b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d3c20 0x7f0f300b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 msgr2=0x7f0f300ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 -- 172.21.15.46:0/3711036961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3015b780 con 0x7f0f300d0320 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.691+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.692+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037040 con 0x7f0f300d0320 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.692+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20042070 con 0x7f0f300d0320 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.692+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20030880 con 0x7f0f300d0320 2024-04-26T23:51:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.692+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f300cae30 con 0x7f0f300d0320 2024-04-26T23:51:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.692+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f300cb3f0 con 0x7f0f300d0320 2024-04-26T23:51:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.693+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f30069530 con 0x7f0f300d0320 2024-04-26T23:51:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.694+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f20030a20 con 0x7f0f300d0320 2024-04-26T23:51:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.695+0000 7f0f27e00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.695+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6187+0+0 (secure 0 0 0) 0x7f0f200ac640 con 0x7f0f300d0320 2024-04-26T23:51:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.696+0000 7f0f2f400640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.697+0000 7f0f2f400640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0012e0 tx=0x7f0f1c007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.700+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f20045360 con 0x7f0f300d0320 2024-04-26T23:51:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:06.905+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f300d34c0 con 0x7f0f300d0320 2024-04-26T23:51:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.442+0000 7f0f27e00640 1 -- 172.21.15.46:0/3711036961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) ==== 181+0+0 (secure 0 0 0) 0x7f0f2007a7f0 con 0x7f0f300d0320 2024-04-26T23:51:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:51:07.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.447+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 msgr2=0x7f0f1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.447+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0012e0 tx=0x7f0f1c007d50 comp rx=0 tx=0).stop 2024-04-26T23:51:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.447+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 msgr2=0x7f0f3015b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.447+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f200046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 shutdown_connections 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f3015f5d0 0x7f0f300ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300d3c20 0x7f0f300b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 --2- 172.21.15.46:0/3711036961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f300d0320 0x7f0f3015b240 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 >> 172.21.15.46:0/3711036961 conn(0x7f0f3009e460 msgr2=0x7f0f300b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 shutdown_connections 2024-04-26T23:51:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.448+0000 7f0f36c00640 1 -- 172.21.15.46:0/3711036961 wait complete. 2024-04-26T23:51:07.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:51:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.683+0000 7f398a200640 1 Processor -- start 2024-04-26T23:51:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.683+0000 7f398a200640 1 -- start start 2024-04-26T23:51:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.684+0000 7f398a200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.684+0000 7f398a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.684+0000 7f398a200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398415f9b0 con 0x7f398415f5d0 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.684+0000 7f398a200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39840d0fc0 con 0x7f39840d3df0 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.684+0000 7f398a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39840d0a00 con 0x7f39840d0150 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.685+0000 7f3983e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.685+0000 7f3982a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.685+0000 7f3982a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:44278/0 (socket says 172.21.15.46:44278) 2024-04-26T23:51:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.685+0000 7f3983e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:58476/0 (socket says 172.21.15.46:58476) 2024-04-26T23:51:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.685+0000 7f3982a00640 1 -- 172.21.15.46:0/444092659 learned_addr learned my addr 172.21.15.46:0/444092659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.686+0000 7f3983e00640 1 -- 172.21.15.46:0/444092659 >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 msgr2=0x7f39840cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.686+0000 7f3983e00640 1 --2- 172.21.15.46:0/444092659 >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.686+0000 7f3983e00640 1 -- 172.21.15.46:0/444092659 >> v1:172.21.15.142:6789/0 conn(0x7f398415f5d0 legacy=0x7f39840d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.686+0000 7f3983e00640 1 -- 172.21.15.46:0/444092659 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840d0ce0 con 0x7f39840d0150 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.686+0000 7f3982a00640 1 --2- 172.21.15.46:0/444092659 >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.688+0000 7f3983e00640 1 --2- 172.21.15.46:0/444092659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f397c009d90 tx=0x7f397c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fbeef3f61a00faa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.688+0000 7f3982000640 1 -- 172.21.15.46:0/444092659 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c007740 con 0x7f39840d0150 2024-04-26T23:51:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.688+0000 7f3982000640 1 -- 172.21.15.46:0/444092659 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c0078e0 con 0x7f39840d0150 2024-04-26T23:51:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.688+0000 7f3982000640 1 -- 172.21.15.46:0/444092659 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c007cb0 con 0x7f39840d0150 2024-04-26T23:51:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 -- 172.21.15.46:0/444092659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 msgr2=0x7f39840b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 --2- 172.21.15.46:0/444092659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f397c009d90 tx=0x7f397c02a660 comp rx=0 tx=0).stop 2024-04-26T23:51:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 -- 172.21.15.46:0/444092659 shutdown_connections 2024-04-26T23:51:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 --2- 172.21.15.46:0/444092659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f39840b25f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 --2- 172.21.15.46:0/444092659 >> v2:172.21.15.142:3300/0 conn(0x7f39840d3df0 0x7f39840cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 -- 172.21.15.46:0/444092659 >> 172.21.15.46:0/444092659 conn(0x7f398409e460 msgr2=0x7f39840d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 -- 172.21.15.46:0/444092659 shutdown_connections 2024-04-26T23:51:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.689+0000 7f398a200640 1 -- 172.21.15.46:0/444092659 wait complete. 2024-04-26T23:51:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.691+0000 7f398a200640 1 Processor -- start 2024-04-26T23:51:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.691+0000 7f398a200640 1 -- start start 2024-04-26T23:51:07.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f3984154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f398415f5d0 0x7f39840b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39840d05e0 con 0x7f39840d3df0 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39840b20d0 con 0x7f398415f5d0 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f398a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39840d0760 con 0x7f39840d0150 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f3982a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f3982a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33328/0 (socket says 172.21.15.46:33328) 2024-04-26T23:51:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.692+0000 7f3982a00640 1 -- 172.21.15.46:0/2002611611 learned_addr learned my addr 172.21.15.46:0/2002611611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3983400640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f3984154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3983e00640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f398415f5d0 0x7f39840b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 -- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 msgr2=0x7f3984154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f3984154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 -- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f398415f5d0 msgr2=0x7f39840b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f398415f5d0 0x7f39840b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 -- 172.21.15.46:0/2002611611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840cace0 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.693+0000 7f3982a00640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e9b0 tx=0x7f396c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.694+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c017070 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.694+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c00c9a0 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.694+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39840caf60 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.694+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c00cd50 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.694+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3984160090 con 0x7f39840d3df0 2024-04-26T23:51:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.695+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398415fba0 con 0x7f39840d3df0 2024-04-26T23:51:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.696+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f396c01a070 con 0x7f39840d3df0 2024-04-26T23:51:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.697+0000 7f397b400640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.697+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5749+0+0 (secure 0 0 0) 0x7f396c08be00 con 0x7f39840d3df0 2024-04-26T23:51:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.698+0000 7f3983400640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:07.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.699+0000 7f3983400640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3974001640 tx=0x7f3974009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.702+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39840cace0 con 0x7f39840d3df0 2024-04-26T23:51:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:07.915+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3984069530 con 0x7f39840d3df0 2024-04-26T23:51:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.446+0000 7f397b400640 1 -- 172.21.15.46:0/2002611611 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) ==== 169+0+0 (secure 0 0 0) 0x7f396c05a160 con 0x7f39840d3df0 2024-04-26T23:51:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.449+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 msgr2=0x7f396006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.449+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3974001640 tx=0x7f3974009040 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.449+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 msgr2=0x7f39840ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.449+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e9b0 tx=0x7f396c00ee70 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 shutdown_connections 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f398415f5d0 0x7f39840b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39840d3df0 0x7f39840ca7a0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 --2- 172.21.15.46:0/2002611611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39840d0150 0x7f3984154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 >> 172.21.15.46:0/2002611611 conn(0x7f398409e460 msgr2=0x7f39840b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 shutdown_connections 2024-04-26T23:51:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.450+0000 7f398a200640 1 -- 172.21.15.46:0/2002611611 wait complete. 2024-04-26T23:51:08.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:51:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.684+0000 7f51d3200640 1 Processor -- start 2024-04-26T23:51:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.684+0000 7f51d3200640 1 -- start start 2024-04-26T23:51:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51d4152ba0 0x7f51d40cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51d40cfdb0 0x7f51d406ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f51d4160530 con 0x7f51d40cfdb0 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f51d415f920 con 0x7f51d4152ba0 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51d415fc00 con 0x7f51d4152f80 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51d40cfdb0 0x7f51d406ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51d40cfdb0 0x7f51d406ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33344/0 (socket says 172.21.15.46:33344) 2024-04-26T23:51:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.686+0000 7f51d1400640 1 -- 172.21.15.46:0/1528077818 learned_addr learned my addr 172.21.15.46:0/1528077818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1e00640 1 --2- 172.21.15.46:0/1528077818 >> v2:172.21.15.142:3300/0 conn(0x7f51d4152ba0 0x7f51d40cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d2800640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1400640 1 -- 172.21.15.46:0/1528077818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 msgr2=0x7f51d406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1400640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d406f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1400640 1 -- 172.21.15.46:0/1528077818 >> v2:172.21.15.142:3300/0 conn(0x7f51d4152ba0 msgr2=0x7f51d40cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1400640 1 --2- 172.21.15.46:0/1528077818 >> v2:172.21.15.142:3300/0 conn(0x7f51d4152ba0 0x7f51d40cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d1400640 1 -- 172.21.15.46:0/1528077818 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d415fee0 con 0x7f51d40cfdb0 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.687+0000 7f51d2800640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.688+0000 7f51d1400640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d40cfdb0 0x7f51d406ef70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b34c0 tx=0x7f51bc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b8b1367829d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.688+0000 7f51d0a00640 1 -- 172.21.15.46:0/1528077818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc037070 con 0x7f51d40cfdb0 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.688+0000 7f51d0a00640 1 -- 172.21.15.46:0/1528077818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc032440 con 0x7f51d40cfdb0 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.688+0000 7f51d0a00640 1 -- 172.21.15.46:0/1528077818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc032810 con 0x7f51d40cfdb0 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 -- 172.21.15.46:0/1528077818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d40cfdb0 msgr2=0x7f51d406ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d40cfdb0 0x7f51d406ef70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b34c0 tx=0x7f51bc02a200 comp rx=0 tx=0).stop 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 -- 172.21.15.46:0/1528077818 shutdown_connections 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 --2- 172.21.15.46:0/1528077818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d40cfdb0 0x7f51d406ef70 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 --2- 172.21.15.46:0/1528077818 >> v2:172.21.15.142:3300/0 conn(0x7f51d4152ba0 0x7f51d40cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 -- 172.21.15.46:0/1528077818 >> 172.21.15.46:0/1528077818 conn(0x7f51d40b26d0 msgr2=0x7f51d40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 -- 172.21.15.46:0/1528077818 shutdown_connections 2024-04-26T23:51:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.689+0000 7f51d3200640 1 -- 172.21.15.46:0/1528077818 wait complete. 2024-04-26T23:51:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.691+0000 7f51d3200640 1 Processor -- start 2024-04-26T23:51:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.691+0000 7f51d3200640 1 -- start start 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.692+0000 7f51d3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.692+0000 7f51d3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d4147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51d41440b0 0x7f51d4072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cf4c0 con 0x7f51d4152ba0 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cf340 con 0x7f51d41440b0 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40cf640 con 0x7f51d4152f80 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33348/0 (socket says 172.21.15.46:33348) 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 -- 172.21.15.46:0/1068937264 learned_addr learned my addr 172.21.15.46:0/1068937264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1400640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d4147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d2800640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51d41440b0 0x7f51d4072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 -- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 msgr2=0x7f51d4147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d4147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 -- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51d41440b0 msgr2=0x7f51d4072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51d41440b0 0x7f51d4072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.693+0000 7f51d1e00640 1 -- 172.21.15.46:0/1068937264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d4144fd0 con 0x7f51d4152ba0 2024-04-26T23:51:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51d1e00640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d800 tx=0x7f51c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400b660 con 0x7f51d4152ba0 2024-04-26T23:51:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c4019070 con 0x7f51d4152ba0 2024-04-26T23:51:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400ba20 con 0x7f51d4152ba0 2024-04-26T23:51:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41452b0 con 0x7f51d4152ba0 2024-04-26T23:51:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.694+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d4145870 con 0x7f51d4152ba0 2024-04-26T23:51:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.695+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d4150170 con 0x7f51d4152ba0 2024-04-26T23:51:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.696+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c400bbc0 con 0x7f51d4152ba0 2024-04-26T23:51:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.698+0000 7f51ca000640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 0x7f51b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.698+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7f51c408be50 con 0x7f51d4152ba0 2024-04-26T23:51:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.699+0000 7f51d1400640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 0x7f51b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.700+0000 7f51d1400640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 0x7f51b006ee00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.703+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51d4150170 con 0x7f51d4152ba0 2024-04-26T23:51:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.910+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f51d4145ab0 con 0x7f51d4152ba0 2024-04-26T23:51:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.916+0000 7f51ca000640 1 -- 172.21.15.46:0/1068937264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f51c405a3d0 con 0x7f51d4152ba0 2024-04-26T23:51:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 msgr2=0x7f51b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 0x7f51b006ee00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 msgr2=0x7f51d41448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d800 tx=0x7f51c400dcc0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 shutdown_connections 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51b006c940 0x7f51b006ee00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51d41440b0 0x7f51d4072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51d4152f80 0x7f51d4147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 --2- 172.21.15.46:0/1068937264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d4152ba0 0x7f51d41448d0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 >> 172.21.15.46:0/1068937264 conn(0x7f51d40b26d0 msgr2=0x7f51d406ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 shutdown_connections 2024-04-26T23:51:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:08.919+0000 7f51d3200640 1 -- 172.21.15.46:0/1068937264 wait complete. 2024-04-26T23:51:08.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-26T23:51:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.140+0000 7fa98cc00640 1 Processor -- start 2024-04-26T23:51:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.140+0000 7fa98cc00640 1 -- start start 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.141+0000 7fa98cc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa988159950 0x7fa9880b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.141+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 0x7fa988162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa98cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98806f430 con 0x7fa9881631d0 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa98cc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa988163640 con 0x7fa988159950 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa98cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880af0c0 con 0x7fa9880ae8e0 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa987e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9881631d0 0x7fa98815f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60062/0 (socket says 172.21.15.46:60062) 2024-04-26T23:51:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa987e00640 1 -- 172.21.15.46:0/2469092785 learned_addr learned my addr 172.21.15.46:0/2469092785 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa987400640 1 --2- 172.21.15.46:0/2469092785 >> v2:172.21.15.142:3300/0 conn(0x7fa988159950 0x7fa9880b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa97fe00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 0x7fa988162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.142+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599425528 0 0) 0x7fa98806f430 con 0x7fa9881631d0 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa96c003520 con 0x7fa9881631d0 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1763338738 0 0) 0x7fa96c003520 con 0x7fa9881631d0 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 msgr2=0x7fa988162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 0x7fa988162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 >> v2:172.21.15.142:3300/0 conn(0x7fa988159950 msgr2=0x7fa9880b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 --2- 172.21.15.46:0/2469092785 >> v2:172.21.15.142:3300/0 conn(0x7fa988159950 0x7fa9880b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9880af3a0 con 0x7fa9881631d0 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa9800019f0 con 0x7fa9881631d0 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 >> v1:172.21.15.46:6789/0 conn(0x7fa9881631d0 legacy=0x7fa98815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa96c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 0x7fa96c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96c004d40 0x7fa96c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa988163640 con 0x7fa96c0039b0 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880af0c0 con 0x7fa96c004d40 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c00b9c0 con 0x7fa9880ae8e0 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.143+0000 7fa987400640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa96c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.144+0000 7fa987400640 1 -- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 msgr2=0x7fa96c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.144+0000 7fa987400640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9880ae8e0 0x7fa96c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.144+0000 7fa987400640 1 -- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96c004d40 msgr2=0x7fa96c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.144+0000 7fa987400640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96c004d40 0x7fa96c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.144+0000 7fa987400640 1 -- 172.21.15.46:0/2469092785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c003520 con 0x7fa96c0039b0 2024-04-26T23:51:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.145+0000 7fa987400640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa96c003dd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa978009af0 tx=0x7fa96c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dca3eb1fa8c8bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.145+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978036070 con 0x7fa96c0039b0 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.145+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978031400 con 0x7fa96c0039b0 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.145+0000 7fa986a00640 1 -- 172.21.15.46:0/2469092785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978009690 con 0x7fa96c0039b0 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 -- 172.21.15.46:0/2469092785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 msgr2=0x7fa96c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2469092785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa96c003dd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa978009af0 tx=0x7fa96c005830 comp rx=0 tx=0).stop 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa987e00640 1 -- 172.21.15.46:0/2469092785 reap_dead start 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 -- 172.21.15.46:0/2469092785 shutdown_connections 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 -- 172.21.15.46:0/2469092785 >> 172.21.15.46:0/2469092785 conn(0x7fa9880b2820 msgr2=0x7fa9880b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 -- 172.21.15.46:0/2469092785 shutdown_connections 2024-04-26T23:51:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.146+0000 7fa98cc00640 1 -- 172.21.15.46:0/2469092785 wait complete. 2024-04-26T23:51:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.148+0000 7fa98cc00640 1 Processor -- start 2024-04-26T23:51:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.148+0000 7fa98cc00640 1 -- start start 2024-04-26T23:51:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 0x7fa988073470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa988151540 0x7fa98814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880afc40 con 0x7fa96c0039b0 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880af960 con 0x7fa988155430 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa98cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880aed70 con 0x7fa988151540 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.149+0000 7fa987400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33364/0 (socket says 172.21.15.46:33364) 2024-04-26T23:51:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 -- 172.21.15.46:0/2158816074 learned_addr learned my addr 172.21.15.46:0/2158816074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa97fe00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 0x7fa988073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987e00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa988151540 0x7fa98814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 -- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa988151540 msgr2=0x7fa98814d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa988151540 0x7fa98814d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 -- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 msgr2=0x7fa988073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 0x7fa988073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 -- 172.21.15.46:0/2158816074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa988156f80 con 0x7fa96c0039b0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa97fe00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 0x7fa988073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.150+0000 7fa987400640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa978034040 tx=0x7fa978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.151+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978041020 con 0x7fa96c0039b0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.151+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97802ade0 con 0x7fa96c0039b0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.151+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978036040 con 0x7fa96c0039b0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.151+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881559a0 con 0x7fa96c0039b0 2024-04-26T23:51:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.151+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa988155f60 con 0x7fa96c0039b0 2024-04-26T23:51:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.152+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9880693b0 con 0x7fa96c0039b0 2024-04-26T23:51:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.153+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa978044020 con 0x7fa96c0039b0 2024-04-26T23:51:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.154+0000 7fa97f400640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 0x7fa96406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.154+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7fa9780abf20 con 0x7fa96c0039b0 2024-04-26T23:51:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.155+0000 7fa97fe00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 0x7fa96406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.156+0000 7fa97fe00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 0x7fa96406ede0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa98815d240 tx=0x7fa970008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:09.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.159+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa97804b030 con 0x7fa96c0039b0 2024-04-26T23:51:09.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:09.462+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) -- 0x7fa988157fe0 con 0x7fa96c0039b0 2024-04-26T23:51:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.471+0000 7fa97f400640 1 -- 172.21.15.46:0/2158816074 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v644) ==== 155+0+0 (secure 0 0 0) 0x7fa97807a420 con 0x7fa96c0039b0 2024-04-26T23:51:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.474+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 msgr2=0x7fa96406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.474+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 0x7fa96406ede0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa98815d240 tx=0x7fa970008040 comp rx=0 tx=0).stop 2024-04-26T23:51:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.474+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 msgr2=0x7fa988156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa978034040 tx=0x7fa978004730 comp rx=0 tx=0).stop 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 shutdown_connections 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa96406c920 0x7fa96406ede0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa988151540 0x7fa98814d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988155430 0x7fa988073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 --2- 172.21.15.46:0/2158816074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96c0039b0 0x7fa988156a40 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 >> 172.21.15.46:0/2158816074 conn(0x7fa9880b2820 msgr2=0x7fa9880d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 shutdown_connections 2024-04-26T23:51:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.475+0000 7fa98cc00640 1 -- 172.21.15.46:0/2158816074 wait complete. 2024-04-26T23:51:10.500 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-26T23:51:10.500 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-26T23:51:10.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:10.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.702+0000 7ff46f200640 1 Processor -- start 2024-04-26T23:51:10.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.703+0000 7ff46f200640 1 -- start start 2024-04-26T23:51:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.704+0000 7ff46f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4700d2130 0x7ff4700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.704+0000 7ff46f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.704+0000 7ff46f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4700d11f0 con 0x7ff47015f4e0 2024-04-26T23:51:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.704+0000 7ff46f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff4700d0ce0 con 0x7ff4700d2130 2024-04-26T23:51:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.704+0000 7ff46f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4700d2990 con 0x7ff4700b4000 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4700d2130 0x7ff4700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4700d2130 0x7ff4700d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33372/0 (socket says 172.21.15.46:33372) 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 -- 172.21.15.46:0/2737929694 learned_addr learned my addr 172.21.15.46:0/2737929694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46e800640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 -- 172.21.15.46:0/2737929694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 msgr2=0x7ff4700b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 -- 172.21.15.46:0/2737929694 >> v1:172.21.15.142:6789/0 conn(0x7ff47015f4e0 legacy=0x7ff4700d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.705+0000 7ff46d400640 1 -- 172.21.15.46:0/2737929694 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4700d5f20 con 0x7ff4700d2130 2024-04-26T23:51:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46d400640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff4700d2df0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009e50 tx=0x7ff45c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b5418b28f06e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46ca00640 1 -- 172.21.15.46:0/2737929694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c036070 con 0x7ff4700d2130 2024-04-26T23:51:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46ca00640 1 -- 172.21.15.46:0/2737929694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45c031600 con 0x7ff4700d2130 2024-04-26T23:51:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46ca00640 1 -- 172.21.15.46:0/2737929694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0319d0 con 0x7ff4700d2130 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46f200640 1 -- 172.21.15.46:0/2737929694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 msgr2=0x7ff4700d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.707+0000 7ff46f200640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff4700d2df0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009e50 tx=0x7ff45c02a710 comp rx=0 tx=0).stop 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 -- 172.21.15.46:0/2737929694 shutdown_connections 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 --2- 172.21.15.46:0/2737929694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff4700d2df0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 -- 172.21.15.46:0/2737929694 >> 172.21.15.46:0/2737929694 conn(0x7ff4700d5010 msgr2=0x7ff4700d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 -- 172.21.15.46:0/2737929694 shutdown_connections 2024-04-26T23:51:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.708+0000 7ff46f200640 1 -- 172.21.15.46:0/2737929694 wait complete. 2024-04-26T23:51:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.710+0000 7ff46f200640 1 Processor -- start 2024-04-26T23:51:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.710+0000 7ff46f200640 1 -- start start 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47015f4e0 0x7ff470155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700b1d60 con 0x7ff4700d2130 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700d5c40 con 0x7ff47015f4e0 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.711+0000 7ff46f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff47015f8c0 con 0x7ff4700b4000 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33374/0 (socket says 172.21.15.46:33374) 2024-04-26T23:51:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 -- 172.21.15.46:0/629172936 learned_addr learned my addr 172.21.15.46:0/629172936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46e800640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47015f4e0 0x7ff470155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46de00640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 -- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 msgr2=0x7ff4700b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 -- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47015f4e0 msgr2=0x7ff470155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47015f4e0 0x7ff470155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46d400640 1 -- 172.21.15.46:0/629172936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4700a20f0 con 0x7ff4700d2130 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.712+0000 7ff46de00640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff46d400640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f70 tx=0x7ff45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c036050 con 0x7ff4700d2130 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45c042070 con 0x7ff4700d2130 2024-04-26T23:51:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c009510 con 0x7ff4700d2130 2024-04-26T23:51:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4700a23d0 con 0x7ff4700d2130 2024-04-26T23:51:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.713+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701f58a0 con 0x7ff4700d2130 2024-04-26T23:51:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.715+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4701f53b0 con 0x7ff4700d2130 2024-04-26T23:51:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.716+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff45c007c70 con 0x7ff4700d2130 2024-04-26T23:51:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.717+0000 7ff466000640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 0x7ff45006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.717+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7ff45c0aca30 con 0x7ff4700d2130 2024-04-26T23:51:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.718+0000 7ff46de00640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 0x7ff45006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.719+0000 7ff46de00640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 0x7ff45006edb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4600015e0 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.722+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff45c07aea0 con 0x7ff4700d2130 2024-04-26T23:51:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.928+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4700d46d0 con 0x7ff4700d2130 2024-04-26T23:51:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.930+0000 7ff466000640 1 -- 172.21.15.46:0/629172936 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+13267 (secure 0 0 0) 0x7ff45c07a7a0 con 0x7ff4700d2130 2024-04-26T23:51:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:10.934 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:08.442515+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/2039062774":"2024-04-27T23:51:05.423895+0000","172.21.15.46:6832/2039062774":"2024-04-27T23:51:05.423895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 msgr2=0x7ff45006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 0x7ff45006edb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4600015e0 tx=0x7ff460009040 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 msgr2=0x7ff470155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f70 tx=0x7ff45c004a40 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 shutdown_connections 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff45006c8f0 0x7ff45006edb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47015f4e0 0x7ff470155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4700d2130 0x7ff470155010 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 --2- 172.21.15.46:0/629172936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4700b4000 0x7ff4700b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 >> 172.21.15.46:0/629172936 conn(0x7ff4700d5010 msgr2=0x7ff4700cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 shutdown_connections 2024-04-26T23:51:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:10.933+0000 7ff46f200640 1 -- 172.21.15.46:0/629172936 wait complete. 2024-04-26T23:51:10.956 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:51:10.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.171+0000 7f03de000640 1 Processor -- start 2024-04-26T23:51:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.171+0000 7f03de000640 1 -- start start 2024-04-26T23:51:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03de000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03d815cec0 0x7f03d80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03de000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80ce7d0 con 0x7f03d8073d00 2024-04-26T23:51:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03de000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80ceab0 con 0x7f03d80cdb30 2024-04-26T23:51:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03de000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f03d80ced90 con 0x7f03d815cec0 2024-04-26T23:51:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03dd600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03d8073d00 0x7f03d80b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60066/0 (socket says 172.21.15.46:60066) 2024-04-26T23:51:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.173+0000 7f03dd600640 1 -- 172.21.15.46:0/1282246860 learned_addr learned my addr 172.21.15.46:0/1282246860 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03dcc00640 1 --2- 172.21.15.46:0/1282246860 >> v2:172.21.15.142:3300/0 conn(0x7f03d815cec0 0x7f03d80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1371478417 0 0) 0x7f03d80ce7d0 con 0x7f03d8073d00 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03c0003520 con 0x7f03d8073d00 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2262140475 0 0) 0x7f03c0003520 con 0x7f03d8073d00 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 >> v2:172.21.15.142:3300/0 conn(0x7f03d815cec0 msgr2=0x7f03d80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 --2- 172.21.15.46:0/1282246860 >> v2:172.21.15.142:3300/0 conn(0x7f03d815cec0 0x7f03d80cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 >> v1:172.21.15.142:6789/0 conn(0x7f03d80cdb30 legacy=0x7f03d80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d80cf070 con 0x7f03d8073d00 2024-04-26T23:51:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f03d00019f0 con 0x7f03d8073d00 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 >> v1:172.21.15.46:6789/0 conn(0x7f03d8073d00 legacy=0x7f03d80b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.174+0000 7f03d7400640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 0x7f03c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7400640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03c0004800 0x7f03c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7400640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00051c0 0x7f03c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03d80ced90 con 0x7f03c00039b0 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03c000c060 con 0x7f03c00051c0 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03c000c260 con 0x7f03c0004800 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 0x7f03c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03d7e00640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03c0004800 0x7f03c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dd600640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00051c0 0x7f03c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 -- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03c0004800 msgr2=0x7f03c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03c0004800 0x7f03c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 -- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00051c0 msgr2=0x7f03c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00051c0 0x7f03c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dcc00640 1 -- 172.21.15.46:0/1282246860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c0003520 con 0x7f03c00039b0 2024-04-26T23:51:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dd600640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00051c0 0x7f03c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.175+0000 7f03dd600640 1 -- 172.21.15.46:0/1282246860 reap_dead start 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.177+0000 7f03dcc00640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 0x7f03c0003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009ac0 tx=0x7f03c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f950d77671fddd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.177+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc036070 con 0x7f03c00039b0 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.177+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc031400 con 0x7f03c00039b0 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.177+0000 7f03d7400640 1 -- 172.21.15.46:0/1282246860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc009690 con 0x7f03c00039b0 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 -- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 msgr2=0x7f03c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 0x7f03c0003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009ac0 tx=0x7f03c0005830 comp rx=0 tx=0).stop 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 -- 172.21.15.46:0/1282246860 shutdown_connections 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 --2- 172.21.15.46:0/1282246860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c00039b0 0x7f03c0003dd0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 -- 172.21.15.46:0/1282246860 >> 172.21.15.46:0/1282246860 conn(0x7f03d80b27b0 msgr2=0x7f03d80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 -- 172.21.15.46:0/1282246860 shutdown_connections 2024-04-26T23:51:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.178+0000 7f03de000640 1 -- 172.21.15.46:0/1282246860 wait complete. 2024-04-26T23:51:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.180+0000 7f03de000640 1 Processor -- start 2024-04-26T23:51:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.180+0000 7f03de000640 1 -- start start 2024-04-26T23:51:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00039b0 0x7f03d80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03d80cb250 0x7f03d80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03d80cd5f0 con 0x7f03d80b2100 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03d815d2a0 con 0x7f03c00039b0 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03de000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03d80cd770 con 0x7f03d80cb250 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33396/0 (socket says 172.21.15.46:33396) 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03dd600640 1 -- 172.21.15.46:0/3161549306 learned_addr learned my addr 172.21.15.46:0/3161549306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03d7e00640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03d80cb250 0x7f03d80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.181+0000 7f03dcc00640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00039b0 0x7f03d80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 -- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03d80cb250 msgr2=0x7f03d80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03d80cb250 0x7f03d80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 -- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00039b0 msgr2=0x7f03d80b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00039b0 0x7f03d80b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 -- 172.21.15.46:0/3161549306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d815d670 con 0x7f03d80b2100 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03dd600640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f03d80cfe50 tx=0x7f03d000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000b800 con 0x7f03d80b2100 2024-04-26T23:51:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d000b9c0 con 0x7f03d80b2100 2024-04-26T23:51:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000bdb0 con 0x7f03d80b2100 2024-04-26T23:51:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.182+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d815d950 con 0x7f03d80b2100 2024-04-26T23:51:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.183+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d80b51d0 con 0x7f03d80b2100 2024-04-26T23:51:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.184+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d80693b0 con 0x7f03d80b2100 2024-04-26T23:51:11.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.185+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03d001b020 con 0x7f03d80b2100 2024-04-26T23:51:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.186+0000 7f03d4c00640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 0x7f03bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.186+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7f03d008b9f0 con 0x7f03d80b2100 2024-04-26T23:51:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.187+0000 7f03dcc00640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 0x7f03bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.188+0000 7f03dcc00640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 0x7f03bc06ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f03cc033000 tx=0x7f03cc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.190+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03d0059f80 con 0x7f03d80b2100 2024-04-26T23:51:11.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.393+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03d80d0110 con 0x7f03d80b2100 2024-04-26T23:51:11.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.394+0000 7f03d4c00640 1 -- 172.21.15.46:0/3161549306 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) ==== 93+0+10 (secure 0 0 0) 0x7f03d0059880 con 0x7f03d80b2100 2024-04-26T23:51:11.397 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 msgr2=0x7f03bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 0x7f03bc06ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f03cc033000 tx=0x7f03cc008030 comp rx=0 tx=0).stop 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 msgr2=0x7f03d80cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f03d80cfe50 tx=0x7f03d000def0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 shutdown_connections 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c920 0x7f03bc06ede0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d80b2100 0x7f03d80cc740 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03d80cb250 0x7f03d80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 --2- 172.21.15.46:0/3161549306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03c00039b0 0x7f03d80b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 >> 172.21.15.46:0/3161549306 conn(0x7f03d80b27b0 msgr2=0x7f03d8071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 shutdown_connections 2024-04-26T23:51:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.396+0000 7f03de000640 1 -- 172.21.15.46:0/3161549306 wait complete. 2024-04-26T23:51:11.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-26T23:51:11.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.626+0000 7f25fbe00640 1 Processor -- start 2024-04-26T23:51:11.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.626+0000 7f25fbe00640 1 -- start start 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.628+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25fc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.628+0000 7f25fbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc073ad0 con 0x7f25fc160910 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.628+0000 7f25fbe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc146a90 con 0x7f25fc1466b0 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.628+0000 7f25fbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc073d70 con 0x7f25fc0720e0 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.629+0000 7f25fb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f25fc160910 0x7f25fc0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60070/0 (socket says 172.21.15.46:60070) 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.629+0000 7f25fb400640 1 -- 172.21.15.46:0/3466079420 learned_addr learned my addr 172.21.15.46:0/3466079420 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.629+0000 7f25f3e00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25fc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.629+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810018316 0 0) 0x7f25fc073ad0 con 0x7f25fc160910 2024-04-26T23:51:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.629+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25e0003520 con 0x7f25fc160910 2024-04-26T23:51:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455107006 0 0) 0x7f25e0003520 con 0x7f25fc160910 2024-04-26T23:51:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 msgr2=0x7f25fc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25fc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 >> v1:172.21.15.142:6789/0 conn(0x7f25fc1466b0 legacy=0x7f25fc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc074050 con 0x7f25fc160910 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f25f40019f0 con 0x7f25fc160910 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 >> v1:172.21.15.46:6789/0 conn(0x7f25fc160910 legacy=0x7f25fc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e00039e0 0x7f25e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0004800 0x7f25e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc073d70 con 0x7f25e00039e0 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25e000bad0 con 0x7f25e0004800 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25e000bcd0 con 0x7f25fc0720e0 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25faa00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e00039e0 0x7f25e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25fb400640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25faa00640 1 -- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 msgr2=0x7f25e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25faa00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25f3e00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0004800 0x7f25e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25faa00640 1 -- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0004800 msgr2=0x7f25e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.630+0000 7f25faa00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0004800 0x7f25e0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.631+0000 7f25faa00640 1 -- 172.21.15.46:0/3466079420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e0003520 con 0x7f25e00039e0 2024-04-26T23:51:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.631+0000 7f25fb400640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25fc0720e0 0x7f25e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.632+0000 7f25faa00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e00039e0 0x7f25e0003e00 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f25ec001720 tx=0x7f25e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5825365a85cd9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.632+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec001370 con 0x7f25e00039e0 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.632+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec0052d0 con 0x7f25e00039e0 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.632+0000 7f25fa000640 1 -- 172.21.15.46:0/3466079420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec02ab90 con 0x7f25e00039e0 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 -- 172.21.15.46:0/3466079420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e00039e0 msgr2=0x7f25e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 --2- 172.21.15.46:0/3466079420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e00039e0 0x7f25e0003e00 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f25ec001720 tx=0x7f25e0005830 comp rx=0 tx=0).stop 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fb400640 1 -- 172.21.15.46:0/3466079420 reap_dead start 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 -- 172.21.15.46:0/3466079420 shutdown_connections 2024-04-26T23:51:11.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 -- 172.21.15.46:0/3466079420 >> 172.21.15.46:0/3466079420 conn(0x7f25fc0b26d0 msgr2=0x7f25fc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:11.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 -- 172.21.15.46:0/3466079420 shutdown_connections 2024-04-26T23:51:11.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.633+0000 7f25fbe00640 1 -- 172.21.15.46:0/3466079420 wait complete. 2024-04-26T23:51:11.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.635+0000 7f25fbe00640 1 Processor -- start 2024-04-26T23:51:11.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.635+0000 7f25fbe00640 1 -- start start 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 0x7f25fc13f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 0x7f25fc15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0cc0a0 con 0x7f25fc15f320 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0cb470 con 0x7f25fc15e320 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc0cc220 con 0x7f25e00039e0 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33420/0 (socket says 172.21.15.46:33420) 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25f3e00640 1 -- 172.21.15.46:0/2891845877 learned_addr learned my addr 172.21.15.46:0/2891845877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25faa00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 0x7f25fc13f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.636+0000 7f25fb400640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 0x7f25fc15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 -- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 msgr2=0x7f25fc13f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 0x7f25fc13f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 -- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 msgr2=0x7f25fc15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 0x7f25fc15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 -- 172.21.15.46:0/2891845877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0cf940 con 0x7f25fc15f320 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25faa00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 0x7f25fc13f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25fb400640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 0x7f25fc15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f3e00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f25e4010860 tx=0x7f25e4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400f040 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e400eb00 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e401c400 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc0cfc20 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.637+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc0c7b70 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.638+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0693b0 con 0x7f25fc15f320 2024-04-26T23:51:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.640+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25e401c5a0 con 0x7f25fc15f320 2024-04-26T23:51:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.640+0000 7f25f2a00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 0x7f25d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.641+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7f25e408bfe0 con 0x7f25fc15f320 2024-04-26T23:51:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.641+0000 7f25faa00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 0x7f25d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.642+0000 7f25faa00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 0x7f25d806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f25ec032000 tx=0x7f25ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.645+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25e4018030 con 0x7f25fc15f320 2024-04-26T23:51:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:11.929+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) -- 0x7f25fc0c7180 con 0x7f25fc15f320 2024-04-26T23:51:12.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:12.409 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.483+0000 7f25f2a00640 1 -- 172.21.15.46:0/2891845877 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v646) ==== 169+0+0 (secure 0 0 0) 0x7f25e4059ef0 con 0x7f25fc15f320 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 msgr2=0x7f25d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 0x7f25d806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f25ec032000 tx=0x7f25ec02e000 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 msgr2=0x7f25fc0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f25e4010860 tx=0x7f25e4010d20 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 shutdown_connections 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25d806c8f0 0x7f25d806edb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc15e320 0x7f25fc15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25fc15f320 0x7f25fc0cf080 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 --2- 172.21.15.46:0/2891845877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00039e0 0x7f25fc13f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 >> 172.21.15.46:0/2891845877 conn(0x7f25fc0b26d0 msgr2=0x7f25fc0726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 shutdown_connections 2024-04-26T23:51:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.486+0000 7f25fbe00640 1 -- 172.21.15.46:0/2891845877 wait complete. 2024-04-26T23:51:12.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:51:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.721+0000 7ff615c00640 1 Processor -- start 2024-04-26T23:51:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.721+0000 7ff615c00640 1 -- start start 2024-04-26T23:51:12.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff615c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff61015d730 0x7ff6100d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff615c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff61015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff615c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61006eb90 con 0x7ff61015e050 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff615c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff6100b4310 con 0x7ff61015d730 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff615c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6100b45f0 con 0x7ff6100b53b0 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff60ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff61015d730 0x7ff6100d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff60ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff61015d730 0x7ff6100d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48856/0 (socket says 172.21.15.46:48856) 2024-04-26T23:51:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.723+0000 7ff60ea00640 1 -- 172.21.15.46:0/3647565858 learned_addr learned my addr 172.21.15.46:0/3647565858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.724+0000 7ff60e000640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff61015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.724+0000 7ff60ea00640 1 -- 172.21.15.46:0/3647565858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 msgr2=0x7ff61015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.724+0000 7ff60ea00640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff61015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.724+0000 7ff60ea00640 1 -- 172.21.15.46:0/3647565858 >> v1:172.21.15.142:6789/0 conn(0x7ff61015e050 legacy=0x7ff61006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.724+0000 7ff60ea00640 1 -- 172.21.15.46:0/3647565858 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6100b48d0 con 0x7ff61015d730 2024-04-26T23:51:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.725+0000 7ff60ea00640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100d00d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff600009870 tx=0x7ff60002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a58adbd9e8ae003 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.725+0000 7ff60d600640 1 -- 172.21.15.46:0/3647565858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600037070 con 0x7ff61015d730 2024-04-26T23:51:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.725+0000 7ff60d600640 1 -- 172.21.15.46:0/3647565858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff600032440 con 0x7ff61015d730 2024-04-26T23:51:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.725+0000 7ff60d600640 1 -- 172.21.15.46:0/3647565858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600032810 con 0x7ff61015d730 2024-04-26T23:51:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 -- 172.21.15.46:0/3647565858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 msgr2=0x7ff6100d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100d00d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff600009870 tx=0x7ff60002a210 comp rx=0 tx=0).stop 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 -- 172.21.15.46:0/3647565858 shutdown_connections 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff61015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 --2- 172.21.15.46:0/3647565858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100d00d0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 -- 172.21.15.46:0/3647565858 >> 172.21.15.46:0/3647565858 conn(0x7ff61006c980 msgr2=0x7ff6100b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 -- 172.21.15.46:0/3647565858 shutdown_connections 2024-04-26T23:51:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.726+0000 7ff615c00640 1 -- 172.21.15.46:0/3647565858 wait complete. 2024-04-26T23:51:12.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.728+0000 7ff615c00640 1 Processor -- start 2024-04-26T23:51:12.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.728+0000 7ff615c00640 1 -- start start 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff610159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff61015e050 0x7ff6100b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff61015e4c0 con 0x7ff61015d730 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6100b4e90 con 0x7ff61015e050 2024-04-26T23:51:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff615c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6100b3ec0 con 0x7ff6100b53b0 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff60e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff60e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48866/0 (socket says 172.21.15.46:48866) 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.729+0000 7ff60e000640 1 -- 172.21.15.46:0/1730097868 learned_addr learned my addr 172.21.15.46:0/1730097868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60ea00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff610159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60f400640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff61015e050 0x7ff6100b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 -- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 msgr2=0x7ff610159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff610159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 -- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff61015e050 msgr2=0x7ff6100b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff61015e050 0x7ff6100b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 -- 172.21.15.46:0/1730097868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6100b1ad0 con 0x7ff61015d730 2024-04-26T23:51:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60ea00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff610159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.730+0000 7ff60e000640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d620 tx=0x7ff5fc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.731+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00b620 con 0x7ff61015d730 2024-04-26T23:51:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.731+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc019070 con 0x7ff61015d730 2024-04-26T23:51:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.731+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00ba00 con 0x7ff61015d730 2024-04-26T23:51:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.731+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6100b1db0 con 0x7ff61015d730 2024-04-26T23:51:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.731+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6100b2370 con 0x7ff61015d730 2024-04-26T23:51:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.732+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff610069530 con 0x7ff61015d730 2024-04-26T23:51:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.733+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5fc00bba0 con 0x7ff61015d730 2024-04-26T23:51:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.735+0000 7ff606a00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.735+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7ff5fc08bde0 con 0x7ff61015d730 2024-04-26T23:51:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.735+0000 7ff60ea00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.736+0000 7ff60ea00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 0x7ff5f006edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff600034000 tx=0x7ff60002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.739+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5fc05a370 con 0x7ff61015d730 2024-04-26T23:51:12.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.943+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff61015dba0 con 0x7ff61015d730 2024-04-26T23:51:12.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.948+0000 7ff606a00640 1 -- 172.21.15.46:0/1730097868 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7ff5fc060030 con 0x7ff61015d730 2024-04-26T23:51:12.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.951+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 msgr2=0x7ff5f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.951+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 0x7ff5f006edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff600034000 tx=0x7ff60002e000 comp rx=0 tx=0).stop 2024-04-26T23:51:12.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.951+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 msgr2=0x7ff6100b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:12.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.951+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d620 tx=0x7ff5fc00dae0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 shutdown_connections 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff61015e050 0x7ff6100b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61015d730 0x7ff6100b0f10 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 --2- 172.21.15.46:0/1730097868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6100b53b0 0x7ff610159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 >> 172.21.15.46:0/1730097868 conn(0x7ff61006c980 msgr2=0x7ff6100b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 shutdown_connections 2024-04-26T23:51:12.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:12.952+0000 7ff615c00640 1 -- 172.21.15.46:0/1730097868 wait complete. 2024-04-26T23:51:12.976 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:51:12.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:51:13.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:51:13.191 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:51:13.191 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:51:13.192 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:13.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:51:13.252 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:51:13.253 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:13.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:51:13.310 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:51:13.310 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:13.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:51:13.370 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:51:13.370 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:13.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:51:13.425 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:51:13.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.642+0000 7fc6eee00640 1 Processor -- start 2024-04-26T23:51:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.642+0000 7fc6eee00640 1 -- start start 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.644+0000 7fc6eee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6e80720e0 0x7fc6e80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.644+0000 7fc6eee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e8073ad0 con 0x7fc6e8160910 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.644+0000 7fc6eee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e8146a90 con 0x7fc6e81466b0 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.644+0000 7fc6eee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e8073d70 con 0x7fc6e80720e0 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6e80720e0 0x7fc6e80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6e80720e0 0x7fc6e80cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48870/0 (socket says 172.21.15.46:48870) 2024-04-26T23:51:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 -- 172.21.15.46:0/997503618 learned_addr learned my addr 172.21.15.46:0/997503618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 -- 172.21.15.46:0/997503618 >> v1:172.21.15.142:6789/0 conn(0x7fc6e81466b0 legacy=0x7fc6e806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 -- 172.21.15.46:0/997503618 >> v1:172.21.15.46:6789/0 conn(0x7fc6e8160910 legacy=0x7fc6e80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.645+0000 7fc6ed000640 1 -- 172.21.15.46:0/997503618 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8074050 con 0x7fc6e80720e0 2024-04-26T23:51:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.646+0000 7fc6ed000640 1 --2- 172.21.15.46:0/997503618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e80720e0 0x7fc6e80cd860 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40098a0 tx=0x7fc6d402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a25cbbf06315756 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:13.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6e7e00640 1 -- 172.21.15.46:0/997503618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4037070 con 0x7fc6e80720e0 2024-04-26T23:51:13.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6e7e00640 1 -- 172.21.15.46:0/997503618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d4032440 con 0x7fc6e80720e0 2024-04-26T23:51:13.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6e7e00640 1 -- 172.21.15.46:0/997503618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4032810 con 0x7fc6e80720e0 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6eee00640 1 -- 172.21.15.46:0/997503618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e80720e0 msgr2=0x7fc6e80cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6eee00640 1 --2- 172.21.15.46:0/997503618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e80720e0 0x7fc6e80cd860 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40098a0 tx=0x7fc6d402a2e0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6eee00640 1 -- 172.21.15.46:0/997503618 shutdown_connections 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6eee00640 1 --2- 172.21.15.46:0/997503618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e80720e0 0x7fc6e80cd860 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.647+0000 7fc6eee00640 1 -- 172.21.15.46:0/997503618 >> 172.21.15.46:0/997503618 conn(0x7fc6e80b26d0 msgr2=0x7fc6e80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.648+0000 7fc6eee00640 1 -- 172.21.15.46:0/997503618 shutdown_connections 2024-04-26T23:51:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.648+0000 7fc6eee00640 1 -- 172.21.15.46:0/997503618 wait complete. 2024-04-26T23:51:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.649+0000 7fc6eee00640 1 Processor -- start 2024-04-26T23:51:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.649+0000 7fc6eee00640 1 -- start start 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e80720e0 0x7fc6e80b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e8160910 0x7fc6e80b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80cdda0 con 0x7fc6e81466b0 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80cb470 con 0x7fc6e80720e0 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.650+0000 7fc6eee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e80cc0a0 con 0x7fc6e8160910 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48878/0 (socket says 172.21.15.46:48878) 2024-04-26T23:51:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 -- 172.21.15.46:0/3965827111 learned_addr learned my addr 172.21.15.46:0/3965827111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 -- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e8160910 msgr2=0x7fc6e80b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e8160910 0x7fc6e80b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 -- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e80720e0 msgr2=0x7fc6e80b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e80720e0 0x7fc6e80b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 -- 172.21.15.46:0/3965827111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e80ce670 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.651+0000 7fc6ed000640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40097f0 tx=0x7fc6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.652+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4041070 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.652+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d4004220 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.652+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4037040 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.652+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e80ce910 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.652+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e80ceed0 con 0x7fc6e81466b0 2024-04-26T23:51:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.653+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e80693b0 con 0x7fc6e81466b0 2024-04-26T23:51:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.654+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6d4003610 con 0x7fc6e81466b0 2024-04-26T23:51:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.655+0000 7fc6e5600640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 0x7fc6c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.655+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7fc6d40abfe0 con 0x7fc6e81466b0 2024-04-26T23:51:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.656+0000 7fc6eda00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 0x7fc6c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.657+0000 7fc6eda00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 0x7fc6c806ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0043c0 tx=0x7fc6dc007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.659+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6d407a5d0 con 0x7fc6e81466b0 2024-04-26T23:51:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.903+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6e80c7180 con 0x7fc6e81466b0 2024-04-26T23:51:13.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.905+0000 7fc6e5600640 1 -- 172.21.15.46:0/3965827111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3874 (secure 0 0 0) 0x7fc6d4079ed0 con 0x7fc6e81466b0 2024-04-26T23:51:13.909 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:13.909 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":111,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3284721193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3284721193},{"type":"v1","addr":"172.21.15.142:6836","nonce":3284721193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/11200390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":11200390},{"type":"v1","addr":"172.21.15.46:6835","nonce":11200390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3174850067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3174850067},{"type":"v1","addr":"172.21.15.142:6837","nonce":3174850067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":9591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4139965057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4139965057},{"type":"v1","addr":"172.21.15.46:6833","nonce":4139965057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[]} 2024-04-26T23:51:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 111 2024-04-26T23:51:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 msgr2=0x7fc6c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 0x7fc6c806ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0043c0 tx=0x7fc6dc007d30 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 msgr2=0x7fc6e80cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40097f0 tx=0x7fc6d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 shutdown_connections 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc6c806c920 0x7fc6c806ede0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e8160910 0x7fc6e80b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e81466b0 0x7fc6e80cf9b0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 --2- 172.21.15.46:0/3965827111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e80720e0 0x7fc6e80b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 >> 172.21.15.46:0/3965827111 conn(0x7fc6e80b26d0 msgr2=0x7fc6e80726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 shutdown_connections 2024-04-26T23:51:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:13.908+0000 7fc6eee00640 1 -- 172.21.15.46:0/3965827111 wait complete. 2024-04-26T23:51:13.932 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:51:13.932 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:51:13.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:51:13.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:51:13.932 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:51:13.932 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:51:13.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:51:13.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:51:13.933 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:51:13.933 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:51:13.933 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:51:13.933 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:51:13.933 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:51:13.933 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:51:13.934 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:51:13.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:51:13.949 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:51:13.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:51:13.951 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:51:13.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:51:13.957 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:51:13.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:51:13.960 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:51:13.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:51:13.995 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:51:13.996 DEBUG:teuthology.parallel:result is None 2024-04-26T23:51:13.999 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:51:13.999 DEBUG:teuthology.parallel:result is None 2024-04-26T23:51:14.007 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:51:14.007 DEBUG:teuthology.parallel:result is None 2024-04-26T23:51:14.013 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:51:14.014 DEBUG:teuthology.parallel:result is None 2024-04-26T23:51:14.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:51:14.145 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:51:14.146 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:51:14.164 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:51:14.164 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:51:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.191+0000 7f205de00640 1 Processor -- start 2024-04-26T23:51:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.191+0000 7f205de00640 1 -- start start 2024-04-26T23:51:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.192+0000 7f205de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f205815f4b0 0x7f20580d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.192+0000 7f205de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.192+0000 7f205de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20580cfd60 con 0x7f20580b5140 2024-04-26T23:51:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.192+0000 7f205de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f205815f890 con 0x7f205815f4b0 2024-04-26T23:51:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.192+0000 7f205de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20580d2910 con 0x7f20580d55e0 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2056a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f205815f4b0 0x7f20580d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:51136/0 (socket says 172.21.15.46:51136) 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 -- 172.21.15.46:0/3420735587 learned_addr learned my addr 172.21.15.46:0/3420735587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 -- 172.21.15.46:0/3420735587 >> v2:172.21.15.46:3300/0 conn(0x7f205815f4b0 msgr2=0x7f20580d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 --2- 172.21.15.46:0/3420735587 >> v2:172.21.15.46:3300/0 conn(0x7f205815f4b0 0x7f20580d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 -- 172.21.15.46:0/3420735587 >> v1:172.21.15.142:6789/0 conn(0x7f20580b5140 legacy=0x7f20580d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.193+0000 7f2057400640 1 -- 172.21.15.46:0/3420735587 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20580b55d0 con 0x7f20580d55e0 2024-04-26T23:51:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.195+0000 7f2057400640 1 --2- 172.21.15.46:0/3420735587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2050009e50 tx=0x7f205002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ec289da5102364 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.195+0000 7f2056000640 1 -- 172.21.15.46:0/3420735587 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050036070 con 0x7f20580d55e0 2024-04-26T23:51:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.195+0000 7f2056000640 1 -- 172.21.15.46:0/3420735587 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2050031600 con 0x7f20580d55e0 2024-04-26T23:51:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.195+0000 7f2056000640 1 -- 172.21.15.46:0/3420735587 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500319d0 con 0x7f20580d55e0 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 -- 172.21.15.46:0/3420735587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 msgr2=0x7f20580d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 --2- 172.21.15.46:0/3420735587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2050009e50 tx=0x7f205002a710 comp rx=0 tx=0).stop 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 -- 172.21.15.46:0/3420735587 shutdown_connections 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 --2- 172.21.15.46:0/3420735587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20580d2f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 --2- 172.21.15.46:0/3420735587 >> v2:172.21.15.46:3300/0 conn(0x7f205815f4b0 0x7f20580d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 -- 172.21.15.46:0/3420735587 >> 172.21.15.46:0/3420735587 conn(0x7f205808eb50 msgr2=0x7f205809e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 -- 172.21.15.46:0/3420735587 shutdown_connections 2024-04-26T23:51:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.196+0000 7f205de00640 1 -- 172.21.15.46:0/3420735587 wait complete. 2024-04-26T23:51:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.197+0000 7f205de00640 1 Processor -- start 2024-04-26T23:51:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.198+0000 7f205de00640 1 -- start start 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.198+0000 7f205de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20581f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.198+0000 7f205de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205815f4b0 0x7f205815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f205de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f205de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20580b5e70 con 0x7f2058161e00 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f205de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20580b5b90 con 0x7f205815f4b0 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f205de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20580b4680 con 0x7f20580d55e0 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48932/0 (socket says 172.21.15.46:48932) 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2056a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20581f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 -- 172.21.15.46:0/2726157395 learned_addr learned my addr 172.21.15.46:0/2726157395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f204fe00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205815f4b0 0x7f205815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 -- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 msgr2=0x7f20581f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20581f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 -- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205815f4b0 msgr2=0x7f205815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205815f4b0 0x7f205815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.199+0000 7f2057400640 1 -- 172.21.15.46:0/2726157395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20581eb580 con 0x7f2058161e00 2024-04-26T23:51:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f2057400640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2050009f70 tx=0x7f2050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205003f020 con 0x7f2058161e00 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2050037040 con 0x7f2058161e00 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2058160d60 con 0x7f2058161e00 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050036070 con 0x7f2058161e00 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.200+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2058161320 con 0x7f2058161e00 2024-04-26T23:51:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.202+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2058069530 con 0x7f2058161e00 2024-04-26T23:51:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.203+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2050042020 con 0x7f2058161e00 2024-04-26T23:51:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.204+0000 7f204f400640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 0x7f203406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.204+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5335+0+0 (secure 0 0 0) 0x7f20500abd70 con 0x7f2058161e00 2024-04-26T23:51:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.204+0000 7f2056a00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 0x7f203406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.205+0000 7f2056a00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 0x7f203406edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f205806e6e0 tx=0x7f2048009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.209+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2050049030 con 0x7f2058161e00 2024-04-26T23:51:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.423+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2058075960 con 0x7f2058161e00 2024-04-26T23:51:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.512+0000 7f204f400640 1 -- 172.21.15.46:0/2726157395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v142) ==== 97+0+0 (secure 0 0 0) 0x7f2050079c20 con 0x7f2058161e00 2024-04-26T23:51:14.517 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:51:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 msgr2=0x7f203406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 0x7f203406edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f205806e6e0 tx=0x7f2048009190 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 msgr2=0x7f20581eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2050009f70 tx=0x7f2050004a40 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 shutdown_connections 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f203406c8f0 0x7f203406edb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2058161e00 0x7f20581eb040 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205815f4b0 0x7f205815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 --2- 172.21.15.46:0/2726157395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20580d55e0 0x7f20581f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 >> 172.21.15.46:0/2726157395 conn(0x7f205808eb50 msgr2=0x7f20580d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 shutdown_connections 2024-04-26T23:51:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.516+0000 7f205de00640 1 -- 172.21.15.46:0/2726157395 wait complete. 2024-04-26T23:51:14.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:51:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.732+0000 7f9ac9a00640 1 Processor -- start 2024-04-26T23:51:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.733+0000 7f9ac9a00640 1 -- start start 2024-04-26T23:51:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac9a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ac40d0c10 0x7f9ac40d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac9a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac40cfe50 con 0x7f9ac40d1990 2024-04-26T23:51:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac9a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac40d0130 con 0x7f9ac41639c0 2024-04-26T23:51:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac9a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac40d0780 con 0x7f9ac40d0c10 2024-04-26T23:51:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ac40d1990 0x7f9ac40d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36304/0 (socket says 172.21.15.46:36304) 2024-04-26T23:51:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.734+0000 7f9ac3e00640 1 -- 172.21.15.46:0/1882936104 learned_addr learned my addr 172.21.15.46:0/1882936104 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac3400640 1 --2- 172.21.15.46:0/1882936104 >> v2:172.21.15.142:3300/0 conn(0x7f9ac40d0c10 0x7f9ac40d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1150452823 0 0) 0x7f9ac40cfe50 con 0x7f9ac40d1990 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aa8003520 con 0x7f9ac40d1990 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4149561228 0 0) 0x7f9aa8003520 con 0x7f9ac40d1990 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 >> v2:172.21.15.142:3300/0 conn(0x7f9ac40d0c10 msgr2=0x7f9ac40d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 --2- 172.21.15.46:0/1882936104 >> v2:172.21.15.142:3300/0 conn(0x7f9ac40d0c10 0x7f9ac40d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 >> v1:172.21.15.142:6789/0 conn(0x7f9ac41639c0 legacy=0x7f9ac40b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac40d0410 con 0x7f9ac40d1990 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9ab4001670 con 0x7f9ac40d1990 2024-04-26T23:51:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 >> v1:172.21.15.46:6789/0 conn(0x7f9ac40d1990 legacy=0x7f9ac40d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80039b0 0x7f9aa8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.735+0000 7f9ac2a00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9aa8004800 0x7f9aa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac2a00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 0x7f9aa800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac40d0780 con 0x7f9aa80051c0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800c0f0 con 0x7f9aa80039b0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa800c2f0 con 0x7f9aa8004800 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 0x7f9aa800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac3e00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80039b0 0x7f9aa8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac3400640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9aa8004800 0x7f9aa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 -- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9aa8004800 msgr2=0x7f9aa8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9aa8004800 0x7f9aa8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 -- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80039b0 msgr2=0x7f9aa8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80039b0 0x7f9aa8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac9000640 1 -- 172.21.15.46:0/1882936104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa8003520 con 0x7f9aa80051c0 2024-04-26T23:51:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.736+0000 7f9ac3e00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80039b0 0x7f9aa8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.737+0000 7f9ac9000640 1 -- 172.21.15.46:0/1882936104 reap_dead start 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.737+0000 7f9ac9000640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 0x7f9aa800b9c0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f9ac408d7e0 tx=0x7f9aa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2439a469f7cc6d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc037070 con 0x7f9aa80051c0 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc02acf0 con 0x7f9aa80051c0 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac2a00640 1 -- 172.21.15.46:0/1882936104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0040c0 con 0x7f9aa80051c0 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 -- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 msgr2=0x7f9aa800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 0x7f9aa800b9c0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f9ac408d7e0 tx=0x7f9aa8005830 comp rx=0 tx=0).stop 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 -- 172.21.15.46:0/1882936104 shutdown_connections 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/1882936104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa80051c0 0x7f9aa800b9c0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 -- 172.21.15.46:0/1882936104 >> 172.21.15.46:0/1882936104 conn(0x7f9ac409e2a0 msgr2=0x7f9ac40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.738+0000 7f9ac9a00640 1 -- 172.21.15.46:0/1882936104 shutdown_connections 2024-04-26T23:51:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.739+0000 7f9ac9a00640 1 -- 172.21.15.46:0/1882936104 wait complete. 2024-04-26T23:51:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.740+0000 7f9ac9a00640 1 Processor -- start 2024-04-26T23:51:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.740+0000 7f9ac9a00640 1 -- start start 2024-04-26T23:51:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80051c0 0x7f9ac41f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ac41f8df0 0x7f9ac41f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac40cfc10 con 0x7f9ac40cc050 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac40cfa90 con 0x7f9aa80051c0 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.741+0000 7f9ac9a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac40d1d70 con 0x7f9ac41f8df0 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48944/0 (socket says 172.21.15.46:48944) 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 -- 172.21.15.46:0/2580203241 learned_addr learned my addr 172.21.15.46:0/2580203241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac3400640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ac41f8df0 0x7f9ac41f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac3e00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80051c0 0x7f9ac41f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 -- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ac41f8df0 msgr2=0x7f9ac41f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ac41f8df0 0x7f9ac41f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 -- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80051c0 msgr2=0x7f9ac41f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80051c0 0x7f9ac41f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.742+0000 7f9ac9000640 1 -- 172.21.15.46:0/2580203241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac40ccd40 con 0x7f9ac40cc050 2024-04-26T23:51:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9ac9000640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9abc02bfd0 tx=0x7f9abc003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc037040 con 0x7f9ac40cc050 2024-04-26T23:51:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc041070 con 0x7f9ac40cc050 2024-04-26T23:51:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac40cd020 con 0x7f9ac40cc050 2024-04-26T23:51:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0040c0 con 0x7f9ac40cc050 2024-04-26T23:51:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.743+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac40cd5e0 con 0x7f9ac40cc050 2024-04-26T23:51:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.744+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac40693b0 con 0x7f9ac40cc050 2024-04-26T23:51:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.745+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9abc032d00 con 0x7f9ac40cc050 2024-04-26T23:51:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.746+0000 7f9abbe00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.746+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5249+0+0 (secure 0 0 0) 0x7f9abc0ab6a0 con 0x7f9ac40cc050 2024-04-26T23:51:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.747+0000 7f9ac3e00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:14.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.748+0000 7f9ac3e00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 0x7f9aa006edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001200 tx=0x7f9ab4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:14.751+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9abc079bf0 con 0x7f9ac40cc050 2024-04-26T23:51:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.024+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9ac40d1770 con 0x7f9ac40cc050 2024-04-26T23:51:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.025+0000 7f9abbe00640 1 -- 172.21.15.46:0/2580203241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f9abc0794f0 con 0x7f9ac40cc050 2024-04-26T23:51:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:15.030 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:51:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:51:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:51:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:51:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:51:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:51:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:51:15.044 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:51:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 msgr2=0x7f9aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 0x7f9aa006edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001200 tx=0x7f9ab4009040 comp rx=0 tx=0).stop 2024-04-26T23:51:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 msgr2=0x7f9ac40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9abc02bfd0 tx=0x7f9abc003a70 comp rx=0 tx=0).stop 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 shutdown_connections 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ac40cc050 0x7f9ac40cc4d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ac41f8df0 0x7f9ac41f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 --2- 172.21.15.46:0/2580203241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa80051c0 0x7f9ac41f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.028+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 >> 172.21.15.46:0/2580203241 conn(0x7f9ac409e2a0 msgr2=0x7f9ac406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.029+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 shutdown_connections 2024-04-26T23:51:15.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.029+0000 7f9ac9a00640 1 -- 172.21.15.46:0/2580203241 wait complete. 2024-04-26T23:51:15.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.261+0000 7f0f0d000640 1 Processor -- start 2024-04-26T23:51:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.261+0000 7f0f0d000640 1 -- start start 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f0d000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f0815f1d0 0x7f0f080cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f0d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 0x7f0f080c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f0d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f080b03c0 con 0x7f0f0815edf0 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f0d000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0f080cf5c0 con 0x7f0f0815f1d0 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f0d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080cb680 con 0x7f0f080cd770 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f07400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f0815edf0 0x7f0f080cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36318/0 (socket says 172.21.15.46:36318) 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.263+0000 7f0f07400640 1 -- 172.21.15.46:0/99360092 learned_addr learned my addr 172.21.15.46:0/99360092 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f07e00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 0x7f0f080c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0efea00640 1 --2- 172.21.15.46:0/99360092 >> v2:172.21.15.142:3300/0 conn(0x7f0f0815f1d0 0x7f0f080cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034480807 0 0) 0x7f0f080b03c0 con 0x7f0f0815edf0 2024-04-26T23:51:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eec003520 con 0x7f0f0815edf0 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3777892306 0 0) 0x7f0eec003520 con 0x7f0f0815edf0 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 msgr2=0x7f0f080c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 0x7f0f080c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 >> v2:172.21.15.142:3300/0 conn(0x7f0f0815f1d0 msgr2=0x7f0f080cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 --2- 172.21.15.46:0/99360092 >> v2:172.21.15.142:3300/0 conn(0x7f0f0815f1d0 0x7f0f080cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f080cb960 con 0x7f0f0815edf0 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0ef8001690 con 0x7f0f0815edf0 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 >> v1:172.21.15.46:6789/0 conn(0x7f0f0815edf0 legacy=0x7f0f080cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 0x7f0eec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eec003f30 0x7f0eec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0eec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080cf5c0 con 0x7f0eec004d40 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080cb680 con 0x7f0eec003f30 2024-04-26T23:51:15.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0eec00baa0 con 0x7f0f080cd770 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.264+0000 7f0efea00640 1 --2- 172.21.15.46:0/99360092 >> v2:172.21.15.142:3300/0 conn(0x7f0f0815f1d0 0x7f0f080cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0eec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 -- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 msgr2=0x7f0eec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080cd770 0x7f0eec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 -- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eec003f30 msgr2=0x7f0eec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eec003f30 0x7f0eec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.265+0000 7f0f07e00640 1 -- 172.21.15.46:0/99360092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec003520 con 0x7f0eec004d40 2024-04-26T23:51:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.266+0000 7f0f07e00640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0eec0051c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009a90 tx=0x7f0eec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb572a906455f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.266+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00036070 con 0x7f0eec004d40 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.266+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f00031400 con 0x7f0eec004d40 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.266+0000 7f0f06a00640 1 -- 172.21.15.46:0/99360092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009690 con 0x7f0eec004d40 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 -- 172.21.15.46:0/99360092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 msgr2=0x7f0eec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 --2- 172.21.15.46:0/99360092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0eec0051c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009a90 tx=0x7f0eec005830 comp rx=0 tx=0).stop 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f07e00640 1 -- 172.21.15.46:0/99360092 reap_dead start 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 -- 172.21.15.46:0/99360092 shutdown_connections 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 -- 172.21.15.46:0/99360092 >> 172.21.15.46:0/99360092 conn(0x7f0f080b26d0 msgr2=0x7f0f080b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 -- 172.21.15.46:0/99360092 shutdown_connections 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.267+0000 7f0f0d000640 1 -- 172.21.15.46:0/99360092 wait complete. 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.269+0000 7f0f0d000640 1 Processor -- start 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.269+0000 7f0f0d000640 1 -- start start 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f0815a320 0x7f0f080b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f0806ef80 0x7f0f0806f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080cc200 con 0x7f0eec004d40 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080cbf20 con 0x7f0f0806ef80 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f0d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080cb1a0 con 0x7f0f0815a320 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48958/0 (socket says 172.21.15.46:48958) 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.270+0000 7f0f07400640 1 -- 172.21.15.46:0/3234365841 learned_addr learned my addr 172.21.15.46:0/3234365841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 -- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f0815a320 msgr2=0x7f0f080b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0efea00640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f0815a320 0x7f0f080b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07e00640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f0806ef80 0x7f0f0806f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f0815a320 0x7f0f080b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 -- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f0806ef80 msgr2=0x7f0f0806f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f0806ef80 0x7f0f0806f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 -- 172.21.15.46:0/3234365841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f080727b0 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f07400640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c8d0 tx=0x7f0ef800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8018070 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef80074e0 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004a20 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.271+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08072a90 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.272+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f08073050 con 0x7f0eec004d40 2024-04-26T23:51:15.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.273+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f080693b0 con 0x7f0eec004d40 2024-04-26T23:51:15.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.274+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef8004bc0 con 0x7f0eec004d40 2024-04-26T23:51:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.275+0000 7f0effe00640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 0x7f0ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.275+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5249+0+0 (secure 0 0 0) 0x7f0ef808afc0 con 0x7f0eec004d40 2024-04-26T23:51:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.276+0000 7f0efea00640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 0x7f0ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.277+0000 7f0efea00640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 0x7f0ee406edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0f080700c0 tx=0x7f0ef000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.280+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ef8059510 con 0x7f0eec004d40 2024-04-26T23:51:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.488+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f0815bad0 con 0x7f0eec004d40 2024-04-26T23:51:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.491+0000 7f0effe00640 1 -- 172.21.15.46:0/3234365841 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+13138 (secure 0 0 0) 0x7f0ef8058e10 con 0x7f0eec004d40 2024-04-26T23:51:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:15.496 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":144,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:14.529651+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 msgr2=0x7f0ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 0x7f0ee406edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0f080700c0 tx=0x7f0ef000b040 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 msgr2=0x7f0f08073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c8d0 tx=0x7f0ef800cd90 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 shutdown_connections 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ee406c8f0 0x7f0ee406edb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f0806ef80 0x7f0f0806f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f0815a320 0x7f0f080b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 --2- 172.21.15.46:0/3234365841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eec004d40 0x7f0f08073850 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 >> 172.21.15.46:0/3234365841 conn(0x7f0f080b26d0 msgr2=0x7f0f080c5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.493+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 shutdown_connections 2024-04-26T23:51:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.494+0000 7f0f0d000640 1 -- 172.21.15.46:0/3234365841 wait complete. 2024-04-26T23:51:15.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:51:15.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.717+0000 7f892fe00640 1 Processor -- start 2024-04-26T23:51:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.717+0000 7f892fe00640 1 -- start start 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 0x7f89301644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8930073af0 0x7f89300cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89301649f0 con 0x7f89300cbec0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f89300cb4f0 con 0x7f8930073af0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8930071e80 con 0x7f89301640d0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89300cbec0 0x7f89300ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36332/0 (socket says 172.21.15.46:36332) 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.719+0000 7f892f400640 1 -- 172.21.15.46:0/4024805064 learned_addr learned my addr 172.21.15.46:0/4024805064 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892e000640 1 --2- 172.21.15.46:0/4024805064 >> v2:172.21.15.142:3300/0 conn(0x7f8930073af0 0x7f89300cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892ea00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 0x7f89301644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625852356 0 0) 0x7f89301649f0 con 0x7f89300cbec0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8914003520 con 0x7f89300cbec0 2024-04-26T23:51:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3270768069 0 0) 0x7f8914003520 con 0x7f89300cbec0 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 msgr2=0x7f89301644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 0x7f89301644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 >> v2:172.21.15.142:3300/0 conn(0x7f8930073af0 msgr2=0x7f89300cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 --2- 172.21.15.46:0/4024805064 >> v2:172.21.15.142:3300/0 conn(0x7f8930073af0 0x7f89300cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930072160 con 0x7f89300cbec0 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f89280019f0 con 0x7f89300cbec0 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 >> v1:172.21.15.46:6789/0 conn(0x7f89300cbec0 legacy=0x7f89300ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89140039b0 0x7f8914003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8914004800 0x7f8914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 0x7f89140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89300cb4f0 con 0x7f89140039b0 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8930071e80 con 0x7f8914004800 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.720+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f891400b9c0 con 0x7f89301640d0 2024-04-26T23:51:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89140039b0 0x7f8914003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892e000640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8914004800 0x7f8914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 -- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 msgr2=0x7f89140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89301640d0 0x7f89140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 -- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8914004800 msgr2=0x7f8914004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8914004800 0x7f8914004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.721+0000 7f892ea00640 1 -- 172.21.15.46:0/4024805064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8914003520 con 0x7f89140039b0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.722+0000 7f892ea00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89140039b0 0x7f8914003dd0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f8920009ac0 tx=0x7f8914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816f564b6359a2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.722+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920036070 con 0x7f89140039b0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.722+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8920031400 con 0x7f89140039b0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.722+0000 7f892d600640 1 -- 172.21.15.46:0/4024805064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920009690 con 0x7f89140039b0 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 -- 172.21.15.46:0/4024805064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89140039b0 msgr2=0x7f8914003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 --2- 172.21.15.46:0/4024805064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89140039b0 0x7f8914003dd0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f8920009ac0 tx=0x7f8914005830 comp rx=0 tx=0).stop 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892f400640 1 -- 172.21.15.46:0/4024805064 reap_dead start 2024-04-26T23:51:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 -- 172.21.15.46:0/4024805064 shutdown_connections 2024-04-26T23:51:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 -- 172.21.15.46:0/4024805064 >> 172.21.15.46:0/4024805064 conn(0x7f89300b27b0 msgr2=0x7f89300b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 -- 172.21.15.46:0/4024805064 shutdown_connections 2024-04-26T23:51:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.723+0000 7f892fe00640 1 -- 172.21.15.46:0/4024805064 wait complete. 2024-04-26T23:51:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.725+0000 7f892fe00640 1 Processor -- start 2024-04-26T23:51:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.725+0000 7f892fe00640 1 -- start start 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89140039b0 0x7f893015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8930160130 0x7f89301605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89300ceee0 con 0x7f89300b3ee0 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8930073f70 con 0x7f89140039b0 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f89300cf060 con 0x7f8930160130 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48978/0 (socket says 172.21.15.46:48978) 2024-04-26T23:51:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.726+0000 7f892f400640 1 -- 172.21.15.46:0/2015885822 learned_addr learned my addr 172.21.15.46:0/2015885822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892e000640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8930160130 0x7f89301605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892ea00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89140039b0 0x7f893015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 -- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8930160130 msgr2=0x7f89301605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8930160130 0x7f89301605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 -- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89140039b0 msgr2=0x7f893015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89140039b0 0x7f893015cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 -- 172.21.15.46:0/2015885822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89300b4880 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892f400640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f89300b6d70 tx=0x7f892800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800ce70 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892800a610 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301f97f0 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.727+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800aa00 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.728+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301f9db0 con 0x7f89300b3ee0 2024-04-26T23:51:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.729+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89300693b0 con 0x7f89300b3ee0 2024-04-26T23:51:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.730+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8928011070 con 0x7f89300b3ee0 2024-04-26T23:51:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.731+0000 7f8926a00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 0x7f891006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.731+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5249+0+0 (secure 0 0 0) 0x7f892808b4e0 con 0x7f89300b3ee0 2024-04-26T23:51:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.731+0000 7f892ea00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 0x7f891006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.732+0000 7f892ea00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 0x7f891006edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8920033000 tx=0x7f8920008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.735+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f892801a060 con 0x7f89300b3ee0 2024-04-26T23:51:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.938+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89300cd720 con 0x7f89300b3ee0 2024-04-26T23:51:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.939+0000 7f8926a00640 1 -- 172.21.15.46:0/2015885822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7f892800f260 con 0x7f89300b3ee0 2024-04-26T23:51:15.943 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:51:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 msgr2=0x7f891006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 0x7f891006edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8920033000 tx=0x7f8920008030 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 msgr2=0x7f89300b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f89300b6d70 tx=0x7f892800cc20 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 shutdown_connections 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f891006c8f0 0x7f891006edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89300b3ee0 0x7f89300b4340 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8930160130 0x7f89301605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 --2- 172.21.15.46:0/2015885822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89140039b0 0x7f893015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 >> 172.21.15.46:0/2015885822 conn(0x7f89300b27b0 msgr2=0x7f89300cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 shutdown_connections 2024-04-26T23:51:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:15.942+0000 7f892fe00640 1 -- 172.21.15.46:0/2015885822 wait complete. 2024-04-26T23:51:15.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:51:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.173+0000 7f712cc00640 1 Processor -- start 2024-04-26T23:51:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.174+0000 7f712cc00640 1 -- start start 2024-04-26T23:51:16.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f712cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d1990 0x7f71280d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f712cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71281639c0 0x7f71280b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f712cc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71280d22b0 con 0x7f71280d0c10 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f712cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f71280d0780 con 0x7f71281639c0 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f712cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f71280d5da0 con 0x7f71280d1990 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f7127e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71281639c0 0x7f71280b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f7127e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71281639c0 0x7f71280b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48988/0 (socket says 172.21.15.46:48988) 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.175+0000 7f7127e00640 1 -- 172.21.15.46:0/1987592171 learned_addr learned my addr 172.21.15.46:0/1987592171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:16.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.176+0000 7f7127400640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d1990 0x7f71280d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.176+0000 7f7127e00640 1 -- 172.21.15.46:0/1987592171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d1990 msgr2=0x7f71280d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.176+0000 7f7127e00640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d1990 0x7f71280d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.176+0000 7f7127e00640 1 -- 172.21.15.46:0/1987592171 >> v1:172.21.15.142:6789/0 conn(0x7f71280d0c10 legacy=0x7f71280d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.176+0000 7f7127e00640 1 -- 172.21.15.46:0/1987592171 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71280cfe50 con 0x7f71281639c0 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.177+0000 7f7127e00640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280b7dd0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7120009870 tx=0x7f712002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49d823faf31e17e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.177+0000 7f7126000640 1 -- 172.21.15.46:0/1987592171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120037070 con 0x7f71281639c0 2024-04-26T23:51:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.177+0000 7f7126000640 1 -- 172.21.15.46:0/1987592171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120032440 con 0x7f71281639c0 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.177+0000 7f7126000640 1 -- 172.21.15.46:0/1987592171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120032810 con 0x7f71281639c0 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 -- 172.21.15.46:0/1987592171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 msgr2=0x7f71280b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280b7dd0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7120009870 tx=0x7f712002a210 comp rx=0 tx=0).stop 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 -- 172.21.15.46:0/1987592171 shutdown_connections 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280b7dd0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 --2- 172.21.15.46:0/1987592171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d1990 0x7f71280d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 -- 172.21.15.46:0/1987592171 >> 172.21.15.46:0/1987592171 conn(0x7f712809e2a0 msgr2=0x7f71280d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 -- 172.21.15.46:0/1987592171 shutdown_connections 2024-04-26T23:51:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.178+0000 7f712cc00640 1 -- 172.21.15.46:0/1987592171 wait complete. 2024-04-26T23:51:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.179+0000 7f712cc00640 1 Processor -- start 2024-04-26T23:51:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.180+0000 7f712cc00640 1 -- start start 2024-04-26T23:51:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.180+0000 7f712cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d0c10 0x7f7128156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f712cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 0x7f71280ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f712cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f712cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f712815f530 con 0x7f71281639c0 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f712cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71280d0410 con 0x7f71280d1990 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f712cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f71280cfa90 con 0x7f71280d0c10 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48998/0 (socket says 172.21.15.46:48998) 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 -- 172.21.15.46:0/391499298 learned_addr learned my addr 172.21.15.46:0/391499298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127400640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d0c10 0x7f7128156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7126a00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 0x7f71280ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 -- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d0c10 msgr2=0x7f7128156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d0c10 0x7f7128156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 -- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 msgr2=0x7f71280ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 0x7f71280ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7127e00640 1 -- 172.21.15.46:0/391499298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71280b8650 con 0x7f71281639c0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.181+0000 7f7126a00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 0x7f71280ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f7127e00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f7120009870 tx=0x7f71200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120037040 con 0x7f71281639c0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120042070 con 0x7f71281639c0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120030920 con 0x7f71281639c0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71280b8930 con 0x7f71281639c0 2024-04-26T23:51:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.182+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71280b8ef0 con 0x7f71281639c0 2024-04-26T23:51:16.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.184+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71280693b0 con 0x7f71281639c0 2024-04-26T23:51:16.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.184+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7120030ac0 con 0x7f71281639c0 2024-04-26T23:51:16.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.185+0000 7f711f400640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 0x7f710806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.186+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5249+0+0 (secure 0 0 0) 0x7f71200ac2b0 con 0x7f71281639c0 2024-04-26T23:51:16.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.186+0000 7f7127400640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 0x7f710806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.187+0000 7f7127400640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 0x7f710806ee30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7118009a60 tx=0x7f7118003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.190+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f712007a230 con 0x7f71281639c0 2024-04-26T23:51:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.387+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f71280d1770 con 0x7f71281639c0 2024-04-26T23:51:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.388+0000 7f711f400640 1 -- 172.21.15.46:0/391499298 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) ==== 84+0+65 (secure 0 0 0) 0x7f712007e530 con 0x7f71281639c0 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:51:16.392 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 msgr2=0x7f710806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 0x7f710806ee30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7118009a60 tx=0x7f7118003990 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 msgr2=0x7f71280ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f7120009870 tx=0x7f71200046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 shutdown_connections 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f710806c970 0x7f710806ee30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71281639c0 0x7f71280ce8a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71280d1990 0x7f71280ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 --2- 172.21.15.46:0/391499298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71280d0c10 0x7f7128156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 >> 172.21.15.46:0/391499298 conn(0x7f712809e2a0 msgr2=0x7f7128160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.390+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 shutdown_connections 2024-04-26T23:51:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.391+0000 7f712cc00640 1 -- 172.21.15.46:0/391499298 wait complete. 2024-04-26T23:51:16.413 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:51:16.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:51:16.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.627+0000 7f6b70a00640 1 Processor -- start 2024-04-26T23:51:16.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.628+0000 7f6b70a00640 1 -- start start 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.629+0000 7f6b70a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d39a0 0x7f6b6c0d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.629+0000 7f6b70a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.629+0000 7f6b70a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b6c0d06d0 con 0x7f6b6c0d5790 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.629+0000 7f6b70a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6b6c0d2390 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.629+0000 7f6b70a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6c0d12e0 con 0x7f6b6c0d39a0 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49014/0 (socket says 172.21.15.46:49014) 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 -- 172.21.15.46:0/3802570806 learned_addr learned my addr 172.21.15.46:0/3802570806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6b400640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d39a0 0x7f6b6c0d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 -- 172.21.15.46:0/3802570806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d39a0 msgr2=0x7f6b6c0d3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d39a0 0x7f6b6c0d3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 -- 172.21.15.46:0/3802570806 >> v1:172.21.15.142:6789/0 conn(0x7f6b6c0d5790 legacy=0x7f6b6c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.630+0000 7f6b6be00640 1 -- 172.21.15.46:0/3802570806 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c0d09b0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.631+0000 7f6b6be00640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009e50 tx=0x7f6b6402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2b4cfd8aeb5195 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.632+0000 7f6b6aa00640 1 -- 172.21.15.46:0/3802570806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64036070 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.632+0000 7f6b6aa00640 1 -- 172.21.15.46:0/3802570806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64031600 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.632+0000 7f6b6aa00640 1 -- 172.21.15.46:0/3802570806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640319d0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.632+0000 7f6b70a00640 1 -- 172.21.15.46:0/3802570806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 msgr2=0x7f6b6c0d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.632+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009e50 tx=0x7f6b6402a630 comp rx=0 tx=0).stop 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 -- 172.21.15.46:0/3802570806 shutdown_connections 2024-04-26T23:51:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c0d1e50 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3802570806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d39a0 0x7f6b6c0d3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 -- 172.21.15.46:0/3802570806 >> 172.21.15.46:0/3802570806 conn(0x7f6b6c09e340 msgr2=0x7f6b6c0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 -- 172.21.15.46:0/3802570806 shutdown_connections 2024-04-26T23:51:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.633+0000 7f6b70a00640 1 -- 172.21.15.46:0/3802570806 wait complete. 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.634+0000 7f6b70a00640 1 Processor -- start 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 -- start start 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b6c0d39a0 0x7f6b6c0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d5790 0x7f6b6c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c0d03e0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c0d0f70 con 0x7f6b6c0d39a0 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.635+0000 7f6b70a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6c0b4bb0 con 0x7f6b6c0d5790 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49028/0 (socket says 172.21.15.46:49028) 2024-04-26T23:51:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 -- 172.21.15.46:0/3801564086 learned_addr learned my addr 172.21.15.46:0/3801564086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b63e00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b6c0d39a0 0x7f6b6c0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6be00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d5790 0x7f6b6c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 -- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d5790 msgr2=0x7f6b6c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d5790 0x7f6b6c0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 -- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b6c0d39a0 msgr2=0x7f6b6c0a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b6c0d39a0 0x7f6b6c0a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.636+0000 7f6b6b400640 1 -- 172.21.15.46:0/3801564086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c0a1250 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b6b400640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00d830 tx=0x7f6b5c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00b6c0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c019070 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00ba80 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c0a1530 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.637+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c0a1af0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.638+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c0693b0 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.639+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b5c01e460 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.640+0000 7f6b63400640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 0x7f6b4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.640+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5249+0+0 (secure 0 0 0) 0x7f6b5c08c170 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.641+0000 7f6b63e00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 0x7f6b4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.642+0000 7f6b63e00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 0x7f6b4806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0cec00 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.645+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b5c05a630 con 0x7f6b6c0d1a00 2024-04-26T23:51:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:16.848+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6b6c09e820 con 0x7f6b6c0d1a00 2024-04-26T23:51:17.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.545+0000 7f6b63400640 1 -- 172.21.15.46:0/3801564086 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v145) ==== 138+0+0 (secure 0 0 0) 0x7f6b5c059f30 con 0x7f6b6c0d1a00 2024-04-26T23:51:17.550 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:51:17.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.549+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 msgr2=0x7f6b4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.549+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 0x7f6b4806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0cec00 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2024-04-26T23:51:17.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.549+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 msgr2=0x7f6b6c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.549+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00d830 tx=0x7f6b5c00dcf0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 shutdown_connections 2024-04-26T23:51:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6b4806c920 0x7f6b4806ede0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b6c0d5790 0x7f6b6c0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b6c0d39a0 0x7f6b6c0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 --2- 172.21.15.46:0/3801564086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c0d1a00 0x7f6b6c159460 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 >> 172.21.15.46:0/3801564086 conn(0x7f6b6c09e340 msgr2=0x7f6b6c0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:17.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 shutdown_connections 2024-04-26T23:51:17.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.550+0000 7f6b70a00640 1 -- 172.21.15.46:0/3801564086 wait complete. 2024-04-26T23:51:17.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:51:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.789+0000 7f2618800640 1 Processor -- start 2024-04-26T23:51:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.790+0000 7f2618800640 1 -- start start 2024-04-26T23:51:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.791+0000 7f2618800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f26100cfdf0 0x7f26100b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2618800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26100b3360 con 0x7f261006d790 2024-04-26T23:51:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2618800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26100b3640 con 0x7f261006e580 2024-04-26T23:51:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2618800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f26100b3f70 con 0x7f26100cfdf0 2024-04-26T23:51:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2615400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f261006d790 0x7f261006de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36342/0 (socket says 172.21.15.46:36342) 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2615400640 1 -- 172.21.15.46:0/3990123755 learned_addr learned my addr 172.21.15.46:0/3990123755 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.792+0000 7f2615e00640 1 --2- 172.21.15.46:0/3990123755 >> v2:172.21.15.142:3300/0 conn(0x7f26100cfdf0 0x7f26100b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888321321 0 0) 0x7f26100b3360 con 0x7f261006d790 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25f8003520 con 0x7f261006d790 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3596242342 0 0) 0x7f25f8003520 con 0x7f261006d790 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 >> v2:172.21.15.142:3300/0 conn(0x7f26100cfdf0 msgr2=0x7f26100b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 --2- 172.21.15.46:0/3990123755 >> v2:172.21.15.142:3300/0 conn(0x7f26100cfdf0 0x7f26100b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 >> v1:172.21.15.142:6789/0 conn(0x7f261006e580 legacy=0x7f26100afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26100b3920 con 0x7f261006d790 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2600001670 con 0x7f261006d790 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 >> v1:172.21.15.46:6789/0 conn(0x7f261006d790 legacy=0x7f261006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 0x7f25f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f8004800 0x7f25f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25f80051c0 0x7f25f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26100b3f70 con 0x7f25f80039b0 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25f800c060 con 0x7f25f80051c0 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.793+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25f800c260 con 0x7f25f8004800 2024-04-26T23:51:17.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 0x7f25f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 -- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f8004800 msgr2=0x7f25f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615400640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f8004800 0x7f25f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2616800640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25f80051c0 0x7f25f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f8004800 0x7f25f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 -- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25f80051c0 msgr2=0x7f25f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25f80051c0 0x7f25f800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2615e00640 1 -- 172.21.15.46:0/3990123755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8003520 con 0x7f25f80039b0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.794+0000 7f2616800640 1 -- 172.21.15.46:0/3990123755 reap_dead start 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.795+0000 7f2615e00640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 0x7f25f8003dd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2604009ac0 tx=0x7f25f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c479654869ee78d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.795+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604036070 con 0x7f25f80039b0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.795+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2604031400 con 0x7f25f80039b0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.795+0000 7f2614a00640 1 -- 172.21.15.46:0/3990123755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604009690 con 0x7f25f80039b0 2024-04-26T23:51:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 -- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 msgr2=0x7f25f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 0x7f25f8003dd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2604009ac0 tx=0x7f25f8005830 comp rx=0 tx=0).stop 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 -- 172.21.15.46:0/3990123755 shutdown_connections 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 --2- 172.21.15.46:0/3990123755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f80039b0 0x7f25f8003dd0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 -- 172.21.15.46:0/3990123755 >> 172.21.15.46:0/3990123755 conn(0x7f26100d2e90 msgr2=0x7f26100d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 -- 172.21.15.46:0/3990123755 shutdown_connections 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.796+0000 7f2618800640 1 -- 172.21.15.46:0/3990123755 wait complete. 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.798+0000 7f2618800640 1 Processor -- start 2024-04-26T23:51:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.798+0000 7f2618800640 1 -- start start 2024-04-26T23:51:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f80039b0 0x7f26101f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f261015a090 0x7f261015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26100d01d0 con 0x7f261014e460 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f261006dbf0 con 0x7f261015a090 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.799+0000 7f2618800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26100b3010 con 0x7f25f80039b0 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49036/0 (socket says 172.21.15.46:49036) 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 -- 172.21.15.46:0/2588992482 learned_addr learned my addr 172.21.15.46:0/2588992482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615e00640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f80039b0 0x7f26101f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 -- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f80039b0 msgr2=0x7f26101f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2616800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f261015a090 0x7f261015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f80039b0 0x7f26101f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 -- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f261015a090 msgr2=0x7f261015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f261015a090 0x7f261015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 -- 172.21.15.46:0/2588992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2610159030 con 0x7f261014e460 2024-04-26T23:51:17.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.800+0000 7f2615400640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f260000d7d0 tx=0x7f260000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.801+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600019070 con 0x7f261014e460 2024-04-26T23:51:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.801+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26000074e0 con 0x7f261014e460 2024-04-26T23:51:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.801+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600004a20 con 0x7f261014e460 2024-04-26T23:51:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.801+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261014e8e0 con 0x7f261014e460 2024-04-26T23:51:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.801+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101595b0 con 0x7f261014e460 2024-04-26T23:51:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.802+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610069530 con 0x7f261014e460 2024-04-26T23:51:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.803+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f260001c070 con 0x7f261014e460 2024-04-26T23:51:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.804+0000 7f260a000640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 0x7f25f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.804+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5619+0+0 (secure 0 0 0) 0x7f260008b4d0 con 0x7f261014e460 2024-04-26T23:51:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.805+0000 7f2615e00640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 0x7f25f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.806+0000 7f2615e00640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 0x7f25f406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2604033000 tx=0x7f2604008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:17.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:17.810+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2600059260 con 0x7f261014e460 2024-04-26T23:51:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.020+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f26100b21e0 con 0x7f261014e460 2024-04-26T23:51:18.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.552+0000 7f260a000640 1 -- 172.21.15.46:0/2588992482 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v146) ==== 170+0+0 (secure 0 0 0) 0x7f260005d560 con 0x7f261014e460 2024-04-26T23:51:18.556 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:51:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 msgr2=0x7f25f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 0x7f25f406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2604033000 tx=0x7f2604008030 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 msgr2=0x7f26100c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f260000d7d0 tx=0x7f260000dc90 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 shutdown_connections 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f25f406c8f0 0x7f25f406edb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f261015a090 0x7f261015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f261014e460 0x7f26100c70a0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 --2- 172.21.15.46:0/2588992482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25f80039b0 0x7f26101f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.556+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 >> 172.21.15.46:0/2588992482 conn(0x7f26100d2e90 msgr2=0x7f2610161b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.558+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 shutdown_connections 2024-04-26T23:51:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.558+0000 7f2618800640 1 -- 172.21.15.46:0/2588992482 wait complete. 2024-04-26T23:51:18.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:51:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.778+0000 7f6f45600640 1 Processor -- start 2024-04-26T23:51:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.778+0000 7f6f45600640 1 -- start start 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f45600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f4015f210 0x7f6f400cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f45600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6f400cd610 0x7f6f400cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f45600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f400af570 con 0x7f6f400ae8f0 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f45600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6f400cdfb0 con 0x7f6f4015f210 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f45600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6f400af850 con 0x7f6f400cd610 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f3fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f4015f210 0x7f6f400cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f3fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f4015f210 0x7f6f400cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49040/0 (socket says 172.21.15.46:49040) 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.780+0000 7f6f3fe00640 1 -- 172.21.15.46:0/3934824969 learned_addr learned my addr 172.21.15.46:0/3934824969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.781+0000 7f6f3f400640 1 --2- 172.21.15.46:0/3934824969 >> v2:172.21.15.142:3300/0 conn(0x7f6f400cd610 0x7f6f400cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.781+0000 7f6f3fe00640 1 -- 172.21.15.46:0/3934824969 >> v2:172.21.15.142:3300/0 conn(0x7f6f400cd610 msgr2=0x7f6f400cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.781+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/3934824969 >> v2:172.21.15.142:3300/0 conn(0x7f6f400cd610 0x7f6f400cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.781+0000 7f6f3fe00640 1 -- 172.21.15.46:0/3934824969 >> v1:172.21.15.142:6789/0 conn(0x7f6f400ae8f0 legacy=0x7f6f400aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.781+0000 7f6f3fe00640 1 -- 172.21.15.46:0/3934824969 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f400cb680 con 0x7f6f4015f210 2024-04-26T23:51:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.782+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/3934824969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015f210 0x7f6f400cc160 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6f400b3710 tx=0x7f6f3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975661f6731bdb88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.782+0000 7f6f3ea00640 1 -- 172.21.15.46:0/3934824969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30037070 con 0x7f6f4015f210 2024-04-26T23:51:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.782+0000 7f6f3ea00640 1 -- 172.21.15.46:0/3934824969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f30032440 con 0x7f6f4015f210 2024-04-26T23:51:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.782+0000 7f6f3ea00640 1 -- 172.21.15.46:0/3934824969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30032810 con 0x7f6f4015f210 2024-04-26T23:51:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 -- 172.21.15.46:0/3934824969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015f210 msgr2=0x7f6f400cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 --2- 172.21.15.46:0/3934824969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015f210 0x7f6f400cc160 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6f400b3710 tx=0x7f6f3002a230 comp rx=0 tx=0).stop 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 -- 172.21.15.46:0/3934824969 shutdown_connections 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 --2- 172.21.15.46:0/3934824969 >> v2:172.21.15.142:3300/0 conn(0x7f6f400cd610 0x7f6f400cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 --2- 172.21.15.46:0/3934824969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015f210 0x7f6f400cc160 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 -- 172.21.15.46:0/3934824969 >> 172.21.15.46:0/3934824969 conn(0x7f6f400b27b0 msgr2=0x7f6f400b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 -- 172.21.15.46:0/3934824969 shutdown_connections 2024-04-26T23:51:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.783+0000 7f6f45600640 1 -- 172.21.15.46:0/3934824969 wait complete. 2024-04-26T23:51:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.785+0000 7f6f45600640 1 Processor -- start 2024-04-26T23:51:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.785+0000 7f6f45600640 1 -- start start 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 0x7f6f400c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 0x7f6f4014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4015f5f0 con 0x7f6f400cd610 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f400cbc40 con 0x7f6f400ae8f0 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f45600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6f40071e80 con 0x7f6f4015f210 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49056/0 (socket says 172.21.15.46:49056) 2024-04-26T23:51:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.786+0000 7f6f3f400640 1 -- 172.21.15.46:0/939446900 learned_addr learned my addr 172.21.15.46:0/939446900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 0x7f6f400c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f44c00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 0x7f6f4014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 -- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 msgr2=0x7f6f4014ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 0x7f6f4014ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 -- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 msgr2=0x7f6f400c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 0x7f6f400c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 -- 172.21.15.46:0/939446900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f400c6110 con 0x7f6f400cd610 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 0x7f6f400c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f44c00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 0x7f6f4014ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.787+0000 7f6f3f400640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c7f0 tx=0x7f6f2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.788+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c015070 con 0x7f6f400cd610 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.788+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c004240 con 0x7f6f400cd610 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.788+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f400c63f0 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.788+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c007620 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.788+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f400c69b0 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.789+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f40073e80 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.790+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f2c0079d0 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.791+0000 7f6f37e00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 0x7f6f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.791+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6001+0+0 (secure 0 0 0) 0x7f6f2c08c1b0 con 0x7f6f400cd610 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.792+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 0x7f6f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.793+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 0x7f6f2006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6f30034000 tx=0x7f6f3002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:18.796+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f40073e80 con 0x7f6f400cd610 2024-04-26T23:51:19.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.045+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6f400693b0 con 0x7f6f400cd610 2024-04-26T23:51:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.601+0000 7f6f37e00640 1 -- 172.21.15.46:0/939446900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v114) ==== 174+0+0 (secure 0 0 0) 0x7f6f2c05a410 con 0x7f6f400cd610 2024-04-26T23:51:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 21 and data pool 22 2024-04-26T23:51:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.606+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 msgr2=0x7f6f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.606+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 0x7f6f2006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6f30034000 tx=0x7f6f3002e000 comp rx=0 tx=0).stop 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.606+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 msgr2=0x7f6f401e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.606+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c7f0 tx=0x7f6f2c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 shutdown_connections 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6f2006c8f0 0x7f6f2006edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f4015f210 0x7f6f4014ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f400cd610 0x7f6f401e9090 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 --2- 172.21.15.46:0/939446900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f400ae8f0 0x7f6f400c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.607+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 >> 172.21.15.46:0/939446900 conn(0x7f6f400b27b0 msgr2=0x7f6f400cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.608+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 shutdown_connections 2024-04-26T23:51:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.608+0000 7f6f45600640 1 -- 172.21.15.46:0/939446900 wait complete. 2024-04-26T23:51:19.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.818+0000 7fe7d3400640 1 Processor -- start 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.818+0000 7fe7d3400640 1 -- start start 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.819+0000 7fe7d3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7cc142280 0x7fe7cc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.819+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc144f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.819+0000 7fe7d3400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7cc0b43f0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.819+0000 7fe7d3400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc0b46d0 con 0x7fe7cc142280 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.819+0000 7fe7d3400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0b49b0 con 0x7fe7cc06ea50 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7cc142280 0x7fe7cc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7cc142280 0x7fe7cc06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49064/0 (socket says 172.21.15.46:49064) 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 -- 172.21.15.46:0/1038709717 learned_addr learned my addr 172.21.15.46:0/1038709717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 -- 172.21.15.46:0/1038709717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 msgr2=0x7fe7cc144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 --2- 172.21.15.46:0/1038709717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 -- 172.21.15.46:0/1038709717 >> v1:172.21.15.142:6789/0 conn(0x7fe7cc0b3e80 legacy=0x7fe7cc144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.820+0000 7fe7d0a00640 1 -- 172.21.15.46:0/1038709717 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc0b4c90 con 0x7fe7cc142280 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.821+0000 7fe7d0a00640 1 --2- 172.21.15.46:0/1038709717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc142280 0x7fe7cc06e510 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009870 tx=0x7fe7c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761cb45854972c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7cb400640 1 -- 172.21.15.46:0/1038709717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0037070 con 0x7fe7cc142280 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7cb400640 1 -- 172.21.15.46:0/1038709717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c0032440 con 0x7fe7cc142280 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7cb400640 1 -- 172.21.15.46:0/1038709717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0032810 con 0x7fe7cc142280 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 -- 172.21.15.46:0/1038709717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc142280 msgr2=0x7fe7cc06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1038709717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc142280 0x7fe7cc06e510 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009870 tx=0x7fe7c002a210 comp rx=0 tx=0).stop 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 -- 172.21.15.46:0/1038709717 shutdown_connections 2024-04-26T23:51:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1038709717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1038709717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc142280 0x7fe7cc06e510 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.822+0000 7fe7d3400640 1 -- 172.21.15.46:0/1038709717 >> 172.21.15.46:0/1038709717 conn(0x7fe7cc06c530 msgr2=0x7fe7cc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.823+0000 7fe7d3400640 1 -- 172.21.15.46:0/1038709717 shutdown_connections 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.823+0000 7fe7d3400640 1 -- 172.21.15.46:0/1038709717 wait complete. 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.824+0000 7fe7d3400640 1 Processor -- start 2024-04-26T23:51:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.824+0000 7fe7d3400640 1 -- start start 2024-04-26T23:51:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 0x7fe7cc15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc142a20 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1428a0 con 0x7fe7cc142280 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.825+0000 7fe7d3400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc145490 con 0x7fe7cc06ea50 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49080/0 (socket says 172.21.15.46:49080) 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 -- 172.21.15.46:0/1016386872 learned_addr learned my addr 172.21.15.46:0/1016386872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7d0a00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7d1400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 0x7fe7cc15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 -- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 msgr2=0x7fe7cc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 -- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 msgr2=0x7fe7cc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 0x7fe7cc15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 -- 172.21.15.46:0/1016386872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc13d6e0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7d1400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 0x7fe7cc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.826+0000 7fe7cbe00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c440 tx=0x7fe7b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.827+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8017070 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.827+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b80035f0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.827+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80039e0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.827+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc13d9c0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.827+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc13d3d0 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.828+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc13db90 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.829+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7b801a070 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.831+0000 7fe7c8c00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.831+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6101+0+0 (secure 0 0 0) 0x7fe7b808b040 con 0x7fe7cc0b3e80 2024-04-26T23:51:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.831+0000 7fe7d0a00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.833+0000 7fe7d0a00640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:19.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:19.836+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7b80592e0 con 0x7fe7cc0b3e80 2024-04-26T23:51:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.042+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7cc069530 con 0x7fe7cc0b3e80 2024-04-26T23:51:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.044+0000 7fe7c8c00640 1 -- 172.21.15.46:0/1016386872 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+16869 (secure 0 0 0) 0x7fe7b80010c0 con 0x7fe7cc0b3e80 2024-04-26T23:51:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:20.049 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":147,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:19.047757+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 msgr2=0x7fe7ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002e000 comp rx=0 tx=0).stop 2024-04-26T23:51:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 msgr2=0x7fe7cc144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c440 tx=0x7fe7b800c900 comp rx=0 tx=0).stop 2024-04-26T23:51:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 shutdown_connections 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc142280 0x7fe7cc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7cc0b3e80 0x7fe7cc144ab0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 --2- 172.21.15.46:0/1016386872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc06ea50 0x7fe7cc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 >> 172.21.15.46:0/1016386872 conn(0x7fe7cc06c530 msgr2=0x7fe7cc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 shutdown_connections 2024-04-26T23:51:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.047+0000 7fe7d3400640 1 -- 172.21.15.46:0/1016386872 wait complete. 2024-04-26T23:51:20.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.274+0000 7fa44c800640 1 Processor -- start 2024-04-26T23:51:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.275+0000 7fa44c800640 1 -- start start 2024-04-26T23:51:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.276+0000 7fa44c800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 0x7fa4440d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.276+0000 7fa44c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 0x7fa4440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.276+0000 7fa44c800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4440ce7b0 con 0x7fa4440d1520 2024-04-26T23:51:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.276+0000 7fa44c800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa4440ce560 con 0x7fa44415d510 2024-04-26T23:51:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.276+0000 7fa44c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa444070230 con 0x7fa4440d3af0 2024-04-26T23:51:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.277+0000 7fa449400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 0x7fa4440d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.277+0000 7fa449400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 0x7fa4440d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:37154/0 (socket says 172.21.15.46:37154) 2024-04-26T23:51:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.277+0000 7fa449400640 1 -- 172.21.15.46:0/3239743754 learned_addr learned my addr 172.21.15.46:0/3239743754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.277+0000 7fa44a800640 1 --2- 172.21.15.46:0/3239743754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 0x7fa4440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa448a00640 1 -- 172.21.15.46:0/3239743754 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367721084 0 0) 0x7fa4440ce7b0 con 0x7fa4440d1520 2024-04-26T23:51:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa448a00640 1 -- 172.21.15.46:0/3239743754 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa430003520 con 0x7fa4440d1520 2024-04-26T23:51:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa44a800640 1 -- 172.21.15.46:0/3239743754 >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 msgr2=0x7fa4440d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa44a800640 1 --2- 172.21.15.46:0/3239743754 >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 0x7fa4440d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa44a800640 1 -- 172.21.15.46:0/3239743754 >> v1:172.21.15.142:6789/0 conn(0x7fa4440d1520 legacy=0x7fa4440d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.278+0000 7fa44a800640 1 -- 172.21.15.46:0/3239743754 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440cea90 con 0x7fa4440d3af0 2024-04-26T23:51:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.279+0000 7fa44a800640 1 --2- 172.21.15.46:0/3239743754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 0x7fa4440706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa440009d90 tx=0x7fa44002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f06598ed7abca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa448a00640 1 -- 172.21.15.46:0/3239743754 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440007740 con 0x7fa4440d3af0 2024-04-26T23:51:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa448a00640 1 -- 172.21.15.46:0/3239743754 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4400078e0 con 0x7fa4440d3af0 2024-04-26T23:51:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa448a00640 1 -- 172.21.15.46:0/3239743754 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440007cb0 con 0x7fa4440d3af0 2024-04-26T23:51:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 -- 172.21.15.46:0/3239743754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 msgr2=0x7fa4440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 --2- 172.21.15.46:0/3239743754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 0x7fa4440706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa440009d90 tx=0x7fa44002a660 comp rx=0 tx=0).stop 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 -- 172.21.15.46:0/3239743754 shutdown_connections 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 --2- 172.21.15.46:0/3239743754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d3af0 0x7fa4440706c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 --2- 172.21.15.46:0/3239743754 >> v2:172.21.15.142:3300/0 conn(0x7fa44415d510 0x7fa4440d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.280+0000 7fa44c800640 1 -- 172.21.15.46:0/3239743754 >> 172.21.15.46:0/3239743754 conn(0x7fa4440d2f40 msgr2=0x7fa4440d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.281+0000 7fa44c800640 1 -- 172.21.15.46:0/3239743754 shutdown_connections 2024-04-26T23:51:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.281+0000 7fa44c800640 1 -- 172.21.15.46:0/3239743754 wait complete. 2024-04-26T23:51:20.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.282+0000 7fa44c800640 1 Processor -- start 2024-04-26T23:51:20.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.283+0000 7fa44c800640 1 -- start start 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d1520 0x7fa444157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44415d510 0x7fa44415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440d11d0 con 0x7fa4440d3af0 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440d1050 con 0x7fa44415d510 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa444070c00 con 0x7fa4440d1520 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49090/0 (socket says 172.21.15.46:49090) 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 -- 172.21.15.46:0/3348583766 learned_addr learned my addr 172.21.15.46:0/3348583766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449e00640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d1520 0x7fa444157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 -- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d1520 msgr2=0x7fa444157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa44a800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44415d510 0x7fa44415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d1520 0x7fa444157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 -- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44415d510 msgr2=0x7fa44415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.284+0000 7fa449400640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44415d510 0x7fa44415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa449400640 1 -- 172.21.15.46:0/3348583766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440ca190 con 0x7fa4440d3af0 2024-04-26T23:51:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa449400640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e870 tx=0x7fa43400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434017070 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43400c940 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400cd30 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4440ca450 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.285+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa444157a60 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.286+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa444069530 con 0x7fa4440d3af0 2024-04-26T23:51:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.287+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa43401a070 con 0x7fa4440d3af0 2024-04-26T23:51:20.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.289+0000 7fa43a000640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.289+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6101+0+0 (secure 0 0 0) 0x7fa43408bbc0 con 0x7fa4440d3af0 2024-04-26T23:51:20.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.290+0000 7fa449e00640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.291+0000 7fa449e00640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.293+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa434059e10 con 0x7fa4440d3af0 2024-04-26T23:51:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.489+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4440cf470 con 0x7fa4440d3af0 2024-04-26T23:51:20.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.492+0000 7fa43a000640 1 -- 172.21.15.46:0/3348583766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+16869 (secure 0 0 0) 0x7fa434059710 con 0x7fa4440d3af0 2024-04-26T23:51:20.498 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:20.498 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":147,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:19.047757+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 msgr2=0x7fa42806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c009040 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 msgr2=0x7fa44415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e870 tx=0x7fa43400ed30 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 shutdown_connections 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa44415d510 0x7fa44415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4440d3af0 0x7fa44415ea60 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 --2- 172.21.15.46:0/3348583766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4440d1520 0x7fa444157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 >> 172.21.15.46:0/3348583766 conn(0x7fa4440d2f40 msgr2=0x7fa4440b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:20.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.495+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 shutdown_connections 2024-04-26T23:51:20.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.496+0000 7fa44c800640 1 -- 172.21.15.46:0/3348583766 wait complete. 2024-04-26T23:51:20.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:51:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.725+0000 7fcc0be00640 1 Processor -- start 2024-04-26T23:51:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.725+0000 7fcc0be00640 1 -- start start 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c160130 0x7fcc0c160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc0c0cdc60 0x7fcc0c072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0c0cbe60 con 0x7fcc0c15cc40 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0c0cca70 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c0cc140 con 0x7fcc0c160130 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc0c0cdc60 0x7fcc0c072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc0c0cdc60 0x7fcc0c072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49094/0 (socket says 172.21.15.46:49094) 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.726+0000 7fcc0b400640 1 -- 172.21.15.46:0/1218223436 learned_addr learned my addr 172.21.15.46:0/1218223436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.727+0000 7fcc0b400640 1 -- 172.21.15.46:0/1218223436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c160130 msgr2=0x7fcc0c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.727+0000 7fcc0b400640 1 --2- 172.21.15.46:0/1218223436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c160130 0x7fcc0c160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.727+0000 7fcc0b400640 1 -- 172.21.15.46:0/1218223436 >> v1:172.21.15.142:6789/0 conn(0x7fcc0c15cc40 legacy=0x7fcc0c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.727+0000 7fcc0b400640 1 -- 172.21.15.46:0/1218223436 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c0cc420 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.728+0000 7fcc0b400640 1 --2- 172.21.15.46:0/1218223436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c072220 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400bb10 tx=0x7fcc0402c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7487beb9261245c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.728+0000 7fcc09600640 1 -- 172.21.15.46:0/1218223436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0403a070 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.728+0000 7fcc09600640 1 -- 172.21.15.46:0/1218223436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc040326f0 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.728+0000 7fcc09600640 1 -- 172.21.15.46:0/1218223436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04032ac0 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 -- 172.21.15.46:0/1218223436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 msgr2=0x7fcc0c072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 --2- 172.21.15.46:0/1218223436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c072220 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400bb10 tx=0x7fcc0402c830 comp rx=0 tx=0).stop 2024-04-26T23:51:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 -- 172.21.15.46:0/1218223436 shutdown_connections 2024-04-26T23:51:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 --2- 172.21.15.46:0/1218223436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c072220 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 --2- 172.21.15.46:0/1218223436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c160130 0x7fcc0c160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 -- 172.21.15.46:0/1218223436 >> 172.21.15.46:0/1218223436 conn(0x7fcc0c0b27b0 msgr2=0x7fcc0c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 -- 172.21.15.46:0/1218223436 shutdown_connections 2024-04-26T23:51:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.729+0000 7fcc0be00640 1 -- 172.21.15.46:0/1218223436 wait complete. 2024-04-26T23:51:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.731+0000 7fcc0be00640 1 Processor -- start 2024-04-26T23:51:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.731+0000 7fcc0be00640 1 -- start start 2024-04-26T23:51:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c15cc40 0x7fcc0c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 0x7fcc0c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c15d220 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c15d0a0 con 0x7fcc0c160130 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c072760 con 0x7fcc0c15cc40 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49098/0 (socket says 172.21.15.46:49098) 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 -- 172.21.15.46:0/613849464 learned_addr learned my addr 172.21.15.46:0/613849464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0a000640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c15cc40 0x7fcc0c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0b400640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 0x7fcc0c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 -- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c15cc40 msgr2=0x7fcc0c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c15cc40 0x7fcc0c0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 -- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 msgr2=0x7fcc0c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 0x7fcc0c0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0aa00640 1 -- 172.21.15.46:0/613849464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c1f5140 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.732+0000 7fcc0b400640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 0x7fcc0c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc0aa00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b410 tx=0x7fcbfc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc015020 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbfc011850 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc011c40 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c0c9990 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.733+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c1f53e0 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.735+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c1f55e0 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.736+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbfc018020 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.736+0000 7fcc02a00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.737+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7fcbfc08bc10 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.737+0000 7fcc0a000640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.738+0000 7fcc0a000640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c15e2b0 tx=0x7fcbf4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.741+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc0c1f55e0 con 0x7fcc0c0cdc60 2024-04-26T23:51:20.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:20.987+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcc0c0693b0 con 0x7fcc0c0cdc60 2024-04-26T23:51:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.608+0000 7fcc02a00640 1 -- 172.21.15.46:0/613849464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v117) ==== 118+0+0 (secure 0 0 0) 0x7fcbfc01f020 con 0x7fcc0c0cdc60 2024-04-26T23:51:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 msgr2=0x7fcbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c15e2b0 tx=0x7fcbf4009040 comp rx=0 tx=0).stop 2024-04-26T23:51:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 msgr2=0x7fcc0c0c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b410 tx=0x7fcbfc00b8d0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 shutdown_connections 2024-04-26T23:51:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c160130 0x7fcc0c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0c15cc40 0x7fcc0c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 --2- 172.21.15.46:0/613849464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c0cdc60 0x7fcc0c0c90d0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.613+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 >> 172.21.15.46:0/613849464 conn(0x7fcc0c0b27b0 msgr2=0x7fcc0c155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.614+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 shutdown_connections 2024-04-26T23:51:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.614+0000 7fcc0be00640 1 -- 172.21.15.46:0/613849464 wait complete. 2024-04-26T23:51:21.643 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:51:21.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:51:21.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.861+0000 7fe857e00640 1 Processor -- start 2024-04-26T23:51:21.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.861+0000 7fe857e00640 1 -- start start 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe857e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe8580cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe857e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8580af570 con 0x7fe85815f210 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe857e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8580af850 con 0x7fe8580ae8f0 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe857e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580cb680 con 0x7fe8580cdd20 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe856a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe85815f210 0x7fe8580cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60192/0 (socket says 172.21.15.46:60192) 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.863+0000 7fe856a00640 1 -- 172.21.15.46:0/686265099 learned_addr learned my addr 172.21.15.46:0/686265099 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe856000640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe8580cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492732122 0 0) 0x7fe8580af570 con 0x7fe85815f210 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe83c003520 con 0x7fe85815f210 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 843325076 0 0) 0x7fe83c003520 con 0x7fe85815f210 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 msgr2=0x7fe8580cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe8580cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 >> v1:172.21.15.142:6789/0 conn(0x7fe8580ae8f0 legacy=0x7fe8580aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8580cb960 con 0x7fe85815f210 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe848001670 con 0x7fe85815f210 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 >> v1:172.21.15.46:6789/0 conn(0x7fe85815f210 legacy=0x7fe8580cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe83c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.864+0000 7fe855600640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c003f20 0x7fe83c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe855600640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe83c004d40 0x7fe83c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580cb680 con 0x7fe83c004d40 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c00ba70 con 0x7fe83c003f20 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c00bc70 con 0x7fe8580cdd20 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe83c004d40 0x7fe83c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe856a00640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe83c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe856000640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c003f20 0x7fe83c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 -- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 msgr2=0x7fe83c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8580cdd20 0x7fe83c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 -- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c003f20 msgr2=0x7fe83c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c003f20 0x7fe83c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.865+0000 7fe857400640 1 -- 172.21.15.46:0/686265099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c003520 con 0x7fe83c004d40 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.866+0000 7fe857400640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe83c004d40 0x7fe83c0051c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fe850001f60 tx=0x7fe83c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c367041e3309472 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850001b30 con 0x7fe83c004d40 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850031620 con 0x7fe83c004d40 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe855600640 1 -- 172.21.15.46:0/686265099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850031a10 con 0x7fe83c004d40 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe857e00640 1 -- 172.21.15.46:0/686265099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe83c004d40 msgr2=0x7fe83c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe857e00640 1 --2- 172.21.15.46:0/686265099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe83c004d40 0x7fe83c0051c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fe850001f60 tx=0x7fe83c005830 comp rx=0 tx=0).stop 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.867+0000 7fe857400640 1 -- 172.21.15.46:0/686265099 reap_dead start 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.868+0000 7fe857e00640 1 -- 172.21.15.46:0/686265099 shutdown_connections 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.868+0000 7fe857e00640 1 -- 172.21.15.46:0/686265099 >> 172.21.15.46:0/686265099 conn(0x7fe8580b27b0 msgr2=0x7fe8580b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.868+0000 7fe857e00640 1 -- 172.21.15.46:0/686265099 shutdown_connections 2024-04-26T23:51:21.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.868+0000 7fe857e00640 1 -- 172.21.15.46:0/686265099 wait complete. 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.869+0000 7fe857e00640 1 Processor -- start 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.870+0000 7fe857e00640 1 -- start start 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.870+0000 7fe857e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83c004d40 0x7fe8581f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.870+0000 7fe857e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8580c7cd0 0x7fe8581f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580cc160 con 0x7fe8581f5590 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe85815f5f0 con 0x7fe8580c7cd0 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580cc2e0 con 0x7fe83c004d40 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33610/0 (socket says 172.21.15.46:33610) 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 -- 172.21.15.46:0/3722459260 learned_addr learned my addr 172.21.15.46:0/3722459260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe856a00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83c004d40 0x7fe8581f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe856000640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8580c7cd0 0x7fe8581f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 -- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83c004d40 msgr2=0x7fe8581f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83c004d40 0x7fe8581f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 -- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8580c7cd0 msgr2=0x7fe8581f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8580c7cd0 0x7fe8581f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.871+0000 7fe857400640 1 -- 172.21.15.46:0/3722459260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85806f740 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe857400640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fe85002aa00 tx=0x7fe8500015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850042070 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8500317d0 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8580c8190 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850031b80 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.872+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8580c8750 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.873+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8580693b0 con 0x7fe8581f5590 2024-04-26T23:51:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.875+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe850045070 con 0x7fe8581f5590 2024-04-26T23:51:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.875+0000 7fe84ea00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.876+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7fe8500abde0 con 0x7fe8581f5590 2024-04-26T23:51:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.876+0000 7fe856a00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.877+0000 7fe856a00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 0x7fe83806edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe848001050 tx=0x7fe848009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:21.880+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe85806f740 con 0x7fe8581f5590 2024-04-26T23:51:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.132+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe8580b09a0 con 0x7fe8581f5590 2024-04-26T23:51:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.619+0000 7fe84ea00640 1 -- 172.21.15.46:0/3722459260 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v118) ==== 122+0+0 (secure 0 0 0) 0x7fe8500799a0 con 0x7fe8581f5590 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.624+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 msgr2=0x7fe83806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.624+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 0x7fe83806edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe848001050 tx=0x7fe848009040 comp rx=0 tx=0).stop 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.624+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 msgr2=0x7fe8580730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.624+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fe85002aa00 tx=0x7fe8500015b0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 shutdown_connections 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8581f5590 0x7fe8580730b0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8580c7cd0 0x7fe8581f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 --2- 172.21.15.46:0/3722459260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83c004d40 0x7fe8581f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 >> 172.21.15.46:0/3722459260 conn(0x7fe8580b27b0 msgr2=0x7fe8580b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 shutdown_connections 2024-04-26T23:51:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.625+0000 7fe857e00640 1 -- 172.21.15.46:0/3722459260 wait complete. 2024-04-26T23:51:22.655 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:51:22.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.867+0000 7fb3a2600640 1 Processor -- start 2024-04-26T23:51:22.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.867+0000 7fb3a2600640 1 -- start start 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb39c15f4e0 0x7fb39c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0b4000 0x7fb39c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb39c0d1f60 0x7fb39c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb39c0d2990 con 0x7fb39c15f4e0 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb39c0d11f0 con 0x7fb39c0d1f60 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.868+0000 7fb3a2600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb39c0d5f20 con 0x7fb39c0b4000 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb39c15f4e0 0x7fb39c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb39c15f4e0 0x7fb39c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33626/0 (socket says 172.21.15.46:33626) 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 -- 172.21.15.46:0/3128565514 learned_addr learned my addr 172.21.15.46:0/3128565514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 -- 172.21.15.46:0/3128565514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0b4000 msgr2=0x7fb39c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 --2- 172.21.15.46:0/3128565514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0b4000 0x7fb39c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 -- 172.21.15.46:0/3128565514 >> v2:172.21.15.142:3300/0 conn(0x7fb39c0d1f60 msgr2=0x7fb39c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 --2- 172.21.15.46:0/3128565514 >> v2:172.21.15.142:3300/0 conn(0x7fb39c0d1f60 0x7fb39c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.869+0000 7fb3a1200640 1 -- 172.21.15.46:0/3128565514 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c0d0f10 con 0x7fb39c15f4e0 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.870+0000 7fb3a1200640 1 --2- 172.21.15.46:0/3128565514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0d2df0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0bb0c0 tx=0x7fb38c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d87b36ffa28820 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb39b400640 1 -- 172.21.15.46:0/3128565514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c037070 con 0x7fb39c15f4e0 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb39b400640 1 -- 172.21.15.46:0/3128565514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c032440 con 0x7fb39c15f4e0 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb39b400640 1 -- 172.21.15.46:0/3128565514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c032810 con 0x7fb39c15f4e0 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb3a2600640 1 -- 172.21.15.46:0/3128565514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c15f4e0 msgr2=0x7fb39c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb3a2600640 1 --2- 172.21.15.46:0/3128565514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0d2df0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0bb0c0 tx=0x7fb38c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb3a2600640 1 -- 172.21.15.46:0/3128565514 shutdown_connections 2024-04-26T23:51:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb3a2600640 1 --2- 172.21.15.46:0/3128565514 >> v2:172.21.15.142:3300/0 conn(0x7fb39c0d1f60 0x7fb39c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.871+0000 7fb3a2600640 1 --2- 172.21.15.46:0/3128565514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0b4000 0x7fb39c0d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.872+0000 7fb3a2600640 1 --2- 172.21.15.46:0/3128565514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0d2df0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.872+0000 7fb3a2600640 1 -- 172.21.15.46:0/3128565514 >> 172.21.15.46:0/3128565514 conn(0x7fb39c0d5010 msgr2=0x7fb39c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.872+0000 7fb3a2600640 1 -- 172.21.15.46:0/3128565514 shutdown_connections 2024-04-26T23:51:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.872+0000 7fb3a2600640 1 -- 172.21.15.46:0/3128565514 wait complete. 2024-04-26T23:51:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.873+0000 7fb3a2600640 1 Processor -- start 2024-04-26T23:51:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.874+0000 7fb3a2600640 1 -- start start 2024-04-26T23:51:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.874+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0d1f60 0x7fb39c0a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c15f8c0 con 0x7fb39c0b4000 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a2600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0b1d60 con 0x7fb39c15f4e0 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a2600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb39c0d0210 con 0x7fb39c0d1f60 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33640/0 (socket says 172.21.15.46:33640) 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 -- 172.21.15.46:0/4061382823 learned_addr learned my addr 172.21.15.46:0/4061382823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb39be00640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0d1f60 0x7fb39c0a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 -- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0d1f60 msgr2=0x7fb39c0a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1c00640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0d1f60 0x7fb39c0a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 -- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c15f4e0 msgr2=0x7fb39c0b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.875+0000 7fb3a1200640 1 -- 172.21.15.46:0/4061382823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c0a1f30 con 0x7fb39c0b4000 2024-04-26T23:51:22.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb3a1200640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:22.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c041070 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c03c870 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c037040 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c0a2210 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.876+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c0bae60 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.877+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c0a2730 con 0x7fb39c0b4000 2024-04-26T23:51:22.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.878+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb38c044070 con 0x7fb39c0b4000 2024-04-26T23:51:22.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.879+0000 7fb398c00640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 0x7fb37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:22.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.879+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7fb38c0ac880 con 0x7fb39c0b4000 2024-04-26T23:51:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.880+0000 7fb39be00640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 0x7fb37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:22.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.881+0000 7fb39be00640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 0x7fb37806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb39c15ec30 tx=0x7fb3840072f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:22.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:22.884+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb39c0a2730 con 0x7fb39c0b4000 2024-04-26T23:51:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.135+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c0693b0 con 0x7fb39c0b4000 2024-04-26T23:51:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.136+0000 7fb398c00640 1 -- 172.21.15.46:0/4061382823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7fb38c07ac30 con 0x7fb39c0b4000 2024-04-26T23:51:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:51:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 msgr2=0x7fb37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 0x7fb37806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb39c15ec30 tx=0x7fb3840072f0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 msgr2=0x7fb39c0ba920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 shutdown_connections 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb37806c8f0 0x7fb37806edb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c15f4e0 0x7fb39c0b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c0d1f60 0x7fb39c0a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 --2- 172.21.15.46:0/4061382823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb39c0b4000 0x7fb39c0ba920 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.139+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 >> 172.21.15.46:0/4061382823 conn(0x7fb39c0d5010 msgr2=0x7fb39c075930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.140+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 shutdown_connections 2024-04-26T23:51:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.140+0000 7fb3a2600640 1 -- 172.21.15.46:0/4061382823 wait complete. 2024-04-26T23:51:23.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.366+0000 7f74dcc00640 1 Processor -- start 2024-04-26T23:51:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.366+0000 7f74dcc00640 1 -- start start 2024-04-26T23:51:23.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.367+0000 7f74dcc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f74d80cee60 0x7f74d80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.367+0000 7f74dcc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.367+0000 7f74dcc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80d2430 con 0x7f74d80d3890 2024-04-26T23:51:23.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.367+0000 7f74dcc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f74d80cf240 con 0x7f74d80cee60 2024-04-26T23:51:23.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.367+0000 7f74dcc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74d80d1fa0 con 0x7f74d8162ec0 2024-04-26T23:51:23.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.368+0000 7f74d7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.368+0000 7f74d7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:53846/0 (socket says 172.21.15.46:53846) 2024-04-26T23:51:23.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.368+0000 7f74d7e00640 1 -- 172.21.15.46:0/4092964593 learned_addr learned my addr 172.21.15.46:0/4092964593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:23.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.368+0000 7f74d7400640 1 --2- 172.21.15.46:0/4092964593 >> v2:172.21.15.142:3300/0 conn(0x7f74d80cee60 0x7f74d80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.369+0000 7f74d7e00640 1 -- 172.21.15.46:0/4092964593 >> v2:172.21.15.142:3300/0 conn(0x7f74d80cee60 msgr2=0x7f74d80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.369+0000 7f74d7e00640 1 --2- 172.21.15.46:0/4092964593 >> v2:172.21.15.142:3300/0 conn(0x7f74d80cee60 0x7f74d80cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.369+0000 7f74d7e00640 1 -- 172.21.15.46:0/4092964593 >> v1:172.21.15.142:6789/0 conn(0x7f74d80d3890 legacy=0x7f74d815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.369+0000 7f74d7e00640 1 -- 172.21.15.46:0/4092964593 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d80d1390 con 0x7f74d8162ec0 2024-04-26T23:51:23.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.370+0000 7f74d7e00640 1 --2- 172.21.15.46:0/4092964593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f153604d6cdfdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.371+0000 7f74d6000640 1 -- 172.21.15.46:0/4092964593 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0037070 con 0x7f74d8162ec0 2024-04-26T23:51:23.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.371+0000 7f74d6000640 1 -- 172.21.15.46:0/4092964593 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d0032440 con 0x7f74d8162ec0 2024-04-26T23:51:23.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.371+0000 7f74d6000640 1 -- 172.21.15.46:0/4092964593 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0032810 con 0x7f74d8162ec0 2024-04-26T23:51:23.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.371+0000 7f74dcc00640 1 -- 172.21.15.46:0/4092964593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 msgr2=0x7f74d80cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.371+0000 7f74dcc00640 1 --2- 172.21.15.46:0/4092964593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d002a270 comp rx=0 tx=0).stop 2024-04-26T23:51:23.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 -- 172.21.15.46:0/4092964593 shutdown_connections 2024-04-26T23:51:23.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 --2- 172.21.15.46:0/4092964593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80cfb30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 --2- 172.21.15.46:0/4092964593 >> v2:172.21.15.142:3300/0 conn(0x7f74d80cee60 0x7f74d80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 -- 172.21.15.46:0/4092964593 >> 172.21.15.46:0/4092964593 conn(0x7f74d80d2d70 msgr2=0x7f74d80d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 -- 172.21.15.46:0/4092964593 shutdown_connections 2024-04-26T23:51:23.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.372+0000 7f74dcc00640 1 -- 172.21.15.46:0/4092964593 wait complete. 2024-04-26T23:51:23.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.374+0000 7f74dcc00640 1 Processor -- start 2024-04-26T23:51:23.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.374+0000 7f74dcc00640 1 -- start start 2024-04-26T23:51:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d80cee60 0x7f74d81f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80d1c30 con 0x7f74d80d3890 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80d1950 con 0x7f74d80cee60 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.375+0000 7f74dcc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f74d80d1040 con 0x7f74d8162ec0 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33654/0 (socket says 172.21.15.46:33654) 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 -- 172.21.15.46:0/2108368231 learned_addr learned my addr 172.21.15.46:0/2108368231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d7400640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d80cee60 0x7f74d81f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d7e00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 -- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 msgr2=0x7f74d80b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 -- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d80cee60 msgr2=0x7f74d81f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d80cee60 0x7f74d81f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.376+0000 7f74d6a00640 1 -- 172.21.15.46:0/2108368231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d80b7c40 con 0x7f74d80d3890 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74d6a00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c480 tx=0x7f74c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40052d0 con 0x7f74d80d3890 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4005470 con 0x7f74d80d3890 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4012870 con 0x7f74d80d3890 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d80b7f20 con 0x7f74d80d3890 2024-04-26T23:51:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.377+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d80b3930 con 0x7f74d80d3890 2024-04-26T23:51:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.378+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d80b3b30 con 0x7f74d80d3890 2024-04-26T23:51:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.380+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74c4012a10 con 0x7f74d80d3890 2024-04-26T23:51:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.380+0000 7f74cf400640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 0x7f74b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.381+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f74c408ba20 con 0x7f74d80d3890 2024-04-26T23:51:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.381+0000 7f74d7400640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 0x7f74b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.382+0000 7f74d7400640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 0x7f74b806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f74c80074a0 tx=0x7f74c800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.386+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74d80b3b30 con 0x7f74d80d3890 2024-04-26T23:51:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.591+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74d8069ac0 con 0x7f74d80d3890 2024-04-26T23:51:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.595+0000 7f74cf400640 1 -- 172.21.15.46:0/2108368231 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16856 (secure 0 0 0) 0x7f74d80d2430 con 0x7f74d80d3890 2024-04-26T23:51:23.600 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:23.600 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":148,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:20.596779+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.597+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 msgr2=0x7f74b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.597+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 0x7f74b806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f74c80074a0 tx=0x7f74c800b040 comp rx=0 tx=0).stop 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.597+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 msgr2=0x7f74d80b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.597+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c480 tx=0x7f74c400c940 comp rx=0 tx=0).stop 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 shutdown_connections 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f74b806c8f0 0x7f74b806edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74d8162ec0 0x7f74d80b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d80d3890 0x7f74d80b74d0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 --2- 172.21.15.46:0/2108368231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d80cee60 0x7f74d81f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 >> 172.21.15.46:0/2108368231 conn(0x7f74d80d2d70 msgr2=0x7f74d81627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 shutdown_connections 2024-04-26T23:51:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.598+0000 7f74dcc00640 1 -- 172.21.15.46:0/2108368231 wait complete. 2024-04-26T23:51:23.621 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:51:23.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:51:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.816+0000 7f7c18800640 1 Processor -- start 2024-04-26T23:51:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.817+0000 7f7c18800640 1 -- start start 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.818+0000 7f7c18800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c1006e510 0x7f7c1006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.818+0000 7f7c18800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c10073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.818+0000 7f7c18800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1006ee30 con 0x7f7c10160bd0 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.818+0000 7f7c18800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7c10161940 con 0x7f7c1006e510 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.818+0000 7f7c18800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c10071f80 con 0x7f7c100732c0 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c15400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c10160bd0 0x7f7c10161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60204/0 (socket says 172.21.15.46:60204) 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c15400640 1 -- 172.21.15.46:0/239170856 learned_addr learned my addr 172.21.15.46:0/239170856 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c15e00640 1 --2- 172.21.15.46:0/239170856 >> v2:172.21.15.46:3300/0 conn(0x7f7c1006e510 0x7f7c1006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c16800640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c10073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c14a00640 1 -- 172.21.15.46:0/239170856 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330363965 0 0) 0x7f7c1006ee30 con 0x7f7c10160bd0 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.819+0000 7f7c14a00640 1 -- 172.21.15.46:0/239170856 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bf8003520 con 0x7f7c10160bd0 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.820+0000 7f7c15e00640 1 -- 172.21.15.46:0/239170856 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 msgr2=0x7f7c10073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.820+0000 7f7c15e00640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c10073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.820+0000 7f7c15e00640 1 -- 172.21.15.46:0/239170856 >> v1:172.21.15.46:6789/0 conn(0x7f7c10160bd0 legacy=0x7f7c10161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.820+0000 7f7c15e00640 1 -- 172.21.15.46:0/239170856 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c10072260 con 0x7f7c1006e510 2024-04-26T23:51:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c15e00640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1006e510 0x7f7c1006e8f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097d0 tx=0x7f7bfc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29fdc312fe621b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c14a00640 1 -- 172.21.15.46:0/239170856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc037070 con 0x7f7c1006e510 2024-04-26T23:51:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c14a00640 1 -- 172.21.15.46:0/239170856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc032440 con 0x7f7c1006e510 2024-04-26T23:51:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c14a00640 1 -- 172.21.15.46:0/239170856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc032810 con 0x7f7c1006e510 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c18800640 1 -- 172.21.15.46:0/239170856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1006e510 msgr2=0x7f7c1006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.821+0000 7f7c18800640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1006e510 0x7f7c1006e8f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097d0 tx=0x7f7bfc02a1c0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 -- 172.21.15.46:0/239170856 shutdown_connections 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c10073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 --2- 172.21.15.46:0/239170856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1006e510 0x7f7c1006e8f0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 -- 172.21.15.46:0/239170856 >> 172.21.15.46:0/239170856 conn(0x7f7c1006c530 msgr2=0x7f7c1006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 -- 172.21.15.46:0/239170856 shutdown_connections 2024-04-26T23:51:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.822+0000 7f7c18800640 1 -- 172.21.15.46:0/239170856 wait complete. 2024-04-26T23:51:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.823+0000 7f7c18800640 1 Processor -- start 2024-04-26T23:51:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 -- start start 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 0x7f7c101f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c100aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10072d40 con 0x7f7c10160bd0 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10072820 con 0x7f7c1006e510 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.824+0000 7f7c18800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c10072ec0 con 0x7f7c100732c0 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33670/0 (socket says 172.21.15.46:33670) 2024-04-26T23:51:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 -- 172.21.15.46:0/421618166 learned_addr learned my addr 172.21.15.46:0/421618166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c15400640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c100aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c15e00640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 0x7f7c101f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 -- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 msgr2=0x7f7c100aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c100aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 -- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 msgr2=0x7f7c101f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 0x7f7c101f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c16800640 1 -- 172.21.15.46:0/421618166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100af190 con 0x7f7c10160bd0 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c15400640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c100aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.825+0000 7f7c15e00640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 0x7f7c101f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c16800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007e70 tx=0x7f7c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c014620 con 0x7f7c10160bd0 2024-04-26T23:51:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c019070 con 0x7f7c10160bd0 2024-04-26T23:51:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c014a20 con 0x7f7c10160bd0 2024-04-26T23:51:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c100aba20 con 0x7f7c10160bd0 2024-04-26T23:51:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.826+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c100abfe0 con 0x7f7c10160bd0 2024-04-26T23:51:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.827+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c10069530 con 0x7f7c10160bd0 2024-04-26T23:51:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.828+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c0c014bc0 con 0x7f7c10160bd0 2024-04-26T23:51:23.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.829+0000 7f7c0a000640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.829+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f7c0c08c690 con 0x7f7c10160bd0 2024-04-26T23:51:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.830+0000 7f7c15e00640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.831+0000 7f7c15e00640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 0x7f7bf006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:23.833+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c0c011260 con 0x7f7c10160bd0 2024-04-26T23:51:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.025+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c1006fb60 con 0x7f7bf006c8f0 2024-04-26T23:51:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.045+0000 7f7c0a000640 1 -- 172.21.15.46:0/421618166 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f7bfc03c460 con 0x7f7bf006c8f0 2024-04-26T23:51:24.054 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:24.055 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 msgr2=0x7f7bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 0x7f7bf006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02e000 comp rx=0 tx=0).stop 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 msgr2=0x7f7c1013c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007e70 tx=0x7f7c0c00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 shutdown_connections 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10160bd0 0x7f7c1013c220 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c100732c0 0x7f7c100aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.051+0000 7f7c18800640 1 --2- 172.21.15.46:0/421618166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c1006e510 0x7f7c101f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.052+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 >> 172.21.15.46:0/421618166 conn(0x7f7c1006c530 msgr2=0x7f7c100ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.052+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 shutdown_connections 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.052+0000 7f7c18800640 1 -- 172.21.15.46:0/421618166 wait complete. 2024-04-26T23:51:24.076 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":572,"stamp":"2024-04-26T23:51:22.208458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430332,"kb_used_data":15660,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754544388,"statfs":{"total":773094113280,"available":772653453312,"internally_reserved":0,"allocated":16035840,"data_stored":8137840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.657137"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622015+0000","last_change":"2024-04-26T23:51:19.621972+0000","last_active":"2024-04-26T23:51:20.622015+0000","last_peered":"2024-04-26T23:51:20.622015+0000","last_clean":"2024-04-26T23:51:20.622015+0000","last_became_active":"2024-04-26T23:51:19.617849+0000","last_became_peered":"2024-04-26T23:51:19.617849+0000","last_unstale":"2024-04-26T23:51:20.622015+0000","last_undegraded":"2024-04-26T23:51:20.622015+0000","last_fullsized":"2024-04-26T23:51:20.622015+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:10.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613743+0000","last_change":"2024-04-26T23:51:19.626476+0000","last_active":"2024-04-26T23:51:20.613743+0000","last_peered":"2024-04-26T23:51:20.613743+0000","last_clean":"2024-04-26T23:51:20.613743+0000","last_became_active":"2024-04-26T23:51:19.624748+0000","last_became_peered":"2024-04-26T23:51:19.624748+0000","last_unstale":"2024-04-26T23:51:20.613743+0000","last_undegraded":"2024-04-26T23:51:20.613743+0000","last_fullsized":"2024-04-26T23:51:20.613743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:08.293570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613291+0000","last_change":"2024-04-26T23:51:19.597522+0000","last_active":"2024-04-26T23:51:20.613291+0000","last_peered":"2024-04-26T23:51:20.613291+0000","last_clean":"2024-04-26T23:51:20.613291+0000","last_became_active":"2024-04-26T23:51:19.596323+0000","last_became_peered":"2024-04-26T23:51:19.596323+0000","last_unstale":"2024-04-26T23:51:20.613291+0000","last_undegraded":"2024-04-26T23:51:20.613291+0000","last_fullsized":"2024-04-26T23:51:20.613291+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:34.079085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722159+0000","last_change":"2024-04-26T23:51:19.621354+0000","last_active":"2024-04-26T23:51:20.722159+0000","last_peered":"2024-04-26T23:51:20.722159+0000","last_clean":"2024-04-26T23:51:20.722159+0000","last_became_active":"2024-04-26T23:51:19.621082+0000","last_became_peered":"2024-04-26T23:51:19.621082+0000","last_unstale":"2024-04-26T23:51:20.722159+0000","last_undegraded":"2024-04-26T23:51:20.722159+0000","last_fullsized":"2024-04-26T23:51:20.722159+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:40.010712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721701+0000","last_change":"2024-04-26T23:51:19.620091+0000","last_active":"2024-04-26T23:51:20.721701+0000","last_peered":"2024-04-26T23:51:20.721701+0000","last_clean":"2024-04-26T23:51:20.721701+0000","last_became_active":"2024-04-26T23:51:19.619409+0000","last_became_peered":"2024-04-26T23:51:19.619409+0000","last_unstale":"2024-04-26T23:51:20.721701+0000","last_undegraded":"2024-04-26T23:51:20.721701+0000","last_fullsized":"2024-04-26T23:51:20.721701+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:12.977331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721958+0000","last_change":"2024-04-26T23:51:19.618710+0000","last_active":"2024-04-26T23:51:20.721958+0000","last_peered":"2024-04-26T23:51:20.721958+0000","last_clean":"2024-04-26T23:51:20.721958+0000","last_became_active":"2024-04-26T23:51:19.618374+0000","last_became_peered":"2024-04-26T23:51:19.618374+0000","last_unstale":"2024-04-26T23:51:20.721958+0000","last_undegraded":"2024-04-26T23:51:20.721958+0000","last_fullsized":"2024-04-26T23:51:20.721958+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:00.404120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621908+0000","last_change":"2024-04-26T23:51:19.621932+0000","last_active":"2024-04-26T23:51:20.621908+0000","last_peered":"2024-04-26T23:51:20.621908+0000","last_clean":"2024-04-26T23:51:20.621908+0000","last_became_active":"2024-04-26T23:51:19.620677+0000","last_became_peered":"2024-04-26T23:51:19.620677+0000","last_unstale":"2024-04-26T23:51:20.621908+0000","last_undegraded":"2024-04-26T23:51:20.621908+0000","last_fullsized":"2024-04-26T23:51:20.621908+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:09.935864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613761+0000","last_change":"2024-04-26T23:51:19.613977+0000","last_active":"2024-04-26T23:51:20.613761+0000","last_peered":"2024-04-26T23:51:20.613761+0000","last_clean":"2024-04-26T23:51:20.613761+0000","last_became_active":"2024-04-26T23:51:19.613448+0000","last_became_peered":"2024-04-26T23:51:19.613448+0000","last_unstale":"2024-04-26T23:51:20.613761+0000","last_undegraded":"2024-04-26T23:51:20.613761+0000","last_fullsized":"2024-04-26T23:51:20.613761+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:22.960830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721813+0000","last_change":"2024-04-26T23:51:19.615307+0000","last_active":"2024-04-26T23:51:20.721813+0000","last_peered":"2024-04-26T23:51:20.721813+0000","last_clean":"2024-04-26T23:51:20.721813+0000","last_became_active":"2024-04-26T23:51:19.614998+0000","last_became_peered":"2024-04-26T23:51:19.614998+0000","last_unstale":"2024-04-26T23:51:20.721813+0000","last_undegraded":"2024-04-26T23:51:20.721813+0000","last_fullsized":"2024-04-26T23:51:20.721813+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:40.662270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722264+0000","last_change":"2024-04-26T23:51:19.624604+0000","last_active":"2024-04-26T23:51:20.722264+0000","last_peered":"2024-04-26T23:51:20.722264+0000","last_clean":"2024-04-26T23:51:20.722264+0000","last_became_active":"2024-04-26T23:51:19.622398+0000","last_became_peered":"2024-04-26T23:51:19.622398+0000","last_unstale":"2024-04-26T23:51:20.722264+0000","last_undegraded":"2024-04-26T23:51:20.722264+0000","last_fullsized":"2024-04-26T23:51:20.722264+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:59.999085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721592+0000","last_change":"2024-04-26T23:51:19.623250+0000","last_active":"2024-04-26T23:51:20.721592+0000","last_peered":"2024-04-26T23:51:20.721592+0000","last_clean":"2024-04-26T23:51:20.721592+0000","last_became_active":"2024-04-26T23:51:19.622870+0000","last_became_peered":"2024-04-26T23:51:19.622870+0000","last_unstale":"2024-04-26T23:51:20.721592+0000","last_undegraded":"2024-04-26T23:51:20.721592+0000","last_fullsized":"2024-04-26T23:51:20.721592+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:17.049158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722222+0000","last_change":"2024-04-26T23:51:19.632948+0000","last_active":"2024-04-26T23:51:20.722222+0000","last_peered":"2024-04-26T23:51:20.722222+0000","last_clean":"2024-04-26T23:51:20.722222+0000","last_became_active":"2024-04-26T23:51:19.630358+0000","last_became_peered":"2024-04-26T23:51:19.630358+0000","last_unstale":"2024-04-26T23:51:20.722222+0000","last_undegraded":"2024-04-26T23:51:20.722222+0000","last_fullsized":"2024-04-26T23:51:20.722222+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:53.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721274+0000","last_change":"2024-04-26T23:51:19.618669+0000","last_active":"2024-04-26T23:51:20.721274+0000","last_peered":"2024-04-26T23:51:20.721274+0000","last_clean":"2024-04-26T23:51:20.721274+0000","last_became_active":"2024-04-26T23:51:19.617258+0000","last_became_peered":"2024-04-26T23:51:19.617258+0000","last_unstale":"2024-04-26T23:51:20.721274+0000","last_undegraded":"2024-04-26T23:51:20.721274+0000","last_fullsized":"2024-04-26T23:51:20.721274+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.417014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721710+0000","last_change":"2024-04-26T23:51:19.618677+0000","last_active":"2024-04-26T23:51:20.721710+0000","last_peered":"2024-04-26T23:51:20.721710+0000","last_clean":"2024-04-26T23:51:20.721710+0000","last_became_active":"2024-04-26T23:51:19.617126+0000","last_became_peered":"2024-04-26T23:51:19.617126+0000","last_unstale":"2024-04-26T23:51:20.721710+0000","last_undegraded":"2024-04-26T23:51:20.721710+0000","last_fullsized":"2024-04-26T23:51:20.721710+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:09.757437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613133+0000","last_change":"2024-04-26T23:51:19.627565+0000","last_active":"2024-04-26T23:51:20.613133+0000","last_peered":"2024-04-26T23:51:20.613133+0000","last_clean":"2024-04-26T23:51:20.613133+0000","last_became_active":"2024-04-26T23:51:19.627281+0000","last_became_peered":"2024-04-26T23:51:19.627281+0000","last_unstale":"2024-04-26T23:51:20.613133+0000","last_undegraded":"2024-04-26T23:51:20.613133+0000","last_fullsized":"2024-04-26T23:51:20.613133+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.919962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720949+0000","last_change":"2024-04-26T23:51:19.610349+0000","last_active":"2024-04-26T23:51:20.720949+0000","last_peered":"2024-04-26T23:51:20.720949+0000","last_clean":"2024-04-26T23:51:20.720949+0000","last_became_active":"2024-04-26T23:51:19.609882+0000","last_became_peered":"2024-04-26T23:51:19.609882+0000","last_unstale":"2024-04-26T23:51:20.720949+0000","last_undegraded":"2024-04-26T23:51:20.720949+0000","last_fullsized":"2024-04-26T23:51:20.720949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:34.252626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721939+0000","last_change":"2024-04-26T23:51:19.621900+0000","last_active":"2024-04-26T23:51:20.721939+0000","last_peered":"2024-04-26T23:51:20.721939+0000","last_clean":"2024-04-26T23:51:20.721939+0000","last_became_active":"2024-04-26T23:51:19.621577+0000","last_became_peered":"2024-04-26T23:51:19.621577+0000","last_unstale":"2024-04-26T23:51:20.721939+0000","last_undegraded":"2024-04-26T23:51:20.721939+0000","last_fullsized":"2024-04-26T23:51:20.721939+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:34.090443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722407+0000","last_change":"2024-04-26T23:51:19.625828+0000","last_active":"2024-04-26T23:51:20.722407+0000","last_peered":"2024-04-26T23:51:20.722407+0000","last_clean":"2024-04-26T23:51:20.722407+0000","last_became_active":"2024-04-26T23:51:19.621356+0000","last_became_peered":"2024-04-26T23:51:19.621356+0000","last_unstale":"2024-04-26T23:51:20.722407+0000","last_undegraded":"2024-04-26T23:51:20.722407+0000","last_fullsized":"2024-04-26T23:51:20.722407+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:14.350308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628651+0000","last_change":"2024-04-26T23:51:19.615702+0000","last_active":"2024-04-26T23:51:20.628651+0000","last_peered":"2024-04-26T23:51:20.628651+0000","last_clean":"2024-04-26T23:51:20.628651+0000","last_became_active":"2024-04-26T23:51:19.614390+0000","last_became_peered":"2024-04-26T23:51:19.614390+0000","last_unstale":"2024-04-26T23:51:20.628651+0000","last_undegraded":"2024-04-26T23:51:20.628651+0000","last_fullsized":"2024-04-26T23:51:20.628651+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.417014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610740+0000","last_change":"2024-04-26T23:51:19.596380+0000","last_active":"2024-04-26T23:51:20.610740+0000","last_peered":"2024-04-26T23:51:20.610740+0000","last_clean":"2024-04-26T23:51:20.610740+0000","last_became_active":"2024-04-26T23:51:19.595545+0000","last_became_peered":"2024-04-26T23:51:19.595545+0000","last_unstale":"2024-04-26T23:51:20.610740+0000","last_undegraded":"2024-04-26T23:51:20.610740+0000","last_fullsized":"2024-04-26T23:51:20.610740+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:04.441710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721517+0000","last_change":"2024-04-26T23:51:19.618246+0000","last_active":"2024-04-26T23:51:20.721517+0000","last_peered":"2024-04-26T23:51:20.721517+0000","last_clean":"2024-04-26T23:51:20.721517+0000","last_became_active":"2024-04-26T23:51:19.617653+0000","last_became_peered":"2024-04-26T23:51:19.617653+0000","last_unstale":"2024-04-26T23:51:20.721517+0000","last_undegraded":"2024-04-26T23:51:20.721517+0000","last_fullsized":"2024-04-26T23:51:20.721517+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:09.604539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621650+0000","last_change":"2024-04-26T23:51:19.621450+0000","last_active":"2024-04-26T23:51:20.621650+0000","last_peered":"2024-04-26T23:51:20.621650+0000","last_clean":"2024-04-26T23:51:20.621650+0000","last_became_active":"2024-04-26T23:51:19.621165+0000","last_became_peered":"2024-04-26T23:51:19.621165+0000","last_unstale":"2024-04-26T23:51:20.621650+0000","last_undegraded":"2024-04-26T23:51:20.621650+0000","last_fullsized":"2024-04-26T23:51:20.621650+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:08.293570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613793+0000","last_change":"2024-04-26T23:51:19.614650+0000","last_active":"2024-04-26T23:51:20.613793+0000","last_peered":"2024-04-26T23:51:20.613793+0000","last_clean":"2024-04-26T23:51:20.613793+0000","last_became_active":"2024-04-26T23:51:19.614352+0000","last_became_peered":"2024-04-26T23:51:19.614352+0000","last_unstale":"2024-04-26T23:51:20.613793+0000","last_undegraded":"2024-04-26T23:51:20.613793+0000","last_fullsized":"2024-04-26T23:51:20.613793+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:06.816362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720694+0000","last_change":"2024-04-26T23:51:19.613042+0000","last_active":"2024-04-26T23:51:20.720694+0000","last_peered":"2024-04-26T23:51:20.720694+0000","last_clean":"2024-04-26T23:51:20.720694+0000","last_became_active":"2024-04-26T23:51:19.612743+0000","last_became_peered":"2024-04-26T23:51:19.612743+0000","last_unstale":"2024-04-26T23:51:20.720694+0000","last_undegraded":"2024-04-26T23:51:20.720694+0000","last_fullsized":"2024-04-26T23:51:20.720694+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:32.286079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622096+0000","last_change":"2024-04-26T23:51:19.612217+0000","last_active":"2024-04-26T23:51:20.622096+0000","last_peered":"2024-04-26T23:51:20.622096+0000","last_clean":"2024-04-26T23:51:20.622096+0000","last_became_active":"2024-04-26T23:51:19.611885+0000","last_became_peered":"2024-04-26T23:51:19.611885+0000","last_unstale":"2024-04-26T23:51:20.622096+0000","last_undegraded":"2024-04-26T23:51:20.622096+0000","last_fullsized":"2024-04-26T23:51:20.622096+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:24.088709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622309+0000","last_change":"2024-04-26T23:51:19.616764+0000","last_active":"2024-04-26T23:51:20.622309+0000","last_peered":"2024-04-26T23:51:20.622309+0000","last_clean":"2024-04-26T23:51:20.622309+0000","last_became_active":"2024-04-26T23:51:19.616341+0000","last_became_peered":"2024-04-26T23:51:19.616341+0000","last_unstale":"2024-04-26T23:51:20.622309+0000","last_undegraded":"2024-04-26T23:51:20.622309+0000","last_fullsized":"2024-04-26T23:51:20.622309+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:33.826967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721989+0000","last_change":"2024-04-26T23:51:19.618541+0000","last_active":"2024-04-26T23:51:20.721989+0000","last_peered":"2024-04-26T23:51:20.721989+0000","last_clean":"2024-04-26T23:51:20.721989+0000","last_became_active":"2024-04-26T23:51:19.618222+0000","last_became_peered":"2024-04-26T23:51:19.618222+0000","last_unstale":"2024-04-26T23:51:20.721989+0000","last_undegraded":"2024-04-26T23:51:20.721989+0000","last_fullsized":"2024-04-26T23:51:20.721989+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.523540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720723+0000","last_change":"2024-04-26T23:51:19.612197+0000","last_active":"2024-04-26T23:51:20.720723+0000","last_peered":"2024-04-26T23:51:20.720723+0000","last_clean":"2024-04-26T23:51:20.720723+0000","last_became_active":"2024-04-26T23:51:19.611895+0000","last_became_peered":"2024-04-26T23:51:19.611895+0000","last_unstale":"2024-04-26T23:51:20.720723+0000","last_undegraded":"2024-04-26T23:51:20.720723+0000","last_fullsized":"2024-04-26T23:51:20.720723+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:03.643122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615043+0000","last_change":"2024-04-26T23:51:19.627125+0000","last_active":"2024-04-26T23:51:20.615043+0000","last_peered":"2024-04-26T23:51:20.615043+0000","last_clean":"2024-04-26T23:51:20.615043+0000","last_became_active":"2024-04-26T23:51:19.626833+0000","last_became_peered":"2024-04-26T23:51:19.626833+0000","last_unstale":"2024-04-26T23:51:20.615043+0000","last_undegraded":"2024-04-26T23:51:20.615043+0000","last_fullsized":"2024-04-26T23:51:20.615043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:33.085937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720911+0000","last_change":"2024-04-26T23:51:19.622983+0000","last_active":"2024-04-26T23:51:20.720911+0000","last_peered":"2024-04-26T23:51:20.720911+0000","last_clean":"2024-04-26T23:51:20.720911+0000","last_became_active":"2024-04-26T23:51:19.621519+0000","last_became_peered":"2024-04-26T23:51:19.621519+0000","last_unstale":"2024-04-26T23:51:20.720911+0000","last_undegraded":"2024-04-26T23:51:20.720911+0000","last_fullsized":"2024-04-26T23:51:20.720911+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:14.252548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721273+0000","last_change":"2024-04-26T23:51:19.613168+0000","last_active":"2024-04-26T23:51:20.721273+0000","last_peered":"2024-04-26T23:51:20.721273+0000","last_clean":"2024-04-26T23:51:20.721273+0000","last_became_active":"2024-04-26T23:51:19.610951+0000","last_became_peered":"2024-04-26T23:51:19.610951+0000","last_unstale":"2024-04-26T23:51:20.721273+0000","last_undegraded":"2024-04-26T23:51:20.721273+0000","last_fullsized":"2024-04-26T23:51:20.721273+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:44.171307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614151+0000","last_change":"2024-04-26T23:51:19.613739+0000","last_active":"2024-04-26T23:51:20.614151+0000","last_peered":"2024-04-26T23:51:20.614151+0000","last_clean":"2024-04-26T23:51:20.614151+0000","last_became_active":"2024-04-26T23:51:19.613053+0000","last_became_peered":"2024-04-26T23:51:19.613053+0000","last_unstale":"2024-04-26T23:51:20.614151+0000","last_undegraded":"2024-04-26T23:51:20.614151+0000","last_fullsized":"2024-04-26T23:51:20.614151+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:12.606703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613900+0000","last_change":"2024-04-26T23:51:19.602894+0000","last_active":"2024-04-26T23:51:20.613900+0000","last_peered":"2024-04-26T23:51:20.613900+0000","last_clean":"2024-04-26T23:51:20.613900+0000","last_became_active":"2024-04-26T23:51:19.602509+0000","last_became_peered":"2024-04-26T23:51:19.602509+0000","last_unstale":"2024-04-26T23:51:20.613900+0000","last_undegraded":"2024-04-26T23:51:20.613900+0000","last_fullsized":"2024-04-26T23:51:20.613900+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:10.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"147'4","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722059+0000","last_change":"2024-04-26T23:51:18.576956+0000","last_active":"2024-04-26T23:51:20.722059+0000","last_peered":"2024-04-26T23:51:20.722059+0000","last_clean":"2024-04-26T23:51:20.722059+0000","last_became_active":"2024-04-26T23:51:18.576654+0000","last_became_peered":"2024-04-26T23:51:18.576654+0000","last_unstale":"2024-04-26T23:51:20.722059+0000","last_undegraded":"2024-04-26T23:51:20.722059+0000","last_fullsized":"2024-04-26T23:51:20.722059+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:34.066906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613773+0000","last_change":"2024-04-26T23:51:19.622153+0000","last_active":"2024-04-26T23:51:20.613773+0000","last_peered":"2024-04-26T23:51:20.613773+0000","last_clean":"2024-04-26T23:51:20.613773+0000","last_became_active":"2024-04-26T23:51:19.621851+0000","last_became_peered":"2024-04-26T23:51:19.621851+0000","last_unstale":"2024-04-26T23:51:20.613773+0000","last_undegraded":"2024-04-26T23:51:20.613773+0000","last_fullsized":"2024-04-26T23:51:20.613773+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:09.935864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627947+0000","last_change":"2024-04-26T23:51:18.594579+0000","last_active":"2024-04-26T23:51:20.627947+0000","last_peered":"2024-04-26T23:51:20.627947+0000","last_clean":"2024-04-26T23:51:20.627947+0000","last_became_active":"2024-04-26T23:51:18.594280+0000","last_became_peered":"2024-04-26T23:51:18.594280+0000","last_unstale":"2024-04-26T23:51:20.627947+0000","last_undegraded":"2024-04-26T23:51:20.627947+0000","last_fullsized":"2024-04-26T23:51:20.627947+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:07.277028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615337+0000","last_change":"2024-04-26T23:51:19.624493+0000","last_active":"2024-04-26T23:51:20.615337+0000","last_peered":"2024-04-26T23:51:20.615337+0000","last_clean":"2024-04-26T23:51:20.615337+0000","last_became_active":"2024-04-26T23:51:19.624184+0000","last_became_peered":"2024-04-26T23:51:19.624184+0000","last_unstale":"2024-04-26T23:51:20.615337+0000","last_undegraded":"2024-04-26T23:51:20.615337+0000","last_fullsized":"2024-04-26T23:51:20.615337+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:25.077822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628714+0000","last_change":"2024-04-26T23:51:18.596813+0000","last_active":"2024-04-26T23:51:20.628714+0000","last_peered":"2024-04-26T23:51:20.628714+0000","last_clean":"2024-04-26T23:51:20.628714+0000","last_became_active":"2024-04-26T23:51:18.596522+0000","last_became_peered":"2024-04-26T23:51:18.596522+0000","last_unstale":"2024-04-26T23:51:20.628714+0000","last_undegraded":"2024-04-26T23:51:20.628714+0000","last_fullsized":"2024-04-26T23:51:20.628714+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:08.919322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628020+0000","last_change":"2024-04-26T23:51:19.608539+0000","last_active":"2024-04-26T23:51:20.628020+0000","last_peered":"2024-04-26T23:51:20.628020+0000","last_clean":"2024-04-26T23:51:20.628020+0000","last_became_active":"2024-04-26T23:51:19.608206+0000","last_became_peered":"2024-04-26T23:51:19.608206+0000","last_unstale":"2024-04-26T23:51:20.628020+0000","last_undegraded":"2024-04-26T23:51:20.628020+0000","last_fullsized":"2024-04-26T23:51:20.628020+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:44.953071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613922+0000","last_change":"2024-04-26T23:51:18.584926+0000","last_active":"2024-04-26T23:51:20.613922+0000","last_peered":"2024-04-26T23:51:20.613922+0000","last_clean":"2024-04-26T23:51:20.613922+0000","last_became_active":"2024-04-26T23:51:18.584626+0000","last_became_peered":"2024-04-26T23:51:18.584626+0000","last_unstale":"2024-04-26T23:51:20.613922+0000","last_undegraded":"2024-04-26T23:51:20.613922+0000","last_fullsized":"2024-04-26T23:51:20.613922+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.590161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.611018+0000","last_change":"2024-04-26T23:51:19.615638+0000","last_active":"2024-04-26T23:51:20.611018+0000","last_peered":"2024-04-26T23:51:20.611018+0000","last_clean":"2024-04-26T23:51:20.611018+0000","last_became_active":"2024-04-26T23:51:19.614596+0000","last_became_peered":"2024-04-26T23:51:19.614596+0000","last_unstale":"2024-04-26T23:51:20.611018+0000","last_undegraded":"2024-04-26T23:51:20.611018+0000","last_fullsized":"2024-04-26T23:51:20.611018+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:12.977331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615243+0000","last_change":"2024-04-26T23:51:18.581856+0000","last_active":"2024-04-26T23:51:20.615243+0000","last_peered":"2024-04-26T23:51:20.615243+0000","last_clean":"2024-04-26T23:51:20.615243+0000","last_became_active":"2024-04-26T23:51:18.580751+0000","last_became_peered":"2024-04-26T23:51:18.580751+0000","last_unstale":"2024-04-26T23:51:20.615243+0000","last_undegraded":"2024-04-26T23:51:20.615243+0000","last_fullsized":"2024-04-26T23:51:20.615243+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:31.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721984+0000","last_change":"2024-04-26T23:51:19.617803+0000","last_active":"2024-04-26T23:51:20.721984+0000","last_peered":"2024-04-26T23:51:20.721984+0000","last_clean":"2024-04-26T23:51:20.721984+0000","last_became_active":"2024-04-26T23:51:19.617478+0000","last_became_peered":"2024-04-26T23:51:19.617478+0000","last_unstale":"2024-04-26T23:51:20.721984+0000","last_undegraded":"2024-04-26T23:51:20.721984+0000","last_fullsized":"2024-04-26T23:51:20.721984+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:23.585513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722256+0000","last_change":"2024-04-26T23:51:18.585993+0000","last_active":"2024-04-26T23:51:20.722256+0000","last_peered":"2024-04-26T23:51:20.722256+0000","last_clean":"2024-04-26T23:51:20.722256+0000","last_became_active":"2024-04-26T23:51:18.585606+0000","last_became_peered":"2024-04-26T23:51:18.585606+0000","last_unstale":"2024-04-26T23:51:20.722256+0000","last_undegraded":"2024-04-26T23:51:20.722256+0000","last_fullsized":"2024-04-26T23:51:20.722256+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:33.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613037+0000","last_change":"2024-04-26T23:51:19.596778+0000","last_active":"2024-04-26T23:51:20.613037+0000","last_peered":"2024-04-26T23:51:20.613037+0000","last_clean":"2024-04-26T23:51:20.613037+0000","last_became_active":"2024-04-26T23:51:19.595899+0000","last_became_peered":"2024-04-26T23:51:19.595899+0000","last_unstale":"2024-04-26T23:51:20.613037+0000","last_undegraded":"2024-04-26T23:51:20.613037+0000","last_fullsized":"2024-04-26T23:51:20.613037+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:14.184589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622766+0000","last_change":"2024-04-26T23:51:18.587355+0000","last_active":"2024-04-26T23:51:20.622766+0000","last_peered":"2024-04-26T23:51:20.622766+0000","last_clean":"2024-04-26T23:51:20.622766+0000","last_became_active":"2024-04-26T23:51:18.585333+0000","last_became_peered":"2024-04-26T23:51:18.585333+0000","last_unstale":"2024-04-26T23:51:20.622766+0000","last_undegraded":"2024-04-26T23:51:20.622766+0000","last_fullsized":"2024-04-26T23:51:20.622766+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:37.800610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622045+0000","last_change":"2024-04-26T23:51:19.616088+0000","last_active":"2024-04-26T23:51:20.622045+0000","last_peered":"2024-04-26T23:51:20.622045+0000","last_clean":"2024-04-26T23:51:20.622045+0000","last_became_active":"2024-04-26T23:51:19.615781+0000","last_became_peered":"2024-04-26T23:51:19.615781+0000","last_unstale":"2024-04-26T23:51:20.622045+0000","last_undegraded":"2024-04-26T23:51:20.622045+0000","last_fullsized":"2024-04-26T23:51:20.622045+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:52.766539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721153+0000","last_change":"2024-04-26T23:51:18.585455+0000","last_active":"2024-04-26T23:51:20.721153+0000","last_peered":"2024-04-26T23:51:20.721153+0000","last_clean":"2024-04-26T23:51:20.721153+0000","last_became_active":"2024-04-26T23:51:18.585121+0000","last_became_peered":"2024-04-26T23:51:18.585121+0000","last_unstale":"2024-04-26T23:51:20.721153+0000","last_undegraded":"2024-04-26T23:51:20.721153+0000","last_fullsized":"2024-04-26T23:51:20.721153+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:26.903420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615573+0000","last_change":"2024-04-26T23:51:19.615898+0000","last_active":"2024-04-26T23:51:20.615573+0000","last_peered":"2024-04-26T23:51:20.615573+0000","last_clean":"2024-04-26T23:51:20.615573+0000","last_became_active":"2024-04-26T23:51:19.615505+0000","last_became_peered":"2024-04-26T23:51:19.615505+0000","last_unstale":"2024-04-26T23:51:20.615573+0000","last_undegraded":"2024-04-26T23:51:20.615573+0000","last_fullsized":"2024-04-26T23:51:20.615573+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:53.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628482+0000","last_change":"2024-04-26T23:51:18.594548+0000","last_active":"2024-04-26T23:51:20.628482+0000","last_peered":"2024-04-26T23:51:20.628482+0000","last_clean":"2024-04-26T23:51:20.628482+0000","last_became_active":"2024-04-26T23:51:18.594228+0000","last_became_peered":"2024-04-26T23:51:18.594228+0000","last_unstale":"2024-04-26T23:51:20.628482+0000","last_undegraded":"2024-04-26T23:51:20.628482+0000","last_fullsized":"2024-04-26T23:51:20.628482+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:51.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720896+0000","last_change":"2024-04-26T23:51:19.621918+0000","last_active":"2024-04-26T23:51:20.720896+0000","last_peered":"2024-04-26T23:51:20.720896+0000","last_clean":"2024-04-26T23:51:20.720896+0000","last_became_active":"2024-04-26T23:51:19.619943+0000","last_became_peered":"2024-04-26T23:51:19.619943+0000","last_unstale":"2024-04-26T23:51:20.720896+0000","last_undegraded":"2024-04-26T23:51:20.720896+0000","last_fullsized":"2024-04-26T23:51:20.720896+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:54.238756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622809+0000","last_change":"2024-04-26T23:51:18.585011+0000","last_active":"2024-04-26T23:51:20.622809+0000","last_peered":"2024-04-26T23:51:20.622809+0000","last_clean":"2024-04-26T23:51:20.622809+0000","last_became_active":"2024-04-26T23:51:18.584154+0000","last_became_peered":"2024-04-26T23:51:18.584154+0000","last_unstale":"2024-04-26T23:51:20.622809+0000","last_undegraded":"2024-04-26T23:51:20.622809+0000","last_fullsized":"2024-04-26T23:51:20.622809+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:13.492617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722643+0000","last_change":"2024-04-26T23:51:19.628819+0000","last_active":"2024-04-26T23:51:20.722643+0000","last_peered":"2024-04-26T23:51:20.722643+0000","last_clean":"2024-04-26T23:51:20.722643+0000","last_became_active":"2024-04-26T23:51:19.621282+0000","last_became_peered":"2024-04-26T23:51:19.621282+0000","last_unstale":"2024-04-26T23:51:20.722643+0000","last_undegraded":"2024-04-26T23:51:20.722643+0000","last_fullsized":"2024-04-26T23:51:20.722643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:49.439975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"147'3","reported_seq":17,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721020+0000","last_change":"2024-04-26T23:51:18.577260+0000","last_active":"2024-04-26T23:51:20.721020+0000","last_peered":"2024-04-26T23:51:20.721020+0000","last_clean":"2024-04-26T23:51:20.721020+0000","last_became_active":"2024-04-26T23:51:18.576874+0000","last_became_peered":"2024-04-26T23:51:18.576874+0000","last_unstale":"2024-04-26T23:51:20.721020+0000","last_undegraded":"2024-04-26T23:51:20.721020+0000","last_fullsized":"2024-04-26T23:51:20.721020+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:18.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614309+0000","last_change":"2024-04-26T23:51:19.619471+0000","last_active":"2024-04-26T23:51:20.614309+0000","last_peered":"2024-04-26T23:51:20.614309+0000","last_clean":"2024-04-26T23:51:20.614309+0000","last_became_active":"2024-04-26T23:51:19.618332+0000","last_became_peered":"2024-04-26T23:51:19.618332+0000","last_unstale":"2024-04-26T23:51:20.614309+0000","last_undegraded":"2024-04-26T23:51:20.614309+0000","last_fullsized":"2024-04-26T23:51:20.614309+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:09.757437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721663+0000","last_change":"2024-04-26T23:51:18.574375+0000","last_active":"2024-04-26T23:51:20.721663+0000","last_peered":"2024-04-26T23:51:20.721663+0000","last_clean":"2024-04-26T23:51:20.721663+0000","last_became_active":"2024-04-26T23:51:18.573955+0000","last_became_peered":"2024-04-26T23:51:18.573955+0000","last_unstale":"2024-04-26T23:51:20.721663+0000","last_undegraded":"2024-04-26T23:51:20.721663+0000","last_fullsized":"2024-04-26T23:51:20.721663+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:51.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613269+0000","last_change":"2024-04-26T23:51:19.618331+0000","last_active":"2024-04-26T23:51:20.613269+0000","last_peered":"2024-04-26T23:51:20.613269+0000","last_clean":"2024-04-26T23:51:20.613269+0000","last_became_active":"2024-04-26T23:51:19.618034+0000","last_became_peered":"2024-04-26T23:51:19.618034+0000","last_unstale":"2024-04-26T23:51:20.613269+0000","last_undegraded":"2024-04-26T23:51:20.613269+0000","last_fullsized":"2024-04-26T23:51:20.613269+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:35.083448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721486+0000","last_change":"2024-04-26T23:51:18.590494+0000","last_active":"2024-04-26T23:51:20.721486+0000","last_peered":"2024-04-26T23:51:20.721486+0000","last_clean":"2024-04-26T23:51:20.721486+0000","last_became_active":"2024-04-26T23:51:18.590174+0000","last_became_peered":"2024-04-26T23:51:18.590174+0000","last_unstale":"2024-04-26T23:51:20.721486+0000","last_undegraded":"2024-04-26T23:51:20.721486+0000","last_fullsized":"2024-04-26T23:51:20.721486+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:31.683032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721983+0000","last_change":"2024-04-26T23:51:19.609490+0000","last_active":"2024-04-26T23:51:20.721983+0000","last_peered":"2024-04-26T23:51:20.721983+0000","last_clean":"2024-04-26T23:51:20.721983+0000","last_became_active":"2024-04-26T23:51:19.609126+0000","last_became_peered":"2024-04-26T23:51:19.609126+0000","last_unstale":"2024-04-26T23:51:20.721983+0000","last_undegraded":"2024-04-26T23:51:20.721983+0000","last_fullsized":"2024-04-26T23:51:20.721983+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:18.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722267+0000","last_change":"2024-04-26T23:51:18.580666+0000","last_active":"2024-04-26T23:51:20.722267+0000","last_peered":"2024-04-26T23:51:20.722267+0000","last_clean":"2024-04-26T23:51:20.722267+0000","last_became_active":"2024-04-26T23:51:18.580358+0000","last_became_peered":"2024-04-26T23:51:18.580358+0000","last_unstale":"2024-04-26T23:51:20.722267+0000","last_undegraded":"2024-04-26T23:51:20.722267+0000","last_fullsized":"2024-04-26T23:51:20.722267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:02.003196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.612973+0000","last_change":"2024-04-26T23:51:19.625377+0000","last_active":"2024-04-26T23:51:20.612973+0000","last_peered":"2024-04-26T23:51:20.612973+0000","last_clean":"2024-04-26T23:51:20.612973+0000","last_became_active":"2024-04-26T23:51:19.622032+0000","last_became_peered":"2024-04-26T23:51:19.622032+0000","last_unstale":"2024-04-26T23:51:20.612973+0000","last_undegraded":"2024-04-26T23:51:20.612973+0000","last_fullsized":"2024-04-26T23:51:20.612973+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:00.505778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"147'8","reported_seq":22,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621447+0000","last_change":"2024-04-26T23:51:18.594857+0000","last_active":"2024-04-26T23:51:20.621447+0000","last_peered":"2024-04-26T23:51:20.621447+0000","last_clean":"2024-04-26T23:51:20.621447+0000","last_became_active":"2024-04-26T23:51:18.594487+0000","last_became_peered":"2024-04-26T23:51:18.594487+0000","last_unstale":"2024-04-26T23:51:20.621447+0000","last_undegraded":"2024-04-26T23:51:20.621447+0000","last_fullsized":"2024-04-26T23:51:20.621447+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:52.882140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628013+0000","last_change":"2024-04-26T23:51:19.622190+0000","last_active":"2024-04-26T23:51:20.628013+0000","last_peered":"2024-04-26T23:51:20.628013+0000","last_clean":"2024-04-26T23:51:20.628013+0000","last_became_active":"2024-04-26T23:51:19.621888+0000","last_became_peered":"2024-04-26T23:51:19.621888+0000","last_unstale":"2024-04-26T23:51:20.628013+0000","last_undegraded":"2024-04-26T23:51:20.628013+0000","last_fullsized":"2024-04-26T23:51:20.628013+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:34.252626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721911+0000","last_change":"2024-04-26T23:51:18.591404+0000","last_active":"2024-04-26T23:51:20.721911+0000","last_peered":"2024-04-26T23:51:20.721911+0000","last_clean":"2024-04-26T23:51:20.721911+0000","last_became_active":"2024-04-26T23:51:18.591089+0000","last_became_peered":"2024-04-26T23:51:18.591089+0000","last_unstale":"2024-04-26T23:51:20.721911+0000","last_undegraded":"2024-04-26T23:51:20.721911+0000","last_fullsized":"2024-04-26T23:51:20.721911+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:42.157357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628531+0000","last_change":"2024-04-26T23:51:19.615983+0000","last_active":"2024-04-26T23:51:20.628531+0000","last_peered":"2024-04-26T23:51:20.628531+0000","last_clean":"2024-04-26T23:51:20.628531+0000","last_became_active":"2024-04-26T23:51:19.614792+0000","last_became_peered":"2024-04-26T23:51:19.614792+0000","last_unstale":"2024-04-26T23:51:20.628531+0000","last_undegraded":"2024-04-26T23:51:20.628531+0000","last_fullsized":"2024-04-26T23:51:20.628531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:44.171307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628133+0000","last_change":"2024-04-26T23:51:18.594371+0000","last_active":"2024-04-26T23:51:20.628133+0000","last_peered":"2024-04-26T23:51:20.628133+0000","last_clean":"2024-04-26T23:51:20.628133+0000","last_became_active":"2024-04-26T23:51:18.593893+0000","last_became_peered":"2024-04-26T23:51:18.593893+0000","last_unstale":"2024-04-26T23:51:20.628133+0000","last_undegraded":"2024-04-26T23:51:20.628133+0000","last_fullsized":"2024-04-26T23:51:20.628133+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.167982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622527+0000","last_change":"2024-04-26T23:51:19.616480+0000","last_active":"2024-04-26T23:51:20.622527+0000","last_peered":"2024-04-26T23:51:20.622527+0000","last_clean":"2024-04-26T23:51:20.622527+0000","last_became_active":"2024-04-26T23:51:19.616165+0000","last_became_peered":"2024-04-26T23:51:19.616165+0000","last_unstale":"2024-04-26T23:51:20.622527+0000","last_undegraded":"2024-04-26T23:51:20.622527+0000","last_fullsized":"2024-04-26T23:51:20.622527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:45.854352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613502+0000","last_change":"2024-04-26T23:51:18.581691+0000","last_active":"2024-04-26T23:51:20.613502+0000","last_peered":"2024-04-26T23:51:20.613502+0000","last_clean":"2024-04-26T23:51:20.613502+0000","last_became_active":"2024-04-26T23:51:18.581381+0000","last_became_peered":"2024-04-26T23:51:18.581381+0000","last_unstale":"2024-04-26T23:51:20.613502+0000","last_undegraded":"2024-04-26T23:51:20.613502+0000","last_fullsized":"2024-04-26T23:51:20.613502+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:08.740895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622771+0000","last_change":"2024-04-26T23:51:19.630625+0000","last_active":"2024-04-26T23:51:20.622771+0000","last_peered":"2024-04-26T23:51:20.622771+0000","last_clean":"2024-04-26T23:51:20.622771+0000","last_became_active":"2024-04-26T23:51:19.630293+0000","last_became_peered":"2024-04-26T23:51:19.630293+0000","last_unstale":"2024-04-26T23:51:20.622771+0000","last_undegraded":"2024-04-26T23:51:20.622771+0000","last_fullsized":"2024-04-26T23:51:20.622771+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.502770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722427+0000","last_change":"2024-04-26T23:51:18.590044+0000","last_active":"2024-04-26T23:51:20.722427+0000","last_peered":"2024-04-26T23:51:20.722427+0000","last_clean":"2024-04-26T23:51:20.722427+0000","last_became_active":"2024-04-26T23:51:18.589461+0000","last_became_peered":"2024-04-26T23:51:18.589461+0000","last_unstale":"2024-04-26T23:51:20.722427+0000","last_undegraded":"2024-04-26T23:51:20.722427+0000","last_fullsized":"2024-04-26T23:51:20.722427+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:41.172650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627743+0000","last_change":"2024-04-26T23:51:19.610229+0000","last_active":"2024-04-26T23:51:20.627743+0000","last_peered":"2024-04-26T23:51:20.627743+0000","last_clean":"2024-04-26T23:51:20.627743+0000","last_became_active":"2024-04-26T23:51:19.609852+0000","last_became_peered":"2024-04-26T23:51:19.609852+0000","last_unstale":"2024-04-26T23:51:20.627743+0000","last_undegraded":"2024-04-26T23:51:20.627743+0000","last_fullsized":"2024-04-26T23:51:20.627743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:37.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610815+0000","last_change":"2024-04-26T23:51:18.579763+0000","last_active":"2024-04-26T23:51:20.610815+0000","last_peered":"2024-04-26T23:51:20.610815+0000","last_clean":"2024-04-26T23:51:20.610815+0000","last_became_active":"2024-04-26T23:51:18.579468+0000","last_became_peered":"2024-04-26T23:51:18.579468+0000","last_unstale":"2024-04-26T23:51:20.610815+0000","last_undegraded":"2024-04-26T23:51:20.610815+0000","last_fullsized":"2024-04-26T23:51:20.610815+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:33.236084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613474+0000","last_change":"2024-04-26T23:51:19.608294+0000","last_active":"2024-04-26T23:51:20.613474+0000","last_peered":"2024-04-26T23:51:20.613474+0000","last_clean":"2024-04-26T23:51:20.613474+0000","last_became_active":"2024-04-26T23:51:19.607994+0000","last_became_peered":"2024-04-26T23:51:19.607994+0000","last_unstale":"2024-04-26T23:51:20.613474+0000","last_undegraded":"2024-04-26T23:51:20.613474+0000","last_fullsized":"2024-04-26T23:51:20.613474+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:40.898557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614533+0000","last_change":"2024-04-26T23:51:18.581588+0000","last_active":"2024-04-26T23:51:20.614533+0000","last_peered":"2024-04-26T23:51:20.614533+0000","last_clean":"2024-04-26T23:51:20.614533+0000","last_became_active":"2024-04-26T23:51:18.580547+0000","last_became_peered":"2024-04-26T23:51:18.580547+0000","last_unstale":"2024-04-26T23:51:20.614533+0000","last_undegraded":"2024-04-26T23:51:20.614533+0000","last_fullsized":"2024-04-26T23:51:20.614533+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:46.306757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627492+0000","last_change":"2024-04-26T23:51:19.609150+0000","last_active":"2024-04-26T23:51:20.627492+0000","last_peered":"2024-04-26T23:51:20.627492+0000","last_clean":"2024-04-26T23:51:20.627492+0000","last_became_active":"2024-04-26T23:51:19.608807+0000","last_became_peered":"2024-04-26T23:51:19.608807+0000","last_unstale":"2024-04-26T23:51:20.627492+0000","last_undegraded":"2024-04-26T23:51:20.627492+0000","last_fullsized":"2024-04-26T23:51:20.627492+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:19.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610594+0000","last_change":"2024-04-26T23:51:18.589413+0000","last_active":"2024-04-26T23:51:20.610594+0000","last_peered":"2024-04-26T23:51:20.610594+0000","last_clean":"2024-04-26T23:51:20.610594+0000","last_became_active":"2024-04-26T23:51:18.589048+0000","last_became_peered":"2024-04-26T23:51:18.589048+0000","last_unstale":"2024-04-26T23:51:20.610594+0000","last_undegraded":"2024-04-26T23:51:20.610594+0000","last_fullsized":"2024-04-26T23:51:20.610594+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:43.936529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613110+0000","last_change":"2024-04-26T23:51:19.627172+0000","last_active":"2024-04-26T23:51:20.613110+0000","last_peered":"2024-04-26T23:51:20.613110+0000","last_clean":"2024-04-26T23:51:20.613110+0000","last_became_active":"2024-04-26T23:51:19.626847+0000","last_became_peered":"2024-04-26T23:51:19.626847+0000","last_unstale":"2024-04-26T23:51:20.613110+0000","last_undegraded":"2024-04-26T23:51:20.613110+0000","last_fullsized":"2024-04-26T23:51:20.613110+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:58.878022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614557+0000","last_change":"2024-04-26T23:51:18.582922+0000","last_active":"2024-04-26T23:51:20.614557+0000","last_peered":"2024-04-26T23:51:20.614557+0000","last_clean":"2024-04-26T23:51:20.614557+0000","last_became_active":"2024-04-26T23:51:18.582639+0000","last_became_peered":"2024-04-26T23:51:18.582639+0000","last_unstale":"2024-04-26T23:51:20.614557+0000","last_undegraded":"2024-04-26T23:51:20.614557+0000","last_fullsized":"2024-04-26T23:51:20.614557+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:44.944411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721600+0000","last_change":"2024-04-26T23:51:19.610136+0000","last_active":"2024-04-26T23:51:20.721600+0000","last_peered":"2024-04-26T23:51:20.721600+0000","last_clean":"2024-04-26T23:51:20.721600+0000","last_became_active":"2024-04-26T23:51:19.609482+0000","last_became_peered":"2024-04-26T23:51:19.609482+0000","last_unstale":"2024-04-26T23:51:20.721600+0000","last_undegraded":"2024-04-26T23:51:20.721600+0000","last_fullsized":"2024-04-26T23:51:20.721600+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:04.441710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722184+0000","last_change":"2024-04-26T23:51:18.591822+0000","last_active":"2024-04-26T23:51:20.722184+0000","last_peered":"2024-04-26T23:51:20.722184+0000","last_clean":"2024-04-26T23:51:20.722184+0000","last_became_active":"2024-04-26T23:51:18.589834+0000","last_became_peered":"2024-04-26T23:51:18.589834+0000","last_unstale":"2024-04-26T23:51:20.722184+0000","last_undegraded":"2024-04-26T23:51:20.722184+0000","last_fullsized":"2024-04-26T23:51:20.722184+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:10.618020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721988+0000","last_change":"2024-04-26T23:51:19.636358+0000","last_active":"2024-04-26T23:51:20.721988+0000","last_peered":"2024-04-26T23:51:20.721988+0000","last_clean":"2024-04-26T23:51:20.721988+0000","last_became_active":"2024-04-26T23:51:19.635750+0000","last_became_peered":"2024-04-26T23:51:19.635750+0000","last_unstale":"2024-04-26T23:51:20.721988+0000","last_undegraded":"2024-04-26T23:51:20.721988+0000","last_fullsized":"2024-04-26T23:51:20.721988+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:33.085937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"148'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.626426+0000","last_change":"2024-04-26T23:51:18.596814+0000","last_active":"2024-04-26T23:51:20.626426+0000","last_peered":"2024-04-26T23:51:20.626426+0000","last_clean":"2024-04-26T23:51:20.626426+0000","last_became_active":"2024-04-26T23:51:18.596494+0000","last_became_peered":"2024-04-26T23:51:18.596494+0000","last_unstale":"2024-04-26T23:51:20.626426+0000","last_undegraded":"2024-04-26T23:51:20.626426+0000","last_fullsized":"2024-04-26T23:51:20.626426+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:22.592932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622793+0000","last_change":"2024-04-26T23:51:19.611244+0000","last_active":"2024-04-26T23:51:20.622793+0000","last_peered":"2024-04-26T23:51:20.622793+0000","last_clean":"2024-04-26T23:51:20.622793+0000","last_became_active":"2024-04-26T23:51:19.610920+0000","last_became_peered":"2024-04-26T23:51:19.610920+0000","last_unstale":"2024-04-26T23:51:20.622793+0000","last_undegraded":"2024-04-26T23:51:20.622793+0000","last_fullsized":"2024-04-26T23:51:20.622793+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:22.096559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613621+0000","last_change":"2024-04-26T23:51:18.584895+0000","last_active":"2024-04-26T23:51:20.613621+0000","last_peered":"2024-04-26T23:51:20.613621+0000","last_clean":"2024-04-26T23:51:20.613621+0000","last_became_active":"2024-04-26T23:51:18.584540+0000","last_became_peered":"2024-04-26T23:51:18.584540+0000","last_unstale":"2024-04-26T23:51:20.613621+0000","last_undegraded":"2024-04-26T23:51:20.613621+0000","last_fullsized":"2024-04-26T23:51:20.613621+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:33.073901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721991+0000","last_change":"2024-04-26T23:51:19.629360+0000","last_active":"2024-04-26T23:51:20.721991+0000","last_peered":"2024-04-26T23:51:20.721991+0000","last_clean":"2024-04-26T23:51:20.721991+0000","last_became_active":"2024-04-26T23:51:19.625375+0000","last_became_peered":"2024-04-26T23:51:19.625375+0000","last_unstale":"2024-04-26T23:51:20.721991+0000","last_undegraded":"2024-04-26T23:51:20.721991+0000","last_fullsized":"2024-04-26T23:51:20.721991+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:27.259776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721623+0000","last_change":"2024-04-26T23:51:18.590151+0000","last_active":"2024-04-26T23:51:20.721623+0000","last_peered":"2024-04-26T23:51:20.721623+0000","last_clean":"2024-04-26T23:51:20.721623+0000","last_became_active":"2024-04-26T23:51:18.589720+0000","last_became_peered":"2024-04-26T23:51:18.589720+0000","last_unstale":"2024-04-26T23:51:20.721623+0000","last_undegraded":"2024-04-26T23:51:20.721623+0000","last_fullsized":"2024-04-26T23:51:20.721623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.882015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614565+0000","last_change":"2024-04-26T23:51:19.609938+0000","last_active":"2024-04-26T23:51:20.614565+0000","last_peered":"2024-04-26T23:51:20.614565+0000","last_clean":"2024-04-26T23:51:20.614565+0000","last_became_active":"2024-04-26T23:51:19.609630+0000","last_became_peered":"2024-04-26T23:51:19.609630+0000","last_unstale":"2024-04-26T23:51:20.614565+0000","last_undegraded":"2024-04-26T23:51:20.614565+0000","last_fullsized":"2024-04-26T23:51:20.614565+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:34.090443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622327+0000","last_change":"2024-04-26T23:51:18.591512+0000","last_active":"2024-04-26T23:51:20.622327+0000","last_peered":"2024-04-26T23:51:20.622327+0000","last_clean":"2024-04-26T23:51:20.622327+0000","last_became_active":"2024-04-26T23:51:18.591278+0000","last_became_peered":"2024-04-26T23:51:18.591278+0000","last_unstale":"2024-04-26T23:51:20.622327+0000","last_undegraded":"2024-04-26T23:51:20.622327+0000","last_fullsized":"2024-04-26T23:51:20.622327+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:48.587919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722038+0000","last_change":"2024-04-26T23:51:19.632032+0000","last_active":"2024-04-26T23:51:20.722038+0000","last_peered":"2024-04-26T23:51:20.722038+0000","last_clean":"2024-04-26T23:51:20.722038+0000","last_became_active":"2024-04-26T23:51:19.631773+0000","last_became_peered":"2024-04-26T23:51:19.631773+0000","last_unstale":"2024-04-26T23:51:20.722038+0000","last_undegraded":"2024-04-26T23:51:20.722038+0000","last_fullsized":"2024-04-26T23:51:20.722038+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:25.077822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721197+0000","last_change":"2024-04-26T23:51:18.586882+0000","last_active":"2024-04-26T23:51:20.721197+0000","last_peered":"2024-04-26T23:51:20.721197+0000","last_clean":"2024-04-26T23:51:20.721197+0000","last_became_active":"2024-04-26T23:51:18.586540+0000","last_became_peered":"2024-04-26T23:51:18.586540+0000","last_unstale":"2024-04-26T23:51:20.721197+0000","last_undegraded":"2024-04-26T23:51:20.721197+0000","last_fullsized":"2024-04-26T23:51:20.721197+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:07.791463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.612961+0000","last_change":"2024-04-26T23:51:19.607712+0000","last_active":"2024-04-26T23:51:20.612961+0000","last_peered":"2024-04-26T23:51:20.612961+0000","last_clean":"2024-04-26T23:51:20.612961+0000","last_became_active":"2024-04-26T23:51:19.607373+0000","last_became_peered":"2024-04-26T23:51:19.607373+0000","last_unstale":"2024-04-26T23:51:20.612961+0000","last_undegraded":"2024-04-26T23:51:20.612961+0000","last_fullsized":"2024-04-26T23:51:20.612961+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:45.402405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622192+0000","last_change":"2024-04-26T23:51:18.587659+0000","last_active":"2024-04-26T23:51:20.622192+0000","last_peered":"2024-04-26T23:51:20.622192+0000","last_clean":"2024-04-26T23:51:20.622192+0000","last_became_active":"2024-04-26T23:51:18.586970+0000","last_became_peered":"2024-04-26T23:51:18.586970+0000","last_unstale":"2024-04-26T23:51:20.622192+0000","last_undegraded":"2024-04-26T23:51:20.622192+0000","last_fullsized":"2024-04-26T23:51:20.622192+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:24.090544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614873+0000","last_change":"2024-04-26T23:51:19.628716+0000","last_active":"2024-04-26T23:51:20.614873+0000","last_peered":"2024-04-26T23:51:20.614873+0000","last_clean":"2024-04-26T23:51:20.614873+0000","last_became_active":"2024-04-26T23:51:19.628376+0000","last_became_peered":"2024-04-26T23:51:19.628376+0000","last_unstale":"2024-04-26T23:51:20.614873+0000","last_undegraded":"2024-04-26T23:51:20.614873+0000","last_fullsized":"2024-04-26T23:51:20.614873+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:23.585513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627478+0000","last_change":"2024-04-26T23:51:18.590456+0000","last_active":"2024-04-26T23:51:20.627478+0000","last_peered":"2024-04-26T23:51:20.627478+0000","last_clean":"2024-04-26T23:51:20.627478+0000","last_became_active":"2024-04-26T23:51:18.590107+0000","last_became_peered":"2024-04-26T23:51:18.590107+0000","last_unstale":"2024-04-26T23:51:20.627478+0000","last_undegraded":"2024-04-26T23:51:20.627478+0000","last_fullsized":"2024-04-26T23:51:20.627478+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.882015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615479+0000","last_change":"2024-04-26T23:51:19.610639+0000","last_active":"2024-04-26T23:51:20.615479+0000","last_peered":"2024-04-26T23:51:20.615479+0000","last_clean":"2024-04-26T23:51:20.615479+0000","last_became_active":"2024-04-26T23:51:19.608202+0000","last_became_peered":"2024-04-26T23:51:19.608202+0000","last_unstale":"2024-04-26T23:51:20.615479+0000","last_undegraded":"2024-04-26T23:51:20.615479+0000","last_fullsized":"2024-04-26T23:51:20.615479+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:40.010712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"147'6","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610675+0000","last_change":"2024-04-26T23:51:18.589384+0000","last_active":"2024-04-26T23:51:20.610675+0000","last_peered":"2024-04-26T23:51:20.610675+0000","last_clean":"2024-04-26T23:51:20.610675+0000","last_became_active":"2024-04-26T23:51:18.589069+0000","last_became_peered":"2024-04-26T23:51:18.589069+0000","last_unstale":"2024-04-26T23:51:20.610675+0000","last_undegraded":"2024-04-26T23:51:20.610675+0000","last_fullsized":"2024-04-26T23:51:20.610675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:43.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627655+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:51:20.627655+0000","last_peered":"2024-04-26T23:51:20.627655+0000","last_clean":"2024-04-26T23:51:20.627655+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:51:20.627655+0000","last_undegraded":"2024-04-26T23:51:20.627655+0000","last_fullsized":"2024-04-26T23:51:20.627655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47472,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324368,"statfs":{"total":96636764160,"available":96588152832,"internally_reserved":0,"allocated":2453504,"data_stored":1463608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":1847296,"data_stored":866596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":1843200,"data_stored":866526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55208,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1855488,"data_stored":868138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55216,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316624,"statfs":{"total":96636764160,"available":96580222976,"internally_reserved":0,"allocated":1863680,"data_stored":873834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59364,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312476,"statfs":{"total":96636764160,"available":96575975424,"internally_reserved":0,"allocated":1851392,"data_stored":866638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55204,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1859584,"data_stored":873288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55796,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316044,"statfs":{"total":96636764160,"available":96579629056,"internally_reserved":0,"allocated":2461696,"data_stored":1459212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:51:24.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:51:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.324+0000 7f9774e00640 1 Processor -- start 2024-04-26T23:51:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.324+0000 7f9774e00640 1 -- start start 2024-04-26T23:51:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f9774e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9770160bd0 0x7f9770161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f9774e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97700732c0 0x7f9770073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f9774e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9770161940 con 0x7f977006e510 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f9774e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9770071f80 con 0x7f97700732c0 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f9774e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9770072260 con 0x7f9770160bd0 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f976de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f977006e510 0x7f977006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60216/0 (socket says 172.21.15.46:60216) 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f976de00640 1 -- 172.21.15.46:0/1536724874 learned_addr learned my addr 172.21.15.46:0/1536724874 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.326+0000 7f976e800640 1 --2- 172.21.15.46:0/1536724874 >> v2:172.21.15.46:3300/0 conn(0x7f97700732c0 0x7f9770073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976ca00640 1 -- 172.21.15.46:0/1536724874 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516832848 0 0) 0x7f9770161940 con 0x7f977006e510 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976ca00640 1 -- 172.21.15.46:0/1536724874 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9754003520 con 0x7f977006e510 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976e800640 1 -- 172.21.15.46:0/1536724874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9770160bd0 msgr2=0x7f9770161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976e800640 1 --2- 172.21.15.46:0/1536724874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9770160bd0 0x7f9770161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976e800640 1 -- 172.21.15.46:0/1536724874 >> v1:172.21.15.46:6789/0 conn(0x7f977006e510 legacy=0x7f977006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.327+0000 7f976e800640 1 -- 172.21.15.46:0/1536724874 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9770072540 con 0x7f97700732c0 2024-04-26T23:51:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.328+0000 7f976e800640 1 --2- 172.21.15.46:0/1536724874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97700732c0 0x7f9770073750 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d90 tx=0x7f976802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7704dd78e4beb24e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f976ca00640 1 -- 172.21.15.46:0/1536724874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768007740 con 0x7f97700732c0 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f976ca00640 1 -- 172.21.15.46:0/1536724874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97680078e0 con 0x7f97700732c0 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f976ca00640 1 -- 172.21.15.46:0/1536724874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768007cb0 con 0x7f97700732c0 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 -- 172.21.15.46:0/1536724874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97700732c0 msgr2=0x7f9770073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 --2- 172.21.15.46:0/1536724874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97700732c0 0x7f9770073750 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d90 tx=0x7f976802a660 comp rx=0 tx=0).stop 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 -- 172.21.15.46:0/1536724874 shutdown_connections 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 --2- 172.21.15.46:0/1536724874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97700732c0 0x7f9770073750 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 --2- 172.21.15.46:0/1536724874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9770160bd0 0x7f9770161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 -- 172.21.15.46:0/1536724874 >> 172.21.15.46:0/1536724874 conn(0x7f977006c530 msgr2=0x7f977006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 -- 172.21.15.46:0/1536724874 shutdown_connections 2024-04-26T23:51:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.329+0000 7f9774e00640 1 -- 172.21.15.46:0/1536724874 wait complete. 2024-04-26T23:51:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.331+0000 7f9774e00640 1 Processor -- start 2024-04-26T23:51:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.331+0000 7f9774e00640 1 -- start start 2024-04-26T23:51:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97700732c0 0x7f977013c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770160bd0 0x7f97701e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97701616f0 con 0x7f977006e510 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9770161570 con 0x7f9770160bd0 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f9774e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9770072d40 con 0x7f97700732c0 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f976de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f976de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33688/0 (socket says 172.21.15.46:33688) 2024-04-26T23:51:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.332+0000 7f976de00640 1 -- 172.21.15.46:0/2073217936 learned_addr learned my addr 172.21.15.46:0/2073217936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 -- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97700732c0 msgr2=0x7f977013c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97700732c0 0x7f977013c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 -- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770160bd0 msgr2=0x7f97701e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770160bd0 0x7f97701e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 -- 172.21.15.46:0/2073217936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700af2f0 con 0x7f977006e510 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f976de00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f976000c390 tx=0x7f976000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760017070 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9760005ce0 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600072f0 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977015f090 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.333+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97700af590 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.335+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770069530 con 0x7f977006e510 2024-04-26T23:51:24.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.336+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f976001a070 con 0x7f977006e510 2024-04-26T23:51:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.336+0000 7f9766000640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.337+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f976008b1a0 con 0x7f977006e510 2024-04-26T23:51:24.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.337+0000 7f976d400640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.339+0000 7f976d400640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 0x7f975006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f977015cae0 tx=0x7f975c006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.341+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97600593a0 con 0x7f977006e510 2024-04-26T23:51:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.532+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f977006fb00 con 0x7f975006c8f0 2024-04-26T23:51:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.551+0000 7f9766000640 1 -- 172.21.15.46:0/2073217936 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f977006fb00 con 0x7f975006c8f0 2024-04-26T23:51:24.557 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:51:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 msgr2=0x7f975006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 0x7f975006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f977015cae0 tx=0x7f975c006c90 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 msgr2=0x7f97700afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f976000c390 tx=0x7f976000c850 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 shutdown_connections 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770160bd0 0x7f97701e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97700732c0 0x7f977013c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 --2- 172.21.15.46:0/2073217936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f977006e510 0x7f97700afd90 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 >> 172.21.15.46:0/2073217936 conn(0x7f977006c530 msgr2=0x7f977015bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.557+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 shutdown_connections 2024-04-26T23:51:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.558+0000 7f9774e00640 1 -- 172.21.15.46:0/2073217936 wait complete. 2024-04-26T23:51:24.579 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":573,"stamp":"2024-04-26T23:51:24.209409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430332,"kb_used_data":15660,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754544388,"statfs":{"total":773094113280,"available":772653453312,"internally_reserved":0,"allocated":16035840,"data_stored":8137840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002187"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622015+0000","last_change":"2024-04-26T23:51:19.621972+0000","last_active":"2024-04-26T23:51:20.622015+0000","last_peered":"2024-04-26T23:51:20.622015+0000","last_clean":"2024-04-26T23:51:20.622015+0000","last_became_active":"2024-04-26T23:51:19.617849+0000","last_became_peered":"2024-04-26T23:51:19.617849+0000","last_unstale":"2024-04-26T23:51:20.622015+0000","last_undegraded":"2024-04-26T23:51:20.622015+0000","last_fullsized":"2024-04-26T23:51:20.622015+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:10.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613743+0000","last_change":"2024-04-26T23:51:19.626476+0000","last_active":"2024-04-26T23:51:20.613743+0000","last_peered":"2024-04-26T23:51:20.613743+0000","last_clean":"2024-04-26T23:51:20.613743+0000","last_became_active":"2024-04-26T23:51:19.624748+0000","last_became_peered":"2024-04-26T23:51:19.624748+0000","last_unstale":"2024-04-26T23:51:20.613743+0000","last_undegraded":"2024-04-26T23:51:20.613743+0000","last_fullsized":"2024-04-26T23:51:20.613743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:08.293570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613291+0000","last_change":"2024-04-26T23:51:19.597522+0000","last_active":"2024-04-26T23:51:20.613291+0000","last_peered":"2024-04-26T23:51:20.613291+0000","last_clean":"2024-04-26T23:51:20.613291+0000","last_became_active":"2024-04-26T23:51:19.596323+0000","last_became_peered":"2024-04-26T23:51:19.596323+0000","last_unstale":"2024-04-26T23:51:20.613291+0000","last_undegraded":"2024-04-26T23:51:20.613291+0000","last_fullsized":"2024-04-26T23:51:20.613291+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:34.079085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722159+0000","last_change":"2024-04-26T23:51:19.621354+0000","last_active":"2024-04-26T23:51:20.722159+0000","last_peered":"2024-04-26T23:51:20.722159+0000","last_clean":"2024-04-26T23:51:20.722159+0000","last_became_active":"2024-04-26T23:51:19.621082+0000","last_became_peered":"2024-04-26T23:51:19.621082+0000","last_unstale":"2024-04-26T23:51:20.722159+0000","last_undegraded":"2024-04-26T23:51:20.722159+0000","last_fullsized":"2024-04-26T23:51:20.722159+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:40.010712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721701+0000","last_change":"2024-04-26T23:51:19.620091+0000","last_active":"2024-04-26T23:51:20.721701+0000","last_peered":"2024-04-26T23:51:20.721701+0000","last_clean":"2024-04-26T23:51:20.721701+0000","last_became_active":"2024-04-26T23:51:19.619409+0000","last_became_peered":"2024-04-26T23:51:19.619409+0000","last_unstale":"2024-04-26T23:51:20.721701+0000","last_undegraded":"2024-04-26T23:51:20.721701+0000","last_fullsized":"2024-04-26T23:51:20.721701+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:12.977331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721958+0000","last_change":"2024-04-26T23:51:19.618710+0000","last_active":"2024-04-26T23:51:20.721958+0000","last_peered":"2024-04-26T23:51:20.721958+0000","last_clean":"2024-04-26T23:51:20.721958+0000","last_became_active":"2024-04-26T23:51:19.618374+0000","last_became_peered":"2024-04-26T23:51:19.618374+0000","last_unstale":"2024-04-26T23:51:20.721958+0000","last_undegraded":"2024-04-26T23:51:20.721958+0000","last_fullsized":"2024-04-26T23:51:20.721958+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:00.404120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621908+0000","last_change":"2024-04-26T23:51:19.621932+0000","last_active":"2024-04-26T23:51:20.621908+0000","last_peered":"2024-04-26T23:51:20.621908+0000","last_clean":"2024-04-26T23:51:20.621908+0000","last_became_active":"2024-04-26T23:51:19.620677+0000","last_became_peered":"2024-04-26T23:51:19.620677+0000","last_unstale":"2024-04-26T23:51:20.621908+0000","last_undegraded":"2024-04-26T23:51:20.621908+0000","last_fullsized":"2024-04-26T23:51:20.621908+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:09.935864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613761+0000","last_change":"2024-04-26T23:51:19.613977+0000","last_active":"2024-04-26T23:51:20.613761+0000","last_peered":"2024-04-26T23:51:20.613761+0000","last_clean":"2024-04-26T23:51:20.613761+0000","last_became_active":"2024-04-26T23:51:19.613448+0000","last_became_peered":"2024-04-26T23:51:19.613448+0000","last_unstale":"2024-04-26T23:51:20.613761+0000","last_undegraded":"2024-04-26T23:51:20.613761+0000","last_fullsized":"2024-04-26T23:51:20.613761+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:22.960830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721813+0000","last_change":"2024-04-26T23:51:19.615307+0000","last_active":"2024-04-26T23:51:20.721813+0000","last_peered":"2024-04-26T23:51:20.721813+0000","last_clean":"2024-04-26T23:51:20.721813+0000","last_became_active":"2024-04-26T23:51:19.614998+0000","last_became_peered":"2024-04-26T23:51:19.614998+0000","last_unstale":"2024-04-26T23:51:20.721813+0000","last_undegraded":"2024-04-26T23:51:20.721813+0000","last_fullsized":"2024-04-26T23:51:20.721813+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:40.662270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722264+0000","last_change":"2024-04-26T23:51:19.624604+0000","last_active":"2024-04-26T23:51:20.722264+0000","last_peered":"2024-04-26T23:51:20.722264+0000","last_clean":"2024-04-26T23:51:20.722264+0000","last_became_active":"2024-04-26T23:51:19.622398+0000","last_became_peered":"2024-04-26T23:51:19.622398+0000","last_unstale":"2024-04-26T23:51:20.722264+0000","last_undegraded":"2024-04-26T23:51:20.722264+0000","last_fullsized":"2024-04-26T23:51:20.722264+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:59.999085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721592+0000","last_change":"2024-04-26T23:51:19.623250+0000","last_active":"2024-04-26T23:51:20.721592+0000","last_peered":"2024-04-26T23:51:20.721592+0000","last_clean":"2024-04-26T23:51:20.721592+0000","last_became_active":"2024-04-26T23:51:19.622870+0000","last_became_peered":"2024-04-26T23:51:19.622870+0000","last_unstale":"2024-04-26T23:51:20.721592+0000","last_undegraded":"2024-04-26T23:51:20.721592+0000","last_fullsized":"2024-04-26T23:51:20.721592+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:17.049158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722222+0000","last_change":"2024-04-26T23:51:19.632948+0000","last_active":"2024-04-26T23:51:20.722222+0000","last_peered":"2024-04-26T23:51:20.722222+0000","last_clean":"2024-04-26T23:51:20.722222+0000","last_became_active":"2024-04-26T23:51:19.630358+0000","last_became_peered":"2024-04-26T23:51:19.630358+0000","last_unstale":"2024-04-26T23:51:20.722222+0000","last_undegraded":"2024-04-26T23:51:20.722222+0000","last_fullsized":"2024-04-26T23:51:20.722222+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:53.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721274+0000","last_change":"2024-04-26T23:51:19.618669+0000","last_active":"2024-04-26T23:51:20.721274+0000","last_peered":"2024-04-26T23:51:20.721274+0000","last_clean":"2024-04-26T23:51:20.721274+0000","last_became_active":"2024-04-26T23:51:19.617258+0000","last_became_peered":"2024-04-26T23:51:19.617258+0000","last_unstale":"2024-04-26T23:51:20.721274+0000","last_undegraded":"2024-04-26T23:51:20.721274+0000","last_fullsized":"2024-04-26T23:51:20.721274+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.417014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721710+0000","last_change":"2024-04-26T23:51:19.618677+0000","last_active":"2024-04-26T23:51:20.721710+0000","last_peered":"2024-04-26T23:51:20.721710+0000","last_clean":"2024-04-26T23:51:20.721710+0000","last_became_active":"2024-04-26T23:51:19.617126+0000","last_became_peered":"2024-04-26T23:51:19.617126+0000","last_unstale":"2024-04-26T23:51:20.721710+0000","last_undegraded":"2024-04-26T23:51:20.721710+0000","last_fullsized":"2024-04-26T23:51:20.721710+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:09.757437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613133+0000","last_change":"2024-04-26T23:51:19.627565+0000","last_active":"2024-04-26T23:51:20.613133+0000","last_peered":"2024-04-26T23:51:20.613133+0000","last_clean":"2024-04-26T23:51:20.613133+0000","last_became_active":"2024-04-26T23:51:19.627281+0000","last_became_peered":"2024-04-26T23:51:19.627281+0000","last_unstale":"2024-04-26T23:51:20.613133+0000","last_undegraded":"2024-04-26T23:51:20.613133+0000","last_fullsized":"2024-04-26T23:51:20.613133+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.919962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720949+0000","last_change":"2024-04-26T23:51:19.610349+0000","last_active":"2024-04-26T23:51:20.720949+0000","last_peered":"2024-04-26T23:51:20.720949+0000","last_clean":"2024-04-26T23:51:20.720949+0000","last_became_active":"2024-04-26T23:51:19.609882+0000","last_became_peered":"2024-04-26T23:51:19.609882+0000","last_unstale":"2024-04-26T23:51:20.720949+0000","last_undegraded":"2024-04-26T23:51:20.720949+0000","last_fullsized":"2024-04-26T23:51:20.720949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:34.252626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721939+0000","last_change":"2024-04-26T23:51:19.621900+0000","last_active":"2024-04-26T23:51:20.721939+0000","last_peered":"2024-04-26T23:51:20.721939+0000","last_clean":"2024-04-26T23:51:20.721939+0000","last_became_active":"2024-04-26T23:51:19.621577+0000","last_became_peered":"2024-04-26T23:51:19.621577+0000","last_unstale":"2024-04-26T23:51:20.721939+0000","last_undegraded":"2024-04-26T23:51:20.721939+0000","last_fullsized":"2024-04-26T23:51:20.721939+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:34.090443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722407+0000","last_change":"2024-04-26T23:51:19.625828+0000","last_active":"2024-04-26T23:51:20.722407+0000","last_peered":"2024-04-26T23:51:20.722407+0000","last_clean":"2024-04-26T23:51:20.722407+0000","last_became_active":"2024-04-26T23:51:19.621356+0000","last_became_peered":"2024-04-26T23:51:19.621356+0000","last_unstale":"2024-04-26T23:51:20.722407+0000","last_undegraded":"2024-04-26T23:51:20.722407+0000","last_fullsized":"2024-04-26T23:51:20.722407+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:14.350308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628651+0000","last_change":"2024-04-26T23:51:19.615702+0000","last_active":"2024-04-26T23:51:20.628651+0000","last_peered":"2024-04-26T23:51:20.628651+0000","last_clean":"2024-04-26T23:51:20.628651+0000","last_became_active":"2024-04-26T23:51:19.614390+0000","last_became_peered":"2024-04-26T23:51:19.614390+0000","last_unstale":"2024-04-26T23:51:20.628651+0000","last_undegraded":"2024-04-26T23:51:20.628651+0000","last_fullsized":"2024-04-26T23:51:20.628651+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.417014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610740+0000","last_change":"2024-04-26T23:51:19.596380+0000","last_active":"2024-04-26T23:51:20.610740+0000","last_peered":"2024-04-26T23:51:20.610740+0000","last_clean":"2024-04-26T23:51:20.610740+0000","last_became_active":"2024-04-26T23:51:19.595545+0000","last_became_peered":"2024-04-26T23:51:19.595545+0000","last_unstale":"2024-04-26T23:51:20.610740+0000","last_undegraded":"2024-04-26T23:51:20.610740+0000","last_fullsized":"2024-04-26T23:51:20.610740+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:04.441710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721517+0000","last_change":"2024-04-26T23:51:19.618246+0000","last_active":"2024-04-26T23:51:20.721517+0000","last_peered":"2024-04-26T23:51:20.721517+0000","last_clean":"2024-04-26T23:51:20.721517+0000","last_became_active":"2024-04-26T23:51:19.617653+0000","last_became_peered":"2024-04-26T23:51:19.617653+0000","last_unstale":"2024-04-26T23:51:20.721517+0000","last_undegraded":"2024-04-26T23:51:20.721517+0000","last_fullsized":"2024-04-26T23:51:20.721517+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:09.604539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621650+0000","last_change":"2024-04-26T23:51:19.621450+0000","last_active":"2024-04-26T23:51:20.621650+0000","last_peered":"2024-04-26T23:51:20.621650+0000","last_clean":"2024-04-26T23:51:20.621650+0000","last_became_active":"2024-04-26T23:51:19.621165+0000","last_became_peered":"2024-04-26T23:51:19.621165+0000","last_unstale":"2024-04-26T23:51:20.621650+0000","last_undegraded":"2024-04-26T23:51:20.621650+0000","last_fullsized":"2024-04-26T23:51:20.621650+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:08.293570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613793+0000","last_change":"2024-04-26T23:51:19.614650+0000","last_active":"2024-04-26T23:51:20.613793+0000","last_peered":"2024-04-26T23:51:20.613793+0000","last_clean":"2024-04-26T23:51:20.613793+0000","last_became_active":"2024-04-26T23:51:19.614352+0000","last_became_peered":"2024-04-26T23:51:19.614352+0000","last_unstale":"2024-04-26T23:51:20.613793+0000","last_undegraded":"2024-04-26T23:51:20.613793+0000","last_fullsized":"2024-04-26T23:51:20.613793+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:06.816362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720694+0000","last_change":"2024-04-26T23:51:19.613042+0000","last_active":"2024-04-26T23:51:20.720694+0000","last_peered":"2024-04-26T23:51:20.720694+0000","last_clean":"2024-04-26T23:51:20.720694+0000","last_became_active":"2024-04-26T23:51:19.612743+0000","last_became_peered":"2024-04-26T23:51:19.612743+0000","last_unstale":"2024-04-26T23:51:20.720694+0000","last_undegraded":"2024-04-26T23:51:20.720694+0000","last_fullsized":"2024-04-26T23:51:20.720694+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:32.286079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622096+0000","last_change":"2024-04-26T23:51:19.612217+0000","last_active":"2024-04-26T23:51:20.622096+0000","last_peered":"2024-04-26T23:51:20.622096+0000","last_clean":"2024-04-26T23:51:20.622096+0000","last_became_active":"2024-04-26T23:51:19.611885+0000","last_became_peered":"2024-04-26T23:51:19.611885+0000","last_unstale":"2024-04-26T23:51:20.622096+0000","last_undegraded":"2024-04-26T23:51:20.622096+0000","last_fullsized":"2024-04-26T23:51:20.622096+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:24.088709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622309+0000","last_change":"2024-04-26T23:51:19.616764+0000","last_active":"2024-04-26T23:51:20.622309+0000","last_peered":"2024-04-26T23:51:20.622309+0000","last_clean":"2024-04-26T23:51:20.622309+0000","last_became_active":"2024-04-26T23:51:19.616341+0000","last_became_peered":"2024-04-26T23:51:19.616341+0000","last_unstale":"2024-04-26T23:51:20.622309+0000","last_undegraded":"2024-04-26T23:51:20.622309+0000","last_fullsized":"2024-04-26T23:51:20.622309+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:33.826967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721989+0000","last_change":"2024-04-26T23:51:19.618541+0000","last_active":"2024-04-26T23:51:20.721989+0000","last_peered":"2024-04-26T23:51:20.721989+0000","last_clean":"2024-04-26T23:51:20.721989+0000","last_became_active":"2024-04-26T23:51:19.618222+0000","last_became_peered":"2024-04-26T23:51:19.618222+0000","last_unstale":"2024-04-26T23:51:20.721989+0000","last_undegraded":"2024-04-26T23:51:20.721989+0000","last_fullsized":"2024-04-26T23:51:20.721989+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.523540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720723+0000","last_change":"2024-04-26T23:51:19.612197+0000","last_active":"2024-04-26T23:51:20.720723+0000","last_peered":"2024-04-26T23:51:20.720723+0000","last_clean":"2024-04-26T23:51:20.720723+0000","last_became_active":"2024-04-26T23:51:19.611895+0000","last_became_peered":"2024-04-26T23:51:19.611895+0000","last_unstale":"2024-04-26T23:51:20.720723+0000","last_undegraded":"2024-04-26T23:51:20.720723+0000","last_fullsized":"2024-04-26T23:51:20.720723+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:03.643122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615043+0000","last_change":"2024-04-26T23:51:19.627125+0000","last_active":"2024-04-26T23:51:20.615043+0000","last_peered":"2024-04-26T23:51:20.615043+0000","last_clean":"2024-04-26T23:51:20.615043+0000","last_became_active":"2024-04-26T23:51:19.626833+0000","last_became_peered":"2024-04-26T23:51:19.626833+0000","last_unstale":"2024-04-26T23:51:20.615043+0000","last_undegraded":"2024-04-26T23:51:20.615043+0000","last_fullsized":"2024-04-26T23:51:20.615043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:33.085937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720911+0000","last_change":"2024-04-26T23:51:19.622983+0000","last_active":"2024-04-26T23:51:20.720911+0000","last_peered":"2024-04-26T23:51:20.720911+0000","last_clean":"2024-04-26T23:51:20.720911+0000","last_became_active":"2024-04-26T23:51:19.621519+0000","last_became_peered":"2024-04-26T23:51:19.621519+0000","last_unstale":"2024-04-26T23:51:20.720911+0000","last_undegraded":"2024-04-26T23:51:20.720911+0000","last_fullsized":"2024-04-26T23:51:20.720911+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:14.252548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721273+0000","last_change":"2024-04-26T23:51:19.613168+0000","last_active":"2024-04-26T23:51:20.721273+0000","last_peered":"2024-04-26T23:51:20.721273+0000","last_clean":"2024-04-26T23:51:20.721273+0000","last_became_active":"2024-04-26T23:51:19.610951+0000","last_became_peered":"2024-04-26T23:51:19.610951+0000","last_unstale":"2024-04-26T23:51:20.721273+0000","last_undegraded":"2024-04-26T23:51:20.721273+0000","last_fullsized":"2024-04-26T23:51:20.721273+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:44.171307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614151+0000","last_change":"2024-04-26T23:51:19.613739+0000","last_active":"2024-04-26T23:51:20.614151+0000","last_peered":"2024-04-26T23:51:20.614151+0000","last_clean":"2024-04-26T23:51:20.614151+0000","last_became_active":"2024-04-26T23:51:19.613053+0000","last_became_peered":"2024-04-26T23:51:19.613053+0000","last_unstale":"2024-04-26T23:51:20.614151+0000","last_undegraded":"2024-04-26T23:51:20.614151+0000","last_fullsized":"2024-04-26T23:51:20.614151+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:12.606703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613900+0000","last_change":"2024-04-26T23:51:19.602894+0000","last_active":"2024-04-26T23:51:20.613900+0000","last_peered":"2024-04-26T23:51:20.613900+0000","last_clean":"2024-04-26T23:51:20.613900+0000","last_became_active":"2024-04-26T23:51:19.602509+0000","last_became_peered":"2024-04-26T23:51:19.602509+0000","last_unstale":"2024-04-26T23:51:20.613900+0000","last_undegraded":"2024-04-26T23:51:20.613900+0000","last_fullsized":"2024-04-26T23:51:20.613900+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:10.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"147'4","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722059+0000","last_change":"2024-04-26T23:51:18.576956+0000","last_active":"2024-04-26T23:51:20.722059+0000","last_peered":"2024-04-26T23:51:20.722059+0000","last_clean":"2024-04-26T23:51:20.722059+0000","last_became_active":"2024-04-26T23:51:18.576654+0000","last_became_peered":"2024-04-26T23:51:18.576654+0000","last_unstale":"2024-04-26T23:51:20.722059+0000","last_undegraded":"2024-04-26T23:51:20.722059+0000","last_fullsized":"2024-04-26T23:51:20.722059+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:34.066906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613773+0000","last_change":"2024-04-26T23:51:19.622153+0000","last_active":"2024-04-26T23:51:20.613773+0000","last_peered":"2024-04-26T23:51:20.613773+0000","last_clean":"2024-04-26T23:51:20.613773+0000","last_became_active":"2024-04-26T23:51:19.621851+0000","last_became_peered":"2024-04-26T23:51:19.621851+0000","last_unstale":"2024-04-26T23:51:20.613773+0000","last_undegraded":"2024-04-26T23:51:20.613773+0000","last_fullsized":"2024-04-26T23:51:20.613773+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:09.935864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627947+0000","last_change":"2024-04-26T23:51:18.594579+0000","last_active":"2024-04-26T23:51:20.627947+0000","last_peered":"2024-04-26T23:51:20.627947+0000","last_clean":"2024-04-26T23:51:20.627947+0000","last_became_active":"2024-04-26T23:51:18.594280+0000","last_became_peered":"2024-04-26T23:51:18.594280+0000","last_unstale":"2024-04-26T23:51:20.627947+0000","last_undegraded":"2024-04-26T23:51:20.627947+0000","last_fullsized":"2024-04-26T23:51:20.627947+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:07.277028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615337+0000","last_change":"2024-04-26T23:51:19.624493+0000","last_active":"2024-04-26T23:51:20.615337+0000","last_peered":"2024-04-26T23:51:20.615337+0000","last_clean":"2024-04-26T23:51:20.615337+0000","last_became_active":"2024-04-26T23:51:19.624184+0000","last_became_peered":"2024-04-26T23:51:19.624184+0000","last_unstale":"2024-04-26T23:51:20.615337+0000","last_undegraded":"2024-04-26T23:51:20.615337+0000","last_fullsized":"2024-04-26T23:51:20.615337+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:25.077822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628714+0000","last_change":"2024-04-26T23:51:18.596813+0000","last_active":"2024-04-26T23:51:20.628714+0000","last_peered":"2024-04-26T23:51:20.628714+0000","last_clean":"2024-04-26T23:51:20.628714+0000","last_became_active":"2024-04-26T23:51:18.596522+0000","last_became_peered":"2024-04-26T23:51:18.596522+0000","last_unstale":"2024-04-26T23:51:20.628714+0000","last_undegraded":"2024-04-26T23:51:20.628714+0000","last_fullsized":"2024-04-26T23:51:20.628714+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:08.919322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628020+0000","last_change":"2024-04-26T23:51:19.608539+0000","last_active":"2024-04-26T23:51:20.628020+0000","last_peered":"2024-04-26T23:51:20.628020+0000","last_clean":"2024-04-26T23:51:20.628020+0000","last_became_active":"2024-04-26T23:51:19.608206+0000","last_became_peered":"2024-04-26T23:51:19.608206+0000","last_unstale":"2024-04-26T23:51:20.628020+0000","last_undegraded":"2024-04-26T23:51:20.628020+0000","last_fullsized":"2024-04-26T23:51:20.628020+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:44.953071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613922+0000","last_change":"2024-04-26T23:51:18.584926+0000","last_active":"2024-04-26T23:51:20.613922+0000","last_peered":"2024-04-26T23:51:20.613922+0000","last_clean":"2024-04-26T23:51:20.613922+0000","last_became_active":"2024-04-26T23:51:18.584626+0000","last_became_peered":"2024-04-26T23:51:18.584626+0000","last_unstale":"2024-04-26T23:51:20.613922+0000","last_undegraded":"2024-04-26T23:51:20.613922+0000","last_fullsized":"2024-04-26T23:51:20.613922+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.590161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.611018+0000","last_change":"2024-04-26T23:51:19.615638+0000","last_active":"2024-04-26T23:51:20.611018+0000","last_peered":"2024-04-26T23:51:20.611018+0000","last_clean":"2024-04-26T23:51:20.611018+0000","last_became_active":"2024-04-26T23:51:19.614596+0000","last_became_peered":"2024-04-26T23:51:19.614596+0000","last_unstale":"2024-04-26T23:51:20.611018+0000","last_undegraded":"2024-04-26T23:51:20.611018+0000","last_fullsized":"2024-04-26T23:51:20.611018+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:12.977331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615243+0000","last_change":"2024-04-26T23:51:18.581856+0000","last_active":"2024-04-26T23:51:20.615243+0000","last_peered":"2024-04-26T23:51:20.615243+0000","last_clean":"2024-04-26T23:51:20.615243+0000","last_became_active":"2024-04-26T23:51:18.580751+0000","last_became_peered":"2024-04-26T23:51:18.580751+0000","last_unstale":"2024-04-26T23:51:20.615243+0000","last_undegraded":"2024-04-26T23:51:20.615243+0000","last_fullsized":"2024-04-26T23:51:20.615243+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:31.005824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721984+0000","last_change":"2024-04-26T23:51:19.617803+0000","last_active":"2024-04-26T23:51:20.721984+0000","last_peered":"2024-04-26T23:51:20.721984+0000","last_clean":"2024-04-26T23:51:20.721984+0000","last_became_active":"2024-04-26T23:51:19.617478+0000","last_became_peered":"2024-04-26T23:51:19.617478+0000","last_unstale":"2024-04-26T23:51:20.721984+0000","last_undegraded":"2024-04-26T23:51:20.721984+0000","last_fullsized":"2024-04-26T23:51:20.721984+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:23.585513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722256+0000","last_change":"2024-04-26T23:51:18.585993+0000","last_active":"2024-04-26T23:51:20.722256+0000","last_peered":"2024-04-26T23:51:20.722256+0000","last_clean":"2024-04-26T23:51:20.722256+0000","last_became_active":"2024-04-26T23:51:18.585606+0000","last_became_peered":"2024-04-26T23:51:18.585606+0000","last_unstale":"2024-04-26T23:51:20.722256+0000","last_undegraded":"2024-04-26T23:51:20.722256+0000","last_fullsized":"2024-04-26T23:51:20.722256+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:33.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613037+0000","last_change":"2024-04-26T23:51:19.596778+0000","last_active":"2024-04-26T23:51:20.613037+0000","last_peered":"2024-04-26T23:51:20.613037+0000","last_clean":"2024-04-26T23:51:20.613037+0000","last_became_active":"2024-04-26T23:51:19.595899+0000","last_became_peered":"2024-04-26T23:51:19.595899+0000","last_unstale":"2024-04-26T23:51:20.613037+0000","last_undegraded":"2024-04-26T23:51:20.613037+0000","last_fullsized":"2024-04-26T23:51:20.613037+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:14.184589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622766+0000","last_change":"2024-04-26T23:51:18.587355+0000","last_active":"2024-04-26T23:51:20.622766+0000","last_peered":"2024-04-26T23:51:20.622766+0000","last_clean":"2024-04-26T23:51:20.622766+0000","last_became_active":"2024-04-26T23:51:18.585333+0000","last_became_peered":"2024-04-26T23:51:18.585333+0000","last_unstale":"2024-04-26T23:51:20.622766+0000","last_undegraded":"2024-04-26T23:51:20.622766+0000","last_fullsized":"2024-04-26T23:51:20.622766+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:37.800610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622045+0000","last_change":"2024-04-26T23:51:19.616088+0000","last_active":"2024-04-26T23:51:20.622045+0000","last_peered":"2024-04-26T23:51:20.622045+0000","last_clean":"2024-04-26T23:51:20.622045+0000","last_became_active":"2024-04-26T23:51:19.615781+0000","last_became_peered":"2024-04-26T23:51:19.615781+0000","last_unstale":"2024-04-26T23:51:20.622045+0000","last_undegraded":"2024-04-26T23:51:20.622045+0000","last_fullsized":"2024-04-26T23:51:20.622045+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:52.766539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721153+0000","last_change":"2024-04-26T23:51:18.585455+0000","last_active":"2024-04-26T23:51:20.721153+0000","last_peered":"2024-04-26T23:51:20.721153+0000","last_clean":"2024-04-26T23:51:20.721153+0000","last_became_active":"2024-04-26T23:51:18.585121+0000","last_became_peered":"2024-04-26T23:51:18.585121+0000","last_unstale":"2024-04-26T23:51:20.721153+0000","last_undegraded":"2024-04-26T23:51:20.721153+0000","last_fullsized":"2024-04-26T23:51:20.721153+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:26.903420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615573+0000","last_change":"2024-04-26T23:51:19.615898+0000","last_active":"2024-04-26T23:51:20.615573+0000","last_peered":"2024-04-26T23:51:20.615573+0000","last_clean":"2024-04-26T23:51:20.615573+0000","last_became_active":"2024-04-26T23:51:19.615505+0000","last_became_peered":"2024-04-26T23:51:19.615505+0000","last_unstale":"2024-04-26T23:51:20.615573+0000","last_undegraded":"2024-04-26T23:51:20.615573+0000","last_fullsized":"2024-04-26T23:51:20.615573+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:53.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628482+0000","last_change":"2024-04-26T23:51:18.594548+0000","last_active":"2024-04-26T23:51:20.628482+0000","last_peered":"2024-04-26T23:51:20.628482+0000","last_clean":"2024-04-26T23:51:20.628482+0000","last_became_active":"2024-04-26T23:51:18.594228+0000","last_became_peered":"2024-04-26T23:51:18.594228+0000","last_unstale":"2024-04-26T23:51:20.628482+0000","last_undegraded":"2024-04-26T23:51:20.628482+0000","last_fullsized":"2024-04-26T23:51:20.628482+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:51.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.720896+0000","last_change":"2024-04-26T23:51:19.621918+0000","last_active":"2024-04-26T23:51:20.720896+0000","last_peered":"2024-04-26T23:51:20.720896+0000","last_clean":"2024-04-26T23:51:20.720896+0000","last_became_active":"2024-04-26T23:51:19.619943+0000","last_became_peered":"2024-04-26T23:51:19.619943+0000","last_unstale":"2024-04-26T23:51:20.720896+0000","last_undegraded":"2024-04-26T23:51:20.720896+0000","last_fullsized":"2024-04-26T23:51:20.720896+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:54.238756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622809+0000","last_change":"2024-04-26T23:51:18.585011+0000","last_active":"2024-04-26T23:51:20.622809+0000","last_peered":"2024-04-26T23:51:20.622809+0000","last_clean":"2024-04-26T23:51:20.622809+0000","last_became_active":"2024-04-26T23:51:18.584154+0000","last_became_peered":"2024-04-26T23:51:18.584154+0000","last_unstale":"2024-04-26T23:51:20.622809+0000","last_undegraded":"2024-04-26T23:51:20.622809+0000","last_fullsized":"2024-04-26T23:51:20.622809+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:13.492617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722643+0000","last_change":"2024-04-26T23:51:19.628819+0000","last_active":"2024-04-26T23:51:20.722643+0000","last_peered":"2024-04-26T23:51:20.722643+0000","last_clean":"2024-04-26T23:51:20.722643+0000","last_became_active":"2024-04-26T23:51:19.621282+0000","last_became_peered":"2024-04-26T23:51:19.621282+0000","last_unstale":"2024-04-26T23:51:20.722643+0000","last_undegraded":"2024-04-26T23:51:20.722643+0000","last_fullsized":"2024-04-26T23:51:20.722643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:49.439975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"147'3","reported_seq":17,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721020+0000","last_change":"2024-04-26T23:51:18.577260+0000","last_active":"2024-04-26T23:51:20.721020+0000","last_peered":"2024-04-26T23:51:20.721020+0000","last_clean":"2024-04-26T23:51:20.721020+0000","last_became_active":"2024-04-26T23:51:18.576874+0000","last_became_peered":"2024-04-26T23:51:18.576874+0000","last_unstale":"2024-04-26T23:51:20.721020+0000","last_undegraded":"2024-04-26T23:51:20.721020+0000","last_fullsized":"2024-04-26T23:51:20.721020+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:18.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614309+0000","last_change":"2024-04-26T23:51:19.619471+0000","last_active":"2024-04-26T23:51:20.614309+0000","last_peered":"2024-04-26T23:51:20.614309+0000","last_clean":"2024-04-26T23:51:20.614309+0000","last_became_active":"2024-04-26T23:51:19.618332+0000","last_became_peered":"2024-04-26T23:51:19.618332+0000","last_unstale":"2024-04-26T23:51:20.614309+0000","last_undegraded":"2024-04-26T23:51:20.614309+0000","last_fullsized":"2024-04-26T23:51:20.614309+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:09.757437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721663+0000","last_change":"2024-04-26T23:51:18.574375+0000","last_active":"2024-04-26T23:51:20.721663+0000","last_peered":"2024-04-26T23:51:20.721663+0000","last_clean":"2024-04-26T23:51:20.721663+0000","last_became_active":"2024-04-26T23:51:18.573955+0000","last_became_peered":"2024-04-26T23:51:18.573955+0000","last_unstale":"2024-04-26T23:51:20.721663+0000","last_undegraded":"2024-04-26T23:51:20.721663+0000","last_fullsized":"2024-04-26T23:51:20.721663+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:51.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613269+0000","last_change":"2024-04-26T23:51:19.618331+0000","last_active":"2024-04-26T23:51:20.613269+0000","last_peered":"2024-04-26T23:51:20.613269+0000","last_clean":"2024-04-26T23:51:20.613269+0000","last_became_active":"2024-04-26T23:51:19.618034+0000","last_became_peered":"2024-04-26T23:51:19.618034+0000","last_unstale":"2024-04-26T23:51:20.613269+0000","last_undegraded":"2024-04-26T23:51:20.613269+0000","last_fullsized":"2024-04-26T23:51:20.613269+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:35.083448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721486+0000","last_change":"2024-04-26T23:51:18.590494+0000","last_active":"2024-04-26T23:51:20.721486+0000","last_peered":"2024-04-26T23:51:20.721486+0000","last_clean":"2024-04-26T23:51:20.721486+0000","last_became_active":"2024-04-26T23:51:18.590174+0000","last_became_peered":"2024-04-26T23:51:18.590174+0000","last_unstale":"2024-04-26T23:51:20.721486+0000","last_undegraded":"2024-04-26T23:51:20.721486+0000","last_fullsized":"2024-04-26T23:51:20.721486+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:31.683032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721983+0000","last_change":"2024-04-26T23:51:19.609490+0000","last_active":"2024-04-26T23:51:20.721983+0000","last_peered":"2024-04-26T23:51:20.721983+0000","last_clean":"2024-04-26T23:51:20.721983+0000","last_became_active":"2024-04-26T23:51:19.609126+0000","last_became_peered":"2024-04-26T23:51:19.609126+0000","last_unstale":"2024-04-26T23:51:20.721983+0000","last_undegraded":"2024-04-26T23:51:20.721983+0000","last_fullsized":"2024-04-26T23:51:20.721983+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:18.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722267+0000","last_change":"2024-04-26T23:51:18.580666+0000","last_active":"2024-04-26T23:51:20.722267+0000","last_peered":"2024-04-26T23:51:20.722267+0000","last_clean":"2024-04-26T23:51:20.722267+0000","last_became_active":"2024-04-26T23:51:18.580358+0000","last_became_peered":"2024-04-26T23:51:18.580358+0000","last_unstale":"2024-04-26T23:51:20.722267+0000","last_undegraded":"2024-04-26T23:51:20.722267+0000","last_fullsized":"2024-04-26T23:51:20.722267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:02.003196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.612973+0000","last_change":"2024-04-26T23:51:19.625377+0000","last_active":"2024-04-26T23:51:20.612973+0000","last_peered":"2024-04-26T23:51:20.612973+0000","last_clean":"2024-04-26T23:51:20.612973+0000","last_became_active":"2024-04-26T23:51:19.622032+0000","last_became_peered":"2024-04-26T23:51:19.622032+0000","last_unstale":"2024-04-26T23:51:20.612973+0000","last_undegraded":"2024-04-26T23:51:20.612973+0000","last_fullsized":"2024-04-26T23:51:20.612973+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:00.505778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"147'8","reported_seq":22,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.621447+0000","last_change":"2024-04-26T23:51:18.594857+0000","last_active":"2024-04-26T23:51:20.621447+0000","last_peered":"2024-04-26T23:51:20.621447+0000","last_clean":"2024-04-26T23:51:20.621447+0000","last_became_active":"2024-04-26T23:51:18.594487+0000","last_became_peered":"2024-04-26T23:51:18.594487+0000","last_unstale":"2024-04-26T23:51:20.621447+0000","last_undegraded":"2024-04-26T23:51:20.621447+0000","last_fullsized":"2024-04-26T23:51:20.621447+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:52.882140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628013+0000","last_change":"2024-04-26T23:51:19.622190+0000","last_active":"2024-04-26T23:51:20.628013+0000","last_peered":"2024-04-26T23:51:20.628013+0000","last_clean":"2024-04-26T23:51:20.628013+0000","last_became_active":"2024-04-26T23:51:19.621888+0000","last_became_peered":"2024-04-26T23:51:19.621888+0000","last_unstale":"2024-04-26T23:51:20.628013+0000","last_undegraded":"2024-04-26T23:51:20.628013+0000","last_fullsized":"2024-04-26T23:51:20.628013+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:34.252626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721911+0000","last_change":"2024-04-26T23:51:18.591404+0000","last_active":"2024-04-26T23:51:20.721911+0000","last_peered":"2024-04-26T23:51:20.721911+0000","last_clean":"2024-04-26T23:51:20.721911+0000","last_became_active":"2024-04-26T23:51:18.591089+0000","last_became_peered":"2024-04-26T23:51:18.591089+0000","last_unstale":"2024-04-26T23:51:20.721911+0000","last_undegraded":"2024-04-26T23:51:20.721911+0000","last_fullsized":"2024-04-26T23:51:20.721911+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:42.157357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628531+0000","last_change":"2024-04-26T23:51:19.615983+0000","last_active":"2024-04-26T23:51:20.628531+0000","last_peered":"2024-04-26T23:51:20.628531+0000","last_clean":"2024-04-26T23:51:20.628531+0000","last_became_active":"2024-04-26T23:51:19.614792+0000","last_became_peered":"2024-04-26T23:51:19.614792+0000","last_unstale":"2024-04-26T23:51:20.628531+0000","last_undegraded":"2024-04-26T23:51:20.628531+0000","last_fullsized":"2024-04-26T23:51:20.628531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:44.171307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.628133+0000","last_change":"2024-04-26T23:51:18.594371+0000","last_active":"2024-04-26T23:51:20.628133+0000","last_peered":"2024-04-26T23:51:20.628133+0000","last_clean":"2024-04-26T23:51:20.628133+0000","last_became_active":"2024-04-26T23:51:18.593893+0000","last_became_peered":"2024-04-26T23:51:18.593893+0000","last_unstale":"2024-04-26T23:51:20.628133+0000","last_undegraded":"2024-04-26T23:51:20.628133+0000","last_fullsized":"2024-04-26T23:51:20.628133+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.167982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622527+0000","last_change":"2024-04-26T23:51:19.616480+0000","last_active":"2024-04-26T23:51:20.622527+0000","last_peered":"2024-04-26T23:51:20.622527+0000","last_clean":"2024-04-26T23:51:20.622527+0000","last_became_active":"2024-04-26T23:51:19.616165+0000","last_became_peered":"2024-04-26T23:51:19.616165+0000","last_unstale":"2024-04-26T23:51:20.622527+0000","last_undegraded":"2024-04-26T23:51:20.622527+0000","last_fullsized":"2024-04-26T23:51:20.622527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:45.854352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613502+0000","last_change":"2024-04-26T23:51:18.581691+0000","last_active":"2024-04-26T23:51:20.613502+0000","last_peered":"2024-04-26T23:51:20.613502+0000","last_clean":"2024-04-26T23:51:20.613502+0000","last_became_active":"2024-04-26T23:51:18.581381+0000","last_became_peered":"2024-04-26T23:51:18.581381+0000","last_unstale":"2024-04-26T23:51:20.613502+0000","last_undegraded":"2024-04-26T23:51:20.613502+0000","last_fullsized":"2024-04-26T23:51:20.613502+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:08.740895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622771+0000","last_change":"2024-04-26T23:51:19.630625+0000","last_active":"2024-04-26T23:51:20.622771+0000","last_peered":"2024-04-26T23:51:20.622771+0000","last_clean":"2024-04-26T23:51:20.622771+0000","last_became_active":"2024-04-26T23:51:19.630293+0000","last_became_peered":"2024-04-26T23:51:19.630293+0000","last_unstale":"2024-04-26T23:51:20.622771+0000","last_undegraded":"2024-04-26T23:51:20.622771+0000","last_fullsized":"2024-04-26T23:51:20.622771+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.502770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722427+0000","last_change":"2024-04-26T23:51:18.590044+0000","last_active":"2024-04-26T23:51:20.722427+0000","last_peered":"2024-04-26T23:51:20.722427+0000","last_clean":"2024-04-26T23:51:20.722427+0000","last_became_active":"2024-04-26T23:51:18.589461+0000","last_became_peered":"2024-04-26T23:51:18.589461+0000","last_unstale":"2024-04-26T23:51:20.722427+0000","last_undegraded":"2024-04-26T23:51:20.722427+0000","last_fullsized":"2024-04-26T23:51:20.722427+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:41.172650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627743+0000","last_change":"2024-04-26T23:51:19.610229+0000","last_active":"2024-04-26T23:51:20.627743+0000","last_peered":"2024-04-26T23:51:20.627743+0000","last_clean":"2024-04-26T23:51:20.627743+0000","last_became_active":"2024-04-26T23:51:19.609852+0000","last_became_peered":"2024-04-26T23:51:19.609852+0000","last_unstale":"2024-04-26T23:51:20.627743+0000","last_undegraded":"2024-04-26T23:51:20.627743+0000","last_fullsized":"2024-04-26T23:51:20.627743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:37.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610815+0000","last_change":"2024-04-26T23:51:18.579763+0000","last_active":"2024-04-26T23:51:20.610815+0000","last_peered":"2024-04-26T23:51:20.610815+0000","last_clean":"2024-04-26T23:51:20.610815+0000","last_became_active":"2024-04-26T23:51:18.579468+0000","last_became_peered":"2024-04-26T23:51:18.579468+0000","last_unstale":"2024-04-26T23:51:20.610815+0000","last_undegraded":"2024-04-26T23:51:20.610815+0000","last_fullsized":"2024-04-26T23:51:20.610815+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:33.236084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613474+0000","last_change":"2024-04-26T23:51:19.608294+0000","last_active":"2024-04-26T23:51:20.613474+0000","last_peered":"2024-04-26T23:51:20.613474+0000","last_clean":"2024-04-26T23:51:20.613474+0000","last_became_active":"2024-04-26T23:51:19.607994+0000","last_became_peered":"2024-04-26T23:51:19.607994+0000","last_unstale":"2024-04-26T23:51:20.613474+0000","last_undegraded":"2024-04-26T23:51:20.613474+0000","last_fullsized":"2024-04-26T23:51:20.613474+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:40.898557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614533+0000","last_change":"2024-04-26T23:51:18.581588+0000","last_active":"2024-04-26T23:51:20.614533+0000","last_peered":"2024-04-26T23:51:20.614533+0000","last_clean":"2024-04-26T23:51:20.614533+0000","last_became_active":"2024-04-26T23:51:18.580547+0000","last_became_peered":"2024-04-26T23:51:18.580547+0000","last_unstale":"2024-04-26T23:51:20.614533+0000","last_undegraded":"2024-04-26T23:51:20.614533+0000","last_fullsized":"2024-04-26T23:51:20.614533+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:46.306757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627492+0000","last_change":"2024-04-26T23:51:19.609150+0000","last_active":"2024-04-26T23:51:20.627492+0000","last_peered":"2024-04-26T23:51:20.627492+0000","last_clean":"2024-04-26T23:51:20.627492+0000","last_became_active":"2024-04-26T23:51:19.608807+0000","last_became_peered":"2024-04-26T23:51:19.608807+0000","last_unstale":"2024-04-26T23:51:20.627492+0000","last_undegraded":"2024-04-26T23:51:20.627492+0000","last_fullsized":"2024-04-26T23:51:20.627492+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:19.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610594+0000","last_change":"2024-04-26T23:51:18.589413+0000","last_active":"2024-04-26T23:51:20.610594+0000","last_peered":"2024-04-26T23:51:20.610594+0000","last_clean":"2024-04-26T23:51:20.610594+0000","last_became_active":"2024-04-26T23:51:18.589048+0000","last_became_peered":"2024-04-26T23:51:18.589048+0000","last_unstale":"2024-04-26T23:51:20.610594+0000","last_undegraded":"2024-04-26T23:51:20.610594+0000","last_fullsized":"2024-04-26T23:51:20.610594+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:43.936529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613110+0000","last_change":"2024-04-26T23:51:19.627172+0000","last_active":"2024-04-26T23:51:20.613110+0000","last_peered":"2024-04-26T23:51:20.613110+0000","last_clean":"2024-04-26T23:51:20.613110+0000","last_became_active":"2024-04-26T23:51:19.626847+0000","last_became_peered":"2024-04-26T23:51:19.626847+0000","last_unstale":"2024-04-26T23:51:20.613110+0000","last_undegraded":"2024-04-26T23:51:20.613110+0000","last_fullsized":"2024-04-26T23:51:20.613110+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:58.878022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614557+0000","last_change":"2024-04-26T23:51:18.582922+0000","last_active":"2024-04-26T23:51:20.614557+0000","last_peered":"2024-04-26T23:51:20.614557+0000","last_clean":"2024-04-26T23:51:20.614557+0000","last_became_active":"2024-04-26T23:51:18.582639+0000","last_became_peered":"2024-04-26T23:51:18.582639+0000","last_unstale":"2024-04-26T23:51:20.614557+0000","last_undegraded":"2024-04-26T23:51:20.614557+0000","last_fullsized":"2024-04-26T23:51:20.614557+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:44.944411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721600+0000","last_change":"2024-04-26T23:51:19.610136+0000","last_active":"2024-04-26T23:51:20.721600+0000","last_peered":"2024-04-26T23:51:20.721600+0000","last_clean":"2024-04-26T23:51:20.721600+0000","last_became_active":"2024-04-26T23:51:19.609482+0000","last_became_peered":"2024-04-26T23:51:19.609482+0000","last_unstale":"2024-04-26T23:51:20.721600+0000","last_undegraded":"2024-04-26T23:51:20.721600+0000","last_fullsized":"2024-04-26T23:51:20.721600+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:04.441710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722184+0000","last_change":"2024-04-26T23:51:18.591822+0000","last_active":"2024-04-26T23:51:20.722184+0000","last_peered":"2024-04-26T23:51:20.722184+0000","last_clean":"2024-04-26T23:51:20.722184+0000","last_became_active":"2024-04-26T23:51:18.589834+0000","last_became_peered":"2024-04-26T23:51:18.589834+0000","last_unstale":"2024-04-26T23:51:20.722184+0000","last_undegraded":"2024-04-26T23:51:20.722184+0000","last_fullsized":"2024-04-26T23:51:20.722184+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:10.618020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721988+0000","last_change":"2024-04-26T23:51:19.636358+0000","last_active":"2024-04-26T23:51:20.721988+0000","last_peered":"2024-04-26T23:51:20.721988+0000","last_clean":"2024-04-26T23:51:20.721988+0000","last_became_active":"2024-04-26T23:51:19.635750+0000","last_became_peered":"2024-04-26T23:51:19.635750+0000","last_unstale":"2024-04-26T23:51:20.721988+0000","last_undegraded":"2024-04-26T23:51:20.721988+0000","last_fullsized":"2024-04-26T23:51:20.721988+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:33.085937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"148'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.626426+0000","last_change":"2024-04-26T23:51:18.596814+0000","last_active":"2024-04-26T23:51:20.626426+0000","last_peered":"2024-04-26T23:51:20.626426+0000","last_clean":"2024-04-26T23:51:20.626426+0000","last_became_active":"2024-04-26T23:51:18.596494+0000","last_became_peered":"2024-04-26T23:51:18.596494+0000","last_unstale":"2024-04-26T23:51:20.626426+0000","last_undegraded":"2024-04-26T23:51:20.626426+0000","last_fullsized":"2024-04-26T23:51:20.626426+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:22.592932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622793+0000","last_change":"2024-04-26T23:51:19.611244+0000","last_active":"2024-04-26T23:51:20.622793+0000","last_peered":"2024-04-26T23:51:20.622793+0000","last_clean":"2024-04-26T23:51:20.622793+0000","last_became_active":"2024-04-26T23:51:19.610920+0000","last_became_peered":"2024-04-26T23:51:19.610920+0000","last_unstale":"2024-04-26T23:51:20.622793+0000","last_undegraded":"2024-04-26T23:51:20.622793+0000","last_fullsized":"2024-04-26T23:51:20.622793+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:22.096559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.613621+0000","last_change":"2024-04-26T23:51:18.584895+0000","last_active":"2024-04-26T23:51:20.613621+0000","last_peered":"2024-04-26T23:51:20.613621+0000","last_clean":"2024-04-26T23:51:20.613621+0000","last_became_active":"2024-04-26T23:51:18.584540+0000","last_became_peered":"2024-04-26T23:51:18.584540+0000","last_unstale":"2024-04-26T23:51:20.613621+0000","last_undegraded":"2024-04-26T23:51:20.613621+0000","last_fullsized":"2024-04-26T23:51:20.613621+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:33.073901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721991+0000","last_change":"2024-04-26T23:51:19.629360+0000","last_active":"2024-04-26T23:51:20.721991+0000","last_peered":"2024-04-26T23:51:20.721991+0000","last_clean":"2024-04-26T23:51:20.721991+0000","last_became_active":"2024-04-26T23:51:19.625375+0000","last_became_peered":"2024-04-26T23:51:19.625375+0000","last_unstale":"2024-04-26T23:51:20.721991+0000","last_undegraded":"2024-04-26T23:51:20.721991+0000","last_fullsized":"2024-04-26T23:51:20.721991+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:27.259776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721623+0000","last_change":"2024-04-26T23:51:18.590151+0000","last_active":"2024-04-26T23:51:20.721623+0000","last_peered":"2024-04-26T23:51:20.721623+0000","last_clean":"2024-04-26T23:51:20.721623+0000","last_became_active":"2024-04-26T23:51:18.589720+0000","last_became_peered":"2024-04-26T23:51:18.589720+0000","last_unstale":"2024-04-26T23:51:20.721623+0000","last_undegraded":"2024-04-26T23:51:20.721623+0000","last_fullsized":"2024-04-26T23:51:20.721623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.882015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614565+0000","last_change":"2024-04-26T23:51:19.609938+0000","last_active":"2024-04-26T23:51:20.614565+0000","last_peered":"2024-04-26T23:51:20.614565+0000","last_clean":"2024-04-26T23:51:20.614565+0000","last_became_active":"2024-04-26T23:51:19.609630+0000","last_became_peered":"2024-04-26T23:51:19.609630+0000","last_unstale":"2024-04-26T23:51:20.614565+0000","last_undegraded":"2024-04-26T23:51:20.614565+0000","last_fullsized":"2024-04-26T23:51:20.614565+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:34.090443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622327+0000","last_change":"2024-04-26T23:51:18.591512+0000","last_active":"2024-04-26T23:51:20.622327+0000","last_peered":"2024-04-26T23:51:20.622327+0000","last_clean":"2024-04-26T23:51:20.622327+0000","last_became_active":"2024-04-26T23:51:18.591278+0000","last_became_peered":"2024-04-26T23:51:18.591278+0000","last_unstale":"2024-04-26T23:51:20.622327+0000","last_undegraded":"2024-04-26T23:51:20.622327+0000","last_fullsized":"2024-04-26T23:51:20.622327+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:48.587919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.722038+0000","last_change":"2024-04-26T23:51:19.632032+0000","last_active":"2024-04-26T23:51:20.722038+0000","last_peered":"2024-04-26T23:51:20.722038+0000","last_clean":"2024-04-26T23:51:20.722038+0000","last_became_active":"2024-04-26T23:51:19.631773+0000","last_became_peered":"2024-04-26T23:51:19.631773+0000","last_unstale":"2024-04-26T23:51:20.722038+0000","last_undegraded":"2024-04-26T23:51:20.722038+0000","last_fullsized":"2024-04-26T23:51:20.722038+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:25.077822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.721197+0000","last_change":"2024-04-26T23:51:18.586882+0000","last_active":"2024-04-26T23:51:20.721197+0000","last_peered":"2024-04-26T23:51:20.721197+0000","last_clean":"2024-04-26T23:51:20.721197+0000","last_became_active":"2024-04-26T23:51:18.586540+0000","last_became_peered":"2024-04-26T23:51:18.586540+0000","last_unstale":"2024-04-26T23:51:20.721197+0000","last_undegraded":"2024-04-26T23:51:20.721197+0000","last_fullsized":"2024-04-26T23:51:20.721197+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:07.791463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.612961+0000","last_change":"2024-04-26T23:51:19.607712+0000","last_active":"2024-04-26T23:51:20.612961+0000","last_peered":"2024-04-26T23:51:20.612961+0000","last_clean":"2024-04-26T23:51:20.612961+0000","last_became_active":"2024-04-26T23:51:19.607373+0000","last_became_peered":"2024-04-26T23:51:19.607373+0000","last_unstale":"2024-04-26T23:51:20.612961+0000","last_undegraded":"2024-04-26T23:51:20.612961+0000","last_fullsized":"2024-04-26T23:51:20.612961+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:45.402405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.622192+0000","last_change":"2024-04-26T23:51:18.587659+0000","last_active":"2024-04-26T23:51:20.622192+0000","last_peered":"2024-04-26T23:51:20.622192+0000","last_clean":"2024-04-26T23:51:20.622192+0000","last_became_active":"2024-04-26T23:51:18.586970+0000","last_became_peered":"2024-04-26T23:51:18.586970+0000","last_unstale":"2024-04-26T23:51:20.622192+0000","last_undegraded":"2024-04-26T23:51:20.622192+0000","last_fullsized":"2024-04-26T23:51:20.622192+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:24.090544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.614873+0000","last_change":"2024-04-26T23:51:19.628716+0000","last_active":"2024-04-26T23:51:20.614873+0000","last_peered":"2024-04-26T23:51:20.614873+0000","last_clean":"2024-04-26T23:51:20.614873+0000","last_became_active":"2024-04-26T23:51:19.628376+0000","last_became_peered":"2024-04-26T23:51:19.628376+0000","last_unstale":"2024-04-26T23:51:20.614873+0000","last_undegraded":"2024-04-26T23:51:20.614873+0000","last_fullsized":"2024-04-26T23:51:20.614873+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:23.585513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627478+0000","last_change":"2024-04-26T23:51:18.590456+0000","last_active":"2024-04-26T23:51:20.627478+0000","last_peered":"2024-04-26T23:51:20.627478+0000","last_clean":"2024-04-26T23:51:20.627478+0000","last_became_active":"2024-04-26T23:51:18.590107+0000","last_became_peered":"2024-04-26T23:51:18.590107+0000","last_unstale":"2024-04-26T23:51:20.627478+0000","last_undegraded":"2024-04-26T23:51:20.627478+0000","last_fullsized":"2024-04-26T23:51:20.627478+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.882015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.615479+0000","last_change":"2024-04-26T23:51:19.610639+0000","last_active":"2024-04-26T23:51:20.615479+0000","last_peered":"2024-04-26T23:51:20.615479+0000","last_clean":"2024-04-26T23:51:20.615479+0000","last_became_active":"2024-04-26T23:51:19.608202+0000","last_became_peered":"2024-04-26T23:51:19.608202+0000","last_unstale":"2024-04-26T23:51:20.615479+0000","last_undegraded":"2024-04-26T23:51:20.615479+0000","last_fullsized":"2024-04-26T23:51:20.615479+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:18.546749+0000","last_clean_scrub_stamp":"2024-04-26T23:51:18.546749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:40.010712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"147'6","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.610675+0000","last_change":"2024-04-26T23:51:18.589384+0000","last_active":"2024-04-26T23:51:20.610675+0000","last_peered":"2024-04-26T23:51:20.610675+0000","last_clean":"2024-04-26T23:51:20.610675+0000","last_became_active":"2024-04-26T23:51:18.589069+0000","last_became_peered":"2024-04-26T23:51:18.589069+0000","last_unstale":"2024-04-26T23:51:20.610675+0000","last_undegraded":"2024-04-26T23:51:20.610675+0000","last_fullsized":"2024-04-26T23:51:20.610675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:17.530207+0000","last_clean_scrub_stamp":"2024-04-26T23:51:17.530207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:43.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T23:51:20.627655+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:51:20.627655+0000","last_peered":"2024-04-26T23:51:20.627655+0000","last_clean":"2024-04-26T23:51:20.627655+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:51:20.627655+0000","last_undegraded":"2024-04-26T23:51:20.627655+0000","last_fullsized":"2024-04-26T23:51:20.627655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47472,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324368,"statfs":{"total":96636764160,"available":96588152832,"internally_reserved":0,"allocated":2453504,"data_stored":1463608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":1847296,"data_stored":866596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":1843200,"data_stored":866526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55208,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1855488,"data_stored":868138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55216,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316624,"statfs":{"total":96636764160,"available":96580222976,"internally_reserved":0,"allocated":1863680,"data_stored":873834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59364,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312476,"statfs":{"total":96636764160,"available":96575975424,"internally_reserved":0,"allocated":1851392,"data_stored":866638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55204,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1859584,"data_stored":873288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55796,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316044,"statfs":{"total":96636764160,"available":96579629056,"internally_reserved":0,"allocated":2461696,"data_stored":1459212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:51:24.616 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:51:24.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:51:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.841+0000 7f41d5400640 1 Processor -- start 2024-04-26T23:51:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.842+0000 7f41d5400640 1 -- start start 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.843+0000 7f41d5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41d015f1f0 0x7f41d006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.843+0000 7f41d5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d0162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.843+0000 7f41d5400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d006f200 con 0x7f41d0163270 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.843+0000 7f41d5400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f41d0163d00 con 0x7f41d015f1f0 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.843+0000 7f41d5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41d006ccf0 con 0x7f41d006c890 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41d015f1f0 0x7f41d006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41d015f1f0 0x7f41d006c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33694/0 (socket says 172.21.15.46:33694) 2024-04-26T23:51:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 -- 172.21.15.46:0/567484357 learned_addr learned my addr 172.21.15.46:0/567484357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41cde00640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d0162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 -- 172.21.15.46:0/567484357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 msgr2=0x7f41d0162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d0162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 -- 172.21.15.46:0/567484357 >> v1:172.21.15.142:6789/0 conn(0x7f41d0163270 legacy=0x7f41d00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.844+0000 7f41ce800640 1 -- 172.21.15.46:0/567484357 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d0163a20 con 0x7f41d015f1f0 2024-04-26T23:51:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.846+0000 7f41ce800640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d015f1f0 0x7f41d006c350 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a8d78b731b09b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.846+0000 7f41cd400640 1 -- 172.21.15.46:0/567484357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037070 con 0x7f41d015f1f0 2024-04-26T23:51:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.846+0000 7f41cd400640 1 -- 172.21.15.46:0/567484357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0032440 con 0x7f41d015f1f0 2024-04-26T23:51:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.846+0000 7f41cd400640 1 -- 172.21.15.46:0/567484357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0032810 con 0x7f41d015f1f0 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 -- 172.21.15.46:0/567484357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d015f1f0 msgr2=0x7f41d006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d015f1f0 0x7f41d006c350 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a210 comp rx=0 tx=0).stop 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 -- 172.21.15.46:0/567484357 shutdown_connections 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d0162d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 --2- 172.21.15.46:0/567484357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d015f1f0 0x7f41d006c350 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 -- 172.21.15.46:0/567484357 >> 172.21.15.46:0/567484357 conn(0x7f41d006da90 msgr2=0x7f41d00d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 -- 172.21.15.46:0/567484357 shutdown_connections 2024-04-26T23:51:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.847+0000 7f41d5400640 1 -- 172.21.15.46:0/567484357 wait complete. 2024-04-26T23:51:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.849+0000 7f41d5400640 1 Processor -- start 2024-04-26T23:51:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.849+0000 7f41d5400640 1 -- start start 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d015f1f0 0x7f41d00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41d015f5d0 con 0x7f41d0163270 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006f810 con 0x7f41d015f1f0 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41d5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41d0163700 con 0x7f41d006c890 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33696/0 (socket says 172.21.15.46:33696) 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.850+0000 7f41cf200640 1 -- 172.21.15.46:0/3449589845 learned_addr learned my addr 172.21.15.46:0/3449589845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cde00640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d015f1f0 0x7f41d00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41ce800640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 -- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 msgr2=0x7f41d015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 -- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d015f1f0 msgr2=0x7f41d00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d015f1f0 0x7f41d00d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 -- 172.21.15.46:0/3449589845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d015a390 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41cf200640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c410 tx=0x7f41c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011790 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c8017070 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.851+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011bb0 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.852+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d00c8300 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.852+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d00c85a0 con 0x7f41d0163270 2024-04-26T23:51:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.853+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d0069530 con 0x7f41d0163270 2024-04-26T23:51:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.854+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41c801c460 con 0x7f41d0163270 2024-04-26T23:51:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.855+0000 7f41c6a00640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.855+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f41c808b8c0 con 0x7f41d0163270 2024-04-26T23:51:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.856+0000 7f41ce800640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.857+0000 7f41ce800640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 0x7f41b006edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:24.859+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41c8021020 con 0x7f41d0163270 2024-04-26T23:51:25.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.144+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f41d00c80e0 con 0x7f41d0163270 2024-04-26T23:51:25.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.147+0000 7f41c6a00640 1 -- 172.21.15.46:0/3449589845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f41c8059b50 con 0x7f41d0163270 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:51:25.152 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:51:25.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 msgr2=0x7f41b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 0x7f41b006edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).stop 2024-04-26T23:51:25.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 msgr2=0x7f41d014fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c410 tx=0x7f41c800c8d0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 shutdown_connections 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d0163270 0x7f41d014fc00 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d015f1f0 0x7f41d00d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 --2- 172.21.15.46:0/3449589845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d006c890 0x7f41d015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 >> 172.21.15.46:0/3449589845 conn(0x7f41d006da90 msgr2=0x7f41d00b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.151+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 shutdown_connections 2024-04-26T23:51:25.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.152+0000 7f41d5400640 1 -- 172.21.15.46:0/3449589845 wait complete. 2024-04-26T23:51:25.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.367+0000 7febef000640 1 Processor -- start 2024-04-26T23:51:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.367+0000 7febef000640 1 -- start start 2024-04-26T23:51:25.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febef000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febe8160910 0x7febe8160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:25.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febef000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe8073ad0 con 0x7febe81466b0 2024-04-26T23:51:25.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febef000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe8146a90 con 0x7febe80720e0 2024-04-26T23:51:25.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febef000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7febe81612a0 con 0x7febe8160910 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febee600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febe8160910 0x7febe8160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febedc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7febe81466b0 0x7febe806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60220/0 (socket says 172.21.15.46:60220) 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.369+0000 7febedc00640 1 -- 172.21.15.46:0/2297523136 learned_addr learned my addr 172.21.15.46:0/2297523136 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.370+0000 7febee600640 1 -- 172.21.15.46:0/2297523136 >> v1:172.21.15.142:6789/0 conn(0x7febe80720e0 legacy=0x7febe80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.370+0000 7febee600640 1 -- 172.21.15.46:0/2297523136 >> v1:172.21.15.46:6789/0 conn(0x7febe81466b0 legacy=0x7febe806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.370+0000 7febee600640 1 -- 172.21.15.46:0/2297523136 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe8073d70 con 0x7febe8160910 2024-04-26T23:51:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.371+0000 7febee600640 1 --2- 172.21.15.46:0/2297523136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe8160910 0x7febe8160d60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7febe0009dc0 tx=0x7febe002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e0fa23f30c2648 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.371+0000 7febe7e00640 1 -- 172.21.15.46:0/2297523136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0007740 con 0x7febe8160910 2024-04-26T23:51:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.371+0000 7febe7e00640 1 -- 172.21.15.46:0/2297523136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe00078e0 con 0x7febe8160910 2024-04-26T23:51:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.371+0000 7febe7e00640 1 -- 172.21.15.46:0/2297523136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0007cb0 con 0x7febe8160910 2024-04-26T23:51:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 -- 172.21.15.46:0/2297523136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe8160910 msgr2=0x7febe8160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 --2- 172.21.15.46:0/2297523136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe8160910 0x7febe8160d60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7febe0009dc0 tx=0x7febe002a6d0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 -- 172.21.15.46:0/2297523136 shutdown_connections 2024-04-26T23:51:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 --2- 172.21.15.46:0/2297523136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe8160910 0x7febe8160d60 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 -- 172.21.15.46:0/2297523136 >> 172.21.15.46:0/2297523136 conn(0x7febe80b26d0 msgr2=0x7febe80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 -- 172.21.15.46:0/2297523136 shutdown_connections 2024-04-26T23:51:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.372+0000 7febef000640 1 -- 172.21.15.46:0/2297523136 wait complete. 2024-04-26T23:51:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.374+0000 7febef000640 1 Processor -- start 2024-04-26T23:51:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.374+0000 7febef000640 1 -- start start 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febe80720e0 0x7febe813bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe8160910 0x7febe8139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febe80cb470 con 0x7febe81466b0 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7febe80cb190 con 0x7febe8160910 2024-04-26T23:51:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febef000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7febe80cb990 con 0x7febe80720e0 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33718/0 (socket says 172.21.15.46:33718) 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.375+0000 7febed200640 1 -- 172.21.15.46:0/297610943 learned_addr learned my addr 172.21.15.46:0/297610943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febee600640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe8160910 0x7febe8139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febedc00640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febe80720e0 0x7febe813bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 -- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febe80720e0 msgr2=0x7febe813bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febe80720e0 0x7febe813bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 -- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe8160910 msgr2=0x7febe8139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe8160910 0x7febe8139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 -- 172.21.15.46:0/297610943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe80cfbf0 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.376+0000 7febed200640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7febd400c460 tx=0x7febd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.377+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4016020 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.377+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd4012870 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.377+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4012c60 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.377+0000 7febef000640 1 -- 172.21.15.46:0/297610943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe80b4300 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.377+0000 7febef000640 1 -- 172.21.15.46:0/297610943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe80ca940 con 0x7febe81466b0 2024-04-26T23:51:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.378+0000 7febef000640 1 -- 172.21.15.46:0/297610943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe80693b0 con 0x7febe81466b0 2024-04-26T23:51:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.379+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febd4019020 con 0x7febe81466b0 2024-04-26T23:51:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.380+0000 7febe5600640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 0x7febc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.381+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7febd408be30 con 0x7febe81466b0 2024-04-26T23:51:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.381+0000 7febedc00640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 0x7febc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.382+0000 7febedc00640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 0x7febc806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7febdc000f60 tx=0x7febdc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.385+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febd4020020 con 0x7febe81466b0 2024-04-26T23:51:25.534 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:51:25.530+0000 7f7076a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:51:25.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.629+0000 7febef000640 1 -- 172.21.15.46:0/297610943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febe80c7180 con 0x7febe81466b0 2024-04-26T23:51:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.635+0000 7febe5600640 1 -- 172.21.15.46:0/297610943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7febd40182c0 con 0x7febe81466b0 2024-04-26T23:51:25.640 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:51:25.640 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:51:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 -- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 msgr2=0x7febc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 0x7febc806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7febdc000f60 tx=0x7febdc009040 comp rx=0 tx=0).stop 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 -- 172.21.15.46:0/297610943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 msgr2=0x7febe815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7febd400c460 tx=0x7febd400c920 comp rx=0 tx=0).stop 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 -- 172.21.15.46:0/297610943 shutdown_connections 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7febc806c8f0 0x7febc806edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe8160910 0x7febe8139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febe81466b0 0x7febe815fe40 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 --2- 172.21.15.46:0/297610943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febe80720e0 0x7febe813bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.638+0000 7febef000640 1 -- 172.21.15.46:0/297610943 >> 172.21.15.46:0/297610943 conn(0x7febe80b26d0 msgr2=0x7febe80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.639+0000 7febef000640 1 -- 172.21.15.46:0/297610943 shutdown_connections 2024-04-26T23:51:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:25.639+0000 7febef000640 1 -- 172.21.15.46:0/297610943 wait complete. 2024-04-26T23:51:26.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:51:19.047682+0000', 'modified': '2024-04-26T23:51:22.134191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9663}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9663': {'gid': 9663, 'name': 'c', 'rank': 0, 'incarnation': 114, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/3522383810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 3522383810}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3522383810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9663, 'qdb_cluster': [9663]} 2024-04-26T23:51:26.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:51:26.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:51:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.285+0000 7f9321c00640 1 Processor -- start 2024-04-26T23:51:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.286+0000 7f9321c00640 1 -- start start 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.287+0000 7f9321c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f931c14c2e0 0x7f931c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f9321c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f9321c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f931c14c6c0 con 0x7f931c0cb0f0 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f9321c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f931c0cb8b0 con 0x7f931c14c2e0 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f9321c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f931c167e10 con 0x7f931c0cdbc0 2024-04-26T23:51:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f9321200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.288+0000 7f931be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f931c14c2e0 0x7f931c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f9321200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:53924/0 (socket says 172.21.15.46:53924) 2024-04-26T23:51:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f9321200640 1 -- 172.21.15.46:0/2319508104 learned_addr learned my addr 172.21.15.46:0/2319508104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f931be00640 1 -- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 msgr2=0x7f931c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f931be00640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f931be00640 1 -- 172.21.15.46:0/2319508104 >> v1:172.21.15.142:6789/0 conn(0x7f931c0cb0f0 legacy=0x7f931c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f931be00640 1 -- 172.21.15.46:0/2319508104 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c167050 con 0x7f931c14c2e0 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.290+0000 7f9321200640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.289+0000 7f931aa00640 1 -- 172.21.15.46:0/2319508104 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096127104 0 0) 0x7f931c14c6c0 con 0x7f931c0cb0f0 2024-04-26T23:51:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.291+0000 7f931be00640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c14c2e0 0x7f931c0ccdd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f930c009d90 tx=0x7f930c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a61c7be697bfe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.291+0000 7f931aa00640 1 -- 172.21.15.46:0/2319508104 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c031600 con 0x7f931c14c2e0 2024-04-26T23:51:26.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f931aa00640 1 -- 172.21.15.46:0/2319508104 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c0317a0 con 0x7f931c14c2e0 2024-04-26T23:51:26.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f931aa00640 1 -- 172.21.15.46:0/2319508104 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c031b70 con 0x7f931c14c2e0 2024-04-26T23:51:26.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 -- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c14c2e0 msgr2=0x7f931c0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c14c2e0 0x7f931c0ccdd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f930c009d90 tx=0x7f930c02a5d0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 -- 172.21.15.46:0/2319508104 shutdown_connections 2024-04-26T23:51:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 --2- 172.21.15.46:0/2319508104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c14c2e0 0x7f931c0ccdd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 -- 172.21.15.46:0/2319508104 >> 172.21.15.46:0/2319508104 conn(0x7f931c0b26d0 msgr2=0x7f931c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 -- 172.21.15.46:0/2319508104 shutdown_connections 2024-04-26T23:51:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.292+0000 7f9321c00640 1 -- 172.21.15.46:0/2319508104 wait complete. 2024-04-26T23:51:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.294+0000 7f9321c00640 1 Processor -- start 2024-04-26T23:51:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.294+0000 7f9321c00640 1 -- start start 2024-04-26T23:51:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 0x7f931c147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1678f0 con 0x7f931c0b43f0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f931c167610 con 0x7f931c0cb0f0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.295+0000 7f9321c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f931c0cb580 con 0x7f931c0cdbc0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33734/0 (socket says 172.21.15.46:33734) 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 -- 172.21.15.46:0/268256751 learned_addr learned my addr 172.21.15.46:0/268256751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f931be00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 0x7f931c147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f931b400640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 -- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 msgr2=0x7f931c0b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 -- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 msgr2=0x7f931c147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 0x7f931c147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f9321200640 1 -- 172.21.15.46:0/268256751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c0b6070 con 0x7f931c0b43f0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f931be00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 0x7f931c147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.296+0000 7f931b400640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9321200640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f931400e850 tx=0x7f931400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9313e00640 1 -- 172.21.15.46:0/268256751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314017020 con 0x7f931c0b43f0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9313e00640 1 -- 172.21.15.46:0/268256751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9314004240 con 0x7f931c0b43f0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c0b6310 con 0x7f931c0b43f0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9313e00640 1 -- 172.21.15.46:0/268256751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931400c730 con 0x7f931c0b43f0 2024-04-26T23:51:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.297+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c0b4930 con 0x7f931c0b43f0 2024-04-26T23:51:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.300+0000 7f9313e00640 1 -- 172.21.15.46:0/268256751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931401a020 con 0x7f931c0b43f0 2024-04-26T23:51:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.301+0000 7f9326061740 1 Processor -- start 2024-04-26T23:51:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.301+0000 7f9313e00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 0x7f92f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.301+0000 7f9326061740 1 -- start start 2024-04-26T23:51:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.301+0000 7f9313e00640 1 -- 172.21.15.46:0/268256751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f931408bce0 con 0x7f931c0b43f0 2024-04-26T23:51:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f931b400640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 0x7f92f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cd876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ccc0600 con 0x55ce1cd98e80 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce1cc5d100 con 0x55ce1cd87270 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.302+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ce1ccbc5c0 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33742/0 (socket says 172.21.15.46:33742) 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 -- 172.21.15.46:0/2305158838 learned_addr learned my addr 172.21.15.46:0/2305158838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931be00640 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f9321200640 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cd876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 -- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 msgr2=0x55ce1cd5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 -- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 msgr2=0x55ce1cd876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cd876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.303+0000 7f931b400640 1 -- 172.21.15.46:0/2305158838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1c9d22d0 con 0x55ce1cd98e80 2024-04-26T23:51:26.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.304+0000 7f931b400640 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f93040186e0 tx=0x7f9304019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c977b93800669155 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.304+0000 7f931b400640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 0x7f92f806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9304016550 tx=0x7f9304001c20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.304+0000 7f9310c00640 1 -- 172.21.15.46:0/2305158838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304026070 con 0x55ce1cd98e80 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.304+0000 7f9310c00640 1 -- 172.21.15.46:0/2305158838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93040050e0 con 0x55ce1cd98e80 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.304+0000 7f9310c00640 1 -- 172.21.15.46:0/2305158838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304010040 con 0x55ce1cd98e80 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 -- 172.21.15.46:0/2305158838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 msgr2=0x55ce1cd5cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f93040186e0 tx=0x7f9304019000 comp rx=0 tx=0).stop 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 -- 172.21.15.46:0/2305158838 shutdown_connections 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cd876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cd5cb90 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 --2- 172.21.15.46:0/2305158838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 -- 172.21.15.46:0/2305158838 >> 172.21.15.46:0/2305158838 conn(0x55ce1ccf0d10 msgr2=0x55ce1cda2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 -- 172.21.15.46:0/2305158838 shutdown_connections 2024-04-26T23:51:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.305+0000 7f9326061740 1 -- 172.21.15.46:0/2305158838 wait complete. 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.306+0000 7f9326061740 1 -- start start 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.306+0000 7f9326061740 1 Processor -- start 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd98e80 0x55ce1cda55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce1cc47f70 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce1cd84dc0 con 0x55ce1cd87270 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ce1cdb2fd0 con 0x55ce1cd98e80 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f9326061740 10 client.0 _ll_register_callbacks cb 0x55ce1ccb6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f931be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f931be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33744/0 (socket says 172.21.15.46:33744) 2024-04-26T23:51:26.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.307+0000 7f931be00640 1 -- 172.21.15.46:0/2022210584 learned_addr learned my addr 172.21.15.46:0/2022210584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f9321200640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd98e80 0x55ce1cda55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931b400640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 -- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd98e80 msgr2=0x55ce1cda55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd98e80 0x55ce1cda55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 -- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 msgr2=0x55ce1cdb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 -- 172.21.15.46:0/2022210584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1ca9c6b0 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931b400640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f931be00640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f930c02ab40 tx=0x7f930c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c3a929aa23325f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.308+0000 7f9309600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c04a060 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f930c03b850 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93040050e0 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f930c050050 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 10 client.9792 fetch_fsmap learned FSMap version 118 2024-04-26T23:51:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55ce1cd9dfd0 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93040050e0 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f930c053020 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 10 client.9792 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 10 client.9792 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 20 client.9792 populate_metadata read hostname 'smithi046' 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd87a80 0x55ce1cd87e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 4 client.9792 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.309+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce1ccce210 con 0x55ce1cd87a80 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.310+0000 7f931b400640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd87a80 0x55ce1cd87e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.311+0000 7f931b400640 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd87a80 0x55ce1cd87e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c6e4da3a9b5959 server_cookie=8577bb28d96427a7 in_seq=0 out_seq=0 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.311+0000 7f9309600640 10 client.9792 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:51:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9309600640 1 -- 172.21.15.46:0/2022210584 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce1ccce210 con 0x55ce1cd87a80 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9309600640 10 client.9792 handle_command_reply: tid=0 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 client.9792 shutdown 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 msgr2=0x55ce1cd8a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f930c02ab40 tx=0x7f930c004ab0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 shutdown_connections 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd87a80 0x55ce1cd87e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd98e80 0x55ce1cda55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 --2- 172.21.15.46:0/2022210584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd5c7b0 0x55ce1cd8a910 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9309600640 0 client.9792 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 >> 172.21.15.46:0/2022210584 conn(0x55ce1ccf0d10 msgr2=0x55ce1cd84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 shutdown_connections 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.312+0000 7f9326061740 1 -- 172.21.15.46:0/2022210584 wait complete. 2024-04-26T23:51:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.313+0000 7f9326061740 20 client.9792 trim_cache size 0 max 16384 2024-04-26T23:51:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.333+0000 7f9326061740 1 Processor -- start 2024-04-26T23:51:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.333+0000 7f9326061740 1 -- start start 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd85a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd98e80 0x55ce1cdbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f931c2053c0 con 0x55ce1cd87270 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f931c14a950 con 0x55ce1cd98e80 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f931c204e50 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33752/0 (socket says 172.21.15.46:33752) 2024-04-26T23:51:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 -- 172.21.15.46:0/2868811327 learned_addr learned my addr 172.21.15.46:0/2868811327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 -- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 msgr2=0x55ce1cd85a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd85a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 -- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd98e80 msgr2=0x55ce1cdbbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd98e80 0x55ce1cdbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.334+0000 7f931b400640 1 -- 172.21.15.46:0/2868811327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1cd98cb0 con 0x55ce1cd87270 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.335+0000 7f931b400640 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9304013000 tx=0x7f9304007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d123aef1b6eabbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.335+0000 7f930be00640 1 -- 172.21.15.46:0/2868811327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304035070 con 0x55ce1cd87270 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.335+0000 7f930be00640 1 -- 172.21.15.46:0/2868811327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304027040 con 0x55ce1cd87270 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.335+0000 7f930be00640 1 -- 172.21.15.46:0/2868811327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304026040 con 0x55ce1cd87270 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 -- 172.21.15.46:0/2868811327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 msgr2=0x55ce1cd85f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9304013000 tx=0x7f9304007f00 comp rx=0 tx=0).stop 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 -- 172.21.15.46:0/2868811327 shutdown_connections 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd98e80 0x55ce1cdbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd87270 0x55ce1cd85f80 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 --2- 172.21.15.46:0/2868811327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cd85a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 -- 172.21.15.46:0/2868811327 >> 172.21.15.46:0/2868811327 conn(0x55ce1ccf0d10 msgr2=0x55ce1cd87e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 -- 172.21.15.46:0/2868811327 shutdown_connections 2024-04-26T23:51:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.336+0000 7f9326061740 1 -- 172.21.15.46:0/2868811327 wait complete. 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.337+0000 7f9326061740 1 -- start start 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.337+0000 7f9326061740 1 Processor -- start 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cdff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce1cd79810 con 0x55ce1cd98e80 2024-04-26T23:51:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce1cd89320 con 0x55ce1cd87270 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ce1cd79d00 con 0x55ce1cd5c7b0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9326061740 10 client.0 _ll_register_callbacks cb 0x7f93140090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33758/0 (socket says 172.21.15.46:33758) 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 -- 172.21.15.46:0/1595445676 learned_addr learned my addr 172.21.15.46:0/1595445676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f931be00640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cdff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f931b400640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 -- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 msgr2=0x55ce1cdff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cdff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 -- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 msgr2=0x55ce1cdc4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdc4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f9321200640 1 -- 172.21.15.46:0/1595445676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1cda11b0 con 0x55ce1cd98e80 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.338+0000 7f931b400640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdc4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9321200640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9314055ce0 tx=0x7f931405a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83435e766ad2024b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9309600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:51:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314088060 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f93140600b0 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9304027040 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f9314062460 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9326061740 10 client.9798 fetch_fsmap learned FSMap version 118 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55ce1cd86f50 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.339+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304027040 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f931405abb0 con 0x55ce1cd98e80 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 10 client.9798 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 10 client.9798 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 20 client.9798 populate_metadata read hostname 'smithi046' 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd85ed0 0x55ce1cd862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 4 client.9798 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "status"}) -- 0x55ce1ccc8080 con 0x55ce1cd85ed0 2024-04-26T23:51:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.340+0000 7f931be00640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd85ed0 0x55ce1cd862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f931be00640 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd85ed0 0x55ce1cd862b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e53c5bc426815c1 server_cookie=458c8a14054973b8 in_seq=0 out_seq=0 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9309600640 10 client.9798 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9309600640 1 -- 172.21.15.46:0/1595445676 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ce1ccc8080 con 0x55ce1cd85ed0 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9309600640 10 client.9798 handle_command_reply: tid=0 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 client.9798 shutdown 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 msgr2=0x55ce1cdc5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9314055ce0 tx=0x7f931405a0c0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 shutdown_connections 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55ce1cd85ed0 0x55ce1cd862b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce1cd98e80 0x55ce1cdc5140 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1cd87270 0x55ce1cdc4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 --2- 172.21.15.46:0/1595445676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ce1cd5c7b0 0x55ce1cdff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 >> 172.21.15.46:0/1595445676 conn(0x55ce1ccf0d10 msgr2=0x55ce1cd87e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9309600640 0 client.9798 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 shutdown_connections 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.341+0000 7f9326061740 1 -- 172.21.15.46:0/1595445676 wait complete. 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9326061740 20 client.9798 trim_cache size 0 max 16384 2024-04-26T23:51:26.349 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9663, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7426807169999998, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 118, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 148, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 148, 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.194763008000001 2024-04-26T23:51:26.350 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 msgr2=0x7f92f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 0x7f92f806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9304016550 tx=0x7f9304001c20 comp rx=0 tx=0).stop 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 msgr2=0x7f931c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f931400e850 tx=0x7f931400ed10 comp rx=0 tx=0).stop 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 shutdown_connections 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f806c8f0 0x7f92f806edb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c0b43f0 0x7f931c0ca520 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f931c0cdbc0 0x7f931c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 --2- 172.21.15.46:0/268256751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931c0cb0f0 0x7f931c147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 >> 172.21.15.46:0/268256751 conn(0x7f931c0b26d0 msgr2=0x7f931c0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 shutdown_connections 2024-04-26T23:51:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.342+0000 7f9321c00640 1 -- 172.21.15.46:0/268256751 wait complete. 2024-04-26T23:51:26.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:51:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.573+0000 7f493c800640 1 Processor -- start 2024-04-26T23:51:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.573+0000 7f493c800640 1 -- start start 2024-04-26T23:51:26.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.575+0000 7f493c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f493406e510 0x7f493406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.575+0000 7f493c800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49340732c0 0x7f4934073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.575+0000 7f493c800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f493406ee30 con 0x7f4934160bd0 2024-04-26T23:51:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.575+0000 7f493c800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4934161940 con 0x7f49340732c0 2024-04-26T23:51:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.575+0000 7f493c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4934071f80 con 0x7f493406e510 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f4939400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4934160bd0 0x7f4934161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60228/0 (socket says 172.21.15.46:60228) 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f4939400640 1 -- 172.21.15.46:0/1874106726 learned_addr learned my addr 172.21.15.46:0/1874106726 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f493a800640 1 --2- 172.21.15.46:0/1874106726 >> v2:172.21.15.46:3300/0 conn(0x7f49340732c0 0x7f4934073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f4939e00640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f493406e510 0x7f493406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f4938a00640 1 -- 172.21.15.46:0/1874106726 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 290931426 0 0) 0x7f493406ee30 con 0x7f4934160bd0 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f4938a00640 1 -- 172.21.15.46:0/1874106726 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f491c003520 con 0x7f4934160bd0 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f493a800640 1 -- 172.21.15.46:0/1874106726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f493406e510 msgr2=0x7f493406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f493a800640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f493406e510 0x7f493406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.576+0000 7f493a800640 1 -- 172.21.15.46:0/1874106726 >> v1:172.21.15.46:6789/0 conn(0x7f4934160bd0 legacy=0x7f4934161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.577+0000 7f493a800640 1 -- 172.21.15.46:0/1874106726 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4934072260 con 0x7f49340732c0 2024-04-26T23:51:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.578+0000 7f493a800640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49340732c0 0x7f4934073750 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f4930009d90 tx=0x7f493002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8123a2489341ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.578+0000 7f4938a00640 1 -- 172.21.15.46:0/1874106726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930007740 con 0x7f49340732c0 2024-04-26T23:51:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.578+0000 7f4938a00640 1 -- 172.21.15.46:0/1874106726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49300078e0 con 0x7f49340732c0 2024-04-26T23:51:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.578+0000 7f4938a00640 1 -- 172.21.15.46:0/1874106726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930007cb0 con 0x7f49340732c0 2024-04-26T23:51:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 -- 172.21.15.46:0/1874106726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49340732c0 msgr2=0x7f4934073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49340732c0 0x7f4934073750 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f4930009d90 tx=0x7f493002a660 comp rx=0 tx=0).stop 2024-04-26T23:51:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 -- 172.21.15.46:0/1874106726 shutdown_connections 2024-04-26T23:51:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49340732c0 0x7f4934073750 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 --2- 172.21.15.46:0/1874106726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f493406e510 0x7f493406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 -- 172.21.15.46:0/1874106726 >> 172.21.15.46:0/1874106726 conn(0x7f493406c530 msgr2=0x7f493406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 -- 172.21.15.46:0/1874106726 shutdown_connections 2024-04-26T23:51:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.579+0000 7f493c800640 1 -- 172.21.15.46:0/1874106726 wait complete. 2024-04-26T23:51:26.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.581+0000 7f493c800640 1 Processor -- start 2024-04-26T23:51:26.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.581+0000 7f493c800640 1 -- start start 2024-04-26T23:51:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f493406e510 0x7f493415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49340732c0 0x7f4934143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4934072d40 con 0x7f4934160bd0 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4934072820 con 0x7f493406e510 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4934072ec0 con 0x7f49340732c0 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.582+0000 7f493a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33776/0 (socket says 172.21.15.46:33776) 2024-04-26T23:51:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 -- 172.21.15.46:0/2181624804 learned_addr learned my addr 172.21.15.46:0/2181624804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:51:26.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f4939e00640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f493406e510 0x7f493415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f4939400640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49340732c0 0x7f4934143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 -- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49340732c0 msgr2=0x7f4934143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49340732c0 0x7f4934143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 -- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f493406e510 msgr2=0x7f493415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f493406e510 0x7f493415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.583+0000 7f493a800640 1 -- 172.21.15.46:0/2181624804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49340b4d00 con 0x7f4934160bd0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f493a800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4930009eb0 tx=0x7f4930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930040070 con 0x7f4934160bd0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4930031bc0 con 0x7f4934160bd0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49340b4fe0 con 0x7f4934160bd0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930009450 con 0x7f4934160bd0 2024-04-26T23:51:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.584+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49340b52c0 con 0x7f4934160bd0 2024-04-26T23:51:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.585+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4934069530 con 0x7f4934160bd0 2024-04-26T23:51:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.586+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4930039040 con 0x7f4934160bd0 2024-04-26T23:51:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.587+0000 7f492e000640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.588+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f49300ad140 con 0x7f4934160bd0 2024-04-26T23:51:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.588+0000 7f4939e00640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.590+0000 7f4939e00640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4928009a30 tx=0x7f4928008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.592+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f493004b340 con 0x7f4934160bd0 2024-04-26T23:51:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.839+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f493406fb00 con 0x7f4934160bd0 2024-04-26T23:51:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.839+0000 7f492e000640 1 -- 172.21.15.46:0/2181624804 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v118) ==== 53+0+73 (secure 0 0 0) 0x7f493007b3d0 con 0x7f4934160bd0 2024-04-26T23:51:26.846 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.842+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 msgr2=0x7f491806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.842+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4928009a30 tx=0x7f4928008040 comp rx=0 tx=0).stop 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.842+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 msgr2=0x7f4934138800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.842+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4930009eb0 tx=0x7f4930004a40 comp rx=0 tx=0).stop 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 shutdown_connections 2024-04-26T23:51:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4934160bd0 0x7f4934138800 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49340732c0 0x7f4934143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 --2- 172.21.15.46:0/2181624804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f493406e510 0x7f493415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 >> 172.21.15.46:0/2181624804 conn(0x7f493406c530 msgr2=0x7f493406e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 shutdown_connections 2024-04-26T23:51:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:26.843+0000 7f493c800640 1 -- 172.21.15.46:0/2181624804 wait complete. 2024-04-26T23:51:26.865 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:51:26.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:51:26.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:51:26.866 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:51:26.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:26.866 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:51:26.917 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:51:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:51:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:51:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41734sec preferred_lft 41734sec 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:51:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:51:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:26.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:26.921 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:51:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:51:27.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:27.012 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:51:27.065 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:51:27.066 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:51:27.066 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:51:27.066 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:51:27.066 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:51:27.066 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:27.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:27.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:27.288 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:51:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:51:27.339 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:51:27.339 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:51:27.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:27.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:51:27.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:51:27.420 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:51:27.472 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:51:27.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:51:27.472 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:51:27.525 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:51:27.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:27.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:51:27.662 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:51:27.663 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:51:27.663 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:51:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:51:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:51:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:51:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:51:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:51:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:51:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:51:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:51:27.716 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:51:27.716 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:51:27.716 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:51:27.716 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:51:27.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:51:27.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:27.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:51:27.802 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:51:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:client6335 2024-04-26T23:51:27.941 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:51:27.941 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.0 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:27.997 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:27.998 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:28.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282321 s, 37.1 MB/s 2024-04-26T23:51:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274601 s, 38.2 MB/s 2024-04-26T23:51:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298781 s, 35.1 MB/s 2024-04-26T23:51:28.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214907 s, 48.8 MB/s 2024-04-26T23:51:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317136 s, 33.1 MB/s 2024-04-26T23:51:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286966 s, 36.5 MB/s 2024-04-26T23:51:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308404 s, 34.0 MB/s 2024-04-26T23:51:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466901 s, 22.5 MB/s 2024-04-26T23:51:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235254 s, 44.6 MB/s 2024-04-26T23:51:28.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275851 s, 38.0 MB/s 2024-04-26T23:51:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342624 s, 30.6 MB/s 2024-04-26T23:51:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252293 s, 41.6 MB/s 2024-04-26T23:51:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245721 s, 42.7 MB/s 2024-04-26T23:51:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.624 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.624 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027231 s, 38.5 MB/s 2024-04-26T23:51:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258177 s, 40.6 MB/s 2024-04-26T23:51:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224333 s, 46.7 MB/s 2024-04-26T23:51:28.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273451 s, 38.3 MB/s 2024-04-26T23:51:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242666 s, 43.2 MB/s 2024-04-26T23:51:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245307 s, 42.7 MB/s 2024-04-26T23:51:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580757 s, 18.1 MB/s 2024-04-26T23:51:28.874 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.874 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.874 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234776 s, 44.7 MB/s 2024-04-26T23:51:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325563 s, 32.2 MB/s 2024-04-26T23:51:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267558 s, 39.2 MB/s 2024-04-26T23:51:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028675 s, 36.6 MB/s 2024-04-26T23:51:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226505 s, 46.3 MB/s 2024-04-26T23:51:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313009 s, 33.5 MB/s 2024-04-26T23:51:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022124 s, 47.4 MB/s 2024-04-26T23:51:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283624 s, 37.0 MB/s 2024-04-26T23:51:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275116 s, 38.1 MB/s 2024-04-26T23:51:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226233 s, 46.3 MB/s 2024-04-26T23:51:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210188 s, 49.9 MB/s 2024-04-26T23:51:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319578 s, 32.8 MB/s 2024-04-26T23:51:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:29.249 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.1 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:29.250 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197356 s, 53.1 MB/s 2024-04-26T23:51:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212936 s, 49.2 MB/s 2024-04-26T23:51:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235088 s, 44.6 MB/s 2024-04-26T23:51:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334557 s, 31.3 MB/s 2024-04-26T23:51:29.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207358 s, 50.6 MB/s 2024-04-26T23:51:29.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274103 s, 38.3 MB/s 2024-04-26T23:51:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225702 s, 46.5 MB/s 2024-04-26T23:51:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234438 s, 44.7 MB/s 2024-04-26T23:51:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239359 s, 43.8 MB/s 2024-04-26T23:51:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265715 s, 39.5 MB/s 2024-04-26T23:51:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248494 s, 42.2 MB/s 2024-04-26T23:51:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02534 s, 41.4 MB/s 2024-04-26T23:51:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295592 s, 35.5 MB/s 2024-04-26T23:51:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233337 s, 44.9 MB/s 2024-04-26T23:51:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246731 s, 42.5 MB/s 2024-04-26T23:51:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225724 s, 46.5 MB/s 2024-04-26T23:51:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219947 s, 47.7 MB/s 2024-04-26T23:51:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.909 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201252 s, 52.1 MB/s 2024-04-26T23:51:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228975 s, 45.8 MB/s 2024-04-26T23:51:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026043 s, 40.3 MB/s 2024-04-26T23:51:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281451 s, 37.3 MB/s 2024-04-26T23:51:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023536 s, 44.6 MB/s 2024-04-26T23:51:30.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234847 s, 44.6 MB/s 2024-04-26T23:51:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237774 s, 44.1 MB/s 2024-04-26T23:51:30.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304019 s, 34.5 MB/s 2024-04-26T23:51:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232355 s, 45.1 MB/s 2024-04-26T23:51:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213404 s, 49.1 MB/s 2024-04-26T23:51:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251147 s, 41.8 MB/s 2024-04-26T23:51:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273015 s, 38.4 MB/s 2024-04-26T23:51:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200933 s, 52.2 MB/s 2024-04-26T23:51:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222139 s, 47.2 MB/s 2024-04-26T23:51:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203909 s, 51.4 MB/s 2024-04-26T23:51:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.2 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:30.343 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:30.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228688 s, 45.9 MB/s 2024-04-26T23:51:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224131 s, 46.8 MB/s 2024-04-26T23:51:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303208 s, 34.6 MB/s 2024-04-26T23:51:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317368 s, 33.0 MB/s 2024-04-26T23:51:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022075 s, 47.5 MB/s 2024-04-26T23:51:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218256 s, 48.0 MB/s 2024-04-26T23:51:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219035 s, 47.9 MB/s 2024-04-26T23:51:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235813 s, 44.5 MB/s 2024-04-26T23:51:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252906 s, 41.5 MB/s 2024-04-26T23:51:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029044 s, 36.1 MB/s 2024-04-26T23:51:30.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236882 s, 44.3 MB/s 2024-04-26T23:51:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326013 s, 32.2 MB/s 2024-04-26T23:51:30.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219146 s, 47.8 MB/s 2024-04-26T23:51:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214854 s, 48.8 MB/s 2024-04-26T23:51:30.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256234 s, 40.9 MB/s 2024-04-26T23:51:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262997 s, 39.9 MB/s 2024-04-26T23:51:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022679 s, 46.2 MB/s 2024-04-26T23:51:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022577 s, 46.4 MB/s 2024-04-26T23:51:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246167 s, 42.6 MB/s 2024-04-26T23:51:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225102 s, 46.6 MB/s 2024-04-26T23:51:31.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.099 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223985 s, 46.8 MB/s 2024-04-26T23:51:31.127 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.127 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.127 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022956 s, 45.7 MB/s 2024-04-26T23:51:31.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253652 s, 41.3 MB/s 2024-04-26T23:51:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216966 s, 48.3 MB/s 2024-04-26T23:51:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231893 s, 45.2 MB/s 2024-04-26T23:51:31.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237921 s, 44.1 MB/s 2024-04-26T23:51:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241725 s, 43.4 MB/s 2024-04-26T23:51:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327752 s, 32.0 MB/s 2024-04-26T23:51:31.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.345 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.345 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232577 s, 45.1 MB/s 2024-04-26T23:51:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021761 s, 48.2 MB/s 2024-04-26T23:51:31.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.403 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024413 s, 43.0 MB/s 2024-04-26T23:51:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024804 s, 42.3 MB/s 2024-04-26T23:51:31.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.3 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:31.447 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:31.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238497 s, 44.0 MB/s 2024-04-26T23:51:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019919 s, 52.6 MB/s 2024-04-26T23:51:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306606 s, 34.2 MB/s 2024-04-26T23:51:31.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.688 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292812 s, 35.8 MB/s 2024-04-26T23:51:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287873 s, 36.4 MB/s 2024-04-26T23:51:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249639 s, 42.0 MB/s 2024-04-26T23:51:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217187 s, 48.3 MB/s 2024-04-26T23:51:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020147 s, 52.0 MB/s 2024-04-26T23:51:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248797 s, 42.1 MB/s 2024-04-26T23:51:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247979 s, 42.3 MB/s 2024-04-26T23:51:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235261 s, 44.6 MB/s 2024-04-26T23:51:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249626 s, 42.0 MB/s 2024-04-26T23:51:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435489 s, 24.1 MB/s 2024-04-26T23:51:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225451 s, 46.5 MB/s 2024-04-26T23:51:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232173 s, 45.2 MB/s 2024-04-26T23:51:32.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240058 s, 43.7 MB/s 2024-04-26T23:51:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021927 s, 47.8 MB/s 2024-04-26T23:51:32.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.124 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.125 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220078 s, 47.6 MB/s 2024-04-26T23:51:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313385 s, 33.5 MB/s 2024-04-26T23:51:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234189 s, 44.8 MB/s 2024-04-26T23:51:32.219 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220933 s, 47.5 MB/s 2024-04-26T23:51:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245884 s, 42.6 MB/s 2024-04-26T23:51:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244696 s, 42.9 MB/s 2024-04-26T23:51:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247545 s, 42.4 MB/s 2024-04-26T23:51:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022732 s, 46.1 MB/s 2024-04-26T23:51:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02071 s, 50.6 MB/s 2024-04-26T23:51:32.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.402 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029845 s, 35.1 MB/s 2024-04-26T23:51:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230751 s, 45.4 MB/s 2024-04-26T23:51:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020817 s, 50.4 MB/s 2024-04-26T23:51:32.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234167 s, 44.8 MB/s 2024-04-26T23:51:32.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254056 s, 41.3 MB/s 2024-04-26T23:51:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249658 s, 42.0 MB/s 2024-04-26T23:51:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.4 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:32.565 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:32.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:32.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.720 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311966 s, 33.6 MB/s 2024-04-26T23:51:32.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234215 s, 44.8 MB/s 2024-04-26T23:51:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214631 s, 48.9 MB/s 2024-04-26T23:51:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219434 s, 47.8 MB/s 2024-04-26T23:51:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215474 s, 48.7 MB/s 2024-04-26T23:51:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224295 s, 46.7 MB/s 2024-04-26T23:51:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226091 s, 46.4 MB/s 2024-04-26T23:51:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241492 s, 43.4 MB/s 2024-04-26T23:51:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277401 s, 37.8 MB/s 2024-04-26T23:51:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245657 s, 42.7 MB/s 2024-04-26T23:51:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273488 s, 38.3 MB/s 2024-04-26T23:51:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232449 s, 45.1 MB/s 2024-04-26T23:51:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225234 s, 46.6 MB/s 2024-04-26T23:51:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023133 s, 45.3 MB/s 2024-04-26T23:51:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226133 s, 46.4 MB/s 2024-04-26T23:51:33.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.161 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245296 s, 42.7 MB/s 2024-04-26T23:51:33.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.191 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228569 s, 45.9 MB/s 2024-04-26T23:51:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024373 s, 43.0 MB/s 2024-04-26T23:51:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211745 s, 49.5 MB/s 2024-04-26T23:51:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215602 s, 48.6 MB/s 2024-04-26T23:51:33.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.308 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.308 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243319 s, 43.1 MB/s 2024-04-26T23:51:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321525 s, 32.6 MB/s 2024-04-26T23:51:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241732 s, 43.4 MB/s 2024-04-26T23:51:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225658 s, 46.5 MB/s 2024-04-26T23:51:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213661 s, 49.1 MB/s 2024-04-26T23:51:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301917 s, 34.7 MB/s 2024-04-26T23:51:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307694 s, 34.1 MB/s 2024-04-26T23:51:33.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.534 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224416 s, 46.7 MB/s 2024-04-26T23:51:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220416 s, 47.6 MB/s 2024-04-26T23:51:33.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214131 s, 49.0 MB/s 2024-04-26T23:51:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019925 s, 52.6 MB/s 2024-04-26T23:51:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297227 s, 35.3 MB/s 2024-04-26T23:51:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.5 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:33.667 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:33.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286067 s, 36.7 MB/s 2024-04-26T23:51:33.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219403 s, 47.8 MB/s 2024-04-26T23:51:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253191 s, 41.4 MB/s 2024-04-26T23:51:33.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.906 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307064 s, 34.1 MB/s 2024-04-26T23:51:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264902 s, 39.6 MB/s 2024-04-26T23:51:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228703 s, 45.8 MB/s 2024-04-26T23:51:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235995 s, 44.4 MB/s 2024-04-26T23:51:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251793 s, 41.6 MB/s 2024-04-26T23:51:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217504 s, 48.2 MB/s 2024-04-26T23:51:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285863 s, 36.7 MB/s 2024-04-26T23:51:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255192 s, 41.1 MB/s 2024-04-26T23:51:34.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.157 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284633 s, 36.8 MB/s 2024-04-26T23:51:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326732 s, 32.1 MB/s 2024-04-26T23:51:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274473 s, 38.2 MB/s 2024-04-26T23:51:34.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.252 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182313 s, 57.5 MB/s 2024-04-26T23:51:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02089 s, 50.2 MB/s 2024-04-26T23:51:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226431 s, 46.3 MB/s 2024-04-26T23:51:34.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.338 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.338 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219697 s, 47.7 MB/s 2024-04-26T23:51:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339373 s, 30.9 MB/s 2024-04-26T23:51:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218896 s, 47.9 MB/s 2024-04-26T23:51:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207949 s, 50.4 MB/s 2024-04-26T23:51:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263207 s, 39.8 MB/s 2024-04-26T23:51:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312005 s, 33.6 MB/s 2024-04-26T23:51:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230825 s, 45.4 MB/s 2024-04-26T23:51:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240385 s, 43.6 MB/s 2024-04-26T23:51:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226324 s, 46.3 MB/s 2024-04-26T23:51:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321746 s, 32.6 MB/s 2024-04-26T23:51:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216116 s, 48.5 MB/s 2024-04-26T23:51:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259099 s, 40.5 MB/s 2024-04-26T23:51:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223261 s, 47.0 MB/s 2024-04-26T23:51:34.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290918 s, 36.0 MB/s 2024-04-26T23:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211019 s, 49.7 MB/s 2024-04-26T23:51:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.6 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:34.810 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:34.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251553 s, 41.7 MB/s 2024-04-26T23:51:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232944 s, 45.0 MB/s 2024-04-26T23:51:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235006 s, 44.6 MB/s 2024-04-26T23:51:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216108 s, 48.5 MB/s 2024-04-26T23:51:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291026 s, 36.0 MB/s 2024-04-26T23:51:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227297 s, 46.1 MB/s 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248621 s, 42.2 MB/s 2024-04-26T23:51:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273328 s, 38.4 MB/s 2024-04-26T23:51:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205406 s, 51.0 MB/s 2024-04-26T23:51:35.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.230 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236641 s, 44.3 MB/s 2024-04-26T23:51:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242612 s, 43.2 MB/s 2024-04-26T23:51:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228409 s, 45.9 MB/s 2024-04-26T23:51:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.321 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245349 s, 42.7 MB/s 2024-04-26T23:51:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.350 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232785 s, 45.0 MB/s 2024-04-26T23:51:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232552 s, 45.1 MB/s 2024-04-26T23:51:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212143 s, 49.4 MB/s 2024-04-26T23:51:35.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.440 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026694 s, 39.3 MB/s 2024-04-26T23:51:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276828 s, 37.9 MB/s 2024-04-26T23:51:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023972 s, 43.7 MB/s 2024-04-26T23:51:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027631 s, 37.9 MB/s 2024-04-26T23:51:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02551 s, 41.1 MB/s 2024-04-26T23:51:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254575 s, 41.2 MB/s 2024-04-26T23:51:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249363 s, 42.1 MB/s 2024-04-26T23:51:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027344 s, 38.3 MB/s 2024-04-26T23:51:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249283 s, 42.1 MB/s 2024-04-26T23:51:35.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.723 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211659 s, 49.5 MB/s 2024-04-26T23:51:35.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.753 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244007 s, 43.0 MB/s 2024-04-26T23:51:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204898 s, 51.2 MB/s 2024-04-26T23:51:35.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246051 s, 42.6 MB/s 2024-04-26T23:51:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246388 s, 42.6 MB/s 2024-04-26T23:51:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024551 s, 42.7 MB/s 2024-04-26T23:51:35.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:35.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:35.900 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227715 s, 46.0 MB/s 2024-04-26T23:51:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.7 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:35.913 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231296 s, 45.3 MB/s 2024-04-26T23:51:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.089 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.089 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247076 s, 42.4 MB/s 2024-04-26T23:51:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01729 s, 60.6 MB/s 2024-04-26T23:51:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224877 s, 46.6 MB/s 2024-04-26T23:51:36.170 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282874 s, 37.1 MB/s 2024-04-26T23:51:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312492 s, 33.6 MB/s 2024-04-26T23:51:36.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266098 s, 39.4 MB/s 2024-04-26T23:51:36.271 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.271 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.271 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249081 s, 42.1 MB/s 2024-04-26T23:51:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211959 s, 49.5 MB/s 2024-04-26T23:51:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176835 s, 59.3 MB/s 2024-04-26T23:51:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235738 s, 44.5 MB/s 2024-04-26T23:51:36.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293706 s, 35.7 MB/s 2024-04-26T23:51:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291144 s, 36.0 MB/s 2024-04-26T23:51:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294062 s, 35.7 MB/s 2024-04-26T23:51:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234653 s, 44.7 MB/s 2024-04-26T23:51:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246693 s, 42.5 MB/s 2024-04-26T23:51:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236506 s, 44.3 MB/s 2024-04-26T23:51:36.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227978 s, 46.0 MB/s 2024-04-26T23:51:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.608 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273221 s, 38.4 MB/s 2024-04-26T23:51:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254272 s, 41.2 MB/s 2024-04-26T23:51:36.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.670 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231453 s, 45.3 MB/s 2024-04-26T23:51:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212708 s, 49.3 MB/s 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233883 s, 44.8 MB/s 2024-04-26T23:51:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243878 s, 43.0 MB/s 2024-04-26T23:51:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251426 s, 41.7 MB/s 2024-04-26T23:51:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254384 s, 41.2 MB/s 2024-04-26T23:51:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206802 s, 50.7 MB/s 2024-04-26T23:51:36.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.877 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241415 s, 43.4 MB/s 2024-04-26T23:51:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231305 s, 45.3 MB/s 2024-04-26T23:51:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296036 s, 35.4 MB/s 2024-04-26T23:51:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223805 s, 46.9 MB/s 2024-04-26T23:51:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269876 s, 38.9 MB/s 2024-04-26T23:51:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:37.016 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.8 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:37.017 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:37.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:37.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.158 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202302 s, 51.8 MB/s 2024-04-26T23:51:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316153 s, 33.2 MB/s 2024-04-26T23:51:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294145 s, 35.6 MB/s 2024-04-26T23:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290607 s, 36.1 MB/s 2024-04-26T23:51:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215631 s, 48.6 MB/s 2024-04-26T23:51:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216066 s, 48.5 MB/s 2024-04-26T23:51:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219563 s, 47.8 MB/s 2024-04-26T23:51:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302297 s, 34.7 MB/s 2024-04-26T23:51:37.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.421 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239865 s, 43.7 MB/s 2024-04-26T23:51:37.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.453 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245071 s, 42.8 MB/s 2024-04-26T23:51:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244836 s, 42.8 MB/s 2024-04-26T23:51:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310065 s, 33.8 MB/s 2024-04-26T23:51:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240421 s, 43.6 MB/s 2024-04-26T23:51:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296266 s, 35.4 MB/s 2024-04-26T23:51:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232213 s, 45.2 MB/s 2024-04-26T23:51:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207113 s, 50.6 MB/s 2024-04-26T23:51:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228151 s, 46.0 MB/s 2024-04-26T23:51:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246893 s, 42.5 MB/s 2024-04-26T23:51:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.754 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291049 s, 36.0 MB/s 2024-04-26T23:51:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225355 s, 46.5 MB/s 2024-04-26T23:51:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250532 s, 41.9 MB/s 2024-04-26T23:51:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250042 s, 41.9 MB/s 2024-04-26T23:51:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226028 s, 46.4 MB/s 2024-04-26T23:51:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242441 s, 43.3 MB/s 2024-04-26T23:51:37.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233532 s, 44.9 MB/s 2024-04-26T23:51:37.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:37.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:37.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261585 s, 40.1 MB/s 2024-04-26T23:51:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338353 s, 31.0 MB/s 2024-04-26T23:51:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254885 s, 41.1 MB/s 2024-04-26T23:51:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310898 s, 33.7 MB/s 2024-04-26T23:51:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234807 s, 44.7 MB/s 2024-04-26T23:51:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232875 s, 45.0 MB/s 2024-04-26T23:51:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260788 s, 40.2 MB/s 2024-04-26T23:51:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.9 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:38.191 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290018 s, 36.2 MB/s 2024-04-26T23:51:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254602 s, 41.2 MB/s 2024-04-26T23:51:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343832 s, 30.5 MB/s 2024-04-26T23:51:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276658 s, 37.9 MB/s 2024-04-26T23:51:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242853 s, 43.2 MB/s 2024-04-26T23:51:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.515 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233094 s, 45.0 MB/s 2024-04-26T23:51:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266982 s, 39.3 MB/s 2024-04-26T23:51:38.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309421 s, 33.9 MB/s 2024-04-26T23:51:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234738 s, 44.7 MB/s 2024-04-26T23:51:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199801 s, 52.5 MB/s 2024-04-26T23:51:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292208 s, 35.9 MB/s 2024-04-26T23:51:38.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.703 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022042 s, 47.6 MB/s 2024-04-26T23:51:38.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.739 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285506 s, 36.7 MB/s 2024-04-26T23:51:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257605 s, 40.7 MB/s 2024-04-26T23:51:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021367 s, 49.1 MB/s 2024-04-26T23:51:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247739 s, 42.3 MB/s 2024-04-26T23:51:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219655 s, 47.7 MB/s 2024-04-26T23:51:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247911 s, 42.3 MB/s 2024-04-26T23:51:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234936 s, 44.6 MB/s 2024-04-26T23:51:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026984 s, 38.9 MB/s 2024-04-26T23:51:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252767 s, 41.5 MB/s 2024-04-26T23:51:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.016 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254032 s, 41.3 MB/s 2024-04-26T23:51:39.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234825 s, 44.7 MB/s 2024-04-26T23:51:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229536 s, 45.7 MB/s 2024-04-26T23:51:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023032 s, 45.5 MB/s 2024-04-26T23:51:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229784 s, 45.6 MB/s 2024-04-26T23:51:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02502 s, 41.9 MB/s 2024-04-26T23:51:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249604 s, 42.0 MB/s 2024-04-26T23:51:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280828 s, 37.3 MB/s 2024-04-26T23:51:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248462 s, 42.2 MB/s 2024-04-26T23:51:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178587 s, 58.7 MB/s 2024-04-26T23:51:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292369 s, 35.9 MB/s 2024-04-26T23:51:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:39.331 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:39.331 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:39.332 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.10 2024-04-26T23:51:39.332 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:39.332 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:39.332 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:39.332 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:39.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.456 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017364 s, 60.4 MB/s 2024-04-26T23:51:39.483 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.483 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.483 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225466 s, 46.5 MB/s 2024-04-26T23:51:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020106 s, 52.2 MB/s 2024-04-26T23:51:39.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.539 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256044 s, 41.0 MB/s 2024-04-26T23:51:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.570 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234693 s, 44.7 MB/s 2024-04-26T23:51:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238541 s, 44.0 MB/s 2024-04-26T23:51:39.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.640 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338284 s, 31.0 MB/s 2024-04-26T23:51:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222864 s, 47.0 MB/s 2024-04-26T23:51:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209239 s, 50.1 MB/s 2024-04-26T23:51:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254376 s, 41.2 MB/s 2024-04-26T23:51:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215892 s, 48.6 MB/s 2024-04-26T23:51:39.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.787 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258695 s, 40.5 MB/s 2024-04-26T23:51:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243146 s, 43.1 MB/s 2024-04-26T23:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221132 s, 47.4 MB/s 2024-04-26T23:51:39.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.875 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244823 s, 42.8 MB/s 2024-04-26T23:51:39.901 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.901 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210486 s, 49.8 MB/s 2024-04-26T23:51:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218896 s, 47.9 MB/s 2024-04-26T23:51:39.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237967 s, 44.1 MB/s 2024-04-26T23:51:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256403 s, 40.9 MB/s 2024-04-26T23:51:40.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182029 s, 57.6 MB/s 2024-04-26T23:51:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226417 s, 46.3 MB/s 2024-04-26T23:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242713 s, 43.2 MB/s 2024-04-26T23:51:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276967 s, 37.9 MB/s 2024-04-26T23:51:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290263 s, 36.1 MB/s 2024-04-26T23:51:40.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.175 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282962 s, 37.1 MB/s 2024-04-26T23:51:40.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.203 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215796 s, 48.6 MB/s 2024-04-26T23:51:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222453 s, 47.1 MB/s 2024-04-26T23:51:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246921 s, 42.5 MB/s 2024-04-26T23:51:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024411 s, 43.0 MB/s 2024-04-26T23:51:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221762 s, 47.3 MB/s 2024-04-26T23:51:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281074 s, 37.3 MB/s 2024-04-26T23:51:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296564 s, 35.4 MB/s 2024-04-26T23:51:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.11 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:40.401 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210406 s, 49.8 MB/s 2024-04-26T23:51:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247186 s, 42.4 MB/s 2024-04-26T23:51:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220571 s, 47.5 MB/s 2024-04-26T23:51:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028483 s, 36.8 MB/s 2024-04-26T23:51:40.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.676 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318333 s, 32.9 MB/s 2024-04-26T23:51:40.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.708 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026921 s, 39.0 MB/s 2024-04-26T23:51:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222753 s, 47.1 MB/s 2024-04-26T23:51:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219195 s, 47.8 MB/s 2024-04-26T23:51:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.792 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226396 s, 46.3 MB/s 2024-04-26T23:51:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252675 s, 41.5 MB/s 2024-04-26T23:51:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230508 s, 45.5 MB/s 2024-04-26T23:51:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255275 s, 41.1 MB/s 2024-04-26T23:51:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216075 s, 48.5 MB/s 2024-04-26T23:51:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241268 s, 43.5 MB/s 2024-04-26T23:51:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241444 s, 43.4 MB/s 2024-04-26T23:51:41.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257859 s, 40.7 MB/s 2024-04-26T23:51:41.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.034 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022402 s, 46.8 MB/s 2024-04-26T23:51:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236764 s, 44.3 MB/s 2024-04-26T23:51:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.096 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268573 s, 39.0 MB/s 2024-04-26T23:51:41.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.125 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240684 s, 43.6 MB/s 2024-04-26T23:51:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197255 s, 53.2 MB/s 2024-04-26T23:51:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190168 s, 55.1 MB/s 2024-04-26T23:51:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.204 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230769 s, 45.4 MB/s 2024-04-26T23:51:41.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230361 s, 45.5 MB/s 2024-04-26T23:51:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189696 s, 55.3 MB/s 2024-04-26T23:51:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275052 s, 38.1 MB/s 2024-04-26T23:51:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234334 s, 44.7 MB/s 2024-04-26T23:51:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232961 s, 45.0 MB/s 2024-04-26T23:51:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236944 s, 44.3 MB/s 2024-04-26T23:51:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199071 s, 52.7 MB/s 2024-04-26T23:51:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241628 s, 43.4 MB/s 2024-04-26T23:51:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253741 s, 41.3 MB/s 2024-04-26T23:51:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.12 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:41.477 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024388 s, 43.0 MB/s 2024-04-26T23:51:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266708 s, 39.3 MB/s 2024-04-26T23:51:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234652 s, 44.7 MB/s 2024-04-26T23:51:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221301 s, 47.4 MB/s 2024-04-26T23:51:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244163 s, 42.9 MB/s 2024-04-26T23:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245559 s, 42.7 MB/s 2024-04-26T23:51:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242168 s, 43.3 MB/s 2024-04-26T23:51:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.829 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187584 s, 55.9 MB/s 2024-04-26T23:51:41.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266389 s, 39.4 MB/s 2024-04-26T23:51:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261618 s, 40.1 MB/s 2024-04-26T23:51:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244216 s, 42.9 MB/s 2024-04-26T23:51:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217613 s, 48.2 MB/s 2024-04-26T23:51:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030627 s, 34.2 MB/s 2024-04-26T23:51:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239864 s, 43.7 MB/s 2024-04-26T23:51:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237287 s, 44.2 MB/s 2024-04-26T23:51:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025455 s, 41.2 MB/s 2024-04-26T23:51:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226115 s, 46.4 MB/s 2024-04-26T23:51:42.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.144 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296406 s, 35.4 MB/s 2024-04-26T23:51:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223925 s, 46.8 MB/s 2024-04-26T23:51:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288221 s, 36.4 MB/s 2024-04-26T23:51:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230386 s, 45.5 MB/s 2024-04-26T23:51:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210602 s, 49.8 MB/s 2024-04-26T23:51:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262155 s, 40.0 MB/s 2024-04-26T23:51:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247276 s, 42.4 MB/s 2024-04-26T23:51:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028051 s, 37.4 MB/s 2024-04-26T23:51:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244945 s, 42.8 MB/s 2024-04-26T23:51:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292122 s, 35.9 MB/s 2024-04-26T23:51:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253482 s, 41.4 MB/s 2024-04-26T23:51:42.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.494 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298049 s, 35.2 MB/s 2024-04-26T23:51:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220456 s, 47.6 MB/s 2024-04-26T23:51:42.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:42.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237125 s, 44.2 MB/s 2024-04-26T23:51:42.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.578 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198168 s, 52.9 MB/s 2024-04-26T23:51:42.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.13 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:42.591 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:42.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237467 s, 44.2 MB/s 2024-04-26T23:51:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210644 s, 49.8 MB/s 2024-04-26T23:51:42.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309308 s, 33.9 MB/s 2024-04-26T23:51:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.868 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245599 s, 42.7 MB/s 2024-04-26T23:51:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326853 s, 32.1 MB/s 2024-04-26T23:51:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279451 s, 37.5 MB/s 2024-04-26T23:51:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174289 s, 60.2 MB/s 2024-04-26T23:51:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276273 s, 38.0 MB/s 2024-04-26T23:51:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.031 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273547 s, 38.3 MB/s 2024-04-26T23:51:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260689 s, 40.2 MB/s 2024-04-26T23:51:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233048 s, 45.0 MB/s 2024-04-26T23:51:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204683 s, 51.2 MB/s 2024-04-26T23:51:43.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189012 s, 55.5 MB/s 2024-04-26T23:51:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274097 s, 38.3 MB/s 2024-04-26T23:51:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300646 s, 34.9 MB/s 2024-04-26T23:51:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.243 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215326 s, 48.7 MB/s 2024-04-26T23:51:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203557 s, 51.5 MB/s 2024-04-26T23:51:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021886 s, 47.9 MB/s 2024-04-26T23:51:43.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.333 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301234 s, 34.8 MB/s 2024-04-26T23:51:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212794 s, 49.3 MB/s 2024-04-26T23:51:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023171 s, 45.3 MB/s 2024-04-26T23:51:43.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.425 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288052 s, 36.4 MB/s 2024-04-26T23:51:43.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.452 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211719 s, 49.5 MB/s 2024-04-26T23:51:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221143 s, 47.4 MB/s 2024-04-26T23:51:43.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285033 s, 36.8 MB/s 2024-04-26T23:51:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027281 s, 38.4 MB/s 2024-04-26T23:51:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216203 s, 48.5 MB/s 2024-04-26T23:51:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218387 s, 48.0 MB/s 2024-04-26T23:51:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284776 s, 36.8 MB/s 2024-04-26T23:51:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303046 s, 34.6 MB/s 2024-04-26T23:51:43.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023475 s, 44.7 MB/s 2024-04-26T23:51:43.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.730 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024119 s, 43.5 MB/s 2024-04-26T23:51:43.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.14 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:43.742 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:43.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255727 s, 41.0 MB/s 2024-04-26T23:51:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227493 s, 46.1 MB/s 2024-04-26T23:51:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302714 s, 34.6 MB/s 2024-04-26T23:51:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329513 s, 31.8 MB/s 2024-04-26T23:51:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223379 s, 46.9 MB/s 2024-04-26T23:51:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232351 s, 45.1 MB/s 2024-04-26T23:51:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219755 s, 47.7 MB/s 2024-04-26T23:51:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211207 s, 49.6 MB/s 2024-04-26T23:51:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212752 s, 49.3 MB/s 2024-04-26T23:51:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243381 s, 43.1 MB/s 2024-04-26T23:51:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238692 s, 43.9 MB/s 2024-04-26T23:51:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227613 s, 46.1 MB/s 2024-04-26T23:51:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213793 s, 49.0 MB/s 2024-04-26T23:51:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.294 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305728 s, 34.3 MB/s 2024-04-26T23:51:44.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.320 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021016 s, 49.9 MB/s 2024-04-26T23:51:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019361 s, 54.2 MB/s 2024-04-26T23:51:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279706 s, 37.5 MB/s 2024-04-26T23:51:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229989 s, 45.6 MB/s 2024-04-26T23:51:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210393 s, 49.8 MB/s 2024-04-26T23:51:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245557 s, 42.7 MB/s 2024-04-26T23:51:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221797 s, 47.3 MB/s 2024-04-26T23:51:44.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.526 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277386 s, 37.8 MB/s 2024-04-26T23:51:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250384 s, 41.9 MB/s 2024-04-26T23:51:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222851 s, 47.1 MB/s 2024-04-26T23:51:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249771 s, 42.0 MB/s 2024-04-26T23:51:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259628 s, 40.4 MB/s 2024-04-26T23:51:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218148 s, 48.1 MB/s 2024-04-26T23:51:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024338 s, 43.1 MB/s 2024-04-26T23:51:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249033 s, 42.1 MB/s 2024-04-26T23:51:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231154 s, 45.4 MB/s 2024-04-26T23:51:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215882 s, 48.6 MB/s 2024-04-26T23:51:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229228 s, 45.7 MB/s 2024-04-26T23:51:44.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:44.835 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.15 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:44.836 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259493 s, 40.4 MB/s 2024-04-26T23:51:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219699 s, 47.7 MB/s 2024-04-26T23:51:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269033 s, 39.0 MB/s 2024-04-26T23:51:45.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214376 s, 48.9 MB/s 2024-04-26T23:51:45.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254842 s, 41.1 MB/s 2024-04-26T23:51:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243482 s, 43.1 MB/s 2024-04-26T23:51:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245883 s, 42.6 MB/s 2024-04-26T23:51:45.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254712 s, 41.2 MB/s 2024-04-26T23:51:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019666 s, 53.3 MB/s 2024-04-26T23:51:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314835 s, 33.3 MB/s 2024-04-26T23:51:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281841 s, 37.2 MB/s 2024-04-26T23:51:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240031 s, 43.7 MB/s 2024-04-26T23:51:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224239 s, 46.8 MB/s 2024-04-26T23:51:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362958 s, 28.9 MB/s 2024-04-26T23:51:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282811 s, 37.1 MB/s 2024-04-26T23:51:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233099 s, 45.0 MB/s 2024-04-26T23:51:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285835 s, 36.7 MB/s 2024-04-26T23:51:45.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.529 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244714 s, 42.8 MB/s 2024-04-26T23:51:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229283 s, 45.7 MB/s 2024-04-26T23:51:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219194 s, 47.8 MB/s 2024-04-26T23:51:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249139 s, 42.1 MB/s 2024-04-26T23:51:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023105 s, 45.4 MB/s 2024-04-26T23:51:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203808 s, 51.4 MB/s 2024-04-26T23:51:45.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.698 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218105 s, 48.1 MB/s 2024-04-26T23:51:45.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299614 s, 35.0 MB/s 2024-04-26T23:51:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227194 s, 46.2 MB/s 2024-04-26T23:51:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210433 s, 49.8 MB/s 2024-04-26T23:51:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260873 s, 40.2 MB/s 2024-04-26T23:51:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255265 s, 41.1 MB/s 2024-04-26T23:51:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230533 s, 45.5 MB/s 2024-04-26T23:51:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224518 s, 46.7 MB/s 2024-04-26T23:51:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240343 s, 43.6 MB/s 2024-04-26T23:51:45.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:45.956 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.16 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:45.957 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303642 s, 34.5 MB/s 2024-04-26T23:51:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272698 s, 38.5 MB/s 2024-04-26T23:51:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225065 s, 46.6 MB/s 2024-04-26T23:51:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288533 s, 36.3 MB/s 2024-04-26T23:51:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259418 s, 40.4 MB/s 2024-04-26T23:51:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264902 s, 39.6 MB/s 2024-04-26T23:51:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238756 s, 43.9 MB/s 2024-04-26T23:51:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272348 s, 38.5 MB/s 2024-04-26T23:51:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212044 s, 49.5 MB/s 2024-04-26T23:51:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238641 s, 43.9 MB/s 2024-04-26T23:51:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254893 s, 41.1 MB/s 2024-04-26T23:51:46.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302101 s, 34.7 MB/s 2024-04-26T23:51:46.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279728 s, 37.5 MB/s 2024-04-26T23:51:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237334 s, 44.2 MB/s 2024-04-26T23:51:46.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240999 s, 43.5 MB/s 2024-04-26T23:51:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306687 s, 34.2 MB/s 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347969 s, 30.1 MB/s 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024406 s, 43.0 MB/s 2024-04-26T23:51:46.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.691 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230601 s, 45.5 MB/s 2024-04-26T23:51:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229425 s, 45.7 MB/s 2024-04-26T23:51:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227239 s, 46.1 MB/s 2024-04-26T23:51:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026832 s, 39.1 MB/s 2024-04-26T23:51:46.811 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.812 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.812 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224278 s, 46.8 MB/s 2024-04-26T23:51:46.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.845 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260796 s, 40.2 MB/s 2024-04-26T23:51:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207379 s, 50.6 MB/s 2024-04-26T23:51:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216399 s, 48.5 MB/s 2024-04-26T23:51:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243515 s, 43.1 MB/s 2024-04-26T23:51:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306307 s, 34.2 MB/s 2024-04-26T23:51:46.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:46.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:46.997 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238967 s, 43.9 MB/s 2024-04-26T23:51:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200182 s, 52.4 MB/s 2024-04-26T23:51:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197986 s, 53.0 MB/s 2024-04-26T23:51:47.082 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.082 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.083 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285116 s, 36.8 MB/s 2024-04-26T23:51:47.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.17 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:47.095 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:47.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202507 s, 51.8 MB/s 2024-04-26T23:51:47.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208697 s, 50.2 MB/s 2024-04-26T23:51:47.284 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200968 s, 52.2 MB/s 2024-04-26T23:51:47.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228726 s, 45.8 MB/s 2024-04-26T23:51:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261912 s, 40.0 MB/s 2024-04-26T23:51:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.379 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259315 s, 40.4 MB/s 2024-04-26T23:51:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230668 s, 45.5 MB/s 2024-04-26T23:51:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289006 s, 36.3 MB/s 2024-04-26T23:51:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247839 s, 42.3 MB/s 2024-04-26T23:51:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189706 s, 55.3 MB/s 2024-04-26T23:51:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288045 s, 36.4 MB/s 2024-04-26T23:51:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249054 s, 42.1 MB/s 2024-04-26T23:51:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235943 s, 44.4 MB/s 2024-04-26T23:51:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260387 s, 40.3 MB/s 2024-04-26T23:51:47.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281144 s, 37.3 MB/s 2024-04-26T23:51:47.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.689 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247323 s, 42.4 MB/s 2024-04-26T23:51:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230953 s, 45.4 MB/s 2024-04-26T23:51:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233508 s, 44.9 MB/s 2024-04-26T23:51:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224488 s, 46.7 MB/s 2024-04-26T23:51:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207515 s, 50.5 MB/s 2024-04-26T23:51:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309619 s, 33.9 MB/s 2024-04-26T23:51:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217162 s, 48.3 MB/s 2024-04-26T23:51:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285575 s, 36.7 MB/s 2024-04-26T23:51:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309009 s, 33.9 MB/s 2024-04-26T23:51:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248407 s, 42.2 MB/s 2024-04-26T23:51:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244155 s, 42.9 MB/s 2024-04-26T23:51:48.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.026 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233002 s, 45.0 MB/s 2024-04-26T23:51:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291659 s, 36.0 MB/s 2024-04-26T23:51:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208259 s, 50.3 MB/s 2024-04-26T23:51:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023253 s, 45.1 MB/s 2024-04-26T23:51:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234933 s, 44.6 MB/s 2024-04-26T23:51:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287533 s, 36.5 MB/s 2024-04-26T23:51:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:48.194 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:48.194 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:48.194 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.18 2024-04-26T23:51:48.194 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:48.195 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:48.195 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:48.195 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:48.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.335 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229633 s, 45.7 MB/s 2024-04-26T23:51:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182356 s, 57.5 MB/s 2024-04-26T23:51:48.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223069 s, 47.0 MB/s 2024-04-26T23:51:48.424 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.425 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.425 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331806 s, 31.6 MB/s 2024-04-26T23:51:48.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.457 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025195 s, 41.6 MB/s 2024-04-26T23:51:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216186 s, 48.5 MB/s 2024-04-26T23:51:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028488 s, 36.8 MB/s 2024-04-26T23:51:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022179 s, 47.3 MB/s 2024-04-26T23:51:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278435 s, 37.7 MB/s 2024-04-26T23:51:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227997 s, 46.0 MB/s 2024-04-26T23:51:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241846 s, 43.4 MB/s 2024-04-26T23:51:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.671 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024502 s, 42.8 MB/s 2024-04-26T23:51:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230929 s, 45.4 MB/s 2024-04-26T23:51:48.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.728 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.728 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207548 s, 50.5 MB/s 2024-04-26T23:51:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262713 s, 39.9 MB/s 2024-04-26T23:51:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230897 s, 45.4 MB/s 2024-04-26T23:51:48.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.823 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272981 s, 38.4 MB/s 2024-04-26T23:51:48.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273905 s, 38.3 MB/s 2024-04-26T23:51:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292671 s, 35.8 MB/s 2024-04-26T23:51:48.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.930 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.931 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031303 s, 33.5 MB/s 2024-04-26T23:51:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220938 s, 47.5 MB/s 2024-04-26T23:51:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272943 s, 38.4 MB/s 2024-04-26T23:51:49.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.026 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.026 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271727 s, 38.6 MB/s 2024-04-26T23:51:49.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.053 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216681 s, 48.4 MB/s 2024-04-26T23:51:49.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.087 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283556 s, 37.0 MB/s 2024-04-26T23:51:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024056 s, 43.6 MB/s 2024-04-26T23:51:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231567 s, 45.3 MB/s 2024-04-26T23:51:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279152 s, 37.6 MB/s 2024-04-26T23:51:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233539 s, 44.9 MB/s 2024-04-26T23:51:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423686 s, 24.7 MB/s 2024-04-26T23:51:49.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.286 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204244 s, 51.3 MB/s 2024-04-26T23:51:49.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.320 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283244 s, 37.0 MB/s 2024-04-26T23:51:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.19 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:49.332 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231474 s, 45.3 MB/s 2024-04-26T23:51:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239685 s, 43.7 MB/s 2024-04-26T23:51:49.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.539 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.539 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324076 s, 32.4 MB/s 2024-04-26T23:51:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221691 s, 47.3 MB/s 2024-04-26T23:51:49.598 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.598 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.598 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256813 s, 40.8 MB/s 2024-04-26T23:51:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304883 s, 34.4 MB/s 2024-04-26T23:51:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245138 s, 42.8 MB/s 2024-04-26T23:51:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250438 s, 41.9 MB/s 2024-04-26T23:51:49.729 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.729 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027165 s, 38.6 MB/s 2024-04-26T23:51:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240416 s, 43.6 MB/s 2024-04-26T23:51:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018924 s, 55.4 MB/s 2024-04-26T23:51:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279819 s, 37.5 MB/s 2024-04-26T23:51:49.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230739 s, 45.4 MB/s 2024-04-26T23:51:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215313 s, 48.7 MB/s 2024-04-26T23:51:49.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269037 s, 39.0 MB/s 2024-04-26T23:51:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255293 s, 41.1 MB/s 2024-04-26T23:51:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272674 s, 38.5 MB/s 2024-04-26T23:51:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236369 s, 44.4 MB/s 2024-04-26T23:51:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215156 s, 48.7 MB/s 2024-04-26T23:51:50.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247474 s, 42.4 MB/s 2024-04-26T23:51:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227234 s, 46.1 MB/s 2024-04-26T23:51:50.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222209 s, 47.2 MB/s 2024-04-26T23:51:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231708 s, 45.3 MB/s 2024-04-26T23:51:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233006 s, 45.0 MB/s 2024-04-26T23:51:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238375 s, 44.0 MB/s 2024-04-26T23:51:50.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208638 s, 50.3 MB/s 2024-04-26T23:51:50.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021291 s, 49.2 MB/s 2024-04-26T23:51:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238719 s, 43.9 MB/s 2024-04-26T23:51:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033767 s, 31.1 MB/s 2024-04-26T23:51:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233438 s, 44.9 MB/s 2024-04-26T23:51:50.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.392 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261994 s, 40.0 MB/s 2024-04-26T23:51:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222324 s, 47.2 MB/s 2024-04-26T23:51:50.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:50.433 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:50.433 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:50.434 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.20 2024-04-26T23:51:50.434 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:50.434 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:50.434 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:50.434 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021876 s, 47.9 MB/s 2024-04-26T23:51:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236159 s, 44.4 MB/s 2024-04-26T23:51:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237927 s, 44.1 MB/s 2024-04-26T23:51:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231986 s, 45.2 MB/s 2024-04-26T23:51:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304701 s, 34.4 MB/s 2024-04-26T23:51:50.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.733 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228185 s, 46.0 MB/s 2024-04-26T23:51:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291994 s, 35.9 MB/s 2024-04-26T23:51:50.795 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.795 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.795 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219123 s, 47.9 MB/s 2024-04-26T23:51:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021117 s, 49.7 MB/s 2024-04-26T23:51:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231739 s, 45.2 MB/s 2024-04-26T23:51:50.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.880 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.880 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203307 s, 51.6 MB/s 2024-04-26T23:51:50.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.909 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222363 s, 47.2 MB/s 2024-04-26T23:51:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334661 s, 31.3 MB/s 2024-04-26T23:51:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:50.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:50.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255012 s, 41.1 MB/s 2024-04-26T23:51:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221517 s, 47.3 MB/s 2024-04-26T23:51:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022838 s, 45.9 MB/s 2024-04-26T23:51:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214134 s, 49.0 MB/s 2024-04-26T23:51:51.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257486 s, 40.7 MB/s 2024-04-26T23:51:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269529 s, 38.9 MB/s 2024-04-26T23:51:51.154 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.155 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.155 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199129 s, 52.7 MB/s 2024-04-26T23:51:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210078 s, 49.9 MB/s 2024-04-26T23:51:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229886 s, 45.6 MB/s 2024-04-26T23:51:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208309 s, 50.3 MB/s 2024-04-26T23:51:51.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200508 s, 52.3 MB/s 2024-04-26T23:51:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332642 s, 31.5 MB/s 2024-04-26T23:51:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276917 s, 37.9 MB/s 2024-04-26T23:51:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027673 s, 37.9 MB/s 2024-04-26T23:51:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211854 s, 49.5 MB/s 2024-04-26T23:51:51.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.423 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204537 s, 51.3 MB/s 2024-04-26T23:51:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224446 s, 46.7 MB/s 2024-04-26T23:51:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272416 s, 38.5 MB/s 2024-04-26T23:51:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221359 s, 47.4 MB/s 2024-04-26T23:51:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.21 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:51.526 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:51.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251337 s, 41.7 MB/s 2024-04-26T23:51:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233271 s, 45.0 MB/s 2024-04-26T23:51:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020223 s, 51.9 MB/s 2024-04-26T23:51:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031068 s, 33.8 MB/s 2024-04-26T23:51:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276468 s, 37.9 MB/s 2024-04-26T23:51:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220659 s, 47.5 MB/s 2024-04-26T23:51:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231549 s, 45.3 MB/s 2024-04-26T23:51:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356232 s, 29.4 MB/s 2024-04-26T23:51:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026248 s, 39.9 MB/s 2024-04-26T23:51:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252817 s, 41.5 MB/s 2024-04-26T23:51:52.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.005 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273223 s, 38.4 MB/s 2024-04-26T23:51:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282251 s, 37.2 MB/s 2024-04-26T23:51:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215112 s, 48.7 MB/s 2024-04-26T23:51:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265123 s, 39.6 MB/s 2024-04-26T23:51:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275807 s, 38.0 MB/s 2024-04-26T23:51:52.169 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.169 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.169 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269526 s, 38.9 MB/s 2024-04-26T23:51:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226915 s, 46.2 MB/s 2024-04-26T23:51:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250881 s, 41.8 MB/s 2024-04-26T23:51:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200879 s, 52.2 MB/s 2024-04-26T23:51:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236544 s, 44.3 MB/s 2024-04-26T23:51:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270266 s, 38.8 MB/s 2024-04-26T23:51:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296322 s, 35.4 MB/s 2024-04-26T23:51:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210048 s, 49.9 MB/s 2024-04-26T23:51:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021483 s, 48.8 MB/s 2024-04-26T23:51:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221689 s, 47.3 MB/s 2024-04-26T23:51:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187982 s, 55.8 MB/s 2024-04-26T23:51:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328716 s, 31.9 MB/s 2024-04-26T23:51:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284979 s, 36.8 MB/s 2024-04-26T23:51:52.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.568 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.568 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307502 s, 34.1 MB/s 2024-04-26T23:51:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225735 s, 46.5 MB/s 2024-04-26T23:51:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231477 s, 45.3 MB/s 2024-04-26T23:51:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212089 s, 49.4 MB/s 2024-04-26T23:51:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.22 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:52.666 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266363 s, 39.4 MB/s 2024-04-26T23:51:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264635 s, 39.6 MB/s 2024-04-26T23:51:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236606 s, 44.3 MB/s 2024-04-26T23:51:52.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.912 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.912 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286705 s, 36.6 MB/s 2024-04-26T23:51:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022277 s, 47.1 MB/s 2024-04-26T23:51:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233298 s, 44.9 MB/s 2024-04-26T23:51:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272065 s, 38.5 MB/s 2024-04-26T23:51:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.034 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267568 s, 39.2 MB/s 2024-04-26T23:51:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202774 s, 51.7 MB/s 2024-04-26T23:51:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252802 s, 41.5 MB/s 2024-04-26T23:51:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292458 s, 35.9 MB/s 2024-04-26T23:51:53.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.158 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.158 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267748 s, 39.2 MB/s 2024-04-26T23:51:53.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.189 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245366 s, 42.7 MB/s 2024-04-26T23:51:53.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.218 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230981 s, 45.4 MB/s 2024-04-26T23:51:53.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.249 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255238 s, 41.1 MB/s 2024-04-26T23:51:53.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.277 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.277 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223441 s, 46.9 MB/s 2024-04-26T23:51:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220998 s, 47.4 MB/s 2024-04-26T23:51:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018487 s, 56.7 MB/s 2024-04-26T23:51:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272558 s, 38.5 MB/s 2024-04-26T23:51:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231168 s, 45.4 MB/s 2024-04-26T23:51:53.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.423 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.423 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270735 s, 38.7 MB/s 2024-04-26T23:51:53.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250629 s, 41.8 MB/s 2024-04-26T23:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221865 s, 47.3 MB/s 2024-04-26T23:51:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283752 s, 37.0 MB/s 2024-04-26T23:51:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240476 s, 43.6 MB/s 2024-04-26T23:51:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276132 s, 38.0 MB/s 2024-04-26T23:51:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206801 s, 50.7 MB/s 2024-04-26T23:51:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225637 s, 46.5 MB/s 2024-04-26T23:51:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176329 s, 59.5 MB/s 2024-04-26T23:51:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291065 s, 36.0 MB/s 2024-04-26T23:51:53.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254969 s, 41.1 MB/s 2024-04-26T23:51:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252014 s, 41.6 MB/s 2024-04-26T23:51:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:53.770 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:53.770 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:53.770 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.23 2024-04-26T23:51:53.770 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:53.771 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:53.771 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:53.771 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:53.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:53.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300158 s, 34.9 MB/s 2024-04-26T23:51:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304295 s, 34.5 MB/s 2024-04-26T23:51:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270191 s, 38.8 MB/s 2024-04-26T23:51:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304822 s, 34.4 MB/s 2024-04-26T23:51:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.057 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.057 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223116 s, 47.0 MB/s 2024-04-26T23:51:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240264 s, 43.6 MB/s 2024-04-26T23:51:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211056 s, 49.7 MB/s 2024-04-26T23:51:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209675 s, 50.0 MB/s 2024-04-26T23:51:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305701 s, 34.3 MB/s 2024-04-26T23:51:54.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315613 s, 33.2 MB/s 2024-04-26T23:51:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233711 s, 44.9 MB/s 2024-04-26T23:51:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018947 s, 55.3 MB/s 2024-04-26T23:51:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266006 s, 39.4 MB/s 2024-04-26T23:51:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219145 s, 47.8 MB/s 2024-04-26T23:51:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195125 s, 53.7 MB/s 2024-04-26T23:51:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233543 s, 44.9 MB/s 2024-04-26T23:51:54.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.417 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249034 s, 42.1 MB/s 2024-04-26T23:51:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228434 s, 45.9 MB/s 2024-04-26T23:51:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224375 s, 46.7 MB/s 2024-04-26T23:51:54.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.504 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238672 s, 43.9 MB/s 2024-04-26T23:51:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242264 s, 43.3 MB/s 2024-04-26T23:51:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297276 s, 35.3 MB/s 2024-04-26T23:51:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231429 s, 45.3 MB/s 2024-04-26T23:51:54.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.625 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197827 s, 53.0 MB/s 2024-04-26T23:51:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247581 s, 42.4 MB/s 2024-04-26T23:51:54.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.683 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215797 s, 48.6 MB/s 2024-04-26T23:51:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272585 s, 38.5 MB/s 2024-04-26T23:51:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.754 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307333 s, 34.1 MB/s 2024-04-26T23:51:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246417 s, 42.6 MB/s 2024-04-26T23:51:54.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272889 s, 38.4 MB/s 2024-04-26T23:51:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262787 s, 39.9 MB/s 2024-04-26T23:51:54.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:54.885 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:54.885 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256929 s, 40.8 MB/s 2024-04-26T23:51:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:54.898 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.24 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:54.899 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:55.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:55.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019479 s, 53.8 MB/s 2024-04-26T23:51:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249717 s, 42.0 MB/s 2024-04-26T23:51:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253926 s, 41.3 MB/s 2024-04-26T23:51:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281977 s, 37.2 MB/s 2024-04-26T23:51:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229332 s, 45.7 MB/s 2024-04-26T23:51:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215137 s, 48.7 MB/s 2024-04-26T23:51:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259279 s, 40.4 MB/s 2024-04-26T23:51:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215895 s, 48.6 MB/s 2024-04-26T23:51:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.279 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.279 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236129 s, 44.4 MB/s 2024-04-26T23:51:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274104 s, 38.3 MB/s 2024-04-26T23:51:55.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.341 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243861 s, 43.0 MB/s 2024-04-26T23:51:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252978 s, 41.4 MB/s 2024-04-26T23:51:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308342 s, 34.0 MB/s 2024-04-26T23:51:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215269 s, 48.7 MB/s 2024-04-26T23:51:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213287 s, 49.2 MB/s 2024-04-26T23:51:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230814 s, 45.4 MB/s 2024-04-26T23:51:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259962 s, 40.3 MB/s 2024-04-26T23:51:55.556 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.556 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219545 s, 47.8 MB/s 2024-04-26T23:51:55.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.591 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293897 s, 35.7 MB/s 2024-04-26T23:51:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02882 s, 36.4 MB/s 2024-04-26T23:51:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217079 s, 48.3 MB/s 2024-04-26T23:51:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222035 s, 47.2 MB/s 2024-04-26T23:51:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213563 s, 49.1 MB/s 2024-04-26T23:51:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249754 s, 42.0 MB/s 2024-04-26T23:51:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207925 s, 50.4 MB/s 2024-04-26T23:51:55.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.803 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309371 s, 33.9 MB/s 2024-04-26T23:51:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02164 s, 48.5 MB/s 2024-04-26T23:51:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323552 s, 32.4 MB/s 2024-04-26T23:51:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238084 s, 44.0 MB/s 2024-04-26T23:51:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232054 s, 45.2 MB/s 2024-04-26T23:51:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239738 s, 43.7 MB/s 2024-04-26T23:51:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209322 s, 50.1 MB/s 2024-04-26T23:51:55.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:55.997 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:55.997 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:55.997 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.25 2024-04-26T23:51:55.997 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:55.998 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:55.998 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:55.998 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:56.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.149 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275946 s, 38.0 MB/s 2024-04-26T23:51:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229749 s, 45.6 MB/s 2024-04-26T23:51:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224874 s, 46.6 MB/s 2024-04-26T23:51:56.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248777 s, 42.1 MB/s 2024-04-26T23:51:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310792 s, 33.7 MB/s 2024-04-26T23:51:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221057 s, 47.4 MB/s 2024-04-26T23:51:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308528 s, 34.0 MB/s 2024-04-26T23:51:56.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.379 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323985 s, 32.4 MB/s 2024-04-26T23:51:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240818 s, 43.5 MB/s 2024-04-26T23:51:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297347 s, 35.3 MB/s 2024-04-26T23:51:56.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.471 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192451 s, 54.5 MB/s 2024-04-26T23:51:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299708 s, 35.0 MB/s 2024-04-26T23:51:56.542 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028532 s, 36.8 MB/s 2024-04-26T23:51:56.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.578 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.578 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290222 s, 36.1 MB/s 2024-04-26T23:51:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235757 s, 44.5 MB/s 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240928 s, 43.5 MB/s 2024-04-26T23:51:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285832 s, 36.7 MB/s 2024-04-26T23:51:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308795 s, 34.0 MB/s 2024-04-26T23:51:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025691 s, 40.8 MB/s 2024-04-26T23:51:56.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268175 s, 39.1 MB/s 2024-04-26T23:51:56.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237506 s, 44.1 MB/s 2024-04-26T23:51:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252513 s, 41.5 MB/s 2024-04-26T23:51:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0351593 s, 29.8 MB/s 2024-04-26T23:51:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246475 s, 42.5 MB/s 2024-04-26T23:51:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295087 s, 35.5 MB/s 2024-04-26T23:51:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374794 s, 28.0 MB/s 2024-04-26T23:51:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251716 s, 41.7 MB/s 2024-04-26T23:51:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263784 s, 39.8 MB/s 2024-04-26T23:51:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219072 s, 47.9 MB/s 2024-04-26T23:51:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226549 s, 46.3 MB/s 2024-04-26T23:51:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229283 s, 45.7 MB/s 2024-04-26T23:51:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206845 s, 50.7 MB/s 2024-04-26T23:51:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.26 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:57.178 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:57.179 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252919 s, 41.5 MB/s 2024-04-26T23:51:57.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.359 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238798 s, 43.9 MB/s 2024-04-26T23:51:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227409 s, 46.1 MB/s 2024-04-26T23:51:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022315 s, 47.0 MB/s 2024-04-26T23:51:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218025 s, 48.1 MB/s 2024-04-26T23:51:57.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246674 s, 42.5 MB/s 2024-04-26T23:51:57.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.509 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.509 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282936 s, 37.1 MB/s 2024-04-26T23:51:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305457 s, 34.3 MB/s 2024-04-26T23:51:57.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.577 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249524 s, 42.0 MB/s 2024-04-26T23:51:57.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019278 s, 54.4 MB/s 2024-04-26T23:51:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231175 s, 45.4 MB/s 2024-04-26T23:51:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242701 s, 43.2 MB/s 2024-04-26T23:51:57.697 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.697 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.698 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264737 s, 39.6 MB/s 2024-04-26T23:51:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226972 s, 46.2 MB/s 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245153 s, 42.8 MB/s 2024-04-26T23:51:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273722 s, 38.3 MB/s 2024-04-26T23:51:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246276 s, 42.6 MB/s 2024-04-26T23:51:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253195 s, 41.4 MB/s 2024-04-26T23:51:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257232 s, 40.8 MB/s 2024-04-26T23:51:57.915 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.916 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221243 s, 47.4 MB/s 2024-04-26T23:51:57.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282396 s, 37.1 MB/s 2024-04-26T23:51:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:57.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263297 s, 39.8 MB/s 2024-04-26T23:51:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023016 s, 45.6 MB/s 2024-04-26T23:51:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271553 s, 38.6 MB/s 2024-04-26T23:51:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239959 s, 43.7 MB/s 2024-04-26T23:51:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229032 s, 45.8 MB/s 2024-04-26T23:51:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298702 s, 35.1 MB/s 2024-04-26T23:51:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208252 s, 50.4 MB/s 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282416 s, 37.1 MB/s 2024-04-26T23:51:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233884 s, 44.8 MB/s 2024-04-26T23:51:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216358 s, 48.5 MB/s 2024-04-26T23:51:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233292 s, 44.9 MB/s 2024-04-26T23:51:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.27 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:58.299 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:58.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:58.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021888 s, 47.9 MB/s 2024-04-26T23:51:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186619 s, 56.2 MB/s 2024-04-26T23:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329205 s, 31.9 MB/s 2024-04-26T23:51:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232046 s, 45.2 MB/s 2024-04-26T23:51:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254767 s, 41.2 MB/s 2024-04-26T23:51:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270189 s, 38.8 MB/s 2024-04-26T23:51:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248678 s, 42.2 MB/s 2024-04-26T23:51:58.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.668 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.668 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240937 s, 43.5 MB/s 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251118 s, 41.8 MB/s 2024-04-26T23:51:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277394 s, 37.8 MB/s 2024-04-26T23:51:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243445 s, 43.1 MB/s 2024-04-26T23:51:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284332 s, 36.9 MB/s 2024-04-26T23:51:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240897 s, 43.5 MB/s 2024-04-26T23:51:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255464 s, 41.0 MB/s 2024-04-26T23:51:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240268 s, 43.6 MB/s 2024-04-26T23:51:58.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.927 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306431 s, 34.2 MB/s 2024-04-26T23:51:58.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025057 s, 41.8 MB/s 2024-04-26T23:51:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254738 s, 41.2 MB/s 2024-04-26T23:51:59.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.019 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248589 s, 42.2 MB/s 2024-04-26T23:51:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287991 s, 36.4 MB/s 2024-04-26T23:51:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308654 s, 34.0 MB/s 2024-04-26T23:51:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025233 s, 41.6 MB/s 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319812 s, 32.8 MB/s 2024-04-26T23:51:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321599 s, 32.6 MB/s 2024-04-26T23:51:59.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.238 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309722 s, 33.9 MB/s 2024-04-26T23:51:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238341 s, 44.0 MB/s 2024-04-26T23:51:59.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218642 s, 48.0 MB/s 2024-04-26T23:51:59.325 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.326 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215473 s, 48.7 MB/s 2024-04-26T23:51:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186227 s, 56.3 MB/s 2024-04-26T23:51:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173556 s, 60.4 MB/s 2024-04-26T23:51:59.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.402 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.402 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226796 s, 46.2 MB/s 2024-04-26T23:51:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243983 s, 43.0 MB/s 2024-04-26T23:51:59.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:59.445 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:51:59.445 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:51:59.445 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.28 2024-04-26T23:51:59.445 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:51:59.445 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:51:59.446 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:51:59.446 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:51:59.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:51:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:59.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281401 s, 37.3 MB/s 2024-04-26T23:51:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286093 s, 36.7 MB/s 2024-04-26T23:51:59.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.661 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214737 s, 48.8 MB/s 2024-04-26T23:51:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021111 s, 49.7 MB/s 2024-04-26T23:51:59.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.727 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.727 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310136 s, 33.8 MB/s 2024-04-26T23:51:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220891 s, 47.5 MB/s 2024-04-26T23:51:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217293 s, 48.3 MB/s 2024-04-26T23:51:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292125 s, 35.9 MB/s 2024-04-26T23:51:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229046 s, 45.8 MB/s 2024-04-26T23:51:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218334 s, 48.0 MB/s 2024-04-26T23:51:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221131 s, 47.4 MB/s 2024-04-26T23:51:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254235 s, 41.2 MB/s 2024-04-26T23:51:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:51:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:51:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252323 s, 41.6 MB/s 2024-04-26T23:52:00.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.003 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222303 s, 47.2 MB/s 2024-04-26T23:52:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296181 s, 35.4 MB/s 2024-04-26T23:52:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259574 s, 40.4 MB/s 2024-04-26T23:52:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020145 s, 52.1 MB/s 2024-04-26T23:52:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256113 s, 40.9 MB/s 2024-04-26T23:52:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.173 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375226 s, 27.9 MB/s 2024-04-26T23:52:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02213 s, 47.4 MB/s 2024-04-26T23:52:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255061 s, 41.1 MB/s 2024-04-26T23:52:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220375 s, 47.6 MB/s 2024-04-26T23:52:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201407 s, 52.1 MB/s 2024-04-26T23:52:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221844 s, 47.3 MB/s 2024-04-26T23:52:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283019 s, 37.0 MB/s 2024-04-26T23:52:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024945 s, 42.0 MB/s 2024-04-26T23:52:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248853 s, 42.1 MB/s 2024-04-26T23:52:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02294 s, 45.7 MB/s 2024-04-26T23:52:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254559 s, 41.2 MB/s 2024-04-26T23:52:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219612 s, 47.7 MB/s 2024-04-26T23:52:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241648 s, 43.4 MB/s 2024-04-26T23:52:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289338 s, 36.2 MB/s 2024-04-26T23:52:00.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.29 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:00.577 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:00.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236469 s, 44.3 MB/s 2024-04-26T23:52:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212498 s, 49.3 MB/s 2024-04-26T23:52:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244585 s, 42.9 MB/s 2024-04-26T23:52:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294212 s, 35.6 MB/s 2024-04-26T23:52:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181023 s, 57.9 MB/s 2024-04-26T23:52:00.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.871 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.871 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226135 s, 46.4 MB/s 2024-04-26T23:52:00.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.900 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.900 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235965 s, 44.4 MB/s 2024-04-26T23:52:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218349 s, 48.0 MB/s 2024-04-26T23:52:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240258 s, 43.6 MB/s 2024-04-26T23:52:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253017 s, 41.4 MB/s 2024-04-26T23:52:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224469 s, 46.7 MB/s 2024-04-26T23:52:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030391 s, 34.5 MB/s 2024-04-26T23:52:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221339 s, 47.4 MB/s 2024-04-26T23:52:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212881 s, 49.3 MB/s 2024-04-26T23:52:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02854 s, 36.7 MB/s 2024-04-26T23:52:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222794 s, 47.1 MB/s 2024-04-26T23:52:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261558 s, 40.1 MB/s 2024-04-26T23:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299935 s, 35.0 MB/s 2024-04-26T23:52:01.268 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.268 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.268 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210795 s, 49.7 MB/s 2024-04-26T23:52:01.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292623 s, 35.8 MB/s 2024-04-26T23:52:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227025 s, 46.2 MB/s 2024-04-26T23:52:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.450 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278427 s, 37.7 MB/s 2024-04-26T23:52:01.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.450 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.450 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250576 s, 41.8 MB/s 2024-04-26T23:52:01.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.456 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311361 s, 33.7 MB/s 2024-04-26T23:52:01.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.487 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261322 s, 40.1 MB/s 2024-04-26T23:52:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023708 s, 44.2 MB/s 2024-04-26T23:52:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230889 s, 45.4 MB/s 2024-04-26T23:52:01.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239711 s, 43.7 MB/s 2024-04-26T23:52:01.604 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.605 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.605 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232506 s, 45.1 MB/s 2024-04-26T23:52:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223482 s, 46.9 MB/s 2024-04-26T23:52:01.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.663 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229935 s, 45.6 MB/s 2024-04-26T23:52:01.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.691 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.691 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228169 s, 46.0 MB/s 2024-04-26T23:52:01.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:01.703 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.30 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:01.704 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:01.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025386 s, 41.3 MB/s 2024-04-26T23:52:01.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.888 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.888 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271632 s, 38.6 MB/s 2024-04-26T23:52:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028959 s, 36.2 MB/s 2024-04-26T23:52:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222262 s, 47.2 MB/s 2024-04-26T23:52:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222615 s, 47.1 MB/s 2024-04-26T23:52:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241929 s, 43.3 MB/s 2024-04-26T23:52:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238497 s, 44.0 MB/s 2024-04-26T23:52:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230981 s, 45.4 MB/s 2024-04-26T23:52:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.101 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258609 s, 40.5 MB/s 2024-04-26T23:52:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276843 s, 37.9 MB/s 2024-04-26T23:52:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238972 s, 43.9 MB/s 2024-04-26T23:52:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245212 s, 42.8 MB/s 2024-04-26T23:52:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235265 s, 44.6 MB/s 2024-04-26T23:52:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274021 s, 38.3 MB/s 2024-04-26T23:52:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227285 s, 46.1 MB/s 2024-04-26T23:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226838 s, 46.2 MB/s 2024-04-26T23:52:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228434 s, 45.9 MB/s 2024-04-26T23:52:02.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.371 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212014 s, 49.5 MB/s 2024-04-26T23:52:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204007 s, 51.4 MB/s 2024-04-26T23:52:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026028 s, 40.3 MB/s 2024-04-26T23:52:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282118 s, 37.2 MB/s 2024-04-26T23:52:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023225 s, 45.1 MB/s 2024-04-26T23:52:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207352 s, 50.6 MB/s 2024-04-26T23:52:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278335 s, 37.7 MB/s 2024-04-26T23:52:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232957 s, 45.0 MB/s 2024-04-26T23:52:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295814 s, 35.4 MB/s 2024-04-26T23:52:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294799 s, 35.6 MB/s 2024-04-26T23:52:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278043 s, 37.7 MB/s 2024-04-26T23:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236412 s, 44.4 MB/s 2024-04-26T23:52:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220749 s, 47.5 MB/s 2024-04-26T23:52:02.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.782 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285385 s, 36.7 MB/s 2024-04-26T23:52:02.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.814 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251951 s, 41.6 MB/s 2024-04-26T23:52:02.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:02.826 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:02.826 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:02.826 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.31 2024-04-26T23:52:02.826 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:02.827 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:02.827 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:02.827 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:02.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296714 s, 35.3 MB/s 2024-04-26T23:52:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029066 s, 36.1 MB/s 2024-04-26T23:52:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248677 s, 42.2 MB/s 2024-04-26T23:52:03.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.074 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229742 s, 45.6 MB/s 2024-04-26T23:52:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030501 s, 34.4 MB/s 2024-04-26T23:52:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236776 s, 44.3 MB/s 2024-04-26T23:52:03.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.175 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256488 s, 40.9 MB/s 2024-04-26T23:52:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301626 s, 34.8 MB/s 2024-04-26T23:52:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217228 s, 48.3 MB/s 2024-04-26T23:52:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328154 s, 32.0 MB/s 2024-04-26T23:52:03.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292392 s, 35.9 MB/s 2024-04-26T23:52:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304792 s, 34.4 MB/s 2024-04-26T23:52:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218749 s, 47.9 MB/s 2024-04-26T23:52:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255453 s, 41.0 MB/s 2024-04-26T23:52:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209867 s, 50.0 MB/s 2024-04-26T23:52:03.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.466 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.466 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237819 s, 44.1 MB/s 2024-04-26T23:52:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208642 s, 50.3 MB/s 2024-04-26T23:52:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224451 s, 46.7 MB/s 2024-04-26T23:52:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230883 s, 45.4 MB/s 2024-04-26T23:52:03.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020597 s, 50.9 MB/s 2024-04-26T23:52:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207457 s, 50.5 MB/s 2024-04-26T23:52:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248556 s, 42.2 MB/s 2024-04-26T23:52:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263676 s, 39.8 MB/s 2024-04-26T23:52:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250246 s, 41.9 MB/s 2024-04-26T23:52:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275446 s, 38.1 MB/s 2024-04-26T23:52:03.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.766 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308241 s, 34.0 MB/s 2024-04-26T23:52:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261715 s, 40.1 MB/s 2024-04-26T23:52:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227556 s, 46.1 MB/s 2024-04-26T23:52:03.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.859 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.860 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237141 s, 44.2 MB/s 2024-04-26T23:52:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02827 s, 37.1 MB/s 2024-04-26T23:52:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233576 s, 44.9 MB/s 2024-04-26T23:52:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246584 s, 42.5 MB/s 2024-04-26T23:52:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:03.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:03.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub start /scrub_control_test_path recursive 2024-04-26T23:52:04.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.178+0000 7f9bd5a00640 1 Processor -- start 2024-04-26T23:52:04.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.178+0000 7f9bd5a00640 1 -- start start 2024-04-26T23:52:04.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd00b64a0 0x7f9bd00b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9bd00d1d90 0x7f9bd00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd00d0790 con 0x7f9bd01628e0 2024-04-26T23:52:04.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd00b4510 con 0x7f9bd00d1d90 2024-04-26T23:52:04.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd00d0a70 con 0x7f9bd00b64a0 2024-04-26T23:52:04.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bcfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9bd01628e0 0x7f9bd00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54896/0 (socket says 172.21.15.46:54896) 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bcfe00640 1 -- 172.21.15.46:0/2603026253 learned_addr learned my addr 172.21.15.46:0/2603026253 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd00b64a0 0x7f9bd00b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.180+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2603026253 >> v2:172.21.15.142:3300/0 conn(0x7f9bd00d1d90 0x7f9bd00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bcf400640 1 -- 172.21.15.46:0/2603026253 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029386077 0 0) 0x7f9bd00d0790 con 0x7f9bd01628e0 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bcf400640 1 -- 172.21.15.46:0/2603026253 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bb4003520 con 0x7f9bd01628e0 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bd5000640 1 -- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd00b64a0 msgr2=0x7f9bd00b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd00b64a0 0x7f9bd00b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bd5000640 1 -- 172.21.15.46:0/2603026253 >> v1:172.21.15.46:6789/0 conn(0x7f9bd01628e0 legacy=0x7f9bd00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.181+0000 7f9bd5000640 1 -- 172.21.15.46:0/2603026253 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd00d0d50 con 0x7f9bd00d1d90 2024-04-26T23:52:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.182+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd00d1d90 0x7f9bd00b4ee0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f2623d74594c46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.182+0000 7f9bcf400640 1 -- 172.21.15.46:0/2603026253 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8031600 con 0x7f9bd00d1d90 2024-04-26T23:52:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bcf400640 1 -- 172.21.15.46:0/2603026253 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc80317a0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bcf400640 1 -- 172.21.15.46:0/2603026253 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8031b50 con 0x7f9bd00d1d90 2024-04-26T23:52:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 -- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd00d1d90 msgr2=0x7f9bd00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd00d1d90 0x7f9bd00b4ee0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc802a5d0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 -- 172.21.15.46:0/2603026253 shutdown_connections 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd00d1d90 0x7f9bd00b4ee0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/2603026253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd00b64a0 0x7f9bd00b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 -- 172.21.15.46:0/2603026253 >> 172.21.15.46:0/2603026253 conn(0x7f9bd00d4e90 msgr2=0x7f9bd00d52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 -- 172.21.15.46:0/2603026253 shutdown_connections 2024-04-26T23:52:04.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.183+0000 7f9bd5a00640 1 -- 172.21.15.46:0/2603026253 wait complete. 2024-04-26T23:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.185+0000 7f9bd5a00640 1 Processor -- start 2024-04-26T23:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.185+0000 7f9bd5a00640 1 -- start start 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.186+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.186+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd01628e0 0x7f9bd0160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.186+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd01552b0 0x7f9bd00b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bd5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd00d59b0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bd5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0162f50 con 0x7f9bd01552b0 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bd5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd00d5b30 con 0x7f9bd01628e0 2024-04-26T23:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47728/0 (socket says 172.21.15.46:47728) 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 -- 172.21.15.46:0/51844251 learned_addr learned my addr 172.21.15.46:0/51844251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd01628e0 0x7f9bd0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bd5000640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd01552b0 0x7f9bd00b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 -- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd01628e0 msgr2=0x7f9bd0160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd01628e0 0x7f9bd0160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 -- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd01552b0 msgr2=0x7f9bd00b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd01552b0 0x7f9bd00b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.187+0000 7f9bcfe00640 1 -- 172.21.15.46:0/51844251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd0161010 con 0x7f9bd00d1d90 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c460 tx=0x7f9bc000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bccc00640 1 -- 172.21.15.46:0/51844251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd00d1d90 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bccc00640 1 -- 172.21.15.46:0/51844251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc0005ce0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01612f0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bccc00640 1 -- 172.21.15.46:0/51844251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0007290 con 0x7f9bd00d1d90 2024-04-26T23:52:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.188+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01618b0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.191+0000 7f9bccc00640 1 -- 172.21.15.46:0/51844251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bc001a070 con 0x7f9bd00d1d90 2024-04-26T23:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.192+0000 7f9bccc00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.192+0000 7f9bccc00640 1 -- 172.21.15.46:0/51844251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f9bc008afe0 con 0x7f9bd00d1d90 2024-04-26T23:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.192+0000 7f9bd9fd5740 1 Processor -- start 2024-04-26T23:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.192+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.192+0000 7f9bd9fd5740 1 -- start start 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff828580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff8580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556ff791f90 con 0x5556ff858620 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5556ff72e1e0 con 0x5556ff875c40 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5556ff78df50 con 0x5556ff858d60 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.193+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 0x7f9bac06ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b5640 tx=0x7f9bb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47732/0 (socket says 172.21.15.46:47732) 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 -- 172.21.15.46:0/3263132727 learned_addr learned my addr 172.21.15.46:0/3263132727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff828580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff8580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 -- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 msgr2=0x5556ff8580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff8580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 -- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 msgr2=0x5556ff828580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff828580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bd5000640 1 -- 172.21.15.46:0/3263132727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ff56fba0 con 0x5556ff858620 2024-04-26T23:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.194+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff8580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.195+0000 7f9bd5000640 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009e80 tx=0x7f9bc802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe0fae6d76e24b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.195+0000 7f9bc4c00640 1 -- 172.21.15.46:0/3263132727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8041070 con 0x5556ff858620 2024-04-26T23:52:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.195+0000 7f9bc4c00640 1 -- 172.21.15.46:0/3263132727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc8003b90 con 0x5556ff858620 2024-04-26T23:52:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.195+0000 7f9bc4c00640 1 -- 172.21.15.46:0/3263132727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8004040 con 0x5556ff858620 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/3263132727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 msgr2=0x5556ff82d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009e80 tx=0x7f9bc802e000 comp rx=0 tx=0).stop 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/3263132727 shutdown_connections 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858620 0x5556ff82d430 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff8580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/3263132727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff828580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/3263132727 >> 172.21.15.46:0/3263132727 conn(0x5556ff82a840 msgr2=0x5556ff8730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/3263132727 shutdown_connections 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.196+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/3263132727 wait complete. 2024-04-26T23:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.197+0000 7f9bd9fd5740 1 -- start start 2024-04-26T23:52:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.197+0000 7f9bd9fd5740 1 Processor -- start 2024-04-26T23:52:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff872020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff869da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556ff82c9f0 con 0x5556ff86a2e0 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5556ff718da0 con 0x5556ff875c40 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5556ff82bde0 con 0x5556ff858d60 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd9fd5740 10 client.0 _ll_register_callbacks cb 0x5556ff745400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47736/0 (socket says 172.21.15.46:47736) 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.198+0000 7f9bd5000640 1 -- 172.21.15.46:0/2707451068 learned_addr learned my addr 172.21.15.46:0/2707451068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff869da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff872020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 -- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 msgr2=0x5556ff872020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff872020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 -- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 msgr2=0x5556ff869da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff869da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 -- 172.21.15.46:0/2707451068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ff463490 con 0x5556ff86a2e0 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff872020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff869da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bd5000640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f9bc802a610 tx=0x7f9bc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe2c95264962af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.199+0000 7f9bbd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8041040 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc8003b90 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f9bc8003990 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f9bc8047da0 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bd9fd5740 10 client.9819 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x5556ff5d0330 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc8003b90 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.200+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f9bc8051470 con 0x5556ff86a2e0 2024-04-26T23:52:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 10 client.9819 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 10 client.9819 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 20 client.9819 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff859cc0 0x5556ff85a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 4 client.9819 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ff79fba0 con 0x5556ff859cc0 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff859cc0 0x5556ff85a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff859cc0 0x5556ff85a140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81e468b96dc8076 server_cookie=bb9d53493d1b7d62 in_seq=0 out_seq=0 2024-04-26T23:52:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.201+0000 7f9bbd600640 10 client.9819 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.204+0000 7f9bbd600640 1 -- 172.21.15.46:0/2707451068 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556ff79fba0 con 0x5556ff859cc0 2024-04-26T23:52:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.204+0000 7f9bbd600640 10 client.9819 handle_command_reply: tid=0 2024-04-26T23:52:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.204+0000 7f9bd9fd5740 1 client.9819 shutdown 2024-04-26T23:52:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.204+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 msgr2=0x5556ff86ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.204+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f9bc802a610 tx=0x7f9bc8004ab0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 shutdown_connections 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff859cc0 0x5556ff85a140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff86a2e0 0x5556ff86ba80 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff869da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/2707451068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff858d60 0x5556ff872020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bbd600640 0 client.9819 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 >> 172.21.15.46:0/2707451068 conn(0x5556ff858620 msgr2=0x5556ff8730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 shutdown_connections 2024-04-26T23:52:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/2707451068 wait complete. 2024-04-26T23:52:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.205+0000 7f9bd9fd5740 20 client.9819 trim_cache size 0 max 16384 2024-04-26T23:52:04.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.225+0000 7f9bd9fd5740 1 Processor -- start 2024-04-26T23:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.225+0000 7f9bd9fd5740 1 -- start start 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff82ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff82e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0201ab0 con 0x5556ff858d60 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd015d9e0 con 0x5556ff875c40 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd01f9710 con 0x5556ff86a2e0 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47746/0 (socket says 172.21.15.46:47746) 2024-04-26T23:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.226+0000 7f9bcfe00640 1 -- 172.21.15.46:0/540668247 learned_addr learned my addr 172.21.15.46:0/540668247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 -- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 msgr2=0x5556ff82ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff82ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 -- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 msgr2=0x5556ff82e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff82e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 -- 172.21.15.46:0/540668247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ff853a90 con 0x5556ff858d60 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9bc005b010 tx=0x7f9bc0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e403b88d50873bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bbfe00640 1 -- 172.21.15.46:0/540668247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0056ea0 con 0x5556ff858d60 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bbfe00640 1 -- 172.21.15.46:0/540668247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc0055210 con 0x5556ff858d60 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.227+0000 7f9bbfe00640 1 -- 172.21.15.46:0/540668247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0061530 con 0x5556ff858d60 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/540668247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 msgr2=0x5556ff872ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9bc005b010 tx=0x7f9bc0000bc0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/540668247 shutdown_connections 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff875c40 0x5556ff82e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff82ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/540668247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff858d60 0x5556ff872ee0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/540668247 >> 172.21.15.46:0/540668247 conn(0x5556ff858620 msgr2=0x5556ff828430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/540668247 shutdown_connections 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.228+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/540668247 wait complete. 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.229+0000 7f9bd9fd5740 1 -- start start 2024-04-26T23:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.229+0000 7f9bd9fd5740 1 Processor -- start 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.230+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff858d60 0x5556ff846970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.230+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff869310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.230+0000 7f9bd9fd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.230+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556ff79d7d0 con 0x5556ff875c40 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd015d390 con 0x5556ff858d60 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd9fd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5556ff873a20 con 0x5556ff86a2e0 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd9fd5740 10 client.0 _ll_register_callbacks cb 0x7f9bb8009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47750/0 (socket says 172.21.15.46:47750) 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bd5000640 1 -- 172.21.15.46:0/526425291 learned_addr learned my addr 172.21.15.46:0/526425291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff869310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.231+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff858d60 0x5556ff846970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 -- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 msgr2=0x5556ff869310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff869310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 -- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff858d60 msgr2=0x5556ff846970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff858d60 0x5556ff846970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 -- 172.21.15.46:0/526425291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ff43c610 con 0x5556ff875c40 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bd5000640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8051c70 tx=0x7f9bc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2afbdeed5151a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.232+0000 7f9bbd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8041040 con 0x5556ff875c40 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc0055210 con 0x5556ff875c40 2024-04-26T23:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f9bc804d050 con 0x5556ff875c40 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f9bc8069050 con 0x5556ff875c40 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bd9fd5740 10 client.9825 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.233+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x5556ff849c90 con 0x5556ff875c40 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc0055210 con 0x5556ff875c40 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f9bc806c020 con 0x5556ff875c40 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 10 client.9825 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 10 client.9825 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 20 client.9825 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff85bef0 0x5556ff8508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 4 client.9825 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.234+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x5556ff86b610 con 0x5556ff85bef0 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.235+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff85bef0 0x5556ff8508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.236+0000 7f9bbd600640 10 client.9825 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.236+0000 7f9bcfe00640 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff85bef0 0x5556ff8508d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d970f9b4aa733cda server_cookie=667d65704b2953bf in_seq=0 out_seq=0 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.298+0000 7f9bbd600640 1 -- 172.21.15.46:0/526425291 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5556ff86b610 con 0x5556ff85bef0 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.298+0000 7f9bbd600640 10 client.9825 handle_command_reply: tid=0 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.298+0000 7f9bd9fd5740 1 client.9825 shutdown 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 msgr2=0x5556ff869850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8051c70 tx=0x7f9bc8004a40 comp rx=0 tx=0).stop 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 shutdown_connections 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x5556ff85bef0 0x5556ff8508d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556ff875c40 0x5556ff869850 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bbd600640 0 client.9825 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5556ff86a2e0 0x5556ff869310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 --2- 172.21.15.46:0/526425291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ff858d60 0x5556ff846970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 >> 172.21.15.46:0/526425291 conn(0x5556ff858620 msgr2=0x5556ff82ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 shutdown_connections 2024-04-26T23:52:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.299+0000 7f9bd9fd5740 1 -- 172.21.15.46:0/526425291 wait complete. 2024-04-26T23:52:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.300+0000 7f9bd9fd5740 20 client.9825 trim_cache size 0 max 16384 2024-04-26T23:52:04.305 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:52:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6bdad86a-0ba0-4ca4-bc3f-6a7717ccafe2", 2024-04-26T23:52:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:52:04.305 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 msgr2=0x7f9bac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 0x7f9bac06ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b5640 tx=0x7f9bb8008040 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 msgr2=0x7f9bd0154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c460 tx=0x7f9bc000c920 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 shutdown_connections 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bd01552b0 0x7f9bd00b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9bd01628e0 0x7f9bd0160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 --2- 172.21.15.46:0/51844251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00d1d90 0x7f9bd0154a40 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.301+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 >> 172.21.15.46:0/51844251 conn(0x7f9bd00d4e90 msgr2=0x7f9bd00b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.302+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 shutdown_connections 2024-04-26T23:52:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.302+0000 7f9bd5a00640 1 -- 172.21.15.46:0/51844251 wait complete. 2024-04-26T23:52:04.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:04.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub pause 2024-04-26T23:52:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.557+0000 7f8827e00640 1 Processor -- start 2024-04-26T23:52:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.557+0000 7f8827e00640 1 -- start start 2024-04-26T23:52:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.558+0000 7f8827e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f882015d730 0x7f88200d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.558+0000 7f8827e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 0x7f882015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.558+0000 7f8827e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882006eb90 con 0x7f882015e050 2024-04-26T23:52:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.558+0000 7f8827e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f88200b4310 con 0x7f882015d730 2024-04-26T23:52:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.558+0000 7f8827e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88200b45f0 con 0x7f88200b53b0 2024-04-26T23:52:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f882015d730 0x7f88200d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f882015e050 0x7f882006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54912/0 (socket says 172.21.15.46:54912) 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825e00640 1 -- 172.21.15.46:0/2529663492 learned_addr learned my addr 172.21.15.46:0/2529663492 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8824a00640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 0x7f882015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825400640 1 -- 172.21.15.46:0/2529663492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 msgr2=0x7f882015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825400640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 0x7f882015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825400640 1 -- 172.21.15.46:0/2529663492 >> v1:172.21.15.46:6789/0 conn(0x7f882015e050 legacy=0x7f882006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8825400640 1 -- 172.21.15.46:0/2529663492 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88200b48d0 con 0x7f882015d730 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f881fe00640 1 -- 172.21.15.46:0/2529663492 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475978469 0 0) 0x7f882006eb90 con 0x7f882015e050 2024-04-26T23:52:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.559+0000 7f8824a00640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 0x7f882015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.561+0000 7f8825400640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200d00d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f88140097a0 tx=0x7f881402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff965d0bd317e1c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.561+0000 7f881fe00640 1 -- 172.21.15.46:0/2529663492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814037070 con 0x7f882015d730 2024-04-26T23:52:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.561+0000 7f881fe00640 1 -- 172.21.15.46:0/2529663492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8814032440 con 0x7f882015d730 2024-04-26T23:52:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f881fe00640 1 -- 172.21.15.46:0/2529663492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814032810 con 0x7f882015d730 2024-04-26T23:52:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 -- 172.21.15.46:0/2529663492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 msgr2=0x7f88200d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200d00d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f88140097a0 tx=0x7f881402a180 comp rx=0 tx=0).stop 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 -- 172.21.15.46:0/2529663492 shutdown_connections 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88200b53b0 0x7f882015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 --2- 172.21.15.46:0/2529663492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200d00d0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 -- 172.21.15.46:0/2529663492 >> 172.21.15.46:0/2529663492 conn(0x7f882006c980 msgr2=0x7f88200b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 -- 172.21.15.46:0/2529663492 shutdown_connections 2024-04-26T23:52:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.562+0000 7f8827e00640 1 -- 172.21.15.46:0/2529663492 wait complete. 2024-04-26T23:52:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.564+0000 7f8827e00640 1 Processor -- start 2024-04-26T23:52:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.564+0000 7f8827e00640 1 -- start start 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.565+0000 7f8827e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 0x7f8820140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.565+0000 7f8827e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8827e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f882015e050 0x7f88200cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8827e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f882015e4c0 con 0x7f882015d730 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8827e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88200b4e90 con 0x7f88200b53b0 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8827e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88200b3ec0 con 0x7f882015e050 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47768/0 (socket says 172.21.15.46:47768) 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 -- 172.21.15.46:0/2875906563 learned_addr learned my addr 172.21.15.46:0/2875906563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8825e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f882015e050 0x7f88200cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8825400640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 0x7f8820140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 -- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f882015e050 msgr2=0x7f88200cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f882015e050 0x7f88200cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 -- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 msgr2=0x7f8820140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 0x7f8820140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8824a00640 1 -- 172.21.15.46:0/2875906563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88201f5110 con 0x7f882015d730 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.566+0000 7f8825400640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 0x7f8820140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f8824a00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c7f0 tx=0x7f880c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f881d600640 1 -- 172.21.15.46:0/2875906563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c015070 con 0x7f882015d730 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f881d600640 1 -- 172.21.15.46:0/2875906563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c007820 con 0x7f882015d730 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f881d600640 1 -- 172.21.15.46:0/2875906563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c007bd0 con 0x7f882015d730 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88200ccc50 con 0x7f882015d730 2024-04-26T23:52:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.567+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88200cd4f0 con 0x7f882015d730 2024-04-26T23:52:04.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.572+0000 7f881d600640 1 -- 172.21.15.46:0/2875906563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f880c018070 con 0x7f882015d730 2024-04-26T23:52:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.573+0000 7f881d600640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 0x7f880006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.573+0000 7f881d600640 1 -- 172.21.15.46:0/2875906563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f880c08be30 con 0x7f882015d730 2024-04-26T23:52:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.573+0000 7f882ad6a740 1 Processor -- start 2024-04-26T23:52:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.573+0000 7f882ad6a740 1 -- start start 2024-04-26T23:52:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.573+0000 7f8825400640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 0x7f880006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 0x557bbbb82d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bbba9e900 con 0x557bbbb3ac30 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557bbba3b1d0 con 0x557bbbb827f0 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557bbba9a8c0 con 0x557bbbb64f50 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47776/0 (socket says 172.21.15.46:47776) 2024-04-26T23:52:04.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 -- 172.21.15.46:0/2945843499 learned_addr learned my addr 172.21.15.46:0/2945843499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825400640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 0x7f880006edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8814034000 tx=0x7f881402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8824a00640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 0x557bbbb82d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825400640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 -- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 msgr2=0x557bbbb65330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb65330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 -- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 msgr2=0x557bbbb82d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 0x557bbbb82d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 -- 172.21.15.46:0/2945843499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbb87a7d0 con 0x557bbbb3ac30 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8824a00640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 0x557bbbb82d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8825e00640 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8818001db0 tx=0x7f881800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e099efb4b22c6032 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.574+0000 7f8811c00640 1 -- 172.21.15.46:0/2945843499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881800de30 con 0x557bbbb3ac30 2024-04-26T23:52:04.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f8811c00640 1 -- 172.21.15.46:0/2945843499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818004b60 con 0x557bbbb3ac30 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f8811c00640 1 -- 172.21.15.46:0/2945843499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88180070d0 con 0x557bbbb3ac30 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 -- 172.21.15.46:0/2945843499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 msgr2=0x557bbbb80eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8818001db0 tx=0x7f881800d520 comp rx=0 tx=0).stop 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 -- 172.21.15.46:0/2945843499 shutdown_connections 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb3ac30 0x557bbbb80eb0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb827f0 0x557bbbb82d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 --2- 172.21.15.46:0/2945843499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb65330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 -- 172.21.15.46:0/2945843499 >> 172.21.15.46:0/2945843499 conn(0x557bbbb37560 msgr2=0x557bbbb7fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 -- 172.21.15.46:0/2945843499 shutdown_connections 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.575+0000 7f882ad6a740 1 -- 172.21.15.46:0/2945843499 wait complete. 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.576+0000 7f882ad6a740 1 -- start start 2024-04-26T23:52:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.576+0000 7f882ad6a740 1 Processor -- start 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 0x557bbbb78770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb78cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bbbb3b9d0 con 0x557bbbb64f50 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557bbba26120 con 0x557bbbb538a0 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.577+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557bbbb35490 con 0x557bbbb827f0 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f882ad6a740 10 client.0 _ll_register_callbacks cb 0x557bbba5e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47788/0 (socket says 172.21.15.46:47788) 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 -- 172.21.15.46:0/377136259 learned_addr learned my addr 172.21.15.46:0/377136259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8824a00640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 0x557bbbb78770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825e00640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb78cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 -- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 msgr2=0x557bbbb78770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 0x557bbbb78770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 -- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 msgr2=0x557bbbb78cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb78cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8825400640 1 -- 172.21.15.46:0/377136259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbb7700f0 con 0x557bbbb64f50 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.578+0000 7f8824a00640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 0x557bbbb78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f8825400640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f881402a770 tx=0x7f881402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa499003ddf86905 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f8806000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881404d060 con 0x557bbbb64f50 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f881403cdc0 con 0x557bbbb64f50 2024-04-26T23:52:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8818004b60 con 0x557bbbb64f50 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f8814053050 con 0x557bbbb64f50 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f882ad6a740 10 client.9840 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.579+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x557bbb8daf60 con 0x557bbbb64f50 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818004b60 con 0x557bbbb64f50 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f8814056020 con 0x557bbbb64f50 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 10 client.9840 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 10 client.9840 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 20 client.9840 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb67020 0x557bbbb68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 4 client.9840 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bbbaac510 con 0x557bbbb67020 2024-04-26T23:52:04.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.580+0000 7f8824a00640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb67020 0x557bbbb68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.581+0000 7f8824a00640 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb67020 0x557bbbb68410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f53a6d81c9a4bf server_cookie=e2be89cb0b9a7066 in_seq=0 out_seq=0 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.581+0000 7f8806000640 10 client.9840 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.581+0000 7f8806000640 1 -- 172.21.15.46:0/377136259 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bbbaac510 con 0x557bbbb67020 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.581+0000 7f8806000640 10 client.9840 handle_command_reply: tid=0 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 client.9840 shutdown 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 msgr2=0x557bbbb77820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f881402a770 tx=0x7f881402c070 comp rx=0 tx=0).stop 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 shutdown_connections 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb67020 0x557bbbb68410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb78cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb827f0 0x557bbbb78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 --2- 172.21.15.46:0/377136259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb64f50 0x557bbbb77820 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f8806000640 0 client.9840 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 >> 172.21.15.46:0/377136259 conn(0x557bbbb3ac30 msgr2=0x557bbbb83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 shutdown_connections 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 1 -- 172.21.15.46:0/377136259 wait complete. 2024-04-26T23:52:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.582+0000 7f882ad6a740 20 client.9840 trim_cache size 0 max 16384 2024-04-26T23:52:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.601+0000 7f882ad6a740 1 Processor -- start 2024-04-26T23:52:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.601+0000 7f882ad6a740 1 -- start start 2024-04-26T23:52:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb538a0 0x557bbbb92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 0x557bbbb76330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8820201cf0 con 0x557bbbb827f0 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8820200d70 con 0x557bbbb64f50 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.602+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f88200ab430 con 0x557bbbb538a0 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47790/0 (socket says 172.21.15.46:47790) 2024-04-26T23:52:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 -- 172.21.15.46:0/1535661033 learned_addr learned my addr 172.21.15.46:0/1535661033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825400640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb538a0 0x557bbbb92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8824a00640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 0x557bbbb76330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 -- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb538a0 msgr2=0x557bbbb92610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb538a0 0x557bbbb92610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 -- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 msgr2=0x557bbbb76330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 0x557bbbb76330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8825e00640 1 -- 172.21.15.46:0/1535661033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbb7491b0 con 0x557bbbb827f0 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.603+0000 7f8824a00640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 0x557bbbb76330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.604+0000 7f8825e00640 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f8818016040 tx=0x7f8818006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d122ec92d32b0281 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.604+0000 7f8811200640 1 -- 172.21.15.46:0/1535661033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818001710 con 0x557bbbb827f0 2024-04-26T23:52:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.604+0000 7f8811200640 1 -- 172.21.15.46:0/1535661033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881800dcb0 con 0x557bbbb827f0 2024-04-26T23:52:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.604+0000 7f8811200640 1 -- 172.21.15.46:0/1535661033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004da0 con 0x557bbbb827f0 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 -- 172.21.15.46:0/1535661033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 msgr2=0x557bbbb3b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f8818016040 tx=0x7f8818006000 comp rx=0 tx=0).stop 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 -- 172.21.15.46:0/1535661033 shutdown_connections 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb3b010 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb64f50 0x557bbbb76330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 --2- 172.21.15.46:0/1535661033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb538a0 0x557bbbb92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 -- 172.21.15.46:0/1535661033 >> 172.21.15.46:0/1535661033 conn(0x557bbbb3ac30 msgr2=0x557bbbb64150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 -- 172.21.15.46:0/1535661033 shutdown_connections 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.605+0000 7f882ad6a740 1 -- 172.21.15.46:0/1535661033 wait complete. 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.606+0000 7f882ad6a740 1 -- start start 2024-04-26T23:52:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.606+0000 7f882ad6a740 1 Processor -- start 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb76e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb77340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8820200720 con 0x557bbbb827f0 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557bbbb61690 con 0x557bbbb538a0 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557bbbb39610 con 0x557bbbb64f50 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f882ad6a740 10 client.0 _ll_register_callbacks cb 0x7f8814035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f8825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47794/0 (socket says 172.21.15.46:47794) 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.607+0000 7f8825e00640 1 -- 172.21.15.46:0/628114109 learned_addr learned my addr 172.21.15.46:0/628114109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8824a00640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb77340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 -- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 msgr2=0x557bbbb77340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825400640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb76e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb77340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 -- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 msgr2=0x557bbbb76e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb76e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 -- 172.21.15.46:0/628114109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbbb926e0 con 0x557bbbb827f0 2024-04-26T23:52:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8825e00640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f88180012e0 tx=0x7f8818001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d1d993290d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8806000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818020860 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f8818020a00 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.608+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881800dcb0 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f8818039050 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 10 client.9846 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x557bbbb393b0 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881800dcb0 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f881803c020 con 0x557bbbb827f0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 10 client.9846 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 10 client.9846 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 20 client.9846 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb3a5a0 0x557bbbb38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 4 client.9846 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x557bbbaa6380 con 0x557bbbb3a5a0 2024-04-26T23:52:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.609+0000 7f8825400640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb3a5a0 0x557bbbb38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f8825400640 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb3a5a0 0x557bbbb38c30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9214dd38e8c50e server_cookie=9ff335d830ddc9b2 in_seq=0 out_seq=0 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f8806000640 10 client.9846 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f8806000640 1 -- 172.21.15.46:0/628114109 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557bbbaa6380 con 0x557bbbb3a5a0 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f8806000640 10 client.9846 handle_command_reply: tid=0 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 client.9846 shutdown 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 msgr2=0x557bbbb386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f88180012e0 tx=0x7f8818001250 comp rx=0 tx=0).stop 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 shutdown_connections 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x557bbbb3a5a0 0x557bbbb38c30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bbbb827f0 0x557bbbb386f0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557bbbb64f50 0x557bbbb77340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 --2- 172.21.15.46:0/628114109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557bbbb538a0 0x557bbbb76e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f8806000640 0 client.9846 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.610+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 >> 172.21.15.46:0/628114109 conn(0x557bbbb3ac30 msgr2=0x557bbbb63fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 shutdown_connections 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f882ad6a740 1 -- 172.21.15.46:0/628114109 wait complete. 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f882ad6a740 20 client.9846 trim_cache size 0 max 16384 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:52:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 msgr2=0x7f880006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 0x7f880006edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8814034000 tx=0x7f881402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 msgr2=0x7f88200cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.611+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c7f0 tx=0x7f880c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 shutdown_connections 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f880006c8f0 0x7f880006edb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f882015e050 0x7f88200cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f882015d730 0x7f88200cc100 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 --2- 172.21.15.46:0/2875906563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88200b53b0 0x7f8820140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 >> 172.21.15.46:0/2875906563 conn(0x7f882006c980 msgr2=0x7f8820142c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 shutdown_connections 2024-04-26T23:52:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.612+0000 7f8827e00640 1 -- 172.21.15.46:0/2875906563 wait complete. 2024-04-26T23:52:04.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:04.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub status 2024-04-26T23:52:04.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.830+0000 7fdeb5400640 1 Processor -- start 2024-04-26T23:52:04.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.830+0000 7fdeb5400640 1 -- start start 2024-04-26T23:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeb5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdeb01612f0 0x7fdeb00d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeb5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb015d640 0x7fdeb00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeb5400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb00d0e50 con 0x7fdeb01616d0 2024-04-26T23:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeb5400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb00d1130 con 0x7fdeb01612f0 2024-04-26T23:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeb5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb00cf230 con 0x7fdeb015d640 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdea7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdeb01616d0 0x7fdeb015cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54918/0 (socket says 172.21.15.46:54918) 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdea7400640 1 -- 172.21.15.46:0/1202143435 learned_addr learned my addr 172.21.15.46:0/1202143435 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.831+0000 7fdeafe00640 1 --2- 172.21.15.46:0/1202143435 >> v2:172.21.15.46:3300/0 conn(0x7fdeb01612f0 0x7fdeb00d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeb4a00640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb015d640 0x7fdeb00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeaf400640 1 -- 172.21.15.46:0/1202143435 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771094334 0 0) 0x7fdeb00d0e50 con 0x7fdeb01616d0 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeaf400640 1 -- 172.21.15.46:0/1202143435 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde94003520 con 0x7fdeb01616d0 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeafe00640 1 -- 172.21.15.46:0/1202143435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb015d640 msgr2=0x7fdeb00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeafe00640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb015d640 0x7fdeb00ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeafe00640 1 -- 172.21.15.46:0/1202143435 >> v1:172.21.15.46:6789/0 conn(0x7fdeb01616d0 legacy=0x7fdeb015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.832+0000 7fdeafe00640 1 -- 172.21.15.46:0/1202143435 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb00cf510 con 0x7fdeb01612f0 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.833+0000 7fdeafe00640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb01612f0 0x7fdeb00d38d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fdea00097a0 tx=0x7fdea002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41bdc903207071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.833+0000 7fdeaf400640 1 -- 172.21.15.46:0/1202143435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0037070 con 0x7fdeb01612f0 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.833+0000 7fdeaf400640 1 -- 172.21.15.46:0/1202143435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea002ae50 con 0x7fdeb01612f0 2024-04-26T23:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.833+0000 7fdeaf400640 1 -- 172.21.15.46:0/1202143435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0032660 con 0x7fdeb01612f0 2024-04-26T23:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 -- 172.21.15.46:0/1202143435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb01612f0 msgr2=0x7fdeb00d38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb01612f0 0x7fdeb00d38d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fdea00097a0 tx=0x7fdea002a100 comp rx=0 tx=0).stop 2024-04-26T23:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 -- 172.21.15.46:0/1202143435 shutdown_connections 2024-04-26T23:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb015d640 0x7fdeb00ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 --2- 172.21.15.46:0/1202143435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb01612f0 0x7fdeb00d38d0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 -- 172.21.15.46:0/1202143435 >> 172.21.15.46:0/1202143435 conn(0x7fdeb00d2db0 msgr2=0x7fdeb00d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 -- 172.21.15.46:0/1202143435 shutdown_connections 2024-04-26T23:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.834+0000 7fdeb5400640 1 -- 172.21.15.46:0/1202143435 wait complete. 2024-04-26T23:52:04.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.836+0000 7fdeb5400640 1 Processor -- start 2024-04-26T23:52:04.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.836+0000 7fdeb5400640 1 -- start start 2024-04-26T23:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb01612f0 0x7fdeb015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb01616d0 0x7fdeb00c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb00afce0 con 0x7fdeb015d640 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb00afb60 con 0x7fdeb01616d0 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeb5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb00afe60 con 0x7fdeb01612f0 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47814/0 (socket says 172.21.15.46:47814) 2024-04-26T23:52:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 -- 172.21.15.46:0/2962205228 learned_addr learned my addr 172.21.15.46:0/2962205228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdea7400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb01612f0 0x7fdeb015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 -- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb01612f0 msgr2=0x7fdeb015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb01612f0 0x7fdeb015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 -- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb01616d0 msgr2=0x7fdeb00c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb01616d0 0x7fdeb00c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 -- 172.21.15.46:0/2962205228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb01f9350 con 0x7fdeb015d640 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.838+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fdea00098c0 tx=0x7fdea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.839+0000 7fdeacc00640 1 -- 172.21.15.46:0/2962205228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0041030 con 0x7fdeb015d640 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.839+0000 7fdeacc00640 1 -- 172.21.15.46:0/2962205228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea002ac50 con 0x7fdeb015d640 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.839+0000 7fdeacc00640 1 -- 172.21.15.46:0/2962205228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0037040 con 0x7fdeb015d640 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.839+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb01f9630 con 0x7fdeb015d640 2024-04-26T23:52:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.839+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb00c75d0 con 0x7fdeb015d640 2024-04-26T23:52:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.843+0000 7fdeacc00640 1 -- 172.21.15.46:0/2962205228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdea0044070 con 0x7fdeb015d640 2024-04-26T23:52:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdeacc00640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdeacc00640 1 -- 172.21.15.46:0/2962205228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fdea00ac330 con 0x7fdeb015d640 2024-04-26T23:52:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdeb978d740 1 Processor -- start 2024-04-26T23:52:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdea7400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdeb978d740 1 -- start start 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.844+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b784d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b78053f0 0x55c3b784b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeb978d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b7715030 con 0x55c3b782f710 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b77307f0 con 0x55c3b784cfb0 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b76eb7e0 con 0x55c3b78053f0 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdea7400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 0x7fde8c06edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00c6e20 tx=0x7fde98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47824/0 (socket says 172.21.15.46:47824) 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 -- 172.21.15.46:0/2119795685 learned_addr learned my addr 172.21.15.46:0/2119795685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 -- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b78053f0 msgr2=0x55c3b784b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b78053f0 0x55c3b784b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 -- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 msgr2=0x55c3b784d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b784d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.845+0000 7fdeafe00640 1 -- 172.21.15.46:0/2119795685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b76f7050 con 0x55c3b782f710 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fdeafe00640 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fdea0035040 tx=0x7fdea00797f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1d2dc618de9e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fde9fe00640 1 -- 172.21.15.46:0/2119795685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0079e90 con 0x55c3b782f710 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fde9fe00640 1 -- 172.21.15.46:0/2119795685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea007e240 con 0x55c3b782f710 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fde9fe00640 1 -- 172.21.15.46:0/2119795685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0082a70 con 0x55c3b782f710 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fdeb978d740 1 -- 172.21.15.46:0/2119795685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 msgr2=0x55c3b782faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.846+0000 7fdeb978d740 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fdea0035040 tx=0x7fdea00797f0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 -- 172.21.15.46:0/2119795685 shutdown_connections 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b78053f0 0x55c3b784b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b784d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 --2- 172.21.15.46:0/2119795685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b782faf0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 -- 172.21.15.46:0/2119795685 >> 172.21.15.46:0/2119795685 conn(0x55c3b7801d20 msgr2=0x55c3b784a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 -- 172.21.15.46:0/2119795685 shutdown_connections 2024-04-26T23:52:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.847+0000 7fdeb978d740 1 -- 172.21.15.46:0/2119795685 wait complete. 2024-04-26T23:52:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.848+0000 7fdeb978d740 1 -- start start 2024-04-26T23:52:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.848+0000 7fdeb978d740 1 Processor -- start 2024-04-26T23:52:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7845f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b784b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b776d200 con 0x55c3b782f710 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b7767ee0 con 0x55c3b784cfb0 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b7774e70 con 0x55c3b785d040 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 10 client.0 _ll_register_callbacks cb 0x55c3b775e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47826/0 (socket says 172.21.15.46:47826) 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 -- 172.21.15.46:0/468122083 learned_addr learned my addr 172.21.15.46:0/468122083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 -- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 msgr2=0x55c3b784b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b784b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 -- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 msgr2=0x55c3b7845f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7845f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 -- 172.21.15.46:0/468122083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b75447d0 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeafe00640 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fdea0035040 tx=0x7fdea0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1b9d071f72d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fde9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0082a70 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fdea0078190 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea007e240 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fdea0099380 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 10 client.9861 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.850+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55c3b747a3f0 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea007e240 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7fdea004e060 con 0x55c3b782f710 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 10 client.9861 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 10 client.9861 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 20 client.9861 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b784daa0 0x55c3b784df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 4 client.9861 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3b77767e0 con 0x55c3b784daa0 2024-04-26T23:52:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.851+0000 7fdea7400640 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b784daa0 0x55c3b784df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.852+0000 7fdea7400640 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b784daa0 0x55c3b784df30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586473f1f45fbb38 server_cookie=367865955ac998e3 in_seq=0 out_seq=0 2024-04-26T23:52:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.852+0000 7fde9cc00640 10 client.9861 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.852+0000 7fde9cc00640 1 -- 172.21.15.46:0/468122083 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3b77767e0 con 0x55c3b784daa0 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.852+0000 7fde9cc00640 10 client.9861 handle_command_reply: tid=0 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 client.9861 shutdown 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 msgr2=0x55c3b78594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fdea0035040 tx=0x7fdea0009190 comp rx=0 tx=0).stop 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 shutdown_connections 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b784daa0 0x55c3b784df30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b784b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7845f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 --2- 172.21.15.46:0/468122083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b78594e0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fde9cc00640 0 client.9861 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 >> 172.21.15.46:0/468122083 conn(0x55c3b78053f0 msgr2=0x55c3b7804950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 shutdown_connections 2024-04-26T23:52:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 1 -- 172.21.15.46:0/468122083 wait complete. 2024-04-26T23:52:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.853+0000 7fdeb978d740 20 client.9861 trim_cache size 0 max 16384 2024-04-26T23:52:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 Processor -- start 2024-04-26T23:52:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 -- start start 2024-04-26T23:52:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7821d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b78499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0201c60 con 0x55c3b782f710 2024-04-26T23:52:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0200cb0 con 0x55c3b784cfb0 2024-04-26T23:52:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.873+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb0158740 con 0x55c3b785d040 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47838/0 (socket says 172.21.15.46:47838) 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 -- 172.21.15.46:0/715624181 learned_addr learned my addr 172.21.15.46:0/715624181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdea7400640 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7821d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeb4a00640 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b78499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 -- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 msgr2=0x55c3b78499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b78499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 -- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 msgr2=0x55c3b7821d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.874+0000 7fdeafe00640 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7821d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.875+0000 7fdeafe00640 1 -- 172.21.15.46:0/715624181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b743a0f0 con 0x55c3b782f710 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.875+0000 7fdeafe00640 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fdea002a6f0 tx=0x7fdea00aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bc433e08d5db8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.875+0000 7fde9f400640 1 -- 172.21.15.46:0/715624181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea007e240 con 0x55c3b782f710 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.875+0000 7fde9f400640 1 -- 172.21.15.46:0/715624181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea00a8040 con 0x55c3b782f710 2024-04-26T23:52:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.875+0000 7fde9f400640 1 -- 172.21.15.46:0/715624181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0082630 con 0x55c3b782f710 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 -- 172.21.15.46:0/715624181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 msgr2=0x55c3b77fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fdea002a6f0 tx=0x7fdea00aa000 comp rx=0 tx=0).stop 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 -- 172.21.15.46:0/715624181 shutdown_connections 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b785d040 0x55c3b78499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b784cfb0 0x55c3b7821d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 --2- 172.21.15.46:0/715624181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b782f710 0x55c3b77fec40 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 -- 172.21.15.46:0/715624181 >> 172.21.15.46:0/715624181 conn(0x55c3b78053f0 msgr2=0x55c3b77fb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 -- 172.21.15.46:0/715624181 shutdown_connections 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.876+0000 7fdeb978d740 1 -- 172.21.15.46:0/715624181 wait complete. 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.877+0000 7fdeb978d740 1 -- start start 2024-04-26T23:52:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.877+0000 7fdeb978d740 1 Processor -- start 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b784cfb0 0x55c3b786c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 0x55c3b786ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b7775fd0 con 0x55c3b786d090 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01f5fa0 con 0x55c3b785d040 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b78222e0 con 0x55c3b784cfb0 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb978d740 10 client.0 _ll_register_callbacks cb 0x7fdea0079cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47854/0 (socket says 172.21.15.46:47854) 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.878+0000 7fdeb4a00640 1 -- 172.21.15.46:0/622996073 learned_addr learned my addr 172.21.15.46:0/622996073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdea7400640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 0x55c3b786ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeafe00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b784cfb0 0x55c3b786c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 -- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b784cfb0 msgr2=0x55c3b786c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b784cfb0 0x55c3b786c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 -- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 msgr2=0x55c3b786ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 0x55c3b786ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 -- 172.21.15.46:0/622996073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b7730a30 con 0x55c3b786d090 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdea7400640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 0x55c3b786ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb4a00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x55c3b784a4f0 tx=0x7fdea8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64d6b1d30348bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fde9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea800fa90 con 0x55c3b786d090 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fdea800fc30 con 0x55c3b786d090 2024-04-26T23:52:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.879+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea00a8040 con 0x55c3b786d090 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fdea8023050 con 0x55c3b786d090 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 10 client.9867 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55c3b7822020 con 0x55c3b786d090 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea00a8040 con 0x55c3b786d090 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7fdea8026020 con 0x55c3b786d090 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 10 client.9867 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 10 client.9867 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 20 client.9867 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b785c1a0 0x55c3b7821900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 4 client.9867 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3b7770650 con 0x55c3b785c1a0 2024-04-26T23:52:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.880+0000 7fdeafe00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b785c1a0 0x55c3b7821900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.881+0000 7fdeafe00640 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b785c1a0 0x55c3b7821900 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e38075f97f84821e server_cookie=c80d306c939ae28e in_seq=0 out_seq=0 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.881+0000 7fde9cc00640 10 client.9867 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.881+0000 7fde9cc00640 1 -- 172.21.15.46:0/622996073 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x55c3b7770650 con 0x55c3b785c1a0 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.881+0000 7fde9cc00640 10 client.9867 handle_command_reply: tid=0 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.881+0000 7fdeb978d740 1 client.9867 shutdown 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 msgr2=0x55c3b78213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x55c3b784a4f0 tx=0x7fdea8007db0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 shutdown_connections 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55c3b785c1a0 0x55c3b7821900 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b786d090 0x55c3b78213c0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b785d040 0x55c3b786ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fde9cc00640 0 client.9867 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 --2- 172.21.15.46:0/622996073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b784cfb0 0x55c3b786c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 >> 172.21.15.46:0/622996073 conn(0x55c3b78053f0 msgr2=0x55c3b785ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 shutdown_connections 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 1 -- 172.21.15.46:0/622996073 wait complete. 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.882+0000 7fdeb978d740 20 client.9867 trim_cache size 0 max 16384 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "6bdad86a-0ba0-4ca4-bc3f-6a7717ccafe2": { 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_control_test_path", 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6bdad86a-0ba0-4ca4-bc3f-6a7717ccafe2", 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:04.890 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 msgr2=0x7fde8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 0x7fde8c06edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00c6e20 tx=0x7fde98009040 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 msgr2=0x7fdeb00c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fdea00098c0 tx=0x7fdea00046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 shutdown_connections 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb01616d0 0x7fdeb00c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeb01612f0 0x7fdeb015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 --2- 172.21.15.46:0/2962205228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb015d640 0x7fdeb00c68e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 >> 172.21.15.46:0/2962205228 conn(0x7fdeb00d2db0 msgr2=0x7fdeb00b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 shutdown_connections 2024-04-26T23:52:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:04.883+0000 7fdeb5400640 1 -- 172.21.15.46:0/2962205228 wait complete. 2024-04-26T23:52:04.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 Processor -- start 2024-04-26T23:52:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 -- start start 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd38c0d2010 0x7fd38c0d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd38c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd38c0d0990 con 0x7fd38c15f200 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd38c0d0c70 con 0x7fd38c0d2010 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd390a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c0d0f50 con 0x7fd38c15f5e0 2024-04-26T23:52:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd38a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd38c15f200 0x7fd38c0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54930/0 (socket says 172.21.15.46:54930) 2024-04-26T23:52:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.112+0000 7fd38a800640 1 -- 172.21.15.46:0/303734298 learned_addr learned my addr 172.21.15.46:0/303734298 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd381e00640 1 --2- 172.21.15.46:0/303734298 >> v2:172.21.15.142:3300/0 conn(0x7fd38c0d2010 0x7fd38c0d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10107099 0 0) 0x7fd38c0d0990 con 0x7fd38c15f200 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd370003520 con 0x7fd38c15f200 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd38b200640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd38c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851316165 0 0) 0x7fd370003520 con 0x7fd38c15f200 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 msgr2=0x7fd38c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd38c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 >> v2:172.21.15.142:3300/0 conn(0x7fd38c0d2010 msgr2=0x7fd38c0d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 --2- 172.21.15.46:0/303734298 >> v2:172.21.15.142:3300/0 conn(0x7fd38c0d2010 0x7fd38c0d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c0d1230 con 0x7fd38c15f200 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd37c001670 con 0x7fd38c15f200 2024-04-26T23:52:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 >> v1:172.21.15.46:6789/0 conn(0x7fd38c15f200 legacy=0x7fd38c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.113+0000 7fd389e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3700039b0 0x7fd370003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd389e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd389e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd3700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c0d0c70 con 0x7fd370004800 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c0d0f50 con 0x7fd3700039b0 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd37000ba70 con 0x7fd38c15f5e0 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd38a800640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3700039b0 0x7fd370003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd38b200640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd3700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 -- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 msgr2=0x7fd3700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c15f5e0 0x7fd3700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 -- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3700039b0 msgr2=0x7fd370003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3700039b0 0x7fd370003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd381e00640 1 -- 172.21.15.46:0/303734298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd370003520 con 0x7fd370004800 2024-04-26T23:52:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.114+0000 7fd38a800640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3700039b0 0x7fd370003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.115+0000 7fd381e00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd370004c80 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fd37400ba00 tx=0x7fd370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58b1d5dea4102b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374030070 con 0x7fd370004800 2024-04-26T23:52:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37402adb0 con 0x7fd370004800 2024-04-26T23:52:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd389e00640 1 -- 172.21.15.46:0/303734298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37400b600 con 0x7fd370004800 2024-04-26T23:52:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd390a00640 1 -- 172.21.15.46:0/303734298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 msgr2=0x7fd370004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd390a00640 1 --2- 172.21.15.46:0/303734298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd370004c80 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fd37400ba00 tx=0x7fd370005830 comp rx=0 tx=0).stop 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd38b200640 1 -- 172.21.15.46:0/303734298 reap_dead start 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd390a00640 1 -- 172.21.15.46:0/303734298 shutdown_connections 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.116+0000 7fd390a00640 1 -- 172.21.15.46:0/303734298 >> 172.21.15.46:0/303734298 conn(0x7fd38c0d4ef0 msgr2=0x7fd38c0d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.117+0000 7fd390a00640 1 -- 172.21.15.46:0/303734298 shutdown_connections 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.117+0000 7fd390a00640 1 -- 172.21.15.46:0/303734298 wait complete. 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.118+0000 7fd390a00640 1 Processor -- start 2024-04-26T23:52:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 -- start start 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c0b8e60 0x7fd38c15fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c1f44b0 0x7fd38c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c0b2f50 con 0x7fd370004800 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c0b2dd0 con 0x7fd38c1f44b0 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd390a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c0b30d0 con 0x7fd38c0b8e60 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47884/0 (socket says 172.21.15.46:47884) 2024-04-26T23:52:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 -- 172.21.15.46:0/3319835814 learned_addr learned my addr 172.21.15.46:0/3319835814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38b200640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c1f44b0 0x7fd38c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd381e00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c0b8e60 0x7fd38c15fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 -- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c0b8e60 msgr2=0x7fd38c15fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c0b8e60 0x7fd38c15fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 -- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c1f44b0 msgr2=0x7fd38c0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c1f44b0 0x7fd38c0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.120+0000 7fd38a800640 1 -- 172.21.15.46:0/3319835814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c0b93d0 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd38a800640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d8e0 tx=0x7fd37c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c019070 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c0074e0 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c0b96b0 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c007690 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.121+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c0b9c70 con 0x7fd370004800 2024-04-26T23:52:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.122+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0693b0 con 0x7fd370004800 2024-04-26T23:52:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.123+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd37c01c070 con 0x7fd370004800 2024-04-26T23:52:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.124+0000 7fd383400640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 0x7fd36806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.124+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fd37c08ba00 con 0x7fd370004800 2024-04-26T23:52:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.125+0000 7fd381e00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 0x7fd36806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.129+0000 7fd381e00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 0x7fd36806ede0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3700057e0 tx=0x7fd374030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.129+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd37c059bd0 con 0x7fd370004800 2024-04-26T23:52:05.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.373+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd38c0ba9b0 con 0x7fd370004800 2024-04-26T23:52:05.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.375+0000 7fd383400640 1 -- 172.21.15.46:0/3319835814 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7fd37c0594d0 con 0x7fd370004800 2024-04-26T23:52:05.378 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:05.379 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 msgr2=0x7fd36806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 0x7fd36806ede0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3700057e0 tx=0x7fd374030000 comp rx=0 tx=0).stop 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 msgr2=0x7fd38c0ba470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d8e0 tx=0x7fd37c00dda0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 shutdown_connections 2024-04-26T23:52:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd36806c920 0x7fd36806ede0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c1f44b0 0x7fd38c0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd38c0b8e60 0x7fd38c15fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 --2- 172.21.15.46:0/3319835814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd370004800 0x7fd38c0ba470 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 >> 172.21.15.46:0/3319835814 conn(0x7fd38c0d4ef0 msgr2=0x7fd38c15e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 shutdown_connections 2024-04-26T23:52:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.378+0000 7fd390a00640 1 -- 172.21.15.46:0/3319835814 wait complete. 2024-04-26T23:52:05.400 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:05.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.602+0000 7f1dea000640 1 Processor -- start 2024-04-26T23:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.602+0000 7f1dea000640 1 -- start start 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1dea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1de4159950 0x7f1de40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1dea000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1de41633a0 0x7f1de406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1dea000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1de40af0c0 con 0x7f1de40ae8e0 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1dea000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1de40af3a0 con 0x7f1de4159950 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1dea000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1de40af680 con 0x7f1de41633a0 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1de8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1de4159950 0x7f1de40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1de3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1de40ae8e0 0x7f1de4162c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54946/0 (socket says 172.21.15.46:54946) 2024-04-26T23:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1de3e00640 1 -- 172.21.15.46:0/4076708131 learned_addr learned my addr 172.21.15.46:0/4076708131 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.604+0000 7f1de9600640 1 --2- 172.21.15.46:0/4076708131 >> v2:172.21.15.142:3300/0 conn(0x7f1de41633a0 0x7f1de406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.605+0000 7f1de3400640 1 -- 172.21.15.46:0/4076708131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525844575 0 0) 0x7f1de40af0c0 con 0x7f1de40ae8e0 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.605+0000 7f1de8c00640 1 -- 172.21.15.46:0/4076708131 >> v2:172.21.15.142:3300/0 conn(0x7f1de41633a0 msgr2=0x7f1de406e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.605+0000 7f1de8c00640 1 --2- 172.21.15.46:0/4076708131 >> v2:172.21.15.142:3300/0 conn(0x7f1de41633a0 0x7f1de406e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.605+0000 7f1de8c00640 1 -- 172.21.15.46:0/4076708131 >> v1:172.21.15.46:6789/0 conn(0x7f1de40ae8e0 legacy=0x7f1de4162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.605+0000 7f1de8c00640 1 -- 172.21.15.46:0/4076708131 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de40af960 con 0x7f1de4159950 2024-04-26T23:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.606+0000 7f1de8c00640 1 --2- 172.21.15.46:0/4076708131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de4159950 0x7f1de40b48a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f1de4071b20 tx=0x7f1dd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcc771bde0f7af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.606+0000 7f1de3400640 1 -- 172.21.15.46:0/4076708131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8037070 con 0x7f1de4159950 2024-04-26T23:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.606+0000 7f1de3400640 1 -- 172.21.15.46:0/4076708131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd8032440 con 0x7f1de4159950 2024-04-26T23:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.606+0000 7f1de3400640 1 -- 172.21.15.46:0/4076708131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8032810 con 0x7f1de4159950 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 -- 172.21.15.46:0/4076708131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de4159950 msgr2=0x7f1de40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 --2- 172.21.15.46:0/4076708131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de4159950 0x7f1de40b48a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f1de4071b20 tx=0x7f1dd802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 -- 172.21.15.46:0/4076708131 shutdown_connections 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 --2- 172.21.15.46:0/4076708131 >> v2:172.21.15.142:3300/0 conn(0x7f1de41633a0 0x7f1de406e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 --2- 172.21.15.46:0/4076708131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de4159950 0x7f1de40b48a0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 -- 172.21.15.46:0/4076708131 >> 172.21.15.46:0/4076708131 conn(0x7f1de40b2820 msgr2=0x7f1de40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 -- 172.21.15.46:0/4076708131 shutdown_connections 2024-04-26T23:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.607+0000 7f1dea000640 1 -- 172.21.15.46:0/4076708131 wait complete. 2024-04-26T23:52:05.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.609+0000 7f1dea000640 1 Processor -- start 2024-04-26T23:52:05.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.609+0000 7f1dea000640 1 -- start start 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de40ae8e0 0x7f1de4156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1de41633a0 0x7f1de40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1de40b0160 con 0x7f1de40b1820 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1de40aed70 con 0x7f1de40ae8e0 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1dea000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1de406e960 con 0x7f1de41633a0 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47908/0 (socket says 172.21.15.46:47908) 2024-04-26T23:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 -- 172.21.15.46:0/661760158 learned_addr learned my addr 172.21.15.46:0/661760158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de8c00640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de40ae8e0 0x7f1de4156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 -- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1de41633a0 msgr2=0x7f1de40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1de41633a0 0x7f1de40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 -- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de40ae8e0 msgr2=0x7f1de4156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de40ae8e0 0x7f1de4156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.610+0000 7f1de9600640 1 -- 172.21.15.46:0/661760158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de415ba60 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1de9600640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc007ec0 tx=0x7f1ddc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc014620 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc019070 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc014a20 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de415bd40 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.611+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de415c300 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.612+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de40693b0 con 0x7f1de40b1820 2024-04-26T23:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.614+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ddc014bc0 con 0x7f1de40b1820 2024-04-26T23:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.614+0000 7f1de0c00640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 0x7f1dc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.615+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f1ddc08c5d0 con 0x7f1de40b1820 2024-04-26T23:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.615+0000 7f1de8c00640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 0x7f1dc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.616+0000 7f1de8c00640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 0x7f1dc806ede0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:05.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.619+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ddc05a7a0 con 0x7f1de40b1820 2024-04-26T23:52:05.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.893+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1de40b35e0 con 0x7f1de40b1820 2024-04-26T23:52:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.894+0000 7f1de0c00640 1 -- 172.21.15.46:0/661760158 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1203 (secure 0 0 0) 0x7f1ddc05a0a0 con 0x7f1de40b1820 2024-04-26T23:52:05.897 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:05.898 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":917,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":963,"data_bytes":987164576,"bytes_used":2386919424,"bytes_avail":770707193856,"bytes_total":773094113280,"recovering_objects_per_sec":3,"recovering_bytes_per_sec":3668401,"recovering_keys_per_sec":0,"num_objects_recovered":28,"num_bytes_recovered":29360128,"num_keys_recovered":0,"write_bytes_sec":18251167,"read_op_per_sec":0,"write_op_per_sec":18},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-26T23:51:06.203709+0000","services":{}},"progress_events":{}} 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 msgr2=0x7f1dc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 0x7f1dc806ede0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd802e000 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 msgr2=0x7f1de40c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc007ec0 tx=0x7f1ddc00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 shutdown_connections 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1dc806c920 0x7f1dc806ede0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1de40b1820 0x7f1de40c7240 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1de41633a0 0x7f1de40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 --2- 172.21.15.46:0/661760158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de40ae8e0 0x7f1de4156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 >> 172.21.15.46:0/661760158 conn(0x7f1de40b2820 msgr2=0x7f1de4162850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 shutdown_connections 2024-04-26T23:52:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:05.897+0000 7f1dea000640 1 -- 172.21.15.46:0/661760158 wait complete. 2024-04-26T23:52:06.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.128+0000 7f7af1600640 1 Processor -- start 2024-04-26T23:52:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.129+0000 7f7af1600640 1 -- start start 2024-04-26T23:52:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.130+0000 7f7af1600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7aec06e510 0x7f7aec06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.130+0000 7f7af1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7aec160bd0 0x7f7aec161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.130+0000 7f7af1600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aec06ee30 con 0x7f7aec15d540 2024-04-26T23:52:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.130+0000 7f7af1600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7aec0742d0 con 0x7f7aec160bd0 2024-04-26T23:52:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.130+0000 7f7af1600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7aec161940 con 0x7f7aec06e510 2024-04-26T23:52:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7aec160bd0 0x7f7aec161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7aec160bd0 0x7f7aec161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47916/0 (socket says 172.21.15.46:47916) 2024-04-26T23:52:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 -- 172.21.15.46:0/2367094200 learned_addr learned my addr 172.21.15.46:0/2367094200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aeaa00640 1 --2- 172.21.15.46:0/2367094200 >> v2:172.21.15.142:3300/0 conn(0x7f7aec06e510 0x7f7aec06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 -- 172.21.15.46:0/2367094200 >> v2:172.21.15.142:3300/0 conn(0x7f7aec06e510 msgr2=0x7f7aec06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 --2- 172.21.15.46:0/2367094200 >> v2:172.21.15.142:3300/0 conn(0x7f7aec06e510 0x7f7aec06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 -- 172.21.15.46:0/2367094200 >> v1:172.21.15.142:6789/0 conn(0x7f7aec15d540 legacy=0x7f7aec15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.131+0000 7f7aea000640 1 -- 172.21.15.46:0/2367094200 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec071f80 con 0x7f7aec160bd0 2024-04-26T23:52:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.133+0000 7f7aea000640 1 --2- 172.21.15.46:0/2367094200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec161060 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009870 tx=0x7f7ad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9102c6d65692b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.134+0000 7f7ae9600640 1 -- 172.21.15.46:0/2367094200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8037070 con 0x7f7aec160bd0 2024-04-26T23:52:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.134+0000 7f7ae9600640 1 -- 172.21.15.46:0/2367094200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad8032440 con 0x7f7aec160bd0 2024-04-26T23:52:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.134+0000 7f7ae9600640 1 -- 172.21.15.46:0/2367094200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8032810 con 0x7f7aec160bd0 2024-04-26T23:52:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.134+0000 7f7af1600640 1 -- 172.21.15.46:0/2367094200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 msgr2=0x7f7aec161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.134+0000 7f7af1600640 1 --2- 172.21.15.46:0/2367094200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec161060 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009870 tx=0x7f7ad802a270 comp rx=0 tx=0).stop 2024-04-26T23:52:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 -- 172.21.15.46:0/2367094200 shutdown_connections 2024-04-26T23:52:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 --2- 172.21.15.46:0/2367094200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec161060 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 --2- 172.21.15.46:0/2367094200 >> v2:172.21.15.142:3300/0 conn(0x7f7aec06e510 0x7f7aec06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 -- 172.21.15.46:0/2367094200 >> 172.21.15.46:0/2367094200 conn(0x7f7aec06c530 msgr2=0x7f7aec06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 -- 172.21.15.46:0/2367094200 shutdown_connections 2024-04-26T23:52:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.135+0000 7f7af1600640 1 -- 172.21.15.46:0/2367094200 wait complete. 2024-04-26T23:52:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.136+0000 7f7af1600640 1 Processor -- start 2024-04-26T23:52:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.137+0000 7f7af1600640 1 -- start start 2024-04-26T23:52:07.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aec06e510 0x7f7aec0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7aec15d540 0x7f7aec15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec072820 con 0x7f7aec160bd0 2024-04-26T23:52:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec072540 con 0x7f7aec06e510 2024-04-26T23:52:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7af1600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec1615a0 con 0x7f7aec15d540 2024-04-26T23:52:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7aeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7aeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47922/0 (socket says 172.21.15.46:47922) 2024-04-26T23:52:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7aeb400640 1 -- 172.21.15.46:0/216769583 learned_addr learned my addr 172.21.15.46:0/216769583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7aea000640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7aec15d540 0x7f7aec15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.138+0000 7f7aeaa00640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aec06e510 0x7f7aec0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 -- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7aec15d540 msgr2=0x7f7aec15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7aec15d540 0x7f7aec15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 -- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aec06e510 msgr2=0x7f7aec0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aec06e510 0x7f7aec0b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 -- 172.21.15.46:0/216769583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec0b5f80 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7aeb400640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c780 tx=0x7f7ae400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004b60 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae4004d00 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec13ec70 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.139+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400aa90 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.140+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec13f210 con 0x7f7aec160bd0 2024-04-26T23:52:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.141+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec069530 con 0x7f7aec160bd0 2024-04-26T23:52:07.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.142+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae400ac30 con 0x7f7aec160bd0 2024-04-26T23:52:07.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.143+0000 7f7ae2a00640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 0x7f7acc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.143+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f7ae408bfa0 con 0x7f7aec160bd0 2024-04-26T23:52:07.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.144+0000 7f7aeaa00640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 0x7f7acc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.145+0000 7f7aeaa00640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 0x7f7acc06ede0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.148+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ae4020070 con 0x7f7aec160bd0 2024-04-26T23:52:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.402+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aec06fb00 con 0x7f7aec160bd0 2024-04-26T23:52:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.403+0000 7f7ae2a00640 1 -- 172.21.15.46:0/216769583 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f7ae405a200 con 0x7f7aec160bd0 2024-04-26T23:52:07.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:07.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.406+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 msgr2=0x7f7acc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.406+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 0x7f7acc06ede0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc008040 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.406+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 msgr2=0x7f7aec0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.406+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c780 tx=0x7f7ae400cc40 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 shutdown_connections 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7acc06c920 0x7f7acc06ede0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aec160bd0 0x7f7aec0ae8e0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7aec15d540 0x7f7aec15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 --2- 172.21.15.46:0/216769583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aec06e510 0x7f7aec0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 >> 172.21.15.46:0/216769583 conn(0x7f7aec06c530 msgr2=0x7f7aec15fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 shutdown_connections 2024-04-26T23:52:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.407+0000 7f7af1600640 1 -- 172.21.15.46:0/216769583 wait complete. 2024-04-26T23:52:07.428 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:07.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:07.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.638+0000 7fda87200640 1 Processor -- start 2024-04-26T23:52:07.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.638+0000 7fda87200640 1 -- start start 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda87200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda8815bf50 0x7fda880b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda87200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda8806f8f0 0x7fda880ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda87200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda880b41b0 con 0x7fda88072c30 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda87200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fda8815c330 con 0x7fda8806f8f0 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda87200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fda880b4490 con 0x7fda8815bf50 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda86800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda88072c30 0x7fda880b5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54956/0 (socket says 172.21.15.46:54956) 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda86800640 1 -- 172.21.15.46:0/3210583772 learned_addr learned my addr 172.21.15.46:0/3210583772 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:07.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda85400640 1 --2- 172.21.15.46:0/3210583772 >> v2:172.21.15.46:3300/0 conn(0x7fda8806f8f0 0x7fda880ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.640+0000 7fda85e00640 1 --2- 172.21.15.46:0/3210583772 >> v2:172.21.15.142:3300/0 conn(0x7fda8815bf50 0x7fda880b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda84a00640 1 -- 172.21.15.46:0/3210583772 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 97132353 0 0) 0x7fda880b41b0 con 0x7fda88072c30 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda84a00640 1 -- 172.21.15.46:0/3210583772 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda6c003520 con 0x7fda88072c30 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda85400640 1 -- 172.21.15.46:0/3210583772 >> v2:172.21.15.142:3300/0 conn(0x7fda8815bf50 msgr2=0x7fda880b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda85400640 1 --2- 172.21.15.46:0/3210583772 >> v2:172.21.15.142:3300/0 conn(0x7fda8815bf50 0x7fda880b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda85400640 1 -- 172.21.15.46:0/3210583772 >> v1:172.21.15.46:6789/0 conn(0x7fda88072c30 legacy=0x7fda880b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.641+0000 7fda85400640 1 -- 172.21.15.46:0/3210583772 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda880b4770 con 0x7fda8806f8f0 2024-04-26T23:52:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.642+0000 7fda85400640 1 --2- 172.21.15.46:0/3210583772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8806f8f0 0x7fda880ce1f0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fda88074b80 tx=0x7fda7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200551597a2ab2de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.642+0000 7fda84a00640 1 -- 172.21.15.46:0/3210583772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74037070 con 0x7fda8806f8f0 2024-04-26T23:52:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.642+0000 7fda84a00640 1 -- 172.21.15.46:0/3210583772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74032440 con 0x7fda8806f8f0 2024-04-26T23:52:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.642+0000 7fda84a00640 1 -- 172.21.15.46:0/3210583772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74032810 con 0x7fda8806f8f0 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 -- 172.21.15.46:0/3210583772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8806f8f0 msgr2=0x7fda880ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 --2- 172.21.15.46:0/3210583772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8806f8f0 0x7fda880ce1f0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fda88074b80 tx=0x7fda7402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 -- 172.21.15.46:0/3210583772 shutdown_connections 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 --2- 172.21.15.46:0/3210583772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8806f8f0 0x7fda880ce1f0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 --2- 172.21.15.46:0/3210583772 >> v2:172.21.15.142:3300/0 conn(0x7fda8815bf50 0x7fda880b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 -- 172.21.15.46:0/3210583772 >> 172.21.15.46:0/3210583772 conn(0x7fda880b2820 msgr2=0x7fda880b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 -- 172.21.15.46:0/3210583772 shutdown_connections 2024-04-26T23:52:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.643+0000 7fda87200640 1 -- 172.21.15.46:0/3210583772 wait complete. 2024-04-26T23:52:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.645+0000 7fda87200640 1 Processor -- start 2024-04-26T23:52:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.645+0000 7fda87200640 1 -- start start 2024-04-26T23:52:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda8806f8f0 0x7fda8814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda88072c30 0x7fda880cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda880739f0 con 0x7fda8815bf50 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda880ce730 con 0x7fda88072c30 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda87200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda880b5960 con 0x7fda8806f8f0 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47942/0 (socket says 172.21.15.46:47942) 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda86800640 1 -- 172.21.15.46:0/401510394 learned_addr learned my addr 172.21.15.46:0/401510394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.646+0000 7fda85e00640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda8806f8f0 0x7fda8814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda85400640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda88072c30 0x7fda880cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 -- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda8806f8f0 msgr2=0x7fda8814c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda8806f8f0 0x7fda8814c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 -- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda88072c30 msgr2=0x7fda880cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda88072c30 0x7fda880cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 -- 172.21.15.46:0/401510394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda880cbb50 con 0x7fda8815bf50 2024-04-26T23:52:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda86800640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fda8000cb70 tx=0x7fda8000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda8815bf50 2024-04-26T23:52:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8000ad40 con 0x7fda8815bf50 2024-04-26T23:52:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80012620 con 0x7fda8815bf50 2024-04-26T23:52:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda880cbe30 con 0x7fda8815bf50 2024-04-26T23:52:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.647+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda880cc3f0 con 0x7fda8815bf50 2024-04-26T23:52:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.649+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880693b0 con 0x7fda8815bf50 2024-04-26T23:52:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.650+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda8001a070 con 0x7fda8815bf50 2024-04-26T23:52:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.651+0000 7fda7e000640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.651+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fda8008bc80 con 0x7fda8815bf50 2024-04-26T23:52:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.651+0000 7fda85e00640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.652+0000 7fda85e00640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 0x7fda6806edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fda78006fd0 tx=0x7fda78008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.655+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda80059e50 con 0x7fda8815bf50 2024-04-26T23:52:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.942+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fda8815ae40 con 0x7fda8815bf50 2024-04-26T23:52:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.943+0000 7fda7e000640 1 -- 172.21.15.46:0/401510394 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1205 (secure 0 0 0) 0x7fda80059750 con 0x7fda8815bf50 2024-04-26T23:52:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":919,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661712,"bytes_used":2605752320,"bytes_avail":770488360960,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":3097661,"recovering_keys_per_sec":0,"num_objects_recovered":28,"num_bytes_recovered":29360128,"num_keys_recovered":0,"write_bytes_sec":24855950,"read_op_per_sec":0,"write_op_per_sec":26},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-26T23:51:06.203709+0000","services":{}},"progress_events":{}} 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.945+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 msgr2=0x7fda6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.945+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 0x7fda6806edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fda78006fd0 tx=0x7fda78008040 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.945+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 msgr2=0x7fda880c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.945+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fda8000cb70 tx=0x7fda8000a430 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 shutdown_connections 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8815bf50 0x7fda880c86c0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda88072c30 0x7fda880cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 --2- 172.21.15.46:0/401510394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda8806f8f0 0x7fda8814c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 >> 172.21.15.46:0/401510394 conn(0x7fda880b2820 msgr2=0x7fda880aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 shutdown_connections 2024-04-26T23:52:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:07.946+0000 7fda87200640 1 -- 172.21.15.46:0/401510394 wait complete. 2024-04-26T23:52:08.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.194+0000 7f9e65600640 1 Processor -- start 2024-04-26T23:52:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.194+0000 7f9e65600640 1 -- start start 2024-04-26T23:52:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e65600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9e6006e780 0x7f9e600b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e65600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e6014d200 0x7f9e60072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e65600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e600b4940 con 0x7f9e600b43c0 2024-04-26T23:52:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e65600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600b4c20 con 0x7f9e6014d200 2024-04-26T23:52:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e65600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600b4f00 con 0x7f9e6006e780 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e5f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e6014d200 0x7f9e60072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.196+0000 7f9e5f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e6014d200 0x7f9e60072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47956/0 (socket says 172.21.15.46:47956) 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5f400640 1 -- 172.21.15.46:0/3844090348 learned_addr learned my addr 172.21.15.46:0/3844090348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5ea00640 1 --2- 172.21.15.46:0/3844090348 >> v2:172.21.15.142:3300/0 conn(0x7f9e6006e780 0x7f9e600b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5f400640 1 -- 172.21.15.46:0/3844090348 >> v2:172.21.15.142:3300/0 conn(0x7f9e6006e780 msgr2=0x7f9e600b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5f400640 1 --2- 172.21.15.46:0/3844090348 >> v2:172.21.15.142:3300/0 conn(0x7f9e6006e780 0x7f9e600b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5f400640 1 -- 172.21.15.46:0/3844090348 >> v1:172.21.15.142:6789/0 conn(0x7f9e600b43c0 legacy=0x7f9e6014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.197+0000 7f9e5f400640 1 -- 172.21.15.46:0/3844090348 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600b51e0 con 0x7f9e6014d200 2024-04-26T23:52:09.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.198+0000 7f9e5f400640 1 --2- 172.21.15.46:0/3844090348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e6014d200 0x7f9e60072360 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f9e60070070 tx=0x7f9e5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a955e761ebc361 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.199+0000 7f9e5d600640 1 -- 172.21.15.46:0/3844090348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58037070 con 0x7f9e6014d200 2024-04-26T23:52:09.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.199+0000 7f9e5d600640 1 -- 172.21.15.46:0/3844090348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e58032440 con 0x7f9e6014d200 2024-04-26T23:52:09.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.199+0000 7f9e5d600640 1 -- 172.21.15.46:0/3844090348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58032810 con 0x7f9e6014d200 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.199+0000 7f9e65600640 1 -- 172.21.15.46:0/3844090348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e6014d200 msgr2=0x7f9e60072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.199+0000 7f9e65600640 1 --2- 172.21.15.46:0/3844090348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e6014d200 0x7f9e60072360 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f9e60070070 tx=0x7f9e5802a230 comp rx=0 tx=0).stop 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 -- 172.21.15.46:0/3844090348 shutdown_connections 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 --2- 172.21.15.46:0/3844090348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e6014d200 0x7f9e60072360 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 --2- 172.21.15.46:0/3844090348 >> v2:172.21.15.142:3300/0 conn(0x7f9e6006e780 0x7f9e600b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 -- 172.21.15.46:0/3844090348 >> 172.21.15.46:0/3844090348 conn(0x7f9e6006c530 msgr2=0x7f9e6006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 -- 172.21.15.46:0/3844090348 shutdown_connections 2024-04-26T23:52:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.200+0000 7f9e65600640 1 -- 172.21.15.46:0/3844090348 wait complete. 2024-04-26T23:52:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.201+0000 7f9e65600640 1 Processor -- start 2024-04-26T23:52:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 -- start start 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6006e780 0x7f9e60160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6014d200 0x7f9e600cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006ece0 con 0x7f9e600b43c0 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.202+0000 7f9e65600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006eb60 con 0x7f9e6014d200 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e65600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6006ee60 con 0x7f9e6006e780 2024-04-26T23:52:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47970/0 (socket says 172.21.15.46:47970) 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 -- 172.21.15.46:0/4037470134 learned_addr learned my addr 172.21.15.46:0/4037470134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5ea00640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6006e780 0x7f9e60160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 -- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6006e780 msgr2=0x7f9e60160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6006e780 0x7f9e60160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 -- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6014d200 msgr2=0x7f9e600cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6014d200 0x7f9e600cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.203+0000 7f9e5e000640 1 -- 172.21.15.46:0/4037470134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6015e0b0 con 0x7f9e600b43c0 2024-04-26T23:52:09.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e5e000640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c400 tx=0x7f9e4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48017070 con 0x7f9e600b43c0 2024-04-26T23:52:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e480035f0 con 0x7f9e600b43c0 2024-04-26T23:52:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6015e320 con 0x7f9e600b43c0 2024-04-26T23:52:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003a20 con 0x7f9e600b43c0 2024-04-26T23:52:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.204+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6015e8e0 con 0x7f9e600b43c0 2024-04-26T23:52:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.205+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e60069530 con 0x7f9e600b43c0 2024-04-26T23:52:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.206+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e4801a070 con 0x7f9e600b43c0 2024-04-26T23:52:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.207+0000 7f9e56a00640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.207+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f9e4808b0c0 con 0x7f9e600b43c0 2024-04-26T23:52:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.208+0000 7f9e5ea00640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.209+0000 7f9e5ea00640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 0x7f9e3c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd940 tx=0x7f9e5000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.212+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e48058c40 con 0x7f9e600b43c0 2024-04-26T23:52:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.456+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e60069e80 con 0x7f9e600b43c0 2024-04-26T23:52:09.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.457+0000 7f9e56a00640 1 -- 172.21.15.46:0/4037470134 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f9e4805cf40 con 0x7f9e600b43c0 2024-04-26T23:52:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:09.461 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:09.461 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 msgr2=0x7f9e3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 0x7f9e3c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd940 tx=0x7f9e5000b040 comp rx=0 tx=0).stop 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 msgr2=0x7f9e60149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c400 tx=0x7f9e4800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 shutdown_connections 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6014d200 0x7f9e600cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e600b43c0 0x7f9e60149d00 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 --2- 172.21.15.46:0/4037470134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6006e780 0x7f9e60160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.460+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 >> 172.21.15.46:0/4037470134 conn(0x7f9e6006c530 msgr2=0x7f9e60161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.461+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 shutdown_connections 2024-04-26T23:52:09.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.461+0000 7f9e65600640 1 -- 172.21.15.46:0/4037470134 wait complete. 2024-04-26T23:52:09.483 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:09.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.670+0000 7fa7c5c00640 1 Processor -- start 2024-04-26T23:52:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.671+0000 7fa7c5c00640 1 -- start start 2024-04-26T23:52:09.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.672+0000 7fa7c5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7c00cf450 0x7fa7c0073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.672+0000 7fa7c5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c0072190 0x7fa7c00cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.672+0000 7fa7c5c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c006f470 con 0x7fa7c00cfd70 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.672+0000 7fa7c5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c0072930 con 0x7fa7c00cf450 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.672+0000 7fa7c5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c0072c10 con 0x7fa7c0072190 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7c5200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7c00cfd70 0x7fa7c006e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54964/0 (socket says 172.21.15.46:54964) 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7c5200640 1 -- 172.21.15.46:0/13360316 learned_addr learned my addr 172.21.15.46:0/13360316 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/13360316 >> v2:172.21.15.46:3300/0 conn(0x7fa7c00cf450 0x7fa7c0073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bf400640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c0072190 0x7fa7c00cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bea00640 1 -- 172.21.15.46:0/13360316 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415591702 0 0) 0x7fa7c006f470 con 0x7fa7c00cfd70 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bfe00640 1 -- 172.21.15.46:0/13360316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c0072190 msgr2=0x7fa7c00cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c0072190 0x7fa7c00cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bfe00640 1 -- 172.21.15.46:0/13360316 >> v1:172.21.15.46:6789/0 conn(0x7fa7c00cfd70 legacy=0x7fa7c006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.673+0000 7fa7bfe00640 1 -- 172.21.15.46:0/13360316 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c0072ef0 con 0x7fa7c00cf450 2024-04-26T23:52:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.674+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c00cf450 0x7fa7c0073db0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaf02d0c50b3438 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7bea00640 1 -- 172.21.15.46:0/13360316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0037070 con 0x7fa7c00cf450 2024-04-26T23:52:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7bea00640 1 -- 172.21.15.46:0/13360316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b0032440 con 0x7fa7c00cf450 2024-04-26T23:52:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7bea00640 1 -- 172.21.15.46:0/13360316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0032810 con 0x7fa7c00cf450 2024-04-26T23:52:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 -- 172.21.15.46:0/13360316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c00cf450 msgr2=0x7fa7c0073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c00cf450 0x7fa7c0073db0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b002a180 comp rx=0 tx=0).stop 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 -- 172.21.15.46:0/13360316 shutdown_connections 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c0072190 0x7fa7c00cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/13360316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c00cf450 0x7fa7c0073db0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 -- 172.21.15.46:0/13360316 >> 172.21.15.46:0/13360316 conn(0x7fa7c00b2820 msgr2=0x7fa7c00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.675+0000 7fa7c5c00640 1 -- 172.21.15.46:0/13360316 shutdown_connections 2024-04-26T23:52:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.676+0000 7fa7c5c00640 1 -- 172.21.15.46:0/13360316 wait complete. 2024-04-26T23:52:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.677+0000 7fa7c5c00640 1 Processor -- start 2024-04-26T23:52:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 -- start start 2024-04-26T23:52:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c00cf450 0x7fa7c01e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c00cfd70 0x7fa7c01e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00d01e0 con 0x7fa7c0072190 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00734b0 con 0x7fa7c00cf450 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.678+0000 7fa7c5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c00725f0 con 0x7fa7c00cfd70 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48000/0 (socket says 172.21.15.46:48000) 2024-04-26T23:52:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 -- 172.21.15.46:0/2357466427 learned_addr learned my addr 172.21.15.46:0/2357466427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 -- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c00cfd70 msgr2=0x7fa7c01e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c00cfd70 0x7fa7c01e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 -- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c00cf450 msgr2=0x7fa7c01e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c00cf450 0x7fa7c01e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.679+0000 7fa7bfe00640 1 -- 172.21.15.46:0/2357466427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c01e3810 con 0x7fa7c0072190 2024-04-26T23:52:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7bfe00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0037050 con 0x7fa7c0072190 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b0038040 con 0x7fa7c0072190 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c01e3af0 con 0x7fa7c0072190 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004490 con 0x7fa7c0072190 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.680+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01e33e0 con 0x7fa7c0072190 2024-04-26T23:52:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.681+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c00cf830 con 0x7fa7c0072190 2024-04-26T23:52:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.682+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7b0030900 con 0x7fa7c0072190 2024-04-26T23:52:09.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.683+0000 7fa7b7e00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 0x7fa7a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.683+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa7b00ac970 con 0x7fa7c0072190 2024-04-26T23:52:09.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.684+0000 7fa7bf400640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 0x7fa7a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.685+0000 7fa7bf400640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 0x7fa7a006edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006fd0 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.688+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7c00cf830 con 0x7fa7c0072190 2024-04-26T23:52:09.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.976+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa7c00693b0 con 0x7fa7c0072190 2024-04-26T23:52:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.977+0000 7fa7b7e00640 1 -- 172.21.15.46:0/2357466427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1205 (secure 0 0 0) 0x7fa7b007aab0 con 0x7fa7c0072190 2024-04-26T23:52:09.980 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":921,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661712,"bytes_used":2605752320,"bytes_avail":770488360960,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2802864,"recovering_keys_per_sec":0,"num_objects_recovered":28,"num_bytes_recovered":29360128,"num_keys_recovered":0,"write_bytes_sec":22490466,"read_op_per_sec":0,"write_op_per_sec":23},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-26T23:51:06.203709+0000","services":{}},"progress_events":{}} 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.979+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 msgr2=0x7fa7a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 0x7fa7a006edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006fd0 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 msgr2=0x7fa7c0136240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b00046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 shutdown_connections 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa7a006c8f0 0x7fa7a006edb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7c00cfd70 0x7fa7c01e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c00cf450 0x7fa7c01e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 --2- 172.21.15.46:0/2357466427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c0072190 0x7fa7c0136240 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 >> 172.21.15.46:0/2357466427 conn(0x7fa7c00b2820 msgr2=0x7fa7c0073b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 shutdown_connections 2024-04-26T23:52:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:09.980+0000 7fa7c5c00640 1 -- 172.21.15.46:0/2357466427 wait complete. 2024-04-26T23:52:11.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.225+0000 7f9a93600640 1 Processor -- start 2024-04-26T23:52:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.225+0000 7f9a93600640 1 -- start start 2024-04-26T23:52:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.226+0000 7f9a93600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a8c06e510 0x7f9a8c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.226+0000 7f9a93600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8c06ee30 con 0x7f9a8c0732c0 2024-04-26T23:52:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.226+0000 7f9a93600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8c161940 con 0x7f9a8c160bd0 2024-04-26T23:52:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.226+0000 7f9a93600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9a8c071f80 con 0x7f9a8c06e510 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a91600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a8c0732c0 0x7f9a8c073710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54980/0 (socket says 172.21.15.46:54980) 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a91600640 1 -- 172.21.15.46:0/2877347339 learned_addr learned my addr 172.21.15.46:0/2877347339 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a90c00640 1 --2- 172.21.15.46:0/2877347339 >> v2:172.21.15.46:3300/0 conn(0x7f9a8c06e510 0x7f9a8c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a8be00640 1 -- 172.21.15.46:0/2877347339 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185950955 0 0) 0x7f9a8c06ee30 con 0x7f9a8c0732c0 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a90c00640 1 -- 172.21.15.46:0/2877347339 >> v1:172.21.15.142:6789/0 conn(0x7f9a8c160bd0 legacy=0x7f9a8c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a90c00640 1 -- 172.21.15.46:0/2877347339 >> v1:172.21.15.46:6789/0 conn(0x7f9a8c0732c0 legacy=0x7f9a8c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.227+0000 7f9a90c00640 1 -- 172.21.15.46:0/2877347339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c072260 con 0x7f9a8c06e510 2024-04-26T23:52:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.228+0000 7f9a90c00640 1 --2- 172.21.15.46:0/2877347339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c06e8f0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097d0 tx=0x7f9a7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13e0a5f5b5a773 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.229+0000 7f9a8be00640 1 -- 172.21.15.46:0/2877347339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c037070 con 0x7f9a8c06e510 2024-04-26T23:52:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.229+0000 7f9a8be00640 1 -- 172.21.15.46:0/2877347339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c032440 con 0x7f9a8c06e510 2024-04-26T23:52:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.229+0000 7f9a8be00640 1 -- 172.21.15.46:0/2877347339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032810 con 0x7f9a8c06e510 2024-04-26T23:52:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.229+0000 7f9a93600640 1 -- 172.21.15.46:0/2877347339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 msgr2=0x7f9a8c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.229+0000 7f9a93600640 1 --2- 172.21.15.46:0/2877347339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c06e8f0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097d0 tx=0x7f9a7c02a210 comp rx=0 tx=0).stop 2024-04-26T23:52:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.230+0000 7f9a93600640 1 -- 172.21.15.46:0/2877347339 shutdown_connections 2024-04-26T23:52:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.230+0000 7f9a93600640 1 --2- 172.21.15.46:0/2877347339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c06e8f0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.230+0000 7f9a93600640 1 -- 172.21.15.46:0/2877347339 >> 172.21.15.46:0/2877347339 conn(0x7f9a8c06c530 msgr2=0x7f9a8c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.230+0000 7f9a93600640 1 -- 172.21.15.46:0/2877347339 shutdown_connections 2024-04-26T23:52:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.230+0000 7f9a93600640 1 -- 172.21.15.46:0/2877347339 wait complete. 2024-04-26T23:52:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.231+0000 7f9a93600640 1 Processor -- start 2024-04-26T23:52:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.232+0000 7f9a93600640 1 -- start start 2024-04-26T23:52:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a8c0732c0 0x7f9a8c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a8c160bd0 0x7f9a8c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c072d40 con 0x7f9a8c06e510 2024-04-26T23:52:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c072820 con 0x7f9a8c0732c0 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a93600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c072ec0 con 0x7f9a8c160bd0 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 -- 172.21.15.46:0/4048652059 learned_addr learned my addr 172.21.15.46:0/4048652059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a83e00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a8c0732c0 0x7f9a8c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 -- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a8c160bd0 msgr2=0x7f9a8c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a8c160bd0 0x7f9a8c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 -- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a8c0732c0 msgr2=0x7f9a8c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a8c0732c0 0x7f9a8c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.233+0000 7f9a90c00640 1 -- 172.21.15.46:0/4048652059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c0ab270 con 0x7f9a8c06e510 2024-04-26T23:52:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a90c00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02a250 tx=0x7f9a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c041070 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c03c930 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c037040 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c0ab550 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.234+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c0abb10 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.235+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c069530 con 0x7f9a8c06e510 2024-04-26T23:52:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.236+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a7c044070 con 0x7f9a8c06e510 2024-04-26T23:52:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.238+0000 7f9a89600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 0x7f9a6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.238+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f9a7c0ac930 con 0x7f9a8c06e510 2024-04-26T23:52:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.238+0000 7f9a83e00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 0x7f9a6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.240+0000 7f9a83e00640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 0x7f9a6806ede0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0b1940 tx=0x7f9a74008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.242+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a7c07acb0 con 0x7f9a8c06e510 2024-04-26T23:52:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.494+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8c06fb00 con 0x7f9a8c06e510 2024-04-26T23:52:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.495+0000 7f9a89600640 1 -- 172.21.15.46:0/4048652059 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f9a7c07a5b0 con 0x7f9a8c06e510 2024-04-26T23:52:11.498 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:11.498 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 msgr2=0x7f9a6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 0x7f9a6806ede0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0b1940 tx=0x7f9a74008040 comp rx=0 tx=0).stop 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 msgr2=0x7f9a8c1f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02a250 tx=0x7f9a7c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 shutdown_connections 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a6806c920 0x7f9a6806ede0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a8c160bd0 0x7f9a8c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a8c0732c0 0x7f9a8c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 --2- 172.21.15.46:0/4048652059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a8c06e510 0x7f9a8c1f68e0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 >> 172.21.15.46:0/4048652059 conn(0x7f9a8c06c530 msgr2=0x7f9a8c06e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 shutdown_connections 2024-04-26T23:52:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.499+0000 7f9a93600640 1 -- 172.21.15.46:0/4048652059 wait complete. 2024-04-26T23:52:11.522 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:11.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:11.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.733+0000 7f222e600640 1 Processor -- start 2024-04-26T23:52:11.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.733+0000 7f222e600640 1 -- start start 2024-04-26T23:52:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222e600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f22280d0c10 0x7f22280d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22281639c0 0x7f22280b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22280cfe50 con 0x7f22280d1990 2024-04-26T23:52:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f22280d0780 con 0x7f22281639c0 2024-04-26T23:52:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222e600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f22280d0130 con 0x7f22280d0c10 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22281639c0 0x7f22280b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.735+0000 7f222d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22280d1990 0x7f22280d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54982/0 (socket says 172.21.15.46:54982) 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f222d200640 1 -- 172.21.15.46:0/3868449633 learned_addr learned my addr 172.21.15.46:0/3868449633 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f2227e00640 1 --2- 172.21.15.46:0/3868449633 >> v2:172.21.15.142:3300/0 conn(0x7f22280d0c10 0x7f22280d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f222dc00640 1 -- 172.21.15.46:0/3868449633 >> v2:172.21.15.142:3300/0 conn(0x7f22280d0c10 msgr2=0x7f22280d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f222dc00640 1 --2- 172.21.15.46:0/3868449633 >> v2:172.21.15.142:3300/0 conn(0x7f22280d0c10 0x7f22280d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f222dc00640 1 -- 172.21.15.46:0/3868449633 >> v1:172.21.15.46:6789/0 conn(0x7f22280d1990 legacy=0x7f22280d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f222dc00640 1 -- 172.21.15.46:0/3868449633 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280d0410 con 0x7f22281639c0 2024-04-26T23:52:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.736+0000 7f2227400640 1 -- 172.21.15.46:0/3868449633 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058664722 0 0) 0x7f22280cfe50 con 0x7f22280d1990 2024-04-26T23:52:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.737+0000 7f222dc00640 1 --2- 172.21.15.46:0/3868449633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22281639c0 0x7f22280b7dd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f22280c8720 tx=0x7f222002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c96ea847da403f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f2227400640 1 -- 172.21.15.46:0/3868449633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220007740 con 0x7f22281639c0 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f2227400640 1 -- 172.21.15.46:0/3868449633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22200078e0 con 0x7f22281639c0 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f2227400640 1 -- 172.21.15.46:0/3868449633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220007cb0 con 0x7f22281639c0 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 -- 172.21.15.46:0/3868449633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22281639c0 msgr2=0x7f22280b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 --2- 172.21.15.46:0/3868449633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22281639c0 0x7f22280b7dd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f22280c8720 tx=0x7f222002a680 comp rx=0 tx=0).stop 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 -- 172.21.15.46:0/3868449633 shutdown_connections 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 --2- 172.21.15.46:0/3868449633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22281639c0 0x7f22280b7dd0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 --2- 172.21.15.46:0/3868449633 >> v2:172.21.15.142:3300/0 conn(0x7f22280d0c10 0x7f22280d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.738+0000 7f222e600640 1 -- 172.21.15.46:0/3868449633 >> 172.21.15.46:0/3868449633 conn(0x7f222809e2a0 msgr2=0x7f22280d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.739+0000 7f222e600640 1 -- 172.21.15.46:0/3868449633 shutdown_connections 2024-04-26T23:52:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.739+0000 7f222e600640 1 -- 172.21.15.46:0/3868449633 wait complete. 2024-04-26T23:52:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.740+0000 7f222e600640 1 Processor -- start 2024-04-26T23:52:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.740+0000 7f222e600640 1 -- start start 2024-04-26T23:52:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280d0c10 0x7f22280cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281639c0 0x7f22280b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22280cfc10 con 0x7f22280d1990 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22280cfa90 con 0x7f22281639c0 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.741+0000 7f222e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22280d5bd0 con 0x7f22280d0c10 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-04-26T23:52:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 -- 172.21.15.46:0/3821215207 learned_addr learned my addr 172.21.15.46:0/3821215207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 -- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280d0c10 msgr2=0x7f22280cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280d0c10 0x7f22280cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 -- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281639c0 msgr2=0x7f22280b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281639c0 0x7f22280b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 -- 172.21.15.46:0/3821215207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280cb770 con 0x7f22280d1990 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.742+0000 7f2227e00640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.743+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214017070 con 0x7f22280d1990 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.743+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221400c9a0 con 0x7f22280d1990 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.743+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400cd90 con 0x7f22280d1990 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.743+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22281f5800 con 0x7f22280d1990 2024-04-26T23:52:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.743+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22281f5dc0 con 0x7f22280d1990 2024-04-26T23:52:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.744+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22280693b0 con 0x7f22280d1990 2024-04-26T23:52:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.745+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f221401a070 con 0x7f22280d1990 2024-04-26T23:52:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.747+0000 7f2224c00640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.747+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f221408c180 con 0x7f22280d1990 2024-04-26T23:52:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.748+0000 7f222d200640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.749+0000 7f222d200640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 0x7f220806edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f221c007fd0 tx=0x7f221c007920 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:11.752+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f221405a350 con 0x7f22280d1990 2024-04-26T23:52:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.030+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f22280d1770 con 0x7f22280d1990 2024-04-26T23:52:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.031+0000 7f2224c00640 1 -- 172.21.15.46:0/3821215207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1205 (secure 0 0 0) 0x7f2214059c50 con 0x7f22280d1990 2024-04-26T23:52:12.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:12.034 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":923,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076661712,"bytes_used":2605752320,"bytes_avail":770488360960,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2445707,"recovering_keys_per_sec":0,"num_objects_recovered":28,"num_bytes_recovered":29360128,"num_keys_recovered":0,"write_bytes_sec":19624600,"read_op_per_sec":0,"write_op_per_sec":20},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-26T23:51:06.203709+0000","services":{}},"progress_events":{}} 2024-04-26T23:52:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.033+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 msgr2=0x7f220806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.033+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 0x7f220806edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f221c007fd0 tx=0x7f221c007920 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.033+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 msgr2=0x7f22280cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.033+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ee70 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 shutdown_connections 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281639c0 0x7f22280b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22280d1990 0x7f22280cca60 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 --2- 172.21.15.46:0/3821215207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22280d0c10 0x7f22280cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 >> 172.21.15.46:0/3821215207 conn(0x7f222809e2a0 msgr2=0x7f222806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 shutdown_connections 2024-04-26T23:52:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:12.034+0000 7f222e600640 1 -- 172.21.15.46:0/3821215207 wait complete. 2024-04-26T23:52:12.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:12.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:13.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:13.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.275+0000 7f48eb200640 1 Processor -- start 2024-04-26T23:52:13.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.276+0000 7f48eb200640 1 -- start start 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48ec1466b0 0x7f48ec06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48ec0720e0 0x7f48ec0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec160910 0x7f48ec160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec073ad0 con 0x7f48ec0720e0 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec146a90 con 0x7f48ec1466b0 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.277+0000 7f48eb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec073d70 con 0x7f48ec160910 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48ec0720e0 0x7f48ec0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48ec0720e0 0x7f48ec0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36900/0 (socket says 172.21.15.46:36900) 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 -- 172.21.15.46:0/2070857182 learned_addr learned my addr 172.21.15.46:0/2070857182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e9e00640 1 --2- 172.21.15.46:0/2070857182 >> v2:172.21.15.142:3300/0 conn(0x7f48ec1466b0 0x7f48ec06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 -- 172.21.15.46:0/2070857182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec160910 msgr2=0x7f48ec160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2070857182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec160910 0x7f48ec160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 -- 172.21.15.46:0/2070857182 >> v2:172.21.15.142:3300/0 conn(0x7f48ec1466b0 msgr2=0x7f48ec06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2070857182 >> v2:172.21.15.142:3300/0 conn(0x7f48ec1466b0 0x7f48ec06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.278+0000 7f48e3e00640 1 -- 172.21.15.46:0/2070857182 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec074050 con 0x7f48ec0720e0 2024-04-26T23:52:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.279+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2070857182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec0720e0 0x7f48ec0cd860 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754781917e23b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48e9400640 1 -- 172.21.15.46:0/2070857182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037070 con 0x7f48ec0720e0 2024-04-26T23:52:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48e9400640 1 -- 172.21.15.46:0/2070857182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d4032440 con 0x7f48ec0720e0 2024-04-26T23:52:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48e9400640 1 -- 172.21.15.46:0/2070857182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4032810 con 0x7f48ec0720e0 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 -- 172.21.15.46:0/2070857182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec0720e0 msgr2=0x7f48ec0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 --2- 172.21.15.46:0/2070857182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec0720e0 0x7f48ec0cd860 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402a210 comp rx=0 tx=0).stop 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 -- 172.21.15.46:0/2070857182 shutdown_connections 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 --2- 172.21.15.46:0/2070857182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec160910 0x7f48ec160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 --2- 172.21.15.46:0/2070857182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec0720e0 0x7f48ec0cd860 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 --2- 172.21.15.46:0/2070857182 >> v2:172.21.15.142:3300/0 conn(0x7f48ec1466b0 0x7f48ec06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.280+0000 7f48eb200640 1 -- 172.21.15.46:0/2070857182 >> 172.21.15.46:0/2070857182 conn(0x7f48ec0b26d0 msgr2=0x7f48ec0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.281+0000 7f48eb200640 1 -- 172.21.15.46:0/2070857182 shutdown_connections 2024-04-26T23:52:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.281+0000 7f48eb200640 1 -- 172.21.15.46:0/2070857182 wait complete. 2024-04-26T23:52:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.283+0000 7f48eb200640 1 Processor -- start 2024-04-26T23:52:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.283+0000 7f48eb200640 1 -- start start 2024-04-26T23:52:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 0x7f48ec1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 0x7f48ec0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec06e3c0 con 0x7f48ec1466b0 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cb470 con 0x7f48ec160910 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48eb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0cdda0 con 0x7f48ec0720e0 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36902/0 (socket says 172.21.15.46:36902) 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 -- 172.21.15.46:0/4157037244 learned_addr learned my addr 172.21.15.46:0/4157037244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48ea800640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 0x7f48ec0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e9e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 0x7f48ec1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 -- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 msgr2=0x7f48ec1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 0x7f48ec1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.284+0000 7f48e3e00640 1 -- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 msgr2=0x7f48ec0b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e3e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 0x7f48ec0b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e3e00640 1 -- 172.21.15.46:0/4157037244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec140240 con 0x7f48ec1466b0 2024-04-26T23:52:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e9e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 0x7f48ec1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48ea800640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 0x7f48ec0b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e3e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4041020 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d4030970 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037040 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec140520 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.285+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec140ae0 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.287+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0693b0 con 0x7f48ec1466b0 2024-04-26T23:52:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.288+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48d4044020 con 0x7f48ec1466b0 2024-04-26T23:52:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.289+0000 7f48e2000640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.289+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f48d40ac7a0 con 0x7f48ec1466b0 2024-04-26T23:52:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.289+0000 7f48e9e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.291+0000 7f48e9e00640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 0x7f48c806edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b820 tx=0x7f48dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.294+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48d404b030 con 0x7f48ec1466b0 2024-04-26T23:52:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.540+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48ec0c7180 con 0x7f48ec1466b0 2024-04-26T23:52:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.542+0000 7f48e2000640 1 -- 172.21.15.46:0/4157037244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f48d407a970 con 0x7f48ec1466b0 2024-04-26T23:52:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:13.545 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 msgr2=0x7f48c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 0x7f48c806edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b820 tx=0x7f48dc008040 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 msgr2=0x7f48ec1f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 shutdown_connections 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec160910 0x7f48ec0b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec1466b0 0x7f48ec1f7e20 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 --2- 172.21.15.46:0/4157037244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48ec0720e0 0x7f48ec1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 >> 172.21.15.46:0/4157037244 conn(0x7f48ec0b26d0 msgr2=0x7f48ec06efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 shutdown_connections 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.545+0000 7f48eb200640 1 -- 172.21.15.46:0/4157037244 wait complete. 2024-04-26T23:52:13.567 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:13.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.772+0000 7fe836200640 1 Processor -- start 2024-04-26T23:52:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.773+0000 7fe836200640 1 -- start start 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.774+0000 7fe836200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 0x7fe8300b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.774+0000 7fe836200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe830072c30 0x7fe830073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.774+0000 7fe836200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8300b41b0 con 0x7fe83015bf50 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.774+0000 7fe836200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe83015c330 con 0x7fe830072c30 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.774+0000 7fe836200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8300735c0 con 0x7fe83006f8f0 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe830072c30 0x7fe830073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe830072c30 0x7fe830073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36910/0 (socket says 172.21.15.46:36910) 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 -- 172.21.15.46:0/1153088404 learned_addr learned my addr 172.21.15.46:0/1153088404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe82fe00640 1 --2- 172.21.15.46:0/1153088404 >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 0x7fe8300b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 -- 172.21.15.46:0/1153088404 >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 msgr2=0x7fe8300b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 --2- 172.21.15.46:0/1153088404 >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 0x7fe8300b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 -- 172.21.15.46:0/1153088404 >> v1:172.21.15.142:6789/0 conn(0x7fe83015bf50 legacy=0x7fe8300ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe835800640 1 -- 172.21.15.46:0/1153088404 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8300b4490 con 0x7fe830072c30 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.775+0000 7fe82fe00640 1 --2- 172.21.15.46:0/1153088404 >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 0x7fe8300b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe835800640 1 --2- 172.21.15.46:0/1153088404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe830073080 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe83006e390 tx=0x7fe82802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa505e877bc0d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe82f400640 1 -- 172.21.15.46:0/1153088404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82800b2e0 con 0x7fe830072c30 2024-04-26T23:52:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe82f400640 1 -- 172.21.15.46:0/1153088404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe828037070 con 0x7fe830072c30 2024-04-26T23:52:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe82f400640 1 -- 172.21.15.46:0/1153088404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82800b480 con 0x7fe830072c30 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe836200640 1 -- 172.21.15.46:0/1153088404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 msgr2=0x7fe830073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.777+0000 7fe836200640 1 --2- 172.21.15.46:0/1153088404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe830073080 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe83006e390 tx=0x7fe82802c940 comp rx=0 tx=0).stop 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 -- 172.21.15.46:0/1153088404 shutdown_connections 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 --2- 172.21.15.46:0/1153088404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe830073080 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 --2- 172.21.15.46:0/1153088404 >> v2:172.21.15.142:3300/0 conn(0x7fe83006f8f0 0x7fe8300b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 -- 172.21.15.46:0/1153088404 >> 172.21.15.46:0/1153088404 conn(0x7fe8300b2820 msgr2=0x7fe8300b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 -- 172.21.15.46:0/1153088404 shutdown_connections 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.778+0000 7fe836200640 1 -- 172.21.15.46:0/1153088404 wait complete. 2024-04-26T23:52:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.780+0000 7fe836200640 1 Processor -- start 2024-04-26T23:52:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.780+0000 7fe836200640 1 -- start start 2024-04-26T23:52:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83006f8f0 0x7fe8300aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83015bf50 0x7fe8301528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8300739f0 con 0x7fe830072c30 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8300b4a50 con 0x7fe83006f8f0 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe836200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8300b5250 con 0x7fe83015bf50 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe82fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe82fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36914/0 (socket says 172.21.15.46:36914) 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe82fe00640 1 -- 172.21.15.46:0/2627078481 learned_addr learned my addr 172.21.15.46:0/2627078481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.781+0000 7fe834e00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83006f8f0 0x7fe8300aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe835800640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83015bf50 0x7fe8301528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 -- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83015bf50 msgr2=0x7fe8301528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83015bf50 0x7fe8301528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 -- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83006f8f0 msgr2=0x7fe8300aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83006f8f0 0x7fe8300aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 -- 172.21.15.46:0/2627078481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83015d2c0 con 0x7fe830072c30 2024-04-26T23:52:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82fe00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e7e0 tx=0x7fe81c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00f040 con 0x7fe830072c30 2024-04-26T23:52:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c004030 con 0x7fe830072c30 2024-04-26T23:52:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c004400 con 0x7fe830072c30 2024-04-26T23:52:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.782+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83015d5a0 con 0x7fe830072c30 2024-04-26T23:52:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.783+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83015db60 con 0x7fe830072c30 2024-04-26T23:52:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.784+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8300693b0 con 0x7fe830072c30 2024-04-26T23:52:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.785+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe81c00c6d0 con 0x7fe830072c30 2024-04-26T23:52:13.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.786+0000 7fe82cc00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.786+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fe81c08bab0 con 0x7fe830072c30 2024-04-26T23:52:13.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.787+0000 7fe834e00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.788+0000 7fe834e00640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 0x7fe81006edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe824007fd0 tx=0x7fe824009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:13.790+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe81c017020 con 0x7fe830072c30 2024-04-26T23:52:14.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.061+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe8300b33d0 con 0x7fe830072c30 2024-04-26T23:52:14.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.062+0000 7fe82cc00640 1 -- 172.21.15.46:0/2627078481 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2229 (secure 0 0 0) 0x7fe81c059d10 con 0x7fe830072c30 2024-04-26T23:52:14.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:14.065 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":925,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076966141,"bytes_used":2606358528,"bytes_avail":770487754752,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2445702,"recovering_keys_per_sec":0,"num_objects_recovered":28,"num_bytes_recovered":29360128,"num_keys_recovered":0,"write_bytes_sec":19650068,"read_op_per_sec":0,"write_op_per_sec":20},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-26T23:52:12.229010+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:14.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 msgr2=0x7fe81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 0x7fe81006edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe824007fd0 tx=0x7fe824009040 comp rx=0 tx=0).stop 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 msgr2=0x7fe8301523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e7e0 tx=0x7fe81c00eca0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 shutdown_connections 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe83015bf50 0x7fe8301528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe830072c30 0x7fe8301523b0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 --2- 172.21.15.46:0/2627078481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83006f8f0 0x7fe8300aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.064+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 >> 172.21.15.46:0/2627078481 conn(0x7fe8300b2820 msgr2=0x7fe8300b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.065+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 shutdown_connections 2024-04-26T23:52:14.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.065+0000 7fe836200640 1 -- 172.21.15.46:0/2627078481 wait complete. 2024-04-26T23:52:14.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:52:14.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:14.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub abort 2024-04-26T23:52:14.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.303+0000 7f0986000640 1 Processor -- start 2024-04-26T23:52:14.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.303+0000 7f0986000640 1 -- start start 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0986000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f0980073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0986000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09800cfed0 0x7f09800d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0986000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f098006ec30 con 0x7f09800d0dc0 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0986000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0980145ff0 con 0x7f09800cfed0 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0986000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09800d1230 con 0x7f0980145c10 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f097fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09800cfed0 0x7f09800d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f097fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09800cfed0 0x7f09800d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36916/0 (socket says 172.21.15.46:36916) 2024-04-26T23:52:14.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f097fe00640 1 -- 172.21.15.46:0/3860041911 learned_addr learned my addr 172.21.15.46:0/3860041911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.305+0000 7f0984c00640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f0980073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.306+0000 7f097fe00640 1 -- 172.21.15.46:0/3860041911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 msgr2=0x7f0980073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.306+0000 7f097fe00640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f0980073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.306+0000 7f097fe00640 1 -- 172.21.15.46:0/3860041911 >> v1:172.21.15.142:6789/0 conn(0x7f09800d0dc0 legacy=0x7f0980071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.306+0000 7f097fe00640 1 -- 172.21.15.46:0/3860041911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09800cf3f0 con 0x7f09800cfed0 2024-04-26T23:52:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.306+0000 7f0984c00640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f0980073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.307+0000 7f097fe00640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800d0880 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0968009e80 tx=0x7f096802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67b3f4dd107feeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.307+0000 7f097f400640 1 -- 172.21.15.46:0/3860041911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968036070 con 0x7f09800cfed0 2024-04-26T23:52:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.307+0000 7f097f400640 1 -- 172.21.15.46:0/3860041911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0968031600 con 0x7f09800cfed0 2024-04-26T23:52:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.307+0000 7f097f400640 1 -- 172.21.15.46:0/3860041911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680319d0 con 0x7f09800cfed0 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 -- 172.21.15.46:0/3860041911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 msgr2=0x7f09800d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800d0880 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0968009e80 tx=0x7f096802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 -- 172.21.15.46:0/3860041911 shutdown_connections 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800d0880 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 --2- 172.21.15.46:0/3860041911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f0980073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 -- 172.21.15.46:0/3860041911 >> 172.21.15.46:0/3860041911 conn(0x7f098006c350 msgr2=0x7f098006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 -- 172.21.15.46:0/3860041911 shutdown_connections 2024-04-26T23:52:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.308+0000 7f0986000640 1 -- 172.21.15.46:0/3860041911 wait complete. 2024-04-26T23:52:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.310+0000 7f0986000640 1 Processor -- start 2024-04-26T23:52:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.310+0000 7f0986000640 1 -- start start 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09800d0dc0 0x7f09800b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f09800b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09800cf0a0 con 0x7f09800cfed0 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09800cf9b0 con 0x7f09800d0dc0 2024-04-26T23:52:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0986000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09801468b0 con 0x7f0980145c10 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36922/0 (socket says 172.21.15.46:36922) 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.311+0000 7f0984c00640 1 -- 172.21.15.46:0/2635288459 learned_addr learned my addr 172.21.15.46:0/2635288459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 -- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 msgr2=0x7f09800b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f09800b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 -- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09800d0dc0 msgr2=0x7f09800b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09800d0dc0 0x7f09800b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 -- 172.21.15.46:0/2635288459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0980143db0 con 0x7f09800cfed0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0984c00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f097000c820 tx=0x7f097000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f097cc00640 1 -- 172.21.15.46:0/2635288459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970015070 con 0x7f09800cfed0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f097cc00640 1 -- 172.21.15.46:0/2635288459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0970007a30 con 0x7f09800cfed0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f097cc00640 1 -- 172.21.15.46:0/2635288459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970007de0 con 0x7f09800cfed0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.312+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0980144090 con 0x7f09800cfed0 2024-04-26T23:52:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.313+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0980144650 con 0x7f09800cfed0 2024-04-26T23:52:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.315+0000 7f097cc00640 1 -- 172.21.15.46:0/2635288459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0970018070 con 0x7f09800cfed0 2024-04-26T23:52:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.316+0000 7f097cc00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.316+0000 7f097cc00640 1 -- 172.21.15.46:0/2635288459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f097008bb70 con 0x7f09800cfed0 2024-04-26T23:52:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.317+0000 7f098a464740 1 Processor -- start 2024-04-26T23:52:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.317+0000 7f098a464740 1 -- start start 2024-04-26T23:52:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.317+0000 7f097fe00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c449e0 0x55e3f4c44f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3f4b60900 con 0x55e3f4bfce20 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e3f4afd1d0 con 0x55e3f4c27140 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e3f4b5c8c0 con 0x55e3f4c449e0 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f097fe00640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 0x7f095c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0968033000 tx=0x7f096802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f0985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f0985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36928/0 (socket says 172.21.15.46:36928) 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.318+0000 7f0985600640 1 -- 172.21.15.46:0/3657558189 learned_addr learned my addr 172.21.15.46:0/3657558189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f097fe00640 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c449e0 0x55e3f4c44f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0984c00640 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 -- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c449e0 msgr2=0x55e3f4c44f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c449e0 0x55e3f4c44f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 -- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 msgr2=0x55e3f4c27520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c27520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 -- 172.21.15.46:0/3657558189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f493c7d0 con 0x55e3f4bfce20 2024-04-26T23:52:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0985600640 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0978003d40 tx=0x7f0978004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ccaa5d6a58d587 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0975200640 1 -- 172.21.15.46:0/3657558189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978017070 con 0x55e3f4bfce20 2024-04-26T23:52:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0975200640 1 -- 172.21.15.46:0/3657558189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0978005480 con 0x55e3f4bfce20 2024-04-26T23:52:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.319+0000 7f0975200640 1 -- 172.21.15.46:0/3657558189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097800c8e0 con 0x55e3f4bfce20 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 -- 172.21.15.46:0/3657558189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 msgr2=0x55e3f4c430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0978003d40 tx=0x7f0978004200 comp rx=0 tx=0).stop 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 -- 172.21.15.46:0/3657558189 shutdown_connections 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c430a0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c449e0 0x55e3f4c44f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 --2- 172.21.15.46:0/3657558189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c27520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 -- 172.21.15.46:0/3657558189 >> 172.21.15.46:0/3657558189 conn(0x55e3f4bf9750 msgr2=0x55e3f4c41f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 -- 172.21.15.46:0/3657558189 shutdown_connections 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.320+0000 7f098a464740 1 -- 172.21.15.46:0/3657558189 wait complete. 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.321+0000 7f098a464740 1 -- start start 2024-04-26T23:52:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.321+0000 7f098a464740 1 Processor -- start 2024-04-26T23:52:14.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c27140 0x55e3f4c56250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c39bc0 0x55e3f4c42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3f4bfdbc0 con 0x55e3f4c449e0 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e3f4ae8120 con 0x55e3f4c39bc0 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e3f4bf7680 con 0x55e3f4c27140 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.322+0000 7f098a464740 10 client.0 _ll_register_callbacks cb 0x55e3f4b20750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36940/0 (socket says 172.21.15.46:36940) 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 -- 172.21.15.46:0/1100257677 learned_addr learned my addr 172.21.15.46:0/1100257677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f0985600640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c39bc0 0x55e3f4c42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f0984c00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c27140 0x55e3f4c56250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 -- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c27140 msgr2=0x55e3f4c56250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c27140 0x55e3f4c56250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 -- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c39bc0 msgr2=0x55e3f4c42b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c39bc0 0x55e3f4c42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 -- 172.21.15.46:0/1100257677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f48320f0 con 0x55e3f4c449e0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f097fe00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0968034040 tx=0x7f0968004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201607970c411adc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.323+0000 7f096d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096804d060 con 0x55e3f4c449e0 2024-04-26T23:52:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f096803b850 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0978005480 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f0968053050 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f098a464740 10 client.9954 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55e3f499cf60 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0978005480 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f0968056020 con 0x55e3f4c449e0 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f098a464740 10 client.9954 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.324+0000 7f098a464740 10 client.9954 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f098a464740 20 client.9954 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c22af0 0x55e3f4c43040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f098a464740 4 client.9954 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3f4b6e510 con 0x55e3f4c22af0 2024-04-26T23:52:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f0984c00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c22af0 0x55e3f4c43040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f0984c00640 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c22af0 0x55e3f4c43040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e95ce0d1fbff06f server_cookie=3805c78a26c84a4d in_seq=0 out_seq=0 2024-04-26T23:52:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.325+0000 7f096d600640 10 client.9954 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.326+0000 7f096d600640 1 -- 172.21.15.46:0/1100257677 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3f4b6e510 con 0x55e3f4c22af0 2024-04-26T23:52:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.326+0000 7f096d600640 10 client.9954 handle_command_reply: tid=0 2024-04-26T23:52:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.326+0000 7f098a464740 1 client.9954 shutdown 2024-04-26T23:52:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 msgr2=0x55e3f4c29ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0968034040 tx=0x7f0968004ab0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 shutdown_connections 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c22af0 0x55e3f4c43040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c39bc0 0x55e3f4c42b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4c449e0 0x55e3f4c29ec0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 --2- 172.21.15.46:0/1100257677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c27140 0x55e3f4c56250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f096d600640 0 client.9954 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 >> 172.21.15.46:0/1100257677 conn(0x55e3f4bfce20 msgr2=0x55e3f4c417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 shutdown_connections 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 1 -- 172.21.15.46:0/1100257677 wait complete. 2024-04-26T23:52:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.327+0000 7f098a464740 20 client.9954 trim_cache size 0 max 16384 2024-04-26T23:52:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.346+0000 7f098a464740 1 Processor -- start 2024-04-26T23:52:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.346+0000 7f098a464740 1 -- start start 2024-04-26T23:52:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c41840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4c38e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0980201e00 con 0x55e3f4bfce20 2024-04-26T23:52:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0980200e70 con 0x55e3f4c27140 2024-04-26T23:52:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0980200230 con 0x55e3f4c39bc0 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36952/0 (socket says 172.21.15.46:36952) 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.347+0000 7f0984c00640 1 -- 172.21.15.46:0/3708051223 learned_addr learned my addr 172.21.15.46:0/3708051223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0985600640 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4c38e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f097fe00640 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c41840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 -- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 msgr2=0x55e3f4c38e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4c38e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 -- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 msgr2=0x55e3f4c41840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c41840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 -- 172.21.15.46:0/3708051223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f480b1b0 con 0x55e3f4bfce20 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f0984c00640 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0970007410 tx=0x7f0970060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1515e0384a10111d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f096fe00640 1 -- 172.21.15.46:0/3708051223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097001f040 con 0x55e3f4bfce20 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f096fe00640 1 -- 172.21.15.46:0/3708051223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097006f070 con 0x55e3f4bfce20 2024-04-26T23:52:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.348+0000 7f096fe00640 1 -- 172.21.15.46:0/3708051223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097000ce10 con 0x55e3f4bfce20 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 -- 172.21.15.46:0/3708051223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 msgr2=0x55e3f4c382e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0970007410 tx=0x7f0970060040 comp rx=0 tx=0).stop 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 -- 172.21.15.46:0/3708051223 shutdown_connections 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4c38e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4c41840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 --2- 172.21.15.46:0/3708051223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c382e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 -- 172.21.15.46:0/3708051223 >> 172.21.15.46:0/3708051223 conn(0x55e3f4c3b080 msgr2=0x55e3f4c42a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 -- 172.21.15.46:0/3708051223 shutdown_connections 2024-04-26T23:52:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.349+0000 7f098a464740 1 -- 172.21.15.46:0/3708051223 wait complete. 2024-04-26T23:52:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.350+0000 7f098a464740 1 -- start start 2024-04-26T23:52:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.350+0000 7f098a464740 1 Processor -- start 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4cb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4cb63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0980200820 con 0x55e3f4bfce20 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e3f4c43150 con 0x55e3f4c27140 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e3f4c39990 con 0x55e3f4c39bc0 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.351+0000 7f098a464740 10 client.0 _ll_register_callbacks cb 0x7f0968034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36958/0 (socket says 172.21.15.46:36958) 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 -- 172.21.15.46:0/1581669330 learned_addr learned my addr 172.21.15.46:0/1581669330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f097fe00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4cb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0985600640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4cb63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 -- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 msgr2=0x55e3f4cb63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4cb63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 -- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 msgr2=0x55e3f4cb5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4cb5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 -- 172.21.15.46:0/1581669330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f4cbf990 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f097fe00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4cb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f0984c00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0970089f40 tx=0x7f0970013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfba09e59f0fb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f096d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.352+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970089070 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f0970055de0 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f097006f070 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f097001f040 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 10 client.9960 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55e3f4c5bba0 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097006f070 con 0x55e3f4bfce20 2024-04-26T23:52:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f0970058bc0 con 0x55e3f4bfce20 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 10 client.9960 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 10 client.9960 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 20 client.9960 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c449e0 0x55e3f4c453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 4 client.9960 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55e3f4b68380 con 0x55e3f4c449e0 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.353+0000 7f097fe00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c449e0 0x55e3f4c453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.354+0000 7f097fe00640 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c449e0 0x55e3f4c453f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ad205abb9af7ba server_cookie=be0518cc892ce25d in_seq=0 out_seq=0 2024-04-26T23:52:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.354+0000 7f096d600640 10 client.9960 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.354+0000 7f096d600640 1 -- 172.21.15.46:0/1581669330 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e3f4b68380 con 0x55e3f4c449e0 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.354+0000 7f096d600640 10 client.9960 handle_command_reply: tid=0 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.354+0000 7f098a464740 1 client.9960 shutdown 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 msgr2=0x55e3f4c427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0970089f40 tx=0x7f0970013040 comp rx=0 tx=0).stop 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 shutdown_connections 2024-04-26T23:52:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55e3f4c449e0 0x55e3f4c453f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e3f4c39bc0 0x55e3f4cb63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3f4c27140 0x55e3f4cb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 --2- 172.21.15.46:0/1581669330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3f4bfce20 0x55e3f4c427f0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 >> 172.21.15.46:0/1581669330 conn(0x55e3f4c3b080 msgr2=0x55e3f4c37970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f096d600640 0 client.9960 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 shutdown_connections 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 1 -- 172.21.15.46:0/1581669330 wait complete. 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.355+0000 7f098a464740 20 client.9960 trim_cache size 0 max 16384 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:52:14.361 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 msgr2=0x7f095c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 0x7f095c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0968033000 tx=0x7f096802e000 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 msgr2=0x7f09800b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f097000c820 tx=0x7f097000cce0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 shutdown_connections 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0980145c10 0x7f09800b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09800d0dc0 0x7f09800b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 --2- 172.21.15.46:0/2635288459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09800cfed0 0x7f09800b0640 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 >> 172.21.15.46:0/2635288459 conn(0x7f098006c350 msgr2=0x7f09800b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 shutdown_connections 2024-04-26T23:52:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.356+0000 7f0986000640 1 -- 172.21.15.46:0/2635288459 wait complete. 2024-04-26T23:52:14.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:14.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub status 2024-04-26T23:52:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.579+0000 7ffa8be00640 1 Processor -- start 2024-04-26T23:52:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.579+0000 7ffa8be00640 1 -- start start 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa8be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa8415d730 0x7ffa840d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415e220 0x7ffa8406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa8be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8406eb90 con 0x7ffa840b53b0 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa8be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffa840b4310 con 0x7ffa8415d730 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa8be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840b45f0 con 0x7ffa8415e220 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa89400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa8415d730 0x7ffa840d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa88a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa840b53b0 0x7ffa8415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54596/0 (socket says 172.21.15.46:54596) 2024-04-26T23:52:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa88a00640 1 -- 172.21.15.46:0/651813672 learned_addr learned my addr 172.21.15.46:0/651813672 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.581+0000 7ffa89e00640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415e220 0x7ffa8406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.582+0000 7ffa89400640 1 -- 172.21.15.46:0/651813672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415e220 msgr2=0x7ffa8406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.582+0000 7ffa89400640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415e220 0x7ffa8406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.582+0000 7ffa89400640 1 -- 172.21.15.46:0/651813672 >> v1:172.21.15.46:6789/0 conn(0x7ffa840b53b0 legacy=0x7ffa8415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.582+0000 7ffa89400640 1 -- 172.21.15.46:0/651813672 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa840b48d0 con 0x7ffa8415d730 2024-04-26T23:52:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.583+0000 7ffa89400640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa8415d730 0x7ffa840d00d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097a0 tx=0x7ffa7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d9f4879922c034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.583+0000 7ffa83e00640 1 -- 172.21.15.46:0/651813672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78037070 con 0x7ffa8415d730 2024-04-26T23:52:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.583+0000 7ffa83e00640 1 -- 172.21.15.46:0/651813672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa78032440 con 0x7ffa8415d730 2024-04-26T23:52:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.583+0000 7ffa83e00640 1 -- 172.21.15.46:0/651813672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78032810 con 0x7ffa8415d730 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 -- 172.21.15.46:0/651813672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa8415d730 msgr2=0x7ffa840d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa8415d730 0x7ffa840d00d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097a0 tx=0x7ffa7802a180 comp rx=0 tx=0).stop 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 -- 172.21.15.46:0/651813672 shutdown_connections 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415e220 0x7ffa8406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 --2- 172.21.15.46:0/651813672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa8415d730 0x7ffa840d00d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 -- 172.21.15.46:0/651813672 >> 172.21.15.46:0/651813672 conn(0x7ffa8406c980 msgr2=0x7ffa840b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 -- 172.21.15.46:0/651813672 shutdown_connections 2024-04-26T23:52:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.584+0000 7ffa8be00640 1 -- 172.21.15.46:0/651813672 wait complete. 2024-04-26T23:52:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.586+0000 7ffa8be00640 1 Processor -- start 2024-04-26T23:52:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.587+0000 7ffa8be00640 1 -- start start 2024-04-26T23:52:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415d730 0x7ffa84140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8415e220 0x7ffa84140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840b3ec0 con 0x7ffa840b53b0 2024-04-26T23:52:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840b4e90 con 0x7ffa8415e220 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa8be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840b4040 con 0x7ffa8415d730 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36984/0 (socket says 172.21.15.46:36984) 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 -- 172.21.15.46:0/282344846 learned_addr learned my addr 172.21.15.46:0/282344846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 -- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415d730 msgr2=0x7ffa84140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415d730 0x7ffa84140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 -- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8415e220 msgr2=0x7ffa84140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89e00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8415e220 0x7ffa84140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8415e220 0x7ffa84140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.588+0000 7ffa89400640 1 -- 172.21.15.46:0/282344846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa841f8ee0 con 0x7ffa840b53b0 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa89400640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ffa780098c0 tx=0x7ffa780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa81600640 1 -- 172.21.15.46:0/282344846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78037040 con 0x7ffa840b53b0 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa81600640 1 -- 172.21.15.46:0/282344846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa78030770 con 0x7ffa840b53b0 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa81600640 1 -- 172.21.15.46:0/282344846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78030b40 con 0x7ffa840b53b0 2024-04-26T23:52:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa841f6230 con 0x7ffa840b53b0 2024-04-26T23:52:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.589+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8413f540 con 0x7ffa840b53b0 2024-04-26T23:52:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.592+0000 7ffa81600640 1 -- 172.21.15.46:0/282344846 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa78044020 con 0x7ffa840b53b0 2024-04-26T23:52:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.593+0000 7ffa81600640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 0x7ffa6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.593+0000 7ffa81600640 1 -- 172.21.15.46:0/282344846 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7ffa780ac200 con 0x7ffa840b53b0 2024-04-26T23:52:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.593+0000 7ffa8ebf8740 1 Processor -- start 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.593+0000 7ffa88a00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 0x7ffa6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.594+0000 7ffa8ebf8740 1 -- start start 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea04920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea21de0 0x560a0ea22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560a0e8ea030 con 0x560a0e9da220 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560a0e9057f0 con 0x560a0ea21de0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa88a00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 0x7ffa6406edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000f60 tx=0x7ffa70007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560a0e8c07e0 con 0x560a0ea04540 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36998/0 (socket says 172.21.15.46:36998) 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa89e00640 1 -- 172.21.15.46:0/450255005 learned_addr learned my addr 172.21.15.46:0/450255005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.595+0000 7ffa88a00640 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea21de0 0x560a0ea22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89400640 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea04920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 -- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 msgr2=0x560a0ea04920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea04920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 -- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea21de0 msgr2=0x560a0ea22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea21de0 0x560a0ea22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 -- 172.21.15.46:0/450255005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0e8cc050 con 0x560a0e9da220 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa89e00640 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c007330 tx=0x7ffa7c0077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8fa4c3a2f502 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa75c00640 1 -- 172.21.15.46:0/450255005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c013780 con 0x560a0e9da220 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa75c00640 1 -- 172.21.15.46:0/450255005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c019070 con 0x560a0e9da220 2024-04-26T23:52:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.596+0000 7ffa75c00640 1 -- 172.21.15.46:0/450255005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c013b40 con 0x560a0e9da220 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/450255005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 msgr2=0x560a0ea204a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c007330 tx=0x7ffa7c0077f0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/450255005 shutdown_connections 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0e9da220 0x560a0ea204a0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea21de0 0x560a0ea22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/450255005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea04920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/450255005 >> 172.21.15.46:0/450255005 conn(0x560a0e9d6b50 msgr2=0x560a0ea1f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/450255005 shutdown_connections 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.597+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/450255005 wait complete. 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.598+0000 7ffa8ebf8740 1 -- start start 2024-04-26T23:52:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.598+0000 7ffa8ebf8740 1 Processor -- start 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0e9f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560a0e942200 con 0x560a0ea21de0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560a0e93cee0 con 0x560a0ea04540 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560a0e949e70 con 0x560a0ea193a0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa8ebf8740 10 client.0 _ll_register_callbacks cb 0x560a0e933270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa88a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37002/0 (socket says 172.21.15.46:37002) 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.599+0000 7ffa88a00640 1 -- 172.21.15.46:0/3364887995 learned_addr learned my addr 172.21.15.46:0/3364887995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa89400640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0e9f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa89e00640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 -- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 msgr2=0x560a0ea176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 -- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 msgr2=0x560a0e9f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0e9f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 -- 172.21.15.46:0/3364887995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0e7197d0 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa89400640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0e9f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa88a00640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ffa700076b0 tx=0x7ffa70000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8a4f4fd292f521 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa6a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70016b50 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7ffa70016cf0 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.600+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c019070 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7ffa70014430 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 10 client.9975 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x560a0e64f3f0 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c019070 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7ffa7002a050 con 0x560a0ea21de0 2024-04-26T23:52:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 10 client.9975 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 10 client.9975 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 20 client.9975 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0ea17bf0 0x560a0ea22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 4 client.9975 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a0e94b7e0 con 0x560a0ea17bf0 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.601+0000 7ffa89400640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0ea17bf0 0x560a0ea22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.602+0000 7ffa89400640 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0ea17bf0 0x560a0ea22da0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6180d7619d76d2f1 server_cookie=ebf9f3a21df01704 in_seq=0 out_seq=0 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.602+0000 7ffa6a000640 10 client.9975 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa6a000640 1 -- 172.21.15.46:0/3364887995 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a0e94b7e0 con 0x560a0ea17bf0 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa6a000640 10 client.9975 handle_command_reply: tid=0 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 client.9975 shutdown 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 msgr2=0x560a0ea1c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ffa700076b0 tx=0x7ffa70000ed0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 shutdown_connections 2024-04-26T23:52:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0ea17bf0 0x560a0ea22da0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea1c1f0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3364887995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0e9f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 >> 172.21.15.46:0/3364887995 conn(0x560a0e9da220 msgr2=0x560a0ea1f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa6a000640 0 client.9975 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 shutdown_connections 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3364887995 wait complete. 2024-04-26T23:52:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.603+0000 7ffa8ebf8740 20 client.9975 trim_cache size 0 max 16384 2024-04-26T23:52:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.623+0000 7ffa8ebf8740 1 Processor -- start 2024-04-26T23:52:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.623+0000 7ffa8ebf8740 1 -- start start 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0ea97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea97d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84201f60 con 0x560a0ea21de0 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84200fe0 con 0x560a0ea04540 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa84073450 con 0x560a0ea193a0 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37010/0 (socket says 172.21.15.46:37010) 2024-04-26T23:52:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa89e00640 1 -- 172.21.15.46:0/2889135140 learned_addr learned my addr 172.21.15.46:0/2889135140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa88a00640 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea97d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.624+0000 7ffa89400640 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0ea97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 -- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 msgr2=0x560a0ea97d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea97d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 -- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 msgr2=0x560a0ea97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0ea97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 -- 172.21.15.46:0/2889135140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0e60f0f0 con 0x560a0ea21de0 2024-04-26T23:52:14.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa89e00640 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c016040 tx=0x7ffa7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d04c32ad369979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.625+0000 7ffa75200640 1 -- 172.21.15.46:0/2889135140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c003b90 con 0x560a0ea21de0 2024-04-26T23:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa75200640 1 -- 172.21.15.46:0/2889135140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c023070 con 0x560a0ea21de0 2024-04-26T23:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa75200640 1 -- 172.21.15.46:0/2889135140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c004040 con 0x560a0ea21de0 2024-04-26T23:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/2889135140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 msgr2=0x560a0ea14ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c016040 tx=0x7ffa7c013000 comp rx=0 tx=0).stop 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/2889135140 shutdown_connections 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea14ed0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea193a0 0x560a0ea97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/2889135140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea04540 0x560a0ea97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/2889135140 >> 172.21.15.46:0/2889135140 conn(0x560a0e9da220 msgr2=0x560a0e9d96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.626+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/2889135140 shutdown_connections 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.627+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/2889135140 wait complete. 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.627+0000 7ffa8ebf8740 1 -- start start 2024-04-26T23:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.627+0000 7ffa8ebf8740 1 Processor -- start 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea193a0 0x560a0ea19d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560a0e94afd0 con 0x560a0ea21de0 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84073aa0 con 0x560a0ea193a0 2024-04-26T23:52:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560a0e9f4c80 con 0x560a0ea04540 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.628+0000 7ffa8ebf8740 10 client.0 _ll_register_callbacks cb 0x7ffa700058b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37024/0 (socket says 172.21.15.46:37024) 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 -- 172.21.15.46:0/3894422491 learned_addr learned my addr 172.21.15.46:0/3894422491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa88a00640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea193a0 0x560a0ea19d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89400640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 -- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 msgr2=0x560a0ea19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea19820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 -- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea193a0 msgr2=0x560a0ea19d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea193a0 0x560a0ea19d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.629+0000 7ffa89e00640 1 -- 172.21.15.46:0/3894422491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0e905a30 con 0x560a0ea21de0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa89e00640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c016040 tx=0x7ffa7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70179c407bc9e75e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c036060 con 0x560a0ea21de0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7ffa7c004450 con 0x560a0ea21de0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c023070 con 0x560a0ea21de0 2024-04-26T23:52:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c03c050 con 0x560a0ea21de0 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa8ebf8740 10 client.9981 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x560a0ea15930 con 0x560a0ea21de0 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c023070 con 0x560a0ea21de0 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.630+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7ffa7c03f020 con 0x560a0ea21de0 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 10 client.9981 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 10 client.9981 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 20 client.9981 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0eaa3c70 0x560a0ea079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 4 client.9981 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a0e945650 con 0x560a0eaa3c70 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa89400640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0eaa3c70 0x560a0ea079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa6a000640 10 client.9981 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.631+0000 7ffa89400640 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0eaa3c70 0x560a0ea079f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79164f2a6af7663c server_cookie=259ea7bb8fb21896 in_seq=0 out_seq=0 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa6a000640 1 -- 172.21.15.46:0/3894422491 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+69 (crc 0 0 0) 0x560a0e945650 con 0x560a0eaa3c70 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa6a000640 10 client.9981 handle_command_reply: tid=0 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 client.9981 shutdown 2024-04-26T23:52:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 msgr2=0x560a0ea074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c016040 tx=0x7ffa7c004ab0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 shutdown_connections 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x560a0eaa3c70 0x560a0ea079f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a0ea21de0 0x560a0ea074b0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a0ea193a0 0x560a0ea19d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 --2- 172.21.15.46:0/3894422491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560a0ea04540 0x560a0ea19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 >> 172.21.15.46:0/3894422491 conn(0x560a0e9da220 msgr2=0x560a0e9da600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa6a000640 0 client.9981 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 shutdown_connections 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 1 -- 172.21.15.46:0/3894422491 wait complete. 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.632+0000 7ffa8ebf8740 20 client.9981 trim_cache size 0 max 16384 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:52:14.645 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 msgr2=0x7ffa6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 0x7ffa6406edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000f60 tx=0x7ffa70007a80 comp rx=0 tx=0).stop 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 msgr2=0x7ffa8415a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ffa780098c0 tx=0x7ffa780046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 shutdown_connections 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa6406c8f0 0x7ffa6406edb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8415e220 0x7ffa84140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415d730 0x7ffa84140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 --2- 172.21.15.46:0/282344846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840b53b0 0x7ffa8415a8f0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 >> 172.21.15.46:0/282344846 conn(0x7ffa8406c980 msgr2=0x7ffa840b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 shutdown_connections 2024-04-26T23:52:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.633+0000 7ffa8be00640 1 -- 172.21.15.46:0/282344846 wait complete. 2024-04-26T23:52:14.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.861+0000 7f7a89c00640 1 Processor -- start 2024-04-26T23:52:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.861+0000 7f7a89c00640 1 -- start start 2024-04-26T23:52:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.862+0000 7f7a89c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7a8415f4e0 0x7f7a840d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.862+0000 7f7a89c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a840b4000 0x7f7a840d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a89c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a840d11f0 con 0x7f7a840d1f60 2024-04-26T23:52:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a89c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7a840d2990 con 0x7f7a8415f4e0 2024-04-26T23:52:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a89c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840d5f20 con 0x7f7a840b4000 2024-04-26T23:52:14.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a89200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7a840d1f60 0x7f7a840cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54604/0 (socket says 172.21.15.46:54604) 2024-04-26T23:52:14.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a89200640 1 -- 172.21.15.46:0/495577600 learned_addr learned my addr 172.21.15.46:0/495577600 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:14.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a83e00640 1 --2- 172.21.15.46:0/495577600 >> v2:172.21.15.142:3300/0 conn(0x7f7a8415f4e0 0x7f7a840d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.863+0000 7f7a7be00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a840b4000 0x7f7a840d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.864+0000 7f7a83e00640 1 -- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a840b4000 msgr2=0x7f7a840d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.864+0000 7f7a83e00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a840b4000 0x7f7a840d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.864+0000 7f7a83e00640 1 -- 172.21.15.46:0/495577600 >> v1:172.21.15.46:6789/0 conn(0x7f7a840d1f60 legacy=0x7f7a840cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.864+0000 7f7a83e00640 1 -- 172.21.15.46:0/495577600 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a840d0f10 con 0x7f7a8415f4e0 2024-04-26T23:52:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.865+0000 7f7a83e00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a8415f4e0 0x7f7a840d2df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a740097a0 tx=0x7f7a7402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5dcb339c18c938 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a83400640 1 -- 172.21.15.46:0/495577600 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74037070 con 0x7f7a8415f4e0 2024-04-26T23:52:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a83400640 1 -- 172.21.15.46:0/495577600 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a74032440 con 0x7f7a8415f4e0 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a83400640 1 -- 172.21.15.46:0/495577600 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74032810 con 0x7f7a8415f4e0 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 -- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a8415f4e0 msgr2=0x7f7a840d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a8415f4e0 0x7f7a840d2df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a740097a0 tx=0x7f7a7402a180 comp rx=0 tx=0).stop 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 -- 172.21.15.46:0/495577600 shutdown_connections 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a840b4000 0x7f7a840d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 --2- 172.21.15.46:0/495577600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a8415f4e0 0x7f7a840d2df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 -- 172.21.15.46:0/495577600 >> 172.21.15.46:0/495577600 conn(0x7f7a840d5010 msgr2=0x7f7a840d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.872+0000 7f7a89c00640 1 -- 172.21.15.46:0/495577600 shutdown_connections 2024-04-26T23:52:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.873+0000 7f7a89c00640 1 -- 172.21.15.46:0/495577600 wait complete. 2024-04-26T23:52:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.874+0000 7f7a89c00640 1 Processor -- start 2024-04-26T23:52:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.874+0000 7f7a89c00640 1 -- start start 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840b4000 0x7f7a841573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a8415f4e0 0x7f7a8415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8415f8c0 con 0x7f7a840d1f60 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a840b1d60 con 0x7f7a840b4000 2024-04-26T23:52:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.875+0000 7f7a89c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840b3290 con 0x7f7a8415f4e0 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37030/0 (socket says 172.21.15.46:37030) 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 -- 172.21.15.46:0/687610102 learned_addr learned my addr 172.21.15.46:0/687610102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a83e00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840b4000 0x7f7a841573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a89200640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a8415f4e0 0x7f7a8415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 -- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a8415f4e0 msgr2=0x7f7a8415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a8415f4e0 0x7f7a8415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 -- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840b4000 msgr2=0x7f7a841573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840b4000 0x7f7a841573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.876+0000 7f7a7be00640 1 -- 172.21.15.46:0/687610102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a840b47f0 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a7be00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00e9b0 tx=0x7f7a6c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c017070 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6c00c9a0 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00cd90 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a840b4ad0 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.877+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a840b5090 con 0x7f7a840d1f60 2024-04-26T23:52:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.878+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a840693b0 con 0x7f7a840d1f60 2024-04-26T23:52:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.879+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a6c01a070 con 0x7f7a840d1f60 2024-04-26T23:52:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.881+0000 7f7a80c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 0x7f7a6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.881+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f7a6c08c0f0 con 0x7f7a840d1f60 2024-04-26T23:52:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.881+0000 7f7a83e00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 0x7f7a6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.883+0000 7f7a83e00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 0x7f7a6006ede0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7a74034000 tx=0x7f7a7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:14.886+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a6c05a2c0 con 0x7f7a840d1f60 2024-04-26T23:52:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.133+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a840b79b0 con 0x7f7a840d1f60 2024-04-26T23:52:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.134+0000 7f7a80c00640 1 -- 172.21.15.46:0/687610102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f7a6c059bc0 con 0x7f7a840d1f60 2024-04-26T23:52:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:15.137 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:15.137 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 msgr2=0x7f7a6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 0x7f7a6006ede0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7a74034000 tx=0x7f7a7402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 msgr2=0x7f7a840b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00e9b0 tx=0x7f7a6c00ee70 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 shutdown_connections 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7a6006c920 0x7f7a6006ede0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a8415f4e0 0x7f7a8415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a840d1f60 0x7f7a840b6260 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 --2- 172.21.15.46:0/687610102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840b4000 0x7f7a841573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 >> 172.21.15.46:0/687610102 conn(0x7f7a840d5010 msgr2=0x7f7a841601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 shutdown_connections 2024-04-26T23:52:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.137+0000 7f7a89c00640 1 -- 172.21.15.46:0/687610102 wait complete. 2024-04-26T23:52:15.160 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:15.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.370+0000 7f23fb200640 1 Processor -- start 2024-04-26T23:52:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.371+0000 7f23fb200640 1 -- start start 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.372+0000 7f23fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4159950 0x7f23f40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.372+0000 7f23fb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23f40ae8e0 0x7f23f4162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.372+0000 7f23fb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f406f430 con 0x7f23f41631d0 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.372+0000 7f23fb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f23f4163620 con 0x7f23f40ae8e0 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.372+0000 7f23fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40af0c0 con 0x7f23f4159950 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23f40ae8e0 0x7f23f4162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23fa800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f23f41631d0 0x7f23f415f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54614/0 (socket says 172.21.15.46:54614) 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23f40ae8e0 0x7f23f4162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37034/0 (socket says 172.21.15.46:37034) 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23fa800640 1 -- 172.21.15.46:0/4112823846 learned_addr learned my addr 172.21.15.46:0/4112823846 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9e00640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4159950 0x7f23f40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 -- 172.21.15.46:0/4112823846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4159950 msgr2=0x7f23f40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4159950 0x7f23f40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 -- 172.21.15.46:0/4112823846 >> v1:172.21.15.46:6789/0 conn(0x7f23f41631d0 legacy=0x7f23f415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f9400640 1 -- 172.21.15.46:0/4112823846 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40af3a0 con 0x7f23f40ae8e0 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.373+0000 7f23f8a00640 1 -- 172.21.15.46:0/4112823846 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294205242 0 0) 0x7f23f406f430 con 0x7f23f41631d0 2024-04-26T23:52:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.374+0000 7f23f9400640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40ae8e0 0x7f23f4162c90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097a0 tx=0x7f23e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db9d089ed9212a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.375+0000 7f23f8a00640 1 -- 172.21.15.46:0/4112823846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4037070 con 0x7f23f40ae8e0 2024-04-26T23:52:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.375+0000 7f23f8a00640 1 -- 172.21.15.46:0/4112823846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e4032440 con 0x7f23f40ae8e0 2024-04-26T23:52:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.375+0000 7f23f8a00640 1 -- 172.21.15.46:0/4112823846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4032810 con 0x7f23f40ae8e0 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.375+0000 7f23fb200640 1 -- 172.21.15.46:0/4112823846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40ae8e0 msgr2=0x7f23f4162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.375+0000 7f23fb200640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40ae8e0 0x7f23f4162c90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097a0 tx=0x7f23e402a180 comp rx=0 tx=0).stop 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 -- 172.21.15.46:0/4112823846 shutdown_connections 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40ae8e0 0x7f23f4162c90 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 --2- 172.21.15.46:0/4112823846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4159950 0x7f23f40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 -- 172.21.15.46:0/4112823846 >> 172.21.15.46:0/4112823846 conn(0x7f23f40b2820 msgr2=0x7f23f40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 -- 172.21.15.46:0/4112823846 shutdown_connections 2024-04-26T23:52:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.376+0000 7f23fb200640 1 -- 172.21.15.46:0/4112823846 wait complete. 2024-04-26T23:52:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.377+0000 7f23fb200640 1 Processor -- start 2024-04-26T23:52:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.378+0000 7f23fb200640 1 -- start start 2024-04-26T23:52:15.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.378+0000 7f23fb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 0x7f23f4144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.378+0000 7f23fb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4141560 0x7f23f40738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23fb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40afc40 con 0x7f23f41631d0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23fb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40af960 con 0x7f23f40ae8e0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40b0160 con 0x7f23f4141560 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37038/0 (socket says 172.21.15.46:37038) 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 -- 172.21.15.46:0/2670187925 learned_addr learned my addr 172.21.15.46:0/2670187925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 -- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4141560 msgr2=0x7f23f40738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 0x7f23f4144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23fa800640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4141560 0x7f23f40738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4141560 0x7f23f40738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 -- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 msgr2=0x7f23f4144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 0x7f23f4144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9400640 1 -- 172.21.15.46:0/2670187925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f415fa50 con 0x7f23f41631d0 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.379+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 0x7f23f4144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:15.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23f9400640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097a0 tx=0x7f23e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4037050 con 0x7f23f41631d0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e4038040 con 0x7f23f41631d0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004470 con 0x7f23f41631d0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f4073df0 con 0x7f23f41631d0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.380+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f4161420 con 0x7f23f41631d0 2024-04-26T23:52:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.381+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40693b0 con 0x7f23f41631d0 2024-04-26T23:52:15.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.382+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23e4030900 con 0x7f23f41631d0 2024-04-26T23:52:15.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.384+0000 7f23ee000640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.384+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f23e40ac910 con 0x7f23f41631d0 2024-04-26T23:52:15.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.385+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.386+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 0x7f23d806edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f23e8006fd0 tx=0x7f23e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.388+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23f4073df0 con 0x7f23f41631d0 2024-04-26T23:52:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.680+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f23f40b35e0 con 0x7f23f41631d0 2024-04-26T23:52:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.681+0000 7f23ee000640 1 -- 172.21.15.46:0/2670187925 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2052 (secure 0 0 0) 0x7f23f415fa50 con 0x7f23f41631d0 2024-04-26T23:52:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:15.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":927,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076966141,"bytes_used":2606358528,"bytes_avail":770487754752,"bytes_total":773094113280,"write_bytes_sec":7481850,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-26T23:52:12.229010+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:15.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 msgr2=0x7f23d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 0x7f23d806edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f23e8006fd0 tx=0x7f23e8008040 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 msgr2=0x7f23f415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097a0 tx=0x7f23e40046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 shutdown_connections 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f4141560 0x7f23f40738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f41631d0 0x7f23f415f510 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 --2- 172.21.15.46:0/2670187925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f40ae8e0 0x7f23f4144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 >> 172.21.15.46:0/2670187925 conn(0x7f23f40b2820 msgr2=0x7f23f40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.684+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 shutdown_connections 2024-04-26T23:52:15.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.685+0000 7f23fb200640 1 -- 172.21.15.46:0/2670187925 wait complete. 2024-04-26T23:52:15.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:52:15.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:15.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub resume 2024-04-26T23:52:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.917+0000 7f1b7cc00640 1 Processor -- start 2024-04-26T23:52:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.917+0000 7f1b7cc00640 1 -- start start 2024-04-26T23:52:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b7cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 0x7f1b780cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b7cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b780b64c0 0x7f1b780d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b7cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780b46f0 con 0x7f1b78162aa0 2024-04-26T23:52:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b7cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780b44a0 con 0x7f1b780b64c0 2024-04-26T23:52:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b7cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780b5020 con 0x7f1b780d3d60 2024-04-26T23:52:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b76800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b780b64c0 0x7f1b780d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.918+0000 7f1b75e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b78162aa0 0x7f1b780d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54622/0 (socket says 172.21.15.46:54622) 2024-04-26T23:52:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b75e00640 1 -- 172.21.15.46:0/110105783 learned_addr learned my addr 172.21.15.46:0/110105783 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b75400640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 0x7f1b780cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b76800640 1 -- 172.21.15.46:0/110105783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 msgr2=0x7f1b780cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b76800640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 0x7f1b780cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b76800640 1 -- 172.21.15.46:0/110105783 >> v1:172.21.15.46:6789/0 conn(0x7f1b78162aa0 legacy=0x7f1b780d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b76800640 1 -- 172.21.15.46:0/110105783 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780b49d0 con 0x7f1b780b64c0 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.919+0000 7f1b75400640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 0x7f1b780cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.920+0000 7f1b76800640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780d01c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097a0 tx=0x7f1b7002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d27b84c60cc23d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.920+0000 7f1b74a00640 1 -- 172.21.15.46:0/110105783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70037070 con 0x7f1b780b64c0 2024-04-26T23:52:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b74a00640 1 -- 172.21.15.46:0/110105783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7002ae40 con 0x7f1b780b64c0 2024-04-26T23:52:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b74a00640 1 -- 172.21.15.46:0/110105783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70032660 con 0x7f1b780b64c0 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 -- 172.21.15.46:0/110105783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 msgr2=0x7f1b780d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780d01c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097a0 tx=0x7f1b7002a0f0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 -- 172.21.15.46:0/110105783 shutdown_connections 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780d01c0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/110105783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b780d3d60 0x7f1b780cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 -- 172.21.15.46:0/110105783 >> 172.21.15.46:0/110105783 conn(0x7f1b780d5070 msgr2=0x7f1b780d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 -- 172.21.15.46:0/110105783 shutdown_connections 2024-04-26T23:52:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.921+0000 7f1b7cc00640 1 -- 172.21.15.46:0/110105783 wait complete. 2024-04-26T23:52:15.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.923+0000 7f1b7cc00640 1 Processor -- start 2024-04-26T23:52:15.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.923+0000 7f1b7cc00640 1 -- start start 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b780d3d60 0x7f1b780ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b78162aa0 0x7f1b7814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780d30f0 con 0x7f1b780b64c0 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78162e80 con 0x7f1b780d3d60 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.924+0000 7f1b7cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780d5b90 con 0x7f1b78162aa0 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37062/0 (socket says 172.21.15.46:37062) 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 -- 172.21.15.46:0/1945705447 learned_addr learned my addr 172.21.15.46:0/1945705447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75400640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b780d3d60 0x7f1b780ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 -- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b78162aa0 msgr2=0x7f1b7814ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b78162aa0 0x7f1b7814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 -- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b780d3d60 msgr2=0x7f1b780ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b780d3d60 0x7f1b780ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.925+0000 7f1b75e00640 1 -- 172.21.15.46:0/1945705447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780bab40 con 0x7f1b780b64c0 2024-04-26T23:52:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b75e00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800ca70 tx=0x7f1b6800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b6e000640 1 -- 172.21.15.46:0/1945705447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68017070 con 0x7f1b780b64c0 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b6e000640 1 -- 172.21.15.46:0/1945705447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6800abd0 con 0x7f1b780b64c0 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b6e000640 1 -- 172.21.15.46:0/1945705447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68012400 con 0x7f1b780b64c0 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b780bae20 con 0x7f1b780b64c0 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.926+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b780c9ed0 con 0x7f1b780b64c0 2024-04-26T23:52:15.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.928+0000 7f1b6e000640 1 -- 172.21.15.46:0/1945705447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b6801a070 con 0x7f1b780b64c0 2024-04-26T23:52:15.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.929+0000 7f1b6e000640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 0x7f1b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.929+0000 7f1b6e000640 1 -- 172.21.15.46:0/1945705447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f1b6808ba60 con 0x7f1b780b64c0 2024-04-26T23:52:15.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.930+0000 7f1b7f9c0740 1 Processor -- start 2024-04-26T23:52:15.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.930+0000 7f1b7f9c0740 1 -- start start 2024-04-26T23:52:15.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003816920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a003834330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0036fc030 con 0x55a0037ec220 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a0037177f0 con 0x55a003833de0 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0036d27e0 con 0x55a003816540 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b76800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b75400640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 0x7f1b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.931+0000 7f1b76800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37076/0 (socket says 172.21.15.46:37076) 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 -- 172.21.15.46:0/3425922586 learned_addr learned my addr 172.21.15.46:0/3425922586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b75400640 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a003834330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 -- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 msgr2=0x55a003816920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003816920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 -- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 msgr2=0x55a003834330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a003834330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 -- 172.21.15.46:0/3425922586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0036de050 con 0x55a0037ec220 2024-04-26T23:52:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b75400640 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a003834330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.932+0000 7f1b76800640 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f1b700098c0 tx=0x7f1b7002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bbcc7515eaab78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.933+0000 7f1b66600640 1 -- 172.21.15.46:0/3425922586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70037040 con 0x55a0037ec220 2024-04-26T23:52:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.933+0000 7f1b66600640 1 -- 172.21.15.46:0/3425922586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70042070 con 0x55a0037ec220 2024-04-26T23:52:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.933+0000 7f1b66600640 1 -- 172.21.15.46:0/3425922586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70032940 con 0x55a0037ec220 2024-04-26T23:52:15.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.933+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3425922586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 msgr2=0x55a0038324a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.933+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f1b700098c0 tx=0x7f1b7002c070 comp rx=0 tx=0).stop 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b75400640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 0x7f1b5406edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55a0037e9830 tx=0x7f1b60006f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3425922586 shutdown_connections 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0037ec220 0x55a0038324a0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a003834330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3425922586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003816920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3425922586 >> 172.21.15.46:0/3425922586 conn(0x55a0037e8b50 msgr2=0x55a003831350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3425922586 shutdown_connections 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.934+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3425922586 wait complete. 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.935+0000 7f1b7f9c0740 1 -- start start 2024-04-26T23:52:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.935+0000 7f1b7f9c0740 1 Processor -- start 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a0038023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a003754200 con 0x55a003833de0 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a00374eee0 con 0x55a003832a00 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a00375be70 con 0x55a003816540 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b7f9c0740 10 client.0 _ll_register_callbacks cb 0x55a003745270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37088/0 (socket says 172.21.15.46:37088) 2024-04-26T23:52:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 -- 172.21.15.46:0/4028978893 learned_addr learned my addr 172.21.15.46:0/4028978893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 -- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 msgr2=0x55a0038324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b76800640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a0038023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.936+0000 7f1b75400640 1 -- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 msgr2=0x55a0038023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b75400640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a0038023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b75400640 1 -- 172.21.15.46:0/4028978893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00352b7d0 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b75400640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f1b600136b0 tx=0x7f1b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0d02636ffc5a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001e5c0 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f1b6001e760 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b70042070 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.937+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f1b6002c050 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 10 client.10008 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55a0034613f0 con 0x55a003833de0 2024-04-26T23:52:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70042070 con 0x55a003833de0 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f1b6002f020 con 0x55a003833de0 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 10 client.10008 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 10 client.10008 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 20 client.10008 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a003802c10 0x55a0038053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 4 client.10008 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a00375d7e0 con 0x55a003802c10 2024-04-26T23:52:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.938+0000 7f1b75e00640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a003802c10 0x55a0038053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.939+0000 7f1b75e00640 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a003802c10 0x55a0038053c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c248969d5212fa server_cookie=b05b4e31b36ab0e5 in_seq=0 out_seq=0 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.939+0000 7f1b5aa00640 10 client.10008 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b5aa00640 1 -- 172.21.15.46:0/4028978893 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a00375d7e0 con 0x55a003802c10 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b5aa00640 10 client.10008 handle_command_reply: tid=0 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 client.10008 shutdown 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 msgr2=0x55a003844ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f1b600136b0 tx=0x7f1b600046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 shutdown_connections 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a003802c10 0x55a0038053c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a0038023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003844ae0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/4028978893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b5aa00640 0 client.10008 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 >> 172.21.15.46:0/4028978893 conn(0x55a0037ec220 msgr2=0x55a003835160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 shutdown_connections 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/4028978893 wait complete. 2024-04-26T23:52:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.940+0000 7f1b7f9c0740 20 client.10008 trim_cache size 0 max 16384 2024-04-26T23:52:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 Processor -- start 2024-04-26T23:52:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 -- start start 2024-04-26T23:52:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003802ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a0037e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78201e80 con 0x55a003832a00 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780ca8c0 con 0x55a003833de0 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.960+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b781f98a0 con 0x55a003816540 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37092/0 (socket says 172.21.15.46:37092) 2024-04-26T23:52:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 -- 172.21.15.46:0/1222501671 learned_addr learned my addr 172.21.15.46:0/1222501671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 -- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 msgr2=0x55a003802ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003802ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 -- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 msgr2=0x55a0037e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b76800640 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a0037e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a0037e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.961+0000 7f1b75400640 1 -- 172.21.15.46:0/1222501671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0034210f0 con 0x55a003832a00 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b75400640 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000ffe0 tx=0x7f1b6001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474467dc9aed2af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b65c00640 1 -- 172.21.15.46:0/1222501671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001ee60 con 0x55a003832a00 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b65c00640 1 -- 172.21.15.46:0/1222501671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60024070 con 0x55a003832a00 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b65c00640 1 -- 172.21.15.46:0/1222501671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001e5c0 con 0x55a003832a00 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/1222501671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 msgr2=0x55a003803530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.962+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000ffe0 tx=0x7f1b6001e000 comp rx=0 tx=0).stop 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/1222501671 shutdown_connections 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003833de0 0x55a0037e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003832a00 0x55a003803530 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/1222501671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a003802ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/1222501671 >> 172.21.15.46:0/1222501671 conn(0x55a0037ec220 msgr2=0x55a0038333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/1222501671 shutdown_connections 2024-04-26T23:52:15.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/1222501671 wait complete. 2024-04-26T23:52:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.963+0000 7f1b7f9c0740 1 -- start start 2024-04-26T23:52:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.964+0000 7f1b7f9c0740 1 Processor -- start 2024-04-26T23:52:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a003826ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a00375cfd0 con 0x55a003833de0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780cecd0 con 0x55a003832a00 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0038444e0 con 0x55a003816540 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b7f9c0740 10 client.0 _ll_register_callbacks cb 0x7f1b6000fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37100/0 (socket says 172.21.15.46:37100) 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 -- 172.21.15.46:0/3100683536 learned_addr learned my addr 172.21.15.46:0/3100683536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b75400640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a003826ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 -- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 msgr2=0x55a0038aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b75e00640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 -- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 msgr2=0x55a003826ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a003826ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.965+0000 7f1b76800640 1 -- 172.21.15.46:0/3100683536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a003717a30 con 0x55a003833de0 2024-04-26T23:52:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b76800640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097a0 tx=0x7f1b70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5923305ac6ac42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7004b070 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f1b70003a20 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b60024070 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f1b70037040 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b7f9c0740 10 client.10014 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.966+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55a0038abbc0 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60024070 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7f1b7003a030 con 0x55a003833de0 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 10 client.10014 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 10 client.10014 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:15.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 20 client.10014 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0037e6920 0x55a0038ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 4 client.10014 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55a00382cea0 con 0x55a0037e6920 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.967+0000 7f1b75e00640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0037e6920 0x55a0038ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b75e00640 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0037e6920 0x55a0038ae4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13104e26dab4e17b server_cookie=137148b196f22d67 in_seq=0 out_seq=0 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b5aa00640 10 client.10014 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b5aa00640 1 -- 172.21.15.46:0/3100683536 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a00382cea0 con 0x55a0037e6920 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b5aa00640 10 client.10014 handle_command_reply: tid=0 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 client.10014 shutdown 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 msgr2=0x55a003827220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097a0 tx=0x7f1b70004730 comp rx=0 tx=0).stop 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 shutdown_connections 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0037e6920 0x55a0038ae4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a003833de0 0x55a003827220 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a003832a00 0x55a003826ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 --2- 172.21.15.46:0/3100683536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a003816540 0x55a0038aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b5aa00640 0 client.10014 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 >> 172.21.15.46:0/3100683536 conn(0x55a0037ec220 msgr2=0x55a003844660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 shutdown_connections 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.968+0000 7f1b7f9c0740 1 -- 172.21.15.46:0/3100683536 wait complete. 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7f9c0740 20 client.10014 trim_cache size 0 max 16384 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:52:15.974 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 msgr2=0x7f1b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 0x7f1b5406edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55a0037e9830 tx=0x7f1b60006f00 comp rx=0 tx=0).stop 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 msgr2=0x7f1b780b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800ca70 tx=0x7f1b6800cf30 comp rx=0 tx=0).stop 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 shutdown_connections 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b5406c8f0 0x7f1b5406edb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b78162aa0 0x7f1b7814ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b780d3d60 0x7f1b780ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/1945705447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b780b64c0 0x7f1b780b7f40 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 >> 172.21.15.46:0/1945705447 conn(0x7f1b780d5070 msgr2=0x7f1b78160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.969+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 shutdown_connections 2024-04-26T23:52:15.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:15.970+0000 7f1b7cc00640 1 -- 172.21.15.46:0/1945705447 wait complete. 2024-04-26T23:52:15.990 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T23:52:15.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.10:0 scrub status 2024-04-26T23:52:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.180+0000 7fa963200640 1 Processor -- start 2024-04-26T23:52:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.180+0000 7fa963200640 1 -- start start 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa963200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa964163bb0 0x7fa964164020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa963200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96406dee0 con 0x7fa9640d0a50 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa963200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96406e1c0 con 0x7fa9640b39d0 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa963200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa96406eca0 con 0x7fa964163bb0 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa961e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9640d0a50 0x7fa9640d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54630/0 (socket says 172.21.15.46:54630) 2024-04-26T23:52:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa961e00640 1 -- 172.21.15.46:0/2460971027 learned_addr learned my addr 172.21.15.46:0/2460971027 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917138355 0 0) 0x7fa96406dee0 con 0x7fa9640d0a50 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa948003520 con 0x7fa9640d0a50 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.182+0000 7fa961400640 1 --2- 172.21.15.46:0/2460971027 >> v2:172.21.15.142:3300/0 conn(0x7fa964163bb0 0x7fa964164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 865525843 0 0) 0x7fa948003520 con 0x7fa9640d0a50 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 >> v2:172.21.15.142:3300/0 conn(0x7fa964163bb0 msgr2=0x7fa964164020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 --2- 172.21.15.46:0/2460971027 >> v2:172.21.15.142:3300/0 conn(0x7fa964163bb0 0x7fa964164020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 >> v1:172.21.15.142:6789/0 conn(0x7fa9640b39d0 legacy=0x7fa9640b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96406e4a0 con 0x7fa9640d0a50 2024-04-26T23:52:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa954001670 con 0x7fa9640d0a50 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 >> v1:172.21.15.46:6789/0 conn(0x7fa9640d0a50 legacy=0x7fa9640d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa948003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa948004800 0x7fa948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480052e0 0x7fa94800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa96406eca0 con 0x7fa9480039b0 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa94800c1b0 con 0x7fa9480052e0 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa94800c3b0 con 0x7fa948004800 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.183+0000 7fa961e00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa948003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa961e00640 1 -- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa948004800 msgr2=0x7fa948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa961e00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa948004800 0x7fa948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa961e00640 1 -- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480052e0 msgr2=0x7fa94800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa961e00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480052e0 0x7fa94800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa961e00640 1 -- 172.21.15.46:0/2460971027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa948003520 con 0x7fa9480039b0 2024-04-26T23:52:16.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.184+0000 7fa962800640 1 -- 172.21.15.46:0/2460971027 reap_dead start 2024-04-26T23:52:16.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.185+0000 7fa961e00640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa948003dd0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c210 tx=0x7fa948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97ff38cf8a048d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.185+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954007cf0 con 0x7fa9480039b0 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.185+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa954005af0 con 0x7fa9480039b0 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.185+0000 7fa960a00640 1 -- 172.21.15.46:0/2460971027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954007070 con 0x7fa9480039b0 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 -- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 msgr2=0x7fa948003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa948003dd0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c210 tx=0x7fa948005830 comp rx=0 tx=0).stop 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 -- 172.21.15.46:0/2460971027 shutdown_connections 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 --2- 172.21.15.46:0/2460971027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa948003dd0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 -- 172.21.15.46:0/2460971027 >> 172.21.15.46:0/2460971027 conn(0x7fa96406c350 msgr2=0x7fa96406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 -- 172.21.15.46:0/2460971027 shutdown_connections 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.186+0000 7fa963200640 1 -- 172.21.15.46:0/2460971027 wait complete. 2024-04-26T23:52:16.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.188+0000 7fa963200640 1 Processor -- start 2024-04-26T23:52:16.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.188+0000 7fa963200640 1 -- start start 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa964155020 0x7fa9641554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 0x7fa9640b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640d0660 con 0x7fa9480039b0 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa964164920 con 0x7fa964155020 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa963200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9640d07e0 con 0x7fa9640b2ad0 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37124/0 (socket says 172.21.15.46:37124) 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 -- 172.21.15.46:0/933934161 learned_addr learned my addr 172.21.15.46:0/933934161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa962800640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 0x7fa9640b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 -- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 msgr2=0x7fa9640b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961400640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa964155020 0x7fa9641554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 0x7fa9640b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 -- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa964155020 msgr2=0x7fa9641554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa964155020 0x7fa9641554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa961e00640 1 -- 172.21.15.46:0/933934161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa964144260 con 0x7fa9480039b0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.189+0000 7fa962800640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 0x7fa9640b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa961e00640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c210 tx=0x7fa96406e390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa95a000640 1 -- 172.21.15.46:0/933934161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954041020 con 0x7fa9480039b0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa95a000640 1 -- 172.21.15.46:0/933934161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa954038040 con 0x7fa9480039b0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa95a000640 1 -- 172.21.15.46:0/933934161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9540072b0 con 0x7fa9480039b0 2024-04-26T23:52:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa964144540 con 0x7fa9480039b0 2024-04-26T23:52:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.190+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa964144b00 con 0x7fa9480039b0 2024-04-26T23:52:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.192+0000 7fa95a000640 1 -- 172.21.15.46:0/933934161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa954044020 con 0x7fa9480039b0 2024-04-26T23:52:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.193+0000 7fa95a000640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 0x7fa94406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.194+0000 7fa95a000640 1 -- 172.21.15.46:0/933934161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa9540abbb0 con 0x7fa9480039b0 2024-04-26T23:52:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.194+0000 7fa961400640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 0x7fa94406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.194+0000 7fa96bfad740 1 Processor -- start 2024-04-26T23:52:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.195+0000 7fa96bfad740 1 -- start start 2024-04-26T23:52:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2077af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa961400640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 0x7fa94406ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa950006fd0 tx=0x7fa950008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2095500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1f5d030 con 0x55a0c204d3f0 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1f787f0 con 0x55a0c2077710 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0c1f337e0 con 0x55a0c2094fb0 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37134/0 (socket says 172.21.15.46:37134) 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 -- 172.21.15.46:0/372912102 learned_addr learned my addr 172.21.15.46:0/372912102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa961400640 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2095500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa961e00640 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 -- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 msgr2=0x55a0c2095500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2095500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 -- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 msgr2=0x55a0c2077af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2077af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.196+0000 7fa962800640 1 -- 172.21.15.46:0/372912102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c1f3f050 con 0x55a0c204d3f0 2024-04-26T23:52:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.197+0000 7fa962800640 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fa964070180 tx=0x7fa95c0020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72796b273925448 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.197+0000 7fa94ea00640 1 -- 172.21.15.46:0/372912102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c018070 con 0x55a0c204d3f0 2024-04-26T23:52:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.197+0000 7fa94ea00640 1 -- 172.21.15.46:0/372912102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c00d7c0 con 0x55a0c204d3f0 2024-04-26T23:52:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.197+0000 7fa94ea00640 1 -- 172.21.15.46:0/372912102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00db70 con 0x55a0c204d3f0 2024-04-26T23:52:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 -- 172.21.15.46:0/372912102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 msgr2=0x55a0c2093670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fa964070180 tx=0x7fa95c0020a0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 -- 172.21.15.46:0/372912102 shutdown_connections 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c204d3f0 0x55a0c2093670 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2095500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 --2- 172.21.15.46:0/372912102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 -- 172.21.15.46:0/372912102 >> 172.21.15.46:0/372912102 conn(0x55a0c2049d20 msgr2=0x55a0c2092520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 -- 172.21.15.46:0/372912102 shutdown_connections 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.198+0000 7fa96bfad740 1 -- 172.21.15.46:0/372912102 wait complete. 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.199+0000 7fa96bfad740 1 -- start start 2024-04-26T23:52:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.199+0000 7fa96bfad740 1 Processor -- start 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c20727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2095fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1fb5200 con 0x55a0c2094fb0 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1fafee0 con 0x55a0c2077710 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0c1fbce70 con 0x55a0c2071d90 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa96bfad740 10 client.0 _ll_register_callbacks cb 0x55a0c1fa6270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa961400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa961400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37144/0 (socket says 172.21.15.46:37144) 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.200+0000 7fa961400640 1 -- 172.21.15.46:0/1011306852 learned_addr learned my addr 172.21.15.46:0/1011306852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961e00640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c20727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa962800640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2095fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 -- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 msgr2=0x55a0c2095fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2095fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 -- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 msgr2=0x55a0c20727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c20727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 -- 172.21.15.46:0/1011306852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c1d8c7d0 con 0x55a0c2094fb0 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961e00640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c20727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa961400640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa950015bd0 tx=0x7fa950004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50534c91d8777af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa943400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000f040 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa95001e040 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.201+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95c00d7c0 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fa95002d050 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 10 client.10026 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55a0c1cc23f0 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c00d7c0 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7fa950030020 con 0x55a0c2094fb0 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 10 client.10026 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 10 client.10026 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 20 client.10026 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c20783d0 0x55a0c2078860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 4 client.10026 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.202+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c1fbe7e0 con 0x55a0c20783d0 2024-04-26T23:52:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.203+0000 7fa961e00640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c20783d0 0x55a0c2078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.203+0000 7fa961e00640 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c20783d0 0x55a0c2078860 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e2d4457a3146be server_cookie=9dcd13998e51e5ed in_seq=0 out_seq=0 2024-04-26T23:52:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.203+0000 7fa943400640 10 client.10026 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa943400640 1 -- 172.21.15.46:0/1011306852 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0c1fbe7e0 con 0x55a0c20783d0 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa943400640 10 client.10026 handle_command_reply: tid=0 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 client.10026 shutdown 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 msgr2=0x55a0c2095aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa950015bd0 tx=0x7fa950004730 comp rx=0 tx=0).stop 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 shutdown_connections 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c20783d0 0x55a0c2078860 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2095fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2095aa0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa943400640 0 client.10026 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 --2- 172.21.15.46:0/1011306852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c20727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 >> 172.21.15.46:0/1011306852 conn(0x55a0c204d3f0 msgr2=0x55a0c2046960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 shutdown_connections 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.204+0000 7fa96bfad740 1 -- 172.21.15.46:0/1011306852 wait complete. 2024-04-26T23:52:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.205+0000 7fa96bfad740 20 client.10026 trim_cache size 0 max 16384 2024-04-26T23:52:16.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.224+0000 7fa96bfad740 1 Processor -- start 2024-04-26T23:52:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.224+0000 7fa96bfad740 1 -- start start 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2071d90 0x55a0c2091760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa964206980 con 0x55a0c2077710 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa964205960 con 0x55a0c2071d90 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa964204c70 con 0x55a0c2094fb0 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37152/0 (socket says 172.21.15.46:37152) 2024-04-26T23:52:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 -- 172.21.15.46:0/2121497947 learned_addr learned my addr 172.21.15.46:0/2121497947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 -- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 msgr2=0x55a0c2076cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961e00640 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2071d90 0x55a0c2091760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 -- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2071d90 msgr2=0x55a0c2091760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.225+0000 7fa961400640 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2071d90 0x55a0c2091760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.226+0000 7fa961400640 1 -- 172.21.15.46:0/2121497947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c1c820f0 con 0x55a0c2077710 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.226+0000 7fa961400640 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fa950015910 tx=0x7fa95001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b005d7bddf1c6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.226+0000 7fa94e000640 1 -- 172.21.15.46:0/2121497947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950036070 con 0x55a0c2077710 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.226+0000 7fa94e000640 1 -- 172.21.15.46:0/2121497947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9500137c0 con 0x55a0c2077710 2024-04-26T23:52:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.226+0000 7fa94e000640 1 -- 172.21.15.46:0/2121497947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950009ef0 con 0x55a0c2077710 2024-04-26T23:52:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 -- 172.21.15.46:0/2121497947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 msgr2=0x55a0c2076770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fa950015910 tx=0x7fa95001f000 comp rx=0 tx=0).stop 2024-04-26T23:52:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 -- 172.21.15.46:0/2121497947 shutdown_connections 2024-04-26T23:52:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2121497947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2071d90 0x55a0c2091760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 -- 172.21.15.46:0/2121497947 >> 172.21.15.46:0/2121497947 conn(0x55a0c204d3f0 msgr2=0x55a0c208b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 -- 172.21.15.46:0/2121497947 shutdown_connections 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.227+0000 7fa96bfad740 1 -- 172.21.15.46:0/2121497947 wait complete. 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.228+0000 7fa96bfad740 1 -- start start 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.228+0000 7fa96bfad740 1 Processor -- start 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2102740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1fbdfd0 con 0x55a0c2094fb0 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa964205290 con 0x55a0c2077710 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a0c2110170 con 0x55a0c2071d90 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa96bfad740 10 client.0 _ll_register_callbacks cb 0x7fa950009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa962800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa962800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37166/0 (socket says 172.21.15.46:37166) 2024-04-26T23:52:16.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa962800640 1 -- 172.21.15.46:0/2626336654 learned_addr learned my addr 172.21.15.46:0/2626336654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa961400640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.229+0000 7fa961e00640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 -- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 msgr2=0x55a0c2102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2102740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 -- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 msgr2=0x55a0c2076770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 -- 172.21.15.46:0/2626336654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c1f78a30 con 0x55a0c2094fb0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa962800640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009010 tx=0x7fa95c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af30ec34ae8961 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa943400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c022070 con 0x55a0c2094fb0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa95c0072d0 con 0x55a0c2094fb0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9500137c0 con 0x55a0c2094fb0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.230+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fa95c02c050 con 0x55a0c2094fb0 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 10 client.10032 fetch_fsmap learned FSMap version 118 2024-04-26T23:52:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55a0c204e7c0 con 0x55a0c2094fb0 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9500137c0 con 0x55a0c2094fb0 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 118) ==== 3494+0+0 (secure 0 0 0) 0x7fa95c02f020 con 0x55a0c2094fb0 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 10 client.10032 fetch_fsmap finished waiting for FSMap version 118 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 10 client.10032 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.c 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 20 client.10032 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c207b160 0x55a0c210f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 4 client.10032 mds_command: new command op to 9663 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 --> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0c1fb8650 con 0x55a0c207b160 2024-04-26T23:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.231+0000 7fa961e00640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c207b160 0x55a0c210f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa961e00640 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c207b160 0x55a0c210f350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d9d64bd7a3b97 server_cookie=58d6317ec93fad6f in_seq=0 out_seq=0 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa943400640 10 client.10032 ms_handle_connect on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa943400640 1 -- 172.21.15.46:0/2626336654 <== mds.0 v2:172.21.15.46:6833/3522383810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0c1fb8650 con 0x55a0c207b160 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa943400640 10 client.10032 handle_command_reply: tid=0 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 client.10032 shutdown 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 msgr2=0x55a0c2076cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009010 tx=0x7fa95c0014a0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 shutdown_connections 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:6833/3522383810,v1:172.21.15.46:6835/3522383810] conn(0x55a0c207b160 0x55a0c210f350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0c2094fb0 0x55a0c2076cb0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a0c2077710 0x55a0c2076770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 --2- 172.21.15.46:0/2626336654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a0c2071d90 0x55a0c2102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa943400640 0 client.10032 ms_handle_reset on v2:172.21.15.46:6833/3522383810 2024-04-26T23:52:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.232+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 >> 172.21.15.46:0/2626336654 conn(0x55a0c204d3f0 msgr2=0x55a0c2043080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 shutdown_connections 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa96bfad740 1 -- 172.21.15.46:0/2626336654 wait complete. 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa96bfad740 20 client.10032 trim_cache size 0 max 16384 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:52:16.240 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 msgr2=0x7fa94406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 0x7fa94406ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa950006fd0 tx=0x7fa950008040 comp rx=0 tx=0).stop 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 msgr2=0x7fa964142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.233+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c210 tx=0x7fa96406e390 comp rx=0 tx=0).stop 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 shutdown_connections 2024-04-26T23:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa94406c920 0x7fa94406ede0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9640b2ad0 0x7fa9640b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa964155020 0x7fa9641554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 --2- 172.21.15.46:0/933934161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9480039b0 0x7fa964142d90 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 >> 172.21.15.46:0/933934161 conn(0x7fa96406c350 msgr2=0x7fa964158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 shutdown_connections 2024-04-26T23:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.234+0000 7fa963200640 1 -- 172.21.15.46:0/933934161 wait complete. 2024-04-26T23:52:16.253 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:52:16.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.445+0000 7fa044a00640 1 Processor -- start 2024-04-26T23:52:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.446+0000 7fa044a00640 1 -- start start 2024-04-26T23:52:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.447+0000 7fa044a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa040160870 0x7fa0400d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.447+0000 7fa044a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0400cebc0 con 0x7fa0400d08f0 2024-04-26T23:52:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.447+0000 7fa044a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0400ce320 con 0x7fa040160490 2024-04-26T23:52:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.447+0000 7fa044a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa0400ce600 con 0x7fa040160870 2024-04-26T23:52:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa040160870 0x7fa0400d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03f200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa0400d08f0 0x7fa0400b6490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54644/0 (socket says 172.21.15.46:54644) 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03f200640 1 -- 172.21.15.46:0/1009929930 learned_addr learned my addr 172.21.15.46:0/1009929930 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03de00640 1 -- 172.21.15.46:0/1009929930 >> v1:172.21.15.142:6789/0 conn(0x7fa040160490 legacy=0x7fa0400d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03de00640 1 -- 172.21.15.46:0/1009929930 >> v1:172.21.15.46:6789/0 conn(0x7fa0400d08f0 legacy=0x7fa0400b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03de00640 1 -- 172.21.15.46:0/1009929930 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400ce8e0 con 0x7fa040160870 2024-04-26T23:52:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.448+0000 7fa03d400640 1 -- 172.21.15.46:0/1009929930 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999836658 0 0) 0x7fa0400cebc0 con 0x7fa0400d08f0 2024-04-26T23:52:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.449+0000 7fa03de00640 1 --2- 172.21.15.46:0/1009929930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0400d14b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354a36de61bf10f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.450+0000 7fa03d400640 1 -- 172.21.15.46:0/1009929930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028007740 con 0x7fa040160870 2024-04-26T23:52:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.450+0000 7fa03d400640 1 -- 172.21.15.46:0/1009929930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0280078e0 con 0x7fa040160870 2024-04-26T23:52:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.450+0000 7fa03d400640 1 -- 172.21.15.46:0/1009929930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028007cb0 con 0x7fa040160870 2024-04-26T23:52:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.450+0000 7fa044a00640 1 -- 172.21.15.46:0/1009929930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 msgr2=0x7fa0400d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.450+0000 7fa044a00640 1 --2- 172.21.15.46:0/1009929930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0400d14b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.451+0000 7fa044a00640 1 -- 172.21.15.46:0/1009929930 shutdown_connections 2024-04-26T23:52:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.451+0000 7fa044a00640 1 --2- 172.21.15.46:0/1009929930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0400d14b0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.451+0000 7fa044a00640 1 -- 172.21.15.46:0/1009929930 >> 172.21.15.46:0/1009929930 conn(0x7fa0400b48f0 msgr2=0x7fa0400b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.451+0000 7fa044a00640 1 -- 172.21.15.46:0/1009929930 shutdown_connections 2024-04-26T23:52:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.451+0000 7fa044a00640 1 -- 172.21.15.46:0/1009929930 wait complete. 2024-04-26T23:52:16.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.452+0000 7fa044a00640 1 Processor -- start 2024-04-26T23:52:16.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.453+0000 7fa044a00640 1 -- start start 2024-04-26T23:52:16.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 0x7fa04015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040160490 0x7fa0400c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400d19f0 con 0x7fa040160870 2024-04-26T23:52:16.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400b5f70 con 0x7fa040160490 2024-04-26T23:52:16.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa044a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa0400d1b70 con 0x7fa0400d08f0 2024-04-26T23:52:16.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa03f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa03f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37176/0 (socket says 172.21.15.46:37176) 2024-04-26T23:52:16.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa03f200640 1 -- 172.21.15.46:0/3427392788 learned_addr learned my addr 172.21.15.46:0/3427392788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa03e800640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 0x7fa04015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.454+0000 7fa03de00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040160490 0x7fa0400c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 -- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 msgr2=0x7fa04015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 0x7fa04015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 -- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040160490 msgr2=0x7fa0400c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040160490 0x7fa0400c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 -- 172.21.15.46:0/3427392788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0401f8ba0 con 0x7fa040160870 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03e800640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 0x7fa04015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa03f200640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c3c0 tx=0x7fa03800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0380074e0 con 0x7fa040160870 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0380047e0 con 0x7fa040160870 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038004c50 con 0x7fa040160870 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0401f8e80 con 0x7fa040160870 2024-04-26T23:52:16.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.455+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0400c8bf0 con 0x7fa040160870 2024-04-26T23:52:16.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.457+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0400c8740 con 0x7fa040160870 2024-04-26T23:52:16.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.458+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa038011070 con 0x7fa040160870 2024-04-26T23:52:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.458+0000 7fa036a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 0x7fa01c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.459+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa03808b820 con 0x7fa040160870 2024-04-26T23:52:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.459+0000 7fa03e800640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 0x7fa01c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.460+0000 7fa03e800640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 0x7fa01c06ee00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa04006f640 tx=0x7fa030009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.463+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa038059b00 con 0x7fa040160870 2024-04-26T23:52:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.706+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa040069ac0 con 0x7fa040160870 2024-04-26T23:52:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.707+0000 7fa036a00640 1 -- 172.21.15.46:0/3427392788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7fa038059400 con 0x7fa040160870 2024-04-26T23:52:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:16.710 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 msgr2=0x7fa01c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 0x7fa01c06ee00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa04006f640 tx=0x7fa030009040 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 msgr2=0x7fa0401f80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c3c0 tx=0x7fa03800c880 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 shutdown_connections 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa01c06c940 0x7fa01c06ee00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa040160870 0x7fa0401f80e0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040160490 0x7fa0400c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 --2- 172.21.15.46:0/3427392788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0400d08f0 0x7fa04015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 >> 172.21.15.46:0/3427392788 conn(0x7fa0400b48f0 msgr2=0x7fa0400b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 shutdown_connections 2024-04-26T23:52:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.710+0000 7fa044a00640 1 -- 172.21.15.46:0/3427392788 wait complete. 2024-04-26T23:52:16.732 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:16.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:16.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.932+0000 7fab01600640 1 Processor -- start 2024-04-26T23:52:16.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.932+0000 7fab01600640 1 -- start start 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7fab01600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faafc06e510 0x7faafc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7fab01600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0732c0 0x7faafc073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7fab01600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faafc06ee30 con 0x7faafc160bd0 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7fab01600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faafc161940 con 0x7faafc06e510 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7fab01600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faafc071f80 con 0x7faafc0732c0 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7faafa000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faafc160bd0 0x7faafc161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54652/0 (socket says 172.21.15.46:54652) 2024-04-26T23:52:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.934+0000 7faafa000640 1 -- 172.21.15.46:0/416162232 learned_addr learned my addr 172.21.15.46:0/416162232 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faafaa00640 1 --2- 172.21.15.46:0/416162232 >> v2:172.21.15.142:3300/0 conn(0x7faafc06e510 0x7faafc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faafb400640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0732c0 0x7faafc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034104773 0 0) 0x7faafc06ee30 con 0x7faafc160bd0 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faae0003520 con 0x7faafc160bd0 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3949840161 0 0) 0x7faae0003520 con 0x7faafc160bd0 2024-04-26T23:52:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0732c0 msgr2=0x7faafc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0732c0 0x7faafc073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 >> v2:172.21.15.142:3300/0 conn(0x7faafc06e510 msgr2=0x7faafc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 --2- 172.21.15.46:0/416162232 >> v2:172.21.15.142:3300/0 conn(0x7faafc06e510 0x7faafc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc072260 con 0x7faafc160bd0 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7faae8001670 con 0x7faafc160bd0 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 >> v1:172.21.15.46:6789/0 conn(0x7faafc160bd0 legacy=0x7faafc161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faae0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0004800 0x7faae0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faae00051c0 0x7faae000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faafc161940 con 0x7faae00039b0 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faafc071f80 con 0x7faae0004800 2024-04-26T23:52:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.935+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faae000bf00 con 0x7faae00051c0 2024-04-26T23:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.936+0000 7faafaa00640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faae0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.936+0000 7faafa000640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0004800 0x7faae0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.936+0000 7faafb400640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faae00051c0 0x7faae000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.936+0000 7faafaa00640 1 -- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faae00051c0 msgr2=0x7faae000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.936+0000 7faafaa00640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faae00051c0 0x7faae000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.937+0000 7faafaa00640 1 -- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0004800 msgr2=0x7faae0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.937+0000 7faafaa00640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0004800 0x7faae0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.937+0000 7faafaa00640 1 -- 172.21.15.46:0/416162232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae0003520 con 0x7faae00039b0 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.937+0000 7faafa000640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0004800 0x7faae0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.937+0000 7faafb400640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faae00051c0 0x7faae000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.938+0000 7faafaa00640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faae0003dd0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e170 tx=0x7faae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d0664c6097bf0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec034020 con 0x7faae00039b0 2024-04-26T23:52:16.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec030430 con 0x7faae00039b0 2024-04-26T23:52:16.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7faaf9600640 1 -- 172.21.15.46:0/416162232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec009c10 con 0x7faae00039b0 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 -- 172.21.15.46:0/416162232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 msgr2=0x7faae0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 --2- 172.21.15.46:0/416162232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faae0003dd0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e170 tx=0x7faae0005830 comp rx=0 tx=0).stop 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7faafb400640 1 -- 172.21.15.46:0/416162232 reap_dead start 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 -- 172.21.15.46:0/416162232 shutdown_connections 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 -- 172.21.15.46:0/416162232 >> 172.21.15.46:0/416162232 conn(0x7faafc06c530 msgr2=0x7faafc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 -- 172.21.15.46:0/416162232 shutdown_connections 2024-04-26T23:52:16.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.939+0000 7fab01600640 1 -- 172.21.15.46:0/416162232 wait complete. 2024-04-26T23:52:16.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.941+0000 7fab01600640 1 Processor -- start 2024-04-26T23:52:16.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.941+0000 7fab01600640 1 -- start start 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 0x7faafc142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc13a0d0 0x7faafc13a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faafc072d40 con 0x7faae00039b0 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faafc072820 con 0x7faafc13a0d0 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.942+0000 7fab01600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faafc072ec0 con 0x7faafc139550 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37186/0 (socket says 172.21.15.46:37186) 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 -- 172.21.15.46:0/3998898528 learned_addr learned my addr 172.21.15.46:0/3998898528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafa000640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 0x7faafc142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 -- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 msgr2=0x7faafc142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafb400640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc13a0d0 0x7faafc13a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 0x7faafc142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 -- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc13a0d0 msgr2=0x7faafc13a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc13a0d0 0x7faafc13a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafaa00640 1 -- 172.21.15.46:0/3998898528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc1446f0 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.943+0000 7faafa000640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 0x7faafc142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7faafaa00640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e170 tx=0x7faaec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec034070 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec003b90 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004020 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc143650 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.944+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc143c10 con 0x7faae00039b0 2024-04-26T23:52:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.945+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faafc069530 con 0x7faae00039b0 2024-04-26T23:52:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.946+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaec037020 con 0x7faae00039b0 2024-04-26T23:52:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.947+0000 7faaf2a00640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 0x7faadc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.947+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7faaec0ac640 con 0x7faae00039b0 2024-04-26T23:52:16.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.948+0000 7faafa000640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 0x7faadc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.949+0000 7faafa000640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 0x7faadc06ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7faae8001eb0 tx=0x7faae8000f50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:16.952+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faaec03f070 con 0x7faae00039b0 2024-04-26T23:52:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.242+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7faafc06fb00 con 0x7faae00039b0 2024-04-26T23:52:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.243+0000 7faaf2a00640 1 -- 172.21.15.46:0/3998898528 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1665 (secure 0 0 0) 0x7faaec07a8a0 con 0x7faae00039b0 2024-04-26T23:52:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:17.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":928,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077264401,"bytes_used":2606956544,"bytes_avail":770487156736,"bytes_total":773094113280,"write_bytes_sec":7507156,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-26T23:52:16.230959+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 msgr2=0x7faadc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 0x7faadc06ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7faae8001eb0 tx=0x7faae8000f50 comp rx=0 tx=0).stop 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 msgr2=0x7faafc15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e170 tx=0x7faaec009750 comp rx=0 tx=0).stop 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 shutdown_connections 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faadc06c920 0x7faadc06ede0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc13a0d0 0x7faafc13a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc139550 0x7faafc142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 --2- 172.21.15.46:0/3998898528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae00039b0 0x7faafc15c220 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 >> 172.21.15.46:0/3998898528 conn(0x7faafc06c530 msgr2=0x7faafc0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 shutdown_connections 2024-04-26T23:52:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:17.246+0000 7fab01600640 1 -- 172.21.15.46:0/3998898528 wait complete. 2024-04-26T23:52:17.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:52:18.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.488+0000 7fd7f5000640 1 Processor -- start 2024-04-26T23:52:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.488+0000 7fd7f5000640 1 -- start start 2024-04-26T23:52:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.489+0000 7fd7f5000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 0x7fd7f00d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.489+0000 7fd7f5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7f00b3fb0 0x7fd7f00d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.489+0000 7fd7f5000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f00d2a70 con 0x7fd7f00d56c0 2024-04-26T23:52:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.489+0000 7fd7f5000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f015f8e0 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.489+0000 7fd7f5000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f00d5b10 con 0x7fd7f015f500 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7f00b3fb0 0x7fd7f00d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7f00b3fb0 0x7fd7f00d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37188/0 (socket says 172.21.15.46:37188) 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 -- 172.21.15.46:0/1783547532 learned_addr learned my addr 172.21.15.46:0/1783547532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ede00640 1 --2- 172.21.15.46:0/1783547532 >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 0x7fd7f00d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 -- 172.21.15.46:0/1783547532 >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 msgr2=0x7fd7f00d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 --2- 172.21.15.46:0/1783547532 >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 0x7fd7f00d2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 -- 172.21.15.46:0/1783547532 >> v1:172.21.15.142:6789/0 conn(0x7fd7f00d56c0 legacy=0x7fd7f0162f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ed400640 1 -- 172.21.15.46:0/1783547532 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00d2f60 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.490+0000 7fd7ede00640 1 --2- 172.21.15.46:0/1783547532 >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 0x7fd7f00d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7ed400640 1 --2- 172.21.15.46:0/1783547532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f00b3fb0 0x7fd7f00d5180 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b7d40 tx=0x7fd7dc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb618ad7947b2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7eca00640 1 -- 172.21.15.46:0/1783547532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc037070 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7eca00640 1 -- 172.21.15.46:0/1783547532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7dc032440 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7eca00640 1 -- 172.21.15.46:0/1783547532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc032810 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7f5000640 1 -- 172.21.15.46:0/1783547532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f00b3fb0 msgr2=0x7fd7f00d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.492+0000 7fd7f5000640 1 --2- 172.21.15.46:0/1783547532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f00b3fb0 0x7fd7f00d5180 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b7d40 tx=0x7fd7dc02a250 comp rx=0 tx=0).stop 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 -- 172.21.15.46:0/1783547532 shutdown_connections 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 --2- 172.21.15.46:0/1783547532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f00b3fb0 0x7fd7f00d5180 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 --2- 172.21.15.46:0/1783547532 >> v2:172.21.15.142:3300/0 conn(0x7fd7f015f500 0x7fd7f00d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 -- 172.21.15.46:0/1783547532 >> 172.21.15.46:0/1783547532 conn(0x7fd7f0074400 msgr2=0x7fd7f009e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 -- 172.21.15.46:0/1783547532 shutdown_connections 2024-04-26T23:52:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.493+0000 7fd7f5000640 1 -- 172.21.15.46:0/1783547532 wait complete. 2024-04-26T23:52:18.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.494+0000 7fd7f5000640 1 Processor -- start 2024-04-26T23:52:18.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.495+0000 7fd7f5000640 1 -- start start 2024-04-26T23:52:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.495+0000 7fd7f5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7f00b3fb0 0x7fd7f00b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.495+0000 7fd7f5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f00d56c0 0x7fd7f00b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7f5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7f5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00d3520 con 0x7fd7f015f500 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7f5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00b5e80 con 0x7fd7f00d56c0 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7f5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f00d3800 con 0x7fd7f00b3fb0 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37192/0 (socket says 172.21.15.46:37192) 2024-04-26T23:52:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 -- 172.21.15.46:0/2744009681 learned_addr learned my addr 172.21.15.46:0/2744009681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ede00640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7f00b3fb0 0x7fd7f00b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ed400640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f00d56c0 0x7fd7f00b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 -- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7f00b3fb0 msgr2=0x7fd7f00b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7f00b3fb0 0x7fd7f00b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 -- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f00d56c0 msgr2=0x7fd7f00b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f00d56c0 0x7fd7f00b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.496+0000 7fd7ee800640 1 -- 172.21.15.46:0/2744009681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00b9cc0 con 0x7fd7f015f500 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7ee800640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c780 tx=0x7fd7e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8017070 con 0x7fd7f015f500 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8003da0 con 0x7fd7f015f500 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80052b0 con 0x7fd7f015f500 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f00b9fa0 con 0x7fd7f015f500 2024-04-26T23:52:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.497+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f00cc410 con 0x7fd7f015f500 2024-04-26T23:52:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.499+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f0069530 con 0x7fd7f015f500 2024-04-26T23:52:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.499+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7e801a070 con 0x7fd7f015f500 2024-04-26T23:52:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.501+0000 7fd7e6000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 0x7fd7d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.501+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fd7e808b420 con 0x7fd7f015f500 2024-04-26T23:52:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.502+0000 7fd7ede00640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 0x7fd7d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.503+0000 7fd7ede00640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 0x7fd7d006edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b7640 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.506+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7e80595f0 con 0x7fd7f015f500 2024-04-26T23:52:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.752+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f00d0ac0 con 0x7fd7f015f500 2024-04-26T23:52:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.753+0000 7fd7e6000640 1 -- 172.21.15.46:0/2744009681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7fd7e8058ef0 con 0x7fd7f015f500 2024-04-26T23:52:18.756 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:18.756 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.755+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 msgr2=0x7fd7d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.755+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 0x7fd7d006edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b7640 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.755+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 msgr2=0x7fd7f00b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.755+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c780 tx=0x7fd7e800cc40 comp rx=0 tx=0).stop 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 shutdown_connections 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd7d006c8f0 0x7fd7d006edb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f015f500 0x7fd7f00b2e10 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f00d56c0 0x7fd7f00b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 --2- 172.21.15.46:0/2744009681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7f00b3fb0 0x7fd7f00b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 >> 172.21.15.46:0/2744009681 conn(0x7fd7f0074400 msgr2=0x7fd7f00b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 shutdown_connections 2024-04-26T23:52:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.756+0000 7fd7f5000640 1 -- 172.21.15.46:0/2744009681 wait complete. 2024-04-26T23:52:18.779 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:18.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.980+0000 7f55a9200640 1 Processor -- start 2024-04-26T23:52:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.980+0000 7f55a9200640 1 -- start start 2024-04-26T23:52:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.981+0000 7f55a9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cff40 0x7f55a4073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.981+0000 7f55a9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55a4072190 0x7f55a4166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.981+0000 7f55a9200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a406f1d0 con 0x7f55a40cf450 2024-04-26T23:52:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f55a406f470 con 0x7f55a4072190 2024-04-26T23:52:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f55a4072930 con 0x7f55a40cff40 2024-04-26T23:52:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55a4072190 0x7f55a4166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55a4072190 0x7f55a4166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37204/0 (socket says 172.21.15.46:37204) 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a3e00640 1 -- 172.21.15.46:0/1145467240 learned_addr learned my addr 172.21.15.46:0/1145467240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.982+0000 7f55a2a00640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cff40 0x7f55a4073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.983+0000 7f55a3e00640 1 -- 172.21.15.46:0/1145467240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cff40 msgr2=0x7f55a4073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.983+0000 7f55a3e00640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cff40 0x7f55a4073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.983+0000 7f55a3e00640 1 -- 172.21.15.46:0/1145467240 >> v1:172.21.15.142:6789/0 conn(0x7f55a40cf450 legacy=0x7f55a40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.983+0000 7f55a3e00640 1 -- 172.21.15.46:0/1145467240 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a4072c10 con 0x7f55a4072190 2024-04-26T23:52:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.984+0000 7f55a3e00640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a4072190 0x7f55a4166b00 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5839f63aa9c2bf27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.985+0000 7f55a2000640 1 -- 172.21.15.46:0/1145467240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c037070 con 0x7f55a4072190 2024-04-26T23:52:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.985+0000 7f55a2000640 1 -- 172.21.15.46:0/1145467240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c032440 con 0x7f55a4072190 2024-04-26T23:52:18.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.985+0000 7f55a2000640 1 -- 172.21.15.46:0/1145467240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c032810 con 0x7f55a4072190 2024-04-26T23:52:18.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.985+0000 7f55a9200640 1 -- 172.21.15.46:0/1145467240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a4072190 msgr2=0x7f55a4166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.985+0000 7f55a9200640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a4072190 0x7f55a4166b00 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c02a270 comp rx=0 tx=0).stop 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 -- 172.21.15.46:0/1145467240 shutdown_connections 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a4072190 0x7f55a4166b00 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 --2- 172.21.15.46:0/1145467240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cff40 0x7f55a4073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 -- 172.21.15.46:0/1145467240 >> 172.21.15.46:0/1145467240 conn(0x7f55a40b2820 msgr2=0x7f55a40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 -- 172.21.15.46:0/1145467240 shutdown_connections 2024-04-26T23:52:18.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.986+0000 7f55a9200640 1 -- 172.21.15.46:0/1145467240 wait complete. 2024-04-26T23:52:18.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.988+0000 7f55a9200640 1 Processor -- start 2024-04-26T23:52:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.988+0000 7f55a9200640 1 -- start start 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 0x7f55a41e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cf450 0x7f55a41dd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40734b0 con 0x7f55a40cff40 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40731d0 con 0x7f55a4072190 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a9200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f55a4072600 con 0x7f55a40cf450 2024-04-26T23:52:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37220/0 (socket says 172.21.15.46:37220) 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 -- 172.21.15.46:0/1454360724 learned_addr learned my addr 172.21.15.46:0/1454360724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a2a00640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cf450 0x7f55a41dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3400640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 0x7f55a41e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 -- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cf450 msgr2=0x7f55a41dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cf450 0x7f55a41dd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 -- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 msgr2=0x7f55a41e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 0x7f55a41e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3e00640 1 -- 172.21.15.46:0/1454360724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41e1180 con 0x7f55a40cff40 2024-04-26T23:52:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.989+0000 7f55a3400640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 0x7f55a41e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f55a3e00640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c041020 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c03d640 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c037040 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41e1460 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.990+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41e1a20 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.991+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a40693b0 con 0x7f55a40cff40 2024-04-26T23:52:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.993+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f559c044020 con 0x7f55a40cff40 2024-04-26T23:52:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.993+0000 7f559b400640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 0x7f558006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:18.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.994+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f559c0ac430 con 0x7f55a40cff40 2024-04-26T23:52:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.994+0000 7f55a3400640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 0x7f558006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.995+0000 7f55a3400640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 0x7f558006edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5594001d70 tx=0x7f5594009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:18.998+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f559c04b030 con 0x7f55a40cff40 2024-04-26T23:52:19.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.296+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f55a41459d0 con 0x7f55a40cff40 2024-04-26T23:52:19.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.298+0000 7f559b400640 1 -- 172.21.15.46:0/1454360724 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1663 (secure 0 0 0) 0x7f559c07a610 con 0x7f55a40cff40 2024-04-26T23:52:19.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:19.301 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":930,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077264401,"bytes_used":2606956544,"bytes_avail":770487156736,"bytes_total":773094113280,"write_bytes_sec":50495,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-26T23:52:16.230959+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 msgr2=0x7f558006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 0x7f558006edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5594001d70 tx=0x7f5594009040 comp rx=0 tx=0).stop 2024-04-26T23:52:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 msgr2=0x7f55a412e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 shutdown_connections 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f558006c8f0 0x7f558006edb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55a40cff40 0x7f55a412e300 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55a40cf450 0x7f55a41dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 --2- 172.21.15.46:0/1454360724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a4072190 0x7f55a41e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.300+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 >> 172.21.15.46:0/1454360724 conn(0x7f55a40b2820 msgr2=0x7f55a4142670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.301+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 shutdown_connections 2024-04-26T23:52:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:19.301+0000 7f55a9200640 1 -- 172.21.15.46:0/1454360724 wait complete. 2024-04-26T23:52:19.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:52:20.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:20.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.539+0000 7f3bad200640 1 Processor -- start 2024-04-26T23:52:20.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.540+0000 7f3bad200640 1 -- start start 2024-04-26T23:52:20.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.541+0000 7f3bad200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ba815f4e0 0x7f3ba80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:20.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.541+0000 7f3bad200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba80d11f0 con 0x7f3ba80b4000 2024-04-26T23:52:20.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3bad200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba80d5f20 con 0x7f3ba80d2130 2024-04-26T23:52:20.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3bad200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba80d0ce0 con 0x7f3ba815f4e0 2024-04-26T23:52:20.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ba815f4e0 0x7f3ba80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:20.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ba80b4000 0x7f3ba80d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54660/0 (socket says 172.21.15.46:54660) 2024-04-26T23:52:20.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba6a00640 1 -- 172.21.15.46:0/1482624158 learned_addr learned my addr 172.21.15.46:0/1482624158 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba7400640 1 -- 172.21.15.46:0/1482624158 >> v1:172.21.15.142:6789/0 conn(0x7f3ba80d2130 legacy=0x7f3ba80b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba7400640 1 -- 172.21.15.46:0/1482624158 >> v1:172.21.15.46:6789/0 conn(0x7f3ba80b4000 legacy=0x7f3ba80d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba6000640 1 -- 172.21.15.46:0/1482624158 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2197232046 0 0) 0x7f3ba80d11f0 con 0x7f3ba80b4000 2024-04-26T23:52:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.542+0000 7f3ba7400640 1 -- 172.21.15.46:0/1482624158 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba80d0f10 con 0x7f3ba815f4e0 2024-04-26T23:52:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.544+0000 7f3ba7400640 1 --2- 172.21.15.46:0/1482624158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba815f4e0 0x7f3ba80d2df0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e771cef223e012b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.544+0000 7f3ba6000640 1 -- 172.21.15.46:0/1482624158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98037070 con 0x7f3ba815f4e0 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.544+0000 7f3ba6000640 1 -- 172.21.15.46:0/1482624158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98032440 con 0x7f3ba815f4e0 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.544+0000 7f3ba6000640 1 -- 172.21.15.46:0/1482624158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98032810 con 0x7f3ba815f4e0 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 -- 172.21.15.46:0/1482624158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba815f4e0 msgr2=0x7f3ba80d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 --2- 172.21.15.46:0/1482624158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba815f4e0 0x7f3ba80d2df0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b9802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 -- 172.21.15.46:0/1482624158 shutdown_connections 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 --2- 172.21.15.46:0/1482624158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba815f4e0 0x7f3ba80d2df0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 -- 172.21.15.46:0/1482624158 >> 172.21.15.46:0/1482624158 conn(0x7f3ba80d5010 msgr2=0x7f3ba80d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 -- 172.21.15.46:0/1482624158 shutdown_connections 2024-04-26T23:52:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.545+0000 7f3bad200640 1 -- 172.21.15.46:0/1482624158 wait complete. 2024-04-26T23:52:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.547+0000 7f3bad200640 1 Processor -- start 2024-04-26T23:52:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.547+0000 7f3bad200640 1 -- start start 2024-04-26T23:52:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 0x7f3ba806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 0x7f3ba806d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba80b1d60 con 0x7f3ba80b4000 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba80d2990 con 0x7f3ba815f4e0 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3bad200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba815f8c0 con 0x7f3ba80d2130 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3ba7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3ba7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37236/0 (socket says 172.21.15.46:37236) 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.548+0000 7f3ba7400640 1 -- 172.21.15.46:0/1668295573 learned_addr learned my addr 172.21.15.46:0/1668295573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 0x7f3ba806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 -- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 msgr2=0x7f3ba806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 0x7f3ba806cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 0x7f3ba806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 -- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 msgr2=0x7f3ba806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 0x7f3ba806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 -- 172.21.15.46:0/1668295573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba80cee20 con 0x7f3ba80b4000 2024-04-26T23:52:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 0x7f3ba806cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7e00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 0x7f3ba806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3ba7400640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98037040 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98004250 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98032e20 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.549+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba80b7e80 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.550+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba8159dc0 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.551+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba80693b0 con 0x7f3ba80b4000 2024-04-26T23:52:20.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.552+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b98030770 con 0x7f3ba80b4000 2024-04-26T23:52:20.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.553+0000 7f3b9f400640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 0x7f3b8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:20.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.553+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f3b980acfe0 con 0x7f3ba80b4000 2024-04-26T23:52:20.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.554+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 0x7f3b8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:20.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.555+0000 7f3ba6a00640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 0x7f3b8806edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3ba815a180 tx=0x7f3b94009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:20.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.558+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b9807b260 con 0x7f3ba80b4000 2024-04-26T23:52:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.795+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ba80d33d0 con 0x7f3ba80b4000 2024-04-26T23:52:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.796+0000 7f3b9f400640 1 -- 172.21.15.46:0/1668295573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f3b9807ab60 con 0x7f3ba80b4000 2024-04-26T23:52:20.799 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:20.802 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 msgr2=0x7f3b8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 0x7f3b8806edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3ba815a180 tx=0x7f3b94009040 comp rx=0 tx=0).stop 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 msgr2=0x7f3ba8159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b980046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 shutdown_connections 2024-04-26T23:52:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b8806c8f0 0x7f3b8806edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba815f4e0 0x7f3ba806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ba80d2130 0x7f3ba806cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 --2- 172.21.15.46:0/1668295573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ba80b4000 0x7f3ba8159880 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 >> 172.21.15.46:0/1668295573 conn(0x7f3ba80d5010 msgr2=0x7f3ba80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 shutdown_connections 2024-04-26T23:52:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:20.799+0000 7f3bad200640 1 -- 172.21.15.46:0/1668295573 wait complete. 2024-04-26T23:52:20.819 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:20.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.023+0000 7f3d0d000640 1 Processor -- start 2024-04-26T23:52:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.023+0000 7f3d0d000640 1 -- start start 2024-04-26T23:52:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.024+0000 7f3d0d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d080d1c90 0x7f3d080d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.024+0000 7f3d0d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d2070 0x7f3d080d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.024+0000 7f3d0d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d080d5e10 con 0x7f3d080d0ce0 2024-04-26T23:52:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.024+0000 7f3d0d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3d080cfc80 con 0x7f3d080d1c90 2024-04-26T23:52:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.024+0000 7f3d0d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d080cff20 con 0x7f3d080d2070 2024-04-26T23:52:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d080d1c90 0x7f3d080d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d080d1c90 0x7f3d080d35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37240/0 (socket says 172.21.15.46:37240) 2024-04-26T23:52:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 -- 172.21.15.46:0/384643800 learned_addr learned my addr 172.21.15.46:0/384643800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d06a00640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d2070 0x7f3d080d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 -- 172.21.15.46:0/384643800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d2070 msgr2=0x7f3d080d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d2070 0x7f3d080d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 -- 172.21.15.46:0/384643800 >> v1:172.21.15.142:6789/0 conn(0x7f3d080d0ce0 legacy=0x7f3d0815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.025+0000 7f3d07400640 1 -- 172.21.15.46:0/384643800 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d080d0200 con 0x7f3d080d1c90 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.026+0000 7f3d07400640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080d35f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3d080b8e70 tx=0x7f3cf802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446e06481505f59a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d06000640 1 -- 172.21.15.46:0/384643800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8036070 con 0x7f3d080d1c90 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d06000640 1 -- 172.21.15.46:0/384643800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf8031600 con 0x7f3d080d1c90 2024-04-26T23:52:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d06000640 1 -- 172.21.15.46:0/384643800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80319d0 con 0x7f3d080d1c90 2024-04-26T23:52:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 -- 172.21.15.46:0/384643800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 msgr2=0x7f3d080d35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080d35f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3d080b8e70 tx=0x7f3cf802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 -- 172.21.15.46:0/384643800 shutdown_connections 2024-04-26T23:52:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d2070 0x7f3d080d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 --2- 172.21.15.46:0/384643800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080d35f0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 -- 172.21.15.46:0/384643800 >> 172.21.15.46:0/384643800 conn(0x7f3d080d5010 msgr2=0x7f3d080d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 -- 172.21.15.46:0/384643800 shutdown_connections 2024-04-26T23:52:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.027+0000 7f3d0d000640 1 -- 172.21.15.46:0/384643800 wait complete. 2024-04-26T23:52:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.029+0000 7f3d0d000640 1 Processor -- start 2024-04-26T23:52:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.029+0000 7f3d0d000640 1 -- start start 2024-04-26T23:52:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d0ce0 0x7f3d080b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d080d2070 0x7f3d080b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0815f580 con 0x7f3d080d1c90 2024-04-26T23:52:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d080d07c0 con 0x7f3d080d2070 2024-04-26T23:52:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.030+0000 7f3d0d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0815f700 con 0x7f3d080d0ce0 2024-04-26T23:52:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37252/0 (socket says 172.21.15.46:37252) 2024-04-26T23:52:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 -- 172.21.15.46:0/3497980329 learned_addr learned my addr 172.21.15.46:0/3497980329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d07400640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d0ce0 0x7f3d080b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 -- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d0ce0 msgr2=0x7f3d080b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d0ce0 0x7f3d080b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 -- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d080d2070 msgr2=0x7f3d080b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d07e00640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d080d2070 0x7f3d080b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d080d2070 0x7f3d080b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 -- 172.21.15.46:0/3497980329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d080b1de0 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.031+0000 7f3d06a00640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000d5f0 tx=0x7f3cf000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.032+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00125f0 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.032+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf0019070 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.032+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00129d0 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.032+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d080b20c0 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.032+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d080b57d0 con 0x7f3d080d1c90 2024-04-26T23:52:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.033+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d08069ac0 con 0x7f3d080d1c90 2024-04-26T23:52:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.034+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cf0012b70 con 0x7f3d080d1c90 2024-04-26T23:52:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.035+0000 7f3cff400640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 0x7f3ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.035+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f3cf008c200 con 0x7f3d080d1c90 2024-04-26T23:52:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.036+0000 7f3d07400640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 0x7f3ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.037+0000 7f3d07400640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 0x7f3ce406edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8033000 tx=0x7f3cf8009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.039+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cf005a3d0 con 0x7f3d080d1c90 2024-04-26T23:52:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.327+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3d080d3220 con 0x7f3d080d1c90 2024-04-26T23:52:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.328+0000 7f3cff400640 1 -- 172.21.15.46:0/3497980329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1663 (secure 0 0 0) 0x7f3cf0059cd0 con 0x7f3d080d1c90 2024-04-26T23:52:21.331 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:21.331 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":932,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077264401,"bytes_used":2606956544,"bytes_avail":770487156736,"bytes_total":773094113280,"write_bytes_sec":50496,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-26T23:52:16.230959+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 msgr2=0x7f3ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 0x7f3ce406edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8033000 tx=0x7f3cf8009000 comp rx=0 tx=0).stop 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 msgr2=0x7f3d080b97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000d5f0 tx=0x7f3cf000dab0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 shutdown_connections 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ce406c8f0 0x7f3ce406edb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d080d2070 0x7f3d080b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d080d1c90 0x7f3d080b97b0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 --2- 172.21.15.46:0/3497980329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d080d0ce0 0x7f3d080b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 >> 172.21.15.46:0/3497980329 conn(0x7f3d080d5010 msgr2=0x7f3d0806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 shutdown_connections 2024-04-26T23:52:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:21.331+0000 7f3d0d000640 1 -- 172.21.15.46:0/3497980329 wait complete. 2024-04-26T23:52:21.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:52:22.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.574+0000 7fd85e000640 1 Processor -- start 2024-04-26T23:52:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.574+0000 7fd85e000640 1 -- start start 2024-04-26T23:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.575+0000 7fd85e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd8580cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd85e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd858073ad0 con 0x7fd8581466b0 2024-04-26T23:52:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd85e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd858146a90 con 0x7fd858160910 2024-04-26T23:52:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd85e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd858073d70 con 0x7fd8580720e0 2024-04-26T23:52:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd85cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8581466b0 0x7fd85806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54830/0 (socket says 172.21.15.46:54830) 2024-04-26T23:52:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd85cc00640 1 -- 172.21.15.46:0/3663068137 learned_addr learned my addr 172.21.15.46:0/3663068137 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360339168 0 0) 0x7fd858073ad0 con 0x7fd8581466b0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd83c003520 con 0x7fd8581466b0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.576+0000 7fd857e00640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd8580cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1028331100 0 0) 0x7fd83c003520 con 0x7fd8581466b0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 msgr2=0x7fd8580cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd8580cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 >> v1:172.21.15.142:6789/0 conn(0x7fd858160910 legacy=0x7fd8580cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd858074050 con 0x7fd8581466b0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fd848001670 con 0x7fd8581466b0 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 >> v1:172.21.15.46:6789/0 conn(0x7fd8581466b0 legacy=0x7fd85806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd83c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd83c004010 0x7fd83c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd83c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd858073d70 con 0x7fd83c004d40 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c00bb80 con 0x7fd83c004010 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd83c00bd80 con 0x7fd8580720e0 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.577+0000 7fd85d600640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd83c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85cc00640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd83c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd857e00640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd83c004010 0x7fd83c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85d600640 1 -- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 msgr2=0x7fd83c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85d600640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8580720e0 0x7fd83c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85d600640 1 -- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd83c004010 msgr2=0x7fd83c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85d600640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd83c004010 0x7fd83c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd85d600640 1 -- 172.21.15.46:0/3663068137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c003520 con 0x7fd83c004d40 2024-04-26T23:52:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.578+0000 7fd857e00640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd83c004010 0x7fd83c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.579+0000 7fd85d600640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd83c0051c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd850001e50 tx=0x7fd83c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31e9b15d0378939 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.579+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850001a70 con 0x7fd83c004d40 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.579+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850003970 con 0x7fd83c004d40 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.579+0000 7fd857400640 1 -- 172.21.15.46:0/3663068137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850003d60 con 0x7fd83c004d40 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 -- 172.21.15.46:0/3663068137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 msgr2=0x7fd83c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 --2- 172.21.15.46:0/3663068137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd83c0051c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd850001e50 tx=0x7fd83c005830 comp rx=0 tx=0).stop 2024-04-26T23:52:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85d600640 1 -- 172.21.15.46:0/3663068137 reap_dead start 2024-04-26T23:52:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 -- 172.21.15.46:0/3663068137 shutdown_connections 2024-04-26T23:52:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 -- 172.21.15.46:0/3663068137 >> 172.21.15.46:0/3663068137 conn(0x7fd8580b26d0 msgr2=0x7fd8580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 -- 172.21.15.46:0/3663068137 shutdown_connections 2024-04-26T23:52:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.580+0000 7fd85e000640 1 -- 172.21.15.46:0/3663068137 wait complete. 2024-04-26T23:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.582+0000 7fd85e000640 1 Processor -- start 2024-04-26T23:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.582+0000 7fd85e000640 1 -- start start 2024-04-26T23:52:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 0x7fd8580b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 0x7fd8581f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8580cc0a0 con 0x7fd83c004d40 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8580cb470 con 0x7fd8580b58e0 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.583+0000 7fd85e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8580cc220 con 0x7fd8581f7fb0 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45246/0 (socket says 172.21.15.46:45246) 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 -- 172.21.15.46:0/3320364668 learned_addr learned my addr 172.21.15.46:0/3320364668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd857e00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 0x7fd8580b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 -- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 msgr2=0x7fd8581f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85d600640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 0x7fd8581f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 0x7fd8581f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 -- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 msgr2=0x7fd8580b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 0x7fd8580b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85cc00640 1 -- 172.21.15.46:0/3320364668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85813fb80 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd857e00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 0x7fd8580b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.584+0000 7fd85d600640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 0x7fd8581f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd85cc00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fd84800e870 tx=0x7fd84800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84800c8d0 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd848018070 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84800ccb0 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8580b5e80 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.585+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8580b6440 con 0x7fd83c004d40 2024-04-26T23:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.586+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8580693b0 con 0x7fd83c004d40 2024-04-26T23:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.587+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd848013420 con 0x7fd83c004d40 2024-04-26T23:52:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.588+0000 7fd854c00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 0x7fd83806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.588+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fd84808b920 con 0x7fd83c004d40 2024-04-26T23:52:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.589+0000 7fd857e00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 0x7fd83806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.590+0000 7fd857e00640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 0x7fd83806ede0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd858157940 tx=0x7fd844008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:22.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.593+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd85813fb80 con 0x7fd83c004d40 2024-04-26T23:52:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.838+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd85806fd90 con 0x7fd83c004d40 2024-04-26T23:52:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.839+0000 7fd854c00640 1 -- 172.21.15.46:0/3320364668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7fd848059ce0 con 0x7fd83c004d40 2024-04-26T23:52:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:22.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 msgr2=0x7fd83806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 0x7fd83806ede0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd858157940 tx=0x7fd844008040 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 msgr2=0x7fd85813f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fd84800e870 tx=0x7fd84800ed30 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 shutdown_connections 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd83806c920 0x7fd83806ede0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8581f7fb0 0x7fd8581f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8580b58e0 0x7fd8580b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 --2- 172.21.15.46:0/3320364668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c004d40 0x7fd85813f640 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 >> 172.21.15.46:0/3320364668 conn(0x7fd8580b26d0 msgr2=0x7fd85806de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 shutdown_connections 2024-04-26T23:52:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:22.842+0000 7fd85e000640 1 -- 172.21.15.46:0/3320364668 wait complete. 2024-04-26T23:52:22.866 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:52:22.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:52:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.077+0000 7fc6b3200640 1 Processor -- start 2024-04-26T23:52:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.078+0000 7fc6b3200640 1 -- start start 2024-04-26T23:52:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.079+0000 7fc6b3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6ac0cff40 0x7fc6ac073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.079+0000 7fc6b3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac072190 0x7fc6ac166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.079+0000 7fc6b3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac06f1d0 con 0x7fc6ac0cf450 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.079+0000 7fc6b3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ac06f470 con 0x7fc6ac0cff40 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.079+0000 7fc6b3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac072930 con 0x7fc6ac072190 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6ac0cff40 0x7fc6ac073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6ac0cff40 0x7fc6ac073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45256/0 (socket says 172.21.15.46:45256) 2024-04-26T23:52:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 -- 172.21.15.46:0/3368571668 learned_addr learned my addr 172.21.15.46:0/3368571668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b2800640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac072190 0x7fc6ac166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 -- 172.21.15.46:0/3368571668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac072190 msgr2=0x7fc6ac166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac072190 0x7fc6ac166b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 -- 172.21.15.46:0/3368571668 >> v1:172.21.15.142:6789/0 conn(0x7fc6ac0cf450 legacy=0x7fc6ac0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.080+0000 7fc6b1400640 1 -- 172.21.15.46:0/3368571668 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac072c10 con 0x7fc6ac0cff40 2024-04-26T23:52:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b1400640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac0cff40 0x7fc6ac073db0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70051ec100b5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b0a00640 1 -- 172.21.15.46:0/3368571668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0037070 con 0x7fc6ac0cff40 2024-04-26T23:52:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b0a00640 1 -- 172.21.15.46:0/3368571668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a0032440 con 0x7fc6ac0cff40 2024-04-26T23:52:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b0a00640 1 -- 172.21.15.46:0/3368571668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0032810 con 0x7fc6ac0cff40 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b3200640 1 -- 172.21.15.46:0/3368571668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac0cff40 msgr2=0x7fc6ac073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.082+0000 7fc6b3200640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac0cff40 0x7fc6ac073db0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002a1c0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 -- 172.21.15.46:0/3368571668 shutdown_connections 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac072190 0x7fc6ac166b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 --2- 172.21.15.46:0/3368571668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac0cff40 0x7fc6ac073db0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 -- 172.21.15.46:0/3368571668 >> 172.21.15.46:0/3368571668 conn(0x7fc6ac0b2820 msgr2=0x7fc6ac0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 -- 172.21.15.46:0/3368571668 shutdown_connections 2024-04-26T23:52:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.083+0000 7fc6b3200640 1 -- 172.21.15.46:0/3368571668 wait complete. 2024-04-26T23:52:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.084+0000 7fc6b3200640 1 Processor -- start 2024-04-26T23:52:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.085+0000 7fc6b3200640 1 -- start start 2024-04-26T23:52:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0cf450 0x7fc6ac1417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac0cff40 0x7fc6ac0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0734b0 con 0x7fc6ac072190 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0731d0 con 0x7fc6ac0cf450 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac0725e0 con 0x7fc6ac0cff40 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45258/0 (socket says 172.21.15.46:45258) 2024-04-26T23:52:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 -- 172.21.15.46:0/578498893 learned_addr learned my addr 172.21.15.46:0/578498893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 -- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac0cff40 msgr2=0x7fc6ac0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac0cff40 0x7fc6ac0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1400640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0cf450 0x7fc6ac1417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 -- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0cf450 msgr2=0x7fc6ac1417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0cf450 0x7fc6ac1417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.086+0000 7fc6b1e00640 1 -- 172.21.15.46:0/578498893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac140750 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6b1e00640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c400 tx=0x7fc69c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c017070 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c0052d0 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c003830 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac140a30 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.087+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac140ff0 con 0x7fc6ac072190 2024-04-26T23:52:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.089+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac0693b0 con 0x7fc6ac072190 2024-04-26T23:52:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.089+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc69c01a070 con 0x7fc6ac072190 2024-04-26T23:52:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.090+0000 7fc6a6000640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.091+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fc69c08b070 con 0x7fc6ac072190 2024-04-26T23:52:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.091+0000 7fc6b1400640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.092+0000 7fc6b1400640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 0x7fc69006edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.095+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc69c0592e0 con 0x7fc6ac072190 2024-04-26T23:52:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.381+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc6ac074100 con 0x7fc6ac072190 2024-04-26T23:52:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.382+0000 7fc6a6000640 1 -- 172.21.15.46:0/578498893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1640 (secure 0 0 0) 0x7fc69c058be0 con 0x7fc6ac072190 2024-04-26T23:52:23.385 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:23.385 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":934,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077577227,"bytes_used":2607579136,"bytes_avail":770486534144,"bytes_total":773094113280,"write_bytes_sec":76598,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":118,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9663}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":33,"modified":"2024-04-26T23:52:22.233190+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:52:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 msgr2=0x7fc69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 0x7fc69006edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a002e000 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 msgr2=0x7fc6ac1f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c400 tx=0x7fc69c00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 shutdown_connections 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6ac0cff40 0x7fc6ac0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0cf450 0x7fc6ac1417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 --2- 172.21.15.46:0/578498893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ac072190 0x7fc6ac1f5e10 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.385+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 >> 172.21.15.46:0/578498893 conn(0x7fc6ac0b2820 msgr2=0x7fc6ac0af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.386+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 shutdown_connections 2024-04-26T23:52:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.386+0000 7fc6b3200640 1 -- 172.21.15.46:0/578498893 wait complete. 2024-04-26T23:52:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:52:23.409 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:52:23.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:52:23.509 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:52:23.624 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:52:23.624 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:52:23.625 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:23.625 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:23.684 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:23.684 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:23.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:23.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.950+0000 7fa9dfe00640 1 Processor -- start 2024-04-26T23:52:23.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.950+0000 7fa9dfe00640 1 -- start start 2024-04-26T23:52:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.951+0000 7fa9dfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 0x7fa9e00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e0073970 0x7fa9e00cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e00c6a50 0x7fa9e00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e0072bb0 con 0x7fa9e0073970 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e0072e90 con 0x7fa9e01454a0 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e0073170 con 0x7fa9e00c6a50 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9d7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e0073970 0x7fa9e00cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9d7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e0073970 0x7fa9e00cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45268/0 (socket says 172.21.15.46:45268) 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9d7e00640 1 -- 172.21.15.46:0/1850414379 learned_addr learned my addr 172.21.15.46:0/1850414379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9df400640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e00c6a50 0x7fa9e00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9dea00640 1 --2- 172.21.15.46:0/1850414379 >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 0x7fa9e00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9d7e00640 1 -- 172.21.15.46:0/1850414379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e00c6a50 msgr2=0x7fa9e00c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.952+0000 7fa9d7e00640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e00c6a50 0x7fa9e00c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.953+0000 7fa9d7e00640 1 -- 172.21.15.46:0/1850414379 >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 msgr2=0x7fa9e00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.953+0000 7fa9d7e00640 1 --2- 172.21.15.46:0/1850414379 >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 0x7fa9e00cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.953+0000 7fa9d7e00640 1 -- 172.21.15.46:0/1850414379 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e0073450 con 0x7fa9e0073970 2024-04-26T23:52:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.953+0000 7fa9dea00640 1 --2- 172.21.15.46:0/1850414379 >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 0x7fa9e00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9d7e00640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0073970 0x7fa9e00cdc30 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00d1ba0 tx=0x7fa9c802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d7d16c34902ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9de000640 1 -- 172.21.15.46:0/1850414379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8032790 con 0x7fa9e0073970 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9de000640 1 -- 172.21.15.46:0/1850414379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c8038070 con 0x7fa9e0073970 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9de000640 1 -- 172.21.15.46:0/1850414379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8032b50 con 0x7fa9e0073970 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1850414379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0073970 msgr2=0x7fa9e00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.954+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0073970 0x7fa9e00cdc30 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00d1ba0 tx=0x7fa9c802c890 comp rx=0 tx=0).stop 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1850414379 shutdown_connections 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e00c6a50 0x7fa9e00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1850414379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0073970 0x7fa9e00cdc30 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1850414379 >> v2:172.21.15.142:3300/0 conn(0x7fa9e01454a0 0x7fa9e00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1850414379 >> 172.21.15.46:0/1850414379 conn(0x7fa9e00b26d0 msgr2=0x7fa9e00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1850414379 shutdown_connections 2024-04-26T23:52:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.955+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1850414379 wait complete. 2024-04-26T23:52:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.956+0000 7fa9dfe00640 1 Processor -- start 2024-04-26T23:52:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.957+0000 7fa9dfe00640 1 -- start start 2024-04-26T23:52:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.957+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e0073970 0x7fa9e01390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.957+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 0x7fa9e015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.957+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9dfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0073e00 con 0x7fa9e01454a0 2024-04-26T23:52:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9dfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0072880 con 0x7fa9e00c6a50 2024-04-26T23:52:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9dfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e0145880 con 0x7fa9e0073970 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45282/0 (socket says 172.21.15.46:45282) 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9df400640 1 -- 172.21.15.46:0/1657430800 learned_addr learned my addr 172.21.15.46:0/1657430800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9d7e00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 0x7fa9e015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.958+0000 7fa9dea00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e0073970 0x7fa9e01390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 -- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e0073970 msgr2=0x7fa9e01390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e0073970 0x7fa9e01390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 -- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 msgr2=0x7fa9e015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 0x7fa9e015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 -- 172.21.15.46:0/1657430800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00caab0 con 0x7fa9e01454a0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9d7e00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 0x7fa9e015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9df400640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8007e60 tx=0x7fa9d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8014420 con 0x7fa9e01454a0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d8019070 con 0x7fa9e01454a0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8014820 con 0x7fa9e01454a0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e013abd0 con 0x7fa9e01454a0 2024-04-26T23:52:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.959+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e013b190 con 0x7fa9e01454a0 2024-04-26T23:52:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.960+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e00693b0 con 0x7fa9e01454a0 2024-04-26T23:52:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.962+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9d80149c0 con 0x7fa9e01454a0 2024-04-26T23:52:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.963+0000 7fa9d6a00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.963+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fa9d808c5e0 con 0x7fa9e01454a0 2024-04-26T23:52:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.963+0000 7fa9dea00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.964+0000 7fa9dea00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 0x7fa9bc06ede0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000a910 tx=0x7fa9d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:23.968+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9d805a7b0 con 0x7fa9e01454a0 2024-04-26T23:52:24.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.181+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9e00cefa0 con 0x7fa9e01454a0 2024-04-26T23:52:24.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.185+0000 7fa9d6a00640 1 -- 172.21.15.46:0/1657430800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16953 (secure 0 0 0) 0x7fa9d805a0b0 con 0x7fa9e01454a0 2024-04-26T23:52:24.188 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":150,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:57.743412+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 msgr2=0x7fa9bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 0x7fa9bc06ede0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000a910 tx=0x7fa9d0008040 comp rx=0 tx=0).stop 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 msgr2=0x7fa9e006de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8007e60 tx=0x7fa9d800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 shutdown_connections 2024-04-26T23:52:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e01454a0 0x7fa9e006de60 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e00c6a50 0x7fa9e015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 --2- 172.21.15.46:0/1657430800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e0073970 0x7fa9e01390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 >> 172.21.15.46:0/1657430800 conn(0x7fa9e00b26d0 msgr2=0x7fa9e00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 shutdown_connections 2024-04-26T23:52:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.189+0000 7fa9dfe00640 1 -- 172.21.15.46:0/1657430800 wait complete. 2024-04-26T23:52:24.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:16.864971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:18.039678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:52:24.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:52:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.418+0000 7f80f5000640 1 Processor -- start 2024-04-26T23:52:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.418+0000 7f80f5000640 1 -- start start 2024-04-26T23:52:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80f5000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80f015cec0 0x7f80f00cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80f5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80f00cdb30 0x7f80f00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80f5000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f00ce7d0 con 0x7f80f0073b30 2024-04-26T23:52:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80f5000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f80f00ceab0 con 0x7f80f00cdb30 2024-04-26T23:52:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80f5000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f80f00ced90 con 0x7f80f015cec0 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80eea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80f00cdb30 0x7f80f00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80eea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80f00cdb30 0x7f80f00735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45284/0 (socket says 172.21.15.46:45284) 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.420+0000 7f80eea00640 1 -- 172.21.15.46:0/1808810868 learned_addr learned my addr 172.21.15.46:0/1808810868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.421+0000 7f80ef400640 1 --2- 172.21.15.46:0/1808810868 >> v2:172.21.15.142:3300/0 conn(0x7f80f015cec0 0x7f80f00cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.421+0000 7f80eea00640 1 -- 172.21.15.46:0/1808810868 >> v2:172.21.15.142:3300/0 conn(0x7f80f015cec0 msgr2=0x7f80f00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.421+0000 7f80eea00640 1 --2- 172.21.15.46:0/1808810868 >> v2:172.21.15.142:3300/0 conn(0x7f80f015cec0 0x7f80f00cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.421+0000 7f80eea00640 1 -- 172.21.15.46:0/1808810868 >> v1:172.21.15.142:6789/0 conn(0x7f80f0073b30 legacy=0x7f80f0071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.421+0000 7f80eea00640 1 -- 172.21.15.46:0/1808810868 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f00cf070 con 0x7f80f00cdb30 2024-04-26T23:52:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.422+0000 7f80eea00640 1 --2- 172.21.15.46:0/1808810868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f00cdb30 0x7f80f00735f0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f80f00b3830 tx=0x7f80dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73d01928f732495 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.422+0000 7f80ee000640 1 -- 172.21.15.46:0/1808810868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc037070 con 0x7f80f00cdb30 2024-04-26T23:52:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.422+0000 7f80ee000640 1 -- 172.21.15.46:0/1808810868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc032440 con 0x7f80f00cdb30 2024-04-26T23:52:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.422+0000 7f80ee000640 1 -- 172.21.15.46:0/1808810868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc032810 con 0x7f80f00cdb30 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 -- 172.21.15.46:0/1808810868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f00cdb30 msgr2=0x7f80f00735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 --2- 172.21.15.46:0/1808810868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f00cdb30 0x7f80f00735f0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f80f00b3830 tx=0x7f80dc02a230 comp rx=0 tx=0).stop 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 -- 172.21.15.46:0/1808810868 shutdown_connections 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 --2- 172.21.15.46:0/1808810868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f00cdb30 0x7f80f00735f0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 --2- 172.21.15.46:0/1808810868 >> v2:172.21.15.142:3300/0 conn(0x7f80f015cec0 0x7f80f00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 -- 172.21.15.46:0/1808810868 >> 172.21.15.46:0/1808810868 conn(0x7f80f00b27b0 msgr2=0x7f80f00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 -- 172.21.15.46:0/1808810868 shutdown_connections 2024-04-26T23:52:24.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.423+0000 7f80f5000640 1 -- 172.21.15.46:0/1808810868 wait complete. 2024-04-26T23:52:24.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.425+0000 7f80f5000640 1 Processor -- start 2024-04-26T23:52:24.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.425+0000 7f80f5000640 1 -- start start 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80f0073b30 0x7f80f00ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80f00cdb30 0x7f80f0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0073f70 con 0x7f80f015cec0 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015d2a0 con 0x7f80f00cdb30 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.426+0000 7f80f5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80f00b4b50 con 0x7f80f0073b30 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45294/0 (socket says 172.21.15.46:45294) 2024-04-26T23:52:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 -- 172.21.15.46:0/2515531817 learned_addr learned my addr 172.21.15.46:0/2515531817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 -- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80f0073b30 msgr2=0x7f80f00ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80f0073b30 0x7f80f00ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 -- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80f00cdb30 msgr2=0x7f80f0154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80eea00640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80f00cdb30 0x7f80f0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80f00cdb30 0x7f80f0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 -- 172.21.15.46:0/2515531817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f00cc930 con 0x7f80f015cec0 2024-04-26T23:52:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.427+0000 7f80efe00640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f80e800c780 tx=0x7f80e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.428+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004b10 con 0x7f80f015cec0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.428+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e8004cb0 con 0x7f80f015cec0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.428+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004e60 con 0x7f80f015cec0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.428+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f00caa20 con 0x7f80f015cec0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.428+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f00c6e10 con 0x7f80f015cec0 2024-04-26T23:52:24.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.429+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f00693b0 con 0x7f80f015cec0 2024-04-26T23:52:24.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.430+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80e800aa90 con 0x7f80f015cec0 2024-04-26T23:52:24.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.432+0000 7f80e7400640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 0x7f80d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.432+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f80e808bf60 con 0x7f80f015cec0 2024-04-26T23:52:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.432+0000 7f80ef400640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 0x7f80d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.434+0000 7f80ef400640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 0x7f80d006edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006fd0 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.437+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80e8020070 con 0x7f80f015cec0 2024-04-26T23:52:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.650+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80f0073450 con 0x7f80f015cec0 2024-04-26T23:52:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.652+0000 7f80e7400640 1 -- 172.21.15.46:0/2515531817 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) ==== 93+0+10 (secure 0 0 0) 0x7f80e805a130 con 0x7f80f015cec0 2024-04-26T23:52:24.654 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:52:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.654+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 msgr2=0x7f80d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.654+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 0x7f80d006edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006fd0 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2024-04-26T23:52:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.654+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 msgr2=0x7f80f015ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.654+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f80e800c780 tx=0x7f80e800cc40 comp rx=0 tx=0).stop 2024-04-26T23:52:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 shutdown_connections 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80d006c8f0 0x7f80d006edb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f015cec0 0x7f80f015ab70 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80f00cdb30 0x7f80f0154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 --2- 172.21.15.46:0/2515531817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80f0073b30 0x7f80f00ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 >> 172.21.15.46:0/2515531817 conn(0x7f80f00b27b0 msgr2=0x7f80f015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 shutdown_connections 2024-04-26T23:52:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.655+0000 7f80f5000640 1 -- 172.21.15.46:0/2515531817 wait complete. 2024-04-26T23:52:24.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:52:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.891+0000 7fecbde00640 1 Processor -- start 2024-04-26T23:52:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.892+0000 7fecbde00640 1 -- start start 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.893+0000 7fecbde00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fecb80b53b0 0x7fecb815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.893+0000 7fecbde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecb815e050 0x7fecb806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.893+0000 7fecbde00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb80d07e0 con 0x7fecb815d730 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.893+0000 7fecbde00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fecb806eb90 con 0x7fecb815e050 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.893+0000 7fecbde00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fecb80b4310 con 0x7fecb80b53b0 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecb815d730 0x7fecb80d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54842/0 (socket says 172.21.15.46:54842) 2024-04-26T23:52:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb6a00640 1 -- 172.21.15.46:0/522529911 learned_addr learned my addr 172.21.15.46:0/522529911 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb7400640 1 --2- 172.21.15.46:0/522529911 >> v2:172.21.15.46:3300/0 conn(0x7fecb815e050 0x7fecb806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb6000640 1 --2- 172.21.15.46:0/522529911 >> v2:172.21.15.142:3300/0 conn(0x7fecb80b53b0 0x7fecb815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb5600640 1 -- 172.21.15.46:0/522529911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003671843 0 0) 0x7fecb80d07e0 con 0x7fecb815d730 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb5600640 1 -- 172.21.15.46:0/522529911 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec9c003520 con 0x7fecb815d730 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb7400640 1 -- 172.21.15.46:0/522529911 >> v2:172.21.15.142:3300/0 conn(0x7fecb80b53b0 msgr2=0x7fecb815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb7400640 1 --2- 172.21.15.46:0/522529911 >> v2:172.21.15.142:3300/0 conn(0x7fecb80b53b0 0x7fecb815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb7400640 1 -- 172.21.15.46:0/522529911 >> v1:172.21.15.46:6789/0 conn(0x7fecb815d730 legacy=0x7fecb80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.894+0000 7fecb7400640 1 -- 172.21.15.46:0/522529911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb80b45f0 con 0x7fecb815e050 2024-04-26T23:52:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.895+0000 7fecb7400640 1 --2- 172.21.15.46:0/522529911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb806dc90 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fecb806ffa0 tx=0x7fecb002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0403c47317f6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecb5600640 1 -- 172.21.15.46:0/522529911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0007740 con 0x7fecb815e050 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecb5600640 1 -- 172.21.15.46:0/522529911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb00078e0 con 0x7fecb815e050 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecb5600640 1 -- 172.21.15.46:0/522529911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0007cb0 con 0x7fecb815e050 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 -- 172.21.15.46:0/522529911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 msgr2=0x7fecb806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 --2- 172.21.15.46:0/522529911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb806dc90 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fecb806ffa0 tx=0x7fecb002a680 comp rx=0 tx=0).stop 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 -- 172.21.15.46:0/522529911 shutdown_connections 2024-04-26T23:52:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 --2- 172.21.15.46:0/522529911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb806dc90 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 --2- 172.21.15.46:0/522529911 >> v2:172.21.15.142:3300/0 conn(0x7fecb80b53b0 0x7fecb815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.896+0000 7fecbde00640 1 -- 172.21.15.46:0/522529911 >> 172.21.15.46:0/522529911 conn(0x7fecb806c980 msgr2=0x7fecb80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.897+0000 7fecbde00640 1 -- 172.21.15.46:0/522529911 shutdown_connections 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.897+0000 7fecbde00640 1 -- 172.21.15.46:0/522529911 wait complete. 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.898+0000 7fecbde00640 1 Processor -- start 2024-04-26T23:52:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.899+0000 7fecbde00640 1 -- start start 2024-04-26T23:52:24.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 0x7fecb8144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecb815d730 0x7fecb813f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80b4e90 con 0x7fecb815e050 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80b4bb0 con 0x7fecb80b53b0 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecbde00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecb815e460 con 0x7fecb815d730 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45322/0 (socket says 172.21.15.46:45322) 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 -- 172.21.15.46:0/2532361093 learned_addr learned my addr 172.21.15.46:0/2532361093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb6a00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 0x7fecb8144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb6000640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecb815d730 0x7fecb813f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 -- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecb815d730 msgr2=0x7fecb813f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecb815d730 0x7fecb813f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 -- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 msgr2=0x7fecb8144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.900+0000 7fecb7400640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 0x7fecb8144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecb7400640 1 -- 172.21.15.46:0/2532361093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb813f680 con 0x7fecb815e050 2024-04-26T23:52:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecb6a00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 0x7fecb8144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecb7400640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fecb002ac70 tx=0x7fecb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0040070 con 0x7fecb815e050 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb0031bc0 con 0x7fecb815e050 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0009450 con 0x7fecb815e050 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.901+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb815d2f0 con 0x7fecb815e050 2024-04-26T23:52:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.902+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb81ec660 con 0x7fecb815e050 2024-04-26T23:52:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.903+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb8069530 con 0x7fecb815e050 2024-04-26T23:52:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.904+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecb0039040 con 0x7fecb815e050 2024-04-26T23:52:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.905+0000 7fecaf400640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 0x7fec9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.905+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fecb00ad510 con 0x7fecb815e050 2024-04-26T23:52:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.905+0000 7fecb6a00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 0x7fec9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.906+0000 7fecb6a00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 0x7fec9406edb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7feca8005880 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:24.909+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecb007b930 con 0x7fecb815e050 2024-04-26T23:52:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.120+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fecb815c1c0 con 0x7fecb815e050 2024-04-26T23:52:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.121+0000 7fecaf400640 1 -- 172.21.15.46:0/2532361093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) ==== 104+0+11 (secure 0 0 0) 0x7fecb007b230 con 0x7fecb815e050 2024-04-26T23:52:25.124 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:52:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 msgr2=0x7fec9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 0x7fec9406edb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7feca8005880 tx=0x7feca8009040 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 msgr2=0x7fecb81f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fecb002ac70 tx=0x7fecb0004a40 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 shutdown_connections 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec9406c8f0 0x7fec9406edb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb815e050 0x7fecb81f4d60 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecb815d730 0x7fecb813f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 --2- 172.21.15.46:0/2532361093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb80b53b0 0x7fecb8144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 >> 172.21.15.46:0/2532361093 conn(0x7fecb806c980 msgr2=0x7fecb80b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 shutdown_connections 2024-04-26T23:52:25.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.124+0000 7fecbde00640 1 -- 172.21.15.46:0/2532361093 wait complete. 2024-04-26T23:52:25.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:52:25.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.363+0000 7f5a2cc00640 1 Processor -- start 2024-04-26T23:52:25.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.363+0000 7f5a2cc00640 1 -- start start 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a2cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a280cfb60 0x7f5a280cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a2cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 0x7f5a280b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a2cc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a280d4090 con 0x7f5a2815f840 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a2cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5a280d3880 con 0x7f5a280cfb60 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a2cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b4020 con 0x7f5a280d0500 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a1ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a280cfb60 0x7f5a280cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a1ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a280cfb60 0x7f5a280cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45328/0 (socket says 172.21.15.46:45328) 2024-04-26T23:52:25.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.365+0000 7f5a1ea00640 1 -- 172.21.15.46:0/3250174965 learned_addr learned my addr 172.21.15.46:0/3250174965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a27e00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 0x7f5a280b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a1ea00640 1 -- 172.21.15.46:0/3250174965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 msgr2=0x7f5a280b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a1ea00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 0x7f5a280b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a1ea00640 1 -- 172.21.15.46:0/3250174965 >> v1:172.21.15.142:6789/0 conn(0x7f5a2815f840 legacy=0x7f5a280d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a1ea00640 1 -- 172.21.15.46:0/3250174965 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280d3ad0 con 0x7f5a280cfb60 2024-04-26T23:52:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.366+0000 7f5a27e00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 0x7f5a280b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.367+0000 7f5a1ea00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280cfb60 0x7f5a280cffc0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009870 tx=0x7f5a1002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61e74c63a94ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.367+0000 7f5a26a00640 1 -- 172.21.15.46:0/3250174965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10037070 con 0x7f5a280cfb60 2024-04-26T23:52:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.367+0000 7f5a26a00640 1 -- 172.21.15.46:0/3250174965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a10032440 con 0x7f5a280cfb60 2024-04-26T23:52:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.367+0000 7f5a26a00640 1 -- 172.21.15.46:0/3250174965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10032810 con 0x7f5a280cfb60 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 -- 172.21.15.46:0/3250174965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280cfb60 msgr2=0x7f5a280cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280cfb60 0x7f5a280cffc0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009870 tx=0x7f5a1002a200 comp rx=0 tx=0).stop 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 -- 172.21.15.46:0/3250174965 shutdown_connections 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280d0500 0x7f5a280b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/3250174965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280cfb60 0x7f5a280cffc0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 -- 172.21.15.46:0/3250174965 >> 172.21.15.46:0/3250174965 conn(0x7f5a280d4f70 msgr2=0x7f5a280d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:25.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 -- 172.21.15.46:0/3250174965 shutdown_connections 2024-04-26T23:52:25.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.368+0000 7f5a2cc00640 1 -- 172.21.15.46:0/3250174965 wait complete. 2024-04-26T23:52:25.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.370+0000 7f5a2cc00640 1 Processor -- start 2024-04-26T23:52:25.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.370+0000 7f5a2cc00640 1 -- start start 2024-04-26T23:52:25.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 0x7f5a28153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a2815f840 0x7f5a28161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280d0990 con 0x7f5a280d0500 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b27d0 con 0x7f5a2815f840 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a2cc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b3230 con 0x7f5a280cfb60 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45340/0 (socket says 172.21.15.46:45340) 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.371+0000 7f5a1ea00640 1 -- 172.21.15.46:0/167879187 learned_addr learned my addr 172.21.15.46:0/167879187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a27400640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 0x7f5a28153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a27e00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a2815f840 0x7f5a28161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 -- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 msgr2=0x7f5a28153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 0x7f5a28153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 -- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a2815f840 msgr2=0x7f5a28161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a2815f840 0x7f5a28161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 -- 172.21.15.46:0/167879187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280cc540 con 0x7f5a280d0500 2024-04-26T23:52:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a27400640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 0x7f5a28153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.372+0000 7f5a1ea00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f5a1002a240 tx=0x7f5a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.373+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10037040 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.373+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a10042070 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.373+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a280b7b20 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.373+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10030bd0 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.373+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a280b80e0 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.374+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a280693b0 con 0x7f5a280d0500 2024-04-26T23:52:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.375+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a1002ad10 con 0x7f5a280d0500 2024-04-26T23:52:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.376+0000 7f5a1fe00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.376+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f5a100ac640 con 0x7f5a280d0500 2024-04-26T23:52:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.376+0000 7f5a27400640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.378+0000 7f5a27400640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800a830 tx=0x7f5a18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.380+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a280d4090 con 0x7f5a280d0500 2024-04-26T23:52:25.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.585+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a280d5d10 con 0x7f5a280d0500 2024-04-26T23:52:25.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.586+0000 7f5a1fe00640 1 -- 172.21.15.46:0/167879187 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) ==== 100+0+11 (secure 0 0 0) 0x7f5a10045360 con 0x7f5a280d0500 2024-04-26T23:52:25.588 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:52:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 msgr2=0x7f5a0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800a830 tx=0x7f5a18008040 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 msgr2=0x7f5a28152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f5a1002a240 tx=0x7f5a100046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 shutdown_connections 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.588+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a2815f840 0x7f5a28161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.589+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a280d0500 0x7f5a28152500 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.589+0000 7f5a2cc00640 1 --2- 172.21.15.46:0/167879187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a280cfb60 0x7f5a28153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.589+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 >> 172.21.15.46:0/167879187 conn(0x7f5a280d4f70 msgr2=0x7f5a2815e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.589+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 shutdown_connections 2024-04-26T23:52:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.589+0000 7f5a2cc00640 1 -- 172.21.15.46:0/167879187 wait complete. 2024-04-26T23:52:25.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.824+0000 7fcd95c00640 1 Processor -- start 2024-04-26T23:52:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.825+0000 7fcd95c00640 1 -- start start 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd95c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd900b4000 0x7fcd900d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd95c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd900d11f0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd95c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd900d5f20 con 0x7fcd900d2130 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd95c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcd900d0ce0 con 0x7fcd900b4000 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd8fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd9015f4e0 0x7fcd900d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54850/0 (socket says 172.21.15.46:54850) 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd8fe00640 1 -- 172.21.15.46:0/3997245920 learned_addr learned my addr 172.21.15.46:0/3997245920 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.826+0000 7fcd95200640 1 --2- 172.21.15.46:0/3997245920 >> v2:172.21.15.46:3300/0 conn(0x7fcd900b4000 0x7fcd900d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.827+0000 7fcd8ea00640 1 -- 172.21.15.46:0/3997245920 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713521285 0 0) 0x7fcd900d11f0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.827+0000 7fcd8ea00640 1 -- 172.21.15.46:0/3997245920 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd74003520 con 0x7fcd9015f4e0 2024-04-26T23:52:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.827+0000 7fcd95200640 1 -- 172.21.15.46:0/3997245920 >> v1:172.21.15.142:6789/0 conn(0x7fcd900d2130 legacy=0x7fcd900b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.827+0000 7fcd95200640 1 -- 172.21.15.46:0/3997245920 >> v1:172.21.15.46:6789/0 conn(0x7fcd9015f4e0 legacy=0x7fcd900d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.827+0000 7fcd95200640 1 -- 172.21.15.46:0/3997245920 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900d0f10 con 0x7fcd900b4000 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.828+0000 7fcd95200640 1 --2- 172.21.15.46:0/3997245920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd900b4000 0x7fcd900d2df0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fcd880097d0 tx=0x7fcd8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dfdc9fa89f26a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.828+0000 7fcd8ea00640 1 -- 172.21.15.46:0/3997245920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88037070 con 0x7fcd900b4000 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.828+0000 7fcd8ea00640 1 -- 172.21.15.46:0/3997245920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd88032440 con 0x7fcd900b4000 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.828+0000 7fcd8ea00640 1 -- 172.21.15.46:0/3997245920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880327f0 con 0x7fcd900b4000 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 -- 172.21.15.46:0/3997245920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd900b4000 msgr2=0x7fcd900d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 --2- 172.21.15.46:0/3997245920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd900b4000 0x7fcd900d2df0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fcd880097d0 tx=0x7fcd8802a210 comp rx=0 tx=0).stop 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 -- 172.21.15.46:0/3997245920 shutdown_connections 2024-04-26T23:52:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 --2- 172.21.15.46:0/3997245920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd900b4000 0x7fcd900d2df0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 -- 172.21.15.46:0/3997245920 >> 172.21.15.46:0/3997245920 conn(0x7fcd900d5010 msgr2=0x7fcd900d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 -- 172.21.15.46:0/3997245920 shutdown_connections 2024-04-26T23:52:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.829+0000 7fcd95c00640 1 -- 172.21.15.46:0/3997245920 wait complete. 2024-04-26T23:52:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.831+0000 7fcd95c00640 1 Processor -- start 2024-04-26T23:52:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.831+0000 7fcd95c00640 1 -- start start 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 0x7fcd900a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd900d2130 0x7fcd900a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900b1d60 con 0x7fcd9015f4e0 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900d2990 con 0x7fcd900d2130 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.832+0000 7fcd95c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9015f8c0 con 0x7fcd900b4000 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45366/0 (socket says 172.21.15.46:45366) 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 -- 172.21.15.46:0/4214916554 learned_addr learned my addr 172.21.15.46:0/4214916554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 0x7fcd900a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 -- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 msgr2=0x7fcd900a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 0x7fcd900a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 -- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd900d2130 msgr2=0x7fcd900a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd900d2130 0x7fcd900a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 -- 172.21.15.46:0/4214916554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900a1620 con 0x7fcd9015f4e0 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 0x7fcd900a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.833+0000 7fcd95200640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009720 tx=0x7fcd880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.834+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88037040 con 0x7fcd9015f4e0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.834+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd88004230 con 0x7fcd9015f4e0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.834+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88003610 con 0x7fcd9015f4e0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.834+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd900a1900 con 0x7fcd9015f4e0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.834+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd900a1ec0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.835+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd900693b0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.836+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd880037b0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.837+0000 7fcd87e00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 0x7fcd6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.837+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fcd880acad0 con 0x7fcd9015f4e0 2024-04-26T23:52:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.838+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 0x7fcd6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.839+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 0x7fcd6c06edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005880 tx=0x7fcd78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:25.841+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd8807aca0 con 0x7fcd9015f4e0 2024-04-26T23:52:26.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.061+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd900d33d0 con 0x7fcd9015f4e0 2024-04-26T23:52:26.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.062+0000 7fcd87e00640 1 -- 172.21.15.46:0/4214916554 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v150) ==== 80+0+0 (secure 0 0 0) 0x7fcd8807a5a0 con 0x7fcd9015f4e0 2024-04-26T23:52:26.065 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:52:26.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.064+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 msgr2=0x7fcd6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.064+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 0x7fcd6c06edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005880 tx=0x7fcd78009040 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.064+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 msgr2=0x7fcd90160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.064+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009720 tx=0x7fcd880046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 shutdown_connections 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd6c06c8f0 0x7fcd6c06edb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd9015f4e0 0x7fcd90160af0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd900d2130 0x7fcd900a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 --2- 172.21.15.46:0/4214916554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd900b4000 0x7fcd900a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 >> 172.21.15.46:0/4214916554 conn(0x7fcd900d5010 msgr2=0x7fcd900b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 shutdown_connections 2024-04-26T23:52:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:26.065+0000 7fcd95c00640 1 -- 172.21.15.46:0/4214916554 wait complete. 2024-04-26T23:52:26.086 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:52:26.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:26.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:52:26.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:52:26.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:26.728 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:52:26.772 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:52:26.772 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:52:26.772 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:26.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:52:26.812 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:52:26.812 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:26.812 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:26.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:52:26.853 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:52:26.854 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:52:26.854 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:26.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:52:26.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.104+0000 7faf79c00640 1 Processor -- start 2024-04-26T23:52:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.105+0000 7faf79c00640 1 -- start start 2024-04-26T23:52:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.106+0000 7faf79c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 0x7faf740cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.106+0000 7faf79c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.106+0000 7faf79c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740b03c0 con 0x7faf740cb1a0 2024-04-26T23:52:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.106+0000 7faf79c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faf740cf5c0 con 0x7faf7415f1d0 2024-04-26T23:52:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.106+0000 7faf79c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf740cb680 con 0x7faf7415edf0 2024-04-26T23:52:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.107+0000 7faf73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 0x7faf740cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.107+0000 7faf6be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.107+0000 7faf6be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59422/0 (socket says 172.21.15.46:59422) 2024-04-26T23:52:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.107+0000 7faf6be00640 1 -- 172.21.15.46:0/1064628969 learned_addr learned my addr 172.21.15.46:0/1064628969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.108+0000 7faf73e00640 1 -- 172.21.15.46:0/1064628969 >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 msgr2=0x7faf740cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.108+0000 7faf73e00640 1 --2- 172.21.15.46:0/1064628969 >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.108+0000 7faf73e00640 1 -- 172.21.15.46:0/1064628969 >> v1:172.21.15.142:6789/0 conn(0x7faf740cb1a0 legacy=0x7faf740ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.108+0000 7faf73e00640 1 -- 172.21.15.46:0/1064628969 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740cb960 con 0x7faf7415edf0 2024-04-26T23:52:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.108+0000 7faf6be00640 1 --2- 172.21.15.46:0/1064628969 >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.109+0000 7faf73e00640 1 --2- 172.21.15.46:0/1064628969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 0x7faf740cf080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf640097a0 tx=0x7faf6402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db9af1cfc49cbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.110+0000 7faf73400640 1 -- 172.21.15.46:0/1064628969 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64037070 con 0x7faf7415edf0 2024-04-26T23:52:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.110+0000 7faf73400640 1 -- 172.21.15.46:0/1064628969 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf64032440 con 0x7faf7415edf0 2024-04-26T23:52:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.110+0000 7faf73400640 1 -- 172.21.15.46:0/1064628969 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64032810 con 0x7faf7415edf0 2024-04-26T23:52:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.110+0000 7faf79c00640 1 -- 172.21.15.46:0/1064628969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 msgr2=0x7faf740cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.110+0000 7faf79c00640 1 --2- 172.21.15.46:0/1064628969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 0x7faf740cf080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf640097a0 tx=0x7faf6402a210 comp rx=0 tx=0).stop 2024-04-26T23:52:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 -- 172.21.15.46:0/1064628969 shutdown_connections 2024-04-26T23:52:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 --2- 172.21.15.46:0/1064628969 >> v2:172.21.15.142:3300/0 conn(0x7faf7415f1d0 0x7faf740cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 --2- 172.21.15.46:0/1064628969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7415edf0 0x7faf740cf080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 -- 172.21.15.46:0/1064628969 >> 172.21.15.46:0/1064628969 conn(0x7faf740b26d0 msgr2=0x7faf740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 -- 172.21.15.46:0/1064628969 shutdown_connections 2024-04-26T23:52:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.111+0000 7faf79c00640 1 -- 172.21.15.46:0/1064628969 wait complete. 2024-04-26T23:52:27.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.113+0000 7faf79c00640 1 Processor -- start 2024-04-26T23:52:27.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.113+0000 7faf79c00640 1 -- start start 2024-04-26T23:52:27.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740cb1a0 0x7faf74160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7414b240 0x7faf7415d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf740cc200 con 0x7faf7415edf0 2024-04-26T23:52:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf740cbf20 con 0x7faf740cb1a0 2024-04-26T23:52:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.114+0000 7faf79c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf740c6950 con 0x7faf7414b240 2024-04-26T23:52:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45374/0 (socket says 172.21.15.46:45374) 2024-04-26T23:52:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 -- 172.21.15.46:0/3973217957 learned_addr learned my addr 172.21.15.46:0/3973217957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:27.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf79200640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7414b240 0x7faf7415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf73e00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740cb1a0 0x7faf74160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 -- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7414b240 msgr2=0x7faf7415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7414b240 0x7faf7415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 -- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740cb1a0 msgr2=0x7faf74160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740cb1a0 0x7faf74160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.115+0000 7faf6be00640 1 -- 172.21.15.46:0/3973217957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7414e9f0 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf6be00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9e0 tx=0x7faf5c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c017070 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c00c9d0 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00cd80 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf741613d0 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.116+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7414df10 con 0x7faf7415edf0 2024-04-26T23:52:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.117+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf740693b0 con 0x7faf7415edf0 2024-04-26T23:52:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.118+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf5c01a070 con 0x7faf7415edf0 2024-04-26T23:52:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.119+0000 7faf70c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.119+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7faf5c08bb90 con 0x7faf7415edf0 2024-04-26T23:52:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.120+0000 7faf73e00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.121+0000 7faf73e00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf64034000 tx=0x7faf6402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.124+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf5c059df0 con 0x7faf7415edf0 2024-04-26T23:52:27.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.368+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf740ce760 con 0x7faf7415edf0 2024-04-26T23:52:27.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.369+0000 7faf70c00640 1 -- 172.21.15.46:0/3973217957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7faf5c0596f0 con 0x7faf7415edf0 2024-04-26T23:52:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:27.373 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 msgr2=0x7faf5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf64034000 tx=0x7faf6402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 msgr2=0x7faf7415ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9e0 tx=0x7faf5c00eea0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 shutdown_connections 2024-04-26T23:52:27.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf7414b240 0x7faf7415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7415edf0 0x7faf7415ffd0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 --2- 172.21.15.46:0/3973217957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740cb1a0 0x7faf74160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.372+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 >> 172.21.15.46:0/3973217957 conn(0x7faf740b26d0 msgr2=0x7faf740b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.373+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 shutdown_connections 2024-04-26T23:52:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.373+0000 7faf79c00640 1 -- 172.21.15.46:0/3973217957 wait complete. 2024-04-26T23:52:27.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.602+0000 7fbb33e00640 1 Processor -- start 2024-04-26T23:52:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.602+0000 7fbb33e00640 1 -- start start 2024-04-26T23:52:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.603+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb3406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.603+0000 7fbb33e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb34160910 0x7fbb34160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.603+0000 7fbb33e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb34073ad0 con 0x7fbb340720e0 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.603+0000 7fbb33e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbb34146a90 con 0x7fbb34160910 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.603+0000 7fbb33e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb34073d70 con 0x7fbb341466b0 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb33400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb34160910 0x7fbb34160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb32000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb340720e0 0x7fbb340cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54862/0 (socket says 172.21.15.46:54862) 2024-04-26T23:52:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb32000640 1 -- 172.21.15.46:0/1215764265 learned_addr learned my addr 172.21.15.46:0/1215764265 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb32a00640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb3406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb33400640 1 -- 172.21.15.46:0/1215764265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 msgr2=0x7fbb3406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb33400640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb3406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb33400640 1 -- 172.21.15.46:0/1215764265 >> v1:172.21.15.46:6789/0 conn(0x7fbb340720e0 legacy=0x7fbb340cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb31600640 1 -- 172.21.15.46:0/1215764265 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980206559 0 0) 0x7fbb34073ad0 con 0x7fbb340720e0 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb33400640 1 -- 172.21.15.46:0/1215764265 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb34074050 con 0x7fbb34160910 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.604+0000 7fbb32a00640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb3406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.605+0000 7fbb33400640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb34160da0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097a0 tx=0x7fbb2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4040c3c636e831bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb31600640 1 -- 172.21.15.46:0/1215764265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c037070 con 0x7fbb34160910 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb31600640 1 -- 172.21.15.46:0/1215764265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2c032440 con 0x7fbb34160910 2024-04-26T23:52:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb31600640 1 -- 172.21.15.46:0/1215764265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c032810 con 0x7fbb34160910 2024-04-26T23:52:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 -- 172.21.15.46:0/1215764265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 msgr2=0x7fbb34160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb34160da0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097a0 tx=0x7fbb2c02a180 comp rx=0 tx=0).stop 2024-04-26T23:52:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 -- 172.21.15.46:0/1215764265 shutdown_connections 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb34160da0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 --2- 172.21.15.46:0/1215764265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb3406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 -- 172.21.15.46:0/1215764265 >> 172.21.15.46:0/1215764265 conn(0x7fbb340b26d0 msgr2=0x7fbb340b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 -- 172.21.15.46:0/1215764265 shutdown_connections 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.606+0000 7fbb33e00640 1 -- 172.21.15.46:0/1215764265 wait complete. 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.608+0000 7fbb33e00640 1 Processor -- start 2024-04-26T23:52:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.608+0000 7fbb33e00640 1 -- start start 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 0x7fbb34160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb340b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3406e3c0 con 0x7fbb34160910 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340cb470 con 0x7fbb340720e0 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.609+0000 7fbb33e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb340cb990 con 0x7fbb341466b0 2024-04-26T23:52:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45394/0 (socket says 172.21.15.46:45394) 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 -- 172.21.15.46:0/3034238578 learned_addr learned my addr 172.21.15.46:0/3034238578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb32000640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb340b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb32a00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 0x7fbb34160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 -- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 msgr2=0x7fbb340b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb340b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 -- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 msgr2=0x7fbb34160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 0x7fbb34160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb33400640 1 -- 172.21.15.46:0/3034238578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb341f5770 con 0x7fbb34160910 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb32000640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb340b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.610+0000 7fbb32a00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 0x7fbb34160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb33400640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c041070 con 0x7fbb34160910 2024-04-26T23:52:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2c03c870 con 0x7fbb34160910 2024-04-26T23:52:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb340ae980 con 0x7fbb34160910 2024-04-26T23:52:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c037040 con 0x7fbb34160910 2024-04-26T23:52:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.611+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb340aef40 con 0x7fbb34160910 2024-04-26T23:52:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.612+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb340693b0 con 0x7fbb34160910 2024-04-26T23:52:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.613+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb2c044070 con 0x7fbb34160910 2024-04-26T23:52:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.614+0000 7fbb2aa00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 0x7fbb1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.615+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7fbb2c0ac8b0 con 0x7fbb34160910 2024-04-26T23:52:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.615+0000 7fbb32a00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 0x7fbb1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.616+0000 7fbb32a00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 0x7fbb1006edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb34144f50 tx=0x7fbb24007550 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.619+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb2c07ab10 con 0x7fbb34160910 2024-04-26T23:52:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.814+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb340c7180 con 0x7fbb34160910 2024-04-26T23:52:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.816+0000 7fbb2aa00640 1 -- 172.21.15.46:0/3034238578 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16953 (secure 0 0 0) 0x7fbb2c07a410 con 0x7fbb34160910 2024-04-26T23:52:27.819 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":150,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:57.743412+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.818+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 msgr2=0x7fbb1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.818+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 0x7fbb1006edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb34144f50 tx=0x7fbb24007550 comp rx=0 tx=0).stop 2024-04-26T23:52:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.818+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 msgr2=0x7fbb340b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.818+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 shutdown_connections 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbb1006c8f0 0x7fbb1006edb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34160910 0x7fbb340b05b0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb341466b0 0x7fbb340b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 --2- 172.21.15.46:0/3034238578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb340720e0 0x7fbb34160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 >> 172.21.15.46:0/3034238578 conn(0x7fbb340b26d0 msgr2=0x7fbb340ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 shutdown_connections 2024-04-26T23:52:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:27.819+0000 7fbb33e00640 1 -- 172.21.15.46:0/3034238578 wait complete. 2024-04-26T23:52:27.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:16.864971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:18.039678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:52:27.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:52:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.047+0000 7ff88be00640 1 Processor -- start 2024-04-26T23:52:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.047+0000 7ff88be00640 1 -- start start 2024-04-26T23:52:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff88c15f210 0x7ff88c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c0af570 con 0x7ff88c0af000 2024-04-26T23:52:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c0af850 con 0x7ff88c0cd610 2024-04-26T23:52:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c0cb680 con 0x7ff88c15f210 2024-04-26T23:52:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff88c0af000 0x7ff88c071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54866/0 (socket says 172.21.15.46:54866) 2024-04-26T23:52:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.049+0000 7ff88b400640 1 -- 172.21.15.46:0/3745246881 learned_addr learned my addr 172.21.15.46:0/3745246881 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88aa00640 1 --2- 172.21.15.46:0/3745246881 >> v2:172.21.15.142:3300/0 conn(0x7ff88c15f210 0x7ff88c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636799267 0 0) 0x7ff88c0af570 con 0x7ff88c0af000 2024-04-26T23:52:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff870003520 con 0x7ff88c0af000 2024-04-26T23:52:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 756157941 0 0) 0x7ff870003520 con 0x7ff88c0af000 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 >> v2:172.21.15.142:3300/0 conn(0x7ff88c15f210 msgr2=0x7ff88c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 --2- 172.21.15.46:0/3745246881 >> v2:172.21.15.142:3300/0 conn(0x7ff88c15f210 0x7ff88c0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 >> v1:172.21.15.142:6789/0 conn(0x7ff88c0cd610 legacy=0x7ff88c0ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.050+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0cb960 con 0x7ff88c0af000 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7ff8840019f0 con 0x7ff88c0af000 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 >> v1:172.21.15.46:6789/0 conn(0x7ff88c0af000 legacy=0x7ff88c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700039b0 0x7ff870003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff870004800 0x7ff870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 0x7ff87000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0cb680 con 0x7ff8700051c0 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff87000c060 con 0x7ff8700039b0 2024-04-26T23:52:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff87000c260 con 0x7ff870004800 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88b400640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 0x7ff87000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88aa00640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700039b0 0x7ff870003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff883e00640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff870004800 0x7ff870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88b400640 1 -- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff870004800 msgr2=0x7ff870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.051+0000 7ff88b400640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff870004800 0x7ff870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.052+0000 7ff88b400640 1 -- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700039b0 msgr2=0x7ff870003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.052+0000 7ff88b400640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700039b0 0x7ff870003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.052+0000 7ff88b400640 1 -- 172.21.15.46:0/3745246881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff870003520 con 0x7ff8700051c0 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.052+0000 7ff88b400640 1 -- 172.21.15.46:0/3745246881 reap_dead start 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88b400640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 0x7ff87000b9c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7ff88400a840 tx=0x7ff870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c2c104511173da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88400a460 con 0x7ff8700051c0 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88402f070 con 0x7ff8700051c0 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88a000640 1 -- 172.21.15.46:0/3745246881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88402bcb0 con 0x7ff8700051c0 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88be00640 1 -- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 msgr2=0x7ff87000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.053+0000 7ff88be00640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 0x7ff87000b9c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7ff88400a840 tx=0x7ff870005830 comp rx=0 tx=0).stop 2024-04-26T23:52:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.054+0000 7ff88be00640 1 -- 172.21.15.46:0/3745246881 shutdown_connections 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.054+0000 7ff88be00640 1 --2- 172.21.15.46:0/3745246881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8700051c0 0x7ff87000b9c0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.054+0000 7ff88be00640 1 -- 172.21.15.46:0/3745246881 >> 172.21.15.46:0/3745246881 conn(0x7ff88c0b27b0 msgr2=0x7ff88c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.054+0000 7ff88be00640 1 -- 172.21.15.46:0/3745246881 shutdown_connections 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.054+0000 7ff88be00640 1 -- 172.21.15.46:0/3745246881 wait complete. 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.055+0000 7ff88be00640 1 Processor -- start 2024-04-26T23:52:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.056+0000 7ff88be00640 1 -- start start 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.056+0000 7ff88be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 0x7ff88c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b37c0 0x7ff88c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0cc160 con 0x7ff88c0c8a00 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c15f5f0 con 0x7ff8700051c0 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0cc2e0 con 0x7ff88c0b37c0 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45412/0 (socket says 172.21.15.46:45412) 2024-04-26T23:52:28.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 -- 172.21.15.46:0/2042541065 learned_addr learned my addr 172.21.15.46:0/2042541065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88aa00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 0x7ff88c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 -- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b37c0 msgr2=0x7ff88c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88b400640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b37c0 0x7ff88c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b37c0 0x7ff88c06f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 -- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 msgr2=0x7ff88c0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 0x7ff88c0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff883e00640 1 -- 172.21.15.46:0/2042541065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c162b00 con 0x7ff88c0c8a00 2024-04-26T23:52:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.057+0000 7ff88aa00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 0x7ff88c0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff883e00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e480 tx=0x7ff87400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8740052e0 con 0x7ff88c0c8a00 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff874018070 con 0x7ff88c0c8a00 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874003830 con 0x7ff88c0c8a00 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c0b4780 con 0x7ff88c0c8a00 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.058+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c0c8200 con 0x7ff88c0c8a00 2024-04-26T23:52:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.059+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c0693b0 con 0x7ff88c0c8a00 2024-04-26T23:52:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.060+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8740039d0 con 0x7ff88c0c8a00 2024-04-26T23:52:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.061+0000 7ff882a00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 0x7ff86806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.061+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7ff87408ba60 con 0x7ff88c0c8a00 2024-04-26T23:52:28.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.062+0000 7ff88aa00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 0x7ff86806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.063+0000 7ff88aa00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 0x7ff86806ede0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009a50 tx=0x7ff87c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.066+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff874059cc0 con 0x7ff88c0c8a00 2024-04-26T23:52:28.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.269+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff88c15c710 con 0x7ff88c0c8a00 2024-04-26T23:52:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.270+0000 7ff882a00640 1 -- 172.21.15.46:0/2042541065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) ==== 93+0+10 (secure 0 0 0) 0x7ff8740595c0 con 0x7ff88c0c8a00 2024-04-26T23:52:28.273 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 msgr2=0x7ff86806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 0x7ff86806ede0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009a50 tx=0x7ff87c008040 comp rx=0 tx=0).stop 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 msgr2=0x7ff88c1532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e480 tx=0x7ff87400e940 comp rx=0 tx=0).stop 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 shutdown_connections 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff86806c920 0x7ff86806ede0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b37c0 0x7ff88c06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff88c0c8a00 0x7ff88c1532e0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 --2- 172.21.15.46:0/2042541065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700051c0 0x7ff88c0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 >> 172.21.15.46:0/2042541065 conn(0x7ff88c0b27b0 msgr2=0x7ff88c0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 shutdown_connections 2024-04-26T23:52:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.273+0000 7ff88be00640 1 -- 172.21.15.46:0/2042541065 wait complete. 2024-04-26T23:52:28.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:52:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.505+0000 7f9641200640 1 Processor -- start 2024-04-26T23:52:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.505+0000 7f9641200640 1 -- start start 2024-04-26T23:52:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9641200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f963c0cdd00 0x7f963c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9641200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c073b30 0x7f963c0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9641200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f963c0ce7d0 con 0x7f963c15cec0 2024-04-26T23:52:28.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9641200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f963c0ceab0 con 0x7f963c0cdd00 2024-04-26T23:52:28.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9641200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f963c0ced90 con 0x7f963c073b30 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9632a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f963c0cdd00 0x7f963c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9632a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f963c0cdd00 0x7f963c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45414/0 (socket says 172.21.15.46:45414) 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.507+0000 7f9632a00640 1 -- 172.21.15.46:0/1017118081 learned_addr learned my addr 172.21.15.46:0/1017118081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.508+0000 7f963be00640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c073b30 0x7f963c0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.508+0000 7f9632a00640 1 -- 172.21.15.46:0/1017118081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c073b30 msgr2=0x7f963c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.508+0000 7f9632a00640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c073b30 0x7f963c0b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.508+0000 7f9632a00640 1 -- 172.21.15.46:0/1017118081 >> v1:172.21.15.142:6789/0 conn(0x7f963c15cec0 legacy=0x7f963c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.508+0000 7f9632a00640 1 -- 172.21.15.46:0/1017118081 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c0cf070 con 0x7f963c0cdd00 2024-04-26T23:52:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.509+0000 7f9632a00640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0cdd00 0x7f963c0735f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f9624009870 tx=0x7f962402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a9f432a73aaac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.509+0000 7f963aa00640 1 -- 172.21.15.46:0/1017118081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624037070 con 0x7f963c0cdd00 2024-04-26T23:52:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.509+0000 7f963aa00640 1 -- 172.21.15.46:0/1017118081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9624032440 con 0x7f963c0cdd00 2024-04-26T23:52:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.509+0000 7f963aa00640 1 -- 172.21.15.46:0/1017118081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624032810 con 0x7f963c0cdd00 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 -- 172.21.15.46:0/1017118081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0cdd00 msgr2=0x7f963c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0cdd00 0x7f963c0735f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f9624009870 tx=0x7f962402a250 comp rx=0 tx=0).stop 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 -- 172.21.15.46:0/1017118081 shutdown_connections 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c073b30 0x7f963c0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 --2- 172.21.15.46:0/1017118081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0cdd00 0x7f963c0735f0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 -- 172.21.15.46:0/1017118081 >> 172.21.15.46:0/1017118081 conn(0x7f963c0b27b0 msgr2=0x7f963c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 -- 172.21.15.46:0/1017118081 shutdown_connections 2024-04-26T23:52:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.510+0000 7f9641200640 1 -- 172.21.15.46:0/1017118081 wait complete. 2024-04-26T23:52:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.512+0000 7f9641200640 1 Processor -- start 2024-04-26T23:52:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.512+0000 7f9641200640 1 -- start start 2024-04-26T23:52:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 0x7f963c0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c15cec0 0x7f963c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f963c073f70 con 0x7f963c073b30 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f963c15d2a0 con 0x7f963c0cdd00 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.513+0000 7f9641200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f963c0b4080 con 0x7f963c15cec0 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45416/0 (socket says 172.21.15.46:45416) 2024-04-26T23:52:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 -- 172.21.15.46:0/1655209479 learned_addr learned my addr 172.21.15.46:0/1655209479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963be00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c15cec0 0x7f963c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f9632a00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 0x7f963c0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 -- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c15cec0 msgr2=0x7f963c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c15cec0 0x7f963c15ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 -- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 msgr2=0x7f963c0c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 0x7f963c0c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f963b400640 1 -- 172.21.15.46:0/1655209479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c0c6c40 con 0x7f963c073b30 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.514+0000 7f9632a00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 0x7f963c0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f963b400640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f962c00c6f0 tx=0x7f962c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c003990 con 0x7f963c073b30 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962c003b50 con 0x7f963c073b30 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00a850 con 0x7f963c073b30 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c0c9e30 con 0x7f963c073b30 2024-04-26T23:52:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.515+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c0ca3f0 con 0x7f963c073b30 2024-04-26T23:52:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.516+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c0693b0 con 0x7f963c073b30 2024-04-26T23:52:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.517+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f962c00a9f0 con 0x7f963c073b30 2024-04-26T23:52:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.518+0000 7f9633e00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.518+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f962c08bbb0 con 0x7f963c073b30 2024-04-26T23:52:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.519+0000 7f9632a00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.520+0000 7f9632a00640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 0x7f961806edb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9624034000 tx=0x7f962402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.523+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f962c059e10 con 0x7f963c073b30 2024-04-26T23:52:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.728+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f963c0cd800 con 0x7f963c073b30 2024-04-26T23:52:28.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.729+0000 7f9633e00640 1 -- 172.21.15.46:0/1655209479 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) ==== 104+0+11 (secure 0 0 0) 0x7f962c059710 con 0x7f963c073b30 2024-04-26T23:52:28.732 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.731+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 msgr2=0x7f961806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 0x7f961806edb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9624034000 tx=0x7f962402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 msgr2=0x7f963c0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f962c00c6f0 tx=0x7f962c00cbb0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 shutdown_connections 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f963c15cec0 0x7f963c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c0cdd00 0x7f963c0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 --2- 172.21.15.46:0/1655209479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c073b30 0x7f963c0c9730 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 >> 172.21.15.46:0/1655209479 conn(0x7f963c0b27b0 msgr2=0x7f963c072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 shutdown_connections 2024-04-26T23:52:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.732+0000 7f9641200640 1 -- 172.21.15.46:0/1655209479 wait complete. 2024-04-26T23:52:28.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.956+0000 7f8b7fe00640 1 Processor -- start 2024-04-26T23:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.956+0000 7f8b7fe00640 1 -- start start 2024-04-26T23:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b800ce8c0 0x7f8b800d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b800cd8a0 0x7f8b80160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b80161350 0x7f8b800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8b800d21a0 con 0x7f8b80161350 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8b80161c20 con 0x7f8b800cd8a0 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b800d1980 con 0x7f8b800ce8c0 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b80161350 0x7f8b800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b80161350 0x7f8b800706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45418/0 (socket says 172.21.15.46:45418) 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7f400640 1 -- 172.21.15.46:0/3759593967 learned_addr learned my addr 172.21.15.46:0/3759593967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b7ea00640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b800ce8c0 0x7f8b800d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.958+0000 7f8b77e00640 1 --2- 172.21.15.46:0/3759593967 >> v2:172.21.15.142:3300/0 conn(0x7f8b800cd8a0 0x7f8b80160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.959+0000 7f8b7f400640 1 -- 172.21.15.46:0/3759593967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b800ce8c0 msgr2=0x7f8b800d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.959+0000 7f8b7f400640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b800ce8c0 0x7f8b800d1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.959+0000 7f8b7f400640 1 -- 172.21.15.46:0/3759593967 >> v2:172.21.15.142:3300/0 conn(0x7f8b800cd8a0 msgr2=0x7f8b80160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.959+0000 7f8b7f400640 1 --2- 172.21.15.46:0/3759593967 >> v2:172.21.15.142:3300/0 conn(0x7f8b800cd8a0 0x7f8b80160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.959+0000 7f8b7f400640 1 -- 172.21.15.46:0/3759593967 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b800701b0 con 0x7f8b80161350 2024-04-26T23:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.960+0000 7f8b7f400640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b80161350 0x7f8b800706d0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8b80073100 tx=0x7f8b7802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de269807f71ae23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.960+0000 7f8b7e000640 1 -- 172.21.15.46:0/3759593967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78036070 con 0x7f8b80161350 2024-04-26T23:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.960+0000 7f8b7e000640 1 -- 172.21.15.46:0/3759593967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78031600 con 0x7f8b80161350 2024-04-26T23:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.960+0000 7f8b7e000640 1 -- 172.21.15.46:0/3759593967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780319d0 con 0x7f8b80161350 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 -- 172.21.15.46:0/3759593967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b80161350 msgr2=0x7f8b800706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b80161350 0x7f8b800706d0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8b80073100 tx=0x7f8b7802a610 comp rx=0 tx=0).stop 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 -- 172.21.15.46:0/3759593967 shutdown_connections 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b80161350 0x7f8b800706d0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/3759593967 >> v2:172.21.15.142:3300/0 conn(0x7f8b800cd8a0 0x7f8b80160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/3759593967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b800ce8c0 0x7f8b800d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 -- 172.21.15.46:0/3759593967 >> 172.21.15.46:0/3759593967 conn(0x7f8b800d2d90 msgr2=0x7f8b800d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 -- 172.21.15.46:0/3759593967 shutdown_connections 2024-04-26T23:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.961+0000 7f8b7fe00640 1 -- 172.21.15.46:0/3759593967 wait complete. 2024-04-26T23:52:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.963+0000 7f8b7fe00640 1 Processor -- start 2024-04-26T23:52:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.964+0000 7f8b7fe00640 1 -- start start 2024-04-26T23:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b800cd8a0 0x7f8b800b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b80161350 0x7f8b800b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800cee20 con 0x7f8b800ce8c0 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800ceca0 con 0x7f8b800cd8a0 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b7fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b800cdd20 con 0x7f8b80161350 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45434/0 (socket says 172.21.15.46:45434) 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.965+0000 7f8b77e00640 1 -- 172.21.15.46:0/957519101 learned_addr learned my addr 172.21.15.46:0/957519101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b7ea00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b800cd8a0 0x7f8b800b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b7f400640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b80161350 0x7f8b800b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 -- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b80161350 msgr2=0x7f8b800b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b80161350 0x7f8b800b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 -- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b800cd8a0 msgr2=0x7f8b800b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b800cd8a0 0x7f8b800b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 -- 172.21.15.46:0/957519101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b800cc370 con 0x7f8b800ce8c0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b77e00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d800 tx=0x7f8b6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.966+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68018030 con 0x7f8b800ce8c0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.967+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b680096a0 con 0x7f8b800ce8c0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.967+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68009870 con 0x7f8b800ce8c0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.967+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b800cc650 con 0x7f8b800ce8c0 2024-04-26T23:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.967+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b800ccc10 con 0x7f8b800ce8c0 2024-04-26T23:52:28.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.968+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b80069ac0 con 0x7f8b800ce8c0 2024-04-26T23:52:28.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.969+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b6801b070 con 0x7f8b800ce8c0 2024-04-26T23:52:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.970+0000 7f8b76a00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.970+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f8b6808bf10 con 0x7f8b800ce8c0 2024-04-26T23:52:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.971+0000 7f8b7ea00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.972+0000 7f8b7ea00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:28.975+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b6805a210 con 0x7f8b800ce8c0 2024-04-26T23:52:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.175+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b800b8d10 con 0x7f8b800ce8c0 2024-04-26T23:52:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.176+0000 7f8b76a00640 1 -- 172.21.15.46:0/957519101 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) ==== 100+0+11 (secure 0 0 0) 0x7f8b68059b10 con 0x7f8b800ce8c0 2024-04-26T23:52:29.179 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:52:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 msgr2=0x7f8b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70008040 comp rx=0 tx=0).stop 2024-04-26T23:52:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 msgr2=0x7f8b800b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d800 tx=0x7f8b6800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 shutdown_connections 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b80161350 0x7f8b800b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b800ce8c0 0x7f8b800b0bb0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 --2- 172.21.15.46:0/957519101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b800cd8a0 0x7f8b800b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.178+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 >> 172.21.15.46:0/957519101 conn(0x7f8b800d2d90 msgr2=0x7f8b8006d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.179+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 shutdown_connections 2024-04-26T23:52:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.179+0000 7f8b7fe00640 1 -- 172.21.15.46:0/957519101 wait complete. 2024-04-26T23:52:29.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.393+0000 7f2079800640 1 Processor -- start 2024-04-26T23:52:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.394+0000 7f2079800640 1 -- start start 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f2079800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f20740d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f2079800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20740d11f0 con 0x7f20740d2130 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f2079800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20740d5f20 con 0x7f207415f4e0 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f2079800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20740d0ce0 con 0x7f20740b4000 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f206b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f20740d2130 0x7f20740b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54878/0 (socket says 172.21.15.46:54878) 2024-04-26T23:52:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.395+0000 7f206b400640 1 -- 172.21.15.46:0/1396485218 learned_addr learned my addr 172.21.15.46:0/1396485218 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506011256 0 0) 0x7f20740d11f0 con 0x7f20740d2130 2024-04-26T23:52:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2078e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f20740d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2058003520 con 0x7f20740d2130 2024-04-26T23:52:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 768906201 0 0) 0x7f2058003520 con 0x7f20740d2130 2024-04-26T23:52:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 msgr2=0x7f20740d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f20740d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 >> v1:172.21.15.142:6789/0 conn(0x7f207415f4e0 legacy=0x7f20740d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20740d0f10 con 0x7f20740d2130 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f205c001670 con 0x7f20740d2130 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.396+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 >> v1:172.21.15.46:6789/0 conn(0x7f20740d2130 legacy=0x7f20740b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f2058003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f2058004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2058004d40 0x7f20580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f205800bb50 con 0x7f20580039e0 2024-04-26T23:52:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f205800bcd0 con 0x7f2058004d40 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f205800bed0 con 0x7f20740b4000 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2073e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f2058003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f206b400640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f2058004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.397+0000 7f2078e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2058004d40 0x7f20580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.398+0000 7f2073e00640 1 -- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 msgr2=0x7f2058004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.398+0000 7f2073e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20740b4000 0x7f2058004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.398+0000 7f2073e00640 1 -- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2058004d40 msgr2=0x7f20580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.398+0000 7f2073e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2058004d40 0x7f20580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.398+0000 7f2073e00640 1 -- 172.21.15.46:0/1396485218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2058003520 con 0x7f20580039e0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2073e00640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f2058003e00 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2064001e50 tx=0x7f2058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c718c73b3b308e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064001a70 con 0x7f20580039e0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064003970 con 0x7f20580039e0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2073400640 1 -- 172.21.15.46:0/1396485218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064003d60 con 0x7f20580039e0 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2079800640 1 -- 172.21.15.46:0/1396485218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 msgr2=0x7f2058003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2079800640 1 --2- 172.21.15.46:0/1396485218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f2058003e00 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2064001e50 tx=0x7f2058005830 comp rx=0 tx=0).stop 2024-04-26T23:52:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.399+0000 7f2078e00640 1 -- 172.21.15.46:0/1396485218 reap_dead start 2024-04-26T23:52:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.400+0000 7f2079800640 1 -- 172.21.15.46:0/1396485218 shutdown_connections 2024-04-26T23:52:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.400+0000 7f2079800640 1 -- 172.21.15.46:0/1396485218 >> 172.21.15.46:0/1396485218 conn(0x7f20740d5010 msgr2=0x7f20740d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.400+0000 7f2079800640 1 -- 172.21.15.46:0/1396485218 shutdown_connections 2024-04-26T23:52:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.400+0000 7f2079800640 1 -- 172.21.15.46:0/1396485218 wait complete. 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.401+0000 7f2079800640 1 Processor -- start 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.401+0000 7f2079800640 1 -- start start 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2074158d90 0x7f2074157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20741f54d0 0x7f20740b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20740b1d60 con 0x7f20580039e0 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20740d2990 con 0x7f2074158d90 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.402+0000 7f2079800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f207415f8c0 con 0x7f20741f54d0 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45446/0 (socket says 172.21.15.46:45446) 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 -- 172.21.15.46:0/2624331462 learned_addr learned my addr 172.21.15.46:0/2624331462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 -- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20741f54d0 msgr2=0x7f20740b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20741f54d0 0x7f20740b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 -- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2074158d90 msgr2=0x7f2074157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2074158d90 0x7f2074157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 -- 172.21.15.46:0/2624331462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2074158130 con 0x7f20580039e0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.403+0000 7f2073e00640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2064001f70 tx=0x7f20640014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.404+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064037040 con 0x7f20580039e0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.404+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064041070 con 0x7f20580039e0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.404+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2074159530 con 0x7f20580039e0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.404+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064007db0 con 0x7f20580039e0 2024-04-26T23:52:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.404+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2074159af0 con 0x7f20580039e0 2024-04-26T23:52:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.405+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20740693b0 con 0x7f20580039e0 2024-04-26T23:52:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.406+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2064005500 con 0x7f20580039e0 2024-04-26T23:52:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.407+0000 7f2070c00640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 0x7f205006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.407+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f20640ac0e0 con 0x7f20580039e0 2024-04-26T23:52:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.407+0000 7f206b400640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 0x7f205006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.409+0000 7f206b400640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 0x7f205006edb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f205c001050 tx=0x7f205c001150 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.411+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20640057f0 con 0x7f20580039e0 2024-04-26T23:52:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.654+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20740d33d0 con 0x7f20580039e0 2024-04-26T23:52:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.655+0000 7f2070c00640 1 -- 172.21.15.46:0/2624331462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f206407a230 con 0x7f20580039e0 2024-04-26T23:52:29.659 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:29.659 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 msgr2=0x7f205006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 0x7f205006edb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f205c001050 tx=0x7f205c001150 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 msgr2=0x7f20740cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2064001f70 tx=0x7f20640014a0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 shutdown_connections 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f205006c8f0 0x7f205006edb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20741f54d0 0x7f20740b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2074158d90 0x7f2074157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 --2- 172.21.15.46:0/2624331462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20580039e0 0x7f20740cdd00 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 >> 172.21.15.46:0/2624331462 conn(0x7f20740d5010 msgr2=0x7f20740b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 shutdown_connections 2024-04-26T23:52:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.658+0000 7f2079800640 1 -- 172.21.15.46:0/2624331462 wait complete. 2024-04-26T23:52:29.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.876+0000 7f8d05e00640 1 Processor -- start 2024-04-26T23:52:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.876+0000 7f8d05e00640 1 -- start start 2024-04-26T23:52:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.878+0000 7f8d05e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8d000720e0 0x7f8d000cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.878+0000 7f8d05e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d00073ad0 con 0x7f8d00160910 2024-04-26T23:52:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.878+0000 7f8d05e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d00146a90 con 0x7f8d001466b0 2024-04-26T23:52:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.878+0000 7f8d05e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8d00073d70 con 0x7f8d000720e0 2024-04-26T23:52:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.879+0000 7f8d05400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d00160910 0x7f8d000cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54880/0 (socket says 172.21.15.46:54880) 2024-04-26T23:52:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.879+0000 7f8d05400640 1 -- 172.21.15.46:0/2016599152 learned_addr learned my addr 172.21.15.46:0/2016599152 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.879+0000 7f8cffe00640 1 --2- 172.21.15.46:0/2016599152 >> v2:172.21.15.142:3300/0 conn(0x7f8d000720e0 0x7f8d000cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.879+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229330574 0 0) 0x7f8d00073ad0 con 0x7f8d00160910 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.879+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ce8003520 con 0x7f8d00160910 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112546774 0 0) 0x7f8d00146a90 con 0x7f8d001466b0 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d00073ad0 con 0x7f8d001466b0 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1338184877 0 0) 0x7f8ce8003520 con 0x7f8d00160910 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 >> v2:172.21.15.142:3300/0 conn(0x7f8d000720e0 msgr2=0x7f8d000cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 --2- 172.21.15.46:0/2016599152 >> v2:172.21.15.142:3300/0 conn(0x7f8d000720e0 0x7f8d000cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 >> v1:172.21.15.142:6789/0 conn(0x7f8d001466b0 legacy=0x7f8d0006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d00074050 con 0x7f8d00160910 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8cf80019f0 con 0x7f8d00160910 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 >> v1:172.21.15.46:6789/0 conn(0x7f8d00160910 legacy=0x7f8d000cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ce8003bc0 0x7f8ce8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ce80051c0 0x7f8ce800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00073d70 con 0x7f8ce8004800 2024-04-26T23:52:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce800c0c0 con 0x7f8ce80051c0 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce800c2c0 con 0x7f8ce8003bc0 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8cffe00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.880+0000 7f8d04a00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ce8003bc0 0x7f8ce8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8cffe00640 1 -- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ce8003bc0 msgr2=0x7f8ce8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8d05400640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ce80051c0 0x7f8ce800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8cffe00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ce8003bc0 0x7f8ce8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8cffe00640 1 -- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ce80051c0 msgr2=0x7f8ce800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8cffe00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ce80051c0 0x7f8ce800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8cffe00640 1 -- 172.21.15.46:0/2016599152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce8003520 con 0x7f8ce8004800 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8d05400640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ce80051c0 0x7f8ce800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.881+0000 7f8d05400640 1 -- 172.21.15.46:0/2016599152 reap_dead start 2024-04-26T23:52:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8cffe00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8ce8004c80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009ac0 tx=0x7f8ce8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd8b85272f8e428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec036070 con 0x7f8ce8004800 2024-04-26T23:52:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec031400 con 0x7f8ce8004800 2024-04-26T23:52:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8cff400640 1 -- 172.21.15.46:0/2016599152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec009690 con 0x7f8ce8004800 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8d05e00640 1 -- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 msgr2=0x7f8ce8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.882+0000 7f8d05e00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8ce8004c80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009ac0 tx=0x7f8ce8005830 comp rx=0 tx=0).stop 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.883+0000 7f8d05e00640 1 -- 172.21.15.46:0/2016599152 shutdown_connections 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.883+0000 7f8d05e00640 1 --2- 172.21.15.46:0/2016599152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8ce8004c80 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.883+0000 7f8d05e00640 1 -- 172.21.15.46:0/2016599152 >> 172.21.15.46:0/2016599152 conn(0x7f8d000b26d0 msgr2=0x7f8d000b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.883+0000 7f8d05e00640 1 -- 172.21.15.46:0/2016599152 shutdown_connections 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.883+0000 7f8d05e00640 1 -- 172.21.15.46:0/2016599152 wait complete. 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.885+0000 7f8d05e00640 1 Processor -- start 2024-04-26T23:52:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.885+0000 7f8d05e00640 1 -- start start 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 0x7f8d000aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d001600d0 0x7f8d00137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00160d60 con 0x7f8ce8004800 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000cb470 con 0x7f8d0015fc50 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d05e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8d00160ee0 con 0x7f8d001600d0 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d04a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d04a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45474/0 (socket says 172.21.15.46:45474) 2024-04-26T23:52:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.886+0000 7f8d04a00640 1 -- 172.21.15.46:0/348582983 learned_addr learned my addr 172.21.15.46:0/348582983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8cffe00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 0x7f8d000aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 -- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d001600d0 msgr2=0x7f8d00137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d05400640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d001600d0 0x7f8d00137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d001600d0 0x7f8d00137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 -- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 msgr2=0x7f8d000aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 0x7f8d000aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 -- 172.21.15.46:0/348582983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d000aeec0 con 0x7f8ce8004800 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8cffe00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 0x7f8d000aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d04a00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e7b0 tx=0x7f8cf400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400ee60 con 0x7f8ce8004800 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf4018070 con 0x7f8ce8004800 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400c6c0 con 0x7f8ce8004800 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d000af1a0 con 0x7f8ce8004800 2024-04-26T23:52:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.887+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d000af760 con 0x7f8ce8004800 2024-04-26T23:52:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.889+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d000693b0 con 0x7f8ce8004800 2024-04-26T23:52:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.890+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cf400c860 con 0x7f8ce8004800 2024-04-26T23:52:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.890+0000 7f8cfcc00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 0x7f8ce006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.891+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f8cf408bbc0 con 0x7f8ce8004800 2024-04-26T23:52:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.891+0000 7f8cffe00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 0x7f8ce006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.892+0000 7f8cffe00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 0x7f8ce006ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8cec033000 tx=0x7f8cec008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:29.895+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cf401a310 con 0x7f8ce8004800 2024-04-26T23:52:30.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.103+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d000c7180 con 0x7f8ce8004800 2024-04-26T23:52:30.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.105+0000 7f8cfcc00640 1 -- 172.21.15.46:0/348582983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16953 (secure 0 0 0) 0x7f8cf4059e30 con 0x7f8ce8004800 2024-04-26T23:52:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":150,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:57.743412+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 msgr2=0x7f8ce006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 0x7f8ce006ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8cec033000 tx=0x7f8cec008030 comp rx=0 tx=0).stop 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 msgr2=0x7f8d000ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e7b0 tx=0x7f8cf400ec70 comp rx=0 tx=0).stop 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 shutdown_connections 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ce006c920 0x7f8ce006ede0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d001600d0 0x7f8d00137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0015fc50 0x7f8d000aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 --2- 172.21.15.46:0/348582983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce8004800 0x7f8d000ae8e0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 >> 172.21.15.46:0/348582983 conn(0x7f8d000b26d0 msgr2=0x7f8d000c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 shutdown_connections 2024-04-26T23:52:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.108+0000 7f8d05e00640 1 -- 172.21.15.46:0/348582983 wait complete. 2024-04-26T23:52:30.131 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:52:30.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.334+0000 7f6eaf200640 1 Processor -- start 2024-04-26T23:52:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.334+0000 7f6eaf200640 1 -- start start 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6eb01466b0 0x7f6eb006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 0x7f6eb00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6eb0160910 0x7f6eb0160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb0073ad0 con 0x7f6eb0160910 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb0146a90 con 0x7f6eb01466b0 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eaf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb0073d70 con 0x7f6eb00720e0 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eae800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6eb0160910 0x7f6eb0160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eae800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6eb0160910 0x7f6eb0160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45490/0 (socket says 172.21.15.46:45490) 2024-04-26T23:52:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.336+0000 7f6eae800640 1 -- 172.21.15.46:0/1174732193 learned_addr learned my addr 172.21.15.46:0/1174732193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eade00640 1 --2- 172.21.15.46:0/1174732193 >> v2:172.21.15.142:3300/0 conn(0x7f6eb01466b0 0x7f6eb006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6ead400640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 0x7f6eb00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eae800640 1 -- 172.21.15.46:0/1174732193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 msgr2=0x7f6eb00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eae800640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 0x7f6eb00cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eae800640 1 -- 172.21.15.46:0/1174732193 >> v2:172.21.15.142:3300/0 conn(0x7f6eb01466b0 msgr2=0x7f6eb006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eae800640 1 --2- 172.21.15.46:0/1174732193 >> v2:172.21.15.142:3300/0 conn(0x7f6eb01466b0 0x7f6eb006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6eae800640 1 -- 172.21.15.46:0/1174732193 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb0074050 con 0x7f6eb0160910 2024-04-26T23:52:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.337+0000 7f6ead400640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 0x7f6eb00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.338+0000 7f6eae800640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb0160910 0x7f6eb0160da0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b3440 tx=0x7f6ea802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf6a0e633a1ec7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.338+0000 7f6eaca00640 1 -- 172.21.15.46:0/1174732193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8036070 con 0x7f6eb0160910 2024-04-26T23:52:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.338+0000 7f6eaca00640 1 -- 172.21.15.46:0/1174732193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea8031600 con 0x7f6eb0160910 2024-04-26T23:52:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.338+0000 7f6eaca00640 1 -- 172.21.15.46:0/1174732193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80319d0 con 0x7f6eb0160910 2024-04-26T23:52:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 -- 172.21.15.46:0/1174732193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb0160910 msgr2=0x7f6eb0160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb0160910 0x7f6eb0160da0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b3440 tx=0x7f6ea802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 -- 172.21.15.46:0/1174732193 shutdown_connections 2024-04-26T23:52:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb0160910 0x7f6eb0160da0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 --2- 172.21.15.46:0/1174732193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb00720e0 0x7f6eb00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 --2- 172.21.15.46:0/1174732193 >> v2:172.21.15.142:3300/0 conn(0x7f6eb01466b0 0x7f6eb006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 -- 172.21.15.46:0/1174732193 >> 172.21.15.46:0/1174732193 conn(0x7f6eb00b26d0 msgr2=0x7f6eb00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 -- 172.21.15.46:0/1174732193 shutdown_connections 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.339+0000 7f6eaf200640 1 -- 172.21.15.46:0/1174732193 wait complete. 2024-04-26T23:52:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.341+0000 7f6eaf200640 1 Processor -- start 2024-04-26T23:52:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.341+0000 7f6eaf200640 1 -- start start 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb01466b0 0x7f6eb01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 0x7f6eb00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb006e3c0 con 0x7f6eb00720e0 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00cb470 con 0x7f6eb0160910 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eaf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00cdda0 con 0x7f6eb01466b0 2024-04-26T23:52:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.342+0000 7f6eade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45504/0 (socket says 172.21.15.46:45504) 2024-04-26T23:52:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 -- 172.21.15.46:0/3449220488 learned_addr learned my addr 172.21.15.46:0/3449220488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6ead400640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb01466b0 0x7f6eb01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eae800640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 0x7f6eb00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 -- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb01466b0 msgr2=0x7f6eb01436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb01466b0 0x7f6eb01436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 -- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 msgr2=0x7f6eb00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 0x7f6eb00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 -- 172.21.15.46:0/3449220488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb01426b0 con 0x7f6eb00720e0 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eae800640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 0x7f6eb00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.343+0000 7f6eade00640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c7f0 tx=0x7f6ea000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.344+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0015070 con 0x7f6eb00720e0 2024-04-26T23:52:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.344+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0004220 con 0x7f6eb00720e0 2024-04-26T23:52:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.344+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb00caa30 con 0x7f6eb00720e0 2024-04-26T23:52:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.344+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0007620 con 0x7f6eb00720e0 2024-04-26T23:52:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.344+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb0142c00 con 0x7f6eb00720e0 2024-04-26T23:52:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.345+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb00693b0 con 0x7f6eb00720e0 2024-04-26T23:52:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.346+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea00079d0 con 0x7f6eb00720e0 2024-04-26T23:52:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.348+0000 7f6ea6000640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 0x7f6e8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.348+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f6ea008bcc0 con 0x7f6eb00720e0 2024-04-26T23:52:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.348+0000 7f6ead400640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 0x7f6e8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.349+0000 7f6ead400640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 0x7f6e8c06ede0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6eb01f80f0 tx=0x7f6e98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.352+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ea0092080 con 0x7f6eb00720e0 2024-04-26T23:52:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.605+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eb00c7180 con 0x7f6eb00720e0 2024-04-26T23:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.606+0000 7f6ea6000640 1 -- 172.21.15.46:0/3449220488 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f6ea0059e90 con 0x7f6eb00720e0 2024-04-26T23:52:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:30.609 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 msgr2=0x7f6e8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 0x7f6e8c06ede0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6eb01f80f0 tx=0x7f6e98008040 comp rx=0 tx=0).stop 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 msgr2=0x7f6eb0141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c7f0 tx=0x7f6ea000ccb0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 shutdown_connections 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6e8c06c920 0x7f6e8c06ede0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb0160910 0x7f6eb00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6eb01466b0 0x7f6eb01436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 --2- 172.21.15.46:0/3449220488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb00720e0 0x7f6eb0141b40 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 >> 172.21.15.46:0/3449220488 conn(0x7f6eb00b26d0 msgr2=0x7f6eb00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.609+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 shutdown_connections 2024-04-26T23:52:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.610+0000 7f6eaf200640 1 -- 172.21.15.46:0/3449220488 wait complete. 2024-04-26T23:52:30.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:30.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.838+0000 7f31ed400640 1 Processor -- start 2024-04-26T23:52:30.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.838+0000 7f31ed400640 1 -- start start 2024-04-26T23:52:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 0x7f31e80d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31e8072e60 0x7f31e80732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f31e80d0c50 con 0x7f31e8072e60 2024-04-26T23:52:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f31e815cf90 con 0x7f31e8145770 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31ed400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f31e815d490 con 0x7f31e815c5c0 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31e8072e60 0x7f31e80732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31e8072e60 0x7f31e80732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45516/0 (socket says 172.21.15.46:45516) 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31dfe00640 1 -- 172.21.15.46:0/1463610781 learned_addr learned my addr 172.21.15.46:0/1463610781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31e6800640 1 --2- 172.21.15.46:0/1463610781 >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 0x7f31e80d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.840+0000 7f31e7200640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31dfe00640 1 -- 172.21.15.46:0/1463610781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 msgr2=0x7f31e815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31dfe00640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e815ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31dfe00640 1 -- 172.21.15.46:0/1463610781 >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 msgr2=0x7f31e80d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31dfe00640 1 --2- 172.21.15.46:0/1463610781 >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 0x7f31e80d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31dfe00640 1 -- 172.21.15.46:0/1463610781 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e80cf950 con 0x7f31e8072e60 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31e6800640 1 --2- 172.21.15.46:0/1463610781 >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 0x7f31e80d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.841+0000 7f31e7200640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.842+0000 7f31dfe00640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e80732c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f31e806d2c0 tx=0x7f31d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbb7f6a366714db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.842+0000 7f31e5e00640 1 -- 172.21.15.46:0/1463610781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037070 con 0x7f31e8072e60 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.842+0000 7f31e5e00640 1 -- 172.21.15.46:0/1463610781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d0032440 con 0x7f31e8072e60 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.842+0000 7f31e5e00640 1 -- 172.21.15.46:0/1463610781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0032810 con 0x7f31e8072e60 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 -- 172.21.15.46:0/1463610781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 msgr2=0x7f31e80732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e80732c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f31e806d2c0 tx=0x7f31d002a210 comp rx=0 tx=0).stop 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 -- 172.21.15.46:0/1463610781 shutdown_connections 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 --2- 172.21.15.46:0/1463610781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e80732c0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 --2- 172.21.15.46:0/1463610781 >> v2:172.21.15.142:3300/0 conn(0x7f31e8145770 0x7f31e80d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 -- 172.21.15.46:0/1463610781 >> 172.21.15.46:0/1463610781 conn(0x7f31e806c530 msgr2=0x7f31e806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 -- 172.21.15.46:0/1463610781 shutdown_connections 2024-04-26T23:52:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.843+0000 7f31ed400640 1 -- 172.21.15.46:0/1463610781 wait complete. 2024-04-26T23:52:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.845+0000 7f31ed400640 1 Processor -- start 2024-04-26T23:52:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.845+0000 7f31ed400640 1 -- start start 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31e8145770 0x7f31e8156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e81683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31e80cff10 con 0x7f31e8072e60 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8073800 con 0x7f31e8145770 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31ed400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f31e80d01f0 con 0x7f31e815c5c0 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45518/0 (socket says 172.21.15.46:45518) 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 -- 172.21.15.46:0/883795571 learned_addr learned my addr 172.21.15.46:0/883795571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 -- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 msgr2=0x7f31e81683e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e81683e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.846+0000 7f31e6800640 1 -- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31e8145770 msgr2=0x7f31e8156f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31e6800640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31e8145770 0x7f31e8156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:30.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31e6800640 1 -- 172.21.15.46:0/883795571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e815acf0 con 0x7f31e8072e60 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31e6800640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c7c0 tx=0x7f31d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8015070 con 0x7f31e8072e60 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d8007a30 con 0x7f31e8072e60 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007e00 con 0x7f31e8072e60 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e815afd0 con 0x7f31e8072e60 2024-04-26T23:52:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.847+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e815b590 con 0x7f31e8072e60 2024-04-26T23:52:30.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.849+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e8069530 con 0x7f31e8072e60 2024-04-26T23:52:30.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.850+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31d8018070 con 0x7f31e8072e60 2024-04-26T23:52:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.851+0000 7f31dea00640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.851+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f31d808bc80 con 0x7f31e8072e60 2024-04-26T23:52:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.852+0000 7f31dfe00640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.853+0000 7f31dfe00640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f31d0034000 tx=0x7f31d002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:30.856+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31d8059ee0 con 0x7f31e8072e60 2024-04-26T23:52:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.102+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e80b1460 con 0x7f31e8072e60 2024-04-26T23:52:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.103+0000 7f31dea00640 1 -- 172.21.15.46:0/883795571 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5197 (secure 0 0 0) 0x7f31d80597e0 con 0x7f31e8072e60 2024-04-26T23:52:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:31.107 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:19.047682+0000","modified":"2024-04-26T23:51:22.134191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9663},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9663":{"gid":9663,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3522383810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3522383810},{"type":"v1","addr":"172.21.15.46:6835","nonce":3522383810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9663,"qdb_cluster":[9663]},"id":10}]} 2024-04-26T23:52:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 118 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 msgr2=0x7f31c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f31d0034000 tx=0x7f31d002c000 comp rx=0 tx=0).stop 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 msgr2=0x7f31e815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c7c0 tx=0x7f31d800cc80 comp rx=0 tx=0).stop 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 shutdown_connections 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31e815c5c0 0x7f31e81683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31e8145770 0x7f31e8156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 --2- 172.21.15.46:0/883795571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31e8072e60 0x7f31e815a680 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 >> 172.21.15.46:0/883795571 conn(0x7f31e806c530 msgr2=0x7f31e80b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 shutdown_connections 2024-04-26T23:52:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.106+0000 7f31ed400640 1 -- 172.21.15.46:0/883795571 wait complete. 2024-04-26T23:52:31.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.341+0000 7f240b400640 1 Processor -- start 2024-04-26T23:52:31.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.341+0000 7f240b400640 1 -- start start 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f240b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f240406e780 0x7f24040b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f240b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f240414d200 0x7f2404072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f240b400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24040b4940 con 0x7f24040b43c0 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f240b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f24040b4c20 con 0x7f240406e780 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f240b400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f24040b4f00 con 0x7f240414d200 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f2408a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f240406e780 0x7f24040b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f2408a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f240406e780 0x7f24040b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45526/0 (socket says 172.21.15.46:45526) 2024-04-26T23:52:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.343+0000 7f2408a00640 1 -- 172.21.15.46:0/1597810450 learned_addr learned my addr 172.21.15.46:0/1597810450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.344+0000 7f2408a00640 1 -- 172.21.15.46:0/1597810450 >> v2:172.21.15.142:3300/0 conn(0x7f240414d200 msgr2=0x7f2404072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.344+0000 7f2408a00640 1 --2- 172.21.15.46:0/1597810450 >> v2:172.21.15.142:3300/0 conn(0x7f240414d200 0x7f2404072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.344+0000 7f2408a00640 1 -- 172.21.15.46:0/1597810450 >> v1:172.21.15.46:6789/0 conn(0x7f24040b43c0 legacy=0x7f240414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.344+0000 7f2408a00640 1 -- 172.21.15.46:0/1597810450 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24040b51e0 con 0x7f240406e780 2024-04-26T23:52:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.345+0000 7f2408a00640 1 --2- 172.21.15.46:0/1597810450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406e780 0x7f24040b3e80 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f2404161730 tx=0x7f23f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a216b6307a5f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.345+0000 7f2403400640 1 -- 172.21.15.46:0/1597810450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8037070 con 0x7f240406e780 2024-04-26T23:52:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.345+0000 7f2403400640 1 -- 172.21.15.46:0/1597810450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f8032440 con 0x7f240406e780 2024-04-26T23:52:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.345+0000 7f2403400640 1 -- 172.21.15.46:0/1597810450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8032810 con 0x7f240406e780 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 -- 172.21.15.46:0/1597810450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406e780 msgr2=0x7f24040b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 --2- 172.21.15.46:0/1597810450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406e780 0x7f24040b3e80 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f2404161730 tx=0x7f23f802a270 comp rx=0 tx=0).stop 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 -- 172.21.15.46:0/1597810450 shutdown_connections 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 --2- 172.21.15.46:0/1597810450 >> v2:172.21.15.142:3300/0 conn(0x7f240414d200 0x7f2404072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 --2- 172.21.15.46:0/1597810450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406e780 0x7f24040b3e80 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 -- 172.21.15.46:0/1597810450 >> 172.21.15.46:0/1597810450 conn(0x7f240406c530 msgr2=0x7f240406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 -- 172.21.15.46:0/1597810450 shutdown_connections 2024-04-26T23:52:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.346+0000 7f240b400640 1 -- 172.21.15.46:0/1597810450 wait complete. 2024-04-26T23:52:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.347+0000 7f240b400640 1 Processor -- start 2024-04-26T23:52:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.348+0000 7f240b400640 1 -- start start 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f240406e780 0x7f240414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f240414d200 0x7f2404167330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f240406ece0 con 0x7f24040b43c0 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f240406eb60 con 0x7f240414d200 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f240b400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f240406ee60 con 0x7f240406e780 2024-04-26T23:52:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45528/0 (socket says 172.21.15.46:45528) 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 -- 172.21.15.46:0/789480247 learned_addr learned my addr 172.21.15.46:0/789480247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2408a00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f240406e780 0x7f240414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 -- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f240406e780 msgr2=0x7f240414a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f240406e780 0x7f240414a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 -- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f240414d200 msgr2=0x7f2404167330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f240414d200 0x7f2404167330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.349+0000 7f2403e00640 1 -- 172.21.15.46:0/789480247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240415f3e0 con 0x7f24040b43c0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f2403e00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f23f400c400 tx=0x7f23f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4017070 con 0x7f24040b43c0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f40052d0 con 0x7f24040b43c0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4003830 con 0x7f24040b43c0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240415f6c0 con 0x7f24040b43c0 2024-04-26T23:52:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.350+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2404164f10 con 0x7f24040b43c0 2024-04-26T23:52:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.352+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2404069530 con 0x7f24040b43c0 2024-04-26T23:52:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.353+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23f401a070 con 0x7f24040b43c0 2024-04-26T23:52:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.354+0000 7f2400c00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 0x7f23e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.354+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f23f408b0d0 con 0x7f24040b43c0 2024-04-26T23:52:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.355+0000 7f2408a00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 0x7f23e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.356+0000 7f2408a00640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 0x7f23e806ede0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.359+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23f4059340 con 0x7f24040b43c0 2024-04-26T23:52:31.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.565+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24040d08d0 con 0x7f24040b43c0 2024-04-26T23:52:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.568+0000 7f2400c00640 1 -- 172.21.15.46:0/789480247 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16953 (secure 0 0 0) 0x7f23f4058c40 con 0x7f24040b43c0 2024-04-26T23:52:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:31.571 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":150,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:51:57.743412+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:16.864971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:18.039678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.570+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 msgr2=0x7f23e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.570+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 0x7f23e806ede0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).stop 2024-04-26T23:52:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.570+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 msgr2=0x7f2404073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.570+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f23f400c400 tx=0x7f23f400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 shutdown_connections 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f23e806c920 0x7f23e806ede0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f240414d200 0x7f2404167330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040b43c0 0x7f2404073570 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 --2- 172.21.15.46:0/789480247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f240406e780 0x7f240414a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 >> 172.21.15.46:0/789480247 conn(0x7f240406c530 msgr2=0x7f2404161890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 shutdown_connections 2024-04-26T23:52:31.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.571+0000 7f240b400640 1 -- 172.21.15.46:0/789480247 wait complete. 2024-04-26T23:52:31.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:52:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.810+0000 7f1d0b200640 1 Processor -- start 2024-04-26T23:52:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.811+0000 7f1d0b200640 1 -- start start 2024-04-26T23:52:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.812+0000 7f1d0b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d0c14c2e0 0x7f1d0c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.812+0000 7f1d0b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.812+0000 7f1d0b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d0c14c6c0 con 0x7f1d0c0cd9f0 2024-04-26T23:52:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.812+0000 7f1d0b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0c0cde60 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.812+0000 7f1d0b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0c0cb8b0 con 0x7f1d0c0cb0f0 2024-04-26T23:52:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.813+0000 7f1d0a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1d0c0cd9f0 0x7f1d0c0c5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54892/0 (socket says 172.21.15.46:54892) 2024-04-26T23:52:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.813+0000 7f1d0a800640 1 -- 172.21.15.46:0/695111128 learned_addr learned my addr 172.21.15.46:0/695111128 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.813+0000 7f1d09e00640 1 --2- 172.21.15.46:0/695111128 >> v2:172.21.15.142:3300/0 conn(0x7f1d0c14c2e0 0x7f1d0c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.813+0000 7f1d03e00640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453971122 0 0) 0x7f1d0c14c6c0 con 0x7f1d0c0cd9f0 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cf0003520 con 0x7f1d0c0cd9f0 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09e00640 1 -- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cb0f0 msgr2=0x7f1d0c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09e00640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09e00640 1 -- 172.21.15.46:0/695111128 >> v1:172.21.15.46:6789/0 conn(0x7f1d0c0cd9f0 legacy=0x7f1d0c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4224888269 0 0) 0x7f1cf0003520 con 0x7f1d0c0cd9f0 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.814+0000 7f1d09e00640 1 -- 172.21.15.46:0/695111128 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c167e10 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.816+0000 7f1d09e00640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c0ccdd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097a0 tx=0x7f1cfc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ead64d4e0b850591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.817+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc037070 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.817+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cfc02ab10 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.817+0000 7f1d09400640 1 -- 172.21.15.46:0/695111128 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d040019f0 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.817+0000 7f1d0b200640 1 -- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c14c2e0 msgr2=0x7f1d0c0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.817+0000 7f1d0b200640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c0ccdd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097a0 tx=0x7f1cfc02a080 comp rx=0 tx=0).stop 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 -- 172.21.15.46:0/695111128 shutdown_connections 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 --2- 172.21.15.46:0/695111128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c0ccdd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 -- 172.21.15.46:0/695111128 >> 172.21.15.46:0/695111128 conn(0x7f1d0c0b26d0 msgr2=0x7f1d0c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 -- 172.21.15.46:0/695111128 shutdown_connections 2024-04-26T23:52:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.818+0000 7f1d0b200640 1 -- 172.21.15.46:0/695111128 wait complete. 2024-04-26T23:52:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.819+0000 7f1d0b200640 1 Processor -- start 2024-04-26T23:52:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.820+0000 7f1d0b200640 1 -- start start 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.820+0000 7f1d0b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.820+0000 7f1d0b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cd9f0 0x7f1d0c0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.820+0000 7f1d0b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c167610 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c167330 con 0x7f1d0c0cb0f0 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0c1678f0 con 0x7f1d0c0cd9f0 2024-04-26T23:52:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45538/0 (socket says 172.21.15.46:45538) 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 -- 172.21.15.46:0/2743925248 learned_addr learned my addr 172.21.15.46:0/2743925248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d09e00640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d03e00640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cd9f0 0x7f1d0c0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 -- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cd9f0 msgr2=0x7f1d0c0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cd9f0 0x7f1d0c0c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 -- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c0cb0f0 msgr2=0x7f1d0c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.821+0000 7f1d0a800640 1 -- 172.21.15.46:0/2743925248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c1f9f80 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d0a800640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400cc00 tx=0x7f1d0400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04017070 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0400ce10 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012600 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c0b3860 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.822+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c0b3e20 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.823+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c0693b0 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.824+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d0401a070 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.825+0000 7f1d02000640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 0x7f1ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.825+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6159+0+0 (secure 0 0 0) 0x7f1d0408bba0 con 0x7f1d0c14c2e0 2024-04-26T23:52:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.826+0000 7f1d09e00640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 0x7f1ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.827+0000 7f1d09e00640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 0x7f1ce806edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc034000 tx=0x7f1cfc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:31.830+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d04059720 con 0x7f1d0c14c2e0 2024-04-26T23:52:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.082+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1d0c0c73a0 con 0x7f1d0c14c2e0 2024-04-26T23:52:32.086 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:52:32.083+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:52:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.087+0000 7f1d02000640 1 -- 172.21.15.46:0/2743925248 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) ==== 161+0+0 (secure 0 0 0) 0x7f1d0405da20 con 0x7f1d0c14c2e0 2024-04-26T23:52:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:52:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 msgr2=0x7f1ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 0x7f1ce806edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc034000 tx=0x7f1cfc02e000 comp rx=0 tx=0).stop 2024-04-26T23:52:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 msgr2=0x7f1d0c1f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400cc00 tx=0x7f1d0400a430 comp rx=0 tx=0).stop 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 shutdown_connections 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1ce806c8f0 0x7f1ce806edb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d0c14c2e0 0x7f1d0c1f9230 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d0c0cd9f0 0x7f1d0c0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 --2- 172.21.15.46:0/2743925248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d0c0cb0f0 0x7f1d0c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 >> 172.21.15.46:0/2743925248 conn(0x7f1d0c0b26d0 msgr2=0x7f1d0c0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 shutdown_connections 2024-04-26T23:52:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.090+0000 7f1d0b200640 1 -- 172.21.15.46:0/2743925248 wait complete. 2024-04-26T23:52:32.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:52:32.203 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:52:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.307+0000 7fe2b6400640 1 Processor -- start 2024-04-26T23:52:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.307+0000 7fe2b6400640 1 -- start start 2024-04-26T23:52:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2b6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2b00b53b0 0x7fe2b015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2b6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b00d07e0 con 0x7fe2b015e050 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2b6400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b006eb90 con 0x7fe2b015d730 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2b6400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b00b4310 con 0x7fe2b00b53b0 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2aea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2b00b53b0 0x7fe2b015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2afe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe2b015e050 0x7fe2b006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54696/0 (socket says 172.21.15.46:54696) 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.309+0000 7fe2afe00640 1 -- 172.21.15.46:0/2119481101 learned_addr learned my addr 172.21.15.46:0/2119481101 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.310+0000 7fe2aea00640 1 -- 172.21.15.46:0/2119481101 >> v1:172.21.15.142:6789/0 conn(0x7fe2b015d730 legacy=0x7fe2b00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.310+0000 7fe2aea00640 1 -- 172.21.15.46:0/2119481101 >> v1:172.21.15.46:6789/0 conn(0x7fe2b015e050 legacy=0x7fe2b006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.310+0000 7fe2ae000640 1 -- 172.21.15.46:0/2119481101 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112777893 0 0) 0x7fe2b00d07e0 con 0x7fe2b015e050 2024-04-26T23:52:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.310+0000 7fe2aea00640 1 -- 172.21.15.46:0/2119481101 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00b45f0 con 0x7fe2b00b53b0 2024-04-26T23:52:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2aea00640 1 --2- 172.21.15.46:0/2119481101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b015db10 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097a0 tx=0x7fe29c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7277771bb87d7fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2ae000640 1 -- 172.21.15.46:0/2119481101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c037070 con 0x7fe2b00b53b0 2024-04-26T23:52:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2ae000640 1 -- 172.21.15.46:0/2119481101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c032440 con 0x7fe2b00b53b0 2024-04-26T23:52:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2ae000640 1 -- 172.21.15.46:0/2119481101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c032810 con 0x7fe2b00b53b0 2024-04-26T23:52:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2b6400640 1 -- 172.21.15.46:0/2119481101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 msgr2=0x7fe2b015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.311+0000 7fe2b6400640 1 --2- 172.21.15.46:0/2119481101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b015db10 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097a0 tx=0x7fe29c02a230 comp rx=0 tx=0).stop 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.312+0000 7fe2b6400640 1 -- 172.21.15.46:0/2119481101 shutdown_connections 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.312+0000 7fe2b6400640 1 --2- 172.21.15.46:0/2119481101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b015db10 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.312+0000 7fe2b6400640 1 -- 172.21.15.46:0/2119481101 >> 172.21.15.46:0/2119481101 conn(0x7fe2b006c980 msgr2=0x7fe2b00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.312+0000 7fe2b6400640 1 -- 172.21.15.46:0/2119481101 shutdown_connections 2024-04-26T23:52:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.312+0000 7fe2b6400640 1 -- 172.21.15.46:0/2119481101 wait complete. 2024-04-26T23:52:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.313+0000 7fe2b6400640 1 Processor -- start 2024-04-26T23:52:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.314+0000 7fe2b6400640 1 -- start start 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.314+0000 7fe2b6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.314+0000 7fe2b6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b015d730 0x7fe2b00c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2b6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b015e050 0x7fe2b00c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2b6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00b4e90 con 0x7fe2b00b53b0 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2b6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00b4bb0 con 0x7fe2b015e050 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2b6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b015e460 con 0x7fe2b015d730 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34990/0 (socket says 172.21.15.46:34990) 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 -- 172.21.15.46:0/1398732432 learned_addr learned my addr 172.21.15.46:0/1398732432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2afe00640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b015e050 0x7fe2b00c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2aea00640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b015d730 0x7fe2b00c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 -- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b015d730 msgr2=0x7fe2b00c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b015d730 0x7fe2b00c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 -- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b015e050 msgr2=0x7fe2b00c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b015e050 0x7fe2b00c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.315+0000 7fe2af400640 1 -- 172.21.15.46:0/1398732432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01f9590 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2af400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c870 tx=0x7fe2a000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003b70 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a0003d30 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000aa90 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b00c5bc0 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.316+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b00c6180 con 0x7fe2b00b53b0 2024-04-26T23:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.317+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0069530 con 0x7fe2b00b53b0 2024-04-26T23:52:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.318+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2a000ac30 con 0x7fe2b00b53b0 2024-04-26T23:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.319+0000 7fe2a7400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.319+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6245+0+0 (secure 0 0 0) 0x7fe2a008bd90 con 0x7fe2b00b53b0 2024-04-26T23:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.320+0000 7fe2aea00640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.321+0000 7fe2aea00640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe29c034000 tx=0x7fe29c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.324+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2a0059f00 con 0x7fe2b00b53b0 2024-04-26T23:52:32.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:32.570+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe2b00cff90 con 0x7fe2b00b53b0 2024-04-26T23:52:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.097+0000 7fe2a7400640 1 -- 172.21.15.46:0/1398732432 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) ==== 104+0+0 (secure 0 0 0) 0x7fe2a00600d0 con 0x7fe2b00b53b0 2024-04-26T23:52:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.100+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 msgr2=0x7fe29006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe29c034000 tx=0x7fe29c02e000 comp rx=0 tx=0).stop 2024-04-26T23:52:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 msgr2=0x7fe2b01f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c870 tx=0x7fe2a000cd30 comp rx=0 tx=0).stop 2024-04-26T23:52:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 shutdown_connections 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b015e050 0x7fe2b00c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2b015d730 0x7fe2b00c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 --2- 172.21.15.46:0/1398732432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b00b53b0 0x7fe2b01f4e70 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 >> 172.21.15.46:0/1398732432 conn(0x7fe2b006c980 msgr2=0x7fe2b00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 shutdown_connections 2024-04-26T23:52:33.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.101+0000 7fe2b6400640 1 -- 172.21.15.46:0/1398732432 wait complete. 2024-04-26T23:52:33.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:52:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.337+0000 7f1eff200640 1 Processor -- start 2024-04-26T23:52:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.337+0000 7f1eff200640 1 -- start start 2024-04-26T23:52:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.338+0000 7f1eff200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 0x7f1f000b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.338+0000 7f1eff200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 0x7f1f000d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.338+0000 7f1eff200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f000b6140 con 0x7f1f000d5ac0 2024-04-26T23:52:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.338+0000 7f1eff200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1f000b82e0 con 0x7f1f000d5660 2024-04-26T23:52:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.338+0000 7f1eff200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f000d2f30 con 0x7f1f001628e0 2024-04-26T23:52:33.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.339+0000 7f1efd400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 0x7f1f000d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.339+0000 7f1efd400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 0x7f1f000d1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:57874/0 (socket says 172.21.15.46:57874) 2024-04-26T23:52:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.339+0000 7f1efd400640 1 -- 172.21.15.46:0/2864040546 learned_addr learned my addr 172.21.15.46:0/2864040546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.339+0000 7f1efde00640 1 --2- 172.21.15.46:0/2864040546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 0x7f1f000b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efca00640 1 -- 172.21.15.46:0/2864040546 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091462430 0 0) 0x7f1ef8001110 con 0x7f1f000d5ac0 2024-04-26T23:52:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efca00640 1 -- 172.21.15.46:0/2864040546 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ee4003520 con 0x7f1f000d5ac0 2024-04-26T23:52:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efde00640 1 -- 172.21.15.46:0/2864040546 >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 msgr2=0x7f1f000d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efde00640 1 --2- 172.21.15.46:0/2864040546 >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 0x7f1f000d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efde00640 1 -- 172.21.15.46:0/2864040546 >> v1:172.21.15.142:6789/0 conn(0x7f1f000d5ac0 legacy=0x7f1f000d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.340+0000 7f1efde00640 1 -- 172.21.15.46:0/2864040546 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f000d0f10 con 0x7f1f001628e0 2024-04-26T23:52:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.342+0000 7f1efde00640 1 --2- 172.21.15.46:0/2864040546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 0x7f1f000b7da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80097a0 tx=0x7f1ee802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8fc0658501c9e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.342+0000 7f1efca00640 1 -- 172.21.15.46:0/2864040546 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8037070 con 0x7f1f001628e0 2024-04-26T23:52:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.342+0000 7f1efca00640 1 -- 172.21.15.46:0/2864040546 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8032440 con 0x7f1f001628e0 2024-04-26T23:52:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.342+0000 7f1efca00640 1 -- 172.21.15.46:0/2864040546 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80327f0 con 0x7f1f001628e0 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 -- 172.21.15.46:0/2864040546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 msgr2=0x7f1f000b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 --2- 172.21.15.46:0/2864040546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 0x7f1f000b7da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80097a0 tx=0x7f1ee802a180 comp rx=0 tx=0).stop 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 -- 172.21.15.46:0/2864040546 shutdown_connections 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 --2- 172.21.15.46:0/2864040546 >> v2:172.21.15.142:3300/0 conn(0x7f1f000d5660 0x7f1f000d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 --2- 172.21.15.46:0/2864040546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f001628e0 0x7f1f000b7da0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 -- 172.21.15.46:0/2864040546 >> 172.21.15.46:0/2864040546 conn(0x7f1f0009e2c0 msgr2=0x7f1f000d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 -- 172.21.15.46:0/2864040546 shutdown_connections 2024-04-26T23:52:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.343+0000 7f1eff200640 1 -- 172.21.15.46:0/2864040546 wait complete. 2024-04-26T23:52:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.345+0000 7f1eff200640 1 Processor -- start 2024-04-26T23:52:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.345+0000 7f1eff200640 1 -- start start 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f000d5ac0 0x7f1f0015ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f001628e0 0x7f1f0015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f000d0c30 con 0x7f1f000d5660 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f000d0950 con 0x7f1f001628e0 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1eff200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f000d2bc0 con 0x7f1f000d5ac0 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1efde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1efde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34994/0 (socket says 172.21.15.46:34994) 2024-04-26T23:52:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.346+0000 7f1efde00640 1 -- 172.21.15.46:0/1063275653 learned_addr learned my addr 172.21.15.46:0/1063275653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 -- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f000d5ac0 msgr2=0x7f1f0015ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efd400640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f000d5ac0 0x7f1f0015ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efe800640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f001628e0 0x7f1f0015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f000d5ac0 0x7f1f0015ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 -- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f001628e0 msgr2=0x7f1f0015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f001628e0 0x7f1f0015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 -- 172.21.15.46:0/1063275653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f000ce300 con 0x7f1f000d5660 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1efde00640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f1ee802a1c0 tx=0x7f1ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.347+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8037040 con 0x7f1f000d5660 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.348+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8042070 con 0x7f1f000d5660 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.348+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee803b930 con 0x7f1f000d5660 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.348+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0015ef10 con 0x7f1f000d5660 2024-04-26T23:52:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.348+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0015f4d0 con 0x7f1f000d5660 2024-04-26T23:52:33.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.349+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f000693b0 con 0x7f1f000d5660 2024-04-26T23:52:33.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.350+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ee803bad0 con 0x7f1f000d5660 2024-04-26T23:52:33.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.351+0000 7f1ef6000640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 0x7f1edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:33.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.351+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6245+0+0 (secure 0 0 0) 0x7f1ee80ac680 con 0x7f1f000d5660 2024-04-26T23:52:33.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.351+0000 7f1efd400640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 0x7f1edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:33.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.353+0000 7f1efd400640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 0x7f1edc06edb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1f00161960 tx=0x7f1ef000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:33.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.355+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ee807a7f0 con 0x7f1f000d5660 2024-04-26T23:52:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:33.560+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1f000d4ff0 con 0x7f1f000d5660 2024-04-26T23:52:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.101+0000 7f1ef6000640 1 -- 172.21.15.46:0/1063275653 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v152) ==== 181+0+0 (secure 0 0 0) 0x7f1ee807a0f0 con 0x7f1f000d5660 2024-04-26T23:52:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:52:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.104+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 msgr2=0x7f1edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.104+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 0x7f1edc06edb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1f00161960 tx=0x7f1ef000b040 comp rx=0 tx=0).stop 2024-04-26T23:52:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.104+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 msgr2=0x7f1f0015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.104+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f1ee802a1c0 tx=0x7f1ee80046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 shutdown_connections 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1edc06c8f0 0x7f1edc06edb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f001628e0 0x7f1f0015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f000d5ac0 0x7f1f0015ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 --2- 172.21.15.46:0/1063275653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f000d5660 0x7f1f0015d590 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 >> 172.21.15.46:0/1063275653 conn(0x7f1f0009e2c0 msgr2=0x7f1f000d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 shutdown_connections 2024-04-26T23:52:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.105+0000 7f1eff200640 1 -- 172.21.15.46:0/1063275653 wait complete. 2024-04-26T23:52:34.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:52:34.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.341+0000 7f355ba00640 1 Processor -- start 2024-04-26T23:52:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.342+0000 7f355ba00640 1 -- start start 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f355ba00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 0x7f355416a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f355ba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35540cfec0 con 0x7f355415f490 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f355ba00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35540d01a0 con 0x7f35540d3e70 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f355ba00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f355415f870 con 0x7f35540d2180 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f3559000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f355415f490 0x7f35540d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54702/0 (socket says 172.21.15.46:54702) 2024-04-26T23:52:34.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.343+0000 7f3559000640 1 -- 172.21.15.46:0/3791385372 learned_addr learned my addr 172.21.15.46:0/3791385372 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202782597 0 0) 0x7f35540cfec0 con 0x7f355415f490 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3538003520 con 0x7f355415f490 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3531178542 0 0) 0x7f3538003520 con 0x7f355415f490 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 msgr2=0x7f355416a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 0x7f355416a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 >> v1:172.21.15.142:6789/0 conn(0x7f35540d3e70 legacy=0x7f35540d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540d0480 con 0x7f355415f490 2024-04-26T23:52:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3544001670 con 0x7f355415f490 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 >> v1:172.21.15.46:6789/0 conn(0x7f355415f490 legacy=0x7f35540d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f3538003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 0x7f3538004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3538004db0 0x7f3538005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f355415f870 con 0x7f35380039e0 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f353800bc90 con 0x7f3538004db0 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.344+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f353800be90 con 0x7f35540d2180 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f3538003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 -- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 msgr2=0x7f3538004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540d2180 0x7f3538004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 -- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3538004db0 msgr2=0x7f3538005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3538004db0 0x7f3538005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.345+0000 7f3559000640 1 -- 172.21.15.46:0/3791385372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3538003520 con 0x7f35380039e0 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.346+0000 7f3559000640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f3538003de0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f354400c210 tx=0x7f3538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d0ee3dfa514c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544007d30 con 0x7f35380039e0 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3544005af0 con 0x7f35380039e0 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f3553e00640 1 -- 172.21.15.46:0/3791385372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544007070 con 0x7f35380039e0 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 -- 172.21.15.46:0/3791385372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 msgr2=0x7f3538003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 --2- 172.21.15.46:0/3791385372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f3538003de0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f354400c210 tx=0x7f3538005830 comp rx=0 tx=0).stop 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f3559a00640 1 -- 172.21.15.46:0/3791385372 reap_dead start 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 -- 172.21.15.46:0/3791385372 shutdown_connections 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 -- 172.21.15.46:0/3791385372 >> 172.21.15.46:0/3791385372 conn(0x7f355409e450 msgr2=0x7f35540d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 -- 172.21.15.46:0/3791385372 shutdown_connections 2024-04-26T23:52:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.347+0000 7f355ba00640 1 -- 172.21.15.46:0/3791385372 wait complete. 2024-04-26T23:52:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.349+0000 7f355ba00640 1 Processor -- start 2024-04-26T23:52:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.349+0000 7f355ba00640 1 -- start start 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355415b5f0 0x7f3554152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540ceb30 0x7f3554160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35540d0a40 con 0x7f35380039e0 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f35540b8fe0 con 0x7f355415b5f0 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.350+0000 7f355ba00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f35540d0f60 con 0x7f35540ceb30 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35014/0 (socket says 172.21.15.46:35014) 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 -- 172.21.15.46:0/1139784110 learned_addr learned my addr 172.21.15.46:0/1139784110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559a00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540ceb30 0x7f3554160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f354be00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355415b5f0 0x7f3554152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 -- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540ceb30 msgr2=0x7f3554160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540ceb30 0x7f3554160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 -- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355415b5f0 msgr2=0x7f3554152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355415b5f0 0x7f3554152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.351+0000 7f3559000640 1 -- 172.21.15.46:0/1139784110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540c9dd0 con 0x7f35380039e0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f3559000640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f354400c270 tx=0x7f35540cfc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544043070 con 0x7f35380039e0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3544038040 con 0x7f35380039e0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544007290 con 0x7f35380039e0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35540ca0b0 con 0x7f35380039e0 2024-04-26T23:52:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.352+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35540ca670 con 0x7f35380039e0 2024-04-26T23:52:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.353+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3554069530 con 0x7f35380039e0 2024-04-26T23:52:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.354+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3544007430 con 0x7f35380039e0 2024-04-26T23:52:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.355+0000 7f3551600640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 0x7f353006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.355+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5807+0+0 (secure 0 0 0) 0x7f35440ab4a0 con 0x7f35380039e0 2024-04-26T23:52:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.356+0000 7f354be00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 0x7f353006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.357+0000 7f354be00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 0x7f353006ee00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.360+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3544001630 con 0x7f35380039e0 2024-04-26T23:52:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:34.569+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35541f56e0 con 0x7f35380039e0 2024-04-26T23:52:35.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.114+0000 7f3551600640 1 -- 172.21.15.46:0/1139784110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v153) ==== 169+0+0 (secure 0 0 0) 0x7f3544005ac0 con 0x7f35380039e0 2024-04-26T23:52:35.118 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:52:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.118+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 msgr2=0x7f353006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 0x7f353006ee00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).stop 2024-04-26T23:52:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 msgr2=0x7f35541f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f354400c270 tx=0x7f35540cfc70 comp rx=0 tx=0).stop 2024-04-26T23:52:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 shutdown_connections 2024-04-26T23:52:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f353006c940 0x7f353006ee00 secure :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).stop 2024-04-26T23:52:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35540ceb30 0x7f3554160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355415b5f0 0x7f3554152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 --2- 172.21.15.46:0/1139784110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35380039e0 0x7f35541f51a0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 >> 172.21.15.46:0/1139784110 conn(0x7f355409e450 msgr2=0x7f35540b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.119+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 shutdown_connections 2024-04-26T23:52:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.120+0000 7f355ba00640 1 -- 172.21.15.46:0/1139784110 wait complete. 2024-04-26T23:52:35.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:52:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.348+0000 7faab8e00640 1 Processor -- start 2024-04-26T23:52:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.348+0000 7faab8e00640 1 -- start start 2024-04-26T23:52:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.349+0000 7faab8e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab4146740 0x7faab415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.349+0000 7faab8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faab40d0330 0x7faab41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.349+0000 7faab8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faab40ce670 0x7faab40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab8e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faab4146b20 con 0x7faab40d0330 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab8e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faab40ccf30 con 0x7faab40ce670 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab8e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faab4160af0 con 0x7faab4146740 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faab40d0330 0x7faab41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faab40d0330 0x7faab41612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35026/0 (socket says 172.21.15.46:35026) 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 -- 172.21.15.46:0/1935735480 learned_addr learned my addr 172.21.15.46:0/1935735480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 -- 172.21.15.46:0/1935735480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab4146740 msgr2=0x7faab415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 --2- 172.21.15.46:0/1935735480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab4146740 0x7faab415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 -- 172.21.15.46:0/1935735480 >> v2:172.21.15.142:3300/0 conn(0x7faab40ce670 msgr2=0x7faab40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 --2- 172.21.15.46:0/1935735480 >> v2:172.21.15.142:3300/0 conn(0x7faab40ce670 0x7faab40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.350+0000 7faab2a00640 1 -- 172.21.15.46:0/1935735480 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab4160dd0 con 0x7faab40d0330 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.351+0000 7faab2a00640 1 --2- 172.21.15.46:0/1935735480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab40d0330 0x7faab41612f0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7faab406f090 tx=0x7faaa002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d700811ee83b9cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab2000640 1 -- 172.21.15.46:0/1935735480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0037070 con 0x7faab40d0330 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab2000640 1 -- 172.21.15.46:0/1935735480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa0032440 con 0x7faab40d0330 2024-04-26T23:52:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab2000640 1 -- 172.21.15.46:0/1935735480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0032810 con 0x7faab40d0330 2024-04-26T23:52:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 -- 172.21.15.46:0/1935735480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab40d0330 msgr2=0x7faab41612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 --2- 172.21.15.46:0/1935735480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab40d0330 0x7faab41612f0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7faab406f090 tx=0x7faaa002a1d0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 -- 172.21.15.46:0/1935735480 shutdown_connections 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 --2- 172.21.15.46:0/1935735480 >> v2:172.21.15.142:3300/0 conn(0x7faab40ce670 0x7faab40ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 --2- 172.21.15.46:0/1935735480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab40d0330 0x7faab41612f0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 --2- 172.21.15.46:0/1935735480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab4146740 0x7faab415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.352+0000 7faab8e00640 1 -- 172.21.15.46:0/1935735480 >> 172.21.15.46:0/1935735480 conn(0x7faab40b26d0 msgr2=0x7faab40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.353+0000 7faab8e00640 1 -- 172.21.15.46:0/1935735480 shutdown_connections 2024-04-26T23:52:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.353+0000 7faab8e00640 1 -- 172.21.15.46:0/1935735480 wait complete. 2024-04-26T23:52:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.354+0000 7faab8e00640 1 Processor -- start 2024-04-26T23:52:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.355+0000 7faab8e00640 1 -- start start 2024-04-26T23:52:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab40ce670 0x7faab40afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab40d0330 0x7faab40b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab40cefe0 con 0x7faab4146740 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faab40ccb70 con 0x7faab40ce670 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab8e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faab40b4450 con 0x7faab40d0330 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35036/0 (socket says 172.21.15.46:35036) 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab3e00640 1 -- 172.21.15.46:0/3304003781 learned_addr learned my addr 172.21.15.46:0/3304003781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab3400640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab40ce670 0x7faab40afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.356+0000 7faab2a00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab40d0330 0x7faab40b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 -- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab40d0330 msgr2=0x7faab40b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab40d0330 0x7faab40b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 -- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab40ce670 msgr2=0x7faab40afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab40ce670 0x7faab40afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 -- 172.21.15.46:0/3304003781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab40c9e70 con 0x7faab4146740 2024-04-26T23:52:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab3e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b970 tx=0x7faaac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac019070 con 0x7faab4146740 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac00cb60 con 0x7faab4146740 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac014440 con 0x7faab4146740 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab40b5140 con 0x7faab4146740 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.357+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab40b5700 con 0x7faab4146740 2024-04-26T23:52:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.359+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab40693b0 con 0x7faab4146740 2024-04-26T23:52:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.359+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaac01c070 con 0x7faab4146740 2024-04-26T23:52:35.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.361+0000 7faaab400640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 0x7faa9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.361+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7faaac08b7f0 con 0x7faab4146740 2024-04-26T23:52:35.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.362+0000 7faab3400640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 0x7faa9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.363+0000 7faab3400640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 0x7faa9406edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009890 tx=0x7faaa4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.367+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faaac059cf0 con 0x7faab4146740 2024-04-26T23:52:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.566+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7faab406f400 con 0x7faab4146740 2024-04-26T23:52:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.573+0000 7faaab400640 1 -- 172.21.15.46:0/3304003781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7faaac0595f0 con 0x7faab4146740 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 msgr2=0x7faa9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 0x7faa9406edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009890 tx=0x7faaa4008040 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 msgr2=0x7faab40c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b970 tx=0x7faaac00be30 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 shutdown_connections 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faa9406c8f0 0x7faa9406edb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab4146740 0x7faab40c8580 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab40d0330 0x7faab40b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 --2- 172.21.15.46:0/3304003781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab40ce670 0x7faab40afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 >> 172.21.15.46:0/3304003781 conn(0x7faab40b26d0 msgr2=0x7faab4143550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 shutdown_connections 2024-04-26T23:52:35.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.576+0000 7faab8e00640 1 -- 172.21.15.46:0/3304003781 wait complete. 2024-04-26T23:52:35.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-26T23:52:35.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.782+0000 7f7c25600640 1 Processor -- start 2024-04-26T23:52:35.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.783+0000 7f7c25600640 1 -- start start 2024-04-26T23:52:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.784+0000 7f7c25600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c200b5140 0x7f7c200d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.784+0000 7f7c25600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7c200d5410 0x7f7c200d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.784+0000 7f7c25600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c200cfd60 con 0x7f7c2015f4b0 2024-04-26T23:52:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.784+0000 7f7c25600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2015f890 con 0x7f7c200b5140 2024-04-26T23:52:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.784+0000 7f7c25600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7c200d58c0 con 0x7f7c200d5410 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c2015f4b0 0x7f7c200d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54716/0 (socket says 172.21.15.46:54716) 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1ea00640 1 -- 172.21.15.46:0/1091938155 learned_addr learned my addr 172.21.15.46:0/1091938155 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1e000640 1 --2- 172.21.15.46:0/1091938155 >> v2:172.21.15.46:3300/0 conn(0x7f7c200b5140 0x7f7c200d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1d600640 1 -- 172.21.15.46:0/1091938155 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590268706 0 0) 0x7f7c10001110 con 0x7f7c2015f4b0 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1d600640 1 -- 172.21.15.46:0/1091938155 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c00003520 con 0x7f7c2015f4b0 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1e000640 1 -- 172.21.15.46:0/1091938155 >> v2:172.21.15.142:3300/0 conn(0x7f7c200d5410 msgr2=0x7f7c200d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1e000640 1 --2- 172.21.15.46:0/1091938155 >> v2:172.21.15.142:3300/0 conn(0x7f7c200d5410 0x7f7c200d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1e000640 1 -- 172.21.15.46:0/1091938155 >> v1:172.21.15.46:6789/0 conn(0x7f7c2015f4b0 legacy=0x7f7c200d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.785+0000 7f7c1e000640 1 -- 172.21.15.46:0/1091938155 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c200b55d0 con 0x7f7c200b5140 2024-04-26T23:52:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.786+0000 7f7c1e000640 1 --2- 172.21.15.46:0/1091938155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200b5140 0x7f7c200d4ed0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f7c200b3e30 tx=0x7f7c0802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e7308318afbb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c1d600640 1 -- 172.21.15.46:0/1091938155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08007740 con 0x7f7c200b5140 2024-04-26T23:52:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c1d600640 1 -- 172.21.15.46:0/1091938155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c080078e0 con 0x7f7c200b5140 2024-04-26T23:52:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c1d600640 1 -- 172.21.15.46:0/1091938155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08007cb0 con 0x7f7c200b5140 2024-04-26T23:52:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 -- 172.21.15.46:0/1091938155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200b5140 msgr2=0x7f7c200d4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 --2- 172.21.15.46:0/1091938155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200b5140 0x7f7c200d4ed0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f7c200b3e30 tx=0x7f7c0802a680 comp rx=0 tx=0).stop 2024-04-26T23:52:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 -- 172.21.15.46:0/1091938155 shutdown_connections 2024-04-26T23:52:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 --2- 172.21.15.46:0/1091938155 >> v2:172.21.15.142:3300/0 conn(0x7f7c200d5410 0x7f7c200d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 --2- 172.21.15.46:0/1091938155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200b5140 0x7f7c200d4ed0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 -- 172.21.15.46:0/1091938155 >> 172.21.15.46:0/1091938155 conn(0x7f7c2008eb50 msgr2=0x7f7c2009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.787+0000 7f7c25600640 1 -- 172.21.15.46:0/1091938155 shutdown_connections 2024-04-26T23:52:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.788+0000 7f7c25600640 1 -- 172.21.15.46:0/1091938155 wait complete. 2024-04-26T23:52:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.789+0000 7f7c25600640 1 Processor -- start 2024-04-26T23:52:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.789+0000 7f7c25600640 1 -- start start 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c2015f4b0 0x7f7c200cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c200cdeb0 0x7f7c20160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200b5e70 con 0x7f7c200d5410 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200b5b90 con 0x7f7c200cdeb0 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c25600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200b4680 con 0x7f7c2015f4b0 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35062/0 (socket says 172.21.15.46:35062) 2024-04-26T23:52:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.790+0000 7f7c1ea00640 1 -- 172.21.15.46:0/4015304604 learned_addr learned my addr 172.21.15.46:0/4015304604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 -- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c2015f4b0 msgr2=0x7f7c200cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c2015f4b0 0x7f7c200cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 -- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c200cdeb0 msgr2=0x7f7c20160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c200cdeb0 0x7f7c20160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 -- 172.21.15.46:0/4015304604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2015fb40 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c1ea00640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c760 tx=0x7f7c1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10017070 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10005ad0 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10007e20 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c200cef80 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.791+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c200ce9c0 con 0x7f7c200d5410 2024-04-26T23:52:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.792+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2015fd10 con 0x7f7c200d5410 2024-04-26T23:52:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.794+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c1001a070 con 0x7f7c200d5410 2024-04-26T23:52:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.795+0000 7f7c16a00640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 0x7f7bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.795+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f7c1008adb0 con 0x7f7c200d5410 2024-04-26T23:52:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.797+0000 7f7c1e000640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 0x7f7bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.798+0000 7f7c1e000640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 0x7f7bfc06edb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:35.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:35.800+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c1008f070 con 0x7f7c200d5410 2024-04-26T23:52:36.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.099+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) -- 0x7f7c20069530 con 0x7f7c200d5410 2024-04-26T23:52:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.116+0000 7f7c16a00640 1 -- 172.21.15.46:0/4015304604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v711) ==== 166+0+0 (secure 0 0 0) 0x7f7c200cfd60 con 0x7f7c200d5410 2024-04-26T23:52:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 msgr2=0x7f7bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 0x7f7bfc06edb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 msgr2=0x7f7c2014d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c760 tx=0x7f7c1000cc20 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 shutdown_connections 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7bfc06c8f0 0x7f7bfc06edb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c200cdeb0 0x7f7c20160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c2015f4b0 0x7f7c200cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 --2- 172.21.15.46:0/4015304604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c200d5410 0x7f7c2014d200 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.120+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 >> 172.21.15.46:0/4015304604 conn(0x7f7c2008eb50 msgr2=0x7f7c200d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.121+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 shutdown_connections 2024-04-26T23:52:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.121+0000 7f7c25600640 1 -- 172.21.15.46:0/4015304604 wait complete. 2024-04-26T23:52:36.141 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-26T23:52:36.141 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-26T23:52:36.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.314+0000 7f89f1c00640 1 Processor -- start 2024-04-26T23:52:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.314+0000 7f89f1c00640 1 -- start start 2024-04-26T23:52:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f89ec0720e0 0x7f89ec0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89ec160910 0x7f89ec160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec073ad0 con 0x7f89ec1466b0 2024-04-26T23:52:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec1612a0 con 0x7f89ec160910 2024-04-26T23:52:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec146a90 con 0x7f89ec0720e0 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89ebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89ec1466b0 0x7f89ec06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54732/0 (socket says 172.21.15.46:54732) 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89ebe00640 1 -- 172.21.15.46:0/3402346300 learned_addr learned my addr 172.21.15.46:0/3402346300 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.316+0000 7f89f1200640 1 --2- 172.21.15.46:0/3402346300 >> v2:172.21.15.46:3300/0 conn(0x7f89ec160910 0x7f89ec160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89e3400640 1 --2- 172.21.15.46:0/3402346300 >> v2:172.21.15.142:3300/0 conn(0x7f89ec0720e0 0x7f89ec0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89eb400640 1 -- 172.21.15.46:0/3402346300 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108853631 0 0) 0x7f89ec073ad0 con 0x7f89ec1466b0 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89eb400640 1 -- 172.21.15.46:0/3402346300 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d0003520 con 0x7f89ec1466b0 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89f1200640 1 -- 172.21.15.46:0/3402346300 >> v2:172.21.15.142:3300/0 conn(0x7f89ec0720e0 msgr2=0x7f89ec0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89f1200640 1 --2- 172.21.15.46:0/3402346300 >> v2:172.21.15.142:3300/0 conn(0x7f89ec0720e0 0x7f89ec0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89f1200640 1 -- 172.21.15.46:0/3402346300 >> v1:172.21.15.46:6789/0 conn(0x7f89ec1466b0 legacy=0x7f89ec06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.317+0000 7f89f1200640 1 -- 172.21.15.46:0/3402346300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec073d70 con 0x7f89ec160910 2024-04-26T23:52:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89f1200640 1 --2- 172.21.15.46:0/3402346300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec160910 0x7f89ec160d60 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009d90 tx=0x7f89e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e26e53bbe99be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89eb400640 1 -- 172.21.15.46:0/3402346300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4007740 con 0x7f89ec160910 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89eb400640 1 -- 172.21.15.46:0/3402346300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e40078e0 con 0x7f89ec160910 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89eb400640 1 -- 172.21.15.46:0/3402346300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4007cb0 con 0x7f89ec160910 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89f1c00640 1 -- 172.21.15.46:0/3402346300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec160910 msgr2=0x7f89ec160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.319+0000 7f89f1c00640 1 --2- 172.21.15.46:0/3402346300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec160910 0x7f89ec160d60 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009d90 tx=0x7f89e402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 -- 172.21.15.46:0/3402346300 shutdown_connections 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 --2- 172.21.15.46:0/3402346300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec160910 0x7f89ec160d60 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 --2- 172.21.15.46:0/3402346300 >> v2:172.21.15.142:3300/0 conn(0x7f89ec0720e0 0x7f89ec0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 -- 172.21.15.46:0/3402346300 >> 172.21.15.46:0/3402346300 conn(0x7f89ec0b26d0 msgr2=0x7f89ec0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 -- 172.21.15.46:0/3402346300 shutdown_connections 2024-04-26T23:52:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.320+0000 7f89f1c00640 1 -- 172.21.15.46:0/3402346300 wait complete. 2024-04-26T23:52:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.321+0000 7f89f1c00640 1 Processor -- start 2024-04-26T23:52:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.322+0000 7f89f1c00640 1 -- start start 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.322+0000 7f89f1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 0x7f89ec137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89ec160910 0x7f89ec0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0cb470 con 0x7f89ec0720e0 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0cb190 con 0x7f89ec1466b0 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec0cdda0 con 0x7f89ec160910 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35070/0 (socket says 172.21.15.46:35070) 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 -- 172.21.15.46:0/1380495150 learned_addr learned my addr 172.21.15.46:0/1380495150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89e3400640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 0x7f89ec137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89f1200640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89ec160910 0x7f89ec0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 -- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89ec160910 msgr2=0x7f89ec0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89ec160910 0x7f89ec0b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 -- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 msgr2=0x7f89ec137580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 0x7f89ec137580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89ebe00640 1 -- 172.21.15.46:0/1380495150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec0b5d60 con 0x7f89ec0720e0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.323+0000 7f89e3400640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 0x7f89ec137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89ebe00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c3c0 tx=0x7f89dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc017070 con 0x7f89ec0720e0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc005ce0 con 0x7f89ec0720e0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0072f0 con 0x7f89ec0720e0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec0b5fe0 con 0x7f89ec0720e0 2024-04-26T23:52:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.324+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec0b65a0 con 0x7f89ec0720e0 2024-04-26T23:52:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.325+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec0693b0 con 0x7f89ec0720e0 2024-04-26T23:52:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.326+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89dc01a070 con 0x7f89ec0720e0 2024-04-26T23:52:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.327+0000 7f89e8c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.327+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f89dc08adb0 con 0x7f89ec0720e0 2024-04-26T23:52:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.328+0000 7f89e3400640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.329+0000 7f89e3400640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 0x7f89c806edb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.332+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89ec0b5d60 con 0x7f89ec0720e0 2024-04-26T23:52:36.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.540+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89ec0c7180 con 0x7f89ec0720e0 2024-04-26T23:52:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.542+0000 7f89e8c00640 1 -- 172.21.15.46:0/1380495150 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+13267 (secure 0 0 0) 0x7f89dc059350 con 0x7f89ec0720e0 2024-04-26T23:52:36.546 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:36.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":153,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:52:35.102117+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/3522383810":"2024-04-27T23:52:32.083507+0000","172.21.15.46:6833/3522383810":"2024-04-27T23:52:32.083507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 msgr2=0x7f89c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 0x7f89c806edb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4008040 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 msgr2=0x7f89ec1342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c3c0 tx=0x7f89dc00c880 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 shutdown_connections 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89ec160910 0x7f89ec0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ec1466b0 0x7f89ec137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.544+0000 7f89f1c00640 1 --2- 172.21.15.46:0/1380495150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0720e0 0x7f89ec1342e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.545+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 >> 172.21.15.46:0/1380495150 conn(0x7f89ec0b26d0 msgr2=0x7f89ec0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.545+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 shutdown_connections 2024-04-26T23:52:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.545+0000 7f89f1c00640 1 -- 172.21.15.46:0/1380495150 wait complete. 2024-04-26T23:52:36.566 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:52:36.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:52:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.772+0000 7f301aa00640 1 Processor -- start 2024-04-26T23:52:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.772+0000 7f301aa00640 1 -- start start 2024-04-26T23:52:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f301aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30140b4b50 0x7f3014073d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f301aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30140ce7d0 con 0x7f301415cec0 2024-04-26T23:52:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f301aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30140ceab0 con 0x7f30140cdd00 2024-04-26T23:52:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f301aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f30140ced90 con 0x7f30140b4b50 2024-04-26T23:52:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f3019600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f301415cec0 0x7f30140cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54740/0 (socket says 172.21.15.46:54740) 2024-04-26T23:52:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f3019600640 1 -- 172.21.15.46:0/3859147932 learned_addr learned my addr 172.21.15.46:0/3859147932 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f301a000640 1 --2- 172.21.15.46:0/3859147932 >> v2:172.21.15.142:3300/0 conn(0x7f30140b4b50 0x7f3014073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.774+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759398713 0 0) 0x7f30140ce7d0 con 0x7f301415cec0 2024-04-26T23:52:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ffc003520 con 0x7f301415cec0 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4012752795 0 0) 0x7f2ffc003520 con 0x7f301415cec0 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 >> v2:172.21.15.142:3300/0 conn(0x7f30140b4b50 msgr2=0x7f3014073d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 --2- 172.21.15.46:0/3859147932 >> v2:172.21.15.142:3300/0 conn(0x7f30140b4b50 0x7f3014073d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 >> v1:172.21.15.142:6789/0 conn(0x7f30140cdd00 legacy=0x7f30140735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30140cf070 con 0x7f301415cec0 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3008001690 con 0x7f301415cec0 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 >> v1:172.21.15.46:6789/0 conn(0x7f301415cec0 legacy=0x7f30140cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ffc003a00 0x7f2ffc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc004800 0x7f2ffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 0x7f2ffc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30140ced90 con 0x7f2ffc0051c0 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00c090 con 0x7f2ffc004800 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc00c290 con 0x7f2ffc003a00 2024-04-26T23:52:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.775+0000 7f301a000640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 0x7f2ffc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f3019600640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ffc003a00 0x7f2ffc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f301a000640 1 -- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ffc003a00 msgr2=0x7f2ffc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f301a000640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ffc003a00 0x7f2ffc003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f301a000640 1 -- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc004800 msgr2=0x7f2ffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f301a000640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc004800 0x7f2ffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.776+0000 7f301a000640 1 -- 172.21.15.46:0/3859147932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc003520 con 0x7f2ffc0051c0 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.777+0000 7f301a000640 1 -- 172.21.15.46:0/3859147932 reap_dead start 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.777+0000 7f301a000640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 0x7f2ffc00b9c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f300c009fc0 tx=0x7f2ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c03272055fa24f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.777+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c035020 con 0x7f2ffc0051c0 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.777+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c003bd0 con 0x7f2ffc0051c0 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.777+0000 7f3013e00640 1 -- 172.21.15.46:0/3859147932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c009c10 con 0x7f2ffc0051c0 2024-04-26T23:52:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 -- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 msgr2=0x7f2ffc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 0x7f2ffc00b9c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f300c009fc0 tx=0x7f2ffc005830 comp rx=0 tx=0).stop 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 -- 172.21.15.46:0/3859147932 shutdown_connections 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 --2- 172.21.15.46:0/3859147932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0051c0 0x7f2ffc00b9c0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 -- 172.21.15.46:0/3859147932 >> 172.21.15.46:0/3859147932 conn(0x7f30140b27b0 msgr2=0x7f30140b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 -- 172.21.15.46:0/3859147932 shutdown_connections 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.778+0000 7f301aa00640 1 -- 172.21.15.46:0/3859147932 wait complete. 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.780+0000 7f301aa00640 1 Processor -- start 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.780+0000 7f301aa00640 1 -- start start 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc0051c0 0x7f30140b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3014072980 0x7f30140cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30140cd5f0 con 0x7f3014153dd0 2024-04-26T23:52:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f301415d2a0 con 0x7f2ffc0051c0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30140cd770 con 0x7f3014072980 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35096/0 (socket says 172.21.15.46:35096) 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 -- 172.21.15.46:0/4249186529 learned_addr learned my addr 172.21.15.46:0/4249186529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 -- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3014072980 msgr2=0x7f30140cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3014072980 0x7f30140cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f3019600640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc0051c0 0x7f30140b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 -- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc0051c0 msgr2=0x7f30140b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.781+0000 7f301a000640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc0051c0 0x7f30140b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f301a000640 1 -- 172.21.15.46:0/4249186529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3014154790 con 0x7f3014153dd0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f301a000640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e230 tx=0x7f300c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c035030 con 0x7f3014153dd0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c043070 con 0x7f3014153dd0 2024-04-26T23:52:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30140af300 con 0x7f3014153dd0 2024-04-26T23:52:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c02d070 con 0x7f3014153dd0 2024-04-26T23:52:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.782+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30140af8c0 con 0x7f3014153dd0 2024-04-26T23:52:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.783+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30140b0080 con 0x7f3014153dd0 2024-04-26T23:52:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.784+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f300c005d10 con 0x7f3014153dd0 2024-04-26T23:52:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.785+0000 7f3011600640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.785+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f300c0ac710 con 0x7f3014153dd0 2024-04-26T23:52:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.786+0000 7f3019600640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.787+0000 7f3019600640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 0x7f2ff406ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3008001800 tx=0x7f3008009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.789+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30140b0080 con 0x7f3014153dd0 2024-04-26T23:52:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.993+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30140693b0 con 0x7f3014153dd0 2024-04-26T23:52:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.994+0000 7f3011600640 1 -- 172.21.15.46:0/4249186529 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) ==== 93+0+10 (secure 0 0 0) 0x7f300c07ac10 con 0x7f3014153dd0 2024-04-26T23:52:36.997 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 msgr2=0x7f2ff406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 0x7f2ff406ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3008001800 tx=0x7f3008009040 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 msgr2=0x7f3014154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e230 tx=0x7f300c009750 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 shutdown_connections 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3014153dd0 0x7f3014154250 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3014072980 0x7f30140cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 --2- 172.21.15.46:0/4249186529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ffc0051c0 0x7f30140b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 >> 172.21.15.46:0/4249186529 conn(0x7f30140b27b0 msgr2=0x7f30140b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 shutdown_connections 2024-04-26T23:52:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:36.997+0000 7f301aa00640 1 -- 172.21.15.46:0/4249186529 wait complete. 2024-04-26T23:52:37.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-26T23:52:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.217+0000 7fed97e00640 1 Processor -- start 2024-04-26T23:52:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.217+0000 7fed97e00640 1 -- start start 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed97e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed90160bd0 0x7fed90161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed97e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed900732c0 0x7fed90073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed97e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed90161940 con 0x7fed9006e510 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed97e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fed90071f80 con 0x7fed90160bd0 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed97e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fed90072260 con 0x7fed900732c0 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed95400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed9006e510 0x7fed9006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54746/0 (socket says 172.21.15.46:54746) 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.219+0000 7fed95400640 1 -- 172.21.15.46:0/4106423009 learned_addr learned my addr 172.21.15.46:0/4106423009 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed94a00640 1 --2- 172.21.15.46:0/4106423009 >> v2:172.21.15.46:3300/0 conn(0x7fed90160bd0 0x7fed90161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed95e00640 1 --2- 172.21.15.46:0/4106423009 >> v2:172.21.15.142:3300/0 conn(0x7fed900732c0 0x7fed90073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed8fe00640 1 -- 172.21.15.46:0/4106423009 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324079353 0 0) 0x7fed90161940 con 0x7fed9006e510 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed94a00640 1 -- 172.21.15.46:0/4106423009 >> v2:172.21.15.142:3300/0 conn(0x7fed900732c0 msgr2=0x7fed90073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed94a00640 1 --2- 172.21.15.46:0/4106423009 >> v2:172.21.15.142:3300/0 conn(0x7fed900732c0 0x7fed90073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed94a00640 1 -- 172.21.15.46:0/4106423009 >> v1:172.21.15.46:6789/0 conn(0x7fed9006e510 legacy=0x7fed9006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.220+0000 7fed94a00640 1 -- 172.21.15.46:0/4106423009 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90072540 con 0x7fed90160bd0 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.221+0000 7fed94a00640 1 --2- 172.21.15.46:0/4106423009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90161060 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6c9533bbbfdf9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.221+0000 7fed8fe00640 1 -- 172.21.15.46:0/4106423009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037070 con 0x7fed90160bd0 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.221+0000 7fed8fe00640 1 -- 172.21.15.46:0/4106423009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed90160bd0 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.221+0000 7fed8fe00640 1 -- 172.21.15.46:0/4106423009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032810 con 0x7fed90160bd0 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 -- 172.21.15.46:0/4106423009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 msgr2=0x7fed90161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 --2- 172.21.15.46:0/4106423009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90161060 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 -- 172.21.15.46:0/4106423009 shutdown_connections 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 --2- 172.21.15.46:0/4106423009 >> v2:172.21.15.142:3300/0 conn(0x7fed900732c0 0x7fed90073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 --2- 172.21.15.46:0/4106423009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90161060 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 -- 172.21.15.46:0/4106423009 >> 172.21.15.46:0/4106423009 conn(0x7fed9006c530 msgr2=0x7fed9006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 -- 172.21.15.46:0/4106423009 shutdown_connections 2024-04-26T23:52:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.222+0000 7fed97e00640 1 -- 172.21.15.46:0/4106423009 wait complete. 2024-04-26T23:52:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.224+0000 7fed97e00640 1 Processor -- start 2024-04-26T23:52:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.224+0000 7fed97e00640 1 -- start start 2024-04-26T23:52:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed9006e510 0x7fed90157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed900732c0 0x7fed90145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed90161720 con 0x7fed90160bd0 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed901615a0 con 0x7fed900732c0 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed97e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed90072d40 con 0x7fed9006e510 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35116/0 (socket says 172.21.15.46:35116) 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.225+0000 7fed95e00640 1 -- 172.21.15.46:0/3639679144 learned_addr learned my addr 172.21.15.46:0/3639679144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed94a00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed900732c0 0x7fed90145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95400640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed9006e510 0x7fed90157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 -- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed9006e510 msgr2=0x7fed90157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed9006e510 0x7fed90157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 -- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed900732c0 msgr2=0x7fed90145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed900732c0 0x7fed90145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 -- 172.21.15.46:0/3639679144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed900b1780 con 0x7fed90160bd0 2024-04-26T23:52:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed95e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fed88007e70 tx=0x7fed8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88014620 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed88019070 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.226+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88014a20 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.227+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed900b1a60 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.227+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed900b2020 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.228+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed90069530 con 0x7fed90160bd0 2024-04-26T23:52:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.229+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed88014bc0 con 0x7fed90160bd0 2024-04-26T23:52:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.230+0000 7fed8d600640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.230+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7fed8808c390 con 0x7fed90160bd0 2024-04-26T23:52:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.230+0000 7fed95400640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.231+0000 7fed95400640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 0x7fed7006ede0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fed84000f60 tx=0x7fed84007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.234+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed88011260 con 0x7fed90160bd0 2024-04-26T23:52:37.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:37.519+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) -- 0x7fed90157f90 con 0x7fed90160bd0 2024-04-26T23:52:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.131+0000 7fed8d600640 1 -- 172.21.15.46:0/3639679144 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v713) ==== 169+0+0 (secure 0 0 0) 0x7fed8805a890 con 0x7fed90160bd0 2024-04-26T23:52:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 msgr2=0x7fed7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 0x7fed7006ede0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fed84000f60 tx=0x7fed84007c00 comp rx=0 tx=0).stop 2024-04-26T23:52:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 msgr2=0x7fed90145690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fed88007e70 tx=0x7fed8800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 shutdown_connections 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed90160bd0 0x7fed90145690 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed900732c0 0x7fed90145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 --2- 172.21.15.46:0/3639679144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed9006e510 0x7fed90157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 >> 172.21.15.46:0/3639679144 conn(0x7fed9006c530 msgr2=0x7fed9006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 shutdown_connections 2024-04-26T23:52:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.134+0000 7fed97e00640 1 -- 172.21.15.46:0/3639679144 wait complete. 2024-04-26T23:52:38.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:52:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.374+0000 7f06d3000640 1 Processor -- start 2024-04-26T23:52:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.374+0000 7f06d3000640 1 -- start start 2024-04-26T23:52:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.376+0000 7f06d3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.376+0000 7f06d3000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f06cc0b4320 0x7f06cc0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.376+0000 7f06d3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06cc0b4b30 con 0x7f06cc0d04c0 2024-04-26T23:52:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.376+0000 7f06d3000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc0b4e10 con 0x7f06cc0b4320 2024-04-26T23:52:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.376+0000 7f06d3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0b50f0 con 0x7f06cc06f4b0 2024-04-26T23:52:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.377+0000 7f06cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.377+0000 7f06d1000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f06cc0b4320 0x7f06cc0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.377+0000 7f06cb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:49132/0 (socket says 172.21.15.46:49132) 2024-04-26T23:52:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.377+0000 7f06cb400640 1 -- 172.21.15.46:0/1267366436 learned_addr learned my addr 172.21.15.46:0/1267366436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.378+0000 7f06cb400640 1 -- 172.21.15.46:0/1267366436 >> v2:172.21.15.142:3300/0 conn(0x7f06cc0b4320 msgr2=0x7f06cc0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.378+0000 7f06cb400640 1 --2- 172.21.15.46:0/1267366436 >> v2:172.21.15.142:3300/0 conn(0x7f06cc0b4320 0x7f06cc0ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.378+0000 7f06cb400640 1 -- 172.21.15.46:0/1267366436 >> v1:172.21.15.142:6789/0 conn(0x7f06cc0d04c0 legacy=0x7f06cc06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.378+0000 7f06cb400640 1 -- 172.21.15.46:0/1267366436 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0b53d0 con 0x7f06cc06f4b0 2024-04-26T23:52:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.379+0000 7f06cb400640 1 --2- 172.21.15.46:0/1267366436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06b400d150 tx=0x7f06b402c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c78c7dd096e8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.379+0000 7f06caa00640 1 -- 172.21.15.46:0/1267366436 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4039070 con 0x7f06cc06f4b0 2024-04-26T23:52:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06caa00640 1 -- 172.21.15.46:0/1267366436 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4009600 con 0x7f06cc06f4b0 2024-04-26T23:52:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06caa00640 1 -- 172.21.15.46:0/1267366436 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40099d0 con 0x7f06cc06f4b0 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 -- 172.21.15.46:0/1267366436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 msgr2=0x7f06cc0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 --2- 172.21.15.46:0/1267366436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06b400d150 tx=0x7f06b402c890 comp rx=0 tx=0).stop 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 -- 172.21.15.46:0/1267366436 shutdown_connections 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 --2- 172.21.15.46:0/1267366436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc0b3de0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 --2- 172.21.15.46:0/1267366436 >> v2:172.21.15.142:3300/0 conn(0x7f06cc0b4320 0x7f06cc0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 -- 172.21.15.46:0/1267366436 >> 172.21.15.46:0/1267366436 conn(0x7f06cc06c980 msgr2=0x7f06cc0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 -- 172.21.15.46:0/1267366436 shutdown_connections 2024-04-26T23:52:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.380+0000 7f06d3000640 1 -- 172.21.15.46:0/1267366436 wait complete. 2024-04-26T23:52:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.383+0000 7f06d3000640 1 Processor -- start 2024-04-26T23:52:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.383+0000 7f06d3000640 1 -- start start 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06cc0d04c0 0x7f06cc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc0b48d0 con 0x7f06cc0b4320 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc0b4750 con 0x7f06cc0d04c0 2024-04-26T23:52:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.384+0000 7f06d3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0ce9b0 con 0x7f06cc06f4b0 2024-04-26T23:52:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35132/0 (socket says 172.21.15.46:35132) 2024-04-26T23:52:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 -- 172.21.15.46:0/3606310147 learned_addr learned my addr 172.21.15.46:0/3606310147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06d1000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06cc0d04c0 0x7f06cc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cbe00640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 -- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 msgr2=0x7f06cc142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 -- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06cc0d04c0 msgr2=0x7f06cc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06cc0d04c0 0x7f06cc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.385+0000 7f06cb400640 1 -- 172.21.15.46:0/3606310147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1e80a0 con 0x7f06cc0b4320 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06cb400640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000f80 tx=0x7f06b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4039040 con 0x7f06cc0b4320 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4009370 con 0x7f06cc0b4320 2024-04-26T23:52:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4034480 con 0x7f06cc0b4320 2024-04-26T23:52:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1e8380 con 0x7f06cc0b4320 2024-04-26T23:52:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.386+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1e8940 con 0x7f06cc0b4320 2024-04-26T23:52:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.388+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc069530 con 0x7f06cc0b4320 2024-04-26T23:52:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.389+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06b4034620 con 0x7f06cc0b4320 2024-04-26T23:52:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.389+0000 7f06c3e00640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.390+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f06b40ac810 con 0x7f06cc0b4320 2024-04-26T23:52:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.390+0000 7f06cbe00640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.392+0000 7f06cbe00640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 0x7f06a806edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007fd0 tx=0x7f06bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.394+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06b4009520 con 0x7f06cc0b4320 2024-04-26T23:52:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.607+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f06cc144c90 con 0x7f06cc0b4320 2024-04-26T23:52:38.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.614+0000 7f06c3e00640 1 -- 172.21.15.46:0/3606310147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f06b407aec0 con 0x7f06cc0b4320 2024-04-26T23:52:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 msgr2=0x7f06a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 0x7f06a806edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007fd0 tx=0x7f06bc009040 comp rx=0 tx=0).stop 2024-04-26T23:52:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 msgr2=0x7f06cc1e75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000f80 tx=0x7f06b4004a40 comp rx=0 tx=0).stop 2024-04-26T23:52:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 shutdown_connections 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06cc0d04c0 0x7f06cc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0b4320 0x7f06cc1e75e0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 --2- 172.21.15.46:0/3606310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06cc06f4b0 0x7f06cc142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.617+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 >> 172.21.15.46:0/3606310147 conn(0x7f06cc06c980 msgr2=0x7f06cc0ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.618+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 shutdown_connections 2024-04-26T23:52:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:38.618+0000 7f06d3000640 1 -- 172.21.15.46:0/3606310147 wait complete. 2024-04-26T23:52:38.642 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:52:38.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:52:38.765 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:52:38.879 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:52:38.880 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:38.880 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:38.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:52:38.940 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:52:38.940 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:38.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:52:38.997 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:38.997 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:39.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:52:39.058 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:52:39.058 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:39.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:52:39.116 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:52:39.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:39.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.333+0000 7f72e4a00640 1 Processor -- start 2024-04-26T23:52:39.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.333+0000 7f72e4a00640 1 -- start start 2024-04-26T23:52:39.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72e4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72e00720e0 0x7f72e00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72e4a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72e0160910 0x7f72e0160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72e4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e0073ad0 con 0x7f72e01466b0 2024-04-26T23:52:39.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72e4a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f72e01612a0 con 0x7f72e00720e0 2024-04-26T23:52:39.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72e4a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f72e0146a90 con 0x7f72e0160910 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72df400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f72e01466b0 0x7f72e006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54748/0 (socket says 172.21.15.46:54748) 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72df400640 1 -- 172.21.15.46:0/4048785066 learned_addr learned my addr 172.21.15.46:0/4048785066 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.335+0000 7f72dea00640 1 --2- 172.21.15.46:0/4048785066 >> v2:172.21.15.46:3300/0 conn(0x7f72e00720e0 0x7f72e00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.336+0000 7f72dfe00640 1 --2- 172.21.15.46:0/4048785066 >> v2:172.21.15.142:3300/0 conn(0x7f72e0160910 0x7f72e0160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.336+0000 7f72dea00640 1 -- 172.21.15.46:0/4048785066 >> v2:172.21.15.142:3300/0 conn(0x7f72e0160910 msgr2=0x7f72e0160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.336+0000 7f72dea00640 1 --2- 172.21.15.46:0/4048785066 >> v2:172.21.15.142:3300/0 conn(0x7f72e0160910 0x7f72e0160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.336+0000 7f72dea00640 1 -- 172.21.15.46:0/4048785066 >> v1:172.21.15.46:6789/0 conn(0x7f72e01466b0 legacy=0x7f72e006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.336+0000 7f72dea00640 1 -- 172.21.15.46:0/4048785066 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e0073d70 con 0x7f72e00720e0 2024-04-26T23:52:39.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.337+0000 7f72dea00640 1 --2- 172.21.15.46:0/4048785066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00720e0 0x7f72e00cd860 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097a0 tx=0x7f72cc02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad2f69bd52dcc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.337+0000 7f72de000640 1 -- 172.21.15.46:0/4048785066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc037070 con 0x7f72e00720e0 2024-04-26T23:52:39.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.337+0000 7f72de000640 1 -- 172.21.15.46:0/4048785066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc032440 con 0x7f72e00720e0 2024-04-26T23:52:39.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.337+0000 7f72de000640 1 -- 172.21.15.46:0/4048785066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc032810 con 0x7f72e00720e0 2024-04-26T23:52:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 -- 172.21.15.46:0/4048785066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00720e0 msgr2=0x7f72e00cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 --2- 172.21.15.46:0/4048785066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00720e0 0x7f72e00cd860 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097a0 tx=0x7f72cc02a1d0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 -- 172.21.15.46:0/4048785066 shutdown_connections 2024-04-26T23:52:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 --2- 172.21.15.46:0/4048785066 >> v2:172.21.15.142:3300/0 conn(0x7f72e0160910 0x7f72e0160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 --2- 172.21.15.46:0/4048785066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00720e0 0x7f72e00cd860 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 -- 172.21.15.46:0/4048785066 >> 172.21.15.46:0/4048785066 conn(0x7f72e00b26d0 msgr2=0x7f72e00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 -- 172.21.15.46:0/4048785066 shutdown_connections 2024-04-26T23:52:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.338+0000 7f72e4a00640 1 -- 172.21.15.46:0/4048785066 wait complete. 2024-04-26T23:52:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.340+0000 7f72e4a00640 1 Processor -- start 2024-04-26T23:52:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.340+0000 7f72e4a00640 1 -- start start 2024-04-26T23:52:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e00720e0 0x7f72e015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72e01466b0 0x7f72e0142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00cb470 con 0x7f72e0160910 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00cb190 con 0x7f72e00720e0 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72e4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72e00cdda0 con 0x7f72e01466b0 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35154/0 (socket says 172.21.15.46:35154) 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.341+0000 7f72dfe00640 1 -- 172.21.15.46:0/767506957 learned_addr learned my addr 172.21.15.46:0/767506957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 -- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72e01466b0 msgr2=0x7f72e0142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72e01466b0 0x7f72e0142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 -- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e00720e0 msgr2=0x7f72e015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e00720e0 0x7f72e015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 -- 172.21.15.46:0/767506957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e0141670 con 0x7f72e0160910 2024-04-26T23:52:39.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72dfe00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f72d800cc30 tx=0x7f72d800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ce40 con 0x7f72e0160910 2024-04-26T23:52:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d8003990 con 0x7f72e0160910 2024-04-26T23:52:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8003d40 con 0x7f72e0160910 2024-04-26T23:52:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.342+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e015d660 con 0x7f72e0160910 2024-04-26T23:52:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.343+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e015dc20 con 0x7f72e0160910 2024-04-26T23:52:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.344+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e00693b0 con 0x7f72e0160910 2024-04-26T23:52:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.345+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72d8003ee0 con 0x7f72e0160910 2024-04-26T23:52:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.346+0000 7f72d7400640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 0x7f72c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.346+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f72d808be60 con 0x7f72e0160910 2024-04-26T23:52:39.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.347+0000 7f72df400640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 0x7f72c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.348+0000 7f72df400640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 0x7f72c006edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f72d0000f60 tx=0x7f72d0007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.351+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72d8014030 con 0x7f72e0160910 2024-04-26T23:52:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.596+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e00c7180 con 0x7f72e0160910 2024-04-26T23:52:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.597+0000 7f72d7400640 1 -- 172.21.15.46:0/767506957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+3881 (secure 0 0 0) 0x7f72d805a400 con 0x7f72e0160910 2024-04-26T23:52:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:39.600 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3194390670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3194390670},{"type":"v1","addr":"172.21.15.142:6837","nonce":3194390670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1739270901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1739270901},{"type":"v1","addr":"172.21.15.142:6836","nonce":1739270901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1800590886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1800590886},{"type":"v1","addr":"172.21.15.46:6834","nonce":1800590886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2847709384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2847709384},{"type":"v1","addr":"172.21.15.46:6835","nonce":2847709384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[]} 2024-04-26T23:52:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 120 2024-04-26T23:52:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 msgr2=0x7f72c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 0x7f72c006edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f72d0000f60 tx=0x7f72d0007a80 comp rx=0 tx=0).stop 2024-04-26T23:52:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 msgr2=0x7f72e0141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f72d800cc30 tx=0x7f72d800a5c0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 shutdown_connections 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f72c006c8f0 0x7f72c006edb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e0160910 0x7f72e0141130 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72e01466b0 0x7f72e0142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 --2- 172.21.15.46:0/767506957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e00720e0 0x7f72e015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 >> 172.21.15.46:0/767506957 conn(0x7f72e00b26d0 msgr2=0x7f72e00c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 shutdown_connections 2024-04-26T23:52:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.600+0000 7f72e4a00640 1 -- 172.21.15.46:0/767506957 wait complete. 2024-04-26T23:52:39.623 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:52:39.624 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:52:39.624 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:52:39.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:52:39.624 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:52:39.624 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:52:39.624 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:52:39.625 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:52:39.625 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:52:39.625 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:52:39.625 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:52:39.625 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:52:39.625 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:52:39.626 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:52:39.626 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:52:39.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:52:39.646 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:52:39.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:52:39.648 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:52:39.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:52:39.650 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:52:39.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:52:39.656 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:52:39.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:52:39.697 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:52:39.697 DEBUG:teuthology.parallel:result is None 2024-04-26T23:52:39.702 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:52:39.703 DEBUG:teuthology.parallel:result is None 2024-04-26T23:52:39.703 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:52:39.703 DEBUG:teuthology.parallel:result is None 2024-04-26T23:52:39.708 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:52:39.708 DEBUG:teuthology.parallel:result is None 2024-04-26T23:52:39.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:52:39.849 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:52:39.851 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:52:39.854 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:52:39.854 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:52:39.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.894+0000 7f3f63200640 1 Processor -- start 2024-04-26T23:52:39.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.894+0000 7f3f63200640 1 -- start start 2024-04-26T23:52:39.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.895+0000 7f3f63200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.895+0000 7f3f63200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c15f980 con 0x7f3f5c0d1bc0 2024-04-26T23:52:39.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.895+0000 7f3f63200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0d20c0 con 0x7f3f5c0ce070 2024-04-26T23:52:39.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.895+0000 7f3f63200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c0cdbe0 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f5c0d1bc0 0x7f3f5c070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54772/0 (socket says 172.21.15.46:54772) 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5b400640 1 -- 172.21.15.46:0/3239527900 learned_addr learned my addr 172.21.15.46:0/3239527900 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5be00640 1 --2- 172.21.15.46:0/3239527900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607447187 0 0) 0x7f3f5c15f980 con 0x7f3f5c0d1bc0 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f40003520 con 0x7f3f5c0d1bc0 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907371840 0 0) 0x7f3f5c0d20c0 con 0x7f3f5c0ce070 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f5c15f980 con 0x7f3f5c0ce070 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5be00640 1 -- 172.21.15.46:0/3239527900 >> v1:172.21.15.142:6789/0 conn(0x7f3f5c0ce070 legacy=0x7f3f5c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5be00640 1 -- 172.21.15.46:0/3239527900 >> v1:172.21.15.46:6789/0 conn(0x7f3f5c0d1bc0 legacy=0x7f3f5c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.896+0000 7f3f5be00640 1 -- 172.21.15.46:0/3239527900 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0d23a0 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.898+0000 7f3f5be00640 1 --2- 172.21.15.46:0/3239527900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c070440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097d0 tx=0x7f3f4c02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e554cc54af160b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.898+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c037070 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.898+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c032440 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.898+0000 7f3f5aa00640 1 -- 172.21.15.46:0/3239527900 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c032810 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 -- 172.21.15.46:0/3239527900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 msgr2=0x7f3f5c070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 --2- 172.21.15.46:0/3239527900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c070440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097d0 tx=0x7f3f4c02a2a0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 -- 172.21.15.46:0/3239527900 shutdown_connections 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 --2- 172.21.15.46:0/3239527900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c070440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 -- 172.21.15.46:0/3239527900 >> 172.21.15.46:0/3239527900 conn(0x7f3f5c0d2f40 msgr2=0x7f3f5c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 -- 172.21.15.46:0/3239527900 shutdown_connections 2024-04-26T23:52:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.899+0000 7f3f63200640 1 -- 172.21.15.46:0/3239527900 wait complete. 2024-04-26T23:52:39.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.901+0000 7f3f63200640 1 Processor -- start 2024-04-26T23:52:39.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.901+0000 7f3f63200640 1 -- start start 2024-04-26T23:52:39.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f5c0d1bc0 0x7f3f5c160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c1607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0d07d0 con 0x7f3f5c0ce070 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0cd870 con 0x7f3f5c0d1bc0 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f63200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c071090 con 0x7f3f5c15f5a0 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35220/0 (socket says 172.21.15.46:35220) 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 -- 172.21.15.46:0/2947385691 learned_addr learned my addr 172.21.15.46:0/2947385691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f61200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c1607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5b400640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f5c0d1bc0 0x7f3f5c160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 -- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 msgr2=0x7f3f5c1607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c1607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 -- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f5c0d1bc0 msgr2=0x7f3f5c160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f5c0d1bc0 0x7f3f5c160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f5be00640 1 -- 172.21.15.46:0/2947385691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c1f59b0 con 0x7f3f5c0ce070 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.902+0000 7f3f61200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f5be00640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0098f0 tx=0x7f3f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c037040 con 0x7f3f5c0ce070 2024-04-26T23:52:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c042070 con 0x7f3f5c0ce070 2024-04-26T23:52:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c03d880 con 0x7f3f5c0ce070 2024-04-26T23:52:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c1f5c90 con 0x7f3f5c0ce070 2024-04-26T23:52:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.903+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1f6250 con 0x7f3f5c0ce070 2024-04-26T23:52:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.904+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c162e30 con 0x7f3f5c0ce070 2024-04-26T23:52:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.905+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f4c03da20 con 0x7f3f5c0ce070 2024-04-26T23:52:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.906+0000 7f3f53e00640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.907+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5335+0+0 (secure 0 0 0) 0x7f3f4c0abff0 con 0x7f3f5c0ce070 2024-04-26T23:52:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.907+0000 7f3f5b400640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.908+0000 7f3f5b400640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c154870 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:39.911+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f4c07a530 con 0x7f3f5c0ce070 2024-04-26T23:52:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.128+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3f5c069530 con 0x7f3f5c0ce070 2024-04-26T23:52:40.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.166+0000 7f3f53e00640 1 -- 172.21.15.46:0/2947385691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v155) ==== 97+0+0 (secure 0 0 0) 0x7f3f4c079e30 con 0x7f3f5c0ce070 2024-04-26T23:52:40.169 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:52:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.170+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 msgr2=0x7f3f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.170+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c154870 tx=0x7f3f48009040 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.170+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 msgr2=0x7f3f5c0b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.170+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0098f0 tx=0x7f3f4c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 shutdown_connections 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f5c15f5a0 0x7f3f5c1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f5c0d1bc0 0x7f3f5c160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 --2- 172.21.15.46:0/2947385691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f5c0ce070 0x7f3f5c0b7e40 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 >> 172.21.15.46:0/2947385691 conn(0x7f3f5c0d2f40 msgr2=0x7f3f5c0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 shutdown_connections 2024-04-26T23:52:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.171+0000 7f3f63200640 1 -- 172.21.15.46:0/2947385691 wait complete. 2024-04-26T23:52:40.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:52:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.396+0000 7fe466000640 1 Processor -- start 2024-04-26T23:52:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.396+0000 7fe466000640 1 -- start start 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe466000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4600cf450 0x7fe460073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe466000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 0x7fe4600cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe466000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46006f470 con 0x7fe4600cfd70 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe466000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe460072930 con 0x7fe4600cf450 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe466000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe460072c10 con 0x7fe460072190 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe465600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe4600cfd70 0x7fe46006e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54782/0 (socket says 172.21.15.46:54782) 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe465600640 1 -- 172.21.15.46:0/699797414 learned_addr learned my addr 172.21.15.46:0/699797414 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.398+0000 7fe464c00640 1 --2- 172.21.15.46:0/699797414 >> v2:172.21.15.46:3300/0 conn(0x7fe4600cf450 0x7fe460073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe45fe00640 1 --2- 172.21.15.46:0/699797414 >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 0x7fe4600cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe45f400640 1 -- 172.21.15.46:0/699797414 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906753640 0 0) 0x7fe46006f470 con 0x7fe4600cfd70 2024-04-26T23:52:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe464c00640 1 -- 172.21.15.46:0/699797414 >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 msgr2=0x7fe4600cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe464c00640 1 --2- 172.21.15.46:0/699797414 >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 0x7fe4600cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe464c00640 1 -- 172.21.15.46:0/699797414 >> v1:172.21.15.46:6789/0 conn(0x7fe4600cfd70 legacy=0x7fe46006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe464c00640 1 -- 172.21.15.46:0/699797414 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe460072ef0 con 0x7fe4600cf450 2024-04-26T23:52:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.399+0000 7fe45fe00640 1 --2- 172.21.15.46:0/699797414 >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 0x7fe4600cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.401+0000 7fe464c00640 1 --2- 172.21.15.46:0/699797414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cf450 0x7fe460073db0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fe4600703e0 tx=0x7fe45402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ae2a1fcfe7e9af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.401+0000 7fe45f400640 1 -- 172.21.15.46:0/699797414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454037070 con 0x7fe4600cf450 2024-04-26T23:52:40.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.401+0000 7fe45f400640 1 -- 172.21.15.46:0/699797414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe454032440 con 0x7fe4600cf450 2024-04-26T23:52:40.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.401+0000 7fe45f400640 1 -- 172.21.15.46:0/699797414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454032810 con 0x7fe4600cf450 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 -- 172.21.15.46:0/699797414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cf450 msgr2=0x7fe460073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 --2- 172.21.15.46:0/699797414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cf450 0x7fe460073db0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fe4600703e0 tx=0x7fe45402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 -- 172.21.15.46:0/699797414 shutdown_connections 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 --2- 172.21.15.46:0/699797414 >> v2:172.21.15.142:3300/0 conn(0x7fe460072190 0x7fe4600cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 --2- 172.21.15.46:0/699797414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cf450 0x7fe460073db0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 -- 172.21.15.46:0/699797414 >> 172.21.15.46:0/699797414 conn(0x7fe4600b2820 msgr2=0x7fe4600b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 -- 172.21.15.46:0/699797414 shutdown_connections 2024-04-26T23:52:40.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.402+0000 7fe466000640 1 -- 172.21.15.46:0/699797414 wait complete. 2024-04-26T23:52:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.404+0000 7fe466000640 1 Processor -- start 2024-04-26T23:52:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.404+0000 7fe466000640 1 -- start start 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe460072190 0x7fe460144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600cf450 0x7fe46013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe460072620 con 0x7fe4600cfd70 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600734b0 con 0x7fe4600cf450 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.405+0000 7fe466000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4600d0180 con 0x7fe460072190 2024-04-26T23:52:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35230/0 (socket says 172.21.15.46:35230) 2024-04-26T23:52:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 -- 172.21.15.46:0/1450476237 learned_addr learned my addr 172.21.15.46:0/1450476237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe45fe00640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600cf450 0x7fe46013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe464c00640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe460072190 0x7fe460144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 -- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe460072190 msgr2=0x7fe460144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe460072190 0x7fe460144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 -- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600cf450 msgr2=0x7fe46013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600cf450 0x7fe46013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 -- 172.21.15.46:0/1450476237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46013fbd0 con 0x7fe4600cfd70 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.406+0000 7fe465600640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c7a0 tx=0x7fe45800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.407+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458018070 con 0x7fe4600cfd70 2024-04-26T23:52:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.407+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45800a640 con 0x7fe4600cfd70 2024-04-26T23:52:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.407+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45800aab0 con 0x7fe4600cfd70 2024-04-26T23:52:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.407+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4601f32c0 con 0x7fe4600cfd70 2024-04-26T23:52:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.407+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601f3880 con 0x7fe4600cfd70 2024-04-26T23:52:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.408+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4600693b0 con 0x7fe4600cfd70 2024-04-26T23:52:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.409+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe45801b070 con 0x7fe4600cfd70 2024-04-26T23:52:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.410+0000 7fe45cc00640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 0x7fe44006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.411+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5249+0+0 (secure 0 0 0) 0x7fe45808afe0 con 0x7fe4600cfd70 2024-04-26T23:52:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.411+0000 7fe464c00640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 0x7fe44006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.412+0000 7fe464c00640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 0x7fe44006edb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe454034000 tx=0x7fe45402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.415+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe46013fbd0 con 0x7fe4600cfd70 2024-04-26T23:52:40.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.695+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe4600b0ba0 con 0x7fe4600cfd70 2024-04-26T23:52:40.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.696+0000 7fe45cc00640 1 -- 172.21.15.46:0/1450476237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fe458059570 con 0x7fe4600cfd70 2024-04-26T23:52:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:40.699 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:52:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:52:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:52:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:52:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:52:40.713 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 msgr2=0x7fe44006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 0x7fe44006edb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe454034000 tx=0x7fe45402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 msgr2=0x7fe4601f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c7a0 tx=0x7fe45800cc60 comp rx=0 tx=0).stop 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 shutdown_connections 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44006c8f0 0x7fe44006edb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4600cfd70 0x7fe4601f2940 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600cf450 0x7fe46013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.699+0000 7fe466000640 1 --2- 172.21.15.46:0/1450476237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe460072190 0x7fe460144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.700+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 >> 172.21.15.46:0/1450476237 conn(0x7fe4600b2820 msgr2=0x7fe4600d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.700+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 shutdown_connections 2024-04-26T23:52:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.700+0000 7fe466000640 1 -- 172.21.15.46:0/1450476237 wait complete. 2024-04-26T23:52:40.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:40.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.912+0000 7f69bdc00640 1 Processor -- start 2024-04-26T23:52:40.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.912+0000 7f69bdc00640 1 -- start start 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f69b806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69bdc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69b806ee30 con 0x7f69b80732c0 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69bdc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69b8161940 con 0x7f69b8160bd0 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69bdc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f69b8071f80 con 0x7f69b806e510 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69b7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f69b80732c0 0x7f69b8073750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54788/0 (socket says 172.21.15.46:54788) 2024-04-26T23:52:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.914+0000 7f69b7400640 1 -- 172.21.15.46:0/4152873229 learned_addr learned my addr 172.21.15.46:0/4152873229 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b6a00640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f69b806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195527188 0 0) 0x7f69b806ee30 con 0x7f69b80732c0 2024-04-26T23:52:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f699c003520 con 0x7f69b80732c0 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1181163184 0 0) 0x7f699c003520 con 0x7f69b80732c0 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 msgr2=0x7f69b806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f69b806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 >> v1:172.21.15.142:6789/0 conn(0x7f69b8160bd0 legacy=0x7f69b8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b8072260 con 0x7f69b80732c0 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f69b00019f0 con 0x7f69b80732c0 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 >> v1:172.21.15.46:6789/0 conn(0x7f69b80732c0 legacy=0x7f69b8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f699c0039e0 0x7f699c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f699c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8071f80 con 0x7f699c004d40 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f699c00ffb0 con 0x7f699c0039e0 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.915+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f699c0101b0 con 0x7f69b806e510 2024-04-26T23:52:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b6000640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f699c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b6a00640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f699c0039e0 0x7f699c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 -- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 msgr2=0x7f699c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b806e510 0x7f699c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 -- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f699c0039e0 msgr2=0x7f699c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f699c0039e0 0x7f699c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.916+0000 7f69b7400640 1 -- 172.21.15.46:0/4152873229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c003520 con 0x7f699c004d40 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.917+0000 7f69b7400640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f69b000a770 tx=0x7f699c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6a39587c10a2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.917+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b000a390 con 0x7f699c004d40 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b002f070 con 0x7f699c004d40 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69b5600640 1 -- 172.21.15.46:0/4152873229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b002bbe0 con 0x7f699c004d40 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 -- 172.21.15.46:0/4152873229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 msgr2=0x7f699c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4152873229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f699c0051c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f69b000a770 tx=0x7f699c005830 comp rx=0 tx=0).stop 2024-04-26T23:52:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69b7400640 1 -- 172.21.15.46:0/4152873229 reap_dead start 2024-04-26T23:52:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 -- 172.21.15.46:0/4152873229 shutdown_connections 2024-04-26T23:52:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 -- 172.21.15.46:0/4152873229 >> 172.21.15.46:0/4152873229 conn(0x7f69b806c530 msgr2=0x7f69b806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 -- 172.21.15.46:0/4152873229 shutdown_connections 2024-04-26T23:52:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.918+0000 7f69bdc00640 1 -- 172.21.15.46:0/4152873229 wait complete. 2024-04-26T23:52:40.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.921+0000 7f69bdc00640 1 Processor -- start 2024-04-26T23:52:40.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.921+0000 7f69bdc00640 1 -- start start 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69b80b20e0 0x7f69b8141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b80b3660 0x7f69b815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8072d40 con 0x7f699c004d40 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8072820 con 0x7f69b80b20e0 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69bdc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f69b8072ec0 con 0x7f69b80b3660 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.922+0000 7f69b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35246/0 (socket says 172.21.15.46:35246) 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 -- 172.21.15.46:0/4223793339 learned_addr learned my addr 172.21.15.46:0/4223793339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6000640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69b80b20e0 0x7f69b8141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b7400640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b80b3660 0x7f69b815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 -- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b80b3660 msgr2=0x7f69b815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b80b3660 0x7f69b815bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 -- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69b80b20e0 msgr2=0x7f69b8141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69b80b20e0 0x7f69b8141780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 -- 172.21.15.46:0/4223793339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b80b25c0 con 0x7f699c004d40 2024-04-26T23:52:40.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.923+0000 7f69b6a00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f69a800f580 tx=0x7f69a800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.924+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8019020 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.924+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a8014870 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.924+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8014c60 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.924+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b80b28a0 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.924+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b80b2e60 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.925+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b8069530 con 0x7f699c004d40 2024-04-26T23:52:40.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.926+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69a801c020 con 0x7f699c004d40 2024-04-26T23:52:40.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.927+0000 7f69aea00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 0x7f699806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:40.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.927+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5249+0+0 (secure 0 0 0) 0x7f69a808c2c0 con 0x7f699c004d40 2024-04-26T23:52:40.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.928+0000 7f69b6000640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 0x7f699806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:40.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.929+0000 7f69b6000640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 0x7f699806edb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007fd0 tx=0x7f69a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:40.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:40.931+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69a8023020 con 0x7f699c004d40 2024-04-26T23:52:41.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.128+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69b80b4a30 con 0x7f699c004d40 2024-04-26T23:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.129+0000 7f69aea00640 1 -- 172.21.15.46:0/4223793339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+13138 (secure 0 0 0) 0x7f69a8026350 con 0x7f699c004d40 2024-04-26T23:52:41.132 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:41.132 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":157,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:52:40.180526+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 msgr2=0x7f699806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 0x7f699806edb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007fd0 tx=0x7f69a4009040 comp rx=0 tx=0).stop 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 msgr2=0x7f69b8141240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f69a800f580 tx=0x7f69a800fa40 comp rx=0 tx=0).stop 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 shutdown_connections 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f699806c8f0 0x7f699806edb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69b80b3660 0x7f69b815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69b80b20e0 0x7f69b8141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 --2- 172.21.15.46:0/4223793339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c004d40 0x7f69b8141240 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 >> 172.21.15.46:0/4223793339 conn(0x7f69b806c530 msgr2=0x7f69b813f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.132+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 shutdown_connections 2024-04-26T23:52:41.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.133+0000 7f69bdc00640 1 -- 172.21.15.46:0/4223793339 wait complete. 2024-04-26T23:52:41.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:52:41.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:52:41.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.359+0000 7fdee7000640 1 Processor -- start 2024-04-26T23:52:41.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.359+0000 7fdee7000640 1 -- start start 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee00cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee7000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee0073d00 0x7fdee00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee7000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00ce7d0 con 0x7fdee00cdb30 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee7000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdee00cdf90 con 0x7fdee0073d00 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00ceab0 con 0x7fdee015cec0 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee0073d00 0x7fdee00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee0073d00 0x7fdee00b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35256/0 (socket says 172.21.15.46:35256) 2024-04-26T23:52:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.361+0000 7fdee6600640 1 -- 172.21.15.46:0/1239217827 learned_addr learned my addr 172.21.15.46:0/1239217827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee5c00640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee00cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee6600640 1 -- 172.21.15.46:0/1239217827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 msgr2=0x7fdee00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee6600640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee00cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee6600640 1 -- 172.21.15.46:0/1239217827 >> v1:172.21.15.142:6789/0 conn(0x7fdee00cdb30 legacy=0x7fdee00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee6600640 1 -- 172.21.15.46:0/1239217827 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00ced90 con 0x7fdee0073d00 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.362+0000 7fdee5c00640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdee6600640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee0073d00 0x7fdee00b3b40 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cb1e0 tx=0x7fded802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c5515239a97ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdedfe00640 1 -- 172.21.15.46:0/1239217827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8036070 con 0x7fdee0073d00 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdedfe00640 1 -- 172.21.15.46:0/1239217827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded8031600 con 0x7fdee0073d00 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdedfe00640 1 -- 172.21.15.46:0/1239217827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80319d0 con 0x7fdee0073d00 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdee7000640 1 -- 172.21.15.46:0/1239217827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee0073d00 msgr2=0x7fdee00b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.363+0000 7fdee7000640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee0073d00 0x7fdee00b3b40 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cb1e0 tx=0x7fded802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 -- 172.21.15.46:0/1239217827 shutdown_connections 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee0073d00 0x7fdee00b3b40 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 --2- 172.21.15.46:0/1239217827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 -- 172.21.15.46:0/1239217827 >> 172.21.15.46:0/1239217827 conn(0x7fdee00b27b0 msgr2=0x7fdee00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 -- 172.21.15.46:0/1239217827 shutdown_connections 2024-04-26T23:52:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.364+0000 7fdee7000640 1 -- 172.21.15.46:0/1239217827 wait complete. 2024-04-26T23:52:41.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.365+0000 7fdee7000640 1 Processor -- start 2024-04-26T23:52:41.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 -- start start 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 0x7fdee00c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdee015d2a0 con 0x7fdee00cdb30 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00cf350 con 0x7fdee0073d00 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.366+0000 7fdee7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00b4080 con 0x7fdee015cec0 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35258/0 (socket says 172.21.15.46:35258) 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 -- 172.21.15.46:0/372261961 learned_addr learned my addr 172.21.15.46:0/372261961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:41.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee6600640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5c00640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 0x7fdee00c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 -- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 msgr2=0x7fdee014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee014ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 -- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 msgr2=0x7fdee00c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 0x7fdee00c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 -- 172.21.15.46:0/372261961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00b0300 con 0x7fdee00cdb30 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee6600640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5c00640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 0x7fdee00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.367+0000 7fdee5200640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c430 tx=0x7fdecc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.368+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011790 con 0x7fdee00cdb30 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.368+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc017070 con 0x7fdee00cdb30 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.368+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee00b05e0 con 0x7fdee00cdb30 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.368+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011bb0 con 0x7fdee00cdb30 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.368+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee014f260 con 0x7fdee00cdb30 2024-04-26T23:52:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.369+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee00693b0 con 0x7fdee00cdb30 2024-04-26T23:52:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.370+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdecc01c460 con 0x7fdee00cdb30 2024-04-26T23:52:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.371+0000 7fdedd600640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 0x7fdec006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.372+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5249+0+0 (secure 0 0 0) 0x7fdecc08b810 con 0x7fdee00cdb30 2024-04-26T23:52:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.372+0000 7fdee5c00640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 0x7fdec006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.373+0000 7fdee5c00640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 0x7fdec006ede0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.376+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdecc059e00 con 0x7fdee00cdb30 2024-04-26T23:52:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.571+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdee00cb250 con 0x7fdee00cdb30 2024-04-26T23:52:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.572+0000 7fdedd600640 1 -- 172.21.15.46:0/372261961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) ==== 93+0+10 (secure 0 0 0) 0x7fdecc059700 con 0x7fdee00cdb30 2024-04-26T23:52:41.574 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 msgr2=0x7fdec006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 0x7fdec006ede0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4008040 comp rx=0 tx=0).stop 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 msgr2=0x7fdee014bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c430 tx=0x7fdecc00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 shutdown_connections 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdec006c920 0x7fdec006ede0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee015cec0 0x7fdee014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee00cdb30 0x7fdee014bc00 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 --2- 172.21.15.46:0/372261961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdee0073d00 0x7fdee00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 >> 172.21.15.46:0/372261961 conn(0x7fdee00b27b0 msgr2=0x7fdee0072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 shutdown_connections 2024-04-26T23:52:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.574+0000 7fdee7000640 1 -- 172.21.15.46:0/372261961 wait complete. 2024-04-26T23:52:41.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:52:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.800+0000 7fadeae00640 1 Processor -- start 2024-04-26T23:52:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.800+0000 7fadeae00640 1 -- start start 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fadeae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fade4073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fadeae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fade40cff40 0x7fade4166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fadeae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fade406f1d0 con 0x7fade4072190 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fadeae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fade406f470 con 0x7fade40cff40 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fadeae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fade4072930 con 0x7fade40cf450 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fade9000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fade4072190 0x7fade40cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54802/0 (socket says 172.21.15.46:54802) 2024-04-26T23:52:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.802+0000 7fade9000640 1 -- 172.21.15.46:0/4267871041 learned_addr learned my addr 172.21.15.46:0/4267871041 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade9a00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fade4073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420756222 0 0) 0x7fade406f1d0 con 0x7fade4072190 2024-04-26T23:52:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fadea400640 1 --2- 172.21.15.46:0/4267871041 >> v2:172.21.15.142:3300/0 conn(0x7fade40cff40 0x7fade4166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadcc003520 con 0x7fade4072190 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 792872496 0 0) 0x7fadcc003520 con 0x7fade4072190 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 msgr2=0x7fade4073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fade4073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 >> v2:172.21.15.142:3300/0 conn(0x7fade40cff40 msgr2=0x7fade4166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 --2- 172.21.15.46:0/4267871041 >> v2:172.21.15.142:3300/0 conn(0x7fade40cff40 0x7fade4166b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4072c10 con 0x7fade4072190 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fadd0001670 con 0x7fade4072190 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 >> v1:172.21.15.46:6789/0 conn(0x7fade4072190 legacy=0x7fade40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fadcc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadcc003f30 0x7fadcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc004d40 0x7fadcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fade406f470 con 0x7fadcc003f30 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fade4072930 con 0x7fadcc004d40 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc00ba10 con 0x7fade40cf450 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.803+0000 7fadea400640 1 --2- 172.21.15.46:0/4267871041 >> v2:172.21.15.142:3300/0 conn(0x7fade40cff40 0x7fade4166b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadcc003f30 0x7fadcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9a00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fadcc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 -- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 msgr2=0x7fadcc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade40cf450 0x7fadcc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 -- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc004d40 msgr2=0x7fadcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc004d40 0x7fadcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.804+0000 7fade9000640 1 -- 172.21.15.46:0/4267871041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc003520 con 0x7fadcc003f30 2024-04-26T23:52:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.805+0000 7fade9000640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadcc003f30 0x7fadcc004800 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c290 tx=0x7fadcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675ff2c37af7e94b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0007d50 con 0x7fadcc003f30 2024-04-26T23:52:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd0005ad0 con 0x7fadcc003f30 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fade3e00640 1 -- 172.21.15.46:0/4267871041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0007070 con 0x7fadcc003f30 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fadeae00640 1 -- 172.21.15.46:0/4267871041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadcc003f30 msgr2=0x7fadcc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fadeae00640 1 --2- 172.21.15.46:0/4267871041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadcc003f30 0x7fadcc004800 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c290 tx=0x7fadcc005830 comp rx=0 tx=0).stop 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fadea400640 1 -- 172.21.15.46:0/4267871041 reap_dead start 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fadeae00640 1 -- 172.21.15.46:0/4267871041 shutdown_connections 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.806+0000 7fadeae00640 1 -- 172.21.15.46:0/4267871041 >> 172.21.15.46:0/4267871041 conn(0x7fade40b2820 msgr2=0x7fade40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.807+0000 7fadeae00640 1 -- 172.21.15.46:0/4267871041 shutdown_connections 2024-04-26T23:52:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.807+0000 7fadeae00640 1 -- 172.21.15.46:0/4267871041 wait complete. 2024-04-26T23:52:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.808+0000 7fadeae00640 1 Processor -- start 2024-04-26T23:52:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.809+0000 7fadeae00640 1 -- start start 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.809+0000 7fadeae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 0x7fade41e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadeae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadeae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 0x7fade41454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadeae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fade40734b0 con 0x7fade40b4680 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadeae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fade40731d0 con 0x7fadcc003f30 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadeae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fade40725f0 con 0x7fade4145050 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35286/0 (socket says 172.21.15.46:35286) 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 -- 172.21.15.46:0/4197027011 learned_addr learned my addr 172.21.15.46:0/4197027011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9a00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 0x7fade41e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 -- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 msgr2=0x7fade41454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadea400640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 0x7fade41454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 0x7fade41454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 -- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 msgr2=0x7fade41e1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 0x7fade41e1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9000640 1 -- 172.21.15.46:0/4197027011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade41e27b0 con 0x7fade40b4680 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fade9a00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 0x7fade41e1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.810+0000 7fadea400640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 0x7fade41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fade9000640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c3b0 tx=0x7fade406fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0042070 con 0x7fade40b4680 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd0007480 con 0x7fade40b4680 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd003e450 con 0x7fade40b4680 2024-04-26T23:52:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41e2a90 con 0x7fade40b4680 2024-04-26T23:52:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.811+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41e3050 con 0x7fade40b4680 2024-04-26T23:52:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.813+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade40693b0 con 0x7fade40b4680 2024-04-26T23:52:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.814+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadd0045070 con 0x7fade40b4680 2024-04-26T23:52:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.814+0000 7fade1600640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 0x7fadc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.815+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5249+0+0 (secure 0 0 0) 0x7fadd00ac590 con 0x7fade40b4680 2024-04-26T23:52:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.815+0000 7fade9a00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 0x7fadc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:41.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.816+0000 7fade9a00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 0x7fadc406edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fadd8006ce0 tx=0x7fadd8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:41.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:41.819+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadd004c030 con 0x7fade40b4680 2024-04-26T23:52:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.027+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fade4069ac0 con 0x7fade40b4680 2024-04-26T23:52:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.028+0000 7fade1600640 1 -- 172.21.15.46:0/4197027011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v157) ==== 84+0+65 (secure 0 0 0) 0x7fadd007aaf0 con 0x7fade40b4680 2024-04-26T23:52:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:52:42.031 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 msgr2=0x7fadc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 0x7fadc406edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fadd8006ce0 tx=0x7fadd8008040 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 msgr2=0x7fade40b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c3b0 tx=0x7fade406fbb0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 shutdown_connections 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fadc406c8f0 0x7fadc406edb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fade4145050 0x7fade41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade40b4680 0x7fade40b4b00 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 --2- 172.21.15.46:0/4197027011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadcc003f30 0x7fade41e1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 >> 172.21.15.46:0/4197027011 conn(0x7fade40b2820 msgr2=0x7fade40b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 shutdown_connections 2024-04-26T23:52:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.031+0000 7fadeae00640 1 -- 172.21.15.46:0/4197027011 wait complete. 2024-04-26T23:52:42.052 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:52:42.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:52:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.266+0000 7f11fb200640 1 Processor -- start 2024-04-26T23:52:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.266+0000 7f11fb200640 1 -- start start 2024-04-26T23:52:42.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.267+0000 7f11fb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11fc15f6e0 0x7f11fc0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.267+0000 7f11fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc0b4500 0x7f11fc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.267+0000 7f11fb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc0b3080 con 0x7f11fc0d1490 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.267+0000 7f11fb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc0b3e60 con 0x7f11fc15f6e0 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.267+0000 7f11fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc0d18c0 con 0x7f11fc0b4500 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11fc15f6e0 0x7f11fc0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11fa800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11fc0d1490 0x7f11fc15fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54576/0 (socket says 172.21.15.46:54576) 2024-04-26T23:52:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11fa800640 1 -- 172.21.15.46:0/941184298 learned_addr learned my addr 172.21.15.46:0/941184298 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9400640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc0b4500 0x7f11fc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9e00640 1 -- 172.21.15.46:0/941184298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc0b4500 msgr2=0x7f11fc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9e00640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc0b4500 0x7f11fc0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9e00640 1 -- 172.21.15.46:0/941184298 >> v1:172.21.15.46:6789/0 conn(0x7f11fc0d1490 legacy=0x7f11fc15fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f9e00640 1 -- 172.21.15.46:0/941184298 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc0b3360 con 0x7f11fc15f6e0 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.268+0000 7f11f8a00640 1 -- 172.21.15.46:0/941184298 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71393287 0 0) 0x7f11fc0b3080 con 0x7f11fc0d1490 2024-04-26T23:52:42.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.269+0000 7f11f9e00640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc15f6e0 0x7f11fc0b3920 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009dc0 tx=0x7f11ec02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3392b3f6fab786b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.270+0000 7f11f8a00640 1 -- 172.21.15.46:0/941184298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec007740 con 0x7f11fc15f6e0 2024-04-26T23:52:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.270+0000 7f11f8a00640 1 -- 172.21.15.46:0/941184298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec0078e0 con 0x7f11fc15f6e0 2024-04-26T23:52:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.270+0000 7f11f8a00640 1 -- 172.21.15.46:0/941184298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec007cb0 con 0x7f11fc15f6e0 2024-04-26T23:52:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.270+0000 7f11fb200640 1 -- 172.21.15.46:0/941184298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc15f6e0 msgr2=0x7f11fc0b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.270+0000 7f11fb200640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc15f6e0 0x7f11fc0b3920 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009dc0 tx=0x7f11ec02a5f0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 -- 172.21.15.46:0/941184298 shutdown_connections 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc0b4500 0x7f11fc0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 --2- 172.21.15.46:0/941184298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc15f6e0 0x7f11fc0b3920 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 -- 172.21.15.46:0/941184298 >> 172.21.15.46:0/941184298 conn(0x7f11fc0b4960 msgr2=0x7f11fc0b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 -- 172.21.15.46:0/941184298 shutdown_connections 2024-04-26T23:52:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.271+0000 7f11fb200640 1 -- 172.21.15.46:0/941184298 wait complete. 2024-04-26T23:52:42.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.272+0000 7f11fb200640 1 Processor -- start 2024-04-26T23:52:42.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.273+0000 7f11fb200640 1 -- start start 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11fc0d1490 0x7f11fc14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 0x7f11fc0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc0b41a0 con 0x7f11fc0b4500 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc0afce0 con 0x7f11fc0d1490 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc0b1210 con 0x7f11fc15f6e0 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50426/0 (socket says 172.21.15.46:50426) 2024-04-26T23:52:42.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 -- 172.21.15.46:0/481005249 learned_addr learned my addr 172.21.15.46:0/481005249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fa800640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 0x7f11fc0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9400640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11fc0d1490 0x7f11fc14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 -- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 msgr2=0x7f11fc0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 0x7f11fc0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 -- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11fc0d1490 msgr2=0x7f11fc14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11fc0d1490 0x7f11fc14ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11f9e00640 1 -- 172.21.15.46:0/481005249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc14e310 con 0x7f11fc0b4500 2024-04-26T23:52:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.274+0000 7f11fa800640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 0x7f11fc0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11f9e00640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009ee0 tx=0x7f11ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec03f070 con 0x7f11fc0b4500 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec031c40 con 0x7f11fc0b4500 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec009450 con 0x7f11fc0b4500 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc0b78a0 con 0x7f11fc0b4500 2024-04-26T23:52:42.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.275+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc0b7e60 con 0x7f11fc0b4500 2024-04-26T23:52:42.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.276+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc06a700 con 0x7f11fc0b4500 2024-04-26T23:52:42.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.277+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11ec043410 con 0x7f11fc0b4500 2024-04-26T23:52:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.278+0000 7f11f2000640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 0x7f11d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:42.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.278+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5249+0+0 (secure 0 0 0) 0x7f11ec0ace30 con 0x7f11fc0b4500 2024-04-26T23:52:42.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.279+0000 7f11f9400640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 0x7f11d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:42.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.280+0000 7f11f9400640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 0x7f11d806edb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f11fc069d00 tx=0x7f11e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.283+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11fc14e310 con 0x7f11fc0b4500 2024-04-26T23:52:42.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:42.478+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f11fc0b23d0 con 0x7f11fc0b4500 2024-04-26T23:52:42.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:42.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:43.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.182+0000 7f11f2000640 1 -- 172.21.15.46:0/481005249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v158) ==== 138+0+0 (secure 0 0 0) 0x7f11ec07b380 con 0x7f11fc0b4500 2024-04-26T23:52:43.189 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:52:43.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 msgr2=0x7f11d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 0x7f11d806edb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f11fc069d00 tx=0x7f11e4008040 comp rx=0 tx=0).stop 2024-04-26T23:52:43.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 msgr2=0x7f11fc14ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009ee0 tx=0x7f11ec004a40 comp rx=0 tx=0).stop 2024-04-26T23:52:43.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 shutdown_connections 2024-04-26T23:52:43.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f11d806c8f0 0x7f11d806edb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11fc15f6e0 0x7f11fc0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11fc0d1490 0x7f11fc14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 --2- 172.21.15.46:0/481005249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11fc0b4500 0x7f11fc14ab70 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.187+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 >> 172.21.15.46:0/481005249 conn(0x7f11fc0b4960 msgr2=0x7f11fc06dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:43.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.188+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 shutdown_connections 2024-04-26T23:52:43.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.189+0000 7f11fb200640 1 -- 172.21.15.46:0/481005249 wait complete. 2024-04-26T23:52:43.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:52:43.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.435+0000 7f0ed3000640 1 Processor -- start 2024-04-26T23:52:43.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.435+0000 7f0ed3000640 1 -- start start 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed3000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ecc0b4b50 0x7f0ecc073d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc0ce7d0 con 0x7f0ecc15cec0 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc0ceab0 con 0x7f0ecc0cdd00 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed3000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0ecc0ced90 con 0x7f0ecc0b4b50 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed1c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ecc15cec0 0x7f0ecc0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54578/0 (socket says 172.21.15.46:54578) 2024-04-26T23:52:43.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.437+0000 7f0ed1c00640 1 -- 172.21.15.46:0/3697147243 learned_addr learned my addr 172.21.15.46:0/3697147243 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:43.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ed2600640 1 --2- 172.21.15.46:0/3697147243 >> v2:172.21.15.142:3300/0 conn(0x7f0ecc0b4b50 0x7f0ecc073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520440627 0 0) 0x7f0ecc0ce7d0 con 0x7f0ecc15cec0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb4003520 con 0x7f0ecc15cec0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2999038150 0 0) 0x7f0eb4003520 con 0x7f0ecc15cec0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 >> v2:172.21.15.142:3300/0 conn(0x7f0ecc0b4b50 msgr2=0x7f0ecc073d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 --2- 172.21.15.46:0/3697147243 >> v2:172.21.15.142:3300/0 conn(0x7f0ecc0b4b50 0x7f0ecc073d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 >> v1:172.21.15.142:6789/0 conn(0x7f0ecc0cdd00 legacy=0x7f0ecc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc0cf070 con 0x7f0ecc15cec0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0ec0001690 con 0x7f0ecc15cec0 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 >> v1:172.21.15.46:6789/0 conn(0x7f0ecc15cec0 legacy=0x7f0ecc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0eb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.438+0000 7f0ecbe00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb4004800 0x7f0eb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ecbe00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0eb40051c0 0x7f0eb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc0ced90 con 0x7f0eb40039b0 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb400c060 con 0x7f0eb4004800 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb400c260 con 0x7f0eb40051c0 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0eb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed2600640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0eb40051c0 0x7f0eb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 -- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0eb40051c0 msgr2=0x7f0eb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1200640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb4004800 0x7f0eb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0eb40051c0 0x7f0eb400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 -- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb4004800 msgr2=0x7f0eb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:43.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb4004800 0x7f0eb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1c00640 1 -- 172.21.15.46:0/3697147243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb4003520 con 0x7f0eb40039b0 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed1200640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb4004800 0x7f0eb4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.439+0000 7f0ed2600640 1 -- 172.21.15.46:0/3697147243 reap_dead start 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.440+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0eb4003dd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c210 tx=0x7f0eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d0216ea7468ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0007d10 con 0x7f0eb40039b0 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec0005af0 con 0x7f0eb40039b0 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ecbe00640 1 -- 172.21.15.46:0/3697147243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0007070 con 0x7f0eb40039b0 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ed3000640 1 -- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 msgr2=0x7f0eb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ed3000640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0eb4003dd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c210 tx=0x7f0eb4005830 comp rx=0 tx=0).stop 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ed3000640 1 -- 172.21.15.46:0/3697147243 shutdown_connections 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ed3000640 1 --2- 172.21.15.46:0/3697147243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0eb4003dd0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.441+0000 7f0ed3000640 1 -- 172.21.15.46:0/3697147243 >> 172.21.15.46:0/3697147243 conn(0x7f0ecc0b27b0 msgr2=0x7f0ecc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.442+0000 7f0ed3000640 1 -- 172.21.15.46:0/3697147243 shutdown_connections 2024-04-26T23:52:43.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.442+0000 7f0ed3000640 1 -- 172.21.15.46:0/3697147243 wait complete. 2024-04-26T23:52:43.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.443+0000 7f0ed3000640 1 Processor -- start 2024-04-26T23:52:43.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.443+0000 7f0ed3000640 1 -- start start 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 0x7f0ecc0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ecc0b5ad0 0x7f0ecc0c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc0cd5f0 con 0x7f0eb40039b0 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc15d2a0 con 0x7f0ecc151cf0 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.444+0000 7f0ed3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc0cd770 con 0x7f0ecc0b5ad0 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50438/0 (socket says 172.21.15.46:50438) 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 -- 172.21.15.46:0/892094721 learned_addr learned my addr 172.21.15.46:0/892094721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed2600640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ecc0b5ad0 0x7f0ecc0c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1200640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 0x7f0ecc0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 -- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ecc0b5ad0 msgr2=0x7f0ecc0c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ecc0b5ad0 0x7f0ecc0c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 -- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 msgr2=0x7f0ecc0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 0x7f0ecc0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1c00640 1 -- 172.21.15.46:0/892094721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc0c7580 con 0x7f0eb40039b0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.445+0000 7f0ed1200640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 0x7f0ecc0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ed1c00640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c210 tx=0x7f0ecc0cec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0037040 con 0x7f0eb40039b0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec00047e0 con 0x7f0eb40039b0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0001c50 con 0x7f0eb40039b0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc14e0d0 con 0x7f0eb40039b0 2024-04-26T23:52:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.446+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc14e690 con 0x7f0eb40039b0 2024-04-26T23:52:43.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.447+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc0b5ee0 con 0x7f0eb40039b0 2024-04-26T23:52:43.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.449+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ec0004b90 con 0x7f0eb40039b0 2024-04-26T23:52:43.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.450+0000 7f0ec9600640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 0x7f0eac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:43.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.450+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5619+0+0 (secure 0 0 0) 0x7f0ec00abf90 con 0x7f0eb40039b0 2024-04-26T23:52:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.450+0000 7f0ed1200640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 0x7f0eac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:43.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.452+0000 7f0ed1200640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 0x7f0eac06edb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8007fd0 tx=0x7f0eb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:43.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.454+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ecc0b5ee0 con 0x7f0eb40039b0 2024-04-26T23:52:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:43.661+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ecc0693e0 con 0x7f0eb40039b0 2024-04-26T23:52:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.199+0000 7f0ec9600640 1 -- 172.21.15.46:0/892094721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v159) ==== 170+0+0 (secure 0 0 0) 0x7f0ecc0693e0 con 0x7f0eb40039b0 2024-04-26T23:52:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:52:44.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 msgr2=0x7f0eac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 0x7f0eac06edb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8007fd0 tx=0x7f0eb8009040 comp rx=0 tx=0).stop 2024-04-26T23:52:44.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 msgr2=0x7f0ecc0c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c210 tx=0x7f0ecc0cec80 comp rx=0 tx=0).stop 2024-04-26T23:52:44.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 shutdown_connections 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0eac06c8f0 0x7f0eac06edb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.204+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ecc0b5ad0 0x7f0ecc0c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.205+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ecc151cf0 0x7f0ecc0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.205+0000 7f0ed3000640 1 --2- 172.21.15.46:0/892094721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb40039b0 0x7f0ecc0c7040 secure :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c210 tx=0x7f0ecc0cec80 comp rx=0 tx=0).stop 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.205+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 >> 172.21.15.46:0/892094721 conn(0x7f0ecc0b27b0 msgr2=0x7f0ecc073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.206+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 shutdown_connections 2024-04-26T23:52:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.207+0000 7f0ed3000640 1 -- 172.21.15.46:0/892094721 wait complete. 2024-04-26T23:52:44.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:52:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.426+0000 7f4e7aa00640 1 Processor -- start 2024-04-26T23:52:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.426+0000 7f4e7aa00640 1 -- start start 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.428+0000 7f4e7aa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e74160130 0x7f4e74160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.428+0000 7f4e7aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 0x7f4e740cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.428+0000 7f4e7aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e740cbe60 con 0x7f4e7415d0a0 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.428+0000 7f4e7aa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4e740cca70 con 0x7f4e74160130 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.428+0000 7f4e7aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e740cc140 con 0x7f4e7415cc40 2024-04-26T23:52:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.429+0000 7f4e78c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 0x7f4e740cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.429+0000 7f4e79600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e74160130 0x7f4e74160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.429+0000 7f4e78c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 0x7f4e740cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:40828/0 (socket says 172.21.15.46:40828) 2024-04-26T23:52:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.429+0000 7f4e78c00640 1 -- 172.21.15.46:0/3336951771 learned_addr learned my addr 172.21.15.46:0/3336951771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.430+0000 7f4e79600640 1 -- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 msgr2=0x7f4e740cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.430+0000 7f4e79600640 1 --2- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 0x7f4e740cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.430+0000 7f4e79600640 1 -- 172.21.15.46:0/3336951771 >> v1:172.21.15.142:6789/0 conn(0x7f4e7415d0a0 legacy=0x7f4e740cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.430+0000 7f4e79600640 1 -- 172.21.15.46:0/3336951771 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e740cc420 con 0x7f4e74160130 2024-04-26T23:52:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.431+0000 7f4e79600640 1 --2- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e74160130 0x7f4e74160510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097a0 tx=0x7f4e6802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469c768cf9e5bac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.432+0000 7f4e73e00640 1 -- 172.21.15.46:0/3336951771 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68037070 con 0x7f4e74160130 2024-04-26T23:52:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.432+0000 7f4e73e00640 1 -- 172.21.15.46:0/3336951771 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6802ae50 con 0x7f4e74160130 2024-04-26T23:52:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.432+0000 7f4e73e00640 1 -- 172.21.15.46:0/3336951771 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68032660 con 0x7f4e74160130 2024-04-26T23:52:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.432+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e74160130 msgr2=0x7f4e74160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.432+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e74160130 0x7f4e74160510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097a0 tx=0x7f4e6802a100 comp rx=0 tx=0).stop 2024-04-26T23:52:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3336951771 shutdown_connections 2024-04-26T23:52:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e7415cc40 0x7f4e740cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3336951771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e74160130 0x7f4e74160510 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3336951771 >> 172.21.15.46:0/3336951771 conn(0x7f4e740b27b0 msgr2=0x7f4e740b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3336951771 shutdown_connections 2024-04-26T23:52:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.433+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3336951771 wait complete. 2024-04-26T23:52:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.434+0000 7f4e7aa00640 1 Processor -- start 2024-04-26T23:52:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 -- start start 2024-04-26T23:52:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e7415d0a0 0x7f4e7414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e74160130 0x7f4e7406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e740733b0 con 0x7f4e7415cc40 2024-04-26T23:52:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74073230 con 0x7f4e7415d0a0 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.435+0000 7f4e7aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e74073530 con 0x7f4e74160130 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50454/0 (socket says 172.21.15.46:50454) 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 -- 172.21.15.46:0/3616244090 learned_addr learned my addr 172.21.15.46:0/3616244090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e78c00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e7415d0a0 0x7f4e7414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e7a000640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e74160130 0x7f4e7406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 -- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e74160130 msgr2=0x7f4e7406e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e74160130 0x7f4e7406e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 -- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e7415d0a0 msgr2=0x7f4e7414d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e7415d0a0 0x7f4e7414d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.436+0000 7f4e79600640 1 -- 172.21.15.46:0/3616244090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e740c6430 con 0x7f4e7415cc40 2024-04-26T23:52:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e79600640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802a140 tx=0x7f4e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68041070 con 0x7f4e7415cc40 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6803c670 con 0x7f4e7415cc40 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68037040 con 0x7f4e7415cc40 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e740c5390 con 0x7f4e7415cc40 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.437+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e740c5950 con 0x7f4e7415cc40 2024-04-26T23:52:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.438+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e740693b0 con 0x7f4e7415cc40 2024-04-26T23:52:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.440+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e68044070 con 0x7f4e7415cc40 2024-04-26T23:52:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.441+0000 7f4e71600640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.441+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 6001+0+0 (secure 0 0 0) 0x7f4e680ac570 con 0x7f4e7415cc40 2024-04-26T23:52:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.442+0000 7f4e78c00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.443+0000 7f4e78c00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 0x7f4e5406edb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4e740cf4e0 tx=0x7f4e60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.445+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e6807a970 con 0x7f4e7415cc40 2024-04-26T23:52:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:44.689+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e740cbb10 con 0x7f4e7415cc40 2024-04-26T23:52:45.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.230+0000 7f4e71600640 1 -- 172.21.15.46:0/3616244090 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v123) ==== 174+0+0 (secure 0 0 0) 0x7f4e6807a270 con 0x7f4e7415cc40 2024-04-26T23:52:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 23 and data pool 24 2024-04-26T23:52:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.239+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 msgr2=0x7f4e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.239+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 0x7f4e5406edb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4e740cf4e0 tx=0x7f4e60008040 comp rx=0 tx=0).stop 2024-04-26T23:52:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.239+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 msgr2=0x7f4e7406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.239+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802a140 tx=0x7f4e680046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 shutdown_connections 2024-04-26T23:52:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e74160130 0x7f4e7406e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e7415d0a0 0x7f4e7414d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 --2- 172.21.15.46:0/3616244090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e7415cc40 0x7f4e7406fb10 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.241+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 >> 172.21.15.46:0/3616244090 conn(0x7f4e740b27b0 msgr2=0x7f4e7415d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.242+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 shutdown_connections 2024-04-26T23:52:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.242+0000 7f4e7aa00640 1 -- 172.21.15.46:0/3616244090 wait complete. 2024-04-26T23:52:45.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:45.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.446+0000 7f5fc1c00640 1 Processor -- start 2024-04-26T23:52:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.446+0000 7f5fc1c00640 1 -- start start 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.447+0000 7f5fc1c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5fbc142280 0x7f5fbc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.447+0000 7f5fc1c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fbc0b43f0 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.447+0000 7f5fc1c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fbc0b46d0 con 0x7f5fbc06ea50 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.447+0000 7f5fc1c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc06eea0 con 0x7f5fbc142280 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.448+0000 7f5fb3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5fbc0b3e80 0x7f5fbc144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54586/0 (socket says 172.21.15.46:54586) 2024-04-26T23:52:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.448+0000 7f5fb3e00640 1 -- 172.21.15.46:0/1674625840 learned_addr learned my addr 172.21.15.46:0/1674625840 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:45.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.448+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431902889 0 0) 0x7f5fbc0b43f0 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.448+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f9c003520 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.448+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1674625840 >> v2:172.21.15.142:3300/0 conn(0x7f5fbc142280 0x7f5fbc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1751973766 0 0) 0x7f5f9c003520 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 >> v2:172.21.15.142:3300/0 conn(0x7f5fbc142280 msgr2=0x7f5fbc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 --2- 172.21.15.46:0/1674625840 >> v2:172.21.15.142:3300/0 conn(0x7f5fbc142280 0x7f5fbc06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 >> v1:172.21.15.142:6789/0 conn(0x7f5fbc06ea50 legacy=0x7f5fbc144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc0b49b0 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5fa4001670 con 0x7f5fbc0b3e80 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 >> v1:172.21.15.46:6789/0 conn(0x7f5fbc0b3e80 legacy=0x7f5fbc144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5f9c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f9c004800 0x7f5f9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c0051c0 0x7f5f9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc06eea0 con 0x7f5f9c0039b0 2024-04-26T23:52:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c00c0f0 con 0x7f5f9c0051c0 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c00c2f0 con 0x7f5f9c004800 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5f9c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f9c004800 msgr2=0x7f5f9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f9c004800 0x7f5f9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.449+0000 7f5fbb400640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c0051c0 0x7f5f9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.450+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c0051c0 msgr2=0x7f5f9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.450+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c0051c0 0x7f5f9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.450+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1674625840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c003520 con 0x7f5f9c0039b0 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.450+0000 7f5fbb400640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c0051c0 0x7f5f9c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.450+0000 7f5fbb400640 1 -- 172.21.15.46:0/1674625840 reap_dead start 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5f9c003db0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00ba00 tx=0x7f5f9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b881ae31b42d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac030070 con 0x7f5f9c0039b0 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fac02adb0 con 0x7f5f9c0039b0 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fba000640 1 -- 172.21.15.46:0/1674625840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00b600 con 0x7f5f9c0039b0 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fc1c00640 1 -- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 msgr2=0x7f5f9c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5f9c003db0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00ba00 tx=0x7f5f9c005830 comp rx=0 tx=0).stop 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fc1c00640 1 -- 172.21.15.46:0/1674625840 shutdown_connections 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/1674625840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5f9c003db0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.451+0000 7f5fc1c00640 1 -- 172.21.15.46:0/1674625840 >> 172.21.15.46:0/1674625840 conn(0x7f5fbc06c530 msgr2=0x7f5fbc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.452+0000 7f5fc1c00640 1 -- 172.21.15.46:0/1674625840 shutdown_connections 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.452+0000 7f5fc1c00640 1 -- 172.21.15.46:0/1674625840 wait complete. 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.453+0000 7f5fc1c00640 1 Processor -- start 2024-04-26T23:52:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.453+0000 7f5fc1c00640 1 -- start start 2024-04-26T23:52:45.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5fbc1596a0 0x7f5fbc1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fbc1f5770 0x7f5fbc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc1428a0 con 0x7f5f9c0039b0 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc0b4f70 con 0x7f5fbc1f5770 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fc1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc142a20 con 0x7f5fbc1596a0 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.454+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50462/0 (socket says 172.21.15.46:50462) 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 -- 172.21.15.46:0/3750499974 learned_addr learned my addr 172.21.15.46:0/3750499974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 -- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5fbc1596a0 msgr2=0x7f5fbc1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5fbc1596a0 0x7f5fbc1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 -- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fbc1f5770 msgr2=0x7f5fbc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fbc1f5770 0x7f5fbc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 -- 172.21.15.46:0/3750499974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc1f5c50 con 0x7f5f9c0039b0 2024-04-26T23:52:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00baf0 tx=0x7f5fac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.455+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac043020 con 0x7f5f9c0039b0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.456+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fac03f680 con 0x7f5f9c0039b0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.456+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00b600 con 0x7f5f9c0039b0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.456+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc1f5f30 con 0x7f5f9c0039b0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.456+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc15a320 con 0x7f5f9c0039b0 2024-04-26T23:52:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.457+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc069530 con 0x7f5f9c0039b0 2024-04-26T23:52:45.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.458+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fac046020 con 0x7f5f9c0039b0 2024-04-26T23:52:45.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.459+0000 7f5fb2a00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 0x7f5f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.459+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6101+0+0 (secure 0 0 0) 0x7f5fac0ac830 con 0x7f5f9c0039b0 2024-04-26T23:52:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.460+0000 7f5fb3e00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 0x7f5f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.461+0000 7f5fb3e00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 0x7f5f9806edb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0cedf0 tx=0x7f5fa4001540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.464+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fac04d030 con 0x7f5f9c0039b0 2024-04-26T23:52:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.668+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fbc160750 con 0x7f5f9c0039b0 2024-04-26T23:52:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.672+0000 7f5fb2a00640 1 -- 172.21.15.46:0/3750499974 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16861 (secure 0 0 0) 0x7f5fac07aad0 con 0x7f5f9c0039b0 2024-04-26T23:52:45.676 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:45.676 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":160,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:52:44.692289+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.674+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 msgr2=0x7f5f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.674+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 0x7f5f9806edb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0cedf0 tx=0x7f5fa4001540 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.674+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 msgr2=0x7f5fbc159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.674+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00baf0 tx=0x7f5fac004730 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 shutdown_connections 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5f9806c8f0 0x7f5f9806edb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fbc1f5770 0x7f5fbc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5fbc1596a0 0x7f5fbc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 --2- 172.21.15.46:0/3750499974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9c0039b0 0x7f5fbc159030 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 >> 172.21.15.46:0/3750499974 conn(0x7f5fbc06c530 msgr2=0x7f5fbc143e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 shutdown_connections 2024-04-26T23:52:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.675+0000 7f5fc1c00640 1 -- 172.21.15.46:0/3750499974 wait complete. 2024-04-26T23:52:45.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.905+0000 7fd080a00640 1 Processor -- start 2024-04-26T23:52:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.905+0000 7fd080a00640 1 -- start start 2024-04-26T23:52:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.906+0000 7fd080a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd07c06e510 0x7fd07c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.906+0000 7fd080a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c160bd0 0x7fd07c161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.906+0000 7fd080a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07c06ee30 con 0x7fd07c15d540 2024-04-26T23:52:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.906+0000 7fd080a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd07c0742d0 con 0x7fd07c06e510 2024-04-26T23:52:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.906+0000 7fd080a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c161940 con 0x7fd07c160bd0 2024-04-26T23:52:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.907+0000 7fd079400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c160bd0 0x7fd07c161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.907+0000 7fd079e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd07c06e510 0x7fd07c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.907+0000 7fd079e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd07c06e510 0x7fd07c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:41772/0 (socket says 172.21.15.46:41772) 2024-04-26T23:52:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.907+0000 7fd079e00640 1 -- 172.21.15.46:0/196501663 learned_addr learned my addr 172.21.15.46:0/196501663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.908+0000 7fd079e00640 1 -- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c160bd0 msgr2=0x7fd07c161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.908+0000 7fd079e00640 1 --2- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c160bd0 0x7fd07c161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.908+0000 7fd079e00640 1 -- 172.21.15.46:0/196501663 >> v1:172.21.15.142:6789/0 conn(0x7fd07c15d540 legacy=0x7fd07c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.908+0000 7fd079e00640 1 -- 172.21.15.46:0/196501663 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c071f80 con 0x7fd07c06e510 2024-04-26T23:52:45.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.909+0000 7fd079e00640 1 --2- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c06e510 0x7fd07c06e8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50f2ada29501b21a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd078a00640 1 -- 172.21.15.46:0/196501663 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c037070 con 0x7fd07c06e510 2024-04-26T23:52:45.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd078a00640 1 -- 172.21.15.46:0/196501663 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c032440 con 0x7fd07c06e510 2024-04-26T23:52:45.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd078a00640 1 -- 172.21.15.46:0/196501663 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c032810 con 0x7fd07c06e510 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 -- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c06e510 msgr2=0x7fd07c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 --2- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c06e510 0x7fd07c06e8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c02a250 comp rx=0 tx=0).stop 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 -- 172.21.15.46:0/196501663 shutdown_connections 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 --2- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c160bd0 0x7fd07c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 --2- 172.21.15.46:0/196501663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c06e510 0x7fd07c06e8f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 -- 172.21.15.46:0/196501663 >> 172.21.15.46:0/196501663 conn(0x7fd07c06c530 msgr2=0x7fd07c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 -- 172.21.15.46:0/196501663 shutdown_connections 2024-04-26T23:52:45.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.910+0000 7fd080a00640 1 -- 172.21.15.46:0/196501663 wait complete. 2024-04-26T23:52:45.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.912+0000 7fd080a00640 1 Processor -- start 2024-04-26T23:52:45.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.912+0000 7fd080a00640 1 -- start start 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 0x7fd07c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c160bd0 0x7fd07c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c072820 con 0x7fd07c15d540 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c072540 con 0x7fd07c160bd0 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd080a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c161570 con 0x7fd07c06e510 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.913+0000 7fd079400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50468/0 (socket says 172.21.15.46:50468) 2024-04-26T23:52:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 -- 172.21.15.46:0/1727651307 learned_addr learned my addr 172.21.15.46:0/1727651307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd07a800640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c160bd0 0x7fd07c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079e00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 0x7fd07c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 -- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 msgr2=0x7fd07c1430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 0x7fd07c1430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 -- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c160bd0 msgr2=0x7fd07c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c160bd0 0x7fd07c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 -- 172.21.15.46:0/1727651307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c143dc0 con 0x7fd07c15d540 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079e00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 0x7fd07c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.914+0000 7fd079400640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fd06400e870 tx=0x7fd06400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.915+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064017070 con 0x7fd07c15d540 2024-04-26T23:52:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.915+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06400c940 con 0x7fd07c15d540 2024-04-26T23:52:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.915+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c12ff80 con 0x7fd07c15d540 2024-04-26T23:52:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.915+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06400cd30 con 0x7fd07c15d540 2024-04-26T23:52:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.915+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c130540 con 0x7fd07c15d540 2024-04-26T23:52:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.916+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07c0b56b0 con 0x7fd07c15d540 2024-04-26T23:52:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.917+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd06401a070 con 0x7fd07c15d540 2024-04-26T23:52:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.918+0000 7fd072000640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.918+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6101+0+0 (secure 0 0 0) 0x7fd06408bf80 con 0x7fd07c15d540 2024-04-26T23:52:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.919+0000 7fd079e00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.920+0000 7fd079e00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 0x7fd05806edb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd06c034000 tx=0x7fd06c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:45.923+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd07c0b56b0 con 0x7fd07c15d540 2024-04-26T23:52:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.128+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd07c069530 con 0x7fd07c15d540 2024-04-26T23:52:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.130+0000 7fd072000640 1 -- 172.21.15.46:0/1727651307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16861 (secure 0 0 0) 0x7fd06405a180 con 0x7fd07c15d540 2024-04-26T23:52:46.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:46.134 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":160,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:52:44.692289+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.132+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 msgr2=0x7fd05806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.132+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 0x7fd05806edb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd06c034000 tx=0x7fd06c02e000 comp rx=0 tx=0).stop 2024-04-26T23:52:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.132+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 msgr2=0x7fd07c143730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.132+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fd06400e870 tx=0x7fd06400ed30 comp rx=0 tx=0).stop 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 shutdown_connections 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c160bd0 0x7fd07c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd07c15d540 0x7fd07c143730 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 --2- 172.21.15.46:0/1727651307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c06e510 0x7fd07c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 >> 172.21.15.46:0/1727651307 conn(0x7fd07c06c530 msgr2=0x7fd07c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 shutdown_connections 2024-04-26T23:52:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.133+0000 7fd080a00640 1 -- 172.21.15.46:0/1727651307 wait complete. 2024-04-26T23:52:46.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:52:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.367+0000 7f83ca200640 1 Processor -- start 2024-04-26T23:52:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.368+0000 7f83ca200640 1 -- start start 2024-04-26T23:52:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.369+0000 7f83ca200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.369+0000 7f83ca200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 0x7f83c406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.369+0000 7f83ca200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c4072880 con 0x7f83c415dd60 2024-04-26T23:52:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.369+0000 7f83ca200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f83c415e140 con 0x7f83c40723e0 2024-04-26T23:52:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.369+0000 7f83ca200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83c4072b40 con 0x7f83c4071f80 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.370+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.370+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:40904/0 (socket says 172.21.15.46:40904) 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.370+0000 7f83bbe00640 1 -- 172.21.15.46:0/4271046593 learned_addr learned my addr 172.21.15.46:0/4271046593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.370+0000 7f83c3e00640 1 --2- 172.21.15.46:0/4271046593 >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 0x7f83c406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.371+0000 7f83bbe00640 1 -- 172.21.15.46:0/4271046593 >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 msgr2=0x7f83c406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.371+0000 7f83bbe00640 1 --2- 172.21.15.46:0/4271046593 >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 0x7f83c406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.371+0000 7f83bbe00640 1 -- 172.21.15.46:0/4271046593 >> v1:172.21.15.142:6789/0 conn(0x7f83c415dd60 legacy=0x7f83c40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.371+0000 7f83bbe00640 1 -- 172.21.15.46:0/4271046593 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4072e20 con 0x7f83c4071f80 2024-04-26T23:52:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.371+0000 7f83c3e00640 1 --2- 172.21.15.46:0/4271046593 >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 0x7f83c406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.372+0000 7f83bbe00640 1 --2- 172.21.15.46:0/4271046593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009870 tx=0x7f83ac02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8363dd4615456a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.373+0000 7f83c2a00640 1 -- 172.21.15.46:0/4271046593 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac037070 con 0x7f83c4071f80 2024-04-26T23:52:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.373+0000 7f83c2a00640 1 -- 172.21.15.46:0/4271046593 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ac032440 con 0x7f83c4071f80 2024-04-26T23:52:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.373+0000 7f83c2a00640 1 -- 172.21.15.46:0/4271046593 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac032810 con 0x7f83c4071f80 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 -- 172.21.15.46:0/4271046593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 msgr2=0x7f83c4163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 --2- 172.21.15.46:0/4271046593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009870 tx=0x7f83ac02a250 comp rx=0 tx=0).stop 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 -- 172.21.15.46:0/4271046593 shutdown_connections 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 --2- 172.21.15.46:0/4271046593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c4071f80 0x7f83c4163bc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 --2- 172.21.15.46:0/4271046593 >> v2:172.21.15.142:3300/0 conn(0x7f83c40723e0 0x7f83c406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 -- 172.21.15.46:0/4271046593 >> 172.21.15.46:0/4271046593 conn(0x7f83c406c980 msgr2=0x7f83c40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 -- 172.21.15.46:0/4271046593 shutdown_connections 2024-04-26T23:52:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.374+0000 7f83ca200640 1 -- 172.21.15.46:0/4271046593 wait complete. 2024-04-26T23:52:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.376+0000 7f83ca200640 1 Processor -- start 2024-04-26T23:52:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.376+0000 7f83ca200640 1 -- start start 2024-04-26T23:52:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.377+0000 7f83ca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.377+0000 7f83ca200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40723e0 0x7f83c41609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83ca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 0x7f83c406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83ca200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b42c0 con 0x7f83c4071f80 2024-04-26T23:52:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83ca200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40733e0 con 0x7f83c415dd60 2024-04-26T23:52:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83ca200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40b4440 con 0x7f83c40723e0 2024-04-26T23:52:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50474/0 (socket says 172.21.15.46:50474) 2024-04-26T23:52:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 -- 172.21.15.46:0/1984444104 learned_addr learned my addr 172.21.15.46:0/1984444104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40723e0 0x7f83c41609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3e00640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 0x7f83c406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 -- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40723e0 msgr2=0x7f83c41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40723e0 0x7f83c41609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 -- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 msgr2=0x7f83c406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 0x7f83c406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3400640 1 -- 172.21.15.46:0/1984444104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4159680 con 0x7f83c4071f80 2024-04-26T23:52:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.378+0000 7f83c3e00640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 0x7f83c406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83c3400640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c3c0 tx=0x7f83b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40052d0 con 0x7f83c4071f80 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4005490 con 0x7f83c4071f80 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4007260 con 0x7f83c4071f80 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c41ee570 con 0x7f83c4071f80 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.379+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c41ed7b0 con 0x7f83c4071f80 2024-04-26T23:52:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.380+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c4069530 con 0x7f83c4071f80 2024-04-26T23:52:46.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.382+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83b4007400 con 0x7f83c4071f80 2024-04-26T23:52:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.383+0000 7f83bb400640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 0x7f83a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.383+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f83b408b1a0 con 0x7f83c4071f80 2024-04-26T23:52:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.384+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 0x7f83a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.385+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 0x7f83a006ede0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f83ac034000 tx=0x7f83ac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.388+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83c4159680 con 0x7f83c4071f80 2024-04-26T23:52:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:46.638+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f83c40b36f0 con 0x7f83c4071f80 2024-04-26T23:52:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.258+0000 7f83bb400640 1 -- 172.21.15.46:0/1984444104 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v126) ==== 118+0+0 (secure 0 0 0) 0x7f83b4059430 con 0x7f83c4071f80 2024-04-26T23:52:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.261+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 msgr2=0x7f83a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.261+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 0x7f83a006ede0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f83ac034000 tx=0x7f83ac02e000 comp rx=0 tx=0).stop 2024-04-26T23:52:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 msgr2=0x7f83c4159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c3c0 tx=0x7f83b400c880 comp rx=0 tx=0).stop 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 shutdown_connections 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f83a006c920 0x7f83a006ede0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c415dd60 0x7f83c406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40723e0 0x7f83c41609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 --2- 172.21.15.46:0/1984444104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83c4071f80 0x7f83c4159140 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 >> 172.21.15.46:0/1984444104 conn(0x7f83c406c980 msgr2=0x7f83c415c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 shutdown_connections 2024-04-26T23:52:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.262+0000 7f83ca200640 1 -- 172.21.15.46:0/1984444104 wait complete. 2024-04-26T23:52:47.286 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:52:47.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:52:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.492+0000 7f169c800640 1 Processor -- start 2024-04-26T23:52:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.493+0000 7f169c800640 1 -- start start 2024-04-26T23:52:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.494+0000 7f169c800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16940d3860 0x7f16940d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.494+0000 7f169c800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1694163e70 con 0x7f16941630a0 2024-04-26T23:52:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.494+0000 7f169c800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16940d0170 con 0x7f169415f480 2024-04-26T23:52:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.494+0000 7f169c800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f16940d0aa0 con 0x7f16940d3860 2024-04-26T23:52:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.495+0000 7f169a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16941630a0 0x7f16940b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54596/0 (socket says 172.21.15.46:54596) 2024-04-26T23:52:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.495+0000 7f169a800640 1 -- 172.21.15.46:0/1745809167 learned_addr learned my addr 172.21.15.46:0/1745809167 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.495+0000 7f1699400640 1 --2- 172.21.15.46:0/1745809167 >> v2:172.21.15.142:3300/0 conn(0x7f16940d3860 0x7f16940d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.495+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296444431 0 0) 0x7f1694163e70 con 0x7f16941630a0 2024-04-26T23:52:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1680003520 con 0x7f16941630a0 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3737053771 0 0) 0x7f1680003520 con 0x7f16941630a0 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 >> v2:172.21.15.142:3300/0 conn(0x7f16940d3860 msgr2=0x7f16940d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 --2- 172.21.15.46:0/1745809167 >> v2:172.21.15.142:3300/0 conn(0x7f16940d3860 0x7f16940d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 >> v1:172.21.15.142:6789/0 conn(0x7f169415f480 legacy=0x7f16940d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16940d0450 con 0x7f16941630a0 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1690001670 con 0x7f16941630a0 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 >> v1:172.21.15.46:6789/0 conn(0x7f16941630a0 legacy=0x7f16940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 0x7f1680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1680004800 0x7f1680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16800051c0 0x7f168000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16940d0aa0 con 0x7f16800039b0 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f168000c070 con 0x7f16800051c0 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f168000c270 con 0x7f1680004800 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1699400640 1 --2- 172.21.15.46:0/1745809167 >> v2:172.21.15.142:3300/0 conn(0x7f16940d3860 0x7f16940d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.496+0000 7f1699e00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 0x7f1680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f169a800640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16800051c0 0x7f168000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f1699e00640 1 -- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1680004800 msgr2=0x7f1680004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f1699e00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1680004800 0x7f1680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f1699e00640 1 -- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16800051c0 msgr2=0x7f168000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f1699e00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16800051c0 0x7f168000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f1699e00640 1 -- 172.21.15.46:0/1745809167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1680003520 con 0x7f16800039b0 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f169a800640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16800051c0 0x7f168000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.497+0000 7f169a800640 1 -- 172.21.15.46:0/1745809167 reap_dead start 2024-04-26T23:52:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.498+0000 7f1699e00640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 0x7f1680003dd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f168c001e50 tx=0x7f1680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93bd1078745040b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.498+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c001a70 con 0x7f16800039b0 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.498+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c02ae40 con 0x7f16800039b0 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.498+0000 7f1698a00640 1 -- 172.21.15.46:0/1745809167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c003bb0 con 0x7f16800039b0 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 -- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 msgr2=0x7f1680003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 0x7f1680003dd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f168c001e50 tx=0x7f1680005830 comp rx=0 tx=0).stop 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 -- 172.21.15.46:0/1745809167 shutdown_connections 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 --2- 172.21.15.46:0/1745809167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16800039b0 0x7f1680003dd0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 -- 172.21.15.46:0/1745809167 >> 172.21.15.46:0/1745809167 conn(0x7f16940d5070 msgr2=0x7f16940d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 -- 172.21.15.46:0/1745809167 shutdown_connections 2024-04-26T23:52:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.499+0000 7f169c800640 1 -- 172.21.15.46:0/1745809167 wait complete. 2024-04-26T23:52:47.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.501+0000 7f169c800640 1 Processor -- start 2024-04-26T23:52:47.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.501+0000 7f169c800640 1 -- start start 2024-04-26T23:52:47.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16800039b0 0x7f16941f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16940cc140 0x7f1694159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f169415f860 con 0x7f16940cee10 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16940cfe60 con 0x7f16940cc140 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f169415f9e0 con 0x7f16800039b0 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50484/0 (socket says 172.21.15.46:50484) 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.502+0000 7f169a800640 1 -- 172.21.15.46:0/3411161040 learned_addr learned my addr 172.21.15.46:0/3411161040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f1699400640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16940cc140 0x7f1694159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f1699e00640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16800039b0 0x7f16941f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 -- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16800039b0 msgr2=0x7f16941f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16800039b0 0x7f16941f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 -- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16940cc140 msgr2=0x7f1694159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16940cc140 0x7f1694159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 -- 172.21.15.46:0/3411161040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16940cc8b0 con 0x7f16940cee10 2024-04-26T23:52:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169a800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f1694163480 tx=0x7f169000d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690007440 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16900075e0 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004a20 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.503+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16940cc5d0 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.504+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16941f8ca0 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.505+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169408d8a0 con 0x7f16940cee10 2024-04-26T23:52:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.506+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1690004bc0 con 0x7f16940cee10 2024-04-26T23:52:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.506+0000 7f168a000640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 0x7f167806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.507+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f169008b800 con 0x7f16940cee10 2024-04-26T23:52:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.507+0000 7f1699e00640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 0x7f167806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.508+0000 7f1699e00640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 0x7f167806edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f168c032000 tx=0x7f168c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.511+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f169408d8a0 con 0x7f16940cee10 2024-04-26T23:52:47.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:47.760+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f169408d0d0 con 0x7f16940cee10 2024-04-26T23:52:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.262+0000 7f168a000640 1 -- 172.21.15.46:0/3411161040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v127) ==== 122+0+0 (secure 0 0 0) 0x7f1690059a00 con 0x7f16940cee10 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 msgr2=0x7f167806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 0x7f167806edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f168c032000 tx=0x7f168c02e000 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 msgr2=0x7f16941ea310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f1694163480 tx=0x7f169000d8e0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 shutdown_connections 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f167806c8f0 0x7f167806edb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16940cee10 0x7f16941ea310 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16940cc140 0x7f1694159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 --2- 172.21.15.46:0/3411161040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16800039b0 0x7f16941f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 >> 172.21.15.46:0/3411161040 conn(0x7f16940d5070 msgr2=0x7f169415fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:48.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 shutdown_connections 2024-04-26T23:52:48.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.265+0000 7f169c800640 1 -- 172.21.15.46:0/3411161040 wait complete. 2024-04-26T23:52:48.293 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:52:48.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.501+0000 7fd8eae00640 1 Processor -- start 2024-04-26T23:52:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.501+0000 7fd8eae00640 1 -- start start 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e41466b0 0x7fd8e406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8e40720e0 0x7fd8e40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8e4160910 0x7fd8e4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e4073ad0 con 0x7fd8e4160910 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e4146a90 con 0x7fd8e40720e0 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8eae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e4073d70 con 0x7fd8e41466b0 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8ea400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8e4160910 0x7fd8e4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8ea400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8e4160910 0x7fd8e4160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50490/0 (socket says 172.21.15.46:50490) 2024-04-26T23:52:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8ea400640 1 -- 172.21.15.46:0/1286444259 learned_addr learned my addr 172.21.15.46:0/1286444259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8e9000640 1 --2- 172.21.15.46:0/1286444259 >> v2:172.21.15.142:3300/0 conn(0x7fd8e40720e0 0x7fd8e40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.503+0000 7fd8e9a00640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e41466b0 0x7fd8e406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.504+0000 7fd8ea400640 1 -- 172.21.15.46:0/1286444259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e41466b0 msgr2=0x7fd8e406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.504+0000 7fd8ea400640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e41466b0 0x7fd8e406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.504+0000 7fd8ea400640 1 -- 172.21.15.46:0/1286444259 >> v2:172.21.15.142:3300/0 conn(0x7fd8e40720e0 msgr2=0x7fd8e40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.504+0000 7fd8ea400640 1 --2- 172.21.15.46:0/1286444259 >> v2:172.21.15.142:3300/0 conn(0x7fd8e40720e0 0x7fd8e40cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.504+0000 7fd8ea400640 1 -- 172.21.15.46:0/1286444259 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e4074050 con 0x7fd8e4160910 2024-04-26T23:52:48.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.505+0000 7fd8ea400640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4160910 0x7fd8e4160da0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40d0990 tx=0x7fd8dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8800ea69c83a160b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.505+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1286444259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8e4160910 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.505+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1286444259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc032440 con 0x7fd8e4160910 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.505+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1286444259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032810 con 0x7fd8e4160910 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 -- 172.21.15.46:0/1286444259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4160910 msgr2=0x7fd8e4160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4160910 0x7fd8e4160da0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40d0990 tx=0x7fd8dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 -- 172.21.15.46:0/1286444259 shutdown_connections 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4160910 0x7fd8e4160da0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1286444259 >> v2:172.21.15.142:3300/0 conn(0x7fd8e40720e0 0x7fd8e40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1286444259 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e41466b0 0x7fd8e406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 -- 172.21.15.46:0/1286444259 >> 172.21.15.46:0/1286444259 conn(0x7fd8e40b26d0 msgr2=0x7fd8e40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 -- 172.21.15.46:0/1286444259 shutdown_connections 2024-04-26T23:52:48.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.506+0000 7fd8eae00640 1 -- 172.21.15.46:0/1286444259 wait complete. 2024-04-26T23:52:48.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.508+0000 7fd8eae00640 1 Processor -- start 2024-04-26T23:52:48.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.508+0000 7fd8eae00640 1 -- start start 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e41466b0 0x7fd8e41f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e4160910 0x7fd8e40b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e406e3c0 con 0x7fd8e40720e0 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40cb470 con 0x7fd8e41466b0 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8eae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e40cdda0 con 0x7fd8e4160910 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50500/0 (socket says 172.21.15.46:50500) 2024-04-26T23:52:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 -- 172.21.15.46:0/1113273828 learned_addr learned my addr 172.21.15.46:0/1113273828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8ea400640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e4160910 0x7fd8e40b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 -- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e4160910 msgr2=0x7fd8e40b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e4160910 0x7fd8e40b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 -- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e41466b0 msgr2=0x7fd8e41f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e41466b0 0x7fd8e41f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.509+0000 7fd8e9a00640 1 -- 172.21.15.46:0/1113273828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e41f3450 con 0x7fd8e40720e0 2024-04-26T23:52:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8e9a00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007e70 tx=0x7fd8d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8014620 con 0x7fd8e40720e0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d8019070 con 0x7fd8e40720e0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8014a00 con 0x7fd8e40720e0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e41f3730 con 0x7fd8e40720e0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.510+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e41f3cf0 con 0x7fd8e40720e0 2024-04-26T23:52:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.511+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e40693b0 con 0x7fd8e40720e0 2024-04-26T23:52:48.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.512+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8d8014ba0 con 0x7fd8e40720e0 2024-04-26T23:52:48.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.513+0000 7fd8e1600640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 0x7fd8c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.513+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fd8d808c4a0 con 0x7fd8e40720e0 2024-04-26T23:52:48.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.514+0000 7fd8e9000640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 0x7fd8c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.515+0000 7fd8e9000640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 0x7fd8c406ee00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.518+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8d8011260 con 0x7fd8e40720e0 2024-04-26T23:52:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.765+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8e40c7180 con 0x7fd8e40720e0 2024-04-26T23:52:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.767+0000 7fd8e1600640 1 -- 172.21.15.46:0/1113273828 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7fd8d805a7b0 con 0x7fd8e40720e0 2024-04-26T23:52:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:48.771 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:52:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 msgr2=0x7fd8c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 0x7fd8c406ee00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d0008040 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 msgr2=0x7fd8e4141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007e70 tx=0x7fd8d800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 shutdown_connections 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd8c406c940 0x7fd8c406ee00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e4160910 0x7fd8e40b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e41466b0 0x7fd8e41f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 --2- 172.21.15.46:0/1113273828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40720e0 0x7fd8e4141640 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 >> 172.21.15.46:0/1113273828 conn(0x7fd8e40b26d0 msgr2=0x7fd8e4160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:48.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 shutdown_connections 2024-04-26T23:52:48.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.770+0000 7fd8eae00640 1 -- 172.21.15.46:0/1113273828 wait complete. 2024-04-26T23:52:48.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.993+0000 7f4085600640 1 Processor -- start 2024-04-26T23:52:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.993+0000 7f4085600640 1 -- start start 2024-04-26T23:52:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.995+0000 7f4085600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.995+0000 7f4085600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f408015ccd0 0x7f40800732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.995+0000 7f4085600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40800cf950 con 0x7f4080072e60 2024-04-26T23:52:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.995+0000 7f4085600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f40800d0c50 con 0x7f408015ccd0 2024-04-26T23:52:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.995+0000 7f4085600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f408015d490 con 0x7f4080145770 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.996+0000 7f407ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.996+0000 7f407ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:40952/0 (socket says 172.21.15.46:40952) 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.996+0000 7f407ea00640 1 -- 172.21.15.46:0/3496087413 learned_addr learned my addr 172.21.15.46:0/3496087413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.996+0000 7f407f400640 1 --2- 172.21.15.46:0/3496087413 >> v2:172.21.15.142:3300/0 conn(0x7f408015ccd0 0x7f40800732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.997+0000 7f407ea00640 1 -- 172.21.15.46:0/3496087413 >> v2:172.21.15.142:3300/0 conn(0x7f408015ccd0 msgr2=0x7f40800732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.997+0000 7f407ea00640 1 --2- 172.21.15.46:0/3496087413 >> v2:172.21.15.142:3300/0 conn(0x7f408015ccd0 0x7f40800732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.997+0000 7f407ea00640 1 -- 172.21.15.46:0/3496087413 >> v1:172.21.15.142:6789/0 conn(0x7f4080072e60 legacy=0x7f408015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.997+0000 7f407ea00640 1 -- 172.21.15.46:0/3496087413 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800cfc30 con 0x7f4080145770 2024-04-26T23:52:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.998+0000 7f407ea00640 1 --2- 172.21.15.46:0/3496087413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4070009870 tx=0x7f407002c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646bb0ffa2b5979b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f407e000640 1 -- 172.21.15.46:0/3496087413 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070038070 con 0x7f4080145770 2024-04-26T23:52:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f407e000640 1 -- 172.21.15.46:0/3496087413 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4070031710 con 0x7f4080145770 2024-04-26T23:52:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f407e000640 1 -- 172.21.15.46:0/3496087413 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070031ae0 con 0x7f4080145770 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 -- 172.21.15.46:0/3496087413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 msgr2=0x7f40800d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 --2- 172.21.15.46:0/3496087413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4070009870 tx=0x7f407002c420 comp rx=0 tx=0).stop 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 -- 172.21.15.46:0/3496087413 shutdown_connections 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 --2- 172.21.15.46:0/3496087413 >> v2:172.21.15.142:3300/0 conn(0x7f408015ccd0 0x7f40800732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 --2- 172.21.15.46:0/3496087413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080145770 0x7f40800d0710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:48.999+0000 7f4085600640 1 -- 172.21.15.46:0/3496087413 >> 172.21.15.46:0/3496087413 conn(0x7f408006c530 msgr2=0x7f408006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.000+0000 7f4085600640 1 -- 172.21.15.46:0/3496087413 shutdown_connections 2024-04-26T23:52:49.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.000+0000 7f4085600640 1 -- 172.21.15.46:0/3496087413 wait complete. 2024-04-26T23:52:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.001+0000 7f4085600640 1 Processor -- start 2024-04-26T23:52:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.001+0000 7f4085600640 1 -- start start 2024-04-26T23:52:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080072e60 0x7f408006f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4080145770 0x7f40801f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40800d01f0 con 0x7f408015ccd0 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4080073800 con 0x7f4080145770 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.002+0000 7f4085600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40800cf5b0 con 0x7f4080072e60 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50516/0 (socket says 172.21.15.46:50516) 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 -- 172.21.15.46:0/3887389199 learned_addr learned my addr 172.21.15.46:0/3887389199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407ea00640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080072e60 0x7f408006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f4076000640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4080145770 0x7f40801f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 -- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080072e60 msgr2=0x7f408006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080072e60 0x7f408006f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 -- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4080145770 msgr2=0x7f40801f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4080145770 0x7f40801f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.003+0000 7f407f400640 1 -- 172.21.15.46:0/3887389199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40801f6150 con 0x7f408015ccd0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f407f400640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f407800efc0 tx=0x7f407800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078018020 con 0x7f408015ccd0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4078014480 con 0x7f408015ccd0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078014870 con 0x7f408015ccd0 2024-04-26T23:52:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40801f6430 con 0x7f408015ccd0 2024-04-26T23:52:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.004+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40801f6d40 con 0x7f408015ccd0 2024-04-26T23:52:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.005+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4080069530 con 0x7f408015ccd0 2024-04-26T23:52:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.007+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4078005d10 con 0x7f408015ccd0 2024-04-26T23:52:49.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.008+0000 7f4077400640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.008+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f407808c7c0 con 0x7f408015ccd0 2024-04-26T23:52:49.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.009+0000 7f407ea00640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.010+0000 7f407ea00640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 0x7f405c06edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4070035000 tx=0x7f4070008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.012+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4078022070 con 0x7f408015ccd0 2024-04-26T23:52:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.221+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f408015bfe0 con 0x7f408015ccd0 2024-04-26T23:52:49.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.225+0000 7f4077400640 1 -- 172.21.15.46:0/3887389199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16848 (secure 0 0 0) 0x7f4078010040 con 0x7f408015ccd0 2024-04-26T23:52:49.229 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:49.229 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":161,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:52:46.230039+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 msgr2=0x7f405c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 0x7f405c06edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4070035000 tx=0x7f4070008030 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 msgr2=0x7f40800c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f407800efc0 tx=0x7f407800c490 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 shutdown_connections 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f408015ccd0 0x7f40800c9620 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4080145770 0x7f40801f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 --2- 172.21.15.46:0/3887389199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4080072e60 0x7f408006f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 >> 172.21.15.46:0/3887389199 conn(0x7f408006c530 msgr2=0x7f40800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 shutdown_connections 2024-04-26T23:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.228+0000 7f4085600640 1 -- 172.21.15.46:0/3887389199 wait complete. 2024-04-26T23:52:49.253 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:52:49.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:52:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.464+0000 7f8f2d600640 1 Processor -- start 2024-04-26T23:52:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.464+0000 7f8f2d600640 1 -- start start 2024-04-26T23:52:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.466+0000 7f8f2d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f2815f930 0x7f8f2806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.466+0000 7f8f2d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f2806c490 0x7f8f2806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.466+0000 7f8f2d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f28163050 con 0x7f8f28160470 2024-04-26T23:52:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.466+0000 7f8f2d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8f28160850 con 0x7f8f2806c490 2024-04-26T23:52:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.466+0000 7f8f2d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8f281632f0 con 0x7f8f2815f930 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f2806c490 0x7f8f2806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f2806c490 0x7f8f2806c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50524/0 (socket says 172.21.15.46:50524) 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 -- 172.21.15.46:0/969460409 learned_addr learned my addr 172.21.15.46:0/969460409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f26000640 1 --2- 172.21.15.46:0/969460409 >> v2:172.21.15.142:3300/0 conn(0x7f8f2815f930 0x7f8f2806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 -- 172.21.15.46:0/969460409 >> v2:172.21.15.142:3300/0 conn(0x7f8f2815f930 msgr2=0x7f8f2806f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 --2- 172.21.15.46:0/969460409 >> v2:172.21.15.142:3300/0 conn(0x7f8f2815f930 0x7f8f2806f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 -- 172.21.15.46:0/969460409 >> v1:172.21.15.142:6789/0 conn(0x7f8f28160470 legacy=0x7f8f2806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.467+0000 7f8f27400640 1 -- 172.21.15.46:0/969460409 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281635d0 con 0x7f8f2806c490 2024-04-26T23:52:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.469+0000 7f8f27400640 1 --2- 172.21.15.46:0/969460409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2806c490 0x7f8f2806c8a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b3ec0 tx=0x7f8f2002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c85e0d000e9c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.469+0000 7f8f25600640 1 -- 172.21.15.46:0/969460409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20036070 con 0x7f8f2806c490 2024-04-26T23:52:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.469+0000 7f8f25600640 1 -- 172.21.15.46:0/969460409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20031600 con 0x7f8f2806c490 2024-04-26T23:52:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.469+0000 7f8f25600640 1 -- 172.21.15.46:0/969460409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200319d0 con 0x7f8f2806c490 2024-04-26T23:52:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 -- 172.21.15.46:0/969460409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2806c490 msgr2=0x7f8f2806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 --2- 172.21.15.46:0/969460409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2806c490 0x7f8f2806c8a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b3ec0 tx=0x7f8f2002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 -- 172.21.15.46:0/969460409 shutdown_connections 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 --2- 172.21.15.46:0/969460409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2806c490 0x7f8f2806c8a0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 --2- 172.21.15.46:0/969460409 >> v2:172.21.15.142:3300/0 conn(0x7f8f2815f930 0x7f8f2806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 -- 172.21.15.46:0/969460409 >> 172.21.15.46:0/969460409 conn(0x7f8f280b38b0 msgr2=0x7f8f280d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 -- 172.21.15.46:0/969460409 shutdown_connections 2024-04-26T23:52:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.470+0000 7f8f2d600640 1 -- 172.21.15.46:0/969460409 wait complete. 2024-04-26T23:52:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.472+0000 7f8f2d600640 1 Processor -- start 2024-04-26T23:52:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.472+0000 7f8f2d600640 1 -- start start 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2806c490 0x7f8f281e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f28160470 0x7f8f2815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28070750 con 0x7f8f2815f930 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28163b90 con 0x7f8f2806c490 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f2d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2806f9f0 con 0x7f8f28160470 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50532/0 (socket says 172.21.15.46:50532) 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.473+0000 7f8f26000640 1 -- 172.21.15.46:0/2900465368 learned_addr learned my addr 172.21.15.46:0/2900465368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26a00640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2806c490 0x7f8f281e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f27400640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f28160470 0x7f8f2815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 -- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f28160470 msgr2=0x7f8f2815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f28160470 0x7f8f2815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 -- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2806c490 msgr2=0x7f8f281e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2806c490 0x7f8f281e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 -- 172.21.15.46:0/2900465368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281e9f30 con 0x7f8f2815f930 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f26000640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400d830 tx=0x7f8f1400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400b6c0 con 0x7f8f2815f930 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14019070 con 0x7f8f2815f930 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.474+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400ba80 con 0x7f8f2815f930 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.475+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2815b7d0 con 0x7f8f2815f930 2024-04-26T23:52:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.475+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281e9c20 con 0x7f8f2815f930 2024-04-26T23:52:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.476+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2815b9a0 con 0x7f8f2815f930 2024-04-26T23:52:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.477+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f1401e460 con 0x7f8f2815f930 2024-04-26T23:52:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.478+0000 7f8f1ea00640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 0x7f8f0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.478+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f8f1408c3d0 con 0x7f8f2815f930 2024-04-26T23:52:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.478+0000 7f8f26a00640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 0x7f8f0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.480+0000 7f8f26a00640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 0x7f8f0806edb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8f18006fd0 tx=0x7f8f18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.482+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f1405a660 con 0x7f8f2815f930 2024-04-26T23:52:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.684+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f28069530 con 0x7f8f0806c8f0 2024-04-26T23:52:49.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.705+0000 7f8f1ea00640 1 -- 172.21.15.46:0/2900465368 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f8f28069530 con 0x7f8f0806c8f0 2024-04-26T23:52:49.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:52:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 msgr2=0x7f8f0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 0x7f8f0806edb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8f18006fd0 tx=0x7f8f18008040 comp rx=0 tx=0).stop 2024-04-26T23:52:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 msgr2=0x7f8f281f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400d830 tx=0x7f8f1400dcf0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 shutdown_connections 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8f0806c8f0 0x7f8f0806edb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f28160470 0x7f8f2815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f2815f930 0x7f8f281f4a60 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 --2- 172.21.15.46:0/2900465368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2806c490 0x7f8f281e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 >> 172.21.15.46:0/2900465368 conn(0x7f8f280b38b0 msgr2=0x7f8f2806eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.711+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 shutdown_connections 2024-04-26T23:52:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.712+0000 7f8f2d600640 1 -- 172.21.15.46:0/2900465368 wait complete. 2024-04-26T23:52:49.735 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2024-04-26T23:52:48.241450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448212,"kb_used_data":16936,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754526508,"statfs":{"total":773094113280,"available":772635144192,"internally_reserved":0,"allocated":17342464,"data_stored":8761332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":441503281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.056251"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245858+0000","last_change":"2024-04-26T23:52:45.242231+0000","last_active":"2024-04-26T23:52:45.245858+0000","last_peered":"2024-04-26T23:52:45.245858+0000","last_clean":"2024-04-26T23:52:45.245858+0000","last_became_active":"2024-04-26T23:52:45.241079+0000","last_became_peered":"2024-04-26T23:52:45.241079+0000","last_unstale":"2024-04-26T23:52:45.245858+0000","last_undegraded":"2024-04-26T23:52:45.245858+0000","last_fullsized":"2024-04-26T23:52:45.245858+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:25.635643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248365+0000","last_change":"2024-04-26T23:52:45.244575+0000","last_active":"2024-04-26T23:52:45.248365+0000","last_peered":"2024-04-26T23:52:45.248365+0000","last_clean":"2024-04-26T23:52:45.248365+0000","last_became_active":"2024-04-26T23:52:45.239682+0000","last_became_peered":"2024-04-26T23:52:45.239682+0000","last_unstale":"2024-04-26T23:52:45.248365+0000","last_undegraded":"2024-04-26T23:52:45.248365+0000","last_fullsized":"2024-04-26T23:52:45.248365+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:47.629201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.247308+0000","last_change":"2024-04-26T23:52:45.242937+0000","last_active":"2024-04-26T23:52:45.247308+0000","last_peered":"2024-04-26T23:52:45.247308+0000","last_clean":"2024-04-26T23:52:45.247308+0000","last_became_active":"2024-04-26T23:52:45.242649+0000","last_became_peered":"2024-04-26T23:52:45.242649+0000","last_unstale":"2024-04-26T23:52:45.247308+0000","last_undegraded":"2024-04-26T23:52:45.247308+0000","last_fullsized":"2024-04-26T23:52:45.247308+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:28.797918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239863+0000","last_change":"2024-04-26T23:52:45.232910+0000","last_active":"2024-04-26T23:52:45.239863+0000","last_peered":"2024-04-26T23:52:45.239863+0000","last_clean":"2024-04-26T23:52:45.239863+0000","last_became_active":"2024-04-26T23:52:45.229308+0000","last_became_peered":"2024-04-26T23:52:45.229308+0000","last_unstale":"2024-04-26T23:52:45.239863+0000","last_undegraded":"2024-04-26T23:52:45.239863+0000","last_fullsized":"2024-04-26T23:52:45.239863+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:06.298828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252906+0000","last_change":"2024-04-26T23:52:45.247689+0000","last_active":"2024-04-26T23:52:45.252906+0000","last_peered":"2024-04-26T23:52:45.252906+0000","last_clean":"2024-04-26T23:52:45.252906+0000","last_became_active":"2024-04-26T23:52:45.247141+0000","last_became_peered":"2024-04-26T23:52:45.247141+0000","last_unstale":"2024-04-26T23:52:45.252906+0000","last_undegraded":"2024-04-26T23:52:45.252906+0000","last_fullsized":"2024-04-26T23:52:45.252906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.447117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.246822+0000","last_change":"2024-04-26T23:52:45.239666+0000","last_active":"2024-04-26T23:52:45.246822+0000","last_peered":"2024-04-26T23:52:45.246822+0000","last_clean":"2024-04-26T23:52:45.246822+0000","last_became_active":"2024-04-26T23:52:45.239342+0000","last_became_peered":"2024-04-26T23:52:45.239342+0000","last_unstale":"2024-04-26T23:52:45.246822+0000","last_undegraded":"2024-04-26T23:52:45.246822+0000","last_fullsized":"2024-04-26T23:52:45.246822+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259361+0000","last_change":"2024-04-26T23:52:45.252760+0000","last_active":"2024-04-26T23:52:45.259361+0000","last_peered":"2024-04-26T23:52:45.259361+0000","last_clean":"2024-04-26T23:52:45.259361+0000","last_became_active":"2024-04-26T23:52:45.251340+0000","last_became_peered":"2024-04-26T23:52:45.251340+0000","last_unstale":"2024-04-26T23:52:45.259361+0000","last_undegraded":"2024-04-26T23:52:45.259361+0000","last_fullsized":"2024-04-26T23:52:45.259361+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257987+0000","last_change":"2024-04-26T23:52:45.243752+0000","last_active":"2024-04-26T23:52:45.257987+0000","last_peered":"2024-04-26T23:52:45.257987+0000","last_clean":"2024-04-26T23:52:45.257987+0000","last_became_active":"2024-04-26T23:52:45.240132+0000","last_became_peered":"2024-04-26T23:52:45.240132+0000","last_unstale":"2024-04-26T23:52:45.257987+0000","last_undegraded":"2024-04-26T23:52:45.257987+0000","last_fullsized":"2024-04-26T23:52:45.257987+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257973+0000","last_change":"2024-04-26T23:52:45.244323+0000","last_active":"2024-04-26T23:52:45.257973+0000","last_peered":"2024-04-26T23:52:45.257973+0000","last_clean":"2024-04-26T23:52:45.257973+0000","last_became_active":"2024-04-26T23:52:45.243563+0000","last_became_peered":"2024-04-26T23:52:45.243563+0000","last_unstale":"2024-04-26T23:52:45.257973+0000","last_undegraded":"2024-04-26T23:52:45.257973+0000","last_fullsized":"2024-04-26T23:52:45.257973+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257848+0000","last_change":"2024-04-26T23:52:45.242508+0000","last_active":"2024-04-26T23:52:45.257848+0000","last_peered":"2024-04-26T23:52:45.257848+0000","last_clean":"2024-04-26T23:52:45.257848+0000","last_became_active":"2024-04-26T23:52:45.240195+0000","last_became_peered":"2024-04-26T23:52:45.240195+0000","last_unstale":"2024-04-26T23:52:45.257848+0000","last_undegraded":"2024-04-26T23:52:45.257848+0000","last_fullsized":"2024-04-26T23:52:45.257848+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:24.382711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239139+0000","last_change":"2024-04-26T23:52:45.235466+0000","last_active":"2024-04-26T23:52:45.239139+0000","last_peered":"2024-04-26T23:52:45.239139+0000","last_clean":"2024-04-26T23:52:45.239139+0000","last_became_active":"2024-04-26T23:52:45.235188+0000","last_became_peered":"2024-04-26T23:52:45.235188+0000","last_unstale":"2024-04-26T23:52:45.239139+0000","last_undegraded":"2024-04-26T23:52:45.239139+0000","last_fullsized":"2024-04-26T23:52:45.239139+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.249684+0000","last_change":"2024-04-26T23:52:45.247006+0000","last_active":"2024-04-26T23:52:45.249684+0000","last_peered":"2024-04-26T23:52:45.249684+0000","last_clean":"2024-04-26T23:52:45.249684+0000","last_became_active":"2024-04-26T23:52:45.244443+0000","last_became_peered":"2024-04-26T23:52:45.244443+0000","last_unstale":"2024-04-26T23:52:45.249684+0000","last_undegraded":"2024-04-26T23:52:45.249684+0000","last_fullsized":"2024-04-26T23:52:45.249684+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:19:18.056046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.235495+0000","last_change":"2024-04-26T23:52:45.233370+0000","last_active":"2024-04-26T23:52:45.235495+0000","last_peered":"2024-04-26T23:52:45.235495+0000","last_clean":"2024-04-26T23:52:45.235495+0000","last_became_active":"2024-04-26T23:52:45.233086+0000","last_became_peered":"2024-04-26T23:52:45.233086+0000","last_unstale":"2024-04-26T23:52:45.235495+0000","last_undegraded":"2024-04-26T23:52:45.235495+0000","last_fullsized":"2024-04-26T23:52:45.235495+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:22.670229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252476+0000","last_change":"2024-04-26T23:52:45.246036+0000","last_active":"2024-04-26T23:52:45.252476+0000","last_peered":"2024-04-26T23:52:45.252476+0000","last_clean":"2024-04-26T23:52:45.252476+0000","last_became_active":"2024-04-26T23:52:45.245736+0000","last_became_peered":"2024-04-26T23:52:45.245736+0000","last_unstale":"2024-04-26T23:52:45.252476+0000","last_undegraded":"2024-04-26T23:52:45.252476+0000","last_fullsized":"2024-04-26T23:52:45.252476+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:18.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256907+0000","last_change":"2024-04-26T23:52:45.254895+0000","last_active":"2024-04-26T23:52:45.256907+0000","last_peered":"2024-04-26T23:52:45.256907+0000","last_clean":"2024-04-26T23:52:45.256907+0000","last_became_active":"2024-04-26T23:52:45.254555+0000","last_became_peered":"2024-04-26T23:52:45.254555+0000","last_unstale":"2024-04-26T23:52:45.256907+0000","last_undegraded":"2024-04-26T23:52:45.256907+0000","last_fullsized":"2024-04-26T23:52:45.256907+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:06.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239578+0000","last_change":"2024-04-26T23:52:45.237436+0000","last_active":"2024-04-26T23:52:45.239578+0000","last_peered":"2024-04-26T23:52:45.239578+0000","last_clean":"2024-04-26T23:52:45.239578+0000","last_became_active":"2024-04-26T23:52:45.236766+0000","last_became_peered":"2024-04-26T23:52:45.236766+0000","last_unstale":"2024-04-26T23:52:45.239578+0000","last_undegraded":"2024-04-26T23:52:45.239578+0000","last_fullsized":"2024-04-26T23:52:45.239578+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:43.676634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.254623+0000","last_change":"2024-04-26T23:52:45.237604+0000","last_active":"2024-04-26T23:52:45.254623+0000","last_peered":"2024-04-26T23:52:45.254623+0000","last_clean":"2024-04-26T23:52:45.254623+0000","last_became_active":"2024-04-26T23:52:45.237338+0000","last_became_peered":"2024-04-26T23:52:45.237338+0000","last_unstale":"2024-04-26T23:52:45.254623+0000","last_undegraded":"2024-04-26T23:52:45.254623+0000","last_fullsized":"2024-04-26T23:52:45.254623+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:04.278595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.255856+0000","last_change":"2024-04-26T23:52:45.254819+0000","last_active":"2024-04-26T23:52:45.255856+0000","last_peered":"2024-04-26T23:52:45.255856+0000","last_clean":"2024-04-26T23:52:45.255856+0000","last_became_active":"2024-04-26T23:52:45.254425+0000","last_became_peered":"2024-04-26T23:52:45.254425+0000","last_unstale":"2024-04-26T23:52:45.255856+0000","last_undegraded":"2024-04-26T23:52:45.255856+0000","last_fullsized":"2024-04-26T23:52:45.255856+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:09.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258202+0000","last_change":"2024-04-26T23:52:45.256946+0000","last_active":"2024-04-26T23:52:45.258202+0000","last_peered":"2024-04-26T23:52:45.258202+0000","last_clean":"2024-04-26T23:52:45.258202+0000","last_became_active":"2024-04-26T23:52:45.256637+0000","last_became_peered":"2024-04-26T23:52:45.256637+0000","last_unstale":"2024-04-26T23:52:45.258202+0000","last_undegraded":"2024-04-26T23:52:45.258202+0000","last_fullsized":"2024-04-26T23:52:45.258202+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:14.669687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243055+0000","last_change":"2024-04-26T23:52:45.237343+0000","last_active":"2024-04-26T23:52:45.243055+0000","last_peered":"2024-04-26T23:52:45.243055+0000","last_clean":"2024-04-26T23:52:45.243055+0000","last_became_active":"2024-04-26T23:52:45.236807+0000","last_became_peered":"2024-04-26T23:52:45.236807+0000","last_unstale":"2024-04-26T23:52:45.243055+0000","last_undegraded":"2024-04-26T23:52:45.243055+0000","last_fullsized":"2024-04-26T23:52:45.243055+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:53.556520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243306+0000","last_change":"2024-04-26T23:52:45.239580+0000","last_active":"2024-04-26T23:52:45.243306+0000","last_peered":"2024-04-26T23:52:45.243306+0000","last_clean":"2024-04-26T23:52:45.243306+0000","last_became_active":"2024-04-26T23:52:45.239259+0000","last_became_peered":"2024-04-26T23:52:45.239259+0000","last_unstale":"2024-04-26T23:52:45.243306+0000","last_undegraded":"2024-04-26T23:52:45.243306+0000","last_fullsized":"2024-04-26T23:52:45.243306+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:02.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.237075+0000","last_change":"2024-04-26T23:52:45.233630+0000","last_active":"2024-04-26T23:52:45.237075+0000","last_peered":"2024-04-26T23:52:45.237075+0000","last_clean":"2024-04-26T23:52:45.237075+0000","last_became_active":"2024-04-26T23:52:45.233339+0000","last_became_peered":"2024-04-26T23:52:45.233339+0000","last_unstale":"2024-04-26T23:52:45.237075+0000","last_undegraded":"2024-04-26T23:52:45.237075+0000","last_fullsized":"2024-04-26T23:52:45.237075+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.254033+0000","last_change":"2024-04-26T23:52:45.248507+0000","last_active":"2024-04-26T23:52:45.254033+0000","last_peered":"2024-04-26T23:52:45.254033+0000","last_clean":"2024-04-26T23:52:45.254033+0000","last_became_active":"2024-04-26T23:52:45.248201+0000","last_became_peered":"2024-04-26T23:52:45.248201+0000","last_unstale":"2024-04-26T23:52:45.254033+0000","last_undegraded":"2024-04-26T23:52:45.254033+0000","last_fullsized":"2024-04-26T23:52:45.254033+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:16.451622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239612+0000","last_change":"2024-04-26T23:52:45.237696+0000","last_active":"2024-04-26T23:52:45.239612+0000","last_peered":"2024-04-26T23:52:45.239612+0000","last_clean":"2024-04-26T23:52:45.239612+0000","last_became_active":"2024-04-26T23:52:45.237157+0000","last_became_peered":"2024-04-26T23:52:45.237157+0000","last_unstale":"2024-04-26T23:52:45.239612+0000","last_undegraded":"2024-04-26T23:52:45.239612+0000","last_fullsized":"2024-04-26T23:52:45.239612+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:33.160098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239400+0000","last_change":"2024-04-26T23:52:45.235282+0000","last_active":"2024-04-26T23:52:45.239400+0000","last_peered":"2024-04-26T23:52:45.239400+0000","last_clean":"2024-04-26T23:52:45.239400+0000","last_became_active":"2024-04-26T23:52:45.234949+0000","last_became_peered":"2024-04-26T23:52:45.234949+0000","last_unstale":"2024-04-26T23:52:45.239400+0000","last_undegraded":"2024-04-26T23:52:45.239400+0000","last_fullsized":"2024-04-26T23:52:45.239400+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:44.200196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258406+0000","last_change":"2024-04-26T23:52:45.249751+0000","last_active":"2024-04-26T23:52:45.258406+0000","last_peered":"2024-04-26T23:52:45.258406+0000","last_clean":"2024-04-26T23:52:45.258406+0000","last_became_active":"2024-04-26T23:52:45.249470+0000","last_became_peered":"2024-04-26T23:52:45.249470+0000","last_unstale":"2024-04-26T23:52:45.258406+0000","last_undegraded":"2024-04-26T23:52:45.258406+0000","last_fullsized":"2024-04-26T23:52:45.258406+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:33.930128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.236688+0000","last_change":"2024-04-26T23:52:45.231174+0000","last_active":"2024-04-26T23:52:45.236688+0000","last_peered":"2024-04-26T23:52:45.236688+0000","last_clean":"2024-04-26T23:52:45.236688+0000","last_became_active":"2024-04-26T23:52:45.230526+0000","last_became_peered":"2024-04-26T23:52:45.230526+0000","last_unstale":"2024-04-26T23:52:45.236688+0000","last_undegraded":"2024-04-26T23:52:45.236688+0000","last_fullsized":"2024-04-26T23:52:45.236688+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.232605+0000","last_change":"2024-04-26T23:52:45.230088+0000","last_active":"2024-04-26T23:52:45.232605+0000","last_peered":"2024-04-26T23:52:45.232605+0000","last_clean":"2024-04-26T23:52:45.232605+0000","last_became_active":"2024-04-26T23:52:45.228832+0000","last_became_peered":"2024-04-26T23:52:45.228832+0000","last_unstale":"2024-04-26T23:52:45.232605+0000","last_undegraded":"2024-04-26T23:52:45.232605+0000","last_fullsized":"2024-04-26T23:52:45.232605+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:31.569776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248400+0000","last_change":"2024-04-26T23:52:45.241806+0000","last_active":"2024-04-26T23:52:45.248400+0000","last_peered":"2024-04-26T23:52:45.248400+0000","last_clean":"2024-04-26T23:52:45.248400+0000","last_became_active":"2024-04-26T23:52:45.241512+0000","last_became_peered":"2024-04-26T23:52:45.241512+0000","last_unstale":"2024-04-26T23:52:45.248400+0000","last_undegraded":"2024-04-26T23:52:45.248400+0000","last_fullsized":"2024-04-26T23:52:45.248400+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:20.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248454+0000","last_change":"2024-04-26T23:52:45.244880+0000","last_active":"2024-04-26T23:52:45.248454+0000","last_peered":"2024-04-26T23:52:45.248454+0000","last_clean":"2024-04-26T23:52:45.248454+0000","last_became_active":"2024-04-26T23:52:45.241714+0000","last_became_peered":"2024-04-26T23:52:45.241714+0000","last_unstale":"2024-04-26T23:52:45.248454+0000","last_undegraded":"2024-04-26T23:52:45.248454+0000","last_fullsized":"2024-04-26T23:52:45.248454+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:10.589629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.250483+0000","last_change":"2024-04-26T23:52:45.248001+0000","last_active":"2024-04-26T23:52:45.250483+0000","last_peered":"2024-04-26T23:52:45.250483+0000","last_clean":"2024-04-26T23:52:45.250483+0000","last_became_active":"2024-04-26T23:52:45.247081+0000","last_became_peered":"2024-04-26T23:52:45.247081+0000","last_unstale":"2024-04-26T23:52:45.250483+0000","last_undegraded":"2024-04-26T23:52:45.250483+0000","last_fullsized":"2024-04-26T23:52:45.250483+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:20.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245291+0000","last_change":"2024-04-26T23:52:45.238717+0000","last_active":"2024-04-26T23:52:45.245291+0000","last_peered":"2024-04-26T23:52:45.245291+0000","last_clean":"2024-04-26T23:52:45.245291+0000","last_became_active":"2024-04-26T23:52:45.238188+0000","last_became_peered":"2024-04-26T23:52:45.238188+0000","last_unstale":"2024-04-26T23:52:45.245291+0000","last_undegraded":"2024-04-26T23:52:45.245291+0000","last_fullsized":"2024-04-26T23:52:45.245291+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:45.350016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248315+0000","last_change":"2024-04-26T23:52:45.243206+0000","last_active":"2024-04-26T23:52:45.248315+0000","last_peered":"2024-04-26T23:52:45.248315+0000","last_clean":"2024-04-26T23:52:45.248315+0000","last_became_active":"2024-04-26T23:52:45.242860+0000","last_became_peered":"2024-04-26T23:52:45.242860+0000","last_unstale":"2024-04-26T23:52:45.248315+0000","last_undegraded":"2024-04-26T23:52:45.248315+0000","last_fullsized":"2024-04-26T23:52:45.248315+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:14.669687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"160'4","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259244+0000","last_change":"2024-04-26T23:52:44.222425+0000","last_active":"2024-04-26T23:52:45.259244+0000","last_peered":"2024-04-26T23:52:45.259244+0000","last_clean":"2024-04-26T23:52:45.259244+0000","last_became_active":"2024-04-26T23:52:44.221707+0000","last_became_peered":"2024-04-26T23:52:44.221707+0000","last_unstale":"2024-04-26T23:52:45.259244+0000","last_undegraded":"2024-04-26T23:52:45.259244+0000","last_fullsized":"2024-04-26T23:52:45.259244+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:37.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.247200+0000","last_change":"2024-04-26T23:52:45.246067+0000","last_active":"2024-04-26T23:52:45.247200+0000","last_peered":"2024-04-26T23:52:45.247200+0000","last_clean":"2024-04-26T23:52:45.247200+0000","last_became_active":"2024-04-26T23:52:45.245802+0000","last_became_peered":"2024-04-26T23:52:45.245802+0000","last_unstale":"2024-04-26T23:52:45.247200+0000","last_undegraded":"2024-04-26T23:52:45.247200+0000","last_fullsized":"2024-04-26T23:52:45.247200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:31.289188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264142+0000","last_change":"2024-04-26T23:52:44.231249+0000","last_active":"2024-04-26T23:52:45.264142+0000","last_peered":"2024-04-26T23:52:45.264142+0000","last_clean":"2024-04-26T23:52:45.264142+0000","last_became_active":"2024-04-26T23:52:44.230957+0000","last_became_peered":"2024-04-26T23:52:44.230957+0000","last_unstale":"2024-04-26T23:52:45.264142+0000","last_undegraded":"2024-04-26T23:52:45.264142+0000","last_fullsized":"2024-04-26T23:52:45.264142+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:21.615078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.250896+0000","last_change":"2024-04-26T23:52:45.241211+0000","last_active":"2024-04-26T23:52:45.250896+0000","last_peered":"2024-04-26T23:52:45.250896+0000","last_clean":"2024-04-26T23:52:45.250896+0000","last_became_active":"2024-04-26T23:52:45.238448+0000","last_became_peered":"2024-04-26T23:52:45.238448+0000","last_unstale":"2024-04-26T23:52:45.250896+0000","last_undegraded":"2024-04-26T23:52:45.250896+0000","last_fullsized":"2024-04-26T23:52:45.250896+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.333700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259941+0000","last_change":"2024-04-26T23:52:44.235027+0000","last_active":"2024-04-26T23:52:45.259941+0000","last_peered":"2024-04-26T23:52:45.259941+0000","last_clean":"2024-04-26T23:52:45.259941+0000","last_became_active":"2024-04-26T23:52:44.234740+0000","last_became_peered":"2024-04-26T23:52:44.234740+0000","last_unstale":"2024-04-26T23:52:45.259941+0000","last_undegraded":"2024-04-26T23:52:45.259941+0000","last_fullsized":"2024-04-26T23:52:45.259941+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:51.528844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.246712+0000","last_change":"2024-04-26T23:52:45.237855+0000","last_active":"2024-04-26T23:52:45.246712+0000","last_peered":"2024-04-26T23:52:45.246712+0000","last_clean":"2024-04-26T23:52:45.246712+0000","last_became_active":"2024-04-26T23:52:45.237238+0000","last_became_peered":"2024-04-26T23:52:45.237238+0000","last_unstale":"2024-04-26T23:52:45.246712+0000","last_undegraded":"2024-04-26T23:52:45.246712+0000","last_fullsized":"2024-04-26T23:52:45.246712+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:38.441388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267628+0000","last_change":"2024-04-26T23:52:44.233273+0000","last_active":"2024-04-26T23:52:45.267628+0000","last_peered":"2024-04-26T23:52:45.267628+0000","last_clean":"2024-04-26T23:52:45.267628+0000","last_became_active":"2024-04-26T23:52:44.232932+0000","last_became_peered":"2024-04-26T23:52:44.232932+0000","last_unstale":"2024-04-26T23:52:45.267628+0000","last_undegraded":"2024-04-26T23:52:45.267628+0000","last_fullsized":"2024-04-26T23:52:45.267628+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:23:23.496996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259218+0000","last_change":"2024-04-26T23:52:45.252362+0000","last_active":"2024-04-26T23:52:45.259218+0000","last_peered":"2024-04-26T23:52:45.259218+0000","last_clean":"2024-04-26T23:52:45.259218+0000","last_became_active":"2024-04-26T23:52:45.252066+0000","last_became_peered":"2024-04-26T23:52:45.252066+0000","last_unstale":"2024-04-26T23:52:45.259218+0000","last_undegraded":"2024-04-26T23:52:45.259218+0000","last_fullsized":"2024-04-26T23:52:45.259218+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.268722+0000","last_change":"2024-04-26T23:52:44.239921+0000","last_active":"2024-04-26T23:52:45.268722+0000","last_peered":"2024-04-26T23:52:45.268722+0000","last_clean":"2024-04-26T23:52:45.268722+0000","last_became_active":"2024-04-26T23:52:44.239623+0000","last_became_peered":"2024-04-26T23:52:44.239623+0000","last_unstale":"2024-04-26T23:52:45.268722+0000","last_undegraded":"2024-04-26T23:52:45.268722+0000","last_fullsized":"2024-04-26T23:52:45.268722+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:06.810802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248675+0000","last_change":"2024-04-26T23:52:45.243368+0000","last_active":"2024-04-26T23:52:45.248675+0000","last_peered":"2024-04-26T23:52:45.248675+0000","last_clean":"2024-04-26T23:52:45.248675+0000","last_became_active":"2024-04-26T23:52:45.237517+0000","last_became_peered":"2024-04-26T23:52:45.237517+0000","last_unstale":"2024-04-26T23:52:45.248675+0000","last_undegraded":"2024-04-26T23:52:45.248675+0000","last_fullsized":"2024-04-26T23:52:45.248675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:59.715643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264600+0000","last_change":"2024-04-26T23:52:44.225024+0000","last_active":"2024-04-26T23:52:45.264600+0000","last_peered":"2024-04-26T23:52:45.264600+0000","last_clean":"2024-04-26T23:52:45.264600+0000","last_became_active":"2024-04-26T23:52:44.223248+0000","last_became_peered":"2024-04-26T23:52:44.223248+0000","last_unstale":"2024-04-26T23:52:45.264600+0000","last_undegraded":"2024-04-26T23:52:45.264600+0000","last_fullsized":"2024-04-26T23:52:45.264600+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:46.611618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258445+0000","last_change":"2024-04-26T23:52:45.251131+0000","last_active":"2024-04-26T23:52:45.258445+0000","last_peered":"2024-04-26T23:52:45.258445+0000","last_clean":"2024-04-26T23:52:45.258445+0000","last_became_active":"2024-04-26T23:52:45.250210+0000","last_became_peered":"2024-04-26T23:52:45.250210+0000","last_unstale":"2024-04-26T23:52:45.258445+0000","last_undegraded":"2024-04-26T23:52:45.258445+0000","last_fullsized":"2024-04-26T23:52:45.258445+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:35.821082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225615+0000","last_change":"2024-04-26T23:52:44.222397+0000","last_active":"2024-04-26T23:52:45.225615+0000","last_peered":"2024-04-26T23:52:45.225615+0000","last_clean":"2024-04-26T23:52:45.225615+0000","last_became_active":"2024-04-26T23:52:44.222106+0000","last_became_peered":"2024-04-26T23:52:44.222106+0000","last_unstale":"2024-04-26T23:52:45.225615+0000","last_undegraded":"2024-04-26T23:52:45.225615+0000","last_fullsized":"2024-04-26T23:52:45.225615+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:10.021380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.244025+0000","last_change":"2024-04-26T23:52:45.240996+0000","last_active":"2024-04-26T23:52:45.244025+0000","last_peered":"2024-04-26T23:52:45.244025+0000","last_clean":"2024-04-26T23:52:45.244025+0000","last_became_active":"2024-04-26T23:52:45.239945+0000","last_became_peered":"2024-04-26T23:52:45.239945+0000","last_unstale":"2024-04-26T23:52:45.244025+0000","last_undegraded":"2024-04-26T23:52:45.244025+0000","last_fullsized":"2024-04-26T23:52:45.244025+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:52.896334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243079+0000","last_change":"2024-04-26T23:52:44.237595+0000","last_active":"2024-04-26T23:52:45.243079+0000","last_peered":"2024-04-26T23:52:45.243079+0000","last_clean":"2024-04-26T23:52:45.243079+0000","last_became_active":"2024-04-26T23:52:44.237178+0000","last_became_peered":"2024-04-26T23:52:44.237178+0000","last_unstale":"2024-04-26T23:52:45.243079+0000","last_undegraded":"2024-04-26T23:52:45.243079+0000","last_fullsized":"2024-04-26T23:52:45.243079+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:11.502021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248030+0000","last_change":"2024-04-26T23:52:45.241243+0000","last_active":"2024-04-26T23:52:45.248030+0000","last_peered":"2024-04-26T23:52:45.248030+0000","last_clean":"2024-04-26T23:52:45.248030+0000","last_became_active":"2024-04-26T23:52:45.240944+0000","last_became_peered":"2024-04-26T23:52:45.240944+0000","last_unstale":"2024-04-26T23:52:45.248030+0000","last_undegraded":"2024-04-26T23:52:45.248030+0000","last_fullsized":"2024-04-26T23:52:45.248030+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225682+0000","last_change":"2024-04-26T23:52:44.231535+0000","last_active":"2024-04-26T23:52:45.225682+0000","last_peered":"2024-04-26T23:52:45.225682+0000","last_clean":"2024-04-26T23:52:45.225682+0000","last_became_active":"2024-04-26T23:52:44.230796+0000","last_became_peered":"2024-04-26T23:52:44.230796+0000","last_unstale":"2024-04-26T23:52:45.225682+0000","last_undegraded":"2024-04-26T23:52:45.225682+0000","last_fullsized":"2024-04-26T23:52:45.225682+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:15.524822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.233267+0000","last_change":"2024-04-26T23:52:45.232017+0000","last_active":"2024-04-26T23:52:45.233267+0000","last_peered":"2024-04-26T23:52:45.233267+0000","last_clean":"2024-04-26T23:52:45.233267+0000","last_became_active":"2024-04-26T23:52:45.231701+0000","last_became_peered":"2024-04-26T23:52:45.231701+0000","last_unstale":"2024-04-26T23:52:45.233267+0000","last_undegraded":"2024-04-26T23:52:45.233267+0000","last_fullsized":"2024-04-26T23:52:45.233267+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:40.187504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"160'3","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258719+0000","last_change":"2024-04-26T23:52:44.234204+0000","last_active":"2024-04-26T23:52:45.258719+0000","last_peered":"2024-04-26T23:52:45.258719+0000","last_clean":"2024-04-26T23:52:45.258719+0000","last_became_active":"2024-04-26T23:52:44.233586+0000","last_became_peered":"2024-04-26T23:52:44.233586+0000","last_unstale":"2024-04-26T23:52:45.258719+0000","last_undegraded":"2024-04-26T23:52:45.258719+0000","last_fullsized":"2024-04-26T23:52:45.258719+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:29.060684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259322+0000","last_change":"2024-04-26T23:52:45.252361+0000","last_active":"2024-04-26T23:52:45.259322+0000","last_peered":"2024-04-26T23:52:45.259322+0000","last_clean":"2024-04-26T23:52:45.259322+0000","last_became_active":"2024-04-26T23:52:45.252066+0000","last_became_peered":"2024-04-26T23:52:45.252066+0000","last_unstale":"2024-04-26T23:52:45.259322+0000","last_undegraded":"2024-04-26T23:52:45.259322+0000","last_fullsized":"2024-04-26T23:52:45.259322+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.403097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"160'6","reported_seq":18,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.271276+0000","last_change":"2024-04-26T23:52:44.230451+0000","last_active":"2024-04-26T23:52:45.271276+0000","last_peered":"2024-04-26T23:52:45.271276+0000","last_clean":"2024-04-26T23:52:45.271276+0000","last_became_active":"2024-04-26T23:52:44.230159+0000","last_became_peered":"2024-04-26T23:52:44.230159+0000","last_unstale":"2024-04-26T23:52:45.271276+0000","last_undegraded":"2024-04-26T23:52:45.271276+0000","last_fullsized":"2024-04-26T23:52:45.271276+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:17.580809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252087+0000","last_change":"2024-04-26T23:52:45.250809+0000","last_active":"2024-04-26T23:52:45.252087+0000","last_peered":"2024-04-26T23:52:45.252087+0000","last_clean":"2024-04-26T23:52:45.252087+0000","last_became_active":"2024-04-26T23:52:45.250356+0000","last_became_peered":"2024-04-26T23:52:45.250356+0000","last_unstale":"2024-04-26T23:52:45.252087+0000","last_undegraded":"2024-04-26T23:52:45.252087+0000","last_fullsized":"2024-04-26T23:52:45.252087+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:59.889184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267915+0000","last_change":"2024-04-26T23:52:44.239118+0000","last_active":"2024-04-26T23:52:45.267915+0000","last_peered":"2024-04-26T23:52:45.267915+0000","last_clean":"2024-04-26T23:52:45.267915+0000","last_became_active":"2024-04-26T23:52:44.235855+0000","last_became_peered":"2024-04-26T23:52:44.235855+0000","last_unstale":"2024-04-26T23:52:45.267915+0000","last_undegraded":"2024-04-26T23:52:45.267915+0000","last_fullsized":"2024-04-26T23:52:45.267915+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:19:17.038463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256911+0000","last_change":"2024-04-26T23:52:45.242412+0000","last_active":"2024-04-26T23:52:45.256911+0000","last_peered":"2024-04-26T23:52:45.256911+0000","last_clean":"2024-04-26T23:52:45.256911+0000","last_became_active":"2024-04-26T23:52:45.241599+0000","last_became_peered":"2024-04-26T23:52:45.241599+0000","last_unstale":"2024-04-26T23:52:45.256911+0000","last_undegraded":"2024-04-26T23:52:45.256911+0000","last_fullsized":"2024-04-26T23:52:45.256911+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:26.921870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.213191+0000","last_change":"2024-04-26T23:52:44.236449+0000","last_active":"2024-04-26T23:52:45.213191+0000","last_peered":"2024-04-26T23:52:45.213191+0000","last_clean":"2024-04-26T23:52:45.213191+0000","last_became_active":"2024-04-26T23:52:44.234867+0000","last_became_peered":"2024-04-26T23:52:44.234867+0000","last_unstale":"2024-04-26T23:52:45.213191+0000","last_undegraded":"2024-04-26T23:52:45.213191+0000","last_fullsized":"2024-04-26T23:52:45.213191+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:50.614442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240768+0000","last_change":"2024-04-26T23:52:45.236621+0000","last_active":"2024-04-26T23:52:45.240768+0000","last_peered":"2024-04-26T23:52:45.240768+0000","last_clean":"2024-04-26T23:52:45.240768+0000","last_became_active":"2024-04-26T23:52:45.236305+0000","last_became_peered":"2024-04-26T23:52:45.236305+0000","last_unstale":"2024-04-26T23:52:45.240768+0000","last_undegraded":"2024-04-26T23:52:45.240768+0000","last_fullsized":"2024-04-26T23:52:45.240768+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:38.441388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.224904+0000","last_change":"2024-04-26T23:52:44.235590+0000","last_active":"2024-04-26T23:52:45.224904+0000","last_peered":"2024-04-26T23:52:45.224904+0000","last_clean":"2024-04-26T23:52:45.224904+0000","last_became_active":"2024-04-26T23:52:44.234988+0000","last_became_peered":"2024-04-26T23:52:44.234988+0000","last_unstale":"2024-04-26T23:52:45.224904+0000","last_undegraded":"2024-04-26T23:52:45.224904+0000","last_fullsized":"2024-04-26T23:52:45.224904+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:11.502021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259055+0000","last_change":"2024-04-26T23:52:45.246185+0000","last_active":"2024-04-26T23:52:45.259055+0000","last_peered":"2024-04-26T23:52:45.259055+0000","last_clean":"2024-04-26T23:52:45.259055+0000","last_became_active":"2024-04-26T23:52:45.245903+0000","last_became_peered":"2024-04-26T23:52:45.245903+0000","last_unstale":"2024-04-26T23:52:45.259055+0000","last_undegraded":"2024-04-26T23:52:45.259055+0000","last_fullsized":"2024-04-26T23:52:45.259055+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:06.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"160'8","reported_seq":20,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.269576+0000","last_change":"2024-04-26T23:52:44.232473+0000","last_active":"2024-04-26T23:52:45.269576+0000","last_peered":"2024-04-26T23:52:45.269576+0000","last_clean":"2024-04-26T23:52:45.269576+0000","last_became_active":"2024-04-26T23:52:44.232148+0000","last_became_peered":"2024-04-26T23:52:44.232148+0000","last_unstale":"2024-04-26T23:52:45.269576+0000","last_undegraded":"2024-04-26T23:52:45.269576+0000","last_fullsized":"2024-04-26T23:52:45.269576+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:37.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.255655+0000","last_change":"2024-04-26T23:52:45.241311+0000","last_active":"2024-04-26T23:52:45.255655+0000","last_peered":"2024-04-26T23:52:45.255655+0000","last_clean":"2024-04-26T23:52:45.255655+0000","last_became_active":"2024-04-26T23:52:45.239714+0000","last_became_peered":"2024-04-26T23:52:45.239714+0000","last_unstale":"2024-04-26T23:52:45.255655+0000","last_undegraded":"2024-04-26T23:52:45.255655+0000","last_fullsized":"2024-04-26T23:52:45.255655+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:55.647019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.265418+0000","last_change":"2024-04-26T23:52:44.225669+0000","last_active":"2024-04-26T23:52:45.265418+0000","last_peered":"2024-04-26T23:52:45.265418+0000","last_clean":"2024-04-26T23:52:45.265418+0000","last_became_active":"2024-04-26T23:52:44.225376+0000","last_became_peered":"2024-04-26T23:52:44.225376+0000","last_unstale":"2024-04-26T23:52:45.265418+0000","last_undegraded":"2024-04-26T23:52:45.265418+0000","last_fullsized":"2024-04-26T23:52:45.265418+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:07.859563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248201+0000","last_change":"2024-04-26T23:52:45.243498+0000","last_active":"2024-04-26T23:52:45.248201+0000","last_peered":"2024-04-26T23:52:45.248201+0000","last_clean":"2024-04-26T23:52:45.248201+0000","last_became_active":"2024-04-26T23:52:45.243112+0000","last_became_peered":"2024-04-26T23:52:45.243112+0000","last_unstale":"2024-04-26T23:52:45.248201+0000","last_undegraded":"2024-04-26T23:52:45.248201+0000","last_fullsized":"2024-04-26T23:52:45.248201+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:03.276791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.268860+0000","last_change":"2024-04-26T23:52:44.230807+0000","last_active":"2024-04-26T23:52:45.268860+0000","last_peered":"2024-04-26T23:52:45.268860+0000","last_clean":"2024-04-26T23:52:45.268860+0000","last_became_active":"2024-04-26T23:52:44.230466+0000","last_became_peered":"2024-04-26T23:52:44.230466+0000","last_unstale":"2024-04-26T23:52:45.268860+0000","last_undegraded":"2024-04-26T23:52:45.268860+0000","last_fullsized":"2024-04-26T23:52:45.268860+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.796144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240439+0000","last_change":"2024-04-26T23:52:45.237656+0000","last_active":"2024-04-26T23:52:45.240439+0000","last_peered":"2024-04-26T23:52:45.240439+0000","last_clean":"2024-04-26T23:52:45.240439+0000","last_became_active":"2024-04-26T23:52:45.237312+0000","last_became_peered":"2024-04-26T23:52:45.237312+0000","last_unstale":"2024-04-26T23:52:45.240439+0000","last_undegraded":"2024-04-26T23:52:45.240439+0000","last_fullsized":"2024-04-26T23:52:45.240439+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264198+0000","last_change":"2024-04-26T23:52:44.224080+0000","last_active":"2024-04-26T23:52:45.264198+0000","last_peered":"2024-04-26T23:52:45.264198+0000","last_clean":"2024-04-26T23:52:45.264198+0000","last_became_active":"2024-04-26T23:52:44.223735+0000","last_became_peered":"2024-04-26T23:52:44.223735+0000","last_unstale":"2024-04-26T23:52:45.264198+0000","last_undegraded":"2024-04-26T23:52:45.264198+0000","last_fullsized":"2024-04-26T23:52:45.264198+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:59.418025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.234589+0000","last_change":"2024-04-26T23:52:45.233392+0000","last_active":"2024-04-26T23:52:45.234589+0000","last_peered":"2024-04-26T23:52:45.234589+0000","last_clean":"2024-04-26T23:52:45.234589+0000","last_became_active":"2024-04-26T23:52:45.232997+0000","last_became_peered":"2024-04-26T23:52:45.232997+0000","last_unstale":"2024-04-26T23:52:45.234589+0000","last_undegraded":"2024-04-26T23:52:45.234589+0000","last_fullsized":"2024-04-26T23:52:45.234589+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240918+0000","last_change":"2024-04-26T23:52:44.240014+0000","last_active":"2024-04-26T23:52:45.240918+0000","last_peered":"2024-04-26T23:52:45.240918+0000","last_clean":"2024-04-26T23:52:45.240918+0000","last_became_active":"2024-04-26T23:52:44.239731+0000","last_became_peered":"2024-04-26T23:52:44.239731+0000","last_unstale":"2024-04-26T23:52:45.240918+0000","last_undegraded":"2024-04-26T23:52:45.240918+0000","last_fullsized":"2024-04-26T23:52:45.240918+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:35.591248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245582+0000","last_change":"2024-04-26T23:52:45.244232+0000","last_active":"2024-04-26T23:52:45.245582+0000","last_peered":"2024-04-26T23:52:45.245582+0000","last_clean":"2024-04-26T23:52:45.245582+0000","last_became_active":"2024-04-26T23:52:45.243960+0000","last_became_peered":"2024-04-26T23:52:45.243960+0000","last_unstale":"2024-04-26T23:52:45.245582+0000","last_undegraded":"2024-04-26T23:52:45.245582+0000","last_fullsized":"2024-04-26T23:52:45.245582+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:12.536494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267458+0000","last_change":"2024-04-26T23:52:44.236986+0000","last_active":"2024-04-26T23:52:45.267458+0000","last_peered":"2024-04-26T23:52:45.267458+0000","last_clean":"2024-04-26T23:52:45.267458+0000","last_became_active":"2024-04-26T23:52:44.235500+0000","last_became_peered":"2024-04-26T23:52:44.235500+0000","last_unstale":"2024-04-26T23:52:45.267458+0000","last_undegraded":"2024-04-26T23:52:45.267458+0000","last_fullsized":"2024-04-26T23:52:45.267458+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:37.437126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258871+0000","last_change":"2024-04-26T23:52:45.256179+0000","last_active":"2024-04-26T23:52:45.258871+0000","last_peered":"2024-04-26T23:52:45.258871+0000","last_clean":"2024-04-26T23:52:45.258871+0000","last_became_active":"2024-04-26T23:52:45.255524+0000","last_became_peered":"2024-04-26T23:52:45.255524+0000","last_unstale":"2024-04-26T23:52:45.258871+0000","last_undegraded":"2024-04-26T23:52:45.258871+0000","last_fullsized":"2024-04-26T23:52:45.258871+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:20.106084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223999+0000","last_change":"2024-04-26T23:52:44.233101+0000","last_active":"2024-04-26T23:52:45.223999+0000","last_peered":"2024-04-26T23:52:45.223999+0000","last_clean":"2024-04-26T23:52:45.223999+0000","last_became_active":"2024-04-26T23:52:44.230391+0000","last_became_peered":"2024-04-26T23:52:44.230391+0000","last_unstale":"2024-04-26T23:52:45.223999+0000","last_undegraded":"2024-04-26T23:52:45.223999+0000","last_fullsized":"2024-04-26T23:52:45.223999+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:25.904286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.234053+0000","last_change":"2024-04-26T23:52:45.231576+0000","last_active":"2024-04-26T23:52:45.234053+0000","last_peered":"2024-04-26T23:52:45.234053+0000","last_clean":"2024-04-26T23:52:45.234053+0000","last_became_active":"2024-04-26T23:52:45.231222+0000","last_became_peered":"2024-04-26T23:52:45.231222+0000","last_unstale":"2024-04-26T23:52:45.234053+0000","last_undegraded":"2024-04-26T23:52:45.234053+0000","last_fullsized":"2024-04-26T23:52:45.234053+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:51.632026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225889+0000","last_change":"2024-04-26T23:52:44.235957+0000","last_active":"2024-04-26T23:52:45.225889+0000","last_peered":"2024-04-26T23:52:45.225889+0000","last_clean":"2024-04-26T23:52:45.225889+0000","last_became_active":"2024-04-26T23:52:44.235007+0000","last_became_peered":"2024-04-26T23:52:44.235007+0000","last_unstale":"2024-04-26T23:52:45.225889+0000","last_undegraded":"2024-04-26T23:52:45.225889+0000","last_fullsized":"2024-04-26T23:52:45.225889+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:25.124752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248440+0000","last_change":"2024-04-26T23:52:45.243385+0000","last_active":"2024-04-26T23:52:45.248440+0000","last_peered":"2024-04-26T23:52:45.248440+0000","last_clean":"2024-04-26T23:52:45.248440+0000","last_became_active":"2024-04-26T23:52:45.243058+0000","last_became_peered":"2024-04-26T23:52:45.243058+0000","last_unstale":"2024-04-26T23:52:45.248440+0000","last_undegraded":"2024-04-26T23:52:45.248440+0000","last_fullsized":"2024-04-26T23:52:45.248440+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:29.011766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.221321+0000","last_change":"2024-04-26T23:52:44.236117+0000","last_active":"2024-04-26T23:52:45.221321+0000","last_peered":"2024-04-26T23:52:45.221321+0000","last_clean":"2024-04-26T23:52:45.221321+0000","last_became_active":"2024-04-26T23:52:44.235727+0000","last_became_peered":"2024-04-26T23:52:44.235727+0000","last_unstale":"2024-04-26T23:52:45.221321+0000","last_undegraded":"2024-04-26T23:52:45.221321+0000","last_fullsized":"2024-04-26T23:52:45.221321+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:51.528844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.249675+0000","last_change":"2024-04-26T23:52:45.246522+0000","last_active":"2024-04-26T23:52:45.249675+0000","last_peered":"2024-04-26T23:52:45.249675+0000","last_clean":"2024-04-26T23:52:45.249675+0000","last_became_active":"2024-04-26T23:52:45.246207+0000","last_became_peered":"2024-04-26T23:52:45.246207+0000","last_unstale":"2024-04-26T23:52:45.249675+0000","last_undegraded":"2024-04-26T23:52:45.249675+0000","last_fullsized":"2024-04-26T23:52:45.249675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264640+0000","last_change":"2024-04-26T23:52:44.235352+0000","last_active":"2024-04-26T23:52:45.264640+0000","last_peered":"2024-04-26T23:52:45.264640+0000","last_clean":"2024-04-26T23:52:45.264640+0000","last_became_active":"2024-04-26T23:52:44.234140+0000","last_became_peered":"2024-04-26T23:52:44.234140+0000","last_unstale":"2024-04-26T23:52:45.264640+0000","last_undegraded":"2024-04-26T23:52:45.264640+0000","last_fullsized":"2024-04-26T23:52:45.264640+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:58.266715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257786+0000","last_change":"2024-04-26T23:52:45.243625+0000","last_active":"2024-04-26T23:52:45.257786+0000","last_peered":"2024-04-26T23:52:45.257786+0000","last_clean":"2024-04-26T23:52:45.257786+0000","last_became_active":"2024-04-26T23:52:45.243325+0000","last_became_peered":"2024-04-26T23:52:45.243325+0000","last_unstale":"2024-04-26T23:52:45.257786+0000","last_undegraded":"2024-04-26T23:52:45.257786+0000","last_fullsized":"2024-04-26T23:52:45.257786+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:52.546427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.222993+0000","last_change":"2024-04-26T23:52:44.227117+0000","last_active":"2024-04-26T23:52:45.222993+0000","last_peered":"2024-04-26T23:52:45.222993+0000","last_clean":"2024-04-26T23:52:45.222993+0000","last_became_active":"2024-04-26T23:52:44.226788+0000","last_became_peered":"2024-04-26T23:52:44.226788+0000","last_unstale":"2024-04-26T23:52:45.222993+0000","last_undegraded":"2024-04-26T23:52:45.222993+0000","last_fullsized":"2024-04-26T23:52:45.222993+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:55.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248001+0000","last_change":"2024-04-26T23:52:45.241229+0000","last_active":"2024-04-26T23:52:45.248001+0000","last_peered":"2024-04-26T23:52:45.248001+0000","last_clean":"2024-04-26T23:52:45.248001+0000","last_became_active":"2024-04-26T23:52:45.240404+0000","last_became_peered":"2024-04-26T23:52:45.240404+0000","last_unstale":"2024-04-26T23:52:45.248001+0000","last_undegraded":"2024-04-26T23:52:45.248001+0000","last_fullsized":"2024-04-26T23:52:45.248001+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:00.720006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223041+0000","last_change":"2024-04-26T23:52:44.235908+0000","last_active":"2024-04-26T23:52:45.223041+0000","last_peered":"2024-04-26T23:52:45.223041+0000","last_clean":"2024-04-26T23:52:45.223041+0000","last_became_active":"2024-04-26T23:52:44.232624+0000","last_became_peered":"2024-04-26T23:52:44.232624+0000","last_unstale":"2024-04-26T23:52:45.223041+0000","last_undegraded":"2024-04-26T23:52:45.223041+0000","last_fullsized":"2024-04-26T23:52:45.223041+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:15.434038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256462+0000","last_change":"2024-04-26T23:52:45.241877+0000","last_active":"2024-04-26T23:52:45.256462+0000","last_peered":"2024-04-26T23:52:45.256462+0000","last_clean":"2024-04-26T23:52:45.256462+0000","last_became_active":"2024-04-26T23:52:45.241248+0000","last_became_peered":"2024-04-26T23:52:45.241248+0000","last_unstale":"2024-04-26T23:52:45.256462+0000","last_undegraded":"2024-04-26T23:52:45.256462+0000","last_fullsized":"2024-04-26T23:52:45.256462+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:57.976389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.229464+0000","last_change":"2024-04-26T23:52:44.235880+0000","last_active":"2024-04-26T23:52:45.229464+0000","last_peered":"2024-04-26T23:52:45.229464+0000","last_clean":"2024-04-26T23:52:45.229464+0000","last_became_active":"2024-04-26T23:52:44.235215+0000","last_became_peered":"2024-04-26T23:52:44.235215+0000","last_unstale":"2024-04-26T23:52:45.229464+0000","last_undegraded":"2024-04-26T23:52:45.229464+0000","last_fullsized":"2024-04-26T23:52:45.229464+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:38.803564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240175+0000","last_change":"2024-04-26T23:52:45.233170+0000","last_active":"2024-04-26T23:52:45.240175+0000","last_peered":"2024-04-26T23:52:45.240175+0000","last_clean":"2024-04-26T23:52:45.240175+0000","last_became_active":"2024-04-26T23:52:45.229793+0000","last_became_peered":"2024-04-26T23:52:45.229793+0000","last_unstale":"2024-04-26T23:52:45.240175+0000","last_undegraded":"2024-04-26T23:52:45.240175+0000","last_fullsized":"2024-04-26T23:52:45.240175+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:26.040678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.211996+0000","last_change":"2024-04-26T23:52:44.225856+0000","last_active":"2024-04-26T23:52:45.211996+0000","last_peered":"2024-04-26T23:52:45.211996+0000","last_clean":"2024-04-26T23:52:45.211996+0000","last_became_active":"2024-04-26T23:52:44.223912+0000","last_became_peered":"2024-04-26T23:52:44.223912+0000","last_unstale":"2024-04-26T23:52:45.211996+0000","last_undegraded":"2024-04-26T23:52:45.211996+0000","last_fullsized":"2024-04-26T23:52:45.211996+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:34.223513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248255+0000","last_change":"2024-04-26T23:52:45.242507+0000","last_active":"2024-04-26T23:52:45.248255+0000","last_peered":"2024-04-26T23:52:45.248255+0000","last_clean":"2024-04-26T23:52:45.248255+0000","last_became_active":"2024-04-26T23:52:45.242209+0000","last_became_peered":"2024-04-26T23:52:45.242209+0000","last_unstale":"2024-04-26T23:52:45.248255+0000","last_undegraded":"2024-04-26T23:52:45.248255+0000","last_fullsized":"2024-04-26T23:52:45.248255+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:06.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.213656+0000","last_change":"2024-04-26T23:52:44.230071+0000","last_active":"2024-04-26T23:52:45.213656+0000","last_peered":"2024-04-26T23:52:45.213656+0000","last_clean":"2024-04-26T23:52:45.213656+0000","last_became_active":"2024-04-26T23:52:44.229296+0000","last_became_peered":"2024-04-26T23:52:44.229296+0000","last_unstale":"2024-04-26T23:52:45.213656+0000","last_undegraded":"2024-04-26T23:52:45.213656+0000","last_fullsized":"2024-04-26T23:52:45.213656+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:50.316116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243264+0000","last_change":"2024-04-26T23:52:45.239068+0000","last_active":"2024-04-26T23:52:45.243264+0000","last_peered":"2024-04-26T23:52:45.243264+0000","last_clean":"2024-04-26T23:52:45.243264+0000","last_became_active":"2024-04-26T23:52:45.238378+0000","last_became_peered":"2024-04-26T23:52:45.238378+0000","last_unstale":"2024-04-26T23:52:45.243264+0000","last_undegraded":"2024-04-26T23:52:45.243264+0000","last_fullsized":"2024-04-26T23:52:45.243264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.926047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.224513+0000","last_change":"2024-04-26T23:52:44.235635+0000","last_active":"2024-04-26T23:52:45.224513+0000","last_peered":"2024-04-26T23:52:45.224513+0000","last_clean":"2024-04-26T23:52:45.224513+0000","last_became_active":"2024-04-26T23:52:44.235366+0000","last_became_peered":"2024-04-26T23:52:44.235366+0000","last_unstale":"2024-04-26T23:52:45.224513+0000","last_undegraded":"2024-04-26T23:52:45.224513+0000","last_fullsized":"2024-04-26T23:52:45.224513+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:23.365128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259112+0000","last_change":"2024-04-26T23:52:45.250689+0000","last_active":"2024-04-26T23:52:45.259112+0000","last_peered":"2024-04-26T23:52:45.259112+0000","last_clean":"2024-04-26T23:52:45.259112+0000","last_became_active":"2024-04-26T23:52:45.250385+0000","last_became_peered":"2024-04-26T23:52:45.250385+0000","last_unstale":"2024-04-26T23:52:45.259112+0000","last_undegraded":"2024-04-26T23:52:45.259112+0000","last_fullsized":"2024-04-26T23:52:45.259112+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:35.572422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.210958+0000","last_change":"2024-04-26T23:52:44.232443+0000","last_active":"2024-04-26T23:52:45.210958+0000","last_peered":"2024-04-26T23:52:45.210958+0000","last_clean":"2024-04-26T23:52:45.210958+0000","last_became_active":"2024-04-26T23:52:44.232159+0000","last_became_peered":"2024-04-26T23:52:44.232159+0000","last_unstale":"2024-04-26T23:52:45.210958+0000","last_undegraded":"2024-04-26T23:52:45.210958+0000","last_fullsized":"2024-04-26T23:52:45.210958+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:27.994182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248988+0000","last_change":"2024-04-26T23:52:45.245828+0000","last_active":"2024-04-26T23:52:45.248988+0000","last_peered":"2024-04-26T23:52:45.248988+0000","last_clean":"2024-04-26T23:52:45.248988+0000","last_became_active":"2024-04-26T23:52:45.245416+0000","last_became_peered":"2024-04-26T23:52:45.245416+0000","last_unstale":"2024-04-26T23:52:45.248988+0000","last_undegraded":"2024-04-26T23:52:45.248988+0000","last_fullsized":"2024-04-26T23:52:45.248988+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:59.284298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223108+0000","last_change":"2024-04-26T23:52:44.227316+0000","last_active":"2024-04-26T23:52:45.223108+0000","last_peered":"2024-04-26T23:52:45.223108+0000","last_clean":"2024-04-26T23:52:45.223108+0000","last_became_active":"2024-04-26T23:52:44.227016+0000","last_became_peered":"2024-04-26T23:52:44.227016+0000","last_unstale":"2024-04-26T23:52:45.223108+0000","last_undegraded":"2024-04-26T23:52:45.223108+0000","last_fullsized":"2024-04-26T23:52:45.223108+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:19.088500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":427,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.220721+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:52:45.220721+0000","last_peered":"2024-04-26T23:52:45.220721+0000","last_clean":"2024-04-26T23:52:45.220721+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:52:45.220721+0000","last_undegraded":"2024-04-26T23:52:45.220721+0000","last_fullsized":"2024-04-26T23:52:45.220721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51772,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320068,"statfs":{"total":96636764160,"available":96583749632,"internally_reserved":0,"allocated":2605056,"data_stored":1537340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2011136,"data_stored":946972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51200,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320640,"statfs":{"total":96636764160,"available":96584335360,"internally_reserved":0,"allocated":2019328,"data_stored":947096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59532,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312308,"statfs":{"total":96636764160,"available":96575803392,"internally_reserved":0,"allocated":2027520,"data_stored":947736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59532,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312308,"statfs":{"total":96636764160,"available":96575803392,"internally_reserved":0,"allocated":2031616,"data_stored":949188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59512,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":2011136,"data_stored":946972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575040,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59524,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312316,"statfs":{"total":96636764160,"available":96575811584,"internally_reserved":0,"allocated":2023424,"data_stored":948584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575040,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55948,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315892,"statfs":{"total":96636764160,"available":96579473408,"internally_reserved":0,"allocated":2613248,"data_stored":1537444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:52:49.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:52:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.989+0000 7f1e7b000640 1 Processor -- start 2024-04-26T23:52:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.990+0000 7f1e7b000640 1 -- start start 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.991+0000 7f1e7b000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e740d3f60 0x7f1e7416a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.991+0000 7f1e7b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e740cfd60 con 0x7f1e7415f4b0 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.991+0000 7f1e7b000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e740b55d0 con 0x7f1e740b5140 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.991+0000 7f1e7b000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1e7415f890 con 0x7f1e740d3f60 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.992+0000 7f1e73e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1e7415f4b0 0x7f1e740d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54610/0 (socket says 172.21.15.46:54610) 2024-04-26T23:52:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.992+0000 7f1e73e00640 1 -- 172.21.15.46:0/4134015598 learned_addr learned my addr 172.21.15.46:0/4134015598 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.992+0000 7f1e79000640 1 --2- 172.21.15.46:0/4134015598 >> v2:172.21.15.142:3300/0 conn(0x7f1e740d3f60 0x7f1e7416a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.992+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980774182 0 0) 0x7f1e64001110 con 0x7f1e7415f4b0 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.992+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e54003520 con 0x7f1e7415f4b0 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1794251666 0 0) 0x7f1e54003520 con 0x7f1e7415f4b0 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 >> v2:172.21.15.142:3300/0 conn(0x7f1e740d3f60 msgr2=0x7f1e7416a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 --2- 172.21.15.46:0/4134015598 >> v2:172.21.15.142:3300/0 conn(0x7f1e740d3f60 0x7f1e7416a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 >> v1:172.21.15.142:6789/0 conn(0x7f1e740b5140 legacy=0x7f1e740d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e740b58b0 con 0x7f1e7415f4b0 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1e64001840 con 0x7f1e7415f4b0 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 >> v1:172.21.15.46:6789/0 conn(0x7f1e7415f4b0 legacy=0x7f1e740d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e540039b0 0x7f1e54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 0x7f1e54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e540051c0 0x7f1e5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7415f890 con 0x7f1e54004800 2024-04-26T23:52:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5400c060 con 0x7f1e540039b0 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5400c260 con 0x7f1e540051c0 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.993+0000 7f1e73400640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 0x7f1e54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e79000640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e540051c0 0x7f1e5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e73400640 1 -- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e540051c0 msgr2=0x7f1e5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e73400640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e540051c0 0x7f1e5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e73400640 1 -- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e540039b0 msgr2=0x7f1e54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e73400640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e540039b0 0x7f1e54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e73400640 1 -- 172.21.15.46:0/4134015598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e54003520 con 0x7f1e54004800 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e79000640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e540051c0 0x7f1e5400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.994+0000 7f1e79000640 1 -- 172.21.15.46:0/4134015598 reap_dead start 2024-04-26T23:52:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.995+0000 7f1e73400640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 0x7f1e54004c80 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c001e50 tx=0x7f1e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5903ee3abe455df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.995+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c001a70 con 0x7f1e54004800 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.995+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c02ae40 con 0x7f1e54004800 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.995+0000 7f1e72a00640 1 -- 172.21.15.46:0/4134015598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c003bb0 con 0x7f1e54004800 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 -- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 msgr2=0x7f1e54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 0x7f1e54004c80 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c001e50 tx=0x7f1e54005830 comp rx=0 tx=0).stop 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 -- 172.21.15.46:0/4134015598 shutdown_connections 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 --2- 172.21.15.46:0/4134015598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e54004800 0x7f1e54004c80 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 -- 172.21.15.46:0/4134015598 >> 172.21.15.46:0/4134015598 conn(0x7f1e7408eb50 msgr2=0x7f1e7409e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 -- 172.21.15.46:0/4134015598 shutdown_connections 2024-04-26T23:52:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.996+0000 7f1e7b000640 1 -- 172.21.15.46:0/4134015598 wait complete. 2024-04-26T23:52:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.998+0000 7f1e7b000640 1 Processor -- start 2024-04-26T23:52:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.998+0000 7f1e7b000640 1 -- start start 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e54004800 0x7f1e741507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e74161340 0x7f1e740b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740b4680 con 0x7f1e740ca590 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740b5e70 con 0x7f1e54004800 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e7b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1e740d23d0 con 0x7f1e74161340 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50542/0 (socket says 172.21.15.46:50542) 2024-04-26T23:52:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:49.999+0000 7f1e73400640 1 -- 172.21.15.46:0/1854699317 learned_addr learned my addr 172.21.15.46:0/1854699317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 -- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e74161340 msgr2=0x7f1e740b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e74161340 0x7f1e740b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 -- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e54004800 msgr2=0x7f1e741507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e54004800 0x7f1e741507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 -- 172.21.15.46:0/1854699317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7415fd60 con 0x7f1e740ca590 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e73400640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f80 tx=0x7f1e5c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.000+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c040030 con 0x7f1e740ca590 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.001+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c02ac40 con 0x7f1e740ca590 2024-04-26T23:52:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.001+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c007db0 con 0x7f1e740ca590 2024-04-26T23:52:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.001+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e74160040 con 0x7f1e740ca590 2024-04-26T23:52:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.001+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e74160600 con 0x7f1e740ca590 2024-04-26T23:52:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.002+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e74069530 con 0x7f1e740ca590 2024-04-26T23:52:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.003+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e5c043070 con 0x7f1e740ca590 2024-04-26T23:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.004+0000 7f1e6be00640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.004+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f1e5c0abde0 con 0x7f1e740ca590 2024-04-26T23:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.004+0000 7f1e73e00640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.006+0000 7f1e73e00640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 0x7f1e5006ede0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1e640013d0 tx=0x7f1e64001d00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.009+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e5c031e20 con 0x7f1e740ca590 2024-04-26T23:52:50.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.213+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e740b4bd0 con 0x7f1e5006c920 2024-04-26T23:52:50.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.232+0000 7f1e6be00640 1 -- 172.21.15.46:0/1854699317 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f1e740b4bd0 con 0x7f1e5006c920 2024-04-26T23:52:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:50.240 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 msgr2=0x7f1e5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 0x7f1e5006ede0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1e640013d0 tx=0x7f1e64001d00 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 msgr2=0x7f1e74160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f80 tx=0x7f1e5c0014a0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 shutdown_connections 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e74161340 0x7f1e740b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e740ca590 0x7f1e74160e00 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 --2- 172.21.15.46:0/1854699317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e54004800 0x7f1e741507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 >> 172.21.15.46:0/1854699317 conn(0x7f1e7408eb50 msgr2=0x7f1e7406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 shutdown_connections 2024-04-26T23:52:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.237+0000 7f1e7b000640 1 -- 172.21.15.46:0/1854699317 wait complete. 2024-04-26T23:52:50.259 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2024-04-26T23:52:48.241450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448212,"kb_used_data":16936,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754526508,"statfs":{"total":773094113280,"available":772635144192,"internally_reserved":0,"allocated":17342464,"data_stored":8761332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":441503281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.056251"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245858+0000","last_change":"2024-04-26T23:52:45.242231+0000","last_active":"2024-04-26T23:52:45.245858+0000","last_peered":"2024-04-26T23:52:45.245858+0000","last_clean":"2024-04-26T23:52:45.245858+0000","last_became_active":"2024-04-26T23:52:45.241079+0000","last_became_peered":"2024-04-26T23:52:45.241079+0000","last_unstale":"2024-04-26T23:52:45.245858+0000","last_undegraded":"2024-04-26T23:52:45.245858+0000","last_fullsized":"2024-04-26T23:52:45.245858+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:25.635643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248365+0000","last_change":"2024-04-26T23:52:45.244575+0000","last_active":"2024-04-26T23:52:45.248365+0000","last_peered":"2024-04-26T23:52:45.248365+0000","last_clean":"2024-04-26T23:52:45.248365+0000","last_became_active":"2024-04-26T23:52:45.239682+0000","last_became_peered":"2024-04-26T23:52:45.239682+0000","last_unstale":"2024-04-26T23:52:45.248365+0000","last_undegraded":"2024-04-26T23:52:45.248365+0000","last_fullsized":"2024-04-26T23:52:45.248365+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:47.629201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.247308+0000","last_change":"2024-04-26T23:52:45.242937+0000","last_active":"2024-04-26T23:52:45.247308+0000","last_peered":"2024-04-26T23:52:45.247308+0000","last_clean":"2024-04-26T23:52:45.247308+0000","last_became_active":"2024-04-26T23:52:45.242649+0000","last_became_peered":"2024-04-26T23:52:45.242649+0000","last_unstale":"2024-04-26T23:52:45.247308+0000","last_undegraded":"2024-04-26T23:52:45.247308+0000","last_fullsized":"2024-04-26T23:52:45.247308+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:28.797918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239863+0000","last_change":"2024-04-26T23:52:45.232910+0000","last_active":"2024-04-26T23:52:45.239863+0000","last_peered":"2024-04-26T23:52:45.239863+0000","last_clean":"2024-04-26T23:52:45.239863+0000","last_became_active":"2024-04-26T23:52:45.229308+0000","last_became_peered":"2024-04-26T23:52:45.229308+0000","last_unstale":"2024-04-26T23:52:45.239863+0000","last_undegraded":"2024-04-26T23:52:45.239863+0000","last_fullsized":"2024-04-26T23:52:45.239863+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:06.298828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252906+0000","last_change":"2024-04-26T23:52:45.247689+0000","last_active":"2024-04-26T23:52:45.252906+0000","last_peered":"2024-04-26T23:52:45.252906+0000","last_clean":"2024-04-26T23:52:45.252906+0000","last_became_active":"2024-04-26T23:52:45.247141+0000","last_became_peered":"2024-04-26T23:52:45.247141+0000","last_unstale":"2024-04-26T23:52:45.252906+0000","last_undegraded":"2024-04-26T23:52:45.252906+0000","last_fullsized":"2024-04-26T23:52:45.252906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.447117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.246822+0000","last_change":"2024-04-26T23:52:45.239666+0000","last_active":"2024-04-26T23:52:45.246822+0000","last_peered":"2024-04-26T23:52:45.246822+0000","last_clean":"2024-04-26T23:52:45.246822+0000","last_became_active":"2024-04-26T23:52:45.239342+0000","last_became_peered":"2024-04-26T23:52:45.239342+0000","last_unstale":"2024-04-26T23:52:45.246822+0000","last_undegraded":"2024-04-26T23:52:45.246822+0000","last_fullsized":"2024-04-26T23:52:45.246822+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259361+0000","last_change":"2024-04-26T23:52:45.252760+0000","last_active":"2024-04-26T23:52:45.259361+0000","last_peered":"2024-04-26T23:52:45.259361+0000","last_clean":"2024-04-26T23:52:45.259361+0000","last_became_active":"2024-04-26T23:52:45.251340+0000","last_became_peered":"2024-04-26T23:52:45.251340+0000","last_unstale":"2024-04-26T23:52:45.259361+0000","last_undegraded":"2024-04-26T23:52:45.259361+0000","last_fullsized":"2024-04-26T23:52:45.259361+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257987+0000","last_change":"2024-04-26T23:52:45.243752+0000","last_active":"2024-04-26T23:52:45.257987+0000","last_peered":"2024-04-26T23:52:45.257987+0000","last_clean":"2024-04-26T23:52:45.257987+0000","last_became_active":"2024-04-26T23:52:45.240132+0000","last_became_peered":"2024-04-26T23:52:45.240132+0000","last_unstale":"2024-04-26T23:52:45.257987+0000","last_undegraded":"2024-04-26T23:52:45.257987+0000","last_fullsized":"2024-04-26T23:52:45.257987+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257973+0000","last_change":"2024-04-26T23:52:45.244323+0000","last_active":"2024-04-26T23:52:45.257973+0000","last_peered":"2024-04-26T23:52:45.257973+0000","last_clean":"2024-04-26T23:52:45.257973+0000","last_became_active":"2024-04-26T23:52:45.243563+0000","last_became_peered":"2024-04-26T23:52:45.243563+0000","last_unstale":"2024-04-26T23:52:45.257973+0000","last_undegraded":"2024-04-26T23:52:45.257973+0000","last_fullsized":"2024-04-26T23:52:45.257973+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257848+0000","last_change":"2024-04-26T23:52:45.242508+0000","last_active":"2024-04-26T23:52:45.257848+0000","last_peered":"2024-04-26T23:52:45.257848+0000","last_clean":"2024-04-26T23:52:45.257848+0000","last_became_active":"2024-04-26T23:52:45.240195+0000","last_became_peered":"2024-04-26T23:52:45.240195+0000","last_unstale":"2024-04-26T23:52:45.257848+0000","last_undegraded":"2024-04-26T23:52:45.257848+0000","last_fullsized":"2024-04-26T23:52:45.257848+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:24.382711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239139+0000","last_change":"2024-04-26T23:52:45.235466+0000","last_active":"2024-04-26T23:52:45.239139+0000","last_peered":"2024-04-26T23:52:45.239139+0000","last_clean":"2024-04-26T23:52:45.239139+0000","last_became_active":"2024-04-26T23:52:45.235188+0000","last_became_peered":"2024-04-26T23:52:45.235188+0000","last_unstale":"2024-04-26T23:52:45.239139+0000","last_undegraded":"2024-04-26T23:52:45.239139+0000","last_fullsized":"2024-04-26T23:52:45.239139+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.249684+0000","last_change":"2024-04-26T23:52:45.247006+0000","last_active":"2024-04-26T23:52:45.249684+0000","last_peered":"2024-04-26T23:52:45.249684+0000","last_clean":"2024-04-26T23:52:45.249684+0000","last_became_active":"2024-04-26T23:52:45.244443+0000","last_became_peered":"2024-04-26T23:52:45.244443+0000","last_unstale":"2024-04-26T23:52:45.249684+0000","last_undegraded":"2024-04-26T23:52:45.249684+0000","last_fullsized":"2024-04-26T23:52:45.249684+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:19:18.056046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.235495+0000","last_change":"2024-04-26T23:52:45.233370+0000","last_active":"2024-04-26T23:52:45.235495+0000","last_peered":"2024-04-26T23:52:45.235495+0000","last_clean":"2024-04-26T23:52:45.235495+0000","last_became_active":"2024-04-26T23:52:45.233086+0000","last_became_peered":"2024-04-26T23:52:45.233086+0000","last_unstale":"2024-04-26T23:52:45.235495+0000","last_undegraded":"2024-04-26T23:52:45.235495+0000","last_fullsized":"2024-04-26T23:52:45.235495+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:22.670229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252476+0000","last_change":"2024-04-26T23:52:45.246036+0000","last_active":"2024-04-26T23:52:45.252476+0000","last_peered":"2024-04-26T23:52:45.252476+0000","last_clean":"2024-04-26T23:52:45.252476+0000","last_became_active":"2024-04-26T23:52:45.245736+0000","last_became_peered":"2024-04-26T23:52:45.245736+0000","last_unstale":"2024-04-26T23:52:45.252476+0000","last_undegraded":"2024-04-26T23:52:45.252476+0000","last_fullsized":"2024-04-26T23:52:45.252476+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:18.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256907+0000","last_change":"2024-04-26T23:52:45.254895+0000","last_active":"2024-04-26T23:52:45.256907+0000","last_peered":"2024-04-26T23:52:45.256907+0000","last_clean":"2024-04-26T23:52:45.256907+0000","last_became_active":"2024-04-26T23:52:45.254555+0000","last_became_peered":"2024-04-26T23:52:45.254555+0000","last_unstale":"2024-04-26T23:52:45.256907+0000","last_undegraded":"2024-04-26T23:52:45.256907+0000","last_fullsized":"2024-04-26T23:52:45.256907+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:06.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239578+0000","last_change":"2024-04-26T23:52:45.237436+0000","last_active":"2024-04-26T23:52:45.239578+0000","last_peered":"2024-04-26T23:52:45.239578+0000","last_clean":"2024-04-26T23:52:45.239578+0000","last_became_active":"2024-04-26T23:52:45.236766+0000","last_became_peered":"2024-04-26T23:52:45.236766+0000","last_unstale":"2024-04-26T23:52:45.239578+0000","last_undegraded":"2024-04-26T23:52:45.239578+0000","last_fullsized":"2024-04-26T23:52:45.239578+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:43.676634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.254623+0000","last_change":"2024-04-26T23:52:45.237604+0000","last_active":"2024-04-26T23:52:45.254623+0000","last_peered":"2024-04-26T23:52:45.254623+0000","last_clean":"2024-04-26T23:52:45.254623+0000","last_became_active":"2024-04-26T23:52:45.237338+0000","last_became_peered":"2024-04-26T23:52:45.237338+0000","last_unstale":"2024-04-26T23:52:45.254623+0000","last_undegraded":"2024-04-26T23:52:45.254623+0000","last_fullsized":"2024-04-26T23:52:45.254623+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:04.278595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.255856+0000","last_change":"2024-04-26T23:52:45.254819+0000","last_active":"2024-04-26T23:52:45.255856+0000","last_peered":"2024-04-26T23:52:45.255856+0000","last_clean":"2024-04-26T23:52:45.255856+0000","last_became_active":"2024-04-26T23:52:45.254425+0000","last_became_peered":"2024-04-26T23:52:45.254425+0000","last_unstale":"2024-04-26T23:52:45.255856+0000","last_undegraded":"2024-04-26T23:52:45.255856+0000","last_fullsized":"2024-04-26T23:52:45.255856+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:09.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258202+0000","last_change":"2024-04-26T23:52:45.256946+0000","last_active":"2024-04-26T23:52:45.258202+0000","last_peered":"2024-04-26T23:52:45.258202+0000","last_clean":"2024-04-26T23:52:45.258202+0000","last_became_active":"2024-04-26T23:52:45.256637+0000","last_became_peered":"2024-04-26T23:52:45.256637+0000","last_unstale":"2024-04-26T23:52:45.258202+0000","last_undegraded":"2024-04-26T23:52:45.258202+0000","last_fullsized":"2024-04-26T23:52:45.258202+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:14.669687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243055+0000","last_change":"2024-04-26T23:52:45.237343+0000","last_active":"2024-04-26T23:52:45.243055+0000","last_peered":"2024-04-26T23:52:45.243055+0000","last_clean":"2024-04-26T23:52:45.243055+0000","last_became_active":"2024-04-26T23:52:45.236807+0000","last_became_peered":"2024-04-26T23:52:45.236807+0000","last_unstale":"2024-04-26T23:52:45.243055+0000","last_undegraded":"2024-04-26T23:52:45.243055+0000","last_fullsized":"2024-04-26T23:52:45.243055+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:53.556520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243306+0000","last_change":"2024-04-26T23:52:45.239580+0000","last_active":"2024-04-26T23:52:45.243306+0000","last_peered":"2024-04-26T23:52:45.243306+0000","last_clean":"2024-04-26T23:52:45.243306+0000","last_became_active":"2024-04-26T23:52:45.239259+0000","last_became_peered":"2024-04-26T23:52:45.239259+0000","last_unstale":"2024-04-26T23:52:45.243306+0000","last_undegraded":"2024-04-26T23:52:45.243306+0000","last_fullsized":"2024-04-26T23:52:45.243306+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:02.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.237075+0000","last_change":"2024-04-26T23:52:45.233630+0000","last_active":"2024-04-26T23:52:45.237075+0000","last_peered":"2024-04-26T23:52:45.237075+0000","last_clean":"2024-04-26T23:52:45.237075+0000","last_became_active":"2024-04-26T23:52:45.233339+0000","last_became_peered":"2024-04-26T23:52:45.233339+0000","last_unstale":"2024-04-26T23:52:45.237075+0000","last_undegraded":"2024-04-26T23:52:45.237075+0000","last_fullsized":"2024-04-26T23:52:45.237075+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:03.577394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.254033+0000","last_change":"2024-04-26T23:52:45.248507+0000","last_active":"2024-04-26T23:52:45.254033+0000","last_peered":"2024-04-26T23:52:45.254033+0000","last_clean":"2024-04-26T23:52:45.254033+0000","last_became_active":"2024-04-26T23:52:45.248201+0000","last_became_peered":"2024-04-26T23:52:45.248201+0000","last_unstale":"2024-04-26T23:52:45.254033+0000","last_undegraded":"2024-04-26T23:52:45.254033+0000","last_fullsized":"2024-04-26T23:52:45.254033+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:16.451622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239612+0000","last_change":"2024-04-26T23:52:45.237696+0000","last_active":"2024-04-26T23:52:45.239612+0000","last_peered":"2024-04-26T23:52:45.239612+0000","last_clean":"2024-04-26T23:52:45.239612+0000","last_became_active":"2024-04-26T23:52:45.237157+0000","last_became_peered":"2024-04-26T23:52:45.237157+0000","last_unstale":"2024-04-26T23:52:45.239612+0000","last_undegraded":"2024-04-26T23:52:45.239612+0000","last_fullsized":"2024-04-26T23:52:45.239612+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:33.160098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.239400+0000","last_change":"2024-04-26T23:52:45.235282+0000","last_active":"2024-04-26T23:52:45.239400+0000","last_peered":"2024-04-26T23:52:45.239400+0000","last_clean":"2024-04-26T23:52:45.239400+0000","last_became_active":"2024-04-26T23:52:45.234949+0000","last_became_peered":"2024-04-26T23:52:45.234949+0000","last_unstale":"2024-04-26T23:52:45.239400+0000","last_undegraded":"2024-04-26T23:52:45.239400+0000","last_fullsized":"2024-04-26T23:52:45.239400+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:44.200196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258406+0000","last_change":"2024-04-26T23:52:45.249751+0000","last_active":"2024-04-26T23:52:45.258406+0000","last_peered":"2024-04-26T23:52:45.258406+0000","last_clean":"2024-04-26T23:52:45.258406+0000","last_became_active":"2024-04-26T23:52:45.249470+0000","last_became_peered":"2024-04-26T23:52:45.249470+0000","last_unstale":"2024-04-26T23:52:45.258406+0000","last_undegraded":"2024-04-26T23:52:45.258406+0000","last_fullsized":"2024-04-26T23:52:45.258406+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:33.930128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.236688+0000","last_change":"2024-04-26T23:52:45.231174+0000","last_active":"2024-04-26T23:52:45.236688+0000","last_peered":"2024-04-26T23:52:45.236688+0000","last_clean":"2024-04-26T23:52:45.236688+0000","last_became_active":"2024-04-26T23:52:45.230526+0000","last_became_peered":"2024-04-26T23:52:45.230526+0000","last_unstale":"2024-04-26T23:52:45.236688+0000","last_undegraded":"2024-04-26T23:52:45.236688+0000","last_fullsized":"2024-04-26T23:52:45.236688+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.232605+0000","last_change":"2024-04-26T23:52:45.230088+0000","last_active":"2024-04-26T23:52:45.232605+0000","last_peered":"2024-04-26T23:52:45.232605+0000","last_clean":"2024-04-26T23:52:45.232605+0000","last_became_active":"2024-04-26T23:52:45.228832+0000","last_became_peered":"2024-04-26T23:52:45.228832+0000","last_unstale":"2024-04-26T23:52:45.232605+0000","last_undegraded":"2024-04-26T23:52:45.232605+0000","last_fullsized":"2024-04-26T23:52:45.232605+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:31.569776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248400+0000","last_change":"2024-04-26T23:52:45.241806+0000","last_active":"2024-04-26T23:52:45.248400+0000","last_peered":"2024-04-26T23:52:45.248400+0000","last_clean":"2024-04-26T23:52:45.248400+0000","last_became_active":"2024-04-26T23:52:45.241512+0000","last_became_peered":"2024-04-26T23:52:45.241512+0000","last_unstale":"2024-04-26T23:52:45.248400+0000","last_undegraded":"2024-04-26T23:52:45.248400+0000","last_fullsized":"2024-04-26T23:52:45.248400+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:20.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248454+0000","last_change":"2024-04-26T23:52:45.244880+0000","last_active":"2024-04-26T23:52:45.248454+0000","last_peered":"2024-04-26T23:52:45.248454+0000","last_clean":"2024-04-26T23:52:45.248454+0000","last_became_active":"2024-04-26T23:52:45.241714+0000","last_became_peered":"2024-04-26T23:52:45.241714+0000","last_unstale":"2024-04-26T23:52:45.248454+0000","last_undegraded":"2024-04-26T23:52:45.248454+0000","last_fullsized":"2024-04-26T23:52:45.248454+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:10.589629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.250483+0000","last_change":"2024-04-26T23:52:45.248001+0000","last_active":"2024-04-26T23:52:45.250483+0000","last_peered":"2024-04-26T23:52:45.250483+0000","last_clean":"2024-04-26T23:52:45.250483+0000","last_became_active":"2024-04-26T23:52:45.247081+0000","last_became_peered":"2024-04-26T23:52:45.247081+0000","last_unstale":"2024-04-26T23:52:45.250483+0000","last_undegraded":"2024-04-26T23:52:45.250483+0000","last_fullsized":"2024-04-26T23:52:45.250483+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:20.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245291+0000","last_change":"2024-04-26T23:52:45.238717+0000","last_active":"2024-04-26T23:52:45.245291+0000","last_peered":"2024-04-26T23:52:45.245291+0000","last_clean":"2024-04-26T23:52:45.245291+0000","last_became_active":"2024-04-26T23:52:45.238188+0000","last_became_peered":"2024-04-26T23:52:45.238188+0000","last_unstale":"2024-04-26T23:52:45.245291+0000","last_undegraded":"2024-04-26T23:52:45.245291+0000","last_fullsized":"2024-04-26T23:52:45.245291+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:45.350016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248315+0000","last_change":"2024-04-26T23:52:45.243206+0000","last_active":"2024-04-26T23:52:45.248315+0000","last_peered":"2024-04-26T23:52:45.248315+0000","last_clean":"2024-04-26T23:52:45.248315+0000","last_became_active":"2024-04-26T23:52:45.242860+0000","last_became_peered":"2024-04-26T23:52:45.242860+0000","last_unstale":"2024-04-26T23:52:45.248315+0000","last_undegraded":"2024-04-26T23:52:45.248315+0000","last_fullsized":"2024-04-26T23:52:45.248315+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:19:14.669687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"160'4","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259244+0000","last_change":"2024-04-26T23:52:44.222425+0000","last_active":"2024-04-26T23:52:45.259244+0000","last_peered":"2024-04-26T23:52:45.259244+0000","last_clean":"2024-04-26T23:52:45.259244+0000","last_became_active":"2024-04-26T23:52:44.221707+0000","last_became_peered":"2024-04-26T23:52:44.221707+0000","last_unstale":"2024-04-26T23:52:45.259244+0000","last_undegraded":"2024-04-26T23:52:45.259244+0000","last_fullsized":"2024-04-26T23:52:45.259244+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:37.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.247200+0000","last_change":"2024-04-26T23:52:45.246067+0000","last_active":"2024-04-26T23:52:45.247200+0000","last_peered":"2024-04-26T23:52:45.247200+0000","last_clean":"2024-04-26T23:52:45.247200+0000","last_became_active":"2024-04-26T23:52:45.245802+0000","last_became_peered":"2024-04-26T23:52:45.245802+0000","last_unstale":"2024-04-26T23:52:45.247200+0000","last_undegraded":"2024-04-26T23:52:45.247200+0000","last_fullsized":"2024-04-26T23:52:45.247200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:31.289188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264142+0000","last_change":"2024-04-26T23:52:44.231249+0000","last_active":"2024-04-26T23:52:45.264142+0000","last_peered":"2024-04-26T23:52:45.264142+0000","last_clean":"2024-04-26T23:52:45.264142+0000","last_became_active":"2024-04-26T23:52:44.230957+0000","last_became_peered":"2024-04-26T23:52:44.230957+0000","last_unstale":"2024-04-26T23:52:45.264142+0000","last_undegraded":"2024-04-26T23:52:45.264142+0000","last_fullsized":"2024-04-26T23:52:45.264142+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:21.615078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.250896+0000","last_change":"2024-04-26T23:52:45.241211+0000","last_active":"2024-04-26T23:52:45.250896+0000","last_peered":"2024-04-26T23:52:45.250896+0000","last_clean":"2024-04-26T23:52:45.250896+0000","last_became_active":"2024-04-26T23:52:45.238448+0000","last_became_peered":"2024-04-26T23:52:45.238448+0000","last_unstale":"2024-04-26T23:52:45.250896+0000","last_undegraded":"2024-04-26T23:52:45.250896+0000","last_fullsized":"2024-04-26T23:52:45.250896+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.333700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259941+0000","last_change":"2024-04-26T23:52:44.235027+0000","last_active":"2024-04-26T23:52:45.259941+0000","last_peered":"2024-04-26T23:52:45.259941+0000","last_clean":"2024-04-26T23:52:45.259941+0000","last_became_active":"2024-04-26T23:52:44.234740+0000","last_became_peered":"2024-04-26T23:52:44.234740+0000","last_unstale":"2024-04-26T23:52:45.259941+0000","last_undegraded":"2024-04-26T23:52:45.259941+0000","last_fullsized":"2024-04-26T23:52:45.259941+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:51.528844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.246712+0000","last_change":"2024-04-26T23:52:45.237855+0000","last_active":"2024-04-26T23:52:45.246712+0000","last_peered":"2024-04-26T23:52:45.246712+0000","last_clean":"2024-04-26T23:52:45.246712+0000","last_became_active":"2024-04-26T23:52:45.237238+0000","last_became_peered":"2024-04-26T23:52:45.237238+0000","last_unstale":"2024-04-26T23:52:45.246712+0000","last_undegraded":"2024-04-26T23:52:45.246712+0000","last_fullsized":"2024-04-26T23:52:45.246712+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:38.441388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267628+0000","last_change":"2024-04-26T23:52:44.233273+0000","last_active":"2024-04-26T23:52:45.267628+0000","last_peered":"2024-04-26T23:52:45.267628+0000","last_clean":"2024-04-26T23:52:45.267628+0000","last_became_active":"2024-04-26T23:52:44.232932+0000","last_became_peered":"2024-04-26T23:52:44.232932+0000","last_unstale":"2024-04-26T23:52:45.267628+0000","last_undegraded":"2024-04-26T23:52:45.267628+0000","last_fullsized":"2024-04-26T23:52:45.267628+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:23:23.496996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259218+0000","last_change":"2024-04-26T23:52:45.252362+0000","last_active":"2024-04-26T23:52:45.259218+0000","last_peered":"2024-04-26T23:52:45.259218+0000","last_clean":"2024-04-26T23:52:45.259218+0000","last_became_active":"2024-04-26T23:52:45.252066+0000","last_became_peered":"2024-04-26T23:52:45.252066+0000","last_unstale":"2024-04-26T23:52:45.259218+0000","last_undegraded":"2024-04-26T23:52:45.259218+0000","last_fullsized":"2024-04-26T23:52:45.259218+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.268722+0000","last_change":"2024-04-26T23:52:44.239921+0000","last_active":"2024-04-26T23:52:45.268722+0000","last_peered":"2024-04-26T23:52:45.268722+0000","last_clean":"2024-04-26T23:52:45.268722+0000","last_became_active":"2024-04-26T23:52:44.239623+0000","last_became_peered":"2024-04-26T23:52:44.239623+0000","last_unstale":"2024-04-26T23:52:45.268722+0000","last_undegraded":"2024-04-26T23:52:45.268722+0000","last_fullsized":"2024-04-26T23:52:45.268722+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:06.810802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248675+0000","last_change":"2024-04-26T23:52:45.243368+0000","last_active":"2024-04-26T23:52:45.248675+0000","last_peered":"2024-04-26T23:52:45.248675+0000","last_clean":"2024-04-26T23:52:45.248675+0000","last_became_active":"2024-04-26T23:52:45.237517+0000","last_became_peered":"2024-04-26T23:52:45.237517+0000","last_unstale":"2024-04-26T23:52:45.248675+0000","last_undegraded":"2024-04-26T23:52:45.248675+0000","last_fullsized":"2024-04-26T23:52:45.248675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:59.715643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264600+0000","last_change":"2024-04-26T23:52:44.225024+0000","last_active":"2024-04-26T23:52:45.264600+0000","last_peered":"2024-04-26T23:52:45.264600+0000","last_clean":"2024-04-26T23:52:45.264600+0000","last_became_active":"2024-04-26T23:52:44.223248+0000","last_became_peered":"2024-04-26T23:52:44.223248+0000","last_unstale":"2024-04-26T23:52:45.264600+0000","last_undegraded":"2024-04-26T23:52:45.264600+0000","last_fullsized":"2024-04-26T23:52:45.264600+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:46.611618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258445+0000","last_change":"2024-04-26T23:52:45.251131+0000","last_active":"2024-04-26T23:52:45.258445+0000","last_peered":"2024-04-26T23:52:45.258445+0000","last_clean":"2024-04-26T23:52:45.258445+0000","last_became_active":"2024-04-26T23:52:45.250210+0000","last_became_peered":"2024-04-26T23:52:45.250210+0000","last_unstale":"2024-04-26T23:52:45.258445+0000","last_undegraded":"2024-04-26T23:52:45.258445+0000","last_fullsized":"2024-04-26T23:52:45.258445+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:35.821082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225615+0000","last_change":"2024-04-26T23:52:44.222397+0000","last_active":"2024-04-26T23:52:45.225615+0000","last_peered":"2024-04-26T23:52:45.225615+0000","last_clean":"2024-04-26T23:52:45.225615+0000","last_became_active":"2024-04-26T23:52:44.222106+0000","last_became_peered":"2024-04-26T23:52:44.222106+0000","last_unstale":"2024-04-26T23:52:45.225615+0000","last_undegraded":"2024-04-26T23:52:45.225615+0000","last_fullsized":"2024-04-26T23:52:45.225615+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:10.021380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.244025+0000","last_change":"2024-04-26T23:52:45.240996+0000","last_active":"2024-04-26T23:52:45.244025+0000","last_peered":"2024-04-26T23:52:45.244025+0000","last_clean":"2024-04-26T23:52:45.244025+0000","last_became_active":"2024-04-26T23:52:45.239945+0000","last_became_peered":"2024-04-26T23:52:45.239945+0000","last_unstale":"2024-04-26T23:52:45.244025+0000","last_undegraded":"2024-04-26T23:52:45.244025+0000","last_fullsized":"2024-04-26T23:52:45.244025+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:52.896334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243079+0000","last_change":"2024-04-26T23:52:44.237595+0000","last_active":"2024-04-26T23:52:45.243079+0000","last_peered":"2024-04-26T23:52:45.243079+0000","last_clean":"2024-04-26T23:52:45.243079+0000","last_became_active":"2024-04-26T23:52:44.237178+0000","last_became_peered":"2024-04-26T23:52:44.237178+0000","last_unstale":"2024-04-26T23:52:45.243079+0000","last_undegraded":"2024-04-26T23:52:45.243079+0000","last_fullsized":"2024-04-26T23:52:45.243079+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:11.502021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248030+0000","last_change":"2024-04-26T23:52:45.241243+0000","last_active":"2024-04-26T23:52:45.248030+0000","last_peered":"2024-04-26T23:52:45.248030+0000","last_clean":"2024-04-26T23:52:45.248030+0000","last_became_active":"2024-04-26T23:52:45.240944+0000","last_became_peered":"2024-04-26T23:52:45.240944+0000","last_unstale":"2024-04-26T23:52:45.248030+0000","last_undegraded":"2024-04-26T23:52:45.248030+0000","last_fullsized":"2024-04-26T23:52:45.248030+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225682+0000","last_change":"2024-04-26T23:52:44.231535+0000","last_active":"2024-04-26T23:52:45.225682+0000","last_peered":"2024-04-26T23:52:45.225682+0000","last_clean":"2024-04-26T23:52:45.225682+0000","last_became_active":"2024-04-26T23:52:44.230796+0000","last_became_peered":"2024-04-26T23:52:44.230796+0000","last_unstale":"2024-04-26T23:52:45.225682+0000","last_undegraded":"2024-04-26T23:52:45.225682+0000","last_fullsized":"2024-04-26T23:52:45.225682+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:15.524822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.233267+0000","last_change":"2024-04-26T23:52:45.232017+0000","last_active":"2024-04-26T23:52:45.233267+0000","last_peered":"2024-04-26T23:52:45.233267+0000","last_clean":"2024-04-26T23:52:45.233267+0000","last_became_active":"2024-04-26T23:52:45.231701+0000","last_became_peered":"2024-04-26T23:52:45.231701+0000","last_unstale":"2024-04-26T23:52:45.233267+0000","last_undegraded":"2024-04-26T23:52:45.233267+0000","last_fullsized":"2024-04-26T23:52:45.233267+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:40.187504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"160'3","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258719+0000","last_change":"2024-04-26T23:52:44.234204+0000","last_active":"2024-04-26T23:52:45.258719+0000","last_peered":"2024-04-26T23:52:45.258719+0000","last_clean":"2024-04-26T23:52:45.258719+0000","last_became_active":"2024-04-26T23:52:44.233586+0000","last_became_peered":"2024-04-26T23:52:44.233586+0000","last_unstale":"2024-04-26T23:52:45.258719+0000","last_undegraded":"2024-04-26T23:52:45.258719+0000","last_fullsized":"2024-04-26T23:52:45.258719+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:29.060684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259322+0000","last_change":"2024-04-26T23:52:45.252361+0000","last_active":"2024-04-26T23:52:45.259322+0000","last_peered":"2024-04-26T23:52:45.259322+0000","last_clean":"2024-04-26T23:52:45.259322+0000","last_became_active":"2024-04-26T23:52:45.252066+0000","last_became_peered":"2024-04-26T23:52:45.252066+0000","last_unstale":"2024-04-26T23:52:45.259322+0000","last_undegraded":"2024-04-26T23:52:45.259322+0000","last_fullsized":"2024-04-26T23:52:45.259322+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.403097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"160'6","reported_seq":18,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.271276+0000","last_change":"2024-04-26T23:52:44.230451+0000","last_active":"2024-04-26T23:52:45.271276+0000","last_peered":"2024-04-26T23:52:45.271276+0000","last_clean":"2024-04-26T23:52:45.271276+0000","last_became_active":"2024-04-26T23:52:44.230159+0000","last_became_peered":"2024-04-26T23:52:44.230159+0000","last_unstale":"2024-04-26T23:52:45.271276+0000","last_undegraded":"2024-04-26T23:52:45.271276+0000","last_fullsized":"2024-04-26T23:52:45.271276+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:17.580809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.252087+0000","last_change":"2024-04-26T23:52:45.250809+0000","last_active":"2024-04-26T23:52:45.252087+0000","last_peered":"2024-04-26T23:52:45.252087+0000","last_clean":"2024-04-26T23:52:45.252087+0000","last_became_active":"2024-04-26T23:52:45.250356+0000","last_became_peered":"2024-04-26T23:52:45.250356+0000","last_unstale":"2024-04-26T23:52:45.252087+0000","last_undegraded":"2024-04-26T23:52:45.252087+0000","last_fullsized":"2024-04-26T23:52:45.252087+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:59.889184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267915+0000","last_change":"2024-04-26T23:52:44.239118+0000","last_active":"2024-04-26T23:52:45.267915+0000","last_peered":"2024-04-26T23:52:45.267915+0000","last_clean":"2024-04-26T23:52:45.267915+0000","last_became_active":"2024-04-26T23:52:44.235855+0000","last_became_peered":"2024-04-26T23:52:44.235855+0000","last_unstale":"2024-04-26T23:52:45.267915+0000","last_undegraded":"2024-04-26T23:52:45.267915+0000","last_fullsized":"2024-04-26T23:52:45.267915+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:19:17.038463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256911+0000","last_change":"2024-04-26T23:52:45.242412+0000","last_active":"2024-04-26T23:52:45.256911+0000","last_peered":"2024-04-26T23:52:45.256911+0000","last_clean":"2024-04-26T23:52:45.256911+0000","last_became_active":"2024-04-26T23:52:45.241599+0000","last_became_peered":"2024-04-26T23:52:45.241599+0000","last_unstale":"2024-04-26T23:52:45.256911+0000","last_undegraded":"2024-04-26T23:52:45.256911+0000","last_fullsized":"2024-04-26T23:52:45.256911+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:26.921870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.213191+0000","last_change":"2024-04-26T23:52:44.236449+0000","last_active":"2024-04-26T23:52:45.213191+0000","last_peered":"2024-04-26T23:52:45.213191+0000","last_clean":"2024-04-26T23:52:45.213191+0000","last_became_active":"2024-04-26T23:52:44.234867+0000","last_became_peered":"2024-04-26T23:52:44.234867+0000","last_unstale":"2024-04-26T23:52:45.213191+0000","last_undegraded":"2024-04-26T23:52:45.213191+0000","last_fullsized":"2024-04-26T23:52:45.213191+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:50.614442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240768+0000","last_change":"2024-04-26T23:52:45.236621+0000","last_active":"2024-04-26T23:52:45.240768+0000","last_peered":"2024-04-26T23:52:45.240768+0000","last_clean":"2024-04-26T23:52:45.240768+0000","last_became_active":"2024-04-26T23:52:45.236305+0000","last_became_peered":"2024-04-26T23:52:45.236305+0000","last_unstale":"2024-04-26T23:52:45.240768+0000","last_undegraded":"2024-04-26T23:52:45.240768+0000","last_fullsized":"2024-04-26T23:52:45.240768+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:38.441388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.224904+0000","last_change":"2024-04-26T23:52:44.235590+0000","last_active":"2024-04-26T23:52:45.224904+0000","last_peered":"2024-04-26T23:52:45.224904+0000","last_clean":"2024-04-26T23:52:45.224904+0000","last_became_active":"2024-04-26T23:52:44.234988+0000","last_became_peered":"2024-04-26T23:52:44.234988+0000","last_unstale":"2024-04-26T23:52:45.224904+0000","last_undegraded":"2024-04-26T23:52:45.224904+0000","last_fullsized":"2024-04-26T23:52:45.224904+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:11.502021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259055+0000","last_change":"2024-04-26T23:52:45.246185+0000","last_active":"2024-04-26T23:52:45.259055+0000","last_peered":"2024-04-26T23:52:45.259055+0000","last_clean":"2024-04-26T23:52:45.259055+0000","last_became_active":"2024-04-26T23:52:45.245903+0000","last_became_peered":"2024-04-26T23:52:45.245903+0000","last_unstale":"2024-04-26T23:52:45.259055+0000","last_undegraded":"2024-04-26T23:52:45.259055+0000","last_fullsized":"2024-04-26T23:52:45.259055+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:06.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"160'8","reported_seq":20,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.269576+0000","last_change":"2024-04-26T23:52:44.232473+0000","last_active":"2024-04-26T23:52:45.269576+0000","last_peered":"2024-04-26T23:52:45.269576+0000","last_clean":"2024-04-26T23:52:45.269576+0000","last_became_active":"2024-04-26T23:52:44.232148+0000","last_became_peered":"2024-04-26T23:52:44.232148+0000","last_unstale":"2024-04-26T23:52:45.269576+0000","last_undegraded":"2024-04-26T23:52:45.269576+0000","last_fullsized":"2024-04-26T23:52:45.269576+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:37.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.255655+0000","last_change":"2024-04-26T23:52:45.241311+0000","last_active":"2024-04-26T23:52:45.255655+0000","last_peered":"2024-04-26T23:52:45.255655+0000","last_clean":"2024-04-26T23:52:45.255655+0000","last_became_active":"2024-04-26T23:52:45.239714+0000","last_became_peered":"2024-04-26T23:52:45.239714+0000","last_unstale":"2024-04-26T23:52:45.255655+0000","last_undegraded":"2024-04-26T23:52:45.255655+0000","last_fullsized":"2024-04-26T23:52:45.255655+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:55.647019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.265418+0000","last_change":"2024-04-26T23:52:44.225669+0000","last_active":"2024-04-26T23:52:45.265418+0000","last_peered":"2024-04-26T23:52:45.265418+0000","last_clean":"2024-04-26T23:52:45.265418+0000","last_became_active":"2024-04-26T23:52:44.225376+0000","last_became_peered":"2024-04-26T23:52:44.225376+0000","last_unstale":"2024-04-26T23:52:45.265418+0000","last_undegraded":"2024-04-26T23:52:45.265418+0000","last_fullsized":"2024-04-26T23:52:45.265418+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:07.859563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248201+0000","last_change":"2024-04-26T23:52:45.243498+0000","last_active":"2024-04-26T23:52:45.248201+0000","last_peered":"2024-04-26T23:52:45.248201+0000","last_clean":"2024-04-26T23:52:45.248201+0000","last_became_active":"2024-04-26T23:52:45.243112+0000","last_became_peered":"2024-04-26T23:52:45.243112+0000","last_unstale":"2024-04-26T23:52:45.248201+0000","last_undegraded":"2024-04-26T23:52:45.248201+0000","last_fullsized":"2024-04-26T23:52:45.248201+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:03.276791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.268860+0000","last_change":"2024-04-26T23:52:44.230807+0000","last_active":"2024-04-26T23:52:45.268860+0000","last_peered":"2024-04-26T23:52:45.268860+0000","last_clean":"2024-04-26T23:52:45.268860+0000","last_became_active":"2024-04-26T23:52:44.230466+0000","last_became_peered":"2024-04-26T23:52:44.230466+0000","last_unstale":"2024-04-26T23:52:45.268860+0000","last_undegraded":"2024-04-26T23:52:45.268860+0000","last_fullsized":"2024-04-26T23:52:45.268860+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.796144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240439+0000","last_change":"2024-04-26T23:52:45.237656+0000","last_active":"2024-04-26T23:52:45.240439+0000","last_peered":"2024-04-26T23:52:45.240439+0000","last_clean":"2024-04-26T23:52:45.240439+0000","last_became_active":"2024-04-26T23:52:45.237312+0000","last_became_peered":"2024-04-26T23:52:45.237312+0000","last_unstale":"2024-04-26T23:52:45.240439+0000","last_undegraded":"2024-04-26T23:52:45.240439+0000","last_fullsized":"2024-04-26T23:52:45.240439+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:06.683558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"160'1","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264198+0000","last_change":"2024-04-26T23:52:44.224080+0000","last_active":"2024-04-26T23:52:45.264198+0000","last_peered":"2024-04-26T23:52:45.264198+0000","last_clean":"2024-04-26T23:52:45.264198+0000","last_became_active":"2024-04-26T23:52:44.223735+0000","last_became_peered":"2024-04-26T23:52:44.223735+0000","last_unstale":"2024-04-26T23:52:45.264198+0000","last_undegraded":"2024-04-26T23:52:45.264198+0000","last_fullsized":"2024-04-26T23:52:45.264198+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:59.418025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.234589+0000","last_change":"2024-04-26T23:52:45.233392+0000","last_active":"2024-04-26T23:52:45.234589+0000","last_peered":"2024-04-26T23:52:45.234589+0000","last_clean":"2024-04-26T23:52:45.234589+0000","last_became_active":"2024-04-26T23:52:45.232997+0000","last_became_peered":"2024-04-26T23:52:45.232997+0000","last_unstale":"2024-04-26T23:52:45.234589+0000","last_undegraded":"2024-04-26T23:52:45.234589+0000","last_fullsized":"2024-04-26T23:52:45.234589+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.252977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240918+0000","last_change":"2024-04-26T23:52:44.240014+0000","last_active":"2024-04-26T23:52:45.240918+0000","last_peered":"2024-04-26T23:52:45.240918+0000","last_clean":"2024-04-26T23:52:45.240918+0000","last_became_active":"2024-04-26T23:52:44.239731+0000","last_became_peered":"2024-04-26T23:52:44.239731+0000","last_unstale":"2024-04-26T23:52:45.240918+0000","last_undegraded":"2024-04-26T23:52:45.240918+0000","last_fullsized":"2024-04-26T23:52:45.240918+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:35.591248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.245582+0000","last_change":"2024-04-26T23:52:45.244232+0000","last_active":"2024-04-26T23:52:45.245582+0000","last_peered":"2024-04-26T23:52:45.245582+0000","last_clean":"2024-04-26T23:52:45.245582+0000","last_became_active":"2024-04-26T23:52:45.243960+0000","last_became_peered":"2024-04-26T23:52:45.243960+0000","last_unstale":"2024-04-26T23:52:45.245582+0000","last_undegraded":"2024-04-26T23:52:45.245582+0000","last_fullsized":"2024-04-26T23:52:45.245582+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:12.536494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.267458+0000","last_change":"2024-04-26T23:52:44.236986+0000","last_active":"2024-04-26T23:52:45.267458+0000","last_peered":"2024-04-26T23:52:45.267458+0000","last_clean":"2024-04-26T23:52:45.267458+0000","last_became_active":"2024-04-26T23:52:44.235500+0000","last_became_peered":"2024-04-26T23:52:44.235500+0000","last_unstale":"2024-04-26T23:52:45.267458+0000","last_undegraded":"2024-04-26T23:52:45.267458+0000","last_fullsized":"2024-04-26T23:52:45.267458+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:37.437126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.258871+0000","last_change":"2024-04-26T23:52:45.256179+0000","last_active":"2024-04-26T23:52:45.258871+0000","last_peered":"2024-04-26T23:52:45.258871+0000","last_clean":"2024-04-26T23:52:45.258871+0000","last_became_active":"2024-04-26T23:52:45.255524+0000","last_became_peered":"2024-04-26T23:52:45.255524+0000","last_unstale":"2024-04-26T23:52:45.258871+0000","last_undegraded":"2024-04-26T23:52:45.258871+0000","last_fullsized":"2024-04-26T23:52:45.258871+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:20.106084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223999+0000","last_change":"2024-04-26T23:52:44.233101+0000","last_active":"2024-04-26T23:52:45.223999+0000","last_peered":"2024-04-26T23:52:45.223999+0000","last_clean":"2024-04-26T23:52:45.223999+0000","last_became_active":"2024-04-26T23:52:44.230391+0000","last_became_peered":"2024-04-26T23:52:44.230391+0000","last_unstale":"2024-04-26T23:52:45.223999+0000","last_undegraded":"2024-04-26T23:52:45.223999+0000","last_fullsized":"2024-04-26T23:52:45.223999+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:25.904286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.234053+0000","last_change":"2024-04-26T23:52:45.231576+0000","last_active":"2024-04-26T23:52:45.234053+0000","last_peered":"2024-04-26T23:52:45.234053+0000","last_clean":"2024-04-26T23:52:45.234053+0000","last_became_active":"2024-04-26T23:52:45.231222+0000","last_became_peered":"2024-04-26T23:52:45.231222+0000","last_unstale":"2024-04-26T23:52:45.234053+0000","last_undegraded":"2024-04-26T23:52:45.234053+0000","last_fullsized":"2024-04-26T23:52:45.234053+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:51.632026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.225889+0000","last_change":"2024-04-26T23:52:44.235957+0000","last_active":"2024-04-26T23:52:45.225889+0000","last_peered":"2024-04-26T23:52:45.225889+0000","last_clean":"2024-04-26T23:52:45.225889+0000","last_became_active":"2024-04-26T23:52:44.235007+0000","last_became_peered":"2024-04-26T23:52:44.235007+0000","last_unstale":"2024-04-26T23:52:45.225889+0000","last_undegraded":"2024-04-26T23:52:45.225889+0000","last_fullsized":"2024-04-26T23:52:45.225889+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:21:25.124752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248440+0000","last_change":"2024-04-26T23:52:45.243385+0000","last_active":"2024-04-26T23:52:45.248440+0000","last_peered":"2024-04-26T23:52:45.248440+0000","last_clean":"2024-04-26T23:52:45.248440+0000","last_became_active":"2024-04-26T23:52:45.243058+0000","last_became_peered":"2024-04-26T23:52:45.243058+0000","last_unstale":"2024-04-26T23:52:45.248440+0000","last_undegraded":"2024-04-26T23:52:45.248440+0000","last_fullsized":"2024-04-26T23:52:45.248440+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:29.011766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.221321+0000","last_change":"2024-04-26T23:52:44.236117+0000","last_active":"2024-04-26T23:52:45.221321+0000","last_peered":"2024-04-26T23:52:45.221321+0000","last_clean":"2024-04-26T23:52:45.221321+0000","last_became_active":"2024-04-26T23:52:44.235727+0000","last_became_peered":"2024-04-26T23:52:44.235727+0000","last_unstale":"2024-04-26T23:52:45.221321+0000","last_undegraded":"2024-04-26T23:52:45.221321+0000","last_fullsized":"2024-04-26T23:52:45.221321+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:51.528844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.249675+0000","last_change":"2024-04-26T23:52:45.246522+0000","last_active":"2024-04-26T23:52:45.249675+0000","last_peered":"2024-04-26T23:52:45.249675+0000","last_clean":"2024-04-26T23:52:45.249675+0000","last_became_active":"2024-04-26T23:52:45.246207+0000","last_became_peered":"2024-04-26T23:52:45.246207+0000","last_unstale":"2024-04-26T23:52:45.249675+0000","last_undegraded":"2024-04-26T23:52:45.249675+0000","last_fullsized":"2024-04-26T23:52:45.249675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:25.692896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"160'2","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.264640+0000","last_change":"2024-04-26T23:52:44.235352+0000","last_active":"2024-04-26T23:52:45.264640+0000","last_peered":"2024-04-26T23:52:45.264640+0000","last_clean":"2024-04-26T23:52:45.264640+0000","last_became_active":"2024-04-26T23:52:44.234140+0000","last_became_peered":"2024-04-26T23:52:44.234140+0000","last_unstale":"2024-04-26T23:52:45.264640+0000","last_undegraded":"2024-04-26T23:52:45.264640+0000","last_fullsized":"2024-04-26T23:52:45.264640+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:58.266715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.257786+0000","last_change":"2024-04-26T23:52:45.243625+0000","last_active":"2024-04-26T23:52:45.257786+0000","last_peered":"2024-04-26T23:52:45.257786+0000","last_clean":"2024-04-26T23:52:45.257786+0000","last_became_active":"2024-04-26T23:52:45.243325+0000","last_became_peered":"2024-04-26T23:52:45.243325+0000","last_unstale":"2024-04-26T23:52:45.257786+0000","last_undegraded":"2024-04-26T23:52:45.257786+0000","last_fullsized":"2024-04-26T23:52:45.257786+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:52.546427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.222993+0000","last_change":"2024-04-26T23:52:44.227117+0000","last_active":"2024-04-26T23:52:45.222993+0000","last_peered":"2024-04-26T23:52:45.222993+0000","last_clean":"2024-04-26T23:52:45.222993+0000","last_became_active":"2024-04-26T23:52:44.226788+0000","last_became_peered":"2024-04-26T23:52:44.226788+0000","last_unstale":"2024-04-26T23:52:45.222993+0000","last_undegraded":"2024-04-26T23:52:45.222993+0000","last_fullsized":"2024-04-26T23:52:45.222993+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:55.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248001+0000","last_change":"2024-04-26T23:52:45.241229+0000","last_active":"2024-04-26T23:52:45.248001+0000","last_peered":"2024-04-26T23:52:45.248001+0000","last_clean":"2024-04-26T23:52:45.248001+0000","last_became_active":"2024-04-26T23:52:45.240404+0000","last_became_peered":"2024-04-26T23:52:45.240404+0000","last_unstale":"2024-04-26T23:52:45.248001+0000","last_undegraded":"2024-04-26T23:52:45.248001+0000","last_fullsized":"2024-04-26T23:52:45.248001+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:00.720006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223041+0000","last_change":"2024-04-26T23:52:44.235908+0000","last_active":"2024-04-26T23:52:45.223041+0000","last_peered":"2024-04-26T23:52:45.223041+0000","last_clean":"2024-04-26T23:52:45.223041+0000","last_became_active":"2024-04-26T23:52:44.232624+0000","last_became_peered":"2024-04-26T23:52:44.232624+0000","last_unstale":"2024-04-26T23:52:45.223041+0000","last_undegraded":"2024-04-26T23:52:45.223041+0000","last_fullsized":"2024-04-26T23:52:45.223041+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:15.434038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.256462+0000","last_change":"2024-04-26T23:52:45.241877+0000","last_active":"2024-04-26T23:52:45.256462+0000","last_peered":"2024-04-26T23:52:45.256462+0000","last_clean":"2024-04-26T23:52:45.256462+0000","last_became_active":"2024-04-26T23:52:45.241248+0000","last_became_peered":"2024-04-26T23:52:45.241248+0000","last_unstale":"2024-04-26T23:52:45.256462+0000","last_undegraded":"2024-04-26T23:52:45.256462+0000","last_fullsized":"2024-04-26T23:52:45.256462+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:57.976389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.229464+0000","last_change":"2024-04-26T23:52:44.235880+0000","last_active":"2024-04-26T23:52:45.229464+0000","last_peered":"2024-04-26T23:52:45.229464+0000","last_clean":"2024-04-26T23:52:45.229464+0000","last_became_active":"2024-04-26T23:52:44.235215+0000","last_became_peered":"2024-04-26T23:52:44.235215+0000","last_unstale":"2024-04-26T23:52:45.229464+0000","last_undegraded":"2024-04-26T23:52:45.229464+0000","last_fullsized":"2024-04-26T23:52:45.229464+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:38.803564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.240175+0000","last_change":"2024-04-26T23:52:45.233170+0000","last_active":"2024-04-26T23:52:45.240175+0000","last_peered":"2024-04-26T23:52:45.240175+0000","last_clean":"2024-04-26T23:52:45.240175+0000","last_became_active":"2024-04-26T23:52:45.229793+0000","last_became_peered":"2024-04-26T23:52:45.229793+0000","last_unstale":"2024-04-26T23:52:45.240175+0000","last_undegraded":"2024-04-26T23:52:45.240175+0000","last_fullsized":"2024-04-26T23:52:45.240175+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:26.040678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.211996+0000","last_change":"2024-04-26T23:52:44.225856+0000","last_active":"2024-04-26T23:52:45.211996+0000","last_peered":"2024-04-26T23:52:45.211996+0000","last_clean":"2024-04-26T23:52:45.211996+0000","last_became_active":"2024-04-26T23:52:44.223912+0000","last_became_peered":"2024-04-26T23:52:44.223912+0000","last_unstale":"2024-04-26T23:52:45.211996+0000","last_undegraded":"2024-04-26T23:52:45.211996+0000","last_fullsized":"2024-04-26T23:52:45.211996+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:34.223513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248255+0000","last_change":"2024-04-26T23:52:45.242507+0000","last_active":"2024-04-26T23:52:45.248255+0000","last_peered":"2024-04-26T23:52:45.248255+0000","last_clean":"2024-04-26T23:52:45.248255+0000","last_became_active":"2024-04-26T23:52:45.242209+0000","last_became_peered":"2024-04-26T23:52:45.242209+0000","last_unstale":"2024-04-26T23:52:45.248255+0000","last_undegraded":"2024-04-26T23:52:45.248255+0000","last_fullsized":"2024-04-26T23:52:45.248255+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:06.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.213656+0000","last_change":"2024-04-26T23:52:44.230071+0000","last_active":"2024-04-26T23:52:45.213656+0000","last_peered":"2024-04-26T23:52:45.213656+0000","last_clean":"2024-04-26T23:52:45.213656+0000","last_became_active":"2024-04-26T23:52:44.229296+0000","last_became_peered":"2024-04-26T23:52:44.229296+0000","last_unstale":"2024-04-26T23:52:45.213656+0000","last_undegraded":"2024-04-26T23:52:45.213656+0000","last_fullsized":"2024-04-26T23:52:45.213656+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:50.316116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.243264+0000","last_change":"2024-04-26T23:52:45.239068+0000","last_active":"2024-04-26T23:52:45.243264+0000","last_peered":"2024-04-26T23:52:45.243264+0000","last_clean":"2024-04-26T23:52:45.243264+0000","last_became_active":"2024-04-26T23:52:45.238378+0000","last_became_peered":"2024-04-26T23:52:45.238378+0000","last_unstale":"2024-04-26T23:52:45.243264+0000","last_undegraded":"2024-04-26T23:52:45.243264+0000","last_fullsized":"2024-04-26T23:52:45.243264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.926047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.224513+0000","last_change":"2024-04-26T23:52:44.235635+0000","last_active":"2024-04-26T23:52:45.224513+0000","last_peered":"2024-04-26T23:52:45.224513+0000","last_clean":"2024-04-26T23:52:45.224513+0000","last_became_active":"2024-04-26T23:52:44.235366+0000","last_became_peered":"2024-04-26T23:52:44.235366+0000","last_unstale":"2024-04-26T23:52:45.224513+0000","last_undegraded":"2024-04-26T23:52:45.224513+0000","last_fullsized":"2024-04-26T23:52:45.224513+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:23.365128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.259112+0000","last_change":"2024-04-26T23:52:45.250689+0000","last_active":"2024-04-26T23:52:45.259112+0000","last_peered":"2024-04-26T23:52:45.259112+0000","last_clean":"2024-04-26T23:52:45.259112+0000","last_became_active":"2024-04-26T23:52:45.250385+0000","last_became_peered":"2024-04-26T23:52:45.250385+0000","last_unstale":"2024-04-26T23:52:45.259112+0000","last_undegraded":"2024-04-26T23:52:45.259112+0000","last_fullsized":"2024-04-26T23:52:45.259112+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:35.572422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.210958+0000","last_change":"2024-04-26T23:52:44.232443+0000","last_active":"2024-04-26T23:52:45.210958+0000","last_peered":"2024-04-26T23:52:45.210958+0000","last_clean":"2024-04-26T23:52:45.210958+0000","last_became_active":"2024-04-26T23:52:44.232159+0000","last_became_peered":"2024-04-26T23:52:44.232159+0000","last_unstale":"2024-04-26T23:52:45.210958+0000","last_undegraded":"2024-04-26T23:52:45.210958+0000","last_fullsized":"2024-04-26T23:52:45.210958+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:27.994182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.248988+0000","last_change":"2024-04-26T23:52:45.245828+0000","last_active":"2024-04-26T23:52:45.248988+0000","last_peered":"2024-04-26T23:52:45.248988+0000","last_clean":"2024-04-26T23:52:45.248988+0000","last_became_active":"2024-04-26T23:52:45.245416+0000","last_became_peered":"2024-04-26T23:52:45.245416+0000","last_unstale":"2024-04-26T23:52:45.248988+0000","last_undegraded":"2024-04-26T23:52:45.248988+0000","last_fullsized":"2024-04-26T23:52:45.248988+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:44.183307+0000","last_clean_scrub_stamp":"2024-04-26T23:52:44.183307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:59.284298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.223108+0000","last_change":"2024-04-26T23:52:44.227316+0000","last_active":"2024-04-26T23:52:45.223108+0000","last_peered":"2024-04-26T23:52:45.223108+0000","last_clean":"2024-04-26T23:52:45.223108+0000","last_became_active":"2024-04-26T23:52:44.227016+0000","last_became_peered":"2024-04-26T23:52:44.227016+0000","last_unstale":"2024-04-26T23:52:45.223108+0000","last_undegraded":"2024-04-26T23:52:45.223108+0000","last_fullsized":"2024-04-26T23:52:45.223108+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:52:43.165723+0000","last_clean_scrub_stamp":"2024-04-26T23:52:43.165723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:19.088500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":427,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-26T23:52:45.220721+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:52:45.220721+0000","last_peered":"2024-04-26T23:52:45.220721+0000","last_clean":"2024-04-26T23:52:45.220721+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:52:45.220721+0000","last_undegraded":"2024-04-26T23:52:45.220721+0000","last_fullsized":"2024-04-26T23:52:45.220721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51772,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320068,"statfs":{"total":96636764160,"available":96583749632,"internally_reserved":0,"allocated":2605056,"data_stored":1537340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2011136,"data_stored":946972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51200,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320640,"statfs":{"total":96636764160,"available":96584335360,"internally_reserved":0,"allocated":2019328,"data_stored":947096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59532,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312308,"statfs":{"total":96636764160,"available":96575803392,"internally_reserved":0,"allocated":2027520,"data_stored":947736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59532,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312308,"statfs":{"total":96636764160,"available":96575803392,"internally_reserved":0,"allocated":2031616,"data_stored":949188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59512,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":2011136,"data_stored":946972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575040,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59524,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312316,"statfs":{"total":96636764160,"available":96575811584,"internally_reserved":0,"allocated":2023424,"data_stored":948584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575040,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55948,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315892,"statfs":{"total":96636764160,"available":96579473408,"internally_reserved":0,"allocated":2613248,"data_stored":1537444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:52:50.265 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:52:50.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:52:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.525+0000 7f8492200640 1 Processor -- start 2024-04-26T23:52:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.525+0000 7f8492200640 1 -- start start 2024-04-26T23:52:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8492200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f848c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8492200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f848c0cdb30 0x7f848c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8492200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f848c0ce7d0 con 0x7f848c073b30 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8492200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f848c0cdf90 con 0x7f848c0cdb30 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8492200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f848c0ceab0 con 0x7f848c15cec0 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8491800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f848c073b30 0x7f848c071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54616/0 (socket says 172.21.15.46:54616) 2024-04-26T23:52:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.527+0000 7f8491800640 1 -- 172.21.15.46:0/843299123 learned_addr learned my addr 172.21.15.46:0/843299123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> v2:172.21.15.142:3300/0 conn(0x7f848c0cdb30 0x7f848c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909437256 0 0) 0x7f848c0ce7d0 con 0x7f848c073b30 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f8490e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f848c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8470003520 con 0x7f848c073b30 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 578153200 0 0) 0x7f8470003520 con 0x7f848c073b30 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 msgr2=0x7f848c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f848c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 >> v2:172.21.15.142:3300/0 conn(0x7f848c0cdb30 msgr2=0x7f848c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 --2- 172.21.15.46:0/843299123 >> v2:172.21.15.142:3300/0 conn(0x7f848c0cdb30 0x7f848c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c0ced90 con 0x7f848c073b30 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8484001690 con 0x7f848c073b30 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f8490e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f848c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 >> v1:172.21.15.46:6789/0 conn(0x7f848c073b30 legacy=0x7f848c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f84700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f8470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8470004d40 0x7f84700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0ceab0 con 0x7f8470003ff0 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f847000ba70 con 0x7f8470004d40 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f847000bc70 con 0x7f848c15cec0 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.528+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> v2:172.21.15.142:3300/0 conn(0x7f848c0cdb30 0x7f848c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f8470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8490e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f84700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8491800640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8470004d40 0x7f84700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 -- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 msgr2=0x7f84700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c15cec0 0x7f84700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 -- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8470004d40 msgr2=0x7f84700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8470004d40 0x7f84700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.529+0000 7f8483e00640 1 -- 172.21.15.46:0/843299123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8470003520 con 0x7f8470003ff0 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.530+0000 7f8483e00640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f8470004800 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f84740098a0 tx=0x7f8470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cae78c3f2491ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474037070 con 0x7f8470003ff0 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847402ad20 con 0x7f8470003ff0 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f848be00640 1 -- 172.21.15.46:0/843299123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847400f040 con 0x7f8470003ff0 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 -- 172.21.15.46:0/843299123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 msgr2=0x7f8470004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 --2- 172.21.15.46:0/843299123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f8470004800 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f84740098a0 tx=0x7f8470005830 comp rx=0 tx=0).stop 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8491800640 1 -- 172.21.15.46:0/843299123 reap_dead start 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 -- 172.21.15.46:0/843299123 shutdown_connections 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 -- 172.21.15.46:0/843299123 >> 172.21.15.46:0/843299123 conn(0x7f848c0b27b0 msgr2=0x7f848c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 -- 172.21.15.46:0/843299123 shutdown_connections 2024-04-26T23:52:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.531+0000 7f8492200640 1 -- 172.21.15.46:0/843299123 wait complete. 2024-04-26T23:52:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.533+0000 7f8492200640 1 Processor -- start 2024-04-26T23:52:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.533+0000 7f8492200640 1 -- start start 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 0x7f848c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 0x7f848c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f848c15d2a0 con 0x7f8470003ff0 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0cf350 con 0x7f848c158d10 2024-04-26T23:52:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.534+0000 7f8492200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f848c073fa0 con 0x7f848c0c5c80 2024-04-26T23:52:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50560/0 (socket says 172.21.15.46:50560) 2024-04-26T23:52:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 -- 172.21.15.46:0/3738245174 learned_addr learned my addr 172.21.15.46:0/3738245174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8491800640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 0x7f848c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8483e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 0x7f848c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 -- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 msgr2=0x7f848c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 0x7f848c0c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 -- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 msgr2=0x7f848c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 0x7f848c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8490e00640 1 -- 172.21.15.46:0/3738245174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c0ccce0 con 0x7f8470003ff0 2024-04-26T23:52:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8491800640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 0x7f848c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.535+0000 7f8483e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 0x7f848c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8490e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f847c004710 tx=0x7f847c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c014610 con 0x7f8470003ff0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c019070 con 0x7f8470003ff0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0149d0 con 0x7f8470003ff0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c0ccf80 con 0x7f8470003ff0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.536+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c0cc280 con 0x7f8470003ff0 2024-04-26T23:52:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.537+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c0693b0 con 0x7f8470003ff0 2024-04-26T23:52:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.538+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f847c014b70 con 0x7f8470003ff0 2024-04-26T23:52:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.539+0000 7f8489600640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 0x7f846806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.539+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f847c08c570 con 0x7f8470003ff0 2024-04-26T23:52:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.540+0000 7f8483e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 0x7f846806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.541+0000 7f8483e00640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 0x7f846806ede0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8474034000 tx=0x7f847402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.543+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f847c05a800 con 0x7f8470003ff0 2024-04-26T23:52:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.819+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f848c15b740 con 0x7f8470003ff0 2024-04-26T23:52:50.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.824+0000 7f8489600640 1 -- 172.21.15.46:0/3738245174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f847c05a100 con 0x7f8470003ff0 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:52:50.828 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 msgr2=0x7f846806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 0x7f846806ede0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8474034000 tx=0x7f847402e000 comp rx=0 tx=0).stop 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 msgr2=0x7f848c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f847c004710 tx=0x7f847c007700 comp rx=0 tx=0).stop 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 shutdown_connections 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f846806c920 0x7f846806ede0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f848c0c5c80 0x7f848c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f848c158d10 0x7f848c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 --2- 172.21.15.46:0/3738245174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8470003ff0 0x7f848c0b1f00 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:50.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 >> 172.21.15.46:0/3738245174 conn(0x7f848c0b27b0 msgr2=0x7f848c0ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:50.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.827+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 shutdown_connections 2024-04-26T23:52:50.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:50.828+0000 7f8492200640 1 -- 172.21.15.46:0/3738245174 wait complete. 2024-04-26T23:52:50.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:51.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.061+0000 7fa89e400640 1 Processor -- start 2024-04-26T23:52:51.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.062+0000 7fa89e400640 1 -- start start 2024-04-26T23:52:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.063+0000 7fa89e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa89815f4e0 0x7fa8980d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.063+0000 7fa89e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8980b4000 0x7fa8980d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.063+0000 7fa89e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8980d11f0 con 0x7fa8980d1f60 2024-04-26T23:52:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.063+0000 7fa89e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa8980d2990 con 0x7fa8980b4000 2024-04-26T23:52:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.063+0000 7fa89e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa8980d5f20 con 0x7fa89815f4e0 2024-04-26T23:52:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8980b4000 0x7fa8980d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa89da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa8980d1f60 0x7fa8980cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54620/0 (socket says 172.21.15.46:54620) 2024-04-26T23:52:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa89da00640 1 -- 172.21.15.46:0/2649077372 learned_addr learned my addr 172.21.15.46:0/2649077372 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa89d000640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa89815f4e0 0x7fa8980d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897400640 1 -- 172.21.15.46:0/2649077372 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748185766 0 0) 0x7fa8980d11f0 con 0x7fa8980d1f60 2024-04-26T23:52:51.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897e00640 1 -- 172.21.15.46:0/2649077372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa89815f4e0 msgr2=0x7fa8980d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:51.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897e00640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa89815f4e0 0x7fa8980d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897e00640 1 -- 172.21.15.46:0/2649077372 >> v1:172.21.15.46:6789/0 conn(0x7fa8980d1f60 legacy=0x7fa8980cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.064+0000 7fa897e00640 1 -- 172.21.15.46:0/2649077372 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8980d0f10 con 0x7fa8980b4000 2024-04-26T23:52:51.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.066+0000 7fa897e00640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8980b4000 0x7fa8980d1a20 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fa8840097a0 tx=0x7fa88402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e8344953e48612 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:51.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.066+0000 7fa897400640 1 -- 172.21.15.46:0/2649077372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884037070 con 0x7fa8980b4000 2024-04-26T23:52:51.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.066+0000 7fa897400640 1 -- 172.21.15.46:0/2649077372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa884032440 con 0x7fa8980b4000 2024-04-26T23:52:51.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.066+0000 7fa897400640 1 -- 172.21.15.46:0/2649077372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884032810 con 0x7fa8980b4000 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 -- 172.21.15.46:0/2649077372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8980b4000 msgr2=0x7fa8980d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8980b4000 0x7fa8980d1a20 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fa8840097a0 tx=0x7fa88402a180 comp rx=0 tx=0).stop 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 -- 172.21.15.46:0/2649077372 shutdown_connections 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8980b4000 0x7fa8980d1a20 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 --2- 172.21.15.46:0/2649077372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa89815f4e0 0x7fa8980d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 -- 172.21.15.46:0/2649077372 >> 172.21.15.46:0/2649077372 conn(0x7fa8980d5010 msgr2=0x7fa8980d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 -- 172.21.15.46:0/2649077372 shutdown_connections 2024-04-26T23:52:51.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.067+0000 7fa89e400640 1 -- 172.21.15.46:0/2649077372 wait complete. 2024-04-26T23:52:51.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.069+0000 7fa89e400640 1 Processor -- start 2024-04-26T23:52:51.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.069+0000 7fa89e400640 1 -- start start 2024-04-26T23:52:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8980b4000 0x7fa8980cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 0x7fa89814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa89815f8c0 con 0x7fa89815f4e0 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980b1d60 con 0x7fa8980d1f60 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa8980b3290 con 0x7fa8980b4000 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50582/0 (socket says 172.21.15.46:50582) 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.070+0000 7fa89da00640 1 -- 172.21.15.46:0/4032917068 learned_addr learned my addr 172.21.15.46:0/4032917068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89d000640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8980b4000 0x7fa8980cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 -- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8980b4000 msgr2=0x7fa8980cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8980b4000 0x7fa8980cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa897e00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 0x7fa89814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 -- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 msgr2=0x7fa89814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 0x7fa89814edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 -- 172.21.15.46:0/4032917068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89814f6c0 con 0x7fa89815f4e0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa897e00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 0x7fa89814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89da00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c7d0 tx=0x7fa89000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890018070 con 0x7fa89815f4e0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa89000a640 con 0x7fa89815f4e0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89000aa70 con 0x7fa89815f4e0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.071+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89814f9a0 con 0x7fa89815f4e0 2024-04-26T23:52:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.072+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89814ff60 con 0x7fa89815f4e0 2024-04-26T23:52:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.073+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8980d1310 con 0x7fa89815f4e0 2024-04-26T23:52:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.074+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa89001b070 con 0x7fa89815f4e0 2024-04-26T23:52:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.075+0000 7fa894c00640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.075+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fa89008b3e0 con 0x7fa89815f4e0 2024-04-26T23:52:51.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.075+0000 7fa89d000640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.076+0000 7fa89d000640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 0x7fa87806edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:51.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.079+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8980d1310 con 0x7fa89815f4e0 2024-04-26T23:52:51.174 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:52:51.170+0000 7fe3df600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:52:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.327+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8980693b0 con 0x7fa89815f4e0 2024-04-26T23:52:51.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.328+0000 7fa894c00640 1 -- 172.21.15.46:0/4032917068 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7fa8900595e0 con 0x7fa89815f4e0 2024-04-26T23:52:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:51.332 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:52:51.332 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 msgr2=0x7fa87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 0x7fa87806edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c008040 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 msgr2=0x7fa8980cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c7d0 tx=0x7fa89000cc90 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 shutdown_connections 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89815f4e0 0x7fa8980cedf0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8980d1f60 0x7fa89814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 --2- 172.21.15.46:0/4032917068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8980b4000 0x7fa8980cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 >> 172.21.15.46:0/4032917068 conn(0x7fa8980d5010 msgr2=0x7fa89806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 shutdown_connections 2024-04-26T23:52:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.332+0000 7fa89e400640 1 -- 172.21.15.46:0/4032917068 wait complete. 2024-04-26T23:52:51.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 127, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:52:44.692203+0000', 'modified': '2024-04-26T23:52:47.762256+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10287': {'gid': 10287, 'name': 'c', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/1585151449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 1585151449}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1585151449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10287, 'qdb_cluster': [10287]} 2024-04-26T23:52:51.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:52:51.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:52:51.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.980+0000 7f0e26800640 1 Processor -- start 2024-04-26T23:52:51.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.980+0000 7f0e26800640 1 -- start start 2024-04-26T23:52:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.981+0000 7f0e26800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 0x7f0e200d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.981+0000 7f0e26800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e200d11f0 con 0x7f0e200b4000 2024-04-26T23:52:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.981+0000 7f0e26800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e200d5f20 con 0x7f0e2015f4e0 2024-04-26T23:52:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.981+0000 7f0e26800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e200d0ce0 con 0x7f0e200d2130 2024-04-26T23:52:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.982+0000 7f0e25e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e200b4000 0x7f0e200b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54370/0 (socket says 172.21.15.46:54370) 2024-04-26T23:52:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.982+0000 7f0e25e00640 1 -- 172.21.15.46:0/1067320243 learned_addr learned my addr 172.21.15.46:0/1067320243 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.982+0000 7f0e24a00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 0x7f0e200d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.982+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459696980 0 0) 0x7f0e200d11f0 con 0x7f0e200b4000 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.982+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e04003520 con 0x7f0e200b4000 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2789494000 0 0) 0x7f0e04003520 con 0x7f0e200b4000 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 msgr2=0x7f0e200d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 0x7f0e200d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 >> v1:172.21.15.142:6789/0 conn(0x7f0e2015f4e0 legacy=0x7f0e200d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e200d0f10 con 0x7f0e200b4000 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0e18001670 con 0x7f0e200b4000 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 >> v1:172.21.15.46:6789/0 conn(0x7f0e200b4000 legacy=0x7f0e200b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e040039e0 0x7f0e04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 0x7f0e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0400b9c0 con 0x7f0e040039e0 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0400bb40 con 0x7f0e04004d40 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0400bd40 con 0x7f0e200d2130 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.983+0000 7f0e25400640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e040039e0 0x7f0e04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25400640 1 -- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 msgr2=0x7f0e04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25e00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25400640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e200d2130 0x7f0e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25400640 1 -- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e04004d40 msgr2=0x7f0e040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25400640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25400640 1 -- 172.21.15.46:0/1067320243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04003520 con 0x7f0e040039e0 2024-04-26T23:52:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.984+0000 7f0e25e00640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.985+0000 7f0e25400640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e040039e0 0x7f0e04003dc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001860 tx=0x7f0e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c932d9d12e326fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.985+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10037070 con 0x7f0e040039e0 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.985+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1002ab60 con 0x7f0e040039e0 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.985+0000 7f0e1fe00640 1 -- 172.21.15.46:0/1067320243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10001430 con 0x7f0e040039e0 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 -- 172.21.15.46:0/1067320243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e040039e0 msgr2=0x7f0e04003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 --2- 172.21.15.46:0/1067320243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e040039e0 0x7f0e04003dc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001860 tx=0x7f0e04005830 comp rx=0 tx=0).stop 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e25e00640 1 -- 172.21.15.46:0/1067320243 reap_dead start 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 -- 172.21.15.46:0/1067320243 shutdown_connections 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 -- 172.21.15.46:0/1067320243 >> 172.21.15.46:0/1067320243 conn(0x7f0e200d5010 msgr2=0x7f0e200d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 -- 172.21.15.46:0/1067320243 shutdown_connections 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.986+0000 7f0e26800640 1 -- 172.21.15.46:0/1067320243 wait complete. 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.988+0000 7f0e26800640 1 Processor -- start 2024-04-26T23:52:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.988+0000 7f0e26800640 1 -- start start 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e040039e0 0x7f0e2009eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e200ca460 0x7f0e200ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200b1d60 con 0x7f0e200a1a40 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200d2990 con 0x7f0e200ca460 2024-04-26T23:52:51.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e26800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2015f8c0 con 0x7f0e040039e0 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37096/0 (socket says 172.21.15.46:37096) 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 -- 172.21.15.46:0/2644971155 learned_addr learned my addr 172.21.15.46:0/2644971155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25400640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e040039e0 0x7f0e2009eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e24a00640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e200ca460 0x7f0e200ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 -- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e040039e0 msgr2=0x7f0e2009eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e040039e0 0x7f0e2009eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 -- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e200ca460 msgr2=0x7f0e200ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.989+0000 7f0e25e00640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e200ca460 0x7f0e200ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e25e00640 1 -- 172.21.15.46:0/2644971155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e200cb200 con 0x7f0e200a1a40 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e25e00640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0e2008f150 tx=0x7f0e1800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e1d600640 1 -- 172.21.15.46:0/2644971155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007440 con 0x7f0e200a1a40 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e1d600640 1 -- 172.21.15.46:0/2644971155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18007600 con 0x7f0e200a1a40 2024-04-26T23:52:51.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e1d600640 1 -- 172.21.15.46:0/2644971155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18004a20 con 0x7f0e200a1a40 2024-04-26T23:52:51.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e200a0f60 con 0x7f0e200a1a40 2024-04-26T23:52:51.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.990+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e200a1520 con 0x7f0e200a1a40 2024-04-26T23:52:51.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.993+0000 7f0e1d600640 1 -- 172.21.15.46:0/2644971155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e1801b020 con 0x7f0e200a1a40 2024-04-26T23:52:51.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.993+0000 7f0e1d600640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 0x7f0e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:51.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.994+0000 7f0e1d600640 1 -- 172.21.15.46:0/2644971155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f0e1808b790 con 0x7f0e200a1a40 2024-04-26T23:52:51.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.994+0000 7f0e25400640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 0x7f0e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:51.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.994+0000 7f0e2ac1b740 1 Processor -- start 2024-04-26T23:52:51.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.994+0000 7f0e2ac1b740 1 -- start start 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e25400640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 0x7f0e0006edb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0e10034000 tx=0x7f0e10009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d5001cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d500476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d4ff80600 con 0x558d5001c7b0 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d4ff1d100 con 0x558d50058e80 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.995+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558d4ff7c5c0 con 0x558d50047270 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37110/0 (socket says 172.21.15.46:37110) 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 -- 172.21.15.46:0/3046303819 learned_addr learned my addr 172.21.15.46:0/3046303819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25e00640 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d500476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 -- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 msgr2=0x558d500476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d500476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e24a00640 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d5001cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 -- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 msgr2=0x558d5001cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d5001cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 -- 172.21.15.46:0/3046303819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4fc922d0 con 0x558d5001c7b0 2024-04-26T23:52:52.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.996+0000 7f0e25400640 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0e10007b40 tx=0x7f0e1002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5dd4f17718ea1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e15c00640 1 -- 172.21.15.46:0/3046303819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1004e070 con 0x558d5001c7b0 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e15c00640 1 -- 172.21.15.46:0/3046303819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10038040 con 0x558d5001c7b0 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e15c00640 1 -- 172.21.15.46:0/3046303819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10037050 con 0x558d5001c7b0 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3046303819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 msgr2=0x558d5001e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0e10007b40 tx=0x7f0e1002e000 comp rx=0 tx=0).stop 2024-04-26T23:52:52.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3046303819 shutdown_connections 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d500476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d5001cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3046303819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5001e910 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.997+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3046303819 >> 172.21.15.46:0/3046303819 conn(0x558d4ffb0d10 msgr2=0x558d50062270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.998+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3046303819 shutdown_connections 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.998+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3046303819 wait complete. 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.998+0000 7f0e2ac1b740 1 -- start start 2024-04-26T23:52:52.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.998+0000 7f0e2ac1b740 1 Processor -- start 2024-04-26T23:52:52.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50047270 0x558d50032110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50058e80 0x558d50033e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d4ff07f70 con 0x558d5001c7b0 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d50044dc0 con 0x558d50047270 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:51.999+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558d50034920 con 0x558d50058e80 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e2ac1b740 10 client.0 _ll_register_callbacks cb 0x558d4ff76480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37112/0 (socket says 172.21.15.46:37112) 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 -- 172.21.15.46:0/2894371248 learned_addr learned my addr 172.21.15.46:0/2894371248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 -- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50058e80 msgr2=0x558d50033e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50058e80 0x558d50033e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 -- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50047270 msgr2=0x558d50032110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50047270 0x558d50032110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 -- 172.21.15.46:0/2894371248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4fd5c6b0 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e25400640 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0e1004c040 tx=0x7f0e1002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5999562add6c4bef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.000+0000 7f0e0a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1004e070 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f0e10054ca0 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e10038040 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f0e10037040 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e2ac1b740 10 client.10410 fetch_fsmap learned FSMap version 127 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x558d500326b0 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10038040 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f0e10003b30 con 0x558d5001c7b0 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.001+0000 7f0e2ac1b740 10 client.10410 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:52:52.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e2ac1b740 10 client.10410 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:52:52.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e2ac1b740 20 client.10410 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:52.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d50059de0 0x558d5007f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e2ac1b740 4 client.10410 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:52:52.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d4ff8e210 con 0x558d50059de0 2024-04-26T23:52:52.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e24a00640 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d50059de0 0x558d5007f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e24a00640 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d50059de0 0x558d5007f830 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1278a9871763fb server_cookie=ca4f49718d83af51 in_seq=0 out_seq=0 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.002+0000 7f0e0a000640 10 client.10410 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.003+0000 7f0e0a000640 1 -- 172.21.15.46:0/2894371248 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d4ff8e210 con 0x558d50059de0 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.003+0000 7f0e0a000640 10 client.10410 handle_command_reply: tid=0 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.003+0000 7f0e2ac1b740 1 client.10410 shutdown 2024-04-26T23:52:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 msgr2=0x558d5005b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0e1004c040 tx=0x7f0e1002c070 comp rx=0 tx=0).stop 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 shutdown_connections 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d50059de0 0x558d5007f830 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50058e80 0x558d50033e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50047270 0x558d50032110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2894371248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005b290 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 >> 172.21.15.46:0/2894371248 conn(0x558d4ffb0d10 msgr2=0x558d500422e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e0a000640 0 client.10410 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 shutdown_connections 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2894371248 wait complete. 2024-04-26T23:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.004+0000 7f0e2ac1b740 20 client.10410 trim_cache size 0 max 16384 2024-04-26T23:52:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.024+0000 7f0e2ac1b740 1 Processor -- start 2024-04-26T23:52:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.024+0000 7f0e2ac1b740 1 -- start start 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d50084a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d50084fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20201180 con 0x558d5001c7b0 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20151800 con 0x558d50058e80 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e20200b60 con 0x558d50047270 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e25400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37126/0 (socket says 172.21.15.46:37126) 2024-04-26T23:52:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.025+0000 7f0e25400640 1 -- 172.21.15.46:0/2379075097 learned_addr learned my addr 172.21.15.46:0/2379075097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 -- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 msgr2=0x558d50084a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d50084a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 -- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 msgr2=0x558d50084fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d50084fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 -- 172.21.15.46:0/2379075097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5001c320 con 0x558d5001c7b0 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e25400640 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0e1002ab10 tx=0x7f0e10042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3f6ed79612664b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e15200640 1 -- 172.21.15.46:0/2379075097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1004e070 con 0x558d5001c7b0 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e15200640 1 -- 172.21.15.46:0/2379075097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10003b30 con 0x558d5001c7b0 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.026+0000 7f0e15200640 1 -- 172.21.15.46:0/2379075097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1005d680 con 0x558d5001c7b0 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2379075097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 msgr2=0x558d5005ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0e1002ab10 tx=0x7f0e10042000 comp rx=0 tx=0).stop 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2379075097 shutdown_connections 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d50058e80 0x558d50084fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d50084a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/2379075097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d5001c7b0 0x558d5005ae40 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2379075097 >> 172.21.15.46:0/2379075097 conn(0x558d4ffb0d10 msgr2=0x558d50033b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2379075097 shutdown_connections 2024-04-26T23:52:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.027+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/2379075097 wait complete. 2024-04-26T23:52:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.028+0000 7f0e2ac1b740 1 -- start start 2024-04-26T23:52:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.028+0000 7f0e2ac1b740 1 Processor -- start 2024-04-26T23:52:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 0x558d50085110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d5001af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d50059c60 con 0x558d50058e80 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d50030960 con 0x558d5001c7b0 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558d5005a290 con 0x558d50047270 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e2ac1b740 10 client.0 _ll_register_callbacks cb 0x7f0e1004c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37142/0 (socket says 172.21.15.46:37142) 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.029+0000 7f0e25e00640 1 -- 172.21.15.46:0/3996005577 learned_addr learned my addr 172.21.15.46:0/3996005577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25400640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 0x558d50085110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 -- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 msgr2=0x558d5001af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e24a00640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d5001af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d5001af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 -- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 msgr2=0x558d50085110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 0x558d50085110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 -- 172.21.15.46:0/3996005577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4ff7c780 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25400640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 0x558d50085110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e25e00640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e010 tx=0x7f0e18059350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46f896a1b92b318 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e0a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e180594f0 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f0e18059690 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e10003b30 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f0e180619e0 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.030+0000 7f0e2ac1b740 10 client.10416 fetch_fsmap learned FSMap version 127 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x558d50073f90 con 0x558d50058e80 2024-04-26T23:52:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10003b30 con 0x558d50058e80 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f0e18004bc0 con 0x558d50058e80 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 10 client.10416 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 10 client.10416 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 20 client.10416 populate_metadata read hostname 'smithi046' 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d500e44a0 0x558d500e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 4 client.10416 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "status"}) -- 0x558d4ff88080 con 0x558d500e44a0 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.031+0000 7f0e25400640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d500e44a0 0x558d500e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e25400640 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d500e44a0 0x558d500e4880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257066dfd722c4ce server_cookie=4f242916617bbbc8 in_seq=0 out_seq=0 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e0a000640 10 client.10416 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e0a000640 1 -- 172.21.15.46:0/3996005577 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x558d4ff88080 con 0x558d500e44a0 2024-04-26T23:52:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e0a000640 10 client.10416 handle_command_reply: tid=0 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 client.10416 shutdown 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 msgr2=0x558d5001b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e010 tx=0x7f0e18059350 comp rx=0 tx=0).stop 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 shutdown_connections 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x558d500e44a0 0x558d500e4880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d50058e80 0x558d5001b4b0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558d50047270 0x558d5001af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 --2- 172.21.15.46:0/3996005577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d5001c7b0 0x558d50085110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e0a000640 0 client.10416 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 >> 172.21.15.46:0/3996005577 conn(0x558d4ffb0d10 msgr2=0x558d5005bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 shutdown_connections 2024-04-26T23:52:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.032+0000 7f0e2ac1b740 1 -- 172.21.15.46:0/3996005577 wait complete. 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e2ac1b740 20 client.10416 trim_cache size 0 max 16384 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 10287, 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:52:52.041 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7968123240000002, 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 127, 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 161, 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 161, 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.179052838 2024-04-26T23:52:52.042 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 msgr2=0x7f0e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 0x7f0e0006edb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0e10034000 tx=0x7f0e10009030 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 msgr2=0x7f0e200a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0e2008f150 tx=0x7f0e1800d9f0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 shutdown_connections 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0e0006c8f0 0x7f0e0006edb0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200a1a40 0x7f0e200a1ec0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e200ca460 0x7f0e200ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 --2- 172.21.15.46:0/2644971155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e040039e0 0x7f0e2009eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 >> 172.21.15.46:0/2644971155 conn(0x7f0e200d5010 msgr2=0x7f0e200d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 shutdown_connections 2024-04-26T23:52:52.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.033+0000 7f0e26800640 1 -- 172.21.15.46:0/2644971155 wait complete. 2024-04-26T23:52:52.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:52:52.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.255+0000 7f2fe2800640 1 Processor -- start 2024-04-26T23:52:52.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.256+0000 7f2fe2800640 1 -- start start 2024-04-26T23:52:52.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cf450 0x7f2fdc073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fdc072190 0x7f2fdc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe2800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc06f470 con 0x7f2fdc0cfd70 2024-04-26T23:52:52.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc072930 con 0x7f2fdc072190 2024-04-26T23:52:52.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc072c10 con 0x7f2fdc0cf450 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.257+0000 7f2fe0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fdc072190 0x7f2fdc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fdc072190 0x7f2fdc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37158/0 (socket says 172.21.15.46:37158) 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 -- 172.21.15.46:0/428925825 learned_addr learned my addr 172.21.15.46:0/428925825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe1400640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cf450 0x7f2fdc073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 -- 172.21.15.46:0/428925825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cf450 msgr2=0x7f2fdc073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cf450 0x7f2fdc073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 -- 172.21.15.46:0/428925825 >> v1:172.21.15.142:6789/0 conn(0x7f2fdc0cfd70 legacy=0x7f2fdc06e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.258+0000 7f2fe0a00640 1 -- 172.21.15.46:0/428925825 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc072ef0 con 0x7f2fdc072190 2024-04-26T23:52:52.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.259+0000 7f2fe0a00640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc072190 0x7f2fdc0cf830 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009870 tx=0x7f2fc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f73db49e0b4a34e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.259+0000 7f2fdbe00640 1 -- 172.21.15.46:0/428925825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8037070 con 0x7f2fdc072190 2024-04-26T23:52:52.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fdbe00640 1 -- 172.21.15.46:0/428925825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc8032440 con 0x7f2fdc072190 2024-04-26T23:52:52.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fdbe00640 1 -- 172.21.15.46:0/428925825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8032810 con 0x7f2fdc072190 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 -- 172.21.15.46:0/428925825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc072190 msgr2=0x7f2fdc0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc072190 0x7f2fdc0cf830 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009870 tx=0x7f2fc802a210 comp rx=0 tx=0).stop 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 -- 172.21.15.46:0/428925825 shutdown_connections 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc072190 0x7f2fdc0cf830 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 --2- 172.21.15.46:0/428925825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cf450 0x7f2fdc073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 -- 172.21.15.46:0/428925825 >> 172.21.15.46:0/428925825 conn(0x7f2fdc0b2820 msgr2=0x7f2fdc0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 -- 172.21.15.46:0/428925825 shutdown_connections 2024-04-26T23:52:52.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.260+0000 7f2fe2800640 1 -- 172.21.15.46:0/428925825 wait complete. 2024-04-26T23:52:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.262+0000 7f2fe2800640 1 Processor -- start 2024-04-26T23:52:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.262+0000 7f2fe2800640 1 -- start start 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc072190 0x7f2fdc13f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 0x7f2fdc1e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0d01e0 con 0x7f2fdc0cf450 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0734b0 con 0x7f2fdc072190 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.263+0000 7f2fe2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0725f0 con 0x7f2fdc0cfd70 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37162/0 (socket says 172.21.15.46:37162) 2024-04-26T23:52:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 -- 172.21.15.46:0/3305705641 learned_addr learned my addr 172.21.15.46:0/3305705641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 -- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 msgr2=0x7f2fdc1e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe1e00640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 0x7f2fdc1e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe1400640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc072190 0x7f2fdc13f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 0x7f2fdc1e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 -- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc072190 msgr2=0x7f2fdc13f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc072190 0x7f2fdc13f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe0a00640 1 -- 172.21.15.46:0/3305705641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc140e40 con 0x7f2fdc0cf450 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.264+0000 7f2fe1e00640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 0x7f2fdc1e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fe0a00640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009990 tx=0x7f2fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8041020 con 0x7f2fdc0cf450 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc8030780 con 0x7f2fdc0cf450 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8037040 con 0x7f2fdc0cf450 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc141120 con 0x7f2fdc0cf450 2024-04-26T23:52:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.265+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1416e0 con 0x7f2fdc0cf450 2024-04-26T23:52:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.266+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc0693b0 con 0x7f2fdc0cf450 2024-04-26T23:52:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.267+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fc8044020 con 0x7f2fdc0cf450 2024-04-26T23:52:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.268+0000 7f2fd9600640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 0x7f2fbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.268+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f2fc80ac580 con 0x7f2fdc0cf450 2024-04-26T23:52:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.269+0000 7f2fe1400640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 0x7f2fbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.270+0000 7f2fe1400640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 0x7f2fbc06ede0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.273+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fc804b030 con 0x7f2fdc0cf450 2024-04-26T23:52:52.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.518+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2fdc073df0 con 0x7f2fdc0cf450 2024-04-26T23:52:52.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.519+0000 7f2fd9600640 1 -- 172.21.15.46:0/3305705641 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v127) ==== 53+0+73 (secure 0 0 0) 0x7f2fc807a6f0 con 0x7f2fdc0cf450 2024-04-26T23:52:52.523 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 msgr2=0x7f2fbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 0x7f2fbc06ede0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0008040 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 msgr2=0x7f2fdc0b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009990 tx=0x7f2fc80046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 shutdown_connections 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2fbc06c920 0x7f2fbc06ede0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0cfd70 0x7f2fdc1e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc0cf450 0x7f2fdc0b3fb0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 --2- 172.21.15.46:0/3305705641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc072190 0x7f2fdc13f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 >> 172.21.15.46:0/3305705641 conn(0x7f2fdc0b2820 msgr2=0x7f2fdc0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 shutdown_connections 2024-04-26T23:52:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52.522+0000 7f2fe2800640 1 -- 172.21.15.46:0/3305705641 wait complete. 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:52:52.545 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:52:52.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:52.545 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:52:52.600 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41648sec preferred_lft 41648sec 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.601 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:52:52.602 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:52:52.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:52.603 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:52:52.683 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:52:52.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:52.688 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:52:52.741 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:52:52.741 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:52:52.741 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:52:52.741 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:52:52.741 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:52:52.741 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:52:52.742 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:52.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:52:52.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:52:52.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:52:52.742 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:52:52.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:52.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:52.967 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:52:53.021 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:52:53.021 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:52:53.022 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:52:53.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:53.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:52:53.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:52:53.113 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:52:53.168 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:52:53.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:52:53.169 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:52:53.222 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:52:53.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:53.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:52:53.352 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:52:53.352 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:52:53.353 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:52:53.374 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:52:53.374 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:52:53.374 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:52:53.374 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:52:53.374 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:52:53.398 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:52:53.398 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:52:53.398 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:52:53.399 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:52:53.399 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:52:53.399 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:52:53.399 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:52:53.399 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:52:53.400 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:52:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:52:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:52:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:52:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:52:53.406 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:52:53.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:53.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:52:53.491 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:52:53.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:53.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:53.641 INFO:teuthology.orchestra.run.smithi046.stdout:client6545 2024-04-26T23:52:53.641 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:52:53.641 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:52:53.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.931+0000 7f0bc9800640 1 Processor -- start 2024-04-26T23:52:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.932+0000 7f0bc9800640 1 -- start start 2024-04-26T23:52:53.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.933+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.933+0000 7f0bc9800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 0x7f0bc40d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.933+0000 7f0bc9800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc40b65e0 con 0x7f0bc40d5a90 2024-04-26T23:52:53.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.933+0000 7f0bc9800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc40d12e0 con 0x7f0bc40b4e20 2024-04-26T23:52:53.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.933+0000 7f0bc9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc40d05f0 con 0x7f0bc4162a60 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.934+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.934+0000 7f0bc8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 0x7f0bc40d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.934+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:51874/0 (socket says 172.21.15.46:51874) 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.934+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2395025496 learned_addr learned my addr 172.21.15.46:0/2395025496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.935+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2395025496 >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 msgr2=0x7f0bc40d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.935+0000 7f0bc3e00640 1 --2- 172.21.15.46:0/2395025496 >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 0x7f0bc40d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.935+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2395025496 >> v1:172.21.15.142:6789/0 conn(0x7f0bc40d5a90 legacy=0x7f0bc40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.935+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2395025496 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40d1530 con 0x7f0bc4162a60 2024-04-26T23:52:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.935+0000 7f0bc8e00640 1 --2- 172.21.15.46:0/2395025496 >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 0x7f0bc40d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.936+0000 7f0bc3e00640 1 --2- 172.21.15.46:0/2395025496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009870 tx=0x7f0bb402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4608250b6950e642 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.937+0000 7f0bc2a00640 1 -- 172.21.15.46:0/2395025496 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4037070 con 0x7f0bc4162a60 2024-04-26T23:52:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.937+0000 7f0bc2a00640 1 -- 172.21.15.46:0/2395025496 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb4032440 con 0x7f0bc4162a60 2024-04-26T23:52:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.937+0000 7f0bc2a00640 1 -- 172.21.15.46:0/2395025496 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4032810 con 0x7f0bc4162a60 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 -- 172.21.15.46:0/2395025496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 msgr2=0x7f0bc40d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2395025496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009870 tx=0x7f0bb402a130 comp rx=0 tx=0).stop 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 -- 172.21.15.46:0/2395025496 shutdown_connections 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2395025496 >> v2:172.21.15.142:3300/0 conn(0x7f0bc40b4e20 0x7f0bc40d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2395025496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc4162a60 0x7f0bc40d0a80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 -- 172.21.15.46:0/2395025496 >> 172.21.15.46:0/2395025496 conn(0x7f0bc40d4fa0 msgr2=0x7f0bc40d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 -- 172.21.15.46:0/2395025496 shutdown_connections 2024-04-26T23:52:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.938+0000 7f0bc9800640 1 -- 172.21.15.46:0/2395025496 wait complete. 2024-04-26T23:52:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.940+0000 7f0bc9800640 1 Processor -- start 2024-04-26T23:52:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.941+0000 7f0bc9800640 1 -- start start 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc40d5a90 0x7f0bc40b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 0x7f0bc40cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40b52d0 con 0x7f0bc40b4e20 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40d1ef0 con 0x7f0bc4162a60 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc40c94b0 con 0x7f0bc40d5a90 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37172/0 (socket says 172.21.15.46:37172) 2024-04-26T23:52:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.942+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2274590241 learned_addr learned my addr 172.21.15.46:0/2274590241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc8e00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 0x7f0bc40cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3400640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc40d5a90 0x7f0bc40b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc40d5a90 msgr2=0x7f0bc40b9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc40d5a90 0x7f0bc40b9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 msgr2=0x7f0bc40cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 0x7f0bc40cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 -- 172.21.15.46:0/2274590241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40b8e20 con 0x7f0bc40b4e20 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc8e00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 0x7f0bc40cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.943+0000 7f0bc3e00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.944+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4043070 con 0x7f0bc40b4e20 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.944+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb4038040 con 0x7f0bc40b4e20 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.944+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4037050 con 0x7f0bc40b4e20 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.944+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc40b9100 con 0x7f0bc40b4e20 2024-04-26T23:52:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.944+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc415a460 con 0x7f0bc40b4e20 2024-04-26T23:52:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.945+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4069ac0 con 0x7f0bc40b4e20 2024-04-26T23:52:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.946+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bb403cb50 con 0x7f0bc40b4e20 2024-04-26T23:52:53.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.947+0000 7f0bbbe00640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:53.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.947+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f0bb40abcb0 con 0x7f0bc40b4e20 2024-04-26T23:52:53.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.948+0000 7f0bc3400640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:53.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.949+0000 7f0bc3400640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 0x7f0ba006edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0bc406e130 tx=0x7f0bac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:53.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:53.952+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bb4079f40 con 0x7f0bc40b4e20 2024-04-26T23:52:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.200+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bc40b4840 con 0x7f0bc40b4e20 2024-04-26T23:52:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.201+0000 7f0bbbe00640 1 -- 172.21.15.46:0/2274590241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7f0bb4079840 con 0x7f0bc40b4e20 2024-04-26T23:52:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:54.206 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:52:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:52:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 msgr2=0x7f0ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 0x7f0ba006edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0bc406e130 tx=0x7f0bac009040 comp rx=0 tx=0).stop 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 msgr2=0x7f0bc4159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb40046c0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 shutdown_connections 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc4162a60 0x7f0bc40cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bc40d5a90 0x7f0bc40b9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 --2- 172.21.15.46:0/2274590241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc40b4e20 0x7f0bc4159f20 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 >> 172.21.15.46:0/2274590241 conn(0x7f0bc40d4fa0 msgr2=0x7f0bc406e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 shutdown_connections 2024-04-26T23:52:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.205+0000 7f0bc9800640 1 -- 172.21.15.46:0/2274590241 wait complete. 2024-04-26T23:52:54.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:54.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.434+0000 7f9a77e00640 1 Processor -- start 2024-04-26T23:52:54.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.434+0000 7f9a77e00640 1 -- start start 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815edf0 0x7f9a780cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a780cd770 0x7f9a780c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a780b03c0 con 0x7f9a7815f1d0 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9a780cf5c0 con 0x7f9a780cd770 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a780cb680 con 0x7f9a7815edf0 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a76000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a7815f1d0 0x7f9a780cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54386/0 (socket says 172.21.15.46:54386) 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a76000640 1 -- 172.21.15.46:0/2238471649 learned_addr learned my addr 172.21.15.46:0/2238471649 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:52:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.436+0000 7f9a77400640 1 --2- 172.21.15.46:0/2238471649 >> v2:172.21.15.46:3300/0 conn(0x7f9a780cd770 0x7f9a780c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a76a00640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815edf0 0x7f9a780cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a75600640 1 -- 172.21.15.46:0/2238471649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902798115 0 0) 0x7f9a780b03c0 con 0x7f9a7815f1d0 2024-04-26T23:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a75600640 1 -- 172.21.15.46:0/2238471649 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a5c003520 con 0x7f9a7815f1d0 2024-04-26T23:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a77400640 1 -- 172.21.15.46:0/2238471649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815edf0 msgr2=0x7f9a780cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a77400640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815edf0 0x7f9a780cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a77400640 1 -- 172.21.15.46:0/2238471649 >> v1:172.21.15.46:6789/0 conn(0x7f9a7815f1d0 legacy=0x7f9a780cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.437+0000 7f9a77400640 1 -- 172.21.15.46:0/2238471649 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a780cb960 con 0x7f9a780cd770 2024-04-26T23:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.438+0000 7f9a77400640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a780c5940 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009d90 tx=0x7f9a7002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7781a5e0e35dd8de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.438+0000 7f9a75600640 1 -- 172.21.15.46:0/2238471649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007740 con 0x7f9a780cd770 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.438+0000 7f9a75600640 1 -- 172.21.15.46:0/2238471649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a700078e0 con 0x7f9a780cd770 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.438+0000 7f9a75600640 1 -- 172.21.15.46:0/2238471649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007cb0 con 0x7f9a780cd770 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 -- 172.21.15.46:0/2238471649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 msgr2=0x7f9a780c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a780c5940 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009d90 tx=0x7f9a7002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 -- 172.21.15.46:0/2238471649 shutdown_connections 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a780c5940 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 --2- 172.21.15.46:0/2238471649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815edf0 0x7f9a780cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 -- 172.21.15.46:0/2238471649 >> 172.21.15.46:0/2238471649 conn(0x7f9a780b26d0 msgr2=0x7f9a780b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 -- 172.21.15.46:0/2238471649 shutdown_connections 2024-04-26T23:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.439+0000 7f9a77e00640 1 -- 172.21.15.46:0/2238471649 wait complete. 2024-04-26T23:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.441+0000 7f9a77e00640 1 Processor -- start 2024-04-26T23:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.441+0000 7f9a77e00640 1 -- start start 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a7815edf0 0x7f9a7814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815ffd0 0x7f9a78160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a780cc200 con 0x7f9a780cd770 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a780cbf20 con 0x7f9a7815edf0 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a77e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9a780cb1a0 con 0x7f9a7815ffd0 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a76a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a76a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37190/0 (socket says 172.21.15.46:37190) 2024-04-26T23:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.442+0000 7f9a76a00640 1 -- 172.21.15.46:0/120005010 learned_addr learned my addr 172.21.15.46:0/120005010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 -- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815ffd0 msgr2=0x7f9a78160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815ffd0 0x7f9a78160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 -- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a7815edf0 msgr2=0x7f9a7814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76000640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a7815edf0 0x7f9a7814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a7815edf0 0x7f9a7814f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 -- 172.21.15.46:0/120005010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a78162080 con 0x7f9a780cd770 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a76a00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d800 tx=0x7f9a6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800b6c0 con 0x7f9a780cd770 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a68019070 con 0x7f9a780cd770 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ba80 con 0x7f9a780cd770 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.443+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a78160d80 con 0x7f9a780cd770 2024-04-26T23:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.444+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a78161340 con 0x7f9a780cd770 2024-04-26T23:52:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.445+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a78068bc0 con 0x7f9a780cd770 2024-04-26T23:52:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.446+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a6800bc20 con 0x7f9a780cd770 2024-04-26T23:52:54.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.447+0000 7f9a6ea00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:54.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.447+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f9a6808c1e0 con 0x7f9a780cd770 2024-04-26T23:52:54.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.447+0000 7f9a76000640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:54.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.448+0000 7f9a76000640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 0x7f9a5806edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9a7815e390 tx=0x7f9a64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.451+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a78068bc0 con 0x7f9a780cd770 2024-04-26T23:52:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.696+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a780693b0 con 0x7f9a780cd770 2024-04-26T23:52:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.698+0000 7f9a6ea00640 1 -- 172.21.15.46:0/120005010 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7f9a6801c360 con 0x7f9a780cd770 2024-04-26T23:52:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:52:54.702 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:52:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:52:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 msgr2=0x7f9a5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 0x7f9a5806edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9a7815e390 tx=0x7f9a64009040 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 msgr2=0x7f9a78161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d800 tx=0x7f9a6800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 shutdown_connections 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a7815ffd0 0x7f9a78160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a7815edf0 0x7f9a7814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 --2- 172.21.15.46:0/120005010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a780cd770 0x7f9a78161b40 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.700+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 >> 172.21.15.46:0/120005010 conn(0x7f9a780b26d0 msgr2=0x7f9a780c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.701+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 shutdown_connections 2024-04-26T23:52:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54.701+0000 7f9a77e00640 1 -- 172.21.15.46:0/120005010 wait complete. 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.0 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:54.724 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:54.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:54.916 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442981 s, 23.7 MB/s 2024-04-26T23:52:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331587 s, 31.6 MB/s 2024-04-26T23:52:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331263 s, 31.7 MB/s 2024-04-26T23:52:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265062 s, 39.6 MB/s 2024-04-26T23:52:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294487 s, 35.6 MB/s 2024-04-26T23:52:55.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.098 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301646 s, 34.8 MB/s 2024-04-26T23:52:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224076 s, 46.8 MB/s 2024-04-26T23:52:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022935 s, 45.7 MB/s 2024-04-26T23:52:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232215 s, 45.2 MB/s 2024-04-26T23:52:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291599 s, 36.0 MB/s 2024-04-26T23:52:55.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334425 s, 31.4 MB/s 2024-04-26T23:52:55.284 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.284 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222456 s, 47.1 MB/s 2024-04-26T23:52:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295893 s, 35.4 MB/s 2024-04-26T23:52:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282812 s, 37.1 MB/s 2024-04-26T23:52:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269217 s, 38.9 MB/s 2024-04-26T23:52:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280987 s, 37.3 MB/s 2024-04-26T23:52:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208536 s, 50.3 MB/s 2024-04-26T23:52:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252071 s, 41.6 MB/s 2024-04-26T23:52:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306261 s, 34.2 MB/s 2024-04-26T23:52:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325283 s, 32.2 MB/s 2024-04-26T23:52:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229153 s, 45.8 MB/s 2024-04-26T23:52:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377446 s, 27.8 MB/s 2024-04-26T23:52:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042795 s, 24.5 MB/s 2024-04-26T23:52:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292359 s, 35.9 MB/s 2024-04-26T23:52:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226826 s, 46.2 MB/s 2024-04-26T23:52:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239785 s, 43.7 MB/s 2024-04-26T23:52:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250823 s, 41.8 MB/s 2024-04-26T23:52:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217876 s, 48.1 MB/s 2024-04-26T23:52:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264505 s, 39.6 MB/s 2024-04-26T23:52:55.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307085 s, 34.1 MB/s 2024-04-26T23:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223166 s, 47.0 MB/s 2024-04-26T23:52:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:55.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300795 s, 34.9 MB/s 2024-04-26T23:52:55.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.1 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:55.975 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252001 s, 41.6 MB/s 2024-04-26T23:52:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294738 s, 35.6 MB/s 2024-04-26T23:52:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286771 s, 36.6 MB/s 2024-04-26T23:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022469 s, 46.7 MB/s 2024-04-26T23:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233735 s, 44.9 MB/s 2024-04-26T23:52:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242759 s, 43.2 MB/s 2024-04-26T23:52:56.323 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.323 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.323 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300371 s, 34.9 MB/s 2024-04-26T23:52:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224353 s, 46.7 MB/s 2024-04-26T23:52:56.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.384 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.384 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230873 s, 45.4 MB/s 2024-04-26T23:52:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245395 s, 42.7 MB/s 2024-04-26T23:52:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311788 s, 33.6 MB/s 2024-04-26T23:52:56.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022741 s, 46.1 MB/s 2024-04-26T23:52:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226304 s, 46.3 MB/s 2024-04-26T23:52:56.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243122 s, 43.1 MB/s 2024-04-26T23:52:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275806 s, 38.0 MB/s 2024-04-26T23:52:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225267 s, 46.5 MB/s 2024-04-26T23:52:56.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250802 s, 41.8 MB/s 2024-04-26T23:52:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274736 s, 38.2 MB/s 2024-04-26T23:52:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248895 s, 42.1 MB/s 2024-04-26T23:52:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223387 s, 46.9 MB/s 2024-04-26T23:52:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190247 s, 55.1 MB/s 2024-04-26T23:52:56.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238725 s, 43.9 MB/s 2024-04-26T23:52:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214284 s, 48.9 MB/s 2024-04-26T23:52:56.865 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308112 s, 34.0 MB/s 2024-04-26T23:52:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213496 s, 49.1 MB/s 2024-04-26T23:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02392 s, 43.8 MB/s 2024-04-26T23:52:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219549 s, 47.8 MB/s 2024-04-26T23:52:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248216 s, 42.2 MB/s 2024-04-26T23:52:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217466 s, 48.2 MB/s 2024-04-26T23:52:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028106 s, 37.3 MB/s 2024-04-26T23:52:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187826 s, 55.8 MB/s 2024-04-26T23:52:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278824 s, 37.6 MB/s 2024-04-26T23:52:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.2 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:57.116 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:57.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266468 s, 39.4 MB/s 2024-04-26T23:52:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274357 s, 38.2 MB/s 2024-04-26T23:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256659 s, 40.9 MB/s 2024-04-26T23:52:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248293 s, 42.2 MB/s 2024-04-26T23:52:57.389 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.390 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.390 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220677 s, 47.5 MB/s 2024-04-26T23:52:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300384 s, 34.9 MB/s 2024-04-26T23:52:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210932 s, 49.7 MB/s 2024-04-26T23:52:57.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.481 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222828 s, 47.1 MB/s 2024-04-26T23:52:57.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.510 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229541 s, 45.7 MB/s 2024-04-26T23:52:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195816 s, 53.5 MB/s 2024-04-26T23:52:57.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.574 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318845 s, 32.9 MB/s 2024-04-26T23:52:57.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.611 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304307 s, 34.5 MB/s 2024-04-26T23:52:57.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243516 s, 43.1 MB/s 2024-04-26T23:52:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.677 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281097 s, 37.3 MB/s 2024-04-26T23:52:57.707 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242681 s, 43.2 MB/s 2024-04-26T23:52:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252079 s, 41.6 MB/s 2024-04-26T23:52:57.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.770 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242096 s, 43.3 MB/s 2024-04-26T23:52:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317666 s, 33.0 MB/s 2024-04-26T23:52:57.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.836 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218229 s, 48.0 MB/s 2024-04-26T23:52:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217474 s, 48.2 MB/s 2024-04-26T23:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262122 s, 40.0 MB/s 2024-04-26T23:52:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226372 s, 46.3 MB/s 2024-04-26T23:52:57.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232473 s, 45.1 MB/s 2024-04-26T23:52:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221194 s, 47.4 MB/s 2024-04-26T23:52:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297766 s, 35.2 MB/s 2024-04-26T23:52:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303007 s, 34.6 MB/s 2024-04-26T23:52:58.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.088 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.088 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236658 s, 44.3 MB/s 2024-04-26T23:52:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021981 s, 47.7 MB/s 2024-04-26T23:52:58.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.139 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175149 s, 59.9 MB/s 2024-04-26T23:52:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216942 s, 48.3 MB/s 2024-04-26T23:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328962 s, 31.9 MB/s 2024-04-26T23:52:58.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265183 s, 39.5 MB/s 2024-04-26T23:52:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.3 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:58.253 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:58.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:58.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222004 s, 47.2 MB/s 2024-04-26T23:52:58.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.426 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210715 s, 49.8 MB/s 2024-04-26T23:52:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233997 s, 44.8 MB/s 2024-04-26T23:52:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278276 s, 37.7 MB/s 2024-04-26T23:52:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023179 s, 45.2 MB/s 2024-04-26T23:52:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258713 s, 40.5 MB/s 2024-04-26T23:52:58.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.582 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.582 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024983 s, 42.0 MB/s 2024-04-26T23:52:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225266 s, 46.5 MB/s 2024-04-26T23:52:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288479 s, 36.3 MB/s 2024-04-26T23:52:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223683 s, 46.9 MB/s 2024-04-26T23:52:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025484 s, 41.1 MB/s 2024-04-26T23:52:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219738 s, 47.7 MB/s 2024-04-26T23:52:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241895 s, 43.3 MB/s 2024-04-26T23:52:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294283 s, 35.6 MB/s 2024-04-26T23:52:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246551 s, 42.5 MB/s 2024-04-26T23:52:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256727 s, 40.8 MB/s 2024-04-26T23:52:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247972 s, 42.3 MB/s 2024-04-26T23:52:58.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.918 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205971 s, 50.9 MB/s 2024-04-26T23:52:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181353 s, 57.8 MB/s 2024-04-26T23:52:58.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:58.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:58.975 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271245 s, 38.7 MB/s 2024-04-26T23:52:59.002 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.002 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.002 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203743 s, 51.5 MB/s 2024-04-26T23:52:59.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201155 s, 52.1 MB/s 2024-04-26T23:52:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263127 s, 39.9 MB/s 2024-04-26T23:52:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213925 s, 49.0 MB/s 2024-04-26T23:52:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021297 s, 49.2 MB/s 2024-04-26T23:52:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211165 s, 49.7 MB/s 2024-04-26T23:52:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022823 s, 45.9 MB/s 2024-04-26T23:52:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278725 s, 37.6 MB/s 2024-04-26T23:52:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248719 s, 42.2 MB/s 2024-04-26T23:52:59.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.260 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.260 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251276 s, 41.7 MB/s 2024-04-26T23:52:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245607 s, 42.7 MB/s 2024-04-26T23:52:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268748 s, 39.0 MB/s 2024-04-26T23:52:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:52:59.342 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.4 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:52:59.343 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:52:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:52:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:52:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261024 s, 40.2 MB/s 2024-04-26T23:52:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315602 s, 33.2 MB/s 2024-04-26T23:52:59.556 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.557 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.557 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021434 s, 48.9 MB/s 2024-04-26T23:52:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248427 s, 42.2 MB/s 2024-04-26T23:52:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235812 s, 44.5 MB/s 2024-04-26T23:52:59.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189011 s, 55.5 MB/s 2024-04-26T23:52:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289609 s, 36.2 MB/s 2024-04-26T23:52:59.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.702 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215138 s, 48.7 MB/s 2024-04-26T23:52:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297711 s, 35.2 MB/s 2024-04-26T23:52:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245434 s, 42.7 MB/s 2024-04-26T23:52:59.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.796 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229384 s, 45.7 MB/s 2024-04-26T23:52:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282753 s, 37.1 MB/s 2024-04-26T23:52:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356549 s, 29.4 MB/s 2024-04-26T23:52:59.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.905 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254722 s, 41.2 MB/s 2024-04-26T23:52:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272195 s, 38.5 MB/s 2024-04-26T23:52:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:52:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:52:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023999 s, 43.7 MB/s 2024-04-26T23:53:00.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.007 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.007 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303735 s, 34.5 MB/s 2024-04-26T23:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256437 s, 40.9 MB/s 2024-04-26T23:53:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194134 s, 54.0 MB/s 2024-04-26T23:53:00.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308464 s, 34.0 MB/s 2024-04-26T23:53:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270604 s, 38.7 MB/s 2024-04-26T23:53:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026024 s, 40.3 MB/s 2024-04-26T23:53:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291206 s, 36.0 MB/s 2024-04-26T23:53:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281939 s, 37.2 MB/s 2024-04-26T23:53:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264876 s, 39.6 MB/s 2024-04-26T23:53:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245317 s, 42.7 MB/s 2024-04-26T23:53:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252907 s, 41.5 MB/s 2024-04-26T23:53:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286228 s, 36.6 MB/s 2024-04-26T23:53:00.398 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263331 s, 39.8 MB/s 2024-04-26T23:53:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274241 s, 38.2 MB/s 2024-04-26T23:53:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220959 s, 47.5 MB/s 2024-04-26T23:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199435 s, 52.6 MB/s 2024-04-26T23:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:00.498 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:00.498 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:00.499 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.5 2024-04-26T23:53:00.499 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:00.499 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:00.499 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:00.499 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241229 s, 43.5 MB/s 2024-04-26T23:53:00.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.676 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224689 s, 46.7 MB/s 2024-04-26T23:53:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276562 s, 37.9 MB/s 2024-04-26T23:53:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214224 s, 48.9 MB/s 2024-04-26T23:53:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240321 s, 43.6 MB/s 2024-04-26T23:53:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312948 s, 33.5 MB/s 2024-04-26T23:53:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221583 s, 47.3 MB/s 2024-04-26T23:53:00.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194735 s, 53.8 MB/s 2024-04-26T23:53:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239562 s, 43.8 MB/s 2024-04-26T23:53:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.922 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271194 s, 38.7 MB/s 2024-04-26T23:53:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308601 s, 34.0 MB/s 2024-04-26T23:53:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025437 s, 41.2 MB/s 2024-04-26T23:53:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239065 s, 43.9 MB/s 2024-04-26T23:53:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241509 s, 43.4 MB/s 2024-04-26T23:53:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222948 s, 47.0 MB/s 2024-04-26T23:53:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225094 s, 46.6 MB/s 2024-04-26T23:53:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287125 s, 36.5 MB/s 2024-04-26T23:53:01.177 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308163 s, 34.0 MB/s 2024-04-26T23:53:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294399 s, 35.6 MB/s 2024-04-26T23:53:01.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.248 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293566 s, 35.7 MB/s 2024-04-26T23:53:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279687 s, 37.5 MB/s 2024-04-26T23:53:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244038 s, 43.0 MB/s 2024-04-26T23:53:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.350 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278738 s, 37.6 MB/s 2024-04-26T23:53:01.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282912 s, 37.1 MB/s 2024-04-26T23:53:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238982 s, 43.9 MB/s 2024-04-26T23:53:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235569 s, 44.5 MB/s 2024-04-26T23:53:01.478 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.478 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.478 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228826 s, 45.8 MB/s 2024-04-26T23:53:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313372 s, 33.5 MB/s 2024-04-26T23:53:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247382 s, 42.4 MB/s 2024-04-26T23:53:01.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.581 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243128 s, 43.1 MB/s 2024-04-26T23:53:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210066 s, 49.9 MB/s 2024-04-26T23:53:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259396 s, 40.4 MB/s 2024-04-26T23:53:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.6 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:01.658 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020383 s, 51.4 MB/s 2024-04-26T23:53:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258015 s, 40.6 MB/s 2024-04-26T23:53:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232731 s, 45.1 MB/s 2024-04-26T23:53:01.896 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.896 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.896 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270779 s, 38.7 MB/s 2024-04-26T23:53:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018187 s, 57.7 MB/s 2024-04-26T23:53:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255706 s, 41.0 MB/s 2024-04-26T23:53:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272115 s, 38.5 MB/s 2024-04-26T23:53:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270037 s, 38.8 MB/s 2024-04-26T23:53:02.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.058 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319929 s, 32.8 MB/s 2024-04-26T23:53:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200056 s, 52.4 MB/s 2024-04-26T23:53:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245646 s, 42.7 MB/s 2024-04-26T23:53:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261409 s, 40.1 MB/s 2024-04-26T23:53:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316569 s, 33.1 MB/s 2024-04-26T23:53:02.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.220 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.220 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263395 s, 39.8 MB/s 2024-04-26T23:53:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273166 s, 38.4 MB/s 2024-04-26T23:53:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229945 s, 45.6 MB/s 2024-04-26T23:53:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225714 s, 46.5 MB/s 2024-04-26T23:53:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231619 s, 45.3 MB/s 2024-04-26T23:53:02.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.376 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.376 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237876 s, 44.1 MB/s 2024-04-26T23:53:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307055 s, 34.1 MB/s 2024-04-26T23:53:02.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263252 s, 39.8 MB/s 2024-04-26T23:53:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250859 s, 41.8 MB/s 2024-04-26T23:53:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248256 s, 42.2 MB/s 2024-04-26T23:53:02.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255947 s, 41.0 MB/s 2024-04-26T23:53:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253877 s, 41.3 MB/s 2024-04-26T23:53:02.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289314 s, 36.2 MB/s 2024-04-26T23:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319096 s, 32.9 MB/s 2024-04-26T23:53:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233885 s, 44.8 MB/s 2024-04-26T23:53:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202088 s, 51.9 MB/s 2024-04-26T23:53:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246069 s, 42.6 MB/s 2024-04-26T23:53:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223305 s, 47.0 MB/s 2024-04-26T23:53:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234717 s, 44.7 MB/s 2024-04-26T23:53:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:02.818 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:02.818 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:02.818 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.7 2024-04-26T23:53:02.818 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:02.818 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:02.819 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:02.819 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:02.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259335 s, 40.4 MB/s 2024-04-26T23:53:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.008 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344989 s, 30.4 MB/s 2024-04-26T23:53:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022313 s, 47.0 MB/s 2024-04-26T23:53:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179344 s, 58.5 MB/s 2024-04-26T23:53:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284876 s, 36.8 MB/s 2024-04-26T23:53:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320495 s, 32.7 MB/s 2024-04-26T23:53:03.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.167 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.167 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242081 s, 43.3 MB/s 2024-04-26T23:53:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199444 s, 52.6 MB/s 2024-04-26T23:53:03.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.222 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218664 s, 48.0 MB/s 2024-04-26T23:53:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261789 s, 40.1 MB/s 2024-04-26T23:53:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264544 s, 39.6 MB/s 2024-04-26T23:53:03.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.317 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216969 s, 48.3 MB/s 2024-04-26T23:53:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269602 s, 38.9 MB/s 2024-04-26T23:53:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230301 s, 45.5 MB/s 2024-04-26T23:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267393 s, 39.2 MB/s 2024-04-26T23:53:03.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.449 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.449 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299919 s, 35.0 MB/s 2024-04-26T23:53:03.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.480 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244189 s, 42.9 MB/s 2024-04-26T23:53:03.511 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259832 s, 40.4 MB/s 2024-04-26T23:53:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206832 s, 50.7 MB/s 2024-04-26T23:53:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216827 s, 48.4 MB/s 2024-04-26T23:53:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282818 s, 37.1 MB/s 2024-04-26T23:53:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273066 s, 38.4 MB/s 2024-04-26T23:53:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293851 s, 35.7 MB/s 2024-04-26T23:53:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273803 s, 38.3 MB/s 2024-04-26T23:53:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227539 s, 46.1 MB/s 2024-04-26T23:53:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213936 s, 49.0 MB/s 2024-04-26T23:53:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281918 s, 37.2 MB/s 2024-04-26T23:53:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286869 s, 36.6 MB/s 2024-04-26T23:53:03.856 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.857 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218866 s, 47.9 MB/s 2024-04-26T23:53:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272502 s, 38.5 MB/s 2024-04-26T23:53:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025527 s, 41.1 MB/s 2024-04-26T23:53:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232871 s, 45.0 MB/s 2024-04-26T23:53:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.8 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:03.964 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:04.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.118 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292234 s, 35.9 MB/s 2024-04-26T23:53:04.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.153 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.153 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296275 s, 35.4 MB/s 2024-04-26T23:53:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292692 s, 35.8 MB/s 2024-04-26T23:53:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230938 s, 45.4 MB/s 2024-04-26T23:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228414 s, 45.9 MB/s 2024-04-26T23:53:04.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270713 s, 38.7 MB/s 2024-04-26T23:53:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250994 s, 41.8 MB/s 2024-04-26T23:53:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226279 s, 46.3 MB/s 2024-04-26T23:53:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232928 s, 45.0 MB/s 2024-04-26T23:53:04.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214567 s, 48.9 MB/s 2024-04-26T23:53:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028151 s, 37.2 MB/s 2024-04-26T23:53:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284121 s, 36.9 MB/s 2024-04-26T23:53:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327792 s, 32.0 MB/s 2024-04-26T23:53:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024652 s, 42.5 MB/s 2024-04-26T23:53:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231474 s, 45.3 MB/s 2024-04-26T23:53:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244911 s, 42.8 MB/s 2024-04-26T23:53:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252085 s, 41.6 MB/s 2024-04-26T23:53:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255708 s, 41.0 MB/s 2024-04-26T23:53:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233234 s, 45.0 MB/s 2024-04-26T23:53:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240008 s, 43.7 MB/s 2024-04-26T23:53:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229854 s, 45.6 MB/s 2024-04-26T23:53:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274395 s, 38.2 MB/s 2024-04-26T23:53:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224304 s, 46.7 MB/s 2024-04-26T23:53:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180033 s, 58.2 MB/s 2024-04-26T23:53:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218685 s, 47.9 MB/s 2024-04-26T23:53:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307883 s, 34.1 MB/s 2024-04-26T23:53:04.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.936 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252482 s, 41.5 MB/s 2024-04-26T23:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196827 s, 53.3 MB/s 2024-04-26T23:53:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205438 s, 51.0 MB/s 2024-04-26T23:53:05.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.013 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162717 s, 64.4 MB/s 2024-04-26T23:53:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228308 s, 45.9 MB/s 2024-04-26T23:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229623 s, 45.7 MB/s 2024-04-26T23:53:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.9 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:05.083 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256609 s, 40.9 MB/s 2024-04-26T23:53:05.257 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023801 s, 44.1 MB/s 2024-04-26T23:53:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233857 s, 44.8 MB/s 2024-04-26T23:53:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243547 s, 43.1 MB/s 2024-04-26T23:53:05.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02484 s, 42.2 MB/s 2024-04-26T23:53:05.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239485 s, 43.8 MB/s 2024-04-26T23:53:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319365 s, 32.8 MB/s 2024-04-26T23:53:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024578 s, 42.7 MB/s 2024-04-26T23:53:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241446 s, 43.4 MB/s 2024-04-26T23:53:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325184 s, 32.2 MB/s 2024-04-26T23:53:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188182 s, 55.7 MB/s 2024-04-26T23:53:05.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.581 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.581 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315068 s, 33.3 MB/s 2024-04-26T23:53:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197326 s, 53.1 MB/s 2024-04-26T23:53:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248776 s, 42.1 MB/s 2024-04-26T23:53:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240603 s, 43.6 MB/s 2024-04-26T23:53:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328689 s, 31.9 MB/s 2024-04-26T23:53:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304931 s, 34.4 MB/s 2024-04-26T23:53:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272357 s, 38.5 MB/s 2024-04-26T23:53:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307561 s, 34.1 MB/s 2024-04-26T23:53:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233455 s, 44.9 MB/s 2024-04-26T23:53:05.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215623 s, 48.6 MB/s 2024-04-26T23:53:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193964 s, 54.1 MB/s 2024-04-26T23:53:05.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321685 s, 32.6 MB/s 2024-04-26T23:53:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025168 s, 41.7 MB/s 2024-04-26T23:53:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227126 s, 46.2 MB/s 2024-04-26T23:53:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239314 s, 43.8 MB/s 2024-04-26T23:53:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279232 s, 37.6 MB/s 2024-04-26T23:53:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300837 s, 34.9 MB/s 2024-04-26T23:53:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258715 s, 40.5 MB/s 2024-04-26T23:53:06.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.150 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017571 s, 59.7 MB/s 2024-04-26T23:53:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.183 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259895 s, 40.3 MB/s 2024-04-26T23:53:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018234 s, 57.5 MB/s 2024-04-26T23:53:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.10 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:06.219 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210823 s, 49.7 MB/s 2024-04-26T23:53:06.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017147 s, 61.2 MB/s 2024-04-26T23:53:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021707 s, 48.3 MB/s 2024-04-26T23:53:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248042 s, 42.3 MB/s 2024-04-26T23:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235569 s, 44.5 MB/s 2024-04-26T23:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234616 s, 44.7 MB/s 2024-04-26T23:53:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.529 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.529 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223737 s, 46.9 MB/s 2024-04-26T23:53:06.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.561 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.561 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262132 s, 40.0 MB/s 2024-04-26T23:53:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229484 s, 45.7 MB/s 2024-04-26T23:53:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307401 s, 34.1 MB/s 2024-04-26T23:53:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248885 s, 42.1 MB/s 2024-04-26T23:53:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202254 s, 51.8 MB/s 2024-04-26T23:53:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295464 s, 35.5 MB/s 2024-04-26T23:53:06.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023041 s, 45.5 MB/s 2024-04-26T23:53:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253197 s, 41.4 MB/s 2024-04-26T23:53:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025487 s, 41.1 MB/s 2024-04-26T23:53:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243832 s, 43.0 MB/s 2024-04-26T23:53:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292236 s, 35.9 MB/s 2024-04-26T23:53:06.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301353 s, 34.8 MB/s 2024-04-26T23:53:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313001 s, 33.5 MB/s 2024-04-26T23:53:06.988 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:06.988 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:06.988 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255399 s, 41.1 MB/s 2024-04-26T23:53:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243242 s, 43.1 MB/s 2024-04-26T23:53:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265661 s, 39.5 MB/s 2024-04-26T23:53:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211151 s, 49.7 MB/s 2024-04-26T23:53:07.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.107 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.107 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216148 s, 48.5 MB/s 2024-04-26T23:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259702 s, 40.4 MB/s 2024-04-26T23:53:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288209 s, 36.4 MB/s 2024-04-26T23:53:07.204 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.205 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.205 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242863 s, 43.2 MB/s 2024-04-26T23:53:07.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.234 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.234 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239278 s, 43.8 MB/s 2024-04-26T23:53:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219257 s, 47.8 MB/s 2024-04-26T23:53:07.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.298 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.298 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281193 s, 37.3 MB/s 2024-04-26T23:53:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024354 s, 43.1 MB/s 2024-04-26T23:53:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.11 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:07.340 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269005 s, 39.0 MB/s 2024-04-26T23:53:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248301 s, 42.2 MB/s 2024-04-26T23:53:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323172 s, 32.4 MB/s 2024-04-26T23:53:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026361 s, 39.8 MB/s 2024-04-26T23:53:07.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.634 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.634 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305492 s, 34.3 MB/s 2024-04-26T23:53:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026925 s, 38.9 MB/s 2024-04-26T23:53:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252165 s, 41.6 MB/s 2024-04-26T23:53:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248728 s, 42.2 MB/s 2024-04-26T23:53:07.764 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.764 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.764 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022701 s, 46.2 MB/s 2024-04-26T23:53:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293665 s, 35.7 MB/s 2024-04-26T23:53:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239227 s, 43.8 MB/s 2024-04-26T23:53:07.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.866 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291077 s, 36.0 MB/s 2024-04-26T23:53:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246147 s, 42.6 MB/s 2024-04-26T23:53:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300086 s, 34.9 MB/s 2024-04-26T23:53:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216955 s, 48.3 MB/s 2024-04-26T23:53:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308659 s, 34.0 MB/s 2024-04-26T23:53:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024365 s, 43.0 MB/s 2024-04-26T23:53:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257166 s, 40.8 MB/s 2024-04-26T23:53:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.094 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199156 s, 52.7 MB/s 2024-04-26T23:53:08.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.122 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.123 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215231 s, 48.7 MB/s 2024-04-26T23:53:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02853 s, 36.8 MB/s 2024-04-26T23:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291474 s, 36.0 MB/s 2024-04-26T23:53:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229553 s, 45.7 MB/s 2024-04-26T23:53:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260443 s, 40.3 MB/s 2024-04-26T23:53:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244233 s, 42.9 MB/s 2024-04-26T23:53:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225471 s, 46.5 MB/s 2024-04-26T23:53:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217017 s, 48.3 MB/s 2024-04-26T23:53:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218293 s, 48.0 MB/s 2024-04-26T23:53:08.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.403 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.403 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235272 s, 44.6 MB/s 2024-04-26T23:53:08.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.432 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.432 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222209 s, 47.2 MB/s 2024-04-26T23:53:08.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.463 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243569 s, 43.1 MB/s 2024-04-26T23:53:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268192 s, 39.1 MB/s 2024-04-26T23:53:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.12 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:08.509 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024288 s, 43.2 MB/s 2024-04-26T23:53:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245712 s, 42.7 MB/s 2024-04-26T23:53:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258711 s, 40.5 MB/s 2024-04-26T23:53:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024585 s, 42.7 MB/s 2024-04-26T23:53:08.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238595 s, 43.9 MB/s 2024-04-26T23:53:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217773 s, 48.1 MB/s 2024-04-26T23:53:08.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.831 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243035 s, 43.1 MB/s 2024-04-26T23:53:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256916 s, 40.8 MB/s 2024-04-26T23:53:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237774 s, 44.1 MB/s 2024-04-26T23:53:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023472 s, 44.7 MB/s 2024-04-26T23:53:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245204 s, 42.8 MB/s 2024-04-26T23:53:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257814 s, 40.7 MB/s 2024-04-26T23:53:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.025 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.025 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324452 s, 32.3 MB/s 2024-04-26T23:53:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285425 s, 36.7 MB/s 2024-04-26T23:53:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247968 s, 42.3 MB/s 2024-04-26T23:53:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229077 s, 45.8 MB/s 2024-04-26T23:53:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234536 s, 44.7 MB/s 2024-04-26T23:53:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302237 s, 34.7 MB/s 2024-04-26T23:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277544 s, 37.8 MB/s 2024-04-26T23:53:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.255 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306307 s, 34.2 MB/s 2024-04-26T23:53:09.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.282 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.282 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203566 s, 51.5 MB/s 2024-04-26T23:53:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.316 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270105 s, 38.8 MB/s 2024-04-26T23:53:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211087 s, 49.7 MB/s 2024-04-26T23:53:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224066 s, 46.8 MB/s 2024-04-26T23:53:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202646 s, 51.7 MB/s 2024-04-26T23:53:09.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.430 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023961 s, 43.8 MB/s 2024-04-26T23:53:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206988 s, 50.7 MB/s 2024-04-26T23:53:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254719 s, 41.2 MB/s 2024-04-26T23:53:09.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.519 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231592 s, 45.3 MB/s 2024-04-26T23:53:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219427 s, 47.8 MB/s 2024-04-26T23:53:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219498 s, 47.8 MB/s 2024-04-26T23:53:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359314 s, 29.2 MB/s 2024-04-26T23:53:09.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:09.631 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:09.631 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:09.632 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.13 2024-04-26T23:53:09.632 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:09.632 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:09.632 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:09.632 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306933 s, 34.2 MB/s 2024-04-26T23:53:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284161 s, 36.9 MB/s 2024-04-26T23:53:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240828 s, 43.5 MB/s 2024-04-26T23:53:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265014 s, 39.6 MB/s 2024-04-26T23:53:09.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.908 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.908 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018855 s, 55.6 MB/s 2024-04-26T23:53:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282241 s, 37.2 MB/s 2024-04-26T23:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383566 s, 27.3 MB/s 2024-04-26T23:53:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235683 s, 44.5 MB/s 2024-04-26T23:53:10.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273559 s, 38.3 MB/s 2024-04-26T23:53:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276601 s, 37.9 MB/s 2024-04-26T23:53:10.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.111 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022845 s, 45.9 MB/s 2024-04-26T23:53:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225353 s, 46.5 MB/s 2024-04-26T23:53:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030021 s, 34.9 MB/s 2024-04-26T23:53:10.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.210 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236944 s, 44.3 MB/s 2024-04-26T23:53:10.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.240 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.240 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236259 s, 44.4 MB/s 2024-04-26T23:53:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192926 s, 54.4 MB/s 2024-04-26T23:53:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243953 s, 43.0 MB/s 2024-04-26T23:53:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028821 s, 36.4 MB/s 2024-04-26T23:53:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272311 s, 38.5 MB/s 2024-04-26T23:53:10.396 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.396 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.396 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245348 s, 42.7 MB/s 2024-04-26T23:53:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026499 s, 39.6 MB/s 2024-04-26T23:53:10.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.461 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.461 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255743 s, 41.0 MB/s 2024-04-26T23:53:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288095 s, 36.4 MB/s 2024-04-26T23:53:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277589 s, 37.8 MB/s 2024-04-26T23:53:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208069 s, 50.4 MB/s 2024-04-26T23:53:10.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.589 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.589 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243937 s, 43.0 MB/s 2024-04-26T23:53:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249912 s, 42.0 MB/s 2024-04-26T23:53:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193719 s, 54.1 MB/s 2024-04-26T23:53:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023222 s, 45.2 MB/s 2024-04-26T23:53:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322683 s, 32.5 MB/s 2024-04-26T23:53:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255462 s, 41.0 MB/s 2024-04-26T23:53:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315422 s, 33.2 MB/s 2024-04-26T23:53:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.14 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:10.805 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:10.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267636 s, 39.2 MB/s 2024-04-26T23:53:10.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:10.987 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:10.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234766 s, 44.7 MB/s 2024-04-26T23:53:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247972 s, 42.3 MB/s 2024-04-26T23:53:11.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.050 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.050 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262216 s, 40.0 MB/s 2024-04-26T23:53:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263819 s, 39.7 MB/s 2024-04-26T23:53:11.118 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262757 s, 39.9 MB/s 2024-04-26T23:53:11.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.150 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240487 s, 43.6 MB/s 2024-04-26T23:53:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225933 s, 46.4 MB/s 2024-04-26T23:53:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226283 s, 46.3 MB/s 2024-04-26T23:53:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02325 s, 45.1 MB/s 2024-04-26T23:53:11.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.278 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.278 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289616 s, 36.2 MB/s 2024-04-26T23:53:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246241 s, 42.6 MB/s 2024-04-26T23:53:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302997 s, 34.6 MB/s 2024-04-26T23:53:11.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.383 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.383 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273864 s, 38.3 MB/s 2024-04-26T23:53:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221297 s, 47.4 MB/s 2024-04-26T23:53:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236505 s, 44.3 MB/s 2024-04-26T23:53:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288343 s, 36.4 MB/s 2024-04-26T23:53:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309112 s, 33.9 MB/s 2024-04-26T23:53:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225116 s, 46.6 MB/s 2024-04-26T23:53:11.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.585 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283324 s, 37.0 MB/s 2024-04-26T23:53:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251181 s, 41.7 MB/s 2024-04-26T23:53:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306954 s, 34.2 MB/s 2024-04-26T23:53:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224401 s, 46.7 MB/s 2024-04-26T23:53:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295811 s, 35.4 MB/s 2024-04-26T23:53:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230761 s, 45.4 MB/s 2024-04-26T23:53:11.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.787 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240992 s, 43.5 MB/s 2024-04-26T23:53:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227996 s, 46.0 MB/s 2024-04-26T23:53:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303365 s, 34.6 MB/s 2024-04-26T23:53:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300225 s, 34.9 MB/s 2024-04-26T23:53:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260467 s, 40.3 MB/s 2024-04-26T23:53:11.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.964 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309946 s, 33.8 MB/s 2024-04-26T23:53:11.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:11.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:11.992 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214773 s, 48.8 MB/s 2024-04-26T23:53:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.15 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:12.004 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:12.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.149 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229793 s, 45.6 MB/s 2024-04-26T23:53:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021088 s, 49.7 MB/s 2024-04-26T23:53:12.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.210 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.210 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270521 s, 38.8 MB/s 2024-04-26T23:53:12.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.245 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.245 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266493 s, 39.3 MB/s 2024-04-26T23:53:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284241 s, 36.9 MB/s 2024-04-26T23:53:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253303 s, 41.4 MB/s 2024-04-26T23:53:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023618 s, 44.4 MB/s 2024-04-26T23:53:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280324 s, 37.4 MB/s 2024-04-26T23:53:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023179 s, 45.2 MB/s 2024-04-26T23:53:12.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.439 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.439 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250223 s, 41.9 MB/s 2024-04-26T23:53:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295678 s, 35.5 MB/s 2024-04-26T23:53:12.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231953 s, 45.2 MB/s 2024-04-26T23:53:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216425 s, 48.4 MB/s 2024-04-26T23:53:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199699 s, 52.5 MB/s 2024-04-26T23:53:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.596 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278613 s, 37.6 MB/s 2024-04-26T23:53:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275576 s, 38.1 MB/s 2024-04-26T23:53:12.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.669 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.669 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317294 s, 33.0 MB/s 2024-04-26T23:53:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.712 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.712 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345772 s, 30.3 MB/s 2024-04-26T23:53:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247006 s, 42.5 MB/s 2024-04-26T23:53:12.770 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.771 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.771 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222718 s, 47.1 MB/s 2024-04-26T23:53:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315107 s, 33.3 MB/s 2024-04-26T23:53:12.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.843 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261016 s, 40.2 MB/s 2024-04-26T23:53:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289789 s, 36.2 MB/s 2024-04-26T23:53:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023404 s, 44.8 MB/s 2024-04-26T23:53:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266901 s, 39.3 MB/s 2024-04-26T23:53:12.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:12.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257687 s, 40.7 MB/s 2024-04-26T23:53:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274441 s, 38.2 MB/s 2024-04-26T23:53:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224833 s, 46.6 MB/s 2024-04-26T23:53:13.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.055 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180511 s, 58.1 MB/s 2024-04-26T23:53:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260288 s, 40.3 MB/s 2024-04-26T23:53:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280914 s, 37.3 MB/s 2024-04-26T23:53:13.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.150 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.150 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229727 s, 45.6 MB/s 2024-04-26T23:53:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.16 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:13.163 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268308 s, 39.1 MB/s 2024-04-26T23:53:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027701 s, 37.9 MB/s 2024-04-26T23:53:13.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280391 s, 37.4 MB/s 2024-04-26T23:53:13.457 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.458 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.458 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254151 s, 41.3 MB/s 2024-04-26T23:53:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214634 s, 48.9 MB/s 2024-04-26T23:53:13.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188817 s, 55.5 MB/s 2024-04-26T23:53:13.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.536 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.536 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182249 s, 57.5 MB/s 2024-04-26T23:53:13.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.573 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.573 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287602 s, 36.5 MB/s 2024-04-26T23:53:13.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.607 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.607 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251754 s, 41.7 MB/s 2024-04-26T23:53:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249733 s, 42.0 MB/s 2024-04-26T23:53:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238074 s, 44.0 MB/s 2024-04-26T23:53:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247848 s, 42.3 MB/s 2024-04-26T23:53:13.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.736 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294004 s, 35.7 MB/s 2024-04-26T23:53:13.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307059 s, 34.1 MB/s 2024-04-26T23:53:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263812 s, 39.7 MB/s 2024-04-26T23:53:13.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.842 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278737 s, 37.6 MB/s 2024-04-26T23:53:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228765 s, 45.8 MB/s 2024-04-26T23:53:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276921 s, 37.9 MB/s 2024-04-26T23:53:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232665 s, 45.1 MB/s 2024-04-26T23:53:13.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:13.970 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:13.970 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292106 s, 35.9 MB/s 2024-04-26T23:53:14.001 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.001 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.001 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244036 s, 43.0 MB/s 2024-04-26T23:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279934 s, 37.5 MB/s 2024-04-26T23:53:14.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.066 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249993 s, 41.9 MB/s 2024-04-26T23:53:14.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225068 s, 46.6 MB/s 2024-04-26T23:53:14.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247523 s, 42.4 MB/s 2024-04-26T23:53:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245252 s, 42.8 MB/s 2024-04-26T23:53:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235185 s, 44.6 MB/s 2024-04-26T23:53:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236537 s, 44.3 MB/s 2024-04-26T23:53:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267095 s, 39.3 MB/s 2024-04-26T23:53:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243733 s, 43.0 MB/s 2024-04-26T23:53:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.314 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245679 s, 42.7 MB/s 2024-04-26T23:53:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278789 s, 37.6 MB/s 2024-04-26T23:53:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.17 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:14.361 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:14.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254935 s, 41.1 MB/s 2024-04-26T23:53:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308314 s, 34.0 MB/s 2024-04-26T23:53:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212557 s, 49.3 MB/s 2024-04-26T23:53:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290727 s, 36.1 MB/s 2024-04-26T23:53:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285894 s, 36.7 MB/s 2024-04-26T23:53:14.680 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247416 s, 42.4 MB/s 2024-04-26T23:53:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285931 s, 36.7 MB/s 2024-04-26T23:53:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229696 s, 45.7 MB/s 2024-04-26T23:53:14.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.776 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.776 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020449 s, 51.3 MB/s 2024-04-26T23:53:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252299 s, 41.6 MB/s 2024-04-26T23:53:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241728 s, 43.4 MB/s 2024-04-26T23:53:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250792 s, 41.8 MB/s 2024-04-26T23:53:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185291 s, 56.6 MB/s 2024-04-26T23:53:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222722 s, 47.1 MB/s 2024-04-26T23:53:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.963 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.964 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02688 s, 39.0 MB/s 2024-04-26T23:53:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251607 s, 41.7 MB/s 2024-04-26T23:53:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213594 s, 49.1 MB/s 2024-04-26T23:53:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253985 s, 41.3 MB/s 2024-04-26T23:53:15.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.084 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.084 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241682 s, 43.4 MB/s 2024-04-26T23:53:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256204 s, 40.9 MB/s 2024-04-26T23:53:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235156 s, 44.6 MB/s 2024-04-26T23:53:15.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235981 s, 44.4 MB/s 2024-04-26T23:53:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239818 s, 43.7 MB/s 2024-04-26T23:53:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259803 s, 40.4 MB/s 2024-04-26T23:53:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224022 s, 46.8 MB/s 2024-04-26T23:53:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312973 s, 33.5 MB/s 2024-04-26T23:53:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267377 s, 39.2 MB/s 2024-04-26T23:53:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291459 s, 36.0 MB/s 2024-04-26T23:53:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261702 s, 40.1 MB/s 2024-04-26T23:53:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267796 s, 39.2 MB/s 2024-04-26T23:53:15.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.473 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311371 s, 33.7 MB/s 2024-04-26T23:53:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283556 s, 37.0 MB/s 2024-04-26T23:53:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.18 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:15.519 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024482 s, 42.8 MB/s 2024-04-26T23:53:15.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.699 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305191 s, 34.4 MB/s 2024-04-26T23:53:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028217 s, 37.2 MB/s 2024-04-26T23:53:15.766 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290195 s, 36.1 MB/s 2024-04-26T23:53:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223161 s, 47.0 MB/s 2024-04-26T23:53:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.830 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278776 s, 37.6 MB/s 2024-04-26T23:53:15.861 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025918 s, 40.5 MB/s 2024-04-26T23:53:15.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.889 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212404 s, 49.4 MB/s 2024-04-26T23:53:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252855 s, 41.5 MB/s 2024-04-26T23:53:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247725 s, 42.3 MB/s 2024-04-26T23:53:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021982 s, 47.7 MB/s 2024-04-26T23:53:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248802 s, 42.1 MB/s 2024-04-26T23:53:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273801 s, 38.3 MB/s 2024-04-26T23:53:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250804 s, 41.8 MB/s 2024-04-26T23:53:16.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.103 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247144 s, 42.4 MB/s 2024-04-26T23:53:16.133 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.133 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.133 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245061 s, 42.8 MB/s 2024-04-26T23:53:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256724 s, 40.8 MB/s 2024-04-26T23:53:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.194 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245118 s, 42.8 MB/s 2024-04-26T23:53:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304129 s, 34.5 MB/s 2024-04-26T23:53:16.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.263 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.263 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228101 s, 46.0 MB/s 2024-04-26T23:53:16.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258129 s, 40.6 MB/s 2024-04-26T23:53:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215715 s, 48.6 MB/s 2024-04-26T23:53:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268939 s, 39.0 MB/s 2024-04-26T23:53:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021234 s, 49.4 MB/s 2024-04-26T23:53:16.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.421 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257423 s, 40.7 MB/s 2024-04-26T23:53:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243994 s, 43.0 MB/s 2024-04-26T23:53:16.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.482 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238872 s, 43.9 MB/s 2024-04-26T23:53:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276616 s, 37.9 MB/s 2024-04-26T23:53:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307475 s, 34.1 MB/s 2024-04-26T23:53:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249563 s, 42.0 MB/s 2024-04-26T23:53:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226189 s, 46.4 MB/s 2024-04-26T23:53:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.649 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.649 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296312 s, 35.4 MB/s 2024-04-26T23:53:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.19 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:16.662 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:16.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.808 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.808 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254436 s, 41.2 MB/s 2024-04-26T23:53:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335297 s, 31.3 MB/s 2024-04-26T23:53:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287738 s, 36.4 MB/s 2024-04-26T23:53:16.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237274 s, 44.2 MB/s 2024-04-26T23:53:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259867 s, 40.4 MB/s 2024-04-26T23:53:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255989 s, 41.0 MB/s 2024-04-26T23:53:17.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245871 s, 42.6 MB/s 2024-04-26T23:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288257 s, 36.4 MB/s 2024-04-26T23:53:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300613 s, 34.9 MB/s 2024-04-26T23:53:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241286 s, 43.5 MB/s 2024-04-26T23:53:17.147 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304771 s, 34.4 MB/s 2024-04-26T23:53:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265977 s, 39.4 MB/s 2024-04-26T23:53:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222741 s, 47.1 MB/s 2024-04-26T23:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285007 s, 36.8 MB/s 2024-04-26T23:53:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306624 s, 34.2 MB/s 2024-04-26T23:53:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238417 s, 44.0 MB/s 2024-04-26T23:53:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295366 s, 35.5 MB/s 2024-04-26T23:53:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261212 s, 40.1 MB/s 2024-04-26T23:53:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284252 s, 36.9 MB/s 2024-04-26T23:53:17.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.441 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.441 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238028 s, 44.1 MB/s 2024-04-26T23:53:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259387 s, 40.4 MB/s 2024-04-26T23:53:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241177 s, 43.5 MB/s 2024-04-26T23:53:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231696 s, 45.3 MB/s 2024-04-26T23:53:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260971 s, 40.2 MB/s 2024-04-26T23:53:17.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299919 s, 35.0 MB/s 2024-04-26T23:53:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301805 s, 34.7 MB/s 2024-04-26T23:53:17.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.674 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.674 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283503 s, 37.0 MB/s 2024-04-26T23:53:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230624 s, 45.5 MB/s 2024-04-26T23:53:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.735 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.735 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249226 s, 42.1 MB/s 2024-04-26T23:53:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027255 s, 38.5 MB/s 2024-04-26T23:53:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022096 s, 47.5 MB/s 2024-04-26T23:53:17.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.832 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.832 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302039 s, 34.7 MB/s 2024-04-26T23:53:17.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.20 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:17.844 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:17.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:17.986 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:17.987 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201935 s, 51.9 MB/s 2024-04-26T23:53:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241854 s, 43.4 MB/s 2024-04-26T23:53:18.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.045 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227662 s, 46.1 MB/s 2024-04-26T23:53:18.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.074 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230896 s, 45.4 MB/s 2024-04-26T23:53:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238206 s, 44.0 MB/s 2024-04-26T23:53:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238792 s, 43.9 MB/s 2024-04-26T23:53:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226216 s, 46.4 MB/s 2024-04-26T23:53:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211696 s, 49.5 MB/s 2024-04-26T23:53:18.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.225 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.225 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305957 s, 34.3 MB/s 2024-04-26T23:53:18.255 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.256 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.256 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242673 s, 43.2 MB/s 2024-04-26T23:53:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217193 s, 48.3 MB/s 2024-04-26T23:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244241 s, 42.9 MB/s 2024-04-26T23:53:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.343 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233281 s, 44.9 MB/s 2024-04-26T23:53:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231033 s, 45.4 MB/s 2024-04-26T23:53:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274133 s, 38.3 MB/s 2024-04-26T23:53:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235621 s, 44.5 MB/s 2024-04-26T23:53:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253873 s, 41.3 MB/s 2024-04-26T23:53:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024459 s, 42.9 MB/s 2024-04-26T23:53:18.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.526 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236669 s, 44.3 MB/s 2024-04-26T23:53:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294147 s, 35.6 MB/s 2024-04-26T23:53:18.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232475 s, 45.1 MB/s 2024-04-26T23:53:18.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.626 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.626 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296789 s, 35.3 MB/s 2024-04-26T23:53:18.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.656 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.657 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253499 s, 41.4 MB/s 2024-04-26T23:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237514 s, 44.1 MB/s 2024-04-26T23:53:18.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234932 s, 44.6 MB/s 2024-04-26T23:53:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308206 s, 34.0 MB/s 2024-04-26T23:53:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257174 s, 40.8 MB/s 2024-04-26T23:53:18.810 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.811 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.811 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021364 s, 49.1 MB/s 2024-04-26T23:53:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244941 s, 42.8 MB/s 2024-04-26T23:53:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287788 s, 36.4 MB/s 2024-04-26T23:53:18.904 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.905 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.905 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223217 s, 47.0 MB/s 2024-04-26T23:53:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280459 s, 37.4 MB/s 2024-04-26T23:53:18.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:18.949 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.21 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:18.950 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:19.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:19.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.105 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.105 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293693 s, 35.7 MB/s 2024-04-26T23:53:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274504 s, 38.2 MB/s 2024-04-26T23:53:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228277 s, 45.9 MB/s 2024-04-26T23:53:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289836 s, 36.2 MB/s 2024-04-26T23:53:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333423 s, 31.4 MB/s 2024-04-26T23:53:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026985 s, 38.9 MB/s 2024-04-26T23:53:19.306 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.306 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256045 s, 41.0 MB/s 2024-04-26T23:53:19.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215338 s, 48.7 MB/s 2024-04-26T23:53:19.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.369 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300132 s, 34.9 MB/s 2024-04-26T23:53:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227642 s, 46.1 MB/s 2024-04-26T23:53:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286303 s, 36.6 MB/s 2024-04-26T23:53:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.465 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.465 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259906 s, 40.3 MB/s 2024-04-26T23:53:19.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.492 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.492 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220929 s, 47.5 MB/s 2024-04-26T23:53:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223007 s, 47.0 MB/s 2024-04-26T23:53:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219932 s, 47.7 MB/s 2024-04-26T23:53:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210704 s, 49.8 MB/s 2024-04-26T23:53:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325569 s, 32.2 MB/s 2024-04-26T23:53:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254505 s, 41.2 MB/s 2024-04-26T23:53:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023035 s, 45.5 MB/s 2024-04-26T23:53:19.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.719 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.719 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341462 s, 30.7 MB/s 2024-04-26T23:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216565 s, 48.4 MB/s 2024-04-26T23:53:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207207 s, 50.6 MB/s 2024-04-26T23:53:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219323 s, 47.8 MB/s 2024-04-26T23:53:19.839 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.839 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.839 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279265 s, 37.5 MB/s 2024-04-26T23:53:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254834 s, 41.1 MB/s 2024-04-26T23:53:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278147 s, 37.7 MB/s 2024-04-26T23:53:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238164 s, 44.0 MB/s 2024-04-26T23:53:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287138 s, 36.5 MB/s 2024-04-26T23:53:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213869 s, 49.0 MB/s 2024-04-26T23:53:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205538 s, 51.0 MB/s 2024-04-26T23:53:20.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.059 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.059 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328167 s, 32.0 MB/s 2024-04-26T23:53:20.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.085 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212462 s, 49.4 MB/s 2024-04-26T23:53:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:20.097 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:20.097 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:20.098 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.22 2024-04-26T23:53:20.098 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:20.098 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:20.098 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:20.098 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190519 s, 55.0 MB/s 2024-04-26T23:53:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254283 s, 41.2 MB/s 2024-04-26T23:53:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239296 s, 43.8 MB/s 2024-04-26T23:53:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322469 s, 32.5 MB/s 2024-04-26T23:53:20.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.364 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225075 s, 46.6 MB/s 2024-04-26T23:53:20.391 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.391 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.391 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212339 s, 49.4 MB/s 2024-04-26T23:53:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200541 s, 52.3 MB/s 2024-04-26T23:53:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296078 s, 35.4 MB/s 2024-04-26T23:53:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332363 s, 31.5 MB/s 2024-04-26T23:53:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293057 s, 35.8 MB/s 2024-04-26T23:53:20.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.559 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.559 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273842 s, 38.3 MB/s 2024-04-26T23:53:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255503 s, 41.0 MB/s 2024-04-26T23:53:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242701 s, 43.2 MB/s 2024-04-26T23:53:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238151 s, 44.0 MB/s 2024-04-26T23:53:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249952 s, 42.0 MB/s 2024-04-26T23:53:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255711 s, 41.0 MB/s 2024-04-26T23:53:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264631 s, 39.6 MB/s 2024-04-26T23:53:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216201 s, 48.5 MB/s 2024-04-26T23:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229726 s, 45.6 MB/s 2024-04-26T23:53:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268852 s, 39.0 MB/s 2024-04-26T23:53:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.862 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208003 s, 50.4 MB/s 2024-04-26T23:53:20.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240094 s, 43.7 MB/s 2024-04-26T23:53:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258335 s, 40.6 MB/s 2024-04-26T23:53:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230064 s, 45.6 MB/s 2024-04-26T23:53:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225044 s, 46.6 MB/s 2024-04-26T23:53:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286531 s, 36.6 MB/s 2024-04-26T23:53:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217041 s, 48.3 MB/s 2024-04-26T23:53:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254648 s, 41.2 MB/s 2024-04-26T23:53:21.110 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.111 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286133 s, 36.6 MB/s 2024-04-26T23:53:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238048 s, 44.0 MB/s 2024-04-26T23:53:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331157 s, 31.7 MB/s 2024-04-26T23:53:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023128 s, 45.3 MB/s 2024-04-26T23:53:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.23 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:21.220 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:21.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283676 s, 37.0 MB/s 2024-04-26T23:53:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372376 s, 28.2 MB/s 2024-04-26T23:53:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241634 s, 43.4 MB/s 2024-04-26T23:53:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259225 s, 40.5 MB/s 2024-04-26T23:53:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237464 s, 44.2 MB/s 2024-04-26T23:53:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358939 s, 29.2 MB/s 2024-04-26T23:53:21.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.576 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.576 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231902 s, 45.2 MB/s 2024-04-26T23:53:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263306 s, 39.8 MB/s 2024-04-26T23:53:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255098 s, 41.1 MB/s 2024-04-26T23:53:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025886 s, 40.5 MB/s 2024-04-26T23:53:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377695 s, 27.8 MB/s 2024-04-26T23:53:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220721 s, 47.5 MB/s 2024-04-26T23:53:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.773 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024966 s, 42.0 MB/s 2024-04-26T23:53:21.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.802 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.802 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224818 s, 46.6 MB/s 2024-04-26T23:53:21.832 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023475 s, 44.7 MB/s 2024-04-26T23:53:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024795 s, 42.3 MB/s 2024-04-26T23:53:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022855 s, 45.9 MB/s 2024-04-26T23:53:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242774 s, 43.2 MB/s 2024-04-26T23:53:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279324 s, 37.5 MB/s 2024-04-26T23:53:21.998 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:21.998 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:21.998 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318979 s, 32.9 MB/s 2024-04-26T23:53:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234282 s, 44.8 MB/s 2024-04-26T23:53:22.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.058 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.058 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237576 s, 44.1 MB/s 2024-04-26T23:53:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211797 s, 49.5 MB/s 2024-04-26T23:53:22.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.112 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.112 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022506 s, 46.6 MB/s 2024-04-26T23:53:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227281 s, 46.1 MB/s 2024-04-26T23:53:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368879 s, 28.4 MB/s 2024-04-26T23:53:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225687 s, 46.5 MB/s 2024-04-26T23:53:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304466 s, 34.4 MB/s 2024-04-26T23:53:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284936 s, 36.8 MB/s 2024-04-26T23:53:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247928 s, 42.3 MB/s 2024-04-26T23:53:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233941 s, 44.8 MB/s 2024-04-26T23:53:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02744 s, 38.2 MB/s 2024-04-26T23:53:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.24 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:22.396 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326203 s, 32.1 MB/s 2024-04-26T23:53:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026271 s, 39.9 MB/s 2024-04-26T23:53:22.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.618 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.618 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233759 s, 44.9 MB/s 2024-04-26T23:53:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302307 s, 34.7 MB/s 2024-04-26T23:53:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238554 s, 44.0 MB/s 2024-04-26T23:53:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.716 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.716 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248774 s, 42.1 MB/s 2024-04-26T23:53:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296659 s, 35.3 MB/s 2024-04-26T23:53:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310512 s, 33.8 MB/s 2024-04-26T23:53:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189983 s, 55.2 MB/s 2024-04-26T23:53:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322556 s, 32.5 MB/s 2024-04-26T23:53:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.896 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348929 s, 30.1 MB/s 2024-04-26T23:53:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187687 s, 55.9 MB/s 2024-04-26T23:53:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247319 s, 42.4 MB/s 2024-04-26T23:53:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274965 s, 38.1 MB/s 2024-04-26T23:53:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.011 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203402 s, 51.6 MB/s 2024-04-26T23:53:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017903 s, 58.6 MB/s 2024-04-26T23:53:23.061 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211447 s, 49.6 MB/s 2024-04-26T23:53:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266241 s, 39.4 MB/s 2024-04-26T23:53:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205256 s, 51.1 MB/s 2024-04-26T23:53:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228162 s, 46.0 MB/s 2024-04-26T23:53:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276959 s, 37.9 MB/s 2024-04-26T23:53:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270382 s, 38.8 MB/s 2024-04-26T23:53:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286442 s, 36.6 MB/s 2024-04-26T23:53:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214727 s, 48.8 MB/s 2024-04-26T23:53:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241408 s, 43.4 MB/s 2024-04-26T23:53:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255845 s, 41.0 MB/s 2024-04-26T23:53:23.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.371 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.372 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235267 s, 44.6 MB/s 2024-04-26T23:53:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220294 s, 47.6 MB/s 2024-04-26T23:53:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219062 s, 47.9 MB/s 2024-04-26T23:53:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283466 s, 37.0 MB/s 2024-04-26T23:53:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024055 s, 43.6 MB/s 2024-04-26T23:53:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214321 s, 48.9 MB/s 2024-04-26T23:53:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.25 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:23.533 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:23.534 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232946 s, 45.0 MB/s 2024-04-26T23:53:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252982 s, 41.4 MB/s 2024-04-26T23:53:23.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.739 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.739 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260188 s, 40.3 MB/s 2024-04-26T23:53:23.777 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.778 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.778 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324954 s, 32.3 MB/s 2024-04-26T23:53:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024597 s, 42.6 MB/s 2024-04-26T23:53:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275796 s, 38.0 MB/s 2024-04-26T23:53:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205146 s, 51.1 MB/s 2024-04-26T23:53:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027789 s, 37.7 MB/s 2024-04-26T23:53:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290342 s, 36.1 MB/s 2024-04-26T23:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253443 s, 41.4 MB/s 2024-04-26T23:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270344 s, 38.8 MB/s 2024-04-26T23:53:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261025 s, 40.2 MB/s 2024-04-26T23:53:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287269 s, 36.5 MB/s 2024-04-26T23:53:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227072 s, 46.2 MB/s 2024-04-26T23:53:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023136 s, 45.3 MB/s 2024-04-26T23:53:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277948 s, 37.7 MB/s 2024-04-26T23:53:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258512 s, 40.6 MB/s 2024-04-26T23:53:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022265 s, 47.1 MB/s 2024-04-26T23:53:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324592 s, 32.3 MB/s 2024-04-26T23:53:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304342 s, 34.5 MB/s 2024-04-26T23:53:24.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269459 s, 38.9 MB/s 2024-04-26T23:53:24.354 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223461 s, 46.9 MB/s 2024-04-26T23:53:24.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.386 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.386 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256284 s, 40.9 MB/s 2024-04-26T23:53:24.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.416 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.416 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252515 s, 41.5 MB/s 2024-04-26T23:53:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222741 s, 47.1 MB/s 2024-04-26T23:53:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.477 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.477 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271545 s, 38.6 MB/s 2024-04-26T23:53:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230083 s, 45.6 MB/s 2024-04-26T23:53:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271383 s, 38.6 MB/s 2024-04-26T23:53:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244874 s, 42.8 MB/s 2024-04-26T23:53:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246414 s, 42.6 MB/s 2024-04-26T23:53:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231298 s, 45.3 MB/s 2024-04-26T23:53:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241206 s, 43.5 MB/s 2024-04-26T23:53:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.26 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:24.675 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250156 s, 41.9 MB/s 2024-04-26T23:53:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255533 s, 41.0 MB/s 2024-04-26T23:53:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288952 s, 36.3 MB/s 2024-04-26T23:53:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217928 s, 48.1 MB/s 2024-04-26T23:53:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231141 s, 45.4 MB/s 2024-04-26T23:53:24.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:24.982 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:24.982 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296325 s, 35.4 MB/s 2024-04-26T23:53:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269669 s, 38.9 MB/s 2024-04-26T23:53:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241199 s, 43.5 MB/s 2024-04-26T23:53:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240988 s, 43.5 MB/s 2024-04-26T23:53:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264421 s, 39.7 MB/s 2024-04-26T23:53:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257764 s, 40.7 MB/s 2024-04-26T23:53:25.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.166 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.166 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230287 s, 45.5 MB/s 2024-04-26T23:53:25.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.197 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255294 s, 41.1 MB/s 2024-04-26T23:53:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206728 s, 50.7 MB/s 2024-04-26T23:53:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294999 s, 35.5 MB/s 2024-04-26T23:53:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311273 s, 33.7 MB/s 2024-04-26T23:53:25.331 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.332 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.332 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290124 s, 36.1 MB/s 2024-04-26T23:53:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282536 s, 37.1 MB/s 2024-04-26T23:53:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218548 s, 48.0 MB/s 2024-04-26T23:53:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275576 s, 38.1 MB/s 2024-04-26T23:53:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185949 s, 56.4 MB/s 2024-04-26T23:53:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272919 s, 38.4 MB/s 2024-04-26T23:53:25.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.520 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.520 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270823 s, 38.7 MB/s 2024-04-26T23:53:25.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.554 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262208 s, 40.0 MB/s 2024-04-26T23:53:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264624 s, 39.6 MB/s 2024-04-26T23:53:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280454 s, 37.4 MB/s 2024-04-26T23:53:25.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.648 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.648 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228591 s, 45.9 MB/s 2024-04-26T23:53:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236076 s, 44.4 MB/s 2024-04-26T23:53:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029524 s, 35.5 MB/s 2024-04-26T23:53:25.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.743 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.743 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023146 s, 45.3 MB/s 2024-04-26T23:53:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232818 s, 45.0 MB/s 2024-04-26T23:53:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292644 s, 35.8 MB/s 2024-04-26T23:53:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.27 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:25.819 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247333 s, 42.4 MB/s 2024-04-26T23:53:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401624 s, 26.1 MB/s 2024-04-26T23:53:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226074 s, 46.4 MB/s 2024-04-26T23:53:26.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.066 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.066 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199185 s, 52.6 MB/s 2024-04-26T23:53:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282113 s, 37.2 MB/s 2024-04-26T23:53:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228158 s, 46.0 MB/s 2024-04-26T23:53:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205062 s, 51.1 MB/s 2024-04-26T23:53:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.186 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243557 s, 43.1 MB/s 2024-04-26T23:53:26.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.213 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.213 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217354 s, 48.2 MB/s 2024-04-26T23:53:26.248 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.249 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.249 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295165 s, 35.5 MB/s 2024-04-26T23:53:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316626 s, 33.1 MB/s 2024-04-26T23:53:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231784 s, 45.2 MB/s 2024-04-26T23:53:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.348 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257294 s, 40.8 MB/s 2024-04-26T23:53:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247551 s, 42.4 MB/s 2024-04-26T23:53:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335209 s, 31.3 MB/s 2024-04-26T23:53:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292968 s, 35.8 MB/s 2024-04-26T23:53:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022041 s, 47.6 MB/s 2024-04-26T23:53:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023079 s, 45.4 MB/s 2024-04-26T23:53:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248987 s, 42.1 MB/s 2024-04-26T23:53:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231877 s, 45.2 MB/s 2024-04-26T23:53:26.600 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.601 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.601 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235729 s, 44.5 MB/s 2024-04-26T23:53:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280717 s, 37.4 MB/s 2024-04-26T23:53:26.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.670 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288218 s, 36.4 MB/s 2024-04-26T23:53:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218604 s, 48.0 MB/s 2024-04-26T23:53:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264764 s, 39.6 MB/s 2024-04-26T23:53:26.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.762 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.762 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255868 s, 41.0 MB/s 2024-04-26T23:53:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214189 s, 49.0 MB/s 2024-04-26T23:53:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236641 s, 44.3 MB/s 2024-04-26T23:53:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231644 s, 45.3 MB/s 2024-04-26T23:53:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226577 s, 46.3 MB/s 2024-04-26T23:53:26.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.914 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.914 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335528 s, 31.3 MB/s 2024-04-26T23:53:26.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:26.953 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:26.953 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329281 s, 31.8 MB/s 2024-04-26T23:53:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.28 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:26.965 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:27.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311594 s, 33.7 MB/s 2024-04-26T23:53:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246974 s, 42.5 MB/s 2024-04-26T23:53:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311408 s, 33.7 MB/s 2024-04-26T23:53:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220903 s, 47.5 MB/s 2024-04-26T23:53:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242755 s, 43.2 MB/s 2024-04-26T23:53:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027091 s, 38.7 MB/s 2024-04-26T23:53:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236972 s, 44.2 MB/s 2024-04-26T23:53:27.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.351 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.351 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412614 s, 25.4 MB/s 2024-04-26T23:53:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279554 s, 37.5 MB/s 2024-04-26T23:53:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307185 s, 34.1 MB/s 2024-04-26T23:53:27.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.454 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.454 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244852 s, 42.8 MB/s 2024-04-26T23:53:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249694 s, 42.0 MB/s 2024-04-26T23:53:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308665 s, 34.0 MB/s 2024-04-26T23:53:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.554 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239936 s, 43.7 MB/s 2024-04-26T23:53:27.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.584 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.584 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223371 s, 46.9 MB/s 2024-04-26T23:53:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247452 s, 42.4 MB/s 2024-04-26T23:53:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021701 s, 48.3 MB/s 2024-04-26T23:53:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205123 s, 51.1 MB/s 2024-04-26T23:53:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197512 s, 53.1 MB/s 2024-04-26T23:53:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229703 s, 45.6 MB/s 2024-04-26T23:53:27.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.755 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.755 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239738 s, 43.7 MB/s 2024-04-26T23:53:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236584 s, 44.3 MB/s 2024-04-26T23:53:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253814 s, 41.3 MB/s 2024-04-26T23:53:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249834 s, 42.0 MB/s 2024-04-26T23:53:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226149 s, 46.4 MB/s 2024-04-26T23:53:27.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.910 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.910 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236297 s, 44.4 MB/s 2024-04-26T23:53:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320275 s, 32.7 MB/s 2024-04-26T23:53:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218815 s, 47.9 MB/s 2024-04-26T23:53:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284633 s, 36.8 MB/s 2024-04-26T23:53:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237034 s, 44.2 MB/s 2024-04-26T23:53:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220421 s, 47.6 MB/s 2024-04-26T23:53:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221837 s, 47.3 MB/s 2024-04-26T23:53:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.29 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:28.114 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:28.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:28.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.270 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.270 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287226 s, 36.5 MB/s 2024-04-26T23:53:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321765 s, 32.6 MB/s 2024-04-26T23:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217964 s, 48.1 MB/s 2024-04-26T23:53:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211589 s, 49.6 MB/s 2024-04-26T23:53:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.395 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284833 s, 36.8 MB/s 2024-04-26T23:53:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249944 s, 42.0 MB/s 2024-04-26T23:53:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022642 s, 46.3 MB/s 2024-04-26T23:53:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252371 s, 41.5 MB/s 2024-04-26T23:53:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250975 s, 41.8 MB/s 2024-04-26T23:53:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249148 s, 42.1 MB/s 2024-04-26T23:53:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211185 s, 49.7 MB/s 2024-04-26T23:53:28.602 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221542 s, 47.3 MB/s 2024-04-26T23:53:28.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.633 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.633 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236629 s, 44.3 MB/s 2024-04-26T23:53:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209027 s, 50.2 MB/s 2024-04-26T23:53:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295715 s, 35.5 MB/s 2024-04-26T23:53:28.730 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029739 s, 35.3 MB/s 2024-04-26T23:53:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279936 s, 37.5 MB/s 2024-04-26T23:53:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229533 s, 45.7 MB/s 2024-04-26T23:53:28.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.822 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.822 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231067 s, 45.4 MB/s 2024-04-26T23:53:28.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.857 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.857 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283855 s, 36.9 MB/s 2024-04-26T23:53:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246539 s, 42.5 MB/s 2024-04-26T23:53:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247694 s, 42.3 MB/s 2024-04-26T23:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204623 s, 51.2 MB/s 2024-04-26T23:53:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262029 s, 40.0 MB/s 2024-04-26T23:53:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258894 s, 40.5 MB/s 2024-04-26T23:53:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407923 s, 25.7 MB/s 2024-04-26T23:53:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216419 s, 48.5 MB/s 2024-04-26T23:53:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304886 s, 34.4 MB/s 2024-04-26T23:53:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221698 s, 47.3 MB/s 2024-04-26T23:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258589 s, 40.5 MB/s 2024-04-26T23:53:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249906 s, 42.0 MB/s 2024-04-26T23:53:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215583 s, 48.6 MB/s 2024-04-26T23:53:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.30 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:29.249 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:29.250 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307624 s, 34.1 MB/s 2024-04-26T23:53:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239155 s, 43.8 MB/s 2024-04-26T23:53:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284098 s, 36.9 MB/s 2024-04-26T23:53:29.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.493 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.493 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257193 s, 40.8 MB/s 2024-04-26T23:53:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228196 s, 46.0 MB/s 2024-04-26T23:53:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243742 s, 43.0 MB/s 2024-04-26T23:53:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215635 s, 48.6 MB/s 2024-04-26T23:53:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242874 s, 43.2 MB/s 2024-04-26T23:53:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294814 s, 35.6 MB/s 2024-04-26T23:53:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247551 s, 42.4 MB/s 2024-04-26T23:53:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321948 s, 32.6 MB/s 2024-04-26T23:53:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278335 s, 37.7 MB/s 2024-04-26T23:53:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276484 s, 37.9 MB/s 2024-04-26T23:53:29.813 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268322 s, 39.1 MB/s 2024-04-26T23:53:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239564 s, 43.8 MB/s 2024-04-26T23:53:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268968 s, 39.0 MB/s 2024-04-26T23:53:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224001 s, 46.8 MB/s 2024-04-26T23:53:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269117 s, 39.0 MB/s 2024-04-26T23:53:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259108 s, 40.5 MB/s 2024-04-26T23:53:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215845 s, 48.6 MB/s 2024-04-26T23:53:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211225 s, 49.6 MB/s 2024-04-26T23:53:30.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.056 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.056 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278689 s, 37.6 MB/s 2024-04-26T23:53:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298048 s, 35.2 MB/s 2024-04-26T23:53:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253916 s, 41.3 MB/s 2024-04-26T23:53:30.171 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398907 s, 26.3 MB/s 2024-04-26T23:53:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025086 s, 41.8 MB/s 2024-04-26T23:53:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029378 s, 35.7 MB/s 2024-04-26T23:53:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265344 s, 39.5 MB/s 2024-04-26T23:53:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315654 s, 33.2 MB/s 2024-04-26T23:53:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299705 s, 35.0 MB/s 2024-04-26T23:53:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278827 s, 37.6 MB/s 2024-04-26T23:53:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245058 s, 42.8 MB/s 2024-04-26T23:53:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:30.423 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p scrub_control_test_path/dir.31 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> for ((i = 0; i < 32; i++)); do 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> done 2024-04-26T23:53:30.424 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:53:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:53:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269334 s, 38.9 MB/s 2024-04-26T23:53:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304076 s, 34.5 MB/s 2024-04-26T23:53:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240466 s, 43.6 MB/s 2024-04-26T23:53:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020816 s, 50.4 MB/s 2024-04-26T23:53:30.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.693 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.693 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212837 s, 49.3 MB/s 2024-04-26T23:53:30.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.720 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.720 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216257 s, 48.5 MB/s 2024-04-26T23:53:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253403 s, 41.4 MB/s 2024-04-26T23:53:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302028 s, 34.7 MB/s 2024-04-26T23:53:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314121 s, 33.4 MB/s 2024-04-26T23:53:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236394 s, 44.4 MB/s 2024-04-26T23:53:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313877 s, 33.4 MB/s 2024-04-26T23:53:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216254 s, 48.5 MB/s 2024-04-26T23:53:30.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.945 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019192 s, 54.6 MB/s 2024-04-26T23:53:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289858 s, 36.2 MB/s 2024-04-26T23:53:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247288 s, 42.4 MB/s 2024-04-26T23:53:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315783 s, 33.2 MB/s 2024-04-26T23:53:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019712 s, 53.2 MB/s 2024-04-26T23:53:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247119 s, 42.4 MB/s 2024-04-26T23:53:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280678 s, 37.4 MB/s 2024-04-26T23:53:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019653 s, 53.4 MB/s 2024-04-26T23:53:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300814 s, 34.9 MB/s 2024-04-26T23:53:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252745 s, 41.5 MB/s 2024-04-26T23:53:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274725 s, 38.2 MB/s 2024-04-26T23:53:31.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.293 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.293 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023335 s, 44.9 MB/s 2024-04-26T23:53:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184264 s, 56.9 MB/s 2024-04-26T23:53:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321097 s, 32.7 MB/s 2024-04-26T23:53:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226793 s, 46.2 MB/s 2024-04-26T23:53:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263143 s, 39.8 MB/s 2024-04-26T23:53:31.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.448 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.448 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237207 s, 44.2 MB/s 2024-04-26T23:53:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024001 s, 43.7 MB/s 2024-04-26T23:53:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231043 s, 45.4 MB/s 2024-04-26T23:53:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-26T23:53:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-26T23:53:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311743 s, 33.6 MB/s 2024-04-26T23:53:31.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:53:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:53:31.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start /scrub_control_test_path recursive 2024-04-26T23:53:31.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.765+0000 7fd453e00640 1 Processor -- start 2024-04-26T23:53:31.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.766+0000 7fd453e00640 1 -- start start 2024-04-26T23:53:31.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.767+0000 7fd453e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd44c1469c0 0x7fd44c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.767+0000 7fd453e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd44c0cf560 0x7fd44c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.767+0000 7fd453e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44c146da0 con 0x7fd44c0cfa10 2024-04-26T23:53:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.767+0000 7fd453e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c0d0ac0 con 0x7fd44c1469c0 2024-04-26T23:53:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.767+0000 7fd453e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c0cfeb0 con 0x7fd44c0cf560 2024-04-26T23:53:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd44c1469c0 0x7fd44c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd44c1469c0 0x7fd44c15d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57840/0 (socket says 172.21.15.46:57840) 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 -- 172.21.15.46:0/1434174039 learned_addr learned my addr 172.21.15.46:0/1434174039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 -- 172.21.15.46:0/1434174039 >> v2:172.21.15.142:3300/0 conn(0x7fd44c0cf560 msgr2=0x7fd44c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 --2- 172.21.15.46:0/1434174039 >> v2:172.21.15.142:3300/0 conn(0x7fd44c0cf560 0x7fd44c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 -- 172.21.15.46:0/1434174039 >> v1:172.21.15.142:6789/0 conn(0x7fd44c0cfa10 legacy=0x7fd44c06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.768+0000 7fd451400640 1 -- 172.21.15.46:0/1434174039 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c0d0190 con 0x7fd44c1469c0 2024-04-26T23:53:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.769+0000 7fd451400640 1 --2- 172.21.15.46:0/1434174039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c1469c0 0x7fd44c15d6d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b6330 tx=0x7fd43c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c3a29786c6d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd44be00640 1 -- 172.21.15.46:0/1434174039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c037070 con 0x7fd44c1469c0 2024-04-26T23:53:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd44be00640 1 -- 172.21.15.46:0/1434174039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c032440 con 0x7fd44c1469c0 2024-04-26T23:53:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd44be00640 1 -- 172.21.15.46:0/1434174039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c032810 con 0x7fd44c1469c0 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 -- 172.21.15.46:0/1434174039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c1469c0 msgr2=0x7fd44c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 --2- 172.21.15.46:0/1434174039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c1469c0 0x7fd44c15d6d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b6330 tx=0x7fd43c02a230 comp rx=0 tx=0).stop 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 -- 172.21.15.46:0/1434174039 shutdown_connections 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 --2- 172.21.15.46:0/1434174039 >> v2:172.21.15.142:3300/0 conn(0x7fd44c0cf560 0x7fd44c0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 --2- 172.21.15.46:0/1434174039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c1469c0 0x7fd44c15d6d0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.770+0000 7fd453e00640 1 -- 172.21.15.46:0/1434174039 >> 172.21.15.46:0/1434174039 conn(0x7fd44c06c770 msgr2=0x7fd44c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.771+0000 7fd453e00640 1 -- 172.21.15.46:0/1434174039 shutdown_connections 2024-04-26T23:53:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.771+0000 7fd453e00640 1 -- 172.21.15.46:0/1434174039 wait complete. 2024-04-26T23:53:31.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.772+0000 7fd453e00640 1 Processor -- start 2024-04-26T23:53:31.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 -- start start 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 0x7fd44c144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd44c1469c0 0x7fd44c072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0737e0 con 0x7fd44c0cf560 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0d0750 con 0x7fd44c0cfa10 2024-04-26T23:53:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.773+0000 7fd453e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c073960 con 0x7fd44c1469c0 2024-04-26T23:53:31.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57842/0 (socket says 172.21.15.46:57842) 2024-04-26T23:53:31.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 -- 172.21.15.46:0/95177633 learned_addr learned my addr 172.21.15.46:0/95177633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd450a00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 0x7fd44c144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 -- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd44c1469c0 msgr2=0x7fd44c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd44c1469c0 0x7fd44c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 -- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 msgr2=0x7fd44c144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 0x7fd44c144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd451400640 1 -- 172.21.15.46:0/95177633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c0b58c0 con 0x7fd44c0cf560 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.774+0000 7fd450a00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 0x7fd44c144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:31.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd451400640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000c00 tx=0x7fd43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd449600640 1 -- 172.21.15.46:0/95177633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c041070 con 0x7fd44c0cf560 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd449600640 1 -- 172.21.15.46:0/95177633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c03c870 con 0x7fd44c0cf560 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd449600640 1 -- 172.21.15.46:0/95177633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c037040 con 0x7fd44c0cf560 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c143830 con 0x7fd44c0cf560 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.775+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c15cad0 con 0x7fd44c0cf560 2024-04-26T23:53:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.777+0000 7fd449600640 1 -- 172.21.15.46:0/95177633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd43c044070 con 0x7fd44c0cf560 2024-04-26T23:53:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.778+0000 7fd449600640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 0x7fd42c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.778+0000 7fd449600640 1 -- 172.21.15.46:0/95177633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fd43c0ac850 con 0x7fd44c0cf560 2024-04-26T23:53:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.779+0000 7fd450a00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 0x7fd42c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.779+0000 7fd456d8e740 1 Processor -- start 2024-04-26T23:53:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.780+0000 7fd456d8e740 1 -- start start 2024-04-26T23:53:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.780+0000 7fd450a00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 0x7fd42c06ee00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd438006fd0 tx=0x7fd438008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9286a70 0x557ef9239400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 0x557ef923e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557ef91a2f90 con 0x557ef9269b90 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557ef913f1e0 con 0x557ef9286a70 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557ef919ef50 con 0x557ef9269450 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57844/0 (socket says 172.21.15.46:57844) 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 -- 172.21.15.46:0/1869700711 learned_addr learned my addr 172.21.15.46:0/1869700711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd451400640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9286a70 0x557ef9239400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd451e00640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 0x557ef923e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 -- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 msgr2=0x557ef923e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 0x557ef923e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 -- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9286a70 msgr2=0x557ef9239400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9286a70 0x557ef9239400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd450a00640 1 -- 172.21.15.46:0/1869700711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef8f80ba0 con 0x557ef9269b90 2024-04-26T23:53:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.781+0000 7fd451e00640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 0x557ef923e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.782+0000 7fd450a00640 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fd43800cac0 tx=0x7fd43800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd89a830daf084a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.782+0000 7fd441c00640 1 -- 172.21.15.46:0/1869700711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438023070 con 0x557ef9269b90 2024-04-26T23:53:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.782+0000 7fd441c00640 1 -- 172.21.15.46:0/1869700711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd438013db0 con 0x557ef9269b90 2024-04-26T23:53:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.782+0000 7fd441c00640 1 -- 172.21.15.46:0/1869700711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43801e600 con 0x557ef9269b90 2024-04-26T23:53:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 -- 172.21.15.46:0/1869700711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 msgr2=0x557ef9268f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fd43800cac0 tx=0x7fd43800f000 comp rx=0 tx=0).stop 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 -- 172.21.15.46:0/1869700711 shutdown_connections 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269450 0x557ef923e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9269b90 0x557ef9268f10 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1869700711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9286a70 0x557ef9239400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 -- 172.21.15.46:0/1869700711 >> 172.21.15.46:0/1869700711 conn(0x557ef923b670 msgr2=0x557ef9283f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 -- 172.21.15.46:0/1869700711 shutdown_connections 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.783+0000 7fd456d8e740 1 -- 172.21.15.46:0/1869700711 wait complete. 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.784+0000 7fd456d8e740 1 -- start start 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.784+0000 7fd456d8e740 1 Processor -- start 2024-04-26T23:53:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269b90 0x557ef926a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 0x557ef9265900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557ef923d820 con 0x557ef9286a70 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557ef9129da0 con 0x557ef926b9b0 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557ef923cc10 con 0x557ef9269b90 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd456d8e740 10 client.0 _ll_register_callbacks cb 0x557ef9156400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57856/0 (socket says 172.21.15.46:57856) 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd450a00640 1 -- 172.21.15.46:0/1119921577 learned_addr learned my addr 172.21.15.46:0/1119921577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.785+0000 7fd451e00640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 0x557ef9265900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 -- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269b90 msgr2=0x557ef926a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269b90 0x557ef926a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 -- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 msgr2=0x557ef9265900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 0x557ef9265900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 -- 172.21.15.46:0/1119921577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef8e74490 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd451e00640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 0x557ef9265900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd450a00640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a1a0 tx=0x7fd4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bf792de25f6296 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd436000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438023040 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fd43801e830 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438013db0 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fd438036050 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd456d8e740 10 client.10446 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.786+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x557ef8fe1330 con 0x557ef9286a70 2024-04-26T23:53:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd438013db0 con 0x557ef9286a70 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7fd438039020 con 0x557ef9286a70 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 10 client.10446 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 10 client.10446 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 20 client.10446 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9265e40 0x557ef9257050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 4 client.10446 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef91b0ba0 con 0x557ef9265e40 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.787+0000 7fd451400640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9265e40 0x557ef9257050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.788+0000 7fd451400640 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9265e40 0x557ef9257050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825e2d93608db6c2 server_cookie=75f2aa15452febf2 in_seq=0 out_seq=0 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.788+0000 7fd436000640 10 client.10446 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd436000640 1 -- 172.21.15.46:0/1119921577 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ef91b0ba0 con 0x557ef9265e40 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd436000640 10 client.10446 handle_command_reply: tid=0 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 client.10446 shutdown 2024-04-26T23:53:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 msgr2=0x557ef9256840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a1a0 tx=0x7fd4380046c0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 shutdown_connections 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9265e40 0x557ef9257050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef926b9b0 0x557ef9265900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef9286a70 0x557ef9256840 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 --2- 172.21.15.46:0/1119921577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9269b90 0x557ef926a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd436000640 0 client.10446 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 >> 172.21.15.46:0/1119921577 conn(0x557ef9269450 msgr2=0x557ef9238b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 shutdown_connections 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 1 -- 172.21.15.46:0/1119921577 wait complete. 2024-04-26T23:53:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.789+0000 7fd456d8e740 20 client.10446 trim_cache size 0 max 16384 2024-04-26T23:53:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.809+0000 7fd456d8e740 1 Processor -- start 2024-04-26T23:53:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.809+0000 7fd456d8e740 1 -- start start 2024-04-26T23:53:31.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef923c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1fdbb0 con 0x557ef926b9b0 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1fcbc0 con 0x557ef9269b90 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c0b4670 con 0x557ef9286a70 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57858/0 (socket says 172.21.15.46:57858) 2024-04-26T23:53:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.810+0000 7fd450a00640 1 -- 172.21.15.46:0/3100494383 learned_addr learned my addr 172.21.15.46:0/3100494383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 -- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 msgr2=0x557ef923cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd451400640 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef923c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 -- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 msgr2=0x557ef923c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef923c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 -- 172.21.15.46:0/3100494383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef8e4d610 con 0x557ef926b9b0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd450a00640 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a330 tx=0x7fd438025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b02db5805756e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd441200640 1 -- 172.21.15.46:0/3100494383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43803e070 con 0x557ef926b9b0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd441200640 1 -- 172.21.15.46:0/3100494383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43802e070 con 0x557ef926b9b0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.811+0000 7fd441200640 1 -- 172.21.15.46:0/3100494383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438023040 con 0x557ef926b9b0 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 -- 172.21.15.46:0/3100494383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 msgr2=0x557ef923f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a330 tx=0x7fd438025000 comp rx=0 tx=0).stop 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 -- 172.21.15.46:0/3100494383 shutdown_connections 2024-04-26T23:53:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef923f140 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 --2- 172.21.15.46:0/3100494383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef923c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 -- 172.21.15.46:0/3100494383 >> 172.21.15.46:0/3100494383 conn(0x557ef9269450 msgr2=0x557ef9265670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 -- 172.21.15.46:0/3100494383 shutdown_connections 2024-04-26T23:53:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.812+0000 7fd456d8e740 1 -- 172.21.15.46:0/3100494383 wait complete. 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.813+0000 7fd456d8e740 1 -- start start 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.813+0000 7fd456d8e740 1 Processor -- start 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef92fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557ef9256f40 con 0x557ef926b9b0 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1fc570 con 0x557ef9269b90 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557ef91a54a0 con 0x557ef9286a70 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd456d8e740 10 client.0 _ll_register_callbacks cb 0x7fd438021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd450a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57874/0 (socket says 172.21.15.46:57874) 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.814+0000 7fd450a00640 1 -- 172.21.15.46:0/398923379 learned_addr learned my addr 172.21.15.46:0/398923379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd451400640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef92fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd451e00640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 -- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 msgr2=0x557ef923cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 -- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 msgr2=0x557ef92fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef92fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 -- 172.21.15.46:0/398923379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef9279b80 con 0x557ef926b9b0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd450a00640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd438021040 tx=0x7fd438012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73195750bd036eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd436000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.815+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438049020 con 0x557ef926b9b0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fd43802b880 con 0x557ef926b9b0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43802e070 con 0x557ef926b9b0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fd43803e040 con 0x557ef926b9b0 2024-04-26T23:53:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 10 client.10452 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x557ef92392c0 con 0x557ef926b9b0 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43802e070 con 0x557ef926b9b0 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7fd438039050 con 0x557ef926b9b0 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 10 client.10452 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 10 client.10452 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 20 client.10452 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9255ce0 0x557ef923d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 4 client.10452 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.816+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x557ef91aaa10 con 0x557ef9255ce0 2024-04-26T23:53:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.817+0000 7fd451400640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9255ce0 0x557ef923d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.817+0000 7fd451400640 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9255ce0 0x557ef923d150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18cf76f7560580b server_cookie=35418e1e70ad6a3e in_seq=0 out_seq=0 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.817+0000 7fd436000640 10 client.10452 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd436000640 1 -- 172.21.15.46:0/398923379 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557ef91aaa10 con 0x557ef9255ce0 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd436000640 10 client.10452 handle_command_reply: tid=0 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 client.10452 shutdown 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 msgr2=0x557ef92557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd438021040 tx=0x7fd438012000 comp rx=0 tx=0).stop 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 shutdown_connections 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x557ef9255ce0 0x557ef923d150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557ef9286a70 0x557ef923cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ef926b9b0 0x557ef92557a0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 --2- 172.21.15.46:0/398923379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ef9269b90 0x557ef92fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd436000640 0 client.10452 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 >> 172.21.15.46:0/398923379 conn(0x557ef9269450 msgr2=0x557ef92640f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 shutdown_connections 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 1 -- 172.21.15.46:0/398923379 wait complete. 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.820+0000 7fd456d8e740 20 client.10452 trim_cache size 0 max 16384 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "164b3e36-9f06-49e5-b190-273bc56a1a3c", 2024-04-26T23:53:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:53:31.826 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:53:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.821+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 msgr2=0x7fd42c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.821+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 0x7fd42c06ee00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd438006fd0 tx=0x7fd438008040 comp rx=0 tx=0).stop 2024-04-26T23:53:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.821+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 msgr2=0x7fd44c15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.821+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000c00 tx=0x7fd43c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 shutdown_connections 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd42c06c940 0x7fd42c06ee00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd44c1469c0 0x7fd44c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd44c0cfa10 0x7fd44c144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 --2- 172.21.15.46:0/95177633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd44c0cf560 0x7fd44c15beb0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 >> 172.21.15.46:0/95177633 conn(0x7fd44c06c770 msgr2=0x7fd44c1419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 shutdown_connections 2024-04-26T23:53:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:31.822+0000 7fd453e00640 1 -- 172.21.15.46:0/95177633 wait complete. 2024-04-26T23:53:31.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:53:31.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub pause 2024-04-26T23:53:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.074+0000 7f10e8800640 1 Processor -- start 2024-04-26T23:53:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.075+0000 7f10e8800640 1 -- start start 2024-04-26T23:53:32.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e8800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10e015cf40 0x7f10e006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e8800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10e006f060 0x7f10e006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e8800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e0160c60 con 0x7f10e00aee20 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e8800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f10e0160f00 con 0x7f10e006f060 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e8800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f10e01611e0 con 0x7f10e015cf40 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10e006f060 0x7f10e006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f10e00aee20 0x7f10e00af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39640/0 (socket says 172.21.15.46:39640) 2024-04-26T23:53:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.076+0000 7f10e6800640 1 -- 172.21.15.46:0/2351963082 learned_addr learned my addr 172.21.15.46:0/2351963082 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e5e00640 1 --2- 172.21.15.46:0/2351963082 >> v2:172.21.15.142:3300/0 conn(0x7f10e015cf40 0x7f10e006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e5400640 1 -- 172.21.15.46:0/2351963082 >> v2:172.21.15.142:3300/0 conn(0x7f10e015cf40 msgr2=0x7f10e006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e5400640 1 --2- 172.21.15.46:0/2351963082 >> v2:172.21.15.142:3300/0 conn(0x7f10e015cf40 0x7f10e006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e5400640 1 -- 172.21.15.46:0/2351963082 >> v1:172.21.15.46:6789/0 conn(0x7f10e00aee20 legacy=0x7f10e00af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e5400640 1 -- 172.21.15.46:0/2351963082 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e01614c0 con 0x7f10e006f060 2024-04-26T23:53:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.077+0000 7f10e4a00640 1 -- 172.21.15.46:0/2351963082 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285866281 0 0) 0x7f10e0160c60 con 0x7f10e00aee20 2024-04-26T23:53:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.078+0000 7f10e5400640 1 --2- 172.21.15.46:0/2351963082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e006f060 0x7f10e006c890 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f10e00d1c10 tx=0x7f10d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4aec931ab7e49ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.078+0000 7f10e4a00640 1 -- 172.21.15.46:0/2351963082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0037070 con 0x7f10e006f060 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.078+0000 7f10e4a00640 1 -- 172.21.15.46:0/2351963082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d0032440 con 0x7f10e006f060 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.078+0000 7f10e4a00640 1 -- 172.21.15.46:0/2351963082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d002ae60 con 0x7f10e006f060 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 -- 172.21.15.46:0/2351963082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e006f060 msgr2=0x7f10e006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 --2- 172.21.15.46:0/2351963082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e006f060 0x7f10e006c890 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f10e00d1c10 tx=0x7f10d002a110 comp rx=0 tx=0).stop 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 -- 172.21.15.46:0/2351963082 shutdown_connections 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 --2- 172.21.15.46:0/2351963082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e006f060 0x7f10e006c890 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 --2- 172.21.15.46:0/2351963082 >> v2:172.21.15.142:3300/0 conn(0x7f10e015cf40 0x7f10e006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 -- 172.21.15.46:0/2351963082 >> 172.21.15.46:0/2351963082 conn(0x7f10e006da90 msgr2=0x7f10e00d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 -- 172.21.15.46:0/2351963082 shutdown_connections 2024-04-26T23:53:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.079+0000 7f10e8800640 1 -- 172.21.15.46:0/2351963082 wait complete. 2024-04-26T23:53:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.081+0000 7f10e8800640 1 Processor -- start 2024-04-26T23:53:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.081+0000 7f10e8800640 1 -- start start 2024-04-26T23:53:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e006f060 0x7f10e00b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e015cf40 0x7f10e015fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00cc840 con 0x7f10e00aee20 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10e015d320 con 0x7f10e015cf40 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e8800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10e00cc9c0 con 0x7f10e006f060 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53406/0 (socket says 172.21.15.46:53406) 2024-04-26T23:53:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.082+0000 7f10e5400640 1 -- 172.21.15.46:0/1834429303 learned_addr learned my addr 172.21.15.46:0/1834429303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 -- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e006f060 msgr2=0x7f10e00b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e006f060 0x7f10e00b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 -- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e015cf40 msgr2=0x7f10e015fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e015cf40 0x7f10e015fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 -- 172.21.15.46:0/1834429303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e00c8f20 con 0x7f10e00aee20 2024-04-26T23:53:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e5400640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f10d0000c00 tx=0x7f10d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10da000640 1 -- 172.21.15.46:0/1834429303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0037040 con 0x7f10e00aee20 2024-04-26T23:53:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10da000640 1 -- 172.21.15.46:0/1834429303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d0042070 con 0x7f10e00aee20 2024-04-26T23:53:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10da000640 1 -- 172.21.15.46:0/1834429303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0030bd0 con 0x7f10e00aee20 2024-04-26T23:53:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e00c91c0 con 0x7f10e00aee20 2024-04-26T23:53:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.083+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e00b6440 con 0x7f10e00aee20 2024-04-26T23:53:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.086+0000 7f10da000640 1 -- 172.21.15.46:0/1834429303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10d0047460 con 0x7f10e00aee20 2024-04-26T23:53:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.087+0000 7f10da000640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.087+0000 7f10da000640 1 -- 172.21.15.46:0/1834429303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f10d00ac950 con 0x7f10e00aee20 2024-04-26T23:53:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.087+0000 7f10eb621740 1 Processor -- start 2024-04-26T23:53:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.087+0000 7f10eb621740 1 -- start start 2024-04-26T23:53:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.087+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 0x55abf5646330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf56637f0 0x55abf5663d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abf557f900 con 0x55abf561bc30 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55abf551c1d0 con 0x55abf56637f0 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55abf557b8c0 con 0x55abf5645f50 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53410/0 (socket says 172.21.15.46:53410) 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.088+0000 7f10e6800640 1 -- 172.21.15.46:0/1136078250 learned_addr learned my addr 172.21.15.46:0/1136078250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 0x55abf5646330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 -- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 msgr2=0x55abf5646330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e5400640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf56637f0 0x55abf5663d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 0x55abf5646330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 -- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf56637f0 msgr2=0x55abf5663d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf56637f0 0x55abf5663d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 -- 172.21.15.46:0/1136078250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf535b7d0 con 0x55abf561bc30 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 0x55abf5646330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10e6800640 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f10dc001d60 tx=0x7f10dc00c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b787f4bf9ffe9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.089+0000 7f10ce600640 1 -- 172.21.15.46:0/1136078250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00c910 con 0x55abf561bc30 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10ce600640 1 -- 172.21.15.46:0/1136078250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc00cab0 con 0x55abf561bc30 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10ce600640 1 -- 172.21.15.46:0/1136078250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00ce60 con 0x55abf561bc30 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 -- 172.21.15.46:0/1136078250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 msgr2=0x55abf5661eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f10dc001d60 tx=0x7f10dc00c0e0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 -- 172.21.15.46:0/1136078250 shutdown_connections 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf561bc30 0x55abf5661eb0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf56637f0 0x55abf5663d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 --2- 172.21.15.46:0/1136078250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5645f50 0x55abf5646330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 -- 172.21.15.46:0/1136078250 >> 172.21.15.46:0/1136078250 conn(0x55abf5618560 msgr2=0x55abf5660d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 -- 172.21.15.46:0/1136078250 shutdown_connections 2024-04-26T23:53:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.090+0000 7f10eb621740 1 -- 172.21.15.46:0/1136078250 wait complete. 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.091+0000 7f10eb621740 1 -- start start 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.091+0000 7f10eb621740 1 Processor -- start 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5649e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abf561c9d0 con 0x55abf56637f0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55abf5507120 con 0x55abf5645f50 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55abf5616490 con 0x55abf5642ab0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10eb621740 10 client.0 _ll_register_callbacks cb 0x55abf553f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53426/0 (socket says 172.21.15.46:53426) 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.092+0000 7f10e5400640 1 -- 172.21.15.46:0/1757416789 learned_addr learned my addr 172.21.15.46:0/1757416789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 -- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 msgr2=0x55abf56155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 -- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 msgr2=0x55abf5649e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5649e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 -- 172.21.15.46:0/1757416789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf52510f0 con 0x55abf56637f0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10e5400640 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f10d007a150 tx=0x7f10d0079e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed03b64b699b1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d00a9060 con 0x55abf56637f0 2024-04-26T23:53:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f10d0045360 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.093+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc00cab0 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f10d0097050 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 10 client.10467 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x55abf53bbf60 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc00cab0 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f10d003a030 con 0x55abf56637f0 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 10 client.10467 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 10 client.10467 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 20 client.10467 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf56582e0 0x55abf5658740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 4 client.10467 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.094+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abf558d510 con 0x55abf56582e0 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.095+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf56582e0 0x55abf5658740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.095+0000 7f10e5e00640 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf56582e0 0x55abf5658740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3338178686e89 server_cookie=ad937b23a4a781ff in_seq=0 out_seq=0 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.095+0000 7f10c2a00640 10 client.10467 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10c2a00640 1 -- 172.21.15.46:0/1757416789 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abf558d510 con 0x55abf56582e0 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10c2a00640 10 client.10467 handle_command_reply: tid=0 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 client.10467 shutdown 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 msgr2=0x55abf5615080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f10d007a150 tx=0x7f10d0079e70 comp rx=0 tx=0).stop 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 shutdown_connections 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf56582e0 0x55abf5658740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5615080 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 --2- 172.21.15.46:0/1757416789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5649e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10c2a00640 0 client.10467 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 >> 172.21.15.46:0/1757416789 conn(0x55abf561bc30 msgr2=0x55abf5660500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.096+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 shutdown_connections 2024-04-26T23:53:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.097+0000 7f10eb621740 1 -- 172.21.15.46:0/1757416789 wait complete. 2024-04-26T23:53:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.097+0000 7f10eb621740 20 client.10467 trim_cache size 0 max 16384 2024-04-26T23:53:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.115+0000 7f10eb621740 1 Processor -- start 2024-04-26T23:53:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 -- start start 2024-04-26T23:53:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5644f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf56637f0 0x55abf56454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0201db0 con 0x55abf5642ab0 2024-04-26T23:53:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0200df0 con 0x55abf5645f50 2024-04-26T23:53:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.116+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10e014c4c0 con 0x55abf56637f0 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53436/0 (socket says 172.21.15.46:53436) 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 -- 172.21.15.46:0/3900882060 learned_addr learned my addr 172.21.15.46:0/3900882060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e6800640 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf56637f0 0x55abf56454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5400640 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5644f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 -- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf56637f0 msgr2=0x55abf56454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf56637f0 0x55abf56454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 -- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 msgr2=0x55abf5644f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5644f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.117+0000 7f10e5e00640 1 -- 172.21.15.46:0/3900882060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf522a1b0 con 0x55abf5642ab0 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f10d400d530 tx=0x7f10d4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf766f75935bf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10cdc00640 1 -- 172.21.15.46:0/3900882060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4014910 con 0x55abf5642ab0 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10cdc00640 1 -- 172.21.15.46:0/3900882060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4014ab0 con 0x55abf5642ab0 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10cdc00640 1 -- 172.21.15.46:0/3900882060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4014e60 con 0x55abf5642ab0 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10eb621740 1 -- 172.21.15.46:0/3900882060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 msgr2=0x55abf56586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.118+0000 7f10eb621740 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f10d400d530 tx=0x7f10d4010000 comp rx=0 tx=0).stop 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 -- 172.21.15.46:0/3900882060 shutdown_connections 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf56637f0 0x55abf56454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5644f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 --2- 172.21.15.46:0/3900882060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf5642ab0 0x55abf56586c0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 -- 172.21.15.46:0/3900882060 >> 172.21.15.46:0/3900882060 conn(0x55abf561bc30 msgr2=0x55abf565a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 -- 172.21.15.46:0/3900882060 shutdown_connections 2024-04-26T23:53:32.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 -- 172.21.15.46:0/3900882060 wait complete. 2024-04-26T23:53:32.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.119+0000 7f10eb621740 1 -- start start 2024-04-26T23:53:32.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.120+0000 7f10eb621740 1 Processor -- start 2024-04-26T23:53:32.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56da210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5637ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00cc020 con 0x55abf56637f0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55abf5638990 con 0x55abf5645f50 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55abf5660380 con 0x55abf5642ab0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10eb621740 10 client.0 _ll_register_callbacks cb 0x7f10d0009110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53442/0 (socket says 172.21.15.46:53442) 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 -- 172.21.15.46:0/3225279251 learned_addr learned my addr 172.21.15.46:0/3225279251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e5400640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5637ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 -- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 msgr2=0x55abf56da210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56da210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 -- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 msgr2=0x55abf5637ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5637ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.121+0000 7f10e6800640 1 -- 172.21.15.46:0/3225279251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf5641780 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10e6800640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000f80 tx=0x7f10dc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd54012156f7315 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0072d0 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f10dc022910 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d4014ab0 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f10dc022b10 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10eb621740 10 client.10473 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.122+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x55abf56855a0 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4014ab0 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f10dc022cb0 con 0x55abf56637f0 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 10 client.10473 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 10 client.10473 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 20 client.10473 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf5645370 0x55abf565a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 4 client.10473 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-26T23:53:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55abf5587380 con 0x55abf5645370 2024-04-26T23:53:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.123+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf5645370 0x55abf565a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10e5e00640 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf5645370 0x55abf565a1b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50178ee53526b6c server_cookie=55261e161e6b524c in_seq=0 out_seq=0 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10c2a00640 10 client.10473 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10c2a00640 1 -- 172.21.15.46:0/3225279251 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55abf5587380 con 0x55abf5645370 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10c2a00640 10 client.10473 handle_command_reply: tid=0 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 client.10473 shutdown 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 msgr2=0x55abf5685060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000f80 tx=0x7f10dc001250 comp rx=0 tx=0).stop 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 shutdown_connections 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55abf5645370 0x55abf565a1b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abf56637f0 0x55abf5685060 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abf5645f50 0x55abf5637ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 --2- 172.21.15.46:0/3225279251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55abf5642ab0 0x55abf56da210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10c2a00640 0 client.10473 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 >> 172.21.15.46:0/3225279251 conn(0x55abf561bc30 msgr2=0x55abf5671bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 shutdown_connections 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.124+0000 7f10eb621740 1 -- 172.21.15.46:0/3225279251 wait complete. 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10eb621740 20 client.10473 trim_cache size 0 max 16384 2024-04-26T23:53:32.131 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 msgr2=0x7f10c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4008040 comp rx=0 tx=0).stop 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 msgr2=0x7f10e01f8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f10d0000c00 tx=0x7f10d00046c0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 shutdown_connections 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e015cf40 0x7f10e015fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e00aee20 0x7f10e01f8b50 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 --2- 172.21.15.46:0/1834429303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e006f060 0x7f10e00b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.125+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 >> 172.21.15.46:0/1834429303 conn(0x7f10e006da90 msgr2=0x7f10e00cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.126+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 shutdown_connections 2024-04-26T23:53:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.126+0000 7f10e8800640 1 -- 172.21.15.46:0/1834429303 wait complete. 2024-04-26T23:53:32.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:53:32.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-26T23:53:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.358+0000 7f8317200640 1 Processor -- start 2024-04-26T23:53:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.359+0000 7f8317200640 1 -- start start 2024-04-26T23:53:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8317200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8310162a70 0x7f83100d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8317200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83100d1cb0 0x7f83100b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8317200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83100d3400 con 0x7f83100d3830 2024-04-26T23:53:32.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8317200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f83100d2140 con 0x7f83100d1cb0 2024-04-26T23:53:32.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8317200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83100d2370 con 0x7f8310162a70 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.360+0000 7f8316800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83100d1cb0 0x7f83100b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83100d1cb0 0x7f83100b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53450/0 (socket says 172.21.15.46:53450) 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8315e00640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8310162a70 0x7f83100d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 -- 172.21.15.46:0/2111262619 learned_addr learned my addr 172.21.15.46:0/2111262619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 -- 172.21.15.46:0/2111262619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8310162a70 msgr2=0x7f83100d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8310162a70 0x7f83100d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 -- 172.21.15.46:0/2111262619 >> v1:172.21.15.142:6789/0 conn(0x7f83100d3830 legacy=0x7f83100d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.361+0000 7f8316800640 1 -- 172.21.15.46:0/2111262619 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100d2650 con 0x7f83100d1cb0 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.362+0000 7f8316800640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83100d1cb0 0x7f83100b29e0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f830c009e50 tx=0x7f830c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbb7a46e7c503db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8314a00640 1 -- 172.21.15.46:0/2111262619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c036070 con 0x7f83100d1cb0 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8314a00640 1 -- 172.21.15.46:0/2111262619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c031600 con 0x7f83100d1cb0 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8314a00640 1 -- 172.21.15.46:0/2111262619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c0319d0 con 0x7f83100d1cb0 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 -- 172.21.15.46:0/2111262619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83100d1cb0 msgr2=0x7f83100b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83100d1cb0 0x7f83100b29e0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f830c009e50 tx=0x7f830c02a7a0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 -- 172.21.15.46:0/2111262619 shutdown_connections 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83100d1cb0 0x7f83100b29e0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 --2- 172.21.15.46:0/2111262619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8310162a70 0x7f83100d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 -- 172.21.15.46:0/2111262619 >> 172.21.15.46:0/2111262619 conn(0x7f83100d4f20 msgr2=0x7f83100d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.363+0000 7f8317200640 1 -- 172.21.15.46:0/2111262619 shutdown_connections 2024-04-26T23:53:32.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.364+0000 7f8317200640 1 -- 172.21.15.46:0/2111262619 wait complete. 2024-04-26T23:53:32.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.365+0000 7f8317200640 1 Processor -- start 2024-04-26T23:53:32.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.365+0000 7f8317200640 1 -- start start 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83100d1cb0 0x7f831015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 0x7f83101f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8310162fd0 con 0x7f8310162a70 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8310162e50 con 0x7f83100d3830 2024-04-26T23:53:32.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8317200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83100b2f20 con 0x7f83100d1cb0 2024-04-26T23:53:32.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53456/0 (socket says 172.21.15.46:53456) 2024-04-26T23:53:32.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 -- 172.21.15.46:0/2054313000 learned_addr learned my addr 172.21.15.46:0/2054313000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8315e00640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83100d1cb0 0x7f831015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8315400640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 0x7f83101f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 -- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83100d1cb0 msgr2=0x7f831015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83100d1cb0 0x7f831015e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 -- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 msgr2=0x7f83101f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 0x7f83101f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.367+0000 7f8316800640 1 -- 172.21.15.46:0/2054313000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100ccb00 con 0x7f8310162a70 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f8315400640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 0x7f83101f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f8316800640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f830c009e50 tx=0x7f830c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f830a000640 1 -- 172.21.15.46:0/2054313000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c036050 con 0x7f8310162a70 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f830a000640 1 -- 172.21.15.46:0/2054313000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c042070 con 0x7f8310162a70 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83100ccde0 con 0x7f8310162a70 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f830a000640 1 -- 172.21.15.46:0/2054313000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c009510 con 0x7f8310162a70 2024-04-26T23:53:32.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.368+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83100cd3a0 con 0x7f8310162a70 2024-04-26T23:53:32.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.372+0000 7f830a000640 1 -- 172.21.15.46:0/2054313000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f830c007b70 con 0x7f8310162a70 2024-04-26T23:53:32.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.373+0000 7f830a000640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 0x7f82f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.373+0000 7f830a000640 1 -- 172.21.15.46:0/2054313000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f830c0acc20 con 0x7f8310162a70 2024-04-26T23:53:32.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.373+0000 7f831b5a2740 1 Processor -- start 2024-04-26T23:53:32.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.373+0000 7f831b5a2740 1 -- start start 2024-04-26T23:53:32.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.373+0000 7f8315e00640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 0x7f82f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d744af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d71a3f0 0x55cf7d760670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cf7d62a030 con 0x55cf7d761fb0 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf7d6457f0 con 0x55cf7d744710 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cf7d6007e0 con 0x55cf7d71a3f0 2024-04-26T23:53:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f8315400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f8315400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53464/0 (socket says 172.21.15.46:53464) 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.374+0000 7f8315400640 1 -- 172.21.15.46:0/789229681 learned_addr learned my addr 172.21.15.46:0/789229681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8316800640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d71a3f0 0x55cf7d760670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 -- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d71a3f0 msgr2=0x55cf7d760670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315e00640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d744af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d71a3f0 0x55cf7d760670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 -- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 msgr2=0x55cf7d744af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d744af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 -- 172.21.15.46:0/789229681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7d60c050 con 0x55cf7d761fb0 2024-04-26T23:53:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315e00640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d744af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315e00640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 0x7f82f406edb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8304009aa0 tx=0x7f8304009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f8315400640 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f8300001710 tx=0x7f8300001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5579236a32251a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f82fe600640 1 -- 172.21.15.46:0/789229681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300018070 con 0x55cf7d761fb0 2024-04-26T23:53:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f82fe600640 1 -- 172.21.15.46:0/789229681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830000d730 con 0x55cf7d761fb0 2024-04-26T23:53:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.375+0000 7f82fe600640 1 -- 172.21.15.46:0/789229681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000db00 con 0x55cf7d761fb0 2024-04-26T23:53:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 -- 172.21.15.46:0/789229681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 msgr2=0x55cf7d762500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f8300001710 tx=0x7f8300001bd0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 -- 172.21.15.46:0/789229681 shutdown_connections 2024-04-26T23:53:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d71a3f0 0x55cf7d760670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d761fb0 0x55cf7d762500 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 --2- 172.21.15.46:0/789229681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d744af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 -- 172.21.15.46:0/789229681 >> 172.21.15.46:0/789229681 conn(0x55cf7d716d20 msgr2=0x55cf7d75f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 -- 172.21.15.46:0/789229681 shutdown_connections 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.376+0000 7f831b5a2740 1 -- 172.21.15.46:0/789229681 wait complete. 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.377+0000 7f831b5a2740 1 -- start start 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.377+0000 7f831b5a2740 1 Processor -- start 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d7731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d761fb0 0x55cf7d76f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cf7d682200 con 0x55cf7d76f6b0 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf7d67cee0 con 0x55cf7d744710 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cf7d689e70 con 0x55cf7d761fb0 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.378+0000 7f831b5a2740 10 client.0 _ll_register_callbacks cb 0x55cf7d673270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53466/0 (socket says 172.21.15.46:53466) 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 -- 172.21.15.46:0/1904125758 learned_addr learned my addr 172.21.15.46:0/1904125758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 -- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d761fb0 msgr2=0x55cf7d76f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d761fb0 0x55cf7d76f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 -- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 msgr2=0x55cf7d7731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d7731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 -- 172.21.15.46:0/1904125758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7d4597d0 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f8316800640 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f830c02a7e0 tx=0x7f830c07a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91017107a743e2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f82f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.379+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c0a9060 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f830c0aaa40 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830000d730 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f830c097050 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 10 client.10485 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x55cf7d38f3f0 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830000d730 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f830c079b70 con 0x55cf7d76f6b0 2024-04-26T23:53:32.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 10 client.10485 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 10 client.10485 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 20 client.10485 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d756b60 0x55cf7d713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 4 client.10485 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.380+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7d68b7e0 con 0x55cf7d756b60 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.381+0000 7f8315e00640 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d756b60 0x55cf7d713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.381+0000 7f8315e00640 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d756b60 0x55cf7d713960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecc8ce4533d7f0c server_cookie=8cd3c2d311eaa5fd in_seq=0 out_seq=0 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.381+0000 7f82f2a00640 10 client.10485 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f82f2a00640 1 -- 172.21.15.46:0/1904125758 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf7d68b7e0 con 0x55cf7d756b60 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f82f2a00640 10 client.10485 handle_command_reply: tid=0 2024-04-26T23:53:32.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 client.10485 shutdown 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 msgr2=0x55cf7d7565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f830c02a7e0 tx=0x7f830c07a2e0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 shutdown_connections 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d756b60 0x55cf7d713960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7565d0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f82f2a00640 0 client.10485 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.382+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d761fb0 0x55cf7d76f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f831b5a2740 1 --2- 172.21.15.46:0/1904125758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d744710 0x55cf7d7731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 >> 172.21.15.46:0/1904125758 conn(0x55cf7d71a3f0 msgr2=0x55cf7d75ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 shutdown_connections 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f831b5a2740 1 -- 172.21.15.46:0/1904125758 wait complete. 2024-04-26T23:53:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.383+0000 7f831b5a2740 20 client.10485 trim_cache size 0 max 16384 2024-04-26T23:53:32.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.402+0000 7f831b5a2740 1 Processor -- start 2024-04-26T23:53:32.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 -- start start 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d744710 0x55cf7d760480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8310201ce0 con 0x55cf7d76f6b0 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f831014c2c0 con 0x55cf7d761fb0 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.403+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f831014db90 con 0x55cf7d744710 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53474/0 (socket says 172.21.15.46:53474) 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 -- 172.21.15.46:0/729757762 learned_addr learned my addr 172.21.15.46:0/729757762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8315400640 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 -- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d744710 msgr2=0x55cf7d760480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d744710 0x55cf7d760480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 -- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 msgr2=0x55cf7d7609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8316800640 1 -- 172.21.15.46:0/729757762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7d713cd0 con 0x55cf7d76f6b0 2024-04-26T23:53:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.404+0000 7f8315400640 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f8316800640 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f830c07a420 tx=0x7f830c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d65c33b5252a3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f82fdc00640 1 -- 172.21.15.46:0/729757762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c07a4f0 con 0x55cf7d76f6b0 2024-04-26T23:53:32.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f82fdc00640 1 -- 172.21.15.46:0/729757762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c037040 con 0x55cf7d76f6b0 2024-04-26T23:53:32.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f82fdc00640 1 -- 172.21.15.46:0/729757762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c031600 con 0x55cf7d76f6b0 2024-04-26T23:53:32.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 -- 172.21.15.46:0/729757762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 msgr2=0x55cf7d7473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f830c07a420 tx=0x7f830c07c000 comp rx=0 tx=0).stop 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 -- 172.21.15.46:0/729757762 shutdown_connections 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d76f6b0 0x55cf7d7473e0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 --2- 172.21.15.46:0/729757762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d744710 0x55cf7d760480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.405+0000 7f831b5a2740 1 -- 172.21.15.46:0/729757762 >> 172.21.15.46:0/729757762 conn(0x55cf7d71a3f0 msgr2=0x55cf7d759100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.406+0000 7f831b5a2740 1 -- 172.21.15.46:0/729757762 shutdown_connections 2024-04-26T23:53:32.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.406+0000 7f831b5a2740 1 -- 172.21.15.46:0/729757762 wait complete. 2024-04-26T23:53:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.406+0000 7f831b5a2740 1 -- start start 2024-04-26T23:53:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.406+0000 7f831b5a2740 1 Processor -- start 2024-04-26T23:53:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 0x55cf7d782b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cf7d68afd0 con 0x55cf7d744710 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f831014bc70 con 0x55cf7d761fb0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cf7d759770 con 0x55cf7d76f6b0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.407+0000 7f831b5a2740 10 client.0 _ll_register_callbacks cb 0x7f830c02aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53478/0 (socket says 172.21.15.46:53478) 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 -- 172.21.15.46:0/3249475427 learned_addr learned my addr 172.21.15.46:0/3249475427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315400640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8316800640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 0x55cf7d782b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 -- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 msgr2=0x55cf7d782b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 0x55cf7d782b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 -- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 msgr2=0x55cf7d7825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 -- 172.21.15.46:0/3249475427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7d34f0f0 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8316800640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 0x55cf7d782b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f8315e00640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f8304011ac0 tx=0x7f83040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d13bebc024fb5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.408+0000 7f82f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304011cc0 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f830400f050 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830c037040 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f8304035050 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f831b5a2740 10 client.10491 fetch_fsmap learned FSMap version 127 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=2+}) -- 0x55cf7d645a30 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c037040 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.409+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 127) ==== 3494+0+0 (secure 0 0 0) 0x7f8304038020 con 0x55cf7d744710 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 10 client.10491 fetch_fsmap finished waiting for FSMap version 127 2024-04-26T23:53:32.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 10 client.10491 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 20 client.10491 populate_metadata read hostname 'smithi046' 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d757360 0x55cf7d757740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 4 client.10491 mds_command: new command op to 10287 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 --> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf7d685650 con 0x55cf7d757360 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f8315400640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d757360 0x55cf7d757740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f8315400640 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d757360 0x55cf7d757740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa0ee240c62a5f7 server_cookie=c826acd9a5a4a12a in_seq=0 out_seq=0 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.410+0000 7f82f2a00640 10 client.10491 ms_handle_connect on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f82f2a00640 1 -- 172.21.15.46:0/3249475427 <== mds.0 v2:172.21.15.46:6833/1585151449 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x55cf7d685650 con 0x55cf7d757360 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f82f2a00640 10 client.10491 handle_command_reply: tid=0 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 client.10491 shutdown 2024-04-26T23:53:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 msgr2=0x55cf7d7758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f8304011ac0 tx=0x7f83040046c0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 shutdown_connections 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:6833/1585151449,v1:172.21.15.46:6835/1585151449] conn(0x55cf7d757360 0x55cf7d757740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cf7d76f6b0 0x55cf7d782b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf7d761fb0 0x55cf7d7825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 --2- 172.21.15.46:0/3249475427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf7d744710 0x55cf7d7758f0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f82f2a00640 0 client.10491 ms_handle_reset on v2:172.21.15.46:6833/1585151449 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 >> 172.21.15.46:0/3249475427 conn(0x55cf7d71a3f0 msgr2=0x55cf7d760480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 shutdown_connections 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.411+0000 7f831b5a2740 1 -- 172.21.15.46:0/3249475427 wait complete. 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.412+0000 7f831b5a2740 20 client.10491 trim_cache size 0 max 16384 2024-04-26T23:53:32.419 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "164b3e36-9f06-49e5-b190-273bc56a1a3c": { 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/scrub_control_test_path", 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "164b3e36-9f06-49e5-b190-273bc56a1a3c", 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:53:32.420 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:53:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.412+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 msgr2=0x7f82f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.412+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 0x7f82f406edb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8304009aa0 tx=0x7f8304009190 comp rx=0 tx=0).stop 2024-04-26T23:53:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.412+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 msgr2=0x7f83101f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.412+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f830c009e50 tx=0x7f830c004a40 comp rx=0 tx=0).stop 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 shutdown_connections 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82f406c8f0 0x7f82f406edb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8310162a70 0x7f83101f8810 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83100d3830 0x7f83101f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 --2- 172.21.15.46:0/2054313000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83100d1cb0 0x7f831015e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 >> 172.21.15.46:0/2054313000 conn(0x7f83100d4f20 msgr2=0x7f8310161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 shutdown_connections 2024-04-26T23:53:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.413+0000 7f8317200640 1 -- 172.21.15.46:0/2054313000 wait complete. 2024-04-26T23:53:32.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.637+0000 7fd5e9000640 1 Processor -- start 2024-04-26T23:53:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.637+0000 7fd5e9000640 1 -- start start 2024-04-26T23:53:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.638+0000 7fd5e9000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5e40d2130 0x7fd5e40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.638+0000 7fd5e9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e40b4000 0x7fd5e40b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.638+0000 7fd5e9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40d11f0 con 0x7fd5e415f4e0 2024-04-26T23:53:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.638+0000 7fd5e9000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40d0ce0 con 0x7fd5e40b4000 2024-04-26T23:53:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.638+0000 7fd5e9000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40d2990 con 0x7fd5e40d2130 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e40b4000 0x7fd5e40b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd5e415f4e0 0x7fd5e40d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39650/0 (socket says 172.21.15.46:39650) 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3400640 1 -- 172.21.15.46:0/2237442820 learned_addr learned my addr 172.21.15.46:0/2237442820 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/2237442820 >> v2:172.21.15.142:3300/0 conn(0x7fd5e40d2130 0x7fd5e40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2237442820 >> v2:172.21.15.142:3300/0 conn(0x7fd5e40d2130 msgr2=0x7fd5e40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2237442820 >> v2:172.21.15.142:3300/0 conn(0x7fd5e40d2130 0x7fd5e40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2237442820 >> v1:172.21.15.46:6789/0 conn(0x7fd5e415f4e0 legacy=0x7fd5e40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.639+0000 7fd5e3e00640 1 -- 172.21.15.46:0/2237442820 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40d5f20 con 0x7fd5e40b4000 2024-04-26T23:53:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.640+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2237442820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40b4000 0x7fd5e40b2280 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097a0 tx=0x7fd5dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315f60c514177de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.641+0000 7fd5e2000640 1 -- 172.21.15.46:0/2237442820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc037070 con 0x7fd5e40b4000 2024-04-26T23:53:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.641+0000 7fd5e2000640 1 -- 172.21.15.46:0/2237442820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc032440 con 0x7fd5e40b4000 2024-04-26T23:53:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.641+0000 7fd5e2000640 1 -- 172.21.15.46:0/2237442820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc032810 con 0x7fd5e40b4000 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.641+0000 7fd5e9000640 1 -- 172.21.15.46:0/2237442820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40b4000 msgr2=0x7fd5e40b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.641+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2237442820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40b4000 0x7fd5e40b2280 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097a0 tx=0x7fd5dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 -- 172.21.15.46:0/2237442820 shutdown_connections 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2237442820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40b4000 0x7fd5e40b2280 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2237442820 >> v2:172.21.15.142:3300/0 conn(0x7fd5e40d2130 0x7fd5e40d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 -- 172.21.15.46:0/2237442820 >> 172.21.15.46:0/2237442820 conn(0x7fd5e40d5010 msgr2=0x7fd5e40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 -- 172.21.15.46:0/2237442820 shutdown_connections 2024-04-26T23:53:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.642+0000 7fd5e9000640 1 -- 172.21.15.46:0/2237442820 wait complete. 2024-04-26T23:53:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.643+0000 7fd5e9000640 1 Processor -- start 2024-04-26T23:53:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.644+0000 7fd5e9000640 1 -- start start 2024-04-26T23:53:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40b4000 0x7fd5e4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 0x7fd5e4161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40b1d60 con 0x7fd5e40d2130 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40d5c40 con 0x7fd5e415f4e0 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e415f8c0 con 0x7fd5e40b4000 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53502/0 (socket says 172.21.15.46:53502) 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 -- 172.21.15.46:0/2932242470 learned_addr learned my addr 172.21.15.46:0/2932242470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 0x7fd5e4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 -- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40b4000 msgr2=0x7fd5e4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40b4000 0x7fd5e4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 -- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 msgr2=0x7fd5e4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 0x7fd5e4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e2a00640 1 -- 172.21.15.46:0/2932242470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40b9560 con 0x7fd5e40d2130 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.645+0000 7fd5e3e00640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 0x7fd5e4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5e2a00640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4007ef0 tx=0x7fd5d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4014620 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d4019070 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4014a00 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e40b9840 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.646+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e40b9e00 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.647+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e40693b0 con 0x7fd5e40d2130 2024-04-26T23:53:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.648+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5d4014ba0 con 0x7fd5e40d2130 2024-04-26T23:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.649+0000 7fd5db400640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 0x7fd5c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.649+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fd5d408c690 con 0x7fd5e40d2130 2024-04-26T23:53:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.650+0000 7fd5e3400640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 0x7fd5c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.652+0000 7fd5e3400640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 0x7fd5c006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000f60 tx=0x7fd5cc007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.655+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5d405a890 con 0x7fd5e40d2130 2024-04-26T23:53:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.901+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e406cdb0 con 0x7fd5e40d2130 2024-04-26T23:53:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.902+0000 7fd5db400640 1 -- 172.21.15.46:0/2932242470 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7fd5d405a190 con 0x7fd5e40d2130 2024-04-26T23:53:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:32.907 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 msgr2=0x7fd5c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 0x7fd5c006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000f60 tx=0x7fd5cc007a80 comp rx=0 tx=0).stop 2024-04-26T23:53:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 msgr2=0x7fd5e40b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4007ef0 tx=0x7fd5d400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 shutdown_connections 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd5c006c8f0 0x7fd5c006edb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e415f4e0 0x7fd5e4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e40d2130 0x7fd5e40b8c80 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 --2- 172.21.15.46:0/2932242470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e40b4000 0x7fd5e4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 >> 172.21.15.46:0/2932242470 conn(0x7fd5e40d5010 msgr2=0x7fd5e40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 shutdown_connections 2024-04-26T23:53:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:32.905+0000 7fd5e9000640 1 -- 172.21.15.46:0/2932242470 wait complete. 2024-04-26T23:53:32.930 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:53:32.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:53:33.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.117+0000 7fae32800640 1 Processor -- start 2024-04-26T23:53:33.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.117+0000 7fae32800640 1 -- start start 2024-04-26T23:53:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.119+0000 7fae32800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.119+0000 7fae32800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fae2c160bd0 0x7fae2c161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.119+0000 7fae32800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae2c06ee30 con 0x7fae2c15d540 2024-04-26T23:53:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.119+0000 7fae32800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fae2c0742d0 con 0x7fae2c160bd0 2024-04-26T23:53:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.119+0000 7fae32800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c161940 con 0x7fae2c06e510 2024-04-26T23:53:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.120+0000 7fae2b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.120+0000 7fae2b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:40090/0 (socket says 172.21.15.46:40090) 2024-04-26T23:53:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.120+0000 7fae2b400640 1 -- 172.21.15.46:0/4040767223 learned_addr learned my addr 172.21.15.46:0/4040767223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.120+0000 7fae22a00640 1 --2- 172.21.15.46:0/4040767223 >> v2:172.21.15.142:3300/0 conn(0x7fae2c160bd0 0x7fae2c161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.120+0000 7fae2b400640 1 -- 172.21.15.46:0/4040767223 >> v2:172.21.15.142:3300/0 conn(0x7fae2c160bd0 msgr2=0x7fae2c161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.121+0000 7fae2b400640 1 --2- 172.21.15.46:0/4040767223 >> v2:172.21.15.142:3300/0 conn(0x7fae2c160bd0 0x7fae2c161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.121+0000 7fae2b400640 1 -- 172.21.15.46:0/4040767223 >> v1:172.21.15.142:6789/0 conn(0x7fae2c15d540 legacy=0x7fae2c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.121+0000 7fae2b400640 1 -- 172.21.15.46:0/4040767223 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c071f80 con 0x7fae2c06e510 2024-04-26T23:53:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.122+0000 7fae2b400640 1 --2- 172.21.15.46:0/4040767223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009e50 tx=0x7fae1c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36eed525981c88fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.122+0000 7fae2aa00640 1 -- 172.21.15.46:0/4040767223 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c036070 con 0x7fae2c06e510 2024-04-26T23:53:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.122+0000 7fae2aa00640 1 -- 172.21.15.46:0/4040767223 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c031600 con 0x7fae2c06e510 2024-04-26T23:53:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.122+0000 7fae2aa00640 1 -- 172.21.15.46:0/4040767223 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0319d0 con 0x7fae2c06e510 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 -- 172.21.15.46:0/4040767223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 msgr2=0x7fae2c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 --2- 172.21.15.46:0/4040767223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009e50 tx=0x7fae1c02a710 comp rx=0 tx=0).stop 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 -- 172.21.15.46:0/4040767223 shutdown_connections 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 --2- 172.21.15.46:0/4040767223 >> v2:172.21.15.142:3300/0 conn(0x7fae2c160bd0 0x7fae2c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 --2- 172.21.15.46:0/4040767223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c06e510 0x7fae2c06e8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 -- 172.21.15.46:0/4040767223 >> 172.21.15.46:0/4040767223 conn(0x7fae2c06c530 msgr2=0x7fae2c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 -- 172.21.15.46:0/4040767223 shutdown_connections 2024-04-26T23:53:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.123+0000 7fae32800640 1 -- 172.21.15.46:0/4040767223 wait complete. 2024-04-26T23:53:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.125+0000 7fae32800640 1 Processor -- start 2024-04-26T23:53:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.125+0000 7fae32800640 1 -- start start 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae2c15d540 0x7fae2c1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c160bd0 0x7fae2c13f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c072820 con 0x7fae2c06e510 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c072540 con 0x7fae2c15d540 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae32800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c161570 con 0x7fae2c160bd0 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53518/0 (socket says 172.21.15.46:53518) 2024-04-26T23:53:33.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.126+0000 7fae2b400640 1 -- 172.21.15.46:0/2444665612 learned_addr learned my addr 172.21.15.46:0/2444665612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2be00640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c160bd0 0x7fae2c13f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae22a00640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae2c15d540 0x7fae2c1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 -- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c160bd0 msgr2=0x7fae2c13f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c160bd0 0x7fae2c13f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 -- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae2c15d540 msgr2=0x7fae2c1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae2c15d540 0x7fae2c1422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 -- 172.21.15.46:0/2444665612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c13e1b0 con 0x7fae2c06e510 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.127+0000 7fae2b400640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fae1c034040 tx=0x7fae1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.128+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c041020 con 0x7fae2c06e510 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.128+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c007820 con 0x7fae2c06e510 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.128+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c036040 con 0x7fae2c06e510 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.128+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c13e490 con 0x7fae2c06e510 2024-04-26T23:53:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.128+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c13ed30 con 0x7fae2c06e510 2024-04-26T23:53:33.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.129+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2c13e660 con 0x7fae2c06e510 2024-04-26T23:53:33.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.130+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae1c044020 con 0x7fae2c06e510 2024-04-26T23:53:33.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.131+0000 7fae23e00640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:33.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.131+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fae1c0ac9d0 con 0x7fae2c06e510 2024-04-26T23:53:33.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.132+0000 7fae22a00640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:33.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.133+0000 7fae22a00640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 0x7fae0806edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0ce440 tx=0x7fae14008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:33.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.135+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae1c04b030 con 0x7fae2c06e510 2024-04-26T23:53:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.424+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fae2c069530 con 0x7fae2c06e510 2024-04-26T23:53:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.425+0000 7fae23e00640 1 -- 172.21.15.46:0/2444665612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1580 (secure 0 0 0) 0x7fae1c07ac60 con 0x7fae2c06e510 2024-04-26T23:53:33.428 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:33.429 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1004,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":161,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1034,"data_bytes":1061867345,"bytes_used":2556534784,"bytes_avail":770537578496,"bytes_total":773094113280,"write_bytes_sec":36595225,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":127,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"c","status":"up:active","gid":10287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":36,"modified":"2024-04-26T23:53:02.248184+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:53:33.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.427+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 msgr2=0x7fae0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 0x7fae0806edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0ce440 tx=0x7fae14008040 comp rx=0 tx=0).stop 2024-04-26T23:53:33.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 msgr2=0x7fae2c13db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:33.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fae1c034040 tx=0x7fae1c004a40 comp rx=0 tx=0).stop 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 shutdown_connections 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae2c160bd0 0x7fae2c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae2c15d540 0x7fae2c1422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 --2- 172.21.15.46:0/2444665612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2c06e510 0x7fae2c13db20 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 >> 172.21.15.46:0/2444665612 conn(0x7fae2c06c530 msgr2=0x7fae2c0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 shutdown_connections 2024-04-26T23:53:33.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:33.428+0000 7fae32800640 1 -- 172.21.15.46:0/2444665612 wait complete. 2024-04-26T23:53:33.451 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:53:34.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.665+0000 7f05ef600640 1 Processor -- start 2024-04-26T23:53:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.665+0000 7f05ef600640 1 -- start start 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ef600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e80d1520 0x7f05e80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ef600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ef600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e80d1e40 con 0x7f05e815d510 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ef600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f05e80d1220 con 0x7f05e80d1520 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ef600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05e80ce560 con 0x7f05e80d3af0 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ecc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e80d1520 0x7f05e80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ecc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e80d1520 0x7f05e80d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53534/0 (socket says 172.21.15.46:53534) 2024-04-26T23:53:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.667+0000 7f05ecc00640 1 -- 172.21.15.46:0/4100410696 learned_addr learned my addr 172.21.15.46:0/4100410696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ed600640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ecc00640 1 -- 172.21.15.46:0/4100410696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 msgr2=0x7f05e80706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ecc00640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e80706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ecc00640 1 -- 172.21.15.46:0/4100410696 >> v1:172.21.15.142:6789/0 conn(0x7f05e815d510 legacy=0x7f05e80d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ecc00640 1 -- 172.21.15.46:0/4100410696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e80ce7b0 con 0x7f05e80d1520 2024-04-26T23:53:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.668+0000 7f05ed600640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e80706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.669+0000 7f05ecc00640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e80d1520 0x7f05e80d1900 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009870 tx=0x7f05dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91d9970e817a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05e7400640 1 -- 172.21.15.46:0/4100410696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc037070 con 0x7f05e80d1520 2024-04-26T23:53:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05e7400640 1 -- 172.21.15.46:0/4100410696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc032440 con 0x7f05e80d1520 2024-04-26T23:53:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05e7400640 1 -- 172.21.15.46:0/4100410696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc032810 con 0x7f05e80d1520 2024-04-26T23:53:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 -- 172.21.15.46:0/4100410696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e80d1520 msgr2=0x7f05e80d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e80d1520 0x7f05e80d1900 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009870 tx=0x7f05dc02a210 comp rx=0 tx=0).stop 2024-04-26T23:53:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 -- 172.21.15.46:0/4100410696 shutdown_connections 2024-04-26T23:53:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e80706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 --2- 172.21.15.46:0/4100410696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e80d1520 0x7f05e80d1900 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.670+0000 7f05ef600640 1 -- 172.21.15.46:0/4100410696 >> 172.21.15.46:0/4100410696 conn(0x7f05e80d2f40 msgr2=0x7f05e80d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.671+0000 7f05ef600640 1 -- 172.21.15.46:0/4100410696 shutdown_connections 2024-04-26T23:53:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.671+0000 7f05ef600640 1 -- 172.21.15.46:0/4100410696 wait complete. 2024-04-26T23:53:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.673+0000 7f05ef600640 1 Processor -- start 2024-04-26T23:53:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.673+0000 7f05ef600640 1 -- start start 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e81f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05e815d510 0x7f05e80cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05e80ced70 con 0x7f05e81f6f40 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8070230 con 0x7f05e815d510 2024-04-26T23:53:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.674+0000 7f05ef600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05e8070c00 con 0x7f05e80d3af0 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53542/0 (socket says 172.21.15.46:53542) 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 -- 172.21.15.46:0/2277662296 learned_addr learned my addr 172.21.15.46:0/2277662296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ecc00640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e81f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05e7e00640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05e815d510 0x7f05e80cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 -- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 msgr2=0x7f05e81f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e81f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 -- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05e815d510 msgr2=0x7f05e80cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05e815d510 0x7f05e80cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.675+0000 7f05ed600640 1 -- 172.21.15.46:0/2277662296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e80cd1d0 con 0x7f05e81f6f40 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05ed600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f05e000cea0 tx=0x7f05e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0003b90 con 0x7f05e81f6f40 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e0003d50 con 0x7f05e81f6f40 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0004260 con 0x7f05e81f6f40 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e806d460 con 0x7f05e81f6f40 2024-04-26T23:53:34.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.676+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e806da20 con 0x7f05e81f6f40 2024-04-26T23:53:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.677+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e8068ec0 con 0x7f05e81f6f40 2024-04-26T23:53:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.678+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05e0004400 con 0x7f05e81f6f40 2024-04-26T23:53:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.679+0000 7f05e4c00640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 0x7f05c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.679+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f05e008c6f0 con 0x7f05e81f6f40 2024-04-26T23:53:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.680+0000 7f05ecc00640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 0x7f05c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.681+0000 7f05ecc00640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 0x7f05c806edb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f05dc034000 tx=0x7f05dc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:34.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.684+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05e8068ec0 con 0x7f05e81f6f40 2024-04-26T23:53:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.937+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e80d1900 con 0x7f05e81f6f40 2024-04-26T23:53:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.938+0000 7f05e4c00640 1 -- 172.21.15.46:0/2277662296 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7f05e80d1900 con 0x7f05e81f6f40 2024-04-26T23:53:34.942 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:34.942 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.941+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 msgr2=0x7f05c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.941+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 0x7f05c806edb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f05dc034000 tx=0x7f05dc02e000 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.941+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 msgr2=0x7f05e80c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f05e000cea0 tx=0x7f05e000cf90 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 shutdown_connections 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f05c806c8f0 0x7f05c806edb0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e81f6f40 0x7f05e80c85a0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05e815d510 0x7f05e80cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 --2- 172.21.15.46:0/2277662296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05e80d3af0 0x7f05e81f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 >> 172.21.15.46:0/2277662296 conn(0x7f05e80d2f40 msgr2=0x7f05e80afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 shutdown_connections 2024-04-26T23:53:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:34.942+0000 7f05ef600640 1 -- 172.21.15.46:0/2277662296 wait complete. 2024-04-26T23:53:34.969 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:53:34.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:53:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.164+0000 7faa14600640 1 Processor -- start 2024-04-26T23:53:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.164+0000 7faa14600640 1 -- start start 2024-04-26T23:53:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa14600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faa0c0d1520 0x7faa0c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa14600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c0d1e40 con 0x7faa0c0d3af0 2024-04-26T23:53:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa14600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c0ce7b0 con 0x7faa0c15d510 2024-04-26T23:53:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa14600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0d1220 con 0x7faa0c0d1520 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa12600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faa0c0d3af0 0x7faa0c0706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39664/0 (socket says 172.21.15.46:39664) 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa12600640 1 -- 172.21.15.46:0/1767009719 learned_addr learned my addr 172.21.15.46:0/1767009719 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa11c00640 1 --2- 172.21.15.46:0/1767009719 >> v2:172.21.15.142:3300/0 conn(0x7faa0c0d1520 0x7faa0c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407727898 0 0) 0x7faa0c0d1e40 con 0x7faa0c0d3af0 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.166+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9f4003520 con 0x7faa0c0d3af0 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 97151129 0 0) 0x7fa9f4003520 con 0x7faa0c0d3af0 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 >> v2:172.21.15.142:3300/0 conn(0x7faa0c0d1520 msgr2=0x7faa0c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 --2- 172.21.15.46:0/1767009719 >> v2:172.21.15.142:3300/0 conn(0x7faa0c0d1520 0x7faa0c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 >> v1:172.21.15.142:6789/0 conn(0x7faa0c15d510 legacy=0x7faa0c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0cea90 con 0x7faa0c0d3af0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7faa040019f0 con 0x7faa0c0d3af0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 >> v1:172.21.15.46:6789/0 conn(0x7faa0c0d3af0 legacy=0x7faa0c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9f40039b0 0x7fa9f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 0x7fa9f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f40051c0 0x7fa9f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0d1220 con 0x7fa9f4004800 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f400c060 con 0x7fa9f40039b0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f400c260 con 0x7fa9f40051c0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.167+0000 7faa11200640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 0x7fa9f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa12600640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f40051c0 0x7fa9f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa11200640 1 -- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f40051c0 msgr2=0x7fa9f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa11200640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f40051c0 0x7fa9f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa11200640 1 -- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9f40039b0 msgr2=0x7fa9f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa11200640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9f40039b0 0x7fa9f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa11200640 1 -- 172.21.15.46:0/1767009719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f4003520 con 0x7fa9f4004800 2024-04-26T23:53:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa12600640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f40051c0 0x7fa9f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:35.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.168+0000 7faa12600640 1 -- 172.21.15.46:0/1767009719 reap_dead start 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.169+0000 7faa11200640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 0x7fa9f4004c80 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8001720 tx=0x7fa9f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918865d4dd8bcf65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.169+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8001370 con 0x7fa9f4004800 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.169+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f80052d0 con 0x7fa9f4004800 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.169+0000 7faa0be00640 1 -- 172.21.15.46:0/1767009719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802ab90 con 0x7fa9f4004800 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 -- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 msgr2=0x7fa9f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 0x7fa9f4004c80 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8001720 tx=0x7fa9f4005830 comp rx=0 tx=0).stop 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 -- 172.21.15.46:0/1767009719 shutdown_connections 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 --2- 172.21.15.46:0/1767009719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f4004800 0x7fa9f4004c80 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 -- 172.21.15.46:0/1767009719 >> 172.21.15.46:0/1767009719 conn(0x7faa0c0d2f40 msgr2=0x7faa0c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 -- 172.21.15.46:0/1767009719 shutdown_connections 2024-04-26T23:53:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.170+0000 7faa14600640 1 -- 172.21.15.46:0/1767009719 wait complete. 2024-04-26T23:53:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.172+0000 7faa14600640 1 Processor -- start 2024-04-26T23:53:35.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.173+0000 7faa14600640 1 -- start start 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f4004800 0x7faa0c156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 0x7faa0c14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c070230 con 0x7faa0c14d490 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0ce560 con 0x7faa0c157300 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa14600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c0d1900 con 0x7fa9f4004800 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa12600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa12600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53556/0 (socket says 172.21.15.46:53556) 2024-04-26T23:53:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa12600640 1 -- 172.21.15.46:0/1053763580 learned_addr learned my addr 172.21.15.46:0/1053763580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.174+0000 7faa11c00640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f4004800 0x7faa0c156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa11200640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 0x7faa0c14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 -- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f4004800 msgr2=0x7faa0c156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f4004800 0x7faa0c156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 -- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 msgr2=0x7faa0c14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 0x7faa0c14cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 -- 172.21.15.46:0/1053763580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1599d0 con 0x7faa0c14d490 2024-04-26T23:53:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa11200640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 0x7faa0c14cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa12600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7faa0c073e60 tx=0x7faa0400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.175+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0400bb80 con 0x7faa0c14d490 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.176+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0400bd20 con 0x7faa0c14d490 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.176+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04007350 con 0x7faa0c14d490 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.176+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c159cb0 con 0x7faa0c14d490 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.176+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c0b2ef0 con 0x7faa0c14d490 2024-04-26T23:53:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.177+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c0b2ce0 con 0x7faa0c14d490 2024-04-26T23:53:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.179+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa0401b020 con 0x7faa0c14d490 2024-04-26T23:53:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.179+0000 7faa09600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.179+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7faa0408bfe0 con 0x7faa0c14d490 2024-04-26T23:53:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.180+0000 7faa11c00640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.181+0000 7faa11c00640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 0x7fa9ec06ede0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faa0c06b270 tx=0x7faa00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.183+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa0405a2e0 con 0x7faa0c14d490 2024-04-26T23:53:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.470+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7faa0c069530 con 0x7faa0c14d490 2024-04-26T23:53:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.471+0000 7faa09600640 1 -- 172.21.15.46:0/1053763580 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1580 (secure 0 0 0) 0x7faa04059be0 con 0x7faa0c14d490 2024-04-26T23:53:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:35.475 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1006,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":161,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1034,"data_bytes":1061867345,"bytes_used":2556534784,"bytes_avail":770537578496,"bytes_total":773094113280,"write_bytes_sec":24252738,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":127,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"c","status":"up:active","gid":10287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":36,"modified":"2024-04-26T23:53:02.248184+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:53:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 msgr2=0x7fa9ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 0x7fa9ec06ede0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faa0c06b270 tx=0x7faa00008040 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 msgr2=0x7faa0c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7faa0c073e60 tx=0x7faa0400b4a0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 shutdown_connections 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa0c14d490 0x7faa0c158d30 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa0c157300 0x7faa0c14cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 --2- 172.21.15.46:0/1053763580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9f4004800 0x7faa0c156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 >> 172.21.15.46:0/1053763580 conn(0x7faa0c0d2f40 msgr2=0x7faa0c15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 shutdown_connections 2024-04-26T23:53:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:35.474+0000 7faa14600640 1 -- 172.21.15.46:0/1053763580 wait complete. 2024-04-26T23:53:35.496 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-26T23:53:36.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.712+0000 7ff7ee600640 1 Processor -- start 2024-04-26T23:53:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.713+0000 7ff7ee600640 1 -- start start 2024-04-26T23:53:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.714+0000 7ff7ee600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 0x7ff7e8161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.714+0000 7ff7ee600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7e80732c0 0x7ff7e8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.714+0000 7ff7ee600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7e8161940 con 0x7ff7e806e510 2024-04-26T23:53:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.714+0000 7ff7ee600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e8071f80 con 0x7ff7e80732c0 2024-04-26T23:53:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.714+0000 7ff7ee600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8072260 con 0x7ff7e8160bd0 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7e80732c0 0x7ff7e8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff7e806e510 0x7ff7e806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39678/0 (socket says 172.21.15.46:39678) 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7400640 1 -- 172.21.15.46:0/3880313959 learned_addr learned my addr 172.21.15.46:0/3880313959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7e00640 1 -- 172.21.15.46:0/3880313959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 msgr2=0x7ff7e8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e6a00640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 0x7ff7e8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7e00640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 0x7ff7e8161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e6000640 1 -- 172.21.15.46:0/3880313959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354829356 0 0) 0x7ff7e8161940 con 0x7ff7e806e510 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7e00640 1 -- 172.21.15.46:0/3880313959 >> v1:172.21.15.46:6789/0 conn(0x7ff7e806e510 legacy=0x7ff7e806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e7e00640 1 -- 172.21.15.46:0/3880313959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e8072540 con 0x7ff7e80732c0 2024-04-26T23:53:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.715+0000 7ff7e6a00640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 0x7ff7e8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.716+0000 7ff7e7e00640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e80732c0 0x7ff7e8073750 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00097a0 tx=0x7ff7e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dea475d883ee075 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.716+0000 7ff7e6000640 1 -- 172.21.15.46:0/3880313959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0037070 con 0x7ff7e80732c0 2024-04-26T23:53:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7e6000640 1 -- 172.21.15.46:0/3880313959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e0032440 con 0x7ff7e80732c0 2024-04-26T23:53:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7e6000640 1 -- 172.21.15.46:0/3880313959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0032810 con 0x7ff7e80732c0 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 -- 172.21.15.46:0/3880313959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e80732c0 msgr2=0x7ff7e8073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e80732c0 0x7ff7e8073750 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00097a0 tx=0x7ff7e002a180 comp rx=0 tx=0).stop 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 -- 172.21.15.46:0/3880313959 shutdown_connections 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e80732c0 0x7ff7e8073750 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3880313959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e8160bd0 0x7ff7e8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 -- 172.21.15.46:0/3880313959 >> 172.21.15.46:0/3880313959 conn(0x7ff7e806c530 msgr2=0x7ff7e806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 -- 172.21.15.46:0/3880313959 shutdown_connections 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.717+0000 7ff7ee600640 1 -- 172.21.15.46:0/3880313959 wait complete. 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.719+0000 7ff7ee600640 1 Processor -- start 2024-04-26T23:53:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.719+0000 7ff7ee600640 1 -- start start 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e80732c0 0x7ff7e8145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8160bd0 0x7ff7e8145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81616f0 con 0x7ff7e806e510 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8161570 con 0x7ff7e8160bd0 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.720+0000 7ff7ee600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8072d40 con 0x7ff7e80732c0 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53568/0 (socket says 172.21.15.46:53568) 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 -- 172.21.15.46:0/3976569355 learned_addr learned my addr 172.21.15.46:0/3976569355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e6a00640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e80732c0 0x7ff7e8145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 -- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e80732c0 msgr2=0x7ff7e8145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7e00640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8160bd0 0x7ff7e8145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e80732c0 0x7ff7e8145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 -- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8160bd0 msgr2=0x7ff7e8145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8160bd0 0x7ff7e8145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.721+0000 7ff7e7400640 1 -- 172.21.15.46:0/3976569355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e81fc7d0 con 0x7ff7e806e510 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7e7400640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c430 tx=0x7ff7d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00074e0 con 0x7ff7e806e510 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d00047e0 con 0x7ff7e806e510 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0004bd0 con 0x7ff7e806e510 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e81fcab0 con 0x7ff7e806e510 2024-04-26T23:53:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.722+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81fd070 con 0x7ff7e806e510 2024-04-26T23:53:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.723+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e8069530 con 0x7ff7e806e510 2024-04-26T23:53:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.724+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7d0011070 con 0x7ff7e806e510 2024-04-26T23:53:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.725+0000 7ff7df400640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.725+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7ff7d008b9d0 con 0x7ff7e806e510 2024-04-26T23:53:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.726+0000 7ff7e6a00640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.727+0000 7ff7e6a00640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 0x7ff7c406edb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80b3550 tx=0x7ff7d80084d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:36.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.729+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7d0059c60 con 0x7ff7e806e510 2024-04-26T23:53:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.978+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7e806fb00 con 0x7ff7e806e510 2024-04-26T23:53:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.979+0000 7ff7df400640 1 -- 172.21.15.46:0/3976569355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7ff7d0059560 con 0x7ff7e806e510 2024-04-26T23:53:36.982 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:36.982 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 msgr2=0x7ff7c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 0x7ff7c406edb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80b3550 tx=0x7ff7d80084d0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 msgr2=0x7ff7e813c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c430 tx=0x7ff7d000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 shutdown_connections 2024-04-26T23:53:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8160bd0 0x7ff7e8145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7e80732c0 0x7ff7e8145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 --2- 172.21.15.46:0/3976569355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7e806e510 0x7ff7e813c700 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.982+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 >> 172.21.15.46:0/3976569355 conn(0x7ff7e806c530 msgr2=0x7ff7e815d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.983+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 shutdown_connections 2024-04-26T23:53:36.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:36.983+0000 7ff7ee600640 1 -- 172.21.15.46:0/3976569355 wait complete. 2024-04-26T23:53:37.005 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-26T23:53:37.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:53:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.212+0000 7fb27d000640 1 Processor -- start 2024-04-26T23:53:37.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.212+0000 7fb27d000640 1 -- start start 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.214+0000 7fb27d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 0x7fb27806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.214+0000 7fb27d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27806ee30 con 0x7fb278160bd0 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.214+0000 7fb27d000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb278161940 con 0x7fb2780732c0 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.214+0000 7fb27d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb278071f80 con 0x7fb27806e510 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.215+0000 7fb275400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb278160bd0 0x7fb278161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39688/0 (socket says 172.21.15.46:39688) 2024-04-26T23:53:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.215+0000 7fb275400640 1 -- 172.21.15.46:0/1688399223 learned_addr learned my addr 172.21.15.46:0/1688399223 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.215+0000 7fb275e00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 0x7fb27806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.215+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416778651 0 0) 0x7fb27806ee30 con 0x7fb278160bd0 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.215+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb25c003520 con 0x7fb278160bd0 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677532025 0 0) 0x7fb25c003520 con 0x7fb278160bd0 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 msgr2=0x7fb27806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 0x7fb27806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 >> v1:172.21.15.142:6789/0 conn(0x7fb2780732c0 legacy=0x7fb278073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb278072260 con 0x7fb278160bd0 2024-04-26T23:53:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb264001670 con 0x7fb278160bd0 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 >> v1:172.21.15.46:6789/0 conn(0x7fb278160bd0 legacy=0x7fb278161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25c0039e0 0x7fb25c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 0x7fb25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb278071f80 con 0x7fb25c0039e0 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c00ba70 con 0x7fb25c004800 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c00bc70 con 0x7fb27806e510 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.216+0000 7fb275e00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25c0039e0 0x7fb25c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275e00640 1 -- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 msgr2=0x7fb25c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275e00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb27806e510 0x7fb25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275e00640 1 -- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25c004800 msgr2=0x7fb25c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275400640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275e00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275e00640 1 -- 172.21.15.46:0/1688399223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c003520 con 0x7fb25c0039e0 2024-04-26T23:53:37.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.217+0000 7fb275400640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb275e00640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25c0039e0 0x7fb25c003e00 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb268009a90 tx=0x7fb25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74fa743ffa19300 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268036070 con 0x7fb25c0039e0 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268031400 con 0x7fb25c0039e0 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb274a00640 1 -- 172.21.15.46:0/1688399223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268009690 con 0x7fb25c0039e0 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb27d000640 1 -- 172.21.15.46:0/1688399223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25c0039e0 msgr2=0x7fb25c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb27d000640 1 --2- 172.21.15.46:0/1688399223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25c0039e0 0x7fb25c003e00 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb268009a90 tx=0x7fb25c005830 comp rx=0 tx=0).stop 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.218+0000 7fb276800640 1 -- 172.21.15.46:0/1688399223 reap_dead start 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.219+0000 7fb27d000640 1 -- 172.21.15.46:0/1688399223 shutdown_connections 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.219+0000 7fb27d000640 1 -- 172.21.15.46:0/1688399223 >> 172.21.15.46:0/1688399223 conn(0x7fb27806c530 msgr2=0x7fb27806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.219+0000 7fb27d000640 1 -- 172.21.15.46:0/1688399223 shutdown_connections 2024-04-26T23:53:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.219+0000 7fb27d000640 1 -- 172.21.15.46:0/1688399223 wait complete. 2024-04-26T23:53:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.220+0000 7fb27d000640 1 Processor -- start 2024-04-26T23:53:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.221+0000 7fb27d000640 1 -- start start 2024-04-26T23:53:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.221+0000 7fb27d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb25c0039e0 0x7fb27813e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.221+0000 7fb27d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2780aa4d0 0x7fb2780aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.221+0000 7fb27d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb27d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb278072d40 con 0x7fb27814a0b0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb27d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb278072820 con 0x7fb2780aa4d0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb27d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb278072ec0 con 0x7fb25c0039e0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53584/0 (socket says 172.21.15.46:53584) 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 -- 172.21.15.46:0/1287462632 learned_addr learned my addr 172.21.15.46:0/1287462632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb275400640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2780aa4d0 0x7fb2780aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb275e00640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb25c0039e0 0x7fb27813e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 -- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb25c0039e0 msgr2=0x7fb27813e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb25c0039e0 0x7fb27813e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 -- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2780aa4d0 msgr2=0x7fb2780aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2780aa4d0 0x7fb2780aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.222+0000 7fb276800640 1 -- 172.21.15.46:0/1287462632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27814bec0 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb276800640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cfed0 tx=0x7fb27000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270017070 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb270005d40 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000a850 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27814ae20 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.223+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27814b3e0 con 0x7fb27814a0b0 2024-04-26T23:53:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.224+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278069530 con 0x7fb27814a0b0 2024-04-26T23:53:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.225+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb27000a9f0 con 0x7fb27814a0b0 2024-04-26T23:53:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.226+0000 7fb26e000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.226+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fb27008acc0 con 0x7fb27814a0b0 2024-04-26T23:53:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.227+0000 7fb275e00640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.228+0000 7fb275e00640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 0x7fb25806ede0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb268033000 tx=0x7fb268008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.231+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb270091080 con 0x7fb27814a0b0 2024-04-26T23:53:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.512+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb27806fb00 con 0x7fb27814a0b0 2024-04-26T23:53:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.512+0000 7fb26e000640 1 -- 172.21.15.46:0/1287462632 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1603 (secure 0 0 0) 0x7fb270058ec0 con 0x7fb27814a0b0 2024-04-26T23:53:37.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:37.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1008,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":161,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076952021,"bytes_used":2624520192,"bytes_avail":770469593088,"bytes_total":773094113280,"write_bytes_sec":25510953,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":127,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"c","status":"up:active","gid":10287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":37,"modified":"2024-04-26T23:53:36.262494+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 msgr2=0x7fb25806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 0x7fb25806ede0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb268033000 tx=0x7fb268008030 comp rx=0 tx=0).stop 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 msgr2=0x7fb27814a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cfed0 tx=0x7fb27000cb90 comp rx=0 tx=0).stop 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 shutdown_connections 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb27814a0b0 0x7fb27814a510 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2780aa4d0 0x7fb2780aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 --2- 172.21.15.46:0/1287462632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb25c0039e0 0x7fb27813e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 >> 172.21.15.46:0/1287462632 conn(0x7fb27806c530 msgr2=0x7fb27815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.515+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 shutdown_connections 2024-04-26T23:53:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:37.516+0000 7fb27d000640 1 -- 172.21.15.46:0/1287462632 wait complete. 2024-04-26T23:53:37.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-26T23:53:37.537 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:53:37.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:53:37.560 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:53:37.560 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'b', 'd', 'a'} 2024-04-26T23:53:37.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-26T23:53:37.817 INFO:teuthology.orchestra.run.smithi046.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-26T23:53:37.817 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-26T23:53:37.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.030+0000 7f517ea00640 1 Processor -- start 2024-04-26T23:53:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.031+0000 7f517ea00640 1 -- start start 2024-04-26T23:53:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51780b4000 0x7f51780d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51780d11f0 con 0x7f517815f4e0 2024-04-26T23:53:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51780d5f20 con 0x7f51780d2130 2024-04-26T23:53:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f51780d0ce0 con 0x7f51780b4000 2024-04-26T23:53:38.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517d600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f517815f4e0 0x7f51780d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39698/0 (socket says 172.21.15.46:39698) 2024-04-26T23:53:38.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.032+0000 7f517d600640 1 -- 172.21.15.46:0/3242630760 learned_addr learned my addr 172.21.15.46:0/3242630760 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f517e000640 1 --2- 172.21.15.46:0/3242630760 >> v2:172.21.15.142:3300/0 conn(0x7f51780b4000 0x7f51780d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747167515 0 0) 0x7f51780d11f0 con 0x7f517815f4e0 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5160003520 con 0x7f517815f4e0 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4246853545 0 0) 0x7f5160003520 con 0x7f517815f4e0 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 >> v2:172.21.15.142:3300/0 conn(0x7f51780b4000 msgr2=0x7f51780d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 --2- 172.21.15.46:0/3242630760 >> v2:172.21.15.142:3300/0 conn(0x7f51780b4000 0x7f51780d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 >> v1:172.21.15.142:6789/0 conn(0x7f51780d2130 legacy=0x7f51780b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51780d0f10 con 0x7f517815f4e0 2024-04-26T23:53:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f516c001670 con 0x7f517815f4e0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.033+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 >> v1:172.21.15.46:6789/0 conn(0x7f517815f4e0 legacy=0x7f51780d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600039e0 0x7f5160003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5160004800 0x7f5160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 0x7f516000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f516000c090 con 0x7f51600051c0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f516000c210 con 0x7f51600039e0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f516000c410 con 0x7f5160004800 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f517e000640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 0x7f516000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f517d600640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600039e0 0x7f5160003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.034+0000 7f517cc00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5160004800 0x7f5160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.035+0000 7f517e000640 1 -- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5160004800 msgr2=0x7f5160004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.035+0000 7f517e000640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5160004800 0x7f5160004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.035+0000 7f517e000640 1 -- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600039e0 msgr2=0x7f5160003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.035+0000 7f517e000640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600039e0 0x7f5160003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.035+0000 7f517e000640 1 -- 172.21.15.46:0/3242630760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5160003520 con 0x7f51600051c0 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.036+0000 7f517e000640 1 -- 172.21.15.46:0/3242630760 reap_dead start 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.036+0000 7f517e000640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 0x7f516000b9c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f5170009ac0 tx=0x7f5160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590ed2c7d426a7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.036+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170009690 con 0x7f51600051c0 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.036+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5170004020 con 0x7f51600051c0 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.036+0000 7f5177e00640 1 -- 172.21.15.46:0/3242630760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170004410 con 0x7f51600051c0 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 -- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 msgr2=0x7f516000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 0x7f516000b9c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f5170009ac0 tx=0x7f5160005830 comp rx=0 tx=0).stop 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 -- 172.21.15.46:0/3242630760 shutdown_connections 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 --2- 172.21.15.46:0/3242630760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51600051c0 0x7f516000b9c0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 -- 172.21.15.46:0/3242630760 >> 172.21.15.46:0/3242630760 conn(0x7f51780d5010 msgr2=0x7f51780d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 -- 172.21.15.46:0/3242630760 shutdown_connections 2024-04-26T23:53:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.037+0000 7f517ea00640 1 -- 172.21.15.46:0/3242630760 wait complete. 2024-04-26T23:53:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.039+0000 7f517ea00640 1 Processor -- start 2024-04-26T23:53:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.039+0000 7f517ea00640 1 -- start start 2024-04-26T23:53:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 0x7f5178161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51780ca350 0x7f51780ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51780b1d60 con 0x7f51781580d0 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51780d2990 con 0x7f51600051c0 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f517815f8c0 con 0x7f51780ca350 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53606/0 (socket says 172.21.15.46:53606) 2024-04-26T23:53:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.040+0000 7f517cc00640 1 -- 172.21.15.46:0/3464475767 learned_addr learned my addr 172.21.15.46:0/3464475767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517d600640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 0x7f5178161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 -- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51780ca350 msgr2=0x7f51780ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51780ca350 0x7f51780ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 -- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 msgr2=0x7f5178161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 0x7f5178161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 -- 172.21.15.46:0/3464475767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5178158a90 con 0x7f51781580d0 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517d600640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 0x7f5178161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517cc00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f516800e820 tx=0x7f516800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168003b80 con 0x7f51781580d0 2024-04-26T23:53:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168018070 con 0x7f51781580d0 2024-04-26T23:53:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51781f4f20 con 0x7f51781580d0 2024-04-26T23:53:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680050d0 con 0x7f51781580d0 2024-04-26T23:53:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.041+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781f54e0 con 0x7f51781580d0 2024-04-26T23:53:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.043+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51780693b0 con 0x7f51781580d0 2024-04-26T23:53:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.044+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5168005270 con 0x7f51781580d0 2024-04-26T23:53:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.046+0000 7f5175600640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 0x7f515c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.046+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7f516808bd80 con 0x7f51781580d0 2024-04-26T23:53:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.047+0000 7f517d600640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 0x7f515c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.048+0000 7f517d600640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 0x7f515c06ede0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f516c0017d0 tx=0x7f516c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.051+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5178158a90 con 0x7f51781580d0 2024-04-26T23:53:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.300+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51780d33d0 con 0x7f51781580d0 2024-04-26T23:53:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.300+0000 7f5175600640 1 -- 172.21.15.46:0/3464475767 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7f516801a310 con 0x7f51781580d0 2024-04-26T23:53:38.303 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:38.303 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 msgr2=0x7f515c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 0x7f515c06ede0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f516c0017d0 tx=0x7f516c009040 comp rx=0 tx=0).stop 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 msgr2=0x7f5178158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f516800e820 tx=0x7f516800ece0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 shutdown_connections 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f515c06c920 0x7f515c06ede0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51780ca350 0x7f51780ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51781580d0 0x7f5178158550 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 --2- 172.21.15.46:0/3464475767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51600051c0 0x7f5178161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.303+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 >> 172.21.15.46:0/3464475767 conn(0x7f51780d5010 msgr2=0x7f51780b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.304+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 shutdown_connections 2024-04-26T23:53:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:38.304+0000 7f517ea00640 1 -- 172.21.15.46:0/3464475767 wait complete. 2024-04-26T23:53:38.324 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T23:53:43.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:43.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:43.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:43.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.540+0000 7fa22e400640 1 Processor -- start 2024-04-26T23:53:43.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.540+0000 7fa22e400640 1 -- start start 2024-04-26T23:53:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2281466b0 0x7fa22806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa228073ad0 con 0x7fa228160910 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22e400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa228146a90 con 0x7fa2280720e0 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa228073d70 con 0x7fa2281466b0 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2281466b0 0x7fa22806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa228160910 0x7fa2280cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60722/0 (socket says 172.21.15.46:60722) 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.542+0000 7fa22da00640 1 -- 172.21.15.46:0/3884905552 learned_addr learned my addr 172.21.15.46:0/3884905552 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:43.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.543+0000 7fa227e00640 1 -- 172.21.15.46:0/3884905552 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221555145 0 0) 0x7fa228073ad0 con 0x7fa228160910 2024-04-26T23:53:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.543+0000 7fa22d000640 1 -- 172.21.15.46:0/3884905552 >> v1:172.21.15.142:6789/0 conn(0x7fa2280720e0 legacy=0x7fa2280cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.543+0000 7fa22d000640 1 -- 172.21.15.46:0/3884905552 >> v1:172.21.15.46:6789/0 conn(0x7fa228160910 legacy=0x7fa2280cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.543+0000 7fa22d000640 1 -- 172.21.15.46:0/3884905552 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa228074050 con 0x7fa2281466b0 2024-04-26T23:53:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.544+0000 7fa22d000640 1 --2- 172.21.15.46:0/3884905552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2281466b0 0x7fa22806de80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fa2180097d0 tx=0x7fa21802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e78a7890c9e3858 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.545+0000 7fa227e00640 1 -- 172.21.15.46:0/3884905552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218037070 con 0x7fa2281466b0 2024-04-26T23:53:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.545+0000 7fa227e00640 1 -- 172.21.15.46:0/3884905552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa218032440 con 0x7fa2281466b0 2024-04-26T23:53:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.545+0000 7fa227e00640 1 -- 172.21.15.46:0/3884905552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218032810 con 0x7fa2281466b0 2024-04-26T23:53:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.545+0000 7fa22e400640 1 -- 172.21.15.46:0/3884905552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2281466b0 msgr2=0x7fa22806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.545+0000 7fa22e400640 1 --2- 172.21.15.46:0/3884905552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2281466b0 0x7fa22806de80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fa2180097d0 tx=0x7fa21802a270 comp rx=0 tx=0).stop 2024-04-26T23:53:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.546+0000 7fa22e400640 1 -- 172.21.15.46:0/3884905552 shutdown_connections 2024-04-26T23:53:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.546+0000 7fa22e400640 1 --2- 172.21.15.46:0/3884905552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2281466b0 0x7fa22806de80 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.546+0000 7fa22e400640 1 -- 172.21.15.46:0/3884905552 >> 172.21.15.46:0/3884905552 conn(0x7fa2280b26d0 msgr2=0x7fa2280b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.546+0000 7fa22e400640 1 -- 172.21.15.46:0/3884905552 shutdown_connections 2024-04-26T23:53:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.546+0000 7fa22e400640 1 -- 172.21.15.46:0/3884905552 wait complete. 2024-04-26T23:53:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.548+0000 7fa22e400640 1 Processor -- start 2024-04-26T23:53:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.548+0000 7fa22e400640 1 -- start start 2024-04-26T23:53:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2280720e0 0x7fa2281e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 0x7fa228141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa22806e3c0 con 0x7fa228160910 2024-04-26T23:53:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2280cb470 con 0x7fa2281466b0 2024-04-26T23:53:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2280cc0a0 con 0x7fa2280720e0 2024-04-26T23:53:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35772/0 (socket says 172.21.15.46:35772) 2024-04-26T23:53:43.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa22da00640 1 -- 172.21.15.46:0/2205457460 learned_addr learned my addr 172.21.15.46:0/2205457460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.549+0000 7fa21fe00640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 0x7fa228141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 -- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2280720e0 msgr2=0x7fa2281e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2280720e0 0x7fa2281e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 -- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 msgr2=0x7fa228141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 0x7fa228141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 -- 172.21.15.46:0/2205457460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22814c1c0 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa21fe00640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 0x7fa228141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22da00640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c780 tx=0x7fa22000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220018070 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa22000a640 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22815fac0 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.550+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22000aa30 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.551+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22814c740 con 0x7fa228160910 2024-04-26T23:53:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.552+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2280693b0 con 0x7fa228160910 2024-04-26T23:53:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.553+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa22001b070 con 0x7fa228160910 2024-04-26T23:53:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.554+0000 7fa225600640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 0x7fa20406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.554+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fa22008b480 con 0x7fa228160910 2024-04-26T23:53:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.555+0000 7fa22d000640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 0x7fa20406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.556+0000 7fa22d000640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 0x7fa20406ee00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa218034000 tx=0x7fa21802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.559+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa220059670 con 0x7fa228160910 2024-04-26T23:53:43.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.798+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa228145c20 con 0x7fa228160910 2024-04-26T23:53:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.799+0000 7fa225600640 1 -- 172.21.15.46:0/2205457460 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7fa220058f70 con 0x7fa228160910 2024-04-26T23:53:43.802 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:43.802 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 msgr2=0x7fa20406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 0x7fa20406ee00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa218034000 tx=0x7fa21802c000 comp rx=0 tx=0).stop 2024-04-26T23:53:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 msgr2=0x7fa22814d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c780 tx=0x7fa22000cc40 comp rx=0 tx=0).stop 2024-04-26T23:53:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 shutdown_connections 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa20406c940 0x7fa20406ee00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa228160910 0x7fa22814d220 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2281466b0 0x7fa228141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 --2- 172.21.15.46:0/2205457460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2280720e0 0x7fa2281e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 >> 172.21.15.46:0/2205457460 conn(0x7fa2280b26d0 msgr2=0x7fa2280b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 shutdown_connections 2024-04-26T23:53:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:43.802+0000 7fa22e400640 1 -- 172.21.15.46:0/2205457460 wait complete. 2024-04-26T23:53:43.824 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T23:53:48.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:49.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.052+0000 7fa781600640 1 Processor -- start 2024-04-26T23:53:49.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.052+0000 7fa781600640 1 -- start start 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa781600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa77c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa781600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c071900 con 0x7fa77c0710d0 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa781600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c160fd0 con 0x7fa77c160a30 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa781600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c140c80 con 0x7fa77c0714b0 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa77be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa77c0710d0 0x7fa77c15dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60726/0 (socket says 172.21.15.46:60726) 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.053+0000 7fa77be00640 1 -- 172.21.15.46:0/478388782 learned_addr learned my addr 172.21.15.46:0/478388782 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164101644 0 0) 0x7fa77c071900 con 0x7fa77c0710d0 2024-04-26T23:53:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa760003520 con 0x7fa77c0710d0 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa780c00640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa77c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3621855696 0 0) 0x7fa760003520 con 0x7fa77c0710d0 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 msgr2=0x7fa77c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa77c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 >> v1:172.21.15.142:6789/0 conn(0x7fa77c160a30 legacy=0x7fa77c140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c1612b0 con 0x7fa77c0710d0 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa76c001670 con 0x7fa77c0710d0 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa780c00640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa77c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.054+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 >> v1:172.21.15.46:6789/0 conn(0x7fa77c0710d0 legacy=0x7fa77c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7600039e0 0x7fa760003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa7600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c140c80 con 0x7fa760004800 2024-04-26T23:53:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa76000bac0 con 0x7fa7600039e0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa76000bcc0 con 0x7fa77c0714b0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa773400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa77be00640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7600039e0 0x7fa760003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa780c00640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa7600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa773400640 1 -- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 msgr2=0x7fa7600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.055+0000 7fa773400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa77c0714b0 0x7fa7600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.056+0000 7fa773400640 1 -- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7600039e0 msgr2=0x7fa760003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.056+0000 7fa773400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7600039e0 0x7fa760003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.056+0000 7fa773400640 1 -- 172.21.15.46:0/478388782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa760003520 con 0x7fa760004800 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa773400640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa760004800 0x7fa760004c80 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa764001750 tx=0x7fa760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff468b3f63a996d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764001370 con 0x7fa760004800 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7640052d0 con 0x7fa760004800 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa77b400640 1 -- 172.21.15.46:0/478388782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76402abe0 con 0x7fa760004800 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa781600640 1 -- 172.21.15.46:0/478388782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa760004800 msgr2=0x7fa760004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa781600640 1 --2- 172.21.15.46:0/478388782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa760004800 0x7fa760004c80 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa764001750 tx=0x7fa760005830 comp rx=0 tx=0).stop 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.057+0000 7fa780c00640 1 -- 172.21.15.46:0/478388782 reap_dead start 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.058+0000 7fa781600640 1 -- 172.21.15.46:0/478388782 shutdown_connections 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.058+0000 7fa781600640 1 -- 172.21.15.46:0/478388782 >> 172.21.15.46:0/478388782 conn(0x7fa77c0d1c70 msgr2=0x7fa77c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.058+0000 7fa781600640 1 -- 172.21.15.46:0/478388782 shutdown_connections 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.058+0000 7fa781600640 1 -- 172.21.15.46:0/478388782 wait complete. 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.059+0000 7fa781600640 1 Processor -- start 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.060+0000 7fa781600640 1 -- start start 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.060+0000 7fa781600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 0x7fa77c15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.060+0000 7fa781600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.060+0000 7fa781600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa77c13f770 0x7fa77c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa781600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c072f50 con 0x7fa77c0b49b0 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa781600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c161870 con 0x7fa77c13f770 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa781600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c06c350 con 0x7fa760004800 2024-04-26T23:53:49.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35786/0 (socket says 172.21.15.46:35786) 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 -- 172.21.15.46:0/281913671 learned_addr learned my addr 172.21.15.46:0/281913671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa77be00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 0x7fa77c15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 -- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 msgr2=0x7fa77c15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa780c00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa77c13f770 0x7fa77c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 0x7fa77c15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 -- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa77c13f770 msgr2=0x7fa77c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa77c13f770 0x7fa77c06f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa773400640 1 -- 172.21.15.46:0/281913671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c1f3220 con 0x7fa77c0b49b0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.061+0000 7fa77be00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 0x7fa77c15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa773400640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fa764008fd0 tx=0x7fa764001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764040020 con 0x7fa77c0b49b0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa764007680 con 0x7fa77c0b49b0 2024-04-26T23:53:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764007a30 con 0x7fa77c0b49b0 2024-04-26T23:53:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c1f3500 con 0x7fa77c0b49b0 2024-04-26T23:53:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.062+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c13fc20 con 0x7fa77c0b49b0 2024-04-26T23:53:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.063+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c069ac0 con 0x7fa77c0b49b0 2024-04-26T23:53:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.064+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa764043020 con 0x7fa77c0b49b0 2024-04-26T23:53:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.065+0000 7fa778c00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 0x7fa75806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.065+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 6101+0+0 (secure 0 0 0) 0x7fa7640abd20 con 0x7fa77c0b49b0 2024-04-26T23:53:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.066+0000 7fa77be00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 0x7fa75806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.067+0000 7fa77be00640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 0x7fa75806edb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa77c1606f0 tx=0x7fa76c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.071+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa76404a030 con 0x7fa77c0b49b0 2024-04-26T23:53:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.324+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa77c0b62b0 con 0x7fa77c0b49b0 2024-04-26T23:53:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.324+0000 7fa778c00640 1 -- 172.21.15.46:0/281913671 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5200 (secure 0 0 0) 0x7fa764079f20 con 0x7fa77c0b49b0 2024-04-26T23:53:49.327 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:49.327 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":127,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:52:47.762256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10287":{"gid":10287,"name":"c","rank":0,"incarnation":123,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1585151449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1585151449},{"type":"v1","addr":"172.21.15.46:6835","nonce":1585151449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10287,"qdb_cluster":[10287]},"id":11}]} 2024-04-26T23:53:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 127 2024-04-26T23:53:49.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 msgr2=0x7fa75806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 0x7fa75806edb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa77c1606f0 tx=0x7fa76c009040 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 msgr2=0x7fa77c1f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fa764008fd0 tx=0x7fa764001120 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 shutdown_connections 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa75806c8f0 0x7fa75806edb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa77c13f770 0x7fa77c06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77c0b49b0 0x7fa77c1f2ab0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 --2- 172.21.15.46:0/281913671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa760004800 0x7fa77c15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 >> 172.21.15.46:0/281913671 conn(0x7fa77c0d1c70 msgr2=0x7fa77c073370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.327+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 shutdown_connections 2024-04-26T23:53:49.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:49.328+0000 7fa781600640 1 -- 172.21.15.46:0/281913671 wait complete. 2024-04-26T23:53:49.351 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T23:53:54.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:54.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.572+0000 7fd262a00640 1 Processor -- start 2024-04-26T23:53:54.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.572+0000 7fd262a00640 1 -- start start 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd262a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd25c1466b0 0x7fd25c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd262a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd262a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25c073ad0 con 0x7fd25c0720e0 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd262a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd25c146a90 con 0x7fd25c1466b0 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd262a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c073d70 con 0x7fd25c160910 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.574+0000 7fd261600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd25c1466b0 0x7fd25c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd25c1466b0 0x7fd25c06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45300/0 (socket says 172.21.15.46:45300) 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 -- 172.21.15.46:0/994317260 learned_addr learned my addr 172.21.15.46:0/994317260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 -- 172.21.15.46:0/994317260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 msgr2=0x7fd25c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:54.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 --2- 172.21.15.46:0/994317260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 -- 172.21.15.46:0/994317260 >> v1:172.21.15.46:6789/0 conn(0x7fd25c0720e0 legacy=0x7fd25c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:54.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.575+0000 7fd261600640 1 -- 172.21.15.46:0/994317260 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c074050 con 0x7fd25c1466b0 2024-04-26T23:53:54.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.576+0000 7fd261600640 1 --2- 172.21.15.46:0/994317260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c1466b0 0x7fd25c06de80 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098a0 tx=0x7fd25002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8af6cb8677d60e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.576+0000 7fd25be00640 1 -- 172.21.15.46:0/994317260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250037070 con 0x7fd25c1466b0 2024-04-26T23:53:54.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.576+0000 7fd25be00640 1 -- 172.21.15.46:0/994317260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250032440 con 0x7fd25c1466b0 2024-04-26T23:53:54.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd25be00640 1 -- 172.21.15.46:0/994317260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250032810 con 0x7fd25c1466b0 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 -- 172.21.15.46:0/994317260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c1466b0 msgr2=0x7fd25c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 --2- 172.21.15.46:0/994317260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c1466b0 0x7fd25c06de80 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098a0 tx=0x7fd25002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 -- 172.21.15.46:0/994317260 shutdown_connections 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 --2- 172.21.15.46:0/994317260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 --2- 172.21.15.46:0/994317260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c1466b0 0x7fd25c06de80 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 -- 172.21.15.46:0/994317260 >> 172.21.15.46:0/994317260 conn(0x7fd25c0b26d0 msgr2=0x7fd25c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 -- 172.21.15.46:0/994317260 shutdown_connections 2024-04-26T23:53:54.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.577+0000 7fd262a00640 1 -- 172.21.15.46:0/994317260 wait complete. 2024-04-26T23:53:54.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.579+0000 7fd262a00640 1 Processor -- start 2024-04-26T23:53:54.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.579+0000 7fd262a00640 1 -- start start 2024-04-26T23:53:54.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25c1466b0 0x7fd25c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c06e3c0 con 0x7fd25c0720e0 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0cb470 con 0x7fd25c1466b0 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd262a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0cb990 con 0x7fd25c160910 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.580+0000 7fd261600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45312/0 (socket says 172.21.15.46:45312) 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 -- 172.21.15.46:0/1730731655 learned_addr learned my addr 172.21.15.46:0/1730731655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 -- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 msgr2=0x7fd25c13d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 -- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25c1466b0 msgr2=0x7fd25c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25c1466b0 0x7fd25c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 -- 172.21.15.46:0/1730731655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c0b56e0 con 0x7fd25c0720e0 2024-04-26T23:53:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.581+0000 7fd261600640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098a0 tx=0x7fd2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.582+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250041020 con 0x7fd25c0720e0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.582+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250030970 con 0x7fd25c0720e0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.582+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250037040 con 0x7fd25c0720e0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.582+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c0b4e40 con 0x7fd25c0720e0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.582+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c0b5400 con 0x7fd25c0720e0 2024-04-26T23:53:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.583+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c0693b0 con 0x7fd25c0720e0 2024-04-26T23:53:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.584+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd250044020 con 0x7fd25c0720e0 2024-04-26T23:53:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.585+0000 7fd259600640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 0x7fd23c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.585+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fd2500ac730 con 0x7fd25c0720e0 2024-04-26T23:53:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.586+0000 7fd260c00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 0x7fd23c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.587+0000 7fd260c00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 0x7fd23c06edb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd248000f60 tx=0x7fd2480079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.590+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd25004b030 con 0x7fd25c0720e0 2024-04-26T23:53:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.837+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd25c0c7180 con 0x7fd25c0720e0 2024-04-26T23:53:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.838+0000 7fd259600640 1 -- 172.21.15.46:0/1730731655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) ==== 95+0+4366 (secure 0 0 0) 0x7fd25007a970 con 0x7fd25c0720e0 2024-04-26T23:53:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:53:54.841 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":128,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:53.908480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":11}]} 2024-04-26T23:53:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 128 2024-04-26T23:53:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 msgr2=0x7fd23c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 0x7fd23c06edb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd248000f60 tx=0x7fd2480079b0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 msgr2=0x7fd25c0d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098a0 tx=0x7fd2500046c0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 shutdown_connections 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23c06c8f0 0x7fd23c06edb0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd25c160910 0x7fd25c13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd25c1466b0 0x7fd25c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 --2- 172.21.15.46:0/1730731655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd25c0720e0 0x7fd25c0d0250 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 >> 172.21.15.46:0/1730731655 conn(0x7fd25c0b26d0 msgr2=0x7fd25c0ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 shutdown_connections 2024-04-26T23:53:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:53:54.841+0000 7fd262a00640 1 -- 172.21.15.46:0/1730731655 wait complete. 2024-04-26T23:53:54.863 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T23:53:59.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.070+0000 7fe46d400640 1 Processor -- start 2024-04-26T23:54:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.071+0000 7fe46d400640 1 -- start start 2024-04-26T23:54:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46815f4e0 0x7fe4680d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4680b4000 0x7fe4680d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4680d1f60 0x7fe4680cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe4680d2990 con 0x7fe4680d1f60 2024-04-26T23:54:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe4680d11f0 con 0x7fe4680b4000 2024-04-26T23:54:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.072+0000 7fe46d400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680d5f20 con 0x7fe46815f4e0 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4680d1f60 0x7fe4680cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4680d1f60 0x7fe4680cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45328/0 (socket says 172.21.15.46:45328) 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 -- 172.21.15.46:0/2190294796 learned_addr learned my addr 172.21.15.46:0/2190294796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe467e00640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46815f4e0 0x7fe4680d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe467400640 1 --2- 172.21.15.46:0/2190294796 >> v2:172.21.15.142:3300/0 conn(0x7fe4680b4000 0x7fe4680d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 -- 172.21.15.46:0/2190294796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46815f4e0 msgr2=0x7fe4680d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46815f4e0 0x7fe4680d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 -- 172.21.15.46:0/2190294796 >> v2:172.21.15.142:3300/0 conn(0x7fe4680b4000 msgr2=0x7fe4680d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 --2- 172.21.15.46:0/2190294796 >> v2:172.21.15.142:3300/0 conn(0x7fe4680b4000 0x7fe4680d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.073+0000 7fe46ca00640 1 -- 172.21.15.46:0/2190294796 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4680d0f10 con 0x7fe4680d1f60 2024-04-26T23:54:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.074+0000 7fe46ca00640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4680d1f60 0x7fe4680cfcd0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe46808d3f0 tx=0x7fe46002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab4998cc2cb0aff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe466a00640 1 -- 172.21.15.46:0/2190294796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460037070 con 0x7fe4680d1f60 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe466a00640 1 -- 172.21.15.46:0/2190294796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460032440 con 0x7fe4680d1f60 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe466a00640 1 -- 172.21.15.46:0/2190294796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460032810 con 0x7fe4680d1f60 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 -- 172.21.15.46:0/2190294796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4680d1f60 msgr2=0x7fe4680cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4680d1f60 0x7fe4680cfcd0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe46808d3f0 tx=0x7fe46002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 -- 172.21.15.46:0/2190294796 shutdown_connections 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4680d1f60 0x7fe4680cfcd0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 --2- 172.21.15.46:0/2190294796 >> v2:172.21.15.142:3300/0 conn(0x7fe4680b4000 0x7fe4680d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 --2- 172.21.15.46:0/2190294796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46815f4e0 0x7fe4680d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.075+0000 7fe46d400640 1 -- 172.21.15.46:0/2190294796 >> 172.21.15.46:0/2190294796 conn(0x7fe4680d5010 msgr2=0x7fe4680d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.076+0000 7fe46d400640 1 -- 172.21.15.46:0/2190294796 shutdown_connections 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.076+0000 7fe46d400640 1 -- 172.21.15.46:0/2190294796 wait complete. 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.077+0000 7fe46d400640 1 Processor -- start 2024-04-26T23:54:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.077+0000 7fe46d400640 1 -- start start 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4680b4000 0x7fe4681554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4680d1f60 0x7fe4680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe46815f8c0 con 0x7fe46815f4e0 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680b1d60 con 0x7fe4680b4000 2024-04-26T23:54:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.078+0000 7fe46d400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680d0210 con 0x7fe4680d1f60 2024-04-26T23:54:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45332/0 (socket says 172.21.15.46:45332) 2024-04-26T23:54:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 -- 172.21.15.46:0/3794321458 learned_addr learned my addr 172.21.15.46:0/3794321458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe467e00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4680b4000 0x7fe4681554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe467400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4680d1f60 0x7fe4680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 -- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4680d1f60 msgr2=0x7fe4680b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4680d1f60 0x7fe4680b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 -- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4680b4000 msgr2=0x7fe4681554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4680b4000 0x7fe4681554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 -- 172.21.15.46:0/3794321458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4680c9380 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.079+0000 7fe46ca00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fe46002a220 tx=0x7fe4600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.080+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460037040 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.080+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460042070 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.080+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460030920 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.080+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4680c9620 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.080+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4680c9be0 con 0x7fe46815f4e0 2024-04-26T23:54:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.081+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe468069e70 con 0x7fe46815f4e0 2024-04-26T23:54:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.082+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe460030ac0 con 0x7fe46815f4e0 2024-04-26T23:54:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.083+0000 7fe45fe00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 0x7fe44406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.083+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fe4600ac660 con 0x7fe46815f4e0 2024-04-26T23:54:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.084+0000 7fe467e00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 0x7fe44406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.085+0000 7fe467e00640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 0x7fe44406ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.088+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe468069e70 con 0x7fe46815f4e0 2024-04-26T23:54:00.181 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:54:00.178+0000 7f91b4c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:54:00.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.339+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4680693b0 con 0x7fe46815f4e0 2024-04-26T23:54:00.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.340+0000 7fe45fe00640 1 -- 172.21.15.46:0/3794321458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7fe46007a250 con 0x7fe46815f4e0 2024-04-26T23:54:00.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:00.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:00.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 msgr2=0x7fe44406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 0x7fe44406ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458008040 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 msgr2=0x7fe4680ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fe46002a220 tx=0x7fe4600046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 shutdown_connections 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe44406c920 0x7fe44406ede0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe46815f4e0 0x7fe4680ce0f0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4680d1f60 0x7fe4680b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 --2- 172.21.15.46:0/3794321458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4680b4000 0x7fe4681554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 >> 172.21.15.46:0/3794321458 conn(0x7fe4680d5010 msgr2=0x7fe4680d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 shutdown_connections 2024-04-26T23:54:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.343+0000 7fe46d400640 1 -- 172.21.15.46:0/3794321458 wait complete. 2024-04-26T23:54:00.364 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-26T23:54:00.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.555+0000 7f6875200640 1 Processor -- start 2024-04-26T23:54:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.555+0000 7f6875200640 1 -- start start 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687015f4e0 0x7f68700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 0x7f68700d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68700d1f60 0x7f68700cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f68700d2990 con 0x7f68700d1f60 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f68700d11f0 con 0x7f68700b4000 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.556+0000 7f6875200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68700d5f20 con 0x7f687015f4e0 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68700d1f60 0x7f68700cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68700d1f60 0x7f68700cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45344/0 (socket says 172.21.15.46:45344) 2024-04-26T23:54:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 -- 172.21.15.46:0/2662995353 learned_addr learned my addr 172.21.15.46:0/2662995353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686f400640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687015f4e0 0x7f68700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686ea00640 1 --2- 172.21.15.46:0/2662995353 >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 0x7f68700d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 -- 172.21.15.46:0/2662995353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687015f4e0 msgr2=0x7f68700d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687015f4e0 0x7f68700d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 -- 172.21.15.46:0/2662995353 >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 msgr2=0x7f68700d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 --2- 172.21.15.46:0/2662995353 >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 0x7f68700d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686fe00640 1 -- 172.21.15.46:0/2662995353 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68700d0f10 con 0x7f68700d1f60 2024-04-26T23:54:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.557+0000 7f686ea00640 1 --2- 172.21.15.46:0/2662995353 >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 0x7f68700d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.558+0000 7f686fe00640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f68700cfcd0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f687008d3f0 tx=0x7f686802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d456876da866d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.559+0000 7f686e000640 1 -- 172.21.15.46:0/2662995353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868036070 con 0x7f68700d1f60 2024-04-26T23:54:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.559+0000 7f686e000640 1 -- 172.21.15.46:0/2662995353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868031600 con 0x7f68700d1f60 2024-04-26T23:54:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.559+0000 7f686e000640 1 -- 172.21.15.46:0/2662995353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68680319d0 con 0x7f68700d1f60 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 -- 172.21.15.46:0/2662995353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 msgr2=0x7f68700cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f68700cfcd0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f687008d3f0 tx=0x7f686802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 -- 172.21.15.46:0/2662995353 shutdown_connections 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f68700cfcd0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 --2- 172.21.15.46:0/2662995353 >> v2:172.21.15.142:3300/0 conn(0x7f68700b4000 0x7f68700d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 --2- 172.21.15.46:0/2662995353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f687015f4e0 0x7f68700d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 -- 172.21.15.46:0/2662995353 >> 172.21.15.46:0/2662995353 conn(0x7f68700d5010 msgr2=0x7f68700d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 -- 172.21.15.46:0/2662995353 shutdown_connections 2024-04-26T23:54:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.560+0000 7f6875200640 1 -- 172.21.15.46:0/2662995353 wait complete. 2024-04-26T23:54:00.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.562+0000 7f6875200640 1 Processor -- start 2024-04-26T23:54:00.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.562+0000 7f6875200640 1 -- start start 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.563+0000 7f6875200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68700b4000 0x7f68700b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f6875200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f6875200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 0x7f6870161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f6875200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f687015f8c0 con 0x7f68700d1f60 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f6875200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68700b1d60 con 0x7f687015f4e0 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f6875200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68700d0210 con 0x7f68700b4000 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45360/0 (socket says 172.21.15.46:45360) 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 -- 172.21.15.46:0/2997420532 learned_addr learned my addr 172.21.15.46:0/2997420532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686fe00640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 0x7f6870161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686f400640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68700b4000 0x7f68700b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 -- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68700b4000 msgr2=0x7f68700b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68700b4000 0x7f68700b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 -- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 msgr2=0x7f6870161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.564+0000 7f686ea00640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 0x7f6870161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f686ea00640 1 -- 172.21.15.46:0/2997420532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687014ede0 con 0x7f68700d1f60 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f686fe00640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 0x7f6870161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f686ea00640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c7f0 tx=0x7f685c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c015070 con 0x7f68700d1f60 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c007a30 con 0x7f68700d1f60 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c007e00 con 0x7f68700d1f60 2024-04-26T23:54:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687014f0c0 con 0x7f68700d1f60 2024-04-26T23:54:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.565+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687014e910 con 0x7f68700d1f60 2024-04-26T23:54:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.567+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68700693b0 con 0x7f68700d1f60 2024-04-26T23:54:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.568+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f685c018070 con 0x7f68700d1f60 2024-04-26T23:54:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.569+0000 7f6867400640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.569+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f685c08bcb0 con 0x7f68700d1f60 2024-04-26T23:54:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.569+0000 7f686f400640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.571+0000 7f686f400640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 0x7f685006edb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6860006fd0 tx=0x7f6860008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:00.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.574+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f685c059e60 con 0x7f68700d1f60 2024-04-26T23:54:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.818+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68700cfd60 con 0x7f68700d1f60 2024-04-26T23:54:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.820+0000 7f6867400640 1 -- 172.21.15.46:0/2997420532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7f685c059760 con 0x7f68700d1f60 2024-04-26T23:54:00.822 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:00.822 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.822+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 msgr2=0x7f685006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.822+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 0x7f685006edb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6860006fd0 tx=0x7f6860008040 comp rx=0 tx=0).stop 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.822+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 msgr2=0x7f687014e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.822+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c7f0 tx=0x7f685c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 shutdown_connections 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687015f4e0 0x7f6870161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68700d1f60 0x7f687014e3d0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 --2- 172.21.15.46:0/2997420532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68700b4000 0x7f68700b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 >> 172.21.15.46:0/2997420532 conn(0x7f68700d5010 msgr2=0x7f68700d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 shutdown_connections 2024-04-26T23:54:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:00.823+0000 7f6875200640 1 -- 172.21.15.46:0/2997420532 wait complete. 2024-04-26T23:54:00.844 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-26T23:54:00.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T23:54:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.050+0000 7fb50a200640 1 Processor -- start 2024-04-26T23:54:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.050+0000 7fb50a200640 1 -- start start 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb50406e780 0x7fb5040b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb50414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb50406eb60 0x7fb50414d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb5040b4940 con 0x7fb50406e780 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb5040b4c20 con 0x7fb50406eb60 2024-04-26T23:54:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb50a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040b4f00 con 0x7fb5040b43c0 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb503400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb50406e780 0x7fb5040b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb503400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb50406e780 0x7fb5040b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45376/0 (socket says 172.21.15.46:45376) 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb503400640 1 -- 172.21.15.46:0/1315702843 learned_addr learned my addr 172.21.15.46:0/1315702843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb503e00640 1 --2- 172.21.15.46:0/1315702843 >> v2:172.21.15.142:3300/0 conn(0x7fb50406eb60 0x7fb50414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.052+0000 7fb502a00640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb50414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.053+0000 7fb503400640 1 -- 172.21.15.46:0/1315702843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 msgr2=0x7fb50414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.053+0000 7fb503400640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb50414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.053+0000 7fb503400640 1 -- 172.21.15.46:0/1315702843 >> v2:172.21.15.142:3300/0 conn(0x7fb50406eb60 msgr2=0x7fb50414d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.053+0000 7fb503400640 1 --2- 172.21.15.46:0/1315702843 >> v2:172.21.15.142:3300/0 conn(0x7fb50406eb60 0x7fb50414d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.053+0000 7fb503400640 1 -- 172.21.15.46:0/1315702843 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040b51e0 con 0x7fb50406e780 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.054+0000 7fb503400640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040b3e80 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fb5040b6550 tx=0x7fb4f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc6f10a1cdd92fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.054+0000 7fb502000640 1 -- 172.21.15.46:0/1315702843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4036070 con 0x7fb50406e780 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.054+0000 7fb502000640 1 -- 172.21.15.46:0/1315702843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f4031600 con 0x7fb50406e780 2024-04-26T23:54:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.054+0000 7fb502000640 1 -- 172.21.15.46:0/1315702843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40319d0 con 0x7fb50406e780 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 -- 172.21.15.46:0/1315702843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 msgr2=0x7fb5040b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040b3e80 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fb5040b6550 tx=0x7fb4f402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 -- 172.21.15.46:0/1315702843 shutdown_connections 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 --2- 172.21.15.46:0/1315702843 >> v2:172.21.15.142:3300/0 conn(0x7fb50406eb60 0x7fb50414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb50414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 --2- 172.21.15.46:0/1315702843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040b3e80 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 -- 172.21.15.46:0/1315702843 >> 172.21.15.46:0/1315702843 conn(0x7fb50406c530 msgr2=0x7fb50406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 -- 172.21.15.46:0/1315702843 shutdown_connections 2024-04-26T23:54:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.055+0000 7fb50a200640 1 -- 172.21.15.46:0/1315702843 wait complete. 2024-04-26T23:54:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.057+0000 7fb50a200640 1 Processor -- start 2024-04-26T23:54:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.057+0000 7fb50a200640 1 -- start start 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50406eb60 0x7fb50413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb5041f9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041672b0 con 0x7fb50406e780 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb504167130 con 0x7fb50406eb60 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb50a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb504167430 con 0x7fb5040b43c0 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45380/0 (socket says 172.21.15.46:45380) 2024-04-26T23:54:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 -- 172.21.15.46:0/3860076779 learned_addr learned my addr 172.21.15.46:0/3860076779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503e00640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb5041f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb502a00640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50406eb60 0x7fb50413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 -- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 msgr2=0x7fb5041f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb5041f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 -- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50406eb60 msgr2=0x7fb50413fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50406eb60 0x7fb50413fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.058+0000 7fb503400640 1 -- 172.21.15.46:0/3860076779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041644e0 con 0x7fb50406e780 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb503400640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000f80 tx=0x7fb4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4036050 con 0x7fb50406e780 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f4042070 con 0x7fb50406e780 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009510 con 0x7fb50406e780 2024-04-26T23:54:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041647c0 con 0x7fb50406e780 2024-04-26T23:54:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.059+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb504164d80 con 0x7fb50406e780 2024-04-26T23:54:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.060+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb504069530 con 0x7fb50406e780 2024-04-26T23:54:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.061+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4f4007c70 con 0x7fb50406e780 2024-04-26T23:54:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.062+0000 7fb4fb400640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.062+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fb4f40acca0 con 0x7fb50406e780 2024-04-26T23:54:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.063+0000 7fb502a00640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.064+0000 7fb502a00640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 0x7fb4e006edb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.067+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4f4045360 con 0x7fb50406e780 2024-04-26T23:54:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.350+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb50414c180 con 0x7fb50406e780 2024-04-26T23:54:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.351+0000 7fb4fb400640 1 -- 172.21.15.46:0/3860076779 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1028 (secure 0 0 0) 0x7fb4f407ae50 con 0x7fb50406e780 2024-04-26T23:54:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:01.354 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1032,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":162,"num_osds":8,"num_up_osds":8,"osd_up_since":1714174620,"num_in_osds":8,"osd_in_since":1714174611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076952021,"bytes_used":2624651264,"bytes_avail":770469462016,"bytes_total":773094113280,"read_bytes_sec":268589,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":131,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"a","status":"up:active","gid":10284}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":40,"modified":"2024-04-26T23:53:54.269203+0000","services":{}},"progress_events":{}} 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 msgr2=0x7fb4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 0x7fb4e006edb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec008040 comp rx=0 tx=0).stop 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 msgr2=0x7fb5040aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000f80 tx=0x7fb4f4004a40 comp rx=0 tx=0).stop 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 shutdown_connections 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5040b43c0 0x7fb5041f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50406eb60 0x7fb50413fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 --2- 172.21.15.46:0/3860076779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb50406e780 0x7fb5040aa4d0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 >> 172.21.15.46:0/3860076779 conn(0x7fb50406c530 msgr2=0x7fb504162e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 shutdown_connections 2024-04-26T23:54:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.354+0000 7fb50a200640 1 -- 172.21.15.46:0/3860076779 wait complete. 2024-04-26T23:54:01.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:54:01.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:54:01.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:54:01.611 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:54:01.611 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:54:01.612 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:01.612 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:01.671 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:01.671 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:01.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:01.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.932+0000 7fd03aa00640 1 Processor -- start 2024-04-26T23:54:01.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.933+0000 7fd03aa00640 1 -- start start 2024-04-26T23:54:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0340b4000 0x7fd0340d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0340d11f0 con 0x7fd0340d2130 2024-04-26T23:54:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0340d5f20 con 0x7fd03415f4e0 2024-04-26T23:54:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd0340d0ce0 con 0x7fd0340b4000 2024-04-26T23:54:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0340b4000 0x7fd0340d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0340b4000 0x7fd0340d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45188/0 (socket says 172.21.15.46:45188) 2024-04-26T23:54:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.934+0000 7fd03a000640 1 -- 172.21.15.46:0/1901037119 learned_addr learned my addr 172.21.15.46:0/1901037119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.935+0000 7fd03a000640 1 -- 172.21.15.46:0/1901037119 >> v1:172.21.15.142:6789/0 conn(0x7fd03415f4e0 legacy=0x7fd0340d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.935+0000 7fd03a000640 1 -- 172.21.15.46:0/1901037119 >> v1:172.21.15.46:6789/0 conn(0x7fd0340d2130 legacy=0x7fd0340b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.935+0000 7fd03a000640 1 -- 172.21.15.46:0/1901037119 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0340d0f10 con 0x7fd0340b4000 2024-04-26T23:54:01.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.936+0000 7fd03a000640 1 --2- 172.21.15.46:0/1901037119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340d2df0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009e80 tx=0x7fd02c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5176568a1a7efc0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.936+0000 7fd033e00640 1 -- 172.21.15.46:0/1901037119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c036070 con 0x7fd0340b4000 2024-04-26T23:54:01.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.936+0000 7fd033e00640 1 -- 172.21.15.46:0/1901037119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c031600 con 0x7fd0340b4000 2024-04-26T23:54:01.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.936+0000 7fd033e00640 1 -- 172.21.15.46:0/1901037119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0319d0 con 0x7fd0340b4000 2024-04-26T23:54:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 -- 172.21.15.46:0/1901037119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 msgr2=0x7fd0340d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 --2- 172.21.15.46:0/1901037119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340d2df0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009e80 tx=0x7fd02c02a7a0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 -- 172.21.15.46:0/1901037119 shutdown_connections 2024-04-26T23:54:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 --2- 172.21.15.46:0/1901037119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340d2df0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 -- 172.21.15.46:0/1901037119 >> 172.21.15.46:0/1901037119 conn(0x7fd0340d5010 msgr2=0x7fd0340d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 -- 172.21.15.46:0/1901037119 shutdown_connections 2024-04-26T23:54:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.937+0000 7fd03aa00640 1 -- 172.21.15.46:0/1901037119 wait complete. 2024-04-26T23:54:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.939+0000 7fd03aa00640 1 Processor -- start 2024-04-26T23:54:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.939+0000 7fd03aa00640 1 -- start start 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 0x7fd0340b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 0x7fd0340b9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0340b1d60 con 0x7fd0340b4000 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd0340d2990 con 0x7fd0340d2130 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd03415f8c0 con 0x7fd03415f4e0 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45196/0 (socket says 172.21.15.46:45196) 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 -- 172.21.15.46:0/538287181 learned_addr learned my addr 172.21.15.46:0/538287181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd038c00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 0x7fd0340b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd03a000640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 0x7fd0340b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 -- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 msgr2=0x7fd0340b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 0x7fd0340b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 -- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 msgr2=0x7fd0340b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 0x7fd0340b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd039600640 1 -- 172.21.15.46:0/538287181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0340ba5c0 con 0x7fd0340b4000 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.940+0000 7fd038c00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 0x7fd0340b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd03a000640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 0x7fd0340b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd039600640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c460 tx=0x7fd02800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028016020 con 0x7fd0340b4000 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028012870 con 0x7fd0340b4000 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028012c60 con 0x7fd0340b4000 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0340ba8a0 con 0x7fd0340b4000 2024-04-26T23:54:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.941+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd034153670 con 0x7fd0340b4000 2024-04-26T23:54:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.943+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0340693b0 con 0x7fd0340b4000 2024-04-26T23:54:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.944+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd028019020 con 0x7fd0340b4000 2024-04-26T23:54:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.944+0000 7fd031600640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 0x7fd01406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.945+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fd02808ba10 con 0x7fd0340b4000 2024-04-26T23:54:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.945+0000 7fd038c00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 0x7fd01406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.946+0000 7fd038c00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 0x7fd01406ede0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd0340b9520 tx=0x7fd020009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:01.949+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd028020030 con 0x7fd0340b4000 2024-04-26T23:54:02.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.160+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0340d33d0 con 0x7fd0340b4000 2024-04-26T23:54:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.162+0000 7fd031600640 1 -- 172.21.15.46:0/538287181 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16977 (secure 0 0 0) 0x7fd028059c50 con 0x7fd0340b4000 2024-04-26T23:54:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:02.165 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":162,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:53:53.908148+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1585151449":"2024-04-27T23:53:53.908053+0000","172.21.15.46:6833/1585151449":"2024-04-27T23:53:53.908053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 msgr2=0x7fd01406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 0x7fd01406ede0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd0340b9520 tx=0x7fd020009040 comp rx=0 tx=0).stop 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 msgr2=0x7fd0340b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c460 tx=0x7fd02800c920 comp rx=0 tx=0).stop 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 shutdown_connections 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd01406c920 0x7fd01406ede0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd03415f4e0 0x7fd0340b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0340d2130 0x7fd0340b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 --2- 172.21.15.46:0/538287181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0340b4000 0x7fd0340b5dd0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.165+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 >> 172.21.15.46:0/538287181 conn(0x7fd0340d5010 msgr2=0x7fd0340d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:02.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.166+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 shutdown_connections 2024-04-26T23:54:02.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.166+0000 7fd03aa00640 1 -- 172.21.15.46:0/538287181 wait complete. 2024-04-26T23:54:02.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:52:42.496744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:52:43.676354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:54:02.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:02.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.402+0000 7f0062800640 1 Processor -- start 2024-04-26T23:54:02.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.402+0000 7f0062800640 1 -- start start 2024-04-26T23:54:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.404+0000 7f0062800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f005c15cec0 0x7f005c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.404+0000 7f0062800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.404+0000 7f0062800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f005c0ce7d0 con 0x7f005c0cdb30 2024-04-26T23:54:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.404+0000 7f0062800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f005c0ceab0 con 0x7f005c15cec0 2024-04-26T23:54:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.404+0000 7f0062800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f005c0ced90 con 0x7f005c073d00 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:59616/0 (socket says 172.21.15.46:59616) 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 -- 172.21.15.46:0/3057939623 learned_addr learned my addr 172.21.15.46:0/3057939623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061400640 1 --2- 172.21.15.46:0/3057939623 >> v2:172.21.15.142:3300/0 conn(0x7f005c15cec0 0x7f005c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 -- 172.21.15.46:0/3057939623 >> v2:172.21.15.142:3300/0 conn(0x7f005c15cec0 msgr2=0x7f005c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:02.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 --2- 172.21.15.46:0/3057939623 >> v2:172.21.15.142:3300/0 conn(0x7f005c15cec0 0x7f005c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 -- 172.21.15.46:0/3057939623 >> v1:172.21.15.142:6789/0 conn(0x7f005c0cdb30 legacy=0x7f005c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.405+0000 7f0061e00640 1 -- 172.21.15.46:0/3057939623 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c0cf070 con 0x7f005c073d00 2024-04-26T23:54:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.407+0000 7f0061e00640 1 --2- 172.21.15.46:0/3057939623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0054009e50 tx=0x7f005402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598214d670452bea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.407+0000 7f005be00640 1 -- 172.21.15.46:0/3057939623 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054036070 con 0x7f005c073d00 2024-04-26T23:54:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.407+0000 7f005be00640 1 -- 172.21.15.46:0/3057939623 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054031600 con 0x7f005c073d00 2024-04-26T23:54:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.407+0000 7f005be00640 1 -- 172.21.15.46:0/3057939623 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540319d0 con 0x7f005c073d00 2024-04-26T23:54:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 -- 172.21.15.46:0/3057939623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 msgr2=0x7f005c0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 --2- 172.21.15.46:0/3057939623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0054009e50 tx=0x7f005402a730 comp rx=0 tx=0).stop 2024-04-26T23:54:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 -- 172.21.15.46:0/3057939623 shutdown_connections 2024-04-26T23:54:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 --2- 172.21.15.46:0/3057939623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c073d00 0x7f005c0b3b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 --2- 172.21.15.46:0/3057939623 >> v2:172.21.15.142:3300/0 conn(0x7f005c15cec0 0x7f005c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 -- 172.21.15.46:0/3057939623 >> 172.21.15.46:0/3057939623 conn(0x7f005c0b27b0 msgr2=0x7f005c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 -- 172.21.15.46:0/3057939623 shutdown_connections 2024-04-26T23:54:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.408+0000 7f0062800640 1 -- 172.21.15.46:0/3057939623 wait complete. 2024-04-26T23:54:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.410+0000 7f0062800640 1 Processor -- start 2024-04-26T23:54:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.410+0000 7f0062800640 1 -- start start 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c073d00 0x7f005c154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c15cec0 0x7f005c15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f005c0b4080 con 0x7f005c0cdb30 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f005c15d2a0 con 0x7f005c073d00 2024-04-26T23:54:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0062800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f005c0b4200 con 0x7f005c15cec0 2024-04-26T23:54:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0060a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0060a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45200/0 (socket says 172.21.15.46:45200) 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.411+0000 7f0060a00640 1 -- 172.21.15.46:0/2292336048 learned_addr learned my addr 172.21.15.46:0/2292336048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0061e00640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c15cec0 0x7f005c15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0061400640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c073d00 0x7f005c154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0060a00640 1 -- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c15cec0 msgr2=0x7f005c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0060a00640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c15cec0 0x7f005c15f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0060a00640 1 -- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c073d00 msgr2=0x7f005c154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0060a00640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c073d00 0x7f005c154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.412+0000 7f0060a00640 1 -- 172.21.15.46:0/2292336048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c155480 con 0x7f005c0cdb30 2024-04-26T23:54:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0060a00640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f004800c430 tx=0x7f004800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048017070 con 0x7f005c0cdb30 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00480052d0 con 0x7f005c0cdb30 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048003850 con 0x7f005c0cdb30 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c0c5230 con 0x7f005c0cdb30 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.413+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c0c57f0 con 0x7f005c0cdb30 2024-04-26T23:54:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.414+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c0693b0 con 0x7f005c0cdb30 2024-04-26T23:54:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.415+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f004801a070 con 0x7f005c0cdb30 2024-04-26T23:54:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.416+0000 7f0059600640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 0x7f003c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.416+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f004808b000 con 0x7f005c0cdb30 2024-04-26T23:54:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.417+0000 7f0061400640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 0x7f003c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.418+0000 7f0061400640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 0x7f003c06edb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0050009aa0 tx=0x7f0050009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.421+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0048058c10 con 0x7f005c0cdb30 2024-04-26T23:54:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.625+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f005c0c4120 con 0x7f005c0cdb30 2024-04-26T23:54:02.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.626+0000 7f0059600640 1 -- 172.21.15.46:0/2292336048 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) ==== 93+0+10 (secure 0 0 0) 0x7f004805cf10 con 0x7f005c0cdb30 2024-04-26T23:54:02.629 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:54:02.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.628+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 msgr2=0x7f003c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 0x7f003c06edb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0050009aa0 tx=0x7f0050009190 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 msgr2=0x7f005c0c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f004800c430 tx=0x7f004800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 shutdown_connections 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c8f0 0x7f003c06edb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f005c15cec0 0x7f005c15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f005c0cdb30 0x7f005c0c6f80 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 --2- 172.21.15.46:0/2292336048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f005c073d00 0x7f005c154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 >> 172.21.15.46:0/2292336048 conn(0x7f005c0b27b0 msgr2=0x7f005c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 shutdown_connections 2024-04-26T23:54:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.629+0000 7f0062800640 1 -- 172.21.15.46:0/2292336048 wait complete. 2024-04-26T23:54:02.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:54:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.866+0000 7f0b9c800640 1 Processor -- start 2024-04-26T23:54:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.866+0000 7f0b9c800640 1 -- start start 2024-04-26T23:54:02.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.868+0000 7f0b9c800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b9415dd60 0x7f0b94163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.868+0000 7f0b9c800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b94072880 con 0x7f0b940723e0 2024-04-26T23:54:02.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.868+0000 7f0b9c800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b94072b40 con 0x7f0b94071f80 2024-04-26T23:54:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.868+0000 7f0b9c800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0b9415e140 con 0x7f0b9415dd60 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.869+0000 7f0b9a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0b940723e0 0x7f0b9406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49352/0 (socket says 172.21.15.46:49352) 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.869+0000 7f0b9a800640 1 -- 172.21.15.46:0/1294933343 learned_addr learned my addr 172.21.15.46:0/1294933343 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.869+0000 7f0b99e00640 1 --2- 172.21.15.46:0/1294933343 >> v2:172.21.15.142:3300/0 conn(0x7f0b9415dd60 0x7f0b94163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.869+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488079270 0 0) 0x7f0b94072880 con 0x7f0b940723e0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.869+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b7c003520 con 0x7f0b940723e0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89050199 0 0) 0x7f0b7c003520 con 0x7f0b940723e0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 >> v2:172.21.15.142:3300/0 conn(0x7f0b9415dd60 msgr2=0x7f0b94163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 --2- 172.21.15.46:0/1294933343 >> v2:172.21.15.142:3300/0 conn(0x7f0b9415dd60 0x7f0b94163bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 >> v1:172.21.15.142:6789/0 conn(0x7f0b94071f80 legacy=0x7f0b940b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b94072e20 con 0x7f0b940723e0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0b900019f0 con 0x7f0b940723e0 2024-04-26T23:54:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 >> v1:172.21.15.46:6789/0 conn(0x7f0b940723e0 legacy=0x7f0b9406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0039b0 0x7f0b7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c004800 0x7f0b7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c00c030 con 0x7f0b7c0051c0 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c00c1e0 con 0x7f0b7c004800 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c00c3e0 con 0x7f0b7c0039b0 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b9a800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.870+0000 7f0b99400640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c004800 0x7f0b7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b9a800640 1 -- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0039b0 msgr2=0x7f0b7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b99e00640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0039b0 0x7f0b7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b9a800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0039b0 0x7f0b7c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b9a800640 1 -- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c004800 msgr2=0x7f0b7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b9a800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c004800 0x7f0b7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.871+0000 7f0b9a800640 1 -- 172.21.15.46:0/1294933343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c003520 con 0x7f0b7c0051c0 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.872+0000 7f0b9a800640 1 -- 172.21.15.46:0/1294933343 reap_dead start 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.872+0000 7f0b9a800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b7c00b9c0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000a840 tx=0x7f0b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3620eedfffc26792 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.872+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000a460 con 0x7f0b7c0051c0 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.872+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9002f070 con 0x7f0b7c0051c0 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.872+0000 7f0b98a00640 1 -- 172.21.15.46:0/1294933343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9002bcb0 con 0x7f0b7c0051c0 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 -- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 msgr2=0x7f0b7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b7c00b9c0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000a840 tx=0x7f0b7c005830 comp rx=0 tx=0).stop 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 -- 172.21.15.46:0/1294933343 shutdown_connections 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1294933343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b7c00b9c0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 -- 172.21.15.46:0/1294933343 >> 172.21.15.46:0/1294933343 conn(0x7f0b9406c980 msgr2=0x7f0b940b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 -- 172.21.15.46:0/1294933343 shutdown_connections 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.873+0000 7f0b9c800640 1 -- 172.21.15.46:0/1294933343 wait complete. 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.875+0000 7f0b9c800640 1 Processor -- start 2024-04-26T23:54:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.875+0000 7f0b9c800640 1 -- start start 2024-04-26T23:54:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 0x7f0b941f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b941f8bf0 0x7f0b94161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94163bc0 con 0x7f0b7c0051c0 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940733e0 con 0x7f0b941f9d00 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b9c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b94163d40 con 0x7f0b941f8bf0 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45204/0 (socket says 172.21.15.46:45204) 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 -- 172.21.15.46:0/1126412392 learned_addr learned my addr 172.21.15.46:0/1126412392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 -- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b941f8bf0 msgr2=0x7f0b94161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b941f8bf0 0x7f0b94161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99e00640 1 -- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 msgr2=0x7f0b941f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.876+0000 7f0b99400640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 0x7f0b941f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b99e00640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 0x7f0b941f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b99e00640 1 -- 172.21.15.46:0/1126412392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940aae70 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b99400640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 0x7f0b941f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b99e00640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f0b88010890 tx=0x7f0b88010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800f040 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8800eb60 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8801c400 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b940ab150 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.877+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b941fa5a0 con 0x7f0b7c0051c0 2024-04-26T23:54:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.879+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b94069530 con 0x7f0b7c0051c0 2024-04-26T23:54:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.880+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b8801c5a0 con 0x7f0b7c0051c0 2024-04-26T23:54:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.881+0000 7f0b8e000640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 0x7f0b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.881+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f0b8808c3d0 con 0x7f0b7c0051c0 2024-04-26T23:54:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.881+0000 7f0b99400640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 0x7f0b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.883+0000 7f0b99400640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 0x7f0b7806edb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0b84000c00 tx=0x7f0b84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:02.885+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b88018030 con 0x7f0b7c0051c0 2024-04-26T23:54:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.094+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b940b36f0 con 0x7f0b7c0051c0 2024-04-26T23:54:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.095+0000 7f0b8e000640 1 -- 172.21.15.46:0/1126412392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) ==== 104+0+11 (secure 0 0 0) 0x7f0b8805a610 con 0x7f0b7c0051c0 2024-04-26T23:54:03.097 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:54:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 msgr2=0x7f0b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 0x7f0b7806edb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0b84000c00 tx=0x7f0b84009040 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 msgr2=0x7f0b940aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f0b88010890 tx=0x7f0b88010d50 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 shutdown_connections 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b7806c8f0 0x7f0b7806edb0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b941f8bf0 0x7f0b94161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b941f9d00 0x7f0b941f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 --2- 172.21.15.46:0/1126412392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c0051c0 0x7f0b940aa4d0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 >> 172.21.15.46:0/1126412392 conn(0x7f0b9406c980 msgr2=0x7f0b9415d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 shutdown_connections 2024-04-26T23:54:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.098+0000 7f0b9c800640 1 -- 172.21.15.46:0/1126412392 wait complete. 2024-04-26T23:54:03.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:54:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.329+0000 7fba9ce00640 1 Processor -- start 2024-04-26T23:54:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.329+0000 7fba9ce00640 1 -- start start 2024-04-26T23:54:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815d730 0x7fba980d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b53b0 0x7fba9815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba9815e050 0x7fba9806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fba9806eb90 con 0x7fba9815d730 2024-04-26T23:54:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fba980b4310 con 0x7fba9815e050 2024-04-26T23:54:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba9ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba980b45f0 con 0x7fba980b53b0 2024-04-26T23:54:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.331+0000 7fba95e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815d730 0x7fba980d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba9815d730 0x7fba980d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45208/0 (socket says 172.21.15.46:45208) 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 -- 172.21.15.46:0/820079183 learned_addr learned my addr 172.21.15.46:0/820079183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba96800640 1 --2- 172.21.15.46:0/820079183 >> v2:172.21.15.142:3300/0 conn(0x7fba9815e050 0x7fba9806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95400640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b53b0 0x7fba9815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 -- 172.21.15.46:0/820079183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b53b0 msgr2=0x7fba9815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b53b0 0x7fba9815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 -- 172.21.15.46:0/820079183 >> v2:172.21.15.142:3300/0 conn(0x7fba9815e050 msgr2=0x7fba9806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 --2- 172.21.15.46:0/820079183 >> v2:172.21.15.142:3300/0 conn(0x7fba9815e050 0x7fba9806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.332+0000 7fba95e00640 1 -- 172.21.15.46:0/820079183 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980b48d0 con 0x7fba9815d730 2024-04-26T23:54:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.334+0000 7fba95e00640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815d730 0x7fba980d00d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fba98074200 tx=0x7fba8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f87b9fc11a6ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.334+0000 7fba94a00640 1 -- 172.21.15.46:0/820079183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88037070 con 0x7fba9815d730 2024-04-26T23:54:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.334+0000 7fba94a00640 1 -- 172.21.15.46:0/820079183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba88032440 con 0x7fba9815d730 2024-04-26T23:54:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.334+0000 7fba94a00640 1 -- 172.21.15.46:0/820079183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88032810 con 0x7fba9815d730 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 -- 172.21.15.46:0/820079183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815d730 msgr2=0x7fba980d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815d730 0x7fba980d00d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fba98074200 tx=0x7fba8802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 -- 172.21.15.46:0/820079183 shutdown_connections 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 --2- 172.21.15.46:0/820079183 >> v2:172.21.15.142:3300/0 conn(0x7fba9815e050 0x7fba9806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba980b53b0 0x7fba9815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 --2- 172.21.15.46:0/820079183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815d730 0x7fba980d00d0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 -- 172.21.15.46:0/820079183 >> 172.21.15.46:0/820079183 conn(0x7fba9806c980 msgr2=0x7fba980b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 -- 172.21.15.46:0/820079183 shutdown_connections 2024-04-26T23:54:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.335+0000 7fba9ce00640 1 -- 172.21.15.46:0/820079183 wait complete. 2024-04-26T23:54:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.337+0000 7fba9ce00640 1 Processor -- start 2024-04-26T23:54:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.337+0000 7fba9ce00640 1 -- start start 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980b53b0 0x7fba981f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba9815d730 0x7fba981e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba9815e4c0 con 0x7fba9815e050 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba980b4e90 con 0x7fba980b53b0 2024-04-26T23:54:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba9ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba980b3ec0 con 0x7fba9815d730 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.338+0000 7fba96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45212/0 (socket says 172.21.15.46:45212) 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 -- 172.21.15.46:0/3260155067 learned_addr learned my addr 172.21.15.46:0/3260155067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba95400640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba9815d730 0x7fba981e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba95e00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980b53b0 0x7fba981f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 -- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba9815d730 msgr2=0x7fba981e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba9815d730 0x7fba981e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 -- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980b53b0 msgr2=0x7fba981f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980b53b0 0x7fba981f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 -- 172.21.15.46:0/3260155067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980caf00 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.339+0000 7fba96800640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e70 tx=0x7fba9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.340+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90014620 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.340+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90019070 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.340+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90014a20 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.340+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba980ca140 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.340+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba980ca700 con 0x7fba9815e050 2024-04-26T23:54:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.341+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba98069530 con 0x7fba9815e050 2024-04-26T23:54:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.342+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba90014bc0 con 0x7fba9815e050 2024-04-26T23:54:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.343+0000 7fba8e000640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.343+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fba9008c560 con 0x7fba9815e050 2024-04-26T23:54:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.344+0000 7fba95e00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.345+0000 7fba95e00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fba88034000 tx=0x7fba8802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.347+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba90011260 con 0x7fba9815e050 2024-04-26T23:54:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.550+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba9806e4d0 con 0x7fba9815e050 2024-04-26T23:54:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.551+0000 7fba8e000640 1 -- 172.21.15.46:0/3260155067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) ==== 100+0+11 (secure 0 0 0) 0x7fba9005a840 con 0x7fba9815e050 2024-04-26T23:54:03.553 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:54:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 msgr2=0x7fba7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fba88034000 tx=0x7fba8802e000 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 msgr2=0x7fba981e4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fba90007e70 tx=0x7fba9000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 shutdown_connections 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba9815e050 0x7fba981e4490 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba9815d730 0x7fba981e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 --2- 172.21.15.46:0/3260155067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980b53b0 0x7fba981f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 >> 172.21.15.46:0/3260155067 conn(0x7fba9806c980 msgr2=0x7fba980734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 shutdown_connections 2024-04-26T23:54:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.553+0000 7fba9ce00640 1 -- 172.21.15.46:0/3260155067 wait complete. 2024-04-26T23:54:03.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:54:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.785+0000 7f8e74a00640 1 Processor -- start 2024-04-26T23:54:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.785+0000 7f8e74a00640 1 -- start start 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e74a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e700cc690 0x7f8e700cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e74a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e70072bb0 con 0x7f8e701454a0 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e74a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e70072e90 con 0x7f8e700c6a50 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e74a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8e70073170 con 0x7f8e700cc690 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e6e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e701454a0 0x7f8e70073970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49366/0 (socket says 172.21.15.46:49366) 2024-04-26T23:54:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e6e800640 1 -- 172.21.15.46:0/1914929290 learned_addr learned my addr 172.21.15.46:0/1914929290 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:03.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.787+0000 7f8e65e00640 1 --2- 172.21.15.46:0/1914929290 >> v2:172.21.15.142:3300/0 conn(0x7f8e700cc690 0x7f8e700cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372566778 0 0) 0x7f8e70072bb0 con 0x7f8e701454a0 2024-04-26T23:54:03.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e54003520 con 0x7f8e701454a0 2024-04-26T23:54:03.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3163673600 0 0) 0x7f8e54003520 con 0x7f8e701454a0 2024-04-26T23:54:03.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 >> v2:172.21.15.142:3300/0 conn(0x7f8e700cc690 msgr2=0x7f8e700cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 --2- 172.21.15.46:0/1914929290 >> v2:172.21.15.142:3300/0 conn(0x7f8e700cc690 0x7f8e700cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 >> v1:172.21.15.142:6789/0 conn(0x7f8e700c6a50 legacy=0x7f8e700cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e70073450 con 0x7f8e701454a0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8e60001670 con 0x7f8e701454a0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 >> v1:172.21.15.46:6789/0 conn(0x7f8e701454a0 legacy=0x7f8e70073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e540039b0 0x7f8e54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 0x7f8e54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e540051c0 0x7f8e5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70073170 con 0x7f8e54004800 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5400c060 con 0x7f8e540051c0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.788+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5400c260 con 0x7f8e540039b0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 0x7f8e54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e6e800640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e540039b0 0x7f8e54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 -- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e540039b0 msgr2=0x7f8e54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:03.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e540039b0 0x7f8e54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 -- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e540051c0 msgr2=0x7f8e5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e540051c0 0x7f8e5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e65e00640 1 -- 172.21.15.46:0/1914929290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e54003520 con 0x7f8e54004800 2024-04-26T23:54:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.789+0000 7f8e6f200640 1 -- 172.21.15.46:0/1914929290 reap_dead start 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.790+0000 7f8e65e00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 0x7f8e54004c80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009ac0 tx=0x7f8e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f27fd1878ad75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.790+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58036070 con 0x7f8e54004800 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.790+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e58031400 con 0x7f8e54004800 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.790+0000 7f8e6de00640 1 -- 172.21.15.46:0/1914929290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58009690 con 0x7f8e54004800 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 -- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 msgr2=0x7f8e54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 0x7f8e54004c80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009ac0 tx=0x7f8e54005830 comp rx=0 tx=0).stop 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 -- 172.21.15.46:0/1914929290 shutdown_connections 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 --2- 172.21.15.46:0/1914929290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e54004800 0x7f8e54004c80 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 -- 172.21.15.46:0/1914929290 >> 172.21.15.46:0/1914929290 conn(0x7f8e700b26d0 msgr2=0x7f8e700b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 -- 172.21.15.46:0/1914929290 shutdown_connections 2024-04-26T23:54:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.791+0000 7f8e74a00640 1 -- 172.21.15.46:0/1914929290 wait complete. 2024-04-26T23:54:03.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.793+0000 7f8e74a00640 1 Processor -- start 2024-04-26T23:54:03.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.793+0000 7f8e74a00640 1 -- start start 2024-04-26T23:54:03.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 0x7f8e7015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70141400 0x7f8e7015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70145a00 con 0x7f8e70140f80 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70145880 con 0x7f8e54004800 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.794+0000 7f8e74a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e700c6ea0 con 0x7f8e70141400 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45236/0 (socket says 172.21.15.46:45236) 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 -- 172.21.15.46:0/2165603323 learned_addr learned my addr 172.21.15.46:0/2165603323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e6f200640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70141400 0x7f8e7015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e6e800640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 0x7f8e7015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 -- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70141400 msgr2=0x7f8e7015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70141400 0x7f8e7015b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 -- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 msgr2=0x7f8e7015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 0x7f8e7015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 -- 172.21.15.46:0/2165603323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7015bef0 con 0x7f8e70140f80 2024-04-26T23:54:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e6e800640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 0x7f8e7015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.795+0000 7f8e65e00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f8e58034040 tx=0x7f8e58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.796+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5803f020 con 0x7f8e70140f80 2024-04-26T23:54:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.796+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e58037040 con 0x7f8e70140f80 2024-04-26T23:54:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.796+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58036070 con 0x7f8e70140f80 2024-04-26T23:54:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.796+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7015c1d0 con 0x7f8e70140f80 2024-04-26T23:54:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.796+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7015c790 con 0x7f8e70140f80 2024-04-26T23:54:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.797+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e700693b0 con 0x7f8e70140f80 2024-04-26T23:54:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.798+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e58042020 con 0x7f8e70140f80 2024-04-26T23:54:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.799+0000 7f8e67400640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 0x7f8e4c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.799+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f8e580abd40 con 0x7f8e70140f80 2024-04-26T23:54:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.799+0000 7f8e6e800640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 0x7f8e4c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.801+0000 7f8e6e800640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 0x7f8e4c06ee30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8e60000f60 tx=0x7f8e60000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:03.803+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8e58049030 con 0x7f8e70140f80 2024-04-26T23:54:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.022+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e700b0640 con 0x7f8e70140f80 2024-04-26T23:54:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.023+0000 7f8e67400640 1 -- 172.21.15.46:0/2165603323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v162) ==== 80+0+122 (secure 0 0 0) 0x7f8e5807a000 con 0x7f8e70140f80 2024-04-26T23:54:04.025 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/1585151449 2024-04-27T23:53:53.908053+0000 2024-04-26T23:54:04.026 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/1585151449 2024-04-27T23:53:53.908053+0000 2024-04-26T23:54:04.026 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-26T23:54:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 msgr2=0x7f8e4c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 0x7f8e4c06ee30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8e60000f60 tx=0x7f8e60000ed0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 msgr2=0x7f8e700b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f8e58034040 tx=0x7f8e58004730 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 shutdown_connections 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8e4c06c970 0x7f8e4c06ee30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70141400 0x7f8e7015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e70140f80 0x7f8e700b52a0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 --2- 172.21.15.46:0/2165603323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e54004800 0x7f8e7015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 >> 172.21.15.46:0/2165603323 conn(0x7f8e700b26d0 msgr2=0x7f8e70142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 shutdown_connections 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.026+0000 7f8e74a00640 1 -- 172.21.15.46:0/2165603323 wait complete. 2024-04-26T23:54:04.046 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:54:04.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:04.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:04.137 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:54:04.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:04.138 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:54:04.191 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:54:04.191 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:54:04.191 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:04.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:54:04.250 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:54:04.250 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:04.250 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:04.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:54:04.310 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:54:04.310 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:54:04.310 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:04.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:54:04.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.577+0000 7f394ec00640 1 Processor -- start 2024-04-26T23:54:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.577+0000 7f394ec00640 1 -- start start 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39480cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394ec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f394815f1d0 0x7f39480cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39480b03c0 con 0x7f39480cb1a0 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394ec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39480cf5c0 con 0x7f394815f1d0 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39480cb680 con 0x7f394815edf0 2024-04-26T23:54:04.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394e200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f39480cb1a0 0x7f39480ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49372/0 (socket says 172.21.15.46:49372) 2024-04-26T23:54:04.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.579+0000 7f394e200640 1 -- 172.21.15.46:0/4181194342 learned_addr learned my addr 172.21.15.46:0/4181194342 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:04.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f394ce00640 1 --2- 172.21.15.46:0/4181194342 >> v2:172.21.15.142:3300/0 conn(0x7f394815f1d0 0x7f39480cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f394d800640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39480cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1288934314 0 0) 0x7f39480b03c0 con 0x7f39480cb1a0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3930003520 con 0x7f39480cb1a0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214850506 0 0) 0x7f3930003520 con 0x7f39480cb1a0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 msgr2=0x7f39480cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39480cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 >> v2:172.21.15.142:3300/0 conn(0x7f394815f1d0 msgr2=0x7f39480cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 --2- 172.21.15.46:0/4181194342 >> v2:172.21.15.142:3300/0 conn(0x7f394815f1d0 0x7f39480cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39480cb960 con 0x7f39480cb1a0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3940001a10 con 0x7f39480cb1a0 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 >> v1:172.21.15.46:6789/0 conn(0x7f39480cb1a0 legacy=0x7f39480ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300039b0 0x7f3930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3930004800 0x7f3930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.580+0000 7f3947e00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39480cf5c0 con 0x7f3930004800 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39480cb680 con 0x7f39300039b0 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f393000b9f0 con 0x7f394815edf0 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3930004800 0x7f3930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394d800640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300039b0 0x7f3930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 -- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 msgr2=0x7f39300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394e200640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 -- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300039b0 msgr2=0x7f3930003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300039b0 0x7f3930003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394ce00640 1 -- 172.21.15.46:0/4181194342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3930003520 con 0x7f3930004800 2024-04-26T23:54:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.581+0000 7f394e200640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f394815edf0 0x7f39300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.582+0000 7f394ce00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3930004800 0x7f3930004c80 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f3934009ac0 tx=0x7f3930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585f576d554daf66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934036070 con 0x7f3930004800 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3934031400 con 0x7f3930004800 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f3947e00640 1 -- 172.21.15.46:0/4181194342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934009690 con 0x7f3930004800 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 -- 172.21.15.46:0/4181194342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3930004800 msgr2=0x7f3930004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 --2- 172.21.15.46:0/4181194342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3930004800 0x7f3930004c80 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f3934009ac0 tx=0x7f3930005830 comp rx=0 tx=0).stop 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394e200640 1 -- 172.21.15.46:0/4181194342 reap_dead start 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 -- 172.21.15.46:0/4181194342 shutdown_connections 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 -- 172.21.15.46:0/4181194342 >> 172.21.15.46:0/4181194342 conn(0x7f39480b26d0 msgr2=0x7f39480b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 -- 172.21.15.46:0/4181194342 shutdown_connections 2024-04-26T23:54:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.583+0000 7f394ec00640 1 -- 172.21.15.46:0/4181194342 wait complete. 2024-04-26T23:54:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.585+0000 7f394ec00640 1 Processor -- start 2024-04-26T23:54:04.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.585+0000 7f394ec00640 1 -- start start 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3930004800 0x7f394815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3948151530 0x7f39480b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39480cc200 con 0x7f39481509a0 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39480cbf20 con 0x7f3948151530 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.586+0000 7f394ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39480c6950 con 0x7f3930004800 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45258/0 (socket says 172.21.15.46:45258) 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 -- 172.21.15.46:0/1840983606 learned_addr learned my addr 172.21.15.46:0/1840983606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394ce00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3948151530 0x7f39480b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394d800640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3930004800 0x7f394815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 -- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3930004800 msgr2=0x7f394815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3930004800 0x7f394815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 -- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3948151530 msgr2=0x7f39480b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3948151530 0x7f39480b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 -- 172.21.15.46:0/1840983606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394806e620 con 0x7f39481509a0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.587+0000 7f394e200640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f39480cb0f0 tx=0x7f394000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.588+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940004d60 con 0x7f39481509a0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.588+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394000bb20 con 0x7f39481509a0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.588+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940007070 con 0x7f39481509a0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.588+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394815cd10 con 0x7f39481509a0 2024-04-26T23:54:04.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.588+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394815d2d0 con 0x7f39481509a0 2024-04-26T23:54:04.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.589+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39480693b0 con 0x7f39481509a0 2024-04-26T23:54:04.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.590+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f394001b020 con 0x7f39481509a0 2024-04-26T23:54:04.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.591+0000 7f3945600640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 0x7f392806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:04.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.591+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f394008be90 con 0x7f39481509a0 2024-04-26T23:54:04.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.592+0000 7f394d800640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 0x7f392806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.593+0000 7f394d800640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 0x7f392806ede0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:04.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.596+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f394005a040 con 0x7f39481509a0 2024-04-26T23:54:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.843+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39480cd1d0 con 0x7f39481509a0 2024-04-26T23:54:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.844+0000 7f3945600640 1 -- 172.21.15.46:0/1840983606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7f394008d020 con 0x7f39481509a0 2024-04-26T23:54:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:04.847 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:04.847 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 msgr2=0x7f392806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 0x7f392806ede0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c008040 comp rx=0 tx=0).stop 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 msgr2=0x7f3948150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f39480cb0f0 tx=0x7f394000b4a0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 shutdown_connections 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f392806c920 0x7f392806ede0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39481509a0 0x7f3948150e20 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3948151530 0x7f39480b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 --2- 172.21.15.46:0/1840983606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3930004800 0x7f394815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.847+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 >> 172.21.15.46:0/1840983606 conn(0x7f39480b26d0 msgr2=0x7f39480cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.848+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 shutdown_connections 2024-04-26T23:54:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:04.848+0000 7f394ec00640 1 -- 172.21.15.46:0/1840983606 wait complete. 2024-04-26T23:54:04.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.071+0000 7faee9000640 1 Processor -- start 2024-04-26T23:54:05.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.071+0000 7faee9000640 1 -- start start 2024-04-26T23:54:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee9000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faee40d1520 0x7faee40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40d1e40 con 0x7faee40d3af0 2024-04-26T23:54:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40ce7b0 con 0x7faee415d510 2024-04-26T23:54:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee9000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faee40d1220 con 0x7faee40d1520 2024-04-26T23:54:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faee40d3af0 0x7faee40706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49380/0 (socket says 172.21.15.46:49380) 2024-04-26T23:54:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.073+0000 7faee2800640 1 -- 172.21.15.46:0/419941081 learned_addr learned my addr 172.21.15.46:0/419941081 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506059148 0 0) 0x7faee40d1e40 con 0x7faee40d3af0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee1e00640 1 --2- 172.21.15.46:0/419941081 >> v2:172.21.15.142:3300/0 conn(0x7faee40d1520 0x7faee40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faec8003520 con 0x7faee40d3af0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3052286031 0 0) 0x7faec8003520 con 0x7faee40d3af0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 >> v2:172.21.15.142:3300/0 conn(0x7faee40d1520 msgr2=0x7faee40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 --2- 172.21.15.46:0/419941081 >> v2:172.21.15.142:3300/0 conn(0x7faee40d1520 0x7faee40d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 >> v1:172.21.15.142:6789/0 conn(0x7faee415d510 legacy=0x7faee40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40cea90 con 0x7faee40d3af0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7faedc0019f0 con 0x7faee40d3af0 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 >> v1:172.21.15.46:6789/0 conn(0x7faee40d3af0 legacy=0x7faee40706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec80039b0 0x7faec8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 0x7faec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec80051c0 0x7faec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d1220 con 0x7faec8004800 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faec800c040 con 0x7faec80039b0 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.074+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faec800c240 con 0x7faec80051c0 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 0x7faec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1e00640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec80039b0 0x7faec8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 -- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec80051c0 msgr2=0x7faec800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee2800640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec80051c0 0x7faec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec80051c0 0x7faec800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 -- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec80039b0 msgr2=0x7faec8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec80039b0 0x7faec8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee1400640 1 -- 172.21.15.46:0/419941081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec8003520 con 0x7faec8004800 2024-04-26T23:54:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.075+0000 7faee2800640 1 -- 172.21.15.46:0/419941081 reap_dead start 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.076+0000 7faee1400640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 0x7faec8004c80 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7faed0003d80 tx=0x7faec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8027805f4248dac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0037070 con 0x7faec8004800 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed002a9d0 con 0x7faec8004800 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee0a00640 1 -- 172.21.15.46:0/419941081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0003980 con 0x7faec8004800 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 -- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 msgr2=0x7faec8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 0x7faec8004c80 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7faed0003d80 tx=0x7faec8005830 comp rx=0 tx=0).stop 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 -- 172.21.15.46:0/419941081 shutdown_connections 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 --2- 172.21.15.46:0/419941081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8004800 0x7faec8004c80 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 -- 172.21.15.46:0/419941081 >> 172.21.15.46:0/419941081 conn(0x7faee40d2f40 msgr2=0x7faee40d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.077+0000 7faee9000640 1 -- 172.21.15.46:0/419941081 shutdown_connections 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.078+0000 7faee9000640 1 -- 172.21.15.46:0/419941081 wait complete. 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.079+0000 7faee9000640 1 Processor -- start 2024-04-26T23:54:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.080+0000 7faee9000640 1 -- start start 2024-04-26T23:54:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.080+0000 7faee9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec8004800 0x7faee414e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 0x7faee40c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee4070230 con 0x7faee40b5430 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faee40ce560 con 0x7faee41f50f0 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d1900 con 0x7faec8004800 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45276/0 (socket says 172.21.15.46:45276) 2024-04-26T23:54:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 -- 172.21.15.46:0/597652472 learned_addr learned my addr 172.21.15.46:0/597652472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee2800640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 0x7faee40c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1e00640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec8004800 0x7faee414e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 -- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec8004800 msgr2=0x7faee414e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec8004800 0x7faee414e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 -- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 msgr2=0x7faee40c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 0x7faee40c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee1400640 1 -- 172.21.15.46:0/597652472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40b5890 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.081+0000 7faee2800640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 0x7faee40c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faee1400640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7faed0035040 tx=0x7faed00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0037050 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed0038040 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0006e50 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee414efb0 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.082+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee414f570 con 0x7faee40b5430 2024-04-26T23:54:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.084+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee41f54d0 con 0x7faee40b5430 2024-04-26T23:54:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.085+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faed0045020 con 0x7faee40b5430 2024-04-26T23:54:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.085+0000 7faeda000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 0x7faec406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.086+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7faed00abb00 con 0x7faee40b5430 2024-04-26T23:54:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.086+0000 7faee1e00640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 0x7faec406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.087+0000 7faee1e00640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 0x7faec406ef80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faee406b1d0 tx=0x7faed4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.090+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faed0079d50 con 0x7faee40b5430 2024-04-26T23:54:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.302+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faee4069530 con 0x7faee40b5430 2024-04-26T23:54:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.304+0000 7faeda000640 1 -- 172.21.15.46:0/597652472 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16977 (secure 0 0 0) 0x7faed0079650 con 0x7faee40b5430 2024-04-26T23:54:05.307 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:05.307 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":162,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:53:53.908148+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1585151449":"2024-04-27T23:53:53.908053+0000","172.21.15.46:6833/1585151449":"2024-04-27T23:53:53.908053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.306+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 msgr2=0x7faec406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.306+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 0x7faec406ef80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faee406b1d0 tx=0x7faed4008040 comp rx=0 tx=0).stop 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.306+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 msgr2=0x7faee41f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7faed0035040 tx=0x7faed00036f0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 shutdown_connections 2024-04-26T23:54:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faec406cac0 0x7faec406ef80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee41f50f0 0x7faee40c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee40b5430 0x7faee41f4bb0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 --2- 172.21.15.46:0/597652472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec8004800 0x7faee414e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 >> 172.21.15.46:0/597652472 conn(0x7faee40d2f40 msgr2=0x7faee415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 shutdown_connections 2024-04-26T23:54:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.307+0000 7faee9000640 1 -- 172.21.15.46:0/597652472 wait complete. 2024-04-26T23:54:05.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:52:42.496744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:52:43.676354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:54:05.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.534+0000 7fbabb200640 1 Processor -- start 2024-04-26T23:54:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.534+0000 7fbabb200640 1 -- start start 2024-04-26T23:54:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.535+0000 7fbabb200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbabc0cdd00 0x7fbabc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.536+0000 7fbabb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.536+0000 7fbabb200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbabc0ce7d0 con 0x7fbabc15cec0 2024-04-26T23:54:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.536+0000 7fbabb200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbabc0ceab0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.536+0000 7fbabb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbabc0ced90 con 0x7fbabc073b30 2024-04-26T23:54:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.536+0000 7fbaba800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.537+0000 7fbab3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbabc0cdd00 0x7fbabc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.537+0000 7fbab3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbabc0cdd00 0x7fbabc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:45810/0 (socket says 172.21.15.46:45810) 2024-04-26T23:54:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.537+0000 7fbab3e00640 1 -- 172.21.15.46:0/1628541696 learned_addr learned my addr 172.21.15.46:0/1628541696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:05.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.538+0000 7fbab3e00640 1 -- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 msgr2=0x7fbabc0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:05.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.538+0000 7fbab3e00640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc0b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.538+0000 7fbab3e00640 1 -- 172.21.15.46:0/1628541696 >> v1:172.21.15.142:6789/0 conn(0x7fbabc15cec0 legacy=0x7fbabc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.538+0000 7fbab3e00640 1 -- 172.21.15.46:0/1628541696 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc0cf070 con 0x7fbabc0cdd00 2024-04-26T23:54:05.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.538+0000 7fbaba800640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:05.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.540+0000 7fbab3e00640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc0cdd00 0x7fbabc0735f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009870 tx=0x7fbaa402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff06e79c408bfae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.540+0000 7fbab9400640 1 -- 172.21.15.46:0/1628541696 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4037070 con 0x7fbabc0cdd00 2024-04-26T23:54:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.540+0000 7fbab9400640 1 -- 172.21.15.46:0/1628541696 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa4032440 con 0x7fbabc0cdd00 2024-04-26T23:54:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbab9400640 1 -- 172.21.15.46:0/1628541696 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4032810 con 0x7fbabc0cdd00 2024-04-26T23:54:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 -- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc0cdd00 msgr2=0x7fbabc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc0cdd00 0x7fbabc0735f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009870 tx=0x7fbaa402a250 comp rx=0 tx=0).stop 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 -- 172.21.15.46:0/1628541696 shutdown_connections 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 --2- 172.21.15.46:0/1628541696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc0cdd00 0x7fbabc0735f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.541+0000 7fbabb200640 1 -- 172.21.15.46:0/1628541696 >> 172.21.15.46:0/1628541696 conn(0x7fbabc0b27b0 msgr2=0x7fbabc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.542+0000 7fbabb200640 1 -- 172.21.15.46:0/1628541696 shutdown_connections 2024-04-26T23:54:05.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.542+0000 7fbabb200640 1 -- 172.21.15.46:0/1628541696 wait complete. 2024-04-26T23:54:05.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.544+0000 7fbabb200640 1 Processor -- start 2024-04-26T23:54:05.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.544+0000 7fbabb200640 1 -- start start 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc15cec0 0x7fbabc159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc073f70 con 0x7fbabc0cdd00 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc15d2a0 con 0x7fbabc15cec0 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.545+0000 7fbabb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbabc0b4080 con 0x7fbabc073b30 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45290/0 (socket says 172.21.15.46:45290) 2024-04-26T23:54:05.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 -- 172.21.15.46:0/2868289433 learned_addr learned my addr 172.21.15.46:0/2868289433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab9e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbaba800640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc15cec0 0x7fbabc159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 -- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 msgr2=0x7fbabc146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 -- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc15cec0 msgr2=0x7fbabc159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc15cec0 0x7fbabc159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.546+0000 7fbab3e00640 1 -- 172.21.15.46:0/2868289433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc1584c0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbab3e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4037040 con 0x7fbabc0cdd00 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa4042070 con 0x7fbabc0cdd00 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc1587a0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4030940 con 0x7fbabc0cdd00 2024-04-26T23:54:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.547+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc158d60 con 0x7fbabc0cdd00 2024-04-26T23:54:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.548+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc0693b0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.549+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaa4030ae0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.550+0000 7fbab2000640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.551+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fbaa40ac640 con 0x7fbabc0cdd00 2024-04-26T23:54:05.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.551+0000 7fbab9e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:05.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.552+0000 7fbab9e00640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 0x7fba9806edb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbaac000f60 tx=0x7fbaac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.555+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbaa407a7f0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.758+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbabc146c50 con 0x7fbabc0cdd00 2024-04-26T23:54:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.759+0000 7fbab2000640 1 -- 172.21.15.46:0/2868289433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) ==== 93+0+10 (secure 0 0 0) 0x7fbaa407a0f0 con 0x7fbabc0cdd00 2024-04-26T23:54:05.761 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.761+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 msgr2=0x7fba9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.761+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 0x7fba9806edb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbaac000f60 tx=0x7fbaac009040 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.761+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 msgr2=0x7fbabc146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.761+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa40046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 shutdown_connections 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc15cec0 0x7fbabc159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbabc0cdd00 0x7fbabc146fb0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 --2- 172.21.15.46:0/2868289433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbabc073b30 0x7fbabc146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 >> 172.21.15.46:0/2868289433 conn(0x7fbabc0b27b0 msgr2=0x7fbabc0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 shutdown_connections 2024-04-26T23:54:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:05.762+0000 7fbabb200640 1 -- 172.21.15.46:0/2868289433 wait complete. 2024-04-26T23:54:05.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:54:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.000+0000 7f4f8e600640 1 Processor -- start 2024-04-26T23:54:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.000+0000 7f4f8e600640 1 -- start start 2024-04-26T23:54:06.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.001+0000 7f4f8e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f881626c0 0x7f4f880b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.001+0000 7f4f8e600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4f880b5fb0 0x7f4f880d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.001+0000 7f4f8e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f880d07d0 con 0x7f4f88162aa0 2024-04-26T23:54:06.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.001+0000 7f4f8e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4f880d0ab0 con 0x7f4f881626c0 2024-04-26T23:54:06.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.001+0000 7f4f8e600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4f880d0d90 con 0x7f4f880b5fb0 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f8d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f881626c0 0x7f4f880b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f8d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f881626c0 0x7f4f880b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45298/0 (socket says 172.21.15.46:45298) 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f8d200640 1 -- 172.21.15.46:0/3106154767 learned_addr learned my addr 172.21.15.46:0/3106154767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f8dc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4f88162aa0 0x7f4f880d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49394/0 (socket says 172.21.15.46:49394) 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f87e00640 1 --2- 172.21.15.46:0/3106154767 >> v2:172.21.15.142:3300/0 conn(0x7f4f880b5fb0 0x7f4f880d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.002+0000 7f4f8d200640 1 -- 172.21.15.46:0/3106154767 >> v2:172.21.15.142:3300/0 conn(0x7f4f880b5fb0 msgr2=0x7f4f880d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.003+0000 7f4f8d200640 1 --2- 172.21.15.46:0/3106154767 >> v2:172.21.15.142:3300/0 conn(0x7f4f880b5fb0 0x7f4f880d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.003+0000 7f4f8d200640 1 -- 172.21.15.46:0/3106154767 >> v1:172.21.15.46:6789/0 conn(0x7f4f88162aa0 legacy=0x7f4f880d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.003+0000 7f4f8d200640 1 -- 172.21.15.46:0/3106154767 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f880d2c80 con 0x7f4f881626c0 2024-04-26T23:54:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.004+0000 7f4f8d200640 1 --2- 172.21.15.46:0/3106154767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f881626c0 0x7f4f880b47c0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b4750 tx=0x7f4f7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574f69e4e0262b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.004+0000 7f4f87400640 1 -- 172.21.15.46:0/3106154767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78036070 con 0x7f4f881626c0 2024-04-26T23:54:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.004+0000 7f4f87400640 1 -- 172.21.15.46:0/3106154767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78031600 con 0x7f4f881626c0 2024-04-26T23:54:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.004+0000 7f4f87400640 1 -- 172.21.15.46:0/3106154767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780319d0 con 0x7f4f881626c0 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 -- 172.21.15.46:0/3106154767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f881626c0 msgr2=0x7f4f880b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 --2- 172.21.15.46:0/3106154767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f881626c0 0x7f4f880b47c0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b4750 tx=0x7f4f7802a730 comp rx=0 tx=0).stop 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 -- 172.21.15.46:0/3106154767 shutdown_connections 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 --2- 172.21.15.46:0/3106154767 >> v2:172.21.15.142:3300/0 conn(0x7f4f880b5fb0 0x7f4f880d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 --2- 172.21.15.46:0/3106154767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f881626c0 0x7f4f880b47c0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 -- 172.21.15.46:0/3106154767 >> 172.21.15.46:0/3106154767 conn(0x7f4f8809e2a0 msgr2=0x7f4f880d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 -- 172.21.15.46:0/3106154767 shutdown_connections 2024-04-26T23:54:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.005+0000 7f4f8e600640 1 -- 172.21.15.46:0/3106154767 wait complete. 2024-04-26T23:54:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.007+0000 7f4f8e600640 1 Processor -- start 2024-04-26T23:54:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.007+0000 7f4f8e600640 1 -- start start 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f880b5fb0 0x7f4f880cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f881626c0 0x7f4f88159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f880d2190 con 0x7f4f88162aa0 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f880d2010 con 0x7f4f881626c0 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f880d31a0 con 0x7f4f880b5fb0 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45302/0 (socket says 172.21.15.46:45302) 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 -- 172.21.15.46:0/2662573485 learned_addr learned my addr 172.21.15.46:0/2662573485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8d200640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f880b5fb0 0x7f4f880cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 -- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f880b5fb0 msgr2=0x7f4f880cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f880b5fb0 0x7f4f880cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f87e00640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f881626c0 0x7f4f88159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.008+0000 7f4f8dc00640 1 -- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f881626c0 msgr2=0x7f4f88159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f8dc00640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f881626c0 0x7f4f88159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f8dc00640 1 -- 172.21.15.46:0/2662573485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f880c8a70 con 0x7f4f88162aa0 2024-04-26T23:54:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f8dc00640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c3c0 tx=0x7f4f8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f800074e0 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f80007680 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004a20 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f880c8d50 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.009+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f880c9310 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.011+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f880c9510 con 0x7f4f88162aa0 2024-04-26T23:54:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.012+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f8001a020 con 0x7f4f88162aa0 2024-04-26T23:54:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.012+0000 7f4f84c00640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 0x7f4f6406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.013+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f4f8008b490 con 0x7f4f88162aa0 2024-04-26T23:54:06.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.013+0000 7f4f8d200640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 0x7f4f6406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.014+0000 7f4f8d200640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 0x7f4f6406ee00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4f78033000 tx=0x7f4f78009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.017+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f800596c0 con 0x7f4f88162aa0 2024-04-26T23:54:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.225+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f880693b0 con 0x7f4f88162aa0 2024-04-26T23:54:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.226+0000 7f4f84c00640 1 -- 172.21.15.46:0/2662573485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) ==== 104+0+11 (secure 0 0 0) 0x7f4f80058fc0 con 0x7f4f88162aa0 2024-04-26T23:54:06.229 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:54:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 msgr2=0x7f4f6406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 0x7f4f6406ee00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4f78033000 tx=0x7f4f78009000 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 msgr2=0x7f4f8806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c3c0 tx=0x7f4f8000c880 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 shutdown_connections 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4f6406c940 0x7f4f6406ee00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f88162aa0 0x7f4f8806cdd0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f881626c0 0x7f4f88159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 --2- 172.21.15.46:0/2662573485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f880b5fb0 0x7f4f880cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 >> 172.21.15.46:0/2662573485 conn(0x7f4f8809e2a0 msgr2=0x7f4f8806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 shutdown_connections 2024-04-26T23:54:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.229+0000 7f4f8e600640 1 -- 172.21.15.46:0/2662573485 wait complete. 2024-04-26T23:54:06.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:54:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.466+0000 7fb7b9800640 1 Processor -- start 2024-04-26T23:54:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.467+0000 7fb7b9800640 1 -- start start 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.468+0000 7fb7b9800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 0x7fb7b415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.468+0000 7fb7b9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7b40d0330 0x7fb7b41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b9800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b4146b20 con 0x7fb7b40ce670 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b9800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40d07c0 con 0x7fb7b40d0330 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b9800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40ccf30 con 0x7fb7b4146740 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7b40d0330 0x7fb7b41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7b40d0330 0x7fb7b41612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45318/0 (socket says 172.21.15.46:45318) 2024-04-26T23:54:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b3400640 1 -- 172.21.15.46:0/2471382607 learned_addr learned my addr 172.21.15.46:0/2471382607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b3400640 1 -- 172.21.15.46:0/2471382607 >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 msgr2=0x7fb7b415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.469+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/2471382607 >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 0x7fb7b415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.470+0000 7fb7b3400640 1 --2- 172.21.15.46:0/2471382607 >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 0x7fb7b415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.470+0000 7fb7b3400640 1 -- 172.21.15.46:0/2471382607 >> v1:172.21.15.142:6789/0 conn(0x7fb7b40ce670 legacy=0x7fb7b40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.470+0000 7fb7b3400640 1 -- 172.21.15.46:0/2471382607 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b4160af0 con 0x7fb7b40d0330 2024-04-26T23:54:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.470+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/2471382607 >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 0x7fb7b415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.471+0000 7fb7b3400640 1 --2- 172.21.15.46:0/2471382607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b40d0330 0x7fb7b41612f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d1b70 tx=0x7fb79c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4640ff0d914ef110 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.471+0000 7fb7b2a00640 1 -- 172.21.15.46:0/2471382607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c037070 con 0x7fb7b40d0330 2024-04-26T23:54:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.471+0000 7fb7b2a00640 1 -- 172.21.15.46:0/2471382607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c032440 con 0x7fb7b40d0330 2024-04-26T23:54:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.471+0000 7fb7b2a00640 1 -- 172.21.15.46:0/2471382607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c032810 con 0x7fb7b40d0330 2024-04-26T23:54:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 -- 172.21.15.46:0/2471382607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b40d0330 msgr2=0x7fb7b41612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 --2- 172.21.15.46:0/2471382607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b40d0330 0x7fb7b41612f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d1b70 tx=0x7fb79c02a230 comp rx=0 tx=0).stop 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 -- 172.21.15.46:0/2471382607 shutdown_connections 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 --2- 172.21.15.46:0/2471382607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b40d0330 0x7fb7b41612f0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 --2- 172.21.15.46:0/2471382607 >> v2:172.21.15.142:3300/0 conn(0x7fb7b4146740 0x7fb7b415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 -- 172.21.15.46:0/2471382607 >> 172.21.15.46:0/2471382607 conn(0x7fb7b40b26d0 msgr2=0x7fb7b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 -- 172.21.15.46:0/2471382607 shutdown_connections 2024-04-26T23:54:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.472+0000 7fb7b9800640 1 -- 172.21.15.46:0/2471382607 wait complete. 2024-04-26T23:54:06.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.474+0000 7fb7b9800640 1 Processor -- start 2024-04-26T23:54:06.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.474+0000 7fb7b9800640 1 -- start start 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 0x7fb7b415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7b40d0330 0x7fb7b40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40cccf0 con 0x7fb7b4146740 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40ccb70 con 0x7fb7b40d0330 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b4161830 con 0x7fb7b40ce670 2024-04-26T23:54:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45326/0 (socket says 172.21.15.46:45326) 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.475+0000 7fb7b8e00640 1 -- 172.21.15.46:0/3164065810 learned_addr learned my addr 172.21.15.46:0/3164065810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 0x7fb7b415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b3400640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7b40d0330 0x7fb7b40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 -- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 msgr2=0x7fb7b415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 0x7fb7b415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 -- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7b40d0330 msgr2=0x7fb7b40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7b40d0330 0x7fb7b40b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 -- 172.21.15.46:0/3164065810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40b5560 con 0x7fb7b4146740 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 0x7fb7b415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7b8e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b410 tx=0x7fb7ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.476+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac015030 con 0x7fb7b4146740 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.477+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac004020 con 0x7fb7b4146740 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.477+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b40b5840 con 0x7fb7b4146740 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.477+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac004410 con 0x7fb7b4146740 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.477+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b40b5e00 con 0x7fb7b4146740 2024-04-26T23:54:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.478+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b40693b0 con 0x7fb7b4146740 2024-04-26T23:54:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.479+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7ac018070 con 0x7fb7b4146740 2024-04-26T23:54:06.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.480+0000 7fb7abe00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 0x7fb79006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.480+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fb7ac08b8b0 con 0x7fb7b4146740 2024-04-26T23:54:06.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.481+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 0x7fb79006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.482+0000 7fb7b3e00640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 0x7fb79006edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d430 tx=0x7fb7a4007530 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.485+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb7ac059a60 con 0x7fb7b4146740 2024-04-26T23:54:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.687+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7b4073c40 con 0x7fb7b4146740 2024-04-26T23:54:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.689+0000 7fb7abe00640 1 -- 172.21.15.46:0/3164065810 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) ==== 100+0+11 (secure 0 0 0) 0x7fb7ac059360 con 0x7fb7b4146740 2024-04-26T23:54:06.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:54:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 msgr2=0x7fb79006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 0x7fb79006edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d430 tx=0x7fb7a4007530 comp rx=0 tx=0).stop 2024-04-26T23:54:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 msgr2=0x7fb7b415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b410 tx=0x7fb7ac00b8d0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 shutdown_connections 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb79006c8f0 0x7fb79006edb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4146740 0x7fb7b415ef50 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7b40d0330 0x7fb7b40b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 --2- 172.21.15.46:0/3164065810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7b40ce670 0x7fb7b415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 >> 172.21.15.46:0/3164065810 conn(0x7fb7b40b26d0 msgr2=0x7fb7b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 shutdown_connections 2024-04-26T23:54:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.691+0000 7fb7b9800640 1 -- 172.21.15.46:0/3164065810 wait complete. 2024-04-26T23:54:06.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.927+0000 7fa69be00640 1 Processor -- start 2024-04-26T23:54:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.928+0000 7fa69be00640 1 -- start start 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.929+0000 7fa69be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa69c1466b0 0x7fa69c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa69c073ad0 con 0x7fa69c160910 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa69c146a90 con 0x7fa69c0720e0 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa69c073d70 con 0x7fa69c1466b0 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa69c1466b0 0x7fa69c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa69c1466b0 0x7fa69c06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45340/0 (socket says 172.21.15.46:45340) 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa69c160910 0x7fa69c0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49400/0 (socket says 172.21.15.46:49400) 2024-04-26T23:54:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69aa00640 1 -- 172.21.15.46:0/2507561663 learned_addr learned my addr 172.21.15.46:0/2507561663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69aa00640 1 -- 172.21.15.46:0/2507561663 >> v1:172.21.15.142:6789/0 conn(0x7fa69c0720e0 legacy=0x7fa69c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.930+0000 7fa69aa00640 1 -- 172.21.15.46:0/2507561663 >> v1:172.21.15.46:6789/0 conn(0x7fa69c160910 legacy=0x7fa69c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.931+0000 7fa69aa00640 1 -- 172.21.15.46:0/2507561663 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c074050 con 0x7fa69c1466b0 2024-04-26T23:54:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.931+0000 7fa699600640 1 -- 172.21.15.46:0/2507561663 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143351572 0 0) 0x7fa69c073ad0 con 0x7fa69c160910 2024-04-26T23:54:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa69aa00640 1 --2- 172.21.15.46:0/2507561663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c1466b0 0x7fa69c06de80 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0098a0 tx=0x7fa68c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8817e2628fc22f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa699600640 1 -- 172.21.15.46:0/2507561663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c037070 con 0x7fa69c1466b0 2024-04-26T23:54:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa699600640 1 -- 172.21.15.46:0/2507561663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c032640 con 0x7fa69c1466b0 2024-04-26T23:54:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa699600640 1 -- 172.21.15.46:0/2507561663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c032a10 con 0x7fa69c1466b0 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa69be00640 1 -- 172.21.15.46:0/2507561663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c1466b0 msgr2=0x7fa69c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.932+0000 7fa69be00640 1 --2- 172.21.15.46:0/2507561663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c1466b0 0x7fa69c06de80 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0098a0 tx=0x7fa68c02a320 comp rx=0 tx=0).stop 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.933+0000 7fa69be00640 1 -- 172.21.15.46:0/2507561663 shutdown_connections 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.933+0000 7fa69be00640 1 --2- 172.21.15.46:0/2507561663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c1466b0 0x7fa69c06de80 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.933+0000 7fa69be00640 1 -- 172.21.15.46:0/2507561663 >> 172.21.15.46:0/2507561663 conn(0x7fa69c0b26d0 msgr2=0x7fa69c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.933+0000 7fa69be00640 1 -- 172.21.15.46:0/2507561663 shutdown_connections 2024-04-26T23:54:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.933+0000 7fa69be00640 1 -- 172.21.15.46:0/2507561663 wait complete. 2024-04-26T23:54:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.935+0000 7fa69be00640 1 Processor -- start 2024-04-26T23:54:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.935+0000 7fa69be00640 1 -- start start 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa69c0720e0 0x7fa69c0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa69c1466b0 0x7fa69c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c06e3c0 con 0x7fa69c160910 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c0cb470 con 0x7fa69c0720e0 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c0cc0a0 con 0x7fa69c1466b0 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45342/0 (socket says 172.21.15.46:45342) 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69b400640 1 -- 172.21.15.46:0/4259254196 learned_addr learned my addr 172.21.15.46:0/4259254196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69aa00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa69c0720e0 0x7fa69c0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.936+0000 7fa69a000640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa69c1466b0 0x7fa69c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 -- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa69c1466b0 msgr2=0x7fa69c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa69c1466b0 0x7fa69c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 -- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa69c0720e0 msgr2=0x7fa69c0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa69c0720e0 0x7fa69c0cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 -- 172.21.15.46:0/4259254196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c0cfb80 con 0x7fa69c160910 2024-04-26T23:54:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69b400640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c730 tx=0x7fa69400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694018070 con 0x7fa69c160910 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69400ce40 con 0x7fa69c160910 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400a860 con 0x7fa69c160910 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c0cfe60 con 0x7fa69c160910 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.937+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c0d1210 con 0x7fa69c160910 2024-04-26T23:54:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.939+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69c0693b0 con 0x7fa69c160910 2024-04-26T23:54:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.940+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa69400aa00 con 0x7fa69c160910 2024-04-26T23:54:06.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.941+0000 7fa693400640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 0x7fa67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:06.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.941+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fa69408b2a0 con 0x7fa69c160910 2024-04-26T23:54:06.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.941+0000 7fa69aa00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 0x7fa67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:06.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.942+0000 7fa69aa00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 0x7fa67806edb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034000 tx=0x7fa68c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:06.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:06.945+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa694059580 con 0x7fa69c160910 2024-04-26T23:54:07.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.196+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa69c161220 con 0x7fa69c160910 2024-04-26T23:54:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.197+0000 7fa693400640 1 -- 172.21.15.46:0/4259254196 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7fa69408b0c0 con 0x7fa69c160910 2024-04-26T23:54:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:07.356 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.199+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 msgr2=0x7fa67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.199+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 0x7fa67806edb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034000 tx=0x7fa68c02e000 comp rx=0 tx=0).stop 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.199+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 msgr2=0x7fa69c1f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.199+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c730 tx=0x7fa69400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 shutdown_connections 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa67806c8f0 0x7fa67806edb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69c160910 0x7fa69c1f5310 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa69c1466b0 0x7fa69c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 --2- 172.21.15.46:0/4259254196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa69c0720e0 0x7fa69c0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 >> 172.21.15.46:0/4259254196 conn(0x7fa69c0b26d0 msgr2=0x7fa69c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 shutdown_connections 2024-04-26T23:54:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.200+0000 7fa69be00640 1 -- 172.21.15.46:0/4259254196 wait complete. 2024-04-26T23:54:07.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:07.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.573+0000 7fe179600640 1 Processor -- start 2024-04-26T23:54:07.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.573+0000 7fe179600640 1 -- start start 2024-04-26T23:54:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1741454a0 0x7fe1740cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 0x7fe1740cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 0x7fe1740c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe174072bb0 con 0x7fe1741454a0 2024-04-26T23:54:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe174072e90 con 0x7fe1740c6a50 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.575+0000 7fe179600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe174073170 con 0x7fe174073970 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1741454a0 0x7fe1740cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1741454a0 0x7fe1740cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45346/0 (socket says 172.21.15.46:45346) 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 -- 172.21.15.46:0/1620583028 learned_addr learned my addr 172.21.15.46:0/1620583028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe178c00640 1 --2- 172.21.15.46:0/1620583028 >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 0x7fe1740c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe16be00640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 0x7fe1740cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 -- 172.21.15.46:0/1620583028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 msgr2=0x7fe1740cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:07.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 0x7fe1740cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 -- 172.21.15.46:0/1620583028 >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 msgr2=0x7fe1740c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 --2- 172.21.15.46:0/1620583028 >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 0x7fe1740c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe173e00640 1 -- 172.21.15.46:0/1620583028 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174073450 con 0x7fe1741454a0 2024-04-26T23:54:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe178c00640 1 --2- 172.21.15.46:0/1620583028 >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 0x7fe1740c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:07.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.576+0000 7fe16be00640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 0x7fe1740cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.577+0000 7fe173e00640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe1740cc690 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fe1740cbd30 tx=0x7fe16402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1bf700457c1c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe173400640 1 -- 172.21.15.46:0/1620583028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164037070 con 0x7fe1741454a0 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe173400640 1 -- 172.21.15.46:0/1620583028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe164032440 con 0x7fe1741454a0 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe173400640 1 -- 172.21.15.46:0/1620583028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164032810 con 0x7fe1741454a0 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 -- 172.21.15.46:0/1620583028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 msgr2=0x7fe1740cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe1740cc690 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fe1740cbd30 tx=0x7fe16402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 -- 172.21.15.46:0/1620583028 shutdown_connections 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 --2- 172.21.15.46:0/1620583028 >> v2:172.21.15.142:3300/0 conn(0x7fe1740c6a50 0x7fe1740c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe174073970 0x7fe1740cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 --2- 172.21.15.46:0/1620583028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe1740cc690 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 -- 172.21.15.46:0/1620583028 >> 172.21.15.46:0/1620583028 conn(0x7fe1740b26d0 msgr2=0x7fe1740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.578+0000 7fe179600640 1 -- 172.21.15.46:0/1620583028 shutdown_connections 2024-04-26T23:54:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.579+0000 7fe179600640 1 -- 172.21.15.46:0/1620583028 wait complete. 2024-04-26T23:54:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.580+0000 7fe179600640 1 Processor -- start 2024-04-26T23:54:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.580+0000 7fe179600640 1 -- start start 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe174073970 0x7fe17413e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1740c6a50 0x7fe17413a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe174073dd0 con 0x7fe1741454a0 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe174072880 con 0x7fe174073970 2024-04-26T23:54:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.581+0000 7fe179600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740b6160 con 0x7fe1740c6a50 2024-04-26T23:54:07.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45358/0 (socket says 172.21.15.46:45358) 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 -- 172.21.15.46:0/1035324683 learned_addr learned my addr 172.21.15.46:0/1035324683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe16be00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1740c6a50 0x7fe17413a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe173e00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe174073970 0x7fe17413e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 -- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1740c6a50 msgr2=0x7fe17413a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1740c6a50 0x7fe17413a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 -- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe174073970 msgr2=0x7fe17413e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe174073970 0x7fe17413e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.582+0000 7fe178c00640 1 -- 172.21.15.46:0/1035324683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17413f280 con 0x7fe1741454a0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe178c00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00cef0 tx=0x7fe16c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c01a070 con 0x7fe1741454a0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c00d860 con 0x7fe1741454a0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17413f560 con 0x7fe1741454a0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00dc50 con 0x7fe1741454a0 2024-04-26T23:54:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.583+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1740b5c40 con 0x7fe1741454a0 2024-04-26T23:54:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.584+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1740693b0 con 0x7fe1741454a0 2024-04-26T23:54:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.585+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe16c01d070 con 0x7fe1741454a0 2024-04-26T23:54:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.586+0000 7fe170c00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 0x7fe15006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.586+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fe16c08c5c0 con 0x7fe1741454a0 2024-04-26T23:54:07.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.587+0000 7fe173e00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 0x7fe15006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.588+0000 7fe173e00640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 0x7fe15006edb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe164034000 tx=0x7fe16402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:07.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.591+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe16c05a8c0 con 0x7fe1741454a0 2024-04-26T23:54:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.789+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe174143fb0 con 0x7fe1741454a0 2024-04-26T23:54:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.792+0000 7fe170c00640 1 -- 172.21.15.46:0/1035324683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16977 (secure 0 0 0) 0x7fe16c05a1c0 con 0x7fe1741454a0 2024-04-26T23:54:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:07.795 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":162,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:53:53.908148+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1585151449":"2024-04-27T23:53:53.908053+0000","172.21.15.46:6833/1585151449":"2024-04-27T23:53:53.908053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 msgr2=0x7fe15006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 0x7fe15006edb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe164034000 tx=0x7fe16402e000 comp rx=0 tx=0).stop 2024-04-26T23:54:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 msgr2=0x7fe174156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00cef0 tx=0x7fe16c00a710 comp rx=0 tx=0).stop 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 shutdown_connections 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe15006c8f0 0x7fe15006edb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1741454a0 0x7fe174156270 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1740c6a50 0x7fe17413a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 --2- 172.21.15.46:0/1035324683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe174073970 0x7fe17413e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 >> 172.21.15.46:0/1035324683 conn(0x7fe1740b26d0 msgr2=0x7fe1740b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 shutdown_connections 2024-04-26T23:54:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:07.795+0000 7fe179600640 1 -- 172.21.15.46:0/1035324683 wait complete. 2024-04-26T23:54:07.817 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:54:07.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:08.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.026+0000 7f5b5fe00640 1 Processor -- start 2024-04-26T23:54:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.026+0000 7f5b5fe00640 1 -- start start 2024-04-26T23:54:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.027+0000 7f5b5fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b601466b0 0x7f5b6006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5b60160910 0x7f5b60160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b60073ad0 con 0x7f5b600720e0 2024-04-26T23:54:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5b601612a0 con 0x7f5b601466b0 2024-04-26T23:54:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5b60146a90 con 0x7f5b60160910 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b601466b0 0x7f5b6006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b601466b0 0x7f5b6006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45364/0 (socket says 172.21.15.46:45364) 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 -- 172.21.15.46:0/3069469876 learned_addr learned my addr 172.21.15.46:0/3069469876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5f400640 1 --2- 172.21.15.46:0/3069469876 >> v2:172.21.15.142:3300/0 conn(0x7f5b60160910 0x7f5b60160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 -- 172.21.15.46:0/3069469876 >> v2:172.21.15.142:3300/0 conn(0x7f5b60160910 msgr2=0x7f5b60160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 --2- 172.21.15.46:0/3069469876 >> v2:172.21.15.142:3300/0 conn(0x7f5b60160910 0x7f5b60160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 -- 172.21.15.46:0/3069469876 >> v1:172.21.15.46:6789/0 conn(0x7f5b600720e0 legacy=0x7f5b600cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.028+0000 7f5b5ea00640 1 -- 172.21.15.46:0/3069469876 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b60073d70 con 0x7f5b601466b0 2024-04-26T23:54:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.029+0000 7f5b5ea00640 1 --2- 172.21.15.46:0/3069469876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b601466b0 0x7f5b6006de80 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009870 tx=0x7f5b5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19982c8bc4c8e060 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5e000640 1 -- 172.21.15.46:0/3069469876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50037070 con 0x7f5b601466b0 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5e000640 1 -- 172.21.15.46:0/3069469876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b50032440 con 0x7f5b601466b0 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5e000640 1 -- 172.21.15.46:0/3069469876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50032810 con 0x7f5b601466b0 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 -- 172.21.15.46:0/3069469876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b601466b0 msgr2=0x7f5b6006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/3069469876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b601466b0 0x7f5b6006de80 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009870 tx=0x7f5b5002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 -- 172.21.15.46:0/3069469876 shutdown_connections 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/3069469876 >> v2:172.21.15.142:3300/0 conn(0x7f5b60160910 0x7f5b60160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/3069469876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b601466b0 0x7f5b6006de80 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.030+0000 7f5b5fe00640 1 -- 172.21.15.46:0/3069469876 >> 172.21.15.46:0/3069469876 conn(0x7f5b600b26d0 msgr2=0x7f5b600b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.031+0000 7f5b5fe00640 1 -- 172.21.15.46:0/3069469876 shutdown_connections 2024-04-26T23:54:08.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.031+0000 7f5b5fe00640 1 -- 172.21.15.46:0/3069469876 wait complete. 2024-04-26T23:54:08.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.032+0000 7f5b5fe00640 1 Processor -- start 2024-04-26T23:54:08.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.032+0000 7f5b5fe00640 1 -- start start 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b601466b0 0x7f5b600c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b60160910 0x7f5b60154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600cb470 con 0x7f5b600720e0 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600cb190 con 0x7f5b601466b0 2024-04-26T23:54:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.033+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6006e3c0 con 0x7f5b60160910 2024-04-26T23:54:08.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45374/0 (socket says 172.21.15.46:45374) 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 -- 172.21.15.46:0/1145131772 learned_addr learned my addr 172.21.15.46:0/1145131772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 -- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b60160910 msgr2=0x7f5b60154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b60160910 0x7f5b60154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 -- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b601466b0 msgr2=0x7f5b600c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b601466b0 0x7f5b600c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 -- 172.21.15.46:0/1145131772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b600ca490 con 0x7f5b600720e0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.034+0000 7f5b5ea00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.035+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50041020 con 0x7f5b600720e0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.035+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b500306c0 con 0x7f5b600720e0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.035+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50037040 con 0x7f5b600720e0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.035+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b600aeb30 con 0x7f5b600720e0 2024-04-26T23:54:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.035+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b600af0f0 con 0x7f5b600720e0 2024-04-26T23:54:08.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.036+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b600693b0 con 0x7f5b600720e0 2024-04-26T23:54:08.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.037+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b50044020 con 0x7f5b600720e0 2024-04-26T23:54:08.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.038+0000 7f5b57400640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 0x7f5b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.038+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f5b500ac550 con 0x7f5b600720e0 2024-04-26T23:54:08.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.039+0000 7f5b56000640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 0x7f5b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.040+0000 7f5b56000640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 0x7f5b3c06edb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000f60 tx=0x7f5b480079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.042+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b5004b030 con 0x7f5b600720e0 2024-04-26T23:54:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.288+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b600c7180 con 0x7f5b600720e0 2024-04-26T23:54:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.290+0000 7f5b57400640 1 -- 172.21.15.46:0/1145131772 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7f5b5007a790 con 0x7f5b600720e0 2024-04-26T23:54:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 msgr2=0x7f5b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 0x7f5b3c06edb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000f60 tx=0x7f5b480079d0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 msgr2=0x7f5b60140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b500046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 shutdown_connections 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5b3c06c8f0 0x7f5b3c06edb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b60160910 0x7f5b60154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b601466b0 0x7f5b600c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 --2- 172.21.15.46:0/1145131772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b600720e0 0x7f5b60140850 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.292+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 >> 172.21.15.46:0/1145131772 conn(0x7f5b600b26d0 msgr2=0x7f5b600c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.293+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 shutdown_connections 2024-04-26T23:54:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.293+0000 7f5b5fe00640 1 -- 172.21.15.46:0/1145131772 wait complete. 2024-04-26T23:54:08.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.524+0000 7fbb08a00640 1 Processor -- start 2024-04-26T23:54:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.524+0000 7fbb08a00640 1 -- start start 2024-04-26T23:54:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.525+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbb040cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.525+0000 7fbb08a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb04073ad0 con 0x7fbb04160910 2024-04-26T23:54:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.526+0000 7fbb08a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb04146a90 con 0x7fbb041466b0 2024-04-26T23:54:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.526+0000 7fbb08a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb04073d70 con 0x7fbb040720e0 2024-04-26T23:54:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.526+0000 7fbb03200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb04160910 0x7fbb040cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49416/0 (socket says 172.21.15.46:49416) 2024-04-26T23:54:08.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.526+0000 7fbb03200640 1 -- 172.21.15.46:0/1360890191 learned_addr learned my addr 172.21.15.46:0/1360890191 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:08.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.526+0000 7fbaf9e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbb040cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081606670 0 0) 0x7fbb04073ad0 con 0x7fbb04160910 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbae8003520 con 0x7fbb04160910 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874105065 0 0) 0x7fbb04146a90 con 0x7fbb041466b0 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb04073ad0 con 0x7fbb041466b0 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1342153744 0 0) 0x7fbae8003520 con 0x7fbb04160910 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 msgr2=0x7fbb040cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbb040cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 >> v1:172.21.15.142:6789/0 conn(0x7fbb041466b0 legacy=0x7fbb0406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb04074050 con 0x7fbb04160910 2024-04-26T23:54:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fbafc0019f0 con 0x7fbb04160910 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 >> v1:172.21.15.46:6789/0 conn(0x7fbb04160910 legacy=0x7fbb040cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae8003b70 0x7fbae8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbae8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae8004d40 0x7fbae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04073d70 con 0x7fbae8004d40 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbae800ba70 con 0x7fbae8003b70 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbae800bc70 con 0x7fbb040720e0 2024-04-26T23:54:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.527+0000 7fbb03200640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae8004d40 0x7fbae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb02800640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae8003b70 0x7fbae8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbaf9e00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbae8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb03200640 1 -- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 msgr2=0x7fbae8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb03200640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb040720e0 0x7fbae8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb03200640 1 -- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae8003b70 msgr2=0x7fbae8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb03200640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae8003b70 0x7fbae8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.528+0000 7fbb03200640 1 -- 172.21.15.46:0/1360890191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae8003520 con 0x7fbae8004d40 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.529+0000 7fbb03200640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae8004d40 0x7fbae80051c0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00a820 tx=0x7fbae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32158b9d00ac690e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.529+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00a450 con 0x7fbae8004d40 2024-04-26T23:54:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.529+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc02f070 con 0x7fbae8004d40 2024-04-26T23:54:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.529+0000 7fbb01e00640 1 -- 172.21.15.46:0/1360890191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc02bc70 con 0x7fbae8004d40 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 -- 172.21.15.46:0/1360890191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae8004d40 msgr2=0x7fbae80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 --2- 172.21.15.46:0/1360890191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae8004d40 0x7fbae80051c0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00a820 tx=0x7fbae8005830 comp rx=0 tx=0).stop 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb03200640 1 -- 172.21.15.46:0/1360890191 reap_dead start 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 -- 172.21.15.46:0/1360890191 shutdown_connections 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 -- 172.21.15.46:0/1360890191 >> 172.21.15.46:0/1360890191 conn(0x7fbb040b26d0 msgr2=0x7fbb040b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:08.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 -- 172.21.15.46:0/1360890191 shutdown_connections 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.530+0000 7fbb08a00640 1 -- 172.21.15.46:0/1360890191 wait complete. 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.532+0000 7fbb08a00640 1 Processor -- start 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.532+0000 7fbb08a00640 1 -- start start 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 0x7fbb040c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb04160090 0x7fbb040b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040cc0a0 con 0x7fbb040ca5d0 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040cb470 con 0x7fbb04160090 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb08a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040cc220 con 0x7fbae8004d40 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbaf9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbaf9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45400/0 (socket says 172.21.15.46:45400) 2024-04-26T23:54:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbaf9e00640 1 -- 172.21.15.46:0/3718163212 learned_addr learned my addr 172.21.15.46:0/3718163212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb02800640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 0x7fbb040c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.533+0000 7fbb03200640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb04160090 0x7fbb040b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 -- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 msgr2=0x7fbb040c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 0x7fbb040c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 -- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb04160090 msgr2=0x7fbb040b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb04160090 0x7fbb040b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 -- 172.21.15.46:0/3718163212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0413a8d0 con 0x7fbb040ca5d0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbb02800640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 0x7fbb040c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbaf9e00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbaec010890 tx=0x7fbaec010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00f040 con 0x7fbb040ca5d0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec00eb60 con 0x7fbb040ca5d0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec01c400 con 0x7fbb040ca5d0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb040aeba0 con 0x7fbb040ca5d0 2024-04-26T23:54:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.534+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb04160510 con 0x7fbb040ca5d0 2024-04-26T23:54:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.536+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb040c9590 con 0x7fbb040ca5d0 2024-04-26T23:54:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.537+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaec01c5a0 con 0x7fbb040ca5d0 2024-04-26T23:54:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.538+0000 7fbafb400640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 0x7fbae006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.538+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fbaec08c3d0 con 0x7fbb040ca5d0 2024-04-26T23:54:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.539+0000 7fbb02800640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 0x7fbae006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.540+0000 7fbb02800640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 0x7fbae006ede0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4007ae0 tx=0x7fbaf4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.543+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb040c9590 con 0x7fbb040ca5d0 2024-04-26T23:54:08.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.791+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb040c7180 con 0x7fbb040ca5d0 2024-04-26T23:54:08.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.792+0000 7fbafb400640 1 -- 172.21.15.46:0/3718163212 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+4376 (secure 0 0 0) 0x7fbaec05d130 con 0x7fbb040ca5d0 2024-04-26T23:54:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:08.794 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:52:44.692203+0000","modified":"2024-04-26T23:53:56.980289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10284":{"gid":10284,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":22,"addr":"172.21.15.46:6834/506880201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":506880201},{"type":"v1","addr":"172.21.15.46:6834","nonce":506880201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10284,"qdb_cluster":[10284]},"id":11}]} 2024-04-26T23:54:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 131 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 msgr2=0x7fbae006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 0x7fbae006ede0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4007ae0 tx=0x7fbaf4009040 comp rx=0 tx=0).stop 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 msgr2=0x7fbb040b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbaec010890 tx=0x7fbaec010d50 comp rx=0 tx=0).stop 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 shutdown_connections 2024-04-26T23:54:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbae006c920 0x7fbae006ede0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb04160090 0x7fbb040b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ca5d0 0x7fbb040b15f0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 --2- 172.21.15.46:0/3718163212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbae8004d40 0x7fbb040c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 >> 172.21.15.46:0/3718163212 conn(0x7fbb040b26d0 msgr2=0x7fbb0406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 shutdown_connections 2024-04-26T23:54:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:08.795+0000 7fbb08a00640 1 -- 172.21.15.46:0/3718163212 wait complete. 2024-04-26T23:54:08.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.027+0000 7fde37200640 1 Processor -- start 2024-04-26T23:54:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.028+0000 7fde37200640 1 -- start start 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.029+0000 7fde37200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde380d2130 0x7fde380d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.029+0000 7fde37200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde380d11f0 con 0x7fde3815f4e0 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde37200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde380d5f20 con 0x7fde380b4000 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde37200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fde380d0ce0 con 0x7fde380d2130 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fde3815f4e0 0x7fde380d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49426/0 (socket says 172.21.15.46:49426) 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35e00640 1 -- 172.21.15.46:0/2038801243 learned_addr learned my addr 172.21.15.46:0/2038801243 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35400640 1 --2- 172.21.15.46:0/2038801243 >> v2:172.21.15.46:3300/0 conn(0x7fde380d2130 0x7fde380d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde34a00640 1 -- 172.21.15.46:0/2038801243 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977747532 0 0) 0x7fde380d11f0 con 0x7fde3815f4e0 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde34a00640 1 -- 172.21.15.46:0/2038801243 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde1c003520 con 0x7fde3815f4e0 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35400640 1 -- 172.21.15.46:0/2038801243 >> v1:172.21.15.142:6789/0 conn(0x7fde380b4000 legacy=0x7fde380b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35400640 1 -- 172.21.15.46:0/2038801243 >> v1:172.21.15.46:6789/0 conn(0x7fde3815f4e0 legacy=0x7fde380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.030+0000 7fde35400640 1 -- 172.21.15.46:0/2038801243 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380d0f10 con 0x7fde380d2130 2024-04-26T23:54:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde35400640 1 --2- 172.21.15.46:0/2038801243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380d2130 0x7fde380d2df0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fde28009dc0 tx=0x7fde2802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5c0380349fe17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde34a00640 1 -- 172.21.15.46:0/2038801243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28007740 con 0x7fde380d2130 2024-04-26T23:54:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde34a00640 1 -- 172.21.15.46:0/2038801243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde280078e0 con 0x7fde380d2130 2024-04-26T23:54:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde34a00640 1 -- 172.21.15.46:0/2038801243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28007c90 con 0x7fde380d2130 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde37200640 1 -- 172.21.15.46:0/2038801243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380d2130 msgr2=0x7fde380d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.032+0000 7fde37200640 1 --2- 172.21.15.46:0/2038801243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380d2130 0x7fde380d2df0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fde28009dc0 tx=0x7fde2802a730 comp rx=0 tx=0).stop 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.033+0000 7fde37200640 1 -- 172.21.15.46:0/2038801243 shutdown_connections 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.033+0000 7fde37200640 1 --2- 172.21.15.46:0/2038801243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380d2130 0x7fde380d2df0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.033+0000 7fde37200640 1 -- 172.21.15.46:0/2038801243 >> 172.21.15.46:0/2038801243 conn(0x7fde380d5010 msgr2=0x7fde380d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.033+0000 7fde37200640 1 -- 172.21.15.46:0/2038801243 shutdown_connections 2024-04-26T23:54:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.033+0000 7fde37200640 1 -- 172.21.15.46:0/2038801243 wait complete. 2024-04-26T23:54:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.034+0000 7fde37200640 1 Processor -- start 2024-04-26T23:54:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 -- start start 2024-04-26T23:54:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde380d2130 0x7fde380ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 0x7fde380ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde380b1d60 con 0x7fde380b4000 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde380d2990 con 0x7fde3815f4e0 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.035+0000 7fde37200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde3815f8c0 con 0x7fde380d2130 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45424/0 (socket says 172.21.15.46:45424) 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 -- 172.21.15.46:0/2809340975 learned_addr learned my addr 172.21.15.46:0/2809340975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 -- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde380d2130 msgr2=0x7fde380ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde36800640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 0x7fde380ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde380d2130 0x7fde380ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 -- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 msgr2=0x7fde380ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 0x7fde380ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde35e00640 1 -- 172.21.15.46:0/2809340975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde381f96f0 con 0x7fde380b4000 2024-04-26T23:54:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.036+0000 7fde36800640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 0x7fde380ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde35e00640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c500 tx=0x7fde2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20017070 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20005ce0 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20007330 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381f6190 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.037+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381f8c10 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.038+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde380693b0 con 0x7fde380b4000 2024-04-26T23:54:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.039+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde2001a070 con 0x7fde380b4000 2024-04-26T23:54:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.040+0000 7fde2e000640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 0x7fde1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.040+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7fde2008b1d0 con 0x7fde380b4000 2024-04-26T23:54:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.041+0000 7fde35400640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 0x7fde1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.042+0000 7fde35400640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 0x7fde1406ede0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fde28034000 tx=0x7fde2802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.045+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde200594d0 con 0x7fde380b4000 2024-04-26T23:54:09.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.258+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde380d33d0 con 0x7fde380b4000 2024-04-26T23:54:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.261+0000 7fde2e000640 1 -- 172.21.15.46:0/2809340975 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16977 (secure 0 0 0) 0x7fde20058dd0 con 0x7fde380b4000 2024-04-26T23:54:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:09.265 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":162,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:53:53.908148+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:52:42.496744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-26T23:52:43.676354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1585151449":"2024-04-27T23:53:53.908053+0000","172.21.15.46:6833/1585151449":"2024-04-27T23:53:53.908053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 msgr2=0x7fde1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 0x7fde1406ede0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fde28034000 tx=0x7fde2802e000 comp rx=0 tx=0).stop 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 msgr2=0x7fde38159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c500 tx=0x7fde2000c9c0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 shutdown_connections 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde1406c920 0x7fde1406ede0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde3815f4e0 0x7fde380ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde380d2130 0x7fde380ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 --2- 172.21.15.46:0/2809340975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380b4000 0x7fde38159080 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 >> 172.21.15.46:0/2809340975 conn(0x7fde380d5010 msgr2=0x7fde3806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 shutdown_connections 2024-04-26T23:54:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.265+0000 7fde37200640 1 -- 172.21.15.46:0/2809340975 wait complete. 2024-04-26T23:54:09.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:54:09.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.501+0000 7f5c9e600640 1 Processor -- start 2024-04-26T23:54:09.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.502+0000 7f5c9e600640 1 -- start start 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.503+0000 7f5c9e600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c981466b0 0x7f5c9806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.503+0000 7f5c9e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c980720e0 0x7f5c980cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.503+0000 7f5c9e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c98073ad0 con 0x7f5c98160910 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.503+0000 7f5c9e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5c98146a90 con 0x7f5c980720e0 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.503+0000 7f5c9e600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5c98073d70 con 0x7f5c981466b0 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c980720e0 0x7f5c980cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c9dc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5c98160910 0x7f5c980cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49442/0 (socket says 172.21.15.46:49442) 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c980720e0 0x7f5c980cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45432/0 (socket says 172.21.15.46:45432) 2024-04-26T23:54:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c9dc00640 1 -- 172.21.15.46:0/637856485 learned_addr learned my addr 172.21.15.46:0/637856485 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 -- 172.21.15.46:0/637856485 >> v2:172.21.15.142:3300/0 conn(0x7f5c981466b0 msgr2=0x7f5c9806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 --2- 172.21.15.46:0/637856485 >> v2:172.21.15.142:3300/0 conn(0x7f5c981466b0 0x7f5c9806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 -- 172.21.15.46:0/637856485 >> v1:172.21.15.46:6789/0 conn(0x7f5c98160910 legacy=0x7f5c980cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.504+0000 7f5c97e00640 1 -- 172.21.15.46:0/637856485 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c98074050 con 0x7f5c980720e0 2024-04-26T23:54:09.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.505+0000 7f5c97e00640 1 --2- 172.21.15.46:0/637856485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c980720e0 0x7f5c980cd860 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5c840097a0 tx=0x7f5c8402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905355c9dcff89fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c97400640 1 -- 172.21.15.46:0/637856485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84037070 con 0x7f5c980720e0 2024-04-26T23:54:09.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c97400640 1 -- 172.21.15.46:0/637856485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c84032440 con 0x7f5c980720e0 2024-04-26T23:54:09.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c97400640 1 -- 172.21.15.46:0/637856485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84032810 con 0x7f5c980720e0 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c9e600640 1 -- 172.21.15.46:0/637856485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c980720e0 msgr2=0x7f5c980cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c9e600640 1 --2- 172.21.15.46:0/637856485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c980720e0 0x7f5c980cd860 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5c840097a0 tx=0x7f5c8402a1d0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c9e600640 1 -- 172.21.15.46:0/637856485 shutdown_connections 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c9e600640 1 --2- 172.21.15.46:0/637856485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c980720e0 0x7f5c980cd860 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.506+0000 7f5c9e600640 1 --2- 172.21.15.46:0/637856485 >> v2:172.21.15.142:3300/0 conn(0x7f5c981466b0 0x7f5c9806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.507+0000 7f5c9e600640 1 -- 172.21.15.46:0/637856485 >> 172.21.15.46:0/637856485 conn(0x7f5c980b26d0 msgr2=0x7f5c980b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.507+0000 7f5c9e600640 1 -- 172.21.15.46:0/637856485 shutdown_connections 2024-04-26T23:54:09.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.507+0000 7f5c9e600640 1 -- 172.21.15.46:0/637856485 wait complete. 2024-04-26T23:54:09.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.508+0000 7f5c9e600640 1 Processor -- start 2024-04-26T23:54:09.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.508+0000 7f5c9e600640 1 -- start start 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.509+0000 7f5c9e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 0x7f5c981e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.509+0000 7f5c9e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c981466b0 0x7f5c981f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.509+0000 7f5c9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9806e3c0 con 0x7f5c98160910 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980cb470 con 0x7f5c980720e0 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c980cdda0 con 0x7f5c981466b0 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45438/0 (socket says 172.21.15.46:45438) 2024-04-26T23:54:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 -- 172.21.15.46:0/2232840098 learned_addr learned my addr 172.21.15.46:0/2232840098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c97e00640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c981466b0 0x7f5c981f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9d200640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 0x7f5c981e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 -- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c981466b0 msgr2=0x7f5c981f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c981466b0 0x7f5c981f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 -- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 msgr2=0x7f5c981e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 0x7f5c981e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9dc00640 1 -- 172.21.15.46:0/2232840098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c980ce670 con 0x7f5c98160910 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.510+0000 7f5c9d200640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 0x7f5c981e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c9dc00640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000cab0 tx=0x7f5c9000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90017070 con 0x7f5c98160910 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9000ac30 con 0x7f5c98160910 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90012400 con 0x7f5c98160910 2024-04-26T23:54:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c980b4ed0 con 0x7f5c98160910 2024-04-26T23:54:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.511+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c980cebf0 con 0x7f5c98160910 2024-04-26T23:54:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.512+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c980693b0 con 0x7f5c98160910 2024-04-26T23:54:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.513+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c9001a070 con 0x7f5c98160910 2024-04-26T23:54:09.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.514+0000 7f5c94c00640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.514+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6187+0+0 (secure 0 0 0) 0x7f5c9008b8b0 con 0x7f5c98160910 2024-04-26T23:54:09.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.515+0000 7f5c9d200640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.516+0000 7f5c9d200640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00a870 tx=0x7f5c8c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.519+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c900594a0 con 0x7f5c98160910 2024-04-26T23:54:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.767+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c980b68d0 con 0x7f5c98160910 2024-04-26T23:54:09.770 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:54:09.768+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:54:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.772+0000 7f5c94c00640 1 -- 172.21.15.46:0/2232840098 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v132) ==== 161+0+0 (secure 0 0 0) 0x7f5c9005d7a0 con 0x7f5c98160910 2024-04-26T23:54:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.776+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 msgr2=0x7f5c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.776+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00a870 tx=0x7f5c8c008040 comp rx=0 tx=0).stop 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 msgr2=0x7f5c980c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000cab0 tx=0x7f5c9000cf70 comp rx=0 tx=0).stop 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 shutdown_connections 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00a870 tx=0x7f5c8c008040 comp rx=0 tx=0).stop 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c98160910 0x7f5c980c7be0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c981466b0 0x7f5c981f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 --2- 172.21.15.46:0/2232840098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c980720e0 0x7f5c981e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.777+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 >> 172.21.15.46:0/2232840098 conn(0x7f5c980b26d0 msgr2=0x7f5c980d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.778+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 shutdown_connections 2024-04-26T23:54:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.778+0000 7f5c9e600640 1 -- 172.21.15.46:0/2232840098 wait complete. 2024-04-26T23:54:09.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:54:09.858 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:54:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.984+0000 7f897e600640 1 Processor -- start 2024-04-26T23:54:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.984+0000 7f897e600640 1 -- start start 2024-04-26T23:54:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f897e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897815d730 0x7f89780d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f897e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f897e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897806eb90 con 0x7f89780b53b0 2024-04-26T23:54:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f897e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f89780b4310 con 0x7f897815d730 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f897e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f89780b45f0 con 0x7f897815e220 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f8977400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897815d730 0x7f89780d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f896fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89780b53b0 0x7f897815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49448/0 (socket says 172.21.15.46:49448) 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f896fe00640 1 -- 172.21.15.46:0/1560153047 learned_addr learned my addr 172.21.15.46:0/1560153047 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.986+0000 7f8977400640 1 --2- 172.21.15.46:0/1560153047 >> v2:172.21.15.46:3300/0 conn(0x7f897815d730 0x7f89780d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45480/0 (socket says 172.21.15.46:45480) 2024-04-26T23:54:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8977e00640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8977400640 1 -- 172.21.15.46:0/1560153047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 msgr2=0x7f897806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8977400640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8977400640 1 -- 172.21.15.46:0/1560153047 >> v1:172.21.15.46:6789/0 conn(0x7f89780b53b0 legacy=0x7f897815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8977400640 1 -- 172.21.15.46:0/1560153047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780b48d0 con 0x7f897815d730 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.987+0000 7f8976a00640 1 -- 172.21.15.46:0/1560153047 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628345299 0 0) 0x7f897806eb90 con 0x7f89780b53b0 2024-04-26T23:54:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.988+0000 7f8977400640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897815d730 0x7f89780d00d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f89680097a0 tx=0x7f896802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d208d67d3c53a0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f8976a00640 1 -- 172.21.15.46:0/1560153047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968037070 con 0x7f897815d730 2024-04-26T23:54:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f8976a00640 1 -- 172.21.15.46:0/1560153047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968032440 con 0x7f897815d730 2024-04-26T23:54:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f8976a00640 1 -- 172.21.15.46:0/1560153047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968032810 con 0x7f897815d730 2024-04-26T23:54:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 -- 172.21.15.46:0/1560153047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897815d730 msgr2=0x7f89780d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897815d730 0x7f89780d00d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f89680097a0 tx=0x7f896802a180 comp rx=0 tx=0).stop 2024-04-26T23:54:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 -- 172.21.15.46:0/1560153047 shutdown_connections 2024-04-26T23:54:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 --2- 172.21.15.46:0/1560153047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897815d730 0x7f89780d00d0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.989+0000 7f897e600640 1 -- 172.21.15.46:0/1560153047 >> 172.21.15.46:0/1560153047 conn(0x7f897806c980 msgr2=0x7f89780b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.990+0000 7f897e600640 1 -- 172.21.15.46:0/1560153047 shutdown_connections 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.990+0000 7f897e600640 1 -- 172.21.15.46:0/1560153047 wait complete. 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.991+0000 7f897e600640 1 Processor -- start 2024-04-26T23:54:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 -- start start 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897815d730 0x7f8978145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897813d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b3ec0 con 0x7f89780b53b0 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b4e90 con 0x7f897815d730 2024-04-26T23:54:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.992+0000 7f897e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f89780b4040 con 0x7f897815e220 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45496/0 (socket says 172.21.15.46:45496) 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 -- 172.21.15.46:0/420787820 learned_addr learned my addr 172.21.15.46:0/420787820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 -- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 msgr2=0x7f897813d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 -- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897815d730 msgr2=0x7f8978145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897815d730 0x7f8978145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 -- 172.21.15.46:0/420787820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8978144b90 con 0x7f89780b53b0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.993+0000 7f8977400640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f8968035040 tx=0x7f89680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.994+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968037040 con 0x7f89780b53b0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.994+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968030770 con 0x7f89780b53b0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.994+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968030b80 con 0x7f89780b53b0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.994+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897813cbd0 con 0x7f89780b53b0 2024-04-26T23:54:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.994+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8978145110 con 0x7f89780b53b0 2024-04-26T23:54:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.995+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8978069530 con 0x7f89780b53b0 2024-04-26T23:54:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.996+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8968044020 con 0x7f89780b53b0 2024-04-26T23:54:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.997+0000 7f896f400640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 0x7f895406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.997+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6273+0+0 (secure 0 0 0) 0x7f89680ac310 con 0x7f89780b53b0 2024-04-26T23:54:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.998+0000 7f896fe00640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 0x7f895406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:09.999+0000 7f896fe00640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 0x7f895406edb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.002+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8968032e20 con 0x7f89780b53b0 2024-04-26T23:54:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.258+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f89780aef80 con 0x7f89780b53b0 2024-04-26T23:54:10.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.775+0000 7f896f400640 1 -- 172.21.15.46:0/420787820 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v133) ==== 104+0+0 (secure 0 0 0) 0x7f896807a4f0 con 0x7f89780b53b0 2024-04-26T23:54:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 msgr2=0x7f895406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 0x7f895406edb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960007960 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 msgr2=0x7f897813fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f8968035040 tx=0x7f89680046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 shutdown_connections 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f895406c8f0 0x7f895406edb0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897815e220 0x7f897813d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897815d730 0x7f8978145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 --2- 172.21.15.46:0/420787820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89780b53b0 0x7f897813fa80 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.779+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 >> 172.21.15.46:0/420787820 conn(0x7f897806c980 msgr2=0x7f89780b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.780+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 shutdown_connections 2024-04-26T23:54:10.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:10.780+0000 7f897e600640 1 -- 172.21.15.46:0/420787820 wait complete. 2024-04-26T23:54:10.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:54:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.011+0000 7f2789a00640 1 Processor -- start 2024-04-26T23:54:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.011+0000 7f2789a00640 1 -- start start 2024-04-26T23:54:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.013+0000 7f2789a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f278415d6d0 0x7f27840b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.013+0000 7f2789a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27840b3400 0x7f27840727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.013+0000 7f2789a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840b3820 con 0x7f27841469c0 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.013+0000 7f2789a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f27840cfc70 con 0x7f278415d6d0 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.013+0000 7f2789a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27840d0ac0 con 0x7f27840b3400 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.014+0000 7f277a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f278415d6d0 0x7f27840b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.014+0000 7f277a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f278415d6d0 0x7f27840b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:45968/0 (socket says 172.21.15.46:45968) 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.014+0000 7f277a000640 1 -- 172.21.15.46:0/985348445 learned_addr learned my addr 172.21.15.46:0/985348445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.014+0000 7f2783400640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27840b3400 0x7f27840727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.015+0000 7f277a000640 1 -- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27840b3400 msgr2=0x7f27840727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.015+0000 7f277a000640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27840b3400 0x7f27840727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.015+0000 7f277a000640 1 -- 172.21.15.46:0/985348445 >> v1:172.21.15.142:6789/0 conn(0x7f27841469c0 legacy=0x7f27840cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.015+0000 7f277a000640 1 -- 172.21.15.46:0/985348445 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2784146da0 con 0x7f278415d6d0 2024-04-26T23:54:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.016+0000 7f277a000640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278415d6d0 0x7f27840b2ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f276c009870 tx=0x7f276c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f797d7b9c54c59df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.017+0000 7f2782000640 1 -- 172.21.15.46:0/985348445 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c037070 con 0x7f278415d6d0 2024-04-26T23:54:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.017+0000 7f2782000640 1 -- 172.21.15.46:0/985348445 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c032440 con 0x7f278415d6d0 2024-04-26T23:54:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.017+0000 7f2782000640 1 -- 172.21.15.46:0/985348445 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c032810 con 0x7f278415d6d0 2024-04-26T23:54:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.017+0000 7f2789a00640 1 -- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278415d6d0 msgr2=0x7f27840b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.017+0000 7f2789a00640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278415d6d0 0x7f27840b2ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f276c009870 tx=0x7f276c02a270 comp rx=0 tx=0).stop 2024-04-26T23:54:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 -- 172.21.15.46:0/985348445 shutdown_connections 2024-04-26T23:54:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27840b3400 0x7f27840727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 --2- 172.21.15.46:0/985348445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278415d6d0 0x7f27840b2ec0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 -- 172.21.15.46:0/985348445 >> 172.21.15.46:0/985348445 conn(0x7f278406c770 msgr2=0x7f278406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 -- 172.21.15.46:0/985348445 shutdown_connections 2024-04-26T23:54:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.018+0000 7f2789a00640 1 -- 172.21.15.46:0/985348445 wait complete. 2024-04-26T23:54:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.020+0000 7f2789a00640 1 Processor -- start 2024-04-26T23:54:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.020+0000 7f2789a00640 1 -- start start 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 0x7f27840ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f278415d6d0 0x7f27840d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27840d0470 con 0x7f27840b3400 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27840d0190 con 0x7f27841469c0 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2789a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27840d0750 con 0x7f278415d6d0 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2782a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2782a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45506/0 (socket says 172.21.15.46:45506) 2024-04-26T23:54:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2782a00640 1 -- 172.21.15.46:0/3724645778 learned_addr learned my addr 172.21.15.46:0/3724645778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f277a000640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 0x7f27840ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.022+0000 7f2783400640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f278415d6d0 0x7f27840d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 -- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f278415d6d0 msgr2=0x7f27840d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f278415d6d0 0x7f27840d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 -- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 msgr2=0x7f27840ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 0x7f27840ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 -- 172.21.15.46:0/3724645778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278413dff0 con 0x7f27840b3400 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f277a000640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 0x7f27840ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2782a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f277400c870 tx=0x7f277400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774003b70 con 0x7f27840b3400 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2774003d30 con 0x7f27840b3400 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277400aa90 con 0x7f27840b3400 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.023+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278415cf00 con 0x7f27840b3400 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.024+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278413e570 con 0x7f27840b3400 2024-04-26T23:54:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.025+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784069530 con 0x7f27840b3400 2024-04-26T23:54:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.026+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f277400ac30 con 0x7f27840b3400 2024-04-26T23:54:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.027+0000 7f277b400640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.027+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6273+0+0 (secure 0 0 0) 0x7f277408bc80 con 0x7f27840b3400 2024-04-26T23:54:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.027+0000 7f277a000640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.029+0000 7f277a000640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 0x7f276006ede0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f276c034000 tx=0x7f276c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.033+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2774059f00 con 0x7f27840b3400 2024-04-26T23:54:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:11.240+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f278406d360 con 0x7f27840b3400 2024-04-26T23:54:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.084+0000 7f277b400640 1 -- 172.21.15.46:0/3724645778 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) ==== 181+0+0 (secure 0 0 0) 0x7f2774059800 con 0x7f27840b3400 2024-04-26T23:54:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.087+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 msgr2=0x7f276006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.087+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 0x7f276006ede0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f276c034000 tx=0x7f276c02e000 comp rx=0 tx=0).stop 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.087+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 msgr2=0x7f278415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.087+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f277400c870 tx=0x7f277400cd30 comp rx=0 tx=0).stop 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 shutdown_connections 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f278415d6d0 0x7f27840d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27841469c0 0x7f27840ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 --2- 172.21.15.46:0/3724645778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27840b3400 0x7f278415c9c0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 >> 172.21.15.46:0/3724645778 conn(0x7f278406c770 msgr2=0x7f27840cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 shutdown_connections 2024-04-26T23:54:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.088+0000 7f2789a00640 1 -- 172.21.15.46:0/3724645778 wait complete. 2024-04-26T23:54:12.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:54:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.324+0000 7f0b84600640 1 Processor -- start 2024-04-26T23:54:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.324+0000 7f0b84600640 1 -- start start 2024-04-26T23:54:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.325+0000 7f0b84600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b7c15f6c0 0x7f0b7c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.325+0000 7f0b84600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0d5820 0x7f0b7c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.325+0000 7f0b84600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7c0d0fe0 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.325+0000 7f0b84600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0d5d00 con 0x7f0b7c15f6c0 2024-04-26T23:54:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.325+0000 7f0b84600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cffc0 con 0x7f0b7c0d5820 2024-04-26T23:54:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b82600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0b7c0d36f0 0x7f0b7c0b1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49112/0 (socket says 172.21.15.46:49112) 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b82600640 1 -- 172.21.15.46:0/3322351579 learned_addr learned my addr 172.21.15.46:0/3322351579 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b81200640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0d5820 0x7f0b7c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b81c00640 1 --2- 172.21.15.46:0/3322351579 >> v2:172.21.15.142:3300/0 conn(0x7f0b7c15f6c0 0x7f0b7c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389947054 0 0) 0x7f0b7c0d0fe0 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.326+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b64003520 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2866463356 0 0) 0x7f0b64003520 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0d5820 msgr2=0x7f0b7c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0d5820 0x7f0b7c0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 >> v2:172.21.15.142:3300/0 conn(0x7f0b7c15f6c0 msgr2=0x7f0b7c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 --2- 172.21.15.46:0/3322351579 >> v2:172.21.15.142:3300/0 conn(0x7f0b7c15f6c0 0x7f0b7c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0d02a0 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0b740019f0 con 0x7f0b7c0d36f0 2024-04-26T23:54:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 >> v1:172.21.15.46:6789/0 conn(0x7f0b7c0d36f0 legacy=0x7f0b7c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b64004800 0x7f0b64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640051c0 0x7f0b6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d5d00 con 0x7f0b640039b0 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cffc0 con 0x7f0b64004800 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6400c030 con 0x7f0b640051c0 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b81200640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b7c0d5820 0x7f0b7c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.327+0000 7f0b81c00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b81c00640 1 -- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640051c0 msgr2=0x7f0b6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b82600640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640051c0 0x7f0b6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b81c00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640051c0 0x7f0b6400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b81c00640 1 -- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b64004800 msgr2=0x7f0b64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b81c00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b64004800 0x7f0b64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.328+0000 7f0b81c00640 1 -- 172.21.15.46:0/3322351579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b64003520 con 0x7f0b640039b0 2024-04-26T23:54:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.329+0000 7f0b81c00640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000ba30 tx=0x7f0b64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c84a6363f665e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.329+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70030070 con 0x7f0b640039b0 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.329+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7002ae10 con 0x7f0b640039b0 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.329+0000 7f0b7be00640 1 -- 172.21.15.46:0/3322351579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000b600 con 0x7f0b640039b0 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 -- 172.21.15.46:0/3322351579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b640039b0 msgr2=0x7f0b64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 --2- 172.21.15.46:0/3322351579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000ba30 tx=0x7f0b64005830 comp rx=0 tx=0).stop 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b82600640 1 -- 172.21.15.46:0/3322351579 reap_dead start 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 -- 172.21.15.46:0/3322351579 shutdown_connections 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 -- 172.21.15.46:0/3322351579 >> 172.21.15.46:0/3322351579 conn(0x7f0b7c09e450 msgr2=0x7f0b7c0d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 -- 172.21.15.46:0/3322351579 shutdown_connections 2024-04-26T23:54:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.330+0000 7f0b84600640 1 -- 172.21.15.46:0/3322351579 wait complete. 2024-04-26T23:54:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.331+0000 7f0b84600640 1 Processor -- start 2024-04-26T23:54:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.332+0000 7f0b84600640 1 -- start start 2024-04-26T23:54:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640039b0 0x7f0b7c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c0b9280 0x7f0b7c0b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cfda0 con 0x7f0b7c161b40 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cfc20 con 0x7f0b7c0b9280 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b84600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d30b0 con 0x7f0b640039b0 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b82600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b82600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54870/0 (socket says 172.21.15.46:54870) 2024-04-26T23:54:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b82600640 1 -- 172.21.15.46:0/2160873496 learned_addr learned my addr 172.21.15.46:0/2160873496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b81c00640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640039b0 0x7f0b7c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b81200640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c0b9280 0x7f0b7c0b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.333+0000 7f0b82600640 1 -- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640039b0 msgr2=0x7f0b7c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b82600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640039b0 0x7f0b7c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b82600640 1 -- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c0b9280 msgr2=0x7f0b7c0b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b82600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c0b9280 0x7f0b7c0b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b82600640 1 -- 172.21.15.46:0/2160873496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0baee0 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b82600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c08f300 tx=0x7f0b7400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400bb50 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7400bcf0 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400bea0 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c0b9e40 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.334+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c0ba400 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.335+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c069530 con 0x7f0b7c161b40 2024-04-26T23:54:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.337+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b7401b020 con 0x7f0b7c161b40 2024-04-26T23:54:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.337+0000 7f0b79600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.338+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5835+0+0 (secure 0 0 0) 0x7f0b7408bde0 con 0x7f0b7c161b40 2024-04-26T23:54:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.338+0000 7f0b81c00640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.339+0000 7f0b81c00640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0b70035000 tx=0x7f0b70030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.342+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b7405a0f0 con 0x7f0b7c161b40 2024-04-26T23:54:12.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:12.547+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b7c0d6020 con 0x7f0b7c161b40 2024-04-26T23:54:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.095+0000 7f0b79600640 1 -- 172.21.15.46:0/2160873496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) ==== 169+0+0 (secure 0 0 0) 0x7f0b740599f0 con 0x7f0b7c161b40 2024-04-26T23:54:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.098+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 msgr2=0x7f0b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.098+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0b70035000 tx=0x7f0b70030000 comp rx=0 tx=0).stop 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 msgr2=0x7f0b7c161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c08f300 tx=0x7f0b7400b4a0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 shutdown_connections 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b7c161b40 0x7f0b7c161fc0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c0b9280 0x7f0b7c0b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 --2- 172.21.15.46:0/2160873496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b640039b0 0x7f0b7c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 >> 172.21.15.46:0/2160873496 conn(0x7f0b7c09e450 msgr2=0x7f0b7c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.099+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 shutdown_connections 2024-04-26T23:54:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.100+0000 7f0b84600640 1 -- 172.21.15.46:0/2160873496 wait complete. 2024-04-26T23:54:13.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:54:13.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:13.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.327+0000 7fedc1e00640 1 Processor -- start 2024-04-26T23:54:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.327+0000 7fedc1e00640 1 -- start start 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedc1e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fedbc162aa0 0x7fedbc0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedc1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc0d3b90 0x7fedbc0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedc1e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedbc0b46f0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedc1e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fedbc0d11d0 con 0x7fedbc162aa0 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedc1e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc0b44a0 con 0x7fedbc0d3b90 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedb3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fedbc0d5b90 0x7fedbc0b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49124/0 (socket says 172.21.15.46:49124) 2024-04-26T23:54:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedb3e00640 1 -- 172.21.15.46:0/3273761617 learned_addr learned my addr 172.21.15.46:0/3273761617 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.329+0000 7fedbaa00640 1 --2- 172.21.15.46:0/3273761617 >> v2:172.21.15.142:3300/0 conn(0x7fedbc162aa0 0x7fedbc0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedbb400640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc0d3b90 0x7fedbc0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedba000640 1 -- 172.21.15.46:0/3273761617 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820081533 0 0) 0x7fedbc0b46f0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedba000640 1 -- 172.21.15.46:0/3273761617 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda0003520 con 0x7fedbc0d5b90 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedbaa00640 1 -- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc0d3b90 msgr2=0x7fedbc0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedbaa00640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc0d3b90 0x7fedbc0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedbaa00640 1 -- 172.21.15.46:0/3273761617 >> v1:172.21.15.46:6789/0 conn(0x7fedbc0d5b90 legacy=0x7fedbc0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.330+0000 7fedbaa00640 1 -- 172.21.15.46:0/3273761617 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc0b49d0 con 0x7fedbc162aa0 2024-04-26T23:54:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.331+0000 7fedbaa00640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc162aa0 0x7fedbc0d3650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fedac0097a0 tx=0x7fedac02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3ba6ab79884fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.332+0000 7fedba000640 1 -- 172.21.15.46:0/3273761617 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac037070 con 0x7fedbc162aa0 2024-04-26T23:54:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.332+0000 7fedba000640 1 -- 172.21.15.46:0/3273761617 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac02adb0 con 0x7fedbc162aa0 2024-04-26T23:54:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.332+0000 7fedba000640 1 -- 172.21.15.46:0/3273761617 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac032640 con 0x7fedbc162aa0 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.332+0000 7fedc1e00640 1 -- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc162aa0 msgr2=0x7fedbc0d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.332+0000 7fedc1e00640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc162aa0 0x7fedbc0d3650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fedac0097a0 tx=0x7fedac02a060 comp rx=0 tx=0).stop 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 -- 172.21.15.46:0/3273761617 shutdown_connections 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc0d3b90 0x7fedbc0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 --2- 172.21.15.46:0/3273761617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc162aa0 0x7fedbc0d3650 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 -- 172.21.15.46:0/3273761617 >> 172.21.15.46:0/3273761617 conn(0x7fedbc0d5070 msgr2=0x7fedbc0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 -- 172.21.15.46:0/3273761617 shutdown_connections 2024-04-26T23:54:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.333+0000 7fedc1e00640 1 -- 172.21.15.46:0/3273761617 wait complete. 2024-04-26T23:54:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.335+0000 7fedc1e00640 1 Processor -- start 2024-04-26T23:54:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.335+0000 7fedc1e00640 1 -- start start 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 0x7fedbc15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc162aa0 0x7fedbc0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc0d3fd0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc0b5020 con 0x7fedbc0d3b90 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.336+0000 7fedc1e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc162e80 con 0x7fedbc162aa0 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54882/0 (socket says 172.21.15.46:54882) 2024-04-26T23:54:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 -- 172.21.15.46:0/148043522 learned_addr learned my addr 172.21.15.46:0/148043522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedbaa00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 0x7fedbc15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedbb400640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc162aa0 0x7fedbc0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 -- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc162aa0 msgr2=0x7fedbc0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc162aa0 0x7fedbc0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 -- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 msgr2=0x7fedbc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 0x7fedbc15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedb3e00640 1 -- 172.21.15.46:0/148043522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc15da60 con 0x7fedbc0d5b90 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.337+0000 7fedbaa00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 0x7fedbc15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedb3e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7feda40010e0 tx=0x7feda400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4017070 con 0x7fedbc0d5b90 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4005d00 con 0x7fedbc0d5b90 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda40072b0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc0cdb00 con 0x7fedbc0d5b90 2024-04-26T23:54:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.338+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc0ce0c0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.340+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc069530 con 0x7fedbc0d5b90 2024-04-26T23:54:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.340+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feda401a070 con 0x7fedbc0d5b90 2024-04-26T23:54:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.342+0000 7fedb3400640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 0x7fed9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.342+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7feda408af00 con 0x7fedbc0d5b90 2024-04-26T23:54:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.343+0000 7fedbaa00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 0x7fed9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.344+0000 7fedbaa00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 0x7fed9806edb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fedac034000 tx=0x7fedac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.347+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feda40593c0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.556+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fedbc161ff0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.562+0000 7fedb3400640 1 -- 172.21.15.46:0/148043522 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7feda4058cc0 con 0x7fedbc0d5b90 2024-04-26T23:54:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 msgr2=0x7fed9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 0x7fed9806edb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fedac034000 tx=0x7fedac02e000 comp rx=0 tx=0).stop 2024-04-26T23:54:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 msgr2=0x7fedbc0cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7feda40010e0 tx=0x7feda400c960 comp rx=0 tx=0).stop 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 shutdown_connections 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fed9806c8f0 0x7fed9806edb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedbc162aa0 0x7fedbc0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedbc0d5b90 0x7fedbc0cb360 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 --2- 172.21.15.46:0/148043522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedbc0d3b90 0x7fedbc15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 >> 172.21.15.46:0/148043522 conn(0x7fedbc0d5070 msgr2=0x7fedbc15e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 shutdown_connections 2024-04-26T23:54:13.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.566+0000 7fedc1e00640 1 -- 172.21.15.46:0/148043522 wait complete. 2024-04-26T23:54:13.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-26T23:54:13.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.844+0000 7f28dbe00640 1 Processor -- start 2024-04-26T23:54:13.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.844+0000 7f28dbe00640 1 -- start start 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.845+0000 7f28dbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28dc1454a0 0x7f28dc0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc073970 0x7f28dc0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28dbe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28dc072bb0 con 0x7f28dc0c6a50 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28dbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f28dc072e90 con 0x7f28dc1454a0 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc073170 con 0x7f28dc073970 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28dc1454a0 0x7f28dc0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28dc1454a0 0x7f28dc0cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54894/0 (socket says 172.21.15.46:54894) 2024-04-26T23:54:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 -- 172.21.15.46:0/3055219828 learned_addr learned my addr 172.21.15.46:0/3055219828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28da000640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc073970 0x7f28dc0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 -- 172.21.15.46:0/3055219828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc073970 msgr2=0x7f28dc0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc073970 0x7f28dc0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 -- 172.21.15.46:0/3055219828 >> v1:172.21.15.142:6789/0 conn(0x7f28dc0c6a50 legacy=0x7f28dc0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.846+0000 7f28daa00640 1 -- 172.21.15.46:0/3055219828 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc073450 con 0x7f28dc1454a0 2024-04-26T23:54:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.848+0000 7f28daa00640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc1454a0 0x7f28dc0cc690 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009870 tx=0x7f28cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b054989c4299fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.848+0000 7f28d9600640 1 -- 172.21.15.46:0/3055219828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc037070 con 0x7f28dc1454a0 2024-04-26T23:54:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.848+0000 7f28d9600640 1 -- 172.21.15.46:0/3055219828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28cc032440 con 0x7f28dc1454a0 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.848+0000 7f28d9600640 1 -- 172.21.15.46:0/3055219828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc032810 con 0x7f28dc1454a0 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 -- 172.21.15.46:0/3055219828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc1454a0 msgr2=0x7f28dc0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc1454a0 0x7f28dc0cc690 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009870 tx=0x7f28cc02a210 comp rx=0 tx=0).stop 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 -- 172.21.15.46:0/3055219828 shutdown_connections 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc073970 0x7f28dc0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 --2- 172.21.15.46:0/3055219828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc1454a0 0x7f28dc0cc690 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 -- 172.21.15.46:0/3055219828 >> 172.21.15.46:0/3055219828 conn(0x7f28dc0b26d0 msgr2=0x7f28dc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 -- 172.21.15.46:0/3055219828 shutdown_connections 2024-04-26T23:54:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.849+0000 7f28dbe00640 1 -- 172.21.15.46:0/3055219828 wait complete. 2024-04-26T23:54:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.851+0000 7f28dbe00640 1 Processor -- start 2024-04-26T23:54:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.851+0000 7f28dbe00640 1 -- start start 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc0c6a50 0x7f28dc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc1454a0 0x7f28dc15de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc073dd0 con 0x7f28dc073970 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc072880 con 0x7f28dc1454a0 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28dbe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc15e790 con 0x7f28dc0c6a50 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54908/0 (socket says 172.21.15.46:54908) 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 -- 172.21.15.46:0/2006006336 learned_addr learned my addr 172.21.15.46:0/2006006336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 -- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc0c6a50 msgr2=0x7f28dc15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc0c6a50 0x7f28dc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 -- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc1454a0 msgr2=0x7f28dc15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc1454a0 0x7f28dc15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.852+0000 7f28daa00640 1 -- 172.21.15.46:0/2006006336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc15b3b0 con 0x7f28dc073970 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.853+0000 7f28daa00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.853+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc041020 con 0x7f28dc073970 2024-04-26T23:54:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.853+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28cc030800 con 0x7f28dc073970 2024-04-26T23:54:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.853+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc037040 con 0x7f28dc073970 2024-04-26T23:54:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.853+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc15e4b0 con 0x7f28dc073970 2024-04-26T23:54:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.854+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc15c6d0 con 0x7f28dc073970 2024-04-26T23:54:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.855+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28dc0693b0 con 0x7f28dc073970 2024-04-26T23:54:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.856+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28cc044020 con 0x7f28dc073970 2024-04-26T23:54:13.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.857+0000 7f28d2a00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:13.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.857+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7f28cc0ac380 con 0x7f28dc073970 2024-04-26T23:54:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.858+0000 7f28da000640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.860+0000 7f28da000640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f28c8006fd0 tx=0x7f28c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:13.861+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28cc04b030 con 0x7f28dc073970 2024-04-26T23:54:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:14.197+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) -- 0x7f28dc0cefa0 con 0x7f28dc073970 2024-04-26T23:54:15.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.109+0000 7f28d2a00640 1 -- 172.21.15.46:0/2006006336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v782) ==== 166+0+0 (secure 0 0 0) 0x7f28cc07a830 con 0x7f28dc073970 2024-04-26T23:54:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 msgr2=0x7f28bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f28c8006fd0 tx=0x7f28c8008040 comp rx=0 tx=0).stop 2024-04-26T23:54:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 msgr2=0x7f28dc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 shutdown_connections 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc1454a0 0x7f28dc15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28dc0c6a50 0x7f28dc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.112+0000 7f28dbe00640 1 --2- 172.21.15.46:0/2006006336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28dc073970 0x7f28dc15ced0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.113+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 >> 172.21.15.46:0/2006006336 conn(0x7f28dc0b26d0 msgr2=0x7f28dc06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.113+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 shutdown_connections 2024-04-26T23:54:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.113+0000 7f28dbe00640 1 -- 172.21.15.46:0/2006006336 wait complete. 2024-04-26T23:54:15.136 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-26T23:54:15.137 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-26T23:54:15.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.343+0000 7f654d200640 1 Processor -- start 2024-04-26T23:54:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.344+0000 7f654d200640 1 -- start start 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.345+0000 7f654d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f65480d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.345+0000 7f654d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f65480b4000 0x7f65480b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.345+0000 7f654d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65480d11f0 con 0x7f654815f4e0 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.345+0000 7f654d200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f65480d0ce0 con 0x7f65480b4000 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.345+0000 7f654d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65480d2990 con 0x7f65480d2130 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6547400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f654815f4e0 0x7f65480d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49138/0 (socket says 172.21.15.46:49138) 2024-04-26T23:54:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6547400640 1 -- 172.21.15.46:0/946217414 learned_addr learned my addr 172.21.15.46:0/946217414 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:15.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6546a00640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f65480d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6547e00640 1 --2- 172.21.15.46:0/946217414 >> v2:172.21.15.142:3300/0 conn(0x7f65480b4000 0x7f65480b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392708904 0 0) 0x7f65480d11f0 con 0x7f654815f4e0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.346+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003520 con 0x7f654815f4e0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 825406363 0 0) 0x7f652c003520 con 0x7f654815f4e0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 msgr2=0x7f65480d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f65480d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 >> v2:172.21.15.142:3300/0 conn(0x7f65480b4000 msgr2=0x7f65480b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 --2- 172.21.15.46:0/946217414 >> v2:172.21.15.142:3300/0 conn(0x7f65480b4000 0x7f65480b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65480d5f20 con 0x7f654815f4e0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6538001670 con 0x7f654815f4e0 2024-04-26T23:54:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 >> v1:172.21.15.46:6789/0 conn(0x7f654815f4e0 legacy=0x7f65480d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c0039b0 0x7f652c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f652c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65480d2990 con 0x7f652c0039b0 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f652c00b9c0 con 0x7f652c004800 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f652c00bb40 con 0x7f65480d2130 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6547400640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c0039b0 0x7f652c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6546a00640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.347+0000 7f6547e00640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f652c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.348+0000 7f6547400640 1 -- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 msgr2=0x7f652c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.348+0000 7f6547400640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65480d2130 0x7f652c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.348+0000 7f6547400640 1 -- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f652c004800 msgr2=0x7f652c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.348+0000 7f6547400640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.348+0000 7f6547400640 1 -- 172.21.15.46:0/946217414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c003520 con 0x7f652c0039b0 2024-04-26T23:54:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f6547400640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c0039b0 0x7f652c003dd0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f653800c160 tx=0x7f652c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545a7dc73086cf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538007c80 con 0x7f652c0039b0 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6538005ad0 con 0x7f652c0039b0 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f6546000640 1 -- 172.21.15.46:0/946217414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538007070 con 0x7f652c0039b0 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f654d200640 1 -- 172.21.15.46:0/946217414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c0039b0 msgr2=0x7f652c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f654d200640 1 --2- 172.21.15.46:0/946217414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c0039b0 0x7f652c003dd0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f653800c160 tx=0x7f652c005830 comp rx=0 tx=0).stop 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.349+0000 7f6547e00640 1 -- 172.21.15.46:0/946217414 reap_dead start 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.350+0000 7f654d200640 1 -- 172.21.15.46:0/946217414 shutdown_connections 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.350+0000 7f654d200640 1 -- 172.21.15.46:0/946217414 >> 172.21.15.46:0/946217414 conn(0x7f65480d5010 msgr2=0x7f65480d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.350+0000 7f654d200640 1 -- 172.21.15.46:0/946217414 shutdown_connections 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.350+0000 7f654d200640 1 -- 172.21.15.46:0/946217414 wait complete. 2024-04-26T23:54:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.351+0000 7f654d200640 1 Processor -- start 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 -- start start 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f652c0039b0 0x7f654815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 0x7f654806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65480b1d60 con 0x7f65480cb5f0 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65480d5c40 con 0x7f654806cd40 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.352+0000 7f654d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f654815f8c0 con 0x7f652c0039b0 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54926/0 (socket says 172.21.15.46:54926) 2024-04-26T23:54:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 -- 172.21.15.46:0/1409948121 learned_addr learned my addr 172.21.15.46:0/1409948121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6547e00640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 0x7f654806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6547400640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f652c0039b0 0x7f654815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 -- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f652c0039b0 msgr2=0x7f654815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f652c0039b0 0x7f654815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 -- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 msgr2=0x7f654806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 0x7f654806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6546a00640 1 -- 172.21.15.46:0/1409948121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65480cd660 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.353+0000 7f6547e00640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 0x7f654806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f6546a00640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f6534011540 tx=0x7f6534011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534015930 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6534015ad0 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400f460 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654815e250 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.354+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65480ccd30 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.355+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65480693b0 con 0x7f65480cb5f0 2024-04-26T23:54:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.356+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f653400f600 con 0x7f65480cb5f0 2024-04-26T23:54:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.357+0000 7f653f400640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 0x7f652806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.357+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7f653408c350 con 0x7f65480cb5f0 2024-04-26T23:54:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.358+0000 7f6547400640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 0x7f652806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.359+0000 7f6547400640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 0x7f652806ede0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6538032000 tx=0x7f6538009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.362+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f654815e250 con 0x7f65480cb5f0 2024-04-26T23:54:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.569+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65480d33d0 con 0x7f65480cb5f0 2024-04-26T23:54:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.571+0000 7f653f400640 1 -- 172.21.15.46:0/1409948121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+13395 (secure 0 0 0) 0x7f6534019030 con 0x7f65480cb5f0 2024-04-26T23:54:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:15.575 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":165,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:13.084830+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/506880201":"2024-04-27T23:54:09.768630+0000","172.21.15.46:6832/506880201":"2024-04-27T23:54:09.768630+0000","172.21.15.46:6835/1585151449":"2024-04-27T23:53:53.908053+0000","172.21.15.46:6833/1585151449":"2024-04-27T23:53:53.908053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 msgr2=0x7f652806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 0x7f652806ede0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6538032000 tx=0x7f6538009030 comp rx=0 tx=0).stop 2024-04-26T23:54:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 msgr2=0x7f65480cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f6534011540 tx=0x7f6534011a00 comp rx=0 tx=0).stop 2024-04-26T23:54:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 shutdown_connections 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f652806c920 0x7f652806ede0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654806cd40 0x7f654806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480cb5f0 0x7f65480cc0e0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 --2- 172.21.15.46:0/1409948121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f652c0039b0 0x7f654815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 >> 172.21.15.46:0/1409948121 conn(0x7f65480d5010 msgr2=0x7f65480b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 shutdown_connections 2024-04-26T23:54:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.574+0000 7f654d200640 1 -- 172.21.15.46:0/1409948121 wait complete. 2024-04-26T23:54:15.597 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:54:15.598 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.812+0000 7f195f200640 1 Processor -- start 2024-04-26T23:54:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.812+0000 7f195f200640 1 -- start start 2024-04-26T23:54:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196015cec0 0x7f19600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1960073d00 0x7f19600b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19600ce7d0 con 0x7f19600cdb30 2024-04-26T23:54:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f19600ceab0 con 0x7f196015cec0 2024-04-26T23:54:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f19600ced90 con 0x7f1960073d00 2024-04-26T23:54:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.814+0000 7f195de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196015cec0 0x7f19600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196015cec0 0x7f19600cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54930/0 (socket says 172.21.15.46:54930) 2024-04-26T23:54:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 -- 172.21.15.46:0/944143402 learned_addr learned my addr 172.21.15.46:0/944143402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195e800640 1 --2- 172.21.15.46:0/944143402 >> v2:172.21.15.142:3300/0 conn(0x7f1960073d00 0x7f19600b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 -- 172.21.15.46:0/944143402 >> v2:172.21.15.142:3300/0 conn(0x7f1960073d00 msgr2=0x7f19600b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 --2- 172.21.15.46:0/944143402 >> v2:172.21.15.142:3300/0 conn(0x7f1960073d00 0x7f19600b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 -- 172.21.15.46:0/944143402 >> v1:172.21.15.142:6789/0 conn(0x7f19600cdb30 legacy=0x7f19600735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:15.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.815+0000 7f195de00640 1 -- 172.21.15.46:0/944143402 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600cf070 con 0x7f196015cec0 2024-04-26T23:54:15.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.817+0000 7f195de00640 1 --2- 172.21.15.46:0/944143402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196015cec0 0x7f19600cd5f0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f19600cd4f0 tx=0x7f195002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca5176fc2b77569 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.817+0000 7f195d400640 1 -- 172.21.15.46:0/944143402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037070 con 0x7f196015cec0 2024-04-26T23:54:15.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.817+0000 7f195d400640 1 -- 172.21.15.46:0/944143402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1950032440 con 0x7f196015cec0 2024-04-26T23:54:15.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.817+0000 7f195d400640 1 -- 172.21.15.46:0/944143402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950032810 con 0x7f196015cec0 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 -- 172.21.15.46:0/944143402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196015cec0 msgr2=0x7f19600cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 --2- 172.21.15.46:0/944143402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196015cec0 0x7f19600cd5f0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f19600cd4f0 tx=0x7f195002a230 comp rx=0 tx=0).stop 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 -- 172.21.15.46:0/944143402 shutdown_connections 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 --2- 172.21.15.46:0/944143402 >> v2:172.21.15.142:3300/0 conn(0x7f1960073d00 0x7f19600b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 --2- 172.21.15.46:0/944143402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196015cec0 0x7f19600cd5f0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 -- 172.21.15.46:0/944143402 >> 172.21.15.46:0/944143402 conn(0x7f19600b27b0 msgr2=0x7f19600b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 -- 172.21.15.46:0/944143402 shutdown_connections 2024-04-26T23:54:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.818+0000 7f195f200640 1 -- 172.21.15.46:0/944143402 wait complete. 2024-04-26T23:54:15.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.820+0000 7f195f200640 1 Processor -- start 2024-04-26T23:54:15.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.821+0000 7f195f200640 1 -- start start 2024-04-26T23:54:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 0x7f19600c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196015cec0 0x7f19600c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19600b4080 con 0x7f1960073d00 2024-04-26T23:54:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f196015d2a0 con 0x7f19600cdb30 2024-04-26T23:54:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19600b4200 con 0x7f196015cec0 2024-04-26T23:54:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54944/0 (socket says 172.21.15.46:54944) 2024-04-26T23:54:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.822+0000 7f195de00640 1 -- 172.21.15.46:0/4102706438 learned_addr learned my addr 172.21.15.46:0/4102706438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:15.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f1957e00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 0x7f19600c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195e800640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196015cec0 0x7f19600c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 -- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196015cec0 msgr2=0x7f19600c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196015cec0 0x7f19600c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 -- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 msgr2=0x7f19600c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 0x7f19600c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 -- 172.21.15.46:0/4102706438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600cb2f0 con 0x7f1960073d00 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f1957e00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 0x7f19600c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.823+0000 7f195de00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f195002a270 tx=0x7f19500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.824+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950041020 con 0x7f1960073d00 2024-04-26T23:54:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.824+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19500306c0 con 0x7f1960073d00 2024-04-26T23:54:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.824+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037040 con 0x7f1960073d00 2024-04-26T23:54:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.824+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19600cb560 con 0x7f1960073d00 2024-04-26T23:54:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.824+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19600cbb20 con 0x7f1960073d00 2024-04-26T23:54:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.825+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19600693b0 con 0x7f1960073d00 2024-04-26T23:54:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.826+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1950044020 con 0x7f1960073d00 2024-04-26T23:54:15.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.827+0000 7f1956000640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.827+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7f19500ac300 con 0x7f1960073d00 2024-04-26T23:54:15.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.828+0000 7f1957e00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.829+0000 7f1957e00640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 0x7f193c06edb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f19600ca6c0 tx=0x7f1948009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:15.831+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f195004b030 con 0x7f1960073d00 2024-04-26T23:54:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.035+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1960160680 con 0x7f1960073d00 2024-04-26T23:54:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.036+0000 7f1956000640 1 -- 172.21.15.46:0/4102706438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) ==== 93+0+10 (secure 0 0 0) 0x7f195007a7b0 con 0x7f1960073d00 2024-04-26T23:54:16.039 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 msgr2=0x7f193c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 0x7f193c06edb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f19600ca6c0 tx=0x7f1948009040 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 msgr2=0x7f19600cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f195002a270 tx=0x7f19500046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 shutdown_connections 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196015cec0 0x7f19600c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19600cdb30 0x7f19600c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 --2- 172.21.15.46:0/4102706438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1960073d00 0x7f19600cca10 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.039+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 >> 172.21.15.46:0/4102706438 conn(0x7f19600b27b0 msgr2=0x7f1960157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.040+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 shutdown_connections 2024-04-26T23:54:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.040+0000 7f195f200640 1 -- 172.21.15.46:0/4102706438 wait complete. 2024-04-26T23:54:16.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-26T23:54:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.274+0000 7f489a000640 1 Processor -- start 2024-04-26T23:54:16.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.274+0000 7f489a000640 1 -- start start 2024-04-26T23:54:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f489a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48940cf450 0x7f4894073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f489a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f489406f1d0 con 0x7f4894072190 2024-04-26T23:54:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f489a000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f489406f470 con 0x7f48940cff40 2024-04-26T23:54:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f489a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4894072930 con 0x7f48940cf450 2024-04-26T23:54:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f488be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4894072190 0x7f48940cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49140/0 (socket says 172.21.15.46:49140) 2024-04-26T23:54:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.276+0000 7f488be00640 1 -- 172.21.15.46:0/3686115231 learned_addr learned my addr 172.21.15.46:0/3686115231 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.277+0000 7f4898c00640 1 --2- 172.21.15.46:0/3686115231 >> v2:172.21.15.142:3300/0 conn(0x7f48940cf450 0x7f4894073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.277+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267916121 0 0) 0x7f489406f1d0 con 0x7f4894072190 2024-04-26T23:54:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.277+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4878003520 con 0x7f4894072190 2024-04-26T23:54:16.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.277+0000 7f4898c00640 1 -- 172.21.15.46:0/3686115231 >> v1:172.21.15.142:6789/0 conn(0x7f48940cff40 legacy=0x7f4894166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.278+0000 7f4898c00640 1 -- 172.21.15.46:0/3686115231 >> v1:172.21.15.46:6789/0 conn(0x7f4894072190 legacy=0x7f48940cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.278+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1097184280 0 0) 0x7f4878003520 con 0x7f4894072190 2024-04-26T23:54:16.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.278+0000 7f4898c00640 1 -- 172.21.15.46:0/3686115231 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4894072c10 con 0x7f48940cf450 2024-04-26T23:54:16.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.279+0000 7f4898c00640 1 --2- 172.21.15.46:0/3686115231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cf450 0x7f4894073db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f48840097a0 tx=0x7f488402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6662c480a31c3fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:16.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.279+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884037070 con 0x7f48940cf450 2024-04-26T23:54:16.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.279+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488402ac50 con 0x7f48940cf450 2024-04-26T23:54:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.279+0000 7f4893e00640 1 -- 172.21.15.46:0/3686115231 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c001670 con 0x7f48940cf450 2024-04-26T23:54:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 -- 172.21.15.46:0/3686115231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cf450 msgr2=0x7f4894073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 --2- 172.21.15.46:0/3686115231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cf450 0x7f4894073db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f48840097a0 tx=0x7f488402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 -- 172.21.15.46:0/3686115231 shutdown_connections 2024-04-26T23:54:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 --2- 172.21.15.46:0/3686115231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cf450 0x7f4894073db0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 -- 172.21.15.46:0/3686115231 >> 172.21.15.46:0/3686115231 conn(0x7f48940b2820 msgr2=0x7f48940b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 -- 172.21.15.46:0/3686115231 shutdown_connections 2024-04-26T23:54:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.280+0000 7f489a000640 1 -- 172.21.15.46:0/3686115231 wait complete. 2024-04-26T23:54:16.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.282+0000 7f489a000640 1 Processor -- start 2024-04-26T23:54:16.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.282+0000 7f489a000640 1 -- start start 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894072190 0x7f48941e7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cff40 0x7f4894144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48940734b0 con 0x7f48940cf450 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48940731d0 con 0x7f48940cff40 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f489a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4894072620 con 0x7f4894072190 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.283+0000 7f488be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54950/0 (socket says 172.21.15.46:54950) 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 -- 172.21.15.46:0/3030703802 learned_addr learned my addr 172.21.15.46:0/3030703802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f4898c00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894072190 0x7f48941e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 -- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894072190 msgr2=0x7f48941e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894072190 0x7f48941e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 -- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cff40 msgr2=0x7f4894144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cff40 0x7f4894144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 -- 172.21.15.46:0/3030703802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4894143e20 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f488be00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c580 tx=0x7f487c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.284+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c017070 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.285+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487c005ce0 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.285+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4894144120 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.285+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c007330 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.285+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941446e0 con 0x7f48940cf450 2024-04-26T23:54:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.286+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48940693b0 con 0x7f48940cf450 2024-04-26T23:54:16.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.287+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f487c01a070 con 0x7f48940cf450 2024-04-26T23:54:16.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.288+0000 7f4891600640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 0x7f487006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:16.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.289+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7f487c08afe0 con 0x7f48940cf450 2024-04-26T23:54:16.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.289+0000 7f4898c00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 0x7f487006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:16.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.290+0000 7f4898c00640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 0x7f487006edb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4884034000 tx=0x7f488402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.293+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f487c0594a0 con 0x7f48940cf450 2024-04-26T23:54:16.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:16.581+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) -- 0x7f4894141f70 con 0x7f48940cf450 2024-04-26T23:54:17.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.120+0000 7f4891600640 1 -- 172.21.15.46:0/3030703802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v784) ==== 143+0+0 (secure 0 0 0) 0x7f487c058da0 con 0x7f48940cf450 2024-04-26T23:54:17.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.124+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 msgr2=0x7f487006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.124+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 0x7f487006edb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4884034000 tx=0x7f488402e000 comp rx=0 tx=0).stop 2024-04-26T23:54:17.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.124+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 msgr2=0x7f4894140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.124+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c580 tx=0x7f487c00ca40 comp rx=0 tx=0).stop 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 shutdown_connections 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f487006c8f0 0x7f487006edb0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48940cff40 0x7f4894144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48940cf450 0x7f4894140af0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 --2- 172.21.15.46:0/3030703802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894072190 0x7f48941e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 >> 172.21.15.46:0/3030703802 conn(0x7f48940b2820 msgr2=0x7f48940cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:17.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 shutdown_connections 2024-04-26T23:54:17.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.125+0000 7f489a000640 1 -- 172.21.15.46:0/3030703802 wait complete. 2024-04-26T23:54:17.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:54:17.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.375+0000 7fcdc2a00640 1 Processor -- start 2024-04-26T23:54:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.375+0000 7fcdc2a00640 1 -- start start 2024-04-26T23:54:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.377+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 0x7fcdbc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.377+0000 7fcdc2a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcdbc0b5fb0 0x7fcdbc0d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.377+0000 7fcdc2a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdbc0d07d0 con 0x7fcdbc162aa0 2024-04-26T23:54:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.377+0000 7fcdc2a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcdbc0d0ab0 con 0x7fcdbc0b5fb0 2024-04-26T23:54:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.377+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc0d0d90 con 0x7fcdbc1626c0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdc2000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcdbc162aa0 0x7fcdbc0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49142/0 (socket says 172.21.15.46:49142) 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdc2000640 1 -- 172.21.15.46:0/2454084324 learned_addr learned my addr 172.21.15.46:0/2454084324 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdc1600640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 0x7fcdbc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdc0c00640 1 --2- 172.21.15.46:0/2454084324 >> v2:172.21.15.142:3300/0 conn(0x7fcdbc0b5fb0 0x7fcdbc0d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887097183 0 0) 0x7fcdbc0d07d0 con 0x7fcdbc162aa0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.378+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcda4003520 con 0x7fcdbc162aa0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2408219488 0 0) 0x7fcda4003520 con 0x7fcdbc162aa0 2024-04-26T23:54:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 msgr2=0x7fcdbc0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 0x7fcdbc0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 >> v2:172.21.15.142:3300/0 conn(0x7fcdbc0b5fb0 msgr2=0x7fcdbc0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 --2- 172.21.15.46:0/2454084324 >> v2:172.21.15.142:3300/0 conn(0x7fcdbc0b5fb0 0x7fcdbc0d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc0d2c80 con 0x7fcdbc162aa0 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcdb40019f0 con 0x7fcdbc162aa0 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 >> v1:172.21.15.46:6789/0 conn(0x7fcdbc162aa0 legacy=0x7fcdbc0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcda4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 0x7fcda4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda4004d40 0x7fcda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0d0ab0 con 0x7fcda40039b0 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0d0d90 con 0x7fcda4004d40 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcda400b9c0 con 0x7fcdbc1626c0 2024-04-26T23:54:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.379+0000 7fcdc1600640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcda4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc1600640 1 -- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 msgr2=0x7fcda4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc1600640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc1626c0 0x7fcda4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc1600640 1 -- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda4004d40 msgr2=0x7fcda40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc2000640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda4004d40 0x7fcda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc1600640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda4004d40 0x7fcda40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.380+0000 7fcdc1600640 1 -- 172.21.15.46:0/2454084324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda4003520 con 0x7fcda40039b0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.381+0000 7fcdc1600640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcda4003dd0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ac0 tx=0x7fcda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56f94b68b5d278d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.381+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0036070 con 0x7fcda40039b0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.381+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb0031400 con 0x7fcda40039b0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.381+0000 7fcdbbe00640 1 -- 172.21.15.46:0/2454084324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0009690 con 0x7fcda40039b0 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 -- 172.21.15.46:0/2454084324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 msgr2=0x7fcda4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/2454084324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcda4003dd0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ac0 tx=0x7fcda4005830 comp rx=0 tx=0).stop 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2000640 1 -- 172.21.15.46:0/2454084324 reap_dead start 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 -- 172.21.15.46:0/2454084324 shutdown_connections 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 -- 172.21.15.46:0/2454084324 >> 172.21.15.46:0/2454084324 conn(0x7fcdbc09e2a0 msgr2=0x7fcdbc0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 -- 172.21.15.46:0/2454084324 shutdown_connections 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.382+0000 7fcdc2a00640 1 -- 172.21.15.46:0/2454084324 wait complete. 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.384+0000 7fcdc2a00640 1 Processor -- start 2024-04-26T23:54:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.384+0000 7fcdc2a00640 1 -- start start 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc156d70 0x7fcdbc1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdbc1f9390 0x7fcdbc158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0d2190 con 0x7fcda40039b0 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0d2010 con 0x7fcdbc1f9390 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc0b47c0 con 0x7fcdbc156d70 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54972/0 (socket says 172.21.15.46:54972) 2024-04-26T23:54:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 -- 172.21.15.46:0/1661609241 learned_addr learned my addr 172.21.15.46:0/1661609241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:17.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 -- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc156d70 msgr2=0x7fcdbc1f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc156d70 0x7fcdbc1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 -- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdbc1f9390 msgr2=0x7fcdbc158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdbc1f9390 0x7fcdbc158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.385+0000 7fcdc1600640 1 -- 172.21.15.46:0/1661609241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc1f8950 con 0x7fcda40039b0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdc1600640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ac0 tx=0x7fcdb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0040020 con 0x7fcda40039b0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb0003bc0 con 0x7fcda40039b0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0036040 con 0x7fcda40039b0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc0cc250 con 0x7fcda40039b0 2024-04-26T23:54:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.386+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc0ccee0 con 0x7fcda40039b0 2024-04-26T23:54:17.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.388+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc0cbfe0 con 0x7fcda40039b0 2024-04-26T23:54:17.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.388+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdb0043020 con 0x7fcda40039b0 2024-04-26T23:54:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.389+0000 7fcdb9600640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.389+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7fcdb00abc10 con 0x7fcda40039b0 2024-04-26T23:54:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.390+0000 7fcdc0c00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.391+0000 7fcdc0c00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009370 tx=0x7fcda80092a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.394+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdb004a070 con 0x7fcda40039b0 2024-04-26T23:54:17.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.599+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcdbc0693b0 con 0x7fcda40039b0 2024-04-26T23:54:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.605+0000 7fcdb9600640 1 -- 172.21.15.46:0/1661609241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fcdb00044e0 con 0x7fcda40039b0 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.607+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 msgr2=0x7fcd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.607+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009370 tx=0x7fcda80092a0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.607+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 msgr2=0x7fcdbc0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.607+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ac0 tx=0x7fcdb0004730 comp rx=0 tx=0).stop 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 shutdown_connections 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdbc1f9390 0x7fcdbc158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdbc156d70 0x7fcdbc1f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 --2- 172.21.15.46:0/1661609241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40039b0 0x7fcdbc0cc9a0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 >> 172.21.15.46:0/1661609241 conn(0x7fcdbc09e2a0 msgr2=0x7fcdbc0d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 shutdown_connections 2024-04-26T23:54:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:17.608+0000 7fcdc2a00640 1 -- 172.21.15.46:0/1661609241 wait complete. 2024-04-26T23:54:17.631 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:54:17.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:54:17.731 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:54:17.850 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:54:17.851 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:17.851 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:17.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:54:17.910 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:54:17.910 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:17.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:54:17.969 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:17.969 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:18.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:54:18.030 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:54:18.030 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:18.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:54:18.088 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:54:18.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.296+0000 7fe887200640 1 Processor -- start 2024-04-26T23:54:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.296+0000 7fe887200640 1 -- start start 2024-04-26T23:54:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe887200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe888148b80 0x7fe8880af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe887200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe88806fdc0 0x7fe8880722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe887200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8880b5580 con 0x7fe8880b4ea0 2024-04-26T23:54:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe887200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe8880b5330 con 0x7fe888148b80 2024-04-26T23:54:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe887200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8880b5860 con 0x7fe88806fdc0 2024-04-26T23:54:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.298+0000 7fe885e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe888148b80 0x7fe8880af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe888148b80 0x7fe8880af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54980/0 (socket says 172.21.15.46:54980) 2024-04-26T23:54:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 -- 172.21.15.46:0/2702662896 learned_addr learned my addr 172.21.15.46:0/2702662896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe886800640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe88806fdc0 0x7fe8880722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 -- 172.21.15.46:0/2702662896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe88806fdc0 msgr2=0x7fe8880722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe88806fdc0 0x7fe8880722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 -- 172.21.15.46:0/2702662896 >> v1:172.21.15.142:6789/0 conn(0x7fe8880b4ea0 legacy=0x7fe88806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.299+0000 7fe885e00640 1 -- 172.21.15.46:0/2702662896 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8880b5b40 con 0x7fe888148b80 2024-04-26T23:54:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.300+0000 7fe885e00640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe8880af7d0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe87802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1321e8dde1f73f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.300+0000 7fe885400640 1 -- 172.21.15.46:0/2702662896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878037070 con 0x7fe888148b80 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.300+0000 7fe885400640 1 -- 172.21.15.46:0/2702662896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878032440 con 0x7fe888148b80 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe885400640 1 -- 172.21.15.46:0/2702662896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878032810 con 0x7fe888148b80 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 -- 172.21.15.46:0/2702662896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 msgr2=0x7fe8880af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe8880af7d0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe87802a250 comp rx=0 tx=0).stop 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 -- 172.21.15.46:0/2702662896 shutdown_connections 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe88806fdc0 0x7fe8880722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 --2- 172.21.15.46:0/2702662896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe8880af7d0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 -- 172.21.15.46:0/2702662896 >> 172.21.15.46:0/2702662896 conn(0x7fe88806e3c0 msgr2=0x7fe88806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 -- 172.21.15.46:0/2702662896 shutdown_connections 2024-04-26T23:54:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.301+0000 7fe887200640 1 -- 172.21.15.46:0/2702662896 wait complete. 2024-04-26T23:54:18.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.303+0000 7fe887200640 1 Processor -- start 2024-04-26T23:54:18.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.303+0000 7fe887200640 1 -- start start 2024-04-26T23:54:18.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 0x7fe88815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8880b4ea0 0x7fe88815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880aeb10 con 0x7fe888148b80 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880b6100 con 0x7fe88806fdc0 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe887200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe888072830 con 0x7fe8880b4ea0 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.304+0000 7fe886800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54996/0 (socket says 172.21.15.46:54996) 2024-04-26T23:54:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 -- 172.21.15.46:0/10700986 learned_addr learned my addr 172.21.15.46:0/10700986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe87d400640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8880b4ea0 0x7fe88815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe885e00640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 0x7fe88815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 -- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8880b4ea0 msgr2=0x7fe88815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8880b4ea0 0x7fe88815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 -- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 msgr2=0x7fe88815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 0x7fe88815f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 -- 172.21.15.46:0/10700986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881f9c80 con 0x7fe888148b80 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe885e00640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 0x7fe88815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.305+0000 7fe886800640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe880007e10 tx=0x7fe88000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.306+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880014420 con 0x7fe888148b80 2024-04-26T23:54:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.306+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe880019070 con 0x7fe888148b80 2024-04-26T23:54:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.306+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881f8d90 con 0x7fe888148b80 2024-04-26T23:54:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.306+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880014820 con 0x7fe888148b80 2024-04-26T23:54:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.306+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881f9350 con 0x7fe888148b80 2024-04-26T23:54:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.307+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888069de0 con 0x7fe888148b80 2024-04-26T23:54:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.308+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8800149c0 con 0x7fe888148b80 2024-04-26T23:54:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.309+0000 7fe87ea00640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.309+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7fe88008c290 con 0x7fe888148b80 2024-04-26T23:54:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.310+0000 7fe885e00640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.311+0000 7fe885e00640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe878034000 tx=0x7fe87802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.314+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe888069de0 con 0x7fe888148b80 2024-04-26T23:54:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.566+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8881483e0 con 0x7fe888148b80 2024-04-26T23:54:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.567+0000 7fe87ea00640 1 -- 172.21.15.46:0/10700986 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) ==== 95+0+3055 (secure 0 0 0) 0x7fe8881483e0 con 0x7fe888148b80 2024-04-26T23:54:18.571 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:18.571 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":133,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1817842283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1817842283},{"type":"v1","addr":"172.21.15.142:6836","nonce":1817842283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1053470379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":1053470379},{"type":"v1","addr":"172.21.15.142:6837","nonce":1053470379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":10692,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2790456729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2790456729},{"type":"v1","addr":"172.21.15.46:6833","nonce":2790456729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":133}],"filesystems":[]} 2024-04-26T23:54:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 133 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.570+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 msgr2=0x7fe86406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.570+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe878034000 tx=0x7fe87802e000 comp rx=0 tx=0).stop 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.570+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 msgr2=0x7fe888160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.570+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe880007e10 tx=0x7fe88000e4b0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 shutdown_connections 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe888148b80 0x7fe888160d30 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8880b4ea0 0x7fe88815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 --2- 172.21.15.46:0/10700986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe88806fdc0 0x7fe88815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 >> 172.21.15.46:0/10700986 conn(0x7fe88806e3c0 msgr2=0x7fe888072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 shutdown_connections 2024-04-26T23:54:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.571+0000 7fe887200640 1 -- 172.21.15.46:0/10700986 wait complete. 2024-04-26T23:54:18.594 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:54:18.594 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:54:18.594 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:54:18.594 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:54:18.594 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:54:18.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:54:18.595 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:54:18.595 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:54:18.595 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:54:18.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:54:18.595 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:54:18.595 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:54:18.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:54:18.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:54:18.600 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:54:18.601 DEBUG:teuthology.parallel:result is None 2024-04-26T23:54:18.614 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:54:18.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:54:18.619 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:54:18.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:54:18.624 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:54:18.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:54:18.660 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:54:18.660 DEBUG:teuthology.parallel:result is None 2024-04-26T23:54:18.669 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:54:18.670 DEBUG:teuthology.parallel:result is None 2024-04-26T23:54:18.676 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:54:18.676 DEBUG:teuthology.parallel:result is None 2024-04-26T23:54:18.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:54:18.764 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:54:18.805 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:54:18.826 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:54:18.826 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:54:18.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.856+0000 7f9455400640 1 Processor -- start 2024-04-26T23:54:18.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.856+0000 7f9455400640 1 -- start start 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.857+0000 7f9455400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9450155560 0x7f9450170370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.857+0000 7f9455400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94500ce590 0x7f94500b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.857+0000 7f9455400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945016c210 con 0x7f94500b54d0 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.857+0000 7f9455400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9450072f00 con 0x7f94500ce590 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.857+0000 7f9455400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f94500722f0 con 0x7f9450155560 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.858+0000 7f9445e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94500ce590 0x7f94500b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.858+0000 7f9445e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94500ce590 0x7f94500b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55060/0 (socket says 172.21.15.46:55060) 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.858+0000 7f9445e00640 1 -- 172.21.15.46:0/941352973 learned_addr learned my addr 172.21.15.46:0/941352973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.858+0000 7f944de00640 1 -- 172.21.15.46:0/941352973 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805552233 0 0) 0x7f945016c210 con 0x7f94500b54d0 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.858+0000 7f944e800640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9450155560 0x7f9450170370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.930+0000 7f944de00640 1 -- 172.21.15.46:0/941352973 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9434007930 con 0x7f94500b54d0 2024-04-26T23:54:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.931+0000 7f9445e00640 1 -- 172.21.15.46:0/941352973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9450155560 msgr2=0x7f9450170370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.931+0000 7f9445e00640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9450155560 0x7f9450170370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.931+0000 7f9445e00640 1 -- 172.21.15.46:0/941352973 >> v1:172.21.15.142:6789/0 conn(0x7f94500b54d0 legacy=0x7f9450073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.931+0000 7f9445e00640 1 -- 172.21.15.46:0/941352973 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94500725d0 con 0x7f94500ce590 2024-04-26T23:54:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.933+0000 7f9445e00640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500ce590 0x7f94500b4f90 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f9438013d10 tx=0x7f9438032b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcecedf05a41a800 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.933+0000 7f944de00640 1 -- 172.21.15.46:0/941352973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438034870 con 0x7f94500ce590 2024-04-26T23:54:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.933+0000 7f944de00640 1 -- 172.21.15.46:0/941352973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943800ae50 con 0x7f94500ce590 2024-04-26T23:54:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.933+0000 7f944de00640 1 -- 172.21.15.46:0/941352973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438034c30 con 0x7f94500ce590 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 -- 172.21.15.46:0/941352973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500ce590 msgr2=0x7f94500b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500ce590 0x7f94500b4f90 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f9438013d10 tx=0x7f9438032b10 comp rx=0 tx=0).stop 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 -- 172.21.15.46:0/941352973 shutdown_connections 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500ce590 0x7f94500b4f90 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 --2- 172.21.15.46:0/941352973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9450155560 0x7f9450170370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.934+0000 7f9455400640 1 -- 172.21.15.46:0/941352973 >> 172.21.15.46:0/941352973 conn(0x7f945006cbd0 msgr2=0x7f94500b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.935+0000 7f9455400640 1 -- 172.21.15.46:0/941352973 shutdown_connections 2024-04-26T23:54:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.935+0000 7f9455400640 1 -- 172.21.15.46:0/941352973 wait complete. 2024-04-26T23:54:18.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.938+0000 7f9455400640 1 Processor -- start 2024-04-26T23:54:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.938+0000 7f9455400640 1 -- start start 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.940+0000 7f9455400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.940+0000 7f9455400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94500ce590 0x7f9450156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.940+0000 7f9455400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 0x7f945008b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f9455400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9450155940 con 0x7f94500b54d0 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f9455400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9450072b90 con 0x7f9450155560 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f9455400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f94500ce9f0 con 0x7f94500ce590 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55064/0 (socket says 172.21.15.46:55064) 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 -- 172.21.15.46:0/2472309612 learned_addr learned my addr 172.21.15.46:0/2472309612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f9445e00640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94500ce590 0x7f9450156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944f200640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 0x7f945008b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 -- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94500ce590 msgr2=0x7f9450156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94500ce590 0x7f9450156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 -- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 msgr2=0x7f945008b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 0x7f945008b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944e800640 1 -- 172.21.15.46:0/2472309612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9450202190 con 0x7f94500b54d0 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.941+0000 7f944f200640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 0x7f945008b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.942+0000 7f944e800640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f944000f8f0 tx=0x7f944000fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.942+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440017070 con 0x7f94500b54d0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.942+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944000dbd0 con 0x7f94500b54d0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.942+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000d040 con 0x7f94500b54d0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.942+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9450156df0 con 0x7f94500b54d0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.943+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501573b0 con 0x7f94500b54d0 2024-04-26T23:54:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.944+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9450069530 con 0x7f94500b54d0 2024-04-26T23:54:18.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.945+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f944001a070 con 0x7f94500b54d0 2024-04-26T23:54:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.946+0000 7f9447400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.946+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5421+0+0 (secure 0 0 0) 0x7f944008bdf0 con 0x7f94500b54d0 2024-04-26T23:54:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.947+0000 7f9445e00640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.948+0000 7f9445e00640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f943803c000 tx=0x7f9438040040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:18.953+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f944005a330 con 0x7f94500b54d0 2024-04-26T23:54:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:19.182+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9450073230 con 0x7f94500b54d0 2024-04-26T23:54:20.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.161+0000 7f9447400640 1 -- 172.21.15.46:0/2472309612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v169) ==== 97+0+0 (secure 0 0 0) 0x7f9440059c30 con 0x7f94500b54d0 2024-04-26T23:54:20.165 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:54:20.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.165+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 msgr2=0x7f942c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.165+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f943803c000 tx=0x7f9438040040 comp rx=0 tx=0).stop 2024-04-26T23:54:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.165+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 msgr2=0x7f9450201c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.165+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f944000f8f0 tx=0x7f944000fdb0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 shutdown_connections 2024-04-26T23:54:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450155560 0x7f945008b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94500ce590 0x7f9450156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 --2- 172.21.15.46:0/2472309612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94500b54d0 0x7f9450201c50 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 >> 172.21.15.46:0/2472309612 conn(0x7f945006cbd0 msgr2=0x7f94500b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 shutdown_connections 2024-04-26T23:54:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.166+0000 7f9455400640 1 -- 172.21.15.46:0/2472309612 wait complete. 2024-04-26T23:54:20.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:54:20.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.393+0000 7f0065400640 1 Processor -- start 2024-04-26T23:54:20.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.394+0000 7f0065400640 1 -- start start 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.395+0000 7f0065400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f00600cf450 0x7f0060073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.395+0000 7f0065400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00600cff40 0x7f0060166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.395+0000 7f0065400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006006f1d0 con 0x7f0060072190 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.395+0000 7f0065400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f006006f470 con 0x7f00600cff40 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.395+0000 7f0065400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0060072930 con 0x7f00600cf450 2024-04-26T23:54:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.396+0000 7f0057e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0060072190 0x7f00600cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49186/0 (socket says 172.21.15.46:49186) 2024-04-26T23:54:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.396+0000 7f0057e00640 1 -- 172.21.15.46:0/3036013615 learned_addr learned my addr 172.21.15.46:0/3036013615 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.396+0000 7f0064a00640 1 --2- 172.21.15.46:0/3036013615 >> v2:172.21.15.46:3300/0 conn(0x7f00600cff40 0x7f0060166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.396+0000 7f005fe00640 1 --2- 172.21.15.46:0/3036013615 >> v2:172.21.15.142:3300/0 conn(0x7f00600cf450 0x7f0060073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.396+0000 7f005f400640 1 -- 172.21.15.46:0/3036013615 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708632726 0 0) 0x7f006006f1d0 con 0x7f0060072190 2024-04-26T23:54:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.397+0000 7f005f400640 1 -- 172.21.15.46:0/3036013615 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0044003520 con 0x7f0060072190 2024-04-26T23:54:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.397+0000 7f0064a00640 1 -- 172.21.15.46:0/3036013615 >> v2:172.21.15.142:3300/0 conn(0x7f00600cf450 msgr2=0x7f0060073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.397+0000 7f0064a00640 1 --2- 172.21.15.46:0/3036013615 >> v2:172.21.15.142:3300/0 conn(0x7f00600cf450 0x7f0060073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.397+0000 7f0064a00640 1 -- 172.21.15.46:0/3036013615 >> v1:172.21.15.46:6789/0 conn(0x7f0060072190 legacy=0x7f00600cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.397+0000 7f0064a00640 1 -- 172.21.15.46:0/3036013615 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0060072c10 con 0x7f00600cff40 2024-04-26T23:54:20.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.398+0000 7f0064a00640 1 --2- 172.21.15.46:0/3036013615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cff40 0x7f0060166b00 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f006006f3a0 tx=0x7f005802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e52b034344545f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.398+0000 7f005f400640 1 -- 172.21.15.46:0/3036013615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058007740 con 0x7f00600cff40 2024-04-26T23:54:20.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.398+0000 7f005f400640 1 -- 172.21.15.46:0/3036013615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00580078e0 con 0x7f00600cff40 2024-04-26T23:54:20.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.398+0000 7f005f400640 1 -- 172.21.15.46:0/3036013615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058007cb0 con 0x7f00600cff40 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 -- 172.21.15.46:0/3036013615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cff40 msgr2=0x7f0060166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 --2- 172.21.15.46:0/3036013615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cff40 0x7f0060166b00 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f006006f3a0 tx=0x7f005802a660 comp rx=0 tx=0).stop 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 -- 172.21.15.46:0/3036013615 shutdown_connections 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 --2- 172.21.15.46:0/3036013615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cff40 0x7f0060166b00 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 --2- 172.21.15.46:0/3036013615 >> v2:172.21.15.142:3300/0 conn(0x7f00600cf450 0x7f0060073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 -- 172.21.15.46:0/3036013615 >> 172.21.15.46:0/3036013615 conn(0x7f00600b2820 msgr2=0x7f00600b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:20.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 -- 172.21.15.46:0/3036013615 shutdown_connections 2024-04-26T23:54:20.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.399+0000 7f0065400640 1 -- 172.21.15.46:0/3036013615 wait complete. 2024-04-26T23:54:20.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.401+0000 7f0065400640 1 Processor -- start 2024-04-26T23:54:20.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.401+0000 7f0065400640 1 -- start start 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0060072190 0x7f0060143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00600cff40 0x7f00601e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00600734b0 con 0x7f00600cf450 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00600731d0 con 0x7f00600cff40 2024-04-26T23:54:20.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0065400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0060072620 con 0x7f0060072190 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55088/0 (socket says 172.21.15.46:55088) 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.402+0000 7f0057e00640 1 -- 172.21.15.46:0/369599230 learned_addr learned my addr 172.21.15.46:0/369599230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0064a00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00600cff40 0x7f00601e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f005fe00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0060072190 0x7f0060143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 -- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0060072190 msgr2=0x7f0060143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0060072190 0x7f0060143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 -- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00600cff40 msgr2=0x7f00601e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00600cff40 0x7f00601e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 -- 172.21.15.46:0/369599230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601f50a0 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f0057e00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f004800c390 tx=0x7f004800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.403+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048017070 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.404+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0048005ce0 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.404+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00601f5380 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.404+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00480072f0 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.404+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601f5940 con 0x7f00600cf450 2024-04-26T23:54:20.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.405+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00600693b0 con 0x7f00600cf450 2024-04-26T23:54:20.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.406+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f004801a070 con 0x7f00600cf450 2024-04-26T23:54:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.407+0000 7f005cc00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 0x7f003c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.407+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5249+0+0 (secure 0 0 0) 0x7f004808ad30 con 0x7f00600cf450 2024-04-26T23:54:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.408+0000 7f005fe00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 0x7f003c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.409+0000 7f005fe00640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 0x7f003c06ede0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0060146cf0 tx=0x7f0050008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.412+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0048059320 con 0x7f00600cf450 2024-04-26T23:54:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.701+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f00600d0c50 con 0x7f00600cf450 2024-04-26T23:54:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.702+0000 7f005cc00640 1 -- 172.21.15.46:0/369599230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f0048058c20 con 0x7f00600cf450 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:54:20.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:54:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.712 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:54:20.713 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:54:20.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:54:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:54:20.717 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:54:20.718 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:54:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 msgr2=0x7f003c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 0x7f003c06ede0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0060146cf0 tx=0x7f0050008040 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 msgr2=0x7f00600b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f004800c390 tx=0x7f004800c850 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 shutdown_connections 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f003c06c920 0x7f003c06ede0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00600cff40 0x7f00601e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00600cf450 0x7f00600b4b70 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 --2- 172.21.15.46:0/369599230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0060072190 0x7f0060143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.705+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 >> 172.21.15.46:0/369599230 conn(0x7f00600b2820 msgr2=0x7f00600cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.706+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 shutdown_connections 2024-04-26T23:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.706+0000 7f0065400640 1 -- 172.21.15.46:0/369599230 wait complete. 2024-04-26T23:54:20.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.933+0000 7f571de00640 1 Processor -- start 2024-04-26T23:54:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.933+0000 7f571de00640 1 -- start start 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.934+0000 7f571de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f571815f4e0 0x7f57180d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.934+0000 7f571de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 0x7f57180d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.934+0000 7f571de00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57180d11f0 con 0x7f57180d1f60 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.934+0000 7f571de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f57180d2990 con 0x7f571815f4e0 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.934+0000 7f571de00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f57180d5f20 con 0x7f57180b4000 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f571815f4e0 0x7f57180d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f57180d1f60 0x7f57180cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49198/0 (socket says 172.21.15.46:49198) 2024-04-26T23:54:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571d400640 1 -- 172.21.15.46:0/326223644 learned_addr learned my addr 172.21.15.46:0/326223644 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f5717e00640 1 --2- 172.21.15.46:0/326223644 >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 0x7f57180d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571ca00640 1 -- 172.21.15.46:0/326223644 >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 msgr2=0x7f57180d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571ca00640 1 --2- 172.21.15.46:0/326223644 >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 0x7f57180d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571ca00640 1 -- 172.21.15.46:0/326223644 >> v1:172.21.15.46:6789/0 conn(0x7f57180d1f60 legacy=0x7f57180cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f5717400640 1 -- 172.21.15.46:0/326223644 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790473907 0 0) 0x7f57180d11f0 con 0x7f57180d1f60 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f571ca00640 1 -- 172.21.15.46:0/326223644 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180d0f10 con 0x7f571815f4e0 2024-04-26T23:54:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.935+0000 7f5717e00640 1 --2- 172.21.15.46:0/326223644 >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 0x7f57180d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.936+0000 7f571ca00640 1 --2- 172.21.15.46:0/326223644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571815f4e0 0x7f57180d2df0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f571808d880 tx=0x7f570002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbd34743e898bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.937+0000 7f5717400640 1 -- 172.21.15.46:0/326223644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700037070 con 0x7f571815f4e0 2024-04-26T23:54:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.937+0000 7f5717400640 1 -- 172.21.15.46:0/326223644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5700032440 con 0x7f571815f4e0 2024-04-26T23:54:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.937+0000 7f5717400640 1 -- 172.21.15.46:0/326223644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700032810 con 0x7f571815f4e0 2024-04-26T23:54:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 -- 172.21.15.46:0/326223644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571815f4e0 msgr2=0x7f57180d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 --2- 172.21.15.46:0/326223644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571815f4e0 0x7f57180d2df0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f571808d880 tx=0x7f570002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 -- 172.21.15.46:0/326223644 shutdown_connections 2024-04-26T23:54:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 --2- 172.21.15.46:0/326223644 >> v2:172.21.15.142:3300/0 conn(0x7f57180b4000 0x7f57180d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 --2- 172.21.15.46:0/326223644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571815f4e0 0x7f57180d2df0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 -- 172.21.15.46:0/326223644 >> 172.21.15.46:0/326223644 conn(0x7f57180d5010 msgr2=0x7f57180d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 -- 172.21.15.46:0/326223644 shutdown_connections 2024-04-26T23:54:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.938+0000 7f571de00640 1 -- 172.21.15.46:0/326223644 wait complete. 2024-04-26T23:54:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.940+0000 7f571de00640 1 Processor -- start 2024-04-26T23:54:20.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.940+0000 7f571de00640 1 -- start start 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57180d1f60 0x7f5718158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f571815f4e0 0x7f57180b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f571815f8c0 con 0x7f57180b4000 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57180b1d60 con 0x7f571815f4e0 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f57180b3290 con 0x7f57180d1f60 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55108/0 (socket says 172.21.15.46:55108) 2024-04-26T23:54:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 -- 172.21.15.46:0/2407430481 learned_addr learned my addr 172.21.15.46:0/2407430481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 -- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57180d1f60 msgr2=0x7f5718158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57180d1f60 0x7f5718158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571d400640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f571815f4e0 0x7f57180b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 -- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f571815f4e0 msgr2=0x7f57180b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f571815f4e0 0x7f57180b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.941+0000 7f571ca00640 1 -- 172.21.15.46:0/2407430481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57181f5030 con 0x7f57180b4000 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f571ca00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f5700009890 tx=0x7f57000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700040020 con 0x7f57180b4000 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5700038040 con 0x7f57180b4000 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700037070 con 0x7f57180b4000 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57181598d0 con 0x7f57180b4000 2024-04-26T23:54:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.942+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181f5590 con 0x7f57180b4000 2024-04-26T23:54:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.944+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57180693b0 con 0x7f57180b4000 2024-04-26T23:54:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.945+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5700043020 con 0x7f57180b4000 2024-04-26T23:54:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.946+0000 7f5714c00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 0x7f56f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.946+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5249+0+0 (secure 0 0 0) 0x7f57000ac140 con 0x7f57180b4000 2024-04-26T23:54:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.946+0000 7f5717e00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 0x7f56f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.947+0000 7f5717e00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 0x7f56f406edb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:20.950+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f570004a030 con 0x7f57180b4000 2024-04-26T23:54:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.158+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57180d33d0 con 0x7f57180b4000 2024-04-26T23:54:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.161+0000 7f5714c00640 1 -- 172.21.15.46:0/2407430481 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+13138 (secure 0 0 0) 0x7f57181598d0 con 0x7f57180b4000 2024-04-26T23:54:21.164 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:21.164 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":169,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:20.159498+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 msgr2=0x7f56f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 0x7f56f406edb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 msgr2=0x7f57180b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f5700009890 tx=0x7f57000046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 shutdown_connections 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f56f406c8f0 0x7f56f406edb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f571815f4e0 0x7f57180b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57180d1f60 0x7f5718158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 --2- 172.21.15.46:0/2407430481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57180b4000 0x7f57180b7c30 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.163+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 >> 172.21.15.46:0/2407430481 conn(0x7f57180d5010 msgr2=0x7f5718161200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.164+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 shutdown_connections 2024-04-26T23:54:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.164+0000 7f571de00640 1 -- 172.21.15.46:0/2407430481 wait complete. 2024-04-26T23:54:21.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:54:21.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.392+0000 7f62f7e00640 1 Processor -- start 2024-04-26T23:54:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.392+0000 7f62f7e00640 1 -- start start 2024-04-26T23:54:21.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.393+0000 7f62f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62f80cdd00 0x7f62f80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.393+0000 7f62f7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f80ce7d0 con 0x7f62f8073b30 2024-04-26T23:54:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.393+0000 7f62f7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f80ceab0 con 0x7f62f815cec0 2024-04-26T23:54:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.393+0000 7f62f7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f62f80ced90 con 0x7f62f80cdd00 2024-04-26T23:54:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62f80cdd00 0x7f62f80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f62f8073b30 0x7f62f80b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49210/0 (socket says 172.21.15.46:49210) 2024-04-26T23:54:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f7400640 1 -- 172.21.15.46:0/4233418383 learned_addr learned my addr 172.21.15.46:0/4233418383 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f6000640 1 -- 172.21.15.46:0/4233418383 >> v1:172.21.15.142:6789/0 conn(0x7f62f815cec0 legacy=0x7f62f80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f6000640 1 -- 172.21.15.46:0/4233418383 >> v1:172.21.15.46:6789/0 conn(0x7f62f8073b30 legacy=0x7f62f80b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.394+0000 7f62f6000640 1 -- 172.21.15.46:0/4233418383 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f80cf070 con 0x7f62f80cdd00 2024-04-26T23:54:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f6000640 1 --2- 172.21.15.46:0/4233418383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80735f0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f62e40097a0 tx=0x7f62e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d68ed0651a08fda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f5600640 1 -- 172.21.15.46:0/4233418383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4037070 con 0x7f62f80cdd00 2024-04-26T23:54:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f5600640 1 -- 172.21.15.46:0/4233418383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4032440 con 0x7f62f80cdd00 2024-04-26T23:54:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f5600640 1 -- 172.21.15.46:0/4233418383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4032810 con 0x7f62f80cdd00 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f7e00640 1 -- 172.21.15.46:0/4233418383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 msgr2=0x7f62f80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.396+0000 7f62f7e00640 1 --2- 172.21.15.46:0/4233418383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80735f0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f62e40097a0 tx=0x7f62e402a230 comp rx=0 tx=0).stop 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.397+0000 7f62f7e00640 1 -- 172.21.15.46:0/4233418383 shutdown_connections 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.397+0000 7f62f7e00640 1 --2- 172.21.15.46:0/4233418383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80735f0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.397+0000 7f62f7e00640 1 -- 172.21.15.46:0/4233418383 >> 172.21.15.46:0/4233418383 conn(0x7f62f80b27b0 msgr2=0x7f62f80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.397+0000 7f62f7e00640 1 -- 172.21.15.46:0/4233418383 shutdown_connections 2024-04-26T23:54:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.397+0000 7f62f7e00640 1 -- 172.21.15.46:0/4233418383 wait complete. 2024-04-26T23:54:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.398+0000 7f62f7e00640 1 Processor -- start 2024-04-26T23:54:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.399+0000 7f62f7e00640 1 -- start start 2024-04-26T23:54:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.399+0000 7f62f7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62f8073b30 0x7f62f8155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f815cec0 0x7f62f80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8073f70 con 0x7f62f80cdd00 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62f815d2a0 con 0x7f62f815cec0 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f62f80b4250 con 0x7f62f8073b30 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55132/0 (socket says 172.21.15.46:55132) 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 -- 172.21.15.46:0/13476776 learned_addr learned my addr 172.21.15.46:0/13476776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 -- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62f8073b30 msgr2=0x7f62f8155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62f8073b30 0x7f62f8155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 -- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f815cec0 msgr2=0x7f62f80c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f815cec0 0x7f62f80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.400+0000 7f62f6000640 1 -- 172.21.15.46:0/13476776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f80cd080 con 0x7f62f80cdd00 2024-04-26T23:54:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62f6000640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f62e4035040 tx=0x7f62e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4037040 con 0x7f62f80cdd00 2024-04-26T23:54:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4030770 con 0x7f62f80cdd00 2024-04-26T23:54:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4030b40 con 0x7f62f80cdd00 2024-04-26T23:54:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f80aeb40 con 0x7f62f80cdd00 2024-04-26T23:54:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.401+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f80af100 con 0x7f62f80cdd00 2024-04-26T23:54:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.402+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f80693b0 con 0x7f62f80cdd00 2024-04-26T23:54:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.403+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62e4044020 con 0x7f62f80cdd00 2024-04-26T23:54:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.404+0000 7f62eea00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.404+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5249+0+0 (secure 0 0 0) 0x7f62e40abf10 con 0x7f62f80cdd00 2024-04-26T23:54:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.405+0000 7f62f6a00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.406+0000 7f62f6a00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 0x7f62d806edb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f62e8000f60 tx=0x7f62e80079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.408+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62e4032e20 con 0x7f62f80cdd00 2024-04-26T23:54:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.603+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62f80b55f0 con 0x7f62f80cdd00 2024-04-26T23:54:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.604+0000 7f62eea00640 1 -- 172.21.15.46:0/13476776 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) ==== 93+0+10 (secure 0 0 0) 0x7f62e407a4f0 con 0x7f62f80cdd00 2024-04-26T23:54:21.607 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 msgr2=0x7f62d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 0x7f62d806edb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f62e8000f60 tx=0x7f62e80079b0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 msgr2=0x7f62f80cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f62e4035040 tx=0x7f62e40046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 shutdown_connections 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f815cec0 0x7f62f80c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f80cdd00 0x7f62f80cbc60 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 --2- 172.21.15.46:0/13476776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62f8073b30 0x7f62f8155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 >> 172.21.15.46:0/13476776 conn(0x7f62f80b27b0 msgr2=0x7f62f8072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 shutdown_connections 2024-04-26T23:54:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.606+0000 7f62f7e00640 1 -- 172.21.15.46:0/13476776 wait complete. 2024-04-26T23:54:21.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:54:21.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.827+0000 7fedd5a00640 1 Processor -- start 2024-04-26T23:54:21.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.828+0000 7fedd5a00640 1 -- start start 2024-04-26T23:54:21.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.829+0000 7fedd5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d1990 0x7fedd00d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.829+0000 7fedd5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fedd01639c0 0x7fedd00b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.829+0000 7fedd5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd00d22b0 con 0x7fedd00d0c10 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.829+0000 7fedd5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fedd00d0780 con 0x7fedd01639c0 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.829+0000 7fedd5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00d5da0 con 0x7fedd00d1990 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedd5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fedd01639c0 0x7fedd00b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedcf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fedd00d0c10 0x7fedd00d5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49222/0 (socket says 172.21.15.46:49222) 2024-04-26T23:54:21.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedcf400640 1 -- 172.21.15.46:0/355781883 learned_addr learned my addr 172.21.15.46:0/355781883 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedcfe00640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d1990 0x7fedd00d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedd5000640 1 -- 172.21.15.46:0/355781883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d1990 msgr2=0x7fedd00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedd5000640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d1990 0x7fedd00d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedd5000640 1 -- 172.21.15.46:0/355781883 >> v1:172.21.15.46:6789/0 conn(0x7fedd00d0c10 legacy=0x7fedd00d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.830+0000 7fedd5000640 1 -- 172.21.15.46:0/355781883 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00cfe50 con 0x7fedd01639c0 2024-04-26T23:54:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.831+0000 7fedd5000640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd00b7dd0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009d90 tx=0x7fedc802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7ea136dd1a824d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedcea00640 1 -- 172.21.15.46:0/355781883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8007740 con 0x7fedd01639c0 2024-04-26T23:54:21.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedcea00640 1 -- 172.21.15.46:0/355781883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc80078e0 con 0x7fedd01639c0 2024-04-26T23:54:21.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedcea00640 1 -- 172.21.15.46:0/355781883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8007cb0 con 0x7fedd01639c0 2024-04-26T23:54:21.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 -- 172.21.15.46:0/355781883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 msgr2=0x7fedd00b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd00b7dd0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009d90 tx=0x7fedc802a660 comp rx=0 tx=0).stop 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 -- 172.21.15.46:0/355781883 shutdown_connections 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd00b7dd0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 --2- 172.21.15.46:0/355781883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d1990 0x7fedd00d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 -- 172.21.15.46:0/355781883 >> 172.21.15.46:0/355781883 conn(0x7fedd009e2a0 msgr2=0x7fedd00d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 -- 172.21.15.46:0/355781883 shutdown_connections 2024-04-26T23:54:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.832+0000 7fedd5a00640 1 -- 172.21.15.46:0/355781883 wait complete. 2024-04-26T23:54:21.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.834+0000 7fedd5a00640 1 Processor -- start 2024-04-26T23:54:21.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.834+0000 7fedd5a00640 1 -- start start 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d0c10 0x7fedd006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd00d1990 0x7fedd015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fedd015f530 con 0x7fedd01639c0 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00d0410 con 0x7fedd00d1990 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00cfa90 con 0x7fedd00d0c10 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55150/0 (socket says 172.21.15.46:55150) 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.835+0000 7fedd5000640 1 -- 172.21.15.46:0/1769870070 learned_addr learned my addr 172.21.15.46:0/1769870070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedcfe00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d0c10 0x7fedd006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 -- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d0c10 msgr2=0x7fedd006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d0c10 0x7fedd006e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedcf400640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd00d1990 0x7fedd015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 -- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd00d1990 msgr2=0x7fedd015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd00d1990 0x7fedd015a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 -- 172.21.15.46:0/1769870070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00cd240 con 0x7fedd01639c0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedd5000640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009eb0 tx=0x7fedc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.836+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc803f070 con 0x7fedd01639c0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.837+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc8031bc0 con 0x7fedd01639c0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.837+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8044460 con 0x7fedd01639c0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.837+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd00cd520 con 0x7fedd01639c0 2024-04-26T23:54:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.837+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd00cdae0 con 0x7fedd01639c0 2024-04-26T23:54:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.838+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd00693b0 con 0x7fedd01639c0 2024-04-26T23:54:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.839+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedc8007790 con 0x7fedd01639c0 2024-04-26T23:54:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.840+0000 7fedc7e00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 0x7fedac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.840+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5249+0+0 (secure 0 0 0) 0x7fedc80ad290 con 0x7fedd01639c0 2024-04-26T23:54:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.840+0000 7fedcfe00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 0x7fedac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.842+0000 7fedcfe00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 0x7fedac06edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:21.845+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fedc807b7e0 con 0x7fedd01639c0 2024-04-26T23:54:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.045+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fedd00d1770 con 0x7fedd01639c0 2024-04-26T23:54:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.046+0000 7fedc7e00640 1 -- 172.21.15.46:0/1769870070 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v169) ==== 84+0+65 (secure 0 0 0) 0x7fedc807b0e0 con 0x7fedd01639c0 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:22.050 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:54:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.048+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 msgr2=0x7fedac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.048+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 0x7fedac06edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0008040 comp rx=0 tx=0).stop 2024-04-26T23:54:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 msgr2=0x7fedd015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009eb0 tx=0x7fedc8004a40 comp rx=0 tx=0).stop 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 shutdown_connections 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fedac06c8f0 0x7fedac06edb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedd01639c0 0x7fedd015a960 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd00d1990 0x7fedd015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 --2- 172.21.15.46:0/1769870070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd00d0c10 0x7fedd006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 >> 172.21.15.46:0/1769870070 conn(0x7fedd009e2a0 msgr2=0x7fedd015deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 shutdown_connections 2024-04-26T23:54:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.049+0000 7fedd5a00640 1 -- 172.21.15.46:0/1769870070 wait complete. 2024-04-26T23:54:22.071 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:54:22.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:54:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.278+0000 7fb599000640 1 Processor -- start 2024-04-26T23:54:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.278+0000 7fb599000640 1 -- start start 2024-04-26T23:54:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb599000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5940d5820 0x7fb5940d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb599000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5940d5d00 con 0x7fb59415f6c0 2024-04-26T23:54:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb599000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5940cffc0 con 0x7fb5940d36f0 2024-04-26T23:54:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb599000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb5940d02a0 con 0x7fb5940d5820 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb591e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb59415f6c0 0x7fb5940d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57640/0 (socket says 172.21.15.46:57640) 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb591e00640 1 -- 172.21.15.46:0/2572888008 learned_addr learned my addr 172.21.15.46:0/2572888008 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.280+0000 7fb591400640 1 --2- 172.21.15.46:0/2572888008 >> v2:172.21.15.46:3300/0 conn(0x7fb5940d5820 0x7fb5940d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.281+0000 7fb591400640 1 -- 172.21.15.46:0/2572888008 >> v1:172.21.15.142:6789/0 conn(0x7fb5940d36f0 legacy=0x7fb5940b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.281+0000 7fb591400640 1 -- 172.21.15.46:0/2572888008 >> v1:172.21.15.46:6789/0 conn(0x7fb59415f6c0 legacy=0x7fb5940d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.281+0000 7fb590a00640 1 -- 172.21.15.46:0/2572888008 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984135440 0 0) 0x7fb5940d5d00 con 0x7fb59415f6c0 2024-04-26T23:54:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.281+0000 7fb591400640 1 -- 172.21.15.46:0/2572888008 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5940d0580 con 0x7fb5940d5820 2024-04-26T23:54:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.282+0000 7fb591400640 1 --2- 172.21.15.46:0/2572888008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb5940d2100 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb180582027d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:22.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.282+0000 7fb590a00640 1 -- 172.21.15.46:0/2572888008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb5940d5820 2024-04-26T23:54:22.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.282+0000 7fb590a00640 1 -- 172.21.15.46:0/2572888008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb580032440 con 0x7fb5940d5820 2024-04-26T23:54:22.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.282+0000 7fb590a00640 1 -- 172.21.15.46:0/2572888008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580032810 con 0x7fb5940d5820 2024-04-26T23:54:22.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 -- 172.21.15.46:0/2572888008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 msgr2=0x7fb5940d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 --2- 172.21.15.46:0/2572888008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb5940d2100 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 -- 172.21.15.46:0/2572888008 shutdown_connections 2024-04-26T23:54:22.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 --2- 172.21.15.46:0/2572888008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb5940d2100 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 -- 172.21.15.46:0/2572888008 >> 172.21.15.46:0/2572888008 conn(0x7fb59409e450 msgr2=0x7fb5940d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:22.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 -- 172.21.15.46:0/2572888008 shutdown_connections 2024-04-26T23:54:22.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.283+0000 7fb599000640 1 -- 172.21.15.46:0/2572888008 wait complete. 2024-04-26T23:54:22.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.285+0000 7fb599000640 1 Processor -- start 2024-04-26T23:54:22.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.285+0000 7fb599000640 1 -- start start 2024-04-26T23:54:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 0x7fb594151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 0x7fb5941f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940d3230 con 0x7fb5940d5820 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940d30b0 con 0x7fb59415f6c0 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb599000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5940b2450 con 0x7fb5940d36f0 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb591400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb591400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49374/0 (socket says 172.21.15.46:49374) 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb591400640 1 -- 172.21.15.46:0/454696713 learned_addr learned my addr 172.21.15.46:0/454696713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb592800640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 0x7fb5941f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb591400640 1 -- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 msgr2=0x7fb594151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.286+0000 7fb591e00640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 0x7fb594151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591400640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 0x7fb594151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591400640 1 -- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 msgr2=0x7fb5941f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591400640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 0x7fb5941f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591400640 1 -- 172.21.15.46:0/454696713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5940b9770 con 0x7fb5940d5820 2024-04-26T23:54:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb592800640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 0x7fb5941f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591e00640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 0x7fb594151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb591400640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fb5800098c0 tx=0x7fb5800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580041070 con 0x7fb5940d5820 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58003c870 con 0x7fb5940d5820 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580037040 con 0x7fb5940d5820 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.287+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5940b99e0 con 0x7fb5940d5820 2024-04-26T23:54:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.288+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5940b9fa0 con 0x7fb5940d5820 2024-04-26T23:54:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.289+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594069530 con 0x7fb5940d5820 2024-04-26T23:54:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.290+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb580044070 con 0x7fb5940d5820 2024-04-26T23:54:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.291+0000 7fb58a000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.291+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5249+0+0 (secure 0 0 0) 0x7fb5800ac7e0 con 0x7fb5940d5820 2024-04-26T23:54:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.291+0000 7fb591e00640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:22.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.292+0000 7fb591e00640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 0x7fb57406edb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.295+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb58007adc0 con 0x7fb5940d5820 2024-04-26T23:54:22.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:22.498+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb5940b3a20 con 0x7fb5940d5820 2024-04-26T23:54:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.173+0000 7fb58a000640 1 -- 172.21.15.46:0/454696713 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v170) ==== 138+0+0 (secure 0 0 0) 0x7fb58007a6c0 con 0x7fb5940d5820 2024-04-26T23:54:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.176+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 msgr2=0x7fb57406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.176+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 0x7fb57406edb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).stop 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 msgr2=0x7fb59415faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fb5800098c0 tx=0x7fb5800046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 shutdown_connections 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59415f6c0 0x7fb5941f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5940d5820 0x7fb59415faa0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 --2- 172.21.15.46:0/454696713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940d36f0 0x7fb594151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 >> 172.21.15.46:0/454696713 conn(0x7fb59409e450 msgr2=0x7fb59406e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 shutdown_connections 2024-04-26T23:54:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.177+0000 7fb599000640 1 -- 172.21.15.46:0/454696713 wait complete. 2024-04-26T23:54:23.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:54:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.421+0000 7f0792c00640 1 Processor -- start 2024-04-26T23:54:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.421+0000 7f0792c00640 1 -- start start 2024-04-26T23:54:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.422+0000 7f0792c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f078c0d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.422+0000 7f0792c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078c0d20b0 con 0x7f078c15f3e0 2024-04-26T23:54:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.422+0000 7f0792c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078c0cdf10 con 0x7f078c0d1bf0 2024-04-26T23:54:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.422+0000 7f0792c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f078c0d2390 con 0x7f078c0cfa00 2024-04-26T23:54:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.423+0000 7f0791800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f078c15f3e0 0x7f078c0d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57646/0 (socket says 172.21.15.46:57646) 2024-04-26T23:54:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.423+0000 7f0791800640 1 -- 172.21.15.46:0/2898151452 learned_addr learned my addr 172.21.15.46:0/2898151452 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.423+0000 7f0790e00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f078c0d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.423+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261666147 0 0) 0x7f078c0d20b0 con 0x7f078c15f3e0 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.423+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0770003520 con 0x7f078c15f3e0 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2822321317 0 0) 0x7f0770003520 con 0x7f078c15f3e0 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 msgr2=0x7f078c0d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f078c0d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 >> v1:172.21.15.142:6789/0 conn(0x7f078c0d1bf0 legacy=0x7f078c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c0ce1f0 con 0x7f078c15f3e0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f077c0019f0 con 0x7f078c15f3e0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 >> v1:172.21.15.46:6789/0 conn(0x7f078c15f3e0 legacy=0x7f078c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07700039e0 0x7f0770003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0770004800 0x7f0770004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f0770005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0d2390 con 0x7f0770004800 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f077000bb50 con 0x7f07700039e0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f077000bd50 con 0x7f078c0cfa00 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.424+0000 7f0790e00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0770004800 0x7f0770004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0792200640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f0770005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0791800640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07700039e0 0x7f0770003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0790e00640 1 -- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 msgr2=0x7f0770005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0790e00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c0cfa00 0x7f0770005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0790e00640 1 -- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07700039e0 msgr2=0x7f0770003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0790e00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07700039e0 0x7f0770003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.425+0000 7f0790e00640 1 -- 172.21.15.46:0/2898151452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0770003520 con 0x7f0770004800 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.426+0000 7f0790e00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0770004800 0x7f0770004c80 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f0778009ac0 tx=0x7f0770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38da3e859d15625a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.426+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778036070 con 0x7f0770004800 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.426+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0778031400 con 0x7f0770004800 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.426+0000 7f078be00640 1 -- 172.21.15.46:0/2898151452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778009690 con 0x7f0770004800 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 -- 172.21.15.46:0/2898151452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0770004800 msgr2=0x7f0770004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 --2- 172.21.15.46:0/2898151452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0770004800 0x7f0770004c80 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f0778009ac0 tx=0x7f0770005830 comp rx=0 tx=0).stop 2024-04-26T23:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792200640 1 -- 172.21.15.46:0/2898151452 reap_dead start 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 -- 172.21.15.46:0/2898151452 shutdown_connections 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 -- 172.21.15.46:0/2898151452 >> 172.21.15.46:0/2898151452 conn(0x7f078c0b48f0 msgr2=0x7f078c0b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 -- 172.21.15.46:0/2898151452 shutdown_connections 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.427+0000 7f0792c00640 1 -- 172.21.15.46:0/2898151452 wait complete. 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.429+0000 7f0792c00640 1 Processor -- start 2024-04-26T23:54:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.429+0000 7f0792c00640 1 -- start start 2024-04-26T23:54:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c1575b0 0x7f078c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078c0b7160 0x7f078c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0b40a0 con 0x7f078c156780 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0ce560 con 0x7f078c0b7160 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0792c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f078c0b4220 con 0x7f078c1575b0 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0791800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0791800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49392/0 (socket says 172.21.15.46:49392) 2024-04-26T23:54:23.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.430+0000 7f0791800640 1 -- 172.21.15.46:0/1046905861 learned_addr learned my addr 172.21.15.46:0/1046905861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 -- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c1575b0 msgr2=0x7f078c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c1575b0 0x7f078c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 -- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078c0b7160 msgr2=0x7f078c15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078c0b7160 0x7f078c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 -- 172.21.15.46:0/1046905861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c0b78e0 con 0x7f078c156780 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0791800640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e410 tx=0x7f077c00e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00ee20 con 0x7f078c156780 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c018070 con 0x7f078c156780 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078c0b7600 con 0x7f078c156780 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.431+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c013660 con 0x7f078c156780 2024-04-26T23:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.432+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078c0b5d80 con 0x7f078c156780 2024-04-26T23:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.433+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078c069ac0 con 0x7f078c156780 2024-04-26T23:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.434+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f077c013800 con 0x7f078c156780 2024-04-26T23:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.435+0000 7f0789600640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 0x7f076c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.435+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5619+0+0 (secure 0 0 0) 0x7f077c08c260 con 0x7f078c156780 2024-04-26T23:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.435+0000 7f0790e00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 0x7f076c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.436+0000 7f0790e00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 0x7f076c06edb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0778009ac0 tx=0x7f077802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:23.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.440+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f077c05a6d0 con 0x7f078c156780 2024-04-26T23:54:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:23.649+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f078c15e370 con 0x7f078c156780 2024-04-26T23:54:24.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.179+0000 7f0789600640 1 -- 172.21.15.46:0/1046905861 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v171) ==== 170+0+0 (secure 0 0 0) 0x7f077c059fd0 con 0x7f078c156780 2024-04-26T23:54:24.183 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:54:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 msgr2=0x7f076c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 0x7f076c06edb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0778009ac0 tx=0x7f077802e000 comp rx=0 tx=0).stop 2024-04-26T23:54:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 msgr2=0x7f078c0b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e410 tx=0x7f077c00e500 comp rx=0 tx=0).stop 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 shutdown_connections 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f076c06c8f0 0x7f076c06edb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078c0b7160 0x7f078c15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078c1575b0 0x7f078c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 --2- 172.21.15.46:0/1046905861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f078c156780 0x7f078c0b6c20 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.186+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 >> 172.21.15.46:0/1046905861 conn(0x7f078c0b48f0 msgr2=0x7f078c157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.187+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 shutdown_connections 2024-04-26T23:54:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.187+0000 7f0792c00640 1 -- 172.21.15.46:0/1046905861 wait complete. 2024-04-26T23:54:24.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.416+0000 7f1fb9000640 1 Processor -- start 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.416+0000 7f1fb9000640 1 -- start start 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.417+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb40d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.417+0000 7f1fb9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb4074070 con 0x7f1fb416be40 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb406dee0 con 0x7f1fb40d1420 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb416c220 con 0x7f1fb40b4750 2024-04-26T23:54:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1fb416be40 0x7f1fb4073640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57656/0 (socket says 172.21.15.46:57656) 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb1e00640 1 -- 172.21.15.46:0/912213980 learned_addr learned my addr 172.21.15.46:0/912213980 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb1400640 1 --2- 172.21.15.46:0/912213980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb40d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.418+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235599742 0 0) 0x7f1fb4074070 con 0x7f1fb416be40 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.626+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f98003500 con 0x7f1fb416be40 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.626+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 535184218 0 0) 0x7f1fb406dee0 con 0x7f1fb40d1420 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.626+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb4074070 con 0x7f1fb40d1420 2024-04-26T23:54:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.627+0000 7f1fb1400640 1 -- 172.21.15.46:0/912213980 >> v1:172.21.15.142:6789/0 conn(0x7f1fb40d1420 legacy=0x7f1fb416f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.627+0000 7f1fb1400640 1 -- 172.21.15.46:0/912213980 >> v1:172.21.15.46:6789/0 conn(0x7f1fb416be40 legacy=0x7f1fb4073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.627+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 441613546 0 0) 0x7f1f98003500 con 0x7f1fb416be40 2024-04-26T23:54:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.627+0000 7f1fb1400640 1 -- 172.21.15.46:0/912213980 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb406e1c0 con 0x7f1fb40b4750 2024-04-26T23:54:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.630+0000 7f1fb1400640 1 --2- 172.21.15.46:0/912213980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb40d0ee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009800 tx=0x7f1fa002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1ec388d4f56723 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:24.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.631+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4001670 con 0x7f1fb40b4750 2024-04-26T23:54:24.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.631+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa00092e0 con 0x7f1fb40b4750 2024-04-26T23:54:24.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.631+0000 7f1fb0a00640 1 -- 172.21.15.46:0/912213980 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa002ab00 con 0x7f1fb40b4750 2024-04-26T23:54:24.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 -- 172.21.15.46:0/912213980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 msgr2=0x7f1fb40d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 --2- 172.21.15.46:0/912213980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb40d0ee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009800 tx=0x7f1fa002a080 comp rx=0 tx=0).stop 2024-04-26T23:54:24.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 -- 172.21.15.46:0/912213980 shutdown_connections 2024-04-26T23:54:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 --2- 172.21.15.46:0/912213980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb40d0ee0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 -- 172.21.15.46:0/912213980 >> 172.21.15.46:0/912213980 conn(0x7f1fb406c530 msgr2=0x7f1fb406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 -- 172.21.15.46:0/912213980 shutdown_connections 2024-04-26T23:54:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.633+0000 7f1fb9000640 1 -- 172.21.15.46:0/912213980 wait complete. 2024-04-26T23:54:24.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.636+0000 7f1fb9000640 1 Processor -- start 2024-04-26T23:54:24.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.636+0000 7f1fb9000640 1 -- start start 2024-04-26T23:54:24.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb4162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb40d1420 0x7f1fb41683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406e530 con 0x7f1fb416be40 2024-04-26T23:54:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406dc90 con 0x7f1fb40d1420 2024-04-26T23:54:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.637+0000 7f1fb9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb406e9c0 con 0x7f1fb40b4750 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.638+0000 7f1fb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.638+0000 7f1fb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49398/0 (socket says 172.21.15.46:49398) 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.638+0000 7f1fb2800640 1 -- 172.21.15.46:0/836495563 learned_addr learned my addr 172.21.15.46:0/836495563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.638+0000 7f1fb1e00640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.638+0000 7f1fb1400640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb40d1420 0x7f1fb41683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 -- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 msgr2=0x7f1fb4162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb4162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 -- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb40d1420 msgr2=0x7f1fb41683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb40d1420 0x7f1fb41683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 -- 172.21.15.46:0/836495563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41688e0 con 0x7f1fb416be40 2024-04-26T23:54:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.639+0000 7f1fb2800640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c790 tx=0x7f1fac00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.640+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003b70 con 0x7f1fb416be40 2024-04-26T23:54:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.640+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac003d10 con 0x7f1fb416be40 2024-04-26T23:54:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.640+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00aa50 con 0x7f1fb416be40 2024-04-26T23:54:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.640+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb4169f50 con 0x7f1fb416be40 2024-04-26T23:54:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.640+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb4169c70 con 0x7f1fb416be40 2024-04-26T23:54:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.641+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb4069530 con 0x7f1fb416be40 2024-04-26T23:54:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.643+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fac00abf0 con 0x7f1fb416be40 2024-04-26T23:54:24.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.644+0000 7f1faa000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 0x7f1f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.644+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 6001+0+0 (secure 0 0 0) 0x7f1fac08bab0 con 0x7f1fb416be40 2024-04-26T23:54:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.645+0000 7f1fb1e00640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 0x7f1f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.646+0000 7f1fb1e00640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 0x7f1f9406edb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40011e0 tx=0x7f1fa40016f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.650+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fb41688e0 con 0x7f1fb416be40 2024-04-26T23:54:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:24.903+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1fb40b4330 con 0x7f1fb416be40 2024-04-26T23:54:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.235+0000 7f1faa000640 1 -- 172.21.15.46:0/836495563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v136) ==== 174+0+0 (secure 0 0 0) 0x7f1fac059750 con 0x7f1fb416be40 2024-04-26T23:54:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 25 and data pool 26 2024-04-26T23:54:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.242+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 msgr2=0x7f1f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.242+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 0x7f1f9406edb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40011e0 tx=0x7f1fa40016f0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.242+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 msgr2=0x7f1fb4201820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.242+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c790 tx=0x7f1fac00cc50 comp rx=0 tx=0).stop 2024-04-26T23:54:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 shutdown_connections 2024-04-26T23:54:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1f9406c8f0 0x7f1f9406edb0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb416be40 0x7f1fb4201820 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb40d1420 0x7f1fb41683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 --2- 172.21.15.46:0/836495563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fb40b4750 0x7f1fb4162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.246+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 >> 172.21.15.46:0/836495563 conn(0x7f1fb406c530 msgr2=0x7f1fb40b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.248+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 shutdown_connections 2024-04-26T23:54:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.248+0000 7f1fb9000640 1 -- 172.21.15.46:0/836495563 wait complete. 2024-04-26T23:54:25.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.457+0000 7fda4ea00640 1 Processor -- start 2024-04-26T23:54:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.457+0000 7fda4ea00640 1 -- start start 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.458+0000 7fda4ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda480d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.458+0000 7fda4ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda48072e60 0x7fda480732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.458+0000 7fda4ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda480cf950 con 0x7fda4815c5c0 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.458+0000 7fda4ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fda480d0c50 con 0x7fda48072e60 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.458+0000 7fda4ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda4815d490 con 0x7fda48145770 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda4ca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda4815c5c0 0x7fda480b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57664/0 (socket says 172.21.15.46:57664) 2024-04-26T23:54:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda4ca00640 1 -- 172.21.15.46:0/1887205789 learned_addr learned my addr 172.21.15.46:0/1887205789 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda47400640 1 --2- 172.21.15.46:0/1887205789 >> v2:172.21.15.142:3300/0 conn(0x7fda48072e60 0x7fda480732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda47e00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda480d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024011120 0 0) 0x7fda480cf950 con 0x7fda4815c5c0 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.459+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda2c003520 con 0x7fda4815c5c0 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1881881212 0 0) 0x7fda2c003520 con 0x7fda4815c5c0 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 msgr2=0x7fda480d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda480d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 >> v2:172.21.15.142:3300/0 conn(0x7fda48072e60 msgr2=0x7fda480732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 --2- 172.21.15.46:0/1887205789 >> v2:172.21.15.142:3300/0 conn(0x7fda48072e60 0x7fda480732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda480cfc30 con 0x7fda4815c5c0 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fda400019f0 con 0x7fda4815c5c0 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 >> v1:172.21.15.46:6789/0 conn(0x7fda4815c5c0 legacy=0x7fda480b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda2c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c004d40 0x7fda2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda480d0c50 con 0x7fda2c003f80 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda4815d490 con 0x7fda2c004d40 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c00b9c0 con 0x7fda48145770 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda47400640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.460+0000 7fda47e00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda2c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda47400640 1 -- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 msgr2=0x7fda2c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda4ca00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c004d40 0x7fda2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda47400640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda48145770 0x7fda2c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda47400640 1 -- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c004d40 msgr2=0x7fda2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda47400640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c004d40 0x7fda2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda47400640 1 -- 172.21.15.46:0/1887205789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c003520 con 0x7fda2c003f80 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.461+0000 7fda4ca00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c004d40 0x7fda2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.462+0000 7fda47400640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda2c004800 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fda34009af0 tx=0x7fda2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3019bf67149b218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.462+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34036070 con 0x7fda2c003f80 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.462+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34031400 con 0x7fda2c003f80 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.462+0000 7fda46a00640 1 -- 172.21.15.46:0/1887205789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34009690 con 0x7fda2c003f80 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 -- 172.21.15.46:0/1887205789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 msgr2=0x7fda2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 --2- 172.21.15.46:0/1887205789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda2c004800 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fda34009af0 tx=0x7fda2c005830 comp rx=0 tx=0).stop 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ca00640 1 -- 172.21.15.46:0/1887205789 reap_dead start 2024-04-26T23:54:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 -- 172.21.15.46:0/1887205789 shutdown_connections 2024-04-26T23:54:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 -- 172.21.15.46:0/1887205789 >> 172.21.15.46:0/1887205789 conn(0x7fda4806c530 msgr2=0x7fda4806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 -- 172.21.15.46:0/1887205789 shutdown_connections 2024-04-26T23:54:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.463+0000 7fda4ea00640 1 -- 172.21.15.46:0/1887205789 wait complete. 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.465+0000 7fda4ea00640 1 Processor -- start 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.465+0000 7fda4ea00640 1 -- start start 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda481e5630 0x7fda480b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda480b5650 0x7fda48141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda480d01f0 con 0x7fda2c003f80 2024-04-26T23:54:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda48073800 con 0x7fda481e5630 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda4ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda480cf5b0 con 0x7fda480b5650 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49420/0 (socket says 172.21.15.46:49420) 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.466+0000 7fda47e00640 1 -- 172.21.15.46:0/3628439207 learned_addr learned my addr 172.21.15.46:0/3628439207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 -- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda480b5650 msgr2=0x7fda48141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda480b5650 0x7fda48141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 -- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda481e5630 msgr2=0x7fda480b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda481e5630 0x7fda480b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 -- 172.21.15.46:0/3628439207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481429d0 con 0x7fda2c003f80 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda47e00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fda38010ac0 tx=0x7fda38010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38011450 con 0x7fda2c003f80 2024-04-26T23:54:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda380115f0 con 0x7fda2c003f80 2024-04-26T23:54:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda380119a0 con 0x7fda2c003f80 2024-04-26T23:54:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481e4870 con 0x7fda2c003f80 2024-04-26T23:54:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.467+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481e4e30 con 0x7fda2c003f80 2024-04-26T23:54:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.469+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda48069530 con 0x7fda2c003f80 2024-04-26T23:54:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.470+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda38011b40 con 0x7fda2c003f80 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.471+0000 7fda3fe00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 0x7fda2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.471+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 6101+0+0 (secure 0 0 0) 0x7fda3808c830 con 0x7fda2c003f80 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.471+0000 7fda47400640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 0x7fda2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.473+0000 7fda47400640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 0x7fda2806ede0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fda34033000 tx=0x7fda3402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.475+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda380162a0 con 0x7fda2c003f80 2024-04-26T23:54:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.684+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda480b6060 con 0x7fda2c003f80 2024-04-26T23:54:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.687+0000 7fda3fe00640 1 -- 172.21.15.46:0/3628439207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) ==== 74+0+16865 (secure 0 0 0) 0x7fda3805aa30 con 0x7fda2c003f80 2024-04-26T23:54:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:25.691 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":172,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:24.905254+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 msgr2=0x7fda2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 0x7fda2806ede0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fda34033000 tx=0x7fda3402e000 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 msgr2=0x7fda4815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fda38010ac0 tx=0x7fda38010f80 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 shutdown_connections 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fda2806c920 0x7fda2806ede0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda480b5650 0x7fda48141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda481e5630 0x7fda480b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 --2- 172.21.15.46:0/3628439207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c003f80 0x7fda4815b600 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 >> 172.21.15.46:0/3628439207 conn(0x7fda4806c530 msgr2=0x7fda480b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 shutdown_connections 2024-04-26T23:54:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.690+0000 7fda4ea00640 1 -- 172.21.15.46:0/3628439207 wait complete. 2024-04-26T23:54:25.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.922+0000 7f833ea00640 1 Processor -- start 2024-04-26T23:54:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.922+0000 7f833ea00640 1 -- start start 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.923+0000 7f833ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83380ccb20 0x7f833806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.923+0000 7f833ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83380d19a0 con 0x7f833806c890 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f833ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8338163a10 con 0x7f83381634a0 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f833ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8338163cf0 con 0x7f83380ccb20 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f833cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f833806c890 0x7f8338162d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57678/0 (socket says 172.21.15.46:57678) 2024-04-26T23:54:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f833cc00640 1 -- 172.21.15.46:0/666640384 learned_addr learned my addr 172.21.15.46:0/666640384 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f833d600640 1 --2- 172.21.15.46:0/666640384 >> v2:172.21.15.142:3300/0 conn(0x7f83380ccb20 0x7f833806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2101894056 0 0) 0x7f83380d19a0 con 0x7f833806c890 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.924+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f831c003520 con 0x7f833806c890 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 956130697 0 0) 0x7f831c003520 con 0x7f833806c890 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 >> v2:172.21.15.142:3300/0 conn(0x7f83380ccb20 msgr2=0x7f833806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 --2- 172.21.15.46:0/666640384 >> v2:172.21.15.142:3300/0 conn(0x7f83380ccb20 0x7f833806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 >> v1:172.21.15.142:6789/0 conn(0x7f83381634a0 legacy=0x7f833815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83380cfeb0 con 0x7f833806c890 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f83240019f0 con 0x7f833806c890 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 >> v1:172.21.15.46:6789/0 conn(0x7f833806c890 legacy=0x7f8338162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c003a00 0x7f831c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 0x7f831c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f831c0051c0 0x7f831c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f831c00c090 con 0x7f831c004800 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f831c00c240 con 0x7f831c0051c0 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f831c00c440 con 0x7f831c003a00 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.925+0000 7f833cc00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 0x7f831c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833d600640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c003a00 0x7f831c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833cc00640 1 -- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c003a00 msgr2=0x7f831c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833cc00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c003a00 0x7f831c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833cc00640 1 -- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f831c0051c0 msgr2=0x7f831c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833cc00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f831c0051c0 0x7f831c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833cc00640 1 -- 172.21.15.46:0/666640384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c003520 con 0x7f831c004800 2024-04-26T23:54:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833e000640 1 -- 172.21.15.46:0/666640384 reap_dead start 2024-04-26T23:54:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.926+0000 7f833d600640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c003a00 0x7f831c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.927+0000 7f833cc00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 0x7f831c004c80 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f832400a7b0 tx=0x7f831c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb78f083941248e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.927+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400a3d0 con 0x7f831c004800 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.927+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832402f070 con 0x7f831c004800 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.927+0000 7f8337e00640 1 -- 172.21.15.46:0/666640384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832402bc90 con 0x7f831c004800 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 -- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 msgr2=0x7f831c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 0x7f831c004c80 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f832400a7b0 tx=0x7f831c005830 comp rx=0 tx=0).stop 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 -- 172.21.15.46:0/666640384 shutdown_connections 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 --2- 172.21.15.46:0/666640384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c004800 0x7f831c004c80 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 -- 172.21.15.46:0/666640384 >> 172.21.15.46:0/666640384 conn(0x7f83380d1c30 msgr2=0x7f83380d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 -- 172.21.15.46:0/666640384 shutdown_connections 2024-04-26T23:54:25.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.928+0000 7f833ea00640 1 -- 172.21.15.46:0/666640384 wait complete. 2024-04-26T23:54:25.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.930+0000 7f833ea00640 1 Processor -- start 2024-04-26T23:54:25.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.930+0000 7f833ea00640 1 -- start start 2024-04-26T23:54:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c004800 0x7f83381e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83380cb920 0x7f833814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83380d0470 con 0x7f83380c96a0 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83380ccf00 con 0x7f83380cb920 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83380ae8f0 con 0x7f831c004800 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49442/0 (socket says 172.21.15.46:49442) 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 -- 172.21.15.46:0/4023793178 learned_addr learned my addr 172.21.15.46:0/4023793178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833cc00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83380cb920 0x7f833814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833d600640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c004800 0x7f83381e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 -- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c004800 msgr2=0x7f83381e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c004800 0x7f83381e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 -- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83380cb920 msgr2=0x7f833814f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83380cb920 0x7f833814f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.931+0000 7f833e000640 1 -- 172.21.15.46:0/4023793178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833814f630 con 0x7f83380c96a0 2024-04-26T23:54:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f833e000640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f833000c390 tx=0x7f833000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330007480 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8330007620 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330005a00 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83381e9090 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.932+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381e9650 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.933+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83380693b0 con 0x7f83380c96a0 2024-04-26T23:54:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.935+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8330011070 con 0x7f83380c96a0 2024-04-26T23:54:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.935+0000 7f8335600640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 0x7f831806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.936+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 6101+0+0 (secure 0 0 0) 0x7f833008b160 con 0x7f83380c96a0 2024-04-26T23:54:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.936+0000 7f833d600640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 0x7f831806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.937+0000 7f833d600640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 0x7f831806ede0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f833806b1d0 tx=0x7f8328008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:25.940+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8330019060 con 0x7f83380c96a0 2024-04-26T23:54:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.146+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83380d2b40 con 0x7f83380c96a0 2024-04-26T23:54:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.147+0000 7f8335600640 1 -- 172.21.15.46:0/4023793178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) ==== 74+0+16865 (secure 0 0 0) 0x7f8330059360 con 0x7f83380c96a0 2024-04-26T23:54:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:26.152 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":172,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:24.905254+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:26.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 msgr2=0x7f831806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 0x7f831806ede0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f833806b1d0 tx=0x7f8328008040 comp rx=0 tx=0).stop 2024-04-26T23:54:26.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 msgr2=0x7f83380c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f833000c390 tx=0x7f833000c850 comp rx=0 tx=0).stop 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 shutdown_connections 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f831806c920 0x7f831806ede0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83380c96a0 0x7f83380c9b20 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83380cb920 0x7f833814f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 --2- 172.21.15.46:0/4023793178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f831c004800 0x7f83381e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 >> 172.21.15.46:0/4023793178 conn(0x7f83380d1c30 msgr2=0x7f83380d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.150+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 shutdown_connections 2024-04-26T23:54:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.151+0000 7f833ea00640 1 -- 172.21.15.46:0/4023793178 wait complete. 2024-04-26T23:54:26.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.394+0000 7f92ae400640 1 Processor -- start 2024-04-26T23:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.394+0000 7f92ae400640 1 -- start start 2024-04-26T23:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92ae400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a81597e0 0x7f92a80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92ae400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92a8072620 0x7f92a80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92ae400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a8073120 con 0x7f92a80b5730 2024-04-26T23:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92ae400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f92a80b60d0 con 0x7f92a8072620 2024-04-26T23:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92ae400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92a8159bc0 con 0x7f92a81597e0 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92a8072620 0x7f92a80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92a8072620 0x7f92a80b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49458/0 (socket says 172.21.15.46:49458) 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.396+0000 7f92a7e00640 1 -- 172.21.15.46:0/206766628 learned_addr learned my addr 172.21.15.46:0/206766628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.397+0000 7f92a7400640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a81597e0 0x7f92a80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.397+0000 7f92a7e00640 1 -- 172.21.15.46:0/206766628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a81597e0 msgr2=0x7f92a80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.397+0000 7f92a7e00640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a81597e0 0x7f92a80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.397+0000 7f92a7e00640 1 -- 172.21.15.46:0/206766628 >> v1:172.21.15.142:6789/0 conn(0x7f92a80b5730 legacy=0x7f92a8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.397+0000 7f92a7e00640 1 -- 172.21.15.46:0/206766628 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a8072b60 con 0x7f92a8072620 2024-04-26T23:54:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.398+0000 7f92a7e00640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a8072620 0x7f92a80b5b90 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e50 tx=0x7f92a002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54140ae78f60fa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.399+0000 7f92a6000640 1 -- 172.21.15.46:0/206766628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0036070 con 0x7f92a8072620 2024-04-26T23:54:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.399+0000 7f92a6000640 1 -- 172.21.15.46:0/206766628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a0031600 con 0x7f92a8072620 2024-04-26T23:54:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.399+0000 7f92a6000640 1 -- 172.21.15.46:0/206766628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00319d0 con 0x7f92a8072620 2024-04-26T23:54:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 -- 172.21.15.46:0/206766628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a8072620 msgr2=0x7f92a80b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a8072620 0x7f92a80b5b90 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e50 tx=0x7f92a002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 -- 172.21.15.46:0/206766628 shutdown_connections 2024-04-26T23:54:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a8072620 0x7f92a80b5b90 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 --2- 172.21.15.46:0/206766628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a81597e0 0x7f92a80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 -- 172.21.15.46:0/206766628 >> 172.21.15.46:0/206766628 conn(0x7f92a806c980 msgr2=0x7f92a80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 -- 172.21.15.46:0/206766628 shutdown_connections 2024-04-26T23:54:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.400+0000 7f92ae400640 1 -- 172.21.15.46:0/206766628 wait complete. 2024-04-26T23:54:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.402+0000 7f92ae400640 1 Processor -- start 2024-04-26T23:54:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.402+0000 7f92ae400640 1 -- start start 2024-04-26T23:54:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a8072620 0x7f92a814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a81597e0 0x7f92a80af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80b43b0 con 0x7f92a80b5730 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80b40d0 con 0x7f92a81597e0 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92ae400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92a80aee20 con 0x7f92a8072620 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49470/0 (socket says 172.21.15.46:49470) 2024-04-26T23:54:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92a6a00640 1 -- 172.21.15.46:0/3798195659 learned_addr learned my addr 172.21.15.46:0/3798195659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92a7e00640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a81597e0 0x7f92a80af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.403+0000 7f92a7400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a8072620 0x7f92a814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 -- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a8072620 msgr2=0x7f92a814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a8072620 0x7f92a814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 -- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a81597e0 msgr2=0x7f92a80af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a81597e0 0x7f92a80af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 -- 172.21.15.46:0/3798195659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a815e450 con 0x7f92a80b5730 2024-04-26T23:54:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92a6a00640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f929000c430 tx=0x7f929000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92900052d0 con 0x7f92a80b5730 2024-04-26T23:54:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9290005490 con 0x7f92a80b5730 2024-04-26T23:54:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290007260 con 0x7f92a80b5730 2024-04-26T23:54:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a815e730 con 0x7f92a80b5730 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.404+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a815ecf0 con 0x7f92a80b5730 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.405+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a8069530 con 0x7f92a80b5730 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.407+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9290007400 con 0x7f92a80b5730 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.407+0000 7f929f400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 0x7f928406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.408+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7f929008b1b0 con 0x7f92a80b5730 2024-04-26T23:54:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.408+0000 7f92a7400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 0x7f928406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.410+0000 7f92a7400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 0x7f928406edb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9298006fd0 tx=0x7f9298008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.412+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92900594e0 con 0x7f92a80b5730 2024-04-26T23:54:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:26.656+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f92a8157f90 con 0x7f92a80b5730 2024-04-26T23:54:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.269+0000 7f929f400640 1 -- 172.21.15.46:0/3798195659 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v139) ==== 118+0+0 (secure 0 0 0) 0x7f9290058de0 con 0x7f92a80b5730 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.272+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 msgr2=0x7f928406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.272+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 0x7f928406edb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9298006fd0 tx=0x7f9298008040 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.272+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 msgr2=0x7f92a815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.272+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f929000c430 tx=0x7f929000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 shutdown_connections 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f928406c8f0 0x7f928406edb0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a81597e0 0x7f92a80af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92a80b5730 0x7f92a815dbb0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 --2- 172.21.15.46:0/3798195659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92a8072620 0x7f92a814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 >> 172.21.15.46:0/3798195659 conn(0x7f92a806c980 msgr2=0x7f92a8163570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 shutdown_connections 2024-04-26T23:54:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.273+0000 7f92ae400640 1 -- 172.21.15.46:0/3798195659 wait complete. 2024-04-26T23:54:27.298 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:54:27.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:54:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.511+0000 7fc30fe00640 1 Processor -- start 2024-04-26T23:54:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.512+0000 7fc30fe00640 1 -- start start 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.513+0000 7fc30fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc31015cec0 0x7fc3100cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.513+0000 7fc30fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3100cdb30 0x7fc3100735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.513+0000 7fc30fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3100ce7d0 con 0x7fc310073b30 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.513+0000 7fc30fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc3100ceab0 con 0x7fc31015cec0 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.513+0000 7fc30fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc3100ced90 con 0x7fc3100cdb30 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc31015cec0 0x7fc3100cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc31015cec0 0x7fc3100cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49480/0 (socket says 172.21.15.46:49480) 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 -- 172.21.15.46:0/52667912 learned_addr learned my addr 172.21.15.46:0/52667912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 -- 172.21.15.46:0/52667912 >> v2:172.21.15.142:3300/0 conn(0x7fc3100cdb30 msgr2=0x7fc3100735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 --2- 172.21.15.46:0/52667912 >> v2:172.21.15.142:3300/0 conn(0x7fc3100cdb30 0x7fc3100735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 -- 172.21.15.46:0/52667912 >> v1:172.21.15.142:6789/0 conn(0x7fc310073b30 legacy=0x7fc310071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.514+0000 7fc30ea00640 1 -- 172.21.15.46:0/52667912 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3100cf070 con 0x7fc31015cec0 2024-04-26T23:54:27.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.515+0000 7fc30ea00640 1 --2- 172.21.15.46:0/52667912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31015cec0 0x7fc3100cd5f0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fc310074300 tx=0x7fc30002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67775a9f7be182fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.515+0000 7fc30d600640 1 -- 172.21.15.46:0/52667912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300037070 con 0x7fc31015cec0 2024-04-26T23:54:27.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.515+0000 7fc30d600640 1 -- 172.21.15.46:0/52667912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300032440 con 0x7fc31015cec0 2024-04-26T23:54:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.515+0000 7fc30d600640 1 -- 172.21.15.46:0/52667912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300032810 con 0x7fc31015cec0 2024-04-26T23:54:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 -- 172.21.15.46:0/52667912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31015cec0 msgr2=0x7fc3100cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 --2- 172.21.15.46:0/52667912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31015cec0 0x7fc3100cd5f0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fc310074300 tx=0x7fc30002a230 comp rx=0 tx=0).stop 2024-04-26T23:54:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 -- 172.21.15.46:0/52667912 shutdown_connections 2024-04-26T23:54:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 --2- 172.21.15.46:0/52667912 >> v2:172.21.15.142:3300/0 conn(0x7fc3100cdb30 0x7fc3100735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 --2- 172.21.15.46:0/52667912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31015cec0 0x7fc3100cd5f0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 -- 172.21.15.46:0/52667912 >> 172.21.15.46:0/52667912 conn(0x7fc3100b27b0 msgr2=0x7fc3100b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 -- 172.21.15.46:0/52667912 shutdown_connections 2024-04-26T23:54:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.516+0000 7fc30fe00640 1 -- 172.21.15.46:0/52667912 wait complete. 2024-04-26T23:54:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.518+0000 7fc30fe00640 1 Processor -- start 2024-04-26T23:54:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.518+0000 7fc30fe00640 1 -- start start 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100cdb30 0x7fc3100b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015cec0 0x7fc3100d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc310073f70 con 0x7fc310073b30 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc31015d2a0 con 0x7fc3100cdb30 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc3100b4b50 con 0x7fc31015cec0 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49496/0 (socket says 172.21.15.46:49496) 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.519+0000 7fc30ea00640 1 -- 172.21.15.46:0/1703449461 learned_addr learned my addr 172.21.15.46:0/1703449461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 -- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015cec0 msgr2=0x7fc3100d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015cec0 0x7fc3100d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 -- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100cdb30 msgr2=0x7fc3100b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100cdb30 0x7fc3100b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 -- 172.21.15.46:0/1703449461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc310160140 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc30ea00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc3000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300041070 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30003c870 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.520+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300037040 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.521+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3101f5a50 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.521+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3101f8490 con 0x7fc310073b30 2024-04-26T23:54:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.522+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3100693b0 con 0x7fc310073b30 2024-04-26T23:54:27.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.523+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc300044070 con 0x7fc310073b30 2024-04-26T23:54:27.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.524+0000 7fc307400640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.524+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fc3000ac970 con 0x7fc310073b30 2024-04-26T23:54:27.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.524+0000 7fc30e000640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.526+0000 7fc30e000640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc3100c7490 tx=0x7fc2f8009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.528+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc310160140 con 0x7fc310073b30 2024-04-26T23:54:27.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:27.778+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc31015dc10 con 0x7fc310073b30 2024-04-26T23:54:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.272+0000 7fc307400640 1 -- 172.21.15.46:0/1703449461 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v140) ==== 122+0+0 (secure 0 0 0) 0x7fc30007ad20 con 0x7fc310073b30 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 msgr2=0x7fc2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc3100c7490 tx=0x7fc2f8009250 comp rx=0 tx=0).stop 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 msgr2=0x7fc3101f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc3000046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 shutdown_connections 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015cec0 0x7fc3100d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100cdb30 0x7fc3100b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 --2- 172.21.15.46:0/1703449461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310073b30 0x7fc3101f8f70 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.276+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 >> 172.21.15.46:0/1703449461 conn(0x7fc3100b27b0 msgr2=0x7fc3100733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.277+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 shutdown_connections 2024-04-26T23:54:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.277+0000 7fc30fe00640 1 -- 172.21.15.46:0/1703449461 wait complete. 2024-04-26T23:54:28.302 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:54:28.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.520+0000 7fcf43600640 1 Processor -- start 2024-04-26T23:54:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.520+0000 7fcf43600640 1 -- start start 2024-04-26T23:54:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.521+0000 7fcf43600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf3c15f5a0 0x7fcf3c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.521+0000 7fcf43600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf3c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.521+0000 7fcf43600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3c15f980 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.521+0000 7fcf43600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3c0cdbe0 con 0x7fcf3c15f5a0 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.521+0000 7fcf43600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d20c0 con 0x7fcf3c0ce070 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf3be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf3c0d1bc0 0x7fcf3c070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57686/0 (socket says 172.21.15.46:57686) 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf3be00640 1 -- 172.21.15.46:0/1523734977 learned_addr learned my addr 172.21.15.46:0/1523734977 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf41600640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf3c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1523734977 >> v2:172.21.15.142:3300/0 conn(0x7fcf3c15f5a0 0x7fcf3c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733703604 0 0) 0x7fcf3c15f980 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.522+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf20003520 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3413251638 0 0) 0x7fcf20003520 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 msgr2=0x7fcf3c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf3c0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 >> v2:172.21.15.142:3300/0 conn(0x7fcf3c15f5a0 msgr2=0x7fcf3c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 --2- 172.21.15.46:0/1523734977 >> v2:172.21.15.142:3300/0 conn(0x7fcf3c15f5a0 0x7fcf3c070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c0d23a0 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcf24001670 con 0x7fcf3c0d1bc0 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 >> v1:172.21.15.46:6789/0 conn(0x7fcf3c0d1bc0 legacy=0x7fcf3c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf200039b0 0x7fcf20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c0cdbe0 con 0x7fcf200039b0 2024-04-26T23:54:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d20c0 con 0x7fcf20004800 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2000b9f0 con 0x7fcf3c0ce070 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.523+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf200039b0 0x7fcf20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf3be00640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf41600640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf40c00640 1 -- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 msgr2=0x7fcf200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf3c0ce070 0x7fcf200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf40c00640 1 -- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf20004800 msgr2=0x7fcf20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.524+0000 7fcf40c00640 1 -- 172.21.15.46:0/1523734977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf20003520 con 0x7fcf200039b0 2024-04-26T23:54:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.525+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf200039b0 0x7fcf20003dd0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fc0 tx=0x7fcf20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf3976770e72b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.525+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c009c10 con 0x7fcf200039b0 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.525+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c036070 con 0x7fcf200039b0 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.525+0000 7fcf3b400640 1 -- 172.21.15.46:0/1523734977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c02d070 con 0x7fcf200039b0 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 -- 172.21.15.46:0/1523734977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf200039b0 msgr2=0x7fcf20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 --2- 172.21.15.46:0/1523734977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf200039b0 0x7fcf20003dd0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fc0 tx=0x7fcf20005830 comp rx=0 tx=0).stop 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf41600640 1 -- 172.21.15.46:0/1523734977 reap_dead start 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 -- 172.21.15.46:0/1523734977 shutdown_connections 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 -- 172.21.15.46:0/1523734977 >> 172.21.15.46:0/1523734977 conn(0x7fcf3c0d2f40 msgr2=0x7fcf3c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 -- 172.21.15.46:0/1523734977 shutdown_connections 2024-04-26T23:54:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.526+0000 7fcf43600640 1 -- 172.21.15.46:0/1523734977 wait complete. 2024-04-26T23:54:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.528+0000 7fcf43600640 1 Processor -- start 2024-04-26T23:54:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.528+0000 7fcf43600640 1 -- start start 2024-04-26T23:54:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 0x7fcf3c1e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1e8ae0 0x7fcf3c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d07d0 con 0x7fcf3c153fa0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c0cd870 con 0x7fcf3c1e8ae0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf43600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c071090 con 0x7fcf200039b0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49508/0 (socket says 172.21.15.46:49508) 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 -- 172.21.15.46:0/1984850422 learned_addr learned my addr 172.21.15.46:0/1984850422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 0x7fcf3c1e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf3be00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1e8ae0 0x7fcf3c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 -- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 msgr2=0x7fcf3c1e85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 0x7fcf3c1e85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 -- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1e8ae0 msgr2=0x7fcf3c0c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1e8ae0 0x7fcf3c0c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf41600640 1 -- 172.21.15.46:0/1984850422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c1e9650 con 0x7fcf3c153fa0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.529+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 0x7fcf3c1e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf41600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c073230 tx=0x7fcf34011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34019020 con 0x7fcf3c153fa0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf34003610 con 0x7fcf3c153fa0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34003a00 con 0x7fcf3c153fa0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c1e9090 con 0x7fcf3c153fa0 2024-04-26T23:54:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.530+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c160990 con 0x7fcf3c153fa0 2024-04-26T23:54:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.531+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c15ece0 con 0x7fcf3c153fa0 2024-04-26T23:54:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.532+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf3401c020 con 0x7fcf3c153fa0 2024-04-26T23:54:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.533+0000 7fcf38c00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 0x7fcf1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.533+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fcf3408bee0 con 0x7fcf3c153fa0 2024-04-26T23:54:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.534+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 0x7fcf1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.535+0000 7fcf40c00640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 0x7fcf1806ede0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c035000 tx=0x7fcf2c02f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.538+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf3c15ece0 con 0x7fcf3c153fa0 2024-04-26T23:54:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.781+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf3c069530 con 0x7fcf3c153fa0 2024-04-26T23:54:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.782+0000 7fcf38c00640 1 -- 172.21.15.46:0/1984850422 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7fcf34023030 con 0x7fcf3c153fa0 2024-04-26T23:54:28.785 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:28.785 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:54:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:54:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 msgr2=0x7fcf1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 0x7fcf1806ede0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c035000 tx=0x7fcf2c02f000 comp rx=0 tx=0).stop 2024-04-26T23:54:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 msgr2=0x7fcf3c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c073230 tx=0x7fcf34011920 comp rx=0 tx=0).stop 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 shutdown_connections 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf1806c920 0x7fcf1806ede0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf3c153fa0 0x7fcf3c0c6fb0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1e8ae0 0x7fcf3c0c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 --2- 172.21.15.46:0/1984850422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf200039b0 0x7fcf3c1e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 >> 172.21.15.46:0/1984850422 conn(0x7fcf3c0d2f40 msgr2=0x7fcf3c070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.785+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 shutdown_connections 2024-04-26T23:54:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:28.786+0000 7fcf43600640 1 -- 172.21.15.46:0/1984850422 wait complete. 2024-04-26T23:54:28.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.019+0000 7fb459400640 1 Processor -- start 2024-04-26T23:54:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.020+0000 7fb459400640 1 -- start start 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb459400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb45415f1d0 0x7fb4540cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb459400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4540b03c0 con 0x7fb45415edf0 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb459400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4540cb680 con 0x7fb4540cd770 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb459400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb4540cf5c0 con 0x7fb45415f1d0 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb453400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb45415f1d0 0x7fb4540cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb453e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb45415edf0 0x7fb4540cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57702/0 (socket says 172.21.15.46:57702) 2024-04-26T23:54:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.022+0000 7fb453e00640 1 -- 172.21.15.46:0/172217398 learned_addr learned my addr 172.21.15.46:0/172217398 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.023+0000 7fb453400640 1 -- 172.21.15.46:0/172217398 >> v1:172.21.15.142:6789/0 conn(0x7fb4540cd770 legacy=0x7fb4540c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.023+0000 7fb453400640 1 -- 172.21.15.46:0/172217398 >> v1:172.21.15.46:6789/0 conn(0x7fb45415edf0 legacy=0x7fb4540cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.023+0000 7fb452a00640 1 -- 172.21.15.46:0/172217398 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227906394 0 0) 0x7fb4540b03c0 con 0x7fb45415edf0 2024-04-26T23:54:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.023+0000 7fb453400640 1 -- 172.21.15.46:0/172217398 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540cb960 con 0x7fb45415f1d0 2024-04-26T23:54:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.024+0000 7fb453400640 1 --2- 172.21.15.46:0/172217398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415f1d0 0x7fb4540cf080 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fb444009d90 tx=0x7fb44402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e26d0af740377fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.024+0000 7fb452a00640 1 -- 172.21.15.46:0/172217398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444007740 con 0x7fb45415f1d0 2024-04-26T23:54:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.024+0000 7fb452a00640 1 -- 172.21.15.46:0/172217398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4440078e0 con 0x7fb45415f1d0 2024-04-26T23:54:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.024+0000 7fb452a00640 1 -- 172.21.15.46:0/172217398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444007cb0 con 0x7fb45415f1d0 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 -- 172.21.15.46:0/172217398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415f1d0 msgr2=0x7fb4540cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 --2- 172.21.15.46:0/172217398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415f1d0 0x7fb4540cf080 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fb444009d90 tx=0x7fb44402a730 comp rx=0 tx=0).stop 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 -- 172.21.15.46:0/172217398 shutdown_connections 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 --2- 172.21.15.46:0/172217398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415f1d0 0x7fb4540cf080 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 -- 172.21.15.46:0/172217398 >> 172.21.15.46:0/172217398 conn(0x7fb4540b26d0 msgr2=0x7fb4540b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 -- 172.21.15.46:0/172217398 shutdown_connections 2024-04-26T23:54:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.025+0000 7fb459400640 1 -- 172.21.15.46:0/172217398 wait complete. 2024-04-26T23:54:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.027+0000 7fb459400640 1 Processor -- start 2024-04-26T23:54:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.027+0000 7fb459400640 1 -- start start 2024-04-26T23:54:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4540cd770 0x7fb454153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 0x7fb4540d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4540cc200 con 0x7fb45415edf0 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4540cbce0 con 0x7fb45406feb0 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb459400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4540cb1a0 con 0x7fb4540cd770 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb453400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb453400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49528/0 (socket says 172.21.15.46:49528) 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.028+0000 7fb453400640 1 -- 172.21.15.46:0/2082518708 learned_addr learned my addr 172.21.15.46:0/2082518708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:29.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453e00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4540cd770 0x7fb454153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb458a00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 0x7fb4540d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 -- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4540cd770 msgr2=0x7fb454153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4540cd770 0x7fb454153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 -- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 msgr2=0x7fb4540d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 0x7fb4540d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 -- 172.21.15.46:0/2082518708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540b5a30 con 0x7fb45415edf0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb458a00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 0x7fb4540d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.029+0000 7fb453400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fb444009d90 tx=0x7fb444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.030+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44403f070 con 0x7fb45415edf0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.030+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb444031d00 con 0x7fb45415edf0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.030+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444044680 con 0x7fb45415edf0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.030+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4540b5d10 con 0x7fb45415edf0 2024-04-26T23:54:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.030+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4540b62d0 con 0x7fb45415edf0 2024-04-26T23:54:29.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.031+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4540b66b0 con 0x7fb45415edf0 2024-04-26T23:54:29.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.032+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb44403b410 con 0x7fb45415edf0 2024-04-26T23:54:29.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.033+0000 7fb44be00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.033+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fb4440ad790 con 0x7fb45415edf0 2024-04-26T23:54:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.034+0000 7fb453e00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.035+0000 7fb453e00640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 0x7fb43006edb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000f60 tx=0x7fb43c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.038+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4540b66b0 con 0x7fb45415edf0 2024-04-26T23:54:29.144 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:54:29.141+0000 7fad74000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:54:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.243+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4540693b0 con 0x7fb45415edf0 2024-04-26T23:54:29.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.245+0000 7fb44be00640 1 -- 172.21.15.46:0/2082518708 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16852 (secure 0 0 0) 0x7fb44407b340 con 0x7fb45415edf0 2024-04-26T23:54:29.248 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:29.249 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":173,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:26.233645+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 msgr2=0x7fb43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 0x7fb43006edb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000f60 tx=0x7fb43c009040 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 msgr2=0x7fb45406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fb444009d90 tx=0x7fb444004a40 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 shutdown_connections 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb45406feb0 0x7fb4540d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45415edf0 0x7fb45406f970 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 --2- 172.21.15.46:0/2082518708 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4540cd770 0x7fb454153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 >> 172.21.15.46:0/2082518708 conn(0x7fb4540b26d0 msgr2=0x7fb4540ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 shutdown_connections 2024-04-26T23:54:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.248+0000 7fb459400640 1 -- 172.21.15.46:0/2082518708 wait complete. 2024-04-26T23:54:29.273 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:54:29.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:54:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.481+0000 7f245e200640 1 Processor -- start 2024-04-26T23:54:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.481+0000 7f245e200640 1 -- start start 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.482+0000 7f245e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f245806d670 0x7f2458160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.482+0000 7f245e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24581639b0 con 0x7f24580cece0 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.482+0000 7f245e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2458163c90 con 0x7f24580cfd00 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.482+0000 7f245e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f245806df10 con 0x7f245806d670 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.483+0000 7f245ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24580cece0 0x7f24580cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57708/0 (socket says 172.21.15.46:57708) 2024-04-26T23:54:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.483+0000 7f245ce00640 1 -- 172.21.15.46:0/2297460493 learned_addr learned my addr 172.21.15.46:0/2297460493 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.483+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> v2:172.21.15.142:3300/0 conn(0x7f245806d670 0x7f2458160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.483+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087268460 0 0) 0x7f24581639b0 con 0x7f24580cece0 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.483+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c003520 con 0x7f24580cece0 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791755443 0 0) 0x7f243c003520 con 0x7f24580cece0 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 >> v2:172.21.15.142:3300/0 conn(0x7f245806d670 msgr2=0x7f2458160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 --2- 172.21.15.46:0/2297460493 >> v2:172.21.15.142:3300/0 conn(0x7f245806d670 0x7f2458160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 >> v1:172.21.15.142:6789/0 conn(0x7f24580cfd00 legacy=0x7f245815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245806e1f0 con 0x7f24580cece0 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f24480019f0 con 0x7f24580cece0 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 >> v1:172.21.15.46:6789/0 conn(0x7f24580cece0 legacy=0x7f24580cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f243c003a00 0x7f243c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f243c004800 0x7f243c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f243c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f245806df10 con 0x7f243c005210 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f243c00c160 con 0x7f243c004800 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f243c00c360 con 0x7f243c003a00 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> v2:172.21.15.142:3300/0 conn(0x7f245806d670 0x7f2458160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f243c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.484+0000 7f245ce00640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f243c003a00 0x7f243c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f2457e00640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f243c004800 0x7f243c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f245d800640 1 -- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f243c003a00 msgr2=0x7f243c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f243c003a00 0x7f243c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f245d800640 1 -- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f243c004800 msgr2=0x7f243c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f243c004800 0x7f243c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f245d800640 1 -- 172.21.15.46:0/2297460493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c003520 con 0x7f243c005210 2024-04-26T23:54:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.485+0000 7f2457e00640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f243c004800 0x7f243c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.486+0000 7f245d800640 1 -- 172.21.15.46:0/2297460493 reap_dead start 2024-04-26T23:54:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.486+0000 7f245d800640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f243c00b9c0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f24500098a0 tx=0x7f243c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bbda94a7287da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.486+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450037070 con 0x7f243c005210 2024-04-26T23:54:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.486+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245002ad40 con 0x7f243c005210 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.486+0000 7f2457400640 1 -- 172.21.15.46:0/2297460493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245000f040 con 0x7f243c005210 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 -- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 msgr2=0x7f243c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f243c00b9c0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f24500098a0 tx=0x7f243c005830 comp rx=0 tx=0).stop 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 -- 172.21.15.46:0/2297460493 shutdown_connections 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 --2- 172.21.15.46:0/2297460493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f243c00b9c0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 -- 172.21.15.46:0/2297460493 >> 172.21.15.46:0/2297460493 conn(0x7f24580d2d70 msgr2=0x7f24580d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 -- 172.21.15.46:0/2297460493 shutdown_connections 2024-04-26T23:54:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.487+0000 7f245e200640 1 -- 172.21.15.46:0/2297460493 wait complete. 2024-04-26T23:54:29.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.489+0000 7f245e200640 1 Processor -- start 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.489+0000 7f245e200640 1 -- start start 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 0x7f2458157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 0x7f24580c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24580cf0c0 con 0x7f243c005210 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24580cd3c0 con 0x7f24580c8a70 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f24580cf240 con 0x7f24580b72b0 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49542/0 (socket says 172.21.15.46:49542) 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.490+0000 7f245ce00640 1 -- 172.21.15.46:0/1668184446 learned_addr learned my addr 172.21.15.46:0/1668184446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f2457e00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 0x7f2458157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245d800640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 0x7f24580c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 -- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 msgr2=0x7f2458157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 0x7f2458157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 -- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 msgr2=0x7f24580c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 0x7f24580c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 -- 172.21.15.46:0/1668184446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24580b7790 con 0x7f243c005210 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245d800640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 0x7f24580c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f2457e00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 0x7f2458157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245ce00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f2448007a10 tx=0x7f244800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244800edf0 con 0x7f243c005210 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2448018070 con 0x7f243c005210 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448013660 con 0x7f243c005210 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.491+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24580b7a70 con 0x7f243c005210 2024-04-26T23:54:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.492+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24580b8030 con 0x7f243c005210 2024-04-26T23:54:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.493+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2458069ac0 con 0x7f243c005210 2024-04-26T23:54:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.494+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2448013800 con 0x7f243c005210 2024-04-26T23:54:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.495+0000 7f2454c00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.495+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7f244808c250 con 0x7f243c005210 2024-04-26T23:54:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.496+0000 7f2457e00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.497+0000 7f2457e00640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 0x7f243406ede0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f24580c96b0 tx=0x7f2440009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.500+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f244805a4f0 con 0x7f243c005210 2024-04-26T23:54:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.705+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f245806f9a0 con 0x7f243406c920 2024-04-26T23:54:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.725+0000 7f2454c00640 1 -- 172.21.15.46:0/1668184446 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f245806f9a0 con 0x7f243406c920 2024-04-26T23:54:29.732 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 msgr2=0x7f243406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 0x7f243406ede0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f24580c96b0 tx=0x7f2440009040 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 msgr2=0x7f24581f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f2448007a10 tx=0x7f244800e530 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 shutdown_connections 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580c8a70 0x7f24580c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24580b72b0 0x7f2458157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 --2- 172.21.15.46:0/1668184446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f243c005210 0x7f24581f4670 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 >> 172.21.15.46:0/1668184446 conn(0x7f24580d2d70 msgr2=0x7f24580cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 shutdown_connections 2024-04-26T23:54:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:29.730+0000 7f245e200640 1 -- 172.21.15.46:0/1668184446 wait complete. 2024-04-26T23:54:29.753 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2024-04-26T23:54:28.280989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":436964,"kb_used_data":18152,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754537756,"statfs":{"total":773094113280,"available":772646662144,"internally_reserved":0,"allocated":18587648,"data_stored":9359276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":428723761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103010"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270341+0000","last_change":"2024-04-26T23:54:25.261837+0000","last_active":"2024-04-26T23:54:25.270341+0000","last_peered":"2024-04-26T23:54:25.270341+0000","last_clean":"2024-04-26T23:54:25.270341+0000","last_became_active":"2024-04-26T23:54:25.261537+0000","last_became_peered":"2024-04-26T23:54:25.261537+0000","last_unstale":"2024-04-26T23:54:25.270341+0000","last_undegraded":"2024-04-26T23:54:25.270341+0000","last_fullsized":"2024-04-26T23:54:25.270341+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.263863+0000","last_change":"2024-04-26T23:54:25.248982+0000","last_active":"2024-04-26T23:54:25.263863+0000","last_peered":"2024-04-26T23:54:25.263863+0000","last_clean":"2024-04-26T23:54:25.263863+0000","last_became_active":"2024-04-26T23:54:25.248618+0000","last_became_peered":"2024-04-26T23:54:25.248618+0000","last_unstale":"2024-04-26T23:54:25.263863+0000","last_undegraded":"2024-04-26T23:54:25.263863+0000","last_fullsized":"2024-04-26T23:54:25.263863+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:37.869900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254426+0000","last_change":"2024-04-26T23:54:25.243080+0000","last_active":"2024-04-26T23:54:25.254426+0000","last_peered":"2024-04-26T23:54:25.254426+0000","last_clean":"2024-04-26T23:54:25.254426+0000","last_became_active":"2024-04-26T23:54:25.242348+0000","last_became_peered":"2024-04-26T23:54:25.242348+0000","last_unstale":"2024-04-26T23:54:25.254426+0000","last_undegraded":"2024-04-26T23:54:25.254426+0000","last_fullsized":"2024-04-26T23:54:25.254426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:31.622675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.243986+0000","last_change":"2024-04-26T23:54:25.242541+0000","last_active":"2024-04-26T23:54:25.243986+0000","last_peered":"2024-04-26T23:54:25.243986+0000","last_clean":"2024-04-26T23:54:25.243986+0000","last_became_active":"2024-04-26T23:54:25.241247+0000","last_became_peered":"2024-04-26T23:54:25.241247+0000","last_unstale":"2024-04-26T23:54:25.243986+0000","last_undegraded":"2024-04-26T23:54:25.243986+0000","last_fullsized":"2024-04-26T23:54:25.243986+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:59.027093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259859+0000","last_change":"2024-04-26T23:54:25.252375+0000","last_active":"2024-04-26T23:54:25.259859+0000","last_peered":"2024-04-26T23:54:25.259859+0000","last_clean":"2024-04-26T23:54:25.259859+0000","last_became_active":"2024-04-26T23:54:25.251918+0000","last_became_peered":"2024-04-26T23:54:25.251918+0000","last_unstale":"2024-04-26T23:54:25.259859+0000","last_undegraded":"2024-04-26T23:54:25.259859+0000","last_fullsized":"2024-04-26T23:54:25.259859+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:17.635409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257722+0000","last_change":"2024-04-26T23:54:25.250136+0000","last_active":"2024-04-26T23:54:25.257722+0000","last_peered":"2024-04-26T23:54:25.257722+0000","last_clean":"2024-04-26T23:54:25.257722+0000","last_became_active":"2024-04-26T23:54:25.249852+0000","last_became_peered":"2024-04-26T23:54:25.249852+0000","last_unstale":"2024-04-26T23:54:25.257722+0000","last_undegraded":"2024-04-26T23:54:25.257722+0000","last_fullsized":"2024-04-26T23:54:25.257722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:26.443604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264174+0000","last_change":"2024-04-26T23:54:25.255878+0000","last_active":"2024-04-26T23:54:25.264174+0000","last_peered":"2024-04-26T23:54:25.264174+0000","last_clean":"2024-04-26T23:54:25.264174+0000","last_became_active":"2024-04-26T23:54:25.255567+0000","last_became_peered":"2024-04-26T23:54:25.255567+0000","last_unstale":"2024-04-26T23:54:25.264174+0000","last_undegraded":"2024-04-26T23:54:25.264174+0000","last_fullsized":"2024-04-26T23:54:25.264174+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:39.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.252654+0000","last_change":"2024-04-26T23:54:25.248208+0000","last_active":"2024-04-26T23:54:25.252654+0000","last_peered":"2024-04-26T23:54:25.252654+0000","last_clean":"2024-04-26T23:54:25.252654+0000","last_became_active":"2024-04-26T23:54:25.247940+0000","last_became_peered":"2024-04-26T23:54:25.247940+0000","last_unstale":"2024-04-26T23:54:25.252654+0000","last_undegraded":"2024-04-26T23:54:25.252654+0000","last_fullsized":"2024-04-26T23:54:25.252654+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267712+0000","last_change":"2024-04-26T23:54:25.254642+0000","last_active":"2024-04-26T23:54:25.267712+0000","last_peered":"2024-04-26T23:54:25.267712+0000","last_clean":"2024-04-26T23:54:25.267712+0000","last_became_active":"2024-04-26T23:54:25.254344+0000","last_became_peered":"2024-04-26T23:54:25.254344+0000","last_unstale":"2024-04-26T23:54:25.267712+0000","last_undegraded":"2024-04-26T23:54:25.267712+0000","last_fullsized":"2024-04-26T23:54:25.267712+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:19.959605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258402+0000","last_change":"2024-04-26T23:54:25.246010+0000","last_active":"2024-04-26T23:54:25.258402+0000","last_peered":"2024-04-26T23:54:25.258402+0000","last_clean":"2024-04-26T23:54:25.258402+0000","last_became_active":"2024-04-26T23:54:25.245698+0000","last_became_peered":"2024-04-26T23:54:25.245698+0000","last_unstale":"2024-04-26T23:54:25.258402+0000","last_undegraded":"2024-04-26T23:54:25.258402+0000","last_fullsized":"2024-04-26T23:54:25.258402+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:45.253157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250814+0000","last_change":"2024-04-26T23:54:25.246889+0000","last_active":"2024-04-26T23:54:25.250814+0000","last_peered":"2024-04-26T23:54:25.250814+0000","last_clean":"2024-04-26T23:54:25.250814+0000","last_became_active":"2024-04-26T23:54:25.246554+0000","last_became_peered":"2024-04-26T23:54:25.246554+0000","last_unstale":"2024-04-26T23:54:25.250814+0000","last_undegraded":"2024-04-26T23:54:25.250814+0000","last_fullsized":"2024-04-26T23:54:25.250814+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266496+0000","last_change":"2024-04-26T23:54:25.256315+0000","last_active":"2024-04-26T23:54:25.266496+0000","last_peered":"2024-04-26T23:54:25.266496+0000","last_clean":"2024-04-26T23:54:25.266496+0000","last_became_active":"2024-04-26T23:54:25.256034+0000","last_became_peered":"2024-04-26T23:54:25.256034+0000","last_unstale":"2024-04-26T23:54:25.266496+0000","last_undegraded":"2024-04-26T23:54:25.266496+0000","last_fullsized":"2024-04-26T23:54:25.266496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:07:03.896763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259638+0000","last_change":"2024-04-26T23:54:25.256879+0000","last_active":"2024-04-26T23:54:25.259638+0000","last_peered":"2024-04-26T23:54:25.259638+0000","last_clean":"2024-04-26T23:54:25.259638+0000","last_became_active":"2024-04-26T23:54:25.254919+0000","last_became_peered":"2024-04-26T23:54:25.254919+0000","last_unstale":"2024-04-26T23:54:25.259638+0000","last_undegraded":"2024-04-26T23:54:25.259638+0000","last_fullsized":"2024-04-26T23:54:25.259638+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:26.481809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264879+0000","last_change":"2024-04-26T23:54:25.260923+0000","last_active":"2024-04-26T23:54:25.264879+0000","last_peered":"2024-04-26T23:54:25.264879+0000","last_clean":"2024-04-26T23:54:25.264879+0000","last_became_active":"2024-04-26T23:54:25.258488+0000","last_became_peered":"2024-04-26T23:54:25.258488+0000","last_unstale":"2024-04-26T23:54:25.264879+0000","last_undegraded":"2024-04-26T23:54:25.264879+0000","last_fullsized":"2024-04-26T23:54:25.264879+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:18.445359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270606+0000","last_change":"2024-04-26T23:54:25.261047+0000","last_active":"2024-04-26T23:54:25.270606+0000","last_peered":"2024-04-26T23:54:25.270606+0000","last_clean":"2024-04-26T23:54:25.270606+0000","last_became_active":"2024-04-26T23:54:25.260772+0000","last_became_peered":"2024-04-26T23:54:25.260772+0000","last_unstale":"2024-04-26T23:54:25.270606+0000","last_undegraded":"2024-04-26T23:54:25.270606+0000","last_fullsized":"2024-04-26T23:54:25.270606+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.261812+0000","last_change":"2024-04-26T23:54:25.256985+0000","last_active":"2024-04-26T23:54:25.261812+0000","last_peered":"2024-04-26T23:54:25.261812+0000","last_clean":"2024-04-26T23:54:25.261812+0000","last_became_active":"2024-04-26T23:54:25.250514+0000","last_became_peered":"2024-04-26T23:54:25.250514+0000","last_unstale":"2024-04-26T23:54:25.261812+0000","last_undegraded":"2024-04-26T23:54:25.261812+0000","last_fullsized":"2024-04-26T23:54:25.261812+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:36.773616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262936+0000","last_change":"2024-04-26T23:54:25.257990+0000","last_active":"2024-04-26T23:54:25.262936+0000","last_peered":"2024-04-26T23:54:25.262936+0000","last_clean":"2024-04-26T23:54:25.262936+0000","last_became_active":"2024-04-26T23:54:25.257562+0000","last_became_peered":"2024-04-26T23:54:25.257562+0000","last_unstale":"2024-04-26T23:54:25.262936+0000","last_undegraded":"2024-04-26T23:54:25.262936+0000","last_fullsized":"2024-04-26T23:54:25.262936+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:46.960946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259385+0000","last_change":"2024-04-26T23:54:25.256550+0000","last_active":"2024-04-26T23:54:25.259385+0000","last_peered":"2024-04-26T23:54:25.259385+0000","last_clean":"2024-04-26T23:54:25.259385+0000","last_became_active":"2024-04-26T23:54:25.256237+0000","last_became_peered":"2024-04-26T23:54:25.256237+0000","last_unstale":"2024-04-26T23:54:25.259385+0000","last_undegraded":"2024-04-26T23:54:25.259385+0000","last_fullsized":"2024-04-26T23:54:25.259385+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.917158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249696+0000","last_change":"2024-04-26T23:54:25.241963+0000","last_active":"2024-04-26T23:54:25.249696+0000","last_peered":"2024-04-26T23:54:25.249696+0000","last_clean":"2024-04-26T23:54:25.249696+0000","last_became_active":"2024-04-26T23:54:25.241670+0000","last_became_peered":"2024-04-26T23:54:25.241670+0000","last_unstale":"2024-04-26T23:54:25.249696+0000","last_undegraded":"2024-04-26T23:54:25.249696+0000","last_fullsized":"2024-04-26T23:54:25.249696+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:05.537898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262941+0000","last_change":"2024-04-26T23:54:25.258358+0000","last_active":"2024-04-26T23:54:25.262941+0000","last_peered":"2024-04-26T23:54:25.262941+0000","last_clean":"2024-04-26T23:54:25.262941+0000","last_became_active":"2024-04-26T23:54:25.258067+0000","last_became_peered":"2024-04-26T23:54:25.258067+0000","last_unstale":"2024-04-26T23:54:25.262941+0000","last_undegraded":"2024-04-26T23:54:25.262941+0000","last_fullsized":"2024-04-26T23:54:25.262941+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:33.590565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269743+0000","last_change":"2024-04-26T23:54:25.258093+0000","last_active":"2024-04-26T23:54:25.269743+0000","last_peered":"2024-04-26T23:54:25.269743+0000","last_clean":"2024-04-26T23:54:25.269743+0000","last_became_active":"2024-04-26T23:54:25.256694+0000","last_became_peered":"2024-04-26T23:54:25.256694+0000","last_unstale":"2024-04-26T23:54:25.269743+0000","last_undegraded":"2024-04-26T23:54:25.269743+0000","last_fullsized":"2024-04-26T23:54:25.269743+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270225+0000","last_change":"2024-04-26T23:54:25.260683+0000","last_active":"2024-04-26T23:54:25.270225+0000","last_peered":"2024-04-26T23:54:25.270225+0000","last_clean":"2024-04-26T23:54:25.270225+0000","last_became_active":"2024-04-26T23:54:25.260289+0000","last_became_peered":"2024-04-26T23:54:25.260289+0000","last_unstale":"2024-04-26T23:54:25.270225+0000","last_undegraded":"2024-04-26T23:54:25.270225+0000","last_fullsized":"2024-04-26T23:54:25.270225+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254193+0000","last_change":"2024-04-26T23:54:25.250736+0000","last_active":"2024-04-26T23:54:25.254193+0000","last_peered":"2024-04-26T23:54:25.254193+0000","last_clean":"2024-04-26T23:54:25.254193+0000","last_became_active":"2024-04-26T23:54:25.248401+0000","last_became_peered":"2024-04-26T23:54:25.248401+0000","last_unstale":"2024-04-26T23:54:25.254193+0000","last_undegraded":"2024-04-26T23:54:25.254193+0000","last_fullsized":"2024-04-26T23:54:25.254193+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270220+0000","last_change":"2024-04-26T23:54:25.258526+0000","last_active":"2024-04-26T23:54:25.270220+0000","last_peered":"2024-04-26T23:54:25.270220+0000","last_clean":"2024-04-26T23:54:25.270220+0000","last_became_active":"2024-04-26T23:54:25.258213+0000","last_became_peered":"2024-04-26T23:54:25.258213+0000","last_unstale":"2024-04-26T23:54:25.270220+0000","last_undegraded":"2024-04-26T23:54:25.270220+0000","last_fullsized":"2024-04-26T23:54:25.270220+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264850+0000","last_change":"2024-04-26T23:54:25.261346+0000","last_active":"2024-04-26T23:54:25.264850+0000","last_peered":"2024-04-26T23:54:25.264850+0000","last_clean":"2024-04-26T23:54:25.264850+0000","last_became_active":"2024-04-26T23:54:25.259492+0000","last_became_peered":"2024-04-26T23:54:25.259492+0000","last_unstale":"2024-04-26T23:54:25.264850+0000","last_undegraded":"2024-04-26T23:54:25.264850+0000","last_fullsized":"2024-04-26T23:54:25.264850+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:40.426257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262161+0000","last_change":"2024-04-26T23:54:25.257230+0000","last_active":"2024-04-26T23:54:25.262161+0000","last_peered":"2024-04-26T23:54:25.262161+0000","last_clean":"2024-04-26T23:54:25.262161+0000","last_became_active":"2024-04-26T23:54:25.255556+0000","last_became_peered":"2024-04-26T23:54:25.255556+0000","last_unstale":"2024-04-26T23:54:25.262161+0000","last_undegraded":"2024-04-26T23:54:25.262161+0000","last_fullsized":"2024-04-26T23:54:25.262161+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254995+0000","last_change":"2024-04-26T23:54:25.251888+0000","last_active":"2024-04-26T23:54:25.254995+0000","last_peered":"2024-04-26T23:54:25.254995+0000","last_clean":"2024-04-26T23:54:25.254995+0000","last_became_active":"2024-04-26T23:54:25.251588+0000","last_became_peered":"2024-04-26T23:54:25.251588+0000","last_unstale":"2024-04-26T23:54:25.254995+0000","last_undegraded":"2024-04-26T23:54:25.254995+0000","last_fullsized":"2024-04-26T23:54:25.254995+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258505+0000","last_change":"2024-04-26T23:54:25.254802+0000","last_active":"2024-04-26T23:54:25.258505+0000","last_peered":"2024-04-26T23:54:25.258505+0000","last_clean":"2024-04-26T23:54:25.258505+0000","last_became_active":"2024-04-26T23:54:25.254510+0000","last_became_peered":"2024-04-26T23:54:25.254510+0000","last_unstale":"2024-04-26T23:54:25.258505+0000","last_undegraded":"2024-04-26T23:54:25.258505+0000","last_fullsized":"2024-04-26T23:54:25.258505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:09.335134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250730+0000","last_change":"2024-04-26T23:54:25.247869+0000","last_active":"2024-04-26T23:54:25.250730+0000","last_peered":"2024-04-26T23:54:25.250730+0000","last_clean":"2024-04-26T23:54:25.250730+0000","last_became_active":"2024-04-26T23:54:25.247575+0000","last_became_peered":"2024-04-26T23:54:25.247575+0000","last_unstale":"2024-04-26T23:54:25.250730+0000","last_undegraded":"2024-04-26T23:54:25.250730+0000","last_fullsized":"2024-04-26T23:54:25.250730+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249217+0000","last_change":"2024-04-26T23:54:25.244372+0000","last_active":"2024-04-26T23:54:25.249217+0000","last_peered":"2024-04-26T23:54:25.249217+0000","last_clean":"2024-04-26T23:54:25.249217+0000","last_became_active":"2024-04-26T23:54:25.244052+0000","last_became_peered":"2024-04-26T23:54:25.244052+0000","last_unstale":"2024-04-26T23:54:25.249217+0000","last_undegraded":"2024-04-26T23:54:25.249217+0000","last_fullsized":"2024-04-26T23:54:25.249217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250520+0000","last_change":"2024-04-26T23:54:25.248477+0000","last_active":"2024-04-26T23:54:25.250520+0000","last_peered":"2024-04-26T23:54:25.250520+0000","last_clean":"2024-04-26T23:54:25.250520+0000","last_became_active":"2024-04-26T23:54:25.247171+0000","last_became_peered":"2024-04-26T23:54:25.247171+0000","last_unstale":"2024-04-26T23:54:25.250520+0000","last_undegraded":"2024-04-26T23:54:25.250520+0000","last_fullsized":"2024-04-26T23:54:25.250520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:46.167514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257326+0000","last_change":"2024-04-26T23:54:25.247313+0000","last_active":"2024-04-26T23:54:25.257326+0000","last_peered":"2024-04-26T23:54:25.257326+0000","last_clean":"2024-04-26T23:54:25.257326+0000","last_became_active":"2024-04-26T23:54:25.246996+0000","last_became_peered":"2024-04-26T23:54:25.246996+0000","last_unstale":"2024-04-26T23:54:25.257326+0000","last_undegraded":"2024-04-26T23:54:25.257326+0000","last_fullsized":"2024-04-26T23:54:25.257326+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:37.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.268644+0000","last_change":"2024-04-26T23:54:25.267504+0000","last_active":"2024-04-26T23:54:25.268644+0000","last_peered":"2024-04-26T23:54:25.268644+0000","last_clean":"2024-04-26T23:54:25.268644+0000","last_became_active":"2024-04-26T23:54:25.267208+0000","last_became_peered":"2024-04-26T23:54:25.267208+0000","last_unstale":"2024-04-26T23:54:25.268644+0000","last_undegraded":"2024-04-26T23:54:25.268644+0000","last_fullsized":"2024-04-26T23:54:25.268644+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:52.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"172'4","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267354+0000","last_change":"2024-04-26T23:54:24.228872+0000","last_active":"2024-04-26T23:54:25.267354+0000","last_peered":"2024-04-26T23:54:25.267354+0000","last_clean":"2024-04-26T23:54:25.267354+0000","last_became_active":"2024-04-26T23:54:24.228459+0000","last_became_peered":"2024-04-26T23:54:24.228459+0000","last_unstale":"2024-04-26T23:54:25.267354+0000","last_undegraded":"2024-04-26T23:54:25.267354+0000","last_fullsized":"2024-04-26T23:54:25.267354+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:35.474770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264811+0000","last_change":"2024-04-26T23:54:25.257415+0000","last_active":"2024-04-26T23:54:25.264811+0000","last_peered":"2024-04-26T23:54:25.264811+0000","last_clean":"2024-04-26T23:54:25.264811+0000","last_became_active":"2024-04-26T23:54:25.257076+0000","last_became_peered":"2024-04-26T23:54:25.257076+0000","last_unstale":"2024-04-26T23:54:25.264811+0000","last_undegraded":"2024-04-26T23:54:25.264811+0000","last_fullsized":"2024-04-26T23:54:25.264811+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:58.046695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.272865+0000","last_change":"2024-04-26T23:54:24.214355+0000","last_active":"2024-04-26T23:54:25.272865+0000","last_peered":"2024-04-26T23:54:25.272865+0000","last_clean":"2024-04-26T23:54:25.272865+0000","last_became_active":"2024-04-26T23:54:24.214068+0000","last_became_peered":"2024-04-26T23:54:24.214068+0000","last_unstale":"2024-04-26T23:54:25.272865+0000","last_undegraded":"2024-04-26T23:54:25.272865+0000","last_fullsized":"2024-04-26T23:54:25.272865+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:55.333276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258295+0000","last_change":"2024-04-26T23:54:25.255571+0000","last_active":"2024-04-26T23:54:25.258295+0000","last_peered":"2024-04-26T23:54:25.258295+0000","last_clean":"2024-04-26T23:54:25.258295+0000","last_became_active":"2024-04-26T23:54:25.255263+0000","last_became_peered":"2024-04-26T23:54:25.255263+0000","last_unstale":"2024-04-26T23:54:25.258295+0000","last_undegraded":"2024-04-26T23:54:25.258295+0000","last_fullsized":"2024-04-26T23:54:25.258295+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269857+0000","last_change":"2024-04-26T23:54:24.213242+0000","last_active":"2024-04-26T23:54:25.269857+0000","last_peered":"2024-04-26T23:54:25.269857+0000","last_clean":"2024-04-26T23:54:25.269857+0000","last_became_active":"2024-04-26T23:54:24.212936+0000","last_became_peered":"2024-04-26T23:54:24.212936+0000","last_unstale":"2024-04-26T23:54:25.269857+0000","last_undegraded":"2024-04-26T23:54:25.269857+0000","last_fullsized":"2024-04-26T23:54:25.269857+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:36.854946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254127+0000","last_change":"2024-04-26T23:54:25.252136+0000","last_active":"2024-04-26T23:54:25.254127+0000","last_peered":"2024-04-26T23:54:25.254127+0000","last_clean":"2024-04-26T23:54:25.254127+0000","last_became_active":"2024-04-26T23:54:25.251799+0000","last_became_peered":"2024-04-26T23:54:25.251799+0000","last_unstale":"2024-04-26T23:54:25.254127+0000","last_undegraded":"2024-04-26T23:54:25.254127+0000","last_fullsized":"2024-04-26T23:54:25.254127+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:58.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269163+0000","last_change":"2024-04-26T23:54:24.216075+0000","last_active":"2024-04-26T23:54:25.269163+0000","last_peered":"2024-04-26T23:54:25.269163+0000","last_clean":"2024-04-26T23:54:25.269163+0000","last_became_active":"2024-04-26T23:54:24.215703+0000","last_became_peered":"2024-04-26T23:54:24.215703+0000","last_unstale":"2024-04-26T23:54:25.269163+0000","last_undegraded":"2024-04-26T23:54:25.269163+0000","last_fullsized":"2024-04-26T23:54:25.269163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:09.914977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.260828+0000","last_change":"2024-04-26T23:54:25.259527+0000","last_active":"2024-04-26T23:54:25.260828+0000","last_peered":"2024-04-26T23:54:25.260828+0000","last_clean":"2024-04-26T23:54:25.260828+0000","last_became_active":"2024-04-26T23:54:25.258807+0000","last_became_peered":"2024-04-26T23:54:25.258807+0000","last_unstale":"2024-04-26T23:54:25.260828+0000","last_undegraded":"2024-04-26T23:54:25.260828+0000","last_fullsized":"2024-04-26T23:54:25.260828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:58.869636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.272928+0000","last_change":"2024-04-26T23:54:24.218389+0000","last_active":"2024-04-26T23:54:25.272928+0000","last_peered":"2024-04-26T23:54:25.272928+0000","last_clean":"2024-04-26T23:54:25.272928+0000","last_became_active":"2024-04-26T23:54:24.218048+0000","last_became_peered":"2024-04-26T23:54:24.218048+0000","last_unstale":"2024-04-26T23:54:25.272928+0000","last_undegraded":"2024-04-26T23:54:25.272928+0000","last_fullsized":"2024-04-26T23:54:25.272928+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:47.057131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.252731+0000","last_change":"2024-04-26T23:54:25.246361+0000","last_active":"2024-04-26T23:54:25.252731+0000","last_peered":"2024-04-26T23:54:25.252731+0000","last_clean":"2024-04-26T23:54:25.252731+0000","last_became_active":"2024-04-26T23:54:25.246067+0000","last_became_peered":"2024-04-26T23:54:25.246067+0000","last_unstale":"2024-04-26T23:54:25.252731+0000","last_undegraded":"2024-04-26T23:54:25.252731+0000","last_fullsized":"2024-04-26T23:54:25.252731+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:36.528174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270461+0000","last_change":"2024-04-26T23:54:24.209707+0000","last_active":"2024-04-26T23:54:25.270461+0000","last_peered":"2024-04-26T23:54:25.270461+0000","last_clean":"2024-04-26T23:54:25.270461+0000","last_became_active":"2024-04-26T23:54:24.208514+0000","last_became_peered":"2024-04-26T23:54:24.208514+0000","last_unstale":"2024-04-26T23:54:25.270461+0000","last_undegraded":"2024-04-26T23:54:25.270461+0000","last_fullsized":"2024-04-26T23:54:25.270461+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.803222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271243+0000","last_change":"2024-04-26T23:54:25.268200+0000","last_active":"2024-04-26T23:54:25.271243+0000","last_peered":"2024-04-26T23:54:25.271243+0000","last_clean":"2024-04-26T23:54:25.271243+0000","last_became_active":"2024-04-26T23:54:25.265951+0000","last_became_peered":"2024-04-26T23:54:25.265951+0000","last_unstale":"2024-04-26T23:54:25.271243+0000","last_undegraded":"2024-04-26T23:54:25.271243+0000","last_fullsized":"2024-04-26T23:54:25.271243+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:05.908781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.230058+0000","last_change":"2024-04-26T23:54:24.217316+0000","last_active":"2024-04-26T23:54:25.230058+0000","last_peered":"2024-04-26T23:54:25.230058+0000","last_clean":"2024-04-26T23:54:25.230058+0000","last_became_active":"2024-04-26T23:54:24.217024+0000","last_became_peered":"2024-04-26T23:54:24.217024+0000","last_unstale":"2024-04-26T23:54:25.230058+0000","last_undegraded":"2024-04-26T23:54:25.230058+0000","last_fullsized":"2024-04-26T23:54:25.230058+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:41.845328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264508+0000","last_change":"2024-04-26T23:54:25.262805+0000","last_active":"2024-04-26T23:54:25.264508+0000","last_peered":"2024-04-26T23:54:25.264508+0000","last_clean":"2024-04-26T23:54:25.264508+0000","last_became_active":"2024-04-26T23:54:25.262415+0000","last_became_peered":"2024-04-26T23:54:25.262415+0000","last_unstale":"2024-04-26T23:54:25.264508+0000","last_undegraded":"2024-04-26T23:54:25.264508+0000","last_fullsized":"2024-04-26T23:54:25.264508+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.235187+0000","last_change":"2024-04-26T23:54:24.213361+0000","last_active":"2024-04-26T23:54:25.235187+0000","last_peered":"2024-04-26T23:54:25.235187+0000","last_clean":"2024-04-26T23:54:25.235187+0000","last_became_active":"2024-04-26T23:54:24.212542+0000","last_became_peered":"2024-04-26T23:54:24.212542+0000","last_unstale":"2024-04-26T23:54:25.235187+0000","last_undegraded":"2024-04-26T23:54:25.235187+0000","last_fullsized":"2024-04-26T23:54:25.235187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:49:12.024881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257611+0000","last_change":"2024-04-26T23:54:25.246219+0000","last_active":"2024-04-26T23:54:25.257611+0000","last_peered":"2024-04-26T23:54:25.257611+0000","last_clean":"2024-04-26T23:54:25.257611+0000","last_became_active":"2024-04-26T23:54:25.245293+0000","last_became_peered":"2024-04-26T23:54:25.245293+0000","last_unstale":"2024-04-26T23:54:25.257611+0000","last_undegraded":"2024-04-26T23:54:25.257611+0000","last_fullsized":"2024-04-26T23:54:25.257611+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:25.394034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.231369+0000","last_change":"2024-04-26T23:54:24.211435+0000","last_active":"2024-04-26T23:54:25.231369+0000","last_peered":"2024-04-26T23:54:25.231369+0000","last_clean":"2024-04-26T23:54:25.231369+0000","last_became_active":"2024-04-26T23:54:24.211135+0000","last_became_peered":"2024-04-26T23:54:24.211135+0000","last_unstale":"2024-04-26T23:54:25.231369+0000","last_undegraded":"2024-04-26T23:54:25.231369+0000","last_fullsized":"2024-04-26T23:54:25.231369+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:44.197712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.251436+0000","last_change":"2024-04-26T23:54:25.245299+0000","last_active":"2024-04-26T23:54:25.251436+0000","last_peered":"2024-04-26T23:54:25.251436+0000","last_clean":"2024-04-26T23:54:25.251436+0000","last_became_active":"2024-04-26T23:54:25.241634+0000","last_became_peered":"2024-04-26T23:54:25.241634+0000","last_unstale":"2024-04-26T23:54:25.251436+0000","last_undegraded":"2024-04-26T23:54:25.251436+0000","last_fullsized":"2024-04-26T23:54:25.251436+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:33.430183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"172'3","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271116+0000","last_change":"2024-04-26T23:54:24.215184+0000","last_active":"2024-04-26T23:54:25.271116+0000","last_peered":"2024-04-26T23:54:25.271116+0000","last_clean":"2024-04-26T23:54:25.271116+0000","last_became_active":"2024-04-26T23:54:24.213522+0000","last_became_peered":"2024-04-26T23:54:24.213522+0000","last_unstale":"2024-04-26T23:54:25.271116+0000","last_undegraded":"2024-04-26T23:54:25.271116+0000","last_fullsized":"2024-04-26T23:54:25.271116+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:57.498903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266001+0000","last_change":"2024-04-26T23:54:25.249362+0000","last_active":"2024-04-26T23:54:25.266001+0000","last_peered":"2024-04-26T23:54:25.266001+0000","last_clean":"2024-04-26T23:54:25.266001+0000","last_became_active":"2024-04-26T23:54:25.249011+0000","last_became_peered":"2024-04-26T23:54:25.249011+0000","last_unstale":"2024-04-26T23:54:25.266001+0000","last_undegraded":"2024-04-26T23:54:25.266001+0000","last_fullsized":"2024-04-26T23:54:25.266001+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:36.545064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"172'6","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.274751+0000","last_change":"2024-04-26T23:54:24.209823+0000","last_active":"2024-04-26T23:54:25.274751+0000","last_peered":"2024-04-26T23:54:25.274751+0000","last_clean":"2024-04-26T23:54:25.274751+0000","last_became_active":"2024-04-26T23:54:24.209485+0000","last_became_peered":"2024-04-26T23:54:24.209485+0000","last_unstale":"2024-04-26T23:54:25.274751+0000","last_undegraded":"2024-04-26T23:54:25.274751+0000","last_fullsized":"2024-04-26T23:54:25.274751+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:10.906275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264129+0000","last_change":"2024-04-26T23:54:25.254853+0000","last_active":"2024-04-26T23:54:25.264129+0000","last_peered":"2024-04-26T23:54:25.264129+0000","last_clean":"2024-04-26T23:54:25.264129+0000","last_became_active":"2024-04-26T23:54:25.254573+0000","last_became_peered":"2024-04-26T23:54:25.254573+0000","last_unstale":"2024-04-26T23:54:25.264129+0000","last_undegraded":"2024-04-26T23:54:25.264129+0000","last_fullsized":"2024-04-26T23:54:25.264129+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:27.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271797+0000","last_change":"2024-04-26T23:54:24.201944+0000","last_active":"2024-04-26T23:54:25.271797+0000","last_peered":"2024-04-26T23:54:25.271797+0000","last_clean":"2024-04-26T23:54:25.271797+0000","last_became_active":"2024-04-26T23:54:24.201549+0000","last_became_peered":"2024-04-26T23:54:24.201549+0000","last_unstale":"2024-04-26T23:54:25.271797+0000","last_undegraded":"2024-04-26T23:54:25.271797+0000","last_fullsized":"2024-04-26T23:54:25.271797+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:51.512188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258350+0000","last_change":"2024-04-26T23:54:25.253361+0000","last_active":"2024-04-26T23:54:25.258350+0000","last_peered":"2024-04-26T23:54:25.258350+0000","last_clean":"2024-04-26T23:54:25.258350+0000","last_became_active":"2024-04-26T23:54:25.250441+0000","last_became_peered":"2024-04-26T23:54:25.250441+0000","last_unstale":"2024-04-26T23:54:25.258350+0000","last_undegraded":"2024-04-26T23:54:25.258350+0000","last_fullsized":"2024-04-26T23:54:25.258350+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.489417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.237106+0000","last_change":"2024-04-26T23:54:24.198686+0000","last_active":"2024-04-26T23:54:25.237106+0000","last_peered":"2024-04-26T23:54:25.237106+0000","last_clean":"2024-04-26T23:54:25.237106+0000","last_became_active":"2024-04-26T23:54:24.197578+0000","last_became_peered":"2024-04-26T23:54:24.197578+0000","last_unstale":"2024-04-26T23:54:25.237106+0000","last_undegraded":"2024-04-26T23:54:25.237106+0000","last_fullsized":"2024-04-26T23:54:25.237106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:58.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.256693+0000","last_change":"2024-04-26T23:54:25.254151+0000","last_active":"2024-04-26T23:54:25.256693+0000","last_peered":"2024-04-26T23:54:25.256693+0000","last_clean":"2024-04-26T23:54:25.256693+0000","last_became_active":"2024-04-26T23:54:25.252802+0000","last_became_peered":"2024-04-26T23:54:25.252802+0000","last_unstale":"2024-04-26T23:54:25.256693+0000","last_undegraded":"2024-04-26T23:54:25.256693+0000","last_fullsized":"2024-04-26T23:54:25.256693+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:55.423762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.215805+0000","last_change":"2024-04-26T23:54:24.212434+0000","last_active":"2024-04-26T23:54:25.215805+0000","last_peered":"2024-04-26T23:54:25.215805+0000","last_clean":"2024-04-26T23:54:25.215805+0000","last_became_active":"2024-04-26T23:54:24.210392+0000","last_became_peered":"2024-04-26T23:54:24.210392+0000","last_unstale":"2024-04-26T23:54:25.215805+0000","last_undegraded":"2024-04-26T23:54:25.215805+0000","last_fullsized":"2024-04-26T23:54:25.215805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:04.893826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257053+0000","last_change":"2024-04-26T23:54:25.252364+0000","last_active":"2024-04-26T23:54:25.257053+0000","last_peered":"2024-04-26T23:54:25.257053+0000","last_clean":"2024-04-26T23:54:25.257053+0000","last_became_active":"2024-04-26T23:54:25.250095+0000","last_became_peered":"2024-04-26T23:54:25.250095+0000","last_unstale":"2024-04-26T23:54:25.257053+0000","last_undegraded":"2024-04-26T23:54:25.257053+0000","last_fullsized":"2024-04-26T23:54:25.257053+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:53.430799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"172'8","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.273183+0000","last_change":"2024-04-26T23:54:24.215012+0000","last_active":"2024-04-26T23:54:25.273183+0000","last_peered":"2024-04-26T23:54:25.273183+0000","last_clean":"2024-04-26T23:54:25.273183+0000","last_became_active":"2024-04-26T23:54:24.214744+0000","last_became_peered":"2024-04-26T23:54:24.214744+0000","last_unstale":"2024-04-26T23:54:25.273183+0000","last_undegraded":"2024-04-26T23:54:25.273183+0000","last_fullsized":"2024-04-26T23:54:25.273183+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:35.530109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250467+0000","last_change":"2024-04-26T23:54:25.246075+0000","last_active":"2024-04-26T23:54:25.250467+0000","last_peered":"2024-04-26T23:54:25.250467+0000","last_clean":"2024-04-26T23:54:25.250467+0000","last_became_active":"2024-04-26T23:54:25.245799+0000","last_became_peered":"2024-04-26T23:54:25.245799+0000","last_unstale":"2024-04-26T23:54:25.250467+0000","last_undegraded":"2024-04-26T23:54:25.250467+0000","last_fullsized":"2024-04-26T23:54:25.250467+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.289408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271838+0000","last_change":"2024-04-26T23:54:24.214503+0000","last_active":"2024-04-26T23:54:25.271838+0000","last_peered":"2024-04-26T23:54:25.271838+0000","last_clean":"2024-04-26T23:54:25.271838+0000","last_became_active":"2024-04-26T23:54:24.212865+0000","last_became_peered":"2024-04-26T23:54:24.212865+0000","last_unstale":"2024-04-26T23:54:25.271838+0000","last_undegraded":"2024-04-26T23:54:25.271838+0000","last_fullsized":"2024-04-26T23:54:25.271838+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:40.201910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257631+0000","last_change":"2024-04-26T23:54:25.246481+0000","last_active":"2024-04-26T23:54:25.257631+0000","last_peered":"2024-04-26T23:54:25.257631+0000","last_clean":"2024-04-26T23:54:25.257631+0000","last_became_active":"2024-04-26T23:54:25.245912+0000","last_became_peered":"2024-04-26T23:54:25.245912+0000","last_unstale":"2024-04-26T23:54:25.257631+0000","last_undegraded":"2024-04-26T23:54:25.257631+0000","last_fullsized":"2024-04-26T23:54:25.257631+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:30.657975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266597+0000","last_change":"2024-04-26T23:54:24.213143+0000","last_active":"2024-04-26T23:54:25.266597+0000","last_peered":"2024-04-26T23:54:25.266597+0000","last_clean":"2024-04-26T23:54:25.266597+0000","last_became_active":"2024-04-26T23:54:24.212848+0000","last_became_peered":"2024-04-26T23:54:24.212848+0000","last_unstale":"2024-04-26T23:54:25.266597+0000","last_undegraded":"2024-04-26T23:54:25.266597+0000","last_fullsized":"2024-04-26T23:54:25.266597+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:20.630576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249171+0000","last_change":"2024-04-26T23:54:25.244991+0000","last_active":"2024-04-26T23:54:25.249171+0000","last_peered":"2024-04-26T23:54:25.249171+0000","last_clean":"2024-04-26T23:54:25.249171+0000","last_became_active":"2024-04-26T23:54:25.244441+0000","last_became_peered":"2024-04-26T23:54:25.244441+0000","last_unstale":"2024-04-26T23:54:25.249171+0000","last_undegraded":"2024-04-26T23:54:25.249171+0000","last_fullsized":"2024-04-26T23:54:25.249171+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267611+0000","last_change":"2024-04-26T23:54:24.215999+0000","last_active":"2024-04-26T23:54:25.267611+0000","last_peered":"2024-04-26T23:54:25.267611+0000","last_clean":"2024-04-26T23:54:25.267611+0000","last_became_active":"2024-04-26T23:54:24.215718+0000","last_became_peered":"2024-04-26T23:54:24.215718+0000","last_unstale":"2024-04-26T23:54:25.267611+0000","last_undegraded":"2024-04-26T23:54:25.267611+0000","last_fullsized":"2024-04-26T23:54:25.267611+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:39.411303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270365+0000","last_change":"2024-04-26T23:54:25.266154+0000","last_active":"2024-04-26T23:54:25.270365+0000","last_peered":"2024-04-26T23:54:25.270365+0000","last_clean":"2024-04-26T23:54:25.270365+0000","last_became_active":"2024-04-26T23:54:25.265783+0000","last_became_peered":"2024-04-26T23:54:25.265783+0000","last_unstale":"2024-04-26T23:54:25.270365+0000","last_undegraded":"2024-04-26T23:54:25.270365+0000","last_fullsized":"2024-04-26T23:54:25.270365+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:52.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.226642+0000","last_change":"2024-04-26T23:54:24.222447+0000","last_active":"2024-04-26T23:54:25.226642+0000","last_peered":"2024-04-26T23:54:25.226642+0000","last_clean":"2024-04-26T23:54:25.226642+0000","last_became_active":"2024-04-26T23:54:24.222017+0000","last_became_peered":"2024-04-26T23:54:24.222017+0000","last_unstale":"2024-04-26T23:54:25.226642+0000","last_undegraded":"2024-04-26T23:54:25.226642+0000","last_fullsized":"2024-04-26T23:54:25.226642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:18.944650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270393+0000","last_change":"2024-04-26T23:54:25.264675+0000","last_active":"2024-04-26T23:54:25.270393+0000","last_peered":"2024-04-26T23:54:25.270393+0000","last_clean":"2024-04-26T23:54:25.270393+0000","last_became_active":"2024-04-26T23:54:25.264326+0000","last_became_peered":"2024-04-26T23:54:25.264326+0000","last_unstale":"2024-04-26T23:54:25.270393+0000","last_undegraded":"2024-04-26T23:54:25.270393+0000","last_fullsized":"2024-04-26T23:54:25.270393+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271848+0000","last_change":"2024-04-26T23:54:24.216362+0000","last_active":"2024-04-26T23:54:25.271848+0000","last_peered":"2024-04-26T23:54:25.271848+0000","last_clean":"2024-04-26T23:54:25.271848+0000","last_became_active":"2024-04-26T23:54:24.214865+0000","last_became_peered":"2024-04-26T23:54:24.214865+0000","last_unstale":"2024-04-26T23:54:25.271848+0000","last_undegraded":"2024-04-26T23:54:25.271848+0000","last_fullsized":"2024-04-26T23:54:25.271848+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:10.264882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.245420+0000","last_change":"2024-04-26T23:54:25.241594+0000","last_active":"2024-04-26T23:54:25.245420+0000","last_peered":"2024-04-26T23:54:25.245420+0000","last_clean":"2024-04-26T23:54:25.245420+0000","last_became_active":"2024-04-26T23:54:25.240299+0000","last_became_peered":"2024-04-26T23:54:25.240299+0000","last_unstale":"2024-04-26T23:54:25.245420+0000","last_undegraded":"2024-04-26T23:54:25.245420+0000","last_fullsized":"2024-04-26T23:54:25.245420+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:07.586180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.216725+0000","last_change":"2024-04-26T23:54:24.202395+0000","last_active":"2024-04-26T23:54:25.216725+0000","last_peered":"2024-04-26T23:54:25.216725+0000","last_clean":"2024-04-26T23:54:25.216725+0000","last_became_active":"2024-04-26T23:54:24.201614+0000","last_became_peered":"2024-04-26T23:54:24.201614+0000","last_unstale":"2024-04-26T23:54:25.216725+0000","last_undegraded":"2024-04-26T23:54:25.216725+0000","last_fullsized":"2024-04-26T23:54:25.216725+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:07.201679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234601+0000","last_change":"2024-04-26T23:54:25.231159+0000","last_active":"2024-04-26T23:54:25.234601+0000","last_peered":"2024-04-26T23:54:25.234601+0000","last_clean":"2024-04-26T23:54:25.234601+0000","last_became_active":"2024-04-26T23:54:25.230766+0000","last_became_peered":"2024-04-26T23:54:25.230766+0000","last_unstale":"2024-04-26T23:54:25.234601+0000","last_undegraded":"2024-04-26T23:54:25.234601+0000","last_fullsized":"2024-04-26T23:54:25.234601+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:27.480961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234787+0000","last_change":"2024-04-26T23:54:24.207845+0000","last_active":"2024-04-26T23:54:25.234787+0000","last_peered":"2024-04-26T23:54:25.234787+0000","last_clean":"2024-04-26T23:54:25.234787+0000","last_became_active":"2024-04-26T23:54:24.206388+0000","last_became_peered":"2024-04-26T23:54:24.206388+0000","last_unstale":"2024-04-26T23:54:25.234787+0000","last_undegraded":"2024-04-26T23:54:25.234787+0000","last_fullsized":"2024-04-26T23:54:25.234787+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:04.668590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259826+0000","last_change":"2024-04-26T23:54:25.245003+0000","last_active":"2024-04-26T23:54:25.259826+0000","last_peered":"2024-04-26T23:54:25.259826+0000","last_clean":"2024-04-26T23:54:25.259826+0000","last_became_active":"2024-04-26T23:54:25.244534+0000","last_became_peered":"2024-04-26T23:54:25.244534+0000","last_unstale":"2024-04-26T23:54:25.259826+0000","last_undegraded":"2024-04-26T23:54:25.259826+0000","last_fullsized":"2024-04-26T23:54:25.259826+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:31.557233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.233526+0000","last_change":"2024-04-26T23:54:24.218850+0000","last_active":"2024-04-26T23:54:25.233526+0000","last_peered":"2024-04-26T23:54:25.233526+0000","last_clean":"2024-04-26T23:54:25.233526+0000","last_became_active":"2024-04-26T23:54:24.218480+0000","last_became_peered":"2024-04-26T23:54:24.218480+0000","last_unstale":"2024-04-26T23:54:25.233526+0000","last_undegraded":"2024-04-26T23:54:25.233526+0000","last_fullsized":"2024-04-26T23:54:25.233526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:17.430404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.260508+0000","last_change":"2024-04-26T23:54:25.253451+0000","last_active":"2024-04-26T23:54:25.260508+0000","last_peered":"2024-04-26T23:54:25.260508+0000","last_clean":"2024-04-26T23:54:25.260508+0000","last_became_active":"2024-04-26T23:54:25.253134+0000","last_became_peered":"2024-04-26T23:54:25.253134+0000","last_unstale":"2024-04-26T23:54:25.260508+0000","last_undegraded":"2024-04-26T23:54:25.260508+0000","last_fullsized":"2024-04-26T23:54:25.260508+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:20:54.660336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269504+0000","last_change":"2024-04-26T23:54:24.217054+0000","last_active":"2024-04-26T23:54:25.269504+0000","last_peered":"2024-04-26T23:54:25.269504+0000","last_clean":"2024-04-26T23:54:25.269504+0000","last_became_active":"2024-04-26T23:54:24.216788+0000","last_became_peered":"2024-04-26T23:54:24.216788+0000","last_unstale":"2024-04-26T23:54:25.269504+0000","last_undegraded":"2024-04-26T23:54:25.269504+0000","last_fullsized":"2024-04-26T23:54:25.269504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:57:43.382251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249865+0000","last_change":"2024-04-26T23:54:25.247374+0000","last_active":"2024-04-26T23:54:25.249865+0000","last_peered":"2024-04-26T23:54:25.249865+0000","last_clean":"2024-04-26T23:54:25.249865+0000","last_became_active":"2024-04-26T23:54:25.247093+0000","last_became_peered":"2024-04-26T23:54:25.247093+0000","last_unstale":"2024-04-26T23:54:25.249865+0000","last_undegraded":"2024-04-26T23:54:25.249865+0000","last_fullsized":"2024-04-26T23:54:25.249865+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:41.216864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234479+0000","last_change":"2024-04-26T23:54:24.210315+0000","last_active":"2024-04-26T23:54:25.234479+0000","last_peered":"2024-04-26T23:54:25.234479+0000","last_clean":"2024-04-26T23:54:25.234479+0000","last_became_active":"2024-04-26T23:54:24.210023+0000","last_became_peered":"2024-04-26T23:54:24.210023+0000","last_unstale":"2024-04-26T23:54:25.234479+0000","last_undegraded":"2024-04-26T23:54:25.234479+0000","last_fullsized":"2024-04-26T23:54:25.234479+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:17.417082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262224+0000","last_change":"2024-04-26T23:54:25.258604+0000","last_active":"2024-04-26T23:54:25.262224+0000","last_peered":"2024-04-26T23:54:25.262224+0000","last_clean":"2024-04-26T23:54:25.262224+0000","last_became_active":"2024-04-26T23:54:25.256686+0000","last_became_peered":"2024-04-26T23:54:25.256686+0000","last_unstale":"2024-04-26T23:54:25.262224+0000","last_undegraded":"2024-04-26T23:54:25.262224+0000","last_fullsized":"2024-04-26T23:54:25.262224+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.232172+0000","last_change":"2024-04-26T23:54:24.214684+0000","last_active":"2024-04-26T23:54:25.232172+0000","last_peered":"2024-04-26T23:54:25.232172+0000","last_clean":"2024-04-26T23:54:25.232172+0000","last_became_active":"2024-04-26T23:54:24.214191+0000","last_became_peered":"2024-04-26T23:54:24.214191+0000","last_unstale":"2024-04-26T23:54:25.232172+0000","last_undegraded":"2024-04-26T23:54:25.232172+0000","last_fullsized":"2024-04-26T23:54:25.232172+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:51.561421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.261142+0000","last_change":"2024-04-26T23:54:25.253833+0000","last_active":"2024-04-26T23:54:25.261142+0000","last_peered":"2024-04-26T23:54:25.261142+0000","last_clean":"2024-04-26T23:54:25.261142+0000","last_became_active":"2024-04-26T23:54:25.253512+0000","last_became_peered":"2024-04-26T23:54:25.253512+0000","last_unstale":"2024-04-26T23:54:25.261142+0000","last_undegraded":"2024-04-26T23:54:25.261142+0000","last_fullsized":"2024-04-26T23:54:25.261142+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:09.002415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.244337+0000","last_change":"2024-04-26T23:54:24.208019+0000","last_active":"2024-04-26T23:54:25.244337+0000","last_peered":"2024-04-26T23:54:25.244337+0000","last_clean":"2024-04-26T23:54:25.244337+0000","last_became_active":"2024-04-26T23:54:24.207743+0000","last_became_peered":"2024-04-26T23:54:24.207743+0000","last_unstale":"2024-04-26T23:54:25.244337+0000","last_undegraded":"2024-04-26T23:54:25.244337+0000","last_fullsized":"2024-04-26T23:54:25.244337+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:12.600132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254374+0000","last_change":"2024-04-26T23:54:25.242824+0000","last_active":"2024-04-26T23:54:25.254374+0000","last_peered":"2024-04-26T23:54:25.254374+0000","last_clean":"2024-04-26T23:54:25.254374+0000","last_became_active":"2024-04-26T23:54:25.240816+0000","last_became_peered":"2024-04-26T23:54:25.240816+0000","last_unstale":"2024-04-26T23:54:25.254374+0000","last_undegraded":"2024-04-26T23:54:25.254374+0000","last_fullsized":"2024-04-26T23:54:25.254374+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:02.623311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.242239+0000","last_change":"2024-04-26T23:54:24.222812+0000","last_active":"2024-04-26T23:54:25.242239+0000","last_peered":"2024-04-26T23:54:25.242239+0000","last_clean":"2024-04-26T23:54:25.242239+0000","last_became_active":"2024-04-26T23:54:24.222501+0000","last_became_peered":"2024-04-26T23:54:24.222501+0000","last_unstale":"2024-04-26T23:54:25.242239+0000","last_undegraded":"2024-04-26T23:54:25.242239+0000","last_fullsized":"2024-04-26T23:54:25.242239+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:35.530109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.256655+0000","last_change":"2024-04-26T23:54:25.253954+0000","last_active":"2024-04-26T23:54:25.256655+0000","last_peered":"2024-04-26T23:54:25.256655+0000","last_clean":"2024-04-26T23:54:25.256655+0000","last_became_active":"2024-04-26T23:54:25.253391+0000","last_became_peered":"2024-04-26T23:54:25.253391+0000","last_unstale":"2024-04-26T23:54:25.256655+0000","last_undegraded":"2024-04-26T23:54:25.256655+0000","last_fullsized":"2024-04-26T23:54:25.256655+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:41.653350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.245322+0000","last_change":"2024-04-26T23:54:24.220453+0000","last_active":"2024-04-26T23:54:25.245322+0000","last_peered":"2024-04-26T23:54:25.245322+0000","last_clean":"2024-04-26T23:54:25.245322+0000","last_became_active":"2024-04-26T23:54:24.220043+0000","last_became_peered":"2024-04-26T23:54:24.220043+0000","last_unstale":"2024-04-26T23:54:25.245322+0000","last_undegraded":"2024-04-26T23:54:25.245322+0000","last_fullsized":"2024-04-26T23:54:25.245322+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:18.944650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271220+0000","last_change":"2024-04-26T23:54:25.267824+0000","last_active":"2024-04-26T23:54:25.271220+0000","last_peered":"2024-04-26T23:54:25.271220+0000","last_clean":"2024-04-26T23:54:25.271220+0000","last_became_active":"2024-04-26T23:54:25.266453+0000","last_became_peered":"2024-04-26T23:54:25.266453+0000","last_unstale":"2024-04-26T23:54:25.271220+0000","last_undegraded":"2024-04-26T23:54:25.271220+0000","last_fullsized":"2024-04-26T23:54:25.271220+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:21.902328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234712+0000","last_change":"2024-04-26T23:54:24.227633+0000","last_active":"2024-04-26T23:54:25.234712+0000","last_peered":"2024-04-26T23:54:25.234712+0000","last_clean":"2024-04-26T23:54:25.234712+0000","last_became_active":"2024-04-26T23:54:24.209171+0000","last_became_peered":"2024-04-26T23:54:24.209171+0000","last_unstale":"2024-04-26T23:54:25.234712+0000","last_undegraded":"2024-04-26T23:54:25.234712+0000","last_fullsized":"2024-04-26T23:54:25.234712+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:45.659252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.253079+0000","last_change":"2024-04-26T23:54:25.251528+0000","last_active":"2024-04-26T23:54:25.253079+0000","last_peered":"2024-04-26T23:54:25.253079+0000","last_clean":"2024-04-26T23:54:25.253079+0000","last_became_active":"2024-04-26T23:54:25.251089+0000","last_became_peered":"2024-04-26T23:54:25.251089+0000","last_unstale":"2024-04-26T23:54:25.253079+0000","last_undegraded":"2024-04-26T23:54:25.253079+0000","last_fullsized":"2024-04-26T23:54:25.253079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.233143+0000","last_change":"2024-04-26T23:54:24.212073+0000","last_active":"2024-04-26T23:54:25.233143+0000","last_peered":"2024-04-26T23:54:25.233143+0000","last_clean":"2024-04-26T23:54:25.233143+0000","last_became_active":"2024-04-26T23:54:24.211739+0000","last_became_peered":"2024-04-26T23:54:24.211739+0000","last_unstale":"2024-04-26T23:54:25.233143+0000","last_undegraded":"2024-04-26T23:54:25.233143+0000","last_fullsized":"2024-04-26T23:54:25.233143+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:57.031740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254129+0000","last_change":"2024-04-26T23:54:25.249406+0000","last_active":"2024-04-26T23:54:25.254129+0000","last_peered":"2024-04-26T23:54:25.254129+0000","last_clean":"2024-04-26T23:54:25.254129+0000","last_became_active":"2024-04-26T23:54:25.249040+0000","last_became_peered":"2024-04-26T23:54:25.249040+0000","last_unstale":"2024-04-26T23:54:25.254129+0000","last_undegraded":"2024-04-26T23:54:25.254129+0000","last_fullsized":"2024-04-26T23:54:25.254129+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:31.324349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.215774+0000","last_change":"2024-04-26T23:54:24.211118+0000","last_active":"2024-04-26T23:54:25.215774+0000","last_peered":"2024-04-26T23:54:25.215774+0000","last_clean":"2024-04-26T23:54:25.215774+0000","last_became_active":"2024-04-26T23:54:24.210851+0000","last_became_peered":"2024-04-26T23:54:24.210851+0000","last_unstale":"2024-04-26T23:54:25.215774+0000","last_undegraded":"2024-04-26T23:54:25.215774+0000","last_fullsized":"2024-04-26T23:54:25.215774+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:20.887373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.240219+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:54:25.240219+0000","last_peered":"2024-04-26T23:54:25.240219+0000","last_clean":"2024-04-26T23:54:25.240219+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:54:25.240219+0000","last_undegraded":"2024-04-26T23:54:25.240219+0000","last_fullsized":"2024-04-26T23:54:25.240219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575059,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51924,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319916,"statfs":{"total":96636764160,"available":96583593984,"internally_reserved":0,"allocated":2760704,"data_stored":1612083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575060,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55508,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316332,"statfs":{"total":96636764160,"available":96579923968,"internally_reserved":0,"allocated":2170880,"data_stored":1021785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575060,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55520,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316320,"statfs":{"total":96636764160,"available":96579911680,"internally_reserved":0,"allocated":2183168,"data_stored":1023349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2183168,"data_stored":1023361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575060,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47200,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324640,"statfs":{"total":96636764160,"available":96588431360,"internally_reserved":0,"allocated":2174976,"data_stored":1021875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59672,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312168,"statfs":{"total":96636764160,"available":96575660032,"internally_reserved":0,"allocated":2170880,"data_stored":1022297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47188,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324652,"statfs":{"total":96636764160,"available":96588443648,"internally_reserved":0,"allocated":2166784,"data_stored":1021715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575060,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60268,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311572,"statfs":{"total":96636764160,"available":96575049728,"internally_reserved":0,"allocated":2777088,"data_stored":1612811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:54:29.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:54:30.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.010+0000 7f236ba00640 1 Processor -- start 2024-04-26T23:54:30.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.010+0000 7f236ba00640 1 -- start start 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.011+0000 7f236ba00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f236406ec20 0x7f23640734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.011+0000 7f236ba00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23640739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f236ba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23640b43f0 con 0x7f2364142280 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f236ba00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f23640b46d0 con 0x7f236406ec20 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f236ba00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23640b49b0 con 0x7f23640b3e80 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f2369000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2364142280 0x7f236406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57714/0 (socket says 172.21.15.46:57714) 2024-04-26T23:54:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f2369000640 1 -- 172.21.15.46:0/2189418587 learned_addr learned my addr 172.21.15.46:0/2189418587 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f2363e00640 1 --2- 172.21.15.46:0/2189418587 >> v2:172.21.15.142:3300/0 conn(0x7f236406ec20 0x7f23640734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f2369a00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23640739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.012+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199940008 0 0) 0x7f23640b43f0 con 0x7f2364142280 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2348003520 con 0x7f2364142280 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3751909625 0 0) 0x7f2348003520 con 0x7f2364142280 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 msgr2=0x7f23640739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23640739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 >> v2:172.21.15.142:3300/0 conn(0x7f236406ec20 msgr2=0x7f23640734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 --2- 172.21.15.46:0/2189418587 >> v2:172.21.15.142:3300/0 conn(0x7f236406ec20 0x7f23640734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640b4c90 con 0x7f2364142280 2024-04-26T23:54:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2354001a10 con 0x7f2364142280 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 >> v1:172.21.15.46:6789/0 conn(0x7f2364142280 legacy=0x7f236406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2348003f80 0x7f2348004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f23480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23640b46d0 con 0x7f2348004d40 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f23640b49b0 con 0x7f2348003f80 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f234800b9c0 con 0x7f23640b3e80 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.013+0000 7f2369a00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f23480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369000640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369a00640 1 -- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 msgr2=0x7f23480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369a00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b3e80 0x7f23480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369a00640 1 -- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2348003f80 msgr2=0x7f2348004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369a00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2348003f80 0x7f2348004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.014+0000 7f2369a00640 1 -- 172.21.15.46:0/2189418587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348003520 con 0x7f2348004d40 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.015+0000 7f2369a00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f23480051c0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f235c009af0 tx=0x7f2348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d4cc86602fbea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.015+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c036070 con 0x7f2348004d40 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.015+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c031400 con 0x7f2348004d40 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.015+0000 7f2363400640 1 -- 172.21.15.46:0/2189418587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c009690 con 0x7f2348004d40 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 -- 172.21.15.46:0/2189418587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 msgr2=0x7f23480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 --2- 172.21.15.46:0/2189418587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f23480051c0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f235c009af0 tx=0x7f2348005830 comp rx=0 tx=0).stop 2024-04-26T23:54:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f2369a00640 1 -- 172.21.15.46:0/2189418587 reap_dead start 2024-04-26T23:54:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 -- 172.21.15.46:0/2189418587 shutdown_connections 2024-04-26T23:54:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 -- 172.21.15.46:0/2189418587 >> 172.21.15.46:0/2189418587 conn(0x7f236406c530 msgr2=0x7f236406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 -- 172.21.15.46:0/2189418587 shutdown_connections 2024-04-26T23:54:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.016+0000 7f236ba00640 1 -- 172.21.15.46:0/2189418587 wait complete. 2024-04-26T23:54:30.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.018+0000 7f236ba00640 1 Processor -- start 2024-04-26T23:54:30.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.018+0000 7f236ba00640 1 -- start start 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 0x7f236415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 0x7f23640b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2364142a20 con 0x7f2348004d40 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f23641428a0 con 0x7f23640cf9b0 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f236ba00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23640739e0 con 0x7f23640b0af0 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f2369000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f2369000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49558/0 (socket says 172.21.15.46:49558) 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f2369000640 1 -- 172.21.15.46:0/1252226537 learned_addr learned my addr 172.21.15.46:0/1252226537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f2369a00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 0x7f23640b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.019+0000 7f2363e00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 0x7f236415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 -- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 msgr2=0x7f23640b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 0x7f23640b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 -- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 msgr2=0x7f236415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 0x7f236415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 -- 172.21.15.46:0/1252226537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640cd9e0 con 0x7f2348004d40 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369a00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 0x7f23640b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2363e00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 0x7f236415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2369000640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f2354007ab0 tx=0x7f235400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354013420 con 0x7f2348004d40 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2354018070 con 0x7f2348004d40 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354013800 con 0x7f2348004d40 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23640cdcc0 con 0x7f2348004d40 2024-04-26T23:54:30.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.020+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23640ce280 con 0x7f2348004d40 2024-04-26T23:54:30.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.022+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23640b14b0 con 0x7f2348004d40 2024-04-26T23:54:30.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.023+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23540139a0 con 0x7f2348004d40 2024-04-26T23:54:30.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.023+0000 7f2360c00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 0x7f234006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.024+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7f235408c3a0 con 0x7f2348004d40 2024-04-26T23:54:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.024+0000 7f2363e00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 0x7f234006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.025+0000 7f2363e00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 0x7f234006ede0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2364142e00 tx=0x7f234c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.028+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f235405a630 con 0x7f2348004d40 2024-04-26T23:54:30.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.228+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2364069530 con 0x7f234006c920 2024-04-26T23:54:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.248+0000 7f2360c00640 1 -- 172.21.15.46:0/1252226537 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f2364069530 con 0x7f234006c920 2024-04-26T23:54:30.255 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 msgr2=0x7f234006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 0x7f234006ede0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2364142e00 tx=0x7f234c008040 comp rx=0 tx=0).stop 2024-04-26T23:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 msgr2=0x7f2364138ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f2354007ab0 tx=0x7f235400e530 comp rx=0 tx=0).stop 2024-04-26T23:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 shutdown_connections 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f234006c920 0x7f234006ede0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23640b0af0 0x7f23640b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23640cf9b0 0x7f236415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 --2- 172.21.15.46:0/1252226537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2348004d40 0x7f2364138ae0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 >> 172.21.15.46:0/1252226537 conn(0x7f236406c530 msgr2=0x7f2364143530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 shutdown_connections 2024-04-26T23:54:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.255+0000 7f236ba00640 1 -- 172.21.15.46:0/1252226537 wait complete. 2024-04-26T23:54:30.278 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2024-04-26T23:54:28.280989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":436964,"kb_used_data":18152,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754537756,"statfs":{"total":773094113280,"available":772646662144,"internally_reserved":0,"allocated":18587648,"data_stored":9359276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":428723761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103010"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270341+0000","last_change":"2024-04-26T23:54:25.261837+0000","last_active":"2024-04-26T23:54:25.270341+0000","last_peered":"2024-04-26T23:54:25.270341+0000","last_clean":"2024-04-26T23:54:25.270341+0000","last_became_active":"2024-04-26T23:54:25.261537+0000","last_became_peered":"2024-04-26T23:54:25.261537+0000","last_unstale":"2024-04-26T23:54:25.270341+0000","last_undegraded":"2024-04-26T23:54:25.270341+0000","last_fullsized":"2024-04-26T23:54:25.270341+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.263863+0000","last_change":"2024-04-26T23:54:25.248982+0000","last_active":"2024-04-26T23:54:25.263863+0000","last_peered":"2024-04-26T23:54:25.263863+0000","last_clean":"2024-04-26T23:54:25.263863+0000","last_became_active":"2024-04-26T23:54:25.248618+0000","last_became_peered":"2024-04-26T23:54:25.248618+0000","last_unstale":"2024-04-26T23:54:25.263863+0000","last_undegraded":"2024-04-26T23:54:25.263863+0000","last_fullsized":"2024-04-26T23:54:25.263863+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:37.869900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254426+0000","last_change":"2024-04-26T23:54:25.243080+0000","last_active":"2024-04-26T23:54:25.254426+0000","last_peered":"2024-04-26T23:54:25.254426+0000","last_clean":"2024-04-26T23:54:25.254426+0000","last_became_active":"2024-04-26T23:54:25.242348+0000","last_became_peered":"2024-04-26T23:54:25.242348+0000","last_unstale":"2024-04-26T23:54:25.254426+0000","last_undegraded":"2024-04-26T23:54:25.254426+0000","last_fullsized":"2024-04-26T23:54:25.254426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:31.622675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.243986+0000","last_change":"2024-04-26T23:54:25.242541+0000","last_active":"2024-04-26T23:54:25.243986+0000","last_peered":"2024-04-26T23:54:25.243986+0000","last_clean":"2024-04-26T23:54:25.243986+0000","last_became_active":"2024-04-26T23:54:25.241247+0000","last_became_peered":"2024-04-26T23:54:25.241247+0000","last_unstale":"2024-04-26T23:54:25.243986+0000","last_undegraded":"2024-04-26T23:54:25.243986+0000","last_fullsized":"2024-04-26T23:54:25.243986+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:59.027093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259859+0000","last_change":"2024-04-26T23:54:25.252375+0000","last_active":"2024-04-26T23:54:25.259859+0000","last_peered":"2024-04-26T23:54:25.259859+0000","last_clean":"2024-04-26T23:54:25.259859+0000","last_became_active":"2024-04-26T23:54:25.251918+0000","last_became_peered":"2024-04-26T23:54:25.251918+0000","last_unstale":"2024-04-26T23:54:25.259859+0000","last_undegraded":"2024-04-26T23:54:25.259859+0000","last_fullsized":"2024-04-26T23:54:25.259859+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:17.635409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257722+0000","last_change":"2024-04-26T23:54:25.250136+0000","last_active":"2024-04-26T23:54:25.257722+0000","last_peered":"2024-04-26T23:54:25.257722+0000","last_clean":"2024-04-26T23:54:25.257722+0000","last_became_active":"2024-04-26T23:54:25.249852+0000","last_became_peered":"2024-04-26T23:54:25.249852+0000","last_unstale":"2024-04-26T23:54:25.257722+0000","last_undegraded":"2024-04-26T23:54:25.257722+0000","last_fullsized":"2024-04-26T23:54:25.257722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:26.443604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264174+0000","last_change":"2024-04-26T23:54:25.255878+0000","last_active":"2024-04-26T23:54:25.264174+0000","last_peered":"2024-04-26T23:54:25.264174+0000","last_clean":"2024-04-26T23:54:25.264174+0000","last_became_active":"2024-04-26T23:54:25.255567+0000","last_became_peered":"2024-04-26T23:54:25.255567+0000","last_unstale":"2024-04-26T23:54:25.264174+0000","last_undegraded":"2024-04-26T23:54:25.264174+0000","last_fullsized":"2024-04-26T23:54:25.264174+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:39.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.252654+0000","last_change":"2024-04-26T23:54:25.248208+0000","last_active":"2024-04-26T23:54:25.252654+0000","last_peered":"2024-04-26T23:54:25.252654+0000","last_clean":"2024-04-26T23:54:25.252654+0000","last_became_active":"2024-04-26T23:54:25.247940+0000","last_became_peered":"2024-04-26T23:54:25.247940+0000","last_unstale":"2024-04-26T23:54:25.252654+0000","last_undegraded":"2024-04-26T23:54:25.252654+0000","last_fullsized":"2024-04-26T23:54:25.252654+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267712+0000","last_change":"2024-04-26T23:54:25.254642+0000","last_active":"2024-04-26T23:54:25.267712+0000","last_peered":"2024-04-26T23:54:25.267712+0000","last_clean":"2024-04-26T23:54:25.267712+0000","last_became_active":"2024-04-26T23:54:25.254344+0000","last_became_peered":"2024-04-26T23:54:25.254344+0000","last_unstale":"2024-04-26T23:54:25.267712+0000","last_undegraded":"2024-04-26T23:54:25.267712+0000","last_fullsized":"2024-04-26T23:54:25.267712+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:19.959605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258402+0000","last_change":"2024-04-26T23:54:25.246010+0000","last_active":"2024-04-26T23:54:25.258402+0000","last_peered":"2024-04-26T23:54:25.258402+0000","last_clean":"2024-04-26T23:54:25.258402+0000","last_became_active":"2024-04-26T23:54:25.245698+0000","last_became_peered":"2024-04-26T23:54:25.245698+0000","last_unstale":"2024-04-26T23:54:25.258402+0000","last_undegraded":"2024-04-26T23:54:25.258402+0000","last_fullsized":"2024-04-26T23:54:25.258402+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:45.253157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250814+0000","last_change":"2024-04-26T23:54:25.246889+0000","last_active":"2024-04-26T23:54:25.250814+0000","last_peered":"2024-04-26T23:54:25.250814+0000","last_clean":"2024-04-26T23:54:25.250814+0000","last_became_active":"2024-04-26T23:54:25.246554+0000","last_became_peered":"2024-04-26T23:54:25.246554+0000","last_unstale":"2024-04-26T23:54:25.250814+0000","last_undegraded":"2024-04-26T23:54:25.250814+0000","last_fullsized":"2024-04-26T23:54:25.250814+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266496+0000","last_change":"2024-04-26T23:54:25.256315+0000","last_active":"2024-04-26T23:54:25.266496+0000","last_peered":"2024-04-26T23:54:25.266496+0000","last_clean":"2024-04-26T23:54:25.266496+0000","last_became_active":"2024-04-26T23:54:25.256034+0000","last_became_peered":"2024-04-26T23:54:25.256034+0000","last_unstale":"2024-04-26T23:54:25.266496+0000","last_undegraded":"2024-04-26T23:54:25.266496+0000","last_fullsized":"2024-04-26T23:54:25.266496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:07:03.896763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259638+0000","last_change":"2024-04-26T23:54:25.256879+0000","last_active":"2024-04-26T23:54:25.259638+0000","last_peered":"2024-04-26T23:54:25.259638+0000","last_clean":"2024-04-26T23:54:25.259638+0000","last_became_active":"2024-04-26T23:54:25.254919+0000","last_became_peered":"2024-04-26T23:54:25.254919+0000","last_unstale":"2024-04-26T23:54:25.259638+0000","last_undegraded":"2024-04-26T23:54:25.259638+0000","last_fullsized":"2024-04-26T23:54:25.259638+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:26.481809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264879+0000","last_change":"2024-04-26T23:54:25.260923+0000","last_active":"2024-04-26T23:54:25.264879+0000","last_peered":"2024-04-26T23:54:25.264879+0000","last_clean":"2024-04-26T23:54:25.264879+0000","last_became_active":"2024-04-26T23:54:25.258488+0000","last_became_peered":"2024-04-26T23:54:25.258488+0000","last_unstale":"2024-04-26T23:54:25.264879+0000","last_undegraded":"2024-04-26T23:54:25.264879+0000","last_fullsized":"2024-04-26T23:54:25.264879+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:18.445359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270606+0000","last_change":"2024-04-26T23:54:25.261047+0000","last_active":"2024-04-26T23:54:25.270606+0000","last_peered":"2024-04-26T23:54:25.270606+0000","last_clean":"2024-04-26T23:54:25.270606+0000","last_became_active":"2024-04-26T23:54:25.260772+0000","last_became_peered":"2024-04-26T23:54:25.260772+0000","last_unstale":"2024-04-26T23:54:25.270606+0000","last_undegraded":"2024-04-26T23:54:25.270606+0000","last_fullsized":"2024-04-26T23:54:25.270606+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.261812+0000","last_change":"2024-04-26T23:54:25.256985+0000","last_active":"2024-04-26T23:54:25.261812+0000","last_peered":"2024-04-26T23:54:25.261812+0000","last_clean":"2024-04-26T23:54:25.261812+0000","last_became_active":"2024-04-26T23:54:25.250514+0000","last_became_peered":"2024-04-26T23:54:25.250514+0000","last_unstale":"2024-04-26T23:54:25.261812+0000","last_undegraded":"2024-04-26T23:54:25.261812+0000","last_fullsized":"2024-04-26T23:54:25.261812+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:36.773616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262936+0000","last_change":"2024-04-26T23:54:25.257990+0000","last_active":"2024-04-26T23:54:25.262936+0000","last_peered":"2024-04-26T23:54:25.262936+0000","last_clean":"2024-04-26T23:54:25.262936+0000","last_became_active":"2024-04-26T23:54:25.257562+0000","last_became_peered":"2024-04-26T23:54:25.257562+0000","last_unstale":"2024-04-26T23:54:25.262936+0000","last_undegraded":"2024-04-26T23:54:25.262936+0000","last_fullsized":"2024-04-26T23:54:25.262936+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:46.960946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259385+0000","last_change":"2024-04-26T23:54:25.256550+0000","last_active":"2024-04-26T23:54:25.259385+0000","last_peered":"2024-04-26T23:54:25.259385+0000","last_clean":"2024-04-26T23:54:25.259385+0000","last_became_active":"2024-04-26T23:54:25.256237+0000","last_became_peered":"2024-04-26T23:54:25.256237+0000","last_unstale":"2024-04-26T23:54:25.259385+0000","last_undegraded":"2024-04-26T23:54:25.259385+0000","last_fullsized":"2024-04-26T23:54:25.259385+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.917158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249696+0000","last_change":"2024-04-26T23:54:25.241963+0000","last_active":"2024-04-26T23:54:25.249696+0000","last_peered":"2024-04-26T23:54:25.249696+0000","last_clean":"2024-04-26T23:54:25.249696+0000","last_became_active":"2024-04-26T23:54:25.241670+0000","last_became_peered":"2024-04-26T23:54:25.241670+0000","last_unstale":"2024-04-26T23:54:25.249696+0000","last_undegraded":"2024-04-26T23:54:25.249696+0000","last_fullsized":"2024-04-26T23:54:25.249696+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:05.537898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262941+0000","last_change":"2024-04-26T23:54:25.258358+0000","last_active":"2024-04-26T23:54:25.262941+0000","last_peered":"2024-04-26T23:54:25.262941+0000","last_clean":"2024-04-26T23:54:25.262941+0000","last_became_active":"2024-04-26T23:54:25.258067+0000","last_became_peered":"2024-04-26T23:54:25.258067+0000","last_unstale":"2024-04-26T23:54:25.262941+0000","last_undegraded":"2024-04-26T23:54:25.262941+0000","last_fullsized":"2024-04-26T23:54:25.262941+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:33.590565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269743+0000","last_change":"2024-04-26T23:54:25.258093+0000","last_active":"2024-04-26T23:54:25.269743+0000","last_peered":"2024-04-26T23:54:25.269743+0000","last_clean":"2024-04-26T23:54:25.269743+0000","last_became_active":"2024-04-26T23:54:25.256694+0000","last_became_peered":"2024-04-26T23:54:25.256694+0000","last_unstale":"2024-04-26T23:54:25.269743+0000","last_undegraded":"2024-04-26T23:54:25.269743+0000","last_fullsized":"2024-04-26T23:54:25.269743+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270225+0000","last_change":"2024-04-26T23:54:25.260683+0000","last_active":"2024-04-26T23:54:25.270225+0000","last_peered":"2024-04-26T23:54:25.270225+0000","last_clean":"2024-04-26T23:54:25.270225+0000","last_became_active":"2024-04-26T23:54:25.260289+0000","last_became_peered":"2024-04-26T23:54:25.260289+0000","last_unstale":"2024-04-26T23:54:25.270225+0000","last_undegraded":"2024-04-26T23:54:25.270225+0000","last_fullsized":"2024-04-26T23:54:25.270225+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254193+0000","last_change":"2024-04-26T23:54:25.250736+0000","last_active":"2024-04-26T23:54:25.254193+0000","last_peered":"2024-04-26T23:54:25.254193+0000","last_clean":"2024-04-26T23:54:25.254193+0000","last_became_active":"2024-04-26T23:54:25.248401+0000","last_became_peered":"2024-04-26T23:54:25.248401+0000","last_unstale":"2024-04-26T23:54:25.254193+0000","last_undegraded":"2024-04-26T23:54:25.254193+0000","last_fullsized":"2024-04-26T23:54:25.254193+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270220+0000","last_change":"2024-04-26T23:54:25.258526+0000","last_active":"2024-04-26T23:54:25.270220+0000","last_peered":"2024-04-26T23:54:25.270220+0000","last_clean":"2024-04-26T23:54:25.270220+0000","last_became_active":"2024-04-26T23:54:25.258213+0000","last_became_peered":"2024-04-26T23:54:25.258213+0000","last_unstale":"2024-04-26T23:54:25.270220+0000","last_undegraded":"2024-04-26T23:54:25.270220+0000","last_fullsized":"2024-04-26T23:54:25.270220+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264850+0000","last_change":"2024-04-26T23:54:25.261346+0000","last_active":"2024-04-26T23:54:25.264850+0000","last_peered":"2024-04-26T23:54:25.264850+0000","last_clean":"2024-04-26T23:54:25.264850+0000","last_became_active":"2024-04-26T23:54:25.259492+0000","last_became_peered":"2024-04-26T23:54:25.259492+0000","last_unstale":"2024-04-26T23:54:25.264850+0000","last_undegraded":"2024-04-26T23:54:25.264850+0000","last_fullsized":"2024-04-26T23:54:25.264850+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:40.426257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262161+0000","last_change":"2024-04-26T23:54:25.257230+0000","last_active":"2024-04-26T23:54:25.262161+0000","last_peered":"2024-04-26T23:54:25.262161+0000","last_clean":"2024-04-26T23:54:25.262161+0000","last_became_active":"2024-04-26T23:54:25.255556+0000","last_became_peered":"2024-04-26T23:54:25.255556+0000","last_unstale":"2024-04-26T23:54:25.262161+0000","last_undegraded":"2024-04-26T23:54:25.262161+0000","last_fullsized":"2024-04-26T23:54:25.262161+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254995+0000","last_change":"2024-04-26T23:54:25.251888+0000","last_active":"2024-04-26T23:54:25.254995+0000","last_peered":"2024-04-26T23:54:25.254995+0000","last_clean":"2024-04-26T23:54:25.254995+0000","last_became_active":"2024-04-26T23:54:25.251588+0000","last_became_peered":"2024-04-26T23:54:25.251588+0000","last_unstale":"2024-04-26T23:54:25.254995+0000","last_undegraded":"2024-04-26T23:54:25.254995+0000","last_fullsized":"2024-04-26T23:54:25.254995+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258505+0000","last_change":"2024-04-26T23:54:25.254802+0000","last_active":"2024-04-26T23:54:25.258505+0000","last_peered":"2024-04-26T23:54:25.258505+0000","last_clean":"2024-04-26T23:54:25.258505+0000","last_became_active":"2024-04-26T23:54:25.254510+0000","last_became_peered":"2024-04-26T23:54:25.254510+0000","last_unstale":"2024-04-26T23:54:25.258505+0000","last_undegraded":"2024-04-26T23:54:25.258505+0000","last_fullsized":"2024-04-26T23:54:25.258505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:09.335134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250730+0000","last_change":"2024-04-26T23:54:25.247869+0000","last_active":"2024-04-26T23:54:25.250730+0000","last_peered":"2024-04-26T23:54:25.250730+0000","last_clean":"2024-04-26T23:54:25.250730+0000","last_became_active":"2024-04-26T23:54:25.247575+0000","last_became_peered":"2024-04-26T23:54:25.247575+0000","last_unstale":"2024-04-26T23:54:25.250730+0000","last_undegraded":"2024-04-26T23:54:25.250730+0000","last_fullsized":"2024-04-26T23:54:25.250730+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:43.312183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249217+0000","last_change":"2024-04-26T23:54:25.244372+0000","last_active":"2024-04-26T23:54:25.249217+0000","last_peered":"2024-04-26T23:54:25.249217+0000","last_clean":"2024-04-26T23:54:25.249217+0000","last_became_active":"2024-04-26T23:54:25.244052+0000","last_became_peered":"2024-04-26T23:54:25.244052+0000","last_unstale":"2024-04-26T23:54:25.249217+0000","last_undegraded":"2024-04-26T23:54:25.249217+0000","last_fullsized":"2024-04-26T23:54:25.249217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:56.556207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250520+0000","last_change":"2024-04-26T23:54:25.248477+0000","last_active":"2024-04-26T23:54:25.250520+0000","last_peered":"2024-04-26T23:54:25.250520+0000","last_clean":"2024-04-26T23:54:25.250520+0000","last_became_active":"2024-04-26T23:54:25.247171+0000","last_became_peered":"2024-04-26T23:54:25.247171+0000","last_unstale":"2024-04-26T23:54:25.250520+0000","last_undegraded":"2024-04-26T23:54:25.250520+0000","last_fullsized":"2024-04-26T23:54:25.250520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:46.167514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257326+0000","last_change":"2024-04-26T23:54:25.247313+0000","last_active":"2024-04-26T23:54:25.257326+0000","last_peered":"2024-04-26T23:54:25.257326+0000","last_clean":"2024-04-26T23:54:25.257326+0000","last_became_active":"2024-04-26T23:54:25.246996+0000","last_became_peered":"2024-04-26T23:54:25.246996+0000","last_unstale":"2024-04-26T23:54:25.257326+0000","last_undegraded":"2024-04-26T23:54:25.257326+0000","last_fullsized":"2024-04-26T23:54:25.257326+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:37.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.268644+0000","last_change":"2024-04-26T23:54:25.267504+0000","last_active":"2024-04-26T23:54:25.268644+0000","last_peered":"2024-04-26T23:54:25.268644+0000","last_clean":"2024-04-26T23:54:25.268644+0000","last_became_active":"2024-04-26T23:54:25.267208+0000","last_became_peered":"2024-04-26T23:54:25.267208+0000","last_unstale":"2024-04-26T23:54:25.268644+0000","last_undegraded":"2024-04-26T23:54:25.268644+0000","last_fullsized":"2024-04-26T23:54:25.268644+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:52.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"172'4","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267354+0000","last_change":"2024-04-26T23:54:24.228872+0000","last_active":"2024-04-26T23:54:25.267354+0000","last_peered":"2024-04-26T23:54:25.267354+0000","last_clean":"2024-04-26T23:54:25.267354+0000","last_became_active":"2024-04-26T23:54:24.228459+0000","last_became_peered":"2024-04-26T23:54:24.228459+0000","last_unstale":"2024-04-26T23:54:25.267354+0000","last_undegraded":"2024-04-26T23:54:25.267354+0000","last_fullsized":"2024-04-26T23:54:25.267354+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:35.474770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264811+0000","last_change":"2024-04-26T23:54:25.257415+0000","last_active":"2024-04-26T23:54:25.264811+0000","last_peered":"2024-04-26T23:54:25.264811+0000","last_clean":"2024-04-26T23:54:25.264811+0000","last_became_active":"2024-04-26T23:54:25.257076+0000","last_became_peered":"2024-04-26T23:54:25.257076+0000","last_unstale":"2024-04-26T23:54:25.264811+0000","last_undegraded":"2024-04-26T23:54:25.264811+0000","last_fullsized":"2024-04-26T23:54:25.264811+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:58.046695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.272865+0000","last_change":"2024-04-26T23:54:24.214355+0000","last_active":"2024-04-26T23:54:25.272865+0000","last_peered":"2024-04-26T23:54:25.272865+0000","last_clean":"2024-04-26T23:54:25.272865+0000","last_became_active":"2024-04-26T23:54:24.214068+0000","last_became_peered":"2024-04-26T23:54:24.214068+0000","last_unstale":"2024-04-26T23:54:25.272865+0000","last_undegraded":"2024-04-26T23:54:25.272865+0000","last_fullsized":"2024-04-26T23:54:25.272865+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:55.333276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258295+0000","last_change":"2024-04-26T23:54:25.255571+0000","last_active":"2024-04-26T23:54:25.258295+0000","last_peered":"2024-04-26T23:54:25.258295+0000","last_clean":"2024-04-26T23:54:25.258295+0000","last_became_active":"2024-04-26T23:54:25.255263+0000","last_became_peered":"2024-04-26T23:54:25.255263+0000","last_unstale":"2024-04-26T23:54:25.258295+0000","last_undegraded":"2024-04-26T23:54:25.258295+0000","last_fullsized":"2024-04-26T23:54:25.258295+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269857+0000","last_change":"2024-04-26T23:54:24.213242+0000","last_active":"2024-04-26T23:54:25.269857+0000","last_peered":"2024-04-26T23:54:25.269857+0000","last_clean":"2024-04-26T23:54:25.269857+0000","last_became_active":"2024-04-26T23:54:24.212936+0000","last_became_peered":"2024-04-26T23:54:24.212936+0000","last_unstale":"2024-04-26T23:54:25.269857+0000","last_undegraded":"2024-04-26T23:54:25.269857+0000","last_fullsized":"2024-04-26T23:54:25.269857+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:36.854946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254127+0000","last_change":"2024-04-26T23:54:25.252136+0000","last_active":"2024-04-26T23:54:25.254127+0000","last_peered":"2024-04-26T23:54:25.254127+0000","last_clean":"2024-04-26T23:54:25.254127+0000","last_became_active":"2024-04-26T23:54:25.251799+0000","last_became_peered":"2024-04-26T23:54:25.251799+0000","last_unstale":"2024-04-26T23:54:25.254127+0000","last_undegraded":"2024-04-26T23:54:25.254127+0000","last_fullsized":"2024-04-26T23:54:25.254127+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:58.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269163+0000","last_change":"2024-04-26T23:54:24.216075+0000","last_active":"2024-04-26T23:54:25.269163+0000","last_peered":"2024-04-26T23:54:25.269163+0000","last_clean":"2024-04-26T23:54:25.269163+0000","last_became_active":"2024-04-26T23:54:24.215703+0000","last_became_peered":"2024-04-26T23:54:24.215703+0000","last_unstale":"2024-04-26T23:54:25.269163+0000","last_undegraded":"2024-04-26T23:54:25.269163+0000","last_fullsized":"2024-04-26T23:54:25.269163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:09.914977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.260828+0000","last_change":"2024-04-26T23:54:25.259527+0000","last_active":"2024-04-26T23:54:25.260828+0000","last_peered":"2024-04-26T23:54:25.260828+0000","last_clean":"2024-04-26T23:54:25.260828+0000","last_became_active":"2024-04-26T23:54:25.258807+0000","last_became_peered":"2024-04-26T23:54:25.258807+0000","last_unstale":"2024-04-26T23:54:25.260828+0000","last_undegraded":"2024-04-26T23:54:25.260828+0000","last_fullsized":"2024-04-26T23:54:25.260828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:58.869636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.272928+0000","last_change":"2024-04-26T23:54:24.218389+0000","last_active":"2024-04-26T23:54:25.272928+0000","last_peered":"2024-04-26T23:54:25.272928+0000","last_clean":"2024-04-26T23:54:25.272928+0000","last_became_active":"2024-04-26T23:54:24.218048+0000","last_became_peered":"2024-04-26T23:54:24.218048+0000","last_unstale":"2024-04-26T23:54:25.272928+0000","last_undegraded":"2024-04-26T23:54:25.272928+0000","last_fullsized":"2024-04-26T23:54:25.272928+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:47.057131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.252731+0000","last_change":"2024-04-26T23:54:25.246361+0000","last_active":"2024-04-26T23:54:25.252731+0000","last_peered":"2024-04-26T23:54:25.252731+0000","last_clean":"2024-04-26T23:54:25.252731+0000","last_became_active":"2024-04-26T23:54:25.246067+0000","last_became_peered":"2024-04-26T23:54:25.246067+0000","last_unstale":"2024-04-26T23:54:25.252731+0000","last_undegraded":"2024-04-26T23:54:25.252731+0000","last_fullsized":"2024-04-26T23:54:25.252731+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:36.528174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270461+0000","last_change":"2024-04-26T23:54:24.209707+0000","last_active":"2024-04-26T23:54:25.270461+0000","last_peered":"2024-04-26T23:54:25.270461+0000","last_clean":"2024-04-26T23:54:25.270461+0000","last_became_active":"2024-04-26T23:54:24.208514+0000","last_became_peered":"2024-04-26T23:54:24.208514+0000","last_unstale":"2024-04-26T23:54:25.270461+0000","last_undegraded":"2024-04-26T23:54:25.270461+0000","last_fullsized":"2024-04-26T23:54:25.270461+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.803222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271243+0000","last_change":"2024-04-26T23:54:25.268200+0000","last_active":"2024-04-26T23:54:25.271243+0000","last_peered":"2024-04-26T23:54:25.271243+0000","last_clean":"2024-04-26T23:54:25.271243+0000","last_became_active":"2024-04-26T23:54:25.265951+0000","last_became_peered":"2024-04-26T23:54:25.265951+0000","last_unstale":"2024-04-26T23:54:25.271243+0000","last_undegraded":"2024-04-26T23:54:25.271243+0000","last_fullsized":"2024-04-26T23:54:25.271243+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:05.908781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.230058+0000","last_change":"2024-04-26T23:54:24.217316+0000","last_active":"2024-04-26T23:54:25.230058+0000","last_peered":"2024-04-26T23:54:25.230058+0000","last_clean":"2024-04-26T23:54:25.230058+0000","last_became_active":"2024-04-26T23:54:24.217024+0000","last_became_peered":"2024-04-26T23:54:24.217024+0000","last_unstale":"2024-04-26T23:54:25.230058+0000","last_undegraded":"2024-04-26T23:54:25.230058+0000","last_fullsized":"2024-04-26T23:54:25.230058+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:41.845328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264508+0000","last_change":"2024-04-26T23:54:25.262805+0000","last_active":"2024-04-26T23:54:25.264508+0000","last_peered":"2024-04-26T23:54:25.264508+0000","last_clean":"2024-04-26T23:54:25.264508+0000","last_became_active":"2024-04-26T23:54:25.262415+0000","last_became_peered":"2024-04-26T23:54:25.262415+0000","last_unstale":"2024-04-26T23:54:25.264508+0000","last_undegraded":"2024-04-26T23:54:25.264508+0000","last_fullsized":"2024-04-26T23:54:25.264508+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.235187+0000","last_change":"2024-04-26T23:54:24.213361+0000","last_active":"2024-04-26T23:54:25.235187+0000","last_peered":"2024-04-26T23:54:25.235187+0000","last_clean":"2024-04-26T23:54:25.235187+0000","last_became_active":"2024-04-26T23:54:24.212542+0000","last_became_peered":"2024-04-26T23:54:24.212542+0000","last_unstale":"2024-04-26T23:54:25.235187+0000","last_undegraded":"2024-04-26T23:54:25.235187+0000","last_fullsized":"2024-04-26T23:54:25.235187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:49:12.024881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257611+0000","last_change":"2024-04-26T23:54:25.246219+0000","last_active":"2024-04-26T23:54:25.257611+0000","last_peered":"2024-04-26T23:54:25.257611+0000","last_clean":"2024-04-26T23:54:25.257611+0000","last_became_active":"2024-04-26T23:54:25.245293+0000","last_became_peered":"2024-04-26T23:54:25.245293+0000","last_unstale":"2024-04-26T23:54:25.257611+0000","last_undegraded":"2024-04-26T23:54:25.257611+0000","last_fullsized":"2024-04-26T23:54:25.257611+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:25.394034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.231369+0000","last_change":"2024-04-26T23:54:24.211435+0000","last_active":"2024-04-26T23:54:25.231369+0000","last_peered":"2024-04-26T23:54:25.231369+0000","last_clean":"2024-04-26T23:54:25.231369+0000","last_became_active":"2024-04-26T23:54:24.211135+0000","last_became_peered":"2024-04-26T23:54:24.211135+0000","last_unstale":"2024-04-26T23:54:25.231369+0000","last_undegraded":"2024-04-26T23:54:25.231369+0000","last_fullsized":"2024-04-26T23:54:25.231369+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:44.197712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.251436+0000","last_change":"2024-04-26T23:54:25.245299+0000","last_active":"2024-04-26T23:54:25.251436+0000","last_peered":"2024-04-26T23:54:25.251436+0000","last_clean":"2024-04-26T23:54:25.251436+0000","last_became_active":"2024-04-26T23:54:25.241634+0000","last_became_peered":"2024-04-26T23:54:25.241634+0000","last_unstale":"2024-04-26T23:54:25.251436+0000","last_undegraded":"2024-04-26T23:54:25.251436+0000","last_fullsized":"2024-04-26T23:54:25.251436+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:33.430183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"172'3","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271116+0000","last_change":"2024-04-26T23:54:24.215184+0000","last_active":"2024-04-26T23:54:25.271116+0000","last_peered":"2024-04-26T23:54:25.271116+0000","last_clean":"2024-04-26T23:54:25.271116+0000","last_became_active":"2024-04-26T23:54:24.213522+0000","last_became_peered":"2024-04-26T23:54:24.213522+0000","last_unstale":"2024-04-26T23:54:25.271116+0000","last_undegraded":"2024-04-26T23:54:25.271116+0000","last_fullsized":"2024-04-26T23:54:25.271116+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:57.498903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266001+0000","last_change":"2024-04-26T23:54:25.249362+0000","last_active":"2024-04-26T23:54:25.266001+0000","last_peered":"2024-04-26T23:54:25.266001+0000","last_clean":"2024-04-26T23:54:25.266001+0000","last_became_active":"2024-04-26T23:54:25.249011+0000","last_became_peered":"2024-04-26T23:54:25.249011+0000","last_unstale":"2024-04-26T23:54:25.266001+0000","last_undegraded":"2024-04-26T23:54:25.266001+0000","last_fullsized":"2024-04-26T23:54:25.266001+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:36.545064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"172'6","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.274751+0000","last_change":"2024-04-26T23:54:24.209823+0000","last_active":"2024-04-26T23:54:25.274751+0000","last_peered":"2024-04-26T23:54:25.274751+0000","last_clean":"2024-04-26T23:54:25.274751+0000","last_became_active":"2024-04-26T23:54:24.209485+0000","last_became_peered":"2024-04-26T23:54:24.209485+0000","last_unstale":"2024-04-26T23:54:25.274751+0000","last_undegraded":"2024-04-26T23:54:25.274751+0000","last_fullsized":"2024-04-26T23:54:25.274751+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:10.906275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.264129+0000","last_change":"2024-04-26T23:54:25.254853+0000","last_active":"2024-04-26T23:54:25.264129+0000","last_peered":"2024-04-26T23:54:25.264129+0000","last_clean":"2024-04-26T23:54:25.264129+0000","last_became_active":"2024-04-26T23:54:25.254573+0000","last_became_peered":"2024-04-26T23:54:25.254573+0000","last_unstale":"2024-04-26T23:54:25.264129+0000","last_undegraded":"2024-04-26T23:54:25.264129+0000","last_fullsized":"2024-04-26T23:54:25.264129+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:27.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271797+0000","last_change":"2024-04-26T23:54:24.201944+0000","last_active":"2024-04-26T23:54:25.271797+0000","last_peered":"2024-04-26T23:54:25.271797+0000","last_clean":"2024-04-26T23:54:25.271797+0000","last_became_active":"2024-04-26T23:54:24.201549+0000","last_became_peered":"2024-04-26T23:54:24.201549+0000","last_unstale":"2024-04-26T23:54:25.271797+0000","last_undegraded":"2024-04-26T23:54:25.271797+0000","last_fullsized":"2024-04-26T23:54:25.271797+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:51.512188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.258350+0000","last_change":"2024-04-26T23:54:25.253361+0000","last_active":"2024-04-26T23:54:25.258350+0000","last_peered":"2024-04-26T23:54:25.258350+0000","last_clean":"2024-04-26T23:54:25.258350+0000","last_became_active":"2024-04-26T23:54:25.250441+0000","last_became_peered":"2024-04-26T23:54:25.250441+0000","last_unstale":"2024-04-26T23:54:25.258350+0000","last_undegraded":"2024-04-26T23:54:25.258350+0000","last_fullsized":"2024-04-26T23:54:25.258350+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.489417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.237106+0000","last_change":"2024-04-26T23:54:24.198686+0000","last_active":"2024-04-26T23:54:25.237106+0000","last_peered":"2024-04-26T23:54:25.237106+0000","last_clean":"2024-04-26T23:54:25.237106+0000","last_became_active":"2024-04-26T23:54:24.197578+0000","last_became_peered":"2024-04-26T23:54:24.197578+0000","last_unstale":"2024-04-26T23:54:25.237106+0000","last_undegraded":"2024-04-26T23:54:25.237106+0000","last_fullsized":"2024-04-26T23:54:25.237106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:58.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.256693+0000","last_change":"2024-04-26T23:54:25.254151+0000","last_active":"2024-04-26T23:54:25.256693+0000","last_peered":"2024-04-26T23:54:25.256693+0000","last_clean":"2024-04-26T23:54:25.256693+0000","last_became_active":"2024-04-26T23:54:25.252802+0000","last_became_peered":"2024-04-26T23:54:25.252802+0000","last_unstale":"2024-04-26T23:54:25.256693+0000","last_undegraded":"2024-04-26T23:54:25.256693+0000","last_fullsized":"2024-04-26T23:54:25.256693+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:55.423762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.215805+0000","last_change":"2024-04-26T23:54:24.212434+0000","last_active":"2024-04-26T23:54:25.215805+0000","last_peered":"2024-04-26T23:54:25.215805+0000","last_clean":"2024-04-26T23:54:25.215805+0000","last_became_active":"2024-04-26T23:54:24.210392+0000","last_became_peered":"2024-04-26T23:54:24.210392+0000","last_unstale":"2024-04-26T23:54:25.215805+0000","last_undegraded":"2024-04-26T23:54:25.215805+0000","last_fullsized":"2024-04-26T23:54:25.215805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:04.893826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257053+0000","last_change":"2024-04-26T23:54:25.252364+0000","last_active":"2024-04-26T23:54:25.257053+0000","last_peered":"2024-04-26T23:54:25.257053+0000","last_clean":"2024-04-26T23:54:25.257053+0000","last_became_active":"2024-04-26T23:54:25.250095+0000","last_became_peered":"2024-04-26T23:54:25.250095+0000","last_unstale":"2024-04-26T23:54:25.257053+0000","last_undegraded":"2024-04-26T23:54:25.257053+0000","last_fullsized":"2024-04-26T23:54:25.257053+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:53.430799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"172'8","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.273183+0000","last_change":"2024-04-26T23:54:24.215012+0000","last_active":"2024-04-26T23:54:25.273183+0000","last_peered":"2024-04-26T23:54:25.273183+0000","last_clean":"2024-04-26T23:54:25.273183+0000","last_became_active":"2024-04-26T23:54:24.214744+0000","last_became_peered":"2024-04-26T23:54:24.214744+0000","last_unstale":"2024-04-26T23:54:25.273183+0000","last_undegraded":"2024-04-26T23:54:25.273183+0000","last_fullsized":"2024-04-26T23:54:25.273183+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:35.530109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.250467+0000","last_change":"2024-04-26T23:54:25.246075+0000","last_active":"2024-04-26T23:54:25.250467+0000","last_peered":"2024-04-26T23:54:25.250467+0000","last_clean":"2024-04-26T23:54:25.250467+0000","last_became_active":"2024-04-26T23:54:25.245799+0000","last_became_peered":"2024-04-26T23:54:25.245799+0000","last_unstale":"2024-04-26T23:54:25.250467+0000","last_undegraded":"2024-04-26T23:54:25.250467+0000","last_fullsized":"2024-04-26T23:54:25.250467+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.289408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271838+0000","last_change":"2024-04-26T23:54:24.214503+0000","last_active":"2024-04-26T23:54:25.271838+0000","last_peered":"2024-04-26T23:54:25.271838+0000","last_clean":"2024-04-26T23:54:25.271838+0000","last_became_active":"2024-04-26T23:54:24.212865+0000","last_became_peered":"2024-04-26T23:54:24.212865+0000","last_unstale":"2024-04-26T23:54:25.271838+0000","last_undegraded":"2024-04-26T23:54:25.271838+0000","last_fullsized":"2024-04-26T23:54:25.271838+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:40.201910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.257631+0000","last_change":"2024-04-26T23:54:25.246481+0000","last_active":"2024-04-26T23:54:25.257631+0000","last_peered":"2024-04-26T23:54:25.257631+0000","last_clean":"2024-04-26T23:54:25.257631+0000","last_became_active":"2024-04-26T23:54:25.245912+0000","last_became_peered":"2024-04-26T23:54:25.245912+0000","last_unstale":"2024-04-26T23:54:25.257631+0000","last_undegraded":"2024-04-26T23:54:25.257631+0000","last_fullsized":"2024-04-26T23:54:25.257631+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:30.657975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.266597+0000","last_change":"2024-04-26T23:54:24.213143+0000","last_active":"2024-04-26T23:54:25.266597+0000","last_peered":"2024-04-26T23:54:25.266597+0000","last_clean":"2024-04-26T23:54:25.266597+0000","last_became_active":"2024-04-26T23:54:24.212848+0000","last_became_peered":"2024-04-26T23:54:24.212848+0000","last_unstale":"2024-04-26T23:54:25.266597+0000","last_undegraded":"2024-04-26T23:54:25.266597+0000","last_fullsized":"2024-04-26T23:54:25.266597+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:20.630576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249171+0000","last_change":"2024-04-26T23:54:25.244991+0000","last_active":"2024-04-26T23:54:25.249171+0000","last_peered":"2024-04-26T23:54:25.249171+0000","last_clean":"2024-04-26T23:54:25.249171+0000","last_became_active":"2024-04-26T23:54:25.244441+0000","last_became_peered":"2024-04-26T23:54:25.244441+0000","last_unstale":"2024-04-26T23:54:25.249171+0000","last_undegraded":"2024-04-26T23:54:25.249171+0000","last_fullsized":"2024-04-26T23:54:25.249171+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:04.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.267611+0000","last_change":"2024-04-26T23:54:24.215999+0000","last_active":"2024-04-26T23:54:25.267611+0000","last_peered":"2024-04-26T23:54:25.267611+0000","last_clean":"2024-04-26T23:54:25.267611+0000","last_became_active":"2024-04-26T23:54:24.215718+0000","last_became_peered":"2024-04-26T23:54:24.215718+0000","last_unstale":"2024-04-26T23:54:25.267611+0000","last_undegraded":"2024-04-26T23:54:25.267611+0000","last_fullsized":"2024-04-26T23:54:25.267611+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:39.411303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270365+0000","last_change":"2024-04-26T23:54:25.266154+0000","last_active":"2024-04-26T23:54:25.270365+0000","last_peered":"2024-04-26T23:54:25.270365+0000","last_clean":"2024-04-26T23:54:25.270365+0000","last_became_active":"2024-04-26T23:54:25.265783+0000","last_became_peered":"2024-04-26T23:54:25.265783+0000","last_unstale":"2024-04-26T23:54:25.270365+0000","last_undegraded":"2024-04-26T23:54:25.270365+0000","last_fullsized":"2024-04-26T23:54:25.270365+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:52.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.226642+0000","last_change":"2024-04-26T23:54:24.222447+0000","last_active":"2024-04-26T23:54:25.226642+0000","last_peered":"2024-04-26T23:54:25.226642+0000","last_clean":"2024-04-26T23:54:25.226642+0000","last_became_active":"2024-04-26T23:54:24.222017+0000","last_became_peered":"2024-04-26T23:54:24.222017+0000","last_unstale":"2024-04-26T23:54:25.226642+0000","last_undegraded":"2024-04-26T23:54:25.226642+0000","last_fullsized":"2024-04-26T23:54:25.226642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:18.944650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.270393+0000","last_change":"2024-04-26T23:54:25.264675+0000","last_active":"2024-04-26T23:54:25.270393+0000","last_peered":"2024-04-26T23:54:25.270393+0000","last_clean":"2024-04-26T23:54:25.270393+0000","last_became_active":"2024-04-26T23:54:25.264326+0000","last_became_peered":"2024-04-26T23:54:25.264326+0000","last_unstale":"2024-04-26T23:54:25.270393+0000","last_undegraded":"2024-04-26T23:54:25.270393+0000","last_fullsized":"2024-04-26T23:54:25.270393+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:48.429407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271848+0000","last_change":"2024-04-26T23:54:24.216362+0000","last_active":"2024-04-26T23:54:25.271848+0000","last_peered":"2024-04-26T23:54:25.271848+0000","last_clean":"2024-04-26T23:54:25.271848+0000","last_became_active":"2024-04-26T23:54:24.214865+0000","last_became_peered":"2024-04-26T23:54:24.214865+0000","last_unstale":"2024-04-26T23:54:25.271848+0000","last_undegraded":"2024-04-26T23:54:25.271848+0000","last_fullsized":"2024-04-26T23:54:25.271848+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:10.264882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.245420+0000","last_change":"2024-04-26T23:54:25.241594+0000","last_active":"2024-04-26T23:54:25.245420+0000","last_peered":"2024-04-26T23:54:25.245420+0000","last_clean":"2024-04-26T23:54:25.245420+0000","last_became_active":"2024-04-26T23:54:25.240299+0000","last_became_peered":"2024-04-26T23:54:25.240299+0000","last_unstale":"2024-04-26T23:54:25.245420+0000","last_undegraded":"2024-04-26T23:54:25.245420+0000","last_fullsized":"2024-04-26T23:54:25.245420+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:02:07.586180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.216725+0000","last_change":"2024-04-26T23:54:24.202395+0000","last_active":"2024-04-26T23:54:25.216725+0000","last_peered":"2024-04-26T23:54:25.216725+0000","last_clean":"2024-04-26T23:54:25.216725+0000","last_became_active":"2024-04-26T23:54:24.201614+0000","last_became_peered":"2024-04-26T23:54:24.201614+0000","last_unstale":"2024-04-26T23:54:25.216725+0000","last_undegraded":"2024-04-26T23:54:25.216725+0000","last_fullsized":"2024-04-26T23:54:25.216725+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:07.201679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234601+0000","last_change":"2024-04-26T23:54:25.231159+0000","last_active":"2024-04-26T23:54:25.234601+0000","last_peered":"2024-04-26T23:54:25.234601+0000","last_clean":"2024-04-26T23:54:25.234601+0000","last_became_active":"2024-04-26T23:54:25.230766+0000","last_became_peered":"2024-04-26T23:54:25.230766+0000","last_unstale":"2024-04-26T23:54:25.234601+0000","last_undegraded":"2024-04-26T23:54:25.234601+0000","last_fullsized":"2024-04-26T23:54:25.234601+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:27.480961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234787+0000","last_change":"2024-04-26T23:54:24.207845+0000","last_active":"2024-04-26T23:54:25.234787+0000","last_peered":"2024-04-26T23:54:25.234787+0000","last_clean":"2024-04-26T23:54:25.234787+0000","last_became_active":"2024-04-26T23:54:24.206388+0000","last_became_peered":"2024-04-26T23:54:24.206388+0000","last_unstale":"2024-04-26T23:54:25.234787+0000","last_undegraded":"2024-04-26T23:54:25.234787+0000","last_fullsized":"2024-04-26T23:54:25.234787+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:04.668590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.259826+0000","last_change":"2024-04-26T23:54:25.245003+0000","last_active":"2024-04-26T23:54:25.259826+0000","last_peered":"2024-04-26T23:54:25.259826+0000","last_clean":"2024-04-26T23:54:25.259826+0000","last_became_active":"2024-04-26T23:54:25.244534+0000","last_became_peered":"2024-04-26T23:54:25.244534+0000","last_unstale":"2024-04-26T23:54:25.259826+0000","last_undegraded":"2024-04-26T23:54:25.259826+0000","last_fullsized":"2024-04-26T23:54:25.259826+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:31.557233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.233526+0000","last_change":"2024-04-26T23:54:24.218850+0000","last_active":"2024-04-26T23:54:25.233526+0000","last_peered":"2024-04-26T23:54:25.233526+0000","last_clean":"2024-04-26T23:54:25.233526+0000","last_became_active":"2024-04-26T23:54:24.218480+0000","last_became_peered":"2024-04-26T23:54:24.218480+0000","last_unstale":"2024-04-26T23:54:25.233526+0000","last_undegraded":"2024-04-26T23:54:25.233526+0000","last_fullsized":"2024-04-26T23:54:25.233526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:17.430404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.260508+0000","last_change":"2024-04-26T23:54:25.253451+0000","last_active":"2024-04-26T23:54:25.260508+0000","last_peered":"2024-04-26T23:54:25.260508+0000","last_clean":"2024-04-26T23:54:25.260508+0000","last_became_active":"2024-04-26T23:54:25.253134+0000","last_became_peered":"2024-04-26T23:54:25.253134+0000","last_unstale":"2024-04-26T23:54:25.260508+0000","last_undegraded":"2024-04-26T23:54:25.260508+0000","last_fullsized":"2024-04-26T23:54:25.260508+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:20:54.660336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"172'2","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.269504+0000","last_change":"2024-04-26T23:54:24.217054+0000","last_active":"2024-04-26T23:54:25.269504+0000","last_peered":"2024-04-26T23:54:25.269504+0000","last_clean":"2024-04-26T23:54:25.269504+0000","last_became_active":"2024-04-26T23:54:24.216788+0000","last_became_peered":"2024-04-26T23:54:24.216788+0000","last_unstale":"2024-04-26T23:54:25.269504+0000","last_undegraded":"2024-04-26T23:54:25.269504+0000","last_fullsized":"2024-04-26T23:54:25.269504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:57:43.382251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.249865+0000","last_change":"2024-04-26T23:54:25.247374+0000","last_active":"2024-04-26T23:54:25.249865+0000","last_peered":"2024-04-26T23:54:25.249865+0000","last_clean":"2024-04-26T23:54:25.249865+0000","last_became_active":"2024-04-26T23:54:25.247093+0000","last_became_peered":"2024-04-26T23:54:25.247093+0000","last_unstale":"2024-04-26T23:54:25.249865+0000","last_undegraded":"2024-04-26T23:54:25.249865+0000","last_fullsized":"2024-04-26T23:54:25.249865+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:41.216864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234479+0000","last_change":"2024-04-26T23:54:24.210315+0000","last_active":"2024-04-26T23:54:25.234479+0000","last_peered":"2024-04-26T23:54:25.234479+0000","last_clean":"2024-04-26T23:54:25.234479+0000","last_became_active":"2024-04-26T23:54:24.210023+0000","last_became_peered":"2024-04-26T23:54:24.210023+0000","last_unstale":"2024-04-26T23:54:25.234479+0000","last_undegraded":"2024-04-26T23:54:25.234479+0000","last_fullsized":"2024-04-26T23:54:25.234479+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:17.417082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.262224+0000","last_change":"2024-04-26T23:54:25.258604+0000","last_active":"2024-04-26T23:54:25.262224+0000","last_peered":"2024-04-26T23:54:25.262224+0000","last_clean":"2024-04-26T23:54:25.262224+0000","last_became_active":"2024-04-26T23:54:25.256686+0000","last_became_peered":"2024-04-26T23:54:25.256686+0000","last_unstale":"2024-04-26T23:54:25.262224+0000","last_undegraded":"2024-04-26T23:54:25.262224+0000","last_fullsized":"2024-04-26T23:54:25.262224+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:52.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.232172+0000","last_change":"2024-04-26T23:54:24.214684+0000","last_active":"2024-04-26T23:54:25.232172+0000","last_peered":"2024-04-26T23:54:25.232172+0000","last_clean":"2024-04-26T23:54:25.232172+0000","last_became_active":"2024-04-26T23:54:24.214191+0000","last_became_peered":"2024-04-26T23:54:24.214191+0000","last_unstale":"2024-04-26T23:54:25.232172+0000","last_undegraded":"2024-04-26T23:54:25.232172+0000","last_fullsized":"2024-04-26T23:54:25.232172+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:51.561421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.261142+0000","last_change":"2024-04-26T23:54:25.253833+0000","last_active":"2024-04-26T23:54:25.261142+0000","last_peered":"2024-04-26T23:54:25.261142+0000","last_clean":"2024-04-26T23:54:25.261142+0000","last_became_active":"2024-04-26T23:54:25.253512+0000","last_became_peered":"2024-04-26T23:54:25.253512+0000","last_unstale":"2024-04-26T23:54:25.261142+0000","last_undegraded":"2024-04-26T23:54:25.261142+0000","last_fullsized":"2024-04-26T23:54:25.261142+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:09.002415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.244337+0000","last_change":"2024-04-26T23:54:24.208019+0000","last_active":"2024-04-26T23:54:25.244337+0000","last_peered":"2024-04-26T23:54:25.244337+0000","last_clean":"2024-04-26T23:54:25.244337+0000","last_became_active":"2024-04-26T23:54:24.207743+0000","last_became_peered":"2024-04-26T23:54:24.207743+0000","last_unstale":"2024-04-26T23:54:25.244337+0000","last_undegraded":"2024-04-26T23:54:25.244337+0000","last_fullsized":"2024-04-26T23:54:25.244337+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:12.600132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254374+0000","last_change":"2024-04-26T23:54:25.242824+0000","last_active":"2024-04-26T23:54:25.254374+0000","last_peered":"2024-04-26T23:54:25.254374+0000","last_clean":"2024-04-26T23:54:25.254374+0000","last_became_active":"2024-04-26T23:54:25.240816+0000","last_became_peered":"2024-04-26T23:54:25.240816+0000","last_unstale":"2024-04-26T23:54:25.254374+0000","last_undegraded":"2024-04-26T23:54:25.254374+0000","last_fullsized":"2024-04-26T23:54:25.254374+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:02.623311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.242239+0000","last_change":"2024-04-26T23:54:24.222812+0000","last_active":"2024-04-26T23:54:25.242239+0000","last_peered":"2024-04-26T23:54:25.242239+0000","last_clean":"2024-04-26T23:54:25.242239+0000","last_became_active":"2024-04-26T23:54:24.222501+0000","last_became_peered":"2024-04-26T23:54:24.222501+0000","last_unstale":"2024-04-26T23:54:25.242239+0000","last_undegraded":"2024-04-26T23:54:25.242239+0000","last_fullsized":"2024-04-26T23:54:25.242239+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:35.530109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.256655+0000","last_change":"2024-04-26T23:54:25.253954+0000","last_active":"2024-04-26T23:54:25.256655+0000","last_peered":"2024-04-26T23:54:25.256655+0000","last_clean":"2024-04-26T23:54:25.256655+0000","last_became_active":"2024-04-26T23:54:25.253391+0000","last_became_peered":"2024-04-26T23:54:25.253391+0000","last_unstale":"2024-04-26T23:54:25.256655+0000","last_undegraded":"2024-04-26T23:54:25.256655+0000","last_fullsized":"2024-04-26T23:54:25.256655+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:41.653350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.245322+0000","last_change":"2024-04-26T23:54:24.220453+0000","last_active":"2024-04-26T23:54:25.245322+0000","last_peered":"2024-04-26T23:54:25.245322+0000","last_clean":"2024-04-26T23:54:25.245322+0000","last_became_active":"2024-04-26T23:54:24.220043+0000","last_became_peered":"2024-04-26T23:54:24.220043+0000","last_unstale":"2024-04-26T23:54:25.245322+0000","last_undegraded":"2024-04-26T23:54:25.245322+0000","last_fullsized":"2024-04-26T23:54:25.245322+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:18.944650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.271220+0000","last_change":"2024-04-26T23:54:25.267824+0000","last_active":"2024-04-26T23:54:25.271220+0000","last_peered":"2024-04-26T23:54:25.271220+0000","last_clean":"2024-04-26T23:54:25.271220+0000","last_became_active":"2024-04-26T23:54:25.266453+0000","last_became_peered":"2024-04-26T23:54:25.266453+0000","last_unstale":"2024-04-26T23:54:25.271220+0000","last_undegraded":"2024-04-26T23:54:25.271220+0000","last_fullsized":"2024-04-26T23:54:25.271220+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:21.902328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.234712+0000","last_change":"2024-04-26T23:54:24.227633+0000","last_active":"2024-04-26T23:54:25.234712+0000","last_peered":"2024-04-26T23:54:25.234712+0000","last_clean":"2024-04-26T23:54:25.234712+0000","last_became_active":"2024-04-26T23:54:24.209171+0000","last_became_peered":"2024-04-26T23:54:24.209171+0000","last_unstale":"2024-04-26T23:54:25.234712+0000","last_undegraded":"2024-04-26T23:54:25.234712+0000","last_fullsized":"2024-04-26T23:54:25.234712+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:45.659252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.253079+0000","last_change":"2024-04-26T23:54:25.251528+0000","last_active":"2024-04-26T23:54:25.253079+0000","last_peered":"2024-04-26T23:54:25.253079+0000","last_clean":"2024-04-26T23:54:25.253079+0000","last_became_active":"2024-04-26T23:54:25.251089+0000","last_became_peered":"2024-04-26T23:54:25.251089+0000","last_unstale":"2024-04-26T23:54:25.253079+0000","last_undegraded":"2024-04-26T23:54:25.253079+0000","last_fullsized":"2024-04-26T23:54:25.253079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:07.065465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.233143+0000","last_change":"2024-04-26T23:54:24.212073+0000","last_active":"2024-04-26T23:54:25.233143+0000","last_peered":"2024-04-26T23:54:25.233143+0000","last_clean":"2024-04-26T23:54:25.233143+0000","last_became_active":"2024-04-26T23:54:24.211739+0000","last_became_peered":"2024-04-26T23:54:24.211739+0000","last_unstale":"2024-04-26T23:54:25.233143+0000","last_undegraded":"2024-04-26T23:54:25.233143+0000","last_fullsized":"2024-04-26T23:54:25.233143+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:57.031740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.254129+0000","last_change":"2024-04-26T23:54:25.249406+0000","last_active":"2024-04-26T23:54:25.254129+0000","last_peered":"2024-04-26T23:54:25.254129+0000","last_clean":"2024-04-26T23:54:25.254129+0000","last_became_active":"2024-04-26T23:54:25.249040+0000","last_became_peered":"2024-04-26T23:54:25.249040+0000","last_unstale":"2024-04-26T23:54:25.254129+0000","last_undegraded":"2024-04-26T23:54:25.254129+0000","last_fullsized":"2024-04-26T23:54:25.254129+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:24.173956+0000","last_clean_scrub_stamp":"2024-04-26T23:54:24.173956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:31.324349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.215774+0000","last_change":"2024-04-26T23:54:24.211118+0000","last_active":"2024-04-26T23:54:25.215774+0000","last_peered":"2024-04-26T23:54:25.215774+0000","last_clean":"2024-04-26T23:54:25.215774+0000","last_became_active":"2024-04-26T23:54:24.210851+0000","last_became_peered":"2024-04-26T23:54:24.210851+0000","last_unstale":"2024-04-26T23:54:25.215774+0000","last_undegraded":"2024-04-26T23:54:25.215774+0000","last_fullsized":"2024-04-26T23:54:25.215774+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:54:23.159001+0000","last_clean_scrub_stamp":"2024-04-26T23:54:23.159001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:20.887373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-04-26T23:54:25.240219+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:54:25.240219+0000","last_peered":"2024-04-26T23:54:25.240219+0000","last_clean":"2024-04-26T23:54:25.240219+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:54:25.240219+0000","last_undegraded":"2024-04-26T23:54:25.240219+0000","last_fullsized":"2024-04-26T23:54:25.240219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575059,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51924,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319916,"statfs":{"total":96636764160,"available":96583593984,"internally_reserved":0,"allocated":2760704,"data_stored":1612083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575060,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55508,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316332,"statfs":{"total":96636764160,"available":96579923968,"internally_reserved":0,"allocated":2170880,"data_stored":1021785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575060,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55520,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316320,"statfs":{"total":96636764160,"available":96579911680,"internally_reserved":0,"allocated":2183168,"data_stored":1023349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2183168,"data_stored":1023361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575060,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47200,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324640,"statfs":{"total":96636764160,"available":96588431360,"internally_reserved":0,"allocated":2174976,"data_stored":1021875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59672,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312168,"statfs":{"total":96636764160,"available":96575660032,"internally_reserved":0,"allocated":2170880,"data_stored":1022297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47188,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324652,"statfs":{"total":96636764160,"available":96588443648,"internally_reserved":0,"allocated":2166784,"data_stored":1021715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575060,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60268,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311572,"statfs":{"total":96636764160,"available":96575049728,"internally_reserved":0,"allocated":2777088,"data_stored":1612811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:54:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:54:30.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:54:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.543+0000 7f3cfb600640 1 Processor -- start 2024-04-26T23:54:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.544+0000 7f3cfb600640 1 -- start start 2024-04-26T23:54:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.545+0000 7f3cfb600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3cf415f310 0x7f3cf40b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.545+0000 7f3cfb600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf406e810 0x7f3cf406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.545+0000 7f3cfb600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf40b4a80 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.545+0000 7f3cfb600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf40b4d60 con 0x7f3cf415f310 2024-04-26T23:54:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.545+0000 7f3cfb600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf40b5040 con 0x7f3cf406e810 2024-04-26T23:54:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.546+0000 7f3cf8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3cf415f310 0x7f3cf40b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.546+0000 7f3cf8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3cf415f310 0x7f3cf40b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49570/0 (socket says 172.21.15.46:49570) 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.546+0000 7f3cf8c00640 1 -- 172.21.15.46:0/1888538348 learned_addr learned my addr 172.21.15.46:0/1888538348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.546+0000 7f3cf9600640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf406e810 0x7f3cf406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.547+0000 7f3cf8c00640 1 -- 172.21.15.46:0/1888538348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf406e810 msgr2=0x7f3cf406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.547+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf406e810 0x7f3cf406ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.547+0000 7f3cf8c00640 1 -- 172.21.15.46:0/1888538348 >> v1:172.21.15.142:6789/0 conn(0x7f3cf40ae8e0 legacy=0x7f3cf40b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.547+0000 7f3cf8c00640 1 -- 172.21.15.46:0/1888538348 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf40b5320 con 0x7f3cf415f310 2024-04-26T23:54:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.548+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf415f310 0x7f3cf40b44a0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009870 tx=0x7f3ce402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf7cc25c319c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.549+0000 7f3cf3e00640 1 -- 172.21.15.46:0/1888538348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4037070 con 0x7f3cf415f310 2024-04-26T23:54:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.549+0000 7f3cf3e00640 1 -- 172.21.15.46:0/1888538348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce4032440 con 0x7f3cf415f310 2024-04-26T23:54:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.549+0000 7f3cf3e00640 1 -- 172.21.15.46:0/1888538348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4032810 con 0x7f3cf415f310 2024-04-26T23:54:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.549+0000 7f3cfb600640 1 -- 172.21.15.46:0/1888538348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf415f310 msgr2=0x7f3cf40b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.549+0000 7f3cfb600640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf415f310 0x7f3cf40b44a0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009870 tx=0x7f3ce402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 -- 172.21.15.46:0/1888538348 shutdown_connections 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf406e810 0x7f3cf406ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 --2- 172.21.15.46:0/1888538348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf415f310 0x7f3cf40b44a0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 -- 172.21.15.46:0/1888538348 >> 172.21.15.46:0/1888538348 conn(0x7f3cf40727e0 msgr2=0x7f3cf4072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 -- 172.21.15.46:0/1888538348 shutdown_connections 2024-04-26T23:54:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.550+0000 7f3cfb600640 1 -- 172.21.15.46:0/1888538348 wait complete. 2024-04-26T23:54:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.552+0000 7f3cfb600640 1 Processor -- start 2024-04-26T23:54:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.552+0000 7f3cfb600640 1 -- start start 2024-04-26T23:54:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 0x7f3cf414a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf415f310 0x7f3cf4154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40b6230 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40735a0 con 0x7f3cf406e810 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cfb600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf40b63b0 con 0x7f3cf415f310 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49586/0 (socket says 172.21.15.46:49586) 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cebe00640 1 -- 172.21.15.46:0/349203315 learned_addr learned my addr 172.21.15.46:0/349203315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cf9600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf415f310 0x7f3cf4154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.553+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 0x7f3cf414a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 -- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf415f310 msgr2=0x7f3cf4154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf415f310 0x7f3cf4154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 -- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 msgr2=0x7f3cf414a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 0x7f3cf414a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 -- 172.21.15.46:0/349203315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf414f870 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 0x7f3cf414a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cebe00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c430 tx=0x7f3cdc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0052d0 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc005490 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc007260 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.554+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf414c3d0 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.555+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf414c990 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.556+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4069530 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.557+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cdc007400 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.558+0000 7f3cf1600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 0x7f3cd006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.558+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7f3cdc08b2f0 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.558+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 0x7f3cd006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.560+0000 7f3cf8c00640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 0x7f3cd006ee00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4034000 tx=0x7f3ce402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.562+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cdc0594e0 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.843+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3cf414fa40 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.848+0000 7f3cf1600640 1 -- 172.21.15.46:0/349203315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f3cdc058de0 con 0x7f3cf40ae8e0 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:54:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:54:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:54:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.850+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 msgr2=0x7f3cd006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.850+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 0x7f3cd006ee00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4034000 tx=0x7f3ce402e000 comp rx=0 tx=0).stop 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 msgr2=0x7f3cf414ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c430 tx=0x7f3cdc00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 shutdown_connections 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3cd006c940 0x7f3cd006ee00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3cf415f310 0x7f3cf4154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ae8e0 0x7f3cf414ba10 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 --2- 172.21.15.46:0/349203315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cf406e810 0x7f3cf414a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 >> 172.21.15.46:0/349203315 conn(0x7f3cf40727e0 msgr2=0x7f3cf406ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 shutdown_connections 2024-04-26T23:54:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:30.851+0000 7f3cfb600640 1 -- 172.21.15.46:0/349203315 wait complete. 2024-04-26T23:54:30.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.083+0000 7fd64be00640 1 Processor -- start 2024-04-26T23:54:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.083+0000 7fd64be00640 1 -- start start 2024-04-26T23:54:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd64be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd64414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd64be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd64414d030 0x7fd644072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd64be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6440b4940 con 0x7fd64406e780 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd64be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd6440b4c20 con 0x7fd64414d030 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd64be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6440b4f00 con 0x7fd64406eb60 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd649400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd64406e780 0x7fd6440b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57730/0 (socket says 172.21.15.46:57730) 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.085+0000 7fd649400640 1 -- 172.21.15.46:0/4008478217 learned_addr learned my addr 172.21.15.46:0/4008478217 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd649e00640 1 --2- 172.21.15.46:0/4008478217 >> v2:172.21.15.46:3300/0 conn(0x7fd64414d030 0x7fd644072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd643e00640 1 -- 172.21.15.46:0/4008478217 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588679682 0 0) 0x7fd6440b4940 con 0x7fd64406e780 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd643e00640 1 -- 172.21.15.46:0/4008478217 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd62c003520 con 0x7fd64406e780 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd648a00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd64414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd649e00640 1 -- 172.21.15.46:0/4008478217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 msgr2=0x7fd64414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd649e00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd64414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd649e00640 1 -- 172.21.15.46:0/4008478217 >> v1:172.21.15.46:6789/0 conn(0x7fd64406e780 legacy=0x7fd6440b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.086+0000 7fd649e00640 1 -- 172.21.15.46:0/4008478217 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6440b51e0 con 0x7fd64414d030 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.087+0000 7fd649e00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644072360 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0097a0 tx=0x7fd63c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb734f1e30d447c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd643e00640 1 -- 172.21.15.46:0/4008478217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c037070 con 0x7fd64414d030 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd643e00640 1 -- 172.21.15.46:0/4008478217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c032440 con 0x7fd64414d030 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd643e00640 1 -- 172.21.15.46:0/4008478217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0327f0 con 0x7fd64414d030 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 -- 172.21.15.46:0/4008478217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 msgr2=0x7fd644072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644072360 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0097a0 tx=0x7fd63c02a180 comp rx=0 tx=0).stop 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 -- 172.21.15.46:0/4008478217 shutdown_connections 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644072360 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 --2- 172.21.15.46:0/4008478217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd64414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 -- 172.21.15.46:0/4008478217 >> 172.21.15.46:0/4008478217 conn(0x7fd64406c530 msgr2=0x7fd64406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 -- 172.21.15.46:0/4008478217 shutdown_connections 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.088+0000 7fd64be00640 1 -- 172.21.15.46:0/4008478217 wait complete. 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.090+0000 7fd64be00640 1 Processor -- start 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.090+0000 7fd64be00640 1 -- start start 2024-04-26T23:54:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64406e780 0x7fd644162700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd644143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440b4590 con 0x7fd64414d030 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440728a0 con 0x7fd64406e780 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.091+0000 7fd64be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd64415fd40 con 0x7fd64406eb60 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49618/0 (socket says 172.21.15.46:49618) 2024-04-26T23:54:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 -- 172.21.15.46:0/1242044773 learned_addr learned my addr 172.21.15.46:0/1242044773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 -- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 msgr2=0x7fd644143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd644143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 -- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64406e780 msgr2=0x7fd644162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64406e780 0x7fd644162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 -- 172.21.15.46:0/1242044773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64415f130 con 0x7fd64414d030 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.092+0000 7fd649e00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fd63c035040 tx=0x7fd63c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.093+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c041030 con 0x7fd64414d030 2024-04-26T23:54:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.093+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c030440 con 0x7fd64414d030 2024-04-26T23:54:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.093+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c037040 con 0x7fd64414d030 2024-04-26T23:54:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.093+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64415f410 con 0x7fd64414d030 2024-04-26T23:54:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.093+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64415f9d0 con 0x7fd64414d030 2024-04-26T23:54:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.094+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64415fec0 con 0x7fd64414d030 2024-04-26T23:54:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.095+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd63c044070 con 0x7fd64414d030 2024-04-26T23:54:31.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.097+0000 7fd641600640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.097+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fd63c0ac4f0 con 0x7fd64414d030 2024-04-26T23:54:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.097+0000 7fd649400640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.098+0000 7fd649400640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 0x7fd62806edb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd634000f60 tx=0x7fd6340078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.101+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd63c07a6f0 con 0x7fd64414d030 2024-04-26T23:54:31.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.351+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd644069530 con 0x7fd64414d030 2024-04-26T23:54:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.353+0000 7fd641600640 1 -- 172.21.15.46:0/1242044773 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7fd63c079ff0 con 0x7fd64414d030 2024-04-26T23:54:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:31.356 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:54:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:54:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.355+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 msgr2=0x7fd62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.355+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 0x7fd62806edb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd634000f60 tx=0x7fd6340078f0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 msgr2=0x7fd644143ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fd63c035040 tx=0x7fd63c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 shutdown_connections 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd64414d030 0x7fd644143ac0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd64406eb60 0x7fd644143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 --2- 172.21.15.46:0/1242044773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64406e780 0x7fd644162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 >> 172.21.15.46:0/1242044773 conn(0x7fd64406c530 msgr2=0x7fd6440b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 shutdown_connections 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:31.356+0000 7fd64be00640 1 -- 172.21.15.46:0/1242044773 wait complete. 2024-04-26T23:54:31.782 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:54:24.905219+0000', 'modified': '2024-04-26T23:54:27.780325+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10770}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10770': {'gid': 10770, 'name': 'a', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/7977112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 7977112}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 7977112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10770, 'qdb_cluster': [10770]} 2024-04-26T23:54:31.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:54:31.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:54:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.003+0000 7fc41b200640 1 Processor -- start 2024-04-26T23:54:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.003+0000 7fc41b200640 1 -- start start 2024-04-26T23:54:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc41c0720e0 0x7fc41c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc41c160910 0x7fc41c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc41c073ad0 con 0x7fc41c160910 2024-04-26T23:54:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc41c146a90 con 0x7fc41c0720e0 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c073d70 con 0x7fc41c1466b0 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc41c160910 0x7fc41c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc41c160910 0x7fc41c160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57236/0 (socket says 172.21.15.46:57236) 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.005+0000 7fc41a800640 1 -- 172.21.15.46:0/1581809613 learned_addr learned my addr 172.21.15.46:0/1581809613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc419e00640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc41a800640 1 -- 172.21.15.46:0/1581809613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 msgr2=0x7fc41c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc41a800640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc41a800640 1 -- 172.21.15.46:0/1581809613 >> v2:172.21.15.142:3300/0 conn(0x7fc41c0720e0 msgr2=0x7fc41c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc413e00640 1 --2- 172.21.15.46:0/1581809613 >> v2:172.21.15.142:3300/0 conn(0x7fc41c0720e0 0x7fc41c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc41a800640 1 --2- 172.21.15.46:0/1581809613 >> v2:172.21.15.142:3300/0 conn(0x7fc41c0720e0 0x7fc41c0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc41a800640 1 -- 172.21.15.46:0/1581809613 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c074050 con 0x7fc41c160910 2024-04-26T23:54:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.006+0000 7fc419e00640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.007+0000 7fc41a800640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c160910 0x7fc41c160da0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0d0990 tx=0x7fc41402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca95a3b32224fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.007+0000 7fc419400640 1 -- 172.21.15.46:0/1581809613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414036070 con 0x7fc41c160910 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.007+0000 7fc419400640 1 -- 172.21.15.46:0/1581809613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414031600 con 0x7fc41c160910 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.007+0000 7fc419400640 1 -- 172.21.15.46:0/1581809613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4140319d0 con 0x7fc41c160910 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 -- 172.21.15.46:0/1581809613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c160910 msgr2=0x7fc41c160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c160910 0x7fc41c160da0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0d0990 tx=0x7fc41402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 -- 172.21.15.46:0/1581809613 shutdown_connections 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c160910 0x7fc41c160da0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 --2- 172.21.15.46:0/1581809613 >> v2:172.21.15.142:3300/0 conn(0x7fc41c0720e0 0x7fc41c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 --2- 172.21.15.46:0/1581809613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 -- 172.21.15.46:0/1581809613 >> 172.21.15.46:0/1581809613 conn(0x7fc41c0b26d0 msgr2=0x7fc41c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 -- 172.21.15.46:0/1581809613 shutdown_connections 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.008+0000 7fc41b200640 1 -- 172.21.15.46:0/1581809613 wait complete. 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.010+0000 7fc41b200640 1 Processor -- start 2024-04-26T23:54:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.010+0000 7fc41b200640 1 -- start start 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 0x7fc41c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c06e3c0 con 0x7fc41c0720e0 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c0cb470 con 0x7fc41c160910 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc41b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c0cdda0 con 0x7fc41c1466b0 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57244/0 (socket says 172.21.15.46:57244) 2024-04-26T23:54:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.011+0000 7fc419e00640 1 -- 172.21.15.46:0/3229899615 learned_addr learned my addr 172.21.15.46:0/3229899615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc413e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc41a800640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 0x7fc41c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 -- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 msgr2=0x7fc41c134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c134140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 -- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 msgr2=0x7fc41c0c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 0x7fc41c0c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 -- 172.21.15.46:0/3229899615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c0b6800 con 0x7fc41c0720e0 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc413e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc41a800640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 0x7fc41c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.012+0000 7fc419e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c7f0 tx=0x7fc40c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.013+0000 7fc412000640 1 -- 172.21.15.46:0/3229899615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c015070 con 0x7fc41c0720e0 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.013+0000 7fc412000640 1 -- 172.21.15.46:0/3229899615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c007820 con 0x7fc41c0720e0 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.013+0000 7fc412000640 1 -- 172.21.15.46:0/3229899615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007bd0 con 0x7fc41c0720e0 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.013+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c134a80 con 0x7fc41c0720e0 2024-04-26T23:54:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.013+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c135040 con 0x7fc41c0720e0 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.015+0000 7fc412000640 1 -- 172.21.15.46:0/3229899615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc40c018070 con 0x7fc41c0720e0 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.017+0000 7fc412000640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 0x7fc3f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.017+0000 7fc423b6d740 1 Processor -- start 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.017+0000 7fc412000640 1 -- 172.21.15.46:0/3229899615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fc40c08be50 con 0x7fc41c0720e0 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.017+0000 7fc423b6d740 1 -- start start 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc413e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 0x7fc3f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890c2e80 0x55d789086b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890b1270 0x55d7890b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d788fea600 con 0x55d7890867b0 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d788f87100 con 0x55d7890b1270 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.018+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d788fe65c0 con 0x55d7890c2e80 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57248/0 (socket says 172.21.15.46:57248) 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 -- 172.21.15.46:0/3853905584 learned_addr learned my addr 172.21.15.46:0/3853905584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 -- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890c2e80 msgr2=0x55d789086b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890c2e80 0x55d789086b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 -- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890b1270 msgr2=0x55d7890b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890b1270 0x55d7890b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 -- 172.21.15.46:0/3853905584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d788cfc2d0 con 0x55d7890867b0 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc413e00640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 0x7fc3f806edb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc41c1e6d70 tx=0x7fc404008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.019+0000 7fc419e00640 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fc40c08a0c0 tx=0x7fc40c0595a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbe2e5f7eb4a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc40aa00640 1 -- 172.21.15.46:0/3853905584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c01f040 con 0x55d7890867b0 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc40aa00640 1 -- 172.21.15.46:0/3853905584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c057c50 con 0x55d7890867b0 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc40aa00640 1 -- 172.21.15.46:0/3853905584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c060070 con 0x55d7890867b0 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 -- 172.21.15.46:0/3853905584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 msgr2=0x55d789088910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fc40c08a0c0 tx=0x7fc40c0595a0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 -- 172.21.15.46:0/3853905584 shutdown_connections 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890b1270 0x55d7890b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890c2e80 0x55d789086b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 --2- 172.21.15.46:0/3853905584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d789088910 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.020+0000 7fc423b6d740 1 -- 172.21.15.46:0/3853905584 >> 172.21.15.46:0/3853905584 conn(0x55d78901ad10 msgr2=0x55d7890cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.021+0000 7fc423b6d740 1 -- 172.21.15.46:0/3853905584 shutdown_connections 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.021+0000 7fc423b6d740 1 -- 172.21.15.46:0/3853905584 wait complete. 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.021+0000 7fc423b6d740 1 -- start start 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.021+0000 7fc423b6d740 1 Processor -- start 2024-04-26T23:54:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.022+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890867b0 0x55d7890b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d788f71f70 con 0x55d7890b1270 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7890aedc0 con 0x55d7890c2e80 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d7890dcda0 con 0x55d7890867b0 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc423b6d740 10 client.0 _ll_register_callbacks cb 0x55d788fe0480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57262/0 (socket says 172.21.15.46:57262) 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 -- 172.21.15.46:0/2616323228 learned_addr learned my addr 172.21.15.46:0/2616323228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc419e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890867b0 0x55d7890b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc41a800640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 -- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890867b0 msgr2=0x55d7890b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890867b0 0x55d7890b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 -- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 msgr2=0x55d7890a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc413e00640 1 -- 172.21.15.46:0/2616323228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d788dc66b0 con 0x55d7890b1270 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.023+0000 7fc41a800640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.024+0000 7fc413e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fc4040146c0 tx=0x7fc404014ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f7a8d580ff7821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.024+0000 7fc3ff400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.024+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404015800 con 0x55d7890b1270 2024-04-26T23:54:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.024+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fc4040159a0 con 0x55d7890b1270 2024-04-26T23:54:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.024+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40c057c50 con 0x55d7890b1270 2024-04-26T23:54:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fc40402b050 con 0x55d7890b1270 2024-04-26T23:54:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 10 client.10929 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55d7890b2d60 con 0x55d7890b1270 2024-04-26T23:54:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c057c50 con 0x55d7890b1270 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fc40402e020 con 0x55d7890b1270 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 10 client.10929 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 10 client.10929 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 20 client.10929 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d7890a0dc0 0x55d7890b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 4 client.10929 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d788ff8210 con 0x55d7890a0dc0 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.025+0000 7fc419e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d7890a0dc0 0x55d7890b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.026+0000 7fc419e00640 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d7890a0dc0 0x55d7890b1840 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83038942ae7c6ac server_cookie=4c680487d0643c49 in_seq=0 out_seq=0 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.026+0000 7fc3ff400640 10 client.10929 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc3ff400640 1 -- 172.21.15.46:0/2616323228 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d788ff8210 con 0x55d7890a0dc0 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc3ff400640 10 client.10929 handle_command_reply: tid=0 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 client.10929 shutdown 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 msgr2=0x55d7890ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fc4040146c0 tx=0x7fc404014ac0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 shutdown_connections 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890b1270 0x55d7890ce7f0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d7890a0dc0 0x55d7890b1840 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2616323228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890867b0 0x55d7890b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 >> 172.21.15.46:0/2616323228 conn(0x55d78901ad10 msgr2=0x55d7890cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.027+0000 7fc3ff400640 0 client.10929 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.028+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 shutdown_connections 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.028+0000 7fc423b6d740 1 -- 172.21.15.46:0/2616323228 wait complete. 2024-04-26T23:54:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.028+0000 7fc423b6d740 20 client.10929 trim_cache size 0 max 16384 2024-04-26T23:54:32.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.048+0000 7fc423b6d740 1 Processor -- start 2024-04-26T23:54:32.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.048+0000 7fc423b6d740 1 -- start start 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c200ed0 con 0x55d7890867b0 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c136010 con 0x55d7890c2e80 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c136660 con 0x55d7890b1270 2024-04-26T23:54:32.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57278/0 (socket says 172.21.15.46:57278) 2024-04-26T23:54:32.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.049+0000 7fc419e00640 1 -- 172.21.15.46:0/2845865844 learned_addr learned my addr 172.21.15.46:0/2845865844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 -- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 msgr2=0x55d7890cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 -- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 msgr2=0x55d7890cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 -- 172.21.15.46:0/2845865844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7890845f0 con 0x55d7890867b0 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc419e00640 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fc40c059b10 tx=0x7fc40c060000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e84af53d841446 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc40a000640 1 -- 172.21.15.46:0/2845865844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c01f040 con 0x55d7890867b0 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc40a000640 1 -- 172.21.15.46:0/2845865844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c074070 con 0x55d7890867b0 2024-04-26T23:54:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.050+0000 7fc40a000640 1 -- 172.21.15.46:0/2845865844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c055bf0 con 0x55d7890867b0 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 -- 172.21.15.46:0/2845865844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 msgr2=0x55d7891419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fc40c059b10 tx=0x7fc40c060000 comp rx=0 tx=0).stop 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 -- 172.21.15.46:0/2845865844 shutdown_connections 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890c2e80 0x55d7890cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 --2- 172.21.15.46:0/2845865844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890867b0 0x55d7891419b0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 -- 172.21.15.46:0/2845865844 >> 172.21.15.46:0/2845865844 conn(0x55d78901ad10 msgr2=0x55d7890ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 -- 172.21.15.46:0/2845865844 shutdown_connections 2024-04-26T23:54:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.051+0000 7fc423b6d740 1 -- 172.21.15.46:0/2845865844 wait complete. 2024-04-26T23:54:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.052+0000 7fc423b6d740 1 -- start start 2024-04-26T23:54:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.052+0000 7fc423b6d740 1 Processor -- start 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890867b0 0x55d789140c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d7890a0c40 con 0x55d7890c2e80 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7890c4b80 con 0x55d7890867b0 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d7890dbf30 con 0x55d7890b1270 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc423b6d740 10 client.0 _ll_register_callbacks cb 0x7fc40400bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.053+0000 7fc41a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57288/0 (socket says 172.21.15.46:57288) 2024-04-26T23:54:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 -- 172.21.15.46:0/4058260944 learned_addr learned my addr 172.21.15.46:0/4058260944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 -- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 msgr2=0x55d7890c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 -- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890867b0 msgr2=0x55d789140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890867b0 0x55d789140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 -- 172.21.15.46:0/4058260944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7890c3a70 con 0x55d7890c2e80 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc41a800640 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc41402a6e0 tx=0x7fc414031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be7052c486fafd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.054+0000 7fc3ff400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414042070 con 0x55d7890c2e80 2024-04-26T23:54:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fc41403b440 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40c074070 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fc414036040 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 10 client.10935 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55d789081580 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c074070 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fc414003990 con 0x55d7890c2e80 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 10 client.10935 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 10 client.10935 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 20 client.10935 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d789084f70 0x55d789085350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 4 client.10935 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.055+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "status"}) -- 0x55d788ff2080 con 0x55d789084f70 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.056+0000 7fc419e00640 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d789084f70 0x55d789085350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.056+0000 7fc419e00640 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d789084f70 0x55d789085350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd65ecc467d724b server_cookie=172fe3d8b570412a in_seq=0 out_seq=0 2024-04-26T23:54:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.056+0000 7fc3ff400640 10 client.10935 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc3ff400640 1 -- 172.21.15.46:0/4058260944 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55d788ff2080 con 0x55d789084f70 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc3ff400640 10 client.10935 handle_command_reply: tid=0 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 client.10935 shutdown 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 msgr2=0x55d7890c2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc41402a6e0 tx=0x7fc414031910 comp rx=0 tx=0).stop 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 shutdown_connections 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7890c2e80 0x55d7890c2660 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55d789084f70 0x55d789085350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d7890b1270 0x55d7890c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 --2- 172.21.15.46:0/4058260944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7890867b0 0x55d789140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 >> 172.21.15.46:0/4058260944 conn(0x55d78901ad10 msgr2=0x55d7890ef920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc3ff400640 0 client.10935 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 shutdown_connections 2024-04-26T23:54:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 1 -- 172.21.15.46:0/4058260944 wait complete. 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.057+0000 7fc423b6d740 20 client.10935 trim_cache size 0 max 16384 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 10770, 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.8209863559999997, 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 140, 2024-04-26T23:54:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 173, 2024-04-26T23:54:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 173, 2024-04-26T23:54:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 13.249524836999999 2024-04-26T23:54:32.064 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:54:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 msgr2=0x7fc3f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 0x7fc3f806edb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc41c1e6d70 tx=0x7fc404008040 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 msgr2=0x7fc41c1e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c7f0 tx=0x7fc40c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 shutdown_connections 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3f806c8f0 0x7fc3f806edb0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc41c160910 0x7fc41c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41c1466b0 0x7fc41c134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 --2- 172.21.15.46:0/3229899615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc41c0720e0 0x7fc41c1e6830 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 >> 172.21.15.46:0/3229899615 conn(0x7fc41c0b26d0 msgr2=0x7fc41c160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 shutdown_connections 2024-04-26T23:54:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.058+0000 7fc41b200640 1 -- 172.21.15.46:0/3229899615 wait complete. 2024-04-26T23:54:32.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:54:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.288+0000 7f3250a00640 1 Processor -- start 2024-04-26T23:54:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.288+0000 7f3250a00640 1 -- start start 2024-04-26T23:54:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.289+0000 7f3250a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c145770 0x7f324c0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.289+0000 7f3250a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f324c15ccd0 0x7f324c0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.289+0000 7f3250a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324c0cf950 con 0x7f324c072e60 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.289+0000 7f3250a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f324c0d0c50 con 0x7f324c15ccd0 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.289+0000 7f3250a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f324c15d490 con 0x7f324c145770 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f324a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f324c15ccd0 0x7f324c0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f3243e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f324c072e60 0x7f324c15c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38572/0 (socket says 172.21.15.46:38572) 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f3243e00640 1 -- 172.21.15.46:0/4078101006 learned_addr learned my addr 172.21.15.46:0/4078101006 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f3249e00640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c145770 0x7f324c0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f324a800640 1 -- 172.21.15.46:0/4078101006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c145770 msgr2=0x7f324c0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f324a800640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c145770 0x7f324c0d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f324a800640 1 -- 172.21.15.46:0/4078101006 >> v1:172.21.15.46:6789/0 conn(0x7f324c072e60 legacy=0x7f324c15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f324a800640 1 -- 172.21.15.46:0/4078101006 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0cfc30 con 0x7f324c15ccd0 2024-04-26T23:54:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.290+0000 7f3249400640 1 -- 172.21.15.46:0/4078101006 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525194760 0 0) 0x7f324c0cf950 con 0x7f324c072e60 2024-04-26T23:54:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.291+0000 7f324a800640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c15ccd0 0x7f324c0732c0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f3244009d90 tx=0x7f324402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebf4cca679b9911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3249400640 1 -- 172.21.15.46:0/4078101006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244007740 con 0x7f324c15ccd0 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3249400640 1 -- 172.21.15.46:0/4078101006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440078e0 con 0x7f324c15ccd0 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3249400640 1 -- 172.21.15.46:0/4078101006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244007cb0 con 0x7f324c15ccd0 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 -- 172.21.15.46:0/4078101006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c15ccd0 msgr2=0x7f324c0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c15ccd0 0x7f324c0732c0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f3244009d90 tx=0x7f324402a660 comp rx=0 tx=0).stop 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 -- 172.21.15.46:0/4078101006 shutdown_connections 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c15ccd0 0x7f324c0732c0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 --2- 172.21.15.46:0/4078101006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c145770 0x7f324c0d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.292+0000 7f3250a00640 1 -- 172.21.15.46:0/4078101006 >> 172.21.15.46:0/4078101006 conn(0x7f324c06c530 msgr2=0x7f324c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.293+0000 7f3250a00640 1 -- 172.21.15.46:0/4078101006 shutdown_connections 2024-04-26T23:54:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.293+0000 7f3250a00640 1 -- 172.21.15.46:0/4078101006 wait complete. 2024-04-26T23:54:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.294+0000 7f3250a00640 1 Processor -- start 2024-04-26T23:54:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.294+0000 7f3250a00640 1 -- start start 2024-04-26T23:54:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 0x7f324c141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 0x7f324c13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f324c0d01f0 con 0x7f324c072e60 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f324c073800 con 0x7f324c145770 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.295+0000 7f3250a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0cf5b0 con 0x7f324c15ccd0 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57306/0 (socket says 172.21.15.46:57306) 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 -- 172.21.15.46:0/533519339 learned_addr learned my addr 172.21.15.46:0/533519339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f324a800640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 0x7f324c13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3243e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 0x7f324c141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 -- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 msgr2=0x7f324c13ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 0x7f324c13ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 -- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 msgr2=0x7f324c141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 0x7f324c141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3249e00640 1 -- 172.21.15.46:0/533519339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c13e8b0 con 0x7f324c072e60 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f324a800640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 0x7f324c13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.296+0000 7f3243e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 0x7f324c141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3249e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d800 tx=0x7f323c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00b6c0 con 0x7f324c072e60 2024-04-26T23:54:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c019070 con 0x7f324c072e60 2024-04-26T23:54:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00ba80 con 0x7f324c072e60 2024-04-26T23:54:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c131bd0 con 0x7f324c072e60 2024-04-26T23:54:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.297+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c132410 con 0x7f324c072e60 2024-04-26T23:54:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.298+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c069530 con 0x7f324c072e60 2024-04-26T23:54:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.299+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f323c00bc20 con 0x7f324c072e60 2024-04-26T23:54:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.300+0000 7f3242000640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 0x7f322806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.300+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7f323c08c1d0 con 0x7f324c072e60 2024-04-26T23:54:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.301+0000 7f3243e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 0x7f322806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.302+0000 7f3243e00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 0x7f322806edb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f324c0b0500 tx=0x7f3234009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.305+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f324c131bd0 con 0x7f324c072e60 2024-04-26T23:54:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.559+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f324c073dc0 con 0x7f324c072e60 2024-04-26T23:54:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.559+0000 7f3242000640 1 -- 172.21.15.46:0/533519339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v140) ==== 53+0+73 (secure 0 0 0) 0x7f323c01c360 con 0x7f324c072e60 2024-04-26T23:54:32.563 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:54:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 msgr2=0x7f322806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 0x7f322806edb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f324c0b0500 tx=0x7f3234009040 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 msgr2=0x7f324c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d800 tx=0x7f323c00dcc0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 shutdown_connections 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f322806c8f0 0x7f322806edb0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f324c15ccd0 0x7f324c13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f324c145770 0x7f324c141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 --2- 172.21.15.46:0/533519339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f324c072e60 0x7f324c13e370 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 >> 172.21.15.46:0/533519339 conn(0x7f324c06c530 msgr2=0x7f324c0b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 shutdown_connections 2024-04-26T23:54:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32.562+0000 7f3250a00640 1 -- 172.21.15.46:0/533519339 wait complete. 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:54:32.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:54:32.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:32.588 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:54:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41548sec preferred_lft 41548sec 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:54:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:32.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:32.642 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:54:32.724 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:54:32.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:32.729 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:54:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:54:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:54:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:54:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:54:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:54:32.785 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:32.785 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:54:32.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:54:32.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:54:32.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:54:32.786 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:54:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:33.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:33.009 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:54:33.061 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:54:33.061 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:54:33.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:54:33.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:33.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:54:33.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:54:33.142 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:54:33.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:54:33.196 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:33.252 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:54:33.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:33.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:54:33.385 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:54:33.385 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:54:33.385 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:54:33.410 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:54:33.410 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:54:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:54:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:54:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:54:33.439 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:54:33.439 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:54:33.439 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:54:33.439 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:54:33.440 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:54:33.440 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:54:33.440 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:54:33.440 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:54:33.444 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:54:33.444 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:54:33.444 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:54:33.444 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:54:33.444 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:54:33.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:54:33.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:33.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:33.525 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:54:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:54:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:33.657 INFO:teuthology.orchestra.run.smithi046.stdout:client6725 2024-04-26T23:54:33.658 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:54:33.658 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:33.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.12:0 damage ls 2024-04-26T23:54:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.922+0000 7fe426400640 1 Processor -- start 2024-04-26T23:54:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.922+0000 7fe426400640 1 -- start start 2024-04-26T23:54:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe426400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe420073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe426400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4200d0f90 0x7fe420071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe426400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42006ec30 con 0x7fe4200cfed0 2024-04-26T23:54:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe426400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe420145ff0 con 0x7fe4200d0f90 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe426400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4200cf3f0 con 0x7fe420145c10 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.924+0000 7fe425a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4200d0f90 0x7fe420071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4200d0f90 0x7fe420071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57312/0 (socket says 172.21.15.46:57312) 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 -- 172.21.15.46:0/1863321792 learned_addr learned my addr 172.21.15.46:0/1863321792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425000640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe420073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 -- 172.21.15.46:0/1863321792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 msgr2=0x7fe420073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe420073c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 -- 172.21.15.46:0/1863321792 >> v1:172.21.15.142:6789/0 conn(0x7fe4200cfed0 legacy=0x7fe4200d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.925+0000 7fe425a00640 1 -- 172.21.15.46:0/1863321792 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4200cf6d0 con 0x7fe4200d0f90 2024-04-26T23:54:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.927+0000 7fe425a00640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200d0f90 0x7fe420071e20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fe418009870 tx=0x7fe41802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8ae147af92464 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.927+0000 7fe41f400640 1 -- 172.21.15.46:0/1863321792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418037070 con 0x7fe4200d0f90 2024-04-26T23:54:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.927+0000 7fe41f400640 1 -- 172.21.15.46:0/1863321792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe418032440 con 0x7fe4200d0f90 2024-04-26T23:54:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.927+0000 7fe41f400640 1 -- 172.21.15.46:0/1863321792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418032810 con 0x7fe4200d0f90 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 -- 172.21.15.46:0/1863321792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200d0f90 msgr2=0x7fe420071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200d0f90 0x7fe420071e20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fe418009870 tx=0x7fe41802a270 comp rx=0 tx=0).stop 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 -- 172.21.15.46:0/1863321792 shutdown_connections 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200d0f90 0x7fe420071e20 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 --2- 172.21.15.46:0/1863321792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe420073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 -- 172.21.15.46:0/1863321792 >> 172.21.15.46:0/1863321792 conn(0x7fe42006c350 msgr2=0x7fe42006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 -- 172.21.15.46:0/1863321792 shutdown_connections 2024-04-26T23:54:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.928+0000 7fe426400640 1 -- 172.21.15.46:0/1863321792 wait complete. 2024-04-26T23:54:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.930+0000 7fe426400640 1 Processor -- start 2024-04-26T23:54:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.930+0000 7fe426400640 1 -- start start 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.931+0000 7fe426400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.931+0000 7fe426400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 0x7fe42015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.931+0000 7fe426400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe42015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe426400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201468b0 con 0x7fe4200cfed0 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe426400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4200cf0a0 con 0x7fe4200d0f90 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe426400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4200d0330 con 0x7fe420145c10 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57318/0 (socket says 172.21.15.46:57318) 2024-04-26T23:54:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 -- 172.21.15.46:0/3191109257 learned_addr learned my addr 172.21.15.46:0/3191109257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 0x7fe42015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425a00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe42015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 -- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 msgr2=0x7fe42015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe42015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 -- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 msgr2=0x7fe42015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 0x7fe42015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425000640 1 -- 172.21.15.46:0/3191109257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe420157d60 con 0x7fe4200cfed0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 0x7fe42015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.932+0000 7fe425a00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe42015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe425000640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ccd0 tx=0x7fe41400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe41cc00640 1 -- 172.21.15.46:0/3191109257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414003990 con 0x7fe4200cfed0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe41cc00640 1 -- 172.21.15.46:0/3191109257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe414003b30 con 0x7fe4200cfed0 2024-04-26T23:54:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4200b3390 con 0x7fe4200cfed0 2024-04-26T23:54:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe41cc00640 1 -- 172.21.15.46:0/3191109257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414004020 con 0x7fe4200cfed0 2024-04-26T23:54:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.933+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4200b1550 con 0x7fe4200cfed0 2024-04-26T23:54:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.936+0000 7fe41cc00640 1 -- 172.21.15.46:0/3191109257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4140041c0 con 0x7fe4200cfed0 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.937+0000 7fe41cc00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 0x7fe40006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.937+0000 7fe41cc00640 1 -- 172.21.15.46:0/3191109257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fe41408c1d0 con 0x7fe4200cfed0 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.937+0000 7fe42a99d740 1 Processor -- start 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.937+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 0x7fe40006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.937+0000 7fe42a99d740 1 -- start start 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb580880 0x55c6eb5831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb553100 0x55c6eb556910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb4bbc70 con 0x55c6eb5804a0 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb474980 con 0x55c6eb580880 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c6eb4b7c30 con 0x55c6eb553100 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.938+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 0x7fe40006ee00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe4200b26d0 tx=0x7fe40c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57332/0 (socket says 172.21.15.46:57332) 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 -- 172.21.15.46:0/3383812226 learned_addr learned my addr 172.21.15.46:0/3383812226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 -- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb553100 msgr2=0x55c6eb556910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb553100 0x55c6eb556910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 -- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb580880 msgr2=0x55c6eb5831f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb580880 0x55c6eb5831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb580880 0x55c6eb5831f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 -- 172.21.15.46:0/3383812226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6eb44c7b0 con 0x55c6eb5804a0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.939+0000 7fe425000640 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fe41408a4c0 tx=0x7fe414059c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb8fa96d5b71722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe411600640 1 -- 172.21.15.46:0/3383812226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414072020 con 0x55c6eb5804a0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe411600640 1 -- 172.21.15.46:0/3383812226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41405e100 con 0x55c6eb5804a0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe411600640 1 -- 172.21.15.46:0/3383812226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41401d040 con 0x55c6eb5804a0 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 -- 172.21.15.46:0/3383812226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 msgr2=0x55c6eb582cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fe41408a4c0 tx=0x7fe414059c50 comp rx=0 tx=0).stop 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 -- 172.21.15.46:0/3383812226 shutdown_connections 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb553100 0x55c6eb556910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb580880 0x55c6eb5831f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3383812226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb582cb0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.940+0000 7fe42a99d740 1 -- 172.21.15.46:0/3383812226 >> 172.21.15.46:0/3383812226 conn(0x55c6eb555370 msgr2=0x55c6eb59dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.941+0000 7fe42a99d740 1 -- 172.21.15.46:0/3383812226 shutdown_connections 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.941+0000 7fe42a99d740 1 -- 172.21.15.46:0/3383812226 wait complete. 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.941+0000 7fe42a99d740 1 -- start start 2024-04-26T23:54:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.941+0000 7fe42a99d740 1 Processor -- start 2024-04-26T23:54:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb580880 0x55c6eb552320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb576d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb4baf80 con 0x55c6eb5804a0 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb468260 con 0x55c6eb576900 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.942+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c6eb4c02a0 con 0x55c6eb580880 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe42a99d740 10 client.0 _ll_register_callbacks cb 0x55c6eb4b2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57336/0 (socket says 172.21.15.46:57336) 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 -- 172.21.15.46:0/2067090559 learned_addr learned my addr 172.21.15.46:0/2067090559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425a00640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb576d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 -- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb580880 msgr2=0x55c6eb552320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb580880 0x55c6eb552320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 -- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 msgr2=0x55c6eb576d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb576d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425000640 1 -- 172.21.15.46:0/2067090559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6eb44a1c0 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.943+0000 7fe425a00640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb576d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe425000640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fe41400cca0 tx=0x7fe414000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0debe0f3ba7f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41401d040 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fe414079300 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41405e100 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fe414072040 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe42a99d740 10 client.10956 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55c6eb1cd3f0 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41405e100 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.944+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fe4140563c0 con 0x55c6eb5804a0 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 10 client.10956 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 10 client.10956 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 20 client.10956 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb5adc10 0x55c6eb5ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 4 client.10956 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6eb4c9880 con 0x55c6eb5adc10 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe41fe00640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb5adc10 0x55c6eb5ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.945+0000 7fe41fe00640 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb5adc10 0x55c6eb5ae080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a2b1a0c3ee2750 server_cookie=68fbf2f34628472c in_seq=0 out_seq=0 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.946+0000 7fe406000640 10 client.10956 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.946+0000 7fe406000640 1 -- 172.21.15.46:0/2067090559 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6eb4c9880 con 0x55c6eb5adc10 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.946+0000 7fe406000640 10 client.10956 handle_command_reply: tid=0 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.946+0000 7fe42a99d740 1 client.10956 shutdown 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 msgr2=0x55c6eb599ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fe41400cca0 tx=0x7fe414000f40 comp rx=0 tx=0).stop 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 shutdown_connections 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb576d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb5adc10 0x55c6eb5ae080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb580880 0x55c6eb552320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 --2- 172.21.15.46:0/2067090559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb5804a0 0x55c6eb599ef0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 >> 172.21.15.46:0/2067090559 conn(0x55c6eb553100 msgr2=0x55c6eb582db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe406000640 0 client.10956 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 shutdown_connections 2024-04-26T23:54:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 1 -- 172.21.15.46:0/2067090559 wait complete. 2024-04-26T23:54:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.947+0000 7fe42a99d740 20 client.10956 trim_cache size 0 max 16384 2024-04-26T23:54:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.962+0000 7fe42a99d740 1 Processor -- start 2024-04-26T23:54:33.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.962+0000 7fe42a99d740 1 -- start start 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb5578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe420201d60 con 0x55c6eb580880 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe420200da0 con 0x55c6eb576900 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.963+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe42002da70 con 0x55c6eb5804a0 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57350/0 (socket says 172.21.15.46:57350) 2024-04-26T23:54:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 -- 172.21.15.46:0/3346416100 learned_addr learned my addr 172.21.15.46:0/3346416100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425000640 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe41fe00640 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb5578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 -- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 msgr2=0x55c6eb5578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb5578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 -- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 msgr2=0x55c6eb5adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.964+0000 7fe425a00640 1 -- 172.21.15.46:0/3346416100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6eb2977d0 con 0x55c6eb580880 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.965+0000 7fe425a00640 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fe41802a2b0 tx=0x7fe41802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e057a6c9b455b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.965+0000 7fe410c00640 1 -- 172.21.15.46:0/3346416100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418042070 con 0x55c6eb580880 2024-04-26T23:54:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.965+0000 7fe410c00640 1 -- 172.21.15.46:0/3346416100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe418032780 con 0x55c6eb580880 2024-04-26T23:54:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.965+0000 7fe410c00640 1 -- 172.21.15.46:0/3346416100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418037040 con 0x55c6eb580880 2024-04-26T23:54:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 -- 172.21.15.46:0/3346416100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 msgr2=0x55c6eb5534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fe41802a2b0 tx=0x7fe41802c000 comp rx=0 tx=0).stop 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 -- 172.21.15.46:0/3346416100 shutdown_connections 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5534e0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb5578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 --2- 172.21.15.46:0/3346416100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 -- 172.21.15.46:0/3346416100 >> 172.21.15.46:0/3346416100 conn(0x55c6eb553100 msgr2=0x55c6eb558480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 -- 172.21.15.46:0/3346416100 shutdown_connections 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.966+0000 7fe42a99d740 1 -- 172.21.15.46:0/3346416100 wait complete. 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.967+0000 7fe42a99d740 1 -- start start 2024-04-26T23:54:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.967+0000 7fe42a99d740 1 Processor -- start 2024-04-26T23:54:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5fc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb59dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe420200750 con 0x55c6eb580880 2024-04-26T23:54:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb552660 con 0x55c6eb576900 2024-04-26T23:54:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c6eb599c20 con 0x55c6eb5804a0 2024-04-26T23:54:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe42a99d740 10 client.0 _ll_register_callbacks cb 0x7fe41405c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe425a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe425a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57360/0 (socket says 172.21.15.46:57360) 2024-04-26T23:54:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.968+0000 7fe425a00640 1 -- 172.21.15.46:0/1497948160 learned_addr learned my addr 172.21.15.46:0/1497948160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425000640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 -- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 msgr2=0x55c6eb59dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe41fe00640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb59dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb59dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 -- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 msgr2=0x55c6eb5fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5fc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 -- 172.21.15.46:0/1497948160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6eb4622a0 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe425a00640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe41802a2b0 tx=0x7fe418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9939ee92efd69835 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe406000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41804b060 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.969+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fe418032b40 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe418032780 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fe418051050 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 10 client.10962 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55c6eb59ea50 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe418032780 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fe418054020 con 0x55c6eb580880 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 10 client.10962 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 10 client.10962 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 20 client.10962 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb619a60 0x55c6eb619e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 4 client.10962 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:54:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.970+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6eb4c36f0 con 0x55c6eb619a60 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.971+0000 7fe425000640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb619a60 0x55c6eb619e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.971+0000 7fe425000640 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb619a60 0x55c6eb619e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168c606959ea33c7 server_cookie=d1a0004578670c1a in_seq=0 out_seq=0 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.971+0000 7fe406000640 10 client.10962 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe406000640 1 -- 172.21.15.46:0/1497948160 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6eb4c36f0 con 0x55c6eb619a60 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe406000640 10 client.10962 handle_command_reply: tid=0 2024-04-26T23:54:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 client.10962 shutdown 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 msgr2=0x55c6eb5996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe41802a2b0 tx=0x7fe418004730 comp rx=0 tx=0).stop 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 shutdown_connections 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6eb580880 0x55c6eb5996e0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c6eb619a60 0x55c6eb619e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6eb5804a0 0x55c6eb59dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 --2- 172.21.15.46:0/1497948160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6eb576900 0x55c6eb5fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 >> 172.21.15.46:0/1497948160 conn(0x55c6eb553100 msgr2=0x55c6eb54e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe406000640 0 client.10962 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 shutdown_connections 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 1 -- 172.21.15.46:0/1497948160 wait complete. 2024-04-26T23:54:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.972+0000 7fe42a99d740 20 client.10962 trim_cache size 0 max 16384 2024-04-26T23:54:33.978 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-26T23:54:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 msgr2=0x7fe40006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 0x7fe40006ee00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe4200b26d0 tx=0x7fe40c009040 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 msgr2=0x7fe420157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ccd0 tx=0x7fe41400a5c0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 shutdown_connections 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe40006c940 0x7fe40006ee00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe420145c10 0x7fe42015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4200d0f90 0x7fe42015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 --2- 172.21.15.46:0/3191109257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4200cfed0 0x7fe420157820 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 >> 172.21.15.46:0/3191109257 conn(0x7fe42006c350 msgr2=0x7fe4200af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 shutdown_connections 2024-04-26T23:54:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:33.973+0000 7fe426400640 1 -- 172.21.15.46:0/3191109257 wait complete. 2024-04-26T23:54:33.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T23:54:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.195+0000 7fd25a200640 1 Processor -- start 2024-04-26T23:54:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.196+0000 7fd25a200640 1 -- start start 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd25a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2540d1990 0x7fd2540d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd25a200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2540d22b0 con 0x7fd2541639c0 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd25a200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2540cfe50 con 0x7fd2540d0c10 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd25a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd2540d0780 con 0x7fd2540d1990 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd259800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd2541639c0 0x7fd2540b7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38580/0 (socket says 172.21.15.46:38580) 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd259800640 1 -- 172.21.15.46:0/2288547762 learned_addr learned my addr 172.21.15.46:0/2288547762 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:54:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.197+0000 7fd258e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2540d1990 0x7fd2540d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.198+0000 7fd258e00640 1 -- 172.21.15.46:0/2288547762 >> v1:172.21.15.142:6789/0 conn(0x7fd2540d0c10 legacy=0x7fd2540d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.198+0000 7fd258e00640 1 -- 172.21.15.46:0/2288547762 >> v1:172.21.15.46:6789/0 conn(0x7fd2541639c0 legacy=0x7fd2540b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.198+0000 7fd258e00640 1 -- 172.21.15.46:0/2288547762 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2540d0130 con 0x7fd2540d1990 2024-04-26T23:54:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.198+0000 7fd253400640 1 -- 172.21.15.46:0/2288547762 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382095564 0 0) 0x7fd2540d22b0 con 0x7fd2541639c0 2024-04-26T23:54:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.199+0000 7fd258e00640 1 --2- 172.21.15.46:0/2288547762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd2540d1d70 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fd2440097a0 tx=0x7fd24402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5609127a0315241d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.199+0000 7fd253400640 1 -- 172.21.15.46:0/2288547762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244037070 con 0x7fd2540d1990 2024-04-26T23:54:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.199+0000 7fd253400640 1 -- 172.21.15.46:0/2288547762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd244032440 con 0x7fd2540d1990 2024-04-26T23:54:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.199+0000 7fd253400640 1 -- 172.21.15.46:0/2288547762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244032810 con 0x7fd2540d1990 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 -- 172.21.15.46:0/2288547762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 msgr2=0x7fd2540d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 --2- 172.21.15.46:0/2288547762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd2540d1d70 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fd2440097a0 tx=0x7fd24402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 -- 172.21.15.46:0/2288547762 shutdown_connections 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 --2- 172.21.15.46:0/2288547762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd2540d1d70 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 -- 172.21.15.46:0/2288547762 >> 172.21.15.46:0/2288547762 conn(0x7fd25409e2a0 msgr2=0x7fd2540d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 -- 172.21.15.46:0/2288547762 shutdown_connections 2024-04-26T23:54:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.200+0000 7fd25a200640 1 -- 172.21.15.46:0/2288547762 wait complete. 2024-04-26T23:54:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.202+0000 7fd25a200640 1 Processor -- start 2024-04-26T23:54:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.202+0000 7fd25a200640 1 -- start start 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2540d0c10 0x7fd25414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 0x7fd25415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd25415f530 con 0x7fd2540d1990 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540d5da0 con 0x7fd2541639c0 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd25a200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2540cfa90 con 0x7fd2540d0c10 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd253e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd253e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57384/0 (socket says 172.21.15.46:57384) 2024-04-26T23:54:34.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd253e00640 1 -- 172.21.15.46:0/1114265214 learned_addr learned my addr 172.21.15.46:0/1114265214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.203+0000 7fd258e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2540d0c10 0x7fd25414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 -- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2540d0c10 msgr2=0x7fd25414a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd259800640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 0x7fd25415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2540d0c10 0x7fd25414a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 -- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 msgr2=0x7fd25415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 0x7fd25415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 -- 172.21.15.46:0/1114265214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2541480e0 con 0x7fd2540d1990 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd259800640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 0x7fd25415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd253e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c430 tx=0x7fd24000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240011790 con 0x7fd2540d1990 2024-04-26T23:54:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240017070 con 0x7fd2540d1990 2024-04-26T23:54:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25415c590 con 0x7fd2540d1990 2024-04-26T23:54:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240011bb0 con 0x7fd2540d1990 2024-04-26T23:54:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.204+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25415cb50 con 0x7fd2540d1990 2024-04-26T23:54:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.206+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2540693b0 con 0x7fd2540d1990 2024-04-26T23:54:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.206+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd24001c460 con 0x7fd2540d1990 2024-04-26T23:54:34.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.207+0000 7fd250c00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.207+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fd24008bb80 con 0x7fd2540d1990 2024-04-26T23:54:34.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.208+0000 7fd258e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.209+0000 7fd258e00640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd244034000 tx=0x7fd24402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.212+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd240059e20 con 0x7fd2540d1990 2024-04-26T23:54:34.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.457+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd2541482b0 con 0x7fd2540d1990 2024-04-26T23:54:34.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.458+0000 7fd250c00640 1 -- 172.21.15.46:0/1114265214 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd240059720 con 0x7fd2540d1990 2024-04-26T23:54:34.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:54:34.461 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T23:54:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 msgr2=0x7fd23406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd244034000 tx=0x7fd24402e000 comp rx=0 tx=0).stop 2024-04-26T23:54:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 msgr2=0x7fd25414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c430 tx=0x7fd24000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 shutdown_connections 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd23406c920 0x7fd23406ede0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2541639c0 0x7fd25415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2540d1990 0x7fd25414a6d0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 --2- 172.21.15.46:0/1114265214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2540d0c10 0x7fd25414a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.461+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 >> 172.21.15.46:0/1114265214 conn(0x7fd25409e2a0 msgr2=0x7fd2540d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.462+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 shutdown_connections 2024-04-26T23:54:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.462+0000 7fd25a200640 1 -- 172.21.15.46:0/1114265214 wait complete. 2024-04-26T23:54:34.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:34.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start '~mdsdir' recursive 2024-04-26T23:54:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.696+0000 7fbf64e00640 1 Processor -- start 2024-04-26T23:54:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.696+0000 7fbf64e00640 1 -- start start 2024-04-26T23:54:34.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.698+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf60160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.698+0000 7fbf64e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf600b6910 0x7fbf6006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.698+0000 7fbf64e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf601607c0 con 0x7fbf60073b80 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.698+0000 7fbf64e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbf600730e0 con 0x7fbf600b6910 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.698+0000 7fbf64e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6006cc70 con 0x7fbf6006c350 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf600b6910 0x7fbf6006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbf600b6910 0x7fbf6006c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57394/0 (socket says 172.21.15.46:57394) 2024-04-26T23:54:34.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 -- 172.21.15.46:0/940379424 learned_addr learned my addr 172.21.15.46:0/940379424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 -- 172.21.15.46:0/940379424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 msgr2=0x7fbf60160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5de00640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf60160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf60160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 -- 172.21.15.46:0/940379424 >> v1:172.21.15.142:6789/0 conn(0x7fbf60073b80 legacy=0x7fbf6006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5d400640 1 -- 172.21.15.46:0/940379424 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600b6db0 con 0x7fbf600b6910 2024-04-26T23:54:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.699+0000 7fbf5de00640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf60160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.700+0000 7fbf5d400640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf600b6910 0x7fbf6006c730 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a4f1c05766da10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.701+0000 7fbf5ca00640 1 -- 172.21.15.46:0/940379424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48037070 con 0x7fbf600b6910 2024-04-26T23:54:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.701+0000 7fbf5ca00640 1 -- 172.21.15.46:0/940379424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48032440 con 0x7fbf600b6910 2024-04-26T23:54:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.701+0000 7fbf5ca00640 1 -- 172.21.15.46:0/940379424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48032810 con 0x7fbf600b6910 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.701+0000 7fbf64e00640 1 -- 172.21.15.46:0/940379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf600b6910 msgr2=0x7fbf6006c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.701+0000 7fbf64e00640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf600b6910 0x7fbf6006c730 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4802a210 comp rx=0 tx=0).stop 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 -- 172.21.15.46:0/940379424 shutdown_connections 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf600b6910 0x7fbf6006c730 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 --2- 172.21.15.46:0/940379424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf60160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 -- 172.21.15.46:0/940379424 >> 172.21.15.46:0/940379424 conn(0x7fbf600b3bb0 msgr2=0x7fbf600b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 -- 172.21.15.46:0/940379424 shutdown_connections 2024-04-26T23:54:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.702+0000 7fbf64e00640 1 -- 172.21.15.46:0/940379424 wait complete. 2024-04-26T23:54:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.703+0000 7fbf64e00640 1 Processor -- start 2024-04-26T23:54:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 -- start start 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf601374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600b6910 0x7fbf600aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60073660 con 0x7fbf60073b80 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.704+0000 7fbf64e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60073380 con 0x7fbf600b6910 2024-04-26T23:54:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf64e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6015d100 con 0x7fbf6006c350 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57396/0 (socket says 172.21.15.46:57396) 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 -- 172.21.15.46:0/216995801 learned_addr learned my addr 172.21.15.46:0/216995801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 -- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 msgr2=0x7fbf601374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf601374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 -- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600b6910 msgr2=0x7fbf600aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600b6910 0x7fbf600aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 -- 172.21.15.46:0/216995801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf60135250 con 0x7fbf60073b80 2024-04-26T23:54:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.705+0000 7fbf5d400640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.706+0000 7fbf56000640 1 -- 172.21.15.46:0/216995801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48037040 con 0x7fbf60073b80 2024-04-26T23:54:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.706+0000 7fbf56000640 1 -- 172.21.15.46:0/216995801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48042070 con 0x7fbf60073b80 2024-04-26T23:54:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.706+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf600d1390 con 0x7fbf60073b80 2024-04-26T23:54:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.706+0000 7fbf56000640 1 -- 172.21.15.46:0/216995801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48030bd0 con 0x7fbf60073b80 2024-04-26T23:54:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.706+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf600d1670 con 0x7fbf60073b80 2024-04-26T23:54:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.708+0000 7fbf56000640 1 -- 172.21.15.46:0/216995801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf48047460 con 0x7fbf60073b80 2024-04-26T23:54:34.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.709+0000 7fbf56000640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.709+0000 7fbf56000640 1 -- 172.21.15.46:0/216995801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fbf480aca90 con 0x7fbf60073b80 2024-04-26T23:54:34.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.710+0000 7fbf67c2d740 1 Processor -- start 2024-04-26T23:54:34.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.710+0000 7fbf5de00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.710+0000 7fbf67c2d740 1 -- start start 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56382f7f7db0 con 0x56382f8be0d0 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56382f794000 con 0x56382f8db6f0 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56382f7f3d70 con 0x56382f8be810 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf5de00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006fd0 tx=0x7fbf50008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf5e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf5e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57410/0 (socket says 172.21.15.46:57410) 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.711+0000 7fbf5e800640 1 -- 172.21.15.46:0/2169995880 learned_addr learned my addr 172.21.15.46:0/2169995880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5de00640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5d400640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 -- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 msgr2=0x56382f8bdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8bdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 -- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 msgr2=0x56382f88e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 -- 172.21.15.46:0/2169995880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382f5d5a20 con 0x56382f8be0d0 2024-04-26T23:54:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5de00640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.712+0000 7fbf5e800640 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009d30 tx=0x7fbf5800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319b9200f2af6d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf4ea00640 1 -- 172.21.15.46:0/2169995880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800bd40 con 0x56382f8be0d0 2024-04-26T23:54:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf4ea00640 1 -- 172.21.15.46:0/2169995880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58003970 con 0x56382f8be0d0 2024-04-26T23:54:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf4ea00640 1 -- 172.21.15.46:0/2169995880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58003b60 con 0x56382f8be0d0 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2169995880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 msgr2=0x56382f892ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009d30 tx=0x7fbf5800b420 comp rx=0 tx=0).stop 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2169995880 shutdown_connections 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8be0d0 0x56382f892ee0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8bdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2169995880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.713+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2169995880 >> 172.21.15.46:0/2169995880 conn(0x56382f890580 msgr2=0x56382f8d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.714+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2169995880 shutdown_connections 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.714+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2169995880 wait complete. 2024-04-26T23:54:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.714+0000 7fbf67c2d740 1 -- start start 2024-04-26T23:54:34.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.715+0000 7fbf67c2d740 1 Processor -- start 2024-04-26T23:54:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f8bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f8aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56382f8924a0 con 0x56382f8db6f0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56382f77ec50 con 0x56382f8be810 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56382f891890 con 0x56382f8bf2b0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf67c2d740 10 client.0 _ll_register_callbacks cb 0x56382f7eefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57414/0 (socket says 172.21.15.46:57414) 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 -- 172.21.15.46:0/3201511904 learned_addr learned my addr 172.21.15.46:0/3201511904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f8bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 -- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 msgr2=0x56382f8aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f8aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 -- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 msgr2=0x56382f8bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f8bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.716+0000 7fbf5d400640 1 -- 172.21.15.46:0/3201511904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382f4c9310 con 0x56382f8db6f0 2024-04-26T23:54:34.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf5d400640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fbf4807a200 tx=0x7fbf4807acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a0a7b142436551 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf43400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48083170 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fbf4807a760 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf58003970 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fbf4807a960 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf67c2d740 10 client.10983 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x56382f6361b0 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.717+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58003970 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fbf4803a030 con 0x56382f8db6f0 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 10 client.10983 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 10 client.10983 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 20 client.10983 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8abdf0 0x56382f8ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 4 client.10983 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56382f8059c0 con 0x56382f8abdf0 2024-04-26T23:54:34.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.718+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8abdf0 0x56382f8ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.719+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8abdf0 0x56382f8ac280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a8b40b60e74d50 server_cookie=74789b2ee247a84d in_seq=0 out_seq=0 2024-04-26T23:54:34.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.719+0000 7fbf43400640 10 client.10983 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.719+0000 7fbf43400640 1 -- 172.21.15.46:0/3201511904 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56382f8059c0 con 0x56382f8abdf0 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.719+0000 7fbf43400640 10 client.10983 handle_command_reply: tid=0 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.719+0000 7fbf67c2d740 1 client.10983 shutdown 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 msgr2=0x56382f8bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fbf4807a200 tx=0x7fbf4807acc0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 shutdown_connections 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8abdf0 0x56382f8ac280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f8aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8bed70 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3201511904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f8bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf43400640 0 client.10983 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 >> 172.21.15.46:0/3201511904 conn(0x56382f8be0d0 msgr2=0x56382f8dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 shutdown_connections 2024-04-26T23:54:34.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3201511904 wait complete. 2024-04-26T23:54:34.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.720+0000 7fbf67c2d740 20 client.10983 trim_cache size 0 max 16384 2024-04-26T23:54:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.740+0000 7fbf67c2d740 1 Processor -- start 2024-04-26T23:54:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.740+0000 7fbf67c2d740 1 -- start start 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60201fa0 con 0x56382f8bf2b0 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60200fa0 con 0x56382f8db6f0 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf60136b10 con 0x56382f8be810 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57422/0 (socket says 172.21.15.46:57422) 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.741+0000 7fbf5d400640 1 -- 172.21.15.46:0/2479655947 learned_addr learned my addr 172.21.15.46:0/2479655947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5de00640 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5e800640 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 -- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 msgr2=0x56382f8ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 -- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 msgr2=0x56382f88dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 -- 172.21.15.46:0/2479655947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382f891430 con 0x56382f8bf2b0 2024-04-26T23:54:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf5d400640 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fbf48076910 tx=0x7fbf4807d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de7c9f36158ad46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf4e000640 1 -- 172.21.15.46:0/2479655947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48083170 con 0x56382f8bf2b0 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf4e000640 1 -- 172.21.15.46:0/2479655947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48082e70 con 0x56382f8bf2b0 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.742+0000 7fbf4e000640 1 -- 172.21.15.46:0/2479655947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480799a0 con 0x56382f8bf2b0 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2479655947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 msgr2=0x56382f8ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fbf48076910 tx=0x7fbf4807d000 comp rx=0 tx=0).stop 2024-04-26T23:54:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2479655947 shutdown_connections 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8db6f0 0x56382f88dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8bf2b0 0x56382f8ac740 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/2479655947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8be810 0x56382f8ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2479655947 >> 172.21.15.46:0/2479655947 conn(0x56382f8be0d0 msgr2=0x56382f88f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2479655947 shutdown_connections 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.743+0000 7fbf67c2d740 1 -- 172.21.15.46:0/2479655947 wait complete. 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.744+0000 7fbf67c2d740 1 -- start start 2024-04-26T23:54:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.744+0000 7fbf67c2d740 1 Processor -- start 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f9523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f88e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56382f892320 con 0x56382f8db6f0 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf602009a0 con 0x56382f8be810 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56382f8ad3a0 con 0x56382f8bf2b0 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf67c2d740 10 client.0 _ll_register_callbacks cb 0x7fbf480094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf5e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf5e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57428/0 (socket says 172.21.15.46:57428) 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.745+0000 7fbf5e800640 1 -- 172.21.15.46:0/3122670344 learned_addr learned my addr 172.21.15.46:0/3122670344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f9523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 -- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 msgr2=0x56382f88e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f88e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 -- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 msgr2=0x56382f9523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f9523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 -- 172.21.15.46:0/3122670344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382f4a2490 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf5e800640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fbf58014040 tx=0x7fbf58004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f43a457a6bcb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf43400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004040 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fbf580041e0 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.746+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf48082e70 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fbf580043e0 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 10 client.10989 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x56382f8ad0e0 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48082e70 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fbf58011070 con 0x56382f8db6f0 2024-04-26T23:54:34.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 10 client.10989 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 10 client.10989 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 20 client.10989 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8ac740 0x56382f8ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 4 client.10989 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) -- 0x56382f7ff830 con 0x56382f8ac740 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.747+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8ac740 0x56382f8ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.748+0000 7fbf5de00640 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8ac740 0x56382f8ce7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9521ddcff04df2a4 server_cookie=9ae4e50be761c3e5 in_seq=0 out_seq=0 2024-04-26T23:54:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.748+0000 7fbf43400640 10 client.10989 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf43400640 1 -- 172.21.15.46:0/3122670344 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56382f7ff830 con 0x56382f8ac740 2024-04-26T23:54:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf43400640 10 client.10989 handle_command_reply: tid=0 2024-04-26T23:54:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 client.10989 shutdown 2024-04-26T23:54:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 msgr2=0x56382f8ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fbf58014040 tx=0x7fbf58004a70 comp rx=0 tx=0).stop 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 shutdown_connections 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56382f8db6f0 0x56382f8ac200 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x56382f8ac740 0x56382f8ce7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56382f8bf2b0 0x56382f88e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 --2- 172.21.15.46:0/3122670344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382f8be810 0x56382f9523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf43400640 0 client.10989 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 >> 172.21.15.46:0/3122670344 conn(0x56382f8be0d0 msgr2=0x56382f8fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 shutdown_connections 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 1 -- 172.21.15.46:0/3122670344 wait complete. 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.749+0000 7fbf67c2d740 20 client.10989 trim_cache size 0 max 16384 2024-04-26T23:54:34.755 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:54:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:54:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a5a6c899-34f0-4805-b461-46d06e9541ab", 2024-04-26T23:54:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:54:34.756 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:54:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 msgr2=0x7fbf3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006fd0 tx=0x7fbf50008040 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 msgr2=0x7fbf601379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 shutdown_connections 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600b6910 0x7fbf600aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf60073b80 0x7fbf601379f0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 --2- 172.21.15.46:0/216995801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf6006c350 0x7fbf601374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 >> 172.21.15.46:0/216995801 conn(0x7fbf600b3bb0 msgr2=0x7fbf600b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 shutdown_connections 2024-04-26T23:54:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.750+0000 7fbf64e00640 1 -- 172.21.15.46:0/216995801 wait complete. 2024-04-26T23:54:34.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:34.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-26T23:54:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.964+0000 7fd1ffe00640 1 Processor -- start 2024-04-26T23:54:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.965+0000 7fd1ffe00640 1 -- start start 2024-04-26T23:54:34.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.966+0000 7fd1ffe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1f815d730 0x7fd1f80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.966+0000 7fd1ffe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 0x7fd1f815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.966+0000 7fd1ffe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f806eb90 con 0x7fd1f815e050 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.966+0000 7fd1ffe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd1f80b4310 con 0x7fd1f815d730 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.966+0000 7fd1ffe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd1f80b45f0 con 0x7fd1f80b53b0 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1f815d730 0x7fd1f80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1f815d730 0x7fd1f80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57432/0 (socket says 172.21.15.46:57432) 2024-04-26T23:54:34.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 -- 172.21.15.46:0/1226057582 learned_addr learned my addr 172.21.15.46:0/1226057582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fca00640 1 --2- 172.21.15.46:0/1226057582 >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 0x7fd1f815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 -- 172.21.15.46:0/1226057582 >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 msgr2=0x7fd1f815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 --2- 172.21.15.46:0/1226057582 >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 0x7fd1f815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 -- 172.21.15.46:0/1226057582 >> v1:172.21.15.142:6789/0 conn(0x7fd1f815e050 legacy=0x7fd1f806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fd400640 1 -- 172.21.15.46:0/1226057582 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f80b48d0 con 0x7fd1f815d730 2024-04-26T23:54:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.967+0000 7fd1fca00640 1 --2- 172.21.15.46:0/1226057582 >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 0x7fd1f815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.968+0000 7fd1fd400640 1 --2- 172.21.15.46:0/1226057582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f815d730 0x7fd1f80d00d0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd1f806ffa0 tx=0x7fd1ec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f4ff0a99837c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.969+0000 7fd1f7e00640 1 -- 172.21.15.46:0/1226057582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec036070 con 0x7fd1f815d730 2024-04-26T23:54:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.969+0000 7fd1f7e00640 1 -- 172.21.15.46:0/1226057582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec031600 con 0x7fd1f815d730 2024-04-26T23:54:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.969+0000 7fd1f7e00640 1 -- 172.21.15.46:0/1226057582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0319d0 con 0x7fd1f815d730 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.969+0000 7fd1ffe00640 1 -- 172.21.15.46:0/1226057582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f815d730 msgr2=0x7fd1f80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.969+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/1226057582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f815d730 0x7fd1f80d00d0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd1f806ffa0 tx=0x7fd1ec02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 -- 172.21.15.46:0/1226057582 shutdown_connections 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/1226057582 >> v2:172.21.15.142:3300/0 conn(0x7fd1f80b53b0 0x7fd1f815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/1226057582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f815d730 0x7fd1f80d00d0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 -- 172.21.15.46:0/1226057582 >> 172.21.15.46:0/1226057582 conn(0x7fd1f806c980 msgr2=0x7fd1f80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 -- 172.21.15.46:0/1226057582 shutdown_connections 2024-04-26T23:54:34.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.970+0000 7fd1ffe00640 1 -- 172.21.15.46:0/1226057582 wait complete. 2024-04-26T23:54:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.972+0000 7fd1ffe00640 1 Processor -- start 2024-04-26T23:54:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.972+0000 7fd1ffe00640 1 -- start start 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 0x7fd1f813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f815e050 0x7fd1f815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f815e460 con 0x7fd1f80b53b0 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f80b4e90 con 0x7fd1f815d730 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f80b3ec0 con 0x7fd1f815e050 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57444/0 (socket says 172.21.15.46:57444) 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 -- 172.21.15.46:0/2168839356 learned_addr learned my addr 172.21.15.46:0/2168839356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 0x7fd1f813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 -- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f815e050 msgr2=0x7fd1f815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f815e050 0x7fd1f815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 -- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 msgr2=0x7fd1f813f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.973+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 0x7fd1f813f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1fd400640 1 -- 172.21.15.46:0/2168839356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f8140270 con 0x7fd1f80b53b0 2024-04-26T23:54:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 0x7fd1f813f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034040 tx=0x7fd1ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1f5600640 1 -- 172.21.15.46:0/2168839356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec036050 con 0x7fd1f80b53b0 2024-04-26T23:54:34.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1f5600640 1 -- 172.21.15.46:0/2168839356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec042070 con 0x7fd1f80b53b0 2024-04-26T23:54:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1f5600640 1 -- 172.21.15.46:0/2168839356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec009510 con 0x7fd1f80b53b0 2024-04-26T23:54:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f8140550 con 0x7fd1f80b53b0 2024-04-26T23:54:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.974+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f8140b10 con 0x7fd1f80b53b0 2024-04-26T23:54:34.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.977+0000 7fd1f5600640 1 -- 172.21.15.46:0/2168839356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1ec007c50 con 0x7fd1f80b53b0 2024-04-26T23:54:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.978+0000 7fd1f5600640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 0x7fd1dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.978+0000 7fd1f5600640 1 -- 172.21.15.46:0/2168839356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fd1ec0ace20 con 0x7fd1f80b53b0 2024-04-26T23:54:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.979+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 0x7fd1dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.979+0000 7fd202c20740 1 Processor -- start 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.979+0000 7fd202c20740 1 -- start start 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f0e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 0x7fd1dc06edb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064ee4220 0x55e064f2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e064df4030 con 0x55e064f2bde0 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e064e0f7f0 con 0x55e064f0e540 2024-04-26T23:54:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e064dca7e0 con 0x55e064ee4220 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd1fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd1fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57448/0 (socket says 172.21.15.46:57448) 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.980+0000 7fd1fca00640 1 -- 172.21.15.46:0/2550317001 learned_addr learned my addr 172.21.15.46:0/2550317001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f0e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fde00640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064ee4220 0x55e064f2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 -- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064ee4220 msgr2=0x55e064f2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064ee4220 0x55e064f2a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 -- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 msgr2=0x55e064f0e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f0e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 -- 172.21.15.46:0/2550317001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e064dd6050 con 0x55e064f2bde0 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f0e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.981+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800db60 tx=0x7fd1e8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32403477616907e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd1e5c00640 1 -- 172.21.15.46:0/2550317001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8019050 con 0x55e064f2bde0 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd1e5c00640 1 -- 172.21.15.46:0/2550317001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e8023070 con 0x55e064f2bde0 2024-04-26T23:54:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd1e5c00640 1 -- 172.21.15.46:0/2550317001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8012b90 con 0x55e064f2bde0 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 -- 172.21.15.46:0/2550317001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 msgr2=0x55e064f2c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800db60 tx=0x7fd1e8010000 comp rx=0 tx=0).stop 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 -- 172.21.15.46:0/2550317001 shutdown_connections 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064ee4220 0x55e064f2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064f2c330 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 --2- 172.21.15.46:0/2550317001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f0e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.982+0000 7fd202c20740 1 -- 172.21.15.46:0/2550317001 >> 172.21.15.46:0/2550317001 conn(0x55e064ee0b50 msgr2=0x55e064f29350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.983+0000 7fd202c20740 1 -- 172.21.15.46:0/2550317001 shutdown_connections 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.983+0000 7fd202c20740 1 -- 172.21.15.46:0/2550317001 wait complete. 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.983+0000 7fd202c20740 1 -- start start 2024-04-26T23:54:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.984+0000 7fd202c20740 1 Processor -- start 2024-04-26T23:54:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f2bde0 0x55e064f39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e064e4c200 con 0x55e064f10630 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e064e46ee0 con 0x55e064f0e540 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e064e53e70 con 0x55e064f2bde0 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd202c20740 10 client.0 _ll_register_callbacks cb 0x55e064e3d270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57460/0 (socket says 172.21.15.46:57460) 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 -- 172.21.15.46:0/3328077166 learned_addr learned my addr 172.21.15.46:0/3328077166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fd400640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f07210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fca00640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f2bde0 0x55e064f39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 -- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f2bde0 msgr2=0x55e064f39440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f2bde0 0x55e064f39440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 -- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 msgr2=0x55e064f07210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f07210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.985+0000 7fd1fde00640 1 -- 172.21.15.46:0/3328077166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e064c237d0 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1fd400640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f07210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1fde00640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005560 tx=0x7fd1f000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dde2b83b27e34fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1da000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00038d0 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e8023070 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fd1f0003a70 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fd1f0020050 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd202c20740 10 client.11001 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.986+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55e064b593f0 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e8023070 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fd1f0023020 con 0x55e064f10630 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 10 client.11001 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 10 client.11001 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 20 client.11001 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f2cf20 0x55e064f0cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 4 client.11001 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e064e557e0 con 0x55e064f2cf20 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.987+0000 7fd1fd400640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f2cf20 0x55e064f0cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.988+0000 7fd1fd400640 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f2cf20 0x55e064f0cfa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae74232ddbc08a6 server_cookie=9077a9bb7e74626e in_seq=0 out_seq=0 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.988+0000 7fd1da000640 10 client.11001 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.988+0000 7fd1da000640 1 -- 172.21.15.46:0/3328077166 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e064e557e0 con 0x55e064f2cf20 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.988+0000 7fd1da000640 10 client.11001 handle_command_reply: tid=0 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.988+0000 7fd202c20740 1 client.11001 shutdown 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 msgr2=0x55e064f10ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005560 tx=0x7fd1f000e530 comp rx=0 tx=0).stop 2024-04-26T23:54:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 shutdown_connections 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f10630 0x55e064f10ab0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f2cf20 0x55e064f0cfa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f2bde0 0x55e064f39440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 --2- 172.21.15.46:0/3328077166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f07210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd1da000640 0 client.11001 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 >> 172.21.15.46:0/3328077166 conn(0x55e064ee4220 msgr2=0x55e064ee3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 shutdown_connections 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 1 -- 172.21.15.46:0/3328077166 wait complete. 2024-04-26T23:54:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:34.989+0000 7fd202c20740 20 client.11001 trim_cache size 0 max 16384 2024-04-26T23:54:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.009+0000 7fd202c20740 1 Processor -- start 2024-04-26T23:54:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.009+0000 7fd202c20740 1 -- start start 2024-04-26T23:54:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064ee4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064f28290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8201de0 con 0x55e064f2bde0 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8200e20 con 0x55e064f0e540 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f8200280 con 0x55e064f10630 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57466/0 (socket says 172.21.15.46:57466) 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 -- 172.21.15.46:0/918560127 learned_addr learned my addr 172.21.15.46:0/918560127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fd400640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064ee4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fca00640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064f28290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 -- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 msgr2=0x55e064f28290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064f28290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 -- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 msgr2=0x55e064ee4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064ee4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fde00640 1 -- 172.21.15.46:0/918560127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e064f0ec90 con 0x55e064f2bde0 2024-04-26T23:54:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.010+0000 7fd1fd400640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064ee4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:35.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.011+0000 7fd1fde00640 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ea30 tx=0x7fd1f000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cf7450007e513d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.011+0000 7fd1e5200640 1 -- 172.21.15.46:0/918560127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00067d0 con 0x55e064f2bde0 2024-04-26T23:54:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.011+0000 7fd1e5200640 1 -- 172.21.15.46:0/918560127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0018070 con 0x55e064f2bde0 2024-04-26T23:54:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.011+0000 7fd1e5200640 1 -- 172.21.15.46:0/918560127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0006b70 con 0x55e064f2bde0 2024-04-26T23:54:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 -- 172.21.15.46:0/918560127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 msgr2=0x55e064ee3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ea30 tx=0x7fd1f000a000 comp rx=0 tx=0).stop 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 -- 172.21.15.46:0/918560127 shutdown_connections 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3540 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064f28290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 --2- 172.21.15.46:0/918560127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064ee4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 -- 172.21.15.46:0/918560127 >> 172.21.15.46:0/918560127 conn(0x55e064ee4220 msgr2=0x55e064ed9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 -- 172.21.15.46:0/918560127 shutdown_connections 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.012+0000 7fd202c20740 1 -- 172.21.15.46:0/918560127 wait complete. 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.013+0000 7fd202c20740 1 -- start start 2024-04-26T23:54:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.013+0000 7fd202c20740 1 Processor -- start 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f99010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064ee3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e064e54fd0 con 0x55e064f2bde0 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f82007d0 con 0x55e064f0e540 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e064fa2a40 con 0x55e064f10630 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd202c20740 10 client.0 _ll_register_callbacks cb 0x7fd1f0007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57470/0 (socket says 172.21.15.46:57470) 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.014+0000 7fd1fde00640 1 -- 172.21.15.46:0/2017172265 learned_addr learned my addr 172.21.15.46:0/2017172265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f99010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064ee3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 -- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 msgr2=0x55e064ee3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064ee3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 -- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 msgr2=0x55e064f99010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f99010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 -- 172.21.15.46:0/2017172265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e064b190f0 con 0x55e064f2bde0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fca00640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064ee3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1fde00640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ebb0 tx=0x7fd1f0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5513dfafc33716be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1da000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0006ce0 con 0x55e064f2bde0 2024-04-26T23:54:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f0018070 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.015+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 6101+0+0 (secure 0 0 0) 0x7fd1f0007040 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7fd1f003c050 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 10 client.11007 fetch_fsmap learned FSMap version 140 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55e064e0fa30 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0018070 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7fd1f003f020 con 0x55e064f2bde0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 10 client.11007 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 10 client.11007 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 20 client.11007 populate_metadata read hostname 'smithi046' 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f4bd50 0x55e064f046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 4 client.11007 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e064e4f650 con 0x55e064f4bd50 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.016+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f4bd50 0x55e064f046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd1fd400640 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f4bd50 0x55e064f046f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5df6492c30cc12e server_cookie=2d44613713a8df in_seq=0 out_seq=0 2024-04-26T23:54:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd1da000640 10 client.11007 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd1da000640 1 -- 172.21.15.46:0/2017172265 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+257 (crc 0 0 0) 0x55e064e4f650 con 0x55e064f4bd50 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd1da000640 10 client.11007 handle_command_reply: tid=0 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 client.11007 shutdown 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 msgr2=0x55e064ee3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ebb0 tx=0x7fd1f0003760 comp rx=0 tx=0).stop 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 shutdown_connections 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e064f2bde0 0x55e064ee3850 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55e064f4bd50 0x55e064f046f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e064f10630 0x55e064ee3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 --2- 172.21.15.46:0/2017172265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e064f0e540 0x55e064f99010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd1da000640 0 client.11007 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.017+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 >> 172.21.15.46:0/2017172265 conn(0x55e064ee4220 msgr2=0x55e064f3bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 shutdown_connections 2024-04-26T23:54:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd202c20740 1 -- 172.21.15.46:0/2017172265 wait complete. 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd202c20740 20 client.11007 trim_cache size 0 max 16384 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stdout: "a5a6c899-34f0-4805-b461-46d06e9541ab": { 2024-04-26T23:54:35.025 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "~mds0", 2024-04-26T23:54:35.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a5a6c899-34f0-4805-b461-46d06e9541ab", 2024-04-26T23:54:35.026 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-26T23:54:35.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:35.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:54:35.026 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 msgr2=0x7fd1dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 0x7fd1dc06edb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).stop 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 msgr2=0x7fd1f8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034040 tx=0x7fd1ec004a40 comp rx=0 tx=0).stop 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 shutdown_connections 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.018+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1dc06c8f0 0x7fd1dc06edb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f815e050 0x7fd1f815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f815d730 0x7fd1f813f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2168839356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f80b53b0 0x7fd1f8073050 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 >> 172.21.15.46:0/2168839356 conn(0x7fd1f806c980 msgr2=0x7fd1f806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 shutdown_connections 2024-04-26T23:54:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:54:35.019+0000 7fd1ffe00640 1 -- 172.21.15.46:0/2168839356 wait complete. 2024-04-26T23:54:35.039 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a6c899-34f0-4805-b461-46d06e9541ab - {'path': '~mds0', 'tag': 'a5a6c899-34f0-4805-b461-46d06e9541ab', 'options': 'recursive'} 2024-04-26T23:54:43.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:43.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:05.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:05.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-26T23:55:05.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.265+0000 7f85f0600640 1 Processor -- start 2024-04-26T23:55:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.266+0000 7f85f0600640 1 -- start start 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85f0600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f85e8159b10 0x7f85e80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85f0600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85f0600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e815fc30 con 0x7f85e806d8a0 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85f0600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f85e8160840 con 0x7f85e8159b10 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85f0600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85e815ff10 con 0x7f85e80aee20 2024-04-26T23:55:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85ed200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85edc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f85e8159b10 0x7f85e80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85ee600640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f85e806d8a0 0x7f85e8163010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:55596/0 (socket says 172.21.15.46:55596) 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.268+0000 7f85ee600640 1 -- 172.21.15.46:0/831827535 learned_addr learned my addr 172.21.15.46:0/831827535 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.269+0000 7f85edc00640 1 -- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 msgr2=0x7f85e806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.269+0000 7f85edc00640 1 --2- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e806d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.269+0000 7f85edc00640 1 -- 172.21.15.46:0/831827535 >> v1:172.21.15.142:6789/0 conn(0x7f85e806d8a0 legacy=0x7f85e8163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.269+0000 7f85edc00640 1 -- 172.21.15.46:0/831827535 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81601f0 con 0x7f85e8159b10 2024-04-26T23:55:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.271+0000 7f85edc00640 1 --2- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e8159b10 0x7f85e80ae8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c008d4a80762c608 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.271+0000 7f85e7e00640 1 -- 172.21.15.46:0/831827535 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8031600 con 0x7f85e8159b10 2024-04-26T23:55:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.271+0000 7f85e7e00640 1 -- 172.21.15.46:0/831827535 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d80317a0 con 0x7f85e8159b10 2024-04-26T23:55:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.271+0000 7f85e7e00640 1 -- 172.21.15.46:0/831827535 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8031b70 con 0x7f85e8159b10 2024-04-26T23:55:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 -- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e8159b10 msgr2=0x7f85e80ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 --2- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e8159b10 0x7f85e80ae8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d802a5d0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 -- 172.21.15.46:0/831827535 shutdown_connections 2024-04-26T23:55:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 --2- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e806d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 --2- 172.21.15.46:0/831827535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e8159b10 0x7f85e80ae8e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.272+0000 7f85f0600640 1 -- 172.21.15.46:0/831827535 >> 172.21.15.46:0/831827535 conn(0x7f85e80727e0 msgr2=0x7f85e8072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.273+0000 7f85f0600640 1 -- 172.21.15.46:0/831827535 shutdown_connections 2024-04-26T23:55:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.273+0000 7f85f0600640 1 -- 172.21.15.46:0/831827535 wait complete. 2024-04-26T23:55:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.275+0000 7f85f0600640 1 Processor -- start 2024-04-26T23:55:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.275+0000 7f85f0600640 1 -- start start 2024-04-26T23:55:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e806d8a0 0x7f85e814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e8150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8073630 con 0x7f85e8159b10 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80734b0 con 0x7f85e806d8a0 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.276+0000 7f85f0600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85e8161d10 con 0x7f85e80aee20 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45532/0 (socket says 172.21.15.46:45532) 2024-04-26T23:55:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 -- 172.21.15.46:0/2934715035 learned_addr learned my addr 172.21.15.46:0/2934715035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85edc00640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e806d8a0 0x7f85e814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ed200640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e8150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 -- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 msgr2=0x7f85e8150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e8150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 -- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e806d8a0 msgr2=0x7f85e814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e806d8a0 0x7f85e814e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.277+0000 7f85ee600640 1 -- 172.21.15.46:0/2934715035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8156970 con 0x7f85e8159b10 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85ee600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c420 tx=0x7f85e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85e5600640 1 -- 172.21.15.46:0/2934715035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0017070 con 0x7f85e8159b10 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85e5600640 1 -- 172.21.15.46:0/2934715035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e0005ce0 con 0x7f85e8159b10 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e8156c50 con 0x7f85e8159b10 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85e5600640 1 -- 172.21.15.46:0/2934715035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00072d0 con 0x7f85e8159b10 2024-04-26T23:55:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.278+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e8157210 con 0x7f85e8159b10 2024-04-26T23:55:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.280+0000 7f85e5600640 1 -- 172.21.15.46:0/2934715035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85e001a070 con 0x7f85e8159b10 2024-04-26T23:55:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.281+0000 7f85e5600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.282+0000 7f85e5600640 1 -- 172.21.15.46:0/2934715035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f85e008b0d0 con 0x7f85e8159b10 2024-04-26T23:55:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.282+0000 7f85f3401740 1 Processor -- start 2024-04-26T23:55:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.282+0000 7f85edc00640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.282+0000 7f85f3401740 1 -- start start 2024-04-26T23:55:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.283+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded923540 0x559ded923920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.283+0000 7f85f3401740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.283+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded8f9220 0x559ded93f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.283+0000 7f85f3401740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559ded809030 con 0x559ded940de0 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ded8247f0 con 0x559ded8f9220 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559ded7df7e0 con 0x559ded923540 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85edc00640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 0x7f85c806edb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f85d8034000 tx=0x7f85d8033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45546/0 (socket says 172.21.15.46:45546) 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 -- 172.21.15.46:0/4129901299 learned_addr learned my addr 172.21.15.46:0/4129901299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ee600640 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded8f9220 0x559ded93f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 -- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded923540 msgr2=0x559ded923920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded923540 0x559ded923920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 -- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded8f9220 msgr2=0x559ded93f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded8f9220 0x559ded93f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.284+0000 7f85ed200640 1 -- 172.21.15.46:0/4129901299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ded7eb050 con 0x559ded940de0 2024-04-26T23:55:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.285+0000 7f85ed200640 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009b30 tx=0x7f85d4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e62425a5064b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.285+0000 7f85de000640 1 -- 172.21.15.46:0/4129901299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018070 con 0x559ded940de0 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.285+0000 7f85de000640 1 -- 172.21.15.46:0/4129901299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4010dc0 con 0x559ded940de0 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.285+0000 7f85de000640 1 -- 172.21.15.46:0/4129901299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400f040 con 0x559ded940de0 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- 172.21.15.46:0/4129901299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 msgr2=0x559ded941330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009b30 tx=0x7f85d4009190 comp rx=0 tx=0).stop 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- 172.21.15.46:0/4129901299 shutdown_connections 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded8f9220 0x559ded93f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded940de0 0x559ded941330 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 --2- 172.21.15.46:0/4129901299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded923540 0x559ded923920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- 172.21.15.46:0/4129901299 >> 172.21.15.46:0/4129901299 conn(0x559ded8f5b50 msgr2=0x559ded93e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- 172.21.15.46:0/4129901299 shutdown_connections 2024-04-26T23:55:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- 172.21.15.46:0/4129901299 wait complete. 2024-04-26T23:55:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.286+0000 7f85f3401740 1 -- start start 2024-04-26T23:55:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.287+0000 7f85f3401740 1 Processor -- start 2024-04-26T23:55:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded9122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded941cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559ded861200 con 0x559ded9384d0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ded85bee0 con 0x559ded923540 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559ded868e70 con 0x559ded940de0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85f3401740 10 client.0 _ll_register_callbacks cb 0x559ded852270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45550/0 (socket says 172.21.15.46:45550) 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.288+0000 7f85ee600640 1 -- 172.21.15.46:0/806045118 learned_addr learned my addr 172.21.15.46:0/806045118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 -- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 msgr2=0x559ded941cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded941cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 -- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 msgr2=0x559ded9122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded9122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 -- 172.21.15.46:0/806045118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ded6387d0 con 0x559ded9384d0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85ee600640 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f85e00892e0 tx=0x7f85e005d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf19e38b7d04c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0087060 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f85e0058040 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d4010dc0 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.289+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7f85e0071070 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 10 client.11016 fetch_fsmap learned FSMap version 140 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x559ded56e3f0 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4010dc0 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7f85e005f0f0 con 0x559ded9384d0 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 10 client.11016 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 10 client.11016 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 20 client.11016 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded90f7a0 0x559ded90fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 4 client.11016 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ded86a7e0 con 0x559ded90f7a0 2024-04-26T23:55:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.290+0000 7f85edc00640 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded90f7a0 0x559ded90fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.291+0000 7f85edc00640 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded90f7a0 0x559ded90fbc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63eff284ac77968e server_cookie=c2252cdd24b76684 in_seq=0 out_seq=0 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.291+0000 7f85d2a00640 10 client.11016 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85d2a00640 1 -- 172.21.15.46:0/806045118 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ded86a7e0 con 0x559ded90f7a0 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85d2a00640 10 client.11016 handle_command_reply: tid=0 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 client.11016 shutdown 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 msgr2=0x559ded938950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f85e00892e0 tx=0x7f85e005d030 comp rx=0 tx=0).stop 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 shutdown_connections 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded938950 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded90f7a0 0x559ded90fbc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded941cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 --2- 172.21.15.46:0/806045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded9122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85d2a00640 0 client.11016 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 >> 172.21.15.46:0/806045118 conn(0x559ded8f9220 msgr2=0x559ded8f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 shutdown_connections 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.292+0000 7f85f3401740 1 -- 172.21.15.46:0/806045118 wait complete. 2024-04-26T23:55:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.293+0000 7f85f3401740 20 client.11016 trim_cache size 0 max 16384 2024-04-26T23:55:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.312+0000 7f85f3401740 1 Processor -- start 2024-04-26T23:55:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.312+0000 7f85f3401740 1 -- start start 2024-04-26T23:55:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 0x559ded8f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded8f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8202110 con 0x559ded923540 2024-04-26T23:55:05.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8201110 con 0x559ded9384d0 2024-04-26T23:55:05.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.313+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85e8154390 con 0x559ded940de0 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45558/0 (socket says 172.21.15.46:45558) 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 -- 172.21.15.46:0/1098758008 learned_addr learned my addr 172.21.15.46:0/1098758008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85ed200640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 0x559ded8f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85ee600640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded8f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 -- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 msgr2=0x559ded8f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded8f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 -- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 msgr2=0x559ded8f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 0x559ded8f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 -- 172.21.15.46:0/1098758008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ded52e0f0 con 0x559ded923540 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85ed200640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 0x559ded8f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.314+0000 7f85edc00640 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009eb0 tx=0x7f85d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3108469468831a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85dd600640 1 -- 172.21.15.46:0/1098758008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d804de60 con 0x559ded923540 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85dd600640 1 -- 172.21.15.46:0/1098758008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d8053070 con 0x559ded923540 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85dd600640 1 -- 172.21.15.46:0/1098758008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8041040 con 0x559ded923540 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85f3401740 1 -- 172.21.15.46:0/1098758008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 msgr2=0x559ded920e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85f3401740 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009eb0 tx=0x7f85d8004ab0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.315+0000 7f85f3401740 1 -- 172.21.15.46:0/1098758008 shutdown_connections 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded8f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded9384d0 0x559ded8f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 --2- 172.21.15.46:0/1098758008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded923540 0x559ded920e50 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 -- 172.21.15.46:0/1098758008 >> 172.21.15.46:0/1098758008 conn(0x559ded8f9220 msgr2=0x559ded941ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 -- 172.21.15.46:0/1098758008 shutdown_connections 2024-04-26T23:55:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 -- 172.21.15.46:0/1098758008 wait complete. 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.316+0000 7f85f3401740 1 -- start start 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.317+0000 7f85f3401740 1 Processor -- start 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded933f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded926750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559ded869fd0 con 0x559ded9384d0 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8200a60 con 0x559ded923540 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559ded926fa0 con 0x559ded940de0 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85f3401740 10 client.0 _ll_register_callbacks cb 0x7f85e005f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45570/0 (socket says 172.21.15.46:45570) 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 -- 172.21.15.46:0/2395691885 learned_addr learned my addr 172.21.15.46:0/2395691885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85edc00640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded933f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ee600640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded926750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 -- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 msgr2=0x559ded926750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded926750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 -- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 msgr2=0x559ded933f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded933f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.318+0000 7f85ed200640 1 -- 172.21.15.46:0/2395691885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ded824a30 con 0x559ded9384d0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85edc00640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded933f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85ed200640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009250 tx=0x7f85d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536c9584ade761f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018040 con 0x559ded9384d0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f85d401d8a0 con 0x559ded9384d0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d8053070 con 0x559ded9384d0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7f85d401dac0 con 0x559ded9384d0 2024-04-26T23:55:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85f3401740 10 client.11022 fetch_fsmap learned FSMap version 140 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.319+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x559ded959330 con 0x559ded9384d0 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d402f070 con 0x559ded9384d0 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7f85d401dc60 con 0x559ded9384d0 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 10 client.11022 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 10 client.11022 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 20 client.11022 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded927340 0x559ded93f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 4 client.11022 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ded9380d0 con 0x559ded927340 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.320+0000 7f85edc00640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded927340 0x559ded93f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85edc00640 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded927340 0x559ded93f710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95b57ae724902f9 server_cookie=f3eb5c8672655eb7 in_seq=0 out_seq=0 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85d2a00640 10 client.11022 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85d2a00640 1 -- 172.21.15.46:0/2395691885 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ded9380d0 con 0x559ded927340 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85d2a00640 10 client.11022 handle_command_reply: tid=0 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 client.11022 shutdown 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 msgr2=0x559ded926210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009250 tx=0x7f85d40046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 shutdown_connections 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x559ded927340 0x559ded93f710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559ded940de0 0x559ded926750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ded9384d0 0x559ded926210 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85d2a00640 0 client.11022 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 --2- 172.21.15.46:0/2395691885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ded923540 0x559ded933f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.321+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 >> 172.21.15.46:0/2395691885 conn(0x559ded8f9220 msgr2=0x559ded8f8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 shutdown_connections 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f3401740 1 -- 172.21.15.46:0/2395691885 wait complete. 2024-04-26T23:55:05.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f3401740 20 client.11022 trim_cache size 0 max 16384 2024-04-26T23:55:05.329 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:05.329 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:55:05.329 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:55:05.329 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 msgr2=0x7f85c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 0x7f85c806edb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f85d8034000 tx=0x7f85d8033040 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 msgr2=0x7f85e8150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c420 tx=0x7f85e000c8e0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 shutdown_connections 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e8159b10 0x7f85e8150ca0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.322+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e80aee20 0x7f85e8150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.323+0000 7f85f0600640 1 --2- 172.21.15.46:0/2934715035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e806d8a0 0x7f85e814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.323+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 >> 172.21.15.46:0/2934715035 conn(0x7f85e80727e0 msgr2=0x7f85e8157d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.323+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 shutdown_connections 2024-04-26T23:55:05.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.323+0000 7f85f0600640 1 -- 172.21.15.46:0/2934715035 wait complete. 2024-04-26T23:55:05.343 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:55:05.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.12:0 damage ls 2024-04-26T23:55:05.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.542+0000 7f66a0c00640 1 Processor -- start 2024-04-26T23:55:05.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.543+0000 7f66a0c00640 1 -- start start 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f66a0c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f669c162a70 0x7f669c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f66a0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f669c0d20c0 con 0x7f669c0d1cb0 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f66a0c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f669c0d3400 con 0x7f669c0d3830 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f66a0c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f669c0d2370 con 0x7f669c162a70 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f669be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f669c0d1cb0 0x7f669c0b29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53570/0 (socket says 172.21.15.46:53570) 2024-04-26T23:55:05.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.544+0000 7f669be00640 1 -- 172.21.15.46:0/1632587531 learned_addr learned my addr 172.21.15.46:0/1632587531 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606157847 0 0) 0x7f669c0d20c0 con 0x7f669c0d1cb0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6680003520 con 0x7f669c0d1cb0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669b400640 1 --2- 172.21.15.46:0/1632587531 >> v2:172.21.15.142:3300/0 conn(0x7f669c162a70 0x7f669c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3072926447 0 0) 0x7f6680003520 con 0x7f669c0d1cb0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 >> v2:172.21.15.142:3300/0 conn(0x7f669c162a70 msgr2=0x7f669c0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 --2- 172.21.15.46:0/1632587531 >> v2:172.21.15.142:3300/0 conn(0x7f669c162a70 0x7f669c0d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 >> v1:172.21.15.142:6789/0 conn(0x7f669c0d3830 legacy=0x7f669c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c0d2650 con 0x7f669c0d1cb0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f66940019f0 con 0x7f669c0d1cb0 2024-04-26T23:55:05.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 >> v1:172.21.15.46:6789/0 conn(0x7f669c0d1cb0 legacy=0x7f669c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66800039b0 0x7f6680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 0x7f6680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66800051c0 0x7f668000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f669c0d2370 con 0x7f6680004800 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f668000c0f0 con 0x7f66800051c0 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.545+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f668000c2f0 con 0x7f66800039b0 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 0x7f6680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669b400640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66800039b0 0x7f6680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669be00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66800051c0 0x7f668000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 -- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66800039b0 msgr2=0x7f6680003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66800039b0 0x7f6680003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 -- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66800051c0 msgr2=0x7f668000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66800051c0 0x7f668000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669aa00640 1 -- 172.21.15.46:0/1632587531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680003520 con 0x7f6680004800 2024-04-26T23:55:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669be00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66800051c0 0x7f668000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.546+0000 7f669be00640 1 -- 172.21.15.46:0/1632587531 reap_dead start 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.547+0000 7f669aa00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 0x7f6680004c80 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f6688001750 tx=0x7f6680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997d575b01f13a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.547+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688001370 con 0x7f6680004800 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.547+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66880052d0 con 0x7f6680004800 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.547+0000 7f669a000640 1 -- 172.21.15.46:0/1632587531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668802abc0 con 0x7f6680004800 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 -- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 msgr2=0x7f6680004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 0x7f6680004c80 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f6688001750 tx=0x7f6680005830 comp rx=0 tx=0).stop 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 -- 172.21.15.46:0/1632587531 shutdown_connections 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 --2- 172.21.15.46:0/1632587531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6680004800 0x7f6680004c80 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 -- 172.21.15.46:0/1632587531 >> 172.21.15.46:0/1632587531 conn(0x7f669c0d4f20 msgr2=0x7f669c0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 -- 172.21.15.46:0/1632587531 shutdown_connections 2024-04-26T23:55:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.548+0000 7f66a0c00640 1 -- 172.21.15.46:0/1632587531 wait complete. 2024-04-26T23:55:05.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.550+0000 7f66a0c00640 1 Processor -- start 2024-04-26T23:55:05.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.550+0000 7f66a0c00640 1 -- start start 2024-04-26T23:55:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6680004800 0x7f669c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 0x7f669c0ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f669c162fd0 con 0x7f669c0c9700 2024-04-26T23:55:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f669c162e50 con 0x7f6680004800 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.551+0000 7f66a0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f669c0d2e20 con 0x7f669c0b9cd0 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45588/0 (socket says 172.21.15.46:45588) 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 -- 172.21.15.46:0/2278888721 learned_addr learned my addr 172.21.15.46:0/2278888721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669b400640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6680004800 0x7f669c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669be00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 0x7f669c0ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 -- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 msgr2=0x7f669c0ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 0x7f669c0ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 -- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6680004800 msgr2=0x7f669c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6680004800 0x7f669c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669aa00640 1 -- 172.21.15.46:0/2278888721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c1f8cb0 con 0x7f669c0c9700 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.552+0000 7f669be00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 0x7f669c0ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f669aa00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f6688001870 tx=0x7f6688001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f6693400640 1 -- 172.21.15.46:0/2278888721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668803f030 con 0x7f669c0c9700 2024-04-26T23:55:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f6693400640 1 -- 172.21.15.46:0/2278888721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6688005480 con 0x7f669c0c9700 2024-04-26T23:55:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f6693400640 1 -- 172.21.15.46:0/2278888721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688037040 con 0x7f669c0c9700 2024-04-26T23:55:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c1f8f90 con 0x7f669c0c9700 2024-04-26T23:55:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.553+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c0c9c20 con 0x7f669c0c9700 2024-04-26T23:55:05.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.556+0000 7f6693400640 1 -- 172.21.15.46:0/2278888721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6688042070 con 0x7f669c0c9700 2024-04-26T23:55:05.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.556+0000 7f6693400640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 0x7f667c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.557+0000 7f6693400640 1 -- 172.21.15.46:0/2278888721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f6688007640 con 0x7f669c0c9700 2024-04-26T23:55:05.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.557+0000 7f66a519b740 1 Processor -- start 2024-04-26T23:55:05.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.557+0000 7f669b400640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 0x7f667c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.557+0000 7f66a519b740 1 -- start start 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8953cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b89541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b888cc70 con 0x55c3b8924100 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8845980 con 0x55c3b89514a0 2024-04-26T23:55:05.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b8888c30 con 0x55c3b8951880 2024-04-26T23:55:05.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.558+0000 7f669be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669b400640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 0x7f667c06ede0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f669c051fc0 tx=0x7f668c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45594/0 (socket says 172.21.15.46:45594) 2024-04-26T23:55:05.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 -- 172.21.15.46:0/1623390427 learned_addr learned my addr 172.21.15.46:0/1623390427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669b400640 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8953cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669aa00640 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b89541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 -- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 msgr2=0x55c3b89541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b89541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 -- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 msgr2=0x55c3b8953cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8953cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 -- 172.21.15.46:0/1623390427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b881d7b0 con 0x55c3b8924100 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.559+0000 7f669be00640 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f6694000f80 tx=0x7f669400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6180f5922cf25e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f6687400640 1 -- 172.21.15.46:0/1623390427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018070 con 0x55c3b8924100 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f6687400640 1 -- 172.21.15.46:0/1623390427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694007110 con 0x55c3b8924100 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f6687400640 1 -- 172.21.15.46:0/1623390427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940074c0 con 0x55c3b8924100 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 -- 172.21.15.46:0/1623390427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 msgr2=0x55c3b8927910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f6694000f80 tx=0x7f669400e650 comp rx=0 tx=0).stop 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 -- 172.21.15.46:0/1623390427 shutdown_connections 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8924100 0x55c3b8927910 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b89541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 --2- 172.21.15.46:0/1623390427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8953cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 -- 172.21.15.46:0/1623390427 >> 172.21.15.46:0/1623390427 conn(0x55c3b8926370 msgr2=0x55c3b896ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.560+0000 7f66a519b740 1 -- 172.21.15.46:0/1623390427 shutdown_connections 2024-04-26T23:55:05.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.561+0000 7f66a519b740 1 -- 172.21.15.46:0/1623390427 wait complete. 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.561+0000 7f66a519b740 1 -- start start 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.561+0000 7f66a519b740 1 Processor -- start 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.562+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8971b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b894fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b888bf80 con 0x55c3b89720a0 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8839260 con 0x55c3b89514a0 2024-04-26T23:55:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b88912a0 con 0x55c3b8951880 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f66a519b740 10 client.0 _ll_register_callbacks cb 0x55c3b8883960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45604/0 (socket says 172.21.15.46:45604) 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 -- 172.21.15.46:0/2952471337 learned_addr learned my addr 172.21.15.46:0/2952471337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669b400640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8971b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669aa00640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b894fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 -- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 msgr2=0x55c3b894fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b894fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 -- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 msgr2=0x55c3b8971b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8971b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.563+0000 7f669be00640 1 -- 172.21.15.46:0/2952471337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b881b1c0 con 0x55c3b89720a0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f669be00640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f6694009010 tx=0x7f6694001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b4bf6b6635b99a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f667be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018040 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f6694013450 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694007110 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7f669402a050 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f66a519b740 10 client.11034 fetch_fsmap learned FSMap version 140 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.564+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55c3b859e3f0 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694007110 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7f669402d020 con 0x55c3b89720a0 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 10 client.11034 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 10 client.11034 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 20 client.11034 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b896a280 0x55c3b896a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 4 client.11034 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3b889a880 con 0x55c3b896a280 2024-04-26T23:55:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.565+0000 7f669b400640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b896a280 0x55c3b896a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.566+0000 7f669b400640 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b896a280 0x55c3b896a6b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7451354b2de8cb1 server_cookie=c5120bf033569e02 in_seq=0 out_seq=0 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.566+0000 7f667be00640 10 client.11034 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f667be00640 1 -- 172.21.15.46:0/2952471337 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3b889a880 con 0x55c3b896a280 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f667be00640 10 client.11034 handle_command_reply: tid=0 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 client.11034 shutdown 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 msgr2=0x55c3b8950060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f6694009010 tx=0x7f6694001dc0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 shutdown_connections 2024-04-26T23:55:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89720a0 0x55c3b8950060 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b896a280 0x55c3b896a6b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f667be00640 0 client.11034 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b894fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 --2- 172.21.15.46:0/2952471337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89514a0 0x55c3b8971b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 >> 172.21.15.46:0/2952471337 conn(0x55c3b8924100 msgr2=0x55c3b8928cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 shutdown_connections 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 1 -- 172.21.15.46:0/2952471337 wait complete. 2024-04-26T23:55:05.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.567+0000 7f66a519b740 20 client.11034 trim_cache size 0 max 16384 2024-04-26T23:55:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.582+0000 7f66a519b740 1 Processor -- start 2024-04-26T23:55:05.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.582+0000 7f66a519b740 1 -- start start 2024-04-26T23:55:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b8922f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8923460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f669c2029b0 con 0x55c3b89514a0 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f669c201a20 con 0x55c3b89720a0 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.583+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f669c0cb470 con 0x55c3b8951880 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45620/0 (socket says 172.21.15.46:45620) 2024-04-26T23:55:05.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 -- 172.21.15.46:0/2860133829 learned_addr learned my addr 172.21.15.46:0/2860133829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669aa00640 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b8922f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669be00640 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8923460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 -- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 msgr2=0x55c3b8922f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b8922f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 -- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 msgr2=0x55c3b8923460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8923460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 -- 172.21.15.46:0/2860133829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b86687d0 con 0x55c3b89514a0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.584+0000 7f669b400640 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f669c08d2a0 tx=0x7f668c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87238b7f40aaa5df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f6686a00640 1 -- 172.21.15.46:0/2860133829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c030070 con 0x55c3b89514a0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f6686a00640 1 -- 172.21.15.46:0/2860133829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c01d040 con 0x55c3b89514a0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f6686a00640 1 -- 172.21.15.46:0/2860133829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c011c80 con 0x55c3b89514a0 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 -- 172.21.15.46:0/2860133829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 msgr2=0x55c3b894ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f669c08d2a0 tx=0x7f668c014000 comp rx=0 tx=0).stop 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 -- 172.21.15.46:0/2860133829 shutdown_connections 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8923460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b8951880 0x55c3b8922f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 --2- 172.21.15.46:0/2860133829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b89514a0 0x55c3b894ba60 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.585+0000 7f66a519b740 1 -- 172.21.15.46:0/2860133829 >> 172.21.15.46:0/2860133829 conn(0x55c3b8924100 msgr2=0x55c3b8929920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.586+0000 7f66a519b740 1 -- 172.21.15.46:0/2860133829 shutdown_connections 2024-04-26T23:55:05.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.586+0000 7f66a519b740 1 -- 172.21.15.46:0/2860133829 wait complete. 2024-04-26T23:55:05.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.586+0000 7f66a519b740 1 -- start start 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.586+0000 7f66a519b740 1 Processor -- start 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b89514a0 0x55c3b894dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8949220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f669c201350 con 0x55c3b8951880 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8945f40 con 0x55c3b89720a0 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c3b894e8e0 con 0x55c3b89514a0 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.587+0000 7f66a519b740 10 client.0 _ll_register_callbacks cb 0x7f6694016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45634/0 (socket says 172.21.15.46:45634) 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 -- 172.21.15.46:0/2776210854 learned_addr learned my addr 172.21.15.46:0/2776210854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 -- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b89514a0 msgr2=0x55c3b894dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b89514a0 0x55c3b894dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 -- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 msgr2=0x55c3b8949220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8949220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 -- 172.21.15.46:0/2776210854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b88332a0 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f669aa00640 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f6688001840 tx=0x7f6688079b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf8a8f97e5c291 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.588+0000 7f667be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688091030 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f6688081ad0 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c01d040 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7f6688081f60 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 10 client.11040 fetch_fsmap learned FSMap version 140 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) -- 0x55c3b8945c80 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c01d040 con 0x55c3b8951880 2024-04-26T23:55:05.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 140) ==== 3494+0+0 (secure 0 0 0) 0x7f668804c060 con 0x55c3b8951880 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 10 client.11040 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 10 client.11040 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 20 client.11040 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b89e5480 0x55c3b896b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 4 client.11040 mds_command: new command op to 10770 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.589+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 --> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3b896f390 con 0x55c3b89e5480 2024-04-26T23:55:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.590+0000 7f669b400640 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b89e5480 0x55c3b896b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.590+0000 7f669b400640 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b89e5480 0x55c3b896b1d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf43038994317545 server_cookie=2d60119046c27577 in_seq=0 out_seq=0 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.590+0000 7f667be00640 10 client.11040 ms_handle_connect on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f667be00640 1 -- 172.21.15.46:0/2776210854 <== mds.0 v2:172.21.15.46:6834/7977112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3b896f390 con 0x55c3b89e5480 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f667be00640 10 client.11040 handle_command_reply: tid=0 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 client.11040 shutdown 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 msgr2=0x55c3b8948ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f6688001840 tx=0x7f6688079b40 comp rx=0 tx=0).stop 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 shutdown_connections 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b89720a0 0x55c3b8949220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8951880 0x55c3b8948ce0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.46:6834/7977112,v1:172.21.15.46:6835/7977112] conn(0x55c3b89e5480 0x55c3b896b1d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 --2- 172.21.15.46:0/2776210854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c3b89514a0 0x55c3b894dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 >> 172.21.15.46:0/2776210854 conn(0x55c3b8924100 msgr2=0x55c3b8970cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f667be00640 0 client.11040 ms_handle_reset on v2:172.21.15.46:6834/7977112 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 shutdown_connections 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 1 -- 172.21.15.46:0/2776210854 wait complete. 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.591+0000 7f66a519b740 20 client.11040 trim_cache size 0 max 16384 2024-04-26T23:55:05.597 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 msgr2=0x7f667c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 0x7f667c06ede0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f669c051fc0 tx=0x7f668c008040 comp rx=0 tx=0).stop 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 msgr2=0x7f669c1f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f6688001870 tx=0x7f6688001120 comp rx=0 tx=0).stop 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 shutdown_connections 2024-04-26T23:55:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f667c06c920 0x7f667c06ede0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669c0b9cd0 0x7f669c0ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669c0c9700 0x7f669c1f8380 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 --2- 172.21.15.46:0/2278888721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6680004800 0x7f669c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 >> 172.21.15.46:0/2278888721 conn(0x7f669c0d4f20 msgr2=0x7f669c160b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 shutdown_connections 2024-04-26T23:55:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.592+0000 7f66a0c00640 1 -- 172.21.15.46:0/2278888721 wait complete. 2024-04-26T23:55:05.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T23:55:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.799+0000 7fdd5dc00640 1 Processor -- start 2024-04-26T23:55:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.799+0000 7fdd5dc00640 1 -- start start 2024-04-26T23:55:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.800+0000 7fdd5dc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdd580d0c10 0x7fdd580d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.800+0000 7fdd5dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.800+0000 7fdd5dc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd580cfe50 con 0x7fdd580d1990 2024-04-26T23:55:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.800+0000 7fdd5dc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdd580d0780 con 0x7fdd580d0c10 2024-04-26T23:55:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.800+0000 7fdd5dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdd580d0130 con 0x7fdd581639c0 2024-04-26T23:55:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.801+0000 7fdd5d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.801+0000 7fdd5d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:36770/0 (socket says 172.21.15.46:36770) 2024-04-26T23:55:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.801+0000 7fdd5d200640 1 -- 172.21.15.46:0/1809153953 learned_addr learned my addr 172.21.15.46:0/1809153953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.801+0000 7fdd57400640 1 --2- 172.21.15.46:0/1809153953 >> v2:172.21.15.142:3300/0 conn(0x7fdd580d0c10 0x7fdd580d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.802+0000 7fdd5d200640 1 -- 172.21.15.46:0/1809153953 >> v2:172.21.15.142:3300/0 conn(0x7fdd580d0c10 msgr2=0x7fdd580d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.802+0000 7fdd5d200640 1 --2- 172.21.15.46:0/1809153953 >> v2:172.21.15.142:3300/0 conn(0x7fdd580d0c10 0x7fdd580d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.802+0000 7fdd56a00640 1 -- 172.21.15.46:0/1809153953 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1696882296 0 0) 0x7fdd580cfe50 con 0x7fdd580d1990 2024-04-26T23:55:05.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.802+0000 7fdd5d200640 1 -- 172.21.15.46:0/1809153953 >> v1:172.21.15.142:6789/0 conn(0x7fdd580d1990 legacy=0x7fdd580d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.802+0000 7fdd5d200640 1 -- 172.21.15.46:0/1809153953 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd580d0410 con 0x7fdd581639c0 2024-04-26T23:55:05.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.804+0000 7fdd5d200640 1 --2- 172.21.15.46:0/1809153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009e50 tx=0x7fdd5002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260ab3d5f91f282e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.804+0000 7fdd56a00640 1 -- 172.21.15.46:0/1809153953 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50036070 con 0x7fdd581639c0 2024-04-26T23:55:05.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.804+0000 7fdd56a00640 1 -- 172.21.15.46:0/1809153953 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd50031600 con 0x7fdd581639c0 2024-04-26T23:55:05.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.804+0000 7fdd56a00640 1 -- 172.21.15.46:0/1809153953 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd500319d0 con 0x7fdd581639c0 2024-04-26T23:55:05.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 -- 172.21.15.46:0/1809153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 msgr2=0x7fdd580b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/1809153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009e50 tx=0x7fdd5002a710 comp rx=0 tx=0).stop 2024-04-26T23:55:05.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 -- 172.21.15.46:0/1809153953 shutdown_connections 2024-04-26T23:55:05.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/1809153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580b7dd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/1809153953 >> v2:172.21.15.142:3300/0 conn(0x7fdd580d0c10 0x7fdd580d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 -- 172.21.15.46:0/1809153953 >> 172.21.15.46:0/1809153953 conn(0x7fdd5809e2a0 msgr2=0x7fdd580d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:05.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 -- 172.21.15.46:0/1809153953 shutdown_connections 2024-04-26T23:55:05.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.805+0000 7fdd5dc00640 1 -- 172.21.15.46:0/1809153953 wait complete. 2024-04-26T23:55:05.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.807+0000 7fdd5dc00640 1 Processor -- start 2024-04-26T23:55:05.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.807+0000 7fdd5dc00640 1 -- start start 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 0x7fdd5815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580cfc10 con 0x7fdd580d1990 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580cfa90 con 0x7fdd580d0c10 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd5dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdd580d5bd0 con 0x7fdd581639c0 2024-04-26T23:55:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45636/0 (socket says 172.21.15.46:45636) 2024-04-26T23:55:05.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.808+0000 7fdd57400640 1 -- 172.21.15.46:0/2197629684 learned_addr learned my addr 172.21.15.46:0/2197629684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 0x7fdd5815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 -- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 msgr2=0x7fdd580d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd5d200640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 -- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 msgr2=0x7fdd5815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 0x7fdd5815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:05.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 -- 172.21.15.46:0/2197629684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5809fbd0 con 0x7fdd580d1990 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 0x7fdd5815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd57400640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e9e0 tx=0x7fdd4400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44017070 con 0x7fdd580d1990 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4400c9d0 con 0x7fdd580d1990 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.809+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400cdc0 con 0x7fdd580d1990 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.810+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd580a1fe0 con 0x7fdd580d1990 2024-04-26T23:55:05.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.810+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd580a25a0 con 0x7fdd580d1990 2024-04-26T23:55:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.811+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580693b0 con 0x7fdd580d1990 2024-04-26T23:55:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.812+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd4401a070 con 0x7fdd580d1990 2024-04-26T23:55:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.813+0000 7fdd4fe00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 0x7fdd3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.813+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7fdd4408c120 con 0x7fdd580d1990 2024-04-26T23:55:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.814+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 0x7fdd3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:05.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.815+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 0x7fdd3806edb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:05.818+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd4405a320 con 0x7fdd580d1990 2024-04-26T23:55:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.066+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd580d1770 con 0x7fdd580d1990 2024-04-26T23:55:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.067+0000 7fdd4fe00640 1 -- 172.21.15.46:0/2197629684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdd44059c20 con 0x7fdd580d1990 2024-04-26T23:55:06.070 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:06.070 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 msgr2=0x7fdd3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 0x7fdd3806edb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48009040 comp rx=0 tx=0).stop 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 msgr2=0x7fdd5809f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e9e0 tx=0x7fdd4400eea0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 shutdown_connections 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd3806c8f0 0x7fdd3806edb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd581639c0 0x7fdd580d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd580d1990 0x7fdd5809f690 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 --2- 172.21.15.46:0/2197629684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd580d0c10 0x7fdd5815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 >> 172.21.15.46:0/2197629684 conn(0x7fdd5809e2a0 msgr2=0x7fdd580b9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 shutdown_connections 2024-04-26T23:55:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.070+0000 7fdd5dc00640 1 -- 172.21.15.46:0/2197629684 wait complete. 2024-04-26T23:55:06.092 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:55:06.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:55:06.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:55:06.322 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:55:06.322 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:55:06.322 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:06.323 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:06.383 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:06.383 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:06.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.656+0000 7faf5ea00640 1 Processor -- start 2024-04-26T23:55:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.656+0000 7faf5ea00640 1 -- start start 2024-04-26T23:55:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf5815edf0 0x7faf580cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf580c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf580b03c0 con 0x7faf5815f1d0 2024-04-26T23:55:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faf580cf5c0 con 0x7faf5815edf0 2024-04-26T23:55:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf580cb680 con 0x7faf580cd770 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf5815edf0 0x7faf580cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faf5815f1d0 0x7faf580cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53572/0 (socket says 172.21.15.46:53572) 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5cc00640 1 -- 172.21.15.46:0/1351592423 learned_addr learned my addr 172.21.15.46:0/1351592423 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5e000640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf580c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf57e00640 1 -- 172.21.15.46:0/1351592423 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151279916 0 0) 0x7faf580b03c0 con 0x7faf5815f1d0 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5d600640 1 -- 172.21.15.46:0/1351592423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 msgr2=0x7faf580c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5d600640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf580c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5d600640 1 -- 172.21.15.46:0/1351592423 >> v1:172.21.15.46:6789/0 conn(0x7faf5815f1d0 legacy=0x7faf580cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.658+0000 7faf5d600640 1 -- 172.21.15.46:0/1351592423 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf580cb960 con 0x7faf5815edf0 2024-04-26T23:55:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.659+0000 7faf5d600640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf580cf080 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0097a0 tx=0x7faf4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e61eccb44b9fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.659+0000 7faf57e00640 1 -- 172.21.15.46:0/1351592423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c037070 con 0x7faf5815edf0 2024-04-26T23:55:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.659+0000 7faf57e00640 1 -- 172.21.15.46:0/1351592423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4c032440 con 0x7faf5815edf0 2024-04-26T23:55:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.659+0000 7faf57e00640 1 -- 172.21.15.46:0/1351592423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c032810 con 0x7faf5815edf0 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 -- 172.21.15.46:0/1351592423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 msgr2=0x7faf580cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf580cf080 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0097a0 tx=0x7faf4c02a210 comp rx=0 tx=0).stop 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 -- 172.21.15.46:0/1351592423 shutdown_connections 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf580c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 --2- 172.21.15.46:0/1351592423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf580cf080 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 -- 172.21.15.46:0/1351592423 >> 172.21.15.46:0/1351592423 conn(0x7faf580b26d0 msgr2=0x7faf580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 -- 172.21.15.46:0/1351592423 shutdown_connections 2024-04-26T23:55:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.660+0000 7faf5ea00640 1 -- 172.21.15.46:0/1351592423 wait complete. 2024-04-26T23:55:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.662+0000 7faf5ea00640 1 Processor -- start 2024-04-26T23:55:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.662+0000 7faf5ea00640 1 -- start start 2024-04-26T23:55:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf58151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 0x7faf5815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf580cc200 con 0x7faf5815edf0 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf580cbf20 con 0x7faf5806f3f0 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faf580cb1a0 con 0x7faf580cd770 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45662/0 (socket says 172.21.15.46:45662) 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.663+0000 7faf5cc00640 1 -- 172.21.15.46:0/3080375146 learned_addr learned my addr 172.21.15.46:0/3080375146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5d600640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf58151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5e000640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 0x7faf5815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 -- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 msgr2=0x7faf58151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf58151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 -- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 msgr2=0x7faf5815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 0x7faf5815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 -- 172.21.15.46:0/3080375146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5815d2f0 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5d600640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf58151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5e000640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 0x7faf5815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.664+0000 7faf5cc00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c470 tx=0x7faf4400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.665+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44017070 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.665+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44005ce0 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.665+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5815d5d0 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.665+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44007270 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.665+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf581516c0 con 0x7faf5815edf0 2024-04-26T23:55:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.666+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5806f800 con 0x7faf5815edf0 2024-04-26T23:55:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.668+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf4401a070 con 0x7faf5815edf0 2024-04-26T23:55:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.670+0000 7faf55600640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 0x7faf3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.670+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7faf4408af30 con 0x7faf5815edf0 2024-04-26T23:55:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.670+0000 7faf5d600640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 0x7faf3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.670+0000 7faf5d600640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 0x7faf3806edb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7faf4c034000 tx=0x7faf4c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.674+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf5806f800 con 0x7faf5815edf0 2024-04-26T23:55:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.885+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf580693b0 con 0x7faf5815edf0 2024-04-26T23:55:06.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.888+0000 7faf55600640 1 -- 172.21.15.46:0/3080375146 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+17089 (secure 0 0 0) 0x7faf44058b80 con 0x7faf5815edf0 2024-04-26T23:55:06.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:06.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":175,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:58.408057+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.21","mappings":[{"from":1,"to":0}]},{"pgid":"26.22","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:06.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 msgr2=0x7faf3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 0x7faf3806edb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7faf4c034000 tx=0x7faf4c02e000 comp rx=0 tx=0).stop 2024-04-26T23:55:06.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 msgr2=0x7faf5806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c470 tx=0x7faf4400c930 comp rx=0 tx=0).stop 2024-04-26T23:55:06.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 shutdown_connections 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faf3806c8f0 0x7faf3806edb0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf5806f3f0 0x7faf5815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5815edf0 0x7faf5806eeb0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 --2- 172.21.15.46:0/3080375146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf580cd770 0x7faf58151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 >> 172.21.15.46:0/3080375146 conn(0x7faf580b26d0 msgr2=0x7faf580af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 shutdown_connections 2024-04-26T23:55:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:06.891+0000 7faf5ea00640 1 -- 172.21.15.46:0/3080375146 wait complete. 2024-04-26T23:55:06.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:54:22.516305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:54:23.663985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:55:06.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:55:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.128+0000 7f73b8a00640 1 Processor -- start 2024-04-26T23:55:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.128+0000 7f73b8a00640 1 -- start start 2024-04-26T23:55:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.130+0000 7f73b8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 0x7f73b40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.130+0000 7f73b8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b40ce7d0 con 0x7f73b4073b30 2024-04-26T23:55:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.130+0000 7f73b8a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b40ceab0 con 0x7f73b415cec0 2024-04-26T23:55:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.130+0000 7f73b8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f73b40ced90 con 0x7f73b40cdd00 2024-04-26T23:55:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73b3200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f73b4073b30 0x7f73b40b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53588/0 (socket says 172.21.15.46:53588) 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73b3200640 1 -- 172.21.15.46:0/1818899777 learned_addr learned my addr 172.21.15.46:0/1818899777 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73a9e00640 1 --2- 172.21.15.46:0/1818899777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 0x7f73b40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73b1e00640 1 -- 172.21.15.46:0/1818899777 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888210334 0 0) 0x7f73b40ce7d0 con 0x7f73b4073b30 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73b1e00640 1 -- 172.21.15.46:0/1818899777 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7398003520 con 0x7f73b4073b30 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73a9e00640 1 -- 172.21.15.46:0/1818899777 >> v1:172.21.15.142:6789/0 conn(0x7f73b415cec0 legacy=0x7f73b40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73a9e00640 1 -- 172.21.15.46:0/1818899777 >> v1:172.21.15.46:6789/0 conn(0x7f73b4073b30 legacy=0x7f73b40b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.131+0000 7f73a9e00640 1 -- 172.21.15.46:0/1818899777 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b40cf070 con 0x7f73b40cdd00 2024-04-26T23:55:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.133+0000 7f73a9e00640 1 --2- 172.21.15.46:0/1818899777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 0x7f73b40735f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f739c0097d0 tx=0x7f739c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86a3f3ced1900f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.133+0000 7f73b1e00640 1 -- 172.21.15.46:0/1818899777 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c037070 con 0x7f73b40cdd00 2024-04-26T23:55:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.133+0000 7f73b1e00640 1 -- 172.21.15.46:0/1818899777 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739c032440 con 0x7f73b40cdd00 2024-04-26T23:55:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.133+0000 7f73b1e00640 1 -- 172.21.15.46:0/1818899777 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c032810 con 0x7f73b40cdd00 2024-04-26T23:55:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 -- 172.21.15.46:0/1818899777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 msgr2=0x7f73b40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 --2- 172.21.15.46:0/1818899777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 0x7f73b40735f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f739c0097d0 tx=0x7f739c02a210 comp rx=0 tx=0).stop 2024-04-26T23:55:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 -- 172.21.15.46:0/1818899777 shutdown_connections 2024-04-26T23:55:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 --2- 172.21.15.46:0/1818899777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b40cdd00 0x7f73b40735f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 -- 172.21.15.46:0/1818899777 >> 172.21.15.46:0/1818899777 conn(0x7f73b40b27b0 msgr2=0x7f73b40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 -- 172.21.15.46:0/1818899777 shutdown_connections 2024-04-26T23:55:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.134+0000 7f73b8a00640 1 -- 172.21.15.46:0/1818899777 wait complete. 2024-04-26T23:55:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.136+0000 7f73b8a00640 1 Processor -- start 2024-04-26T23:55:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.136+0000 7f73b8a00640 1 -- start start 2024-04-26T23:55:07.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b4073b30 0x7f73b40c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b40cdd00 0x7f73b40c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4073f70 con 0x7f73b415cec0 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73b415d2a0 con 0x7f73b40cdd00 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.137+0000 7f73b8a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f73b40b4250 con 0x7f73b4073b30 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45664/0 (socket says 172.21.15.46:45664) 2024-04-26T23:55:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 -- 172.21.15.46:0/992319074 learned_addr learned my addr 172.21.15.46:0/992319074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b2800640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b4073b30 0x7f73b40c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 -- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b4073b30 msgr2=0x7f73b40c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b4073b30 0x7f73b40c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 -- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b40cdd00 msgr2=0x7f73b40c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b40cdd00 0x7f73b40c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.138+0000 7f73b3200640 1 -- 172.21.15.46:0/992319074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b4159970 con 0x7f73b415cec0 2024-04-26T23:55:07.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73b3200640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00cad0 tx=0x7f73ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac017070 con 0x7f73b415cec0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac00ac30 con 0x7f73b415cec0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac012400 con 0x7f73b415cec0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b41f61f0 con 0x7f73b415cec0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.139+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b40ca180 con 0x7f73b415cec0 2024-04-26T23:55:07.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.141+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b40693b0 con 0x7f73b415cec0 2024-04-26T23:55:07.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.143+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73ac01a070 con 0x7f73b415cec0 2024-04-26T23:55:07.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.143+0000 7f73ab400640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 0x7f739006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.144+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f73ac08bb90 con 0x7f73b415cec0 2024-04-26T23:55:07.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.144+0000 7f73b2800640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 0x7f739006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.145+0000 7f73b2800640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 0x7f739006ee00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fd0 tx=0x7f73a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.148+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73ac059d80 con 0x7f73b415cec0 2024-04-26T23:55:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.354+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73b406e0d0 con 0x7f73b415cec0 2024-04-26T23:55:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.355+0000 7f73ab400640 1 -- 172.21.15.46:0/992319074 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) ==== 93+0+10 (secure 0 0 0) 0x7f73ac059680 con 0x7f73b415cec0 2024-04-26T23:55:07.359 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 msgr2=0x7f739006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 0x7f739006ee00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fd0 tx=0x7f73a4009040 comp rx=0 tx=0).stop 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 msgr2=0x7f73b4159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00cad0 tx=0x7f73ac00cf90 comp rx=0 tx=0).stop 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 shutdown_connections 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f739006c940 0x7f739006ee00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b415cec0 0x7f73b4159430 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b40cdd00 0x7f73b40c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 --2- 172.21.15.46:0/992319074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73b4073b30 0x7f73b40c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 >> 172.21.15.46:0/992319074 conn(0x7f73b40b27b0 msgr2=0x7f73b4071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 shutdown_connections 2024-04-26T23:55:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.358+0000 7f73b8a00640 1 -- 172.21.15.46:0/992319074 wait complete. 2024-04-26T23:55:07.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:55:07.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.606+0000 7fc412400640 1 Processor -- start 2024-04-26T23:55:07.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.607+0000 7fc412400640 1 -- start start 2024-04-26T23:55:07.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc412400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc40c1626c0 0x7fc40c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc412400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0d07d0 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc412400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0d0ab0 con 0x7fc40c162aa0 2024-04-26T23:55:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc412400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc40c0d0d90 con 0x7fc40c1626c0 2024-04-26T23:55:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc40be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc40c0b5fb0 0x7fc40c0d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53592/0 (socket says 172.21.15.46:53592) 2024-04-26T23:55:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc40be00640 1 -- 172.21.15.46:0/922354467 learned_addr learned my addr 172.21.15.46:0/922354467 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc411000640 1 --2- 172.21.15.46:0/922354467 >> v2:172.21.15.142:3300/0 conn(0x7fc40c1626c0 0x7fc40c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942449668 0 0) 0x7fc40c0d07d0 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.608+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3f0003520 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2457924854 0 0) 0x7fc3f0003520 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 >> v2:172.21.15.142:3300/0 conn(0x7fc40c1626c0 msgr2=0x7fc40c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 --2- 172.21.15.46:0/922354467 >> v2:172.21.15.142:3300/0 conn(0x7fc40c1626c0 0x7fc40c0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 >> v1:172.21.15.142:6789/0 conn(0x7fc40c162aa0 legacy=0x7fc40c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c0d2c80 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fc3f40017f0 con 0x7fc40c0b5fb0 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 >> v1:172.21.15.46:6789/0 conn(0x7fc40c0b5fb0 legacy=0x7fc40c0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc3f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0d0d90 con 0x7fc3f00051c0 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f000c060 con 0x7fc3f00039b0 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f000c260 con 0x7fc3f0004800 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc411a00640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc3f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.609+0000 7fc411000640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc40be00640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc411a00640 1 -- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f0004800 msgr2=0x7fc3f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc411a00640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc411a00640 1 -- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00039b0 msgr2=0x7fc3f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc411a00640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.611+0000 7fc411a00640 1 -- 172.21.15.46:0/922354467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f0003520 con 0x7fc3f00051c0 2024-04-26T23:55:07.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.612+0000 7fc411a00640 1 -- 172.21.15.46:0/922354467 reap_dead start 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.612+0000 7fc411a00640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc3f000b9c0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc404001750 tx=0x7fc3f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ab14c1d7155d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.612+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404001370 con 0x7fc3f00051c0 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.612+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4040052f0 con 0x7fc3f00051c0 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.612+0000 7fc40b400640 1 -- 172.21.15.46:0/922354467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40402abe0 con 0x7fc3f00051c0 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 -- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 msgr2=0x7fc3f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc3f000b9c0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc404001750 tx=0x7fc3f0005830 comp rx=0 tx=0).stop 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 -- 172.21.15.46:0/922354467 shutdown_connections 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 --2- 172.21.15.46:0/922354467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc3f000b9c0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 -- 172.21.15.46:0/922354467 >> 172.21.15.46:0/922354467 conn(0x7fc40c09e2a0 msgr2=0x7fc40c0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 -- 172.21.15.46:0/922354467 shutdown_connections 2024-04-26T23:55:07.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.613+0000 7fc412400640 1 -- 172.21.15.46:0/922354467 wait complete. 2024-04-26T23:55:07.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 Processor -- start 2024-04-26T23:55:07.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 -- start start 2024-04-26T23:55:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 0x7fc40c0d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40c0d1e30 0x7fc40c146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0b4b50 con 0x7fc3f00051c0 2024-04-26T23:55:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0b49d0 con 0x7fc40c15def0 2024-04-26T23:55:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc412400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d5850 con 0x7fc40c0d1e30 2024-04-26T23:55:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc411000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc411000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45688/0 (socket says 172.21.15.46:45688) 2024-04-26T23:55:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.615+0000 7fc411000640 1 -- 172.21.15.46:0/3563770431 learned_addr learned my addr 172.21.15.46:0/3563770431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc40be00640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 0x7fc40c0d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 -- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40c0d1e30 msgr2=0x7fc40c146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40c0d1e30 0x7fc40c146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 -- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 msgr2=0x7fc40c0d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 0x7fc40c0d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 -- 172.21.15.46:0/3563770431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c147960 con 0x7fc3f00051c0 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc40be00640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 0x7fc40c0d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc411000640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009fb0 tx=0x7fc3fc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc01a020 con 0x7fc3f00051c0 2024-04-26T23:55:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc016470 con 0x7fc3f00051c0 2024-04-26T23:55:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc016860 con 0x7fc3f00051c0 2024-04-26T23:55:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c147c40 con 0x7fc3f00051c0 2024-04-26T23:55:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.617+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c15d6f0 con 0x7fc3f00051c0 2024-04-26T23:55:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.619+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c0d2250 con 0x7fc3f00051c0 2024-04-26T23:55:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.620+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3fc005d10 con 0x7fc3f00051c0 2024-04-26T23:55:07.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.620+0000 7fc408c00640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 0x7fc3e806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:07.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.620+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7fc3fc08cb70 con 0x7fc3f00051c0 2024-04-26T23:55:07.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.622+0000 7fc40be00640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 0x7fc3e806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.623+0000 7fc40be00640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 0x7fc3e806ee30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000ef0 tx=0x7fc3f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:07.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.626+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3fc024030 con 0x7fc3f00051c0 2024-04-26T23:55:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.830+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc40c0693b0 con 0x7fc3f00051c0 2024-04-26T23:55:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.831+0000 7fc408c00640 1 -- 172.21.15.46:0/3563770431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) ==== 104+0+11 (secure 0 0 0) 0x7fc3fc05adf0 con 0x7fc3f00051c0 2024-04-26T23:55:07.833 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 msgr2=0x7fc3e806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 0x7fc3e806ee30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000ef0 tx=0x7fc3f4009040 comp rx=0 tx=0).stop 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 msgr2=0x7fc40c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009fb0 tx=0x7fc3fc0115a0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 shutdown_connections 2024-04-26T23:55:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc3e806c970 0x7fc3e806ee30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc40c0d1e30 0x7fc40c146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc40c15def0 0x7fc40c0d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 --2- 172.21.15.46:0/3563770431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3f00051c0 0x7fc40c0ce990 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 >> 172.21.15.46:0/3563770431 conn(0x7fc40c09e2a0 msgr2=0x7fc40c1ea030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 shutdown_connections 2024-04-26T23:55:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:07.834+0000 7fc412400640 1 -- 172.21.15.46:0/3563770431 wait complete. 2024-04-26T23:55:07.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:55:08.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.080+0000 7fe422800640 1 Processor -- start 2024-04-26T23:55:08.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.080+0000 7fe422800640 1 -- start start 2024-04-26T23:55:08.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.081+0000 7fe422800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe41c146740 0x7fe41c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.081+0000 7fe422800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe41c0d0330 0x7fe41c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.081+0000 7fe422800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41c146b20 con 0x7fe41c0ce670 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.081+0000 7fe422800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe41c0d07c0 con 0x7fe41c146740 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.081+0000 7fe422800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe41c0ccf30 con 0x7fe41c0d0330 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe41c146740 0x7fe41c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe41c0ce670 0x7fe41c0cea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53608/0 (socket says 172.21.15.46:53608) 2024-04-26T23:55:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421e00640 1 -- 172.21.15.46:0/39721954 learned_addr learned my addr 172.21.15.46:0/39721954 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe420a00640 1 --2- 172.21.15.46:0/39721954 >> v2:172.21.15.142:3300/0 conn(0x7fe41c0d0330 0x7fe41c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421400640 1 -- 172.21.15.46:0/39721954 >> v2:172.21.15.142:3300/0 conn(0x7fe41c0d0330 msgr2=0x7fe41c1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421400640 1 --2- 172.21.15.46:0/39721954 >> v2:172.21.15.142:3300/0 conn(0x7fe41c0d0330 0x7fe41c1612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421400640 1 -- 172.21.15.46:0/39721954 >> v1:172.21.15.46:6789/0 conn(0x7fe41c0ce670 legacy=0x7fe41c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe421400640 1 -- 172.21.15.46:0/39721954 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c160af0 con 0x7fe41c146740 2024-04-26T23:55:08.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.082+0000 7fe41be00640 1 -- 172.21.15.46:0/39721954 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840798244 0 0) 0x7fe41c146b20 con 0x7fe41c0ce670 2024-04-26T23:55:08.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe421400640 1 --2- 172.21.15.46:0/39721954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c146740 0x7fe41c15df90 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0cfe60 tx=0x7fe40c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186f265d70c34a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe41be00640 1 -- 172.21.15.46:0/39721954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c037070 con 0x7fe41c146740 2024-04-26T23:55:08.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe41be00640 1 -- 172.21.15.46:0/39721954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c032440 con 0x7fe41c146740 2024-04-26T23:55:08.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe41be00640 1 -- 172.21.15.46:0/39721954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c032810 con 0x7fe41c146740 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe422800640 1 -- 172.21.15.46:0/39721954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c146740 msgr2=0x7fe41c15df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.084+0000 7fe422800640 1 --2- 172.21.15.46:0/39721954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c146740 0x7fe41c15df90 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0cfe60 tx=0x7fe40c02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 -- 172.21.15.46:0/39721954 shutdown_connections 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 --2- 172.21.15.46:0/39721954 >> v2:172.21.15.142:3300/0 conn(0x7fe41c0d0330 0x7fe41c1612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 --2- 172.21.15.46:0/39721954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c146740 0x7fe41c15df90 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 -- 172.21.15.46:0/39721954 >> 172.21.15.46:0/39721954 conn(0x7fe41c0b26d0 msgr2=0x7fe41c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 -- 172.21.15.46:0/39721954 shutdown_connections 2024-04-26T23:55:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.085+0000 7fe422800640 1 -- 172.21.15.46:0/39721954 wait complete. 2024-04-26T23:55:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.086+0000 7fe422800640 1 Processor -- start 2024-04-26T23:55:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.087+0000 7fe422800640 1 -- start start 2024-04-26T23:55:08.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.087+0000 7fe422800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.087+0000 7fe422800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c0d0330 0x7fe41c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe422800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 0x7fe41c0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe422800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0cccf0 con 0x7fe41c0ce670 2024-04-26T23:55:08.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe422800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0ccb70 con 0x7fe41c0d0330 2024-04-26T23:55:08.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe422800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c161830 con 0x7fe41c146740 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45700/0 (socket says 172.21.15.46:45700) 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 -- 172.21.15.46:0/2779559099 learned_addr learned my addr 172.21.15.46:0/2779559099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe420a00640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c0d0330 0x7fe41c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421e00640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 0x7fe41c0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 -- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 msgr2=0x7fe41c0c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 0x7fe41c0c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 -- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c0d0330 msgr2=0x7fe41c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c0d0330 0x7fe41c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421400640 1 -- 172.21.15.46:0/2779559099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0b4260 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.088+0000 7fe421e00640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 0x7fe41c0c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe421400640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fe40c035040 tx=0x7fe40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c037050 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c038040 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c004470 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c0b4540 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.089+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c0b4b00 con 0x7fe41c0ce670 2024-04-26T23:55:08.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.091+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c0693b0 con 0x7fe41c0ce670 2024-04-26T23:55:08.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.092+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe40c030980 con 0x7fe41c0ce670 2024-04-26T23:55:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.093+0000 7fe419600640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 0x7fe3fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.093+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7fe40c0aca40 con 0x7fe41c0ce670 2024-04-26T23:55:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.094+0000 7fe420a00640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 0x7fe3fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.095+0000 7fe420a00640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 0x7fe3fc06edb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe408006fd0 tx=0x7fe408008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.097+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe40c07ad60 con 0x7fe41c0ce670 2024-04-26T23:55:08.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.300+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe41c0afb30 con 0x7fe41c0ce670 2024-04-26T23:55:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.301+0000 7fe419600640 1 -- 172.21.15.46:0/2779559099 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) ==== 100+0+11 (secure 0 0 0) 0x7fe40c07a660 con 0x7fe41c0ce670 2024-04-26T23:55:08.303 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 msgr2=0x7fe3fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 0x7fe3fc06edb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe408006fd0 tx=0x7fe408008040 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 msgr2=0x7fe41c15c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fe40c035040 tx=0x7fe40c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 shutdown_connections 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3fc06c8f0 0x7fe3fc06edb0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41c146740 0x7fe41c0c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c0d0330 0x7fe41c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 --2- 172.21.15.46:0/2779559099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe41c0ce670 0x7fe41c15c7f0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 >> 172.21.15.46:0/2779559099 conn(0x7fe41c0b26d0 msgr2=0x7fe41c0cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:08.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 shutdown_connections 2024-04-26T23:55:08.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.304+0000 7fe422800640 1 -- 172.21.15.46:0/2779559099 wait complete. 2024-04-26T23:55:08.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:55:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.544+0000 7f1d2e200640 1 Processor -- start 2024-04-26T23:55:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.545+0000 7f1d2e200640 1 -- start start 2024-04-26T23:55:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.546+0000 7f1d2e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d280d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.546+0000 7f1d2e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 0x7f1d280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.546+0000 7f1d2e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d280d1e40 con 0x7f1d2815d510 2024-04-26T23:55:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.546+0000 7f1d2e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1d280d1220 con 0x7f1d280d3af0 2024-04-26T23:55:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.546+0000 7f1d2e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d280ce560 con 0x7f1d280d1520 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.547+0000 7f1d27400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d280d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.547+0000 7f1d27e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 0x7f1d280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.547+0000 7f1d1fe00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f1d2815d510 0x7f1d280d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:55628/0 (socket says 172.21.15.46:55628) 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.547+0000 7f1d1fe00640 1 -- 172.21.15.46:0/531595982 learned_addr learned my addr 172.21.15.46:0/531595982 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d26a00640 1 -- 172.21.15.46:0/531595982 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1160813901 0 0) 0x7f1d280d1e40 con 0x7f1d2815d510 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d26a00640 1 -- 172.21.15.46:0/531595982 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d0c003520 con 0x7f1d2815d510 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d27400640 1 -- 172.21.15.46:0/531595982 >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 msgr2=0x7f1d280706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d27400640 1 --2- 172.21.15.46:0/531595982 >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 0x7f1d280706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d27400640 1 -- 172.21.15.46:0/531595982 >> v1:172.21.15.142:6789/0 conn(0x7f1d2815d510 legacy=0x7f1d280d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d27400640 1 -- 172.21.15.46:0/531595982 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d280ce7b0 con 0x7f1d280d1520 2024-04-26T23:55:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.548+0000 7f1d27e00640 1 --2- 172.21.15.46:0/531595982 >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 0x7f1d280706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.549+0000 7f1d27400640 1 --2- 172.21.15.46:0/531595982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d280d1900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d180097a0 tx=0x7f1d1802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883fda878f4ad8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.550+0000 7f1d26a00640 1 -- 172.21.15.46:0/531595982 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18037070 con 0x7f1d280d1520 2024-04-26T23:55:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.550+0000 7f1d26a00640 1 -- 172.21.15.46:0/531595982 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18032440 con 0x7f1d280d1520 2024-04-26T23:55:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.550+0000 7f1d26a00640 1 -- 172.21.15.46:0/531595982 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d180327f0 con 0x7f1d280d1520 2024-04-26T23:55:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 -- 172.21.15.46:0/531595982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 msgr2=0x7f1d280d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 --2- 172.21.15.46:0/531595982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d280d1900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d180097a0 tx=0x7f1d1802a180 comp rx=0 tx=0).stop 2024-04-26T23:55:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 -- 172.21.15.46:0/531595982 shutdown_connections 2024-04-26T23:55:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 --2- 172.21.15.46:0/531595982 >> v2:172.21.15.142:3300/0 conn(0x7f1d280d3af0 0x7f1d280706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 --2- 172.21.15.46:0/531595982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d280d1900 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 -- 172.21.15.46:0/531595982 >> 172.21.15.46:0/531595982 conn(0x7f1d280d2f40 msgr2=0x7f1d280d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 -- 172.21.15.46:0/531595982 shutdown_connections 2024-04-26T23:55:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.551+0000 7f1d2e200640 1 -- 172.21.15.46:0/531595982 wait complete. 2024-04-26T23:55:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.553+0000 7f1d2e200640 1 Processor -- start 2024-04-26T23:55:08.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.553+0000 7f1d2e200640 1 -- start start 2024-04-26T23:55:08.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d281f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d280d3af0 0x7f1d280b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280ced70 con 0x7f1d2815d510 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28070230 con 0x7f1d280d3af0 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d2e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d28070c00 con 0x7f1d280d1520 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45712/0 (socket says 172.21.15.46:45712) 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.554+0000 7f1d27e00640 1 -- 172.21.15.46:0/3748399306 learned_addr learned my addr 172.21.15.46:0/3748399306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27400640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d281f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d1fe00640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d280d3af0 0x7f1d280b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 -- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 msgr2=0x7f1d281f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d281f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 -- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d280d3af0 msgr2=0x7f1d280b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d280d3af0 0x7f1d280b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 -- 172.21.15.46:0/3748399306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d281f5710 con 0x7f1d2815d510 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d27e00640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000efb0 tx=0x7f1d2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20018020 con 0x7f1d2815d510 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d20014480 con 0x7f1d2815d510 2024-04-26T23:55:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20014850 con 0x7f1d2815d510 2024-04-26T23:55:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.555+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d280b3410 con 0x7f1d2815d510 2024-04-26T23:55:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.556+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d280b39d0 con 0x7f1d2815d510 2024-04-26T23:55:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.557+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d280b3bd0 con 0x7f1d2815d510 2024-04-26T23:55:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.558+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d20005d10 con 0x7f1d2815d510 2024-04-26T23:55:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.559+0000 7f1d1f400640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 0x7f1d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.559+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f1d2008cb90 con 0x7f1d2815d510 2024-04-26T23:55:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.560+0000 7f1d27400640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 0x7f1d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:08.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.561+0000 7f1d27400640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 0x7f1d0406edb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d18034000 tx=0x7f1d1802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.563+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d20022030 con 0x7f1d2815d510 2024-04-26T23:55:08.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.778+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d28069530 con 0x7f1d2815d510 2024-04-26T23:55:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.779+0000 7f1d1f400640 1 -- 172.21.15.46:0/3748399306 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v175) ==== 80+0+0 (secure 0 0 0) 0x7f1d20010040 con 0x7f1d2815d510 2024-04-26T23:55:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 msgr2=0x7f1d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 0x7f1d0406edb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d18034000 tx=0x7f1d1802e000 comp rx=0 tx=0).stop 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 msgr2=0x7f1d280b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000efb0 tx=0x7f1d2000c490 comp rx=0 tx=0).stop 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 shutdown_connections 2024-04-26T23:55:08.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1d0406c8f0 0x7f1d0406edb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d2815d510 0x7f1d280b2de0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d280d3af0 0x7f1d280b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 --2- 172.21.15.46:0/3748399306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d280d1520 0x7f1d281f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 >> 172.21.15.46:0/3748399306 conn(0x7f1d280d2f40 msgr2=0x7f1d280b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 shutdown_connections 2024-04-26T23:55:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:08.782+0000 7f1d2e200640 1 -- 172.21.15.46:0/3748399306 wait complete. 2024-04-26T23:55:08.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:55:08.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:08.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:55:08.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:55:08.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:08.902 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:55:08.959 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:55:08.960 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:55:08.960 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:09.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:55:09.018 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:55:09.018 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:09.018 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:09.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:55:09.077 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:55:09.077 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:55:09.078 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:09.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:55:09.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.355+0000 7ff8b0e00640 1 Processor -- start 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.355+0000 7ff8b0e00640 1 -- start start 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.356+0000 7ff8b0e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 0x7ff8ac06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8b0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8ac160bd0 0x7ff8ac161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8b0e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ac06ee30 con 0x7ff8ac15d540 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8b0e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ac0742d0 con 0x7ff8ac160bd0 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8b0e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ac161940 con 0x7ff8ac06e510 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8a9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8ac160bd0 0x7ff8ac161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8aa800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8ac15d540 0x7ff8ac15d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53610/0 (socket says 172.21.15.46:53610) 2024-04-26T23:55:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8aa800640 1 -- 172.21.15.46:0/1965135959 learned_addr learned my addr 172.21.15.46:0/1965135959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.357+0000 7ff8a9e00640 1 --2- 172.21.15.46:0/1965135959 >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 0x7ff8ac06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a9400640 1 -- 172.21.15.46:0/1965135959 >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 msgr2=0x7ff8ac06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a9400640 1 --2- 172.21.15.46:0/1965135959 >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 0x7ff8ac06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a9400640 1 -- 172.21.15.46:0/1965135959 >> v1:172.21.15.46:6789/0 conn(0x7ff8ac15d540 legacy=0x7ff8ac15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a9400640 1 -- 172.21.15.46:0/1965135959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac071f80 con 0x7ff8ac160bd0 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a8a00640 1 -- 172.21.15.46:0/1965135959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345122162 0 0) 0x7ff8ac06ee30 con 0x7ff8ac15d540 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.358+0000 7ff8a9e00640 1 --2- 172.21.15.46:0/1965135959 >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 0x7ff8ac06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.359+0000 7ff8a9400640 1 --2- 172.21.15.46:0/1965135959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac161060 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff89402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211bfb43e3aad145 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.359+0000 7ff8a8a00640 1 -- 172.21.15.46:0/1965135959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894007740 con 0x7ff8ac160bd0 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.359+0000 7ff8a8a00640 1 -- 172.21.15.46:0/1965135959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8940078e0 con 0x7ff8ac160bd0 2024-04-26T23:55:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.359+0000 7ff8a8a00640 1 -- 172.21.15.46:0/1965135959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894007cb0 con 0x7ff8ac160bd0 2024-04-26T23:55:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 -- 172.21.15.46:0/1965135959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac160bd0 msgr2=0x7ff8ac161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/1965135959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac161060 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff89402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 -- 172.21.15.46:0/1965135959 shutdown_connections 2024-04-26T23:55:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/1965135959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac161060 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/1965135959 >> v2:172.21.15.142:3300/0 conn(0x7ff8ac06e510 0x7ff8ac06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 -- 172.21.15.46:0/1965135959 >> 172.21.15.46:0/1965135959 conn(0x7ff8ac06c530 msgr2=0x7ff8ac06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 -- 172.21.15.46:0/1965135959 shutdown_connections 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.360+0000 7ff8b0e00640 1 -- 172.21.15.46:0/1965135959 wait complete. 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.362+0000 7ff8b0e00640 1 Processor -- start 2024-04-26T23:55:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.362+0000 7ff8b0e00640 1 -- start start 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac06e510 0x7ff8ac13cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac072820 con 0x7ff8ac15d540 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac072540 con 0x7ff8ac160bd0 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ac1615a0 con 0x7ff8ac06e510 2024-04-26T23:55:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45742/0 (socket says 172.21.15.46:45742) 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 -- 172.21.15.46:0/360784450 learned_addr learned my addr 172.21.15.46:0/360784450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 -- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac06e510 msgr2=0x7ff8ac13cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac06e510 0x7ff8ac13cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 -- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac160bd0 msgr2=0x7ff8ac142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.363+0000 7ff8a9400640 1 -- 172.21.15.46:0/360784450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac13fce0 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8a9400640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff894004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89403f070 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff894031c40 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894003990 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac13ffc0 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.364+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac140580 con 0x7ff8ac15d540 2024-04-26T23:55:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.366+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac140950 con 0x7ff8ac15d540 2024-04-26T23:55:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.366+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff894007790 con 0x7ff8ac15d540 2024-04-26T23:55:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.367+0000 7ff8a2000640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 0x7ff88806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.367+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7ff8940ad190 con 0x7ff8ac15d540 2024-04-26T23:55:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.368+0000 7ff8a9e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 0x7ff88806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.369+0000 7ff8a9e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 0x7ff88806edb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006fd0 tx=0x7ff89c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.372+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff89407b270 con 0x7ff8ac15d540 2024-04-26T23:55:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.620+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8ac06fb00 con 0x7ff8ac15d540 2024-04-26T23:55:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.621+0000 7ff8a2000640 1 -- 172.21.15.46:0/360784450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7ff89407ab70 con 0x7ff8ac15d540 2024-04-26T23:55:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:09.623 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:55:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 msgr2=0x7ff88806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 0x7ff88806edb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006fd0 tx=0x7ff89c008040 comp rx=0 tx=0).stop 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 msgr2=0x7ff8ac13f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff894004a40 comp rx=0 tx=0).stop 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 shutdown_connections 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff88806c8f0 0x7ff88806edb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac160bd0 0x7ff8ac142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8ac15d540 0x7ff8ac13f2e0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 --2- 172.21.15.46:0/360784450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac06e510 0x7ff8ac13cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 >> 172.21.15.46:0/360784450 conn(0x7ff8ac06c530 msgr2=0x7ff8ac06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 shutdown_connections 2024-04-26T23:55:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.624+0000 7ff8b0e00640 1 -- 172.21.15.46:0/360784450 wait complete. 2024-04-26T23:55:09.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.848+0000 7f652a400640 1 Processor -- start 2024-04-26T23:55:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.848+0000 7f652a400640 1 -- start start 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f652a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65240720e0 0x7f65240cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f652a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6524160910 0x7f6524160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f652a400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6524073ad0 con 0x7f65241466b0 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f652a400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6524146a90 con 0x7f6524160910 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f652a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6524073d70 con 0x7f65240720e0 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f6529a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6524160910 0x7f6524160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f6529a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6524160910 0x7f6524160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45744/0 (socket says 172.21.15.46:45744) 2024-04-26T23:55:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.850+0000 7f6529a00640 1 -- 172.21.15.46:0/3381928323 learned_addr learned my addr 172.21.15.46:0/3381928323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.851+0000 7f6523e00640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65240720e0 0x7f65240cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.851+0000 7f6529a00640 1 -- 172.21.15.46:0/3381928323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65240720e0 msgr2=0x7f65240cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.851+0000 7f6529a00640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65240720e0 0x7f65240cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.851+0000 7f6529a00640 1 -- 172.21.15.46:0/3381928323 >> v1:172.21.15.142:6789/0 conn(0x7f65241466b0 legacy=0x7f652406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.851+0000 7f6529a00640 1 -- 172.21.15.46:0/3381928323 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524074050 con 0x7f6524160910 2024-04-26T23:55:09.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.852+0000 7f6529a00640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f6524160da0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f651c009e50 tx=0x7f651c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f574dff84268b53f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.853+0000 7f6523400640 1 -- 172.21.15.46:0/3381928323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c036070 con 0x7f6524160910 2024-04-26T23:55:09.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.853+0000 7f6523400640 1 -- 172.21.15.46:0/3381928323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c031600 con 0x7f6524160910 2024-04-26T23:55:09.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.853+0000 7f6523400640 1 -- 172.21.15.46:0/3381928323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0319d0 con 0x7f6524160910 2024-04-26T23:55:09.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.853+0000 7f652a400640 1 -- 172.21.15.46:0/3381928323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 msgr2=0x7f6524160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.853+0000 7f652a400640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f6524160da0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f651c009e50 tx=0x7f651c02a710 comp rx=0 tx=0).stop 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 -- 172.21.15.46:0/3381928323 shutdown_connections 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f6524160da0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 --2- 172.21.15.46:0/3381928323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65240720e0 0x7f65240cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 -- 172.21.15.46:0/3381928323 >> 172.21.15.46:0/3381928323 conn(0x7f65240b26d0 msgr2=0x7f65240b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 -- 172.21.15.46:0/3381928323 shutdown_connections 2024-04-26T23:55:09.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.854+0000 7f652a400640 1 -- 172.21.15.46:0/3381928323 wait complete. 2024-04-26T23:55:09.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.855+0000 7f652a400640 1 Processor -- start 2024-04-26T23:55:09.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.855+0000 7f652a400640 1 -- start start 2024-04-26T23:55:09.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 0x7f652415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65241466b0 0x7f65241f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65240cdda0 con 0x7f6524160910 2024-04-26T23:55:09.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65240cb470 con 0x7f65240720e0 2024-04-26T23:55:09.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.856+0000 7f652a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65240cb990 con 0x7f65241466b0 2024-04-26T23:55:09.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45752/0 (socket says 172.21.15.46:45752) 2024-04-26T23:55:09.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 -- 172.21.15.46:0/110033304 learned_addr learned my addr 172.21.15.46:0/110033304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6523e00640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65241466b0 0x7f65241f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529000640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 0x7f652415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 -- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65241466b0 msgr2=0x7f65241f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65241466b0 0x7f65241f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 -- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 msgr2=0x7f652415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 0x7f652415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529a00640 1 -- 172.21.15.46:0/110033304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524160050 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.857+0000 7f6529000640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 0x7f652415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f6529a00640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f651c009e50 tx=0x7f651c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c041020 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c037040 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c036070 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65240b08a0 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.858+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65240b0e60 con 0x7f6524160910 2024-04-26T23:55:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.859+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65240693b0 con 0x7f6524160910 2024-04-26T23:55:09.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.860+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f651c044020 con 0x7f6524160910 2024-04-26T23:55:09.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.861+0000 7f6520c00640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 0x7f650406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:09.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.861+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f651c0ac920 con 0x7f6524160910 2024-04-26T23:55:09.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.862+0000 7f6529000640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 0x7f650406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.863+0000 7f6529000640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 0x7f650406edb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6518001680 tx=0x7f6518009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:09.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:09.865+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f651c04b030 con 0x7f6524160910 2024-04-26T23:55:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.062+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6524072d30 con 0x7f6524160910 2024-04-26T23:55:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.065+0000 7f6520c00640 1 -- 172.21.15.46:0/110033304 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+17089 (secure 0 0 0) 0x7f651c07abe0 con 0x7f6524160910 2024-04-26T23:55:10.068 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:10.068 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":175,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:58.408057+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.21","mappings":[{"from":1,"to":0}]},{"pgid":"26.22","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 msgr2=0x7f650406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 0x7f650406edb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6518001680 tx=0x7f6518009040 comp rx=0 tx=0).stop 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 msgr2=0x7f65241f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f651c009e50 tx=0x7f651c004a40 comp rx=0 tx=0).stop 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 shutdown_connections 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f650406c8f0 0x7f650406edb0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6524160910 0x7f65241f87c0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65241466b0 0x7f65241f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 --2- 172.21.15.46:0/110033304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240720e0 0x7f652415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 >> 172.21.15.46:0/110033304 conn(0x7f65240b26d0 msgr2=0x7f65240cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 shutdown_connections 2024-04-26T23:55:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.068+0000 7f652a400640 1 -- 172.21.15.46:0/110033304 wait complete. 2024-04-26T23:55:10.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:54:22.516305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:54:23.663985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:55:10.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:55:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.298+0000 7f5ace200640 1 Processor -- start 2024-04-26T23:55:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.298+0000 7f5ace200640 1 -- start start 2024-04-26T23:55:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5ace200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ac8160130 0x7f5ac8160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5ace200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac8072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5ace200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac80cbe60 con 0x7f5ac815cc40 2024-04-26T23:55:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5ace200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80cca70 con 0x7f5ac8160130 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5ace200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac80cc140 con 0x7f5ac80cdc60 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5acce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ac8160130 0x7f5ac8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5abfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ac815cc40 0x7f5ac80cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53612/0 (socket says 172.21.15.46:53612) 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.300+0000 7f5abfe00640 1 -- 172.21.15.46:0/2305574572 learned_addr learned my addr 172.21.15.46:0/2305574572 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.301+0000 7f5acd800640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac8072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.301+0000 7f5acce00640 1 -- 172.21.15.46:0/2305574572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 msgr2=0x7f5ac8072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.301+0000 7f5acce00640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac8072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.301+0000 7f5acce00640 1 -- 172.21.15.46:0/2305574572 >> v1:172.21.15.46:6789/0 conn(0x7f5ac815cc40 legacy=0x7f5ac80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.301+0000 7f5acce00640 1 -- 172.21.15.46:0/2305574572 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80cc420 con 0x7f5ac8160130 2024-04-26T23:55:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.302+0000 7f5acce00640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac8160510 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80097a0 tx=0x7f5ab802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0b244caccd4d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.303+0000 7f5ac7e00640 1 -- 172.21.15.46:0/2305574572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8037070 con 0x7f5ac8160130 2024-04-26T23:55:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.303+0000 7f5ac7e00640 1 -- 172.21.15.46:0/2305574572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab802ae40 con 0x7f5ac8160130 2024-04-26T23:55:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.303+0000 7f5ac7e00640 1 -- 172.21.15.46:0/2305574572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8032660 con 0x7f5ac8160130 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 -- 172.21.15.46:0/2305574572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 msgr2=0x7f5ac8160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac8160510 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80097a0 tx=0x7f5ab802a0f0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 -- 172.21.15.46:0/2305574572 shutdown_connections 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac8072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 --2- 172.21.15.46:0/2305574572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac8160510 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 -- 172.21.15.46:0/2305574572 >> 172.21.15.46:0/2305574572 conn(0x7f5ac80b27b0 msgr2=0x7f5ac80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 -- 172.21.15.46:0/2305574572 shutdown_connections 2024-04-26T23:55:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.304+0000 7f5ace200640 1 -- 172.21.15.46:0/2305574572 wait complete. 2024-04-26T23:55:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.306+0000 7f5ace200640 1 Processor -- start 2024-04-26T23:55:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.306+0000 7f5ace200640 1 -- start start 2024-04-26T23:55:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac815cc40 0x7f5ac815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815d220 con 0x7f5ac8160130 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815d0a0 con 0x7f5ac815cc40 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5ace200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8072760 con 0x7f5ac80cdc60 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5acd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5acd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45770/0 (socket says 172.21.15.46:45770) 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5acd800640 1 -- 172.21.15.46:0/3885925768 learned_addr learned my addr 172.21.15.46:0/3885925768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5abfe00640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac815cc40 0x7f5ac815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.307+0000 7f5acce00640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 -- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 msgr2=0x7f5ac814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac814c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 -- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac815cc40 msgr2=0x7f5ac815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac815cc40 0x7f5ac815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 -- 172.21.15.46:0/3885925768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80c89a0 con 0x7f5ac8160130 2024-04-26T23:55:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acce00640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5acd800640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000efc0 tx=0x7f5ac000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0018020 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0009b10 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac80c8c80 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.308+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00146c0 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.309+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80c9240 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.310+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac80693b0 con 0x7f5ac8160130 2024-04-26T23:55:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.310+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ac0005d10 con 0x7f5ac8160130 2024-04-26T23:55:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.311+0000 7f5ac5600640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 0x7f5aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.311+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f5ac008c7a0 con 0x7f5ac8160130 2024-04-26T23:55:10.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.312+0000 7f5acce00640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 0x7f5aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.313+0000 7f5acce00640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 0x7f5aa406edb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8034000 tx=0x7f5ab802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.317+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ac0022070 con 0x7f5ac8160130 2024-04-26T23:55:10.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.523+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ac80cbb10 con 0x7f5ac8160130 2024-04-26T23:55:10.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.524+0000 7f5ac5600640 1 -- 172.21.15.46:0/3885925768 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) ==== 93+0+10 (secure 0 0 0) 0x7f5ac0010040 con 0x7f5ac8160130 2024-04-26T23:55:10.526 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:55:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 msgr2=0x7f5aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 0x7f5aa406edb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8034000 tx=0x7f5ab802e000 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 msgr2=0x7f5ac815b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000efc0 tx=0x7f5ac000c490 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 shutdown_connections 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5aa406c8f0 0x7f5aa406edb0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac8160130 0x7f5ac815b310 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac815cc40 0x7f5ac815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 --2- 172.21.15.46:0/3885925768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ac80cdc60 0x7f5ac814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.526+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 >> 172.21.15.46:0/3885925768 conn(0x7f5ac80b27b0 msgr2=0x7f5ac80c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.527+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 shutdown_connections 2024-04-26T23:55:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.527+0000 7f5ace200640 1 -- 172.21.15.46:0/3885925768 wait complete. 2024-04-26T23:55:10.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:55:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.748+0000 7fdcf8e00640 1 Processor -- start 2024-04-26T23:55:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.749+0000 7fdcf8e00640 1 -- start start 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcf4160510 0x7fdcf40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 0x7fdcf4072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf8e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf40cbe60 con 0x7fdcf4160130 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf8e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40cca70 con 0x7fdcf4160510 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf8e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf415d350 con 0x7fdcf40cda90 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.750+0000 7fdcf3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdcf4160130 0x7fdcf415cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53622/0 (socket says 172.21.15.46:53622) 2024-04-26T23:55:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcf3400640 1 -- 172.21.15.46:0/672468744 learned_addr learned my addr 172.21.15.46:0/672468744 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcebe00640 1 --2- 172.21.15.46:0/672468744 >> v2:172.21.15.46:3300/0 conn(0x7fdcf4160510 0x7fdcf40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcf2a00640 1 -- 172.21.15.46:0/672468744 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217271760 0 0) 0x7fdcf40cbe60 con 0x7fdcf4160130 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcf2a00640 1 -- 172.21.15.46:0/672468744 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcd8003520 con 0x7fdcf4160130 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcf3e00640 1 --2- 172.21.15.46:0/672468744 >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 0x7fdcf4072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcebe00640 1 -- 172.21.15.46:0/672468744 >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 msgr2=0x7fdcf4072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcebe00640 1 --2- 172.21.15.46:0/672468744 >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 0x7fdcf4072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcebe00640 1 -- 172.21.15.46:0/672468744 >> v1:172.21.15.46:6789/0 conn(0x7fdcf4160130 legacy=0x7fdcf415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcebe00640 1 -- 172.21.15.46:0/672468744 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf40cc140 con 0x7fdcf4160510 2024-04-26T23:55:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.751+0000 7fdcf3e00640 1 --2- 172.21.15.46:0/672468744 >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 0x7fdcf4072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.752+0000 7fdcebe00640 1 --2- 172.21.15.46:0/672468744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40cd550 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40cd190 tx=0x7fdcdc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dc601aeae46297 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf2a00640 1 -- 172.21.15.46:0/672468744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc037070 con 0x7fdcf4160510 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf2a00640 1 -- 172.21.15.46:0/672468744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc032440 con 0x7fdcf4160510 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf2a00640 1 -- 172.21.15.46:0/672468744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02ae60 con 0x7fdcf4160510 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 -- 172.21.15.46:0/672468744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 msgr2=0x7fdcf40cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/672468744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40cd550 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40cd190 tx=0x7fdcdc02a110 comp rx=0 tx=0).stop 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 -- 172.21.15.46:0/672468744 shutdown_connections 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/672468744 >> v2:172.21.15.142:3300/0 conn(0x7fdcf40cda90 0x7fdcf4072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/672468744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40cd550 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 -- 172.21.15.46:0/672468744 >> 172.21.15.46:0/672468744 conn(0x7fdcf40b27b0 msgr2=0x7fdcf40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.753+0000 7fdcf8e00640 1 -- 172.21.15.46:0/672468744 shutdown_connections 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.754+0000 7fdcf8e00640 1 -- 172.21.15.46:0/672468744 wait complete. 2024-04-26T23:55:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.755+0000 7fdcf8e00640 1 Processor -- start 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.755+0000 7fdcf8e00640 1 -- start start 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 0x7fdcf41f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcf4160130 0x7fdcf40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4072760 con 0x7fdcf4160510 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf40cc700 con 0x7fdcf4160130 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.756+0000 7fdcf8e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40728e0 con 0x7fdcf40cda90 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45788/0 (socket says 172.21.15.46:45788) 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 -- 172.21.15.46:0/4126125199 learned_addr learned my addr 172.21.15.46:0/4126125199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3400640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 0x7fdcf41f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 -- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 msgr2=0x7fdcf41f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 0x7fdcf41f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 -- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcf4160130 msgr2=0x7fdcf40c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcebe00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcf4160130 0x7fdcf40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcf4160130 0x7fdcf40c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 -- 172.21.15.46:0/4126125199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf41f4be0 con 0x7fdcf4160510 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3400640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 0x7fdcf41f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.757+0000 7fdcf3e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00ce40 tx=0x7fdcec00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.758+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec01a070 con 0x7fdcf4160510 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.758+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec00d840 con 0x7fdcf4160510 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.758+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00dc30 con 0x7fdcf4160510 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.758+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf41f4ec0 con 0x7fdcf4160510 2024-04-26T23:55:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.758+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf41f5480 con 0x7fdcf4160510 2024-04-26T23:55:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.759+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf40693b0 con 0x7fdcf4160510 2024-04-26T23:55:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.760+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcec01d070 con 0x7fdcf4160510 2024-04-26T23:55:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.761+0000 7fdceb400640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.761+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7fdcec08c620 con 0x7fdcf4160510 2024-04-26T23:55:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.762+0000 7fdcf3400640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.763+0000 7fdcf3400640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 0x7fdcd006edb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005880 tx=0x7fdce4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.765+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcec05a790 con 0x7fdcf4160510 2024-04-26T23:55:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.969+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcf40cbb10 con 0x7fdcf4160510 2024-04-26T23:55:10.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.969+0000 7fdceb400640 1 -- 172.21.15.46:0/4126125199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) ==== 104+0+11 (secure 0 0 0) 0x7fdcec05a090 con 0x7fdcf4160510 2024-04-26T23:55:10.971 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:55:10.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 msgr2=0x7fdcd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 0x7fdcd006edb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005880 tx=0x7fdce4009040 comp rx=0 tx=0).stop 2024-04-26T23:55:10.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 msgr2=0x7fdcf40c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00ce40 tx=0x7fdcec00a710 comp rx=0 tx=0).stop 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 shutdown_connections 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcf4160510 0x7fdcf40c8f50 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcf4160130 0x7fdcf40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 --2- 172.21.15.46:0/4126125199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcf40cda90 0x7fdcf41f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 >> 172.21.15.46:0/4126125199 conn(0x7fdcf40b27b0 msgr2=0x7fdcf4072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 shutdown_connections 2024-04-26T23:55:10.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:10.972+0000 7fdcf8e00640 1 -- 172.21.15.46:0/4126125199 wait complete. 2024-04-26T23:55:10.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:55:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.205+0000 7fa35b200640 1 Processor -- start 2024-04-26T23:55:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.206+0000 7fa35b200640 1 -- start start 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.207+0000 7fa35b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa35c16a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.207+0000 7fa35b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0d4090 con 0x7fa35c0cfb60 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.207+0000 7fa35b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0b4020 con 0x7fa35c15f840 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.207+0000 7fa35b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c0d3880 con 0x7fa35c0d06d0 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.208+0000 7fa359400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa35c0cfb60 0x7fa35c0cffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53628/0 (socket says 172.21.15.46:53628) 2024-04-26T23:55:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.208+0000 7fa359400640 1 -- 172.21.15.46:0/2734544825 learned_addr learned my addr 172.21.15.46:0/2734544825 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.208+0000 7fa35a800640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa35c16a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.208+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095495670 0 0) 0x7fa344001490 con 0x7fa35c0cfb60 2024-04-26T23:55:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.208+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa33c003520 con 0x7fa35c0cfb60 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3991792305 0 0) 0x7fa33c003520 con 0x7fa35c0cfb60 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 msgr2=0x7fa35c16a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa35c16a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 >> v1:172.21.15.142:6789/0 conn(0x7fa35c15f840 legacy=0x7fa35c0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c0d3ad0 con 0x7fa35c0cfb60 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa344001bc0 con 0x7fa35c0cfb60 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 >> v1:172.21.15.46:6789/0 conn(0x7fa35c0cfb60 legacy=0x7fa35c0cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa33c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa33c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c004d40 0x7fa33c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0d3880 con 0x7fa33c0039e0 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c00b9c0 con 0x7fa33c004d40 2024-04-26T23:55:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c00bbc0 con 0x7fa35c0d06d0 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.209+0000 7fa359e00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa33c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359400640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa33c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa35a800640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c004d40 0x7fa33c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359e00640 1 -- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 msgr2=0x7fa33c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359e00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa33c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359e00640 1 -- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c004d40 msgr2=0x7fa33c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359e00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c004d40 0x7fa33c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359e00640 1 -- 172.21.15.46:0/2734544825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c003520 con 0x7fa33c0039e0 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa359400640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0d06d0 0x7fa33c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.210+0000 7fa35a800640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa33c004d40 0x7fa33c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.211+0000 7fa359e00640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa33c003dc0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fa34c001860 tx=0x7fa33c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a106871563057d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.211+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c037070 con 0x7fa33c0039e0 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.211+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c02ab60 con 0x7fa33c0039e0 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.211+0000 7fa358a00640 1 -- 172.21.15.46:0/2734544825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c001430 con 0x7fa33c0039e0 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 -- 172.21.15.46:0/2734544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 msgr2=0x7fa33c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 --2- 172.21.15.46:0/2734544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa33c003dc0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fa34c001860 tx=0x7fa33c005830 comp rx=0 tx=0).stop 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35a800640 1 -- 172.21.15.46:0/2734544825 reap_dead start 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 -- 172.21.15.46:0/2734544825 shutdown_connections 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 -- 172.21.15.46:0/2734544825 >> 172.21.15.46:0/2734544825 conn(0x7fa35c0d4f70 msgr2=0x7fa35c0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 -- 172.21.15.46:0/2734544825 shutdown_connections 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.212+0000 7fa35b200640 1 -- 172.21.15.46:0/2734544825 wait complete. 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.214+0000 7fa35b200640 1 Processor -- start 2024-04-26T23:55:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.214+0000 7fa35b200640 1 -- start start 2024-04-26T23:55:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa35c161e80 0x7fa35c162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0cdf00 0x7fa35c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0b3d00 con 0x7fa33c0039e0 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0b27d0 con 0x7fa35c161e80 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa35b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c0b3e80 con 0x7fa35c0cdf00 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa359e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa359e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45806/0 (socket says 172.21.15.46:45806) 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.215+0000 7fa359e00640 1 -- 172.21.15.46:0/465587172 learned_addr learned my addr 172.21.15.46:0/465587172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa35a800640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0cdf00 0x7fa35c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359400640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa35c161e80 0x7fa35c162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 -- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0cdf00 msgr2=0x7fa35c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0cdf00 0x7fa35c0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 -- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa35c161e80 msgr2=0x7fa35c162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa35c161e80 0x7fa35c162300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 -- 172.21.15.46:0/465587172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c0b8830 con 0x7fa33c0039e0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa359e00640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c03f020 con 0x7fa33c0039e0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c005270 con 0x7fa33c0039e0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c037040 con 0x7fa33c0039e0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.216+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c0b8b10 con 0x7fa33c0039e0 2024-04-26T23:55:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.217+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c0cd9e0 con 0x7fa33c0039e0 2024-04-26T23:55:11.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.218+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c0693b0 con 0x7fa33c0039e0 2024-04-26T23:55:11.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.219+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34c042020 con 0x7fa33c0039e0 2024-04-26T23:55:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.220+0000 7fa352000640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 0x7fa33806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.220+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7fa34c0abcb0 con 0x7fa33c0039e0 2024-04-26T23:55:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.221+0000 7fa359400640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 0x7fa33806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.222+0000 7fa359400640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 0x7fa33806ede0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa344007c40 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.225+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa34c049030 con 0x7fa33c0039e0 2024-04-26T23:55:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.427+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa35c0d5d10 con 0x7fa33c0039e0 2024-04-26T23:55:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.428+0000 7fa352000640 1 -- 172.21.15.46:0/465587172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) ==== 100+0+11 (secure 0 0 0) 0x7fa34c079ed0 con 0x7fa33c0039e0 2024-04-26T23:55:11.429 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 msgr2=0x7fa33806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 0x7fa33806ede0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa344007c40 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-26T23:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 msgr2=0x7fa35c15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c001230 comp rx=0 tx=0).stop 2024-04-26T23:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 shutdown_connections 2024-04-26T23:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa33806c920 0x7fa33806ede0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa35c0cdf00 0x7fa35c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa35c161e80 0x7fa35c162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 --2- 172.21.15.46:0/465587172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33c0039e0 0x7fa35c15b820 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.430+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 >> 172.21.15.46:0/465587172 conn(0x7fa35c0d4f70 msgr2=0x7fa35c0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.431+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 shutdown_connections 2024-04-26T23:55:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.431+0000 7fa35b200640 1 -- 172.21.15.46:0/465587172 wait complete. 2024-04-26T23:55:11.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.658+0000 7f41e4a00640 1 Processor -- start 2024-04-26T23:55:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.658+0000 7f41e4a00640 1 -- start start 2024-04-26T23:55:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41e4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41e0160520 0x7f41e00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41e4a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41e0072f50 0x7f41e0073380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41e4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e00cdc10 con 0x7f41e015be40 2024-04-26T23:55:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41e4a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f41e00cd9e0 con 0x7f41e0160520 2024-04-26T23:55:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41e4a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f41e00ce1d0 con 0x7f41e0072f50 2024-04-26T23:55:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41de800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f41e015be40 0x7f41e00d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53642/0 (socket says 172.21.15.46:53642) 2024-04-26T23:55:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41de800640 1 -- 172.21.15.46:0/1208274116 learned_addr learned my addr 172.21.15.46:0/1208274116 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.660+0000 7f41d5e00640 1 --2- 172.21.15.46:0/1208274116 >> v2:172.21.15.46:3300/0 conn(0x7f41e0160520 0x7f41e00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41df200640 1 --2- 172.21.15.46:0/1208274116 >> v2:172.21.15.142:3300/0 conn(0x7f41e0072f50 0x7f41e0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41d5e00640 1 -- 172.21.15.46:0/1208274116 >> v2:172.21.15.142:3300/0 conn(0x7f41e0072f50 msgr2=0x7f41e0073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41d5e00640 1 --2- 172.21.15.46:0/1208274116 >> v2:172.21.15.142:3300/0 conn(0x7f41e0072f50 0x7f41e0073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41d5e00640 1 -- 172.21.15.46:0/1208274116 >> v1:172.21.15.46:6789/0 conn(0x7f41e015be40 legacy=0x7f41e00d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41d5e00640 1 -- 172.21.15.46:0/1208274116 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e00cdef0 con 0x7f41e0160520 2024-04-26T23:55:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.661+0000 7f41dde00640 1 -- 172.21.15.46:0/1208274116 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394863168 0 0) 0x7f41e00cdc10 con 0x7f41e015be40 2024-04-26T23:55:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.662+0000 7f41d5e00640 1 --2- 172.21.15.46:0/1208274116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e00cd4a0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f41e00ce350 tx=0x7f41c802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431fb42ca05e25d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41dde00640 1 -- 172.21.15.46:0/1208274116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8037070 con 0x7f41e0160520 2024-04-26T23:55:11.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41dde00640 1 -- 172.21.15.46:0/1208274116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c802add0 con 0x7f41e0160520 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41dde00640 1 -- 172.21.15.46:0/1208274116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8032660 con 0x7f41e0160520 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 -- 172.21.15.46:0/1208274116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 msgr2=0x7f41e00cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 --2- 172.21.15.46:0/1208274116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e00cd4a0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f41e00ce350 tx=0x7f41c802a080 comp rx=0 tx=0).stop 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 -- 172.21.15.46:0/1208274116 shutdown_connections 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 --2- 172.21.15.46:0/1208274116 >> v2:172.21.15.142:3300/0 conn(0x7f41e0072f50 0x7f41e0073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 --2- 172.21.15.46:0/1208274116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e00cd4a0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 -- 172.21.15.46:0/1208274116 >> 172.21.15.46:0/1208274116 conn(0x7f41e00d1c30 msgr2=0x7f41e00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 -- 172.21.15.46:0/1208274116 shutdown_connections 2024-04-26T23:55:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.663+0000 7f41e4a00640 1 -- 172.21.15.46:0/1208274116 wait complete. 2024-04-26T23:55:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.665+0000 7f41e4a00640 1 Processor -- start 2024-04-26T23:55:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.665+0000 7f41e4a00640 1 -- start start 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e0072f50 0x7f41e014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 0x7f41e00aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00cfc10 con 0x7f41e0160520 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00738c0 con 0x7f41e0072f50 2024-04-26T23:55:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.666+0000 7f41e4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41e015c220 con 0x7f41e015be40 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45828/0 (socket says 172.21.15.46:45828) 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 -- 172.21.15.46:0/190224607 learned_addr learned my addr 172.21.15.46:0/190224607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41de800640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e0072f50 0x7f41e014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41d5e00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 0x7f41e00aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 -- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 msgr2=0x7f41e00aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 0x7f41e00aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 -- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e0072f50 msgr2=0x7f41e014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e0072f50 0x7f41e014dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41df200640 1 -- 172.21.15.46:0/190224607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e015d540 con 0x7f41e0160520 2024-04-26T23:55:11.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.667+0000 7f41d5e00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 0x7f41e00aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41df200640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007e60 tx=0x7f41d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8014420 con 0x7f41e0160520 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d8019070 con 0x7f41e0160520 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8014820 con 0x7f41e0160520 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e015d820 con 0x7f41e0160520 2024-04-26T23:55:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.668+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e015dde0 con 0x7f41e0160520 2024-04-26T23:55:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.669+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e0073380 con 0x7f41e0160520 2024-04-26T23:55:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.670+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d80149c0 con 0x7f41e0160520 2024-04-26T23:55:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.671+0000 7f41d7400640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.671+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f41d808c560 con 0x7f41e0160520 2024-04-26T23:55:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.672+0000 7f41de800640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.673+0000 7f41de800640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 0x7f41bc06edb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f41d0000f60 tx=0x7f41d0007d00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.676+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41e0073380 con 0x7f41e0160520 2024-04-26T23:55:11.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.923+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41e00693b0 con 0x7f41e0160520 2024-04-26T23:55:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.924+0000 7f41d7400640 1 -- 172.21.15.46:0/190224607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7f41d805a760 con 0x7f41e0160520 2024-04-26T23:55:11.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:11.926 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:55:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:55:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 msgr2=0x7f41bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 0x7f41bc06edb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f41d0000f60 tx=0x7f41d0007d00 comp rx=0 tx=0).stop 2024-04-26T23:55:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 msgr2=0x7f41e015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f41d8007e60 tx=0x7f41d800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 shutdown_connections 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e0160520 0x7f41e015cdf0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015be40 0x7f41e00aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 --2- 172.21.15.46:0/190224607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e0072f50 0x7f41e014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 >> 172.21.15.46:0/190224607 conn(0x7f41e00d1c30 msgr2=0x7f41e006ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 shutdown_connections 2024-04-26T23:55:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:11.927+0000 7f41e4a00640 1 -- 172.21.15.46:0/190224607 wait complete. 2024-04-26T23:55:11.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.163+0000 7f6403800640 1 Processor -- start 2024-04-26T23:55:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.163+0000 7f6403800640 1 -- start start 2024-04-26T23:55:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.164+0000 7f6403800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63fc162980 0x7f63fc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.164+0000 7f6403800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d31f0 0x7f63fc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.164+0000 7f6403800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63fc15fd30 con 0x7f63fc0d2030 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.164+0000 7f6403800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f63fc162d60 con 0x7f63fc162980 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.164+0000 7f6403800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f63fc0d24a0 con 0x7f63fc0d31f0 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6400e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63fc162980 0x7f63fc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6401800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f63fc0d2030 0x7f63fc15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52128/0 (socket says 172.21.15.46:52128) 2024-04-26T23:55:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6401800640 1 -- 172.21.15.46:0/1712996656 learned_addr learned my addr 172.21.15.46:0/1712996656 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f63f3e00640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d31f0 0x7f63fc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6400e00640 1 -- 172.21.15.46:0/1712996656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d31f0 msgr2=0x7f63fc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6400e00640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d31f0 0x7f63fc0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6400e00640 1 -- 172.21.15.46:0/1712996656 >> v1:172.21.15.46:6789/0 conn(0x7f63fc0d2030 legacy=0x7f63fc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f6400e00640 1 -- 172.21.15.46:0/1712996656 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc0d1050 con 0x7f63fc162980 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.165+0000 7f63fbe00640 1 -- 172.21.15.46:0/1712996656 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122945705 0 0) 0x7f63fc15fd30 con 0x7f63fc0d2030 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.166+0000 7f6400e00640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0d2cb0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f63e40097a0 tx=0x7f63e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cfb262b655b5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f63fbe00640 1 -- 172.21.15.46:0/1712996656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4037070 con 0x7f63fc162980 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f63fbe00640 1 -- 172.21.15.46:0/1712996656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e4032440 con 0x7f63fc162980 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f63fbe00640 1 -- 172.21.15.46:0/1712996656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4032810 con 0x7f63fc162980 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 -- 172.21.15.46:0/1712996656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 msgr2=0x7f63fc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0d2cb0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f63e40097a0 tx=0x7f63e402a180 comp rx=0 tx=0).stop 2024-04-26T23:55:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 -- 172.21.15.46:0/1712996656 shutdown_connections 2024-04-26T23:55:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d31f0 0x7f63fc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 --2- 172.21.15.46:0/1712996656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0d2cb0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.167+0000 7f6403800640 1 -- 172.21.15.46:0/1712996656 >> 172.21.15.46:0/1712996656 conn(0x7f63fc09e420 msgr2=0x7f63fc0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.168+0000 7f6403800640 1 -- 172.21.15.46:0/1712996656 shutdown_connections 2024-04-26T23:55:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.168+0000 7f6403800640 1 -- 172.21.15.46:0/1712996656 wait complete. 2024-04-26T23:55:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.169+0000 7f6403800640 1 Processor -- start 2024-04-26T23:55:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.170+0000 7f6403800640 1 -- start start 2024-04-26T23:55:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d2030 0x7f63fc1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 0x7f63fc1f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1605d0 con 0x7f63fc162980 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1602f0 con 0x7f63fc0d31f0 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6403800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f63fc1608b0 con 0x7f63fc0d2030 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50482/0 (socket says 172.21.15.46:50482) 2024-04-26T23:55:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 -- 172.21.15.46:0/2953749084 learned_addr learned my addr 172.21.15.46:0/2953749084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6400e00640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d2030 0x7f63fc1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f63f3e00640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 0x7f63fc1f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 -- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d2030 msgr2=0x7f63fc1f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d2030 0x7f63fc1f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 -- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 msgr2=0x7f63fc1f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 0x7f63fc1f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f6401800640 1 -- 172.21.15.46:0/2953749084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc1f6790 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.171+0000 7f63f3e00640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 0x7f63fc1f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f6401800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c700 tx=0x7f63f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4018070 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f400ce10 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f400a860 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc0b80a0 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.172+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc0b8660 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.173+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc069530 con 0x7f63fc162980 2024-04-26T23:55:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.175+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63f400aa00 con 0x7f63fc162980 2024-04-26T23:55:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.176+0000 7f63f9600640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.176+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f63f408b410 con 0x7f63fc162980 2024-04-26T23:55:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.176+0000 7f6400e00640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.177+0000 7f6400e00640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f63e4034000 tx=0x7f63e402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.180+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63fc1f6790 con 0x7f63fc162980 2024-04-26T23:55:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.386+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63fc0d15f0 con 0x7f63fc162980 2024-04-26T23:55:12.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.388+0000 7f63f9600640 1 -- 172.21.15.46:0/2953749084 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+17089 (secure 0 0 0) 0x7f63f4059580 con 0x7f63fc162980 2024-04-26T23:55:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:12.391 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":175,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:58.408057+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.21","mappings":[{"from":1,"to":0}]},{"pgid":"26.22","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 msgr2=0x7f63d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f63e4034000 tx=0x7f63e402e000 comp rx=0 tx=0).stop 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 msgr2=0x7f63fc0cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c700 tx=0x7f63f400cbc0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 shutdown_connections 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63fc162980 0x7f63fc0cacb0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63fc0d31f0 0x7f63fc1f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 --2- 172.21.15.46:0/2953749084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63fc0d2030 0x7f63fc1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 >> 172.21.15.46:0/2953749084 conn(0x7f63fc09e420 msgr2=0x7f63fc0b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:12.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 shutdown_connections 2024-04-26T23:55:12.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.391+0000 7f6403800640 1 -- 172.21.15.46:0/2953749084 wait complete. 2024-04-26T23:55:12.413 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:55:12.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.615+0000 7f7064e00640 1 Processor -- start 2024-04-26T23:55:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.615+0000 7f7064e00640 1 -- start start 2024-04-26T23:55:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f7064e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70600720e0 0x7f70600cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f7064e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7060073ad0 con 0x7f70601466b0 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f7064e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7060146a90 con 0x7f7060160910 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f7064e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7060073d70 con 0x7f70600720e0 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f705ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70600720e0 0x7f70600cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f705f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70601466b0 0x7f706006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52138/0 (socket says 172.21.15.46:52138) 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.617+0000 7f705f400640 1 -- 172.21.15.46:0/710322682 learned_addr learned my addr 172.21.15.46:0/710322682 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.618+0000 7f705ea00640 1 -- 172.21.15.46:0/710322682 >> v1:172.21.15.142:6789/0 conn(0x7f7060160910 legacy=0x7f70600cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.618+0000 7f705ea00640 1 -- 172.21.15.46:0/710322682 >> v1:172.21.15.46:6789/0 conn(0x7f70601466b0 legacy=0x7f706006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.618+0000 7f705ea00640 1 -- 172.21.15.46:0/710322682 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7060074050 con 0x7f70600720e0 2024-04-26T23:55:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.619+0000 7f705ea00640 1 --2- 172.21.15.46:0/710322682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70600720e0 0x7f70600cd860 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f70500097d0 tx=0x7f705002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccbfb04f46ae01b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.619+0000 7f705e000640 1 -- 172.21.15.46:0/710322682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050037070 con 0x7f70600720e0 2024-04-26T23:55:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.619+0000 7f705e000640 1 -- 172.21.15.46:0/710322682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7050032440 con 0x7f70600720e0 2024-04-26T23:55:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.619+0000 7f705e000640 1 -- 172.21.15.46:0/710322682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050032810 con 0x7f70600720e0 2024-04-26T23:55:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 -- 172.21.15.46:0/710322682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70600720e0 msgr2=0x7f70600cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 --2- 172.21.15.46:0/710322682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70600720e0 0x7f70600cd860 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f70500097d0 tx=0x7f705002a210 comp rx=0 tx=0).stop 2024-04-26T23:55:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 -- 172.21.15.46:0/710322682 shutdown_connections 2024-04-26T23:55:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 --2- 172.21.15.46:0/710322682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70600720e0 0x7f70600cd860 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 -- 172.21.15.46:0/710322682 >> 172.21.15.46:0/710322682 conn(0x7f70600b26d0 msgr2=0x7f70600b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 -- 172.21.15.46:0/710322682 shutdown_connections 2024-04-26T23:55:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.620+0000 7f7064e00640 1 -- 172.21.15.46:0/710322682 wait complete. 2024-04-26T23:55:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.621+0000 7f7064e00640 1 Processor -- start 2024-04-26T23:55:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.622+0000 7f7064e00640 1 -- start start 2024-04-26T23:55:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.622+0000 7f7064e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70600720e0 0x7f70600b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.622+0000 7f7064e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70601466b0 0x7f70600c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f7064e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f7064e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70600cdda0 con 0x7f7060160910 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f7064e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70600cb470 con 0x7f70600720e0 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f7064e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f70600cc0a0 con 0x7f70601466b0 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50508/0 (socket says 172.21.15.46:50508) 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 -- 172.21.15.46:0/417905111 learned_addr learned my addr 172.21.15.46:0/417905111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705f400640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70600720e0 0x7f70600b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705ea00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70601466b0 0x7f70600c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 -- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70601466b0 msgr2=0x7f70600c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70601466b0 0x7f70600c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 -- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70600720e0 msgr2=0x7f70600b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70600720e0 0x7f70600b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.623+0000 7f705fe00640 1 -- 172.21.15.46:0/417905111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70600b4cf0 con 0x7f7060160910 2024-04-26T23:55:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f705fe00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f705800b990 tx=0x7f705800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058019070 con 0x7f7060160910 2024-04-26T23:55:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705800cba0 con 0x7f7060160910 2024-04-26T23:55:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058014440 con 0x7f7060160910 2024-04-26T23:55:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70600b4fd0 con 0x7f7060160910 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.624+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70600b5590 con 0x7f7060160910 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.626+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70600693b0 con 0x7f7060160910 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.626+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f705801c070 con 0x7f7060160910 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.627+0000 7f7057400640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.627+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f705808bb90 con 0x7f7060160910 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.628+0000 7f705f400640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.629+0000 7f705f400640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 0x7f703c06edb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7048000f60 tx=0x7f7048009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.632+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7058059d00 con 0x7f7060160910 2024-04-26T23:55:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.874+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70600b6810 con 0x7f7060160910 2024-04-26T23:55:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.875+0000 7f7057400640 1 -- 172.21.15.46:0/417905111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7f7058059600 con 0x7f7060160910 2024-04-26T23:55:12.877 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:12.878 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:55:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:55:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 msgr2=0x7f703c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 0x7f703c06edb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7048000f60 tx=0x7f7048009040 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 msgr2=0x7f70601f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f705800b990 tx=0x7f705800be50 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 shutdown_connections 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7060160910 0x7f70601f8a90 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70601466b0 0x7f70600c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 --2- 172.21.15.46:0/417905111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70600720e0 0x7f70600b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 >> 172.21.15.46:0/417905111 conn(0x7f70600b26d0 msgr2=0x7f706006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 shutdown_connections 2024-04-26T23:55:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:12.878+0000 7f7064e00640 1 -- 172.21.15.46:0/417905111 wait complete. 2024-04-26T23:55:12.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.104+0000 7f970f000640 1 Processor -- start 2024-04-26T23:55:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.105+0000 7f970f000640 1 -- start start 2024-04-26T23:55:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.106+0000 7f970f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97080d2130 0x7f97080d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.106+0000 7f970f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97080d11f0 con 0x7f97080b4000 2024-04-26T23:55:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.106+0000 7f970f000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97080d5f20 con 0x7f970815f4e0 2024-04-26T23:55:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.106+0000 7f970f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f97080d0ce0 con 0x7f97080d2130 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f970e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97080b4000 0x7f97080b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52154/0 (socket says 172.21.15.46:52154) 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f970e600640 1 -- 172.21.15.46:0/3514267129 learned_addr learned my addr 172.21.15.46:0/3514267129 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f970d200640 1 --2- 172.21.15.46:0/3514267129 >> v2:172.21.15.142:3300/0 conn(0x7f97080d2130 0x7f97080d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667536615 0 0) 0x7f97080d11f0 con 0x7f97080b4000 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ec003520 con 0x7f97080b4000 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518964131 0 0) 0x7f97080d5f20 con 0x7f970815f4e0 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.107+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97080d11f0 con 0x7f970815f4e0 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1834635201 0 0) 0x7f96ec003520 con 0x7f97080b4000 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 >> v2:172.21.15.142:3300/0 conn(0x7f97080d2130 msgr2=0x7f97080d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 --2- 172.21.15.46:0/3514267129 >> v2:172.21.15.142:3300/0 conn(0x7f97080d2130 0x7f97080d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 >> v1:172.21.15.142:6789/0 conn(0x7f970815f4e0 legacy=0x7f97080d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080d0f10 con 0x7f97080b4000 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f97000019f0 con 0x7f97080b4000 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 >> v1:172.21.15.46:6789/0 conn(0x7f97080b4000 legacy=0x7f97080b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96ec003b40 0x7f96ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 0x7f96ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec0051c0 0x7f96ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00c010 con 0x7f96ec004800 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00c1c0 con 0x7f96ec0051c0 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec00c3c0 con 0x7f96ec003b40 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f970d200640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 0x7f96ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f970dc00640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96ec003b40 0x7f96ec003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.108+0000 7f970e600640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec0051c0 0x7f96ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970d200640 1 -- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96ec003b40 msgr2=0x7f96ec003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970d200640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96ec003b40 0x7f96ec003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970d200640 1 -- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec0051c0 msgr2=0x7f96ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970d200640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec0051c0 0x7f96ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970d200640 1 -- 172.21.15.46:0/3514267129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec003520 con 0x7f96ec004800 2024-04-26T23:55:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.109+0000 7f970e600640 1 -- 172.21.15.46:0/3514267129 reap_dead start 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.110+0000 7f970d200640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 0x7f96ec004c80 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009ac0 tx=0x7f96ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb57a703e1e5dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.110+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4036070 con 0x7f96ec004800 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.110+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f4031400 con 0x7f96ec004800 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.110+0000 7f9707e00640 1 -- 172.21.15.46:0/3514267129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4009690 con 0x7f96ec004800 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 -- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 msgr2=0x7f96ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 0x7f96ec004c80 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009ac0 tx=0x7f96ec005830 comp rx=0 tx=0).stop 2024-04-26T23:55:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 -- 172.21.15.46:0/3514267129 shutdown_connections 2024-04-26T23:55:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 --2- 172.21.15.46:0/3514267129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec004800 0x7f96ec004c80 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 -- 172.21.15.46:0/3514267129 >> 172.21.15.46:0/3514267129 conn(0x7f97080d5010 msgr2=0x7f97080d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 -- 172.21.15.46:0/3514267129 shutdown_connections 2024-04-26T23:55:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.111+0000 7f970f000640 1 -- 172.21.15.46:0/3514267129 wait complete. 2024-04-26T23:55:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.113+0000 7f970f000640 1 Processor -- start 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.113+0000 7f970f000640 1 -- start start 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec004800 0x7f9708159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9708161180 0x7f9708161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97080b1d60 con 0x7f97080c93f0 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97080d2990 con 0x7f96ec004800 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f970815f8c0 con 0x7f9708161180 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50522/0 (socket says 172.21.15.46:50522) 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970d200640 1 -- 172.21.15.46:0/2143665741 learned_addr learned my addr 172.21.15.46:0/2143665741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.114+0000 7f970e600640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9708161180 0x7f9708161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970dc00640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec004800 0x7f9708159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 -- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9708161180 msgr2=0x7f9708161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9708161180 0x7f9708161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 -- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec004800 msgr2=0x7f9708159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec004800 0x7f9708159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 -- 172.21.15.46:0/2143665741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970815a5f0 con 0x7f97080c93f0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970d200640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034040 tx=0x7f96f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f403f020 con 0x7f97080c93f0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f4037040 con 0x7f97080c93f0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4036070 con 0x7f97080c93f0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.115+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97081599e0 con 0x7f97080c93f0 2024-04-26T23:55:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.116+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708159fa0 con 0x7f97080c93f0 2024-04-26T23:55:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.117+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97080693b0 con 0x7f97080c93f0 2024-04-26T23:55:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.118+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96f4042020 con 0x7f97080c93f0 2024-04-26T23:55:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.118+0000 7f9705600640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.118+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f96f40abae0 con 0x7f97080c93f0 2024-04-26T23:55:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.119+0000 7f970dc00640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.120+0000 7f970dc00640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007ae0 tx=0x7f96f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.123+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96f4049070 con 0x7f97080c93f0 2024-04-26T23:55:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.368+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97080d46d0 con 0x7f97080c93f0 2024-04-26T23:55:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.370+0000 7f9705600640 1 -- 172.21.15.46:0/2143665741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5191 (secure 0 0 0) 0x7f97080d46d0 con 0x7f97080c93f0 2024-04-26T23:55:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:13.372 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:54:24.905219+0000","modified":"2024-04-26T23:54:27.780325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10770":{"gid":10770,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/7977112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":7977112},{"type":"v1","addr":"172.21.15.46:6835","nonce":7977112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10770,"qdb_cluster":[10770]},"id":12}]} 2024-04-26T23:55:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 140 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.372+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 msgr2=0x7f96e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.372+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007ae0 tx=0x7f96f8009040 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.372+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 msgr2=0x7f9708155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.372+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034040 tx=0x7f96f4004730 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 shutdown_connections 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9708161180 0x7f9708161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97080c93f0 0x7f9708155540 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 --2- 172.21.15.46:0/2143665741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96ec004800 0x7f9708159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 >> 172.21.15.46:0/2143665741 conn(0x7f97080d5010 msgr2=0x7f9708160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 shutdown_connections 2024-04-26T23:55:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.373+0000 7f970f000640 1 -- 172.21.15.46:0/2143665741 wait complete. 2024-04-26T23:55:13.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:13.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:13.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:13.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.580+0000 7f7ac7000640 1 Processor -- start 2024-04-26T23:55:13.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.580+0000 7f7ac7000640 1 -- start start 2024-04-26T23:55:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7ac7000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ac0073570 0x7f7ac00d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7ac7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac015c5c0 0x7f7ac015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7ac7000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac00cf950 con 0x7f7ac0145770 2024-04-26T23:55:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7ac7000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac00d0c50 con 0x7f7ac0073570 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7ac7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac015d490 con 0x7f7ac015c5c0 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ac0073570 0x7f7ac00d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ac0073570 0x7f7ac00d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50528/0 (socket says 172.21.15.46:50528) 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 -- 172.21.15.46:0/3704732069 learned_addr learned my addr 172.21.15.46:0/3704732069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 -- 172.21.15.46:0/3704732069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac015c5c0 msgr2=0x7f7ac015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 --2- 172.21.15.46:0/3704732069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac015c5c0 0x7f7ac015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 -- 172.21.15.46:0/3704732069 >> v1:172.21.15.142:6789/0 conn(0x7f7ac0145770 legacy=0x7f7ac0072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.582+0000 7f7abf400640 1 -- 172.21.15.46:0/3704732069 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00cfc30 con 0x7f7ac0073570 2024-04-26T23:55:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.583+0000 7f7abf400640 1 --2- 172.21.15.46:0/3704732069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac0073570 0x7f7ac00d0710 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00bb10 tx=0x7f7aac02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b7154974d48497 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7abea00640 1 -- 172.21.15.46:0/3704732069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac032790 con 0x7f7ac0073570 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7abea00640 1 -- 172.21.15.46:0/3704732069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac038070 con 0x7f7ac0073570 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7abea00640 1 -- 172.21.15.46:0/3704732069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac032b50 con 0x7f7ac0073570 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 -- 172.21.15.46:0/3704732069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac0073570 msgr2=0x7f7ac00d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 --2- 172.21.15.46:0/3704732069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac0073570 0x7f7ac00d0710 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00bb10 tx=0x7f7aac02c8d0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 -- 172.21.15.46:0/3704732069 shutdown_connections 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 --2- 172.21.15.46:0/3704732069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac015c5c0 0x7f7ac015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 --2- 172.21.15.46:0/3704732069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac0073570 0x7f7ac00d0710 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 -- 172.21.15.46:0/3704732069 >> 172.21.15.46:0/3704732069 conn(0x7f7ac006c530 msgr2=0x7f7ac006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 -- 172.21.15.46:0/3704732069 shutdown_connections 2024-04-26T23:55:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.584+0000 7f7ac7000640 1 -- 172.21.15.46:0/3704732069 wait complete. 2024-04-26T23:55:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.586+0000 7f7ac7000640 1 Processor -- start 2024-04-26T23:55:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.586+0000 7f7ac7000640 1 -- start start 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac0073570 0x7f7ac01e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 0x7f7ac013b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac00d01f0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac015cf60 con 0x7f7ac0073570 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac7000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac00cf5b0 con 0x7f7ac0145770 2024-04-26T23:55:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50544/0 (socket says 172.21.15.46:50544) 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.587+0000 7f7ac5000640 1 -- 172.21.15.46:0/663437665 learned_addr learned my addr 172.21.15.46:0/663437665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7abfe00640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac0073570 0x7f7ac01e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7abf400640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 0x7f7ac013b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 -- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 msgr2=0x7f7ac013b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 0x7f7ac013b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 -- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac0073570 msgr2=0x7f7ac01e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac0073570 0x7f7ac01e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 -- 172.21.15.46:0/663437665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac01e54c0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7abf400640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 0x7f7ac013b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ac5000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b970 tx=0x7f7ab800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.588+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8019070 con 0x7f7ac015c5c0 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.589+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab800cb60 con 0x7f7ac015c5c0 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.589+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac01e57a0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.589+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8014440 con 0x7f7ac015c5c0 2024-04-26T23:55:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.589+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac01e5d60 con 0x7f7ac015c5c0 2024-04-26T23:55:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.590+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac01e6530 con 0x7f7ac015c5c0 2024-04-26T23:55:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.591+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ab801c070 con 0x7f7ac015c5c0 2024-04-26T23:55:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.592+0000 7f7ab7e00640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 0x7f7aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.592+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f7ab808ba10 con 0x7f7ac015c5c0 2024-04-26T23:55:13.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.592+0000 7f7abfe00640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 0x7f7aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:13.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.593+0000 7f7abfe00640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 0x7f7aa006edb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00017f0 tx=0x7f7ab0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.596+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ab80595c0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.802+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac015bfe0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.804+0000 7f7ab7e00640 1 -- 172.21.15.46:0/663437665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+17089 (secure 0 0 0) 0x7f7ab805d8c0 con 0x7f7ac015c5c0 2024-04-26T23:55:13.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":175,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:54:58.408057+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:54:22.516305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-26T23:54:23.663985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.21","mappings":[{"from":1,"to":0}]},{"pgid":"26.22","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 msgr2=0x7f7aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 0x7f7aa006edb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00017f0 tx=0x7f7ab0009040 comp rx=0 tx=0).stop 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 msgr2=0x7f7ac013ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b970 tx=0x7f7ab800be30 comp rx=0 tx=0).stop 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 shutdown_connections 2024-04-26T23:55:13.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7aa006c8f0 0x7f7aa006edb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac015c5c0 0x7f7ac013ca30 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac0145770 0x7f7ac013b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 --2- 172.21.15.46:0/663437665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac0073570 0x7f7ac01e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 >> 172.21.15.46:0/663437665 conn(0x7f7ac006c530 msgr2=0x7f7ac006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 shutdown_connections 2024-04-26T23:55:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:13.807+0000 7f7ac7000640 1 -- 172.21.15.46:0/663437665 wait complete. 2024-04-26T23:55:13.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:55:14.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.084+0000 7f46fb200640 1 Processor -- start 2024-04-26T23:55:14.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.084+0000 7f46fb200640 1 -- start start 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.085+0000 7f46fb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46f41466b0 0x7f46f406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.085+0000 7f46fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.085+0000 7f46fb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f4073ad0 con 0x7f46f40720e0 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.085+0000 7f46fb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f46f4146a90 con 0x7f46f41466b0 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.085+0000 7f46fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46f4073d70 con 0x7f46f4160910 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46f41466b0 0x7f46f406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f46f40720e0 0x7f46f40cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52166/0 (socket says 172.21.15.46:52166) 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9400640 1 -- 172.21.15.46:0/373314357 learned_addr learned my addr 172.21.15.46:0/373314357 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:14.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46fa800640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9e00640 1 -- 172.21.15.46:0/373314357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 msgr2=0x7f46f4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:14.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9e00640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f4160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9e00640 1 -- 172.21.15.46:0/373314357 >> v1:172.21.15.46:6789/0 conn(0x7f46f40720e0 legacy=0x7f46f40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.086+0000 7f46f9e00640 1 -- 172.21.15.46:0/373314357 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f4074050 con 0x7f46f41466b0 2024-04-26T23:55:14.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.087+0000 7f46f9e00640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f406de80 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f46e80097d0 tx=0x7f46e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289572b2b4e52c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.088+0000 7f46f8a00640 1 -- 172.21.15.46:0/373314357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8037070 con 0x7f46f41466b0 2024-04-26T23:55:14.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.088+0000 7f46f8a00640 1 -- 172.21.15.46:0/373314357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8032440 con 0x7f46f41466b0 2024-04-26T23:55:14.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.088+0000 7f46f8a00640 1 -- 172.21.15.46:0/373314357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8032810 con 0x7f46f41466b0 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.088+0000 7f46fb200640 1 -- 172.21.15.46:0/373314357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 msgr2=0x7f46f406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.088+0000 7f46fb200640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f406de80 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f46e80097d0 tx=0x7f46e802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 -- 172.21.15.46:0/373314357 shutdown_connections 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f4160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 --2- 172.21.15.46:0/373314357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f406de80 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 -- 172.21.15.46:0/373314357 >> 172.21.15.46:0/373314357 conn(0x7f46f40b26d0 msgr2=0x7f46f40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 -- 172.21.15.46:0/373314357 shutdown_connections 2024-04-26T23:55:14.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.089+0000 7f46fb200640 1 -- 172.21.15.46:0/373314357 wait complete. 2024-04-26T23:55:14.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.090+0000 7f46fb200640 1 Processor -- start 2024-04-26T23:55:14.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.091+0000 7f46fb200640 1 -- start start 2024-04-26T23:55:14.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.091+0000 7f46fb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46f40720e0 0x7f46f40b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.091+0000 7f46fb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46fb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f41e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46fb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406e3c0 con 0x7f46f41466b0 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46fb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40cb470 con 0x7f46f40720e0 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46fb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40cb990 con 0x7f46f4160910 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50558/0 (socket says 172.21.15.46:50558) 2024-04-26T23:55:14.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 -- 172.21.15.46:0/3054886013 learned_addr learned my addr 172.21.15.46:0/3054886013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:14.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 -- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 msgr2=0x7f46f41e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f41e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 -- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46f40720e0 msgr2=0x7f46f40b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46f40720e0 0x7f46f40b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.092+0000 7f46f9400640 1 -- 172.21.15.46:0/3054886013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f41f6800 con 0x7f46f41466b0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46f9400640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c400 tx=0x7f46e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4017070 con 0x7f46f41466b0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4005ce0 con 0x7f46f41466b0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40072f0 con 0x7f46f41466b0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f40b5380 con 0x7f46f41466b0 2024-04-26T23:55:14.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.093+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40b5940 con 0x7f46f41466b0 2024-04-26T23:55:14.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.095+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f41f51f0 con 0x7f46f41466b0 2024-04-26T23:55:14.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.095+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46e401a070 con 0x7f46f41466b0 2024-04-26T23:55:14.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.096+0000 7f46ee000640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 0x7f46d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.096+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6246+0+0 (secure 0 0 0) 0x7f46e408b180 con 0x7f46f41466b0 2024-04-26T23:55:14.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.097+0000 7f46f9e00640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 0x7f46d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.098+0000 7f46f9e00640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 0x7f46d806edb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f46e8034000 tx=0x7f46e802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.102+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46e4059390 con 0x7f46f41466b0 2024-04-26T23:55:14.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.344+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46f40c7180 con 0x7f46f41466b0 2024-04-26T23:55:14.347 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:55:14.345+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:55:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.349+0000 7f46ee000640 1 -- 172.21.15.46:0/3054886013 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v141) ==== 161+0+0 (secure 0 0 0) 0x7f46e4058c90 con 0x7f46f41466b0 2024-04-26T23:55:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:55:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 msgr2=0x7f46d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 0x7f46d806edb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f46e8034000 tx=0x7f46e802c000 comp rx=0 tx=0).stop 2024-04-26T23:55:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 msgr2=0x7f46f41f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c400 tx=0x7f46e400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 shutdown_connections 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f46d806c8f0 0x7f46d806edb0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46f4160910 0x7f46f41e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46f41466b0 0x7f46f41f62c0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 --2- 172.21.15.46:0/3054886013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46f40720e0 0x7f46f40b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 >> 172.21.15.46:0/3054886013 conn(0x7f46f40b26d0 msgr2=0x7f46f40ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.353+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 shutdown_connections 2024-04-26T23:55:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.354+0000 7f46fb200640 1 -- 172.21.15.46:0/3054886013 wait complete. 2024-04-26T23:55:14.368 INFO:tasks.ceph.mds.a.smithi046.stderr: -3701> 2024-04-26T23:54:29.141+0000 7fad74000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:55:14.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:55:14.447 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:55:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.572+0000 7f270be00640 1 Processor -- start 2024-04-26T23:55:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.573+0000 7f270be00640 1 -- start start 2024-04-26T23:55:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.574+0000 7f270be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27040d04c0 0x7f270406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.574+0000 7f270be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.574+0000 7f270be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27040b4b30 con 0x7f270406f2e0 2024-04-26T23:55:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.574+0000 7f270be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f27040b4e10 con 0x7f27040d04c0 2024-04-26T23:55:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.574+0000 7f270be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27040b50f0 con 0x7f27040b44f0 2024-04-26T23:55:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.575+0000 7f2709e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.575+0000 7f2709e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:50714/0 (socket says 172.21.15.46:50714) 2024-04-26T23:55:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.575+0000 7f2709e00640 1 -- 172.21.15.46:0/3335008695 learned_addr learned my addr 172.21.15.46:0/3335008695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.575+0000 7f2709400640 1 --2- 172.21.15.46:0/3335008695 >> v2:172.21.15.142:3300/0 conn(0x7f27040d04c0 0x7f270406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.576+0000 7f2709e00640 1 -- 172.21.15.46:0/3335008695 >> v2:172.21.15.142:3300/0 conn(0x7f27040d04c0 msgr2=0x7f270406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.576+0000 7f2709e00640 1 --2- 172.21.15.46:0/3335008695 >> v2:172.21.15.142:3300/0 conn(0x7f27040d04c0 0x7f270406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.576+0000 7f2709e00640 1 -- 172.21.15.46:0/3335008695 >> v1:172.21.15.142:6789/0 conn(0x7f270406f2e0 legacy=0x7f27040b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.576+0000 7f2709e00640 1 -- 172.21.15.46:0/3335008695 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27040b53d0 con 0x7f27040b44f0 2024-04-26T23:55:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.577+0000 7f2709e00640 1 --2- 172.21.15.46:0/3335008695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009e50 tx=0x7f26fc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f270fa29e85131 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.578+0000 7f2703e00640 1 -- 172.21.15.46:0/3335008695 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc036070 con 0x7f27040b44f0 2024-04-26T23:55:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.578+0000 7f2703e00640 1 -- 172.21.15.46:0/3335008695 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc031600 con 0x7f27040b44f0 2024-04-26T23:55:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.578+0000 7f2703e00640 1 -- 172.21.15.46:0/3335008695 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0319d0 con 0x7f27040b44f0 2024-04-26T23:55:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.578+0000 7f270be00640 1 -- 172.21.15.46:0/3335008695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 msgr2=0x7f27040ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.578+0000 7f270be00640 1 --2- 172.21.15.46:0/3335008695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009e50 tx=0x7f26fc02a680 comp rx=0 tx=0).stop 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 -- 172.21.15.46:0/3335008695 shutdown_connections 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 --2- 172.21.15.46:0/3335008695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27040b44f0 0x7f27040ce470 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 --2- 172.21.15.46:0/3335008695 >> v2:172.21.15.142:3300/0 conn(0x7f27040d04c0 0x7f270406eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 -- 172.21.15.46:0/3335008695 >> 172.21.15.46:0/3335008695 conn(0x7f270406c980 msgr2=0x7f27040b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 -- 172.21.15.46:0/3335008695 shutdown_connections 2024-04-26T23:55:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.579+0000 7f270be00640 1 -- 172.21.15.46:0/3335008695 wait complete. 2024-04-26T23:55:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.581+0000 7f270be00640 1 Processor -- start 2024-04-26T23:55:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.581+0000 7f270be00640 1 -- start start 2024-04-26T23:55:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 0x7f270415e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27040b44f0 0x7f270415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27040ceb30 con 0x7f27040d04c0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27040ce9b0 con 0x7f27040b44f0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f270be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f27040cecb0 con 0x7f270406f2e0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50588/0 (socket says 172.21.15.46:50588) 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 -- 172.21.15.46:0/4193303438 learned_addr learned my addr 172.21.15.46:0/4193303438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709400640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 0x7f270415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2708a00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27040b44f0 0x7f270415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 -- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 msgr2=0x7f270415e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 0x7f270415e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 -- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27040b44f0 msgr2=0x7f270415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.582+0000 7f2709e00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27040b44f0 0x7f270415edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2709e00640 1 -- 172.21.15.46:0/4193303438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270415f300 con 0x7f27040d04c0 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2709400640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 0x7f270415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2709e00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f26fc02ac70 tx=0x7f26fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc041030 con 0x7f27040d04c0 2024-04-26T23:55:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc031400 con 0x7f27040d04c0 2024-04-26T23:55:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270415d270 con 0x7f27040d04c0 2024-04-26T23:55:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc036040 con 0x7f27040d04c0 2024-04-26T23:55:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.583+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270415d830 con 0x7f27040d04c0 2024-04-26T23:55:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.585+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2704069530 con 0x7f27040d04c0 2024-04-26T23:55:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.586+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26fc044070 con 0x7f27040d04c0 2024-04-26T23:55:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.587+0000 7f2701600640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 0x7f26e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.587+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 6332+0+0 (secure 0 0 0) 0x7f26fc0ac160 con 0x7f27040d04c0 2024-04-26T23:55:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.588+0000 7f2709400640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 0x7f26e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.589+0000 7f2709400640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 0x7f26e406ede0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009a20 tx=0x7f26f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.592+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26fc07a280 con 0x7f27040d04c0 2024-04-26T23:55:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:14.839+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f27040cf460 con 0x7f27040d04c0 2024-04-26T23:55:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.354+0000 7f2701600640 1 -- 172.21.15.46:0/4193303438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v142) ==== 104+0+0 (secure 0 0 0) 0x7f26fc079b80 con 0x7f27040d04c0 2024-04-26T23:55:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 msgr2=0x7f26e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 0x7f26e406ede0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009a20 tx=0x7f26f8008040 comp rx=0 tx=0).stop 2024-04-26T23:55:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 msgr2=0x7f2704157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f26fc02ac70 tx=0x7f26fc004a40 comp rx=0 tx=0).stop 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 shutdown_connections 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f26e406c920 0x7f26e406ede0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27040d04c0 0x7f2704157550 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27040b44f0 0x7f270415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 --2- 172.21.15.46:0/4193303438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f270406f2e0 0x7f270415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.357+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 >> 172.21.15.46:0/4193303438 conn(0x7f270406c980 msgr2=0x7f270413e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.358+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 shutdown_connections 2024-04-26T23:55:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.358+0000 7f270be00640 1 -- 172.21.15.46:0/4193303438 wait complete. 2024-04-26T23:55:15.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:55:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.600+0000 7f9c13200640 1 Processor -- start 2024-04-26T23:55:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.600+0000 7f9c13200640 1 -- start start 2024-04-26T23:55:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.601+0000 7f9c13200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 0x7f9c0c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.601+0000 7f9c13200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 0x7f9c0c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.602+0000 7f9c13200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c0b0520 con 0x7f9c0c0d11b0 2024-04-26T23:55:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.602+0000 7f9c13200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c0b0800 con 0x7f9c0c0afad0 2024-04-26T23:55:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.602+0000 7f9c13200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0b0ae0 con 0x7f9c0c15d160 2024-04-26T23:55:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.602+0000 7f9c11400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 0x7f9c0c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.603+0000 7f9c12800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f9c0c0d11b0 0x7f9c0c0b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:53680/0 (socket says 172.21.15.46:53680) 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.603+0000 7f9c12800640 1 -- 172.21.15.46:0/3801227095 learned_addr learned my addr 172.21.15.46:0/3801227095 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.603+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3801227095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 0x7f9c0c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.604+0000 7f9c11e00640 1 -- 172.21.15.46:0/3801227095 >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 msgr2=0x7f9c0c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.604+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3801227095 >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 0x7f9c0c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.604+0000 7f9c11e00640 1 -- 172.21.15.46:0/3801227095 >> v1:172.21.15.142:6789/0 conn(0x7f9c0c0d11b0 legacy=0x7f9c0c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.604+0000 7f9c11e00640 1 -- 172.21.15.46:0/3801227095 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c0b0dc0 con 0x7f9c0c15d160 2024-04-26T23:55:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.604+0000 7f9c11400640 1 --2- 172.21.15.46:0/3801227095 >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 0x7f9c0c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.605+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3801227095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 0x7f9c0c0cd740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009d90 tx=0x7f9c0402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79af6b9df0f494a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.605+0000 7f9c10a00640 1 -- 172.21.15.46:0/3801227095 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04007740 con 0x7f9c0c15d160 2024-04-26T23:55:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c10a00640 1 -- 172.21.15.46:0/3801227095 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c040078e0 con 0x7f9c0c15d160 2024-04-26T23:55:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c10a00640 1 -- 172.21.15.46:0/3801227095 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04007cb0 con 0x7f9c0c15d160 2024-04-26T23:55:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 -- 172.21.15.46:0/3801227095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 msgr2=0x7f9c0c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 --2- 172.21.15.46:0/3801227095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 0x7f9c0c0cd740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009d90 tx=0x7f9c0402a660 comp rx=0 tx=0).stop 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 -- 172.21.15.46:0/3801227095 shutdown_connections 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 --2- 172.21.15.46:0/3801227095 >> v2:172.21.15.142:3300/0 conn(0x7f9c0c0afad0 0x7f9c0c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 --2- 172.21.15.46:0/3801227095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c15d160 0x7f9c0c0cd740 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 -- 172.21.15.46:0/3801227095 >> 172.21.15.46:0/3801227095 conn(0x7f9c0c09ba70 msgr2=0x7f9c0c09be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 -- 172.21.15.46:0/3801227095 shutdown_connections 2024-04-26T23:55:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.606+0000 7f9c13200640 1 -- 172.21.15.46:0/3801227095 wait complete. 2024-04-26T23:55:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.608+0000 7f9c13200640 1 Processor -- start 2024-04-26T23:55:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.608+0000 7f9c13200640 1 -- start start 2024-04-26T23:55:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 0x7f9c0c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c0c0d11b0 0x7f9c0c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c071b70 con 0x7f9c0c15d160 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0719f0 con 0x7f9c0c0d11b0 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.609+0000 7f9c13200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0d15c0 con 0x7f9c0c0afad0 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50600/0 (socket says 172.21.15.46:50600) 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 -- 172.21.15.46:0/3835896450 learned_addr learned my addr 172.21.15.46:0/3835896450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c11400640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c0c0d11b0 0x7f9c0c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 0x7f9c0c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 -- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 msgr2=0x7f9c0c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 0x7f9c0c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 -- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c0c0d11b0 msgr2=0x7f9c0c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c0c0d11b0 0x7f9c0c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 -- 172.21.15.46:0/3835896450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c153170 con 0x7f9c0c15d160 2024-04-26T23:55:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 0x7f9c0c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.610+0000 7f9c12800640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c420 tx=0x7f9c0800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.611+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08017070 con 0x7f9c0c15d160 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.611+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08005ce0 con 0x7f9c0c15d160 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.611+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080072d0 con 0x7f9c0c15d160 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.611+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c0c9fd0 con 0x7f9c0c15d160 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.611+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c0ca590 con 0x7f9c0c15d160 2024-04-26T23:55:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.612+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c069ac0 con 0x7f9c0c15d160 2024-04-26T23:55:15.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.613+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c0801a070 con 0x7f9c0c15d160 2024-04-26T23:55:15.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.614+0000 7f9c02000640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 0x7f9bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:15.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.614+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 6332+0+0 (secure 0 0 0) 0x7f9c0808b0e0 con 0x7f9c0c15d160 2024-04-26T23:55:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.615+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 0x7f9bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.616+0000 7f9c11e00640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 0x7f9bf006edb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c04033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:15.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.619+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c08058bb0 con 0x7f9c0c15d160 2024-04-26T23:55:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:15.827+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c0c06f450 con 0x7f9c0c15d160 2024-04-26T23:55:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.528+0000 7f9c02000640 1 -- 172.21.15.46:0/3835896450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v177) ==== 181+0+0 (secure 0 0 0) 0x7f9c0805ceb0 con 0x7f9c0c15d160 2024-04-26T23:55:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:55:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.533+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 msgr2=0x7f9bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.533+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 0x7f9bf006edb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c04033040 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.533+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 msgr2=0x7f9c0c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.533+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c420 tx=0x7f9c0800c8e0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 shutdown_connections 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9bf006c8f0 0x7f9bf006edb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0c15d160 0x7f9c0c0cc240 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c0c0d11b0 0x7f9c0c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 --2- 172.21.15.46:0/3835896450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c0c0afad0 0x7f9c0c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.534+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 >> 172.21.15.46:0/3835896450 conn(0x7f9c0c09ba70 msgr2=0x7f9c0c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.536+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 shutdown_connections 2024-04-26T23:55:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.536+0000 7f9c13200640 1 -- 172.21.15.46:0/3835896450 wait complete. 2024-04-26T23:55:16.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:55:16.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.766+0000 7f63fd200640 1 Processor -- start 2024-04-26T23:55:16.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.767+0000 7f63fd200640 1 -- start start 2024-04-26T23:55:16.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.768+0000 7f63fd200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63f80d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.768+0000 7f63fd200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f80d01a0 con 0x7f63f80d3cc0 2024-04-26T23:55:16.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.768+0000 7f63fd200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f80d0480 con 0x7f63f81627f0 2024-04-26T23:55:16.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.768+0000 7f63fd200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f63f80d25c0 con 0x7f63f80cfbc0 2024-04-26T23:55:16.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.769+0000 7f63f6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f63f80d3cc0 0x7f63f80b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52182/0 (socket says 172.21.15.46:52182) 2024-04-26T23:55:16.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.769+0000 7f63f6a00640 1 -- 172.21.15.46:0/1804834381 learned_addr learned my addr 172.21.15.46:0/1804834381 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.769+0000 7f63f7e00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63f80d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.769+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123885629 0 0) 0x7f63f80d01a0 con 0x7f63f80d3cc0 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.769+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63dc003520 con 0x7f63f80d3cc0 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2535182500 0 0) 0x7f63dc003520 con 0x7f63f80d3cc0 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 msgr2=0x7f63f80d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63f80d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 >> v1:172.21.15.142:6789/0 conn(0x7f63f81627f0 legacy=0x7f63f80d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f80d0760 con 0x7f63f80d3cc0 2024-04-26T23:55:16.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f63e4001670 con 0x7f63f80d3cc0 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 >> v1:172.21.15.46:6789/0 conn(0x7f63f80d3cc0 legacy=0x7f63f80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63f80d25c0 con 0x7f63dc004d40 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc00ba20 con 0x7f63dc003f20 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc00bc20 con 0x7f63f80cfbc0 2024-04-26T23:55:16.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.770+0000 7f63f7e00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f6a00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7400640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7e00640 1 -- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 msgr2=0x7f63dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7e00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63f80cfbc0 0x7f63dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7e00640 1 -- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63dc003f20 msgr2=0x7f63dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7e00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.771+0000 7f63f7e00640 1 -- 172.21.15.46:0/1804834381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc003520 con 0x7f63dc004d40 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.772+0000 7f63f7e00640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009fc0 tx=0x7f63dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176fc68578e60cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.772+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0009c10 con 0x7f63dc004d40 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.772+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0036070 con 0x7f63dc004d40 2024-04-26T23:55:16.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.772+0000 7f63f6000640 1 -- 172.21.15.46:0/1804834381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0031880 con 0x7f63dc004d40 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 -- 172.21.15.46:0/1804834381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63dc004d40 msgr2=0x7f63dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 --2- 172.21.15.46:0/1804834381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009fc0 tx=0x7f63dc005830 comp rx=0 tx=0).stop 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63f7e00640 1 -- 172.21.15.46:0/1804834381 reap_dead start 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 -- 172.21.15.46:0/1804834381 shutdown_connections 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 -- 172.21.15.46:0/1804834381 >> 172.21.15.46:0/1804834381 conn(0x7f63f809e380 msgr2=0x7f63f80d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 -- 172.21.15.46:0/1804834381 shutdown_connections 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.773+0000 7f63fd200640 1 -- 172.21.15.46:0/1804834381 wait complete. 2024-04-26T23:55:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.775+0000 7f63fd200640 1 Processor -- start 2024-04-26T23:55:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.775+0000 7f63fd200640 1 -- start start 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63dc004d40 0x7f63f81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f81f8810 0x7f63f81f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8162bd0 con 0x7f63f8160670 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f63f80d0d20 con 0x7f63f81f8810 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63fd200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f63f80d2080 con 0x7f63dc004d40 2024-04-26T23:55:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50626/0 (socket says 172.21.15.46:50626) 2024-04-26T23:55:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.776+0000 7f63f7e00640 1 -- 172.21.15.46:0/3645057986 learned_addr learned my addr 172.21.15.46:0/3645057986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 -- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63dc004d40 msgr2=0x7f63f81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63dc004d40 0x7f63f81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f6a00640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f81f8810 0x7f63f81f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 -- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f81f8810 msgr2=0x7f63f81f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f81f8810 0x7f63f81f8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 -- 172.21.15.46:0/3645057986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f8159af0 con 0x7f63f8160670 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63f7e00640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f63f0033040 tx=0x7f63f0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.777+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0009c10 con 0x7f63f8160670 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.778+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0031df0 con 0x7f63f8160670 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.778+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0003bd0 con 0x7f63f8160670 2024-04-26T23:55:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.778+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81f93d0 con 0x7f63f8160670 2024-04-26T23:55:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.778+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f80c8af0 con 0x7f63f8160670 2024-04-26T23:55:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.779+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f80693b0 con 0x7f63f8160670 2024-04-26T23:55:16.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.780+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63f0005d10 con 0x7f63f8160670 2024-04-26T23:55:16.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.781+0000 7f63ef400640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.782+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 5894+0+0 (secure 0 0 0) 0x7f63f00acbd0 con 0x7f63f8160670 2024-04-26T23:55:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.782+0000 7f63f7400640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.785+0000 7f63f7400640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007fd0 tx=0x7f63e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.789+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63f8159af0 con 0x7f63f8160670 2024-04-26T23:55:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:16.997+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63f80d53e0 con 0x7f63f8160670 2024-04-26T23:55:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.532+0000 7f63ef400640 1 -- 172.21.15.46:0/3645057986 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v178) ==== 169+0+0 (secure 0 0 0) 0x7f63f007afd0 con 0x7f63f8160670 2024-04-26T23:55:17.534 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:55:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.537+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 msgr2=0x7f63d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.537+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007fd0 tx=0x7f63e8009040 comp rx=0 tx=0).stop 2024-04-26T23:55:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.537+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 msgr2=0x7f63f8160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.537+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f63f0033040 tx=0x7f63f0009750 comp rx=0 tx=0).stop 2024-04-26T23:55:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 shutdown_connections 2024-04-26T23:55:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f63d806c8f0 0x7f63d806edb0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8160670 0x7f63f8160af0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f81f8810 0x7f63f81f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 --2- 172.21.15.46:0/3645057986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63dc004d40 0x7f63f81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.540+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 >> 172.21.15.46:0/3645057986 conn(0x7f63f809e380 msgr2=0x7f63f80b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:17.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.542+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 shutdown_connections 2024-04-26T23:55:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.543+0000 7f63fd200640 1 -- 172.21.15.46:0/3645057986 wait complete. 2024-04-26T23:55:17.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:55:17.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.758+0000 7fb30be00640 1 Processor -- start 2024-04-26T23:55:17.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.758+0000 7fb30be00640 1 -- start start 2024-04-26T23:55:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 0x7fb30c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb30c0d0330 0x7fb30c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb30c146b20 con 0x7fb30c0ce670 2024-04-26T23:55:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb30c0ccf30 con 0x7fb30c0d0330 2024-04-26T23:55:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c160af0 con 0x7fb30c146740 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.760+0000 7fb30aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 0x7fb30c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30b400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fb30c0ce670 0x7fb30c0ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:53696/0 (socket says 172.21.15.46:53696) 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb30c0d0330 0x7fb30c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30b400640 1 -- 172.21.15.46:0/2448323772 learned_addr learned my addr 172.21.15.46:0/2448323772 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 0x7fb30c15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:50776/0 (socket says 172.21.15.46:50776) 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30a000640 1 -- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 msgr2=0x7fb30c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30a000640 1 --2- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 0x7fb30c15df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30a000640 1 -- 172.21.15.46:0/2448323772 >> v1:172.21.15.142:6789/0 conn(0x7fb30c0ce670 legacy=0x7fb30c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.761+0000 7fb30a000640 1 -- 172.21.15.46:0/2448323772 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c160dd0 con 0x7fb30c0d0330 2024-04-26T23:55:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.763+0000 7fb30a000640 1 --2- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c1612f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097a0 tx=0x7fb2f802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48242fdd5f21dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:17.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb309600640 1 -- 172.21.15.46:0/2448323772 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8037070 con 0x7fb30c0d0330 2024-04-26T23:55:17.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb309600640 1 -- 172.21.15.46:0/2448323772 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f8032440 con 0x7fb30c0d0330 2024-04-26T23:55:17.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb309600640 1 -- 172.21.15.46:0/2448323772 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8032810 con 0x7fb30c0d0330 2024-04-26T23:55:17.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 -- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 msgr2=0x7fb30c1612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 --2- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c1612f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097a0 tx=0x7fb2f802a180 comp rx=0 tx=0).stop 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 -- 172.21.15.46:0/2448323772 shutdown_connections 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 --2- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c1612f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 --2- 172.21.15.46:0/2448323772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c146740 0x7fb30c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.764+0000 7fb30be00640 1 -- 172.21.15.46:0/2448323772 >> 172.21.15.46:0/2448323772 conn(0x7fb30c0b26d0 msgr2=0x7fb30c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.765+0000 7fb30be00640 1 -- 172.21.15.46:0/2448323772 shutdown_connections 2024-04-26T23:55:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.765+0000 7fb30be00640 1 -- 172.21.15.46:0/2448323772 wait complete. 2024-04-26T23:55:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.767+0000 7fb30be00640 1 Processor -- start 2024-04-26T23:55:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.767+0000 7fb30be00640 1 -- start start 2024-04-26T23:55:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c0ce670 0x7fb30c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0cccf0 con 0x7fb30c146740 2024-04-26T23:55:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0ccb70 con 0x7fb30c0d0330 2024-04-26T23:55:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.768+0000 7fb30be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c161830 con 0x7fb30c0ce670 2024-04-26T23:55:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50638/0 (socket says 172.21.15.46:50638) 2024-04-26T23:55:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 -- 172.21.15.46:0/672169049 learned_addr learned my addr 172.21.15.46:0/672169049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30a000640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30aa00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c0ce670 0x7fb30c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 -- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c0ce670 msgr2=0x7fb30c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c0ce670 0x7fb30c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 -- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 msgr2=0x7fb30c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.769+0000 7fb30b400640 1 -- 172.21.15.46:0/672169049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c0c7ca0 con 0x7fb30c146740 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb30b400640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fb30400cbc0 tx=0x7fb30400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304017070 con 0x7fb30c146740 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30400ad40 con 0x7fb30c146740 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304012600 con 0x7fb30c146740 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c0c7f80 con 0x7fb30c146740 2024-04-26T23:55:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.770+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c0c8540 con 0x7fb30c146740 2024-04-26T23:55:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.772+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c0c8a40 con 0x7fb30c146740 2024-04-26T23:55:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.773+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb30401a070 con 0x7fb30c146740 2024-04-26T23:55:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.774+0000 7fb302a00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 0x7fb2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.774+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7fb30408b620 con 0x7fb30c146740 2024-04-26T23:55:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.774+0000 7fb30aa00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 0x7fb2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.775+0000 7fb30aa00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 0x7fb2ec06edb0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009a30 tx=0x7fb2fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:17.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.778+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb30c0c8a40 con 0x7fb30c146740 2024-04-26T23:55:17.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.985+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb30c0693b0 con 0x7fb30c146740 2024-04-26T23:55:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.992+0000 7fb302a00640 1 -- 172.21.15.46:0/672169049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7fb304059600 con 0x7fb30c146740 2024-04-26T23:55:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 msgr2=0x7fb2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 0x7fb2ec06edb0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009a30 tx=0x7fb2fc008040 comp rx=0 tx=0).stop 2024-04-26T23:55:17.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 msgr2=0x7fb30c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fb30400cbc0 tx=0x7fb30400a430 comp rx=0 tx=0).stop 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 shutdown_connections 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb2ec06c8f0 0x7fb2ec06edb0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c146740 0x7fb30c15bfd0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb30c0d0330 0x7fb30c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 --2- 172.21.15.46:0/672169049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb30c0ce670 0x7fb30c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 >> 172.21.15.46:0/672169049 conn(0x7fb30c0b26d0 msgr2=0x7fb30c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.995+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 shutdown_connections 2024-04-26T23:55:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:17.996+0000 7fb30be00640 1 -- 172.21.15.46:0/672169049 wait complete. 2024-04-26T23:55:18.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-26T23:55:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.229+0000 7fcf85000640 1 Processor -- start 2024-04-26T23:55:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.229+0000 7fcf85000640 1 -- start start 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf85000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf801466b0 0x7fcf8006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf85000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf80160910 0x7fcf80160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf85000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf80073ad0 con 0x7fcf800720e0 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf85000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcf801612a0 con 0x7fcf801466b0 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf85000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf80146a90 con 0x7fcf80160910 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf7ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf800720e0 0x7fcf800cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52190/0 (socket says 172.21.15.46:52190) 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf7ea00640 1 -- 172.21.15.46:0/524611156 learned_addr learned my addr 172.21.15.46:0/524611156 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.231+0000 7fcf7f400640 1 --2- 172.21.15.46:0/524611156 >> v2:172.21.15.46:3300/0 conn(0x7fcf801466b0 0x7fcf8006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7fe00640 1 --2- 172.21.15.46:0/524611156 >> v2:172.21.15.142:3300/0 conn(0x7fcf80160910 0x7fcf80160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7f400640 1 -- 172.21.15.46:0/524611156 >> v2:172.21.15.142:3300/0 conn(0x7fcf80160910 msgr2=0x7fcf80160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7f400640 1 --2- 172.21.15.46:0/524611156 >> v2:172.21.15.142:3300/0 conn(0x7fcf80160910 0x7fcf80160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7f400640 1 -- 172.21.15.46:0/524611156 >> v1:172.21.15.46:6789/0 conn(0x7fcf800720e0 legacy=0x7fcf800cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7e000640 1 -- 172.21.15.46:0/524611156 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244997823 0 0) 0x7fcf80073ad0 con 0x7fcf800720e0 2024-04-26T23:55:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.232+0000 7fcf7f400640 1 -- 172.21.15.46:0/524611156 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf80073d70 con 0x7fcf801466b0 2024-04-26T23:55:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.233+0000 7fcf7f400640 1 --2- 172.21.15.46:0/524611156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf801466b0 0x7fcf8006de80 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fcf700097a0 tx=0x7fcf7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2b5567053c1e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.233+0000 7fcf7e000640 1 -- 172.21.15.46:0/524611156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70037070 con 0x7fcf801466b0 2024-04-26T23:55:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.233+0000 7fcf7e000640 1 -- 172.21.15.46:0/524611156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf70032440 con 0x7fcf801466b0 2024-04-26T23:55:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.233+0000 7fcf7e000640 1 -- 172.21.15.46:0/524611156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70032810 con 0x7fcf801466b0 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 -- 172.21.15.46:0/524611156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf801466b0 msgr2=0x7fcf8006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 --2- 172.21.15.46:0/524611156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf801466b0 0x7fcf8006de80 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fcf700097a0 tx=0x7fcf7002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 -- 172.21.15.46:0/524611156 shutdown_connections 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 --2- 172.21.15.46:0/524611156 >> v2:172.21.15.142:3300/0 conn(0x7fcf80160910 0x7fcf80160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 --2- 172.21.15.46:0/524611156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf801466b0 0x7fcf8006de80 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 -- 172.21.15.46:0/524611156 >> 172.21.15.46:0/524611156 conn(0x7fcf800b26d0 msgr2=0x7fcf800b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 -- 172.21.15.46:0/524611156 shutdown_connections 2024-04-26T23:55:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.234+0000 7fcf85000640 1 -- 172.21.15.46:0/524611156 wait complete. 2024-04-26T23:55:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.236+0000 7fcf85000640 1 Processor -- start 2024-04-26T23:55:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.236+0000 7fcf85000640 1 -- start start 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf800720e0 0x7fcf8013b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801466b0 0x7fcf8015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800cb470 con 0x7fcf80160910 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800cb190 con 0x7fcf801466b0 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf85000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8006e3c0 con 0x7fcf800720e0 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50656/0 (socket says 172.21.15.46:50656) 2024-04-26T23:55:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.237+0000 7fcf7fe00640 1 -- 172.21.15.46:0/49084836 learned_addr learned my addr 172.21.15.46:0/49084836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 -- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf800720e0 msgr2=0x7fcf8013b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf800720e0 0x7fcf8013b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 -- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801466b0 msgr2=0x7fcf8015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801466b0 0x7fcf8015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 -- 172.21.15.46:0/49084836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8013e7c0 con 0x7fcf80160910 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf7fe00640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fcf78007e60 tx=0x7fcf7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78014420 con 0x7fcf80160910 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78019070 con 0x7fcf80160910 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8013df20 con 0x7fcf80160910 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.238+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78014800 con 0x7fcf80160910 2024-04-26T23:55:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.239+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8013e4e0 con 0x7fcf80160910 2024-04-26T23:55:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.240+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf800693b0 con 0x7fcf80160910 2024-04-26T23:55:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.241+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf780149a0 con 0x7fcf80160910 2024-04-26T23:55:18.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.242+0000 7fcf77400640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 0x7fcf6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.243+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7fcf7808c1d0 con 0x7fcf80160910 2024-04-26T23:55:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.243+0000 7fcf7f400640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 0x7fcf6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.244+0000 7fcf7f400640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 0x7fcf6006edb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcf70034000 tx=0x7fcf7002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.247+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf7805a800 con 0x7fcf80160910 2024-04-26T23:55:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.537+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) -- 0x7fcf800cdbb0 con 0x7fcf80160910 2024-04-26T23:55:18.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.547+0000 7fcf77400640 1 -- 172.21.15.46:0/49084836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v830) ==== 140+0+0 (secure 0 0 0) 0x7fcf7805a100 con 0x7fcf80160910 2024-04-26T23:55:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 msgr2=0x7fcf6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 0x7fcf6006edb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcf70034000 tx=0x7fcf7002c000 comp rx=0 tx=0).stop 2024-04-26T23:55:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 msgr2=0x7fcf8015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fcf78007e60 tx=0x7fcf7800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 shutdown_connections 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf6006c8f0 0x7fcf6006edb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf80160910 0x7fcf8015f940 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801466b0 0x7fcf8015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 --2- 172.21.15.46:0/49084836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf800720e0 0x7fcf8013b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.550+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 >> 172.21.15.46:0/49084836 conn(0x7fcf800b26d0 msgr2=0x7fcf800cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.551+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 shutdown_connections 2024-04-26T23:55:18.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.551+0000 7fcf85000640 1 -- 172.21.15.46:0/49084836 wait complete. 2024-04-26T23:55:18.573 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-26T23:55:18.573 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-26T23:55:18.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.784+0000 7fe866200640 1 Processor -- start 2024-04-26T23:55:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.784+0000 7fe866200640 1 -- start start 2024-04-26T23:55:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.786+0000 7fe866200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 0x7fe8600d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.786+0000 7fe866200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8600d11f0 con 0x7fe8600d2130 2024-04-26T23:55:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.786+0000 7fe866200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8600d5f20 con 0x7fe8600b4000 2024-04-26T23:55:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.786+0000 7fe866200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8600d0ce0 con 0x7fe86015f4e0 2024-04-26T23:55:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.787+0000 7fe865800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe8600d2130 0x7fe8600b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52200/0 (socket says 172.21.15.46:52200) 2024-04-26T23:55:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.787+0000 7fe865800640 1 -- 172.21.15.46:0/3795628699 learned_addr learned my addr 172.21.15.46:0/3795628699 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.787+0000 7fe864e00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 0x7fe8600d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.787+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479632874 0 0) 0x7fe8600d11f0 con 0x7fe8600d2130 2024-04-26T23:55:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.787+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe848003520 con 0x7fe8600d2130 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407726308 0 0) 0x7fe8600d5f20 con 0x7fe8600b4000 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8600d11f0 con 0x7fe8600b4000 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3518879329 0 0) 0x7fe848003520 con 0x7fe8600d2130 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 msgr2=0x7fe8600d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 0x7fe8600d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 >> v1:172.21.15.142:6789/0 conn(0x7fe8600b4000 legacy=0x7fe8600d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8600d0f10 con 0x7fe8600d2130 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe8580019f0 con 0x7fe8600d2130 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 >> v1:172.21.15.46:6789/0 conn(0x7fe8600d2130 legacy=0x7fe8600b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe848003b70 0x7fe848003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe848004800 0x7fe848004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 0x7fe8480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe84800ba70 con 0x7fe848004800 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe84800bbf0 con 0x7fe848003b70 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe84800bdf0 con 0x7fe86015f4e0 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.788+0000 7fe85fe00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe848004800 0x7fe848004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe864e00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe848003b70 0x7fe848003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe85fe00640 1 -- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 msgr2=0x7fe8480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe85fe00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe86015f4e0 0x7fe8480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe85fe00640 1 -- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe848003b70 msgr2=0x7fe848003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe85fe00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe848003b70 0x7fe848003f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.789+0000 7fe85fe00640 1 -- 172.21.15.46:0/3795628699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe848003520 con 0x7fe848004800 2024-04-26T23:55:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.790+0000 7fe85fe00640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe848004800 0x7fe848004c80 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0018d0 tx=0x7fe848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7aff731847554c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.790+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0014f0 con 0x7fe848004800 2024-04-26T23:55:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.790+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c0052d0 con 0x7fe848004800 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.790+0000 7fe85f400640 1 -- 172.21.15.46:0/3795628699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c02aca0 con 0x7fe848004800 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 -- 172.21.15.46:0/3795628699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe848004800 msgr2=0x7fe848004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 --2- 172.21.15.46:0/3795628699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe848004800 0x7fe848004c80 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0018d0 tx=0x7fe848005830 comp rx=0 tx=0).stop 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe865800640 1 -- 172.21.15.46:0/3795628699 reap_dead start 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 -- 172.21.15.46:0/3795628699 shutdown_connections 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 -- 172.21.15.46:0/3795628699 >> 172.21.15.46:0/3795628699 conn(0x7fe8600d5010 msgr2=0x7fe8600d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 -- 172.21.15.46:0/3795628699 shutdown_connections 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.791+0000 7fe866200640 1 -- 172.21.15.46:0/3795628699 wait complete. 2024-04-26T23:55:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.793+0000 7fe866200640 1 Processor -- start 2024-04-26T23:55:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.793+0000 7fe866200640 1 -- start start 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe848004800 0x7fe8600c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8600ba780 0x7fe86015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8600b1d60 con 0x7fe8600ba300 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8600d2990 con 0x7fe8600ba780 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe866200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe86015f8c0 con 0x7fe848004800 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50678/0 (socket says 172.21.15.46:50678) 2024-04-26T23:55:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.794+0000 7fe85fe00640 1 -- 172.21.15.46:0/39059523 learned_addr learned my addr 172.21.15.46:0/39059523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe864e00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe848004800 0x7fe8600c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 -- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe848004800 msgr2=0x7fe8600c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe848004800 0x7fe8600c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 -- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8600ba780 msgr2=0x7fe86015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe865800640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8600ba780 0x7fe86015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8600ba780 0x7fe86015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 -- 172.21.15.46:0/39059523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8600cb6f0 con 0x7fe8600ba300 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85fe00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0018d0 tx=0x7fe84c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c040020 con 0x7fe8600ba300 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c005480 con 0x7fe8600ba300 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0014f0 con 0x7fe8600ba300 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.795+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8600c8ca0 con 0x7fe8600ba300 2024-04-26T23:55:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.796+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8600c9540 con 0x7fe8600ba300 2024-04-26T23:55:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.797+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8600693b0 con 0x7fe8600ba300 2024-04-26T23:55:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.798+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe84c043020 con 0x7fe8600ba300 2024-04-26T23:55:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.799+0000 7fe85cc00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 0x7fe84006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.799+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7fe84c0abb60 con 0x7fe8600ba300 2024-04-26T23:55:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.800+0000 7fe864e00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 0x7fe84006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.801+0000 7fe864e00640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 0x7fe84006ee30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe854009830 tx=0x7fe854008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:18.803+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe84c04a030 con 0x7fe8600ba300 2024-04-26T23:55:19.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.011+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8600d33d0 con 0x7fe8600ba300 2024-04-26T23:55:19.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.013+0000 7fe85cc00640 1 -- 172.21.15.46:0/39059523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) ==== 74+0+13261 (secure 0 0 0) 0x7fe84c07a0e0 con 0x7fe8600ba300 2024-04-26T23:55:19.015 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:19.015 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":178,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:17.528485+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/7977112":"2024-04-27T23:55:14.345633+0000","172.21.15.46:6834/7977112":"2024-04-27T23:55:14.345633+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 msgr2=0x7fe84006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 0x7fe84006ee30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe854009830 tx=0x7fe854008040 comp rx=0 tx=0).stop 2024-04-26T23:55:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 msgr2=0x7fe8600ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0018d0 tx=0x7fe84c0012d0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 shutdown_connections 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe84006c970 0x7fe84006ee30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8600ba780 0x7fe86015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8600ba300 0x7fe8600ce3a0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 --2- 172.21.15.46:0/39059523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe848004800 0x7fe8600c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 >> 172.21.15.46:0/39059523 conn(0x7fe8600d5010 msgr2=0x7fe860161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 shutdown_connections 2024-04-26T23:55:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.016+0000 7fe866200640 1 -- 172.21.15.46:0/39059523 wait complete. 2024-04-26T23:55:19.038 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:55:19.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:55:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.254+0000 7f8312c00640 1 Processor -- start 2024-04-26T23:55:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.255+0000 7f8312c00640 1 -- start start 2024-04-26T23:55:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.256+0000 7f8312c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f830c15cec0 0x7f830c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.256+0000 7f8312c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f830c0ce7d0 con 0x7f830c073d00 2024-04-26T23:55:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.256+0000 7f8312c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f830c0ceab0 con 0x7f830c0cdb30 2024-04-26T23:55:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.256+0000 7f8312c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f830c0ced90 con 0x7f830c15cec0 2024-04-26T23:55:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.257+0000 7f8312200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f830c073d00 0x7f830c0b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52210/0 (socket says 172.21.15.46:52210) 2024-04-26T23:55:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.257+0000 7f8312200640 1 -- 172.21.15.46:0/592894954 learned_addr learned my addr 172.21.15.46:0/592894954 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.257+0000 7f8311800640 1 --2- 172.21.15.46:0/592894954 >> v2:172.21.15.142:3300/0 conn(0x7f830c15cec0 0x7f830c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.257+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869809832 0 0) 0x7f830c0ce7d0 con 0x7f830c073d00 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.257+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82f4003520 con 0x7f830c073d00 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1535198772 0 0) 0x7f82f4003520 con 0x7f830c073d00 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 >> v2:172.21.15.142:3300/0 conn(0x7f830c15cec0 msgr2=0x7f830c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 --2- 172.21.15.46:0/592894954 >> v2:172.21.15.142:3300/0 conn(0x7f830c15cec0 0x7f830c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 >> v1:172.21.15.142:6789/0 conn(0x7f830c0cdb30 legacy=0x7f830c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c0cf070 con 0x7f830c073d00 2024-04-26T23:55:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f83040019f0 con 0x7f830c073d00 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f8311800640 1 --2- 172.21.15.46:0/592894954 >> v2:172.21.15.142:3300/0 conn(0x7f830c15cec0 0x7f830c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 >> v1:172.21.15.46:6789/0 conn(0x7f830c073d00 legacy=0x7f830c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f4003a00 0x7f82f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f4004800 0x7f82f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f82f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f830c0ced90 con 0x7f82f4005210 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400c160 con 0x7f82f4003a00 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82f400c360 con 0x7f82f4004800 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.258+0000 7f8312200640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f82f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8310e00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f4004800 0x7f82f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8311800640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f4003a00 0x7f82f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8312200640 1 -- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f4004800 msgr2=0x7f82f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8312200640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f4004800 0x7f82f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8312200640 1 -- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f4003a00 msgr2=0x7f82f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8312200640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f4003a00 0x7f82f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8312200640 1 -- 172.21.15.46:0/592894954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f4003520 con 0x7f82f4005210 2024-04-26T23:55:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.259+0000 7f8311800640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f4003a00 0x7f82f4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.260+0000 7f8312200640 1 -- 172.21.15.46:0/592894954 reap_dead start 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.260+0000 7f8312200640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f82f400b9c0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f830400a790 tx=0x7f82f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92805c1e88409e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.261+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400a3b0 con 0x7f82f4005210 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.261+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830402f070 con 0x7f82f4005210 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.261+0000 7f830be00640 1 -- 172.21.15.46:0/592894954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830402bc20 con 0x7f82f4005210 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.261+0000 7f8312c00640 1 -- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 msgr2=0x7f82f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.261+0000 7f8312c00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f82f400b9c0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f830400a790 tx=0x7f82f4005830 comp rx=0 tx=0).stop 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.262+0000 7f8312c00640 1 -- 172.21.15.46:0/592894954 shutdown_connections 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.262+0000 7f8312c00640 1 --2- 172.21.15.46:0/592894954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f82f400b9c0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.262+0000 7f8312c00640 1 -- 172.21.15.46:0/592894954 >> 172.21.15.46:0/592894954 conn(0x7f830c0b27b0 msgr2=0x7f830c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.262+0000 7f8312c00640 1 -- 172.21.15.46:0/592894954 shutdown_connections 2024-04-26T23:55:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.262+0000 7f8312c00640 1 -- 172.21.15.46:0/592894954 wait complete. 2024-04-26T23:55:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.264+0000 7f8312c00640 1 Processor -- start 2024-04-26T23:55:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.264+0000 7f8312c00640 1 -- start start 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c15ba10 0x7f830c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c15ee30 0x7f830c150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f830c0cd5f0 con 0x7f82f4005210 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f830c15d2a0 con 0x7f830c15ee30 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8312c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0cd770 con 0x7f830c15ba10 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8311800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8311800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50694/0 (socket says 172.21.15.46:50694) 2024-04-26T23:55:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.265+0000 7f8311800640 1 -- 172.21.15.46:0/2028365975 learned_addr learned my addr 172.21.15.46:0/2028365975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8310e00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c15ba10 0x7f830c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8312200640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c15ee30 0x7f830c150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 -- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c15ba10 msgr2=0x7f830c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c15ba10 0x7f830c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 -- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c15ee30 msgr2=0x7f830c150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c15ee30 0x7f830c150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 -- 172.21.15.46:0/2028365975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c15dd90 con 0x7f82f4005210 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8311800640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f8300010860 tx=0x7f8300010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830001a070 con 0x7f82f4005210 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830000cb40 con 0x7f82f4005210 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c15e070 con 0x7f82f4005210 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000f040 con 0x7f82f4005210 2024-04-26T23:55:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.266+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c15e630 con 0x7f82f4005210 2024-04-26T23:55:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.268+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c0693b0 con 0x7f82f4005210 2024-04-26T23:55:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.269+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f830001d070 con 0x7f82f4005210 2024-04-26T23:55:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.270+0000 7f8309600640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 0x7f82ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.270+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7f830008be40 con 0x7f82f4005210 2024-04-26T23:55:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.271+0000 7f8310e00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 0x7f82ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.272+0000 7f8310e00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 0x7f82ec06edb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007fd0 tx=0x7f82f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.275+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f830005a3c0 con 0x7f82f4005210 2024-04-26T23:55:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.482+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f830c0d0160 con 0x7f82f4005210 2024-04-26T23:55:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.483+0000 7f8309600640 1 -- 172.21.15.46:0/2028365975 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) ==== 93+0+10 (secure 0 0 0) 0x7f8300059cc0 con 0x7f82f4005210 2024-04-26T23:55:19.485 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.485+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 msgr2=0x7f82ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.485+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 0x7f82ec06edb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007fd0 tx=0x7f82f8009040 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.485+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 msgr2=0x7f830c071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.485+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f8300010860 tx=0x7f8300010d20 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 shutdown_connections 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f82ec06c8f0 0x7f82ec06edb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c15ee30 0x7f830c150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c15ba10 0x7f830c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 --2- 172.21.15.46:0/2028365975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4005210 0x7f830c071ff0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 >> 172.21.15.46:0/2028365975 conn(0x7f830c0b27b0 msgr2=0x7f830c0cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 shutdown_connections 2024-04-26T23:55:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.486+0000 7f8312c00640 1 -- 172.21.15.46:0/2028365975 wait complete. 2024-04-26T23:55:19.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-26T23:55:19.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.716+0000 7f2be7600640 1 Processor -- start 2024-04-26T23:55:19.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.716+0000 7f2be7600640 1 -- start start 2024-04-26T23:55:19.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.718+0000 7f2be7600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2be006e780 0x7f2be00b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.718+0000 7f2be7600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00b4940 con 0x7f2be014d200 2024-04-26T23:55:19.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.718+0000 7f2be7600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00b4c20 con 0x7f2be00b43c0 2024-04-26T23:55:19.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.718+0000 7f2be7600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2be00b4f00 con 0x7f2be006e780 2024-04-26T23:55:19.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.719+0000 7f2be5600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2be014d200 0x7f2be0072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52214/0 (socket says 172.21.15.46:52214) 2024-04-26T23:55:19.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.719+0000 7f2be5600640 1 -- 172.21.15.46:0/2048600213 learned_addr learned my addr 172.21.15.46:0/2048600213 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.719+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905819272 0 0) 0x7f2be00b4940 con 0x7f2be014d200 2024-04-26T23:55:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.719+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bc4003520 con 0x7f2be014d200 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.719+0000 7f2be4c00640 1 --2- 172.21.15.46:0/2048600213 >> v2:172.21.15.142:3300/0 conn(0x7f2be006e780 0x7f2be00b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3385963684 0 0) 0x7f2bc4003520 con 0x7f2be014d200 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 >> v2:172.21.15.142:3300/0 conn(0x7f2be006e780 msgr2=0x7f2be00b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 --2- 172.21.15.46:0/2048600213 >> v2:172.21.15.142:3300/0 conn(0x7f2be006e780 0x7f2be00b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 >> v1:172.21.15.142:6789/0 conn(0x7f2be00b43c0 legacy=0x7f2be014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be00b51e0 con 0x7f2be014d200 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2bd8001670 con 0x7f2be014d200 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 >> v1:172.21.15.46:6789/0 conn(0x7f2be014d200 legacy=0x7f2be0072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4003a00 0x7f2bc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc4004800 0x7f2bc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 0x7f2bc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00b4f00 con 0x7f2bc4005210 2024-04-26T23:55:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc400c160 con 0x7f2bc4004800 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc400c360 con 0x7f2bc4003a00 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.720+0000 7f2be5600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 0x7f2bc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2bd7e00640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc4004800 0x7f2bc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be4c00640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4003a00 0x7f2bc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be5600640 1 -- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4003a00 msgr2=0x7f2bc4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be5600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4003a00 0x7f2bc4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be5600640 1 -- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc4004800 msgr2=0x7f2bc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be5600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc4004800 0x7f2bc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.721+0000 7f2be5600640 1 -- 172.21.15.46:0/2048600213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc4003520 con 0x7f2bc4005210 2024-04-26T23:55:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.722+0000 7f2be5600640 1 -- 172.21.15.46:0/2048600213 reap_dead start 2024-04-26T23:55:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.722+0000 7f2be5600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 0x7f2bc400b9c0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c0e0 tx=0x7f2bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce7b8ad4e7d8023 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.722+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007c20 con 0x7f2bc4005210 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.722+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8005af0 con 0x7f2bc4005210 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.722+0000 7f2bdfe00640 1 -- 172.21.15.46:0/2048600213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007070 con 0x7f2bc4005210 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 -- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 msgr2=0x7f2bc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 0x7f2bc400b9c0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c0e0 tx=0x7f2bc4005830 comp rx=0 tx=0).stop 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 -- 172.21.15.46:0/2048600213 shutdown_connections 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 --2- 172.21.15.46:0/2048600213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bc4005210 0x7f2bc400b9c0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 -- 172.21.15.46:0/2048600213 >> 172.21.15.46:0/2048600213 conn(0x7f2be006c530 msgr2=0x7f2be006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 -- 172.21.15.46:0/2048600213 shutdown_connections 2024-04-26T23:55:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.723+0000 7f2be7600640 1 -- 172.21.15.46:0/2048600213 wait complete. 2024-04-26T23:55:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.725+0000 7f2be7600640 1 Processor -- start 2024-04-26T23:55:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.725+0000 7f2be7600640 1 -- start start 2024-04-26T23:55:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4005210 0x7f2be0141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be0162960 0x7f2be0162de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be006ece0 con 0x7f2be015fd80 2024-04-26T23:55:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2be006eb60 con 0x7f2be0162960 2024-04-26T23:55:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2be7600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2be006ee60 con 0x7f2bc4005210 2024-04-26T23:55:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50708/0 (socket says 172.21.15.46:50708) 2024-04-26T23:55:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.726+0000 7f2bd7e00640 1 -- 172.21.15.46:0/3181428422 learned_addr learned my addr 172.21.15.46:0/3181428422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2be4c00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4005210 0x7f2be0141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 -- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4005210 msgr2=0x7f2be0141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4005210 0x7f2be0141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2be5600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be0162960 0x7f2be0162de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 -- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be0162960 msgr2=0x7f2be0162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be0162960 0x7f2be0162de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 -- 172.21.15.46:0/3181428422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01f9ab0 con 0x7f2be015fd80 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bd7e00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e470 tx=0x7f2bc800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.727+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8019070 con 0x7f2be015fd80 2024-04-26T23:55:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.728+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc80052e0 con 0x7f2be015fd80 2024-04-26T23:55:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.728+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be01f9d30 con 0x7f2be015fd80 2024-04-26T23:55:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.728+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80037f0 con 0x7f2be015fd80 2024-04-26T23:55:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.728+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01fa2f0 con 0x7f2be015fd80 2024-04-26T23:55:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.729+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0069530 con 0x7f2be015fd80 2024-04-26T23:55:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.730+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bc801c070 con 0x7f2be015fd80 2024-04-26T23:55:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.731+0000 7f2bdd600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.731+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7f2bc808af40 con 0x7f2be015fd80 2024-04-26T23:55:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.732+0000 7f2be4c00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.733+0000 7f2be4c00640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009df0 tx=0x7f2bd000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:19.736+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2be01f9ab0 con 0x7f2be015fd80 2024-04-26T23:55:20.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.022+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) -- 0x7f2be01423c0 con 0x7f2be015fd80 2024-04-26T23:55:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.561+0000 7f2bdd600640 1 -- 172.21.15.46:0/3181428422 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v832) ==== 136+0+0 (secure 0 0 0) 0x7f2bc8058df0 con 0x7f2be015fd80 2024-04-26T23:55:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.565+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 msgr2=0x7f2bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.565+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009df0 tx=0x7f2bd000b040 comp rx=0 tx=0).stop 2024-04-26T23:55:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.565+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 msgr2=0x7f2be01f9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.565+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e470 tx=0x7f2bc800e930 comp rx=0 tx=0).stop 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 shutdown_connections 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be0162960 0x7f2be0162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be015fd80 0x7f2be01f9570 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 --2- 172.21.15.46:0/3181428422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bc4005210 0x7f2be0141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 >> 172.21.15.46:0/3181428422 conn(0x7f2be006c530 msgr2=0x7f2be0072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 shutdown_connections 2024-04-26T23:55:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.566+0000 7f2be7600640 1 -- 172.21.15.46:0/3181428422 wait complete. 2024-04-26T23:55:20.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:55:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.809+0000 7f0f36200640 1 Processor -- start 2024-04-26T23:55:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.810+0000 7f0f36200640 1 -- start start 2024-04-26T23:55:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.811+0000 7f0f36200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f301626c0 0x7f0f300b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.811+0000 7f0f36200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f300d07d0 con 0x7f0f300b5fb0 2024-04-26T23:55:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.811+0000 7f0f36200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f300d0ab0 con 0x7f0f30162aa0 2024-04-26T23:55:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.811+0000 7f0f36200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300d0d90 con 0x7f0f301626c0 2024-04-26T23:55:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.812+0000 7f0f2fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f300b5fb0 0x7f0f300d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52226/0 (socket says 172.21.15.46:52226) 2024-04-26T23:55:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.812+0000 7f0f2fe00640 1 -- 172.21.15.46:0/736014127 learned_addr learned my addr 172.21.15.46:0/736014127 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.812+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3572751445 0 0) 0x7f0f300d07d0 con 0x7f0f300b5fb0 2024-04-26T23:55:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.812+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f301626c0 0x7f0f300b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.812+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f18003520 con 0x7f0f300b5fb0 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3198676353 0 0) 0x7f0f18003520 con 0x7f0f300b5fb0 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f301626c0 msgr2=0x7f0f300b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f301626c0 0x7f0f300b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 >> v1:172.21.15.142:6789/0 conn(0x7f0f30162aa0 legacy=0x7f0f300d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300d2c80 con 0x7f0f300b5fb0 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0f1c0017f0 con 0x7f0f300b5fb0 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 >> v1:172.21.15.46:6789/0 conn(0x7f0f300b5fb0 legacy=0x7f0f300d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f180039e0 0x7f0f18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f18004800 0x7f0f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180051c0 0x7f0f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300d0d90 con 0x7f0f180039e0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1800c090 con 0x7f0f180051c0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1800c290 con 0x7f0f18004800 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f301626c0 0x7f0f300b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f180039e0 0x7f0f18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.813+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f18004800 0x7f0f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f35800640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180051c0 0x7f0f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f34e00640 1 -- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f18004800 msgr2=0x7f0f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f18004800 0x7f0f18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f34e00640 1 -- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180051c0 msgr2=0x7f0f1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180051c0 0x7f0f1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f34e00640 1 -- 172.21.15.46:0/736014127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f18003520 con 0x7f0f180039e0 2024-04-26T23:55:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.814+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f18004800 0x7f0f18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.815+0000 7f0f34e00640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f180039e0 0x7f0f18003e00 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd003a00600451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.815+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24037070 con 0x7f0f180039e0 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.815+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2402aae0 con 0x7f0f180039e0 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.815+0000 7f0f2f400640 1 -- 172.21.15.46:0/736014127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400f040 con 0x7f0f180039e0 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 -- 172.21.15.46:0/736014127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f180039e0 msgr2=0x7f0f18003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 --2- 172.21.15.46:0/736014127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f180039e0 0x7f0f18003e00 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f18005830 comp rx=0 tx=0).stop 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f35800640 1 -- 172.21.15.46:0/736014127 reap_dead start 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 -- 172.21.15.46:0/736014127 shutdown_connections 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 -- 172.21.15.46:0/736014127 >> 172.21.15.46:0/736014127 conn(0x7f0f3009e2a0 msgr2=0x7f0f300d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 -- 172.21.15.46:0/736014127 shutdown_connections 2024-04-26T23:55:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.816+0000 7f0f36200640 1 -- 172.21.15.46:0/736014127 wait complete. 2024-04-26T23:55:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.818+0000 7f0f36200640 1 Processor -- start 2024-04-26T23:55:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.818+0000 7f0f36200640 1 -- start start 2024-04-26T23:55:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180039e0 0x7f0f301f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 0x7f0f300cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300b47c0 con 0x7f0f301f5160 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300b4940 con 0x7f0f180039e0 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f36200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300b4ac0 con 0x7f0f300cd520 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f35800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f35800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50728/0 (socket says 172.21.15.46:50728) 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.819+0000 7f0f35800640 1 -- 172.21.15.46:0/2303358736 learned_addr learned my addr 172.21.15.46:0/2303358736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 0x7f0f300cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f34e00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180039e0 0x7f0f301f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 -- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 msgr2=0x7f0f300cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 0x7f0f300cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 -- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180039e0 msgr2=0x7f0f301f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180039e0 0x7f0f301f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 -- 172.21.15.46:0/2303358736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f301f5f00 con 0x7f0f301f5160 2024-04-26T23:55:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f2fe00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 0x7f0f300cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f35800640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0f30162f10 tx=0x7f0f2800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28003970 con 0x7f0f301f5160 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28018070 con 0x7f0f301f5160 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.820+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28003d50 con 0x7f0f301f5160 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.821+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f300ca9a0 con 0x7f0f301f5160 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.821+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f300caf60 con 0x7f0f301f5160 2024-04-26T23:55:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.822+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f300693b0 con 0x7f0f301f5160 2024-04-26T23:55:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.823+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f280050d0 con 0x7f0f301f5160 2024-04-26T23:55:20.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.825+0000 7f0f2cc00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:20.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.825+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7f0f2808b740 con 0x7f0f301f5160 2024-04-26T23:55:20.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.825+0000 7f0f34e00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.827+0000 7f0f34e00640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f2402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:20.829+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f28059c40 con 0x7f0f301f5160 2024-04-26T23:55:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.032+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0f300d25b0 con 0x7f0f301f5160 2024-04-26T23:55:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.037+0000 7f0f2cc00640 1 -- 172.21.15.46:0/2303358736 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f0f28059540 con 0x7f0f301f5160 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 msgr2=0x7f0f1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f2402e000 comp rx=0 tx=0).stop 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 msgr2=0x7f0f301f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0f30162f10 tx=0x7f0f2800eb90 comp rx=0 tx=0).stop 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 shutdown_connections 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0f1006c8f0 0x7f0f1006edb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f301f5160 0x7f0f301f55e0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f300cd520 0x7f0f300cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 --2- 172.21.15.46:0/2303358736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f180039e0 0x7f0f301f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 >> 172.21.15.46:0/2303358736 conn(0x7f0f3009e2a0 msgr2=0x7f0f300b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 shutdown_connections 2024-04-26T23:55:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.040+0000 7f0f36200640 1 -- 172.21.15.46:0/2303358736 wait complete. 2024-04-26T23:55:21.061 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:55:21.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:55:21.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:55:21.282 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:55:21.282 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:21.282 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:21.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:55:21.341 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:55:21.341 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:21.399 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:55:21.399 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:21.399 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:21.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:55:21.460 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:55:21.460 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:21.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:55:21.513 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:55:21.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:21.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.731+0000 7f2996400640 1 Processor -- start 2024-04-26T23:55:21.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.731+0000 7f2996400640 1 -- start start 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2996400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29900d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2996400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29900d11f0 con 0x7f299015f4e0 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2996400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29900d5f20 con 0x7f29900d2130 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2996400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29900d0ce0 con 0x7f29900b4000 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2995000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f299015f4e0 0x7f29900d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52230/0 (socket says 172.21.15.46:52230) 2024-04-26T23:55:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.733+0000 7f2995000640 1 -- 172.21.15.46:0/1582809352 learned_addr learned my addr 172.21.15.46:0/1582809352 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f2995a00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29900d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207586250 0 0) 0x7f29900d11f0 con 0x7f299015f4e0 2024-04-26T23:55:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2978003520 con 0x7f299015f4e0 2024-04-26T23:55:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987515473 0 0) 0x7f2978003520 con 0x7f299015f4e0 2024-04-26T23:55:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 msgr2=0x7f29900d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29900d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 >> v1:172.21.15.142:6789/0 conn(0x7f29900d2130 legacy=0x7f29900b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29900d0f10 con 0x7f299015f4e0 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.734+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2984001670 con 0x7f299015f4e0 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 >> v1:172.21.15.46:6789/0 conn(0x7f299015f4e0 legacy=0x7f29900d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29780039e0 0x7f2978003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f2978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f297800ba70 con 0x7f2978004800 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f297800bc20 con 0x7f29780039e0 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f297800be20 con 0x7f29900b4000 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f2978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f2995a00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f2995000640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29780039e0 0x7f2978003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 -- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 msgr2=0x7f29780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 -- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29780039e0 msgr2=0x7f2978003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29780039e0 0x7f2978003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f298fe00640 1 -- 172.21.15.46:0/1582809352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2978003520 con 0x7f2978004800 2024-04-26T23:55:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.735+0000 7f2995a00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b4000 0x7f29780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.736+0000 7f298fe00640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f2978004c80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f297c001720 tx=0x7f2978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198cbf7268a9db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c001370 con 0x7f2978004800 2024-04-26T23:55:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0052d0 con 0x7f2978004800 2024-04-26T23:55:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f298f400640 1 -- 172.21.15.46:0/1582809352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c02ab90 con 0x7f2978004800 2024-04-26T23:55:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f2996400640 1 -- 172.21.15.46:0/1582809352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 msgr2=0x7f2978004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f2996400640 1 --2- 172.21.15.46:0/1582809352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f2978004c80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f297c001720 tx=0x7f2978005830 comp rx=0 tx=0).stop 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.737+0000 7f2995a00640 1 -- 172.21.15.46:0/1582809352 reap_dead start 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.738+0000 7f2996400640 1 -- 172.21.15.46:0/1582809352 shutdown_connections 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.738+0000 7f2996400640 1 -- 172.21.15.46:0/1582809352 >> 172.21.15.46:0/1582809352 conn(0x7f29900d5010 msgr2=0x7f29900d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.738+0000 7f2996400640 1 -- 172.21.15.46:0/1582809352 shutdown_connections 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.738+0000 7f2996400640 1 -- 172.21.15.46:0/1582809352 wait complete. 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.739+0000 7f2996400640 1 Processor -- start 2024-04-26T23:55:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 -- start start 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 0x7f29901556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b5ca0 0x7f29900b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29900b1d60 con 0x7f2978004800 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29900d2990 con 0x7f29900cc0e0 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.740+0000 7f2996400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f299015f8c0 con 0x7f29900b5ca0 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50740/0 (socket says 172.21.15.46:50740) 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 -- 172.21.15.46:0/1889916865 learned_addr learned my addr 172.21.15.46:0/1889916865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995a00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b5ca0 0x7f29900b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f298fe00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 0x7f29901556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 -- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b5ca0 msgr2=0x7f29900b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b5ca0 0x7f29900b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 -- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 msgr2=0x7f29901556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 0x7f29901556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f2995000640 1 -- 172.21.15.46:0/1889916865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29900cdf70 con 0x7f2978004800 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.741+0000 7f298fe00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 0x7f29901556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f2995000640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f298400d7a0 tx=0x7f298400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984019070 con 0x7f2978004800 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29840074e0 con 0x7f2978004800 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984004a00 con 0x7f2978004800 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29900cc6a0 con 0x7f2978004800 2024-04-26T23:55:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.742+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29900ccc60 con 0x7f2978004800 2024-04-26T23:55:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.743+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29900693b0 con 0x7f2978004800 2024-04-26T23:55:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.745+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2984004ba0 con 0x7f2978004800 2024-04-26T23:55:21.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.745+0000 7f298cc00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 0x7f297006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:21.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.746+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7f298408b350 con 0x7f2978004800 2024-04-26T23:55:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.746+0000 7f298fe00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 0x7f297006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.747+0000 7f298fe00640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 0x7f297006edb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f297c032000 tx=0x7f297c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.750+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2984059980 con 0x7f2978004800 2024-04-26T23:55:22.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:21.999+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29900cf5f0 con 0x7f2978004800 2024-04-26T23:55:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.000+0000 7f298cc00640 1 -- 172.21.15.46:0/1889916865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+3879 (secure 0 0 0) 0x7f2984059280 con 0x7f2978004800 2024-04-26T23:55:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:22.002 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":142,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/3938040627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3938040627},{"type":"v1","addr":"172.21.15.142:6836","nonce":3938040627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":6674,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/261858641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":261858641},{"type":"v1","addr":"172.21.15.142:6837","nonce":261858641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":135},{"gid":10761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/4084717801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4084717801},{"type":"v1","addr":"172.21.15.46:6833","nonce":4084717801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134},{"gid":11172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1440738179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1440738179},{"type":"v1","addr":"172.21.15.46:6835","nonce":1440738179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":142}],"filesystems":[]} 2024-04-26T23:55:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 142 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 msgr2=0x7f297006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 0x7f297006edb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f297c032000 tx=0x7f297c02e000 comp rx=0 tx=0).stop 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 msgr2=0x7f29900cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f298400d7a0 tx=0x7f298400dc60 comp rx=0 tx=0).stop 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 shutdown_connections 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f297006c8f0 0x7f297006edb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29900b5ca0 0x7f29900b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29900cc0e0 0x7f29901556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 --2- 172.21.15.46:0/1889916865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2978004800 0x7f29900cda30 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 >> 172.21.15.46:0/1889916865 conn(0x7f29900d5010 msgr2=0x7f29900b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 shutdown_connections 2024-04-26T23:55:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.003+0000 7f2996400640 1 -- 172.21.15.46:0/1889916865 wait complete. 2024-04-26T23:55:22.026 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:55:22.026 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:55:22.026 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:55:22.026 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:55:22.026 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:55:22.026 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:55:22.026 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:55:22.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:55:22.027 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:55:22.027 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:55:22.027 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:55:22.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:55:22.027 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:55:22.027 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:55:22.027 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:55:22.028 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:55:22.045 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:55:22.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:55:22.047 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:55:22.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:55:22.051 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:55:22.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:55:22.052 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:55:22.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:55:22.094 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:55:22.095 DEBUG:teuthology.parallel:result is None 2024-04-26T23:55:22.099 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:55:22.099 DEBUG:teuthology.parallel:result is None 2024-04-26T23:55:22.101 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:55:22.101 DEBUG:teuthology.parallel:result is None 2024-04-26T23:55:22.105 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:55:22.105 DEBUG:teuthology.parallel:result is None 2024-04-26T23:55:22.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:55:22.250 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:55:22.252 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:55:22.253 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:55:22.254 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:55:22.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.286+0000 7f883d200640 1 Processor -- start 2024-04-26T23:55:22.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.286+0000 7f883d200640 1 -- start start 2024-04-26T23:55:22.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.287+0000 7f883d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88380d2130 0x7f88380d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.287+0000 7f883d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 0x7f88380b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.287+0000 7f883d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88380d11f0 con 0x7f883815f4e0 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.287+0000 7f883d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f88380d0ce0 con 0x7f88380d2130 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.287+0000 7f883d200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f88380d2990 con 0x7f88380b4000 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f882fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88380d2130 0x7f88380d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f8837400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f883815f4e0 0x7f88380d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40968/0 (socket says 172.21.15.46:40968) 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f8837400640 1 -- 172.21.15.46:0/1405484730 learned_addr learned my addr 172.21.15.46:0/1405484730 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f8837e00640 1 --2- 172.21.15.46:0/1405484730 >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 0x7f88380b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f882fe00640 1 -- 172.21.15.46:0/1405484730 >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 msgr2=0x7f88380b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f882fe00640 1 --2- 172.21.15.46:0/1405484730 >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 0x7f88380b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f882fe00640 1 -- 172.21.15.46:0/1405484730 >> v1:172.21.15.46:6789/0 conn(0x7f883815f4e0 legacy=0x7f88380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f882fe00640 1 -- 172.21.15.46:0/1405484730 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380d5f20 con 0x7f88380d2130 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.288+0000 7f8837e00640 1 --2- 172.21.15.46:0/1405484730 >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 0x7f88380b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:22.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.289+0000 7f882fe00640 1 --2- 172.21.15.46:0/1405484730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88380d2130 0x7f88380d2df0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f88200097a0 tx=0x7f882002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db98bb08a1f5ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.289+0000 7f8836a00640 1 -- 172.21.15.46:0/1405484730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820037070 con 0x7f88380d2130 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f8836a00640 1 -- 172.21.15.46:0/1405484730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8820032440 con 0x7f88380d2130 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f8836a00640 1 -- 172.21.15.46:0/1405484730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820032810 con 0x7f88380d2130 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 -- 172.21.15.46:0/1405484730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88380d2130 msgr2=0x7f88380d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 --2- 172.21.15.46:0/1405484730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88380d2130 0x7f88380d2df0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f88200097a0 tx=0x7f882002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 -- 172.21.15.46:0/1405484730 shutdown_connections 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 --2- 172.21.15.46:0/1405484730 >> v2:172.21.15.142:3300/0 conn(0x7f88380b4000 0x7f88380b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 --2- 172.21.15.46:0/1405484730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88380d2130 0x7f88380d2df0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 -- 172.21.15.46:0/1405484730 >> 172.21.15.46:0/1405484730 conn(0x7f88380d5010 msgr2=0x7f88380d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 -- 172.21.15.46:0/1405484730 shutdown_connections 2024-04-26T23:55:22.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.290+0000 7f883d200640 1 -- 172.21.15.46:0/1405484730 wait complete. 2024-04-26T23:55:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.292+0000 7f883d200640 1 Processor -- start 2024-04-26T23:55:22.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.292+0000 7f883d200640 1 -- start start 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88380b4000 0x7f88381564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 0x7f88380ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88380b1d60 con 0x7f883815f4e0 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88380d5c40 con 0x7f88380d2130 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f883d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f883815f8c0 con 0x7f88380b4000 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f8837e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f8837e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33002/0 (socket says 172.21.15.46:33002) 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.293+0000 7f8837e00640 1 -- 172.21.15.46:0/3593544965 learned_addr learned my addr 172.21.15.46:0/3593544965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837400640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88380b4000 0x7f88381564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f882fe00640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 0x7f88380ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 -- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88380b4000 msgr2=0x7f88381564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88380b4000 0x7f88381564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 -- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 msgr2=0x7f88380ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 0x7f88380ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 -- 172.21.15.46:0/3593544965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380b8290 con 0x7f883815f4e0 2024-04-26T23:55:22.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f882fe00640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 0x7f88380ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:22.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f8837e00640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f8830007df0 tx=0x7f883000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883001a070 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.294+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8830003990 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.295+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830003d80 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.295+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88380b8570 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.295+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88380b8b30 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.296+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88380693b0 con 0x7f883815f4e0 2024-04-26T23:55:22.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.297+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f883001d070 con 0x7f883815f4e0 2024-04-26T23:55:22.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.298+0000 7f882f400640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 0x7f881406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.298+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5335+0+0 (secure 0 0 0) 0x7f883008bc40 con 0x7f883815f4e0 2024-04-26T23:55:22.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.298+0000 7f8837400640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 0x7f881406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.299+0000 7f8837400640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 0x7f881406ede0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f60 tx=0x7f8828009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.302+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8830010070 con 0x7f883815f4e0 2024-04-26T23:55:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.522+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f88380d33d0 con 0x7f883815f4e0 2024-04-26T23:55:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.586+0000 7f882f400640 1 -- 172.21.15.46:0/3593544965 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) ==== 97+0+0 (secure 0 0 0) 0x7f883005a1d0 con 0x7f883815f4e0 2024-04-26T23:55:22.588 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:55:22.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 msgr2=0x7f881406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 0x7f881406ede0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f60 tx=0x7f8828009040 comp rx=0 tx=0).stop 2024-04-26T23:55:22.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 msgr2=0x7f88380b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f8830007df0 tx=0x7f883000d4a0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 shutdown_connections 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f881406c920 0x7f881406ede0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883815f4e0 0x7f88380b2850 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88380d2130 0x7f88380ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 --2- 172.21.15.46:0/3593544965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88380b4000 0x7f88381564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.590+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 >> 172.21.15.46:0/3593544965 conn(0x7f88380d5010 msgr2=0x7f88380d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.591+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 shutdown_connections 2024-04-26T23:55:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.591+0000 7f883d200640 1 -- 172.21.15.46:0/3593544965 wait complete. 2024-04-26T23:55:22.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:55:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.810+0000 7fde77e00640 1 Processor -- start 2024-04-26T23:55:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.810+0000 7fde77e00640 1 -- start start 2024-04-26T23:55:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde780d1990 0x7fde780d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde77e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde780d0c10 0x7fde780d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde77e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde780d22b0 con 0x7fde781639c0 2024-04-26T23:55:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde77e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fde780d0780 con 0x7fde780d1990 2024-04-26T23:55:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde77e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fde780cfe50 con 0x7fde780d0c10 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.812+0000 7fde76a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde780d1990 0x7fde780d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde780d1990 0x7fde780d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33012/0 (socket says 172.21.15.46:33012) 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 -- 172.21.15.46:0/1382445275 learned_addr learned my addr 172.21.15.46:0/1382445275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76000640 1 --2- 172.21.15.46:0/1382445275 >> v2:172.21.15.142:3300/0 conn(0x7fde780d0c10 0x7fde780d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 -- 172.21.15.46:0/1382445275 >> v2:172.21.15.142:3300/0 conn(0x7fde780d0c10 msgr2=0x7fde780d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 --2- 172.21.15.46:0/1382445275 >> v2:172.21.15.142:3300/0 conn(0x7fde780d0c10 0x7fde780d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 -- 172.21.15.46:0/1382445275 >> v1:172.21.15.142:6789/0 conn(0x7fde781639c0 legacy=0x7fde780d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.813+0000 7fde76a00640 1 -- 172.21.15.46:0/1382445275 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde780d0130 con 0x7fde780d1990 2024-04-26T23:55:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.815+0000 7fde76a00640 1 --2- 172.21.15.46:0/1382445275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780d1d70 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fde780c8720 tx=0x7fde6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6416f1fa2c768f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.815+0000 7fde75600640 1 -- 172.21.15.46:0/1382445275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68037070 con 0x7fde780d1990 2024-04-26T23:55:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.815+0000 7fde75600640 1 -- 172.21.15.46:0/1382445275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde68032440 con 0x7fde780d1990 2024-04-26T23:55:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.815+0000 7fde75600640 1 -- 172.21.15.46:0/1382445275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68032810 con 0x7fde780d1990 2024-04-26T23:55:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 -- 172.21.15.46:0/1382445275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 msgr2=0x7fde780d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 --2- 172.21.15.46:0/1382445275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780d1d70 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fde780c8720 tx=0x7fde6802a270 comp rx=0 tx=0).stop 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 -- 172.21.15.46:0/1382445275 shutdown_connections 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 --2- 172.21.15.46:0/1382445275 >> v2:172.21.15.142:3300/0 conn(0x7fde780d0c10 0x7fde780d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 --2- 172.21.15.46:0/1382445275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780d1d70 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 -- 172.21.15.46:0/1382445275 >> 172.21.15.46:0/1382445275 conn(0x7fde7809e2a0 msgr2=0x7fde780d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 -- 172.21.15.46:0/1382445275 shutdown_connections 2024-04-26T23:55:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.816+0000 7fde77e00640 1 -- 172.21.15.46:0/1382445275 wait complete. 2024-04-26T23:55:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.818+0000 7fde77e00640 1 Processor -- start 2024-04-26T23:55:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.818+0000 7fde77e00640 1 -- start start 2024-04-26T23:55:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.819+0000 7fde77e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 0x7fde780a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde781639c0 0x7fde78151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde780cfa90 con 0x7fde780d1990 2024-04-26T23:55:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde7815f530 con 0x7fde781639c0 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde780cfc10 con 0x7fde780d0c10 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33016/0 (socket says 172.21.15.46:33016) 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 -- 172.21.15.46:0/1980335709 learned_addr learned my addr 172.21.15.46:0/1980335709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde77400640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde781639c0 0x7fde78151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76a00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 0x7fde780a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 -- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 msgr2=0x7fde780a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 0x7fde780a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 -- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde781639c0 msgr2=0x7fde78151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde781639c0 0x7fde78151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76000640 1 -- 172.21.15.46:0/1980335709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde780a1570 con 0x7fde780d1990 2024-04-26T23:55:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.820+0000 7fde76a00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 0x7fde780a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde76000640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e870 tx=0x7fde6400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64017070 con 0x7fde780d1990 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6400c940 con 0x7fde780d1990 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde780a1850 con 0x7fde780d1990 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6400cd30 con 0x7fde780d1990 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.821+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde780a1e10 con 0x7fde780d1990 2024-04-26T23:55:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.823+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde780ce890 con 0x7fde780d1990 2024-04-26T23:55:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.824+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde6401a070 con 0x7fde780d1990 2024-04-26T23:55:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.825+0000 7fde6f400640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 0x7fde5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.825+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5249+0+0 (secure 0 0 0) 0x7fde6408bba0 con 0x7fde780d1990 2024-04-26T23:55:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.826+0000 7fde76a00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 0x7fde5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.827+0000 7fde76a00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 0x7fde5806ede0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fde68034000 tx=0x7fde6802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:22.831+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde6405a180 con 0x7fde780d1990 2024-04-26T23:55:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.111+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fde780b6170 con 0x7fde780d1990 2024-04-26T23:55:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.112+0000 7fde6f400640 1 -- 172.21.15.46:0/1980335709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7fde64059a80 con 0x7fde780d1990 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:55:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:55:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:55:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:55:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:55:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:55:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:55:23.137 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 msgr2=0x7fde5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 0x7fde5806ede0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fde68034000 tx=0x7fde6802e000 comp rx=0 tx=0).stop 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 msgr2=0x7fde780ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e870 tx=0x7fde6400ed30 comp rx=0 tx=0).stop 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 shutdown_connections 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde5806c920 0x7fde5806ede0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde781639c0 0x7fde78151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d1990 0x7fde780ce350 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 --2- 172.21.15.46:0/1980335709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde780d0c10 0x7fde780a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 >> 172.21.15.46:0/1980335709 conn(0x7fde7809e2a0 msgr2=0x7fde780ba4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 shutdown_connections 2024-04-26T23:55:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.115+0000 7fde77e00640 1 -- 172.21.15.46:0/1980335709 wait complete. 2024-04-26T23:55:23.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:23.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.343+0000 7f2166c00640 1 Processor -- start 2024-04-26T23:55:23.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.343+0000 7f2166c00640 1 -- start start 2024-04-26T23:55:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.344+0000 7f2166c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21600d2130 0x7f21600d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.344+0000 7f2166c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600b4000 0x7f21600b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.344+0000 7f2166c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21600d11f0 con 0x7f216015f4e0 2024-04-26T23:55:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.344+0000 7f2166c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f21600d0ce0 con 0x7f21600d2130 2024-04-26T23:55:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.344+0000 7f2166c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21600d2990 con 0x7f21600b4000 2024-04-26T23:55:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2165800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f216015f4e0 0x7f21600d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40980/0 (socket says 172.21.15.46:40980) 2024-04-26T23:55:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2164e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21600d2130 0x7f21600d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2165800640 1 -- 172.21.15.46:0/2682575963 learned_addr learned my addr 172.21.15.46:0/2682575963 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2166200640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600b4000 0x7f21600b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2164e00640 1 -- 172.21.15.46:0/2682575963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600b4000 msgr2=0x7f21600b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2164e00640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600b4000 0x7f21600b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f2164e00640 1 -- 172.21.15.46:0/2682575963 >> v1:172.21.15.46:6789/0 conn(0x7f216015f4e0 legacy=0x7f21600d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.345+0000 7f215fe00640 1 -- 172.21.15.46:0/2682575963 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143028245 0 0) 0x7f21600d11f0 con 0x7f216015f4e0 2024-04-26T23:55:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.346+0000 7f2164e00640 1 -- 172.21.15.46:0/2682575963 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21600d5f20 con 0x7f21600d2130 2024-04-26T23:55:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.347+0000 7f2164e00640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21600d2130 0x7f21600d2df0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda6aa7f5b6ffb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.347+0000 7f215fe00640 1 -- 172.21.15.46:0/2682575963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c007740 con 0x7f21600d2130 2024-04-26T23:55:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.347+0000 7f215fe00640 1 -- 172.21.15.46:0/2682575963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c0078e0 con 0x7f21600d2130 2024-04-26T23:55:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.347+0000 7f215fe00640 1 -- 172.21.15.46:0/2682575963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c007cb0 con 0x7f21600d2130 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 -- 172.21.15.46:0/2682575963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21600d2130 msgr2=0x7f21600d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21600d2130 0x7f21600d2df0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c02a660 comp rx=0 tx=0).stop 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 -- 172.21.15.46:0/2682575963 shutdown_connections 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600b4000 0x7f21600b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 --2- 172.21.15.46:0/2682575963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21600d2130 0x7f21600d2df0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 -- 172.21.15.46:0/2682575963 >> 172.21.15.46:0/2682575963 conn(0x7f21600d5010 msgr2=0x7f21600d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 -- 172.21.15.46:0/2682575963 shutdown_connections 2024-04-26T23:55:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.348+0000 7f2166c00640 1 -- 172.21.15.46:0/2682575963 wait complete. 2024-04-26T23:55:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.350+0000 7f2166c00640 1 Processor -- start 2024-04-26T23:55:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.350+0000 7f2166c00640 1 -- start start 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21600b4000 0x7f2160156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600d2130 0x7f216015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21600b1d60 con 0x7f216015f4e0 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21600d5c40 con 0x7f21600b4000 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.351+0000 7f2166c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f216015f8c0 con 0x7f21600d2130 2024-04-26T23:55:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33026/0 (socket says 172.21.15.46:33026) 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 -- 172.21.15.46:0/2843388372 learned_addr learned my addr 172.21.15.46:0/2843388372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2165800640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21600b4000 0x7f2160156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2164e00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600d2130 0x7f216015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 -- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600d2130 msgr2=0x7f216015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600d2130 0x7f216015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 -- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21600b4000 msgr2=0x7f2160156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21600b4000 0x7f2160156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.352+0000 7f2166200640 1 -- 172.21.15.46:0/2843388372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21600b9a60 con 0x7f216015f4e0 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f2166200640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f215800d830 tx=0x7f215800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800b6c0 con 0x7f216015f4e0 2024-04-26T23:55:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2158019070 con 0x7f216015f4e0 2024-04-26T23:55:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800bac0 con 0x7f216015f4e0 2024-04-26T23:55:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21600b9d40 con 0x7f216015f4e0 2024-04-26T23:55:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.353+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21600cc180 con 0x7f216015f4e0 2024-04-26T23:55:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.354+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21600693b0 con 0x7f216015f4e0 2024-04-26T23:55:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.355+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f215801e460 con 0x7f216015f4e0 2024-04-26T23:55:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.356+0000 7f215d600640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 0x7f214006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.356+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5249+0+0 (secure 0 0 0) 0x7f215808c100 con 0x7f216015f4e0 2024-04-26T23:55:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.357+0000 7f2165800640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 0x7f214006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.358+0000 7f2165800640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 0x7f214006edb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2150000f60 tx=0x7f2150007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.361+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f215805a660 con 0x7f216015f4e0 2024-04-26T23:55:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.563+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21600d33d0 con 0x7f216015f4e0 2024-04-26T23:55:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.565+0000 7f215d600640 1 -- 172.21.15.46:0/2843388372 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) ==== 74+0+13138 (secure 0 0 0) 0x7f2158059f60 con 0x7f216015f4e0 2024-04-26T23:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":182,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:22.599196+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 msgr2=0x7f214006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 0x7f214006edb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2150000f60 tx=0x7f2150007c00 comp rx=0 tx=0).stop 2024-04-26T23:55:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 msgr2=0x7f21600cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f215800d830 tx=0x7f215800dcf0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 shutdown_connections 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f214006c8f0 0x7f214006edb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216015f4e0 0x7f21600cd7d0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21600d2130 0x7f216015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 --2- 172.21.15.46:0/2843388372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21600b4000 0x7f2160156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 >> 172.21.15.46:0/2843388372 conn(0x7f21600d5010 msgr2=0x7f216006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 shutdown_connections 2024-04-26T23:55:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.568+0000 7f2166c00640 1 -- 172.21.15.46:0/2843388372 wait complete. 2024-04-26T23:55:23.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:55:23.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:55:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.791+0000 7f8cab200640 1 Processor -- start 2024-04-26T23:55:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.791+0000 7f8cab200640 1 -- start start 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.792+0000 7f8cab200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cac0d0e00 0x7f8cac0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8cab200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8cab200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 0x7f8cac160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8cab200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8cac0d05d0 con 0x7f8cac0d0e00 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8cab200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8cac0cf530 con 0x7f8cac160b60 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8cab200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8cac0cf810 con 0x7f8cac0d11e0 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8ca9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cac0d0e00 0x7f8cac0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8ca9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cac0d0e00 0x7f8cac0d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33036/0 (socket says 172.21.15.46:33036) 2024-04-26T23:55:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8ca9e00640 1 -- 172.21.15.46:0/37710460 learned_addr learned my addr 172.21.15.46:0/37710460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:23.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8caa800640 1 --2- 172.21.15.46:0/37710460 >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 0x7f8cac160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.793+0000 7f8ca9400640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9e00640 1 -- 172.21.15.46:0/37710460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 msgr2=0x7f8cac15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:23.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac15ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9e00640 1 -- 172.21.15.46:0/37710460 >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 msgr2=0x7f8cac160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/37710460 >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 0x7f8cac160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9e00640 1 -- 172.21.15.46:0/37710460 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac0cfaf0 con 0x7f8cac0d0e00 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8caa800640 1 --2- 172.21.15.46:0/37710460 >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 0x7f8cac160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.794+0000 7f8ca9400640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac15ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:23.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.795+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac0d38a0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0cf2d0 tx=0x7f8c9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a35088ff6c1ade server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.795+0000 7f8ca8a00640 1 -- 172.21.15.46:0/37710460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c037070 con 0x7f8cac0d0e00 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.795+0000 7f8ca8a00640 1 -- 172.21.15.46:0/37710460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c032440 con 0x7f8cac0d0e00 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.795+0000 7f8ca8a00640 1 -- 172.21.15.46:0/37710460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c032810 con 0x7f8cac0d0e00 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 -- 172.21.15.46:0/37710460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 msgr2=0x7f8cac0d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac0d38a0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0cf2d0 tx=0x7f8c9c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 -- 172.21.15.46:0/37710460 shutdown_connections 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 --2- 172.21.15.46:0/37710460 >> v2:172.21.15.142:3300/0 conn(0x7f8cac160b60 0x7f8cac160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac15ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 --2- 172.21.15.46:0/37710460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac0d38a0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 -- 172.21.15.46:0/37710460 >> 172.21.15.46:0/37710460 conn(0x7f8cac0d2d80 msgr2=0x7f8cac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 -- 172.21.15.46:0/37710460 shutdown_connections 2024-04-26T23:55:23.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.796+0000 7f8cab200640 1 -- 172.21.15.46:0/37710460 wait complete. 2024-04-26T23:55:23.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.798+0000 7f8cab200640 1 Processor -- start 2024-04-26T23:55:23.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.798+0000 7f8cab200640 1 -- start start 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 0x7f8cac14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac0d1640 con 0x7f8cac0d0e00 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac0d00b0 con 0x7f8cac160b60 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8cab200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8cac15e330 con 0x7f8cac0d11e0 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33050/0 (socket says 172.21.15.46:33050) 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 -- 172.21.15.46:0/3125777835 learned_addr learned my addr 172.21.15.46:0/3125777835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9400640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8caa800640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 0x7f8cac14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 -- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 msgr2=0x7f8cac0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 -- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 msgr2=0x7f8cac14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 0x7f8cac14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8ca9e00640 1 -- 172.21.15.46:0/3125777835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac15b840 con 0x7f8cac0d0e00 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.799+0000 7f8caa800640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 0x7f8cac14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8ca9400640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8ca9e00640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c041020 con 0x7f8cac0d0e00 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c030970 con 0x7f8cac0d0e00 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c037040 con 0x7f8cac0d0e00 2024-04-26T23:55:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac15bb20 con 0x7f8cac0d0e00 2024-04-26T23:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.800+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac15c0e0 con 0x7f8cac0d0e00 2024-04-26T23:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.802+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac069ac0 con 0x7f8cac0d0e00 2024-04-26T23:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.803+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c9c044020 con 0x7f8cac0d0e00 2024-04-26T23:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.803+0000 7f8ca2000640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.804+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5249+0+0 (secure 0 0 0) 0x7f8c9c0ac410 con 0x7f8cac0d0e00 2024-04-26T23:55:23.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.804+0000 7f8ca9400640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.806+0000 7f8ca9400640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 0x7f8c8806edb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8c94006fd0 tx=0x7f8c94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:23.808+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c9c04b030 con 0x7f8cac0d0e00 2024-04-26T23:55:24.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.016+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cac074330 con 0x7f8cac0d0e00 2024-04-26T23:55:24.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.017+0000 7f8ca2000640 1 -- 172.21.15.46:0/3125777835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) ==== 93+0+10 (secure 0 0 0) 0x7f8c9c07a9f0 con 0x7f8cac0d0e00 2024-04-26T23:55:24.019 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.020+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 msgr2=0x7f8c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.020+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 0x7f8c8806edb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8c94006fd0 tx=0x7f8c94008040 comp rx=0 tx=0).stop 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.020+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 msgr2=0x7f8cac159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.020+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 shutdown_connections 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cac160b60 0x7f8cac14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cac0d11e0 0x7f8cac0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 --2- 172.21.15.46:0/3125777835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cac0d0e00 0x7f8cac159d80 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 >> 172.21.15.46:0/3125777835 conn(0x7f8cac0d2d80 msgr2=0x7f8cac15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:24.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 shutdown_connections 2024-04-26T23:55:24.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.021+0000 7f8cab200640 1 -- 172.21.15.46:0/3125777835 wait complete. 2024-04-26T23:55:24.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:55:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.243+0000 7f1bc1800640 1 Processor -- start 2024-04-26T23:55:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.244+0000 7f1bc1800640 1 -- start start 2024-04-26T23:55:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.245+0000 7f1bc1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1bbc15f4b0 0x7f1bbc0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.245+0000 7f1bc1800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bbc0cfd60 con 0x7f1bbc0d55e0 2024-04-26T23:55:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.245+0000 7f1bc1800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bbc0d2910 con 0x7f1bbc0b5140 2024-04-26T23:55:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bc1800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1bbc15f890 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1bbc0d55e0 0x7f1bbc0d2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40992/0 (socket says 172.21.15.46:40992) 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbb400640 1 -- 172.21.15.46:0/1036524855 learned_addr learned my addr 172.21.15.46:0/1036524855 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbaa00640 1 --2- 172.21.15.46:0/1036524855 >> v2:172.21.15.46:3300/0 conn(0x7f1bbc15f4b0 0x7f1bbc0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bb9600640 1 -- 172.21.15.46:0/1036524855 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224472642 0 0) 0x7f1bb4001110 con 0x7f1bbc0d55e0 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bb9600640 1 -- 172.21.15.46:0/1036524855 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b9c003520 con 0x7f1bbc0d55e0 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbaa00640 1 -- 172.21.15.46:0/1036524855 >> v1:172.21.15.142:6789/0 conn(0x7f1bbc0b5140 legacy=0x7f1bbc0d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbaa00640 1 -- 172.21.15.46:0/1036524855 >> v1:172.21.15.46:6789/0 conn(0x7f1bbc0d55e0 legacy=0x7f1bbc0d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.246+0000 7f1bbaa00640 1 -- 172.21.15.46:0/1036524855 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc0b55d0 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bbaa00640 1 --2- 172.21.15.46:0/1036524855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0d23d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0097a0 tx=0x7f1bac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ababac2478ef4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bb9600640 1 -- 172.21.15.46:0/1036524855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac037070 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bb9600640 1 -- 172.21.15.46:0/1036524855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bac032440 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bb9600640 1 -- 172.21.15.46:0/1036524855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac032810 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bc1800640 1 -- 172.21.15.46:0/1036524855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc15f4b0 msgr2=0x7f1bbc0d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.248+0000 7f1bc1800640 1 --2- 172.21.15.46:0/1036524855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0d23d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0097a0 tx=0x7f1bac02a230 comp rx=0 tx=0).stop 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.249+0000 7f1bc1800640 1 -- 172.21.15.46:0/1036524855 shutdown_connections 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.249+0000 7f1bc1800640 1 --2- 172.21.15.46:0/1036524855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0d23d0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.249+0000 7f1bc1800640 1 -- 172.21.15.46:0/1036524855 >> 172.21.15.46:0/1036524855 conn(0x7f1bbc08eb50 msgr2=0x7f1bbc09e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.249+0000 7f1bc1800640 1 -- 172.21.15.46:0/1036524855 shutdown_connections 2024-04-26T23:55:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.249+0000 7f1bc1800640 1 -- 172.21.15.46:0/1036524855 wait complete. 2024-04-26T23:55:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.250+0000 7f1bc1800640 1 Processor -- start 2024-04-26T23:55:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.250+0000 7f1bc1800640 1 -- start start 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0d55e0 0x7f1bbc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc0b5e70 con 0x7f1bbc0cc170 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc0b5b90 con 0x7f1bbc15f4b0 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.251+0000 7f1bc1800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1bbc0b4680 con 0x7f1bbc0d55e0 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33068/0 (socket says 172.21.15.46:33068) 2024-04-26T23:55:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 -- 172.21.15.46:0/3776605836 learned_addr learned my addr 172.21.15.46:0/3776605836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbaa00640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0d55e0 0x7f1bbc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bba000640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 -- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0d55e0 msgr2=0x7f1bbc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0d55e0 0x7f1bbc0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 -- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15f4b0 msgr2=0x7f1bbc0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.252+0000 7f1bbb400640 1 -- 172.21.15.46:0/3776605836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc0ce160 con 0x7f1bbc0cc170 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bbb400640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400c730 tx=0x7f1bb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4017070 con 0x7f1bbc0cc170 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb40058b0 con 0x7f1bbc0cc170 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4005cc0 con 0x7f1bbc0cc170 2024-04-26T23:55:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc0cce60 con 0x7f1bbc0cc170 2024-04-26T23:55:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.253+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc0cd420 con 0x7f1bbc0cc170 2024-04-26T23:55:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.254+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc069530 con 0x7f1bbc0cc170 2024-04-26T23:55:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.256+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bb401a070 con 0x7f1bbc0cc170 2024-04-26T23:55:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.257+0000 7f1bb2a00640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 0x7f1b9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.257+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5249+0+0 (secure 0 0 0) 0x7f1bb408afd0 con 0x7f1bbc0cc170 2024-04-26T23:55:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.257+0000 7f1bbaa00640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 0x7f1b9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.259+0000 7f1bbaa00640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 0x7f1b9806edb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1bac034000 tx=0x7f1bac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.262+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bbc0cfd60 con 0x7f1bbc0cc170 2024-04-26T23:55:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.467+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1bbc0d4f70 con 0x7f1bbc0cc170 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.468+0000 7f1bb2a00640 1 -- 172.21.15.46:0/3776605836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v182) ==== 84+0+65 (secure 0 0 0) 0x7f1bb4059520 con 0x7f1bbc0cc170 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:55:24.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 msgr2=0x7f1b9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 0x7f1b9806edb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1bac034000 tx=0x7f1bac02e000 comp rx=0 tx=0).stop 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 msgr2=0x7f1bbc15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400c730 tx=0x7f1bb400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 shutdown_connections 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1b9806c8f0 0x7f1b9806edb0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc0cc170 0x7f1bbc15a1f0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15f4b0 0x7f1bbc0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 --2- 172.21.15.46:0/3776605836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0d55e0 0x7f1bbc0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 >> 172.21.15.46:0/3776605836 conn(0x7f1bbc08eb50 msgr2=0x7f1bbc0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:24.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 shutdown_connections 2024-04-26T23:55:24.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.471+0000 7f1bc1800640 1 -- 172.21.15.46:0/3776605836 wait complete. 2024-04-26T23:55:24.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:55:24.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:55:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.708+0000 7f1c26000640 1 Processor -- start 2024-04-26T23:55:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.708+0000 7f1c26000640 1 -- start start 2024-04-26T23:55:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.709+0000 7f1c26000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c201627f0 0x7f1c200cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.709+0000 7f1c26000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 0x7f1c200b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.709+0000 7f1c26000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c200d01a0 con 0x7f1c200d2080 2024-04-26T23:55:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.709+0000 7f1c26000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1c200d0480 con 0x7f1c201627f0 2024-04-26T23:55:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.709+0000 7f1c26000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1c200d0760 con 0x7f1c200d3cc0 2024-04-26T23:55:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c24c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c201627f0 0x7f1c200cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c1fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c200d2080 0x7f1c200d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41004/0 (socket says 172.21.15.46:41004) 2024-04-26T23:55:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c1fe00640 1 -- 172.21.15.46:0/347385142 learned_addr learned my addr 172.21.15.46:0/347385142 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c25600640 1 --2- 172.21.15.46:0/347385142 >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 0x7f1c200b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c24c00640 1 -- 172.21.15.46:0/347385142 >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 msgr2=0x7f1c200b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c24c00640 1 --2- 172.21.15.46:0/347385142 >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 0x7f1c200b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.710+0000 7f1c24c00640 1 -- 172.21.15.46:0/347385142 >> v1:172.21.15.46:6789/0 conn(0x7f1c200d2080 legacy=0x7f1c200d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.711+0000 7f1c24c00640 1 -- 172.21.15.46:0/347385142 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c200d0a40 con 0x7f1c201627f0 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.711+0000 7f1c1f400640 1 -- 172.21.15.46:0/347385142 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648911505 0 0) 0x7f1c200d01a0 con 0x7f1c200d2080 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.711+0000 7f1c25600640 1 --2- 172.21.15.46:0/347385142 >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 0x7f1c200b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c24c00640 1 --2- 172.21.15.46:0/347385142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200cfbc0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f1c2006a670 tx=0x7f1c1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ef342680d77af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c1f400640 1 -- 172.21.15.46:0/347385142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10037070 con 0x7f1c201627f0 2024-04-26T23:55:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c1f400640 1 -- 172.21.15.46:0/347385142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c10032440 con 0x7f1c201627f0 2024-04-26T23:55:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c1f400640 1 -- 172.21.15.46:0/347385142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10032810 con 0x7f1c201627f0 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c26000640 1 -- 172.21.15.46:0/347385142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 msgr2=0x7f1c200cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.712+0000 7f1c26000640 1 --2- 172.21.15.46:0/347385142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200cfbc0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f1c2006a670 tx=0x7f1c1002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 -- 172.21.15.46:0/347385142 shutdown_connections 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 --2- 172.21.15.46:0/347385142 >> v2:172.21.15.142:3300/0 conn(0x7f1c200d3cc0 0x7f1c200b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 --2- 172.21.15.46:0/347385142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200cfbc0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 -- 172.21.15.46:0/347385142 >> 172.21.15.46:0/347385142 conn(0x7f1c2009e380 msgr2=0x7f1c200d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 -- 172.21.15.46:0/347385142 shutdown_connections 2024-04-26T23:55:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.713+0000 7f1c26000640 1 -- 172.21.15.46:0/347385142 wait complete. 2024-04-26T23:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.714+0000 7f1c26000640 1 Processor -- start 2024-04-26T23:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.715+0000 7f1c26000640 1 -- start start 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.715+0000 7f1c26000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c200d2080 0x7f1c2015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c26000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c200d3cc0 0x7f1c200b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c26000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c26000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20162bd0 con 0x7f1c201627f0 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c26000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1c200d24f0 con 0x7f1c200d2080 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c26000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1c200b4a90 con 0x7f1c200d3cc0 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33084/0 (socket says 172.21.15.46:33084) 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 -- 172.21.15.46:0/743522135 learned_addr learned my addr 172.21.15.46:0/743522135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c24c00640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c200d2080 0x7f1c2015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c1fe00640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c200d3cc0 0x7f1c200b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 -- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c200d3cc0 msgr2=0x7f1c200b9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c200d3cc0 0x7f1c200b9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 -- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c200d2080 msgr2=0x7f1c2015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c200d2080 0x7f1c2015c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.716+0000 7f1c25600640 1 -- 172.21.15.46:0/743522135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c200ba8a0 con 0x7f1c201627f0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c25600640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7f0 tx=0x7f1c1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18015070 con 0x7f1c201627f0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c18004240 con 0x7f1c201627f0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18007620 con 0x7f1c201627f0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c200ccf00 con 0x7f1c201627f0 2024-04-26T23:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.717+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c200cd4c0 con 0x7f1c201627f0 2024-04-26T23:55:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.718+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c200693b0 con 0x7f1c201627f0 2024-04-26T23:55:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.719+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c180079d0 con 0x7f1c201627f0 2024-04-26T23:55:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.720+0000 7f1c1cc00640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.720+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5249+0+0 (secure 0 0 0) 0x7f1c1808bb70 con 0x7f1c201627f0 2024-04-26T23:55:24.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.721+0000 7f1c24c00640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.722+0000 7f1c24c00640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c10034000 tx=0x7f1c1002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.725+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c1805a0c0 con 0x7f1c201627f0 2024-04-26T23:55:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:24.929+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1c200d56e0 con 0x7f1c201627f0 2024-04-26T23:55:25.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.624+0000 7f1c1cc00640 1 -- 172.21.15.46:0/743522135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v183) ==== 138+0+0 (secure 0 0 0) 0x7f1c180599c0 con 0x7f1c201627f0 2024-04-26T23:55:25.626 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:55:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 msgr2=0x7f1bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c10034000 tx=0x7f1c1002e000 comp rx=0 tx=0).stop 2024-04-26T23:55:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 msgr2=0x7f1c200b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7f0 tx=0x7f1c1800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 shutdown_connections 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c201627f0 0x7f1c200b5ca0 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c200d3cc0 0x7f1c200b9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 --2- 172.21.15.46:0/743522135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c200d2080 0x7f1c2015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.628+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 >> 172.21.15.46:0/743522135 conn(0x7f1c2009e380 msgr2=0x7f1c200d5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.629+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 shutdown_connections 2024-04-26T23:55:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.629+0000 7f1c26000640 1 -- 172.21.15.46:0/743522135 wait complete. 2024-04-26T23:55:25.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:55:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.862+0000 7fb528c00640 1 Processor -- start 2024-04-26T23:55:25.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.863+0000 7fb528c00640 1 -- start start 2024-04-26T23:55:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.864+0000 7fb528c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb52415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.864+0000 7fb528c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5240d07e0 con 0x7fb52415e050 2024-04-26T23:55:25.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.864+0000 7fb528c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52406eb90 con 0x7fb52415d730 2024-04-26T23:55:25.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.864+0000 7fb528c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5240b4310 con 0x7fb5240b53b0 2024-04-26T23:55:25.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.865+0000 7fb522800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb52415e050 0x7fb52406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41018/0 (socket says 172.21.15.46:41018) 2024-04-26T23:55:25.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.865+0000 7fb522800640 1 -- 172.21.15.46:0/3061576239 learned_addr learned my addr 172.21.15.46:0/3061576239 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.865+0000 7fb521400640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb52415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.865+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630387144 0 0) 0x7fb5240d07e0 con 0x7fb52415e050 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.865+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb508003520 con 0x7fb52415e050 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1389766801 0 0) 0x7fb508003520 con 0x7fb52415e050 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 msgr2=0x7fb52415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb52415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 >> v1:172.21.15.142:6789/0 conn(0x7fb52415d730 legacy=0x7fb5240d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5240b45f0 con 0x7fb52415e050 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb51c0019f0 con 0x7fb52415e050 2024-04-26T23:55:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 >> v1:172.21.15.46:6789/0 conn(0x7fb52415e050 legacy=0x7fb52406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb5080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb508004060 0x7fb508004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb508004d40 0x7fb5080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5240b4310 con 0x7fb508004060 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb50800bc00 con 0x7fb508004d40 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.866+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb50800be00 con 0x7fb5240b53b0 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb508004060 0x7fb508004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb522800640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb508004d40 0x7fb5080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521e00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb5080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 -- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 msgr2=0x7fb5080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5240b53b0 0x7fb5080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 -- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb508004d40 msgr2=0x7fb5080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb508004d40 0x7fb5080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.867+0000 7fb521400640 1 -- 172.21.15.46:0/3061576239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb508003520 con 0x7fb508004060 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.869+0000 7fb521400640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb508004060 0x7fb508004800 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fb51000ba30 tx=0x7fb508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34deb0853ce6122 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.869+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510030070 con 0x7fb508004060 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.869+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51002ade0 con 0x7fb508004060 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.869+0000 7fb520a00640 1 -- 172.21.15.46:0/3061576239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000b600 con 0x7fb508004060 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.869+0000 7fb528c00640 1 -- 172.21.15.46:0/3061576239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb508004060 msgr2=0x7fb508004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb528c00640 1 --2- 172.21.15.46:0/3061576239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb508004060 0x7fb508004800 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fb51000ba30 tx=0x7fb508005830 comp rx=0 tx=0).stop 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb522800640 1 -- 172.21.15.46:0/3061576239 reap_dead start 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb528c00640 1 -- 172.21.15.46:0/3061576239 shutdown_connections 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb528c00640 1 -- 172.21.15.46:0/3061576239 >> 172.21.15.46:0/3061576239 conn(0x7fb52406c980 msgr2=0x7fb5240b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb528c00640 1 -- 172.21.15.46:0/3061576239 shutdown_connections 2024-04-26T23:55:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.870+0000 7fb528c00640 1 -- 172.21.15.46:0/3061576239 wait complete. 2024-04-26T23:55:25.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.872+0000 7fb528c00640 1 Processor -- start 2024-04-26T23:55:25.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.872+0000 7fb528c00640 1 -- start start 2024-04-26T23:55:25.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb508004060 0x7fb52415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb52415c100 0x7fb52415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5240b4e90 con 0x7fb524137220 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5240b4bb0 con 0x7fb52415c100 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb528c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb52415e4c0 con 0x7fb508004060 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb521400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb521400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33096/0 (socket says 172.21.15.46:33096) 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb521400640 1 -- 172.21.15.46:0/4162252448 learned_addr learned my addr 172.21.15.46:0/4162252448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.873+0000 7fb521e00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb508004060 0x7fb52415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb522800640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb52415c100 0x7fb52415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 -- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb508004060 msgr2=0x7fb52415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb508004060 0x7fb52415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 -- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb52415c100 msgr2=0x7fb52415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb52415c100 0x7fb52415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 -- 172.21.15.46:0/4162252448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52415ada0 con 0x7fb524137220 2024-04-26T23:55:25.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.874+0000 7fb521400640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fb51000bb50 tx=0x7fb5100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:25.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.875+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510043030 con 0x7fb524137220 2024-04-26T23:55:25.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.875+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51002abe0 con 0x7fb524137220 2024-04-26T23:55:25.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.875+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000b600 con 0x7fb524137220 2024-04-26T23:55:25.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.875+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb524137770 con 0x7fb524137220 2024-04-26T23:55:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.875+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb524137d30 con 0x7fb524137220 2024-04-26T23:55:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.876+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52406faa0 con 0x7fb524137220 2024-04-26T23:55:25.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.877+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb510046070 con 0x7fb524137220 2024-04-26T23:55:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.878+0000 7fb51a000640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 0x7fb50406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.878+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5619+0+0 (secure 0 0 0) 0x7fb5100acd00 con 0x7fb524137220 2024-04-26T23:55:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.879+0000 7fb521e00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 0x7fb50406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.880+0000 7fb521e00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 0x7fb50406edb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb514007fd0 tx=0x7fb514009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:25.883+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb52406faa0 con 0x7fb524137220 2024-04-26T23:55:26.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.087+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb524069530 con 0x7fb524137220 2024-04-26T23:55:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.630+0000 7fb51a000640 1 -- 172.21.15.46:0/4162252448 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v184) ==== 170+0+0 (secure 0 0 0) 0x7fb51007b260 con 0x7fb524137220 2024-04-26T23:55:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.635+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 msgr2=0x7fb50406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.635+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 0x7fb50406edb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb514007fd0 tx=0x7fb514009040 comp rx=0 tx=0).stop 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.635+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 msgr2=0x7fb5241e37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.635+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fb51000bb50 tx=0x7fb5100046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 shutdown_connections 2024-04-26T23:55:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb50406c8f0 0x7fb50406edb0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb52415c100 0x7fb52415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb524137220 0x7fb5241e37e0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 --2- 172.21.15.46:0/4162252448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb508004060 0x7fb52415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 >> 172.21.15.46:0/4162252448 conn(0x7fb52406c980 msgr2=0x7fb52415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.636+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 shutdown_connections 2024-04-26T23:55:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.637+0000 7fb528c00640 1 -- 172.21.15.46:0/4162252448 wait complete. 2024-04-26T23:55:26.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:55:26.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.850+0000 7f8e16600640 1 Processor -- start 2024-04-26T23:55:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.850+0000 7f8e16600640 1 -- start start 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e16600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e10160510 0x7f8e100cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e16600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e100cbe60 con 0x7f8e100cda90 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e16600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e100cc140 con 0x7f8e10160130 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e16600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8e100cca70 con 0x7f8e10160510 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e15c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e100cda90 0x7f8e10072220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41034/0 (socket says 172.21.15.46:41034) 2024-04-26T23:55:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.852+0000 7f8e15c00640 1 -- 172.21.15.46:0/1991314412 learned_addr learned my addr 172.21.15.46:0/1991314412 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e07e00640 1 --2- 172.21.15.46:0/1991314412 >> v2:172.21.15.142:3300/0 conn(0x7f8e10160510 0x7f8e100cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523431678 0 0) 0x7f8e100cbe60 con 0x7f8e100cda90 2024-04-26T23:55:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8df0003520 con 0x7f8e100cda90 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2467228097 0 0) 0x7f8df0003520 con 0x7f8e100cda90 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 >> v2:172.21.15.142:3300/0 conn(0x7f8e10160510 msgr2=0x7f8e100cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 --2- 172.21.15.46:0/1991314412 >> v2:172.21.15.142:3300/0 conn(0x7f8e10160510 0x7f8e100cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 >> v1:172.21.15.142:6789/0 conn(0x7f8e10160130 legacy=0x7f8e1015cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e100cc420 con 0x7f8e100cda90 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8e080019f0 con 0x7f8e100cda90 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 >> v1:172.21.15.46:6789/0 conn(0x7f8e100cda90 legacy=0x7f8e10072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.853+0000 7f8e0fe00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e0fe00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e0fe00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8df00051c0 0x7f8df000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e100cca70 con 0x7f8df0004800 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8df000c160 con 0x7f8df00039b0 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8df000c360 con 0x7f8df00051c0 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e15200640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e15c00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8df00051c0 0x7f8df000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 -- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8df00051c0 msgr2=0x7f8df000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8df00051c0 0x7f8df000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 -- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df00039b0 msgr2=0x7f8df0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e07e00640 1 -- 172.21.15.46:0/1991314412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df0003520 con 0x7f8df0004800 2024-04-26T23:55:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.854+0000 7f8e15c00640 1 -- 172.21.15.46:0/1991314412 reap_dead start 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.855+0000 7f8e07e00640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea7744a0cea1e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.856+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00036070 con 0x7f8df0004800 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.856+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e00031400 con 0x7f8df0004800 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.856+0000 7f8e0fe00640 1 -- 172.21.15.46:0/1991314412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00009690 con 0x7f8df0004800 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.856+0000 7f8e16600640 1 -- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 msgr2=0x7f8df0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.856+0000 7f8e16600640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8df0005830 comp rx=0 tx=0).stop 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.857+0000 7f8e16600640 1 -- 172.21.15.46:0/1991314412 shutdown_connections 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.857+0000 7f8e16600640 1 --2- 172.21.15.46:0/1991314412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.857+0000 7f8e16600640 1 -- 172.21.15.46:0/1991314412 >> 172.21.15.46:0/1991314412 conn(0x7f8e100b27b0 msgr2=0x7f8e100b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.857+0000 7f8e16600640 1 -- 172.21.15.46:0/1991314412 shutdown_connections 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.857+0000 7f8e16600640 1 -- 172.21.15.46:0/1991314412 wait complete. 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.858+0000 7f8e16600640 1 Processor -- start 2024-04-26T23:55:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.859+0000 7f8e16600640 1 -- start start 2024-04-26T23:55:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df0004800 0x7f8e10147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 0x7f8e101e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e100cd6d0 con 0x7f8e101e5600 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e100cd550 con 0x7f8df0004800 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e16600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8e100cd850 con 0x7f8e101e7820 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33116/0 (socket says 172.21.15.46:33116) 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 -- 172.21.15.46:0/388320617 learned_addr learned my addr 172.21.15.46:0/388320617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e15c00640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 0x7f8e101e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e15200640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df0004800 0x7f8e10147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 -- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 msgr2=0x7f8e101e7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 0x7f8e101e7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 -- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df0004800 msgr2=0x7f8e10147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df0004800 0x7f8e10147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.860+0000 7f8e07e00640 1 -- 172.21.15.46:0/388320617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e10155200 con 0x7f8e101e5600 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e15c00640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 0x7f8e101e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e07e00640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f8e00034040 tx=0x7f8e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0003f020 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e00037040 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00036070 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e101e5a90 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.861+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101e6050 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.863+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e100693b0 con 0x7f8e101e5600 2024-04-26T23:55:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.864+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e00042020 con 0x7f8e101e5600 2024-04-26T23:55:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.865+0000 7f8e0d600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 0x7f8dec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.865+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 6001+0+0 (secure 0 0 0) 0x7f8e000abe90 con 0x7f8e101e5600 2024-04-26T23:55:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.865+0000 7f8e15200640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 0x7f8dec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:26.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.866+0000 7f8e15200640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 0x7f8dec06ede0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8df8007fd0 tx=0x7f8df8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:26.871+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8e00049030 con 0x7f8e101e5600 2024-04-26T23:55:27.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.116+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8e100cbb10 con 0x7f8e101e5600 2024-04-26T23:55:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.666+0000 7f8e0d600640 1 -- 172.21.15.46:0/388320617 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v145) ==== 174+0+0 (secure 0 0 0) 0x7f8e000ad070 con 0x7f8e101e5600 2024-04-26T23:55:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 27 and data pool 28 2024-04-26T23:55:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 msgr2=0x7f8dec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 0x7f8dec06ede0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8df8007fd0 tx=0x7f8df8009040 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 msgr2=0x7f8e100730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f8e00034040 tx=0x7f8e00004730 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 shutdown_connections 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8dec06c920 0x7f8dec06ede0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e101e7820 0x7f8e101e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e101e5600 0x7f8e100730b0 secure :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f8e00034040 tx=0x7f8e00004730 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 --2- 172.21.15.46:0/388320617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df0004800 0x7f8e10147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.673+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 >> 172.21.15.46:0/388320617 conn(0x7f8e100b27b0 msgr2=0x7f8e1015ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.675+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 shutdown_connections 2024-04-26T23:55:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.676+0000 7f8e16600640 1 -- 172.21.15.46:0/388320617 wait complete. 2024-04-26T23:55:27.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:27.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.889+0000 7fe41f000640 1 Processor -- start 2024-04-26T23:55:27.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.889+0000 7fe41f000640 1 -- start start 2024-04-26T23:55:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.890+0000 7fe41f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe41806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.890+0000 7fe41f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4180732c0 0x7fe418073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.890+0000 7fe41f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41806ee30 con 0x7fe418160bd0 2024-04-26T23:55:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.890+0000 7fe41f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe418161940 con 0x7fe4180732c0 2024-04-26T23:55:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.890+0000 7fe41f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe418071f80 con 0x7fe41806e510 2024-04-26T23:55:27.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe417400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe418160bd0 0x7fe418161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41040/0 (socket says 172.21.15.46:41040) 2024-04-26T23:55:27.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe417400640 1 -- 172.21.15.46:0/1998306170 learned_addr learned my addr 172.21.15.46:0/1998306170 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe417e00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe41806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe41d000640 1 --2- 172.21.15.46:0/1998306170 >> v2:172.21.15.142:3300/0 conn(0x7fe4180732c0 0x7fe418073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886981822 0 0) 0x7fe41806ee30 con 0x7fe418160bd0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.891+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3fc003520 con 0x7fe418160bd0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2386718093 0 0) 0x7fe3fc003520 con 0x7fe418160bd0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 msgr2=0x7fe41806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe41806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 >> v2:172.21.15.142:3300/0 conn(0x7fe4180732c0 msgr2=0x7fe418073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 --2- 172.21.15.46:0/1998306170 >> v2:172.21.15.142:3300/0 conn(0x7fe4180732c0 0x7fe418073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe418072260 con 0x7fe418160bd0 2024-04-26T23:55:27.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe400001670 con 0x7fe418160bd0 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 >> v1:172.21.15.46:6789/0 conn(0x7fe418160bd0 legacy=0x7fe418161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe3fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3fc003f30 0x7fe3fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc004d40 0x7fe3fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe418161940 con 0x7fe3fc003f30 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe418071f80 con 0x7fe3fc004d40 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc00b9c0 con 0x7fe41806e510 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe417400640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3fc003f30 0x7fe3fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe41d000640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc004d40 0x7fe3fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.892+0000 7fe417e00640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe3fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.893+0000 7fe417400640 1 -- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 msgr2=0x7fe3fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.893+0000 7fe417400640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe41806e510 0x7fe3fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.893+0000 7fe417400640 1 -- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc004d40 msgr2=0x7fe3fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:27.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.893+0000 7fe417400640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc004d40 0x7fe3fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.893+0000 7fe417400640 1 -- 172.21.15.46:0/1998306170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc003520 con 0x7fe3fc003f30 2024-04-26T23:55:27.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe417400640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3fc003f30 0x7fe3fc004800 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c240 tx=0x7fe3fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a24bc0c1ada86a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:27.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400007d30 con 0x7fe3fc003f30 2024-04-26T23:55:27.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe400005ad0 con 0x7fe3fc003f30 2024-04-26T23:55:27.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe416a00640 1 -- 172.21.15.46:0/1998306170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400007070 con 0x7fe3fc003f30 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe41f000640 1 -- 172.21.15.46:0/1998306170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3fc003f30 msgr2=0x7fe3fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe41f000640 1 --2- 172.21.15.46:0/1998306170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3fc003f30 0x7fe3fc004800 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c240 tx=0x7fe3fc005830 comp rx=0 tx=0).stop 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.894+0000 7fe41d000640 1 -- 172.21.15.46:0/1998306170 reap_dead start 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.895+0000 7fe41f000640 1 -- 172.21.15.46:0/1998306170 shutdown_connections 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.895+0000 7fe41f000640 1 -- 172.21.15.46:0/1998306170 >> 172.21.15.46:0/1998306170 conn(0x7fe41806c530 msgr2=0x7fe41806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.895+0000 7fe41f000640 1 -- 172.21.15.46:0/1998306170 shutdown_connections 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.895+0000 7fe41f000640 1 -- 172.21.15.46:0/1998306170 wait complete. 2024-04-26T23:55:27.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.896+0000 7fe41f000640 1 Processor -- start 2024-04-26T23:55:27.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.897+0000 7fe41f000640 1 -- start start 2024-04-26T23:55:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.897+0000 7fe41f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc003f30 0x7fe4181f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4181f7ef0 0x7fe4180af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe418072d40 con 0x7fe4180af5f0 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe418072820 con 0x7fe3fc003f30 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe418072ec0 con 0x7fe4181f7ef0 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33126/0 (socket says 172.21.15.46:33126) 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 -- 172.21.15.46:0/1870715152 learned_addr learned my addr 172.21.15.46:0/1870715152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe417e00640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc003f30 0x7fe4181f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe417400640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4181f7ef0 0x7fe4180af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 -- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4181f7ef0 msgr2=0x7fe4180af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4181f7ef0 0x7fe4180af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 -- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc003f30 msgr2=0x7fe4181f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc003f30 0x7fe4181f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.898+0000 7fe41d000640 1 -- 172.21.15.46:0/1870715152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe418143790 con 0x7fe4180af5f0 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe41d000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe4180d0490 tx=0x7fe4100115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410011e10 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41001b070 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe418144170 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410003990 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.899+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe418144730 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.900+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe418069530 con 0x7fe4180af5f0 2024-04-26T23:55:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.901+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe410005d10 con 0x7fe4180af5f0 2024-04-26T23:55:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.903+0000 7fe40fe00640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 0x7fe3f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.903+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 6101+0+0 (secure 0 0 0) 0x7fe41008c2b0 con 0x7fe4180af5f0 2024-04-26T23:55:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.903+0000 7fe417e00640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 0x7fe3f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.904+0000 7fe417e00640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 0x7fe3f406ede0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe4180afdb0 tx=0x7fe408008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:27.908+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe410025020 con 0x7fe4180af5f0 2024-04-26T23:55:28.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.111+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe41806fb00 con 0x7fe4180af5f0 2024-04-26T23:55:28.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.115+0000 7fe40fe00640 1 -- 172.21.15.46:0/1870715152 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) ==== 74+0+16929 (secure 0 0 0) 0x7fe41001d310 con 0x7fe4180af5f0 2024-04-26T23:55:28.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:28.117 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":185,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:27.117170+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.117+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 msgr2=0x7fe3f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.117+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 0x7fe3f406ede0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe4180afdb0 tx=0x7fe408008040 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.117+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 msgr2=0x7fe418142a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.117+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe4180d0490 tx=0x7fe4100115a0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 shutdown_connections 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe3f406c920 0x7fe3f406ede0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4180af5f0 0x7fe418142a20 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4181f7ef0 0x7fe4180af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 --2- 172.21.15.46:0/1870715152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3fc003f30 0x7fe4181f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 >> 172.21.15.46:0/1870715152 conn(0x7fe41806c530 msgr2=0x7fe418141f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 shutdown_connections 2024-04-26T23:55:28.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.118+0000 7fe41f000640 1 -- 172.21.15.46:0/1870715152 wait complete. 2024-04-26T23:55:28.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:28.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.350+0000 7f1099c00640 1 Processor -- start 2024-04-26T23:55:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.350+0000 7f1099c00640 1 -- start start 2024-04-26T23:55:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1099c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10940d1a60 0x7f10940cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1099c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10940d3c30 con 0x7f109415d080 2024-04-26T23:55:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1099c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10940cdb40 con 0x7f10940d0be0 2024-04-26T23:55:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1099c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f10940cdde0 con 0x7f10940d1a60 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1093e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f109415d080 0x7f10940d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41056/0 (socket says 172.21.15.46:41056) 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.351+0000 7f1093e00640 1 -- 172.21.15.46:0/461153758 learned_addr learned my addr 172.21.15.46:0/461153758 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093485826 0 0) 0x7f10940d3c30 con 0x7f109415d080 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1078003520 con 0x7f109415d080 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1093400640 1 --2- 172.21.15.46:0/461153758 >> v2:172.21.15.142:3300/0 conn(0x7f10940d1a60 0x7f10940cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3182160050 0 0) 0x7f1078003520 con 0x7f109415d080 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 >> v2:172.21.15.142:3300/0 conn(0x7f10940d1a60 msgr2=0x7f10940cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 --2- 172.21.15.46:0/461153758 >> v2:172.21.15.142:3300/0 conn(0x7f10940d1a60 0x7f10940cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 >> v1:172.21.15.142:6789/0 conn(0x7f10940d0be0 legacy=0x7f10940b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10940ce0c0 con 0x7f109415d080 2024-04-26T23:55:28.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f10840019f0 con 0x7f109415d080 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 >> v1:172.21.15.46:6789/0 conn(0x7f109415d080 legacy=0x7f10940d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f1078003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1078004800 0x7f1078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10780052e0 0x7f107800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10940cdde0 con 0x7f10780039b0 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f107800c1d0 con 0x7f1078004800 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f107800c3d0 con 0x7f10780052e0 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1093400640 1 --2- 172.21.15.46:0/461153758 >> v2:172.21.15.142:3300/0 conn(0x7f10940d1a60 0x7f10940cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.352+0000 7f1093e00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f1078003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1093e00640 1 -- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10780052e0 msgr2=0x7f107800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1093e00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10780052e0 0x7f107800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1093e00640 1 -- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1078004800 msgr2=0x7f1078004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1093e00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1078004800 0x7f1078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1093e00640 1 -- 172.21.15.46:0/461153758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1078003520 con 0x7f10780039b0 2024-04-26T23:55:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.353+0000 7f1099200640 1 -- 172.21.15.46:0/461153758 reap_dead start 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.354+0000 7f1093e00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f1078003db0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f108400a8f0 tx=0x7f1078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eedffd73ab8a2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.354+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108400a4c0 con 0x7f10780039b0 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.354+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108402f070 con 0x7f10780039b0 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.354+0000 7f1092a00640 1 -- 172.21.15.46:0/461153758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108402bcb0 con 0x7f10780039b0 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 -- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 msgr2=0x7f1078003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f1078003db0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f108400a8f0 tx=0x7f1078005830 comp rx=0 tx=0).stop 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 -- 172.21.15.46:0/461153758 shutdown_connections 2024-04-26T23:55:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 --2- 172.21.15.46:0/461153758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f1078003db0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 -- 172.21.15.46:0/461153758 >> 172.21.15.46:0/461153758 conn(0x7f10940d2d70 msgr2=0x7f10940d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 -- 172.21.15.46:0/461153758 shutdown_connections 2024-04-26T23:55:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.355+0000 7f1099c00640 1 -- 172.21.15.46:0/461153758 wait complete. 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.357+0000 7f1099c00640 1 Processor -- start 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.357+0000 7f1099c00640 1 -- start start 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10941545c0 0x7f10940b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 0x7f1094150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10940d3a10 con 0x7f10780039b0 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10940d3890 con 0x7f10941545c0 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1099c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f109415d460 con 0x7f10940b5ed0 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1093e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1093e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33148/0 (socket says 172.21.15.46:33148) 2024-04-26T23:55:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.358+0000 7f1093e00640 1 -- 172.21.15.46:0/2164800626 learned_addr learned my addr 172.21.15.46:0/2164800626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1099200640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 0x7f1094150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093400640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10941545c0 0x7f10940b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 -- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 msgr2=0x7f1094150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 0x7f1094150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 -- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10941545c0 msgr2=0x7f10940b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10941545c0 0x7f10940b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 -- 172.21.15.46:0/2164800626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10940caf30 con 0x7f10780039b0 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1099200640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 0x7f1094150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1093e00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f108402b500 tx=0x7f108402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108402bd20 con 0x7f10780039b0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1084034470 con 0x7f10780039b0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084034820 con 0x7f10780039b0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10940cb1a0 con 0x7f10780039b0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.359+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10940c7600 con 0x7f10780039b0 2024-04-26T23:55:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.360+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1094069ac0 con 0x7f10780039b0 2024-04-26T23:55:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.361+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10840349c0 con 0x7f10780039b0 2024-04-26T23:55:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.362+0000 7f108be00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 0x7f107006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.362+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 6101+0+0 (secure 0 0 0) 0x7f10840ac1e0 con 0x7f10780039b0 2024-04-26T23:55:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.363+0000 7f1093400640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 0x7f107006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.364+0000 7f1093400640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 0x7f107006edb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f107c006fd0 tx=0x7f107c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.367+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f108407a480 con 0x7f10780039b0 2024-04-26T23:55:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.568+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10940b62b0 con 0x7f10780039b0 2024-04-26T23:55:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.571+0000 7f108be00640 1 -- 172.21.15.46:0/2164800626 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) ==== 74+0+16929 (secure 0 0 0) 0x7f1084079d80 con 0x7f10780039b0 2024-04-26T23:55:28.574 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:28.574 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":185,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:27.117170+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 msgr2=0x7f107006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 0x7f107006edb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f107c006fd0 tx=0x7f107c008040 comp rx=0 tx=0).stop 2024-04-26T23:55:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 msgr2=0x7f10940c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f108402b500 tx=0x7f108402d070 comp rx=0 tx=0).stop 2024-04-26T23:55:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 shutdown_connections 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f107006c8f0 0x7f107006edb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10940b5ed0 0x7f1094150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10941545c0 0x7f10940b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 --2- 172.21.15.46:0/2164800626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10780039b0 0x7f10940c70c0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 >> 172.21.15.46:0/2164800626 conn(0x7f10940d2d70 msgr2=0x7f109415c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 shutdown_connections 2024-04-26T23:55:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.574+0000 7f1099c00640 1 -- 172.21.15.46:0/2164800626 wait complete. 2024-04-26T23:55:28.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:55:28.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.802+0000 7feafa000640 1 Processor -- start 2024-04-26T23:55:28.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.803+0000 7feafa000640 1 -- start start 2024-04-26T23:55:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.804+0000 7feafa000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feaf4160510 0x7feaf40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.804+0000 7feafa000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf40cbe60 con 0x7feaf4160130 2024-04-26T23:55:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.804+0000 7feafa000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf40cc140 con 0x7feaf40cda90 2024-04-26T23:55:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.804+0000 7feafa000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7feaf40cca70 con 0x7feaf4160510 2024-04-26T23:55:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.804+0000 7feaf8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feaf4160130 0x7feaf415cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41072/0 (socket says 172.21.15.46:41072) 2024-04-26T23:55:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf8c00640 1 -- 172.21.15.46:0/3282370721 learned_addr learned my addr 172.21.15.46:0/3282370721 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3e00640 1 --2- 172.21.15.46:0/3282370721 >> v2:172.21.15.46:3300/0 conn(0x7feaf4160510 0x7feaf40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965299239 0 0) 0x7feaf40cbe60 con 0x7feaf4160130 2024-04-26T23:55:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feadc003520 con 0x7feaf4160130 2024-04-26T23:55:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3e00640 1 -- 172.21.15.46:0/3282370721 >> v1:172.21.15.142:6789/0 conn(0x7feaf40cda90 legacy=0x7feaf4072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3e00640 1 -- 172.21.15.46:0/3282370721 >> v1:172.21.15.46:6789/0 conn(0x7feaf4160130 legacy=0x7feaf415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3515115010 0 0) 0x7feadc003520 con 0x7feaf4160130 2024-04-26T23:55:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.805+0000 7feaf3e00640 1 -- 172.21.15.46:0/3282370721 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf40cc420 con 0x7feaf4160510 2024-04-26T23:55:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.806+0000 7feaf3e00640 1 --2- 172.21.15.46:0/3282370721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40cd550 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7feae00097a0 tx=0x7feae002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97834705f982994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.807+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0037070 con 0x7feaf4160510 2024-04-26T23:55:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.807+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae002ab60 con 0x7feaf4160510 2024-04-26T23:55:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.807+0000 7feaf3400640 1 -- 172.21.15.46:0/3282370721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8001670 con 0x7feaf4160510 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.807+0000 7feafa000640 1 -- 172.21.15.46:0/3282370721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 msgr2=0x7feaf40cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.807+0000 7feafa000640 1 --2- 172.21.15.46:0/3282370721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40cd550 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7feae00097a0 tx=0x7feae002a0d0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.808+0000 7feafa000640 1 -- 172.21.15.46:0/3282370721 shutdown_connections 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.808+0000 7feafa000640 1 --2- 172.21.15.46:0/3282370721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40cd550 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.808+0000 7feafa000640 1 -- 172.21.15.46:0/3282370721 >> 172.21.15.46:0/3282370721 conn(0x7feaf40b27b0 msgr2=0x7feaf40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.808+0000 7feafa000640 1 -- 172.21.15.46:0/3282370721 shutdown_connections 2024-04-26T23:55:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.808+0000 7feafa000640 1 -- 172.21.15.46:0/3282370721 wait complete. 2024-04-26T23:55:28.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.809+0000 7feafa000640 1 Processor -- start 2024-04-26T23:55:28.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.810+0000 7feafa000640 1 -- start start 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 0x7feaf414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4160130 0x7feaf40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4072930 con 0x7feaf4160510 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feaf415d350 con 0x7feaf4160130 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feafa000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feaf4072ab0 con 0x7feaf40cda90 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33166/0 (socket says 172.21.15.46:33166) 2024-04-26T23:55:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 -- 172.21.15.46:0/391544741 learned_addr learned my addr 172.21.15.46:0/391544741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf3e00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4160130 0x7feaf40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf8c00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 0x7feaf414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 -- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 msgr2=0x7feaf414a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 0x7feaf414a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 -- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4160130 msgr2=0x7feaf40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4160130 0x7feaf40ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.811+0000 7feaf9600640 1 -- 172.21.15.46:0/391544741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf414a570 con 0x7feaf4160510 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feaf8c00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 0x7feaf414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feaf9600640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c7b0 tx=0x7feaec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec00ce60 con 0x7feaf4160510 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec004b10 con 0x7feaf4160510 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec00a850 con 0x7feaf4160510 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf41f6c70 con 0x7feaf4160510 2024-04-26T23:55:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.812+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf4159950 con 0x7feaf4160510 2024-04-26T23:55:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.814+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf40693b0 con 0x7feaf4160510 2024-04-26T23:55:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.815+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaec00a9f0 con 0x7feaf4160510 2024-04-26T23:55:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.816+0000 7feaf0c00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.816+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7feaec08bf40 con 0x7feaf4160510 2024-04-26T23:55:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.816+0000 7feaf8c00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:28.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.818+0000 7feaf8c00640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 0x7fead406edb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feae80011e0 tx=0x7feae8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:28.820+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feaec020070 con 0x7feaf4160510 2024-04-26T23:55:29.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.063+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feaf40cbb10 con 0x7feaf4160510 2024-04-26T23:55:29.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.680+0000 7feaf0c00640 1 -- 172.21.15.46:0/391544741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v148) ==== 118+0+0 (secure 0 0 0) 0x7feaf414a570 con 0x7feaf4160510 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 msgr2=0x7fead406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 0x7fead406edb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feae80011e0 tx=0x7feae8009040 comp rx=0 tx=0).stop 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 msgr2=0x7feaf40ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c7b0 tx=0x7feaec00cc70 comp rx=0 tx=0).stop 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 shutdown_connections 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaf4160510 0x7feaf40ca680 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4160130 0x7feaf40ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 --2- 172.21.15.46:0/391544741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf40cda90 0x7feaf414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.684+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 >> 172.21.15.46:0/391544741 conn(0x7feaf40b27b0 msgr2=0x7feaf4072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:29.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.685+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 shutdown_connections 2024-04-26T23:55:29.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.685+0000 7feafa000640 1 -- 172.21.15.46:0/391544741 wait complete. 2024-04-26T23:55:29.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:55:29.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:55:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.924+0000 7fcb56600640 1 Processor -- start 2024-04-26T23:55:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.924+0000 7fcb56600640 1 -- start start 2024-04-26T23:55:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.925+0000 7fcb56600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb500cdd20 0x7fcb500cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.925+0000 7fcb56600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.925+0000 7fcb56600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb500af570 con 0x7fcb5015f210 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.925+0000 7fcb56600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcb500af850 con 0x7fcb500cdd20 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.925+0000 7fcb56600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb500cb680 con 0x7fcb500ae8f0 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb500cdd20 0x7fcb500cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb500cdd20 0x7fcb500cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33180/0 (socket says 172.21.15.46:33180) 2024-04-26T23:55:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 -- 172.21.15.46:0/1993030187 learned_addr learned my addr 172.21.15.46:0/1993030187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb55c00640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 -- 172.21.15.46:0/1993030187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 msgr2=0x7fcb500aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 -- 172.21.15.46:0/1993030187 >> v1:172.21.15.142:6789/0 conn(0x7fcb5015f210 legacy=0x7fcb500cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.926+0000 7fcb4fe00640 1 -- 172.21.15.46:0/1993030187 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb500cb960 con 0x7fcb500cdd20 2024-04-26T23:55:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.927+0000 7fcb4fe00640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb500cdd20 0x7fcb500cc160 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009870 tx=0x7fcb4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2306647965439d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.928+0000 7fcb4f400640 1 -- 172.21.15.46:0/1993030187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40037070 con 0x7fcb500cdd20 2024-04-26T23:55:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.928+0000 7fcb4f400640 1 -- 172.21.15.46:0/1993030187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb40032440 con 0x7fcb500cdd20 2024-04-26T23:55:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.928+0000 7fcb4f400640 1 -- 172.21.15.46:0/1993030187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40032810 con 0x7fcb500cdd20 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.928+0000 7fcb56600640 1 -- 172.21.15.46:0/1993030187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb500cdd20 msgr2=0x7fcb500cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.928+0000 7fcb56600640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb500cdd20 0x7fcb500cc160 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009870 tx=0x7fcb4002a210 comp rx=0 tx=0).stop 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 -- 172.21.15.46:0/1993030187 shutdown_connections 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 --2- 172.21.15.46:0/1993030187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb500cdd20 0x7fcb500cc160 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 -- 172.21.15.46:0/1993030187 >> 172.21.15.46:0/1993030187 conn(0x7fcb500b27b0 msgr2=0x7fcb500b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 -- 172.21.15.46:0/1993030187 shutdown_connections 2024-04-26T23:55:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.929+0000 7fcb56600640 1 -- 172.21.15.46:0/1993030187 wait complete. 2024-04-26T23:55:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.931+0000 7fcb56600640 1 Processor -- start 2024-04-26T23:55:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.931+0000 7fcb56600640 1 -- start start 2024-04-26T23:55:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 0x7fcb500ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5015f5f0 con 0x7fcb5015f210 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500af280 con 0x7fcb500cdd20 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb56600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb50071e80 con 0x7fcb500ae8f0 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33184/0 (socket says 172.21.15.46:33184) 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 -- 172.21.15.46:0/126552054 learned_addr learned my addr 172.21.15.46:0/126552054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb4fe00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 0x7fcb500ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55200640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 -- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 msgr2=0x7fcb500c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 -- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 msgr2=0x7fcb500ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 0x7fcb500ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.932+0000 7fcb55c00640 1 -- 172.21.15.46:0/126552054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb500c6670 con 0x7fcb5015f210 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb55200640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb4fe00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 0x7fcb500ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb55c00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fcb48007d90 tx=0x7fcb4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48014420 con 0x7fcb5015f210 2024-04-26T23:55:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb48019070 con 0x7fcb5015f210 2024-04-26T23:55:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48014820 con 0x7fcb5015f210 2024-04-26T23:55:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb500b3af0 con 0x7fcb5015f210 2024-04-26T23:55:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.933+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb500caee0 con 0x7fcb5015f210 2024-04-26T23:55:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.935+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb500693b0 con 0x7fcb5015f210 2024-04-26T23:55:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.935+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb480149c0 con 0x7fcb5015f210 2024-04-26T23:55:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.937+0000 7fcb4cc00640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 0x7fcb3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.937+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fcb4808c5b0 con 0x7fcb5015f210 2024-04-26T23:55:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.937+0000 7fcb55200640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 0x7fcb3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.939+0000 7fcb55200640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 0x7fcb3406ede0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400a830 tx=0x7fcb44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:29.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:29.941+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb4805a7b0 con 0x7fcb5015f210 2024-04-26T23:55:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.195+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcb50073bd0 con 0x7fcb5015f210 2024-04-26T23:55:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.684+0000 7fcb4cc00640 1 -- 172.21.15.46:0/126552054 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v149) ==== 122+0+0 (secure 0 0 0) 0x7fcb4805a0b0 con 0x7fcb5015f210 2024-04-26T23:55:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.687+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 msgr2=0x7fcb3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.687+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 0x7fcb3406ede0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400a830 tx=0x7fcb44008040 comp rx=0 tx=0).stop 2024-04-26T23:55:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.687+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 msgr2=0x7fcb50072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.687+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fcb48007d90 tx=0x7fcb4800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.687+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 shutdown_connections 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcb3406c920 0x7fcb3406ede0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb5015f210 0x7fcb50072e00 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb500cdd20 0x7fcb500ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 --2- 172.21.15.46:0/126552054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb500ae8f0 0x7fcb500c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 >> 172.21.15.46:0/126552054 conn(0x7fcb500b27b0 msgr2=0x7fcb500cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 shutdown_connections 2024-04-26T23:55:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.688+0000 7fcb56600640 1 -- 172.21.15.46:0/126552054 wait complete. 2024-04-26T23:55:30.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:55:30.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:30.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.928+0000 7f7845800640 1 Processor -- start 2024-04-26T23:55:30.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.928+0000 7f7845800640 1 -- start start 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7845800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78401466b0 0x7f784006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7845800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7840073ad0 con 0x7f7840160910 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7845800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7840146a90 con 0x7f78400720e0 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7845800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7840073d70 con 0x7f78401466b0 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7844e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7840160910 0x7f78400cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41076/0 (socket says 172.21.15.46:41076) 2024-04-26T23:55:30.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.929+0000 7f7844e00640 1 -- 172.21.15.46:0/2394497109 learned_addr learned my addr 172.21.15.46:0/2394497109 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606494010 0 0) 0x7f7840073ad0 con 0x7f7840160910 2024-04-26T23:55:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7824003520 con 0x7f7840160910 2024-04-26T23:55:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783fe00640 1 --2- 172.21.15.46:0/2394497109 >> v2:172.21.15.142:3300/0 conn(0x7f78401466b0 0x7f784006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500075625 0 0) 0x7f7824003520 con 0x7f7840160910 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 >> v2:172.21.15.142:3300/0 conn(0x7f78401466b0 msgr2=0x7f784006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 --2- 172.21.15.46:0/2394497109 >> v2:172.21.15.142:3300/0 conn(0x7f78401466b0 0x7f784006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 >> v1:172.21.15.142:6789/0 conn(0x7f78400720e0 legacy=0x7f78400cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7840074050 con 0x7f7840160910 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f78380019f0 con 0x7f7840160910 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.930+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 >> v1:172.21.15.46:6789/0 conn(0x7f7840160910 legacy=0x7f78400cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 0x7f7824003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824004800 0x7f7824004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78240051c0 0x7f782400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7840073d70 con 0x7f7824003a00 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f782400c0c0 con 0x7f78240051c0 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f782400c2c0 con 0x7f7824004800 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 0x7f7824003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783f400640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824004800 0x7f7824004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 -- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824004800 msgr2=0x7f7824004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824004800 0x7f7824004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 -- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78240051c0 msgr2=0x7f782400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78240051c0 0x7f782400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783fe00640 1 -- 172.21.15.46:0/2394497109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7824003520 con 0x7f7824003a00 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f7844e00640 1 -- 172.21.15.46:0/2394497109 reap_dead start 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.931+0000 7f783f400640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824004800 0x7f7824004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.932+0000 7f783fe00640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 0x7f7824003e20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f7828009ac0 tx=0x7f7824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7a4c494666773e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828036070 con 0x7f7824003a00 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828031400 con 0x7f7824003a00 2024-04-26T23:55:30.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f783ea00640 1 -- 172.21.15.46:0/2394497109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828009690 con 0x7f7824003a00 2024-04-26T23:55:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 -- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 msgr2=0x7f7824003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 0x7f7824003e20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f7828009ac0 tx=0x7f7824005830 comp rx=0 tx=0).stop 2024-04-26T23:55:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 -- 172.21.15.46:0/2394497109 shutdown_connections 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 --2- 172.21.15.46:0/2394497109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824003a00 0x7f7824003e20 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 -- 172.21.15.46:0/2394497109 >> 172.21.15.46:0/2394497109 conn(0x7f78400b26d0 msgr2=0x7f78400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 -- 172.21.15.46:0/2394497109 shutdown_connections 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.933+0000 7f7845800640 1 -- 172.21.15.46:0/2394497109 wait complete. 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.935+0000 7f7845800640 1 Processor -- start 2024-04-26T23:55:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.935+0000 7f7845800640 1 -- start start 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824003a00 0x7f7840143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 0x7f78400c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7840160d60 con 0x7f7840140120 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78400cb470 con 0x7f78400b5170 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7845800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7840160ee0 con 0x7f7824003a00 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.936+0000 7f7844e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33206/0 (socket says 172.21.15.46:33206) 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 -- 172.21.15.46:0/2736333613 learned_addr learned my addr 172.21.15.46:0/2736333613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f783f400640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 0x7f78400c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f783fe00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824003a00 0x7f7840143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 -- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824003a00 msgr2=0x7f7840143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824003a00 0x7f7840143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 -- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 msgr2=0x7f78400c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 0x7f78400c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 -- 172.21.15.46:0/2736333613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78400b5630 con 0x7f7840140120 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f783f400640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 0x7f78400c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7844e00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f78400742f0 tx=0x7f783800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.937+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783800cea0 con 0x7f7840140120 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.938+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783800a640 con 0x7f7840140120 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.938+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783800aa30 con 0x7f7840140120 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.938+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78400b5910 con 0x7f7840140120 2024-04-26T23:55:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.938+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78400b5ed0 con 0x7f7840140120 2024-04-26T23:55:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.939+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78400693b0 con 0x7f7840140120 2024-04-26T23:55:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.940+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f783800abd0 con 0x7f7840140120 2024-04-26T23:55:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.941+0000 7f7837e00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 0x7f781c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:30.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.941+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7f783808b5f0 con 0x7f7840140120 2024-04-26T23:55:30.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.942+0000 7f783fe00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 0x7f781c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.943+0000 7f783fe00640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 0x7f781c06ee00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7828033000 tx=0x7f7828008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:30.945+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7838059870 con 0x7f7840140120 2024-04-26T23:55:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.189+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78400cda20 con 0x7f7840140120 2024-04-26T23:55:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.190+0000 7f7837e00640 1 -- 172.21.15.46:0/2736333613 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f7838059170 con 0x7f7840140120 2024-04-26T23:55:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:31.192 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:55:31.192 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.192+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 msgr2=0x7f781c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.192+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 0x7f781c06ee00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7828033000 tx=0x7f7828008030 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 msgr2=0x7f78401405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f78400742f0 tx=0x7f783800cb90 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 shutdown_connections 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f781c06c940 0x7f781c06ee00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7840140120 0x7f78401405b0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78400b5170 0x7f78400c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 --2- 172.21.15.46:0/2736333613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7824003a00 0x7f7840143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 >> 172.21.15.46:0/2736333613 conn(0x7f78400b26d0 msgr2=0x7f784006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 shutdown_connections 2024-04-26T23:55:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.193+0000 7f7845800640 1 -- 172.21.15.46:0/2736333613 wait complete. 2024-04-26T23:55:31.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:55:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.424+0000 7f5a05e00640 1 Processor -- start 2024-04-26T23:55:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.425+0000 7f5a05e00640 1 -- start start 2024-04-26T23:55:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.426+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f5a000d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.426+0000 7f5a05e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a000d11f0 con 0x7f5a000b4000 2024-04-26T23:55:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.426+0000 7f5a05e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a000d5f20 con 0x7f5a0015f4e0 2024-04-26T23:55:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.426+0000 7f5a05e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a000d0ce0 con 0x7f5a000d2130 2024-04-26T23:55:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.427+0000 7f5a05400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5a000b4000 0x7f5a000b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41082/0 (socket says 172.21.15.46:41082) 2024-04-26T23:55:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.427+0000 7f5a05400640 1 -- 172.21.15.46:0/211431678 learned_addr learned my addr 172.21.15.46:0/211431678 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.427+0000 7f59ffe00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f5a000d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.427+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838708791 0 0) 0x7f5a000d11f0 con 0x7f5a000b4000 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.427+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e8003520 con 0x7f5a000b4000 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 204530441 0 0) 0x7f59e8003520 con 0x7f5a000b4000 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 msgr2=0x7f5a000d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f5a000d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 >> v1:172.21.15.142:6789/0 conn(0x7f5a0015f4e0 legacy=0x7f5a000d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000d0f10 con 0x7f5a000b4000 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f59f80019f0 con 0x7f5a000b4000 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 >> v1:172.21.15.46:6789/0 conn(0x7f5a000b4000 legacy=0x7f5a000b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f59e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e8003f60 0x7f59e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e8004d40 0x7f59e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59e800ba70 con 0x7f59e8003f60 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59e800bc20 con 0x7f59e8004d40 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59e800be20 con 0x7f5a000d2130 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.428+0000 7f59ffe00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e8003f60 0x7f59e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f5a04a00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f59e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f5a05400640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e8004d40 0x7f59e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f59ffe00640 1 -- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 msgr2=0x7f59e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f59ffe00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a000d2130 0x7f59e80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f59ffe00640 1 -- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e8004d40 msgr2=0x7f59e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f59ffe00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e8004d40 0x7f59e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.429+0000 7f59ffe00640 1 -- 172.21.15.46:0/211431678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e8003520 con 0x7f59e8003f60 2024-04-26T23:55:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.430+0000 7f59ffe00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e8003f60 0x7f59e8004800 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009ac0 tx=0x7f59e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f78f631fef52f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.430+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec036070 con 0x7f59e8003f60 2024-04-26T23:55:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.430+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec031400 con 0x7f59e8003f60 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.430+0000 7f59ff400640 1 -- 172.21.15.46:0/211431678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec009690 con 0x7f59e8003f60 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 -- 172.21.15.46:0/211431678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e8003f60 msgr2=0x7f59e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 --2- 172.21.15.46:0/211431678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e8003f60 0x7f59e8004800 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009ac0 tx=0x7f59e8005830 comp rx=0 tx=0).stop 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05400640 1 -- 172.21.15.46:0/211431678 reap_dead start 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 -- 172.21.15.46:0/211431678 shutdown_connections 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 -- 172.21.15.46:0/211431678 >> 172.21.15.46:0/211431678 conn(0x7f5a000d5010 msgr2=0x7f5a000d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 -- 172.21.15.46:0/211431678 shutdown_connections 2024-04-26T23:55:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.431+0000 7f5a05e00640 1 -- 172.21.15.46:0/211431678 wait complete. 2024-04-26T23:55:31.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.433+0000 7f5a05e00640 1 Processor -- start 2024-04-26T23:55:31.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.434+0000 7f5a05e00640 1 -- start start 2024-04-26T23:55:31.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e8003f60 0x7f5a000cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 0x7f5a000b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000b1d60 con 0x7f5a0014d000 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000d2990 con 0x7f5a000cce00 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0015f8c0 con 0x7f59e8003f60 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33222/0 (socket says 172.21.15.46:33222) 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a05400640 1 -- 172.21.15.46:0/3128265378 learned_addr learned my addr 172.21.15.46:0/3128265378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.435+0000 7f5a04a00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e8003f60 0x7f5a000cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f59ffe00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 0x7f5a000b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 -- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e8003f60 msgr2=0x7f5a000cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e8003f60 0x7f5a000cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 -- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 msgr2=0x7f5a000b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 0x7f5a000b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 -- 172.21.15.46:0/3128265378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000cd980 con 0x7f5a0014d000 2024-04-26T23:55:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f59ffe00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 0x7f5a000b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05400640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f5a0008d650 tx=0x7f59f800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8007aa0 con 0x7f5a0014d000 2024-04-26T23:55:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f800ee20 con 0x7f5a0014d000 2024-04-26T23:55:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8014690 con 0x7f5a0014d000 2024-04-26T23:55:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a000cd3c0 con 0x7f5a0014d000 2024-04-26T23:55:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.436+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0015b8d0 con 0x7f5a0014d000 2024-04-26T23:55:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.438+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a000693b0 con 0x7f5a0014d000 2024-04-26T23:55:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.439+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59f8014830 con 0x7f5a0014d000 2024-04-26T23:55:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.439+0000 7f59fcc00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 0x7f59e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.439+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7f59f808bbd0 con 0x7f5a0014d000 2024-04-26T23:55:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.440+0000 7f5a04a00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 0x7f59e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.441+0000 7f5a04a00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 0x7f59e006ede0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f59f4007fd0 tx=0x7f59f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.444+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59f8059dd0 con 0x7f5a0014d000 2024-04-26T23:55:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.654+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a000c8260 con 0x7f5a0014d000 2024-04-26T23:55:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.656+0000 7f59fcc00640 1 -- 172.21.15.46:0/3128265378 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+16916 (secure 0 0 0) 0x7f59f80596d0 con 0x7f5a0014d000 2024-04-26T23:55:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":186,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:28.648251+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.659+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 msgr2=0x7f59e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.659+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 0x7f59e006ede0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f59f4007fd0 tx=0x7f59f4009040 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.659+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 msgr2=0x7f5a0014d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.659+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f5a0008d650 tx=0x7f59f800e530 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 shutdown_connections 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f59e006c920 0x7f59e006ede0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a0014d000 0x7f5a0014d480 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a000cce00 0x7f5a000b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 --2- 172.21.15.46:0/3128265378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e8003f60 0x7f5a000cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 >> 172.21.15.46:0/3128265378 conn(0x7f5a000d5010 msgr2=0x7f5a000d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 shutdown_connections 2024-04-26T23:55:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.660+0000 7f5a05e00640 1 -- 172.21.15.46:0/3128265378 wait complete. 2024-04-26T23:55:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:55:31.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:55:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.892+0000 7f6db7200640 1 Processor -- start 2024-04-26T23:55:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.893+0000 7f6db7200640 1 -- start start 2024-04-26T23:55:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.894+0000 7f6db7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db80d5a90 0x7f6db80d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.894+0000 7f6db7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6db80b4e20 0x7f6db80d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.894+0000 7f6db7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db80b65e0 con 0x7f6db8162a60 2024-04-26T23:55:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.894+0000 7f6db7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6db80d12e0 con 0x7f6db80b4e20 2024-04-26T23:55:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.894+0000 7f6db7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6db80d05f0 con 0x7f6db80d5a90 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6db80b4e20 0x7f6db80d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6db80b4e20 0x7f6db80d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58444/0 (socket says 172.21.15.46:58444) 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 -- 172.21.15.46:0/142812452 learned_addr learned my addr 172.21.15.46:0/142812452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db5400640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db80d5a90 0x7f6db80d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 -- 172.21.15.46:0/142812452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db80d5a90 msgr2=0x7f6db80d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db80d5a90 0x7f6db80d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 -- 172.21.15.46:0/142812452 >> v1:172.21.15.142:6789/0 conn(0x7f6db8162a60 legacy=0x7f6db80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.895+0000 7f6db6800640 1 -- 172.21.15.46:0/142812452 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80d1530 con 0x7f6db80b4e20 2024-04-26T23:55:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.896+0000 7f6db6800640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80b4e20 0x7f6db80d19b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009870 tx=0x7f6db002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da243d5999f8481 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.897+0000 7f6db4a00640 1 -- 172.21.15.46:0/142812452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0037070 con 0x7f6db80b4e20 2024-04-26T23:55:31.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.897+0000 7f6db4a00640 1 -- 172.21.15.46:0/142812452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db002ae10 con 0x7f6db80b4e20 2024-04-26T23:55:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.897+0000 7f6db4a00640 1 -- 172.21.15.46:0/142812452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0032660 con 0x7f6db80b4e20 2024-04-26T23:55:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.897+0000 7f6db7200640 1 -- 172.21.15.46:0/142812452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80b4e20 msgr2=0x7f6db80d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.897+0000 7f6db7200640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80b4e20 0x7f6db80d19b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009870 tx=0x7f6db002a100 comp rx=0 tx=0).stop 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 -- 172.21.15.46:0/142812452 shutdown_connections 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80b4e20 0x7f6db80d19b0 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 --2- 172.21.15.46:0/142812452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db80d5a90 0x7f6db80d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 -- 172.21.15.46:0/142812452 >> 172.21.15.46:0/142812452 conn(0x7f6db80d4fa0 msgr2=0x7f6db80d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 -- 172.21.15.46:0/142812452 shutdown_connections 2024-04-26T23:55:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.898+0000 7f6db7200640 1 -- 172.21.15.46:0/142812452 wait complete. 2024-04-26T23:55:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.899+0000 7f6db7200640 1 Processor -- start 2024-04-26T23:55:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.900+0000 7f6db7200640 1 -- start start 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 0x7f6db8152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db8162a60 0x7f6db80d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80b52d0 con 0x7f6db80d5a90 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80d1ef0 con 0x7f6db80b4e20 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6db8152950 con 0x7f6db8162a60 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58450/0 (socket says 172.21.15.46:58450) 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db5400640 1 -- 172.21.15.46:0/62053922 learned_addr learned my addr 172.21.15.46:0/62053922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db5e00640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 0x7f6db8152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.901+0000 7f6db6800640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db8162a60 0x7f6db80d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 -- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db8162a60 msgr2=0x7f6db80d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db8162a60 0x7f6db80d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 -- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 msgr2=0x7f6db8152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 0x7f6db8152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 -- 172.21.15.46:0/62053922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80ca860 con 0x7f6db80d5a90 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5e00640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 0x7f6db8152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db5400640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009fd0 tx=0x7f6da800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8014620 con 0x7f6db80d5a90 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da8019070 con 0x7f6db80d5a90 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8014a00 con 0x7f6db80d5a90 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.902+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db80cab40 con 0x7f6db80d5a90 2024-04-26T23:55:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.903+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db80cb100 con 0x7f6db80d5a90 2024-04-26T23:55:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.904+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db8069ac0 con 0x7f6db80d5a90 2024-04-26T23:55:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.905+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6da8014ba0 con 0x7f6db80d5a90 2024-04-26T23:55:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.906+0000 7f6dae000640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 0x7f6d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.906+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7f6da808c590 con 0x7f6db80d5a90 2024-04-26T23:55:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.907+0000 7f6db5e00640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 0x7f6d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.908+0000 7f6db5e00640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 0x7f6d9406edb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006fd0 tx=0x7f6da0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:31.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:31.911+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6da805a820 con 0x7f6db80d5a90 2024-04-26T23:55:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.114+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6db8161590 con 0x7f6d9406c8f0 2024-04-26T23:55:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.133+0000 7f6dae000640 1 -- 172.21.15.46:0/62053922 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240304 (secure 0 0 0) 0x7f6db8161590 con 0x7f6d9406c8f0 2024-04-26T23:55:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:32.139 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:55:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 msgr2=0x7f6d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 0x7f6d9406edb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006fd0 tx=0x7f6da0008040 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 msgr2=0x7f6db814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009fd0 tx=0x7f6da800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 shutdown_connections 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d9406c8f0 0x7f6d9406edb0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6db8162a60 0x7f6db80d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db80d5a90 0x7f6db814f020 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 --2- 172.21.15.46:0/62053922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80b4e20 0x7f6db8152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 >> 172.21.15.46:0/62053922 conn(0x7f6db80d4fa0 msgr2=0x7f6db80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 shutdown_connections 2024-04-26T23:55:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.138+0000 7f6db7200640 1 -- 172.21.15.46:0/62053922 wait complete. 2024-04-26T23:55:32.159 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":734,"stamp":"2024-04-26T23:55:30.303896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":437976,"kb_used_data":19172,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754536744,"statfs":{"total":773094113280,"available":772645625856,"internally_reserved":0,"allocated":19632128,"data_stored":9872805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":428723761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.676674"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.640322+0000","last_change":"2024-04-26T23:55:25.640322+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.640322+0000","last_undegraded":"2024-04-26T23:55:25.640322+0000","last_fullsized":"2024-04-26T23:55:25.640322+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.640518+0000","last_change":"2024-04-26T23:55:25.640518+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.640518+0000","last_undegraded":"2024-04-26T23:55:25.640518+0000","last_fullsized":"2024-04-26T23:55:25.640518+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.631660+0000","last_change":"2024-04-26T23:55:25.631660+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.631660+0000","last_undegraded":"2024-04-26T23:55:25.631660+0000","last_fullsized":"2024-04-26T23:55:25.631660+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.635009+0000","last_change":"2024-04-26T23:55:25.635009+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.635009+0000","last_undegraded":"2024-04-26T23:55:25.635009+0000","last_fullsized":"2024-04-26T23:55:25.635009+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.644916+0000","last_change":"2024-04-26T23:55:25.644916+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.644916+0000","last_undegraded":"2024-04-26T23:55:25.644916+0000","last_fullsized":"2024-04-26T23:55:25.644916+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.643689+0000","last_change":"2024-04-26T23:55:25.643689+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.643689+0000","last_undegraded":"2024-04-26T23:55:25.643689+0000","last_fullsized":"2024-04-26T23:55:25.643689+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.635658+0000","last_change":"2024-04-26T23:55:25.635658+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.635658+0000","last_undegraded":"2024-04-26T23:55:25.635658+0000","last_fullsized":"2024-04-26T23:55:25.635658+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.637059+0000","last_change":"2024-04-26T23:55:25.637059+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.637059+0000","last_undegraded":"2024-04-26T23:55:25.637059+0000","last_fullsized":"2024-04-26T23:55:25.637059+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.645698+0000","last_change":"2024-04-26T23:55:25.645698+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.645698+0000","last_undegraded":"2024-04-26T23:55:25.645698+0000","last_fullsized":"2024-04-26T23:55:25.645698+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.632504+0000","last_change":"2024-04-26T23:55:25.632504+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.632504+0000","last_undegraded":"2024-04-26T23:55:25.632504+0000","last_fullsized":"2024-04-26T23:55:25.632504+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.641644+0000","last_change":"2024-04-26T23:55:25.641644+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.641644+0000","last_undegraded":"2024-04-26T23:55:25.641644+0000","last_fullsized":"2024-04-26T23:55:25.641644+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.630856+0000","last_change":"2024-04-26T23:55:25.630856+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.630856+0000","last_undegraded":"2024-04-26T23:55:25.630856+0000","last_fullsized":"2024-04-26T23:55:25.630856+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.636178+0000","last_change":"2024-04-26T23:55:25.636178+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.636178+0000","last_undegraded":"2024-04-26T23:55:25.636178+0000","last_fullsized":"2024-04-26T23:55:25.636178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.633459+0000","last_change":"2024-04-26T23:55:25.633459+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.633459+0000","last_undegraded":"2024-04-26T23:55:25.633459+0000","last_fullsized":"2024-04-26T23:55:25.633459+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.636416+0000","last_change":"2024-04-26T23:55:25.636416+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.636416+0000","last_undegraded":"2024-04-26T23:55:25.636416+0000","last_fullsized":"2024-04-26T23:55:25.636416+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.638460+0000","last_change":"2024-04-26T23:55:25.638460+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.638460+0000","last_undegraded":"2024-04-26T23:55:25.638460+0000","last_fullsized":"2024-04-26T23:55:25.638460+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.639417+0000","last_change":"2024-04-26T23:55:25.639417+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.639417+0000","last_undegraded":"2024-04-26T23:55:25.639417+0000","last_fullsized":"2024-04-26T23:55:25.639417+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.630856+0000","last_change":"2024-04-26T23:55:25.630856+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.630856+0000","last_undegraded":"2024-04-26T23:55:25.630856+0000","last_fullsized":"2024-04-26T23:55:25.630856+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.637111+0000","last_change":"2024-04-26T23:55:25.637111+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.637111+0000","last_undegraded":"2024-04-26T23:55:25.637111+0000","last_fullsized":"2024-04-26T23:55:25.637111+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.636209+0000","last_change":"2024-04-26T23:55:25.636209+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.636209+0000","last_undegraded":"2024-04-26T23:55:25.636209+0000","last_fullsized":"2024-04-26T23:55:25.636209+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.631127+0000","last_change":"2024-04-26T23:55:25.631127+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.631127+0000","last_undegraded":"2024-04-26T23:55:25.631127+0000","last_fullsized":"2024-04-26T23:55:25.631127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.633020+0000","last_change":"2024-04-26T23:55:25.633020+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.633020+0000","last_undegraded":"2024-04-26T23:55:25.633020+0000","last_fullsized":"2024-04-26T23:55:25.633020+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.636967+0000","last_change":"2024-04-26T23:55:25.636967+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.636967+0000","last_undegraded":"2024-04-26T23:55:25.636967+0000","last_fullsized":"2024-04-26T23:55:25.636967+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.632881+0000","last_change":"2024-04-26T23:55:25.632881+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.632881+0000","last_undegraded":"2024-04-26T23:55:25.632881+0000","last_fullsized":"2024-04-26T23:55:25.632881+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.642680+0000","last_change":"2024-04-26T23:55:25.642680+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.642680+0000","last_undegraded":"2024-04-26T23:55:25.642680+0000","last_fullsized":"2024-04-26T23:55:25.642680+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.633317+0000","last_change":"2024-04-26T23:55:25.633317+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.633317+0000","last_undegraded":"2024-04-26T23:55:25.633317+0000","last_fullsized":"2024-04-26T23:55:25.633317+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.639313+0000","last_change":"2024-04-26T23:55:25.639313+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.639313+0000","last_undegraded":"2024-04-26T23:55:25.639313+0000","last_fullsized":"2024-04-26T23:55:25.639313+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.640475+0000","last_change":"2024-04-26T23:55:25.640475+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.640475+0000","last_undegraded":"2024-04-26T23:55:25.640475+0000","last_fullsized":"2024-04-26T23:55:25.640475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.632330+0000","last_change":"2024-04-26T23:55:25.632330+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.632330+0000","last_undegraded":"2024-04-26T23:55:25.632330+0000","last_fullsized":"2024-04-26T23:55:25.632330+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.636926+0000","last_change":"2024-04-26T23:55:25.636926+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.636926+0000","last_undegraded":"2024-04-26T23:55:25.636926+0000","last_fullsized":"2024-04-26T23:55:25.636926+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.635782+0000","last_change":"2024-04-26T23:55:25.635782+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.635782+0000","last_undegraded":"2024-04-26T23:55:25.635782+0000","last_fullsized":"2024-04-26T23:55:25.635782+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:55:26.623878+0000","last_change":"2024-04-26T23:55:26.623878+0000","last_active":"2024-04-26T23:55:26.623878+0000","last_peered":"2024-04-26T23:55:26.623878+0000","last_clean":"2024-04-26T23:55:26.623878+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:26.623878+0000","last_undegraded":"2024-04-26T23:55:26.623878+0000","last_fullsized":"2024-04-26T23:55:26.623878+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2024-04-26T23:55:25.634905+0000","last_change":"2024-04-26T23:55:25.634905+0000","last_active":"2024-04-26T23:55:25.606992+0000","last_peered":"2024-04-26T23:55:25.606992+0000","last_clean":"2024-04-26T23:55:25.606992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:55:25.634905+0000","last_undegraded":"2024-04-26T23:55:25.634905+0000","last_fullsized":"2024-04-26T23:55:25.634905+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":472,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-26T23:55:25.648298+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:55:25.648298+0000","last_peered":"2024-04-26T23:55:25.648298+0000","last_clean":"2024-04-26T23:55:25.648298+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:55:25.648298+0000","last_undegraded":"2024-04-26T23:55:25.648298+0000","last_fullsized":"2024-04-26T23:55:25.648298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575071,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52064,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319776,"statfs":{"total":96636764160,"available":96583450624,"internally_reserved":0,"allocated":2904064,"data_stored":1679205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55640,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316200,"statfs":{"total":96636764160,"available":96579788800,"internally_reserved":0,"allocated":2297856,"data_stored":1082628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1082628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575071,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2310144,"data_stored":1088837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2310144,"data_stored":1088837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2310144,"data_stored":1088837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47312,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324528,"statfs":{"total":96636764160,"available":96588316672,"internally_reserved":0,"allocated":2297856,"data_stored":1082628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575072,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":2904064,"data_stored":1679205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:55:32.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:55:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.418+0000 7fd625600640 1 Processor -- start 2024-04-26T23:55:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.418+0000 7fd625600640 1 -- start start 2024-04-26T23:55:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.420+0000 7fd625600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6201466b0 0x7fd62006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.420+0000 7fd625600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.420+0000 7fd625600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd620073ad0 con 0x7fd620160910 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.420+0000 7fd625600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd620146a90 con 0x7fd6201466b0 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.420+0000 7fd625600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd620073d70 con 0x7fd6200720e0 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.421+0000 7fd61f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.421+0000 7fd61f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:46408/0 (socket says 172.21.15.46:46408) 2024-04-26T23:55:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.421+0000 7fd61f400640 1 -- 172.21.15.46:0/694984489 learned_addr learned my addr 172.21.15.46:0/694984489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.421+0000 7fd61fe00640 1 --2- 172.21.15.46:0/694984489 >> v2:172.21.15.142:3300/0 conn(0x7fd6201466b0 0x7fd62006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.421+0000 7fd61fe00640 1 -- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 msgr2=0x7fd6200cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.422+0000 7fd61fe00640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.422+0000 7fd61fe00640 1 -- 172.21.15.46:0/694984489 >> v1:172.21.15.142:6789/0 conn(0x7fd620160910 legacy=0x7fd6200cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.422+0000 7fd61fe00640 1 -- 172.21.15.46:0/694984489 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd620074050 con 0x7fd6201466b0 2024-04-26T23:55:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.422+0000 7fd61f400640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.423+0000 7fd61fe00640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6201466b0 0x7fd62006de80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd6100097a0 tx=0x7fd61002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3bd33b4462a1bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.423+0000 7fd61ea00640 1 -- 172.21.15.46:0/694984489 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610037070 con 0x7fd6201466b0 2024-04-26T23:55:32.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.423+0000 7fd61ea00640 1 -- 172.21.15.46:0/694984489 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd610032440 con 0x7fd6201466b0 2024-04-26T23:55:32.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd61ea00640 1 -- 172.21.15.46:0/694984489 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610032810 con 0x7fd6201466b0 2024-04-26T23:55:32.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 -- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6201466b0 msgr2=0x7fd62006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6201466b0 0x7fd62006de80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd6100097a0 tx=0x7fd61002a180 comp rx=0 tx=0).stop 2024-04-26T23:55:32.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 -- 172.21.15.46:0/694984489 shutdown_connections 2024-04-26T23:55:32.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6200720e0 0x7fd6200cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 --2- 172.21.15.46:0/694984489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6201466b0 0x7fd62006de80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 -- 172.21.15.46:0/694984489 >> 172.21.15.46:0/694984489 conn(0x7fd6200b26d0 msgr2=0x7fd6200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 -- 172.21.15.46:0/694984489 shutdown_connections 2024-04-26T23:55:32.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.424+0000 7fd625600640 1 -- 172.21.15.46:0/694984489 wait complete. 2024-04-26T23:55:32.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.427+0000 7fd625600640 1 Processor -- start 2024-04-26T23:55:32.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.427+0000 7fd625600640 1 -- start start 2024-04-26T23:55:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6201466b0 0x7fd6201e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 0x7fd6200d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006e3c0 con 0x7fd6200720e0 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200cb470 con 0x7fd620160910 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd625600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200cdda0 con 0x7fd6201466b0 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.428+0000 7fd61fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58462/0 (socket says 172.21.15.46:58462) 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 -- 172.21.15.46:0/3403765709 learned_addr learned my addr 172.21.15.46:0/3403765709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61f400640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6201466b0 0x7fd6201e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd624c00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 0x7fd6200d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 -- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6201466b0 msgr2=0x7fd6201e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6201466b0 0x7fd6201e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 -- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 msgr2=0x7fd6200d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 0x7fd6200d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd61fe00640 1 -- 172.21.15.46:0/3403765709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6201e70c0 con 0x7fd6200720e0 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.429+0000 7fd624c00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 0x7fd6200d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd61fe00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fd6100098c0 tx=0x7fd6100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610042020 con 0x7fd6200720e0 2024-04-26T23:55:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61003d730 con 0x7fd6200720e0 2024-04-26T23:55:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610037040 con 0x7fd6200720e0 2024-04-26T23:55:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6201e73a0 con 0x7fd6200720e0 2024-04-26T23:55:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.430+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62013f760 con 0x7fd6200720e0 2024-04-26T23:55:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.431+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6200693b0 con 0x7fd6200720e0 2024-04-26T23:55:32.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.432+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd610045020 con 0x7fd6200720e0 2024-04-26T23:55:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.433+0000 7fd617e00640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.434+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fd6100ac860 con 0x7fd6200720e0 2024-04-26T23:55:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.434+0000 7fd61f400640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.435+0000 7fd61f400640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 0x7fd60006edb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd620072bf0 tx=0x7fd60c009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.438+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd61004c030 con 0x7fd6200720e0 2024-04-26T23:55:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.637+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6200c7180 con 0x7fd60006c8f0 2024-04-26T23:55:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.655+0000 7fd617e00640 1 -- 172.21.15.46:0/3403765709 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252492 (secure 0 0 0) 0x7fd6200c7180 con 0x7fd60006c8f0 2024-04-26T23:55:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:55:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 msgr2=0x7fd60006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 0x7fd60006edb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd620072bf0 tx=0x7fd60c009240 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 msgr2=0x7fd62013e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fd6100098c0 tx=0x7fd6100046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 shutdown_connections 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd620160910 0x7fd6200d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6201466b0 0x7fd6201e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 --2- 172.21.15.46:0/3403765709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6200720e0 0x7fd62013e430 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 >> 172.21.15.46:0/3403765709 conn(0x7fd6200b26d0 msgr2=0x7fd6200ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.661+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 shutdown_connections 2024-04-26T23:55:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.662+0000 7fd625600640 1 -- 172.21.15.46:0/3403765709 wait complete. 2024-04-26T23:55:32.683 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":735,"stamp":"2024-04-26T23:55:32.304489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":438368,"kb_used_data":19560,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754536352,"statfs":{"total":773094113280,"available":772645224448,"internally_reserved":0,"allocated":20029440,"data_stored":10060460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":428723761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002012"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381682+0000","last_change":"2024-04-26T23:55:27.700749+0000","last_active":"2024-04-26T23:55:29.381682+0000","last_peered":"2024-04-26T23:55:29.381682+0000","last_clean":"2024-04-26T23:55:29.381682+0000","last_became_active":"2024-04-26T23:55:27.700115+0000","last_became_peered":"2024-04-26T23:55:27.700115+0000","last_unstale":"2024-04-26T23:55:29.381682+0000","last_undegraded":"2024-04-26T23:55:29.381682+0000","last_fullsized":"2024-04-26T23:55:29.381682+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:41.555369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383910+0000","last_change":"2024-04-26T23:55:27.685394+0000","last_active":"2024-04-26T23:55:29.383910+0000","last_peered":"2024-04-26T23:55:29.383910+0000","last_clean":"2024-04-26T23:55:29.383910+0000","last_became_active":"2024-04-26T23:55:27.685035+0000","last_became_peered":"2024-04-26T23:55:27.685035+0000","last_unstale":"2024-04-26T23:55:29.383910+0000","last_undegraded":"2024-04-26T23:55:29.383910+0000","last_fullsized":"2024-04-26T23:55:29.383910+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:15.043366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657738+0000","last_change":"2024-04-26T23:55:27.683569+0000","last_active":"2024-04-26T23:55:28.657738+0000","last_peered":"2024-04-26T23:55:28.657738+0000","last_clean":"2024-04-26T23:55:28.657738+0000","last_became_active":"2024-04-26T23:55:27.683248+0000","last_became_peered":"2024-04-26T23:55:27.683248+0000","last_unstale":"2024-04-26T23:55:28.657738+0000","last_undegraded":"2024-04-26T23:55:28.657738+0000","last_fullsized":"2024-04-26T23:55:28.657738+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:41.555369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659469+0000","last_change":"2024-04-26T23:55:27.685374+0000","last_active":"2024-04-26T23:55:28.659469+0000","last_peered":"2024-04-26T23:55:28.659469+0000","last_clean":"2024-04-26T23:55:28.659469+0000","last_became_active":"2024-04-26T23:55:27.681938+0000","last_became_peered":"2024-04-26T23:55:27.681938+0000","last_unstale":"2024-04-26T23:55:28.659469+0000","last_undegraded":"2024-04-26T23:55:28.659469+0000","last_fullsized":"2024-04-26T23:55:28.659469+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:48.959458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381901+0000","last_change":"2024-04-26T23:55:27.692890+0000","last_active":"2024-04-26T23:55:29.381901+0000","last_peered":"2024-04-26T23:55:29.381901+0000","last_clean":"2024-04-26T23:55:29.381901+0000","last_became_active":"2024-04-26T23:55:27.692627+0000","last_became_peered":"2024-04-26T23:55:27.692627+0000","last_unstale":"2024-04-26T23:55:29.381901+0000","last_undegraded":"2024-04-26T23:55:29.381901+0000","last_fullsized":"2024-04-26T23:55:29.381901+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:45.762105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.658806+0000","last_change":"2024-04-26T23:55:27.698766+0000","last_active":"2024-04-26T23:55:28.658806+0000","last_peered":"2024-04-26T23:55:28.658806+0000","last_clean":"2024-04-26T23:55:28.658806+0000","last_became_active":"2024-04-26T23:55:27.698453+0000","last_became_peered":"2024-04-26T23:55:27.698453+0000","last_unstale":"2024-04-26T23:55:28.658806+0000","last_undegraded":"2024-04-26T23:55:28.658806+0000","last_fullsized":"2024-04-26T23:55:28.658806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:42.475439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382116+0000","last_change":"2024-04-26T23:55:27.687046+0000","last_active":"2024-04-26T23:55:29.382116+0000","last_peered":"2024-04-26T23:55:29.382116+0000","last_clean":"2024-04-26T23:55:29.382116+0000","last_became_active":"2024-04-26T23:55:27.686766+0000","last_became_peered":"2024-04-26T23:55:27.686766+0000","last_unstale":"2024-04-26T23:55:29.382116+0000","last_undegraded":"2024-04-26T23:55:29.382116+0000","last_fullsized":"2024-04-26T23:55:29.382116+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:55.018313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.685674+0000","last_change":"2024-04-26T23:55:27.696062+0000","last_active":"2024-04-26T23:55:28.685674+0000","last_peered":"2024-04-26T23:55:28.685674+0000","last_clean":"2024-04-26T23:55:28.685674+0000","last_became_active":"2024-04-26T23:55:27.695789+0000","last_became_peered":"2024-04-26T23:55:27.695789+0000","last_unstale":"2024-04-26T23:55:28.685674+0000","last_undegraded":"2024-04-26T23:55:28.685674+0000","last_fullsized":"2024-04-26T23:55:28.685674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:09.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654192+0000","last_change":"2024-04-26T23:55:27.689176+0000","last_active":"2024-04-26T23:55:28.654192+0000","last_peered":"2024-04-26T23:55:28.654192+0000","last_clean":"2024-04-26T23:55:28.654192+0000","last_became_active":"2024-04-26T23:55:27.688857+0000","last_became_peered":"2024-04-26T23:55:27.688857+0000","last_unstale":"2024-04-26T23:55:28.654192+0000","last_undegraded":"2024-04-26T23:55:28.654192+0000","last_fullsized":"2024-04-26T23:55:28.654192+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:10:45.806894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.687180+0000","last_change":"2024-04-26T23:55:27.689301+0000","last_active":"2024-04-26T23:55:28.687180+0000","last_peered":"2024-04-26T23:55:28.687180+0000","last_clean":"2024-04-26T23:55:28.687180+0000","last_became_active":"2024-04-26T23:55:27.689014+0000","last_became_peered":"2024-04-26T23:55:27.689014+0000","last_unstale":"2024-04-26T23:55:28.687180+0000","last_undegraded":"2024-04-26T23:55:28.687180+0000","last_fullsized":"2024-04-26T23:55:28.687180+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:48:10.666556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657763+0000","last_change":"2024-04-26T23:55:27.687534+0000","last_active":"2024-04-26T23:55:28.657763+0000","last_peered":"2024-04-26T23:55:28.657763+0000","last_clean":"2024-04-26T23:55:28.657763+0000","last_became_active":"2024-04-26T23:55:27.687149+0000","last_became_peered":"2024-04-26T23:55:27.687149+0000","last_unstale":"2024-04-26T23:55:28.657763+0000","last_undegraded":"2024-04-26T23:55:28.657763+0000","last_fullsized":"2024-04-26T23:55:28.657763+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:10.878466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382535+0000","last_change":"2024-04-26T23:55:27.691454+0000","last_active":"2024-04-26T23:55:29.382535+0000","last_peered":"2024-04-26T23:55:29.382535+0000","last_clean":"2024-04-26T23:55:29.382535+0000","last_became_active":"2024-04-26T23:55:27.691181+0000","last_became_peered":"2024-04-26T23:55:27.691181+0000","last_unstale":"2024-04-26T23:55:29.382535+0000","last_undegraded":"2024-04-26T23:55:29.382535+0000","last_fullsized":"2024-04-26T23:55:29.382535+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:08.358703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659250+0000","last_change":"2024-04-26T23:55:27.692408+0000","last_active":"2024-04-26T23:55:28.659250+0000","last_peered":"2024-04-26T23:55:28.659250+0000","last_clean":"2024-04-26T23:55:28.659250+0000","last_became_active":"2024-04-26T23:55:27.691583+0000","last_became_peered":"2024-04-26T23:55:27.691583+0000","last_unstale":"2024-04-26T23:55:28.659250+0000","last_undegraded":"2024-04-26T23:55:28.659250+0000","last_fullsized":"2024-04-26T23:55:28.659250+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:41.555369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.685818+0000","last_change":"2024-04-26T23:55:27.688078+0000","last_active":"2024-04-26T23:55:28.685818+0000","last_peered":"2024-04-26T23:55:28.685818+0000","last_clean":"2024-04-26T23:55:28.685818+0000","last_became_active":"2024-04-26T23:55:27.686999+0000","last_became_peered":"2024-04-26T23:55:27.686999+0000","last_unstale":"2024-04-26T23:55:28.685818+0000","last_undegraded":"2024-04-26T23:55:28.685818+0000","last_fullsized":"2024-04-26T23:55:28.685818+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:45.762105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381021+0000","last_change":"2024-04-26T23:55:27.682493+0000","last_active":"2024-04-26T23:55:29.381021+0000","last_peered":"2024-04-26T23:55:29.381021+0000","last_clean":"2024-04-26T23:55:29.381021+0000","last_became_active":"2024-04-26T23:55:27.682199+0000","last_became_peered":"2024-04-26T23:55:27.682199+0000","last_unstale":"2024-04-26T23:55:29.381021+0000","last_undegraded":"2024-04-26T23:55:29.381021+0000","last_fullsized":"2024-04-26T23:55:29.381021+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:15.489757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657321+0000","last_change":"2024-04-26T23:55:27.686011+0000","last_active":"2024-04-26T23:55:28.657321+0000","last_peered":"2024-04-26T23:55:28.657321+0000","last_clean":"2024-04-26T23:55:28.657321+0000","last_became_active":"2024-04-26T23:55:27.685341+0000","last_became_peered":"2024-04-26T23:55:27.685341+0000","last_unstale":"2024-04-26T23:55:28.657321+0000","last_undegraded":"2024-04-26T23:55:28.657321+0000","last_fullsized":"2024-04-26T23:55:28.657321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:34.172112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654352+0000","last_change":"2024-04-26T23:55:27.684622+0000","last_active":"2024-04-26T23:55:28.654352+0000","last_peered":"2024-04-26T23:55:28.654352+0000","last_clean":"2024-04-26T23:55:28.654352+0000","last_became_active":"2024-04-26T23:55:27.684110+0000","last_became_peered":"2024-04-26T23:55:27.684110+0000","last_unstale":"2024-04-26T23:55:28.654352+0000","last_undegraded":"2024-04-26T23:55:28.654352+0000","last_fullsized":"2024-04-26T23:55:28.654352+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:02.087445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.653768+0000","last_change":"2024-04-26T23:55:27.670707+0000","last_active":"2024-04-26T23:55:28.653768+0000","last_peered":"2024-04-26T23:55:28.653768+0000","last_clean":"2024-04-26T23:55:28.653768+0000","last_became_active":"2024-04-26T23:55:27.670193+0000","last_became_peered":"2024-04-26T23:55:27.670193+0000","last_unstale":"2024-04-26T23:55:28.653768+0000","last_undegraded":"2024-04-26T23:55:28.653768+0000","last_fullsized":"2024-04-26T23:55:28.653768+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:36.390265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.687247+0000","last_change":"2024-04-26T23:55:27.682898+0000","last_active":"2024-04-26T23:55:28.687247+0000","last_peered":"2024-04-26T23:55:28.687247+0000","last_clean":"2024-04-26T23:55:28.687247+0000","last_became_active":"2024-04-26T23:55:27.682479+0000","last_became_peered":"2024-04-26T23:55:27.682479+0000","last_unstale":"2024-04-26T23:55:28.687247+0000","last_undegraded":"2024-04-26T23:55:28.687247+0000","last_fullsized":"2024-04-26T23:55:28.687247+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:00.963779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686761+0000","last_change":"2024-04-26T23:55:27.687360+0000","last_active":"2024-04-26T23:55:28.686761+0000","last_peered":"2024-04-26T23:55:28.686761+0000","last_clean":"2024-04-26T23:55:28.686761+0000","last_became_active":"2024-04-26T23:55:27.686396+0000","last_became_peered":"2024-04-26T23:55:27.686396+0000","last_unstale":"2024-04-26T23:55:28.686761+0000","last_undegraded":"2024-04-26T23:55:28.686761+0000","last_fullsized":"2024-04-26T23:55:28.686761+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:26.268099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686388+0000","last_change":"2024-04-26T23:55:27.690058+0000","last_active":"2024-04-26T23:55:28.686388+0000","last_peered":"2024-04-26T23:55:28.686388+0000","last_clean":"2024-04-26T23:55:28.686388+0000","last_became_active":"2024-04-26T23:55:27.689203+0000","last_became_peered":"2024-04-26T23:55:27.689203+0000","last_unstale":"2024-04-26T23:55:28.686388+0000","last_undegraded":"2024-04-26T23:55:28.686388+0000","last_fullsized":"2024-04-26T23:55:28.686388+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:24.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686039+0000","last_change":"2024-04-26T23:55:27.686984+0000","last_active":"2024-04-26T23:55:28.686039+0000","last_peered":"2024-04-26T23:55:28.686039+0000","last_clean":"2024-04-26T23:55:28.686039+0000","last_became_active":"2024-04-26T23:55:27.686676+0000","last_became_peered":"2024-04-26T23:55:27.686676+0000","last_unstale":"2024-04-26T23:55:28.686039+0000","last_undegraded":"2024-04-26T23:55:28.686039+0000","last_fullsized":"2024-04-26T23:55:28.686039+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:55.095407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381999+0000","last_change":"2024-04-26T23:55:27.690707+0000","last_active":"2024-04-26T23:55:29.381999+0000","last_peered":"2024-04-26T23:55:29.381999+0000","last_clean":"2024-04-26T23:55:29.381999+0000","last_became_active":"2024-04-26T23:55:27.690222+0000","last_became_peered":"2024-04-26T23:55:27.690222+0000","last_unstale":"2024-04-26T23:55:29.381999+0000","last_undegraded":"2024-04-26T23:55:29.381999+0000","last_fullsized":"2024-04-26T23:55:29.381999+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:32.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381972+0000","last_change":"2024-04-26T23:55:27.686184+0000","last_active":"2024-04-26T23:55:29.381972+0000","last_peered":"2024-04-26T23:55:29.381972+0000","last_clean":"2024-04-26T23:55:29.381972+0000","last_became_active":"2024-04-26T23:55:27.685879+0000","last_became_peered":"2024-04-26T23:55:27.685879+0000","last_unstale":"2024-04-26T23:55:29.381972+0000","last_undegraded":"2024-04-26T23:55:29.381972+0000","last_fullsized":"2024-04-26T23:55:29.381972+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:55.095407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654460+0000","last_change":"2024-04-26T23:55:27.673451+0000","last_active":"2024-04-26T23:55:28.654460+0000","last_peered":"2024-04-26T23:55:28.654460+0000","last_clean":"2024-04-26T23:55:28.654460+0000","last_became_active":"2024-04-26T23:55:27.673135+0000","last_became_peered":"2024-04-26T23:55:27.673135+0000","last_unstale":"2024-04-26T23:55:28.654460+0000","last_undegraded":"2024-04-26T23:55:28.654460+0000","last_fullsized":"2024-04-26T23:55:28.654460+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:36.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382222+0000","last_change":"2024-04-26T23:55:27.684545+0000","last_active":"2024-04-26T23:55:29.382222+0000","last_peered":"2024-04-26T23:55:29.382222+0000","last_clean":"2024-04-26T23:55:29.382222+0000","last_became_active":"2024-04-26T23:55:27.683788+0000","last_became_peered":"2024-04-26T23:55:27.683788+0000","last_unstale":"2024-04-26T23:55:29.382222+0000","last_undegraded":"2024-04-26T23:55:29.382222+0000","last_fullsized":"2024-04-26T23:55:29.382222+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:10.878466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.658006+0000","last_change":"2024-04-26T23:55:27.686955+0000","last_active":"2024-04-26T23:55:28.658006+0000","last_peered":"2024-04-26T23:55:28.658006+0000","last_clean":"2024-04-26T23:55:28.658006+0000","last_became_active":"2024-04-26T23:55:27.686634+0000","last_became_peered":"2024-04-26T23:55:27.686634+0000","last_unstale":"2024-04-26T23:55:28.658006+0000","last_undegraded":"2024-04-26T23:55:28.658006+0000","last_fullsized":"2024-04-26T23:55:28.658006+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:24.469652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382852+0000","last_change":"2024-04-26T23:55:27.701781+0000","last_active":"2024-04-26T23:55:29.382852+0000","last_peered":"2024-04-26T23:55:29.382852+0000","last_clean":"2024-04-26T23:55:29.382852+0000","last_became_active":"2024-04-26T23:55:27.700135+0000","last_became_peered":"2024-04-26T23:55:27.700135+0000","last_unstale":"2024-04-26T23:55:29.382852+0000","last_undegraded":"2024-04-26T23:55:29.382852+0000","last_fullsized":"2024-04-26T23:55:29.382852+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:26.268099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654575+0000","last_change":"2024-04-26T23:55:27.664728+0000","last_active":"2024-04-26T23:55:28.654575+0000","last_peered":"2024-04-26T23:55:28.654575+0000","last_clean":"2024-04-26T23:55:28.654575+0000","last_became_active":"2024-04-26T23:55:27.663978+0000","last_became_peered":"2024-04-26T23:55:27.663978+0000","last_unstale":"2024-04-26T23:55:28.654575+0000","last_undegraded":"2024-04-26T23:55:28.654575+0000","last_fullsized":"2024-04-26T23:55:28.654575+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:30.328095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381419+0000","last_change":"2024-04-26T23:55:27.695165+0000","last_active":"2024-04-26T23:55:29.381419+0000","last_peered":"2024-04-26T23:55:29.381419+0000","last_clean":"2024-04-26T23:55:29.381419+0000","last_became_active":"2024-04-26T23:55:27.694162+0000","last_became_peered":"2024-04-26T23:55:27.694162+0000","last_unstale":"2024-04-26T23:55:29.381419+0000","last_undegraded":"2024-04-26T23:55:29.381419+0000","last_fullsized":"2024-04-26T23:55:29.381419+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:16.065009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659449+0000","last_change":"2024-04-26T23:55:27.692168+0000","last_active":"2024-04-26T23:55:28.659449+0000","last_peered":"2024-04-26T23:55:28.659449+0000","last_clean":"2024-04-26T23:55:28.659449+0000","last_became_active":"2024-04-26T23:55:27.691318+0000","last_became_peered":"2024-04-26T23:55:27.691318+0000","last_unstale":"2024-04-26T23:55:28.659449+0000","last_undegraded":"2024-04-26T23:55:28.659449+0000","last_fullsized":"2024-04-26T23:55:28.659449+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:10.878466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.658668+0000","last_change":"2024-04-26T23:55:27.690180+0000","last_active":"2024-04-26T23:55:28.658668+0000","last_peered":"2024-04-26T23:55:28.658668+0000","last_clean":"2024-04-26T23:55:28.658668+0000","last_became_active":"2024-04-26T23:55:27.689794+0000","last_became_peered":"2024-04-26T23:55:27.689794+0000","last_unstale":"2024-04-26T23:55:28.658668+0000","last_undegraded":"2024-04-26T23:55:28.658668+0000","last_fullsized":"2024-04-26T23:55:28.658668+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:09.658367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381900+0000","last_change":"2024-04-26T23:55:27.690576+0000","last_active":"2024-04-26T23:55:29.381900+0000","last_peered":"2024-04-26T23:55:29.381900+0000","last_clean":"2024-04-26T23:55:29.381900+0000","last_became_active":"2024-04-26T23:55:27.690240+0000","last_became_peered":"2024-04-26T23:55:27.690240+0000","last_unstale":"2024-04-26T23:55:29.381900+0000","last_undegraded":"2024-04-26T23:55:29.381900+0000","last_fullsized":"2024-04-26T23:55:29.381900+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:39.465894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"185'4","reported_seq":18,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381315+0000","last_change":"2024-04-26T23:55:26.672837+0000","last_active":"2024-04-26T23:55:29.381315+0000","last_peered":"2024-04-26T23:55:29.381315+0000","last_clean":"2024-04-26T23:55:29.381315+0000","last_became_active":"2024-04-26T23:55:26.672503+0000","last_became_peered":"2024-04-26T23:55:26.672503+0000","last_unstale":"2024-04-26T23:55:29.381315+0000","last_undegraded":"2024-04-26T23:55:29.381315+0000","last_fullsized":"2024-04-26T23:55:29.381315+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:00.460129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.380403+0000","last_change":"2024-04-26T23:55:27.681492+0000","last_active":"2024-04-26T23:55:29.380403+0000","last_peered":"2024-04-26T23:55:29.380403+0000","last_clean":"2024-04-26T23:55:29.380403+0000","last_became_active":"2024-04-26T23:55:27.681200+0000","last_became_peered":"2024-04-26T23:55:27.681200+0000","last_unstale":"2024-04-26T23:55:29.380403+0000","last_undegraded":"2024-04-26T23:55:29.380403+0000","last_fullsized":"2024-04-26T23:55:29.380403+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:20.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381890+0000","last_change":"2024-04-26T23:55:26.665720+0000","last_active":"2024-04-26T23:55:29.381890+0000","last_peered":"2024-04-26T23:55:29.381890+0000","last_clean":"2024-04-26T23:55:29.381890+0000","last_became_active":"2024-04-26T23:55:26.664975+0000","last_became_peered":"2024-04-26T23:55:26.664975+0000","last_unstale":"2024-04-26T23:55:29.381890+0000","last_undegraded":"2024-04-26T23:55:29.381890+0000","last_fullsized":"2024-04-26T23:55:29.381890+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:43.593928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686239+0000","last_change":"2024-04-26T23:55:27.697777+0000","last_active":"2024-04-26T23:55:28.686239+0000","last_peered":"2024-04-26T23:55:28.686239+0000","last_clean":"2024-04-26T23:55:28.686239+0000","last_became_active":"2024-04-26T23:55:27.697257+0000","last_became_peered":"2024-04-26T23:55:27.697257+0000","last_unstale":"2024-04-26T23:55:28.686239+0000","last_undegraded":"2024-04-26T23:55:28.686239+0000","last_fullsized":"2024-04-26T23:55:28.686239+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:55.018313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383655+0000","last_change":"2024-04-26T23:55:26.665770+0000","last_active":"2024-04-26T23:55:29.383655+0000","last_peered":"2024-04-26T23:55:29.383655+0000","last_clean":"2024-04-26T23:55:29.383655+0000","last_became_active":"2024-04-26T23:55:26.665459+0000","last_became_peered":"2024-04-26T23:55:26.665459+0000","last_unstale":"2024-04-26T23:55:29.383655+0000","last_undegraded":"2024-04-26T23:55:29.383655+0000","last_fullsized":"2024-04-26T23:55:29.383655+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:29.311210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382080+0000","last_change":"2024-04-26T23:55:27.690850+0000","last_active":"2024-04-26T23:55:29.382080+0000","last_peered":"2024-04-26T23:55:29.382080+0000","last_clean":"2024-04-26T23:55:29.382080+0000","last_became_active":"2024-04-26T23:55:27.690558+0000","last_became_peered":"2024-04-26T23:55:27.690558+0000","last_unstale":"2024-04-26T23:55:29.382080+0000","last_undegraded":"2024-04-26T23:55:29.382080+0000","last_fullsized":"2024-04-26T23:55:29.382080+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:50.879329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.653819+0000","last_change":"2024-04-26T23:55:26.655646+0000","last_active":"2024-04-26T23:55:28.653819+0000","last_peered":"2024-04-26T23:55:28.653819+0000","last_clean":"2024-04-26T23:55:28.653819+0000","last_became_active":"2024-04-26T23:55:26.655135+0000","last_became_peered":"2024-04-26T23:55:26.655135+0000","last_unstale":"2024-04-26T23:55:28.653819+0000","last_undegraded":"2024-04-26T23:55:28.653819+0000","last_fullsized":"2024-04-26T23:55:28.653819+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:23.615002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382149+0000","last_change":"2024-04-26T23:55:27.688131+0000","last_active":"2024-04-26T23:55:29.382149+0000","last_peered":"2024-04-26T23:55:29.382149+0000","last_clean":"2024-04-26T23:55:29.382149+0000","last_became_active":"2024-04-26T23:55:27.687709+0000","last_became_peered":"2024-04-26T23:55:27.687709+0000","last_unstale":"2024-04-26T23:55:29.382149+0000","last_undegraded":"2024-04-26T23:55:29.382149+0000","last_fullsized":"2024-04-26T23:55:29.382149+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:27.843956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659055+0000","last_change":"2024-04-26T23:55:26.670222+0000","last_active":"2024-04-26T23:55:28.659055+0000","last_peered":"2024-04-26T23:55:28.659055+0000","last_clean":"2024-04-26T23:55:28.659055+0000","last_became_active":"2024-04-26T23:55:26.669929+0000","last_became_peered":"2024-04-26T23:55:26.669929+0000","last_unstale":"2024-04-26T23:55:28.659055+0000","last_undegraded":"2024-04-26T23:55:28.659055+0000","last_fullsized":"2024-04-26T23:55:28.659055+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:00.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382114+0000","last_change":"2024-04-26T23:55:27.693940+0000","last_active":"2024-04-26T23:55:29.382114+0000","last_peered":"2024-04-26T23:55:29.382114+0000","last_clean":"2024-04-26T23:55:29.382114+0000","last_became_active":"2024-04-26T23:55:27.693632+0000","last_became_peered":"2024-04-26T23:55:27.693632+0000","last_unstale":"2024-04-26T23:55:29.382114+0000","last_undegraded":"2024-04-26T23:55:29.382114+0000","last_fullsized":"2024-04-26T23:55:29.382114+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:34.007155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.658482+0000","last_change":"2024-04-26T23:55:26.666472+0000","last_active":"2024-04-26T23:55:28.658482+0000","last_peered":"2024-04-26T23:55:28.658482+0000","last_clean":"2024-04-26T23:55:28.658482+0000","last_became_active":"2024-04-26T23:55:26.666177+0000","last_became_peered":"2024-04-26T23:55:26.666177+0000","last_unstale":"2024-04-26T23:55:28.658482+0000","last_undegraded":"2024-04-26T23:55:28.658482+0000","last_fullsized":"2024-04-26T23:55:28.658482+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:06.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654607+0000","last_change":"2024-04-26T23:55:27.670090+0000","last_active":"2024-04-26T23:55:28.654607+0000","last_peered":"2024-04-26T23:55:28.654607+0000","last_clean":"2024-04-26T23:55:28.654607+0000","last_became_active":"2024-04-26T23:55:27.668951+0000","last_became_peered":"2024-04-26T23:55:27.668951+0000","last_unstale":"2024-04-26T23:55:28.654607+0000","last_undegraded":"2024-04-26T23:55:28.654607+0000","last_fullsized":"2024-04-26T23:55:28.654607+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:20.074381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657337+0000","last_change":"2024-04-26T23:55:26.667782+0000","last_active":"2024-04-26T23:55:28.657337+0000","last_peered":"2024-04-26T23:55:28.657337+0000","last_clean":"2024-04-26T23:55:28.657337+0000","last_became_active":"2024-04-26T23:55:26.667470+0000","last_became_peered":"2024-04-26T23:55:26.667470+0000","last_unstale":"2024-04-26T23:55:28.657337+0000","last_undegraded":"2024-04-26T23:55:28.657337+0000","last_fullsized":"2024-04-26T23:55:28.657337+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:55.909677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382744+0000","last_change":"2024-04-26T23:55:27.674074+0000","last_active":"2024-04-26T23:55:29.382744+0000","last_peered":"2024-04-26T23:55:29.382744+0000","last_clean":"2024-04-26T23:55:29.382744+0000","last_became_active":"2024-04-26T23:55:27.673700+0000","last_became_peered":"2024-04-26T23:55:27.673700+0000","last_unstale":"2024-04-26T23:55:29.382744+0000","last_undegraded":"2024-04-26T23:55:29.382744+0000","last_fullsized":"2024-04-26T23:55:29.382744+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:33.107897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382575+0000","last_change":"2024-04-26T23:55:26.667612+0000","last_active":"2024-04-26T23:55:29.382575+0000","last_peered":"2024-04-26T23:55:29.382575+0000","last_clean":"2024-04-26T23:55:29.382575+0000","last_became_active":"2024-04-26T23:55:26.667319+0000","last_became_peered":"2024-04-26T23:55:26.667319+0000","last_unstale":"2024-04-26T23:55:29.382575+0000","last_undegraded":"2024-04-26T23:55:29.382575+0000","last_fullsized":"2024-04-26T23:55:29.382575+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:39.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657393+0000","last_change":"2024-04-26T23:55:27.687214+0000","last_active":"2024-04-26T23:55:28.657393+0000","last_peered":"2024-04-26T23:55:28.657393+0000","last_clean":"2024-04-26T23:55:28.657393+0000","last_became_active":"2024-04-26T23:55:27.686910+0000","last_became_peered":"2024-04-26T23:55:27.686910+0000","last_unstale":"2024-04-26T23:55:28.657393+0000","last_undegraded":"2024-04-26T23:55:28.657393+0000","last_fullsized":"2024-04-26T23:55:28.657393+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:09.658367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.660736+0000","last_change":"2024-04-26T23:55:26.671146+0000","last_active":"2024-04-26T23:55:28.660736+0000","last_peered":"2024-04-26T23:55:28.660736+0000","last_clean":"2024-04-26T23:55:28.660736+0000","last_became_active":"2024-04-26T23:55:26.670686+0000","last_became_peered":"2024-04-26T23:55:26.670686+0000","last_unstale":"2024-04-26T23:55:28.660736+0000","last_undegraded":"2024-04-26T23:55:28.660736+0000","last_fullsized":"2024-04-26T23:55:28.660736+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:40.585210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654004+0000","last_change":"2024-04-26T23:55:27.681523+0000","last_active":"2024-04-26T23:55:28.654004+0000","last_peered":"2024-04-26T23:55:28.654004+0000","last_clean":"2024-04-26T23:55:28.654004+0000","last_became_active":"2024-04-26T23:55:27.681167+0000","last_became_peered":"2024-04-26T23:55:27.681167+0000","last_unstale":"2024-04-26T23:55:28.654004+0000","last_undegraded":"2024-04-26T23:55:28.654004+0000","last_fullsized":"2024-04-26T23:55:28.654004+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:08.695684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"185'3","reported_seq":17,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.653854+0000","last_change":"2024-04-26T23:55:26.656669+0000","last_active":"2024-04-26T23:55:28.653854+0000","last_peered":"2024-04-26T23:55:28.653854+0000","last_clean":"2024-04-26T23:55:28.653854+0000","last_became_active":"2024-04-26T23:55:26.656384+0000","last_became_peered":"2024-04-26T23:55:26.656384+0000","last_unstale":"2024-04-26T23:55:28.653854+0000","last_undegraded":"2024-04-26T23:55:28.653854+0000","last_fullsized":"2024-04-26T23:55:28.653854+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:55.877492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382325+0000","last_change":"2024-04-26T23:55:27.705414+0000","last_active":"2024-04-26T23:55:29.382325+0000","last_peered":"2024-04-26T23:55:29.382325+0000","last_clean":"2024-04-26T23:55:29.382325+0000","last_became_active":"2024-04-26T23:55:27.705117+0000","last_became_peered":"2024-04-26T23:55:27.705117+0000","last_unstale":"2024-04-26T23:55:29.382325+0000","last_undegraded":"2024-04-26T23:55:29.382325+0000","last_fullsized":"2024-04-26T23:55:29.382325+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:24.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"185'6","reported_seq":20,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659443+0000","last_change":"2024-04-26T23:55:26.674344+0000","last_active":"2024-04-26T23:55:28.659443+0000","last_peered":"2024-04-26T23:55:28.659443+0000","last_clean":"2024-04-26T23:55:28.659443+0000","last_became_active":"2024-04-26T23:55:26.673972+0000","last_became_peered":"2024-04-26T23:55:26.673972+0000","last_unstale":"2024-04-26T23:55:28.659443+0000","last_undegraded":"2024-04-26T23:55:28.659443+0000","last_fullsized":"2024-04-26T23:55:28.659443+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:43.593928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383637+0000","last_change":"2024-04-26T23:55:27.692722+0000","last_active":"2024-04-26T23:55:29.383637+0000","last_peered":"2024-04-26T23:55:29.383637+0000","last_clean":"2024-04-26T23:55:29.383637+0000","last_became_active":"2024-04-26T23:55:27.692133+0000","last_became_peered":"2024-04-26T23:55:27.692133+0000","last_unstale":"2024-04-26T23:55:29.383637+0000","last_undegraded":"2024-04-26T23:55:29.383637+0000","last_fullsized":"2024-04-26T23:55:29.383637+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:28.009957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383306+0000","last_change":"2024-04-26T23:55:26.654783+0000","last_active":"2024-04-26T23:55:29.383306+0000","last_peered":"2024-04-26T23:55:29.383306+0000","last_clean":"2024-04-26T23:55:29.383306+0000","last_became_active":"2024-04-26T23:55:26.654439+0000","last_became_peered":"2024-04-26T23:55:26.654439+0000","last_unstale":"2024-04-26T23:55:29.383306+0000","last_undegraded":"2024-04-26T23:55:29.383306+0000","last_fullsized":"2024-04-26T23:55:29.383306+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:19.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657981+0000","last_change":"2024-04-26T23:55:27.672295+0000","last_active":"2024-04-26T23:55:28.657981+0000","last_peered":"2024-04-26T23:55:28.657981+0000","last_clean":"2024-04-26T23:55:28.657981+0000","last_became_active":"2024-04-26T23:55:27.671945+0000","last_became_peered":"2024-04-26T23:55:27.671945+0000","last_unstale":"2024-04-26T23:55:28.657981+0000","last_undegraded":"2024-04-26T23:55:28.657981+0000","last_fullsized":"2024-04-26T23:55:28.657981+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:02.634591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657409+0000","last_change":"2024-04-26T23:55:26.660467+0000","last_active":"2024-04-26T23:55:28.657409+0000","last_peered":"2024-04-26T23:55:28.657409+0000","last_clean":"2024-04-26T23:55:28.657409+0000","last_became_active":"2024-04-26T23:55:26.659561+0000","last_became_peered":"2024-04-26T23:55:26.659561+0000","last_unstale":"2024-04-26T23:55:28.657409+0000","last_undegraded":"2024-04-26T23:55:28.657409+0000","last_fullsized":"2024-04-26T23:55:28.657409+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:40.585210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654047+0000","last_change":"2024-04-26T23:55:27.691270+0000","last_active":"2024-04-26T23:55:28.654047+0000","last_peered":"2024-04-26T23:55:28.654047+0000","last_clean":"2024-04-26T23:55:28.654047+0000","last_became_active":"2024-04-26T23:55:27.690962+0000","last_became_peered":"2024-04-26T23:55:27.690962+0000","last_unstale":"2024-04-26T23:55:28.654047+0000","last_undegraded":"2024-04-26T23:55:28.654047+0000","last_fullsized":"2024-04-26T23:55:28.654047+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:41.572083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.653438+0000","last_change":"2024-04-26T23:55:26.664822+0000","last_active":"2024-04-26T23:55:28.653438+0000","last_peered":"2024-04-26T23:55:28.653438+0000","last_clean":"2024-04-26T23:55:28.653438+0000","last_became_active":"2024-04-26T23:55:26.662519+0000","last_became_peered":"2024-04-26T23:55:26.662519+0000","last_unstale":"2024-04-26T23:55:28.653438+0000","last_undegraded":"2024-04-26T23:55:28.653438+0000","last_fullsized":"2024-04-26T23:55:28.653438+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:39.321880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.660995+0000","last_change":"2024-04-26T23:55:27.692692+0000","last_active":"2024-04-26T23:55:28.660995+0000","last_peered":"2024-04-26T23:55:28.660995+0000","last_clean":"2024-04-26T23:55:28.660995+0000","last_became_active":"2024-04-26T23:55:27.690087+0000","last_became_peered":"2024-04-26T23:55:27.690087+0000","last_unstale":"2024-04-26T23:55:28.660995+0000","last_undegraded":"2024-04-26T23:55:28.660995+0000","last_fullsized":"2024-04-26T23:55:28.660995+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:56.096024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"185'8","reported_seq":22,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.380674+0000","last_change":"2024-04-26T23:55:26.674180+0000","last_active":"2024-04-26T23:55:29.380674+0000","last_peered":"2024-04-26T23:55:29.380674+0000","last_clean":"2024-04-26T23:55:29.380674+0000","last_became_active":"2024-04-26T23:55:26.673851+0000","last_became_peered":"2024-04-26T23:55:26.673851+0000","last_unstale":"2024-04-26T23:55:29.380674+0000","last_undegraded":"2024-04-26T23:55:29.380674+0000","last_fullsized":"2024-04-26T23:55:29.380674+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.645703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.685672+0000","last_change":"2024-04-26T23:55:27.691228+0000","last_active":"2024-04-26T23:55:28.685672+0000","last_peered":"2024-04-26T23:55:28.685672+0000","last_clean":"2024-04-26T23:55:28.685672+0000","last_became_active":"2024-04-26T23:55:27.690943+0000","last_became_peered":"2024-04-26T23:55:27.690943+0000","last_unstale":"2024-04-26T23:55:28.685672+0000","last_undegraded":"2024-04-26T23:55:28.685672+0000","last_fullsized":"2024-04-26T23:55:28.685672+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:07.348152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381621+0000","last_change":"2024-04-26T23:55:26.657441+0000","last_active":"2024-04-26T23:55:29.381621+0000","last_peered":"2024-04-26T23:55:29.381621+0000","last_clean":"2024-04-26T23:55:29.381621+0000","last_became_active":"2024-04-26T23:55:26.657029+0000","last_became_peered":"2024-04-26T23:55:26.657029+0000","last_unstale":"2024-04-26T23:55:29.381621+0000","last_undegraded":"2024-04-26T23:55:29.381621+0000","last_fullsized":"2024-04-26T23:55:29.381621+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:16.758447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.380673+0000","last_change":"2024-04-26T23:55:27.688631+0000","last_active":"2024-04-26T23:55:29.380673+0000","last_peered":"2024-04-26T23:55:29.380673+0000","last_clean":"2024-04-26T23:55:29.380673+0000","last_became_active":"2024-04-26T23:55:27.688338+0000","last_became_peered":"2024-04-26T23:55:27.688338+0000","last_unstale":"2024-04-26T23:55:29.380673+0000","last_undegraded":"2024-04-26T23:55:29.380673+0000","last_fullsized":"2024-04-26T23:55:29.380673+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:53.034308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382154+0000","last_change":"2024-04-26T23:55:26.669149+0000","last_active":"2024-04-26T23:55:29.382154+0000","last_peered":"2024-04-26T23:55:29.382154+0000","last_clean":"2024-04-26T23:55:29.382154+0000","last_became_active":"2024-04-26T23:55:26.668869+0000","last_became_peered":"2024-04-26T23:55:26.668869+0000","last_unstale":"2024-04-26T23:55:29.382154+0000","last_undegraded":"2024-04-26T23:55:29.382154+0000","last_fullsized":"2024-04-26T23:55:29.382154+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:00.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381451+0000","last_change":"2024-04-26T23:55:27.693570+0000","last_active":"2024-04-26T23:55:29.381451+0000","last_peered":"2024-04-26T23:55:29.381451+0000","last_clean":"2024-04-26T23:55:29.381451+0000","last_became_active":"2024-04-26T23:55:27.693239+0000","last_became_peered":"2024-04-26T23:55:27.693239+0000","last_unstale":"2024-04-26T23:55:29.381451+0000","last_undegraded":"2024-04-26T23:55:29.381451+0000","last_fullsized":"2024-04-26T23:55:29.381451+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:42.475439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383860+0000","last_change":"2024-04-26T23:55:26.665188+0000","last_active":"2024-04-26T23:55:29.383860+0000","last_peered":"2024-04-26T23:55:29.383860+0000","last_clean":"2024-04-26T23:55:29.383860+0000","last_became_active":"2024-04-26T23:55:26.663507+0000","last_became_peered":"2024-04-26T23:55:26.663507+0000","last_unstale":"2024-04-26T23:55:29.383860+0000","last_undegraded":"2024-04-26T23:55:29.383860+0000","last_fullsized":"2024-04-26T23:55:29.383860+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:11.135537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686487+0000","last_change":"2024-04-26T23:55:27.692702+0000","last_active":"2024-04-26T23:55:28.686487+0000","last_peered":"2024-04-26T23:55:28.686487+0000","last_clean":"2024-04-26T23:55:28.686487+0000","last_became_active":"2024-04-26T23:55:27.689787+0000","last_became_peered":"2024-04-26T23:55:27.689787+0000","last_unstale":"2024-04-26T23:55:28.686487+0000","last_undegraded":"2024-04-26T23:55:28.686487+0000","last_fullsized":"2024-04-26T23:55:28.686487+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:08.358703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382687+0000","last_change":"2024-04-26T23:55:26.661133+0000","last_active":"2024-04-26T23:55:29.382687+0000","last_peered":"2024-04-26T23:55:29.382687+0000","last_clean":"2024-04-26T23:55:29.382687+0000","last_became_active":"2024-04-26T23:55:26.660821+0000","last_became_peered":"2024-04-26T23:55:26.660821+0000","last_unstale":"2024-04-26T23:55:29.382687+0000","last_undegraded":"2024-04-26T23:55:29.382687+0000","last_fullsized":"2024-04-26T23:55:29.382687+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:50.574742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.383876+0000","last_change":"2024-04-26T23:55:27.692481+0000","last_active":"2024-04-26T23:55:29.383876+0000","last_peered":"2024-04-26T23:55:29.383876+0000","last_clean":"2024-04-26T23:55:29.383876+0000","last_became_active":"2024-04-26T23:55:27.691753+0000","last_became_peered":"2024-04-26T23:55:27.691753+0000","last_unstale":"2024-04-26T23:55:29.383876+0000","last_undegraded":"2024-04-26T23:55:29.383876+0000","last_fullsized":"2024-04-26T23:55:29.383876+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.014109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382238+0000","last_change":"2024-04-26T23:55:26.662155+0000","last_active":"2024-04-26T23:55:29.382238+0000","last_peered":"2024-04-26T23:55:29.382238+0000","last_clean":"2024-04-26T23:55:29.382238+0000","last_became_active":"2024-04-26T23:55:26.661883+0000","last_became_peered":"2024-04-26T23:55:26.661883+0000","last_unstale":"2024-04-26T23:55:29.382238+0000","last_undegraded":"2024-04-26T23:55:29.382238+0000","last_fullsized":"2024-04-26T23:55:29.382238+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:36.122175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.659902+0000","last_change":"2024-04-26T23:55:27.691947+0000","last_active":"2024-04-26T23:55:28.659902+0000","last_peered":"2024-04-26T23:55:28.659902+0000","last_clean":"2024-04-26T23:55:28.659902+0000","last_became_active":"2024-04-26T23:55:27.691194+0000","last_became_peered":"2024-04-26T23:55:27.691194+0000","last_unstale":"2024-04-26T23:55:28.659902+0000","last_undegraded":"2024-04-26T23:55:28.659902+0000","last_fullsized":"2024-04-26T23:55:28.659902+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:02.634591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.382390+0000","last_change":"2024-04-26T23:55:26.657266+0000","last_active":"2024-04-26T23:55:29.382390+0000","last_peered":"2024-04-26T23:55:29.382390+0000","last_clean":"2024-04-26T23:55:29.382390+0000","last_became_active":"2024-04-26T23:55:26.656981+0000","last_became_peered":"2024-04-26T23:55:26.656981+0000","last_unstale":"2024-04-26T23:55:29.382390+0000","last_undegraded":"2024-04-26T23:55:29.382390+0000","last_fullsized":"2024-04-26T23:55:29.382390+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:35.941444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654209+0000","last_change":"2024-04-26T23:55:27.686347+0000","last_active":"2024-04-26T23:55:28.654209+0000","last_peered":"2024-04-26T23:55:28.654209+0000","last_clean":"2024-04-26T23:55:28.654209+0000","last_became_active":"2024-04-26T23:55:27.686080+0000","last_became_peered":"2024-04-26T23:55:27.686080+0000","last_unstale":"2024-04-26T23:55:28.654209+0000","last_undegraded":"2024-04-26T23:55:28.654209+0000","last_fullsized":"2024-04-26T23:55:28.654209+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:49.689779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381377+0000","last_change":"2024-04-26T23:55:26.669413+0000","last_active":"2024-04-26T23:55:29.381377+0000","last_peered":"2024-04-26T23:55:29.381377+0000","last_clean":"2024-04-26T23:55:29.381377+0000","last_became_active":"2024-04-26T23:55:26.666763+0000","last_became_peered":"2024-04-26T23:55:26.666763+0000","last_unstale":"2024-04-26T23:55:29.381377+0000","last_undegraded":"2024-04-26T23:55:29.381377+0000","last_fullsized":"2024-04-26T23:55:29.381377+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:18:28.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.687581+0000","last_change":"2024-04-26T23:55:27.697644+0000","last_active":"2024-04-26T23:55:28.687581+0000","last_peered":"2024-04-26T23:55:28.687581+0000","last_clean":"2024-04-26T23:55:28.687581+0000","last_became_active":"2024-04-26T23:55:27.697353+0000","last_became_peered":"2024-04-26T23:55:27.697353+0000","last_unstale":"2024-04-26T23:55:28.687581+0000","last_undegraded":"2024-04-26T23:55:28.687581+0000","last_fullsized":"2024-04-26T23:55:28.687581+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:38.939646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.380086+0000","last_change":"2024-04-26T23:55:26.657224+0000","last_active":"2024-04-26T23:55:29.380086+0000","last_peered":"2024-04-26T23:55:29.380086+0000","last_clean":"2024-04-26T23:55:29.380086+0000","last_became_active":"2024-04-26T23:55:26.656834+0000","last_became_peered":"2024-04-26T23:55:26.656834+0000","last_unstale":"2024-04-26T23:55:29.380086+0000","last_undegraded":"2024-04-26T23:55:29.380086+0000","last_fullsized":"2024-04-26T23:55:29.380086+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:19:06.601107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686174+0000","last_change":"2024-04-26T23:55:27.687552+0000","last_active":"2024-04-26T23:55:28.686174+0000","last_peered":"2024-04-26T23:55:28.686174+0000","last_clean":"2024-04-26T23:55:28.686174+0000","last_became_active":"2024-04-26T23:55:27.687251+0000","last_became_peered":"2024-04-26T23:55:27.687251+0000","last_unstale":"2024-04-26T23:55:28.686174+0000","last_undegraded":"2024-04-26T23:55:28.686174+0000","last_fullsized":"2024-04-26T23:55:28.686174+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:50.879329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654541+0000","last_change":"2024-04-26T23:55:26.672031+0000","last_active":"2024-04-26T23:55:28.654541+0000","last_peered":"2024-04-26T23:55:28.654541+0000","last_clean":"2024-04-26T23:55:28.654541+0000","last_became_active":"2024-04-26T23:55:26.671772+0000","last_became_peered":"2024-04-26T23:55:26.671772+0000","last_unstale":"2024-04-26T23:55:28.654541+0000","last_undegraded":"2024-04-26T23:55:28.654541+0000","last_fullsized":"2024-04-26T23:55:28.654541+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:00.980972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657418+0000","last_change":"2024-04-26T23:55:27.684404+0000","last_active":"2024-04-26T23:55:28.657418+0000","last_peered":"2024-04-26T23:55:28.657418+0000","last_clean":"2024-04-26T23:55:28.657418+0000","last_became_active":"2024-04-26T23:55:27.684121+0000","last_became_peered":"2024-04-26T23:55:27.684121+0000","last_unstale":"2024-04-26T23:55:28.657418+0000","last_undegraded":"2024-04-26T23:55:28.657418+0000","last_fullsized":"2024-04-26T23:55:28.657418+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:42.475439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"186'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.687061+0000","last_change":"2024-04-26T23:55:26.669694+0000","last_active":"2024-04-26T23:55:28.687061+0000","last_peered":"2024-04-26T23:55:28.687061+0000","last_clean":"2024-04-26T23:55:28.687061+0000","last_became_active":"2024-04-26T23:55:26.669375+0000","last_became_peered":"2024-04-26T23:55:26.669375+0000","last_unstale":"2024-04-26T23:55:28.687061+0000","last_undegraded":"2024-04-26T23:55:28.687061+0000","last_fullsized":"2024-04-26T23:55:28.687061+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:56.703467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.653627+0000","last_change":"2024-04-26T23:55:27.695172+0000","last_active":"2024-04-26T23:55:28.653627+0000","last_peered":"2024-04-26T23:55:28.653627+0000","last_clean":"2024-04-26T23:55:28.653627+0000","last_became_active":"2024-04-26T23:55:27.694742+0000","last_became_peered":"2024-04-26T23:55:27.694742+0000","last_unstale":"2024-04-26T23:55:28.653627+0000","last_undegraded":"2024-04-26T23:55:28.653627+0000","last_fullsized":"2024-04-26T23:55:28.653627+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:18.785093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381589+0000","last_change":"2024-04-26T23:55:26.652329+0000","last_active":"2024-04-26T23:55:29.381589+0000","last_peered":"2024-04-26T23:55:29.381589+0000","last_clean":"2024-04-26T23:55:29.381589+0000","last_became_active":"2024-04-26T23:55:26.651690+0000","last_became_peered":"2024-04-26T23:55:26.651690+0000","last_unstale":"2024-04-26T23:55:29.381589+0000","last_undegraded":"2024-04-26T23:55:29.381589+0000","last_fullsized":"2024-04-26T23:55:29.381589+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:09.019216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381396+0000","last_change":"2024-04-26T23:55:27.685905+0000","last_active":"2024-04-26T23:55:29.381396+0000","last_peered":"2024-04-26T23:55:29.381396+0000","last_clean":"2024-04-26T23:55:29.381396+0000","last_became_active":"2024-04-26T23:55:27.685614+0000","last_became_peered":"2024-04-26T23:55:27.685614+0000","last_unstale":"2024-04-26T23:55:29.381396+0000","last_undegraded":"2024-04-26T23:55:29.381396+0000","last_fullsized":"2024-04-26T23:55:29.381396+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:09.658367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381884+0000","last_change":"2024-04-26T23:55:26.664502+0000","last_active":"2024-04-26T23:55:29.381884+0000","last_peered":"2024-04-26T23:55:29.381884+0000","last_clean":"2024-04-26T23:55:29.381884+0000","last_became_active":"2024-04-26T23:55:26.664195+0000","last_became_peered":"2024-04-26T23:55:26.664195+0000","last_unstale":"2024-04-26T23:55:29.381884+0000","last_undegraded":"2024-04-26T23:55:29.381884+0000","last_fullsized":"2024-04-26T23:55:29.381884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:46.122354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686822+0000","last_change":"2024-04-26T23:55:27.695219+0000","last_active":"2024-04-26T23:55:28.686822+0000","last_peered":"2024-04-26T23:55:28.686822+0000","last_clean":"2024-04-26T23:55:28.686822+0000","last_became_active":"2024-04-26T23:55:27.694935+0000","last_became_peered":"2024-04-26T23:55:27.694935+0000","last_unstale":"2024-04-26T23:55:28.686822+0000","last_undegraded":"2024-04-26T23:55:28.686822+0000","last_fullsized":"2024-04-26T23:55:28.686822+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:24.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.686514+0000","last_change":"2024-04-26T23:55:26.673198+0000","last_active":"2024-04-26T23:55:28.686514+0000","last_peered":"2024-04-26T23:55:28.686514+0000","last_clean":"2024-04-26T23:55:28.686514+0000","last_became_active":"2024-04-26T23:55:26.672369+0000","last_became_peered":"2024-04-26T23:55:26.672369+0000","last_unstale":"2024-04-26T23:55:28.686514+0000","last_undegraded":"2024-04-26T23:55:28.686514+0000","last_fullsized":"2024-04-26T23:55:28.686514+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:40.236899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.660081+0000","last_change":"2024-04-26T23:55:27.695960+0000","last_active":"2024-04-26T23:55:28.660081+0000","last_peered":"2024-04-26T23:55:28.660081+0000","last_clean":"2024-04-26T23:55:28.660081+0000","last_became_active":"2024-04-26T23:55:27.691622+0000","last_became_peered":"2024-04-26T23:55:27.691622+0000","last_unstale":"2024-04-26T23:55:28.660081+0000","last_undegraded":"2024-04-26T23:55:28.660081+0000","last_fullsized":"2024-04-26T23:55:28.660081+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:24.469652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654074+0000","last_change":"2024-04-26T23:55:26.651389+0000","last_active":"2024-04-26T23:55:28.654074+0000","last_peered":"2024-04-26T23:55:28.654074+0000","last_clean":"2024-04-26T23:55:28.654074+0000","last_became_active":"2024-04-26T23:55:26.651038+0000","last_became_peered":"2024-04-26T23:55:26.651038+0000","last_unstale":"2024-04-26T23:55:28.654074+0000","last_undegraded":"2024-04-26T23:55:28.654074+0000","last_fullsized":"2024-04-26T23:55:28.654074+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:19.775622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.660973+0000","last_change":"2024-04-26T23:55:27.696501+0000","last_active":"2024-04-26T23:55:28.660973+0000","last_peered":"2024-04-26T23:55:28.660973+0000","last_clean":"2024-04-26T23:55:28.660973+0000","last_became_active":"2024-04-26T23:55:27.692287+0000","last_became_peered":"2024-04-26T23:55:27.692287+0000","last_unstale":"2024-04-26T23:55:28.660973+0000","last_undegraded":"2024-04-26T23:55:28.660973+0000","last_fullsized":"2024-04-26T23:55:28.660973+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:51.229107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.380748+0000","last_change":"2024-04-26T23:55:26.667527+0000","last_active":"2024-04-26T23:55:29.380748+0000","last_peered":"2024-04-26T23:55:29.380748+0000","last_clean":"2024-04-26T23:55:29.380748+0000","last_became_active":"2024-04-26T23:55:26.667230+0000","last_became_peered":"2024-04-26T23:55:26.667230+0000","last_unstale":"2024-04-26T23:55:29.380748+0000","last_undegraded":"2024-04-26T23:55:29.380748+0000","last_fullsized":"2024-04-26T23:55:29.380748+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:06.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.657900+0000","last_change":"2024-04-26T23:55:27.687596+0000","last_active":"2024-04-26T23:55:28.657900+0000","last_peered":"2024-04-26T23:55:28.657900+0000","last_clean":"2024-04-26T23:55:28.657900+0000","last_became_active":"2024-04-26T23:55:27.686923+0000","last_became_peered":"2024-04-26T23:55:27.686923+0000","last_unstale":"2024-04-26T23:55:28.657900+0000","last_undegraded":"2024-04-26T23:55:28.657900+0000","last_fullsized":"2024-04-26T23:55:28.657900+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:48.959458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381285+0000","last_change":"2024-04-26T23:55:26.667602+0000","last_active":"2024-04-26T23:55:29.381285+0000","last_peered":"2024-04-26T23:55:29.381285+0000","last_clean":"2024-04-26T23:55:29.381285+0000","last_became_active":"2024-04-26T23:55:26.667305+0000","last_became_peered":"2024-04-26T23:55:26.667305+0000","last_unstale":"2024-04-26T23:55:29.381285+0000","last_undegraded":"2024-04-26T23:55:29.381285+0000","last_fullsized":"2024-04-26T23:55:29.381285+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:00.816014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:29.381161+0000","last_change":"2024-04-26T23:55:27.699917+0000","last_active":"2024-04-26T23:55:29.381161+0000","last_peered":"2024-04-26T23:55:29.381161+0000","last_clean":"2024-04-26T23:55:29.381161+0000","last_became_active":"2024-04-26T23:55:27.699555+0000","last_became_peered":"2024-04-26T23:55:27.699555+0000","last_unstale":"2024-04-26T23:55:29.381161+0000","last_undegraded":"2024-04-26T23:55:29.381161+0000","last_fullsized":"2024-04-26T23:55:29.381161+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:26.623878+0000","last_clean_scrub_stamp":"2024-04-26T23:55:26.623878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:34.172112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.654011+0000","last_change":"2024-04-26T23:55:26.664184+0000","last_active":"2024-04-26T23:55:28.654011+0000","last_peered":"2024-04-26T23:55:28.654011+0000","last_clean":"2024-04-26T23:55:28.654011+0000","last_became_active":"2024-04-26T23:55:26.663866+0000","last_became_peered":"2024-04-26T23:55:26.663866+0000","last_unstale":"2024-04-26T23:55:28.654011+0000","last_undegraded":"2024-04-26T23:55:28.654011+0000","last_fullsized":"2024-04-26T23:55:28.654011+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:55:25.606992+0000","last_clean_scrub_stamp":"2024-04-26T23:55:25.606992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:52.017422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":478,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-26T23:55:28.658544+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:55:28.658544+0000","last_peered":"2024-04-26T23:55:28.658544+0000","last_clean":"2024-04-26T23:55:28.658544+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:55:28.658544+0000","last_undegraded":"2024-04-26T23:55:28.658544+0000","last_fullsized":"2024-04-26T23:55:28.658544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575072,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52116,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319724,"statfs":{"total":96636764160,"available":96583397376,"internally_reserved":0,"allocated":2957312,"data_stored":1701413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575073,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":2363392,"data_stored":1111557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575073,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55692,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316148,"statfs":{"total":96636764160,"available":96579735552,"internally_reserved":0,"allocated":2355200,"data_stored":1109419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575072,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2351104,"data_stored":1109945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575073,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47372,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324468,"statfs":{"total":96636764160,"available":96588255232,"internally_reserved":0,"allocated":2359296,"data_stored":1109489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575073,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2351104,"data_stored":1109453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575073,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2347008,"data_stored":1109363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575073,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":2945024,"data_stored":1699821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:55:32.687 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:55:32.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:55:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.952+0000 7efc9d600640 1 Processor -- start 2024-04-26T23:55:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.952+0000 7efc9d600640 1 -- start start 2024-04-26T23:55:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc9d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc980d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc9d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc980d3b90 0x7efc980d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc9d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc980b46f0 con 0x7efc980d5b90 2024-04-26T23:55:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc9d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7efc980d11d0 con 0x7efc980d3b90 2024-04-26T23:55:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc9d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc980b44a0 con 0x7efc98162aa0 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc97400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc980d3b90 0x7efc980d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc97400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc980d3b90 0x7efc980d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58466/0 (socket says 172.21.15.46:58466) 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.954+0000 7efc97400640 1 -- 172.21.15.46:0/2750914535 learned_addr learned my addr 172.21.15.46:0/2750914535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.955+0000 7efc97400640 1 -- 172.21.15.46:0/2750914535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 msgr2=0x7efc980d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.955+0000 7efc97400640 1 --2- 172.21.15.46:0/2750914535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc980d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.955+0000 7efc97400640 1 -- 172.21.15.46:0/2750914535 >> v1:172.21.15.142:6789/0 conn(0x7efc980d5b90 legacy=0x7efc980b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.955+0000 7efc97400640 1 -- 172.21.15.46:0/2750914535 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc980b49d0 con 0x7efc980d3b90 2024-04-26T23:55:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.956+0000 7efc97400640 1 --2- 172.21.15.46:0/2750914535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc980d0c90 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7efc90009870 tx=0x7efc9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ee63281ba8dd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.956+0000 7efc95600640 1 -- 172.21.15.46:0/2750914535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90037070 con 0x7efc980d3b90 2024-04-26T23:55:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.956+0000 7efc95600640 1 -- 172.21.15.46:0/2750914535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc90032440 con 0x7efc980d3b90 2024-04-26T23:55:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.956+0000 7efc95600640 1 -- 172.21.15.46:0/2750914535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90032810 con 0x7efc980d3b90 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 -- 172.21.15.46:0/2750914535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 msgr2=0x7efc980d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 --2- 172.21.15.46:0/2750914535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc980d0c90 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7efc90009870 tx=0x7efc9002a180 comp rx=0 tx=0).stop 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 -- 172.21.15.46:0/2750914535 shutdown_connections 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 --2- 172.21.15.46:0/2750914535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc980d0c90 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 --2- 172.21.15.46:0/2750914535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc980d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 -- 172.21.15.46:0/2750914535 >> 172.21.15.46:0/2750914535 conn(0x7efc980d5070 msgr2=0x7efc980d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.957+0000 7efc9d600640 1 -- 172.21.15.46:0/2750914535 shutdown_connections 2024-04-26T23:55:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.958+0000 7efc9d600640 1 -- 172.21.15.46:0/2750914535 wait complete. 2024-04-26T23:55:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.959+0000 7efc9d600640 1 Processor -- start 2024-04-26T23:55:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.959+0000 7efc9d600640 1 -- start start 2024-04-26T23:55:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc980d5b90 0x7efc980c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc981f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc980d3fd0 con 0x7efc980d3b90 2024-04-26T23:55:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc980b5020 con 0x7efc980d5b90 2024-04-26T23:55:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.960+0000 7efc9d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc98162e80 con 0x7efc98162aa0 2024-04-26T23:55:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58476/0 (socket says 172.21.15.46:58476) 2024-04-26T23:55:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 -- 172.21.15.46:0/511136473 learned_addr learned my addr 172.21.15.46:0/511136473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc97400640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc981f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 -- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 msgr2=0x7efc981f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc981f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 -- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc980d5b90 msgr2=0x7efc980c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc980d5b90 0x7efc980c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 -- 172.21.15.46:0/511136473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc980caa70 con 0x7efc980d3b90 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.961+0000 7efc96a00640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ba50 tx=0x7efc8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.962+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88019070 con 0x7efc980d3b90 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.962+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8800cc00 con 0x7efc980d3b90 2024-04-26T23:55:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.962+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88014440 con 0x7efc980d3b90 2024-04-26T23:55:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.962+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc980b8000 con 0x7efc980d3b90 2024-04-26T23:55:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.962+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc980b85c0 con 0x7efc980d3b90 2024-04-26T23:55:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.963+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc98069530 con 0x7efc980d3b90 2024-04-26T23:55:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.964+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc8801c070 con 0x7efc980d3b90 2024-04-26T23:55:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.965+0000 7efc8ea00640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 0x7efc7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.965+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7efc8808ba90 con 0x7efc980d3b90 2024-04-26T23:55:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.966+0000 7efc96000640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 0x7efc7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.967+0000 7efc96000640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 0x7efc7806edb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efc980cb190 tx=0x7efc840079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:32.970+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc88059d20 con 0x7efc980d3b90 2024-04-26T23:55:33.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.251+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7efc9806d960 con 0x7efc980d3b90 2024-04-26T23:55:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.255+0000 7efc8ea00640 1 -- 172.21.15.46:0/511136473 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7efc8808b020 con 0x7efc980d3b90 2024-04-26T23:55:33.257 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:55:33.258 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:55:33.258 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:55:33.258 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:55:33.258 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:55:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 msgr2=0x7efc7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 0x7efc7806edb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efc980cb190 tx=0x7efc840079b0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 msgr2=0x7efc981f5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ba50 tx=0x7efc8800bf10 comp rx=0 tx=0).stop 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 shutdown_connections 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efc7806c8f0 0x7efc7806edb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc98162aa0 0x7efc981f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc980d5b90 0x7efc980c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 --2- 172.21.15.46:0/511136473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc980d3b90 0x7efc981f5f70 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 >> 172.21.15.46:0/511136473 conn(0x7efc980d5070 msgr2=0x7efc98160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 shutdown_connections 2024-04-26T23:55:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.258+0000 7efc9d600640 1 -- 172.21.15.46:0/511136473 wait complete. 2024-04-26T23:55:33.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.473+0000 7fa183000640 1 Processor -- start 2024-04-26T23:55:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.474+0000 7fa183000640 1 -- start start 2024-04-26T23:55:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.475+0000 7fa183000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.475+0000 7fa183000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa17c0732c0 0x7fa17c073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.475+0000 7fa183000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17c161940 con 0x7fa17c06e510 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.475+0000 7fa183000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa17c071f80 con 0x7fa17c0732c0 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.475+0000 7fa183000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c072260 con 0x7fa17c160bd0 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa17c0732c0 0x7fa17c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa17c0732c0 0x7fa17c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58484/0 (socket says 172.21.15.46:58484) 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 -- 172.21.15.46:0/139384388 learned_addr learned my addr 172.21.15.46:0/139384388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa17b400640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 -- 172.21.15.46:0/139384388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 msgr2=0x7fa17c161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 -- 172.21.15.46:0/139384388 >> v1:172.21.15.142:6789/0 conn(0x7fa17c06e510 legacy=0x7fa17c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa181000640 1 -- 172.21.15.46:0/139384388 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c072540 con 0x7fa17c0732c0 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.476+0000 7fa17b400640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.477+0000 7fa181000640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c073750 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fa174009e50 tx=0x7fa17402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fab9271b38a7628 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.478+0000 7fa17aa00640 1 -- 172.21.15.46:0/139384388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174036070 con 0x7fa17c0732c0 2024-04-26T23:55:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.478+0000 7fa17aa00640 1 -- 172.21.15.46:0/139384388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174031600 con 0x7fa17c0732c0 2024-04-26T23:55:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.478+0000 7fa17aa00640 1 -- 172.21.15.46:0/139384388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1740319d0 con 0x7fa17c0732c0 2024-04-26T23:55:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.478+0000 7fa183000640 1 -- 172.21.15.46:0/139384388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 msgr2=0x7fa17c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.478+0000 7fa183000640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c073750 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fa174009e50 tx=0x7fa17402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 -- 172.21.15.46:0/139384388 shutdown_connections 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c073750 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 --2- 172.21.15.46:0/139384388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 -- 172.21.15.46:0/139384388 >> 172.21.15.46:0/139384388 conn(0x7fa17c06c530 msgr2=0x7fa17c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 -- 172.21.15.46:0/139384388 shutdown_connections 2024-04-26T23:55:33.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.479+0000 7fa183000640 1 -- 172.21.15.46:0/139384388 wait complete. 2024-04-26T23:55:33.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.480+0000 7fa183000640 1 Processor -- start 2024-04-26T23:55:33.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.481+0000 7fa183000640 1 -- start start 2024-04-26T23:55:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa17c06e510 0x7fa17c1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1616f0 con 0x7fa17c0732c0 2024-04-26T23:55:33.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c161570 con 0x7fa17c06e510 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa183000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c072d40 con 0x7fa17c160bd0 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58492/0 (socket says 172.21.15.46:58492) 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 -- 172.21.15.46:0/1391767414 learned_addr learned my addr 172.21.15.46:0/1391767414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 -- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 msgr2=0x7fa17c141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa181000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 -- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa17c06e510 msgr2=0x7fa17c1f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa17c06e510 0x7fa17c1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.482+0000 7fa17b400640 1 -- 172.21.15.46:0/1391767414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c0af230 con 0x7fa17c0732c0 2024-04-26T23:55:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa17b400640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c7f0 tx=0x7fa16400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164015070 con 0x7fa17c0732c0 2024-04-26T23:55:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164007a30 con 0x7fa17c0732c0 2024-04-26T23:55:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164007e00 con 0x7fa17c0732c0 2024-04-26T23:55:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c0af510 con 0x7fa17c0732c0 2024-04-26T23:55:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.483+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c0b0b30 con 0x7fa17c0732c0 2024-04-26T23:55:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.484+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c069530 con 0x7fa17c0732c0 2024-04-26T23:55:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.485+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa164018070 con 0x7fa17c0732c0 2024-04-26T23:55:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.486+0000 7fa173e00640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.486+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fa16408bd90 con 0x7fa17c0732c0 2024-04-26T23:55:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.487+0000 7fa17be00640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:33.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.488+0000 7fa17be00640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 0x7fa15806edb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006fd0 tx=0x7fa16c006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.491+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa164059f90 con 0x7fa17c0732c0 2024-04-26T23:55:33.607 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:55:33.605+0000 7f7b3c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:55:33.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.736+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa17c0cf760 con 0x7fa17c0732c0 2024-04-26T23:55:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.738+0000 7fa173e00640 1 -- 172.21.15.46:0/1391767414 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7fa164059890 con 0x7fa17c0732c0 2024-04-26T23:55:33.740 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:33.740 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:55:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:55:33.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.740+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 msgr2=0x7fa15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.740+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 0x7fa15806edb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006fd0 tx=0x7fa16c006bc0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.740+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 msgr2=0x7fa17c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.740+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c7f0 tx=0x7fa16400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 shutdown_connections 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa17c160bd0 0x7fa17c141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa17c0732c0 0x7fa17c0ae8e0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 --2- 172.21.15.46:0/1391767414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa17c06e510 0x7fa17c1f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 >> 172.21.15.46:0/1391767414 conn(0x7fa17c06c530 msgr2=0x7fa17c0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 shutdown_connections 2024-04-26T23:55:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:33.741+0000 7fa183000640 1 -- 172.21.15.46:0/1391767414 wait complete. 2024-04-26T23:55:34.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 149, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:55:27.117121+0000', 'modified': '2024-04-26T23:55:30.196334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11247': {'gid': 11247, 'name': 'c', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/2117263293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2117263293}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2117263293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11247, 'qdb_cluster': [11247]} 2024-04-26T23:55:34.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:55:34.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:55:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.382+0000 7fce53200640 1 Processor -- start 2024-04-26T23:55:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.382+0000 7fce53200640 1 -- start start 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce5415f4e0 0x7fce540d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce540b4000 0x7fce540d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce540d1f60 0x7fce540cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fce540d2990 con 0x7fce540b4000 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fce540d11f0 con 0x7fce5415f4e0 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce53200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce540d5f20 con 0x7fce540d1f60 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce4be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce540b4000 0x7fce540d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce4be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce540b4000 0x7fce540d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58504/0 (socket says 172.21.15.46:58504) 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.384+0000 7fce4be00640 1 -- 172.21.15.46:0/1265132486 learned_addr learned my addr 172.21.15.46:0/1265132486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.385+0000 7fce4be00640 1 -- 172.21.15.46:0/1265132486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce540d1f60 msgr2=0x7fce540cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.385+0000 7fce4be00640 1 --2- 172.21.15.46:0/1265132486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce540d1f60 0x7fce540cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.385+0000 7fce4be00640 1 -- 172.21.15.46:0/1265132486 >> v2:172.21.15.142:3300/0 conn(0x7fce5415f4e0 msgr2=0x7fce540d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.385+0000 7fce4be00640 1 --2- 172.21.15.46:0/1265132486 >> v2:172.21.15.142:3300/0 conn(0x7fce5415f4e0 0x7fce540d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.385+0000 7fce4be00640 1 -- 172.21.15.46:0/1265132486 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce540d0f10 con 0x7fce540b4000 2024-04-26T23:55:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.386+0000 7fce4be00640 1 --2- 172.21.15.46:0/1265132486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce540d1a20 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fce5408d8b0 tx=0x7fce3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655649bff2b041b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.386+0000 7fce51400640 1 -- 172.21.15.46:0/1265132486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c037070 con 0x7fce540b4000 2024-04-26T23:55:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.386+0000 7fce51400640 1 -- 172.21.15.46:0/1265132486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c032440 con 0x7fce540b4000 2024-04-26T23:55:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.386+0000 7fce51400640 1 -- 172.21.15.46:0/1265132486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c032810 con 0x7fce540b4000 2024-04-26T23:55:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 -- 172.21.15.46:0/1265132486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 msgr2=0x7fce540d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 --2- 172.21.15.46:0/1265132486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce540d1a20 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fce5408d8b0 tx=0x7fce3c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 -- 172.21.15.46:0/1265132486 shutdown_connections 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 --2- 172.21.15.46:0/1265132486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce540d1f60 0x7fce540cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 --2- 172.21.15.46:0/1265132486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce540d1a20 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 --2- 172.21.15.46:0/1265132486 >> v2:172.21.15.142:3300/0 conn(0x7fce5415f4e0 0x7fce540d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 -- 172.21.15.46:0/1265132486 >> 172.21.15.46:0/1265132486 conn(0x7fce540d5010 msgr2=0x7fce540d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 -- 172.21.15.46:0/1265132486 shutdown_connections 2024-04-26T23:55:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.387+0000 7fce53200640 1 -- 172.21.15.46:0/1265132486 wait complete. 2024-04-26T23:55:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.389+0000 7fce53200640 1 Processor -- start 2024-04-26T23:55:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.390+0000 7fce53200640 1 -- start start 2024-04-26T23:55:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 0x7fce54161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 0x7fce54150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce5415f8c0 con 0x7fce540b4000 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce540b1d60 con 0x7fce540d1f60 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce53200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce540d0210 con 0x7fce5415f4e0 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58514/0 (socket says 172.21.15.46:58514) 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce51e00640 1 -- 172.21.15.46:0/793180673 learned_addr learned my addr 172.21.15.46:0/793180673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.391+0000 7fce4be00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 0x7fce54161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 -- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 msgr2=0x7fce54150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce52800640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 0x7fce54150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 0x7fce54150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 -- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 msgr2=0x7fce54161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 0x7fce54161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 -- 172.21.15.46:0/793180673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce540cc330 con 0x7fce540b4000 2024-04-26T23:55:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce52800640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 0x7fce54150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce4be00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 0x7fce54161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce51e00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b410 tx=0x7fce4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce4a000640 1 -- 172.21.15.46:0/793180673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44015020 con 0x7fce540b4000 2024-04-26T23:55:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce4a000640 1 -- 172.21.15.46:0/793180673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce44011850 con 0x7fce540b4000 2024-04-26T23:55:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce4a000640 1 -- 172.21.15.46:0/793180673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44011c40 con 0x7fce540b4000 2024-04-26T23:55:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce540cc610 con 0x7fce540b4000 2024-04-26T23:55:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.392+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce540ccbd0 con 0x7fce540b4000 2024-04-26T23:55:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.395+0000 7fce4a000640 1 -- 172.21.15.46:0/793180673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce44018020 con 0x7fce540b4000 2024-04-26T23:55:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.396+0000 7fce4a000640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 0x7fce3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.396+0000 7fce5b9f3740 1 Processor -- start 2024-04-26T23:55:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.396+0000 7fce5b9f3740 1 -- start start 2024-04-26T23:55:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.396+0000 7fce4a000640 1 -- 172.21.15.46:0/793180673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fce4408bc80 con 0x7fce540b4000 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce4be00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 0x7fce3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ecb7b0 0x5582e6ecd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6f07e80 0x5582e6ecbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5582e6e2f600 con 0x5582e6ef6270 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5582e6dcc100 con 0x5582e6ecb7b0 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5582e6e2b5c0 con 0x5582e6f07e80 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58528/0 (socket says 172.21.15.46:58528) 2024-04-26T23:55:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.397+0000 7fce52800640 1 -- 172.21.15.46:0/633853194 learned_addr learned my addr 172.21.15.46:0/633853194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 -- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6f07e80 msgr2=0x5582e6ecbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6f07e80 0x5582e6ecbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 -- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ecb7b0 msgr2=0x5582e6ecd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ecb7b0 0x5582e6ecd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 -- 172.21.15.46:0/633853194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e6b412d0 con 0x5582e6ef6270 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce52800640 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007f60 tx=0x7fce4c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b0152513bac4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.398+0000 7fce4be00640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 0x7fce3006edb0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fce3c034000 tx=0x7fce3c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce42600640 1 -- 172.21.15.46:0/633853194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c014070 con 0x5582e6ef6270 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce42600640 1 -- 172.21.15.46:0/633853194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c00f850 con 0x5582e6ef6270 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce42600640 1 -- 172.21.15.46:0/633853194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00fc20 con 0x5582e6ef6270 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 -- 172.21.15.46:0/633853194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 msgr2=0x5582e6ef66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007f60 tx=0x7fce4c007f90 comp rx=0 tx=0).stop 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 -- 172.21.15.46:0/633853194 shutdown_connections 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ef6270 0x5582e6ef66f0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6f07e80 0x5582e6ecbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/633853194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ecb7b0 0x5582e6ecd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 -- 172.21.15.46:0/633853194 >> 172.21.15.46:0/633853194 conn(0x5582e6e5fd10 msgr2=0x5582e6f11270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 -- 172.21.15.46:0/633853194 shutdown_connections 2024-04-26T23:55:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.399+0000 7fce5b9f3740 1 -- 172.21.15.46:0/633853194 wait complete. 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.400+0000 7fce5b9f3740 1 -- start start 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.400+0000 7fce5b9f3740 1 Processor -- start 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f09940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6ee33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5582e6db6f70 con 0x5582e6f07e80 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5582e6ef3dc0 con 0x5582e6ef6270 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5582e6ee40c0 con 0x5582e6ecb7b0 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce5b9f3740 10 client.0 _ll_register_callbacks cb 0x5582e6e25480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.401+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58530/0 (socket says 172.21.15.46:58530) 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 -- 172.21.15.46:0/1165312824 learned_addr learned my addr 172.21.15.46:0/1165312824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce51e00640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f09940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce4be00640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6ee33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 -- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 msgr2=0x5582e6f09940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f09940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 -- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 msgr2=0x5582e6ee33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6ee33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 -- 172.21.15.46:0/1165312824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e6c0b6b0 con 0x5582e6f07e80 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce51e00640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f09940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce52800640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000f80 tx=0x7fce4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5342f18552f27d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.402+0000 7fce36a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c029060 con 0x5582e6f07e80 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce4c00f850 con 0x5582e6f07e80 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fce4c024470 con 0x5582e6f07e80 2024-04-26T23:55:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fce4c02f050 con 0x5582e6f07e80 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 10 client.11382 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x5582e6ee3e00 con 0x5582e6f07e80 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c00f850 con 0x5582e6f07e80 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fce4c032020 con 0x5582e6f07e80 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 10 client.11382 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 10 client.11382 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 20 client.11382 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6f2cb80 0x5582e6f2ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 4 client.11382 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.403+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582e6e3d210 con 0x5582e6f2cb80 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.404+0000 7fce51e00640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6f2cb80 0x5582e6f2ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.404+0000 7fce51e00640 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6f2cb80 0x5582e6f2ef70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc4beac1c722813 server_cookie=90e786fd5d031df5 in_seq=0 out_seq=0 2024-04-26T23:55:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.404+0000 7fce36a00640 10 client.11382 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce36a00640 1 -- 172.21.15.46:0/1165312824 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582e6e3d210 con 0x5582e6f2cb80 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce36a00640 10 client.11382 handle_command_reply: tid=0 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 client.11382 shutdown 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 msgr2=0x5582e6f0a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000f80 tx=0x7fce4c004a40 comp rx=0 tx=0).stop 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 shutdown_connections 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6f2cb80 0x5582e6f2ef70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.405+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f0a8f0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6ee33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/1165312824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f09940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce36a00640 0 client.11382 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 >> 172.21.15.46:0/1165312824 conn(0x5582e6e5fd10 msgr2=0x5582e6f146f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 shutdown_connections 2024-04-26T23:55:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 1 -- 172.21.15.46:0/1165312824 wait complete. 2024-04-26T23:55:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.406+0000 7fce5b9f3740 20 client.11382 trim_cache size 0 max 16384 2024-04-26T23:55:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.426+0000 7fce5b9f3740 1 Processor -- start 2024-04-26T23:55:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.426+0000 7fce5b9f3740 1 -- start start 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ef6270 0x5582e6ef9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6f07e80 0x5582e6f148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce54200ee0 con 0x5582e6ecb7b0 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce54160800 con 0x5582e6f07e80 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fce540cdd40 con 0x5582e6ef6270 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58540/0 (socket says 172.21.15.46:58540) 2024-04-26T23:55:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 -- 172.21.15.46:0/2846088870 learned_addr learned my addr 172.21.15.46:0/2846088870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 -- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ef6270 msgr2=0x5582e6ef9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ef6270 0x5582e6ef9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 -- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6f07e80 msgr2=0x5582e6f148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.427+0000 7fce51e00640 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6f07e80 0x5582e6f148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.428+0000 7fce51e00640 1 -- 172.21.15.46:0/2846088870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e6ef5860 con 0x5582e6ecb7b0 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.428+0000 7fce51e00640 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fce44055c70 tx=0x7fce440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68afad9428dcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.428+0000 7fce41c00640 1 -- 172.21.15.46:0/2846088870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44087070 con 0x5582e6ecb7b0 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.428+0000 7fce41c00640 1 -- 172.21.15.46:0/2846088870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4408a090 con 0x5582e6ecb7b0 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.428+0000 7fce41c00640 1 -- 172.21.15.46:0/2846088870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44060040 con 0x5582e6ecb7b0 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2846088870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 msgr2=0x5582e6f20b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fce44055c70 tx=0x7fce440046c0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2846088870 shutdown_connections 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6f07e80 0x5582e6f148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ef6270 0x5582e6ef9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2846088870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6ecb7b0 0x5582e6f20b00 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2846088870 >> 172.21.15.46:0/2846088870 conn(0x5582e6e5fd10 msgr2=0x5582e6ef0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2846088870 shutdown_connections 2024-04-26T23:55:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2846088870 wait complete. 2024-04-26T23:55:34.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.429+0000 7fce5b9f3740 1 -- start start 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.430+0000 7fce5b9f3740 1 Processor -- start 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f8a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6f1fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5582e6f082e0 con 0x5582e6f07e80 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5582e6ef1100 con 0x5582e6ef6270 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5582e6ee2d00 con 0x5582e6ecb7b0 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce5b9f3740 10 client.0 _ll_register_callbacks cb 0x7fce4c007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58550/0 (socket says 172.21.15.46:58550) 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 -- 172.21.15.46:0/2840714533 learned_addr learned my addr 172.21.15.46:0/2840714533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce4be00640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6f1fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce51e00640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f8a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 -- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 msgr2=0x5582e6f8a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f8a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 -- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 msgr2=0x5582e6f1fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6f1fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.431+0000 7fce52800640 1 -- 172.21.15.46:0/2840714533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e6eca310 con 0x5582e6f07e80 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce52800640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0078f0 tx=0x7fce4c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4de7c21820835 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce36a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c038060 con 0x5582e6f07e80 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fce4c024470 con 0x5582e6f07e80 2024-04-26T23:55:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce4408a090 con 0x5582e6f07e80 2024-04-26T23:55:34.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fce4c03e050 con 0x5582e6f07e80 2024-04-26T23:55:34.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce5b9f3740 10 client.11388 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:34.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.432+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x5582e6f32f90 con 0x5582e6f07e80 2024-04-26T23:55:34.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4408a090 con 0x5582e6f07e80 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fce4c041020 con 0x5582e6f07e80 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 10 client.11388 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 10 client.11388 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 20 client.11388 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6ecb070 0x5582e6ee00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 4 client.11388 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "status"}) -- 0x5582e6e37080 con 0x5582e6ecb070 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.433+0000 7fce51e00640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6ecb070 0x5582e6ee00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce51e00640 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6ecb070 0x5582e6ee00d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60545f970552e73 server_cookie=b22aafc4d84b61e4 in_seq=0 out_seq=0 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce36a00640 10 client.11388 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce36a00640 1 -- 172.21.15.46:0/2840714533 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x7fce44076160 con 0x5582e6ecb070 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce36a00640 10 client.11388 handle_command_reply: tid=0 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 client.11388 shutdown 2024-04-26T23:55:34.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 msgr2=0x5582e6f1fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0078f0 tx=0x7fce4c004b50 comp rx=0 tx=0).stop 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 shutdown_connections 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x5582e6ecb070 0x5582e6ee00d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582e6f07e80 0x5582e6f1fff0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e6ef6270 0x5582e6f1fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce36a00640 0 client.11388 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 --2- 172.21.15.46:0/2840714533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5582e6ecb7b0 0x5582e6f8a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 >> 172.21.15.46:0/2840714533 conn(0x5582e6e5fd10 msgr2=0x5582e6ec1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 shutdown_connections 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 1 -- 172.21.15.46:0/2840714533 wait complete. 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.434+0000 7fce5b9f3740 20 client.11388 trim_cache size 0 max 16384 2024-04-26T23:55:34.439 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 11247, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7659684020000004, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 149, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 186, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 186, 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.177170746 2024-04-26T23:55:34.440 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 msgr2=0x7fce3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 0x7fce3006edb0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fce3c034000 tx=0x7fce3c02c000 comp rx=0 tx=0).stop 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 msgr2=0x7fce54150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b410 tx=0x7fce4400b8d0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 shutdown_connections 2024-04-26T23:55:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fce3006c8f0 0x7fce3006edb0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce5415f4e0 0x7fce54150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce540d1f60 0x7fce54161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 --2- 172.21.15.46:0/793180673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce540b4000 0x7fce54150d90 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 >> 172.21.15.46:0/793180673 conn(0x7fce540d5010 msgr2=0x7fce5415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 shutdown_connections 2024-04-26T23:55:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.435+0000 7fce53200640 1 -- 172.21.15.46:0/793180673 wait complete. 2024-04-26T23:55:34.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:55:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.646+0000 7fb647e00640 1 Processor -- start 2024-04-26T23:55:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.646+0000 7fb647e00640 1 -- start start 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.647+0000 7fb647e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb648073db0 0x7fb648072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.647+0000 7fb647e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64806f1d0 con 0x7fb6480cff40 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.647+0000 7fb647e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6480741c0 con 0x7fb6480cf450 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.647+0000 7fb647e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb6480726d0 con 0x7fb648073db0 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.648+0000 7fb646000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb6480cff40 0x7fb648164af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58430/0 (socket says 172.21.15.46:58430) 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.648+0000 7fb646000640 1 -- 172.21.15.46:0/1233061066 learned_addr learned my addr 172.21.15.46:0/1233061066 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.648+0000 7fb647400640 1 --2- 172.21.15.46:0/1233061066 >> v2:172.21.15.46:3300/0 conn(0x7fb648073db0 0x7fb648072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.648+0000 7fb645600640 1 -- 172.21.15.46:0/1233061066 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388723521 0 0) 0x7fb64806f1d0 con 0x7fb6480cff40 2024-04-26T23:55:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.649+0000 7fb645600640 1 -- 172.21.15.46:0/1233061066 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb62c003520 con 0x7fb6480cff40 2024-04-26T23:55:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.649+0000 7fb647400640 1 -- 172.21.15.46:0/1233061066 >> v1:172.21.15.142:6789/0 conn(0x7fb6480cf450 legacy=0x7fb6480cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.649+0000 7fb647400640 1 -- 172.21.15.46:0/1233061066 >> v1:172.21.15.46:6789/0 conn(0x7fb6480cff40 legacy=0x7fb648164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.649+0000 7fb647400640 1 -- 172.21.15.46:0/1233061066 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64806f470 con 0x7fb648073db0 2024-04-26T23:55:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.650+0000 7fb647400640 1 --2- 172.21.15.46:0/1233061066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb648073db0 0x7fb648072190 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb6400097a0 tx=0x7fb64002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cc6510cece59f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb645600640 1 -- 172.21.15.46:0/1233061066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640037070 con 0x7fb648073db0 2024-04-26T23:55:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb645600640 1 -- 172.21.15.46:0/1233061066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640032440 con 0x7fb648073db0 2024-04-26T23:55:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb645600640 1 -- 172.21.15.46:0/1233061066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640032810 con 0x7fb648073db0 2024-04-26T23:55:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb647e00640 1 -- 172.21.15.46:0/1233061066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb648073db0 msgr2=0x7fb648072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb647e00640 1 --2- 172.21.15.46:0/1233061066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb648073db0 0x7fb648072190 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb6400097a0 tx=0x7fb64002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb647e00640 1 -- 172.21.15.46:0/1233061066 shutdown_connections 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb647e00640 1 --2- 172.21.15.46:0/1233061066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb648073db0 0x7fb648072190 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.651+0000 7fb647e00640 1 -- 172.21.15.46:0/1233061066 >> 172.21.15.46:0/1233061066 conn(0x7fb6480b2820 msgr2=0x7fb6480b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.652+0000 7fb647e00640 1 -- 172.21.15.46:0/1233061066 shutdown_connections 2024-04-26T23:55:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.652+0000 7fb647e00640 1 -- 172.21.15.46:0/1233061066 wait complete. 2024-04-26T23:55:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.653+0000 7fb647e00640 1 Processor -- start 2024-04-26T23:55:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.653+0000 7fb647e00640 1 -- start start 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb648073db0 0x7fb648140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6480cff40 0x7fb6481dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb648072ef0 con 0x7fb6480cf450 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb648072c10 con 0x7fb6480cff40 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.654+0000 7fb647e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6480731d0 con 0x7fb648073db0 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58572/0 (socket says 172.21.15.46:58572) 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 -- 172.21.15.46:0/1036389835 learned_addr learned my addr 172.21.15.46:0/1036389835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646a00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb648073db0 0x7fb648140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb647400640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6480cff40 0x7fb6481dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 -- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb648073db0 msgr2=0x7fb648140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb648073db0 0x7fb648140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 -- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6480cff40 msgr2=0x7fb6481dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6480cff40 0x7fb6481dc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.655+0000 7fb646000640 1 -- 172.21.15.46:0/1036389835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64813f910 con 0x7fb6480cf450 2024-04-26T23:55:34.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb646000640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c530 tx=0x7fb63000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630017070 con 0x7fb6480cf450 2024-04-26T23:55:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb630005ce0 con 0x7fb6480cf450 2024-04-26T23:55:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300072b0 con 0x7fb6480cf450 2024-04-26T23:55:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6481dc800 con 0x7fb6480cf450 2024-04-26T23:55:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.656+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6481dcdc0 con 0x7fb6480cf450 2024-04-26T23:55:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.657+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6480693b0 con 0x7fb6480cf450 2024-04-26T23:55:34.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.658+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63001a070 con 0x7fb6480cf450 2024-04-26T23:55:34.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.659+0000 7fb63ea00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.660+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fb63008b1b0 con 0x7fb6480cf450 2024-04-26T23:55:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.660+0000 7fb646a00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.662+0000 7fb646a00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 0x7fb62406edb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb6480d1870 tx=0x7fb638009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.665+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb630059470 con 0x7fb6480cf450 2024-04-26T23:55:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.914+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6481dd160 con 0x7fb6480cf450 2024-04-26T23:55:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.915+0000 7fb63ea00640 1 -- 172.21.15.46:0/1036389835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v149) ==== 53+0+73 (secure 0 0 0) 0x7fb630058d70 con 0x7fb6480cf450 2024-04-26T23:55:34.917 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.917+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 msgr2=0x7fb62406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.917+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 0x7fb62406edb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb6480d1870 tx=0x7fb638009040 comp rx=0 tx=0).stop 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.917+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 msgr2=0x7fb6481dbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.917+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c530 tx=0x7fb63000c9f0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 shutdown_connections 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6480cff40 0x7fb6481dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6480cf450 0x7fb6481dbc90 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 --2- 172.21.15.46:0/1036389835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb648073db0 0x7fb648140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 >> 172.21.15.46:0/1036389835 conn(0x7fb6480b2820 msgr2=0x7fb6481e3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 shutdown_connections 2024-04-26T23:55:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:34.918+0000 7fb647e00640 1 -- 172.21.15.46:0/1036389835 wait complete. 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:55:34.940 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:55:34.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:34.941 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:55:34.991 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41486sec preferred_lft 41486sec 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:55:34.992 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:55:34.993 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:55:34.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:34.994 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:55:35.075 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:55:35.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:35.079 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:55:35.135 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:55:35.135 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:55:35.135 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:55:35.135 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:55:35.135 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:55:35.136 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:55:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:55:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:55:35.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:35.364 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:55:35.415 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:55:35.415 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:55:35.416 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:55:35.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:35.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:55:35.504 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:55:35.504 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:55:35.564 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:55:35.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:55:35.565 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:55:35.618 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:55:35.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:35.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:55:35.753 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:55:35.753 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:55:35.754 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:55:35.775 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:55:35.775 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:55:35.776 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:55:35.776 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:55:35.776 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:55:35.802 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:55:35.803 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:55:35.803 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:55:35.803 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:55:35.803 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:55:35.803 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:55:35.804 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:55:35.804 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:55:35.804 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:55:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:55:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:55:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:55:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:55:35.808 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:55:35.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:35.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:55:35.883 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:55:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:55:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:55:36.014 INFO:teuthology.orchestra.run.smithi046.stdout:client7891 2024-04-26T23:55:36.015 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:36.015 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:36.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:36.068 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-26T23:55:36.133 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:55:36.133 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:55:36.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:36.192 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-26T23:55:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-26T23:55:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-26T23:55:36.369 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.120645 s, 52.1 MB/s 2024-04-26T23:55:36.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:55:36.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-26T23:55:36.519 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:55:36.520 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.521 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-04-26T23:55:36.522 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.523 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-26T23:55:36.524 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.525 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-26T23:55:36.526 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-04-26T23:55:36.527 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.528 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: mds cache max size = 1000 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: mds log max segments = 1 2024-04-26T23:55:36.529 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = False 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-26T23:55:36.530 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:55:36.531 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:55:36.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.533 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.534 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-26T23:55:36.535 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.536 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.537 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-26T23:55:36.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-26T23:55:36.539 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds cache max size = 1000 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds log max segments = 1 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.540 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:55:36.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:55:36.700 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.701 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:55:36.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-04-26T23:55:36.703 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-26T23:55:36.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:55:36.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-26T23:55:36.706 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-26T23:55:36.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:55:36.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:55:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: mds cache max size = 1000 2024-04-26T23:55:36.711 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = False 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds log max segments = 1 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = False 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-26T23:55:36.712 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-26T23:55:36.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:55:36.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-26T23:55:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:55:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-26T23:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-26T23:55:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:55:36.722 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds cache max size = 1000 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds log max segments = 1 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:55:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:55:36.747 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-04-26T23:55:36.748 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:55:36.895 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mode": 33188, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ino": 1099511627777, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_dev": 41, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_nlink": 1, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_uid": 1000, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_gid": 1316, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_size": 6291456, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_atime": 1714175736.244109, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mtime": 1714175736.2861085, 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ctime": 1714175736.2861085 2024-04-26T23:55:36.896 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:36.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:55:36.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:55:37.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.168+0000 7f3587600640 1 Processor -- start 2024-04-26T23:55:37.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.168+0000 7f3587600640 1 -- start start 2024-04-26T23:55:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3587600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 0x7f35800d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3587600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35800d2a70 con 0x7f35800d5890 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3587600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35800d2f60 con 0x7f35800b3fb0 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3587600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f358015f8e0 con 0x7f358015f500 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3585600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f35800d5890 0x7f35800b4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58436/0 (socket says 172.21.15.46:58436) 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.170+0000 7f3585600640 1 -- 172.21.15.46:0/931741405 learned_addr learned my addr 172.21.15.46:0/931741405 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f3584c00640 1 --2- 172.21.15.46:0/931741405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 0x7f35800d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f357f400640 1 -- 172.21.15.46:0/931741405 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514091148 0 0) 0x7f35800d2a70 con 0x7f35800d5890 2024-04-26T23:55:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f357f400640 1 -- 172.21.15.46:0/931741405 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3564003520 con 0x7f35800d5890 2024-04-26T23:55:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f3584c00640 1 -- 172.21.15.46:0/931741405 >> v1:172.21.15.142:6789/0 conn(0x7f35800b3fb0 legacy=0x7f35800d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f3584c00640 1 -- 172.21.15.46:0/931741405 >> v1:172.21.15.46:6789/0 conn(0x7f35800d5890 legacy=0x7f35800b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.171+0000 7f3584c00640 1 -- 172.21.15.46:0/931741405 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35800d3240 con 0x7f358015f500 2024-04-26T23:55:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.173+0000 7f3584c00640 1 --2- 172.21.15.46:0/931741405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 0x7f35800d2530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35700097d0 tx=0x7f357002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f810ff366b1246 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.173+0000 7f357f400640 1 -- 172.21.15.46:0/931741405 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570037070 con 0x7f358015f500 2024-04-26T23:55:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.173+0000 7f357f400640 1 -- 172.21.15.46:0/931741405 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3570032440 con 0x7f358015f500 2024-04-26T23:55:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.173+0000 7f357f400640 1 -- 172.21.15.46:0/931741405 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570032810 con 0x7f358015f500 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 -- 172.21.15.46:0/931741405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 msgr2=0x7f35800d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 --2- 172.21.15.46:0/931741405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 0x7f35800d2530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35700097d0 tx=0x7f357002a210 comp rx=0 tx=0).stop 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 -- 172.21.15.46:0/931741405 shutdown_connections 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 --2- 172.21.15.46:0/931741405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f358015f500 0x7f35800d2530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 -- 172.21.15.46:0/931741405 >> 172.21.15.46:0/931741405 conn(0x7f3580074400 msgr2=0x7f358009e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 -- 172.21.15.46:0/931741405 shutdown_connections 2024-04-26T23:55:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.174+0000 7f3587600640 1 -- 172.21.15.46:0/931741405 wait complete. 2024-04-26T23:55:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.176+0000 7f3587600640 1 Processor -- start 2024-04-26T23:55:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.176+0000 7f3587600640 1 -- start start 2024-04-26T23:55:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35800b3fb0 0x7f35800b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35800d5890 0x7f358015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35800d3ae0 con 0x7f358015f500 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f35800d3800 con 0x7f35800b3fb0 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.177+0000 7f3587600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f35800d4000 con 0x7f35800d5890 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58584/0 (socket says 172.21.15.46:58584) 2024-04-26T23:55:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 -- 172.21.15.46:0/714842665 learned_addr learned my addr 172.21.15.46:0/714842665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3584c00640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35800b3fb0 0x7f35800b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f357fe00640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35800d5890 0x7f358015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 -- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35800d5890 msgr2=0x7f358015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35800d5890 0x7f358015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 -- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35800b3fb0 msgr2=0x7f35800b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35800b3fb0 0x7f35800b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.178+0000 7f3585600640 1 -- 172.21.15.46:0/714842665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35800ba160 con 0x7f358015f500 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f3585600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f357800cae0 tx=0x7f357800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578017070 con 0x7f358015f500 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357800ac40 con 0x7f358015f500 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578012640 con 0x7f358015f500 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35800c8f70 con 0x7f358015f500 2024-04-26T23:55:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.179+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358015c050 con 0x7f358015f500 2024-04-26T23:55:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.181+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580069530 con 0x7f358015f500 2024-04-26T23:55:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.182+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f357801a070 con 0x7f358015f500 2024-04-26T23:55:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.183+0000 7f357cc00640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 0x7f356006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.183+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7f357808ba90 con 0x7f358015f500 2024-04-26T23:55:37.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.183+0000 7f3584c00640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 0x7f356006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.185+0000 7f3584c00640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 0x7f356006edb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3570034000 tx=0x7f357002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.188+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3578059d20 con 0x7f358015f500 2024-04-26T23:55:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.435+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35800cf520 con 0x7f358015f500 2024-04-26T23:55:37.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.437+0000 7f357cc00640 1 -- 172.21.15.46:0/714842665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f357808b020 con 0x7f358015f500 2024-04-26T23:55:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:55:37.439 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:55:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:55:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.439+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 msgr2=0x7f356006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.439+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 0x7f356006edb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3570034000 tx=0x7f357002e000 comp rx=0 tx=0).stop 2024-04-26T23:55:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 msgr2=0x7f35800cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f357800cae0 tx=0x7f357800a060 comp rx=0 tx=0).stop 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 shutdown_connections 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f356006c8f0 0x7f356006edb0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f358015f500 0x7f35800cade0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35800d5890 0x7f358015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 --2- 172.21.15.46:0/714842665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35800b3fb0 0x7f35800b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 >> 172.21.15.46:0/714842665 conn(0x7f3580074400 msgr2=0x7f35800d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 shutdown_connections 2024-04-26T23:55:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.440+0000 7f3587600640 1 -- 172.21.15.46:0/714842665 wait complete. 2024-04-26T23:55:37.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T23:55:37.711 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-26T23:55:37.712 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T23:55:37.712 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-26T23:55:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.811+0000 7f4543e34f40 1 Processor -- start 2024-04-26T23:55:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.811+0000 7f4543e34f40 1 -- start start 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393d215c0 0x561393db27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393db0b40 0x561393daf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561393c80d20 con 0x561393d255c0 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561393cd72f0 con 0x561393db0b40 2024-04-26T23:55:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.812+0000 7f4543e34f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561393db0070 con 0x561393d215c0 2024-04-26T23:55:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58600/0 (socket says 172.21.15.46:58600) 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 -- 172.21.15.46:0/1599398700 learned_addr learned my addr 172.21.15.46:0/1599398700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4541a00640 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393db0b40 0x561393daf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4541000640 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393d215c0 0x561393db27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 -- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393d215c0 msgr2=0x561393db27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393d215c0 0x561393db27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 -- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393db0b40 msgr2=0x561393daf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393db0b40 0x561393daf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.813+0000 7f4540600640 1 -- 172.21.15.46:0/1599398700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561393db02c0 con 0x561393d255c0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f4540600640 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098b0 tx=0x7f452c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6255386f0884ca32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f453f200640 1 -- 172.21.15.46:0/1599398700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c008930 con 0x561393d255c0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f453f200640 1 -- 172.21.15.46:0/1599398700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c018070 con 0x561393d255c0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f453f200640 1 -- 172.21.15.46:0/1599398700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c008cd0 con 0x561393d255c0 2024-04-26T23:55:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f4543e34f40 1 -- 172.21.15.46:0/1599398700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 msgr2=0x561393daef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f4543e34f40 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098b0 tx=0x7f452c005a00 comp rx=0 tx=0).stop 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f4543e34f40 1 -- 172.21.15.46:0/1599398700 shutdown_connections 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.814+0000 7f4543e34f40 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393db0b40 0x561393daf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d255c0 0x561393daef60 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 --2- 172.21.15.46:0/1599398700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393d215c0 0x561393db27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 -- 172.21.15.46:0/1599398700 >> 172.21.15.46:0/1599398700 conn(0x561393cdea20 msgr2=0x561393d0c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 -- 172.21.15.46:0/1599398700 shutdown_connections 2024-04-26T23:55:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 -- 172.21.15.46:0/1599398700 wait complete. 2024-04-26T23:55:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.815+0000 7f4543e34f40 1 Processor -- start 2024-04-26T23:55:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.816+0000 7f4543e34f40 1 -- start start 2024-04-26T23:55:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.816+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 0x561393daa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393db0b40 0x561393da3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4543e34f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393daaf60 0x561393d0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4543e34f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561393dae760 con 0x561393d215c0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4543e34f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561393daaa20 con 0x561393daaf60 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4543e34f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561393daea40 con 0x561393db0b40 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 0x561393daa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 0x561393daa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58612/0 (socket says 172.21.15.46:58612) 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 -- 172.21.15.46:0/3121029858 learned_addr learned my addr 172.21.15.46:0/3121029858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541a00640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393daaf60 0x561393d0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4540600640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393db0b40 0x561393da3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 -- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393db0b40 msgr2=0x561393da3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561393db0b40 0x561393da3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 -- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393daaf60 msgr2=0x561393d0ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561393daaf60 0x561393d0ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.817+0000 7f4541000640 1 -- 172.21.15.46:0/3121029858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561393e476c0 con 0x561393d215c0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f4541000640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 0x561393daa1b0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f4530007000 tx=0x7f4530003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f453de00640 1 -- 172.21.15.46:0/3121029858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530005a80 con 0x561393d215c0 2024-04-26T23:55:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f453de00640 1 -- 172.21.15.46:0/3121029858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530054e10 con 0x561393d215c0 2024-04-26T23:55:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f453de00640 1 -- 172.21.15.46:0/3121029858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45300696c0 con 0x561393d215c0 2024-04-26T23:55:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561393e479a0 con 0x561393d215c0 2024-04-26T23:55:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.818+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561393e47f60 con 0x561393d215c0 2024-04-26T23:55:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.820+0000 7f453de00640 1 -- 172.21.15.46:0/3121029858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4530069860 con 0x561393d215c0 2024-04-26T23:55:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.821+0000 7f453de00640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f452006b300 0x7f452006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.821+0000 7f453de00640 1 -- 172.21.15.46:0/3121029858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7f45300cf020 con 0x561393d215c0 2024-04-26T23:55:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.822+0000 7f4543e34f40 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x561393dad0b0 0x561393dad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.822+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- osd_op(unknown.0.0:1 28.7 28:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186) -- 0x561393dada50 con 0x561393dad0b0 2024-04-26T23:55:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.822+0000 7f4541a00640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x561393dad0b0 0x561393dad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.822+0000 7f4540600640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f452006b300 0x7f452006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.823+0000 7f4541a00640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x561393dad0b0 0x561393dad510 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.823+0000 7f4540600640 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f452006b300 0x7f452006d7c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x561393d08bd0 tx=0x7f452c012000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4541a00640 1 -- 172.21.15.46:0/3121029858 <== osd.1 v2:172.21.15.46:6816/3879162742 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v186'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4538009040 con 0x561393dad0b0 2024-04-26T23:55:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x561393dad0b0 msgr2=0x561393dad510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x561393dad0b0 0x561393dad510 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f452006b300 msgr2=0x7f452006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f452006b300 0x7f452006d7c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x561393d08bd0 tx=0x7f452c012000 comp rx=0 tx=0).stop 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 msgr2=0x561393daa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 --2- 172.21.15.46:0/3121029858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561393d215c0 0x561393daa1b0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f4530007000 tx=0x7f4530003330 comp rx=0 tx=0).stop 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4541a00640 1 -- 172.21.15.46:0/3121029858 reap_dead start 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 shutdown_connections 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 >> 172.21.15.46:0/3121029858 conn(0x561393cdea20 msgr2=0x561393db1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 shutdown_connections 2024-04-26T23:55:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:37.826+0000 7f4543e34f40 1 -- 172.21.15.46:0/3121029858 wait complete. 2024-04-26T23:55:37.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:37.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub start / recursive,repair 2024-04-26T23:55:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.036+0000 7fdd9fe00640 1 Processor -- start 2024-04-26T23:55:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.036+0000 7fdd9fe00640 1 -- start start 2024-04-26T23:55:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdda01627f0 0x7fdda00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00d01a0 con 0x7fdda00d3cc0 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00d0480 con 0x7fdda00d2080 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00d0760 con 0x7fdda01627f0 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdda01627f0 0x7fdda00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdda01627f0 0x7fdda00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58626/0 (socket says 172.21.15.46:58626) 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.038+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3395907444 learned_addr learned my addr 172.21.15.46:0/3395907444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.039+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3395907444 >> v1:172.21.15.142:6789/0 conn(0x7fdda00d2080 legacy=0x7fdda00d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.039+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3395907444 >> v1:172.21.15.46:6789/0 conn(0x7fdda00d3cc0 legacy=0x7fdda00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.039+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3395907444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00d0a40 con 0x7fdda01627f0 2024-04-26T23:55:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.040+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/3395907444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda01627f0 0x7fdda00cfbc0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b7e331ca6aabd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.040+0000 7fdd9d600640 1 -- 172.21.15.46:0/3395907444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90037070 con 0x7fdda01627f0 2024-04-26T23:55:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.040+0000 7fdd9d600640 1 -- 172.21.15.46:0/3395907444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd90032440 con 0x7fdda01627f0 2024-04-26T23:55:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.040+0000 7fdd9d600640 1 -- 172.21.15.46:0/3395907444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90032810 con 0x7fdda01627f0 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 -- 172.21.15.46:0/3395907444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda01627f0 msgr2=0x7fdda00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/3395907444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda01627f0 0x7fdda00cfbc0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 -- 172.21.15.46:0/3395907444 shutdown_connections 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/3395907444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda01627f0 0x7fdda00cfbc0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 -- 172.21.15.46:0/3395907444 >> 172.21.15.46:0/3395907444 conn(0x7fdda009e380 msgr2=0x7fdda00d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 -- 172.21.15.46:0/3395907444 shutdown_connections 2024-04-26T23:55:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.041+0000 7fdd9fe00640 1 -- 172.21.15.46:0/3395907444 wait complete. 2024-04-26T23:55:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.043+0000 7fdd9fe00640 1 Processor -- start 2024-04-26T23:55:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.043+0000 7fdd9fe00640 1 -- start start 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 0x7fdda00cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda01627f0 0x7fdda00c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0162bd0 con 0x7fdda00d3cc0 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00d24f0 con 0x7fdda01627f0 2024-04-26T23:55:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdda00b4c60 con 0x7fdda00d2080 2024-04-26T23:55:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.044+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58628/0 (socket says 172.21.15.46:58628) 2024-04-26T23:55:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 -- 172.21.15.46:0/538368417 learned_addr learned my addr 172.21.15.46:0/538368417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 0x7fdda00cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 -- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 msgr2=0x7fdda00cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 0x7fdda00cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 -- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda01627f0 msgr2=0x7fdda00c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9f400640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda01627f0 0x7fdda00c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda01627f0 0x7fdda00c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 -- 172.21.15.46:0/538368417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00c8b40 con 0x7fdda00d3cc0 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 0x7fdda00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.045+0000 7fdd9e000640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c460 tx=0x7fdd8c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.046+0000 7fdd96a00640 1 -- 172.21.15.46:0/538368417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c016020 con 0x7fdda00d3cc0 2024-04-26T23:55:38.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.046+0000 7fdd96a00640 1 -- 172.21.15.46:0/538368417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c012870 con 0x7fdda00d3cc0 2024-04-26T23:55:38.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.046+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda015e070 con 0x7fdda00d3cc0 2024-04-26T23:55:38.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.046+0000 7fdd96a00640 1 -- 172.21.15.46:0/538368417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c012c60 con 0x7fdda00d3cc0 2024-04-26T23:55:38.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.046+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda015e630 con 0x7fdda00d3cc0 2024-04-26T23:55:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.048+0000 7fdd96a00640 1 -- 172.21.15.46:0/538368417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd8c019020 con 0x7fdda00d3cc0 2024-04-26T23:55:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.049+0000 7fdd96a00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 0x7fdd8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.049+0000 7fdd96a00640 1 -- 172.21.15.46:0/538368417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fdd8c08b770 con 0x7fdda00d3cc0 2024-04-26T23:55:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.050+0000 7fdda85b9740 1 Processor -- start 2024-04-26T23:55:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.050+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 0x7fdd8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.050+0000 7fdda85b9740 1 -- start start 2024-04-26T23:55:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a32820 0x558591a32c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55859196c980 con 0x558591a08500 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558591909220 con 0x558591a500c0 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558591968940 con 0x558591a32820 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 0x7fdd8006ede0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58640/0 (socket says 172.21.15.46:58640) 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9f400640 1 -- 172.21.15.46:0/1610326969 learned_addr learned my addr 172.21.15.46:0/1610326969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a32820 0x558591a32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.051+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 -- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a32820 msgr2=0x558591a32c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a32820 0x558591a32c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 -- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 msgr2=0x558591a50610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a50610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 -- 172.21.15.46:0/1610326969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585917467e0 con 0x558591a08500 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd9f400640 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdd98001cd0 tx=0x7fdd98002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173685d11c3cfae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd8b400640 1 -- 172.21.15.46:0/1610326969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98018070 con 0x558591a08500 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.052+0000 7fdd8b400640 1 -- 172.21.15.46:0/1610326969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98003da0 con 0x558591a08500 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdd8b400640 1 -- 172.21.15.46:0/1610326969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800dab0 con 0x558591a08500 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 -- 172.21.15.46:0/1610326969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 msgr2=0x558591a4e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdd98001cd0 tx=0x7fdd98002190 comp rx=0 tx=0).stop 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 -- 172.21.15.46:0/1610326969 shutdown_connections 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a08500 0x558591a4e780 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1610326969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a32820 0x558591a32c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 -- 172.21.15.46:0/1610326969 >> 172.21.15.46:0/1610326969 conn(0x558591a04e30 msgr2=0x558591a4d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 -- 172.21.15.46:0/1610326969 shutdown_connections 2024-04-26T23:55:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.053+0000 7fdda85b9740 1 -- 172.21.15.46:0/1610326969 wait complete. 2024-04-26T23:55:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.054+0000 7fdda85b9740 1 -- start start 2024-04-26T23:55:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.054+0000 7fdda85b9740 1 Processor -- start 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a2fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591a337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558591a092a0 con 0x558591a32820 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585918f4170 con 0x558591a500c0 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558591a02d60 con 0x558591a30140 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdda85b9740 10 client.0 _ll_register_callbacks cb 0x558591963770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.055+0000 7fdd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58642/0 (socket says 172.21.15.46:58642) 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3687152880 learned_addr learned my addr 172.21.15.46:0/3687152880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 msgr2=0x558591a337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591a337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 msgr2=0x558591a2fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a2fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 -- 172.21.15.46:0/3687152880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585917a6f70 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fdd9002a890 tx=0x7fdd9002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee587420290927b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.056+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9004e060 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd98003da0 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fdd9003edc0 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fdd90054050 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 10 client.11430 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x55859163e170 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98003da0 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fdd90057020 con 0x558591a32820 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 10 client.11430 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 10 client.11430 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:55:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 20 client.11430 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a341e0 0x558591a5dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 4 client.11430 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.057+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55859197a590 con 0x558591a341e0 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.058+0000 7fdd9e000640 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a341e0 0x558591a5dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.058+0000 7fdd9e000640 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a341e0 0x558591a5dbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61545df1ce776863 server_cookie=b2c6139a094c3d15 in_seq=0 out_seq=0 2024-04-26T23:55:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.058+0000 7fdd7fe00640 10 client.11430 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdd7fe00640 1 -- 172.21.15.46:0/3687152880 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55859197a590 con 0x558591a341e0 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdd7fe00640 10 client.11430 handle_command_reply: tid=0 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 client.11430 shutdown 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 msgr2=0x558591a35460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fdd9002a890 tx=0x7fdd9002c070 comp rx=0 tx=0).stop 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 shutdown_connections 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a341e0 0x558591a5dbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591a337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a2fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 --2- 172.21.15.46:0/3687152880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a35460 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 >> 172.21.15.46:0/3687152880 conn(0x558591a08500 msgr2=0x558591a51440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.059+0000 7fdd7fe00640 0 client.11430 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.060+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 shutdown_connections 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.060+0000 7fdda85b9740 1 -- 172.21.15.46:0/3687152880 wait complete. 2024-04-26T23:55:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.060+0000 7fdda85b9740 20 client.11430 trim_cache size 0 max 16384 2024-04-26T23:55:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.079+0000 7fdda85b9740 1 Processor -- start 2024-04-26T23:55:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.079+0000 7fdda85b9740 1 -- start start 2024-04-26T23:55:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a30140 0x558591a088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a500c0 0x558591a42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0201cc0 con 0x558591a32820 2024-04-26T23:55:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0159230 con 0x558591a30140 2024-04-26T23:55:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.080+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdda0158590 con 0x558591a500c0 2024-04-26T23:55:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a30140 0x558591a088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58644/0 (socket says 172.21.15.46:58644) 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 -- 172.21.15.46:0/1723172937 learned_addr learned my addr 172.21.15.46:0/1723172937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9f400640 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a500c0 0x558591a42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 -- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a500c0 msgr2=0x558591a42fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a500c0 0x558591a42fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 -- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a30140 msgr2=0x558591a088e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a30140 0x558591a088e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.081+0000 7fdd9e000640 1 -- 172.21.15.46:0/1723172937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558591a029f0 con 0x558591a32820 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c430 tx=0x7fdd8c0590d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2139165fd3625d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdd8aa00640 1 -- 172.21.15.46:0/1723172937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c057600 con 0x558591a32820 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdd8aa00640 1 -- 172.21.15.46:0/1723172937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c088070 con 0x558591a32820 2024-04-26T23:55:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdd8aa00640 1 -- 172.21.15.46:0/1723172937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c055970 con 0x558591a32820 2024-04-26T23:55:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 -- 172.21.15.46:0/1723172937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 msgr2=0x558591a08e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c430 tx=0x7fdd8c0590d0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 -- 172.21.15.46:0/1723172937 shutdown_connections 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a500c0 0x558591a42fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a08e20 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.082+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1723172937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a30140 0x558591a088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.083+0000 7fdda85b9740 1 -- 172.21.15.46:0/1723172937 >> 172.21.15.46:0/1723172937 conn(0x558591a08500 msgr2=0x558591a334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.083+0000 7fdda85b9740 1 -- 172.21.15.46:0/1723172937 shutdown_connections 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.083+0000 7fdda85b9740 1 -- 172.21.15.46:0/1723172937 wait complete. 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.083+0000 7fdda85b9740 1 -- start start 2024-04-26T23:55:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.083+0000 7fdda85b9740 1 Processor -- start 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591acf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0158be0 con 0x558591a32820 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558591a05d80 con 0x558591a500c0 2024-04-26T23:55:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558591a70200 con 0x558591a30140 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.084+0000 7fdda85b9740 10 client.0 _ll_register_callbacks cb 0x7fdd90035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58658/0 (socket says 172.21.15.46:58658) 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 -- 172.21.15.46:0/1223149485 learned_addr learned my addr 172.21.15.46:0/1223149485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591acf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 -- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 msgr2=0x558591acf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591acf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 -- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 msgr2=0x558591a44710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a44710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 -- 172.21.15.46:0/1223149485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558591a33ad0 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd9e000640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0593b0 tx=0x7fdd8c01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6500bfc297ed23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.085+0000 7fdd7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c078020 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fdd8c055970 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8c088070 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c082050 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 10 client.11436 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x558591a6ff40 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c088070 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fdd8c058750 con 0x558591a32820 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 10 client.11436 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 10 client.11436 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 20 client.11436 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a44d90 0x558591a2eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 4 client.11436 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-26T23:55:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.086+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x558591974400 con 0x558591a44d90 2024-04-26T23:55:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.087+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a44d90 0x558591a2eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.087+0000 7fdd9ea00640 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a44d90 0x558591a2eb70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc51cb184ace3c9 server_cookie=2c1be03d83cefad2 in_seq=0 out_seq=0 2024-04-26T23:55:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.087+0000 7fdd7fe00640 10 client.11436 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdd7fe00640 1 -- 172.21.15.46:0/1223149485 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558591974400 con 0x558591a44d90 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdd7fe00640 10 client.11436 handle_command_reply: tid=0 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 client.11436 shutdown 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 msgr2=0x558591a098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0593b0 tx=0x7fdd8c01d040 comp rx=0 tx=0).stop 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 shutdown_connections 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x558591a44d90 0x558591a2eb70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558591a500c0 0x558591a44710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558591a32820 0x558591a098d0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 --2- 172.21.15.46:0/1223149485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558591a30140 0x558591acf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdd7fe00640 0 client.11436 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.088+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 >> 172.21.15.46:0/1223149485 conn(0x558591a08500 msgr2=0x558591a08b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.094 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:55:38.088+0000 7f7b37400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-26T23:55:36.280108+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={7891=0-12582912@1} caps={7891=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=7891 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5570bde70c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(28)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-26T23:55:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 shutdown_connections 2024-04-26T23:55:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdda85b9740 1 -- 172.21.15.46:0/1223149485 wait complete. 2024-04-26T23:55:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdda85b9740 20 client.11436 trim_cache size 0 max 16384 2024-04-26T23:55:38.095 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:38.095 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:55:38.095 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "35abcf76-3b51-421f-9601-da4372605bbf", 2024-04-26T23:55:38.095 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:55:38.095 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 msgr2=0x7fdd8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 0x7fdd8006ede0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).stop 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 msgr2=0x7fdda015f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.089+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c460 tx=0x7fdd8c00c920 comp rx=0 tx=0).stop 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 shutdown_connections 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd8006c920 0x7fdd8006ede0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda01627f0 0x7fdda00c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda00d3cc0 0x7fdda015f540 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 --2- 172.21.15.46:0/538368417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdda00d2080 0x7fdda00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 >> 172.21.15.46:0/538368417 conn(0x7fdda009e380 msgr2=0x7fdda00d5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 shutdown_connections 2024-04-26T23:55:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.090+0000 7fdd9fe00640 1 -- 172.21.15.46:0/538368417 wait complete. 2024-04-26T23:55:38.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:55:38.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-26T23:55:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.307+0000 7fb2bca00640 1 Processor -- start 2024-04-26T23:55:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.307+0000 7fb2bca00640 1 -- start start 2024-04-26T23:55:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.308+0000 7fb2bca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2b8162a70 0x7fb2b80d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2bca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 0x7fb2b80b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2bca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b80d20c0 con 0x7fb2b80d3830 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2bca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b80d3400 con 0x7fb2b8162a70 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2bca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b80d2370 con 0x7fb2b80d1cb0 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2b8162a70 0x7fb2b80d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2ade00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb2b80d3830 0x7fb2b80d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58456/0 (socket says 172.21.15.46:58456) 2024-04-26T23:55:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2ade00640 1 -- 172.21.15.46:0/1621469138 learned_addr learned my addr 172.21.15.46:0/1621469138 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b7200640 1 --2- 172.21.15.46:0/1621469138 >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 0x7fb2b80b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b6800640 1 -- 172.21.15.46:0/1621469138 >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 msgr2=0x7fb2b80b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b6800640 1 --2- 172.21.15.46:0/1621469138 >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 0x7fb2b80b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b6800640 1 -- 172.21.15.46:0/1621469138 >> v1:172.21.15.46:6789/0 conn(0x7fb2b80d3830 legacy=0x7fb2b80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.309+0000 7fb2b5e00640 1 -- 172.21.15.46:0/1621469138 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897947559 0 0) 0x7fb2b80d20c0 con 0x7fb2b80d3830 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.310+0000 7fb2b6800640 1 -- 172.21.15.46:0/1621469138 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80d2650 con 0x7fb2b8162a70 2024-04-26T23:55:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.310+0000 7fb2b7200640 1 --2- 172.21.15.46:0/1621469138 >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 0x7fb2b80b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.311+0000 7fb2b6800640 1 --2- 172.21.15.46:0/1621469138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b80d2e20 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80cf500 tx=0x7fb2a802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5c9dcab788009e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.311+0000 7fb2b5e00640 1 -- 172.21.15.46:0/1621469138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8037070 con 0x7fb2b8162a70 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.311+0000 7fb2b5e00640 1 -- 172.21.15.46:0/1621469138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a8032440 con 0x7fb2b8162a70 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.311+0000 7fb2b5e00640 1 -- 172.21.15.46:0/1621469138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8032810 con 0x7fb2b8162a70 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 -- 172.21.15.46:0/1621469138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 msgr2=0x7fb2b80d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 --2- 172.21.15.46:0/1621469138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b80d2e20 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80cf500 tx=0x7fb2a802a250 comp rx=0 tx=0).stop 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 -- 172.21.15.46:0/1621469138 shutdown_connections 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 --2- 172.21.15.46:0/1621469138 >> v2:172.21.15.142:3300/0 conn(0x7fb2b80d1cb0 0x7fb2b80b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 --2- 172.21.15.46:0/1621469138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b80d2e20 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 -- 172.21.15.46:0/1621469138 >> 172.21.15.46:0/1621469138 conn(0x7fb2b80d4f20 msgr2=0x7fb2b80d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 -- 172.21.15.46:0/1621469138 shutdown_connections 2024-04-26T23:55:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.312+0000 7fb2bca00640 1 -- 172.21.15.46:0/1621469138 wait complete. 2024-04-26T23:55:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.314+0000 7fb2bca00640 1 Processor -- start 2024-04-26T23:55:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.314+0000 7fb2bca00640 1 -- start start 2024-04-26T23:55:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2b80d1cb0 0x7fb2b80ba350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2b80d3830 0x7fb2b81f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8162fd0 con 0x7fb2b8162a70 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8162e50 con 0x7fb2b80d3830 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2bca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80b2f20 con 0x7fb2b80d1cb0 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58674/0 (socket says 172.21.15.46:58674) 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.315+0000 7fb2b7200640 1 -- 172.21.15.46:0/4158778515 learned_addr learned my addr 172.21.15.46:0/4158778515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 -- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2b80d1cb0 msgr2=0x7fb2b80ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2b80d1cb0 0x7fb2b80ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 -- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2b80d3830 msgr2=0x7fb2b81f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2b80d3830 0x7fb2b81f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 -- 172.21.15.46:0/4158778515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80cc480 con 0x7fb2b8162a70 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2b7200640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000cef0 tx=0x7fb2b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2af400640 1 -- 172.21.15.46:0/4158778515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b001a070 con 0x7fb2b8162a70 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2af400640 1 -- 172.21.15.46:0/4158778515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b000d860 con 0x7fb2b8162a70 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b80ca5e0 con 0x7fb2b8162a70 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.316+0000 7fb2af400640 1 -- 172.21.15.46:0/4158778515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000dc50 con 0x7fb2b8162a70 2024-04-26T23:55:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.317+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b80caba0 con 0x7fb2b8162a70 2024-04-26T23:55:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.319+0000 7fb2af400640 1 -- 172.21.15.46:0/4158778515 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b001d070 con 0x7fb2b8162a70 2024-04-26T23:55:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.320+0000 7fb2af400640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 0x7fb29406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.320+0000 7fb2af400640 1 -- 172.21.15.46:0/4158778515 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fb2b008c6e0 con 0x7fb2b8162a70 2024-04-26T23:55:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.320+0000 7fb2c0279740 1 Processor -- start 2024-04-26T23:55:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.321+0000 7fb2b6800640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 0x7fb29406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.321+0000 7fb2c0279740 1 -- start start 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 0x560dbb11d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560dbafe5030 con 0x560dbb0d53f0 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560dbb0007f0 con 0x560dbb0ff710 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560dbafbb7e0 con 0x560dbb11cfb0 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b6800640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 0x7fb29406edb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8034000 tx=0x7fb2a802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58684/0 (socket says 172.21.15.46:58684) 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 -- 172.21.15.46:0/2588014555 learned_addr learned my addr 172.21.15.46:0/2588014555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 0x560dbb11d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 -- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 msgr2=0x560dbb11d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 0x560dbb11d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 -- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 msgr2=0x560dbb0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2b7200640 1 -- 172.21.15.46:0/2588014555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbafc7050 con 0x560dbb0d53f0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.322+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 0x560dbb11d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.323+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0059f10 tx=0x7fb2b005a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22e8990b1dc6f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.323+0000 7fb2a7000640 1 -- 172.21.15.46:0/2588014555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0058510 con 0x560dbb0d53f0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.323+0000 7fb2a7000640 1 -- 172.21.15.46:0/2588014555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0056880 con 0x560dbb0d53f0 2024-04-26T23:55:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.323+0000 7fb2a7000640 1 -- 172.21.15.46:0/2588014555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b005e370 con 0x560dbb0d53f0 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 -- 172.21.15.46:0/2588014555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 msgr2=0x560dbb11b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0059f10 tx=0x7fb2b005a2d0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 -- 172.21.15.46:0/2588014555 shutdown_connections 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb0d53f0 0x560dbb11b670 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb11cfb0 0x560dbb11d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2588014555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 -- 172.21.15.46:0/2588014555 >> 172.21.15.46:0/2588014555 conn(0x560dbb0d1d20 msgr2=0x560dbb11a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 -- 172.21.15.46:0/2588014555 shutdown_connections 2024-04-26T23:55:38.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.324+0000 7fb2c0279740 1 -- 172.21.15.46:0/2588014555 wait complete. 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.325+0000 7fb2c0279740 1 -- start start 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.325+0000 7fb2c0279740 1 Processor -- start 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb101e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb12de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560dbb03d200 con 0x560dbb11cfb0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560dbb037ee0 con 0x560dbb1023b0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560dbb044e70 con 0x560dbb0ff710 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2c0279740 10 client.0 _ll_register_callbacks cb 0x560dbb02e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58694/0 (socket says 172.21.15.46:58694) 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.326+0000 7fb2ade00640 1 -- 172.21.15.46:0/2453668890 learned_addr learned my addr 172.21.15.46:0/2453668890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb12de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 -- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 msgr2=0x560dbb101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 -- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 msgr2=0x560dbb12de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb12de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 -- 172.21.15.46:0/2453668890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbae147d0 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0001ea0 tx=0x7fb2a000d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bce610b05850aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb29b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00072d0 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fb2a0007470 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.327+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b0056880 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb2a0007670 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 10 client.11451 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x560dbad4a3f0 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0056880 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fb2a00047e0 con 0x560dbb11cfb0 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 10 client.11451 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 10 client.11451 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:55:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 20 client.11451 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11bdd0 0x560dbb11c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 4 client.11451 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbb0467e0 con 0x560dbb11bdd0 2024-04-26T23:55:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.328+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11bdd0 0x560dbb11c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.329+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11bdd0 0x560dbb11c1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d22743a2d73d12 server_cookie=70eafe43640bc615 in_seq=0 out_seq=0 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.329+0000 7fb29b400640 10 client.11451 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb29b400640 1 -- 172.21.15.46:0/2453668890 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dbb0467e0 con 0x560dbb11bdd0 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb29b400640 10 client.11451 handle_command_reply: tid=0 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 client.11451 shutdown 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 msgr2=0x560dbb12d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0001ea0 tx=0x7fb2a000d5e0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 shutdown_connections 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11bdd0 0x560dbb11c1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb12de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb12d8f0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2453668890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb29b400640 0 client.11451 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 >> 172.21.15.46:0/2453668890 conn(0x560dbb0d53f0 msgr2=0x560dbb11e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 shutdown_connections 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.330+0000 7fb2c0279740 1 -- 172.21.15.46:0/2453668890 wait complete. 2024-04-26T23:55:38.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.331+0000 7fb2c0279740 20 client.11451 trim_cache size 0 max 16384 2024-04-26T23:55:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.350+0000 7fb2c0279740 1 Processor -- start 2024-04-26T23:55:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.350+0000 7fb2c0279740 1 -- start start 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb1023b0 0x560dbb112d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8201a70 con 0x560dbb11cfb0 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81482e0 con 0x560dbb0ff710 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80ce810 con 0x560dbb1023b0 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.351+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58706/0 (socket says 172.21.15.46:58706) 2024-04-26T23:55:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 -- 172.21.15.46:0/1834594658 learned_addr learned my addr 172.21.15.46:0/1834594658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b6800640 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2ade00640 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb1023b0 0x560dbb112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 -- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb1023b0 msgr2=0x560dbb112d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb1023b0 0x560dbb112d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 -- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 msgr2=0x560dbb0fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.352+0000 7fb2b7200640 1 -- 172.21.15.46:0/1834594658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbb11c520 con 0x560dbb11cfb0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.353+0000 7fb2b7200640 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000ac10 tx=0x7fb2b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a6d13f4b4db9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.353+0000 7fb2a6600640 1 -- 172.21.15.46:0/1834594658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0058510 con 0x560dbb11cfb0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.353+0000 7fb2a6600640 1 -- 172.21.15.46:0/1834594658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0056880 con 0x560dbb11cfb0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.353+0000 7fb2a6600640 1 -- 172.21.15.46:0/1834594658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b005e390 con 0x560dbb11cfb0 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 -- 172.21.15.46:0/1834594658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 msgr2=0x560dbb1132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000ac10 tx=0x7fb2b0017000 comp rx=0 tx=0).stop 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 -- 172.21.15.46:0/1834594658 shutdown_connections 2024-04-26T23:55:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb1132d0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb1023b0 0x560dbb112d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 --2- 172.21.15.46:0/1834594658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb0ff710 0x560dbb0fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 -- 172.21.15.46:0/1834594658 >> 172.21.15.46:0/1834594658 conn(0x560dbb0d53f0 msgr2=0x560dbb1109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 -- 172.21.15.46:0/1834594658 shutdown_connections 2024-04-26T23:55:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.354+0000 7fb2c0279740 1 -- 172.21.15.46:0/1834594658 wait complete. 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.355+0000 7fb2c0279740 1 -- start start 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.355+0000 7fb2c0279740 1 Processor -- start 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb193a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb13d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560dbb045fd0 con 0x560dbb11cfb0 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8147c90 con 0x560dbb1023b0 2024-04-26T23:55:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x560dbb101c30 con 0x560dbb0ff710 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2c0279740 10 client.0 _ll_register_callbacks cb 0x7fb2a0007c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58708/0 (socket says 172.21.15.46:58708) 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2b7200640 1 -- 172.21.15.46:0/2981911675 learned_addr learned my addr 172.21.15.46:0/2981911675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2ade00640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb13d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.356+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb193a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 -- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 msgr2=0x560dbb193a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb193a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 -- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 msgr2=0x560dbb13d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb13d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 -- 172.21.15.46:0/2981911675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbad0a0f0 con 0x560dbb11cfb0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2b7200640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0062950 tx=0x7fb2b0017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61645ed2038a8ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb29b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b007e060 con 0x560dbb11cfb0 2024-04-26T23:55:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(186..186 src has 1..186) ==== 6101+0+0 (secure 0 0 0) 0x7fb2b0058510 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.357+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b0056880 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb2b0084050 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 10 client.11457 fetch_fsmap learned FSMap version 149 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x560dbb000a30 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0056880 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7fb2b00629f0 con 0x560dbb11cfb0 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 10 client.11457 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 10 client.11457 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 20 client.11457 populate_metadata read hostname 'smithi046' 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11a070 0x560dbb0fe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 4 client.11457 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560dbb040650 con 0x560dbb11a070 2024-04-26T23:55:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.358+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11a070 0x560dbb0fe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2b6800640 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11a070 0x560dbb0fe110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5f5554e63cef31 server_cookie=3364d87c75ef0181 in_seq=0 out_seq=0 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb29b400640 10 client.11457 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb29b400640 1 -- 172.21.15.46:0/2981911675 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x560dbb040650 con 0x560dbb11a070 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb29b400640 10 client.11457 handle_command_reply: tid=0 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 client.11457 shutdown 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 msgr2=0x560dbb134bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0062950 tx=0x7fb2b0017070 comp rx=0 tx=0).stop 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 shutdown_connections 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x560dbb11a070 0x560dbb0fe110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.359+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dbb11cfb0 0x560dbb134bc0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dbb1023b0 0x560dbb13d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 1 --2- 172.21.15.46:0/2981911675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560dbb0ff710 0x560dbb193a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb29b400640 0 client.11457 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 >> 172.21.15.46:0/2981911675 conn(0x560dbb0d53f0 msgr2=0x560dbb0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 shutdown_connections 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 1 -- 172.21.15.46:0/2981911675 wait complete. 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2c0279740 20 client.11457 trim_cache size 0 max 16384 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "35abcf76-3b51-421f-9601-da4372605bbf": { 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "35abcf76-3b51-421f-9601-da4372605bbf", 2024-04-26T23:55:38.365 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-26T23:55:38.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:38.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:55:38.366 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 msgr2=0x7fb29406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 0x7fb29406edb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8034000 tx=0x7fb2a802c000 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 msgr2=0x7fb2b81f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.360+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000cef0 tx=0x7fb2b000a710 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 shutdown_connections 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb29406c8f0 0x7fb29406edb0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b8162a70 0x7fb2b81f4490 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2b80d3830 0x7fb2b81f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 --2- 172.21.15.46:0/4158778515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2b80d1cb0 0x7fb2b80ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 >> 172.21.15.46:0/4158778515 conn(0x7fb2b80d4f20 msgr2=0x7fb2b806e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 shutdown_connections 2024-04-26T23:55:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:55:38.361+0000 7fb2bca00640 1 -- 172.21.15.46:0/4158778515 wait complete. 2024-04-26T23:55:38.380 INFO:tasks.cephfs.filesystem:scrub status for tag:35abcf76-3b51-421f-9601-da4372605bbf - {'path': '/', 'tag': '35abcf76-3b51-421f-9601-da4372605bbf', 'options': 'recursive,repair'} 2024-04-26T23:55:43.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:43.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:08.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:08.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-26T23:56:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.606+0000 7f330fe00640 1 Processor -- start 2024-04-26T23:56:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.606+0000 7f330fe00640 1 -- start start 2024-04-26T23:56:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.607+0000 7f330fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f33080cfdf0 0x7f33080b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.607+0000 7f330fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33080b3360 con 0x7f330806e580 2024-04-26T23:56:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.607+0000 7f330fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33080b3640 con 0x7f330806d790 2024-04-26T23:56:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.607+0000 7f330fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f33080b3f70 con 0x7f33080cfdf0 2024-04-26T23:56:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.608+0000 7f330de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f330806e580 0x7f33080afb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34044/0 (socket says 172.21.15.46:34044) 2024-04-26T23:56:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.608+0000 7f330de00640 1 -- 172.21.15.46:0/1232163336 learned_addr learned my addr 172.21.15.46:0/1232163336 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.608+0000 7f330d400640 1 --2- 172.21.15.46:0/1232163336 >> v2:172.21.15.142:3300/0 conn(0x7f33080cfdf0 0x7f33080b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.608+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143180756 0 0) 0x7f33080b3360 con 0x7f330806e580 2024-04-26T23:56:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.608+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f0003520 con 0x7f330806e580 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1338738608 0 0) 0x7f32f0003520 con 0x7f330806e580 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 >> v2:172.21.15.142:3300/0 conn(0x7f33080cfdf0 msgr2=0x7f33080b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 --2- 172.21.15.46:0/1232163336 >> v2:172.21.15.142:3300/0 conn(0x7f33080cfdf0 0x7f33080b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 >> v1:172.21.15.142:6789/0 conn(0x7f330806d790 legacy=0x7f330806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33080b3920 con 0x7f330806e580 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3300001670 con 0x7f330806e580 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 >> v1:172.21.15.46:6789/0 conn(0x7f330806e580 legacy=0x7f33080afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f00039b0 0x7f32f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 0x7f32f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f00051c0 0x7f32f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33080b3f70 con 0x7f32f0004800 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32f000c060 con 0x7f32f00051c0 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32f000c260 con 0x7f32f00039b0 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.609+0000 7f330ca00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 0x7f32f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330d400640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f00039b0 0x7f32f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330de00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f00051c0 0x7f32f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330ca00640 1 -- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f00039b0 msgr2=0x7f32f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330ca00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f00039b0 0x7f32f0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330ca00640 1 -- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f00051c0 msgr2=0x7f32f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330ca00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f00051c0 0x7f32f000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330ca00640 1 -- 172.21.15.46:0/1232163336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f0003520 con 0x7f32f0004800 2024-04-26T23:56:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.610+0000 7f330de00640 1 -- 172.21.15.46:0/1232163336 reap_dead start 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.611+0000 7f330ca00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 0x7f32f0004c80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f32fc001e50 tx=0x7f32f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a7b99dcc78df1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.611+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc001a70 con 0x7f32f0004800 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.611+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc02ae40 con 0x7f32f0004800 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.611+0000 7f3307e00640 1 -- 172.21.15.46:0/1232163336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc003bb0 con 0x7f32f0004800 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 -- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 msgr2=0x7f32f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 0x7f32f0004c80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f32fc001e50 tx=0x7f32f0005830 comp rx=0 tx=0).stop 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 -- 172.21.15.46:0/1232163336 shutdown_connections 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 --2- 172.21.15.46:0/1232163336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f0004800 0x7f32f0004c80 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 -- 172.21.15.46:0/1232163336 >> 172.21.15.46:0/1232163336 conn(0x7f33080d2e90 msgr2=0x7f33080d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 -- 172.21.15.46:0/1232163336 shutdown_connections 2024-04-26T23:56:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.612+0000 7f330fe00640 1 -- 172.21.15.46:0/1232163336 wait complete. 2024-04-26T23:56:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.614+0000 7f330fe00640 1 Processor -- start 2024-04-26T23:56:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.614+0000 7f330fe00640 1 -- start start 2024-04-26T23:56:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f0004800 0x7f3308153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33080ca970 0x7f33080cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33080d01d0 con 0x7f33080cadf0 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f330806dbf0 con 0x7f33080ca970 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f33080b3010 con 0x7f32f0004800 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54038/0 (socket says 172.21.15.46:54038) 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330de00640 1 -- 172.21.15.46:0/1645782728 learned_addr learned my addr 172.21.15.46:0/1645782728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330ca00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33080ca970 0x7f33080cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.615+0000 7f330d400640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f0004800 0x7f3308153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 -- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f0004800 msgr2=0x7f3308153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f0004800 0x7f3308153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 -- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33080ca970 msgr2=0x7f33080cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33080ca970 0x7f33080cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 -- 172.21.15.46:0/1645782728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081536b0 con 0x7f33080cadf0 2024-04-26T23:56:08.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330de00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f330806f5a0 tx=0x7f330000d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f3305600640 1 -- 172.21.15.46:0/1645782728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300007440 con 0x7f33080cadf0 2024-04-26T23:56:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f3305600640 1 -- 172.21.15.46:0/1645782728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33000075e0 con 0x7f33080cadf0 2024-04-26T23:56:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f3305600640 1 -- 172.21.15.46:0/1645782728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300004a20 con 0x7f33080cadf0 2024-04-26T23:56:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.616+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33080cb350 con 0x7f33080cadf0 2024-04-26T23:56:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.617+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33080cb910 con 0x7f33080cadf0 2024-04-26T23:56:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.619+0000 7f3305600640 1 -- 172.21.15.46:0/1645782728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3300004bc0 con 0x7f33080cadf0 2024-04-26T23:56:08.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.620+0000 7f3305600640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 0x7f32ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.621+0000 7f3305600640 1 -- 172.21.15.46:0/1645782728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f330008b550 con 0x7f33080cadf0 2024-04-26T23:56:08.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.621+0000 7f3312bf0740 1 Processor -- start 2024-04-26T23:56:08.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.621+0000 7f330d400640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 0x7f32ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.621+0000 7f3312bf0740 1 -- start start 2024-04-26T23:56:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d4d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 0x55a584d4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a584c15030 con 0x55a584d2f540 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a584c307f0 con 0x55a584d05220 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a584beb7e0 con 0x55a584d4cde0 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f330d400640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 0x7f32ec06ede0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f330806b270 tx=0x7f32f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.622+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54048/0 (socket says 172.21.15.46:54048) 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 -- 172.21.15.46:0/2991388144 learned_addr learned my addr 172.21.15.46:0/2991388144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330ca00640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d4d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330de00640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 0x55a584d4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 -- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 msgr2=0x55a584d4d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d4d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 -- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 msgr2=0x55a584d4b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 0x55a584d4b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 -- 172.21.15.46:0/2991388144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a584bf7050 con 0x55a584d2f540 2024-04-26T23:56:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330de00640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 0x55a584d4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.623+0000 7f330d400640 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f32f8012d00 tx=0x7f32f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8defd8e2c3beab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f32f5c00640 1 -- 172.21.15.46:0/2991388144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800f040 con 0x55a584d2f540 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f32f5c00640 1 -- 172.21.15.46:0/2991388144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8027070 con 0x55a584d2f540 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f32f5c00640 1 -- 172.21.15.46:0/2991388144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8022640 con 0x55a584d2f540 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 -- 172.21.15.46:0/2991388144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 msgr2=0x55a584d2f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f32f8012d00 tx=0x7f32f8015000 comp rx=0 tx=0).stop 2024-04-26T23:56:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 -- 172.21.15.46:0/2991388144 shutdown_connections 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d05220 0x55a584d4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d4d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2991388144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d2f920 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.624+0000 7f3312bf0740 1 -- 172.21.15.46:0/2991388144 >> 172.21.15.46:0/2991388144 conn(0x55a584d01b50 msgr2=0x55a584d4a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.625+0000 7f3312bf0740 1 -- 172.21.15.46:0/2991388144 shutdown_connections 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.625+0000 7f3312bf0740 1 -- 172.21.15.46:0/2991388144 wait complete. 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.625+0000 7f3312bf0740 1 -- start start 2024-04-26T23:56:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.625+0000 7f3312bf0740 1 Processor -- start 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a584c6d200 con 0x55a584d2f540 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a584c67ee0 con 0x55a584d1cbc0 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a584c74e70 con 0x55a584d4cde0 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.626+0000 7f3312bf0740 10 client.0 _ll_register_callbacks cb 0x55a584c5e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54054/0 (socket says 172.21.15.46:54054) 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 -- 172.21.15.46:0/2342274354 learned_addr learned my addr 172.21.15.46:0/2342274354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330de00640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 -- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 msgr2=0x55a584d41400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330ca00640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d41400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 -- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 msgr2=0x55a584d4d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d4d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.627+0000 7f330d400640 1 -- 172.21.15.46:0/2342274354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a584a447d0 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f330d400640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f32f80128c0 tx=0x7f32f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843494989f3f50bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f32ea000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8004040 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f32f8037cc0 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f8027070 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f32f803e050 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f3312bf0740 10 client.11469 fetch_fsmap learned FSMap version 149 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x55a58497a3f0 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.628+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8027070 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7f32f8041020 con 0x55a584d2f540 2024-04-26T23:56:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 10 client.11469 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 10 client.11469 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 20 client.11469 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d329c0 0x55a584d32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 4 client.11469 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a584c767e0 con 0x55a584d329c0 2024-04-26T23:56:08.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.629+0000 7f330ca00640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d329c0 0x55a584d32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.630+0000 7f330ca00640 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d329c0 0x55a584d32e00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43b1dedb96aeb65 server_cookie=4a7bb651d60fbb0e in_seq=0 out_seq=0 2024-04-26T23:56:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.630+0000 7f32ea000640 10 client.11469 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:56:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.630+0000 7f32ea000640 1 -- 172.21.15.46:0/2342274354 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a584c767e0 con 0x55a584d329c0 2024-04-26T23:56:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.630+0000 7f32ea000640 10 client.11469 handle_command_reply: tid=0 2024-04-26T23:56:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.630+0000 7f3312bf0740 1 client.11469 shutdown 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 msgr2=0x55a584d1d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f32f80128c0 tx=0x7f32f8004730 comp rx=0 tx=0).stop 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 shutdown_connections 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d329c0 0x55a584d32e00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d41400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 --2- 172.21.15.46:0/2342274354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1d490 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 >> 172.21.15.46:0/2342274354 conn(0x55a584d05220 msgr2=0x55a584d49bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f32ea000640 0 client.11469 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 shutdown_connections 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 1 -- 172.21.15.46:0/2342274354 wait complete. 2024-04-26T23:56:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.631+0000 7f3312bf0740 20 client.11469 trim_cache size 0 max 16384 2024-04-26T23:56:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.650+0000 7f3312bf0740 1 Processor -- start 2024-04-26T23:56:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 -- start start 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d2f540 0x55a584d30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3308202710 con 0x55a584d1cbc0 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33082016f0 con 0x55a584d2f540 2024-04-26T23:56:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.651+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3308200a50 con 0x55a584d4cde0 2024-04-26T23:56:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54070/0 (socket says 172.21.15.46:54070) 2024-04-26T23:56:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 -- 172.21.15.46:0/822941438 learned_addr learned my addr 172.21.15.46:0/822941438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330de00640 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330ca00640 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d2f540 0x55a584d30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 -- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 msgr2=0x55a584d30c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d30c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 -- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d2f540 msgr2=0x55a584d30720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d2f540 0x55a584d30720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 -- 172.21.15.46:0/822941438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58493a0f0 con 0x55a584d1cbc0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.652+0000 7f330d400640 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f32f80044a0 tx=0x7f32f802f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83b86cbb7322523 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f32f5200640 1 -- 172.21.15.46:0/822941438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8012c80 con 0x55a584d1cbc0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f32f5200640 1 -- 172.21.15.46:0/822941438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8027040 con 0x55a584d1cbc0 2024-04-26T23:56:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f32f5200640 1 -- 172.21.15.46:0/822941438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8003850 con 0x55a584d1cbc0 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 -- 172.21.15.46:0/822941438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 msgr2=0x55a584dbe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f32f80044a0 tx=0x7f32f802f070 comp rx=0 tx=0).stop 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 -- 172.21.15.46:0/822941438 shutdown_connections 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d30c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d2f540 0x55a584d30720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 --2- 172.21.15.46:0/822941438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d1cbc0 0x55a584dbe230 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 -- 172.21.15.46:0/822941438 >> 172.21.15.46:0/822941438 conn(0x55a584d05220 msgr2=0x55a584d4d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.653+0000 7f3312bf0740 1 -- 172.21.15.46:0/822941438 shutdown_connections 2024-04-26T23:56:08.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.654+0000 7f3312bf0740 1 -- 172.21.15.46:0/822941438 wait complete. 2024-04-26T23:56:08.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.654+0000 7f3312bf0740 1 -- start start 2024-04-26T23:56:08.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.654+0000 7f3312bf0740 1 Processor -- start 2024-04-26T23:56:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d1b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a584c75fd0 con 0x55a584d2f540 2024-04-26T23:56:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33082010a0 con 0x55a584d1cbc0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a584d1bfc0 con 0x55a584d4cde0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.655+0000 7f3312bf0740 10 client.0 _ll_register_callbacks cb 0x7f32f8012bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54084/0 (socket says 172.21.15.46:54084) 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 -- 172.21.15.46:0/1400220848 learned_addr learned my addr 172.21.15.46:0/1400220848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 -- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 msgr2=0x55a584d1b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 -- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 msgr2=0x55a584d408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330d400640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 -- 172.21.15.46:0/1400220848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a584c30a30 con 0x55a584d2f540 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f330ca00640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f32fc001e50 tx=0x7f32fc02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5b7b3d60858de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.656+0000 7f32ea000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc037040 con 0x55a584d2f540 2024-04-26T23:56:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f32fc006d50 con 0x55a584d2f540 2024-04-26T23:56:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f8027040 con 0x55a584d2f540 2024-04-26T23:56:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f32fc050050 con 0x55a584d2f540 2024-04-26T23:56:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 10 client.11475 fetch_fsmap learned FSMap version 149 2024-04-26T23:56:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=2+}) -- 0x55a584dbdb40 con 0x55a584d2f540 2024-04-26T23:56:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8027040 con 0x55a584d2f540 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 149) ==== 3494+0+0 (secure 0 0 0) 0x7f32fc053020 con 0x55a584d2f540 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 10 client.11475 fetch_fsmap finished waiting for FSMap version 149 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 10 client.11475 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 20 client.11475 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d48990 0x55a584d30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 4 client.11475 mds_command: new command op to 11247 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.657+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 --> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a584d30570 con 0x55a584d48990 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.658+0000 7f330d400640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d48990 0x55a584d30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.658+0000 7f330d400640 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d48990 0x55a584d30720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b905428b2c95ab8 server_cookie=5f55d0f6f8be34c9 in_seq=0 out_seq=0 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.658+0000 7f32ea000640 10 client.11475 ms_handle_connect on v2:172.21.15.46:6834/2117263293 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f32ea000640 1 -- 172.21.15.46:0/1400220848 <== mds.0 v2:172.21.15.46:6834/2117263293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a584d30570 con 0x55a584d48990 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f32ea000640 10 client.11475 handle_command_reply: tid=0 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 client.11475 shutdown 2024-04-26T23:56:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 msgr2=0x55a584d1b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f32fc001e50 tx=0x7f32fc02c070 comp rx=0 tx=0).stop 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 shutdown_connections 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:6834/2117263293,v1:172.21.15.46:6835/2117263293] conn(0x55a584d48990 0x55a584d30720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a584d4cde0 0x55a584d1b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a584d2f540 0x55a584d1b1c0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 --2- 172.21.15.46:0/1400220848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a584d1cbc0 0x55a584d408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f32ea000640 0 client.11475 ms_handle_reset on v2:172.21.15.46:6834/2117263293 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 >> 172.21.15.46:0/1400220848 conn(0x55a584d05220 msgr2=0x55a584cffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 shutdown_connections 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 1 -- 172.21.15.46:0/1400220848 wait complete. 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.659+0000 7f3312bf0740 20 client.11475 trim_cache size 0 max 16384 2024-04-26T23:56:08.664 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 msgr2=0x7f32ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 0x7f32ec06ede0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f330806b270 tx=0x7f32f8008040 comp rx=0 tx=0).stop 2024-04-26T23:56:08.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 msgr2=0x7f33080cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f330806f5a0 tx=0x7f330000d950 comp rx=0 tx=0).stop 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 shutdown_connections 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f32ec06c920 0x7f32ec06ede0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33080cadf0 0x7f33080cccd0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33080ca970 0x7f33080cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 --2- 172.21.15.46:0/1645782728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f0004800 0x7f3308153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 >> 172.21.15.46:0/1645782728 conn(0x7f33080d2e90 msgr2=0x7f33080ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 shutdown_connections 2024-04-26T23:56:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08.660+0000 7f330fe00640 1 -- 172.21.15.46:0/1645782728 wait complete. 2024-04-26T23:56:08.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T23:56:08.681 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-04-26T23:56:08.682 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mode": 33188, 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ino": 1099511627777, 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "st_dev": 41, 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "st_nlink": 1, 2024-04-26T23:56:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "st_uid": 1000, 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "st_gid": 1316, 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "st_size": 6291456, 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "st_atime": 1714175736.244109, 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "st_mtime": 1714175736.2861085, 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "st_ctime": 1714175736.2861085 2024-04-26T23:56:08.821 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:08.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:56:08.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.088+0000 7fa75ec00640 1 Processor -- start 2024-04-26T23:56:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.088+0000 7fa75ec00640 1 -- start start 2024-04-26T23:56:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.089+0000 7fa75ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa75815f4e0 0x7fa7580d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.089+0000 7fa75ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7580d2130 0x7fa7580b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.089+0000 7fa75ec00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7580d11f0 con 0x7fa7580b4000 2024-04-26T23:56:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.089+0000 7fa75ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580d0ce0 con 0x7fa7580d2130 2024-04-26T23:56:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.089+0000 7fa75ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580d2990 con 0x7fa75815f4e0 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7580d2130 0x7fa7580b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7580d2130 0x7fa7580b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54096/0 (socket says 172.21.15.46:54096) 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75e200640 1 -- 172.21.15.46:0/2036306594 learned_addr learned my addr 172.21.15.46:0/2036306594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75d800640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa75815f4e0 0x7fa7580d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75e200640 1 -- 172.21.15.46:0/2036306594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa75815f4e0 msgr2=0x7fa7580d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.090+0000 7fa75e200640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa75815f4e0 0x7fa7580d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.091+0000 7fa75e200640 1 -- 172.21.15.46:0/2036306594 >> v1:172.21.15.142:6789/0 conn(0x7fa7580b4000 legacy=0x7fa7580d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.091+0000 7fa75e200640 1 -- 172.21.15.46:0/2036306594 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580d5f20 con 0x7fa7580d2130 2024-04-26T23:56:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.092+0000 7fa75e200640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa7580b2280 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa750009870 tx=0x7fa75002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc26dd5366811689 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.092+0000 7fa757e00640 1 -- 172.21.15.46:0/2036306594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037070 con 0x7fa7580d2130 2024-04-26T23:56:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.092+0000 7fa757e00640 1 -- 172.21.15.46:0/2036306594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750032440 con 0x7fa7580d2130 2024-04-26T23:56:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.092+0000 7fa757e00640 1 -- 172.21.15.46:0/2036306594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750032810 con 0x7fa7580d2130 2024-04-26T23:56:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 -- 172.21.15.46:0/2036306594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 msgr2=0x7fa7580b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa7580b2280 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa750009870 tx=0x7fa75002a200 comp rx=0 tx=0).stop 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 -- 172.21.15.46:0/2036306594 shutdown_connections 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa7580b2280 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 --2- 172.21.15.46:0/2036306594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa75815f4e0 0x7fa7580d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 -- 172.21.15.46:0/2036306594 >> 172.21.15.46:0/2036306594 conn(0x7fa7580d5010 msgr2=0x7fa7580d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 -- 172.21.15.46:0/2036306594 shutdown_connections 2024-04-26T23:56:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.093+0000 7fa75ec00640 1 -- 172.21.15.46:0/2036306594 wait complete. 2024-04-26T23:56:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.095+0000 7fa75ec00640 1 Processor -- start 2024-04-26T23:56:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.095+0000 7fa75ec00640 1 -- start start 2024-04-26T23:56:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7580b4000 0x7fa758154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 0x7fa758162380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580b1d60 con 0x7fa7580d2130 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580d5c40 con 0x7fa75815f4e0 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.096+0000 7fa75ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa75815f8c0 con 0x7fa7580b4000 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54112/0 (socket says 172.21.15.46:54112) 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 -- 172.21.15.46:0/3250450117 learned_addr learned my addr 172.21.15.46:0/3250450117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75d800640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7580b4000 0x7fa758154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75e200640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 0x7fa758162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 -- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7580b4000 msgr2=0x7fa758154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7580b4000 0x7fa758154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 -- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 msgr2=0x7fa758162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 0x7fa758162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75ce00640 1 -- 172.21.15.46:0/3250450117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7581f8d70 con 0x7fa7580d2130 2024-04-26T23:56:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.097+0000 7fa75e200640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 0x7fa758162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa75ce00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c7c0 tx=0x7fa74800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748016020 con 0x7fa7580d2130 2024-04-26T23:56:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74800aa50 con 0x7fa7580d2130 2024-04-26T23:56:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74800ae40 con 0x7fa7580d2130 2024-04-26T23:56:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7581f9050 con 0x7fa7580d2130 2024-04-26T23:56:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.098+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7581f9610 con 0x7fa7580d2130 2024-04-26T23:56:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.099+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7580693b0 con 0x7fa7580d2130 2024-04-26T23:56:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.101+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa748019020 con 0x7fa7580d2130 2024-04-26T23:56:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.102+0000 7fa755600640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 0x7fa73c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.102+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7fa74808bde0 con 0x7fa7580d2130 2024-04-26T23:56:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.102+0000 7fa75d800640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 0x7fa73c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.104+0000 7fa75d800640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 0x7fa73c06edb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009a30 tx=0x7fa74c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.107+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa748020030 con 0x7fa7580d2130 2024-04-26T23:56:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.362+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7580d33d0 con 0x7fa7580d2130 2024-04-26T23:56:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.363+0000 7fa755600640 1 -- 172.21.15.46:0/3250450117 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7fa74805a020 con 0x7fa7580d2130 2024-04-26T23:56:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:09.366 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:56:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:56:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.365+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 msgr2=0x7fa73c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.365+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 0x7fa73c06edb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009a30 tx=0x7fa74c008040 comp rx=0 tx=0).stop 2024-04-26T23:56:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.365+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 msgr2=0x7fa758161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.365+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c7c0 tx=0x7fa74800cc80 comp rx=0 tx=0).stop 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 shutdown_connections 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa73c06c8f0 0x7fa73c06edb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa75815f4e0 0x7fa758162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7580d2130 0x7fa758161e40 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 --2- 172.21.15.46:0/3250450117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7580b4000 0x7fa758154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 >> 172.21.15.46:0/3250450117 conn(0x7fa7580d5010 msgr2=0x7fa7580d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 shutdown_connections 2024-04-26T23:56:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.366+0000 7fa75ec00640 1 -- 172.21.15.46:0/3250450117 wait complete. 2024-04-26T23:56:09.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T23:56:09.632 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-26T23:56:09.632 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T23:56:09.633 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-26T23:56:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.743+0000 7fd846e7cf40 1 Processor -- start 2024-04-26T23:56:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.743+0000 7fd846e7cf40 1 -- start start 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaee161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfc2f0 0x556eaeea3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556eaed70d20 con 0x556eaedfbe70 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556eaed74d60 con 0x556eaedfc2f0 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556eaedf8640 con 0x7fd828003ab0 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.744+0000 7fd843600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54132/0 (socket says 172.21.15.46:54132) 2024-04-26T23:56:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 -- 172.21.15.46:0/1888487472 learned_addr learned my addr 172.21.15.46:0/1888487472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd844000640 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaee161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 -- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 msgr2=0x556eaee161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaee161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 -- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfc2f0 msgr2=0x556eaeea3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfc2f0 0x556eaeea3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 -- 172.21.15.46:0/1888487472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaee16730 con 0x556eaedfbe70 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.745+0000 7fd843600640 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c3e0 tx=0x7fd82c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab41625ff679bde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd842200640 1 -- 172.21.15.46:0/1888487472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00e600 con 0x556eaedfbe70 2024-04-26T23:56:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd842200640 1 -- 172.21.15.46:0/1888487472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c00e7a0 con 0x556eaedfbe70 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd842200640 1 -- 172.21.15.46:0/1888487472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00e990 con 0x556eaedfbe70 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 -- 172.21.15.46:0/1888487472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 msgr2=0x556eaee9e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c3e0 tx=0x7fd82c001e70 comp rx=0 tx=0).stop 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 -- 172.21.15.46:0/1888487472 shutdown_connections 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfc2f0 0x556eaeea3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfbe70 0x556eaee9e960 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/1888487472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaee161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 -- 172.21.15.46:0/1888487472 >> 172.21.15.46:0/1888487472 conn(0x556eaedcea00 msgr2=0x556eaedfadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 -- 172.21.15.46:0/1888487472 shutdown_connections 2024-04-26T23:56:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.746+0000 7fd846e7cf40 1 -- 172.21.15.46:0/1888487472 wait complete. 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.747+0000 7fd846e7cf40 1 Processor -- start 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.747+0000 7fd846e7cf40 1 -- start start 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfbe70 0x556eaef36570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 0x556eaedfa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaedfa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556eaee169c0 con 0x556eaedfc2f0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556eaee9f410 con 0x556eaedfbe70 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd846e7cf40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556eaedfb500 con 0x7fd828003ab0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.748+0000 7fd843600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 0x556eaedfa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 0x556eaedfa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54140/0 (socket says 172.21.15.46:54140) 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 -- 172.21.15.46:0/3023150079 learned_addr learned my addr 172.21.15.46:0/3023150079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd844000640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfbe70 0x556eaef36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd844a00640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaedfa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 -- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 msgr2=0x556eaedfa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd828003ab0 0x556eaedfa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 -- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfbe70 msgr2=0x556eaef36570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfbe70 0x556eaef36570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 -- 172.21.15.46:0/3023150079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaef35510 con 0x556eaedfc2f0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd844000640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556eaedfbe70 0x556eaef36570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.749+0000 7fd843600640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 0x556eaedfa0d0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00cd30 tx=0x7fd82c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.750+0000 7fd840e00640 1 -- 172.21.15.46:0/3023150079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0016b0 con 0x556eaedfc2f0 2024-04-26T23:56:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.750+0000 7fd840e00640 1 -- 172.21.15.46:0/3023150079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c022020 con 0x556eaedfc2f0 2024-04-26T23:56:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.750+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556eaef357b0 con 0x556eaedfc2f0 2024-04-26T23:56:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.750+0000 7fd840e00640 1 -- 172.21.15.46:0/3023150079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c001380 con 0x556eaedfc2f0 2024-04-26T23:56:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.750+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaef35d70 con 0x556eaedfc2f0 2024-04-26T23:56:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.752+0000 7fd840e00640 1 -- 172.21.15.46:0/3023150079 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd82c018070 con 0x556eaedfc2f0 2024-04-26T23:56:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.753+0000 7fd840e00640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd82006b4a0 0x7fd82006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.753+0000 7fd840e00640 1 -- 172.21.15.46:0/3023150079 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7fd82c08cad0 con 0x556eaedfc2f0 2024-04-26T23:56:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.753+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x556eaee9b080 0x556eaee9be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.753+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- osd_op(unknown.0.0:1 28.7 28:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e188) -- 0x556eaee9c360 con 0x556eaee9b080 2024-04-26T23:56:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.754+0000 7fd844000640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd82006b4a0 0x7fd82006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.754+0000 7fd844a00640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x556eaee9b080 0x556eaee9be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.754+0000 7fd844a00640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x556eaee9b080 0x556eaee9be20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd844a00640 1 -- 172.21.15.46:0/3023150079 <== osd.1 v2:172.21.15.46:6816/3879162742 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fd83c001730 con 0x556eaee9b080 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd844000640 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd82006b4a0 0x7fd82006d960 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd834005fd0 tx=0x7fd83400c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x556eaee9b080 msgr2=0x556eaee9be20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x556eaee9b080 0x556eaee9be20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd82006b4a0 msgr2=0x7fd82006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd82006b4a0 0x7fd82006d960 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd834005fd0 tx=0x7fd83400c040 comp rx=0 tx=0).stop 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 msgr2=0x556eaedfa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 --2- 172.21.15.46:0/3023150079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eaedfc2f0 0x556eaedfa0d0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00cd30 tx=0x7fd82c001190 comp rx=0 tx=0).stop 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd844a00640 1 -- 172.21.15.46:0/3023150079 reap_dead start 2024-04-26T23:56:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 shutdown_connections 2024-04-26T23:56:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 >> 172.21.15.46:0/3023150079 conn(0x556eaedcea00 msgr2=0x556eaee9f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 shutdown_connections 2024-04-26T23:56:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:09.755+0000 7fd846e7cf40 1 -- 172.21.15.46:0/3023150079 wait complete. 2024-04-26T23:56:09.762 INFO:teuthology.orchestra.run.smithi046.stdout:Ysixmegssubdir  2024-04-26T23:56:09.763 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "sixmegs", 2024-04-26T23:56:09.919 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 8 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "subdir", 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 11 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 28, 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:09.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:09.938 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:56:09.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:10.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:10.169 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:56:10.169 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:10.169 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:10.169 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:10.231 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:10.231 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:10.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.484+0000 7f6d9f400640 1 Processor -- start 2024-04-26T23:56:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.484+0000 7f6d9f400640 1 -- start start 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9f400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d9806eb60 0x7f6d9814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d9814d030 0x7f6d98072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9f400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d980b4940 con 0x7f6d9806e780 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6d980b4c20 con 0x7f6d9814d030 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9f400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6d980b4f00 con 0x7f6d9806eb60 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d9814d030 0x7f6d98072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d9814d030 0x7f6d98072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54154/0 (socket says 172.21.15.46:54154) 2024-04-26T23:56:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.486+0000 7f6d9d400640 1 -- 172.21.15.46:0/4041229733 learned_addr learned my addr 172.21.15.46:0/4041229733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:10.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.487+0000 7f6d9d400640 1 -- 172.21.15.46:0/4041229733 >> v2:172.21.15.142:3300/0 conn(0x7f6d9806eb60 msgr2=0x7f6d9814caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:10.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.487+0000 7f6d9d400640 1 --2- 172.21.15.46:0/4041229733 >> v2:172.21.15.142:3300/0 conn(0x7f6d9806eb60 0x7f6d9814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.487+0000 7f6d9d400640 1 -- 172.21.15.46:0/4041229733 >> v1:172.21.15.142:6789/0 conn(0x7f6d9806e780 legacy=0x7f6d980b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.487+0000 7f6d9d400640 1 -- 172.21.15.46:0/4041229733 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d980b51e0 con 0x7f6d9814d030 2024-04-26T23:56:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.488+0000 7f6d9d400640 1 --2- 172.21.15.46:0/4041229733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d98072360 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6d980b6550 tx=0x7f6d9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cb16d93768600e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.488+0000 7f6d97400640 1 -- 172.21.15.46:0/4041229733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90036070 con 0x7f6d9814d030 2024-04-26T23:56:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d97400640 1 -- 172.21.15.46:0/4041229733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90031600 con 0x7f6d9814d030 2024-04-26T23:56:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d97400640 1 -- 172.21.15.46:0/4041229733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900319d0 con 0x7f6d9814d030 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 -- 172.21.15.46:0/4041229733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 msgr2=0x7f6d98072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 --2- 172.21.15.46:0/4041229733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d98072360 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6d980b6550 tx=0x7f6d9002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 -- 172.21.15.46:0/4041229733 shutdown_connections 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 --2- 172.21.15.46:0/4041229733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d98072360 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 --2- 172.21.15.46:0/4041229733 >> v2:172.21.15.142:3300/0 conn(0x7f6d9806eb60 0x7f6d9814caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 -- 172.21.15.46:0/4041229733 >> 172.21.15.46:0/4041229733 conn(0x7f6d9806c530 msgr2=0x7f6d9806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 -- 172.21.15.46:0/4041229733 shutdown_connections 2024-04-26T23:56:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.489+0000 7f6d9f400640 1 -- 172.21.15.46:0/4041229733 wait complete. 2024-04-26T23:56:10.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.491+0000 7f6d9f400640 1 Processor -- start 2024-04-26T23:56:10.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.491+0000 7f6d9f400640 1 -- start start 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 0x7f6d98149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 0x7f6d98073270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980b4590 con 0x7f6d9814d030 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980728a0 con 0x7f6d9806eb60 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9815f0f0 con 0x7f6d9806e780 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.492+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54160/0 (socket says 172.21.15.46:54160) 2024-04-26T23:56:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 -- 172.21.15.46:0/3055376447 learned_addr learned my addr 172.21.15.46:0/3055376447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 -- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 msgr2=0x7f6d98149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9ca00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 0x7f6d98149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d97e00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 0x7f6d98073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 0x7f6d98149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 -- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 msgr2=0x7f6d98073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 0x7f6d98073270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 -- 172.21.15.46:0/3055376447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d981fa080 con 0x7f6d9814d030 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9ca00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 0x7f6d98149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d97e00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 0x7f6d98073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.493+0000 7f6d9d400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f6d900093e0 tx=0x7f6d90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.494+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90041020 con 0x7f6d9814d030 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.494+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90037040 con 0x7f6d9814d030 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.494+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90036070 con 0x7f6d9814d030 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.494+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d981f92c0 con 0x7f6d9814d030 2024-04-26T23:56:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.494+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981f9880 con 0x7f6d9814d030 2024-04-26T23:56:10.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.495+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d98069530 con 0x7f6d9814d030 2024-04-26T23:56:10.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.496+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d90044020 con 0x7f6d9814d030 2024-04-26T23:56:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.497+0000 7f6d94c00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 0x7f6d7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.497+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f6d900aca00 con 0x7f6d9814d030 2024-04-26T23:56:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.498+0000 7f6d9ca00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 0x7f6d7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.499+0000 7f6d9ca00640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 0x7f6d7806ede0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6d98161200 tx=0x7f6d8c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.502+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d9004b030 con 0x7f6d9814d030 2024-04-26T23:56:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.706+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d9814cc00 con 0x7f6d9814d030 2024-04-26T23:56:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.710+0000 7f6d94c00640 1 -- 172.21.15.46:0/3055376447 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17001 (secure 0 0 0) 0x7f6d9007ad00 con 0x7f6d9814d030 2024-04-26T23:56:10.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:10.713 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":188,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:58.805182+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.25","mappings":[{"from":6,"to":4}]},{"pgid":"28.38","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 msgr2=0x7f6d7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 0x7f6d7806ede0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6d98161200 tx=0x7f6d8c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 msgr2=0x7f6d980737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f6d900093e0 tx=0x7f6d90004a40 comp rx=0 tx=0).stop 2024-04-26T23:56:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 shutdown_connections 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6d7806c920 0x7f6d7806ede0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9814d030 0x7f6d980737b0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9806eb60 0x7f6d98073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 --2- 172.21.15.46:0/3055376447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9806e780 0x7f6d98149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.712+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 >> 172.21.15.46:0/3055376447 conn(0x7f6d9806c530 msgr2=0x7f6d980d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.713+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 shutdown_connections 2024-04-26T23:56:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.713+0000 7f6d9f400640 1 -- 172.21.15.46:0/3055376447 wait complete. 2024-04-26T23:56:10.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:55:24.945034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:55:26.104963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:10.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.945+0000 7f23a7e00640 1 Processor -- start 2024-04-26T23:56:10.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.945+0000 7f23a7e00640 1 -- start start 2024-04-26T23:56:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.946+0000 7f23a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23a81626c0 0x7f23a80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.946+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a8162aa0 0x7f23a80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a80d07d0 con 0x7f23a80b5fb0 2024-04-26T23:56:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f23a80d0ab0 con 0x7f23a81626c0 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23a80d0d90 con 0x7f23a8162aa0 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23a81626c0 0x7f23a80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f23a80b5fb0 0x7f23a80d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34064/0 (socket says 172.21.15.46:34064) 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6000640 1 -- 172.21.15.46:0/1673788788 learned_addr learned my addr 172.21.15.46:0/1673788788 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6a00640 1 -- 172.21.15.46:0/1673788788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a8162aa0 msgr2=0x7f23a80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6a00640 1 --2- 172.21.15.46:0/1673788788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a8162aa0 0x7f23a80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6a00640 1 -- 172.21.15.46:0/1673788788 >> v1:172.21.15.46:6789/0 conn(0x7f23a80b5fb0 legacy=0x7f23a80d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.947+0000 7f23a6a00640 1 -- 172.21.15.46:0/1673788788 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a80d2c80 con 0x7f23a81626c0 2024-04-26T23:56:10.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.948+0000 7f23a6a00640 1 --2- 172.21.15.46:0/1673788788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a81626c0 0x7f23a80b47c0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f23980097a0 tx=0x7f239802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803f19ea5c828abb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.949+0000 7f23a5600640 1 -- 172.21.15.46:0/1673788788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398037070 con 0x7f23a81626c0 2024-04-26T23:56:10.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.949+0000 7f23a5600640 1 -- 172.21.15.46:0/1673788788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2398032440 con 0x7f23a81626c0 2024-04-26T23:56:10.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.949+0000 7f23a5600640 1 -- 172.21.15.46:0/1673788788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398032810 con 0x7f23a81626c0 2024-04-26T23:56:10.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 -- 172.21.15.46:0/1673788788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a81626c0 msgr2=0x7f23a80b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 --2- 172.21.15.46:0/1673788788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a81626c0 0x7f23a80b47c0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f23980097a0 tx=0x7f239802a180 comp rx=0 tx=0).stop 2024-04-26T23:56:10.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 -- 172.21.15.46:0/1673788788 shutdown_connections 2024-04-26T23:56:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 --2- 172.21.15.46:0/1673788788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a8162aa0 0x7f23a80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 --2- 172.21.15.46:0/1673788788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a81626c0 0x7f23a80b47c0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 -- 172.21.15.46:0/1673788788 >> 172.21.15.46:0/1673788788 conn(0x7f23a809e2a0 msgr2=0x7f23a80d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 -- 172.21.15.46:0/1673788788 shutdown_connections 2024-04-26T23:56:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.950+0000 7f23a7e00640 1 -- 172.21.15.46:0/1673788788 wait complete. 2024-04-26T23:56:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.952+0000 7f23a7e00640 1 Processor -- start 2024-04-26T23:56:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.952+0000 7f23a7e00640 1 -- start start 2024-04-26T23:56:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a80b5fb0 0x7f23a81563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a81626c0 0x7f23a815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23a80d2210 con 0x7f23a8162aa0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f23a80d2090 con 0x7f23a81626c0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f23a80d31a0 con 0x7f23a80b5fb0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54170/0 (socket says 172.21.15.46:54170) 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a7400640 1 -- 172.21.15.46:0/2001376431 learned_addr learned my addr 172.21.15.46:0/2001376431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.954+0000 7f23a6a00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a80b5fb0 0x7f23a81563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a6000640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a81626c0 0x7f23a815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 -- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a80b5fb0 msgr2=0x7f23a81563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a80b5fb0 0x7f23a81563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 -- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a81626c0 msgr2=0x7f23a815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a81626c0 0x7f23a815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 -- 172.21.15.46:0/2001376431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a80cb760 con 0x7f23a8162aa0 2024-04-26T23:56:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f23a7400640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b970 tx=0x7f23a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0019070 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a000cb60 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.955+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0014440 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.956+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a8156900 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.956+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a80caec0 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.957+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a80693b0 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.958+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23a001c070 con 0x7f23a8162aa0 2024-04-26T23:56:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.959+0000 7f239f400640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:10.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.959+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f23a008bb10 con 0x7f23a8162aa0 2024-04-26T23:56:10.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.960+0000 7f23a6a00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.961+0000 7f23a6a00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 0x7f238406edb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2398034000 tx=0x7f239802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:10.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:10.965+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23a0059cc0 con 0x7f23a8162aa0 2024-04-26T23:56:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.172+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23a80b41f0 con 0x7f23a8162aa0 2024-04-26T23:56:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.173+0000 7f239f400640 1 -- 172.21.15.46:0/2001376431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) ==== 93+0+10 (secure 0 0 0) 0x7f23a00595c0 con 0x7f23a8162aa0 2024-04-26T23:56:11.176 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 msgr2=0x7f238406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 0x7f238406edb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2398034000 tx=0x7f239802e000 comp rx=0 tx=0).stop 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 msgr2=0x7f23a80b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b970 tx=0x7f23a000be30 comp rx=0 tx=0).stop 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 shutdown_connections 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a8162aa0 0x7f23a80b1b90 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a81626c0 0x7f23a815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 --2- 172.21.15.46:0/2001376431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a80b5fb0 0x7f23a81563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 >> 172.21.15.46:0/2001376431 conn(0x7f23a809e2a0 msgr2=0x7f23a806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 shutdown_connections 2024-04-26T23:56:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.176+0000 7f23a7e00640 1 -- 172.21.15.46:0/2001376431 wait complete. 2024-04-26T23:56:11.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:56:11.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.413+0000 7f5a4f200640 1 Processor -- start 2024-04-26T23:56:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.413+0000 7f5a4f200640 1 -- start start 2024-04-26T23:56:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.414+0000 7f5a4f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a500cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.414+0000 7f5a4f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a500cd610 0x7f5a500cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.414+0000 7f5a4f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a500af570 con 0x7f5a500ae8f0 2024-04-26T23:56:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.414+0000 7f5a4f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5a500cdfb0 con 0x7f5a500cd610 2024-04-26T23:56:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.414+0000 7f5a4f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a500af850 con 0x7f5a5015f210 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.415+0000 7f5a4e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5a500ae8f0 0x7f5a500aed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34074/0 (socket says 172.21.15.46:34074) 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.415+0000 7f5a4e800640 1 -- 172.21.15.46:0/4248178496 learned_addr learned my addr 172.21.15.46:0/4248178496 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.415+0000 7f5a4de00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a500cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.415+0000 7f5a47e00640 1 --2- 172.21.15.46:0/4248178496 >> v2:172.21.15.142:3300/0 conn(0x7f5a500cd610 0x7f5a500cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.415+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544519097 0 0) 0x7f5a500af570 con 0x7f5a500ae8f0 2024-04-26T23:56:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a34003520 con 0x7f5a500ae8f0 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1120731340 0 0) 0x7f5a34003520 con 0x7f5a500ae8f0 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 msgr2=0x7f5a500cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a500cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 >> v2:172.21.15.142:3300/0 conn(0x7f5a500cd610 msgr2=0x7f5a500cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 --2- 172.21.15.46:0/4248178496 >> v2:172.21.15.142:3300/0 conn(0x7f5a500cd610 0x7f5a500cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a500cb680 con 0x7f5a500ae8f0 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f5a480019f0 con 0x7f5a500ae8f0 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 >> v1:172.21.15.46:6789/0 conn(0x7f5a500ae8f0 legacy=0x7f5a500aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a34004d40 0x7f5a340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500af850 con 0x7f5a34003f30 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3400b9c0 con 0x7f5a34004d40 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3400bb40 con 0x7f5a5015f210 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.416+0000 7f5a47e00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a4de00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a4e800640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a34004d40 0x7f5a340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a47e00640 1 -- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 msgr2=0x7f5a340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a47e00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a47e00640 1 -- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a34004d40 msgr2=0x7f5a340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a47e00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a34004d40 0x7f5a340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a47e00640 1 -- 172.21.15.46:0/4248178496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a34003520 con 0x7f5a34003f30 2024-04-26T23:56:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.417+0000 7f5a4de00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5015f210 0x7f5a340039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.418+0000 7f5a47e00640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a34004800 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ac0 tx=0x7f5a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a99af1b953656f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.418+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38036070 con 0x7f5a34003f30 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.418+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a38031400 con 0x7f5a34003f30 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.418+0000 7f5a4d400640 1 -- 172.21.15.46:0/4248178496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38009690 con 0x7f5a34003f30 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 -- 172.21.15.46:0/4248178496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 msgr2=0x7f5a34004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 --2- 172.21.15.46:0/4248178496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a34004800 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ac0 tx=0x7f5a34005830 comp rx=0 tx=0).stop 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4e800640 1 -- 172.21.15.46:0/4248178496 reap_dead start 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 -- 172.21.15.46:0/4248178496 shutdown_connections 2024-04-26T23:56:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 -- 172.21.15.46:0/4248178496 >> 172.21.15.46:0/4248178496 conn(0x7f5a500b27b0 msgr2=0x7f5a500b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 -- 172.21.15.46:0/4248178496 shutdown_connections 2024-04-26T23:56:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.419+0000 7f5a4f200640 1 -- 172.21.15.46:0/4248178496 wait complete. 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.421+0000 7f5a4f200640 1 Processor -- start 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.421+0000 7f5a4f200640 1 -- start start 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a500c5780 0x7f5a500c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 0x7f5a500b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5015f5f0 con 0x7f5a34003f30 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500cbc40 con 0x7f5a500c5780 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a500cc160 con 0x7f5a500b42a0 2024-04-26T23:56:11.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54190/0 (socket says 172.21.15.46:54190) 2024-04-26T23:56:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.422+0000 7f5a4de00640 1 -- 172.21.15.46:0/2029964126 learned_addr learned my addr 172.21.15.46:0/2029964126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4e800640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 0x7f5a500b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 -- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 msgr2=0x7f5a500b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 0x7f5a500b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 -- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a500c5780 msgr2=0x7f5a500c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a500c5780 0x7f5a500c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 -- 172.21.15.46:0/2029964126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a500c5f40 con 0x7f5a34003f30 2024-04-26T23:56:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4e800640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 0x7f5a500b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4de00640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5a400108b0 tx=0x7f5a40010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000f040 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4000eb60 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4001c400 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.423+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a500c6220 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.424+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a500c67e0 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.425+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a500cc590 con 0x7f5a34003f30 2024-04-26T23:56:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.426+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a4001c5a0 con 0x7f5a34003f30 2024-04-26T23:56:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.427+0000 7f5a46000640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 0x7f5a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.427+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f5a4008c410 con 0x7f5a34003f30 2024-04-26T23:56:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.427+0000 7f5a47e00640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 0x7f5a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.428+0000 7f5a47e00640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 0x7f5a2c06edb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a38033000 tx=0x7f5a3802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.432+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a500cc590 con 0x7f5a34003f30 2024-04-26T23:56:11.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.636+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a500693b0 con 0x7f5a34003f30 2024-04-26T23:56:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.637+0000 7f5a46000640 1 -- 172.21.15.46:0/2029964126 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v188) ==== 104+0+11 (secure 0 0 0) 0x7f5a40018030 con 0x7f5a34003f30 2024-04-26T23:56:11.640 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.639+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 msgr2=0x7f5a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.639+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 0x7f5a2c06edb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a38033000 tx=0x7f5a3802e000 comp rx=0 tx=0).stop 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.639+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 msgr2=0x7f5a500cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.639+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5a400108b0 tx=0x7f5a40010d70 comp rx=0 tx=0).stop 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 shutdown_connections 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5a2c06c8f0 0x7f5a2c06edb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a500b42a0 0x7f5a500b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a500c5780 0x7f5a500c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 --2- 172.21.15.46:0/2029964126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a34003f30 0x7f5a500cadf0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 >> 172.21.15.46:0/2029964126 conn(0x7f5a500b27b0 msgr2=0x7f5a500cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 shutdown_connections 2024-04-26T23:56:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.640+0000 7f5a4f200640 1 -- 172.21.15.46:0/2029964126 wait complete. 2024-04-26T23:56:11.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:56:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.876+0000 7fd3b6000640 1 Processor -- start 2024-04-26T23:56:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.876+0000 7fd3b6000640 1 -- start start 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3b6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3b6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b00b4320 0x7fd3b00ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3b6000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b00b4b30 con 0x7fd3b00d04c0 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3b6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b00b4e10 con 0x7fd3b00b4320 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3b6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b00b50f0 con 0x7fd3b006f4b0 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b00b4320 0x7fd3b00ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b00b4320 0x7fd3b00ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50370/0 (socket says 172.21.15.46:50370) 2024-04-26T23:56:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.878+0000 7fd3afe00640 1 -- 172.21.15.46:0/2074066454 learned_addr learned my addr 172.21.15.46:0/2074066454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:11.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3aea00640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3afe00640 1 -- 172.21.15.46:0/2074066454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 msgr2=0x7fd3b00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3afe00640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3afe00640 1 -- 172.21.15.46:0/2074066454 >> v1:172.21.15.142:6789/0 conn(0x7fd3b00d04c0 legacy=0x7fd3b006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3afe00640 1 -- 172.21.15.46:0/2074066454 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b00b53d0 con 0x7fd3b00b4320 2024-04-26T23:56:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.879+0000 7fd3aea00640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.880+0000 7fd3afe00640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b00ce470 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009e50 tx=0x7fd3a802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c02c9afc6052453 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.880+0000 7fd3ae000640 1 -- 172.21.15.46:0/2074066454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8036070 con 0x7fd3b00b4320 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.880+0000 7fd3ae000640 1 -- 172.21.15.46:0/2074066454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a8031600 con 0x7fd3b00b4320 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3ae000640 1 -- 172.21.15.46:0/2074066454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80319d0 con 0x7fd3b00b4320 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 -- 172.21.15.46:0/2074066454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 msgr2=0x7fd3b00ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b00ce470 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009e50 tx=0x7fd3a802a630 comp rx=0 tx=0).stop 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 -- 172.21.15.46:0/2074066454 shutdown_connections 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b00ce470 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 --2- 172.21.15.46:0/2074066454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 -- 172.21.15.46:0/2074066454 >> 172.21.15.46:0/2074066454 conn(0x7fd3b006c980 msgr2=0x7fd3b00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 -- 172.21.15.46:0/2074066454 shutdown_connections 2024-04-26T23:56:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.881+0000 7fd3b6000640 1 -- 172.21.15.46:0/2074066454 wait complete. 2024-04-26T23:56:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.883+0000 7fd3b6000640 1 Processor -- start 2024-04-26T23:56:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.883+0000 7fd3b6000640 1 -- start start 2024-04-26T23:56:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00d04c0 0x7fd3b00ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b00b48d0 con 0x7fd3b00b4320 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b00b4750 con 0x7fd3b00d04c0 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.884+0000 7fd3b6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b00ce9b0 con 0x7fd3b006f4b0 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50372/0 (socket says 172.21.15.46:50372) 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 -- 172.21.15.46:0/3639873396 learned_addr learned my addr 172.21.15.46:0/3639873396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3afe00640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00d04c0 0x7fd3b00ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3af400640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 -- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 msgr2=0x7fd3b00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 -- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00d04c0 msgr2=0x7fd3b00ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00d04c0 0x7fd3b00ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 -- 172.21.15.46:0/3639873396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b00ccc30 con 0x7fd3b00b4320 2024-04-26T23:56:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.885+0000 7fd3aea00640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c820 tx=0x7fd39c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.886+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c015070 con 0x7fd3b00b4320 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.886+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c007a30 con 0x7fd3b00b4320 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.886+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c007e00 con 0x7fd3b00b4320 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.886+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b015bee0 con 0x7fd3b00b4320 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.886+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b00cc950 con 0x7fd3b00b4320 2024-04-26T23:56:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.887+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b0069530 con 0x7fd3b00b4320 2024-04-26T23:56:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.888+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd39c018070 con 0x7fd3b00b4320 2024-04-26T23:56:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.889+0000 7fd3a7400640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 0x7fd39006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.889+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7fd39c08bda0 con 0x7fd3b00b4320 2024-04-26T23:56:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.890+0000 7fd3af400640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 0x7fd39006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.891+0000 7fd3af400640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 0x7fd39006edb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000a750 tx=0x7fd3a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:11.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:11.894+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3b015bee0 con 0x7fd3b00b4320 2024-04-26T23:56:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.094+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3b0076550 con 0x7fd3b00b4320 2024-04-26T23:56:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.095+0000 7fd3a7400640 1 -- 172.21.15.46:0/3639873396 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v188) ==== 100+0+11 (secure 0 0 0) 0x7fd39c059f50 con 0x7fd3b00b4320 2024-04-26T23:56:12.098 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:56:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 msgr2=0x7fd39006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 0x7fd39006edb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000a750 tx=0x7fd3a0008040 comp rx=0 tx=0).stop 2024-04-26T23:56:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 msgr2=0x7fd3b015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c820 tx=0x7fd39c00cc20 comp rx=0 tx=0).stop 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 shutdown_connections 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39006c8f0 0x7fd39006edb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00d04c0 0x7fd3b00ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b00b4320 0x7fd3b015b9a0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 --2- 172.21.15.46:0/3639873396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b006f4b0 0x7fd3b00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.098+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 >> 172.21.15.46:0/3639873396 conn(0x7fd3b006c980 msgr2=0x7fd3b015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.099+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 shutdown_connections 2024-04-26T23:56:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.099+0000 7fd3b6000640 1 -- 172.21.15.46:0/3639873396 wait complete. 2024-04-26T23:56:12.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:56:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.334+0000 7f58c3200640 1 Processor -- start 2024-04-26T23:56:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.334+0000 7f58c3200640 1 -- start start 2024-04-26T23:56:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.335+0000 7f58c3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58c40d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c40d0990 con 0x7f58c40d2010 2024-04-26T23:56:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c3200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c40d0c70 con 0x7f58c415f5e0 2024-04-26T23:56:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f58c40d0f50 con 0x7f58c415f200 2024-04-26T23:56:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58c40d2010 0x7f58c40d2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49000/0 (socket says 172.21.15.46:49000) 2024-04-26T23:56:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c2800640 1 -- 172.21.15.46:0/312883491 learned_addr learned my addr 172.21.15.46:0/312883491 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c1e00640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58c40d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.336+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049985202 0 0) 0x7f58c40d0990 con 0x7f58c40d2010 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58a8003520 con 0x7f58c40d2010 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3237321065 0 0) 0x7f58a8003520 con 0x7f58c40d2010 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 msgr2=0x7f58c40d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58c40d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 >> v1:172.21.15.142:6789/0 conn(0x7f58c415f5e0 legacy=0x7f58c40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c40d1230 con 0x7f58c40d2010 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f58bc0019f0 con 0x7f58c40d2010 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 >> v1:172.21.15.46:6789/0 conn(0x7f58c40d2010 legacy=0x7f58c40d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58a80039e0 0x7f58a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a8004d40 0x7f58a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40d0f50 con 0x7f58a8004d40 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800bba0 con 0x7f58a80039e0 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f58a800bda0 con 0x7f58c415f200 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.337+0000 7f58c2800640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a8004d40 0x7f58a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c1e00640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58a80039e0 0x7f58a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58bbe00640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c2800640 1 -- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 msgr2=0x7f58a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c2800640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58c415f200 0x7f58a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c2800640 1 -- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58a80039e0 msgr2=0x7f58a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c2800640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58a80039e0 0x7f58a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.338+0000 7f58c2800640 1 -- 172.21.15.46:0/312883491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a8003520 con 0x7f58a8004d40 2024-04-26T23:56:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.339+0000 7f58c2800640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a8004d40 0x7f58a80051c0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cf4f0 tx=0x7f58a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e97cabe44a1e0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.339+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00a390 con 0x7f58a8004d40 2024-04-26T23:56:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.339+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc02f070 con 0x7f58a8004d40 2024-04-26T23:56:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.339+0000 7f58c1400640 1 -- 172.21.15.46:0/312883491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc02bb90 con 0x7f58a8004d40 2024-04-26T23:56:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 -- 172.21.15.46:0/312883491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a8004d40 msgr2=0x7f58a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 --2- 172.21.15.46:0/312883491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a8004d40 0x7f58a80051c0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cf4f0 tx=0x7f58a8005830 comp rx=0 tx=0).stop 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c2800640 1 -- 172.21.15.46:0/312883491 reap_dead start 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 -- 172.21.15.46:0/312883491 shutdown_connections 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 -- 172.21.15.46:0/312883491 >> 172.21.15.46:0/312883491 conn(0x7f58c40d4ef0 msgr2=0x7f58c40d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 -- 172.21.15.46:0/312883491 shutdown_connections 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.340+0000 7f58c3200640 1 -- 172.21.15.46:0/312883491 wait complete. 2024-04-26T23:56:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.342+0000 7f58c3200640 1 Processor -- start 2024-04-26T23:56:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.342+0000 7f58c3200640 1 -- start start 2024-04-26T23:56:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a8004d40 0x7f58c415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c41f52a0 0x7f58c415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40b2f50 con 0x7f58c41f4e20 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40b2dd0 con 0x7f58c41f52a0 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.343+0000 7f58c3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f58c40b30d0 con 0x7f58a8004d40 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50390/0 (socket says 172.21.15.46:50390) 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 -- 172.21.15.46:0/1058558798 learned_addr learned my addr 172.21.15.46:0/1058558798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58c1e00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a8004d40 0x7f58c415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58c2800640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c41f52a0 0x7f58c415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 -- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a8004d40 msgr2=0x7f58c415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a8004d40 0x7f58c415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 -- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c41f52a0 msgr2=0x7f58c415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c41f52a0 0x7f58c415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 -- 172.21.15.46:0/1058558798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c41f5870 con 0x7f58c41f4e20 2024-04-26T23:56:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.344+0000 7f58bbe00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e550 tx=0x7f58ac00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.345+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0052e0 con 0x7f58c41f4e20 2024-04-26T23:56:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.345+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac018070 con 0x7f58c41f4e20 2024-04-26T23:56:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.345+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac003810 con 0x7f58c41f4e20 2024-04-26T23:56:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.345+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c41f5b50 con 0x7f58c41f4e20 2024-04-26T23:56:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.345+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c415cad0 con 0x7f58c41f4e20 2024-04-26T23:56:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.346+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c40693b0 con 0x7f58c41f4e20 2024-04-26T23:56:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.347+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58ac0039b0 con 0x7f58c41f4e20 2024-04-26T23:56:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.348+0000 7f58ba000640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 0x7f58a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.348+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f58ac08ba70 con 0x7f58c41f4e20 2024-04-26T23:56:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.349+0000 7f58c1e00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 0x7f58a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.350+0000 7f58c1e00640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 0x7f58a006edb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f58b4006fd0 tx=0x7f58b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.353+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58ac059c20 con 0x7f58c41f4e20 2024-04-26T23:56:12.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.572+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58c40cf2b0 con 0x7f58c41f4e20 2024-04-26T23:56:12.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.573+0000 7f58ba000640 1 -- 172.21.15.46:0/1058558798 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v188) ==== 80+0+0 (secure 0 0 0) 0x7f58ac059520 con 0x7f58c41f4e20 2024-04-26T23:56:12.576 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:56:12.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.575+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 msgr2=0x7f58a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.575+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 0x7f58a006edb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f58b4006fd0 tx=0x7f58b4008040 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.575+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 msgr2=0x7f58c415ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.575+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e550 tx=0x7f58ac00ea10 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 shutdown_connections 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f58a006c8f0 0x7f58a006edb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c41f52a0 0x7f58c415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c41f4e20 0x7f58c415ce40 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 --2- 172.21.15.46:0/1058558798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a8004d40 0x7f58c415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 >> 172.21.15.46:0/1058558798 conn(0x7f58c40d4ef0 msgr2=0x7f58c40b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 shutdown_connections 2024-04-26T23:56:12.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:12.576+0000 7f58c3200640 1 -- 172.21.15.46:0/1058558798 wait complete. 2024-04-26T23:56:12.600 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:56:12.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:12.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:12.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:56:12.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:12.713 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:56:12.773 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:12.773 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:12.773 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:12.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:12.813 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:12.813 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:12.813 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:12.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:12.853 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:12.853 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:12.854 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:12.900 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:12.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.106+0000 7f646d200640 1 Processor -- start 2024-04-26T23:56:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.107+0000 7f646d200640 1 -- start start 2024-04-26T23:56:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.108+0000 7f646d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f646806e510 0x7f646806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.108+0000 7f646d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f6468073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f646d200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646806ee30 con 0x7f6468160bd0 2024-04-26T23:56:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f646d200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6468161940 con 0x7f646806e510 2024-04-26T23:56:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f646d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6468071f80 con 0x7f64680732c0 2024-04-26T23:56:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f6466800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f6468073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f6465e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f646806e510 0x7f646806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f6465400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f6468160bd0 0x7f6468161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47380/0 (socket says 172.21.15.46:47380) 2024-04-26T23:56:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.109+0000 7f6465400640 1 -- 172.21.15.46:0/1382875353 learned_addr learned my addr 172.21.15.46:0/1382875353 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.110+0000 7f6465e00640 1 -- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 msgr2=0x7f6468073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.110+0000 7f6465e00640 1 --2- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f6468073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.110+0000 7f6465e00640 1 -- 172.21.15.46:0/1382875353 >> v1:172.21.15.142:6789/0 conn(0x7f6468160bd0 legacy=0x7f6468161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.110+0000 7f6465e00640 1 -- 172.21.15.46:0/1382875353 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6468072260 con 0x7f646806e510 2024-04-26T23:56:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.112+0000 7f6465e00640 1 --2- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646806e510 0x7f646806e8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64580097d0 tx=0x7f645802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999097eae18b93fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.112+0000 7f6464a00640 1 -- 172.21.15.46:0/1382875353 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458037070 con 0x7f646806e510 2024-04-26T23:56:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.112+0000 7f6464a00640 1 -- 172.21.15.46:0/1382875353 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458032440 con 0x7f646806e510 2024-04-26T23:56:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.112+0000 7f6464a00640 1 -- 172.21.15.46:0/1382875353 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458032810 con 0x7f646806e510 2024-04-26T23:56:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 -- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646806e510 msgr2=0x7f646806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 --2- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646806e510 0x7f646806e8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64580097d0 tx=0x7f645802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 -- 172.21.15.46:0/1382875353 shutdown_connections 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 --2- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f6468073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 --2- 172.21.15.46:0/1382875353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646806e510 0x7f646806e8f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 -- 172.21.15.46:0/1382875353 >> 172.21.15.46:0/1382875353 conn(0x7f646806c530 msgr2=0x7f646806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 -- 172.21.15.46:0/1382875353 shutdown_connections 2024-04-26T23:56:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.113+0000 7f646d200640 1 -- 172.21.15.46:0/1382875353 wait complete. 2024-04-26T23:56:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.116+0000 7f646d200640 1 Processor -- start 2024-04-26T23:56:13.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.116+0000 7f646d200640 1 -- start start 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f64681562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6468160bd0 0x7f6468157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6468072d40 con 0x7f646806e510 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6468072820 con 0x7f6468160bd0 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.117+0000 7f646d200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6468072ec0 con 0x7f64680732c0 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50404/0 (socket says 172.21.15.46:50404) 2024-04-26T23:56:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 -- 172.21.15.46:0/2701236972 learned_addr learned my addr 172.21.15.46:0/2701236972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6466800640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6468160bd0 0x7f6468157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465400640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f64681562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 -- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 msgr2=0x7f64681562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f64681562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 -- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6468160bd0 msgr2=0x7f6468157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6468160bd0 0x7f6468157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465e00640 1 -- 172.21.15.46:0/2701236972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6468156b00 con 0x7f646806e510 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.118+0000 7f6465400640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f64681562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f6465e00640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f645802a7b0 tx=0x7f64580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458037050 con 0x7f646806e510 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458043070 con 0x7f646806e510 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645803cb30 con 0x7f646806e510 2024-04-26T23:56:13.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6468156de0 con 0x7f646806e510 2024-04-26T23:56:13.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.119+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64680b15c0 con 0x7f646806e510 2024-04-26T23:56:13.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.120+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64680b13b0 con 0x7f646806e510 2024-04-26T23:56:13.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.121+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f645803ccd0 con 0x7f646806e510 2024-04-26T23:56:13.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.122+0000 7f645e000640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 0x7f644406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.122+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f64580acb30 con 0x7f646806e510 2024-04-26T23:56:13.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.123+0000 7f6465400640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 0x7f644406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.124+0000 7f6465400640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 0x7f644406edb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f64681578a0 tx=0x7f6450009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.127+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f645807ace0 con 0x7f646806e510 2024-04-26T23:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.373+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6468069530 con 0x7f646806e510 2024-04-26T23:56:13.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.374+0000 7f645e000640 1 -- 172.21.15.46:0/2701236972 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f645807a5e0 con 0x7f646806e510 2024-04-26T23:56:13.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:13.377 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:56:13.377 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 msgr2=0x7f644406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 0x7f644406edb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f64681578a0 tx=0x7f6450009040 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 msgr2=0x7f64681345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f645802a7b0 tx=0x7f64580046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 shutdown_connections 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f644406c8f0 0x7f644406edb0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6468160bd0 0x7f6468157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64680732c0 0x7f64681562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 --2- 172.21.15.46:0/2701236972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f646806e510 0x7f64681345c0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 >> 172.21.15.46:0/2701236972 conn(0x7f646806c530 msgr2=0x7f6468141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 shutdown_connections 2024-04-26T23:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.377+0000 7f646d200640 1 -- 172.21.15.46:0/2701236972 wait complete. 2024-04-26T23:56:13.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:13.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.614+0000 7fca4f200640 1 Processor -- start 2024-04-26T23:56:13.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.615+0000 7fca4f200640 1 -- start start 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.616+0000 7fca4f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca500720e0 0x7fca500cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.616+0000 7fca4f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca50073ad0 con 0x7fca501466b0 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.616+0000 7fca4f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca50146a90 con 0x7fca50160910 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.616+0000 7fca4f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fca50073d70 con 0x7fca500720e0 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca47e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca500720e0 0x7fca500cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca4de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca501466b0 0x7fca5006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49004/0 (socket says 172.21.15.46:49004) 2024-04-26T23:56:13.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca4de00640 1 -- 172.21.15.46:0/1957860901 learned_addr learned my addr 172.21.15.46:0/1957860901 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:13.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca47e00640 1 -- 172.21.15.46:0/1957860901 >> v1:172.21.15.142:6789/0 conn(0x7fca50160910 legacy=0x7fca500cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:13.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca47e00640 1 -- 172.21.15.46:0/1957860901 >> v1:172.21.15.46:6789/0 conn(0x7fca501466b0 legacy=0x7fca5006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca4d400640 1 -- 172.21.15.46:0/1957860901 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595311426 0 0) 0x7fca50073ad0 con 0x7fca501466b0 2024-04-26T23:56:13.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.617+0000 7fca47e00640 1 -- 172.21.15.46:0/1957860901 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca50074050 con 0x7fca500720e0 2024-04-26T23:56:13.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.618+0000 7fca47e00640 1 --2- 172.21.15.46:0/1957860901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca500cd860 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fca380097d0 tx=0x7fca3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a84f75b3503afce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.619+0000 7fca4d400640 1 -- 172.21.15.46:0/1957860901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38037070 con 0x7fca500720e0 2024-04-26T23:56:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.619+0000 7fca4d400640 1 -- 172.21.15.46:0/1957860901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca38032440 con 0x7fca500720e0 2024-04-26T23:56:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.619+0000 7fca4d400640 1 -- 172.21.15.46:0/1957860901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38032810 con 0x7fca500720e0 2024-04-26T23:56:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 -- 172.21.15.46:0/1957860901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 msgr2=0x7fca500cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 --2- 172.21.15.46:0/1957860901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca500cd860 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fca380097d0 tx=0x7fca3802a250 comp rx=0 tx=0).stop 2024-04-26T23:56:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 -- 172.21.15.46:0/1957860901 shutdown_connections 2024-04-26T23:56:13.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 --2- 172.21.15.46:0/1957860901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca500cd860 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 -- 172.21.15.46:0/1957860901 >> 172.21.15.46:0/1957860901 conn(0x7fca500b26d0 msgr2=0x7fca500b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:13.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 -- 172.21.15.46:0/1957860901 shutdown_connections 2024-04-26T23:56:13.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.620+0000 7fca4f200640 1 -- 172.21.15.46:0/1957860901 wait complete. 2024-04-26T23:56:13.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.622+0000 7fca4f200640 1 Processor -- start 2024-04-26T23:56:13.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.622+0000 7fca4f200640 1 -- start start 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.623+0000 7fca4f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.623+0000 7fca4f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca501466b0 0x7fca5015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 0x7fca500c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca500cdda0 con 0x7fca500720e0 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca500cb470 con 0x7fca50160910 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca500cc0a0 con 0x7fca501466b0 2024-04-26T23:56:13.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50426/0 (socket says 172.21.15.46:50426) 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 -- 172.21.15.46:0/1291721108 learned_addr learned my addr 172.21.15.46:0/1291721108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4e800640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 0x7fca500c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 -- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca501466b0 msgr2=0x7fca5015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca501466b0 0x7fca5015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 -- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 msgr2=0x7fca500c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 0x7fca500c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4de00640 1 -- 172.21.15.46:0/1291721108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca500c8720 con 0x7fca500720e0 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.624+0000 7fca4e800640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 0x7fca500c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:13.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca4de00640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c4a0 tx=0x7fca4000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40018070 con 0x7fca500720e0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca400074e0 con 0x7fca500720e0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca500c8a00 con 0x7fca500720e0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40004a20 con 0x7fca500720e0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.625+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5015d420 con 0x7fca500720e0 2024-04-26T23:56:13.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.626+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca500693b0 con 0x7fca500720e0 2024-04-26T23:56:13.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.627+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca4001b070 con 0x7fca500720e0 2024-04-26T23:56:13.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.628+0000 7fca46000640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:13.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.628+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7fca4008b150 con 0x7fca500720e0 2024-04-26T23:56:13.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.629+0000 7fca47e00640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:13.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.630+0000 7fca47e00640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 0x7fca2c06edb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fca38034000 tx=0x7fca3802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:13.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.633+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca40059390 con 0x7fca500720e0 2024-04-26T23:56:13.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:13.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:13.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.839+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca500c7180 con 0x7fca500720e0 2024-04-26T23:56:13.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.842+0000 7fca46000640 1 -- 172.21.15.46:0/1291721108 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17001 (secure 0 0 0) 0x7fca40058c90 con 0x7fca500720e0 2024-04-26T23:56:13.846 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:13.846 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":188,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:58.805182+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.25","mappings":[{"from":6,"to":4}]},{"pgid":"28.38","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 msgr2=0x7fca2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 0x7fca2c06edb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fca38034000 tx=0x7fca3802c000 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 msgr2=0x7fca501413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c4a0 tx=0x7fca4000c960 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 shutdown_connections 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca50160910 0x7fca500c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca501466b0 0x7fca5015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 --2- 172.21.15.46:0/1291721108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca500720e0 0x7fca501413a0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 >> 172.21.15.46:0/1291721108 conn(0x7fca500b26d0 msgr2=0x7fca50141f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 shutdown_connections 2024-04-26T23:56:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:13.845+0000 7fca4f200640 1 -- 172.21.15.46:0/1291721108 wait complete. 2024-04-26T23:56:13.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:55:24.945034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:55:26.104963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:13.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:14.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.122+0000 7f3bbf200640 1 Processor -- start 2024-04-26T23:56:14.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.122+0000 7f3bbf200640 1 -- start start 2024-04-26T23:56:14.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.124+0000 7f3bbf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3bc00cdd00 0x7f3bc00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.124+0000 7f3bbf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.124+0000 7f3bbf200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc00ce7d0 con 0x7f3bc015cec0 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.124+0000 7f3bbf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc00ceab0 con 0x7f3bc00cdd00 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.124+0000 7f3bbf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc00ced90 con 0x7f3bc0073b30 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:45576/0 (socket says 172.21.15.46:45576) 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 -- 172.21.15.46:0/1207801135 learned_addr learned my addr 172.21.15.46:0/1207801135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bb5400640 1 --2- 172.21.15.46:0/1207801135 >> v2:172.21.15.142:3300/0 conn(0x7f3bc00cdd00 0x7f3bc00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 -- 172.21.15.46:0/1207801135 >> v2:172.21.15.142:3300/0 conn(0x7f3bc00cdd00 msgr2=0x7f3bc00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 --2- 172.21.15.46:0/1207801135 >> v2:172.21.15.142:3300/0 conn(0x7f3bc00cdd00 0x7f3bc00735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.125+0000 7f3bbe800640 1 -- 172.21.15.46:0/1207801135 >> v1:172.21.15.142:6789/0 conn(0x7f3bc015cec0 legacy=0x7f3bc00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.126+0000 7f3bbe800640 1 -- 172.21.15.46:0/1207801135 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc00cf070 con 0x7f3bc0073b30 2024-04-26T23:56:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.127+0000 7f3bbe800640 1 --2- 172.21.15.46:0/1207801135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009870 tx=0x7f3bb802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc80c03fdf9b210c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.127+0000 7f3bbd400640 1 -- 172.21.15.46:0/1207801135 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8037070 con 0x7f3bc0073b30 2024-04-26T23:56:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.127+0000 7f3bbd400640 1 -- 172.21.15.46:0/1207801135 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb8032440 con 0x7f3bc0073b30 2024-04-26T23:56:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.127+0000 7f3bbd400640 1 -- 172.21.15.46:0/1207801135 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8032810 con 0x7f3bc0073b30 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 -- 172.21.15.46:0/1207801135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 msgr2=0x7f3bc00b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 --2- 172.21.15.46:0/1207801135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009870 tx=0x7f3bb802a270 comp rx=0 tx=0).stop 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 -- 172.21.15.46:0/1207801135 shutdown_connections 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 --2- 172.21.15.46:0/1207801135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b3b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 --2- 172.21.15.46:0/1207801135 >> v2:172.21.15.142:3300/0 conn(0x7f3bc00cdd00 0x7f3bc00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 -- 172.21.15.46:0/1207801135 >> 172.21.15.46:0/1207801135 conn(0x7f3bc00b27b0 msgr2=0x7f3bc00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 -- 172.21.15.46:0/1207801135 shutdown_connections 2024-04-26T23:56:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.128+0000 7f3bbf200640 1 -- 172.21.15.46:0/1207801135 wait complete. 2024-04-26T23:56:14.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.130+0000 7f3bbf200640 1 Processor -- start 2024-04-26T23:56:14.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.130+0000 7f3bbf200640 1 -- start start 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc015cec0 0x7f3bc0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0073f70 con 0x7f3bc00cdd00 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc015d2a0 con 0x7f3bc015cec0 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bbf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc00b4080 con 0x7f3bc0073b30 2024-04-26T23:56:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.131+0000 7f3bb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50442/0 (socket says 172.21.15.46:50442) 2024-04-26T23:56:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 -- 172.21.15.46:0/2535592602 learned_addr learned my addr 172.21.15.46:0/2535592602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:14.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bbe800640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc015cec0 0x7f3bc0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bbde00640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 -- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 msgr2=0x7f3bc00b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 -- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc015cec0 msgr2=0x7f3bc0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc015cec0 0x7f3bc0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 -- 172.21.15.46:0/2535592602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc00c7580 con 0x7f3bc00cdd00 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.132+0000 7f3bb5400640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800e9b0 tx=0x7f3ba800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.133+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8017070 con 0x7f3bc00cdd00 2024-04-26T23:56:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.133+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba800c9a0 con 0x7f3bc00cdd00 2024-04-26T23:56:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.133+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800cd90 con 0x7f3bc00cdd00 2024-04-26T23:56:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.133+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc00b17a0 con 0x7f3bc00cdd00 2024-04-26T23:56:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.133+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc00b11e0 con 0x7f3bc00cdd00 2024-04-26T23:56:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.134+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc00693b0 con 0x7f3bc00cdd00 2024-04-26T23:56:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.135+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ba801a070 con 0x7f3bc00cdd00 2024-04-26T23:56:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.136+0000 7f3bb6a00640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 0x7f3b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.136+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f3ba808bd80 con 0x7f3bc00cdd00 2024-04-26T23:56:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.137+0000 7f3bbde00640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 0x7f3b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.138+0000 7f3bbde00640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 0x7f3b9c06edb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00012e0 tx=0x7f3bb0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.141+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ba805a060 con 0x7f3bc00cdd00 2024-04-26T23:56:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.350+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bc0072740 con 0x7f3bc00cdd00 2024-04-26T23:56:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.351+0000 7f3bb6a00640 1 -- 172.21.15.46:0/2535592602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) ==== 93+0+10 (secure 0 0 0) 0x7f3ba8059960 con 0x7f3bc00cdd00 2024-04-26T23:56:14.354 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 msgr2=0x7f3b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 0x7f3b9c06edb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00012e0 tx=0x7f3bb0009040 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 msgr2=0x7f3bc00c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800e9b0 tx=0x7f3ba800ee70 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 shutdown_connections 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b9c06c8f0 0x7f3b9c06edb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc015cec0 0x7f3bc0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc00cdd00 0x7f3bc00c7040 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 --2- 172.21.15.46:0/2535592602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc0073b30 0x7f3bc00b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.353+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 >> 172.21.15.46:0/2535592602 conn(0x7f3bc00b27b0 msgr2=0x7f3bc00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.354+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 shutdown_connections 2024-04-26T23:56:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.354+0000 7f3bbf200640 1 -- 172.21.15.46:0/2535592602 wait complete. 2024-04-26T23:56:14.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:56:14.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.591+0000 7fcf81800640 1 Processor -- start 2024-04-26T23:56:14.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.591+0000 7fcf81800640 1 -- start start 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf81800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf7c0b5fb0 0x7fcf7c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf81800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7c0d07d0 con 0x7fcf7c1626c0 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf81800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7c0d0ab0 con 0x7fcf7c162aa0 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf81800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7c0d0d90 con 0x7fcf7c0b5fb0 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf7be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf7c1626c0 0x7fcf7c0d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49010/0 (socket says 172.21.15.46:49010) 2024-04-26T23:56:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.593+0000 7fcf7be00640 1 -- 172.21.15.46:0/642471013 learned_addr learned my addr 172.21.15.46:0/642471013 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7b400640 1 --2- 172.21.15.46:0/642471013 >> v2:172.21.15.142:3300/0 conn(0x7fcf7c0b5fb0 0x7fcf7c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764521203 0 0) 0x7fcf7c0d07d0 con 0x7fcf7c1626c0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf60003520 con 0x7fcf7c1626c0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2979685874 0 0) 0x7fcf60003520 con 0x7fcf7c1626c0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 >> v2:172.21.15.142:3300/0 conn(0x7fcf7c0b5fb0 msgr2=0x7fcf7c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 --2- 172.21.15.46:0/642471013 >> v2:172.21.15.142:3300/0 conn(0x7fcf7c0b5fb0 0x7fcf7c0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 >> v1:172.21.15.142:6789/0 conn(0x7fcf7c162aa0 legacy=0x7fcf7c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c0d2c80 con 0x7fcf7c1626c0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcf6c001670 con 0x7fcf7c1626c0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 >> v1:172.21.15.46:6789/0 conn(0x7fcf7c1626c0 legacy=0x7fcf7c0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 0x7fcf60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60004800 0x7fcf60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf600051c0 0x7fcf6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c0d0d90 con 0x7fcf600039b0 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6000c060 con 0x7fcf60004800 2024-04-26T23:56:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.594+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6000c260 con 0x7fcf600051c0 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 0x7fcf60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 -- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf600051c0 msgr2=0x7fcf6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf600051c0 0x7fcf6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 -- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60004800 msgr2=0x7fcf60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60004800 0x7fcf60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf7be00640 1 -- 172.21.15.46:0/642471013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60003520 con 0x7fcf600039b0 2024-04-26T23:56:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.595+0000 7fcf80e00640 1 -- 172.21.15.46:0/642471013 reap_dead start 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.596+0000 7fcf7be00640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 0x7fcf60003dd0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c210 tx=0x7fcf60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8915742c9aa8b285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c007d30 con 0x7fcf600039b0 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c005ad0 con 0x7fcf600039b0 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf7aa00640 1 -- 172.21.15.46:0/642471013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c007070 con 0x7fcf600039b0 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf81800640 1 -- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 msgr2=0x7fcf60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf81800640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 0x7fcf60003dd0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c210 tx=0x7fcf60005830 comp rx=0 tx=0).stop 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf81800640 1 -- 172.21.15.46:0/642471013 shutdown_connections 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf81800640 1 --2- 172.21.15.46:0/642471013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf600039b0 0x7fcf60003dd0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.597+0000 7fcf81800640 1 -- 172.21.15.46:0/642471013 >> 172.21.15.46:0/642471013 conn(0x7fcf7c09e2a0 msgr2=0x7fcf7c0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.598+0000 7fcf81800640 1 -- 172.21.15.46:0/642471013 shutdown_connections 2024-04-26T23:56:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.598+0000 7fcf81800640 1 -- 172.21.15.46:0/642471013 wait complete. 2024-04-26T23:56:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.599+0000 7fcf81800640 1 Processor -- start 2024-04-26T23:56:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 -- start start 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf600039b0 0x7fcf7c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7c0c9450 0x7fcf7c1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c0b4b50 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c0b49d0 con 0x7fcf600039b0 2024-04-26T23:56:14.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.600+0000 7fcf81800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c0d5850 con 0x7fcf7c0c9450 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50464/0 (socket says 172.21.15.46:50464) 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 -- 172.21.15.46:0/430269559 learned_addr learned my addr 172.21.15.46:0/430269559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf7b400640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7c0c9450 0x7fcf7c1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 -- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7c0c9450 msgr2=0x7fcf7c1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf7be00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf600039b0 0x7fcf7c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7c0c9450 0x7fcf7c1f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 -- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf600039b0 msgr2=0x7fcf7c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf600039b0 0x7fcf7c0b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.601+0000 7fcf80e00640 1 -- 172.21.15.46:0/430269559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c0c9c60 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf80e00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0cf510 tx=0x7fcf7400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74007540 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf74004020 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004410 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c0c9f40 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.602+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c0ca500 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.603+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c0d4820 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.605+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf74011070 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.606+0000 7fcf73e00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 0x7fcf5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.606+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7fcf7408b0e0 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.606+0000 7fcf7be00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 0x7fcf5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:14.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.608+0000 7fcf7be00640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 0x7fcf5c06ede0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c033000 tx=0x7fcf6c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.610+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf74059200 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.815+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf7c0693b0 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.816+0000 7fcf73e00640 1 -- 172.21.15.46:0/430269559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v188) ==== 104+0+11 (secure 0 0 0) 0x7fcf74058b00 con 0x7fcf7c15c1c0 2024-04-26T23:56:14.819 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:56:14.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.819+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 msgr2=0x7fcf5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.819+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 0x7fcf5c06ede0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c033000 tx=0x7fcf6c02e000 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.819+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 msgr2=0x7fcf7c15c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.819+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0cf510 tx=0x7fcf7400c820 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 shutdown_connections 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf5c06c920 0x7fcf5c06ede0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c15c1c0 0x7fcf7c15c670 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7c0c9450 0x7fcf7c1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 --2- 172.21.15.46:0/430269559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf600039b0 0x7fcf7c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 >> 172.21.15.46:0/430269559 conn(0x7fcf7c09e2a0 msgr2=0x7fcf7c163300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 shutdown_connections 2024-04-26T23:56:14.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:14.820+0000 7fcf81800640 1 -- 172.21.15.46:0/430269559 wait complete. 2024-04-26T23:56:14.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:56:15.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.056+0000 7f00a0c00640 1 Processor -- start 2024-04-26T23:56:15.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.056+0000 7f00a0c00640 1 -- start start 2024-04-26T23:56:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f00a0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009c0d2530 0x7f009c0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f00a0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f009c0d2970 con 0x7f009c0d2080 2024-04-26T23:56:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f00a0c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f009c0d3020 con 0x7f009c15f290 2024-04-26T23:56:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f00a0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f009c0d0e10 con 0x7f009c0d2530 2024-04-26T23:56:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f009be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009c0d2530 0x7f009c0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f009aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f009c0d2080 0x7f009c0d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49020/0 (socket says 172.21.15.46:49020) 2024-04-26T23:56:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f009aa00640 1 -- 172.21.15.46:0/3173675539 learned_addr learned my addr 172.21.15.46:0/3173675539 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f009be00640 1 -- 172.21.15.46:0/3173675539 >> v1:172.21.15.142:6789/0 conn(0x7f009c15f290 legacy=0x7f009c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.058+0000 7f009be00640 1 -- 172.21.15.46:0/3173675539 >> v1:172.21.15.46:6789/0 conn(0x7f009c0d2080 legacy=0x7f009c0d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.059+0000 7f009be00640 1 -- 172.21.15.46:0/3173675539 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c0d32e0 con 0x7f009c0d2530 2024-04-26T23:56:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.060+0000 7f009be00640 1 --2- 172.21.15.46:0/3173675539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c0d06c0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f00940097a0 tx=0x7f009402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c86bce2b19b2188 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.060+0000 7f009a000640 1 -- 172.21.15.46:0/3173675539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094037070 con 0x7f009c0d2530 2024-04-26T23:56:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.060+0000 7f009a000640 1 -- 172.21.15.46:0/3173675539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0094032440 con 0x7f009c0d2530 2024-04-26T23:56:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.060+0000 7f009a000640 1 -- 172.21.15.46:0/3173675539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094032810 con 0x7f009c0d2530 2024-04-26T23:56:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 -- 172.21.15.46:0/3173675539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 msgr2=0x7f009c0d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 --2- 172.21.15.46:0/3173675539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c0d06c0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f00940097a0 tx=0x7f009402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 -- 172.21.15.46:0/3173675539 shutdown_connections 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 --2- 172.21.15.46:0/3173675539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c0d06c0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 -- 172.21.15.46:0/3173675539 >> 172.21.15.46:0/3173675539 conn(0x7f009c0d4fa0 msgr2=0x7f009c0d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 -- 172.21.15.46:0/3173675539 shutdown_connections 2024-04-26T23:56:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.061+0000 7f00a0c00640 1 -- 172.21.15.46:0/3173675539 wait complete. 2024-04-26T23:56:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.063+0000 7f00a0c00640 1 Processor -- start 2024-04-26T23:56:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.063+0000 7f00a0c00640 1 -- start start 2024-04-26T23:56:15.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f009c0d2080 0x7f009c14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009c15f290 0x7f009c1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f009c15f7f0 con 0x7f009c0d2530 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f009c15f670 con 0x7f009c0d2080 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f00a0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f009c0d41f0 con 0x7f009c15f290 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50472/0 (socket says 172.21.15.46:50472) 2024-04-26T23:56:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 -- 172.21.15.46:0/622926260 learned_addr learned my addr 172.21.15.46:0/622926260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 -- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009c15f290 msgr2=0x7f009c1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009c15f290 0x7f009c1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009aa00640 1 -- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f009c0d2080 msgr2=0x7f009c14bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.064+0000 7f009b400640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f009c0d2080 0x7f009c14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f009aa00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f009c0d2080 0x7f009c14bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f009aa00640 1 -- 172.21.15.46:0/622926260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c1615c0 con 0x7f009c0d2530 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f009aa00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f008400cb10 tx=0x7f008400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084017070 con 0x7f009c0d2530 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008400ac10 con 0x7f009c0d2530 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084012620 con 0x7f009c0d2530 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c161860 con 0x7f009c0d2530 2024-04-26T23:56:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.065+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c161e20 con 0x7f009c0d2530 2024-04-26T23:56:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.067+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c069ac0 con 0x7f009c0d2530 2024-04-26T23:56:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.068+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f008401a070 con 0x7f009c0d2530 2024-04-26T23:56:15.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.069+0000 7f0093400640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.069+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f008408ba90 con 0x7f009c0d2530 2024-04-26T23:56:15.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.069+0000 7f009b400640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.071+0000 7f009b400640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 0x7f007806edb0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f008c006fd0 tx=0x7f008c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.074+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f009c0d2970 con 0x7f009c0d2530 2024-04-26T23:56:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.280+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f009c0d08f0 con 0x7f009c0d2530 2024-04-26T23:56:15.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.281+0000 7f0093400640 1 -- 172.21.15.46:0/622926260 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v188) ==== 100+0+11 (secure 0 0 0) 0x7f0084059ce0 con 0x7f009c0d2530 2024-04-26T23:56:15.283 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:56:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 msgr2=0x7f007806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 0x7f007806edb0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f008c006fd0 tx=0x7f008c008040 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 msgr2=0x7f009c15fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f008400cb10 tx=0x7f008400a060 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 shutdown_connections 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009c15f290 0x7f009c1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d2530 0x7f009c15fa00 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 --2- 172.21.15.46:0/622926260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f009c0d2080 0x7f009c14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.283+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 >> 172.21.15.46:0/622926260 conn(0x7f009c0d4fa0 msgr2=0x7f009c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.284+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 shutdown_connections 2024-04-26T23:56:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.284+0000 7f00a0c00640 1 -- 172.21.15.46:0/622926260 wait complete. 2024-04-26T23:56:15.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.513+0000 7f7df5a00640 1 Processor -- start 2024-04-26T23:56:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.513+0000 7f7df5a00640 1 -- start start 2024-04-26T23:56:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.515+0000 7f7df5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7df00720e0 0x7f7df00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.515+0000 7f7df5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df0073ad0 con 0x7f7df0160910 2024-04-26T23:56:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.515+0000 7f7df5a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df0146a90 con 0x7f7df01466b0 2024-04-26T23:56:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.515+0000 7f7df5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7df0073d70 con 0x7f7df00720e0 2024-04-26T23:56:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.516+0000 7f7df5000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7df0160910 0x7f7df00cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49028/0 (socket says 172.21.15.46:49028) 2024-04-26T23:56:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.516+0000 7f7df5000640 1 -- 172.21.15.46:0/2478170490 learned_addr learned my addr 172.21.15.46:0/2478170490 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.516+0000 7f7def400640 1 --2- 172.21.15.46:0/2478170490 >> v2:172.21.15.142:3300/0 conn(0x7f7df00720e0 0x7f7df00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.516+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584249000 0 0) 0x7f7df0073ad0 con 0x7f7df0160910 2024-04-26T23:56:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.516+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dd4003520 con 0x7f7df0160910 2024-04-26T23:56:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2597209024 0 0) 0x7f7dd4003520 con 0x7f7df0160910 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 >> v2:172.21.15.142:3300/0 conn(0x7f7df00720e0 msgr2=0x7f7df00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 --2- 172.21.15.46:0/2478170490 >> v2:172.21.15.142:3300/0 conn(0x7f7df00720e0 0x7f7df00cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 >> v1:172.21.15.142:6789/0 conn(0x7f7df01466b0 legacy=0x7f7df006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0074050 con 0x7f7df0160910 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f7de80019f0 con 0x7f7df0160910 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 >> v1:172.21.15.46:6789/0 conn(0x7f7df0160910 legacy=0x7f7df00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4003a30 0x7f7dd4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 0x7f7dd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40051c0 0x7f7dd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0073d70 con 0x7f7dd4004800 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd400c0f0 con 0x7f7dd40051c0 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd400c2f0 con 0x7f7dd4003a30 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7def400640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 0x7f7dd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7defe00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4003a30 0x7f7dd4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.517+0000 7f7def400640 1 -- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4003a30 msgr2=0x7f7dd4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7def400640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4003a30 0x7f7dd4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7def400640 1 -- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40051c0 msgr2=0x7f7dd400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7df5000640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40051c0 0x7f7dd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7def400640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40051c0 0x7f7dd400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7def400640 1 -- 172.21.15.46:0/2478170490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd4003520 con 0x7f7dd4004800 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7df5000640 1 -- 172.21.15.46:0/2478170490 reap_dead start 2024-04-26T23:56:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.518+0000 7f7defe00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4003a30 0x7f7dd4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7def400640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 0x7f7dd4004c80 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009ac0 tx=0x7f7dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0084aeca0c07636 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc036070 con 0x7f7dd4004800 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc031400 con 0x7f7dd4004800 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7deea00640 1 -- 172.21.15.46:0/2478170490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc009690 con 0x7f7dd4004800 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7df5a00640 1 -- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 msgr2=0x7f7dd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.519+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 0x7f7dd4004c80 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009ac0 tx=0x7f7dd4005830 comp rx=0 tx=0).stop 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.520+0000 7f7df5a00640 1 -- 172.21.15.46:0/2478170490 shutdown_connections 2024-04-26T23:56:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.520+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2478170490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dd4004800 0x7f7dd4004c80 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.520+0000 7f7df5a00640 1 -- 172.21.15.46:0/2478170490 >> 172.21.15.46:0/2478170490 conn(0x7f7df00b26d0 msgr2=0x7f7df00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:15.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.520+0000 7f7df5a00640 1 -- 172.21.15.46:0/2478170490 shutdown_connections 2024-04-26T23:56:15.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.520+0000 7f7df5a00640 1 -- 172.21.15.46:0/2478170490 wait complete. 2024-04-26T23:56:15.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.522+0000 7f7df5a00640 1 Processor -- start 2024-04-26T23:56:15.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.522+0000 7f7df5a00640 1 -- start start 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 0x7f7df015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00d0020 0x7f7df00cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0160d60 con 0x7f7df015f8f0 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00cb470 con 0x7f7dd4004800 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7df5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7df0160ee0 con 0x7f7df00d0020 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7def400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7def400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50492/0 (socket says 172.21.15.46:50492) 2024-04-26T23:56:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.523+0000 7f7def400640 1 -- 172.21.15.46:0/2135930502 learned_addr learned my addr 172.21.15.46:0/2135930502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7defe00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 0x7f7df015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7df5000640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00d0020 0x7f7df00cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 -- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00d0020 msgr2=0x7f7df00cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00d0020 0x7f7df00cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 -- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 msgr2=0x7f7df015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 0x7f7df015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 -- 172.21.15.46:0/2135930502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00cfd00 con 0x7f7df015f8f0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7defe00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 0x7f7df015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7def400640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc034040 tx=0x7f7ddc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc03f020 con 0x7f7df015f8f0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc037040 con 0x7f7df015f8f0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc036070 con 0x7f7df015f8f0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.524+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00c7b70 con 0x7f7df015f8f0 2024-04-26T23:56:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.525+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df0143690 con 0x7f7df015f8f0 2024-04-26T23:56:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.526+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00693b0 con 0x7f7df015f8f0 2024-04-26T23:56:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.527+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ddc042020 con 0x7f7df015f8f0 2024-04-26T23:56:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.528+0000 7f7de7e00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 0x7f7dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.528+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f7ddc0abd60 con 0x7f7df015f8f0 2024-04-26T23:56:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.528+0000 7f7defe00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 0x7f7dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.529+0000 7f7defe00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 0x7f7dd006edb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7df006b1d0 tx=0x7f7de0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.532+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ddc049030 con 0x7f7df015f8f0 2024-04-26T23:56:15.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.780+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df00d1760 con 0x7f7df015f8f0 2024-04-26T23:56:15.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.782+0000 7f7de7e00640 1 -- 172.21.15.46:0/2135930502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f7ddc07a100 con 0x7f7df015f8f0 2024-04-26T23:56:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:15.785 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:56:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:56:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 msgr2=0x7f7dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 0x7f7dd006edb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7df006b1d0 tx=0x7f7de0009040 comp rx=0 tx=0).stop 2024-04-26T23:56:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 msgr2=0x7f7df00cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc034040 tx=0x7f7ddc004730 comp rx=0 tx=0).stop 2024-04-26T23:56:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 shutdown_connections 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7dd006c8f0 0x7f7dd006edb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00d0020 0x7f7df00cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df015f8f0 0x7f7df00cee50 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 --2- 172.21.15.46:0/2135930502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4004800 0x7f7df015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.784+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 >> 172.21.15.46:0/2135930502 conn(0x7f7df00b26d0 msgr2=0x7f7df00c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.785+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 shutdown_connections 2024-04-26T23:56:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:15.785+0000 7f7df5a00640 1 -- 172.21.15.46:0/2135930502 wait complete. 2024-04-26T23:56:15.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.023+0000 7f2008c00640 1 Processor -- start 2024-04-26T23:56:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.024+0000 7f2008c00640 1 -- start start 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.025+0000 7f2008c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2004142280 0x7f200406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.025+0000 7f2008c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f200406ea50 0x7f2004144f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.025+0000 7f2008c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20040b43f0 con 0x7f20040b3e80 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.025+0000 7f2008c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f200406eea0 con 0x7f2004142280 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.025+0000 7f2008c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f20040b46d0 con 0x7f200406ea50 2024-04-26T23:56:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2004142280 0x7f200406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f20040b3e80 0x7f2004144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49044/0 (socket says 172.21.15.46:49044) 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001400640 1 -- 172.21.15.46:0/1494260512 learned_addr learned my addr 172.21.15.46:0/1494260512 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2002800640 1 --2- 172.21.15.46:0/1494260512 >> v2:172.21.15.142:3300/0 conn(0x7f200406ea50 0x7f2004144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2000a00640 1 -- 172.21.15.46:0/1494260512 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854197331 0 0) 0x7f20040b43f0 con 0x7f20040b3e80 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001e00640 1 -- 172.21.15.46:0/1494260512 >> v2:172.21.15.142:3300/0 conn(0x7f200406ea50 msgr2=0x7f2004144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001e00640 1 --2- 172.21.15.46:0/1494260512 >> v2:172.21.15.142:3300/0 conn(0x7f200406ea50 0x7f2004144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001e00640 1 -- 172.21.15.46:0/1494260512 >> v1:172.21.15.46:6789/0 conn(0x7f20040b3e80 legacy=0x7f2004144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.026+0000 7f2001e00640 1 -- 172.21.15.46:0/1494260512 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20040b49b0 con 0x7f2004142280 2024-04-26T23:56:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.028+0000 7f2001e00640 1 --2- 172.21.15.46:0/1494260512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200406e510 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f20041459d0 tx=0x7f1ff402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3bca5457d51ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.028+0000 7f2000a00640 1 -- 172.21.15.46:0/1494260512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4037070 con 0x7f2004142280 2024-04-26T23:56:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.028+0000 7f2000a00640 1 -- 172.21.15.46:0/1494260512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff4032440 con 0x7f2004142280 2024-04-26T23:56:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.028+0000 7f2000a00640 1 -- 172.21.15.46:0/1494260512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4032810 con 0x7f2004142280 2024-04-26T23:56:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 -- 172.21.15.46:0/1494260512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 msgr2=0x7f200406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 --2- 172.21.15.46:0/1494260512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200406e510 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f20041459d0 tx=0x7f1ff402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 -- 172.21.15.46:0/1494260512 shutdown_connections 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 --2- 172.21.15.46:0/1494260512 >> v2:172.21.15.142:3300/0 conn(0x7f200406ea50 0x7f2004144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 --2- 172.21.15.46:0/1494260512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200406e510 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 -- 172.21.15.46:0/1494260512 >> 172.21.15.46:0/1494260512 conn(0x7f200406c530 msgr2=0x7f200406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 -- 172.21.15.46:0/1494260512 shutdown_connections 2024-04-26T23:56:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.029+0000 7f2008c00640 1 -- 172.21.15.46:0/1494260512 wait complete. 2024-04-26T23:56:16.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.031+0000 7f2008c00640 1 Processor -- start 2024-04-26T23:56:16.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.031+0000 7f2008c00640 1 -- start start 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200406ea50 0x7f2004158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20040b3e80 0x7f200415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20041428a0 con 0x7f2004142280 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20040b4f70 con 0x7f200406ea50 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2008c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2004142a20 con 0x7f20040b3e80 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2002800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2002800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50504/0 (socket says 172.21.15.46:50504) 2024-04-26T23:56:16.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2002800640 1 -- 172.21.15.46:0/599138246 learned_addr learned my addr 172.21.15.46:0/599138246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2001e00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200406ea50 0x7f2004158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.032+0000 7f2001400640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20040b3e80 0x7f200415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 -- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20040b3e80 msgr2=0x7f200415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20040b3e80 0x7f200415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 -- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200406ea50 msgr2=0x7f2004158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200406ea50 0x7f2004158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 -- 172.21.15.46:0/599138246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200415f480 con 0x7f2004142280 2024-04-26T23:56:16.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2002800640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d800 tx=0x7f1ffc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00b6c0 con 0x7f2004142280 2024-04-26T23:56:16.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc019070 con 0x7f2004142280 2024-04-26T23:56:16.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00bac0 con 0x7f2004142280 2024-04-26T23:56:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200415f720 con 0x7f2004142280 2024-04-26T23:56:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.033+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200415fce0 con 0x7f2004142280 2024-04-26T23:56:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.035+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2004069530 con 0x7f2004142280 2024-04-26T23:56:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.036+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ffc00bc60 con 0x7f2004142280 2024-04-26T23:56:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.037+0000 7f1ffa000640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.037+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f1ffc08c230 con 0x7f2004142280 2024-04-26T23:56:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.037+0000 7f2001e00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.038+0000 7f2001e00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 0x7f1fe006edb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4034000 tx=0x7f1ff402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.041+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ffc05a3f0 con 0x7f2004142280 2024-04-26T23:56:16.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.248+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2004069c60 con 0x7f2004142280 2024-04-26T23:56:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.250+0000 7f1ffa000640 1 -- 172.21.15.46:0/599138246 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17001 (secure 0 0 0) 0x7f1ffc059cf0 con 0x7f2004142280 2024-04-26T23:56:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:16.254 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":188,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:58.805182+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.25","mappings":[{"from":6,"to":4}]},{"pgid":"28.38","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 msgr2=0x7f1fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 0x7f1fe006edb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4034000 tx=0x7f1ff402e000 comp rx=0 tx=0).stop 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 msgr2=0x7f200413f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d800 tx=0x7f1ffc00dcc0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 shutdown_connections 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004142280 0x7f200413f110 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20040b3e80 0x7f200415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 --2- 172.21.15.46:0/599138246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200406ea50 0x7f2004158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 >> 172.21.15.46:0/599138246 conn(0x7f200406c530 msgr2=0x7f2004142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 shutdown_connections 2024-04-26T23:56:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.253+0000 7f2008c00640 1 -- 172.21.15.46:0/599138246 wait complete. 2024-04-26T23:56:16.276 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:56:16.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:16.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.488+0000 7f1383e00640 1 Processor -- start 2024-04-26T23:56:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.488+0000 7f1383e00640 1 -- start start 2024-04-26T23:56:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.490+0000 7f1383e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 0x7f13840d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.490+0000 7f1383e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.490+0000 7f1383e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13840d0990 con 0x7f138415f200 2024-04-26T23:56:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.490+0000 7f1383e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f13840d0c70 con 0x7f13840d2010 2024-04-26T23:56:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.490+0000 7f1383e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f13840d0f50 con 0x7f138415f5e0 2024-04-26T23:56:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1383400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1383400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:45646/0 (socket says 172.21.15.46:45646) 2024-04-26T23:56:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1383400640 1 -- 172.21.15.46:0/404856630 learned_addr learned my addr 172.21.15.46:0/404856630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1382a00640 1 --1- 172.21.15.46:0/404856630 >> v1:172.21.15.142:6789/0 conn(0x7f138415f200 0x7f13840d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47444/0 (socket says 172.21.15.46:47444) 2024-04-26T23:56:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1382000640 1 --2- 172.21.15.46:0/404856630 >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 0x7f13840d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1381600640 1 -- 172.21.15.46:0/404856630 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1166259556 0 0) 0x7f13840d0990 con 0x7f138415f200 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.491+0000 7f1381600640 1 -- 172.21.15.46:0/404856630 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1364003520 con 0x7f138415f200 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.492+0000 7f1383400640 1 -- 172.21.15.46:0/404856630 >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 msgr2=0x7f13840d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.492+0000 7f1383400640 1 --2- 172.21.15.46:0/404856630 >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 0x7f13840d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.492+0000 7f1383400640 1 -- 172.21.15.46:0/404856630 >> v1:172.21.15.142:6789/0 conn(0x7f138415f200 legacy=0x7f13840d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.492+0000 7f1383400640 1 -- 172.21.15.46:0/404856630 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13840d1230 con 0x7f138415f5e0 2024-04-26T23:56:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.492+0000 7f1382000640 1 --2- 172.21.15.46:0/404856630 >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 0x7f13840d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.493+0000 7f1383400640 1 --2- 172.21.15.46:0/404856630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f137c009e50 tx=0x7f137c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26f0652e45817ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.494+0000 7f1381600640 1 -- 172.21.15.46:0/404856630 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c036070 con 0x7f138415f5e0 2024-04-26T23:56:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.494+0000 7f1381600640 1 -- 172.21.15.46:0/404856630 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c031600 con 0x7f138415f5e0 2024-04-26T23:56:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.494+0000 7f1381600640 1 -- 172.21.15.46:0/404856630 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0319d0 con 0x7f138415f5e0 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 -- 172.21.15.46:0/404856630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 msgr2=0x7f13840d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 --2- 172.21.15.46:0/404856630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f137c009e50 tx=0x7f137c02a710 comp rx=0 tx=0).stop 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 -- 172.21.15.46:0/404856630 shutdown_connections 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 --2- 172.21.15.46:0/404856630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f5e0 0x7f13840d2cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 --2- 172.21.15.46:0/404856630 >> v2:172.21.15.142:3300/0 conn(0x7f13840d2010 0x7f13840d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 -- 172.21.15.46:0/404856630 >> 172.21.15.46:0/404856630 conn(0x7f13840d4ef0 msgr2=0x7f13840d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 -- 172.21.15.46:0/404856630 shutdown_connections 2024-04-26T23:56:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.495+0000 7f1383e00640 1 -- 172.21.15.46:0/404856630 wait complete. 2024-04-26T23:56:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.497+0000 7f1383e00640 1 Processor -- start 2024-04-26T23:56:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.497+0000 7f1383e00640 1 -- start start 2024-04-26T23:56:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 0x7f138414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f138415f5e0 0x7f13841f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13840b2f50 con 0x7f13840d2010 2024-04-26T23:56:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13840b2dd0 con 0x7f138415f5e0 2024-04-26T23:56:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.498+0000 7f1383e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f13840b30d0 con 0x7f138415f200 2024-04-26T23:56:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50516/0 (socket says 172.21.15.46:50516) 2024-04-26T23:56:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 -- 172.21.15.46:0/1188626737 learned_addr learned my addr 172.21.15.46:0/1188626737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382000640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 0x7f138414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 -- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 msgr2=0x7f138414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 0x7f138414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 -- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f138415f5e0 msgr2=0x7f13841f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f138415f5e0 0x7f13841f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382a00640 1 -- 172.21.15.46:0/1188626737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13840caa60 con 0x7f13840d2010 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.499+0000 7f1382000640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 0x7f138414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f1382a00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f137400c460 tx=0x7f137400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740074e0 con 0x7f13840d2010 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13740047e0 con 0x7f13840d2010 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374004bd0 con 0x7f13840d2010 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13840cad40 con 0x7f13840d2010 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.500+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13840cb300 con 0x7f13840d2010 2024-04-26T23:56:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.501+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13840693b0 con 0x7f13840d2010 2024-04-26T23:56:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.502+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f137401a020 con 0x7f13840d2010 2024-04-26T23:56:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.504+0000 7f137aa00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 0x7f136006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.504+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f137408b5c0 con 0x7f13840d2010 2024-04-26T23:56:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.504+0000 7f1382000640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 0x7f136006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.506+0000 7f1382000640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 0x7f136006ede0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f13841f7e70 tx=0x7f136c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.508+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1374010300 con 0x7f13840d2010 2024-04-26T23:56:16.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.748+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13840c8640 con 0x7f13840d2010 2024-04-26T23:56:16.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.749+0000 7f137aa00640 1 -- 172.21.15.46:0/1188626737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f1374059830 con 0x7f13840d2010 2024-04-26T23:56:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:16.752 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:56:16.752 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 msgr2=0x7f136006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 0x7f136006ede0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f13841f7e70 tx=0x7f136c008040 comp rx=0 tx=0).stop 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 msgr2=0x7f13840ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f137400c460 tx=0x7f137400c920 comp rx=0 tx=0).stop 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 shutdown_connections 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f136006c920 0x7f136006ede0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f138415f5e0 0x7f13841f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f138415f200 0x7f138414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 --2- 172.21.15.46:0/1188626737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13840d2010 0x7f13840ca3a0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 >> 172.21.15.46:0/1188626737 conn(0x7f13840d4ef0 msgr2=0x7f138406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 shutdown_connections 2024-04-26T23:56:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.751+0000 7f1383e00640 1 -- 172.21.15.46:0/1188626737 wait complete. 2024-04-26T23:56:16.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.976+0000 7f4066c00640 1 Processor -- start 2024-04-26T23:56:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.977+0000 7f4066c00640 1 -- start start 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.978+0000 7f4066c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40600d1af0 0x7f40600d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.978+0000 7f4066c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 0x7f406015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.978+0000 7f4066c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406015fd30 con 0x7f4060162980 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.978+0000 7f4066c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4060162d60 con 0x7f40600d1af0 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.978+0000 7f4066c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40600d1050 con 0x7f406015ed30 2024-04-26T23:56:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40600d1af0 0x7f40600d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4060162980 0x7f40600d2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49058/0 (socket says 172.21.15.46:49058) 2024-04-26T23:56:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405fe00640 1 -- 172.21.15.46:0/2147492047 learned_addr learned my addr 172.21.15.46:0/2147492047 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f4064c00640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 0x7f406015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405f400640 1 -- 172.21.15.46:0/2147492047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 msgr2=0x7f406015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405f400640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 0x7f406015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405f400640 1 -- 172.21.15.46:0/2147492047 >> v1:172.21.15.46:6789/0 conn(0x7f4060162980 legacy=0x7f40600d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405ea00640 1 -- 172.21.15.46:0/2147492047 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606102255 0 0) 0x7f406015fd30 con 0x7f4060162980 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f405f400640 1 -- 172.21.15.46:0/2147492047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4060160010 con 0x7f40600d1af0 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.979+0000 7f4064c00640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 0x7f406015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.980+0000 7f405f400640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40600d1af0 0x7f40600d1f80 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f40480097a0 tx=0x7f404802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a5dcb80de55196 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.980+0000 7f405ea00640 1 -- 172.21.15.46:0/2147492047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048037070 con 0x7f40600d1af0 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f405ea00640 1 -- 172.21.15.46:0/2147492047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4048032440 con 0x7f40600d1af0 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f405ea00640 1 -- 172.21.15.46:0/2147492047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048032810 con 0x7f40600d1af0 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 -- 172.21.15.46:0/2147492047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40600d1af0 msgr2=0x7f40600d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40600d1af0 0x7f40600d1f80 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f40480097a0 tx=0x7f404802a180 comp rx=0 tx=0).stop 2024-04-26T23:56:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 -- 172.21.15.46:0/2147492047 shutdown_connections 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406015ed30 0x7f406015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 --2- 172.21.15.46:0/2147492047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40600d1af0 0x7f40600d1f80 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 -- 172.21.15.46:0/2147492047 >> 172.21.15.46:0/2147492047 conn(0x7f406009e420 msgr2=0x7f40600d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 -- 172.21.15.46:0/2147492047 shutdown_connections 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.981+0000 7f4066c00640 1 -- 172.21.15.46:0/2147492047 wait complete. 2024-04-26T23:56:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.983+0000 7f4066c00640 1 Processor -- start 2024-04-26T23:56:16.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.983+0000 7f4066c00640 1 -- start start 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40600d1af0 0x7f40601eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406015ed30 0x7f40600ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40601608b0 con 0x7f4060162980 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40601605d0 con 0x7f406015ed30 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.984+0000 7f4066c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4060160d50 con 0x7f40600d1af0 2024-04-26T23:56:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50542/0 (socket says 172.21.15.46:50542) 2024-04-26T23:56:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 -- 172.21.15.46:0/3282757787 learned_addr learned my addr 172.21.15.46:0/3282757787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f405fe00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40600d1af0 0x7f40601eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 -- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40600d1af0 msgr2=0x7f40601eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f405f400640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406015ed30 0x7f40600ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40600d1af0 0x7f40601eb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 -- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406015ed30 msgr2=0x7f40600ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406015ed30 0x7f40600ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 -- 172.21.15.46:0/3282757787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40600b8780 con 0x7f4060162980 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.985+0000 7f4064c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f405800c7c0 tx=0x7f405800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.986+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058015020 con 0x7f4060162980 2024-04-26T23:56:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.986+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4058004230 con 0x7f4060162980 2024-04-26T23:56:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.986+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058007620 con 0x7f4060162980 2024-04-26T23:56:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.986+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40600b8a60 con 0x7f4060162980 2024-04-26T23:56:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.986+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40600b9020 con 0x7f4060162980 2024-04-26T23:56:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.987+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060069530 con 0x7f4060162980 2024-04-26T23:56:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.988+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4058018020 con 0x7f4060162980 2024-04-26T23:56:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.989+0000 7f4057e00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.989+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f405808be80 con 0x7f4060162980 2024-04-26T23:56:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.990+0000 7f405fe00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.991+0000 7f405fe00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f40500012e0 tx=0x7f4050009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:16.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:16.993+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f405801f020 con 0x7f4060162980 2024-04-26T23:56:17.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.234+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40600cf4a0 con 0x7f4060162980 2024-04-26T23:56:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.236+0000 7f4057e00640 1 -- 172.21.15.46:0/3282757787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+5200 (secure 0 0 0) 0x7f405805a0f0 con 0x7f4060162980 2024-04-26T23:56:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:17.239 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:55:27.117121+0000","modified":"2024-04-26T23:55:30.196334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11247":{"gid":11247,"name":"c","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2117263293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2117263293},{"type":"v1","addr":"172.21.15.46:6835","nonce":2117263293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11247,"qdb_cluster":[11247]},"id":13}]} 2024-04-26T23:56:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 149 2024-04-26T23:56:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 msgr2=0x7f403c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f40500012e0 tx=0x7f4050009040 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 msgr2=0x7f40600ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f405800c7c0 tx=0x7f405800cc80 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 shutdown_connections 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4060162980 0x7f40600ce9f0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406015ed30 0x7f40600ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 --2- 172.21.15.46:0/3282757787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40600d1af0 0x7f40601eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.238+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 >> 172.21.15.46:0/3282757787 conn(0x7f406009e420 msgr2=0x7f40600d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.239+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 shutdown_connections 2024-04-26T23:56:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.239+0000 7f4066c00640 1 -- 172.21.15.46:0/3282757787 wait complete. 2024-04-26T23:56:17.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.474+0000 7f3ae7200640 1 Processor -- start 2024-04-26T23:56:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.474+0000 7f3ae7200640 1 -- start start 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae815f4e0 0x7f3ae80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80d2130 0x7f3ae80b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae80d11f0 con 0x7f3ae80b4000 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae80d0ce0 con 0x7f3ae815f4e0 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae80d2990 con 0x7f3ae80d2130 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae815f4e0 0x7f3ae80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae815f4e0 0x7f3ae80d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50552/0 (socket says 172.21.15.46:50552) 2024-04-26T23:56:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.476+0000 7f3ae5e00640 1 -- 172.21.15.46:0/1797042843 learned_addr learned my addr 172.21.15.46:0/1797042843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.477+0000 7f3ae6800640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80d2130 0x7f3ae80b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.477+0000 7f3ae5e00640 1 -- 172.21.15.46:0/1797042843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80d2130 msgr2=0x7f3ae80b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.477+0000 7f3ae5e00640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80d2130 0x7f3ae80b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.477+0000 7f3ae5e00640 1 -- 172.21.15.46:0/1797042843 >> v1:172.21.15.142:6789/0 conn(0x7f3ae80b4000 legacy=0x7f3ae80d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.477+0000 7f3ae5e00640 1 -- 172.21.15.46:0/1797042843 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80d5f20 con 0x7f3ae815f4e0 2024-04-26T23:56:17.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.479+0000 7f3ae5e00640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80d2df0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009870 tx=0x7f3ad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28f0075aa94f666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.479+0000 7f3ae4a00640 1 -- 172.21.15.46:0/1797042843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8037070 con 0x7f3ae815f4e0 2024-04-26T23:56:17.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.479+0000 7f3ae4a00640 1 -- 172.21.15.46:0/1797042843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad8032440 con 0x7f3ae815f4e0 2024-04-26T23:56:17.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.479+0000 7f3ae4a00640 1 -- 172.21.15.46:0/1797042843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8032810 con 0x7f3ae815f4e0 2024-04-26T23:56:17.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 -- 172.21.15.46:0/1797042843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae815f4e0 msgr2=0x7f3ae80d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80d2df0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009870 tx=0x7f3ad802a270 comp rx=0 tx=0).stop 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 -- 172.21.15.46:0/1797042843 shutdown_connections 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80d2130 0x7f3ae80b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1797042843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80d2df0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 -- 172.21.15.46:0/1797042843 >> 172.21.15.46:0/1797042843 conn(0x7f3ae80d5010 msgr2=0x7f3ae80d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 -- 172.21.15.46:0/1797042843 shutdown_connections 2024-04-26T23:56:17.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.480+0000 7f3ae7200640 1 -- 172.21.15.46:0/1797042843 wait complete. 2024-04-26T23:56:17.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.482+0000 7f3ae7200640 1 Processor -- start 2024-04-26T23:56:17.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.482+0000 7f3ae7200640 1 -- start start 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80b4000 0x7f3ae81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80b1d60 con 0x7f3ae80d2130 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80d5c40 con 0x7f3ae815f4e0 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae815f8c0 con 0x7f3ae80b4000 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50560/0 (socket says 172.21.15.46:50560) 2024-04-26T23:56:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae5400640 1 -- 172.21.15.46:0/1872783705 learned_addr learned my addr 172.21.15.46:0/1872783705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:17.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae6800640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.483+0000 7f3ae5e00640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80b4000 0x7f3ae81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 -- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80b4000 msgr2=0x7f3ae81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80b4000 0x7f3ae81566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 -- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae815f4e0 msgr2=0x7f3ae80b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 -- 172.21.15.46:0/1872783705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80b7400 con 0x7f3ae80d2130 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ae5400640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c400 tx=0x7f3ad400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40052d0 con 0x7f3ae80d2130 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4005490 con 0x7f3ae80d2130 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.484+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007260 con 0x7f3ae80d2130 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.485+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae80b4f70 con 0x7f3ae80d2130 2024-04-26T23:56:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.485+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae80b5530 con 0x7f3ae80d2130 2024-04-26T23:56:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.486+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae80693b0 con 0x7f3ae80d2130 2024-04-26T23:56:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.487+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ad4007400 con 0x7f3ae80d2130 2024-04-26T23:56:17.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.488+0000 7f3ade000640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 0x7f3ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.488+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f3ad408b230 con 0x7f3ae80d2130 2024-04-26T23:56:17.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.488+0000 7f3ae5e00640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 0x7f3ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.489+0000 7f3ae5e00640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 0x7f3ac806edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8034000 tx=0x7f3ad802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.493+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ad4059510 con 0x7f3ae80d2130 2024-04-26T23:56:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.699+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ae80cb970 con 0x7f3ae80d2130 2024-04-26T23:56:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.702+0000 7f3ade000640 1 -- 172.21.15.46:0/1872783705 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17001 (secure 0 0 0) 0x7f3ad4058e10 con 0x7f3ae80d2130 2024-04-26T23:56:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:17.706 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":188,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:55:58.805182+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:55:24.945034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-26T23:55:26.104963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.25","mappings":[{"from":6,"to":4}]},{"pgid":"28.38","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 msgr2=0x7f3ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 0x7f3ac806edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8034000 tx=0x7f3ad802e000 comp rx=0 tx=0).stop 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 msgr2=0x7f3ae80b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c400 tx=0x7f3ad400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 shutdown_connections 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3ac806c8f0 0x7f3ac806edb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae815f4e0 0x7f3ae80b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae80d2130 0x7f3ae80b6ec0 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 --2- 172.21.15.46:0/1872783705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ae80b4000 0x7f3ae81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.705+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 >> 172.21.15.46:0/1872783705 conn(0x7f3ae80d5010 msgr2=0x7f3ae81602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.706+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 shutdown_connections 2024-04-26T23:56:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.706+0000 7f3ae7200640 1 -- 172.21.15.46:0/1872783705 wait complete. 2024-04-26T23:56:17.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:56:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.937+0000 7f7357200640 1 Processor -- start 2024-04-26T23:56:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.938+0000 7f7357200640 1 -- start start 2024-04-26T23:56:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.939+0000 7f7357200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f73581466b0 0x7f735806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.939+0000 7f7357200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.939+0000 7f7357200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7358073ad0 con 0x7f7358160910 2024-04-26T23:56:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.939+0000 7f7357200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7358146a90 con 0x7f73581466b0 2024-04-26T23:56:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.939+0000 7f7357200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7358073d70 con 0x7f73580720e0 2024-04-26T23:56:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.940+0000 7f7355400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.940+0000 7f7355e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f73581466b0 0x7f735806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.940+0000 7f7355400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:45718/0 (socket says 172.21.15.46:45718) 2024-04-26T23:56:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.940+0000 7f7355400640 1 -- 172.21.15.46:0/1873188119 learned_addr learned my addr 172.21.15.46:0/1873188119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.940+0000 7f7356800640 1 --1- 172.21.15.46:0/1873188119 >> v1:172.21.15.142:6789/0 conn(0x7f7358160910 0x7f73580cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:47456/0 (socket says 172.21.15.46:47456) 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.941+0000 7f7355e00640 1 -- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 msgr2=0x7f73580cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.941+0000 7f7355e00640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.941+0000 7f7355e00640 1 -- 172.21.15.46:0/1873188119 >> v1:172.21.15.142:6789/0 conn(0x7f7358160910 legacy=0x7f73580cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.942+0000 7f7355e00640 1 -- 172.21.15.46:0/1873188119 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7358074050 con 0x7f73581466b0 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.941+0000 7f7354a00640 1 -- 172.21.15.46:0/1873188119 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2690476654 0 0) 0x7f7358073ad0 con 0x7f7358160910 2024-04-26T23:56:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.942+0000 7f7355400640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.943+0000 7f7355e00640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735806de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f73480097a0 tx=0x7f734802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce7c6ee75ccf256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.943+0000 7f7354a00640 1 -- 172.21.15.46:0/1873188119 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348037070 con 0x7f73581466b0 2024-04-26T23:56:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.943+0000 7f7354a00640 1 -- 172.21.15.46:0/1873188119 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348032440 con 0x7f73581466b0 2024-04-26T23:56:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.943+0000 7f7354a00640 1 -- 172.21.15.46:0/1873188119 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348032810 con 0x7f73581466b0 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 -- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 msgr2=0x7f735806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735806de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f73480097a0 tx=0x7f734802a180 comp rx=0 tx=0).stop 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 -- 172.21.15.46:0/1873188119 shutdown_connections 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73580720e0 0x7f73580cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 --2- 172.21.15.46:0/1873188119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735806de80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 -- 172.21.15.46:0/1873188119 >> 172.21.15.46:0/1873188119 conn(0x7f73580b26d0 msgr2=0x7f73580b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 -- 172.21.15.46:0/1873188119 shutdown_connections 2024-04-26T23:56:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.944+0000 7f7357200640 1 -- 172.21.15.46:0/1873188119 wait complete. 2024-04-26T23:56:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.946+0000 7f7357200640 1 Processor -- start 2024-04-26T23:56:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.946+0000 7f7357200640 1 -- start start 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735813eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7358160910 0x7f73580c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f735806e3c0 con 0x7f73580720e0 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73580cb470 con 0x7f73581466b0 2024-04-26T23:56:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.947+0000 7f7357200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f73580cdda0 con 0x7f7358160910 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50572/0 (socket says 172.21.15.46:50572) 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 -- 172.21.15.46:0/1932018001 learned_addr learned my addr 172.21.15.46:0/1932018001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7356800640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7358160910 0x7f73580c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355400640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735813eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 -- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7358160910 msgr2=0x7f73580c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7358160910 0x7f73580c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 -- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 msgr2=0x7f735813eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735813eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.948+0000 7f7355e00640 1 -- 172.21.15.46:0/1932018001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7358140b00 con 0x7f73580720e0 2024-04-26T23:56:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f7355e00640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f73480098c0 tx=0x7f73480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348041070 con 0x7f73580720e0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f734803c870 con 0x7f73580720e0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348037040 con 0x7f73580720e0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7358140de0 con 0x7f73580720e0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.949+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73581413a0 con 0x7f73580720e0 2024-04-26T23:56:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.950+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73580693b0 con 0x7f73580720e0 2024-04-26T23:56:17.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.951+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7348044070 con 0x7f73580720e0 2024-04-26T23:56:17.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.952+0000 7f734e000640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:17.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.952+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6188+0+0 (secure 0 0 0) 0x7f73480ac7d0 con 0x7f73580720e0 2024-04-26T23:56:17.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.953+0000 7f7355400640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:17.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.954+0000 7f7355400640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 0x7f733806edb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f73580c7f20 tx=0x7f7344008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:17.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:17.957+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f734807abd0 con 0x7f73580720e0 2024-04-26T23:56:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.204+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f73580c7180 con 0x7f73580720e0 2024-04-26T23:56:18.208 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:56:18.206+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:56:18.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.210+0000 7f734e000640 1 -- 172.21.15.46:0/1932018001 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v150) ==== 161+0+0 (secure 0 0 0) 0x7f734807a4d0 con 0x7f73580720e0 2024-04-26T23:56:18.212 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:56:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.213+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 msgr2=0x7f733806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.213+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 0x7f733806edb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f73580c7f20 tx=0x7f7344008040 comp rx=0 tx=0).stop 2024-04-26T23:56:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.213+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 msgr2=0x7f7358140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.213+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f73480098c0 tx=0x7f73480046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 shutdown_connections 2024-04-26T23:56:18.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7358160910 0x7f73580c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73581466b0 0x7f735813eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 --2- 172.21.15.46:0/1932018001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73580720e0 0x7f7358140200 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.214+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 >> 172.21.15.46:0/1932018001 conn(0x7f73580b26d0 msgr2=0x7f73581459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.215+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 shutdown_connections 2024-04-26T23:56:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.215+0000 7f7357200640 1 -- 172.21.15.46:0/1932018001 wait complete. 2024-04-26T23:56:18.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:56:18.251 INFO:tasks.ceph.mds.c.smithi046.stderr: -6526> 2024-04-26T23:55:33.605+0000 7f7b3c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:56:18.263 INFO:tasks.ceph.mds.c.smithi046.stderr: -3416> 2024-04-26T23:55:38.088+0000 7f7b37400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-26T23:55:36.280108+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={7891=0-12582912@1} caps={7891=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=7891 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5570bde70c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(28)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-26T23:56:18.325 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:56:18.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.438+0000 7ff452200640 1 Processor -- start 2024-04-26T23:56:18.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.438+0000 7ff452200640 1 -- start start 2024-04-26T23:56:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff452200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff452200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff44c0d1cb0 0x7ff44c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff452200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff44c0d3400 con 0x7ff44c0d3830 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff452200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff44c0d2140 con 0x7ff44c0d1cb0 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff452200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff44c0d2370 con 0x7ff44c162a70 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff44be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff44c0d3830 0x7ff44c0d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49088/0 (socket says 172.21.15.46:49088) 2024-04-26T23:56:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.440+0000 7ff44be00640 1 -- 172.21.15.46:0/1404661686 learned_addr learned my addr 172.21.15.46:0/1404661686 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff450e00640 1 --2- 172.21.15.46:0/1404661686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff451800640 1 --2- 172.21.15.46:0/1404661686 >> v2:172.21.15.142:3300/0 conn(0x7ff44c0d1cb0 0x7ff44c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235931683 0 0) 0x7ff44c0d3400 con 0x7ff44c0d3830 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff430003520 con 0x7ff44c0d3830 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff450e00640 1 -- 172.21.15.46:0/1404661686 >> v2:172.21.15.142:3300/0 conn(0x7ff44c0d1cb0 msgr2=0x7ff44c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff450e00640 1 --2- 172.21.15.46:0/1404661686 >> v2:172.21.15.142:3300/0 conn(0x7ff44c0d1cb0 0x7ff44c0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1110071055 0 0) 0x7ff430003520 con 0x7ff44c0d3830 2024-04-26T23:56:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff450e00640 1 -- 172.21.15.46:0/1404661686 >> v1:172.21.15.46:6789/0 conn(0x7ff44c0d3830 legacy=0x7ff44c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.441+0000 7ff450e00640 1 -- 172.21.15.46:0/1404661686 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c0d2650 con 0x7ff44c162a70 2024-04-26T23:56:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.443+0000 7ff450e00640 1 --2- 172.21.15.46:0/1404661686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c0d2e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009d90 tx=0x7ff43c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f9a88faf6dca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.443+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438001670 con 0x7ff44c162a70 2024-04-26T23:56:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.443+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43c007540 con 0x7ff44c162a70 2024-04-26T23:56:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.443+0000 7ff44b400640 1 -- 172.21.15.46:0/1404661686 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c007910 con 0x7ff44c162a70 2024-04-26T23:56:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 -- 172.21.15.46:0/1404661686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 msgr2=0x7ff44c0d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 --2- 172.21.15.46:0/1404661686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c0d2e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009d90 tx=0x7ff43c02a680 comp rx=0 tx=0).stop 2024-04-26T23:56:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 -- 172.21.15.46:0/1404661686 shutdown_connections 2024-04-26T23:56:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 --2- 172.21.15.46:0/1404661686 >> v2:172.21.15.142:3300/0 conn(0x7ff44c0d1cb0 0x7ff44c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 --2- 172.21.15.46:0/1404661686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c0d2e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 -- 172.21.15.46:0/1404661686 >> 172.21.15.46:0/1404661686 conn(0x7ff44c0d4f20 msgr2=0x7ff44c0d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 -- 172.21.15.46:0/1404661686 shutdown_connections 2024-04-26T23:56:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.444+0000 7ff452200640 1 -- 172.21.15.46:0/1404661686 wait complete. 2024-04-26T23:56:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.446+0000 7ff452200640 1 Processor -- start 2024-04-26T23:56:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.446+0000 7ff452200640 1 -- start start 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.447+0000 7ff452200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 0x7ff44c15db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.447+0000 7ff452200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.447+0000 7ff452200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff452200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c162fd0 con 0x7ff44c0d3830 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff452200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c162e50 con 0x7ff44c0d1cb0 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff452200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff44c0b2f20 con 0x7ff44c162a70 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50592/0 (socket says 172.21.15.46:50592) 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 -- 172.21.15.46:0/4048742458 learned_addr learned my addr 172.21.15.46:0/4048742458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff450e00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 0x7ff44c15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff451800640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 -- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 msgr2=0x7ff44c14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 -- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 msgr2=0x7ff44c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 0x7ff44c15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff44be00640 1 -- 172.21.15.46:0/4048742458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c0a22a0 con 0x7ff44c0d3830 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff451800640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.448+0000 7ff450e00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 0x7ff44c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:18.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff44be00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c780 tx=0x7ff43800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x7ff44c0d3830 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff438005ad0 con 0x7ff44c0d3830 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c0a2580 con 0x7ff44c0d3830 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438007070 con 0x7ff44c0d3830 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.449+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c0a2b40 con 0x7ff44c0d3830 2024-04-26T23:56:18.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.450+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44c0693b0 con 0x7ff44c0d3830 2024-04-26T23:56:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.451+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff43801a070 con 0x7ff44c0d3830 2024-04-26T23:56:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.452+0000 7ff448c00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 0x7ff42c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.452+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6274+0+0 (secure 0 0 0) 0x7ff43808b420 con 0x7ff44c0d3830 2024-04-26T23:56:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.453+0000 7ff450e00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 0x7ff42c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.454+0000 7ff450e00640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 0x7ff42c06edb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.457+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff438059570 con 0x7ff44c0d3830 2024-04-26T23:56:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:18.707+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff44c076270 con 0x7ff44c0d3830 2024-04-26T23:56:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.223+0000 7ff448c00640 1 -- 172.21.15.46:0/4048742458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v151) ==== 104+0+0 (secure 0 0 0) 0x7ff438058e70 con 0x7ff44c0d3830 2024-04-26T23:56:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 msgr2=0x7ff42c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 0x7ff42c06edb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c033040 comp rx=0 tx=0).stop 2024-04-26T23:56:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 msgr2=0x7ff44c0caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c780 tx=0x7ff43800cc40 comp rx=0 tx=0).stop 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 shutdown_connections 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff42c06c8f0 0x7ff42c06edb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff44c162a70 0x7ff44c14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff44c0d3830 0x7ff44c0caa10 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 --2- 172.21.15.46:0/4048742458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44c0d1cb0 0x7ff44c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 >> 172.21.15.46:0/4048742458 conn(0x7ff44c0d4f20 msgr2=0x7ff44c1e81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 shutdown_connections 2024-04-26T23:56:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.227+0000 7ff452200640 1 -- 172.21.15.46:0/4048742458 wait complete. 2024-04-26T23:56:19.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:56:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.521+0000 7fa9efc00640 1 Processor -- start 2024-04-26T23:56:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.522+0000 7fa9efc00640 1 -- start start 2024-04-26T23:56:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.523+0000 7fa9efc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9e815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.523+0000 7fa9efc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e815fdf0 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.523+0000 7fa9efc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e81600d0 con 0x7fa9e806eac0 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.523+0000 7fa9efc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e81603b0 con 0x7fa9e80b47b0 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.524+0000 7fa9ed200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9e80b2ec0 0x7fa9e80b32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49102/0 (socket says 172.21.15.46:49102) 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.524+0000 7fa9ed200640 1 -- 172.21.15.46:0/1119483335 learned_addr learned my addr 172.21.15.46:0/1119483335 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.524+0000 7fa9edc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9e815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.524+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650782432 0 0) 0x7fa9e815fdf0 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.524+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9d0003520 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2515679064 0 0) 0x7fa9d0003520 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 msgr2=0x7fa9e815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9e815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 >> v1:172.21.15.142:6789/0 conn(0x7fa9e806eac0 legacy=0x7fa9e8072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8160690 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa9d8001670 con 0x7fa9e80b2ec0 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 >> v1:172.21.15.46:6789/0 conn(0x7fa9e80b2ec0 legacy=0x7fa9e80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d0003f60 0x7fa9d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d0004d40 0x7fa9d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81603b0 con 0x7fa9d0004d40 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d000baa0 con 0x7fa9d0003f60 2024-04-26T23:56:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d000bca0 con 0x7fa9e80b47b0 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9edc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d0004d40 0x7fa9d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9ed200640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9d00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.525+0000 7fa9e7e00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d0003f60 0x7fa9d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.526+0000 7fa9edc00640 1 -- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 msgr2=0x7fa9d00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.526+0000 7fa9edc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b47b0 0x7fa9d00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.526+0000 7fa9edc00640 1 -- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d0003f60 msgr2=0x7fa9d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.526+0000 7fa9edc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d0003f60 0x7fa9d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.526+0000 7fa9edc00640 1 -- 172.21.15.46:0/1119483335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8160690 con 0x7fa9d0004d40 2024-04-26T23:56:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.527+0000 7fa9edc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d0004d40 0x7fa9d00051c0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009fc0 tx=0x7fa9d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3a7874488e35ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.527+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0009c10 con 0x7fa9d0004d40 2024-04-26T23:56:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.527+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0036070 con 0x7fa9d0004d40 2024-04-26T23:56:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.527+0000 7fa9e7400640 1 -- 172.21.15.46:0/1119483335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0031880 con 0x7fa9d0004d40 2024-04-26T23:56:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 -- 172.21.15.46:0/1119483335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d0004d40 msgr2=0x7fa9d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 --2- 172.21.15.46:0/1119483335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d0004d40 0x7fa9d00051c0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009fc0 tx=0x7fa9d0005830 comp rx=0 tx=0).stop 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9edc00640 1 -- 172.21.15.46:0/1119483335 reap_dead start 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 -- 172.21.15.46:0/1119483335 shutdown_connections 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 -- 172.21.15.46:0/1119483335 >> 172.21.15.46:0/1119483335 conn(0x7fa9e806c770 msgr2=0x7fa9e806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 -- 172.21.15.46:0/1119483335 shutdown_connections 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.528+0000 7fa9efc00640 1 -- 172.21.15.46:0/1119483335 wait complete. 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.530+0000 7fa9efc00640 1 Processor -- start 2024-04-26T23:56:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.530+0000 7fa9efc00640 1 -- start start 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e81f5490 0x7fa9e80b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b1fd0 0x7fa9e815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80b4040 con 0x7fa9e8153520 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80b3ec0 con 0x7fa9e81f5490 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9efc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e8072160 con 0x7fa9e80b1fd0 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9ed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50608/0 (socket says 172.21.15.46:50608) 2024-04-26T23:56:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.531+0000 7fa9ed200640 1 -- 172.21.15.46:0/822790852 learned_addr learned my addr 172.21.15.46:0/822790852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 -- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b1fd0 msgr2=0x7fa9e815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b1fd0 0x7fa9e815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 -- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e81f5490 msgr2=0x7fa9e80b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e81f5490 0x7fa9e80b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 -- 172.21.15.46:0/822790852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8153900 con 0x7fa9e8153520 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9ed200640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8004840 tx=0x7fa9d8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800c8d0 con 0x7fa9e8153520 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d8018070 con 0x7fa9e8153520 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.532+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800ccd0 con 0x7fa9e8153520 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.533+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e815a060 con 0x7fa9e8153520 2024-04-26T23:56:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.533+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e815a620 con 0x7fa9e8153520 2024-04-26T23:56:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.534+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8069530 con 0x7fa9e8153520 2024-04-26T23:56:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.535+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9d801d460 con 0x7fa9e8153520 2024-04-26T23:56:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.536+0000 7fa9e4c00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 0x7fa9cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.536+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6274+0+0 (secure 0 0 0) 0x7fa9d808bd00 con 0x7fa9e8153520 2024-04-26T23:56:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.537+0000 7fa9e7e00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 0x7fa9cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.538+0000 7fa9e7e00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 0x7fa9cc06ede0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007fd0 tx=0x7fa9dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.542+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9d8059f80 con 0x7fa9e8153520 2024-04-26T23:56:19.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:19.752+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9e8069c60 con 0x7fa9e8153520 2024-04-26T23:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.227+0000 7fa9e4c00640 1 -- 172.21.15.46:0/822790852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v190) ==== 181+0+0 (secure 0 0 0) 0x7fa9d8059880 con 0x7fa9e8153520 2024-04-26T23:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:56:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.231+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 msgr2=0x7fa9cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 0x7fa9cc06ede0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007fd0 tx=0x7fa9dc009040 comp rx=0 tx=0).stop 2024-04-26T23:56:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 msgr2=0x7fa9e81f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8004840 tx=0x7fa9d8005e10 comp rx=0 tx=0).stop 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 shutdown_connections 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa9cc06c920 0x7fa9cc06ede0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9e80b1fd0 0x7fa9e815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e81f5490 0x7fa9e80b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 --2- 172.21.15.46:0/822790852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e8153520 0x7fa9e81f4f50 secure :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8004840 tx=0x7fa9d8005e10 comp rx=0 tx=0).stop 2024-04-26T23:56:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.232+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 >> 172.21.15.46:0/822790852 conn(0x7fa9e806c770 msgr2=0x7fa9e80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.236+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 shutdown_connections 2024-04-26T23:56:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.236+0000 7fa9efc00640 1 -- 172.21.15.46:0/822790852 wait complete. 2024-04-26T23:56:20.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:56:20.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.460+0000 7f9248a00640 1 Processor -- start 2024-04-26T23:56:20.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.460+0000 7f9248a00640 1 -- start start 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9248a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f924415d6d0 0x7f92440b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9248a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9244146da0 con 0x7f92440b3400 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9248a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92440cfeb0 con 0x7f92441469c0 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9248a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f92440cfc70 con 0x7f924415d6d0 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9242800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92440b3400 0x7f92440727d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49106/0 (socket says 172.21.15.46:49106) 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9242800640 1 -- 172.21.15.46:0/3245539050 learned_addr learned my addr 172.21.15.46:0/3245539050 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.462+0000 7f9241400640 1 --2- 172.21.15.46:0/3245539050 >> v2:172.21.15.46:3300/0 conn(0x7f924415d6d0 0x7f92440b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.463+0000 7f9240a00640 1 -- 172.21.15.46:0/3245539050 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290256275 0 0) 0x7f9244146da0 con 0x7f92440b3400 2024-04-26T23:56:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.463+0000 7f9240a00640 1 -- 172.21.15.46:0/3245539050 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9228003520 con 0x7f92440b3400 2024-04-26T23:56:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.463+0000 7f9241400640 1 -- 172.21.15.46:0/3245539050 >> v1:172.21.15.142:6789/0 conn(0x7f92441469c0 legacy=0x7f92440cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.463+0000 7f9241400640 1 -- 172.21.15.46:0/3245539050 >> v1:172.21.15.46:6789/0 conn(0x7f92440b3400 legacy=0x7f92440727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.463+0000 7f9241400640 1 -- 172.21.15.46:0/3245539050 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92440d0190 con 0x7f924415d6d0 2024-04-26T23:56:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.464+0000 7f9241400640 1 --2- 172.21.15.46:0/3245539050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924415d6d0 0x7f92440b2ec0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f922c0097a0 tx=0x7f922c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57623a15861f2af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.464+0000 7f9240a00640 1 -- 172.21.15.46:0/3245539050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c037070 con 0x7f924415d6d0 2024-04-26T23:56:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.464+0000 7f9240a00640 1 -- 172.21.15.46:0/3245539050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c032440 con 0x7f924415d6d0 2024-04-26T23:56:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9240a00640 1 -- 172.21.15.46:0/3245539050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c0327f0 con 0x7f924415d6d0 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 -- 172.21.15.46:0/3245539050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924415d6d0 msgr2=0x7f92440b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 --2- 172.21.15.46:0/3245539050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924415d6d0 0x7f92440b2ec0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f922c0097a0 tx=0x7f922c02a230 comp rx=0 tx=0).stop 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 -- 172.21.15.46:0/3245539050 shutdown_connections 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 --2- 172.21.15.46:0/3245539050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f924415d6d0 0x7f92440b2ec0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 -- 172.21.15.46:0/3245539050 >> 172.21.15.46:0/3245539050 conn(0x7f924406c770 msgr2=0x7f924406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 -- 172.21.15.46:0/3245539050 shutdown_connections 2024-04-26T23:56:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.465+0000 7f9248a00640 1 -- 172.21.15.46:0/3245539050 wait complete. 2024-04-26T23:56:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.467+0000 7f9248a00640 1 Processor -- start 2024-04-26T23:56:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.467+0000 7f9248a00640 1 -- start start 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92440b3400 0x7f924415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924415d6d0 0x7f92440b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92440d0ac0 con 0x7f92441469c0 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92440b3850 con 0x7f92440b3400 2024-04-26T23:56:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.468+0000 7f9248a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92440d0750 con 0x7f924415d6d0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9241400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9241400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50624/0 (socket says 172.21.15.46:50624) 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9241400640 1 -- 172.21.15.46:0/1035359636 learned_addr learned my addr 172.21.15.46:0/1035359636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9242800640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924415d6d0 0x7f92440b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9241e00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92440b3400 0x7f924415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.469+0000 7f9241400640 1 -- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924415d6d0 msgr2=0x7f92440b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9241400640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924415d6d0 0x7f92440b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9241400640 1 -- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92440b3400 msgr2=0x7f924415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9241400640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92440b3400 0x7f924415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9241400640 1 -- 172.21.15.46:0/1035359636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92440b4b00 con 0x7f92441469c0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9241400640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f922c0097a0 tx=0x7f922c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c037050 con 0x7f92441469c0 2024-04-26T23:56:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c038040 con 0x7f92441469c0 2024-04-26T23:56:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c004470 con 0x7f92441469c0 2024-04-26T23:56:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92440b4de0 con 0x7f92441469c0 2024-04-26T23:56:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.470+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92440b53a0 con 0x7f92441469c0 2024-04-26T23:56:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.471+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244069530 con 0x7f92441469c0 2024-04-26T23:56:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.472+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f922c03c6d0 con 0x7f92441469c0 2024-04-26T23:56:20.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.473+0000 7f923a000640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:20.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.474+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5836+0+0 (secure 0 0 0) 0x7f922c0ac690 con 0x7f92441469c0 2024-04-26T23:56:20.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.474+0000 7f9241e00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.475+0000 7f9241e00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 0x7f922006edb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9234001640 tx=0x7f9234009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.478+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f922c07aa40 con 0x7f92441469c0 2024-04-26T23:56:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:20.685+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f924406ced0 con 0x7f92441469c0 2024-04-26T23:56:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.239+0000 7f923a000640 1 -- 172.21.15.46:0/1035359636 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v191) ==== 169+0+0 (secure 0 0 0) 0x7f922c07a340 con 0x7f92441469c0 2024-04-26T23:56:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 msgr2=0x7f922006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 0x7f922006edb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9234001640 tx=0x7f9234009040 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 msgr2=0x7f92440c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f922c0097a0 tx=0x7f922c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 shutdown_connections 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f924415d6d0 0x7f92440b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92441469c0 0x7f92440c6600 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 --2- 172.21.15.46:0/1035359636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92440b3400 0x7f924415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 >> 172.21.15.46:0/1035359636 conn(0x7f924406c770 msgr2=0x7f9244072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 shutdown_connections 2024-04-26T23:56:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.244+0000 7f9248a00640 1 -- 172.21.15.46:0/1035359636 wait complete. 2024-04-26T23:56:21.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:56:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.464+0000 7f83a4a00640 1 Processor -- start 2024-04-26T23:56:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.464+0000 7f83a4a00640 1 -- start start 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.465+0000 7f83a4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a015f840 0x7f83a00d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.465+0000 7f83a4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a00d4090 con 0x7f83a00d24f0 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.465+0000 7f83a4a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a00b4020 con 0x7f83a00d1980 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.465+0000 7f83a4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00d3880 con 0x7f83a015f840 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.466+0000 7f839f200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83a00d24f0 0x7f83a00b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49110/0 (socket says 172.21.15.46:49110) 2024-04-26T23:56:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.466+0000 7f839f200640 1 -- 172.21.15.46:0/2725888291 learned_addr learned my addr 172.21.15.46:0/2725888291 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:21.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.466+0000 7f839e800640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a015f840 0x7f83a00d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.466+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965102330 0 0) 0x7f8398001110 con 0x7f83a00d24f0 2024-04-26T23:56:21.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.466+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8384003520 con 0x7f83a00d24f0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1658067981 0 0) 0x7f8384003520 con 0x7f83a00d24f0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a015f840 msgr2=0x7f83a00d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a015f840 0x7f83a00d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 >> v1:172.21.15.142:6789/0 conn(0x7f83a00d1980 legacy=0x7f83a00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00d3ad0 con 0x7f83a00d24f0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f8398001840 con 0x7f83a00d24f0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 >> v1:172.21.15.46:6789/0 conn(0x7f83a00d24f0 legacy=0x7f83a00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83840039e0 0x7f8384003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840043d0 0x7f83840048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83840050a0 0x7f838400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d3880 con 0x7f83840050a0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f838400bfb0 con 0x7f83840043d0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f838400c1b0 con 0x7f83840039e0 2024-04-26T23:56:21.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839f200640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83840050a0 0x7f838400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.467+0000 7f839e800640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83840039e0 0x7f8384003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839de00640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840043d0 0x7f83840048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839f200640 1 -- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83840039e0 msgr2=0x7f8384003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839f200640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83840039e0 0x7f8384003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839f200640 1 -- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840043d0 msgr2=0x7f83840048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839f200640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840043d0 0x7f83840048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.468+0000 7f839f200640 1 -- 172.21.15.46:0/2725888291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8384003520 con 0x7f83840050a0 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.469+0000 7f839f200640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83840050a0 0x7f838400b9c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f839800c490 tx=0x7f8384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f54e5e86031c6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.469+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839800c0e0 con 0x7f83840050a0 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.469+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8398034070 con 0x7f83840050a0 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.469+0000 7f839d400640 1 -- 172.21.15.46:0/2725888291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398005db0 con 0x7f83840050a0 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 -- 172.21.15.46:0/2725888291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83840050a0 msgr2=0x7f838400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2725888291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83840050a0 0x7f838400b9c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f839800c490 tx=0x7f8384005830 comp rx=0 tx=0).stop 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f839f200640 1 -- 172.21.15.46:0/2725888291 reap_dead start 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 -- 172.21.15.46:0/2725888291 shutdown_connections 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 -- 172.21.15.46:0/2725888291 >> 172.21.15.46:0/2725888291 conn(0x7f83a00d4f70 msgr2=0x7f83a00d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 -- 172.21.15.46:0/2725888291 shutdown_connections 2024-04-26T23:56:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.470+0000 7f83a4a00640 1 -- 172.21.15.46:0/2725888291 wait complete. 2024-04-26T23:56:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.472+0000 7f83a4a00640 1 Processor -- start 2024-04-26T23:56:21.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.472+0000 7f83a4a00640 1 -- start start 2024-04-26T23:56:21.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840050a0 0x7f83a014b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a00b8050 0x7f83a00ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d2e70 con 0x7f83a00b7bd0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00b27d0 con 0x7f83840050a0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f83a4a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00d2ff0 con 0x7f83a00b8050 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f839de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f839de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50632/0 (socket says 172.21.15.46:50632) 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f839de00640 1 -- 172.21.15.46:0/2789024331 learned_addr learned my addr 172.21.15.46:0/2789024331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f839e800640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840050a0 0x7f83a014b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.473+0000 7f839f200640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a00b8050 0x7f83a00ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 -- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a00b8050 msgr2=0x7f83a00ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a00b8050 0x7f83a00ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 -- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840050a0 msgr2=0x7f83a014b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840050a0 0x7f83a014b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 -- 172.21.15.46:0/2789024331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00bac10 con 0x7f83a00b7bd0 2024-04-26T23:56:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f839de00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f838800e4c0 tx=0x7f838800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880052e0 con 0x7f83a00b7bd0 2024-04-26T23:56:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8388018070 con 0x7f83a00b7bd0 2024-04-26T23:56:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a00ba690 con 0x7f83a00b7bd0 2024-04-26T23:56:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.474+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388003830 con 0x7f83a00b7bd0 2024-04-26T23:56:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.475+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a014b840 con 0x7f83a00b7bd0 2024-04-26T23:56:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.476+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00cda70 con 0x7f83a00b7bd0 2024-04-26T23:56:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.477+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83880039d0 con 0x7f83a00b7bd0 2024-04-26T23:56:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.478+0000 7f8396a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.478+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7f838808b660 con 0x7f83a00b7bd0 2024-04-26T23:56:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.478+0000 7f839e800640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.480+0000 7f839e800640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 0x7f837c06edb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8390009fd0 tx=0x7f839000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.483+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8388059cb0 con 0x7f83a00b7bd0 2024-04-26T23:56:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.689+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f83a00693b0 con 0x7f83a00b7bd0 2024-04-26T23:56:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.696+0000 7f8396a00640 1 -- 172.21.15.46:0/2789024331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v47) ==== 104+0+0 (secure 0 0 0) 0x7f83880595b0 con 0x7f83a00b7bd0 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 msgr2=0x7f837c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 0x7f837c06edb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8390009fd0 tx=0x7f839000b040 comp rx=0 tx=0).stop 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 msgr2=0x7f83a00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f838800e4c0 tx=0x7f838800e980 comp rx=0 tx=0).stop 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 shutdown_connections 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83a00b8050 0x7f83a00ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83a00b7bd0 0x7f83a00ce0b0 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 --2- 172.21.15.46:0/2789024331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83840050a0 0x7f83a014b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 >> 172.21.15.46:0/2789024331 conn(0x7f83a00d4f70 msgr2=0x7f83a015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 shutdown_connections 2024-04-26T23:56:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.699+0000 7f83a4a00640 1 -- 172.21.15.46:0/2789024331 wait complete. 2024-04-26T23:56:21.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-26T23:56:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.930+0000 7f6cbca00640 1 Processor -- start 2024-04-26T23:56:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.930+0000 7f6cbca00640 1 -- start start 2024-04-26T23:56:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cbca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cb80cf450 0x7f6cb8073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cbca00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6cb80cff40 0x7f6cb8166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cbca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb806f1d0 con 0x7f6cb8072190 2024-04-26T23:56:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cbca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb806f470 con 0x7f6cb80cf450 2024-04-26T23:56:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cbca00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb8072930 con 0x7f6cb80cff40 2024-04-26T23:56:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6caea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6cb8072190 0x7f6cb80cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49426/0 (socket says 172.21.15.46:49426) 2024-04-26T23:56:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6cb7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cb80cf450 0x7f6cb8073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.932+0000 7f6caea00640 1 -- 172.21.15.46:0/1328261420 learned_addr learned my addr 172.21.15.46:0/1328261420 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.933+0000 7f6cb7e00640 1 --2- 172.21.15.46:0/1328261420 >> v2:172.21.15.142:3300/0 conn(0x7f6cb80cff40 0x7f6cb8166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.933+0000 7f6cb7400640 1 -- 172.21.15.46:0/1328261420 >> v2:172.21.15.142:3300/0 conn(0x7f6cb80cff40 msgr2=0x7f6cb8166b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.933+0000 7f6cb7400640 1 --2- 172.21.15.46:0/1328261420 >> v2:172.21.15.142:3300/0 conn(0x7f6cb80cff40 0x7f6cb8166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.933+0000 7f6cb7400640 1 -- 172.21.15.46:0/1328261420 >> v1:172.21.15.46:6789/0 conn(0x7f6cb8072190 legacy=0x7f6cb80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.933+0000 7f6cb7400640 1 -- 172.21.15.46:0/1328261420 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8072c10 con 0x7f6cb80cf450 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.934+0000 7f6cb7400640 1 --2- 172.21.15.46:0/1328261420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cf450 0x7f6cb8073db0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f6cb806f3a0 tx=0x7f6ca802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f913c6651e4ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.934+0000 7f6cb6a00640 1 -- 172.21.15.46:0/1328261420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8037070 con 0x7f6cb80cf450 2024-04-26T23:56:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.934+0000 7f6cb6a00640 1 -- 172.21.15.46:0/1328261420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8032440 con 0x7f6cb80cf450 2024-04-26T23:56:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.934+0000 7f6cb6a00640 1 -- 172.21.15.46:0/1328261420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8032810 con 0x7f6cb80cf450 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 -- 172.21.15.46:0/1328261420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cf450 msgr2=0x7f6cb8073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 --2- 172.21.15.46:0/1328261420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cf450 0x7f6cb8073db0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f6cb806f3a0 tx=0x7f6ca802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 -- 172.21.15.46:0/1328261420 shutdown_connections 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 --2- 172.21.15.46:0/1328261420 >> v2:172.21.15.142:3300/0 conn(0x7f6cb80cff40 0x7f6cb8166b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 --2- 172.21.15.46:0/1328261420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cf450 0x7f6cb8073db0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 -- 172.21.15.46:0/1328261420 >> 172.21.15.46:0/1328261420 conn(0x7f6cb80b2820 msgr2=0x7f6cb80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 -- 172.21.15.46:0/1328261420 shutdown_connections 2024-04-26T23:56:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.935+0000 7f6cbca00640 1 -- 172.21.15.46:0/1328261420 wait complete. 2024-04-26T23:56:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.937+0000 7f6cbca00640 1 Processor -- start 2024-04-26T23:56:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.937+0000 7f6cbca00640 1 -- start start 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6cb8072190 0x7f6cb813f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80cf450 0x7f6cb81e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80734b0 con 0x7f6cb80cff40 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80731d0 con 0x7f6cb80cf450 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cbca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb8072620 con 0x7f6cb8072190 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37594/0 (socket says 172.21.15.46:37594) 2024-04-26T23:56:21.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.938+0000 7f6cb7e00640 1 -- 172.21.15.46:0/4190849535 learned_addr learned my addr 172.21.15.46:0/4190849535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6caea00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80cf450 0x7f6cb81e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7400640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6cb8072190 0x7f6cb813f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 -- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6cb8072190 msgr2=0x7f6cb813f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6cb8072190 0x7f6cb813f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 -- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80cf450 msgr2=0x7f6cb81e7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80cf450 0x7f6cb81e7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 -- 172.21.15.46:0/4190849535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb81f6550 con 0x7f6cb80cff40 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.939+0000 7f6cb7e00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000cd70 tx=0x7f6cb000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.940+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0003b90 con 0x7f6cb80cff40 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.940+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb0003d30 con 0x7f6cb80cff40 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.940+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0004240 con 0x7f6cb80cff40 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.940+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb81f6830 con 0x7f6cb80cff40 2024-04-26T23:56:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.940+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb81f6df0 con 0x7f6cb80cff40 2024-04-26T23:56:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.941+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb81f6ff0 con 0x7f6cb80cff40 2024-04-26T23:56:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.942+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cb00043e0 con 0x7f6cb80cff40 2024-04-26T23:56:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.943+0000 7f6cafe00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.943+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7f6cb008c5b0 con 0x7f6cb80cff40 2024-04-26T23:56:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.944+0000 7f6cb7400640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:21.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.945+0000 7f6cb7400640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 0x7f6c9406edb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8034000 tx=0x7f6ca802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:21.948+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6cb81f6ff0 con 0x7f6cb80cff40 2024-04-26T23:56:22.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.225+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) -- 0x7f6cb80693b0 con 0x7f6cb80cff40 2024-04-26T23:56:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.242+0000 7f6cafe00640 1 -- 172.21.15.46:0/4190849535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v877) ==== 133+0+0 (secure 0 0 0) 0x7f6cb001d340 con 0x7f6cb80cff40 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.245+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 msgr2=0x7f6c9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.245+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 0x7f6c9406edb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8034000 tx=0x7f6ca802e000 comp rx=0 tx=0).stop 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.245+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 msgr2=0x7f6cb81e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.245+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000cd70 tx=0x7f6cb000a660 comp rx=0 tx=0).stop 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.245+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 shutdown_connections 2024-04-26T23:56:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb80cff40 0x7f6cb81e8090 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80cf450 0x7f6cb81e7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 --2- 172.21.15.46:0/4190849535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6cb8072190 0x7f6cb813f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 >> 172.21.15.46:0/4190849535 conn(0x7f6cb80b2820 msgr2=0x7f6cb80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 shutdown_connections 2024-04-26T23:56:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.246+0000 7f6cbca00640 1 -- 172.21.15.46:0/4190849535 wait complete. 2024-04-26T23:56:22.269 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-26T23:56:22.269 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-26T23:56:22.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.456+0000 7fee7d600640 1 Processor -- start 2024-04-26T23:56:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.457+0000 7fee7d600640 1 -- start start 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee7d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee7815ccd0 0x7fee780d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee7d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee780cf950 con 0x7fee78073570 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee7d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee780cfc30 con 0x7fee78145770 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee7d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fee780d0c50 con 0x7fee7815ccd0 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee6e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fee78073570 0x7fee7815c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49428/0 (socket says 172.21.15.46:49428) 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee6e000640 1 -- 172.21.15.46:0/1022254731 learned_addr learned my addr 172.21.15.46:0/1022254731 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.458+0000 7fee77400640 1 --2- 172.21.15.46:0/1022254731 >> v2:172.21.15.46:3300/0 conn(0x7fee7815ccd0 0x7fee780d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.459+0000 7fee76000640 1 -- 172.21.15.46:0/1022254731 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160302495 0 0) 0x7fee780cf950 con 0x7fee78073570 2024-04-26T23:56:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.459+0000 7fee76000640 1 -- 172.21.15.46:0/1022254731 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee5c005730 con 0x7fee78073570 2024-04-26T23:56:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.459+0000 7fee77400640 1 -- 172.21.15.46:0/1022254731 >> v1:172.21.15.142:6789/0 conn(0x7fee78145770 legacy=0x7fee78072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.459+0000 7fee77400640 1 -- 172.21.15.46:0/1022254731 >> v1:172.21.15.46:6789/0 conn(0x7fee78073570 legacy=0x7fee7815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.459+0000 7fee77400640 1 -- 172.21.15.46:0/1022254731 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee780cff10 con 0x7fee7815ccd0 2024-04-26T23:56:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.460+0000 7fee77400640 1 --2- 172.21.15.46:0/1022254731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7815ccd0 0x7fee780d0710 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fee70009d90 tx=0x7fee7002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eea653a418f484f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.460+0000 7fee76000640 1 -- 172.21.15.46:0/1022254731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70007740 con 0x7fee7815ccd0 2024-04-26T23:56:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.460+0000 7fee76000640 1 -- 172.21.15.46:0/1022254731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee700078e0 con 0x7fee7815ccd0 2024-04-26T23:56:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.460+0000 7fee76000640 1 -- 172.21.15.46:0/1022254731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70007c90 con 0x7fee7815ccd0 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 -- 172.21.15.46:0/1022254731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7815ccd0 msgr2=0x7fee780d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 --2- 172.21.15.46:0/1022254731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7815ccd0 0x7fee780d0710 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fee70009d90 tx=0x7fee7002a660 comp rx=0 tx=0).stop 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 -- 172.21.15.46:0/1022254731 shutdown_connections 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 --2- 172.21.15.46:0/1022254731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7815ccd0 0x7fee780d0710 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 -- 172.21.15.46:0/1022254731 >> 172.21.15.46:0/1022254731 conn(0x7fee7806c530 msgr2=0x7fee7806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 -- 172.21.15.46:0/1022254731 shutdown_connections 2024-04-26T23:56:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.461+0000 7fee7d600640 1 -- 172.21.15.46:0/1022254731 wait complete. 2024-04-26T23:56:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.463+0000 7fee7d600640 1 Processor -- start 2024-04-26T23:56:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.463+0000 7fee7d600640 1 -- start start 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee78073570 0x7fee78144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 0x7fee7815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee780cf5b0 con 0x7fee78145770 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee7815d490 con 0x7fee7815ccd0 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee7d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee780cf730 con 0x7fee78073570 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37610/0 (socket says 172.21.15.46:37610) 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 -- 172.21.15.46:0/200606696 learned_addr learned my addr 172.21.15.46:0/200606696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee76a00640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee78073570 0x7fee78144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 -- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee78073570 msgr2=0x7fee78144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee77400640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 0x7fee7815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee78073570 0x7fee78144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.464+0000 7fee6e000640 1 -- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 msgr2=0x7fee7815e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6e000640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 0x7fee7815e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6e000640 1 -- 172.21.15.46:0/200606696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee781f10b0 con 0x7fee78145770 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee77400640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 0x7fee7815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6e000640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fee60005880 tx=0x7fee6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60005ce0 con 0x7fee78145770 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee60007070 con 0x7fee78145770 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60007440 con 0x7fee78145770 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee781f1390 con 0x7fee78145770 2024-04-26T23:56:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.465+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee78142fb0 con 0x7fee78145770 2024-04-26T23:56:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.466+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee78069530 con 0x7fee78145770 2024-04-26T23:56:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.467+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee600075e0 con 0x7fee78145770 2024-04-26T23:56:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.469+0000 7fee6f400640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 0x7fee5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.469+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7fee6008b730 con 0x7fee78145770 2024-04-26T23:56:22.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.469+0000 7fee76a00640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 0x7fee5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.471+0000 7fee76a00640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 0x7fee5406edb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fee68000f60 tx=0x7fee68009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.473+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee60059c30 con 0x7fee78145770 2024-04-26T23:56:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.679+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee78143180 con 0x7fee78145770 2024-04-26T23:56:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.681+0000 7fee6f400640 1 -- 172.21.15.46:0/200606696 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) ==== 74+0+13267 (secure 0 0 0) 0x7fee60059530 con 0x7fee78145770 2024-04-26T23:56:22.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:22.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":191,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:21.229008+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2117263293":"2024-04-27T23:56:18.207111+0000","172.21.15.46:6834/2117263293":"2024-04-27T23:56:18.207111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 msgr2=0x7fee5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 0x7fee5406edb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fee68000f60 tx=0x7fee68009040 comp rx=0 tx=0).stop 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 msgr2=0x7fee781434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fee60005880 tx=0x7fee6000c960 comp rx=0 tx=0).stop 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 shutdown_connections 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee5406c8f0 0x7fee5406edb0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee7815ccd0 0x7fee7815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee78145770 0x7fee781434d0 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 --2- 172.21.15.46:0/200606696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee78073570 0x7fee78144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 >> 172.21.15.46:0/200606696 conn(0x7fee7806c530 msgr2=0x7fee780b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.684+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 shutdown_connections 2024-04-26T23:56:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.685+0000 7fee7d600640 1 -- 172.21.15.46:0/200606696 wait complete. 2024-04-26T23:56:22.707 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:22.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.918+0000 7eff2ae00640 1 Processor -- start 2024-04-26T23:56:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.919+0000 7eff2ae00640 1 -- start start 2024-04-26T23:56:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.920+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff240b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.920+0000 7eff2ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff240b5fb0 0x7eff240d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.920+0000 7eff2ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff240d07d0 con 0x7eff24162aa0 2024-04-26T23:56:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.920+0000 7eff2ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7eff240d0ab0 con 0x7eff240b5fb0 2024-04-26T23:56:22.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.920+0000 7eff2ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7eff240d0d90 con 0x7eff241626c0 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff29000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff240b5fb0 0x7eff240d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff2a400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7eff24162aa0 0x7eff240d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49444/0 (socket says 172.21.15.46:49444) 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff2a400640 1 -- 172.21.15.46:0/1247300449 learned_addr learned my addr 172.21.15.46:0/1247300449 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff29a00640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff240b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff23e00640 1 -- 172.21.15.46:0/1247300449 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627140301 0 0) 0x7eff240d07d0 con 0x7eff24162aa0 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff29000640 1 -- 172.21.15.46:0/1247300449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 msgr2=0x7eff240b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff29000640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff240b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.921+0000 7eff29000640 1 -- 172.21.15.46:0/1247300449 >> v1:172.21.15.46:6789/0 conn(0x7eff24162aa0 legacy=0x7eff240d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.922+0000 7eff29000640 1 -- 172.21.15.46:0/1247300449 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff240d2c80 con 0x7eff240b5fb0 2024-04-26T23:56:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.923+0000 7eff29000640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240d5850 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7eff10009790 tx=0x7eff1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b2051ebe166aff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.923+0000 7eff23e00640 1 -- 172.21.15.46:0/1247300449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10037070 con 0x7eff240b5fb0 2024-04-26T23:56:22.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.923+0000 7eff23e00640 1 -- 172.21.15.46:0/1247300449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff10032440 con 0x7eff240b5fb0 2024-04-26T23:56:22.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.923+0000 7eff23e00640 1 -- 172.21.15.46:0/1247300449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10032810 con 0x7eff240b5fb0 2024-04-26T23:56:22.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 -- 172.21.15.46:0/1247300449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 msgr2=0x7eff240d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240d5850 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7eff10009790 tx=0x7eff1002a180 comp rx=0 tx=0).stop 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 -- 172.21.15.46:0/1247300449 shutdown_connections 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240d5850 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 --2- 172.21.15.46:0/1247300449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff240b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 -- 172.21.15.46:0/1247300449 >> 172.21.15.46:0/1247300449 conn(0x7eff2409e2a0 msgr2=0x7eff240d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 -- 172.21.15.46:0/1247300449 shutdown_connections 2024-04-26T23:56:22.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.924+0000 7eff2ae00640 1 -- 172.21.15.46:0/1247300449 wait complete. 2024-04-26T23:56:22.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.926+0000 7eff2ae00640 1 Processor -- start 2024-04-26T23:56:22.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.926+0000 7eff2ae00640 1 -- start start 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff2415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24162aa0 0x7eff240b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff240d2190 con 0x7eff240b5fb0 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff240d2010 con 0x7eff24162aa0 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7eff240d31a0 con 0x7eff241626c0 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37628/0 (socket says 172.21.15.46:37628) 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 -- 172.21.15.46:0/802812386 learned_addr learned my addr 172.21.15.46:0/802812386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29000640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff2415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 -- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 msgr2=0x7eff2415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff2415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 -- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24162aa0 msgr2=0x7eff240b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff2a400640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24162aa0 0x7eff240b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.928+0000 7eff29a00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24162aa0 0x7eff240b8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff29a00640 1 -- 172.21.15.46:0/802812386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff240a10b0 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff29a00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7eff1800e9b0 tx=0x7eff1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18017070 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1800c9a0 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff240a1390 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1800cd90 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.929+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff240a1950 con 0x7eff240b5fb0 2024-04-26T23:56:22.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.930+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2408f090 con 0x7eff240b5fb0 2024-04-26T23:56:22.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.931+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff1801a070 con 0x7eff240b5fb0 2024-04-26T23:56:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.932+0000 7eff21600640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 0x7eff0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.933+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7eff1808bdc0 con 0x7eff240b5fb0 2024-04-26T23:56:22.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.933+0000 7eff29000640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 0x7eff0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:22.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.935+0000 7eff29000640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 0x7eff0406edb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7eff10034000 tx=0x7eff1002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:22.937+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff2408f090 con 0x7eff240b5fb0 2024-04-26T23:56:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.146+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff240693b0 con 0x7eff240b5fb0 2024-04-26T23:56:23.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.147+0000 7eff21600640 1 -- 172.21.15.46:0/802812386 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) ==== 93+0+10 (secure 0 0 0) 0x7eff1805a350 con 0x7eff240b5fb0 2024-04-26T23:56:23.150 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 msgr2=0x7eff0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 0x7eff0406edb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7eff10034000 tx=0x7eff1002e000 comp rx=0 tx=0).stop 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 msgr2=0x7eff240b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7eff1800e9b0 tx=0x7eff1800ee70 comp rx=0 tx=0).stop 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 shutdown_connections 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7eff0406c8f0 0x7eff0406edb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24162aa0 0x7eff240b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff241626c0 0x7eff2415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 --2- 172.21.15.46:0/802812386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff240b5fb0 0x7eff240b84a0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.150+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 >> 172.21.15.46:0/802812386 conn(0x7eff2409e2a0 msgr2=0x7eff240d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.151+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 shutdown_connections 2024-04-26T23:56:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.151+0000 7eff2ae00640 1 -- 172.21.15.46:0/802812386 wait complete. 2024-04-26T23:56:23.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-26T23:56:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.384+0000 7fd3b6c00640 1 Processor -- start 2024-04-26T23:56:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.384+0000 7fd3b6c00640 1 -- start start 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b6c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b014cf10 0x7fd3b006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b6c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b0071ef0 0x7fd3b0072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b6c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b006eff0 con 0x7fd3b0164190 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b6c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b006f2d0 con 0x7fd3b014cf10 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b6c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b006f5b0 con 0x7fd3b0071ef0 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b014cf10 0x7fd3b006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3b014cf10 0x7fd3b006e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37644/0 (socket says 172.21.15.46:37644) 2024-04-26T23:56:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.386+0000 7fd3b5800640 1 -- 172.21.15.46:0/1228125690 learned_addr learned my addr 172.21.15.46:0/1228125690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.387+0000 7fd3b5800640 1 -- 172.21.15.46:0/1228125690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b0071ef0 msgr2=0x7fd3b0072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.387+0000 7fd3b5800640 1 --2- 172.21.15.46:0/1228125690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b0071ef0 0x7fd3b0072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.387+0000 7fd3b5800640 1 -- 172.21.15.46:0/1228125690 >> v1:172.21.15.142:6789/0 conn(0x7fd3b0164190 legacy=0x7fd3b0164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.387+0000 7fd3b5800640 1 -- 172.21.15.46:0/1228125690 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b006f890 con 0x7fd3b014cf10 2024-04-26T23:56:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.388+0000 7fd3b5800640 1 --2- 172.21.15.46:0/1228125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b014cf10 0x7fd3b006e960 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009870 tx=0x7fd3a002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fb18a3763394b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.388+0000 7fd3afe00640 1 -- 172.21.15.46:0/1228125690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0037070 con 0x7fd3b014cf10 2024-04-26T23:56:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.388+0000 7fd3afe00640 1 -- 172.21.15.46:0/1228125690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a0032440 con 0x7fd3b014cf10 2024-04-26T23:56:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.388+0000 7fd3afe00640 1 -- 172.21.15.46:0/1228125690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0032810 con 0x7fd3b014cf10 2024-04-26T23:56:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 -- 172.21.15.46:0/1228125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b014cf10 msgr2=0x7fd3b006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/1228125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b014cf10 0x7fd3b006e960 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009870 tx=0x7fd3a002a200 comp rx=0 tx=0).stop 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 -- 172.21.15.46:0/1228125690 shutdown_connections 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/1228125690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b0071ef0 0x7fd3b0072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/1228125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b014cf10 0x7fd3b006e960 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 -- 172.21.15.46:0/1228125690 >> 172.21.15.46:0/1228125690 conn(0x7fd3b00b2820 msgr2=0x7fd3b00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 -- 172.21.15.46:0/1228125690 shutdown_connections 2024-04-26T23:56:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.389+0000 7fd3b6c00640 1 -- 172.21.15.46:0/1228125690 wait complete. 2024-04-26T23:56:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.391+0000 7fd3b6c00640 1 Processor -- start 2024-04-26T23:56:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.391+0000 7fd3b6c00640 1 -- start start 2024-04-26T23:56:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b014cf10 0x7fd3b0162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b0164190 0x7fd3b00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b00b4f30 con 0x7fd3b0071ef0 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0163bd0 con 0x7fd3b0164190 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b6c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b00b50b0 con 0x7fd3b014cf10 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37646/0 (socket says 172.21.15.46:37646) 2024-04-26T23:56:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 -- 172.21.15.46:0/3430546762 learned_addr learned my addr 172.21.15.46:0/3430546762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 -- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b014cf10 msgr2=0x7fd3b0162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b014cf10 0x7fd3b0162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 -- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b0164190 msgr2=0x7fd3b00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b0164190 0x7fd3b00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.393+0000 7fd3b5800640 1 -- 172.21.15.46:0/3430546762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b0153c00 con 0x7fd3b0071ef0 2024-04-26T23:56:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3b5800640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009870 tx=0x7fd3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0041070 con 0x7fd3b0071ef0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a003c870 con 0x7fd3b0071ef0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b0153ee0 con 0x7fd3b0071ef0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0037040 con 0x7fd3b0071ef0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.394+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01544a0 con 0x7fd3b0071ef0 2024-04-26T23:56:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.396+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b006ead0 con 0x7fd3b0071ef0 2024-04-26T23:56:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.397+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3a0044070 con 0x7fd3b0071ef0 2024-04-26T23:56:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.398+0000 7fd3ad600640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 0x7fd39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.398+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7fd3a00ac540 con 0x7fd3b0071ef0 2024-04-26T23:56:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.399+0000 7fd3b4e00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 0x7fd39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.400+0000 7fd3b4e00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 0x7fd39406edb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd3b014bb90 tx=0x7fd3a4006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.402+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3b006ead0 con 0x7fd3b0071ef0 2024-04-26T23:56:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:23.678+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) -- 0x7fd3b00693b0 con 0x7fd3b0071ef0 2024-04-26T23:56:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.256+0000 7fd3ad600640 1 -- 172.21.15.46:0/3430546762 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v879) ==== 150+0+0 (secure 0 0 0) 0x7fd3a007aae0 con 0x7fd3b0071ef0 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 msgr2=0x7fd39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 0x7fd39406edb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd3b014bb90 tx=0x7fd3a4006c90 comp rx=0 tx=0).stop 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 msgr2=0x7fd3b00cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009870 tx=0x7fd3a00046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 shutdown_connections 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd39406c8f0 0x7fd39406edb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b0164190 0x7fd3b00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd3b014cf10 0x7fd3b0162800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 --2- 172.21.15.46:0/3430546762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3b0071ef0 0x7fd3b00cdc10 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 >> 172.21.15.46:0/3430546762 conn(0x7fd3b00b2820 msgr2=0x7fd3b01668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.260+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 shutdown_connections 2024-04-26T23:56:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.261+0000 7fd3b6c00640 1 -- 172.21.15.46:0/3430546762 wait complete. 2024-04-26T23:56:24.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:56:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.496+0000 7f67b7200640 1 Processor -- start 2024-04-26T23:56:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.496+0000 7f67b7200640 1 -- start start 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.497+0000 7f67b7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b8160130 0x7f67b8160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.497+0000 7f67b7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 0x7f67b8072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b80cbe60 con 0x7f67b815cc40 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f67b80cca70 con 0x7f67b8160130 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67b80cc140 con 0x7f67b80cdc60 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b8160130 0x7f67b8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b8160130 0x7f67b8160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37650/0 (socket says 172.21.15.46:37650) 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 -- 172.21.15.46:0/1905498602 learned_addr learned my addr 172.21.15.46:0/1905498602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b6800640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 0x7f67b8072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 -- 172.21.15.46:0/1905498602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 msgr2=0x7f67b8072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 0x7f67b8072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 -- 172.21.15.46:0/1905498602 >> v1:172.21.15.142:6789/0 conn(0x7f67b815cc40 legacy=0x7f67b80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:24.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b5e00640 1 -- 172.21.15.46:0/1905498602 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b80cc420 con 0x7f67b8160130 2024-04-26T23:56:24.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.498+0000 7f67b6800640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 0x7f67b8072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:24.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.499+0000 7f67b5e00640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160130 0x7f67b8160510 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009870 tx=0x7f67a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64ff96d3fc43c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.500+0000 7f67b5400640 1 -- 172.21.15.46:0/1905498602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8037070 con 0x7f67b8160130 2024-04-26T23:56:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.500+0000 7f67b5400640 1 -- 172.21.15.46:0/1905498602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a8032440 con 0x7f67b8160130 2024-04-26T23:56:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.500+0000 7f67b5400640 1 -- 172.21.15.46:0/1905498602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8032810 con 0x7f67b8160130 2024-04-26T23:56:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.500+0000 7f67b7200640 1 -- 172.21.15.46:0/1905498602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160130 msgr2=0x7f67b8160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.500+0000 7f67b7200640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160130 0x7f67b8160510 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009870 tx=0x7f67a802a180 comp rx=0 tx=0).stop 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 -- 172.21.15.46:0/1905498602 shutdown_connections 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b80cdc60 0x7f67b8072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 --2- 172.21.15.46:0/1905498602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160130 0x7f67b8160510 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 -- 172.21.15.46:0/1905498602 >> 172.21.15.46:0/1905498602 conn(0x7f67b80b27b0 msgr2=0x7f67b80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 -- 172.21.15.46:0/1905498602 shutdown_connections 2024-04-26T23:56:24.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.501+0000 7f67b7200640 1 -- 172.21.15.46:0/1905498602 wait complete. 2024-04-26T23:56:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.502+0000 7f67b7200640 1 Processor -- start 2024-04-26T23:56:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.502+0000 7f67b7200640 1 -- start start 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b815cc40 0x7f67b81e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b8160130 0x7f67b81e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b815d220 con 0x7f67b80cdc60 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67b815d0a0 con 0x7f67b815cc40 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.503+0000 7f67b7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67b8072760 con 0x7f67b8160130 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37666/0 (socket says 172.21.15.46:37666) 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 -- 172.21.15.46:0/1754833729 learned_addr learned my addr 172.21.15.46:0/1754833729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 -- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b8160130 msgr2=0x7f67b81e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b8160130 0x7f67b81e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 -- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b815cc40 msgr2=0x7f67b81e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b815cc40 0x7f67b81e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 -- 172.21.15.46:0/1754833729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b80c4730 con 0x7f67b80cdc60 2024-04-26T23:56:24.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.504+0000 7f67b5e00640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009990 tx=0x7f67a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:24.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.505+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8037040 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.505+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a8042070 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.505+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8030a40 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.505+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b80c4a10 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.505+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b80c4fd0 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.506+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b80693b0 con 0x7f67b80cdc60 2024-04-26T23:56:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.507+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67a8030be0 con 0x7f67b80cdc60 2024-04-26T23:56:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.508+0000 7f67aea00640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 0x7f679406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.509+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7f67a80ac350 con 0x7f67b80cdc60 2024-04-26T23:56:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.509+0000 7f67ad400640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 0x7f679406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:24.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.510+0000 7f67ad400640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 0x7f679406edb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f67b815bce0 tx=0x7f67a00079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.513+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67a807a980 con 0x7f67b80cdc60 2024-04-26T23:56:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.717+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f67b80cbb10 con 0x7f67b80cdc60 2024-04-26T23:56:24.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.722+0000 7f67aea00640 1 -- 172.21.15.46:0/1754833729 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f67a807a280 con 0x7f67b80cdc60 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 msgr2=0x7f679406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 0x7f679406edb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f67b815bce0 tx=0x7f67a00079b0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 msgr2=0x7f67b815b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009990 tx=0x7f67a80046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 shutdown_connections 2024-04-26T23:56:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f679406c8f0 0x7f679406edb0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67b8160130 0x7f67b81e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b815cc40 0x7f67b81e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 --2- 172.21.15.46:0/1754833729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cdc60 0x7f67b815b040 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 >> 172.21.15.46:0/1754833729 conn(0x7f67b80b27b0 msgr2=0x7f67b80ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.725+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 shutdown_connections 2024-04-26T23:56:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:24.726+0000 7f67b7200640 1 -- 172.21.15.46:0/1754833729 wait complete. 2024-04-26T23:56:24.749 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:56:24.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:24.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:24.977 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:56:24.977 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:24.978 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:25.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:25.036 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:25.036 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:25.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:25.094 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:25.094 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:25.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:25.156 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:25.156 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:25.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:25.212 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:56:25.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.435+0000 7f9409a00640 1 Processor -- start 2024-04-26T23:56:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.435+0000 7f9409a00640 1 -- start start 2024-04-26T23:56:25.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.436+0000 7f9409a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94040b4000 0x7f94040d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.436+0000 7f9409a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040d11f0 con 0x7f94040d2130 2024-04-26T23:56:25.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040d5f20 con 0x7f940415f4e0 2024-04-26T23:56:25.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f94040d0ce0 con 0x7f94040b4000 2024-04-26T23:56:25.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94040b4000 0x7f94040d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9403400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94040d2130 0x7f94040b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49452/0 (socket says 172.21.15.46:49452) 2024-04-26T23:56:25.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9403400640 1 -- 172.21.15.46:0/3162444533 learned_addr learned my addr 172.21.15.46:0/3162444533 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:25.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409000640 1 -- 172.21.15.46:0/3162444533 >> v1:172.21.15.142:6789/0 conn(0x7f940415f4e0 legacy=0x7f94040d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409000640 1 -- 172.21.15.46:0/3162444533 >> v1:172.21.15.46:6789/0 conn(0x7f94040d2130 legacy=0x7f94040b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.437+0000 7f9409000640 1 -- 172.21.15.46:0/3162444533 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040d0f10 con 0x7f94040b4000 2024-04-26T23:56:25.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409000640 1 --2- 172.21.15.46:0/3162444533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040d2df0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0097d0 tx=0x7f93fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d91891dc54deda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9402a00640 1 -- 172.21.15.46:0/3162444533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc037070 con 0x7f94040b4000 2024-04-26T23:56:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9402a00640 1 -- 172.21.15.46:0/3162444533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc032440 con 0x7f94040b4000 2024-04-26T23:56:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9402a00640 1 -- 172.21.15.46:0/3162444533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc032810 con 0x7f94040b4000 2024-04-26T23:56:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409a00640 1 -- 172.21.15.46:0/3162444533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 msgr2=0x7f94040d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409a00640 1 --2- 172.21.15.46:0/3162444533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040d2df0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0097d0 tx=0x7f93fc02a210 comp rx=0 tx=0).stop 2024-04-26T23:56:25.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409a00640 1 -- 172.21.15.46:0/3162444533 shutdown_connections 2024-04-26T23:56:25.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409a00640 1 --2- 172.21.15.46:0/3162444533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040d2df0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.439+0000 7f9409a00640 1 -- 172.21.15.46:0/3162444533 >> 172.21.15.46:0/3162444533 conn(0x7f94040d5010 msgr2=0x7f94040d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.440+0000 7f9409a00640 1 -- 172.21.15.46:0/3162444533 shutdown_connections 2024-04-26T23:56:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.440+0000 7f9409a00640 1 -- 172.21.15.46:0/3162444533 wait complete. 2024-04-26T23:56:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.441+0000 7f9409a00640 1 Processor -- start 2024-04-26T23:56:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.441+0000 7f9409a00640 1 -- start start 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 0x7f94040b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415f4e0 0x7f94040b9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f94040b1d60 con 0x7f94040b4000 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f94040d2990 con 0x7f940415f4e0 2024-04-26T23:56:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.442+0000 7f9409a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f940415f8c0 con 0x7f94040d2130 2024-04-26T23:56:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37676/0 (socket says 172.21.15.46:37676) 2024-04-26T23:56:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 -- 172.21.15.46:0/2178773406 learned_addr learned my addr 172.21.15.46:0/2178773406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:25.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9409000640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415f4e0 0x7f94040b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403400640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 0x7f94040b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 -- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 msgr2=0x7f94040b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 0x7f94040b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 -- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415f4e0 msgr2=0x7f94040b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415f4e0 0x7f94040b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403e00640 1 -- 172.21.15.46:0/2178773406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040ba5c0 con 0x7f94040b4000 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.443+0000 7f9403400640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 0x7f94040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f9403e00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c7f0 tx=0x7f93f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4016020 con 0x7f94040b4000 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f400aa50 con 0x7f94040b4000 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94040ba8a0 con 0x7f94040b4000 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f400ae40 con 0x7f94040b4000 2024-04-26T23:56:25.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.444+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9404153670 con 0x7f94040b4000 2024-04-26T23:56:25.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.445+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94040693b0 con 0x7f94040b4000 2024-04-26T23:56:25.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.446+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93f4019020 con 0x7f94040b4000 2024-04-26T23:56:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.447+0000 7f93fbe00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 0x7f93e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.447+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7f93f408ba20 con 0x7f94040b4000 2024-04-26T23:56:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.448+0000 7f9403400640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 0x7f93e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.449+0000 7f9403400640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 0x7f93e406ede0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f94040b9520 tx=0x7f93f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.452+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93f4020030 con 0x7f94040b4000 2024-04-26T23:56:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.700+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94040d33d0 con 0x7f94040b4000 2024-04-26T23:56:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.702+0000 7f93fbe00640 1 -- 172.21.15.46:0/2178773406 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+3876 (secure 0 0 0) 0x7f93f4059fb0 con 0x7f94040b4000 2024-04-26T23:56:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:25.705 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":151,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1616586294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1616586294},{"type":"v1","addr":"172.21.15.142:6836","nonce":1616586294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":6839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3047164882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3047164882},{"type":"v1","addr":"172.21.15.142:6837","nonce":3047164882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":144},{"gid":11241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/476013643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":476013643},{"type":"v1","addr":"172.21.15.46:6833","nonce":476013643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143},{"gid":11598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/869903440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":869903440},{"type":"v1","addr":"172.21.15.46:6835","nonce":869903440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":151}],"filesystems":[]} 2024-04-26T23:56:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 151 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 msgr2=0x7f93e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 0x7f93e406ede0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f94040b9520 tx=0x7f93f0009040 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 msgr2=0x7f94040b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c7f0 tx=0x7f93f400ccb0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 shutdown_connections 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f93e406c920 0x7f93e406ede0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415f4e0 0x7f94040b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94040d2130 0x7f94040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 --2- 172.21.15.46:0/2178773406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94040b4000 0x7f94040b5dd0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 >> 172.21.15.46:0/2178773406 conn(0x7f94040d5010 msgr2=0x7f94040d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 shutdown_connections 2024-04-26T23:56:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.705+0000 7f9409a00640 1 -- 172.21.15.46:0/2178773406 wait complete. 2024-04-26T23:56:25.726 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:56:25.726 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:56:25.727 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:56:25.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:25.727 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:56:25.727 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:56:25.727 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:56:25.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:25.727 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:56:25.728 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:56:25.728 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:56:25.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:25.728 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:56:25.728 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:56:25.728 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:56:25.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:25.749 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:56:25.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:56:25.750 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:56:25.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:56:25.751 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:56:25.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:56:25.752 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:56:25.753 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:56:25.798 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:56:25.798 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:25.799 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:56:25.799 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:25.804 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:56:25.804 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:25.806 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:56:25.806 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:25.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:56:25.947 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:56:25.947 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:56:25.951 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:56:25.955 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:56:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.983+0000 7f0b3ca00640 1 Processor -- start 2024-04-26T23:56:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.983+0000 7f0b3ca00640 1 -- start start 2024-04-26T23:56:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.984+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.984+0000 7f0b3ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380b4000 0x7f0b380b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.984+0000 7f0b3ca00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b380d11f0 con 0x7f0b3815f4e0 2024-04-26T23:56:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.984+0000 7f0b3ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0b380d0ce0 con 0x7f0b380b4000 2024-04-26T23:56:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.984+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b380d2990 con 0x7f0b380d2130 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380b4000 0x7f0b380b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b380b4000 0x7f0b380b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37716/0 (socket says 172.21.15.46:37716) 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 -- 172.21.15.46:0/2581703505 learned_addr learned my addr 172.21.15.46:0/2581703505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b35e00640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 -- 172.21.15.46:0/2581703505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 msgr2=0x7f0b380d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 -- 172.21.15.46:0/2581703505 >> v1:172.21.15.142:6789/0 conn(0x7f0b3815f4e0 legacy=0x7f0b380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.985+0000 7f0b37200640 1 -- 172.21.15.46:0/2581703505 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b380d5f20 con 0x7f0b380b4000 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.986+0000 7f0b37200640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380b4000 0x7f0b380b2280 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009e50 tx=0x7f0b3002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616381d792367aef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.986+0000 7f0b35400640 1 -- 172.21.15.46:0/2581703505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30036070 con 0x7f0b380b4000 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.986+0000 7f0b35400640 1 -- 172.21.15.46:0/2581703505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b30031600 con 0x7f0b380b4000 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.986+0000 7f0b35400640 1 -- 172.21.15.46:0/2581703505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b300319d0 con 0x7f0b380b4000 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 -- 172.21.15.46:0/2581703505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380b4000 msgr2=0x7f0b380b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380b4000 0x7f0b380b2280 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009e50 tx=0x7f0b3002a710 comp rx=0 tx=0).stop 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 -- 172.21.15.46:0/2581703505 shutdown_connections 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b380b4000 0x7f0b380b2280 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/2581703505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 -- 172.21.15.46:0/2581703505 >> 172.21.15.46:0/2581703505 conn(0x7f0b380d5010 msgr2=0x7f0b380d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 -- 172.21.15.46:0/2581703505 shutdown_connections 2024-04-26T23:56:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.987+0000 7f0b3ca00640 1 -- 172.21.15.46:0/2581703505 wait complete. 2024-04-26T23:56:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.989+0000 7f0b3ca00640 1 Processor -- start 2024-04-26T23:56:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.989+0000 7f0b3ca00640 1 -- start start 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 0x7f0b381598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380ba360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380b1d60 con 0x7f0b3815f4e0 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380d5c40 con 0x7f0b380b4000 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3815f8c0 con 0x7f0b380d2130 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37724/0 (socket says 172.21.15.46:37724) 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 -- 172.21.15.46:0/850216776 learned_addr learned my addr 172.21.15.46:0/850216776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b36800640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 0x7f0b381598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b35e00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 -- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 msgr2=0x7f0b380ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 -- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 msgr2=0x7f0b381598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.990+0000 7f0b37200640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 0x7f0b381598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b37200640 1 -- 172.21.15.46:0/850216776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38160f90 con 0x7f0b3815f4e0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b35e00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b36800640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 0x7f0b381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b37200640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f0b30034040 tx=0x7f0b30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30036050 con 0x7f0b3815f4e0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b30042070 con 0x7f0b3815f4e0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b38161230 con 0x7f0b3815f4e0 2024-04-26T23:56:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009510 con 0x7f0b3815f4e0 2024-04-26T23:56:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.991+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b38161ad0 con 0x7f0b3815f4e0 2024-04-26T23:56:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.992+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38161400 con 0x7f0b3815f4e0 2024-04-26T23:56:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.993+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b30007bf0 con 0x7f0b3815f4e0 2024-04-26T23:56:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.995+0000 7f0b2ea00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 0x7f0b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.995+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5335+0+0 (secure 0 0 0) 0x7f0b300ac7f0 con 0x7f0b3815f4e0 2024-04-26T23:56:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.995+0000 7f0b36800640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 0x7f0b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.996+0000 7f0b36800640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 0x7f0b1806edb0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000f60 tx=0x7f0b28009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:25.999+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b30045360 con 0x7f0b3815f4e0 2024-04-26T23:56:26.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.214+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0b380d46d0 con 0x7f0b3815f4e0 2024-04-26T23:56:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.288+0000 7f0b2ea00640 1 -- 172.21.15.46:0/850216776 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v193) ==== 97+0+0 (secure 0 0 0) 0x7f0b3007ad80 con 0x7f0b3815f4e0 2024-04-26T23:56:26.291 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:56:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 msgr2=0x7f0b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 0x7f0b1806edb0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000f60 tx=0x7f0b28009040 comp rx=0 tx=0).stop 2024-04-26T23:56:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 msgr2=0x7f0b380b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f0b30034040 tx=0x7f0b30004a40 comp rx=0 tx=0).stop 2024-04-26T23:56:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 shutdown_connections 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0b1806c8f0 0x7f0b1806edb0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3815f4e0 0x7f0b380b5640 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b380d2130 0x7f0b380ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 --2- 172.21.15.46:0/850216776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b380b4000 0x7f0b381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.291+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 >> 172.21.15.46:0/850216776 conn(0x7f0b380d5010 msgr2=0x7f0b3806e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.292+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 shutdown_connections 2024-04-26T23:56:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.292+0000 7f0b3ca00640 1 -- 172.21.15.46:0/850216776 wait complete. 2024-04-26T23:56:26.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:56:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.518+0000 7f9312e00640 1 Processor -- start 2024-04-26T23:56:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.518+0000 7f9312e00640 1 -- start start 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.519+0000 7f9312e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f930c15cec0 0x7f930c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.519+0000 7f9312e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c160bf0 con 0x7f930c06f7a0 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.519+0000 7f9312e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c160ed0 con 0x7f930c06c890 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.519+0000 7f9312e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f930c06ccf0 con 0x7f930c15cec0 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.520+0000 7f9312400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f930c06f7a0 0x7f930c0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49468/0 (socket says 172.21.15.46:49468) 2024-04-26T23:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.520+0000 7f9312400640 1 -- 172.21.15.46:0/2600495192 learned_addr learned my addr 172.21.15.46:0/2600495192 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.520+0000 7f9311a00640 1 --2- 172.21.15.46:0/2600495192 >> v2:172.21.15.142:3300/0 conn(0x7f930c15cec0 0x7f930c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.520+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251808530 0 0) 0x7f930c160bf0 con 0x7f930c06f7a0 2024-04-26T23:56:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.520+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f4003520 con 0x7f930c06f7a0 2024-04-26T23:56:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2976521493 0 0) 0x7f92f4003520 con 0x7f930c06f7a0 2024-04-26T23:56:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 >> v2:172.21.15.142:3300/0 conn(0x7f930c15cec0 msgr2=0x7f930c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 --2- 172.21.15.46:0/2600495192 >> v2:172.21.15.142:3300/0 conn(0x7f930c15cec0 0x7f930c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 >> v1:172.21.15.142:6789/0 conn(0x7f930c06c890 legacy=0x7f930c06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1611b0 con 0x7f930c06f7a0 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f9304001670 con 0x7f930c06f7a0 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 >> v1:172.21.15.46:6789/0 conn(0x7f930c06f7a0 legacy=0x7f930c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40039b0 0x7f92f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4004800 0x7f92f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 0x7f92f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400c030 con 0x7f92f40051c0 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400c1e0 con 0x7f92f4004800 2024-04-26T23:56:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92f400c3e0 con 0x7f92f40039b0 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 0x7f92f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9311a00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40039b0 0x7f92f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 -- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40039b0 msgr2=0x7f92f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40039b0 0x7f92f4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 -- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4004800 msgr2=0x7f92f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4004800 0x7f92f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.521+0000 7f9312400640 1 -- 172.21.15.46:0/2600495192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f4003520 con 0x7f92f40051c0 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.522+0000 7f9312400640 1 -- 172.21.15.46:0/2600495192 reap_dead start 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312400640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 0x7f92f400b9c0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f930400c240 tx=0x7f92f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4bb744eaaf108a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304007d10 con 0x7f92f40051c0 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304005ad0 con 0x7f92f40051c0 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f930be00640 1 -- 172.21.15.46:0/2600495192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304007070 con 0x7f92f40051c0 2024-04-26T23:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312e00640 1 -- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 msgr2=0x7f92f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312e00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 0x7f92f400b9c0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f930400c240 tx=0x7f92f4005830 comp rx=0 tx=0).stop 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312e00640 1 -- 172.21.15.46:0/2600495192 shutdown_connections 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312e00640 1 --2- 172.21.15.46:0/2600495192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40051c0 0x7f92f400b9c0 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.523+0000 7f9312e00640 1 -- 172.21.15.46:0/2600495192 >> 172.21.15.46:0/2600495192 conn(0x7f930c0d1d30 msgr2=0x7f930c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.524+0000 7f9312e00640 1 -- 172.21.15.46:0/2600495192 shutdown_connections 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.524+0000 7f9312e00640 1 -- 172.21.15.46:0/2600495192 wait complete. 2024-04-26T23:56:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.525+0000 7f9312e00640 1 Processor -- start 2024-04-26T23:56:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.526+0000 7f9312e00640 1 -- start start 2024-04-26T23:56:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.526+0000 7f9312e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40051c0 0x7f930c1f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9312e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9312e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f930c0b5900 0x7f930c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9312e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f930c15d2a0 con 0x7f930c0c8150 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9312e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f930c161770 con 0x7f930c0b5900 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9312e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f930c0b4bc0 con 0x7f92f40051c0 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37732/0 (socket says 172.21.15.46:37732) 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 -- 172.21.15.46:0/3998059591 learned_addr learned my addr 172.21.15.46:0/3998059591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311a00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40051c0 0x7f930c1f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 -- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40051c0 msgr2=0x7f930c1f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40051c0 0x7f930c1f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 -- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f930c0b5900 msgr2=0x7f930c0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f930c0b5900 0x7f930c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.527+0000 7f9311000640 1 -- 172.21.15.46:0/3998059591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1585e0 con 0x7f930c0c8150 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9311000640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f930000c800 tx=0x7f930000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300003b70 con 0x7f930c0c8150 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9300003d30 con 0x7f930c0c8150 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1f8930 con 0x7f930c0c8150 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930000aa90 con 0x7f930c0c8150 2024-04-26T23:56:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.528+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1f8ef0 con 0x7f930c0c8150 2024-04-26T23:56:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.529+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c0b3670 con 0x7f930c0c8150 2024-04-26T23:56:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.530+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f930000ac30 con 0x7f930c0c8150 2024-04-26T23:56:26.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.532+0000 7f9309600640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 0x7f92f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:26.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.532+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 5249+0+0 (secure 0 0 0) 0x7f930008ba00 con 0x7f930c0c8150 2024-04-26T23:56:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.532+0000 7f9311a00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 0x7f92f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:26.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.533+0000 7f9311a00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 0x7f92f006edb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f930c15c0e0 tx=0x7f92fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:26.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.536+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f930c0b3670 con 0x7f930c0c8150 2024-04-26T23:56:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.816+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f930c069ac0 con 0x7f930c0c8150 2024-04-26T23:56:26.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.817+0000 7f9309600640 1 -- 172.21.15.46:0/3998059591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7f930005ee20 con 0x7f930c0c8150 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:56:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:56:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:56:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:56:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:56:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:56:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:56:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:56:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:56:26.832 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 msgr2=0x7f92f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 0x7f92f006edb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f930c15c0e0 tx=0x7f92fc008040 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 msgr2=0x7f930c1513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f930000c800 tx=0x7f930000ccc0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 shutdown_connections 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f92f006c8f0 0x7f92f006edb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f930c0b5900 0x7f930c0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f930c0c8150 0x7f930c1513e0 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 --2- 172.21.15.46:0/3998059591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92f40051c0 0x7f930c1f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 >> 172.21.15.46:0/3998059591 conn(0x7f930c0d1d30 msgr2=0x7f930c15ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:26.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 shutdown_connections 2024-04-26T23:56:26.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:26.820+0000 7f9312e00640 1 -- 172.21.15.46:0/3998059591 wait complete. 2024-04-26T23:56:26.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.053+0000 7f7d8ee00640 1 Processor -- start 2024-04-26T23:56:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.053+0000 7f7d8ee00640 1 -- start start 2024-04-26T23:56:27.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.054+0000 7f7d8ee00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d880720e0 0x7f7d880cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.054+0000 7f7d8ee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d88160910 0x7f7d88160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.054+0000 7f7d8ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d88073ad0 con 0x7f7d881466b0 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.054+0000 7f7d8ee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7d881612a0 con 0x7f7d88160910 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.054+0000 7f7d8ee00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7d88146a90 con 0x7f7d880720e0 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.055+0000 7f7d8da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7d881466b0 0x7f7d8806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49476/0 (socket says 172.21.15.46:49476) 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.055+0000 7f7d8da00640 1 -- 172.21.15.46:0/2136038270 learned_addr learned my addr 172.21.15.46:0/2136038270 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.055+0000 7f7d8e400640 1 --2- 172.21.15.46:0/2136038270 >> v2:172.21.15.46:3300/0 conn(0x7f7d88160910 0x7f7d88160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.055+0000 7f7d8d000640 1 --2- 172.21.15.46:0/2136038270 >> v2:172.21.15.142:3300/0 conn(0x7f7d880720e0 0x7f7d880cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d87e00640 1 -- 172.21.15.46:0/2136038270 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647623248 0 0) 0x7f7d88073ad0 con 0x7f7d881466b0 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d87e00640 1 -- 172.21.15.46:0/2136038270 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d70003520 con 0x7f7d881466b0 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d8e400640 1 -- 172.21.15.46:0/2136038270 >> v2:172.21.15.142:3300/0 conn(0x7f7d880720e0 msgr2=0x7f7d880cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d8e400640 1 --2- 172.21.15.46:0/2136038270 >> v2:172.21.15.142:3300/0 conn(0x7f7d880720e0 0x7f7d880cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d8e400640 1 -- 172.21.15.46:0/2136038270 >> v1:172.21.15.46:6789/0 conn(0x7f7d881466b0 legacy=0x7f7d8806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.056+0000 7f7d8e400640 1 -- 172.21.15.46:0/2136038270 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d88073d70 con 0x7f7d88160910 2024-04-26T23:56:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.057+0000 7f7d8e400640 1 --2- 172.21.15.46:0/2136038270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d88160910 0x7f7d88160d60 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ade3638da05a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.057+0000 7f7d87e00640 1 -- 172.21.15.46:0/2136038270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80007740 con 0x7f7d88160910 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.057+0000 7f7d87e00640 1 -- 172.21.15.46:0/2136038270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d800078e0 con 0x7f7d88160910 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.057+0000 7f7d87e00640 1 -- 172.21.15.46:0/2136038270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80007cb0 con 0x7f7d88160910 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 -- 172.21.15.46:0/2136038270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d88160910 msgr2=0x7f7d88160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/2136038270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d88160910 0x7f7d88160d60 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 -- 172.21.15.46:0/2136038270 shutdown_connections 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/2136038270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d88160910 0x7f7d88160d60 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/2136038270 >> v2:172.21.15.142:3300/0 conn(0x7f7d880720e0 0x7f7d880cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 -- 172.21.15.46:0/2136038270 >> 172.21.15.46:0/2136038270 conn(0x7f7d880b26d0 msgr2=0x7f7d880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 -- 172.21.15.46:0/2136038270 shutdown_connections 2024-04-26T23:56:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.058+0000 7f7d8ee00640 1 -- 172.21.15.46:0/2136038270 wait complete. 2024-04-26T23:56:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.060+0000 7f7d8ee00640 1 Processor -- start 2024-04-26T23:56:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.060+0000 7f7d8ee00640 1 -- start start 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880720e0 0x7f7d880b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88160910 0x7f7d880b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d880cb470 con 0x7f7d881466b0 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d880cb190 con 0x7f7d88160910 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7d880cdda0 con 0x7f7d880720e0 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37736/0 (socket says 172.21.15.46:37736) 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.061+0000 7f7d8d000640 1 -- 172.21.15.46:0/3256159685 learned_addr learned my addr 172.21.15.46:0/3256159685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8da00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880720e0 0x7f7d880b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 -- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880720e0 msgr2=0x7f7d880b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880720e0 0x7f7d880b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 -- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88160910 msgr2=0x7f7d880b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8e400640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88160910 0x7f7d880b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88160910 0x7f7d880b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 -- 172.21.15.46:0/3256159685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d880aaf70 con 0x7f7d881466b0 2024-04-26T23:56:27.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8d000640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e870 tx=0x7f7d7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74017070 con 0x7f7d881466b0 2024-04-26T23:56:27.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7400c940 con 0x7f7d881466b0 2024-04-26T23:56:27.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400cd30 con 0x7f7d881466b0 2024-04-26T23:56:27.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.062+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d880ab530 con 0x7f7d881466b0 2024-04-26T23:56:27.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.063+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d880abaf0 con 0x7f7d881466b0 2024-04-26T23:56:27.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.064+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d880693b0 con 0x7f7d881466b0 2024-04-26T23:56:27.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.065+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d7401a070 con 0x7f7d881466b0 2024-04-26T23:56:27.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.066+0000 7f7d85600640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 0x7f7d6806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.066+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 5249+0+0 (secure 0 0 0) 0x7f7d7408bc00 con 0x7f7d881466b0 2024-04-26T23:56:27.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.067+0000 7f7d8da00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 0x7f7d6806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.068+0000 7f7d8da00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 0x7f7d6806ee00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.071+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d7405a1d0 con 0x7f7d881466b0 2024-04-26T23:56:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.273+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d880c7180 con 0x7f7d881466b0 2024-04-26T23:56:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.276+0000 7f7d85600640 1 -- 172.21.15.46:0/3256159685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) ==== 74+0+13138 (secure 0 0 0) 0x7f7d74059ad0 con 0x7f7d881466b0 2024-04-26T23:56:27.279 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:27.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":195,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:26.303360+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 msgr2=0x7f7d6806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 0x7f7d6806ee00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 msgr2=0x7f7d880aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e870 tx=0x7f7d7400ed30 comp rx=0 tx=0).stop 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 shutdown_connections 2024-04-26T23:56:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6806c940 0x7f7d6806ee00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88160910 0x7f7d880b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881466b0 0x7f7d880aa4d0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 --2- 172.21.15.46:0/3256159685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880720e0 0x7f7d880b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.278+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 >> 172.21.15.46:0/3256159685 conn(0x7f7d880b26d0 msgr2=0x7f7d880c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.279+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 shutdown_connections 2024-04-26T23:56:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.279+0000 7f7d8ee00640 1 -- 172.21.15.46:0/3256159685 wait complete. 2024-04-26T23:56:27.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:27.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.508+0000 7f0692800640 1 Processor -- start 2024-04-26T23:56:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.508+0000 7f0692800640 1 -- start start 2024-04-26T23:56:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.509+0000 7f0692800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f068c162aa0 0x7f068c0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.509+0000 7f0692800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f068c0d5b90 0x7f068c0d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f0692800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f068c0b46f0 con 0x7f068c0b54b0 2024-04-26T23:56:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f0692800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f068c0b44a0 con 0x7f068c0d5b90 2024-04-26T23:56:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f0692800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f068c0b5020 con 0x7f068c162aa0 2024-04-26T23:56:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f068aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f068c0d5b90 0x7f068c0d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f068aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f068c0d5b90 0x7f068c0d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37744/0 (socket says 172.21.15.46:37744) 2024-04-26T23:56:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f068aa00640 1 -- 172.21.15.46:0/4028727285 learned_addr learned my addr 172.21.15.46:0/4028727285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.510+0000 7f068b400640 1 --2- 172.21.15.46:0/4028727285 >> v2:172.21.15.142:3300/0 conn(0x7f068c162aa0 0x7f068c0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.511+0000 7f068aa00640 1 -- 172.21.15.46:0/4028727285 >> v2:172.21.15.142:3300/0 conn(0x7f068c162aa0 msgr2=0x7f068c0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.511+0000 7f068aa00640 1 --2- 172.21.15.46:0/4028727285 >> v2:172.21.15.142:3300/0 conn(0x7f068c162aa0 0x7f068c0d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.511+0000 7f068aa00640 1 -- 172.21.15.46:0/4028727285 >> v1:172.21.15.142:6789/0 conn(0x7f068c0b54b0 legacy=0x7f068c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.511+0000 7f068aa00640 1 -- 172.21.15.46:0/4028727285 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c0b49d0 con 0x7f068c0d5b90 2024-04-26T23:56:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.512+0000 7f068aa00640 1 --2- 172.21.15.46:0/4028727285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0d3b90 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f068c0b7bc0 tx=0x7f067402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbbb48c19e5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.512+0000 7f068a000640 1 -- 172.21.15.46:0/4028727285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674037070 con 0x7f068c0d5b90 2024-04-26T23:56:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.512+0000 7f068a000640 1 -- 172.21.15.46:0/4028727285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0674032440 con 0x7f068c0d5b90 2024-04-26T23:56:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.512+0000 7f068a000640 1 -- 172.21.15.46:0/4028727285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674032810 con 0x7f068c0d5b90 2024-04-26T23:56:27.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 -- 172.21.15.46:0/4028727285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 msgr2=0x7f068c0d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 --2- 172.21.15.46:0/4028727285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0d3b90 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f068c0b7bc0 tx=0x7f067402a150 comp rx=0 tx=0).stop 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 -- 172.21.15.46:0/4028727285 shutdown_connections 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 --2- 172.21.15.46:0/4028727285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0d3b90 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 --2- 172.21.15.46:0/4028727285 >> v2:172.21.15.142:3300/0 conn(0x7f068c162aa0 0x7f068c0d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 -- 172.21.15.46:0/4028727285 >> 172.21.15.46:0/4028727285 conn(0x7f068c0d5070 msgr2=0x7f068c0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 -- 172.21.15.46:0/4028727285 shutdown_connections 2024-04-26T23:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.513+0000 7f0692800640 1 -- 172.21.15.46:0/4028727285 wait complete. 2024-04-26T23:56:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.515+0000 7f0692800640 1 Processor -- start 2024-04-26T23:56:27.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.515+0000 7f0692800640 1 -- start start 2024-04-26T23:56:27.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c0b54b0 0x7f068c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 0x7f068c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f068c0d30f0 con 0x7f068c0d5b90 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f068c162e80 con 0x7f068c0b54b0 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f0692800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f068c0d0c90 con 0x7f068c162aa0 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f068aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f068aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37758/0 (socket says 172.21.15.46:37758) 2024-04-26T23:56:27.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.516+0000 7f068aa00640 1 -- 172.21.15.46:0/1898948110 learned_addr learned my addr 172.21.15.46:0/1898948110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:27.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068be00640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 0x7f068c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068b400640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c0b54b0 0x7f068c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 -- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 msgr2=0x7f068c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 0x7f068c159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 -- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c0b54b0 msgr2=0x7f068c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c0b54b0 0x7f068c1551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 -- 172.21.15.46:0/1898948110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c0c9ac0 con 0x7f068c0d5b90 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068be00640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 0x7f068c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f068aa00640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f067402a190 tx=0x7f06740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674041020 con 0x7f068c0d5b90 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0674030760 con 0x7f068c0d5b90 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674037040 con 0x7f068c0d5b90 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.517+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c0c9da0 con 0x7f068c0d5b90 2024-04-26T23:56:27.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.518+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c0ca360 con 0x7f068c0d5b90 2024-04-26T23:56:27.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.519+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068c069530 con 0x7f068c0d5b90 2024-04-26T23:56:27.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.520+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0674044020 con 0x7f068c0d5b90 2024-04-26T23:56:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.521+0000 7f0683400640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 0x7f066806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.521+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 5249+0+0 (secure 0 0 0) 0x7f06740ac370 con 0x7f068c0d5b90 2024-04-26T23:56:27.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.522+0000 7f068b400640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 0x7f066806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.523+0000 7f068b400640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 0x7f066806ede0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f067c009e90 tx=0x7f067c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.526+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f067404b030 con 0x7f068c0d5b90 2024-04-26T23:56:27.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.727+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f068c0d14a0 con 0x7f068c0d5b90 2024-04-26T23:56:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.728+0000 7f0683400640 1 -- 172.21.15.46:0/1898948110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) ==== 93+0+10 (secure 0 0 0) 0x7f067407a830 con 0x7f068c0d5b90 2024-04-26T23:56:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 msgr2=0x7f066806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 0x7f066806ede0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f067c009e90 tx=0x7f067c00b040 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 msgr2=0x7f068c0b8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f067402a190 tx=0x7f06740046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 shutdown_connections 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f066806c920 0x7f066806ede0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f068c162aa0 0x7f068c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f068c0d5b90 0x7f068c0b8560 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 --2- 172.21.15.46:0/1898948110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c0b54b0 0x7f068c1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 >> 172.21.15.46:0/1898948110 conn(0x7f068c0d5070 msgr2=0x7f068c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 shutdown_connections 2024-04-26T23:56:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.731+0000 7f0692800640 1 -- 172.21.15.46:0/1898948110 wait complete. 2024-04-26T23:56:27.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:56:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.964+0000 7f14aae00640 1 Processor -- start 2024-04-26T23:56:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.964+0000 7f14aae00640 1 -- start start 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.965+0000 7f14aae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 0x7f14a40d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.965+0000 7f14aae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14a40d56c0 0x7f14a40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.965+0000 7f14aae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40d2f60 con 0x7f14a415f500 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.965+0000 7f14aae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f14a415f8e0 con 0x7f14a40d56c0 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.965+0000 7f14aae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f14a40d5b30 con 0x7f14a40b3fb0 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14a40d56c0 0x7f14a40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f14a415f500 0x7f14a40d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49484/0 (socket says 172.21.15.46:49484) 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a3e00640 1 -- 172.21.15.46:0/1073194828 learned_addr learned my addr 172.21.15.46:0/1073194828 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a3400640 1 --2- 172.21.15.46:0/1073194828 >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 0x7f14a40d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a2a00640 1 -- 172.21.15.46:0/1073194828 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407498306 0 0) 0x7f14a40d2f60 con 0x7f14a415f500 2024-04-26T23:56:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a2a00640 1 -- 172.21.15.46:0/1073194828 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1488003520 con 0x7f14a415f500 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a8e00640 1 -- 172.21.15.46:0/1073194828 >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 msgr2=0x7f14a40d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a8e00640 1 --2- 172.21.15.46:0/1073194828 >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 0x7f14a40d5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a8e00640 1 -- 172.21.15.46:0/1073194828 >> v1:172.21.15.46:6789/0 conn(0x7f14a415f500 legacy=0x7f14a40d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.966+0000 7f14a8e00640 1 -- 172.21.15.46:0/1073194828 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40d3240 con 0x7f14a40d56c0 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.967+0000 7f14a3400640 1 --2- 172.21.15.46:0/1073194828 >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 0x7f14a40d5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.968+0000 7f14a8e00640 1 --2- 172.21.15.46:0/1073194828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40d56c0 0x7f14a40b4e70 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b7d10 tx=0x7f149c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6be03e16efce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.968+0000 7f14a2a00640 1 -- 172.21.15.46:0/1073194828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007740 con 0x7f14a40d56c0 2024-04-26T23:56:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.968+0000 7f14a2a00640 1 -- 172.21.15.46:0/1073194828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c0078e0 con 0x7f14a40d56c0 2024-04-26T23:56:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.968+0000 7f14a2a00640 1 -- 172.21.15.46:0/1073194828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007cb0 con 0x7f14a40d56c0 2024-04-26T23:56:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 -- 172.21.15.46:0/1073194828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40d56c0 msgr2=0x7f14a40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 --2- 172.21.15.46:0/1073194828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40d56c0 0x7f14a40b4e70 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b7d10 tx=0x7f149c02a680 comp rx=0 tx=0).stop 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 -- 172.21.15.46:0/1073194828 shutdown_connections 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 --2- 172.21.15.46:0/1073194828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40d56c0 0x7f14a40b4e70 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 --2- 172.21.15.46:0/1073194828 >> v2:172.21.15.142:3300/0 conn(0x7f14a40b3fb0 0x7f14a40d5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 -- 172.21.15.46:0/1073194828 >> 172.21.15.46:0/1073194828 conn(0x7f14a4074400 msgr2=0x7f14a409e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 -- 172.21.15.46:0/1073194828 shutdown_connections 2024-04-26T23:56:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.969+0000 7f14aae00640 1 -- 172.21.15.46:0/1073194828 wait complete. 2024-04-26T23:56:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.971+0000 7f14aae00640 1 Processor -- start 2024-04-26T23:56:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.971+0000 7f14aae00640 1 -- start start 2024-04-26T23:56:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a40d56c0 0x7f14a41f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a415f500 0x7f14a4156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40d3ae0 con 0x7f14a40b3fb0 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40d3800 con 0x7f14a40d56c0 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14aae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40d4000 con 0x7f14a415f500 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37778/0 (socket says 172.21.15.46:37778) 2024-04-26T23:56:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 -- 172.21.15.46:0/2599547773 learned_addr learned my addr 172.21.15.46:0/2599547773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3400640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a40d56c0 0x7f14a41f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a8e00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a415f500 0x7f14a4156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 -- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a415f500 msgr2=0x7f14a4156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a415f500 0x7f14a4156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 -- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a40d56c0 msgr2=0x7f14a41f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.972+0000 7f14a3e00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a40d56c0 0x7f14a41f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f14a3e00640 1 -- 172.21.15.46:0/2599547773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a41f8490 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f14a3e00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f149400c390 tx=0x7f149400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494017070 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1494005ce0 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940072f0 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a40cf400 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.973+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41f8730 con 0x7f14a40b3fb0 2024-04-26T23:56:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.975+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4069530 con 0x7f14a40b3fb0 2024-04-26T23:56:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.976+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f149401a070 con 0x7f14a40b3fb0 2024-04-26T23:56:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.976+0000 7f149be00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.977+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 5249+0+0 (secure 0 0 0) 0x7f149408adc0 con 0x7f14a40b3fb0 2024-04-26T23:56:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.977+0000 7f14a3400640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.978+0000 7f14a3400640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 0x7f148406edb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f14a40ca230 tx=0x7f1490008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:27.981+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14940593a0 con 0x7f14a40b3fb0 2024-04-26T23:56:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.189+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f14a40b5dd0 con 0x7f14a40b3fb0 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.190+0000 7f149be00640 1 -- 172.21.15.46:0/2599547773 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v195) ==== 84+0+65 (secure 0 0 0) 0x7f1494058ca0 con 0x7f14a40b3fb0 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:56:28.193 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 msgr2=0x7f148406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 0x7f148406edb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f14a40ca230 tx=0x7f1490008040 comp rx=0 tx=0).stop 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 msgr2=0x7f14a41f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f149400c390 tx=0x7f149400c850 comp rx=0 tx=0).stop 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 shutdown_connections 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a415f500 0x7f14a4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a40d56c0 0x7f14a41f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 --2- 172.21.15.46:0/2599547773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14a40b3fb0 0x7f14a41f8f30 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 >> 172.21.15.46:0/2599547773 conn(0x7f14a4074400 msgr2=0x7f14a406d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 shutdown_connections 2024-04-26T23:56:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.193+0000 7f14aae00640 1 -- 172.21.15.46:0/2599547773 wait complete. 2024-04-26T23:56:28.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:56:28.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:56:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.434+0000 7f5854400640 1 Processor -- start 2024-04-26T23:56:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.435+0000 7f5854400640 1 -- start start 2024-04-26T23:56:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.436+0000 7f5854400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f584c1469c0 0x7f584c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.436+0000 7f5854400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f584c0cfa10 0x7f584c0727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.436+0000 7f5854400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c146da0 con 0x7f584c0cf560 2024-04-26T23:56:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.436+0000 7f5854400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f584c0d0ac0 con 0x7f584c1469c0 2024-04-26T23:56:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.436+0000 7f5854400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f584c0cfeb0 con 0x7f584c0cfa10 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f584c1469c0 0x7f584c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f584c1469c0 0x7f584c15d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37786/0 (socket says 172.21.15.46:37786) 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 -- 172.21.15.46:0/4206696145 learned_addr learned my addr 172.21.15.46:0/4206696145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 -- 172.21.15.46:0/4206696145 >> v2:172.21.15.142:3300/0 conn(0x7f584c0cfa10 msgr2=0x7f584c0727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 --2- 172.21.15.46:0/4206696145 >> v2:172.21.15.142:3300/0 conn(0x7f584c0cfa10 0x7f584c0727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 -- 172.21.15.46:0/4206696145 >> v1:172.21.15.142:6789/0 conn(0x7f584c0cf560 legacy=0x7f584c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.437+0000 7f5851a00640 1 -- 172.21.15.46:0/4206696145 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c0d0190 con 0x7f584c1469c0 2024-04-26T23:56:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.438+0000 7f5851a00640 1 --2- 172.21.15.46:0/4206696145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c1469c0 0x7f584c15d6d0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f584c0d0a30 tx=0x7f583c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788f6266b8ceeaaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.439+0000 7f584be00640 1 -- 172.21.15.46:0/4206696145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c037070 con 0x7f584c1469c0 2024-04-26T23:56:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.439+0000 7f584be00640 1 -- 172.21.15.46:0/4206696145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583c032440 con 0x7f584c1469c0 2024-04-26T23:56:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.439+0000 7f584be00640 1 -- 172.21.15.46:0/4206696145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c032810 con 0x7f584c1469c0 2024-04-26T23:56:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.439+0000 7f5854400640 1 -- 172.21.15.46:0/4206696145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c1469c0 msgr2=0x7f584c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.439+0000 7f5854400640 1 --2- 172.21.15.46:0/4206696145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c1469c0 0x7f584c15d6d0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f584c0d0a30 tx=0x7f583c02a230 comp rx=0 tx=0).stop 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 -- 172.21.15.46:0/4206696145 shutdown_connections 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 --2- 172.21.15.46:0/4206696145 >> v2:172.21.15.142:3300/0 conn(0x7f584c0cfa10 0x7f584c0727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 --2- 172.21.15.46:0/4206696145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c1469c0 0x7f584c15d6d0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 -- 172.21.15.46:0/4206696145 >> 172.21.15.46:0/4206696145 conn(0x7f584c06c770 msgr2=0x7f584c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 -- 172.21.15.46:0/4206696145 shutdown_connections 2024-04-26T23:56:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.440+0000 7f5854400640 1 -- 172.21.15.46:0/4206696145 wait complete. 2024-04-26T23:56:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.442+0000 7f5854400640 1 Processor -- start 2024-04-26T23:56:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.442+0000 7f5854400640 1 -- start start 2024-04-26T23:56:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f584c0cf560 0x7f584c1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584c1469c0 0x7f584c145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0b35d0 con 0x7f584c0cfa10 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0d0750 con 0x7f584c0cf560 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.443+0000 7f5854400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f584c0b3750 con 0x7f584c1469c0 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37800/0 (socket says 172.21.15.46:37800) 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 -- 172.21.15.46:0/1567860554 learned_addr learned my addr 172.21.15.46:0/1567860554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 -- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584c1469c0 msgr2=0x7f584c145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584c1469c0 0x7f584c145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851a00640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f584c0cf560 0x7f584c1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 -- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f584c0cf560 msgr2=0x7f584c1f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f584c0cf560 0x7f584c1f7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.444+0000 7f5851000640 1 -- 172.21.15.46:0/1567860554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c0b4210 con 0x7f584c0cfa10 2024-04-26T23:56:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5851000640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f583800c440 tx=0x7f583800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838017070 con 0x7f584c0cfa10 2024-04-26T23:56:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58380035f0 con 0x7f584c0cfa10 2024-04-26T23:56:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380039e0 con 0x7f584c0cfa10 2024-04-26T23:56:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c0b44f0 con 0x7f584c0cfa10 2024-04-26T23:56:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.445+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c0b4ab0 con 0x7f584c0cfa10 2024-04-26T23:56:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.446+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c069530 con 0x7f584c0cfa10 2024-04-26T23:56:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.447+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5838003b80 con 0x7f584c0cfa10 2024-04-26T23:56:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.448+0000 7f5849600640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.448+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 5249+0+0 (secure 0 0 0) 0x7f583808ad50 con 0x7f584c0cfa10 2024-04-26T23:56:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.449+0000 7f5851a00640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:28.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.450+0000 7f5851a00640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 0x7f582c06edb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f583c034000 tx=0x7f583c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.454+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58380592a0 con 0x7f584c0cfa10 2024-04-26T23:56:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:28.668+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f584c0734c0 con 0x7f584c0cfa10 2024-04-26T23:56:29.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.308+0000 7f5849600640 1 -- 172.21.15.46:0/1567860554 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v196) ==== 138+0+0 (secure 0 0 0) 0x7f58380010c0 con 0x7f584c0cfa10 2024-04-26T23:56:29.311 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:56:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.313+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 msgr2=0x7f582c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.313+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 0x7f582c06edb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f583c034000 tx=0x7f583c02e000 comp rx=0 tx=0).stop 2024-04-26T23:56:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.313+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 msgr2=0x7f584c13a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.313+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f583800c440 tx=0x7f583800c900 comp rx=0 tx=0).stop 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 shutdown_connections 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584c1469c0 0x7f584c145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f584c0cfa10 0x7f584c13a160 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 --2- 172.21.15.46:0/1567860554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f584c0cf560 0x7f584c1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.314+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 >> 172.21.15.46:0/1567860554 conn(0x7f584c06c770 msgr2=0x7f584c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.315+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 shutdown_connections 2024-04-26T23:56:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.315+0000 7f5854400640 1 -- 172.21.15.46:0/1567860554 wait complete. 2024-04-26T23:56:29.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:56:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.558+0000 7f4325a00640 1 Processor -- start 2024-04-26T23:56:29.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.558+0000 7f4325a00640 1 -- start start 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43200b5fb0 0x7f43200b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43200d07d0 con 0x7f4320162aa0 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43200d0ab0 con 0x7f43201626c0 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f43200d0d90 con 0x7f43200b5fb0 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4320162aa0 0x7f43200d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49488/0 (socket says 172.21.15.46:49488) 2024-04-26T23:56:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.559+0000 7f4325000640 1 -- 172.21.15.46:0/1135940505 learned_addr learned my addr 172.21.15.46:0/1135940505 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.560+0000 7f431f400640 1 --2- 172.21.15.46:0/1135940505 >> v2:172.21.15.142:3300/0 conn(0x7f43200b5fb0 0x7f43200b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.560+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701659931 0 0) 0x7f43200d07d0 con 0x7f4320162aa0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.560+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4304003520 con 0x7f4320162aa0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42908826 0 0) 0x7f4304003520 con 0x7f4320162aa0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 >> v2:172.21.15.142:3300/0 conn(0x7f43200b5fb0 msgr2=0x7f43200b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 --2- 172.21.15.46:0/1135940505 >> v2:172.21.15.142:3300/0 conn(0x7f43200b5fb0 0x7f43200b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 >> v1:172.21.15.142:6789/0 conn(0x7f43201626c0 legacy=0x7f43200d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43200d2c80 con 0x7f4320162aa0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4318001670 con 0x7f4320162aa0 2024-04-26T23:56:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 >> v1:172.21.15.46:6789/0 conn(0x7f4320162aa0 legacy=0x7f43200d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 0x7f4304003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4304004800 0x7f4304004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040051c0 0x7f430400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43200d0d90 con 0x7f43040039b0 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f430400c060 con 0x7f43040051c0 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f430400c260 con 0x7f4304004800 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431fe00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 0x7f4304003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431f400640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4304004800 0x7f4304004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.561+0000 7f431fe00640 1 -- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4304004800 msgr2=0x7f4304004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f431fe00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4304004800 0x7f4304004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f431fe00640 1 -- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040051c0 msgr2=0x7f430400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f4325000640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040051c0 0x7f430400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f431fe00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040051c0 0x7f430400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f431fe00640 1 -- 172.21.15.46:0/1135940505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4304003520 con 0x7f43040039b0 2024-04-26T23:56:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f4325000640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040051c0 0x7f430400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.562+0000 7f4325000640 1 -- 172.21.15.46:0/1135940505 reap_dead start 2024-04-26T23:56:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.563+0000 7f431fe00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 0x7f4304003dd0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f4310001e50 tx=0x7f4304005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384fb96292e9bb08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.563+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310001a70 con 0x7f43040039b0 2024-04-26T23:56:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.563+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431002ae40 con 0x7f43040039b0 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.563+0000 7f431ea00640 1 -- 172.21.15.46:0/1135940505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310003bb0 con 0x7f43040039b0 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 -- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 msgr2=0x7f4304003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 0x7f4304003dd0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f4310001e50 tx=0x7f4304005830 comp rx=0 tx=0).stop 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 -- 172.21.15.46:0/1135940505 shutdown_connections 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 --2- 172.21.15.46:0/1135940505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43040039b0 0x7f4304003dd0 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 -- 172.21.15.46:0/1135940505 >> 172.21.15.46:0/1135940505 conn(0x7f432009e2a0 msgr2=0x7f43200d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 -- 172.21.15.46:0/1135940505 shutdown_connections 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.564+0000 7f4325a00640 1 -- 172.21.15.46:0/1135940505 wait complete. 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.566+0000 7f4325a00640 1 Processor -- start 2024-04-26T23:56:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.566+0000 7f4325a00640 1 -- start start 2024-04-26T23:56:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040039b0 0x7f43200cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43200cdb90 0x7f43200ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43200b4b50 con 0x7f43201f5a00 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43200b49d0 con 0x7f43040039b0 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f43200d5850 con 0x7f43200cdb90 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37824/0 (socket says 172.21.15.46:37824) 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f4325000640 1 -- 172.21.15.46:0/2068574810 learned_addr learned my addr 172.21.15.46:0/2068574810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f431fe00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040039b0 0x7f43200cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.567+0000 7f431f400640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43200cdb90 0x7f43200ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 -- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43200cdb90 msgr2=0x7f43200ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43200cdb90 0x7f43200ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 -- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040039b0 msgr2=0x7f43200cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040039b0 0x7f43200cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 -- 172.21.15.46:0/2068574810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43200cb860 con 0x7f43201f5a00 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325000640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f432008f2b0 tx=0x7f431800d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180074e0 con 0x7f43201f5a00 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43180047e0 con 0x7f43201f5a00 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318004b90 con 0x7f43201f5a00 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43200cb580 con 0x7f43201f5a00 2024-04-26T23:56:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.568+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43200c8e50 con 0x7f43201f5a00 2024-04-26T23:56:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.569+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43200693b0 con 0x7f43201f5a00 2024-04-26T23:56:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.571+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431801f480 con 0x7f43201f5a00 2024-04-26T23:56:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.572+0000 7f4317e00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.572+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 5619+0+0 (secure 0 0 0) 0x7f431808b6f0 con 0x7f43201f5a00 2024-04-26T23:56:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.573+0000 7f431fe00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.574+0000 7f431fe00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 0x7f42fc06edb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4310032000 tx=0x7f431002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.576+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4318059be0 con 0x7f43201f5a00 2024-04-26T23:56:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:29.779+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f43200d25b0 con 0x7f43201f5a00 2024-04-26T23:56:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.315+0000 7f4317e00640 1 -- 172.21.15.46:0/2068574810 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v197) ==== 170+0+0 (secure 0 0 0) 0x7f43180594e0 con 0x7f43201f5a00 2024-04-26T23:56:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:56:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.320+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 msgr2=0x7f42fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.320+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 0x7f42fc06edb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4310032000 tx=0x7f431002e000 comp rx=0 tx=0).stop 2024-04-26T23:56:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.320+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 msgr2=0x7f43201f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.320+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f432008f2b0 tx=0x7f431800d880 comp rx=0 tx=0).stop 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 shutdown_connections 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43201f5a00 0x7f43201f5e80 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43200cdb90 0x7f43200ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 --2- 172.21.15.46:0/2068574810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43040039b0 0x7f43200cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 >> 172.21.15.46:0/2068574810 conn(0x7f432009e2a0 msgr2=0x7f4320160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 shutdown_connections 2024-04-26T23:56:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.321+0000 7f4325a00640 1 -- 172.21.15.46:0/2068574810 wait complete. 2024-04-26T23:56:30.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:56:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.545+0000 7f3d96c00640 1 Processor -- start 2024-04-26T23:56:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.545+0000 7f3d96c00640 1 -- start start 2024-04-26T23:56:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d96c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d9015cec0 0x7f3d900cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d96c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 0x7f3d900735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d96c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d900ce7d0 con 0x7f3d90073b30 2024-04-26T23:56:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d96c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3d900ceab0 con 0x7f3d9015cec0 2024-04-26T23:56:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d96c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3d900ced90 con 0x7f3d900cdb30 2024-04-26T23:56:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d95800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d9015cec0 0x7f3d900cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d95800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d9015cec0 0x7f3d900cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37828/0 (socket says 172.21.15.46:37828) 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.547+0000 7f3d95800640 1 -- 172.21.15.46:0/4249566422 learned_addr learned my addr 172.21.15.46:0/4249566422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d94e00640 1 --2- 172.21.15.46:0/4249566422 >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 0x7f3d900735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d95800640 1 -- 172.21.15.46:0/4249566422 >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 msgr2=0x7f3d900735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d95800640 1 --2- 172.21.15.46:0/4249566422 >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 0x7f3d900735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d95800640 1 -- 172.21.15.46:0/4249566422 >> v1:172.21.15.142:6789/0 conn(0x7f3d90073b30 legacy=0x7f3d90071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d95800640 1 -- 172.21.15.46:0/4249566422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900cf070 con 0x7f3d9015cec0 2024-04-26T23:56:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.548+0000 7f3d94e00640 1 --2- 172.21.15.46:0/4249566422 >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 0x7f3d900735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.550+0000 7f3d95800640 1 --2- 172.21.15.46:0/4249566422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d900cd5f0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f3d90074300 tx=0x7f3d8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8a958e169990c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.550+0000 7f3d8fe00640 1 -- 172.21.15.46:0/4249566422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80037070 con 0x7f3d9015cec0 2024-04-26T23:56:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.550+0000 7f3d8fe00640 1 -- 172.21.15.46:0/4249566422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d80032440 con 0x7f3d9015cec0 2024-04-26T23:56:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.550+0000 7f3d8fe00640 1 -- 172.21.15.46:0/4249566422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80032810 con 0x7f3d9015cec0 2024-04-26T23:56:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 -- 172.21.15.46:0/4249566422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 msgr2=0x7f3d900cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 --2- 172.21.15.46:0/4249566422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d900cd5f0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f3d90074300 tx=0x7f3d8002a270 comp rx=0 tx=0).stop 2024-04-26T23:56:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 -- 172.21.15.46:0/4249566422 shutdown_connections 2024-04-26T23:56:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 --2- 172.21.15.46:0/4249566422 >> v2:172.21.15.142:3300/0 conn(0x7f3d900cdb30 0x7f3d900735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 --2- 172.21.15.46:0/4249566422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d900cd5f0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 -- 172.21.15.46:0/4249566422 >> 172.21.15.46:0/4249566422 conn(0x7f3d900b27b0 msgr2=0x7f3d900b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 -- 172.21.15.46:0/4249566422 shutdown_connections 2024-04-26T23:56:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.551+0000 7f3d96c00640 1 -- 172.21.15.46:0/4249566422 wait complete. 2024-04-26T23:56:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.553+0000 7f3d96c00640 1 Processor -- start 2024-04-26T23:56:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.553+0000 7f3d96c00640 1 -- start start 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 0x7f3d9014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 0x7f3d90149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90073f70 con 0x7f3d9015cec0 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9015d2a0 con 0x7f3d900cdb30 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d900b4b50 con 0x7f3d90073b30 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37842/0 (socket says 172.21.15.46:37842) 2024-04-26T23:56:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.554+0000 7f3d96200640 1 -- 172.21.15.46:0/2518667594 learned_addr learned my addr 172.21.15.46:0/2518667594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d95800640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 0x7f3d9014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d94e00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 0x7f3d90149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 -- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 msgr2=0x7f3d9014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 0x7f3d9014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 -- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 msgr2=0x7f3d90149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 0x7f3d90149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 -- 172.21.15.46:0/2518667594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900c58b0 con 0x7f3d9015cec0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d95800640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 0x7f3d9014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d94e00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 0x7f3d90149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d96200640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c730 tx=0x7f3d8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.555+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003b70 con 0x7f3d9015cec0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.556+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88003d30 con 0x7f3d9015cec0 2024-04-26T23:56:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.556+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d900c5b90 con 0x7f3d9015cec0 2024-04-26T23:56:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.556+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88005310 con 0x7f3d9015cec0 2024-04-26T23:56:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.556+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9014a360 con 0x7f3d9015cec0 2024-04-26T23:56:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.557+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d900693b0 con 0x7f3d9015cec0 2024-04-26T23:56:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.558+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d880056c0 con 0x7f3d9015cec0 2024-04-26T23:56:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.559+0000 7f3d8d600640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.559+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) ==== 6001+0+0 (secure 0 0 0) 0x7f3d8808bcd0 con 0x7f3d9015cec0 2024-04-26T23:56:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.560+0000 7f3d95800640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.561+0000 7f3d95800640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3d80034000 tx=0x7f3d8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:30.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.564+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d88059fc0 con 0x7f3d9015cec0 2024-04-26T23:56:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:30.809+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3d900cf980 con 0x7f3d9015cec0 2024-04-26T23:56:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.367+0000 7f3d8d600640 1 -- 172.21.15.46:0/2518667594 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v154) ==== 174+0+0 (secure 0 0 0) 0x7f3d880598c0 con 0x7f3d9015cec0 2024-04-26T23:56:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 29 and data pool 30 2024-04-26T23:56:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 msgr2=0x7f3d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3d80034000 tx=0x7f3d8002e000 comp rx=0 tx=0).stop 2024-04-26T23:56:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 msgr2=0x7f3d90149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c730 tx=0x7f3d8800cbf0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 shutdown_connections 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3d7006c8f0 0x7f3d7006edb0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d9015cec0 0x7f3d90149e20 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d900cdb30 0x7f3d90149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 --2- 172.21.15.46:0/2518667594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d90073b30 0x7f3d9014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.371+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 >> 172.21.15.46:0/2518667594 conn(0x7f3d900b27b0 msgr2=0x7f3d90071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.372+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 shutdown_connections 2024-04-26T23:56:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.372+0000 7f3d96c00640 1 -- 172.21.15.46:0/2518667594 wait complete. 2024-04-26T23:56:31.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.573+0000 7f8bd1a00640 1 Processor -- start 2024-04-26T23:56:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.573+0000 7f8bd1a00640 1 -- start start 2024-04-26T23:56:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bd1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bd1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bcc160bd0 0x7f8bcc161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bd1a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc06ee30 con 0x7f8bcc15d540 2024-04-26T23:56:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bd1a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc0742d0 con 0x7f8bcc160bd0 2024-04-26T23:56:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bd1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc161940 con 0x7f8bcc06e510 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bc2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bcc160bd0 0x7f8bcc161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bcb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8bcc15d540 0x7f8bcc15d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49500/0 (socket says 172.21.15.46:49500) 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bc2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bcc160bd0 0x7f8bcc161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37858/0 (socket says 172.21.15.46:37858) 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.575+0000 7f8bcb400640 1 -- 172.21.15.46:0/521775812 learned_addr learned my addr 172.21.15.46:0/521775812 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bcaa00640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bc2000640 1 -- 172.21.15.46:0/521775812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 msgr2=0x7f8bcc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bc2000640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bc2000640 1 -- 172.21.15.46:0/521775812 >> v1:172.21.15.46:6789/0 conn(0x7f8bcc15d540 legacy=0x7f8bcc15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bc2000640 1 -- 172.21.15.46:0/521775812 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc071f80 con 0x7f8bcc160bd0 2024-04-26T23:56:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.576+0000 7f8bca000640 1 -- 172.21.15.46:0/521775812 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341624950 0 0) 0x7f8bcc06ee30 con 0x7f8bcc15d540 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.577+0000 7f8bc2000640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc161030 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097a0 tx=0x7f8bb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1ee6e44517752a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.577+0000 7f8bca000640 1 -- 172.21.15.46:0/521775812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4037070 con 0x7f8bcc160bd0 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.577+0000 7f8bca000640 1 -- 172.21.15.46:0/521775812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb4032440 con 0x7f8bcc160bd0 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.577+0000 7f8bca000640 1 -- 172.21.15.46:0/521775812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4032810 con 0x7f8bcc160bd0 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 -- 172.21.15.46:0/521775812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 msgr2=0x7f8bcc161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc161030 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097a0 tx=0x7f8bb402a180 comp rx=0 tx=0).stop 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 -- 172.21.15.46:0/521775812 shutdown_connections 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc161030 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/521775812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 -- 172.21.15.46:0/521775812 >> 172.21.15.46:0/521775812 conn(0x7f8bcc06c530 msgr2=0x7f8bcc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 -- 172.21.15.46:0/521775812 shutdown_connections 2024-04-26T23:56:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.578+0000 7f8bd1a00640 1 -- 172.21.15.46:0/521775812 wait complete. 2024-04-26T23:56:31.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.580+0000 7f8bd1a00640 1 Processor -- start 2024-04-26T23:56:31.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.580+0000 7f8bd1a00640 1 -- start start 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.581+0000 7f8bd1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc13c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.581+0000 7f8bd1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc15d540 0x7f8bcc13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bd1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bd1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc072820 con 0x7f8bcc160bd0 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bd1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc072540 con 0x7f8bcc15d540 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bd1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc161570 con 0x7f8bcc06e510 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37870/0 (socket says 172.21.15.46:37870) 2024-04-26T23:56:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 -- 172.21.15.46:0/13321176 learned_addr learned my addr 172.21.15.46:0/13321176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcaa00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc13c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 -- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 msgr2=0x7f8bcc13c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc13c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 -- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc15d540 msgr2=0x7f8bcc13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc15d540 0x7f8bcc13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.582+0000 7f8bcb400640 1 -- 172.21.15.46:0/13321176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc13e370 con 0x7f8bcc160bd0 2024-04-26T23:56:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bcb400640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c480 tx=0x7f8bc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:31.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4018070 con 0x7f8bcc160bd0 2024-04-26T23:56:31.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4007500 con 0x7f8bcc160bd0 2024-04-26T23:56:31.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4004a20 con 0x7f8bcc160bd0 2024-04-26T23:56:31.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc0b2f50 con 0x7f8bcc160bd0 2024-04-26T23:56:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.583+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc0b3510 con 0x7f8bcc160bd0 2024-04-26T23:56:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.584+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc069530 con 0x7f8bcc160bd0 2024-04-26T23:56:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.586+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bc401b070 con 0x7f8bcc160bd0 2024-04-26T23:56:31.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.587+0000 7f8bc3400640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:31.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.587+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6101+0+0 (secure 0 0 0) 0x7f8bc408afe0 con 0x7f8bcc160bd0 2024-04-26T23:56:31.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.588+0000 7f8bcaa00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:31.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.589+0000 7f8bcaa00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009a60 tx=0x7f8bbc003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.591+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8bcc13e370 con 0x7f8bcc160bd0 2024-04-26T23:56:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.797+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bcc1461b0 con 0x7f8bcc160bd0 2024-04-26T23:56:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.800+0000 7f8bc3400640 1 -- 172.21.15.46:0/13321176 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) ==== 74+0+16861 (secure 0 0 0) 0x7f8bc4059290 con 0x7f8bcc160bd0 2024-04-26T23:56:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:31.803 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":198,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:30.811742+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.802+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 msgr2=0x7f8ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.802+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009a60 tx=0x7f8bbc003990 comp rx=0 tx=0).stop 2024-04-26T23:56:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.802+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 msgr2=0x7f8bcc142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.802+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c480 tx=0x7f8bc400c940 comp rx=0 tx=0).stop 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 shutdown_connections 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc160bd0 0x7f8bcc142a10 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc15d540 0x7f8bcc13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 --2- 172.21.15.46:0/13321176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bcc06e510 0x7f8bcc13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 >> 172.21.15.46:0/13321176 conn(0x7f8bcc06c530 msgr2=0x7f8bcc0ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 shutdown_connections 2024-04-26T23:56:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:31.803+0000 7f8bd1a00640 1 -- 172.21.15.46:0/13321176 wait complete. 2024-04-26T23:56:31.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.027+0000 7f9802c00640 1 Processor -- start 2024-04-26T23:56:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.028+0000 7f9802c00640 1 -- start start 2024-04-26T23:56:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.029+0000 7f9802c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.029+0000 7f9802c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97fc0b4000 0x7f97fc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.029+0000 7f9802c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97fc0d11f0 con 0x7f97fc15f4e0 2024-04-26T23:56:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.029+0000 7f9802c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f97fc0d0ce0 con 0x7f97fc0b4000 2024-04-26T23:56:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.029+0000 7f9802c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc0d2990 con 0x7f97fc0d2130 2024-04-26T23:56:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.030+0000 7f9800e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.030+0000 7f9800e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:43524/0 (socket says 172.21.15.46:43524) 2024-04-26T23:56:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.030+0000 7f9800e00640 1 -- 172.21.15.46:0/4084393527 learned_addr learned my addr 172.21.15.46:0/4084393527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.030+0000 7f9802200640 1 --2- 172.21.15.46:0/4084393527 >> v2:172.21.15.142:3300/0 conn(0x7f97fc0b4000 0x7f97fc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.031+0000 7f9800e00640 1 -- 172.21.15.46:0/4084393527 >> v2:172.21.15.142:3300/0 conn(0x7f97fc0b4000 msgr2=0x7f97fc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.031+0000 7f9800e00640 1 --2- 172.21.15.46:0/4084393527 >> v2:172.21.15.142:3300/0 conn(0x7f97fc0b4000 0x7f97fc0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.031+0000 7f9800e00640 1 -- 172.21.15.46:0/4084393527 >> v1:172.21.15.142:6789/0 conn(0x7f97fc15f4e0 legacy=0x7f97fc0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.031+0000 7f9800e00640 1 -- 172.21.15.46:0/4084393527 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc0d5f20 con 0x7f97fc0d2130 2024-04-26T23:56:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.032+0000 7f9800e00640 1 --2- 172.21.15.46:0/4084393527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e50 tx=0x7f97e802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce173a7d75b17e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.033+0000 7f97fbe00640 1 -- 172.21.15.46:0/4084393527 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8036070 con 0x7f97fc0d2130 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.033+0000 7f97fbe00640 1 -- 172.21.15.46:0/4084393527 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e8031600 con 0x7f97fc0d2130 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.033+0000 7f97fbe00640 1 -- 172.21.15.46:0/4084393527 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80319d0 con 0x7f97fc0d2130 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.033+0000 7f9802c00640 1 -- 172.21.15.46:0/4084393527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 msgr2=0x7f97fc0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.033+0000 7f9802c00640 1 --2- 172.21.15.46:0/4084393527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e50 tx=0x7f97e802a710 comp rx=0 tx=0).stop 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 -- 172.21.15.46:0/4084393527 shutdown_connections 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 --2- 172.21.15.46:0/4084393527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0d2130 0x7f97fc0d2df0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 --2- 172.21.15.46:0/4084393527 >> v2:172.21.15.142:3300/0 conn(0x7f97fc0b4000 0x7f97fc0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 -- 172.21.15.46:0/4084393527 >> 172.21.15.46:0/4084393527 conn(0x7f97fc0d5010 msgr2=0x7f97fc0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 -- 172.21.15.46:0/4084393527 shutdown_connections 2024-04-26T23:56:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.034+0000 7f9802c00640 1 -- 172.21.15.46:0/4084393527 wait complete. 2024-04-26T23:56:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.035+0000 7f9802c00640 1 Processor -- start 2024-04-26T23:56:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.036+0000 7f9802c00640 1 -- start start 2024-04-26T23:56:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.036+0000 7f9802c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 0x7f97fc0a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.036+0000 7f9802c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 0x7f97fc0ba780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0b1d60 con 0x7f97fc15f4e0 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0d5c40 con 0x7f97fc0d2130 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc15f8c0 con 0x7f97fc0b4000 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48018/0 (socket says 172.21.15.46:48018) 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 -- 172.21.15.46:0/1874493982 learned_addr learned my addr 172.21.15.46:0/1874493982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9801800640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 0x7f97fc0a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9800e00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 0x7f97fc0ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 -- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 msgr2=0x7f97fc0a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 0x7f97fc0a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 -- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 msgr2=0x7f97fc0ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 0x7f97fc0ba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9802200640 1 -- 172.21.15.46:0/1874493982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc0a0e80 con 0x7f97fc15f4e0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9801800640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 0x7f97fc0a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.037+0000 7f9800e00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 0x7f97fc0ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f9802200640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f97f400d830 tx=0x7f97f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f400b6c0 con 0x7f97fc15f4e0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f4019070 con 0x7f97fc15f4e0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc0a2c30 con 0x7f97fc15f4e0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f400bac0 con 0x7f97fc15f4e0 2024-04-26T23:56:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.038+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc0a1400 con 0x7f97fc15f4e0 2024-04-26T23:56:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.039+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc0693b0 con 0x7f97fc15f4e0 2024-04-26T23:56:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.040+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97f401e460 con 0x7f97fc15f4e0 2024-04-26T23:56:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.041+0000 7f97f9600640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 0x7f97dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.042+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6101+0+0 (secure 0 0 0) 0x7f97f408c3a0 con 0x7f97fc15f4e0 2024-04-26T23:56:32.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.042+0000 7f9801800640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 0x7f97dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.043+0000 7f9801800640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 0x7f97dc06edb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0015e0 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.046+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97f405a640 con 0x7f97fc15f4e0 2024-04-26T23:56:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.250+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97fc0d3780 con 0x7f97fc15f4e0 2024-04-26T23:56:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.253+0000 7f97f9600640 1 -- 172.21.15.46:0/1874493982 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) ==== 74+0+16861 (secure 0 0 0) 0x7f97f4059f40 con 0x7f97fc15f4e0 2024-04-26T23:56:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":198,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:30.811742+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 msgr2=0x7f97dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 0x7f97dc06edb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0015e0 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 msgr2=0x7f97fc0a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f97f400d830 tx=0x7f97f400dcf0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 shutdown_connections 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f97dc06c8f0 0x7f97dc06edb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc15f4e0 0x7f97fc0a26f0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97fc0d2130 0x7f97fc0ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 --2- 172.21.15.46:0/1874493982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97fc0b4000 0x7f97fc0a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 >> 172.21.15.46:0/1874493982 conn(0x7f97fc0d5010 msgr2=0x7f97fc0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 shutdown_connections 2024-04-26T23:56:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.256+0000 7f9802c00640 1 -- 172.21.15.46:0/1874493982 wait complete. 2024-04-26T23:56:32.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:56:32.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.486+0000 7f1e4e600640 1 Processor -- start 2024-04-26T23:56:32.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.486+0000 7f1e4e600640 1 -- start start 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e4e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e480d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e4e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4806eb90 con 0x7f1e480b53b0 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e4e600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480b4310 con 0x7f1e4815e220 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e4e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1e480b45f0 con 0x7f1e4815d730 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e46a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1e480b53b0 0x7f1e4815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59584/0 (socket says 172.21.15.46:59584) 2024-04-26T23:56:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.488+0000 7f1e46a00640 1 -- 172.21.15.46:0/2082119772 learned_addr learned my addr 172.21.15.46:0/2082119772 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e47400640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e480d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674420810 0 0) 0x7f1e4806eb90 con 0x7f1e480b53b0 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e2c003520 con 0x7f1e480b53b0 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3662274101 0 0) 0x7f1e2c003520 con 0x7f1e480b53b0 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 msgr2=0x7f1e480d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e480d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 >> v1:172.21.15.142:6789/0 conn(0x7f1e4815e220 legacy=0x7f1e4806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e480b48d0 con 0x7f1e480b53b0 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f1e34001670 con 0x7f1e480b53b0 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.489+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 >> v1:172.21.15.46:6789/0 conn(0x7f1e480b53b0 legacy=0x7f1e4815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c0039e0 0x7f1e2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2c004800 0x7f1e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480b45f0 con 0x7f1e2c0039e0 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c00ba70 con 0x7f1e2c004800 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2c00bc70 con 0x7f1e4815d730 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c0039e0 0x7f1e2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 -- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 msgr2=0x7f1e2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47e00640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e46a00640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2c004800 0x7f1e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 -- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2c004800 msgr2=0x7f1e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2c004800 0x7f1e2c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47400640 1 -- 172.21.15.46:0/2082119772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c003520 con 0x7f1e2c0039e0 2024-04-26T23:56:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.490+0000 7f1e47e00640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e4815d730 0x7f1e2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.491+0000 7f1e47400640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c0039e0 0x7f1e2c003e00 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009a90 tx=0x7f1e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08876c91c3e977e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38036070 con 0x7f1e2c0039e0 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38031400 con 0x7f1e2c0039e0 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e46000640 1 -- 172.21.15.46:0/2082119772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38009690 con 0x7f1e2c0039e0 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e4e600640 1 -- 172.21.15.46:0/2082119772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c0039e0 msgr2=0x7f1e2c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e4e600640 1 --2- 172.21.15.46:0/2082119772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c0039e0 0x7f1e2c003e00 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009a90 tx=0x7f1e2c005830 comp rx=0 tx=0).stop 2024-04-26T23:56:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e47e00640 1 -- 172.21.15.46:0/2082119772 reap_dead start 2024-04-26T23:56:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e4e600640 1 -- 172.21.15.46:0/2082119772 shutdown_connections 2024-04-26T23:56:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.492+0000 7f1e4e600640 1 -- 172.21.15.46:0/2082119772 >> 172.21.15.46:0/2082119772 conn(0x7f1e4806c980 msgr2=0x7f1e480b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.493+0000 7f1e4e600640 1 -- 172.21.15.46:0/2082119772 shutdown_connections 2024-04-26T23:56:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.493+0000 7f1e4e600640 1 -- 172.21.15.46:0/2082119772 wait complete. 2024-04-26T23:56:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.494+0000 7f1e4e600640 1 Processor -- start 2024-04-26T23:56:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.495+0000 7f1e4e600640 1 -- start start 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e2c0039e0 0x7f1e4815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 0x7f1e480af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480b3ec0 con 0x7f1e48152950 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480b4e90 con 0x7f1e480af360 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e4e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1e480b4040 con 0x7f1e2c0039e0 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48034/0 (socket says 172.21.15.46:48034) 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 -- 172.21.15.46:0/3563745847 learned_addr learned my addr 172.21.15.46:0/3563745847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e47400640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e2c0039e0 0x7f1e4815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e47e00640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 0x7f1e480af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 -- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e2c0039e0 msgr2=0x7f1e4815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e2c0039e0 0x7f1e4815a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 -- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 msgr2=0x7f1e480af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 0x7f1e480af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.496+0000 7f1e46a00640 1 -- 172.21.15.46:0/3563745847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4813ff60 con 0x7f1e48152950 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e47e00640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 0x7f1e480af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e46a00640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400d890 tx=0x7f1e3400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34019070 con 0x7f1e48152950 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e340074e0 con 0x7f1e48152950 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34004a00 con 0x7f1e48152950 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e480b00d0 con 0x7f1e48152950 2024-04-26T23:56:32.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.497+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e48150df0 con 0x7f1e48152950 2024-04-26T23:56:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.499+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e480b2a30 con 0x7f1e48152950 2024-04-26T23:56:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.500+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e34004ba0 con 0x7f1e48152950 2024-04-26T23:56:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.500+0000 7f1e3f400640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 0x7f1e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.501+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f1e3408b6c0 con 0x7f1e48152950 2024-04-26T23:56:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.501+0000 7f1e47400640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 0x7f1e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:32.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.502+0000 7f1e47400640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 0x7f1e2806edb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1e38033000 tx=0x7f1e38008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:32.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.505+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e480b2a30 con 0x7f1e48152950 2024-04-26T23:56:32.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:32.754+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1e4806edb0 con 0x7f1e48152950 2024-04-26T23:56:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.390+0000 7f1e3f400640 1 -- 172.21.15.46:0/3563745847 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v157) ==== 118+0+0 (secure 0 0 0) 0x7f1e4806edb0 con 0x7f1e48152950 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 msgr2=0x7f1e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 0x7f1e2806edb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1e38033000 tx=0x7f1e38008030 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 msgr2=0x7f1e48152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400d890 tx=0x7f1e3400dd50 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 shutdown_connections 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1e2806c8f0 0x7f1e2806edb0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e480af360 0x7f1e480af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e48152950 0x7f1e48152dd0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 --2- 172.21.15.46:0/3563745847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e2c0039e0 0x7f1e4815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 >> 172.21.15.46:0/3563745847 conn(0x7f1e4806c980 msgr2=0x7f1e480d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 shutdown_connections 2024-04-26T23:56:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.394+0000 7f1e4e600640 1 -- 172.21.15.46:0/3563745847 wait complete. 2024-04-26T23:56:33.421 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:56:33.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:56:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.635+0000 7f3905800640 1 Processor -- start 2024-04-26T23:56:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.635+0000 7f3905800640 1 -- start start 2024-04-26T23:56:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 0x7f39000cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39000cd610 0x7f39000cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 0x7f39000aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f39000cdfb0 con 0x7f39000cd610 2024-04-26T23:56:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f39000af2c0 con 0x7f39000ae8f0 2024-04-26T23:56:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.636+0000 7f3905800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39000af570 con 0x7f390015f210 2024-04-26T23:56:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39000cd610 0x7f39000cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39000cd610 0x7f39000cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-04-26T23:56:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 -- 172.21.15.46:0/1153042814 learned_addr learned my addr 172.21.15.46:0/1153042814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ffe00640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 0x7f39000cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f3904e00640 1 --2- 172.21.15.46:0/1153042814 >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 0x7f39000aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 -- 172.21.15.46:0/1153042814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 msgr2=0x7f39000cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 0x7f39000cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 -- 172.21.15.46:0/1153042814 >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 msgr2=0x7f39000aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 --2- 172.21.15.46:0/1153042814 >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 0x7f39000aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ff400640 1 -- 172.21.15.46:0/1153042814 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39000af850 con 0x7f39000cd610 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f3904e00640 1 --2- 172.21.15.46:0/1153042814 >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 0x7f39000aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.637+0000 7f38ffe00640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 0x7f39000cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.638+0000 7f38ff400640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000cd610 0x7f39000cda70 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f39000cb1e0 tx=0x7f38e802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2a222e11af2258 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.639+0000 7f38fea00640 1 -- 172.21.15.46:0/1153042814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8037070 con 0x7f39000cd610 2024-04-26T23:56:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.639+0000 7f38fea00640 1 -- 172.21.15.46:0/1153042814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e8032440 con 0x7f39000cd610 2024-04-26T23:56:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.639+0000 7f38fea00640 1 -- 172.21.15.46:0/1153042814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8032810 con 0x7f39000cd610 2024-04-26T23:56:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.639+0000 7f3905800640 1 -- 172.21.15.46:0/1153042814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000cd610 msgr2=0x7f39000cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.639+0000 7f3905800640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000cd610 0x7f39000cda70 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f39000cb1e0 tx=0x7f38e802a1d0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 -- 172.21.15.46:0/1153042814 shutdown_connections 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 --2- 172.21.15.46:0/1153042814 >> v2:172.21.15.142:3300/0 conn(0x7f39000ae8f0 0x7f39000aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000cd610 0x7f39000cda70 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 --2- 172.21.15.46:0/1153042814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f390015f210 0x7f39000cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 -- 172.21.15.46:0/1153042814 >> 172.21.15.46:0/1153042814 conn(0x7f39000b27b0 msgr2=0x7f39000b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 -- 172.21.15.46:0/1153042814 shutdown_connections 2024-04-26T23:56:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.640+0000 7f3905800640 1 -- 172.21.15.46:0/1153042814 wait complete. 2024-04-26T23:56:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.642+0000 7f3905800640 1 Processor -- start 2024-04-26T23:56:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.642+0000 7f3905800640 1 -- start start 2024-04-26T23:56:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39000cd610 0x7f39000ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f390015f210 0x7f390006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39000cbc40 con 0x7f39000ae8f0 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39000cb960 con 0x7f390015f210 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.643+0000 7f3905800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f390015f5f0 con 0x7f39000cd610 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48058/0 (socket says 172.21.15.46:48058) 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 -- 172.21.15.46:0/3099905191 learned_addr learned my addr 172.21.15.46:0/3099905191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 -- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39000cd610 msgr2=0x7f39000ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39000cd610 0x7f39000ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 -- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f390015f210 msgr2=0x7f390006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f390015f210 0x7f390006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 -- 172.21.15.46:0/3099905191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39000cb1b0 con 0x7f39000ae8f0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.644+0000 7f38ffe00640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c7f0 tx=0x7f38f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.645+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0015070 con 0x7f39000ae8f0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.645+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f0007820 con 0x7f39000ae8f0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.645+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0007bd0 con 0x7f39000ae8f0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.645+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39000ca920 con 0x7f39000ae8f0 2024-04-26T23:56:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.645+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39000caed0 con 0x7f39000ae8f0 2024-04-26T23:56:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.646+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39000693b0 con 0x7f39000ae8f0 2024-04-26T23:56:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.647+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38f0018070 con 0x7f39000ae8f0 2024-04-26T23:56:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.648+0000 7f38f7e00640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 0x7f38dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.648+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f38f008be00 con 0x7f39000ae8f0 2024-04-26T23:56:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.649+0000 7f38ff400640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 0x7f38dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.650+0000 7f38ff400640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 0x7f38dc06edb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f38e8034000 tx=0x7f38e802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.653+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38f005a090 con 0x7f39000ae8f0 2024-04-26T23:56:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:33.901+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f39000ceb90 con 0x7f39000ae8f0 2024-04-26T23:56:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.403+0000 7f38f7e00640 1 -- 172.21.15.46:0/3099905191 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v158) ==== 122+0+0 (secure 0 0 0) 0x7f38f0059990 con 0x7f39000ae8f0 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 msgr2=0x7f38dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 0x7f38dc06edb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f38e8034000 tx=0x7f38e802c000 comp rx=0 tx=0).stop 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 msgr2=0x7f39000c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c7f0 tx=0x7f38f000ccb0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 shutdown_connections 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38dc06c8f0 0x7f38dc06edb0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f390015f210 0x7f390006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39000cd610 0x7f39000ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 --2- 172.21.15.46:0/3099905191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39000ae8f0 0x7f39000c55e0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.407+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 >> 172.21.15.46:0/3099905191 conn(0x7f39000b27b0 msgr2=0x7f390006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.408+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 shutdown_connections 2024-04-26T23:56:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.408+0000 7f3905800640 1 -- 172.21.15.46:0/3099905191 wait complete. 2024-04-26T23:56:34.435 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:56:34.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.645+0000 7f2083e00640 1 Processor -- start 2024-04-26T23:56:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.645+0000 7f2083e00640 1 -- start start 2024-04-26T23:56:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.646+0000 7f2083e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f20841466b0 0x7f208406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.646+0000 7f2083e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.646+0000 7f2083e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2084073ad0 con 0x7f2084160910 2024-04-26T23:56:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.646+0000 7f2083e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2084146a90 con 0x7f20841466b0 2024-04-26T23:56:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.646+0000 7f2083e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2084073d70 con 0x7f20840720e0 2024-04-26T23:56:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.647+0000 7f2082000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.647+0000 7f2082000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:43576/0 (socket says 172.21.15.46:43576) 2024-04-26T23:56:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.647+0000 7f2082000640 1 -- 172.21.15.46:0/550726986 learned_addr learned my addr 172.21.15.46:0/550726986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.647+0000 7f2083400640 1 --1- 172.21.15.46:0/550726986 >> v1:172.21.15.142:6789/0 conn(0x7f2084160910 0x7f20840cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:52398/0 (socket says 172.21.15.46:52398) 2024-04-26T23:56:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.647+0000 7f2082a00640 1 --2- 172.21.15.46:0/550726986 >> v2:172.21.15.142:3300/0 conn(0x7f20841466b0 0x7f208406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.648+0000 7f2082000640 1 -- 172.21.15.46:0/550726986 >> v2:172.21.15.142:3300/0 conn(0x7f20841466b0 msgr2=0x7f208406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.648+0000 7f2082000640 1 --2- 172.21.15.46:0/550726986 >> v2:172.21.15.142:3300/0 conn(0x7f20841466b0 0x7f208406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.648+0000 7f2082000640 1 -- 172.21.15.46:0/550726986 >> v1:172.21.15.142:6789/0 conn(0x7f2084160910 legacy=0x7f20840cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.648+0000 7f2082000640 1 -- 172.21.15.46:0/550726986 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2084074050 con 0x7f20840720e0 2024-04-26T23:56:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.650+0000 7f2082000640 1 --2- 172.21.15.46:0/550726986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2070009870 tx=0x7f207002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee3a658cbf1056f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.650+0000 7f2081600640 1 -- 172.21.15.46:0/550726986 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070037070 con 0x7f20840720e0 2024-04-26T23:56:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.650+0000 7f2081600640 1 -- 172.21.15.46:0/550726986 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2070032440 con 0x7f20840720e0 2024-04-26T23:56:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.650+0000 7f2081600640 1 -- 172.21.15.46:0/550726986 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070032810 con 0x7f20840720e0 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 -- 172.21.15.46:0/550726986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 msgr2=0x7f20840cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 --2- 172.21.15.46:0/550726986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2070009870 tx=0x7f207002a250 comp rx=0 tx=0).stop 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 -- 172.21.15.46:0/550726986 shutdown_connections 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 --2- 172.21.15.46:0/550726986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20840cd860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 --2- 172.21.15.46:0/550726986 >> v2:172.21.15.142:3300/0 conn(0x7f20841466b0 0x7f208406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 -- 172.21.15.46:0/550726986 >> 172.21.15.46:0/550726986 conn(0x7f20840b26d0 msgr2=0x7f20840b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 -- 172.21.15.46:0/550726986 shutdown_connections 2024-04-26T23:56:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.651+0000 7f2083e00640 1 -- 172.21.15.46:0/550726986 wait complete. 2024-04-26T23:56:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.653+0000 7f2083e00640 1 Processor -- start 2024-04-26T23:56:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.653+0000 7f2083e00640 1 -- start start 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20841600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2084160910 0x7f20840b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f208406e3c0 con 0x7f20841466b0 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20840cb470 con 0x7f2084160910 2024-04-26T23:56:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.654+0000 7f2083e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20840cdda0 con 0x7f20840720e0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48062/0 (socket says 172.21.15.46:48062) 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 -- 172.21.15.46:0/256742509 learned_addr learned my addr 172.21.15.46:0/256742509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082a00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20841600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2083400640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2084160910 0x7f20840b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 -- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 msgr2=0x7f20841600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20841600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 -- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2084160910 msgr2=0x7f20840b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2084160910 0x7f20840b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082000640 1 -- 172.21.15.46:0/256742509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840b2060 con 0x7f20841466b0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.655+0000 7f2082a00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20841600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f2082000640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f2070009870 tx=0x7f20700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:34.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070041070 con 0x7f20841466b0 2024-04-26T23:56:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207003c870 con 0x7f20841466b0 2024-04-26T23:56:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2084145950 con 0x7f20841466b0 2024-04-26T23:56:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070037040 con 0x7f20841466b0 2024-04-26T23:56:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.656+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20840af5c0 con 0x7f20841466b0 2024-04-26T23:56:34.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.658+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20840693b0 con 0x7f20841466b0 2024-04-26T23:56:34.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.659+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2070044070 con 0x7f20841466b0 2024-04-26T23:56:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.660+0000 7f207aa00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 0x7f206406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.660+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f20700ac710 con 0x7f20841466b0 2024-04-26T23:56:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.661+0000 7f2082a00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 0x7f206406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.662+0000 7f2082a00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 0x7f206406edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2074009fd0 tx=0x7f207400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.664+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f207007a910 con 0x7f20841466b0 2024-04-26T23:56:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.913+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20840c7180 con 0x7f20841466b0 2024-04-26T23:56:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.914+0000 7f207aa00640 1 -- 172.21.15.46:0/256742509 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7f207007a210 con 0x7f20841466b0 2024-04-26T23:56:34.917 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:34.918 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:34.918 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 msgr2=0x7f206406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 0x7f206406edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2074009fd0 tx=0x7f207400b040 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 msgr2=0x7f20841e6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f2070009870 tx=0x7f20700046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 shutdown_connections 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f206406c8f0 0x7f206406edb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2084160910 0x7f20840b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20841466b0 0x7f20841e6750 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 --2- 172.21.15.46:0/256742509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840720e0 0x7f20841600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.917+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 >> 172.21.15.46:0/256742509 conn(0x7f20840b26d0 msgr2=0x7f20840c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.918+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 shutdown_connections 2024-04-26T23:56:34.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:34.918+0000 7f2083e00640 1 -- 172.21.15.46:0/256742509 wait complete. 2024-04-26T23:56:34.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:35.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.140+0000 7f4821e00640 1 Processor -- start 2024-04-26T23:56:35.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.140+0000 7f4821e00640 1 -- start start 2024-04-26T23:56:35.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f4821e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f481c160bd0 0x7f481c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f4821e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f481c0732c0 0x7f481c073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f4821e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481c161940 con 0x7f481c06e510 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f4821e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f481c071f80 con 0x7f481c0732c0 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f4821e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f481c072260 con 0x7f481c160bd0 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f481aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f481c06e510 0x7f481c06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59596/0 (socket says 172.21.15.46:59596) 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f481aa00640 1 -- 172.21.15.46:0/4210319772 learned_addr learned my addr 172.21.15.46:0/4210319772 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f481b400640 1 --2- 172.21.15.46:0/4210319772 >> v2:172.21.15.46:3300/0 conn(0x7f481c0732c0 0x7f481c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f481a000640 1 --2- 172.21.15.46:0/4210319772 >> v2:172.21.15.142:3300/0 conn(0x7f481c160bd0 0x7f481c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.142+0000 7f481b400640 1 -- 172.21.15.46:0/4210319772 >> v2:172.21.15.142:3300/0 conn(0x7f481c160bd0 msgr2=0x7f481c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.143+0000 7f481b400640 1 --2- 172.21.15.46:0/4210319772 >> v2:172.21.15.142:3300/0 conn(0x7f481c160bd0 0x7f481c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.143+0000 7f481b400640 1 -- 172.21.15.46:0/4210319772 >> v1:172.21.15.46:6789/0 conn(0x7f481c06e510 legacy=0x7f481c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.143+0000 7f481b400640 1 -- 172.21.15.46:0/4210319772 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c072540 con 0x7f481c0732c0 2024-04-26T23:56:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.143+0000 7f4819600640 1 -- 172.21.15.46:0/4210319772 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387634056 0 0) 0x7f481c161940 con 0x7f481c06e510 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f481b400640 1 --2- 172.21.15.46:0/4210319772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0732c0 0x7f481c073730 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4814009d90 tx=0x7f481402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1714237823019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f4819600640 1 -- 172.21.15.46:0/4210319772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814007740 con 0x7f481c0732c0 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f4819600640 1 -- 172.21.15.46:0/4210319772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48140078e0 con 0x7f481c0732c0 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f4819600640 1 -- 172.21.15.46:0/4210319772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814007cb0 con 0x7f481c0732c0 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f4821e00640 1 -- 172.21.15.46:0/4210319772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0732c0 msgr2=0x7f481c073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.144+0000 7f4821e00640 1 --2- 172.21.15.46:0/4210319772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0732c0 0x7f481c073730 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4814009d90 tx=0x7f481402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 -- 172.21.15.46:0/4210319772 shutdown_connections 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 --2- 172.21.15.46:0/4210319772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0732c0 0x7f481c073730 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 --2- 172.21.15.46:0/4210319772 >> v2:172.21.15.142:3300/0 conn(0x7f481c160bd0 0x7f481c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 -- 172.21.15.46:0/4210319772 >> 172.21.15.46:0/4210319772 conn(0x7f481c06c530 msgr2=0x7f481c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 -- 172.21.15.46:0/4210319772 shutdown_connections 2024-04-26T23:56:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.145+0000 7f4821e00640 1 -- 172.21.15.46:0/4210319772 wait complete. 2024-04-26T23:56:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.146+0000 7f4821e00640 1 Processor -- start 2024-04-26T23:56:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 -- start start 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f481c06e510 0x7f481c13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f481c0732c0 0x7f481c13e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f481c161720 con 0x7f481c160bd0 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1615a0 con 0x7f481c0732c0 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.147+0000 7f4821e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f481c072d40 con 0x7f481c06e510 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48078/0 (socket says 172.21.15.46:48078) 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 -- 172.21.15.46:0/1190823263 learned_addr learned my addr 172.21.15.46:0/1190823263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481a000640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f481c0732c0 0x7f481c13e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481aa00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f481c06e510 0x7f481c13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 -- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f481c06e510 msgr2=0x7f481c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f481c06e510 0x7f481c13c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 -- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f481c0732c0 msgr2=0x7f481c13e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f481c0732c0 0x7f481c13e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.148+0000 7f481b400640 1 -- 172.21.15.46:0/1190823263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c13e550 con 0x7f481c160bd0 2024-04-26T23:56:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f481b400640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f481402ac90 tx=0x7f4814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814040070 con 0x7f481c160bd0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4814031bc0 con 0x7f481c160bd0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814045460 con 0x7f481c160bd0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c13e820 con 0x7f481c160bd0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.149+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c13ede0 con 0x7f481c160bd0 2024-04-26T23:56:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.150+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c069530 con 0x7f481c160bd0 2024-04-26T23:56:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.151+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4814039040 con 0x7f481c160bd0 2024-04-26T23:56:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.152+0000 7f4813400640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 0x7f47fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.152+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f48140ad7b0 con 0x7f481c160bd0 2024-04-26T23:56:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.153+0000 7f481aa00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 0x7f47fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.154+0000 7f481aa00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 0x7f47fc06edb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f60 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.157+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f481407bad0 con 0x7f481c160bd0 2024-04-26T23:56:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.363+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f481c06fb00 con 0x7f481c160bd0 2024-04-26T23:56:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.367+0000 7f4813400640 1 -- 172.21.15.46:0/1190823263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) ==== 74+0+16848 (secure 0 0 0) 0x7f481407b3d0 con 0x7f481c160bd0 2024-04-26T23:56:35.370 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:35.370 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":199,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:32.320531+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.369+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 msgr2=0x7f47fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 0x7f47fc06edb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f60 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 msgr2=0x7f481c13f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f481402ac90 tx=0x7f4814004a40 comp rx=0 tx=0).stop 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 shutdown_connections 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47fc06c8f0 0x7f47fc06edb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c160bd0 0x7f481c13f900 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f481c0732c0 0x7f481c13e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 --2- 172.21.15.46:0/1190823263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f481c06e510 0x7f481c13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 >> 172.21.15.46:0/1190823263 conn(0x7f481c06c530 msgr2=0x7f481c15f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 shutdown_connections 2024-04-26T23:56:35.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.370+0000 7f4821e00640 1 -- 172.21.15.46:0/1190823263 wait complete. 2024-04-26T23:56:35.392 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:56:35.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:56:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.588+0000 7ff6c5e00640 1 Processor -- start 2024-04-26T23:56:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.588+0000 7ff6c5e00640 1 -- start start 2024-04-26T23:56:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.589+0000 7ff6c5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6c01634a0 0x7ff6c015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.589+0000 7ff6c5e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c00d19a0 con 0x7ff6c00ccb20 2024-04-26T23:56:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.589+0000 7ff6c5e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c0163a10 con 0x7ff6c006ca60 2024-04-26T23:56:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.589+0000 7ff6c5e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c0163cf0 con 0x7ff6c01634a0 2024-04-26T23:56:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6c01634a0 0x7ff6c015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c4a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6c00ccb20 0x7ff6c006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59612/0 (socket says 172.21.15.46:59612) 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c4a00640 1 -- 172.21.15.46:0/2193542887 learned_addr learned my addr 172.21.15.46:0/2193542887 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c5400640 1 -- 172.21.15.46:0/2193542887 >> v1:172.21.15.142:6789/0 conn(0x7ff6c006ca60 legacy=0x7ff6c0162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6bfe00640 1 -- 172.21.15.46:0/2193542887 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147659772 0 0) 0x7ff6c00d19a0 con 0x7ff6c00ccb20 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c5400640 1 -- 172.21.15.46:0/2193542887 >> v1:172.21.15.46:6789/0 conn(0x7ff6c00ccb20 legacy=0x7ff6c006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.590+0000 7ff6c5400640 1 -- 172.21.15.46:0/2193542887 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c00cfeb0 con 0x7ff6c01634a0 2024-04-26T23:56:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.592+0000 7ff6c5400640 1 --2- 172.21.15.46:0/2193542887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c01634a0 0x7ff6c015e8b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009d90 tx=0x7ff6b802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cfe752fd7bb056 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.592+0000 7ff6bfe00640 1 -- 172.21.15.46:0/2193542887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8007740 con 0x7ff6c01634a0 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.592+0000 7ff6bfe00640 1 -- 172.21.15.46:0/2193542887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b80078e0 con 0x7ff6c01634a0 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.592+0000 7ff6bfe00640 1 -- 172.21.15.46:0/2193542887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8007cb0 con 0x7ff6c01634a0 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2193542887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c01634a0 msgr2=0x7ff6c015e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2193542887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c01634a0 0x7ff6c015e8b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009d90 tx=0x7ff6b802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2193542887 shutdown_connections 2024-04-26T23:56:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2193542887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c01634a0 0x7ff6c015e8b0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2193542887 >> 172.21.15.46:0/2193542887 conn(0x7ff6c00d1c30 msgr2=0x7ff6c00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2193542887 shutdown_connections 2024-04-26T23:56:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.593+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2193542887 wait complete. 2024-04-26T23:56:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.595+0000 7ff6c5e00640 1 Processor -- start 2024-04-26T23:56:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.595+0000 7ff6c5e00640 1 -- start start 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6c006ca60 0x7ff6c0155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 0x7ff6c006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c00ccf00 con 0x7ff6c00ccb20 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c00d0470 con 0x7ff6c006ca60 2024-04-26T23:56:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c00ae8f0 con 0x7ff6c01634a0 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48102/0 (socket says 172.21.15.46:48102) 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 -- 172.21.15.46:0/2118642102 learned_addr learned my addr 172.21.15.46:0/2118642102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c4a00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6c006ca60 0x7ff6c0155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5400640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 0x7ff6c006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 -- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 msgr2=0x7ff6c006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 0x7ff6c006f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 -- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6c006ca60 msgr2=0x7ff6c0155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6c006ca60 0x7ff6c0155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6b7e00640 1 -- 172.21.15.46:0/2118642102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c01f5a80 con 0x7ff6c00ccb20 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.596+0000 7ff6c5400640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 0x7ff6c006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6b7e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c450 tx=0x7ff6b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00052d0 con 0x7ff6c00ccb20 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0005470 con 0x7ff6c00ccb20 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0012870 con 0x7ff6c00ccb20 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c01f84a0 con 0x7ff6c00ccb20 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.597+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c01f8a60 con 0x7ff6c00ccb20 2024-04-26T23:56:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.599+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c01f8c60 con 0x7ff6c00ccb20 2024-04-26T23:56:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.600+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6b0012a10 con 0x7ff6c00ccb20 2024-04-26T23:56:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.600+0000 7ff6bd600640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.601+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7ff6b008b8f0 con 0x7ff6c00ccb20 2024-04-26T23:56:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.601+0000 7ff6c4a00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.602+0000 7ff6c4a00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 0x7ff69c06edb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000f60 tx=0x7ff6a8007bb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.605+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6c01f8c60 con 0x7ff6c00ccb20 2024-04-26T23:56:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.808+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6c00693b0 con 0x7ff69c06c8f0 2024-04-26T23:56:35.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.826+0000 7ff6bd600640 1 -- 172.21.15.46:0/2118642102 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247097 (secure 0 0 0) 0x7ff6c00693b0 con 0x7ff69c06c8f0 2024-04-26T23:56:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:56:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 msgr2=0x7ff69c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 0x7ff69c06edb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000f60 tx=0x7ff6a8007bb0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 msgr2=0x7ff6c006ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c450 tx=0x7ff6b000c910 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 shutdown_connections 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6c01634a0 0x7ff6c006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00ccb20 0x7ff6c006ef70 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 --2- 172.21.15.46:0/2118642102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6c006ca60 0x7ff6c0155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 >> 172.21.15.46:0/2118642102 conn(0x7ff6c00d1c30 msgr2=0x7ff6c0161b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 shutdown_connections 2024-04-26T23:56:35.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:35.833+0000 7ff6c5e00640 1 -- 172.21.15.46:0/2118642102 wait complete. 2024-04-26T23:56:35.855 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":779,"stamp":"2024-04-26T23:56:34.327872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439420,"kb_used_data":20616,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754535300,"statfs":{"total":773094113280,"available":772644147200,"internally_reserved":0,"allocated":21110784,"data_stored":10589760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":428723761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013808"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.327667+0000","last_change":"2024-04-26T23:56:30.327667+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.327667+0000","last_undegraded":"2024-04-26T23:56:30.327667+0000","last_fullsized":"2024-04-26T23:56:30.327667+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.341987+0000","last_change":"2024-04-26T23:56:30.341987+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.341987+0000","last_undegraded":"2024-04-26T23:56:30.341987+0000","last_fullsized":"2024-04-26T23:56:30.341987+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.330864+0000","last_change":"2024-04-26T23:56:30.330864+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.330864+0000","last_undegraded":"2024-04-26T23:56:30.330864+0000","last_fullsized":"2024-04-26T23:56:30.330864+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.338294+0000","last_change":"2024-04-26T23:56:30.338294+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.338294+0000","last_undegraded":"2024-04-26T23:56:30.338294+0000","last_fullsized":"2024-04-26T23:56:30.338294+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324281+0000","last_change":"2024-04-26T23:56:30.324281+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324281+0000","last_undegraded":"2024-04-26T23:56:30.324281+0000","last_fullsized":"2024-04-26T23:56:30.324281+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.334808+0000","last_change":"2024-04-26T23:56:30.334808+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.334808+0000","last_undegraded":"2024-04-26T23:56:30.334808+0000","last_fullsized":"2024-04-26T23:56:30.334808+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.329469+0000","last_change":"2024-04-26T23:56:30.329469+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.329469+0000","last_undegraded":"2024-04-26T23:56:30.329469+0000","last_fullsized":"2024-04-26T23:56:30.329469+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.341926+0000","last_change":"2024-04-26T23:56:30.341926+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.341926+0000","last_undegraded":"2024-04-26T23:56:30.341926+0000","last_fullsized":"2024-04-26T23:56:30.341926+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.325489+0000","last_change":"2024-04-26T23:56:30.325489+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.325489+0000","last_undegraded":"2024-04-26T23:56:30.325489+0000","last_fullsized":"2024-04-26T23:56:30.325489+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.338945+0000","last_change":"2024-04-26T23:56:30.338945+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.338945+0000","last_undegraded":"2024-04-26T23:56:30.338945+0000","last_fullsized":"2024-04-26T23:56:30.338945+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.342031+0000","last_change":"2024-04-26T23:56:30.342031+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.342031+0000","last_undegraded":"2024-04-26T23:56:30.342031+0000","last_fullsized":"2024-04-26T23:56:30.342031+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.343336+0000","last_change":"2024-04-26T23:56:30.343336+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.343336+0000","last_undegraded":"2024-04-26T23:56:30.343336+0000","last_fullsized":"2024-04-26T23:56:30.343336+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.341259+0000","last_change":"2024-04-26T23:56:30.341259+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.341259+0000","last_undegraded":"2024-04-26T23:56:30.341259+0000","last_fullsized":"2024-04-26T23:56:30.341259+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.351352+0000","last_change":"2024-04-26T23:56:30.351352+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.351352+0000","last_undegraded":"2024-04-26T23:56:30.351352+0000","last_fullsized":"2024-04-26T23:56:30.351352+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.329102+0000","last_change":"2024-04-26T23:56:30.329102+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.329102+0000","last_undegraded":"2024-04-26T23:56:30.329102+0000","last_fullsized":"2024-04-26T23:56:30.329102+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.341000+0000","last_change":"2024-04-26T23:56:30.341000+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.341000+0000","last_undegraded":"2024-04-26T23:56:30.341000+0000","last_fullsized":"2024-04-26T23:56:30.341000+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.349910+0000","last_change":"2024-04-26T23:56:30.349910+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.349910+0000","last_undegraded":"2024-04-26T23:56:30.349910+0000","last_fullsized":"2024-04-26T23:56:30.349910+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.338540+0000","last_change":"2024-04-26T23:56:30.338540+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.338540+0000","last_undegraded":"2024-04-26T23:56:30.338540+0000","last_fullsized":"2024-04-26T23:56:30.338540+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.328002+0000","last_change":"2024-04-26T23:56:30.328002+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.328002+0000","last_undegraded":"2024-04-26T23:56:30.328002+0000","last_fullsized":"2024-04-26T23:56:30.328002+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.334296+0000","last_change":"2024-04-26T23:56:30.334296+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.334296+0000","last_undegraded":"2024-04-26T23:56:30.334296+0000","last_fullsized":"2024-04-26T23:56:30.334296+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.331398+0000","last_change":"2024-04-26T23:56:30.331398+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.331398+0000","last_undegraded":"2024-04-26T23:56:30.331398+0000","last_fullsized":"2024-04-26T23:56:30.331398+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.330871+0000","last_change":"2024-04-26T23:56:30.330871+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.330871+0000","last_undegraded":"2024-04-26T23:56:30.330871+0000","last_fullsized":"2024-04-26T23:56:30.330871+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324903+0000","last_change":"2024-04-26T23:56:30.324903+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324903+0000","last_undegraded":"2024-04-26T23:56:30.324903+0000","last_fullsized":"2024-04-26T23:56:30.324903+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324569+0000","last_change":"2024-04-26T23:56:30.324569+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324569+0000","last_undegraded":"2024-04-26T23:56:30.324569+0000","last_fullsized":"2024-04-26T23:56:30.324569+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.326109+0000","last_change":"2024-04-26T23:56:30.326109+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.326109+0000","last_undegraded":"2024-04-26T23:56:30.326109+0000","last_fullsized":"2024-04-26T23:56:30.326109+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.337291+0000","last_change":"2024-04-26T23:56:30.337291+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.337291+0000","last_undegraded":"2024-04-26T23:56:30.337291+0000","last_fullsized":"2024-04-26T23:56:30.337291+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.339458+0000","last_change":"2024-04-26T23:56:30.339458+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.339458+0000","last_undegraded":"2024-04-26T23:56:30.339458+0000","last_fullsized":"2024-04-26T23:56:30.339458+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.342616+0000","last_change":"2024-04-26T23:56:30.342616+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.342616+0000","last_undegraded":"2024-04-26T23:56:30.342616+0000","last_fullsized":"2024-04-26T23:56:30.342616+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.342406+0000","last_change":"2024-04-26T23:56:30.342406+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.342406+0000","last_undegraded":"2024-04-26T23:56:30.342406+0000","last_fullsized":"2024-04-26T23:56:30.342406+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.336109+0000","last_change":"2024-04-26T23:56:30.336109+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.336109+0000","last_undegraded":"2024-04-26T23:56:30.336109+0000","last_fullsized":"2024-04-26T23:56:30.336109+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324103+0000","last_change":"2024-04-26T23:56:30.324103+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324103+0000","last_undegraded":"2024-04-26T23:56:30.324103+0000","last_fullsized":"2024-04-26T23:56:30.324103+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.331177+0000","last_change":"2024-04-26T23:56:30.331177+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.331177+0000","last_undegraded":"2024-04-26T23:56:30.331177+0000","last_fullsized":"2024-04-26T23:56:30.331177+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.333785+0000","last_change":"2024-04-26T23:56:30.333785+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.333785+0000","last_undegraded":"2024-04-26T23:56:30.333785+0000","last_fullsized":"2024-04-26T23:56:30.333785+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.338226+0000","last_change":"2024-04-26T23:56:30.335217+0000","last_active":"2024-04-26T23:56:30.338226+0000","last_peered":"2024-04-26T23:56:30.338226+0000","last_clean":"2024-04-26T23:56:30.338226+0000","last_became_active":"2024-04-26T23:56:30.334736+0000","last_became_peered":"2024-04-26T23:56:30.334736+0000","last_unstale":"2024-04-26T23:56:30.338226+0000","last_undegraded":"2024-04-26T23:56:30.338226+0000","last_fullsized":"2024-04-26T23:56:30.338226+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:31.563891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.330040+0000","last_change":"2024-04-26T23:56:30.330040+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.330040+0000","last_undegraded":"2024-04-26T23:56:30.330040+0000","last_fullsized":"2024-04-26T23:56:30.330040+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360713+0000","last_change":"2024-04-26T23:56:30.358279+0000","last_active":"2024-04-26T23:56:30.360713+0000","last_peered":"2024-04-26T23:56:30.360713+0000","last_clean":"2024-04-26T23:56:30.360713+0000","last_became_active":"2024-04-26T23:56:30.357695+0000","last_became_peered":"2024-04-26T23:56:30.357695+0000","last_unstale":"2024-04-26T23:56:30.360713+0000","last_undegraded":"2024-04-26T23:56:30.360713+0000","last_fullsized":"2024-04-26T23:56:30.360713+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:49.809604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.344547+0000","last_change":"2024-04-26T23:56:30.344547+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.344547+0000","last_undegraded":"2024-04-26T23:56:30.344547+0000","last_fullsized":"2024-04-26T23:56:30.344547+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.362527+0000","last_change":"2024-04-26T23:56:30.361728+0000","last_active":"2024-04-26T23:56:30.362527+0000","last_peered":"2024-04-26T23:56:30.362527+0000","last_clean":"2024-04-26T23:56:30.362527+0000","last_became_active":"2024-04-26T23:56:30.361416+0000","last_became_peered":"2024-04-26T23:56:30.361416+0000","last_unstale":"2024-04-26T23:56:30.362527+0000","last_undegraded":"2024-04-26T23:56:30.362527+0000","last_fullsized":"2024-04-26T23:56:30.362527+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:06.762279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.332272+0000","last_change":"2024-04-26T23:56:30.332272+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.332272+0000","last_undegraded":"2024-04-26T23:56:30.332272+0000","last_fullsized":"2024-04-26T23:56:30.332272+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.357979+0000","last_change":"2024-04-26T23:56:30.357016+0000","last_active":"2024-04-26T23:56:30.357979+0000","last_peered":"2024-04-26T23:56:30.357979+0000","last_clean":"2024-04-26T23:56:30.357979+0000","last_became_active":"2024-04-26T23:56:30.356716+0000","last_became_peered":"2024-04-26T23:56:30.356716+0000","last_unstale":"2024-04-26T23:56:30.357979+0000","last_undegraded":"2024-04-26T23:56:30.357979+0000","last_fullsized":"2024-04-26T23:56:30.357979+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:30.680322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.349289+0000","last_change":"2024-04-26T23:56:30.349289+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.349289+0000","last_undegraded":"2024-04-26T23:56:30.349289+0000","last_fullsized":"2024-04-26T23:56:30.349289+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360210+0000","last_change":"2024-04-26T23:56:30.358799+0000","last_active":"2024-04-26T23:56:30.360210+0000","last_peered":"2024-04-26T23:56:30.360210+0000","last_clean":"2024-04-26T23:56:30.360210+0000","last_became_active":"2024-04-26T23:56:30.358506+0000","last_became_peered":"2024-04-26T23:56:30.358506+0000","last_unstale":"2024-04-26T23:56:30.360210+0000","last_undegraded":"2024-04-26T23:56:30.360210+0000","last_fullsized":"2024-04-26T23:56:30.360210+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:46:46.531883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.330265+0000","last_change":"2024-04-26T23:56:30.330265+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.330265+0000","last_undegraded":"2024-04-26T23:56:30.330265+0000","last_fullsized":"2024-04-26T23:56:30.330265+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.356415+0000","last_change":"2024-04-26T23:56:30.354074+0000","last_active":"2024-04-26T23:56:30.356415+0000","last_peered":"2024-04-26T23:56:30.356415+0000","last_clean":"2024-04-26T23:56:30.356415+0000","last_became_active":"2024-04-26T23:56:30.353806+0000","last_became_peered":"2024-04-26T23:56:30.353806+0000","last_unstale":"2024-04-26T23:56:30.356415+0000","last_undegraded":"2024-04-26T23:56:30.356415+0000","last_fullsized":"2024-04-26T23:56:30.356415+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:59.564743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.325106+0000","last_change":"2024-04-26T23:56:30.325106+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.325106+0000","last_undegraded":"2024-04-26T23:56:30.325106+0000","last_fullsized":"2024-04-26T23:56:30.325106+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.343978+0000","last_change":"2024-04-26T23:56:30.341747+0000","last_active":"2024-04-26T23:56:30.343978+0000","last_peered":"2024-04-26T23:56:30.343978+0000","last_clean":"2024-04-26T23:56:30.343978+0000","last_became_active":"2024-04-26T23:56:30.341227+0000","last_became_peered":"2024-04-26T23:56:30.341227+0000","last_unstale":"2024-04-26T23:56:30.343978+0000","last_undegraded":"2024-04-26T23:56:30.343978+0000","last_fullsized":"2024-04-26T23:56:30.343978+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:58.285053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.333819+0000","last_change":"2024-04-26T23:56:30.333819+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.333819+0000","last_undegraded":"2024-04-26T23:56:30.333819+0000","last_fullsized":"2024-04-26T23:56:30.333819+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.355806+0000","last_change":"2024-04-26T23:56:30.354063+0000","last_active":"2024-04-26T23:56:30.355806+0000","last_peered":"2024-04-26T23:56:30.355806+0000","last_clean":"2024-04-26T23:56:30.355806+0000","last_became_active":"2024-04-26T23:56:30.353776+0000","last_became_peered":"2024-04-26T23:56:30.353776+0000","last_unstale":"2024-04-26T23:56:30.355806+0000","last_undegraded":"2024-04-26T23:56:30.355806+0000","last_fullsized":"2024-04-26T23:56:30.355806+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:17.713731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.335126+0000","last_change":"2024-04-26T23:56:30.335126+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.335126+0000","last_undegraded":"2024-04-26T23:56:30.335126+0000","last_fullsized":"2024-04-26T23:56:30.335126+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.361328+0000","last_change":"2024-04-26T23:56:30.356537+0000","last_active":"2024-04-26T23:56:30.361328+0000","last_peered":"2024-04-26T23:56:30.361328+0000","last_clean":"2024-04-26T23:56:30.361328+0000","last_became_active":"2024-04-26T23:56:30.356232+0000","last_became_peered":"2024-04-26T23:56:30.356232+0000","last_unstale":"2024-04-26T23:56:30.361328+0000","last_undegraded":"2024-04-26T23:56:30.361328+0000","last_fullsized":"2024-04-26T23:56:30.361328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.302253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.335403+0000","last_change":"2024-04-26T23:56:30.335403+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.335403+0000","last_undegraded":"2024-04-26T23:56:30.335403+0000","last_fullsized":"2024-04-26T23:56:30.335403+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.354670+0000","last_change":"2024-04-26T23:56:30.352656+0000","last_active":"2024-04-26T23:56:30.354670+0000","last_peered":"2024-04-26T23:56:30.354670+0000","last_clean":"2024-04-26T23:56:30.354670+0000","last_became_active":"2024-04-26T23:56:30.349500+0000","last_became_peered":"2024-04-26T23:56:30.349500+0000","last_unstale":"2024-04-26T23:56:30.354670+0000","last_undegraded":"2024-04-26T23:56:30.354670+0000","last_fullsized":"2024-04-26T23:56:30.354670+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:49.684474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.345574+0000","last_change":"2024-04-26T23:56:30.345574+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.345574+0000","last_undegraded":"2024-04-26T23:56:30.345574+0000","last_fullsized":"2024-04-26T23:56:30.345574+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.362278+0000","last_change":"2024-04-26T23:56:30.360988+0000","last_active":"2024-04-26T23:56:30.362278+0000","last_peered":"2024-04-26T23:56:30.362278+0000","last_clean":"2024-04-26T23:56:30.362278+0000","last_became_active":"2024-04-26T23:56:30.360611+0000","last_became_peered":"2024-04-26T23:56:30.360611+0000","last_unstale":"2024-04-26T23:56:30.362278+0000","last_undegraded":"2024-04-26T23:56:30.362278+0000","last_fullsized":"2024-04-26T23:56:30.362278+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:06.261901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.341196+0000","last_change":"2024-04-26T23:56:30.341196+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.341196+0000","last_undegraded":"2024-04-26T23:56:30.341196+0000","last_fullsized":"2024-04-26T23:56:30.341196+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360388+0000","last_change":"2024-04-26T23:56:30.357684+0000","last_active":"2024-04-26T23:56:30.360388+0000","last_peered":"2024-04-26T23:56:30.360388+0000","last_clean":"2024-04-26T23:56:30.360388+0000","last_became_active":"2024-04-26T23:56:30.357048+0000","last_became_peered":"2024-04-26T23:56:30.357048+0000","last_unstale":"2024-04-26T23:56:30.360388+0000","last_undegraded":"2024-04-26T23:56:30.360388+0000","last_fullsized":"2024-04-26T23:56:30.360388+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:38.710852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.346437+0000","last_change":"2024-04-26T23:56:30.346437+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.346437+0000","last_undegraded":"2024-04-26T23:56:30.346437+0000","last_fullsized":"2024-04-26T23:56:30.346437+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.335578+0000","last_change":"2024-04-26T23:56:30.330647+0000","last_active":"2024-04-26T23:56:30.335578+0000","last_peered":"2024-04-26T23:56:30.335578+0000","last_clean":"2024-04-26T23:56:30.335578+0000","last_became_active":"2024-04-26T23:56:30.330241+0000","last_became_peered":"2024-04-26T23:56:30.330241+0000","last_unstale":"2024-04-26T23:56:30.335578+0000","last_undegraded":"2024-04-26T23:56:30.335578+0000","last_fullsized":"2024-04-26T23:56:30.335578+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:59.814054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.329615+0000","last_change":"2024-04-26T23:56:30.329615+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.329615+0000","last_undegraded":"2024-04-26T23:56:30.329615+0000","last_fullsized":"2024-04-26T23:56:30.329615+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.353778+0000","last_change":"2024-04-26T23:56:30.349893+0000","last_active":"2024-04-26T23:56:30.353778+0000","last_peered":"2024-04-26T23:56:30.353778+0000","last_clean":"2024-04-26T23:56:30.353778+0000","last_became_active":"2024-04-26T23:56:30.348198+0000","last_became_peered":"2024-04-26T23:56:30.348198+0000","last_unstale":"2024-04-26T23:56:30.353778+0000","last_undegraded":"2024-04-26T23:56:30.353778+0000","last_fullsized":"2024-04-26T23:56:30.353778+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:10.240151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.331691+0000","last_change":"2024-04-26T23:56:30.331691+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.331691+0000","last_undegraded":"2024-04-26T23:56:30.331691+0000","last_fullsized":"2024-04-26T23:56:30.331691+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.354814+0000","last_change":"2024-04-26T23:56:30.353426+0000","last_active":"2024-04-26T23:56:30.354814+0000","last_peered":"2024-04-26T23:56:30.354814+0000","last_clean":"2024-04-26T23:56:30.354814+0000","last_became_active":"2024-04-26T23:56:30.352514+0000","last_became_peered":"2024-04-26T23:56:30.352514+0000","last_unstale":"2024-04-26T23:56:30.354814+0000","last_undegraded":"2024-04-26T23:56:30.354814+0000","last_fullsized":"2024-04-26T23:56:30.354814+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.081232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.339308+0000","last_change":"2024-04-26T23:56:30.339308+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.339308+0000","last_undegraded":"2024-04-26T23:56:30.339308+0000","last_fullsized":"2024-04-26T23:56:30.339308+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.358060+0000","last_change":"2024-04-26T23:56:30.357263+0000","last_active":"2024-04-26T23:56:30.358060+0000","last_peered":"2024-04-26T23:56:30.358060+0000","last_clean":"2024-04-26T23:56:30.358060+0000","last_became_active":"2024-04-26T23:56:30.356822+0000","last_became_peered":"2024-04-26T23:56:30.356822+0000","last_unstale":"2024-04-26T23:56:30.358060+0000","last_undegraded":"2024-04-26T23:56:30.358060+0000","last_fullsized":"2024-04-26T23:56:30.358060+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:37.623206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.326821+0000","last_change":"2024-04-26T23:56:30.326821+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.326821+0000","last_undegraded":"2024-04-26T23:56:30.326821+0000","last_fullsized":"2024-04-26T23:56:30.326821+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.343093+0000","last_change":"2024-04-26T23:56:30.342101+0000","last_active":"2024-04-26T23:56:30.343093+0000","last_peered":"2024-04-26T23:56:30.343093+0000","last_clean":"2024-04-26T23:56:30.343093+0000","last_became_active":"2024-04-26T23:56:30.341785+0000","last_became_peered":"2024-04-26T23:56:30.341785+0000","last_unstale":"2024-04-26T23:56:30.343093+0000","last_undegraded":"2024-04-26T23:56:30.343093+0000","last_fullsized":"2024-04-26T23:56:30.343093+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:23.609703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.329839+0000","last_change":"2024-04-26T23:56:30.329839+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.329839+0000","last_undegraded":"2024-04-26T23:56:30.329839+0000","last_fullsized":"2024-04-26T23:56:30.329839+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.364115+0000","last_change":"2024-04-26T23:56:30.363295+0000","last_active":"2024-04-26T23:56:30.364115+0000","last_peered":"2024-04-26T23:56:30.364115+0000","last_clean":"2024-04-26T23:56:30.364115+0000","last_became_active":"2024-04-26T23:56:30.363014+0000","last_became_peered":"2024-04-26T23:56:30.363014+0000","last_unstale":"2024-04-26T23:56:30.364115+0000","last_undegraded":"2024-04-26T23:56:30.364115+0000","last_fullsized":"2024-04-26T23:56:30.364115+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:59.596927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.349443+0000","last_change":"2024-04-26T23:56:30.349443+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.349443+0000","last_undegraded":"2024-04-26T23:56:30.349443+0000","last_fullsized":"2024-04-26T23:56:30.349443+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.363809+0000","last_change":"2024-04-26T23:56:30.363170+0000","last_active":"2024-04-26T23:56:30.363809+0000","last_peered":"2024-04-26T23:56:30.363809+0000","last_clean":"2024-04-26T23:56:30.363809+0000","last_became_active":"2024-04-26T23:56:30.362784+0000","last_became_peered":"2024-04-26T23:56:30.362784+0000","last_unstale":"2024-04-26T23:56:30.363809+0000","last_undegraded":"2024-04-26T23:56:30.363809+0000","last_fullsized":"2024-04-26T23:56:30.363809+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:44.272460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.342209+0000","last_change":"2024-04-26T23:56:30.342209+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.342209+0000","last_undegraded":"2024-04-26T23:56:30.342209+0000","last_fullsized":"2024-04-26T23:56:30.342209+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360912+0000","last_change":"2024-04-26T23:56:30.358916+0000","last_active":"2024-04-26T23:56:30.360912+0000","last_peered":"2024-04-26T23:56:30.360912+0000","last_clean":"2024-04-26T23:56:30.360912+0000","last_became_active":"2024-04-26T23:56:30.358236+0000","last_became_peered":"2024-04-26T23:56:30.358236+0000","last_unstale":"2024-04-26T23:56:30.360912+0000","last_undegraded":"2024-04-26T23:56:30.360912+0000","last_fullsized":"2024-04-26T23:56:30.360912+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:20.445697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324584+0000","last_change":"2024-04-26T23:56:30.324584+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324584+0000","last_undegraded":"2024-04-26T23:56:30.324584+0000","last_fullsized":"2024-04-26T23:56:30.324584+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.349533+0000","last_change":"2024-04-26T23:56:30.348804+0000","last_active":"2024-04-26T23:56:30.349533+0000","last_peered":"2024-04-26T23:56:30.349533+0000","last_clean":"2024-04-26T23:56:30.349533+0000","last_became_active":"2024-04-26T23:56:30.348363+0000","last_became_peered":"2024-04-26T23:56:30.348363+0000","last_unstale":"2024-04-26T23:56:30.349533+0000","last_undegraded":"2024-04-26T23:56:30.349533+0000","last_fullsized":"2024-04-26T23:56:30.349533+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:42.136259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.338607+0000","last_change":"2024-04-26T23:56:30.338607+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.338607+0000","last_undegraded":"2024-04-26T23:56:30.338607+0000","last_fullsized":"2024-04-26T23:56:30.338607+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.349745+0000","last_change":"2024-04-26T23:56:30.348944+0000","last_active":"2024-04-26T23:56:30.349745+0000","last_peered":"2024-04-26T23:56:30.349745+0000","last_clean":"2024-04-26T23:56:30.349745+0000","last_became_active":"2024-04-26T23:56:30.347911+0000","last_became_peered":"2024-04-26T23:56:30.347911+0000","last_unstale":"2024-04-26T23:56:30.349745+0000","last_undegraded":"2024-04-26T23:56:30.349745+0000","last_fullsized":"2024-04-26T23:56:30.349745+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:34.679261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.330890+0000","last_change":"2024-04-26T23:56:30.330890+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.330890+0000","last_undegraded":"2024-04-26T23:56:30.330890+0000","last_fullsized":"2024-04-26T23:56:30.330890+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.342810+0000","last_change":"2024-04-26T23:56:30.339407+0000","last_active":"2024-04-26T23:56:30.342810+0000","last_peered":"2024-04-26T23:56:30.342810+0000","last_clean":"2024-04-26T23:56:30.342810+0000","last_became_active":"2024-04-26T23:56:30.336661+0000","last_became_peered":"2024-04-26T23:56:30.336661+0000","last_unstale":"2024-04-26T23:56:30.342810+0000","last_undegraded":"2024-04-26T23:56:30.342810+0000","last_fullsized":"2024-04-26T23:56:30.342810+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:22.755695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.338659+0000","last_change":"2024-04-26T23:56:30.338659+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.338659+0000","last_undegraded":"2024-04-26T23:56:30.338659+0000","last_fullsized":"2024-04-26T23:56:30.338659+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360858+0000","last_change":"2024-04-26T23:56:30.359227+0000","last_active":"2024-04-26T23:56:30.360858+0000","last_peered":"2024-04-26T23:56:30.360858+0000","last_clean":"2024-04-26T23:56:30.360858+0000","last_became_active":"2024-04-26T23:56:30.358933+0000","last_became_peered":"2024-04-26T23:56:30.358933+0000","last_unstale":"2024-04-26T23:56:30.360858+0000","last_undegraded":"2024-04-26T23:56:30.360858+0000","last_fullsized":"2024-04-26T23:56:30.360858+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:47.281179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.325099+0000","last_change":"2024-04-26T23:56:30.325099+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.325099+0000","last_undegraded":"2024-04-26T23:56:30.325099+0000","last_fullsized":"2024-04-26T23:56:30.325099+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.349516+0000","last_change":"2024-04-26T23:56:30.348292+0000","last_active":"2024-04-26T23:56:30.349516+0000","last_peered":"2024-04-26T23:56:30.349516+0000","last_clean":"2024-04-26T23:56:30.349516+0000","last_became_active":"2024-04-26T23:56:30.347878+0000","last_became_peered":"2024-04-26T23:56:30.347878+0000","last_unstale":"2024-04-26T23:56:30.349516+0000","last_undegraded":"2024-04-26T23:56:30.349516+0000","last_fullsized":"2024-04-26T23:56:30.349516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:04.269891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.340034+0000","last_change":"2024-04-26T23:56:30.340034+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.340034+0000","last_undegraded":"2024-04-26T23:56:30.340034+0000","last_fullsized":"2024-04-26T23:56:30.340034+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.342700+0000","last_change":"2024-04-26T23:56:30.341651+0000","last_active":"2024-04-26T23:56:30.342700+0000","last_peered":"2024-04-26T23:56:30.342700+0000","last_clean":"2024-04-26T23:56:30.342700+0000","last_became_active":"2024-04-26T23:56:30.341271+0000","last_became_peered":"2024-04-26T23:56:30.341271+0000","last_unstale":"2024-04-26T23:56:30.342700+0000","last_undegraded":"2024-04-26T23:56:30.342700+0000","last_fullsized":"2024-04-26T23:56:30.342700+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:40.510631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.324595+0000","last_change":"2024-04-26T23:56:30.324595+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.324595+0000","last_undegraded":"2024-04-26T23:56:30.324595+0000","last_fullsized":"2024-04-26T23:56:30.324595+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360663+0000","last_change":"2024-04-26T23:56:30.358203+0000","last_active":"2024-04-26T23:56:30.360663+0000","last_peered":"2024-04-26T23:56:30.360663+0000","last_clean":"2024-04-26T23:56:30.360663+0000","last_became_active":"2024-04-26T23:56:30.357769+0000","last_became_peered":"2024-04-26T23:56:30.357769+0000","last_unstale":"2024-04-26T23:56:30.360663+0000","last_undegraded":"2024-04-26T23:56:30.360663+0000","last_fullsized":"2024-04-26T23:56:30.360663+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:20:10.288358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.329658+0000","last_change":"2024-04-26T23:56:30.329658+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.329658+0000","last_undegraded":"2024-04-26T23:56:30.329658+0000","last_fullsized":"2024-04-26T23:56:30.329658+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360931+0000","last_change":"2024-04-26T23:56:30.359102+0000","last_active":"2024-04-26T23:56:30.360931+0000","last_peered":"2024-04-26T23:56:30.360931+0000","last_clean":"2024-04-26T23:56:30.360931+0000","last_became_active":"2024-04-26T23:56:30.357370+0000","last_became_peered":"2024-04-26T23:56:30.357370+0000","last_unstale":"2024-04-26T23:56:30.360931+0000","last_undegraded":"2024-04-26T23:56:30.360931+0000","last_fullsized":"2024-04-26T23:56:30.360931+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:09.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.328823+0000","last_change":"2024-04-26T23:56:30.328823+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.328823+0000","last_undegraded":"2024-04-26T23:56:30.328823+0000","last_fullsized":"2024-04-26T23:56:30.328823+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.348903+0000","last_change":"2024-04-26T23:56:30.347666+0000","last_active":"2024-04-26T23:56:30.348903+0000","last_peered":"2024-04-26T23:56:30.348903+0000","last_clean":"2024-04-26T23:56:30.348903+0000","last_became_active":"2024-04-26T23:56:30.346147+0000","last_became_peered":"2024-04-26T23:56:30.346147+0000","last_unstale":"2024-04-26T23:56:30.348903+0000","last_undegraded":"2024-04-26T23:56:30.348903+0000","last_fullsized":"2024-04-26T23:56:30.348903+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:43.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.325582+0000","last_change":"2024-04-26T23:56:30.325582+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.325582+0000","last_undegraded":"2024-04-26T23:56:30.325582+0000","last_fullsized":"2024-04-26T23:56:30.325582+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.353521+0000","last_change":"2024-04-26T23:56:30.350199+0000","last_active":"2024-04-26T23:56:30.353521+0000","last_peered":"2024-04-26T23:56:30.353521+0000","last_clean":"2024-04-26T23:56:30.353521+0000","last_became_active":"2024-04-26T23:56:30.348661+0000","last_became_peered":"2024-04-26T23:56:30.348661+0000","last_unstale":"2024-04-26T23:56:30.353521+0000","last_undegraded":"2024-04-26T23:56:30.353521+0000","last_fullsized":"2024-04-26T23:56:30.353521+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:30.680322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.350719+0000","last_change":"2024-04-26T23:56:30.350719+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.350719+0000","last_undegraded":"2024-04-26T23:56:30.350719+0000","last_fullsized":"2024-04-26T23:56:30.350719+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.360875+0000","last_change":"2024-04-26T23:56:30.359350+0000","last_active":"2024-04-26T23:56:30.360875+0000","last_peered":"2024-04-26T23:56:30.360875+0000","last_clean":"2024-04-26T23:56:30.360875+0000","last_became_active":"2024-04-26T23:56:30.359057+0000","last_became_peered":"2024-04-26T23:56:30.359057+0000","last_unstale":"2024-04-26T23:56:30.360875+0000","last_undegraded":"2024-04-26T23:56:30.360875+0000","last_fullsized":"2024-04-26T23:56:30.360875+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:04.488356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":197,"state":"creating+peering","last_fresh":"2024-04-26T23:56:30.332821+0000","last_change":"2024-04-26T23:56:30.332821+0000","last_active":"2024-04-26T23:56:30.309570+0000","last_peered":"2024-04-26T23:56:30.309570+0000","last_clean":"2024-04-26T23:56:30.309570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:56:30.332821+0000","last_undegraded":"2024-04-26T23:56:30.332821+0000","last_fullsized":"2024-04-26T23:56:30.332821+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.347388+0000","last_change":"2024-04-26T23:56:30.345798+0000","last_active":"2024-04-26T23:56:30.347388+0000","last_peered":"2024-04-26T23:56:30.347388+0000","last_clean":"2024-04-26T23:56:30.347388+0000","last_became_active":"2024-04-26T23:56:30.345187+0000","last_became_peered":"2024-04-26T23:56:30.345187+0000","last_unstale":"2024-04-26T23:56:30.347388+0000","last_undegraded":"2024-04-26T23:56:30.347388+0000","last_fullsized":"2024-04-26T23:56:30.347388+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:34.679261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":499,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-26T23:56:30.330008+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:56:30.330008+0000","last_peered":"2024-04-26T23:56:30.330008+0000","last_clean":"2024-04-26T23:56:30.330008+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:56:30.330008+0000","last_undegraded":"2024-04-26T23:56:30.330008+0000","last_fullsized":"2024-04-26T23:56:30.330008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575084,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52244,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319596,"statfs":{"total":96636764160,"available":96583266304,"internally_reserved":0,"allocated":3084288,"data_stored":1766496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55824,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316016,"statfs":{"total":96636764160,"available":96579600384,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575085,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55828,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316012,"statfs":{"total":96636764160,"available":96579596288,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575085,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47504,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324336,"statfs":{"total":96636764160,"available":96588120064,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575085,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47500,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324340,"statfs":{"total":96636764160,"available":96588124160,"internally_reserved":0,"allocated":2490368,"data_stored":1176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575085,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60560,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311280,"statfs":{"total":96636764160,"available":96574750720,"internally_reserved":0,"allocated":3084288,"data_stored":1766496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:56:35.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:56:36.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.110+0000 7f85e9c00640 1 Processor -- start 2024-04-26T23:56:36.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.111+0000 7f85e9c00640 1 -- start start 2024-04-26T23:56:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.112+0000 7f85e9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85e40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.112+0000 7f85e9c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e40d11f0 con 0x7f85e415f4e0 2024-04-26T23:56:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.112+0000 7f85e9c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e40d5f20 con 0x7f85e40b4000 2024-04-26T23:56:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.112+0000 7f85e9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85e40d0ce0 con 0x7f85e40d2130 2024-04-26T23:56:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.113+0000 7f85e3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f85e415f4e0 0x7f85e40d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59620/0 (socket says 172.21.15.46:59620) 2024-04-26T23:56:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.113+0000 7f85e3e00640 1 -- 172.21.15.46:0/3474081374 learned_addr learned my addr 172.21.15.46:0/3474081374 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.113+0000 7f85e3400640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85e40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.113+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377518965 0 0) 0x7f85e40d11f0 con 0x7f85e415f4e0 2024-04-26T23:56:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.113+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85c4003520 con 0x7f85e415f4e0 2024-04-26T23:56:36.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4155906185 0 0) 0x7f85c4003520 con 0x7f85e415f4e0 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 msgr2=0x7f85e40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85e40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 >> v1:172.21.15.142:6789/0 conn(0x7f85e40b4000 legacy=0x7f85e40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40d0f10 con 0x7f85e415f4e0 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f85d4001670 con 0x7f85e415f4e0 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 >> v1:172.21.15.46:6789/0 conn(0x7f85e415f4e0 legacy=0x7f85e40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c4004d40 0x7f85c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400ba70 con 0x7f85c40039e0 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400bbf0 con 0x7f85c4004d40 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85c400bda0 con 0x7f85e40d2130 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.114+0000 7f85e3e00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3400640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e9200640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c4004d40 0x7f85c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3e00640 1 -- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 msgr2=0x7f85c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3e00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3e00640 1 -- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c4004d40 msgr2=0x7f85c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3e00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c4004d40 0x7f85c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3e00640 1 -- 172.21.15.46:0/3474081374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c4003520 con 0x7f85c40039e0 2024-04-26T23:56:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.115+0000 7f85e3400640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e40d2130 0x7f85c4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.116+0000 7f85e3e00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85c4003dc0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c110 tx=0x7f85c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f264c3efa167fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.116+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007c40 con 0x7f85c40039e0 2024-04-26T23:56:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.116+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4005ad0 con 0x7f85c40039e0 2024-04-26T23:56:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.116+0000 7f85e2a00640 1 -- 172.21.15.46:0/3474081374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007070 con 0x7f85c40039e0 2024-04-26T23:56:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 -- 172.21.15.46:0/3474081374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 msgr2=0x7f85c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 --2- 172.21.15.46:0/3474081374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85c4003dc0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c110 tx=0x7f85c4005830 comp rx=0 tx=0).stop 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9200640 1 -- 172.21.15.46:0/3474081374 reap_dead start 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 -- 172.21.15.46:0/3474081374 shutdown_connections 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 -- 172.21.15.46:0/3474081374 >> 172.21.15.46:0/3474081374 conn(0x7f85e40d5010 msgr2=0x7f85e40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 -- 172.21.15.46:0/3474081374 shutdown_connections 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.117+0000 7f85e9c00640 1 -- 172.21.15.46:0/3474081374 wait complete. 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.119+0000 7f85e9c00640 1 Processor -- start 2024-04-26T23:56:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.119+0000 7f85e9c00640 1 -- start start 2024-04-26T23:56:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e41fa070 0x7f85e40cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 0x7f85e409f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85e40b1d60 con 0x7f85c40039e0 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85e40d2990 con 0x7f85e409ec60 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f85e415f8c0 con 0x7f85e41fa070 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48108/0 (socket says 172.21.15.46:48108) 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.120+0000 7f85e3e00640 1 -- 172.21.15.46:0/2957043131 learned_addr learned my addr 172.21.15.46:0/2957043131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e9200640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 0x7f85e409f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3400640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e41fa070 0x7f85e40cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 -- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e41fa070 msgr2=0x7f85e40cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e41fa070 0x7f85e40cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 -- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 msgr2=0x7f85e409f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 0x7f85e409f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 -- 172.21.15.46:0/2957043131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40cd980 con 0x7f85c40039e0 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e9200640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 0x7f85e409f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e3e00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c230 tx=0x7f85e40d10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4041020 con 0x7f85c40039e0 2024-04-26T23:56:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4001430 con 0x7f85c40039e0 2024-04-26T23:56:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007070 con 0x7f85c40039e0 2024-04-26T23:56:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.121+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e40cdc60 con 0x7f85c40039e0 2024-04-26T23:56:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.122+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e40ce220 con 0x7f85c40039e0 2024-04-26T23:56:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.123+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e40693b0 con 0x7f85c40039e0 2024-04-26T23:56:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.124+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85d4044020 con 0x7f85c40039e0 2024-04-26T23:56:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.125+0000 7f85dbe00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 0x7f85c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.125+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f85d40ac010 con 0x7f85c40039e0 2024-04-26T23:56:36.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.125+0000 7f85e3400640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 0x7f85c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.126+0000 7f85e3400640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 0x7f85c006ede0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f85e40b3980 tx=0x7f85cc00d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.129+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85d407a3e0 con 0x7f85c40039e0 2024-04-26T23:56:36.290 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-26T23:56:36.287+0000 7f417c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:56:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.330+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85e40d33d0 con 0x7f85c006c920 2024-04-26T23:56:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.348+0000 7f85dbe00640 1 -- 172.21.15.46:0/2957043131 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252489 (secure 0 0 0) 0x7f85e40d33d0 con 0x7f85c006c920 2024-04-26T23:56:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 msgr2=0x7f85c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 0x7f85c006ede0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f85e40b3980 tx=0x7f85cc00d040 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 msgr2=0x7f85e41f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c230 tx=0x7f85e40d10e0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 shutdown_connections 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f85c006c920 0x7f85c006ede0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e409ec60 0x7f85e409f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85e41fa070 0x7f85e40cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 --2- 172.21.15.46:0/2957043131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c40039e0 0x7f85e41f94f0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.354+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 >> 172.21.15.46:0/2957043131 conn(0x7f85e40d5010 msgr2=0x7f85e4160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.355+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 shutdown_connections 2024-04-26T23:56:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.355+0000 7f85e9c00640 1 -- 172.21.15.46:0/2957043131 wait complete. 2024-04-26T23:56:36.377 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":780,"stamp":"2024-04-26T23:56:36.328535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443860,"kb_used_data":20872,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754530860,"statfs":{"total":773094113280,"available":772639600640,"internally_reserved":0,"allocated":21372928,"data_stored":10705452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":432983601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.009221"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607581+0000","last_change":"2024-04-26T23:56:31.354314+0000","last_active":"2024-04-26T23:56:32.607581+0000","last_peered":"2024-04-26T23:56:32.607581+0000","last_clean":"2024-04-26T23:56:32.607581+0000","last_became_active":"2024-04-26T23:56:31.353590+0000","last_became_peered":"2024-04-26T23:56:31.353590+0000","last_unstale":"2024-04-26T23:56:32.607581+0000","last_undegraded":"2024-04-26T23:56:32.607581+0000","last_fullsized":"2024-04-26T23:56:32.607581+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:00.606021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326490+0000","last_change":"2024-04-26T23:56:31.359164+0000","last_active":"2024-04-26T23:56:32.326490+0000","last_peered":"2024-04-26T23:56:32.326490+0000","last_clean":"2024-04-26T23:56:32.326490+0000","last_became_active":"2024-04-26T23:56:31.358763+0000","last_became_peered":"2024-04-26T23:56:31.358763+0000","last_unstale":"2024-04-26T23:56:32.326490+0000","last_undegraded":"2024-04-26T23:56:32.326490+0000","last_fullsized":"2024-04-26T23:56:32.326490+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:13.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395205+0000","last_change":"2024-04-26T23:56:31.359784+0000","last_active":"2024-04-26T23:56:32.395205+0000","last_peered":"2024-04-26T23:56:32.395205+0000","last_clean":"2024-04-26T23:56:32.395205+0000","last_became_active":"2024-04-26T23:56:31.359488+0000","last_became_peered":"2024-04-26T23:56:31.359488+0000","last_unstale":"2024-04-26T23:56:32.395205+0000","last_undegraded":"2024-04-26T23:56:32.395205+0000","last_fullsized":"2024-04-26T23:56:32.395205+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:52.482823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.331776+0000","last_change":"2024-04-26T23:56:31.353151+0000","last_active":"2024-04-26T23:56:32.331776+0000","last_peered":"2024-04-26T23:56:32.331776+0000","last_clean":"2024-04-26T23:56:32.331776+0000","last_became_active":"2024-04-26T23:56:31.352552+0000","last_became_peered":"2024-04-26T23:56:31.352552+0000","last_unstale":"2024-04-26T23:56:32.331776+0000","last_undegraded":"2024-04-26T23:56:32.331776+0000","last_fullsized":"2024-04-26T23:56:32.331776+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:14.564159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605994+0000","last_change":"2024-04-26T23:56:31.359319+0000","last_active":"2024-04-26T23:56:32.605994+0000","last_peered":"2024-04-26T23:56:32.605994+0000","last_clean":"2024-04-26T23:56:32.605994+0000","last_became_active":"2024-04-26T23:56:31.359015+0000","last_became_peered":"2024-04-26T23:56:31.359015+0000","last_unstale":"2024-04-26T23:56:32.605994+0000","last_undegraded":"2024-04-26T23:56:32.605994+0000","last_fullsized":"2024-04-26T23:56:32.605994+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:19.956078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605500+0000","last_change":"2024-04-26T23:56:31.354420+0000","last_active":"2024-04-26T23:56:32.605500+0000","last_peered":"2024-04-26T23:56:32.605500+0000","last_clean":"2024-04-26T23:56:32.605500+0000","last_became_active":"2024-04-26T23:56:31.354096+0000","last_became_peered":"2024-04-26T23:56:31.354096+0000","last_unstale":"2024-04-26T23:56:32.605500+0000","last_undegraded":"2024-04-26T23:56:32.605500+0000","last_fullsized":"2024-04-26T23:56:32.605500+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:03.437385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394735+0000","last_change":"2024-04-26T23:56:31.365697+0000","last_active":"2024-04-26T23:56:32.394735+0000","last_peered":"2024-04-26T23:56:32.394735+0000","last_clean":"2024-04-26T23:56:32.394735+0000","last_became_active":"2024-04-26T23:56:31.363072+0000","last_became_peered":"2024-04-26T23:56:31.363072+0000","last_unstale":"2024-04-26T23:56:32.394735+0000","last_undegraded":"2024-04-26T23:56:32.394735+0000","last_fullsized":"2024-04-26T23:56:32.394735+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:01.641476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.603761+0000","last_change":"2024-04-26T23:56:31.352194+0000","last_active":"2024-04-26T23:56:32.603761+0000","last_peered":"2024-04-26T23:56:32.603761+0000","last_clean":"2024-04-26T23:56:32.603761+0000","last_became_active":"2024-04-26T23:56:31.348581+0000","last_became_peered":"2024-04-26T23:56:31.348581+0000","last_unstale":"2024-04-26T23:56:32.603761+0000","last_undegraded":"2024-04-26T23:56:32.603761+0000","last_fullsized":"2024-04-26T23:56:32.603761+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:34.488001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604551+0000","last_change":"2024-04-26T23:56:31.355453+0000","last_active":"2024-04-26T23:56:32.604551+0000","last_peered":"2024-04-26T23:56:32.604551+0000","last_clean":"2024-04-26T23:56:32.604551+0000","last_became_active":"2024-04-26T23:56:31.355122+0000","last_became_peered":"2024-04-26T23:56:31.355122+0000","last_unstale":"2024-04-26T23:56:32.604551+0000","last_undegraded":"2024-04-26T23:56:32.604551+0000","last_fullsized":"2024-04-26T23:56:32.604551+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:44.024459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395155+0000","last_change":"2024-04-26T23:56:31.365538+0000","last_active":"2024-04-26T23:56:32.395155+0000","last_peered":"2024-04-26T23:56:32.395155+0000","last_clean":"2024-04-26T23:56:32.395155+0000","last_became_active":"2024-04-26T23:56:31.364719+0000","last_became_peered":"2024-04-26T23:56:31.364719+0000","last_unstale":"2024-04-26T23:56:32.395155+0000","last_undegraded":"2024-04-26T23:56:32.395155+0000","last_fullsized":"2024-04-26T23:56:32.395155+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.230253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.332023+0000","last_change":"2024-04-26T23:56:31.361835+0000","last_active":"2024-04-26T23:56:32.332023+0000","last_peered":"2024-04-26T23:56:32.332023+0000","last_clean":"2024-04-26T23:56:32.332023+0000","last_became_active":"2024-04-26T23:56:31.361412+0000","last_became_peered":"2024-04-26T23:56:31.361412+0000","last_unstale":"2024-04-26T23:56:32.332023+0000","last_undegraded":"2024-04-26T23:56:32.332023+0000","last_fullsized":"2024-04-26T23:56:32.332023+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:06.320284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607667+0000","last_change":"2024-04-26T23:56:31.362113+0000","last_active":"2024-04-26T23:56:32.607667+0000","last_peered":"2024-04-26T23:56:32.607667+0000","last_clean":"2024-04-26T23:56:32.607667+0000","last_became_active":"2024-04-26T23:56:31.361760+0000","last_became_peered":"2024-04-26T23:56:31.361760+0000","last_unstale":"2024-04-26T23:56:32.607667+0000","last_undegraded":"2024-04-26T23:56:32.607667+0000","last_fullsized":"2024-04-26T23:56:32.607667+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:40.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604294+0000","last_change":"2024-04-26T23:56:31.349622+0000","last_active":"2024-04-26T23:56:32.604294+0000","last_peered":"2024-04-26T23:56:32.604294+0000","last_clean":"2024-04-26T23:56:32.604294+0000","last_became_active":"2024-04-26T23:56:31.349307+0000","last_became_peered":"2024-04-26T23:56:31.349307+0000","last_unstale":"2024-04-26T23:56:32.604294+0000","last_undegraded":"2024-04-26T23:56:32.604294+0000","last_fullsized":"2024-04-26T23:56:32.604294+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:52.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607146+0000","last_change":"2024-04-26T23:56:31.361374+0000","last_active":"2024-04-26T23:56:32.607146+0000","last_peered":"2024-04-26T23:56:32.607146+0000","last_clean":"2024-04-26T23:56:32.607146+0000","last_became_active":"2024-04-26T23:56:31.355997+0000","last_became_peered":"2024-04-26T23:56:31.355997+0000","last_unstale":"2024-04-26T23:56:32.607146+0000","last_undegraded":"2024-04-26T23:56:32.607146+0000","last_fullsized":"2024-04-26T23:56:32.607146+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:05.773137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328394+0000","last_change":"2024-04-26T23:56:31.359897+0000","last_active":"2024-04-26T23:56:32.328394+0000","last_peered":"2024-04-26T23:56:32.328394+0000","last_clean":"2024-04-26T23:56:32.328394+0000","last_became_active":"2024-04-26T23:56:31.357267+0000","last_became_peered":"2024-04-26T23:56:31.357267+0000","last_unstale":"2024-04-26T23:56:32.328394+0000","last_undegraded":"2024-04-26T23:56:32.328394+0000","last_fullsized":"2024-04-26T23:56:32.328394+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:53.185959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604179+0000","last_change":"2024-04-26T23:56:31.346583+0000","last_active":"2024-04-26T23:56:32.604179+0000","last_peered":"2024-04-26T23:56:32.604179+0000","last_clean":"2024-04-26T23:56:32.604179+0000","last_became_active":"2024-04-26T23:56:31.346151+0000","last_became_peered":"2024-04-26T23:56:31.346151+0000","last_unstale":"2024-04-26T23:56:32.604179+0000","last_undegraded":"2024-04-26T23:56:32.604179+0000","last_fullsized":"2024-04-26T23:56:32.604179+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:11.114153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608508+0000","last_change":"2024-04-26T23:56:31.362041+0000","last_active":"2024-04-26T23:56:32.608508+0000","last_peered":"2024-04-26T23:56:32.608508+0000","last_clean":"2024-04-26T23:56:32.608508+0000","last_became_active":"2024-04-26T23:56:31.358089+0000","last_became_peered":"2024-04-26T23:56:31.358089+0000","last_unstale":"2024-04-26T23:56:32.608508+0000","last_undegraded":"2024-04-26T23:56:32.608508+0000","last_fullsized":"2024-04-26T23:56:32.608508+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:38:23.760074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395266+0000","last_change":"2024-04-26T23:56:31.363171+0000","last_active":"2024-04-26T23:56:32.395266+0000","last_peered":"2024-04-26T23:56:32.395266+0000","last_clean":"2024-04-26T23:56:32.395266+0000","last_became_active":"2024-04-26T23:56:31.362632+0000","last_became_peered":"2024-04-26T23:56:31.362632+0000","last_unstale":"2024-04-26T23:56:32.395266+0000","last_undegraded":"2024-04-26T23:56:32.395266+0000","last_fullsized":"2024-04-26T23:56:32.395266+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:45.559197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606394+0000","last_change":"2024-04-26T23:56:31.359214+0000","last_active":"2024-04-26T23:56:32.606394+0000","last_peered":"2024-04-26T23:56:32.606394+0000","last_clean":"2024-04-26T23:56:32.606394+0000","last_became_active":"2024-04-26T23:56:31.358462+0000","last_became_peered":"2024-04-26T23:56:31.358462+0000","last_unstale":"2024-04-26T23:56:32.606394+0000","last_undegraded":"2024-04-26T23:56:32.606394+0000","last_fullsized":"2024-04-26T23:56:32.606394+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:36.852873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606849+0000","last_change":"2024-04-26T23:56:31.357962+0000","last_active":"2024-04-26T23:56:32.606849+0000","last_peered":"2024-04-26T23:56:32.606849+0000","last_clean":"2024-04-26T23:56:32.606849+0000","last_became_active":"2024-04-26T23:56:31.357647+0000","last_became_peered":"2024-04-26T23:56:31.357647+0000","last_unstale":"2024-04-26T23:56:32.606849+0000","last_undegraded":"2024-04-26T23:56:32.606849+0000","last_fullsized":"2024-04-26T23:56:32.606849+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.251860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394843+0000","last_change":"2024-04-26T23:56:31.355989+0000","last_active":"2024-04-26T23:56:32.394843+0000","last_peered":"2024-04-26T23:56:32.394843+0000","last_clean":"2024-04-26T23:56:32.394843+0000","last_became_active":"2024-04-26T23:56:31.355645+0000","last_became_peered":"2024-04-26T23:56:31.355645+0000","last_unstale":"2024-04-26T23:56:32.394843+0000","last_undegraded":"2024-04-26T23:56:32.394843+0000","last_fullsized":"2024-04-26T23:56:32.394843+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:40.029766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328416+0000","last_change":"2024-04-26T23:56:31.365645+0000","last_active":"2024-04-26T23:56:32.328416+0000","last_peered":"2024-04-26T23:56:32.328416+0000","last_clean":"2024-04-26T23:56:32.328416+0000","last_became_active":"2024-04-26T23:56:31.365276+0000","last_became_peered":"2024-04-26T23:56:31.365276+0000","last_unstale":"2024-04-26T23:56:32.328416+0000","last_undegraded":"2024-04-26T23:56:32.328416+0000","last_fullsized":"2024-04-26T23:56:32.328416+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:07.938823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604121+0000","last_change":"2024-04-26T23:56:31.357655+0000","last_active":"2024-04-26T23:56:32.604121+0000","last_peered":"2024-04-26T23:56:32.604121+0000","last_clean":"2024-04-26T23:56:32.604121+0000","last_became_active":"2024-04-26T23:56:31.357271+0000","last_became_peered":"2024-04-26T23:56:31.357271+0000","last_unstale":"2024-04-26T23:56:32.604121+0000","last_undegraded":"2024-04-26T23:56:32.604121+0000","last_fullsized":"2024-04-26T23:56:32.604121+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:00.612255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604449+0000","last_change":"2024-04-26T23:56:31.350070+0000","last_active":"2024-04-26T23:56:32.604449+0000","last_peered":"2024-04-26T23:56:32.604449+0000","last_clean":"2024-04-26T23:56:32.604449+0000","last_became_active":"2024-04-26T23:56:31.349785+0000","last_became_peered":"2024-04-26T23:56:31.349785+0000","last_unstale":"2024-04-26T23:56:32.604449+0000","last_undegraded":"2024-04-26T23:56:32.604449+0000","last_fullsized":"2024-04-26T23:56:32.604449+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:43.402370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607632+0000","last_change":"2024-04-26T23:56:31.359791+0000","last_active":"2024-04-26T23:56:32.607632+0000","last_peered":"2024-04-26T23:56:32.607632+0000","last_clean":"2024-04-26T23:56:32.607632+0000","last_became_active":"2024-04-26T23:56:31.359496+0000","last_became_peered":"2024-04-26T23:56:31.359496+0000","last_unstale":"2024-04-26T23:56:32.607632+0000","last_undegraded":"2024-04-26T23:56:32.607632+0000","last_fullsized":"2024-04-26T23:56:32.607632+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:40.246270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395389+0000","last_change":"2024-04-26T23:56:31.362909+0000","last_active":"2024-04-26T23:56:32.395389+0000","last_peered":"2024-04-26T23:56:32.395389+0000","last_clean":"2024-04-26T23:56:32.395389+0000","last_became_active":"2024-04-26T23:56:31.362220+0000","last_became_peered":"2024-04-26T23:56:31.362220+0000","last_unstale":"2024-04-26T23:56:32.395389+0000","last_undegraded":"2024-04-26T23:56:32.395389+0000","last_fullsized":"2024-04-26T23:56:32.395389+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:02.419522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.331907+0000","last_change":"2024-04-26T23:56:31.358851+0000","last_active":"2024-04-26T23:56:32.331907+0000","last_peered":"2024-04-26T23:56:32.331907+0000","last_clean":"2024-04-26T23:56:32.331907+0000","last_became_active":"2024-04-26T23:56:31.358462+0000","last_became_peered":"2024-04-26T23:56:31.358462+0000","last_unstale":"2024-04-26T23:56:32.331907+0000","last_undegraded":"2024-04-26T23:56:32.331907+0000","last_fullsized":"2024-04-26T23:56:32.331907+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:52.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608297+0000","last_change":"2024-04-26T23:56:31.351740+0000","last_active":"2024-04-26T23:56:32.608297+0000","last_peered":"2024-04-26T23:56:32.608297+0000","last_clean":"2024-04-26T23:56:32.608297+0000","last_became_active":"2024-04-26T23:56:31.351371+0000","last_became_peered":"2024-04-26T23:56:31.351371+0000","last_unstale":"2024-04-26T23:56:32.608297+0000","last_undegraded":"2024-04-26T23:56:32.608297+0000","last_fullsized":"2024-04-26T23:56:32.608297+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:34.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326709+0000","last_change":"2024-04-26T23:56:31.354662+0000","last_active":"2024-04-26T23:56:32.326709+0000","last_peered":"2024-04-26T23:56:32.326709+0000","last_clean":"2024-04-26T23:56:32.326709+0000","last_became_active":"2024-04-26T23:56:31.351233+0000","last_became_peered":"2024-04-26T23:56:31.351233+0000","last_unstale":"2024-04-26T23:56:32.326709+0000","last_undegraded":"2024-04-26T23:56:32.326709+0000","last_fullsized":"2024-04-26T23:56:32.326709+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:36.852873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604791+0000","last_change":"2024-04-26T23:56:31.362027+0000","last_active":"2024-04-26T23:56:32.604791+0000","last_peered":"2024-04-26T23:56:32.604791+0000","last_clean":"2024-04-26T23:56:32.604791+0000","last_became_active":"2024-04-26T23:56:31.361580+0000","last_became_peered":"2024-04-26T23:56:31.361580+0000","last_unstale":"2024-04-26T23:56:32.604791+0000","last_undegraded":"2024-04-26T23:56:32.604791+0000","last_fullsized":"2024-04-26T23:56:32.604791+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:53.185959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.603683+0000","last_change":"2024-04-26T23:56:31.356123+0000","last_active":"2024-04-26T23:56:32.603683+0000","last_peered":"2024-04-26T23:56:32.603683+0000","last_clean":"2024-04-26T23:56:32.603683+0000","last_became_active":"2024-04-26T23:56:31.353970+0000","last_became_peered":"2024-04-26T23:56:31.353970+0000","last_unstale":"2024-04-26T23:56:32.603683+0000","last_undegraded":"2024-04-26T23:56:32.603683+0000","last_fullsized":"2024-04-26T23:56:32.603683+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:05.503684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604552+0000","last_change":"2024-04-26T23:56:31.361112+0000","last_active":"2024-04-26T23:56:32.604552+0000","last_peered":"2024-04-26T23:56:32.604552+0000","last_clean":"2024-04-26T23:56:32.604552+0000","last_became_active":"2024-04-26T23:56:31.360830+0000","last_became_peered":"2024-04-26T23:56:31.360830+0000","last_unstale":"2024-04-26T23:56:32.604552+0000","last_undegraded":"2024-04-26T23:56:32.604552+0000","last_fullsized":"2024-04-26T23:56:32.604552+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:44.254388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394881+0000","last_change":"2024-04-26T23:56:31.361576+0000","last_active":"2024-04-26T23:56:32.394881+0000","last_peered":"2024-04-26T23:56:32.394881+0000","last_clean":"2024-04-26T23:56:32.394881+0000","last_became_active":"2024-04-26T23:56:31.361185+0000","last_became_peered":"2024-04-26T23:56:31.361185+0000","last_unstale":"2024-04-26T23:56:32.394881+0000","last_undegraded":"2024-04-26T23:56:32.394881+0000","last_fullsized":"2024-04-26T23:56:32.394881+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:47.583746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"198'4","reported_seq":18,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604583+0000","last_change":"2024-04-26T23:56:30.335217+0000","last_active":"2024-04-26T23:56:32.604583+0000","last_peered":"2024-04-26T23:56:32.604583+0000","last_clean":"2024-04-26T23:56:32.604583+0000","last_became_active":"2024-04-26T23:56:30.334736+0000","last_became_peered":"2024-04-26T23:56:30.334736+0000","last_unstale":"2024-04-26T23:56:32.604583+0000","last_undegraded":"2024-04-26T23:56:32.604583+0000","last_fullsized":"2024-04-26T23:56:32.604583+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.081232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395201+0000","last_change":"2024-04-26T23:56:31.359131+0000","last_active":"2024-04-26T23:56:32.395201+0000","last_peered":"2024-04-26T23:56:32.395201+0000","last_clean":"2024-04-26T23:56:32.395201+0000","last_became_active":"2024-04-26T23:56:31.358490+0000","last_became_peered":"2024-04-26T23:56:31.358490+0000","last_unstale":"2024-04-26T23:56:32.395201+0000","last_undegraded":"2024-04-26T23:56:32.395201+0000","last_fullsized":"2024-04-26T23:56:32.395201+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:08.407437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608526+0000","last_change":"2024-04-26T23:56:30.358279+0000","last_active":"2024-04-26T23:56:32.608526+0000","last_peered":"2024-04-26T23:56:32.608526+0000","last_clean":"2024-04-26T23:56:32.608526+0000","last_became_active":"2024-04-26T23:56:30.357695+0000","last_became_peered":"2024-04-26T23:56:30.357695+0000","last_unstale":"2024-04-26T23:56:32.608526+0000","last_undegraded":"2024-04-26T23:56:32.608526+0000","last_fullsized":"2024-04-26T23:56:32.608526+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:12.328732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608564+0000","last_change":"2024-04-26T23:56:31.356728+0000","last_active":"2024-04-26T23:56:32.608564+0000","last_peered":"2024-04-26T23:56:32.608564+0000","last_clean":"2024-04-26T23:56:32.608564+0000","last_became_active":"2024-04-26T23:56:31.355851+0000","last_became_peered":"2024-04-26T23:56:31.355851+0000","last_unstale":"2024-04-26T23:56:32.608564+0000","last_undegraded":"2024-04-26T23:56:32.608564+0000","last_fullsized":"2024-04-26T23:56:32.608564+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:15.838115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605137+0000","last_change":"2024-04-26T23:56:30.361728+0000","last_active":"2024-04-26T23:56:32.605137+0000","last_peered":"2024-04-26T23:56:32.605137+0000","last_clean":"2024-04-26T23:56:32.605137+0000","last_became_active":"2024-04-26T23:56:30.361416+0000","last_became_peered":"2024-04-26T23:56:30.361416+0000","last_unstale":"2024-04-26T23:56:32.605137+0000","last_undegraded":"2024-04-26T23:56:32.605137+0000","last_fullsized":"2024-04-26T23:56:32.605137+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:04.269891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394392+0000","last_change":"2024-04-26T23:56:31.350352+0000","last_active":"2024-04-26T23:56:32.394392+0000","last_peered":"2024-04-26T23:56:32.394392+0000","last_clean":"2024-04-26T23:56:32.394392+0000","last_became_active":"2024-04-26T23:56:31.350008+0000","last_became_peered":"2024-04-26T23:56:31.350008+0000","last_unstale":"2024-04-26T23:56:32.394392+0000","last_undegraded":"2024-04-26T23:56:32.394392+0000","last_fullsized":"2024-04-26T23:56:32.394392+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:11.142082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328577+0000","last_change":"2024-04-26T23:56:30.357016+0000","last_active":"2024-04-26T23:56:32.328577+0000","last_peered":"2024-04-26T23:56:32.328577+0000","last_clean":"2024-04-26T23:56:32.328577+0000","last_became_active":"2024-04-26T23:56:30.356716+0000","last_became_peered":"2024-04-26T23:56:30.356716+0000","last_unstale":"2024-04-26T23:56:32.328577+0000","last_undegraded":"2024-04-26T23:56:32.328577+0000","last_fullsized":"2024-04-26T23:56:32.328577+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:04.615985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605999+0000","last_change":"2024-04-26T23:56:31.353792+0000","last_active":"2024-04-26T23:56:32.605999+0000","last_peered":"2024-04-26T23:56:32.605999+0000","last_clean":"2024-04-26T23:56:32.605999+0000","last_became_active":"2024-04-26T23:56:31.353185+0000","last_became_peered":"2024-04-26T23:56:31.353185+0000","last_unstale":"2024-04-26T23:56:32.605999+0000","last_undegraded":"2024-04-26T23:56:32.605999+0000","last_fullsized":"2024-04-26T23:56:32.605999+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:45.257776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604147+0000","last_change":"2024-04-26T23:56:30.358799+0000","last_active":"2024-04-26T23:56:32.604147+0000","last_peered":"2024-04-26T23:56:32.604147+0000","last_clean":"2024-04-26T23:56:32.604147+0000","last_became_active":"2024-04-26T23:56:30.358506+0000","last_became_peered":"2024-04-26T23:56:30.358506+0000","last_unstale":"2024-04-26T23:56:32.604147+0000","last_undegraded":"2024-04-26T23:56:32.604147+0000","last_fullsized":"2024-04-26T23:56:32.604147+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:08.867444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328443+0000","last_change":"2024-04-26T23:56:31.360165+0000","last_active":"2024-04-26T23:56:32.328443+0000","last_peered":"2024-04-26T23:56:32.328443+0000","last_clean":"2024-04-26T23:56:32.328443+0000","last_became_active":"2024-04-26T23:56:31.357712+0000","last_became_peered":"2024-04-26T23:56:31.357712+0000","last_unstale":"2024-04-26T23:56:32.328443+0000","last_undegraded":"2024-04-26T23:56:32.328443+0000","last_fullsized":"2024-04-26T23:56:32.328443+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:03.437385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.330940+0000","last_change":"2024-04-26T23:56:30.354074+0000","last_active":"2024-04-26T23:56:32.330940+0000","last_peered":"2024-04-26T23:56:32.330940+0000","last_clean":"2024-04-26T23:56:32.330940+0000","last_became_active":"2024-04-26T23:56:30.353806+0000","last_became_peered":"2024-04-26T23:56:30.353806+0000","last_unstale":"2024-04-26T23:56:32.330940+0000","last_undegraded":"2024-04-26T23:56:32.330940+0000","last_fullsized":"2024-04-26T23:56:32.330940+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:04.668223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604330+0000","last_change":"2024-04-26T23:56:31.347959+0000","last_active":"2024-04-26T23:56:32.604330+0000","last_peered":"2024-04-26T23:56:32.604330+0000","last_clean":"2024-04-26T23:56:32.604330+0000","last_became_active":"2024-04-26T23:56:31.347413+0000","last_became_peered":"2024-04-26T23:56:31.347413+0000","last_unstale":"2024-04-26T23:56:32.604330+0000","last_undegraded":"2024-04-26T23:56:32.604330+0000","last_fullsized":"2024-04-26T23:56:32.604330+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:48.594773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394555+0000","last_change":"2024-04-26T23:56:30.341747+0000","last_active":"2024-04-26T23:56:32.394555+0000","last_peered":"2024-04-26T23:56:32.394555+0000","last_clean":"2024-04-26T23:56:32.394555+0000","last_became_active":"2024-04-26T23:56:30.341227+0000","last_became_peered":"2024-04-26T23:56:30.341227+0000","last_unstale":"2024-04-26T23:56:32.394555+0000","last_undegraded":"2024-04-26T23:56:32.394555+0000","last_fullsized":"2024-04-26T23:56:32.394555+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:59.590693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328837+0000","last_change":"2024-04-26T23:56:31.355396+0000","last_active":"2024-04-26T23:56:32.328837+0000","last_peered":"2024-04-26T23:56:32.328837+0000","last_clean":"2024-04-26T23:56:32.328837+0000","last_became_active":"2024-04-26T23:56:31.354129+0000","last_became_peered":"2024-04-26T23:56:31.354129+0000","last_unstale":"2024-04-26T23:56:32.328837+0000","last_undegraded":"2024-04-26T23:56:32.328837+0000","last_fullsized":"2024-04-26T23:56:32.328837+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:43.402370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607856+0000","last_change":"2024-04-26T23:56:30.354063+0000","last_active":"2024-04-26T23:56:32.607856+0000","last_peered":"2024-04-26T23:56:32.607856+0000","last_clean":"2024-04-26T23:56:32.607856+0000","last_became_active":"2024-04-26T23:56:30.353776+0000","last_became_peered":"2024-04-26T23:56:30.353776+0000","last_unstale":"2024-04-26T23:56:32.607856+0000","last_undegraded":"2024-04-26T23:56:32.607856+0000","last_fullsized":"2024-04-26T23:56:32.607856+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:25:40.626699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328968+0000","last_change":"2024-04-26T23:56:31.356868+0000","last_active":"2024-04-26T23:56:32.328968+0000","last_peered":"2024-04-26T23:56:32.328968+0000","last_clean":"2024-04-26T23:56:32.328968+0000","last_became_active":"2024-04-26T23:56:31.354215+0000","last_became_peered":"2024-04-26T23:56:31.354215+0000","last_unstale":"2024-04-26T23:56:32.328968+0000","last_undegraded":"2024-04-26T23:56:32.328968+0000","last_fullsized":"2024-04-26T23:56:32.328968+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:13.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.332132+0000","last_change":"2024-04-26T23:56:30.356537+0000","last_active":"2024-04-26T23:56:32.332132+0000","last_peered":"2024-04-26T23:56:32.332132+0000","last_clean":"2024-04-26T23:56:32.332132+0000","last_became_active":"2024-04-26T23:56:30.356232+0000","last_became_peered":"2024-04-26T23:56:30.356232+0000","last_unstale":"2024-04-26T23:56:32.332132+0000","last_undegraded":"2024-04-26T23:56:32.332132+0000","last_fullsized":"2024-04-26T23:56:32.332132+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:23.462873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395462+0000","last_change":"2024-04-26T23:56:31.356260+0000","last_active":"2024-04-26T23:56:32.395462+0000","last_peered":"2024-04-26T23:56:32.395462+0000","last_clean":"2024-04-26T23:56:32.395462+0000","last_became_active":"2024-04-26T23:56:31.355971+0000","last_became_peered":"2024-04-26T23:56:31.355971+0000","last_unstale":"2024-04-26T23:56:32.395462+0000","last_undegraded":"2024-04-26T23:56:32.395462+0000","last_fullsized":"2024-04-26T23:56:32.395462+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:30.855820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"198'3","reported_seq":17,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607286+0000","last_change":"2024-04-26T23:56:30.352656+0000","last_active":"2024-04-26T23:56:32.607286+0000","last_peered":"2024-04-26T23:56:32.607286+0000","last_clean":"2024-04-26T23:56:32.607286+0000","last_became_active":"2024-04-26T23:56:30.349500+0000","last_became_peered":"2024-04-26T23:56:30.349500+0000","last_unstale":"2024-04-26T23:56:32.607286+0000","last_undegraded":"2024-04-26T23:56:32.607286+0000","last_fullsized":"2024-04-26T23:56:32.607286+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:59.814054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.332174+0000","last_change":"2024-04-26T23:56:31.363162+0000","last_active":"2024-04-26T23:56:32.332174+0000","last_peered":"2024-04-26T23:56:32.332174+0000","last_clean":"2024-04-26T23:56:32.332174+0000","last_became_active":"2024-04-26T23:56:31.360916+0000","last_became_peered":"2024-04-26T23:56:31.360916+0000","last_unstale":"2024-04-26T23:56:32.332174+0000","last_undegraded":"2024-04-26T23:56:32.332174+0000","last_fullsized":"2024-04-26T23:56:32.332174+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:28.155345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"198'6","reported_seq":20,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607098+0000","last_change":"2024-04-26T23:56:30.360988+0000","last_active":"2024-04-26T23:56:32.607098+0000","last_peered":"2024-04-26T23:56:32.607098+0000","last_clean":"2024-04-26T23:56:32.607098+0000","last_became_active":"2024-04-26T23:56:30.360611+0000","last_became_peered":"2024-04-26T23:56:30.360611+0000","last_unstale":"2024-04-26T23:56:32.607098+0000","last_undegraded":"2024-04-26T23:56:32.607098+0000","last_fullsized":"2024-04-26T23:56:32.607098+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:34.679261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326264+0000","last_change":"2024-04-26T23:56:31.350734+0000","last_active":"2024-04-26T23:56:32.326264+0000","last_peered":"2024-04-26T23:56:32.326264+0000","last_clean":"2024-04-26T23:56:32.326264+0000","last_became_active":"2024-04-26T23:56:31.350265+0000","last_became_peered":"2024-04-26T23:56:31.350265+0000","last_unstale":"2024-04-26T23:56:32.326264+0000","last_undegraded":"2024-04-26T23:56:32.326264+0000","last_fullsized":"2024-04-26T23:56:32.326264+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:43.402370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607509+0000","last_change":"2024-04-26T23:56:30.357684+0000","last_active":"2024-04-26T23:56:32.607509+0000","last_peered":"2024-04-26T23:56:32.607509+0000","last_clean":"2024-04-26T23:56:32.607509+0000","last_became_active":"2024-04-26T23:56:30.357048+0000","last_became_peered":"2024-04-26T23:56:30.357048+0000","last_unstale":"2024-04-26T23:56:32.607509+0000","last_undegraded":"2024-04-26T23:56:32.607509+0000","last_fullsized":"2024-04-26T23:56:32.607509+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:23.462873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606738+0000","last_change":"2024-04-26T23:56:31.360994+0000","last_active":"2024-04-26T23:56:32.606738+0000","last_peered":"2024-04-26T23:56:32.606738+0000","last_clean":"2024-04-26T23:56:32.606738+0000","last_became_active":"2024-04-26T23:56:31.360600+0000","last_became_peered":"2024-04-26T23:56:31.360600+0000","last_unstale":"2024-04-26T23:56:32.606738+0000","last_undegraded":"2024-04-26T23:56:32.606738+0000","last_fullsized":"2024-04-26T23:56:32.606738+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:11:49.492587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326820+0000","last_change":"2024-04-26T23:56:30.330647+0000","last_active":"2024-04-26T23:56:32.326820+0000","last_peered":"2024-04-26T23:56:32.326820+0000","last_clean":"2024-04-26T23:56:32.326820+0000","last_became_active":"2024-04-26T23:56:30.330241+0000","last_became_peered":"2024-04-26T23:56:30.330241+0000","last_unstale":"2024-04-26T23:56:32.326820+0000","last_undegraded":"2024-04-26T23:56:32.326820+0000","last_fullsized":"2024-04-26T23:56:32.326820+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:33.472673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394747+0000","last_change":"2024-04-26T23:56:31.359481+0000","last_active":"2024-04-26T23:56:32.394747+0000","last_peered":"2024-04-26T23:56:32.394747+0000","last_clean":"2024-04-26T23:56:32.394747+0000","last_became_active":"2024-04-26T23:56:31.359163+0000","last_became_peered":"2024-04-26T23:56:31.359163+0000","last_unstale":"2024-04-26T23:56:32.394747+0000","last_undegraded":"2024-04-26T23:56:32.394747+0000","last_fullsized":"2024-04-26T23:56:32.394747+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:48.771335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604619+0000","last_change":"2024-04-26T23:56:30.349893+0000","last_active":"2024-04-26T23:56:32.604619+0000","last_peered":"2024-04-26T23:56:32.604619+0000","last_clean":"2024-04-26T23:56:32.604619+0000","last_became_active":"2024-04-26T23:56:30.348198+0000","last_became_peered":"2024-04-26T23:56:30.348198+0000","last_unstale":"2024-04-26T23:56:32.604619+0000","last_undegraded":"2024-04-26T23:56:32.604619+0000","last_fullsized":"2024-04-26T23:56:32.604619+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:42.136259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328502+0000","last_change":"2024-04-26T23:56:31.358576+0000","last_active":"2024-04-26T23:56:32.328502+0000","last_peered":"2024-04-26T23:56:32.328502+0000","last_clean":"2024-04-26T23:56:32.328502+0000","last_became_active":"2024-04-26T23:56:31.358271+0000","last_became_peered":"2024-04-26T23:56:31.358271+0000","last_unstale":"2024-04-26T23:56:32.328502+0000","last_undegraded":"2024-04-26T23:56:32.328502+0000","last_fullsized":"2024-04-26T23:56:32.328502+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.251860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"198'8","reported_seq":22,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605741+0000","last_change":"2024-04-26T23:56:30.353426+0000","last_active":"2024-04-26T23:56:32.605741+0000","last_peered":"2024-04-26T23:56:32.605741+0000","last_clean":"2024-04-26T23:56:32.605741+0000","last_became_active":"2024-04-26T23:56:30.352514+0000","last_became_peered":"2024-04-26T23:56:30.352514+0000","last_unstale":"2024-04-26T23:56:32.605741+0000","last_undegraded":"2024-04-26T23:56:32.605741+0000","last_fullsized":"2024-04-26T23:56:32.605741+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:24.711586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326572+0000","last_change":"2024-04-26T23:56:31.361676+0000","last_active":"2024-04-26T23:56:32.326572+0000","last_peered":"2024-04-26T23:56:32.326572+0000","last_clean":"2024-04-26T23:56:32.326572+0000","last_became_active":"2024-04-26T23:56:31.361380+0000","last_became_peered":"2024-04-26T23:56:31.361380+0000","last_unstale":"2024-04-26T23:56:32.326572+0000","last_undegraded":"2024-04-26T23:56:32.326572+0000","last_fullsized":"2024-04-26T23:56:32.326572+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:19.956078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395673+0000","last_change":"2024-04-26T23:56:30.357263+0000","last_active":"2024-04-26T23:56:32.395673+0000","last_peered":"2024-04-26T23:56:32.395673+0000","last_clean":"2024-04-26T23:56:32.395673+0000","last_became_active":"2024-04-26T23:56:30.356822+0000","last_became_peered":"2024-04-26T23:56:30.356822+0000","last_unstale":"2024-04-26T23:56:32.395673+0000","last_undegraded":"2024-04-26T23:56:32.395673+0000","last_fullsized":"2024-04-26T23:56:32.395673+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:14.590845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606746+0000","last_change":"2024-04-26T23:56:31.352970+0000","last_active":"2024-04-26T23:56:32.606746+0000","last_peered":"2024-04-26T23:56:32.606746+0000","last_clean":"2024-04-26T23:56:32.606746+0000","last_became_active":"2024-04-26T23:56:31.352664+0000","last_became_peered":"2024-04-26T23:56:31.352664+0000","last_unstale":"2024-04-26T23:56:32.606746+0000","last_undegraded":"2024-04-26T23:56:32.606746+0000","last_fullsized":"2024-04-26T23:56:32.606746+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:59.300381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604176+0000","last_change":"2024-04-26T23:56:30.342101+0000","last_active":"2024-04-26T23:56:32.604176+0000","last_peered":"2024-04-26T23:56:32.604176+0000","last_clean":"2024-04-26T23:56:32.604176+0000","last_became_active":"2024-04-26T23:56:30.341785+0000","last_became_peered":"2024-04-26T23:56:30.341785+0000","last_unstale":"2024-04-26T23:56:32.604176+0000","last_undegraded":"2024-04-26T23:56:32.604176+0000","last_fullsized":"2024-04-26T23:56:32.604176+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:41.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395520+0000","last_change":"2024-04-26T23:56:31.359663+0000","last_active":"2024-04-26T23:56:32.395520+0000","last_peered":"2024-04-26T23:56:32.395520+0000","last_clean":"2024-04-26T23:56:32.395520+0000","last_became_active":"2024-04-26T23:56:31.358242+0000","last_became_peered":"2024-04-26T23:56:31.358242+0000","last_unstale":"2024-04-26T23:56:32.395520+0000","last_undegraded":"2024-04-26T23:56:32.395520+0000","last_fullsized":"2024-04-26T23:56:32.395520+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:46.468374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607299+0000","last_change":"2024-04-26T23:56:30.363295+0000","last_active":"2024-04-26T23:56:32.607299+0000","last_peered":"2024-04-26T23:56:32.607299+0000","last_clean":"2024-04-26T23:56:32.607299+0000","last_became_active":"2024-04-26T23:56:30.363014+0000","last_became_peered":"2024-04-26T23:56:30.363014+0000","last_unstale":"2024-04-26T23:56:32.607299+0000","last_undegraded":"2024-04-26T23:56:32.607299+0000","last_fullsized":"2024-04-26T23:56:32.607299+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.704673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608764+0000","last_change":"2024-04-26T23:56:31.357793+0000","last_active":"2024-04-26T23:56:32.608764+0000","last_peered":"2024-04-26T23:56:32.608764+0000","last_clean":"2024-04-26T23:56:32.608764+0000","last_became_active":"2024-04-26T23:56:31.355701+0000","last_became_peered":"2024-04-26T23:56:31.355701+0000","last_unstale":"2024-04-26T23:56:32.608764+0000","last_undegraded":"2024-04-26T23:56:32.608764+0000","last_fullsized":"2024-04-26T23:56:32.608764+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:48.992026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608182+0000","last_change":"2024-04-26T23:56:30.363170+0000","last_active":"2024-04-26T23:56:32.608182+0000","last_peered":"2024-04-26T23:56:32.608182+0000","last_clean":"2024-04-26T23:56:32.608182+0000","last_became_active":"2024-04-26T23:56:30.362784+0000","last_became_peered":"2024-04-26T23:56:30.362784+0000","last_unstale":"2024-04-26T23:56:32.608182+0000","last_undegraded":"2024-04-26T23:56:32.608182+0000","last_fullsized":"2024-04-26T23:56:32.608182+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:13.548831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605396+0000","last_change":"2024-04-26T23:56:31.352576+0000","last_active":"2024-04-26T23:56:32.605396+0000","last_peered":"2024-04-26T23:56:32.605396+0000","last_clean":"2024-04-26T23:56:32.605396+0000","last_became_active":"2024-04-26T23:56:31.352309+0000","last_became_peered":"2024-04-26T23:56:31.352309+0000","last_unstale":"2024-04-26T23:56:32.605396+0000","last_undegraded":"2024-04-26T23:56:32.605396+0000","last_fullsized":"2024-04-26T23:56:32.605396+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:12.381377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607779+0000","last_change":"2024-04-26T23:56:30.358916+0000","last_active":"2024-04-26T23:56:32.607779+0000","last_peered":"2024-04-26T23:56:32.607779+0000","last_clean":"2024-04-26T23:56:32.607779+0000","last_became_active":"2024-04-26T23:56:30.358236+0000","last_became_peered":"2024-04-26T23:56:30.358236+0000","last_unstale":"2024-04-26T23:56:32.607779+0000","last_undegraded":"2024-04-26T23:56:32.607779+0000","last_fullsized":"2024-04-26T23:56:32.607779+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:45.145804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605204+0000","last_change":"2024-04-26T23:56:31.357226+0000","last_active":"2024-04-26T23:56:32.605204+0000","last_peered":"2024-04-26T23:56:32.605204+0000","last_clean":"2024-04-26T23:56:32.605204+0000","last_became_active":"2024-04-26T23:56:31.356927+0000","last_became_peered":"2024-04-26T23:56:31.356927+0000","last_unstale":"2024-04-26T23:56:32.605204+0000","last_undegraded":"2024-04-26T23:56:32.605204+0000","last_fullsized":"2024-04-26T23:56:32.605204+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:13.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328566+0000","last_change":"2024-04-26T23:56:30.348804+0000","last_active":"2024-04-26T23:56:32.328566+0000","last_peered":"2024-04-26T23:56:32.328566+0000","last_clean":"2024-04-26T23:56:32.328566+0000","last_became_active":"2024-04-26T23:56:30.348363+0000","last_became_peered":"2024-04-26T23:56:30.348363+0000","last_unstale":"2024-04-26T23:56:32.328566+0000","last_undegraded":"2024-04-26T23:56:32.328566+0000","last_fullsized":"2024-04-26T23:56:32.328566+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.968299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326901+0000","last_change":"2024-04-26T23:56:31.364914+0000","last_active":"2024-04-26T23:56:32.326901+0000","last_peered":"2024-04-26T23:56:32.326901+0000","last_clean":"2024-04-26T23:56:32.326901+0000","last_became_active":"2024-04-26T23:56:31.363306+0000","last_became_peered":"2024-04-26T23:56:31.363306+0000","last_unstale":"2024-04-26T23:56:32.326901+0000","last_undegraded":"2024-04-26T23:56:32.326901+0000","last_fullsized":"2024-04-26T23:56:32.326901+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:38.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328364+0000","last_change":"2024-04-26T23:56:30.348944+0000","last_active":"2024-04-26T23:56:32.328364+0000","last_peered":"2024-04-26T23:56:32.328364+0000","last_clean":"2024-04-26T23:56:32.328364+0000","last_became_active":"2024-04-26T23:56:30.347911+0000","last_became_peered":"2024-04-26T23:56:30.347911+0000","last_unstale":"2024-04-26T23:56:32.328364+0000","last_undegraded":"2024-04-26T23:56:32.328364+0000","last_fullsized":"2024-04-26T23:56:32.328364+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:17.713731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395024+0000","last_change":"2024-04-26T23:56:31.363697+0000","last_active":"2024-04-26T23:56:32.395024+0000","last_peered":"2024-04-26T23:56:32.395024+0000","last_clean":"2024-04-26T23:56:32.395024+0000","last_became_active":"2024-04-26T23:56:31.361930+0000","last_became_peered":"2024-04-26T23:56:31.361930+0000","last_unstale":"2024-04-26T23:56:32.395024+0000","last_undegraded":"2024-04-26T23:56:32.395024+0000","last_fullsized":"2024-04-26T23:56:32.395024+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.492032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.603579+0000","last_change":"2024-04-26T23:56:30.339407+0000","last_active":"2024-04-26T23:56:32.603579+0000","last_peered":"2024-04-26T23:56:32.603579+0000","last_clean":"2024-04-26T23:56:32.603579+0000","last_became_active":"2024-04-26T23:56:30.336661+0000","last_became_peered":"2024-04-26T23:56:30.336661+0000","last_unstale":"2024-04-26T23:56:32.603579+0000","last_undegraded":"2024-04-26T23:56:32.603579+0000","last_fullsized":"2024-04-26T23:56:32.603579+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:48:14.455420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.326800+0000","last_change":"2024-04-26T23:56:31.346808+0000","last_active":"2024-04-26T23:56:32.326800+0000","last_peered":"2024-04-26T23:56:32.326800+0000","last_clean":"2024-04-26T23:56:32.326800+0000","last_became_active":"2024-04-26T23:56:31.345760+0000","last_became_peered":"2024-04-26T23:56:31.345760+0000","last_unstale":"2024-04-26T23:56:32.326800+0000","last_undegraded":"2024-04-26T23:56:32.326800+0000","last_fullsized":"2024-04-26T23:56:32.326800+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:59.300381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608834+0000","last_change":"2024-04-26T23:56:30.359227+0000","last_active":"2024-04-26T23:56:32.608834+0000","last_peered":"2024-04-26T23:56:32.608834+0000","last_clean":"2024-04-26T23:56:32.608834+0000","last_became_active":"2024-04-26T23:56:30.358933+0000","last_became_peered":"2024-04-26T23:56:30.358933+0000","last_unstale":"2024-04-26T23:56:32.608834+0000","last_undegraded":"2024-04-26T23:56:32.608834+0000","last_fullsized":"2024-04-26T23:56:32.608834+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:27.140017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607078+0000","last_change":"2024-04-26T23:56:31.359468+0000","last_active":"2024-04-26T23:56:32.607078+0000","last_peered":"2024-04-26T23:56:32.607078+0000","last_clean":"2024-04-26T23:56:32.607078+0000","last_became_active":"2024-04-26T23:56:31.358898+0000","last_became_peered":"2024-04-26T23:56:31.358898+0000","last_unstale":"2024-04-26T23:56:32.607078+0000","last_undegraded":"2024-04-26T23:56:32.607078+0000","last_fullsized":"2024-04-26T23:56:32.607078+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:38.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"199'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.329140+0000","last_change":"2024-04-26T23:56:30.348292+0000","last_active":"2024-04-26T23:56:32.329140+0000","last_peered":"2024-04-26T23:56:32.329140+0000","last_clean":"2024-04-26T23:56:32.329140+0000","last_became_active":"2024-04-26T23:56:30.347878+0000","last_became_peered":"2024-04-26T23:56:30.347878+0000","last_unstale":"2024-04-26T23:56:32.329140+0000","last_undegraded":"2024-04-26T23:56:32.329140+0000","last_fullsized":"2024-04-26T23:56:32.329140+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:08.867444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605381+0000","last_change":"2024-04-26T23:56:31.362318+0000","last_active":"2024-04-26T23:56:32.605381+0000","last_peered":"2024-04-26T23:56:32.605381+0000","last_clean":"2024-04-26T23:56:32.605381+0000","last_became_active":"2024-04-26T23:56:31.362041+0000","last_became_peered":"2024-04-26T23:56:31.362041+0000","last_unstale":"2024-04-26T23:56:32.605381+0000","last_undegraded":"2024-04-26T23:56:32.605381+0000","last_fullsized":"2024-04-26T23:56:32.605381+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:07.938823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.328920+0000","last_change":"2024-04-26T23:56:30.341651+0000","last_active":"2024-04-26T23:56:32.328920+0000","last_peered":"2024-04-26T23:56:32.328920+0000","last_clean":"2024-04-26T23:56:32.328920+0000","last_became_active":"2024-04-26T23:56:30.341271+0000","last_became_peered":"2024-04-26T23:56:30.341271+0000","last_unstale":"2024-04-26T23:56:32.328920+0000","last_undegraded":"2024-04-26T23:56:32.328920+0000","last_fullsized":"2024-04-26T23:56:32.328920+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:46:46.531883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604188+0000","last_change":"2024-04-26T23:56:31.355351+0000","last_active":"2024-04-26T23:56:32.604188+0000","last_peered":"2024-04-26T23:56:32.604188+0000","last_clean":"2024-04-26T23:56:32.604188+0000","last_became_active":"2024-04-26T23:56:31.354609+0000","last_became_peered":"2024-04-26T23:56:31.354609+0000","last_unstale":"2024-04-26T23:56:32.604188+0000","last_undegraded":"2024-04-26T23:56:32.604188+0000","last_fullsized":"2024-04-26T23:56:32.604188+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:25.726914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.608443+0000","last_change":"2024-04-26T23:56:30.358203+0000","last_active":"2024-04-26T23:56:32.608443+0000","last_peered":"2024-04-26T23:56:32.608443+0000","last_clean":"2024-04-26T23:56:32.608443+0000","last_became_active":"2024-04-26T23:56:30.357769+0000","last_became_peered":"2024-04-26T23:56:30.357769+0000","last_unstale":"2024-04-26T23:56:32.608443+0000","last_undegraded":"2024-04-26T23:56:32.608443+0000","last_fullsized":"2024-04-26T23:56:32.608443+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:51.629823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394622+0000","last_change":"2024-04-26T23:56:31.365207+0000","last_active":"2024-04-26T23:56:32.394622+0000","last_peered":"2024-04-26T23:56:32.394622+0000","last_clean":"2024-04-26T23:56:32.394622+0000","last_became_active":"2024-04-26T23:56:31.361947+0000","last_became_peered":"2024-04-26T23:56:31.361947+0000","last_unstale":"2024-04-26T23:56:32.394622+0000","last_undegraded":"2024-04-26T23:56:32.394622+0000","last_fullsized":"2024-04-26T23:56:32.394622+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:06.901972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607819+0000","last_change":"2024-04-26T23:56:30.359102+0000","last_active":"2024-04-26T23:56:32.607819+0000","last_peered":"2024-04-26T23:56:32.607819+0000","last_clean":"2024-04-26T23:56:32.607819+0000","last_became_active":"2024-04-26T23:56:30.357370+0000","last_became_peered":"2024-04-26T23:56:30.357370+0000","last_unstale":"2024-04-26T23:56:32.607819+0000","last_undegraded":"2024-04-26T23:56:32.607819+0000","last_fullsized":"2024-04-26T23:56:32.607819+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:36.842477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.394948+0000","last_change":"2024-04-26T23:56:31.366729+0000","last_active":"2024-04-26T23:56:32.394948+0000","last_peered":"2024-04-26T23:56:32.394948+0000","last_clean":"2024-04-26T23:56:32.394948+0000","last_became_active":"2024-04-26T23:56:31.364868+0000","last_became_peered":"2024-04-26T23:56:31.364868+0000","last_unstale":"2024-04-26T23:56:32.394948+0000","last_undegraded":"2024-04-26T23:56:32.394948+0000","last_fullsized":"2024-04-26T23:56:32.394948+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:01.864134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.331295+0000","last_change":"2024-04-26T23:56:30.347666+0000","last_active":"2024-04-26T23:56:32.331295+0000","last_peered":"2024-04-26T23:56:32.331295+0000","last_clean":"2024-04-26T23:56:32.331295+0000","last_became_active":"2024-04-26T23:56:30.346147+0000","last_became_peered":"2024-04-26T23:56:30.346147+0000","last_unstale":"2024-04-26T23:56:32.331295+0000","last_undegraded":"2024-04-26T23:56:32.331295+0000","last_fullsized":"2024-04-26T23:56:32.331295+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.704673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604021+0000","last_change":"2024-04-26T23:56:31.351565+0000","last_active":"2024-04-26T23:56:32.604021+0000","last_peered":"2024-04-26T23:56:32.604021+0000","last_clean":"2024-04-26T23:56:32.604021+0000","last_became_active":"2024-04-26T23:56:31.348774+0000","last_became_peered":"2024-04-26T23:56:31.348774+0000","last_unstale":"2024-04-26T23:56:32.604021+0000","last_undegraded":"2024-04-26T23:56:32.604021+0000","last_fullsized":"2024-04-26T23:56:32.604021+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:45.287788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.605417+0000","last_change":"2024-04-26T23:56:30.350199+0000","last_active":"2024-04-26T23:56:32.605417+0000","last_peered":"2024-04-26T23:56:32.605417+0000","last_clean":"2024-04-26T23:56:32.605417+0000","last_became_active":"2024-04-26T23:56:30.348661+0000","last_became_peered":"2024-04-26T23:56:30.348661+0000","last_unstale":"2024-04-26T23:56:32.605417+0000","last_undegraded":"2024-04-26T23:56:32.605417+0000","last_fullsized":"2024-04-26T23:56:32.605417+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.968299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.607061+0000","last_change":"2024-04-26T23:56:31.352943+0000","last_active":"2024-04-26T23:56:32.607061+0000","last_peered":"2024-04-26T23:56:32.607061+0000","last_clean":"2024-04-26T23:56:32.607061+0000","last_became_active":"2024-04-26T23:56:31.351999+0000","last_became_peered":"2024-04-26T23:56:31.351999+0000","last_unstale":"2024-04-26T23:56:32.607061+0000","last_undegraded":"2024-04-26T23:56:32.607061+0000","last_fullsized":"2024-04-26T23:56:32.607061+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:53.375472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606057+0000","last_change":"2024-04-26T23:56:30.359350+0000","last_active":"2024-04-26T23:56:32.606057+0000","last_peered":"2024-04-26T23:56:32.606057+0000","last_clean":"2024-04-26T23:56:32.606057+0000","last_became_active":"2024-04-26T23:56:30.359057+0000","last_became_peered":"2024-04-26T23:56:30.359057+0000","last_unstale":"2024-04-26T23:56:32.606057+0000","last_undegraded":"2024-04-26T23:56:32.606057+0000","last_fullsized":"2024-04-26T23:56:32.606057+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:47.579445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.395293+0000","last_change":"2024-04-26T23:56:31.363959+0000","last_active":"2024-04-26T23:56:32.395293+0000","last_peered":"2024-04-26T23:56:32.395293+0000","last_clean":"2024-04-26T23:56:32.395293+0000","last_became_active":"2024-04-26T23:56:31.361532+0000","last_became_peered":"2024-04-26T23:56:31.361532+0000","last_unstale":"2024-04-26T23:56:32.395293+0000","last_undegraded":"2024-04-26T23:56:32.395293+0000","last_fullsized":"2024-04-26T23:56:32.395293+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:30.309570+0000","last_clean_scrub_stamp":"2024-04-26T23:56:30.309570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:38.703867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.606177+0000","last_change":"2024-04-26T23:56:30.345798+0000","last_active":"2024-04-26T23:56:32.606177+0000","last_peered":"2024-04-26T23:56:32.606177+0000","last_clean":"2024-04-26T23:56:32.606177+0000","last_became_active":"2024-04-26T23:56:30.345187+0000","last_became_peered":"2024-04-26T23:56:30.345187+0000","last_unstale":"2024-04-26T23:56:32.606177+0000","last_undegraded":"2024-04-26T23:56:32.606177+0000","last_fullsized":"2024-04-26T23:56:32.606177+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:56:29.294242+0000","last_clean_scrub_stamp":"2024-04-26T23:56:29.294242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:04.615985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-26T23:56:32.604692+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:56:32.604692+0000","last_peered":"2024-04-26T23:56:32.604692+0000","last_clean":"2024-04-26T23:56:32.604692+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:56:32.604692+0000","last_undegraded":"2024-04-26T23:56:32.604692+0000","last_fullsized":"2024-04-26T23:56:32.604692+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575085,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52276,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319564,"statfs":{"total":96636764160,"available":96583233536,"internally_reserved":0,"allocated":3117056,"data_stored":1781027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575086,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2539520,"data_stored":1191725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575086,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55864,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315976,"statfs":{"total":96636764160,"available":96579559424,"internally_reserved":0,"allocated":2531328,"data_stored":1191669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575085,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60016,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311824,"statfs":{"total":96636764160,"available":96575307776,"internally_reserved":0,"allocated":2519040,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575086,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47528,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324312,"statfs":{"total":96636764160,"available":96588095488,"internally_reserved":0,"allocated":2519040,"data_stored":1190009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575086,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64180,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307660,"statfs":{"total":96636764160,"available":96571043840,"internally_reserved":0,"allocated":2523136,"data_stored":1190603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575086,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47532,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324308,"statfs":{"total":96636764160,"available":96588091392,"internally_reserved":0,"allocated":2514944,"data_stored":1189987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575086,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60592,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311248,"statfs":{"total":96636764160,"available":96574717952,"internally_reserved":0,"allocated":3108864,"data_stored":1780355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:56:36.380 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:56:36.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:56:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.649+0000 7fd1cf200640 1 Processor -- start 2024-04-26T23:56:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.649+0000 7fd1cf200640 1 -- start start 2024-04-26T23:56:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1c8162aa0 0x7fd1c80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cf200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1c80d07d0 con 0x7fd1c81626c0 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80d0ab0 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80d0d90 con 0x7fd1c8162aa0 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48112/0 (socket says 172.21.15.46:48112) 2024-04-26T23:56:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1cd400640 1 -- 172.21.15.46:0/2236331844 learned_addr learned my addr 172.21.15.46:0/2236331844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.651+0000 7fd1ce800640 1 --2- 172.21.15.46:0/2236331844 >> v2:172.21.15.142:3300/0 conn(0x7fd1c8162aa0 0x7fd1c80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.652+0000 7fd1cd400640 1 -- 172.21.15.46:0/2236331844 >> v2:172.21.15.142:3300/0 conn(0x7fd1c8162aa0 msgr2=0x7fd1c80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.652+0000 7fd1cd400640 1 --2- 172.21.15.46:0/2236331844 >> v2:172.21.15.142:3300/0 conn(0x7fd1c8162aa0 0x7fd1c80d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.652+0000 7fd1cd400640 1 -- 172.21.15.46:0/2236331844 >> v1:172.21.15.142:6789/0 conn(0x7fd1c81626c0 legacy=0x7fd1c80d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.652+0000 7fd1cd400640 1 -- 172.21.15.46:0/2236331844 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c80d2c80 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.653+0000 7fd1cd400640 1 --2- 172.21.15.46:0/2236331844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80bafe0 tx=0x7fd1b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fc2456617b1cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.653+0000 7fd1cca00640 1 -- 172.21.15.46:0/2236331844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8004030 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.653+0000 7fd1cca00640 1 -- 172.21.15.46:0/2236331844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b80041d0 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.653+0000 7fd1cca00640 1 -- 172.21.15.46:0/2236331844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003610 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 -- 172.21.15.46:0/2236331844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c80b5fb0 msgr2=0x7fd1c80b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 --2- 172.21.15.46:0/2236331844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80bafe0 tx=0x7fd1b802a210 comp rx=0 tx=0).stop 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 -- 172.21.15.46:0/2236331844 shutdown_connections 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 --2- 172.21.15.46:0/2236331844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b47c0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 --2- 172.21.15.46:0/2236331844 >> v2:172.21.15.142:3300/0 conn(0x7fd1c8162aa0 0x7fd1c80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 -- 172.21.15.46:0/2236331844 >> 172.21.15.46:0/2236331844 conn(0x7fd1c809e2a0 msgr2=0x7fd1c80d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 -- 172.21.15.46:0/2236331844 shutdown_connections 2024-04-26T23:56:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.654+0000 7fd1cf200640 1 -- 172.21.15.46:0/2236331844 wait complete. 2024-04-26T23:56:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.656+0000 7fd1cf200640 1 Processor -- start 2024-04-26T23:56:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.656+0000 7fd1cf200640 1 -- start start 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1c8162aa0 0x7fd1c8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d2210 con 0x7fd1c81626c0 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d2090 con 0x7fd1c80b5fb0 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c80d31a0 con 0x7fd1c8162aa0 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48122/0 (socket says 172.21.15.46:48122) 2024-04-26T23:56:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.657+0000 7fd1cd400640 1 -- 172.21.15.46:0/496918121 learned_addr learned my addr 172.21.15.46:0/496918121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cde00640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 -- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1c8162aa0 msgr2=0x7fd1c8152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1c8162aa0 0x7fd1c8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 -- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 msgr2=0x7fd1c80b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 -- 172.21.15.46:0/496918121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c80ba370 con 0x7fd1c81626c0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cde00640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.658+0000 7fd1cd400640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000c00 tx=0x7fd1b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.659+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8041020 con 0x7fd1c81626c0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.659+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b8003bd0 con 0x7fd1c81626c0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.659+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8004030 con 0x7fd1c81626c0 2024-04-26T23:56:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.659+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c80ba650 con 0x7fd1c81626c0 2024-04-26T23:56:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.659+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c80bac10 con 0x7fd1c81626c0 2024-04-26T23:56:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.660+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c80693b0 con 0x7fd1c81626c0 2024-04-26T23:56:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.661+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1b8044020 con 0x7fd1c81626c0 2024-04-26T23:56:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.662+0000 7fd1c2000640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.662+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7fd1b80aca00 con 0x7fd1c81626c0 2024-04-26T23:56:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.663+0000 7fd1cde00640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.664+0000 7fd1cde00640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc003eb0 tx=0x7fd1bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:36.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.666+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1b804b030 con 0x7fd1c81626c0 2024-04-26T23:56:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.956+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd1c80d25b0 con 0x7fd1c81626c0 2024-04-26T23:56:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.961+0000 7fd1c2000640 1 -- 172.21.15.46:0/496918121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7fd1b807ac00 con 0x7fd1c81626c0 2024-04-26T23:56:36.964 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:56:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:56:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:56:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:56:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:56:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:56:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 msgr2=0x7fd1ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc003eb0 tx=0x7fd1bc009040 comp rx=0 tx=0).stop 2024-04-26T23:56:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 msgr2=0x7fd1c80ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000c00 tx=0x7fd1b80046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 shutdown_connections 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1c8162aa0 0x7fd1c8152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c81626c0 0x7fd1c80ce960 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 --2- 172.21.15.46:0/496918121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c80b5fb0 0x7fd1c80b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.964+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 >> 172.21.15.46:0/496918121 conn(0x7fd1c809e2a0 msgr2=0x7fd1c806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.965+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 shutdown_connections 2024-04-26T23:56:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:36.965+0000 7fd1cf200640 1 -- 172.21.15.46:0/496918121 wait complete. 2024-04-26T23:56:36.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:37.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.194+0000 7fd74e000640 1 Processor -- start 2024-04-26T23:56:37.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.195+0000 7fd74e000640 1 -- start start 2024-04-26T23:56:37.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74e000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd74815f4e0 0x7fd7480d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7480d11f0 con 0x7fd7480b4000 2024-04-26T23:56:37.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74e000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd7480d0ce0 con 0x7fd74815f4e0 2024-04-26T23:56:37.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd7480d2990 con 0x7fd7480d2130 2024-04-26T23:56:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:43628/0 (socket says 172.21.15.46:43628) 2024-04-26T23:56:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74d600640 1 -- 172.21.15.46:0/861653121 learned_addr learned my addr 172.21.15.46:0/861653121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.197+0000 7fd74cc00640 1 --2- 172.21.15.46:0/861653121 >> v2:172.21.15.142:3300/0 conn(0x7fd74815f4e0 0x7fd7480d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.198+0000 7fd74d600640 1 -- 172.21.15.46:0/861653121 >> v2:172.21.15.142:3300/0 conn(0x7fd74815f4e0 msgr2=0x7fd7480d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.198+0000 7fd74d600640 1 --2- 172.21.15.46:0/861653121 >> v2:172.21.15.142:3300/0 conn(0x7fd74815f4e0 0x7fd7480d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.198+0000 7fd747400640 1 -- 172.21.15.46:0/861653121 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248545224 0 0) 0x7fd7480d11f0 con 0x7fd7480b4000 2024-04-26T23:56:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.198+0000 7fd74d600640 1 -- 172.21.15.46:0/861653121 >> v1:172.21.15.142:6789/0 conn(0x7fd7480b4000 legacy=0x7fd7480d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.198+0000 7fd74d600640 1 -- 172.21.15.46:0/861653121 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7480d5f20 con 0x7fd7480d2130 2024-04-26T23:56:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.199+0000 7fd74d600640 1 --2- 172.21.15.46:0/861653121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd740009870 tx=0x7fd74002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09f4e82133bf9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.200+0000 7fd747400640 1 -- 172.21.15.46:0/861653121 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740037070 con 0x7fd7480d2130 2024-04-26T23:56:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.200+0000 7fd747400640 1 -- 172.21.15.46:0/861653121 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd740032440 con 0x7fd7480d2130 2024-04-26T23:56:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.200+0000 7fd747400640 1 -- 172.21.15.46:0/861653121 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740032810 con 0x7fd7480d2130 2024-04-26T23:56:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 -- 172.21.15.46:0/861653121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 msgr2=0x7fd7480b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 --2- 172.21.15.46:0/861653121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd740009870 tx=0x7fd74002a250 comp rx=0 tx=0).stop 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 -- 172.21.15.46:0/861653121 shutdown_connections 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 --2- 172.21.15.46:0/861653121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7480d2130 0x7fd7480b2280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 --2- 172.21.15.46:0/861653121 >> v2:172.21.15.142:3300/0 conn(0x7fd74815f4e0 0x7fd7480d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 -- 172.21.15.46:0/861653121 >> 172.21.15.46:0/861653121 conn(0x7fd7480d5010 msgr2=0x7fd7480d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 -- 172.21.15.46:0/861653121 shutdown_connections 2024-04-26T23:56:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.201+0000 7fd74e000640 1 -- 172.21.15.46:0/861653121 wait complete. 2024-04-26T23:56:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.203+0000 7fd74e000640 1 Processor -- start 2024-04-26T23:56:37.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.203+0000 7fd74e000640 1 -- start start 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7480b4000 0x7fd7480ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd74815f4e0 0x7fd74815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480b1d60 con 0x7fd7480d2130 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480d5c40 con 0x7fd7480b4000 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd74815f8c0 con 0x7fd74815f4e0 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48128/0 (socket says 172.21.15.46:48128) 2024-04-26T23:56:37.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd747e00640 1 -- 172.21.15.46:0/3344722384 learned_addr learned my addr 172.21.15.46:0/3344722384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74d600640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd74815f4e0 0x7fd74815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.205+0000 7fd74cc00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7480b4000 0x7fd7480ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 -- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd74815f4e0 msgr2=0x7fd74815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd74815f4e0 0x7fd74815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 -- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7480b4000 msgr2=0x7fd7480ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7480b4000 0x7fd7480ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 -- 172.21.15.46:0/3344722384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7480ccc90 con 0x7fd7480d2130 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd747e00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c7e0 tx=0x7fd73800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738003da0 con 0x7fd7480d2130 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd738005070 con 0x7fd7480d2130 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738005460 con 0x7fd7480d2130 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.206+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7480ccf70 con 0x7fd7480d2130 2024-04-26T23:56:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.207+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7480cd530 con 0x7fd7480d2130 2024-04-26T23:56:37.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.208+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7480693b0 con 0x7fd7480d2130 2024-04-26T23:56:37.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.209+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd738005600 con 0x7fd7480d2130 2024-04-26T23:56:37.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.210+0000 7fd744c00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:37.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.210+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7fd73808bbe0 con 0x7fd7480d2130 2024-04-26T23:56:37.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.211+0000 7fd74cc00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.212+0000 7fd74cc00640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd74815e5c0 tx=0x7fd730008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.215+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd738059f10 con 0x7fd7480d2130 2024-04-26T23:56:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.458+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7480b79b0 con 0x7fd7480d2130 2024-04-26T23:56:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.460+0000 7fd744c00640 1 -- 172.21.15.46:0/3344722384 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7fd738059810 con 0x7fd7480d2130 2024-04-26T23:56:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:37.462 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 msgr2=0x7fd72406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd74815e5c0 tx=0x7fd730008040 comp rx=0 tx=0).stop 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 msgr2=0x7fd7480ba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c7e0 tx=0x7fd73800cca0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 shutdown_connections 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd72406c8f0 0x7fd72406edb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd74815f4e0 0x7fd74815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7480d2130 0x7fd7480ba6e0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 --2- 172.21.15.46:0/3344722384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7480b4000 0x7fd7480ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.462+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 >> 172.21.15.46:0/3344722384 conn(0x7fd7480d5010 msgr2=0x7fd74806e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.463+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 shutdown_connections 2024-04-26T23:56:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:37.463+0000 7fd74e000640 1 -- 172.21.15.46:0/3344722384 wait complete. 2024-04-26T23:56:37.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 158, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:56:30.811636+0000', 'modified': '2024-04-26T23:56:33.903516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11688}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11688': {'gid': 11688, 'name': 'c', 'rank': 0, 'incarnation': 154, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6834/4111191104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 4111191104}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 4111191104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11688, 'qdb_cluster': [11688]} 2024-04-26T23:56:37.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:56:37.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T23:56:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.110+0000 7f962e200640 1 Processor -- start 2024-04-26T23:56:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.110+0000 7f962e200640 1 -- start start 2024-04-26T23:56:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96281466b0 0x7f962806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962e200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f96280720e0 0x7f96280cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9628073ad0 con 0x7f9628160910 2024-04-26T23:56:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9628146a90 con 0x7f96281466b0 2024-04-26T23:56:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962e200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9628073d70 con 0x7f96280720e0 2024-04-26T23:56:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9628160910 0x7f96280cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59636/0 (socket says 172.21.15.46:59636) 2024-04-26T23:56:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962d800640 1 -- 172.21.15.46:0/3241674794 learned_addr learned my addr 172.21.15.46:0/3241674794 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f962ce00640 1 --2- 172.21.15.46:0/3241674794 >> v2:172.21.15.46:3300/0 conn(0x7f96281466b0 0x7f962806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.111+0000 7f961fe00640 1 --2- 172.21.15.46:0/3241674794 >> v2:172.21.15.142:3300/0 conn(0x7f96280720e0 0x7f96280cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.112+0000 7f9627e00640 1 -- 172.21.15.46:0/3241674794 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947570490 0 0) 0x7f9628073ad0 con 0x7f9628160910 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.112+0000 7f962ce00640 1 -- 172.21.15.46:0/3241674794 >> v2:172.21.15.142:3300/0 conn(0x7f96280720e0 msgr2=0x7f96280cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.112+0000 7f962ce00640 1 --2- 172.21.15.46:0/3241674794 >> v2:172.21.15.142:3300/0 conn(0x7f96280720e0 0x7f96280cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.112+0000 7f962ce00640 1 -- 172.21.15.46:0/3241674794 >> v1:172.21.15.46:6789/0 conn(0x7f9628160910 legacy=0x7f96280cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.112+0000 7f962ce00640 1 -- 172.21.15.46:0/3241674794 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9628074050 con 0x7f96281466b0 2024-04-26T23:56:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.113+0000 7f962ce00640 1 --2- 172.21.15.46:0/3241674794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f962806de80 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f96180097a0 tx=0x7f961802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ffdb32512144f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.114+0000 7f9627e00640 1 -- 172.21.15.46:0/3241674794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618037070 con 0x7f96281466b0 2024-04-26T23:56:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.114+0000 7f9627e00640 1 -- 172.21.15.46:0/3241674794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9618032440 con 0x7f96281466b0 2024-04-26T23:56:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.114+0000 7f9627e00640 1 -- 172.21.15.46:0/3241674794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618032810 con 0x7f96281466b0 2024-04-26T23:56:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 -- 172.21.15.46:0/3241674794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 msgr2=0x7f962806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 --2- 172.21.15.46:0/3241674794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f962806de80 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f96180097a0 tx=0x7f961802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 -- 172.21.15.46:0/3241674794 shutdown_connections 2024-04-26T23:56:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 --2- 172.21.15.46:0/3241674794 >> v2:172.21.15.142:3300/0 conn(0x7f96280720e0 0x7f96280cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 --2- 172.21.15.46:0/3241674794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f962806de80 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 -- 172.21.15.46:0/3241674794 >> 172.21.15.46:0/3241674794 conn(0x7f96280b26d0 msgr2=0x7f96280b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 -- 172.21.15.46:0/3241674794 shutdown_connections 2024-04-26T23:56:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.116+0000 7f962e200640 1 -- 172.21.15.46:0/3241674794 wait complete. 2024-04-26T23:56:38.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.117+0000 7f962e200640 1 Processor -- start 2024-04-26T23:56:38.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.117+0000 7f962e200640 1 -- start start 2024-04-26T23:56:38.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 0x7f96281f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9628160910 0x7f9628133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f962806e3c0 con 0x7f96281466b0 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96280cb470 con 0x7f96280720e0 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f962e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f96280cdda0 con 0x7f9628160910 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f961fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f961fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48136/0 (socket says 172.21.15.46:48136) 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.118+0000 7f961fe00640 1 -- 172.21.15.46:0/272827905 learned_addr learned my addr 172.21.15.46:0/272827905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f962ce00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 0x7f96281f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f962d800640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9628160910 0x7f9628133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 -- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9628160910 msgr2=0x7f9628133c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9628160910 0x7f9628133c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 -- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 msgr2=0x7f96281f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 0x7f96281f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 -- 172.21.15.46:0/272827905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96280ca600 con 0x7f96281466b0 2024-04-26T23:56:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f962ce00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 0x7f96281f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f961fe00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f961000e870 tx=0x7f961000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f9625600640 1 -- 172.21.15.46:0/272827905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610017070 con 0x7f96281466b0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f9625600640 1 -- 172.21.15.46:0/272827905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961000c940 con 0x7f96281466b0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.119+0000 7f9625600640 1 -- 172.21.15.46:0/272827905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961000cd30 con 0x7f96281466b0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.120+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96280c9a80 con 0x7f96281466b0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.120+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96280ca040 con 0x7f96281466b0 2024-04-26T23:56:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.122+0000 7f9625600640 1 -- 172.21.15.46:0/272827905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f961001a070 con 0x7f96281466b0 2024-04-26T23:56:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.123+0000 7f9625600640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.123+0000 7f9625600640 1 -- 172.21.15.46:0/272827905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f961008bbc0 con 0x7f96281466b0 2024-04-26T23:56:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.123+0000 7f96327be740 1 Processor -- start 2024-04-26T23:56:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.124+0000 7f96327be740 1 -- start start 2024-04-26T23:56:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.124+0000 7f962ce00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.124+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1de9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f96327be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f96327be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5568a1d4b600 con 0x5568a1e23e80 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5568a1ce8100 con 0x5568a1de77b0 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5568a1d475c0 con 0x5568a1e12270 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f962ce00640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9618034000 tx=0x7f961802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48146/0 (socket says 172.21.15.46:48146) 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 -- 172.21.15.46:0/2040002938 learned_addr learned my addr 172.21.15.46:0/2040002938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f962ce00640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1de9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f962d800640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 -- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 msgr2=0x5568a1e126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 -- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 msgr2=0x5568a1de9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1de9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f961fe00640 1 -- 172.21.15.46:0/2040002938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a1a5d2d0 con 0x5568a1e23e80 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.125+0000 7f962ce00640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1de9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f961fe00640 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f9610061e00 tx=0x7f96100591e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec7f80639be87a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f961d600640 1 -- 172.21.15.46:0/2040002938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610021070 con 0x5568a1e23e80 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f961d600640 1 -- 172.21.15.46:0/2040002938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961005db00 con 0x5568a1e23e80 2024-04-26T23:56:38.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f961d600640 1 -- 172.21.15.46:0/2040002938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610062330 con 0x5568a1e23e80 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f96327be740 1 -- 172.21.15.46:0/2040002938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 msgr2=0x5568a1de7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.126+0000 7f96327be740 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f9610061e00 tx=0x7f96100591e0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 -- 172.21.15.46:0/2040002938 shutdown_connections 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1de7b90 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 --2- 172.21.15.46:0/2040002938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1de9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 -- 172.21.15.46:0/2040002938 >> 172.21.15.46:0/2040002938 conn(0x5568a1d7bd10 msgr2=0x5568a1e2d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 -- 172.21.15.46:0/2040002938 shutdown_connections 2024-04-26T23:56:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 -- 172.21.15.46:0/2040002938 wait complete. 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.127+0000 7f96327be740 1 -- start start 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.128+0000 7f96327be740 1 Processor -- start 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 0x5568a1e26860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5568a1cd2f70 con 0x5568a1de77b0 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5568a1e0fdc0 con 0x5568a1e23e80 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5568a1e02040 con 0x5568a1e12270 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f96327be740 10 client.0 _ll_register_callbacks cb 0x5568a1d41480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48148/0 (socket says 172.21.15.46:48148) 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 -- 172.21.15.46:0/2055286041 learned_addr learned my addr 172.21.15.46:0/2055286041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962d800640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 0x5568a1e26860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 -- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 msgr2=0x5568a1e01b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e01b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 -- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 msgr2=0x5568a1e26860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 0x5568a1e26860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962ce00640 1 -- 172.21.15.46:0/2055286041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a1b276b0 con 0x5568a1de77b0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.129+0000 7f962d800640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 0x5568a1e26860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f962ce00640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f961802a7d0 tx=0x7f961802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f977086e082549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f9616000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961804d060 con 0x5568a1de77b0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f961803c870 con 0x5568a1de77b0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961005db00 con 0x5568a1de77b0 2024-04-26T23:56:38.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7f9618053050 con 0x5568a1de77b0 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f96327be740 10 client.11826 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.130+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x5568a1e254d0 con 0x5568a1de77b0 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961005db00 con 0x5568a1de77b0 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7f9618056020 con 0x5568a1de77b0 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 10 client.11826 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 10 client.11826 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 20 client.11826 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e13fd0 0x5568a1e4a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 4 client.11826 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a1d59210 con 0x5568a1e13fd0 2024-04-26T23:56:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.131+0000 7f961fe00640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e13fd0 0x5568a1e4a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.132+0000 7f961fe00640 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e13fd0 0x5568a1e4a950 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd95d7bc7acdea8 server_cookie=447b9b938c58bffb in_seq=0 out_seq=0 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.132+0000 7f9616000640 10 client.11826 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f9616000640 1 -- 172.21.15.46:0/2055286041 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568a1d59210 con 0x5568a1e13fd0 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f9616000640 10 client.11826 handle_command_reply: tid=0 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 client.11826 shutdown 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 msgr2=0x5568a1e242c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f961802a7d0 tx=0x7f961802c070 comp rx=0 tx=0).stop 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 shutdown_connections 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e23e80 0x5568a1e26860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e13fd0 0x5568a1e4a950 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e01b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 --2- 172.21.15.46:0/2055286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e242c0 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f9616000640 0 client.11826 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 >> 172.21.15.46:0/2055286041 conn(0x5568a1d7bd10 msgr2=0x5568a1e101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 shutdown_connections 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 1 -- 172.21.15.46:0/2055286041 wait complete. 2024-04-26T23:56:38.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.134+0000 7f96327be740 20 client.11826 trim_cache size 0 max 16384 2024-04-26T23:56:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 Processor -- start 2024-04-26T23:56:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 -- start start 2024-04-26T23:56:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1e30ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9628200d60 con 0x5568a1e23e80 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96281f50f0 con 0x5568a1de77b0 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.154+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f96281376a0 con 0x5568a1e12270 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48154/0 (socket says 172.21.15.46:48154) 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 -- 172.21.15.46:0/4034433387 learned_addr learned my addr 172.21.15.46:0/4034433387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962ce00640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1e30ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f961fe00640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 -- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 msgr2=0x5568a1e0ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e0ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 -- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 msgr2=0x5568a1e30ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1e30ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f962d800640 1 -- 172.21.15.46:0/4034433387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a1d11b30 con 0x5568a1e23e80 2024-04-26T23:56:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.155+0000 7f961fe00640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f962d800640 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f9620001b30 tx=0x7f9620001ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f912d8d5a46c017c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f961cc00640 1 -- 172.21.15.46:0/4034433387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620004d70 con 0x5568a1e23e80 2024-04-26T23:56:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f961cc00640 1 -- 172.21.15.46:0/4034433387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962000f810 con 0x5568a1e23e80 2024-04-26T23:56:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f961cc00640 1 -- 172.21.15.46:0/4034433387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962000fc00 con 0x5568a1e23e80 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f96327be740 1 -- 172.21.15.46:0/4034433387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 msgr2=0x5568a1e10f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.156+0000 7f96327be740 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f9620001b30 tx=0x7f9620001ff0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 -- 172.21.15.46:0/4034433387 shutdown_connections 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1e23e80 0x5568a1e10f00 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e12270 0x5568a1e0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 --2- 172.21.15.46:0/4034433387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1de77b0 0x5568a1e30ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 -- 172.21.15.46:0/4034433387 >> 172.21.15.46:0/4034433387 conn(0x5568a1d7bd10 msgr2=0x5568a1e26670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 -- 172.21.15.46:0/4034433387 shutdown_connections 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 -- 172.21.15.46:0/4034433387 wait complete. 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 -- start start 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.157+0000 7f96327be740 1 Processor -- start 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.158+0000 7f96327be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.158+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e12270 0x5568a1e3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.158+0000 7f96327be740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e23e80 0x5568a1e22c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f96327be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5568a1e13e50 con 0x5568a1de77b0 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5568a1e126e0 con 0x5568a1e12270 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f96327be740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5568a1e144a0 con 0x5568a1e23e80 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f96327be740 10 client.0 _ll_register_callbacks cb 0x7f9618035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48162/0 (socket says 172.21.15.46:48162) 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 -- 172.21.15.46:0/2413562088 learned_addr learned my addr 172.21.15.46:0/2413562088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 -- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e23e80 msgr2=0x5568a1e22c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e23e80 0x5568a1e22c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 -- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e12270 msgr2=0x5568a1e3c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e12270 0x5568a1e3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 -- 172.21.15.46:0/2413562088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a1e10980 con 0x5568a1de77b0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f962ce00640 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f961802a7d0 tx=0x7f9618004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3b808259f910b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.159+0000 7f9616000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961805b020 con 0x5568a1de77b0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f9618003810 con 0x5568a1de77b0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962000f810 con 0x5568a1de77b0 2024-04-26T23:56:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7f9618065050 con 0x5568a1de77b0 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 10 client.11832 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x5568a1e0cbc0 con 0x5568a1de77b0 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962000f810 con 0x5568a1de77b0 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7f9618068020 con 0x5568a1de77b0 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 10 client.11832 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 10 client.11832 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 20 client.11832 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e0b960 0x5568a1e0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 4 client.11832 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.160+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "status"}) -- 0x5568a1d53080 con 0x5568a1e0b960 2024-04-26T23:56:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f961fe00640 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e0b960 0x5568a1e0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f961fe00640 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e0b960 0x5568a1e0bd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89bab8225c6364 server_cookie=d4eccd95b9aa9a42 in_seq=0 out_seq=0 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f9616000640 10 client.11832 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f9616000640 1 -- 172.21.15.46:0/2413562088 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5568a1d53080 con 0x5568a1e0b960 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f9616000640 10 client.11832 handle_command_reply: tid=0 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.161+0000 7f96327be740 1 client.11832 shutdown 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 msgr2=0x5568a1e00af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f961802a7d0 tx=0x7f9618004730 comp rx=0 tx=0).stop 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 shutdown_connections 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5568a1e0b960 0x5568a1e0bd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5568a1e23e80 0x5568a1e22c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568a1e12270 0x5568a1e3c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 --2- 172.21.15.46:0/2413562088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568a1de77b0 0x5568a1e00af0 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 >> 172.21.15.46:0/2413562088 conn(0x5568a1d7bd10 msgr2=0x5568a1e0f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f9616000640 0 client.11832 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 shutdown_connections 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 1 -- 172.21.15.46:0/2413562088 wait complete. 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f96327be740 20 client.11832 trim_cache size 0 max 16384 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:56:38.169 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 11688, 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.793717934, 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 158, 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 199, 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 199, 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.212598127 2024-04-26T23:56:38.170 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 msgr2=0x7f960406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9618034000 tx=0x7f961802e000 comp rx=0 tx=0).stop 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 msgr2=0x7f96281e6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f961000e870 tx=0x7f961000ed30 comp rx=0 tx=0).stop 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 shutdown_connections 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9628160910 0x7f9628133c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281466b0 0x7f96281e6750 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 --2- 172.21.15.46:0/272827905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96280720e0 0x7f96281f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 >> 172.21.15.46:0/272827905 conn(0x7f96280b26d0 msgr2=0x7f96281601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 shutdown_connections 2024-04-26T23:56:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.163+0000 7f962e200640 1 -- 172.21.15.46:0/272827905 wait complete. 2024-04-26T23:56:38.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:56:38.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.374+0000 7f4070e00640 1 Processor -- start 2024-04-26T23:56:38.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.374+0000 7f4070e00640 1 -- start start 2024-04-26T23:56:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.375+0000 7f4070e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.375+0000 7f4070e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f406c072190 0x7f406c166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.375+0000 7f4070e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c06f1d0 con 0x7f406c0cf450 2024-04-26T23:56:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.375+0000 7f4070e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f406c06f470 con 0x7f406c072190 2024-04-26T23:56:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.375+0000 7f4070e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f406c072930 con 0x7f406c0cff40 2024-04-26T23:56:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f406c0cf450 0x7f406c0cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59638/0 (socket says 172.21.15.46:59638) 2024-04-26T23:56:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406b400640 1 -- 172.21.15.46:0/810917113 learned_addr learned my addr 172.21.15.46:0/810917113 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406be00640 1 --2- 172.21.15.46:0/810917113 >> v2:172.21.15.46:3300/0 conn(0x7f406c072190 0x7f406c166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406aa00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406a000640 1 -- 172.21.15.46:0/810917113 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090097977 0 0) 0x7f406c06f1d0 con 0x7f406c0cf450 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406a000640 1 -- 172.21.15.46:0/810917113 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4050003520 con 0x7f406c0cf450 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406be00640 1 -- 172.21.15.46:0/810917113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 msgr2=0x7f406c073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406be00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406be00640 1 -- 172.21.15.46:0/810917113 >> v1:172.21.15.46:6789/0 conn(0x7f406c0cf450 legacy=0x7f406c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.376+0000 7f406be00640 1 -- 172.21.15.46:0/810917113 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c072c10 con 0x7f406c072190 2024-04-26T23:56:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.377+0000 7f406be00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c072190 0x7f406c166b00 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67256293f28352 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f406a000640 1 -- 172.21.15.46:0/810917113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064037070 con 0x7f406c072190 2024-04-26T23:56:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f406a000640 1 -- 172.21.15.46:0/810917113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064032440 con 0x7f406c072190 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f406a000640 1 -- 172.21.15.46:0/810917113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640327f0 con 0x7f406c072190 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 -- 172.21.15.46:0/810917113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c072190 msgr2=0x7f406c166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c072190 0x7f406c166b00 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a180 comp rx=0 tx=0).stop 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 -- 172.21.15.46:0/810917113 shutdown_connections 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c072190 0x7f406c166b00 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 --2- 172.21.15.46:0/810917113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.378+0000 7f4070e00640 1 -- 172.21.15.46:0/810917113 >> 172.21.15.46:0/810917113 conn(0x7f406c0b2820 msgr2=0x7f406c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.379+0000 7f4070e00640 1 -- 172.21.15.46:0/810917113 shutdown_connections 2024-04-26T23:56:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.379+0000 7f4070e00640 1 -- 172.21.15.46:0/810917113 wait complete. 2024-04-26T23:56:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.380+0000 7f4070e00640 1 Processor -- start 2024-04-26T23:56:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.381+0000 7f4070e00640 1 -- start start 2024-04-26T23:56:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.381+0000 7f4070e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 0x7f406c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f4070e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f4070e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c1fc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f4070e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0734b0 con 0x7f406c0cf450 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f4070e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0731d0 con 0x7f406c072190 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f4070e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f406c072600 con 0x7f406c0cff40 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48182/0 (socket says 172.21.15.46:48182) 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 -- 172.21.15.46:0/3476694390 learned_addr learned my addr 172.21.15.46:0/3476694390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406b400640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 0x7f406c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406be00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c1fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 -- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 msgr2=0x7f406c1fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c1fc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 -- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 msgr2=0x7f406c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 0x7f406c0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406aa00640 1 -- 172.21.15.46:0/3476694390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c1e7850 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.382+0000 7f406b400640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 0x7f406c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f406aa00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f405800d800 tx=0x7f405800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800b6c0 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4058019070 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c1e7de0 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800ba80 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.383+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c1e83a0 con 0x7f406c0cf450 2024-04-26T23:56:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.385+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c0b4b80 con 0x7f406c0cf450 2024-04-26T23:56:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.385+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f405800bc20 con 0x7f406c0cf450 2024-04-26T23:56:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.386+0000 7f4063400640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 0x7f404c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.387+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f405808c1e0 con 0x7f406c0cf450 2024-04-26T23:56:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.387+0000 7f406b400640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 0x7f404c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.390+0000 7f406b400640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 0x7f404c06edb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f405c005c00 tx=0x7f405c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.392+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f405801c360 con 0x7f406c0cf450 2024-04-26T23:56:38.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.642+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f406c0693b0 con 0x7f406c0cf450 2024-04-26T23:56:38.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.644+0000 7f4063400640 1 -- 172.21.15.46:0/3476694390 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v158) ==== 53+0+73 (secure 0 0 0) 0x7f405805a470 con 0x7f406c0cf450 2024-04-26T23:56:38.646 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:56:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.647+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 msgr2=0x7f404c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.647+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 0x7f404c06edb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f405c005c00 tx=0x7f405c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:38.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.647+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 msgr2=0x7f406c13d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.647+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f405800d800 tx=0x7f405800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 shutdown_connections 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f404c06c8f0 0x7f404c06edb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f406c0cff40 0x7f406c1fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0cf450 0x7f406c13d600 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 --2- 172.21.15.46:0/3476694390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f406c072190 0x7f406c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 >> 172.21.15.46:0/3476694390 conn(0x7f406c0b2820 msgr2=0x7f406c142de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 shutdown_connections 2024-04-26T23:56:38.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38.648+0000 7f4070e00640 1 -- 172.21.15.46:0/3476694390 wait complete. 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:56:38.668 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:56:38.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:38.669 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:56:38.719 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:56:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41422sec preferred_lft 41422sec 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.721 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:56:38.722 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:56:38.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:38.723 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:56:38.804 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:56:38.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:38.808 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:56:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:56:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:56:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:56:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:56:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:56:38.862 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:56:38.862 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:56:38.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:56:38.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:56:38.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:56:38.863 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:56:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:56:39.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:56:39.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:39.074 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:56:39.120 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:56:39.120 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:56:39.121 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:56:39.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:39.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:56:39.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:56:39.254 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:56:39.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:56:39.310 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:39.366 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:56:39.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:39.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:56:39.498 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:56:39.498 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:56:39.498 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:56:39.520 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:56:39.520 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:56:39.520 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:56:39.520 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:56:39.520 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:56:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:56:39.548 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:56:39.548 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:56:39.549 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:56:39.549 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:56:39.549 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:56:39.549 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:56:39.549 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:56:39.554 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:56:39.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:39.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:39.631 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:56:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:56:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:56:39.762 INFO:teuthology.orchestra.run.smithi046.stdout:client8128 2024-04-26T23:56:39.762 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:39.762 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:39.820 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:39.820 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> import uuid 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> n = 1000 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = False 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> if 0 > 0: 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> create_hard_links = True 2024-04-26T23:56:39.876 DEBUG:teuthology.orchestra.run.smithi046:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> dpath = os.path.dirname(path) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> fnameprefix = os.path.basename(path) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> os.makedirs(dpath, exist_ok=True) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> for i in range(n): 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> f.write(f"{i}") 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> f.flush() 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(f.fileno()) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> os.unlink(fpath) 2024-04-26T23:56:39.877 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> if create_hard_links: 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> for j in range(0): 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> os.fsync(dirfd) 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> finally: 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> os.close(dirfd) 2024-04-26T23:56:39.878 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-26T23:56:42.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:56:42.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:42.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-26T23:56:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.688+0000 7f1563e00640 1 Processor -- start 2024-04-26T23:56:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.688+0000 7f1563e00640 1 -- start start 2024-04-26T23:56:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.689+0000 7f1563e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f15640d3830 0x7f15640d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.689+0000 7f1563e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.689+0000 7f1563e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15640d2100 con 0x7f1564162a70 2024-04-26T23:56:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.689+0000 7f1563e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f15640d3400 con 0x7f15640d3830 2024-04-26T23:56:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.689+0000 7f1563e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f15640d2370 con 0x7f15640d1cb0 2024-04-26T23:56:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.690+0000 7f1563400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.690+0000 7f1562000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f15640d3830 0x7f15640d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.690+0000 7f1563400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:57846/0 (socket says 172.21.15.46:57846) 2024-04-26T23:56:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.690+0000 7f1563400640 1 -- 172.21.15.46:0/3479517761 learned_addr learned my addr 172.21.15.46:0/3479517761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.691+0000 7f1563400640 1 -- 172.21.15.46:0/3479517761 >> v2:172.21.15.142:3300/0 conn(0x7f15640d3830 msgr2=0x7f15640d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.691+0000 7f1563400640 1 --2- 172.21.15.46:0/3479517761 >> v2:172.21.15.142:3300/0 conn(0x7f15640d3830 0x7f15640d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.691+0000 7f1563400640 1 -- 172.21.15.46:0/3479517761 >> v1:172.21.15.142:6789/0 conn(0x7f1564162a70 legacy=0x7f15640d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.691+0000 7f1563400640 1 -- 172.21.15.46:0/3479517761 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15640d2650 con 0x7f15640d1cb0 2024-04-26T23:56:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.691+0000 7f1561600640 1 -- 172.21.15.46:0/3479517761 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060850516 0 0) 0x7f15640d2100 con 0x7f1564162a70 2024-04-26T23:56:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.692+0000 7f1563400640 1 --2- 172.21.15.46:0/3479517761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f155c009870 tx=0x7f155c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f495be3953a43624 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.693+0000 7f1561600640 1 -- 172.21.15.46:0/3479517761 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c037070 con 0x7f15640d1cb0 2024-04-26T23:56:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.693+0000 7f1561600640 1 -- 172.21.15.46:0/3479517761 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155c02ae50 con 0x7f15640d1cb0 2024-04-26T23:56:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.693+0000 7f1561600640 1 -- 172.21.15.46:0/3479517761 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c032860 con 0x7f15640d1cb0 2024-04-26T23:56:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 -- 172.21.15.46:0/3479517761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 msgr2=0x7f15640b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 --2- 172.21.15.46:0/3479517761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f155c009870 tx=0x7f155c02a300 comp rx=0 tx=0).stop 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 -- 172.21.15.46:0/3479517761 shutdown_connections 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 --2- 172.21.15.46:0/3479517761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d1cb0 0x7f15640b29e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 --2- 172.21.15.46:0/3479517761 >> v2:172.21.15.142:3300/0 conn(0x7f15640d3830 0x7f15640d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 -- 172.21.15.46:0/3479517761 >> 172.21.15.46:0/3479517761 conn(0x7f15640d4f20 msgr2=0x7f15640d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 -- 172.21.15.46:0/3479517761 shutdown_connections 2024-04-26T23:56:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.694+0000 7f1563e00640 1 -- 172.21.15.46:0/3479517761 wait complete. 2024-04-26T23:56:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.696+0000 7f1563e00640 1 Processor -- start 2024-04-26T23:56:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.696+0000 7f1563e00640 1 -- start start 2024-04-26T23:56:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15640d1cb0 0x7f15640b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d3830 0x7f15641f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1564162fd0 con 0x7f1564162a70 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1564162e50 con 0x7f15640d1cb0 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f15640b2f20 con 0x7f15640d3830 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40828/0 (socket says 172.21.15.46:40828) 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1563400640 1 -- 172.21.15.46:0/2947496422 learned_addr learned my addr 172.21.15.46:0/2947496422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1562a00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15640d1cb0 0x7f15640b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.697+0000 7f1562000640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d3830 0x7f15641f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 -- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d3830 msgr2=0x7f15641f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d3830 0x7f15641f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 -- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15640d1cb0 msgr2=0x7f15640b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15640d1cb0 0x7f15640b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 -- 172.21.15.46:0/2947496422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15640ba7a0 con 0x7f1564162a70 2024-04-26T23:56:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563400640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f155c009990 tx=0x7f155c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f155b400640 1 -- 172.21.15.46:0/2947496422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c041020 con 0x7f1564162a70 2024-04-26T23:56:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f155b400640 1 -- 172.21.15.46:0/2947496422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155c03c870 con 0x7f1564162a70 2024-04-26T23:56:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.698+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15640baa80 con 0x7f1564162a70 2024-04-26T23:56:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.699+0000 7f155b400640 1 -- 172.21.15.46:0/2947496422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c037040 con 0x7f1564162a70 2024-04-26T23:56:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.699+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15640c8e60 con 0x7f1564162a70 2024-04-26T23:56:42.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.701+0000 7f155b400640 1 -- 172.21.15.46:0/2947496422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f155c044020 con 0x7f1564162a70 2024-04-26T23:56:42.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.702+0000 7f155b400640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 0x7f154406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.702+0000 7f155b400640 1 -- 172.21.15.46:0/2947496422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f155c0ac4d0 con 0x7f1564162a70 2024-04-26T23:56:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.703+0000 7f156cc0e740 1 Processor -- start 2024-04-26T23:56:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.703+0000 7f1562a00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 0x7f154406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.703+0000 7f156cc0e740 1 -- start start 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd968fb0 0x563fcd969500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd9213f0 0x563fcd967670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fcd831030 con 0x563fcd94b710 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fcd84c7f0 con 0x563fcd9213f0 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563fcd8077e0 con 0x563fcd968fb0 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f1562a00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 0x7f154406edb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1554000f60 tx=0x7f1554009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f1562a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f1562a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40840/0 (socket says 172.21.15.46:40840) 2024-04-26T23:56:42.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f1562a00640 1 -- 172.21.15.46:0/3486876281 learned_addr learned my addr 172.21.15.46:0/3486876281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.704+0000 7f1562000640 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd968fb0 0x563fcd969500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1563400640 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd9213f0 0x563fcd967670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 -- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd968fb0 msgr2=0x563fcd969500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd968fb0 0x563fcd969500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 -- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd9213f0 msgr2=0x563fcd967670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd9213f0 0x563fcd967670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 -- 172.21.15.46:0/3486876281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcd813050 con 0x563fcd94b710 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f1562a00640 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f1554005300 tx=0x7f1554011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2375596e4cafcb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f154f400640 1 -- 172.21.15.46:0/3486876281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400e500 con 0x563fcd94b710 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f154f400640 1 -- 172.21.15.46:0/3486876281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155400e6a0 con 0x563fcd94b710 2024-04-26T23:56:42.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.705+0000 7f154f400640 1 -- 172.21.15.46:0/3486876281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400ea50 con 0x563fcd94b710 2024-04-26T23:56:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 -- 172.21.15.46:0/3486876281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 msgr2=0x563fcd94baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f1554005300 tx=0x7f1554011000 comp rx=0 tx=0).stop 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 -- 172.21.15.46:0/3486876281 shutdown_connections 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd9213f0 0x563fcd967670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd968fb0 0x563fcd969500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3486876281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd94baf0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 -- 172.21.15.46:0/3486876281 >> 172.21.15.46:0/3486876281 conn(0x563fcd91dd20 msgr2=0x563fcd966520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 -- 172.21.15.46:0/3486876281 shutdown_connections 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.706+0000 7f156cc0e740 1 -- 172.21.15.46:0/3486876281 wait complete. 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.707+0000 7f156cc0e740 1 -- start start 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.707+0000 7f156cc0e740 1 Processor -- start 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd966150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fcd889200 con 0x563fcd968fb0 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fcd883ee0 con 0x563fcd94b710 2024-04-26T23:56:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563fcd890e70 con 0x563fcd94dd80 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.708+0000 7f156cc0e740 10 client.0 _ll_register_callbacks cb 0x563fcd87a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40852/0 (socket says 172.21.15.46:40852) 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 -- 172.21.15.46:0/388157037 learned_addr learned my addr 172.21.15.46:0/388157037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1563400640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562a00640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd966150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 -- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 msgr2=0x563fcd95e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 -- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 msgr2=0x563fcd966150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd966150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562000640 1 -- 172.21.15.46:0/388157037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcd6607d0 con 0x563fcd968fb0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.709+0000 7f1562a00640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd966150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f1562000640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f1550007e00 tx=0x7f1550012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaa8e0e9a018d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f1543e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155001c020 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f1550018670 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f155400e6a0 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7f1550026050 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f156cc0e740 10 client.11850 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.710+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x563fcd5963f0 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155400e6a0 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7f1550029020 con 0x563fcd968fb0 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 10 client.11850 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 10 client.11850 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 20 client.11850 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd939780 0x563fcd939bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 4 client.11850 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fcd8927e0 con 0x563fcd939780 2024-04-26T23:56:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.711+0000 7f1562a00640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd939780 0x563fcd939bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.712+0000 7f1562a00640 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd939780 0x563fcd939bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb347cb479348dd server_cookie=23d34645780888a3 in_seq=0 out_seq=0 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.712+0000 7f1543e00640 10 client.11850 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.712+0000 7f1543e00640 1 -- 172.21.15.46:0/388157037 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fcd8927e0 con 0x563fcd939780 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.712+0000 7f1543e00640 10 client.11850 handle_command_reply: tid=0 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.712+0000 7f156cc0e740 1 client.11850 shutdown 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 msgr2=0x563fcd94d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f1550007e00 tx=0x7f1550012610 comp rx=0 tx=0).stop 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 shutdown_connections 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd939780 0x563fcd939bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd94d840 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 --2- 172.21.15.46:0/388157037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd966150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f1543e00640 0 client.11850 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 >> 172.21.15.46:0/388157037 conn(0x563fcd9213f0 msgr2=0x563fcd96a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 shutdown_connections 2024-04-26T23:56:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 1 -- 172.21.15.46:0/388157037 wait complete. 2024-04-26T23:56:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.713+0000 7f156cc0e740 20 client.11850 trim_cache size 0 max 16384 2024-04-26T23:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.732+0000 7f156cc0e740 1 Processor -- start 2024-04-26T23:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.732+0000 7f156cc0e740 1 -- start start 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd9da100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 0x563fcd9758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1564201ad0 con 0x563fcd94b710 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1564200bb0 con 0x563fcd968fb0 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.733+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f15641f55c0 con 0x563fcd94dd80 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40864/0 (socket says 172.21.15.46:40864) 2024-04-26T23:56:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 -- 172.21.15.46:0/534828903 learned_addr learned my addr 172.21.15.46:0/534828903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 -- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 msgr2=0x563fcd9da100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd9da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1563400640 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 0x563fcd9758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 -- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 msgr2=0x563fcd9758f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 0x563fcd9758f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1562a00640 1 -- 172.21.15.46:0/534828903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcd5560f0 con 0x563fcd94b710 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.734+0000 7f1563400640 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 0x563fcd9758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f1562a00640 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f155401e000 tx=0x7f155401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d994fefc23905a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f154ea00640 1 -- 172.21.15.46:0/534828903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155402c450 con 0x563fcd94b710 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f154ea00640 1 -- 172.21.15.46:0/534828903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155402c5f0 con 0x563fcd94b710 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f154ea00640 1 -- 172.21.15.46:0/534828903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155402c7a0 con 0x563fcd94b710 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 -- 172.21.15.46:0/534828903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 msgr2=0x563fcd9227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f155401e000 tx=0x7f155401d000 comp rx=0 tx=0).stop 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 -- 172.21.15.46:0/534828903 shutdown_connections 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd968fb0 0x563fcd9758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd9da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 --2- 172.21.15.46:0/534828903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd94b710 0x563fcd9227c0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.735+0000 7f156cc0e740 1 -- 172.21.15.46:0/534828903 >> 172.21.15.46:0/534828903 conn(0x563fcd9213f0 msgr2=0x563fcd95e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.736+0000 7f156cc0e740 1 -- 172.21.15.46:0/534828903 shutdown_connections 2024-04-26T23:56:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.736+0000 7f156cc0e740 1 -- 172.21.15.46:0/534828903 wait complete. 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.736+0000 7f156cc0e740 1 -- start start 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.736+0000 7f156cc0e740 1 Processor -- start 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd94ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fcd891fd0 con 0x563fcd968fb0 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15641f5c10 con 0x563fcd94b710 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563fcd91c520 con 0x563fcd94dd80 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.737+0000 7f156cc0e740 10 client.0 _ll_register_callbacks cb 0x7f1550009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40870/0 (socket says 172.21.15.46:40870) 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 -- 172.21.15.46:0/3801224895 learned_addr learned my addr 172.21.15.46:0/3801224895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1562a00640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd94ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 -- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 msgr2=0x563fcd95c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 -- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 msgr2=0x563fcd94ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd94ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 -- 172.21.15.46:0/3801224895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcd84ca30 con 0x563fcd968fb0 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1563400640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f155c0aa7d0 tx=0x7f155c07a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e710a5dfcfb51041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.738+0000 7f1543e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c092020 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f155c0ac0b0 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f155402c5f0 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7f155c097050 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 10 client.11856 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x563fcd9d8b70 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155402c5f0 con 0x563fcd968fb0 2024-04-26T23:56:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7f155c07a2c0 con 0x563fcd968fb0 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 10 client.11856 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 10 client.11856 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 20 client.11856 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd917620 0x563fcd94c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 4 client.11856 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.739+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fcd88c650 con 0x563fcd917620 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.740+0000 7f1562a00640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd917620 0x563fcd94c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.740+0000 7f1562a00640 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd917620 0x563fcd94c3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96decd3f9fa5063 server_cookie=18c54ce59f62478c in_seq=0 out_seq=0 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.740+0000 7f1543e00640 10 client.11856 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.740+0000 7f1543e00640 1 -- 172.21.15.46:0/3801224895 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fcd88c650 con 0x563fcd917620 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.740+0000 7f1543e00640 10 client.11856 handle_command_reply: tid=0 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 client.11856 shutdown 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 msgr2=0x563fcd975b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f155c0aa7d0 tx=0x7f155c07a760 comp rx=0 tx=0).stop 2024-04-26T23:56:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 shutdown_connections 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fcd968fb0 0x563fcd975b90 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x563fcd917620 0x563fcd94c3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563fcd94dd80 0x563fcd95c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 --2- 172.21.15.46:0/3801224895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fcd94b710 0x563fcd94ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f1543e00640 0 client.11856 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 >> 172.21.15.46:0/3801224895 conn(0x563fcd9213f0 msgr2=0x563fcd95d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 shutdown_connections 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 1 -- 172.21.15.46:0/3801224895 wait complete. 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.741+0000 7f156cc0e740 20 client.11856 trim_cache size 0 max 16384 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:56:42.745 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:56:42.746 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:56:42.746 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 msgr2=0x7f154406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 0x7f154406edb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1554000f60 tx=0x7f1554009040 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 msgr2=0x7f15641542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f155c009990 tx=0x7f155c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 shutdown_connections 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f154406c8f0 0x7f154406edb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1564162a70 0x7f15641542b0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640d3830 0x7f15641f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 --2- 172.21.15.46:0/2947496422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15640d1cb0 0x7f15640b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 >> 172.21.15.46:0/2947496422 conn(0x7f15640d4f20 msgr2=0x7f1564161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 shutdown_connections 2024-04-26T23:56:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.742+0000 7f1563e00640 1 -- 172.21.15.46:0/2947496422 wait complete. 2024-04-26T23:56:42.761 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:56:42.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.14:0 damage ls 2024-04-26T23:56:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.973+0000 7feb31600640 1 Processor -- start 2024-04-26T23:56:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.973+0000 7feb31600640 1 -- start start 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.975+0000 7feb31600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb2c145c10 0x7feb2c073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.975+0000 7feb31600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb2c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.975+0000 7feb31600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c06ec30 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.975+0000 7feb31600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c145ff0 con 0x7feb2c145c10 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.975+0000 7feb31600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c0d1230 con 0x7feb2c0cfed0 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb30c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feb2c0d0dc0 0x7feb2c071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33018/0 (socket says 172.21.15.46:33018) 2024-04-26T23:56:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb30c00640 1 -- 172.21.15.46:0/4227220368 learned_addr learned my addr 172.21.15.46:0/4227220368 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb2be00640 1 --2- 172.21.15.46:0/4227220368 >> v2:172.21.15.142:3300/0 conn(0x7feb2c145c10 0x7feb2c073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb2b400640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb2c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711278639 0 0) 0x7feb2c06ec30 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.976+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb10003520 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201363390 0 0) 0x7feb10003520 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 msgr2=0x7feb2c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb2c0d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 >> v2:172.21.15.142:3300/0 conn(0x7feb2c145c10 msgr2=0x7feb2c073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 --2- 172.21.15.46:0/4227220368 >> v2:172.21.15.142:3300/0 conn(0x7feb2c145c10 0x7feb2c073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0cf3f0 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7feb240019f0 con 0x7feb2c0d0dc0 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 >> v1:172.21.15.46:6789/0 conn(0x7feb2c0d0dc0 legacy=0x7feb2c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb100039e0 0x7feb10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c145ff0 con 0x7feb10004d40 2024-04-26T23:56:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0d1230 con 0x7feb100039e0 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb1000fdd0 con 0x7feb2c0cfed0 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb30c00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2b400640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.977+0000 7feb2be00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb100039e0 0x7feb10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.978+0000 7feb30c00640 1 -- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 msgr2=0x7feb10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.978+0000 7feb30c00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c0cfed0 0x7feb10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.978+0000 7feb30c00640 1 -- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb100039e0 msgr2=0x7feb10003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.978+0000 7feb30c00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb100039e0 0x7feb10003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.978+0000 7feb30c00640 1 -- 172.21.15.46:0/4227220368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10003520 con 0x7feb10004d40 2024-04-26T23:56:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb30c00640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb100051c0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a840 tx=0x7feb10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49d07a27050191c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400a460 con 0x7feb10004d40 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2402f070 con 0x7feb10004d40 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb2aa00640 1 -- 172.21.15.46:0/4227220368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2402bca0 con 0x7feb10004d40 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb31600640 1 -- 172.21.15.46:0/4227220368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 msgr2=0x7feb100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.979+0000 7feb31600640 1 --2- 172.21.15.46:0/4227220368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb100051c0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a840 tx=0x7feb10005830 comp rx=0 tx=0).stop 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.980+0000 7feb30c00640 1 -- 172.21.15.46:0/4227220368 reap_dead start 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.980+0000 7feb31600640 1 -- 172.21.15.46:0/4227220368 shutdown_connections 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.980+0000 7feb31600640 1 -- 172.21.15.46:0/4227220368 >> 172.21.15.46:0/4227220368 conn(0x7feb2c06c350 msgr2=0x7feb2c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.980+0000 7feb31600640 1 -- 172.21.15.46:0/4227220368 shutdown_connections 2024-04-26T23:56:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.980+0000 7feb31600640 1 -- 172.21.15.46:0/4227220368 wait complete. 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.982+0000 7feb31600640 1 Processor -- start 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.982+0000 7feb31600640 1 -- start start 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c1434a0 0x7feb2c143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c13e670 0x7feb2c13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0cf0a0 con 0x7feb10004d40 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0cf9b0 con 0x7feb2c13e670 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb31600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c1468b0 con 0x7feb2c1434a0 2024-04-26T23:56:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40882/0 (socket says 172.21.15.46:40882) 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 -- 172.21.15.46:0/1756570959 learned_addr learned my addr 172.21.15.46:0/1756570959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2b400640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c1434a0 0x7feb2c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 -- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c1434a0 msgr2=0x7feb2c143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c1434a0 0x7feb2c143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 -- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c13e670 msgr2=0x7feb2c13ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c13e670 0x7feb2c13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.983+0000 7feb2be00640 1 -- 172.21.15.46:0/1756570959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c13b490 con 0x7feb10004d40 2024-04-26T23:56:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb2be00640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7feb1c011630 tx=0x7feb1c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb23e00640 1 -- 172.21.15.46:0/1756570959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c015870 con 0x7feb10004d40 2024-04-26T23:56:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb23e00640 1 -- 172.21.15.46:0/1756570959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c015a10 con 0x7feb10004d40 2024-04-26T23:56:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb23e00640 1 -- 172.21.15.46:0/1756570959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c015dc0 con 0x7feb10004d40 2024-04-26T23:56:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c1420f0 con 0x7feb10004d40 2024-04-26T23:56:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.984+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c1426b0 con 0x7feb10004d40 2024-04-26T23:56:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.987+0000 7feb23e00640 1 -- 172.21.15.46:0/1756570959 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb1c00f490 con 0x7feb10004d40 2024-04-26T23:56:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.988+0000 7feb23e00640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 0x7feb0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.989+0000 7feb23e00640 1 -- 172.21.15.46:0/1756570959 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7feb1c08c680 con 0x7feb10004d40 2024-04-26T23:56:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.989+0000 7feb35bae740 1 Processor -- start 2024-04-26T23:56:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.989+0000 7feb2b400640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 0x7feb0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.989+0000 7feb35bae740 1 -- start start 2024-04-26T23:56:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x5616414b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x5616414b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616413eac70 con 0x561641482100 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5616413a3980 con 0x5616414af880 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5616413e6c30 con 0x5616414af4a0 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb2b400640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 0x7feb0c06edb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7feb2c073100 tx=0x7feb18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40892/0 (socket says 172.21.15.46:40892) 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.990+0000 7feb30c00640 1 -- 172.21.15.46:0/1116146407 learned_addr learned my addr 172.21.15.46:0/1116146407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb2be00640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x5616414b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb2b400640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x5616414b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 -- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 msgr2=0x5616414b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x5616414b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 -- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 msgr2=0x5616414b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x5616414b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 -- 172.21.15.46:0/1116146407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164137b7b0 con 0x561641482100 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb2b400640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x5616414b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb30c00640 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a960 tx=0x7feb2402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d331a30372c752a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb17e00640 1 -- 172.21.15.46:0/1116146407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240019b0 con 0x561641482100 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb17e00640 1 -- 172.21.15.46:0/1116146407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2402f040 con 0x561641482100 2024-04-26T23:56:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.991+0000 7feb17e00640 1 -- 172.21.15.46:0/1116146407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24004d80 con 0x561641482100 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 -- 172.21.15.46:0/1116146407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 msgr2=0x561641485910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a960 tx=0x7feb2402d000 comp rx=0 tx=0).stop 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 -- 172.21.15.46:0/1116146407 shutdown_connections 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561641482100 0x561641485910 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x5616414b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 --2- 172.21.15.46:0/1116146407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x5616414b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 -- 172.21.15.46:0/1116146407 >> 172.21.15.46:0/1116146407 conn(0x561641484370 msgr2=0x5616414cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 -- 172.21.15.46:0/1116146407 shutdown_connections 2024-04-26T23:56:42.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.992+0000 7feb35bae740 1 -- 172.21.15.46:0/1116146407 wait complete. 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.993+0000 7feb35bae740 1 -- start start 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.993+0000 7feb35bae740 1 Processor -- start 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af4a0 0x5616414a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414cf000 0x5616414cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616413e9f80 con 0x5616414af880 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561641397260 con 0x5616414af4a0 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5616413ef2a0 con 0x5616414cf000 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb35bae740 10 client.0 _ll_register_callbacks cb 0x5616413e1960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40902/0 (socket says 172.21.15.46:40902) 2024-04-26T23:56:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.994+0000 7feb2b400640 1 -- 172.21.15.46:0/3453809181 learned_addr learned my addr 172.21.15.46:0/3453809181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb30c00640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414cf000 0x5616414cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2be00640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af4a0 0x5616414a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 -- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414cf000 msgr2=0x5616414cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414cf000 0x5616414cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 -- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af4a0 msgr2=0x5616414a8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af4a0 0x5616414a8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 -- 172.21.15.46:0/3453809181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616413791c0 con 0x5616414af880 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb2b400640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7feb180157a0 tx=0x7feb18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4480721207817c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.995+0000 7feb14c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18026070 con 0x5616414af880 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2402f040 con 0x5616414af880 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7feb18014b30 con 0x5616414af880 2024-04-26T23:56:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7feb18031050 con 0x5616414af880 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 10 client.11868 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x5616410fc3f0 con 0x5616414af880 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2402f040 con 0x5616414af880 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7feb18034020 con 0x5616414af880 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 10 client.11868 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 10 client.11868 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 20 client.11868 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414a8b40 0x5616414cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 4 client.11868 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.996+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616413f8880 con 0x5616414a8b40 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.997+0000 7feb2be00640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414a8b40 0x5616414cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.997+0000 7feb2be00640 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414a8b40 0x5616414cdde0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a4aec04f08781b server_cookie=ffcb90689bc151df in_seq=0 out_seq=0 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.997+0000 7feb14c00640 10 client.11868 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:43.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb14c00640 1 -- 172.21.15.46:0/3453809181 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616413f8880 con 0x5616414a8b40 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb14c00640 10 client.11868 handle_command_reply: tid=0 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 client.11868 shutdown 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 msgr2=0x5616414cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7feb180157a0 tx=0x7feb18004730 comp rx=0 tx=0).stop 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 shutdown_connections 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414a8b40 0x5616414cdde0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414cf000 0x5616414cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af880 0x5616414cfca0 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb14c00640 0 client.11868 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 --2- 172.21.15.46:0/3453809181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af4a0 0x5616414a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.998+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 >> 172.21.15.46:0/3453809181 conn(0x561641482100 msgr2=0x561641487730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.999+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 shutdown_connections 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.999+0000 7feb35bae740 1 -- 172.21.15.46:0/3453809181 wait complete. 2024-04-26T23:56:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:42.999+0000 7feb35bae740 20 client.11868 trim_cache size 0 max 16384 2024-04-26T23:56:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.014+0000 7feb35bae740 1 Processor -- start 2024-04-26T23:56:43.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.014+0000 7feb35bae740 1 -- start start 2024-04-26T23:56:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af880 0x5616414effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414cf000 0x5616414f0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c1fe050 con 0x5616414af4a0 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c1fd050 con 0x5616414cf000 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c1f22e0 con 0x5616414af880 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40906/0 (socket says 172.21.15.46:40906) 2024-04-26T23:56:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 -- 172.21.15.46:0/1070430822 learned_addr learned my addr 172.21.15.46:0/1070430822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 -- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af880 msgr2=0x5616414effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af880 0x5616414effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 -- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414cf000 msgr2=0x5616414f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414cf000 0x5616414f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.015+0000 7feb2be00640 1 -- 172.21.15.46:0/1070430822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616411c67d0 con 0x5616414af4a0 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb2be00640 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00f460 tx=0x7feb1c08a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1c1381ea1b9333 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb17400640 1 -- 172.21.15.46:0/1070430822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c08aa90 con 0x5616414af4a0 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb17400640 1 -- 172.21.15.46:0/1070430822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c058480 con 0x5616414af4a0 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb17400640 1 -- 172.21.15.46:0/1070430822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c05e120 con 0x5616414af4a0 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb35bae740 1 -- 172.21.15.46:0/1070430822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 msgr2=0x5616414c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.016+0000 7feb35bae740 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00f460 tx=0x7feb1c08a850 comp rx=0 tx=0).stop 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 -- 172.21.15.46:0/1070430822 shutdown_connections 2024-04-26T23:56:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414cf000 0x5616414f0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af880 0x5616414effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 --2- 172.21.15.46:0/1070430822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414af4a0 0x5616414c9840 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 -- 172.21.15.46:0/1070430822 >> 172.21.15.46:0/1070430822 conn(0x561641482100 msgr2=0x5616414827d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 -- 172.21.15.46:0/1070430822 shutdown_connections 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 -- 172.21.15.46:0/1070430822 wait complete. 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.017+0000 7feb35bae740 1 -- start start 2024-04-26T23:56:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.018+0000 7feb35bae740 1 Processor -- start 2024-04-26T23:56:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x561641487920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x561641547ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616413ed110 con 0x5616414cf000 2024-04-26T23:56:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c1fca00 con 0x5616414af880 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5616414b5880 con 0x5616414af4a0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb35bae740 10 client.0 _ll_register_callbacks cb 0x7feb18006dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40912/0 (socket says 172.21.15.46:40912) 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 -- 172.21.15.46:0/3363667540 learned_addr learned my addr 172.21.15.46:0/3363667540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb2be00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x561641487920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 -- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 msgr2=0x561641487920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x561641487920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 -- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 msgr2=0x561641547ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x561641547ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.019+0000 7feb30c00640 1 -- 172.21.15.46:0/3363667540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616413912a0 con 0x5616414cf000 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb30c00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a960 tx=0x7feb2c06ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4cfbd711565f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb14c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:56:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2404e060 con 0x5616414cf000 2024-04-26T23:56:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7feb24004d70 con 0x5616414cf000 2024-04-26T23:56:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb1c058480 con 0x5616414cf000 2024-04-26T23:56:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) ==== 24+0+0 (secure 0 0 0) 0x7feb2402baf0 con 0x5616414cf000 2024-04-26T23:56:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb35bae740 10 client.11874 fetch_fsmap learned FSMap version 158 2024-04-26T23:56:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.020+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=2+}) -- 0x561641487e60 con 0x5616414cf000 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c058480 con 0x5616414cf000 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 158) ==== 3494+0+0 (secure 0 0 0) 0x7feb24057020 con 0x5616414cf000 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 10 client.11874 fetch_fsmap finished waiting for FSMap version 158 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 10 client.11874 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 20 client.11874 populate_metadata read hostname 'smithi046' 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414cccb0 0x5616414cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 4 client.11874 mds_command: new command op to 11688 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 --> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616413f26f0 con 0x5616414cccb0 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.021+0000 7feb2be00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414cccb0 0x5616414cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb2be00640 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414cccb0 0x5616414cdde0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7caf105fa038e0b server_cookie=b10c54f97d0eff6b in_seq=0 out_seq=0 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb14c00640 10 client.11874 ms_handle_connect on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb14c00640 1 -- 172.21.15.46:0/3363667540 <== mds.0 v2:172.21.15.46:6832/4111191104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616413f26f0 con 0x5616414cccb0 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb14c00640 10 client.11874 handle_command_reply: tid=0 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 client.11874 shutdown 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 msgr2=0x561641548530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7feb2400a960 tx=0x7feb2c06ee00 comp rx=0 tx=0).stop 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 shutdown_connections 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616414cf000 0x561641548530 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5616414af880 0x561641547ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.46:6832/4111191104,v1:172.21.15.46:6834/4111191104] conn(0x5616414cccb0 0x5616414cdde0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 --2- 172.21.15.46:0/3363667540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5616414af4a0 0x561641487920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 >> 172.21.15.46:0/3363667540 conn(0x561641482100 msgr2=0x56164147d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.022+0000 7feb14c00640 0 client.11874 ms_handle_reset on v2:172.21.15.46:6832/4111191104 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 shutdown_connections 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb35bae740 1 -- 172.21.15.46:0/3363667540 wait complete. 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb35bae740 20 client.11874 trim_cache size 0 max 16384 2024-04-26T23:56:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-26T23:56:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 msgr2=0x7feb0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 0x7feb0c06edb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7feb2c073100 tx=0x7feb18008040 comp rx=0 tx=0).stop 2024-04-26T23:56:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 msgr2=0x7feb2c13af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.023+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7feb1c011630 tx=0x7feb1c011af0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 shutdown_connections 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7feb0c06c8f0 0x7feb0c06edb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c13e670 0x7feb2c13ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c1434a0 0x7feb2c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 --2- 172.21.15.46:0/1756570959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10004d40 0x7feb2c13af50 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 >> 172.21.15.46:0/1756570959 conn(0x7feb2c06c350 msgr2=0x7feb2c15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 shutdown_connections 2024-04-26T23:56:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.024+0000 7feb31600640 1 -- 172.21.15.46:0/1756570959 wait complete. 2024-04-26T23:56:43.042 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:56:43.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:43.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:43.252 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:56:43.252 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:43.252 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:43.252 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:43.310 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:43.310 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:43.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:43.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.574+0000 7f344f200640 1 Processor -- start 2024-04-26T23:56:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.574+0000 7f344f200640 1 -- start start 2024-04-26T23:56:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34500c6d50 0x7f3450073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f345015f740 0x7f34500cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500cbcb0 con 0x7f345015fcb0 2024-04-26T23:56:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f34500cba60 con 0x7f345015f740 2024-04-26T23:56:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f34500cbf90 con 0x7f34500c6d50 2024-04-26T23:56:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f345015fcb0 0x7f345015f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33028/0 (socket says 172.21.15.46:33028) 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344d400640 1 -- 172.21.15.46:0/4278992439 learned_addr learned my addr 172.21.15.46:0/4278992439 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.576+0000 7f344e800640 1 --2- 172.21.15.46:0/4278992439 >> v2:172.21.15.46:3300/0 conn(0x7f345015f740 0x7f34500cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.577+0000 7f344de00640 1 --2- 172.21.15.46:0/4278992439 >> v2:172.21.15.142:3300/0 conn(0x7f34500c6d50 0x7f3450073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.577+0000 7f344e800640 1 -- 172.21.15.46:0/4278992439 >> v2:172.21.15.142:3300/0 conn(0x7f34500c6d50 msgr2=0x7f3450073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.577+0000 7f344e800640 1 --2- 172.21.15.46:0/4278992439 >> v2:172.21.15.142:3300/0 conn(0x7f34500c6d50 0x7f3450073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.577+0000 7f344e800640 1 -- 172.21.15.46:0/4278992439 >> v1:172.21.15.46:6789/0 conn(0x7f345015fcb0 legacy=0x7f345015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.577+0000 7f344e800640 1 -- 172.21.15.46:0/4278992439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450072cd0 con 0x7f345015f740 2024-04-26T23:56:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.578+0000 7f344e800640 1 --2- 172.21.15.46:0/4278992439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345015f740 0x7f34500cee70 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f34500cded0 tx=0x7f344802b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd8897b7b2b4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344ca00640 1 -- 172.21.15.46:0/4278992439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448037070 con 0x7f345015f740 2024-04-26T23:56:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344ca00640 1 -- 172.21.15.46:0/4278992439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448008640 con 0x7f345015f740 2024-04-26T23:56:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344ca00640 1 -- 172.21.15.46:0/4278992439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448008a10 con 0x7f345015f740 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 -- 172.21.15.46:0/4278992439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345015f740 msgr2=0x7f34500cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 --2- 172.21.15.46:0/4278992439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345015f740 0x7f34500cee70 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f34500cded0 tx=0x7f344802b6f0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 -- 172.21.15.46:0/4278992439 shutdown_connections 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 --2- 172.21.15.46:0/4278992439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345015f740 0x7f34500cee70 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 --2- 172.21.15.46:0/4278992439 >> v2:172.21.15.142:3300/0 conn(0x7f34500c6d50 0x7f3450073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.579+0000 7f344f200640 1 -- 172.21.15.46:0/4278992439 >> 172.21.15.46:0/4278992439 conn(0x7f34500b26d0 msgr2=0x7f34500b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.580+0000 7f344f200640 1 -- 172.21.15.46:0/4278992439 shutdown_connections 2024-04-26T23:56:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.580+0000 7f344f200640 1 -- 172.21.15.46:0/4278992439 wait complete. 2024-04-26T23:56:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.581+0000 7f344f200640 1 Processor -- start 2024-04-26T23:56:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.582+0000 7f344f200640 1 -- start start 2024-04-26T23:56:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.582+0000 7f344f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.582+0000 7f344f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345015f740 0x7f345015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345015fcb0 0x7f345015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34500cd6a0 con 0x7f34500c6d50 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34500cd520 con 0x7f345015f740 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34500cd820 con 0x7f345015fcb0 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40928/0 (socket says 172.21.15.46:40928) 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 -- 172.21.15.46:0/2663702593 learned_addr learned my addr 172.21.15.46:0/2663702593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 -- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345015fcb0 msgr2=0x7f345015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345015fcb0 0x7f345015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 -- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345015f740 msgr2=0x7f345015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345015f740 0x7f345015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.583+0000 7f344de00640 1 -- 172.21.15.46:0/2663702593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500c8f00 con 0x7f34500c6d50 2024-04-26T23:56:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f344de00640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f343800d800 tx=0x7f343800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343800b6c0 con 0x7f34500c6d50 2024-04-26T23:56:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3438019070 con 0x7f34500c6d50 2024-04-26T23:56:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34501e9430 con 0x7f34500c6d50 2024-04-26T23:56:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343800ba80 con 0x7f34500c6d50 2024-04-26T23:56:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.584+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34501e99f0 con 0x7f34500c6d50 2024-04-26T23:56:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.585+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34500693b0 con 0x7f34500c6d50 2024-04-26T23:56:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.586+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f343800bc20 con 0x7f34500c6d50 2024-04-26T23:56:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.587+0000 7f3446000640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.587+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f343808c1e0 con 0x7f34500c6d50 2024-04-26T23:56:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.588+0000 7f344d400640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.589+0000 7f344d400640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 0x7f342c06edb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.592+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f343801c360 con 0x7f34500c6d50 2024-04-26T23:56:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.800+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3450073ea0 con 0x7f34500c6d50 2024-04-26T23:56:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.802+0000 7f3446000640 1 -- 172.21.15.46:0/2663702593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) ==== 74+0+16848 (secure 0 0 0) 0x7f343805a470 con 0x7f34500c6d50 2024-04-26T23:56:43.804 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:43.805 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":199,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:32.320531+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 msgr2=0x7f342c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 0x7f342c06edb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440007a80 comp rx=0 tx=0).stop 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 msgr2=0x7f3450153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f343800d800 tx=0x7f343800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 shutdown_connections 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345015fcb0 0x7f345015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345015f740 0x7f345015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 --2- 172.21.15.46:0/2663702593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34500c6d50 0x7f3450153780 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 >> 172.21.15.46:0/2663702593 conn(0x7f34500b26d0 msgr2=0x7f3450155590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.805+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 shutdown_connections 2024-04-26T23:56:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:43.806+0000 7f344f200640 1 -- 172.21.15.46:0/2663702593 wait complete. 2024-04-26T23:56:43.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:56:28.688282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:56:29.797117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:43.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:43.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:43.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.030+0000 7f95fc800640 1 Processor -- start 2024-04-26T23:56:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.030+0000 7f95fc800640 1 -- start start 2024-04-26T23:56:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fc800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 0x7f95f40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fc800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fc800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f40b46f0 con 0x7f95f4162aa0 2024-04-26T23:56:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fc800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f95f40b44a0 con 0x7f95f40d3d60 2024-04-26T23:56:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fc800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40b5020 con 0x7f95f40b64c0 2024-04-26T23:56:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95f9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 0x7f95f40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95fa800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95f9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 0x7f95f40cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:49996/0 (socket says 172.21.15.46:49996) 2024-04-26T23:56:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95fa800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:57952/0 (socket says 172.21.15.46:57952) 2024-04-26T23:56:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95f9400640 1 -- 172.21.15.46:0/1356858974 learned_addr learned my addr 172.21.15.46:0/1356858974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.032+0000 7f95f9e00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f95f4162aa0 0x7f95f40d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:50970/0 (socket says 172.21.15.46:50970) 2024-04-26T23:56:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95fa800640 1 -- 172.21.15.46:0/1356858974 >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 msgr2=0x7f95f40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95fa800640 1 --2- 172.21.15.46:0/1356858974 >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 0x7f95f40cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95fa800640 1 -- 172.21.15.46:0/1356858974 >> v1:172.21.15.142:6789/0 conn(0x7f95f4162aa0 legacy=0x7f95f40d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.033+0000 7f95fa800640 1 -- 172.21.15.46:0/1356858974 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f40b49d0 con 0x7f95f40b64c0 2024-04-26T23:56:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.035+0000 7f95fa800640 1 --2- 172.21.15.46:0/1356858974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b1f4860392ec2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.035+0000 7f95f8a00640 1 -- 172.21.15.46:0/1356858974 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0031600 con 0x7f95f40b64c0 2024-04-26T23:56:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.035+0000 7f95f8a00640 1 -- 172.21.15.46:0/1356858974 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f00317a0 con 0x7f95f40b64c0 2024-04-26T23:56:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.035+0000 7f95f8a00640 1 -- 172.21.15.46:0/1356858974 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0031b70 con 0x7f95f40b64c0 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 -- 172.21.15.46:0/1356858974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 msgr2=0x7f95f40d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 --2- 172.21.15.46:0/1356858974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f002a5d0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 -- 172.21.15.46:0/1356858974 shutdown_connections 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 --2- 172.21.15.46:0/1356858974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f40d01c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 --2- 172.21.15.46:0/1356858974 >> v2:172.21.15.142:3300/0 conn(0x7f95f40d3d60 0x7f95f40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 -- 172.21.15.46:0/1356858974 >> 172.21.15.46:0/1356858974 conn(0x7f95f40d5070 msgr2=0x7f95f40d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 -- 172.21.15.46:0/1356858974 shutdown_connections 2024-04-26T23:56:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.036+0000 7f95fc800640 1 -- 172.21.15.46:0/1356858974 wait complete. 2024-04-26T23:56:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.038+0000 7f95fc800640 1 Processor -- start 2024-04-26T23:56:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.038+0000 7f95fc800640 1 -- start start 2024-04-26T23:56:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95f40d3d60 0x7f95f40cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40d30f0 con 0x7f95f4162aa0 2024-04-26T23:56:44.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4162e80 con 0x7f95f40d3d60 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fc800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40d5b90 con 0x7f95f40b64c0 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.039+0000 7f95fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40936/0 (socket says 172.21.15.46:40936) 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 -- 172.21.15.46:0/974503026 learned_addr learned my addr 172.21.15.46:0/974503026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95f9e00640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95f9400640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95f40d3d60 0x7f95f40cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 -- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 msgr2=0x7f95f4159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f4159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 -- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95f40d3d60 msgr2=0x7f95f40cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95f40d3d60 0x7f95f40cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.040+0000 7f95fa800640 1 -- 172.21.15.46:0/974503026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41f8ae0 con 0x7f95f4162aa0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95fa800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009eb0 tx=0x7f95f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0040070 con 0x7f95f4162aa0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f0030ae0 con 0x7f95f4162aa0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0045460 con 0x7f95f4162aa0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f40cd430 con 0x7f95f4162aa0 2024-04-26T23:56:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.041+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f40ce800 con 0x7f95f4162aa0 2024-04-26T23:56:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.042+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f4069530 con 0x7f95f4162aa0 2024-04-26T23:56:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.043+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95f0039040 con 0x7f95f4162aa0 2024-04-26T23:56:44.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.045+0000 7f95ee000640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 0x7f95d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.045+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f95f00ad3e0 con 0x7f95f4162aa0 2024-04-26T23:56:44.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.045+0000 7f95f9e00640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 0x7f95d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.046+0000 7f95f9e00640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 0x7f95d806edb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000f60 tx=0x7f95e8007ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.050+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95f007b790 con 0x7f95f4162aa0 2024-04-26T23:56:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.252+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95f40cbf00 con 0x7f95f4162aa0 2024-04-26T23:56:44.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.253+0000 7f95ee000640 1 -- 172.21.15.46:0/974503026 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) ==== 93+0+10 (secure 0 0 0) 0x7f95f007b090 con 0x7f95f4162aa0 2024-04-26T23:56:44.255 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 msgr2=0x7f95d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 0x7f95d806edb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000f60 tx=0x7f95e8007ce0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 msgr2=0x7f95f40cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009eb0 tx=0x7f95f0004a40 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 shutdown_connections 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f95d806c8f0 0x7f95d806edb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f4162aa0 0x7f95f40cc680 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95f40d3d60 0x7f95f40cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 --2- 172.21.15.46:0/974503026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95f40b64c0 0x7f95f4159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 >> 172.21.15.46:0/974503026 conn(0x7f95f40d5070 msgr2=0x7f95f406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 shutdown_connections 2024-04-26T23:56:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.256+0000 7f95fc800640 1 -- 172.21.15.46:0/974503026 wait complete. 2024-04-26T23:56:44.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:56:44.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.544+0000 7ff66b200640 1 Processor -- start 2024-04-26T23:56:44.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.544+0000 7ff66b200640 1 -- start start 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff66c073af0 0x7ff66c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c0cb4f0 con 0x7ff66c0cbec0 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c1649f0 con 0x7ff66c073af0 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.545+0000 7ff66b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c071e80 con 0x7ff66c1640d0 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40938/0 (socket says 172.21.15.46:40938) 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 -- 172.21.15.46:0/3512615326 learned_addr learned my addr 172.21.15.46:0/3512615326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff669400640 1 --2- 172.21.15.46:0/3512615326 >> v2:172.21.15.142:3300/0 conn(0x7ff66c073af0 0x7ff66c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff669e00640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 -- 172.21.15.46:0/3512615326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 msgr2=0x7ff66c1644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c1644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 -- 172.21.15.46:0/3512615326 >> v2:172.21.15.142:3300/0 conn(0x7ff66c073af0 msgr2=0x7ff66c0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 --2- 172.21.15.46:0/3512615326 >> v2:172.21.15.142:3300/0 conn(0x7ff66c073af0 0x7ff66c0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff66a800640 1 -- 172.21.15.46:0/3512615326 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c072160 con 0x7ff66c0cbec0 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.546+0000 7ff669e00640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:44.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.547+0000 7ff66a800640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0b3740 tx=0x7ff66402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72131b06c08778e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.548+0000 7ff668a00640 1 -- 172.21.15.46:0/3512615326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664037070 con 0x7ff66c0cbec0 2024-04-26T23:56:44.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.548+0000 7ff668a00640 1 -- 172.21.15.46:0/3512615326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff664032440 con 0x7ff66c0cbec0 2024-04-26T23:56:44.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.548+0000 7ff668a00640 1 -- 172.21.15.46:0/3512615326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664032810 con 0x7ff66c0cbec0 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 -- 172.21.15.46:0/3512615326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c0cbec0 msgr2=0x7ff66c0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0b3740 tx=0x7ff66402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 -- 172.21.15.46:0/3512615326 shutdown_connections 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c0cbec0 0x7ff66c0ce7d0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 --2- 172.21.15.46:0/3512615326 >> v2:172.21.15.142:3300/0 conn(0x7ff66c073af0 0x7ff66c0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 --2- 172.21.15.46:0/3512615326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 -- 172.21.15.46:0/3512615326 >> 172.21.15.46:0/3512615326 conn(0x7ff66c0b27b0 msgr2=0x7ff66c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 -- 172.21.15.46:0/3512615326 shutdown_connections 2024-04-26T23:56:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.549+0000 7ff66b200640 1 -- 172.21.15.46:0/3512615326 wait complete. 2024-04-26T23:56:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.551+0000 7ff66b200640 1 Processor -- start 2024-04-26T23:56:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.551+0000 7ff66b200640 1 -- start start 2024-04-26T23:56:44.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 0x7ff66c1f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c0ced10 con 0x7ff66c073af0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c073f70 con 0x7ff66c0cbec0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c0cee90 con 0x7ff66c1640d0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40954/0 (socket says 172.21.15.46:40954) 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 -- 172.21.15.46:0/1278387403 learned_addr learned my addr 172.21.15.46:0/1278387403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669400640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 0x7ff66c1f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff66a800640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 -- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 msgr2=0x7ff66c0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c0b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 -- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 msgr2=0x7ff66c1f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 0x7ff66c1f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669e00640 1 -- 172.21.15.46:0/1278387403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1534d0 con 0x7ff66c073af0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.552+0000 7ff669400640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 0x7ff66c1f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff669e00640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00cf60 tx=0x7ff65c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c01a070 con 0x7ff66c073af0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c00d840 con 0x7ff66c073af0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00dc10 con 0x7ff66c073af0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c0c8a00 con 0x7ff66c073af0 2024-04-26T23:56:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.553+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c0c8fc0 con 0x7ff66c073af0 2024-04-26T23:56:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.555+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c0693b0 con 0x7ff66c073af0 2024-04-26T23:56:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.556+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff65c01d070 con 0x7ff66c073af0 2024-04-26T23:56:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.556+0000 7ff662000640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 0x7ff64806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.557+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7ff65c08c430 con 0x7ff66c073af0 2024-04-26T23:56:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.557+0000 7ff669400640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 0x7ff64806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.558+0000 7ff669400640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 0x7ff64806ee00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff654006fd0 tx=0x7ff654008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:44.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.561+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff65c05a0d0 con 0x7ff66c073af0 2024-04-26T23:56:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.765+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff66c0ccd90 con 0x7ff66c073af0 2024-04-26T23:56:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.766+0000 7ff662000640 1 -- 172.21.15.46:0/1278387403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) ==== 104+0+11 (secure 0 0 0) 0x7ff65c05e3d0 con 0x7ff66c073af0 2024-04-26T23:56:44.768 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 msgr2=0x7ff64806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 0x7ff64806ee00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff654006fd0 tx=0x7ff654008040 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 msgr2=0x7ff66c152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00cf60 tx=0x7ff65c00cf90 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 shutdown_connections 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff64806c940 0x7ff64806ee00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff66c1640d0 0x7ff66c0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff66c0cbec0 0x7ff66c1f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 --2- 172.21.15.46:0/1278387403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff66c073af0 0x7ff66c152f90 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 >> 172.21.15.46:0/1278387403 conn(0x7ff66c0b27b0 msgr2=0x7ff66c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 shutdown_connections 2024-04-26T23:56:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:44.769+0000 7ff66b200640 1 -- 172.21.15.46:0/1278387403 wait complete. 2024-04-26T23:56:44.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:56:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.009+0000 7fcf5ce00640 1 Processor -- start 2024-04-26T23:56:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.010+0000 7fcf5ce00640 1 -- start start 2024-04-26T23:56:45.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf5ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf580d3860 0x7fcf580d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf5ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf58163e70 con 0x7fcf581630a0 2024-04-26T23:56:45.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf5ce00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf580d0170 con 0x7fcf5815f480 2024-04-26T23:56:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf5ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fcf580d0aa0 con 0x7fcf580d3860 2024-04-26T23:56:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf56800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf581630a0 0x7fcf580b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33044/0 (socket says 172.21.15.46:33044) 2024-04-26T23:56:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf56800640 1 -- 172.21.15.46:0/148881316 learned_addr learned my addr 172.21.15.46:0/148881316 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.011+0000 7fcf55400640 1 --2- 172.21.15.46:0/148881316 >> v2:172.21.15.46:3300/0 conn(0x7fcf580d3860 0x7fcf580d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371417570 0 0) 0x7fcf58163e70 con 0x7fcf581630a0 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf38003520 con 0x7fcf581630a0 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf55400640 1 -- 172.21.15.46:0/148881316 >> v1:172.21.15.142:6789/0 conn(0x7fcf5815f480 legacy=0x7fcf580d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf55400640 1 -- 172.21.15.46:0/148881316 >> v1:172.21.15.46:6789/0 conn(0x7fcf581630a0 legacy=0x7fcf580b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3847881692 0 0) 0x7fcf38003520 con 0x7fcf581630a0 2024-04-26T23:56:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.012+0000 7fcf55400640 1 -- 172.21.15.46:0/148881316 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf580d0450 con 0x7fcf580d3860 2024-04-26T23:56:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.013+0000 7fcf55400640 1 --2- 172.21.15.46:0/148881316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf580d3860 0x7fcf580d3cd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba67e4050bbee92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50001670 con 0x7fcf580d3860 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40007540 con 0x7fcf580d3860 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf54a00640 1 -- 172.21.15.46:0/148881316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf400078f0 con 0x7fcf580d3860 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 -- 172.21.15.46:0/148881316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf580d3860 msgr2=0x7fcf580d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/148881316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf580d3860 0x7fcf580d3cd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4002a710 comp rx=0 tx=0).stop 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 -- 172.21.15.46:0/148881316 shutdown_connections 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/148881316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf580d3860 0x7fcf580d3cd0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 -- 172.21.15.46:0/148881316 >> 172.21.15.46:0/148881316 conn(0x7fcf580d5070 msgr2=0x7fcf580d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 -- 172.21.15.46:0/148881316 shutdown_connections 2024-04-26T23:56:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.014+0000 7fcf5ce00640 1 -- 172.21.15.46:0/148881316 wait complete. 2024-04-26T23:56:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.016+0000 7fcf5ce00640 1 Processor -- start 2024-04-26T23:56:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.016+0000 7fcf5ce00640 1 -- start start 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 0x7fcf580a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf5815f480 0x7fcf5815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580cfe60 con 0x7fcf581630a0 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580d1640 con 0x7fcf5815f480 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.017+0000 7fcf5ce00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5815f860 con 0x7fcf580d3860 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40966/0 (socket says 172.21.15.46:40966) 2024-04-26T23:56:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 -- 172.21.15.46:0/405761315 learned_addr learned my addr 172.21.15.46:0/405761315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf55400640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf5815f480 0x7fcf5815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf55e00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 0x7fcf580a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 -- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 msgr2=0x7fcf580a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 0x7fcf580a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 -- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf5815f480 msgr2=0x7fcf5815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf5815f480 0x7fcf5815dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 -- 172.21.15.46:0/405761315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5809ee70 con 0x7fcf581630a0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf55e00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 0x7fcf580a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.018+0000 7fcf56800640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c560 tx=0x7fcf5000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.019+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50017070 con 0x7fcf581630a0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.019+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50005ce0 con 0x7fcf581630a0 2024-04-26T23:56:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.019+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5809f150 con 0x7fcf581630a0 2024-04-26T23:56:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.019+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500072b0 con 0x7fcf581630a0 2024-04-26T23:56:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.019+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf580a15b0 con 0x7fcf581630a0 2024-04-26T23:56:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.020+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5809f320 con 0x7fcf581630a0 2024-04-26T23:56:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.021+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf5001a070 con 0x7fcf581630a0 2024-04-26T23:56:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.022+0000 7fcf4e000640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 0x7fcf3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.023+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7fcf5008b220 con 0x7fcf581630a0 2024-04-26T23:56:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.023+0000 7fcf55e00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 0x7fcf3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.024+0000 7fcf55e00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 0x7fcf3406edb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcf48001680 tx=0x7fcf48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.027+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf500594b0 con 0x7fcf581630a0 2024-04-26T23:56:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.224+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf5808d8a0 con 0x7fcf581630a0 2024-04-26T23:56:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.225+0000 7fcf4e000640 1 -- 172.21.15.46:0/405761315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) ==== 100+0+11 (secure 0 0 0) 0x7fcf5808d8a0 con 0x7fcf581630a0 2024-04-26T23:56:45.300 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:56:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 msgr2=0x7fcf3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 0x7fcf3406edb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcf48001680 tx=0x7fcf48009040 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 msgr2=0x7fcf580b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c560 tx=0x7fcf5000ca20 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 shutdown_connections 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcf3406c8f0 0x7fcf3406edb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf581630a0 0x7fcf580b26d0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf5815f480 0x7fcf5815dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 --2- 172.21.15.46:0/405761315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf580d3860 0x7fcf580a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 >> 172.21.15.46:0/405761315 conn(0x7fcf580d5070 msgr2=0x7fcf580d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 shutdown_connections 2024-04-26T23:56:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.228+0000 7fcf5ce00640 1 -- 172.21.15.46:0/405761315 wait complete. 2024-04-26T23:56:45.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:56:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.567+0000 7f0d3c200640 1 Processor -- start 2024-04-26T23:56:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.567+0000 7f0d3c200640 1 -- start start 2024-04-26T23:56:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.570+0000 7f0d3c200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 0x7f0d340b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.570+0000 7f0d3c200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d340b3fd0 con 0x7f0d34163530 2024-04-26T23:56:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.570+0000 7f0d3c200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d340b4db0 con 0x7f0d34167ab0 2024-04-26T23:56:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.570+0000 7f0d3c200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d340b5090 con 0x7f0d3406fb00 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.571+0000 7f0d3ac00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0d34167ab0 0x7f0d34165010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:50976/0 (socket says 172.21.15.46:50976) 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.572+0000 7f0d3ac00640 1 -- 172.21.15.46:0/2843990319 learned_addr learned my addr 172.21.15.46:0/2843990319 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.572+0000 7f0d39800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0d34163530 0x7f0d3406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33054/0 (socket says 172.21.15.46:33054) 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.574+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2843990319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 0x7f0d340b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.574+0000 7f0d38e00640 1 -- 172.21.15.46:0/2843990319 >> v1:172.21.15.142:6789/0 conn(0x7f0d34167ab0 legacy=0x7f0d34165010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.574+0000 7f0d38e00640 1 -- 172.21.15.46:0/2843990319 >> v1:172.21.15.46:6789/0 conn(0x7f0d34163530 legacy=0x7f0d3406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.574+0000 7f0d38e00640 1 -- 172.21.15.46:0/2843990319 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d340b5370 con 0x7f0d3406fb00 2024-04-26T23:56:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.578+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2843990319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 0x7f0d340b6b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d200097d0 tx=0x7f0d20029f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1efbf207e4e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.578+0000 7f0d33e00640 1 -- 172.21.15.46:0/2843990319 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20037070 con 0x7f0d3406fb00 2024-04-26T23:56:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.579+0000 7f0d33e00640 1 -- 172.21.15.46:0/2843990319 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2002ac20 con 0x7f0d3406fb00 2024-04-26T23:56:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.579+0000 7f0d33e00640 1 -- 172.21.15.46:0/2843990319 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20032440 con 0x7f0d3406fb00 2024-04-26T23:56:45.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.581+0000 7f0d3c200640 1 -- 172.21.15.46:0/2843990319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 msgr2=0x7f0d340b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.581+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2843990319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 0x7f0d340b6b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d200097d0 tx=0x7f0d20029f90 comp rx=0 tx=0).stop 2024-04-26T23:56:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.582+0000 7f0d3c200640 1 -- 172.21.15.46:0/2843990319 shutdown_connections 2024-04-26T23:56:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.582+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2843990319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d3406fb00 0x7f0d340b6b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.582+0000 7f0d3c200640 1 -- 172.21.15.46:0/2843990319 >> 172.21.15.46:0/2843990319 conn(0x7f0d340b2b10 msgr2=0x7f0d340b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:45.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.582+0000 7f0d3c200640 1 -- 172.21.15.46:0/2843990319 shutdown_connections 2024-04-26T23:56:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.583+0000 7f0d3c200640 1 -- 172.21.15.46:0/2843990319 wait complete. 2024-04-26T23:56:45.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.589+0000 7f0d3c200640 1 Processor -- start 2024-04-26T23:56:45.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.589+0000 7f0d3c200640 1 -- start start 2024-04-26T23:56:45.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.592+0000 7f0d3c200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.594+0000 7f0d39800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.594+0000 7f0d3c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 0x7f0d341525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.594+0000 7f0d39800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:50052/0 (socket says 172.21.15.46:50052) 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.594+0000 7f0d39800640 1 -- 172.21.15.46:0/2615345553 learned_addr learned my addr 172.21.15.46:0/2615345553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.594+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d34167ab0 0x7f0d341626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.595+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34165720 con 0x7f0d34163530 2024-04-26T23:56:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.595+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34164920 con 0x7f0d3406fb00 2024-04-26T23:56:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.595+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0d341658a0 con 0x7f0d34167ab0 2024-04-26T23:56:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.596+0000 7f0d3ac00640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d34167ab0 0x7f0d341626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.596+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 0x7f0d341525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.598+0000 7f0d39800640 1 -- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d34167ab0 msgr2=0x7f0d341626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.598+0000 7f0d39800640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d34167ab0 0x7f0d341626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.598+0000 7f0d39800640 1 -- 172.21.15.46:0/2615345553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 msgr2=0x7f0d341525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.598+0000 7f0d39800640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 0x7f0d341525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.598+0000 7f0d39800640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d34152eb0 con 0x7f0d3406fb00 2024-04-26T23:56:45.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.599+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 0x7f0d341525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:45.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.601+0000 7f0d39800640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c7b0 tx=0x7f0d2800cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.602+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28018070 con 0x7f0d3406fb00 2024-04-26T23:56:45.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.602+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d280072d0 con 0x7f0d3406fb00 2024-04-26T23:56:45.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.602+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d280047e0 con 0x7f0d3406fb00 2024-04-26T23:56:45.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.603+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d340b0b20 con 0x7f0d3406fb00 2024-04-26T23:56:45.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.604+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d340b10e0 con 0x7f0d3406fb00 2024-04-26T23:56:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.607+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d34069530 con 0x7f0d3406fb00 2024-04-26T23:56:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.609+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d2801b070 con 0x7f0d3406fb00 2024-04-26T23:56:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.611+0000 7f0d31600640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 0x7f0d1406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.611+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f0d2808b230 con 0x7f0d3406fb00 2024-04-26T23:56:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.612+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 0x7f0d1406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.614+0000 7f0d38e00640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 0x7f0d1406ee80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0d20034000 tx=0x7f0d2002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.619+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d28058dd0 con 0x7f0d3406fb00 2024-04-26T23:56:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.889+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0d340b3aa0 con 0x7f0d3406fb00 2024-04-26T23:56:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.890+0000 7f0d31600640 1 -- 172.21.15.46:0/2615345553 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v199) ==== 80+0+0 (secure 0 0 0) 0x7f0d2805d0d0 con 0x7f0d3406fb00 2024-04-26T23:56:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:56:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 msgr2=0x7f0d1406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 0x7f0d1406ee80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0d20034000 tx=0x7f0d2002e000 comp rx=0 tx=0).stop 2024-04-26T23:56:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 msgr2=0x7f0d34160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c7b0 tx=0x7f0d2800cc70 comp rx=0 tx=0).stop 2024-04-26T23:56:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 shutdown_connections 2024-04-26T23:56:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0d1406c9c0 0x7f0d1406ee80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d34167ab0 0x7f0d341626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d34163530 0x7f0d341525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 --2- 172.21.15.46:0/2615345553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d3406fb00 0x7f0d34160b20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 >> 172.21.15.46:0/2615345553 conn(0x7f0d340b2b10 msgr2=0x7f0d3406eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 shutdown_connections 2024-04-26T23:56:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:45.893+0000 7f0d3c200640 1 -- 172.21.15.46:0/2615345553 wait complete. 2024-04-26T23:56:45.911 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:56:45.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:45.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:56:46.502 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:56:46.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:56:46.502 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:56:46.541 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:46.541 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:46.541 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:46.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:46.595 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:46.595 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:46.596 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:46.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:46.642 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:56:46.642 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:46.642 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:46.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:46.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:46.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.905+0000 7f09f1000640 1 Processor -- start 2024-04-26T23:56:46.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.905+0000 7f09f1000640 1 -- start start 2024-04-26T23:56:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09f1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09ec0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09f1000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f09ec160910 0x7f09ec160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09f1000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ec073ad0 con 0x7f09ec1466b0 2024-04-26T23:56:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09f1000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec146a90 con 0x7f09ec160910 2024-04-26T23:56:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09f1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec073d70 con 0x7f09ec0720e0 2024-04-26T23:56:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09eb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f09ec1466b0 0x7f09ec06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33060/0 (socket says 172.21.15.46:33060) 2024-04-26T23:56:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.907+0000 7f09eb400640 1 -- 172.21.15.46:0/359799456 learned_addr learned my addr 172.21.15.46:0/359799456 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4115478711 0 0) 0x7f09ec073ad0 con 0x7f09ec1466b0 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09d0003520 con 0x7f09ec1466b0 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ebe00640 1 --2- 172.21.15.46:0/359799456 >> v2:172.21.15.142:3300/0 conn(0x7f09ec160910 0x7f09ec160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09eaa00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09ec0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2871259564 0 0) 0x7f09d0003520 con 0x7f09ec1466b0 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 msgr2=0x7f09ec0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09ec0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 >> v2:172.21.15.142:3300/0 conn(0x7f09ec160910 msgr2=0x7f09ec160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 --2- 172.21.15.46:0/359799456 >> v2:172.21.15.142:3300/0 conn(0x7f09ec160910 0x7f09ec160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec074050 con 0x7f09ec1466b0 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f09dc001670 con 0x7f09ec1466b0 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 >> v1:172.21.15.46:6789/0 conn(0x7f09ec1466b0 legacy=0x7f09ec06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09d00039b0 0x7f09d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09d0004800 0x7f09d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec146a90 con 0x7f09d0004800 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec073d70 con 0x7f09d00039b0 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09d000ba40 con 0x7f09ec0720e0 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.908+0000 7f09eaa00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09d0004800 0x7f09d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09ebe00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eb400640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09d00039b0 0x7f09d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eaa00640 1 -- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 msgr2=0x7f09d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eaa00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09ec0720e0 0x7f09d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eaa00640 1 -- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09d00039b0 msgr2=0x7f09d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eaa00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09d00039b0 0x7f09d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.909+0000 7f09eaa00640 1 -- 172.21.15.46:0/359799456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d0003520 con 0x7f09d0004800 2024-04-26T23:56:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.910+0000 7f09eaa00640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09d0004800 0x7f09d0004c80 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f09d800ba30 tx=0x7f09d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b076dcbf5554900e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.910+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8030070 con 0x7f09d0004800 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.910+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d802ade0 con 0x7f09d0004800 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.910+0000 7f09ea000640 1 -- 172.21.15.46:0/359799456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800b600 con 0x7f09d0004800 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 -- 172.21.15.46:0/359799456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09d0004800 msgr2=0x7f09d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 --2- 172.21.15.46:0/359799456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09d0004800 0x7f09d0004c80 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f09d800ba30 tx=0x7f09d0005830 comp rx=0 tx=0).stop 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09ebe00640 1 -- 172.21.15.46:0/359799456 reap_dead start 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 -- 172.21.15.46:0/359799456 shutdown_connections 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 -- 172.21.15.46:0/359799456 >> 172.21.15.46:0/359799456 conn(0x7f09ec0b26d0 msgr2=0x7f09ec0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 -- 172.21.15.46:0/359799456 shutdown_connections 2024-04-26T23:56:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.911+0000 7f09f1000640 1 -- 172.21.15.46:0/359799456 wait complete. 2024-04-26T23:56:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.913+0000 7f09f1000640 1 Processor -- start 2024-04-26T23:56:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.913+0000 7f09f1000640 1 -- start start 2024-04-26T23:56:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09d0004800 0x7f09ec0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 0x7f09ec0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0cd860 con 0x7f09ec15faa0 2024-04-26T23:56:46.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0cb470 con 0x7f09ec0b5660 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09f1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0cd9e0 con 0x7f09d0004800 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40992/0 (socket says 172.21.15.46:40992) 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.914+0000 7f09eaa00640 1 -- 172.21.15.46:0/2535248416 learned_addr learned my addr 172.21.15.46:0/2535248416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09ebe00640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 0x7f09ec0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eb400640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09d0004800 0x7f09ec0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 -- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09d0004800 msgr2=0x7f09ec0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09d0004800 0x7f09ec0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 -- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 msgr2=0x7f09ec0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 0x7f09ec0b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 -- 172.21.15.46:0/2535248416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec0b3c80 con 0x7f09ec15faa0 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09ebe00640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 0x7f09ec0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:46.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09eaa00640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f09d800bb50 tx=0x7f09d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8043020 con 0x7f09ec15faa0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d803f680 con 0x7f09ec15faa0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800b600 con 0x7f09ec15faa0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.915+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec0b3f60 con 0x7f09ec15faa0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.916+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec0b4520 con 0x7f09ec15faa0 2024-04-26T23:56:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.917+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec0693b0 con 0x7f09ec15faa0 2024-04-26T23:56:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.918+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09d8046020 con 0x7f09ec15faa0 2024-04-26T23:56:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.919+0000 7f09e3400640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 0x7f09cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.919+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f09d80ac7b0 con 0x7f09ec15faa0 2024-04-26T23:56:46.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.920+0000 7f09eb400640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 0x7f09cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.921+0000 7f09eb400640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 0x7f09cc06ede0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000c00 tx=0x7f09dc000fb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:46.923+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09d804d030 con 0x7f09ec15faa0 2024-04-26T23:56:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.164+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09ec0b39c0 con 0x7f09ec15faa0 2024-04-26T23:56:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.165+0000 7f09e3400640 1 -- 172.21.15.46:0/2535248416 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7f09d807aa50 con 0x7f09ec15faa0 2024-04-26T23:56:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:47.167 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 msgr2=0x7f09cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 0x7f09cc06ede0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000c00 tx=0x7f09dc000fb0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 msgr2=0x7f09ec0b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f09d800bb50 tx=0x7f09d80046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 shutdown_connections 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f09cc06c920 0x7f09cc06ede0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09ec0b5660 0x7f09ec0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec15faa0 0x7f09ec0b1b20 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 --2- 172.21.15.46:0/2535248416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09d0004800 0x7f09ec0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.168+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 >> 172.21.15.46:0/2535248416 conn(0x7f09ec0b26d0 msgr2=0x7f09ec0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.169+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 shutdown_connections 2024-04-26T23:56:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.169+0000 7f09f1000640 1 -- 172.21.15.46:0/2535248416 wait complete. 2024-04-26T23:56:47.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.386+0000 7f9e38e00640 1 Processor -- start 2024-04-26T23:56:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.387+0000 7f9e38e00640 1 -- start start 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.388+0000 7f9e38e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9e3415ef20 0x7f9e3406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.388+0000 7f9e38e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e340b4c80 0x7f9e3406d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.388+0000 7f9e38e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e340705b0 con 0x7f9e340b5be0 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.388+0000 7f9e38e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9e340b4980 con 0x7f9e340b4c80 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.388+0000 7f9e38e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9e340702d0 con 0x7f9e3415ef20 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e340b4c80 0x7f9e3406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e340b4c80 0x7f9e3406d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41008/0 (socket says 172.21.15.46:41008) 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 -- 172.21.15.46:0/3200308542 learned_addr learned my addr 172.21.15.46:0/3200308542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 -- 172.21.15.46:0/3200308542 >> v2:172.21.15.142:3300/0 conn(0x7f9e3415ef20 msgr2=0x7f9e3406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 --2- 172.21.15.46:0/3200308542 >> v2:172.21.15.142:3300/0 conn(0x7f9e3415ef20 0x7f9e3406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 -- 172.21.15.46:0/3200308542 >> v1:172.21.15.142:6789/0 conn(0x7f9e340b5be0 legacy=0x7f9e340af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.389+0000 7f9e32a00640 1 -- 172.21.15.46:0/3200308542 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3406e650 con 0x7f9e340b4c80 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.390+0000 7f9e32a00640 1 --2- 172.21.15.46:0/3200308542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e3406d360 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f9e34069120 tx=0x7f9e1c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ab57fd2fabae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.390+0000 7f9e32000640 1 -- 172.21.15.46:0/3200308542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c037070 con 0x7f9e340b4c80 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e32000640 1 -- 172.21.15.46:0/3200308542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e1c032440 con 0x7f9e340b4c80 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e32000640 1 -- 172.21.15.46:0/3200308542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c032810 con 0x7f9e340b4c80 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 -- 172.21.15.46:0/3200308542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 msgr2=0x7f9e3406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3200308542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e3406d360 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f9e34069120 tx=0x7f9e1c02a230 comp rx=0 tx=0).stop 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 -- 172.21.15.46:0/3200308542 shutdown_connections 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3200308542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e3406d360 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3200308542 >> v2:172.21.15.142:3300/0 conn(0x7f9e3415ef20 0x7f9e3406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 -- 172.21.15.46:0/3200308542 >> 172.21.15.46:0/3200308542 conn(0x7f9e34072710 msgr2=0x7f9e34072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 -- 172.21.15.46:0/3200308542 shutdown_connections 2024-04-26T23:56:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.391+0000 7f9e38e00640 1 -- 172.21.15.46:0/3200308542 wait complete. 2024-04-26T23:56:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.393+0000 7f9e38e00640 1 Processor -- start 2024-04-26T23:56:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.393+0000 7f9e38e00640 1 -- start start 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e340b5be0 0x7f9e341e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e3415ef20 0x7f9e3414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3415f300 con 0x7f9e340b4c80 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3406ec10 con 0x7f9e3415ef20 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e38e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3415f480 con 0x7f9e340b5be0 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41022/0 (socket says 172.21.15.46:41022) 2024-04-26T23:56:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.394+0000 7f9e33400640 1 -- 172.21.15.46:0/3725048902 learned_addr learned my addr 172.21.15.46:0/3725048902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 -- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e340b5be0 msgr2=0x7f9e341e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e340b5be0 0x7f9e341e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 -- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e3415ef20 msgr2=0x7f9e3414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e3415ef20 0x7f9e3414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 -- 172.21.15.46:0/3725048902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3414df90 con 0x7f9e340b4c80 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e33400640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b4a0 tx=0x7f9e2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24014070 con 0x7f9e340b4c80 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e24010610 con 0x7f9e340b4c80 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e34156380 con 0x7f9e340b4c80 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.395+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24010a00 con 0x7f9e340b4c80 2024-04-26T23:56:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.396+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e34156940 con 0x7f9e340b4c80 2024-04-26T23:56:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.397+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e34076370 con 0x7f9e340b4c80 2024-04-26T23:56:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.398+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e24017070 con 0x7f9e340b4c80 2024-04-26T23:56:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.399+0000 7f9e2b400640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.400+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f9e2408b800 con 0x7f9e340b4c80 2024-04-26T23:56:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.400+0000 7f9e32a00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.401+0000 7f9e32a00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 0x7f9e1006edb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c034000 tx=0x7f9e1c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.404+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e24059a00 con 0x7f9e340b4c80 2024-04-26T23:56:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.605+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e34154ca0 con 0x7f9e340b4c80 2024-04-26T23:56:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.608+0000 7f9e2b400640 1 -- 172.21.15.46:0/3725048902 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) ==== 74+0+16848 (secure 0 0 0) 0x7f9e24059300 con 0x7f9e340b4c80 2024-04-26T23:56:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:47.611 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":199,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:32.320531+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 msgr2=0x7f9e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 0x7f9e1006edb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c034000 tx=0x7f9e1c02c000 comp rx=0 tx=0).stop 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 msgr2=0x7f9e34155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b4a0 tx=0x7f9e2400b960 comp rx=0 tx=0).stop 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 shutdown_connections 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e3415ef20 0x7f9e3414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e340b5be0 0x7f9e341e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 --2- 172.21.15.46:0/3725048902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e340b4c80 0x7f9e34155a10 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 >> 172.21.15.46:0/3725048902 conn(0x7f9e34072710 msgr2=0x7f9e34158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 shutdown_connections 2024-04-26T23:56:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.611+0000 7f9e38e00640 1 -- 172.21.15.46:0/3725048902 wait complete. 2024-04-26T23:56:47.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:56:28.688282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:56:29.797117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:47.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.842+0000 7f77e0200640 1 Processor -- start 2024-04-26T23:56:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.843+0000 7f77e0200640 1 -- start start 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.844+0000 7f77e0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77d80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.844+0000 7f77e0200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80b46f0 con 0x7f77d80b64c0 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.844+0000 7f77e0200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80b49d0 con 0x7f77d8162aa0 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.844+0000 7f77e0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80b44a0 con 0x7f77d80d3d60 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.845+0000 7f77de200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f77d80b64c0 0x7f77d80d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33066/0 (socket says 172.21.15.46:33066) 2024-04-26T23:56:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.845+0000 7f77de200640 1 -- 172.21.15.46:0/1687391083 learned_addr learned my addr 172.21.15.46:0/1687391083 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.845+0000 7f77dce00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77d80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.845+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197512739 0 0) 0x7f77d80b46f0 con 0x7f77d80b64c0 2024-04-26T23:56:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.845+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77bc003520 con 0x7f77d80b64c0 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1330337864 0 0) 0x7f77bc003520 con 0x7f77d80b64c0 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 msgr2=0x7f77d80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77d80cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 >> v1:172.21.15.142:6789/0 conn(0x7f77d8162aa0 legacy=0x7f77d80d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80b4cb0 con 0x7f77d80b64c0 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f77d00019f0 con 0x7f77d80b64c0 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 >> v1:172.21.15.46:6789/0 conn(0x7f77d80b64c0 legacy=0x7f77d80d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77bc003fb0 0x7f77bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc004d40 0x7f77bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80b44a0 con 0x7f77bc003fb0 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc00bbb0 con 0x7f77bc004d40 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc00bdb0 con 0x7f77d80d3d60 2024-04-26T23:56:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77dce00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77bc003fb0 0x7f77bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.846+0000 7f77dd800640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77dce00640 1 -- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 msgr2=0x7f77bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77dce00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80d3d60 0x7f77bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77de200640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc004d40 0x7f77bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77dce00640 1 -- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc004d40 msgr2=0x7f77bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77dce00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc004d40 0x7f77bc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.847+0000 7f77dce00640 1 -- 172.21.15.46:0/1687391083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc003520 con 0x7f77bc003fb0 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.848+0000 7f77dce00640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77bc003fb0 0x7f77bc004800 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009ac0 tx=0x7f77bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2b7f72dccdbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.848+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4036070 con 0x7f77bc003fb0 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.848+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c4031400 con 0x7f77bc003fb0 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.848+0000 7f77d7e00640 1 -- 172.21.15.46:0/1687391083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4009690 con 0x7f77bc003fb0 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 -- 172.21.15.46:0/1687391083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77bc003fb0 msgr2=0x7f77bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 --2- 172.21.15.46:0/1687391083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77bc003fb0 0x7f77bc004800 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009ac0 tx=0x7f77bc005830 comp rx=0 tx=0).stop 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77de200640 1 -- 172.21.15.46:0/1687391083 reap_dead start 2024-04-26T23:56:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 -- 172.21.15.46:0/1687391083 shutdown_connections 2024-04-26T23:56:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 -- 172.21.15.46:0/1687391083 >> 172.21.15.46:0/1687391083 conn(0x7f77d80d5070 msgr2=0x7f77d80d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 -- 172.21.15.46:0/1687391083 shutdown_connections 2024-04-26T23:56:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.849+0000 7f77e0200640 1 -- 172.21.15.46:0/1687391083 wait complete. 2024-04-26T23:56:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.850+0000 7f77e0200640 1 Processor -- start 2024-04-26T23:56:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.851+0000 7f77e0200640 1 -- start start 2024-04-26T23:56:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.851+0000 7f77e0200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 0x7f77d815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.851+0000 7f77e0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.851+0000 7f77e0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80bacf0 0x7f77d81f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77e0200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d30f0 con 0x7f77d80cde50 2024-04-26T23:56:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77e0200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80b5020 con 0x7f77bc003fb0 2024-04-26T23:56:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77e0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80d5b90 con 0x7f77d80bacf0 2024-04-26T23:56:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41052/0 (socket says 172.21.15.46:41052) 2024-04-26T23:56:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 -- 172.21.15.46:0/3018522605 learned_addr learned my addr 172.21.15.46:0/3018522605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dd800640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 0x7f77d815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77de200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80bacf0 0x7f77d81f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 -- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80bacf0 msgr2=0x7f77d81f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80bacf0 0x7f77d81f83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 -- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 msgr2=0x7f77d815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 0x7f77d815b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dce00640 1 -- 172.21.15.46:0/3018522605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80ce2d0 con 0x7f77d80cde50 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.852+0000 7f77dd800640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 0x7f77d815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77dce00640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034040 tx=0x7f77c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4040020 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c4003c60 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4036040 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d80c8f40 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.853+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d80c9500 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.854+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8069530 con 0x7f77d80cde50 2024-04-26T23:56:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.855+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77c4043020 con 0x7f77d80cde50 2024-04-26T23:56:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.856+0000 7f77d5600640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 0x7f77b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.856+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f77c40abf30 con 0x7f77d80cde50 2024-04-26T23:56:47.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.857+0000 7f77dd800640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 0x7f77b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:47.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.858+0000 7f77dd800640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 0x7f77b806edb0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f77c80016b0 tx=0x7f77c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:47.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:47.860+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77c404a070 con 0x7f77d80cde50 2024-04-26T23:56:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.061+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77d80b2610 con 0x7f77d80cde50 2024-04-26T23:56:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.062+0000 7f77d5600640 1 -- 172.21.15.46:0/3018522605 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) ==== 93+0+10 (secure 0 0 0) 0x7f77c40044e0 con 0x7f77d80cde50 2024-04-26T23:56:48.064 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.064+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 msgr2=0x7f77b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.064+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 0x7f77b806edb0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f77c80016b0 tx=0x7f77c8009040 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.064+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 msgr2=0x7f77d80ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.064+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034040 tx=0x7f77c4004730 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 shutdown_connections 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f77b806c8f0 0x7f77b806edb0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77d80bacf0 0x7f77d81f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d80cde50 0x7f77d80ba7b0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 --2- 172.21.15.46:0/3018522605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77bc003fb0 0x7f77d815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 >> 172.21.15.46:0/3018522605 conn(0x7f77d80d5070 msgr2=0x7f77d80d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 shutdown_connections 2024-04-26T23:56:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.065+0000 7f77e0200640 1 -- 172.21.15.46:0/3018522605 wait complete. 2024-04-26T23:56:48.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:56:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.290+0000 7f41efa00640 1 Processor -- start 2024-04-26T23:56:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.290+0000 7f41efa00640 1 -- start start 2024-04-26T23:56:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41efa00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41e81597e0 0x7f41e80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41efa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41e8072620 0x7f41e80b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41efa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e8073120 con 0x7f41e80b5730 2024-04-26T23:56:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41efa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f41e8159bc0 con 0x7f41e8072620 2024-04-26T23:56:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41efa00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f41e8072b60 con 0x7f41e81597e0 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41e8072620 0x7f41e80b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41e8072620 0x7f41e80b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41058/0 (socket says 172.21.15.46:41058) 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 -- 172.21.15.46:0/1200813859 learned_addr learned my addr 172.21.15.46:0/1200813859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 -- 172.21.15.46:0/1200813859 >> v2:172.21.15.142:3300/0 conn(0x7f41e81597e0 msgr2=0x7f41e80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 --2- 172.21.15.46:0/1200813859 >> v2:172.21.15.142:3300/0 conn(0x7f41e81597e0 0x7f41e80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 -- 172.21.15.46:0/1200813859 >> v1:172.21.15.142:6789/0 conn(0x7f41e80b5730 legacy=0x7f41e8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.292+0000 7f41eda00640 1 -- 172.21.15.46:0/1200813859 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e8072e40 con 0x7f41e8072620 2024-04-26T23:56:48.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.294+0000 7f41eda00640 1 --2- 172.21.15.46:0/1200813859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e8072620 0x7f41e80b5bc0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f41e80b0420 tx=0x7f41e002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96750410deea45ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.294+0000 7f41e7400640 1 -- 172.21.15.46:0/1200813859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0036070 con 0x7f41e8072620 2024-04-26T23:56:48.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.294+0000 7f41e7400640 1 -- 172.21.15.46:0/1200813859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e0031600 con 0x7f41e8072620 2024-04-26T23:56:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.294+0000 7f41e7400640 1 -- 172.21.15.46:0/1200813859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00319d0 con 0x7f41e8072620 2024-04-26T23:56:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 -- 172.21.15.46:0/1200813859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e8072620 msgr2=0x7f41e80b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 --2- 172.21.15.46:0/1200813859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e8072620 0x7f41e80b5bc0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f41e80b0420 tx=0x7f41e002a780 comp rx=0 tx=0).stop 2024-04-26T23:56:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 -- 172.21.15.46:0/1200813859 shutdown_connections 2024-04-26T23:56:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 --2- 172.21.15.46:0/1200813859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e8072620 0x7f41e80b5bc0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 --2- 172.21.15.46:0/1200813859 >> v2:172.21.15.142:3300/0 conn(0x7f41e81597e0 0x7f41e80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 -- 172.21.15.46:0/1200813859 >> 172.21.15.46:0/1200813859 conn(0x7f41e806c980 msgr2=0x7f41e80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 -- 172.21.15.46:0/1200813859 shutdown_connections 2024-04-26T23:56:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.295+0000 7f41efa00640 1 -- 172.21.15.46:0/1200813859 wait complete. 2024-04-26T23:56:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.297+0000 7f41efa00640 1 Processor -- start 2024-04-26T23:56:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.297+0000 7f41efa00640 1 -- start start 2024-04-26T23:56:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e8072620 0x7f41e80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 0x7f41e814a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41e80aee20 con 0x7f41e80b5730 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41e80b43b0 con 0x7f41e81597e0 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41efa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41e80aefa0 con 0x7f41e8072620 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41060/0 (socket says 172.21.15.46:41060) 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41e7e00640 1 -- 172.21.15.46:0/2727095886 learned_addr learned my addr 172.21.15.46:0/2727095886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41ed000640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e8072620 0x7f41e80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41eda00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 0x7f41e814a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.298+0000 7f41e7e00640 1 -- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e8072620 msgr2=0x7f41e80d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e7e00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e8072620 0x7f41e80d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e7e00640 1 -- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 msgr2=0x7f41e814a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e7e00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 0x7f41e814a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e7e00640 1 -- 172.21.15.46:0/2727095886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80ce600 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41eda00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 0x7f41e814a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e7e00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c430 tx=0x7f41d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011790 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d4017070 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e80ce880 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011b70 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.299+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e80cee40 con 0x7f41e80b5730 2024-04-26T23:56:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.300+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e8069530 con 0x7f41e80b5730 2024-04-26T23:56:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.301+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d4011d10 con 0x7f41e80b5730 2024-04-26T23:56:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.302+0000 7f41e4c00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 0x7f41c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.303+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f41d408bc60 con 0x7f41e80b5730 2024-04-26T23:56:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.303+0000 7f41ed000640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 0x7f41c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.304+0000 7f41ed000640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 0x7f41c806edb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f41d800b810 tx=0x7f41d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.307+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41e80ce600 con 0x7f41e80b5730 2024-04-26T23:56:48.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.515+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41e806f670 con 0x7f41e80b5730 2024-04-26T23:56:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.516+0000 7f41e4c00640 1 -- 172.21.15.46:0/2727095886 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) ==== 104+0+11 (secure 0 0 0) 0x7f41d4059dd0 con 0x7f41e80b5730 2024-04-26T23:56:48.518 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 msgr2=0x7f41c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 0x7f41c806edb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f41d800b810 tx=0x7f41d8008040 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 msgr2=0x7f41e80ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c430 tx=0x7f41d400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 shutdown_connections 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41c806c8f0 0x7f41c806edb0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e81597e0 0x7f41e814a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41e80b5730 0x7f41e80ce0c0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 --2- 172.21.15.46:0/2727095886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e8072620 0x7f41e80d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 >> 172.21.15.46:0/2727095886 conn(0x7f41e806c980 msgr2=0x7f41e8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 shutdown_connections 2024-04-26T23:56:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.519+0000 7f41efa00640 1 -- 172.21.15.46:0/2727095886 wait complete. 2024-04-26T23:56:48.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:56:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.750+0000 7f0a28c00640 1 Processor -- start 2024-04-26T23:56:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.751+0000 7f0a28c00640 1 -- start start 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.752+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a2416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.752+0000 7f0a28c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a240d4090 con 0x7f0a240cfb60 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.752+0000 7f0a28c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a240b4020 con 0x7f0a2415f840 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.752+0000 7f0a28c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a240d3880 con 0x7f0a240d06d0 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.753+0000 7f0a1be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0a240cfb60 0x7f0a240cffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33078/0 (socket says 172.21.15.46:33078) 2024-04-26T23:56:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.753+0000 7f0a1be00640 1 -- 172.21.15.46:0/3500060263 learned_addr learned my addr 172.21.15.46:0/3500060263 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.753+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a2416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.753+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189550200 0 0) 0x7f0a0c001110 con 0x7f0a240cfb60 2024-04-26T23:56:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.753+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a08003520 con 0x7f0a240cfb60 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532560946 0 0) 0x7f0a240b4020 con 0x7f0a2415f840 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a0c001110 con 0x7f0a2415f840 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785687609 0 0) 0x7f0a08003520 con 0x7f0a240cfb60 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 msgr2=0x7f0a2416a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a2416a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 >> v1:172.21.15.142:6789/0 conn(0x7f0a2415f840 legacy=0x7f0a240d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a240d3ad0 con 0x7f0a240cfb60 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0a0c0017c0 con 0x7f0a240cfb60 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 >> v1:172.21.15.46:6789/0 conn(0x7f0a240cfb60 legacy=0x7f0a240cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a08003b70 0x7f0a08003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a08004d40 0x7f0a080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240d3880 con 0x7f0a08004d40 2024-04-26T23:56:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0800ba70 con 0x7f0a08003b70 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0800bc70 con 0x7f0a240d06d0 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.754+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a08004d40 0x7f0a080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23400640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a08003b70 0x7f0a08003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23e00640 1 -- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 msgr2=0x7f0a08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a1be00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23e00640 1 -- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a08003b70 msgr2=0x7f0a08003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a08003b70 0x7f0a08003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23e00640 1 -- 172.21.15.46:0/3500060263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a08003520 con 0x7f0a08004d40 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a23400640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a08003b70 0x7f0a08003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.755+0000 7f0a1be00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a240d06d0 0x7f0a08004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.756+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a08004d40 0x7f0a080051c0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009fc0 tx=0x7f0a08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f17874a8716b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.756+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c009c10 con 0x7f0a08004d40 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.756+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c036070 con 0x7f0a08004d40 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.756+0000 7f0a22a00640 1 -- 172.21.15.46:0/3500060263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c031880 con 0x7f0a08004d40 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 -- 172.21.15.46:0/3500060263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a08004d40 msgr2=0x7f0a080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3500060263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a08004d40 0x7f0a080051c0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009fc0 tx=0x7f0a08005830 comp rx=0 tx=0).stop 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a23e00640 1 -- 172.21.15.46:0/3500060263 reap_dead start 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 -- 172.21.15.46:0/3500060263 shutdown_connections 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 -- 172.21.15.46:0/3500060263 >> 172.21.15.46:0/3500060263 conn(0x7f0a240d4f70 msgr2=0x7f0a240d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 -- 172.21.15.46:0/3500060263 shutdown_connections 2024-04-26T23:56:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.757+0000 7f0a28c00640 1 -- 172.21.15.46:0/3500060263 wait complete. 2024-04-26T23:56:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.759+0000 7f0a28c00640 1 Processor -- start 2024-04-26T23:56:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.759+0000 7f0a28c00640 1 -- start start 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241524e0 0x7f0a240ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 0x7f0a2415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240b3d00 con 0x7f0a241601f0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240b27d0 con 0x7f0a241524e0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.760+0000 7f0a28c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a240b3e80 con 0x7f0a241528c0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41070/0 (socket says 172.21.15.46:41070) 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 -- 172.21.15.46:0/3462237381 learned_addr learned my addr 172.21.15.46:0/3462237381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a1be00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 0x7f0a2415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23400640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241524e0 0x7f0a240ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 -- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 msgr2=0x7f0a2415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 0x7f0a2415fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 -- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241524e0 msgr2=0x7f0a240ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241524e0 0x7f0a240ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a23e00640 1 -- 172.21.15.46:0/3462237381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a240cd1d0 con 0x7f0a241601f0 2024-04-26T23:56:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.761+0000 7f0a1be00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 0x7f0a2415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a23e00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02aee0 tx=0x7f0a1c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c009c10 con 0x7f0a241601f0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c031cc0 con 0x7f0a241601f0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c003bd0 con 0x7f0a241601f0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241f5d10 con 0x7f0a241601f0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.762+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a240b90a0 con 0x7f0a241601f0 2024-04-26T23:56:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.764+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a240693b0 con 0x7f0a241601f0 2024-04-26T23:56:48.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.765+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a1c005d10 con 0x7f0a241601f0 2024-04-26T23:56:48.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.765+0000 7f0a1b400640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 0x7f0a0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:48.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.766+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f0a1c0acd70 con 0x7f0a241601f0 2024-04-26T23:56:48.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.766+0000 7f0a23400640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 0x7f0a0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.767+0000 7f0a23400640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 0x7f0a0006ede0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a140018d0 tx=0x7f0a14009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.770+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a1c04a060 con 0x7f0a241601f0 2024-04-26T23:56:48.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.974+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a240d5d10 con 0x7f0a241601f0 2024-04-26T23:56:48.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.974+0000 7f0a1b400640 1 -- 172.21.15.46:0/3462237381 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) ==== 100+0+11 (secure 0 0 0) 0x7f0a1c0393b0 con 0x7f0a241601f0 2024-04-26T23:56:48.976 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 msgr2=0x7f0a0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 0x7f0a0006ede0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a140018d0 tx=0x7f0a14009040 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 msgr2=0x7f0a241f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02aee0 tx=0x7f0a1c009750 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 shutdown_connections 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f0a0006c920 0x7f0a0006ede0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a241601f0 0x7f0a241f4fa0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a241528c0 0x7f0a2415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 --2- 172.21.15.46:0/3462237381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241524e0 0x7f0a240ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 >> 172.21.15.46:0/3462237381 conn(0x7f0a240d4f70 msgr2=0x7f0a240d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 shutdown_connections 2024-04-26T23:56:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:48.977+0000 7f0a28c00640 1 -- 172.21.15.46:0/3462237381 wait complete. 2024-04-26T23:56:48.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.209+0000 7f2926200640 1 Processor -- start 2024-04-26T23:56:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.209+0000 7f2926200640 1 -- start start 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2926200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f292006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2926200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2920160910 0x7f2920160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2926200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2920073ad0 con 0x7f29200720e0 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2926200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2920146a90 con 0x7f2920160910 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2926200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2920073d70 con 0x7f29201466b0 2024-04-26T23:56:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2925800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2920160910 0x7f2920160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f291fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29200720e0 0x7f29200cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33094/0 (socket says 172.21.15.46:33094) 2024-04-26T23:56:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f291fe00640 1 -- 172.21.15.46:0/920461569 learned_addr learned my addr 172.21.15.46:0/920461569 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.211+0000 7f2925800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2920160910 0x7f2920160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41078/0 (socket says 172.21.15.46:41078) 2024-04-26T23:56:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.212+0000 7f2924e00640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f292006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.212+0000 7f2925800640 1 -- 172.21.15.46:0/920461569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 msgr2=0x7f292006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.212+0000 7f2925800640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f292006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.212+0000 7f2925800640 1 -- 172.21.15.46:0/920461569 >> v1:172.21.15.46:6789/0 conn(0x7f29200720e0 legacy=0x7f29200cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.212+0000 7f2925800640 1 -- 172.21.15.46:0/920461569 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2920074050 con 0x7f2920160910 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.213+0000 7f2925800640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2920160910 0x7f2920160da0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d90 tx=0x7f291802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251efc88962f8255 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f291f400640 1 -- 172.21.15.46:0/920461569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918007740 con 0x7f2920160910 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f291f400640 1 -- 172.21.15.46:0/920461569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29180078e0 con 0x7f2920160910 2024-04-26T23:56:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f291f400640 1 -- 172.21.15.46:0/920461569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918007cb0 con 0x7f2920160910 2024-04-26T23:56:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 -- 172.21.15.46:0/920461569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2920160910 msgr2=0x7f2920160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2920160910 0x7f2920160da0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d90 tx=0x7f291802a660 comp rx=0 tx=0).stop 2024-04-26T23:56:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 -- 172.21.15.46:0/920461569 shutdown_connections 2024-04-26T23:56:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2920160910 0x7f2920160da0 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 --2- 172.21.15.46:0/920461569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f292006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 -- 172.21.15.46:0/920461569 >> 172.21.15.46:0/920461569 conn(0x7f29200b26d0 msgr2=0x7f29200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.214+0000 7f2926200640 1 -- 172.21.15.46:0/920461569 shutdown_connections 2024-04-26T23:56:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.215+0000 7f2926200640 1 -- 172.21.15.46:0/920461569 wait complete. 2024-04-26T23:56:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.216+0000 7f2926200640 1 Processor -- start 2024-04-26T23:56:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.216+0000 7f2926200640 1 -- start start 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f29200b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2920160910 0x7f2920143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f292006e3c0 con 0x7f29200720e0 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29200cb470 con 0x7f2920160910 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.217+0000 7f2926200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f29200cb990 con 0x7f29201466b0 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41080/0 (socket says 172.21.15.46:41080) 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 -- 172.21.15.46:0/3938305267 learned_addr learned my addr 172.21.15.46:0/3938305267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f291fe00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f29200b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 -- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 msgr2=0x7f29200b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f29200b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 -- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2920160910 msgr2=0x7f2920143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2925800640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2920160910 0x7f2920143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2920160910 0x7f2920143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f2924e00640 1 -- 172.21.15.46:0/3938305267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29200b1320 con 0x7f29200720e0 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.218+0000 7f291fe00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f29200b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f2924e00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f291400d830 tx=0x7f291400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400b6c0 con 0x7f29200720e0 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914019070 con 0x7f29200720e0 2024-04-26T23:56:49.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400ba80 con 0x7f29200720e0 2024-04-26T23:56:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29200b1600 con 0x7f29200720e0 2024-04-26T23:56:49.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.219+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29200ab9e0 con 0x7f29200720e0 2024-04-26T23:56:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.220+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29200693b0 con 0x7f29200720e0 2024-04-26T23:56:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.221+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f291401e460 con 0x7f29200720e0 2024-04-26T23:56:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.222+0000 7f291cc00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.222+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f291408c330 con 0x7f29200720e0 2024-04-26T23:56:49.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.223+0000 7f291fe00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.224+0000 7f291fe00640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f29200afe30 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.227+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f291405a660 con 0x7f29200720e0 2024-04-26T23:56:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.472+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29200c7180 con 0x7f29200720e0 2024-04-26T23:56:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.473+0000 7f291cc00640 1 -- 172.21.15.46:0/3938305267 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7f2914059f60 con 0x7f29200720e0 2024-04-26T23:56:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:49.475 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 msgr2=0x7f290006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f29200afe30 tx=0x7f290c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 msgr2=0x7f29200aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f291400d830 tx=0x7f291400dcf0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 shutdown_connections 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2920160910 0x7f2920143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29201466b0 0x7f29200b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 --2- 172.21.15.46:0/3938305267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29200720e0 0x7f29200aaf60 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 >> 172.21.15.46:0/3938305267 conn(0x7f29200b26d0 msgr2=0x7f29200cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.476+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 shutdown_connections 2024-04-26T23:56:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.477+0000 7f2926200640 1 -- 172.21.15.46:0/3938305267 wait complete. 2024-04-26T23:56:49.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:49.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.703+0000 7fecb1c00640 1 Processor -- start 2024-04-26T23:56:49.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.704+0000 7fecb1c00640 1 -- start start 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fecac0d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecb1c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecac0d0990 con 0x7fecac15f200 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecb1c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecac0d0c70 con 0x7fecac15f5e0 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecb1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecac0d0f50 con 0x7fecac0d2010 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecabe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecac15f200 0x7fecac0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33100/0 (socket says 172.21.15.46:33100) 2024-04-26T23:56:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.705+0000 7fecabe00640 1 -- 172.21.15.46:0/3223767607 learned_addr learned my addr 172.21.15.46:0/3223767607 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7feca3400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fecac0d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979168094 0 0) 0x7fecac0d0990 con 0x7fecac15f200 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec90003520 con 0x7fecac15f200 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1561673515 0 0) 0x7fec90003520 con 0x7fecac15f200 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 msgr2=0x7fecac0d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fecac0d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 >> v1:172.21.15.142:6789/0 conn(0x7fecac15f5e0 legacy=0x7fecac0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac0d1230 con 0x7fecac15f200 2024-04-26T23:56:49.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fec9c001670 con 0x7fecac15f200 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 >> v1:172.21.15.46:6789/0 conn(0x7fecac15f200 legacy=0x7fecac0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.706+0000 7fecab400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec900039e0 0x7fec90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecab400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fec90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecab400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fec900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0d0f50 con 0x7fec90004d40 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec9000ba70 con 0x7fec900039e0 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec9000bc70 con 0x7fecac0d2010 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fec900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7feca3400640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fec90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecabe00640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec900039e0 0x7fec90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 -- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 msgr2=0x7fec90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:49.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0d2010 0x7fec90004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 -- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec900039e0 msgr2=0x7fec90003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec900039e0 0x7fec90003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.707+0000 7fecb1200640 1 -- 172.21.15.46:0/3223767607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec90003520 con 0x7fec90004d40 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.708+0000 7fecb1200640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fec900051c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7feca4001e20 tx=0x7fec90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c9473021c178b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4001a70 con 0x7fec90004d40 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca402ae40 con 0x7fec90004d40 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecab400640 1 -- 172.21.15.46:0/3223767607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4003bb0 con 0x7fec90004d40 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 -- 172.21.15.46:0/3223767607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 msgr2=0x7fec900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 --2- 172.21.15.46:0/3223767607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fec900051c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7feca4001e20 tx=0x7fec90005830 comp rx=0 tx=0).stop 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1200640 1 -- 172.21.15.46:0/3223767607 reap_dead start 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 -- 172.21.15.46:0/3223767607 shutdown_connections 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 -- 172.21.15.46:0/3223767607 >> 172.21.15.46:0/3223767607 conn(0x7fecac0d4ef0 msgr2=0x7fecac0d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 -- 172.21.15.46:0/3223767607 shutdown_connections 2024-04-26T23:56:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.709+0000 7fecb1c00640 1 -- 172.21.15.46:0/3223767607 wait complete. 2024-04-26T23:56:49.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.711+0000 7fecb1c00640 1 Processor -- start 2024-04-26T23:56:49.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.711+0000 7fecb1c00640 1 -- start start 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac158430 0x7fecac0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0b5a30 0x7fecac0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0b2f50 con 0x7fec90004d40 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0b2dd0 con 0x7fecac158430 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecb1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fecac0b30d0 con 0x7fecac0b5a30 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41090/0 (socket says 172.21.15.46:41090) 2024-04-26T23:56:49.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.712+0000 7fecabe00640 1 -- 172.21.15.46:0/2427382586 learned_addr learned my addr 172.21.15.46:0/2427382586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecb1200640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0b5a30 0x7fecac0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 -- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0b5a30 msgr2=0x7fecac0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0b5a30 0x7fecac0c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 -- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac158430 msgr2=0x7fecac0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac158430 0x7fecac0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 -- 172.21.15.46:0/2427382586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac0c9e10 con 0x7fec90004d40 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.713+0000 7fecabe00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7fec9c000ff0 tx=0x7fec9c00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.714+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c019070 con 0x7fec90004d40 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.714+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9c0074e0 con 0x7fec90004d40 2024-04-26T23:56:49.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.714+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c004a00 con 0x7fec90004d40 2024-04-26T23:56:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.714+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac15fc30 con 0x7fec90004d40 2024-04-26T23:56:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.714+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1601f0 con 0x7fec90004d40 2024-04-26T23:56:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.715+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecac0693b0 con 0x7fec90004d40 2024-04-26T23:56:49.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.716+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec9c004ba0 con 0x7fec90004d40 2024-04-26T23:56:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.717+0000 7feca8c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 0x7fec8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.717+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7fec9c08b890 con 0x7fec90004d40 2024-04-26T23:56:49.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.718+0000 7feca3400640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 0x7fec8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.719+0000 7feca3400640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 0x7fec8806edb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fecac15c130 tx=0x7fec94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:49.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.722+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec9c059a00 con 0x7fec90004d40 2024-04-26T23:56:49.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.926+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecac0b5e10 con 0x7fec90004d40 2024-04-26T23:56:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.930+0000 7feca8c00640 1 -- 172.21.15.46:0/2427382586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) ==== 74+0+16848 (secure 0 0 0) 0x7fec9c059300 con 0x7fec90004d40 2024-04-26T23:56:49.932 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:49.932 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":199,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:32.320531+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.932+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 msgr2=0x7fec8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.932+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 0x7fec8806edb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fecac15c130 tx=0x7fec94008040 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 msgr2=0x7fecac160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7fec9c000ff0 tx=0x7fec9c00dc70 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 shutdown_connections 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fec8806c8f0 0x7fec8806edb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecac0b5a30 0x7fecac0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac158430 0x7fecac0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 --2- 172.21.15.46:0/2427382586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec90004d40 0x7fecac160710 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 >> 172.21.15.46:0/2427382586 conn(0x7fecac0d4ef0 msgr2=0x7fecac06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 shutdown_connections 2024-04-26T23:56:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:49.933+0000 7fecb1c00640 1 -- 172.21.15.46:0/2427382586 wait complete. 2024-04-26T23:56:49.954 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:56:49.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.167+0000 7f2ca5a00640 1 Processor -- start 2024-04-26T23:56:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.167+0000 7f2ca5a00640 1 -- start start 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.168+0000 7f2ca5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ca006e510 0x7f2ca006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.168+0000 7f2ca5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ca00732c0 0x7f2ca0073710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.168+0000 7f2ca5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca006ee30 con 0x7f2ca0160bd0 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.168+0000 7f2ca5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca0161940 con 0x7f2ca00732c0 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2ca5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca0071f80 con 0x7f2ca006e510 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2ca0160bd0 0x7f2ca0161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33106/0 (socket says 172.21.15.46:33106) 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9e000640 1 -- 172.21.15.46:0/945038696 learned_addr learned my addr 172.21.15.46:0/945038696 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9f400640 1 --2- 172.21.15.46:0/945038696 >> v2:172.21.15.46:3300/0 conn(0x7f2ca00732c0 0x7f2ca0073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9ea00640 1 --2- 172.21.15.46:0/945038696 >> v2:172.21.15.142:3300/0 conn(0x7f2ca006e510 0x7f2ca006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9f400640 1 -- 172.21.15.46:0/945038696 >> v2:172.21.15.142:3300/0 conn(0x7f2ca006e510 msgr2=0x7f2ca006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9f400640 1 --2- 172.21.15.46:0/945038696 >> v2:172.21.15.142:3300/0 conn(0x7f2ca006e510 0x7f2ca006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9d600640 1 -- 172.21.15.46:0/945038696 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2570628209 0 0) 0x7f2ca006ee30 con 0x7f2ca0160bd0 2024-04-26T23:56:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9f400640 1 -- 172.21.15.46:0/945038696 >> v1:172.21.15.46:6789/0 conn(0x7f2ca0160bd0 legacy=0x7f2ca0161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.169+0000 7f2c9f400640 1 -- 172.21.15.46:0/945038696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca0072260 con 0x7f2ca00732c0 2024-04-26T23:56:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2c9f400640 1 --2- 172.21.15.46:0/945038696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca0073710 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f2c980097a0 tx=0x7f2c9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d271487a4cd90b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2c9d600640 1 -- 172.21.15.46:0/945038696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98037070 con 0x7f2ca00732c0 2024-04-26T23:56:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2c9d600640 1 -- 172.21.15.46:0/945038696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98032440 con 0x7f2ca00732c0 2024-04-26T23:56:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2c9d600640 1 -- 172.21.15.46:0/945038696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98032810 con 0x7f2ca00732c0 2024-04-26T23:56:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 -- 172.21.15.46:0/945038696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 msgr2=0x7f2ca0073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/945038696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca0073710 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f2c980097a0 tx=0x7f2c9802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 -- 172.21.15.46:0/945038696 shutdown_connections 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/945038696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca0073710 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/945038696 >> v2:172.21.15.142:3300/0 conn(0x7f2ca006e510 0x7f2ca006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.171+0000 7f2ca5a00640 1 -- 172.21.15.46:0/945038696 >> 172.21.15.46:0/945038696 conn(0x7f2ca006c530 msgr2=0x7f2ca006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.172+0000 7f2ca5a00640 1 -- 172.21.15.46:0/945038696 shutdown_connections 2024-04-26T23:56:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.172+0000 7f2ca5a00640 1 -- 172.21.15.46:0/945038696 wait complete. 2024-04-26T23:56:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.173+0000 7f2ca5a00640 1 Processor -- start 2024-04-26T23:56:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.173+0000 7f2ca5a00640 1 -- start start 2024-04-26T23:56:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ca006e510 0x7f2ca013e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ca0160bd0 0x7f2ca013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0072d40 con 0x7f2ca00732c0 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0072820 con 0x7f2ca0160bd0 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.174+0000 7f2ca5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca0072ec0 con 0x7f2ca006e510 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41122/0 (socket says 172.21.15.46:41122) 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 -- 172.21.15.46:0/3912287516 learned_addr learned my addr 172.21.15.46:0/3912287516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 -- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ca006e510 msgr2=0x7f2ca013e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ca006e510 0x7f2ca013e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 -- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ca0160bd0 msgr2=0x7f2ca013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ca0160bd0 0x7f2ca013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 -- 172.21.15.46:0/3912287516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca0139730 con 0x7f2ca00732c0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.175+0000 7f2c9e000640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c4d0 tx=0x7f2c8800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.176+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88018070 con 0x7f2ca00732c0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.176+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c88007500 con 0x7f2ca00732c0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.176+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88004a20 con 0x7f2ca00732c0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.176+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca013d7a0 con 0x7f2ca00732c0 2024-04-26T23:56:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.176+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca013dd60 con 0x7f2ca00732c0 2024-04-26T23:56:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.177+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca0069530 con 0x7f2ca00732c0 2024-04-26T23:56:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.178+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c88004bc0 con 0x7f2ca00732c0 2024-04-26T23:56:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.179+0000 7f2c96a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 0x7f2c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.180+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f2c8808af80 con 0x7f2ca00732c0 2024-04-26T23:56:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.180+0000 7f2c9ea00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 0x7f2c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.182+0000 7f2c9ea00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 0x7f2c7c06edb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2c900070b0 tx=0x7f2c90007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.184+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c88059220 con 0x7f2ca00732c0 2024-04-26T23:56:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.434+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca006fb00 con 0x7f2ca00732c0 2024-04-26T23:56:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.435+0000 7f2c96a00640 1 -- 172.21.15.46:0/3912287516 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7f2c88058b20 con 0x7f2ca00732c0 2024-04-26T23:56:50.437 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:50.438 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.438+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 msgr2=0x7f2c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.438+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 0x7f2c7c06edb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2c900070b0 tx=0x7f2c90007040 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.438+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 msgr2=0x7f2ca01391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.438+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c4d0 tx=0x7f2c8800c990 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 shutdown_connections 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2c7c06c8f0 0x7f2c7c06edb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ca0160bd0 0x7f2ca013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca00732c0 0x7f2ca01391f0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 --2- 172.21.15.46:0/3912287516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ca006e510 0x7f2ca013e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 >> 172.21.15.46:0/3912287516 conn(0x7f2ca006c530 msgr2=0x7f2ca015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 shutdown_connections 2024-04-26T23:56:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.439+0000 7f2ca5a00640 1 -- 172.21.15.46:0/3912287516 wait complete. 2024-04-26T23:56:50.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:50.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.672+0000 7f4bbae00640 1 Processor -- start 2024-04-26T23:56:50.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.673+0000 7f4bbae00640 1 -- start start 2024-04-26T23:56:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.674+0000 7f4bbae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.674+0000 7f4bbae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 0x7f4bb40b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.674+0000 7f4bbae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb40d11f0 con 0x7f4bb40b4000 2024-04-26T23:56:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.674+0000 7f4bbae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb40d0ce0 con 0x7f4bb40d2130 2024-04-26T23:56:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.674+0000 7f4bbae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb40d2990 con 0x7f4bb415f4e0 2024-04-26T23:56:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.675+0000 7f4bb9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.675+0000 7f4bb9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:58166/0 (socket says 172.21.15.46:58166) 2024-04-26T23:56:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.675+0000 7f4bb9a00640 1 -- 172.21.15.46:0/3496498959 learned_addr learned my addr 172.21.15.46:0/3496498959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.675+0000 7f4bb9000640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f4bb40b4000 0x7f4bb40d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:51028/0 (socket says 172.21.15.46:51028) 2024-04-26T23:56:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.675+0000 7f4bba400640 1 --2- 172.21.15.46:0/3496498959 >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 0x7f4bb40b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.676+0000 7f4bb9a00640 1 -- 172.21.15.46:0/3496498959 >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 msgr2=0x7f4bb40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.676+0000 7f4bb9a00640 1 --2- 172.21.15.46:0/3496498959 >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 0x7f4bb40b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.676+0000 7f4bb9a00640 1 -- 172.21.15.46:0/3496498959 >> v1:172.21.15.142:6789/0 conn(0x7f4bb40b4000 legacy=0x7f4bb40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.676+0000 7f4bb9a00640 1 -- 172.21.15.46:0/3496498959 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb40d5f20 con 0x7f4bb415f4e0 2024-04-26T23:56:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.676+0000 7f4bba400640 1 --2- 172.21.15.46:0/3496498959 >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 0x7f4bb40b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.678+0000 7f4bb9a00640 1 --2- 172.21.15.46:0/3496498959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426a6532a48b62d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bb3e00640 1 -- 172.21.15.46:0/3496498959 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4037070 con 0x7f4bb415f4e0 2024-04-26T23:56:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bb3e00640 1 -- 172.21.15.46:0/3496498959 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba4032440 con 0x7f4bb415f4e0 2024-04-26T23:56:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bb3e00640 1 -- 172.21.15.46:0/3496498959 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032810 con 0x7f4bb415f4e0 2024-04-26T23:56:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 -- 172.21.15.46:0/3496498959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 msgr2=0x7f4bb40d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 --2- 172.21.15.46:0/3496498959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba402a250 comp rx=0 tx=0).stop 2024-04-26T23:56:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 -- 172.21.15.46:0/3496498959 shutdown_connections 2024-04-26T23:56:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 --2- 172.21.15.46:0/3496498959 >> v2:172.21.15.142:3300/0 conn(0x7f4bb40d2130 0x7f4bb40b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 --2- 172.21.15.46:0/3496498959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb40d2df0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 -- 172.21.15.46:0/3496498959 >> 172.21.15.46:0/3496498959 conn(0x7f4bb40d5010 msgr2=0x7f4bb40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:50.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 -- 172.21.15.46:0/3496498959 shutdown_connections 2024-04-26T23:56:50.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.679+0000 7f4bbae00640 1 -- 172.21.15.46:0/3496498959 wait complete. 2024-04-26T23:56:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.681+0000 7f4bbae00640 1 Processor -- start 2024-04-26T23:56:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.681+0000 7f4bbae00640 1 -- start start 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40b4000 0x7f4bb414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb409ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb40b1d60 con 0x7f4bb40d2130 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb40d5c40 con 0x7f4bb40b4000 2024-04-26T23:56:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.682+0000 7f4bbae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb415f8c0 con 0x7f4bb415f4e0 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41128/0 (socket says 172.21.15.46:41128) 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 -- 172.21.15.46:0/20287232 learned_addr learned my addr 172.21.15.46:0/20287232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bba400640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb409ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9a00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40b4000 0x7f4bb414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 -- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 msgr2=0x7f4bb409ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb409ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 -- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40b4000 msgr2=0x7f4bb414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40b4000 0x7f4bb414f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.683+0000 7f4bb9000640 1 -- 172.21.15.46:0/20287232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb40cdc40 con 0x7f4bb40d2130 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bb9000640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c790 tx=0x7f4ba000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0018070 con 0x7f4bb40d2130 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba0004d60 con 0x7f4bb40d2130 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000a840 con 0x7f4bb40d2130 2024-04-26T23:56:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb40cdf20 con 0x7f4bb40d2130 2024-04-26T23:56:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.684+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb40ce4e0 con 0x7f4bb40d2130 2024-04-26T23:56:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.685+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb40693b0 con 0x7f4bb40d2130 2024-04-26T23:56:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.687+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ba000abf0 con 0x7f4bb40d2130 2024-04-26T23:56:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.687+0000 7f4bb1600640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 0x7f4b9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.688+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7f4ba008b540 con 0x7f4bb40d2130 2024-04-26T23:56:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.688+0000 7f4bb9a00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 0x7f4b9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:50.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.689+0000 7f4bb9a00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 0x7f4b9406ede0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034000 tx=0x7f4ba402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:50.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.692+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ba0059750 con 0x7f4bb40d2130 2024-04-26T23:56:50.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.937+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bb40d33d0 con 0x7f4bb40d2130 2024-04-26T23:56:50.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.938+0000 7f4bb1600640 1 -- 172.21.15.46:0/20287232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5196 (secure 0 0 0) 0x7f4ba0059050 con 0x7f4bb40d2130 2024-04-26T23:56:50.940 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:50.940 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":158,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:56:30.811636+0000","modified":"2024-04-26T23:56:33.903516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11688":{"gid":11688,"name":"c","rank":0,"incarnation":154,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/4111191104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4111191104},{"type":"v1","addr":"172.21.15.46:6834","nonce":4111191104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11688,"qdb_cluster":[11688]},"id":14}]} 2024-04-26T23:56:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 158 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 msgr2=0x7f4b9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 0x7f4b9406ede0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034000 tx=0x7f4ba402e000 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 msgr2=0x7f4bb40cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c790 tx=0x7f4ba000cb90 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 shutdown_connections 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4b9406c920 0x7f4b9406ede0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb415f4e0 0x7f4bb409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb40d2130 0x7f4bb40cd1a0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 --2- 172.21.15.46:0/20287232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40b4000 0x7f4bb414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.940+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 >> 172.21.15.46:0/20287232 conn(0x7f4bb40d5010 msgr2=0x7f4bb40b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.941+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 shutdown_connections 2024-04-26T23:56:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:50.941+0000 7f4bbae00640 1 -- 172.21.15.46:0/20287232 wait complete. 2024-04-26T23:56:50.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.175+0000 7fb87d600640 1 Processor -- start 2024-04-26T23:56:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.176+0000 7fb87d600640 1 -- start start 2024-04-26T23:56:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.177+0000 7fb87d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.177+0000 7fb87d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb8780b4e20 0x7fb8780d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.177+0000 7fb87d600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8780b65e0 con 0x7fb8780d5a90 2024-04-26T23:56:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.177+0000 7fb87d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb8780d12e0 con 0x7fb8780b4e20 2024-04-26T23:56:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.177+0000 7fb87d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb8780d05f0 con 0x7fb878162a60 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.178+0000 7fb877e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.178+0000 7fb877e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:58176/0 (socket says 172.21.15.46:58176) 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.178+0000 7fb877e00640 1 -- 172.21.15.46:0/804185184 learned_addr learned my addr 172.21.15.46:0/804185184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.178+0000 7fb87cc00640 1 --2- 172.21.15.46:0/804185184 >> v2:172.21.15.142:3300/0 conn(0x7fb8780b4e20 0x7fb8780d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.179+0000 7fb877e00640 1 -- 172.21.15.46:0/804185184 >> v2:172.21.15.142:3300/0 conn(0x7fb8780b4e20 msgr2=0x7fb8780d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.179+0000 7fb877e00640 1 --2- 172.21.15.46:0/804185184 >> v2:172.21.15.142:3300/0 conn(0x7fb8780b4e20 0x7fb8780d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.179+0000 7fb877e00640 1 -- 172.21.15.46:0/804185184 >> v1:172.21.15.142:6789/0 conn(0x7fb8780d5a90 legacy=0x7fb8780b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.179+0000 7fb876a00640 1 -- 172.21.15.46:0/804185184 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972523473 0 0) 0x7fb8780b65e0 con 0x7fb8780d5a90 2024-04-26T23:56:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.179+0000 7fb877e00640 1 -- 172.21.15.46:0/804185184 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8780d1530 con 0x7fb878162a60 2024-04-26T23:56:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.180+0000 7fb877e00640 1 --2- 172.21.15.46:0/804185184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb868009e50 tx=0x7fb86802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b151c7841abefb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.181+0000 7fb876a00640 1 -- 172.21.15.46:0/804185184 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868036070 con 0x7fb878162a60 2024-04-26T23:56:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.181+0000 7fb876a00640 1 -- 172.21.15.46:0/804185184 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb868031600 con 0x7fb878162a60 2024-04-26T23:56:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.181+0000 7fb876a00640 1 -- 172.21.15.46:0/804185184 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680319d0 con 0x7fb878162a60 2024-04-26T23:56:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 -- 172.21.15.46:0/804185184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 msgr2=0x7fb8780d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 --2- 172.21.15.46:0/804185184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb868009e50 tx=0x7fb86802a5f0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 -- 172.21.15.46:0/804185184 shutdown_connections 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 --2- 172.21.15.46:0/804185184 >> v2:172.21.15.142:3300/0 conn(0x7fb8780b4e20 0x7fb8780d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 --2- 172.21.15.46:0/804185184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb878162a60 0x7fb8780d0a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 -- 172.21.15.46:0/804185184 >> 172.21.15.46:0/804185184 conn(0x7fb8780d4fa0 msgr2=0x7fb8780d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 -- 172.21.15.46:0/804185184 shutdown_connections 2024-04-26T23:56:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.182+0000 7fb87d600640 1 -- 172.21.15.46:0/804185184 wait complete. 2024-04-26T23:56:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.184+0000 7fb87d600640 1 Processor -- start 2024-04-26T23:56:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.184+0000 7fb87d600640 1 -- start start 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8780b4e20 0x7fb8780b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8780d5a90 0x7fb878154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8780b52d0 con 0x7fb878162a60 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8780d1ef0 con 0x7fb8780b4e20 2024-04-26T23:56:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb8780b7bc0 con 0x7fb8780d5a90 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41136/0 (socket says 172.21.15.46:41136) 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb87cc00640 1 -- 172.21.15.46:0/617189736 learned_addr learned my addr 172.21.15.46:0/617189736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb877e00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8780b4e20 0x7fb8780b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.186+0000 7fb877400640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8780d5a90 0x7fb878154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 -- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8780d5a90 msgr2=0x7fb878154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8780d5a90 0x7fb878154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 -- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8780b4e20 msgr2=0x7fb8780b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8780b4e20 0x7fb8780b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 -- 172.21.15.46:0/617189736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8780b7ea0 con 0x7fb878162a60 2024-04-26T23:56:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87cc00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e930 tx=0x7fb87000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870017070 con 0x7fb878162a60 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87000c980 con 0x7fb878162a60 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87000cd70 con 0x7fb878162a60 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.187+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8780b8180 con 0x7fb878162a60 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.188+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb878154840 con 0x7fb878162a60 2024-04-26T23:56:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.189+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb878069ac0 con 0x7fb878162a60 2024-04-26T23:56:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.190+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb87001a070 con 0x7fb878162a60 2024-04-26T23:56:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.191+0000 7fb86fe00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 0x7fb85406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.191+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7fb87008bff0 con 0x7fb878162a60 2024-04-26T23:56:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.192+0000 7fb877e00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 0x7fb85406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.193+0000 7fb877e00640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 0x7fb85406edb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb868033000 tx=0x7fb868009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.196+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb87005a380 con 0x7fb878162a60 2024-04-26T23:56:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.400+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb87806d8d0 con 0x7fb878162a60 2024-04-26T23:56:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.403+0000 7fb86fe00640 1 -- 172.21.15.46:0/617189736 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) ==== 74+0+16848 (secure 0 0 0) 0x7fb870059c80 con 0x7fb878162a60 2024-04-26T23:56:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:51.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":199,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:32.320531+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:56:28.688282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-26T23:56:29.797117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 msgr2=0x7fb85406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 0x7fb85406edb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb868033000 tx=0x7fb868009000 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 msgr2=0x7fb878161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e930 tx=0x7fb87000edf0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 shutdown_connections 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb85406c8f0 0x7fb85406edb0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb878162a60 0x7fb878161920 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8780d5a90 0x7fb878154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 --2- 172.21.15.46:0/617189736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8780b4e20 0x7fb8780b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 >> 172.21.15.46:0/617189736 conn(0x7fb8780d4fa0 msgr2=0x7fb8780b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 shutdown_connections 2024-04-26T23:56:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.407+0000 7fb87d600640 1 -- 172.21.15.46:0/617189736 wait complete. 2024-04-26T23:56:51.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:56:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.642+0000 7ffa13200640 1 Processor -- start 2024-04-26T23:56:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.642+0000 7ffa13200640 1 -- start start 2024-04-26T23:56:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.643+0000 7ffa13200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa141466b0 0x7ffa1406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa13200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa140720e0 0x7ffa140cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa13200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa14073ad0 con 0x7ffa14160910 2024-04-26T23:56:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa13200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffa14146a90 con 0x7ffa141466b0 2024-04-26T23:56:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa13200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa14073d70 con 0x7ffa140720e0 2024-04-26T23:56:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa11e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa141466b0 0x7ffa1406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa12800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa14160910 0x7ffa140cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33116/0 (socket says 172.21.15.46:33116) 2024-04-26T23:56:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa12800640 1 -- 172.21.15.46:0/2547558307 learned_addr learned my addr 172.21.15.46:0/2547558307 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.644+0000 7ffa0be00640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa140720e0 0x7ffa140cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.645+0000 7ffa11e00640 1 -- 172.21.15.46:0/2547558307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa140720e0 msgr2=0x7ffa140cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.645+0000 7ffa11e00640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa140720e0 0x7ffa140cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.645+0000 7ffa11e00640 1 -- 172.21.15.46:0/2547558307 >> v1:172.21.15.46:6789/0 conn(0x7ffa14160910 legacy=0x7ffa140cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.645+0000 7ffa11e00640 1 -- 172.21.15.46:0/2547558307 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14074050 con 0x7ffa141466b0 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.646+0000 7ffa11e00640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa141466b0 0x7ffa1406de80 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212d95d9eadc42b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.646+0000 7ffa11400640 1 -- 172.21.15.46:0/2547558307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ffa141466b0 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.646+0000 7ffa11400640 1 -- 172.21.15.46:0/2547558307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04032440 con 0x7ffa141466b0 2024-04-26T23:56:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.646+0000 7ffa11400640 1 -- 172.21.15.46:0/2547558307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04032810 con 0x7ffa141466b0 2024-04-26T23:56:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 -- 172.21.15.46:0/2547558307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa141466b0 msgr2=0x7ffa1406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa141466b0 0x7ffa1406de80 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402a180 comp rx=0 tx=0).stop 2024-04-26T23:56:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 -- 172.21.15.46:0/2547558307 shutdown_connections 2024-04-26T23:56:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa140720e0 0x7ffa140cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 --2- 172.21.15.46:0/2547558307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa141466b0 0x7ffa1406de80 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 -- 172.21.15.46:0/2547558307 >> 172.21.15.46:0/2547558307 conn(0x7ffa140b26d0 msgr2=0x7ffa140b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 -- 172.21.15.46:0/2547558307 shutdown_connections 2024-04-26T23:56:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.647+0000 7ffa13200640 1 -- 172.21.15.46:0/2547558307 wait complete. 2024-04-26T23:56:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.649+0000 7ffa13200640 1 Processor -- start 2024-04-26T23:56:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.649+0000 7ffa13200640 1 -- start start 2024-04-26T23:56:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa141466b0 0x7ffa141510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14160910 0x7ffa140b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1406e3c0 con 0x7ffa140720e0 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140cb470 con 0x7ffa141466b0 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa13200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa140cdda0 con 0x7ffa14160910 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41164/0 (socket says 172.21.15.46:41164) 2024-04-26T23:56:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.650+0000 7ffa11e00640 1 -- 172.21.15.46:0/224524583 learned_addr learned my addr 172.21.15.46:0/224524583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa0be00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa141466b0 0x7ffa141510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa12800640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14160910 0x7ffa140b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 -- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14160910 msgr2=0x7ffa140b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14160910 0x7ffa140b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 -- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa141466b0 msgr2=0x7ffa141510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa141466b0 0x7ffa141510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 -- 172.21.15.46:0/224524583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14151860 con 0x7ffa140720e0 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa11e00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04041070 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.651+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0403c930 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.652+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037040 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.652+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa14151b40 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.652+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140ce810 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.653+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14152060 con 0x7ffa140720e0 2024-04-26T23:56:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.654+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa04044070 con 0x7ffa140720e0 2024-04-26T23:56:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.655+0000 7ffa0a000640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 0x7ff9f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.655+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 6101+0+0 (secure 0 0 0) 0x7ffa040ac890 con 0x7ffa140720e0 2024-04-26T23:56:51.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.655+0000 7ffa0be00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 0x7ff9f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:51.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.657+0000 7ffa0be00640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 0x7ff9f006ede0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:51.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.659+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa14152060 con 0x7ffa140720e0 2024-04-26T23:56:51.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.903+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffa140693b0 con 0x7ffa140720e0 2024-04-26T23:56:51.905 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:56:51.904+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:56:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.907+0000 7ffa0a000640 1 -- 172.21.15.46:0/224524583 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v159) ==== 161+0+0 (secure 0 0 0) 0x7ffa0407abb0 con 0x7ffa140720e0 2024-04-26T23:56:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:56:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 msgr2=0x7ff9f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 0x7ff9f006ede0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc008040 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 msgr2=0x7ffa141f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa040046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 shutdown_connections 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff9f006c920 0x7ff9f006ede0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa14160910 0x7ffa140b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa141466b0 0x7ffa141510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 --2- 172.21.15.46:0/224524583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa140720e0 0x7ffa141f6150 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 >> 172.21.15.46:0/224524583 conn(0x7ffa140b26d0 msgr2=0x7ffa14145020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 shutdown_connections 2024-04-26T23:56:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:51.910+0000 7ffa13200640 1 -- 172.21.15.46:0/224524583 wait complete. 2024-04-26T23:56:51.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:56:52.018 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:56:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.120+0000 7f7956600640 1 Processor -- start 2024-04-26T23:56:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.120+0000 7f7956600640 1 -- start start 2024-04-26T23:56:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.121+0000 7f7956600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f795015d730 0x7f79500d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.122+0000 7f7956600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.122+0000 7f7956600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795006eb90 con 0x7f79500b53b0 2024-04-26T23:56:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.122+0000 7f7956600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79500b4310 con 0x7f795015d730 2024-04-26T23:56:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.122+0000 7f7956600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79500b45f0 con 0x7f795015e220 2024-04-26T23:56:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.122+0000 7f794fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794f400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f795015d730 0x7f79500d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:37536/0 (socket says 172.21.15.46:37536) 2024-04-26T23:56:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794ea00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f79500b53b0 0x7f795015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:55824/0 (socket says 172.21.15.46:55824) 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794fe00640 1 -- 172.21.15.46:0/1181310974 learned_addr learned my addr 172.21.15.46:0/1181310974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794f400640 1 -- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 msgr2=0x7f795006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794f400640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794f400640 1 -- 172.21.15.46:0/1181310974 >> v1:172.21.15.142:6789/0 conn(0x7f79500b53b0 legacy=0x7f795015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794f400640 1 -- 172.21.15.46:0/1181310974 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500b48d0 con 0x7f795015d730 2024-04-26T23:56:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.123+0000 7f794fe00640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.125+0000 7f794f400640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f79500d00d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f79400097a0 tx=0x7f794002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0dfc62f83563f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.126+0000 7f794e000640 1 -- 172.21.15.46:0/1181310974 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940037070 con 0x7f795015d730 2024-04-26T23:56:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.126+0000 7f794e000640 1 -- 172.21.15.46:0/1181310974 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7940032440 con 0x7f795015d730 2024-04-26T23:56:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.126+0000 7f794e000640 1 -- 172.21.15.46:0/1181310974 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940032810 con 0x7f795015d730 2024-04-26T23:56:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.126+0000 7f7956600640 1 -- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 msgr2=0x7f79500d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.126+0000 7f7956600640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f79500d00d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f79400097a0 tx=0x7f794002a180 comp rx=0 tx=0).stop 2024-04-26T23:56:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 -- 172.21.15.46:0/1181310974 shutdown_connections 2024-04-26T23:56:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f795015e220 0x7f795006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 --2- 172.21.15.46:0/1181310974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f79500d00d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 -- 172.21.15.46:0/1181310974 >> 172.21.15.46:0/1181310974 conn(0x7f795006c980 msgr2=0x7f79500b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 -- 172.21.15.46:0/1181310974 shutdown_connections 2024-04-26T23:56:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.127+0000 7f7956600640 1 -- 172.21.15.46:0/1181310974 wait complete. 2024-04-26T23:56:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.129+0000 7f7956600640 1 Processor -- start 2024-04-26T23:56:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.129+0000 7f7956600640 1 -- start start 2024-04-26T23:56:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79500b53b0 0x7f7950130440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f795015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79500b3ec0 con 0x7f795015e220 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79500b4e90 con 0x7f795015d730 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f7956600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79500b4040 con 0x7f79500b53b0 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f794fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f794fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48000/0 (socket says 172.21.15.46:48000) 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f794fe00640 1 -- 172.21.15.46:0/738244563 learned_addr learned my addr 172.21.15.46:0/738244563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f794f400640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79500b53b0 0x7f7950130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.130+0000 7f794ea00640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f795015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 -- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79500b53b0 msgr2=0x7f7950130440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79500b53b0 0x7f7950130440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 -- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 msgr2=0x7f795015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f795015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 -- 172.21.15.46:0/738244563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500b26c0 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.131+0000 7f794fe00640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f794800efc0 tx=0x7f794800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.132+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800cbe0 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.132+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794800cd80 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.132+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79501379f0 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.132+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948003bb0 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.132+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7950137fb0 con 0x7f795015e220 2024-04-26T23:56:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.133+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7950069530 con 0x7f795015e220 2024-04-26T23:56:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.134+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794801b030 con 0x7f795015e220 2024-04-26T23:56:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.135+0000 7f7947400640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 0x7f793006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.135+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(200..200 src has 1..200) ==== 6187+0+0 (secure 0 0 0) 0x7f794808c060 con 0x7f795015e220 2024-04-26T23:56:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.136+0000 7f794f400640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 0x7f793006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.137+0000 7f794f400640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 0x7f793006ede0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:52.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.139+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7948022020 con 0x7f795015e220 2024-04-26T23:56:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.388+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f795015d230 con 0x7f795015e220 2024-04-26T23:56:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.920+0000 7f7947400640 1 -- 172.21.15.46:0/738244563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v160) ==== 104+0+0 (secure 0 0 0) 0x7f7948010040 con 0x7f795015e220 2024-04-26T23:56:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.924+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 msgr2=0x7f793006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.924+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 0x7f793006ede0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).stop 2024-04-26T23:56:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.924+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 msgr2=0x7f7950131110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.924+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f794800efc0 tx=0x7f794800c490 comp rx=0 tx=0).stop 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 shutdown_connections 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f793006c920 0x7f793006ede0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795015e220 0x7f7950131110 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f795015d730 0x7f795015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 --2- 172.21.15.46:0/738244563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79500b53b0 0x7f7950130440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 >> 172.21.15.46:0/738244563 conn(0x7f795006c980 msgr2=0x7f795015c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.925+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 shutdown_connections 2024-04-26T23:56:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:52.926+0000 7f7956600640 1 -- 172.21.15.46:0/738244563 wait complete. 2024-04-26T23:56:52.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:56:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.171+0000 7faebf200640 1 Processor -- start 2024-04-26T23:56:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.171+0000 7faebf200640 1 -- start start 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.172+0000 7faebf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faec01628e0 0x7faec00b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.172+0000 7faebf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec00d1fa0 0x7faec00d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.172+0000 7faebf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faec00b6140 con 0x7faec00d5660 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.172+0000 7faebf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faec00b82e0 con 0x7faec01628e0 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.172+0000 7faebf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faec00d2f30 con 0x7faec00d1fa0 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebd400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faec00d5660 0x7faec00d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42094/0 (socket says 172.21.15.46:42094) 2024-04-26T23:56:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebd400640 1 -- 172.21.15.46:0/31235216 learned_addr learned my addr 172.21.15.46:0/31235216 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebe800640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec00d1fa0 0x7faec00d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebde00640 1 --2- 172.21.15.46:0/31235216 >> v2:172.21.15.142:3300/0 conn(0x7faec01628e0 0x7faec00b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790526390 0 0) 0x7faeac001110 con 0x7faec00d5660 2024-04-26T23:56:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.173+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faea4003520 con 0x7faec00d5660 2024-04-26T23:56:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4133621605 0 0) 0x7faea4003520 con 0x7faec00d5660 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec00d1fa0 msgr2=0x7faec00d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec00d1fa0 0x7faec00d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 >> v2:172.21.15.142:3300/0 conn(0x7faec01628e0 msgr2=0x7faec00b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 --2- 172.21.15.46:0/31235216 >> v2:172.21.15.142:3300/0 conn(0x7faec01628e0 0x7faec00b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec00d0f10 con 0x7faec00d5660 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7faeac001840 con 0x7faec00d5660 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 >> v1:172.21.15.46:6789/0 conn(0x7faec00d5660 legacy=0x7faec00d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faea40039b0 0x7faea4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faea4004800 0x7faea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea40051c0 0x7faea400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faec00d2f30 con 0x7faea4004800 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faea400bf00 con 0x7faea40039b0 2024-04-26T23:56:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faea400c0b0 con 0x7faea40051c0 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebe800640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec00d1fa0 0x7faec00d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faea4004800 0x7faea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebde00640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faea40039b0 0x7faea4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 -- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea40051c0 msgr2=0x7faea400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea40051c0 0x7faea400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 -- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faea40039b0 msgr2=0x7faea4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faea40039b0 0x7faea4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.174+0000 7faebd400640 1 -- 172.21.15.46:0/31235216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea4003520 con 0x7faea4004800 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebd400640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faea4004800 0x7faea4004c80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c3e0 tx=0x7faea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73298f588e5f7774 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac032020 con 0x7faea4004800 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeac005af0 con 0x7faea4004800 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebca00640 1 -- 172.21.15.46:0/31235216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac00c0e0 con 0x7faea4004800 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebf200640 1 -- 172.21.15.46:0/31235216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faea4004800 msgr2=0x7faea4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebf200640 1 --2- 172.21.15.46:0/31235216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faea4004800 0x7faea4004c80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c3e0 tx=0x7faea4005830 comp rx=0 tx=0).stop 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebe800640 1 -- 172.21.15.46:0/31235216 reap_dead start 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebf200640 1 -- 172.21.15.46:0/31235216 shutdown_connections 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.176+0000 7faebf200640 1 -- 172.21.15.46:0/31235216 >> 172.21.15.46:0/31235216 conn(0x7faec009e2c0 msgr2=0x7faec00d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.177+0000 7faebf200640 1 -- 172.21.15.46:0/31235216 shutdown_connections 2024-04-26T23:56:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.177+0000 7faebf200640 1 -- 172.21.15.46:0/31235216 wait complete. 2024-04-26T23:56:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.178+0000 7faebf200640 1 Processor -- start 2024-04-26T23:56:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.179+0000 7faebf200640 1 -- start start 2024-04-26T23:56:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.179+0000 7faebf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea4004800 0x7faec01f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.179+0000 7faebf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec00cc1e0 0x7faec00cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faec00d0c30 con 0x7faec015ce00 2024-04-26T23:56:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faec00d0950 con 0x7faec00cc1e0 2024-04-26T23:56:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faec00d2bc0 con 0x7faea4004800 2024-04-26T23:56:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-04-26T23:56:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 -- 172.21.15.46:0/2014985665 learned_addr learned my addr 172.21.15.46:0/2014985665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebe800640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec00cc1e0 0x7faec00cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebde00640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea4004800 0x7faec01f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 -- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea4004800 msgr2=0x7faec01f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea4004800 0x7faec01f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 -- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec00cc1e0 msgr2=0x7faec00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.180+0000 7faebd400640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec00cc1e0 0x7faec00cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faebd400640 1 -- 172.21.15.46:0/2014985665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec00ccf30 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faebd400640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7faeac031040 tx=0x7faeac001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac007270 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeac007410 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac007800 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec00cd210 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.181+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec00cd7d0 con 0x7faec015ce00 2024-04-26T23:56:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.183+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec00693b0 con 0x7faec015ce00 2024-04-26T23:56:53.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.184+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeac035020 con 0x7faec015ce00 2024-04-26T23:56:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.185+0000 7faeb6000640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 0x7faea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.185+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(200..200 src has 1..200) ==== 6187+0+0 (secure 0 0 0) 0x7faeac0ac0a0 con 0x7faec015ce00 2024-04-26T23:56:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.185+0000 7faebde00640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 0x7faea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.186+0000 7faebde00640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 0x7faea006edb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009a90 tx=0x7faeb0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.189+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faec00b6140 con 0x7faec015ce00 2024-04-26T23:56:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.391+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faec00d2590 con 0x7faec015ce00 2024-04-26T23:56:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.924+0000 7faeb6000640 1 -- 172.21.15.46:0/2014985665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v201) ==== 181+0+0 (secure 0 0 0) 0x7faeac07a380 con 0x7faec015ce00 2024-04-26T23:56:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 msgr2=0x7faea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 0x7faea006edb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009a90 tx=0x7faeb0008040 comp rx=0 tx=0).stop 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 msgr2=0x7faec015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7faeac031040 tx=0x7faeac001c10 comp rx=0 tx=0).stop 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 shutdown_connections 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea006c8f0 0x7faea006edb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec00cc1e0 0x7faec00cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec015ce00 0x7faec015d280 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 --2- 172.21.15.46:0/2014985665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faea4004800 0x7faec01f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.928+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 >> 172.21.15.46:0/2014985665 conn(0x7faec009e2c0 msgr2=0x7faec0160340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.929+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 shutdown_connections 2024-04-26T23:56:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:53.930+0000 7faebf200640 1 -- 172.21.15.46:0/2014985665 wait complete. 2024-04-26T23:56:53.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:56:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.158+0000 7f8b69200640 1 Processor -- start 2024-04-26T23:56:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.158+0000 7f8b69200640 1 -- start start 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.159+0000 7f8b69200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b641627f0 0x7f8b640cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.159+0000 7f8b69200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b640d2080 0x7f8b640d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.159+0000 7f8b69200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b640d01a0 con 0x7f8b640d3af0 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.159+0000 7f8b69200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8b640d0480 con 0x7f8b640d2080 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.159+0000 7f8b69200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8b640d0760 con 0x7f8b641627f0 2024-04-26T23:56:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b640d2080 0x7f8b640d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b640d2080 0x7f8b640d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48044/0 (socket says 172.21.15.46:48044) 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 -- 172.21.15.46:0/3685353538 learned_addr learned my addr 172.21.15.46:0/3685353538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 -- 172.21.15.46:0/3685353538 >> v2:172.21.15.142:3300/0 conn(0x7f8b641627f0 msgr2=0x7f8b640cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 --2- 172.21.15.46:0/3685353538 >> v2:172.21.15.142:3300/0 conn(0x7f8b641627f0 0x7f8b640cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 -- 172.21.15.46:0/3685353538 >> v1:172.21.15.142:6789/0 conn(0x7f8b640d3af0 legacy=0x7f8b64162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.160+0000 7f8b62a00640 1 -- 172.21.15.46:0/3685353538 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b640d0a40 con 0x7f8b640d2080 2024-04-26T23:56:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.161+0000 7f8b62a00640 1 --2- 172.21.15.46:0/3685353538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d2080 0x7f8b640d35b0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f8b640b7ad0 tx=0x7f8b5002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a238e5b4020556a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.162+0000 7f8b62000640 1 -- 172.21.15.46:0/3685353538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50037070 con 0x7f8b640d2080 2024-04-26T23:56:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.162+0000 7f8b62000640 1 -- 172.21.15.46:0/3685353538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b50032440 con 0x7f8b640d2080 2024-04-26T23:56:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.162+0000 7f8b62000640 1 -- 172.21.15.46:0/3685353538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50032810 con 0x7f8b640d2080 2024-04-26T23:56:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.162+0000 7f8b69200640 1 -- 172.21.15.46:0/3685353538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d2080 msgr2=0x7f8b640d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.162+0000 7f8b69200640 1 --2- 172.21.15.46:0/3685353538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d2080 0x7f8b640d35b0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f8b640b7ad0 tx=0x7f8b5002a230 comp rx=0 tx=0).stop 2024-04-26T23:56:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 -- 172.21.15.46:0/3685353538 shutdown_connections 2024-04-26T23:56:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 --2- 172.21.15.46:0/3685353538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d2080 0x7f8b640d35b0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 --2- 172.21.15.46:0/3685353538 >> v2:172.21.15.142:3300/0 conn(0x7f8b641627f0 0x7f8b640cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 -- 172.21.15.46:0/3685353538 >> 172.21.15.46:0/3685353538 conn(0x7f8b6409e380 msgr2=0x7f8b640d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 -- 172.21.15.46:0/3685353538 shutdown_connections 2024-04-26T23:56:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.163+0000 7f8b69200640 1 -- 172.21.15.46:0/3685353538 wait complete. 2024-04-26T23:56:54.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.164+0000 7f8b69200640 1 Processor -- start 2024-04-26T23:56:54.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 -- start start 2024-04-26T23:56:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b640d2080 0x7f8b640ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b641627f0 0x7f8b6415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64162bd0 con 0x7f8b640d3af0 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b640d24f0 con 0x7f8b641627f0 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.165+0000 7f8b69200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b640b5560 con 0x7f8b640d2080 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48056/0 (socket says 172.21.15.46:48056) 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 -- 172.21.15.46:0/2515641996 learned_addr learned my addr 172.21.15.46:0/2515641996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 -- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b640d2080 msgr2=0x7f8b640ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b640d2080 0x7f8b640ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 -- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b641627f0 msgr2=0x7f8b6415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b641627f0 0x7f8b6415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.166+0000 7f8b62a00640 1 -- 172.21.15.46:0/2515641996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6415da30 con 0x7f8b640d3af0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b62a00640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50041070 con 0x7f8b640d3af0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5003c870 con 0x7f8b640d3af0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b640cdfc0 con 0x7f8b640d3af0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50037040 con 0x7f8b640d3af0 2024-04-26T23:56:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.167+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b640ce580 con 0x7f8b640d3af0 2024-04-26T23:56:54.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.168+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b640693b0 con 0x7f8b640d3af0 2024-04-26T23:56:54.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.169+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b50044070 con 0x7f8b640d3af0 2024-04-26T23:56:54.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.170+0000 7f8b5b400640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 0x7f8b4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:54.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.171+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(201..201 src has 1..201) ==== 5749+0+0 (secure 0 0 0) 0x7f8b500ac7a0 con 0x7f8b640d3af0 2024-04-26T23:56:54.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.171+0000 7f8b63400640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 0x7f8b4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:54.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.172+0000 7f8b63400640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 0x7f8b4406edb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400ab90 tx=0x7f8b54009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:54.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.175+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b5007ab90 con 0x7f8b640d3af0 2024-04-26T23:56:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.391+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8b6415df60 con 0x7f8b640d3af0 2024-04-26T23:56:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.928+0000 7f8b5b400640 1 -- 172.21.15.46:0/2515641996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v202) ==== 169+0+0 (secure 0 0 0) 0x7f8b5007a490 con 0x7f8b640d3af0 2024-04-26T23:56:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:56:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.933+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 msgr2=0x7f8b4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.933+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 0x7f8b4406edb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400ab90 tx=0x7f8b54009250 comp rx=0 tx=0).stop 2024-04-26T23:56:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.933+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 msgr2=0x7f8b640cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.933+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b500046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 shutdown_connections 2024-04-26T23:56:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8b4406c8f0 0x7f8b4406edb0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b641627f0 0x7f8b6415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640d3af0 0x7f8b640cb4b0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 --2- 172.21.15.46:0/2515641996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b640d2080 0x7f8b640ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.935+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 >> 172.21.15.46:0/2515641996 conn(0x7f8b6409e380 msgr2=0x7f8b640b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.936+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 shutdown_connections 2024-04-26T23:56:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:54.936+0000 7f8b69200640 1 -- 172.21.15.46:0/2515641996 wait complete. 2024-04-26T23:56:54.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:56:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.153+0000 7f42c6400640 1 Processor -- start 2024-04-26T23:56:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.153+0000 7f42c6400640 1 -- start start 2024-04-26T23:56:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42c6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42c00ced80 0x7f42c00d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42c6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c015e420 con 0x7f42c015df90 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42c6400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c0146b20 con 0x7f42c0146740 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42c6400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f42c0161a00 con 0x7f42c00ced80 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42c5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42c00ced80 0x7f42c00d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42bfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42c015df90 0x7f42c00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42108/0 (socket says 172.21.15.46:42108) 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.155+0000 7f42bfe00640 1 -- 172.21.15.46:0/1900948899 learned_addr learned my addr 172.21.15.46:0/1900948899 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:55.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.156+0000 7f42c5a00640 1 -- 172.21.15.46:0/1900948899 >> v1:172.21.15.142:6789/0 conn(0x7f42c0146740 legacy=0x7f42c01612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:55.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.156+0000 7f42c5a00640 1 -- 172.21.15.46:0/1900948899 >> v1:172.21.15.46:6789/0 conn(0x7f42c015df90 legacy=0x7f42c00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.156+0000 7f42c5a00640 1 -- 172.21.15.46:0/1900948899 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00ccf30 con 0x7f42c00ced80 2024-04-26T23:56:55.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.156+0000 7f42bf400640 1 -- 172.21.15.46:0/1900948899 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2220265525 0 0) 0x7f42ac001110 con 0x7f42c015df90 2024-04-26T23:56:55.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.157+0000 7f42c5a00640 1 --2- 172.21.15.46:0/1900948899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c00ced80 0x7f42c00d0330 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009d90 tx=0x7f42b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b41abb15bd9d032 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.157+0000 7f42bf400640 1 -- 172.21.15.46:0/1900948899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8007740 con 0x7f42c00ced80 2024-04-26T23:56:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.157+0000 7f42bf400640 1 -- 172.21.15.46:0/1900948899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b80078e0 con 0x7f42c00ced80 2024-04-26T23:56:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.157+0000 7f42bf400640 1 -- 172.21.15.46:0/1900948899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8007cb0 con 0x7f42c00ced80 2024-04-26T23:56:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 -- 172.21.15.46:0/1900948899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c00ced80 msgr2=0x7f42c00d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 --2- 172.21.15.46:0/1900948899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c00ced80 0x7f42c00d0330 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009d90 tx=0x7f42b802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 -- 172.21.15.46:0/1900948899 shutdown_connections 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 --2- 172.21.15.46:0/1900948899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c00ced80 0x7f42c00d0330 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 -- 172.21.15.46:0/1900948899 >> 172.21.15.46:0/1900948899 conn(0x7f42c00b26d0 msgr2=0x7f42c00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 -- 172.21.15.46:0/1900948899 shutdown_connections 2024-04-26T23:56:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.158+0000 7f42c6400640 1 -- 172.21.15.46:0/1900948899 wait complete. 2024-04-26T23:56:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.160+0000 7f42c6400640 1 Processor -- start 2024-04-26T23:56:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.160+0000 7f42c6400640 1 -- start start 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 0x7f42c015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42c015df90 0x7f42c013ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00ccb70 con 0x7f42c0146740 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0160dd0 con 0x7f42c00ced80 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42c6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42c00cccf0 con 0x7f42c015df90 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48084/0 (socket says 172.21.15.46:48084) 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.161+0000 7f42bfe00640 1 -- 172.21.15.46:0/34255421 learned_addr learned my addr 172.21.15.46:0/34255421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42c5000640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 0x7f42c015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42c5a00640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42c015df90 0x7f42c013ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 -- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42c015df90 msgr2=0x7f42c013ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42c015df90 0x7f42c013ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 -- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 msgr2=0x7f42c015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 0x7f42c015ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 -- 172.21.15.46:0/34255421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00c6d60 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42c5000640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 0x7f42c015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bfe00640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00c430 tx=0x7f42ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0074e0 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac0047e0 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac004bd0 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.162+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c00c72f0 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.163+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c015b3d0 con 0x7f42c0146740 2024-04-26T23:56:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.164+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c00693b0 con 0x7f42c0146740 2024-04-26T23:56:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.165+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42ac011070 con 0x7f42c0146740 2024-04-26T23:56:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.166+0000 7f42bcc00640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 0x7f42a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.166+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7f42ac08b6b0 con 0x7f42c0146740 2024-04-26T23:56:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.166+0000 7f42c5000640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 0x7f42a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.167+0000 7f42c5000640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 0x7f42a006edb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f42c00cade0 tx=0x7f42b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.170+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42c015e420 con 0x7f42c0146740 2024-04-26T23:56:55.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.376+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f42c0069cc0 con 0x7f42c0146740 2024-04-26T23:56:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.382+0000 7f42bcc00640 1 -- 172.21.15.46:0/34255421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f42ac00e280 con 0x7f42c0146740 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.385+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 msgr2=0x7f42a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.385+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 0x7f42a006edb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f42c00cade0 tx=0x7f42b4009040 comp rx=0 tx=0).stop 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.385+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 msgr2=0x7f42c00c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.385+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00c430 tx=0x7f42ac00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 shutdown_connections 2024-04-26T23:56:55.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f42a006c8f0 0x7f42a006edb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42c015df90 0x7f42c013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42c0146740 0x7f42c00c6350 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 --2- 172.21.15.46:0/34255421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42c00ced80 0x7f42c015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 >> 172.21.15.46:0/34255421 conn(0x7f42c00b26d0 msgr2=0x7f42c01612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 shutdown_connections 2024-04-26T23:56:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.386+0000 7f42c6400640 1 -- 172.21.15.46:0/34255421 wait complete. 2024-04-26T23:56:55.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-26T23:56:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.612+0000 7fdf22200640 1 Processor -- start 2024-04-26T23:56:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.613+0000 7fdf22200640 1 -- start start 2024-04-26T23:56:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.614+0000 7fdf22200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf1c06e780 0x7fdf1c0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf22200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf22200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0b4940 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf22200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0b4c20 con 0x7fdf1c06e780 2024-04-26T23:56:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf22200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0b4f00 con 0x7fdf1c14d200 2024-04-26T23:56:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf1aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf1c0b43c0 0x7fdf1c14caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42116/0 (socket says 172.21.15.46:42116) 2024-04-26T23:56:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf1aa00640 1 -- 172.21.15.46:0/144787260 learned_addr learned my addr 172.21.15.46:0/144787260 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:56:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf1be00640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.615+0000 7fdf1b400640 1 --2- 172.21.15.46:0/144787260 >> v2:172.21.15.142:3300/0 conn(0x7fdf1c06e780 0x7fdf1c0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924490482 0 0) 0x7fdf1c0b4940 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf00003520 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1b400640 1 -- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 msgr2=0x7fdf1c072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1b400640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 92685414 0 0) 0x7fdf00003520 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1b400640 1 -- 172.21.15.46:0/144787260 >> v1:172.21.15.46:6789/0 conn(0x7fdf1c0b43c0 legacy=0x7fdf1c14caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.616+0000 7fdf1b400640 1 -- 172.21.15.46:0/144787260 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c0b51e0 con 0x7fdf1c06e780 2024-04-26T23:56:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.618+0000 7fdf1b400640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c0b3e80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0097a0 tx=0x7fdf0c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07ffce8636d8b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.618+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c037070 con 0x7fdf1c06e780 2024-04-26T23:56:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.618+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c02aba0 con 0x7fdf1c06e780 2024-04-26T23:56:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.618+0000 7fdf1a000640 1 -- 172.21.15.46:0/144787260 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08001670 con 0x7fdf1c06e780 2024-04-26T23:56:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 -- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 msgr2=0x7fdf1c0b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c0b3e80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0097a0 tx=0x7fdf0c02a110 comp rx=0 tx=0).stop 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 -- 172.21.15.46:0/144787260 shutdown_connections 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 --2- 172.21.15.46:0/144787260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c0b3e80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 -- 172.21.15.46:0/144787260 >> 172.21.15.46:0/144787260 conn(0x7fdf1c06c530 msgr2=0x7fdf1c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 -- 172.21.15.46:0/144787260 shutdown_connections 2024-04-26T23:56:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.619+0000 7fdf22200640 1 -- 172.21.15.46:0/144787260 wait complete. 2024-04-26T23:56:55.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.621+0000 7fdf22200640 1 Processor -- start 2024-04-26T23:56:55.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.621+0000 7fdf22200640 1 -- start start 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c06ece0 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c06eb60 con 0x7fdf1c06e780 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf22200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c06ee60 con 0x7fdf1c14d200 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.622+0000 7fdf1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48090/0 (socket says 172.21.15.46:48090) 2024-04-26T23:56:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 -- 172.21.15.46:0/3047637630 learned_addr learned my addr 172.21.15.46:0/3047637630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1be00640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1b400640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 -- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 msgr2=0x7fdf1c1644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c1644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 -- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 msgr2=0x7fdf1c1675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c1675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.623+0000 7fdf1aa00640 1 -- 172.21.15.46:0/3047637630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c166550 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf1aa00640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fdf080011e0 tx=0x7fdf0800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08017070 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08005ad0 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08007070 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c166830 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.624+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c166df0 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.626+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c069530 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.626+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf0801a070 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.628+0000 7fdf13400640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 0x7fdefc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:55.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.628+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7fdf0808adc0 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.628+0000 7fdf1b400640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 0x7fdefc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.630+0000 7fdf1b400640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 0x7fdefc06edb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c034000 tx=0x7fdf0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.632+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf08058d00 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.914+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) -- 0x7fdf1c06d360 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.944+0000 7fdf13400640 1 -- 172.21.15.46:0/3047637630 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v908) ==== 147+0+0 (secure 0 0 0) 0x7fdf0805d000 con 0x7fdf1c0b43c0 2024-04-26T23:56:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.946+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 msgr2=0x7fdefc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.946+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 0x7fdefc06edb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c034000 tx=0x7fdf0c02e000 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.946+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 msgr2=0x7fdf1c148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.946+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fdf080011e0 tx=0x7fdf0800cbe0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 shutdown_connections 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdefc06c8f0 0x7fdefc06edb0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf1c14d200 0x7fdf1c1644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1c0b43c0 0x7fdf1c148040 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 --2- 172.21.15.46:0/3047637630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf1c06e780 0x7fdf1c1675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 >> 172.21.15.46:0/3047637630 conn(0x7fdf1c06c530 msgr2=0x7fdf1c06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 shutdown_connections 2024-04-26T23:56:55.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:55.947+0000 7fdf22200640 1 -- 172.21.15.46:0/3047637630 wait complete. 2024-04-26T23:56:55.969 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-26T23:56:55.969 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-26T23:56:55.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:56:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.172+0000 7f80de600640 1 Processor -- start 2024-04-26T23:56:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.172+0000 7f80de600640 1 -- start start 2024-04-26T23:56:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.174+0000 7f80de600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.174+0000 7f80de600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 0x7f80d80c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.174+0000 7f80de600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d80b03c0 con 0x7f80d815edf0 2024-04-26T23:56:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.174+0000 7f80de600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f80d80cf5c0 con 0x7f80d80cd770 2024-04-26T23:56:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.174+0000 7f80de600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80d80cb680 con 0x7f80d815f1d0 2024-04-26T23:56:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.175+0000 7f80d7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.175+0000 7f80ddc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 0x7f80d80c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.175+0000 7f80d7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:37640/0 (socket says 172.21.15.46:37640) 2024-04-26T23:56:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.175+0000 7f80d7e00640 1 -- 172.21.15.46:0/1925114375 learned_addr learned my addr 172.21.15.46:0/1925114375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.176+0000 7f80d7e00640 1 -- 172.21.15.46:0/1925114375 >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 msgr2=0x7f80d80c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.176+0000 7f80d7e00640 1 --2- 172.21.15.46:0/1925114375 >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 0x7f80d80c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.176+0000 7f80d7e00640 1 -- 172.21.15.46:0/1925114375 >> v1:172.21.15.142:6789/0 conn(0x7f80d815edf0 legacy=0x7f80d80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.176+0000 7f80d7e00640 1 -- 172.21.15.46:0/1925114375 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d80cb960 con 0x7f80d815f1d0 2024-04-26T23:56:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.176+0000 7f80ddc00640 1 --2- 172.21.15.46:0/1925114375 >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 0x7f80d80c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:56.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.178+0000 7f80d7e00640 1 --2- 172.21.15.46:0/1925114375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009870 tx=0x7f80c402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3f0e26ee79717e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.178+0000 7f80d7400640 1 -- 172.21.15.46:0/1925114375 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4037070 con 0x7f80d815f1d0 2024-04-26T23:56:56.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.178+0000 7f80d7400640 1 -- 172.21.15.46:0/1925114375 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c402ae50 con 0x7f80d815f1d0 2024-04-26T23:56:56.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.178+0000 7f80d7400640 1 -- 172.21.15.46:0/1925114375 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4032860 con 0x7f80d815f1d0 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 -- 172.21.15.46:0/1925114375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 msgr2=0x7f80d80cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 --2- 172.21.15.46:0/1925114375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009870 tx=0x7f80c402a300 comp rx=0 tx=0).stop 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 -- 172.21.15.46:0/1925114375 shutdown_connections 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 --2- 172.21.15.46:0/1925114375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815f1d0 0x7f80d80cf080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 --2- 172.21.15.46:0/1925114375 >> v2:172.21.15.142:3300/0 conn(0x7f80d80cd770 0x7f80d80c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 -- 172.21.15.46:0/1925114375 >> 172.21.15.46:0/1925114375 conn(0x7f80d80b26d0 msgr2=0x7f80d80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 -- 172.21.15.46:0/1925114375 shutdown_connections 2024-04-26T23:56:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.179+0000 7f80de600640 1 -- 172.21.15.46:0/1925114375 wait complete. 2024-04-26T23:56:56.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.181+0000 7f80de600640 1 Processor -- start 2024-04-26T23:56:56.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.182+0000 7f80de600640 1 -- start start 2024-04-26T23:56:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d80cd770 0x7f80d815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815edf0 0x7f80d8158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80d80cc200 con 0x7f80d815aee0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80d80cbf20 con 0x7f80d80cd770 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80de600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f80d80cb1a0 con 0x7f80d815edf0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48104/0 (socket says 172.21.15.46:48104) 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.183+0000 7f80ddc00640 1 -- 172.21.15.46:0/2026220393 learned_addr learned my addr 172.21.15.46:0/2026220393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80dd200640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d80cd770 0x7f80d815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80d7e00640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815edf0 0x7f80d8158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 -- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815edf0 msgr2=0x7f80d8158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815edf0 0x7f80d8158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 -- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d80cd770 msgr2=0x7f80d815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d80cd770 0x7f80d815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 -- 172.21.15.46:0/2026220393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d8072880 con 0x7f80d815aee0 2024-04-26T23:56:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80ddc00640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009f00 tx=0x7f80d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0018030 con 0x7f80d815aee0 2024-04-26T23:56:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.184+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d000ce50 con 0x7f80d815aee0 2024-04-26T23:56:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.185+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0003bb0 con 0x7f80d815aee0 2024-04-26T23:56:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.185+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d8072b60 con 0x7f80d815aee0 2024-04-26T23:56:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.185+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d8073640 con 0x7f80d815aee0 2024-04-26T23:56:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.186+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d80693b0 con 0x7f80d815aee0 2024-04-26T23:56:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.187+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80d001b070 con 0x7f80d815aee0 2024-04-26T23:56:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.188+0000 7f80d4c00640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 0x7f80b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.188+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7f80d008c460 con 0x7f80d815aee0 2024-04-26T23:56:56.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.189+0000 7f80dd200640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 0x7f80b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.190+0000 7f80dd200640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 0x7f80b806edb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000f60 tx=0x7f80cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.192+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80d0010040 con 0x7f80d815aee0 2024-04-26T23:56:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.400+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80d815e270 con 0x7f80d815aee0 2024-04-26T23:56:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.401+0000 7f80d4c00640 1 -- 172.21.15.46:0/2026220393 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) ==== 74+0+13267 (secure 0 0 0) 0x7f80d0012340 con 0x7f80d815aee0 2024-04-26T23:56:56.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:56.403 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":202,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:56:54.924403+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/4111191104":"2024-04-27T23:56:51.904474+0000","172.21.15.46:6832/4111191104":"2024-04-27T23:56:51.904474+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:56:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 msgr2=0x7f80b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 0x7f80b806edb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000f60 tx=0x7f80cc009040 comp rx=0 tx=0).stop 2024-04-26T23:56:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 msgr2=0x7f80d815b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009f00 tx=0x7f80d000c490 comp rx=0 tx=0).stop 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 shutdown_connections 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f80b806c8f0 0x7f80b806edb0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80d815aee0 0x7f80d815b360 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80d815edf0 0x7f80d8158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 --2- 172.21.15.46:0/2026220393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d80cd770 0x7f80d815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.404+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 >> 172.21.15.46:0/2026220393 conn(0x7f80d80b26d0 msgr2=0x7f80d80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.405+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 shutdown_connections 2024-04-26T23:56:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.405+0000 7f80de600640 1 -- 172.21.15.46:0/2026220393 wait complete. 2024-04-26T23:56:56.427 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:56:56.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:56:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.657+0000 7fac74a00640 1 Processor -- start 2024-04-26T23:56:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.657+0000 7fac74a00640 1 -- start start 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.659+0000 7fac74a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac700cfdf0 0x7fac700b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.659+0000 7fac74a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 0x7fac700afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.659+0000 7fac74a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac700b3360 con 0x7fac7006d790 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.659+0000 7fac74a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fac700b3f70 con 0x7fac700cfdf0 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.659+0000 7fac74a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac700b3640 con 0x7fac7006e580 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac700cfdf0 0x7fac700b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac700cfdf0 0x7fac700b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48110/0 (socket says 172.21.15.46:48110) 2024-04-26T23:56:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 -- 172.21.15.46:0/3526799405 learned_addr learned my addr 172.21.15.46:0/3526799405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 -- 172.21.15.46:0/3526799405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 msgr2=0x7fac700afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6e800640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 0x7fac700afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 0x7fac700afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 -- 172.21.15.46:0/3526799405 >> v1:172.21.15.46:6789/0 conn(0x7fac7006d790 legacy=0x7fac7006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6de00640 1 -- 172.21.15.46:0/3526799405 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac700b3920 con 0x7fac700cfdf0 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6ca00640 1 -- 172.21.15.46:0/3526799405 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488313533 0 0) 0x7fac700b3360 con 0x7fac7006d790 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.660+0000 7fac6e800640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 0x7fac700afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.661+0000 7fac6de00640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac700cfdf0 0x7fac700b4400 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac6002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882a034ddbd7f41d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac6ca00640 1 -- 172.21.15.46:0/3526799405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60037070 con 0x7fac700cfdf0 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac6ca00640 1 -- 172.21.15.46:0/3526799405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac60032440 con 0x7fac700cfdf0 2024-04-26T23:56:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac6ca00640 1 -- 172.21.15.46:0/3526799405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60032810 con 0x7fac700cfdf0 2024-04-26T23:56:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 -- 172.21.15.46:0/3526799405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac700cfdf0 msgr2=0x7fac700b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac700cfdf0 0x7fac700b4400 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac6002a250 comp rx=0 tx=0).stop 2024-04-26T23:56:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 -- 172.21.15.46:0/3526799405 shutdown_connections 2024-04-26T23:56:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac7006e580 0x7fac700afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 --2- 172.21.15.46:0/3526799405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac700cfdf0 0x7fac700b4400 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.662+0000 7fac74a00640 1 -- 172.21.15.46:0/3526799405 >> 172.21.15.46:0/3526799405 conn(0x7fac700d2e90 msgr2=0x7fac700d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.663+0000 7fac74a00640 1 -- 172.21.15.46:0/3526799405 shutdown_connections 2024-04-26T23:56:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.663+0000 7fac74a00640 1 -- 172.21.15.46:0/3526799405 wait complete. 2024-04-26T23:56:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.664+0000 7fac74a00640 1 Processor -- start 2024-04-26T23:56:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.665+0000 7fac74a00640 1 -- start start 2024-04-26T23:56:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.665+0000 7fac74a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.665+0000 7fac74a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac7006e580 0x7fac70161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac74a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac700cfdf0 0x7fac7015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac74a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac700d01d0 con 0x7fac7006d790 2024-04-26T23:56:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac74a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fac7006dbf0 con 0x7fac7006e580 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac74a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac700b3010 con 0x7fac700cfdf0 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48118/0 (socket says 172.21.15.46:48118) 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 -- 172.21.15.46:0/951419498 learned_addr learned my addr 172.21.15.46:0/951419498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 -- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac700cfdf0 msgr2=0x7fac7015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac700cfdf0 0x7fac7015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 -- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac7006e580 msgr2=0x7fac70161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6d400640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac7006e580 0x7fac70161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac7006e580 0x7fac70161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.666+0000 7fac6de00640 1 -- 172.21.15.46:0/951419498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7015da80 con 0x7fac7006d790 2024-04-26T23:56:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.667+0000 7fac6de00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.667+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60037040 con 0x7fac7006d790 2024-04-26T23:56:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.667+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac60004230 con 0x7fac7006d790 2024-04-26T23:56:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.667+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60003610 con 0x7fac7006d790 2024-04-26T23:56:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.667+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7015dd60 con 0x7fac7006d790 2024-04-26T23:56:56.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.668+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7015e320 con 0x7fac7006d790 2024-04-26T23:56:56.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.669+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac70069530 con 0x7fac7006d790 2024-04-26T23:56:56.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.670+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac600037b0 con 0x7fac7006d790 2024-04-26T23:56:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.671+0000 7fac66000640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.671+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7fac600ac9e0 con 0x7fac7006d790 2024-04-26T23:56:56.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.672+0000 7fac6d400640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:56.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.673+0000 7fac6d400640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 0x7fac5006edb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fac700c9e60 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.676+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac6007af70 con 0x7fac7006d790 2024-04-26T23:56:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.885+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac700ce240 con 0x7fac7006d790 2024-04-26T23:56:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.885+0000 7fac66000640 1 -- 172.21.15.46:0/951419498 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) ==== 93+0+10 (secure 0 0 0) 0x7fac6007a870 con 0x7fac7006d790 2024-04-26T23:56:56.887 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 msgr2=0x7fac5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 0x7fac5006edb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fac700c9e60 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 msgr2=0x7fac70157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac600046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 shutdown_connections 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac700cfdf0 0x7fac7015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac7006e580 0x7fac70161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 --2- 172.21.15.46:0/951419498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7006d790 0x7fac70157350 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 >> 172.21.15.46:0/951419498 conn(0x7fac700d2e90 msgr2=0x7fac700b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 shutdown_connections 2024-04-26T23:56:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:56.888+0000 7fac74a00640 1 -- 172.21.15.46:0/951419498 wait complete. 2024-04-26T23:56:56.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-26T23:56:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.114+0000 7fe92a400640 1 Processor -- start 2024-04-26T23:56:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.115+0000 7fe92a400640 1 -- start start 2024-04-26T23:56:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.116+0000 7fe92a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cff40 0x7fe924073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.116+0000 7fe92a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe924072190 0x7fe924166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.116+0000 7fe92a400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92406f1d0 con 0x7fe9240cf450 2024-04-26T23:56:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.116+0000 7fe92a400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe92406f470 con 0x7fe924072190 2024-04-26T23:56:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.116+0000 7fe92a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe924072930 con 0x7fe9240cff40 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe924072190 0x7fe924166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe924072190 0x7fe924166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48126/0 (socket says 172.21.15.46:48126) 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 -- 172.21.15.46:0/440537529 learned_addr learned my addr 172.21.15.46:0/440537529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 -- 172.21.15.46:0/440537529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cff40 msgr2=0x7fe924073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe923e00640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cff40 0x7fe924073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cff40 0x7fe924073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 -- 172.21.15.46:0/440537529 >> v1:172.21.15.142:6789/0 conn(0x7fe9240cf450 legacy=0x7fe9240cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.117+0000 7fe929a00640 1 -- 172.21.15.46:0/440537529 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe924072c10 con 0x7fe924072190 2024-04-26T23:56:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.118+0000 7fe929a00640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe924166b00 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009e50 tx=0x7fe91c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc561aef00aee476 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.119+0000 7fe923400640 1 -- 172.21.15.46:0/440537529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c036070 con 0x7fe924072190 2024-04-26T23:56:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.119+0000 7fe923400640 1 -- 172.21.15.46:0/440537529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c031600 con 0x7fe924072190 2024-04-26T23:56:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.119+0000 7fe923400640 1 -- 172.21.15.46:0/440537529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0319d0 con 0x7fe924072190 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.119+0000 7fe92a400640 1 -- 172.21.15.46:0/440537529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 msgr2=0x7fe924166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.119+0000 7fe92a400640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe924166b00 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009e50 tx=0x7fe91c02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 -- 172.21.15.46:0/440537529 shutdown_connections 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe924166b00 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 --2- 172.21.15.46:0/440537529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cff40 0x7fe924073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 -- 172.21.15.46:0/440537529 >> 172.21.15.46:0/440537529 conn(0x7fe9240b2820 msgr2=0x7fe9240b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 -- 172.21.15.46:0/440537529 shutdown_connections 2024-04-26T23:56:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.120+0000 7fe92a400640 1 -- 172.21.15.46:0/440537529 wait complete. 2024-04-26T23:56:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.122+0000 7fe92a400640 1 Processor -- start 2024-04-26T23:56:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.122+0000 7fe92a400640 1 -- start start 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 0x7fe924141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9240cff40 0x7fe9241416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9240734b0 con 0x7fe924072190 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9240731d0 con 0x7fe9240cff40 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe92a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe924072600 con 0x7fe9240cf450 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe929000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe929000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48140/0 (socket says 172.21.15.46:48140) 2024-04-26T23:56:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe929000640 1 -- 172.21.15.46:0/3829915702 learned_addr learned my addr 172.21.15.46:0/3829915702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.123+0000 7fe923e00640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 0x7fe924141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 -- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 msgr2=0x7fe924141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 0x7fe924141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 -- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9240cff40 msgr2=0x7fe9241416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9240cff40 0x7fe9241416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 -- 172.21.15.46:0/3829915702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9240c5b30 con 0x7fe924072190 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe923e00640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 0x7fe924141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe929000640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c460 tx=0x7fe91800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918016020 con 0x7fe924072190 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe918012870 con 0x7fe924072190 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918012c60 con 0x7fe924072190 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9240c5e10 con 0x7fe924072190 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.124+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9240c63d0 con 0x7fe924072190 2024-04-26T23:56:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.125+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9240693b0 con 0x7fe924072190 2024-04-26T23:56:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.126+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe918019020 con 0x7fe924072190 2024-04-26T23:56:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.127+0000 7fe920c00640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 0x7fe90406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.128+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7fe91808b7c0 con 0x7fe924072190 2024-04-26T23:56:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.128+0000 7fe923e00640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 0x7fe90406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.129+0000 7fe923e00640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 0x7fe90406edb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe9241f8370 tx=0x7fe910008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.132+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe918020030 con 0x7fe924072190 2024-04-26T23:56:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.408+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) -- 0x7fe9240cec50 con 0x7fe924072190 2024-04-26T23:56:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.958+0000 7fe920c00640 1 -- 172.21.15.46:0/3829915702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v910) ==== 136+0+0 (secure 0 0 0) 0x7fe918059cc0 con 0x7fe924072190 2024-04-26T23:56:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.961+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 msgr2=0x7fe90406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.961+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 0x7fe90406edb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe9241f8370 tx=0x7fe910008040 comp rx=0 tx=0).stop 2024-04-26T23:56:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.961+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 msgr2=0x7fe92413f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.961+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c460 tx=0x7fe91800c920 comp rx=0 tx=0).stop 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 shutdown_connections 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe90406c8f0 0x7fe90406edb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9240cff40 0x7fe9241416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9240cf450 0x7fe924141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 --2- 172.21.15.46:0/3829915702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe924072190 0x7fe92413f840 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 >> 172.21.15.46:0/3829915702 conn(0x7fe9240b2820 msgr2=0x7fe9240cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 shutdown_connections 2024-04-26T23:56:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:57.962+0000 7fe92a400640 1 -- 172.21.15.46:0/3829915702 wait complete. 2024-04-26T23:56:57.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:56:58.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.194+0000 7f301fe00640 1 Processor -- start 2024-04-26T23:56:58.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.194+0000 7f301fe00640 1 -- start start 2024-04-26T23:56:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f302015cec0 0x7f30200cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30200cdb30 0x7f30200735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30200ce7d0 con 0x7f3020073b30 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f30200ceab0 con 0x7f302015cec0 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f30200ced90 con 0x7f30200cdb30 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f302015cec0 0x7f30200cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f302015cec0 0x7f30200cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48156/0 (socket says 172.21.15.46:48156) 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301ea00640 1 -- 172.21.15.46:0/670553927 learned_addr learned my addr 172.21.15.46:0/670553927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.196+0000 7f301f400640 1 --1- 172.21.15.46:0/670553927 >> v1:172.21.15.46:6789/0 conn(0x7f3020073b30 0x7f3020071e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42130/0 (socket says 172.21.15.46:42130) 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301e000640 1 --2- 172.21.15.46:0/670553927 >> v2:172.21.15.142:3300/0 conn(0x7f30200cdb30 0x7f30200735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301ea00640 1 -- 172.21.15.46:0/670553927 >> v2:172.21.15.142:3300/0 conn(0x7f30200cdb30 msgr2=0x7f30200735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301ea00640 1 --2- 172.21.15.46:0/670553927 >> v2:172.21.15.142:3300/0 conn(0x7f30200cdb30 0x7f30200735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301ea00640 1 -- 172.21.15.46:0/670553927 >> v1:172.21.15.46:6789/0 conn(0x7f3020073b30 legacy=0x7f3020071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301ea00640 1 -- 172.21.15.46:0/670553927 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30200cf070 con 0x7f302015cec0 2024-04-26T23:56:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.197+0000 7f301d600640 1 -- 172.21.15.46:0/670553927 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416219752 0 0) 0x7f30200ce7d0 con 0x7f3020073b30 2024-04-26T23:56:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.198+0000 7f301ea00640 1 --2- 172.21.15.46:0/670553927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200cd5f0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f3020074300 tx=0x7f301002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52113e128bf9f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.198+0000 7f301d600640 1 -- 172.21.15.46:0/670553927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010036070 con 0x7f302015cec0 2024-04-26T23:56:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.198+0000 7f301d600640 1 -- 172.21.15.46:0/670553927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010031600 con 0x7f302015cec0 2024-04-26T23:56:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.198+0000 7f301d600640 1 -- 172.21.15.46:0/670553927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100319d0 con 0x7f302015cec0 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 -- 172.21.15.46:0/670553927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 msgr2=0x7f30200cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 --2- 172.21.15.46:0/670553927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200cd5f0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f3020074300 tx=0x7f301002a730 comp rx=0 tx=0).stop 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 -- 172.21.15.46:0/670553927 shutdown_connections 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 --2- 172.21.15.46:0/670553927 >> v2:172.21.15.142:3300/0 conn(0x7f30200cdb30 0x7f30200735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 --2- 172.21.15.46:0/670553927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200cd5f0 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 -- 172.21.15.46:0/670553927 >> 172.21.15.46:0/670553927 conn(0x7f30200b27b0 msgr2=0x7f30200b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 -- 172.21.15.46:0/670553927 shutdown_connections 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.199+0000 7f301fe00640 1 -- 172.21.15.46:0/670553927 wait complete. 2024-04-26T23:56:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.201+0000 7f301fe00640 1 Processor -- start 2024-04-26T23:56:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.201+0000 7f301fe00640 1 -- start start 2024-04-26T23:56:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 0x7f3020156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30200cdb30 0x7f3020157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3020073f70 con 0x7f302015cec0 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f302015d2a0 con 0x7f3020073b30 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.202+0000 7f301fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f30200b4b50 con 0x7f30200cdb30 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48160/0 (socket says 172.21.15.46:48160) 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 -- 172.21.15.46:0/2656674801 learned_addr learned my addr 172.21.15.46:0/2656674801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301e000640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30200cdb30 0x7f3020157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301ea00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 0x7f3020156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 -- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30200cdb30 msgr2=0x7f3020157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30200cdb30 0x7f3020157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 -- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 msgr2=0x7f3020156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 0x7f3020156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 -- 172.21.15.46:0/2656674801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30200c6590 con 0x7f302015cec0 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301ea00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 0x7f3020156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:56:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.203+0000 7f301f400640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f301800c3c0 tx=0x7f301800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.204+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30180074e0 con 0x7f302015cec0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.204+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30180047e0 con 0x7f302015cec0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.204+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018004c10 con 0x7f302015cec0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.204+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302015d550 con 0x7f302015cec0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.204+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302015db10 con 0x7f302015cec0 2024-04-26T23:56:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.205+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30200b68e0 con 0x7f302015cec0 2024-04-26T23:56:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.206+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3018011070 con 0x7f302015cec0 2024-04-26T23:56:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.207+0000 7f3016a00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 0x7f2ffc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.207+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7f301808b5a0 con 0x7f302015cec0 2024-04-26T23:56:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.208+0000 7f301ea00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 0x7f2ffc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.210+0000 7f301ea00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 0x7f2ffc06edb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3010033000 tx=0x7f3010009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:58.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.212+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f301801a2b0 con 0x7f302015cec0 2024-04-26T23:56:58.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.418+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f30200693b0 con 0x7f302015cec0 2024-04-26T23:56:58.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.426+0000 7f3016a00640 1 -- 172.21.15.46:0/2656674801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f3018059b30 con 0x7f302015cec0 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 msgr2=0x7f2ffc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 0x7f2ffc06edb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3010033000 tx=0x7f3010009000 comp rx=0 tx=0).stop 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 msgr2=0x7f30200b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f301800c3c0 tx=0x7f301800c880 comp rx=0 tx=0).stop 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 shutdown_connections 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2ffc06c8f0 0x7f2ffc06edb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f302015cec0 0x7f30200b5420 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30200cdb30 0x7f3020157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 --2- 172.21.15.46:0/2656674801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3020073b30 0x7f3020156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.429+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 >> 172.21.15.46:0/2656674801 conn(0x7f30200b27b0 msgr2=0x7f30200758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.430+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 shutdown_connections 2024-04-26T23:56:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:58.430+0000 7f301fe00640 1 -- 172.21.15.46:0/2656674801 wait complete. 2024-04-26T23:56:58.452 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:56:58.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:58.576 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:56:58.694 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:56:58.694 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:58.694 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:58.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:58.755 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:58.756 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:58.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:58.813 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:56:58.813 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:58.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:56:58.875 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:56:58.875 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:56:58.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:56:58.931 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:56:58.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:56:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.151+0000 7fb257400640 1 Processor -- start 2024-04-26T23:56:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.151+0000 7fb257400640 1 -- start start 2024-04-26T23:56:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb257400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb25006e510 0x7fb25006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb257400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb250073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb257400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25006ee30 con 0x7fb250160bd0 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb257400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb250161940 con 0x7fb25006e510 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb257400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb250071f80 con 0x7fb2500732c0 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb254a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb25006e510 0x7fb25006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb254a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb25006e510 0x7fb25006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48176/0 (socket says 172.21.15.46:48176) 2024-04-26T23:56:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.153+0000 7fb254a00640 1 -- 172.21.15.46:0/2646513624 learned_addr learned my addr 172.21.15.46:0/2646513624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.154+0000 7fb255400640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb250073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.154+0000 7fb254a00640 1 -- 172.21.15.46:0/2646513624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 msgr2=0x7fb250073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.154+0000 7fb254a00640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb250073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.154+0000 7fb254a00640 1 -- 172.21.15.46:0/2646513624 >> v1:172.21.15.142:6789/0 conn(0x7fb250160bd0 legacy=0x7fb250161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.154+0000 7fb254a00640 1 -- 172.21.15.46:0/2646513624 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb250072260 con 0x7fb25006e510 2024-04-26T23:56:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.155+0000 7fb254a00640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25006e510 0x7fb25006e8f0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fb2440098a0 tx=0x7fb24402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe54b912df6d31f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.155+0000 7fb24f400640 1 -- 172.21.15.46:0/2646513624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244037070 con 0x7fb25006e510 2024-04-26T23:56:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.155+0000 7fb24f400640 1 -- 172.21.15.46:0/2646513624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb244032440 con 0x7fb25006e510 2024-04-26T23:56:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.155+0000 7fb24f400640 1 -- 172.21.15.46:0/2646513624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244032810 con 0x7fb25006e510 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 -- 172.21.15.46:0/2646513624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25006e510 msgr2=0x7fb25006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25006e510 0x7fb25006e8f0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fb2440098a0 tx=0x7fb24402a250 comp rx=0 tx=0).stop 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 -- 172.21.15.46:0/2646513624 shutdown_connections 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb250073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 --2- 172.21.15.46:0/2646513624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25006e510 0x7fb25006e8f0 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 -- 172.21.15.46:0/2646513624 >> 172.21.15.46:0/2646513624 conn(0x7fb25006c530 msgr2=0x7fb25006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 -- 172.21.15.46:0/2646513624 shutdown_connections 2024-04-26T23:56:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.156+0000 7fb257400640 1 -- 172.21.15.46:0/2646513624 wait complete. 2024-04-26T23:56:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.158+0000 7fb257400640 1 Processor -- start 2024-04-26T23:56:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.159+0000 7fb257400640 1 -- start start 2024-04-26T23:56:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25006e510 0x7fb25013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb2500b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb250072d40 con 0x7fb250160bd0 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb250072820 con 0x7fb25006e510 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb257400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb250072ec0 con 0x7fb2500732c0 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb255400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb255400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48184/0 (socket says 172.21.15.46:48184) 2024-04-26T23:56:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.160+0000 7fb255400640 1 -- 172.21.15.46:0/573008265 learned_addr learned my addr 172.21.15.46:0/573008265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb254a00640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25006e510 0x7fb25013c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb24fe00640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb2500b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 -- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 msgr2=0x7fb2500b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb2500b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 -- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25006e510 msgr2=0x7fb25013c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25006e510 0x7fb25013c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 -- 172.21.15.46:0/573008265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2500b45e0 con 0x7fb250160bd0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb255400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fb24800cd70 tx=0x7fb24800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.161+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248003b90 con 0x7fb250160bd0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.162+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248003d30 con 0x7fb250160bd0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.162+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248004240 con 0x7fb250160bd0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.162+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2500b48c0 con 0x7fb250160bd0 2024-04-26T23:56:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.162+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2500b4e80 con 0x7fb250160bd0 2024-04-26T23:56:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.163+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250069530 con 0x7fb250160bd0 2024-04-26T23:56:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.164+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2480043e0 con 0x7fb250160bd0 2024-04-26T23:56:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.165+0000 7fb24cc00640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 0x7fb23006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.165+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7fb24808c540 con 0x7fb250160bd0 2024-04-26T23:56:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.166+0000 7fb254a00640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 0x7fb23006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.167+0000 7fb254a00640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 0x7fb23006ede0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb244034000 tx=0x7fb24402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.170+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb24805aad0 con 0x7fb250160bd0 2024-04-26T23:56:59.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.413+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb25015f5a0 con 0x7fb250160bd0 2024-04-26T23:56:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.414+0000 7fb24cc00640 1 -- 172.21.15.46:0/573008265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+3872 (secure 0 0 0) 0x7fb24805a3d0 con 0x7fb250160bd0 2024-04-26T23:56:59.416 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:56:59.416 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":160,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1720824451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1720824451},{"type":"v1","addr":"172.21.15.142:6837","nonce":1720824451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8041,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3871883750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3871883750},{"type":"v1","addr":"172.21.15.46:6835","nonce":3871883750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/75514027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":75514027},{"type":"v1","addr":"172.21.15.142:6835","nonce":75514027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":153},{"gid":11982,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/843868406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":843868406},{"type":"v1","addr":"172.21.15.46:6834","nonce":843868406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[]} 2024-04-26T23:56:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 160 2024-04-26T23:56:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.416+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 msgr2=0x7fb23006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.416+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 0x7fb23006ede0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb244034000 tx=0x7fb24402e000 comp rx=0 tx=0).stop 2024-04-26T23:56:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 msgr2=0x7fb2500b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fb24800cd70 tx=0x7fb24800a660 comp rx=0 tx=0).stop 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 shutdown_connections 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb23006c920 0x7fb23006ede0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb250160bd0 0x7fb2500b5bc0 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2500732c0 0x7fb2500b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 --2- 172.21.15.46:0/573008265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb25006e510 0x7fb25013c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 >> 172.21.15.46:0/573008265 conn(0x7fb25006c530 msgr2=0x7fb2500b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 shutdown_connections 2024-04-26T23:56:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.417+0000 7fb257400640 1 -- 172.21.15.46:0/573008265 wait complete. 2024-04-26T23:56:59.440 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:56:59.440 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:56:59.440 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:56:59.440 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:59.441 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:56:59.441 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:56:59.441 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:56:59.441 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:59.441 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:56:59.441 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:56:59.442 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:56:59.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:59.442 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:56:59.442 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:56:59.442 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:56:59.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:56:59.459 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:56:59.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:56:59.460 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:56:59.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:56:59.461 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:56:59.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:56:59.466 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:56:59.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:56:59.505 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:56:59.505 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:59.509 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:56:59.509 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:59.512 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:56:59.513 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:59.519 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:56:59.519 DEBUG:teuthology.parallel:result is None 2024-04-26T23:56:59.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:56:59.654 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-26T23:56:59.655 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:56:59.667 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-04-26T23:56:59.669 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-04-26T23:56:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.708+0000 7f3731200640 1 Processor -- start 2024-04-26T23:56:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.708+0000 7f3731200640 1 -- start start 2024-04-26T23:56:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.709+0000 7f3731200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f372c160a30 0x7f372c0714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.709+0000 7f3731200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f372c140570 0x7f372c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.709+0000 7f3731200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c160fd0 con 0x7f372c0710d0 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.709+0000 7f3731200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f372c1612b0 con 0x7f372c160a30 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.709+0000 7f3731200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f372c161590 con 0x7f372c140570 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f372c160a30 0x7f372c0714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f372c160a30 0x7f372c0714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48204/0 (socket says 172.21.15.46:48204) 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 -- 172.21.15.46:0/3557384380 learned_addr learned my addr 172.21.15.46:0/3557384380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372be00640 1 --2- 172.21.15.46:0/3557384380 >> v2:172.21.15.142:3300/0 conn(0x7f372c140570 0x7f372c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 -- 172.21.15.46:0/3557384380 >> v2:172.21.15.142:3300/0 conn(0x7f372c140570 msgr2=0x7f372c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 --2- 172.21.15.46:0/3557384380 >> v2:172.21.15.142:3300/0 conn(0x7f372c140570 0x7f372c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 -- 172.21.15.46:0/3557384380 >> v1:172.21.15.142:6789/0 conn(0x7f372c0710d0 legacy=0x7f372c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.710+0000 7f372aa00640 1 -- 172.21.15.46:0/3557384380 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c161870 con 0x7f372c160a30 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.711+0000 7f372aa00640 1 --2- 172.21.15.46:0/3557384380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c160a30 0x7f372c0714b0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f372c1435c0 tx=0x7f371402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637c62cee885e97b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.711+0000 7f372a000640 1 -- 172.21.15.46:0/3557384380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714037070 con 0x7f372c160a30 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f372a000640 1 -- 172.21.15.46:0/3557384380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3714032440 con 0x7f372c160a30 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f372a000640 1 -- 172.21.15.46:0/3557384380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714032810 con 0x7f372c160a30 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 -- 172.21.15.46:0/3557384380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c160a30 msgr2=0x7f372c0714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 --2- 172.21.15.46:0/3557384380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c160a30 0x7f372c0714b0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f372c1435c0 tx=0x7f371402a230 comp rx=0 tx=0).stop 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 -- 172.21.15.46:0/3557384380 shutdown_connections 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 --2- 172.21.15.46:0/3557384380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c160a30 0x7f372c0714b0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 --2- 172.21.15.46:0/3557384380 >> v2:172.21.15.142:3300/0 conn(0x7f372c140570 0x7f372c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 -- 172.21.15.46:0/3557384380 >> 172.21.15.46:0/3557384380 conn(0x7f372c0d1c70 msgr2=0x7f372c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 -- 172.21.15.46:0/3557384380 shutdown_connections 2024-04-26T23:56:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.712+0000 7f3731200640 1 -- 172.21.15.46:0/3557384380 wait complete. 2024-04-26T23:56:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.714+0000 7f3731200640 1 Processor -- start 2024-04-26T23:56:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.714+0000 7f3731200640 1 -- start start 2024-04-26T23:56:59.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f372c0710d0 0x7f372c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f372c160a30 0x7f372c0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f372c140b00 con 0x7f372c140570 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f372c140980 con 0x7f372c0710d0 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f3731200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f372c140c80 con 0x7f372c160a30 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.715+0000 7f372aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48214/0 (socket says 172.21.15.46:48214) 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 -- 172.21.15.46:0/3433949250 learned_addr learned my addr 172.21.15.46:0/3433949250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372b400640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f372c0710d0 0x7f372c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372be00640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f372c160a30 0x7f372c0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 -- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f372c160a30 msgr2=0x7f372c0af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f372c160a30 0x7f372c0af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 -- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f372c0710d0 msgr2=0x7f372c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f372c0710d0 0x7f372c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 -- 172.21.15.46:0/3433949250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c0b09b0 con 0x7f372c140570 2024-04-26T23:56:59.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.716+0000 7f372aa00640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f37140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.717+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714041020 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.717+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37140306c0 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.717+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714037040 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.717+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c0afda0 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.717+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c0b0360 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.718+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c069ac0 con 0x7f372c140570 2024-04-26T23:56:59.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.719+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3714044020 con 0x7f372c140570 2024-04-26T23:56:59.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.720+0000 7f3723400640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 0x7f370806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:59.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.720+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5335+0+0 (secure 0 0 0) 0x7f37140ac100 con 0x7f372c140570 2024-04-26T23:56:59.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.721+0000 7f372b400640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 0x7f370806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:59.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.721+0000 7f372b400640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 0x7f370806ede0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f371c006fd0 tx=0x7f371c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:59.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.724+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f371404b030 con 0x7f372c140570 2024-04-26T23:56:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.942+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f372c0b49b0 con 0x7f372c140570 2024-04-26T23:56:59.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.990+0000 7f3723400640 1 -- 172.21.15.46:0/3433949250 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v204) ==== 97+0+0 (secure 0 0 0) 0x7f371407a040 con 0x7f372c140570 2024-04-26T23:56:59.992 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 msgr2=0x7f370806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 0x7f370806ede0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f371c006fd0 tx=0x7f371c009060 comp rx=0 tx=0).stop 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 msgr2=0x7f372c0af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f37140046c0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 shutdown_connections 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f370806c920 0x7f370806ede0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f372c160a30 0x7f372c0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c140570 0x7f372c0af120 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 --2- 172.21.15.46:0/3433949250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f372c0710d0 0x7f372c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:59.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.994+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 >> 172.21.15.46:0/3433949250 conn(0x7f372c0d1c70 msgr2=0x7f372c0c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:59.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.995+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 shutdown_connections 2024-04-26T23:56:59.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:56:59.995+0000 7f3731200640 1 -- 172.21.15.46:0/3433949250 wait complete. 2024-04-26T23:57:00.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:57:00.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.218+0000 7f4d8d800640 1 Processor -- start 2024-04-26T23:57:00.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.219+0000 7f4d8d800640 1 -- start start 2024-04-26T23:57:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.220+0000 7f4d8d800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4d881639c0 0x7f4d8816a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.220+0000 7f4d8d800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d880cfe50 con 0x7f4d880d1990 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.220+0000 7f4d8d800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d880d0130 con 0x7f4d880d0c10 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.220+0000 7f4d8d800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880d0780 con 0x7f4d881639c0 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.221+0000 7f4d87e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4d880d1990 0x7f4d880d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42160/0 (socket says 172.21.15.46:42160) 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.221+0000 7f4d87e00640 1 -- 172.21.15.46:0/345159274 learned_addr learned my addr 172.21.15.46:0/345159274 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.221+0000 7f4d8ce00640 1 --2- 172.21.15.46:0/345159274 >> v2:172.21.15.142:3300/0 conn(0x7f4d881639c0 0x7f4d8816a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.221+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613879552 0 0) 0x7f4d880cfe50 con 0x7f4d880d1990 2024-04-26T23:57:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.221+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d6c003520 con 0x7f4d880d1990 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2708568504 0 0) 0x7f4d6c003520 con 0x7f4d880d1990 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 >> v2:172.21.15.142:3300/0 conn(0x7f4d881639c0 msgr2=0x7f4d8816a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 --2- 172.21.15.46:0/345159274 >> v2:172.21.15.142:3300/0 conn(0x7f4d881639c0 0x7f4d8816a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 >> v1:172.21.15.142:6789/0 conn(0x7f4d880d0c10 legacy=0x7f4d880d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d880d0410 con 0x7f4d880d1990 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4d78001670 con 0x7f4d880d1990 2024-04-26T23:57:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 >> v1:172.21.15.46:6789/0 conn(0x7f4d880d1990 legacy=0x7f4d880d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d6c004800 0x7f4d6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d6c0051c0 0x7f4d6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880d0780 con 0x7f4d6c0039b0 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c00c060 con 0x7f4d6c004800 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6c00c260 con 0x7f4d6c0051c0 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87e00640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d7f400640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d6c004800 0x7f4d6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87e00640 1 -- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d6c0051c0 msgr2=0x7f4d6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87e00640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d6c0051c0 0x7f4d6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.222+0000 7f4d87e00640 1 -- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d6c004800 msgr2=0x7f4d6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.223+0000 7f4d87e00640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d6c004800 0x7f4d6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.223+0000 7f4d8ce00640 1 -- 172.21.15.46:0/345159274 reap_dead start 2024-04-26T23:57:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.223+0000 7f4d87e00640 1 -- 172.21.15.46:0/345159274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c003520 con 0x7f4d6c0039b0 2024-04-26T23:57:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.223+0000 7f4d7f400640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d6c004800 0x7f4d6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.224+0000 7f4d87e00640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d6c003dd0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c210 tx=0x7f4d6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dc917368333b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.224+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78007d30 con 0x7f4d6c0039b0 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.224+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d780058d0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.224+0000 7f4d87400640 1 -- 172.21.15.46:0/345159274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78005d20 con 0x7f4d6c0039b0 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 -- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 msgr2=0x7f4d6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d6c003dd0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c210 tx=0x7f4d6c005830 comp rx=0 tx=0).stop 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 -- 172.21.15.46:0/345159274 shutdown_connections 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 --2- 172.21.15.46:0/345159274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d6c003dd0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 -- 172.21.15.46:0/345159274 >> 172.21.15.46:0/345159274 conn(0x7f4d8809e2a0 msgr2=0x7f4d880d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 -- 172.21.15.46:0/345159274 shutdown_connections 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.225+0000 7f4d8d800640 1 -- 172.21.15.46:0/345159274 wait complete. 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.227+0000 7f4d8d800640 1 Processor -- start 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.227+0000 7f4d8d800640 1 -- start start 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d880ce740 0x7f4d881eab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d880cebc0 0x7f4d881eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880cfc10 con 0x7f4d6c0039b0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880cfa90 con 0x7f4d880cebc0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d8d800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4d880d1d70 con 0x7f4d880ce740 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48242/0 (socket says 172.21.15.46:48242) 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.228+0000 7f4d87e00640 1 -- 172.21.15.46:0/1323933489 learned_addr learned my addr 172.21.15.46:0/1323933489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d7f400640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d880ce740 0x7f4d881eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 -- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d880ce740 msgr2=0x7f4d881eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d880ce740 0x7f4d881eab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 -- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d880cebc0 msgr2=0x7f4d881eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d880cebc0 0x7f4d881eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 -- 172.21.15.46:0/1323933489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d881e8ef0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.229+0000 7f4d87e00640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c210 tx=0x7f4d78001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.230+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78038040 con 0x7f4d6c0039b0 2024-04-26T23:57:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.230+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d78042070 con 0x7f4d6c0039b0 2024-04-26T23:57:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.230+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78000ed0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.230+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d881e91d0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.230+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881e9790 con 0x7f4d6c0039b0 2024-04-26T23:57:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.231+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880693b0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.232+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d78007070 con 0x7f4d6c0039b0 2024-04-26T23:57:00.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.233+0000 7f4d84c00640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 0x7f4d6406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.233+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 5249+0+0 (secure 0 0 0) 0x7f4d780ab6a0 con 0x7f4d6c0039b0 2024-04-26T23:57:00.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.234+0000 7f4d7f400640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 0x7f4d6406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.235+0000 7f4d7f400640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 0x7f4d6406ef80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.239+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d78079630 con 0x7f4d6c0039b0 2024-04-26T23:57:00.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.513+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4d880cb630 con 0x7f4d6c0039b0 2024-04-26T23:57:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.514+0000 7f4d84c00640 1 -- 172.21.15.46:0/1323933489 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5304 (secure 0 0 0) 0x7f4d7807d930 con 0x7f4d6c0039b0 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmwiF1HRAAX1UAGggy2rwMQgzjxQ8VUg==", 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.516 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmSZqLNhAAszdR4vpAaUbDctYziJTj6g==", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmysWnKBAA5CB1opz6mg2wUeVVNNRq/g==", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-26T23:57:00.518 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmBRFQCRAAq/R/syZYbFhcs88b6vfKYw==", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBtOixmg+FVHRAA6yACMoYHM5Gam+KEI/syRw==", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.519 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBwOixmmC6tKRAAaa3bZ1CKlGdgNISfOxVgcw==", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB0OixmbyJcCBAA0UFOi1FqsPR9uYBEMKBiGw==", 2024-04-26T23:57:00.520 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB3OixmIT3PIhAA5MMHPl77TkeHUGqcfaJWKw==", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB+OixmAixLDBAAk4jHpEaJE2q9pJrBmTF+4Q==", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCBOixmwwRzGBAAdaFHcEg+/h/Pz+FRUfhs2Q==", 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.522 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCEOixmc7csIhAApO0HqrwR1uN6AQ4/O67nNw==", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCHOixmymckOxAAHO67XJfaaywnWDxc3lpzlw==", 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-26T23:57:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA==", 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm7ytaJhAAtWF3u0QoSqTxKnB5My/vKg==", 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.525 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixmInmAHBAAeE19HgK4lJgVnx4eNrYVlQ==", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:57:00.526 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBrOixm8VKPMBAA0HVWgYsThYgrY1Rm8qa4xA==", 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.527 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBpOixmvOA3GxAAMO2XCgGs82h7WDx5NSL+Kg==", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmrMQ0FRAAPWdudU6UivXPuSLc+gdRMw==", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBqOixmqXblCRAABQVfP2wY+osJupbSw3SGDg==", 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-26T23:57:00.529 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 msgr2=0x7f4d6406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 0x7f4d6406ef80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d70009040 comp rx=0 tx=0).stop 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 msgr2=0x7f4d88159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c210 tx=0x7f4d78001a40 comp rx=0 tx=0).stop 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 shutdown_connections 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4d6406cac0 0x7f4d6406ef80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d880cebc0 0x7f4d881eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d880ce740 0x7f4d881eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 --2- 172.21.15.46:0/1323933489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d6c0039b0 0x7f4d88159e70 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.516+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 >> 172.21.15.46:0/1323933489 conn(0x7f4d8809e2a0 msgr2=0x7f4d880b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.517+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 shutdown_connections 2024-04-26T23:57:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.517+0000 7f4d8d800640 1 -- 172.21.15.46:0/1323933489 wait complete. 2024-04-26T23:57:00.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:57:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.720+0000 7fee51a00640 1 Processor -- start 2024-04-26T23:57:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.720+0000 7fee51a00640 1 -- start start 2024-04-26T23:57:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.722+0000 7fee51a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 0x7fee4c073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.722+0000 7fee51a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee4c1635d0 0x7fee4c0739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.722+0000 7fee51a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4c0cdc90 con 0x7fee4c1631f0 2024-04-26T23:57:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.722+0000 7fee51a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fee4c073ee0 con 0x7fee4c1635d0 2024-04-26T23:57:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.722+0000 7fee51a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c0d0ef0 con 0x7fee4c163bb0 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee4c1635d0 0x7fee4c0739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fee4c1635d0 0x7fee4c0739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48250/0 (socket says 172.21.15.46:48250) 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 -- 172.21.15.46:0/3403430811 learned_addr learned my addr 172.21.15.46:0/3403430811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4a000640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 0x7fee4c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 -- 172.21.15.46:0/3403430811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 msgr2=0x7fee4c073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 0x7fee4c073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 -- 172.21.15.46:0/3403430811 >> v1:172.21.15.142:6789/0 conn(0x7fee4c1631f0 legacy=0x7fee4c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4b400640 1 -- 172.21.15.46:0/3403430811 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c0cdf70 con 0x7fee4c1635d0 2024-04-26T23:57:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.723+0000 7fee4a000640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 0x7fee4c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.724+0000 7fee4b400640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c1635d0 0x7fee4c0739a0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fee44009e80 tx=0x7fee4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b917bb0aa73b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee49600640 1 -- 172.21.15.46:0/3403430811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44036070 con 0x7fee4c1635d0 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee49600640 1 -- 172.21.15.46:0/3403430811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee44031600 con 0x7fee4c1635d0 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee49600640 1 -- 172.21.15.46:0/3403430811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440319d0 con 0x7fee4c1635d0 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 -- 172.21.15.46:0/3403430811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c1635d0 msgr2=0x7fee4c0739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c1635d0 0x7fee4c0739a0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fee44009e80 tx=0x7fee4402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 -- 172.21.15.46:0/3403430811 shutdown_connections 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c1635d0 0x7fee4c0739a0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 --2- 172.21.15.46:0/3403430811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c163bb0 0x7fee4c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 -- 172.21.15.46:0/3403430811 >> 172.21.15.46:0/3403430811 conn(0x7fee4c0b2cf0 msgr2=0x7fee4c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 -- 172.21.15.46:0/3403430811 shutdown_connections 2024-04-26T23:57:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.725+0000 7fee51a00640 1 -- 172.21.15.46:0/3403430811 wait complete. 2024-04-26T23:57:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.727+0000 7fee51a00640 1 Processor -- start 2024-04-26T23:57:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.727+0000 7fee51a00640 1 -- start start 2024-04-26T23:57:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 0x7fee4c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 0x7fee4c0ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c15edf0 con 0x7fee4c163bb0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c0ce530 con 0x7fee4c1635d0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee51a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c15ef70 con 0x7fee4c1631f0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48252/0 (socket says 172.21.15.46:48252) 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 -- 172.21.15.46:0/434864792 learned_addr learned my addr 172.21.15.46:0/434864792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4aa00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 0x7fee4c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 -- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 msgr2=0x7fee4c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 0x7fee4c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 -- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 msgr2=0x7fee4c0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4a000640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 0x7fee4c0ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 0x7fee4c0ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4b400640 1 -- 172.21.15.46:0/434864792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c161720 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.729+0000 7fee4aa00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 0x7fee4c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee4a000640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 0x7fee4c0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee4b400640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fee44034040 tx=0x7fee44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44041020 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee44007780 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44036040 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c161a00 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.730+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c161fc0 con 0x7fee4c163bb0 2024-04-26T23:57:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.731+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c0ce770 con 0x7fee4c163bb0 2024-04-26T23:57:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.732+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee44044020 con 0x7fee4c163bb0 2024-04-26T23:57:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.733+0000 7fee42a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 0x7fee2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.733+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 5249+0+0 (secure 0 0 0) 0x7fee440ac6d0 con 0x7fee4c163bb0 2024-04-26T23:57:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.734+0000 7fee4aa00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 0x7fee2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.735+0000 7fee4aa00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 0x7fee2c06ede0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fee3c007fd0 tx=0x7fee3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.738+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee4c0ce770 con 0x7fee4c163bb0 2024-04-26T23:57:00.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.936+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee4c06f9a0 con 0x7fee4c163bb0 2024-04-26T23:57:00.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.938+0000 7fee42a00640 1 -- 172.21.15.46:0/434864792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) ==== 74+0+13138 (secure 0 0 0) 0x7fee4404b030 con 0x7fee4c163bb0 2024-04-26T23:57:00.940 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:00.940 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":206,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:00.002525+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 msgr2=0x7fee2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 0x7fee2c06ede0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fee3c007fd0 tx=0x7fee3c009040 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 msgr2=0x7fee4c0ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fee44034040 tx=0x7fee44004a40 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 shutdown_connections 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fee2c06c920 0x7fee2c06ede0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee4c163bb0 0x7fee4c0ae7b0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee4c1635d0 0x7fee4c0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 --2- 172.21.15.46:0/434864792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee4c1631f0 0x7fee4c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 >> 172.21.15.46:0/434864792 conn(0x7fee4c0b2cf0 msgr2=0x7fee4c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 shutdown_connections 2024-04-26T23:57:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:00.941+0000 7fee51a00640 1 -- 172.21.15.46:0/434864792 wait complete. 2024-04-26T23:57:00.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:57:00.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:57:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.155+0000 7f86b5a00640 1 Processor -- start 2024-04-26T23:57:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.155+0000 7f86b5a00640 1 -- start start 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.156+0000 7f86b5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b00d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.156+0000 7f86b5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b01630a0 0x7f86b0163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86b5a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b0163e70 con 0x7f86b00d1470 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86b5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f86b00d0aa0 con 0x7f86b01630a0 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86b5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f86b00d0170 con 0x7f86b015f480 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b01630a0 0x7f86b0163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b01630a0 0x7f86b0163550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48256/0 (socket says 172.21.15.46:48256) 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 -- 172.21.15.46:0/3132349593 learned_addr learned my addr 172.21.15.46:0/3132349593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86aea00640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b00d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 -- 172.21.15.46:0/3132349593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 msgr2=0x7f86b00d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b00d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.157+0000 7f86af400640 1 -- 172.21.15.46:0/3132349593 >> v1:172.21.15.142:6789/0 conn(0x7f86b00d1470 legacy=0x7f86b00d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.158+0000 7f86af400640 1 -- 172.21.15.46:0/3132349593 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00d0450 con 0x7f86b01630a0 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86af400640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b01630a0 0x7f86b0163550 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009870 tx=0x7f86a802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e6e29ea4b6f33c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86ad600640 1 -- 172.21.15.46:0/3132349593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8037070 con 0x7f86b01630a0 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86ad600640 1 -- 172.21.15.46:0/3132349593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8032440 con 0x7f86b01630a0 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86ad600640 1 -- 172.21.15.46:0/3132349593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8032810 con 0x7f86b01630a0 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86b5a00640 1 -- 172.21.15.46:0/3132349593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b01630a0 msgr2=0x7f86b0163550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.159+0000 7f86b5a00640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b01630a0 0x7f86b0163550 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009870 tx=0x7f86a802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 -- 172.21.15.46:0/3132349593 shutdown_connections 2024-04-26T23:57:01.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b01630a0 0x7f86b0163550 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 --2- 172.21.15.46:0/3132349593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b00d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 -- 172.21.15.46:0/3132349593 >> 172.21.15.46:0/3132349593 conn(0x7f86b00d5070 msgr2=0x7f86b00d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:01.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 -- 172.21.15.46:0/3132349593 shutdown_connections 2024-04-26T23:57:01.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.160+0000 7f86b5a00640 1 -- 172.21.15.46:0/3132349593 wait complete. 2024-04-26T23:57:01.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.161+0000 7f86b5a00640 1 Processor -- start 2024-04-26T23:57:01.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 -- start start 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b01630a0 0x7f86b015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86b015f860 con 0x7f86b00d1470 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00cfe60 con 0x7f86b01630a0 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.162+0000 7f86b5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f86b015f9e0 con 0x7f86b015f480 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48266/0 (socket says 172.21.15.46:48266) 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 -- 172.21.15.46:0/4155457218 learned_addr learned my addr 172.21.15.46:0/4155457218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 -- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 msgr2=0x7f86b0154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 -- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b01630a0 msgr2=0x7f86b015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b01630a0 0x7f86b015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.163+0000 7f86aea00640 1 -- 172.21.15.46:0/4155457218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00b8070 con 0x7f86b00d1470 2024-04-26T23:57:01.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86aea00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d800 tx=0x7f86a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a000b6c0 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a0019070 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a000ba80 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b00b8350 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.164+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b00b8910 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.165+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b008d8a0 con 0x7f86b00d1470 2024-04-26T23:57:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.166+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86a000bc20 con 0x7f86b00d1470 2024-04-26T23:57:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.167+0000 7f86a6a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 0x7f868c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.167+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 5249+0+0 (secure 0 0 0) 0x7f86a008be90 con 0x7f86b00d1470 2024-04-26T23:57:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.168+0000 7f86ae000640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 0x7f868c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.169+0000 7f86ae000640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 0x7f868c06edb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f86b00cee30 tx=0x7f8698007d10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.172+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86b008d8a0 con 0x7f86b00d1470 2024-04-26T23:57:01.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.373+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86b0069530 con 0x7f86b00d1470 2024-04-26T23:57:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.375+0000 7f86a6a00640 1 -- 172.21.15.46:0/4155457218 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) ==== 93+0+10 (secure 0 0 0) 0x7f86a008d0a0 con 0x7f86b00d1470 2024-04-26T23:57:01.376 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 msgr2=0x7f868c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 0x7f868c06edb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f86b00cee30 tx=0x7f8698007d10 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 msgr2=0x7f86b01f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d800 tx=0x7f86a000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 shutdown_connections 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f868c06c8f0 0x7f868c06edb0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b01630a0 0x7f86b015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86b015f480 0x7f86b0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 --2- 172.21.15.46:0/4155457218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b00d1470 0x7f86b01f94e0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 >> 172.21.15.46:0/4155457218 conn(0x7f86b00d5070 msgr2=0x7f86b0160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 shutdown_connections 2024-04-26T23:57:01.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.377+0000 7f86b5a00640 1 -- 172.21.15.46:0/4155457218 wait complete. 2024-04-26T23:57:01.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:57:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.607+0000 7fadcd000640 1 Processor -- start 2024-04-26T23:57:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.608+0000 7fadcd000640 1 -- start start 2024-04-26T23:57:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.609+0000 7fadcd000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fadc815f2d0 0x7fadc80d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.609+0000 7fadcd000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc80d5970 con 0x7fadc80d3ba0 2024-04-26T23:57:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.609+0000 7fadcd000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc80d07d0 con 0x7fadc80d1d40 2024-04-26T23:57:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.609+0000 7fadcd000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fadc80d0ab0 con 0x7fadc815f2d0 2024-04-26T23:57:01.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fadc815f2d0 0x7fadc80d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fadc80d3ba0 0x7fadc80b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42162/0 (socket says 172.21.15.46:42162) 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7e00640 1 -- 172.21.15.46:0/3245570613 learned_addr learned my addr 172.21.15.46:0/3245570613 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7400640 1 -- 172.21.15.46:0/3245570613 >> v1:172.21.15.142:6789/0 conn(0x7fadc80d1d40 legacy=0x7fadc80b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc6000640 1 -- 172.21.15.46:0/3245570613 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203558663 0 0) 0x7fadc80d5970 con 0x7fadc80d3ba0 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7400640 1 -- 172.21.15.46:0/3245570613 >> v1:172.21.15.46:6789/0 conn(0x7fadc80d3ba0 legacy=0x7fadc80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.610+0000 7fadc7400640 1 -- 172.21.15.46:0/3245570613 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80d0d90 con 0x7fadc815f2d0 2024-04-26T23:57:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.611+0000 7fadc7400640 1 --2- 172.21.15.46:0/3245570613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc815f2d0 0x7fadc80d1800 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fadb80097a0 tx=0x7fadb802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c3903fb35d35f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.611+0000 7fadc6000640 1 -- 172.21.15.46:0/3245570613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8037070 con 0x7fadc815f2d0 2024-04-26T23:57:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.611+0000 7fadc6000640 1 -- 172.21.15.46:0/3245570613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb8032440 con 0x7fadc815f2d0 2024-04-26T23:57:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.611+0000 7fadc6000640 1 -- 172.21.15.46:0/3245570613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8032810 con 0x7fadc815f2d0 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 -- 172.21.15.46:0/3245570613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc815f2d0 msgr2=0x7fadc80d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 --2- 172.21.15.46:0/3245570613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc815f2d0 0x7fadc80d1800 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fadb80097a0 tx=0x7fadb802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 -- 172.21.15.46:0/3245570613 shutdown_connections 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 --2- 172.21.15.46:0/3245570613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc815f2d0 0x7fadc80d1800 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 -- 172.21.15.46:0/3245570613 >> 172.21.15.46:0/3245570613 conn(0x7fadc809e290 msgr2=0x7fadc80d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 -- 172.21.15.46:0/3245570613 shutdown_connections 2024-04-26T23:57:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.612+0000 7fadcd000640 1 -- 172.21.15.46:0/3245570613 wait complete. 2024-04-26T23:57:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.614+0000 7fadcd000640 1 Processor -- start 2024-04-26T23:57:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.614+0000 7fadcd000640 1 -- start start 2024-04-26T23:57:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 0x7fadc80ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc815f2d0 0x7fadc80b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadc80d2340 con 0x7fadc80d3ba0 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadc80d21c0 con 0x7fadc815f2d0 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadcd000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fadc80d24c0 con 0x7fadc80d1d40 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadc6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadc6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48284/0 (socket says 172.21.15.46:48284) 2024-04-26T23:57:01.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.615+0000 7fadc6a00640 1 -- 172.21.15.46:0/3597749430 learned_addr learned my addr 172.21.15.46:0/3597749430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc7400640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 0x7fadc80ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc7e00640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc815f2d0 0x7fadc80b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 -- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 msgr2=0x7fadc80ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 0x7fadc80ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 -- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc815f2d0 msgr2=0x7fadc80b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc815f2d0 0x7fadc80b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 -- 172.21.15.46:0/3597749430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80c9dc0 con 0x7fadc80d3ba0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc7400640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 0x7fadc80ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadc6a00640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b3f0 tx=0x7fadb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.616+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0015020 con 0x7fadc80d3ba0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.617+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb0011830 con 0x7fadc80d3ba0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.617+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0011c20 con 0x7fadc80d3ba0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.617+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc80ca0a0 con 0x7fadc80d3ba0 2024-04-26T23:57:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.617+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc80ca660 con 0x7fadc80d3ba0 2024-04-26T23:57:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.618+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc80693b0 con 0x7fadc80d3ba0 2024-04-26T23:57:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.619+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadb0018020 con 0x7fadc80d3ba0 2024-04-26T23:57:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.620+0000 7fadbf400640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 0x7fada406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.620+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 5249+0+0 (secure 0 0 0) 0x7fadb008b5e0 con 0x7fadc80d3ba0 2024-04-26T23:57:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.620+0000 7fadc7400640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 0x7fada406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.622+0000 7fadc7400640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 0x7fada406ede0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fadb8034000 tx=0x7fadb802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.624+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadb001f030 con 0x7fadc80d3ba0 2024-04-26T23:57:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.829+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fadc80ca830 con 0x7fadc80d3ba0 2024-04-26T23:57:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.830+0000 7fadbf400640 1 -- 172.21.15.46:0/3597749430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v206) ==== 84+0+65 (secure 0 0 0) 0x7fadb00172c0 con 0x7fadc80d3ba0 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:01.832 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-26T23:57:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.832+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 msgr2=0x7fada406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.832+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 0x7fada406ede0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fadb8034000 tx=0x7fadb802e000 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 msgr2=0x7fadc81503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b3f0 tx=0x7fadb000b8b0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 shutdown_connections 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fada406c920 0x7fada406ede0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc815f2d0 0x7fadc80b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadc80d3ba0 0x7fadc81503a0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 --2- 172.21.15.46:0/3597749430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadc80d1d40 0x7fadc80ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 >> 172.21.15.46:0/3597749430 conn(0x7fadc809e290 msgr2=0x7fadc80b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 shutdown_connections 2024-04-26T23:57:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:01.833+0000 7fadcd000640 1 -- 172.21.15.46:0/3597749430 wait complete. 2024-04-26T23:57:01.855 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:57:01.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:57:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.069+0000 7f2cfae00640 1 Processor -- start 2024-04-26T23:57:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.069+0000 7f2cfae00640 1 -- start start 2024-04-26T23:57:02.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cfae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cf40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cfae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf4146da0 con 0x7f2cf40b3400 2024-04-26T23:57:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cfae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf40cfeb0 con 0x7f2cf41469c0 2024-04-26T23:57:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cfae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf40cfc70 con 0x7f2cf415d6d0 2024-04-26T23:57:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cf8e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cf40b3400 0x7f2cf40727d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37276/0 (socket says 172.21.15.46:37276) 2024-04-26T23:57:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.071+0000 7f2cf8e00640 1 -- 172.21.15.46:0/1863785102 learned_addr learned my addr 172.21.15.46:0/1863785102 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf3400640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cf40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200786063 0 0) 0x7f2cf4146da0 con 0x7f2cf40b3400 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cd8003520 con 0x7f2cf40b3400 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3897326864 0 0) 0x7f2cd8003520 con 0x7f2cf40b3400 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 msgr2=0x7f2cf40b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cf40b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 >> v1:172.21.15.142:6789/0 conn(0x7f2cf41469c0 legacy=0x7f2cf40cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf40d0190 con 0x7f2cf40b3400 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2cec0019f0 con 0x7f2cf40b3400 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf3400640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cf40b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 >> v1:172.21.15.46:6789/0 conn(0x7f2cf40b3400 legacy=0x7f2cf40727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd80039e0 0x7f2cd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.072+0000 7f2cf2a00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf2a00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd8004d40 0x7f2cd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40cfc70 con 0x7f2cd8004d40 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd800bb00 con 0x7f2cd80039e0 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd800bd00 con 0x7f2cf415d6d0 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd8004d40 0x7f2cd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf3400640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf3e00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd80039e0 0x7f2cd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 -- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 msgr2=0x7f2cd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415d6d0 0x7f2cd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 -- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd80039e0 msgr2=0x7f2cd8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd80039e0 0x7f2cd8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.073+0000 7f2cf8e00640 1 -- 172.21.15.46:0/1863785102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd8003520 con 0x7f2cd8004d40 2024-04-26T23:57:02.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.075+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd8004d40 0x7f2cd80051c0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00a810 tx=0x7f2cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cae783c122f45b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:02.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.075+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00a460 con 0x7f2cd8004d40 2024-04-26T23:57:02.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.075+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec02f070 con 0x7f2cd8004d40 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.075+0000 7f2cf2a00640 1 -- 172.21.15.46:0/1863785102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec02bc90 con 0x7f2cd8004d40 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 -- 172.21.15.46:0/1863785102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd8004d40 msgr2=0x7f2cd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 --2- 172.21.15.46:0/1863785102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd8004d40 0x7f2cd80051c0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00a810 tx=0x7f2cd8005830 comp rx=0 tx=0).stop 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cf8e00640 1 -- 172.21.15.46:0/1863785102 reap_dead start 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 -- 172.21.15.46:0/1863785102 shutdown_connections 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 -- 172.21.15.46:0/1863785102 >> 172.21.15.46:0/1863785102 conn(0x7f2cf406c770 msgr2=0x7f2cf406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 -- 172.21.15.46:0/1863785102 shutdown_connections 2024-04-26T23:57:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.076+0000 7f2cfae00640 1 -- 172.21.15.46:0/1863785102 wait complete. 2024-04-26T23:57:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.078+0000 7f2cfae00640 1 Processor -- start 2024-04-26T23:57:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.078+0000 7f2cfae00640 1 -- start start 2024-04-26T23:57:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415c870 0x7f2cf40b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cf4140170 0x7f2cf413c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40d0750 con 0x7f2cf413cdc0 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40d0ac0 con 0x7f2cf4140170 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cfae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf40b2ec0 con 0x7f2cf415c870 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cf8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cf8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35396/0 (socket says 172.21.15.46:35396) 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.079+0000 7f2cf8e00640 1 -- 172.21.15.46:0/450489709 learned_addr learned my addr 172.21.15.46:0/450489709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf3e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415c870 0x7f2cf40b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf3400640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cf4140170 0x7f2cf413c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 -- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415c870 msgr2=0x7f2cf40b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415c870 0x7f2cf40b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 -- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cf4140170 msgr2=0x7f2cf413c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cf4140170 0x7f2cf413c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 -- 172.21.15.46:0/450489709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf40b43e0 con 0x7f2cf413cdc0 2024-04-26T23:57:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cf8e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00a930 tx=0x7f2cf40d0080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.080+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00a460 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.081+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec034470 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.081+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec001fb0 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.081+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf40b4680 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.081+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf40b4c40 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.082+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf4069530 con 0x7f2cf413cdc0 2024-04-26T23:57:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.083+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cec034840 con 0x7f2cf413cdc0 2024-04-26T23:57:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.084+0000 7f2cebe00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 0x7f2cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.084+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 5249+0+0 (secure 0 0 0) 0x7f2cec0ab5b0 con 0x7f2cf413cdc0 2024-04-26T23:57:02.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.085+0000 7f2cf3e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 0x7f2cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:02.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.086+0000 7f2cf3e00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 0x7f2cd406edb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007fd0 tx=0x7f2ce4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.089+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cec079b00 con 0x7f2cf413cdc0 2024-04-26T23:57:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:02.298+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2cf40739f0 con 0x7f2cf413cdc0 2024-04-26T23:57:03.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.014+0000 7f2cebe00640 1 -- 172.21.15.46:0/450489709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v207) ==== 138+0+0 (secure 0 0 0) 0x7f2cec079400 con 0x7f2cf413cdc0 2024-04-26T23:57:03.016 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-26T23:57:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 msgr2=0x7f2cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 0x7f2cd406edb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007fd0 tx=0x7f2ce4009040 comp rx=0 tx=0).stop 2024-04-26T23:57:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 msgr2=0x7f2cf41f87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00a930 tx=0x7f2cf40d0080 comp rx=0 tx=0).stop 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 shutdown_connections 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2cd406c8f0 0x7f2cd406edb0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf413cdc0 0x7f2cf41f87d0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cf4140170 0x7f2cf413c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 --2- 172.21.15.46:0/450489709 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cf415c870 0x7f2cf40b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 >> 172.21.15.46:0/450489709 conn(0x7f2cf406c770 msgr2=0x7f2cf40726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 shutdown_connections 2024-04-26T23:57:03.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.020+0000 7f2cfae00640 1 -- 172.21.15.46:0/450489709 wait complete. 2024-04-26T23:57:03.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:57:03.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.260+0000 7f6680a00640 1 Processor -- start 2024-04-26T23:57:03.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.261+0000 7f6680a00640 1 -- start start 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.262+0000 7f6680a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f667c1626c0 0x7f667c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.262+0000 7f6680a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f667c0d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.262+0000 7f6680a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c0d07d0 con 0x7f667c162aa0 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.262+0000 7f6680a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f667c0d0ab0 con 0x7f667c1626c0 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.262+0000 7f6680a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0d0d90 con 0x7f667c0b5fb0 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f667c162aa0 0x7f667c0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37282/0 (socket says 172.21.15.46:37282) 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667be00640 1 -- 172.21.15.46:0/3478084308 learned_addr learned my addr 172.21.15.46:0/3478084308 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667b400640 1 --2- 172.21.15.46:0/3478084308 >> v2:172.21.15.142:3300/0 conn(0x7f667c1626c0 0x7f667c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667aa00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f667c0d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352678871 0 0) 0x7f667c0d07d0 con 0x7f667c162aa0 2024-04-26T23:57:03.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6660003520 con 0x7f667c162aa0 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.263+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4126574809 0 0) 0x7f6660003520 con 0x7f667c162aa0 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 msgr2=0x7f667c0d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f667c0d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 >> v2:172.21.15.142:3300/0 conn(0x7f667c1626c0 msgr2=0x7f667c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 --2- 172.21.15.46:0/3478084308 >> v2:172.21.15.142:3300/0 conn(0x7f667c1626c0 0x7f667c0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c0d2c80 con 0x7f667c162aa0 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f66740019f0 con 0x7f667c162aa0 2024-04-26T23:57:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 >> v1:172.21.15.46:6789/0 conn(0x7f667c162aa0 legacy=0x7f667c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66600039b0 0x7f6660003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f6660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660004d40 0x7f66600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d0ab0 con 0x7f6660004d40 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d0d90 con 0x7f66600039b0 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f666000b9c0 con 0x7f667c0b5fb0 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.264+0000 7f667be00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660004d40 0x7f66600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667b400640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66600039b0 0x7f6660003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667aa00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f6660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667be00640 1 -- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 msgr2=0x7f6660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667be00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c0b5fb0 0x7f6660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667be00640 1 -- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66600039b0 msgr2=0x7f6660003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667be00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66600039b0 0x7f6660003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.265+0000 7f667be00640 1 -- 172.21.15.46:0/3478084308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660003520 con 0x7f6660004d40 2024-04-26T23:57:03.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.266+0000 7f667be00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660004d40 0x7f66600051c0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f667400a8a0 tx=0x7f6660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b5f5c1ad5ba907 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:03.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.266+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667400a4c0 con 0x7f6660004d40 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.266+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667402f070 con 0x7f6660004d40 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.266+0000 7f667a000640 1 -- 172.21.15.46:0/3478084308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667402bcf0 con 0x7f6660004d40 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 -- 172.21.15.46:0/3478084308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660004d40 msgr2=0x7f66600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 --2- 172.21.15.46:0/3478084308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660004d40 0x7f66600051c0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f667400a8a0 tx=0x7f6660005830 comp rx=0 tx=0).stop 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f667be00640 1 -- 172.21.15.46:0/3478084308 reap_dead start 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 -- 172.21.15.46:0/3478084308 shutdown_connections 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 -- 172.21.15.46:0/3478084308 >> 172.21.15.46:0/3478084308 conn(0x7f667c09e2a0 msgr2=0x7f667c0d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 -- 172.21.15.46:0/3478084308 shutdown_connections 2024-04-26T23:57:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.267+0000 7f6680a00640 1 -- 172.21.15.46:0/3478084308 wait complete. 2024-04-26T23:57:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.269+0000 7f6680a00640 1 Processor -- start 2024-04-26T23:57:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.269+0000 7f6680a00640 1 -- start start 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 0x7f667c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 0x7f667c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d5850 con 0x7f667c1f9360 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d2220 con 0x7f667c0ca1e0 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f6680a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0d59d0 con 0x7f667c1f8f80 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.270+0000 7f667be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35416/0 (socket says 172.21.15.46:35416) 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 -- 172.21.15.46:0/2363446030 learned_addr learned my addr 172.21.15.46:0/2363446030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667b400640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 0x7f667c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667aa00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 0x7f667c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 -- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 msgr2=0x7f667c0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 0x7f667c0c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 -- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 msgr2=0x7f667c1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 0x7f667c1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667be00640 1 -- 172.21.15.46:0/2363446030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c0ca680 con 0x7f667c1f9360 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667b400640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 0x7f667c0c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.271+0000 7f667aa00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 0x7f667c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f667be00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f667402b580 tx=0x7f6674001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667400a4c0 con 0x7f667c1f9360 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66740019e0 con 0x7f667c1f9360 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674002010 con 0x7f667c1f9360 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c0ca960 con 0x7f667c1f9360 2024-04-26T23:57:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.272+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c0caf20 con 0x7f667c1f9360 2024-04-26T23:57:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.273+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c0693b0 con 0x7f667c1f9360 2024-04-26T23:57:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.274+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66740346b0 con 0x7f667c1f9360 2024-04-26T23:57:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.275+0000 7f6673400640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 0x7f665c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.275+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 5619+0+0 (secure 0 0 0) 0x7f66740ab770 con 0x7f667c1f9360 2024-04-26T23:57:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.276+0000 7f667b400640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 0x7f665c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.277+0000 7f667b400640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 0x7f665c06edb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f667c0cc580 tx=0x7f666c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.280+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6674047030 con 0x7f667c1f9360 2024-04-26T23:57:03.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:03.488+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f667c0d25b0 con 0x7f667c1f9360 2024-04-26T23:57:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.021+0000 7f6673400640 1 -- 172.21.15.46:0/2363446030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v208) ==== 170+0+0 (secure 0 0 0) 0x7f6674079b50 con 0x7f667c1f9360 2024-04-26T23:57:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-26T23:57:04.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.024+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 msgr2=0x7f665c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.024+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 0x7f665c06edb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f667c0cc580 tx=0x7f666c008040 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.024+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 msgr2=0x7f667c158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.024+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f667402b580 tx=0x7f6674001e30 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 shutdown_connections 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f665c06c8f0 0x7f665c06edb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f667c1f9360 0x7f667c158730 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f667c0ca1e0 0x7f667c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 --2- 172.21.15.46:0/2363446030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f667c1f8f80 0x7f667c0c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 >> 172.21.15.46:0/2363446030 conn(0x7f667c09e2a0 msgr2=0x7f667c161d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 shutdown_connections 2024-04-26T23:57:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.025+0000 7f6680a00640 1 -- 172.21.15.46:0/2363446030 wait complete. 2024-04-26T23:57:04.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:57:04.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.243+0000 7fd29a000640 1 Processor -- start 2024-04-26T23:57:04.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.244+0000 7fd29a000640 1 -- start start 2024-04-26T23:57:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.245+0000 7fd29a000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd2940b64c0 0x7fd2940d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.245+0000 7fd29a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2940b46f0 con 0x7fd294162aa0 2024-04-26T23:57:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.245+0000 7fd29a000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2940b49d0 con 0x7fd2940d3d60 2024-04-26T23:57:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.245+0000 7fd29a000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd2940b44a0 con 0x7fd2940b64c0 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.246+0000 7fd293400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd294162aa0 0x7fd2940d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37296/0 (socket says 172.21.15.46:37296) 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.246+0000 7fd293400640 1 -- 172.21.15.46:0/1831372755 learned_addr learned my addr 172.21.15.46:0/1831372755 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.246+0000 7fd293e00640 1 --2- 172.21.15.46:0/1831372755 >> v2:172.21.15.142:3300/0 conn(0x7fd2940b64c0 0x7fd2940d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.246+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2040945936 0 0) 0x7fd2940b46f0 con 0x7fd294162aa0 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.247+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd278003520 con 0x7fd294162aa0 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.247+0000 7fd293e00640 1 -- 172.21.15.46:0/1831372755 >> v1:172.21.15.142:6789/0 conn(0x7fd2940d3d60 legacy=0x7fd2940cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.247+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3508347483 0 0) 0x7fd278003520 con 0x7fd294162aa0 2024-04-26T23:57:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.247+0000 7fd293e00640 1 -- 172.21.15.46:0/1831372755 >> v1:172.21.15.46:6789/0 conn(0x7fd294162aa0 legacy=0x7fd2940d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.247+0000 7fd293e00640 1 -- 172.21.15.46:0/1831372755 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2940b4cb0 con 0x7fd2940b64c0 2024-04-26T23:57:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.249+0000 7fd293e00640 1 --2- 172.21.15.46:0/1831372755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940d0390 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03e25b35f44d46c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.249+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284001670 con 0x7fd2940b64c0 2024-04-26T23:57:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.249+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c007540 con 0x7fd2940b64c0 2024-04-26T23:57:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.249+0000 7fd292000640 1 -- 172.21.15.46:0/1831372755 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c007910 con 0x7fd2940b64c0 2024-04-26T23:57:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 -- 172.21.15.46:0/1831372755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 msgr2=0x7fd2940d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 --2- 172.21.15.46:0/1831372755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940d0390 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c02a5f0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 -- 172.21.15.46:0/1831372755 shutdown_connections 2024-04-26T23:57:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 --2- 172.21.15.46:0/1831372755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940d0390 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 -- 172.21.15.46:0/1831372755 >> 172.21.15.46:0/1831372755 conn(0x7fd2940d5070 msgr2=0x7fd2940d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 -- 172.21.15.46:0/1831372755 shutdown_connections 2024-04-26T23:57:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.250+0000 7fd29a000640 1 -- 172.21.15.46:0/1831372755 wait complete. 2024-04-26T23:57:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.252+0000 7fd29a000640 1 Processor -- start 2024-04-26T23:57:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.252+0000 7fd29a000640 1 -- start start 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2940d3d60 0x7fd294154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2940d30f0 con 0x7fd294162aa0 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2940b5020 con 0x7fd2940b64c0 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd29a000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2940d5b90 con 0x7fd2940d3d60 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd293e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd293e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35426/0 (socket says 172.21.15.46:35426) 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.253+0000 7fd293e00640 1 -- 172.21.15.46:0/96480915 learned_addr learned my addr 172.21.15.46:0/96480915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293400640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd292a00640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2940d3d60 0x7fd294154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 -- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2940d3d60 msgr2=0x7fd294154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2940d3d60 0x7fd294154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 -- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 msgr2=0x7fd2940ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 -- 172.21.15.46:0/96480915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2940b7bf0 con 0x7fd294162aa0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.254+0000 7fd293e00640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.255+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c040070 con 0x7fd294162aa0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.255+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c031bc0 con 0x7fd294162aa0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.255+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c009450 con 0x7fd294162aa0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.255+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2940cd100 con 0x7fd294162aa0 2024-04-26T23:57:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.255+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2940b7eb0 con 0x7fd294162aa0 2024-04-26T23:57:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.256+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd294069530 con 0x7fd294162aa0 2024-04-26T23:57:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.258+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd28c039040 con 0x7fd294162aa0 2024-04-26T23:57:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.259+0000 7fd28b400640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 0x7fd27006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.259+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(208..208 src has 1..208) ==== 6001+0+0 (secure 0 0 0) 0x7fd28c0ad310 con 0x7fd294162aa0 2024-04-26T23:57:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.259+0000 7fd293400640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 0x7fd27006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.260+0000 7fd293400640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 0x7fd27006edb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd284005880 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:04.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.264+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd28c04c350 con 0x7fd294162aa0 2024-04-26T23:57:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:04.512+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd29406d290 con 0x7fd294162aa0 2024-04-26T23:57:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.045+0000 7fd28b400640 1 -- 172.21.15.46:0/96480915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v163) ==== 174+0+0 (secure 0 0 0) 0x7fd28c07b4f0 con 0x7fd294162aa0 2024-04-26T23:57:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 31 and data pool 32 2024-04-26T23:57:05.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.049+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 msgr2=0x7fd27006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.049+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 0x7fd27006edb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd284005880 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-04-26T23:57:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.049+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 msgr2=0x7fd2940c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.049+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c004a40 comp rx=0 tx=0).stop 2024-04-26T23:57:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 shutdown_connections 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fd27006c8f0 0x7fd27006edb0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294162aa0 0x7fd2940c9580 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2940d3d60 0x7fd294154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 --2- 172.21.15.46:0/96480915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940b64c0 0x7fd2940ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 >> 172.21.15.46:0/96480915 conn(0x7fd2940d5070 msgr2=0x7fd2941614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 shutdown_connections 2024-04-26T23:57:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.050+0000 7fd29a000640 1 -- 172.21.15.46:0/96480915 wait complete. 2024-04-26T23:57:05.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:57:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.276+0000 7f9108600640 1 Processor -- start 2024-04-26T23:57:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.277+0000 7f9108600640 1 -- start start 2024-04-26T23:57:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9108600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f910006ca60 0x7f910016a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9108600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91001632a0 con 0x7f910015f150 2024-04-26T23:57:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9108600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9100163580 con 0x7f9100162d50 2024-04-26T23:57:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9108600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f91000cf970 con 0x7f910006ca60 2024-04-26T23:57:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9105c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f910015f150 0x7f9100071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37306/0 (socket says 172.21.15.46:37306) 2024-04-26T23:57:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.278+0000 7f9105c00640 1 -- 172.21.15.46:0/3600963419 learned_addr learned my addr 172.21.15.46:0/3600963419 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f9106600640 1 --2- 172.21.15.46:0/3600963419 >> v2:172.21.15.142:3300/0 conn(0x7f910006ca60 0x7f910016a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426916114 0 0) 0x7f91001632a0 con 0x7f910015f150 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e4003520 con 0x7f910015f150 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3432989415 0 0) 0x7f90e4003520 con 0x7f910015f150 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 >> v2:172.21.15.142:3300/0 conn(0x7f910006ca60 msgr2=0x7f910016a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 --2- 172.21.15.46:0/3600963419 >> v2:172.21.15.142:3300/0 conn(0x7f910006ca60 0x7f910016a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 >> v1:172.21.15.142:6789/0 conn(0x7f9100162d50 legacy=0x7f910006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9100163860 con 0x7f910015f150 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f90f00019f0 con 0x7f910015f150 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 >> v1:172.21.15.46:6789/0 conn(0x7f910015f150 legacy=0x7f9100071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 0x7f90e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90e4004800 0x7f90e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e40052e0 0x7f90e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91000cf970 con 0x7f90e4003a00 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90e400c200 con 0x7f90e4004800 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.279+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90e400c400 con 0x7f90e40052e0 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 0x7f90e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105200640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90e4004800 0x7f90e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 -- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e40052e0 msgr2=0x7f90e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e40052e0 0x7f90e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 -- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90e4004800 msgr2=0x7f90e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90e4004800 0x7f90e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9105c00640 1 -- 172.21.15.46:0/3600963419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e4003520 con 0x7f90e4003a00 2024-04-26T23:57:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.280+0000 7f9106600640 1 -- 172.21.15.46:0/3600963419 reap_dead start 2024-04-26T23:57:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.281+0000 7f9105c00640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 0x7f90e4003e20 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f90f000a770 tx=0x7f90e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c38056c72f9ce1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.281+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000a390 con 0x7f90e4003a00 2024-04-26T23:57:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.281+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f002f070 con 0x7f90e4003a00 2024-04-26T23:57:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.281+0000 7f90ffe00640 1 -- 172.21.15.46:0/3600963419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f002bc20 con 0x7f90e4003a00 2024-04-26T23:57:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 -- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 msgr2=0x7f90e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 0x7f90e4003e20 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f90f000a770 tx=0x7f90e4005830 comp rx=0 tx=0).stop 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 -- 172.21.15.46:0/3600963419 shutdown_connections 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 --2- 172.21.15.46:0/3600963419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90e4003a00 0x7f90e4003e20 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 -- 172.21.15.46:0/3600963419 >> 172.21.15.46:0/3600963419 conn(0x7f910006da90 msgr2=0x7f91000d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 -- 172.21.15.46:0/3600963419 shutdown_connections 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.282+0000 7f9108600640 1 -- 172.21.15.46:0/3600963419 wait complete. 2024-04-26T23:57:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.284+0000 7f9108600640 1 Processor -- start 2024-04-26T23:57:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.284+0000 7f9108600640 1 -- start start 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e4003a00 0x7f910014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91001f97d0 0x7f91000b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f910006c350 con 0x7f91000b49a0 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f910015f530 con 0x7f91001f97d0 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9108600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f910006c4d0 con 0x7f90e4003a00 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9106600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9106600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35450/0 (socket says 172.21.15.46:35450) 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.285+0000 7f9106600640 1 -- 172.21.15.46:0/3014212661 learned_addr learned my addr 172.21.15.46:0/3014212661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9105c00640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e4003a00 0x7f910014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9105200640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91001f97d0 0x7f91000b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9106600640 1 -- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e4003a00 msgr2=0x7f910014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9106600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e4003a00 0x7f910014f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9106600640 1 -- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91001f97d0 msgr2=0x7f91000b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9106600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91001f97d0 0x7f91000b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.286+0000 7f9106600640 1 -- 172.21.15.46:0/3014212661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91000c7160 con 0x7f91000b49a0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f9106600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f910006e340 tx=0x7f90f800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004020 con 0x7f91000b49a0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f80041c0 con 0x7f91000b49a0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80043b0 con 0x7f91000b49a0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91000c7440 con 0x7f91000b49a0 2024-04-26T23:57:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.287+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91000c7a00 con 0x7f91000b49a0 2024-04-26T23:57:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.288+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9100069530 con 0x7f91000b49a0 2024-04-26T23:57:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.289+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90f801a020 con 0x7f91000b49a0 2024-04-26T23:57:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.290+0000 7f90fd600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.290+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 6101+0+0 (secure 0 0 0) 0x7f90f808b350 con 0x7f91000b49a0 2024-04-26T23:57:05.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.291+0000 7f9105c00640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.292+0000 7f9105c00640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 0x7f90e006edb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f90f0033000 tx=0x7f90f0007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.295+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90f8059550 con 0x7f91000b49a0 2024-04-26T23:57:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.498+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91000b7d20 con 0x7f91000b49a0 2024-04-26T23:57:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.500+0000 7f90fd600640 1 -- 172.21.15.46:0/3014212661 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) ==== 74+0+16929 (secure 0 0 0) 0x7f90f8058e50 con 0x7f91000b49a0 2024-04-26T23:57:05.502 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:05.502 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":209,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:04.513040+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.502+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 msgr2=0x7f90e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.502+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 0x7f90e006edb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f90f0033000 tx=0x7f90f0007000 comp rx=0 tx=0).stop 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.502+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 msgr2=0x7f91001f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.502+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f910006e340 tx=0x7f90f800d820 comp rx=0 tx=0).stop 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 shutdown_connections 2024-04-26T23:57:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91000b49a0 0x7f91001f8bf0 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91001f97d0 0x7f91000b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 --2- 172.21.15.46:0/3014212661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90e4003a00 0x7f910014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 >> 172.21.15.46:0/3014212661 conn(0x7f910006da90 msgr2=0x7f91000d1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 shutdown_connections 2024-04-26T23:57:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.503+0000 7f9108600640 1 -- 172.21.15.46:0/3014212661 wait complete. 2024-04-26T23:57:05.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:57:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.725+0000 7f47d5600640 1 Processor -- start 2024-04-26T23:57:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.725+0000 7f47d5600640 1 -- start start 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47d5600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f47d00d2cf0 0x7f47d00d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47d5600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00cfe00 con 0x7f47d00d0e60 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47d5600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d015f870 con 0x7f47d00d1920 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47d5600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f47d00d09d0 con 0x7f47d00d2cf0 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47c7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f47d00d0e60 0x7f47d00b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37322/0 (socket says 172.21.15.46:37322) 2024-04-26T23:57:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.727+0000 7f47c7e00640 1 -- 172.21.15.46:0/1287577994 learned_addr learned my addr 172.21.15.46:0/1287577994 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47d4c00640 1 --2- 172.21.15.46:0/1287577994 >> v2:172.21.15.142:3300/0 conn(0x7f47d00d2cf0 0x7f47d00d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148555503 0 0) 0x7f47d00cfe00 con 0x7f47d00d0e60 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b4003520 con 0x7f47d00d0e60 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1291859061 0 0) 0x7f47b4003520 con 0x7f47d00d0e60 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 >> v2:172.21.15.142:3300/0 conn(0x7f47d00d2cf0 msgr2=0x7f47d00d5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 --2- 172.21.15.46:0/1287577994 >> v2:172.21.15.142:3300/0 conn(0x7f47d00d2cf0 0x7f47d00d5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 >> v1:172.21.15.142:6789/0 conn(0x7f47d00d1920 legacy=0x7f47d00d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00d00a0 con 0x7f47d00d0e60 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f47b8001670 con 0x7f47d00d0e60 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.728+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 >> v1:172.21.15.46:6789/0 conn(0x7f47d00d0e60 legacy=0x7f47d00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003a30 0x7f47b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d09d0 con 0x7f47b40051c0 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47b400c0c0 con 0x7f47b4004800 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47b400c2c0 con 0x7f47b4003a30 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47c7e00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 -- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003a30 msgr2=0x7f47b4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cfe00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003a30 0x7f47b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003a30 0x7f47b4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 -- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 msgr2=0x7f47b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47d4c00640 1 -- 172.21.15.46:0/1287577994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b4003520 con 0x7f47b40051c0 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47c7e00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.729+0000 7f47cfe00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b4003a30 0x7f47b4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:05.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.730+0000 7f47d4c00640 1 -- 172.21.15.46:0/1287577994 reap_dead start 2024-04-26T23:57:05.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.730+0000 7f47d4c00640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fc0 tx=0x7f47b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bac9cc3b41eaf3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8035020 con 0x7f47b40051c0 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c8003b90 con 0x7f47b40051c0 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47cf400640 1 -- 172.21.15.46:0/1287577994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8009c10 con 0x7f47b40051c0 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 -- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 msgr2=0x7f47b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fc0 tx=0x7f47b4005830 comp rx=0 tx=0).stop 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 -- 172.21.15.46:0/1287577994 shutdown_connections 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 --2- 172.21.15.46:0/1287577994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 -- 172.21.15.46:0/1287577994 >> 172.21.15.46:0/1287577994 conn(0x7f47d00d4f10 msgr2=0x7f47d00d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.731+0000 7f47d5600640 1 -- 172.21.15.46:0/1287577994 shutdown_connections 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.732+0000 7f47d5600640 1 -- 172.21.15.46:0/1287577994 wait complete. 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.733+0000 7f47d5600640 1 Processor -- start 2024-04-26T23:57:05.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.733+0000 7f47d5600640 1 -- start start 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00b2920 0x7f47d00b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d5a30 con 0x7f47d00c8a80 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d0660 con 0x7f47b40051c0 2024-04-26T23:57:05.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.734+0000 7f47d5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47d00d5bb0 con 0x7f47d00b2920 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35468/0 (socket says 172.21.15.46:35468) 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 -- 172.21.15.46:0/3196336623 learned_addr learned my addr 172.21.15.46:0/3196336623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47cfe00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 -- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00b2920 msgr2=0x7f47d00b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47d4c00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00b2920 0x7f47d00b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00b2920 0x7f47d00b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 -- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 msgr2=0x7f47d015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 -- 172.21.15.46:0/3196336623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00c94d0 con 0x7f47d00c8a80 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.735+0000 7f47c7e00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d770 tx=0x7f47b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.736+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8019070 con 0x7f47d00c8a80 2024-04-26T23:57:05.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.736+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b80074e0 con 0x7f47d00c8a80 2024-04-26T23:57:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.736+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004a00 con 0x7f47d00c8a80 2024-04-26T23:57:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.736+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d00c97b0 con 0x7f47d00c8a80 2024-04-26T23:57:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.736+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d00c9d70 con 0x7f47d00c8a80 2024-04-26T23:57:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.737+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d008d980 con 0x7f47d00c8a80 2024-04-26T23:57:05.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.738+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b8004ba0 con 0x7f47d00c8a80 2024-04-26T23:57:05.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.739+0000 7f47ccc00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:05.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.739+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 6101+0+0 (secure 0 0 0) 0x7f47b808b700 con 0x7f47d00c8a80 2024-04-26T23:57:05.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.740+0000 7f47cfe00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.741+0000 7f47cfe00640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001750 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.744+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47d008d980 con 0x7f47d00c8a80 2024-04-26T23:57:05.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.944+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47d00693b0 con 0x7f47d00c8a80 2024-04-26T23:57:05.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.946+0000 7f47ccc00640 1 -- 172.21.15.46:0/3196336623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) ==== 74+0+16929 (secure 0 0 0) 0x7f47b8089e20 con 0x7f47d00c8a80 2024-04-26T23:57:05.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":209,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:04.513040+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:57:05.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 msgr2=0x7f47ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001750 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2024-04-26T23:57:05.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 msgr2=0x7f47d00c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:05.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d770 tx=0x7f47b800dc30 comp rx=0 tx=0).stop 2024-04-26T23:57:05.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 shutdown_connections 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47d00b2920 0x7f47d00b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47d00c8a80 0x7f47d00c8ee0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 --2- 172.21.15.46:0/3196336623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b40051c0 0x7f47d015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.948+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 >> 172.21.15.46:0/3196336623 conn(0x7f47d00d4f10 msgr2=0x7f47d006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.949+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 shutdown_connections 2024-04-26T23:57:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:05.949+0000 7f47d5600640 1 -- 172.21.15.46:0/3196336623 wait complete. 2024-04-26T23:57:05.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:57:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.177+0000 7fe195200640 1 Processor -- start 2024-04-26T23:57:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.178+0000 7fe195200640 1 -- start start 2024-04-26T23:57:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.179+0000 7fe195200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.179+0000 7fe195200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1900b5fb0 0x7fe1900d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.179+0000 7fe195200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1900d07d0 con 0x7fe190162aa0 2024-04-26T23:57:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.179+0000 7fe195200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe1900d0ab0 con 0x7fe1900b5fb0 2024-04-26T23:57:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.179+0000 7fe195200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1900d0d90 con 0x7fe1901626c0 2024-04-26T23:57:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.180+0000 7fe18f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.180+0000 7fe18f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:42254/0 (socket says 172.21.15.46:42254) 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.180+0000 7fe18f400640 1 -- 172.21.15.46:0/3421445383 learned_addr learned my addr 172.21.15.46:0/3421445383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.180+0000 7fe18ea00640 1 --2- 172.21.15.46:0/3421445383 >> v2:172.21.15.142:3300/0 conn(0x7fe1900b5fb0 0x7fe1900d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.181+0000 7fe18f400640 1 -- 172.21.15.46:0/3421445383 >> v2:172.21.15.142:3300/0 conn(0x7fe1900b5fb0 msgr2=0x7fe1900d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.181+0000 7fe18f400640 1 --2- 172.21.15.46:0/3421445383 >> v2:172.21.15.142:3300/0 conn(0x7fe1900b5fb0 0x7fe1900d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.181+0000 7fe18f400640 1 -- 172.21.15.46:0/3421445383 >> v1:172.21.15.142:6789/0 conn(0x7fe190162aa0 legacy=0x7fe1900d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.181+0000 7fe18f400640 1 -- 172.21.15.46:0/3421445383 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1900d2c80 con 0x7fe1901626c0 2024-04-26T23:57:06.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.182+0000 7fe18f400640 1 --2- 172.21.15.46:0/3421445383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe180009870 tx=0x7fe18002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33119bc245cc4568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:06.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.183+0000 7fe18e000640 1 -- 172.21.15.46:0/3421445383 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180037070 con 0x7fe1901626c0 2024-04-26T23:57:06.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.183+0000 7fe18e000640 1 -- 172.21.15.46:0/3421445383 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe180032440 con 0x7fe1901626c0 2024-04-26T23:57:06.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.183+0000 7fe18e000640 1 -- 172.21.15.46:0/3421445383 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180032810 con 0x7fe1901626c0 2024-04-26T23:57:06.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 -- 172.21.15.46:0/3421445383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 msgr2=0x7fe1900b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 --2- 172.21.15.46:0/3421445383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe180009870 tx=0x7fe18002a270 comp rx=0 tx=0).stop 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 -- 172.21.15.46:0/3421445383 shutdown_connections 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 --2- 172.21.15.46:0/3421445383 >> v2:172.21.15.142:3300/0 conn(0x7fe1900b5fb0 0x7fe1900d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 --2- 172.21.15.46:0/3421445383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1900b47c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 -- 172.21.15.46:0/3421445383 >> 172.21.15.46:0/3421445383 conn(0x7fe19009e2a0 msgr2=0x7fe1900d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 -- 172.21.15.46:0/3421445383 shutdown_connections 2024-04-26T23:57:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.184+0000 7fe195200640 1 -- 172.21.15.46:0/3421445383 wait complete. 2024-04-26T23:57:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.186+0000 7fe195200640 1 Processor -- start 2024-04-26T23:57:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.186+0000 7fe195200640 1 -- start start 2024-04-26T23:57:06.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900b5fb0 0x7fe1900cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1901596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900d2190 con 0x7fe190162aa0 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900d2010 con 0x7fe1900b5fb0 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe195200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1900d31a0 con 0x7fe1901626c0 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe18fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe18fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35476/0 (socket says 172.21.15.46:35476) 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe18fe00640 1 -- 172.21.15.46:0/3565561650 learned_addr learned my addr 172.21.15.46:0/3565561650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe18ea00640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1901596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.188+0000 7fe18f400640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900b5fb0 0x7fe1900cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 -- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 msgr2=0x7fe1901596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:06.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1901596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 -- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900b5fb0 msgr2=0x7fe1900cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900b5fb0 0x7fe1900cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 -- 172.21.15.46:0/3565561650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19009fe90 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe18fe00640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fe18800c3f0 tx=0x7fe18800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880052d0 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188005470 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1900a0130 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188007260 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.189+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1900a06f0 con 0x7fe190162aa0 2024-04-26T23:57:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.191+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1900693b0 con 0x7fe190162aa0 2024-04-26T23:57:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.192+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe188007400 con 0x7fe190162aa0 2024-04-26T23:57:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.192+0000 7fe187400640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 0x7fe16c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.193+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7fe18808b090 con 0x7fe190162aa0 2024-04-26T23:57:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.193+0000 7fe18f400640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 0x7fe16c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:06.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.194+0000 7fe18f400640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 0x7fe16c06edb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe180034000 tx=0x7fe18002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:06.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.197+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe188059320 con 0x7fe190162aa0 2024-04-26T23:57:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:06.443+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe1900d25b0 con 0x7fe190162aa0 2024-04-26T23:57:07.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.086+0000 7fe187400640 1 -- 172.21.15.46:0/3565561650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v166) ==== 118+0+0 (secure 0 0 0) 0x7fe188058c20 con 0x7fe190162aa0 2024-04-26T23:57:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.089+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 msgr2=0x7fe16c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.089+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 0x7fe16c06edb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe180034000 tx=0x7fe18002e000 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 msgr2=0x7fe1901fc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fe18800c3f0 tx=0x7fe18800c8b0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 shutdown_connections 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fe16c06c8f0 0x7fe16c06edb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe190162aa0 0x7fe1901fc920 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1901626c0 0x7fe1901596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 --2- 172.21.15.46:0/3565561650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900b5fb0 0x7fe1900cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 >> 172.21.15.46:0/3565561650 conn(0x7fe19009e2a0 msgr2=0x7fe1900d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 shutdown_connections 2024-04-26T23:57:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.090+0000 7fe195200640 1 -- 172.21.15.46:0/3565561650 wait complete. 2024-04-26T23:57:07.118 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:57:07.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:57:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.329+0000 7f7fff600640 1 Processor -- start 2024-04-26T23:57:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.329+0000 7f7fff600640 1 -- start start 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.330+0000 7f7fff600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ff806f4b0 0x7f7ff80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.330+0000 7f7fff600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff80b4320 0x7f7ff80ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.330+0000 7f7fff600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff80b4b30 con 0x7f7ff80d04c0 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.330+0000 7f7fff600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff80b4e10 con 0x7f7ff80b4320 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.330+0000 7f7fff600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff80b50f0 con 0x7f7ff806f4b0 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff80b4320 0x7f7ff80ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff80b4320 0x7f7ff80ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35490/0 (socket says 172.21.15.46:35490) 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 -- 172.21.15.46:0/320363294 learned_addr learned my addr 172.21.15.46:0/320363294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/320363294 >> v2:172.21.15.142:3300/0 conn(0x7f7ff806f4b0 0x7f7ff80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 -- 172.21.15.46:0/320363294 >> v2:172.21.15.142:3300/0 conn(0x7f7ff806f4b0 msgr2=0x7f7ff80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 --2- 172.21.15.46:0/320363294 >> v2:172.21.15.142:3300/0 conn(0x7f7ff806f4b0 0x7f7ff80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 -- 172.21.15.46:0/320363294 >> v1:172.21.15.142:6789/0 conn(0x7f7ff80d04c0 legacy=0x7f7ff806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.331+0000 7f7ffd600640 1 -- 172.21.15.46:0/320363294 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff80b53d0 con 0x7f7ff80b4320 2024-04-26T23:57:07.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.333+0000 7f7ffd600640 1 --2- 172.21.15.46:0/320363294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff80ce470 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b62a0 tx=0x7f7ff002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d86f272973d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.333+0000 7f7ff7400640 1 -- 172.21.15.46:0/320363294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0037070 con 0x7f7ff80b4320 2024-04-26T23:57:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.333+0000 7f7ff7400640 1 -- 172.21.15.46:0/320363294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff0032440 con 0x7f7ff80b4320 2024-04-26T23:57:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.333+0000 7f7ff7400640 1 -- 172.21.15.46:0/320363294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0032810 con 0x7f7ff80b4320 2024-04-26T23:57:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 -- 172.21.15.46:0/320363294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 msgr2=0x7f7ff80ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 --2- 172.21.15.46:0/320363294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff80ce470 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b62a0 tx=0x7f7ff002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 -- 172.21.15.46:0/320363294 shutdown_connections 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 --2- 172.21.15.46:0/320363294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff80ce470 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 --2- 172.21.15.46:0/320363294 >> v2:172.21.15.142:3300/0 conn(0x7f7ff806f4b0 0x7f7ff80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 -- 172.21.15.46:0/320363294 >> 172.21.15.46:0/320363294 conn(0x7f7ff806c980 msgr2=0x7f7ff80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 -- 172.21.15.46:0/320363294 shutdown_connections 2024-04-26T23:57:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.334+0000 7f7fff600640 1 -- 172.21.15.46:0/320363294 wait complete. 2024-04-26T23:57:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.336+0000 7f7fff600640 1 Processor -- start 2024-04-26T23:57:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.336+0000 7f7fff600640 1 -- start start 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 0x7f7ff813fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff80d04c0 0x7f7ff813d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80b4910 con 0x7f7ff80b4320 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80b4790 con 0x7f7ff806f4b0 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7fff600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff80ce9b0 con 0x7f7ff80d04c0 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35506/0 (socket says 172.21.15.46:35506) 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.337+0000 7f7ff7e00640 1 -- 172.21.15.46:0/3014761337 learned_addr learned my addr 172.21.15.46:0/3014761337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 -- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff80d04c0 msgr2=0x7f7ff813d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ffd600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff80d04c0 0x7f7ff813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff80d04c0 0x7f7ff813d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ffcc00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 0x7f7ff813fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 -- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 msgr2=0x7f7ff813fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 0x7f7ff813fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 -- 172.21.15.46:0/3014761337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff8140220 con 0x7f7ff80b4320 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ffcc00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 0x7f7ff813fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000d800 tx=0x7f7fe000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000b6c0 con 0x7f7ff80b4320 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0019070 con 0x7f7ff80b4320 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000ba80 con 0x7f7ff80b4320 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.338+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff8140500 con 0x7f7ff80b4320 2024-04-26T23:57:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.339+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff8140ac0 con 0x7f7ff80b4320 2024-04-26T23:57:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.340+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff8069530 con 0x7f7ff80b4320 2024-04-26T23:57:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.341+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fe000bc20 con 0x7f7ff80b4320 2024-04-26T23:57:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.341+0000 7f7ff4c00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 0x7f7fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.342+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f7fe008c1f0 con 0x7f7ff80b4320 2024-04-26T23:57:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.342+0000 7f7ffcc00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 0x7f7fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.343+0000 7f7ffcc00640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 0x7f7fd406edb0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8007e90 tx=0x7f7fe80073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.346+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fe005a480 con 0x7f7ff80b4320 2024-04-26T23:57:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:07.589+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7ff80737f0 con 0x7f7ff80b4320 2024-04-26T23:57:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.089+0000 7f7ff4c00640 1 -- 172.21.15.46:0/3014761337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v167) ==== 122+0+0 (secure 0 0 0) 0x7f7fe0059d80 con 0x7f7ff80b4320 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 msgr2=0x7f7fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 0x7f7fd406edb0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8007e90 tx=0x7f7fe80073a0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 msgr2=0x7f7ff815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000d800 tx=0x7f7fe000dcc0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 shutdown_connections 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7fd406c8f0 0x7f7fd406edb0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff80d04c0 0x7f7ff813d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80b4320 0x7f7ff815c4b0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 --2- 172.21.15.46:0/3014761337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff806f4b0 0x7f7ff813fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 >> 172.21.15.46:0/3014761337 conn(0x7f7ff806c980 msgr2=0x7f7ff806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.094+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 shutdown_connections 2024-04-26T23:57:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.095+0000 7f7fff600640 1 -- 172.21.15.46:0/3014761337 wait complete. 2024-04-26T23:57:08.120 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:57:08.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.324+0000 7f49f2000640 1 Processor -- start 2024-04-26T23:57:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.324+0000 7f49f2000640 1 -- start start 2024-04-26T23:57:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49ec1466b0 0x7f49ec06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49ec160910 0x7f49ec160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ec073ad0 con 0x7f49ec0720e0 2024-04-26T23:57:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f2000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f49ec1612a0 con 0x7f49ec1466b0 2024-04-26T23:57:08.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f49ec146a90 con 0x7f49ec160910 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49f0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49ec1466b0 0x7f49ec06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49ebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f49ec0720e0 0x7f49ec0cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37326/0 (socket says 172.21.15.46:37326) 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.326+0000 7f49ebe00640 1 -- 172.21.15.46:0/2418797008 learned_addr learned my addr 172.21.15.46:0/2418797008 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.327+0000 7f49f1600640 1 --2- 172.21.15.46:0/2418797008 >> v2:172.21.15.142:3300/0 conn(0x7f49ec160910 0x7f49ec160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.327+0000 7f49f0c00640 1 -- 172.21.15.46:0/2418797008 >> v2:172.21.15.142:3300/0 conn(0x7f49ec160910 msgr2=0x7f49ec160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.327+0000 7f49f0c00640 1 --2- 172.21.15.46:0/2418797008 >> v2:172.21.15.142:3300/0 conn(0x7f49ec160910 0x7f49ec160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.327+0000 7f49f0c00640 1 -- 172.21.15.46:0/2418797008 >> v1:172.21.15.46:6789/0 conn(0x7f49ec0720e0 legacy=0x7f49ec0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.327+0000 7f49f0c00640 1 -- 172.21.15.46:0/2418797008 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec073d70 con 0x7f49ec1466b0 2024-04-26T23:57:08.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.328+0000 7f49f0c00640 1 --2- 172.21.15.46:0/2418797008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec06de80 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097a0 tx=0x7f49e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab8264994d56a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.328+0000 7f49eb400640 1 -- 172.21.15.46:0/2418797008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0037070 con 0x7f49ec1466b0 2024-04-26T23:57:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.328+0000 7f49eb400640 1 -- 172.21.15.46:0/2418797008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0032440 con 0x7f49ec1466b0 2024-04-26T23:57:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.328+0000 7f49eb400640 1 -- 172.21.15.46:0/2418797008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0032810 con 0x7f49ec1466b0 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 -- 172.21.15.46:0/2418797008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 msgr2=0x7f49ec06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 --2- 172.21.15.46:0/2418797008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec06de80 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097a0 tx=0x7f49e002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 -- 172.21.15.46:0/2418797008 shutdown_connections 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 --2- 172.21.15.46:0/2418797008 >> v2:172.21.15.142:3300/0 conn(0x7f49ec160910 0x7f49ec160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 --2- 172.21.15.46:0/2418797008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec06de80 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 -- 172.21.15.46:0/2418797008 >> 172.21.15.46:0/2418797008 conn(0x7f49ec0b26d0 msgr2=0x7f49ec0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 -- 172.21.15.46:0/2418797008 shutdown_connections 2024-04-26T23:57:08.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.329+0000 7f49f2000640 1 -- 172.21.15.46:0/2418797008 wait complete. 2024-04-26T23:57:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.331+0000 7f49f2000640 1 Processor -- start 2024-04-26T23:57:08.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.331+0000 7f49f2000640 1 -- start start 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49ec0720e0 0x7f49ec140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49ec160910 0x7f49ec154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec0cb470 con 0x7f49ec1466b0 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec0cb190 con 0x7f49ec160910 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49f2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec06e3c0 con 0x7f49ec0720e0 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35528/0 (socket says 172.21.15.46:35528) 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.332+0000 7f49ebe00640 1 -- 172.21.15.46:0/3189675084 learned_addr learned my addr 172.21.15.46:0/3189675084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49f1600640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49ec160910 0x7f49ec154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49f0c00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49ec0720e0 0x7f49ec140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 -- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49ec0720e0 msgr2=0x7f49ec140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49ec0720e0 0x7f49ec140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 -- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49ec160910 msgr2=0x7f49ec154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49ec160910 0x7f49ec154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 -- 172.21.15.46:0/3189675084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec0ca490 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49ebe00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3f0 tx=0x7f49d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8017070 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8005ce0 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80072f0 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec0aeb30 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.333+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec0af0f0 con 0x7f49ec1466b0 2024-04-26T23:57:08.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.334+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ec0693b0 con 0x7f49ec1466b0 2024-04-26T23:57:08.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.336+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49d801a070 con 0x7f49ec1466b0 2024-04-26T23:57:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.336+0000 7f49e8c00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 0x7f49cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.337+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f49d808aff0 con 0x7f49ec1466b0 2024-04-26T23:57:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.337+0000 7f49f0c00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 0x7f49cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.338+0000 7f49f0c00640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 0x7f49cc06edb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f49e0034000 tx=0x7f49e002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.341+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49d8059280 con 0x7f49ec1466b0 2024-04-26T23:57:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.588+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49ec0c7180 con 0x7f49ec1466b0 2024-04-26T23:57:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.590+0000 7f49e8c00640 1 -- 172.21.15.46:0/3189675084 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+5203 (secure 0 0 0) 0x7f49d8058b80 con 0x7f49ec1466b0 2024-04-26T23:57:08.591 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":167,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:07.590672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12054":{"gid":12054,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3782420006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3782420006},{"type":"v1","addr":"172.21.15.46:6835","nonce":3782420006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12054,"qdb_cluster":[12054]},"id":15}]} 2024-04-26T23:57:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 167 2024-04-26T23:57:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.592+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 msgr2=0x7f49cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.592+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 0x7f49cc06edb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f49e0034000 tx=0x7f49e002e000 comp rx=0 tx=0).stop 2024-04-26T23:57:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 msgr2=0x7f49ec0c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3f0 tx=0x7f49d800c8b0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 shutdown_connections 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f49cc06c8f0 0x7f49cc06edb0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49ec160910 0x7f49ec154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec1466b0 0x7f49ec0c9f50 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 --2- 172.21.15.46:0/3189675084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49ec0720e0 0x7f49ec140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 >> 172.21.15.46:0/3189675084 conn(0x7f49ec0b26d0 msgr2=0x7f49ec0c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 shutdown_connections 2024-04-26T23:57:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.593+0000 7f49f2000640 1 -- 172.21.15.46:0/3189675084 wait complete. 2024-04-26T23:57:08.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:57:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.810+0000 7f8353a00640 1 Processor -- start 2024-04-26T23:57:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.811+0000 7f8353a00640 1 -- start start 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.812+0000 7f8353a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f834c0d1520 0x7f834c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.812+0000 7f8353a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834c0d1e40 con 0x7f834c0d3af0 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.812+0000 7f8353a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834c0ce7b0 con 0x7f834c15d510 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.812+0000 7f8353a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f834c0d1220 con 0x7f834c0d1520 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f834c0d3af0 0x7f834c0706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f834c0d1520 0x7f834c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351a00640 1 -- 172.21.15.46:0/529171587 learned_addr learned my addr 172.21.15.46:0/529171587 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f834b400640 1 -- 172.21.15.46:0/529171587 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753235959 0 0) 0x7f834c0d1e40 con 0x7f834c0d3af0 2024-04-26T23:57:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f834b400640 1 -- 172.21.15.46:0/529171587 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8330003520 con 0x7f834c0d3af0 2024-04-26T23:57:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351000640 1 -- 172.21.15.46:0/529171587 >> v1:172.21.15.142:6789/0 conn(0x7f834c15d510 legacy=0x7f834c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351000640 1 -- 172.21.15.46:0/529171587 >> v1:172.21.15.46:6789/0 conn(0x7f834c0d3af0 legacy=0x7f834c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.813+0000 7f8351000640 1 -- 172.21.15.46:0/529171587 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c0cea90 con 0x7f834c0d1520 2024-04-26T23:57:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.814+0000 7f8351000640 1 --2- 172.21.15.46:0/529171587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c0d1900 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f833c009d90 tx=0x7f833c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19628a0cc320ea73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f834b400640 1 -- 172.21.15.46:0/529171587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c007740 con 0x7f834c0d1520 2024-04-26T23:57:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f834b400640 1 -- 172.21.15.46:0/529171587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0078e0 con 0x7f834c0d1520 2024-04-26T23:57:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f834b400640 1 -- 172.21.15.46:0/529171587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c007c90 con 0x7f834c0d1520 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f8353a00640 1 -- 172.21.15.46:0/529171587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 msgr2=0x7f834c0d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f8353a00640 1 --2- 172.21.15.46:0/529171587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c0d1900 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f833c009d90 tx=0x7f833c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f8353a00640 1 -- 172.21.15.46:0/529171587 shutdown_connections 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f8353a00640 1 --2- 172.21.15.46:0/529171587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c0d1900 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.815+0000 7f8353a00640 1 -- 172.21.15.46:0/529171587 >> 172.21.15.46:0/529171587 conn(0x7f834c0d2f40 msgr2=0x7f834c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.816+0000 7f8353a00640 1 -- 172.21.15.46:0/529171587 shutdown_connections 2024-04-26T23:57:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.816+0000 7f8353a00640 1 -- 172.21.15.46:0/529171587 wait complete. 2024-04-26T23:57:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.817+0000 7f8353a00640 1 Processor -- start 2024-04-26T23:57:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.818+0000 7f8353a00640 1 -- start start 2024-04-26T23:57:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834c0d3af0 0x7f834c0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f834c15d510 0x7f834c09e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f834c070230 con 0x7f834c0d1520 2024-04-26T23:57:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f834c0ce560 con 0x7f834c0d3af0 2024-04-26T23:57:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8353a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f834c070dd0 con 0x7f834c15d510 2024-04-26T23:57:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35538/0 (socket says 172.21.15.46:35538) 2024-04-26T23:57:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 -- 172.21.15.46:0/2105353061 learned_addr learned my addr 172.21.15.46:0/2105353061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 -- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f834c15d510 msgr2=0x7f834c09e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f834c15d510 0x7f834c09e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 -- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834c0d3af0 msgr2=0x7f834c0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834c0d3af0 0x7f834c0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.819+0000 7f8351000640 1 -- 172.21.15.46:0/2105353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c0cb430 con 0x7f834c0d1520 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8351000640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f833c009d90 tx=0x7f833c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c009450 con 0x7f834c0d1520 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c004240 con 0x7f834c0d1520 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c0045f0 con 0x7f834c0d1520 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834c0cb710 con 0x7f834c0d1520 2024-04-26T23:57:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.820+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834c0cbcd0 con 0x7f834c0d1520 2024-04-26T23:57:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.821+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834c069530 con 0x7f834c0d1520 2024-04-26T23:57:08.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.823+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f833c039040 con 0x7f834c0d1520 2024-04-26T23:57:08.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.823+0000 7f8348c00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 0x7f832c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:08.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.824+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f833c0ad1e0 con 0x7f834c0d1520 2024-04-26T23:57:08.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.824+0000 7f834be00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 0x7f832c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:08.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.825+0000 7f834be00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 0x7f832c06edb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8338000f60 tx=0x7f83380079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:08.828+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f833c04b030 con 0x7f834c0d1520 2024-04-26T23:57:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.033+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f834c0cf470 con 0x7f834c0d1520 2024-04-26T23:57:09.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.036+0000 7f8348c00640 1 -- 172.21.15.46:0/2105353061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) ==== 74+0+16916 (secure 0 0 0) 0x7f833c040030 con 0x7f834c0d1520 2024-04-26T23:57:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:09.038 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":210,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:06.062873+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 msgr2=0x7f832c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 0x7f832c06edb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8338000f60 tx=0x7f83380079b0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 msgr2=0x7f834c15bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f833c009d90 tx=0x7f833c004a40 comp rx=0 tx=0).stop 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 shutdown_connections 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f832c06c8f0 0x7f832c06edb0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f834c15d510 0x7f834c09e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834c0d3af0 0x7f834c0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 --2- 172.21.15.46:0/2105353061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834c0d1520 0x7f834c15bf90 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.038+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 >> 172.21.15.46:0/2105353061 conn(0x7f834c0d2f40 msgr2=0x7f834c0b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.039+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 shutdown_connections 2024-04-26T23:57:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.039+0000 7f8353a00640 1 -- 172.21.15.46:0/2105353061 wait complete. 2024-04-26T23:57:09.059 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:57:09.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:57:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.271+0000 7f2f01400640 1 Processor -- start 2024-04-26T23:57:09.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.271+0000 7f2f01400640 1 -- start start 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f01400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2efc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f01400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2efc0720e0 0x7f2efc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f01400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2efc073ad0 con 0x7f2efc160910 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f01400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2efc146a90 con 0x7f2efc0720e0 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f01400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2efc073d70 con 0x7f2efc1466b0 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f00a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2efc160910 0x7f2efc0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37340/0 (socket says 172.21.15.46:37340) 2024-04-26T23:57:09.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.273+0000 7f2f00a00640 1 -- 172.21.15.46:0/278529419 learned_addr learned my addr 172.21.15.46:0/278529419 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efb400640 1 --2- 172.21.15.46:0/278529419 >> v2:172.21.15.142:3300/0 conn(0x7f2efc0720e0 0x7f2efc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efbe00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2efc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686788396 0 0) 0x7f2efc073ad0 con 0x7f2efc160910 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee0003520 con 0x7f2efc160910 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2823802922 0 0) 0x7f2ee0003520 con 0x7f2efc160910 2024-04-26T23:57:09.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 msgr2=0x7f2efc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2efc06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 >> v2:172.21.15.142:3300/0 conn(0x7f2efc0720e0 msgr2=0x7f2efc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 --2- 172.21.15.46:0/278529419 >> v2:172.21.15.142:3300/0 conn(0x7f2efc0720e0 0x7f2efc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc074050 con 0x7f2efc160910 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.274+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f2ef40019f0 con 0x7f2efc160910 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 >> v1:172.21.15.46:6789/0 conn(0x7f2efc160910 legacy=0x7f2efc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ee00039b0 0x7f2ee0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0004800 0x7f2ee0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2ee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc146a90 con 0x7f2ee00039b0 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc073d70 con 0x7f2ee0004800 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee000b9c0 con 0x7f2efc1466b0 2024-04-26T23:57:09.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ee00039b0 0x7f2ee0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efb400640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0004800 0x7f2ee0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2f00a00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2ee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 -- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 msgr2=0x7f2ee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2ee00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 -- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0004800 msgr2=0x7f2ee0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0004800 0x7f2ee0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2efbe00640 1 -- 172.21.15.46:0/278529419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee0003520 con 0x7f2ee00039b0 2024-04-26T23:57:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.275+0000 7f2f00a00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc1466b0 0x7f2ee00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.276+0000 7f2efbe00640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ee00039b0 0x7f2ee0003dd0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77498baf1df448c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec036070 con 0x7f2ee00039b0 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eec031400 con 0x7f2ee00039b0 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2efaa00640 1 -- 172.21.15.46:0/278529419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec009690 con 0x7f2ee00039b0 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2f01400640 1 -- 172.21.15.46:0/278529419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ee00039b0 msgr2=0x7f2ee0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2f01400640 1 --2- 172.21.15.46:0/278529419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ee00039b0 0x7f2ee0003dd0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2ee0005830 comp rx=0 tx=0).stop 2024-04-26T23:57:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2f00a00640 1 -- 172.21.15.46:0/278529419 reap_dead start 2024-04-26T23:57:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2f01400640 1 -- 172.21.15.46:0/278529419 shutdown_connections 2024-04-26T23:57:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.277+0000 7f2f01400640 1 -- 172.21.15.46:0/278529419 >> 172.21.15.46:0/278529419 conn(0x7f2efc0b26d0 msgr2=0x7f2efc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.278+0000 7f2f01400640 1 -- 172.21.15.46:0/278529419 shutdown_connections 2024-04-26T23:57:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.278+0000 7f2f01400640 1 -- 172.21.15.46:0/278529419 wait complete. 2024-04-26T23:57:09.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.280+0000 7f2f01400640 1 Processor -- start 2024-04-26T23:57:09.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.280+0000 7f2f01400640 1 -- start start 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2efc1451c0 0x7f2efc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc0b0c00 0x7f2efc142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc0cdda0 con 0x7f2efc144de0 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc0cb470 con 0x7f2efc1451c0 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.281+0000 7f2f01400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2efc0cc0a0 con 0x7f2efc0b0c00 2024-04-26T23:57:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35548/0 (socket says 172.21.15.46:35548) 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 -- 172.21.15.46:0/1730476472 learned_addr learned my addr 172.21.15.46:0/1730476472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efb400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2efc1451c0 0x7f2efc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 -- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc0b0c00 msgr2=0x7f2efc142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc0b0c00 0x7f2efc142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 -- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2efc1451c0 msgr2=0x7f2efc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2efc1451c0 0x7f2efc0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.282+0000 7f2efbe00640 1 -- 172.21.15.46:0/1730476472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc0c7ef0 con 0x7f2efc144de0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2efbe00640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec041020 con 0x7f2efc144de0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eec003810 con 0x7f2efc144de0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec036040 con 0x7f2efc144de0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc0c81d0 con 0x7f2efc144de0 2024-04-26T23:57:09.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.283+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc0c8790 con 0x7f2efc144de0 2024-04-26T23:57:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.284+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc0693b0 con 0x7f2efc144de0 2024-04-26T23:57:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.285+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2eec044020 con 0x7f2efc144de0 2024-04-26T23:57:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.286+0000 7f2ef3e00640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 0x7f2edc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.286+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f2eec0abff0 con 0x7f2efc144de0 2024-04-26T23:57:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.287+0000 7f2efb400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 0x7f2edc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.288+0000 7f2efb400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 0x7f2edc06ee30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2efc1455e0 tx=0x7f2ee8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.293+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2eec04b070 con 0x7f2efc144de0 2024-04-26T23:57:09.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.495+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2efc0c7180 con 0x7f2edc06c970 2024-04-26T23:57:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.514+0000 7f2ef3e00640 1 -- 172.21.15.46:0/1730476472 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252441 (secure 0 0 0) 0x7f2efc0c7180 con 0x7f2edc06c970 2024-04-26T23:57:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:09.520 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:57:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 msgr2=0x7f2edc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 0x7f2edc06ee30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2efc1455e0 tx=0x7f2ee8008040 comp rx=0 tx=0).stop 2024-04-26T23:57:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 msgr2=0x7f2efc0c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2eec004730 comp rx=0 tx=0).stop 2024-04-26T23:57:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 shutdown_connections 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2edc06c970 0x7f2edc06ee30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2efc0b0c00 0x7f2efc142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2efc1451c0 0x7f2efc0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 --2- 172.21.15.46:0/1730476472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc144de0 0x7f2efc0c8f90 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.519+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 >> 172.21.15.46:0/1730476472 conn(0x7f2efc0b26d0 msgr2=0x7f2efc06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.520+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 shutdown_connections 2024-04-26T23:57:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.520+0000 7f2f01400640 1 -- 172.21.15.46:0/1730476472 wait complete. 2024-04-26T23:57:09.540 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":807,"stamp":"2024-04-26T23:57:08.340163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453200,"kb_used_data":21908,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754521520,"statfs":{"total":773094113280,"available":772630036480,"internally_reserved":0,"allocated":22433792,"data_stored":11198704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":441503281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.323023"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090511+0000","last_change":"2024-04-26T23:57:05.085398+0000","last_active":"2024-04-26T23:57:05.090511+0000","last_peered":"2024-04-26T23:57:05.090511+0000","last_clean":"2024-04-26T23:57:05.090511+0000","last_became_active":"2024-04-26T23:57:05.084309+0000","last_became_peered":"2024-04-26T23:57:05.084309+0000","last_unstale":"2024-04-26T23:57:05.090511+0000","last_undegraded":"2024-04-26T23:57:05.090511+0000","last_fullsized":"2024-04-26T23:57:05.090511+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:53.503417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093225+0000","last_change":"2024-04-26T23:57:05.085968+0000","last_active":"2024-04-26T23:57:05.093225+0000","last_peered":"2024-04-26T23:57:05.093225+0000","last_clean":"2024-04-26T23:57:05.093225+0000","last_became_active":"2024-04-26T23:57:05.085639+0000","last_became_peered":"2024-04-26T23:57:05.085639+0000","last_unstale":"2024-04-26T23:57:05.093225+0000","last_undegraded":"2024-04-26T23:57:05.093225+0000","last_fullsized":"2024-04-26T23:57:05.093225+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:46.086737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094069+0000","last_change":"2024-04-26T23:57:05.089249+0000","last_active":"2024-04-26T23:57:05.094069+0000","last_peered":"2024-04-26T23:57:05.094069+0000","last_clean":"2024-04-26T23:57:05.094069+0000","last_became_active":"2024-04-26T23:57:05.088947+0000","last_became_peered":"2024-04-26T23:57:05.088947+0000","last_unstale":"2024-04-26T23:57:05.094069+0000","last_undegraded":"2024-04-26T23:57:05.094069+0000","last_fullsized":"2024-04-26T23:57:05.094069+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.086906+0000","last_change":"2024-04-26T23:57:05.079820+0000","last_active":"2024-04-26T23:57:05.086906+0000","last_peered":"2024-04-26T23:57:05.086906+0000","last_clean":"2024-04-26T23:57:05.086906+0000","last_became_active":"2024-04-26T23:57:05.079533+0000","last_became_peered":"2024-04-26T23:57:05.079533+0000","last_unstale":"2024-04-26T23:57:05.086906+0000","last_undegraded":"2024-04-26T23:57:05.086906+0000","last_fullsized":"2024-04-26T23:57:05.086906+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.470427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.089698+0000","last_change":"2024-04-26T23:57:05.086275+0000","last_active":"2024-04-26T23:57:05.089698+0000","last_peered":"2024-04-26T23:57:05.089698+0000","last_clean":"2024-04-26T23:57:05.089698+0000","last_became_active":"2024-04-26T23:57:05.085833+0000","last_became_peered":"2024-04-26T23:57:05.085833+0000","last_unstale":"2024-04-26T23:57:05.089698+0000","last_undegraded":"2024-04-26T23:57:05.089698+0000","last_fullsized":"2024-04-26T23:57:05.089698+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:17.959749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098924+0000","last_change":"2024-04-26T23:57:05.094461+0000","last_active":"2024-04-26T23:57:05.098924+0000","last_peered":"2024-04-26T23:57:05.098924+0000","last_clean":"2024-04-26T23:57:05.098924+0000","last_became_active":"2024-04-26T23:57:05.094179+0000","last_became_peered":"2024-04-26T23:57:05.094179+0000","last_unstale":"2024-04-26T23:57:05.098924+0000","last_undegraded":"2024-04-26T23:57:05.098924+0000","last_fullsized":"2024-04-26T23:57:05.098924+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:24.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080895+0000","last_change":"2024-04-26T23:57:05.076217+0000","last_active":"2024-04-26T23:57:06.080895+0000","last_peered":"2024-04-26T23:57:06.080895+0000","last_clean":"2024-04-26T23:57:06.080895+0000","last_became_active":"2024-04-26T23:57:05.075790+0000","last_became_peered":"2024-04-26T23:57:05.075790+0000","last_unstale":"2024-04-26T23:57:06.080895+0000","last_undegraded":"2024-04-26T23:57:06.080895+0000","last_fullsized":"2024-04-26T23:57:06.080895+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090952+0000","last_change":"2024-04-26T23:57:05.089835+0000","last_active":"2024-04-26T23:57:05.090952+0000","last_peered":"2024-04-26T23:57:05.090952+0000","last_clean":"2024-04-26T23:57:05.090952+0000","last_became_active":"2024-04-26T23:57:05.089489+0000","last_became_peered":"2024-04-26T23:57:05.089489+0000","last_unstale":"2024-04-26T23:57:05.090952+0000","last_undegraded":"2024-04-26T23:57:05.090952+0000","last_fullsized":"2024-04-26T23:57:05.090952+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:37.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099803+0000","last_change":"2024-04-26T23:57:05.098470+0000","last_active":"2024-04-26T23:57:05.099803+0000","last_peered":"2024-04-26T23:57:05.099803+0000","last_clean":"2024-04-26T23:57:05.099803+0000","last_became_active":"2024-04-26T23:57:05.098173+0000","last_became_peered":"2024-04-26T23:57:05.098173+0000","last_unstale":"2024-04-26T23:57:05.099803+0000","last_undegraded":"2024-04-26T23:57:05.099803+0000","last_fullsized":"2024-04-26T23:57:05.099803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:25.524521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094549+0000","last_change":"2024-04-26T23:57:05.092389+0000","last_active":"2024-04-26T23:57:05.094549+0000","last_peered":"2024-04-26T23:57:05.094549+0000","last_clean":"2024-04-26T23:57:05.094549+0000","last_became_active":"2024-04-26T23:57:05.092095+0000","last_became_peered":"2024-04-26T23:57:05.092095+0000","last_unstale":"2024-04-26T23:57:05.094549+0000","last_undegraded":"2024-04-26T23:57:05.094549+0000","last_fullsized":"2024-04-26T23:57:05.094549+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:57.197392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095212+0000","last_change":"2024-04-26T23:57:05.093199+0000","last_active":"2024-04-26T23:57:05.095212+0000","last_peered":"2024-04-26T23:57:05.095212+0000","last_clean":"2024-04-26T23:57:05.095212+0000","last_became_active":"2024-04-26T23:57:05.092916+0000","last_became_peered":"2024-04-26T23:57:05.092916+0000","last_unstale":"2024-04-26T23:57:05.095212+0000","last_undegraded":"2024-04-26T23:57:05.095212+0000","last_fullsized":"2024-04-26T23:57:05.095212+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:10.935614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.078237+0000","last_change":"2024-04-26T23:57:05.072630+0000","last_active":"2024-04-26T23:57:05.078237+0000","last_peered":"2024-04-26T23:57:05.078237+0000","last_clean":"2024-04-26T23:57:05.078237+0000","last_became_active":"2024-04-26T23:57:05.072294+0000","last_became_peered":"2024-04-26T23:57:05.072294+0000","last_unstale":"2024-04-26T23:57:05.078237+0000","last_undegraded":"2024-04-26T23:57:05.078237+0000","last_fullsized":"2024-04-26T23:57:05.078237+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:15.405571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083157+0000","last_change":"2024-04-26T23:57:05.080701+0000","last_active":"2024-04-26T23:57:06.083157+0000","last_peered":"2024-04-26T23:57:06.083157+0000","last_clean":"2024-04-26T23:57:06.083157+0000","last_became_active":"2024-04-26T23:57:05.080369+0000","last_became_peered":"2024-04-26T23:57:05.080369+0000","last_unstale":"2024-04-26T23:57:06.083157+0000","last_undegraded":"2024-04-26T23:57:06.083157+0000","last_fullsized":"2024-04-26T23:57:06.083157+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:17.959749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077790+0000","last_change":"2024-04-26T23:57:05.066410+0000","last_active":"2024-04-26T23:57:06.077790+0000","last_peered":"2024-04-26T23:57:06.077790+0000","last_clean":"2024-04-26T23:57:06.077790+0000","last_became_active":"2024-04-26T23:57:05.066018+0000","last_became_peered":"2024-04-26T23:57:05.066018+0000","last_unstale":"2024-04-26T23:57:06.077790+0000","last_undegraded":"2024-04-26T23:57:06.077790+0000","last_fullsized":"2024-04-26T23:57:06.077790+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:26.188183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096336+0000","last_change":"2024-04-26T23:57:05.094305+0000","last_active":"2024-04-26T23:57:05.096336+0000","last_peered":"2024-04-26T23:57:05.096336+0000","last_clean":"2024-04-26T23:57:05.096336+0000","last_became_active":"2024-04-26T23:57:05.093724+0000","last_became_peered":"2024-04-26T23:57:05.093724+0000","last_unstale":"2024-04-26T23:57:05.096336+0000","last_undegraded":"2024-04-26T23:57:05.096336+0000","last_fullsized":"2024-04-26T23:57:05.096336+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.092882+0000","last_change":"2024-04-26T23:57:05.083768+0000","last_active":"2024-04-26T23:57:05.092882+0000","last_peered":"2024-04-26T23:57:05.092882+0000","last_clean":"2024-04-26T23:57:05.092882+0000","last_became_active":"2024-04-26T23:57:05.081409+0000","last_became_peered":"2024-04-26T23:57:05.081409+0000","last_unstale":"2024-04-26T23:57:05.092882+0000","last_undegraded":"2024-04-26T23:57:05.092882+0000","last_fullsized":"2024-04-26T23:57:05.092882+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084783+0000","last_change":"2024-04-26T23:57:05.087854+0000","last_active":"2024-04-26T23:57:06.084783+0000","last_peered":"2024-04-26T23:57:06.084783+0000","last_clean":"2024-04-26T23:57:06.084783+0000","last_became_active":"2024-04-26T23:57:05.087553+0000","last_became_peered":"2024-04-26T23:57:05.087553+0000","last_unstale":"2024-04-26T23:57:06.084783+0000","last_undegraded":"2024-04-26T23:57:06.084783+0000","last_fullsized":"2024-04-26T23:57:06.084783+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:37.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094525+0000","last_change":"2024-04-26T23:57:05.086399+0000","last_active":"2024-04-26T23:57:05.094525+0000","last_peered":"2024-04-26T23:57:05.094525+0000","last_clean":"2024-04-26T23:57:05.094525+0000","last_became_active":"2024-04-26T23:57:05.086101+0000","last_became_peered":"2024-04-26T23:57:05.086101+0000","last_unstale":"2024-04-26T23:57:05.094525+0000","last_undegraded":"2024-04-26T23:57:05.094525+0000","last_fullsized":"2024-04-26T23:57:05.094525+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:21.289106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096400+0000","last_change":"2024-04-26T23:57:05.094900+0000","last_active":"2024-04-26T23:57:05.096400+0000","last_peered":"2024-04-26T23:57:05.096400+0000","last_clean":"2024-04-26T23:57:05.096400+0000","last_became_active":"2024-04-26T23:57:05.094587+0000","last_became_peered":"2024-04-26T23:57:05.094587+0000","last_unstale":"2024-04-26T23:57:05.096400+0000","last_undegraded":"2024-04-26T23:57:05.096400+0000","last_fullsized":"2024-04-26T23:57:05.096400+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094263+0000","last_change":"2024-04-26T23:57:05.089125+0000","last_active":"2024-04-26T23:57:05.094263+0000","last_peered":"2024-04-26T23:57:05.094263+0000","last_clean":"2024-04-26T23:57:05.094263+0000","last_became_active":"2024-04-26T23:57:05.088784+0000","last_became_peered":"2024-04-26T23:57:05.088784+0000","last_unstale":"2024-04-26T23:57:05.094263+0000","last_undegraded":"2024-04-26T23:57:05.094263+0000","last_fullsized":"2024-04-26T23:57:05.094263+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:27.080832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080667+0000","last_change":"2024-04-26T23:57:05.088432+0000","last_active":"2024-04-26T23:57:06.080667+0000","last_peered":"2024-04-26T23:57:06.080667+0000","last_clean":"2024-04-26T23:57:06.080667+0000","last_became_active":"2024-04-26T23:57:05.088162+0000","last_became_peered":"2024-04-26T23:57:05.088162+0000","last_unstale":"2024-04-26T23:57:06.080667+0000","last_undegraded":"2024-04-26T23:57:06.080667+0000","last_fullsized":"2024-04-26T23:57:06.080667+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.470427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083116+0000","last_change":"2024-04-26T23:57:05.083617+0000","last_active":"2024-04-26T23:57:06.083116+0000","last_peered":"2024-04-26T23:57:06.083116+0000","last_clean":"2024-04-26T23:57:06.083116+0000","last_became_active":"2024-04-26T23:57:05.083328+0000","last_became_peered":"2024-04-26T23:57:05.083328+0000","last_unstale":"2024-04-26T23:57:06.083116+0000","last_undegraded":"2024-04-26T23:57:06.083116+0000","last_fullsized":"2024-04-26T23:57:06.083116+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:26.695978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096316+0000","last_change":"2024-04-26T23:57:05.093708+0000","last_active":"2024-04-26T23:57:05.096316+0000","last_peered":"2024-04-26T23:57:05.096316+0000","last_clean":"2024-04-26T23:57:05.096316+0000","last_became_active":"2024-04-26T23:57:05.093377+0000","last_became_peered":"2024-04-26T23:57:05.093377+0000","last_unstale":"2024-04-26T23:57:05.096316+0000","last_undegraded":"2024-04-26T23:57:05.096316+0000","last_fullsized":"2024-04-26T23:57:05.096316+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080540+0000","last_change":"2024-04-26T23:57:05.075657+0000","last_active":"2024-04-26T23:57:06.080540+0000","last_peered":"2024-04-26T23:57:06.080540+0000","last_clean":"2024-04-26T23:57:06.080540+0000","last_became_active":"2024-04-26T23:57:05.075338+0000","last_became_peered":"2024-04-26T23:57:05.075338+0000","last_unstale":"2024-04-26T23:57:06.080540+0000","last_undegraded":"2024-04-26T23:57:06.080540+0000","last_fullsized":"2024-04-26T23:57:06.080540+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:08.193361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.086510+0000","last_change":"2024-04-26T23:57:05.078784+0000","last_active":"2024-04-26T23:57:05.086510+0000","last_peered":"2024-04-26T23:57:05.086510+0000","last_clean":"2024-04-26T23:57:05.086510+0000","last_became_active":"2024-04-26T23:57:05.078483+0000","last_became_peered":"2024-04-26T23:57:05.078483+0000","last_unstale":"2024-04-26T23:57:05.086510+0000","last_undegraded":"2024-04-26T23:57:05.086510+0000","last_fullsized":"2024-04-26T23:57:05.086510+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:56.176146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082011+0000","last_change":"2024-04-26T23:57:05.079690+0000","last_active":"2024-04-26T23:57:06.082011+0000","last_peered":"2024-04-26T23:57:06.082011+0000","last_clean":"2024-04-26T23:57:06.082011+0000","last_became_active":"2024-04-26T23:57:05.079400+0000","last_became_peered":"2024-04-26T23:57:05.079400+0000","last_unstale":"2024-04-26T23:57:06.082011+0000","last_undegraded":"2024-04-26T23:57:06.082011+0000","last_fullsized":"2024-04-26T23:57:06.082011+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090981+0000","last_change":"2024-04-26T23:57:05.089934+0000","last_active":"2024-04-26T23:57:05.090981+0000","last_peered":"2024-04-26T23:57:05.090981+0000","last_clean":"2024-04-26T23:57:05.090981+0000","last_became_active":"2024-04-26T23:57:05.089618+0000","last_became_peered":"2024-04-26T23:57:05.089618+0000","last_unstale":"2024-04-26T23:57:05.090981+0000","last_undegraded":"2024-04-26T23:57:05.090981+0000","last_fullsized":"2024-04-26T23:57:05.090981+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:49.957221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099645+0000","last_change":"2024-04-26T23:57:05.096391+0000","last_active":"2024-04-26T23:57:05.099645+0000","last_peered":"2024-04-26T23:57:05.099645+0000","last_clean":"2024-04-26T23:57:05.099645+0000","last_became_active":"2024-04-26T23:57:05.096100+0000","last_became_peered":"2024-04-26T23:57:05.096100+0000","last_unstale":"2024-04-26T23:57:05.099645+0000","last_undegraded":"2024-04-26T23:57:05.099645+0000","last_fullsized":"2024-04-26T23:57:05.099645+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.897714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077990+0000","last_change":"2024-04-26T23:57:05.069929+0000","last_active":"2024-04-26T23:57:06.077990+0000","last_peered":"2024-04-26T23:57:06.077990+0000","last_clean":"2024-04-26T23:57:06.077990+0000","last_became_active":"2024-04-26T23:57:05.069607+0000","last_became_peered":"2024-04-26T23:57:05.069607+0000","last_unstale":"2024-04-26T23:57:06.077990+0000","last_undegraded":"2024-04-26T23:57:06.077990+0000","last_fullsized":"2024-04-26T23:57:06.077990+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:12.409228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077460+0000","last_change":"2024-04-26T23:57:05.087321+0000","last_active":"2024-04-26T23:57:06.077460+0000","last_peered":"2024-04-26T23:57:06.077460+0000","last_clean":"2024-04-26T23:57:06.077460+0000","last_became_active":"2024-04-26T23:57:05.087022+0000","last_became_peered":"2024-04-26T23:57:05.087022+0000","last_unstale":"2024-04-26T23:57:06.077460+0000","last_undegraded":"2024-04-26T23:57:06.077460+0000","last_fullsized":"2024-04-26T23:57:06.077460+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:42.112797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095073+0000","last_change":"2024-04-26T23:57:05.093090+0000","last_active":"2024-04-26T23:57:05.095073+0000","last_peered":"2024-04-26T23:57:05.095073+0000","last_clean":"2024-04-26T23:57:05.095073+0000","last_became_active":"2024-04-26T23:57:05.091823+0000","last_became_peered":"2024-04-26T23:57:05.091823+0000","last_unstale":"2024-04-26T23:57:05.095073+0000","last_undegraded":"2024-04-26T23:57:05.095073+0000","last_fullsized":"2024-04-26T23:57:05.095073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:39.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099734+0000","last_change":"2024-04-26T23:57:05.097991+0000","last_active":"2024-04-26T23:57:05.099734+0000","last_peered":"2024-04-26T23:57:05.099734+0000","last_clean":"2024-04-26T23:57:05.099734+0000","last_became_active":"2024-04-26T23:57:05.095679+0000","last_became_peered":"2024-04-26T23:57:05.095679+0000","last_unstale":"2024-04-26T23:57:05.099734+0000","last_undegraded":"2024-04-26T23:57:05.099734+0000","last_fullsized":"2024-04-26T23:57:05.099734+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:40.282754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090749+0000","last_change":"2024-04-26T23:57:05.085630+0000","last_active":"2024-04-26T23:57:05.090749+0000","last_peered":"2024-04-26T23:57:05.090749+0000","last_clean":"2024-04-26T23:57:05.090749+0000","last_became_active":"2024-04-26T23:57:05.083187+0000","last_became_peered":"2024-04-26T23:57:05.083187+0000","last_unstale":"2024-04-26T23:57:05.090749+0000","last_undegraded":"2024-04-26T23:57:05.090749+0000","last_fullsized":"2024-04-26T23:57:05.090749+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:13.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"209'4","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099168+0000","last_change":"2024-04-26T23:57:04.056445+0000","last_active":"2024-04-26T23:57:05.099168+0000","last_peered":"2024-04-26T23:57:05.099168+0000","last_clean":"2024-04-26T23:57:05.099168+0000","last_became_active":"2024-04-26T23:57:04.056166+0000","last_became_peered":"2024-04-26T23:57:04.056166+0000","last_unstale":"2024-04-26T23:57:05.099168+0000","last_undegraded":"2024-04-26T23:57:05.099168+0000","last_fullsized":"2024-04-26T23:57:05.099168+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:42.833864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098034+0000","last_change":"2024-04-26T23:57:05.097233+0000","last_active":"2024-04-26T23:57:05.098034+0000","last_peered":"2024-04-26T23:57:05.098034+0000","last_clean":"2024-04-26T23:57:05.098034+0000","last_became_active":"2024-04-26T23:57:05.096935+0000","last_became_peered":"2024-04-26T23:57:05.096935+0000","last_unstale":"2024-04-26T23:57:05.098034+0000","last_undegraded":"2024-04-26T23:57:05.098034+0000","last_fullsized":"2024-04-26T23:57:05.098034+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:21.289106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"209'2","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083973+0000","last_change":"2024-04-26T23:57:04.063135+0000","last_active":"2024-04-26T23:57:06.083973+0000","last_peered":"2024-04-26T23:57:06.083973+0000","last_clean":"2024-04-26T23:57:06.083973+0000","last_became_active":"2024-04-26T23:57:04.062804+0000","last_became_peered":"2024-04-26T23:57:04.062804+0000","last_unstale":"2024-04-26T23:57:06.083973+0000","last_undegraded":"2024-04-26T23:57:06.083973+0000","last_fullsized":"2024-04-26T23:57:06.083973+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:40.465823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095843+0000","last_change":"2024-04-26T23:57:05.090168+0000","last_active":"2024-04-26T23:57:05.095843+0000","last_peered":"2024-04-26T23:57:05.095843+0000","last_clean":"2024-04-26T23:57:05.095843+0000","last_became_active":"2024-04-26T23:57:05.089909+0000","last_became_peered":"2024-04-26T23:57:05.089909+0000","last_unstale":"2024-04-26T23:57:05.095843+0000","last_undegraded":"2024-04-26T23:57:05.095843+0000","last_fullsized":"2024-04-26T23:57:05.095843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:08.172802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"209'2","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078038+0000","last_change":"2024-04-26T23:57:04.050049+0000","last_active":"2024-04-26T23:57:06.078038+0000","last_peered":"2024-04-26T23:57:06.078038+0000","last_clean":"2024-04-26T23:57:06.078038+0000","last_became_active":"2024-04-26T23:57:04.048921+0000","last_became_peered":"2024-04-26T23:57:04.048921+0000","last_unstale":"2024-04-26T23:57:06.078038+0000","last_undegraded":"2024-04-26T23:57:06.078038+0000","last_fullsized":"2024-04-26T23:57:06.078038+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:39.590188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077935+0000","last_change":"2024-04-26T23:57:05.079620+0000","last_active":"2024-04-26T23:57:06.077935+0000","last_peered":"2024-04-26T23:57:06.077935+0000","last_clean":"2024-04-26T23:57:06.077935+0000","last_became_active":"2024-04-26T23:57:05.079311+0000","last_became_peered":"2024-04-26T23:57:05.079311+0000","last_unstale":"2024-04-26T23:57:06.077935+0000","last_undegraded":"2024-04-26T23:57:06.077935+0000","last_fullsized":"2024-04-26T23:57:06.077935+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:19.264558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"209'1","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093894+0000","last_change":"2024-04-26T23:57:04.046528+0000","last_active":"2024-04-26T23:57:05.093894+0000","last_peered":"2024-04-26T23:57:05.093894+0000","last_clean":"2024-04-26T23:57:05.093894+0000","last_became_active":"2024-04-26T23:57:04.046146+0000","last_became_peered":"2024-04-26T23:57:04.046146+0000","last_unstale":"2024-04-26T23:57:05.093894+0000","last_undegraded":"2024-04-26T23:57:05.093894+0000","last_fullsized":"2024-04-26T23:57:05.093894+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081671+0000","last_change":"2024-04-26T23:57:05.088567+0000","last_active":"2024-04-26T23:57:06.081671+0000","last_peered":"2024-04-26T23:57:06.081671+0000","last_clean":"2024-04-26T23:57:06.081671+0000","last_became_active":"2024-04-26T23:57:05.088263+0000","last_became_peered":"2024-04-26T23:57:05.088263+0000","last_unstale":"2024-04-26T23:57:06.081671+0000","last_undegraded":"2024-04-26T23:57:06.081671+0000","last_fullsized":"2024-04-26T23:57:06.081671+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077674+0000","last_change":"2024-04-26T23:57:04.045360+0000","last_active":"2024-04-26T23:57:06.077674+0000","last_peered":"2024-04-26T23:57:06.077674+0000","last_clean":"2024-04-26T23:57:06.077674+0000","last_became_active":"2024-04-26T23:57:04.044970+0000","last_became_peered":"2024-04-26T23:57:04.044970+0000","last_unstale":"2024-04-26T23:57:06.077674+0000","last_undegraded":"2024-04-26T23:57:06.077674+0000","last_fullsized":"2024-04-26T23:57:06.077674+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087544+0000","last_change":"2024-04-26T23:57:05.081272+0000","last_active":"2024-04-26T23:57:05.087544+0000","last_peered":"2024-04-26T23:57:05.087544+0000","last_clean":"2024-04-26T23:57:05.087544+0000","last_became_active":"2024-04-26T23:57:05.080986+0000","last_became_peered":"2024-04-26T23:57:05.080986+0000","last_unstale":"2024-04-26T23:57:05.087544+0000","last_undegraded":"2024-04-26T23:57:05.087544+0000","last_fullsized":"2024-04-26T23:57:05.087544+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.075180+0000","last_change":"2024-04-26T23:57:04.052584+0000","last_active":"2024-04-26T23:57:05.075180+0000","last_peered":"2024-04-26T23:57:05.075180+0000","last_clean":"2024-04-26T23:57:05.075180+0000","last_became_active":"2024-04-26T23:57:04.052300+0000","last_became_peered":"2024-04-26T23:57:04.052300+0000","last_unstale":"2024-04-26T23:57:05.075180+0000","last_undegraded":"2024-04-26T23:57:05.075180+0000","last_fullsized":"2024-04-26T23:57:05.075180+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090830+0000","last_change":"2024-04-26T23:57:05.088368+0000","last_active":"2024-04-26T23:57:05.090830+0000","last_peered":"2024-04-26T23:57:05.090830+0000","last_clean":"2024-04-26T23:57:05.090830+0000","last_became_active":"2024-04-26T23:57:05.088089+0000","last_became_peered":"2024-04-26T23:57:05.088089+0000","last_unstale":"2024-04-26T23:57:05.090830+0000","last_undegraded":"2024-04-26T23:57:05.090830+0000","last_fullsized":"2024-04-26T23:57:05.090830+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:41.644183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.057705+0000","last_change":"2024-04-26T23:57:04.053036+0000","last_active":"2024-04-26T23:57:05.057705+0000","last_peered":"2024-04-26T23:57:05.057705+0000","last_clean":"2024-04-26T23:57:05.057705+0000","last_became_active":"2024-04-26T23:57:04.052701+0000","last_became_peered":"2024-04-26T23:57:04.052701+0000","last_unstale":"2024-04-26T23:57:05.057705+0000","last_undegraded":"2024-04-26T23:57:05.057705+0000","last_fullsized":"2024-04-26T23:57:05.057705+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:44.516642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080427+0000","last_change":"2024-04-26T23:57:05.092053+0000","last_active":"2024-04-26T23:57:06.080427+0000","last_peered":"2024-04-26T23:57:06.080427+0000","last_clean":"2024-04-26T23:57:06.080427+0000","last_became_active":"2024-04-26T23:57:05.091789+0000","last_became_peered":"2024-04-26T23:57:05.091789+0000","last_unstale":"2024-04-26T23:57:06.080427+0000","last_undegraded":"2024-04-26T23:57:06.080427+0000","last_fullsized":"2024-04-26T23:57:06.080427+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:26.662514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078416+0000","last_change":"2024-04-26T23:57:04.053259+0000","last_active":"2024-04-26T23:57:06.078416+0000","last_peered":"2024-04-26T23:57:06.078416+0000","last_clean":"2024-04-26T23:57:06.078416+0000","last_became_active":"2024-04-26T23:57:04.052008+0000","last_became_peered":"2024-04-26T23:57:04.052008+0000","last_unstale":"2024-04-26T23:57:06.078416+0000","last_undegraded":"2024-04-26T23:57:06.078416+0000","last_fullsized":"2024-04-26T23:57:06.078416+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:11.453284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082359+0000","last_change":"2024-04-26T23:57:05.079198+0000","last_active":"2024-04-26T23:57:06.082359+0000","last_peered":"2024-04-26T23:57:06.082359+0000","last_clean":"2024-04-26T23:57:06.082359+0000","last_became_active":"2024-04-26T23:57:05.078660+0000","last_became_peered":"2024-04-26T23:57:05.078660+0000","last_unstale":"2024-04-26T23:57:06.082359+0000","last_undegraded":"2024-04-26T23:57:06.082359+0000","last_fullsized":"2024-04-26T23:57:06.082359+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:44.819513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.074507+0000","last_change":"2024-04-26T23:57:04.063553+0000","last_active":"2024-04-26T23:57:05.074507+0000","last_peered":"2024-04-26T23:57:05.074507+0000","last_clean":"2024-04-26T23:57:05.074507+0000","last_became_active":"2024-04-26T23:57:04.063114+0000","last_became_peered":"2024-04-26T23:57:04.063114+0000","last_unstale":"2024-04-26T23:57:05.074507+0000","last_undegraded":"2024-04-26T23:57:05.074507+0000","last_fullsized":"2024-04-26T23:57:05.074507+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.648721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094131+0000","last_change":"2024-04-26T23:57:05.088479+0000","last_active":"2024-04-26T23:57:05.094131+0000","last_peered":"2024-04-26T23:57:05.094131+0000","last_clean":"2024-04-26T23:57:05.094131+0000","last_became_active":"2024-04-26T23:57:05.087773+0000","last_became_peered":"2024-04-26T23:57:05.087773+0000","last_unstale":"2024-04-26T23:57:05.094131+0000","last_undegraded":"2024-04-26T23:57:05.094131+0000","last_fullsized":"2024-04-26T23:57:05.094131+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"209'3","reported_seq":15,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096972+0000","last_change":"2024-04-26T23:57:04.053271+0000","last_active":"2024-04-26T23:57:05.096972+0000","last_peered":"2024-04-26T23:57:05.096972+0000","last_clean":"2024-04-26T23:57:05.096972+0000","last_became_active":"2024-04-26T23:57:04.052882+0000","last_became_peered":"2024-04-26T23:57:04.052882+0000","last_unstale":"2024-04-26T23:57:05.096972+0000","last_undegraded":"2024-04-26T23:57:05.096972+0000","last_fullsized":"2024-04-26T23:57:05.096972+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:43.802370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081300+0000","last_change":"2024-04-26T23:57:05.080628+0000","last_active":"2024-04-26T23:57:06.081300+0000","last_peered":"2024-04-26T23:57:06.081300+0000","last_clean":"2024-04-26T23:57:06.081300+0000","last_became_active":"2024-04-26T23:57:05.080332+0000","last_became_peered":"2024-04-26T23:57:05.080332+0000","last_unstale":"2024-04-26T23:57:06.081300+0000","last_undegraded":"2024-04-26T23:57:06.081300+0000","last_fullsized":"2024-04-26T23:57:06.081300+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"209'6","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.101081+0000","last_change":"2024-04-26T23:57:04.048030+0000","last_active":"2024-04-26T23:57:05.101081+0000","last_peered":"2024-04-26T23:57:05.101081+0000","last_clean":"2024-04-26T23:57:05.101081+0000","last_became_active":"2024-04-26T23:57:04.047721+0000","last_became_peered":"2024-04-26T23:57:04.047721+0000","last_unstale":"2024-04-26T23:57:05.101081+0000","last_undegraded":"2024-04-26T23:57:05.101081+0000","last_fullsized":"2024-04-26T23:57:05.101081+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:01.790838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087154+0000","last_change":"2024-04-26T23:57:05.078581+0000","last_active":"2024-04-26T23:57:05.087154+0000","last_peered":"2024-04-26T23:57:05.087154+0000","last_clean":"2024-04-26T23:57:05.087154+0000","last_became_active":"2024-04-26T23:57:05.078278+0000","last_became_peered":"2024-04-26T23:57:05.078278+0000","last_unstale":"2024-04-26T23:57:05.087154+0000","last_undegraded":"2024-04-26T23:57:05.087154+0000","last_fullsized":"2024-04-26T23:57:05.087154+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:33.487077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099105+0000","last_change":"2024-04-26T23:57:04.049339+0000","last_active":"2024-04-26T23:57:05.099105+0000","last_peered":"2024-04-26T23:57:05.099105+0000","last_clean":"2024-04-26T23:57:05.099105+0000","last_became_active":"2024-04-26T23:57:04.048885+0000","last_became_peered":"2024-04-26T23:57:04.048885+0000","last_unstale":"2024-04-26T23:57:05.099105+0000","last_undegraded":"2024-04-26T23:57:05.099105+0000","last_fullsized":"2024-04-26T23:57:05.099105+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:25.645371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081803+0000","last_change":"2024-04-26T23:57:05.080283+0000","last_active":"2024-04-26T23:57:06.081803+0000","last_peered":"2024-04-26T23:57:06.081803+0000","last_clean":"2024-04-26T23:57:06.081803+0000","last_became_active":"2024-04-26T23:57:05.078092+0000","last_became_peered":"2024-04-26T23:57:05.078092+0000","last_unstale":"2024-04-26T23:57:06.081803+0000","last_undegraded":"2024-04-26T23:57:06.081803+0000","last_fullsized":"2024-04-26T23:57:06.081803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.069045+0000","last_change":"2024-04-26T23:57:04.053420+0000","last_active":"2024-04-26T23:57:05.069045+0000","last_peered":"2024-04-26T23:57:05.069045+0000","last_clean":"2024-04-26T23:57:05.069045+0000","last_became_active":"2024-04-26T23:57:04.053081+0000","last_became_peered":"2024-04-26T23:57:04.053081+0000","last_unstale":"2024-04-26T23:57:05.069045+0000","last_undegraded":"2024-04-26T23:57:05.069045+0000","last_fullsized":"2024-04-26T23:57:05.069045+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:14.330244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096284+0000","last_change":"2024-04-26T23:57:05.093868+0000","last_active":"2024-04-26T23:57:05.096284+0000","last_peered":"2024-04-26T23:57:05.096284+0000","last_clean":"2024-04-26T23:57:05.096284+0000","last_became_active":"2024-04-26T23:57:05.093556+0000","last_became_peered":"2024-04-26T23:57:05.093556+0000","last_unstale":"2024-04-26T23:57:05.096284+0000","last_undegraded":"2024-04-26T23:57:05.096284+0000","last_fullsized":"2024-04-26T23:57:05.096284+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.078086+0000","last_change":"2024-04-26T23:57:04.062067+0000","last_active":"2024-04-26T23:57:05.078086+0000","last_peered":"2024-04-26T23:57:05.078086+0000","last_clean":"2024-04-26T23:57:05.078086+0000","last_became_active":"2024-04-26T23:57:04.061669+0000","last_became_peered":"2024-04-26T23:57:04.061669+0000","last_unstale":"2024-04-26T23:57:05.078086+0000","last_undegraded":"2024-04-26T23:57:05.078086+0000","last_fullsized":"2024-04-26T23:57:05.078086+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:07.176218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097606+0000","last_change":"2024-04-26T23:57:05.096573+0000","last_active":"2024-04-26T23:57:05.097606+0000","last_peered":"2024-04-26T23:57:05.097606+0000","last_clean":"2024-04-26T23:57:05.097606+0000","last_became_active":"2024-04-26T23:57:05.095841+0000","last_became_peered":"2024-04-26T23:57:05.095841+0000","last_unstale":"2024-04-26T23:57:05.097606+0000","last_undegraded":"2024-04-26T23:57:05.097606+0000","last_fullsized":"2024-04-26T23:57:05.097606+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:47.477177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"209'8","reported_seq":20,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.091032+0000","last_change":"2024-04-26T23:57:04.051545+0000","last_active":"2024-04-26T23:57:05.091032+0000","last_peered":"2024-04-26T23:57:05.091032+0000","last_clean":"2024-04-26T23:57:05.091032+0000","last_became_active":"2024-04-26T23:57:04.051205+0000","last_became_peered":"2024-04-26T23:57:04.051205+0000","last_unstale":"2024-04-26T23:57:05.091032+0000","last_undegraded":"2024-04-26T23:57:05.091032+0000","last_fullsized":"2024-04-26T23:57:05.091032+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:00.843562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096405+0000","last_change":"2024-04-26T23:57:05.095286+0000","last_active":"2024-04-26T23:57:05.096405+0000","last_peered":"2024-04-26T23:57:05.096405+0000","last_clean":"2024-04-26T23:57:05.096405+0000","last_became_active":"2024-04-26T23:57:05.094948+0000","last_became_peered":"2024-04-26T23:57:05.094948+0000","last_unstale":"2024-04-26T23:57:05.096405+0000","last_undegraded":"2024-04-26T23:57:05.096405+0000","last_fullsized":"2024-04-26T23:57:05.096405+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:17.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"209'1","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097410+0000","last_change":"2024-04-26T23:57:04.060733+0000","last_active":"2024-04-26T23:57:05.097410+0000","last_peered":"2024-04-26T23:57:05.097410+0000","last_clean":"2024-04-26T23:57:05.097410+0000","last_became_active":"2024-04-26T23:57:04.060349+0000","last_became_peered":"2024-04-26T23:57:04.060349+0000","last_unstale":"2024-04-26T23:57:05.097410+0000","last_undegraded":"2024-04-26T23:57:05.097410+0000","last_fullsized":"2024-04-26T23:57:05.097410+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:51.799965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095827+0000","last_change":"2024-04-26T23:57:05.086368+0000","last_active":"2024-04-26T23:57:05.095827+0000","last_peered":"2024-04-26T23:57:05.095827+0000","last_clean":"2024-04-26T23:57:05.095827+0000","last_became_active":"2024-04-26T23:57:05.086030+0000","last_became_peered":"2024-04-26T23:57:05.086030+0000","last_unstale":"2024-04-26T23:57:05.095827+0000","last_undegraded":"2024-04-26T23:57:05.095827+0000","last_fullsized":"2024-04-26T23:57:05.095827+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:26.891319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.088220+0000","last_change":"2024-04-26T23:57:04.049142+0000","last_active":"2024-04-26T23:57:05.088220+0000","last_peered":"2024-04-26T23:57:05.088220+0000","last_clean":"2024-04-26T23:57:05.088220+0000","last_became_active":"2024-04-26T23:57:04.048780+0000","last_became_peered":"2024-04-26T23:57:04.048780+0000","last_unstale":"2024-04-26T23:57:05.088220+0000","last_undegraded":"2024-04-26T23:57:05.088220+0000","last_fullsized":"2024-04-26T23:57:05.088220+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:27.603016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098171+0000","last_change":"2024-04-26T23:57:05.096793+0000","last_active":"2024-04-26T23:57:05.098171+0000","last_peered":"2024-04-26T23:57:05.098171+0000","last_clean":"2024-04-26T23:57:05.098171+0000","last_became_active":"2024-04-26T23:57:05.096484+0000","last_became_peered":"2024-04-26T23:57:05.096484+0000","last_unstale":"2024-04-26T23:57:05.098171+0000","last_undegraded":"2024-04-26T23:57:05.098171+0000","last_fullsized":"2024-04-26T23:57:05.098171+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:14.349383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084152+0000","last_change":"2024-04-26T23:57:04.064277+0000","last_active":"2024-04-26T23:57:06.084152+0000","last_peered":"2024-04-26T23:57:06.084152+0000","last_clean":"2024-04-26T23:57:06.084152+0000","last_became_active":"2024-04-26T23:57:04.062632+0000","last_became_peered":"2024-04-26T23:57:04.062632+0000","last_unstale":"2024-04-26T23:57:06.084152+0000","last_undegraded":"2024-04-26T23:57:06.084152+0000","last_fullsized":"2024-04-26T23:57:06.084152+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:14.214176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093861+0000","last_change":"2024-04-26T23:57:05.087622+0000","last_active":"2024-04-26T23:57:05.093861+0000","last_peered":"2024-04-26T23:57:05.093861+0000","last_clean":"2024-04-26T23:57:05.093861+0000","last_became_active":"2024-04-26T23:57:05.087199+0000","last_became_peered":"2024-04-26T23:57:05.087199+0000","last_unstale":"2024-04-26T23:57:05.093861+0000","last_undegraded":"2024-04-26T23:57:05.093861+0000","last_fullsized":"2024-04-26T23:57:05.093861+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.079735+0000","last_change":"2024-04-26T23:57:04.063664+0000","last_active":"2024-04-26T23:57:05.079735+0000","last_peered":"2024-04-26T23:57:05.079735+0000","last_clean":"2024-04-26T23:57:05.079735+0000","last_became_active":"2024-04-26T23:57:04.062388+0000","last_became_peered":"2024-04-26T23:57:04.062388+0000","last_unstale":"2024-04-26T23:57:05.079735+0000","last_undegraded":"2024-04-26T23:57:05.079735+0000","last_fullsized":"2024-04-26T23:57:05.079735+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:11.453284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093899+0000","last_change":"2024-04-26T23:57:05.089980+0000","last_active":"2024-04-26T23:57:05.093899+0000","last_peered":"2024-04-26T23:57:05.093899+0000","last_clean":"2024-04-26T23:57:05.093899+0000","last_became_active":"2024-04-26T23:57:05.089449+0000","last_became_peered":"2024-04-26T23:57:05.089449+0000","last_unstale":"2024-04-26T23:57:05.093899+0000","last_undegraded":"2024-04-26T23:57:05.093899+0000","last_fullsized":"2024-04-26T23:57:05.093899+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:35.569494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.092792+0000","last_change":"2024-04-26T23:57:04.060736+0000","last_active":"2024-04-26T23:57:05.092792+0000","last_peered":"2024-04-26T23:57:05.092792+0000","last_clean":"2024-04-26T23:57:05.092792+0000","last_became_active":"2024-04-26T23:57:04.060416+0000","last_became_peered":"2024-04-26T23:57:04.060416+0000","last_unstale":"2024-04-26T23:57:05.092792+0000","last_undegraded":"2024-04-26T23:57:05.092792+0000","last_fullsized":"2024-04-26T23:57:05.092792+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:19.316285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093388+0000","last_change":"2024-04-26T23:57:05.085624+0000","last_active":"2024-04-26T23:57:05.093388+0000","last_peered":"2024-04-26T23:57:05.093388+0000","last_clean":"2024-04-26T23:57:05.093388+0000","last_became_active":"2024-04-26T23:57:05.082384+0000","last_became_peered":"2024-04-26T23:57:05.082384+0000","last_unstale":"2024-04-26T23:57:05.093388+0000","last_undegraded":"2024-04-26T23:57:05.093388+0000","last_fullsized":"2024-04-26T23:57:05.093388+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:18.963136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083357+0000","last_change":"2024-04-26T23:57:04.061762+0000","last_active":"2024-04-26T23:57:06.083357+0000","last_peered":"2024-04-26T23:57:06.083357+0000","last_clean":"2024-04-26T23:57:06.083357+0000","last_became_active":"2024-04-26T23:57:04.061445+0000","last_became_peered":"2024-04-26T23:57:04.061445+0000","last_unstale":"2024-04-26T23:57:06.083357+0000","last_undegraded":"2024-04-26T23:57:06.083357+0000","last_fullsized":"2024-04-26T23:57:06.083357+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:08.382806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084456+0000","last_change":"2024-04-26T23:57:05.090226+0000","last_active":"2024-04-26T23:57:06.084456+0000","last_peered":"2024-04-26T23:57:06.084456+0000","last_clean":"2024-04-26T23:57:06.084456+0000","last_became_active":"2024-04-26T23:57:05.089933+0000","last_became_peered":"2024-04-26T23:57:05.089933+0000","last_unstale":"2024-04-26T23:57:06.084456+0000","last_undegraded":"2024-04-26T23:57:06.084456+0000","last_fullsized":"2024-04-26T23:57:06.084456+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:26.891319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082192+0000","last_change":"2024-04-26T23:57:04.064554+0000","last_active":"2024-04-26T23:57:06.082192+0000","last_peered":"2024-04-26T23:57:06.082192+0000","last_clean":"2024-04-26T23:57:06.082192+0000","last_became_active":"2024-04-26T23:57:04.063846+0000","last_became_peered":"2024-04-26T23:57:04.063846+0000","last_unstale":"2024-04-26T23:57:06.082192+0000","last_undegraded":"2024-04-26T23:57:06.082192+0000","last_fullsized":"2024-04-26T23:57:06.082192+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:37.973436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084847+0000","last_change":"2024-04-26T23:57:05.093760+0000","last_active":"2024-04-26T23:57:06.084847+0000","last_peered":"2024-04-26T23:57:06.084847+0000","last_clean":"2024-04-26T23:57:06.084847+0000","last_became_active":"2024-04-26T23:57:05.093441+0000","last_became_peered":"2024-04-26T23:57:05.093441+0000","last_unstale":"2024-04-26T23:57:06.084847+0000","last_undegraded":"2024-04-26T23:57:06.084847+0000","last_fullsized":"2024-04-26T23:57:06.084847+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:49.957221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077242+0000","last_change":"2024-04-26T23:57:04.049458+0000","last_active":"2024-04-26T23:57:06.077242+0000","last_peered":"2024-04-26T23:57:06.077242+0000","last_clean":"2024-04-26T23:57:06.077242+0000","last_became_active":"2024-04-26T23:57:04.049137+0000","last_became_peered":"2024-04-26T23:57:04.049137+0000","last_unstale":"2024-04-26T23:57:06.077242+0000","last_undegraded":"2024-04-26T23:57:06.077242+0000","last_fullsized":"2024-04-26T23:57:06.077242+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:34.329692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080354+0000","last_change":"2024-04-26T23:57:05.083452+0000","last_active":"2024-04-26T23:57:06.080354+0000","last_peered":"2024-04-26T23:57:06.080354+0000","last_clean":"2024-04-26T23:57:06.080354+0000","last_became_active":"2024-04-26T23:57:05.082946+0000","last_became_peered":"2024-04-26T23:57:05.082946+0000","last_unstale":"2024-04-26T23:57:06.080354+0000","last_undegraded":"2024-04-26T23:57:06.080354+0000","last_fullsized":"2024-04-26T23:57:06.080354+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:22.166396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.091062+0000","last_change":"2024-04-26T23:57:04.055654+0000","last_active":"2024-04-26T23:57:05.091062+0000","last_peered":"2024-04-26T23:57:05.091062+0000","last_clean":"2024-04-26T23:57:05.091062+0000","last_became_active":"2024-04-26T23:57:04.055349+0000","last_became_peered":"2024-04-26T23:57:04.055349+0000","last_unstale":"2024-04-26T23:57:05.091062+0000","last_undegraded":"2024-04-26T23:57:05.091062+0000","last_fullsized":"2024-04-26T23:57:05.091062+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:21.466960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095466+0000","last_change":"2024-04-26T23:57:05.092079+0000","last_active":"2024-04-26T23:57:05.095466+0000","last_peered":"2024-04-26T23:57:05.095466+0000","last_clean":"2024-04-26T23:57:05.095466+0000","last_became_active":"2024-04-26T23:57:05.091740+0000","last_became_peered":"2024-04-26T23:57:05.091740+0000","last_unstale":"2024-04-26T23:57:05.095466+0000","last_undegraded":"2024-04-26T23:57:05.095466+0000","last_fullsized":"2024-04-26T23:57:05.095466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.085211+0000","last_change":"2024-04-26T23:57:04.059909+0000","last_active":"2024-04-26T23:57:05.085211+0000","last_peered":"2024-04-26T23:57:05.085211+0000","last_clean":"2024-04-26T23:57:05.085211+0000","last_became_active":"2024-04-26T23:57:04.059369+0000","last_became_peered":"2024-04-26T23:57:04.059369+0000","last_unstale":"2024-04-26T23:57:05.085211+0000","last_undegraded":"2024-04-26T23:57:05.085211+0000","last_fullsized":"2024-04-26T23:57:05.085211+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:25.678834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.084073+0000","last_change":"2024-04-26T23:57:05.077101+0000","last_active":"2024-04-26T23:57:05.084073+0000","last_peered":"2024-04-26T23:57:05.084073+0000","last_clean":"2024-04-26T23:57:05.084073+0000","last_became_active":"2024-04-26T23:57:05.074834+0000","last_became_peered":"2024-04-26T23:57:05.074834+0000","last_unstale":"2024-04-26T23:57:05.084073+0000","last_undegraded":"2024-04-26T23:57:05.084073+0000","last_fullsized":"2024-04-26T23:57:05.084073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:26.662514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078406+0000","last_change":"2024-04-26T23:57:04.049882+0000","last_active":"2024-04-26T23:57:06.078406+0000","last_peered":"2024-04-26T23:57:06.078406+0000","last_clean":"2024-04-26T23:57:06.078406+0000","last_became_active":"2024-04-26T23:57:04.049552+0000","last_became_peered":"2024-04-26T23:57:04.049552+0000","last_unstale":"2024-04-26T23:57:06.078406+0000","last_undegraded":"2024-04-26T23:57:06.078406+0000","last_fullsized":"2024-04-26T23:57:06.078406+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:43.830299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097639+0000","last_change":"2024-04-26T23:57:05.096587+0000","last_active":"2024-04-26T23:57:05.097639+0000","last_peered":"2024-04-26T23:57:05.097639+0000","last_clean":"2024-04-26T23:57:05.097639+0000","last_became_active":"2024-04-26T23:57:05.096302+0000","last_became_peered":"2024-04-26T23:57:05.096302+0000","last_unstale":"2024-04-26T23:57:05.097639+0000","last_undegraded":"2024-04-26T23:57:05.097639+0000","last_fullsized":"2024-04-26T23:57:05.097639+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:53.661439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078141+0000","last_change":"2024-04-26T23:57:04.059981+0000","last_active":"2024-04-26T23:57:06.078141+0000","last_peered":"2024-04-26T23:57:06.078141+0000","last_clean":"2024-04-26T23:57:06.078141+0000","last_became_active":"2024-04-26T23:57:04.059656+0000","last_became_peered":"2024-04-26T23:57:04.059656+0000","last_unstale":"2024-04-26T23:57:06.078141+0000","last_undegraded":"2024-04-26T23:57:06.078141+0000","last_fullsized":"2024-04-26T23:57:06.078141+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:25.645371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078101+0000","last_change":"2024-04-26T23:57:05.069705+0000","last_active":"2024-04-26T23:57:06.078101+0000","last_peered":"2024-04-26T23:57:06.078101+0000","last_clean":"2024-04-26T23:57:06.078101+0000","last_became_active":"2024-04-26T23:57:05.069377+0000","last_became_peered":"2024-04-26T23:57:05.069377+0000","last_unstale":"2024-04-26T23:57:06.078101+0000","last_undegraded":"2024-04-26T23:57:06.078101+0000","last_fullsized":"2024-04-26T23:57:06.078101+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:36.124883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083098+0000","last_change":"2024-04-26T23:57:04.062696+0000","last_active":"2024-04-26T23:57:06.083098+0000","last_peered":"2024-04-26T23:57:06.083098+0000","last_clean":"2024-04-26T23:57:06.083098+0000","last_became_active":"2024-04-26T23:57:04.062379+0000","last_became_peered":"2024-04-26T23:57:04.062379+0000","last_unstale":"2024-04-26T23:57:06.083098+0000","last_undegraded":"2024-04-26T23:57:06.083098+0000","last_fullsized":"2024-04-26T23:57:06.083098+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:04.383866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087484+0000","last_change":"2024-04-26T23:57:05.077681+0000","last_active":"2024-04-26T23:57:05.087484+0000","last_peered":"2024-04-26T23:57:05.087484+0000","last_clean":"2024-04-26T23:57:05.087484+0000","last_became_active":"2024-04-26T23:57:05.072588+0000","last_became_peered":"2024-04-26T23:57:05.072588+0000","last_unstale":"2024-04-26T23:57:05.087484+0000","last_undegraded":"2024-04-26T23:57:05.087484+0000","last_fullsized":"2024-04-26T23:57:05.087484+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:09.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.073746+0000","last_change":"2024-04-26T23:57:04.063932+0000","last_active":"2024-04-26T23:57:05.073746+0000","last_peered":"2024-04-26T23:57:05.073746+0000","last_clean":"2024-04-26T23:57:05.073746+0000","last_became_active":"2024-04-26T23:57:04.062057+0000","last_became_peered":"2024-04-26T23:57:04.062057+0000","last_unstale":"2024-04-26T23:57:05.073746+0000","last_undegraded":"2024-04-26T23:57:05.073746+0000","last_fullsized":"2024-04-26T23:57:05.073746+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:33.517599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093316+0000","last_change":"2024-04-26T23:57:05.089010+0000","last_active":"2024-04-26T23:57:05.093316+0000","last_peered":"2024-04-26T23:57:05.093316+0000","last_clean":"2024-04-26T23:57:05.093316+0000","last_became_active":"2024-04-26T23:57:05.088726+0000","last_became_peered":"2024-04-26T23:57:05.088726+0000","last_unstale":"2024-04-26T23:57:05.093316+0000","last_undegraded":"2024-04-26T23:57:05.093316+0000","last_fullsized":"2024-04-26T23:57:05.093316+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:23.197948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.071189+0000","last_change":"2024-04-26T23:57:04.063636+0000","last_active":"2024-04-26T23:57:05.071189+0000","last_peered":"2024-04-26T23:57:05.071189+0000","last_clean":"2024-04-26T23:57:05.071189+0000","last_became_active":"2024-04-26T23:57:04.063308+0000","last_became_peered":"2024-04-26T23:57:04.063308+0000","last_unstale":"2024-04-26T23:57:05.071189+0000","last_undegraded":"2024-04-26T23:57:05.071189+0000","last_fullsized":"2024-04-26T23:57:05.071189+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:50.629440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084460+0000","last_change":"2024-04-26T23:57:05.083345+0000","last_active":"2024-04-26T23:57:06.084460+0000","last_peered":"2024-04-26T23:57:06.084460+0000","last_clean":"2024-04-26T23:57:06.084460+0000","last_became_active":"2024-04-26T23:57:05.082969+0000","last_became_peered":"2024-04-26T23:57:05.082969+0000","last_unstale":"2024-04-26T23:57:06.084460+0000","last_undegraded":"2024-04-26T23:57:06.084460+0000","last_fullsized":"2024-04-26T23:57:06.084460+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:41.644183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081164+0000","last_change":"2024-04-26T23:57:04.059376+0000","last_active":"2024-04-26T23:57:06.081164+0000","last_peered":"2024-04-26T23:57:06.081164+0000","last_clean":"2024-04-26T23:57:06.081164+0000","last_became_active":"2024-04-26T23:57:04.059067+0000","last_became_peered":"2024-04-26T23:57:04.059067+0000","last_unstale":"2024-04-26T23:57:06.081164+0000","last_undegraded":"2024-04-26T23:57:06.081164+0000","last_fullsized":"2024-04-26T23:57:06.081164+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:15.839804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.085835+0000","last_change":"2024-04-26T23:57:05.076351+0000","last_active":"2024-04-26T23:57:05.085835+0000","last_peered":"2024-04-26T23:57:05.085835+0000","last_clean":"2024-04-26T23:57:05.085835+0000","last_became_active":"2024-04-26T23:57:05.076040+0000","last_became_peered":"2024-04-26T23:57:05.076040+0000","last_unstale":"2024-04-26T23:57:05.085835+0000","last_undegraded":"2024-04-26T23:57:05.085835+0000","last_fullsized":"2024-04-26T23:57:05.085835+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:45.999550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082281+0000","last_change":"2024-04-26T23:57:04.050717+0000","last_active":"2024-04-26T23:57:06.082281+0000","last_peered":"2024-04-26T23:57:06.082281+0000","last_clean":"2024-04-26T23:57:06.082281+0000","last_became_active":"2024-04-26T23:57:04.050323+0000","last_became_peered":"2024-04-26T23:57:04.050323+0000","last_unstale":"2024-04-26T23:57:06.082281+0000","last_undegraded":"2024-04-26T23:57:06.082281+0000","last_fullsized":"2024-04-26T23:57:06.082281+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:29.535894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":522,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.081061+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:57:05.081061+0000","last_peered":"2024-04-26T23:57:05.081061+0000","last_clean":"2024-04-26T23:57:05.081061+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:57:05.081061+0000","last_undegraded":"2024-04-26T23:57:05.081061+0000","last_fullsized":"2024-04-26T23:57:05.081061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56560,"kb_used_data":3172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315280,"statfs":{"total":96636764160,"available":96578846720,"internally_reserved":0,"allocated":3248128,"data_stored":1841299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575092,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55980,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315860,"statfs":{"total":96636764160,"available":96579440640,"internally_reserved":0,"allocated":2654208,"data_stored":1250919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575092,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55992,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315848,"statfs":{"total":96636764160,"available":96579428352,"internally_reserved":0,"allocated":2658304,"data_stored":1257499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60140,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311700,"statfs":{"total":96636764160,"available":96575180800,"internally_reserved":0,"allocated":2650112,"data_stored":1250877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47676,"kb_used_data":2608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324164,"statfs":{"total":96636764160,"available":96587943936,"internally_reserved":0,"allocated":2670592,"data_stored":1253057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64304,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307536,"statfs":{"total":96636764160,"available":96570916864,"internally_reserved":0,"allocated":2654208,"data_stored":1251459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575092,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2584,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2646016,"data_stored":1250807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575092,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64892,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306948,"statfs":{"total":96636764160,"available":96570314752,"internally_reserved":0,"allocated":3252224,"data_stored":1842787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:57:09.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:57:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.799+0000 7efe96000640 1 Processor -- start 2024-04-26T23:57:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.799+0000 7efe96000640 1 -- start start 2024-04-26T23:57:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe96000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe900720e0 0x7efe900cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe96000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe90160910 0x7efe90160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe96000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe90073ad0 con 0x7efe901466b0 2024-04-26T23:57:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe96000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7efe901612a0 con 0x7efe900720e0 2024-04-26T23:57:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe96000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efe90146a90 con 0x7efe90160910 2024-04-26T23:57:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.801+0000 7efe94c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efe901466b0 0x7efe9006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37354/0 (socket says 172.21.15.46:37354) 2024-04-26T23:57:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe94c00640 1 -- 172.21.15.46:0/10226631 learned_addr learned my addr 172.21.15.46:0/10226631 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe95600640 1 --2- 172.21.15.46:0/10226631 >> v2:172.21.15.142:3300/0 conn(0x7efe90160910 0x7efe90160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8fe00640 1 --2- 172.21.15.46:0/10226631 >> v2:172.21.15.46:3300/0 conn(0x7efe900720e0 0x7efe900cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8f400640 1 -- 172.21.15.46:0/10226631 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516794386 0 0) 0x7efe90073ad0 con 0x7efe901466b0 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8f400640 1 -- 172.21.15.46:0/10226631 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe78003520 con 0x7efe901466b0 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8fe00640 1 -- 172.21.15.46:0/10226631 >> v2:172.21.15.142:3300/0 conn(0x7efe90160910 msgr2=0x7efe90160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8fe00640 1 --2- 172.21.15.46:0/10226631 >> v2:172.21.15.142:3300/0 conn(0x7efe90160910 0x7efe90160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8fe00640 1 -- 172.21.15.46:0/10226631 >> v1:172.21.15.46:6789/0 conn(0x7efe901466b0 legacy=0x7efe9006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.802+0000 7efe8fe00640 1 -- 172.21.15.46:0/10226631 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe90073d70 con 0x7efe900720e0 2024-04-26T23:57:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.804+0000 7efe8fe00640 1 --2- 172.21.15.46:0/10226631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe900720e0 0x7efe900cd860 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0097a0 tx=0x7efe7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af5946cb5f274e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.804+0000 7efe8f400640 1 -- 172.21.15.46:0/10226631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c037070 con 0x7efe900720e0 2024-04-26T23:57:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.804+0000 7efe8f400640 1 -- 172.21.15.46:0/10226631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c032440 con 0x7efe900720e0 2024-04-26T23:57:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.804+0000 7efe8f400640 1 -- 172.21.15.46:0/10226631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0327f0 con 0x7efe900720e0 2024-04-26T23:57:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 -- 172.21.15.46:0/10226631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe900720e0 msgr2=0x7efe900cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 --2- 172.21.15.46:0/10226631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe900720e0 0x7efe900cd860 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0097a0 tx=0x7efe7c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 -- 172.21.15.46:0/10226631 shutdown_connections 2024-04-26T23:57:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 --2- 172.21.15.46:0/10226631 >> v2:172.21.15.142:3300/0 conn(0x7efe90160910 0x7efe90160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 --2- 172.21.15.46:0/10226631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe900720e0 0x7efe900cd860 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 -- 172.21.15.46:0/10226631 >> 172.21.15.46:0/10226631 conn(0x7efe900b26d0 msgr2=0x7efe900b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 -- 172.21.15.46:0/10226631 shutdown_connections 2024-04-26T23:57:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.805+0000 7efe96000640 1 -- 172.21.15.46:0/10226631 wait complete. 2024-04-26T23:57:09.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.807+0000 7efe96000640 1 Processor -- start 2024-04-26T23:57:09.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.807+0000 7efe96000640 1 -- start start 2024-04-26T23:57:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 0x7efe9013c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 0x7efe90151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe900cb470 con 0x7efe901466b0 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe900cb190 con 0x7efe90160910 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe96000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efe900cdda0 con 0x7efe900720e0 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35554/0 (socket says 172.21.15.46:35554) 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe8fe00640 1 -- 172.21.15.46:0/2219849777 learned_addr learned my addr 172.21.15.46:0/2219849777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe95600640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 0x7efe90151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe94c00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 0x7efe9013c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.808+0000 7efe8fe00640 1 -- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 msgr2=0x7efe9013c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8fe00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 0x7efe9013c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8fe00640 1 -- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 msgr2=0x7efe90151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8fe00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 0x7efe90151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8fe00640 1 -- 172.21.15.46:0/2219849777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9013caa0 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe95600640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 0x7efe90151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe94c00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 0x7efe9013c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8fe00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02a220 tx=0x7efe7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c037040 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c042070 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe900af930 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c03b930 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.809+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe900afef0 con 0x7efe901466b0 2024-04-26T23:57:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.811+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe900693b0 con 0x7efe901466b0 2024-04-26T23:57:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.811+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe7c03bad0 con 0x7efe901466b0 2024-04-26T23:57:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.812+0000 7efe8cc00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.812+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7efe7c0ac580 con 0x7efe901466b0 2024-04-26T23:57:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.813+0000 7efe94c00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.814+0000 7efe94c00640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 0x7efe7006edb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efe84005880 tx=0x7efe84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:09.817+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe9013caa0 con 0x7efe901466b0 2024-04-26T23:57:10.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.017+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe900d0660 con 0x7efe7006c8f0 2024-04-26T23:57:10.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.036+0000 7efe8cc00640 1 -- 172.21.15.46:0/2219849777 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252441 (secure 0 0 0) 0x7efe900d0660 con 0x7efe7006c8f0 2024-04-26T23:57:10.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.041+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 msgr2=0x7efe7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.041+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 0x7efe7006edb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efe84005880 tx=0x7efe84009040 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 msgr2=0x7efe900b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02a220 tx=0x7efe7c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 shutdown_connections 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe90160910 0x7efe90151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901466b0 0x7efe900b3be0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 --2- 172.21.15.46:0/2219849777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe900720e0 0x7efe9013c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 >> 172.21.15.46:0/2219849777 conn(0x7efe900b26d0 msgr2=0x7efe9006de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 shutdown_connections 2024-04-26T23:57:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.042+0000 7efe96000640 1 -- 172.21.15.46:0/2219849777 wait complete. 2024-04-26T23:57:10.062 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":807,"stamp":"2024-04-26T23:57:08.340163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453200,"kb_used_data":21908,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754521520,"statfs":{"total":773094113280,"available":772630036480,"internally_reserved":0,"allocated":22433792,"data_stored":11198704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":441503281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.323023"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090511+0000","last_change":"2024-04-26T23:57:05.085398+0000","last_active":"2024-04-26T23:57:05.090511+0000","last_peered":"2024-04-26T23:57:05.090511+0000","last_clean":"2024-04-26T23:57:05.090511+0000","last_became_active":"2024-04-26T23:57:05.084309+0000","last_became_peered":"2024-04-26T23:57:05.084309+0000","last_unstale":"2024-04-26T23:57:05.090511+0000","last_undegraded":"2024-04-26T23:57:05.090511+0000","last_fullsized":"2024-04-26T23:57:05.090511+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:53.503417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093225+0000","last_change":"2024-04-26T23:57:05.085968+0000","last_active":"2024-04-26T23:57:05.093225+0000","last_peered":"2024-04-26T23:57:05.093225+0000","last_clean":"2024-04-26T23:57:05.093225+0000","last_became_active":"2024-04-26T23:57:05.085639+0000","last_became_peered":"2024-04-26T23:57:05.085639+0000","last_unstale":"2024-04-26T23:57:05.093225+0000","last_undegraded":"2024-04-26T23:57:05.093225+0000","last_fullsized":"2024-04-26T23:57:05.093225+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:46.086737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094069+0000","last_change":"2024-04-26T23:57:05.089249+0000","last_active":"2024-04-26T23:57:05.094069+0000","last_peered":"2024-04-26T23:57:05.094069+0000","last_clean":"2024-04-26T23:57:05.094069+0000","last_became_active":"2024-04-26T23:57:05.088947+0000","last_became_peered":"2024-04-26T23:57:05.088947+0000","last_unstale":"2024-04-26T23:57:05.094069+0000","last_undegraded":"2024-04-26T23:57:05.094069+0000","last_fullsized":"2024-04-26T23:57:05.094069+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.086906+0000","last_change":"2024-04-26T23:57:05.079820+0000","last_active":"2024-04-26T23:57:05.086906+0000","last_peered":"2024-04-26T23:57:05.086906+0000","last_clean":"2024-04-26T23:57:05.086906+0000","last_became_active":"2024-04-26T23:57:05.079533+0000","last_became_peered":"2024-04-26T23:57:05.079533+0000","last_unstale":"2024-04-26T23:57:05.086906+0000","last_undegraded":"2024-04-26T23:57:05.086906+0000","last_fullsized":"2024-04-26T23:57:05.086906+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.470427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.089698+0000","last_change":"2024-04-26T23:57:05.086275+0000","last_active":"2024-04-26T23:57:05.089698+0000","last_peered":"2024-04-26T23:57:05.089698+0000","last_clean":"2024-04-26T23:57:05.089698+0000","last_became_active":"2024-04-26T23:57:05.085833+0000","last_became_peered":"2024-04-26T23:57:05.085833+0000","last_unstale":"2024-04-26T23:57:05.089698+0000","last_undegraded":"2024-04-26T23:57:05.089698+0000","last_fullsized":"2024-04-26T23:57:05.089698+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:17.959749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098924+0000","last_change":"2024-04-26T23:57:05.094461+0000","last_active":"2024-04-26T23:57:05.098924+0000","last_peered":"2024-04-26T23:57:05.098924+0000","last_clean":"2024-04-26T23:57:05.098924+0000","last_became_active":"2024-04-26T23:57:05.094179+0000","last_became_peered":"2024-04-26T23:57:05.094179+0000","last_unstale":"2024-04-26T23:57:05.098924+0000","last_undegraded":"2024-04-26T23:57:05.098924+0000","last_fullsized":"2024-04-26T23:57:05.098924+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:24.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080895+0000","last_change":"2024-04-26T23:57:05.076217+0000","last_active":"2024-04-26T23:57:06.080895+0000","last_peered":"2024-04-26T23:57:06.080895+0000","last_clean":"2024-04-26T23:57:06.080895+0000","last_became_active":"2024-04-26T23:57:05.075790+0000","last_became_peered":"2024-04-26T23:57:05.075790+0000","last_unstale":"2024-04-26T23:57:06.080895+0000","last_undegraded":"2024-04-26T23:57:06.080895+0000","last_fullsized":"2024-04-26T23:57:06.080895+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090952+0000","last_change":"2024-04-26T23:57:05.089835+0000","last_active":"2024-04-26T23:57:05.090952+0000","last_peered":"2024-04-26T23:57:05.090952+0000","last_clean":"2024-04-26T23:57:05.090952+0000","last_became_active":"2024-04-26T23:57:05.089489+0000","last_became_peered":"2024-04-26T23:57:05.089489+0000","last_unstale":"2024-04-26T23:57:05.090952+0000","last_undegraded":"2024-04-26T23:57:05.090952+0000","last_fullsized":"2024-04-26T23:57:05.090952+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:37.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099803+0000","last_change":"2024-04-26T23:57:05.098470+0000","last_active":"2024-04-26T23:57:05.099803+0000","last_peered":"2024-04-26T23:57:05.099803+0000","last_clean":"2024-04-26T23:57:05.099803+0000","last_became_active":"2024-04-26T23:57:05.098173+0000","last_became_peered":"2024-04-26T23:57:05.098173+0000","last_unstale":"2024-04-26T23:57:05.099803+0000","last_undegraded":"2024-04-26T23:57:05.099803+0000","last_fullsized":"2024-04-26T23:57:05.099803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:25.524521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094549+0000","last_change":"2024-04-26T23:57:05.092389+0000","last_active":"2024-04-26T23:57:05.094549+0000","last_peered":"2024-04-26T23:57:05.094549+0000","last_clean":"2024-04-26T23:57:05.094549+0000","last_became_active":"2024-04-26T23:57:05.092095+0000","last_became_peered":"2024-04-26T23:57:05.092095+0000","last_unstale":"2024-04-26T23:57:05.094549+0000","last_undegraded":"2024-04-26T23:57:05.094549+0000","last_fullsized":"2024-04-26T23:57:05.094549+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:57.197392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095212+0000","last_change":"2024-04-26T23:57:05.093199+0000","last_active":"2024-04-26T23:57:05.095212+0000","last_peered":"2024-04-26T23:57:05.095212+0000","last_clean":"2024-04-26T23:57:05.095212+0000","last_became_active":"2024-04-26T23:57:05.092916+0000","last_became_peered":"2024-04-26T23:57:05.092916+0000","last_unstale":"2024-04-26T23:57:05.095212+0000","last_undegraded":"2024-04-26T23:57:05.095212+0000","last_fullsized":"2024-04-26T23:57:05.095212+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:10.935614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.078237+0000","last_change":"2024-04-26T23:57:05.072630+0000","last_active":"2024-04-26T23:57:05.078237+0000","last_peered":"2024-04-26T23:57:05.078237+0000","last_clean":"2024-04-26T23:57:05.078237+0000","last_became_active":"2024-04-26T23:57:05.072294+0000","last_became_peered":"2024-04-26T23:57:05.072294+0000","last_unstale":"2024-04-26T23:57:05.078237+0000","last_undegraded":"2024-04-26T23:57:05.078237+0000","last_fullsized":"2024-04-26T23:57:05.078237+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:15.405571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083157+0000","last_change":"2024-04-26T23:57:05.080701+0000","last_active":"2024-04-26T23:57:06.083157+0000","last_peered":"2024-04-26T23:57:06.083157+0000","last_clean":"2024-04-26T23:57:06.083157+0000","last_became_active":"2024-04-26T23:57:05.080369+0000","last_became_peered":"2024-04-26T23:57:05.080369+0000","last_unstale":"2024-04-26T23:57:06.083157+0000","last_undegraded":"2024-04-26T23:57:06.083157+0000","last_fullsized":"2024-04-26T23:57:06.083157+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:42:17.959749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077790+0000","last_change":"2024-04-26T23:57:05.066410+0000","last_active":"2024-04-26T23:57:06.077790+0000","last_peered":"2024-04-26T23:57:06.077790+0000","last_clean":"2024-04-26T23:57:06.077790+0000","last_became_active":"2024-04-26T23:57:05.066018+0000","last_became_peered":"2024-04-26T23:57:05.066018+0000","last_unstale":"2024-04-26T23:57:06.077790+0000","last_undegraded":"2024-04-26T23:57:06.077790+0000","last_fullsized":"2024-04-26T23:57:06.077790+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:26.188183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096336+0000","last_change":"2024-04-26T23:57:05.094305+0000","last_active":"2024-04-26T23:57:05.096336+0000","last_peered":"2024-04-26T23:57:05.096336+0000","last_clean":"2024-04-26T23:57:05.096336+0000","last_became_active":"2024-04-26T23:57:05.093724+0000","last_became_peered":"2024-04-26T23:57:05.093724+0000","last_unstale":"2024-04-26T23:57:05.096336+0000","last_undegraded":"2024-04-26T23:57:05.096336+0000","last_fullsized":"2024-04-26T23:57:05.096336+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.092882+0000","last_change":"2024-04-26T23:57:05.083768+0000","last_active":"2024-04-26T23:57:05.092882+0000","last_peered":"2024-04-26T23:57:05.092882+0000","last_clean":"2024-04-26T23:57:05.092882+0000","last_became_active":"2024-04-26T23:57:05.081409+0000","last_became_peered":"2024-04-26T23:57:05.081409+0000","last_unstale":"2024-04-26T23:57:05.092882+0000","last_undegraded":"2024-04-26T23:57:05.092882+0000","last_fullsized":"2024-04-26T23:57:05.092882+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084783+0000","last_change":"2024-04-26T23:57:05.087854+0000","last_active":"2024-04-26T23:57:06.084783+0000","last_peered":"2024-04-26T23:57:06.084783+0000","last_clean":"2024-04-26T23:57:06.084783+0000","last_became_active":"2024-04-26T23:57:05.087553+0000","last_became_peered":"2024-04-26T23:57:05.087553+0000","last_unstale":"2024-04-26T23:57:06.084783+0000","last_undegraded":"2024-04-26T23:57:06.084783+0000","last_fullsized":"2024-04-26T23:57:06.084783+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:37.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094525+0000","last_change":"2024-04-26T23:57:05.086399+0000","last_active":"2024-04-26T23:57:05.094525+0000","last_peered":"2024-04-26T23:57:05.094525+0000","last_clean":"2024-04-26T23:57:05.094525+0000","last_became_active":"2024-04-26T23:57:05.086101+0000","last_became_peered":"2024-04-26T23:57:05.086101+0000","last_unstale":"2024-04-26T23:57:05.094525+0000","last_undegraded":"2024-04-26T23:57:05.094525+0000","last_fullsized":"2024-04-26T23:57:05.094525+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:21.289106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096400+0000","last_change":"2024-04-26T23:57:05.094900+0000","last_active":"2024-04-26T23:57:05.096400+0000","last_peered":"2024-04-26T23:57:05.096400+0000","last_clean":"2024-04-26T23:57:05.096400+0000","last_became_active":"2024-04-26T23:57:05.094587+0000","last_became_peered":"2024-04-26T23:57:05.094587+0000","last_unstale":"2024-04-26T23:57:05.096400+0000","last_undegraded":"2024-04-26T23:57:05.096400+0000","last_fullsized":"2024-04-26T23:57:05.096400+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094263+0000","last_change":"2024-04-26T23:57:05.089125+0000","last_active":"2024-04-26T23:57:05.094263+0000","last_peered":"2024-04-26T23:57:05.094263+0000","last_clean":"2024-04-26T23:57:05.094263+0000","last_became_active":"2024-04-26T23:57:05.088784+0000","last_became_peered":"2024-04-26T23:57:05.088784+0000","last_unstale":"2024-04-26T23:57:05.094263+0000","last_undegraded":"2024-04-26T23:57:05.094263+0000","last_fullsized":"2024-04-26T23:57:05.094263+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:27.080832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080667+0000","last_change":"2024-04-26T23:57:05.088432+0000","last_active":"2024-04-26T23:57:06.080667+0000","last_peered":"2024-04-26T23:57:06.080667+0000","last_clean":"2024-04-26T23:57:06.080667+0000","last_became_active":"2024-04-26T23:57:05.088162+0000","last_became_peered":"2024-04-26T23:57:05.088162+0000","last_unstale":"2024-04-26T23:57:06.080667+0000","last_undegraded":"2024-04-26T23:57:06.080667+0000","last_fullsized":"2024-04-26T23:57:06.080667+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.470427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083116+0000","last_change":"2024-04-26T23:57:05.083617+0000","last_active":"2024-04-26T23:57:06.083116+0000","last_peered":"2024-04-26T23:57:06.083116+0000","last_clean":"2024-04-26T23:57:06.083116+0000","last_became_active":"2024-04-26T23:57:05.083328+0000","last_became_peered":"2024-04-26T23:57:05.083328+0000","last_unstale":"2024-04-26T23:57:06.083116+0000","last_undegraded":"2024-04-26T23:57:06.083116+0000","last_fullsized":"2024-04-26T23:57:06.083116+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:26.695978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096316+0000","last_change":"2024-04-26T23:57:05.093708+0000","last_active":"2024-04-26T23:57:05.096316+0000","last_peered":"2024-04-26T23:57:05.096316+0000","last_clean":"2024-04-26T23:57:05.096316+0000","last_became_active":"2024-04-26T23:57:05.093377+0000","last_became_peered":"2024-04-26T23:57:05.093377+0000","last_unstale":"2024-04-26T23:57:05.096316+0000","last_undegraded":"2024-04-26T23:57:05.096316+0000","last_fullsized":"2024-04-26T23:57:05.096316+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080540+0000","last_change":"2024-04-26T23:57:05.075657+0000","last_active":"2024-04-26T23:57:06.080540+0000","last_peered":"2024-04-26T23:57:06.080540+0000","last_clean":"2024-04-26T23:57:06.080540+0000","last_became_active":"2024-04-26T23:57:05.075338+0000","last_became_peered":"2024-04-26T23:57:05.075338+0000","last_unstale":"2024-04-26T23:57:06.080540+0000","last_undegraded":"2024-04-26T23:57:06.080540+0000","last_fullsized":"2024-04-26T23:57:06.080540+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:08.193361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.086510+0000","last_change":"2024-04-26T23:57:05.078784+0000","last_active":"2024-04-26T23:57:05.086510+0000","last_peered":"2024-04-26T23:57:05.086510+0000","last_clean":"2024-04-26T23:57:05.086510+0000","last_became_active":"2024-04-26T23:57:05.078483+0000","last_became_peered":"2024-04-26T23:57:05.078483+0000","last_unstale":"2024-04-26T23:57:05.086510+0000","last_undegraded":"2024-04-26T23:57:05.086510+0000","last_fullsized":"2024-04-26T23:57:05.086510+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:56.176146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082011+0000","last_change":"2024-04-26T23:57:05.079690+0000","last_active":"2024-04-26T23:57:06.082011+0000","last_peered":"2024-04-26T23:57:06.082011+0000","last_clean":"2024-04-26T23:57:06.082011+0000","last_became_active":"2024-04-26T23:57:05.079400+0000","last_became_peered":"2024-04-26T23:57:05.079400+0000","last_unstale":"2024-04-26T23:57:06.082011+0000","last_undegraded":"2024-04-26T23:57:06.082011+0000","last_fullsized":"2024-04-26T23:57:06.082011+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:13.735126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090981+0000","last_change":"2024-04-26T23:57:05.089934+0000","last_active":"2024-04-26T23:57:05.090981+0000","last_peered":"2024-04-26T23:57:05.090981+0000","last_clean":"2024-04-26T23:57:05.090981+0000","last_became_active":"2024-04-26T23:57:05.089618+0000","last_became_peered":"2024-04-26T23:57:05.089618+0000","last_unstale":"2024-04-26T23:57:05.090981+0000","last_undegraded":"2024-04-26T23:57:05.090981+0000","last_fullsized":"2024-04-26T23:57:05.090981+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:49.957221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099645+0000","last_change":"2024-04-26T23:57:05.096391+0000","last_active":"2024-04-26T23:57:05.099645+0000","last_peered":"2024-04-26T23:57:05.099645+0000","last_clean":"2024-04-26T23:57:05.099645+0000","last_became_active":"2024-04-26T23:57:05.096100+0000","last_became_peered":"2024-04-26T23:57:05.096100+0000","last_unstale":"2024-04-26T23:57:05.099645+0000","last_undegraded":"2024-04-26T23:57:05.099645+0000","last_fullsized":"2024-04-26T23:57:05.099645+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.897714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077990+0000","last_change":"2024-04-26T23:57:05.069929+0000","last_active":"2024-04-26T23:57:06.077990+0000","last_peered":"2024-04-26T23:57:06.077990+0000","last_clean":"2024-04-26T23:57:06.077990+0000","last_became_active":"2024-04-26T23:57:05.069607+0000","last_became_peered":"2024-04-26T23:57:05.069607+0000","last_unstale":"2024-04-26T23:57:06.077990+0000","last_undegraded":"2024-04-26T23:57:06.077990+0000","last_fullsized":"2024-04-26T23:57:06.077990+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:12.409228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077460+0000","last_change":"2024-04-26T23:57:05.087321+0000","last_active":"2024-04-26T23:57:06.077460+0000","last_peered":"2024-04-26T23:57:06.077460+0000","last_clean":"2024-04-26T23:57:06.077460+0000","last_became_active":"2024-04-26T23:57:05.087022+0000","last_became_peered":"2024-04-26T23:57:05.087022+0000","last_unstale":"2024-04-26T23:57:06.077460+0000","last_undegraded":"2024-04-26T23:57:06.077460+0000","last_fullsized":"2024-04-26T23:57:06.077460+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:42.112797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095073+0000","last_change":"2024-04-26T23:57:05.093090+0000","last_active":"2024-04-26T23:57:05.095073+0000","last_peered":"2024-04-26T23:57:05.095073+0000","last_clean":"2024-04-26T23:57:05.095073+0000","last_became_active":"2024-04-26T23:57:05.091823+0000","last_became_peered":"2024-04-26T23:57:05.091823+0000","last_unstale":"2024-04-26T23:57:05.095073+0000","last_undegraded":"2024-04-26T23:57:05.095073+0000","last_fullsized":"2024-04-26T23:57:05.095073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:39.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099734+0000","last_change":"2024-04-26T23:57:05.097991+0000","last_active":"2024-04-26T23:57:05.099734+0000","last_peered":"2024-04-26T23:57:05.099734+0000","last_clean":"2024-04-26T23:57:05.099734+0000","last_became_active":"2024-04-26T23:57:05.095679+0000","last_became_peered":"2024-04-26T23:57:05.095679+0000","last_unstale":"2024-04-26T23:57:05.099734+0000","last_undegraded":"2024-04-26T23:57:05.099734+0000","last_fullsized":"2024-04-26T23:57:05.099734+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:40.282754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090749+0000","last_change":"2024-04-26T23:57:05.085630+0000","last_active":"2024-04-26T23:57:05.090749+0000","last_peered":"2024-04-26T23:57:05.090749+0000","last_clean":"2024-04-26T23:57:05.090749+0000","last_became_active":"2024-04-26T23:57:05.083187+0000","last_became_peered":"2024-04-26T23:57:05.083187+0000","last_unstale":"2024-04-26T23:57:05.090749+0000","last_undegraded":"2024-04-26T23:57:05.090749+0000","last_fullsized":"2024-04-26T23:57:05.090749+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:13.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"209'4","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099168+0000","last_change":"2024-04-26T23:57:04.056445+0000","last_active":"2024-04-26T23:57:05.099168+0000","last_peered":"2024-04-26T23:57:05.099168+0000","last_clean":"2024-04-26T23:57:05.099168+0000","last_became_active":"2024-04-26T23:57:04.056166+0000","last_became_peered":"2024-04-26T23:57:04.056166+0000","last_unstale":"2024-04-26T23:57:05.099168+0000","last_undegraded":"2024-04-26T23:57:05.099168+0000","last_fullsized":"2024-04-26T23:57:05.099168+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:42.833864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098034+0000","last_change":"2024-04-26T23:57:05.097233+0000","last_active":"2024-04-26T23:57:05.098034+0000","last_peered":"2024-04-26T23:57:05.098034+0000","last_clean":"2024-04-26T23:57:05.098034+0000","last_became_active":"2024-04-26T23:57:05.096935+0000","last_became_peered":"2024-04-26T23:57:05.096935+0000","last_unstale":"2024-04-26T23:57:05.098034+0000","last_undegraded":"2024-04-26T23:57:05.098034+0000","last_fullsized":"2024-04-26T23:57:05.098034+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:21.289106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"209'2","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083973+0000","last_change":"2024-04-26T23:57:04.063135+0000","last_active":"2024-04-26T23:57:06.083973+0000","last_peered":"2024-04-26T23:57:06.083973+0000","last_clean":"2024-04-26T23:57:06.083973+0000","last_became_active":"2024-04-26T23:57:04.062804+0000","last_became_peered":"2024-04-26T23:57:04.062804+0000","last_unstale":"2024-04-26T23:57:06.083973+0000","last_undegraded":"2024-04-26T23:57:06.083973+0000","last_fullsized":"2024-04-26T23:57:06.083973+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:40.465823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095843+0000","last_change":"2024-04-26T23:57:05.090168+0000","last_active":"2024-04-26T23:57:05.095843+0000","last_peered":"2024-04-26T23:57:05.095843+0000","last_clean":"2024-04-26T23:57:05.095843+0000","last_became_active":"2024-04-26T23:57:05.089909+0000","last_became_peered":"2024-04-26T23:57:05.089909+0000","last_unstale":"2024-04-26T23:57:05.095843+0000","last_undegraded":"2024-04-26T23:57:05.095843+0000","last_fullsized":"2024-04-26T23:57:05.095843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:08.172802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"209'2","reported_seq":16,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078038+0000","last_change":"2024-04-26T23:57:04.050049+0000","last_active":"2024-04-26T23:57:06.078038+0000","last_peered":"2024-04-26T23:57:06.078038+0000","last_clean":"2024-04-26T23:57:06.078038+0000","last_became_active":"2024-04-26T23:57:04.048921+0000","last_became_peered":"2024-04-26T23:57:04.048921+0000","last_unstale":"2024-04-26T23:57:06.078038+0000","last_undegraded":"2024-04-26T23:57:06.078038+0000","last_fullsized":"2024-04-26T23:57:06.078038+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:39.590188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077935+0000","last_change":"2024-04-26T23:57:05.079620+0000","last_active":"2024-04-26T23:57:06.077935+0000","last_peered":"2024-04-26T23:57:06.077935+0000","last_clean":"2024-04-26T23:57:06.077935+0000","last_became_active":"2024-04-26T23:57:05.079311+0000","last_became_peered":"2024-04-26T23:57:05.079311+0000","last_unstale":"2024-04-26T23:57:06.077935+0000","last_undegraded":"2024-04-26T23:57:06.077935+0000","last_fullsized":"2024-04-26T23:57:06.077935+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:19.264558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"209'1","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093894+0000","last_change":"2024-04-26T23:57:04.046528+0000","last_active":"2024-04-26T23:57:05.093894+0000","last_peered":"2024-04-26T23:57:05.093894+0000","last_clean":"2024-04-26T23:57:05.093894+0000","last_became_active":"2024-04-26T23:57:04.046146+0000","last_became_peered":"2024-04-26T23:57:04.046146+0000","last_unstale":"2024-04-26T23:57:05.093894+0000","last_undegraded":"2024-04-26T23:57:05.093894+0000","last_fullsized":"2024-04-26T23:57:05.093894+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081671+0000","last_change":"2024-04-26T23:57:05.088567+0000","last_active":"2024-04-26T23:57:06.081671+0000","last_peered":"2024-04-26T23:57:06.081671+0000","last_clean":"2024-04-26T23:57:06.081671+0000","last_became_active":"2024-04-26T23:57:05.088263+0000","last_became_peered":"2024-04-26T23:57:05.088263+0000","last_unstale":"2024-04-26T23:57:06.081671+0000","last_undegraded":"2024-04-26T23:57:06.081671+0000","last_fullsized":"2024-04-26T23:57:06.081671+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077674+0000","last_change":"2024-04-26T23:57:04.045360+0000","last_active":"2024-04-26T23:57:06.077674+0000","last_peered":"2024-04-26T23:57:06.077674+0000","last_clean":"2024-04-26T23:57:06.077674+0000","last_became_active":"2024-04-26T23:57:04.044970+0000","last_became_peered":"2024-04-26T23:57:04.044970+0000","last_unstale":"2024-04-26T23:57:06.077674+0000","last_undegraded":"2024-04-26T23:57:06.077674+0000","last_fullsized":"2024-04-26T23:57:06.077674+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087544+0000","last_change":"2024-04-26T23:57:05.081272+0000","last_active":"2024-04-26T23:57:05.087544+0000","last_peered":"2024-04-26T23:57:05.087544+0000","last_clean":"2024-04-26T23:57:05.087544+0000","last_became_active":"2024-04-26T23:57:05.080986+0000","last_became_peered":"2024-04-26T23:57:05.080986+0000","last_unstale":"2024-04-26T23:57:05.087544+0000","last_undegraded":"2024-04-26T23:57:05.087544+0000","last_fullsized":"2024-04-26T23:57:05.087544+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.075180+0000","last_change":"2024-04-26T23:57:04.052584+0000","last_active":"2024-04-26T23:57:05.075180+0000","last_peered":"2024-04-26T23:57:05.075180+0000","last_clean":"2024-04-26T23:57:05.075180+0000","last_became_active":"2024-04-26T23:57:04.052300+0000","last_became_peered":"2024-04-26T23:57:04.052300+0000","last_unstale":"2024-04-26T23:57:05.075180+0000","last_undegraded":"2024-04-26T23:57:05.075180+0000","last_fullsized":"2024-04-26T23:57:05.075180+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.149252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.090830+0000","last_change":"2024-04-26T23:57:05.088368+0000","last_active":"2024-04-26T23:57:05.090830+0000","last_peered":"2024-04-26T23:57:05.090830+0000","last_clean":"2024-04-26T23:57:05.090830+0000","last_became_active":"2024-04-26T23:57:05.088089+0000","last_became_peered":"2024-04-26T23:57:05.088089+0000","last_unstale":"2024-04-26T23:57:05.090830+0000","last_undegraded":"2024-04-26T23:57:05.090830+0000","last_fullsized":"2024-04-26T23:57:05.090830+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:41.644183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.057705+0000","last_change":"2024-04-26T23:57:04.053036+0000","last_active":"2024-04-26T23:57:05.057705+0000","last_peered":"2024-04-26T23:57:05.057705+0000","last_clean":"2024-04-26T23:57:05.057705+0000","last_became_active":"2024-04-26T23:57:04.052701+0000","last_became_peered":"2024-04-26T23:57:04.052701+0000","last_unstale":"2024-04-26T23:57:05.057705+0000","last_undegraded":"2024-04-26T23:57:05.057705+0000","last_fullsized":"2024-04-26T23:57:05.057705+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:44.516642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080427+0000","last_change":"2024-04-26T23:57:05.092053+0000","last_active":"2024-04-26T23:57:06.080427+0000","last_peered":"2024-04-26T23:57:06.080427+0000","last_clean":"2024-04-26T23:57:06.080427+0000","last_became_active":"2024-04-26T23:57:05.091789+0000","last_became_peered":"2024-04-26T23:57:05.091789+0000","last_unstale":"2024-04-26T23:57:06.080427+0000","last_undegraded":"2024-04-26T23:57:06.080427+0000","last_fullsized":"2024-04-26T23:57:06.080427+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:26.662514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078416+0000","last_change":"2024-04-26T23:57:04.053259+0000","last_active":"2024-04-26T23:57:06.078416+0000","last_peered":"2024-04-26T23:57:06.078416+0000","last_clean":"2024-04-26T23:57:06.078416+0000","last_became_active":"2024-04-26T23:57:04.052008+0000","last_became_peered":"2024-04-26T23:57:04.052008+0000","last_unstale":"2024-04-26T23:57:06.078416+0000","last_undegraded":"2024-04-26T23:57:06.078416+0000","last_fullsized":"2024-04-26T23:57:06.078416+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:11.453284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082359+0000","last_change":"2024-04-26T23:57:05.079198+0000","last_active":"2024-04-26T23:57:06.082359+0000","last_peered":"2024-04-26T23:57:06.082359+0000","last_clean":"2024-04-26T23:57:06.082359+0000","last_became_active":"2024-04-26T23:57:05.078660+0000","last_became_peered":"2024-04-26T23:57:05.078660+0000","last_unstale":"2024-04-26T23:57:06.082359+0000","last_undegraded":"2024-04-26T23:57:06.082359+0000","last_fullsized":"2024-04-26T23:57:06.082359+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:44.819513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.074507+0000","last_change":"2024-04-26T23:57:04.063553+0000","last_active":"2024-04-26T23:57:05.074507+0000","last_peered":"2024-04-26T23:57:05.074507+0000","last_clean":"2024-04-26T23:57:05.074507+0000","last_became_active":"2024-04-26T23:57:04.063114+0000","last_became_peered":"2024-04-26T23:57:04.063114+0000","last_unstale":"2024-04-26T23:57:05.074507+0000","last_undegraded":"2024-04-26T23:57:05.074507+0000","last_fullsized":"2024-04-26T23:57:05.074507+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:30.648721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.094131+0000","last_change":"2024-04-26T23:57:05.088479+0000","last_active":"2024-04-26T23:57:05.094131+0000","last_peered":"2024-04-26T23:57:05.094131+0000","last_clean":"2024-04-26T23:57:05.094131+0000","last_became_active":"2024-04-26T23:57:05.087773+0000","last_became_peered":"2024-04-26T23:57:05.087773+0000","last_unstale":"2024-04-26T23:57:05.094131+0000","last_undegraded":"2024-04-26T23:57:05.094131+0000","last_fullsized":"2024-04-26T23:57:05.094131+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"209'3","reported_seq":15,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096972+0000","last_change":"2024-04-26T23:57:04.053271+0000","last_active":"2024-04-26T23:57:05.096972+0000","last_peered":"2024-04-26T23:57:05.096972+0000","last_clean":"2024-04-26T23:57:05.096972+0000","last_became_active":"2024-04-26T23:57:04.052882+0000","last_became_peered":"2024-04-26T23:57:04.052882+0000","last_unstale":"2024-04-26T23:57:05.096972+0000","last_undegraded":"2024-04-26T23:57:05.096972+0000","last_fullsized":"2024-04-26T23:57:05.096972+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:01:43.802370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081300+0000","last_change":"2024-04-26T23:57:05.080628+0000","last_active":"2024-04-26T23:57:06.081300+0000","last_peered":"2024-04-26T23:57:06.081300+0000","last_clean":"2024-04-26T23:57:06.081300+0000","last_became_active":"2024-04-26T23:57:05.080332+0000","last_became_peered":"2024-04-26T23:57:05.080332+0000","last_unstale":"2024-04-26T23:57:06.081300+0000","last_undegraded":"2024-04-26T23:57:06.081300+0000","last_fullsized":"2024-04-26T23:57:06.081300+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:40.607332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"209'6","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.101081+0000","last_change":"2024-04-26T23:57:04.048030+0000","last_active":"2024-04-26T23:57:05.101081+0000","last_peered":"2024-04-26T23:57:05.101081+0000","last_clean":"2024-04-26T23:57:05.101081+0000","last_became_active":"2024-04-26T23:57:04.047721+0000","last_became_peered":"2024-04-26T23:57:04.047721+0000","last_unstale":"2024-04-26T23:57:05.101081+0000","last_undegraded":"2024-04-26T23:57:05.101081+0000","last_fullsized":"2024-04-26T23:57:05.101081+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:01.790838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087154+0000","last_change":"2024-04-26T23:57:05.078581+0000","last_active":"2024-04-26T23:57:05.087154+0000","last_peered":"2024-04-26T23:57:05.087154+0000","last_clean":"2024-04-26T23:57:05.087154+0000","last_became_active":"2024-04-26T23:57:05.078278+0000","last_became_peered":"2024-04-26T23:57:05.078278+0000","last_unstale":"2024-04-26T23:57:05.087154+0000","last_undegraded":"2024-04-26T23:57:05.087154+0000","last_fullsized":"2024-04-26T23:57:05.087154+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:33.487077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.099105+0000","last_change":"2024-04-26T23:57:04.049339+0000","last_active":"2024-04-26T23:57:05.099105+0000","last_peered":"2024-04-26T23:57:05.099105+0000","last_clean":"2024-04-26T23:57:05.099105+0000","last_became_active":"2024-04-26T23:57:04.048885+0000","last_became_peered":"2024-04-26T23:57:04.048885+0000","last_unstale":"2024-04-26T23:57:05.099105+0000","last_undegraded":"2024-04-26T23:57:05.099105+0000","last_fullsized":"2024-04-26T23:57:05.099105+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:25.645371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081803+0000","last_change":"2024-04-26T23:57:05.080283+0000","last_active":"2024-04-26T23:57:06.081803+0000","last_peered":"2024-04-26T23:57:06.081803+0000","last_clean":"2024-04-26T23:57:06.081803+0000","last_became_active":"2024-04-26T23:57:05.078092+0000","last_became_peered":"2024-04-26T23:57:05.078092+0000","last_unstale":"2024-04-26T23:57:06.081803+0000","last_undegraded":"2024-04-26T23:57:06.081803+0000","last_fullsized":"2024-04-26T23:57:06.081803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:22.476695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.069045+0000","last_change":"2024-04-26T23:57:04.053420+0000","last_active":"2024-04-26T23:57:05.069045+0000","last_peered":"2024-04-26T23:57:05.069045+0000","last_clean":"2024-04-26T23:57:05.069045+0000","last_became_active":"2024-04-26T23:57:04.053081+0000","last_became_peered":"2024-04-26T23:57:04.053081+0000","last_unstale":"2024-04-26T23:57:05.069045+0000","last_undegraded":"2024-04-26T23:57:05.069045+0000","last_fullsized":"2024-04-26T23:57:05.069045+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:14.330244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096284+0000","last_change":"2024-04-26T23:57:05.093868+0000","last_active":"2024-04-26T23:57:05.096284+0000","last_peered":"2024-04-26T23:57:05.096284+0000","last_clean":"2024-04-26T23:57:05.096284+0000","last_became_active":"2024-04-26T23:57:05.093556+0000","last_became_peered":"2024-04-26T23:57:05.093556+0000","last_unstale":"2024-04-26T23:57:05.096284+0000","last_undegraded":"2024-04-26T23:57:05.096284+0000","last_fullsized":"2024-04-26T23:57:05.096284+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.078086+0000","last_change":"2024-04-26T23:57:04.062067+0000","last_active":"2024-04-26T23:57:05.078086+0000","last_peered":"2024-04-26T23:57:05.078086+0000","last_clean":"2024-04-26T23:57:05.078086+0000","last_became_active":"2024-04-26T23:57:04.061669+0000","last_became_peered":"2024-04-26T23:57:04.061669+0000","last_unstale":"2024-04-26T23:57:05.078086+0000","last_undegraded":"2024-04-26T23:57:05.078086+0000","last_fullsized":"2024-04-26T23:57:05.078086+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:07.176218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097606+0000","last_change":"2024-04-26T23:57:05.096573+0000","last_active":"2024-04-26T23:57:05.097606+0000","last_peered":"2024-04-26T23:57:05.097606+0000","last_clean":"2024-04-26T23:57:05.097606+0000","last_became_active":"2024-04-26T23:57:05.095841+0000","last_became_peered":"2024-04-26T23:57:05.095841+0000","last_unstale":"2024-04-26T23:57:05.097606+0000","last_undegraded":"2024-04-26T23:57:05.097606+0000","last_fullsized":"2024-04-26T23:57:05.097606+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:47.477177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"209'8","reported_seq":20,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.091032+0000","last_change":"2024-04-26T23:57:04.051545+0000","last_active":"2024-04-26T23:57:05.091032+0000","last_peered":"2024-04-26T23:57:05.091032+0000","last_clean":"2024-04-26T23:57:05.091032+0000","last_became_active":"2024-04-26T23:57:04.051205+0000","last_became_peered":"2024-04-26T23:57:04.051205+0000","last_unstale":"2024-04-26T23:57:05.091032+0000","last_undegraded":"2024-04-26T23:57:05.091032+0000","last_fullsized":"2024-04-26T23:57:05.091032+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:00.843562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.096405+0000","last_change":"2024-04-26T23:57:05.095286+0000","last_active":"2024-04-26T23:57:05.096405+0000","last_peered":"2024-04-26T23:57:05.096405+0000","last_clean":"2024-04-26T23:57:05.096405+0000","last_became_active":"2024-04-26T23:57:05.094948+0000","last_became_peered":"2024-04-26T23:57:05.094948+0000","last_unstale":"2024-04-26T23:57:05.096405+0000","last_undegraded":"2024-04-26T23:57:05.096405+0000","last_fullsized":"2024-04-26T23:57:05.096405+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:17.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"209'1","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097410+0000","last_change":"2024-04-26T23:57:04.060733+0000","last_active":"2024-04-26T23:57:05.097410+0000","last_peered":"2024-04-26T23:57:05.097410+0000","last_clean":"2024-04-26T23:57:05.097410+0000","last_became_active":"2024-04-26T23:57:04.060349+0000","last_became_peered":"2024-04-26T23:57:04.060349+0000","last_unstale":"2024-04-26T23:57:05.097410+0000","last_undegraded":"2024-04-26T23:57:05.097410+0000","last_fullsized":"2024-04-26T23:57:05.097410+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:51.799965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095827+0000","last_change":"2024-04-26T23:57:05.086368+0000","last_active":"2024-04-26T23:57:05.095827+0000","last_peered":"2024-04-26T23:57:05.095827+0000","last_clean":"2024-04-26T23:57:05.095827+0000","last_became_active":"2024-04-26T23:57:05.086030+0000","last_became_peered":"2024-04-26T23:57:05.086030+0000","last_unstale":"2024-04-26T23:57:05.095827+0000","last_undegraded":"2024-04-26T23:57:05.095827+0000","last_fullsized":"2024-04-26T23:57:05.095827+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:26.891319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.088220+0000","last_change":"2024-04-26T23:57:04.049142+0000","last_active":"2024-04-26T23:57:05.088220+0000","last_peered":"2024-04-26T23:57:05.088220+0000","last_clean":"2024-04-26T23:57:05.088220+0000","last_became_active":"2024-04-26T23:57:04.048780+0000","last_became_peered":"2024-04-26T23:57:04.048780+0000","last_unstale":"2024-04-26T23:57:05.088220+0000","last_undegraded":"2024-04-26T23:57:05.088220+0000","last_fullsized":"2024-04-26T23:57:05.088220+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:27.603016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.098171+0000","last_change":"2024-04-26T23:57:05.096793+0000","last_active":"2024-04-26T23:57:05.098171+0000","last_peered":"2024-04-26T23:57:05.098171+0000","last_clean":"2024-04-26T23:57:05.098171+0000","last_became_active":"2024-04-26T23:57:05.096484+0000","last_became_peered":"2024-04-26T23:57:05.096484+0000","last_unstale":"2024-04-26T23:57:05.098171+0000","last_undegraded":"2024-04-26T23:57:05.098171+0000","last_fullsized":"2024-04-26T23:57:05.098171+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:14.349383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084152+0000","last_change":"2024-04-26T23:57:04.064277+0000","last_active":"2024-04-26T23:57:06.084152+0000","last_peered":"2024-04-26T23:57:06.084152+0000","last_clean":"2024-04-26T23:57:06.084152+0000","last_became_active":"2024-04-26T23:57:04.062632+0000","last_became_peered":"2024-04-26T23:57:04.062632+0000","last_unstale":"2024-04-26T23:57:06.084152+0000","last_undegraded":"2024-04-26T23:57:06.084152+0000","last_fullsized":"2024-04-26T23:57:06.084152+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:14.214176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093861+0000","last_change":"2024-04-26T23:57:05.087622+0000","last_active":"2024-04-26T23:57:05.093861+0000","last_peered":"2024-04-26T23:57:05.093861+0000","last_clean":"2024-04-26T23:57:05.093861+0000","last_became_active":"2024-04-26T23:57:05.087199+0000","last_became_peered":"2024-04-26T23:57:05.087199+0000","last_unstale":"2024-04-26T23:57:05.093861+0000","last_undegraded":"2024-04-26T23:57:05.093861+0000","last_fullsized":"2024-04-26T23:57:05.093861+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:35.346836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.079735+0000","last_change":"2024-04-26T23:57:04.063664+0000","last_active":"2024-04-26T23:57:05.079735+0000","last_peered":"2024-04-26T23:57:05.079735+0000","last_clean":"2024-04-26T23:57:05.079735+0000","last_became_active":"2024-04-26T23:57:04.062388+0000","last_became_peered":"2024-04-26T23:57:04.062388+0000","last_unstale":"2024-04-26T23:57:05.079735+0000","last_undegraded":"2024-04-26T23:57:05.079735+0000","last_fullsized":"2024-04-26T23:57:05.079735+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:11.453284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093899+0000","last_change":"2024-04-26T23:57:05.089980+0000","last_active":"2024-04-26T23:57:05.093899+0000","last_peered":"2024-04-26T23:57:05.093899+0000","last_clean":"2024-04-26T23:57:05.093899+0000","last_became_active":"2024-04-26T23:57:05.089449+0000","last_became_peered":"2024-04-26T23:57:05.089449+0000","last_unstale":"2024-04-26T23:57:05.093899+0000","last_undegraded":"2024-04-26T23:57:05.093899+0000","last_fullsized":"2024-04-26T23:57:05.093899+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:35.569494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.092792+0000","last_change":"2024-04-26T23:57:04.060736+0000","last_active":"2024-04-26T23:57:05.092792+0000","last_peered":"2024-04-26T23:57:05.092792+0000","last_clean":"2024-04-26T23:57:05.092792+0000","last_became_active":"2024-04-26T23:57:04.060416+0000","last_became_peered":"2024-04-26T23:57:04.060416+0000","last_unstale":"2024-04-26T23:57:05.092792+0000","last_undegraded":"2024-04-26T23:57:05.092792+0000","last_fullsized":"2024-04-26T23:57:05.092792+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:19.316285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093388+0000","last_change":"2024-04-26T23:57:05.085624+0000","last_active":"2024-04-26T23:57:05.093388+0000","last_peered":"2024-04-26T23:57:05.093388+0000","last_clean":"2024-04-26T23:57:05.093388+0000","last_became_active":"2024-04-26T23:57:05.082384+0000","last_became_peered":"2024-04-26T23:57:05.082384+0000","last_unstale":"2024-04-26T23:57:05.093388+0000","last_undegraded":"2024-04-26T23:57:05.093388+0000","last_fullsized":"2024-04-26T23:57:05.093388+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:18.963136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083357+0000","last_change":"2024-04-26T23:57:04.061762+0000","last_active":"2024-04-26T23:57:06.083357+0000","last_peered":"2024-04-26T23:57:06.083357+0000","last_clean":"2024-04-26T23:57:06.083357+0000","last_became_active":"2024-04-26T23:57:04.061445+0000","last_became_peered":"2024-04-26T23:57:04.061445+0000","last_unstale":"2024-04-26T23:57:06.083357+0000","last_undegraded":"2024-04-26T23:57:06.083357+0000","last_fullsized":"2024-04-26T23:57:06.083357+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:08.382806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084456+0000","last_change":"2024-04-26T23:57:05.090226+0000","last_active":"2024-04-26T23:57:06.084456+0000","last_peered":"2024-04-26T23:57:06.084456+0000","last_clean":"2024-04-26T23:57:06.084456+0000","last_became_active":"2024-04-26T23:57:05.089933+0000","last_became_peered":"2024-04-26T23:57:05.089933+0000","last_unstale":"2024-04-26T23:57:06.084456+0000","last_undegraded":"2024-04-26T23:57:06.084456+0000","last_fullsized":"2024-04-26T23:57:06.084456+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:26.891319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082192+0000","last_change":"2024-04-26T23:57:04.064554+0000","last_active":"2024-04-26T23:57:06.082192+0000","last_peered":"2024-04-26T23:57:06.082192+0000","last_clean":"2024-04-26T23:57:06.082192+0000","last_became_active":"2024-04-26T23:57:04.063846+0000","last_became_peered":"2024-04-26T23:57:04.063846+0000","last_unstale":"2024-04-26T23:57:06.082192+0000","last_undegraded":"2024-04-26T23:57:06.082192+0000","last_fullsized":"2024-04-26T23:57:06.082192+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:37.973436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084847+0000","last_change":"2024-04-26T23:57:05.093760+0000","last_active":"2024-04-26T23:57:06.084847+0000","last_peered":"2024-04-26T23:57:06.084847+0000","last_clean":"2024-04-26T23:57:06.084847+0000","last_became_active":"2024-04-26T23:57:05.093441+0000","last_became_peered":"2024-04-26T23:57:05.093441+0000","last_unstale":"2024-04-26T23:57:06.084847+0000","last_undegraded":"2024-04-26T23:57:06.084847+0000","last_fullsized":"2024-04-26T23:57:06.084847+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:49.957221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.077242+0000","last_change":"2024-04-26T23:57:04.049458+0000","last_active":"2024-04-26T23:57:06.077242+0000","last_peered":"2024-04-26T23:57:06.077242+0000","last_clean":"2024-04-26T23:57:06.077242+0000","last_became_active":"2024-04-26T23:57:04.049137+0000","last_became_peered":"2024-04-26T23:57:04.049137+0000","last_unstale":"2024-04-26T23:57:06.077242+0000","last_undegraded":"2024-04-26T23:57:06.077242+0000","last_fullsized":"2024-04-26T23:57:06.077242+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:34.329692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.080354+0000","last_change":"2024-04-26T23:57:05.083452+0000","last_active":"2024-04-26T23:57:06.080354+0000","last_peered":"2024-04-26T23:57:06.080354+0000","last_clean":"2024-04-26T23:57:06.080354+0000","last_became_active":"2024-04-26T23:57:05.082946+0000","last_became_peered":"2024-04-26T23:57:05.082946+0000","last_unstale":"2024-04-26T23:57:06.080354+0000","last_undegraded":"2024-04-26T23:57:06.080354+0000","last_fullsized":"2024-04-26T23:57:06.080354+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:22.166396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"209'2","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.091062+0000","last_change":"2024-04-26T23:57:04.055654+0000","last_active":"2024-04-26T23:57:05.091062+0000","last_peered":"2024-04-26T23:57:05.091062+0000","last_clean":"2024-04-26T23:57:05.091062+0000","last_became_active":"2024-04-26T23:57:04.055349+0000","last_became_peered":"2024-04-26T23:57:04.055349+0000","last_unstale":"2024-04-26T23:57:05.091062+0000","last_undegraded":"2024-04-26T23:57:05.091062+0000","last_fullsized":"2024-04-26T23:57:05.091062+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:21.466960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.095466+0000","last_change":"2024-04-26T23:57:05.092079+0000","last_active":"2024-04-26T23:57:05.095466+0000","last_peered":"2024-04-26T23:57:05.095466+0000","last_clean":"2024-04-26T23:57:05.095466+0000","last_became_active":"2024-04-26T23:57:05.091740+0000","last_became_peered":"2024-04-26T23:57:05.091740+0000","last_unstale":"2024-04-26T23:57:05.095466+0000","last_undegraded":"2024-04-26T23:57:05.095466+0000","last_fullsized":"2024-04-26T23:57:05.095466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:44.847443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.085211+0000","last_change":"2024-04-26T23:57:04.059909+0000","last_active":"2024-04-26T23:57:05.085211+0000","last_peered":"2024-04-26T23:57:05.085211+0000","last_clean":"2024-04-26T23:57:05.085211+0000","last_became_active":"2024-04-26T23:57:04.059369+0000","last_became_peered":"2024-04-26T23:57:04.059369+0000","last_unstale":"2024-04-26T23:57:05.085211+0000","last_undegraded":"2024-04-26T23:57:05.085211+0000","last_fullsized":"2024-04-26T23:57:05.085211+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:25.678834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.084073+0000","last_change":"2024-04-26T23:57:05.077101+0000","last_active":"2024-04-26T23:57:05.084073+0000","last_peered":"2024-04-26T23:57:05.084073+0000","last_clean":"2024-04-26T23:57:05.084073+0000","last_became_active":"2024-04-26T23:57:05.074834+0000","last_became_peered":"2024-04-26T23:57:05.074834+0000","last_unstale":"2024-04-26T23:57:05.084073+0000","last_undegraded":"2024-04-26T23:57:05.084073+0000","last_fullsized":"2024-04-26T23:57:05.084073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:26.662514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078406+0000","last_change":"2024-04-26T23:57:04.049882+0000","last_active":"2024-04-26T23:57:06.078406+0000","last_peered":"2024-04-26T23:57:06.078406+0000","last_clean":"2024-04-26T23:57:06.078406+0000","last_became_active":"2024-04-26T23:57:04.049552+0000","last_became_peered":"2024-04-26T23:57:04.049552+0000","last_unstale":"2024-04-26T23:57:06.078406+0000","last_undegraded":"2024-04-26T23:57:06.078406+0000","last_fullsized":"2024-04-26T23:57:06.078406+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:31:43.830299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.097639+0000","last_change":"2024-04-26T23:57:05.096587+0000","last_active":"2024-04-26T23:57:05.097639+0000","last_peered":"2024-04-26T23:57:05.097639+0000","last_clean":"2024-04-26T23:57:05.097639+0000","last_became_active":"2024-04-26T23:57:05.096302+0000","last_became_peered":"2024-04-26T23:57:05.096302+0000","last_unstale":"2024-04-26T23:57:05.097639+0000","last_undegraded":"2024-04-26T23:57:05.097639+0000","last_fullsized":"2024-04-26T23:57:05.097639+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:53.661439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078141+0000","last_change":"2024-04-26T23:57:04.059981+0000","last_active":"2024-04-26T23:57:06.078141+0000","last_peered":"2024-04-26T23:57:06.078141+0000","last_clean":"2024-04-26T23:57:06.078141+0000","last_became_active":"2024-04-26T23:57:04.059656+0000","last_became_peered":"2024-04-26T23:57:04.059656+0000","last_unstale":"2024-04-26T23:57:06.078141+0000","last_undegraded":"2024-04-26T23:57:06.078141+0000","last_fullsized":"2024-04-26T23:57:06.078141+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:25.645371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.078101+0000","last_change":"2024-04-26T23:57:05.069705+0000","last_active":"2024-04-26T23:57:06.078101+0000","last_peered":"2024-04-26T23:57:06.078101+0000","last_clean":"2024-04-26T23:57:06.078101+0000","last_became_active":"2024-04-26T23:57:05.069377+0000","last_became_peered":"2024-04-26T23:57:05.069377+0000","last_unstale":"2024-04-26T23:57:06.078101+0000","last_undegraded":"2024-04-26T23:57:06.078101+0000","last_fullsized":"2024-04-26T23:57:06.078101+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:36.124883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.083098+0000","last_change":"2024-04-26T23:57:04.062696+0000","last_active":"2024-04-26T23:57:06.083098+0000","last_peered":"2024-04-26T23:57:06.083098+0000","last_clean":"2024-04-26T23:57:06.083098+0000","last_became_active":"2024-04-26T23:57:04.062379+0000","last_became_peered":"2024-04-26T23:57:04.062379+0000","last_unstale":"2024-04-26T23:57:06.083098+0000","last_undegraded":"2024-04-26T23:57:06.083098+0000","last_fullsized":"2024-04-26T23:57:06.083098+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:04.383866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.087484+0000","last_change":"2024-04-26T23:57:05.077681+0000","last_active":"2024-04-26T23:57:05.087484+0000","last_peered":"2024-04-26T23:57:05.087484+0000","last_clean":"2024-04-26T23:57:05.087484+0000","last_became_active":"2024-04-26T23:57:05.072588+0000","last_became_peered":"2024-04-26T23:57:05.072588+0000","last_unstale":"2024-04-26T23:57:05.087484+0000","last_undegraded":"2024-04-26T23:57:05.087484+0000","last_fullsized":"2024-04-26T23:57:05.087484+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:09.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.073746+0000","last_change":"2024-04-26T23:57:04.063932+0000","last_active":"2024-04-26T23:57:05.073746+0000","last_peered":"2024-04-26T23:57:05.073746+0000","last_clean":"2024-04-26T23:57:05.073746+0000","last_became_active":"2024-04-26T23:57:04.062057+0000","last_became_peered":"2024-04-26T23:57:04.062057+0000","last_unstale":"2024-04-26T23:57:05.073746+0000","last_undegraded":"2024-04-26T23:57:05.073746+0000","last_fullsized":"2024-04-26T23:57:05.073746+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:33.517599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.093316+0000","last_change":"2024-04-26T23:57:05.089010+0000","last_active":"2024-04-26T23:57:05.093316+0000","last_peered":"2024-04-26T23:57:05.093316+0000","last_clean":"2024-04-26T23:57:05.093316+0000","last_became_active":"2024-04-26T23:57:05.088726+0000","last_became_peered":"2024-04-26T23:57:05.088726+0000","last_unstale":"2024-04-26T23:57:05.093316+0000","last_undegraded":"2024-04-26T23:57:05.093316+0000","last_fullsized":"2024-04-26T23:57:05.093316+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:23.197948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.071189+0000","last_change":"2024-04-26T23:57:04.063636+0000","last_active":"2024-04-26T23:57:05.071189+0000","last_peered":"2024-04-26T23:57:05.071189+0000","last_clean":"2024-04-26T23:57:05.071189+0000","last_became_active":"2024-04-26T23:57:04.063308+0000","last_became_peered":"2024-04-26T23:57:04.063308+0000","last_unstale":"2024-04-26T23:57:05.071189+0000","last_undegraded":"2024-04-26T23:57:05.071189+0000","last_fullsized":"2024-04-26T23:57:05.071189+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:50.629440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.084460+0000","last_change":"2024-04-26T23:57:05.083345+0000","last_active":"2024-04-26T23:57:06.084460+0000","last_peered":"2024-04-26T23:57:06.084460+0000","last_clean":"2024-04-26T23:57:06.084460+0000","last_became_active":"2024-04-26T23:57:05.082969+0000","last_became_peered":"2024-04-26T23:57:05.082969+0000","last_unstale":"2024-04-26T23:57:06.084460+0000","last_undegraded":"2024-04-26T23:57:06.084460+0000","last_fullsized":"2024-04-26T23:57:06.084460+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:41.644183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.081164+0000","last_change":"2024-04-26T23:57:04.059376+0000","last_active":"2024-04-26T23:57:06.081164+0000","last_peered":"2024-04-26T23:57:06.081164+0000","last_clean":"2024-04-26T23:57:06.081164+0000","last_became_active":"2024-04-26T23:57:04.059067+0000","last_became_peered":"2024-04-26T23:57:04.059067+0000","last_unstale":"2024-04-26T23:57:06.081164+0000","last_undegraded":"2024-04-26T23:57:06.081164+0000","last_fullsized":"2024-04-26T23:57:06.081164+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:15.839804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.085835+0000","last_change":"2024-04-26T23:57:05.076351+0000","last_active":"2024-04-26T23:57:05.085835+0000","last_peered":"2024-04-26T23:57:05.085835+0000","last_clean":"2024-04-26T23:57:05.085835+0000","last_became_active":"2024-04-26T23:57:05.076040+0000","last_became_peered":"2024-04-26T23:57:05.076040+0000","last_unstale":"2024-04-26T23:57:05.085835+0000","last_undegraded":"2024-04-26T23:57:05.085835+0000","last_fullsized":"2024-04-26T23:57:05.085835+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:04.014931+0000","last_clean_scrub_stamp":"2024-04-26T23:57:04.014931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:45.999550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"209'1","reported_seq":15,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-04-26T23:57:06.082281+0000","last_change":"2024-04-26T23:57:04.050717+0000","last_active":"2024-04-26T23:57:06.082281+0000","last_peered":"2024-04-26T23:57:06.082281+0000","last_clean":"2024-04-26T23:57:06.082281+0000","last_became_active":"2024-04-26T23:57:04.050323+0000","last_became_peered":"2024-04-26T23:57:04.050323+0000","last_unstale":"2024-04-26T23:57:06.082281+0000","last_undegraded":"2024-04-26T23:57:06.082281+0000","last_fullsized":"2024-04-26T23:57:06.082281+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:57:02.997787+0000","last_clean_scrub_stamp":"2024-04-26T23:57:02.997787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:29.535894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":522,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-04-26T23:57:05.081061+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:57:05.081061+0000","last_peered":"2024-04-26T23:57:05.081061+0000","last_clean":"2024-04-26T23:57:05.081061+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:57:05.081061+0000","last_undegraded":"2024-04-26T23:57:05.081061+0000","last_fullsized":"2024-04-26T23:57:05.081061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56560,"kb_used_data":3172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315280,"statfs":{"total":96636764160,"available":96578846720,"internally_reserved":0,"allocated":3248128,"data_stored":1841299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575092,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55980,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315860,"statfs":{"total":96636764160,"available":96579440640,"internally_reserved":0,"allocated":2654208,"data_stored":1250919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575092,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55992,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315848,"statfs":{"total":96636764160,"available":96579428352,"internally_reserved":0,"allocated":2658304,"data_stored":1257499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60140,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311700,"statfs":{"total":96636764160,"available":96575180800,"internally_reserved":0,"allocated":2650112,"data_stored":1250877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47676,"kb_used_data":2608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324164,"statfs":{"total":96636764160,"available":96587943936,"internally_reserved":0,"allocated":2670592,"data_stored":1253057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64304,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307536,"statfs":{"total":96636764160,"available":96570916864,"internally_reserved":0,"allocated":2654208,"data_stored":1251459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575092,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2584,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2646016,"data_stored":1250807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575092,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64892,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306948,"statfs":{"total":96636764160,"available":96570314752,"internally_reserved":0,"allocated":3252224,"data_stored":1842787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:57:10.066 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:57:10.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:57:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.329+0000 7fa369800640 1 Processor -- start 2024-04-26T23:57:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.330+0000 7fa369800640 1 -- start start 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.331+0000 7fa369800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa3641626c0 0x7fa3640b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.331+0000 7fa369800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3640d07d0 con 0x7fa3640b5fb0 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.331+0000 7fa369800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3640d0ab0 con 0x7fa364162aa0 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.331+0000 7fa369800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa3640d0d90 con 0x7fa3641626c0 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa363400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3640b5fb0 0x7fa3640d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37358/0 (socket says 172.21.15.46:37358) 2024-04-26T23:57:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa363400640 1 -- 172.21.15.46:0/3111182020 learned_addr learned my addr 172.21.15.46:0/3111182020 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa363e00640 1 --2- 172.21.15.46:0/3111182020 >> v2:172.21.15.142:3300/0 conn(0x7fa3641626c0 0x7fa3640b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845250979 0 0) 0x7fa3640d07d0 con 0x7fa3640b5fb0 2024-04-26T23:57:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa348003520 con 0x7fa3640b5fb0 2024-04-26T23:57:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.332+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3916980617 0 0) 0x7fa348003520 con 0x7fa3640b5fb0 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 >> v2:172.21.15.142:3300/0 conn(0x7fa3641626c0 msgr2=0x7fa3640b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 --2- 172.21.15.46:0/3111182020 >> v2:172.21.15.142:3300/0 conn(0x7fa3641626c0 0x7fa3640b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 >> v1:172.21.15.142:6789/0 conn(0x7fa364162aa0 legacy=0x7fa3640d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3640d2c80 con 0x7fa3640b5fb0 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fa34c0017f0 con 0x7fa3640b5fb0 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 >> v1:172.21.15.46:6789/0 conn(0x7fa3640b5fb0 legacy=0x7fa3640d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 0x7fa348003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3640d0d90 con 0x7fa348003a00 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa34800c0c0 con 0x7fa348004800 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa34800c2c0 con 0x7fa3480051c0 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa363e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 0x7fa348003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.333+0000 7fa363400640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa368e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa363e00640 1 -- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3480051c0 msgr2=0x7fa34800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa363e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa363e00640 1 -- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348004800 msgr2=0x7fa348004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa363e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa363e00640 1 -- 172.21.15.46:0/3111182020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa348003520 con 0x7fa348003a00 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa368e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.334+0000 7fa368e00640 1 -- 172.21.15.46:0/3111182020 reap_dead start 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.335+0000 7fa363e00640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 0x7fa348003e20 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fc0 tx=0x7fa348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e4419997905034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.335+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354009c10 con 0x7fa348003a00 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.335+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa354036070 con 0x7fa348003a00 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.335+0000 7fa362a00640 1 -- 172.21.15.46:0/3111182020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35402d070 con 0x7fa348003a00 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 -- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 msgr2=0x7fa348003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 0x7fa348003e20 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fc0 tx=0x7fa348005830 comp rx=0 tx=0).stop 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 -- 172.21.15.46:0/3111182020 shutdown_connections 2024-04-26T23:57:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 --2- 172.21.15.46:0/3111182020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa348003a00 0x7fa348003e20 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 -- 172.21.15.46:0/3111182020 >> 172.21.15.46:0/3111182020 conn(0x7fa36409e2a0 msgr2=0x7fa3640d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 -- 172.21.15.46:0/3111182020 shutdown_connections 2024-04-26T23:57:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.336+0000 7fa369800640 1 -- 172.21.15.46:0/3111182020 wait complete. 2024-04-26T23:57:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.338+0000 7fa369800640 1 Processor -- start 2024-04-26T23:57:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.338+0000 7fa369800640 1 -- start start 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348003a00 0x7fa3640cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 0x7fa364158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3640b4b50 con 0x7fa3640cae10 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3640b49d0 con 0x7fa348003a00 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.339+0000 7fa369800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa3640d5850 con 0x7fa36415b1c0 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35566/0 (socket says 172.21.15.46:35566) 2024-04-26T23:57:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 -- 172.21.15.46:0/778845817 learned_addr learned my addr 172.21.15.46:0/778845817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa368e00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 0x7fa364158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363e00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348003a00 0x7fa3640cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 -- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 msgr2=0x7fa364158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 0x7fa364158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 -- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348003a00 msgr2=0x7fa3640cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348003a00 0x7fa3640cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa363400640 1 -- 172.21.15.46:0/778845817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3640c9a90 con 0x7fa3640cae10 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.340+0000 7fa368e00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 0x7fa364158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa363400640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e7e0 tx=0x7fa34c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0070d0 con 0x7fa3640cae10 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c018070 con 0x7fa3640cae10 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0074d0 con 0x7fa3640cae10 2024-04-26T23:57:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3640c9d70 con 0x7fa3640cae10 2024-04-26T23:57:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.341+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3640ca330 con 0x7fa3640cae10 2024-04-26T23:57:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.342+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3640693b0 con 0x7fa3640cae10 2024-04-26T23:57:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.344+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34c0057a0 con 0x7fa3640cae10 2024-04-26T23:57:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.344+0000 7fa35be00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 0x7fa34006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.345+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7fa34c08bd10 con 0x7fa3640cae10 2024-04-26T23:57:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.345+0000 7fa363e00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 0x7fa34006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.347+0000 7fa363e00640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 0x7fa34006edb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa354035000 tx=0x7fa35402f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.350+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa34c091020 con 0x7fa3640cae10 2024-04-26T23:57:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.626+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa3640d25b0 con 0x7fa3640cae10 2024-04-26T23:57:10.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.632+0000 7fa35be00640 1 -- 172.21.15.46:0/778845817 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7fa34c059f10 con 0x7fa3640cae10 2024-04-26T23:57:10.633 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-26T23:57:10.633 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBrOixmKu3BFBAA3+sdxv04QceN9uWqPb8XJA== 2024-04-26T23:57:10.633 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-26T23:57:10.634 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-26T23:57:10.634 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:57:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-26T23:57:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.634+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 msgr2=0x7fa34006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.634+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 0x7fa34006edb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa354035000 tx=0x7fa35402f000 comp rx=0 tx=0).stop 2024-04-26T23:57:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.634+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 msgr2=0x7fa36415ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.634+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e7e0 tx=0x7fa34c00ebe0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 shutdown_connections 2024-04-26T23:57:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa34006c8f0 0x7fa34006edb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa36415b1c0 0x7fa364158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3640cae10 0x7fa36415ac80 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 --2- 172.21.15.46:0/778845817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa348003a00 0x7fa3640cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 >> 172.21.15.46:0/778845817 conn(0x7fa36409e2a0 msgr2=0x7fa3640b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 shutdown_connections 2024-04-26T23:57:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.635+0000 7fa369800640 1 -- 172.21.15.46:0/778845817 wait complete. 2024-04-26T23:57:10.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:10.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.854+0000 7ff17e600640 1 Processor -- start 2024-04-26T23:57:10.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.854+0000 7ff17e600640 1 -- start start 2024-04-26T23:57:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.856+0000 7ff17e600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff17815edf0 0x7ff1780cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.856+0000 7ff17e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff17815f1d0 0x7ff1780cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.856+0000 7ff17e600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1780b03c0 con 0x7ff1780cb1a0 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.856+0000 7ff17e600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff1780cf5c0 con 0x7ff17815edf0 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.856+0000 7ff17e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1780cb680 con 0x7ff17815f1d0 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.857+0000 7ff17d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff17815edf0 0x7ff1780cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.857+0000 7ff177e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff17815f1d0 0x7ff1780cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.857+0000 7ff17d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff17815edf0 0x7ff1780cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:49308/0 (socket says 172.21.15.46:49308) 2024-04-26T23:57:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.857+0000 7ff17d200640 1 -- 172.21.15.46:0/2958188587 learned_addr learned my addr 172.21.15.46:0/2958188587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.858+0000 7ff17d200640 1 -- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff17815f1d0 msgr2=0x7ff1780cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.858+0000 7ff17d200640 1 --2- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff17815f1d0 0x7ff1780cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.858+0000 7ff17d200640 1 -- 172.21.15.46:0/2958188587 >> v1:172.21.15.142:6789/0 conn(0x7ff1780cb1a0 legacy=0x7ff1780ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.858+0000 7ff17d200640 1 -- 172.21.15.46:0/2958188587 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1780cb960 con 0x7ff17815edf0 2024-04-26T23:57:10.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.859+0000 7ff17d200640 1 --2- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff1780cf080 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009870 tx=0x7ff16c02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9dc834770ee4498 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.860+0000 7ff177400640 1 -- 172.21.15.46:0/2958188587 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037070 con 0x7ff17815edf0 2024-04-26T23:57:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.860+0000 7ff177400640 1 -- 172.21.15.46:0/2958188587 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c02ae50 con 0x7ff17815edf0 2024-04-26T23:57:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.860+0000 7ff177400640 1 -- 172.21.15.46:0/2958188587 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c032860 con 0x7ff17815edf0 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.860+0000 7ff17e600640 1 -- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 msgr2=0x7ff1780cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.860+0000 7ff17e600640 1 --2- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff1780cf080 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009870 tx=0x7ff16c02a300 comp rx=0 tx=0).stop 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 -- 172.21.15.46:0/2958188587 shutdown_connections 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 --2- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff17815f1d0 0x7ff1780cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 --2- 172.21.15.46:0/2958188587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff1780cf080 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 -- 172.21.15.46:0/2958188587 >> 172.21.15.46:0/2958188587 conn(0x7ff1780b26d0 msgr2=0x7ff1780b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 -- 172.21.15.46:0/2958188587 shutdown_connections 2024-04-26T23:57:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.861+0000 7ff17e600640 1 -- 172.21.15.46:0/2958188587 wait complete. 2024-04-26T23:57:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.863+0000 7ff17e600640 1 Processor -- start 2024-04-26T23:57:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.863+0000 7ff17e600640 1 -- start start 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff17815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 0x7ff17806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1780cc200 con 0x7ff1780cb1a0 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1780cbf20 con 0x7ff17815edf0 2024-04-26T23:57:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff1780c6950 con 0x7ff1780c8680 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35582/0 (socket says 172.21.15.46:35582) 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17d200640 1 -- 172.21.15.46:0/1378462649 learned_addr learned my addr 172.21.15.46:0/1378462649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17d200640 1 -- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 msgr2=0x7ff17806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff17dc00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 0x7ff17806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.864+0000 7ff177e00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff17815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17d200640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 0x7ff17806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17d200640 1 -- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 msgr2=0x7ff17815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17d200640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff17815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17d200640 1 -- 172.21.15.46:0/1378462649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1780c7c10 con 0x7ff1780cb1a0 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff177e00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff17815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17dc00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 0x7ff17806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17d200640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009990 tx=0x7ff16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c041020 con 0x7ff1780cb1a0 2024-04-26T23:57:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c03c890 con 0x7ff1780cb1a0 2024-04-26T23:57:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037040 con 0x7ff1780cb1a0 2024-04-26T23:57:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff178151870 con 0x7ff1780cb1a0 2024-04-26T23:57:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.865+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17815e2a0 con 0x7ff1780cb1a0 2024-04-26T23:57:10.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.867+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff178072500 con 0x7ff1780cb1a0 2024-04-26T23:57:10.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.868+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff16c044020 con 0x7ff1780cb1a0 2024-04-26T23:57:10.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.868+0000 7ff174c00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:10.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.869+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7ff16c0ac420 con 0x7ff1780cb1a0 2024-04-26T23:57:10.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.869+0000 7ff177e00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.870+0000 7ff177e00640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 0x7ff15806edb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1780cd520 tx=0x7ff164008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:10.873+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff178072500 con 0x7ff1780cb1a0 2024-04-26T23:57:11.003 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:57:11.001+0000 7f6c1d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:57:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.109+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1780cf0b0 con 0x7ff1780cb1a0 2024-04-26T23:57:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.110+0000 7ff174c00640 1 -- 172.21.15.46:0/1378462649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+5203 (secure 0 0 0) 0x7ff16c04b070 con 0x7ff1780cb1a0 2024-04-26T23:57:11.112 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:11.112 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":167,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:07.590672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12054":{"gid":12054,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3782420006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3782420006},{"type":"v1","addr":"172.21.15.46:6835","nonce":3782420006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12054,"qdb_cluster":[12054]},"id":15}]} 2024-04-26T23:57:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 167 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 msgr2=0x7ff15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 0x7ff15806edb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1780cd520 tx=0x7ff164008040 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 msgr2=0x7ff178151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009990 tx=0x7ff16c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 shutdown_connections 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1780c8680 0x7ff17806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17815edf0 0x7ff17815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 --2- 172.21.15.46:0/1378462649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1780cb1a0 0x7ff178151330 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 >> 172.21.15.46:0/1378462649 conn(0x7ff1780b26d0 msgr2=0x7ff1780b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 shutdown_connections 2024-04-26T23:57:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.113+0000 7ff17e600640 1 -- 172.21.15.46:0/1378462649 wait complete. 2024-04-26T23:57:11.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 167, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:57:04.513003+0000', 'modified': '2024-04-26T23:57:07.590672+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12054': {'gid': 12054, 'name': 'a', 'rank': 0, 'incarnation': 163, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/3782420006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 3782420006}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3782420006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12054, 'qdb_cluster': [12054]} 2024-04-26T23:57:11.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:57:11.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:57:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.747+0000 7f105e200640 1 Processor -- start 2024-04-26T23:57:11.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.748+0000 7f105e200640 1 -- start start 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.749+0000 7f105e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105806eb60 0x7f105814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.749+0000 7f105e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10580b4940 con 0x7f105814d030 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.749+0000 7f105e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10580b4c20 con 0x7f105806e780 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.749+0000 7f105e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f10580b4f00 con 0x7f105806eb60 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105806eb60 0x7f105814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105806eb60 0x7f105814caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35598/0 (socket says 172.21.15.46:35598) 2024-04-26T23:57:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 -- 172.21.15.46:0/1768552338 learned_addr learned my addr 172.21.15.46:0/1768552338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 -- 172.21.15.46:0/1768552338 >> v1:172.21.15.142:6789/0 conn(0x7f105806e780 legacy=0x7f10580b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 -- 172.21.15.46:0/1768552338 >> v1:172.21.15.46:6789/0 conn(0x7f105814d030 legacy=0x7f1058072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.750+0000 7f1056a00640 1 -- 172.21.15.46:0/1768552338 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10580b51e0 con 0x7f105806eb60 2024-04-26T23:57:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.751+0000 7f1056a00640 1 --2- 172.21.15.46:0/1768552338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105806eb60 0x7f105814caf0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f1040009870 tx=0x7f104002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dc77eac11ab5be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.752+0000 7f1056000640 1 -- 172.21.15.46:0/1768552338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040037070 con 0x7f105806eb60 2024-04-26T23:57:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.752+0000 7f1056000640 1 -- 172.21.15.46:0/1768552338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040032440 con 0x7f105806eb60 2024-04-26T23:57:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.752+0000 7f1056000640 1 -- 172.21.15.46:0/1768552338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040032810 con 0x7f105806eb60 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.752+0000 7f105e200640 1 -- 172.21.15.46:0/1768552338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105806eb60 msgr2=0x7f105814caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.752+0000 7f105e200640 1 --2- 172.21.15.46:0/1768552338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105806eb60 0x7f105814caf0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f1040009870 tx=0x7f104002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.753+0000 7f105e200640 1 -- 172.21.15.46:0/1768552338 shutdown_connections 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.753+0000 7f105e200640 1 --2- 172.21.15.46:0/1768552338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105806eb60 0x7f105814caf0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.753+0000 7f105e200640 1 -- 172.21.15.46:0/1768552338 >> 172.21.15.46:0/1768552338 conn(0x7f105806c530 msgr2=0x7f105806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.753+0000 7f105e200640 1 -- 172.21.15.46:0/1768552338 shutdown_connections 2024-04-26T23:57:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.753+0000 7f105e200640 1 -- 172.21.15.46:0/1768552338 wait complete. 2024-04-26T23:57:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.754+0000 7f105e200640 1 Processor -- start 2024-04-26T23:57:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.755+0000 7f105e200640 1 -- start start 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f105806e780 0x7f105815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 0x7f10580b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10580b4590 con 0x7f105814d030 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1058072a70 con 0x7f105806e780 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f105e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1058164f40 con 0x7f105806eb60 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35606/0 (socket says 172.21.15.46:35606) 2024-04-26T23:57:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 -- 172.21.15.46:0/1421760151 learned_addr learned my addr 172.21.15.46:0/1421760151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057400640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f105806e780 0x7f105815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1056a00640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 0x7f10580b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 -- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 msgr2=0x7f10580b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 0x7f10580b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 -- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f105806e780 msgr2=0x7f105815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.756+0000 7f1057e00640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f105806e780 0x7f105815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f1057e00640 1 -- 172.21.15.46:0/1421760151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058165220 con 0x7f105814d030 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f1056a00640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 0x7f10580b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f1057e00640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f105000c760 tx=0x7f105000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f104f400640 1 -- 172.21.15.46:0/1421760151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050016020 con 0x7f105814d030 2024-04-26T23:57:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f104f400640 1 -- 172.21.15.46:0/1421760151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050006d50 con 0x7f105814d030 2024-04-26T23:57:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1058165500 con 0x7f105814d030 2024-04-26T23:57:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f104f400640 1 -- 172.21.15.46:0/1421760151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003bb0 con 0x7f105814d030 2024-04-26T23:57:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.757+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1058165ac0 con 0x7f105814d030 2024-04-26T23:57:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.760+0000 7f104f400640 1 -- 172.21.15.46:0/1421760151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10500050d0 con 0x7f105814d030 2024-04-26T23:57:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.761+0000 7f104f400640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 0x7f103406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.761+0000 7f104f400640 1 -- 172.21.15.46:0/1421760151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f105008b830 con 0x7f105814d030 2024-04-26T23:57:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.762+0000 7f1057400640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 0x7f103406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.762+0000 7f106101a740 1 Processor -- start 2024-04-26T23:57:11.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.762+0000 7f106101a740 1 -- start start 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d3739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55739d2d7600 con 0x55739d3735e0 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55739d274100 con 0x55739d3afcb0 2024-04-26T23:57:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55739d2d35c0 con 0x55739d39e0a0 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f1057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f1057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35608/0 (socket says 172.21.15.46:35608) 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.763+0000 7f1057400640 1 -- 172.21.15.46:0/1857788934 learned_addr learned my addr 172.21.15.46:0/1857788934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 0x7f103406edb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1048004ea0 tx=0x7f1048009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 -- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 msgr2=0x55739d39e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 -- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 msgr2=0x55739d3739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1056a00640 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d3739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d3739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 -- 172.21.15.46:0/1857788934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739cfe92d0 con 0x55739d3735e0 2024-04-26T23:57:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1057400640 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f104800e840 tx=0x7f1048011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31964711bbcf775e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1047400640 1 -- 172.21.15.46:0/1857788934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10480159b0 con 0x55739d3735e0 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1047400640 1 -- 172.21.15.46:0/1857788934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048015b50 con 0x55739d3735e0 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.764+0000 7f1047400640 1 -- 172.21.15.46:0/1857788934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048007020 con 0x55739d3735e0 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 -- 172.21.15.46:0/1857788934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 msgr2=0x55739d375740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f104800e840 tx=0x7f1048011000 comp rx=0 tx=0).stop 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 -- 172.21.15.46:0/1857788934 shutdown_connections 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d3739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 --2- 172.21.15.46:0/1857788934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d375740 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 -- 172.21.15.46:0/1857788934 >> 172.21.15.46:0/1857788934 conn(0x55739d307d10 msgr2=0x55739d3b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 -- 172.21.15.46:0/1857788934 shutdown_connections 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.765+0000 7f106101a740 1 -- 172.21.15.46:0/1857788934 wait complete. 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.766+0000 7f106101a740 1 -- start start 2024-04-26T23:57:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.766+0000 7f106101a740 1 Processor -- start 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d39aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55739d25ef70 con 0x55739d3735e0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55739d39bbf0 con 0x55739d3afcb0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55739d3bc090 con 0x55739d39e0a0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.767+0000 7f106101a740 10 client.0 _ll_register_callbacks cb 0x55739d2cd480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35624/0 (socket says 172.21.15.46:35624) 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 -- 172.21.15.46:0/1672076624 learned_addr learned my addr 172.21.15.46:0/1672076624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1056a00640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057e00640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d39aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 -- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 msgr2=0x55739d39e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 -- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 msgr2=0x55739d39aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d39aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 -- 172.21.15.46:0/1672076624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739d0b36b0 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1056a00640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f1057400640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f1048009010 tx=0x7f10480036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3b3c2b16ec786c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.768+0000 7f103be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048007020 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f10480071c0 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1048015b50 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=167) ==== 24+0+0 (secure 0 0 0) 0x7f1048034050 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f106101a740 10 client.12195 fetch_fsmap learned FSMap version 167 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=167,monmap=2+}) -- 0x55739d3bbe30 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048015b50 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 167) ==== 3494+0+0 (secure 0 0 0) 0x7f1048037020 con 0x55739d3735e0 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f106101a740 10 client.12195 fetch_fsmap finished waiting for FSMap version 167 2024-04-26T23:57:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.769+0000 7f106101a740 10 client.12195 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f106101a740 20 client.12195 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d3cc680 0x55739d3bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f106101a740 4 client.12195 mds_command: new command op to 12054 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 --> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55739d2e5210 con 0x55739d3cc680 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f1056a00640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d3cc680 0x55739d3bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f1056a00640 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d3cc680 0x55739d3bafd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3afddc5ef5271045 server_cookie=631560b294aaf8fa in_seq=0 out_seq=0 2024-04-26T23:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.770+0000 7f103be00640 10 client.12195 ms_handle_connect on v2:172.21.15.46:6833/3782420006 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.771+0000 7f103be00640 1 -- 172.21.15.46:0/1672076624 <== mds.0 v2:172.21.15.46:6833/3782420006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55739d2e5210 con 0x55739d3cc680 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.771+0000 7f103be00640 10 client.12195 handle_command_reply: tid=0 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.771+0000 7f106101a740 1 client.12195 shutdown 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 msgr2=0x55739d38aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f1048009010 tx=0x7f10480036f0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 shutdown_connections 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d3cc680 0x55739d3bafd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3afcb0 0x55739d39aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d39e0a0 0x55739d39e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f103be00640 0 client.12195 ms_handle_reset on v2:172.21.15.46:6833/3782420006 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 --2- 172.21.15.46:0/1672076624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3735e0 0x55739d38aa10 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 >> 172.21.15.46:0/1672076624 conn(0x55739d307d10 msgr2=0x55739d39bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 shutdown_connections 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 1 -- 172.21.15.46:0/1672076624 wait complete. 2024-04-26T23:57:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.772+0000 7f106101a740 20 client.12195 trim_cache size 0 max 16384 2024-04-26T23:57:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.792+0000 7f106101a740 1 Processor -- start 2024-04-26T23:57:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.792+0000 7f106101a740 1 -- start start 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3735e0 0x55739d3bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d39e0a0 0x55739d3cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1058205570 con 0x55739d3afcb0 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10581fa570 con 0x55739d39e0a0 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.793+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1058205040 con 0x55739d3735e0 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35634/0 (socket says 172.21.15.46:35634) 2024-04-26T23:57:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 -- 172.21.15.46:0/1505585303 learned_addr learned my addr 172.21.15.46:0/1505585303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 -- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3735e0 msgr2=0x55739d3bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3735e0 0x55739d3bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 -- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d39e0a0 msgr2=0x55739d3cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d39e0a0 0x55739d3cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 -- 172.21.15.46:0/1505585303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739d374f80 con 0x55739d3afcb0 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1057e00640 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f105000c730 tx=0x7f1050000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8135dab96880348e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1046a00640 1 -- 172.21.15.46:0/1505585303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050071020 con 0x55739d3afcb0 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.794+0000 7f1046a00640 1 -- 172.21.15.46:0/1505585303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050062040 con 0x55739d3afcb0 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f1046a00640 1 -- 172.21.15.46:0/1505585303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050057760 con 0x55739d3afcb0 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 -- 172.21.15.46:0/1505585303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 msgr2=0x55739d39bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f105000c730 tx=0x7f1050000f40 comp rx=0 tx=0).stop 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 -- 172.21.15.46:0/1505585303 shutdown_connections 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d3afcb0 0x55739d39bbb0 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d39e0a0 0x55739d3cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 --2- 172.21.15.46:0/1505585303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3735e0 0x55739d3bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 -- 172.21.15.46:0/1505585303 >> 172.21.15.46:0/1505585303 conn(0x55739d307d10 msgr2=0x55739d3cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 -- 172.21.15.46:0/1505585303 shutdown_connections 2024-04-26T23:57:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.795+0000 7f106101a740 1 -- 172.21.15.46:0/1505585303 wait complete. 2024-04-26T23:57:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.796+0000 7f106101a740 1 -- start start 2024-04-26T23:57:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.796+0000 7f106101a740 1 Processor -- start 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 0x55739d432bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3afcb0 0x55739d3aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55739d3cc500 con 0x55739d39e0a0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55739d39b080 con 0x55739d3735e0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55739d3ccb20 con 0x55739d3afcb0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f106101a740 10 client.0 _ll_register_callbacks cb 0x7f1048005540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.797+0000 7f1056a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35638/0 (socket says 172.21.15.46:35638) 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 -- 172.21.15.46:0/3503280050 learned_addr learned my addr 172.21.15.46:0/3503280050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1057400640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 0x55739d432bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1057e00640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3afcb0 0x55739d3aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 -- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3afcb0 msgr2=0x55739d3aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3afcb0 0x55739d3aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 -- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 msgr2=0x55739d432bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 0x55739d432bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 -- 172.21.15.46:0/3503280050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739d399c90 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1057400640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 0x55739d432bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f1056a00640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f104002a2e0 tx=0x7f10400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8c26104c014982 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f103be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.798+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040048060 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f10400339f0 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1050062040 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=167) ==== 24+0+0 (secure 0 0 0) 0x7f104004e050 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 10 client.12201 fetch_fsmap learned FSMap version 167 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=167,monmap=2+}) -- 0x55739d39d370 con 0x55739d39e0a0 2024-04-26T23:57:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050062040 con 0x55739d39e0a0 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 167) ==== 3494+0+0 (secure 0 0 0) 0x7f1040051020 con 0x55739d39e0a0 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 10 client.12201 fetch_fsmap finished waiting for FSMap version 167 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 10 client.12201 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 20 client.12201 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d39c230 0x55739d39c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 4 client.12201 mds_command: new command op to 12054 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:57:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.799+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 --> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] -- command(tid 0: {"prefix": "status"}) -- 0x55739d2df080 con 0x55739d39c230 2024-04-26T23:57:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f1057400640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d39c230 0x55739d39c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f1057400640 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d39c230 0x55739d39c610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1944516d44c8b7 server_cookie=6659cc4f09832d11 in_seq=0 out_seq=0 2024-04-26T23:57:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f103be00640 10 client.12201 ms_handle_connect on v2:172.21.15.46:6833/3782420006 2024-04-26T23:57:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f103be00640 1 -- 172.21.15.46:0/3503280050 <== mds.0 v2:172.21.15.46:6833/3782420006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55739d2df080 con 0x55739d39c230 2024-04-26T23:57:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f103be00640 10 client.12201 handle_command_reply: tid=0 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f106101a740 1 client.12201 shutdown 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 msgr2=0x55739d3aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.800+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f104002a2e0 tx=0x7f10400046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 shutdown_connections 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:6833/3782420006,v1:172.21.15.46:6835/3782420006] conn(0x55739d39c230 0x55739d39c610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55739d3afcb0 0x55739d3aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55739d39e0a0 0x55739d3aea30 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f103be00640 0 client.12201 ms_handle_reset on v2:172.21.15.46:6833/3782420006 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 --2- 172.21.15.46:0/3503280050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55739d3735e0 0x55739d432bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 >> 172.21.15.46:0/3503280050 conn(0x55739d307d10 msgr2=0x55739d3cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 shutdown_connections 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 1 -- 172.21.15.46:0/3503280050 wait complete. 2024-04-26T23:57:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f106101a740 20 client.12201 trim_cache size 0 max 16384 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 12054, 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.7533219129999997, 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 167, 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 210, 2024-04-26T23:57:11.806 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 210, 2024-04-26T23:57:11.807 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.142393367 2024-04-26T23:57:11.807 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 msgr2=0x7f103406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.801+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 0x7f103406edb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1048004ea0 tx=0x7f1048009040 comp rx=0 tx=0).stop 2024-04-26T23:57:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 msgr2=0x7f10581601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f105000c760 tx=0x7f105000cb60 comp rx=0 tx=0).stop 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 shutdown_connections 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f103406c8f0 0x7f103406edb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105814d030 0x7f10581601e0 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f105806eb60 0x7f10580b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 --2- 172.21.15.46:0/1421760151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f105806e780 0x7f105815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 >> 172.21.15.46:0/1421760151 conn(0x7f105806c530 msgr2=0x7f105806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 shutdown_connections 2024-04-26T23:57:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:11.802+0000 7f105e200640 1 -- 172.21.15.46:0/1421760151 wait complete. 2024-04-26T23:57:11.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:57:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.008+0000 7f38c8c00640 1 Processor -- start 2024-04-26T23:57:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.009+0000 7f38c8c00640 1 -- start start 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.010+0000 7f38c8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38c40b3fb0 0x7f38c40d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.010+0000 7f38c8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c40d2f60 con 0x7f38c415f500 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.010+0000 7f38c8c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c40d3240 con 0x7f38c40d56c0 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.010+0000 7f38c8c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f38c415f8e0 con 0x7f38c40b3fb0 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38c415f500 0x7f38c40d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38978/0 (socket says 172.21.15.46:38978) 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1e00640 1 -- 172.21.15.46:0/1656561294 learned_addr learned my addr 172.21.15.46:0/1656561294 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1400640 1 --2- 172.21.15.46:0/1656561294 >> v2:172.21.15.46:3300/0 conn(0x7f38c40b3fb0 0x7f38c40d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c0a00640 1 -- 172.21.15.46:0/1656561294 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761271677 0 0) 0x7f38c40d2f60 con 0x7f38c415f500 2024-04-26T23:57:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c0a00640 1 -- 172.21.15.46:0/1656561294 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38a8003520 con 0x7f38c415f500 2024-04-26T23:57:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1400640 1 -- 172.21.15.46:0/1656561294 >> v1:172.21.15.142:6789/0 conn(0x7f38c40d56c0 legacy=0x7f38c40b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1400640 1 -- 172.21.15.46:0/1656561294 >> v1:172.21.15.46:6789/0 conn(0x7f38c415f500 legacy=0x7f38c40d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.011+0000 7f38c1400640 1 -- 172.21.15.46:0/1656561294 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c40d3520 con 0x7f38c40b3fb0 2024-04-26T23:57:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.012+0000 7f38c1400640 1 --2- 172.21.15.46:0/1656561294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40b3fb0 0x7f38c40d5180 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0097a0 tx=0x7f38ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387b2df75c2587 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.013+0000 7f38c0a00640 1 -- 172.21.15.46:0/1656561294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037070 con 0x7f38c40b3fb0 2024-04-26T23:57:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.013+0000 7f38c0a00640 1 -- 172.21.15.46:0/1656561294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac032440 con 0x7f38c40b3fb0 2024-04-26T23:57:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.013+0000 7f38c0a00640 1 -- 172.21.15.46:0/1656561294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0327f0 con 0x7f38c40b3fb0 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.013+0000 7f38c8c00640 1 -- 172.21.15.46:0/1656561294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40b3fb0 msgr2=0x7f38c40d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.013+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1656561294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40b3fb0 0x7f38c40d5180 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0097a0 tx=0x7f38ac02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.014+0000 7f38c8c00640 1 -- 172.21.15.46:0/1656561294 shutdown_connections 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.014+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1656561294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40b3fb0 0x7f38c40d5180 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.014+0000 7f38c8c00640 1 -- 172.21.15.46:0/1656561294 >> 172.21.15.46:0/1656561294 conn(0x7f38c4074400 msgr2=0x7f38c409e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.014+0000 7f38c8c00640 1 -- 172.21.15.46:0/1656561294 shutdown_connections 2024-04-26T23:57:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.014+0000 7f38c8c00640 1 -- 172.21.15.46:0/1656561294 wait complete. 2024-04-26T23:57:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.015+0000 7f38c8c00640 1 Processor -- start 2024-04-26T23:57:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.016+0000 7f38c8c00640 1 -- start start 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.016+0000 7f38c8c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 0x7f38c40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c8c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 0x7f38c4163410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40d3ae0 con 0x7f38c40d56c0 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c8c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40d5b30 con 0x7f38c40b3fb0 2024-04-26T23:57:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c8c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f38c40d4000 con 0x7f38c415f500 2024-04-26T23:57:12.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:12.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44102/0 (socket says 172.21.15.46:44102) 2024-04-26T23:57:12.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 -- 172.21.15.46:0/1891151157 learned_addr learned my addr 172.21.15.46:0/1891151157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c2800640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 0x7f38c4163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1e00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 0x7f38c40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 -- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 msgr2=0x7f38c4163410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 0x7f38c4163410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 -- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 msgr2=0x7f38c40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 0x7f38c40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1400640 1 -- 172.21.15.46:0/1891151157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c40b8810 con 0x7f38c40d56c0 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c2800640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 0x7f38c4163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.017+0000 7f38c1e00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 0x7f38c40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38c1400640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f38ac000c00 tx=0x7f38ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037040 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac042070 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac030900 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c40b8ad0 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.018+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c40b9090 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.019+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4069530 con 0x7f38c40d56c0 2024-04-26T23:57:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.020+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38ac030aa0 con 0x7f38c40d56c0 2024-04-26T23:57:12.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.021+0000 7f38ba000640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:12.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.022+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f38ac0ac730 con 0x7f38c40d56c0 2024-04-26T23:57:12.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.022+0000 7f38c1e00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.023+0000 7f38c1e00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f38b4005880 tx=0x7f38b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:12.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.026+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38ac045360 con 0x7f38c40d56c0 2024-04-26T23:57:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.273+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f38c40b4c50 con 0x7f38c40d56c0 2024-04-26T23:57:12.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.274+0000 7f38ba000640 1 -- 172.21.15.46:0/1891151157 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v167) ==== 53+0+73 (secure 0 0 0) 0x7f38ac07a930 con 0x7f38c40d56c0 2024-04-26T23:57:12.275 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:57:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.276+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 msgr2=0x7f38a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.276+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f38b4005880 tx=0x7f38b4009040 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.276+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 msgr2=0x7f38c4162ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.276+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f38ac000c00 tx=0x7f38ac0046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 shutdown_connections 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38c415f500 0x7f38c4163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c40d56c0 0x7f38c4162ed0 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 --2- 172.21.15.46:0/1891151157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38c40b3fb0 0x7f38c40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 >> 172.21.15.46:0/1891151157 conn(0x7f38c4074400 msgr2=0x7f38c406e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 shutdown_connections 2024-04-26T23:57:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12.277+0000 7f38c8c00640 1 -- 172.21.15.46:0/1891151157 wait complete. 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:57:12.298 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:57:12.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.299 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:57:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41389sec preferred_lft 41389sec 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:57:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::8458:d0ff:fe57:1ea8/64 scope link 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 62:db:7f:1f:07:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::60db:7fff:fe1f:77d/64 scope link 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 66:aa:9f:67:3e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::64aa:9fff:fe67:3e5e/64 scope link 2024-04-26T23:57:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:57:12.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.355 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:57:12.433 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-26T23:57:12.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.437 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-26T23:57:12.492 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-26T23:57:12.493 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:57:12.493 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:57:12.493 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:57:12.493 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:57:12.493 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-26T23:57:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:57:12.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:57:12.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.725 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:57:12.775 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:57:12.775 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:57:12.775 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:57:12.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-26T23:57:12.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:57:12.859 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:57:12.917 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:57:12.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:57:12.918 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:57:12.972 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:57:12.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:12.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:57:13.112 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:57:13.113 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:57:13.113 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:57:13.135 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:57:13.135 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:57:13.135 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:57:13.135 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:57:13.135 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790" 2024-04-26T23:57:13.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2024-04-26T23:57:13.161 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=554dfe58-9c53-4ba0-a961-d2b323744aa7" will pass to kernel 2024-04-26T23:57:13.162 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:57:13.162 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:57:13.162 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:57:13.162 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:57:13.162 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:57:13.166 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:57:13.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:13.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:57:13.236 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:57:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:57:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:57:13.368 INFO:teuthology.orchestra.run.smithi046.stdout:client12213 2024-04-26T23:57:13.369 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:57:13.369 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:57:13.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:13.421 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-26T23:57:13.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:13.488 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-26T23:57:13.546 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:57:13.547 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:57:13.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:13.602 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-26T23:57:13.773 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-26T23:57:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-26T23:57:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.120078 s, 52.4 MB/s 2024-04-26T23:57:13.775 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:57:13.776 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:57:13.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:13.831 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-26T23:57:14.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:14.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-26T23:57:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-26T23:57:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.14967 s, 42.0 MB/s 2024-04-26T23:57:14.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:57:14.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.205 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.206 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.207 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.208 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.209 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:57:14.210 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:57:14.211 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:57:14.212 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-26T23:57:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:57:14.214 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds cache max size = 1000 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = False 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds log max segments = 1 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = False 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.215 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-26T23:57:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:57:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:57:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.222 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:57:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-26T23:57:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-26T23:57:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds cache max size = 1000 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds log max segments = 1 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:57:14.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-26T23:57:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-26T23:57:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-26T23:57:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-26T23:57:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:14.383 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-26T23:57:14.384 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds cache max size = 1000 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds log max segments = 1 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-26T23:57:14.386 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.387 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.388 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.46,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 554dfe58-9c53-4ba0-a961-d2b323744aa7 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:57:14.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-26T23:57:14.390 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-26T23:57:14.391 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.392 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.393 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-04-26T23:57:14.394 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-26T23:57:14.395 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds cache max size = 1000 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = False 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds log max segments = 1 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = False 2024-04-26T23:57:14.396 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-04-26T23:57:14.397 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-04-26T23:57:14.397 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-04-26T23:57:14.415 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:57:14.415 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:57:14.469 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:57:14.470 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:57:14.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:57:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.778+0000 7f86a3000640 1 Processor -- start 2024-04-26T23:57:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.778+0000 7f86a3000640 1 -- start start 2024-04-26T23:57:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f86a3000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f869c0d1bc0 0x7f869c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f86a3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f86a3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f869c15f980 con 0x7f869c15f5a0 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f86a3000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f869c0cdbe0 con 0x7f869c0d1bc0 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f86a3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f869c0d20c0 con 0x7f869c0ce070 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f869b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f869c0d1bc0 0x7f869c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f869b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f869c0d1bc0 0x7f869c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44108/0 (socket says 172.21.15.46:44108) 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.780+0000 7f869b400640 1 -- 172.21.15.46:0/847383485 learned_addr learned my addr 172.21.15.46:0/847383485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f86a1000640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f869b400640 1 -- 172.21.15.46:0/847383485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 msgr2=0x7f869c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f869b400640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f869b400640 1 -- 172.21.15.46:0/847383485 >> v1:172.21.15.142:6789/0 conn(0x7f869c15f5a0 legacy=0x7f869c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f869b400640 1 -- 172.21.15.46:0/847383485 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c0d23a0 con 0x7f869c0d1bc0 2024-04-26T23:57:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.781+0000 7f86a1000640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.782+0000 7f869b400640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c0d1bc0 0x7f869c070b50 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f8688009870 tx=0x7f868802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795ebcfe901e38f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.782+0000 7f869aa00640 1 -- 172.21.15.46:0/847383485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688037070 con 0x7f869c0d1bc0 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.782+0000 7f869aa00640 1 -- 172.21.15.46:0/847383485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8688032440 con 0x7f869c0d1bc0 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.782+0000 7f869aa00640 1 -- 172.21.15.46:0/847383485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688032810 con 0x7f869c0d1bc0 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 -- 172.21.15.46:0/847383485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c0d1bc0 msgr2=0x7f869c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c0d1bc0 0x7f869c070b50 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f8688009870 tx=0x7f868802a2a0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 -- 172.21.15.46:0/847383485 shutdown_connections 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 --2- 172.21.15.46:0/847383485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c0d1bc0 0x7f869c070b50 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 -- 172.21.15.46:0/847383485 >> 172.21.15.46:0/847383485 conn(0x7f869c0d2f40 msgr2=0x7f869c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 -- 172.21.15.46:0/847383485 shutdown_connections 2024-04-26T23:57:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.783+0000 7f86a3000640 1 -- 172.21.15.46:0/847383485 wait complete. 2024-04-26T23:57:14.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.785+0000 7f86a3000640 1 Processor -- start 2024-04-26T23:57:14.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.785+0000 7f86a3000640 1 -- start start 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 0x7f869c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f869c0d07d0 con 0x7f869c15f5a0 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f869c0cd870 con 0x7f869c0d1bc0 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f869c071090 con 0x7f869c0ce070 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.786+0000 7f86a1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44112/0 (socket says 172.21.15.46:44112) 2024-04-26T23:57:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 -- 172.21.15.46:0/3018776202 learned_addr learned my addr 172.21.15.46:0/3018776202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f869b400640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 0x7f869c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f869be00640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 -- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 msgr2=0x7f869c0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 -- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 msgr2=0x7f869c15a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 0x7f869c15a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 -- 172.21.15.46:0/3018776202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c0b83e0 con 0x7f869c15f5a0 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f869be00640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f869b400640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 0x7f869c15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.787+0000 7f86a1000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f869400cef0 tx=0x7f869400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.788+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869401a070 con 0x7f869c15f5a0 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.788+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869400d860 con 0x7f869c15f5a0 2024-04-26T23:57:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.788+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869400dc50 con 0x7f869c15f5a0 2024-04-26T23:57:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.788+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869c0b86c0 con 0x7f869c15f5a0 2024-04-26T23:57:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.788+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869c15aa70 con 0x7f869c15f5a0 2024-04-26T23:57:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.789+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869c06a550 con 0x7f869c15f5a0 2024-04-26T23:57:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.790+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f869401d070 con 0x7f869c15f5a0 2024-04-26T23:57:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.792+0000 7f8693e00640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 0x7f867c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.792+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f869408c670 con 0x7f869c15f5a0 2024-04-26T23:57:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.793+0000 7f869be00640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 0x7f867c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.794+0000 7f869be00640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 0x7f867c06edb0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a830 tx=0x7f868c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:14.797+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f869c06a550 con 0x7f869c15f5a0 2024-04-26T23:57:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.002+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f869c069530 con 0x7f869c15f5a0 2024-04-26T23:57:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.004+0000 7f8693e00640 1 -- 172.21.15.46:0/3018776202 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) ==== 74+0+16916 (secure 0 0 0) 0x7f869405a930 con 0x7f869c15f5a0 2024-04-26T23:57:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:15.006 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":210,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:06.062873+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:57:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 msgr2=0x7f867c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 0x7f867c06edb0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a830 tx=0x7f868c008040 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 msgr2=0x7f869c0ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f869400cef0 tx=0x7f869400a710 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 shutdown_connections 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f867c06c8f0 0x7f867c06edb0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f869c15f5a0 0x7f869c0ca9b0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f869c0d1bc0 0x7f869c15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 --2- 172.21.15.46:0/3018776202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f869c0ce070 0x7f869c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 >> 172.21.15.46:0/3018776202 conn(0x7f869c0d2f40 msgr2=0x7f869c0b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 shutdown_connections 2024-04-26T23:57:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.007+0000 7f86a3000640 1 -- 172.21.15.46:0/3018776202 wait complete. 2024-04-26T23:57:15.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:57:02.313551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:57:03.507616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:57:15.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:57:15.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.245+0000 7ff973200640 1 Processor -- start 2024-04-26T23:57:15.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.246+0000 7ff973200640 1 -- start start 2024-04-26T23:57:15.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff973200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9740b5fb0 0x7ff9740b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff973200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9740d07d0 con 0x7ff9741626c0 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff973200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9740d0ab0 con 0x7ff974162aa0 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff973200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff9740d0d90 con 0x7ff9740b5fb0 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff96be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9740b5fb0 0x7ff9740b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff96be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9740b5fb0 0x7ff9740b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44114/0 (socket says 172.21.15.46:44114) 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff971e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9741626c0 0x7ff9740d5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38990/0 (socket says 172.21.15.46:38990) 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.247+0000 7ff96be00640 1 -- 172.21.15.46:0/2744663756 learned_addr learned my addr 172.21.15.46:0/2744663756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:15.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.248+0000 7ff96be00640 1 -- 172.21.15.46:0/2744663756 >> v1:172.21.15.142:6789/0 conn(0x7ff974162aa0 legacy=0x7ff9740d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:15.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.248+0000 7ff96be00640 1 -- 172.21.15.46:0/2744663756 >> v1:172.21.15.46:6789/0 conn(0x7ff9741626c0 legacy=0x7ff9740d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.248+0000 7ff96be00640 1 -- 172.21.15.46:0/2744663756 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740d2c80 con 0x7ff9740b5fb0 2024-04-26T23:57:15.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.249+0000 7ff96be00640 1 --2- 172.21.15.46:0/2744663756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9740b5fb0 0x7ff9740b47c0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009850 tx=0x7ff95c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1e1d176797d0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.249+0000 7ff971400640 1 -- 172.21.15.46:0/2744663756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004030 con 0x7ff9740b5fb0 2024-04-26T23:57:15.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.249+0000 7ff971400640 1 -- 172.21.15.46:0/2744663756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c0041d0 con 0x7ff9740b5fb0 2024-04-26T23:57:15.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.249+0000 7ff971400640 1 -- 172.21.15.46:0/2744663756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c003610 con 0x7ff9740b5fb0 2024-04-26T23:57:15.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 -- 172.21.15.46:0/2744663756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9740b5fb0 msgr2=0x7ff9740b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 --2- 172.21.15.46:0/2744663756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9740b5fb0 0x7ff9740b47c0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009850 tx=0x7ff95c02a290 comp rx=0 tx=0).stop 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 -- 172.21.15.46:0/2744663756 shutdown_connections 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 --2- 172.21.15.46:0/2744663756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9740b5fb0 0x7ff9740b47c0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 -- 172.21.15.46:0/2744663756 >> 172.21.15.46:0/2744663756 conn(0x7ff97409e2a0 msgr2=0x7ff9740d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 -- 172.21.15.46:0/2744663756 shutdown_connections 2024-04-26T23:57:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.250+0000 7ff973200640 1 -- 172.21.15.46:0/2744663756 wait complete. 2024-04-26T23:57:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.252+0000 7ff973200640 1 Processor -- start 2024-04-26T23:57:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.252+0000 7ff973200640 1 -- start start 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9740b5fb0 0x7ff9740b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9741626c0 0x7ff9740ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff97416a2e0 con 0x7ff974162aa0 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740d2260 con 0x7ff9741626c0 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff973200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff97416a460 con 0x7ff9740b5fb0 2024-04-26T23:57:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff972800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff972800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44118/0 (socket says 172.21.15.46:44118) 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.253+0000 7ff972800640 1 -- 172.21.15.46:0/2867599841 learned_addr learned my addr 172.21.15.46:0/2867599841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff971e00640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9740b5fb0 0x7ff9740b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff96be00640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9741626c0 0x7ff9740ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 -- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9740b5fb0 msgr2=0x7ff9740b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9740b5fb0 0x7ff9740b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 -- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9741626c0 msgr2=0x7ff9740ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9741626c0 0x7ff9740ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 -- 172.21.15.46:0/2867599841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff974161800 con 0x7ff974162aa0 2024-04-26T23:57:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff972800640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b9c0 tx=0x7ff96c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c019070 con 0x7ff974162aa0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c00cba0 con 0x7ff974162aa0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c014440 con 0x7ff974162aa0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.254+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff974161ae0 con 0x7ff974162aa0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.255+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9740ce280 con 0x7ff974162aa0 2024-04-26T23:57:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.256+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9740cde00 con 0x7ff974162aa0 2024-04-26T23:57:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.257+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff96c01c070 con 0x7ff974162aa0 2024-04-26T23:57:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.258+0000 7ff96a000640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.258+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7ff96c08bb10 con 0x7ff974162aa0 2024-04-26T23:57:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.259+0000 7ff971e00640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.260+0000 7ff971e00640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff964000f60 tx=0x7ff964007cc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.263+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff96c059d20 con 0x7ff974162aa0 2024-04-26T23:57:15.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.472+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9740693b0 con 0x7ff974162aa0 2024-04-26T23:57:15.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.473+0000 7ff96a000640 1 -- 172.21.15.46:0/2867599841 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v210) ==== 93+0+10 (secure 0 0 0) 0x7ff96c059620 con 0x7ff974162aa0 2024-04-26T23:57:15.475 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 msgr2=0x7ff95006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff964000f60 tx=0x7ff964007cc0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 msgr2=0x7ff9741566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b9c0 tx=0x7ff96c00be80 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 shutdown_connections 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff974162aa0 0x7ff9741566b0 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9741626c0 0x7ff9740ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 --2- 172.21.15.46:0/2867599841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9740b5fb0 0x7ff9740b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 >> 172.21.15.46:0/2867599841 conn(0x7ff97409e2a0 msgr2=0x7ff9740cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 shutdown_connections 2024-04-26T23:57:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.476+0000 7ff973200640 1 -- 172.21.15.46:0/2867599841 wait complete. 2024-04-26T23:57:15.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:57:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.708+0000 7f0587200640 1 Processor -- start 2024-04-26T23:57:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.708+0000 7f0587200640 1 -- start start 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0587200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05880b3d60 0x7f05880d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0587200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 0x7f05880cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0587200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0588162d00 con 0x7f0588162920 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0587200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f05880d2980 con 0x7f05880b3d60 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0587200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05880d24f0 con 0x7f05880cfb10 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0585e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0588162920 0x7f05880d3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39000/0 (socket says 172.21.15.46:39000) 2024-04-26T23:57:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.709+0000 7f0585e00640 1 -- 172.21.15.46:0/2625988071 learned_addr learned my addr 172.21.15.46:0/2625988071 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867092807 0 0) 0x7f0588162d00 con 0x7f0588162920 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f056c003520 con 0x7f0588162920 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0586800640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 0x7f05880cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0585400640 1 --2- 172.21.15.46:0/2625988071 >> v2:172.21.15.142:3300/0 conn(0x7f05880b3d60 0x7f05880d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 257806214 0 0) 0x7f056c003520 con 0x7f0588162920 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 msgr2=0x7f05880cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 0x7f05880cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 >> v2:172.21.15.142:3300/0 conn(0x7f05880b3d60 msgr2=0x7f05880d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 --2- 172.21.15.46:0/2625988071 >> v2:172.21.15.142:3300/0 conn(0x7f05880b3d60 0x7f05880d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05880d3030 con 0x7f0588162920 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f0570001670 con 0x7f0588162920 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 >> v1:172.21.15.46:6789/0 conn(0x7f0588162920 legacy=0x7f05880d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.710+0000 7f0584a00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c004800 0x7f056c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0584a00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 0x7f056c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05880d2980 con 0x7f056c004800 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05880d24f0 con 0x7f056c0039b0 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f056c00bb50 con 0x7f05880cfb10 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c004800 0x7f056c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585e00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 -- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 msgr2=0x7f056c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05880cfb10 0x7f056c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 -- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f056c0039b0 msgr2=0x7f056c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585400640 1 -- 172.21.15.46:0/2625988071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c003520 con 0x7f056c004800 2024-04-26T23:57:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.711+0000 7f0585e00640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.712+0000 7f0585400640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c004800 0x7f056c004c80 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0578007330 tx=0x7f056c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27d97e47d0b2d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578033070 con 0x7f056c004800 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057802f660 con 0x7f056c004800 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0584a00640 1 -- 172.21.15.46:0/2625988071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578009be0 con 0x7f056c004800 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 -- 172.21.15.46:0/2625988071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c004800 msgr2=0x7f056c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 --2- 172.21.15.46:0/2625988071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c004800 0x7f056c004c80 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0578007330 tx=0x7f056c005830 comp rx=0 tx=0).stop 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0586800640 1 -- 172.21.15.46:0/2625988071 reap_dead start 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 -- 172.21.15.46:0/2625988071 shutdown_connections 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 -- 172.21.15.46:0/2625988071 >> 172.21.15.46:0/2625988071 conn(0x7f05880d5000 msgr2=0x7f05880d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 -- 172.21.15.46:0/2625988071 shutdown_connections 2024-04-26T23:57:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.713+0000 7f0587200640 1 -- 172.21.15.46:0/2625988071 wait complete. 2024-04-26T23:57:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.715+0000 7f0587200640 1 Processor -- start 2024-04-26T23:57:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.715+0000 7f0587200640 1 -- start start 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f056c004800 0x7f058815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0588160190 0x7f0588160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05880b4340 con 0x7f05881f46b0 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05880d2180 con 0x7f0588160190 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.716+0000 7f0587200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f05880b44c0 con 0x7f056c004800 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44134/0 (socket says 172.21.15.46:44134) 2024-04-26T23:57:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 -- 172.21.15.46:0/2003413987 learned_addr learned my addr 172.21.15.46:0/2003413987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585e00640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f056c004800 0x7f058815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0586800640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0588160190 0x7f0588160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 -- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f056c004800 msgr2=0x7f058815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f056c004800 0x7f058815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 -- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0588160190 msgr2=0x7f0588160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0588160190 0x7f0588160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.717+0000 7f0585400640 1 -- 172.21.15.46:0/2003413987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0588160ea0 con 0x7f05881f46b0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f0585400640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f057802b040 tx=0x7f0578004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578046070 con 0x7f05881f46b0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057802f460 con 0x7f05881f46b0 2024-04-26T23:57:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05880b7be0 con 0x7f05881f46b0 2024-04-26T23:57:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578033040 con 0x7f05881f46b0 2024-04-26T23:57:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.718+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05880b81a0 con 0x7f05881f46b0 2024-04-26T23:57:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.719+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0588069ac0 con 0x7f05881f46b0 2024-04-26T23:57:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.720+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0578049070 con 0x7f05881f46b0 2024-04-26T23:57:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.722+0000 7f057e000640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 0x7f056406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.722+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f05780ac670 con 0x7f05881f46b0 2024-04-26T23:57:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.723+0000 7f0585e00640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 0x7f056406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.724+0000 7f0585e00640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 0x7f056406ede0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0570001860 tx=0x7f0570009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.727+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f057807a870 con 0x7f05881f46b0 2024-04-26T23:57:15.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.934+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05880cef50 con 0x7f05881f46b0 2024-04-26T23:57:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.935+0000 7f057e000640 1 -- 172.21.15.46:0/2003413987 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v210) ==== 104+0+11 (secure 0 0 0) 0x7f057807a170 con 0x7f05881f46b0 2024-04-26T23:57:16.182 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:57:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 msgr2=0x7f056406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 0x7f056406ede0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0570001860 tx=0x7f0570009040 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 msgr2=0x7f058815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f057802b040 tx=0x7f0578004ab0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 shutdown_connections 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f056406c920 0x7f056406ede0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0588160190 0x7f0588160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05881f46b0 0x7f058815e250 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 --2- 172.21.15.46:0/2003413987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f056c004800 0x7f058815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.937+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 >> 172.21.15.46:0/2003413987 conn(0x7f05880d5000 msgr2=0x7f05880b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.938+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 shutdown_connections 2024-04-26T23:57:16.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:15.938+0000 7f0587200640 1 -- 172.21.15.46:0/2003413987 wait complete. 2024-04-26T23:57:16.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:57:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.411+0000 7f3691600640 1 Processor -- start 2024-04-26T23:57:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.411+0000 7f3691600640 1 -- start start 2024-04-26T23:57:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.412+0000 7f3691600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f368c06f060 0x7f368c0af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.412+0000 7f3691600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f368c160c60 con 0x7f368c06ca60 2024-04-26T23:57:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.412+0000 7f3691600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f368c160f00 con 0x7f368c15cf40 2024-04-26T23:57:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.412+0000 7f3691600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f368c1611e0 con 0x7f368c06f060 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f3683e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f368c06ca60 0x7f368c0aee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39014/0 (socket says 172.21.15.46:39014) 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f3683e00640 1 -- 172.21.15.46:0/2508223902 learned_addr learned my addr 172.21.15.46:0/2508223902 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f368b400640 1 --2- 172.21.15.46:0/2508223902 >> v2:172.21.15.46:3300/0 conn(0x7f368c06f060 0x7f368c0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f368b400640 1 -- 172.21.15.46:0/2508223902 >> v1:172.21.15.142:6789/0 conn(0x7f368c15cf40 legacy=0x7f368c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f368b400640 1 -- 172.21.15.46:0/2508223902 >> v1:172.21.15.46:6789/0 conn(0x7f368c06ca60 legacy=0x7f368c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.413+0000 7f368b400640 1 -- 172.21.15.46:0/2508223902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c1614c0 con 0x7f368c06f060 2024-04-26T23:57:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.414+0000 7f368b400640 1 --2- 172.21.15.46:0/2508223902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06f060 0x7f368c0af530 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f368c06cfc0 tx=0x7f368402a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e8b8d419e304c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.415+0000 7f368a000640 1 -- 172.21.15.46:0/2508223902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684007740 con 0x7f368c06f060 2024-04-26T23:57:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.415+0000 7f368a000640 1 -- 172.21.15.46:0/2508223902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36840078e0 con 0x7f368c06f060 2024-04-26T23:57:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.415+0000 7f368a000640 1 -- 172.21.15.46:0/2508223902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684007cb0 con 0x7f368c06f060 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 -- 172.21.15.46:0/2508223902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06f060 msgr2=0x7f368c0af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 --2- 172.21.15.46:0/2508223902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06f060 0x7f368c0af530 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f368c06cfc0 tx=0x7f368402a550 comp rx=0 tx=0).stop 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 -- 172.21.15.46:0/2508223902 shutdown_connections 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 --2- 172.21.15.46:0/2508223902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06f060 0x7f368c0af530 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 -- 172.21.15.46:0/2508223902 >> 172.21.15.46:0/2508223902 conn(0x7f368c06da90 msgr2=0x7f368c0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 -- 172.21.15.46:0/2508223902 shutdown_connections 2024-04-26T23:57:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.416+0000 7f3691600640 1 -- 172.21.15.46:0/2508223902 wait complete. 2024-04-26T23:57:16.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.417+0000 7f3691600640 1 Processor -- start 2024-04-26T23:57:16.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.418+0000 7f3691600640 1 -- start start 2024-04-26T23:57:16.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 0x7f368c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15cf40 0x7f368c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f368c167ec0 con 0x7f368c06ca60 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f368c15d320 con 0x7f368c15cf40 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3691600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f368c168040 con 0x7f368c06f060 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44160/0 (socket says 172.21.15.46:44160) 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 -- 172.21.15.46:0/111540783 learned_addr learned my addr 172.21.15.46:0/111540783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368b400640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15cf40 0x7f368c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f3683e00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 0x7f368c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 -- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 msgr2=0x7f368c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 0x7f368c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 -- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15cf40 msgr2=0x7f368c145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15cf40 0x7f368c145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.419+0000 7f368aa00640 1 -- 172.21.15.46:0/111540783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c0d12b0 con 0x7f368c06ca60 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3683e00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 0x7f368c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f368aa00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f367c00c460 tx=0x7f367c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c016020 con 0x7f368c06ca60 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c012870 con 0x7f368c06ca60 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c012c60 con 0x7f368c06ca60 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c0d0510 con 0x7f368c06ca60 2024-04-26T23:57:16.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.420+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c0d0ab0 con 0x7f368c06ca60 2024-04-26T23:57:16.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.422+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c069530 con 0x7f368c06ca60 2024-04-26T23:57:16.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.423+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f367c019020 con 0x7f368c06ca60 2024-04-26T23:57:16.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.423+0000 7f3682a00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.424+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f367c08bb80 con 0x7f368c06ca60 2024-04-26T23:57:16.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.424+0000 7f3683e00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.425+0000 7f3683e00640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 0x7f366806edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f368c142fe0 tx=0x7f3674009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.428+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f367c020030 con 0x7f368c06ca60 2024-04-26T23:57:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.634+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f368c167b10 con 0x7f368c06ca60 2024-04-26T23:57:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.635+0000 7f3682a00640 1 -- 172.21.15.46:0/111540783 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v210) ==== 100+0+11 (secure 0 0 0) 0x7f367c0182c0 con 0x7f368c06ca60 2024-04-26T23:57:16.636 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:57:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.637+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 msgr2=0x7f366806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.637+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 0x7f366806edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f368c142fe0 tx=0x7f3674009040 comp rx=0 tx=0).stop 2024-04-26T23:57:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.637+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 msgr2=0x7f368c0b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.637+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f367c00c460 tx=0x7f367c00c920 comp rx=0 tx=0).stop 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 shutdown_connections 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15cf40 0x7f368c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c06f060 0x7f368c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 --2- 172.21.15.46:0/111540783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368c06ca60 0x7f368c0b4a20 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 >> 172.21.15.46:0/111540783 conn(0x7f368c06da90 msgr2=0x7f368c0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 shutdown_connections 2024-04-26T23:57:16.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.638+0000 7f3691600640 1 -- 172.21.15.46:0/111540783 wait complete. 2024-04-26T23:57:16.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:57:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.869+0000 7f6af9000640 1 Processor -- start 2024-04-26T23:57:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.869+0000 7f6af9000640 1 -- start start 2024-04-26T23:57:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6af41466b0 0x7f6af406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af9000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6af4160910 0x7f6af4160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af4073ad0 con 0x7f6af40720e0 2024-04-26T23:57:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af9000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6af41612a0 con 0x7f6af41466b0 2024-04-26T23:57:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af9000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6af4146a90 con 0x7f6af4160910 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6aebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6af40720e0 0x7f6af40cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39018/0 (socket says 172.21.15.46:39018) 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6aebe00640 1 -- 172.21.15.46:0/1176913826 learned_addr learned my addr 172.21.15.46:0/1176913826 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.871+0000 7f6af3400640 1 --2- 172.21.15.46:0/1176913826 >> v2:172.21.15.46:3300/0 conn(0x7f6af41466b0 0x7f6af406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af3e00640 1 --2- 172.21.15.46:0/1176913826 >> v2:172.21.15.142:3300/0 conn(0x7f6af4160910 0x7f6af4160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af3400640 1 -- 172.21.15.46:0/1176913826 >> v2:172.21.15.142:3300/0 conn(0x7f6af4160910 msgr2=0x7f6af4160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af3400640 1 --2- 172.21.15.46:0/1176913826 >> v2:172.21.15.142:3300/0 conn(0x7f6af4160910 0x7f6af4160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af3400640 1 -- 172.21.15.46:0/1176913826 >> v1:172.21.15.46:6789/0 conn(0x7f6af40720e0 legacy=0x7f6af40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af3400640 1 -- 172.21.15.46:0/1176913826 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af4073d70 con 0x7f6af41466b0 2024-04-26T23:57:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.872+0000 7f6af2a00640 1 -- 172.21.15.46:0/1176913826 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030969491 0 0) 0x7f6af4073ad0 con 0x7f6af40720e0 2024-04-26T23:57:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.873+0000 7f6af3400640 1 --2- 172.21.15.46:0/1176913826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af406de80 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40097a0 tx=0x7f6ae402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d6a76225b642b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af2a00640 1 -- 172.21.15.46:0/1176913826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4037070 con 0x7f6af41466b0 2024-04-26T23:57:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af2a00640 1 -- 172.21.15.46:0/1176913826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4032440 con 0x7f6af41466b0 2024-04-26T23:57:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af2a00640 1 -- 172.21.15.46:0/1176913826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4032810 con 0x7f6af41466b0 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 -- 172.21.15.46:0/1176913826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 msgr2=0x7f6af406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 --2- 172.21.15.46:0/1176913826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af406de80 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40097a0 tx=0x7f6ae402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 -- 172.21.15.46:0/1176913826 shutdown_connections 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 --2- 172.21.15.46:0/1176913826 >> v2:172.21.15.142:3300/0 conn(0x7f6af4160910 0x7f6af4160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 --2- 172.21.15.46:0/1176913826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af406de80 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.874+0000 7f6af9000640 1 -- 172.21.15.46:0/1176913826 >> 172.21.15.46:0/1176913826 conn(0x7f6af40b26d0 msgr2=0x7f6af40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.875+0000 7f6af9000640 1 -- 172.21.15.46:0/1176913826 shutdown_connections 2024-04-26T23:57:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.875+0000 7f6af9000640 1 -- 172.21.15.46:0/1176913826 wait complete. 2024-04-26T23:57:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.876+0000 7f6af9000640 1 Processor -- start 2024-04-26T23:57:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.877+0000 7f6af9000640 1 -- start start 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.877+0000 7f6af9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af40720e0 0x7f6af4140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.877+0000 7f6af9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.877+0000 7f6af9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af4160910 0x7f6af40b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6af9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40cb470 con 0x7f6af41466b0 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6af9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40cb190 con 0x7f6af40720e0 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6af9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6af406e3c0 con 0x7f6af4160910 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44174/0 (socket says 172.21.15.46:44174) 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 -- 172.21.15.46:0/3336337026 learned_addr learned my addr 172.21.15.46:0/3336337026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6af3400640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af40720e0 0x7f6af4140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6af3e00640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af4160910 0x7f6af40b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 -- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af4160910 msgr2=0x7f6af40b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af4160910 0x7f6af40b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 -- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af40720e0 msgr2=0x7f6af4140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af40720e0 0x7f6af4140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.878+0000 7f6aebe00640 1 -- 172.21.15.46:0/3336337026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af4141bc0 con 0x7f6af41466b0 2024-04-26T23:57:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6aebe00640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c460 tx=0x7f6adc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc018070 con 0x7f6af41466b0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6adc007500 con 0x7f6af41466b0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc004a20 con 0x7f6af41466b0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af4141600 con 0x7f6af41466b0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.879+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40c8d20 con 0x7f6af41466b0 2024-04-26T23:57:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.880+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af40693b0 con 0x7f6af41466b0 2024-04-26T23:57:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.881+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6adc004bc0 con 0x7f6af41466b0 2024-04-26T23:57:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.882+0000 7f6aeb400640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 0x7f6ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.883+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7f6adc08b090 con 0x7f6af41466b0 2024-04-26T23:57:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.883+0000 7f6af3400640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 0x7f6ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.884+0000 7f6af3400640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 0x7f6ad006edb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4034000 tx=0x7f6ae402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:16.887+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6af4141bc0 con 0x7f6af41466b0 2024-04-26T23:57:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.107+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6af40c7180 con 0x7f6af41466b0 2024-04-26T23:57:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.108+0000 7f6aeb400640 1 -- 172.21.15.46:0/3336337026 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v210) ==== 80+0+0 (secure 0 0 0) 0x7f6adc059290 con 0x7f6af41466b0 2024-04-26T23:57:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-26T23:57:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 msgr2=0x7f6ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 0x7f6ad006edb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4034000 tx=0x7f6ae402e000 comp rx=0 tx=0).stop 2024-04-26T23:57:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 msgr2=0x7f6af415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c460 tx=0x7f6adc00c920 comp rx=0 tx=0).stop 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 shutdown_connections 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6ad006c8f0 0x7f6ad006edb0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af4160910 0x7f6af40b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af41466b0 0x7f6af415ecc0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 --2- 172.21.15.46:0/3336337026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af40720e0 0x7f6af4140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 >> 172.21.15.46:0/3336337026 conn(0x7f6af40b26d0 msgr2=0x7f6af40ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 shutdown_connections 2024-04-26T23:57:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.111+0000 7f6af9000640 1 -- 172.21.15.46:0/3336337026 wait complete. 2024-04-26T23:57:17.134 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:57:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:17.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:57:17.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:57:17.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:17.265 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:57:17.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.502+0000 7fbd2aa00640 1 Processor -- start 2024-04-26T23:57:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.502+0000 7fbd2aa00640 1 -- start start 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd2416e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd2aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240b4940 con 0x7fbd2406e780 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd2aa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240b4c20 con 0x7fbd2406eb60 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbd240b4f00 con 0x7fbd2414d200 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd23e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd2406e780 0x7fbd240b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39032/0 (socket says 172.21.15.46:39032) 2024-04-26T23:57:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.504+0000 7fbd23e00640 1 -- 172.21.15.46:0/1916638615 learned_addr learned my addr 172.21.15.46:0/1916638615 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd28a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd2416e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689874155 0 0) 0x7fbd240b4940 con 0x7fbd2406e780 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd08003520 con 0x7fbd2406e780 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3297216077 0 0) 0x7fbd08003520 con 0x7fbd2406e780 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 msgr2=0x7fbd2416e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd2416e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 >> v1:172.21.15.142:6789/0 conn(0x7fbd2406eb60 legacy=0x7fbd2414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd240b51e0 con 0x7fbd2406e780 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fbd14001670 con 0x7fbd2406e780 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 >> v1:172.21.15.46:6789/0 conn(0x7fbd2406e780 legacy=0x7fbd240b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd08003f60 0x7fbd08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240b4f00 con 0x7fbd08004d40 2024-04-26T23:57:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0800baa0 con 0x7fbd08003f60 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.505+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0800bca0 con 0x7fbd2414d200 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd23400640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd08003f60 0x7fbd08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 -- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 msgr2=0x7fbd080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2414d200 0x7fbd080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 -- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd08003f60 msgr2=0x7fbd08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd08003f60 0x7fbd08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd28a00640 1 -- 172.21.15.46:0/1916638615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd08003520 con 0x7fbd08004d40 2024-04-26T23:57:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.506+0000 7fbd23400640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd08003f60 0x7fbd08004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.507+0000 7fbd28a00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd080051c0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009fc0 tx=0x7fbd08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47245b3a2b6cd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.507+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c009c10 con 0x7fbd08004d40 2024-04-26T23:57:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c036070 con 0x7fbd08004d40 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd22a00640 1 -- 172.21.15.46:0/1916638615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c031880 con 0x7fbd08004d40 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 -- 172.21.15.46:0/1916638615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 msgr2=0x7fbd080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/1916638615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd080051c0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009fc0 tx=0x7fbd08005830 comp rx=0 tx=0).stop 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd28a00640 1 -- 172.21.15.46:0/1916638615 reap_dead start 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 -- 172.21.15.46:0/1916638615 shutdown_connections 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 -- 172.21.15.46:0/1916638615 >> 172.21.15.46:0/1916638615 conn(0x7fbd2406c530 msgr2=0x7fbd2406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 -- 172.21.15.46:0/1916638615 shutdown_connections 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.508+0000 7fbd2aa00640 1 -- 172.21.15.46:0/1916638615 wait complete. 2024-04-26T23:57:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.510+0000 7fbd2aa00640 1 Processor -- start 2024-04-26T23:57:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.510+0000 7fbd2aa00640 1 -- start start 2024-04-26T23:57:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd240b1fc0 0x7fbd240b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd24143b60 0x7fbd24143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240b4590 con 0x7fbd08004d40 2024-04-26T23:57:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24073370 con 0x7fbd24143b60 2024-04-26T23:57:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.511+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2415f190 con 0x7fbd240b1fc0 2024-04-26T23:57:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44190/0 (socket says 172.21.15.46:44190) 2024-04-26T23:57:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 -- 172.21.15.46:0/3618263668 learned_addr learned my addr 172.21.15.46:0/3618263668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23400640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd240b1fc0 0x7fbd240b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd28a00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd24143b60 0x7fbd24143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 -- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd240b1fc0 msgr2=0x7fbd240b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd240b1fc0 0x7fbd240b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 -- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd24143b60 msgr2=0x7fbd24143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd24143b60 0x7fbd24143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 -- 172.21.15.46:0/3618263668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd24141400 con 0x7fbd08004d40 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.512+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e840 tx=0x7fbd1400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.513+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400c870 con 0x7fbd08004d40 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.513+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd14018070 con 0x7fbd08004d40 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.513+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400cc50 con 0x7fbd08004d40 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.513+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd240b3920 con 0x7fbd08004d40 2024-04-26T23:57:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.513+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd240b2730 con 0x7fbd08004d40 2024-04-26T23:57:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.514+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd24069530 con 0x7fbd08004d40 2024-04-26T23:57:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.515+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd1401d460 con 0x7fbd08004d40 2024-04-26T23:57:17.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.516+0000 7fbd1be00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:17.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.516+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7fbd1408bc50 con 0x7fbd08004d40 2024-04-26T23:57:17.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.517+0000 7fbd23400640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:17.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.518+0000 7fbd23400640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 0x7fbd0406edb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbd24144cb0 tx=0x7fbd10009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:17.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.521+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd14059dc0 con 0x7fbd08004d40 2024-04-26T23:57:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.768+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd24163450 con 0x7fbd08004d40 2024-04-26T23:57:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.769+0000 7fbd1be00640 1 -- 172.21.15.46:0/3618263668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+5203 (secure 0 0 0) 0x7fbd140596c0 con 0x7fbd08004d40 2024-04-26T23:57:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:17.772 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":167,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:07.590672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12054":{"gid":12054,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3782420006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3782420006},{"type":"v1","addr":"172.21.15.46:6835","nonce":3782420006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12054,"qdb_cluster":[12054]},"id":15}]} 2024-04-26T23:57:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 167 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 msgr2=0x7fbd0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 0x7fbd0406edb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbd24144cb0 tx=0x7fbd10009040 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 msgr2=0x7fbd2415d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e840 tx=0x7fbd1400ed00 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 shutdown_connections 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd24143b60 0x7fbd24143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd240b1fc0 0x7fbd240b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 --2- 172.21.15.46:0/3618263668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08004d40 0x7fbd2415d150 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 >> 172.21.15.46:0/3618263668 conn(0x7fbd2406c530 msgr2=0x7fbd240cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 shutdown_connections 2024-04-26T23:57:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:17.772+0000 7fbd2aa00640 1 -- 172.21.15.46:0/3618263668 wait complete. 2024-04-26T23:57:17.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-26T23:57:18.051 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-26T23:57:18.052 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T23:57:18.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:57:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.257+0000 7fcdee200640 1 Processor -- start 2024-04-26T23:57:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.257+0000 7fcdee200640 1 -- start start 2024-04-26T23:57:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcdee200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcde8160bd0 0x7fcde8161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcdee200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde8161940 con 0x7fcde806e510 2024-04-26T23:57:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcdee200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde8071f80 con 0x7fcde80732c0 2024-04-26T23:57:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcdee200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcde8072260 con 0x7fcde8160bd0 2024-04-26T23:57:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcde7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcde806e510 0x7fcde806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39036/0 (socket says 172.21.15.46:39036) 2024-04-26T23:57:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.259+0000 7fcde7400640 1 -- 172.21.15.46:0/2814667520 learned_addr learned my addr 172.21.15.46:0/2814667520 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3788091197 0 0) 0x7fcde8161940 con 0x7fcde806e510 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcddea00640 1 --2- 172.21.15.46:0/2814667520 >> v2:172.21.15.142:3300/0 conn(0x7fcde8160bd0 0x7fcde8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdcc003520 con 0x7fcde806e510 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 690328263 0 0) 0x7fcdcc003520 con 0x7fcde806e510 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 >> v2:172.21.15.142:3300/0 conn(0x7fcde8160bd0 msgr2=0x7fcde8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 --2- 172.21.15.46:0/2814667520 >> v2:172.21.15.142:3300/0 conn(0x7fcde8160bd0 0x7fcde8161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 >> v1:172.21.15.142:6789/0 conn(0x7fcde80732c0 legacy=0x7fcde8073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde8072540 con 0x7fcde806e510 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fcdd80019f0 con 0x7fcde806e510 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.260+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 >> v1:172.21.15.46:6789/0 conn(0x7fcde806e510 legacy=0x7fcde806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdcc0039e0 0x7fcdcc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 0x7fcdcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc0051c0 0x7fcdcc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcde8072260 con 0x7fcdcc004800 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc00c090 con 0x7fcdcc0051c0 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcdcc00c290 con 0x7fcdcc0039e0 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 0x7fcdcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 -- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdcc0039e0 msgr2=0x7fcdcc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdcc0039e0 0x7fcdcc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 -- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc0051c0 msgr2=0x7fcdcc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc0051c0 0x7fcdcc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcddea00640 1 -- 172.21.15.46:0/2814667520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc003520 con 0x7fcdcc004800 2024-04-26T23:57:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.261+0000 7fcde7e00640 1 -- 172.21.15.46:0/2814667520 reap_dead start 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.262+0000 7fcddea00640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 0x7fcdcc004c80 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ac0 tx=0x7fcdcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd1c6070cfa286b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0036070 con 0x7fcdcc004800 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0031400 con 0x7fcdcc004800 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcde6a00640 1 -- 172.21.15.46:0/2814667520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0009690 con 0x7fcdcc004800 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 -- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 msgr2=0x7fcdcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 0x7fcdcc004c80 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ac0 tx=0x7fcdcc005830 comp rx=0 tx=0).stop 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 -- 172.21.15.46:0/2814667520 shutdown_connections 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 --2- 172.21.15.46:0/2814667520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc004800 0x7fcdcc004c80 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 -- 172.21.15.46:0/2814667520 >> 172.21.15.46:0/2814667520 conn(0x7fcde806c530 msgr2=0x7fcde806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 -- 172.21.15.46:0/2814667520 shutdown_connections 2024-04-26T23:57:18.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.263+0000 7fcdee200640 1 -- 172.21.15.46:0/2814667520 wait complete. 2024-04-26T23:57:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.265+0000 7fcdee200640 1 Processor -- start 2024-04-26T23:57:18.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.265+0000 7fcdee200640 1 -- start start 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc004800 0x7fcde813ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 0x7fcde80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcde8072ec0 con 0x7fcde81f5270 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcde8072d40 con 0x7fcdcc004800 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.266+0000 7fcdee200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcde8146b10 con 0x7fcde815ab60 2024-04-26T23:57:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44210/0 (socket says 172.21.15.46:44210) 2024-04-26T23:57:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 -- 172.21.15.46:0/1740072685 learned_addr learned my addr 172.21.15.46:0/1740072685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcddea00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 0x7fcde80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7400640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc004800 0x7fcde813ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 -- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 msgr2=0x7fcde80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 0x7fcde80b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 -- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc004800 msgr2=0x7fcde813ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc004800 0x7fcde813ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcde7e00640 1 -- 172.21.15.46:0/1740072685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80b3960 con 0x7fcde81f5270 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.267+0000 7fcddea00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 0x7fcde80b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcde7e00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ce520 tx=0x7fcde000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0017070 con 0x7fcde81f5270 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde00072f0 con 0x7fcde81f5270 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00057c0 con 0x7fcde81f5270 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde8154220 con 0x7fcde81f5270 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.268+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde81595c0 con 0x7fcde81f5270 2024-04-26T23:57:18.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.270+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde8069530 con 0x7fcde81f5270 2024-04-26T23:57:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.271+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcde001a070 con 0x7fcde81f5270 2024-04-26T23:57:18.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.272+0000 7fcddfe00640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 0x7fcdc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.273+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6101+0+0 (secure 0 0 0) 0x7fcde008b0f0 con 0x7fcde81f5270 2024-04-26T23:57:18.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.273+0000 7fcde7400640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 0x7fcdc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.274+0000 7fcde7400640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 0x7fcdc406ede0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005fd0 tx=0x7fcdd80019c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.276+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcde0021020 con 0x7fcde81f5270 2024-04-26T23:57:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.525+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcde806fb00 con 0x7fcde81f5270 2024-04-26T23:57:18.528 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:57:18.526+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:57:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.529+0000 7fcddfe00640 1 -- 172.21.15.46:0/1740072685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v168) ==== 161+0+0 (secure 0 0 0) 0x7fcde005f030 con 0x7fcde81f5270 2024-04-26T23:57:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:57:18.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.534+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 msgr2=0x7fcdc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.534+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 0x7fcdc406ede0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005fd0 tx=0x7fcdd80019c0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.534+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 msgr2=0x7fcde815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.534+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ce520 tx=0x7fcde000c820 comp rx=0 tx=0).stop 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 shutdown_connections 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fcdc406c920 0x7fcdc406ede0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde81f5270 0x7fcde815b5a0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde815ab60 0x7fcde80b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 --2- 172.21.15.46:0/1740072685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdcc004800 0x7fcde813ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.535+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 >> 172.21.15.46:0/1740072685 conn(0x7fcde806c530 msgr2=0x7fcde80b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.536+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 shutdown_connections 2024-04-26T23:57:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.536+0000 7fcdee200640 1 -- 172.21.15.46:0/1740072685 wait complete. 2024-04-26T23:57:18.556 INFO:tasks.ceph.mds.a.smithi046.stderr: -5608> 2024-04-26T23:57:11.001+0000 7f6c1d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:57:18.563 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-26T23:57:18.631 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:57:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.672+0000 7f1950af2f40 1 Processor -- start 2024-04-26T23:57:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.672+0000 7f1950af2f40 1 -- start start 2024-04-26T23:57:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2dfaa5c0 0x562c2e035be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e039b40 0x562c2e03e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c2df09d20 con 0x562c2e042780 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c2df0dd60 con 0x562c2e039b40 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.673+0000 7f1950af2f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562c2e03dca0 con 0x562c2dfaa5c0 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44252/0 (socket says 172.21.15.46:44252) 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 -- 172.21.15.46:0/1368983154 learned_addr learned my addr 172.21.15.46:0/1368983154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 -- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2dfaa5c0 msgr2=0x562c2e035be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2dfaa5c0 0x562c2e035be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 -- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e039b40 msgr2=0x562c2e03e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e039b40 0x562c2e03e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 -- 172.21.15.46:0/1368983154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e03ded0 con 0x562c2e042780 2024-04-26T23:57:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.674+0000 7f194e600640 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f1948025a80 tx=0x7f1948051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c706cbba025fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1947400640 1 -- 172.21.15.46:0/1368983154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948055040 con 0x562c2e042780 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1947400640 1 -- 172.21.15.46:0/1368983154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19480518e0 con 0x562c2e042780 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1947400640 1 -- 172.21.15.46:0/1368983154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948051c90 con 0x562c2e042780 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 -- 172.21.15.46:0/1368983154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 msgr2=0x562c2e0380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f1948025a80 tx=0x7f1948051000 comp rx=0 tx=0).stop 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 -- 172.21.15.46:0/1368983154 shutdown_connections 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e042780 0x562c2e0380f0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e039b40 0x562c2e03e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 --2- 172.21.15.46:0/1368983154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2dfaa5c0 0x562c2e035be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 -- 172.21.15.46:0/1368983154 >> 172.21.15.46:0/1368983154 conn(0x562c2df67a20 msgr2=0x562c2e038cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 -- 172.21.15.46:0/1368983154 shutdown_connections 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.675+0000 7f1950af2f40 1 -- 172.21.15.46:0/1368983154 wait complete. 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.676+0000 7f1950af2f40 1 Processor -- start 2024-04-26T23:57:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.676+0000 7f1950af2f40 1 -- start start 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 0x562c2e03b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2e039b40 0x562c2df60d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e042780 0x562c2df61240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c2e03a620 con 0x562c2dfaa5c0 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c2e03e1f0 con 0x562c2e042780 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.677+0000 7f1950af2f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562c2e036120 con 0x562c2e039b40 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 0x562c2e03b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 0x562c2e03b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44254/0 (socket says 172.21.15.46:44254) 2024-04-26T23:57:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 -- 172.21.15.46:0/2866619247 learned_addr learned my addr 172.21.15.46:0/2866619247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 -- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2e039b40 msgr2=0x562c2df60d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194d200640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2e039b40 0x562c2df60d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194e600640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e042780 0x562c2df61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562c2e039b40 0x562c2df60d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 -- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e042780 msgr2=0x562c2df61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c2e042780 0x562c2df61240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.678+0000 7f194dc00640 1 -- 172.21.15.46:0/2866619247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e02cff0 con 0x562c2dfaa5c0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f194dc00640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 0x562c2e03b3c0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f194000bdb0 tx=0x7f1940009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f1946000640 1 -- 172.21.15.46:0/2866619247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940008070 con 0x562c2dfaa5c0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f1946000640 1 -- 172.21.15.46:0/2866619247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194000f650 con 0x562c2dfaa5c0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f1946000640 1 -- 172.21.15.46:0/2866619247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000fa40 con 0x562c2dfaa5c0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c2e042110 con 0x562c2dfaa5c0 2024-04-26T23:57:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.679+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2e02c670 con 0x562c2dfaa5c0 2024-04-26T23:57:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.681+0000 7f1946000640 1 -- 172.21.15.46:0/2866619247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f194000fbe0 con 0x562c2dfaa5c0 2024-04-26T23:57:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.682+0000 7f1946000640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193006b300 0x7f193006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.682+0000 7f1946000640 1 -- 172.21.15.46:0/2866619247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f194008ab40 con 0x562c2dfaa5c0 2024-04-26T23:57:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.682+0000 7f1950af2f40 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x562c2e036f70 0x562c2e03ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.682+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- osd_op(unknown.0.0:1 31.10 31:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e211) -- 0x562c2e03b900 con 0x562c2e036f70 2024-04-26T23:57:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.683+0000 7f194d200640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193006b300 0x7f193006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.683+0000 7f194e600640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x562c2e036f70 0x562c2e03ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.683+0000 7f194e600640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x562c2e036f70 0x562c2e03ec70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.684+0000 7f194d200640 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193006b300 0x7f193006d7c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x562c2e035fd0 tx=0x7f193c002360 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f194e600640 1 -- 172.21.15.46:0/2866619247 <== osd.2 v2:172.21.15.46:6808/2486201425 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=596b] v0'0 uv10 ondisk = 0) ==== 164+0+596 (crc 0 0 0) 0x7f1948057030 con 0x562c2e036f70 2024-04-26T23:57:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:Writing to - 2024-04-26T23:57:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x562c2e036f70 msgr2=0x562c2e03ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x562c2e036f70 0x562c2e03ec70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193006b300 msgr2=0x7f193006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f193006b300 0x7f193006d7c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x562c2e035fd0 tx=0x7f193c002360 comp rx=0 tx=0).stop 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 msgr2=0x562c2e03b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f1950af2f40 1 --2- 172.21.15.46:0/2866619247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2dfaa5c0 0x562c2e03b3c0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f194000bdb0 tx=0x7f1940009560 comp rx=0 tx=0).stop 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.686+0000 7f194e600640 1 -- 172.21.15.46:0/2866619247 reap_dead start 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.687+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 shutdown_connections 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.687+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 >> 172.21.15.46:0/2866619247 conn(0x562c2df67a20 msgr2=0x562c2e038cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.687+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 shutdown_connections 2024-04-26T23:57:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.687+0000 7f1950af2f40 1 -- 172.21.15.46:0/2866619247 wait complete. 2024-04-26T23:57:18.694 INFO:teuthology.orchestra.run.smithi046.stdout:i* �Y?,fJ<�)���$@@ `��������Y?,fJ<�)Y?,f� �&(8`Y?,fJ<�)8`Y?,fJ<�)��������Y?,f� �&����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 2024-04-26T23:57:18.694 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-26T23:57:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.790+0000 7f6036514f40 1 Processor -- start 2024-04-26T23:57:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.790+0000 7f6036514f40 1 -- start start 2024-04-26T23:57:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e3834a20 0x55f4e38bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f4e378ed20 con 0x55f4e38c7270 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f4e3792d60 con 0x55f4e3834a20 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6036514f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f4e38be2c0 con 0x55f4e38bce10 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44272/0 (socket says 172.21.15.46:44272) 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 -- 172.21.15.46:0/3923107084 learned_addr learned my addr 172.21.15.46:0/3923107084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 -- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 msgr2=0x55f4e38c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6033800640 1 -- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e3834a20 msgr2=0x55f4e38bc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.791+0000 7f6032e00640 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e3834a20 0x55f4e38bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6033800640 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e3834a20 0x55f4e38bc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6033800640 1 -- 172.21.15.46:0/3923107084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e38c2a70 con 0x55f4e38c7270 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6033800640 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f6024006e30 tx=0x7f6024003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5624233564a96b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6031a00640 1 -- 172.21.15.46:0/3923107084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024018070 con 0x55f4e38c7270 2024-04-26T23:57:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6031a00640 1 -- 172.21.15.46:0/3923107084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6024003320 con 0x55f4e38c7270 2024-04-26T23:57:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.792+0000 7f6031a00640 1 -- 172.21.15.46:0/3923107084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024001830 con 0x55f4e38c7270 2024-04-26T23:57:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 -- 172.21.15.46:0/3923107084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 msgr2=0x55f4e37e5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f6024006e30 tx=0x7f6024003a00 comp rx=0 tx=0).stop 2024-04-26T23:57:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 -- 172.21.15.46:0/3923107084 shutdown_connections 2024-04-26T23:57:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e3834a20 0x55f4e38bc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 --2- 172.21.15.46:0/3923107084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e38c7270 0x55f4e37e5c90 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 -- 172.21.15.46:0/3923107084 >> 172.21.15.46:0/3923107084 conn(0x55f4e37eca00 msgr2=0x55f4e3834310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 -- 172.21.15.46:0/3923107084 shutdown_connections 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.793+0000 7f6036514f40 1 -- 172.21.15.46:0/3923107084 wait complete. 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.794+0000 7f6036514f40 1 Processor -- start 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.794+0000 7f6036514f40 1 -- start start 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 0x55f4e37e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e38c7270 0x55f4e38b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f4e38c3030 con 0x55f4e3834a20 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f4e38c2d50 con 0x55f4e38c7270 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6036514f40 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55f4e38c3310 con 0x55f4e38bce10 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6033800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 0x55f4e37e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6033800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 0x55f4e37e5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44276/0 (socket says 172.21.15.46:44276) 2024-04-26T23:57:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6033800640 1 -- 172.21.15.46:0/1764338310 learned_addr learned my addr 172.21.15.46:0/1764338310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6034200640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e38c7270 0x55f4e38b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.795+0000 7f6032e00640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 -- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 msgr2=0x55f4e38b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 -- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e38c7270 msgr2=0x55f4e38b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4e38c7270 0x55f4e38b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 -- 172.21.15.46:0/1764338310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e394c8f0 con 0x55f4e3834a20 2024-04-26T23:57:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6032e00640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55f4e38bce10 0x55f4e38b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6033800640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 0x55f4e37e5e80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x55f4e38c2460 tx=0x7f6024001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f602be00640 1 -- 172.21.15.46:0/1764338310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024024070 con 0x55f4e3834a20 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f602be00640 1 -- 172.21.15.46:0/1764338310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6024019040 con 0x55f4e3834a20 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f602be00640 1 -- 172.21.15.46:0/1764338310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024018030 con 0x55f4e3834a20 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.796+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f4e394cbd0 con 0x55f4e3834a20 2024-04-26T23:57:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.797+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e394d190 con 0x55f4e3834a20 2024-04-26T23:57:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.799+0000 7f602be00640 1 -- 172.21.15.46:0/1764338310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6024027070 con 0x55f4e3834a20 2024-04-26T23:57:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.799+0000 7f602be00640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601406b300 0x7f601406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.799+0000 7f602be00640 1 -- 172.21.15.46:0/1764338310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f602408b1e0 con 0x55f4e3834a20 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.800+0000 7f6036514f40 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x55f4e37e63c0 0x55f4e394f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.800+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- osd_op(unknown.0.0:1 31.10 31:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=592b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211) -- 0x55f4e38c4990 con 0x55f4e37e63c0 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.800+0000 7f6032e00640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601406b300 0x7f601406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.800+0000 7f6034200640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x55f4e37e63c0 0x55f4e394f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.801+0000 7f6034200640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x55f4e37e63c0 0x55f4e394f550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.801+0000 7f6032e00640 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601406b300 0x7f601406d7c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x55f4e38c0b00 tx=0x7f6020001500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.802+0000 7f6034200640 1 -- 172.21.15.46:0/1764338310 <== osd.2 v2:172.21.15.46:6808/2486201425 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v211'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f602c052880 con 0x55f4e37e63c0 2024-04-26T23:57:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x55f4e37e63c0 msgr2=0x55f4e394f550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x55f4e37e63c0 0x55f4e394f550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601406b300 msgr2=0x7f601406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f601406b300 0x7f601406d7c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x55f4e38c0b00 tx=0x7f6020001500 comp rx=0 tx=0).stop 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 msgr2=0x55f4e37e5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 --2- 172.21.15.46:0/1764338310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4e3834a20 0x55f4e37e5e80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x55f4e38c2460 tx=0x7f6024001510 comp rx=0 tx=0).stop 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6034200640 1 -- 172.21.15.46:0/1764338310 reap_dead start 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 shutdown_connections 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 >> 172.21.15.46:0/1764338310 conn(0x55f4e37eca00 msgr2=0x55f4e38342d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 shutdown_connections 2024-04-26T23:57:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:18.803+0000 7f6036514f40 1 -- 172.21.15.46:0/1764338310 wait complete. 2024-04-26T23:57:18.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-26T23:57:19.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.013+0000 7fa34fe00640 1 Processor -- start 2024-04-26T23:57:19.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.014+0000 7fa34fe00640 1 -- start start 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.015+0000 7fa34fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3500b3d60 0x7fa3500d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.015+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3500cfb10 0x7fa3500cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.015+0000 7fa34fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa350162d00 con 0x7fa350162920 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.015+0000 7fa34fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa3500d2980 con 0x7fa3500b3d60 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.015+0000 7fa34fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500d24f0 con 0x7fa3500cfb10 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3500b3d60 0x7fa3500d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3500b3d60 0x7fa3500d3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44282/0 (socket says 172.21.15.46:44282) 2024-04-26T23:57:19.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 -- 172.21.15.46:0/2315995858 learned_addr learned my addr 172.21.15.46:0/2315995858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34f400640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3500cfb10 0x7fa3500cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 -- 172.21.15.46:0/2315995858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3500cfb10 msgr2=0x7fa3500cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3500cfb10 0x7fa3500cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 -- 172.21.15.46:0/2315995858 >> v1:172.21.15.142:6789/0 conn(0x7fa350162920 legacy=0x7fa3500d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:19.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.016+0000 7fa34e000640 1 -- 172.21.15.46:0/2315995858 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3500d3030 con 0x7fa3500b3d60 2024-04-26T23:57:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.017+0000 7fa34e000640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500b3d60 0x7fa3500d3a20 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fa338009e50 tx=0x7fa33802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae4b2474d578300 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34d600640 1 -- 172.21.15.46:0/2315995858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338036070 con 0x7fa3500b3d60 2024-04-26T23:57:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34d600640 1 -- 172.21.15.46:0/2315995858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338031600 con 0x7fa3500b3d60 2024-04-26T23:57:19.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34d600640 1 -- 172.21.15.46:0/2315995858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3380319d0 con 0x7fa3500b3d60 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34fe00640 1 -- 172.21.15.46:0/2315995858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500b3d60 msgr2=0x7fa3500d3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34fe00640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500b3d60 0x7fa3500d3a20 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fa338009e50 tx=0x7fa33802a630 comp rx=0 tx=0).stop 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.018+0000 7fa34fe00640 1 -- 172.21.15.46:0/2315995858 shutdown_connections 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.019+0000 7fa34fe00640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3500cfb10 0x7fa3500cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.019+0000 7fa34fe00640 1 --2- 172.21.15.46:0/2315995858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500b3d60 0x7fa3500d3a20 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.019+0000 7fa34fe00640 1 -- 172.21.15.46:0/2315995858 >> 172.21.15.46:0/2315995858 conn(0x7fa3500d5000 msgr2=0x7fa3500d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.019+0000 7fa34fe00640 1 -- 172.21.15.46:0/2315995858 shutdown_connections 2024-04-26T23:57:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.019+0000 7fa34fe00640 1 -- 172.21.15.46:0/2315995858 wait complete. 2024-04-26T23:57:19.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.020+0000 7fa34fe00640 1 Processor -- start 2024-04-26T23:57:19.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.020+0000 7fa34fe00640 1 -- start start 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3500b3d60 0x7fa3500cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa350162920 0x7fa3501f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500b4340 con 0x7fa3500cfb10 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500d2180 con 0x7fa3500b3d60 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.021+0000 7fa34fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500b44c0 con 0x7fa350162920 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44286/0 (socket says 172.21.15.46:44286) 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 -- 172.21.15.46:0/4193650078 learned_addr learned my addr 172.21.15.46:0/4193650078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 -- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa350162920 msgr2=0x7fa3501f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa350162920 0x7fa3501f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 -- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3500b3d60 msgr2=0x7fa3500cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3500b3d60 0x7fa3500cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.022+0000 7fa34e000640 1 -- 172.21.15.46:0/4193650078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3500ccf70 con 0x7fa3500cfb10 2024-04-26T23:57:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa34e000640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fa338034040 tx=0x7fa338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338036050 con 0x7fa3500cfb10 2024-04-26T23:57:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338042070 con 0x7fa3500cfb10 2024-04-26T23:57:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338009510 con 0x7fa3500cfb10 2024-04-26T23:57:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501f50e0 con 0x7fa3500cfb10 2024-04-26T23:57:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.023+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501f56a0 con 0x7fa3500cfb10 2024-04-26T23:57:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.024+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa350069ac0 con 0x7fa3500cfb10 2024-04-26T23:57:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.026+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa338007bb0 con 0x7fa3500cfb10 2024-04-26T23:57:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.026+0000 7fa346a00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 0x7fa32c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.027+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7fa3380acc70 con 0x7fa3500cfb10 2024-04-26T23:57:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.027+0000 7fa34ea00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 0x7fa32c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.028+0000 7fa34ea00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 0x7fa32c06edb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.032+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa338045360 con 0x7fa3500cfb10 2024-04-26T23:57:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.235+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) -- 0x7fa3500b2560 con 0x7fa3500cfb10 2024-04-26T23:57:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.241+0000 7fa346a00640 1 -- 172.21.15.46:0/4193650078 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v51)=0 v51) ==== 140+0+0 (secure 0 0 0) 0x7fa33807ae20 con 0x7fa3500cfb10 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 msgr2=0x7fa32c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 0x7fa32c06edb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340006bc0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 msgr2=0x7fa3500ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fa338034040 tx=0x7fa338004a40 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 shutdown_connections 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa32c06c8f0 0x7fa32c06edb0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa350162920 0x7fa3501f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3500cfb10 0x7fa3500ce8e0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 --2- 172.21.15.46:0/4193650078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3500b3d60 0x7fa3500cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 >> 172.21.15.46:0/4193650078 conn(0x7fa3500d5000 msgr2=0x7fa3500b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 shutdown_connections 2024-04-26T23:57:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.244+0000 7fa34fe00640 1 -- 172.21.15.46:0/4193650078 wait complete. 2024-04-26T23:57:19.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T23:57:19.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.476+0000 7f4786200640 1 Processor -- start 2024-04-26T23:57:19.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.476+0000 7f4786200640 1 -- start start 2024-04-26T23:57:19.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f478015cec0 0x7f47800cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 0x7f47800735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 0x7f4780071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f47800cdf90 con 0x7f478015cec0 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f47800ce7d0 con 0x7f4780073b30 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4786200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47800ceab0 con 0x7f47800cdb30 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4784e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f478015cec0 0x7f47800cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4784e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f478015cec0 0x7f47800cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44298/0 (socket says 172.21.15.46:44298) 2024-04-26T23:57:19.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.478+0000 7f4784e00640 1 -- 172.21.15.46:0/1862078337 learned_addr learned my addr 172.21.15.46:0/1862078337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f477fe00640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 0x7f47800735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4785800640 1 --2- 172.21.15.46:0/1862078337 >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 0x7f4780071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4784e00640 1 -- 172.21.15.46:0/1862078337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 msgr2=0x7f47800735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4784e00640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 0x7f47800735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4784e00640 1 -- 172.21.15.46:0/1862078337 >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 msgr2=0x7f4780071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4784e00640 1 --2- 172.21.15.46:0/1862078337 >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 0x7f4780071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4784e00640 1 -- 172.21.15.46:0/1862078337 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800ced90 con 0x7f478015cec0 2024-04-26T23:57:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f4785800640 1 --2- 172.21.15.46:0/1862078337 >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 0x7f4780071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.479+0000 7f477fe00640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 0x7f47800735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.480+0000 7f4784e00640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015cec0 0x7f47800cd5f0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f47800cb1e0 tx=0x7f477402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c6122d3c91a256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.480+0000 7f477f400640 1 -- 172.21.15.46:0/1862078337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774037070 con 0x7f478015cec0 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.480+0000 7f477f400640 1 -- 172.21.15.46:0/1862078337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4774032440 con 0x7f478015cec0 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.480+0000 7f477f400640 1 -- 172.21.15.46:0/1862078337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774032810 con 0x7f478015cec0 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 -- 172.21.15.46:0/1862078337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015cec0 msgr2=0x7f47800cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015cec0 0x7f47800cd5f0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f47800cb1e0 tx=0x7f477402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 -- 172.21.15.46:0/1862078337 shutdown_connections 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 --2- 172.21.15.46:0/1862078337 >> v2:172.21.15.142:3300/0 conn(0x7f4780073b30 0x7f4780071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47800cdb30 0x7f47800735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 --2- 172.21.15.46:0/1862078337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015cec0 0x7f47800cd5f0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 -- 172.21.15.46:0/1862078337 >> 172.21.15.46:0/1862078337 conn(0x7f47800b27b0 msgr2=0x7f47800b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 -- 172.21.15.46:0/1862078337 shutdown_connections 2024-04-26T23:57:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.481+0000 7f4786200640 1 -- 172.21.15.46:0/1862078337 wait complete. 2024-04-26T23:57:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.483+0000 7f4786200640 1 Processor -- start 2024-04-26T23:57:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.483+0000 7f4786200640 1 -- start start 2024-04-26T23:57:19.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4780073b30 0x7f47801f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f478015cec0 0x7f478015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f478015d2a0 con 0x7f47800cdb30 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47800cf350 con 0x7f4780073b30 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f4786200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4780073fa0 con 0x7f478015cec0 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f477fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f477fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44312/0 (socket says 172.21.15.46:44312) 2024-04-26T23:57:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.484+0000 7f477fe00640 1 -- 172.21.15.46:0/2582997963 learned_addr learned my addr 172.21.15.46:0/2582997963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f4784e00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4780073b30 0x7f47801f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f4785800640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f478015cec0 0x7f478015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 -- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f478015cec0 msgr2=0x7f478015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f478015cec0 0x7f478015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 -- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4780073b30 msgr2=0x7f47801f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4780073b30 0x7f47801f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 -- 172.21.15.46:0/2582997963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800c59d0 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.485+0000 7f477fe00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c7f0 tx=0x7f476c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.486+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c015070 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.486+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c007a30 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.486+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c007e00 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.486+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47800c5cb0 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.486+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47800c6270 con 0x7f47800cdb30 2024-04-26T23:57:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.487+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47800693b0 con 0x7f47800cdb30 2024-04-26T23:57:19.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.488+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f476c018070 con 0x7f47800cdb30 2024-04-26T23:57:19.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.489+0000 7f477cc00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:19.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.489+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f476c08bcb0 con 0x7f47800cdb30 2024-04-26T23:57:19.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.489+0000 7f4784e00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:19.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.490+0000 7f4784e00640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 0x7f476006edb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4774034000 tx=0x7f477402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:19.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.493+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f476c059ef0 con 0x7f47800cdb30 2024-04-26T23:57:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:19.740+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f47800b2cc0 con 0x7f47800cdb30 2024-04-26T23:57:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.544+0000 7f477cc00640 1 -- 172.21.15.46:0/2582997963 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v170) ==== 162+0+0 (secure 0 0 0) 0x7f476c0597f0 con 0x7f47800cdb30 2024-04-26T23:57:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 msgr2=0x7f476006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 0x7f476006edb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4774034000 tx=0x7f477402e000 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 msgr2=0x7f478015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c7f0 tx=0x7f476c00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 shutdown_connections 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f478015cec0 0x7f478015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800cdb30 0x7f478015d420 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 --2- 172.21.15.46:0/2582997963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4780073b30 0x7f47801f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.548+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 >> 172.21.15.46:0/2582997963 conn(0x7f47800b27b0 msgr2=0x7f4780159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.549+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 shutdown_connections 2024-04-26T23:57:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.549+0000 7f4786200640 1 -- 172.21.15.46:0/2582997963 wait complete. 2024-04-26T23:57:20.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.762+0000 7f0786c00640 1 Processor -- start 2024-04-26T23:57:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.763+0000 7f0786c00640 1 -- start start 2024-04-26T23:57:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.764+0000 7f0786c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0780142280 0x7f078006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.764+0000 7f0786c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078006ea50 0x7f0780144f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.764+0000 7f0786c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07800b43f0 con 0x7f07800b3e80 2024-04-26T23:57:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.764+0000 7f0786c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f07800b46d0 con 0x7f0780142280 2024-04-26T23:57:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.764+0000 7f0786c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07800b49b0 con 0x7f078006ea50 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0780142280 0x7f078006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f0777e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f07800b3e80 0x7f0780144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39076/0 (socket says 172.21.15.46:39076) 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f0777e00640 1 -- 172.21.15.46:0/1559932034 learned_addr learned my addr 172.21.15.46:0/1559932034 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f0784c00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078006ea50 0x7f0780144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077fe00640 1 -- 172.21.15.46:0/1559932034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078006ea50 msgr2=0x7f0780144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077fe00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078006ea50 0x7f0780144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077f400640 1 -- 172.21.15.46:0/1559932034 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292269730 0 0) 0x7f07800b43f0 con 0x7f07800b3e80 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077fe00640 1 -- 172.21.15.46:0/1559932034 >> v1:172.21.15.46:6789/0 conn(0x7f07800b3e80 legacy=0x7f0780144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.765+0000 7f077fe00640 1 -- 172.21.15.46:0/1559932034 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07800b4c90 con 0x7f0780142280 2024-04-26T23:57:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.766+0000 7f077fe00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078006e510 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f07700097a0 tx=0x7f077002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a58ac9c5df85431 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f077f400640 1 -- 172.21.15.46:0/1559932034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770037070 con 0x7f0780142280 2024-04-26T23:57:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f077f400640 1 -- 172.21.15.46:0/1559932034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0770032440 con 0x7f0780142280 2024-04-26T23:57:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f077f400640 1 -- 172.21.15.46:0/1559932034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770032810 con 0x7f0780142280 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 -- 172.21.15.46:0/1559932034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 msgr2=0x7f078006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078006e510 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f07700097a0 tx=0x7f077002a180 comp rx=0 tx=0).stop 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 -- 172.21.15.46:0/1559932034 shutdown_connections 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078006ea50 0x7f0780144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 --2- 172.21.15.46:0/1559932034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078006e510 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.767+0000 7f0786c00640 1 -- 172.21.15.46:0/1559932034 >> 172.21.15.46:0/1559932034 conn(0x7f078006c530 msgr2=0x7f078006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.768+0000 7f0786c00640 1 -- 172.21.15.46:0/1559932034 shutdown_connections 2024-04-26T23:57:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.768+0000 7f0786c00640 1 -- 172.21.15.46:0/1559932034 wait complete. 2024-04-26T23:57:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.769+0000 7f0786c00640 1 Processor -- start 2024-04-26T23:57:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.769+0000 7f0786c00640 1 -- start start 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078006ea50 0x7f0780158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07800b3e80 0x7f078015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0780142a20 con 0x7f0780142280 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07801428a0 con 0x7f078006ea50 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.770+0000 7f0786c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0780145490 con 0x7f07800b3e80 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44326/0 (socket says 172.21.15.46:44326) 2024-04-26T23:57:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 -- 172.21.15.46:0/3525551575 learned_addr learned my addr 172.21.15.46:0/3525551575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 -- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07800b3e80 msgr2=0x7f078015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07800b3e80 0x7f078015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 -- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078006ea50 msgr2=0x7f0780158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078006ea50 0x7f0780158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 -- 172.21.15.46:0/3525551575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0780138aa0 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.771+0000 7f0784c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f0778007e70 tx=0x7f077800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.772+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778014620 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.772+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0778019070 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.772+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778014a20 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.772+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07801f8900 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.772+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801f8ec0 con 0x7f0780142280 2024-04-26T23:57:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.773+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07800cd7e0 con 0x7f0780142280 2024-04-26T23:57:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.774+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0778014bc0 con 0x7f0780142280 2024-04-26T23:57:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.775+0000 7f077cc00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 0x7f075c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.775+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f077808c6d0 con 0x7f0780142280 2024-04-26T23:57:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.776+0000 7f077fe00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 0x7f075c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.777+0000 7f077fe00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 0x7f075c06edb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0770034000 tx=0x7f077002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:20.781+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f077805a880 con 0x7f0780142280 2024-04-26T23:57:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.033+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0780069530 con 0x7f0780142280 2024-04-26T23:57:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.034+0000 7f077cc00640 1 -- 172.21.15.46:0/3525551575 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) ==== 95+0+5196 (secure 0 0 0) 0x7f077805a180 con 0x7f0780142280 2024-04-26T23:57:21.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":171,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:20.551006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-04-26T23:57:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 171 2024-04-26T23:57:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 msgr2=0x7f075c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 0x7f075c06edb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0770034000 tx=0x7f077002c000 comp rx=0 tx=0).stop 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 msgr2=0x7f078015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f0778007e70 tx=0x7f077800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 shutdown_connections 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f075c06c8f0 0x7f075c06edb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0780142280 0x7f078015bcc0 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07800b3e80 0x7f078015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 --2- 172.21.15.46:0/3525551575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f078006ea50 0x7f0780158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.037+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 >> 172.21.15.46:0/3525551575 conn(0x7f078006c530 msgr2=0x7f07800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.038+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 shutdown_connections 2024-04-26T23:57:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:21.038+0000 7f0786c00640 1 -- 172.21.15.46:0/3525551575 wait complete. 2024-04-26T23:57:21.463 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 171, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:57:04.513003+0000', 'modified': '2024-04-26T23:57:20.551006+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 211, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12273': {'gid': 12273, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/2610577522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2610577522}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2610577522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:57:21.463 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_12273:up:replay 2024-04-26T23:57:22.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.683+0000 7f2a37200640 1 Processor -- start 2024-04-26T23:57:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.683+0000 7f2a37200640 1 -- start start 2024-04-26T23:57:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.685+0000 7f2a37200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a3814c2e0 0x7f2a380ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.685+0000 7f2a37200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.685+0000 7f2a37200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3814c6c0 con 0x7f2a380cd9f0 2024-04-26T23:57:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.685+0000 7f2a37200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2a380cde60 con 0x7f2a3814c2e0 2024-04-26T23:57:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.685+0000 7f2a37200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a380cb8b0 con 0x7f2a380cb0f0 2024-04-26T23:57:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:56058/0 (socket says 172.21.15.46:56058) 2024-04-26T23:57:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a2d400640 1 -- 172.21.15.46:0/3868258976 learned_addr learned my addr 172.21.15.46:0/3868258976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a35e00640 1 --2- 172.21.15.46:0/3868258976 >> v2:172.21.15.142:3300/0 conn(0x7f2a3814c2e0 0x7f2a380ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a2d400640 1 -- 172.21.15.46:0/3868258976 >> v2:172.21.15.142:3300/0 conn(0x7f2a3814c2e0 msgr2=0x7f2a380ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.686+0000 7f2a2d400640 1 --2- 172.21.15.46:0/3868258976 >> v2:172.21.15.142:3300/0 conn(0x7f2a3814c2e0 0x7f2a380ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.687+0000 7f2a2d400640 1 -- 172.21.15.46:0/3868258976 >> v1:172.21.15.142:6789/0 conn(0x7f2a380cd9f0 legacy=0x7f2a380c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.687+0000 7f2a2d400640 1 -- 172.21.15.46:0/3868258976 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a38167e10 con 0x7f2a380cb0f0 2024-04-26T23:57:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.688+0000 7f2a2d400640 1 --2- 172.21.15.46:0/3868258976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009870 tx=0x7f2a2002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3cc2746911fc735 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.688+0000 7f2a35400640 1 -- 172.21.15.46:0/3868258976 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20037070 con 0x7f2a380cb0f0 2024-04-26T23:57:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.688+0000 7f2a35400640 1 -- 172.21.15.46:0/3868258976 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a20032440 con 0x7f2a380cb0f0 2024-04-26T23:57:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.688+0000 7f2a35400640 1 -- 172.21.15.46:0/3868258976 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20032810 con 0x7f2a380cb0f0 2024-04-26T23:57:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 -- 172.21.15.46:0/3868258976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 msgr2=0x7f2a380cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 --2- 172.21.15.46:0/3868258976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009870 tx=0x7f2a2002a1c0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 -- 172.21.15.46:0/3868258976 shutdown_connections 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 --2- 172.21.15.46:0/3868258976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a380cb0f0 0x7f2a380cd4b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 --2- 172.21.15.46:0/3868258976 >> v2:172.21.15.142:3300/0 conn(0x7f2a3814c2e0 0x7f2a380ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.689+0000 7f2a37200640 1 -- 172.21.15.46:0/3868258976 >> 172.21.15.46:0/3868258976 conn(0x7f2a380b26d0 msgr2=0x7f2a380b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.690+0000 7f2a37200640 1 -- 172.21.15.46:0/3868258976 shutdown_connections 2024-04-26T23:57:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.690+0000 7f2a37200640 1 -- 172.21.15.46:0/3868258976 wait complete. 2024-04-26T23:57:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.691+0000 7f2a37200640 1 Processor -- start 2024-04-26T23:57:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.691+0000 7f2a37200640 1 -- start start 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a380cb0f0 0x7f2a38161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3814c2e0 0x7f2a38145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38167610 con 0x7f2a380cd9f0 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38167330 con 0x7f2a380cb0f0 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.692+0000 7f2a37200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a381678f0 con 0x7f2a3814c2e0 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45136/0 (socket says 172.21.15.46:45136) 2024-04-26T23:57:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 -- 172.21.15.46:0/3398716641 learned_addr learned my addr 172.21.15.46:0/3398716641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a35e00640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a380cb0f0 0x7f2a38161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a36800640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3814c2e0 0x7f2a38145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 -- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3814c2e0 msgr2=0x7f2a38145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3814c2e0 0x7f2a38145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 -- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a380cb0f0 msgr2=0x7f2a38161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a380cb0f0 0x7f2a38161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.693+0000 7f2a2d400640 1 -- 172.21.15.46:0/3398716641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a38162360 con 0x7f2a380cd9f0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a2d400640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20037040 con 0x7f2a380cd9f0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a20042070 con 0x7f2a380cd9f0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20030860 con 0x7f2a380cd9f0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3806e360 con 0x7f2a380cd9f0 2024-04-26T23:57:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.694+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3806e920 con 0x7f2a380cd9f0 2024-04-26T23:57:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.695+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a380693b0 con 0x7f2a380cd9f0 2024-04-26T23:57:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.696+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a2002ad10 con 0x7f2a380cd9f0 2024-04-26T23:57:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.697+0000 7f2a2ea00640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 0x7f2a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.697+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f2a200ac390 con 0x7f2a380cd9f0 2024-04-26T23:57:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.698+0000 7f2a35e00640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 0x7f2a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:22.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.699+0000 7f2a35e00640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 0x7f2a1406edb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009a30 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.702+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a2007a540 con 0x7f2a380cd9f0 2024-04-26T23:57:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.943+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a381634d0 con 0x7f2a380cd9f0 2024-04-26T23:57:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.944+0000 7f2a2ea00640 1 -- 172.21.15.46:0/3398716641 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) ==== 95+0+5196 (secure 0 0 0) 0x7f2a20079e40 con 0x7f2a380cd9f0 2024-04-26T23:57:22.947 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:22.947 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":173,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:21.563296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-04-26T23:57:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 173 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.946+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 msgr2=0x7f2a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.946+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 0x7f2a1406edb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009a30 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.946+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 msgr2=0x7f2a3806f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.946+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a200046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 shutdown_connections 2024-04-26T23:57:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f2a1406c8f0 0x7f2a1406edb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3814c2e0 0x7f2a38145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a380cd9f0 0x7f2a3806f8b0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 --2- 172.21.15.46:0/3398716641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a380cb0f0 0x7f2a38161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 >> 172.21.15.46:0/3398716641 conn(0x7f2a380b26d0 msgr2=0x7f2a38162ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 shutdown_connections 2024-04-26T23:57:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:22.947+0000 7f2a37200640 1 -- 172.21.15.46:0/3398716641 wait complete. 2024-04-26T23:57:23.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 173, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:57:04.513003+0000', 'modified': '2024-04-26T23:57:21.563296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 211, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12273': {'gid': 12273, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/2610577522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2610577522}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2610577522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T23:57:23.373 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_12273:up:rejoin 2024-04-26T23:57:24.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:57:24.549 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-26T23:57:24.546+0000 7f44f6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:57:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.591+0000 7f07b6000640 1 Processor -- start 2024-04-26T23:57:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.591+0000 7f07b6000640 1 -- start start 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.593+0000 7f07b6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00d2130 0x7f07b00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.593+0000 7f07b6000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00d11f0 con 0x7f07b00b4000 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.593+0000 7f07b6000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00d5f20 con 0x7f07b015f4e0 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.593+0000 7f07b6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f07b00d0ce0 con 0x7f07b00d2130 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.593+0000 7f07afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00d2130 0x7f07b00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.594+0000 7f07afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00d2130 0x7f07b00d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45152/0 (socket says 172.21.15.46:45152) 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.594+0000 7f07afe00640 1 -- 172.21.15.46:0/1455971609 learned_addr learned my addr 172.21.15.46:0/1455971609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.594+0000 7f07afe00640 1 -- 172.21.15.46:0/1455971609 >> v1:172.21.15.142:6789/0 conn(0x7f07b015f4e0 legacy=0x7f07b00d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.594+0000 7f07afe00640 1 -- 172.21.15.46:0/1455971609 >> v1:172.21.15.46:6789/0 conn(0x7f07b00b4000 legacy=0x7f07b00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.594+0000 7f07afe00640 1 -- 172.21.15.46:0/1455971609 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00d0f10 con 0x7f07b00d2130 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.595+0000 7f07afe00640 1 --2- 172.21.15.46:0/1455971609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00d2130 0x7f07b00d2df0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd775dab99738b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07af400640 1 -- 172.21.15.46:0/1455971609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c037070 con 0x7f07b00d2130 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07af400640 1 -- 172.21.15.46:0/1455971609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c032640 con 0x7f07b00d2130 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07af400640 1 -- 172.21.15.46:0/1455971609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c032a10 con 0x7f07b00d2130 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 -- 172.21.15.46:0/1455971609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00d2130 msgr2=0x7f07b00d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 --2- 172.21.15.46:0/1455971609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00d2130 0x7f07b00d2df0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c02a320 comp rx=0 tx=0).stop 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 -- 172.21.15.46:0/1455971609 shutdown_connections 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 --2- 172.21.15.46:0/1455971609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00d2130 0x7f07b00d2df0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 -- 172.21.15.46:0/1455971609 >> 172.21.15.46:0/1455971609 conn(0x7f07b00d5010 msgr2=0x7f07b00d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.596+0000 7f07b6000640 1 -- 172.21.15.46:0/1455971609 shutdown_connections 2024-04-26T23:57:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.597+0000 7f07b6000640 1 -- 172.21.15.46:0/1455971609 wait complete. 2024-04-26T23:57:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.598+0000 7f07b6000640 1 Processor -- start 2024-04-26T23:57:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.598+0000 7f07b6000640 1 -- start start 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07b00d2130 0x7f07b014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 0x7f07b014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07b00b1d60 con 0x7f07b00b4000 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07b00d2990 con 0x7f07b00d2130 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.599+0000 7f07b6000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07b015f8c0 con 0x7f07b015f4e0 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45166/0 (socket says 172.21.15.46:45166) 2024-04-26T23:57:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 -- 172.21.15.46:0/888215124 learned_addr learned my addr 172.21.15.46:0/888215124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07afe00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07b00d2130 0x7f07b014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b5600640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 0x7f07b014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 -- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 msgr2=0x7f07b014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 0x7f07b014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 -- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07b00d2130 msgr2=0x7f07b014ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07b00d2130 0x7f07b014ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 -- 172.21.15.46:0/888215124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b014faa0 con 0x7f07b00b4000 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b5600640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 0x7f07b014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.600+0000 7f07b4c00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c450 tx=0x7f07a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.601+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40052d0 con 0x7f07b00b4000 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.601+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a4005470 con 0x7f07b00b4000 2024-04-26T23:57:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.601+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4012870 con 0x7f07b00b4000 2024-04-26T23:57:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.601+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b014fd80 con 0x7f07b00b4000 2024-04-26T23:57:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.601+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b0150340 con 0x7f07b00b4000 2024-04-26T23:57:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.602+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b00693b0 con 0x7f07b00b4000 2024-04-26T23:57:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.603+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07a4012a10 con 0x7f07b00b4000 2024-04-26T23:57:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.604+0000 7f07acc00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.605+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f07a408b970 con 0x7f07b00b4000 2024-04-26T23:57:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.605+0000 7f07afe00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:24.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.607+0000 7f07afe00640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 0x7f079006edb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f079c034000 tx=0x7f079c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.610+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07a4059bb0 con 0x7f07b00b4000 2024-04-26T23:57:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.857+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b00d33d0 con 0x7f07b00b4000 2024-04-26T23:57:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.859+0000 7f07acc00640 1 -- 172.21.15.46:0/888215124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f07a40594b0 con 0x7f07b00b4000 2024-04-26T23:57:24.861 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:57:24.861 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:57:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 msgr2=0x7f079006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 0x7f079006edb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f079c034000 tx=0x7f079c02e000 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 msgr2=0x7f07b0161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c450 tx=0x7f07a400c910 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 shutdown_connections 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07b015f4e0 0x7f07b014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07b00d2130 0x7f07b014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 --2- 172.21.15.46:0/888215124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00b4000 0x7f07b0161700 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.861+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 >> 172.21.15.46:0/888215124 conn(0x7f07b00d5010 msgr2=0x7f07b00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.862+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 shutdown_connections 2024-04-26T23:57:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:24.862+0000 7f07b6000640 1 -- 172.21.15.46:0/888215124 wait complete. 2024-04-26T23:57:25.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:57:04.513003+0000', 'modified': '2024-04-26T23:57:23.570033+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 211, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12273': {'gid': 12273, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2610577522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2610577522}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2610577522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12273, 'qdb_cluster': [12273]} 2024-04-26T23:57:25.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:57:25.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:57:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.510+0000 7f3216a00640 1 Processor -- start 2024-04-26T23:57:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.510+0000 7f3216a00640 1 -- start start 2024-04-26T23:57:25.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.511+0000 7f3216a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f321015edf0 0x7f32100cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.511+0000 7f3216a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32100c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3216a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32100b03c0 con 0x7f321015f1d0 2024-04-26T23:57:25.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3216a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f32100cf5c0 con 0x7f321015edf0 2024-04-26T23:57:25.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3216a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32100cb680 con 0x7f32100cd770 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3215600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f321015edf0 0x7f32100cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3216000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32100c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3214c00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f321015f1d0 0x7f32100cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.46:46906/0 (socket says 172.21.15.46:46906) 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.512+0000 7f3214c00640 1 -- 172.21.15.46:0/884321421 learned_addr learned my addr 172.21.15.46:0/884321421 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.513+0000 7f3215600640 1 -- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 msgr2=0x7f32100c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.513+0000 7f3215600640 1 --2- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32100c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.513+0000 7f3215600640 1 -- 172.21.15.46:0/884321421 >> v1:172.21.15.142:6789/0 conn(0x7f321015f1d0 legacy=0x7f32100cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.513+0000 7f3215600640 1 -- 172.21.15.46:0/884321421 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32100cb960 con 0x7f321015edf0 2024-04-26T23:57:25.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.515+0000 7f3215600640 1 --2- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f321015edf0 0x7f32100cf080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231d21e20ec5a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.515+0000 7f320fe00640 1 -- 172.21.15.46:0/884321421 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204007740 con 0x7f321015edf0 2024-04-26T23:57:25.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.515+0000 7f320fe00640 1 -- 172.21.15.46:0/884321421 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32040078e0 con 0x7f321015edf0 2024-04-26T23:57:25.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.515+0000 7f320fe00640 1 -- 172.21.15.46:0/884321421 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204007cb0 con 0x7f321015edf0 2024-04-26T23:57:25.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 -- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f321015edf0 msgr2=0x7f32100cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 --2- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f321015edf0 0x7f32100cf080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 -- 172.21.15.46:0/884321421 shutdown_connections 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 --2- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32100c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 --2- 172.21.15.46:0/884321421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f321015edf0 0x7f32100cf080 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 -- 172.21.15.46:0/884321421 >> 172.21.15.46:0/884321421 conn(0x7f32100b26d0 msgr2=0x7f32100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 -- 172.21.15.46:0/884321421 shutdown_connections 2024-04-26T23:57:25.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.516+0000 7f3216a00640 1 -- 172.21.15.46:0/884321421 wait complete. 2024-04-26T23:57:25.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.518+0000 7f3216a00640 1 Processor -- start 2024-04-26T23:57:25.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.519+0000 7f3216a00640 1 -- start start 2024-04-26T23:57:25.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32101f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3210151530 0x7f321015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32100cc200 con 0x7f321015edf0 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32100cbf20 con 0x7f3210151530 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3216a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32100cb1a0 con 0x7f32100cd770 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3214c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3214c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45178/0 (socket says 172.21.15.46:45178) 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3214c00640 1 -- 172.21.15.46:0/1417768958 learned_addr learned my addr 172.21.15.46:0/1417768958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.520+0000 7f3215600640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32101f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3216000640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3210151530 0x7f321015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 -- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 msgr2=0x7f32101f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32101f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 -- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3210151530 msgr2=0x7f321015bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3210151530 0x7f321015bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 -- 172.21.15.46:0/1417768958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32101f9660 con 0x7f321015edf0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.521+0000 7f3214c00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c3c0 tx=0x7f31fc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.522+0000 7f320d600640 1 -- 172.21.15.46:0/1417768958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc017070 con 0x7f321015edf0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.522+0000 7f320d600640 1 -- 172.21.15.46:0/1417768958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc005ce0 con 0x7f321015edf0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.522+0000 7f320d600640 1 -- 172.21.15.46:0/1417768958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc007310 con 0x7f321015edf0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.522+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321015c8b0 con 0x7f321015edf0 2024-04-26T23:57:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.522+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101f8920 con 0x7f321015edf0 2024-04-26T23:57:25.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.524+0000 7f320d600640 1 -- 172.21.15.46:0/1417768958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31fc01a070 con 0x7f321015edf0 2024-04-26T23:57:25.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.526+0000 7f320d600640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.526+0000 7f320d600640 1 -- 172.21.15.46:0/1417768958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f31fc08afa0 con 0x7f321015edf0 2024-04-26T23:57:25.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.526+0000 7f321af6e740 1 Processor -- start 2024-04-26T23:57:25.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.526+0000 7f321af6e740 1 -- start start 2024-04-26T23:57:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f3215600640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e47910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e45b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563af7da9600 con 0x563af7e457b0 2024-04-26T23:57:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563af7d46100 con 0x563af7e81e80 2024-04-26T23:57:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.527+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563af7da55c0 con 0x563af7e70270 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3215600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e47910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3215600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e47910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45182/0 (socket says 172.21.15.46:45182) 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3215600640 1 -- 172.21.15.46:0/774534071 learned_addr learned my addr 172.21.15.46:0/774534071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e45b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3216000640 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 -- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 msgr2=0x563af7e706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 -- 172.21.15.46:0/774534071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 msgr2=0x563af7e47910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e47910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.528+0000 7f3214c00640 1 -- 172.21.15.46:0/774534071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af7abb2d0 con 0x563af7e81e80 2024-04-26T23:57:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.529+0000 7f3215600640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 0x7f31f006edb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f32040095c0 tx=0x7f320402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.529+0000 7f3214c00640 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e45b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31fc089350 tx=0x7f31fc058d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d3032a099e22a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.529+0000 7f3201c00640 1 -- 172.21.15.46:0/774534071 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc071030 con 0x563af7e81e80 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.529+0000 7f3201c00640 1 -- 172.21.15.46:0/774534071 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc056f30 con 0x563af7e81e80 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.529+0000 7f3201c00640 1 -- 172.21.15.46:0/774534071 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0074b0 con 0x563af7e81e80 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 -- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 msgr2=0x563af7e45b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e45b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31fc089350 tx=0x7f31fc058d70 comp rx=0 tx=0).stop 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 -- 172.21.15.46:0/774534071 shutdown_connections 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e45b90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 --2- 172.21.15.46:0/774534071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e47910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 -- 172.21.15.46:0/774534071 >> 172.21.15.46:0/774534071 conn(0x563af7dd9d10 msgr2=0x563af7e8b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 -- 172.21.15.46:0/774534071 shutdown_connections 2024-04-26T23:57:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.530+0000 7f321af6e740 1 -- 172.21.15.46:0/774534071 wait complete. 2024-04-26T23:57:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.531+0000 7f321af6e740 1 -- start start 2024-04-26T23:57:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.531+0000 7f321af6e740 1 Processor -- start 2024-04-26T23:57:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563af7d30f70 con 0x563af7e457b0 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563af7e6ddc0 con 0x563af7e81e80 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563af7e65290 con 0x563af7e70270 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.532+0000 7f321af6e740 10 client.0 _ll_register_callbacks cb 0x563af7d9f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45192/0 (socket says 172.21.15.46:45192) 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 -- 172.21.15.46:0/3774793292 learned_addr learned my addr 172.21.15.46:0/3774793292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3214c00640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 -- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 msgr2=0x563af7e6c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e6c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 -- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 msgr2=0x563af7e6cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.533+0000 7f3215600640 1 -- 172.21.15.46:0/3774793292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af7b856b0 con 0x563af7e457b0 2024-04-26T23:57:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f3215600640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f320402ae90 tx=0x7f3204004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3bbc42509e156 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f31f6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204050070 con 0x563af7e457b0 2024-04-26T23:57:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f320403fb60 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc056f30 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f3204046040 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f321af6e740 10 client.12336 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.534+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x563af7e64fd0 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc056f30 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f320405e080 con 0x563af7e457b0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 10 client.12336 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 10 client.12336 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 20 client.12336 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e65c00 0x563af7e645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 4 client.12336 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af7db7210 con 0x563af7e65c00 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.535+0000 7f3214c00640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e65c00 0x563af7e645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.536+0000 7f3214c00640 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e65c00 0x563af7e645d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7267122d501a473f server_cookie=b7143022407a5a75 in_seq=0 out_seq=0 2024-04-26T23:57:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.536+0000 7f31f6000640 10 client.12336 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f31f6000640 1 -- 172.21.15.46:0/3774793292 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563af7db7210 con 0x563af7e65c00 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f31f6000640 10 client.12336 handle_command_reply: tid=0 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 client.12336 shutdown 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 msgr2=0x563af7e6ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f320402ae90 tx=0x7f3204004ab0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 shutdown_connections 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e81e80 0x563af7e6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e65c00 0x563af7e645d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 --2- 172.21.15.46:0/3774793292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e457b0 0x563af7e6ada0 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f31f6000640 0 client.12336 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 >> 172.21.15.46:0/3774793292 conn(0x563af7dd9d10 msgr2=0x563af7e6e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 shutdown_connections 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 1 -- 172.21.15.46:0/3774793292 wait complete. 2024-04-26T23:57:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.537+0000 7f321af6e740 20 client.12336 trim_cache size 0 max 16384 2024-04-26T23:57:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.557+0000 7f321af6e740 1 Processor -- start 2024-04-26T23:57:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 -- start start 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 0x563af7e3f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3210201020 con 0x563af7e81e80 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32100b40a0 con 0x563af7e457b0 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.558+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3210200ad0 con 0x563af7e70270 2024-04-26T23:57:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45200/0 (socket says 172.21.15.46:45200) 2024-04-26T23:57:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 -- 172.21.15.46:0/3289139623 learned_addr learned my addr 172.21.15.46:0/3289139623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3215600640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 0x563af7e3f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3214c00640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 -- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 msgr2=0x563af7e738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 -- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 msgr2=0x563af7e3f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 0x563af7e3f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3216000640 1 -- 172.21.15.46:0/3289139623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af7e9ae60 con 0x563af7e81e80 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.559+0000 7f3215600640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 0x563af7e3f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.560+0000 7f3216000640 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f3208009c60 tx=0x7f3208014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b6284c1b4954a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.560+0000 7f3201200640 1 -- 172.21.15.46:0/3289139623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208015420 con 0x563af7e81e80 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.560+0000 7f3201200640 1 -- 172.21.15.46:0/3289139623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208019070 con 0x563af7e81e80 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.560+0000 7f3201200640 1 -- 172.21.15.46:0/3289139623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080157c0 con 0x563af7e81e80 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 -- 172.21.15.46:0/3289139623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 msgr2=0x563af7e73de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f3208009c60 tx=0x7f3208014780 comp rx=0 tx=0).stop 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 -- 172.21.15.46:0/3289139623 shutdown_connections 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e73de0 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e70270 0x563af7e738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 --2- 172.21.15.46:0/3289139623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e457b0 0x563af7e3f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 -- 172.21.15.46:0/3289139623 >> 172.21.15.46:0/3289139623 conn(0x563af7dd9d10 msgr2=0x563af7e3bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 -- 172.21.15.46:0/3289139623 shutdown_connections 2024-04-26T23:57:25.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.561+0000 7f321af6e740 1 -- 172.21.15.46:0/3289139623 wait complete. 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.562+0000 7f321af6e740 1 -- start start 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.562+0000 7f321af6e740 1 Processor -- start 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 0x563af7e9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 0x563af7e60b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563af7e65a80 con 0x563af7e81e80 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563af7e61de0 con 0x563af7e70270 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563af7e66110 con 0x563af7e457b0 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f321af6e740 10 client.0 _ll_register_callbacks cb 0x7f3204007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f3216000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f3216000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45216/0 (socket says 172.21.15.46:45216) 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.563+0000 7f3216000640 1 -- 172.21.15.46:0/579190478 learned_addr learned my addr 172.21.15.46:0/579190478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3214c00640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 0x563af7e60b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3215600640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 0x563af7e9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 -- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 msgr2=0x563af7e9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 0x563af7e9d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 -- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 msgr2=0x563af7e60b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 0x563af7e60b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 -- 172.21.15.46:0/579190478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af7e6c9a0 con 0x563af7e81e80 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3214c00640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 0x563af7e60b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3215600640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 0x563af7e9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f3216000640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f320801f040 tx=0x7f3208009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591bb6c3708a4103 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f31f6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320802c020 con 0x563af7e81e80 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f3208003820 con 0x563af7e81e80 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.564+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3208019070 con 0x563af7e81e80 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f3208003a40 con 0x563af7e81e80 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 10 client.12342 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:25.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x563af7e9d740 con 0x563af7e81e80 2024-04-26T23:57:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208019070 con 0x563af7e81e80 2024-04-26T23:57:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f3208003be0 con 0x563af7e81e80 2024-04-26T23:57:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 10 client.12342 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 10 client.12342 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 20 client.12342 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e99970 0x563af7e99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 4 client.12342 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "status"}) -- 0x563af7db1080 con 0x563af7e99970 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.565+0000 7f3215600640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e99970 0x563af7e99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f3215600640 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e99970 0x563af7e99d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f992b679b7ccff67 server_cookie=e65208cf8e3c78de in_seq=0 out_seq=0 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f31f6000640 10 client.12342 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f31f6000640 1 -- 172.21.15.46:0/579190478 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x563af7db1080 con 0x563af7e99970 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f31f6000640 10 client.12342 handle_command_reply: tid=0 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 client.12342 shutdown 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 msgr2=0x563af7e8e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f320801f040 tx=0x7f3208009190 comp rx=0 tx=0).stop 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 shutdown_connections 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563af7e99970 0x563af7e99d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563af7e81e80 0x563af7e8e3b0 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563af7e70270 0x563af7e60b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 --2- 172.21.15.46:0/579190478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563af7e457b0 0x563af7e9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f31f6000640 0 client.12342 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.566+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 >> 172.21.15.46:0/579190478 conn(0x563af7dd9d10 msgr2=0x563af7e738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.567+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 shutdown_connections 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.567+0000 7f321af6e740 1 -- 172.21.15.46:0/579190478 wait complete. 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.567+0000 7f321af6e740 20 client.12342 trim_cache size 0 max 16384 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:57:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 12273, 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.0117450699999999, 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 174, 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 211, 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 211, 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 6.9329551079999998 2024-04-26T23:57:25.573 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 msgr2=0x7f31f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 0x7f31f006edb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f32040095c0 tx=0x7f320402c000 comp rx=0 tx=0).stop 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 msgr2=0x7f32100b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c3c0 tx=0x7f31fc00c7c0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 shutdown_connections 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f31f006c8f0 0x7f31f006edb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3210151530 0x7f321015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321015edf0 0x7f32100b54d0 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 --2- 172.21.15.46:0/1417768958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32100cd770 0x7f32101f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 >> 172.21.15.46:0/1417768958 conn(0x7f32100b26d0 msgr2=0x7f32100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 shutdown_connections 2024-04-26T23:57:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.568+0000 7f3216a00640 1 -- 172.21.15.46:0/1417768958 wait complete. 2024-04-26T23:57:25.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:25.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub start / recursive,repair 2024-04-26T23:57:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.803+0000 7f4ac2000640 1 Processor -- start 2024-04-26T23:57:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.803+0000 7f4ac2000640 1 -- start start 2024-04-26T23:57:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.805+0000 7f4ac2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4abc15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.805+0000 7f4ac2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc15fdf0 con 0x7f4abc0b47b0 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.805+0000 7f4ac2000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc1600d0 con 0x7f4abc0b2ec0 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.805+0000 7f4ac2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc1603b0 con 0x7f4abc06eac0 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.806+0000 7f4abbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4abc0b47b0 0x7f4abc072780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35692/0 (socket says 172.21.15.46:35692) 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.806+0000 7f4abbe00640 1 -- 172.21.15.46:0/2798394027 learned_addr learned my addr 172.21.15.46:0/2798394027 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.806+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4abc15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.806+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928357096 0 0) 0x7f4abc15fdf0 con 0x7f4abc0b47b0 2024-04-26T23:57:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.806+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aa0003520 con 0x7f4abc0b47b0 2024-04-26T23:57:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3601370612 0 0) 0x7f4aa0003520 con 0x7f4abc0b47b0 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 msgr2=0x7f4abc15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4abc15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 >> v1:172.21.15.142:6789/0 conn(0x7f4abc0b2ec0 legacy=0x7f4abc0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc160690 con 0x7f4abc0b47b0 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4ab4001670 con 0x7f4abc0b47b0 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 >> v1:172.21.15.46:6789/0 conn(0x7f4abc0b47b0 legacy=0x7f4abc072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa00039e0 0x7f4aa0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4aa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa0004d40 0x7f4aa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1603b0 con 0x7f4aa00039e0 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000b9c0 con 0x7f4aa0004d40 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa000bbc0 con 0x7f4abc06eac0 2024-04-26T23:57:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.807+0000 7f4abb400640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa00039e0 0x7f4aa0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4aa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abbe00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa0004d40 0x7f4aa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abb400640 1 -- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 msgr2=0x7f4aa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abb400640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc06eac0 0x7f4aa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abb400640 1 -- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa0004d40 msgr2=0x7f4aa00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abb400640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa0004d40 0x7f4aa00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abb400640 1 -- 172.21.15.46:0/2798394027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc160690 con 0x7f4aa00039e0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.808+0000 7f4abbe00640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa0004d40 0x7f4aa00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.809+0000 7f4abb400640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa00039e0 0x7f4aa0003dc0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f4aac001f30 tx=0x7f4aa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c78d4052f6d8ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.809+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac037070 con 0x7f4aa00039e0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aac032440 con 0x7f4aa00039e0 2024-04-26T23:57:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4abaa00640 1 -- 172.21.15.46:0/2798394027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac001b00 con 0x7f4aa00039e0 2024-04-26T23:57:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4ac2000640 1 -- 172.21.15.46:0/2798394027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa00039e0 msgr2=0x7f4aa0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4ac2000640 1 --2- 172.21.15.46:0/2798394027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa00039e0 0x7f4aa0003dc0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f4aac001f30 tx=0x7f4aa0005830 comp rx=0 tx=0).stop 2024-04-26T23:57:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4abbe00640 1 -- 172.21.15.46:0/2798394027 reap_dead start 2024-04-26T23:57:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4ac2000640 1 -- 172.21.15.46:0/2798394027 shutdown_connections 2024-04-26T23:57:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.810+0000 7f4ac2000640 1 -- 172.21.15.46:0/2798394027 >> 172.21.15.46:0/2798394027 conn(0x7f4abc06c770 msgr2=0x7f4abc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.811+0000 7f4ac2000640 1 -- 172.21.15.46:0/2798394027 shutdown_connections 2024-04-26T23:57:25.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.811+0000 7f4ac2000640 1 -- 172.21.15.46:0/2798394027 wait complete. 2024-04-26T23:57:25.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.812+0000 7f4ac2000640 1 Processor -- start 2024-04-26T23:57:25.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.812+0000 7f4ac2000640 1 -- start start 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.813+0000 7f4ac2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 0x7f4abc1f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.813+0000 7f4ac2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.813+0000 7f4ac2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 0x7f4abc0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.813+0000 7f4ac2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0b4040 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ac2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0b3ec0 con 0x7f4aa00039e0 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ac2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc072160 con 0x7f4abc0b1a60 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45244/0 (socket says 172.21.15.46:45244) 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1348260914 learned_addr learned my addr 172.21.15.46:0/1348260914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4abb400640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 0x7f4abc1f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4abbe00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 0x7f4abc0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 msgr2=0x7f4abc0b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 0x7f4abc0b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 msgr2=0x7f4abc1f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 0x7f4abc1f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1348260914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc1f4b10 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4abb400640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 0x7f4abc1f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.814+0000 7f4abbe00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 0x7f4abc0b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4011560 tx=0x7f4aa4011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ab3e00640 1 -- 172.21.15.46:0/1348260914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004220 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ab3e00640 1 -- 172.21.15.46:0/1348260914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa40043c0 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ab3e00640 1 -- 172.21.15.46:0/1348260914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4015870 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc144270 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.815+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc0aca20 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.818+0000 7f4ab3e00640 1 -- 172.21.15.46:0/1348260914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4aa4015a10 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.819+0000 7f4ab3e00640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 0x7f4a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.819+0000 7f4ab3e00640 1 -- 172.21.15.46:0/1348260914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4aa408bf90 con 0x7f4abc0ac2b0 2024-04-26T23:57:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.820+0000 7f4ac4fdc740 1 Processor -- start 2024-04-26T23:57:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.820+0000 7f4abb400640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 0x7f4a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.820+0000 7f4ac4fdc740 1 -- start start 2024-04-26T23:57:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6eef29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eec82f0 0x559c6ef0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c6ee2c980 con 0x559c6ef0feb0 2024-04-26T23:57:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c6edc9220 con 0x559c6eef2610 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559c6ee28940 con 0x559c6eec82f0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.821+0000 7f4abb400640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 0x7f4a9806edb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4aac034000 tx=0x7f4aac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45246/0 (socket says 172.21.15.46:45246) 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 -- 172.21.15.46:0/14688686 learned_addr learned my addr 172.21.15.46:0/14688686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4abb400640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6eef29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4abbe00640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eec82f0 0x559c6ef0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 -- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eec82f0 msgr2=0x559c6ef0e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eec82f0 0x559c6ef0e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 -- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 msgr2=0x559c6eef29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6eef29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 -- 172.21.15.46:0/14688686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6ec067e0 con 0x559c6ef0feb0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4abb400640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6eef29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.822+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40599d0 tx=0x7f4aa4059b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5493b80e09e3bb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4aab400640 1 -- 172.21.15.46:0/14688686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4022030 con 0x559c6ef0feb0 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4aab400640 1 -- 172.21.15.46:0/14688686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa4057d40 con 0x559c6ef0feb0 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4aab400640 1 -- 172.21.15.46:0/14688686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4083ba0 con 0x559c6ef0feb0 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/14688686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 msgr2=0x559c6ef10400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40599d0 tx=0x7f4aa4059b00 comp rx=0 tx=0).stop 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/14688686 shutdown_connections 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eec82f0 0x559c6ef0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6ef0feb0 0x559c6ef10400 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/14688686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6eef29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/14688686 >> 172.21.15.46:0/14688686 conn(0x559c6eec4c20 msgr2=0x559c6ef0d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.823+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/14688686 shutdown_connections 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.824+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/14688686 wait complete. 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.824+0000 7f4ac4fdc740 1 -- start start 2024-04-26T23:57:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.824+0000 7f4ac4fdc740 1 Processor -- start 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 0x559c6ef1d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 0x559c6ef0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c6eec9090 con 0x559c6eef6480 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c6edb4170 con 0x559c6ef0feb0 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559c6eec2b50 con 0x559c6eef2610 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.825+0000 7f4ac4fdc740 10 client.0 _ll_register_callbacks cb 0x559c6ee23770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45248/0 (socket says 172.21.15.46:45248) 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 -- 172.21.15.46:0/2142326727 learned_addr learned my addr 172.21.15.46:0/2142326727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 0x559c6ef0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abb400640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 0x559c6ef1d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 -- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 msgr2=0x559c6ef1d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 0x559c6ef1d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 -- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 msgr2=0x559c6ef0e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 0x559c6ef0e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abbe00640 1 -- 172.21.15.46:0/2142326727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6ec66f70 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 0x559c6ef0e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.826+0000 7f4abb400640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 0x559c6ef1d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4abbe00640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f4abc06ea10 tx=0x7f4ab400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b093133e5048c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4019060 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4ab4014ce0 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4057d40 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f4ab40072e0 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4ac4fdc740 10 client.12354 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x559c6eafe170 con 0x559c6eef6480 2024-04-26T23:57:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.827+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa4057d40 con 0x559c6eef6480 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f4ab4007480 con 0x559c6eef6480 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 10 client.12354 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 10 client.12354 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 20 client.12354 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef20f20 0x559c6ef21330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 4 client.12354 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c6ee3a590 con 0x559c6ef20f20 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.828+0000 7f4abb400640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef20f20 0x559c6ef21330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.829+0000 7f4abb400640 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef20f20 0x559c6ef21330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14338a9a36d37689 server_cookie=1972c9761845f9dd in_seq=0 out_seq=0 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.829+0000 7f4a9fe00640 10 client.12354 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.829+0000 7f4a9fe00640 1 -- 172.21.15.46:0/2142326727 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c6ee3a590 con 0x559c6ef20f20 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.829+0000 7f4a9fe00640 10 client.12354 handle_command_reply: tid=0 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 client.12354 shutdown 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 msgr2=0x559c6ef0ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f4abc06ea10 tx=0x7f4ab400f6c0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 shutdown_connections 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef20f20 0x559c6ef21330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef0ea10 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6ef0feb0 0x559c6ef0e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/2142326727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6eef2610 0x559c6ef1d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4a9fe00640 0 client.12354 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 >> 172.21.15.46:0/2142326727 conn(0x559c6eec82f0 msgr2=0x559c6ef11230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 shutdown_connections 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/2142326727 wait complete. 2024-04-26T23:57:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.830+0000 7f4ac4fdc740 20 client.12354 trim_cache size 0 max 16384 2024-04-26T23:57:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.850+0000 7f4ac4fdc740 1 Processor -- start 2024-04-26T23:57:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.850+0000 7f4ac4fdc740 1 -- start start 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef1fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6eebde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1fe1e0 con 0x559c6eef6480 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1fd220 con 0x559c6eef2610 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc0ae800 con 0x559c6ef0feb0 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45256/0 (socket says 172.21.15.46:45256) 2024-04-26T23:57:25.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 -- 172.21.15.46:0/576028256 learned_addr learned my addr 172.21.15.46:0/576028256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4abbe00640 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6eebde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4abb400640 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef1fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 -- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 msgr2=0x559c6eebde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6eebde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 -- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 msgr2=0x559c6ef1fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef1fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.851+0000 7f4ab2a00640 1 -- 172.21.15.46:0/576028256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6ef7e0a0 con 0x559c6eef6480 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.852+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40596f0 tx=0x7f4aa4059940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689c5207bb2b4037 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.852+0000 7f4aaaa00640 1 -- 172.21.15.46:0/576028256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4022070 con 0x559c6eef6480 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.852+0000 7f4aaaa00640 1 -- 172.21.15.46:0/576028256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa40560b0 con 0x559c6eef6480 2024-04-26T23:57:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.852+0000 7f4aaaa00640 1 -- 172.21.15.46:0/576028256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4083de0 con 0x559c6eef6480 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/576028256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 msgr2=0x559c6ef7db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40596f0 tx=0x7f4aa4059940 comp rx=0 tx=0).stop 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/576028256 shutdown_connections 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6eebde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef7db60 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/576028256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef1fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/576028256 >> 172.21.15.46:0/576028256 conn(0x559c6eec82f0 msgr2=0x559c6ef06600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/576028256 shutdown_connections 2024-04-26T23:57:25.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.853+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/576028256 wait complete. 2024-04-26T23:57:25.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.854+0000 7f4ac4fdc740 1 -- start start 2024-04-26T23:57:25.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.854+0000 7f4ac4fdc740 1 Processor -- start 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef75830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6ef107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1fcc00 con 0x559c6eef6480 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c6eec3d60 con 0x559c6eef2610 2024-04-26T23:57:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x559c6ef7c950 con 0x559c6ef0feb0 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ac4fdc740 10 client.0 _ll_register_callbacks cb 0x7f4ab4001610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.855+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45268/0 (socket says 172.21.15.46:45268) 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1458325721 learned_addr learned my addr 172.21.15.46:0/1458325721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4abbe00640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6ef107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4abb400640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef75830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 msgr2=0x559c6ef107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6ef107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 msgr2=0x559c6ef75830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef75830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 -- 172.21.15.46:0/1458325721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6ef78400 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4ab2a00640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40599d0 tx=0x7f4aa401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32067dc5c87a6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.856+0000 7f4a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4022070 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4aa407bc90 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa40560b0 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f4aa407be90 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 10 client.12360 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x559c6ef7c690 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa40560b0 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f4aa4025060 con 0x559c6eef6480 2024-04-26T23:57:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 10 client.12360 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 10 client.12360 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 20 client.12360 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef7db60 0x559c6ef10d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 4 client.12360 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.857+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x559c6ee34400 con 0x559c6ef7db60 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.858+0000 7f4abb400640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef7db60 0x559c6ef10d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.858+0000 7f4abb400640 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef7db60 0x559c6ef10d30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1464eb018c4d9 server_cookie=38987f0365143edd in_seq=0 out_seq=0 2024-04-26T23:57:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.858+0000 7f4a9fe00640 10 client.12360 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.859+0000 7f4a9fe00640 1 -- 172.21.15.46:0/1458325721 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c6ee34400 con 0x559c6ef7db60 2024-04-26T23:57:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.859+0000 7f4a9fe00640 10 client.12360 handle_command_reply: tid=0 2024-04-26T23:57:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.859+0000 7f4ac4fdc740 1 client.12360 shutdown 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.859+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 msgr2=0x559c6ef03800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.859+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40599d0 tx=0x7f4aa401f040 comp rx=0 tx=0).stop 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 shutdown_connections 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559c6ef0feb0 0x559c6ef107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x559c6ef7db60 0x559c6ef10d30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c6eef6480 0x559c6ef03800 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 --2- 172.21.15.46:0/1458325721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c6eef2610 0x559c6ef75830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 >> 172.21.15.46:0/1458325721 conn(0x559c6eec82f0 msgr2=0x559c6ef1fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4a9fe00640 0 client.12360 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 shutdown_connections 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 1 -- 172.21.15.46:0/1458325721 wait complete. 2024-04-26T23:57:25.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.860+0000 7f4ac4fdc740 20 client.12360 trim_cache size 0 max 16384 2024-04-26T23:57:25.865 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:25.865 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:57:25.865 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "2c4a3ad9-c0d1-4ab6-870b-002ce4acf2b4", 2024-04-26T23:57:25.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:57:25.865 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 msgr2=0x7f4a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 0x7f4a9806edb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4aac034000 tx=0x7f4aac02e000 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 msgr2=0x7f4abc146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4011560 tx=0x7f4aa4011a20 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 shutdown_connections 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a9806c8f0 0x7f4a9806edb0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4abc0b1a60 0x7f4abc0b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4abc0ac2b0 0x7f4abc146b70 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 --2- 172.21.15.46:0/1348260914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa00039e0 0x7f4abc1f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.861+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 >> 172.21.15.46:0/1348260914 conn(0x7f4abc06c770 msgr2=0x7f4abc158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.862+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 shutdown_connections 2024-04-26T23:57:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:25.862+0000 7f4ac2000640 1 -- 172.21.15.46:0/1348260914 wait complete. 2024-04-26T23:57:25.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:25.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub status 2024-04-26T23:57:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.091+0000 7f4a28e00640 1 Processor -- start 2024-04-26T23:57:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.092+0000 7f4a28e00640 1 -- start start 2024-04-26T23:57:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.093+0000 7f4a28e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4a2415d730 0x7f4a240d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.093+0000 7f4a28e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a2406eb90 con 0x7f4a240b53b0 2024-04-26T23:57:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.093+0000 7f4a28e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a240b4310 con 0x7f4a2415e220 2024-04-26T23:57:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.093+0000 7f4a28e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4a240b45f0 con 0x7f4a2415d730 2024-04-26T23:57:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.094+0000 7f4a21400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4a240b53b0 0x7f4a2415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35708/0 (socket says 172.21.15.46:35708) 2024-04-26T23:57:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.094+0000 7f4a21400640 1 -- 172.21.15.46:0/2084198926 learned_addr learned my addr 172.21.15.46:0/2084198926 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.094+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2084198926 >> v2:172.21.15.142:3300/0 conn(0x7f4a2415d730 0x7f4a240d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.094+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795944109 0 0) 0x7f4a2406eb90 con 0x7f4a240b53b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.094+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a04003520 con 0x7f4a240b53b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 107834399 0 0) 0x7f4a04003520 con 0x7f4a240b53b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 >> v2:172.21.15.142:3300/0 conn(0x7f4a2415d730 msgr2=0x7f4a240d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 --2- 172.21.15.46:0/2084198926 >> v2:172.21.15.142:3300/0 conn(0x7f4a2415d730 0x7f4a240d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 >> v1:172.21.15.142:6789/0 conn(0x7f4a2415e220 legacy=0x7f4a2406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a240b48d0 con 0x7f4a240b53b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f4a0c0019f0 con 0x7f4a240b53b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 >> v1:172.21.15.46:6789/0 conn(0x7f4a240b53b0 legacy=0x7f4a2415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039b0 0x7f4a04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a04004800 0x7f4a04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b45f0 con 0x7f4a040051c0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0400c060 con 0x7f4a040039b0 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0400c260 con 0x7f4a04004800 2024-04-26T23:57:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a22800640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.095+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039b0 0x7f4a04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a21400640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a04004800 0x7f4a04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a22800640 1 -- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a04004800 msgr2=0x7f4a04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a22800640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a04004800 0x7f4a04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a22800640 1 -- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039b0 msgr2=0x7f4a04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a22800640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039b0 0x7f4a04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a22800640 1 -- 172.21.15.46:0/2084198926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a04003520 con 0x7f4a040051c0 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a21400640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a04004800 0x7f4a04004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.096+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039b0 0x7f4a04003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.097+0000 7f4a22800640 1 -- 172.21.15.46:0/2084198926 reap_dead start 2024-04-26T23:57:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.097+0000 7f4a22800640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a0400b9c0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c001750 tx=0x7f4a04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfe2a0406fb92e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.097+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c001370 con 0x7f4a040051c0 2024-04-26T23:57:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.097+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c0052f0 con 0x7f4a040051c0 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.097+0000 7f4a20a00640 1 -- 172.21.15.46:0/2084198926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c02abe0 con 0x7f4a040051c0 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 -- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 msgr2=0x7f4a0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a0400b9c0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c001750 tx=0x7f4a04005830 comp rx=0 tx=0).stop 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 -- 172.21.15.46:0/2084198926 shutdown_connections 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 --2- 172.21.15.46:0/2084198926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a0400b9c0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 -- 172.21.15.46:0/2084198926 >> 172.21.15.46:0/2084198926 conn(0x7f4a2406c980 msgr2=0x7f4a240b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 -- 172.21.15.46:0/2084198926 shutdown_connections 2024-04-26T23:57:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.098+0000 7f4a28e00640 1 -- 172.21.15.46:0/2084198926 wait complete. 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.100+0000 7f4a28e00640 1 Processor -- start 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.100+0000 7f4a28e00640 1 -- start start 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 0x7f4a2415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 0x7f4a2413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b3ec0 con 0x7f4a040051c0 2024-04-26T23:57:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b4e90 con 0x7f4a2413e380 2024-04-26T23:57:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a28e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a240b4040 con 0x7f4a241e61e0 2024-04-26T23:57:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45292/0 (socket says 172.21.15.46:45292) 2024-04-26T23:57:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a21e00640 1 -- 172.21.15.46:0/3671198376 learned_addr learned my addr 172.21.15.46:0/3671198376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a21400640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 0x7f4a2415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.101+0000 7f4a22800640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 0x7f4a2413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 -- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 msgr2=0x7f4a2415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 0x7f4a2415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 -- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 msgr2=0x7f4a2413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 0x7f4a2413e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 -- 172.21.15.46:0/3671198376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2413ed40 con 0x7f4a040051c0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21400640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 0x7f4a2415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a22800640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 0x7f4a2413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a21e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f4a14011540 tx=0x7f4a14011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a1a000640 1 -- 172.21.15.46:0/3671198376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14015870 con 0x7f4a040051c0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a1a000640 1 -- 172.21.15.46:0/3671198376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a14015a10 con 0x7f4a040051c0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a1a000640 1 -- 172.21.15.46:0/3671198376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14015dc0 con 0x7f4a040051c0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.102+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a241e7250 con 0x7f4a040051c0 2024-04-26T23:57:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.103+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a241e7810 con 0x7f4a040051c0 2024-04-26T23:57:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.105+0000 7f4a1a000640 1 -- 172.21.15.46:0/3671198376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a1400f460 con 0x7f4a040051c0 2024-04-26T23:57:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.106+0000 7f4a1a000640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.106+0000 7f4a1a000640 1 -- 172.21.15.46:0/3671198376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4a1408c520 con 0x7f4a040051c0 2024-04-26T23:57:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.106+0000 7f4a21400640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.107+0000 7f4a2bbcb740 1 Processor -- start 2024-04-26T23:57:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.107+0000 7f4a2bbcb740 1 -- start start 2024-04-26T23:57:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a21400640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 0x7f4a0006edb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001580 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c0d540 0x55a5a0c0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c2b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0af3030 con 0x55a5a0be3220 2024-04-26T23:57:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0b0e7f0 con 0x55a5a0c2ade0 2024-04-26T23:57:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a5a0ac97e0 con 0x55a5a0c0d540 2024-04-26T23:57:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45296/0 (socket says 172.21.15.46:45296) 2024-04-26T23:57:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a22800640 1 -- 172.21.15.46:0/1705299223 learned_addr learned my addr 172.21.15.46:0/1705299223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:26.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c0d540 0x55a5a0c0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.108+0000 7f4a21400640 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c2b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 -- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c0d540 msgr2=0x55a5a0c0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c0d540 0x55a5a0c0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 -- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c2ade0 msgr2=0x55a5a0c2b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c2b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 -- 172.21.15.46:0/1705299223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a0ad5050 con 0x55a5a0be3220 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a22800640 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a590 tx=0x7f4a1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33d06ca8a205b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a12a00640 1 -- 172.21.15.46:0/1705299223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0050d0 con 0x55a5a0be3220 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a12a00640 1 -- 172.21.15.46:0/1705299223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c005270 con 0x55a5a0be3220 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.109+0000 7f4a12a00640 1 -- 172.21.15.46:0/1705299223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c02a9a0 con 0x55a5a0be3220 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1705299223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 msgr2=0x55a5a0c294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a590 tx=0x7f4a1c02c000 comp rx=0 tx=0).stop 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1705299223 shutdown_connections 2024-04-26T23:57:26.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0be3220 0x55a5a0c294a0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c2b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1705299223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c0d540 0x55a5a0c0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1705299223 >> 172.21.15.46:0/1705299223 conn(0x55a5a0bdfb50 msgr2=0x55a5a0c28350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1705299223 shutdown_connections 2024-04-26T23:57:26.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.110+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1705299223 wait complete. 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.111+0000 7f4a2bbcb740 1 -- start start 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.111+0000 7f4a2bbcb740 1 Processor -- start 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0bffe20 0x55a5a0c223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0b4b200 con 0x55a5a0c2ade0 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0b45ee0 con 0x55a5a0c0d540 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a5a0b52e70 con 0x55a5a0bffe20 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.112+0000 7f4a2bbcb740 10 client.0 _ll_register_callbacks cb 0x55a5a0b3c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45298/0 (socket says 172.21.15.46:45298) 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 -- 172.21.15.46:0/1217706664 learned_addr learned my addr 172.21.15.46:0/1217706664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a22800640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0bffe20 0x55a5a0c223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 -- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0bffe20 msgr2=0x55a5a0c223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0bffe20 0x55a5a0c223a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 -- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 msgr2=0x55a5a0c0fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.113+0000 7f4a21400640 1 -- 172.21.15.46:0/1217706664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a09227d0 con 0x55a5a0c2ade0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a21400640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0076c0 tx=0x7f4a240b44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6de70d4b032dca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c027060 con 0x55a5a0c2ade0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c005270 con 0x55a5a0c2ade0 2024-04-26T23:57:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4a0c022c70 con 0x55a5a0c2ade0 2024-04-26T23:57:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c015a20 con 0x55a5a0c2ade0 2024-04-26T23:57:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a2bbcb740 10 client.12372 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.114+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x55a5a08583f0 con 0x55a5a0c2ade0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c005270 con 0x55a5a0c2ade0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f4a0c015bc0 con 0x55a5a0c2ade0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 10 client.12372 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 10 client.12372 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 20 client.12372 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c3b400 0x55a5a0c3b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 4 client.12372 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5a0b547e0 con 0x55a5a0c3b400 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.115+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c3b400 0x55a5a0c3b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.116+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c3b400 0x55a5a0c3b840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd8b0a1b9065412 server_cookie=5595c683e1e24c8b in_seq=0 out_seq=0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.116+0000 7f4a0b400640 10 client.12372 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.116+0000 7f4a0b400640 1 -- 172.21.15.46:0/1217706664 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5a0b547e0 con 0x55a5a0c3b400 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.116+0000 7f4a0b400640 10 client.12372 handle_command_reply: tid=0 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.116+0000 7f4a2bbcb740 1 client.12372 shutdown 2024-04-26T23:57:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 msgr2=0x55a5a0c102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0076c0 tx=0x7f4a240b44e0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 shutdown_connections 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0bffe20 0x55a5a0c223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c3b400 0x55a5a0c3b840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c2ade0 0x55a5a0c102c0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1217706664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a0b400640 0 client.12372 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 >> 172.21.15.46:0/1217706664 conn(0x55a5a0be3220 msgr2=0x55a5a0c2c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 shutdown_connections 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1217706664 wait complete. 2024-04-26T23:57:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.117+0000 7f4a2bbcb740 20 client.12372 trim_cache size 0 max 16384 2024-04-26T23:57:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.136+0000 7f4a2bbcb740 1 Processor -- start 2024-04-26T23:57:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 -- start start 2024-04-26T23:57:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c0be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24202600 con 0x55a5a0bffe20 2024-04-26T23:57:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a242015e0 con 0x55a5a0c0d540 2024-04-26T23:57:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.137+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4a24200a40 con 0x55a5a0c2ade0 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45302/0 (socket says 172.21.15.46:45302) 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 -- 172.21.15.46:0/1411169782 learned_addr learned my addr 172.21.15.46:0/1411169782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21400640 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 -- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 msgr2=0x55a5a0c0be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c0be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 -- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 msgr2=0x55a5a0c0b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.138+0000 7f4a21e00640 1 -- 172.21.15.46:0/1411169782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a08180f0 con 0x55a5a0bffe20 2024-04-26T23:57:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a21e00640 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f4a1405a090 tx=0x7f4a1408a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd4520a195ce1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a12000640 1 -- 172.21.15.46:0/1411169782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14022040 con 0x55a5a0bffe20 2024-04-26T23:57:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a12000640 1 -- 172.21.15.46:0/1411169782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a14058400 con 0x55a5a0bffe20 2024-04-26T23:57:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a12000640 1 -- 172.21.15.46:0/1411169782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1405e0c0 con 0x55a5a0bffe20 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1411169782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 msgr2=0x55a5a0c0ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f4a1405a090 tx=0x7f4a1408a7d0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1411169782 shutdown_connections 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c0be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0c0d540 0x55a5a0c0b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/1411169782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0bffe20 0x55a5a0c0ff70 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.139+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1411169782 >> 172.21.15.46:0/1411169782 conn(0x55a5a0be3220 msgr2=0x55a5a0bd8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.140+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1411169782 shutdown_connections 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.140+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/1411169782 wait complete. 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.140+0000 7f4a2bbcb740 1 -- start start 2024-04-26T23:57:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.140+0000 7f4a2bbcb740 1 Processor -- start 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0bffe20 0x55a5a0c2ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0b53fd0 con 0x55a5a0c0d540 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24200fc0 con 0x55a5a0bffe20 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a5a0c4b460 con 0x55a5a0c2ade0 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.141+0000 7f4a2bbcb740 10 client.0 _ll_register_callbacks cb 0x7f4a0c001ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45308/0 (socket says 172.21.15.46:45308) 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 -- 172.21.15.46:0/2194681452 learned_addr learned my addr 172.21.15.46:0/2194681452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a22800640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 -- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 msgr2=0x55a5a0c206c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 -- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0bffe20 msgr2=0x55a5a0c2ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0bffe20 0x55a5a0c2ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0bffe20 0x55a5a0c2ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 -- 172.21.15.46:0/2194681452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a0b0ea30 con 0x55a5a0c0d540 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a21400640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009010 tx=0x7f4a0c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c0731ed28a76cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.142+0000 7f4a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c022660 con 0x55a5a0c0d540 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f4a0c01e040 con 0x55a5a0c0d540 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a14058400 con 0x55a5a0c0d540 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c022800 con 0x55a5a0c0d540 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 10 client.12378 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x55a5a0c4b1a0 con 0x55a5a0c0d540 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a14058400 con 0x55a5a0c0d540 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f4a0c03e050 con 0x55a5a0c0d540 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 10 client.12378 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 10 client.12378 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 20 client.12378 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c0c9f0 0x55a5a0c20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 4 client.12378 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.143+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5a0c24310 con 0x55a5a0c0c9f0 2024-04-26T23:57:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.144+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c0c9f0 0x55a5a0c20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.144+0000 7f4a21e00640 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c0c9f0 0x55a5a0c20c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9fbf4b391105888 server_cookie=97cc742a808dbb09 in_seq=0 out_seq=0 2024-04-26T23:57:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.144+0000 7f4a0b400640 10 client.12378 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.144+0000 7f4a0b400640 1 -- 172.21.15.46:0/2194681452 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55a5a0c24310 con 0x55a5a0c0c9f0 2024-04-26T23:57:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a0b400640 10 client.12378 handle_command_reply: tid=0 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 client.12378 shutdown 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 msgr2=0x55a5a0c4a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009010 tx=0x7f4a0c007000 comp rx=0 tx=0).stop 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 shutdown_connections 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a5a0c2ade0 0x55a5a0c206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55a5a0c0c9f0 0x55a5a0c20c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a0c0d540 0x55a5a0c4a890 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 --2- 172.21.15.46:0/2194681452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5a0bffe20 0x55a5a0c2ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a0b400640 0 client.12378 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 >> 172.21.15.46:0/2194681452 conn(0x55a5a0be3220 msgr2=0x55a5a0c10120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 shutdown_connections 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 1 -- 172.21.15.46:0/2194681452 wait complete. 2024-04-26T23:57:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.145+0000 7f4a2bbcb740 20 client.12378 trim_cache size 0 max 16384 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "2c4a3ad9-c0d1-4ab6-870b-002ce4acf2b4": { 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "2c4a3ad9-c0d1-4ab6-870b-002ce4acf2b4", 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:26.151 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 msgr2=0x7f4a0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 0x7f4a0006edb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001580 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2024-04-26T23:57:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 msgr2=0x7f4a241e6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f4a14011540 tx=0x7f4a14011a00 comp rx=0 tx=0).stop 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 shutdown_connections 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a2413e380 0x7f4a2413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a241e61e0 0x7f4a2415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 --2- 172.21.15.46:0/3671198376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a040051c0 0x7f4a241e6ae0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 >> 172.21.15.46:0/3671198376 conn(0x7f4a2406c980 msgr2=0x7f4a240aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 shutdown_connections 2024-04-26T23:57:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:26.146+0000 7f4a28e00640 1 -- 172.21.15.46:0/3671198376 wait complete. 2024-04-26T23:57:26.168 INFO:tasks.cephfs.filesystem:scrub status for tag:2c4a3ad9-c0d1-4ab6-870b-002ce4acf2b4 - {'path': '/', 'tag': '2c4a3ad9-c0d1-4ab6-870b-002ce4acf2b4', 'options': 'recursive,repair'} 2024-04-26T23:57:44.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:44.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:56.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:56.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub status 2024-04-26T23:57:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.388+0000 7f9ff7e00640 1 Processor -- start 2024-04-26T23:57:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.389+0000 7f9ff7e00640 1 -- start start 2024-04-26T23:57:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.390+0000 7f9ff7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ff80d3830 0x7f9ff80d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.390+0000 7f9ff7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ff80d1cb0 0x7f9ff80b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.390+0000 7f9ff7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff80d20c0 con 0x7f9ff8162a70 2024-04-26T23:57:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.390+0000 7f9ff7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff80d3400 con 0x7f9ff80d3830 2024-04-26T23:57:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.390+0000 7f9ff7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff80d2370 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ff8162a70 0x7f9ff80d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44496/0 (socket says 172.21.15.46:44496) 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ff80d3830 0x7f9ff80d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff6a00640 1 -- 172.21.15.46:0/1882034591 learned_addr learned my addr 172.21.15.46:0/1882034591 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff7400640 1 --2- 172.21.15.46:0/1882034591 >> v2:172.21.15.142:3300/0 conn(0x7f9ff80d1cb0 0x7f9ff80b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff6000640 1 -- 172.21.15.46:0/1882034591 >> v2:172.21.15.142:3300/0 conn(0x7f9ff80d1cb0 msgr2=0x7f9ff80b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.391+0000 7f9ff6000640 1 --2- 172.21.15.46:0/1882034591 >> v2:172.21.15.142:3300/0 conn(0x7f9ff80d1cb0 0x7f9ff80b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.392+0000 7f9ff6000640 1 -- 172.21.15.46:0/1882034591 >> v1:172.21.15.46:6789/0 conn(0x7f9ff8162a70 legacy=0x7f9ff80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.392+0000 7f9ff6000640 1 -- 172.21.15.46:0/1882034591 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff80d2650 con 0x7f9ff80d3830 2024-04-26T23:57:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.392+0000 7f9ff5600640 1 -- 172.21.15.46:0/1882034591 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1275114611 0 0) 0x7f9ff80d20c0 con 0x7f9ff8162a70 2024-04-26T23:57:56.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff6000640 1 --2- 172.21.15.46:0/1882034591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d3830 0x7f9ff80d2e20 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80cf500 tx=0x7f9fe002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd82b08d4822930c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff5600640 1 -- 172.21.15.46:0/1882034591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007740 con 0x7f9ff80d3830 2024-04-26T23:57:56.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff5600640 1 -- 172.21.15.46:0/1882034591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe00078e0 con 0x7f9ff80d3830 2024-04-26T23:57:56.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff5600640 1 -- 172.21.15.46:0/1882034591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007cb0 con 0x7f9ff80d3830 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff7e00640 1 -- 172.21.15.46:0/1882034591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d3830 msgr2=0x7f9ff80d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.393+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/1882034591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d3830 0x7f9ff80d2e20 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80cf500 tx=0x7f9fe002a710 comp rx=0 tx=0).stop 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 -- 172.21.15.46:0/1882034591 shutdown_connections 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/1882034591 >> v2:172.21.15.142:3300/0 conn(0x7f9ff80d1cb0 0x7f9ff80b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/1882034591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d3830 0x7f9ff80d2e20 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 -- 172.21.15.46:0/1882034591 >> 172.21.15.46:0/1882034591 conn(0x7f9ff80d4f20 msgr2=0x7f9ff80d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 -- 172.21.15.46:0/1882034591 shutdown_connections 2024-04-26T23:57:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.394+0000 7f9ff7e00640 1 -- 172.21.15.46:0/1882034591 wait complete. 2024-04-26T23:57:56.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.396+0000 7f9ff7e00640 1 Processor -- start 2024-04-26T23:57:56.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.396+0000 7f9ff7e00640 1 -- start start 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ff80d3830 0x7f9ff80ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8162a70 0x7f9ff814b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8162fd0 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8162e50 con 0x7f9ff8162a70 2024-04-26T23:57:56.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff80b2f20 con 0x7f9ff80d3830 2024-04-26T23:57:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60094/0 (socket says 172.21.15.46:60094) 2024-04-26T23:57:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4206274848 learned_addr learned my addr 172.21.15.46:0/4206274848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff6000640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ff80d3830 0x7f9ff80ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.397+0000 7f9ff7400640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8162a70 0x7f9ff814b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ff80d3830 msgr2=0x7f9ff80ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ff80d3830 0x7f9ff80ba5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8162a70 msgr2=0x7f9ff814b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8162a70 0x7f9ff814b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4206274848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff814bde0 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800c460 tx=0x7f9fe800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9feea00640 1 -- 172.21.15.46:0/4206274848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8018070 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9feea00640 1 -- 172.21.15.46:0/4206274848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe8007500 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9feea00640 1 -- 172.21.15.46:0/4206274848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8004a20 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff80b9520 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.398+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff80b9ae0 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.401+0000 7f9feea00640 1 -- 172.21.15.46:0/4206274848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fe801b070 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.402+0000 7f9feea00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 0x7f9fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.402+0000 7f9feea00640 1 -- 172.21.15.46:0/4206274848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f9fe808aff0 con 0x7f9ff80d1cb0 2024-04-26T23:57:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.403+0000 7fa000bef740 1 Processor -- start 2024-04-26T23:57:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.403+0000 7f9ff6000640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 0x7f9fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.403+0000 7fa000bef740 1 -- start start 2024-04-26T23:57:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f0a7a5030 con 0x561f0a8953f0 2024-04-26T23:57:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f0a7c07f0 con 0x561f0a8dcfb0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561f0a77b7e0 con 0x561f0a8bf710 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60100/0 (socket says 172.21.15.46:60100) 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff7400640 1 -- 172.21.15.46:0/48582301 learned_addr learned my addr 172.21.15.46:0/48582301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff6000640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 0x7f9fd406edb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff6000640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff7400640 1 -- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 msgr2=0x561f0a8bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.404+0000 7f9ff7400640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9ff7400640 1 -- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 msgr2=0x561f0a8dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9ff7400640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8dd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9ff7400640 1 -- 172.21.15.46:0/48582301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f0a787050 con 0x561f0a8953f0 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9ff6000640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9ff7400640 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009930 tx=0x7f9ff0009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe664fcb6497610 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9fe7400640 1 -- 172.21.15.46:0/48582301 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0017020 con 0x561f0a8953f0 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9fe7400640 1 -- 172.21.15.46:0/48582301 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff000ea10 con 0x561f0a8953f0 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.405+0000 7f9fe7400640 1 -- 172.21.15.46:0/48582301 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000f040 con 0x561f0a8953f0 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 -- 172.21.15.46:0/48582301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 msgr2=0x561f0a8db670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009930 tx=0x7f9ff0009df0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 -- 172.21.15.46:0/48582301 shutdown_connections 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8953f0 0x561f0a8db670 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 --2- 172.21.15.46:0/48582301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 -- 172.21.15.46:0/48582301 >> 172.21.15.46:0/48582301 conn(0x561f0a891d20 msgr2=0x561f0a8da520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 -- 172.21.15.46:0/48582301 shutdown_connections 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.406+0000 7fa000bef740 1 -- 172.21.15.46:0/48582301 wait complete. 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.407+0000 7fa000bef740 1 -- start start 2024-04-26T23:57:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.407+0000 7fa000bef740 1 Processor -- start 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8dcfb0 0x561f0a8bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8bcfe0 0x561f0a8daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f0a7fd200 con 0x561f0a8bf710 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f0a7f7ee0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561f0a804e70 con 0x561f0a8dcfb0 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7fa000bef740 10 client.0 _ll_register_callbacks cb 0x561f0a7ee270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60104/0 (socket says 172.21.15.46:60104) 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.408+0000 7f9ff6a00640 1 -- 172.21.15.46:0/954504437 learned_addr learned my addr 172.21.15.46:0/954504437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 -- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8dcfb0 msgr2=0x561f0a8bc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8dcfb0 0x561f0a8bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 -- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8bcfe0 msgr2=0x561f0a8daf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8bcfe0 0x561f0a8daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 -- 172.21.15.46:0/954504437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f0a5d47d0 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800cfd0 tx=0x7f9fe8059350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9c7fe7aeb27b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8004dc0 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f9fe8057f30 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.409+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff000ea10 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f9fe8058130 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 10 client.12393 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x561f0a50a3f0 con 0x561f0a8bf710 2024-04-26T23:57:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff000ea10 con 0x561f0a8bf710 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f9fe805f0a0 con 0x561f0a8bf710 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 10 client.12393 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 10 client.12393 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 20 client.12393 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8edaa0 0x561f0a8edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 4 client.12393 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f0a8067e0 con 0x561f0a8edaa0 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.410+0000 7f9ff6000640 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8edaa0 0x561f0a8edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.411+0000 7f9ff6000640 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8edaa0 0x561f0a8edf00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d610ae866a24b272 server_cookie=9f28d6afe7aedc9 in_seq=0 out_seq=0 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.411+0000 7f9fdfe00640 10 client.12393 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7f9fdfe00640 1 -- 172.21.15.46:0/954504437 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f0a8067e0 con 0x561f0a8edaa0 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7f9fdfe00640 10 client.12393 handle_command_reply: tid=0 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 client.12393 shutdown 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 msgr2=0x561f0a8d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800cfd0 tx=0x7f9fe8059350 comp rx=0 tx=0).stop 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 shutdown_connections 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8bcfe0 0x561f0a8daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8dcfb0 0x561f0a8bc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8edaa0 0x561f0a8edf00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 --2- 172.21.15.46:0/954504437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bf710 0x561f0a8d5490 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 >> 172.21.15.46:0/954504437 conn(0x561f0a8953f0 msgr2=0x561f0a88f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7f9fdfe00640 0 client.12393 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 shutdown_connections 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.412+0000 7fa000bef740 1 -- 172.21.15.46:0/954504437 wait complete. 2024-04-26T23:57:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.413+0000 7fa000bef740 20 client.12393 trim_cache size 0 max 16384 2024-04-26T23:57:56.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.432+0000 7fa000bef740 1 Processor -- start 2024-04-26T23:57:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.432+0000 7fa000bef740 1 -- start start 2024-04-26T23:57:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8201d10 con 0x561f0a8bcfe0 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8200f70 con 0x561f0a8dcfb0 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff806daa0 con 0x561f0a8bf710 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60108/0 (socket says 172.21.15.46:60108) 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.433+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4032860879 learned_addr learned my addr 172.21.15.46:0/4032860879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 msgr2=0x561f0a8fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 msgr2=0x561f0a8fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 -- 172.21.15.46:0/4032860879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f0a4ca0f0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800ce20 tx=0x7f9fe805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97e5cf5281bff76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9fe6a00640 1 -- 172.21.15.46:0/4032860879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8004bc0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9fe6a00640 1 -- 172.21.15.46:0/4032860879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe8004d80 con 0x561f0a8bcfe0 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.434+0000 7f9fe6a00640 1 -- 172.21.15.46:0/4032860879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8054a70 con 0x561f0a8bcfe0 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 -- 172.21.15.46:0/4032860879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 msgr2=0x561f0a8d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800ce20 tx=0x7f9fe805c000 comp rx=0 tx=0).stop 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 -- 172.21.15.46:0/4032860879 shutdown_connections 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a8fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a8fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 --2- 172.21.15.46:0/4032860879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a8d05f0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 -- 172.21.15.46:0/4032860879 >> 172.21.15.46:0/4032860879 conn(0x561f0a8953f0 msgr2=0x561f0a8dd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 -- 172.21.15.46:0/4032860879 shutdown_connections 2024-04-26T23:57:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.435+0000 7fa000bef740 1 -- 172.21.15.46:0/4032860879 wait complete. 2024-04-26T23:57:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.436+0000 7fa000bef740 1 -- start start 2024-04-26T23:57:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.436+0000 7fa000bef740 1 Processor -- start 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a88ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a88f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f0a805fd0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff806e0f0 con 0x561f0a8dcfb0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x561f0a8d9fa0 con 0x561f0a8bf710 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.437+0000 7fa000bef740 10 client.0 _ll_register_callbacks cb 0x7f9fe80588b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60112/0 (socket says 172.21.15.46:60112) 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 -- 172.21.15.46:0/3816671256 learned_addr learned my addr 172.21.15.46:0/3816671256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6000640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a88ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff7400640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a88f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 -- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 msgr2=0x561f0a88ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a88ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 -- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 msgr2=0x561f0a88f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a88f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 -- 172.21.15.46:0/3816671256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f0a7c0a30 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff7400640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a88f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.438+0000 7f9ff6a00640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8061190 tx=0x7f9fe80891b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3c43ce741f9c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8058440 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f9fe8057bb0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe8004d80 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f9fe8004bc0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 10 client.12399 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x561f0a8d9ce0 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe8004d80 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f9fe8082480 con 0x561f0a8bcfe0 2024-04-26T23:57:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 10 client.12399 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 10 client.12399 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 20 client.12399 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8d0b90 0x561f0a8d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 4 client.12399 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.439+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f0a800650 con 0x561f0a8d0b90 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7f9ff6000640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8d0b90 0x561f0a8d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7f9ff6000640 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8d0b90 0x561f0a8d8f30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e1de1315ee8f85 server_cookie=14faca64661045b9 in_seq=0 out_seq=0 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7f9fdfe00640 10 client.12399 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7f9fdfe00640 1 -- 172.21.15.46:0/3816671256 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f0a800650 con 0x561f0a8d0b90 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7f9fdfe00640 10 client.12399 handle_command_reply: tid=0 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.440+0000 7fa000bef740 1 client.12399 shutdown 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 msgr2=0x561f0a953ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8061190 tx=0x7f9fe80891b0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 shutdown_connections 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f0a8dcfb0 0x561f0a88f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x561f0a8bf710 0x561f0a88ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x561f0a8d0b90 0x561f0a8d8f30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 --2- 172.21.15.46:0/3816671256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f0a8bcfe0 0x561f0a953ad0 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 >> 172.21.15.46:0/3816671256 conn(0x561f0a8953f0 msgr2=0x561f0a8b9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7f9fdfe00640 0 client.12399 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 shutdown_connections 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 1 -- 172.21.15.46:0/3816671256 wait complete. 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.441+0000 7fa000bef740 20 client.12399 trim_cache size 0 max 16384 2024-04-26T23:57:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:57:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:57:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 msgr2=0x7f9fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 0x7f9fd406edb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002f000 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 msgr2=0x7f9ff80b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800c460 tx=0x7f9fe800c920 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 shutdown_connections 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f9fd406c8f0 0x7f9fd406edb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8162a70 0x7f9ff814b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ff80d3830 0x7f9ff80ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 --2- 172.21.15.46:0/4206274848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff80d1cb0 0x7f9ff80b7a90 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 >> 172.21.15.46:0/4206274848 conn(0x7f9ff80d4f20 msgr2=0x7f9ff815e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 shutdown_connections 2024-04-26T23:57:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.442+0000 7f9ff7e00640 1 -- 172.21.15.46:0/4206274848 wait complete. 2024-04-26T23:57:56.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:57:56.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:56.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub start / recursive,repair 2024-04-26T23:57:56.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.676+0000 7fb6eee00640 1 Processor -- start 2024-04-26T23:57:56.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.676+0000 7fb6eee00640 1 -- start start 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.677+0000 7fb6eee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6e80cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.677+0000 7fb6eee00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb6e80d2080 0x7fb6e80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.677+0000 7fb6eee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e80d01a0 con 0x7fb6e81627f0 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.677+0000 7fb6eee00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e80d0480 con 0x7fb6e80d2080 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.677+0000 7fb6eee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e80d0760 con 0x7fb6e80d3cc0 2024-04-26T23:57:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.678+0000 7fb6eda00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb6e81627f0 0x7fb6e80d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44512/0 (socket says 172.21.15.46:44512) 2024-04-26T23:57:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.678+0000 7fb6eda00640 1 -- 172.21.15.46:0/4259522894 learned_addr learned my addr 172.21.15.46:0/4259522894 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:56.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.678+0000 7fb6ee400640 1 --2- 172.21.15.46:0/4259522894 >> v2:172.21.15.142:3300/0 conn(0x7fb6e80d2080 0x7fb6e80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.678+0000 7fb6ed000640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6e80cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870947246 0 0) 0x7fb6e80d01a0 con 0x7fb6e81627f0 2024-04-26T23:57:56.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d0003520 con 0x7fb6e81627f0 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4079700092 0 0) 0x7fb6d0003520 con 0x7fb6e81627f0 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 msgr2=0x7fb6e80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6e80cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 >> v2:172.21.15.142:3300/0 conn(0x7fb6e80d2080 msgr2=0x7fb6e80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 --2- 172.21.15.46:0/4259522894 >> v2:172.21.15.142:3300/0 conn(0x7fb6e80d2080 0x7fb6e80b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e80d0a40 con 0x7fb6e81627f0 2024-04-26T23:57:56.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb6dc0019f0 con 0x7fb6e81627f0 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 >> v1:172.21.15.46:6789/0 conn(0x7fb6e81627f0 legacy=0x7fb6e80d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d0004d40 0x7fb6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80d0480 con 0x7fb6d0004d40 2024-04-26T23:57:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80d0760 con 0x7fb6d00039b0 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.679+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d000ba40 con 0x7fb6e80d3cc0 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d0004d40 0x7fb6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6eda00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ed000640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 -- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 msgr2=0x7fb6d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e80d3cc0 0x7fb6d0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 -- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6d00039b0 msgr2=0x7fb6d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6ee400640 1 -- 172.21.15.46:0/4259522894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0003520 con 0x7fb6d0004d40 2024-04-26T23:57:56.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.680+0000 7fb6eda00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:56.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.682+0000 7fb6ee400640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d0004d40 0x7fb6d00051c0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684f73d14f3d20a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.682+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0036070 con 0x7fb6d0004d40 2024-04-26T23:57:56.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.682+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0031400 con 0x7fb6d0004d40 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.682+0000 7fb6e7e00640 1 -- 172.21.15.46:0/4259522894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0009690 con 0x7fb6d0004d40 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 -- 172.21.15.46:0/4259522894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d0004d40 msgr2=0x7fb6d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 --2- 172.21.15.46:0/4259522894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d0004d40 0x7fb6d00051c0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d0005830 comp rx=0 tx=0).stop 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6ee400640 1 -- 172.21.15.46:0/4259522894 reap_dead start 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 -- 172.21.15.46:0/4259522894 shutdown_connections 2024-04-26T23:57:56.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 -- 172.21.15.46:0/4259522894 >> 172.21.15.46:0/4259522894 conn(0x7fb6e809e380 msgr2=0x7fb6e80d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 -- 172.21.15.46:0/4259522894 shutdown_connections 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.683+0000 7fb6eee00640 1 -- 172.21.15.46:0/4259522894 wait complete. 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.685+0000 7fb6eee00640 1 Processor -- start 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.685+0000 7fb6eee00640 1 -- start start 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e8150490 0x7fb6e80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 0x7fb6e806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8162bd0 con 0x7fb6e8150870 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80d2500 con 0x7fb6e806df40 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6eee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e80cfbc0 con 0x7fb6e8150490 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60124/0 (socket says 172.21.15.46:60124) 2024-04-26T23:57:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.686+0000 7fb6ed000640 1 -- 172.21.15.46:0/872590949 learned_addr learned my addr 172.21.15.46:0/872590949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ee400640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 0x7fb6e806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6eda00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e8150490 0x7fb6e80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 -- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e8150490 msgr2=0x7fb6e80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e8150490 0x7fb6e80b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 -- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 msgr2=0x7fb6e806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 0x7fb6e806e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 -- 172.21.15.46:0/872590949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e80b6360 con 0x7fb6e8150870 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ee400640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 0x7fb6e806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.687+0000 7fb6ed000640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4010890 tx=0x7fb6d4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.688+0000 7fb6e5600640 1 -- 172.21.15.46:0/872590949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400f040 con 0x7fb6e8150870 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.688+0000 7fb6e5600640 1 -- 172.21.15.46:0/872590949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d400eb60 con 0x7fb6e8150870 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.688+0000 7fb6e5600640 1 -- 172.21.15.46:0/872590949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d401c400 con 0x7fb6e8150870 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.688+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e80b6640 con 0x7fb6e8150870 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.688+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e80b6c00 con 0x7fb6e8150870 2024-04-26T23:57:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.691+0000 7fb6e5600640 1 -- 172.21.15.46:0/872590949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6d401c5a0 con 0x7fb6e8150870 2024-04-26T23:57:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.692+0000 7fb6e5600640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.692+0000 7fb6f3215740 1 Processor -- start 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.692+0000 7fb6e5600640 1 -- 172.21.15.46:0/872590949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7fb6d408c400 con 0x7fb6e8150870 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.692+0000 7fb6f3215740 1 -- start start 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6eda00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d955610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d90d500 0x556b1d953780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b1d871980 con 0x556b1d937820 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556b1d80e220 con 0x556b1d90d500 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.693+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556b1d86d940 con 0x556b1d9550c0 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60126/0 (socket says 172.21.15.46:60126) 2024-04-26T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 -- 172.21.15.46:0/605458355 learned_addr learned my addr 172.21.15.46:0/605458355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6ed000640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d955610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 -- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 msgr2=0x556b1d955610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6ee400640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d90d500 0x556b1d953780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d955610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 -- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d90d500 msgr2=0x556b1d953780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d90d500 0x556b1d953780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6eda00640 1 -- 172.21.15.46:0/605458355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d64b7e0 con 0x556b1d937820 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.694+0000 7fb6ed000640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d955610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.695+0000 7fb6eda00640 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc018b60 tx=0x7fb6dc001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78daa535d1c027d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.695+0000 7fb6da000640 1 -- 172.21.15.46:0/605458355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc018e50 con 0x556b1d937820 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.695+0000 7fb6da000640 1 -- 172.21.15.46:0/605458355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc01e070 con 0x556b1d937820 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.695+0000 7fb6da000640 1 -- 172.21.15.46:0/605458355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0236b0 con 0x556b1d937820 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 -- 172.21.15.46:0/605458355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 msgr2=0x556b1d937c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6eda00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 0x7fb6c806edb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007a00 tx=0x7fb6dc0074f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc018b60 tx=0x7fb6dc001d60 comp rx=0 tx=0).stop 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 -- 172.21.15.46:0/605458355 shutdown_connections 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d90d500 0x556b1d953780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d955610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 --2- 172.21.15.46:0/605458355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d937c00 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 -- 172.21.15.46:0/605458355 >> 172.21.15.46:0/605458355 conn(0x556b1d909e30 msgr2=0x556b1d952630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 -- 172.21.15.46:0/605458355 shutdown_connections 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.696+0000 7fb6f3215740 1 -- 172.21.15.46:0/605458355 wait complete. 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.697+0000 7fb6f3215740 1 -- start start 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.697+0000 7fb6f3215740 1 Processor -- start 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d937820 0x556b1d94ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d9550c0 0x556b1d9493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b1d90e2a0 con 0x556b1d949910 2024-04-26T23:57:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556b1d7f9170 con 0x556b1d9550c0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556b1d907d60 con 0x556b1d937820 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6f3215740 10 client.0 _ll_register_callbacks cb 0x556b1d868770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6ee400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6ee400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60130/0 (socket says 172.21.15.46:60130) 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.698+0000 7fb6ee400640 1 -- 172.21.15.46:0/1920363984 learned_addr learned my addr 172.21.15.46:0/1920363984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6eda00640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d937820 0x556b1d94ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ed000640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d9550c0 0x556b1d9493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 -- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d937820 msgr2=0x556b1d94ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d937820 0x556b1d94ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 -- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d9550c0 msgr2=0x556b1d9493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d9550c0 0x556b1d9493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 -- 172.21.15.46:0/1920363984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d6abf70 con 0x556b1d949910 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6ee400640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009be0 tx=0x7fb6e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56808f934aa35a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.699+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0036070 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7fb6e0003810 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6dc01e070 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7fb6e000f080 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 10 client.12411 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x556b1d543170 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc01e070 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7fb6e004c020 con 0x556b1d949910 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 10 client.12411 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 10 client.12411 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 20 client.12411 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d93a110 0x556b1d93a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 4 client.12411 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.700+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1d87f590 con 0x556b1d93a110 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.701+0000 7fb6eda00640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d93a110 0x556b1d93a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.701+0000 7fb6eda00640 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d93a110 0x556b1d93a520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0cea98d5c8df33 server_cookie=669e9e47d4da08ac in_seq=0 out_seq=0 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.701+0000 7fb6cea00640 10 client.12411 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.702+0000 7fb6cea00640 1 -- 172.21.15.46:0/1920363984 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b1d87f590 con 0x556b1d93a110 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.702+0000 7fb6cea00640 10 client.12411 handle_command_reply: tid=0 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.702+0000 7fb6f3215740 1 client.12411 shutdown 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.702+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 msgr2=0x556b1d9523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.702+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009be0 tx=0x7fb6e00046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 shutdown_connections 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d93a110 0x556b1d93a520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9523b0 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d9550c0 0x556b1d9493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 --2- 172.21.15.46:0/1920363984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d937820 0x556b1d94ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6cea00640 0 client.12411 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 >> 172.21.15.46:0/1920363984 conn(0x556b1d90d500 msgr2=0x556b1d956440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 shutdown_connections 2024-04-26T23:57:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 1 -- 172.21.15.46:0/1920363984 wait complete. 2024-04-26T23:57:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.703+0000 7fb6f3215740 20 client.12411 trim_cache size 0 max 16384 2024-04-26T23:57:56.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.723+0000 7fb6f3215740 1 Processor -- start 2024-04-26T23:57:56.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.723+0000 7fb6f3215740 1 -- start start 2024-04-26T23:57:56.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d949910 0x556b1d9c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d951fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e82029b0 con 0x556b1d937820 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e82019d0 con 0x556b1d949910 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e8200d30 con 0x556b1d9550c0 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60142/0 (socket says 172.21.15.46:60142) 2024-04-26T23:57:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.724+0000 7fb6eda00640 1 -- 172.21.15.46:0/705288826 learned_addr learned my addr 172.21.15.46:0/705288826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 -- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 msgr2=0x556b1d951fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d951fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 -- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d949910 msgr2=0x556b1d9c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d949910 0x556b1d9c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 -- 172.21.15.46:0/705288826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d949740 con 0x556b1d937820 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6eda00640 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01f000 tx=0x7fb6dc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7489ffa04deeba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6d9600640 1 -- 172.21.15.46:0/705288826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc018c50 con 0x556b1d937820 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6d9600640 1 -- 172.21.15.46:0/705288826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc01e040 con 0x556b1d937820 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.725+0000 7fb6d9600640 1 -- 172.21.15.46:0/705288826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc018df0 con 0x556b1d937820 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 -- 172.21.15.46:0/705288826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 msgr2=0x556b1d9619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01f000 tx=0x7fb6dc01d000 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 -- 172.21.15.46:0/705288826 shutdown_connections 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d951fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d949910 0x556b1d9c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 --2- 172.21.15.46:0/705288826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d937820 0x556b1d9619e0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 -- 172.21.15.46:0/705288826 >> 172.21.15.46:0/705288826 conn(0x556b1d90d500 msgr2=0x556b1d90d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 -- 172.21.15.46:0/705288826 shutdown_connections 2024-04-26T23:57:56.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.726+0000 7fb6f3215740 1 -- 172.21.15.46:0/705288826 wait complete. 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.727+0000 7fb6f3215740 1 -- start start 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.727+0000 7fb6f3215740 1 Processor -- start 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d937820 0x556b1d927a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d9cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8201380 con 0x556b1d949910 2024-04-26T23:57:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556b1d908d40 con 0x556b1d937820 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x556b1d954b00 con 0x556b1d9550c0 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6f3215740 10 client.0 _ll_register_callbacks cb 0x7fb6e0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.728+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60150/0 (socket says 172.21.15.46:60150) 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 -- 172.21.15.46:0/164614932 learned_addr learned my addr 172.21.15.46:0/164614932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 -- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 msgr2=0x556b1d9cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d9cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 -- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d937820 msgr2=0x556b1d927a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d937820 0x556b1d927a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 -- 172.21.15.46:0/164614932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d927f50 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6ed000640 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4059e40 tx=0x7fb6d405a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9298b4600473a5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.729+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4072030 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7fb6d408c0c0 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6dc01e040 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7fb6d4062a90 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 10 client.12417 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x556b1d96c960 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc01e040 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7fb6d4024030 con 0x556b1d949910 2024-04-26T23:57:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 10 client.12417 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 10 client.12417 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 20 client.12417 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d9c2190 0x556b1d9d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 4 client.12417 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x556b1d879400 con 0x556b1d9c2190 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.730+0000 7fb6eda00640 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d9c2190 0x556b1d9d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.731+0000 7fb6eda00640 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d9c2190 0x556b1d9d00c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367623b9326efb49 server_cookie=33e1ccb38b078a8 in_seq=0 out_seq=0 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.731+0000 7fb6cea00640 10 client.12417 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6cea00640 1 -- 172.21.15.46:0/164614932 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b1d879400 con 0x556b1d9c2190 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6cea00640 10 client.12417 handle_command_reply: tid=0 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 client.12417 shutdown 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 msgr2=0x556b1d9541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4059e40 tx=0x7fb6d405a670 comp rx=0 tx=0).stop 2024-04-26T23:57:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 shutdown_connections 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x556b1d9550c0 0x556b1d9cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x556b1d9c2190 0x556b1d9d00c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b1d949910 0x556b1d9541b0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 --2- 172.21.15.46:0/164614932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b1d937820 0x556b1d927a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 >> 172.21.15.46:0/164614932 conn(0x556b1d90d500 msgr2=0x556b1d906a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.732+0000 7fb6cea00640 0 client.12417 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 shutdown_connections 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6f3215740 1 -- 172.21.15.46:0/164614932 wait complete. 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6f3215740 20 client.12417 trim_cache size 0 max 16384 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:56.737 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-26T23:57:56.738 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "12c08ac1-4dba-43bb-a7d6-73f6e1be1d78", 2024-04-26T23:57:56.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-26T23:57:56.738 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 msgr2=0x7fb6c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 0x7fb6c806edb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007a00 tx=0x7fb6dc0074f0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 msgr2=0x7fb6e80cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.733+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4010890 tx=0x7fb6d4010d50 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 shutdown_connections 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e806df40 0x7fb6e806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6e8150870 0x7fb6e80cc180 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 --2- 172.21.15.46:0/872590949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6e8150490 0x7fb6e80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 >> 172.21.15.46:0/872590949 conn(0x7fb6e809e380 msgr2=0x7fb6e80d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 shutdown_connections 2024-04-26T23:57:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.734+0000 7fb6eee00640 1 -- 172.21.15.46:0/872590949 wait complete. 2024-04-26T23:57:56.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:57:56.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub status 2024-04-26T23:57:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.953+0000 7f3b31600640 1 Processor -- start 2024-04-26T23:57:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.953+0000 7f3b31600640 1 -- start start 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.955+0000 7f3b31600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b2c073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.955+0000 7f3b31600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c06ec30 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.955+0000 7f3b31600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0cf3f0 con 0x7f3b2c0cfed0 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.955+0000 7f3b31600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c145ff0 con 0x7f3b2c145c10 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.956+0000 7f3b30c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3b2c0d0f90 0x7f3b2c071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44526/0 (socket says 172.21.15.46:44526) 2024-04-26T23:57:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.956+0000 7f3b30c00640 1 -- 172.21.15.46:0/2078328609 learned_addr learned my addr 172.21.15.46:0/2078328609 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:57:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.956+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b2c073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.956+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206980704 0 0) 0x7f3b2c06ec30 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.956+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b10003520 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1590912061 0 0) 0x7f3b10003520 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 msgr2=0x7f3b2c073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b2c073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 >> v1:172.21.15.142:6789/0 conn(0x7f3b2c0cfed0 legacy=0x7f3b2c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0cf6d0 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f3b240019f0 con 0x7f3b2c0d0f90 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 >> v1:172.21.15.46:6789/0 conn(0x7f3b2c0d0f90 legacy=0x7f3b2c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b100039e0 0x7f3b10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b10004390 0x7f3b10004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b100050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c145ff0 con 0x7f3b100039e0 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1000b9c0 con 0x7f3b10004390 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1000bbc0 con 0x7f3b2c145c10 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.957+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b100039e0 0x7f3b10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b23e00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b10004390 0x7f3b10004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b30c00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b100050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b2be00640 1 -- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 msgr2=0x7f3b100050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c145c10 0x7f3b100050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b2be00640 1 -- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b10004390 msgr2=0x7f3b10004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b10004390 0x7f3b10004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b2be00640 1 -- 172.21.15.46:0/2078328609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10003520 con 0x7f3b100039e0 2024-04-26T23:57:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.958+0000 7f3b23e00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b10004390 0x7f3b10004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.959+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b100039e0 0x7f3b10003dc0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8386bfa9fe126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.959+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c036070 con 0x7f3b100039e0 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.959+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c031400 con 0x7f3b100039e0 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.959+0000 7f3b2b400640 1 -- 172.21.15.46:0/2078328609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c009690 con 0x7f3b100039e0 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 -- 172.21.15.46:0/2078328609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b100039e0 msgr2=0x7f3b10003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 --2- 172.21.15.46:0/2078328609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b100039e0 0x7f3b10003dc0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b10005830 comp rx=0 tx=0).stop 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b30c00640 1 -- 172.21.15.46:0/2078328609 reap_dead start 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 -- 172.21.15.46:0/2078328609 shutdown_connections 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 -- 172.21.15.46:0/2078328609 >> 172.21.15.46:0/2078328609 conn(0x7f3b2c06c350 msgr2=0x7f3b2c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 -- 172.21.15.46:0/2078328609 shutdown_connections 2024-04-26T23:57:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.960+0000 7f3b31600640 1 -- 172.21.15.46:0/2078328609 wait complete. 2024-04-26T23:57:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.962+0000 7f3b31600640 1 Processor -- start 2024-04-26T23:57:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.963+0000 7f3b31600640 1 -- start start 2024-04-26T23:57:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.963+0000 7f3b31600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b100039e0 0x7f3b2c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b31600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c13c0e0 0x7f3b2c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b31600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b31600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1468b0 con 0x7f3b2c0b2910 2024-04-26T23:57:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b31600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0cf0a0 con 0x7f3b100039e0 2024-04-26T23:57:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b31600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0d0330 con 0x7f3b2c13c0e0 2024-04-26T23:57:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60168/0 (socket says 172.21.15.46:60168) 2024-04-26T23:57:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b30c00640 1 -- 172.21.15.46:0/4217194608 learned_addr learned my addr 172.21.15.46:0/4217194608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b2be00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b100039e0 0x7f3b2c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b23e00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c13c0e0 0x7f3b2c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b30c00640 1 -- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c13c0e0 msgr2=0x7f3b2c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.964+0000 7f3b30c00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c13c0e0 0x7f3b2c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b30c00640 1 -- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b100039e0 msgr2=0x7f3b2c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b30c00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b100039e0 0x7f3b2c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b30c00640 1 -- 172.21.15.46:0/4217194608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0b2d90 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b30c00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0b05e0 tx=0x7f3b2400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b28c00640 1 -- 172.21.15.46:0/4217194608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400de70 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b28c00640 1 -- 172.21.15.46:0/4217194608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2400bb80 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b28c00640 1 -- 172.21.15.46:0/4217194608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24007070 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c0b3070 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.965+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c15d650 con 0x7f3b2c0b2910 2024-04-26T23:57:56.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.968+0000 7f3b28c00640 1 -- 172.21.15.46:0/4217194608 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b2401b020 con 0x7f3b2c0b2910 2024-04-26T23:57:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.969+0000 7f3b28c00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 0x7f3b0806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.969+0000 7f3b28c00640 1 -- 172.21.15.46:0/4217194608 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f3b2408beb0 con 0x7f3b2c0b2910 2024-04-26T23:57:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.970+0000 7f3b35ba6740 1 Processor -- start 2024-04-26T23:57:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.970+0000 7f3b35ba6740 1 -- start start 2024-04-26T23:57:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.970+0000 7f3b2be00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 0x7f3b0806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1d23f0 0x55c27f218670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c27f0e2030 con 0x55c27f219fb0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c27f0fd7f0 con 0x55c27f1d23f0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c27f0b87e0 con 0x55c27f1fc710 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60172/0 (socket says 172.21.15.46:60172) 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b23e00640 1 -- 172.21.15.46:0/158159975 learned_addr learned my addr 172.21.15.46:0/158159975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.971+0000 7f3b2be00640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 0x7f3b0806ee30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b1c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b2be00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b30c00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1d23f0 0x55c27f218670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 -- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 msgr2=0x55c27f1fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 -- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1d23f0 msgr2=0x55c27f218670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1d23f0 0x55c27f218670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 -- 172.21.15.46:0/158159975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27f0c4050 con 0x55c27f219fb0 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b2be00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b23e00640 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001930 tx=0x7f3b14001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eab65f072ca1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b1be00640 1 -- 172.21.15.46:0/158159975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b140052e0 con 0x55c27f219fb0 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b1be00640 1 -- 172.21.15.46:0/158159975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b14018070 con 0x55c27f219fb0 2024-04-26T23:57:56.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.972+0000 7f3b1be00640 1 -- 172.21.15.46:0/158159975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400fb50 con 0x55c27f219fb0 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 -- 172.21.15.46:0/158159975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 msgr2=0x55c27f21a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001930 tx=0x7f3b14001df0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 -- 172.21.15.46:0/158159975 shutdown_connections 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1d23f0 0x55c27f218670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f21a500 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/158159975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 -- 172.21.15.46:0/158159975 >> 172.21.15.46:0/158159975 conn(0x55c27f1ced20 msgr2=0x55c27f217520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 -- 172.21.15.46:0/158159975 shutdown_connections 2024-04-26T23:57:56.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.973+0000 7f3b35ba6740 1 -- 172.21.15.46:0/158159975 wait complete. 2024-04-26T23:57:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.974+0000 7f3b35ba6740 1 -- start start 2024-04-26T23:57:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.974+0000 7f3b35ba6740 1 Processor -- start 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1ee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 0x55c27f1ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c27f13a200 con 0x55c27f200030 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c27f134ee0 con 0x55c27f219fb0 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c27f141e70 con 0x55c27f1fc710 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.975+0000 7f3b35ba6740 10 client.0 _ll_register_callbacks cb 0x55c27f12b270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60188/0 (socket says 172.21.15.46:60188) 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 -- 172.21.15.46:0/718551193 learned_addr learned my addr 172.21.15.46:0/718551193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b2be00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1ee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b23e00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 0x55c27f1ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 -- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 msgr2=0x55c27f1ee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1ee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 -- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 msgr2=0x55c27f1ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 0x55c27f1ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b30c00640 1 -- 172.21.15.46:0/718551193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c27ef117d0 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.976+0000 7f3b23e00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 0x55c27f1ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b30c00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f3b24062150 tx=0x7f3b24059b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dd41c980a1e775 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b18c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240623f0 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f3b2401b9d0 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b14018070 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f3b2401bbd0 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b35ba6740 10 client.12429 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x55c27ee473f0 con 0x55c27f200030 2024-04-26T23:57:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.977+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b14018070 con 0x55c27f200030 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f3b24083b40 con 0x55c27f200030 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 10 client.12429 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 10 client.12429 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 20 client.12429 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f1fe6a0 0x55c27f1feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 4 client.12429 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c27f1437e0 con 0x55c27f1fe6a0 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.978+0000 7f3b2be00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f1fe6a0 0x55c27f1feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.979+0000 7f3b2be00640 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f1fe6a0 0x55c27f1feab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69873abe3427e969 server_cookie=e2eadf3f4825801b in_seq=0 out_seq=0 2024-04-26T23:57:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.979+0000 7f3b18c00640 10 client.12429 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.979+0000 7f3b18c00640 1 -- 172.21.15.46:0/718551193 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c27f1437e0 con 0x55c27f1fe6a0 2024-04-26T23:57:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.979+0000 7f3b18c00640 10 client.12429 handle_command_reply: tid=0 2024-04-26T23:57:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 client.12429 shutdown 2024-04-26T23:57:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 msgr2=0x55c27f1cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f3b24062150 tx=0x7f3b24059b60 comp rx=0 tx=0).stop 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 shutdown_connections 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f1fe6a0 0x55c27f1feab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f200030 0x55c27f1cb960 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f219fb0 0x55c27f1ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/718551193 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b18c00640 0 client.12429 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 >> 172.21.15.46:0/718551193 conn(0x55c27f1d23f0 msgr2=0x55c27f216d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 shutdown_connections 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 1 -- 172.21.15.46:0/718551193 wait complete. 2024-04-26T23:57:56.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.980+0000 7f3b35ba6740 20 client.12429 trim_cache size 0 max 16384 2024-04-26T23:57:57.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:56.999+0000 7f3b35ba6740 1 Processor -- start 2024-04-26T23:57:57.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 -- start start 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f200030 0x55c27f28ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c202290 con 0x55c27f219fb0 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.000+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c2012b0 con 0x55c27f200030 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c200640 con 0x55c27f1fc710 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60204/0 (socket says 172.21.15.46:60204) 2024-04-26T23:57:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 -- 172.21.15.46:0/2409296255 learned_addr learned my addr 172.21.15.46:0/2409296255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b2be00640 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 -- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 msgr2=0x55c27f1d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b23e00640 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f200030 0x55c27f28ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 -- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f200030 msgr2=0x55c27f28ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f200030 0x55c27f28ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.001+0000 7f3b30c00640 1 -- 172.21.15.46:0/2409296255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27ee070f0 con 0x55c27f219fb0 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.002+0000 7f3b30c00640 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001cf0 tx=0x7f3b2405d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634f7d70e22c8221 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.002+0000 7f3b1b400640 1 -- 172.21.15.46:0/2409296255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24083b40 con 0x55c27f219fb0 2024-04-26T23:57:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.002+0000 7f3b1b400640 1 -- 172.21.15.46:0/2409296255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b240621f0 con 0x55c27f219fb0 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.002+0000 7f3b1b400640 1 -- 172.21.15.46:0/2409296255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24058ef0 con 0x55c27f219fb0 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 -- 172.21.15.46:0/2409296255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 msgr2=0x55c27f2187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001cf0 tx=0x7f3b2405d000 comp rx=0 tx=0).stop 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 -- 172.21.15.46:0/2409296255 shutdown_connections 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f2187f0 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f200030 0x55c27f28ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/2409296255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f1fc710 0x55c27f1d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 -- 172.21.15.46:0/2409296255 >> 172.21.15.46:0/2409296255 conn(0x55c27f1d23f0 msgr2=0x55c27f1ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 -- 172.21.15.46:0/2409296255 shutdown_connections 2024-04-26T23:57:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.003+0000 7f3b35ba6740 1 -- 172.21.15.46:0/2409296255 wait complete. 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.004+0000 7f3b35ba6740 1 -- start start 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.004+0000 7f3b35ba6740 1 Processor -- start 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1fc710 0x55c27f1d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f200030 0x55c27f2388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c27f142fd0 con 0x55c27f219fb0 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c200c90 con 0x55c27f1fc710 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55c27f1cbd00 con 0x55c27f200030 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b35ba6740 10 client.0 _ll_register_callbacks cb 0x7f3b24055d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60208/0 (socket says 172.21.15.46:60208) 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 -- 172.21.15.46:0/1475818993 learned_addr learned my addr 172.21.15.46:0/1475818993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 -- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f200030 msgr2=0x55c27f2388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f200030 0x55c27f2388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 -- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1fc710 msgr2=0x55c27f1d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:57.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.005+0000 7f3b30c00640 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1fc710 0x55c27f1d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b30c00640 1 -- 172.21.15.46:0/1475818993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c27f0fda30 con 0x55c27f219fb0 2024-04-26T23:57:57.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b30c00640 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f3b24062840 tx=0x7f3b2408a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ea4c093aac6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:57.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b18c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24083b40 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(211..211 src has 1..211) ==== 6187+0+0 (secure 0 0 0) 0x7f3b24060070 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b240621f0 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f3b24068050 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b35ba6740 10 client.12435 fetch_fsmap learned FSMap version 174 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.006+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x55c27f0fda30 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b240621f0 con 0x55c27f219fb0 2024-04-26T23:57:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f3b2406b020 con 0x55c27f219fb0 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 10 client.12435 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 10 client.12435 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 20 client.12435 populate_metadata read hostname 'smithi046' 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f215e00 0x55c27f2187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 4 client.12435 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c27f13d650 con 0x55c27f215e00 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.007+0000 7f3b2be00640 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f215e00 0x55c27f2187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b2be00640 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f215e00 0x55c27f2187f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce727d0dc31972a7 server_cookie=9c868cb798e9414a in_seq=0 out_seq=0 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b18c00640 10 client.12435 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b18c00640 1 -- 172.21.15.46:0/1475818993 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55c27f13d650 con 0x55c27f215e00 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b18c00640 10 client.12435 handle_command_reply: tid=0 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 client.12435 shutdown 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 msgr2=0x55c27f238e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f3b24062840 tx=0x7f3b2408a0b0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 shutdown_connections 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x55c27f215e00 0x55c27f2187f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27f219fb0 0x55c27f238e10 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c27f200030 0x55c27f2388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 --2- 172.21.15.46:0/1475818993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c27f1fc710 0x55c27f1d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b18c00640 0 client.12435 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 >> 172.21.15.46:0/1475818993 conn(0x55c27f1d23f0 msgr2=0x55c27f1ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 shutdown_connections 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.008+0000 7f3b35ba6740 1 -- 172.21.15.46:0/1475818993 wait complete. 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b35ba6740 20 client.12435 trim_cache size 0 max 16384 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:57:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: "12c08ac1-4dba-43bb-a7d6-73f6e1be1d78": { 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "12c08ac1-4dba-43bb-a7d6-73f6e1be1d78", 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-26T23:57:57.014 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 msgr2=0x7f3b0806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 0x7f3b0806ee30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b1c008030 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 msgr2=0x7f3b2c072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0b05e0 tx=0x7f3b2400b4a0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 shutdown_connections 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f3b0806c970 0x7f3b0806ee30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c0b2910 0x7f3b2c072730 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b2c13c0e0 0x7f3b2c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 --2- 172.21.15.46:0/4217194608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b100039e0 0x7f3b2c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 >> 172.21.15.46:0/4217194608 conn(0x7f3b2c06c350 msgr2=0x7f3b2c071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:57.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.009+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 shutdown_connections 2024-04-26T23:57:57.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:57:57.010+0000 7f3b31600640 1 -- 172.21.15.46:0/4217194608 wait complete. 2024-04-26T23:57:57.029 INFO:tasks.cephfs.filesystem:scrub status for tag:12c08ac1-4dba-43bb-a7d6-73f6e1be1d78 - {'path': '/', 'tag': '12c08ac1-4dba-43bb-a7d6-73f6e1be1d78', 'options': 'recursive,repair'} 2024-04-26T23:58:14.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:14.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:27.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:27.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.15:0 scrub status 2024-04-26T23:58:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.253+0000 7f023f200640 1 Processor -- start 2024-04-26T23:58:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.253+0000 7f023f200640 1 -- start start 2024-04-26T23:58:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bbb0 0x7f02400709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f024015bf90 0x7f0240070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0240071520 con 0x7f0240072f70 2024-04-26T23:58:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f02400cc260 con 0x7f024015bf90 2024-04-26T23:58:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0240071800 con 0x7f024015bbb0 2024-04-26T23:58:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f024015bf90 0x7f0240070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0240072f70 0x7f024015f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-04-26T23:58:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023e800640 1 -- 172.21.15.46:0/643528167 learned_addr learned my addr 172.21.15.46:0/643528167 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:27.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.255+0000 7f023de00640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bbb0 0x7f02400709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.256+0000 7f023d400640 1 -- 172.21.15.46:0/643528167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bbb0 msgr2=0x7f02400709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.256+0000 7f023d400640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bbb0 0x7f02400709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.256+0000 7f023d400640 1 -- 172.21.15.46:0/643528167 >> v1:172.21.15.46:6789/0 conn(0x7f0240072f70 legacy=0x7f024015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.256+0000 7f023d400640 1 -- 172.21.15.46:0/643528167 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400cc4b0 con 0x7f024015bf90 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.256+0000 7f023ca00640 1 -- 172.21.15.46:0/643528167 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185901798 0 0) 0x7f0240071520 con 0x7f0240072f70 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.257+0000 7f023d400640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bf90 0x7f0240070ef0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d90 tx=0x7f022802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f8b2df556d1bba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.257+0000 7f023ca00640 1 -- 172.21.15.46:0/643528167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228007740 con 0x7f024015bf90 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.257+0000 7f023ca00640 1 -- 172.21.15.46:0/643528167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280078e0 con 0x7f024015bf90 2024-04-26T23:58:27.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.257+0000 7f023ca00640 1 -- 172.21.15.46:0/643528167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228007cb0 con 0x7f024015bf90 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 -- 172.21.15.46:0/643528167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bf90 msgr2=0x7f0240070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bf90 0x7f0240070ef0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d90 tx=0x7f022802a660 comp rx=0 tx=0).stop 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 -- 172.21.15.46:0/643528167 shutdown_connections 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bf90 0x7f0240070ef0 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 --2- 172.21.15.46:0/643528167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bbb0 0x7f02400709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 -- 172.21.15.46:0/643528167 >> 172.21.15.46:0/643528167 conn(0x7f02400d1c10 msgr2=0x7f02400d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 -- 172.21.15.46:0/643528167 shutdown_connections 2024-04-26T23:58:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.258+0000 7f023f200640 1 -- 172.21.15.46:0/643528167 wait complete. 2024-04-26T23:58:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.260+0000 7f023f200640 1 Processor -- start 2024-04-26T23:58:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.260+0000 7f023f200640 1 -- start start 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0240072f70 0x7f02400c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 0x7f0240155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02400cdcc0 con 0x7f024015bbb0 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0240145850 con 0x7f0240072f70 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02400cde40 con 0x7f024015bf90 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48508/0 (socket says 172.21.15.46:48508) 2024-04-26T23:58:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023d400640 1 -- 172.21.15.46:0/3604286937 learned_addr learned my addr 172.21.15.46:0/3604286937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023e800640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 0x7f0240155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.261+0000 7f023de00640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0240072f70 0x7f02400c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 -- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 msgr2=0x7f0240155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 0x7f0240155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 -- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0240072f70 msgr2=0x7f02400c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0240072f70 0x7f02400c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 -- 172.21.15.46:0/3604286937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400d05a0 con 0x7f024015bbb0 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023e800640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 0x7f0240155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023d400640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f0228009eb0 tx=0x7f0228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f0236000640 1 -- 172.21.15.46:0/3604286937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022803f070 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f0236000640 1 -- 172.21.15.46:0/3604286937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228031c40 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f0236000640 1 -- 172.21.15.46:0/3604286937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228043410 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.262+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02400d13c0 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.263+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02400d0b20 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.265+0000 7f0236000640 1 -- 172.21.15.46:0/3604286937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02280435b0 con 0x7f024015bbb0 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.266+0000 7f0236000640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 0x7f021c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.266+0000 7f0236000640 1 -- 172.21.15.46:0/3604286937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f02280ad2f0 con 0x7f024015bbb0 2024-04-26T23:58:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.266+0000 7f0248003740 1 Processor -- start 2024-04-26T23:58:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.266+0000 7f023de00640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 0x7f021c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 -- start start 2024-04-26T23:58:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441247fb0 0x562441248500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5624412003f0 0x562441246670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562441110030 con 0x56244122a710 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56244112b7f0 con 0x562441247fb0 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.267+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5624410e67e0 con 0x5624412003f0 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 0x7f021c06edb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f0230006fd0 tx=0x7f0230008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48516/0 (socket says 172.21.15.46:48516) 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 -- 172.21.15.46:0/2988074555 learned_addr learned my addr 172.21.15.46:0/2988074555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023e800640 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5624412003f0 0x562441246670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 -- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5624412003f0 msgr2=0x562441246670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5624412003f0 0x562441246670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 -- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441247fb0 msgr2=0x562441248500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441247fb0 0x562441248500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.268+0000 7f023de00640 1 -- 172.21.15.46:0/2988074555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624410f2050 con 0x56244122a710 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.269+0000 7f023de00640 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f023000fa50 tx=0x7f0230012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2a10f4d6d911f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.269+0000 7f022ea00640 1 -- 172.21.15.46:0/2988074555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023001b050 con 0x56244122a710 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.269+0000 7f022ea00640 1 -- 172.21.15.46:0/2988074555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230025070 con 0x56244122a710 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.269+0000 7f022ea00640 1 -- 172.21.15.46:0/2988074555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230014de0 con 0x56244122a710 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 -- 172.21.15.46:0/2988074555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 msgr2=0x56244122aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f023000fa50 tx=0x7f0230012000 comp rx=0 tx=0).stop 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 -- 172.21.15.46:0/2988074555 shutdown_connections 2024-04-26T23:58:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5624412003f0 0x562441246670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441247fb0 0x562441248500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 --2- 172.21.15.46:0/2988074555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x56244122aaf0 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 -- 172.21.15.46:0/2988074555 >> 172.21.15.46:0/2988074555 conn(0x5624411fcd20 msgr2=0x562441245520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 -- 172.21.15.46:0/2988074555 shutdown_connections 2024-04-26T23:58:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.270+0000 7f0248003740 1 -- 172.21.15.46:0/2988074555 wait complete. 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.271+0000 7f0248003740 1 -- start start 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.271+0000 7f0248003740 1 Processor -- start 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56244122a710 0x5624412166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x5624412193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562441168200 con 0x562441247fb0 2024-04-26T23:58:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562441162ee0 con 0x562441218f40 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56244116fe70 con 0x56244122a710 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f0248003740 10 client.0 _ll_register_callbacks cb 0x562441159270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48526/0 (socket says 172.21.15.46:48526) 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.272+0000 7f023d400640 1 -- 172.21.15.46:0/1302963305 learned_addr learned my addr 172.21.15.46:0/1302963305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023de00640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56244122a710 0x5624412166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 -- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56244122a710 msgr2=0x5624412166d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56244122a710 0x5624412166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 -- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 msgr2=0x5624412193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x5624412193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 -- 172.21.15.46:0/1302963305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562440f3f7d0 con 0x562441247fb0 2024-04-26T23:58:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f023d400640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f0228009e80 tx=0x7f0228007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df82a76ff7762c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f0227400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228048070 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f02280931b0 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.273+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0230025070 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f02280a9050 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 10 client.12450 fetch_fsmap learned FSMap version 174 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x562440e753f0 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230025070 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f0228083490 con 0x562441247fb0 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 10 client.12450 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 10 client.12450 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:58:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 20 client.12450 populate_metadata read hostname 'smithi046' 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x56244121a870 0x56244122c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 4 client.12450 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624411717e0 con 0x56244121a870 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.274+0000 7f023de00640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x56244121a870 0x56244122c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.275+0000 7f023de00640 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x56244121a870 0x56244122c520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb95f50075e2e27 server_cookie=34e94a36ac6a5279 in_seq=0 out_seq=0 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.275+0000 7f0227400640 10 client.12450 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0227400640 1 -- 172.21.15.46:0/1302963305 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624411717e0 con 0x56244121a870 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0227400640 10 client.12450 handle_command_reply: tid=0 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 client.12450 shutdown 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 msgr2=0x562441216eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f0228009e80 tx=0x7f0228007900 comp rx=0 tx=0).stop 2024-04-26T23:58:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 shutdown_connections 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x5624412193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x56244121a870 0x56244122c520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216eb0 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 --2- 172.21.15.46:0/1302963305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56244122a710 0x5624412166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 >> 172.21.15.46:0/1302963305 conn(0x5624412003f0 msgr2=0x5624411fa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0227400640 0 client.12450 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.276+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 shutdown_connections 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.277+0000 7f0248003740 1 -- 172.21.15.46:0/1302963305 wait complete. 2024-04-26T23:58:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.277+0000 7f0248003740 20 client.12450 trim_cache size 0 max 16384 2024-04-26T23:58:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.296+0000 7f0248003740 1 Processor -- start 2024-04-26T23:58:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.296+0000 7f0248003740 1 -- start start 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x562441244140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441247fb0 0x562441248460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0240201d20 con 0x56244122a710 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0240200d60 con 0x562441218f40 2024-04-26T23:58:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0240154380 con 0x562441247fb0 2024-04-26T23:58:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f023d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48534/0 (socket says 172.21.15.46:48534) 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.297+0000 7f023d400640 1 -- 172.21.15.46:0/3422215118 learned_addr learned my addr 172.21.15.46:0/3422215118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023de00640 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x562441244140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023e800640 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441247fb0 0x562441248460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 -- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441247fb0 msgr2=0x562441248460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441247fb0 0x562441248460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 -- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 msgr2=0x562441244140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x562441244140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 -- 172.21.15.46:0/3422215118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562440e350f0 con 0x56244122a710 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f023d400640 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f0228000f80 tx=0x7f0228007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce9fcc777ff8441 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.298+0000 7f022e000640 1 -- 172.21.15.46:0/3422215118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228048070 con 0x56244122a710 2024-04-26T23:58:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f022e000640 1 -- 172.21.15.46:0/3422215118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022807f0f0 con 0x56244122a710 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f022e000640 1 -- 172.21.15.46:0/3422215118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280931b0 con 0x56244122a710 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 -- 172.21.15.46:0/3422215118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 msgr2=0x5624412157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f0228000f80 tx=0x7f0228007970 comp rx=0 tx=0).stop 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 -- 172.21.15.46:0/3422215118 shutdown_connections 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441247fb0 0x562441248460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56244122a710 0x5624412157b0 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 --2- 172.21.15.46:0/3422215118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562441218f40 0x562441244140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 -- 172.21.15.46:0/3422215118 >> 172.21.15.46:0/3422215118 conn(0x5624412003f0 msgr2=0x56244121ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 -- 172.21.15.46:0/3422215118 shutdown_connections 2024-04-26T23:58:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.299+0000 7f0248003740 1 -- 172.21.15.46:0/3422215118 wait complete. 2024-04-26T23:58:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.300+0000 7f0248003740 1 -- start start 2024-04-26T23:58:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.300+0000 7f0248003740 1 Processor -- start 2024-04-26T23:58:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441218f40 0x5624412be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56244122a710 0x5624412007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562441170fd0 con 0x562441247fb0 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0240200710 con 0x56244122a710 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562441268f20 con 0x562441218f40 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f0248003740 10 client.0 _ll_register_callbacks cb 0x7f0228007230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.301+0000 7f023e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48546/0 (socket says 172.21.15.46:48546) 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 -- 172.21.15.46:0/1171334466 learned_addr learned my addr 172.21.15.46:0/1171334466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023d400640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56244122a710 0x5624412007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 -- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441218f40 msgr2=0x5624412be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441218f40 0x5624412be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 -- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56244122a710 msgr2=0x5624412007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56244122a710 0x5624412007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 -- 172.21.15.46:0/1171334466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244112ba30 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f023e800640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fec42184054334 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f0227400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.302+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238010040 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f0238001ad0 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022807f0f0 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f0238022050 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 10 client.12456 fetch_fsmap learned FSMap version 174 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x562441200d10 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022807f0f0 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f0238025020 con 0x562441247fb0 2024-04-26T23:58:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 10 client.12456 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 10 client.12456 resolve_mds: resolved 15:0 to role '15:0' aka daemon mds.a 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 20 client.12456 populate_metadata read hostname 'smithi046' 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x5624412172f0 0x562441267900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 4 client.12456 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.303+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56244116b650 con 0x5624412172f0 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f023de00640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x5624412172f0 0x562441267900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f023de00640 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x5624412172f0 0x562441267900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dea786581651af4 server_cookie=936e350302fe3ea3 in_seq=0 out_seq=0 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0227400640 10 client.12456 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0227400640 1 -- 172.21.15.46:0/1171334466 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56244116b650 con 0x5624412172f0 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0227400640 10 client.12456 handle_command_reply: tid=0 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0248003740 1 client.12456 shutdown 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 msgr2=0x562441216db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.304+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238007780 comp rx=0 tx=0).stop 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 shutdown_connections 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x5624412172f0 0x562441267900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562441247fb0 0x562441216db0 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56244122a710 0x5624412007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 --2- 172.21.15.46:0/1171334466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562441218f40 0x5624412be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0227400640 0 client.12456 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 >> 172.21.15.46:0/1171334466 conn(0x5624412003f0 msgr2=0x5624411f62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 shutdown_connections 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 1 -- 172.21.15.46:0/1171334466 wait complete. 2024-04-26T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f0248003740 20 client.12456 trim_cache size 0 max 16384 2024-04-26T23:58:27.311 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:27.311 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-26T23:58:27.311 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-26T23:58:27.311 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 msgr2=0x7f021c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.305+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 0x7f021c06edb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f0230006fd0 tx=0x7f0230008040 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 msgr2=0x7f02401552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f0228009eb0 tx=0x7f0228004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 shutdown_connections 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f021c06c8f0 0x7f021c06edb0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f024015bf90 0x7f0240155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024015bbb0 0x7f02401552f0 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 --2- 172.21.15.46:0/3604286937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0240072f70 0x7f02400c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 >> 172.21.15.46:0/3604286937 conn(0x7f02400d1c10 msgr2=0x7f02400b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 shutdown_connections 2024-04-26T23:58:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.306+0000 7f023f200640 1 -- 172.21.15.46:0/3604286937 wait complete. 2024-04-26T23:58:27.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T23:58:27.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:58:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.949+0000 7f03e2000640 1 Processor -- start 2024-04-26T23:58:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.950+0000 7f03e2000640 1 -- start start 2024-04-26T23:58:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03dc15f0c0 0x7f03dc0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 0x7f03dc163850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03dc0d0350 con 0x7f03dc15f4a0 2024-04-26T23:58:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e2000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc163d90 con 0x7f03dc15f0c0 2024-04-26T23:58:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0d0f60 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.951+0000 7f03e0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03dc15f0c0 0x7f03dc0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03dbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03dc15f4a0 0x7f03dc0d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48032/0 (socket says 172.21.15.46:48032) 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03dbe00640 1 -- 172.21.15.46:0/1337509786 learned_addr learned my addr 172.21.15.46:0/1337509786 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e1600640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 0x7f03dc163850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e0c00640 1 -- 172.21.15.46:0/1337509786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 msgr2=0x7f03dc163850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e0c00640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 0x7f03dc163850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e0c00640 1 -- 172.21.15.46:0/1337509786 >> v1:172.21.15.46:6789/0 conn(0x7f03dc15f4a0 legacy=0x7f03dc0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e0c00640 1 -- 172.21.15.46:0/1337509786 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc0d0630 con 0x7f03dc15f0c0 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03e1600640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 0x7f03dc163850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.952+0000 7f03db400640 1 -- 172.21.15.46:0/1337509786 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565162816 0 0) 0x7f03dc0d0350 con 0x7f03dc15f4a0 2024-04-26T23:58:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.953+0000 7f03e0c00640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc15f0c0 0x7f03dc0cfc90 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097a0 tx=0x7f03cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bace860d2e4e7536 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03db400640 1 -- 172.21.15.46:0/1337509786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc037070 con 0x7f03dc15f0c0 2024-04-26T23:58:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03db400640 1 -- 172.21.15.46:0/1337509786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc032440 con 0x7f03dc15f0c0 2024-04-26T23:58:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03db400640 1 -- 172.21.15.46:0/1337509786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc032810 con 0x7f03dc15f0c0 2024-04-26T23:58:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 -- 172.21.15.46:0/1337509786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc15f0c0 msgr2=0x7f03dc0cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc15f0c0 0x7f03dc0cfc90 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097a0 tx=0x7f03cc02a180 comp rx=0 tx=0).stop 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 -- 172.21.15.46:0/1337509786 shutdown_connections 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc0d3cc0 0x7f03dc163850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 --2- 172.21.15.46:0/1337509786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc15f0c0 0x7f03dc0cfc90 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 -- 172.21.15.46:0/1337509786 >> 172.21.15.46:0/1337509786 conn(0x7f03dc09e240 msgr2=0x7f03dc0d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 -- 172.21.15.46:0/1337509786 shutdown_connections 2024-04-26T23:58:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.954+0000 7f03e2000640 1 -- 172.21.15.46:0/1337509786 wait complete. 2024-04-26T23:58:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.956+0000 7f03e2000640 1 Processor -- start 2024-04-26T23:58:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.956+0000 7f03e2000640 1 -- start start 2024-04-26T23:58:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc15f0c0 0x7f03dc0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03dc15f4a0 0x7f03dc1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0b5b70 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0d0bf0 con 0x7f03dc15f4a0 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0b5cf0 con 0x7f03dc15f0c0 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.957+0000 7f03e0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48558/0 (socket says 172.21.15.46:48558) 2024-04-26T23:58:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 -- 172.21.15.46:0/549558277 learned_addr learned my addr 172.21.15.46:0/549558277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 -- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc15f0c0 msgr2=0x7f03dc0c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc15f0c0 0x7f03dc0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 -- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03dc15f4a0 msgr2=0x7f03dc1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03dc15f4a0 0x7f03dc1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 -- 172.21.15.46:0/549558277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc160e20 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.958+0000 7f03e0c00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098c0 tx=0x7f03cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.959+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc037040 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.959+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc042070 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.959+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc030a70 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.959+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc161100 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.959+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1616c0 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.960+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc0693b0 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.961+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03cc030c10 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.962+0000 7f03d8c00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 0x7f03bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.962+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f03cc0ac830 con 0x7f03dc0d3cc0 2024-04-26T23:58:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.963+0000 7f03dbe00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 0x7f03bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.964+0000 7f03dbe00640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 0x7f03bc06edb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f03c8007fd0 tx=0x7f03c8007ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:27.967+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03cc07aa10 con 0x7f03dc0d3cc0 2024-04-26T23:58:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.214+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03dc0d2650 con 0x7f03dc0d3cc0 2024-04-26T23:58:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.215+0000 7f03d8c00640 1 -- 172.21.15.46:0/549558277 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f03cc07a310 con 0x7f03dc0d3cc0 2024-04-26T23:58:28.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:28.217 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:58:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:58:28.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 msgr2=0x7f03bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 0x7f03bc06edb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f03c8007fd0 tx=0x7f03c8007ce0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 msgr2=0x7f03dc15c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098c0 tx=0x7f03cc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 shutdown_connections 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f03bc06c8f0 0x7f03bc06edb0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03dc15f4a0 0x7f03dc1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03dc15f0c0 0x7f03dc0c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 --2- 172.21.15.46:0/549558277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03dc0d3cc0 0x7f03dc15c5e0 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 >> 172.21.15.46:0/549558277 conn(0x7f03dc09e240 msgr2=0x7f03dc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 shutdown_connections 2024-04-26T23:58:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.218+0000 7f03e2000640 1 -- 172.21.15.46:0/549558277 wait complete. 2024-04-26T23:58:28.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 174, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:57:04.513003+0000', 'modified': '2024-04-26T23:57:23.570033+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 211, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12273': {'gid': 12273, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2610577522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2610577522}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2610577522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12273, 'qdb_cluster': [12273]} 2024-04-26T23:58:28.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:58:28.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:58:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.440+0000 7f8777200640 1 Processor -- start 2024-04-26T23:58:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.440+0000 7f8777200640 1 -- start start 2024-04-26T23:58:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.441+0000 7f8777200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 0x7f87780cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8777200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87780cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8777200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877806e6b0 0x7f877815e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8777200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f87780cf190 con 0x7f877806e6b0 2024-04-26T23:58:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8777200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f87780d13f0 con 0x7f87780cedb0 2024-04-26T23:58:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8777200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87780d16d0 con 0x7f87780d21b0 2024-04-26T23:58:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8776800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877806e6b0 0x7f877815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8776800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877806e6b0 0x7f877815e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48574/0 (socket says 172.21.15.46:48574) 2024-04-26T23:58:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8776800640 1 -- 172.21.15.46:0/709273164 learned_addr learned my addr 172.21.15.46:0/709273164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8775400640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87780cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.442+0000 7f8775e00640 1 --2- 172.21.15.46:0/709273164 >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 0x7f87780cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8776800640 1 -- 172.21.15.46:0/709273164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 msgr2=0x7f87780cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8776800640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87780cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8776800640 1 -- 172.21.15.46:0/709273164 >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 msgr2=0x7f87780cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8776800640 1 --2- 172.21.15.46:0/709273164 >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 0x7f87780cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8776800640 1 -- 172.21.15.46:0/709273164 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87780d19b0 con 0x7f877806e6b0 2024-04-26T23:58:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.443+0000 7f8775e00640 1 --2- 172.21.15.46:0/709273164 >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 0x7f87780cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.444+0000 7f8776800640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815e930 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f8778073420 tx=0x7f877002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86abe34262fcccc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.444+0000 7f8774a00640 1 -- 172.21.15.46:0/709273164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770036070 con 0x7f877806e6b0 2024-04-26T23:58:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.444+0000 7f8774a00640 1 -- 172.21.15.46:0/709273164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770031600 con 0x7f877806e6b0 2024-04-26T23:58:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.444+0000 7f8774a00640 1 -- 172.21.15.46:0/709273164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700319d0 con 0x7f877806e6b0 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 -- 172.21.15.46:0/709273164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 msgr2=0x7f877815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815e930 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f8778073420 tx=0x7f877002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 -- 172.21.15.46:0/709273164 shutdown_connections 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815e930 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 --2- 172.21.15.46:0/709273164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87780cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 --2- 172.21.15.46:0/709273164 >> v2:172.21.15.142:3300/0 conn(0x7f87780cedb0 0x7f87780cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 -- 172.21.15.46:0/709273164 >> 172.21.15.46:0/709273164 conn(0x7f87780d2cc0 msgr2=0x7f87780d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 -- 172.21.15.46:0/709273164 shutdown_connections 2024-04-26T23:58:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.445+0000 7f8777200640 1 -- 172.21.15.46:0/709273164 wait complete. 2024-04-26T23:58:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.447+0000 7f8777200640 1 Processor -- start 2024-04-26T23:58:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.447+0000 7f8777200640 1 -- start start 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87780cedb0 0x7f877815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87781f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87780d11b0 con 0x7f877806e6b0 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87780d1030 con 0x7f87780cedb0 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8777200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f877806d700 con 0x7f87780d21b0 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48578/0 (socket says 172.21.15.46:48578) 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.448+0000 7f8775e00640 1 -- 172.21.15.46:0/2639030520 learned_addr learned my addr 172.21.15.46:0/2639030520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775400640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87780cedb0 0x7f877815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8776800640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87781f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 -- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 msgr2=0x7f87781f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87781f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 -- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87780cedb0 msgr2=0x7f877815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87780cedb0 0x7f877815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 -- 172.21.15.46:0/2639030520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877809cb00 con 0x7f877806e6b0 2024-04-26T23:58:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f8775e00640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f876800c7c0 tx=0x7f876800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f876e000640 1 -- 172.21.15.46:0/2639030520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768015070 con 0x7f877806e6b0 2024-04-26T23:58:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f876e000640 1 -- 172.21.15.46:0/2639030520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768007a30 con 0x7f877806e6b0 2024-04-26T23:58:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.449+0000 7f876e000640 1 -- 172.21.15.46:0/2639030520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768007e00 con 0x7f877806e6b0 2024-04-26T23:58:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.450+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877809cd70 con 0x7f877806e6b0 2024-04-26T23:58:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.450+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877815e9a0 con 0x7f877806e6b0 2024-04-26T23:58:28.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.452+0000 7f876e000640 1 -- 172.21.15.46:0/2639030520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8768018070 con 0x7f877806e6b0 2024-04-26T23:58:28.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.454+0000 7f876e000640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 0x7f875406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.454+0000 7f876e000640 1 -- 172.21.15.46:0/2639030520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f876808bce0 con 0x7f877806e6b0 2024-04-26T23:58:28.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.454+0000 7f877fdd5740 1 Processor -- start 2024-04-26T23:58:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.454+0000 7f8775400640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 0x7f875406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.454+0000 7f877fdd5740 1 -- start start 2024-04-26T23:58:28.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d380d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d38039600 con 0x563d38100270 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d37fd6100 con 0x563d38111e80 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563d380355c0 con 0x563d380d57b0 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.455+0000 7f8775400640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 0x7f875406ede0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8760006fd0 tx=0x7f8760008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48588/0 (socket says 172.21.15.46:48588) 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 -- 172.21.15.46:0/3547067049 learned_addr learned my addr 172.21.15.46:0/3547067049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8775400640 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 -- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 msgr2=0x563d380d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d380d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 -- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 msgr2=0x563d380d5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380d5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 -- 172.21.15.46:0/3547067049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d37d4b2d0 con 0x563d38100270 2024-04-26T23:58:28.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.456+0000 7f8776800640 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f877002ac90 tx=0x7f877002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f08d401dd13996 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.457+0000 7f8766a00640 1 -- 172.21.15.46:0/3547067049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770036040 con 0x563d38100270 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.457+0000 7f8766a00640 1 -- 172.21.15.46:0/3547067049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770042070 con 0x563d38100270 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.457+0000 7f8766a00640 1 -- 172.21.15.46:0/3547067049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877003c8b0 con 0x563d38100270 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.457+0000 7f877fdd5740 1 -- 172.21.15.46:0/3547067049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 msgr2=0x563d381006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.457+0000 7f877fdd5740 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f877002ac90 tx=0x7f877002e000 comp rx=0 tx=0).stop 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 -- 172.21.15.46:0/3547067049 shutdown_connections 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d381006f0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 --2- 172.21.15.46:0/3547067049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d380d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 -- 172.21.15.46:0/3547067049 >> 172.21.15.46:0/3547067049 conn(0x563d38069d10 msgr2=0x563d3811b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 -- 172.21.15.46:0/3547067049 shutdown_connections 2024-04-26T23:58:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 -- 172.21.15.46:0/3547067049 wait complete. 2024-04-26T23:58:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.458+0000 7f877fdd5740 1 -- start start 2024-04-26T23:58:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.459+0000 7f877fdd5740 1 Processor -- start 2024-04-26T23:58:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d38100670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d37fc0f70 con 0x563d38100270 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d380fddc0 con 0x563d38111e80 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563d3812b350 con 0x563d380d57b0 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f877fdd5740 10 client.0 _ll_register_callbacks cb 0x563d3802f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48598/0 (socket says 172.21.15.46:48598) 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 -- 172.21.15.46:0/2475456265 learned_addr learned my addr 172.21.15.46:0/2475456265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8776800640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775e00640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d38100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 -- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 msgr2=0x563d38100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d38100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 -- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 msgr2=0x563d380fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8775400640 1 -- 172.21.15.46:0/2475456265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d37e156b0 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.460+0000 7f8776800640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f8775400640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f876000b1d0 tx=0x7f8760004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cdfc40d5a2a368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f875b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760025020 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f8760021830 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8770042070 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f876002f050 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f877fdd5740 10 client.12477 fetch_fsmap learned FSMap version 174 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.461+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x563d3812b0a0 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770042070 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f8760032020 con 0x563d38100270 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 10 client.12477 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:58:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 10 client.12477 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:58:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 20 client.12477 populate_metadata read hostname 'smithi046' 2024-04-26T23:58:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381029f0 0x563d38102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 4 client.12477 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d38047210 con 0x563d381029f0 2024-04-26T23:58:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.462+0000 7f8775e00640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381029f0 0x563d38102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.463+0000 7f8775e00640 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381029f0 0x563d38102dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53833889c9de5227 server_cookie=4b1706eb7d33d632 in_seq=0 out_seq=0 2024-04-26T23:58:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.463+0000 7f875b400640 10 client.12477 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f875b400640 1 -- 172.21.15.46:0/2475456265 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d38047210 con 0x563d381029f0 2024-04-26T23:58:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f875b400640 10 client.12477 handle_command_reply: tid=0 2024-04-26T23:58:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 client.12477 shutdown 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 msgr2=0x563d380fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f876000b1d0 tx=0x7f8760004730 comp rx=0 tx=0).stop 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 shutdown_connections 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38111e80 0x563d380fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381029f0 0x563d38102dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d38100270 0x563d380fc970 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 --2- 172.21.15.46:0/2475456265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d380d57b0 0x563d38100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f875b400640 0 client.12477 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 >> 172.21.15.46:0/2475456265 conn(0x563d38069d10 msgr2=0x563d380fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 shutdown_connections 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 1 -- 172.21.15.46:0/2475456265 wait complete. 2024-04-26T23:58:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.464+0000 7f877fdd5740 20 client.12477 trim_cache size 0 max 16384 2024-04-26T23:58:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.485+0000 7f877fdd5740 1 Processor -- start 2024-04-26T23:58:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.485+0000 7f877fdd5740 1 -- start start 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.485+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.485+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d380fc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d380fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f877fdd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8778200f00 con 0x563d380d57b0 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87781f9140 con 0x563d38100270 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87782008b0 con 0x563d38111e80 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48614/0 (socket says 172.21.15.46:48614) 2024-04-26T23:58:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 -- 172.21.15.46:0/919601014 learned_addr learned my addr 172.21.15.46:0/919601014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 -- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 msgr2=0x563d380fca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d380fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 -- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 msgr2=0x563d380fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d380fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.486+0000 7f8775e00640 1 -- 172.21.15.46:0/919601014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d380d16d0 con 0x563d380d57b0 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f8775e00640 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f8768007410 tx=0x7f8768006f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23c929e048bffb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f8766000640 1 -- 172.21.15.46:0/919601014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876801f040 con 0x563d380d57b0 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f8766000640 1 -- 172.21.15.46:0/919601014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768076070 con 0x563d380d57b0 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f8766000640 1 -- 172.21.15.46:0/919601014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876805dae0 con 0x563d380d57b0 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f877fdd5740 1 -- 172.21.15.46:0/919601014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 msgr2=0x563d380fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.487+0000 7f877fdd5740 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f8768007410 tx=0x7f8768006f50 comp rx=0 tx=0).stop 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 -- 172.21.15.46:0/919601014 shutdown_connections 2024-04-26T23:58:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d380fca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d380fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 --2- 172.21.15.46:0/919601014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d380fda40 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 -- 172.21.15.46:0/919601014 >> 172.21.15.46:0/919601014 conn(0x563d38069d10 msgr2=0x563d380fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 -- 172.21.15.46:0/919601014 shutdown_connections 2024-04-26T23:58:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 -- 172.21.15.46:0/919601014 wait complete. 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 -- start start 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.488+0000 7f877fdd5740 1 Processor -- start 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.489+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.489+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d38114270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.489+0000 7f877fdd5740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d381147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f877fdd5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d380efee0 con 0x563d380d57b0 2024-04-26T23:58:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d3811c430 con 0x563d38100270 2024-04-26T23:58:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f877fdd5740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563d3812ebe0 con 0x563d38111e80 2024-04-26T23:58:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f877fdd5740 10 client.0 _ll_register_callbacks cb 0x7f876000bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48620/0 (socket says 172.21.15.46:48620) 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 -- 172.21.15.46:0/1972025730 learned_addr learned my addr 172.21.15.46:0/1972025730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 -- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 msgr2=0x563d381147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d381147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 -- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 msgr2=0x563d38114270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d38114270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 -- 172.21.15.46:0/1972025730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d380fa0b0 con 0x563d380d57b0 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f8775e00640 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f8768059360 tx=0x7f8760015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23034e1f4fef192c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.490+0000 7f875b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876801f040 con 0x563d380d57b0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f8768089fb0 con 0x563d380d57b0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8768076070 con 0x563d380d57b0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) ==== 24+0+0 (secure 0 0 0) 0x7f876805dae0 con 0x563d380d57b0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 10 client.12483 fetch_fsmap learned FSMap version 174 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=2+}) -- 0x563d3812a3c0 con 0x563d380d57b0 2024-04-26T23:58:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876807d030 con 0x563d380d57b0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 174) ==== 3494+0+0 (secure 0 0 0) 0x7f8768055da0 con 0x563d380d57b0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 10 client.12483 fetch_fsmap finished waiting for FSMap version 174 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 10 client.12483 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 20 client.12483 populate_metadata read hostname 'smithi046' 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381989a0 0x563d3812d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 4 client.12483 mds_command: new command op to 12273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.491+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 --> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] -- command(tid 0: {"prefix": "status"}) -- 0x563d38041080 con 0x563d381989a0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f8775400640 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381989a0 0x563d3812d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f8775400640 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381989a0 0x563d3812d4f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3358c5a7c4ced288 server_cookie=5baff6d38c2cf848 in_seq=0 out_seq=0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f875b400640 10 client.12483 ms_handle_connect on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f875b400640 1 -- 172.21.15.46:0/1972025730 <== mds.0 v2:172.21.15.46:6833/2610577522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x563d38041080 con 0x563d381989a0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f875b400640 10 client.12483 handle_command_reply: tid=0 2024-04-26T23:58:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.492+0000 7f877fdd5740 1 client.12483 shutdown 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 msgr2=0x563d38110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f8768059360 tx=0x7f8760015000 comp rx=0 tx=0).stop 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 shutdown_connections 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563d38111e80 0x563d381147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d38100270 0x563d38114270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:6833/2610577522,v1:172.21.15.46:6835/2610577522] conn(0x563d381989a0 0x563d3812d4f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 --2- 172.21.15.46:0/1972025730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d380d57b0 0x563d38110c00 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f875b400640 0 client.12483 ms_handle_reset on v2:172.21.15.46:6833/2610577522 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 >> 172.21.15.46:0/1972025730 conn(0x563d38069d10 msgr2=0x563d38103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 shutdown_connections 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 1 -- 172.21.15.46:0/1972025730 wait complete. 2024-04-26T23:58:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.493+0000 7f877fdd5740 20 client.12483 trim_cache size 0 max 16384 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "554dfe58-9c53-4ba0-a961-d2b323744aa7", 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 12273, 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 67.936961527999998, 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 174, 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 213, 2024-04-26T23:58:28.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 211, 2024-04-26T23:58:28.501 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 69.858172791000001 2024-04-26T23:58:28.501 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 msgr2=0x7f875406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 0x7f875406ede0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8760006fd0 tx=0x7f8760008040 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 msgr2=0x7f877815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f876800c7c0 tx=0x7f876800cc80 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 shutdown_connections 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f875406c920 0x7f875406ede0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87780d21b0 0x7f87781f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87780cedb0 0x7f877815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 --2- 172.21.15.46:0/2639030520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877806e6b0 0x7f877815cff0 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 >> 172.21.15.46:0/2639030520 conn(0x7f87780d2cc0 msgr2=0x7f87780b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 shutdown_connections 2024-04-26T23:58:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:28.494+0000 7f8777200640 1 -- 172.21.15.46:0/2639030520 wait complete. 2024-04-26T23:58:28.514 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:58:28.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:58:28.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:58:28.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:58:28.743 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:58:28.743 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:58:28.743 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:28.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:58:28.804 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:58:28.804 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:58:28.804 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:28.862 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:58:28.862 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:58:28.863 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:58:28.863 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:28.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:58:28.923 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:58:28.923 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:58:28.923 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:28.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:58:28.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:58:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.208+0000 7f7791200640 1 Processor -- start 2024-04-26T23:58:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.208+0000 7f7791200640 1 -- start start 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.209+0000 7f7791200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f778c160910 0x7f778c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f7791200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778c073ad0 con 0x7f778c0720e0 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f7791200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778c146a90 con 0x7f778c1466b0 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f7791200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f778c1612a0 con 0x7f778c160910 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f778aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f778c0720e0 0x7f778c0cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f778aa00640 1 -- 172.21.15.46:0/2395826838 learned_addr learned my addr 172.21.15.46:0/2395826838 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.210+0000 7f778be00640 1 --2- 172.21.15.46:0/2395826838 >> v2:172.21.15.46:3300/0 conn(0x7f778c160910 0x7f778c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.211+0000 7f778be00640 1 -- 172.21.15.46:0/2395826838 >> v1:172.21.15.142:6789/0 conn(0x7f778c1466b0 legacy=0x7f778c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.211+0000 7f778a000640 1 -- 172.21.15.46:0/2395826838 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321246862 0 0) 0x7f778c073ad0 con 0x7f778c0720e0 2024-04-26T23:58:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.211+0000 7f778be00640 1 -- 172.21.15.46:0/2395826838 >> v1:172.21.15.46:6789/0 conn(0x7f778c0720e0 legacy=0x7f778c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.211+0000 7f778be00640 1 -- 172.21.15.46:0/2395826838 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c073d70 con 0x7f778c160910 2024-04-26T23:58:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.212+0000 7f778be00640 1 --2- 172.21.15.46:0/2395826838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c160d60 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f7784009dc0 tx=0x7f778402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb71ed62db6c0b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.212+0000 7f778a000640 1 -- 172.21.15.46:0/2395826838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784007740 con 0x7f778c160910 2024-04-26T23:58:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.212+0000 7f778a000640 1 -- 172.21.15.46:0/2395826838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77840078e0 con 0x7f778c160910 2024-04-26T23:58:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.212+0000 7f778a000640 1 -- 172.21.15.46:0/2395826838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784007cb0 con 0x7f778c160910 2024-04-26T23:58:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 -- 172.21.15.46:0/2395826838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 msgr2=0x7f778c160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 --2- 172.21.15.46:0/2395826838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c160d60 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f7784009dc0 tx=0x7f778402a730 comp rx=0 tx=0).stop 2024-04-26T23:58:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 -- 172.21.15.46:0/2395826838 shutdown_connections 2024-04-26T23:58:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 --2- 172.21.15.46:0/2395826838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c160d60 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 -- 172.21.15.46:0/2395826838 >> 172.21.15.46:0/2395826838 conn(0x7f778c0b26d0 msgr2=0x7f778c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 -- 172.21.15.46:0/2395826838 shutdown_connections 2024-04-26T23:58:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.213+0000 7f7791200640 1 -- 172.21.15.46:0/2395826838 wait complete. 2024-04-26T23:58:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.215+0000 7f7791200640 1 Processor -- start 2024-04-26T23:58:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.215+0000 7f7791200640 1 -- start start 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f778c0720e0 0x7f778c142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 0x7f778c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f778c0cb470 con 0x7f778c160910 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f778c0cb190 con 0x7f778c1466b0 2024-04-26T23:58:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.216+0000 7f7791200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f778c0cb990 con 0x7f778c0720e0 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48638/0 (socket says 172.21.15.46:48638) 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 -- 172.21.15.46:0/680464006 learned_addr learned my addr 172.21.15.46:0/680464006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778b400640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f778c0720e0 0x7f778c142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778aa00640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 0x7f778c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 -- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f778c0720e0 msgr2=0x7f778c142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f778c0720e0 0x7f778c142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 -- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 msgr2=0x7f778c0b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 0x7f778c0b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778be00640 1 -- 172.21.15.46:0/680464006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c0cfa30 con 0x7f778c160910 2024-04-26T23:58:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.217+0000 7f778aa00640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 0x7f778c0b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f778be00640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f7784009dc0 tx=0x7f7784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778403f070 con 0x7f778c160910 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784031c40 con 0x7f778c160910 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784003990 con 0x7f778c160910 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c0cfd10 con 0x7f778c160910 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.218+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c0d02d0 con 0x7f778c160910 2024-04-26T23:58:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.219+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c0693b0 con 0x7f778c160910 2024-04-26T23:58:29.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.220+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7784003b30 con 0x7f778c160910 2024-04-26T23:58:29.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.221+0000 7f7783400640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 0x7f776806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.221+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f77840ad0f0 con 0x7f778c160910 2024-04-26T23:58:29.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.222+0000 7f778b400640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 0x7f776806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.223+0000 7f778b400640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 0x7f776806edb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f777c000f60 tx=0x7f777c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.226+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f778407b240 con 0x7f778c160910 2024-04-26T23:58:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.473+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f778c0c7180 con 0x7f778c160910 2024-04-26T23:58:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.475+0000 7f7783400640 1 -- 172.21.15.46:0/680464006 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f778407ab40 con 0x7f778c160910 2024-04-26T23:58:29.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:29.476 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:58:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:58:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 msgr2=0x7f776806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 0x7f776806edb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f777c000f60 tx=0x7f777c009040 comp rx=0 tx=0).stop 2024-04-26T23:58:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 msgr2=0x7f778c0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f7784009dc0 tx=0x7f7784004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 shutdown_connections 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f776806c8f0 0x7f776806edb0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f778c160910 0x7f778c0b4140 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f778c1466b0 0x7f778c0b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 --2- 172.21.15.46:0/680464006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f778c0720e0 0x7f778c142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 >> 172.21.15.46:0/680464006 conn(0x7f778c0b26d0 msgr2=0x7f778c0cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 shutdown_connections 2024-04-26T23:58:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.478+0000 7f7791200640 1 -- 172.21.15.46:0/680464006 wait complete. 2024-04-26T23:58:29.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:58:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.710+0000 7f48c1c00640 1 Processor -- start 2024-04-26T23:58:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.710+0000 7f48c1c00640 1 -- start start 2024-04-26T23:58:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48c1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 0x7f48bc160b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48c1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48bc0b75f0 0x7f48bc146760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48c1c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48bc146ca0 con 0x7f48bc0d0630 2024-04-26T23:58:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48c1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f48bc1611c0 con 0x7f48bc0b75f0 2024-04-26T23:58:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48c1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc1614a0 con 0x7f48bc0b7210 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48ba000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48bc0b75f0 0x7f48bc146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48ba000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48bc0b75f0 0x7f48bc146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48650/0 (socket says 172.21.15.46:48650) 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.712+0000 7f48ba000640 1 -- 172.21.15.46:0/1439095135 learned_addr learned my addr 172.21.15.46:0/1439095135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48ba000640 1 -- 172.21.15.46:0/1439095135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 msgr2=0x7f48bc160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48baa00640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 0x7f48bc160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48ba000640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 0x7f48bc160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48ba000640 1 -- 172.21.15.46:0/1439095135 >> v1:172.21.15.142:6789/0 conn(0x7f48bc0d0630 legacy=0x7f48bc15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48ba000640 1 -- 172.21.15.46:0/1439095135 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc161780 con 0x7f48bc0b75f0 2024-04-26T23:58:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.713+0000 7f48baa00640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 0x7f48bc160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.714+0000 7f48ba000640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b75f0 0x7f48bc146760 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009870 tx=0x7f48a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d8920b5b592f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.714+0000 7f48b9600640 1 -- 172.21.15.46:0/1439095135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037070 con 0x7f48bc0b75f0 2024-04-26T23:58:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.714+0000 7f48b9600640 1 -- 172.21.15.46:0/1439095135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8032440 con 0x7f48bc0b75f0 2024-04-26T23:58:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.714+0000 7f48b9600640 1 -- 172.21.15.46:0/1439095135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8032810 con 0x7f48bc0b75f0 2024-04-26T23:58:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 -- 172.21.15.46:0/1439095135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b75f0 msgr2=0x7f48bc146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b75f0 0x7f48bc146760 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009870 tx=0x7f48a802a180 comp rx=0 tx=0).stop 2024-04-26T23:58:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 -- 172.21.15.46:0/1439095135 shutdown_connections 2024-04-26T23:58:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b75f0 0x7f48bc146760 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 --2- 172.21.15.46:0/1439095135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0b7210 0x7f48bc160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 -- 172.21.15.46:0/1439095135 >> 172.21.15.46:0/1439095135 conn(0x7f48bc0d1e50 msgr2=0x7f48bc0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 -- 172.21.15.46:0/1439095135 shutdown_connections 2024-04-26T23:58:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.715+0000 7f48c1c00640 1 -- 172.21.15.46:0/1439095135 wait complete. 2024-04-26T23:58:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.717+0000 7f48c1c00640 1 Processor -- start 2024-04-26T23:58:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.717+0000 7f48c1c00640 1 -- start start 2024-04-26T23:58:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 0x7f48bc14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 0x7f48bc06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0b7a50 con 0x7f48bc0b7210 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc15d670 con 0x7f48bc0b75f0 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48c1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc0d1b00 con 0x7f48bc0d0630 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48666/0 (socket says 172.21.15.46:48666) 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.718+0000 7f48baa00640 1 -- 172.21.15.46:0/2996543962 learned_addr learned my addr 172.21.15.46:0/2996543962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48ba000640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 0x7f48bc14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48bb400640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 0x7f48bc06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 -- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 msgr2=0x7f48bc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 0x7f48bc06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 -- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 msgr2=0x7f48bc14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 0x7f48bc14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 -- 172.21.15.46:0/2996543962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc14f890 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48ba000640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 0x7f48bc14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48bb400640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 0x7f48bc06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48baa00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c7f0 tx=0x7f48ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.719+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac015070 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.720+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac007820 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.720+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007bd0 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.720+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc1f31c0 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.720+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc1f3480 con 0x7f48bc0b7210 2024-04-26T23:58:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.721+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc069530 con 0x7f48bc0b7210 2024-04-26T23:58:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.722+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48ac018070 con 0x7f48bc0b7210 2024-04-26T23:58:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.723+0000 7f48b2a00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 0x7f489c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.723+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f48ac08bf40 con 0x7f48bc0b7210 2024-04-26T23:58:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.724+0000 7f48ba000640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 0x7f489c0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.725+0000 7f48ba000640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 0x7f489c0731c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f48a8034000 tx=0x7f48a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.728+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48bc14f890 con 0x7f48bc0b7210 2024-04-26T23:58:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.932+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48bc0738c0 con 0x7f48bc0b7210 2024-04-26T23:58:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.934+0000 7f48b2a00640 1 -- 172.21.15.46:0/2996543962 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) ==== 74+0+17189 (secure 0 0 0) 0x7f48ac05a090 con 0x7f48bc0b7210 2024-04-26T23:58:29.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:29.936 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":213,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:59.707896+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.6","mappings":[{"from":2,"to":0}]},{"pgid":"32.c","mappings":[{"from":6,"to":5}]},{"pgid":"32.17","mappings":[{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/3782420006":"2024-04-27T23:57:18.527554+0000","172.21.15.46:6833/3782420006":"2024-04-27T23:57:18.527554+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:58:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.937+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 msgr2=0x7f489c0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.937+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 0x7f489c0731c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f48a8034000 tx=0x7f48a802e000 comp rx=0 tx=0).stop 2024-04-26T23:58:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.937+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 msgr2=0x7f48bc15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.937+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c7f0 tx=0x7f48ac00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 shutdown_connections 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f489c070d00 0x7f489c0731c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48bc0d0630 0x7f48bc06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48bc0b75f0 0x7f48bc14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 --2- 172.21.15.46:0/2996543962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48bc0b7210 0x7f48bc15c830 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 >> 172.21.15.46:0/2996543962 conn(0x7f48bc0d1e50 msgr2=0x7f48bc0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:29.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 shutdown_connections 2024-04-26T23:58:29.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:29.938+0000 7f48c1c00640 1 -- 172.21.15.46:0/2996543962 wait complete. 2024-04-26T23:58:29.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:37:01.917968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:57:02.313551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:57:03.507616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:58:29.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:58:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.179+0000 7f7d97a00640 1 Processor -- start 2024-04-26T23:58:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.180+0000 7f7d97a00640 1 -- start start 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.181+0000 7f7d97a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d900d3860 0x7f7d900d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.181+0000 7f7d97a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d901630a0 0x7f7d901634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.181+0000 7f7d97a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d90163e70 con 0x7f7d9015f480 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.181+0000 7f7d97a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7d900d0aa0 con 0x7f7d901630a0 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.181+0000 7f7d97a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7d90163a10 con 0x7f7d900d3860 2024-04-26T23:58:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d901630a0 0x7f7d901634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d901630a0 0x7f7d901634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48682/0 (socket says 172.21.15.46:48682) 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 -- 172.21.15.46:0/1297740126 learned_addr learned my addr 172.21.15.46:0/1297740126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d8fe00640 1 --2- 172.21.15.46:0/1297740126 >> v2:172.21.15.142:3300/0 conn(0x7f7d900d3860 0x7f7d900d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 -- 172.21.15.46:0/1297740126 >> v2:172.21.15.142:3300/0 conn(0x7f7d900d3860 msgr2=0x7f7d900d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 --2- 172.21.15.46:0/1297740126 >> v2:172.21.15.142:3300/0 conn(0x7f7d900d3860 0x7f7d900d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 -- 172.21.15.46:0/1297740126 >> v1:172.21.15.142:6789/0 conn(0x7f7d9015f480 legacy=0x7f7d900d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.182+0000 7f7d95a00640 1 -- 172.21.15.46:0/1297740126 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900d0170 con 0x7f7d901630a0 2024-04-26T23:58:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.184+0000 7f7d95a00640 1 --2- 172.21.15.46:0/1297740126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d901630a0 0x7f7d901634d0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f7d9006cd20 tx=0x7f7d8802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47abbc0918de45b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.184+0000 7f7d8f400640 1 -- 172.21.15.46:0/1297740126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88037070 con 0x7f7d901630a0 2024-04-26T23:58:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.184+0000 7f7d8f400640 1 -- 172.21.15.46:0/1297740126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88032440 con 0x7f7d901630a0 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.184+0000 7f7d8f400640 1 -- 172.21.15.46:0/1297740126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88032810 con 0x7f7d901630a0 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 -- 172.21.15.46:0/1297740126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d901630a0 msgr2=0x7f7d901634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 --2- 172.21.15.46:0/1297740126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d901630a0 0x7f7d901634d0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f7d9006cd20 tx=0x7f7d8802a2c0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 -- 172.21.15.46:0/1297740126 shutdown_connections 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 --2- 172.21.15.46:0/1297740126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d901630a0 0x7f7d901634d0 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 --2- 172.21.15.46:0/1297740126 >> v2:172.21.15.142:3300/0 conn(0x7f7d900d3860 0x7f7d900d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 -- 172.21.15.46:0/1297740126 >> 172.21.15.46:0/1297740126 conn(0x7f7d900d5070 msgr2=0x7f7d900d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 -- 172.21.15.46:0/1297740126 shutdown_connections 2024-04-26T23:58:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.185+0000 7f7d97a00640 1 -- 172.21.15.46:0/1297740126 wait complete. 2024-04-26T23:58:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.187+0000 7f7d97a00640 1 Processor -- start 2024-04-26T23:58:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.187+0000 7f7d97a00640 1 -- start start 2024-04-26T23:58:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d900d3860 0x7f7d901f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d901630a0 0x7f7d900ba370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900d1640 con 0x7f7d9015f480 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900d0730 con 0x7f7d900d3860 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d97a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900cfe60 con 0x7f7d901630a0 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48690/0 (socket says 172.21.15.46:48690) 2024-04-26T23:58:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.188+0000 7f7d8fe00640 1 -- 172.21.15.46:0/973193437 learned_addr learned my addr 172.21.15.46:0/973193437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 -- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d901630a0 msgr2=0x7f7d900ba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d901630a0 0x7f7d900ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 -- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d900d3860 msgr2=0x7f7d901f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d900d3860 0x7f7d901f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 -- 172.21.15.46:0/973193437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900cc4e0 con 0x7f7d9015f480 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8fe00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007ef0 tx=0x7f7d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78014620 con 0x7f7d9015f480 2024-04-26T23:58:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78019070 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78014a00 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.189+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d900cc7c0 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.190+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d900ccd80 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.191+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9008d8a0 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.192+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d78014ba0 con 0x7f7d9015f480 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.193+0000 7f7d8cc00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 0x7f7d6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.193+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f7d7808c770 con 0x7f7d9015f480 2024-04-26T23:58:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.194+0000 7f7d95000640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 0x7f7d6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.195+0000 7f7d95000640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 0x7f7d6c06edb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7d80003cc0 tx=0x7f7d800079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.197+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d9008d8a0 con 0x7f7d9015f480 2024-04-26T23:58:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.400+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d900a13e0 con 0x7f7d9015f480 2024-04-26T23:58:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.401+0000 7f7d8cc00640 1 -- 172.21.15.46:0/973193437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v213) ==== 93+0+10 (secure 0 0 0) 0x7f7d7805a8c0 con 0x7f7d9015f480 2024-04-26T23:58:30.402 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 msgr2=0x7f7d6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 0x7f7d6c06edb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7d80003cc0 tx=0x7f7d800079b0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 msgr2=0x7f7d900b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007ef0 tx=0x7f7d7800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 shutdown_connections 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7d6c06c8f0 0x7f7d6c06edb0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d901630a0 0x7f7d900ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d9015f480 0x7f7d900b9e30 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 --2- 172.21.15.46:0/973193437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d900d3860 0x7f7d901f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.403+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 >> 172.21.15.46:0/973193437 conn(0x7f7d900d5070 msgr2=0x7f7d901624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.404+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 shutdown_connections 2024-04-26T23:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.404+0000 7f7d97a00640 1 -- 172.21.15.46:0/973193437 wait complete. 2024-04-26T23:58:30.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.644+0000 7f6a7da00640 1 Processor -- start 2024-04-26T23:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.644+0000 7f6a7da00640 1 -- start start 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a7da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780d3d60 0x7f6a780cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a7da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a780b64c0 0x7f6a780d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a7da00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a780b46f0 con 0x7f6a78162aa0 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a7da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6a780b44a0 con 0x7f6a780b64c0 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a7da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a780b5020 con 0x7f6a780d3d60 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a77400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a780b64c0 0x7f6a780d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a77400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a780b64c0 0x7f6a780d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48706/0 (socket says 172.21.15.46:48706) 2024-04-26T23:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.646+0000 7f6a77400640 1 -- 172.21.15.46:0/2931709004 learned_addr learned my addr 172.21.15.46:0/2931709004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.647+0000 7f6a77400640 1 -- 172.21.15.46:0/2931709004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780d3d60 msgr2=0x7f6a780cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.647+0000 7f6a77400640 1 --2- 172.21.15.46:0/2931709004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780d3d60 0x7f6a780cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.647+0000 7f6a77400640 1 -- 172.21.15.46:0/2931709004 >> v1:172.21.15.142:6789/0 conn(0x7f6a78162aa0 legacy=0x7f6a780d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.647+0000 7f6a77400640 1 -- 172.21.15.46:0/2931709004 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a780b49d0 con 0x7f6a780b64c0 2024-04-26T23:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.648+0000 7f6a77400640 1 --2- 172.21.15.46:0/2931709004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780b64c0 0x7f6a780d01c0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009870 tx=0x7f6a7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322baa0c58a443e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.648+0000 7f6a75600640 1 -- 172.21.15.46:0/2931709004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70037070 con 0x7f6a780b64c0 2024-04-26T23:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.648+0000 7f6a75600640 1 -- 172.21.15.46:0/2931709004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70032440 con 0x7f6a780b64c0 2024-04-26T23:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.648+0000 7f6a75600640 1 -- 172.21.15.46:0/2931709004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70032810 con 0x7f6a780b64c0 2024-04-26T23:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 -- 172.21.15.46:0/2931709004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780b64c0 msgr2=0x7f6a780d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2931709004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780b64c0 0x7f6a780d01c0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009870 tx=0x7f6a7002a180 comp rx=0 tx=0).stop 2024-04-26T23:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 -- 172.21.15.46:0/2931709004 shutdown_connections 2024-04-26T23:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2931709004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780b64c0 0x7f6a780d01c0 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2931709004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780d3d60 0x7f6a780cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 -- 172.21.15.46:0/2931709004 >> 172.21.15.46:0/2931709004 conn(0x7f6a780d5070 msgr2=0x7f6a780d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 -- 172.21.15.46:0/2931709004 shutdown_connections 2024-04-26T23:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.649+0000 7f6a7da00640 1 -- 172.21.15.46:0/2931709004 wait complete. 2024-04-26T23:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.651+0000 7f6a7da00640 1 Processor -- start 2024-04-26T23:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.651+0000 7f6a7da00640 1 -- start start 2024-04-26T23:58:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a780b64c0 0x7f6a781f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a78162aa0 0x7f6a7814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a780d30f0 con 0x7f6a780d3d60 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78162e80 con 0x7f6a780b64c0 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a7da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6a780d5b90 con 0x7f6a78162aa0 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48714/0 (socket says 172.21.15.46:48714) 2024-04-26T23:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.652+0000 7f6a76000640 1 -- 172.21.15.46:0/2610094853 learned_addr learned my addr 172.21.15.46:0/2610094853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76a00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a780b64c0 0x7f6a781f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 -- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a78162aa0 msgr2=0x7f6a7814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a78162aa0 0x7f6a7814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 -- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a780b64c0 msgr2=0x7f6a781f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a780b64c0 0x7f6a781f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 -- 172.21.15.46:0/2610094853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a780cbd50 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.653+0000 7f6a76000640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b4a0 tx=0x7f6a6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.654+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60014070 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.654+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60010610 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.654+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60010a00 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.654+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a780bada0 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.654+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a780cd580 con 0x7f6a780d3d60 2024-04-26T23:58:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.655+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a78069530 con 0x7f6a780d3d60 2024-04-26T23:58:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.656+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a60017070 con 0x7f6a780d3d60 2024-04-26T23:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.658+0000 7f6a6ea00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 0x7f6a5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.658+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f6a6008b820 con 0x7f6a780d3d60 2024-04-26T23:58:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.658+0000 7f6a76a00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 0x7f6a5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.659+0000 7f6a76a00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 0x7f6a5406edb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009f20 tx=0x7f6a680094d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.663+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a780bada0 con 0x7f6a780d3d60 2024-04-26T23:58:30.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.872+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a780764e0 con 0x7f6a780d3d60 2024-04-26T23:58:30.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.872+0000 7f6a6ea00640 1 -- 172.21.15.46:0/2610094853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v213) ==== 104+0+11 (secure 0 0 0) 0x7f6a60059a00 con 0x7f6a780d3d60 2024-04-26T23:58:30.874 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-26T23:58:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 msgr2=0x7f6a5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 0x7f6a5406edb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009f20 tx=0x7f6a680094d0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 msgr2=0x7f6a780ba860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b4a0 tx=0x7f6a6000b960 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 shutdown_connections 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6a5406c8f0 0x7f6a5406edb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a78162aa0 0x7f6a7814fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a780d3d60 0x7f6a780ba860 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 --2- 172.21.15.46:0/2610094853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a780b64c0 0x7f6a781f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 >> 172.21.15.46:0/2610094853 conn(0x7f6a780d5070 msgr2=0x7f6a780d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.875+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 shutdown_connections 2024-04-26T23:58:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:30.876+0000 7f6a7da00640 1 -- 172.21.15.46:0/2610094853 wait complete. 2024-04-26T23:58:30.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:58:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.110+0000 7fbf6fe00640 1 Processor -- start 2024-04-26T23:58:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.110+0000 7fbf6fe00640 1 -- start start 2024-04-26T23:58:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.111+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7006f680 0x7fbf700ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.111+0000 7fbf6fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf700cff80 con 0x7fbf70152f80 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.111+0000 7fbf6fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7015f920 con 0x7fbf70152ba0 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.111+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf70160530 con 0x7fbf7006f680 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.112+0000 7fbf67e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbf70152f80 0x7fbf7006ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48050/0 (socket says 172.21.15.46:48050) 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.112+0000 7fbf67e00640 1 -- 172.21.15.46:0/1906827123 learned_addr learned my addr 172.21.15.46:0/1906827123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.112+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297150090 0 0) 0x7fbf700cff80 con 0x7fbf70152f80 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.112+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7006f680 0x7fbf700ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.112+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf54005730 con 0x7fbf70152f80 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3694931520 0 0) 0x7fbf54005730 con 0x7fbf70152f80 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7006f680 msgr2=0x7fbf700ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7006f680 0x7fbf700ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 >> v1:172.21.15.142:6789/0 conn(0x7fbf70152ba0 legacy=0x7fbf700cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7015fc00 con 0x7fbf70152f80 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fbf58001670 con 0x7fbf70152f80 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 >> v1:172.21.15.46:6789/0 conn(0x7fbf70152f80 legacy=0x7fbf7006ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf54005bf0 0x7fbf54006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf54006a10 0x7fbf54006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf540073d0 0x7fbf5400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70160530 con 0x7fbf540073d0 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5400e2d0 con 0x7fbf54006a10 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5400e4d0 con 0x7fbf54005bf0 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7006f680 0x7fbf700ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf540073d0 0x7fbf5400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf6ea00640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf54005bf0 0x7fbf54006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.113+0000 7fbf67e00640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf54006a10 0x7fbf54006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.114+0000 7fbf6f400640 1 -- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf54005bf0 msgr2=0x7fbf54006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.114+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf54005bf0 0x7fbf54006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.114+0000 7fbf6f400640 1 -- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf54006a10 msgr2=0x7fbf54006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.114+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf54006a10 0x7fbf54006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.114+0000 7fbf6f400640 1 -- 172.21.15.46:0/1906827123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf54005730 con 0x7fbf540073d0 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.115+0000 7fbf6f400640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf540073d0 0x7fbf5400dbd0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009f00 tx=0x7fbf54007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf45468fdd257ab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.115+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68009b50 con 0x7fbf540073d0 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.115+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf68003990 con 0x7fbf540073d0 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.115+0000 7fbf6e000640 1 -- 172.21.15.46:0/1906827123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003d60 con 0x7fbf540073d0 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 -- 172.21.15.46:0/1906827123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf540073d0 msgr2=0x7fbf5400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/1906827123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf540073d0 0x7fbf5400dbd0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009f00 tx=0x7fbf54007a40 comp rx=0 tx=0).stop 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6f400640 1 -- 172.21.15.46:0/1906827123 reap_dead start 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 -- 172.21.15.46:0/1906827123 shutdown_connections 2024-04-26T23:58:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 -- 172.21.15.46:0/1906827123 >> 172.21.15.46:0/1906827123 conn(0x7fbf700b26d0 msgr2=0x7fbf700b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 -- 172.21.15.46:0/1906827123 shutdown_connections 2024-04-26T23:58:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.116+0000 7fbf6fe00640 1 -- 172.21.15.46:0/1906827123 wait complete. 2024-04-26T23:58:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.118+0000 7fbf6fe00640 1 Processor -- start 2024-04-26T23:58:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.118+0000 7fbf6fe00640 1 -- start start 2024-04-26T23:58:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf540073d0 0x7fbf7014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70142190 0x7fbf701f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf700cf340 con 0x7fbf70073870 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701601c0 con 0x7fbf70142190 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf700cf4c0 con 0x7fbf540073d0 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.119+0000 7fbf6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48724/0 (socket says 172.21.15.46:48724) 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 -- 172.21.15.46:0/142096307 learned_addr learned my addr 172.21.15.46:0/142096307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf67e00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70142190 0x7fbf701f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6ea00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf540073d0 0x7fbf7014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 -- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf540073d0 msgr2=0x7fbf7014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf540073d0 0x7fbf7014d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 -- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70142190 msgr2=0x7fbf701f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70142190 0x7fbf701f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 -- 172.21.15.46:0/142096307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf70142610 con 0x7fbf70073870 2024-04-26T23:58:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.120+0000 7fbf6f400640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fbf6802ae30 tx=0x7fbf68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.121+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68043070 con 0x7fbf70073870 2024-04-26T23:58:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.121+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf680047c0 con 0x7fbf70073870 2024-04-26T23:58:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.121+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6803e6a0 con 0x7fbf70073870 2024-04-26T23:58:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.121+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf700b18e0 con 0x7fbf70073870 2024-04-26T23:58:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.121+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf70073350 con 0x7fbf70073870 2024-04-26T23:58:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.122+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf70073550 con 0x7fbf70073870 2024-04-26T23:58:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.123+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf68046070 con 0x7fbf70073870 2024-04-26T23:58:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.125+0000 7fbf67400640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.125+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7fbf680abc20 con 0x7fbf70073870 2024-04-26T23:58:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.125+0000 7fbf6ea00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.126+0000 7fbf6ea00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 0x7fbf4c06ede0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005490 tx=0x7fbf60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.129+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf70073550 con 0x7fbf70073870 2024-04-26T23:58:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.335+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf700693b0 con 0x7fbf70073870 2024-04-26T23:58:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.336+0000 7fbf67400640 1 -- 172.21.15.46:0/142096307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v213) ==== 100+0+11 (secure 0 0 0) 0x7fbf70142610 con 0x7fbf70073870 2024-04-26T23:58:31.337 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-26T23:58:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 msgr2=0x7fbf4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 0x7fbf4c06ede0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005490 tx=0x7fbf60009040 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 msgr2=0x7fbf70073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fbf6802ae30 tx=0x7fbf68004ab0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 shutdown_connections 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf70073870 0x7fbf70073cf0 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70142190 0x7fbf701f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 --2- 172.21.15.46:0/142096307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf540073d0 0x7fbf7014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 >> 172.21.15.46:0/142096307 conn(0x7fbf700b26d0 msgr2=0x7fbf7006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 shutdown_connections 2024-04-26T23:58:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.339+0000 7fbf6fe00640 1 -- 172.21.15.46:0/142096307 wait complete. 2024-04-26T23:58:31.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:58:31.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.567+0000 7f2673e00640 1 Processor -- start 2024-04-26T23:58:31.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.568+0000 7f2673e00640 1 -- start start 2024-04-26T23:58:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.569+0000 7f2673e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26740ccb20 0x7f267406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.569+0000 7f2673e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f267406c890 0x7f2674162d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.569+0000 7f2673e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674163760 con 0x7f26741632d0 2024-04-26T23:58:31.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.569+0000 7f2673e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2674163cf0 con 0x7f267406c890 2024-04-26T23:58:31.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.569+0000 7f2673e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f267406ccf0 con 0x7f26740ccb20 2024-04-26T23:58:31.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f267406c890 0x7f2674162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f267406c890 0x7f2674162d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48734/0 (socket says 172.21.15.46:48734) 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 -- 172.21.15.46:0/1805956631 learned_addr learned my addr 172.21.15.46:0/1805956631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672a00640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26740ccb20 0x7f267406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 -- 172.21.15.46:0/1805956631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26740ccb20 msgr2=0x7f267406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26740ccb20 0x7f267406c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 -- 172.21.15.46:0/1805956631 >> v1:172.21.15.142:6789/0 conn(0x7f26741632d0 legacy=0x7f267415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.570+0000 7f2672000640 1 -- 172.21.15.46:0/1805956631 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740d19a0 con 0x7f267406c890 2024-04-26T23:58:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2672000640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f267406c890 0x7f2674162d90 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f2660009e80 tx=0x7f266002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d8a37af449edf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2671600640 1 -- 172.21.15.46:0/1805956631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660036070 con 0x7f267406c890 2024-04-26T23:58:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2671600640 1 -- 172.21.15.46:0/1805956631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2660031600 con 0x7f267406c890 2024-04-26T23:58:31.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2671600640 1 -- 172.21.15.46:0/1805956631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600319d0 con 0x7f267406c890 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2673e00640 1 -- 172.21.15.46:0/1805956631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f267406c890 msgr2=0x7f2674162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.572+0000 7f2673e00640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f267406c890 0x7f2674162d90 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f2660009e80 tx=0x7f266002a750 comp rx=0 tx=0).stop 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 -- 172.21.15.46:0/1805956631 shutdown_connections 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f267406c890 0x7f2674162d90 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 --2- 172.21.15.46:0/1805956631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26740ccb20 0x7f267406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 -- 172.21.15.46:0/1805956631 >> 172.21.15.46:0/1805956631 conn(0x7f26740d1c30 msgr2=0x7f26740d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 -- 172.21.15.46:0/1805956631 shutdown_connections 2024-04-26T23:58:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.573+0000 7f2673e00640 1 -- 172.21.15.46:0/1805956631 wait complete. 2024-04-26T23:58:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.575+0000 7f2673e00640 1 Processor -- start 2024-04-26T23:58:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.575+0000 7f2673e00640 1 -- start start 2024-04-26T23:58:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f267406c890 0x7f26740c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 0x7f26740aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26740d0190 con 0x7f26741632d0 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26740cfeb0 con 0x7f26740ccb20 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26740d0470 con 0x7f267406c890 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48746/0 (socket says 172.21.15.46:48746) 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2673400640 1 -- 172.21.15.46:0/2402011591 learned_addr learned my addr 172.21.15.46:0/2402011591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2672a00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f267406c890 0x7f26740c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.576+0000 7f2672000640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 0x7f26740aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 -- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f267406c890 msgr2=0x7f26740c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f267406c890 0x7f26740c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 -- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 msgr2=0x7f26740aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 0x7f26740aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 -- 172.21.15.46:0/2402011591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740c8d00 con 0x7f26741632d0 2024-04-26T23:58:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2672000640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 0x7f26740aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673400640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f266c00c3c0 tx=0x7f266c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c0052d0 con 0x7f26741632d0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c005470 con 0x7f26741632d0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c007240 con 0x7f26741632d0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741f8c90 con 0x7f26741632d0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.577+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26740c8fa0 con 0x7f26741632d0 2024-04-26T23:58:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.578+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26740693b0 con 0x7f26741632d0 2024-04-26T23:58:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.580+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f266c0073e0 con 0x7f26741632d0 2024-04-26T23:58:31.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.581+0000 7f266aa00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 0x7f265406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:31.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.581+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f266c08b120 con 0x7f26741632d0 2024-04-26T23:58:31.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.582+0000 7f2672a00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 0x7f265406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.583+0000 7f2672a00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 0x7f265406ede0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.585+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f266c059300 con 0x7f26741632d0 2024-04-26T23:58:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.831+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2674076430 con 0x7f26741632d0 2024-04-26T23:58:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.832+0000 7f266aa00640 1 -- 172.21.15.46:0/2402011591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f266c058c00 con 0x7f26741632d0 2024-04-26T23:58:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:31.834 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:58:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:58:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 msgr2=0x7f265406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 0x7f265406ede0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664008040 comp rx=0 tx=0).stop 2024-04-26T23:58:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 msgr2=0x7f26741f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f266c00c3c0 tx=0x7f266c00c880 comp rx=0 tx=0).stop 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 shutdown_connections 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f265406c920 0x7f265406ede0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26741632d0 0x7f26741f81f0 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26740ccb20 0x7f26740aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 --2- 172.21.15.46:0/2402011591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f267406c890 0x7f26740c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 >> 172.21.15.46:0/2402011591 conn(0x7f26740d1c30 msgr2=0x7f267406eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 shutdown_connections 2024-04-26T23:58:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:31.835+0000 7f2673e00640 1 -- 172.21.15.46:0/2402011591 wait complete. 2024-04-26T23:58:31.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:58:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.064+0000 7f4fd6400640 1 Processor -- start 2024-04-26T23:58:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.064+0000 7f4fd6400640 1 -- start start 2024-04-26T23:58:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4fd01454a0 0x7f4fd00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd0072bb0 con 0x7f4fd0073970 2024-04-26T23:58:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd6400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd0072e90 con 0x7f4fd01454a0 2024-04-26T23:58:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd0073170 con 0x7f4fd00c6a50 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fd5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4fd01454a0 0x7f4fd00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fcfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4fd0073970 0x7f4fd00cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36564/0 (socket says 172.21.15.46:36564) 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.066+0000 7f4fcfe00640 1 -- 172.21.15.46:0/2601655123 learned_addr learned my addr 172.21.15.46:0/2601655123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.067+0000 7f4fd5000640 1 -- 172.21.15.46:0/2601655123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 msgr2=0x7f4fd00c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.067+0000 7f4fd5000640 1 --2- 172.21.15.46:0/2601655123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.067+0000 7f4fd5000640 1 -- 172.21.15.46:0/2601655123 >> v1:172.21.15.46:6789/0 conn(0x7f4fd0073970 legacy=0x7f4fd00cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.067+0000 7f4fd5000640 1 -- 172.21.15.46:0/2601655123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd0073450 con 0x7f4fd01454a0 2024-04-26T23:58:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.068+0000 7f4fd5000640 1 --2- 172.21.15.46:0/2601655123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd00cc690 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40097a0 tx=0x7f4fc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991357a5a573b08c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.068+0000 7f4fcf400640 1 -- 172.21.15.46:0/2601655123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4037070 con 0x7f4fd01454a0 2024-04-26T23:58:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.068+0000 7f4fcf400640 1 -- 172.21.15.46:0/2601655123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc4032440 con 0x7f4fd01454a0 2024-04-26T23:58:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.068+0000 7f4fcf400640 1 -- 172.21.15.46:0/2601655123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4032810 con 0x7f4fd01454a0 2024-04-26T23:58:32.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 -- 172.21.15.46:0/2601655123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 msgr2=0x7f4fd00cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 --2- 172.21.15.46:0/2601655123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd00cc690 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40097a0 tx=0x7f4fc402a180 comp rx=0 tx=0).stop 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 -- 172.21.15.46:0/2601655123 shutdown_connections 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 --2- 172.21.15.46:0/2601655123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 --2- 172.21.15.46:0/2601655123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd00cc690 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 -- 172.21.15.46:0/2601655123 >> 172.21.15.46:0/2601655123 conn(0x7f4fd00b26d0 msgr2=0x7f4fd00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 -- 172.21.15.46:0/2601655123 shutdown_connections 2024-04-26T23:58:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.069+0000 7f4fd6400640 1 -- 172.21.15.46:0/2601655123 wait complete. 2024-04-26T23:58:32.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.071+0000 7f4fd6400640 1 Processor -- start 2024-04-26T23:58:32.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.071+0000 7f4fd6400640 1 -- start start 2024-04-26T23:58:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fd0073970 0x7f4fd015fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd0145880 con 0x7f4fd01454a0 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd0073e00 con 0x7f4fd0073970 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd6400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd0145a00 con 0x7f4fd00c6a50 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60046/0 (socket says 172.21.15.46:60046) 2024-04-26T23:58:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.072+0000 7f4fd5a00640 1 -- 172.21.15.46:0/355487034 learned_addr learned my addr 172.21.15.46:0/355487034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 -- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 msgr2=0x7f4fd015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 -- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fd0073970 msgr2=0x7f4fd015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fd0073970 0x7f4fd015fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 -- 172.21.15.46:0/355487034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd00ca2d0 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd5a00640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b970 tx=0x7f4fc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8019070 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc800cb60 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8014440 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.073+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd00ca5b0 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.074+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd015d660 con 0x7f4fd01454a0 2024-04-26T23:58:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.075+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd015d860 con 0x7f4fd01454a0 2024-04-26T23:58:32.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.076+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fc801c070 con 0x7f4fd01454a0 2024-04-26T23:58:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.077+0000 7f4fccc00640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 0x7f4fb006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.077+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f4fc808b9f0 con 0x7f4fd01454a0 2024-04-26T23:58:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.078+0000 7f4fd5000640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 0x7f4fb006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.079+0000 7f4fd5000640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 0x7f4fb006ee00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4034000 tx=0x7f4fc402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.081+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fd015d860 con 0x7f4fd01454a0 2024-04-26T23:58:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.286+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fd00693b0 con 0x7f4fd01454a0 2024-04-26T23:58:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.288+0000 7f4fccc00640 1 -- 172.21.15.46:0/355487034 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) ==== 74+0+17189 (secure 0 0 0) 0x7f4fc8059d50 con 0x7f4fd01454a0 2024-04-26T23:58:32.290 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":213,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:59.707896+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.6","mappings":[{"from":2,"to":0}]},{"pgid":"32.c","mappings":[{"from":6,"to":5}]},{"pgid":"32.17","mappings":[{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/3782420006":"2024-04-27T23:57:18.527554+0000","172.21.15.46:6833/3782420006":"2024-04-27T23:57:18.527554+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:58:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 msgr2=0x7f4fb006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 0x7f4fb006ee00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4034000 tx=0x7f4fc402c000 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 msgr2=0x7f4fd0156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b970 tx=0x7f4fc800be30 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 shutdown_connections 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f4fb006c940 0x7f4fb006ee00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd01454a0 0x7f4fd0156300 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4fd00c6a50 0x7f4fd015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 --2- 172.21.15.46:0/355487034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fd0073970 0x7f4fd015fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.291+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 >> 172.21.15.46:0/355487034 conn(0x7f4fd00b26d0 msgr2=0x7f4fd00cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.292+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 shutdown_connections 2024-04-26T23:58:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.292+0000 7f4fd6400640 1 -- 172.21.15.46:0/355487034 wait complete. 2024-04-26T23:58:32.311 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:58:32.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:58:32.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.523+0000 7f1ccb200640 1 Processor -- start 2024-04-26T23:58:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.523+0000 7f1ccb200640 1 -- start start 2024-04-26T23:58:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.525+0000 7f1ccb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ccc1466b0 0x7f1ccc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.525+0000 7f1ccb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.525+0000 7f1ccb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ccc073ad0 con 0x7f1ccc160910 2024-04-26T23:58:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.525+0000 7f1ccb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1ccc146a90 con 0x7f1ccc1466b0 2024-04-26T23:58:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.525+0000 7f1ccb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc073d70 con 0x7f1ccc0720e0 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cca800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ccc160910 0x7f1ccc0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36568/0 (socket says 172.21.15.46:36568) 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ccc1466b0 0x7f1ccc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cca800640 1 -- 172.21.15.46:0/2414809287 learned_addr learned my addr 172.21.15.46:0/2414809287 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9400640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc8a00640 1 -- 172.21.15.46:0/2414809287 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741398084 0 0) 0x7f1ccc073ad0 con 0x7f1ccc160910 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc8a00640 1 -- 172.21.15.46:0/2414809287 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cb0003520 con 0x7f1ccc160910 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9e00640 1 -- 172.21.15.46:0/2414809287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 msgr2=0x7f1ccc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9e00640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9e00640 1 -- 172.21.15.46:0/2414809287 >> v1:172.21.15.46:6789/0 conn(0x7f1ccc160910 legacy=0x7f1ccc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.526+0000 7f1cc9e00640 1 -- 172.21.15.46:0/2414809287 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc074050 con 0x7f1ccc1466b0 2024-04-26T23:58:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.528+0000 7f1cc9e00640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc1466b0 0x7f1ccc06de80 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0097a0 tx=0x7f1cbc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd53a71308e1d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1cc8a00640 1 -- 172.21.15.46:0/2414809287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc037070 con 0x7f1ccc1466b0 2024-04-26T23:58:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1cc8a00640 1 -- 172.21.15.46:0/2414809287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cbc032440 con 0x7f1ccc1466b0 2024-04-26T23:58:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1cc8a00640 1 -- 172.21.15.46:0/2414809287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc032810 con 0x7f1ccc1466b0 2024-04-26T23:58:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 -- 172.21.15.46:0/2414809287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc1466b0 msgr2=0x7f1ccc06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc1466b0 0x7f1ccc06de80 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0097a0 tx=0x7f1cbc02a180 comp rx=0 tx=0).stop 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 -- 172.21.15.46:0/2414809287 shutdown_connections 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 --2- 172.21.15.46:0/2414809287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc1466b0 0x7f1ccc06de80 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.529+0000 7f1ccb200640 1 -- 172.21.15.46:0/2414809287 >> 172.21.15.46:0/2414809287 conn(0x7f1ccc0b26d0 msgr2=0x7f1ccc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.530+0000 7f1ccb200640 1 -- 172.21.15.46:0/2414809287 shutdown_connections 2024-04-26T23:58:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.530+0000 7f1ccb200640 1 -- 172.21.15.46:0/2414809287 wait complete. 2024-04-26T23:58:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.531+0000 7f1ccb200640 1 Processor -- start 2024-04-26T23:58:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.531+0000 7f1ccb200640 1 -- start start 2024-04-26T23:58:32.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ccc0c8440 0x7f1ccc0af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc06e3c0 con 0x7f1ccc160910 2024-04-26T23:58:32.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc0cb470 con 0x7f1ccc0c8440 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.532+0000 7f1ccb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc0cdda0 con 0x7f1ccc0720e0 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60074/0 (socket says 172.21.15.46:60074) 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 -- 172.21.15.46:0/1433758265 learned_addr learned my addr 172.21.15.46:0/1433758265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9e00640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cca800640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ccc0c8440 0x7f1ccc0af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 -- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 msgr2=0x7f1ccc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 -- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ccc0c8440 msgr2=0x7f1ccc0af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ccc0c8440 0x7f1ccc0af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.533+0000 7f1cc9400640 1 -- 172.21.15.46:0/1433758265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc0c9bf0 con 0x7f1ccc160910 2024-04-26T23:58:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1cc9400640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e870 tx=0x7f1cb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8017070 con 0x7f1ccc160910 2024-04-26T23:58:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb800c940 con 0x7f1ccc160910 2024-04-26T23:58:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800cd30 con 0x7f1ccc160910 2024-04-26T23:58:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc0c9ed0 con 0x7f1ccc160910 2024-04-26T23:58:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.534+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc0aebc0 con 0x7f1ccc160910 2024-04-26T23:58:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.535+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc0693b0 con 0x7f1ccc160910 2024-04-26T23:58:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.536+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cb801a070 con 0x7f1ccc160910 2024-04-26T23:58:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.537+0000 7f1cc2000640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.537+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f1cb808bff0 con 0x7f1ccc160910 2024-04-26T23:58:32.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.538+0000 7f1cc9e00640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:32.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.539+0000 7f1cc9e00640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 0x7f1cac06edb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc034000 tx=0x7f1cbc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.542+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cb805a1d0 con 0x7f1ccc160910 2024-04-26T23:58:32.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.786+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ccc0c7180 con 0x7f1ccc160910 2024-04-26T23:58:32.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.788+0000 7f1cc2000640 1 -- 172.21.15.46:0/1433758265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f1cb8059ad0 con 0x7f1ccc160910 2024-04-26T23:58:32.789 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:32.789 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:58:32.789 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.790+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 msgr2=0x7f1cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.790+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 0x7f1cac06edb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc034000 tx=0x7f1cbc02e000 comp rx=0 tx=0).stop 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.790+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 msgr2=0x7f1ccc0c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.790+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e870 tx=0x7f1cb800ed30 comp rx=0 tx=0).stop 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 shutdown_connections 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ccc0c8440 0x7f1ccc0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ccc160910 0x7f1ccc0c92c0 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 --2- 172.21.15.46:0/1433758265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ccc0720e0 0x7f1ccc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:32.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 >> 172.21.15.46:0/1433758265 conn(0x7f1ccc0b26d0 msgr2=0x7f1ccc0c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:32.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 shutdown_connections 2024-04-26T23:58:32.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:32.791+0000 7f1ccb200640 1 -- 172.21.15.46:0/1433758265 wait complete. 2024-04-26T23:58:32.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:58:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.017+0000 7f7b8ea00640 1 Processor -- start 2024-04-26T23:58:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.018+0000 7f7b8ea00640 1 -- start start 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.019+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b881466b0 0x7f7b8806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b88160910 0x7f7b88160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b88073ad0 con 0x7f7b880720e0 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7b88146a90 con 0x7f7b88160910 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b88073d70 con 0x7f7b881466b0 2024-04-26T23:58:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b88160910 0x7f7b88160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b88160910 0x7f7b88160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60090/0 (socket says 172.21.15.46:60090) 2024-04-26T23:58:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8e000640 1 -- 172.21.15.46:0/1595823011 learned_addr learned my addr 172.21.15.46:0/1595823011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.020+0000 7f7b8d600640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b881466b0 0x7f7b8806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.021+0000 7f7b8e000640 1 -- 172.21.15.46:0/1595823011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b881466b0 msgr2=0x7f7b8806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.021+0000 7f7b8e000640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b881466b0 0x7f7b8806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.021+0000 7f7b8e000640 1 -- 172.21.15.46:0/1595823011 >> v1:172.21.15.142:6789/0 conn(0x7f7b880720e0 legacy=0x7f7b880cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.021+0000 7f7b8e000640 1 -- 172.21.15.46:0/1595823011 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b88074050 con 0x7f7b88160910 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.022+0000 7f7b8e000640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88160910 0x7f7b88160da0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009e50 tx=0x7f7b8002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee78e1b04a725545 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.022+0000 7f7b87e00640 1 -- 172.21.15.46:0/1595823011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80036070 con 0x7f7b88160910 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.022+0000 7f7b87e00640 1 -- 172.21.15.46:0/1595823011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80031600 con 0x7f7b88160910 2024-04-26T23:58:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.022+0000 7f7b87e00640 1 -- 172.21.15.46:0/1595823011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800319d0 con 0x7f7b88160910 2024-04-26T23:58:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 -- 172.21.15.46:0/1595823011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88160910 msgr2=0x7f7b88160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88160910 0x7f7b88160da0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009e50 tx=0x7f7b8002a710 comp rx=0 tx=0).stop 2024-04-26T23:58:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 -- 172.21.15.46:0/1595823011 shutdown_connections 2024-04-26T23:58:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88160910 0x7f7b88160da0 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/1595823011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b881466b0 0x7f7b8806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 -- 172.21.15.46:0/1595823011 >> 172.21.15.46:0/1595823011 conn(0x7f7b880b26d0 msgr2=0x7f7b880b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 -- 172.21.15.46:0/1595823011 shutdown_connections 2024-04-26T23:58:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.023+0000 7f7b8ea00640 1 -- 172.21.15.46:0/1595823011 wait complete. 2024-04-26T23:58:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.025+0000 7f7b8ea00640 1 Processor -- start 2024-04-26T23:58:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.025+0000 7f7b8ea00640 1 -- start start 2024-04-26T23:58:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b880720e0 0x7f7b8813ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b88160910 0x7f7b880cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8806e3c0 con 0x7f7b881466b0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880cb470 con 0x7f7b880720e0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b880cb990 con 0x7f7b88160910 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.026+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60092/0 (socket says 172.21.15.46:60092) 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 -- 172.21.15.46:0/3992754330 learned_addr learned my addr 172.21.15.46:0/3992754330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8d600640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b880720e0 0x7f7b8813ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8e000640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b88160910 0x7f7b880cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 -- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b88160910 msgr2=0x7f7b880cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b88160910 0x7f7b880cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 -- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b880720e0 msgr2=0x7f7b8813ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b880720e0 0x7f7b8813ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 -- 172.21.15.46:0/3992754330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8813c8c0 con 0x7f7b881466b0 2024-04-26T23:58:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.027+0000 7f7b8cc00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c400 tx=0x7f7b7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.028+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740052d0 con 0x7f7b881466b0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.028+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b74005490 con 0x7f7b881466b0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.028+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74007260 con 0x7f7b881466b0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.028+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b880b4900 con 0x7f7b881466b0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.028+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8813d460 con 0x7f7b881466b0 2024-04-26T23:58:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.029+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b880693b0 con 0x7f7b881466b0 2024-04-26T23:58:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.030+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b74007400 con 0x7f7b881466b0 2024-04-26T23:58:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.031+0000 7f7b85600640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 0x7f7b6806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.031+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f7b7408b170 con 0x7f7b881466b0 2024-04-26T23:58:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.032+0000 7f7b8d600640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 0x7f7b6806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.033+0000 7f7b8d600640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 0x7f7b6806ee00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ad0 tx=0x7f7b7c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.035+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b8813c8c0 con 0x7f7b881466b0 2024-04-26T23:58:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.274+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b880c7180 con 0x7f7b881466b0 2024-04-26T23:58:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.275+0000 7f7b85600640 1 -- 172.21.15.46:0/3992754330 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+5205 (secure 0 0 0) 0x7f7b740594d0 con 0x7f7b881466b0 2024-04-26T23:58:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:33.277 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":174,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:57:04.513003+0000","modified":"2024-04-26T23:57:23.570033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":211,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12273":{"gid":12273,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2610577522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2610577522},{"type":"v1","addr":"172.21.15.46:6835","nonce":2610577522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12273,"qdb_cluster":[12273]},"id":15}]} 2024-04-26T23:58:33.277 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 174 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.277+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 msgr2=0x7f7b6806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.277+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 0x7f7b6806ee00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ad0 tx=0x7f7b7c008040 comp rx=0 tx=0).stop 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 msgr2=0x7f7b8813c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c400 tx=0x7f7b7400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 shutdown_connections 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f7b6806c940 0x7f7b6806ee00 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b88160910 0x7f7b880cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b881466b0 0x7f7b8813c380 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 --2- 172.21.15.46:0/3992754330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b880720e0 0x7f7b8813ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 >> 172.21.15.46:0/3992754330 conn(0x7f7b880b26d0 msgr2=0x7f7b880ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 shutdown_connections 2024-04-26T23:58:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.278+0000 7f7b8ea00640 1 -- 172.21.15.46:0/3992754330 wait complete. 2024-04-26T23:58:33.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:58:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.492+0000 7f8fe7e00640 1 Processor -- start 2024-04-26T23:58:33.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.492+0000 7f8fe7e00640 1 -- start start 2024-04-26T23:58:33.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fe81466b0 0x7f8fe806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8fe80720e0 0x7f8fe80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe7e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe8073ad0 con 0x7f8fe8160910 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe8146a90 con 0x7f8fe81466b0 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe8073d70 con 0x7f8fe80720e0 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fe81466b0 0x7f8fe806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8fe81466b0 0x7f8fe806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60106/0 (socket says 172.21.15.46:60106) 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe6a00640 1 -- 172.21.15.46:0/1479398199 learned_addr learned my addr 172.21.15.46:0/1479398199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe6a00640 1 -- 172.21.15.46:0/1479398199 >> v2:172.21.15.142:3300/0 conn(0x7f8fe80720e0 msgr2=0x7f8fe80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.494+0000 7f8fe6a00640 1 --2- 172.21.15.46:0/1479398199 >> v2:172.21.15.142:3300/0 conn(0x7f8fe80720e0 0x7f8fe80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.495+0000 7f8fe6a00640 1 -- 172.21.15.46:0/1479398199 >> v1:172.21.15.142:6789/0 conn(0x7f8fe8160910 legacy=0x7f8fe80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.495+0000 7f8fe6a00640 1 -- 172.21.15.46:0/1479398199 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe8074050 con 0x7f8fe81466b0 2024-04-26T23:58:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe6a00640 1 --2- 172.21.15.46:0/1479398199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe81466b0 0x7f8fe806de80 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009870 tx=0x7f8fd802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9716a7f344808d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe6000640 1 -- 172.21.15.46:0/1479398199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8037070 con 0x7f8fe81466b0 2024-04-26T23:58:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe6000640 1 -- 172.21.15.46:0/1479398199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd8032440 con 0x7f8fe81466b0 2024-04-26T23:58:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe6000640 1 -- 172.21.15.46:0/1479398199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8032810 con 0x7f8fe81466b0 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe7e00640 1 -- 172.21.15.46:0/1479398199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe81466b0 msgr2=0x7f8fe806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.496+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/1479398199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe81466b0 0x7f8fe806de80 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009870 tx=0x7f8fd802a250 comp rx=0 tx=0).stop 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 -- 172.21.15.46:0/1479398199 shutdown_connections 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/1479398199 >> v2:172.21.15.142:3300/0 conn(0x7f8fe80720e0 0x7f8fe80cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/1479398199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe81466b0 0x7f8fe806de80 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 -- 172.21.15.46:0/1479398199 >> 172.21.15.46:0/1479398199 conn(0x7f8fe80b26d0 msgr2=0x7f8fe80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 -- 172.21.15.46:0/1479398199 shutdown_connections 2024-04-26T23:58:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.497+0000 7f8fe7e00640 1 -- 172.21.15.46:0/1479398199 wait complete. 2024-04-26T23:58:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.498+0000 7f8fe7e00640 1 Processor -- start 2024-04-26T23:58:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 -- start start 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fe80720e0 0x7f8fe8137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 0x7f8fe8134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe806e3c0 con 0x7f8fe8160910 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80cb470 con 0x7f8fe81466b0 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.499+0000 7f8fe7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe80cdda0 con 0x7f8fe80720e0 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60116/0 (socket says 172.21.15.46:60116) 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 -- 172.21.15.46:0/641960690 learned_addr learned my addr 172.21.15.46:0/641960690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe6a00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fe80720e0 0x7f8fe8137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 -- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fe80720e0 msgr2=0x7f8fe8137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fe80720e0 0x7f8fe8137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 -- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 msgr2=0x7f8fe8134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fdfe00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 0x7f8fe8134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 0x7f8fe8134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fe7400640 1 -- 172.21.15.46:0/641960690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe8137a70 con 0x7f8fe8160910 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.500+0000 7f8fdfe00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 0x7f8fe8134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fe7400640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000b990 tx=0x7f8fe000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0019070 con 0x7f8fe8160910 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe000cba0 con 0x7f8fe8160910 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0014440 con 0x7f8fe8160910 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe8134e70 con 0x7f8fe8160910 2024-04-26T23:58:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.501+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe8135710 con 0x7f8fe8160910 2024-04-26T23:58:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.502+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe80693b0 con 0x7f8fe8160910 2024-04-26T23:58:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.504+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fe001c070 con 0x7f8fe8160910 2024-04-26T23:58:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.504+0000 7f8fdea00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 0x7f8fc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.505+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f8fe008baa0 con 0x7f8fe8160910 2024-04-26T23:58:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.505+0000 7f8fe6a00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 0x7f8fc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.506+0000 7f8fe6a00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 0x7f8fc406ede0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8034000 tx=0x7f8fd802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.509+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8fe8137a70 con 0x7f8fe8160910 2024-04-26T23:58:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.711+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fe80b2440 con 0x7f8fe8160910 2024-04-26T23:58:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.714+0000 7f8fdea00640 1 -- 172.21.15.46:0/641960690 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) ==== 74+0+17189 (secure 0 0 0) 0x7f8fe0059c80 con 0x7f8fe8160910 2024-04-26T23:58:33.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:33.716 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":213,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:57:59.707896+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:57:02.313551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-04-26T23:57:03.507616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.6","mappings":[{"from":2,"to":0}]},{"pgid":"32.c","mappings":[{"from":6,"to":5}]},{"pgid":"32.17","mappings":[{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/3782420006":"2024-04-27T23:57:18.527554+0000","172.21.15.46:6833/3782420006":"2024-04-27T23:57:18.527554+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:58:33.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 msgr2=0x7f8fc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 0x7f8fc406ede0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8034000 tx=0x7f8fd802e000 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 msgr2=0x7f8fe8137f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000b990 tx=0x7f8fe000be50 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 shutdown_connections 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f8fc406c920 0x7f8fc406ede0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe8160910 0x7f8fe8137f80 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe81466b0 0x7f8fe8134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 --2- 172.21.15.46:0/641960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8fe80720e0 0x7f8fe8137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 >> 172.21.15.46:0/641960690 conn(0x7f8fe80b26d0 msgr2=0x7f8fe81600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 shutdown_connections 2024-04-26T23:58:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.717+0000 7f8fe7e00640 1 -- 172.21.15.46:0/641960690 wait complete. 2024-04-26T23:58:33.737 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:58:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.944+0000 7f973fe00640 1 Processor -- start 2024-04-26T23:58:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.944+0000 7f973fe00640 1 -- start start 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.945+0000 7f973fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974015edf0 0x7f97400cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f974015f1d0 0x7f97400cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97400b03c0 con 0x7f97400cb1a0 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f97400cf5c0 con 0x7f974015f1d0 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97400cb680 con 0x7f974015edf0 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97400cb1a0 0x7f97400ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36576/0 (socket says 172.21.15.46:36576) 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973f400640 1 -- 172.21.15.46:0/3771516172 learned_addr learned my addr 172.21.15.46:0/3771516172 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:33.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973e000640 1 --2- 172.21.15.46:0/3771516172 >> v2:172.21.15.46:3300/0 conn(0x7f974015f1d0 0x7f97400cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973ea00640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974015edf0 0x7f97400cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.946+0000 7f973d600640 1 -- 172.21.15.46:0/3771516172 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313474670 0 0) 0x7f97400b03c0 con 0x7f97400cb1a0 2024-04-26T23:58:33.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.947+0000 7f973e000640 1 -- 172.21.15.46:0/3771516172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974015edf0 msgr2=0x7f97400cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:33.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.947+0000 7f973e000640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974015edf0 0x7f97400cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.947+0000 7f973e000640 1 -- 172.21.15.46:0/3771516172 >> v1:172.21.15.46:6789/0 conn(0x7f97400cb1a0 legacy=0x7f97400ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.947+0000 7f973e000640 1 -- 172.21.15.46:0/3771516172 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97400cb960 con 0x7f974015f1d0 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973e000640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015f1d0 0x7f97400cd060 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f97280097a0 tx=0x7f972802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a9d86aecd910de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973d600640 1 -- 172.21.15.46:0/3771516172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037070 con 0x7f974015f1d0 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973d600640 1 -- 172.21.15.46:0/3771516172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728032440 con 0x7f974015f1d0 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973d600640 1 -- 172.21.15.46:0/3771516172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728032810 con 0x7f974015f1d0 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973fe00640 1 -- 172.21.15.46:0/3771516172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015f1d0 msgr2=0x7f97400cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.948+0000 7f973fe00640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015f1d0 0x7f97400cd060 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f97280097a0 tx=0x7f972802a210 comp rx=0 tx=0).stop 2024-04-26T23:58:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 -- 172.21.15.46:0/3771516172 shutdown_connections 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015f1d0 0x7f97400cd060 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 --2- 172.21.15.46:0/3771516172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974015edf0 0x7f97400cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 -- 172.21.15.46:0/3771516172 >> 172.21.15.46:0/3771516172 conn(0x7f97400b26d0 msgr2=0x7f97400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 -- 172.21.15.46:0/3771516172 shutdown_connections 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.949+0000 7f973fe00640 1 -- 172.21.15.46:0/3771516172 wait complete. 2024-04-26T23:58:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.951+0000 7f973fe00640 1 Processor -- start 2024-04-26T23:58:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.951+0000 7f973fe00640 1 -- start start 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97400cb1a0 0x7f97400c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f974015edf0 0x7f97400c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cc200 con 0x7f974006f9b0 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cbf20 con 0x7f974015edf0 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97400c6950 con 0x7f97400cb1a0 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60128/0 (socket says 172.21.15.46:60128) 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 -- 172.21.15.46:0/516629789 learned_addr learned my addr 172.21.15.46:0/516629789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 -- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97400cb1a0 msgr2=0x7f97400c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973ea00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97400cb1a0 0x7f97400c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973e000640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f974015edf0 0x7f97400c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97400cb1a0 0x7f97400c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 -- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f974015edf0 msgr2=0x7f97400c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f974015edf0 0x7f97400c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.952+0000 7f973f400640 1 -- 172.21.15.46:0/516629789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974006fe60 con 0x7f974006f9b0 2024-04-26T23:58:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f973f400640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f973800cb20 tx=0x7f973800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800aca0 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973800ae40 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738012640 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974014dfc0 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.953+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97400ca460 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.954+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97400693b0 con 0x7f974006f9b0 2024-04-26T23:58:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.955+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97380127e0 con 0x7f974006f9b0 2024-04-26T23:58:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.956+0000 7f9736a00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.956+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 6274+0+0 (secure 0 0 0) 0x7f973808c420 con 0x7f974006f9b0 2024-04-26T23:58:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.957+0000 7f973ea00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:33.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.958+0000 7f973ea00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:33.961+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f973805a570 con 0x7f974006f9b0 2024-04-26T23:58:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.207+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f974015ddc0 con 0x7f974006f9b0 2024-04-26T23:58:34.209 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-26T23:58:34.208+0000 7f0752800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:58:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.212+0000 7f9736a00640 1 -- 172.21.15.46:0/516629789 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v175) ==== 161+0+0 (secure 0 0 0) 0x7f9738059e70 con 0x7f974006f9b0 2024-04-26T23:58:34.213 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:58:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.215+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 msgr2=0x7f971c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.215+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730008040 comp rx=0 tx=0).stop 2024-04-26T23:58:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 msgr2=0x7f97400b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f973800cb20 tx=0x7f973800a060 comp rx=0 tx=0).stop 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 shutdown_connections 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974006f9b0 0x7f97400b1690 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f974015edf0 0x7f97400c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 --2- 172.21.15.46:0/516629789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97400cb1a0 0x7f97400c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 >> 172.21.15.46:0/516629789 conn(0x7f97400b26d0 msgr2=0x7f97400b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 shutdown_connections 2024-04-26T23:58:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.216+0000 7f973fe00640 1 -- 172.21.15.46:0/516629789 wait complete. 2024-04-26T23:58:34.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:58:34.244 INFO:tasks.ceph.mds.a.smithi046.stderr: -3196> 2024-04-26T23:57:24.546+0000 7f44f6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:58:34.299 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-26T23:58:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.417+0000 7f30a2400640 1 Processor -- start 2024-04-26T23:58:34.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.418+0000 7f30a2400640 1 -- start start 2024-04-26T23:58:34.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 0x7f309c073c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f309c0cfed0 0x7f309c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f309c0d0dc0 0x7f309c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f309c145ff0 con 0x7f309c0d0dc0 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f309c0d1230 con 0x7f309c0cfed0 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a2400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f309c06ec30 con 0x7f309c145c10 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f309c0d0dc0 0x7f309c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f309c0d0dc0 0x7f309c071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60154/0 (socket says 172.21.15.46:60154) 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.419+0000 7f30a1a00640 1 -- 172.21.15.46:0/1068820824 learned_addr learned my addr 172.21.15.46:0/1068820824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f309be00640 1 --2- 172.21.15.46:0/1068820824 >> v2:172.21.15.142:3300/0 conn(0x7f309c0cfed0 0x7f309c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1000640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 0x7f309c073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1a00640 1 -- 172.21.15.46:0/1068820824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 msgr2=0x7f309c073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 0x7f309c073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1a00640 1 -- 172.21.15.46:0/1068820824 >> v2:172.21.15.142:3300/0 conn(0x7f309c0cfed0 msgr2=0x7f309c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1068820824 >> v2:172.21.15.142:3300/0 conn(0x7f309c0cfed0 0x7f309c0d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1a00640 1 -- 172.21.15.46:0/1068820824 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c0cf3f0 con 0x7f309c0d0dc0 2024-04-26T23:58:34.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.420+0000 7f30a1000640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 0x7f309c073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.421+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c0d0dc0 0x7f309c071e20 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f309c06ff10 tx=0x7f309402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d97d3ef3fae63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.421+0000 7f309b400640 1 -- 172.21.15.46:0/1068820824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094036070 con 0x7f309c0d0dc0 2024-04-26T23:58:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.421+0000 7f309b400640 1 -- 172.21.15.46:0/1068820824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3094031600 con 0x7f309c0d0dc0 2024-04-26T23:58:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.421+0000 7f309b400640 1 -- 172.21.15.46:0/1068820824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940319d0 con 0x7f309c0d0dc0 2024-04-26T23:58:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 -- 172.21.15.46:0/1068820824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c0d0dc0 msgr2=0x7f309c071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c0d0dc0 0x7f309c071e20 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f309c06ff10 tx=0x7f309402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 -- 172.21.15.46:0/1068820824 shutdown_connections 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c0d0dc0 0x7f309c071e20 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 --2- 172.21.15.46:0/1068820824 >> v2:172.21.15.142:3300/0 conn(0x7f309c0cfed0 0x7f309c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 --2- 172.21.15.46:0/1068820824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c145c10 0x7f309c073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 -- 172.21.15.46:0/1068820824 >> 172.21.15.46:0/1068820824 conn(0x7f309c06c350 msgr2=0x7f309c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 -- 172.21.15.46:0/1068820824 shutdown_connections 2024-04-26T23:58:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.422+0000 7f30a2400640 1 -- 172.21.15.46:0/1068820824 wait complete. 2024-04-26T23:58:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.424+0000 7f30a2400640 1 Processor -- start 2024-04-26T23:58:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.424+0000 7f30a2400640 1 -- start start 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c0cfed0 0x7f309c15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 0x7f309c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f309c0cf0a0 con 0x7f309c145c10 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f309c0cf9b0 con 0x7f309c0d0dc0 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a2400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f309c1468b0 con 0x7f309c0cfed0 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60166/0 (socket says 172.21.15.46:60166) 2024-04-26T23:58:34.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.425+0000 7f30a1a00640 1 -- 172.21.15.46:0/1761935036 learned_addr learned my addr 172.21.15.46:0/1761935036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 -- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c0cfed0 msgr2=0x7f309c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f309be00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 0x7f309c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1000640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c0cfed0 0x7f309c15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c0cfed0 0x7f309c15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 -- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 msgr2=0x7f309c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 0x7f309c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 -- 172.21.15.46:0/1761935036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c15fa70 con 0x7f309c145c10 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f309be00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 0x7f309c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.426+0000 7f30a1a00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f3094009f40 tx=0x7f3094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.427+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094036050 con 0x7f309c145c10 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.427+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3094042070 con 0x7f309c145c10 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.427+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094009510 con 0x7f309c145c10 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.427+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c15fd50 con 0x7f309c145c10 2024-04-26T23:58:34.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.427+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c15a2c0 con 0x7f309c145c10 2024-04-26T23:58:34.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.428+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309c0693b0 con 0x7f309c145c10 2024-04-26T23:58:34.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.429+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3094007bd0 con 0x7f309c145c10 2024-04-26T23:58:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.430+0000 7f3098c00640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 0x7f307c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.430+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(214..214 src has 1..214) ==== 6360+0+0 (secure 0 0 0) 0x7f30940ad830 con 0x7f309c145c10 2024-04-26T23:58:34.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.431+0000 7f30a1000640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 0x7f307c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:34.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.432+0000 7f30a1000640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 0x7f307c06edb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f309000b7c0 tx=0x7f3090008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.434+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f309407b9b0 con 0x7f309c145c10 2024-04-26T23:58:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:34.669+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f309c160790 con 0x7f309c145c10 2024-04-26T23:58:35.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.215+0000 7f3098c00640 1 -- 172.21.15.46:0/1761935036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v176) ==== 104+0+0 (secure 0 0 0) 0x7f309407b2b0 con 0x7f309c145c10 2024-04-26T23:58:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.218+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 msgr2=0x7f307c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.218+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 0x7f307c06edb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f309000b7c0 tx=0x7f3090008040 comp rx=0 tx=0).stop 2024-04-26T23:58:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.218+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 msgr2=0x7f309c0b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f3094009f40 tx=0x7f3094004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 shutdown_connections 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f307c06c8f0 0x7f307c06edb0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c145c10 0x7f309c0b3ad0 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f309c0d0dc0 0x7f309c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 --2- 172.21.15.46:0/1761935036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f309c0cfed0 0x7f309c15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 >> 172.21.15.46:0/1761935036 conn(0x7f309c06c350 msgr2=0x7f309c06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 shutdown_connections 2024-04-26T23:58:35.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.219+0000 7f30a2400640 1 -- 172.21.15.46:0/1761935036 wait complete. 2024-04-26T23:58:35.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:58:35.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.451+0000 7ff7cd400640 1 Processor -- start 2024-04-26T23:58:35.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.452+0000 7ff7cd400640 1 -- start start 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.453+0000 7ff7cd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7c81469c0 0x7ff7c815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.453+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c80b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.453+0000 7ff7cd400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c8146da0 con 0x7ff7c80cfa10 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.453+0000 7ff7cd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c80d0ac0 con 0x7ff7c81469c0 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.453+0000 7ff7cd400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c80cfeb0 con 0x7ff7c80cf560 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7c81469c0 0x7ff7c815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7c81469c0 0x7ff7c815d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60170/0 (socket says 172.21.15.46:60170) 2024-04-26T23:58:35.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 -- 172.21.15.46:0/3869188457 learned_addr learned my addr 172.21.15.46:0/3869188457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c5e00640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c80b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 -- 172.21.15.46:0/3869188457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 msgr2=0x7ff7c80b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c80b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 -- 172.21.15.46:0/3869188457 >> v1:172.21.15.142:6789/0 conn(0x7ff7c80cfa10 legacy=0x7ff7c806e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.454+0000 7ff7c6800640 1 -- 172.21.15.46:0/3869188457 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c80d0190 con 0x7ff7c81469c0 2024-04-26T23:58:35.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.456+0000 7ff7c6800640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c815d6d0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009870 tx=0x7ff7b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c2b1d72d4b214a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.456+0000 7ff7c5400640 1 -- 172.21.15.46:0/3869188457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8037070 con 0x7ff7c81469c0 2024-04-26T23:58:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.456+0000 7ff7c5400640 1 -- 172.21.15.46:0/3869188457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b8032440 con 0x7ff7c81469c0 2024-04-26T23:58:35.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.456+0000 7ff7c5400640 1 -- 172.21.15.46:0/3869188457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8032810 con 0x7ff7c81469c0 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 -- 172.21.15.46:0/3869188457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 msgr2=0x7ff7c815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c815d6d0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009870 tx=0x7ff7b802a210 comp rx=0 tx=0).stop 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 -- 172.21.15.46:0/3869188457 shutdown_connections 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c80b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 --2- 172.21.15.46:0/3869188457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c815d6d0 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 -- 172.21.15.46:0/3869188457 >> 172.21.15.46:0/3869188457 conn(0x7ff7c806c770 msgr2=0x7ff7c806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 -- 172.21.15.46:0/3869188457 shutdown_connections 2024-04-26T23:58:35.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.457+0000 7ff7cd400640 1 -- 172.21.15.46:0/3869188457 wait complete. 2024-04-26T23:58:35.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.459+0000 7ff7cd400640 1 Processor -- start 2024-04-26T23:58:35.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.459+0000 7ff7cd400640 1 -- start start 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c8143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7c80cfa10 0x7ff7c8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c80737e0 con 0x7ff7c81469c0 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c80d0750 con 0x7ff7c80cfa10 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7cd400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c8073960 con 0x7ff7c80cf560 2024-04-26T23:58:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:35.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60174/0 (socket says 172.21.15.46:60174) 2024-04-26T23:58:35.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7c7200640 1 -- 172.21.15.46:0/1422422686 learned_addr learned my addr 172.21.15.46:0/1422422686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7c7200640 1 -- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 msgr2=0x7ff7c8143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.460+0000 7ff7c7200640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c8143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7c5e00640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7c80cfa10 0x7ff7c8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7c7200640 1 -- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7c80cfa10 msgr2=0x7ff7c8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7c7200640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7c80cfa10 0x7ff7c8155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7c7200640 1 -- 172.21.15.46:0/1422422686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c8144190 con 0x7ff7c81469c0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7c7200640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c700 tx=0x7ff7c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0016020 con 0x7ff7c81469c0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c000aa50 con 0x7ff7c81469c0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000ce10 con 0x7ff7c81469c0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c8144470 con 0x7ff7c81469c0 2024-04-26T23:58:35.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.461+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c8144a30 con 0x7ff7c81469c0 2024-04-26T23:58:35.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.463+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c8069530 con 0x7ff7c81469c0 2024-04-26T23:58:35.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.464+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7c0019020 con 0x7ff7c81469c0 2024-04-26T23:58:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.465+0000 7ff7bea00640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 0x7ff7a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.465+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(214..214 src has 1..214) ==== 6360+0+0 (secure 0 0 0) 0x7ff7c008c1a0 con 0x7ff7c81469c0 2024-04-26T23:58:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.465+0000 7ff7c6800640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 0x7ff7a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.466+0000 7ff7c6800640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 0x7ff7a806edb0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8034000 tx=0x7ff7b802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:35.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.469+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7c0020020 con 0x7ff7c81469c0 2024-04-26T23:58:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:35.668+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff7c80af7e0 con 0x7ff7c81469c0 2024-04-26T23:58:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.228+0000 7ff7bea00640 1 -- 172.21.15.46:0/1422422686 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v215) ==== 181+0+0 (secure 0 0 0) 0x7ff7c005a2a0 con 0x7ff7c81469c0 2024-04-26T23:58:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:58:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.231+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 msgr2=0x7ff7a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.231+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 0x7ff7a806edb0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8034000 tx=0x7ff7b802c000 comp rx=0 tx=0).stop 2024-04-26T23:58:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.231+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 msgr2=0x7ff7c8145230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.231+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c700 tx=0x7ff7c000cbc0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 shutdown_connections 2024-04-26T23:58:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff7a806c8f0 0x7ff7a806edb0 secure :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8034000 tx=0x7ff7b802c000 comp rx=0 tx=0).stop 2024-04-26T23:58:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c81469c0 0x7ff7c8145230 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7c80cfa10 0x7ff7c8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 --2- 172.21.15.46:0/1422422686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7c80cf560 0x7ff7c8143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 >> 172.21.15.46:0/1422422686 conn(0x7ff7c806c770 msgr2=0x7ff7c815fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.232+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 shutdown_connections 2024-04-26T23:58:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.233+0000 7ff7cd400640 1 -- 172.21.15.46:0/1422422686 wait complete. 2024-04-26T23:58:36.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:58:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.452+0000 7f16bee00640 1 Processor -- start 2024-04-26T23:58:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.453+0000 7f16bee00640 1 -- start start 2024-04-26T23:58:36.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.454+0000 7f16bee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16b8167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.454+0000 7f16bee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b80b3ce0 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.454+0000 7f16bee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b80b3fc0 con 0x7f16b8146b20 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.454+0000 7f16bee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16b80b48f0 con 0x7f16b80d17f0 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.455+0000 7f16bda00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16b8146740 0x7f16b80cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36596/0 (socket says 172.21.15.46:36596) 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.455+0000 7f16bda00640 1 -- 172.21.15.46:0/4228318542 learned_addr learned my addr 172.21.15.46:0/4228318542 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.455+0000 7f16be400640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16b8167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.455+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288547179 0 0) 0x7f16b80b3ce0 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.455+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16a0003520 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 221503585 0 0) 0x7f16a0003520 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 msgr2=0x7f16b8167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16b8167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 >> v1:172.21.15.142:6789/0 conn(0x7f16b8146b20 legacy=0x7f16b80d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b80b42a0 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f16ac001670 con 0x7f16b8146740 2024-04-26T23:58:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 >> v1:172.21.15.46:6789/0 conn(0x7f16b8146740 legacy=0x7f16b80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a0003f60 0x7f16a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a0004d40 0x7f16a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80b48f0 con 0x7f16a0003f60 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16a000bb50 con 0x7f16a0004d40 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16a000bd50 con 0x7f16b80d17f0 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16bd000640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a0003f60 0x7f16a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.456+0000 7f16bda00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16be400640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a0004d40 0x7f16a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16bd000640 1 -- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 msgr2=0x7f16a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16bd000640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b80d17f0 0x7f16a00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16bd000640 1 -- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a0004d40 msgr2=0x7f16a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16bd000640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a0004d40 0x7f16a00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.457+0000 7f16bd000640 1 -- 172.21.15.46:0/4228318542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a0003520 con 0x7f16a0003f60 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16bd000640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a0003f60 0x7f16a0004800 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f16a4001750 tx=0x7f16a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e551e5deb2510b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4001370 con 0x7f16a0003f60 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a40052d0 con 0x7f16a0003f60 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16b7e00640 1 -- 172.21.15.46:0/4228318542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a402abe0 con 0x7f16a0003f60 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16bee00640 1 -- 172.21.15.46:0/4228318542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a0003f60 msgr2=0x7f16a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16bee00640 1 --2- 172.21.15.46:0/4228318542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a0003f60 0x7f16a0004800 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f16a4001750 tx=0x7f16a0005830 comp rx=0 tx=0).stop 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.458+0000 7f16be400640 1 -- 172.21.15.46:0/4228318542 reap_dead start 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.459+0000 7f16bee00640 1 -- 172.21.15.46:0/4228318542 shutdown_connections 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.459+0000 7f16bee00640 1 -- 172.21.15.46:0/4228318542 >> 172.21.15.46:0/4228318542 conn(0x7f16b806c770 msgr2=0x7f16b806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.459+0000 7f16bee00640 1 -- 172.21.15.46:0/4228318542 shutdown_connections 2024-04-26T23:58:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.459+0000 7f16bee00640 1 -- 172.21.15.46:0/4228318542 wait complete. 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.461+0000 7f16bee00640 1 Processor -- start 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.461+0000 7f16bee00640 1 -- start start 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16a0003f60 0x7f16b813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b815c790 0x7f16b8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80cdde0 con 0x7f16b813d4c0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16b815e560 con 0x7f16b815c790 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16b80d05a0 con 0x7f16a0003f60 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16be400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16be400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60186/0 (socket says 172.21.15.46:60186) 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16be400640 1 -- 172.21.15.46:0/3563859938 learned_addr learned my addr 172.21.15.46:0/3563859938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bda00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16a0003f60 0x7f16b813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.462+0000 7f16bd000640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b815c790 0x7f16b8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 -- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16a0003f60 msgr2=0x7f16b813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16a0003f60 0x7f16b813ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 -- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b815c790 msgr2=0x7f16b8154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b815c790 0x7f16b8154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 -- 172.21.15.46:0/3563859938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b813ddc0 con 0x7f16b813d4c0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16be400640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7f16b806feb0 tx=0x7f16b000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0003990 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b0003b50 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004020 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b813e0a0 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.463+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b813e660 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.465+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b80693b0 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.466+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16b0005d10 con 0x7f16b813d4c0 2024-04-26T23:58:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.467+0000 7f16b5600640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 0x7f169806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.467+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 5922+0+0 (secure 0 0 0) 0x7f16b008bfe0 con 0x7f16b813d4c0 2024-04-26T23:58:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.468+0000 7f16bda00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 0x7f169806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.469+0000 7f16bda00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 0x7f169806ede0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f16ac001200 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.472+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16b0012070 con 0x7f16b813d4c0 2024-04-26T23:58:36.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:36.679+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16b80b5e60 con 0x7f16b813d4c0 2024-04-26T23:58:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.242+0000 7f16b5600640 1 -- 172.21.15.46:0/3563859938 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v216) ==== 169+0+0 (secure 0 0 0) 0x7f16b005a2b0 con 0x7f16b813d4c0 2024-04-26T23:58:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-26T23:58:37.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.246+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 msgr2=0x7f169806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.246+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 0x7f169806ede0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f16ac001200 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2024-04-26T23:58:37.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.246+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 msgr2=0x7f16b815f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.246+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7f16b806feb0 tx=0x7f16b000f4a0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 shutdown_connections 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f169806c920 0x7f169806ede0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b813d4c0 0x7f16b815f730 unknown :-1 s=CLOSED pgs=3004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b815c790 0x7f16b8154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 --2- 172.21.15.46:0/3563859938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16a0003f60 0x7f16b813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.247+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 >> 172.21.15.46:0/3563859938 conn(0x7f16b806c770 msgr2=0x7f16b8145a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:37.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.248+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 shutdown_connections 2024-04-26T23:58:37.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.249+0000 7f16bee00640 1 -- 172.21.15.46:0/3563859938 wait complete. 2024-04-26T23:58:37.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-26T23:58:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.491+0000 7fa23b600640 1 Processor -- start 2024-04-26T23:58:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.491+0000 7fa23b600640 1 -- start start 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.492+0000 7fa23b600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa23415f300 0x7fa2340d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.492+0000 7fa23b600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa23406d460 0x7fa2340d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa23b600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa23415f6e0 con 0x7fa23406d8c0 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa23b600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa2340cfc50 con 0x7fa23415f300 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa23b600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa2340cff30 con 0x7fa23406d460 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa23415f300 0x7fa2340d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa23415f300 0x7fa2340d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60196/0 (socket says 172.21.15.46:60196) 2024-04-26T23:58:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 -- 172.21.15.46:0/1520484530 learned_addr learned my addr 172.21.15.46:0/1520484530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa233e00640 1 --2- 172.21.15.46:0/1520484530 >> v2:172.21.15.142:3300/0 conn(0x7fa23406d460 0x7fa2340d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 -- 172.21.15.46:0/1520484530 >> v2:172.21.15.142:3300/0 conn(0x7fa23406d460 msgr2=0x7fa2340d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 --2- 172.21.15.46:0/1520484530 >> v2:172.21.15.142:3300/0 conn(0x7fa23406d460 0x7fa2340d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.493+0000 7fa238c00640 1 -- 172.21.15.46:0/1520484530 >> v1:172.21.15.142:6789/0 conn(0x7fa23406d8c0 legacy=0x7fa234163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.494+0000 7fa238c00640 1 -- 172.21.15.46:0/1520484530 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340d0210 con 0x7fa23415f300 2024-04-26T23:58:37.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.495+0000 7fa238c00640 1 --2- 172.21.15.46:0/1520484530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2340d10a0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7fa2340c69d0 tx=0x7fa22402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59db64a9bdef79e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.495+0000 7fa233400640 1 -- 172.21.15.46:0/1520484530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224037070 con 0x7fa23415f300 2024-04-26T23:58:37.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.495+0000 7fa233400640 1 -- 172.21.15.46:0/1520484530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa224032440 con 0x7fa23415f300 2024-04-26T23:58:37.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.495+0000 7fa233400640 1 -- 172.21.15.46:0/1520484530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224032810 con 0x7fa23415f300 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 -- 172.21.15.46:0/1520484530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 msgr2=0x7fa2340d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 --2- 172.21.15.46:0/1520484530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2340d10a0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7fa2340c69d0 tx=0x7fa22402a230 comp rx=0 tx=0).stop 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 -- 172.21.15.46:0/1520484530 shutdown_connections 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 --2- 172.21.15.46:0/1520484530 >> v2:172.21.15.142:3300/0 conn(0x7fa23406d460 0x7fa2340d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 --2- 172.21.15.46:0/1520484530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2340d10a0 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 -- 172.21.15.46:0/1520484530 >> 172.21.15.46:0/1520484530 conn(0x7fa23409ba70 msgr2=0x7fa23409be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 -- 172.21.15.46:0/1520484530 shutdown_connections 2024-04-26T23:58:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.496+0000 7fa23b600640 1 -- 172.21.15.46:0/1520484530 wait complete. 2024-04-26T23:58:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.498+0000 7fa23b600640 1 Processor -- start 2024-04-26T23:58:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.498+0000 7fa23b600640 1 -- start start 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 0x7fa2340b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa23406d8c0 0x7fa2340b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa2340d15e0 con 0x7fa23415f300 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2340cf8e0 con 0x7fa23406d460 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa23b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2340d1760 con 0x7fa23406d8c0 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60200/0 (socket says 172.21.15.46:60200) 2024-04-26T23:58:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 -- 172.21.15.46:0/821983231 learned_addr learned my addr 172.21.15.46:0/821983231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa238c00640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 0x7fa2340b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 -- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa23406d8c0 msgr2=0x7fa2340b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa23406d8c0 0x7fa2340b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 -- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 msgr2=0x7fa2340b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 0x7fa2340b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa239600640 1 -- 172.21.15.46:0/821983231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341f6750 con 0x7fa23415f300 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.499+0000 7fa238c00640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 0x7fa2340b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa239600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c7b0 tx=0x7fa22c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c017070 con 0x7fa23415f300 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa22c003da0 con 0x7fa23415f300 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341f6a30 con 0x7fa23415f300 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00ce60 con 0x7fa23415f300 2024-04-26T23:58:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.500+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341f6ff0 con 0x7fa23415f300 2024-04-26T23:58:37.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.501+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa234069530 con 0x7fa23415f300 2024-04-26T23:58:37.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.502+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa22c01a070 con 0x7fa23415f300 2024-04-26T23:58:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.503+0000 7fa230c00640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 0x7fa21406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.503+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fa22c08afe0 con 0x7fa23415f300 2024-04-26T23:58:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.504+0000 7fa238c00640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 0x7fa21406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.505+0000 7fa238c00640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 0x7fa21406ede0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa224034000 tx=0x7fa22402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.508+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa22c059560 con 0x7fa23415f300 2024-04-26T23:58:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.717+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) -- 0x7fa2340b8e50 con 0x7fa23415f300 2024-04-26T23:58:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.723+0000 7fa230c00640 1 -- 172.21.15.46:0/821983231 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7fa22c058e60 con 0x7fa23415f300 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 msgr2=0x7fa21406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 0x7fa21406ede0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa224034000 tx=0x7fa22402e000 comp rx=0 tx=0).stop 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 msgr2=0x7fa2341f8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c7b0 tx=0x7fa22c00cc70 comp rx=0 tx=0).stop 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 shutdown_connections 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fa21406c920 0x7fa21406ede0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa23415f300 0x7fa2341f8930 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa23406d8c0 0x7fa2340b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 --2- 172.21.15.46:0/821983231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa23406d460 0x7fa2340b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 >> 172.21.15.46:0/821983231 conn(0x7fa23409ba70 msgr2=0x7fa23406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 shutdown_connections 2024-04-26T23:58:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.726+0000 7fa23b600640 1 -- 172.21.15.46:0/821983231 wait complete. 2024-04-26T23:58:37.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:58:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.956+0000 7f6c1fe00640 1 Processor -- start 2024-04-26T23:58:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.956+0000 7f6c1fe00640 1 -- start start 2024-04-26T23:58:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c20146740 0x7f6c2015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c20146b20 con 0x7f6c200d0330 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c200d07c0 con 0x7f6c200ce670 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6c20161a00 con 0x7f6c20146740 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6c200d0330 0x7f6c201612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36606/0 (socket says 172.21.15.46:36606) 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.958+0000 7f6c1e000640 1 -- 172.21.15.46:0/3082553938 learned_addr learned my addr 172.21.15.46:0/3082553938 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782616560 0 0) 0x7f6c20146b20 con 0x7f6c200d0330 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c04003520 con 0x7f6c200d0330 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1ea00640 1 --2- 172.21.15.46:0/3082553938 >> v2:172.21.15.142:3300/0 conn(0x7f6c20146740 0x7f6c2015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3619186066 0 0) 0x7f6c04003520 con 0x7f6c200d0330 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 >> v2:172.21.15.142:3300/0 conn(0x7f6c20146740 msgr2=0x7f6c2015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 --2- 172.21.15.46:0/3082553938 >> v2:172.21.15.142:3300/0 conn(0x7f6c20146740 0x7f6c2015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 >> v1:172.21.15.142:6789/0 conn(0x7f6c200ce670 legacy=0x7f6c200cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c200ccf30 con 0x7f6c200d0330 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f6c0c001670 con 0x7f6c200d0330 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.959+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 >> v1:172.21.15.46:6789/0 conn(0x7f6c200d0330 legacy=0x7f6c201612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c040039b0 0x7f6c04003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 0x7f6c0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20161a00 con 0x7f6c040051c0 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400c0f0 con 0x7f6c040039b0 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0400c2f0 con 0x7f6c04004800 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1f400640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 0x7f6c0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1ea00640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c040039b0 0x7f6c04003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1f400640 1 -- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c04004800 msgr2=0x7f6c04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.960+0000 7f6c1e000640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.961+0000 7f6c1f400640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.961+0000 7f6c1f400640 1 -- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c040039b0 msgr2=0x7f6c04003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.961+0000 7f6c1f400640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c040039b0 0x7f6c04003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.961+0000 7f6c1f400640 1 -- 172.21.15.46:0/3082553938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c04003520 con 0x7f6c040051c0 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.961+0000 7f6c1f400640 1 -- 172.21.15.46:0/3082553938 reap_dead start 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.962+0000 7f6c1f400640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 0x7f6c0400b9c0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f6c18004480 tx=0x7f6c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e41f4ee7f9e19f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.962+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004080 con 0x7f6c040051c0 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.962+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1802ad80 con 0x7f6c040051c0 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.962+0000 7f6c1d600640 1 -- 172.21.15.46:0/3082553938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18006d90 con 0x7f6c040051c0 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 -- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 msgr2=0x7f6c0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 0x7f6c0400b9c0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f6c18004480 tx=0x7f6c04005830 comp rx=0 tx=0).stop 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 -- 172.21.15.46:0/3082553938 shutdown_connections 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/3082553938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c040051c0 0x7f6c0400b9c0 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 -- 172.21.15.46:0/3082553938 >> 172.21.15.46:0/3082553938 conn(0x7f6c200b26d0 msgr2=0x7f6c200b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 -- 172.21.15.46:0/3082553938 shutdown_connections 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.963+0000 7f6c1fe00640 1 -- 172.21.15.46:0/3082553938 wait complete. 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.965+0000 7f6c1fe00640 1 Processor -- start 2024-04-26T23:58:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.965+0000 7f6c1fe00640 1 -- start start 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c040051c0 0x7f6c200b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c200aaa10 0x7f6c200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c200ccb70 con 0x7f6c2015f540 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20160dd0 con 0x7f6c200aaa10 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6c200cccf0 con 0x7f6c040051c0 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60218/0 (socket says 172.21.15.46:60218) 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 -- 172.21.15.46:0/1850751984 learned_addr learned my addr 172.21.15.46:0/1850751984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1ea00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c040051c0 0x7f6c200b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1f400640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c200aaa10 0x7f6c200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 -- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c040051c0 msgr2=0x7f6c200b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c040051c0 0x7f6c200b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 -- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c200aaa10 msgr2=0x7f6c200b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c200aaa10 0x7f6c200b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.966+0000 7f6c1e000640 1 -- 172.21.15.46:0/1850751984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c200afcb0 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c1e000640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d840 tx=0x7f6c0c00dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c019070 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0074e0 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004a20 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c200aff90 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.967+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c200b5620 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.969+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c200693b0 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.970+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c0c01c070 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.971+0000 7f6c16a00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.971+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f6c0c08b510 con 0x7f6c2015f540 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.972+0000 7f6c1ea00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.973+0000 7f6c1ea00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:37.976+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c0c0599c0 con 0x7f6c2015f540 2024-04-26T23:58:38.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.183+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6c200b67c0 con 0x7f6c2015f540 2024-04-26T23:58:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.190+0000 7f6c16a00640 1 -- 172.21.15.46:0/1850751984 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7f6c0c0592c0 con 0x7f6c2015f540 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 msgr2=0x7f6c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10008040 comp rx=0 tx=0).stop 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 msgr2=0x7f6c200aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d840 tx=0x7f6c0c00dd00 comp rx=0 tx=0).stop 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 shutdown_connections 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c200aaa10 0x7f6c200b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c2015f540 0x7f6c200aa4d0 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 --2- 172.21.15.46:0/1850751984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c040051c0 0x7f6c200b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 >> 172.21.15.46:0/1850751984 conn(0x7f6c200b26d0 msgr2=0x7f6c200b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 shutdown_connections 2024-04-26T23:58:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.193+0000 7f6c1fe00640 1 -- 172.21.15.46:0/1850751984 wait complete. 2024-04-26T23:58:38.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-26T23:58:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.424+0000 7f5c70c00640 1 Processor -- start 2024-04-26T23:58:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.424+0000 7f5c70c00640 1 -- start start 2024-04-26T23:58:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c70c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c6c160bd0 0x7f5c6c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c70c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c6c0732c0 0x7f5c6c073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c70c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c161940 con 0x7f5c6c06e510 2024-04-26T23:58:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c70c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c071f80 con 0x7f5c6c0732c0 2024-04-26T23:58:38.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c70c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c072260 con 0x7f5c6c160bd0 2024-04-26T23:58:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c69e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5c6c06e510 0x7f5c6c06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36620/0 (socket says 172.21.15.46:36620) 2024-04-26T23:58:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c69e00640 1 -- 172.21.15.46:0/2499843263 learned_addr learned my addr 172.21.15.46:0/2499843263 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.426+0000 7f5c6a800640 1 --2- 172.21.15.46:0/2499843263 >> v2:172.21.15.46:3300/0 conn(0x7f5c6c0732c0 0x7f5c6c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c69400640 1 --2- 172.21.15.46:0/2499843263 >> v2:172.21.15.142:3300/0 conn(0x7f5c6c160bd0 0x7f5c6c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c68a00640 1 -- 172.21.15.46:0/2499843263 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621052641 0 0) 0x7f5c6c161940 con 0x7f5c6c06e510 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c68a00640 1 -- 172.21.15.46:0/2499843263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c50003520 con 0x7f5c6c06e510 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c6a800640 1 -- 172.21.15.46:0/2499843263 >> v2:172.21.15.142:3300/0 conn(0x7f5c6c160bd0 msgr2=0x7f5c6c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c6a800640 1 --2- 172.21.15.46:0/2499843263 >> v2:172.21.15.142:3300/0 conn(0x7f5c6c160bd0 0x7f5c6c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c6a800640 1 -- 172.21.15.46:0/2499843263 >> v1:172.21.15.46:6789/0 conn(0x7f5c6c06e510 legacy=0x7f5c6c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.427+0000 7f5c6a800640 1 -- 172.21.15.46:0/2499843263 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c072540 con 0x7f5c6c0732c0 2024-04-26T23:58:38.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.428+0000 7f5c6a800640 1 --2- 172.21.15.46:0/2499843263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c073730 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a7d2e62daba74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:38.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.428+0000 7f5c68a00640 1 -- 172.21.15.46:0/2499843263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64007740 con 0x7f5c6c0732c0 2024-04-26T23:58:38.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c68a00640 1 -- 172.21.15.46:0/2499843263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c640078e0 con 0x7f5c6c0732c0 2024-04-26T23:58:38.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c68a00640 1 -- 172.21.15.46:0/2499843263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64007c90 con 0x7f5c6c0732c0 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 -- 172.21.15.46:0/2499843263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c0732c0 msgr2=0x7f5c6c073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 --2- 172.21.15.46:0/2499843263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c073730 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 -- 172.21.15.46:0/2499843263 shutdown_connections 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 --2- 172.21.15.46:0/2499843263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c073730 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 --2- 172.21.15.46:0/2499843263 >> v2:172.21.15.142:3300/0 conn(0x7f5c6c160bd0 0x7f5c6c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 -- 172.21.15.46:0/2499843263 >> 172.21.15.46:0/2499843263 conn(0x7f5c6c06c530 msgr2=0x7f5c6c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 -- 172.21.15.46:0/2499843263 shutdown_connections 2024-04-26T23:58:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.429+0000 7f5c70c00640 1 -- 172.21.15.46:0/2499843263 wait complete. 2024-04-26T23:58:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.431+0000 7f5c70c00640 1 Processor -- start 2024-04-26T23:58:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.431+0000 7f5c70c00640 1 -- start start 2024-04-26T23:58:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 0x7f5c6c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c13f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c161720 con 0x7f5c6c160bd0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1615a0 con 0x7f5c6c0732c0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.432+0000 7f5c70c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c072d40 con 0x7f5c6c06e510 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60226/0 (socket says 172.21.15.46:60226) 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 -- 172.21.15.46:0/4100367093 learned_addr learned my addr 172.21.15.46:0/4100367093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c69400640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c13f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 -- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 msgr2=0x7f5c6c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c69e00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 0x7f5c6c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 0x7f5c6c13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 -- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 msgr2=0x7f5c6c13f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c13f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 -- 172.21.15.46:0/4100367093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0b2590 con 0x7f5c6c160bd0 2024-04-26T23:58:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c69e00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 0x7f5c6c13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c69400640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.433+0000 7f5c6a800640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009eb0 tx=0x7f5c64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.434+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64040070 con 0x7f5c6c160bd0 2024-04-26T23:58:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.434+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c64031c60 con 0x7f5c6c160bd0 2024-04-26T23:58:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.434+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c0b2870 con 0x7f5c6c160bd0 2024-04-26T23:58:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.434+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64009450 con 0x7f5c6c160bd0 2024-04-26T23:58:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.434+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c0b2e30 con 0x7f5c6c160bd0 2024-04-26T23:58:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.435+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c0b3320 con 0x7f5c6c160bd0 2024-04-26T23:58:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.436+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c64039040 con 0x7f5c6c160bd0 2024-04-26T23:58:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.437+0000 7f5c62000640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.437+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f5c640acf40 con 0x7f5c6c160bd0 2024-04-26T23:58:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.438+0000 7f5c69e00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.439+0000 7f5c69e00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005880 tx=0x7f5c5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.442+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c6407b480 con 0x7f5c6c160bd0 2024-04-26T23:58:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:38.727+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) -- 0x7f5c6c143da0 con 0x7f5c6c160bd0 2024-04-26T23:58:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.258+0000 7f5c62000640 1 -- 172.21.15.46:0/4100367093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v980) ==== 133+0+0 (secure 0 0 0) 0x7f5c6407ad80 con 0x7f5c6c160bd0 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 msgr2=0x7f5c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005880 tx=0x7f5c5c009040 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 msgr2=0x7f5c6c13f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009eb0 tx=0x7f5c64004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 shutdown_connections 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c160bd0 0x7f5c6c13f790 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c6c0732c0 0x7f5c6c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 --2- 172.21.15.46:0/4100367093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c6c06e510 0x7f5c6c13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.261+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 >> 172.21.15.46:0/4100367093 conn(0x7f5c6c06c530 msgr2=0x7f5c6c0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.262+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 shutdown_connections 2024-04-26T23:58:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:39.262+0000 7f5c70c00640 1 -- 172.21.15.46:0/4100367093 wait complete. 2024-04-26T23:58:39.287 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-26T23:58:39.287 INFO:tasks.cephfs_test_runner: 2024-04-26T23:58:39.287 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T23:58:39.287 INFO:tasks.cephfs_test_runner:Ran 14 tests in 1223.366s 2024-04-26T23:58:39.287 INFO:tasks.cephfs_test_runner: 2024-04-26T23:58:39.288 INFO:tasks.cephfs_test_runner:OK 2024-04-26T23:58:39.288 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T23:58:39.299 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T23:58:39.306 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T23:58:39.306 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:58:39.307 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:39.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:58:39.367 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-26T23:58:39.367 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:39.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053507,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:58:39.421 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:58:39.421 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:39.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:58:39.478 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-26T23:58:39.478 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:58:39.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:58:39.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.532 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:58:39.592 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:58:39.592 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:58:39.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:58:39.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:58:39.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-26T23:58:39.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.875 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-26T23:58:39.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:39.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-26T23:58:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-04-26T23:58:40.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:58:40.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.057 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:58:40.114 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T23:58:40.114 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T23:58:40.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.115 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:58:40.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T23:58:40.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-04-26T23:58:40.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.358 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-04-26T23:58:40.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:58:40.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-04-26T23:58:40.516 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-04-26T23:58:40.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:58:40.519 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T23:58:40.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T23:58:40.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:58:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.749+0000 7faece600640 1 Processor -- start 2024-04-26T23:58:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.749+0000 7faece600640 1 -- start start 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faec815fcb0 0x7faec8073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faec80c6d50 0x7faec80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec815f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7faec815f9c0 con 0x7faec815fcb0 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faec80cba60 con 0x7faec80c6d50 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.751+0000 7faece600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faec80cbcb0 con 0x7faec815f030 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faec815fcb0 0x7faec8073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faec815fcb0 0x7faec8073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60236/0 (socket says 172.21.15.46:60236) 2024-04-26T23:58:40.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 -- 172.21.15.46:0/3298522606 learned_addr learned my addr 172.21.15.46:0/3298522606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 -- 172.21.15.46:0/3298522606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 msgr2=0x7faec815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 --2- 172.21.15.46:0/3298522606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 -- 172.21.15.46:0/3298522606 >> v2:172.21.15.142:3300/0 conn(0x7faec80c6d50 msgr2=0x7faec80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 --2- 172.21.15.46:0/3298522606 >> v2:172.21.15.142:3300/0 conn(0x7faec80c6d50 0x7faec80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.752+0000 7faecd200640 1 -- 172.21.15.46:0/3298522606 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec80cbf90 con 0x7faec815fcb0 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.753+0000 7faecd200640 1 --2- 172.21.15.46:0/3298522606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec815fcb0 0x7faec8073be0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7faec80ce710 tx=0x7faeb802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88ed9d012c3c456 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.754+0000 7faec7400640 1 -- 172.21.15.46:0/3298522606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8037070 con 0x7faec815fcb0 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.754+0000 7faec7400640 1 -- 172.21.15.46:0/3298522606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb8032440 con 0x7faec815fcb0 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.754+0000 7faec7400640 1 -- 172.21.15.46:0/3298522606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8032810 con 0x7faec815fcb0 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.754+0000 7faece600640 1 -- 172.21.15.46:0/3298522606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec815fcb0 msgr2=0x7faec8073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.754+0000 7faece600640 1 --2- 172.21.15.46:0/3298522606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec815fcb0 0x7faec8073be0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7faec80ce710 tx=0x7faeb802a140 comp rx=0 tx=0).stop 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 -- 172.21.15.46:0/3298522606 shutdown_connections 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 --2- 172.21.15.46:0/3298522606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 --2- 172.21.15.46:0/3298522606 >> v2:172.21.15.142:3300/0 conn(0x7faec80c6d50 0x7faec80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 --2- 172.21.15.46:0/3298522606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec815fcb0 0x7faec8073be0 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 -- 172.21.15.46:0/3298522606 >> 172.21.15.46:0/3298522606 conn(0x7faec80b26d0 msgr2=0x7faec80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 -- 172.21.15.46:0/3298522606 shutdown_connections 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.755+0000 7faece600640 1 -- 172.21.15.46:0/3298522606 wait complete. 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.757+0000 7faece600640 1 Processor -- start 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.757+0000 7faece600640 1 -- start start 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec80b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec815fcb0 0x7faec815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faec80cda60 con 0x7faec80c6d50 2024-04-26T23:58:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faec80c71c0 con 0x7faec815fcb0 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.758+0000 7faece600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faec80cdbe0 con 0x7faec815f030 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60250/0 (socket says 172.21.15.46:60250) 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 -- 172.21.15.46:0/397270283 learned_addr learned my addr 172.21.15.46:0/397270283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 -- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 msgr2=0x7faec80b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec80b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 -- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec815fcb0 msgr2=0x7faec815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec815fcb0 0x7faec815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 -- 172.21.15.46:0/397270283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec814eb80 con 0x7faec80c6d50 2024-04-26T23:58:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.759+0000 7faecd200640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.760+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8041070 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.760+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb803c870 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.760+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8037040 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.760+0000 7faece600640 1 -- 172.21.15.46:0/397270283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec80b6780 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.760+0000 7faece600640 1 -- 172.21.15.46:0/397270283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec815ad10 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.761+0000 7faece600640 1 -- 172.21.15.46:0/397270283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec814ed50 con 0x7faec80c6d50 2024-04-26T23:58:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.762+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeb8044070 con 0x7faec80c6d50 2024-04-26T23:58:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.763+0000 7faec4c00640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 0x7faea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.763+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7faeb80ac6b0 con 0x7faec80c6d50 2024-04-26T23:58:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.764+0000 7faec7e00640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 0x7faea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.765+0000 7faec7e00640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 0x7faea806edb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faec80cac30 tx=0x7faeb4006bf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.769+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faeb807ad10 con 0x7faec80c6d50 2024-04-26T23:58:40.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.974+0000 7faece600640 1 -- 172.21.15.46:0/397270283 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faec80693b0 con 0x7faea806c8f0 2024-04-26T23:58:40.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.977+0000 7faec4c00640 1 -- 172.21.15.46:0/397270283 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13528 (secure 0 0 0) 0x7faec80693b0 con 0x7faea806c8f0 2024-04-26T23:58:40.978 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:40.979 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 -- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 msgr2=0x7faea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 0x7faea806edb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faec80cac30 tx=0x7faeb4006bf0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 -- 172.21.15.46:0/397270283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 msgr2=0x7faec814f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb80046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 -- 172.21.15.46:0/397270283 shutdown_connections 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7faea806c8f0 0x7faea806edb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faec815fcb0 0x7faec815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faec815f030 0x7faec80b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 --2- 172.21.15.46:0/397270283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec80c6d50 0x7faec814f060 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.979+0000 7faece600640 1 -- 172.21.15.46:0/397270283 >> 172.21.15.46:0/397270283 conn(0x7faec80b26d0 msgr2=0x7faec815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.980+0000 7faece600640 1 -- 172.21.15.46:0/397270283 shutdown_connections 2024-04-26T23:58:40.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:40.980+0000 7faece600640 1 -- 172.21.15.46:0/397270283 wait complete. 2024-04-26T23:58:41.000 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":859,"stamp":"2024-04-26T23:58:40.377392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461616,"kb_used_data":46976,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754513104,"statfs":{"total":773094113280,"available":772621418496,"internally_reserved":0,"allocated":48103424,"data_stored":36672456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162287"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":532,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-04-26T23:58:35.512468+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:58:35.512468+0000","last_peered":"2024-04-26T23:58:35.512468+0000","last_clean":"2024-04-26T23:58:35.512468+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:58:35.512468+0000","last_undegraded":"2024-04-26T23:58:35.512468+0000","last_fullsized":"2024-04-26T23:58:35.512468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575109,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56612,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315228,"statfs":{"total":96636764160,"available":96578793472,"internally_reserved":0,"allocated":3297280,"data_stored":1867539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60128,"kb_used_data":6736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311712,"statfs":{"total":96636764160,"available":96575193088,"internally_reserved":0,"allocated":6897664,"data_stored":5471447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575110,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58080,"kb_used_data":4692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313760,"statfs":{"total":96636764160,"available":96577290240,"internally_reserved":0,"allocated":4804608,"data_stored":3380829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68388,"kb_used_data":10840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303452,"statfs":{"total":96636764160,"available":96566734848,"internally_reserved":0,"allocated":11100160,"data_stored":9672355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49820,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322020,"statfs":{"total":96636764160,"available":96585748480,"internally_reserved":0,"allocated":4866048,"data_stored":3422201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49760,"kb_used_data":4688,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322080,"statfs":{"total":96636764160,"available":96585809920,"internally_reserved":0,"allocated":4800512,"data_stored":3374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49752,"kb_used_data":4680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322088,"statfs":{"total":96636764160,"available":96585818112,"internally_reserved":0,"allocated":4792320,"data_stored":3374183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69076,"kb_used_data":7368,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302764,"statfs":{"total":96636764160,"available":96566030336,"internally_reserved":0,"allocated":7544832,"data_stored":6109067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:58:41.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:58:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.216+0000 7f32a3000640 1 Processor -- start 2024-04-26T23:58:41.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.216+0000 7f32a3000640 1 -- start start 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a3000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f329c1466b0 0x7f329c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329c073ad0 con 0x7f329c0720e0 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a3000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f329c146a90 con 0x7f329c1466b0 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f329c073d70 con 0x7f329c160910 2024-04-26T23:58:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f329c1466b0 0x7f329c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f329c1466b0 0x7f329c06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60252/0 (socket says 172.21.15.46:60252) 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.218+0000 7f32a1c00640 1 -- 172.21.15.46:0/1505025474 learned_addr learned my addr 172.21.15.46:0/1505025474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a2600640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a1c00640 1 -- 172.21.15.46:0/1505025474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 msgr2=0x7f329c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a1c00640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a1c00640 1 -- 172.21.15.46:0/1505025474 >> v1:172.21.15.46:6789/0 conn(0x7f329c0720e0 legacy=0x7f329c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a1c00640 1 -- 172.21.15.46:0/1505025474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c074050 con 0x7f329c1466b0 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.219+0000 7f32a2600640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.220+0000 7f32a1c00640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c1466b0 0x7f329c06de80 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7f328c009e80 tx=0x7f328c02a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8488aaedbefb44a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.220+0000 7f329be00640 1 -- 172.21.15.46:0/1505025474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c036070 con 0x7f329c1466b0 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.220+0000 7f329be00640 1 -- 172.21.15.46:0/1505025474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328c031600 con 0x7f329c1466b0 2024-04-26T23:58:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.220+0000 7f329be00640 1 -- 172.21.15.46:0/1505025474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0319d0 con 0x7f329c1466b0 2024-04-26T23:58:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 -- 172.21.15.46:0/1505025474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c1466b0 msgr2=0x7f329c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c1466b0 0x7f329c06de80 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7f328c009e80 tx=0x7f328c02a740 comp rx=0 tx=0).stop 2024-04-26T23:58:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 -- 172.21.15.46:0/1505025474 shutdown_connections 2024-04-26T23:58:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 --2- 172.21.15.46:0/1505025474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c1466b0 0x7f329c06de80 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 -- 172.21.15.46:0/1505025474 >> 172.21.15.46:0/1505025474 conn(0x7f329c0b26d0 msgr2=0x7f329c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 -- 172.21.15.46:0/1505025474 shutdown_connections 2024-04-26T23:58:41.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.221+0000 7f32a3000640 1 -- 172.21.15.46:0/1505025474 wait complete. 2024-04-26T23:58:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.223+0000 7f32a3000640 1 Processor -- start 2024-04-26T23:58:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.223+0000 7f32a3000640 1 -- start start 2024-04-26T23:58:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f329c1466b0 0x7f329c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f329c06e3c0 con 0x7f329c0720e0 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0cb470 con 0x7f329c1466b0 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f329c0cb990 con 0x7f329c160910 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60264/0 (socket says 172.21.15.46:60264) 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.224+0000 7f32a1c00640 1 -- 172.21.15.46:0/1368912137 learned_addr learned my addr 172.21.15.46:0/1368912137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 -- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 msgr2=0x7f329c0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1200640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f329c1466b0 0x7f329c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 -- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f329c1466b0 msgr2=0x7f329c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f329c1466b0 0x7f329c0cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 -- 172.21.15.46:0/1368912137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c13ce10 con 0x7f329c0720e0 2024-04-26T23:58:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f32a1c00640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f328c034040 tx=0x7f328c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.225+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c036050 con 0x7f329c0720e0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.226+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328c042070 con 0x7f329c0720e0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.226+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c009510 con 0x7f329c0720e0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.226+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c0b4b20 con 0x7f329c0720e0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.226+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c0b50e0 con 0x7f329c0720e0 2024-04-26T23:58:41.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.227+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c0693b0 con 0x7f329c0720e0 2024-04-26T23:58:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.228+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f328c007c30 con 0x7f329c0720e0 2024-04-26T23:58:41.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.229+0000 7f3299600640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 0x7f327c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.229+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f328c0ac900 con 0x7f329c0720e0 2024-04-26T23:58:41.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.230+0000 7f32a1200640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 0x7f327c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.231+0000 7f32a1200640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 0x7f327c06edb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f329c1f8450 tx=0x7f3288009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.233+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f328c07ae50 con 0x7f329c0720e0 2024-04-26T23:58:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.439+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f329c144eb0 con 0x7f327c06c8f0 2024-04-26T23:58:41.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.441+0000 7f3299600640 1 -- 172.21.15.46:0/1368912137 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13528 (secure 0 0 0) 0x7f329c144eb0 con 0x7f327c06c8f0 2024-04-26T23:58:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:41.443 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:58:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 msgr2=0x7f327c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 0x7f327c06edb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f329c1f8450 tx=0x7f3288009040 comp rx=0 tx=0).stop 2024-04-26T23:58:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 msgr2=0x7f329c160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f328c034040 tx=0x7f328c004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 shutdown_connections 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f327c06c8f0 0x7f327c06edb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c160910 0x7f329c0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f329c1466b0 0x7f329c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 --2- 172.21.15.46:0/1368912137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f329c0720e0 0x7f329c160010 unknown :-1 s=CLOSED pgs=3017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 >> 172.21.15.46:0/1368912137 conn(0x7f329c0b26d0 msgr2=0x7f329c0ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 shutdown_connections 2024-04-26T23:58:41.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.444+0000 7f32a3000640 1 -- 172.21.15.46:0/1368912137 wait complete. 2024-04-26T23:58:41.465 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":859,"stamp":"2024-04-26T23:58:40.377392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461616,"kb_used_data":46976,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754513104,"statfs":{"total":773094113280,"available":772621418496,"internally_reserved":0,"allocated":48103424,"data_stored":36672456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162287"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":532,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-04-26T23:58:35.512468+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:58:35.512468+0000","last_peered":"2024-04-26T23:58:35.512468+0000","last_clean":"2024-04-26T23:58:35.512468+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:58:35.512468+0000","last_undegraded":"2024-04-26T23:58:35.512468+0000","last_fullsized":"2024-04-26T23:58:35.512468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575109,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56612,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315228,"statfs":{"total":96636764160,"available":96578793472,"internally_reserved":0,"allocated":3297280,"data_stored":1867539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60128,"kb_used_data":6736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311712,"statfs":{"total":96636764160,"available":96575193088,"internally_reserved":0,"allocated":6897664,"data_stored":5471447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575110,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58080,"kb_used_data":4692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313760,"statfs":{"total":96636764160,"available":96577290240,"internally_reserved":0,"allocated":4804608,"data_stored":3380829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68388,"kb_used_data":10840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303452,"statfs":{"total":96636764160,"available":96566734848,"internally_reserved":0,"allocated":11100160,"data_stored":9672355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49820,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322020,"statfs":{"total":96636764160,"available":96585748480,"internally_reserved":0,"allocated":4866048,"data_stored":3422201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49760,"kb_used_data":4688,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322080,"statfs":{"total":96636764160,"available":96585809920,"internally_reserved":0,"allocated":4800512,"data_stored":3374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49752,"kb_used_data":4680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322088,"statfs":{"total":96636764160,"available":96585818112,"internally_reserved":0,"allocated":4792320,"data_stored":3374183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69076,"kb_used_data":7368,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302764,"statfs":{"total":96636764160,"available":96566030336,"internally_reserved":0,"allocated":7544832,"data_stored":6109067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:58:41.466 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T23:58:41.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:58:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.677+0000 7ff6e2800640 1 Processor -- start 2024-04-26T23:58:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.677+0000 7ff6e2800640 1 -- start start 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.678+0000 7ff6e2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6dc1466b0 0x7ff6dc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.678+0000 7ff6e2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6dc073ad0 con 0x7ff6dc160910 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e2800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6dc146a90 con 0x7ff6dc0720e0 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff6dc073d70 con 0x7ff6dc1466b0 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6dc160910 0x7ff6dc0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36636/0 (socket says 172.21.15.46:36636) 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6dc1466b0 0x7ff6dc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1e00640 1 -- 172.21.15.46:0/3138036843 learned_addr learned my addr 172.21.15.46:0/3138036843 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1400640 1 -- 172.21.15.46:0/3138036843 >> v1:172.21.15.142:6789/0 conn(0x7ff6dc0720e0 legacy=0x7ff6dc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1400640 1 -- 172.21.15.46:0/3138036843 >> v1:172.21.15.46:6789/0 conn(0x7ff6dc160910 legacy=0x7ff6dc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.679+0000 7ff6e1400640 1 -- 172.21.15.46:0/3138036843 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc074050 con 0x7ff6dc1466b0 2024-04-26T23:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e1400640 1 --2- 172.21.15.46:0/3138036843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1466b0 0x7ff6dc06de80 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00097d0 tx=0x7ff6d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2a91342a9df36d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6dbe00640 1 -- 172.21.15.46:0/3138036843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0037070 con 0x7ff6dc1466b0 2024-04-26T23:58:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6dbe00640 1 -- 172.21.15.46:0/3138036843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d0032440 con 0x7ff6dc1466b0 2024-04-26T23:58:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6dbe00640 1 -- 172.21.15.46:0/3138036843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0032810 con 0x7ff6dc1466b0 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e2800640 1 -- 172.21.15.46:0/3138036843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1466b0 msgr2=0x7ff6dc06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e2800640 1 --2- 172.21.15.46:0/3138036843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1466b0 0x7ff6dc06de80 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00097d0 tx=0x7ff6d002a210 comp rx=0 tx=0).stop 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e2800640 1 -- 172.21.15.46:0/3138036843 shutdown_connections 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e2800640 1 --2- 172.21.15.46:0/3138036843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1466b0 0x7ff6dc06de80 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.681+0000 7ff6e2800640 1 -- 172.21.15.46:0/3138036843 >> 172.21.15.46:0/3138036843 conn(0x7ff6dc0b26d0 msgr2=0x7ff6dc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.682+0000 7ff6e2800640 1 -- 172.21.15.46:0/3138036843 shutdown_connections 2024-04-26T23:58:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.682+0000 7ff6e2800640 1 -- 172.21.15.46:0/3138036843 wait complete. 2024-04-26T23:58:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.683+0000 7ff6e2800640 1 Processor -- start 2024-04-26T23:58:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.683+0000 7ff6e2800640 1 -- start start 2024-04-26T23:58:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6dc0720e0 0x7ff6dc0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6dc1466b0 0x7ff6dc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc06e3c0 con 0x7ff6dc160910 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc0cb470 con 0x7ff6dc0720e0 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.684+0000 7ff6e2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc0cc0a0 con 0x7ff6dc1466b0 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60288/0 (socket says 172.21.15.46:60288) 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 -- 172.21.15.46:0/2655203308 learned_addr learned my addr 172.21.15.46:0/2655203308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1400640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6dc0720e0 0x7ff6dc0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 -- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6dc1466b0 msgr2=0x7ff6dc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6dc1466b0 0x7ff6dc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 -- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6dc0720e0 msgr2=0x7ff6dc0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6dc0720e0 0x7ff6dc0b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 -- 172.21.15.46:0/2655203308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc1606d0 con 0x7ff6dc160910 2024-04-26T23:58:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.685+0000 7ff6e1e00640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400ca80 tx=0x7ff6d400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.686+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4017070 con 0x7ff6dc160910 2024-04-26T23:58:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.686+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d400ac30 con 0x7ff6dc160910 2024-04-26T23:58:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.686+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4012600 con 0x7ff6dc160910 2024-04-26T23:58:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.686+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc0b41f0 con 0x7ff6dc160910 2024-04-26T23:58:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.686+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc0b47b0 con 0x7ff6dc160910 2024-04-26T23:58:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.687+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc0693b0 con 0x7ff6dc160910 2024-04-26T23:58:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.688+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6d401a070 con 0x7ff6dc160910 2024-04-26T23:58:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.689+0000 7ff6d9600640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 0x7ff6bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.689+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7ff6d408b5e0 con 0x7ff6dc160910 2024-04-26T23:58:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.690+0000 7ff6e1400640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 0x7ff6bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.691+0000 7ff6e1400640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 0x7ff6bc06edb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0034000 tx=0x7ff6d002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.694+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6d400ae20 con 0x7ff6dc160910 2024-04-26T23:58:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.892+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6dc161200 con 0x7ff6bc06c8f0 2024-04-26T23:58:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.894+0000 7ff6d9600640 1 -- 172.21.15.46:0/2655203308 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13528 (secure 0 0 0) 0x7ff6dc161200 con 0x7ff6bc06c8f0 2024-04-26T23:58:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:58:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 msgr2=0x7ff6bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 0x7ff6bc06edb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0034000 tx=0x7ff6d002e000 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 msgr2=0x7ff6dc1f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400ca80 tx=0x7ff6d400cf40 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 shutdown_connections 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff6bc06c8f0 0x7ff6bc06edb0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc160910 0x7ff6dc1f4dc0 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff6dc1466b0 0x7ff6dc0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 --2- 172.21.15.46:0/2655203308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6dc0720e0 0x7ff6dc0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 >> 172.21.15.46:0/2655203308 conn(0x7ff6dc0b26d0 msgr2=0x7ff6dc0c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.897+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 shutdown_connections 2024-04-26T23:58:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:41.898+0000 7ff6e2800640 1 -- 172.21.15.46:0/2655203308 wait complete. 2024-04-26T23:58:41.917 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":859,"stamp":"2024-04-26T23:58:40.377392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461616,"kb_used_data":46976,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754513104,"statfs":{"total":773094113280,"available":772621418496,"internally_reserved":0,"allocated":48103424,"data_stored":36672456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162287"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":532,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-04-26T23:58:35.512468+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:58:35.512468+0000","last_peered":"2024-04-26T23:58:35.512468+0000","last_clean":"2024-04-26T23:58:35.512468+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:58:35.512468+0000","last_undegraded":"2024-04-26T23:58:35.512468+0000","last_fullsized":"2024-04-26T23:58:35.512468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575109,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56612,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315228,"statfs":{"total":96636764160,"available":96578793472,"internally_reserved":0,"allocated":3297280,"data_stored":1867539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60128,"kb_used_data":6736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311712,"statfs":{"total":96636764160,"available":96575193088,"internally_reserved":0,"allocated":6897664,"data_stored":5471447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575110,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58080,"kb_used_data":4692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313760,"statfs":{"total":96636764160,"available":96577290240,"internally_reserved":0,"allocated":4804608,"data_stored":3380829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68388,"kb_used_data":10840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303452,"statfs":{"total":96636764160,"available":96566734848,"internally_reserved":0,"allocated":11100160,"data_stored":9672355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49820,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322020,"statfs":{"total":96636764160,"available":96585748480,"internally_reserved":0,"allocated":4866048,"data_stored":3422201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49760,"kb_used_data":4688,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322080,"statfs":{"total":96636764160,"available":96585809920,"internally_reserved":0,"allocated":4800512,"data_stored":3374835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49752,"kb_used_data":4680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322088,"statfs":{"total":96636764160,"available":96585818112,"internally_reserved":0,"allocated":4792320,"data_stored":3374183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575110,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69076,"kb_used_data":7368,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302764,"statfs":{"total":96636764160,"available":96566030336,"internally_reserved":0,"allocated":7544832,"data_stored":6109067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:58:41.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:58:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.123+0000 7fe01be00640 1 Processor -- start 2024-04-26T23:58:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.124+0000 7fe01be00640 1 -- start start 2024-04-26T23:58:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe01be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe01c15f4e0 0x7fe01c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe01be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0d11f0 con 0x7fe01c0b4000 2024-04-26T23:58:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe01be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0d5f20 con 0x7fe01c0d2130 2024-04-26T23:58:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe01be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c0d0ce0 con 0x7fe01c15f4e0 2024-04-26T23:58:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe013e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe01c0b4000 0x7fe01c0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41264/0 (socket says 172.21.15.46:41264) 2024-04-26T23:58:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.125+0000 7fe013e00640 1 -- 172.21.15.46:0/846872382 learned_addr learned my addr 172.21.15.46:0/846872382 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01aa00640 1 --2- 172.21.15.46:0/846872382 >> v2:172.21.15.142:3300/0 conn(0x7fe01c15f4e0 0x7fe01c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759377968 0 0) 0x7fe01c0d11f0 con 0x7fe01c0b4000 2024-04-26T23:58:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe000003520 con 0x7fe01c0b4000 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3569233084 0 0) 0x7fe000003520 con 0x7fe01c0b4000 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 >> v2:172.21.15.142:3300/0 conn(0x7fe01c15f4e0 msgr2=0x7fe01c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 --2- 172.21.15.46:0/846872382 >> v2:172.21.15.142:3300/0 conn(0x7fe01c15f4e0 0x7fe01c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 >> v1:172.21.15.142:6789/0 conn(0x7fe01c0d2130 legacy=0x7fe01c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0d0f10 con 0x7fe01c0b4000 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fe004001670 con 0x7fe01c0b4000 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 >> v1:172.21.15.46:6789/0 conn(0x7fe01c0b4000 legacy=0x7fe01c0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe000003a00 0x7fe000003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.126+0000 7fe01a000640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0000051c0 0x7fe00000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe00000c090 con 0x7fe000004800 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe00000c240 con 0x7fe000003a00 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe00000c440 con 0x7fe0000051c0 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 -- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0000051c0 msgr2=0x7fe00000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0000051c0 0x7fe00000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe01aa00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe000003a00 0x7fe000003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 -- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe000003a00 msgr2=0x7fe000003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe000003a00 0x7fe000003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe01b400640 1 -- 172.21.15.46:0/846872382 reap_dead start 2024-04-26T23:58:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.127+0000 7fe013e00640 1 -- 172.21.15.46:0/846872382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000003520 con 0x7fe000004800 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.128+0000 7fe013e00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe000004c80 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c110 tx=0x7fe000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4790504908cde01b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.128+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004007c40 con 0x7fe000004800 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.128+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe004005af0 con 0x7fe000004800 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.128+0000 7fe01a000640 1 -- 172.21.15.46:0/846872382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004007070 con 0x7fe000004800 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 -- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 msgr2=0x7fe000004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe000004c80 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c110 tx=0x7fe000005830 comp rx=0 tx=0).stop 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 -- 172.21.15.46:0/846872382 shutdown_connections 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 --2- 172.21.15.46:0/846872382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe000004c80 unknown :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 -- 172.21.15.46:0/846872382 >> 172.21.15.46:0/846872382 conn(0x7fe01c0d5010 msgr2=0x7fe01c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 -- 172.21.15.46:0/846872382 shutdown_connections 2024-04-26T23:58:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.129+0000 7fe01be00640 1 -- 172.21.15.46:0/846872382 wait complete. 2024-04-26T23:58:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.131+0000 7fe01be00640 1 Processor -- start 2024-04-26T23:58:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.131+0000 7fe01be00640 1 -- start start 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.132+0000 7fe01be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.132+0000 7fe01be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe01c1eae30 0x7fe01c157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.132+0000 7fe01be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 0x7fe01c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0b1d60 con 0x7fe000004800 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0d2990 con 0x7fe01c1eae30 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c15f8c0 con 0x7fe01c1eb2b0 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42958/0 (socket says 172.21.15.46:42958) 2024-04-26T23:58:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 -- 172.21.15.46:0/3800934402 learned_addr learned my addr 172.21.15.46:0/3800934402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01b400640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 0x7fe01c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe013e00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe01c1eae30 0x7fe01c157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 -- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 msgr2=0x7fe01c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 0x7fe01c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 -- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe01c1eae30 msgr2=0x7fe01c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe01c1eae30 0x7fe01c157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01aa00640 1 -- 172.21.15.46:0/3800934402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0c9590 con 0x7fe000004800 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.133+0000 7fe01b400640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 0x7fe01c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe01aa00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe00c010810 tx=0x7fe00c010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00f040 con 0x7fe000004800 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00c00eb60 con 0x7fe000004800 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01c400 con 0x7fe000004800 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c0c9870 con 0x7fe000004800 2024-04-26T23:58:42.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.134+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0c9e30 con 0x7fe000004800 2024-04-26T23:58:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.135+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c0693b0 con 0x7fe000004800 2024-04-26T23:58:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.136+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe00c01c5a0 con 0x7fe000004800 2024-04-26T23:58:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.137+0000 7fe012a00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 0x7fdff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.137+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fe00c08c0b0 con 0x7fe000004800 2024-04-26T23:58:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.138+0000 7fe013e00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 0x7fdff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:42.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.139+0000 7fe013e00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 0x7fdff806edb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe004032000 tx=0x7fe004009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:42.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.142+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe00c018030 con 0x7fe000004800 2024-04-26T23:58:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.348+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe01c0d33d0 con 0x7fe000004800 2024-04-26T23:58:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.350+0000 7fe012a00640 1 -- 172.21.15.46:0/3800934402 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) ==== 74+0+13397 (secure 0 0 0) 0x7fe00c059f10 con 0x7fe000004800 2024-04-26T23:58:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:42.351 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":216,"fsid":"554dfe58-9c53-4ba0-a961-d2b323744aa7","created":"2024-04-26T23:36:48.636947+0000","modified":"2024-04-26T23:58:37.229137+0000","last_up_change":"2024-04-26T23:37:00.631953+0000","last_in_change":"2024-04-26T23:36:51.976402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:37:01.917968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f8ebab0-bd0d-42b2-9960-b5e6dcf1bf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6801","nonce":3755207586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6803","nonce":3755207586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6807","nonce":3755207586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3755207586},{"type":"v1","addr":"172.21.15.46:6805","nonce":3755207586}]},"public_addr":"172.21.15.46:6801/3755207586","cluster_addr":"172.21.15.46:6803/3755207586","heartbeat_back_addr":"172.21.15.46:6807/3755207586","heartbeat_front_addr":"172.21.15.46:6805/3755207586","state":["exists","up"]},{"osd":1,"uuid":"95298e5e-636c-4b28-a96c-2c87dec49625","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6817","nonce":3879162742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6819","nonce":3879162742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6823","nonce":3879162742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3879162742},{"type":"v1","addr":"172.21.15.46:6821","nonce":3879162742}]},"public_addr":"172.21.15.46:6817/3879162742","cluster_addr":"172.21.15.46:6819/3879162742","heartbeat_back_addr":"172.21.15.46:6823/3879162742","heartbeat_front_addr":"172.21.15.46:6821/3879162742","state":["exists","up"]},{"osd":2,"uuid":"ddd74cdc-da9f-49c8-a006-719a401998cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6809","nonce":2486201425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6811","nonce":2486201425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6815","nonce":2486201425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2486201425},{"type":"v1","addr":"172.21.15.46:6813","nonce":2486201425}]},"public_addr":"172.21.15.46:6809/2486201425","cluster_addr":"172.21.15.46:6811/2486201425","heartbeat_back_addr":"172.21.15.46:6815/2486201425","heartbeat_front_addr":"172.21.15.46:6813/2486201425","state":["exists","up"]},{"osd":3,"uuid":"1798bf3a-bfea-4dbb-8467-b047a259e4d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6825","nonce":1706950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6827","nonce":1706950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6831","nonce":1706950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1706950386},{"type":"v1","addr":"172.21.15.46:6829","nonce":1706950386}]},"public_addr":"172.21.15.46:6825/1706950386","cluster_addr":"172.21.15.46:6827/1706950386","heartbeat_back_addr":"172.21.15.46:6831/1706950386","heartbeat_front_addr":"172.21.15.46:6829/1706950386","state":["exists","up"]},{"osd":4,"uuid":"d52e92f6-d76c-4e1d-9f36-20d9cf6a27e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6825","nonce":2432053156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6827","nonce":2432053156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6831","nonce":2432053156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2432053156},{"type":"v1","addr":"172.21.15.142:6829","nonce":2432053156}]},"public_addr":"172.21.15.142:6825/2432053156","cluster_addr":"172.21.15.142:6827/2432053156","heartbeat_back_addr":"172.21.15.142:6831/2432053156","heartbeat_front_addr":"172.21.15.142:6829/2432053156","state":["exists","up"]},{"osd":5,"uuid":"2df71ca1-4fe1-49cb-b231-4998fa699f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6802","nonce":82441601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6805","nonce":82441601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6811","nonce":82441601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":82441601},{"type":"v1","addr":"172.21.15.142:6808","nonce":82441601}]},"public_addr":"172.21.15.142:6802/82441601","cluster_addr":"172.21.15.142:6805/82441601","heartbeat_back_addr":"172.21.15.142:6811/82441601","heartbeat_front_addr":"172.21.15.142:6808/82441601","state":["exists","up"]},{"osd":6,"uuid":"7fb1ccc0-6622-4d7d-8396-ce094d195e98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6803","nonce":715004401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6809","nonce":715004401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6815","nonce":715004401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":715004401},{"type":"v1","addr":"172.21.15.142:6813","nonce":715004401}]},"public_addr":"172.21.15.142:6803/715004401","cluster_addr":"172.21.15.142:6809/715004401","heartbeat_back_addr":"172.21.15.142:6815/715004401","heartbeat_front_addr":"172.21.15.142:6813/715004401","state":["exists","up"]},{"osd":7,"uuid":"51cd5087-296d-45f2-9aa7-832ca1a521ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6817","nonce":631850901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6819","nonce":631850901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6823","nonce":631850901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":631850901},{"type":"v1","addr":"172.21.15.142:6821","nonce":631850901}]},"public_addr":"172.21.15.142:6817/631850901","cluster_addr":"172.21.15.142:6819/631850901","heartbeat_back_addr":"172.21.15.142:6823/631850901","heartbeat_front_addr":"172.21.15.142:6821/631850901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.596898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.379359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.321626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:56.855357+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.664523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.394348+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.657677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:36:57.662814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2610577522":"2024-04-27T23:58:34.208933+0000","172.21.15.46:6833/2610577522":"2024-04-27T23:58:34.208933+0000","172.21.15.46:6835/3782420006":"2024-04-27T23:57:18.527554+0000","172.21.15.46:6833/3782420006":"2024-04-27T23:57:18.527554+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:58:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 msgr2=0x7fdff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 0x7fdff806edb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe004032000 tx=0x7fe004009030 comp rx=0 tx=0).stop 2024-04-26T23:58:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 msgr2=0x7fe01c0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe00c010810 tx=0x7fe00c010cd0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 shutdown_connections 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdff806c8f0 0x7fdff806edb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe01c1eb2b0 0x7fe01c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe01c1eae30 0x7fe01c157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.352+0000 7fe01be00640 1 --2- 172.21.15.46:0/3800934402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe000004800 0x7fe01c0cb390 unknown :-1 s=CLOSED pgs=3023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.353+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 >> 172.21.15.46:0/3800934402 conn(0x7fe01c0d5010 msgr2=0x7fe01c15faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.353+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 shutdown_connections 2024-04-26T23:58:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:42.353+0000 7fe01be00640 1 -- 172.21.15.46:0/3800934402 wait complete. 2024-04-26T23:58:43.374 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T23:58:43.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.601+0000 7f78d1200640 1 Processor -- start 2024-04-26T23:58:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.602+0000 7f78d1200640 1 -- start start 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.603+0000 7f78d1200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78cc0cfbc0 0x7f78cc0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.603+0000 7f78d1200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc0d01a0 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78d1200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc0d0480 con 0x7f78cc1627f0 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78d1200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f78cc0d25c0 con 0x7f78cc0cfbc0 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78caa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78cc0d3cc0 0x7f78cc0b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41266/0 (socket says 172.21.15.46:41266) 2024-04-26T23:58:43.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78caa00640 1 -- 172.21.15.46:0/4173816740 learned_addr learned my addr 172.21.15.46:0/4173816740 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78cbe00640 1 --2- 172.21.15.46:0/4173816740 >> v2:172.21.15.142:3300/0 conn(0x7f78cc0cfbc0 0x7f78cc0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873772302 0 0) 0x7f78cc0d01a0 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.604+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78b0003520 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1785734391 0 0) 0x7f78b0003520 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 >> v2:172.21.15.142:3300/0 conn(0x7f78cc0cfbc0 msgr2=0x7f78cc0d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 --2- 172.21.15.46:0/4173816740 >> v2:172.21.15.142:3300/0 conn(0x7f78cc0cfbc0 0x7f78cc0d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 >> v1:172.21.15.142:6789/0 conn(0x7f78cc1627f0 legacy=0x7f78cc0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0d0760 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f78b8001670 con 0x7f78cc0d3cc0 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 >> v1:172.21.15.46:6789/0 conn(0x7f78cc0d3cc0 legacy=0x7f78cc0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b0003a00 0x7f78b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b0004800 0x7f78b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 0x7f78b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0d25c0 con 0x7f78b00051c0 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78b000c0a0 con 0x7f78b0004800 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78b000c2a0 con 0x7f78b0003a00 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.605+0000 7f78cbe00640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 0x7f78b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78caa00640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b0004800 0x7f78b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cbe00640 1 -- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b0003a00 msgr2=0x7f78b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cb400640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b0003a00 0x7f78b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cbe00640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b0003a00 0x7f78b0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cbe00640 1 -- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b0004800 msgr2=0x7f78b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cbe00640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b0004800 0x7f78b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.606+0000 7f78cbe00640 1 -- 172.21.15.46:0/4173816740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b0003520 con 0x7f78b00051c0 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.607+0000 7f78cbe00640 1 -- 172.21.15.46:0/4173816740 reap_dead start 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.607+0000 7f78cbe00640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 0x7f78b000b9c0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009ac0 tx=0x7f78b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c6332ac37d458f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.607+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4009690 con 0x7f78b00051c0 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.607+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c4004020 con 0x7f78b00051c0 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.607+0000 7f78ca000640 1 -- 172.21.15.46:0/4173816740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4004410 con 0x7f78b00051c0 2024-04-26T23:58:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 -- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 msgr2=0x7f78b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 0x7f78b000b9c0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009ac0 tx=0x7f78b0005830 comp rx=0 tx=0).stop 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 -- 172.21.15.46:0/4173816740 shutdown_connections 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 --2- 172.21.15.46:0/4173816740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78b00051c0 0x7f78b000b9c0 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 -- 172.21.15.46:0/4173816740 >> 172.21.15.46:0/4173816740 conn(0x7f78cc09e380 msgr2=0x7f78cc0d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 -- 172.21.15.46:0/4173816740 shutdown_connections 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.608+0000 7f78d1200640 1 -- 172.21.15.46:0/4173816740 wait complete. 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.610+0000 7f78d1200640 1 Processor -- start 2024-04-26T23:58:43.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.610+0000 7f78d1200640 1 -- start start 2024-04-26T23:58:43.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b00051c0 0x7f78cc15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 0x7f78cc15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc15f840 0x7f78cc15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc162bd0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0d0d20 con 0x7f78cc15f840 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78d1200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc0d2080 con 0x7f78b00051c0 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 0x7f78cc15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 0x7f78cc15cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42982/0 (socket says 172.21.15.46:42982) 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.611+0000 7f78caa00640 1 -- 172.21.15.46:0/2886018872 learned_addr learned my addr 172.21.15.46:0/2886018872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78cb400640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b00051c0 0x7f78cc15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78cbe00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc15f840 0x7f78cc15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 -- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b00051c0 msgr2=0x7f78cc15c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b00051c0 0x7f78cc15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 -- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc15f840 msgr2=0x7f78cc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc15f840 0x7f78cc15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0b8080 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78cbe00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc15f840 0x7f78cc15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78caa00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 0x7f78cc15cfe0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f78b800e730 tx=0x7f78b800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800c6c0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8018070 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800cac0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.612+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc0b8360 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.613+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc0b8920 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.613+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78cc0693b0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.615+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78b800cc60 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78c3400640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78ac06c8f0 0x7f78ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f78b808b8e0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78c3400640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f78ac072e30 0x7f78ac075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b8008e50 con 0x7f78ac072e30 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f78b80902a0 con 0x7f78cc0b7bf0 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78cb400640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78ac06c8f0 0x7f78ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.616+0000 7f78cbe00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f78ac072e30 0x7f78ac075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.617+0000 7f78cbe00640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f78ac072e30 0x7f78ac075270 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.617+0000 7f78cb400640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78ac06c8f0 0x7f78ac06edb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f78bc007fd0 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.618+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== osd.0 v2:172.21.15.46:6800/3755207586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78b8008e50 con 0x7f78ac072e30 2024-04-26T23:58:43.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.633+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 --> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f78cc0d3390 con 0x7f78ac072e30 2024-04-26T23:58:43.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.633+0000 7f78c3400640 1 -- 172.21.15.46:0/2886018872 <== osd.0 v2:172.21.15.46:6800/3755207586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f78cc0d3390 con 0x7f78ac072e30 2024-04-26T23:58:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:43.635 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f78ac072e30 msgr2=0x7f78ac075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:6800/3755207586,v1:172.21.15.46:6801/3755207586] conn(0x7f78ac072e30 0x7f78ac075270 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78ac06c8f0 msgr2=0x7f78ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f78ac06c8f0 0x7f78ac06edb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f78bc007fd0 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 msgr2=0x7f78cc15cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 --2- 172.21.15.46:0/2886018872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78cc0b7bf0 0x7f78cc15cfe0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f78b800e730 tx=0x7f78b800ebf0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78cbe00640 1 -- 172.21.15.46:0/2886018872 reap_dead start 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 shutdown_connections 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 >> 172.21.15.46:0/2886018872 conn(0x7f78cc09e380 msgr2=0x7f78cc06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 shutdown_connections 2024-04-26T23:58:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.634+0000 7f78d1200640 1 -- 172.21.15.46:0/2886018872 wait complete. 2024-04-26T23:58:43.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T23:58:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.862+0000 7fab49400640 1 Processor -- start 2024-04-26T23:58:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.862+0000 7fab49400640 1 -- start start 2024-04-26T23:58:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab49400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab440d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab49400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab440d11f0 con 0x7fab440b4000 2024-04-26T23:58:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab49400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab440d5f20 con 0x7fab440d2130 2024-04-26T23:58:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab49400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab440d0ce0 con 0x7fab4415f4e0 2024-04-26T23:58:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab3b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fab440b4000 0x7fab440d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41270/0 (socket says 172.21.15.46:41270) 2024-04-26T23:58:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.863+0000 7fab3b400640 1 -- 172.21.15.46:0/473644425 learned_addr learned my addr 172.21.15.46:0/473644425 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43e00640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab440d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671851772 0 0) 0x7fab440d11f0 con 0x7fab440b4000 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab24003520 con 0x7fab440b4000 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2782955615 0 0) 0x7fab24003520 con 0x7fab440b4000 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 msgr2=0x7fab440d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab440d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 >> v1:172.21.15.142:6789/0 conn(0x7fab440d2130 legacy=0x7fab440b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab440d0f10 con 0x7fab440b4000 2024-04-26T23:58:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fab2c001670 con 0x7fab440b4000 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.864+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 >> v1:172.21.15.46:6789/0 conn(0x7fab440b4000 legacy=0x7fab440d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab24004d40 0x7fab240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab2400ba70 con 0x7fab24003f60 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fab2400bc20 con 0x7fab24004d40 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab2400be20 con 0x7fab4415f4e0 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab3b400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab43e00640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab48a00640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab24004d40 0x7fab240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab3b400640 1 -- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 msgr2=0x7fab240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab3b400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4415f4e0 0x7fab240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab3b400640 1 -- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab24004d40 msgr2=0x7fab240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.865+0000 7fab3b400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab24004d40 0x7fab240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.866+0000 7fab3b400640 1 -- 172.21.15.46:0/473644425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab24003520 con 0x7fab24003f60 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.867+0000 7fab3b400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab24004800 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00c240 tx=0x7fab24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecf4cb99c31da9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.867+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c007d10 con 0x7fab24003f60 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.867+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab2c005ad0 con 0x7fab24003f60 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.867+0000 7fab43400640 1 -- 172.21.15.46:0/473644425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c007070 con 0x7fab24003f60 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 -- 172.21.15.46:0/473644425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 msgr2=0x7fab24004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 --2- 172.21.15.46:0/473644425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab24004800 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00c240 tx=0x7fab24005830 comp rx=0 tx=0).stop 2024-04-26T23:58:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab48a00640 1 -- 172.21.15.46:0/473644425 reap_dead start 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 -- 172.21.15.46:0/473644425 shutdown_connections 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 -- 172.21.15.46:0/473644425 >> 172.21.15.46:0/473644425 conn(0x7fab440d5010 msgr2=0x7fab440d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 -- 172.21.15.46:0/473644425 shutdown_connections 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.868+0000 7fab49400640 1 -- 172.21.15.46:0/473644425 wait complete. 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.870+0000 7fab49400640 1 Processor -- start 2024-04-26T23:58:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.870+0000 7fab49400640 1 -- start start 2024-04-26T23:58:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 0x7fab4415b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab44157eb0 0x7fab44158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab440b1d60 con 0x7fab24003f60 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fab440d2990 con 0x7fab44157eb0 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab49400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab4415f8c0 con 0x7fab4409eef0 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43006/0 (socket says 172.21.15.46:43006) 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab43e00640 1 -- 172.21.15.46:0/1674031674 learned_addr learned my addr 172.21.15.46:0/1674031674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab3b400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 0x7fab4415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.871+0000 7fab43e00640 1 -- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 msgr2=0x7fab4415b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab43e00640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 0x7fab4415b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab43e00640 1 -- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab44157eb0 msgr2=0x7fab44158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab43e00640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab44157eb0 0x7fab44158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab43e00640 1 -- 172.21.15.46:0/1674031674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab440a1440 con 0x7fab24003f60 2024-04-26T23:58:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab3b400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 0x7fab4415b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab43e00640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fab34011600 tx=0x7fab34011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3400f920 con 0x7fab24003f60 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab3400fac0 con 0x7fab24003f60 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3400fe70 con 0x7fab24003f60 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab440a1720 con 0x7fab24003f60 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.872+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab44158850 con 0x7fab24003f60 2024-04-26T23:58:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.874+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab440a1e40 con 0x7fab24003f60 2024-04-26T23:58:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.875+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab34015460 con 0x7fab24003f60 2024-04-26T23:58:43.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.876+0000 7fab40c00640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 0x7fab2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:43.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.876+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fab3408c330 con 0x7fab24003f60 2024-04-26T23:58:43.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.877+0000 7fab3b400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 0x7fab2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.878+0000 7fab3b400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 0x7fab2006ede0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fab2c033000 tx=0x7fab2c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:43.880+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab34019030 con 0x7fab24003f60 2024-04-26T23:58:44.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.091+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fab440693b0 con 0x7fab2006c920 2024-04-26T23:58:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.092+0000 7fab40c00640 1 -- 172.21.15.46:0/1674031674 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fab440693b0 con 0x7fab2006c920 2024-04-26T23:58:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.095+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 msgr2=0x7fab2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.095+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 0x7fab2006ede0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fab2c033000 tx=0x7fab2c02e000 comp rx=0 tx=0).stop 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.095+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 msgr2=0x7fab44159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.095+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fab34011600 tx=0x7fab34011ac0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 shutdown_connections 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fab2006c920 0x7fab2006ede0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab44157eb0 0x7fab44158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab4409eef0 0x7fab4415b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 --2- 172.21.15.46:0/1674031674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab24003f60 0x7fab44159480 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 >> 172.21.15.46:0/1674031674 conn(0x7fab440d5010 msgr2=0x7fab44161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 shutdown_connections 2024-04-26T23:58:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.096+0000 7fab49400640 1 -- 172.21.15.46:0/1674031674 wait complete. 2024-04-26T23:58:44.118 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T23:58:44.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.331+0000 7fc96ca00640 1 Processor -- start 2024-04-26T23:58:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.331+0000 7fc96ca00640 1 -- start start 2024-04-26T23:58:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.332+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 0x7fc9680cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.332+0000 7fc96ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc9680d3cc0 0x7fc9680b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.332+0000 7fc96ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9680d01a0 con 0x7fc9680d2080 2024-04-26T23:58:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.332+0000 7fc96ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d0480 con 0x7fc9680d3cc0 2024-04-26T23:58:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.332+0000 7fc96ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680d0760 con 0x7fc9681627f0 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.333+0000 7fc966a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc9680d2080 0x7fc9680d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41282/0 (socket says 172.21.15.46:41282) 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.333+0000 7fc966a00640 1 -- 172.21.15.46:0/262000786 learned_addr learned my addr 172.21.15.46:0/262000786 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.333+0000 7fc967e00640 1 --2- 172.21.15.46:0/262000786 >> v2:172.21.15.46:3300/0 conn(0x7fc9680d3cc0 0x7fc9680b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.333+0000 7fc967400640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 0x7fc9680cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.333+0000 7fc966000640 1 -- 172.21.15.46:0/262000786 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149790607 0 0) 0x7fc9680d01a0 con 0x7fc9680d2080 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc966000640 1 -- 172.21.15.46:0/262000786 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc94c003520 con 0x7fc9680d2080 2024-04-26T23:58:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc967e00640 1 -- 172.21.15.46:0/262000786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 msgr2=0x7fc9680cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc967e00640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 0x7fc9680cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc967e00640 1 -- 172.21.15.46:0/262000786 >> v1:172.21.15.46:6789/0 conn(0x7fc9680d2080 legacy=0x7fc9680d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc967e00640 1 -- 172.21.15.46:0/262000786 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680d0a40 con 0x7fc9680d3cc0 2024-04-26T23:58:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.334+0000 7fc967400640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 0x7fc9680cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.335+0000 7fc967e00640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b4550 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7fc9600097a0 tx=0x7fc96002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db8b27d7abc2db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.335+0000 7fc966000640 1 -- 172.21.15.46:0/262000786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960037070 con 0x7fc9680d3cc0 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.335+0000 7fc966000640 1 -- 172.21.15.46:0/262000786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc960032440 con 0x7fc9680d3cc0 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.335+0000 7fc966000640 1 -- 172.21.15.46:0/262000786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960032810 con 0x7fc9680d3cc0 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 -- 172.21.15.46:0/262000786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9680d3cc0 msgr2=0x7fc9680b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b4550 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7fc9600097a0 tx=0x7fc96002a230 comp rx=0 tx=0).stop 2024-04-26T23:58:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 -- 172.21.15.46:0/262000786 shutdown_connections 2024-04-26T23:58:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b4550 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 --2- 172.21.15.46:0/262000786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9681627f0 0x7fc9680cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 -- 172.21.15.46:0/262000786 >> 172.21.15.46:0/262000786 conn(0x7fc96809e380 msgr2=0x7fc9680d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 -- 172.21.15.46:0/262000786 shutdown_connections 2024-04-26T23:58:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.336+0000 7fc96ca00640 1 -- 172.21.15.46:0/262000786 wait complete. 2024-04-26T23:58:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.338+0000 7fc96ca00640 1 Processor -- start 2024-04-26T23:58:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.338+0000 7fc96ca00640 1 -- start start 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9680d2080 0x7fc9680c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 0x7fc9680b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc968162bd0 con 0x7fc9681627f0 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680d24c0 con 0x7fc9680d3cc0 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc96ca00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680b4a90 con 0x7fc9680d2080 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 0x7fc9680b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 0x7fc9680b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43030/0 (socket says 172.21.15.46:43030) 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc967e00640 1 -- 172.21.15.46:0/3741791499 learned_addr learned my addr 172.21.15.46:0/3741791499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc966a00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.339+0000 7fc967400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9680d2080 0x7fc9680c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 -- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9680d2080 msgr2=0x7fc9680c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9680d2080 0x7fc9680c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 -- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9680d3cc0 msgr2=0x7fc9680b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680c9460 con 0x7fc9681627f0 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc966a00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9680d3cc0 0x7fc9680b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9680d2080 0x7fc9680c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc967e00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 0x7fc9680b6520 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fc960035040 tx=0x7fc9600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960037040 con 0x7fc9681627f0 2024-04-26T23:58:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc960042070 con 0x7fc9681627f0 2024-04-26T23:58:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9600308a0 con 0x7fc9681627f0 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.340+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9680c9740 con 0x7fc9681627f0 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.341+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9680cd2e0 con 0x7fc9681627f0 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.341+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9680693b0 con 0x7fc9681627f0 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.342+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc960030a40 con 0x7fc9681627f0 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.343+0000 7fc95f400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc94406c8f0 0x7fc94406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fc9600ac320 con 0x7fc9681627f0 2024-04-26T23:58:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc95f400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7fc944072e30 0x7fc944075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc96000af30 con 0x7fc944072e30 2024-04-26T23:58:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7fc9600b3050 con 0x7fc9681627f0 2024-04-26T23:58:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc966a00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7fc944072e30 0x7fc944075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.344+0000 7fc967400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc94406c8f0 0x7fc94406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.345+0000 7fc966a00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7fc944072e30 0x7fc944075270 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.345+0000 7fc967400640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc94406c8f0 0x7fc94406edb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc9580094c0 tx=0x7fc95800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.346+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== osd.1 v2:172.21.15.46:6816/3879162742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc96000af30 con 0x7fc944072e30 2024-04-26T23:58:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.360+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 --> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc9680ca760 con 0x7fc944072e30 2024-04-26T23:58:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.361+0000 7fc95f400640 1 -- 172.21.15.46:0/3741791499 <== osd.1 v2:172.21.15.46:6816/3879162742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc9680ca760 con 0x7fc944072e30 2024-04-26T23:58:44.362 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:44.362 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.361+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7fc944072e30 msgr2=0x7fc944075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.361+0000 7fc96ca00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:6816/3879162742,v1:172.21.15.46:6817/3879162742] conn(0x7fc944072e30 0x7fc944075270 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc94406c8f0 msgr2=0x7fc94406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc94406c8f0 0x7fc94406edb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc9580094c0 tx=0x7fc95800b040 comp rx=0 tx=0).stop 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 msgr2=0x7fc9680b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 --2- 172.21.15.46:0/3741791499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9681627f0 0x7fc9680b6520 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fc960035040 tx=0x7fc9600046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc967e00640 1 -- 172.21.15.46:0/3741791499 reap_dead start 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 shutdown_connections 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 >> 172.21.15.46:0/3741791499 conn(0x7fc96809e380 msgr2=0x7fc9680d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 shutdown_connections 2024-04-26T23:58:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.362+0000 7fc96ca00640 1 -- 172.21.15.46:0/3741791499 wait complete. 2024-04-26T23:58:44.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T23:58:44.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:44.434 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.586+0000 7ffa78a00640 1 Processor -- start 2024-04-26T23:58:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.587+0000 7ffa78a00640 1 -- start start 2024-04-26T23:58:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.588+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa7415edf0 0x7ffa740cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.588+0000 7ffa78a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa7415f1d0 0x7ffa740cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.588+0000 7ffa78a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa740b03c0 con 0x7ffa740cb1a0 2024-04-26T23:58:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.588+0000 7ffa78a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffa740cf5c0 con 0x7ffa7415f1d0 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.588+0000 7ffa78a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa740cb680 con 0x7ffa7415edf0 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa7415f1d0 0x7ffa740cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa73e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa740cb1a0 0x7ffa740ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41292/0 (socket says 172.21.15.46:41292) 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa73e00640 1 -- 172.21.15.46:0/2094152794 learned_addr learned my addr 172.21.15.46:0/2094152794 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa73400640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa7415edf0 0x7ffa740cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72a00640 1 -- 172.21.15.46:0/2094152794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa7415edf0 msgr2=0x7ffa740cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72a00640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa7415edf0 0x7ffa740cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72a00640 1 -- 172.21.15.46:0/2094152794 >> v1:172.21.15.46:6789/0 conn(0x7ffa740cb1a0 legacy=0x7ffa740ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72000640 1 -- 172.21.15.46:0/2094152794 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581922100 0 0) 0x7ffa740b03c0 con 0x7ffa740cb1a0 2024-04-26T23:58:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.589+0000 7ffa72a00640 1 -- 172.21.15.46:0/2094152794 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa740cb960 con 0x7ffa7415f1d0 2024-04-26T23:58:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa72a00640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415f1d0 0x7ffa740cd060 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7ffa600097a0 tx=0x7ffa6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90043648490129ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa72000640 1 -- 172.21.15.46:0/2094152794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60037070 con 0x7ffa7415f1d0 2024-04-26T23:58:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa72000640 1 -- 172.21.15.46:0/2094152794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa60032440 con 0x7ffa7415f1d0 2024-04-26T23:58:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa72000640 1 -- 172.21.15.46:0/2094152794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60032810 con 0x7ffa7415f1d0 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa78a00640 1 -- 172.21.15.46:0/2094152794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415f1d0 msgr2=0x7ffa740cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.591+0000 7ffa78a00640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415f1d0 0x7ffa740cd060 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7ffa600097a0 tx=0x7ffa6002a210 comp rx=0 tx=0).stop 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 -- 172.21.15.46:0/2094152794 shutdown_connections 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415f1d0 0x7ffa740cd060 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 --2- 172.21.15.46:0/2094152794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa7415edf0 0x7ffa740cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 -- 172.21.15.46:0/2094152794 >> 172.21.15.46:0/2094152794 conn(0x7ffa740b26d0 msgr2=0x7ffa740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 -- 172.21.15.46:0/2094152794 shutdown_connections 2024-04-26T23:58:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.592+0000 7ffa78a00640 1 -- 172.21.15.46:0/2094152794 wait complete. 2024-04-26T23:58:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.593+0000 7ffa78a00640 1 Processor -- start 2024-04-26T23:58:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.594+0000 7ffa78a00640 1 -- start start 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 0x7ffa7415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa741f8960 0x7ffa741f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa740cc200 con 0x7ffa7415edf0 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa740cbf20 con 0x7ffa740cb1a0 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa78a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa740c6950 con 0x7ffa741f8960 2024-04-26T23:58:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43040/0 (socket says 172.21.15.46:43040) 2024-04-26T23:58:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 -- 172.21.15.46:0/1530771503 learned_addr learned my addr 172.21.15.46:0/1530771503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa73e00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa741f8960 0x7ffa741f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa73400640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 0x7ffa7415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 -- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa741f8960 msgr2=0x7ffa741f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa741f8960 0x7ffa741f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 -- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 msgr2=0x7ffa7415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 0x7ffa7415ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa72a00640 1 -- 172.21.15.46:0/1530771503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa74160970 con 0x7ffa7415edf0 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.595+0000 7ffa73400640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 0x7ffa7415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa72a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7ffa600098c0 tx=0x7ffa600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60037040 con 0x7ffa7415edf0 2024-04-26T23:58:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa60042070 con 0x7ffa7415edf0 2024-04-26T23:58:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60030bb0 con 0x7ffa7415edf0 2024-04-26T23:58:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa74160c50 con 0x7ffa7415edf0 2024-04-26T23:58:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.596+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa74161210 con 0x7ffa7415edf0 2024-04-26T23:58:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.598+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa74068bc0 con 0x7ffa7415edf0 2024-04-26T23:58:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.599+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa60047460 con 0x7ffa7415edf0 2024-04-26T23:58:44.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.600+0000 7ffa6b400640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 0x7ffa5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:44.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.600+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7ffa600ac670 con 0x7ffa7415edf0 2024-04-26T23:58:44.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.600+0000 7ffa73400640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 0x7ffa5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:44.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.601+0000 7ffa73400640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 0x7ffa5406edb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.604+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa74068bc0 con 0x7ffa7415edf0 2024-04-26T23:58:44.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.808+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ffa740693b0 con 0x7ffa5406c8f0 2024-04-26T23:58:44.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.809+0000 7ffa6b400640 1 -- 172.21.15.46:0/1530771503 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffa740693b0 con 0x7ffa5406c8f0 2024-04-26T23:58:44.810 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 msgr2=0x7ffa5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 0x7ffa5406edb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64008040 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 msgr2=0x7ffa741f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7ffa600098c0 tx=0x7ffa600046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 shutdown_connections 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffa5406c8f0 0x7ffa5406edb0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa741f8960 0x7ffa741f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa7415edf0 0x7ffa741f8420 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 --2- 172.21.15.46:0/1530771503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740cb1a0 0x7ffa7415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.811+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 >> 172.21.15.46:0/1530771503 conn(0x7ffa740b26d0 msgr2=0x7ffa740cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.812+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 shutdown_connections 2024-04-26T23:58:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:44.812+0000 7ffa78a00640 1 -- 172.21.15.46:0/1530771503 wait complete. 2024-04-26T23:58:44.832 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T23:58:44.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:45.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.092+0000 7fbd25e00640 1 Processor -- start 2024-04-26T23:58:45.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.092+0000 7fbd25e00640 1 -- start start 2024-04-26T23:58:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.094+0000 7fbd25e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbd2006e3d0 0x7fbd20071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.094+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.094+0000 7fbd25e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd20072450 con 0x7fbd20164d20 2024-04-26T23:58:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.094+0000 7fbd25e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbd200d1a80 con 0x7fbd2006e3d0 2024-04-26T23:58:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.094+0000 7fbd25e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbd20164420 con 0x7fbd200d10e0 2024-04-26T23:58:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.095+0000 7fbd24a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbd2006e3d0 0x7fbd20071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.095+0000 7fbd25400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.095+0000 7fbd25400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.46:56612/0 (socket says 172.21.15.46:56612) 2024-04-26T23:58:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.095+0000 7fbd25400640 1 -- 172.21.15.46:0/3221495474 learned_addr learned my addr 172.21.15.46:0/3221495474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.096+0000 7fbd25400640 1 -- 172.21.15.46:0/3221495474 >> v2:172.21.15.142:3300/0 conn(0x7fbd2006e3d0 msgr2=0x7fbd20071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.096+0000 7fbd25400640 1 --2- 172.21.15.46:0/3221495474 >> v2:172.21.15.142:3300/0 conn(0x7fbd2006e3d0 0x7fbd20071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.096+0000 7fbd25400640 1 -- 172.21.15.46:0/3221495474 >> v1:172.21.15.142:6789/0 conn(0x7fbd20164d20 legacy=0x7fbd20163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.096+0000 7fbd25400640 1 -- 172.21.15.46:0/3221495474 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd20164700 con 0x7fbd200d10e0 2024-04-26T23:58:45.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.097+0000 7fbd25400640 1 --2- 172.21.15.46:0/3221495474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10d654c28b8827b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.098+0000 7fbd1f400640 1 -- 172.21.15.46:0/3221495474 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18037070 con 0x7fbd200d10e0 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.098+0000 7fbd1f400640 1 -- 172.21.15.46:0/3221495474 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd18032440 con 0x7fbd200d10e0 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.098+0000 7fbd1f400640 1 -- 172.21.15.46:0/3221495474 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18032810 con 0x7fbd200d10e0 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 -- 172.21.15.46:0/3221495474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 msgr2=0x7fbd200d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3221495474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1802a270 comp rx=0 tx=0).stop 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 -- 172.21.15.46:0/3221495474 shutdown_connections 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3221495474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd200d10e0 0x7fbd200d1540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3221495474 >> v2:172.21.15.142:3300/0 conn(0x7fbd2006e3d0 0x7fbd20071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 -- 172.21.15.46:0/3221495474 >> 172.21.15.46:0/3221495474 conn(0x7fbd2006c770 msgr2=0x7fbd2006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 -- 172.21.15.46:0/3221495474 shutdown_connections 2024-04-26T23:58:45.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.099+0000 7fbd25e00640 1 -- 172.21.15.46:0/3221495474 wait complete. 2024-04-26T23:58:45.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.101+0000 7fbd25e00640 1 Processor -- start 2024-04-26T23:58:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.101+0000 7fbd25e00640 1 -- start start 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2006e3d0 0x7fbd20146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 0x7fbd2015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd20164d20 0x7fbd201592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd201651d0 con 0x7fbd200d10e0 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd200726f0 con 0x7fbd20164d20 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2006e7b0 con 0x7fbd2006e3d0 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 0x7fbd2015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 0x7fbd2015ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43042/0 (socket says 172.21.15.46:43042) 2024-04-26T23:58:45.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd1fe00640 1 -- 172.21.15.46:0/3300185687 learned_addr learned my addr 172.21.15.46:0/3300185687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd24a00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2006e3d0 0x7fbd20146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.102+0000 7fbd25400640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd20164d20 0x7fbd201592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 -- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2006e3d0 msgr2=0x7fbd20146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd2006e3d0 0x7fbd20146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 -- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd20164d20 msgr2=0x7fbd201592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd20164d20 0x7fbd201592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd20159c40 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd25400640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd20164d20 0x7fbd201592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1fe00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 0x7fbd2015ace0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00c370 tx=0x7fbd0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c017070 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0c0052d0 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd20159f20 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c003850 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.103+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2015a4e0 con 0x7fbd200d10e0 2024-04-26T23:58:45.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.104+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd200693b0 con 0x7fbd200d10e0 2024-04-26T23:58:45.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.106+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd0c01a070 con 0x7fbd200d10e0 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd1cc00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fbd0c08ad90 con 0x7fbd200d10e0 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd1cc00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7fbd00072e30 0x7fbd00075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd0c015e70 con 0x7fbd00072e30 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7fbd0c08d430 con 0x7fbd200d10e0 2024-04-26T23:58:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd24a00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.107+0000 7fbd25400640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7fbd00072e30 0x7fbd00075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.108+0000 7fbd25400640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7fbd00072e30 0x7fbd00075270 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.108+0000 7fbd24a00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbd100074a0 tx=0x7fbd1000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.109+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== osd.2 v2:172.21.15.46:6808/2486201425 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd0c015e70 con 0x7fbd00072e30 2024-04-26T23:58:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.124+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 --> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbd20159810 con 0x7fbd00072e30 2024-04-26T23:58:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.125+0000 7fbd1cc00640 1 -- 172.21.15.46:0/3300185687 <== osd.2 v2:172.21.15.46:6808/2486201425 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbd20159810 con 0x7fbd00072e30 2024-04-26T23:58:45.126 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:45.126 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:45.126 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:45.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.125+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7fbd00072e30 msgr2=0x7fbd00075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.125+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:6808/2486201425,v1:172.21.15.46:6809/2486201425] conn(0x7fbd00072e30 0x7fbd00075270 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0006c8f0 msgr2=0x7fbd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbd100074a0 tx=0x7fbd1000b040 comp rx=0 tx=0).stop 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 msgr2=0x7fbd2015ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 --2- 172.21.15.46:0/3300185687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd200d10e0 0x7fbd2015ace0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00c370 tx=0x7fbd0c00c830 comp rx=0 tx=0).stop 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25400640 1 -- 172.21.15.46:0/3300185687 reap_dead start 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 shutdown_connections 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 >> 172.21.15.46:0/3300185687 conn(0x7fbd2006c770 msgr2=0x7fbd200b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 shutdown_connections 2024-04-26T23:58:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.126+0000 7fbd25e00640 1 -- 172.21.15.46:0/3300185687 wait complete. 2024-04-26T23:58:45.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T23:58:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.343+0000 7f90c2800640 1 Processor -- start 2024-04-26T23:58:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.344+0000 7f90c2800640 1 -- start start 2024-04-26T23:58:45.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.345+0000 7f90c2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90bc0d5a90 0x7f90bc0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.345+0000 7f90c2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90bc0b65e0 con 0x7f90bc162a60 2024-04-26T23:58:45.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.345+0000 7f90c2800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90bc0d1530 con 0x7f90bc0b4e20 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.345+0000 7f90c2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f90bc0d12e0 con 0x7f90bc0d5a90 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.345+0000 7f90c0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90bc0d5a90 0x7f90bc0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90c1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f90bc162a60 0x7f90bc0b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41306/0 (socket says 172.21.15.46:41306) 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90c1400640 1 -- 172.21.15.46:0/4080954248 learned_addr learned my addr 172.21.15.46:0/4080954248 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90bbe00640 1 -- 172.21.15.46:0/4080954248 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 629567900 0 0) 0x7f90bc0b65e0 con 0x7f90bc162a60 2024-04-26T23:58:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90bbe00640 1 -- 172.21.15.46:0/4080954248 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90a0003520 con 0x7f90bc162a60 2024-04-26T23:58:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90c0a00640 1 -- 172.21.15.46:0/4080954248 >> v1:172.21.15.142:6789/0 conn(0x7f90bc0b4e20 legacy=0x7f90bc0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90c0a00640 1 -- 172.21.15.46:0/4080954248 >> v1:172.21.15.46:6789/0 conn(0x7f90bc162a60 legacy=0x7f90bc0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.346+0000 7f90c0a00640 1 -- 172.21.15.46:0/4080954248 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc0b5ef0 con 0x7f90bc0d5a90 2024-04-26T23:58:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.347+0000 7f90c0a00640 1 --2- 172.21.15.46:0/4080954248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc0d5a90 0x7f90bc0d0a80 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f90a80097a0 tx=0x7f90a802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6108ab5f7c3513e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.348+0000 7f90bbe00640 1 -- 172.21.15.46:0/4080954248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8037070 con 0x7f90bc0d5a90 2024-04-26T23:58:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.348+0000 7f90bbe00640 1 -- 172.21.15.46:0/4080954248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a802adc0 con 0x7f90bc0d5a90 2024-04-26T23:58:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.348+0000 7f90bbe00640 1 -- 172.21.15.46:0/4080954248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8032640 con 0x7f90bc0d5a90 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.348+0000 7f90c2800640 1 -- 172.21.15.46:0/4080954248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc0d5a90 msgr2=0x7f90bc0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.348+0000 7f90c2800640 1 --2- 172.21.15.46:0/4080954248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc0d5a90 0x7f90bc0d0a80 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f90a80097a0 tx=0x7f90a802a0b0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.349+0000 7f90c2800640 1 -- 172.21.15.46:0/4080954248 shutdown_connections 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.349+0000 7f90c2800640 1 --2- 172.21.15.46:0/4080954248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc0d5a90 0x7f90bc0d0a80 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.349+0000 7f90c2800640 1 -- 172.21.15.46:0/4080954248 >> 172.21.15.46:0/4080954248 conn(0x7f90bc0d4fa0 msgr2=0x7f90bc0d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.349+0000 7f90c2800640 1 -- 172.21.15.46:0/4080954248 shutdown_connections 2024-04-26T23:58:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.349+0000 7f90c2800640 1 -- 172.21.15.46:0/4080954248 wait complete. 2024-04-26T23:58:45.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.350+0000 7f90c2800640 1 Processor -- start 2024-04-26T23:58:45.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.351+0000 7f90c2800640 1 -- start start 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.351+0000 7f90c2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc0b4e20 0x7f90bc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0d5a90 0x7f90bc0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc0d20c0 con 0x7f90bc162a60 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc0d05f0 con 0x7f90bc0b4e20 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc0d2240 con 0x7f90bc0d5a90 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43056/0 (socket says 172.21.15.46:43056) 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 -- 172.21.15.46:0/1548138033 learned_addr learned my addr 172.21.15.46:0/1548138033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1400640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc0b4e20 0x7f90bc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 -- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0d5a90 msgr2=0x7f90bc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0d5a90 0x7f90bc0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 -- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc0b4e20 msgr2=0x7f90bc156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc0b4e20 0x7f90bc156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.352+0000 7f90c1e00640 1 -- 172.21.15.46:0/1548138033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc1567d0 con 0x7f90bc162a60 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90c1e00640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c410 tx=0x7f90b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4011790 con 0x7f90bc162a60 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b4017070 con 0x7f90bc162a60 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4011bb0 con 0x7f90bc162a60 2024-04-26T23:58:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc0cee80 con 0x7f90bc162a60 2024-04-26T23:58:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.353+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc0cf440 con 0x7f90bc162a60 2024-04-26T23:58:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.354+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc069ac0 con 0x7f90bc162a60 2024-04-26T23:58:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.355+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90b401c460 con 0x7f90bc162a60 2024-04-26T23:58:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.356+0000 7f90b9600640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.356+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f90b408b600 con 0x7f90bc162a60 2024-04-26T23:58:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.357+0000 7f90c1400640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.358+0000 7f90c1400640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 0x7f909c06edb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0012e0 tx=0x7f90ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.361+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90b4021020 con 0x7f90bc162a60 2024-04-26T23:58:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.564+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f90bc161320 con 0x7f909c06c8f0 2024-04-26T23:58:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.565+0000 7f90b9600640 1 -- 172.21.15.46:0/1548138033 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f90bc161320 con 0x7f909c06c8f0 2024-04-26T23:58:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T23:58:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 msgr2=0x7f909c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 0x7f909c06edb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0012e0 tx=0x7f90ac009040 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 msgr2=0x7f90bc0cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c410 tx=0x7f90b400c8d0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 shutdown_connections 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90bc162a60 0x7f90bc0cca00 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0d5a90 0x7f90bc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 --2- 172.21.15.46:0/1548138033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc0b4e20 0x7f90bc156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 >> 172.21.15.46:0/1548138033 conn(0x7f90bc0d4fa0 msgr2=0x7f90bc0d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 shutdown_connections 2024-04-26T23:58:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.568+0000 7f90c2800640 1 -- 172.21.15.46:0/1548138033 wait complete. 2024-04-26T23:58:45.589 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T23:58:45.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.793+0000 7fc490a00640 1 Processor -- start 2024-04-26T23:58:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.794+0000 7fc490a00640 1 -- start start 2024-04-26T23:58:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc490a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 0x7fc48c06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc490a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc48c0b4690 0x7fc48c0d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc490a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48c0d2180 con 0x7fc48c15f2b0 2024-04-26T23:58:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc490a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc48c0cfcb0 con 0x7fc48c0b4690 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc490a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc48c0d2740 con 0x7fc48c15f690 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc48b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc48c0b4690 0x7fc48c0d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc48b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc48c0b4690 0x7fc48c0d3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43068/0 (socket says 172.21.15.46:43068) 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc48b200640 1 -- 172.21.15.46:0/3048258386 learned_addr learned my addr 172.21.15.46:0/3048258386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.796+0000 7fc489e00640 1 --2- 172.21.15.46:0/3048258386 >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 0x7fc48c06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.797+0000 7fc48b200640 1 -- 172.21.15.46:0/3048258386 >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 msgr2=0x7fc48c06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.797+0000 7fc48b200640 1 --2- 172.21.15.46:0/3048258386 >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 0x7fc48c06cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.797+0000 7fc48b200640 1 -- 172.21.15.46:0/3048258386 >> v1:172.21.15.142:6789/0 conn(0x7fc48c15f2b0 legacy=0x7fc48c0d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.797+0000 7fc48b200640 1 -- 172.21.15.46:0/3048258386 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c0d2460 con 0x7fc48c0b4690 2024-04-26T23:58:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.797+0000 7fc489e00640 1 --2- 172.21.15.46:0/3048258386 >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 0x7fc48c06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.798+0000 7fc48b200640 1 --2- 172.21.15.46:0/3048258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c0d3680 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0d28c0 tx=0x7fc48402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a7530961de424c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.799+0000 7fc489400640 1 -- 172.21.15.46:0/3048258386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc48c0b4690 2024-04-26T23:58:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.799+0000 7fc489400640 1 -- 172.21.15.46:0/3048258386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484032440 con 0x7fc48c0b4690 2024-04-26T23:58:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.799+0000 7fc489400640 1 -- 172.21.15.46:0/3048258386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484032810 con 0x7fc48c0b4690 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.799+0000 7fc490a00640 1 -- 172.21.15.46:0/3048258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 msgr2=0x7fc48c0d3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.799+0000 7fc490a00640 1 --2- 172.21.15.46:0/3048258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c0d3680 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0d28c0 tx=0x7fc48402a180 comp rx=0 tx=0).stop 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 -- 172.21.15.46:0/3048258386 shutdown_connections 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 --2- 172.21.15.46:0/3048258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c0d3680 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 --2- 172.21.15.46:0/3048258386 >> v2:172.21.15.142:3300/0 conn(0x7fc48c15f690 0x7fc48c06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 -- 172.21.15.46:0/3048258386 >> 172.21.15.46:0/3048258386 conn(0x7fc48c09e3c0 msgr2=0x7fc48c0d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 -- 172.21.15.46:0/3048258386 shutdown_connections 2024-04-26T23:58:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.800+0000 7fc490a00640 1 -- 172.21.15.46:0/3048258386 wait complete. 2024-04-26T23:58:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.802+0000 7fc490a00640 1 Processor -- start 2024-04-26T23:58:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.802+0000 7fc490a00640 1 -- start start 2024-04-26T23:58:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48c15f2b0 0x7fc48c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48c0b7750 0x7fc48c14b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c06d430 con 0x7fc48c0b4690 2024-04-26T23:58:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c0d0250 con 0x7fc48c15f2b0 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc490a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc48c06d5b0 con 0x7fc48c0b7750 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43070/0 (socket says 172.21.15.46:43070) 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48a800640 1 -- 172.21.15.46:0/1284205335 learned_addr learned my addr 172.21.15.46:0/1284205335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc489e00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48c15f2b0 0x7fc48c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48b200640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48c0b7750 0x7fc48c14b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48a800640 1 -- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48c0b7750 msgr2=0x7fc48c14b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.803+0000 7fc48a800640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48c0b7750 0x7fc48c14b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc48a800640 1 -- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48c15f2b0 msgr2=0x7fc48c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc48a800640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48c15f2b0 0x7fc48c15da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc48a800640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c1613c0 con 0x7fc48c0b4690 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc48b200640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48c0b7750 0x7fc48c14b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc489e00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48c15f2b0 0x7fc48c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc48a800640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c15d550 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c430 tx=0x7fc47c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c011790 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c017070 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c011b70 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c161660 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.804+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c161c20 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.805+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc48c0baf70 con 0x7fc48c0b4690 2024-04-26T23:58:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.807+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc47c011d10 con 0x7fc48c0b4690 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc482a00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fc47c08b890 con 0x7fc48c0b4690 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc482a00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7fc468072e30 0x7fc468075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc47c018ec0 con 0x7fc468072e30 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7fc47c0902c0 con 0x7fc48c0b4690 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc48b200640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7fc468072e30 0x7fc468075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.808+0000 7fc489e00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.809+0000 7fc48b200640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7fc468072e30 0x7fc468075270 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.809+0000 7fc489e00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc474009540 tx=0x7fc47400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.810+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== osd.3 v2:172.21.15.46:6824/1706950386 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc47c018ec0 con 0x7fc468072e30 2024-04-26T23:58:45.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.825+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 --> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc48c069ac0 con 0x7fc468072e30 2024-04-26T23:58:45.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.825+0000 7fc482a00640 1 -- 172.21.15.46:0/1284205335 <== osd.3 v2:172.21.15.46:6824/1706950386 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc48c069ac0 con 0x7fc468072e30 2024-04-26T23:58:45.827 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:45.827 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7fc468072e30 msgr2=0x7fc468075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:6824/1706950386,v1:172.21.15.46:6825/1706950386] conn(0x7fc468072e30 0x7fc468075270 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc46806c8f0 msgr2=0x7fc46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc474009540 tx=0x7fc47400b040 comp rx=0 tx=0).stop 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 msgr2=0x7fc48c15d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 --2- 172.21.15.46:0/1284205335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc48c0b4690 0x7fc48c15d550 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c430 tx=0x7fc47c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc48b200640 1 -- 172.21.15.46:0/1284205335 reap_dead start 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 shutdown_connections 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.826+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 >> 172.21.15.46:0/1284205335 conn(0x7fc48c09e3c0 msgr2=0x7fc48c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.827+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 shutdown_connections 2024-04-26T23:58:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:45.827+0000 7fc490a00640 1 -- 172.21.15.46:0/1284205335 wait complete. 2024-04-26T23:58:45.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T23:58:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.060+0000 7fb92b600640 1 Processor -- start 2024-04-26T23:58:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.061+0000 7fb92b600640 1 -- start start 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.062+0000 7fb92b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb92416e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.062+0000 7fb92b600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9240b4940 con 0x7fb92406eb60 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.062+0000 7fb92b600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9240b4c20 con 0x7fb92406e780 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.062+0000 7fb92b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240b4f00 con 0x7fb92414d200 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.063+0000 7fb923e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb92406eb60 0x7fb92414caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41314/0 (socket says 172.21.15.46:41314) 2024-04-26T23:58:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.063+0000 7fb923e00640 1 -- 172.21.15.46:0/4068151979 learned_addr learned my addr 172.21.15.46:0/4068151979 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.063+0000 7fb929600640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb92416e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.063+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437217446 0 0) 0x7fb9240b4940 con 0x7fb92406eb60 2024-04-26T23:58:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.063+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb908003520 con 0x7fb92406eb60 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3335459316 0 0) 0x7fb908003520 con 0x7fb92406eb60 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 msgr2=0x7fb92416e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb92416e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 >> v1:172.21.15.142:6789/0 conn(0x7fb92406e780 legacy=0x7fb9240b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9240b51e0 con 0x7fb92406eb60 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7fb90c001670 con 0x7fb92406eb60 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 >> v1:172.21.15.46:6789/0 conn(0x7fb92406eb60 legacy=0x7fb92414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb9080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb908003f60 0x7fb908004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb908004d40 0x7fb9080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240b4f00 con 0x7fb908003f60 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb90800baa0 con 0x7fb908004d40 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb90800bca0 con 0x7fb92414d200 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.064+0000 7fb923e00640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb908003f60 0x7fb908004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb928c00640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb9080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb929600640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb908004d40 0x7fb9080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb923e00640 1 -- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 msgr2=0x7fb9080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb923e00640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb92414d200 0x7fb9080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb923e00640 1 -- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb908004d40 msgr2=0x7fb9080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb923e00640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb908004d40 0x7fb9080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.065+0000 7fb923e00640 1 -- 172.21.15.46:0/4068151979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb908003520 con 0x7fb908003f60 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.066+0000 7fb923e00640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb908003f60 0x7fb908004800 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00c210 tx=0x7fb908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7ac85790066416 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.066+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c007cf0 con 0x7fb908003f60 2024-04-26T23:58:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.066+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c005af0 con 0x7fb908003f60 2024-04-26T23:58:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.066+0000 7fb923400640 1 -- 172.21.15.46:0/4068151979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c007070 con 0x7fb908003f60 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 -- 172.21.15.46:0/4068151979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb908003f60 msgr2=0x7fb908004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 --2- 172.21.15.46:0/4068151979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb908003f60 0x7fb908004800 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00c210 tx=0x7fb908005830 comp rx=0 tx=0).stop 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb929600640 1 -- 172.21.15.46:0/4068151979 reap_dead start 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 -- 172.21.15.46:0/4068151979 shutdown_connections 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 -- 172.21.15.46:0/4068151979 >> 172.21.15.46:0/4068151979 conn(0x7fb92406c530 msgr2=0x7fb92406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 -- 172.21.15.46:0/4068151979 shutdown_connections 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.067+0000 7fb92b600640 1 -- 172.21.15.46:0/4068151979 wait complete. 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.069+0000 7fb92b600640 1 Processor -- start 2024-04-26T23:58:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.069+0000 7fb92b600640 1 -- start start 2024-04-26T23:58:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb908003f60 0x7fb9240b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9240b32b0 0x7fb9241fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240b4590 con 0x7fb924146b50 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb924073370 con 0x7fb9240b32b0 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb92b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb9241476a0 con 0x7fb908003f60 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43092/0 (socket says 172.21.15.46:43092) 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 -- 172.21.15.46:0/3792322402 learned_addr learned my addr 172.21.15.46:0/3792322402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb928c00640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb908003f60 0x7fb9240b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb923e00640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9240b32b0 0x7fb9241fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 -- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb908003f60 msgr2=0x7fb9240b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb908003f60 0x7fb9240b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 -- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9240b32b0 msgr2=0x7fb9241fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9240b32b0 0x7fb9241fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.070+0000 7fb929600640 1 -- 172.21.15.46:0/3792322402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241fdb20 con 0x7fb924146b50 2024-04-26T23:58:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb929600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b6520 tx=0x7fb91c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c004440 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c01b070 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c003850 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9240b3750 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.071+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92415f4c0 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.073+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb924069530 con 0x7fb924146b50 2024-04-26T23:58:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.074+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb91c0039f0 con 0x7fb924146b50 2024-04-26T23:58:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.075+0000 7fb920c00640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 0x7fb90006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.075+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fb91c08b870 con 0x7fb924146b50 2024-04-26T23:58:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.075+0000 7fb928c00640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 0x7fb90006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.076+0000 7fb928c00640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 0x7fb90006ee00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb914007fd0 tx=0x7fb914009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.079+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb91c059ea0 con 0x7fb924146b50 2024-04-26T23:58:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.285+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb9240d02b0 con 0x7fb90006c940 2024-04-26T23:58:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.286+0000 7fb920c00640 1 -- 172.21.15.46:0/3792322402 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb9240d02b0 con 0x7fb90006c940 2024-04-26T23:58:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T23:58:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 msgr2=0x7fb90006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 0x7fb90006ee00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb914007fd0 tx=0x7fb914009040 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 msgr2=0x7fb9241639a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b6520 tx=0x7fb91c011920 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 shutdown_connections 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fb90006c940 0x7fb90006ee00 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb924146b50 0x7fb9241639a0 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9240b32b0 0x7fb9241fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 --2- 172.21.15.46:0/3792322402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb908003f60 0x7fb9240b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 >> 172.21.15.46:0/3792322402 conn(0x7fb92406c530 msgr2=0x7fb92414c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 shutdown_connections 2024-04-26T23:58:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.289+0000 7fb92b600640 1 -- 172.21.15.46:0/3792322402 wait complete. 2024-04-26T23:58:46.309 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T23:58:46.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.524+0000 7ffb0b200640 1 Processor -- start 2024-04-26T23:58:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.524+0000 7ffb0b200640 1 -- start start 2024-04-26T23:58:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146b20 0x7ffb0c0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb0c0d1620 0x7ffb0c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb0c0b3ce0 con 0x7ffb0c146740 2024-04-26T23:58:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffb0c0b48f0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0c0b3fc0 con 0x7ffb0c146b20 2024-04-26T23:58:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb09e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb0c146740 0x7ffb0c0cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41318/0 (socket says 172.21.15.46:41318) 2024-04-26T23:58:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb09e00640 1 -- 172.21.15.46:0/578033042 learned_addr learned my addr 172.21.15.46:0/578033042 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.526+0000 7ffb0a800640 1 --2- 172.21.15.46:0/578033042 >> v2:172.21.15.46:3300/0 conn(0x7ffb0c0d1620 0x7ffb0c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb03e00640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146b20 0x7ffb0c0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb09400640 1 -- 172.21.15.46:0/578033042 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403457073 0 0) 0x7ffb0c0b3ce0 con 0x7ffb0c146740 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb09400640 1 -- 172.21.15.46:0/578033042 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf0003520 con 0x7ffb0c146740 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb0a800640 1 -- 172.21.15.46:0/578033042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146b20 msgr2=0x7ffb0c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb0a800640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146b20 0x7ffb0c0d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb0a800640 1 -- 172.21.15.46:0/578033042 >> v1:172.21.15.46:6789/0 conn(0x7ffb0c146740 legacy=0x7ffb0c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.527+0000 7ffb0a800640 1 -- 172.21.15.46:0/578033042 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c0b42a0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.528+0000 7ffb0a800640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c0cf590 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009d90 tx=0x7ffb0402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bf39052d75078c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.528+0000 7ffb09400640 1 -- 172.21.15.46:0/578033042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04031600 con 0x7ffb0c0d1620 2024-04-26T23:58:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.528+0000 7ffb09400640 1 -- 172.21.15.46:0/578033042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb040317a0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.528+0000 7ffb09400640 1 -- 172.21.15.46:0/578033042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04031b50 con 0x7ffb0c0d1620 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 -- 172.21.15.46:0/578033042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 msgr2=0x7ffb0c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c0cf590 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009d90 tx=0x7ffb0402a5d0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 -- 172.21.15.46:0/578033042 shutdown_connections 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c0cf590 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 --2- 172.21.15.46:0/578033042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146b20 0x7ffb0c0d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 -- 172.21.15.46:0/578033042 >> 172.21.15.46:0/578033042 conn(0x7ffb0c06c770 msgr2=0x7ffb0c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 -- 172.21.15.46:0/578033042 shutdown_connections 2024-04-26T23:58:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.529+0000 7ffb0b200640 1 -- 172.21.15.46:0/578033042 wait complete. 2024-04-26T23:58:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.531+0000 7ffb0b200640 1 Processor -- start 2024-04-26T23:58:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.531+0000 7ffb0b200640 1 -- start start 2024-04-26T23:58:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146740 0x7ffb0c142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c146b20 0x7ffb0c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c0cdde0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c15e560 con 0x7ffb0c146b20 2024-04-26T23:58:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.532+0000 7ffb0b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0c0cfad0 con 0x7ffb0c146740 2024-04-26T23:58:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.533+0000 7ffb09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.533+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146740 0x7ffb0c142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.533+0000 7ffb09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43116/0 (socket says 172.21.15.46:43116) 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.533+0000 7ffb09e00640 1 -- 172.21.15.46:0/2923392402 learned_addr learned my addr 172.21.15.46:0/2923392402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.533+0000 7ffb0a800640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c146b20 0x7ffb0c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 -- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146740 msgr2=0x7ffb0c142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb0c146740 0x7ffb0c142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 -- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c146b20 msgr2=0x7ffb0c145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c146b20 0x7ffb0c145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c0b1c00 con 0x7ffb0c0d1620 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb09e00640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c142080 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c430 tx=0x7ffafc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc017070 con 0x7ffb0c0d1620 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffafc005ce0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0072f0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c1430d0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.534+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c143690 con 0x7ffb0c0d1620 2024-04-26T23:58:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.535+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb0c0693b0 con 0x7ffb0c0d1620 2024-04-26T23:58:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.537+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffafc01a070 con 0x7ffb0c0d1620 2024-04-26T23:58:46.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb02000640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffae806c940 0x7ffae806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7ffafc08ad20 con 0x7ffb0c0d1620 2024-04-26T23:58:46.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb02000640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7ffae8072e80 0x7ffae80752c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffafc009e50 con 0x7ffae8072e80 2024-04-26T23:58:46.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7ffafc08d470 con 0x7ffb0c0d1620 2024-04-26T23:58:46.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.538+0000 7ffb03e00640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffae806c940 0x7ffae806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.539+0000 7ffb0a800640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7ffae8072e80 0x7ffae80752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.540+0000 7ffb03e00640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffae806c940 0x7ffae806ee00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c1425c0 tx=0x7ffaf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.540+0000 7ffb0a800640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7ffae8072e80 0x7ffae80752c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.542+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== osd.4 v2:172.21.15.142:6824/2432053156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffafc009e50 con 0x7ffae8072e80 2024-04-26T23:58:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.557+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 --> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffb0c0b5e60 con 0x7ffae8072e80 2024-04-26T23:58:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.558+0000 7ffb02000640 1 -- 172.21.15.46:0/2923392402 <== osd.4 v2:172.21.15.142:6824/2432053156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffb0c0b5e60 con 0x7ffae8072e80 2024-04-26T23:58:46.560 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:46.560 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:46.560 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7ffae8072e80 msgr2=0x7ffae80752c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6824/2432053156,v1:172.21.15.142:6825/2432053156] conn(0x7ffae8072e80 0x7ffae80752c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffae806c940 msgr2=0x7ffae806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffae806c940 0x7ffae806ee00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c1425c0 tx=0x7ffaf4008040 comp rx=0 tx=0).stop 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 msgr2=0x7ffb0c142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 --2- 172.21.15.46:0/2923392402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb0c0d1620 0x7ffb0c142080 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c430 tx=0x7ffafc00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0a800640 1 -- 172.21.15.46:0/2923392402 reap_dead start 2024-04-26T23:58:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 shutdown_connections 2024-04-26T23:58:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 >> 172.21.15.46:0/2923392402 conn(0x7ffb0c06c770 msgr2=0x7ffb0c0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.559+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 shutdown_connections 2024-04-26T23:58:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.560+0000 7ffb0b200640 1 -- 172.21.15.46:0/2923392402 wait complete. 2024-04-26T23:58:46.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T23:58:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.774+0000 7fddb9000640 1 Processor -- start 2024-04-26T23:58:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.774+0000 7fddb9000640 1 -- start start 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.775+0000 7fddb9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 0x7fddb40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.775+0000 7fddb9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb4160910 0x7fddb4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.775+0000 7fddb9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb4073ad0 con 0x7fddb41466b0 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.775+0000 7fddb9000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fddb4146a90 con 0x7fddb4160910 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.775+0000 7fddb9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fddb4073d70 con 0x7fddb40720e0 2024-04-26T23:58:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb4160910 0x7fddb4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb4160910 0x7fddb4160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43120/0 (socket says 172.21.15.46:43120) 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 -- 172.21.15.46:0/1024597823 learned_addr learned my addr 172.21.15.46:0/1024597823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 -- 172.21.15.46:0/1024597823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 msgr2=0x7fddb40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddaaa00640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 0x7fddb40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 0x7fddb40cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 -- 172.21.15.46:0/1024597823 >> v1:172.21.15.142:6789/0 conn(0x7fddb41466b0 legacy=0x7fddb406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddb3e00640 1 -- 172.21.15.46:0/1024597823 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4074050 con 0x7fddb4160910 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.776+0000 7fddaaa00640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 0x7fddb40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.777+0000 7fddb3e00640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4160910 0x7fddb4160da0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7fddac009870 tx=0x7fddac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d0a8b9e0bbf85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb2a00640 1 -- 172.21.15.46:0/1024597823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac037070 con 0x7fddb4160910 2024-04-26T23:58:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb2a00640 1 -- 172.21.15.46:0/1024597823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac032440 con 0x7fddb4160910 2024-04-26T23:58:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb2a00640 1 -- 172.21.15.46:0/1024597823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac032810 con 0x7fddb4160910 2024-04-26T23:58:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 -- 172.21.15.46:0/1024597823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4160910 msgr2=0x7fddb4160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4160910 0x7fddb4160da0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7fddac009870 tx=0x7fddac02a210 comp rx=0 tx=0).stop 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 -- 172.21.15.46:0/1024597823 shutdown_connections 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4160910 0x7fddb4160da0 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 --2- 172.21.15.46:0/1024597823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb40720e0 0x7fddb40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.778+0000 7fddb9000640 1 -- 172.21.15.46:0/1024597823 >> 172.21.15.46:0/1024597823 conn(0x7fddb40b26d0 msgr2=0x7fddb40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.779+0000 7fddb9000640 1 -- 172.21.15.46:0/1024597823 shutdown_connections 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.779+0000 7fddb9000640 1 -- 172.21.15.46:0/1024597823 wait complete. 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.780+0000 7fddb9000640 1 Processor -- start 2024-04-26T23:58:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.781+0000 7fddb9000640 1 -- start start 2024-04-26T23:58:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 0x7fddb40ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb4160910 0x7fddb41f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cdda0 con 0x7fddb41466b0 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cb470 con 0x7fddb40720e0 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40cb990 con 0x7fddb4160910 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43128/0 (socket says 172.21.15.46:43128) 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 -- 172.21.15.46:0/3510304033 learned_addr learned my addr 172.21.15.46:0/3510304033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb3400640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 0x7fddb40ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 -- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb4160910 msgr2=0x7fddb41f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb4160910 0x7fddb41f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 -- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 msgr2=0x7fddb40ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 0x7fddb40ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddaaa00640 1 -- 172.21.15.46:0/3510304033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41f85a0 con 0x7fddb41466b0 2024-04-26T23:58:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.782+0000 7fddb3400640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 0x7fddb40ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddaaa00640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c820 tx=0x7fdd9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c015070 con 0x7fddb41466b0 2024-04-26T23:58:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c007a30 con 0x7fddb41466b0 2024-04-26T23:58:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb41f8880 con 0x7fddb41466b0 2024-04-26T23:58:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007e00 con 0x7fddb41466b0 2024-04-26T23:58:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.783+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb415e680 con 0x7fddb41466b0 2024-04-26T23:58:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.785+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb40693b0 con 0x7fddb41466b0 2024-04-26T23:58:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.785+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd9c018070 con 0x7fddb41466b0 2024-04-26T23:58:46.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.786+0000 7fddabe00640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:46.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.786+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fdd9c08bac0 con 0x7fddb41466b0 2024-04-26T23:58:46.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.787+0000 7fddb3400640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:46.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.788+0000 7fddb3400640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdda4004520 tx=0x7fdda4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:46.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.791+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd9c059f70 con 0x7fddb41466b0 2024-04-26T23:58:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.998+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fddb40c7180 con 0x7fdd9006c8f0 2024-04-26T23:58:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:46.999+0000 7fddabe00640 1 -- 172.21.15.46:0/3510304033 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fddb40c7180 con 0x7fdd9006c8f0 2024-04-26T23:58:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T23:58:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 msgr2=0x7fdd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdda4004520 tx=0x7fdda4009040 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 msgr2=0x7fddb40b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c820 tx=0x7fdd9c00cc20 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 shutdown_connections 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fddb4160910 0x7fddb41f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41466b0 0x7fddb40b1500 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 --2- 172.21.15.46:0/3510304033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddb40720e0 0x7fddb40ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 >> 172.21.15.46:0/3510304033 conn(0x7fddb40b26d0 msgr2=0x7fddb40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 shutdown_connections 2024-04-26T23:58:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.002+0000 7fddb9000640 1 -- 172.21.15.46:0/3510304033 wait complete. 2024-04-26T23:58:47.023 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T23:58:47.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.241+0000 7f02f5a00640 1 Processor -- start 2024-04-26T23:58:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.241+0000 7f02f5a00640 1 -- start start 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.243+0000 7f02f5a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f02f00d3cc0 0x7f02f00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.243+0000 7f02f5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02f00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02f5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f00d01a0 con 0x7f02f01627f0 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02f5a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f02f00d0480 con 0x7f02f00d3cc0 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02f5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00d0760 con 0x7f02f00d2080 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02efe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f02f01627f0 0x7f02f00d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41332/0 (socket says 172.21.15.46:41332) 2024-04-26T23:58:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02efe00640 1 -- 172.21.15.46:0/795158639 learned_addr learned my addr 172.21.15.46:0/795158639 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02f5000640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02f00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02e7e00640 1 --2- 172.21.15.46:0/795158639 >> v2:172.21.15.142:3300/0 conn(0x7f02f00d3cc0 0x7f02f00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.244+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971700299 0 0) 0x7f02f00d01a0 con 0x7f02f01627f0 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02d4003520 con 0x7f02f01627f0 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2503660662 0 0) 0x7f02d4003520 con 0x7f02f01627f0 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 msgr2=0x7f02f00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02f00b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 >> v2:172.21.15.142:3300/0 conn(0x7f02f00d3cc0 msgr2=0x7f02f00cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 --2- 172.21.15.46:0/795158639 >> v2:172.21.15.142:3300/0 conn(0x7f02f00d3cc0 0x7f02f00cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f00d0a40 con 0x7f02f01627f0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2335712352 0 0) 0x7f02e0001670 con 0x7f02f01627f0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 >> v1:172.21.15.46:6789/0 conn(0x7f02f01627f0 legacy=0x7f02f00d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02d4004d40 0x7f02d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d0480 con 0x7f02d4004d40 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d0760 con 0x7f02d40039b0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.245+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02d400b9f0 con 0x7f02f00d2080 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02d4004d40 0x7f02d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02efe00640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02e7e00640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 -- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 msgr2=0x7f02d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f00d2080 0x7f02d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 -- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40039b0 msgr2=0x7f02d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.246+0000 7f02f5000640 1 -- 172.21.15.46:0/795158639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d4003520 con 0x7f02d4004d40 2024-04-26T23:58:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.247+0000 7f02f5000640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02d4004d40 0x7f02d40051c0 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009fc0 tx=0x7f02d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3578af91b6bb305a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.247+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8009c10 con 0x7f02d4004d40 2024-04-26T23:58:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.247+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e8036070 con 0x7f02d4004d40 2024-04-26T23:58:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02ef400640 1 -- 172.21.15.46:0/795158639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8031860 con 0x7f02d4004d40 2024-04-26T23:58:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 -- 172.21.15.46:0/795158639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02d4004d40 msgr2=0x7f02d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 --2- 172.21.15.46:0/795158639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02d4004d40 0x7f02d40051c0 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009fc0 tx=0x7f02d4005830 comp rx=0 tx=0).stop 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5000640 1 -- 172.21.15.46:0/795158639 reap_dead start 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 -- 172.21.15.46:0/795158639 shutdown_connections 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 -- 172.21.15.46:0/795158639 >> 172.21.15.46:0/795158639 conn(0x7f02f009e380 msgr2=0x7f02f00d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 -- 172.21.15.46:0/795158639 shutdown_connections 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.248+0000 7f02f5a00640 1 -- 172.21.15.46:0/795158639 wait complete. 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.250+0000 7f02f5a00640 1 Processor -- start 2024-04-26T23:58:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.250+0000 7f02f5a00640 1 -- start start 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f01f5040 0x7f02f00b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 0x7f02f00b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02f01f5420 0x7f02f00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0162bd0 con 0x7f02f00b5da0 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d2530 con 0x7f02f01f5420 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02f5a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00b4550 con 0x7f02f01f5040 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.251+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 0x7f02f00b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 0x7f02f00b8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43138/0 (socket says 172.21.15.46:43138) 2024-04-26T23:58:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 -- 172.21.15.46:0/228811109 learned_addr learned my addr 172.21.15.46:0/228811109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02efe00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f01f5040 0x7f02f00b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02f5000640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02f01f5420 0x7f02f00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 -- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f01f5040 msgr2=0x7f02f00b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02f01f5040 0x7f02f00b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 -- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02f01f5420 msgr2=0x7f02f00b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02f01f5420 0x7f02f00b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f00b84f0 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.252+0000 7f02e7e00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 0x7f02f00b8f80 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f02d80116d0 tx=0x7f02d8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.253+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d800f960 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.253+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d800fb00 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.253+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8015460 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.253+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f00b87d0 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.253+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01545f0 con 0x7f02f00b5da0 2024-04-26T23:58:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.254+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02f00d2c30 con 0x7f02f00b5da0 2024-04-26T23:58:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.255+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02d8015600 con 0x7f02f00b5da0 2024-04-26T23:58:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02ecc00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f02cc06c920 0x7f02cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f02d808c610 con 0x7f02f00b5da0 2024-04-26T23:58:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02ecc00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7f02cc072e60 0x7f02cc0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02d8001d30 con 0x7f02cc072e60 2024-04-26T23:58:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f02d808caa0 con 0x7f02f00b5da0 2024-04-26T23:58:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.256+0000 7f02efe00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f02cc06c920 0x7f02cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.257+0000 7f02f5000640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7f02cc072e60 0x7f02cc0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.258+0000 7f02efe00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f02cc06c920 0x7f02cc06ede0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001050 tx=0x7f02e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.258+0000 7f02f5000640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7f02cc072e60 0x7f02cc0752a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.260+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== osd.5 v2:172.21.15.142:6801/82441601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02d8001d30 con 0x7f02cc072e60 2024-04-26T23:58:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.275+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 --> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f02f00693b0 con 0x7f02cc072e60 2024-04-26T23:58:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.276+0000 7f02ecc00640 1 -- 172.21.15.46:0/228811109 <== osd.5 v2:172.21.15.142:6801/82441601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f02f00693b0 con 0x7f02cc072e60 2024-04-26T23:58:47.277 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:47.277 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.276+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7f02cc072e60 msgr2=0x7f02cc0752a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.276+0000 7f02f5a00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6801/82441601,v1:172.21.15.142:6802/82441601] conn(0x7f02cc072e60 0x7f02cc0752a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f02cc06c920 msgr2=0x7f02cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f02cc06c920 0x7f02cc06ede0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001050 tx=0x7f02e0009040 comp rx=0 tx=0).stop 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 msgr2=0x7f02f00b8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 --2- 172.21.15.46:0/228811109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02f00b5da0 0x7f02f00b8f80 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f02d80116d0 tx=0x7f02d8011b90 comp rx=0 tx=0).stop 2024-04-26T23:58:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5000640 1 -- 172.21.15.46:0/228811109 reap_dead start 2024-04-26T23:58:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 shutdown_connections 2024-04-26T23:58:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 >> 172.21.15.46:0/228811109 conn(0x7f02f009e380 msgr2=0x7f02f00b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 shutdown_connections 2024-04-26T23:58:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.277+0000 7f02f5a00640 1 -- 172.21.15.46:0/228811109 wait complete. 2024-04-26T23:58:47.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T23:58:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.507+0000 7ffb4fa00640 1 Processor -- start 2024-04-26T23:58:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.507+0000 7ffb4fa00640 1 -- start start 2024-04-26T23:58:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4fa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb48160bd0 0x7ffb48161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4fa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb480732c0 0x7ffb48073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb48161940 con 0x7ffb4806e510 2024-04-26T23:58:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4fa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ffb48071f80 con 0x7ffb48160bd0 2024-04-26T23:58:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4fa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb48072260 con 0x7ffb480732c0 2024-04-26T23:58:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb4806e510 0x7ffb4806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41346/0 (socket says 172.21.15.46:41346) 2024-04-26T23:58:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb4d000640 1 -- 172.21.15.46:0/986065756 learned_addr learned my addr 172.21.15.46:0/986065756 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.509+0000 7ffb47e00640 1 --2- 172.21.15.46:0/986065756 >> v2:172.21.15.46:3300/0 conn(0x7ffb48160bd0 0x7ffb48161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb4da00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb480732c0 0x7ffb48073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47400640 1 -- 172.21.15.46:0/986065756 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277204788 0 0) 0x7ffb48161940 con 0x7ffb4806e510 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47400640 1 -- 172.21.15.46:0/986065756 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb2c003520 con 0x7ffb4806e510 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47e00640 1 -- 172.21.15.46:0/986065756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb480732c0 msgr2=0x7ffb48073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47e00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb480732c0 0x7ffb48073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47e00640 1 -- 172.21.15.46:0/986065756 >> v1:172.21.15.46:6789/0 conn(0x7ffb4806e510 legacy=0x7ffb4806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.510+0000 7ffb47e00640 1 -- 172.21.15.46:0/986065756 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb48072540 con 0x7ffb48160bd0 2024-04-26T23:58:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.511+0000 7ffb47e00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb48160bd0 0x7ffb48161060 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009dc0 tx=0x7ffb3802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d0479c50dbd6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.511+0000 7ffb47400640 1 -- 172.21.15.46:0/986065756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802ae60 con 0x7ffb48160bd0 2024-04-26T23:58:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.511+0000 7ffb47400640 1 -- 172.21.15.46:0/986065756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb38007740 con 0x7ffb48160bd0 2024-04-26T23:58:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.511+0000 7ffb47400640 1 -- 172.21.15.46:0/986065756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38007af0 con 0x7ffb48160bd0 2024-04-26T23:58:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 -- 172.21.15.46:0/986065756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb48160bd0 msgr2=0x7ffb48161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb48160bd0 0x7ffb48161060 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009dc0 tx=0x7ffb3802a680 comp rx=0 tx=0).stop 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 -- 172.21.15.46:0/986065756 shutdown_connections 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb480732c0 0x7ffb48073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/986065756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb48160bd0 0x7ffb48161060 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 -- 172.21.15.46:0/986065756 >> 172.21.15.46:0/986065756 conn(0x7ffb4806c530 msgr2=0x7ffb4806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 -- 172.21.15.46:0/986065756 shutdown_connections 2024-04-26T23:58:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.512+0000 7ffb4fa00640 1 -- 172.21.15.46:0/986065756 wait complete. 2024-04-26T23:58:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.514+0000 7ffb4fa00640 1 Processor -- start 2024-04-26T23:58:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.514+0000 7ffb4fa00640 1 -- start start 2024-04-26T23:58:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb4806e510 0x7ffb4813c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb48160bd0 0x7ffb4815f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48161720 con 0x7ffb480732c0 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb481615a0 con 0x7ffb48160bd0 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.515+0000 7ffb4fa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb48072d40 con 0x7ffb4806e510 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43160/0 (socket says 172.21.15.46:43160) 2024-04-26T23:58:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 -- 172.21.15.46:0/1624911487 learned_addr learned my addr 172.21.15.46:0/1624911487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 -- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb4806e510 msgr2=0x7ffb4813c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb4806e510 0x7ffb4813c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 -- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb48160bd0 msgr2=0x7ffb4815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb48160bd0 0x7ffb4815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 -- 172.21.15.46:0/1624911487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4813cfa0 con 0x7ffb480732c0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.516+0000 7ffb47e00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ee0 tx=0x7ffb38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.517+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38009450 con 0x7ffb480732c0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.517+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb38004250 con 0x7ffb480732c0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.517+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb481403c0 con 0x7ffb480732c0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.517+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38004400 con 0x7ffb480732c0 2024-04-26T23:58:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.517+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4813e9d0 con 0x7ffb480732c0 2024-04-26T23:58:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.518+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb48069530 con 0x7ffb480732c0 2024-04-26T23:58:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.519+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb38039040 con 0x7ffb480732c0 2024-04-26T23:58:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.520+0000 7ffb44c00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 0x7ffb2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.520+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7ffb380acda0 con 0x7ffb480732c0 2024-04-26T23:58:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.520+0000 7ffb4d000640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 0x7ffb2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.522+0000 7ffb4d000640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 0x7ffb2406edb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000f60 tx=0x7ffb300078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.524+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb4813cfa0 con 0x7ffb480732c0 2024-04-26T23:58:47.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.729+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ffb4806fb00 con 0x7ffb2406c8f0 2024-04-26T23:58:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.731+0000 7ffb44c00640 1 -- 172.21.15.46:0/1624911487 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb4806fb00 con 0x7ffb2406c8f0 2024-04-26T23:58:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 msgr2=0x7ffb2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 0x7ffb2406edb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000f60 tx=0x7ffb300078f0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 msgr2=0x7ffb481425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ee0 tx=0x7ffb38004a40 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 shutdown_connections 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ffb2406c8f0 0x7ffb2406edb0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb48160bd0 0x7ffb4815f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb480732c0 0x7ffb481425d0 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 --2- 172.21.15.46:0/1624911487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb4806e510 0x7ffb4813c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.733+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 >> 172.21.15.46:0/1624911487 conn(0x7ffb4806c530 msgr2=0x7ffb48073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.734+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 shutdown_connections 2024-04-26T23:58:47.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.734+0000 7ffb4fa00640 1 -- 172.21.15.46:0/1624911487 wait complete. 2024-04-26T23:58:47.754 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T23:58:47.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.969+0000 7f84aa000640 1 Processor -- start 2024-04-26T23:58:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.969+0000 7f84aa000640 1 -- start start 2024-04-26T23:58:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.970+0000 7f84aa000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84a406ca60 0x7f84a415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84aa000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a4071900 con 0x7f84a41595b0 2024-04-26T23:58:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84aa000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a41602c0 con 0x7f84a4159990 2024-04-26T23:58:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84a8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84a41595b0 0x7f84a406fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41360/0 (socket says 172.21.15.46:41360) 2024-04-26T23:58:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84a8c00640 1 -- 172.21.15.46:0/433075517 learned_addr learned my addr 172.21.15.46:0/433075517 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f84a406f600 con 0x7f84a406ca60 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84a9600640 1 --2- 172.21.15.46:0/433075517 >> v2:172.21.15.46:3300/0 conn(0x7f84a406ca60 0x7f84a415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84a3400640 1 -- 172.21.15.46:0/433075517 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800650347 0 0) 0x7f84a4071900 con 0x7f84a41595b0 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.971+0000 7f84a3400640 1 -- 172.21.15.46:0/433075517 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8488003520 con 0x7f84a41595b0 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.972+0000 7f84a9600640 1 -- 172.21.15.46:0/433075517 >> v1:172.21.15.142:6789/0 conn(0x7f84a4159990 legacy=0x7f84a406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.972+0000 7f84a9600640 1 -- 172.21.15.46:0/433075517 >> v1:172.21.15.46:6789/0 conn(0x7f84a41595b0 legacy=0x7f84a406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.972+0000 7f84a9600640 1 -- 172.21.15.46:0/433075517 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a41605a0 con 0x7f84a406ca60 2024-04-26T23:58:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84a9600640 1 --2- 172.21.15.46:0/433075517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a406ca60 0x7f84a415f6e0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f849c0097a0 tx=0x7f849c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644e5202129c69b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84a3400640 1 -- 172.21.15.46:0/433075517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c037070 con 0x7f84a406ca60 2024-04-26T23:58:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84a3400640 1 -- 172.21.15.46:0/433075517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849c032440 con 0x7f84a406ca60 2024-04-26T23:58:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84a3400640 1 -- 172.21.15.46:0/433075517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c032810 con 0x7f84a406ca60 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a406ca60 msgr2=0x7f84a415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.973+0000 7f84aa000640 1 --2- 172.21.15.46:0/433075517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a406ca60 0x7f84a415f6e0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f849c0097a0 tx=0x7f849c02a230 comp rx=0 tx=0).stop 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.974+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 shutdown_connections 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.974+0000 7f84aa000640 1 --2- 172.21.15.46:0/433075517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a406ca60 0x7f84a415f6e0 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.974+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 >> 172.21.15.46:0/433075517 conn(0x7f84a40b38c0 msgr2=0x7f84a40b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.974+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 shutdown_connections 2024-04-26T23:58:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.974+0000 7f84aa000640 1 -- 172.21.15.46:0/433075517 wait complete. 2024-04-26T23:58:47.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.976+0000 7f84aa000640 1 Processor -- start 2024-04-26T23:58:47.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.976+0000 7f84aa000640 1 -- start start 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a406ca60 0x7f84a40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 0x7f84a4145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a4159990 0x7f84a4153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84a415fc20 con 0x7f84a41595b0 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84a406f290 con 0x7f84a406ca60 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.977+0000 7f84aa000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84a415fda0 con 0x7f84a4159990 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 0x7f84a4145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 0x7f84a4145a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43176/0 (socket says 172.21.15.46:43176) 2024-04-26T23:58:47.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 -- 172.21.15.46:0/1722196573 learned_addr learned my addr 172.21.15.46:0/1722196573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a8c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a406ca60 0x7f84a40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a9600640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a4159990 0x7f84a4153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 -- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a4159990 msgr2=0x7f84a4153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a4159990 0x7f84a4153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 -- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a406ca60 msgr2=0x7f84a40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a406ca60 0x7f84a40ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a40cec20 con 0x7f84a41595b0 2024-04-26T23:58:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a8c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a406ca60 0x7f84a40ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.978+0000 7f84a3e00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 0x7f84a4145a50 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c3f0 tx=0x7f848c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.979+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c017070 con 0x7f84a41595b0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.979+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c0052f0 con 0x7f84a41595b0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.979+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c003830 con 0x7f84a41595b0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.979+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a4153560 con 0x7f84a41595b0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.979+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a4146720 con 0x7f84a41595b0 2024-04-26T23:58:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.980+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84a4069ac0 con 0x7f84a41595b0 2024-04-26T23:58:47.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.981+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f848c01a070 con 0x7f84a41595b0 2024-04-26T23:58:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.982+0000 7f84a0c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848006c8f0 0x7f848006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.982+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f848c08ad70 con 0x7f84a41595b0 2024-04-26T23:58:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.982+0000 7f84a0c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f8480072e30 0x7f8480075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.982+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f848c0054a0 con 0x7f8480072e30 2024-04-26T23:58:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.982+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f848c08d470 con 0x7f84a41595b0 2024-04-26T23:58:47.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.983+0000 7f84a8c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848006c8f0 0x7f848006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.983+0000 7f84a9600640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f8480072e30 0x7f8480075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:47.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.984+0000 7f84a8c00640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848006c8f0 0x7f848006edb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8494001560 tx=0x7f8494009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.984+0000 7f84a9600640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f8480072e30 0x7f8480075270 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:47.986+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== osd.6 v2:172.21.15.142:6800/715004401 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f848c0054a0 con 0x7f8480072e30 2024-04-26T23:58:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.001+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 --> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f84a406c3f0 con 0x7f8480072e30 2024-04-26T23:58:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84a0c00640 1 -- 172.21.15.46:0/1722196573 <== osd.6 v2:172.21.15.142:6800/715004401 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f84a406c3f0 con 0x7f8480072e30 2024-04-26T23:58:48.003 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:48.003 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f8480072e30 msgr2=0x7f8480075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6800/715004401,v1:172.21.15.142:6803/715004401] conn(0x7f8480072e30 0x7f8480075270 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848006c8f0 msgr2=0x7f848006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f848006c8f0 0x7f848006edb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8494001560 tx=0x7f8494009040 comp rx=0 tx=0).stop 2024-04-26T23:58:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 msgr2=0x7f84a4145a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 --2- 172.21.15.46:0/1722196573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a41595b0 0x7f84a4145a50 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c3f0 tx=0x7f848c00c8b0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84a9600640 1 -- 172.21.15.46:0/1722196573 reap_dead start 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 shutdown_connections 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.002+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 >> 172.21.15.46:0/1722196573 conn(0x7f84a40b38c0 msgr2=0x7f84a40d2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.003+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 shutdown_connections 2024-04-26T23:58:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.003+0000 7f84aa000640 1 -- 172.21.15.46:0/1722196573 wait complete. 2024-04-26T23:58:48.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T23:58:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.221+0000 7fde4e200640 1 Processor -- start 2024-04-26T23:58:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.222+0000 7fde4e200640 1 -- start start 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde480cc690 0x7fde480cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde48072bb0 con 0x7fde480c6a50 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4e200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde48072e90 con 0x7fde481454a0 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fde48073170 con 0x7fde480cc690 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde47e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde480cc690 0x7fde480cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fde480c6a50 0x7fde480cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41368/0 (socket says 172.21.15.46:41368) 2024-04-26T23:58:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.223+0000 7fde4d800640 1 -- 172.21.15.46:0/4089646516 learned_addr learned my addr 172.21.15.46:0/4089646516 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.224+0000 7fde47e00640 1 -- 172.21.15.46:0/4089646516 >> v1:172.21.15.142:6789/0 conn(0x7fde481454a0 legacy=0x7fde48073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.224+0000 7fde47e00640 1 -- 172.21.15.46:0/4089646516 >> v1:172.21.15.46:6789/0 conn(0x7fde480c6a50 legacy=0x7fde480cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.224+0000 7fde47e00640 1 -- 172.21.15.46:0/4089646516 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde48073450 con 0x7fde480cc690 2024-04-26T23:58:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.225+0000 7fde47e00640 1 --2- 172.21.15.46:0/4089646516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480cdc30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fde340097a0 tx=0x7fde3402c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af2158ecb0ea1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.225+0000 7fde47400640 1 -- 172.21.15.46:0/4089646516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34039070 con 0x7fde480cc690 2024-04-26T23:58:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.225+0000 7fde47400640 1 -- 172.21.15.46:0/4089646516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde34034640 con 0x7fde480cc690 2024-04-26T23:58:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.225+0000 7fde47400640 1 -- 172.21.15.46:0/4089646516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34034a10 con 0x7fde480cc690 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 -- 172.21.15.46:0/4089646516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 msgr2=0x7fde480cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 --2- 172.21.15.46:0/4089646516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480cdc30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fde340097a0 tx=0x7fde3402c3f0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 -- 172.21.15.46:0/4089646516 shutdown_connections 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 --2- 172.21.15.46:0/4089646516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480cdc30 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 -- 172.21.15.46:0/4089646516 >> 172.21.15.46:0/4089646516 conn(0x7fde480b26d0 msgr2=0x7fde480b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 -- 172.21.15.46:0/4089646516 shutdown_connections 2024-04-26T23:58:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.226+0000 7fde4e200640 1 -- 172.21.15.46:0/4089646516 wait complete. 2024-04-26T23:58:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.228+0000 7fde4e200640 1 Processor -- start 2024-04-26T23:58:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.228+0000 7fde4e200640 1 -- start start 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde480c6a50 0x7fde480b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde481454a0 0x7fde480caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde48145880 con 0x7fde480cc690 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde48072880 con 0x7fde481454a0 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde4e200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fde48145a00 con 0x7fde480c6a50 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43192/0 (socket says 172.21.15.46:43192) 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.229+0000 7fde47e00640 1 -- 172.21.15.46:0/2831910837 learned_addr learned my addr 172.21.15.46:0/2831910837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 -- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde480c6a50 msgr2=0x7fde480b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde480c6a50 0x7fde480b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 -- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde481454a0 msgr2=0x7fde480caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde481454a0 0x7fde480caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 -- 172.21.15.46:0/2831910837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde480cb370 con 0x7fde480cc690 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde47e00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.230+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34039070 con 0x7fde480cc690 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.231+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde34032980 con 0x7fde480cc690 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.231+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34032d70 con 0x7fde480cc690 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.231+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde480cb650 con 0x7fde480cc690 2024-04-26T23:58:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.231+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde480b3e60 con 0x7fde480cc690 2024-04-26T23:58:48.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.232+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde480693b0 con 0x7fde480cc690 2024-04-26T23:58:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.233+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde3403f440 con 0x7fde480cc690 2024-04-26T23:58:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.234+0000 7fde44c00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 0x7fde2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.234+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fde340ac820 con 0x7fde480cc690 2024-04-26T23:58:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.234+0000 7fde4ce00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 0x7fde2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.235+0000 7fde4ce00640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 0x7fde2806edb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.238+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde3407adf0 con 0x7fde480cc690 2024-04-26T23:58:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.444+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fde480cefa0 con 0x7fde2806c8f0 2024-04-26T23:58:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.445+0000 7fde44c00640 1 -- 172.21.15.46:0/2831910837 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fde480cefa0 con 0x7fde2806c8f0 2024-04-26T23:58:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T23:58:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.447+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 msgr2=0x7fde2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.447+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 0x7fde2806edb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006bc0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.447+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 msgr2=0x7fde480ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde340046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 shutdown_connections 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fde2806c8f0 0x7fde2806edb0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde481454a0 0x7fde480caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde480cc690 0x7fde480ca5a0 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 --2- 172.21.15.46:0/2831910837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde480c6a50 0x7fde480b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 >> 172.21.15.46:0/2831910837 conn(0x7fde480b26d0 msgr2=0x7fde480afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 shutdown_connections 2024-04-26T23:58:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.448+0000 7fde4e200640 1 -- 172.21.15.46:0/2831910837 wait complete. 2024-04-26T23:58:48.468 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T23:58:48.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:58:48.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.683+0000 7f41cfe00640 1 Processor -- start 2024-04-26T23:58:48.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.684+0000 7f41cfe00640 1 -- start start 2024-04-26T23:58:48.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.685+0000 7f41cfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41d01627f0 0x7f41d00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.685+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.685+0000 7f41cfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d00d01a0 con 0x7f41d00d2080 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.685+0000 7f41cfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f41d00d0480 con 0x7f41d01627f0 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.685+0000 7f41cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00d0760 con 0x7f41d00d3cc0 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.686+0000 7f41cea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41d01627f0 0x7f41d00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.686+0000 7f41cea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41d01627f0 0x7f41d00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:52422/0 (socket says 172.21.15.46:52422) 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.686+0000 7f41cea00640 1 -- 172.21.15.46:0/2364913122 learned_addr learned my addr 172.21.15.46:0/2364913122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:48.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.686+0000 7f41cf400640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.687+0000 7f41cea00640 1 -- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 msgr2=0x7f41d00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:48.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.687+0000 7f41cea00640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.687+0000 7f41cea00640 1 -- 172.21.15.46:0/2364913122 >> v1:172.21.15.142:6789/0 conn(0x7f41d00d2080 legacy=0x7f41d00d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.687+0000 7f41cea00640 1 -- 172.21.15.46:0/2364913122 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d00d0a40 con 0x7f41d01627f0 2024-04-26T23:58:48.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.688+0000 7f41cea00640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d01627f0 0x7f41d00cfbc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0232dcecab83f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.689+0000 7f41cd600640 1 -- 172.21.15.46:0/2364913122 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037070 con 0x7f41d01627f0 2024-04-26T23:58:48.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.689+0000 7f41cd600640 1 -- 172.21.15.46:0/2364913122 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0032640 con 0x7f41d01627f0 2024-04-26T23:58:48.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.689+0000 7f41cd600640 1 -- 172.21.15.46:0/2364913122 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0032a10 con 0x7f41d01627f0 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.689+0000 7f41cfe00640 1 -- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d01627f0 msgr2=0x7f41d00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.689+0000 7f41cfe00640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d01627f0 0x7f41d00cfbc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a2f0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 -- 172.21.15.46:0/2364913122 shutdown_connections 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 --2- 172.21.15.46:0/2364913122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d01627f0 0x7f41d00cfbc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 -- 172.21.15.46:0/2364913122 >> 172.21.15.46:0/2364913122 conn(0x7f41d009e380 msgr2=0x7f41d00d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 -- 172.21.15.46:0/2364913122 shutdown_connections 2024-04-26T23:58:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.690+0000 7f41cfe00640 1 -- 172.21.15.46:0/2364913122 wait complete. 2024-04-26T23:58:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.692+0000 7f41cfe00640 1 Processor -- start 2024-04-26T23:58:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.692+0000 7f41cfe00640 1 -- start start 2024-04-26T23:58:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d00d2080 0x7f41d0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 0x7f41d00b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0162bd0 con 0x7f41d01627f0 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41d00d24f0 con 0x7f41d00d2080 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cfe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00b4a90 con 0x7f41d00d3cc0 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 0x7f41d00b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 0x7f41d00b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43208/0 (socket says 172.21.15.46:43208) 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cf400640 1 -- 172.21.15.46:0/4248667534 learned_addr learned my addr 172.21.15.46:0/4248667534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.694+0000 7f41cea00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d00d2080 0x7f41d0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41ce000640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 -- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 msgr2=0x7f41d00b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41d00d3cc0 0x7f41d00b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 -- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d00d2080 msgr2=0x7f41d0159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41d00d2080 0x7f41d0159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d00b90a0 con 0x7f41d01627f0 2024-04-26T23:58:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cf400640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 0x7f41d00b5fc0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e850 tx=0x7f41c800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8017070 con 0x7f41d01627f0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c800c8e0 con 0x7f41d01627f0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d00b9380 con 0x7f41d01627f0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.695+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800ccd0 con 0x7f41d01627f0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.696+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d00b9940 con 0x7f41d01627f0 2024-04-26T23:58:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.696+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41d00693b0 con 0x7f41d01627f0 2024-04-26T23:58:48.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.698+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41c801a070 con 0x7f41d01627f0 2024-04-26T23:58:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.699+0000 7f41c6a00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41ac06c8f0 0x7f41ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.699+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f41c808c340 con 0x7f41d01627f0 2024-04-26T23:58:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.699+0000 7f41c6a00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7f41ac072e30 0x7f41ac075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.699+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41c8026e50 con 0x7f41ac072e30 2024-04-26T23:58:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.700+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f41c808c6f0 con 0x7f41d01627f0 2024-04-26T23:58:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.700+0000 7f41cea00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41ac06c8f0 0x7f41ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.700+0000 7f41ce000640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7f41ac072e30 0x7f41ac075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.701+0000 7f41cea00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41ac06c8f0 0x7f41ac06edb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.702+0000 7f41ce000640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7f41ac072e30 0x7f41ac075270 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.704+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== osd.7 v2:172.21.15.142:6816/631850901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41c8026e50 con 0x7f41ac072e30 2024-04-26T23:58:48.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.718+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 --> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41d00d56e0 con 0x7f41ac072e30 2024-04-26T23:58:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.719+0000 7f41c6a00640 1 -- 172.21.15.46:0/4248667534 <== osd.7 v2:172.21.15.142:6816/631850901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41d00d56e0 con 0x7f41ac072e30 2024-04-26T23:58:48.722 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-26T23:58:48.722 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:58:48.722 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7f41ac072e30 msgr2=0x7f41ac075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6816/631850901,v1:172.21.15.142:6817/631850901] conn(0x7f41ac072e30 0x7f41ac075270 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41ac06c8f0 msgr2=0x7f41ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f41ac06c8f0 0x7f41ac06edb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).stop 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 msgr2=0x7f41d00b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 --2- 172.21.15.46:0/4248667534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41d01627f0 0x7f41d00b5fc0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e850 tx=0x7f41c800ed10 comp rx=0 tx=0).stop 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cf400640 1 -- 172.21.15.46:0/4248667534 reap_dead start 2024-04-26T23:58:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 shutdown_connections 2024-04-26T23:58:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.720+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 >> 172.21.15.46:0/4248667534 conn(0x7f41d009e380 msgr2=0x7f41d00d5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.721+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 shutdown_connections 2024-04-26T23:58:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.721+0000 7f41cfe00640 1 -- 172.21.15.46:0/4248667534 wait complete. 2024-04-26T23:58:48.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T23:58:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.950+0000 7f51f6200640 1 Processor -- start 2024-04-26T23:58:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.950+0000 7f51f6200640 1 -- start start 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.952+0000 7f51f6200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51f0160bd0 0x7f51f0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.952+0000 7f51f6200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f0161940 con 0x7f51f006e510 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.952+0000 7f51f6200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f0071f80 con 0x7f51f00732c0 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.952+0000 7f51f6200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f51f0072260 con 0x7f51f0160bd0 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51ef400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f51f006e510 0x7f51f006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41378/0 (socket says 172.21.15.46:41378) 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51ef400640 1 -- 172.21.15.46:0/3347353277 learned_addr learned my addr 172.21.15.46:0/3347353277 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51eea00640 1 --2- 172.21.15.46:0/3347353277 >> v2:172.21.15.142:3300/0 conn(0x7f51f0160bd0 0x7f51f0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51ee000640 1 -- 172.21.15.46:0/3347353277 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855025759 0 0) 0x7f51f0161940 con 0x7f51f006e510 2024-04-26T23:58:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51ee000640 1 -- 172.21.15.46:0/3347353277 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d4003520 con 0x7f51f006e510 2024-04-26T23:58:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51eea00640 1 -- 172.21.15.46:0/3347353277 >> v1:172.21.15.142:6789/0 conn(0x7f51f00732c0 legacy=0x7f51f0073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51eea00640 1 -- 172.21.15.46:0/3347353277 >> v1:172.21.15.46:6789/0 conn(0x7f51f006e510 legacy=0x7f51f006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.953+0000 7f51eea00640 1 -- 172.21.15.46:0/3347353277 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f0072540 con 0x7f51f0160bd0 2024-04-26T23:58:48.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.955+0000 7f51eea00640 1 --2- 172.21.15.46:0/3347353277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f0161060 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0097d0 tx=0x7f51dc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d6216e8d0491e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.955+0000 7f51ee000640 1 -- 172.21.15.46:0/3347353277 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc037070 con 0x7f51f0160bd0 2024-04-26T23:58:48.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.955+0000 7f51ee000640 1 -- 172.21.15.46:0/3347353277 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc032440 con 0x7f51f0160bd0 2024-04-26T23:58:48.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.955+0000 7f51ee000640 1 -- 172.21.15.46:0/3347353277 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0327f0 con 0x7f51f0160bd0 2024-04-26T23:58:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 -- 172.21.15.46:0/3347353277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 msgr2=0x7f51f0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 --2- 172.21.15.46:0/3347353277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f0161060 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0097d0 tx=0x7f51dc02a250 comp rx=0 tx=0).stop 2024-04-26T23:58:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 -- 172.21.15.46:0/3347353277 shutdown_connections 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 --2- 172.21.15.46:0/3347353277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f0161060 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 -- 172.21.15.46:0/3347353277 >> 172.21.15.46:0/3347353277 conn(0x7f51f006c530 msgr2=0x7f51f006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 -- 172.21.15.46:0/3347353277 shutdown_connections 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.956+0000 7f51f6200640 1 -- 172.21.15.46:0/3347353277 wait complete. 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.958+0000 7f51f6200640 1 Processor -- start 2024-04-26T23:58:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.958+0000 7f51f6200640 1 -- start start 2024-04-26T23:58:48.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f006e510 0x7f51f0158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f00af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0161720 con 0x7f51f00732c0 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01615a0 con 0x7f51f0160bd0 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51f6200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51f0072d40 con 0x7f51f006e510 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43214/0 (socket says 172.21.15.46:43214) 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 -- 172.21.15.46:0/3891524019 learned_addr learned my addr 172.21.15.46:0/3891524019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 -- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f006e510 msgr2=0x7f51f0158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51efe00640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f00af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f006e510 0x7f51f0158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 -- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 msgr2=0x7f51f00af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f00af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:48.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51eea00640 1 -- 172.21.15.46:0/3891524019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f00ae8e0 con 0x7f51f00732c0 2024-04-26T23:58:48.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.959+0000 7f51efe00640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:58:48.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51eea00640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0097d0 tx=0x7f51dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc037050 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc038040 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f00aebc0 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004470 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.960+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f00af180 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.961+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f0069530 con 0x7f51f00732c0 2024-04-26T23:58:48.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.962+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51dc03c6d0 con 0x7f51f00732c0 2024-04-26T23:58:48.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.963+0000 7f51e7400640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:48.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.963+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f51dc0ac420 con 0x7f51f00732c0 2024-04-26T23:58:48.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.964+0000 7f51ef400640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.965+0000 7f51ef400640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001610 tx=0x7f51e0007930 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:48.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:48.967+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51dc07aa00 con 0x7f51f00732c0 2024-04-26T23:58:49.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.166+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f51f015c930 con 0x7f51d006c8f0 2024-04-26T23:58:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.167+0000 7f51e7400640 1 -- 172.21.15.46:0/3891524019 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f51f015c930 con 0x7f51d006c8f0 2024-04-26T23:58:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.169+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 msgr2=0x7f51d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.169+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001610 tx=0x7f51e0007930 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.169+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 msgr2=0x7f51f013d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.169+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0097d0 tx=0x7f51dc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 shutdown_connections 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f0160bd0 0x7f51f00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f00732c0 0x7f51f013d600 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 --2- 172.21.15.46:0/3891524019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f006e510 0x7f51f0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 >> 172.21.15.46:0/3891524019 conn(0x7f51f006c530 msgr2=0x7f51f015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 shutdown_connections 2024-04-26T23:58:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.170+0000 7f51f6200640 1 -- 172.21.15.46:0/3891524019 wait complete. 2024-04-26T23:58:49.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:58:49.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.401+0000 7fca17000640 1 Processor -- start 2024-04-26T23:58:49.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.402+0000 7fca17000640 1 -- start start 2024-04-26T23:58:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.403+0000 7fca17000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca1015f4b0 0x7fca100d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.403+0000 7fca17000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100b5140 0x7fca100d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.403+0000 7fca17000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca100cfd60 con 0x7fca100d5410 2024-04-26T23:58:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.403+0000 7fca17000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fca1015f890 con 0x7fca1015f4b0 2024-04-26T23:58:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.403+0000 7fca17000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca100d2910 con 0x7fca100b5140 2024-04-26T23:58:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca15000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca100d5410 0x7fca10162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41388/0 (socket says 172.21.15.46:41388) 2024-04-26T23:58:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca15000640 1 -- 172.21.15.46:0/2685837153 learned_addr learned my addr 172.21.15.46:0/2685837153 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2685837153 >> v2:172.21.15.46:3300/0 conn(0x7fca1015f4b0 0x7fca100d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca07e00640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100b5140 0x7fca100d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0f400640 1 -- 172.21.15.46:0/2685837153 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1174793877 0 0) 0x7fca08001490 con 0x7fca100d5410 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0f400640 1 -- 172.21.15.46:0/2685837153 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9f4003520 con 0x7fca100d5410 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0fe00640 1 -- 172.21.15.46:0/2685837153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100b5140 msgr2=0x7fca100d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100b5140 0x7fca100d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0fe00640 1 -- 172.21.15.46:0/2685837153 >> v1:172.21.15.46:6789/0 conn(0x7fca100d5410 legacy=0x7fca10162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.404+0000 7fca0fe00640 1 -- 172.21.15.46:0/2685837153 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca100d58a0 con 0x7fca1015f4b0 2024-04-26T23:58:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.405+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca1015f4b0 0x7fca100d23d0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fca000097a0 tx=0x7fca0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d986f6e8715fc853 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.406+0000 7fca0f400640 1 -- 172.21.15.46:0/2685837153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00037070 con 0x7fca1015f4b0 2024-04-26T23:58:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.406+0000 7fca0f400640 1 -- 172.21.15.46:0/2685837153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca00032440 con 0x7fca1015f4b0 2024-04-26T23:58:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.406+0000 7fca0f400640 1 -- 172.21.15.46:0/2685837153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000327f0 con 0x7fca1015f4b0 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.406+0000 7fca17000640 1 -- 172.21.15.46:0/2685837153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca1015f4b0 msgr2=0x7fca100d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.406+0000 7fca17000640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca1015f4b0 0x7fca100d23d0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fca000097a0 tx=0x7fca0002a180 comp rx=0 tx=0).stop 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 -- 172.21.15.46:0/2685837153 shutdown_connections 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100b5140 0x7fca100d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 --2- 172.21.15.46:0/2685837153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca1015f4b0 0x7fca100d23d0 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 -- 172.21.15.46:0/2685837153 >> 172.21.15.46:0/2685837153 conn(0x7fca1008eb50 msgr2=0x7fca1009e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 -- 172.21.15.46:0/2685837153 shutdown_connections 2024-04-26T23:58:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.407+0000 7fca17000640 1 -- 172.21.15.46:0/2685837153 wait complete. 2024-04-26T23:58:49.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.408+0000 7fca17000640 1 Processor -- start 2024-04-26T23:58:49.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 -- start start 2024-04-26T23:58:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100d5410 0x7fca1015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1015f4b0 0x7fca1015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca100b5b90 con 0x7fca100b92f0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca100b58b0 con 0x7fca1015f4b0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.409+0000 7fca17000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca100b5e70 con 0x7fca100d5410 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43232/0 (socket says 172.21.15.46:43232) 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 -- 172.21.15.46:0/2290760534 learned_addr learned my addr 172.21.15.46:0/2290760534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100d5410 0x7fca1015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca07e00640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1015f4b0 0x7fca1015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 -- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100d5410 msgr2=0x7fca1015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100d5410 0x7fca1015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 -- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1015f4b0 msgr2=0x7fca1015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1015f4b0 0x7fca1015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.410+0000 7fca15000640 1 -- 172.21.15.46:0/2290760534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca10160310 con 0x7fca100b92f0 2024-04-26T23:58:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca15000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fca0800cb50 tx=0x7fca0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:49.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08017070 con 0x7fca100b92f0 2024-04-26T23:58:49.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0800ad40 con 0x7fca100b92f0 2024-04-26T23:58:49.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08012620 con 0x7fca100b92f0 2024-04-26T23:58:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca101605f0 con 0x7fca100b92f0 2024-04-26T23:58:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.411+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca10160bb0 con 0x7fca100b92f0 2024-04-26T23:58:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.412+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca10069530 con 0x7fca100b92f0 2024-04-26T23:58:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.413+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca0801a070 con 0x7fca100b92f0 2024-04-26T23:58:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.414+0000 7fca0cc00640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 0x7fc9ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.415+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fca0808b7b0 con 0x7fca100b92f0 2024-04-26T23:58:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.415+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 0x7fc9ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:49.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.416+0000 7fca0fe00640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 0x7fc9ec06ee00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fca00034000 tx=0x7fca0002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.419+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca100cfd60 con 0x7fca100b92f0 2024-04-26T23:58:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.625+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca100b4bd0 con 0x7fc9ec06c940 2024-04-26T23:58:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.628+0000 7fca0cc00640 1 -- 172.21.15.46:0/2290760534 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13377 (secure 0 0 0) 0x7fca100b4bd0 con 0x7fc9ec06c940 2024-04-26T23:58:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:58:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.630+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 msgr2=0x7fc9ec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.630+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 0x7fc9ec06ee00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fca00034000 tx=0x7fca0002e000 comp rx=0 tx=0).stop 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 msgr2=0x7fca10161c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fca0800cb50 tx=0x7fca0800a430 comp rx=0 tx=0).stop 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 shutdown_connections 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fc9ec06c940 0x7fc9ec06ee00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca100b92f0 0x7fca10161c40 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1015f4b0 0x7fca1015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 --2- 172.21.15.46:0/2290760534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca100d5410 0x7fca1015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 >> 172.21.15.46:0/2290760534 conn(0x7fca1008eb50 msgr2=0x7fca100d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 shutdown_connections 2024-04-26T23:58:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:58:49.631+0000 7fca17000640 1 -- 172.21.15.46:0/2290760534 wait complete. 2024-04-26T23:58:49.651 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":863,"stamp":"2024-04-26T23:58:48.379723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":437212,"kb_used_data":22568,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754537508,"statfs":{"total":773094113280,"available":772646408192,"internally_reserved":0,"allocated":23109632,"data_stored":11551448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.131807"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":536,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-04-26T23:58:37.253422+0000","last_change":"2024-04-26T23:37:03.662460+0000","last_active":"2024-04-26T23:58:37.253422+0000","last_peered":"2024-04-26T23:58:37.253422+0000","last_clean":"2024-04-26T23:58:37.253422+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:58:37.253422+0000","last_undegraded":"2024-04-26T23:58:37.253422+0000","last_fullsized":"2024-04-26T23:58:37.253422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:37:02.639666+0000","last_clean_scrub_stamp":"2024-04-26T23:37:02.639666+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:17.680568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56652,"kb_used_data":3256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315188,"statfs":{"total":96636764160,"available":96578752512,"internally_reserved":0,"allocated":3334144,"data_stored":1886707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575112,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56072,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315768,"statfs":{"total":96636764160,"available":96579346432,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575112,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56064,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575111,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60224,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311616,"statfs":{"total":96636764160,"available":96575094784,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575112,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575112,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575112,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575112,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64968,"kb_used_data":3256,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306872,"statfs":{"total":96636764160,"available":96570236928,"internally_reserved":0,"allocated":3334144,"data_stored":1886707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:58:49.653 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T23:37:02.639666+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=23, tm_min=37, tm_sec=2, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=23, tm_min=58, tm_sec=42, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T23:58:49.654 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T23:59:09.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:59:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.868+0000 7ff48f200640 1 Processor -- start 2024-04-26T23:59:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.869+0000 7ff48f200640 1 -- start start 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.870+0000 7ff48f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48815f4e0 0x7ff4880d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.870+0000 7ff48f200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4880b4000 0x7ff4880d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.870+0000 7ff48f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4880d11f0 con 0x7ff4880d1f60 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.870+0000 7ff48f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff4880d2990 con 0x7ff48815f4e0 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.870+0000 7ff48f200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff4880d5f20 con 0x7ff4880b4000 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48815f4e0 0x7ff4880d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48815f4e0 0x7ff4880d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42220/0 (socket says 172.21.15.46:42220) 2024-04-26T23:59:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 -- 172.21.15.46:0/4241905902 learned_addr learned my addr 172.21.15.46:0/4241905902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48d400640 1 --2- 172.21.15.46:0/4241905902 >> v2:172.21.15.142:3300/0 conn(0x7ff4880b4000 0x7ff4880d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 -- 172.21.15.46:0/4241905902 >> v2:172.21.15.142:3300/0 conn(0x7ff4880b4000 msgr2=0x7ff4880d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 --2- 172.21.15.46:0/4241905902 >> v2:172.21.15.142:3300/0 conn(0x7ff4880b4000 0x7ff4880d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.871+0000 7ff48de00640 1 -- 172.21.15.46:0/4241905902 >> v1:172.21.15.142:6789/0 conn(0x7ff4880d1f60 legacy=0x7ff4880cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.872+0000 7ff48de00640 1 -- 172.21.15.46:0/4241905902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4880d0f10 con 0x7ff48815f4e0 2024-04-26T23:59:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.873+0000 7ff48de00640 1 --2- 172.21.15.46:0/4241905902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff4880d2df0 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7ff48808d880 tx=0x7ff47c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd00f624c03f36d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.874+0000 7ff48ca00640 1 -- 172.21.15.46:0/4241905902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c037070 con 0x7ff48815f4e0 2024-04-26T23:59:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.874+0000 7ff48ca00640 1 -- 172.21.15.46:0/4241905902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c032440 con 0x7ff48815f4e0 2024-04-26T23:59:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.874+0000 7ff48ca00640 1 -- 172.21.15.46:0/4241905902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c032810 con 0x7ff48815f4e0 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 -- 172.21.15.46:0/4241905902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 msgr2=0x7ff4880d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 --2- 172.21.15.46:0/4241905902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff4880d2df0 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7ff48808d880 tx=0x7ff47c02a230 comp rx=0 tx=0).stop 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 -- 172.21.15.46:0/4241905902 shutdown_connections 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 --2- 172.21.15.46:0/4241905902 >> v2:172.21.15.142:3300/0 conn(0x7ff4880b4000 0x7ff4880d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 --2- 172.21.15.46:0/4241905902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff4880d2df0 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 -- 172.21.15.46:0/4241905902 >> 172.21.15.46:0/4241905902 conn(0x7ff4880d5010 msgr2=0x7ff4880d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 -- 172.21.15.46:0/4241905902 shutdown_connections 2024-04-26T23:59:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.875+0000 7ff48f200640 1 -- 172.21.15.46:0/4241905902 wait complete. 2024-04-26T23:59:09.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.877+0000 7ff48f200640 1 Processor -- start 2024-04-26T23:59:09.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.878+0000 7ff48f200640 1 -- start start 2024-04-26T23:59:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4880b4000 0x7ff4880ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4880d1f60 0x7ff48809ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff48815f8c0 con 0x7ff48815f4e0 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4880b1d60 con 0x7ff4880b4000 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4880b3290 con 0x7ff4880d1f60 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42226/0 (socket says 172.21.15.46:42226) 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48e800640 1 -- 172.21.15.46:0/3248038504 learned_addr learned my addr 172.21.15.46:0/3248038504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48de00640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4880b4000 0x7ff4880ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.879+0000 7ff48d400640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4880d1f60 0x7ff48809ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 -- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4880d1f60 msgr2=0x7ff48809ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4880d1f60 0x7ff48809ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 -- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4880b4000 msgr2=0x7ff4880ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4880b4000 0x7ff4880ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 -- 172.21.15.46:0/3248038504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4880a18c0 con 0x7ff48815f4e0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48e800640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c780 tx=0x7ff48400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484004b60 con 0x7ff48815f4e0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff484004d00 con 0x7ff48815f4e0 2024-04-26T23:59:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400aa30 con 0x7ff48815f4e0 2024-04-26T23:59:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.880+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4880a1ba0 con 0x7ff48815f4e0 2024-04-26T23:59:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.881+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4880a2160 con 0x7ff48815f4e0 2024-04-26T23:59:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.882+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4880693b0 con 0x7ff48815f4e0 2024-04-26T23:59:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.883+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48400abd0 con 0x7ff48815f4e0 2024-04-26T23:59:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.884+0000 7ff482000640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 0x7ff46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.884+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7ff48408bbe0 con 0x7ff48815f4e0 2024-04-26T23:59:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.885+0000 7ff48de00640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 0x7ff46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:09.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.886+0000 7ff48de00640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 0x7ff46c06edb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff47c034000 tx=0x7ff47c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:09.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:09.889+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff484020070 con 0x7ff48815f4e0 2024-04-26T23:59:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.094+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 --> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4880b7870 con 0x7ff46c06c8f0 2024-04-26T23:59:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.096+0000 7ff482000640 1 -- 172.21.15.46:0/3248038504 <== mgr.4111 v2:172.21.15.142:6832/1556001348 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13387 (secure 0 0 0) 0x7ff4880b7870 con 0x7ff46c06c8f0 2024-04-26T23:59:10.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:59:10.099 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.098+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 msgr2=0x7ff46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.098+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 0x7ff46c06edb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff47c034000 tx=0x7ff47c02e000 comp rx=0 tx=0).stop 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.098+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 msgr2=0x7ff488152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.098+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c780 tx=0x7ff48400cc40 comp rx=0 tx=0).stop 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 shutdown_connections 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7ff46c06c8f0 0x7ff46c06edb0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48815f4e0 0x7ff488152da0 unknown :-1 s=CLOSED pgs=3069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4880d1f60 0x7ff48809ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 --2- 172.21.15.46:0/3248038504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4880b4000 0x7ff4880ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 >> 172.21.15.46:0/3248038504 conn(0x7ff4880d5010 msgr2=0x7ff4880d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 shutdown_connections 2024-04-26T23:59:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.099+0000 7ff48f200640 1 -- 172.21.15.46:0/3248038504 wait complete. 2024-04-26T23:59:10.119 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":873,"stamp":"2024-04-26T23:59:08.385270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":437192,"kb_used_data":22568,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754537528,"statfs":{"total":773094113280,"available":772646428672,"internally_reserved":0,"allocated":23109632,"data_stored":11551448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"1.0","version":"216'119","reported_seq":548,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-04-26T23:58:49.487901+0000","last_change":"2024-04-26T23:58:49.487901+0000","last_active":"2024-04-26T23:58:49.487901+0000","last_peered":"2024-04-26T23:58:49.487901+0000","last_clean":"2024-04-26T23:58:49.487901+0000","last_became_active":"2024-04-26T23:37:03.661558+0000","last_became_peered":"2024-04-26T23:37:03.661558+0000","last_unstale":"2024-04-26T23:58:49.487901+0000","last_undegraded":"2024-04-26T23:58:49.487901+0000","last_fullsized":"2024-04-26T23:58:49.487901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"216'119","last_scrub_stamp":"2024-04-26T23:58:49.487644+0000","last_deep_scrub":"216'119","last_deep_scrub_stamp":"2024-04-26T23:58:49.487644+0000","last_clean_scrub_stamp":"2024-04-26T23:58:49.487644+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:19.018488+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575115,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56640,"kb_used_data":3256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315200,"statfs":{"total":96636764160,"available":96578764800,"internally_reserved":0,"allocated":3334144,"data_stored":1886707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575116,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56072,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315768,"statfs":{"total":96636764160,"available":96579346432,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575116,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56064,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60224,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311616,"statfs":{"total":96636764160,"available":96575094784,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575116,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575116,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575116,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47744,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":2740224,"data_stored":1296339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575116,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":3256,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":3334144,"data_stored":1886707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:59:10.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T23:59:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.318+0000 7f1903200640 1 Processor -- start 2024-04-26T23:59:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.318+0000 7f1903200640 1 -- start start 2024-04-26T23:59:10.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.319+0000 7f1903200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f190415cec0 0x7f19040cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.319+0000 7f1903200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1904073d00 0x7f19040b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.319+0000 7f1903200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19040ce7d0 con 0x7f19040cdb30 2024-04-26T23:59:10.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.319+0000 7f1903200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f19040ceab0 con 0x7f190415cec0 2024-04-26T23:59:10.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.319+0000 7f1903200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19040ced90 con 0x7f1904073d00 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f190415cec0 0x7f19040cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f190415cec0 0x7f19040cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42232/0 (socket says 172.21.15.46:42232) 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 -- 172.21.15.46:0/3918719156 learned_addr learned my addr 172.21.15.46:0/3918719156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 -- 172.21.15.46:0/3918719156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1904073d00 msgr2=0x7f19040b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 --2- 172.21.15.46:0/3918719156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1904073d00 0x7f19040b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 -- 172.21.15.46:0/3918719156 >> v1:172.21.15.46:6789/0 conn(0x7f19040cdb30 legacy=0x7f19040735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.320+0000 7f1901e00640 1 -- 172.21.15.46:0/3918719156 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19040cf070 con 0x7f190415cec0 2024-04-26T23:59:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.321+0000 7f1901e00640 1 --2- 172.21.15.46:0/3918719156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190415cec0 0x7f19040cd5f0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f19040b3830 tx=0x7f18f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c191f49fa6864 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1900a00640 1 -- 172.21.15.46:0/3918719156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037070 con 0x7f190415cec0 2024-04-26T23:59:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1900a00640 1 -- 172.21.15.46:0/3918719156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f4032440 con 0x7f190415cec0 2024-04-26T23:59:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1900a00640 1 -- 172.21.15.46:0/3918719156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4032810 con 0x7f190415cec0 2024-04-26T23:59:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 -- 172.21.15.46:0/3918719156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190415cec0 msgr2=0x7f19040cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 --2- 172.21.15.46:0/3918719156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190415cec0 0x7f19040cd5f0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f19040b3830 tx=0x7f18f402a1e0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 -- 172.21.15.46:0/3918719156 shutdown_connections 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 --2- 172.21.15.46:0/3918719156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1904073d00 0x7f19040b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 --2- 172.21.15.46:0/3918719156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190415cec0 0x7f19040cd5f0 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.322+0000 7f1903200640 1 -- 172.21.15.46:0/3918719156 >> 172.21.15.46:0/3918719156 conn(0x7f19040b27b0 msgr2=0x7f19040b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.323+0000 7f1903200640 1 -- 172.21.15.46:0/3918719156 shutdown_connections 2024-04-26T23:59:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.323+0000 7f1903200640 1 -- 172.21.15.46:0/3918719156 wait complete. 2024-04-26T23:59:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.324+0000 7f1903200640 1 Processor -- start 2024-04-26T23:59:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.324+0000 7f1903200640 1 -- start start 2024-04-26T23:59:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040cdb30 0x7f19040c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f190415cec0 0x7f19040c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19040b4080 con 0x7f1904073d00 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f190415d2a0 con 0x7f190415cec0 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.325+0000 7f1903200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19040b4200 con 0x7f19040cdb30 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42236/0 (socket says 172.21.15.46:42236) 2024-04-26T23:59:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 -- 172.21.15.46:0/3032228861 learned_addr learned my addr 172.21.15.46:0/3032228861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 -- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040cdb30 msgr2=0x7f19040c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040cdb30 0x7f19040c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 -- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f190415cec0 msgr2=0x7f19040c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f190415cec0 0x7f19040c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 -- 172.21.15.46:0/3032228861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041557c0 con 0x7f1904073d00 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.326+0000 7f1901e00640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000c00 tx=0x7f18f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.327+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4041070 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.327+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f403c870 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.327+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037040 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.327+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190415ed70 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.327+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190415da70 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.328+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19040693b0 con 0x7f1904073d00 2024-04-26T23:59:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.329+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18f4044070 con 0x7f1904073d00 2024-04-26T23:59:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.330+0000 7f18fa000640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.330+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7f18f40ac650 con 0x7f1904073d00 2024-04-26T23:59:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.330+0000 7f1901400640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.331+0000 7f1901400640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 0x7f18e406edb0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f19040c6850 tx=0x7f18f00079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.334+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18f407ab90 con 0x7f1904073d00 2024-04-26T23:59:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.529+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f190415e450 con 0x7f1904073d00 2024-04-26T23:59:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.535+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v54)=0 v54) ==== 123+0+0 (secure 0 0 0) 0x7f18f407a490 con 0x7f1904073d00 2024-04-26T23:59:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.536+0000 7f18fa000640 1 -- 172.21.15.46:0/3032228861 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f18f407e840 con 0x7f1904073d00 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 msgr2=0x7f18e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 0x7f18e406edb0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f19040c6850 tx=0x7f18f00079b0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 msgr2=0x7f190415e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000c00 tx=0x7f18f40046c0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 shutdown_connections 2024-04-26T23:59:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f190415cec0 0x7f19040c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040cdb30 0x7f19040c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 --2- 172.21.15.46:0/3032228861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1904073d00 0x7f190415e830 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 >> 172.21.15.46:0/3032228861 conn(0x7f19040b27b0 msgr2=0x7f1904075a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 shutdown_connections 2024-04-26T23:59:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.538+0000 7f1903200640 1 -- 172.21.15.46:0/3032228861 wait complete. 2024-04-26T23:59:10.558 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:59:10.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:59:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.753+0000 7fef7da00640 1 Processor -- start 2024-04-26T23:59:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.753+0000 7fef7da00640 1 -- start start 2024-04-26T23:59:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef781466b0 0x7fef7806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780720e0 0x7fef780cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef78160910 0x7fef78160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fef78073ad0 con 0x7fef78160910 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fef78146a90 con 0x7fef781466b0 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.755+0000 7fef7da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fef78073d70 con 0x7fef780720e0 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef78160910 0x7fef78160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef78160910 0x7fef78160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42252/0 (socket says 172.21.15.46:42252) 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 -- 172.21.15.46:0/3074831898 learned_addr learned my addr 172.21.15.46:0/3074831898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef77e00640 1 --2- 172.21.15.46:0/3074831898 >> v2:172.21.15.142:3300/0 conn(0x7fef781466b0 0x7fef7806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef6f400640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780720e0 0x7fef780cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 -- 172.21.15.46:0/3074831898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780720e0 msgr2=0x7fef780cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780720e0 0x7fef780cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 -- 172.21.15.46:0/3074831898 >> v2:172.21.15.142:3300/0 conn(0x7fef781466b0 msgr2=0x7fef7806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 --2- 172.21.15.46:0/3074831898 >> v2:172.21.15.142:3300/0 conn(0x7fef781466b0 0x7fef7806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.756+0000 7fef7d000640 1 -- 172.21.15.46:0/3074831898 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef78074050 con 0x7fef78160910 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.757+0000 7fef7d000640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef78160910 0x7fef78160da0 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fef780b3440 tx=0x7fef7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caf9fd3c98e1ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.758+0000 7fef77400640 1 -- 172.21.15.46:0/3074831898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70037070 con 0x7fef78160910 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.758+0000 7fef77400640 1 -- 172.21.15.46:0/3074831898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fef70032440 con 0x7fef78160910 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.758+0000 7fef77400640 1 -- 172.21.15.46:0/3074831898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70032830 con 0x7fef78160910 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 -- 172.21.15.46:0/3074831898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef78160910 msgr2=0x7fef78160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef78160910 0x7fef78160da0 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fef780b3440 tx=0x7fef7002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 -- 172.21.15.46:0/3074831898 shutdown_connections 2024-04-26T23:59:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef78160910 0x7fef78160da0 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 --2- 172.21.15.46:0/3074831898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780720e0 0x7fef780cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 --2- 172.21.15.46:0/3074831898 >> v2:172.21.15.142:3300/0 conn(0x7fef781466b0 0x7fef7806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 -- 172.21.15.46:0/3074831898 >> 172.21.15.46:0/3074831898 conn(0x7fef780b26d0 msgr2=0x7fef780b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 -- 172.21.15.46:0/3074831898 shutdown_connections 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.759+0000 7fef7da00640 1 -- 172.21.15.46:0/3074831898 wait complete. 2024-04-26T23:59:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.761+0000 7fef7da00640 1 Processor -- start 2024-04-26T23:59:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.761+0000 7fef7da00640 1 -- start start 2024-04-26T23:59:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef781466b0 0x7fef78141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef78160910 0x7fef781418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef7806e3c0 con 0x7fef780720e0 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef780cb470 con 0x7fef78160910 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7da00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fef780cdda0 con 0x7fef781466b0 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42256/0 (socket says 172.21.15.46:42256) 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 -- 172.21.15.46:0/3159766041 learned_addr learned my addr 172.21.15.46:0/3159766041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef6f400640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef781466b0 0x7fef78141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 -- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef781466b0 msgr2=0x7fef78141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef781466b0 0x7fef78141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 -- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef78160910 msgr2=0x7fef781418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef7d000640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef78160910 0x7fef781418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef78160910 0x7fef781418a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.762+0000 7fef77e00640 1 -- 172.21.15.46:0/3159766041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef780c8580 con 0x7fef780720e0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef77e00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7fef68007e40 tx=0x7fef6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68014620 con 0x7fef780720e0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fef68019070 con 0x7fef780720e0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68014a20 con 0x7fef780720e0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef780c8860 con 0x7fef780720e0 2024-04-26T23:59:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.763+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef780c8e20 con 0x7fef780720e0 2024-04-26T23:59:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.764+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef780693b0 con 0x7fef780720e0 2024-04-26T23:59:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.765+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef68014bc0 con 0x7fef780720e0 2024-04-26T23:59:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.766+0000 7fef74c00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.766+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 5421+0+0 (secure 0 0 0) 0x7fef6808c3b0 con 0x7fef780720e0 2024-04-26T23:59:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.767+0000 7fef6f400640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.768+0000 7fef6f400640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 0x7fef5406edb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:10.771+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef6805a7d0 con 0x7fef780720e0 2024-04-26T23:59:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.013+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef780c6510 con 0x7fef780720e0 2024-04-26T23:59:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.013+0000 7fef74c00640 1 -- 172.21.15.46:0/3159766041 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) ==== 95+0+3882 (secure 0 0 0) 0x7fef6805a0d0 con 0x7fef780720e0 2024-04-26T23:59:11.016 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-26T23:59:11.016 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":176,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4170405980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":4170405980},{"type":"v1","addr":"172.21.15.142:6837","nonce":4170405980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":8272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/2549318828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2549318828},{"type":"v1","addr":"172.21.15.142:6836","nonce":2549318828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162},{"gid":12051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1774274160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1774274160},{"type":"v1","addr":"172.21.15.46:6834","nonce":1774274160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161},{"gid":12570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3430923039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3430923039},{"type":"v1","addr":"172.21.15.46:6835","nonce":3430923039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[]} 2024-04-26T23:59:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 176 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 msgr2=0x7fef5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 0x7fef5406edb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60008040 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 msgr2=0x7fef780b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7fef68007e40 tx=0x7fef6800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 shutdown_connections 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:6832/1556001348,v1:172.21.15.142:6833/1556001348] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef78160910 0x7fef781418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef781466b0 0x7fef78141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 --2- 172.21.15.46:0/3159766041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef780720e0 0x7fef780b5910 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.016+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 >> 172.21.15.46:0/3159766041 conn(0x7fef780b26d0 msgr2=0x7fef78160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.017+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 shutdown_connections 2024-04-26T23:59:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-26T23:59:11.017+0000 7fef7da00640 1 -- 172.21.15.46:0/3159766041 wait complete. 2024-04-26T23:59:11.038 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T23:59:11.038 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T23:59:11.039 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:59:11.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.061 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:59:11.061 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:59:11.061 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.135 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:59:11.136 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:59:11.136 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.156 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:59:11.156 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:59:11.156 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.216 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:59:11.216 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T23:59:11.216 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T23:59:11.216 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.308 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T23:59:11.309 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T23:59:11.309 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.386 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T23:59:11.387 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T23:59:11.387 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.469 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T23:59:11.469 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T23:59:11.469 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.538 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T23:59:11.539 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T23:59:11.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.596 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T23:59:11.596 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T23:59:11.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:11.677 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T23:59:11.677 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T23:59:11.677 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:13.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-26T23:59:14.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:14.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:17.778 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T23:59:17.779 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T23:59:17.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:17.854 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T23:59:17.854 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T23:59:17.854 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T23:59:17.855 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:19.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-26T23:59:19.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-26T23:59:23.956 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T23:59:23.957 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T23:59:23.957 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:24.041 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T23:59:24.041 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T23:59:24.041 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T23:59:24.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:24.099 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T23:59:24.099 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T23:59:24.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:24.133 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T23:59:24.134 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T23:59:24.134 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:59:24.207 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T23:59:24.207 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T23:59:24.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-26T23:59:24.323 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:59:24.323 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T23:59:24.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-26T23:59:24.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:59:24.779 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T23:59:24.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:59:24.920 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T23:59:25.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-26T23:59:25.088 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T23:59:25.268 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:59:25.269 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T23:59:25.535 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:59:25.536 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T23:59:25.688 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:59:25.688 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T23:59:25.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-26T23:59:25.821 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T23:59:25.971 INFO:tasks.ceph:Archiving mon data... 2024-04-26T23:59:25.973 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/data/mon.a.tgz 2024-04-26T23:59:25.973 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-04-26T23:59:26.033 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.AmFkQdVULW 2024-04-26T23:59:26.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AmFkQdVULW 2024-04-26T23:59:26.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.AmFkQdVULW 2024-04-26T23:59:26.379 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.AmFkQdVULW is 473KB 2024-04-26T23:59:26.493 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.AmFkQdVULW 2024-04-26T23:59:26.555 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/data/mon.b.tgz 2024-04-26T23:59:26.555 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-04-26T23:59:26.612 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.N9rtQGAh4f 2024-04-26T23:59:26.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.N9rtQGAh4f 2024-04-26T23:59:26.811 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.N9rtQGAh4f 2024-04-26T23:59:26.950 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.N9rtQGAh4f is 473KB 2024-04-26T23:59:27.051 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.N9rtQGAh4f 2024-04-26T23:59:27.108 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/data/mon.c.tgz 2024-04-26T23:59:27.108 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-04-26T23:59:27.160 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.AGDP39GfDV 2024-04-26T23:59:27.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.AGDP39GfDV 2024-04-26T23:59:27.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.AGDP39GfDV 2024-04-26T23:59:27.484 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.AGDP39GfDV is 473KB 2024-04-26T23:59:27.596 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.AGDP39GfDV 2024-04-26T23:59:27.651 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T23:59:27.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T23:59:27.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T23:59:27.799 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T23:59:27.856 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T23:59:27.856 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T23:59:27.858 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi046/crash 2024-04-26T23:59:27.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T23:59:27.951 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi142/crash 2024-04-26T23:59:27.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T23:59:28.039 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T23:59:28.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:28.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:28.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:28.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:28.345 INFO:tasks.ceph:Compressing logs... 2024-04-26T23:59:28.346 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:59:28.353 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:59:28.434 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33289.log 2024-04-26T23:59:28.435 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T23:59:28.436 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39119.log 2024-04-26T23:59:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63221.log 2024-04-26T23:59:28.437 INFO:teuthology.orchestra.run.smithi142.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33289.log.gz 2024-04-26T23:59:28.437 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T23:59:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107667.log 2024-04-26T23:59:28.439 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.39119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39119.log.gz 2024-04-26T23:59:28.439 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T23:59:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43208.log 2024-04-26T23:59:28.439 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63221.log.gz 2024-04-26T23:59:28.440 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T23:59:28.441 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T23:59:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92775.log 2024-04-26T23:59:28.441 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107667.log.gz 2024-04-26T23:59:28.442 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33118.log 2024-04-26T23:59:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58464.log 2024-04-26T23:59:28.442 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43208.log.gz 2024-04-26T23:59:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56697.log 2024-04-26T23:59:28.443 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92775.log.gz 2024-04-26T23:59:28.444 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T23:59:28.444 INFO:teuthology.orchestra.run.smithi142.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T23:59:28.444 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33158.log 2024-04-26T23:59:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88132.log 2024-04-26T23:59:28.445 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58464.log.gz 2024-04-26T23:59:28.446 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33118.log: /var/log/ceph/ceph-osd.4.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33118.log.gz 2024-04-26T23:59:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2024-04-26T23:59:28.446 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56697.log.gz 2024-04-26T23:59:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90570.log 2024-04-26T23:59:28.447 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33409.log 2024-04-26T23:59:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44405.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.88132.log.gz 2024-04-26T23:59:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71797.log 2024-04-26T23:59:28.448 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2024-04-26T23:59:28.448 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33329.log 2024-04-26T23:59:28.448 INFO:teuthology.orchestra.run.smithi142.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33158.log.gz 2024-04-26T23:59:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87730.log 2024-04-26T23:59:28.449 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.90570.log.gz 2024-04-26T23:59:28.450 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2024-04-26T23:59:28.450 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71797.log.gz 2024-04-26T23:59:28.451 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33409.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T23:59:28.451 INFO:teuthology.orchestra.run.smithi142.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T23:59:28.451 INFO:teuthology.orchestra.run.smithi142.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33409.log.gz 2024-04-26T23:59:28.451 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33329.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T23:59:28.451 INFO:teuthology.orchestra.run.smithi142.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33329.log.gz 2024-04-26T23:59:28.452 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33202.log 2024-04-26T23:59:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41304.log 2024-04-26T23:59:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44243.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87730.log.gz 2024-04-26T23:59:28.452 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2024-04-26T23:59:28.452 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90402.log 2024-04-26T23:59:28.453 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101924.log 2024-04-26T23:59:28.453 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41304.log.gz 2024-04-26T23:59:28.454 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T23:59:28.454 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90402.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90402.log.gz 2024-04-26T23:59:28.454 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-04-26T23:59:28.455 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60475.log 2024-04-26T23:59:28.455 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101924.log.gz 2024-04-26T23:59:28.456 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33202.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33202.log.gz 2024-04-26T23:59:28.456 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95286.log 2024-04-26T23:59:28.456 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-04-26T23:59:28.457 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40096.log 2024-04-26T23:59:28.457 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60475.log.gz 2024-04-26T23:59:28.457 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33369.log 2024-04-26T23:59:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95286.log.gz 2024-04-26T23:59:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96036.log 2024-04-26T23:59:28.459 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84887.log 2024-04-26T23:59:28.459 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39100.log 2024-04-26T23:59:28.460 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40096.log.gz 2024-04-26T23:59:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99947.log 2024-04-26T23:59:28.460 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96036.log.gz 2024-04-26T23:59:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112085.log 2024-04-26T23:59:28.461 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84887.log.gz 2024-04-26T23:59:28.461 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.33369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39514.log 2024-04-26T23:59:28.461 INFO:teuthology.orchestra.run.smithi142.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33369.log.gz 2024-04-26T23:59:28.462 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33249.log 2024-04-26T23:59:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44573.log 2024-04-26T23:59:28.463 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99947.log.gz 2024-04-26T23:59:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112085.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112085.log.gz 2024-04-26T23:59:28.463 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.3.39514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39514.log.gz 2024-04-26T23:59:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70738.log 2024-04-26T23:59:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44573.log.gz 2024-04-26T23:59:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112043.log 2024-04-26T23:59:28.466 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.39100.log: /var/log/ceph/ceph-client.admin.33249.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39518.log 2024-04-26T23:59:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46656.log 2024-04-26T23:59:28.467 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70738.log.gz 2024-04-26T23:59:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112043.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112043.log.gz 2024-04-26T23:59:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71372.log 2024-04-26T23:59:28.468 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T23:59:28.468 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39100.log.gz 2024-04-26T23:59:28.468 INFO:teuthology.orchestra.run.smithi142.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33249.log.gz 2024-04-26T23:59:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72157.log 2024-04-26T23:59:28.469 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46656.log.gz 2024-04-26T23:59:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71372.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71372.log.gz 2024-04-26T23:59:28.469 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67753.log 2024-04-26T23:59:28.470 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.72157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-04-26T23:59:28.470 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72157.log.gz 2024-04-26T23:59:28.471 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38196.log 2024-04-26T23:59:28.472 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.3.39518.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T23:59:28.472 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67753.log.gz 2024-04-26T23:59:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45851.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-04-26T23:59:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68832.log 2024-04-26T23:59:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2024-04-26T23:59:28.474 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0%/var/log/ceph/ceph-mon.b.log: -- replaced with /var/log/ceph/ceph-client.3.39518.log.gz 2024-04-26T23:59:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68832.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.38196.log.gz 2024-04-26T23:59:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93856.log 2024-04-26T23:59:28.475 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68832.log.gz 2024-04-26T23:59:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39918.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2024-04-26T23:59:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-04-26T23:59:28.477 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104221.log 2024-04-26T23:59:28.477 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93856.log.gz 2024-04-26T23:59:28.477 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54497.log 2024-04-26T23:59:28.478 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-04-26T23:59:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.104221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.104221.log.gz 2024-04-26T23:59:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93028.log 2024-04-26T23:59:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90188.log 2024-04-26T23:59:28.480 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54497.log.gz 2024-04-26T23:59:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46572.log 2024-04-26T23:59:28.481 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93028.log.gz 2024-04-26T23:59:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79373.log 2024-04-26T23:59:28.482 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90188.log.gz 2024-04-26T23:59:28.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54454.log 2024-04-26T23:59:28.483 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46572.log.gz 2024-04-26T23:59:28.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90234.log 2024-04-26T23:59:28.484 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79373.log.gz 2024-04-26T23:59:28.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54454.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54454.log.gz 2024-04-26T23:59:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61646.log 2024-04-26T23:59:28.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98596.log 2024-04-26T23:59:28.487 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90234.log.gz 2024-04-26T23:59:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65934.log 2024-04-26T23:59:28.488 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61646.log.gz 2024-04-26T23:59:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95952.log 2024-04-26T23:59:28.489 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98596.log.gz 2024-04-26T23:59:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63366.log 2024-04-26T23:59:28.490 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65934.log.gz 2024-04-26T23:59:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65425.log 2024-04-26T23:59:28.491 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95952.log.gz 2024-04-26T23:59:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100893.log 2024-04-26T23:59:28.492 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63366.log.gz 2024-04-26T23:59:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108014.log 2024-04-26T23:59:28.493 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65425.log.gz 2024-04-26T23:59:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44663.log 2024-04-26T23:59:28.494 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100893.log.gz 2024-04-26T23:59:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95413.log 2024-04-26T23:59:28.495 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108014.log.gz 2024-04-26T23:59:28.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44663.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44663.log.gz -5 2024-04-26T23:59:28.496 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84505.log 2024-04-26T23:59:28.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65402.log 2024-04-26T23:59:28.497 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95413.log.gz 2024-04-26T23:59:28.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61279.log 2024-04-26T23:59:28.498 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84505.log.gz 2024-04-26T23:59:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65402.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65402.log.gz 2024-04-26T23:59:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70780.log 2024-04-26T23:59:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.61279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56953.log 2024-04-26T23:59:28.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61279.log.gz 2024-04-26T23:59:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111209.log 2024-04-26T23:59:28.501 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70780.log.gz 2024-04-26T23:59:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100456.log 2024-04-26T23:59:28.502 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56953.log.gz 2024-04-26T23:59:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-04-26T23:59:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-04-26T23:59:28.504 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.111209.log.gz 2024-04-26T23:59:28.505 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100456.log.gz 2024-04-26T23:59:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46786.log 2024-04-26T23:59:28.505 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-04-26T23:59:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45161.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gzgzip 2024-04-26T23:59:28.506 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80828.log 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46786.log.gz 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94904.log 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80828.log.gz 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104469.log 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107446.log 2024-04-26T23:59:28.511 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94904.log.gz 2024-04-26T23:59:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-04-26T23:59:28.512 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104469.log.gz 2024-04-26T23:59:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2024-04-26T23:59:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46391.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.107446.log.gz 2024-04-26T23:59:28.514 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-04-26T23:59:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60391.log 2024-04-26T23:59:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55325.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2024-04-26T23:59:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105374.log 2024-04-26T23:59:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102582.log 2024-04-26T23:59:28.516 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60391.log.gz 2024-04-26T23:59:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67838.log 2024-04-26T23:59:28.517 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105374.log.gz 2024-04-26T23:59:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102582.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102582.log.gz -5 2024-04-26T23:59:28.518 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.108919.log 2024-04-26T23:59:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100679.log 2024-04-26T23:59:28.519 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67838.log.gz 2024-04-26T23:59:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67245.log 2024-04-26T23:59:28.520 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108919.log.gz 2024-04-26T23:59:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100679.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100679.log.gz 2024-04-26T23:59:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40769.log 2024-04-26T23:59:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46521.log 2024-04-26T23:59:28.523 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67245.log.gz 2024-04-26T23:59:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95612.log 2024-04-26T23:59:28.523 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40769.log.gz 2024-04-26T23:59:28.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112711.log 2024-04-26T23:59:28.525 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46521.log.gz 2024-04-26T23:59:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81380.log 2024-04-26T23:59:28.526 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95612.log.gz 2024-04-26T23:59:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79919.log 2024-04-26T23:59:28.526 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112711.log.gz 2024-04-26T23:59:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.81380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81380.log.gz 2024-04-26T23:59:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72007.log 2024-04-26T23:59:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60225.log 2024-04-26T23:59:28.529 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79919.log.gz 2024-04-26T23:59:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.72007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68294.log 2024-04-26T23:59:28.530 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72007.log.gz 2024-04-26T23:59:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60225.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60225.log.gz 2024-04-26T23:59:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108537.log 2024-04-26T23:59:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94423.log 2024-04-26T23:59:28.532 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68294.log.gz 2024-04-26T23:59:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103154.log 2024-04-26T23:59:28.533 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108537.log.gz 2024-04-26T23:59:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42995.log 2024-04-26T23:59:28.534 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94423.log.gz 2024-04-26T23:59:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84764.log 2024-04-26T23:59:28.535 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103154.log.gz 2024-04-26T23:59:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105094.log 2024-04-26T23:59:28.536 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42995.log.gz 2024-04-26T23:59:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65868.log 2024-04-26T23:59:28.537 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84764.log.gz 2024-04-26T23:59:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96102.log 2024-04-26T23:59:28.538 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105094.log.gz 2024-04-26T23:59:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56092.log 2024-04-26T23:59:28.540 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65868.log.gz 2024-04-26T23:59:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57385.log 2024-04-26T23:59:28.540 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96102.log.gz 2024-04-26T23:59:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64443.log 2024-04-26T23:59:28.541 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56092.log.gz 2024-04-26T23:59:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60773.log 2024-04-26T23:59:28.542 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57385.log.gz 2024-04-26T23:59:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94077.log 2024-04-26T23:59:28.543 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64443.log.gz 2024-04-26T23:59:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-04-26T23:59:28.544 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60773.log.gz 2024-04-26T23:59:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102642.log 2024-04-26T23:59:28.545 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94077.log.gz 2024-04-26T23:59:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46870.log 2024-04-26T23:59:28.546 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-04-26T23:59:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105458.log 2024-04-26T23:59:28.547 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102642.log.gz 2024-04-26T23:59:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42758.log 2024-04-26T23:59:28.548 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46870.log.gz 2024-04-26T23:59:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90276.log 2024-04-26T23:59:28.549 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105458.log.gz 2024-04-26T23:59:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59527.log 2024-04-26T23:59:28.550 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42758.log.gz 2024-04-26T23:59:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105729.log 2024-04-26T23:59:28.551 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90276.log.gz 2024-04-26T23:59:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60605.log 2024-04-26T23:59:28.552 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59527.log.gz 2024-04-26T23:59:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105729.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105729.log.gz 2024-04-26T23:59:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-04-26T23:59:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60605.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60605.log.gz 2024-04-26T23:59:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2024-04-26T23:59:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81122.log 2024-04-26T23:59:28.556 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-26T23:59:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101966.log 2024-04-26T23:59:28.557 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2024-04-26T23:59:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.81122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101463.log 2024-04-26T23:59:28.558 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.81122.log.gz 2024-04-26T23:59:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69978.log 2024-04-26T23:59:28.559 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101966.log.gz 2024-04-26T23:59:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68126.log 2024-04-26T23:59:28.560 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101463.log.gz 2024-04-26T23:59:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70425.log 2024-04-26T23:59:28.561 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69978.log.gz 2024-04-26T23:59:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92901.log 2024-04-26T23:59:28.562 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68126.log.gz 2024-04-26T23:59:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-04-26T23:59:28.563 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70425.log.gz 2024-04-26T23:59:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87880.log 2024-04-26T23:59:28.564 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92901.log.gz 2024-04-26T23:59:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33677.log 2024-04-26T23:59:28.565 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-04-26T23:59:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90486.log 2024-04-26T23:59:28.566 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87880.log.gz 2024-04-26T23:59:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64611.log 2024-04-26T23:59:28.567 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33677.log.gz 2024-04-26T23:59:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54012.log 2024-04-26T23:59:28.568 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90486.log.gz 2024-04-26T23:59:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67016.log 2024-04-26T23:59:28.569 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64611.log.gz 2024-04-26T23:59:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83308.log 2024-04-26T23:59:28.570 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54012.log.gz 2024-04-26T23:59:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85058.log 2024-04-26T23:59:28.571 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67016.log.gz 2024-04-26T23:59:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96294.log 2024-04-26T23:59:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85058.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.83308.log.gz 2024-04-26T23:59:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69724.log 2024-04-26T23:59:28.573 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85058.log.gz 2024-04-26T23:59:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.96294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96294.log.gz 2024-04-26T23:59:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93130.log 2024-04-26T23:59:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69724.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69724.log.gz 2024-04-26T23:59:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83637.log 2024-04-26T23:59:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79961.log 2024-04-26T23:59:28.576 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93130.log.gz 2024-04-26T23:59:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45725.log 2024-04-26T23:59:28.577 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83637.log.gz 2024-04-26T23:59:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89582.log 2024-04-26T23:59:28.578 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79961.log.gz 2024-04-26T23:59:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39636.log 2024-04-26T23:59:28.579 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45725.log.gz 2024-04-26T23:59:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94549.log 2024-04-26T23:59:28.580 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89582.log.gz 2024-04-26T23:59:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-04-26T23:59:28.581 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39636.log.gz 2024-04-26T23:59:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99448.log 2024-04-26T23:59:28.582 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94549.log.gz 2024-04-26T23:59:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92709.log 2024-04-26T23:59:28.583 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-04-26T23:59:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99448.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.99448.log.gz 2024-04-26T23:59:28.584 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59840.log 2024-04-26T23:59:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2024-04-26T23:59:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59840.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.92709.log.gz 2024-04-26T23:59:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T23:59:28.586 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59840.log.gz 2024-04-26T23:59:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44789.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2024-04-26T23:59:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110635.log 2024-04-26T23:59:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109046.log 2024-04-26T23:59:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44747.log 2024-04-26T23:59:28.589 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110635.log.gz 2024-04-26T23:59:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109046.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109046.log.gz 2024-04-26T23:59:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39468.log 2024-04-26T23:59:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59966.log 2024-04-26T23:59:28.592 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44747.log.gz 2024-04-26T23:59:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-04-26T23:59:28.592 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39468.log.gz 2024-04-26T23:59:28.593 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T23:59:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60899.log 2024-04-26T23:59:28.594 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59966.log.gz 2024-04-26T23:59:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56284.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-04-26T23:59:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83875.log 2024-04-26T23:59:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-04-26T23:59:28.596 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60899.log.gz 2024-04-26T23:59:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93496.log 2024-04-26T23:59:28.597 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83875.log.gz 2024-04-26T23:59:28.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93172.log 2024-04-26T23:59:28.598 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-04-26T23:59:28.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90360.log 2024-04-26T23:59:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93172.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.93496.log.gz 2024-04-26T23:59:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84421.log 2024-04-26T23:59:28.600 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93172.log.gz 2024-04-26T23:59:28.601 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83595.log 2024-04-26T23:59:28.601 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90360.log.gz 2024-04-26T23:59:28.602 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70467.log 2024-04-26T23:59:28.602 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84421.log.gz 2024-04-26T23:59:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107930.log 2024-04-26T23:59:28.603 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83595.log.gz 2024-04-26T23:59:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112169.log 2024-04-26T23:59:28.604 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-04-26T23:59:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39834.log 2024-04-26T23:59:28.605 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107930.log.gz 2024-04-26T23:59:28.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71671.log 2024-04-26T23:59:28.606 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112169.log.gz 2024-04-26T23:59:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39834.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39834.log.gz 2024-04-26T23:59:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100542.log 2024-04-26T23:59:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105939.log 2024-04-26T23:59:28.609 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71671.log.gz 2024-04-26T23:59:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100542.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100542.log.gz 2024-04-26T23:59:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110439.log 2024-04-26T23:59:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59695.log 2024-04-26T23:59:28.611 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105939.log.gz 2024-04-26T23:59:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54257.log 2024-04-26T23:59:28.612 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110439.log.gz 2024-04-26T23:59:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63137.log 2024-04-26T23:59:28.613 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-04-26T23:59:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113091.log 2024-04-26T23:59:28.614 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54257.log.gz 2024-04-26T23:59:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44831.log 2024-04-26T23:59:28.615 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63137.log.gz 2024-04-26T23:59:28.616 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105813.log 2024-04-26T23:59:28.616 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113091.log.gz 2024-04-26T23:59:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-04-26T23:59:28.617 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44831.log.gz 2024-04-26T23:59:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93430.log 2024-04-26T23:59:28.618 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105813.log.gz 2024-04-26T23:59:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80003.log 2024-04-26T23:59:28.619 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-04-26T23:59:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112839.log 2024-04-26T23:59:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80003.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.93430.log.gz 2024-04-26T23:59:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T23:59:28.621 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80003.log.gz 2024-04-26T23:59:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69085.log 2024-04-26T23:59:28.622 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112839.log.gz 2024-04-26T23:59:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89041.log 2024-04-26T23:59:28.624 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94683.log 2024-04-26T23:59:28.624 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69085.log.gz 2024-04-26T23:59:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79751.log 2024-04-26T23:59:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94683.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89041.log.gz 2024-04-26T23:59:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63762.log 2024-04-26T23:59:28.626 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94683.log.gz 2024-04-26T23:59:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79751.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.102186.log 2024-04-26T23:59:28.627 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79751.log.gz 2024-04-26T23:59:28.628 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109004.log 2024-04-26T23:59:28.629 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63762.log.gz 2024-04-26T23:59:28.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67203.log 2024-04-26T23:59:28.629 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102186.log.gz 2024-04-26T23:59:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109004.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.109004.log.gz /var/log/ceph/ceph-client.admin.94591.log 2024-04-26T23:59:28.630 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.632 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2024-04-26T23:59:28.632 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67203.log.gz 2024-04-26T23:59:28.632 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85495.log 2024-04-26T23:59:28.633 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94591.log.gz 2024-04-26T23:59:28.634 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94946.log 2024-04-26T23:59:28.634 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2024-04-26T23:59:28.635 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85537.log 2024-04-26T23:59:28.635 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85495.log.gz 2024-04-26T23:59:28.636 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38996.log 2024-04-26T23:59:28.636 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94946.log.gz 2024-04-26T23:59:28.637 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70058.log 2024-04-26T23:59:28.637 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85537.log.gz 2024-04-26T23:59:28.638 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68038.log 2024-04-26T23:59:28.638 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38996.log.gz 2024-04-26T23:59:28.639 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112443.log 2024-04-26T23:59:28.639 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70058.log.gz 2024-04-26T23:59:28.640 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106969.log 2024-04-26T23:59:28.640 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68038.log.gz 2024-04-26T23:59:28.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112443.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112443.log.gz 2024-04-26T23:59:28.641 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46614.log 2024-04-26T23:59:28.642 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84253.log 2024-04-26T23:59:28.642 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106969.log.gz 2024-04-26T23:59:28.643 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93993.log 2024-04-26T23:59:28.643 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46614.log.gz 2024-04-26T23:59:28.644 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99364.log 2024-04-26T23:59:28.644 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84253.log.gz 2024-04-26T23:59:28.645 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93993.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93993.log.gz 2024-04-26T23:59:28.645 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58408.log 2024-04-26T23:59:28.646 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37986.log 2024-04-26T23:59:28.646 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99364.log.gz 2024-04-26T23:59:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58408.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58408.log.gz 2024-04-26T23:59:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106321.log 2024-04-26T23:59:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94465.log 2024-04-26T23:59:28.648 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37986.log.gz 2024-04-26T23:59:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71211.log 2024-04-26T23:59:28.649 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106321.log.gz 2024-04-26T23:59:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100725.log 2024-04-26T23:59:28.651 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94465.log.gz 2024-04-26T23:59:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71211.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71211.log.gz 2024-04-26T23:59:28.651 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39552.log 2024-04-26T23:59:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80744.log 2024-04-26T23:59:28.652 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100725.log.gz 2024-04-26T23:59:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54215.log 2024-04-26T23:59:28.653 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39552.log.gz 2024-04-26T23:59:28.654 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100372.log 2024-04-26T23:59:28.655 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80744.log.gz 2024-04-26T23:59:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54215.log.gz 2024-04-26T23:59:28.655 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61361.log 2024-04-26T23:59:28.656 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98366.log 2024-04-26T23:59:28.656 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100372.log.gz 2024-04-26T23:59:28.657 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68528.log 2024-04-26T23:59:28.657 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61361.log.gz 2024-04-26T23:59:28.658 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-04-26T23:59:28.659 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98366.log.gz 2024-04-26T23:59:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68528.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68528.log.gz 2024-04-26T23:59:28.659 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88789.log 2024-04-26T23:59:28.660 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-04-26T23:59:28.660 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-04-26T23:59:28.661 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84211.log 2024-04-26T23:59:28.661 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88789.log.gz 2024-04-26T23:59:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95784.log 2024-04-26T23:59:28.663 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-04-26T23:59:28.663 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84211.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84211.log.gzgzip 2024-04-26T23:59:28.663 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.83553.log 2024-04-26T23:59:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108838.log 2024-04-26T23:59:28.664 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95784.log.gz 2024-04-26T23:59:28.665 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88048.log 2024-04-26T23:59:28.665 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83553.log.gz 2024-04-26T23:59:28.666 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54131.log 2024-04-26T23:59:28.667 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108838.log.gz 2024-04-26T23:59:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88048.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88048.log.gz 2024-04-26T23:59:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83176.log 2024-04-26T23:59:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109088.log 2024-04-26T23:59:28.669 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54131.log.gz 2024-04-26T23:59:28.669 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66764.log 2024-04-26T23:59:28.670 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83176.log.gz 2024-04-26T23:59:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89270.log 2024-04-26T23:59:28.671 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109088.log.gz 2024-04-26T23:59:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107011.log 2024-04-26T23:59:28.672 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66764.log.gz 2024-04-26T23:59:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-04-26T23:59:28.673 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89270.log.gz 2024-04-26T23:59:28.673 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67287.log 2024-04-26T23:59:28.674 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.107011.log.gz 2024-04-26T23:59:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109441.log 2024-04-26T23:59:28.675 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-04-26T23:59:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55230.log 2024-04-26T23:59:28.676 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67287.log.gz 2024-04-26T23:59:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-04-26T23:59:28.676 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109441.log.gz 2024-04-26T23:59:28.677 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57639.log 2024-04-26T23:59:28.678 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55230.log.gz 2024-04-26T23:59:28.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98126.log 2024-04-26T23:59:28.679 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-04-26T23:59:28.680 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65826.log 2024-04-26T23:59:28.680 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57639.log.gz 2024-04-26T23:59:28.680 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99144.log 2024-04-26T23:59:28.682 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.98126.log.gz 2024-04-26T23:59:28.682 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65826.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T23:59:28.682 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65826.log.gz 2024-04-26T23:59:28.683 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57843.log 2024-04-26T23:59:28.683 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99144.log.gz 2024-04-26T23:59:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44873.log 2024-04-26T23:59:28.685 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59275.log 2024-04-26T23:59:28.685 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57843.log.gz 2024-04-26T23:59:28.686 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44873.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.44873.log.gz --verbose 2024-04-26T23:59:28.686 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.0.101315.log 2024-04-26T23:59:28.687 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-04-26T23:59:28.687 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59275.log.gz 2024-04-26T23:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.101315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101315.log.gz 2024-04-26T23:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87796.log 2024-04-26T23:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T23:59:28.689 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38912.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-04-26T23:59:28.689 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58037.log 2024-04-26T23:59:28.690 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38112.log 2024-04-26T23:59:28.690 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87796.log.gz 2024-04-26T23:59:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110361.log 2024-04-26T23:59:28.691 INFO:teuthology.orchestra.run.smithi046.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.58037.log.gz 2024-04-26T23:59:28.692 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60563.log 2024-04-26T23:59:28.692 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38112.log.gz 2024-04-26T23:59:28.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110361.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110361.log.gz 2024-04-26T23:59:28.693 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84001.log 2024-04-26T23:59:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107709.log 2024-04-26T23:59:28.694 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60563.log.gz 2024-04-26T23:59:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65540.log 2024-04-26T23:59:28.695 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84001.log.gz 2024-04-26T23:59:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66680.log 2024-04-26T23:59:28.697 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107709.log.gz 2024-04-26T23:59:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65540.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65540.log.gz 2024-04-26T23:59:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57019.log 2024-04-26T23:59:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101061.log 2024-04-26T23:59:28.699 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66680.log.gz 2024-04-26T23:59:28.699 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63179.log 2024-04-26T23:59:28.699 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57019.log.gz 2024-04-26T23:59:28.701 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90636.log 2024-04-26T23:59:28.701 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.101061.log.gz 2024-04-26T23:59:28.702 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103585.log 2024-04-26T23:59:28.702 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63179.log.gz 2024-04-26T23:59:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59443.log 2024-04-26T23:59:28.703 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90636.log.gz 2024-04-26T23:59:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70653.log 2024-04-26T23:59:28.704 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103585.log.gz 2024-04-26T23:59:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-04-26T23:59:28.705 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59443.log.gz 2024-04-26T23:59:28.706 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102312.log 2024-04-26T23:59:28.706 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70653.log.gz 2024-04-26T23:59:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-04-26T23:59:28.708 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-04-26T23:59:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104801.log 2024-04-26T23:59:28.709 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102312.log.gz 2024-04-26T23:59:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95517.log 2024-04-26T23:59:28.710 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-04-26T23:59:28.711 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65780.log 2024-04-26T23:59:28.711 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.104801.log.gz 2024-04-26T23:59:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80702.log 2024-04-26T23:59:28.712 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95517.log.gz 2024-04-26T23:59:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98656.log 2024-04-26T23:59:28.713 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65780.log.gz 2024-04-26T23:59:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54299.log 2024-04-26T23:59:28.714 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80702.log.gz 2024-04-26T23:59:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85579.log 2024-04-26T23:59:28.715 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98656.log.gz 2024-04-26T23:59:28.716 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57681.log 2024-04-26T23:59:28.716 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54299.log.gz 2024-04-26T23:59:28.718 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112797.log 2024-04-26T23:59:28.718 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.85579.log.gz 2024-04-26T23:59:28.719 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58530.log 2024-04-26T23:59:28.719 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112797.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57681.log.gz 2024-04-26T23:59:28.720 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103715.log 2024-04-26T23:59:28.720 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112797.log.gz 2024-04-26T23:59:28.721 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103757.log 2024-04-26T23:59:28.722 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58530.log.gz 2024-04-26T23:59:28.722 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101505.log 2024-04-26T23:59:28.722 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103715.log.gz 2024-04-26T23:59:28.723 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57164.log 2024-04-26T23:59:28.723 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103757.log.gz 2024-04-26T23:59:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109311.log 2024-04-26T23:59:28.725 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.101505.log.gz 2024-04-26T23:59:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100498.log 2024-04-26T23:59:28.726 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57164.log.gz 2024-04-26T23:59:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98324.log 2024-04-26T23:59:28.727 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109311.log.gz 2024-04-26T23:59:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100498.log.gz 2024-04-26T23:59:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68168.log 2024-04-26T23:59:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56476.log 2024-04-26T23:59:28.729 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98324.log.gz 2024-04-26T23:59:28.730 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111646.log 2024-04-26T23:59:28.730 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68168.log.gz 2024-04-26T23:59:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.56476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56476.log.gz 2024-04-26T23:59:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56635.log 2024-04-26T23:59:28.732 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89757.log 2024-04-26T23:59:28.732 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111646.log.gz 2024-04-26T23:59:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63678.log 2024-04-26T23:59:28.733 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56635.log.gz 2024-04-26T23:59:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89757.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.89757.log.gz --verbose 2024-04-26T23:59:28.734 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.62558.log 2024-04-26T23:59:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110731.log 2024-04-26T23:59:28.736 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63678.log.gz 2024-04-26T23:59:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95125.log 2024-04-26T23:59:28.737 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62558.log.gz 2024-04-26T23:59:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110731.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110731.log.gz 2024-04-26T23:59:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55836.log 2024-04-26T23:59:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106364.log 2024-04-26T23:59:28.739 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95125.log.gz 2024-04-26T23:59:28.739 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47500.log 2024-04-26T23:59:28.739 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55836.log.gz 2024-04-26T23:59:28.741 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72367.log 2024-04-26T23:59:28.741 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106364.log.gz 2024-04-26T23:59:28.741 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70864.log 2024-04-26T23:59:28.742 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47500.log.gz 2024-04-26T23:59:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.72367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72367.log.gz 2024-04-26T23:59:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T23:59:28.744 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70864.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70864.log.gzgzip 2024-04-26T23:59:28.744 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.89480.log 2024-04-26T23:59:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39594.log 2024-04-26T23:59:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99280.log 2024-04-26T23:59:28.746 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89480.log.gz 2024-04-26T23:59:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39264.log 2024-04-26T23:59:28.747 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39594.log.gz 2024-04-26T23:59:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60857.log 2024-04-26T23:59:28.748 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99280.log.gz 2024-04-26T23:59:28.749 INFO:teuthology.orchestra.run.smithi142.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T23:59:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56176.log 2024-04-26T23:59:28.750 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz 2024-04-26T23:59:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103673.log 2024-04-26T23:59:28.751 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60857.log.gz 2024-04-26T23:59:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57081.log 2024-04-26T23:59:28.752 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56176.log.gz 2024-04-26T23:59:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103673.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.100586.log -- replaced with /var/log/ceph/ceph-client.admin.103673.log.gz 2024-04-26T23:59:28.753 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65240.log 2024-04-26T23:59:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100586.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57081.log.gz 2024-04-26T23:59:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-04-26T23:59:28.755 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100586.log.gz 2024-04-26T23:59:28.756 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113383.log 2024-04-26T23:59:28.757 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65240.log.gz 2024-04-26T23:59:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85102.log 2024-04-26T23:59:28.758 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-04-26T23:59:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70509.log 2024-04-26T23:59:28.759 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113383.log.gz 2024-04-26T23:59:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89167.log 2024-04-26T23:59:28.760 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85102.log.gz 2024-04-26T23:59:28.761 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85154.log 2024-04-26T23:59:28.761 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70509.log.gz 2024-04-26T23:59:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110773.log 2024-04-26T23:59:28.762 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89167.log.gz 2024-04-26T23:59:28.763 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85154.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85154.log.gz 2024-04-26T23:59:28.763 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-04-26T23:59:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110773.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110773.log.gz 2024-04-26T23:59:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-04-26T23:59:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109717.log 2024-04-26T23:59:28.766 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 2024-04-26T23:59:28.767 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88831.log 2024-04-26T23:59:28.767 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-04-26T23:59:28.768 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109717.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.109717.log.gz 2024-04-26T23:59:28.768 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37816.log 2024-04-26T23:59:28.769 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66848.log 2024-04-26T23:59:28.769 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88831.log.gz 2024-04-26T23:59:28.770 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-04-26T23:59:28.770 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37816.log.gz 2024-04-26T23:59:28.772 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84547.log 2024-04-26T23:59:28.772 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66848.log.gz 2024-04-26T23:59:28.772 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103320.log 2024-04-26T23:59:28.774 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103631.log 2024-04-26T23:59:28.774 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84547.log.gz 2024-04-26T23:59:28.774 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-04-26T23:59:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71923.log 2024-04-26T23:59:28.775 INFO:teuthology.orchestra.run.smithi046.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.103320.log.gz 2024-04-26T23:59:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64359.log 2024-04-26T23:59:28.776 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103631.log.gz 2024-04-26T23:59:28.777 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84085.log 2024-04-26T23:59:28.777 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71923.log.gz 2024-04-26T23:59:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80660.log 2024-04-26T23:59:28.778 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64359.log.gz 2024-04-26T23:59:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63011.log 2024-04-26T23:59:28.780 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84085.log.gz 2024-04-26T23:59:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56008.log 2024-04-26T23:59:28.781 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80660.log.gz 2024-04-26T23:59:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100414.log 2024-04-26T23:59:28.782 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63011.log.gz 2024-04-26T23:59:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56008.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.56008.log.gz /var/log/ceph/ceph-client.admin.44951.log 2024-04-26T23:59:28.783 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58103.log 2024-04-26T23:59:28.784 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100414.log.gz 2024-04-26T23:59:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99238.log 2024-04-26T23:59:28.785 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-04-26T23:59:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88390.log 2024-04-26T23:59:28.787 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58103.log.gz 2024-04-26T23:59:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102438.log 2024-04-26T23:59:28.788 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99238.log.gz 2024-04-26T23:59:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2024-04-26T23:59:28.789 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88390.log.gz 2024-04-26T23:59:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2024-04-26T23:59:28.790 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102438.log.gz 2024-04-26T23:59:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83833.log 2024-04-26T23:59:28.791 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2024-04-26T23:59:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-04-26T23:59:28.793 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2024-04-26T23:59:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101882.log 2024-04-26T23:59:28.794 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83833.log.gz 2024-04-26T23:59:28.795 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64401.log 2024-04-26T23:59:28.795 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-04-26T23:59:28.796 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111520.log 2024-04-26T23:59:28.796 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101882.log.gz 2024-04-26T23:59:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55613.log 2024-04-26T23:59:28.797 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64401.log.gz 2024-04-26T23:59:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57971.log 2024-04-26T23:59:28.798 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111520.log.gz 2024-04-26T23:59:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94778.log 2024-04-26T23:59:28.800 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55613.log.gz 2024-04-26T23:59:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99803.log 2024-04-26T23:59:28.801 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57971.log.gz 2024-04-26T23:59:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102480.log 2024-04-26T23:59:28.802 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94778.log.gz 2024-04-26T23:59:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108758.log 2024-04-26T23:59:28.803 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99803.log.gz 2024-04-26T23:59:28.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66460.log 2024-04-26T23:59:28.805 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102480.log.gz 2024-04-26T23:59:28.805 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108758.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108758.log.gz 2024-04-26T23:59:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T23:59:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39222.log 2024-04-26T23:59:28.807 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66460.log.gz 2024-04-26T23:59:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108224.log 2024-04-26T23:59:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39222.log.gz 2024-04-26T23:59:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93346.log 2024-04-26T23:59:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-04-26T23:59:28.811 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108224.log: /var/log/ceph/ceph-client.admin.93346.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108224.log.gz 2024-04-26T23:59:28.811 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103925.log 2024-04-26T23:59:28.812 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93346.log.gz 2024-04-26T23:59:28.812 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99532.log 2024-04-26T23:59:28.813 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-04-26T23:59:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103925.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103925.log.gz 2024-04-26T23:59:28.814 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71541.log 2024-04-26T23:59:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68462.log 2024-04-26T23:59:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71541.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99532.log.gz 2024-04-26T23:59:28.816 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.71541.log.gz -5 2024-04-26T23:59:28.816 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95371.log 2024-04-26T23:59:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106198.log 2024-04-26T23:59:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95371.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68462.log.gz 2024-04-26T23:59:28.819 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95371.log.gz 2024-04-26T23:59:28.819 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70341.log 2024-04-26T23:59:28.819 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93730.log 2024-04-26T23:59:28.820 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106198.log.gz 2024-04-26T23:59:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88915.log 2024-04-26T23:59:28.821 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70341.log.gz 2024-04-26T23:59:28.822 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37668.log 2024-04-26T23:59:28.822 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93730.log.gz 2024-04-26T23:59:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90095.log 2024-04-26T23:59:28.823 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88915.log.gz 2024-04-26T23:59:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54707.log 2024-04-26T23:59:28.824 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37668.log.gz 2024-04-26T23:59:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62686.log 2024-04-26T23:59:28.826 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90095.log.gz 2024-04-26T23:59:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105676.log 2024-04-26T23:59:28.827 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54707.log.gz 2024-04-26T23:59:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62686.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.62686.log.gz 2024-04-26T23:59:28.828 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104591.log 2024-04-26T23:59:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42184.log 2024-04-26T23:59:28.829 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105676.log.gz 2024-04-26T23:59:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63636.log 2024-04-26T23:59:28.831 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104591.log.gz 2024-04-26T23:59:28.831 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42184.log.gz 2024-04-26T23:59:28.831 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95205.log 2024-04-26T23:59:28.833 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-04-26T23:59:28.833 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63636.log.gz 2024-04-26T23:59:28.834 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113259.log 2024-04-26T23:59:28.834 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95205.log.gz 2024-04-26T23:59:28.835 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61025.log 2024-04-26T23:59:28.835 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-04-26T23:59:28.836 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102937.log 2024-04-26T23:59:28.837 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113259.log.gz 2024-04-26T23:59:28.837 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39390.log 2024-04-26T23:59:28.838 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61025.log.gz 2024-04-26T23:59:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2024-04-26T23:59:28.839 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102937.log.gz 2024-04-26T23:59:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33574.log 2024-04-26T23:59:28.840 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39390.log.gz 2024-04-26T23:59:28.841 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69001.log 2024-04-26T23:59:28.841 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2024-04-26T23:59:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33574.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33574.log.gz 2024-04-26T23:59:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79793.log 2024-04-26T23:59:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.69001.log: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T23:59:28.844 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69001.log.gz 2024-04-26T23:59:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65538.log 2024-04-26T23:59:28.845 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79793.log.gz 2024-04-26T23:59:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112611.log 2024-04-26T23:59:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65538.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65538.log.gz 2024-04-26T23:59:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T23:59:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-04-26T23:59:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112611.log.gz 2024-04-26T23:59:28.850 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79069.log 2024-04-26T23:59:28.851 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42842.log 2024-04-26T23:59:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-04-26T23:59:28.852 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79069.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79069.log.gz 2024-04-26T23:59:28.853 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83679.log 2024-04-26T23:59:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42842.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44705.log 2024-04-26T23:59:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42842.log.gz 2024-04-26T23:59:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102853.log 2024-04-26T23:59:28.856 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83679.log.gz 2024-04-26T23:59:28.857 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44705.log.gz 2024-04-26T23:59:28.857 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108621.log 2024-04-26T23:59:28.858 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102853.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102853.log.gz 2024-04-26T23:59:28.858 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69205.log 2024-04-26T23:59:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111143.log 2024-04-26T23:59:28.860 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108621.log.gz 2024-04-26T23:59:28.862 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69205.log: 91.1%/var/log/ceph/ceph-client.admin.111143.log: -- replaced with /var/log/ceph/ceph-client.admin.69205.log.gz 2024-04-26T23:59:28.863 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65324.log 2024-04-26T23:59:28.863 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.111143.log.gz 2024-04-26T23:59:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-04-26T23:59:28.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-04-26T23:59:28.865 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65324.log.gz 2024-04-26T23:59:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46349.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-04-26T23:59:28.867 INFO:teuthology.orchestra.run.smithi046.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T23:59:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-04-26T23:59:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45931.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-04-26T23:59:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92817.log 2024-04-26T23:59:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70299.log 2024-04-26T23:59:28.870 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-04-26T23:59:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88999.log 2024-04-26T23:59:28.871 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92817.log.gz 2024-04-26T23:59:28.872 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70299.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70299.log.gz 2024-04-26T23:59:28.872 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60145.log 2024-04-26T23:59:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.88999.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65198.log 2024-04-26T23:59:28.874 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88999.log.gz 2024-04-26T23:59:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80870.log 2024-04-26T23:59:28.875 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60145.log.gz 2024-04-26T23:59:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94988.log 2024-04-26T23:59:28.876 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65198.log.gz 2024-04-26T23:59:28.877 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84379.log 2024-04-26T23:59:28.877 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80870.log.gz 2024-04-26T23:59:28.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99718.log 2024-04-26T23:59:28.879 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94988.log.gz 2024-04-26T23:59:28.880 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90007.log 2024-04-26T23:59:28.880 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84379.log.gz 2024-04-26T23:59:28.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2024-04-26T23:59:28.881 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99718.log.gz 2024-04-26T23:59:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90007.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90007.log.gz 2024-04-26T23:59:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44447.log 2024-04-26T23:59:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70593.log 2024-04-26T23:59:28.884 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2024-04-26T23:59:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63720.log 2024-04-26T23:59:28.885 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44447.log.gz 2024-04-26T23:59:28.886 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70593.log: gzip -5 88.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.70593.log.gz -- 2024-04-26T23:59:28.886 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.69850.log 2024-04-26T23:59:28.887 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63720.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T23:59:28.887 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63720.log.gz 2024-04-26T23:59:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69850.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69850.log.gz 2024-04-26T23:59:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109905.log 2024-04-26T23:59:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-04-26T23:59:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.0.109905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109905.log.gz 2024-04-26T23:59:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60008.log 2024-04-26T23:59:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39876.log: -- /var/log/ceph/ceph-client.admin.108140.log 2024-04-26T23:59:28.892 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-04-26T23:59:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40429.log 2024-04-26T23:59:28.893 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60008.log.gz 2024-04-26T23:59:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108140.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108140.log.gz 2024-04-26T23:59:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55020.log 2024-04-26T23:59:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40100.log 2024-04-26T23:59:28.895 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40429.log.gz 2024-04-26T23:59:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2024-04-26T23:59:28.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55020.log.gz 2024-04-26T23:59:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40100.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40100.log.gz 2024-04-26T23:59:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54539.log 2024-04-26T23:59:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106448.log 2024-04-26T23:59:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54539.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2024-04-26T23:59:28.900 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54539.log.gz 2024-04-26T23:59:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94736.log 2024-04-26T23:59:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106448.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106448.log.gz 2024-04-26T23:59:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2024-04-26T23:59:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105626.log 2024-04-26T23:59:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94736.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94736.log.gz 2024-04-26T23:59:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54173.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2024-04-26T23:59:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108369.log 2024-04-26T23:59:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105626.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105626.log.gz 2024-04-26T23:59:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58282.log 2024-04-26T23:59:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55486.log 2024-04-26T23:59:28.909 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108369.log.gz 2024-04-26T23:59:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58282.log.gz 2024-04-26T23:59:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2024-04-26T23:59:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55486.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55486.log.gz 2024-04-26T23:59:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59924.log 2024-04-26T23:59:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111814.log 2024-04-26T23:59:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54623.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2024-04-26T23:59:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83959.log 2024-04-26T23:59:28.914 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59924.log.gz 2024-04-26T23:59:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111814.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111814.log.gz 2024-04-26T23:59:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78668.log 2024-04-26T23:59:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64231.log 2024-04-26T23:59:28.917 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83959.log.gz 2024-04-26T23:59:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.78668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2024-04-26T23:59:28.919 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78668.log.gz 2024-04-26T23:59:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69658.log 2024-04-26T23:59:28.920 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64231.log.gz 2024-04-26T23:59:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53796.log 2024-04-26T23:59:28.921 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-04-26T23:59:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55062.log 2024-04-26T23:59:28.923 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1%/var/log/ceph/ceph-client.admin.53796.log: -- replaced with /var/log/ceph/ceph-client.admin.69658.log.gz 2024-04-26T23:59:28.923 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53796.log.gz 2024-04-26T23:59:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89880.log 2024-04-26T23:59:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40088.log 2024-04-26T23:59:28.925 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55062.log.gz 2024-04-26T23:59:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89880.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.70551.log -- replaced with /var/log/ceph/ceph-client.admin.89880.log.gz 2024-04-26T23:59:28.927 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40088.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40088.log.gz 2024-04-26T23:59:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105332.log 2024-04-26T23:59:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65282.log 2024-04-26T23:59:28.930 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70551.log.gz 2024-04-26T23:59:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85014.log 88.5% 2024-04-26T23:59:28.931 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105332.log.gz 2024-04-26T23:59:28.933 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58993.log 2024-04-26T23:59:28.933 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65282.log.gz 2024-04-26T23:59:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42621.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85014.log.gz 2024-04-26T23:59:28.934 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58993.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58993.log.gz 2024-04-26T23:59:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109269.log 2024-04-26T23:59:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.42621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42621.log.gz 2024-04-26T23:59:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62018.log 2024-04-26T23:59:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56134.log 2024-04-26T23:59:28.938 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109269.log.gz 2024-04-26T23:59:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62018.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62018.log.gz 2024-04-26T23:59:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108495.log 2024-04-26T23:59:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106587.log 2024-04-26T23:59:28.941 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56134.log.gz 2024-04-26T23:59:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108495.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108495.log.gz 2024-04-26T23:59:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99060.log 2024-04-26T23:59:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106587.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106587.log.gz 2024-04-26T23:59:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-04-26T23:59:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80385.log 2024-04-26T23:59:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99060.log.gz 2024-04-26T23:59:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2024-04-26T23:59:28.947 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-04-26T23:59:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111940.log 2024-04-26T23:59:28.948 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80385.log.gz 2024-04-26T23:59:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39791.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz -5 2024-04-26T23:59:28.949 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44075.log 2024-04-26T23:59:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42930.log 2024-04-26T23:59:28.951 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111940.log.gz 2024-04-26T23:59:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44075.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44075.log.gz 2024-04-26T23:59:28.952 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.45809.log 2024-04-26T23:59:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42930.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42930.log.gz 2024-04-26T23:59:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71457.log 2024-04-26T23:59:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45809.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-04-26T23:59:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105500.log 2024-04-26T23:59:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99322.log 2024-04-26T23:59:28.957 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71457.log.gz 2024-04-26T23:59:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105500.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105500.log.gz 2024-04-26T23:59:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98881.log 2024-04-26T23:59:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99322.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99322.log.gz 2024-04-26T23:59:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2024-04-26T23:59:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112000.log 2024-04-26T23:59:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98881.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98881.log.gz 2024-04-26T23:59:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44993.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2024-04-26T23:59:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41052.log 2024-04-26T23:59:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112000.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112000.log.gz 2024-04-26T23:59:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41640.log 2024-04-26T23:59:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53730.log 2024-04-26T23:59:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41052.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41052.log.gz 2024-04-26T23:59:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53862.log 2024-04-26T23:59:28.966 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41640.log.gz 2024-04-26T23:59:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85243.log 2024-04-26T23:59:28.968 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53730.log.gz 2024-04-26T23:59:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68080.log 2024-04-26T23:59:28.970 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53862.log.gz/var/log/ceph/ceph-client.admin.85243.log: 2024-04-26T23:59:28.971 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85243.log.gz 2024-04-26T23:59:28.971 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.85285.log 2024-04-26T23:59:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68080.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64018.log 2024-04-26T23:59:28.972 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68080.log.gz 2024-04-26T23:59:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43712.log 2024-04-26T23:59:28.973 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85285.log.gz 2024-04-26T23:59:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68420.log 2024-04-26T23:59:28.975 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64018.log.gz 2024-04-26T23:59:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100637.log 2024-04-26T23:59:28.976 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43712.log.gz 2024-04-26T23:59:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68420.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68420.log.gz 2024-04-26T23:59:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92577.log 2024-04-26T23:59:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72363.log 2024-04-26T23:59:28.979 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100637.log.gz 2024-04-26T23:59:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54791.log 2024-04-26T23:59:28.981 INFO:teuthology.orchestra.run.smithi046.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.92577.log.gz 2024-04-26T23:59:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.72363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72363.log.gz 2024-04-26T23:59:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59653.log 2024-04-26T23:59:28.982 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98492.log 2024-04-26T23:59:28.983 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54791.log.gz/var/log/ceph/ceph-client.admin.59653.log: 2024-04-26T23:59:28.984 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59653.log.gz 2024-04-26T23:59:28.984 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70696.log 2024-04-26T23:59:28.985 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98492.log.gz 2024-04-26T23:59:28.986 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70383.log 2024-04-26T23:59:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41346.log 2024-04-26T23:59:28.987 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70696.log.gz 2024-04-26T23:59:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66638.log 2024-04-26T23:59:28.989 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70383.log.gz 2024-04-26T23:59:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41346.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41346.log.gz 2024-04-26T23:59:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112653.log 2024-04-26T23:59:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108579.log 2024-04-26T23:59:28.991 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66638.log.gz 2024-04-26T23:59:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62412.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112653.log.gz 2024-04-26T23:59:28.993 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108579.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108579.log.gz 2024-04-26T23:59:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108182.log 2024-04-26T23:59:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62412.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62412.log.gz 2024-04-26T23:59:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66418.log 2024-04-26T23:59:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85837.log 2024-04-26T23:59:28.998 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108182.log.gz 2024-04-26T23:59:28.999 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66418.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66418.log.gz 2024-04-26T23:59:28.999 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-04-26T23:59:28.999 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.85837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85837.log.gz 2024-04-26T23:59:29.000 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-04-26T23:59:29.001 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55146.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz 2024-04-26T23:59:29.001 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84043.log 2024-04-26T23:59:29.003 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.41094.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93070.log 2024-04-26T23:59:29.003 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-04-26T23:59:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84043.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84043.log.gz 2024-04-26T23:59:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66144.log 2024-04-26T23:59:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93070.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93070.log.gzgzip 2024-04-26T23:59:29.006 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65718.log 2024-04-26T23:59:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.66144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85369.log 2024-04-26T23:59:29.007 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66144.log.gz 2024-04-26T23:59:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65718.log.gz 2024-04-26T23:59:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100168.log 2024-04-26T23:59:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85369.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85369.log.gz 2024-04-26T23:59:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110319.log 2024-04-26T23:59:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108663.log 2024-04-26T23:59:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100168.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100168.log.gz 2024-04-26T23:59:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110319.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110319.log.gz 2024-04-26T23:59:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85645.log 2024-04-26T23:59:29.014 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108663.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108663.log.gz 2024-04-26T23:59:29.014 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43376.log 2024-04-26T23:59:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.85645.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63324.log 2024-04-26T23:59:29.016 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85645.log.gz 2024-04-26T23:59:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43376.log.gz 2024-04-26T23:59:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63053.log 2024-04-26T23:59:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63324.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63324.log.gz 2024-04-26T23:59:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70906.log 2024-04-26T23:59:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88006.log 2024-04-26T23:59:29.021 INFO:teuthology.orchestra.run.smithi046.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63053.log.gz 2024-04-26T23:59:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70906.log.gz 2024-04-26T23:59:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79237.log 2024-04-26T23:59:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88006.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88006.log.gz 2024-04-26T23:59:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41178.log 2024-04-26T23:59:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105981.log 2024-04-26T23:59:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79237.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79237.log.gz 2024-04-26T23:59:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41178.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41178.log.gz 2024-04-26T23:59:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80912.log 2024-04-26T23:59:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105981.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105981.log.gz 2024-04-26T23:59:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64317.log 2024-04-26T23:59:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101397.log 2024-04-26T23:59:29.029 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80912.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80912.log.gz 2024-04-26T23:59:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60689.log 2024-04-26T23:59:29.030 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64317.log.gz 2024-04-26T23:59:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101397.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.101397.log.gz 2024-04-26T23:59:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67100.log 2024-04-26T23:59:29.033 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65541.log 2024-04-26T23:59:29.033 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60689.log.gz 2024-04-26T23:59:29.034 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37774.log 2024-04-26T23:59:29.034 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67100.log.gz 2024-04-26T23:59:29.035 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65541.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65541.log.gz 2024-04-26T23:59:29.035 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89838.log 2024-04-26T23:59:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99616.log 2024-04-26T23:59:29.037 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37774.log.gz 2024-04-26T23:59:29.038 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88569.log 2024-04-26T23:59:29.038 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89838.log.gz 2024-04-26T23:59:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99616.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66376.log 2024-04-26T23:59:29.040 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99616.log.gz 2024-04-26T23:59:29.041 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93951.log 2024-04-26T23:59:29.041 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88569.log.gz 2024-04-26T23:59:29.042 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79709.log 2024-04-26T23:59:29.042 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66376.log.gz 2024-04-26T23:59:29.044 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62165.log 2024-04-26T23:59:29.044 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93951.log.gz 2024-04-26T23:59:29.045 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79709.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79709.log.gz 2024-04-26T23:59:29.045 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2024-04-26T23:59:29.046 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60731.log 2024-04-26T23:59:29.047 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62165.log.gz 2024-04-26T23:59:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44363.log 2024-04-26T23:59:29.048 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2024-04-26T23:59:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60731.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60731.log.gz 2024-04-26T23:59:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112569.log 2024-04-26T23:59:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88306.log 2024-04-26T23:59:29.051 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44363.log.gz 2024-04-26T23:59:29.052 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58836.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112569.log.gz 2024-04-26T23:59:29.052 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88306.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88306.log.gz 2024-04-26T23:59:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55743.log 2024-04-26T23:59:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58836.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58836.log.gz -5 2024-04-26T23:59:29.055 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2024-04-26T23:59:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96372.log 2024-04-26T23:59:29.056 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55743.log.gz 2024-04-26T23:59:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94381.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2024-04-26T23:59:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69330.log 2024-04-26T23:59:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96372.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96372.log.gz 2024-04-26T23:59:29.060 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41430.log 2024-04-26T23:59:29.060 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95654.log 2024-04-26T23:59:29.061 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69330.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69330.log.gz 2024-04-26T23:59:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41430.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41430.log.gz 2024-04-26T23:59:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113175.log 2024-04-26T23:59:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95654.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95654.log.gz 2024-04-26T23:59:29.064 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88485.log 2024-04-26T23:59:29.065 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.113175.log: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T23:59:29.065 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113175.log.gz 2024-04-26T23:59:29.066 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88485.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88485.log.gz 2024-04-26T23:59:29.067 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67371.log 2024-04-26T23:59:29.068 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98408.log 2024-04-26T23:59:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67371.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67371.log.gz 2024-04-26T23:59:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98408.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98408.log.gz 2024-04-26T23:59:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93646.log 2024-04-26T23:59:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46744.log 2024-04-26T23:59:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93646.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93646.log.gz 2024-04-26T23:59:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46744.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46744.log.gz 2024-04-26T23:59:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94119.log 2024-04-26T23:59:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88090.log 2024-04-26T23:59:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94119.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94119.log.gz 2024-04-26T23:59:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88090.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88090.log.gz 2024-04-26T23:59:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112755.log 2024-04-26T23:59:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-04-26T23:59:29.084 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112755.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112755.log.gz 2024-04-26T23:59:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-04-26T23:59:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101721.log 2024-04-26T23:59:29.088 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-04-26T23:59:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101721.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101721.log.gz 2024-04-26T23:59:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109567.log 2024-04-26T23:59:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38028.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-04-26T23:59:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101127.log 2024-04-26T23:59:29.093 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109567.log.gz 2024-04-26T23:59:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101127.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101127.log.gz 2024-04-26T23:59:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111898.log 2024-04-26T23:59:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95742.log 2024-04-26T23:59:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111898.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111898.log.gz 2024-04-26T23:59:29.100 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95742.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95742.log.gz 2024-04-26T23:59:29.100 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63408.log 2024-04-26T23:59:29.102 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92943.log 2024-04-26T23:59:29.104 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63408.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63408.log.gz 2024-04-26T23:59:29.105 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92943.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92943.log.gz 2024-04-26T23:59:29.105 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67796.log 2024-04-26T23:59:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41136.log 2024-04-26T23:59:29.108 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67796.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67796.log.gz 2024-04-26T23:59:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41136.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41136.log.gz 2024-04-26T23:59:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105188.log 2024-04-26T23:59:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110819.log 2024-04-26T23:59:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105188.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105188.log.gz 2024-04-26T23:59:29.114 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110819.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110819.log.gz 2024-04-26T23:59:29.115 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85833.log 2024-04-26T23:59:29.117 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2024-04-26T23:59:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.85833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85833.log.gz 2024-04-26T23:59:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43039.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2024-04-26T23:59:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64653.log 2024-04-26T23:59:29.122 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62258.log 2024-04-26T23:59:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64653.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64653.log.gz 2024-04-26T23:59:29.124 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62258.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62258.log.gz 2024-04-26T23:59:29.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102354.log 2024-04-26T23:59:29.131 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104926.log 2024-04-26T23:59:29.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102354.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102354.log.gz 2024-04-26T23:59:29.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99574.log 2024-04-26T23:59:29.131 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104926.log.gz 2024-04-26T23:59:29.131 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71585.log 2024-04-26T23:59:29.133 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99574.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99574.log.gz 2024-04-26T23:59:29.133 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40436.log/var/log/ceph/ceph-client.admin.71585.log: 2024-04-26T23:59:29.133 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71585.log.gz 2024-04-26T23:59:29.134 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107846.log 2024-04-26T23:59:29.136 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40436.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40436.log.gz 2024-04-26T23:59:29.136 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83917.log 2024-04-26T23:59:29.137 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107846.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107846.log.gz 2024-04-26T23:59:29.139 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83917.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83917.log.gz 2024-04-26T23:59:29.140 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71713.log 2024-04-26T23:59:29.141 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102811.log 2024-04-26T23:59:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71713.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71713.log.gz 2024-04-26T23:59:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102811.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102811.log.gz 2024-04-26T23:59:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108266.log 2024-04-26T23:59:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 2024-04-26T23:59:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108266.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108266.log.gz 2024-04-26T23:59:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38070.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-04-26T23:59:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67455.log 2024-04-26T23:59:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72445.log 2024-04-26T23:59:29.153 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67455.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67455.log.gz 2024-04-26T23:59:29.154 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.72445.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72445.log.gz 2024-04-26T23:59:29.154 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55405.log 2024-04-26T23:59:29.156 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107565.log 2024-04-26T23:59:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55405.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55405.log.gz 2024-04-26T23:59:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107565.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107565.log.gz 2024-04-26T23:59:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66140.log 2024-04-26T23:59:29.161 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84589.log 2024-04-26T23:59:29.162 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.66140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66140.log.gz 2024-04-26T23:59:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84589.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84589.log.gz 2024-04-26T23:59:29.164 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62016.log 2024-04-26T23:59:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60519.log 2024-04-26T23:59:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62016.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62016.log.gz 2024-04-26T23:59:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60519.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60519.log.gz 2024-04-26T23:59:29.169 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64060.log 2024-04-26T23:59:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109357.log 2024-04-26T23:59:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64060.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64060.log.gz 2024-04-26T23:59:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.109357.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.100767.log 2024-04-26T23:59:29.173 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109357.log.gz 2024-04-26T23:59:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111730.log 2024-04-26T23:59:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100767.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100767.log.gz 2024-04-26T23:59:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79195.log 2024-04-26T23:59:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111730.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111730.log.gz 2024-04-26T23:59:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39348.log 2024-04-26T23:59:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79195.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79195.log.gz 2024-04-26T23:59:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69324.log 2024-04-26T23:59:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39348.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39348.log.gz 2024-04-26T23:59:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69324.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69324.log.gz 2024-04-26T23:59:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80224.log 2024-04-26T23:59:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71839.log 2024-04-26T23:59:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80224.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80224.log.gz 2024-04-26T23:59:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59077.log 2024-04-26T23:59:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71839.log.gz 2024-04-26T23:59:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104968.log 2024-04-26T23:59:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59077.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59077.log.gz 2024-04-26T23:59:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104033.log 2024-04-26T23:59:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104968.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104968.log.gz 2024-04-26T23:59:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98450.log 2024-04-26T23:59:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104033.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104033.log.gz 2024-04-26T23:59:29.194 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84169.log 2024-04-26T23:59:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98450.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98450.log.gz 2024-04-26T23:59:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64527.log 2024-04-26T23:59:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84169.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84169.log.gz 2024-04-26T23:59:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-04-26T23:59:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64527.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64527.log.gz 2024-04-26T23:59:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106759.log 2024-04-26T23:59:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40761.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 2024-04-26T23:59:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106759.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106759.log.gz 2024-04-26T23:59:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40421.log 2024-04-26T23:59:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62801.log 2024-04-26T23:59:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40421.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40421.log.gz 2024-04-26T23:59:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98976.log 2024-04-26T23:59:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62801.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62801.log.gz 2024-04-26T23:59:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62454.log 2024-04-26T23:59:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98976.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98976.log.gz 2024-04-26T23:59:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62454.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62454.log.gz 2024-04-26T23:59:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88264.log 2024-04-26T23:59:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2024-04-26T23:59:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88264.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88264.log.gz 2024-04-26T23:59:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81038.log 2024-04-26T23:59:29.215 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T23:59:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46828.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2024-04-26T23:59:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104525.log 2024-04-26T23:59:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.81038.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81038.log.gz 2024-04-26T23:59:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63450.log 2024-04-26T23:59:29.219 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104525.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.104525.log.gz 2024-04-26T23:59:29.220 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56472.log 2024-04-26T23:59:29.221 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63450.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63450.log.gz 2024-04-26T23:59:29.222 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70215.log 2024-04-26T23:59:29.223 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.56472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56472.log.gz 2024-04-26T23:59:29.224 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54936.log 2024-04-26T23:59:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70215.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70215.log.gz 2024-04-26T23:59:29.226 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54936.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54936.log.gz 2024-04-26T23:59:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59485.log 2024-04-26T23:59:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81376.log 2024-04-26T23:59:29.230 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59485.log.gz 2024-04-26T23:59:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92986.log 2024-04-26T23:59:29.232 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.81376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81376.log.gz 2024-04-26T23:59:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56825.log 2024-04-26T23:59:29.234 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92986.log.gz 2024-04-26T23:59:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61091.log 2024-04-26T23:59:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110193.log 2024-04-26T23:59:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56825.log: /var/log/ceph/ceph-client.admin.61091.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61091.log.gz 2024-04-26T23:59:29.238 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56825.log.gz 2024-04-26T23:59:29.239 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62843.log 2024-04-26T23:59:29.240 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110193.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110193.log.gz 2024-04-26T23:59:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70173.log 2024-04-26T23:59:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62843.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62843.log.gz 2024-04-26T23:59:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70173.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2024-04-26T23:59:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37858.log 2024-04-26T23:59:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40417.log 2024-04-26T23:59:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37858.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37858.log.gz 2024-04-26T23:59:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109525.log 2024-04-26T23:59:29.248 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40417.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40417.log.gz 2024-04-26T23:59:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42625.log 2024-04-26T23:59:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109525.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109525.log.gz 2024-04-26T23:59:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.42625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42625.log.gz 2024-04-26T23:59:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108411.log 2024-04-26T23:59:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103448.log 2024-04-26T23:59:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108411.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108411.log.gz 2024-04-26T23:59:29.255 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T23:59:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69043.log 2024-04-26T23:59:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103448.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103448.log.gz 2024-04-26T23:59:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113301.log 2024-04-26T23:59:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69043.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69043.log.gz 2024-04-26T23:59:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45665.log 2024-04-26T23:59:29.260 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113301.log.gz 2024-04-26T23:59:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68210.log 2024-04-26T23:59:29.262 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100249.log 2024-04-26T23:59:29.262 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45665.log.gz 2024-04-26T23:59:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68210.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68210.log.gz 2024-04-26T23:59:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113133.log 2024-04-26T23:59:29.265 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100249.log.gz 2024-04-26T23:59:29.267 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113133.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113133.log.gz 2024-04-26T23:59:29.268 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80786.log 2024-04-26T23:59:29.269 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55794.log 2024-04-26T23:59:29.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80786.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80786.log.gz 2024-04-26T23:59:29.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66974.log 2024-04-26T23:59:29.272 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55794.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55794.log.gz 2024-04-26T23:59:29.273 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53946.log 2024-04-26T23:59:29.274 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66974.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66974.log.gz 2024-04-26T23:59:29.276 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53946.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53946.log.gz 2024-04-26T23:59:29.276 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89312.log 2024-04-26T23:59:29.278 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71881.log 2024-04-26T23:59:29.280 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89312.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89312.log.gz 2024-04-26T23:59:29.280 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69912.log 2024-04-26T23:59:29.281 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71881.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71881.log.gz 2024-04-26T23:59:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90528.log 2024-04-26T23:59:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69912.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69912.log.gz 2024-04-26T23:59:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90528.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90528.log.gz 2024-04-26T23:59:29.285 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.104758.log 2024-04-26T23:59:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44117.log 2024-04-26T23:59:29.288 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104758.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104758.log.gz 2024-04-26T23:59:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102144.log 2024-04-26T23:59:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44117.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44117.log.gz 2024-04-26T23:59:29.290 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79457.log 2024-04-26T23:59:29.291 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102144.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102144.log.gz 2024-04-26T23:59:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71499.log 2024-04-26T23:59:29.293 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79457.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79457.log.gz 2024-04-26T23:59:29.295 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71499.log.gz 2024-04-26T23:59:29.295 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81080.log 2024-04-26T23:59:29.297 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98192.log 2024-04-26T23:59:29.298 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.81080.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81080.log.gz 2024-04-26T23:59:29.299 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42165.log 2024-04-26T23:59:29.300 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98192.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.98192.log.gz 2024-04-26T23:59:29.301 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102685.log 2024-04-26T23:59:29.301 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42165.log.gz 2024-04-26T23:59:29.302 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64695.log 2024-04-26T23:59:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102685.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102685.log.gz 2024-04-26T23:59:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95030.log 2024-04-26T23:59:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64695.log.gz 2024-04-26T23:59:29.306 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85453.log 2024-04-26T23:59:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95030.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95030.log.gz 2024-04-26T23:59:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80045.log 2024-04-26T23:59:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85453.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85453.log.gz 2024-04-26T23:59:29.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80045.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80045.log.gz 2024-04-26T23:59:29.311 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65539.log 2024-04-26T23:59:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-04-26T23:59:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65539.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65539.log.gz 2024-04-26T23:59:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54978.log 2024-04-26T23:59:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43880.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-04-26T23:59:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41514.log 2024-04-26T23:59:29.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54978.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54978.log.gz 2024-04-26T23:59:29.319 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41514.log: -- /var/log/ceph/ceph-client.admin.94213.log 2024-04-26T23:59:29.319 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41514.log.gz 2024-04-26T23:59:29.320 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107265.log 2024-04-26T23:59:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94213.log.gz 2024-04-26T23:59:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111293.log 2024-04-26T23:59:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.107265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107265.log.gz 2024-04-26T23:59:29.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109132.log 2024-04-26T23:59:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111293.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.111293.log.gz 2024-04-26T23:59:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109132.log.gz 2024-04-26T23:59:29.327 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89354.log 2024-04-26T23:59:29.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54665.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.89354.log: /var/log/ceph/ceph-client.0.109909.log 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89354.log.gz 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105010.log 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.109909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109909.log.gz 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37712.log 2024-04-26T23:59:29.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105010.log.gz 2024-04-26T23:59:29.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37712.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37712.log.gz 2024-04-26T23:59:29.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61283.log 2024-04-26T23:59:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-04-26T23:59:29.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.61283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61283.log.gz 2024-04-26T23:59:29.340 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94862.log 2024-04-26T23:59:29.342 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37625.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-04-26T23:59:29.343 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39038.log/var/log/ceph/ceph-client.admin.94862.log: 2024-04-26T23:59:29.343 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94862.log.gz 2024-04-26T23:59:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109609.log 2024-04-26T23:59:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39038.log.gz 2024-04-26T23:59:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111772.log 2024-04-26T23:59:29.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109609.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109609.log.gz 2024-04-26T23:59:29.348 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79415.log 2024-04-26T23:59:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111772.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111772.log.gz 2024-04-26T23:59:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80470.log 2024-04-26T23:59:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79415.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79415.log.gz 2024-04-26T23:59:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80470.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80470.log.gz 2024-04-26T23:59:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98258.log 2024-04-26T23:59:29.355 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69616.log 2024-04-26T23:59:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98258.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.98258.log.gz 2024-04-26T23:59:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44285.log 2024-04-26T23:59:29.358 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69616.log.gz 2024-04-26T23:59:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59233.log 2024-04-26T23:59:29.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44285.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44285.log.gz 2024-04-26T23:59:29.361 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.59233.log: --verbose -- /var/log/ceph/ceph-client.admin.94507.log 2024-04-26T23:59:29.361 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59233.log.gz 2024-04-26T23:59:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40770.log 2024-04-26T23:59:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94507.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94507.log.gz 2024-04-26T23:59:29.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40093.log 2024-04-26T23:59:29.366 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40770.log.gz 2024-04-26T23:59:29.367 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83242.log 2024-04-26T23:59:29.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40093.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40093.log.gz 2024-04-26T23:59:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.83242.log: /var/log/ceph/ceph-client.admin.93562.log 2024-04-26T23:59:29.370 INFO:teuthology.orchestra.run.smithi046.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.83242.log.gz 2024-04-26T23:59:29.371 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98534.log 2024-04-26T23:59:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93562.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93562.log.gz 2024-04-26T23:59:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99102.log 2024-04-26T23:59:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98534.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98534.log.gz 2024-04-26T23:59:29.375 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67943.log 2024-04-26T23:59:29.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99102.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99102.log.gz 2024-04-26T23:59:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67943.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67943.log.gz 2024-04-26T23:59:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102270.log 2024-04-26T23:59:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99761.log 2024-04-26T23:59:29.381 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102270.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102270.log.gz 2024-04-26T23:59:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-04-26T23:59:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99761.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99761.log.gz 2024-04-26T23:59:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100809.log 2024-04-26T23:59:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37576.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-04-26T23:59:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-04-26T23:59:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100809.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100809.log.gz 2024-04-26T23:59:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57507.log 2024-04-26T23:59:29.388 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43250.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-04-26T23:59:29.390 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57507.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57507.log.gz 2024-04-26T23:59:29.390 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103543.log 2024-04-26T23:59:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65156.log 2024-04-26T23:59:29.394 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103543.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103543.log.gz 2024-04-26T23:59:29.394 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69327.log 2024-04-26T23:59:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.65156.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65156.log.gz 2024-04-26T23:59:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-04-26T23:59:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69327.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69327.log.gz 2024-04-26T23:59:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39747.log: /var/log/ceph/ceph-client.admin.94255.log 2024-04-26T23:59:29.398 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-04-26T23:59:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87598.log 2024-04-26T23:59:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94255.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94255.log.gz 2024-04-26T23:59:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92643.log 2024-04-26T23:59:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87598.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87598.log.gz 2024-04-26T23:59:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.92643.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80572.log 2024-04-26T23:59:29.404 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.92643.log.gz 2024-04-26T23:59:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41388.log 2024-04-26T23:59:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80572.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80572.log.gz 2024-04-26T23:59:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56891.log 2024-04-26T23:59:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41388.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41388.log.gz 2024-04-26T23:59:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96414.log 2024-04-26T23:59:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56891.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56891.log.gz 2024-04-26T23:59:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103492.log 2024-04-26T23:59:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96414.log.gz 2024-04-26T23:59:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89396.log 2024-04-26T23:59:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103492.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103492.log.gz 2024-04-26T23:59:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110677.log 2024-04-26T23:59:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89396.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89396.log.gz 2024-04-26T23:59:29.417 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.110677.log: /var/log/ceph/ceph-client.admin.111604.log 2024-04-26T23:59:29.417 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110677.log.gz 2024-04-26T23:59:29.418 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93280.log 2024-04-26T23:59:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111604.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111604.log.gz 2024-04-26T23:59:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-04-26T23:59:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93280.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93280.log.gz 2024-04-26T23:59:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-04-26T23:59:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47122.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-04-26T23:59:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38700.log 2024-04-26T23:59:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45119.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-04-26T23:59:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105584.log 2024-04-26T23:59:29.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38700.log: /var/log/ceph/ceph-client.admin.105584.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105584.log.gz 2024-04-26T23:59:29.430 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 92.2% -- replaced with /var/log/ceph/ceph-client.admin.38700.log.gz 2024-04-26T23:59:29.430 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106629.log 2024-04-26T23:59:29.431 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63857.log 2024-04-26T23:59:29.433 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106629.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106629.log.gz 2024-04-26T23:59:29.433 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80616.log 2024-04-26T23:59:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63857.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63857.log.gz 2024-04-26T23:59:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80954.log 2024-04-26T23:59:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80616.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80616.log.gz 2024-04-26T23:59:29.437 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80954.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80954.log.gz 2024-04-26T23:59:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-04-26T23:59:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94339.log 2024-04-26T23:59:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45035.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-04-26T23:59:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55655.log 2024-04-26T23:59:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94339.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94339.log.gz 2024-04-26T23:59:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68716.log 2024-04-26T23:59:29.444 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55655.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55655.log.gz 2024-04-26T23:59:29.445 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106536.log 2024-04-26T23:59:29.445 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.68716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68716.log.gz 2024-04-26T23:59:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-04-26T23:59:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106536.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106536.log.gz 2024-04-26T23:59:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104225.log 2024-04-26T23:59:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38262.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-04-26T23:59:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67630.log 2024-04-26T23:59:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.104225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.104225.log.gz 2024-04-26T23:59:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39122.log 2024-04-26T23:59:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67630.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67630.log.gz 2024-04-26T23:59:29.454 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103277.log 2024-04-26T23:59:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39122.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39122.log.gz 2024-04-26T23:59:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-04-26T23:59:29.457 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103277.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103277.log.gz 2024-04-26T23:59:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-04-26T23:59:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57441.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-04-26T23:59:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89083.log 2024-04-26T23:59:29.461 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-04-26T23:59:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85411.log 2024-04-26T23:59:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89083.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89083.log.gz 2024-04-26T23:59:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.85411.log: /var/log/ceph/ceph.tmp-client.admin.25910.log 2024-04-26T23:59:29.464 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85411.log.gz 2024-04-26T23:59:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94820.log 2024-04-26T23:59:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.25910.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25910.log.gz 2024-04-26T23:59:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80129.log 2024-04-26T23:59:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94820.log.gz 2024-04-26T23:59:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113217.log 2024-04-26T23:59:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80129.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80129.log.gz 2024-04-26T23:59:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113217.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113217.log.gz 2024-04-26T23:59:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106406.log 2024-04-26T23:59:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57747.log 2024-04-26T23:59:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106406.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106406.log.gz 2024-04-26T23:59:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60348.log 2024-04-26T23:59:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57747.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57747.log.gz 2024-04-26T23:59:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69127.log/var/log/ceph/ceph-client.admin.60348.log: 2024-04-26T23:59:29.478 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60348.log.gz 2024-04-26T23:59:29.479 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-04-26T23:59:29.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69127.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69127.log.gz 2024-04-26T23:59:29.481 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90828.log 2024-04-26T23:59:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43334.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2024-04-26T23:59:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61688.log 2024-04-26T23:59:29.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.90828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90828.log.gz 2024-04-26T23:59:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99845.log 2024-04-26T23:59:29.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61688.log.gz 2024-04-26T23:59:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102895.log 2024-04-26T23:59:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99845.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99845.log.gz 2024-04-26T23:59:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43838.log 2024-04-26T23:59:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102895.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102895.log.gz 2024-04-26T23:59:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43838.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43838.log.gz 2024-04-26T23:59:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70990.log 2024-04-26T23:59:29.493 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108308.log 2024-04-26T23:59:29.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70990.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70990.log.gz 2024-04-26T23:59:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84337.log 2024-04-26T23:59:29.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108308.log.gz 2024-04-26T23:59:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113049.log 2024-04-26T23:59:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84337.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84337.log.gz 2024-04-26T23:59:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105290.log 2024-04-26T23:59:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113049.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113049.log.gz 2024-04-26T23:59:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100935.log 2024-04-26T23:59:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105290.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105290.log.gz 2024-04-26T23:59:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105230.log 2024-04-26T23:59:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100935.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100935.log.gz 2024-04-26T23:59:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64103.log 2024-04-26T23:59:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105230.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105230.log.gz 2024-04-26T23:59:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71116.log 2024-04-26T23:59:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64103.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64103.log.gz 2024-04-26T23:59:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81188.log 2024-04-26T23:59:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71116.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71116.log.gz 2024-04-26T23:59:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102979.log 2024-04-26T23:59:29.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.81188.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81188.log.gz 2024-04-26T23:59:29.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93814.log 2024-04-26T23:59:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102979.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102979.log.gz 2024-04-26T23:59:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-04-26T23:59:29.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93814.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93814.log.gz 2024-04-26T23:59:29.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80427.log 2024-04-26T23:59:29.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43124.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-04-26T23:59:29.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80427.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80427.log.gz 2024-04-26T23:59:29.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-04-26T23:59:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89522.log 2024-04-26T23:59:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45413.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-04-26T23:59:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67987.log 2024-04-26T23:59:29.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89522.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89522.log.gz 2024-04-26T23:59:29.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107888.log 2024-04-26T23:59:29.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67987.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67987.log.gz 2024-04-26T23:59:29.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71414.log 2024-04-26T23:59:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107888.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107888.log.gz 2024-04-26T23:59:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68336.log 2024-04-26T23:59:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71414.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71414.log.gz 2024-04-26T23:59:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68336.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68336.log.gz 2024-04-26T23:59:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106801.log 2024-04-26T23:59:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62019.log 2024-04-26T23:59:29.533 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106801.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106801.log.gz 2024-04-26T23:59:29.533 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60815.log 2024-04-26T23:59:29.534 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62019.log.gz 2024-04-26T23:59:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98698.log 2024-04-26T23:59:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60815.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60815.log.gz 2024-04-26T23:59:29.538 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98698.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98698.log.gz 2024-04-26T23:59:29.538 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80512.log 2024-04-26T23:59:29.544 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94633.log 2024-04-26T23:59:29.544 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80512.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80512.log.gz 2024-04-26T23:59:29.545 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45245.log 2024-04-26T23:59:29.545 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94633.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94633.log.gz 2024-04-26T23:59:29.545 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100330.log 2024-04-26T23:59:29.546 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45245.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45245.log.gz 2024-04-26T23:59:29.546 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61445.log 2024-04-26T23:59:29.547 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100330.log.gz 2024-04-26T23:59:29.547 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110277.log 2024-04-26T23:59:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61445.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61445.log.gz 2024-04-26T23:59:29.549 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108056.log 2024-04-26T23:59:29.551 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110277.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110277.log.gz 2024-04-26T23:59:29.551 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-04-26T23:59:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108056.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108056.log.gz 2024-04-26T23:59:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37516.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-04-26T23:59:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90824.log 2024-04-26T23:59:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105416.log 2024-04-26T23:59:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.90824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90824.log.gz 2024-04-26T23:59:29.559 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101571.log 2024-04-26T23:59:29.559 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105416.log.gz 2024-04-26T23:59:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109399.log 2024-04-26T23:59:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101571.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.101571.log.gz 2024-04-26T23:59:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63534.log 2024-04-26T23:59:29.563 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109399.log.gz 2024-04-26T23:59:29.564 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105542.log 2024-04-26T23:59:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63534.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63534.log.gz 2024-04-26T23:59:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64145.log 2024-04-26T23:59:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105542.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105542.log.gz 2024-04-26T23:59:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79625.log 2024-04-26T23:59:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64145.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64145.log.gz 2024-04-26T23:59:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72091.log/var/log/ceph/ceph-client.admin.79625.log: 2024-04-26T23:59:29.570 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79625.log.gz 2024-04-26T23:59:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83110.log 2024-04-26T23:59:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.72091.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72091.log.gz 2024-04-26T23:59:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89677.log 2024-04-26T23:59:29.574 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83110.log.gz 2024-04-26T23:59:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106927.log 2024-04-26T23:59:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89677.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89677.log.gz 2024-04-26T23:59:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40767.log 2024-04-26T23:59:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106927.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106927.log.gz 2024-04-26T23:59:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87664.log 2024-04-26T23:59:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40767.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40767.log.gz 2024-04-26T23:59:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102050.log 2024-04-26T23:59:29.582 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87664.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.87664.log.gz 2024-04-26T23:59:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.102050.log: -- /var/log/ceph/ceph-client.0.101319.log 2024-04-26T23:59:29.584 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102050.log.gz 2024-04-26T23:59:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.101319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101319.log.gz 2024-04-26T23:59:29.586 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60941.log 2024-04-26T23:59:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72049.log 2024-04-26T23:59:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60941.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60941.log.gz 2024-04-26T23:59:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55188.log 2024-04-26T23:59:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.72049.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72049.log.gz 2024-04-26T23:59:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110235.log 2024-04-26T23:59:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55188.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55188.log.gz 2024-04-26T23:59:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110235.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110235.log.gz 2024-04-26T23:59:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106492.log 2024-04-26T23:59:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79541.log 2024-04-26T23:59:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106492.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106492.log.gz 2024-04-26T23:59:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111077.log 2024-04-26T23:59:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79541.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79541.log.gz 2024-04-26T23:59:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110861.log 2024-04-26T23:59:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111077.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.111077.log.gz 2024-04-26T23:59:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-04-26T23:59:29.602 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110861.log.gz 2024-04-26T23:59:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107621.log 2024-04-26T23:59:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57905.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-04-26T23:59:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58186.log 2024-04-26T23:59:29.606 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107621.log.gz 2024-04-26T23:59:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94297.log 2024-04-26T23:59:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58186.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58186.log.gz 2024-04-26T23:59:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94297.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94297.log.gz 2024-04-26T23:59:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99018.log 2024-04-26T23:59:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-04-26T23:59:29.613 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99018.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99018.log.gz 2024-04-26T23:59:29.614 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46433.log: /var/log/ceph/ceph-client.admin.55699.log 2024-04-26T23:59:29.614 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-04-26T23:59:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108961.log 2024-04-26T23:59:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55699.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55699.log.gz 2024-04-26T23:59:29.618 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.108961.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90318.log 2024-04-26T23:59:29.618 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108961.log.gz 2024-04-26T23:59:29.619 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106671.log 2024-04-26T23:59:29.620 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90318.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90318.log.gz 2024-04-26T23:59:29.621 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102527.log/var/log/ceph/ceph-client.admin.106671.log: 2024-04-26T23:59:29.622 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106671.log.gz 2024-04-26T23:59:29.623 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111562.log 2024-04-26T23:59:29.624 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102527.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102527.log.gz 2024-04-26T23:59:29.626 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111562.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111562.log.gz 2024-04-26T23:59:29.626 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87964.log 2024-04-26T23:59:29.628 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59317.log 2024-04-26T23:59:29.630 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87964.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87964.log.gz 2024-04-26T23:59:29.630 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59317.log: -- /var/log/ceph/ceph-client.admin.68252.log 2024-04-26T23:59:29.630 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59317.log.gz 2024-04-26T23:59:29.631 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54833.log 2024-04-26T23:59:29.633 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68252.log.gz 2024-04-26T23:59:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102008.log 2024-04-26T23:59:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54833.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54833.log.gz 2024-04-26T23:59:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84930.log 2024-04-26T23:59:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102008.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102008.log.gz 2024-04-26T23:59:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58662.log 2024-04-26T23:59:29.638 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84930.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84930.log.gz 2024-04-26T23:59:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2024-04-26T23:59:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58662.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58662.log.gz 2024-04-26T23:59:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47080.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2024-04-26T23:59:29.642 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101019.log 2024-04-26T23:59:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106023.log 2024-04-26T23:59:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101019.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101019.log.gz 2024-04-26T23:59:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66239.log 2024-04-26T23:59:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106023.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106023.log.gz 2024-04-26T23:59:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69570.log/var/log/ceph/ceph-client.admin.66239.log: 2024-04-26T23:59:29.648 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66239.log.gz 2024-04-26T23:59:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69570.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69570.log.gz 2024-04-26T23:59:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109176.log 2024-04-26T23:59:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84845.log 2024-04-26T23:59:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109176.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109176.log.gz 2024-04-26T23:59:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55528.log 2024-04-26T23:59:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84845.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84845.log.gz 2024-04-26T23:59:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80087.log 2024-04-26T23:59:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55528.log.gz 2024-04-26T23:59:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79835.log 2024-04-26T23:59:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80087.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80087.log.gz 2024-04-26T23:59:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62927.log 2024-04-26T23:59:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79835.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79835.log.gz 2024-04-26T23:59:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-04-26T23:59:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62927.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62927.log.gz 2024-04-26T23:59:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112211.log 2024-04-26T23:59:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45203.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-04-26T23:59:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112211.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112211.log.gz 2024-04-26T23:59:29.666 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64569.log 2024-04-26T23:59:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88611.log 2024-04-26T23:59:29.669 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64569.log.gz 2024-04-26T23:59:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69808.log 2024-04-26T23:59:29.672 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88611.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88611.log.gz 2024-04-26T23:59:29.672 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93688.log/var/log/ceph/ceph-client.admin.69808.log: 2024-04-26T23:59:29.673 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69808.log.gz 2024-04-26T23:59:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92859.log 2024-04-26T23:59:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93688.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93688.log.gz 2024-04-26T23:59:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88653.log 2024-04-26T23:59:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.92859.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92859.log.gz 2024-04-26T23:59:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40766.log 2024-04-26T23:59:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88653.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88653.log.gz 2024-04-26T23:59:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110993.log/var/log/ceph/ceph-client.admin.40766.log: 2024-04-26T23:59:29.680 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40766.log.gz 2024-04-26T23:59:29.681 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61994.log 2024-04-26T23:59:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110993.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110993.log.gz 2024-04-26T23:59:29.684 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42885.log 2024-04-26T23:59:29.684 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61994.log.gz 2024-04-26T23:59:29.685 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66890.log 2024-04-26T23:59:29.686 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42885.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42885.log.gz 2024-04-26T23:59:29.687 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69528.log 2024-04-26T23:59:29.688 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66890.log.gz 2024-04-26T23:59:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69528.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69528.log.gz 2024-04-26T23:59:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-04-26T23:59:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102727.log 2024-04-26T23:59:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-04-26T23:59:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41472.log 2024-04-26T23:59:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102727.log.gz 2024-04-26T23:59:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79583.log 2024-04-26T23:59:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41472.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41472.log.gz 2024-04-26T23:59:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93604.log 2024-04-26T23:59:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79583.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79583.log.gz 2024-04-26T23:59:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59882.log 2024-04-26T23:59:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93604.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.93604.log.gz 2024-04-26T23:59:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59882.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59882.log.gz 2024-04-26T23:59:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64275.log 2024-04-26T23:59:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58728.log 2024-04-26T23:59:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64275.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64275.log.gz 2024-04-26T23:59:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99490.log 2024-04-26T23:59:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58728.log.gz 2024-04-26T23:59:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112127.log 2024-04-26T23:59:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99490.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99490.log.gz 2024-04-26T23:59:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104675.log 2024-04-26T23:59:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112127.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112127.log.gz 2024-04-26T23:59:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71629.log 2024-04-26T23:59:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104675.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104675.log.gz 2024-04-26T23:59:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66932.log 2024-04-26T23:59:29.714 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71629.log.gz 2024-04-26T23:59:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99989.log 2024-04-26T23:59:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66932.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66932.log.gz 2024-04-26T23:59:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62496.log 2024-04-26T23:59:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99989.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99989.log.gz 2024-04-26T23:59:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-04-26T23:59:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62496.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62496.log.gz 2024-04-26T23:59:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-04-26T23:59:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45329.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-04-26T23:59:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71074.log 2024-04-26T23:59:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47164.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-04-26T23:59:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83374.log 2024-04-26T23:59:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71074.log.gz 2024-04-26T23:59:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2024-04-26T23:59:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83374.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83374.log.gz 2024-04-26T23:59:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62969.log 2024-04-26T23:59:29.729 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2024-04-26T23:59:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109651.log 2024-04-26T23:59:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62969.log.gz 2024-04-26T23:59:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41846.log 2024-04-26T23:59:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109651.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.109651.log.gz 2024-04-26T23:59:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41846.log: -- /var/log/ceph/ceph-client.admin.39679.log 2024-04-26T23:59:29.734 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41846.log.gz 2024-04-26T23:59:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56763.log 2024-04-26T23:59:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39679.log.gz 2024-04-26T23:59:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56763.log: -- /var/log/ceph/ceph-client.admin.95561.log 2024-04-26T23:59:29.738 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56763.log.gz 2024-04-26T23:59:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54393.log 2024-04-26T23:59:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95561.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95561.log.gz 2024-04-26T23:59:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46996.log 2024-04-26T23:59:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54393.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54393.log.gz 2024-04-26T23:59:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46996.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46996.log.gz 2024-04-26T23:59:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2024-04-26T23:59:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84972.log 2024-04-26T23:59:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45455.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2024-04-26T23:59:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47418.log 2024-04-26T23:59:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84972.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84972.log.gz 2024-04-26T23:59:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78950.log 2024-04-26T23:59:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.47418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47418.log.gz 2024-04-26T23:59:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95868.log 2024-04-26T23:59:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.78950.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78950.log.gz 2024-04-26T23:59:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109227.log 2024-04-26T23:59:29.754 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95868.log.gz 2024-04-26T23:59:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85197.log 2024-04-26T23:59:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109227.log.gz 2024-04-26T23:59:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107380.log/var/log/ceph/ceph-client.admin.85197.log: 2024-04-26T23:59:29.763 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85197.log.gz 2024-04-26T23:59:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93214.log 2024-04-26T23:59:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107380.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.107380.log.gz 2024-04-26T23:59:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-04-26T23:59:29.764 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93214.log.gz 2024-04-26T23:59:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69331.log 2024-04-26T23:59:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-04-26T23:59:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89227.log 2024-04-26T23:59:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.69331.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69331.log.gz 2024-04-26T23:59:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40434.log 2024-04-26T23:59:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89227.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89227.log.gz 2024-04-26T23:59:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40434.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40434.log.gz 2024-04-26T23:59:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66806.log 2024-04-26T23:59:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112965.log 2024-04-26T23:59:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66806.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66806.log.gz 2024-04-26T23:59:29.774 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80304.log 2024-04-26T23:59:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112965.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112965.log.gz 2024-04-26T23:59:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-04-26T23:59:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80304.log.gz 2024-04-26T23:59:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39180.log 2024-04-26T23:59:29.778 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37460.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-04-26T23:59:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39180.log: -- /var/log/ceph/ceph-client.admin.43544.log 2024-04-26T23:59:29.780 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39180.log.gz 2024-04-26T23:59:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41262.log 2024-04-26T23:59:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43544.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-04-26T23:59:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66722.log 2024-04-26T23:59:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41262.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41262.log.gz 2024-04-26T23:59:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41556.log 2024-04-26T23:59:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66722.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66722.log.gz 2024-04-26T23:59:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104426.log 2024-04-26T23:59:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41556.log.gz 2024-04-26T23:59:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102396.log 2024-04-26T23:59:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104426.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104426.log.gz 2024-04-26T23:59:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56218.log/var/log/ceph/ceph-client.admin.102396.log: 2024-04-26T23:59:29.791 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102396.log.gz 2024-04-26T23:59:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100977.log 2024-04-26T23:59:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56218.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56218.log.gz 2024-04-26T23:59:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93772.log 2024-04-26T23:59:29.795 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100977.log.gz 2024-04-26T23:59:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40097.log 2024-04-26T23:59:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93772.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93772.log.gz 2024-04-26T23:59:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.40097.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-04-26T23:59:29.799 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-04-26T23:59:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62346.log 2024-04-26T23:59:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46307.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-04-26T23:59:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63576.log 2024-04-26T23:59:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62346.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62346.log.gz 2024-04-26T23:59:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44531.log 2024-04-26T23:59:29.804 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63576.log.gz 2024-04-26T23:59:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80996.log 2024-04-26T23:59:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44531.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44531.log.gz 2024-04-26T23:59:29.807 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99406.log 2024-04-26T23:59:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.80996.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80996.log.gz 2024-04-26T23:59:29.810 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103235.log 2024-04-26T23:59:29.810 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99406.log.gz 2024-04-26T23:59:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78800.log 2024-04-26T23:59:29.812 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103235.log.gz 2024-04-26T23:59:29.813 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.78800.log: /var/log/ceph/ceph-client.admin.79331.log 2024-04-26T23:59:29.814 INFO:teuthology.orchestra.run.smithi046.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.78800.log.gz 2024-04-26T23:59:29.815 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47038.log 2024-04-26T23:59:29.816 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79331.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79331.log.gz 2024-04-26T23:59:29.817 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2024-04-26T23:59:29.818 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47038.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47038.log.gz 2024-04-26T23:59:29.820 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93388.log 2024-04-26T23:59:29.820 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2024-04-26T23:59:29.821 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84295.log 2024-04-26T23:59:29.823 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.93388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93388.log.gz 2024-04-26T23:59:29.823 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2024-04-26T23:59:29.824 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84295.log.gz 2024-04-26T23:59:29.825 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98751.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2024-04-26T23:59:29.826 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88957.log 2024-04-26T23:59:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105855.log 2024-04-26T23:59:29.829 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88957.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88957.log.gz 2024-04-26T23:59:29.829 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112485.log 2024-04-26T23:59:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105855.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105855.log.gz 2024-04-26T23:59:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110524.log 2024-04-26T23:59:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112485.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112485.log.gz 2024-04-26T23:59:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110524.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110524.log.gz 2024-04-26T23:59:29.835 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67161.log 2024-04-26T23:59:29.836 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-04-26T23:59:29.837 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67161.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67161.log.gz 2024-04-26T23:59:29.838 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105052.log 2024-04-26T23:59:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-04-26T23:59:29.840 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.105052.log: /var/log/ceph/ceph-client.admin.38954.log 2024-04-26T23:59:29.840 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105052.log.gz 2024-04-26T23:59:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44489.log 2024-04-26T23:59:29.844 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-04-26T23:59:29.844 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61893.log/var/log/ceph/ceph-client.admin.44489.log: 2024-04-26T23:59:29.845 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44489.log.gz 2024-04-26T23:59:29.845 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95826.log 2024-04-26T23:59:29.847 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61893.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61893.log.gz 2024-04-26T23:59:29.848 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61773.log 2024-04-26T23:59:29.848 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95826.log.gz 2024-04-26T23:59:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95994.log 2024-04-26T23:59:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61773.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61773.log.gz 2024-04-26T23:59:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63281.log 2024-04-26T23:59:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95994.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95994.log.gz 2024-04-26T23:59:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89923.log 2024-04-26T23:59:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63281.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63281.log.gz 2024-04-26T23:59:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79667.log/var/log/ceph/ceph-client.admin.89923.log: 2024-04-26T23:59:29.856 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89923.log.gz 2024-04-26T23:59:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-04-26T23:59:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79667.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79667.log.gz 2024-04-26T23:59:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.45581.log: --verbose -- /var/log/ceph/ceph-client.0.64991.log 2024-04-26T23:59:29.860 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-04-26T23:59:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79877.log 2024-04-26T23:59:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.64991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64991.log.gz 2024-04-26T23:59:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107751.log 2024-04-26T23:59:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79877.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79877.log.gz 2024-04-26T23:59:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59569.log 2024-04-26T23:59:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107751.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107751.log.gz 2024-04-26T23:59:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71032.log 2024-04-26T23:59:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59569.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59569.log.gz 2024-04-26T23:59:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88873.log 2024-04-26T23:59:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.71032.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71032.log.gz 2024-04-26T23:59:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88873.log.gz 2024-04-26T23:59:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59359.log 2024-04-26T23:59:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43922.log 2024-04-26T23:59:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59359.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59359.log.gz 2024-04-26T23:59:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96290.log 2024-04-26T23:59:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43922.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43922.log.gz 2024-04-26T23:59:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79499.log 2024-04-26T23:59:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.96290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96290.log.gz 2024-04-26T23:59:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57259.log 2024-04-26T23:59:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79499.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79499.log.gz 2024-04-26T23:59:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84684.log 2024-04-26T23:59:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57259.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57259.log.gz 2024-04-26T23:59:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55104.log 2024-04-26T23:59:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84684.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84684.log.gz 2024-04-26T23:59:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62300.log 2024-04-26T23:59:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55104.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55104.log.gz 2024-04-26T23:59:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62300.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62300.log.gz 2024-04-26T23:59:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2024-04-26T23:59:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2024-04-26T23:59:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2024-04-26T23:59:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84463.log 2024-04-26T23:59:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43670.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2024-04-26T23:59:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106279.log 2024-04-26T23:59:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84463.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84463.log.gz 2024-04-26T23:59:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106279.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106279.log.gz 2024-04-26T23:59:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107972.log 2024-04-26T23:59:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T23:59:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107972.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107972.log.gz 2024-04-26T23:59:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83511.log 2024-04-26T23:59:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.83511.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83511.log.gz 2024-04-26T23:59:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103362.log 2024-04-26T23:59:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57573.log 2024-04-26T23:59:29.906 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103362.log.gz 2024-04-26T23:59:29.907 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94035.log 2024-04-26T23:59:29.909 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.57573.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90146.log 2024-04-26T23:59:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.94035.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94035.log.gz 2024-04-26T23:59:29.911 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57573.log.gz 2024-04-26T23:59:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54749.log 2024-04-26T23:59:29.914 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90146.log.gz 2024-04-26T23:59:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95696.log 2024-04-26T23:59:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54749.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54749.log.gz 2024-04-26T23:59:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95696.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95696.log.gz 2024-04-26T23:59:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60647.log 2024-04-26T23:59:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113341.log 2024-04-26T23:59:29.923 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60647.log.gz 2024-04-26T23:59:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61403.log 2024-04-26T23:59:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113341.log: /var/log/ceph/ceph-client.admin.61403.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79153.log 2024-04-26T23:59:29.929 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113341.log.gz 2024-04-26T23:59:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47422.log 2024-04-26T23:59:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79153.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61403.log.gz 2024-04-26T23:59:29.931 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79153.log.gz 2024-04-26T23:59:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.47422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47422.log.gz 2024-04-26T23:59:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66544.log 2024-04-26T23:59:29.936 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.66544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90444.log 2024-04-26T23:59:29.938 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66544.log.gz 2024-04-26T23:59:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71291.log 2024-04-26T23:59:29.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90444.log: /var/log/ceph/ceph-client.admin.71291.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37238.log 2024-04-26T23:59:29.944 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90444.log.gz 2024-04-26T23:59:29.945 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71291.log.gz 2024-04-26T23:59:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37944.log 2024-04-26T23:59:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37238.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39306.log 2024-04-26T23:59:29.951 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37238.log.gz 2024-04-26T23:59:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37944.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37944.log.gz 2024-04-26T23:59:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101787.log 2024-04-26T23:59:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39306.log: /var/log/ceph/ceph-client.admin.101787.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107269.log 2024-04-26T23:59:29.958 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39306.log.gz 2024-04-26T23:59:29.959 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101787.log.gz 2024-04-26T23:59:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.107269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-04-26T23:59:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68959.log 2024-04-26T23:59:29.964 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0%/var/log/ceph/ceph-client.admin.43082.log: -- replaced with /var/log/ceph/ceph-client.0.107269.log.gz 2024-04-26T23:59:29.965 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-04-26T23:59:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61731.log 2024-04-26T23:59:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68959.log: /var/log/ceph/ceph-client.admin.61731.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113007.log 2024-04-26T23:59:29.971 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68959.log.gz 2024-04-26T23:59:29.972 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61731.log.gz 2024-04-26T23:59:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95328.log 2024-04-26T23:59:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113007.log: /var/log/ceph/ceph-client.admin.95328.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64995.log 2024-04-26T23:59:29.979 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113007.log.gz 2024-04-26T23:59:29.980 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95328.log.gz 2024-04-26T23:59:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.64995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85327.log 2024-04-26T23:59:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.85327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64995.log.gz 2024-04-26T23:59:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62885.log 2024-04-26T23:59:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108098.log 2024-04-26T23:59:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62885.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85327.log.gz 2024-04-26T23:59:29.990 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62885.log.gz 2024-04-26T23:59:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108098.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108098.log.gz 2024-04-26T23:59:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98797.log 2024-04-26T23:59:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59119.log 2024-04-26T23:59:29.996 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98797.log.gz 2024-04-26T23:59:29.998 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110579.log 2024-04-26T23:59:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59119.log: /var/log/ceph/ceph-client.admin.110579.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112401.log 2024-04-26T23:59:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112401.log: -- replaced with /var/log/ceph/ceph-client.admin.59119.log.gz 2024-04-26T23:59:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46912.log 2024-04-26T23:59:30.006 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.110579.log.gz 2024-04-26T23:59:30.006 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112401.log.gz 2024-04-26T23:59:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103841.log 2024-04-26T23:59:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103841.log: /var/log/ceph/ceph-client.admin.46912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83773.log 2024-04-26T23:59:30.013 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46912.log.gz 2024-04-26T23:59:30.014 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103841.log.gz 2024-04-26T23:59:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79111.log 2024-04-26T23:59:30.020 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.79111.log: gzip -5/var/log/ceph/ceph-client.admin.83773.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83773.log.gz 2024-04-26T23:59:30.020 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110927.log 2024-04-26T23:59:30.021 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79111.log.gz 2024-04-26T23:59:30.021 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40718.log 2024-04-26T23:59:30.024 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40718.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40718.log.gz 2024-04-26T23:59:30.024 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60983.log 2024-04-26T23:59:30.027 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110927.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110927.log.gz 2024-04-26T23:59:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60983.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60983.log.gz 2024-04-26T23:59:30.030 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63492.log 2024-04-26T23:59:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89125.log 2024-04-26T23:59:30.034 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63492.log.gz 2024-04-26T23:59:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2024-04-26T23:59:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89125.log: /var/log/ceph/ceph-client.admin.44201.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-04-26T23:59:30.041 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89125.log.gz 2024-04-26T23:59:30.041 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gz 2024-04-26T23:59:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55966.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-04-26T23:59:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60306.log 2024-04-26T23:59:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-04-26T23:59:30.047 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60306.log.gz 2024-04-26T23:59:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111856.log 2024-04-26T23:59:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40435.log: /var/log/ceph/ceph-client.admin.111856.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-04-26T23:59:30.053 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-04-26T23:59:30.054 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111856.log.gz 2024-04-26T23:59:30.056 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47230.log 2024-04-26T23:59:30.061 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46265.log: /var/log/ceph/ceph-client.admin.47230.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-04-26T23:59:30.061 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111401.log 2024-04-26T23:59:30.062 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47230.log.gz 2024-04-26T23:59:30.062 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95910.log 2024-04-26T23:59:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111401.log: /var/log/ceph/ceph-client.admin.95910.log: 91.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59798.log 2024-04-26T23:59:30.068 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111401.log.gz 2024-04-26T23:59:30.068 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95910.log.gz 2024-04-26T23:59:30.069 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67899.log 2024-04-26T23:59:30.074 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67899.log: /var/log/ceph/ceph-client.admin.59798.log: gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.55882.log 2024-04-26T23:59:30.075 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59798.log.gz 2024-04-26T23:59:30.075 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67899.log.gz 2024-04-26T23:59:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88348.log 2024-04-26T23:59:30.077 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T23:59:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55882.log: /var/log/ceph/ceph-client.admin.88348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106843.log 2024-04-26T23:59:30.080 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55882.log.gz 2024-04-26T23:59:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106843.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112527.log 2024-04-26T23:59:30.083 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88348.log.gz 2024-04-26T23:59:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87922.log 87.3% 2024-04-26T23:59:30.084 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106843.log.gz 2024-04-26T23:59:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64187.log 2024-04-26T23:59:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.87922.log: /var/log/ceph/ceph-client.admin.64187.log: /var/log/ceph/ceph-client.admin.112527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87922.log.gz 2024-04-26T23:59:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59401.log 2024-04-26T23:59:30.091 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112527.log.gz 2024-04-26T23:59:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59401.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59401.log.gz 2024-04-26T23:59:30.095 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64187.log.gz 2024-04-26T23:59:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103883.log 2024-04-26T23:59:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33306.log 2024-04-26T23:59:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33306.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103883.log.gz 2024-04-26T23:59:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103074.log 2024-04-26T23:59:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103967.log 2024-04-26T23:59:30.104 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33306.log.gz 2024-04-26T23:59:30.106 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103074.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103074.log.gz 2024-04-26T23:59:30.108 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69766.log 2024-04-26T23:59:30.112 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103967.log: /var/log/ceph/ceph-client.admin.69766.log: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90051.log 2024-04-26T23:59:30.114 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103967.log.gz 2024-04-26T23:59:30.114 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69766.log.gz 2024-04-26T23:59:30.115 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99905.log 2024-04-26T23:59:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99905.log: /var/log/ceph/ceph-client.admin.90051.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106885.log 2024-04-26T23:59:30.121 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.90051.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.62624.log 2024-04-26T23:59:30.121 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:30.123 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63937.log 2024-04-26T23:59:30.123 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99905.log.gz 2024-04-26T23:59:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62624.log: /var/log/ceph/ceph-client.admin.63937.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62624.log.gz 2024-04-26T23:59:30.127 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5%/var/log/ceph/ceph-client.admin.106885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41010.log 2024-04-26T23:59:30.128 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106885.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64737.log 2024-04-26T23:59:30.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41010.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41010.log.gz 2024-04-26T23:59:30.129 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63937.log.gz 2024-04-26T23:59:30.130 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:30.131 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64803.log 2024-04-26T23:59:30.132 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70948.log 2024-04-26T23:59:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103799.log 2024-04-26T23:59:30.136 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70948.log: /var/log/ceph/ceph-client.admin.64803.log: 87.9% 90.6% -- replaced with /var/log/ceph/ceph-client.admin.64737.log.gz 2024-04-26T23:59:30.136 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64803.log.gz 2024-04-26T23:59:30.137 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56050.log 2024-04-26T23:59:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103799.log: 88.3%gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56050.log: /var/log/ceph/ceph-client.admin.88198.log 2024-04-26T23:59:30.141 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56050.log.gz 2024-04-26T23:59:30.141 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103799.log.gz 2024-04-26T23:59:30.142 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70948.log.gz 2024-04-26T23:59:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88198.log: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40771.log 2024-04-26T23:59:30.146 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88198.log.gz 2024-04-26T23:59:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111688.log 2024-04-26T23:59:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40771.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40771.log.gz 2024-04-26T23:59:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110501.log 2024-04-26T23:59:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-04-26T23:59:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110501.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111688.log.gz 2024-04-26T23:59:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87838.log 2024-04-26T23:59:30.159 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110501.log.gz 2024-04-26T23:59:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45077.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz/var/log/ceph/ceph-client.admin.87838.log: 2024-04-26T23:59:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87838.log.gz 2024-04-26T23:59:30.160 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.100073.log 2024-04-26T23:59:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88527.log 2024-04-26T23:59:30.163 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100073.log.gz 2024-04-26T23:59:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67711.log 2024-04-26T23:59:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.88527.log: /var/log/ceph/ceph-client.admin.67711.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95455.log 2024-04-26T23:59:30.168 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88527.log.gz 2024-04-26T23:59:30.169 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67711.log.gz 2024-04-26T23:59:30.171 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-04-26T23:59:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95455.log: /var/log/ceph/ceph-client.admin.43964.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112881.log 2024-04-26T23:59:30.175 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95455.log.gz 2024-04-26T23:59:30.177 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-04-26T23:59:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71755.log 2024-04-26T23:59:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112881.log: /var/log/ceph/ceph-client.admin.71755.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112923.log 2024-04-26T23:59:30.183 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112881.log.gz 2024-04-26T23:59:30.184 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71755.log.gz 2024-04-26T23:59:30.185 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102228.log 2024-04-26T23:59:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112923.log: /var/log/ceph/ceph-client.admin.102228.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89438.log 2024-04-26T23:59:30.190 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112923.log.gz 2024-04-26T23:59:30.191 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102228.log.gz 2024-04-26T23:59:30.192 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42800.log 2024-04-26T23:59:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.89438.log: /var/log/ceph/ceph-client.admin.42800.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43754.log 2024-04-26T23:59:30.197 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89438.log.gz 2024-04-26T23:59:30.198 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42800.log.gz 2024-04-26T23:59:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99658.log 2024-04-26T23:59:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99658.log: /var/log/ceph/ceph-client.admin.43754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102769.log 2024-04-26T23:59:30.205 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43754.log.gz 2024-04-26T23:59:30.206 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99658.log.gz 2024-04-26T23:59:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-04-26T23:59:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54893.log: /var/log/ceph/ceph-client.admin.102769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88747.log 2024-04-26T23:59:30.211 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102769.log.gz 2024-04-26T23:59:30.212 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-04-26T23:59:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60433.log 2024-04-26T23:59:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60433.log: /var/log/ceph/ceph-client.admin.88747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40099.log 2024-04-26T23:59:30.219 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88747.log.gz 2024-04-26T23:59:30.219 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60433.log.gz 2024-04-26T23:59:30.221 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55571.log 2024-04-26T23:59:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55571.log: /var/log/ceph/ceph-client.admin.40099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89965.log 2024-04-26T23:59:30.226 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40099.log.gz 2024-04-26T23:59:30.227 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55571.log.gz 2024-04-26T23:59:30.227 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106118.log 2024-04-26T23:59:30.230 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.89965.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.105897.log 2024-04-26T23:59:30.232 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106118.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106118.log.gz 2024-04-26T23:59:30.233 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89965.log.gz 2024-04-26T23:59:30.234 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78884.log 2024-04-26T23:59:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.78884.log: /var/log/ceph/ceph-client.admin.105897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54581.log 2024-04-26T23:59:30.240 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105897.log.gz 2024-04-26T23:59:30.241 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78884.log.gz 2024-04-26T23:59:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63095.log 2024-04-26T23:59:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63095.log: /var/log/ceph/ceph-client.admin.54581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105771.log 2024-04-26T23:59:30.248 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54581.log.gz 2024-04-26T23:59:30.248 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63095.log.gz 2024-04-26T23:59:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109483.log 2024-04-26T23:59:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109483.log: /var/log/ceph/ceph-client.admin.105771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59035.log 2024-04-26T23:59:30.255 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105771.log.gz 2024-04-26T23:59:30.255 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109483.log.gz 2024-04-26T23:59:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84127.log 2024-04-26T23:59:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.84127.log: /var/log/ceph/ceph-client.admin.59035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71965.log 2024-04-26T23:59:30.263 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59035.log.gz 2024-04-26T23:59:30.263 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84127.log.gz 2024-04-26T23:59:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59756.log 2024-04-26T23:59:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59756.log: /var/log/ceph/ceph-client.admin.71965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68720.log 2024-04-26T23:59:30.270 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71965.log.gz 2024-04-26T23:59:30.270 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59756.log.gz 2024-04-26T23:59:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60050.log 2024-04-26T23:59:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60050.log: /var/log/ceph/ceph-client.0.68720.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70257.log 2024-04-26T23:59:30.275 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.0.68720.log.gz 2024-04-26T23:59:30.276 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60050.log.gz 2024-04-26T23:59:30.278 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100031.log 2024-04-26T23:59:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100031.log: /var/log/ceph/ceph-client.admin.70257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83416.log 2024-04-26T23:59:30.283 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70257.log.gz 2024-04-26T23:59:30.283 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100031.log.gz 2024-04-26T23:59:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103404.log 2024-04-26T23:59:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41598.log 2024-04-26T23:59:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.83416.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98839.log 2024-04-26T23:59:30.290 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83416.log.gz 2024-04-26T23:59:30.291 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39510.log 2024-04-26T23:59:30.291 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103404.log.gz 2024-04-26T23:59:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98839.log: /var/log/ceph/ceph-client.admin.41598.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41598.log.gz 2024-04-26T23:59:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39510.log: 89.2% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98839.log.gz 2024-04-26T23:59:30.294 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39510.log.gz 2024-04-26T23:59:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78734.log 2024-04-26T23:59:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.78734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67413.log 2024-04-26T23:59:30.300 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78734.log.gz 2024-04-26T23:59:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107077.log 2024-04-26T23:59:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67058.log 2024-04-26T23:59:30.305 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107077.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67413.log.gz 2024-04-26T23:59:30.306 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107077.log.gz 2024-04-26T23:59:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61815.log 2024-04-26T23:59:30.310 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0%/var/log/ceph/ceph-client.admin.61815.log: -- replaced with /var/log/ceph/ceph-client.admin.67058.log.gz 2024-04-26T23:59:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106717.log 2024-04-26T23:59:30.312 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61815.log.gz 2024-04-26T23:59:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64485.log 2024-04-26T23:59:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.64485.log: /var/log/ceph/ceph-client.admin.106717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70822.log 2024-04-26T23:59:30.318 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58596.log 2024-04-26T23:59:30.319 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106717.log.gz 2024-04-26T23:59:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58596.log: -- replaced with /var/log/ceph/ceph-client.admin.64485.log.gz 2024-04-26T23:59:30.323 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58596.log.gz 2024-04-26T23:59:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.70822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67550.log 2024-04-26T23:59:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.67550.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67550.log.gz 2024-04-26T23:59:30.328 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111359.log 2024-04-26T23:59:30.331 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70822.log.gz 2024-04-26T23:59:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100851.log 2024-04-26T23:59:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111359.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111359.log.gz 2024-04-26T23:59:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68378.log 2024-04-26T23:59:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.68378.log: /var/log/ceph/ceph-client.admin.100851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58361.log 2024-04-26T23:59:30.339 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100851.log.gz 2024-04-26T23:59:30.340 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68378.log.gz 2024-04-26T23:59:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45767.log 2024-04-26T23:59:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45767.log: /var/log/ceph/ceph-client.admin.58361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108453.log 2024-04-26T23:59:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101655.log 2024-04-26T23:59:30.349 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% 91.1%/var/log/ceph/ceph-client.admin.101655.log: -- replaced with /var/log/ceph/ceph-client.admin.58361.log.gz 2024-04-26T23:59:30.349 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45767.log.gz 2024-04-26T23:59:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108453.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.101655.log.gz 2024-04-26T23:59:30.351 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39080.log 2024-04-26T23:59:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41682.log 2024-04-26T23:59:30.353 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108453.log.gz 2024-04-26T23:59:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39080.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39080.log.gz 2024-04-26T23:59:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41682.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41682.log.gz 2024-04-26T23:59:30.682 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T23:59:32.789 INFO:teuthology.orchestra.run.smithi046.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T23:59:33.219 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T23:59:33.998 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T23:59:34.475 INFO:teuthology.orchestra.run.smithi046.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T23:59:34.694 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T23:59:34.761 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T23:59:35.089 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T23:59:35.127 INFO:teuthology.orchestra.run.smithi046.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T23:59:35.935 INFO:teuthology.orchestra.run.smithi046.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T23:59:50.210 INFO:teuthology.orchestra.run.smithi142.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T23:59:50.214 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-04-26T23:59:50.214 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m21.809s 2024-04-26T23:59:50.214 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m44.604s 2024-04-26T23:59:50.214 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m3.341s 2024-04-26T23:59:59.410 INFO:teuthology.orchestra.run.smithi046.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T23:59:59.414 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-26T23:59:59.414 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m31.012s 2024-04-26T23:59:59.414 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m57.003s 2024-04-26T23:59:59.414 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m6.908s 2024-04-26T23:59:59.415 INFO:tasks.ceph:Archiving logs... 2024-04-26T23:59:59.416 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi046/log 2024-04-26T23:59:59.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T00:00:08.013 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi142/log 2024-04-27T00:00:08.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T00:00:14.321 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T00:00:14.330 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T00:00:14.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T00:00:14.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T00:00:14.479 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T00:00:14.489 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T00:00:14.489 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T00:00:14.495 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T00:00:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 7 377 111 +1330us[+1305us] +/- 42ms 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 112 -409us[ -434us] +/- 57ms 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 110 -1493us[-1493us] +/- 67ms 2024-04-27T00:00:14.537 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T00:00:14.550 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +1318us[+1325us] +/- 43ms 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -424us[ -416us] +/- 58ms 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -1477us[-1470us] +/- 68ms 2024-04-27T00:00:14.560 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T00:00:14.562 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T00:00:14.582 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T00:00:14.583 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T00:00:14.591 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T00:00:14.811 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T00:00:14.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T00:00:15.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:00:15.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T00:00:15.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:00:15.175 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T00:00:15.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T00:00:15.208 INFO:teuthology.task.internal:Duration was 1922.902505 seconds 2024-04-27T00:00:15.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T00:00:15.219 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T00:00:15.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T00:00:15.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T00:00:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T00:00:15.370 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T00:00:15.804 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T00:00:15.804 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-04-27T00:00:15.804 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T00:00:15.866 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-04-27T00:00:15.866 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T00:00:15.953 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T00:00:15.954 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T00:00:15.962 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T00:00:16.044 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T00:00:16.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T00:00:16.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T00:00:16.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T00:00:16.390 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T00:00:16.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T00:00:16.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T00:00:16.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T00:00:16.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:00:16.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:00:16.722 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-04-27T00:00:16.734 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-04-27T00:00:16.776 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:00:16.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:00:16.828 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:00:16.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:00:16.882 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T00:00:16.942 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T00:00:16.942 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi046 2024-04-27T00:00:16.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T00:00:17.045 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675316/remote/smithi142 2024-04-27T00:00:17.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T00:00:17.139 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T00:00:17.139 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T00:00:17.147 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T00:00:17.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T00:00:17.237 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T00:00:17.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T00:00:17.268 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T00:00:17.268 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T00:00:17.276 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T00:00:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 00:00 /home/ubuntu/cephtest 2024-04-27T00:00:17.333 INFO:teuthology.orchestra.run.smithi142.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 00:00 /home/ubuntu/cephtest 2024-04-27T00:00:17.336 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T00:00:17.360 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T00:00:17.385 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1922.902505159378 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-27T00:00:17.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T00:00:17.472 INFO:teuthology.run:pass